[2023-10-13 20:53:00,207][59943] Saving configuration to ./train_atari/atari_pitfall_APPO/config.json... [2023-10-13 20:53:00,551][59943] Rollout worker 0 uses device cpu [2023-10-13 20:53:00,552][59943] Rollout worker 1 uses device cpu [2023-10-13 20:53:00,552][59943] Rollout worker 2 uses device cpu [2023-10-13 20:53:00,553][59943] Rollout worker 3 uses device cpu [2023-10-13 20:53:00,553][59943] Rollout worker 4 uses device cpu [2023-10-13 20:53:00,554][59943] Rollout worker 5 uses device cpu [2023-10-13 20:53:00,554][59943] Rollout worker 6 uses device cpu [2023-10-13 20:53:00,555][59943] Rollout worker 7 uses device cpu [2023-10-13 20:53:00,555][59943] Rollout worker 8 uses device cpu [2023-10-13 20:53:00,556][59943] Rollout worker 9 uses device cpu [2023-10-13 20:53:00,556][59943] Rollout worker 10 uses device cpu [2023-10-13 20:53:00,557][59943] Rollout worker 11 uses device cpu [2023-10-13 20:53:00,557][59943] Rollout worker 12 uses device cpu [2023-10-13 20:53:00,557][59943] Rollout worker 13 uses device cpu [2023-10-13 20:53:00,558][59943] Rollout worker 14 uses device cpu [2023-10-13 20:53:00,558][59943] Rollout worker 15 uses device cpu [2023-10-13 20:53:00,853][59943] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-13 20:53:00,853][59943] InferenceWorker_p0-w0: min num requests: 2 [2023-10-13 20:53:00,857][59943] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-13 20:53:00,857][59943] InferenceWorker_p1-w0: min num requests: 2 [2023-10-13 20:53:00,903][59943] Starting all processes... [2023-10-13 20:53:00,903][59943] Starting process learner_proc0 [2023-10-13 20:53:02,594][59943] Starting process learner_proc1 [2023-10-13 20:53:02,597][60695] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-13 20:53:02,598][60695] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-10-13 20:53:02,616][60695] Num visible devices: 1 [2023-10-13 20:53:02,634][60695] Setting fixed seed 1234 [2023-10-13 20:53:02,635][60695] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-13 20:53:02,636][60695] Initializing actor-critic model on device cuda:0 [2023-10-13 20:53:02,636][60695] RunningMeanStd input shape: (4, 84, 84) [2023-10-13 20:53:02,636][60695] RunningMeanStd input shape: (1,) [2023-10-13 20:53:02,648][60695] ConvEncoder: input_channels=4 [2023-10-13 20:53:02,818][60695] Conv encoder output size: 512 [2023-10-13 20:53:02,821][60695] Created Actor Critic model with architecture: [2023-10-13 20:53:02,822][60695] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=18, bias=True) ) ) [2023-10-13 20:53:03,412][60695] Using optimizer [2023-10-13 20:53:03,413][60695] No checkpoints found [2023-10-13 20:53:03,413][60695] Did not load from checkpoint, starting from scratch! [2023-10-13 20:53:03,413][60695] Initialized policy 0 weights for model version 0 [2023-10-13 20:53:03,415][60695] LearnerWorker_p0 finished initialization! [2023-10-13 20:53:03,415][60695] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-13 20:53:04,348][59943] Starting all processes... [2023-10-13 20:53:04,351][60828] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-13 20:53:04,352][60828] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1 [2023-10-13 20:53:04,357][59943] Starting process inference_proc0-0 [2023-10-13 20:53:04,357][59943] Starting process inference_proc1-0 [2023-10-13 20:53:04,358][59943] Starting process rollout_proc0 [2023-10-13 20:53:04,370][60828] Num visible devices: 1 [2023-10-13 20:53:04,358][59943] Starting process rollout_proc1 [2023-10-13 20:53:04,358][59943] Starting process rollout_proc2 [2023-10-13 20:53:04,385][60828] Setting fixed seed 1234 [2023-10-13 20:53:04,386][60828] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-13 20:53:04,386][60828] Initializing actor-critic model on device cuda:0 [2023-10-13 20:53:04,387][60828] RunningMeanStd input shape: (4, 84, 84) [2023-10-13 20:53:04,387][60828] RunningMeanStd input shape: (1,) [2023-10-13 20:53:04,358][59943] Starting process rollout_proc3 [2023-10-13 20:53:04,359][59943] Starting process rollout_proc4 [2023-10-13 20:53:04,363][59943] Starting process rollout_proc5 [2023-10-13 20:53:04,368][59943] Starting process rollout_proc6 [2023-10-13 20:53:04,369][59943] Starting process rollout_proc7 [2023-10-13 20:53:04,399][60828] ConvEncoder: input_channels=4 [2023-10-13 20:53:04,373][59943] Starting process rollout_proc8 [2023-10-13 20:53:04,376][59943] Starting process rollout_proc9 [2023-10-13 20:53:04,377][59943] Starting process rollout_proc10 [2023-10-13 20:53:04,379][59943] Starting process rollout_proc11 [2023-10-13 20:53:04,385][59943] Starting process rollout_proc12 [2023-10-13 20:53:04,386][59943] Starting process rollout_proc13 [2023-10-13 20:53:04,810][60828] Conv encoder output size: 512 [2023-10-13 20:53:04,837][60828] Created Actor Critic model with architecture: [2023-10-13 20:53:04,837][60828] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=18, bias=True) ) ) [2023-10-13 20:53:05,647][60828] Using optimizer [2023-10-13 20:53:05,648][60828] No checkpoints found [2023-10-13 20:53:05,648][60828] Did not load from checkpoint, starting from scratch! [2023-10-13 20:53:05,648][60828] Initialized policy 1 weights for model version 0 [2023-10-13 20:53:05,650][60828] LearnerWorker_p1 finished initialization! [2023-10-13 20:53:05,650][60828] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-13 20:53:06,593][59943] Starting process rollout_proc14 [2023-10-13 20:53:06,597][59943] Starting process rollout_proc15 [2023-10-13 20:53:06,598][60934] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-13 20:53:06,598][60934] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1 [2023-10-13 20:53:06,601][60973] Worker 4 uses CPU cores [8, 9] [2023-10-13 20:53:06,604][60998] Worker 11 uses CPU cores [22, 23] [2023-10-13 20:53:06,617][60934] Num visible devices: 1 [2023-10-13 20:53:06,712][60997] Worker 13 uses CPU cores [26, 27] [2023-10-13 20:53:06,776][60968] Worker 0 uses CPU cores [0, 1] [2023-10-13 20:53:06,835][60976] Worker 7 uses CPU cores [14, 15] [2023-10-13 20:53:06,966][60935] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-13 20:53:06,966][60935] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-10-13 20:53:06,976][60970] Worker 1 uses CPU cores [2, 3] [2023-10-13 20:53:06,979][60974] Worker 5 uses CPU cores [10, 11] [2023-10-13 20:53:06,985][60935] Num visible devices: 1 [2023-10-13 20:53:06,997][60984] Worker 8 uses CPU cores [16, 17] [2023-10-13 20:53:07,033][60986] Worker 10 uses CPU cores [20, 21] [2023-10-13 20:53:07,119][60975] Worker 6 uses CPU cores [12, 13] [2023-10-13 20:53:07,137][60971] Worker 2 uses CPU cores [4, 5] [2023-10-13 20:53:07,161][60972] Worker 3 uses CPU cores [6, 7] [2023-10-13 20:53:07,270][60983] Worker 9 uses CPU cores [18, 19] [2023-10-13 20:53:07,289][60996] Worker 12 uses CPU cores [24, 25] [2023-10-13 20:53:07,365][60934] RunningMeanStd input shape: (4, 84, 84) [2023-10-13 20:53:07,366][60934] RunningMeanStd input shape: (1,) [2023-10-13 20:53:07,379][60934] ConvEncoder: input_channels=4 [2023-10-13 20:53:07,484][60934] Conv encoder output size: 512 [2023-10-13 20:53:07,634][60935] RunningMeanStd input shape: (4, 84, 84) [2023-10-13 20:53:07,635][60935] RunningMeanStd input shape: (1,) [2023-10-13 20:53:07,646][60935] ConvEncoder: input_channels=4 [2023-10-13 20:53:07,748][60935] Conv encoder output size: 512 [2023-10-13 20:53:08,511][61664] Worker 15 uses CPU cores [30, 31] [2023-10-13 20:53:08,577][59943] Inference worker 1-0 is ready! [2023-10-13 20:53:08,578][59943] Inference worker 0-0 is ready! [2023-10-13 20:53:08,578][61663] Worker 14 uses CPU cores [28, 29] [2023-10-13 20:53:08,579][59943] All inference workers are ready! Signal rollout workers to start! [2023-10-13 20:53:08,580][60997] EnvRunner 13-0 uses policy 1 [2023-10-13 20:53:08,580][60971] EnvRunner 2-0 uses policy 0 [2023-10-13 20:53:08,580][60984] EnvRunner 8-0 uses policy 0 [2023-10-13 20:53:08,580][60968] EnvRunner 0-0 uses policy 0 [2023-10-13 20:53:08,580][60976] EnvRunner 7-0 uses policy 1 [2023-10-13 20:53:08,580][60970] EnvRunner 1-0 uses policy 1 [2023-10-13 20:53:08,580][60973] EnvRunner 4-0 uses policy 0 [2023-10-13 20:53:08,580][60974] EnvRunner 5-0 uses policy 1 [2023-10-13 20:53:08,580][60996] EnvRunner 12-0 uses policy 0 [2023-10-13 20:53:08,580][60975] EnvRunner 6-0 uses policy 0 [2023-10-13 20:53:08,580][60986] EnvRunner 10-0 uses policy 0 [2023-10-13 20:53:08,580][60983] EnvRunner 9-0 uses policy 1 [2023-10-13 20:53:08,580][60972] EnvRunner 3-0 uses policy 1 [2023-10-13 20:53:08,580][59943] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-13 20:53:08,580][60998] EnvRunner 11-0 uses policy 1 [2023-10-13 20:53:08,700][61663] EnvRunner 14-0 uses policy 0 [2023-10-13 20:53:08,764][61664] EnvRunner 15-0 uses policy 1 [2023-10-13 20:53:10,841][59943] Heartbeat connected on Batcher_0 [2023-10-13 20:53:10,843][59943] Heartbeat connected on LearnerWorker_p0 [2023-10-13 20:53:10,847][59943] Heartbeat connected on Batcher_1 [2023-10-13 20:53:10,849][59943] Heartbeat connected on LearnerWorker_p1 [2023-10-13 20:53:10,858][59943] Heartbeat connected on InferenceWorker_p0-w0 [2023-10-13 20:53:10,860][59943] Heartbeat connected on InferenceWorker_p1-w0 [2023-10-13 20:53:10,863][59943] Heartbeat connected on RolloutWorker_w1 [2023-10-13 20:53:10,867][59943] Heartbeat connected on RolloutWorker_w0 [2023-10-13 20:53:10,870][59943] Heartbeat connected on RolloutWorker_w3 [2023-10-13 20:53:10,871][59943] Heartbeat connected on RolloutWorker_w4 [2023-10-13 20:53:10,872][59943] Heartbeat connected on RolloutWorker_w2 [2023-10-13 20:53:10,874][59943] Heartbeat connected on RolloutWorker_w5 [2023-10-13 20:53:10,880][59943] Heartbeat connected on RolloutWorker_w6 [2023-10-13 20:53:10,880][59943] Heartbeat connected on RolloutWorker_w7 [2023-10-13 20:53:10,886][59943] Heartbeat connected on RolloutWorker_w8 [2023-10-13 20:53:10,888][59943] Heartbeat connected on RolloutWorker_w10 [2023-10-13 20:53:10,890][59943] Heartbeat connected on RolloutWorker_w11 [2023-10-13 20:53:10,890][59943] Heartbeat connected on RolloutWorker_w9 [2023-10-13 20:53:10,897][59943] Heartbeat connected on RolloutWorker_w12 [2023-10-13 20:53:10,902][59943] Heartbeat connected on RolloutWorker_w13 [2023-10-13 20:53:10,902][59943] Heartbeat connected on RolloutWorker_w15 [2023-10-13 20:53:10,904][59943] Heartbeat connected on RolloutWorker_w14 [2023-10-13 20:53:11,248][59943] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 582.4, 1: 371.1. Samples: 2544. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-13 20:53:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:53:16,248][59943] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 973.9, 1: 885.0. Samples: 14254. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-13 20:53:16,249][59943] Avg episode reward: [(0, '-0.762'), (1, '-3.750')] [2023-10-13 20:53:18,496][60935] Updated weights for policy 0, policy_version 10 (0.0009) [2023-10-13 20:53:18,657][60934] Updated weights for policy 1, policy_version 10 (0.0008) [2023-10-13 20:53:18,866][60935] Updated weights for policy 0, policy_version 20 (0.0008) [2023-10-13 20:53:19,015][60934] Updated weights for policy 1, policy_version 20 (0.0008) [2023-10-13 20:53:19,235][60935] Updated weights for policy 0, policy_version 30 (0.0009) [2023-10-13 20:53:19,374][60934] Updated weights for policy 1, policy_version 30 (0.0009) [2023-10-13 20:53:21,248][59943] Fps is (10 sec: 6553.7, 60 sec: 5173.4, 300 sec: 5173.4). Total num frames: 65536. Throughput: 0: 1243.3, 1: 1187.6. Samples: 30794. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 20:53:21,249][59943] Avg episode reward: [(0, '-2.438'), (1, '-2.610')] [2023-10-13 20:53:21,782][60935] Updated weights for policy 0, policy_version 40 (0.0009) [2023-10-13 20:53:22,099][60934] Updated weights for policy 1, policy_version 40 (0.0009) [2023-10-13 20:53:22,146][60935] Updated weights for policy 0, policy_version 50 (0.0010) [2023-10-13 20:53:22,465][60934] Updated weights for policy 1, policy_version 50 (0.0007) [2023-10-13 20:53:22,520][60935] Updated weights for policy 0, policy_version 60 (0.0008) [2023-10-13 20:53:22,828][60934] Updated weights for policy 1, policy_version 60 (0.0008) [2023-10-13 20:53:25,951][60934] Updated weights for policy 1, policy_version 70 (0.0009) [2023-10-13 20:53:26,114][60935] Updated weights for policy 0, policy_version 70 (0.0009) [2023-10-13 20:53:26,248][59943] Fps is (10 sec: 13107.5, 60 sec: 7418.7, 300 sec: 7418.7). Total num frames: 131072. Throughput: 0: 1467.5, 1: 1436.3. Samples: 51304. Policy #0 lag: (min: 33.0, avg: 33.0, max: 33.0) [2023-10-13 20:53:26,249][59943] Avg episode reward: [(0, '-2.021'), (1, '-2.430')] [2023-10-13 20:53:26,323][60934] Updated weights for policy 1, policy_version 80 (0.0007) [2023-10-13 20:53:26,481][60935] Updated weights for policy 0, policy_version 80 (0.0010) [2023-10-13 20:53:26,681][60934] Updated weights for policy 1, policy_version 90 (0.0007) [2023-10-13 20:53:26,841][60935] Updated weights for policy 0, policy_version 90 (0.0009) [2023-10-13 20:53:30,470][60934] Updated weights for policy 1, policy_version 100 (0.0009) [2023-10-13 20:53:30,555][60935] Updated weights for policy 0, policy_version 100 (0.0007) [2023-10-13 20:53:30,824][60934] Updated weights for policy 1, policy_version 110 (0.0009) [2023-10-13 20:53:30,927][60935] Updated weights for policy 0, policy_version 110 (0.0007) [2023-10-13 20:53:31,193][60934] Updated weights for policy 1, policy_version 120 (0.0008) [2023-10-13 20:53:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 8673.4, 300 sec: 8673.4). Total num frames: 196608. Throughput: 0: 1347.7, 1: 1322.0. Samples: 60516. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 20:53:31,249][59943] Avg episode reward: [(0, '-3.305'), (1, '-2.134')] [2023-10-13 20:53:31,296][60935] Updated weights for policy 0, policy_version 120 (0.0007) [2023-10-13 20:53:35,360][60934] Updated weights for policy 1, policy_version 130 (0.0007) [2023-10-13 20:53:35,534][60935] Updated weights for policy 0, policy_version 130 (0.0008) [2023-10-13 20:53:35,726][60934] Updated weights for policy 1, policy_version 140 (0.0007) [2023-10-13 20:53:35,889][60935] Updated weights for policy 0, policy_version 140 (0.0008) [2023-10-13 20:53:36,078][60934] Updated weights for policy 1, policy_version 150 (0.0008) [2023-10-13 20:53:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 9474.7, 300 sec: 9474.7). Total num frames: 262144. Throughput: 0: 1477.1, 1: 1462.1. Samples: 81322. Policy #0 lag: (min: 22.0, avg: 27.3, max: 54.0) [2023-10-13 20:53:36,249][59943] Avg episode reward: [(0, '-2.901'), (1, '-2.020')] [2023-10-13 20:53:36,256][60935] Updated weights for policy 0, policy_version 150 (0.0008) [2023-10-13 20:53:36,444][60828] Saving new best policy, reward=-2.020! [2023-10-13 20:53:36,444][60934] Updated weights for policy 1, policy_version 160 (0.0007) [2023-10-13 20:53:36,621][60695] Saving new best policy, reward=-2.901! [2023-10-13 20:53:36,626][60935] Updated weights for policy 0, policy_version 160 (0.0010) [2023-10-13 20:53:40,464][60934] Updated weights for policy 1, policy_version 170 (0.0008) [2023-10-13 20:53:40,756][60935] Updated weights for policy 0, policy_version 170 (0.0008) [2023-10-13 20:53:40,825][60934] Updated weights for policy 1, policy_version 180 (0.0007) [2023-10-13 20:53:41,128][60935] Updated weights for policy 0, policy_version 180 (0.0007) [2023-10-13 20:53:41,187][60934] Updated weights for policy 1, policy_version 190 (0.0008) [2023-10-13 20:53:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 10030.6, 300 sec: 10030.6). Total num frames: 327680. Throughput: 0: 1556.5, 1: 1545.9. Samples: 101346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:53:41,249][59943] Avg episode reward: [(0, '-6.066'), (1, '-0.890')] [2023-10-13 20:53:41,261][60828] Saving new best policy, reward=-0.890! [2023-10-13 20:53:41,490][60935] Updated weights for policy 0, policy_version 190 (0.0009) [2023-10-13 20:53:45,189][60934] Updated weights for policy 1, policy_version 200 (0.0008) [2023-10-13 20:53:45,478][60935] Updated weights for policy 0, policy_version 200 (0.0009) [2023-10-13 20:53:45,552][60934] Updated weights for policy 1, policy_version 210 (0.0008) [2023-10-13 20:53:45,849][60935] Updated weights for policy 0, policy_version 210 (0.0007) [2023-10-13 20:53:45,925][60934] Updated weights for policy 1, policy_version 220 (0.0008) [2023-10-13 20:53:46,212][60935] Updated weights for policy 0, policy_version 220 (0.0007) [2023-10-13 20:53:46,248][59943] Fps is (10 sec: 16383.6, 60 sec: 11308.9, 300 sec: 11308.9). Total num frames: 425984. Throughput: 0: 1482.7, 1: 1477.5. Samples: 111506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:53:46,249][59943] Avg episode reward: [(0, '-6.780'), (1, '-1.440')] [2023-10-13 20:53:49,980][60934] Updated weights for policy 1, policy_version 230 (0.0007) [2023-10-13 20:53:50,234][60935] Updated weights for policy 0, policy_version 230 (0.0009) [2023-10-13 20:53:50,352][60934] Updated weights for policy 1, policy_version 240 (0.0008) [2023-10-13 20:53:50,599][60935] Updated weights for policy 0, policy_version 240 (0.0008) [2023-10-13 20:53:50,717][60934] Updated weights for policy 1, policy_version 250 (0.0007) [2023-10-13 20:53:50,969][60935] Updated weights for policy 0, policy_version 250 (0.0010) [2023-10-13 20:53:51,248][59943] Fps is (10 sec: 19660.2, 60 sec: 12287.6, 300 sec: 12287.6). Total num frames: 524288. Throughput: 0: 1547.1, 1: 1547.8. Samples: 132054. Policy #0 lag: (min: 4.0, avg: 13.2, max: 36.0) [2023-10-13 20:53:51,250][59943] Avg episode reward: [(0, '-6.220'), (1, '-2.100')] [2023-10-13 20:53:54,557][60934] Updated weights for policy 1, policy_version 260 (0.0007) [2023-10-13 20:53:54,918][60934] Updated weights for policy 1, policy_version 270 (0.0009) [2023-10-13 20:53:54,944][60935] Updated weights for policy 0, policy_version 260 (0.0011) [2023-10-13 20:53:55,286][60934] Updated weights for policy 1, policy_version 280 (0.0008) [2023-10-13 20:53:55,309][60935] Updated weights for policy 0, policy_version 270 (0.0008) [2023-10-13 20:53:55,681][60935] Updated weights for policy 0, policy_version 280 (0.0008) [2023-10-13 20:53:56,248][59943] Fps is (10 sec: 16383.8, 60 sec: 12373.5, 300 sec: 12373.5). Total num frames: 589824. Throughput: 0: 1645.1, 1: 1654.8. Samples: 151038. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-13 20:53:56,250][59943] Avg episode reward: [(0, '-5.620'), (1, '-1.960')] [2023-10-13 20:53:59,615][60934] Updated weights for policy 1, policy_version 290 (0.0008) [2023-10-13 20:53:59,680][60935] Updated weights for policy 0, policy_version 290 (0.0009) [2023-10-13 20:54:00,007][60934] Updated weights for policy 1, policy_version 300 (0.0009) [2023-10-13 20:54:00,085][60935] Updated weights for policy 0, policy_version 300 (0.0009) [2023-10-13 20:54:00,372][60934] Updated weights for policy 1, policy_version 310 (0.0008) [2023-10-13 20:54:00,459][60935] Updated weights for policy 0, policy_version 310 (0.0009) [2023-10-13 20:54:00,737][60934] Updated weights for policy 1, policy_version 320 (0.0007) [2023-10-13 20:54:00,823][60935] Updated weights for policy 0, policy_version 320 (0.0010) [2023-10-13 20:54:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 12443.2, 300 sec: 12443.2). Total num frames: 655360. Throughput: 0: 1638.7, 1: 1651.9. Samples: 162328. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 20:54:01,249][59943] Avg episode reward: [(0, '-1.630'), (1, '-1.830')] [2023-10-13 20:54:01,250][60695] Saving new best policy, reward=-1.630! [2023-10-13 20:54:04,722][60934] Updated weights for policy 1, policy_version 330 (0.0008) [2023-10-13 20:54:04,896][60935] Updated weights for policy 0, policy_version 330 (0.0009) [2023-10-13 20:54:05,081][60934] Updated weights for policy 1, policy_version 340 (0.0009) [2023-10-13 20:54:05,275][60935] Updated weights for policy 0, policy_version 340 (0.0007) [2023-10-13 20:54:05,444][60934] Updated weights for policy 1, policy_version 350 (0.0008) [2023-10-13 20:54:05,641][60935] Updated weights for policy 0, policy_version 350 (0.0009) [2023-10-13 20:54:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 12500.8, 300 sec: 12500.8). Total num frames: 720896. Throughput: 0: 1671.8, 1: 1692.7. Samples: 182194. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-13 20:54:06,249][59943] Avg episode reward: [(0, '-1.070'), (1, '-1.280')] [2023-10-13 20:54:06,250][60695] Saving new best policy, reward=-1.070! [2023-10-13 20:54:09,656][60934] Updated weights for policy 1, policy_version 360 (0.0009) [2023-10-13 20:54:09,869][60935] Updated weights for policy 0, policy_version 360 (0.0008) [2023-10-13 20:54:10,014][60934] Updated weights for policy 1, policy_version 370 (0.0009) [2023-10-13 20:54:10,229][60935] Updated weights for policy 0, policy_version 370 (0.0007) [2023-10-13 20:54:10,374][60934] Updated weights for policy 1, policy_version 380 (0.0008) [2023-10-13 20:54:10,600][60935] Updated weights for policy 0, policy_version 380 (0.0007) [2023-10-13 20:54:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12549.2). Total num frames: 786432. Throughput: 0: 1651.0, 1: 1670.4. Samples: 200764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:54:11,249][59943] Avg episode reward: [(0, '-1.230'), (1, '-0.730')] [2023-10-13 20:54:11,254][60828] Saving new best policy, reward=-0.730! [2023-10-13 20:54:14,392][60934] Updated weights for policy 1, policy_version 390 (0.0009) [2023-10-13 20:54:14,517][60935] Updated weights for policy 0, policy_version 390 (0.0008) [2023-10-13 20:54:14,746][60934] Updated weights for policy 1, policy_version 400 (0.0009) [2023-10-13 20:54:14,873][60935] Updated weights for policy 0, policy_version 400 (0.0008) [2023-10-13 20:54:15,105][60934] Updated weights for policy 1, policy_version 410 (0.0009) [2023-10-13 20:54:15,246][60935] Updated weights for policy 0, policy_version 410 (0.0007) [2023-10-13 20:54:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 12590.4). Total num frames: 851968. Throughput: 0: 1683.2, 1: 1697.3. Samples: 212642. Policy #0 lag: (min: 22.0, avg: 25.0, max: 54.0) [2023-10-13 20:54:16,249][59943] Avg episode reward: [(0, '-0.890'), (1, '-0.590')] [2023-10-13 20:54:16,250][60695] Saving new best policy, reward=-0.890! [2023-10-13 20:54:16,251][60828] Saving new best policy, reward=-0.590! [2023-10-13 20:54:19,139][60934] Updated weights for policy 1, policy_version 420 (0.0009) [2023-10-13 20:54:19,352][60935] Updated weights for policy 0, policy_version 420 (0.0009) [2023-10-13 20:54:19,503][60934] Updated weights for policy 1, policy_version 430 (0.0009) [2023-10-13 20:54:19,716][60935] Updated weights for policy 0, policy_version 430 (0.0007) [2023-10-13 20:54:19,871][60934] Updated weights for policy 1, policy_version 440 (0.0008) [2023-10-13 20:54:20,090][60935] Updated weights for policy 0, policy_version 440 (0.0008) [2023-10-13 20:54:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 12626.0). Total num frames: 917504. Throughput: 0: 1671.7, 1: 1685.9. Samples: 232412. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-13 20:54:21,249][59943] Avg episode reward: [(0, '-0.840'), (1, '-0.920')] [2023-10-13 20:54:21,250][60695] Saving new best policy, reward=-0.840! [2023-10-13 20:54:23,740][60934] Updated weights for policy 1, policy_version 450 (0.0008) [2023-10-13 20:54:24,108][60934] Updated weights for policy 1, policy_version 460 (0.0008) [2023-10-13 20:54:24,146][60935] Updated weights for policy 0, policy_version 450 (0.0009) [2023-10-13 20:54:24,478][60934] Updated weights for policy 1, policy_version 470 (0.0009) [2023-10-13 20:54:24,525][60935] Updated weights for policy 0, policy_version 460 (0.0008) [2023-10-13 20:54:24,838][60934] Updated weights for policy 1, policy_version 480 (0.0008) [2023-10-13 20:54:24,891][60935] Updated weights for policy 0, policy_version 470 (0.0007) [2023-10-13 20:54:25,254][60935] Updated weights for policy 0, policy_version 480 (0.0008) [2023-10-13 20:54:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 12656.9). Total num frames: 983040. Throughput: 0: 1664.1, 1: 1682.9. Samples: 251960. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-10-13 20:54:26,249][59943] Avg episode reward: [(0, '-0.570'), (1, '-0.750')] [2023-10-13 20:54:26,261][60695] Saving new best policy, reward=-0.570! [2023-10-13 20:54:29,135][60934] Updated weights for policy 1, policy_version 490 (0.0008) [2023-10-13 20:54:29,280][60935] Updated weights for policy 0, policy_version 490 (0.0007) [2023-10-13 20:54:29,487][60934] Updated weights for policy 1, policy_version 500 (0.0008) [2023-10-13 20:54:29,645][60935] Updated weights for policy 0, policy_version 500 (0.0007) [2023-10-13 20:54:29,862][60934] Updated weights for policy 1, policy_version 510 (0.0008) [2023-10-13 20:54:30,012][60935] Updated weights for policy 0, policy_version 510 (0.0009) [2023-10-13 20:54:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 12684.2). Total num frames: 1048576. Throughput: 0: 1683.2, 1: 1695.0. Samples: 263526. Policy #0 lag: (min: 26.0, avg: 26.3, max: 37.0) [2023-10-13 20:54:31,249][59943] Avg episode reward: [(0, '-0.730'), (1, '-0.710')] [2023-10-13 20:54:33,908][60934] Updated weights for policy 1, policy_version 520 (0.0008) [2023-10-13 20:54:34,123][60935] Updated weights for policy 0, policy_version 520 (0.0008) [2023-10-13 20:54:34,278][60934] Updated weights for policy 1, policy_version 530 (0.0008) [2023-10-13 20:54:34,493][60935] Updated weights for policy 0, policy_version 530 (0.0009) [2023-10-13 20:54:34,638][60934] Updated weights for policy 1, policy_version 540 (0.0009) [2023-10-13 20:54:34,855][60935] Updated weights for policy 0, policy_version 540 (0.0008) [2023-10-13 20:54:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 12708.3). Total num frames: 1114112. Throughput: 0: 1662.2, 1: 1674.5. Samples: 282204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:54:36,249][59943] Avg episode reward: [(0, '-0.660'), (1, '-0.350')] [2023-10-13 20:54:36,249][60828] Saving new best policy, reward=-0.350! [2023-10-13 20:54:38,591][60934] Updated weights for policy 1, policy_version 550 (0.0008) [2023-10-13 20:54:38,932][60935] Updated weights for policy 0, policy_version 550 (0.0007) [2023-10-13 20:54:38,964][60934] Updated weights for policy 1, policy_version 560 (0.0007) [2023-10-13 20:54:39,299][60935] Updated weights for policy 0, policy_version 560 (0.0008) [2023-10-13 20:54:39,327][60934] Updated weights for policy 1, policy_version 570 (0.0008) [2023-10-13 20:54:39,670][60935] Updated weights for policy 0, policy_version 570 (0.0007) [2023-10-13 20:54:41,248][59943] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 12729.8). Total num frames: 1179648. Throughput: 0: 1675.9, 1: 1690.3. Samples: 302518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:54:41,249][59943] Avg episode reward: [(0, '-0.510'), (1, '-0.270')] [2023-10-13 20:54:41,260][60695] Saving new best policy, reward=-0.510! [2023-10-13 20:54:41,261][60828] Saving new best policy, reward=-0.270! [2023-10-13 20:54:43,596][60934] Updated weights for policy 1, policy_version 580 (0.0009) [2023-10-13 20:54:43,923][60935] Updated weights for policy 0, policy_version 580 (0.0007) [2023-10-13 20:54:43,961][60934] Updated weights for policy 1, policy_version 590 (0.0010) [2023-10-13 20:54:44,286][60935] Updated weights for policy 0, policy_version 590 (0.0009) [2023-10-13 20:54:44,334][60934] Updated weights for policy 1, policy_version 600 (0.0008) [2023-10-13 20:54:44,665][60935] Updated weights for policy 0, policy_version 600 (0.0008) [2023-10-13 20:54:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 12749.1). Total num frames: 1245184. Throughput: 0: 1675.8, 1: 1690.4. Samples: 313804. Policy #0 lag: (min: 17.0, avg: 19.8, max: 49.0) [2023-10-13 20:54:46,249][59943] Avg episode reward: [(0, '-0.550'), (1, '-0.270')] [2023-10-13 20:54:48,436][60934] Updated weights for policy 1, policy_version 610 (0.0008) [2023-10-13 20:54:48,831][60935] Updated weights for policy 0, policy_version 610 (0.0008) [2023-10-13 20:54:48,836][60934] Updated weights for policy 1, policy_version 620 (0.0009) [2023-10-13 20:54:49,200][60934] Updated weights for policy 1, policy_version 630 (0.0007) [2023-10-13 20:54:49,230][60935] Updated weights for policy 0, policy_version 620 (0.0008) [2023-10-13 20:54:49,567][60934] Updated weights for policy 1, policy_version 640 (0.0008) [2023-10-13 20:54:49,589][60935] Updated weights for policy 0, policy_version 630 (0.0010) [2023-10-13 20:54:49,967][60935] Updated weights for policy 0, policy_version 640 (0.0008) [2023-10-13 20:54:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12766.6). Total num frames: 1310720. Throughput: 0: 1662.1, 1: 1669.6. Samples: 332122. Policy #0 lag: (min: 28.0, avg: 33.5, max: 60.0) [2023-10-13 20:54:51,249][59943] Avg episode reward: [(0, '-0.500'), (1, '-0.270')] [2023-10-13 20:54:51,251][60695] Saving new best policy, reward=-0.500! [2023-10-13 20:54:53,504][60934] Updated weights for policy 1, policy_version 650 (0.0007) [2023-10-13 20:54:53,870][60934] Updated weights for policy 1, policy_version 660 (0.0009) [2023-10-13 20:54:53,876][60935] Updated weights for policy 0, policy_version 650 (0.0008) [2023-10-13 20:54:54,241][60934] Updated weights for policy 1, policy_version 670 (0.0008) [2023-10-13 20:54:54,249][60935] Updated weights for policy 0, policy_version 660 (0.0009) [2023-10-13 20:54:54,616][60935] Updated weights for policy 0, policy_version 670 (0.0011) [2023-10-13 20:54:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 12782.4). Total num frames: 1376256. Throughput: 0: 1684.8, 1: 1694.4. Samples: 352830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:54:56,249][59943] Avg episode reward: [(0, '-0.520'), (1, '-0.130')] [2023-10-13 20:54:56,257][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000000672_688128.pth... [2023-10-13 20:54:56,257][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000000672_688128.pth... [2023-10-13 20:54:56,294][60828] Saving new best policy, reward=-0.130! [2023-10-13 20:54:57,989][60934] Updated weights for policy 1, policy_version 680 (0.0007) [2023-10-13 20:54:58,358][60934] Updated weights for policy 1, policy_version 690 (0.0009) [2023-10-13 20:54:58,727][60935] Updated weights for policy 0, policy_version 680 (0.0008) [2023-10-13 20:54:58,730][60934] Updated weights for policy 1, policy_version 700 (0.0007) [2023-10-13 20:54:59,102][60935] Updated weights for policy 0, policy_version 690 (0.0008) [2023-10-13 20:54:59,464][60935] Updated weights for policy 0, policy_version 700 (0.0009) [2023-10-13 20:55:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12796.8). Total num frames: 1441792. Throughput: 0: 1668.4, 1: 1676.3. Samples: 363152. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-13 20:55:01,249][59943] Avg episode reward: [(0, '-0.540'), (1, '-0.020')] [2023-10-13 20:55:01,251][60828] Saving new best policy, reward=-0.020! [2023-10-13 20:55:02,843][60934] Updated weights for policy 1, policy_version 710 (0.0007) [2023-10-13 20:55:03,202][60934] Updated weights for policy 1, policy_version 720 (0.0007) [2023-10-13 20:55:03,562][60934] Updated weights for policy 1, policy_version 730 (0.0009) [2023-10-13 20:55:03,580][60935] Updated weights for policy 0, policy_version 710 (0.0010) [2023-10-13 20:55:03,946][60935] Updated weights for policy 0, policy_version 720 (0.0007) [2023-10-13 20:55:04,319][60935] Updated weights for policy 0, policy_version 730 (0.0007) [2023-10-13 20:55:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12810.0). Total num frames: 1507328. Throughput: 0: 1661.9, 1: 1674.1. Samples: 382534. Policy #0 lag: (min: 4.0, avg: 11.7, max: 36.0) [2023-10-13 20:55:06,248][59943] Avg episode reward: [(0, '-0.550'), (1, '-0.020')] [2023-10-13 20:55:07,742][60934] Updated weights for policy 1, policy_version 740 (0.0009) [2023-10-13 20:55:08,105][60934] Updated weights for policy 1, policy_version 750 (0.0009) [2023-10-13 20:55:08,257][60935] Updated weights for policy 0, policy_version 740 (0.0009) [2023-10-13 20:55:08,468][60934] Updated weights for policy 1, policy_version 760 (0.0009) [2023-10-13 20:55:08,631][60935] Updated weights for policy 0, policy_version 750 (0.0008) [2023-10-13 20:55:08,998][60935] Updated weights for policy 0, policy_version 760 (0.0008) [2023-10-13 20:55:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12822.1). Total num frames: 1572864. Throughput: 0: 1684.5, 1: 1680.1. Samples: 403368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:55:11,249][59943] Avg episode reward: [(0, '-0.330'), (1, '-0.020')] [2023-10-13 20:55:11,262][60695] Saving new best policy, reward=-0.330! [2023-10-13 20:55:12,648][60934] Updated weights for policy 1, policy_version 770 (0.0009) [2023-10-13 20:55:13,017][60934] Updated weights for policy 1, policy_version 780 (0.0010) [2023-10-13 20:55:13,149][60935] Updated weights for policy 0, policy_version 770 (0.0007) [2023-10-13 20:55:13,387][60934] Updated weights for policy 1, policy_version 790 (0.0010) [2023-10-13 20:55:13,521][60935] Updated weights for policy 0, policy_version 780 (0.0007) [2023-10-13 20:55:13,749][60934] Updated weights for policy 1, policy_version 800 (0.0008) [2023-10-13 20:55:13,891][60935] Updated weights for policy 0, policy_version 790 (0.0009) [2023-10-13 20:55:14,249][60935] Updated weights for policy 0, policy_version 800 (0.0011) [2023-10-13 20:55:16,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 12833.3). Total num frames: 1638400. Throughput: 0: 1664.4, 1: 1660.6. Samples: 413152. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-10-13 20:55:16,249][59943] Avg episode reward: [(0, '-0.270'), (1, '-0.120')] [2023-10-13 20:55:16,251][60695] Saving new best policy, reward=-0.270! [2023-10-13 20:55:17,852][60934] Updated weights for policy 1, policy_version 810 (0.0008) [2023-10-13 20:55:18,225][60934] Updated weights for policy 1, policy_version 820 (0.0008) [2023-10-13 20:55:18,295][60935] Updated weights for policy 0, policy_version 810 (0.0009) [2023-10-13 20:55:18,595][60934] Updated weights for policy 1, policy_version 830 (0.0007) [2023-10-13 20:55:18,672][60935] Updated weights for policy 0, policy_version 820 (0.0009) [2023-10-13 20:55:19,037][60935] Updated weights for policy 0, policy_version 830 (0.0009) [2023-10-13 20:55:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12843.6). Total num frames: 1703936. Throughput: 0: 1684.2, 1: 1671.2. Samples: 433198. Policy #0 lag: (min: 12.0, avg: 16.9, max: 44.0) [2023-10-13 20:55:21,249][59943] Avg episode reward: [(0, '-0.260'), (1, '-0.120')] [2023-10-13 20:55:21,250][60695] Saving new best policy, reward=-0.260! [2023-10-13 20:55:22,655][60934] Updated weights for policy 1, policy_version 840 (0.0009) [2023-10-13 20:55:23,021][60934] Updated weights for policy 1, policy_version 850 (0.0007) [2023-10-13 20:55:23,049][60935] Updated weights for policy 0, policy_version 840 (0.0008) [2023-10-13 20:55:23,387][60934] Updated weights for policy 1, policy_version 860 (0.0009) [2023-10-13 20:55:23,418][60935] Updated weights for policy 0, policy_version 850 (0.0008) [2023-10-13 20:55:23,793][60935] Updated weights for policy 0, policy_version 860 (0.0007) [2023-10-13 20:55:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12853.2). Total num frames: 1769472. Throughput: 0: 1688.5, 1: 1675.3. Samples: 453888. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-10-13 20:55:26,249][59943] Avg episode reward: [(0, '-0.260'), (1, '0.000')] [2023-10-13 20:55:26,258][60828] Saving new best policy, reward=0.000! [2023-10-13 20:55:27,705][60934] Updated weights for policy 1, policy_version 870 (0.0008) [2023-10-13 20:55:27,876][60935] Updated weights for policy 0, policy_version 870 (0.0009) [2023-10-13 20:55:28,075][60934] Updated weights for policy 1, policy_version 880 (0.0008) [2023-10-13 20:55:28,238][60935] Updated weights for policy 0, policy_version 880 (0.0008) [2023-10-13 20:55:28,446][60934] Updated weights for policy 1, policy_version 890 (0.0008) [2023-10-13 20:55:28,604][60935] Updated weights for policy 0, policy_version 890 (0.0009) [2023-10-13 20:55:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12862.1). Total num frames: 1835008. Throughput: 0: 1666.8, 1: 1651.9. Samples: 463142. Policy #0 lag: (min: 23.0, avg: 46.1, max: 48.0) [2023-10-13 20:55:31,249][59943] Avg episode reward: [(0, '-0.280'), (1, '0.000')] [2023-10-13 20:55:32,430][60934] Updated weights for policy 1, policy_version 900 (0.0010) [2023-10-13 20:55:32,628][60935] Updated weights for policy 0, policy_version 900 (0.0008) [2023-10-13 20:55:32,789][60934] Updated weights for policy 1, policy_version 910 (0.0009) [2023-10-13 20:55:32,995][60935] Updated weights for policy 0, policy_version 910 (0.0007) [2023-10-13 20:55:33,166][60934] Updated weights for policy 1, policy_version 920 (0.0009) [2023-10-13 20:55:33,366][60935] Updated weights for policy 0, policy_version 920 (0.0008) [2023-10-13 20:55:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12870.4). Total num frames: 1900544. Throughput: 0: 1691.2, 1: 1672.5. Samples: 483486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:55:36,249][59943] Avg episode reward: [(0, '-0.330'), (1, '0.000')] [2023-10-13 20:55:37,388][60934] Updated weights for policy 1, policy_version 930 (0.0009) [2023-10-13 20:55:37,568][60935] Updated weights for policy 0, policy_version 930 (0.0009) [2023-10-13 20:55:37,788][60934] Updated weights for policy 1, policy_version 940 (0.0008) [2023-10-13 20:55:37,958][60935] Updated weights for policy 0, policy_version 940 (0.0009) [2023-10-13 20:55:38,151][60934] Updated weights for policy 1, policy_version 950 (0.0010) [2023-10-13 20:55:38,331][60935] Updated weights for policy 0, policy_version 950 (0.0008) [2023-10-13 20:55:38,514][60934] Updated weights for policy 1, policy_version 960 (0.0009) [2023-10-13 20:55:38,701][60935] Updated weights for policy 0, policy_version 960 (0.0011) [2023-10-13 20:55:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12878.2). Total num frames: 1966080. Throughput: 0: 1690.6, 1: 1667.0. Samples: 503920. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) [2023-10-13 20:55:41,249][59943] Avg episode reward: [(0, '-0.510'), (1, '0.000')] [2023-10-13 20:55:42,549][60934] Updated weights for policy 1, policy_version 970 (0.0008) [2023-10-13 20:55:42,716][60935] Updated weights for policy 0, policy_version 970 (0.0008) [2023-10-13 20:55:42,917][60934] Updated weights for policy 1, policy_version 980 (0.0007) [2023-10-13 20:55:43,081][60935] Updated weights for policy 0, policy_version 980 (0.0009) [2023-10-13 20:55:43,286][60934] Updated weights for policy 1, policy_version 990 (0.0010) [2023-10-13 20:55:43,457][60935] Updated weights for policy 0, policy_version 990 (0.0009) [2023-10-13 20:55:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12885.4). Total num frames: 2031616. Throughput: 0: 1673.8, 1: 1656.9. Samples: 513032. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 20:55:46,249][59943] Avg episode reward: [(0, '-0.440'), (1, '0.000')] [2023-10-13 20:55:47,420][60934] Updated weights for policy 1, policy_version 1000 (0.0008) [2023-10-13 20:55:47,549][60935] Updated weights for policy 0, policy_version 1000 (0.0008) [2023-10-13 20:55:47,788][60934] Updated weights for policy 1, policy_version 1010 (0.0007) [2023-10-13 20:55:47,913][60935] Updated weights for policy 0, policy_version 1010 (0.0008) [2023-10-13 20:55:48,157][60934] Updated weights for policy 1, policy_version 1020 (0.0008) [2023-10-13 20:55:48,290][60935] Updated weights for policy 0, policy_version 1020 (0.0010) [2023-10-13 20:55:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12892.2). Total num frames: 2097152. Throughput: 0: 1689.0, 1: 1669.0. Samples: 533644. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 20:55:51,249][59943] Avg episode reward: [(0, '-0.270'), (1, '-0.010')] [2023-10-13 20:55:52,262][60934] Updated weights for policy 1, policy_version 1030 (0.0009) [2023-10-13 20:55:52,564][60935] Updated weights for policy 0, policy_version 1030 (0.0009) [2023-10-13 20:55:52,628][60934] Updated weights for policy 1, policy_version 1040 (0.0008) [2023-10-13 20:55:52,941][60935] Updated weights for policy 0, policy_version 1040 (0.0008) [2023-10-13 20:55:52,989][60934] Updated weights for policy 1, policy_version 1050 (0.0008) [2023-10-13 20:55:53,318][60935] Updated weights for policy 0, policy_version 1050 (0.0010) [2023-10-13 20:55:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12898.6). Total num frames: 2162688. Throughput: 0: 1677.2, 1: 1677.8. Samples: 554342. Policy #0 lag: (min: 3.0, avg: 5.2, max: 35.0) [2023-10-13 20:55:56,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 20:55:56,259][60695] Saving new best policy, reward=-0.010! [2023-10-13 20:55:57,073][60934] Updated weights for policy 1, policy_version 1060 (0.0008) [2023-10-13 20:55:57,281][60935] Updated weights for policy 0, policy_version 1060 (0.0009) [2023-10-13 20:55:57,445][60934] Updated weights for policy 1, policy_version 1070 (0.0010) [2023-10-13 20:55:57,651][60935] Updated weights for policy 0, policy_version 1070 (0.0009) [2023-10-13 20:55:57,801][60934] Updated weights for policy 1, policy_version 1080 (0.0008) [2023-10-13 20:55:58,027][60935] Updated weights for policy 0, policy_version 1080 (0.0008) [2023-10-13 20:56:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12904.7). Total num frames: 2228224. Throughput: 0: 1668.7, 1: 1670.0. Samples: 563392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:56:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:56:01,250][60695] Saving new best policy, reward=0.000! [2023-10-13 20:56:01,841][60934] Updated weights for policy 1, policy_version 1090 (0.0009) [2023-10-13 20:56:02,030][60935] Updated weights for policy 0, policy_version 1090 (0.0010) [2023-10-13 20:56:02,211][60934] Updated weights for policy 1, policy_version 1100 (0.0011) [2023-10-13 20:56:02,409][60935] Updated weights for policy 0, policy_version 1100 (0.0007) [2023-10-13 20:56:02,577][60934] Updated weights for policy 1, policy_version 1110 (0.0007) [2023-10-13 20:56:02,780][60935] Updated weights for policy 0, policy_version 1110 (0.0007) [2023-10-13 20:56:02,945][60934] Updated weights for policy 1, policy_version 1120 (0.0007) [2023-10-13 20:56:03,142][60935] Updated weights for policy 0, policy_version 1120 (0.0007) [2023-10-13 20:56:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 12910.4). Total num frames: 2293760. Throughput: 0: 1675.3, 1: 1679.5. Samples: 584164. Policy #0 lag: (min: 6.0, avg: 8.0, max: 37.0) [2023-10-13 20:56:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:56:06,951][60934] Updated weights for policy 1, policy_version 1130 (0.0010) [2023-10-13 20:56:07,312][60934] Updated weights for policy 1, policy_version 1140 (0.0008) [2023-10-13 20:56:07,336][60935] Updated weights for policy 0, policy_version 1130 (0.0008) [2023-10-13 20:56:07,683][60934] Updated weights for policy 1, policy_version 1150 (0.0008) [2023-10-13 20:56:07,712][60935] Updated weights for policy 0, policy_version 1140 (0.0008) [2023-10-13 20:56:08,073][60935] Updated weights for policy 0, policy_version 1150 (0.0010) [2023-10-13 20:56:11,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 12915.8). Total num frames: 2359296. Throughput: 0: 1673.1, 1: 1678.1. Samples: 604694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:56:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:56:11,808][60934] Updated weights for policy 1, policy_version 1160 (0.0008) [2023-10-13 20:56:11,930][60935] Updated weights for policy 0, policy_version 1160 (0.0008) [2023-10-13 20:56:12,183][60934] Updated weights for policy 1, policy_version 1170 (0.0010) [2023-10-13 20:56:12,305][60935] Updated weights for policy 0, policy_version 1170 (0.0008) [2023-10-13 20:56:12,541][60934] Updated weights for policy 1, policy_version 1180 (0.0007) [2023-10-13 20:56:12,669][60935] Updated weights for policy 0, policy_version 1180 (0.0010) [2023-10-13 20:56:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12920.9). Total num frames: 2424832. Throughput: 0: 1671.6, 1: 1675.6. Samples: 613766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:56:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:56:16,742][60934] Updated weights for policy 1, policy_version 1190 (0.0008) [2023-10-13 20:56:16,822][60935] Updated weights for policy 0, policy_version 1190 (0.0008) [2023-10-13 20:56:17,106][60934] Updated weights for policy 1, policy_version 1200 (0.0009) [2023-10-13 20:56:17,194][60935] Updated weights for policy 0, policy_version 1200 (0.0009) [2023-10-13 20:56:17,479][60934] Updated weights for policy 1, policy_version 1210 (0.0008) [2023-10-13 20:56:17,567][60935] Updated weights for policy 0, policy_version 1210 (0.0009) [2023-10-13 20:56:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12925.7). Total num frames: 2490368. Throughput: 0: 1673.1, 1: 1680.4. Samples: 634392. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 20:56:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:56:21,556][60935] Updated weights for policy 0, policy_version 1220 (0.0008) [2023-10-13 20:56:21,583][60934] Updated weights for policy 1, policy_version 1220 (0.0007) [2023-10-13 20:56:21,915][60935] Updated weights for policy 0, policy_version 1230 (0.0009) [2023-10-13 20:56:21,949][60934] Updated weights for policy 1, policy_version 1230 (0.0008) [2023-10-13 20:56:22,289][60935] Updated weights for policy 0, policy_version 1240 (0.0009) [2023-10-13 20:56:22,314][60934] Updated weights for policy 1, policy_version 1240 (0.0008) [2023-10-13 20:56:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12930.3). Total num frames: 2555904. Throughput: 0: 1684.0, 1: 1683.9. Samples: 655480. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-13 20:56:26,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 20:56:26,357][60935] Updated weights for policy 0, policy_version 1250 (0.0007) [2023-10-13 20:56:26,435][60934] Updated weights for policy 1, policy_version 1250 (0.0007) [2023-10-13 20:56:26,767][60935] Updated weights for policy 0, policy_version 1260 (0.0009) [2023-10-13 20:56:26,835][60934] Updated weights for policy 1, policy_version 1260 (0.0008) [2023-10-13 20:56:27,141][60935] Updated weights for policy 0, policy_version 1270 (0.0009) [2023-10-13 20:56:27,194][60934] Updated weights for policy 1, policy_version 1270 (0.0009) [2023-10-13 20:56:27,513][60935] Updated weights for policy 0, policy_version 1280 (0.0009) [2023-10-13 20:56:27,561][60934] Updated weights for policy 1, policy_version 1280 (0.0008) [2023-10-13 20:56:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12934.7). Total num frames: 2621440. Throughput: 0: 1680.1, 1: 1682.3. Samples: 664340. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 20:56:31,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 20:56:31,614][60934] Updated weights for policy 1, policy_version 1290 (0.0009) [2023-10-13 20:56:31,691][60935] Updated weights for policy 0, policy_version 1290 (0.0009) [2023-10-13 20:56:31,976][60934] Updated weights for policy 1, policy_version 1300 (0.0008) [2023-10-13 20:56:32,051][60935] Updated weights for policy 0, policy_version 1300 (0.0009) [2023-10-13 20:56:32,346][60934] Updated weights for policy 1, policy_version 1310 (0.0009) [2023-10-13 20:56:32,410][60935] Updated weights for policy 0, policy_version 1310 (0.0010) [2023-10-13 20:56:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12938.8). Total num frames: 2686976. Throughput: 0: 1683.0, 1: 1678.0. Samples: 684888. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) [2023-10-13 20:56:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:56:36,363][60934] Updated weights for policy 1, policy_version 1320 (0.0007) [2023-10-13 20:56:36,466][60935] Updated weights for policy 0, policy_version 1320 (0.0009) [2023-10-13 20:56:36,724][60934] Updated weights for policy 1, policy_version 1330 (0.0008) [2023-10-13 20:56:36,839][60935] Updated weights for policy 0, policy_version 1330 (0.0009) [2023-10-13 20:56:37,091][60934] Updated weights for policy 1, policy_version 1340 (0.0009) [2023-10-13 20:56:37,204][60935] Updated weights for policy 0, policy_version 1340 (0.0008) [2023-10-13 20:56:41,132][60934] Updated weights for policy 1, policy_version 1350 (0.0008) [2023-10-13 20:56:41,221][60935] Updated weights for policy 0, policy_version 1350 (0.0009) [2023-10-13 20:56:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12942.8). Total num frames: 2752512. Throughput: 0: 1687.7, 1: 1665.3. Samples: 705226. Policy #0 lag: (min: 28.0, avg: 30.8, max: 60.0) [2023-10-13 20:56:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:56:41,502][60934] Updated weights for policy 1, policy_version 1360 (0.0008) [2023-10-13 20:56:41,595][60935] Updated weights for policy 0, policy_version 1360 (0.0009) [2023-10-13 20:56:41,870][60934] Updated weights for policy 1, policy_version 1370 (0.0007) [2023-10-13 20:56:41,967][60935] Updated weights for policy 0, policy_version 1370 (0.0010) [2023-10-13 20:56:46,022][60934] Updated weights for policy 1, policy_version 1380 (0.0009) [2023-10-13 20:56:46,194][60935] Updated weights for policy 0, policy_version 1380 (0.0007) [2023-10-13 20:56:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12946.6). Total num frames: 2818048. Throughput: 0: 1684.5, 1: 1666.8. Samples: 714198. Policy #0 lag: (min: 10.0, avg: 12.7, max: 42.0) [2023-10-13 20:56:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:56:46,385][60934] Updated weights for policy 1, policy_version 1390 (0.0008) [2023-10-13 20:56:46,569][60935] Updated weights for policy 0, policy_version 1390 (0.0009) [2023-10-13 20:56:46,760][60934] Updated weights for policy 1, policy_version 1400 (0.0008) [2023-10-13 20:56:46,933][60935] Updated weights for policy 0, policy_version 1400 (0.0010) [2023-10-13 20:56:50,897][60935] Updated weights for policy 0, policy_version 1410 (0.0010) [2023-10-13 20:56:50,980][60934] Updated weights for policy 1, policy_version 1410 (0.0008) [2023-10-13 20:56:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12950.2). Total num frames: 2883584. Throughput: 0: 1681.3, 1: 1663.6. Samples: 734682. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-13 20:56:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:56:51,271][60935] Updated weights for policy 0, policy_version 1420 (0.0009) [2023-10-13 20:56:51,342][60934] Updated weights for policy 1, policy_version 1420 (0.0009) [2023-10-13 20:56:51,639][60935] Updated weights for policy 0, policy_version 1430 (0.0009) [2023-10-13 20:56:51,719][60934] Updated weights for policy 1, policy_version 1430 (0.0009) [2023-10-13 20:56:52,011][60935] Updated weights for policy 0, policy_version 1440 (0.0010) [2023-10-13 20:56:52,080][60934] Updated weights for policy 1, policy_version 1440 (0.0008) [2023-10-13 20:56:55,992][60934] Updated weights for policy 1, policy_version 1450 (0.0009) [2023-10-13 20:56:56,064][60935] Updated weights for policy 0, policy_version 1450 (0.0010) [2023-10-13 20:56:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12953.6). Total num frames: 2949120. Throughput: 0: 1679.7, 1: 1670.2. Samples: 755440. Policy #0 lag: (min: 15.0, avg: 15.3, max: 27.0) [2023-10-13 20:56:56,249][59943] Avg episode reward: [(0, '-0.130'), (1, '0.000')] [2023-10-13 20:56:56,371][60934] Updated weights for policy 1, policy_version 1460 (0.0009) [2023-10-13 20:56:56,438][60935] Updated weights for policy 0, policy_version 1460 (0.0008) [2023-10-13 20:56:56,732][60934] Updated weights for policy 1, policy_version 1470 (0.0008) [2023-10-13 20:56:56,800][60935] Updated weights for policy 0, policy_version 1470 (0.0009) [2023-10-13 20:56:56,804][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000001472_1507328.pth... [2023-10-13 20:56:56,874][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000001472_1507328.pth... [2023-10-13 20:57:00,767][60935] Updated weights for policy 0, policy_version 1480 (0.0008) [2023-10-13 20:57:00,772][60934] Updated weights for policy 1, policy_version 1480 (0.0009) [2023-10-13 20:57:01,133][60934] Updated weights for policy 1, policy_version 1490 (0.0009) [2023-10-13 20:57:01,146][60935] Updated weights for policy 0, policy_version 1490 (0.0009) [2023-10-13 20:57:01,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12956.9). Total num frames: 3014656. Throughput: 0: 1682.1, 1: 1671.6. Samples: 764680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:57:01,249][59943] Avg episode reward: [(0, '-0.030'), (1, '0.000')] [2023-10-13 20:57:01,500][60934] Updated weights for policy 1, policy_version 1500 (0.0008) [2023-10-13 20:57:01,510][60935] Updated weights for policy 0, policy_version 1500 (0.0009) [2023-10-13 20:57:05,606][60935] Updated weights for policy 0, policy_version 1510 (0.0008) [2023-10-13 20:57:05,681][60934] Updated weights for policy 1, policy_version 1510 (0.0008) [2023-10-13 20:57:05,978][60935] Updated weights for policy 0, policy_version 1520 (0.0008) [2023-10-13 20:57:06,064][60934] Updated weights for policy 1, policy_version 1520 (0.0009) [2023-10-13 20:57:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12960.1). Total num frames: 3080192. Throughput: 0: 1681.2, 1: 1668.4. Samples: 785120. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-13 20:57:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:57:06,356][60935] Updated weights for policy 0, policy_version 1530 (0.0009) [2023-10-13 20:57:06,425][60934] Updated weights for policy 1, policy_version 1530 (0.0007) [2023-10-13 20:57:10,554][60935] Updated weights for policy 0, policy_version 1540 (0.0010) [2023-10-13 20:57:10,620][60934] Updated weights for policy 1, policy_version 1540 (0.0009) [2023-10-13 20:57:10,926][60935] Updated weights for policy 0, policy_version 1550 (0.0010) [2023-10-13 20:57:10,995][60934] Updated weights for policy 1, policy_version 1550 (0.0008) [2023-10-13 20:57:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12963.1). Total num frames: 3145728. Throughput: 0: 1660.1, 1: 1660.2. Samples: 804894. Policy #0 lag: (min: 5.0, avg: 10.2, max: 37.0) [2023-10-13 20:57:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:57:11,299][60935] Updated weights for policy 0, policy_version 1560 (0.0011) [2023-10-13 20:57:11,361][60934] Updated weights for policy 1, policy_version 1560 (0.0008) [2023-10-13 20:57:15,392][60935] Updated weights for policy 0, policy_version 1570 (0.0008) [2023-10-13 20:57:15,717][60934] Updated weights for policy 1, policy_version 1570 (0.0008) [2023-10-13 20:57:15,813][60935] Updated weights for policy 0, policy_version 1580 (0.0009) [2023-10-13 20:57:16,132][60934] Updated weights for policy 1, policy_version 1580 (0.0007) [2023-10-13 20:57:16,177][60935] Updated weights for policy 0, policy_version 1590 (0.0009) [2023-10-13 20:57:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12966.0). Total num frames: 3211264. Throughput: 0: 1675.9, 1: 1661.8. Samples: 814534. Policy #0 lag: (min: 17.0, avg: 17.5, max: 33.0) [2023-10-13 20:57:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:57:16,507][60934] Updated weights for policy 1, policy_version 1590 (0.0009) [2023-10-13 20:57:16,552][60935] Updated weights for policy 0, policy_version 1600 (0.0008) [2023-10-13 20:57:16,873][60934] Updated weights for policy 1, policy_version 1600 (0.0009) [2023-10-13 20:57:20,658][60935] Updated weights for policy 0, policy_version 1610 (0.0008) [2023-10-13 20:57:21,032][60935] Updated weights for policy 0, policy_version 1620 (0.0008) [2023-10-13 20:57:21,042][60934] Updated weights for policy 1, policy_version 1610 (0.0008) [2023-10-13 20:57:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12968.8). Total num frames: 3276800. Throughput: 0: 1668.6, 1: 1654.4. Samples: 834426. Policy #0 lag: (min: 30.0, avg: 30.6, max: 47.0) [2023-10-13 20:57:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:57:21,403][60934] Updated weights for policy 1, policy_version 1620 (0.0008) [2023-10-13 20:57:21,406][60935] Updated weights for policy 0, policy_version 1630 (0.0008) [2023-10-13 20:57:21,775][60934] Updated weights for policy 1, policy_version 1630 (0.0010) [2023-10-13 20:57:25,442][60935] Updated weights for policy 0, policy_version 1640 (0.0007) [2023-10-13 20:57:25,807][60935] Updated weights for policy 0, policy_version 1650 (0.0008) [2023-10-13 20:57:25,870][60934] Updated weights for policy 1, policy_version 1640 (0.0009) [2023-10-13 20:57:26,182][60935] Updated weights for policy 0, policy_version 1660 (0.0007) [2023-10-13 20:57:26,241][60934] Updated weights for policy 1, policy_version 1650 (0.0008) [2023-10-13 20:57:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12971.5). Total num frames: 3342336. Throughput: 0: 1654.5, 1: 1656.5. Samples: 854220. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 20:57:26,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 20:57:26,613][60934] Updated weights for policy 1, policy_version 1660 (0.0007) [2023-10-13 20:57:30,466][60935] Updated weights for policy 0, policy_version 1670 (0.0008) [2023-10-13 20:57:30,732][60934] Updated weights for policy 1, policy_version 1670 (0.0009) [2023-10-13 20:57:30,824][60935] Updated weights for policy 0, policy_version 1680 (0.0007) [2023-10-13 20:57:31,103][60934] Updated weights for policy 1, policy_version 1680 (0.0008) [2023-10-13 20:57:31,201][60935] Updated weights for policy 0, policy_version 1690 (0.0007) [2023-10-13 20:57:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12974.1). Total num frames: 3407872. Throughput: 0: 1668.8, 1: 1653.3. Samples: 863694. Policy #0 lag: (min: 4.0, avg: 11.4, max: 36.0) [2023-10-13 20:57:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:57:31,475][60934] Updated weights for policy 1, policy_version 1690 (0.0009) [2023-10-13 20:57:35,196][60935] Updated weights for policy 0, policy_version 1700 (0.0008) [2023-10-13 20:57:35,565][60935] Updated weights for policy 0, policy_version 1710 (0.0011) [2023-10-13 20:57:35,593][60934] Updated weights for policy 1, policy_version 1700 (0.0009) [2023-10-13 20:57:35,938][60935] Updated weights for policy 0, policy_version 1720 (0.0008) [2023-10-13 20:57:35,965][60934] Updated weights for policy 1, policy_version 1710 (0.0008) [2023-10-13 20:57:36,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13099.0). Total num frames: 3506176. Throughput: 0: 1670.7, 1: 1653.5. Samples: 884270. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 20:57:36,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 20:57:36,327][60934] Updated weights for policy 1, policy_version 1720 (0.0009) [2023-10-13 20:57:40,062][60935] Updated weights for policy 0, policy_version 1730 (0.0009) [2023-10-13 20:57:40,435][60935] Updated weights for policy 0, policy_version 1740 (0.0010) [2023-10-13 20:57:40,540][60934] Updated weights for policy 1, policy_version 1730 (0.0008) [2023-10-13 20:57:40,794][60935] Updated weights for policy 0, policy_version 1750 (0.0008) [2023-10-13 20:57:40,910][60934] Updated weights for policy 1, policy_version 1740 (0.0008) [2023-10-13 20:57:41,168][60935] Updated weights for policy 0, policy_version 1760 (0.0010) [2023-10-13 20:57:41,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13099.1). Total num frames: 3571712. Throughput: 0: 1650.4, 1: 1641.9. Samples: 903594. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-13 20:57:41,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 20:57:41,284][60934] Updated weights for policy 1, policy_version 1750 (0.0008) [2023-10-13 20:57:41,651][60934] Updated weights for policy 1, policy_version 1760 (0.0007) [2023-10-13 20:57:45,282][60935] Updated weights for policy 0, policy_version 1770 (0.0007) [2023-10-13 20:57:45,648][60935] Updated weights for policy 0, policy_version 1780 (0.0008) [2023-10-13 20:57:45,720][60934] Updated weights for policy 1, policy_version 1770 (0.0009) [2023-10-13 20:57:46,012][60935] Updated weights for policy 0, policy_version 1790 (0.0008) [2023-10-13 20:57:46,086][60934] Updated weights for policy 1, policy_version 1780 (0.0009) [2023-10-13 20:57:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13099.3). Total num frames: 3637248. Throughput: 0: 1665.8, 1: 1641.2. Samples: 913492. Policy #0 lag: (min: 8.0, avg: 22.2, max: 40.0) [2023-10-13 20:57:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:57:46,452][60934] Updated weights for policy 1, policy_version 1790 (0.0009) [2023-10-13 20:57:50,104][60935] Updated weights for policy 0, policy_version 1800 (0.0007) [2023-10-13 20:57:50,476][60935] Updated weights for policy 0, policy_version 1810 (0.0009) [2023-10-13 20:57:50,590][60934] Updated weights for policy 1, policy_version 1800 (0.0008) [2023-10-13 20:57:50,846][60935] Updated weights for policy 0, policy_version 1820 (0.0008) [2023-10-13 20:57:50,960][60934] Updated weights for policy 1, policy_version 1810 (0.0007) [2023-10-13 20:57:51,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13099.4). Total num frames: 3702784. Throughput: 0: 1660.0, 1: 1645.1. Samples: 933850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:57:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:57:51,328][60934] Updated weights for policy 1, policy_version 1820 (0.0011) [2023-10-13 20:57:54,998][60935] Updated weights for policy 0, policy_version 1830 (0.0008) [2023-10-13 20:57:55,368][60935] Updated weights for policy 0, policy_version 1840 (0.0008) [2023-10-13 20:57:55,554][60934] Updated weights for policy 1, policy_version 1830 (0.0008) [2023-10-13 20:57:55,738][60935] Updated weights for policy 0, policy_version 1850 (0.0009) [2023-10-13 20:57:55,925][60934] Updated weights for policy 1, policy_version 1840 (0.0006) [2023-10-13 20:57:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13099.5). Total num frames: 3768320. Throughput: 0: 1648.3, 1: 1641.3. Samples: 952926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:57:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:57:56,290][60934] Updated weights for policy 1, policy_version 1850 (0.0007) [2023-10-13 20:57:59,809][60935] Updated weights for policy 0, policy_version 1860 (0.0008) [2023-10-13 20:58:00,176][60935] Updated weights for policy 0, policy_version 1870 (0.0008) [2023-10-13 20:58:00,379][60934] Updated weights for policy 1, policy_version 1860 (0.0008) [2023-10-13 20:58:00,552][60935] Updated weights for policy 0, policy_version 1880 (0.0008) [2023-10-13 20:58:00,771][60934] Updated weights for policy 1, policy_version 1870 (0.0008) [2023-10-13 20:58:01,142][60934] Updated weights for policy 1, policy_version 1880 (0.0007) [2023-10-13 20:58:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13099.7). Total num frames: 3833856. Throughput: 0: 1660.0, 1: 1645.5. Samples: 963282. Policy #0 lag: (min: 26.0, avg: 27.5, max: 53.0) [2023-10-13 20:58:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:58:04,861][60935] Updated weights for policy 0, policy_version 1890 (0.0008) [2023-10-13 20:58:05,217][60934] Updated weights for policy 1, policy_version 1890 (0.0010) [2023-10-13 20:58:05,237][60935] Updated weights for policy 0, policy_version 1900 (0.0008) [2023-10-13 20:58:05,591][60934] Updated weights for policy 1, policy_version 1900 (0.0008) [2023-10-13 20:58:05,599][60935] Updated weights for policy 0, policy_version 1910 (0.0008) [2023-10-13 20:58:05,951][60934] Updated weights for policy 1, policy_version 1910 (0.0007) [2023-10-13 20:58:05,967][60935] Updated weights for policy 0, policy_version 1920 (0.0009) [2023-10-13 20:58:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 3899392. Throughput: 0: 1660.2, 1: 1655.5. Samples: 983634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:58:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:58:06,317][60934] Updated weights for policy 1, policy_version 1920 (0.0007) [2023-10-13 20:58:10,133][60935] Updated weights for policy 0, policy_version 1930 (0.0008) [2023-10-13 20:58:10,501][60934] Updated weights for policy 1, policy_version 1930 (0.0010) [2023-10-13 20:58:10,506][60935] Updated weights for policy 0, policy_version 1940 (0.0009) [2023-10-13 20:58:10,864][60934] Updated weights for policy 1, policy_version 1940 (0.0007) [2023-10-13 20:58:10,881][60935] Updated weights for policy 0, policy_version 1950 (0.0009) [2023-10-13 20:58:11,231][60934] Updated weights for policy 1, policy_version 1950 (0.0011) [2023-10-13 20:58:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 3964928. Throughput: 0: 1646.3, 1: 1646.6. Samples: 1002402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:58:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:58:15,034][60935] Updated weights for policy 0, policy_version 1960 (0.0008) [2023-10-13 20:58:15,386][60934] Updated weights for policy 1, policy_version 1960 (0.0010) [2023-10-13 20:58:15,411][60935] Updated weights for policy 0, policy_version 1970 (0.0008) [2023-10-13 20:58:15,756][60934] Updated weights for policy 1, policy_version 1970 (0.0009) [2023-10-13 20:58:15,769][60935] Updated weights for policy 0, policy_version 1980 (0.0007) [2023-10-13 20:58:16,129][60934] Updated weights for policy 1, policy_version 1980 (0.0009) [2023-10-13 20:58:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 4030464. Throughput: 0: 1655.6, 1: 1659.3. Samples: 1012866. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 20:58:16,249][59943] Avg episode reward: [(0, '-0.120'), (1, '0.000')] [2023-10-13 20:58:19,897][60935] Updated weights for policy 0, policy_version 1990 (0.0008) [2023-10-13 20:58:20,269][60935] Updated weights for policy 0, policy_version 2000 (0.0010) [2023-10-13 20:58:20,282][60934] Updated weights for policy 1, policy_version 1990 (0.0010) [2023-10-13 20:58:20,637][60935] Updated weights for policy 0, policy_version 2010 (0.0007) [2023-10-13 20:58:20,648][60934] Updated weights for policy 1, policy_version 2000 (0.0008) [2023-10-13 20:58:21,008][60934] Updated weights for policy 1, policy_version 2010 (0.0010) [2023-10-13 20:58:21,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 4128768. Throughput: 0: 1649.0, 1: 1656.7. Samples: 1033026. Policy #0 lag: (min: 1.0, avg: 6.7, max: 33.0) [2023-10-13 20:58:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:58:24,668][60935] Updated weights for policy 0, policy_version 2020 (0.0009) [2023-10-13 20:58:25,041][60935] Updated weights for policy 0, policy_version 2030 (0.0008) [2023-10-13 20:58:25,203][60934] Updated weights for policy 1, policy_version 2020 (0.0009) [2023-10-13 20:58:25,403][60935] Updated weights for policy 0, policy_version 2040 (0.0009) [2023-10-13 20:58:25,577][60934] Updated weights for policy 1, policy_version 2030 (0.0009) [2023-10-13 20:58:25,938][60934] Updated weights for policy 1, policy_version 2040 (0.0008) [2023-10-13 20:58:26,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 4194304. Throughput: 0: 1648.8, 1: 1646.3. Samples: 1051874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-13 20:58:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:58:29,566][60935] Updated weights for policy 0, policy_version 2050 (0.0008) [2023-10-13 20:58:29,937][60935] Updated weights for policy 0, policy_version 2060 (0.0007) [2023-10-13 20:58:30,004][60934] Updated weights for policy 1, policy_version 2050 (0.0008) [2023-10-13 20:58:30,306][60935] Updated weights for policy 0, policy_version 2070 (0.0007) [2023-10-13 20:58:30,369][60934] Updated weights for policy 1, policy_version 2060 (0.0008) [2023-10-13 20:58:30,676][60935] Updated weights for policy 0, policy_version 2080 (0.0007) [2023-10-13 20:58:30,729][60934] Updated weights for policy 1, policy_version 2070 (0.0008) [2023-10-13 20:58:31,098][60934] Updated weights for policy 1, policy_version 2080 (0.0009) [2023-10-13 20:58:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 4259840. Throughput: 0: 1658.0, 1: 1659.7. Samples: 1062790. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-13 20:58:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:58:35,024][60935] Updated weights for policy 0, policy_version 2090 (0.0009) [2023-10-13 20:58:35,125][60934] Updated weights for policy 1, policy_version 2090 (0.0007) [2023-10-13 20:58:35,396][60935] Updated weights for policy 0, policy_version 2100 (0.0008) [2023-10-13 20:58:35,494][60934] Updated weights for policy 1, policy_version 2100 (0.0007) [2023-10-13 20:58:35,767][60935] Updated weights for policy 0, policy_version 2110 (0.0007) [2023-10-13 20:58:35,855][60934] Updated weights for policy 1, policy_version 2110 (0.0008) [2023-10-13 20:58:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 4325376. Throughput: 0: 1656.7, 1: 1660.1. Samples: 1083106. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-13 20:58:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:58:39,734][60935] Updated weights for policy 0, policy_version 2120 (0.0008) [2023-10-13 20:58:40,006][60934] Updated weights for policy 1, policy_version 2120 (0.0009) [2023-10-13 20:58:40,105][60935] Updated weights for policy 0, policy_version 2130 (0.0008) [2023-10-13 20:58:40,385][60934] Updated weights for policy 1, policy_version 2130 (0.0008) [2023-10-13 20:58:40,466][60935] Updated weights for policy 0, policy_version 2140 (0.0008) [2023-10-13 20:58:40,754][60934] Updated weights for policy 1, policy_version 2140 (0.0007) [2023-10-13 20:58:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 4390912. Throughput: 0: 1657.0, 1: 1649.5. Samples: 1101716. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-13 20:58:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:58:44,472][60935] Updated weights for policy 0, policy_version 2150 (0.0007) [2023-10-13 20:58:44,793][60934] Updated weights for policy 1, policy_version 2150 (0.0007) [2023-10-13 20:58:44,842][60935] Updated weights for policy 0, policy_version 2160 (0.0009) [2023-10-13 20:58:45,163][60934] Updated weights for policy 1, policy_version 2160 (0.0010) [2023-10-13 20:58:45,218][60935] Updated weights for policy 0, policy_version 2170 (0.0007) [2023-10-13 20:58:45,539][60934] Updated weights for policy 1, policy_version 2170 (0.0008) [2023-10-13 20:58:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 4456448. Throughput: 0: 1659.4, 1: 1666.1. Samples: 1112932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:58:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:58:49,353][60935] Updated weights for policy 0, policy_version 2180 (0.0010) [2023-10-13 20:58:49,634][60934] Updated weights for policy 1, policy_version 2180 (0.0010) [2023-10-13 20:58:49,740][60935] Updated weights for policy 0, policy_version 2190 (0.0008) [2023-10-13 20:58:50,009][60934] Updated weights for policy 1, policy_version 2190 (0.0007) [2023-10-13 20:58:50,107][60935] Updated weights for policy 0, policy_version 2200 (0.0007) [2023-10-13 20:58:50,379][60934] Updated weights for policy 1, policy_version 2200 (0.0007) [2023-10-13 20:58:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 4521984. Throughput: 0: 1648.4, 1: 1664.3. Samples: 1132706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:58:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:58:54,270][60935] Updated weights for policy 0, policy_version 2210 (0.0009) [2023-10-13 20:58:54,461][60934] Updated weights for policy 1, policy_version 2210 (0.0007) [2023-10-13 20:58:54,651][60935] Updated weights for policy 0, policy_version 2220 (0.0009) [2023-10-13 20:58:54,838][60934] Updated weights for policy 1, policy_version 2220 (0.0007) [2023-10-13 20:58:55,009][60935] Updated weights for policy 0, policy_version 2230 (0.0010) [2023-10-13 20:58:55,196][60934] Updated weights for policy 1, policy_version 2230 (0.0009) [2023-10-13 20:58:55,383][60935] Updated weights for policy 0, policy_version 2240 (0.0009) [2023-10-13 20:58:55,566][60934] Updated weights for policy 1, policy_version 2240 (0.0009) [2023-10-13 20:58:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 4587520. Throughput: 0: 1659.4, 1: 1652.5. Samples: 1151436. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-13 20:58:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:58:56,258][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000002240_2293760.pth... [2023-10-13 20:58:56,259][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000002240_2293760.pth... [2023-10-13 20:58:56,288][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000000672_688128.pth [2023-10-13 20:58:56,297][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000000672_688128.pth [2023-10-13 20:58:59,521][60935] Updated weights for policy 0, policy_version 2250 (0.0008) [2023-10-13 20:58:59,754][60934] Updated weights for policy 1, policy_version 2250 (0.0007) [2023-10-13 20:58:59,901][60935] Updated weights for policy 0, policy_version 2260 (0.0008) [2023-10-13 20:59:00,129][60934] Updated weights for policy 1, policy_version 2260 (0.0008) [2023-10-13 20:59:00,267][60935] Updated weights for policy 0, policy_version 2270 (0.0009) [2023-10-13 20:59:00,489][60934] Updated weights for policy 1, policy_version 2270 (0.0009) [2023-10-13 20:59:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 4653056. Throughput: 0: 1668.6, 1: 1666.5. Samples: 1162948. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 20:59:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:59:04,337][60935] Updated weights for policy 0, policy_version 2280 (0.0008) [2023-10-13 20:59:04,682][60934] Updated weights for policy 1, policy_version 2280 (0.0010) [2023-10-13 20:59:04,704][60935] Updated weights for policy 0, policy_version 2290 (0.0010) [2023-10-13 20:59:05,049][60934] Updated weights for policy 1, policy_version 2290 (0.0008) [2023-10-13 20:59:05,069][60935] Updated weights for policy 0, policy_version 2300 (0.0007) [2023-10-13 20:59:05,407][60934] Updated weights for policy 1, policy_version 2300 (0.0008) [2023-10-13 20:59:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 4718592. Throughput: 0: 1656.3, 1: 1662.5. Samples: 1182370. Policy #0 lag: (min: 9.0, avg: 18.3, max: 41.0) [2023-10-13 20:59:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:59:09,318][60935] Updated weights for policy 0, policy_version 2310 (0.0008) [2023-10-13 20:59:09,473][60934] Updated weights for policy 1, policy_version 2310 (0.0008) [2023-10-13 20:59:09,688][60935] Updated weights for policy 0, policy_version 2320 (0.0007) [2023-10-13 20:59:09,851][60934] Updated weights for policy 1, policy_version 2320 (0.0009) [2023-10-13 20:59:10,063][60935] Updated weights for policy 0, policy_version 2330 (0.0008) [2023-10-13 20:59:10,222][60934] Updated weights for policy 1, policy_version 2330 (0.0009) [2023-10-13 20:59:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 4784128. Throughput: 0: 1666.5, 1: 1651.6. Samples: 1201192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:59:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:59:14,319][60935] Updated weights for policy 0, policy_version 2340 (0.0009) [2023-10-13 20:59:14,406][60934] Updated weights for policy 1, policy_version 2340 (0.0009) [2023-10-13 20:59:14,684][60935] Updated weights for policy 0, policy_version 2350 (0.0009) [2023-10-13 20:59:14,782][60934] Updated weights for policy 1, policy_version 2350 (0.0009) [2023-10-13 20:59:15,065][60935] Updated weights for policy 0, policy_version 2360 (0.0008) [2023-10-13 20:59:15,144][60934] Updated weights for policy 1, policy_version 2360 (0.0010) [2023-10-13 20:59:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 4849664. Throughput: 0: 1662.9, 1: 1668.3. Samples: 1212696. Policy #0 lag: (min: 14.0, avg: 14.9, max: 34.0) [2023-10-13 20:59:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:59:19,129][60935] Updated weights for policy 0, policy_version 2370 (0.0009) [2023-10-13 20:59:19,259][60934] Updated weights for policy 1, policy_version 2370 (0.0008) [2023-10-13 20:59:19,506][60935] Updated weights for policy 0, policy_version 2380 (0.0007) [2023-10-13 20:59:19,631][60934] Updated weights for policy 1, policy_version 2380 (0.0007) [2023-10-13 20:59:19,880][60935] Updated weights for policy 0, policy_version 2390 (0.0007) [2023-10-13 20:59:19,995][60934] Updated weights for policy 1, policy_version 2390 (0.0007) [2023-10-13 20:59:20,243][60935] Updated weights for policy 0, policy_version 2400 (0.0007) [2023-10-13 20:59:20,364][60934] Updated weights for policy 1, policy_version 2400 (0.0009) [2023-10-13 20:59:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4915200. Throughput: 0: 1652.1, 1: 1658.5. Samples: 1232084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:59:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:59:24,206][60935] Updated weights for policy 0, policy_version 2410 (0.0009) [2023-10-13 20:59:24,424][60934] Updated weights for policy 1, policy_version 2410 (0.0007) [2023-10-13 20:59:24,577][60935] Updated weights for policy 0, policy_version 2420 (0.0009) [2023-10-13 20:59:24,785][60934] Updated weights for policy 1, policy_version 2420 (0.0008) [2023-10-13 20:59:24,944][60935] Updated weights for policy 0, policy_version 2430 (0.0009) [2023-10-13 20:59:25,157][60934] Updated weights for policy 1, policy_version 2430 (0.0009) [2023-10-13 20:59:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 4980736. Throughput: 0: 1670.2, 1: 1658.5. Samples: 1251508. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-13 20:59:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:59:28,885][60935] Updated weights for policy 0, policy_version 2440 (0.0008) [2023-10-13 20:59:29,251][60935] Updated weights for policy 0, policy_version 2450 (0.0009) [2023-10-13 20:59:29,419][60934] Updated weights for policy 1, policy_version 2440 (0.0008) [2023-10-13 20:59:29,620][60935] Updated weights for policy 0, policy_version 2460 (0.0008) [2023-10-13 20:59:29,789][60934] Updated weights for policy 1, policy_version 2450 (0.0009) [2023-10-13 20:59:30,148][60934] Updated weights for policy 1, policy_version 2460 (0.0007) [2023-10-13 20:59:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5046272. Throughput: 0: 1666.4, 1: 1662.9. Samples: 1262748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:59:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:59:33,779][60935] Updated weights for policy 0, policy_version 2470 (0.0008) [2023-10-13 20:59:34,152][60935] Updated weights for policy 0, policy_version 2480 (0.0008) [2023-10-13 20:59:34,356][60934] Updated weights for policy 1, policy_version 2470 (0.0007) [2023-10-13 20:59:34,525][60935] Updated weights for policy 0, policy_version 2490 (0.0008) [2023-10-13 20:59:34,747][60934] Updated weights for policy 1, policy_version 2480 (0.0009) [2023-10-13 20:59:35,123][60934] Updated weights for policy 1, policy_version 2490 (0.0009) [2023-10-13 20:59:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5111808. Throughput: 0: 1661.0, 1: 1649.6. Samples: 1281680. Policy #0 lag: (min: 21.0, avg: 23.9, max: 53.0) [2023-10-13 20:59:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:59:38,685][60935] Updated weights for policy 0, policy_version 2500 (0.0009) [2023-10-13 20:59:39,075][60935] Updated weights for policy 0, policy_version 2510 (0.0008) [2023-10-13 20:59:39,216][60934] Updated weights for policy 1, policy_version 2500 (0.0008) [2023-10-13 20:59:39,449][60935] Updated weights for policy 0, policy_version 2520 (0.0008) [2023-10-13 20:59:39,583][60934] Updated weights for policy 1, policy_version 2510 (0.0008) [2023-10-13 20:59:39,965][60934] Updated weights for policy 1, policy_version 2520 (0.0009) [2023-10-13 20:59:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5177344. Throughput: 0: 1671.7, 1: 1653.0. Samples: 1301046. Policy #0 lag: (min: 18.0, avg: 22.7, max: 50.0) [2023-10-13 20:59:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:59:43,525][60935] Updated weights for policy 0, policy_version 2530 (0.0008) [2023-10-13 20:59:43,903][60935] Updated weights for policy 0, policy_version 2540 (0.0008) [2023-10-13 20:59:44,114][60934] Updated weights for policy 1, policy_version 2530 (0.0009) [2023-10-13 20:59:44,270][60935] Updated weights for policy 0, policy_version 2550 (0.0007) [2023-10-13 20:59:44,479][60934] Updated weights for policy 1, policy_version 2540 (0.0009) [2023-10-13 20:59:44,641][60935] Updated weights for policy 0, policy_version 2560 (0.0008) [2023-10-13 20:59:44,857][60934] Updated weights for policy 1, policy_version 2550 (0.0008) [2023-10-13 20:59:45,226][60934] Updated weights for policy 1, policy_version 2560 (0.0009) [2023-10-13 20:59:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5242880. Throughput: 0: 1659.4, 1: 1652.9. Samples: 1312000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:59:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:59:48,733][60935] Updated weights for policy 0, policy_version 2570 (0.0010) [2023-10-13 20:59:49,105][60935] Updated weights for policy 0, policy_version 2580 (0.0008) [2023-10-13 20:59:49,376][60934] Updated weights for policy 1, policy_version 2570 (0.0009) [2023-10-13 20:59:49,477][60935] Updated weights for policy 0, policy_version 2590 (0.0007) [2023-10-13 20:59:49,743][60934] Updated weights for policy 1, policy_version 2580 (0.0008) [2023-10-13 20:59:50,108][60934] Updated weights for policy 1, policy_version 2590 (0.0007) [2023-10-13 20:59:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5308416. Throughput: 0: 1651.6, 1: 1646.5. Samples: 1330786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 20:59:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:59:53,657][60935] Updated weights for policy 0, policy_version 2600 (0.0009) [2023-10-13 20:59:54,028][60935] Updated weights for policy 0, policy_version 2610 (0.0008) [2023-10-13 20:59:54,270][60934] Updated weights for policy 1, policy_version 2600 (0.0007) [2023-10-13 20:59:54,400][60935] Updated weights for policy 0, policy_version 2620 (0.0009) [2023-10-13 20:59:54,633][60934] Updated weights for policy 1, policy_version 2610 (0.0009) [2023-10-13 20:59:55,000][60934] Updated weights for policy 1, policy_version 2620 (0.0010) [2023-10-13 20:59:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 5373952. Throughput: 0: 1665.4, 1: 1656.5. Samples: 1350678. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 20:59:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 20:59:58,468][60935] Updated weights for policy 0, policy_version 2630 (0.0007) [2023-10-13 20:59:58,829][60935] Updated weights for policy 0, policy_version 2640 (0.0008) [2023-10-13 20:59:58,999][60934] Updated weights for policy 1, policy_version 2630 (0.0008) [2023-10-13 20:59:59,198][60935] Updated weights for policy 0, policy_version 2650 (0.0011) [2023-10-13 20:59:59,364][60934] Updated weights for policy 1, policy_version 2640 (0.0007) [2023-10-13 20:59:59,733][60934] Updated weights for policy 1, policy_version 2650 (0.0009) [2023-10-13 21:00:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 5439488. Throughput: 0: 1656.8, 1: 1653.5. Samples: 1361660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:00:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:00:03,308][60935] Updated weights for policy 0, policy_version 2660 (0.0009) [2023-10-13 21:00:03,683][60935] Updated weights for policy 0, policy_version 2670 (0.0008) [2023-10-13 21:00:03,906][60934] Updated weights for policy 1, policy_version 2660 (0.0009) [2023-10-13 21:00:04,044][60935] Updated weights for policy 0, policy_version 2680 (0.0008) [2023-10-13 21:00:04,276][60934] Updated weights for policy 1, policy_version 2670 (0.0007) [2023-10-13 21:00:04,644][60934] Updated weights for policy 1, policy_version 2680 (0.0007) [2023-10-13 21:00:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5505024. Throughput: 0: 1657.6, 1: 1646.6. Samples: 1380772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:00:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:00:08,362][60935] Updated weights for policy 0, policy_version 2690 (0.0009) [2023-10-13 21:00:08,645][60934] Updated weights for policy 1, policy_version 2690 (0.0007) [2023-10-13 21:00:08,735][60935] Updated weights for policy 0, policy_version 2700 (0.0009) [2023-10-13 21:00:09,006][60934] Updated weights for policy 1, policy_version 2700 (0.0007) [2023-10-13 21:00:09,116][60935] Updated weights for policy 0, policy_version 2710 (0.0007) [2023-10-13 21:00:09,375][60934] Updated weights for policy 1, policy_version 2710 (0.0009) [2023-10-13 21:00:09,483][60935] Updated weights for policy 0, policy_version 2720 (0.0007) [2023-10-13 21:00:09,743][60934] Updated weights for policy 1, policy_version 2720 (0.0011) [2023-10-13 21:00:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5570560. Throughput: 0: 1655.6, 1: 1661.9. Samples: 1400794. Policy #0 lag: (min: 14.0, avg: 15.9, max: 45.0) [2023-10-13 21:00:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:00:13,589][60935] Updated weights for policy 0, policy_version 2730 (0.0012) [2023-10-13 21:00:13,963][60935] Updated weights for policy 0, policy_version 2740 (0.0009) [2023-10-13 21:00:13,981][60934] Updated weights for policy 1, policy_version 2730 (0.0008) [2023-10-13 21:00:14,344][60935] Updated weights for policy 0, policy_version 2750 (0.0008) [2023-10-13 21:00:14,353][60934] Updated weights for policy 1, policy_version 2740 (0.0008) [2023-10-13 21:00:14,718][60934] Updated weights for policy 1, policy_version 2750 (0.0009) [2023-10-13 21:00:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 5636096. Throughput: 0: 1647.2, 1: 1661.9. Samples: 1411658. Policy #0 lag: (min: 15.0, avg: 19.0, max: 47.0) [2023-10-13 21:00:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:00:18,532][60935] Updated weights for policy 0, policy_version 2760 (0.0010) [2023-10-13 21:00:18,896][60934] Updated weights for policy 1, policy_version 2760 (0.0009) [2023-10-13 21:00:18,900][60935] Updated weights for policy 0, policy_version 2770 (0.0009) [2023-10-13 21:00:19,263][60934] Updated weights for policy 1, policy_version 2770 (0.0009) [2023-10-13 21:00:19,283][60935] Updated weights for policy 0, policy_version 2780 (0.0009) [2023-10-13 21:00:19,634][60934] Updated weights for policy 1, policy_version 2780 (0.0009) [2023-10-13 21:00:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5701632. Throughput: 0: 1651.5, 1: 1654.0. Samples: 1430428. Policy #0 lag: (min: 25.0, avg: 39.5, max: 57.0) [2023-10-13 21:00:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:00:23,492][60935] Updated weights for policy 0, policy_version 2790 (0.0009) [2023-10-13 21:00:23,862][60935] Updated weights for policy 0, policy_version 2800 (0.0008) [2023-10-13 21:00:23,949][60934] Updated weights for policy 1, policy_version 2790 (0.0008) [2023-10-13 21:00:24,250][60935] Updated weights for policy 0, policy_version 2810 (0.0007) [2023-10-13 21:00:24,335][60934] Updated weights for policy 1, policy_version 2800 (0.0009) [2023-10-13 21:00:24,699][60934] Updated weights for policy 1, policy_version 2810 (0.0009) [2023-10-13 21:00:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5767168. Throughput: 0: 1651.1, 1: 1663.4. Samples: 1450198. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 21:00:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:00:28,435][60935] Updated weights for policy 0, policy_version 2820 (0.0007) [2023-10-13 21:00:28,788][60934] Updated weights for policy 1, policy_version 2820 (0.0008) [2023-10-13 21:00:28,806][60935] Updated weights for policy 0, policy_version 2830 (0.0008) [2023-10-13 21:00:29,156][60934] Updated weights for policy 1, policy_version 2830 (0.0008) [2023-10-13 21:00:29,166][60935] Updated weights for policy 0, policy_version 2840 (0.0009) [2023-10-13 21:00:29,535][60934] Updated weights for policy 1, policy_version 2840 (0.0009) [2023-10-13 21:00:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5832704. Throughput: 0: 1648.0, 1: 1666.3. Samples: 1461142. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-13 21:00:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:00:33,384][60935] Updated weights for policy 0, policy_version 2850 (0.0008) [2023-10-13 21:00:33,739][60934] Updated weights for policy 1, policy_version 2850 (0.0010) [2023-10-13 21:00:33,768][60935] Updated weights for policy 0, policy_version 2860 (0.0008) [2023-10-13 21:00:34,100][60934] Updated weights for policy 1, policy_version 2860 (0.0008) [2023-10-13 21:00:34,132][60935] Updated weights for policy 0, policy_version 2870 (0.0008) [2023-10-13 21:00:34,477][60934] Updated weights for policy 1, policy_version 2870 (0.0009) [2023-10-13 21:00:34,499][60935] Updated weights for policy 0, policy_version 2880 (0.0008) [2023-10-13 21:00:34,843][60934] Updated weights for policy 1, policy_version 2880 (0.0008) [2023-10-13 21:00:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5898240. Throughput: 0: 1650.2, 1: 1656.8. Samples: 1479604. Policy #0 lag: (min: 44.0, avg: 47.9, max: 48.0) [2023-10-13 21:00:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:00:38,720][60935] Updated weights for policy 0, policy_version 2890 (0.0008) [2023-10-13 21:00:38,982][60934] Updated weights for policy 1, policy_version 2890 (0.0008) [2023-10-13 21:00:39,102][60935] Updated weights for policy 0, policy_version 2900 (0.0009) [2023-10-13 21:00:39,339][60934] Updated weights for policy 1, policy_version 2900 (0.0007) [2023-10-13 21:00:39,475][60935] Updated weights for policy 0, policy_version 2910 (0.0009) [2023-10-13 21:00:39,702][60934] Updated weights for policy 1, policy_version 2910 (0.0007) [2023-10-13 21:00:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 5963776. Throughput: 0: 1643.1, 1: 1662.5. Samples: 1499430. Policy #0 lag: (min: 15.0, avg: 21.7, max: 47.0) [2023-10-13 21:00:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:00:43,571][60934] Updated weights for policy 1, policy_version 2920 (0.0008) [2023-10-13 21:00:43,634][60935] Updated weights for policy 0, policy_version 2920 (0.0008) [2023-10-13 21:00:43,944][60934] Updated weights for policy 1, policy_version 2930 (0.0008) [2023-10-13 21:00:44,003][60935] Updated weights for policy 0, policy_version 2930 (0.0007) [2023-10-13 21:00:44,304][60934] Updated weights for policy 1, policy_version 2940 (0.0008) [2023-10-13 21:00:44,374][60935] Updated weights for policy 0, policy_version 2940 (0.0008) [2023-10-13 21:00:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 6029312. Throughput: 0: 1642.0, 1: 1659.7. Samples: 1510238. Policy #0 lag: (min: 3.0, avg: 5.1, max: 35.0) [2023-10-13 21:00:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:00:48,517][60934] Updated weights for policy 1, policy_version 2950 (0.0008) [2023-10-13 21:00:48,612][60935] Updated weights for policy 0, policy_version 2950 (0.0009) [2023-10-13 21:00:48,891][60934] Updated weights for policy 1, policy_version 2960 (0.0008) [2023-10-13 21:00:48,989][60935] Updated weights for policy 0, policy_version 2960 (0.0007) [2023-10-13 21:00:49,247][60934] Updated weights for policy 1, policy_version 2970 (0.0007) [2023-10-13 21:00:49,365][60935] Updated weights for policy 0, policy_version 2970 (0.0008) [2023-10-13 21:00:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6094848. Throughput: 0: 1640.4, 1: 1647.1. Samples: 1528710. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-13 21:00:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:00:53,344][60934] Updated weights for policy 1, policy_version 2980 (0.0009) [2023-10-13 21:00:53,418][60935] Updated weights for policy 0, policy_version 2980 (0.0009) [2023-10-13 21:00:53,720][60934] Updated weights for policy 1, policy_version 2990 (0.0007) [2023-10-13 21:00:53,785][60935] Updated weights for policy 0, policy_version 2990 (0.0008) [2023-10-13 21:00:54,086][60934] Updated weights for policy 1, policy_version 3000 (0.0008) [2023-10-13 21:00:54,160][60935] Updated weights for policy 0, policy_version 3000 (0.0008) [2023-10-13 21:00:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 6160384. Throughput: 0: 1642.7, 1: 1653.8. Samples: 1549136. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-13 21:00:56,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:00:56,262][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000003008_3080192.pth... [2023-10-13 21:00:56,263][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000003008_3080192.pth... [2023-10-13 21:00:56,298][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000001472_1507328.pth [2023-10-13 21:00:56,299][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000001472_1507328.pth [2023-10-13 21:00:58,127][60934] Updated weights for policy 1, policy_version 3010 (0.0008) [2023-10-13 21:00:58,284][60935] Updated weights for policy 0, policy_version 3010 (0.0008) [2023-10-13 21:00:58,493][60934] Updated weights for policy 1, policy_version 3020 (0.0007) [2023-10-13 21:00:58,651][60935] Updated weights for policy 0, policy_version 3020 (0.0008) [2023-10-13 21:00:58,854][60934] Updated weights for policy 1, policy_version 3030 (0.0007) [2023-10-13 21:00:59,023][60935] Updated weights for policy 0, policy_version 3030 (0.0008) [2023-10-13 21:00:59,219][60934] Updated weights for policy 1, policy_version 3040 (0.0007) [2023-10-13 21:00:59,389][60935] Updated weights for policy 0, policy_version 3040 (0.0008) [2023-10-13 21:01:01,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6225920. Throughput: 0: 1637.7, 1: 1645.5. Samples: 1559404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:01:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:01:03,392][60934] Updated weights for policy 1, policy_version 3050 (0.0007) [2023-10-13 21:01:03,566][60935] Updated weights for policy 0, policy_version 3050 (0.0007) [2023-10-13 21:01:03,758][60934] Updated weights for policy 1, policy_version 3060 (0.0008) [2023-10-13 21:01:03,929][60935] Updated weights for policy 0, policy_version 3060 (0.0009) [2023-10-13 21:01:04,132][60934] Updated weights for policy 1, policy_version 3070 (0.0007) [2023-10-13 21:01:04,298][60935] Updated weights for policy 0, policy_version 3070 (0.0009) [2023-10-13 21:01:06,248][59943] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6291456. Throughput: 0: 1638.3, 1: 1648.6. Samples: 1578338. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-13 21:01:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:01:08,468][60934] Updated weights for policy 1, policy_version 3080 (0.0010) [2023-10-13 21:01:08,551][60935] Updated weights for policy 0, policy_version 3080 (0.0008) [2023-10-13 21:01:08,843][60934] Updated weights for policy 1, policy_version 3090 (0.0008) [2023-10-13 21:01:08,921][60935] Updated weights for policy 0, policy_version 3090 (0.0008) [2023-10-13 21:01:09,210][60934] Updated weights for policy 1, policy_version 3100 (0.0007) [2023-10-13 21:01:09,302][60935] Updated weights for policy 0, policy_version 3100 (0.0008) [2023-10-13 21:01:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6356992. Throughput: 0: 1641.4, 1: 1655.7. Samples: 1598566. Policy #0 lag: (min: 3.0, avg: 18.3, max: 35.0) [2023-10-13 21:01:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:01:13,276][60934] Updated weights for policy 1, policy_version 3110 (0.0007) [2023-10-13 21:01:13,358][60935] Updated weights for policy 0, policy_version 3110 (0.0009) [2023-10-13 21:01:13,647][60934] Updated weights for policy 1, policy_version 3120 (0.0010) [2023-10-13 21:01:13,728][60935] Updated weights for policy 0, policy_version 3120 (0.0007) [2023-10-13 21:01:14,014][60934] Updated weights for policy 1, policy_version 3130 (0.0008) [2023-10-13 21:01:14,105][60935] Updated weights for policy 0, policy_version 3130 (0.0007) [2023-10-13 21:01:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6422528. Throughput: 0: 1637.2, 1: 1646.6. Samples: 1608912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:01:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:01:18,032][60934] Updated weights for policy 1, policy_version 3140 (0.0009) [2023-10-13 21:01:18,400][60934] Updated weights for policy 1, policy_version 3150 (0.0008) [2023-10-13 21:01:18,442][60935] Updated weights for policy 0, policy_version 3140 (0.0008) [2023-10-13 21:01:18,774][60934] Updated weights for policy 1, policy_version 3160 (0.0007) [2023-10-13 21:01:18,817][60935] Updated weights for policy 0, policy_version 3150 (0.0010) [2023-10-13 21:01:19,182][60935] Updated weights for policy 0, policy_version 3160 (0.0009) [2023-10-13 21:01:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6488064. Throughput: 0: 1640.3, 1: 1655.1. Samples: 1627898. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-13 21:01:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:01:22,914][60934] Updated weights for policy 1, policy_version 3170 (0.0009) [2023-10-13 21:01:23,289][60934] Updated weights for policy 1, policy_version 3180 (0.0009) [2023-10-13 21:01:23,360][60935] Updated weights for policy 0, policy_version 3170 (0.0008) [2023-10-13 21:01:23,656][60934] Updated weights for policy 1, policy_version 3190 (0.0009) [2023-10-13 21:01:23,733][60935] Updated weights for policy 0, policy_version 3180 (0.0007) [2023-10-13 21:01:24,021][60934] Updated weights for policy 1, policy_version 3200 (0.0007) [2023-10-13 21:01:24,101][60935] Updated weights for policy 0, policy_version 3190 (0.0009) [2023-10-13 21:01:24,471][60935] Updated weights for policy 0, policy_version 3200 (0.0008) [2023-10-13 21:01:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 6553600. Throughput: 0: 1639.2, 1: 1669.2. Samples: 1648308. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-13 21:01:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:01:28,020][60934] Updated weights for policy 1, policy_version 3210 (0.0009) [2023-10-13 21:01:28,388][60934] Updated weights for policy 1, policy_version 3220 (0.0007) [2023-10-13 21:01:28,647][60935] Updated weights for policy 0, policy_version 3210 (0.0009) [2023-10-13 21:01:28,759][60934] Updated weights for policy 1, policy_version 3230 (0.0007) [2023-10-13 21:01:29,018][60935] Updated weights for policy 0, policy_version 3220 (0.0010) [2023-10-13 21:01:29,397][60935] Updated weights for policy 0, policy_version 3230 (0.0011) [2023-10-13 21:01:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6619136. Throughput: 0: 1638.8, 1: 1652.2. Samples: 1658330. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-13 21:01:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:01:33,065][60934] Updated weights for policy 1, policy_version 3240 (0.0007) [2023-10-13 21:01:33,429][60934] Updated weights for policy 1, policy_version 3250 (0.0007) [2023-10-13 21:01:33,498][60935] Updated weights for policy 0, policy_version 3240 (0.0008) [2023-10-13 21:01:33,793][60934] Updated weights for policy 1, policy_version 3260 (0.0007) [2023-10-13 21:01:33,870][60935] Updated weights for policy 0, policy_version 3250 (0.0009) [2023-10-13 21:01:34,247][60935] Updated weights for policy 0, policy_version 3260 (0.0011) [2023-10-13 21:01:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6684672. Throughput: 0: 1638.0, 1: 1672.4. Samples: 1677682. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-13 21:01:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:01:37,851][60934] Updated weights for policy 1, policy_version 3270 (0.0008) [2023-10-13 21:01:38,217][60934] Updated weights for policy 1, policy_version 3280 (0.0009) [2023-10-13 21:01:38,318][60935] Updated weights for policy 0, policy_version 3270 (0.0010) [2023-10-13 21:01:38,575][60934] Updated weights for policy 1, policy_version 3290 (0.0008) [2023-10-13 21:01:38,690][60935] Updated weights for policy 0, policy_version 3280 (0.0009) [2023-10-13 21:01:39,059][60935] Updated weights for policy 0, policy_version 3290 (0.0009) [2023-10-13 21:01:41,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6750208. Throughput: 0: 1640.1, 1: 1669.3. Samples: 1698060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:01:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:01:42,727][60934] Updated weights for policy 1, policy_version 3300 (0.0008) [2023-10-13 21:01:43,099][60934] Updated weights for policy 1, policy_version 3310 (0.0009) [2023-10-13 21:01:43,356][60935] Updated weights for policy 0, policy_version 3300 (0.0009) [2023-10-13 21:01:43,472][60934] Updated weights for policy 1, policy_version 3320 (0.0007) [2023-10-13 21:01:43,730][60935] Updated weights for policy 0, policy_version 3310 (0.0009) [2023-10-13 21:01:44,109][60935] Updated weights for policy 0, policy_version 3320 (0.0009) [2023-10-13 21:01:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6815744. Throughput: 0: 1639.7, 1: 1658.9. Samples: 1707844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:01:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:01:47,532][60934] Updated weights for policy 1, policy_version 3330 (0.0008) [2023-10-13 21:01:47,909][60934] Updated weights for policy 1, policy_version 3340 (0.0010) [2023-10-13 21:01:48,266][60934] Updated weights for policy 1, policy_version 3350 (0.0009) [2023-10-13 21:01:48,402][60935] Updated weights for policy 0, policy_version 3330 (0.0010) [2023-10-13 21:01:48,636][60934] Updated weights for policy 1, policy_version 3360 (0.0009) [2023-10-13 21:01:48,767][60935] Updated weights for policy 0, policy_version 3340 (0.0009) [2023-10-13 21:01:49,129][60935] Updated weights for policy 0, policy_version 3350 (0.0010) [2023-10-13 21:01:49,501][60935] Updated weights for policy 0, policy_version 3360 (0.0010) [2023-10-13 21:01:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6881280. Throughput: 0: 1635.7, 1: 1668.0. Samples: 1727004. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-13 21:01:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:01:52,587][60934] Updated weights for policy 1, policy_version 3370 (0.0009) [2023-10-13 21:01:52,958][60934] Updated weights for policy 1, policy_version 3380 (0.0008) [2023-10-13 21:01:53,323][60934] Updated weights for policy 1, policy_version 3390 (0.0007) [2023-10-13 21:01:53,481][60935] Updated weights for policy 0, policy_version 3370 (0.0009) [2023-10-13 21:01:53,851][60935] Updated weights for policy 0, policy_version 3380 (0.0011) [2023-10-13 21:01:54,216][60935] Updated weights for policy 0, policy_version 3390 (0.0011) [2023-10-13 21:01:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 6946816. Throughput: 0: 1640.0, 1: 1675.2. Samples: 1747752. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-13 21:01:56,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 21:01:57,459][60934] Updated weights for policy 1, policy_version 3400 (0.0007) [2023-10-13 21:01:57,841][60934] Updated weights for policy 1, policy_version 3410 (0.0007) [2023-10-13 21:01:58,207][60934] Updated weights for policy 1, policy_version 3420 (0.0008) [2023-10-13 21:01:58,389][60935] Updated weights for policy 0, policy_version 3400 (0.0009) [2023-10-13 21:01:58,767][60935] Updated weights for policy 0, policy_version 3410 (0.0010) [2023-10-13 21:01:59,143][60935] Updated weights for policy 0, policy_version 3420 (0.0007) [2023-10-13 21:02:01,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7012352. Throughput: 0: 1638.8, 1: 1654.9. Samples: 1757126. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-13 21:02:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:02:02,423][60934] Updated weights for policy 1, policy_version 3430 (0.0009) [2023-10-13 21:02:02,796][60934] Updated weights for policy 1, policy_version 3440 (0.0007) [2023-10-13 21:02:03,162][60934] Updated weights for policy 1, policy_version 3450 (0.0007) [2023-10-13 21:02:03,279][60935] Updated weights for policy 0, policy_version 3430 (0.0008) [2023-10-13 21:02:03,644][60935] Updated weights for policy 0, policy_version 3440 (0.0009) [2023-10-13 21:02:04,014][60935] Updated weights for policy 0, policy_version 3450 (0.0009) [2023-10-13 21:02:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7077888. Throughput: 0: 1645.2, 1: 1674.2. Samples: 1777268. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-13 21:02:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:02:07,279][60934] Updated weights for policy 1, policy_version 3460 (0.0007) [2023-10-13 21:02:07,643][60934] Updated weights for policy 1, policy_version 3470 (0.0007) [2023-10-13 21:02:08,012][60934] Updated weights for policy 1, policy_version 3480 (0.0007) [2023-10-13 21:02:08,152][60935] Updated weights for policy 0, policy_version 3460 (0.0008) [2023-10-13 21:02:08,522][60935] Updated weights for policy 0, policy_version 3470 (0.0008) [2023-10-13 21:02:08,890][60935] Updated weights for policy 0, policy_version 3480 (0.0009) [2023-10-13 21:02:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7143424. Throughput: 0: 1654.1, 1: 1674.0. Samples: 1798070. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 21:02:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:02:11,995][60934] Updated weights for policy 1, policy_version 3490 (0.0008) [2023-10-13 21:02:12,367][60934] Updated weights for policy 1, policy_version 3500 (0.0007) [2023-10-13 21:02:12,722][60934] Updated weights for policy 1, policy_version 3510 (0.0007) [2023-10-13 21:02:12,995][60935] Updated weights for policy 0, policy_version 3490 (0.0009) [2023-10-13 21:02:13,092][60934] Updated weights for policy 1, policy_version 3520 (0.0007) [2023-10-13 21:02:13,373][60935] Updated weights for policy 0, policy_version 3500 (0.0008) [2023-10-13 21:02:13,748][60935] Updated weights for policy 0, policy_version 3510 (0.0008) [2023-10-13 21:02:14,125][60935] Updated weights for policy 0, policy_version 3520 (0.0007) [2023-10-13 21:02:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 7208960. Throughput: 0: 1645.6, 1: 1666.2. Samples: 1807362. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-13 21:02:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:02:17,256][60934] Updated weights for policy 1, policy_version 3530 (0.0008) [2023-10-13 21:02:17,626][60934] Updated weights for policy 1, policy_version 3540 (0.0007) [2023-10-13 21:02:17,988][60934] Updated weights for policy 1, policy_version 3550 (0.0007) [2023-10-13 21:02:18,041][60935] Updated weights for policy 0, policy_version 3530 (0.0008) [2023-10-13 21:02:18,403][60935] Updated weights for policy 0, policy_version 3540 (0.0009) [2023-10-13 21:02:18,776][60935] Updated weights for policy 0, policy_version 3550 (0.0009) [2023-10-13 21:02:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 7274496. Throughput: 0: 1657.5, 1: 1676.3. Samples: 1827700. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-13 21:02:21,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 21:02:21,973][60934] Updated weights for policy 1, policy_version 3560 (0.0009) [2023-10-13 21:02:22,336][60934] Updated weights for policy 1, policy_version 3570 (0.0008) [2023-10-13 21:02:22,702][60934] Updated weights for policy 1, policy_version 3580 (0.0007) [2023-10-13 21:02:22,970][60935] Updated weights for policy 0, policy_version 3560 (0.0009) [2023-10-13 21:02:23,351][60935] Updated weights for policy 0, policy_version 3570 (0.0011) [2023-10-13 21:02:23,734][60935] Updated weights for policy 0, policy_version 3580 (0.0009) [2023-10-13 21:02:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 7340032. Throughput: 0: 1657.3, 1: 1679.4. Samples: 1848212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:02:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:02:26,664][60934] Updated weights for policy 1, policy_version 3590 (0.0008) [2023-10-13 21:02:27,033][60934] Updated weights for policy 1, policy_version 3600 (0.0008) [2023-10-13 21:02:27,390][60934] Updated weights for policy 1, policy_version 3610 (0.0010) [2023-10-13 21:02:27,942][60935] Updated weights for policy 0, policy_version 3590 (0.0010) [2023-10-13 21:02:28,307][60935] Updated weights for policy 0, policy_version 3600 (0.0009) [2023-10-13 21:02:28,680][60935] Updated weights for policy 0, policy_version 3610 (0.0010) [2023-10-13 21:02:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 7405568. Throughput: 0: 1650.1, 1: 1672.9. Samples: 1857380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:02:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:02:31,565][60934] Updated weights for policy 1, policy_version 3620 (0.0008) [2023-10-13 21:02:31,935][60934] Updated weights for policy 1, policy_version 3630 (0.0008) [2023-10-13 21:02:32,305][60934] Updated weights for policy 1, policy_version 3640 (0.0008) [2023-10-13 21:02:32,718][60935] Updated weights for policy 0, policy_version 3620 (0.0008) [2023-10-13 21:02:33,088][60935] Updated weights for policy 0, policy_version 3630 (0.0009) [2023-10-13 21:02:33,458][60935] Updated weights for policy 0, policy_version 3640 (0.0008) [2023-10-13 21:02:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 7471104. Throughput: 0: 1665.1, 1: 1687.4. Samples: 1877866. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:02:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:02:36,290][60934] Updated weights for policy 1, policy_version 3650 (0.0007) [2023-10-13 21:02:36,662][60934] Updated weights for policy 1, policy_version 3660 (0.0009) [2023-10-13 21:02:37,023][60934] Updated weights for policy 1, policy_version 3670 (0.0009) [2023-10-13 21:02:37,397][60934] Updated weights for policy 1, policy_version 3680 (0.0010) [2023-10-13 21:02:37,658][60935] Updated weights for policy 0, policy_version 3650 (0.0009) [2023-10-13 21:02:38,023][60935] Updated weights for policy 0, policy_version 3660 (0.0012) [2023-10-13 21:02:38,388][60935] Updated weights for policy 0, policy_version 3670 (0.0008) [2023-10-13 21:02:38,765][60935] Updated weights for policy 0, policy_version 3680 (0.0007) [2023-10-13 21:02:41,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 7536640. Throughput: 0: 1664.8, 1: 1687.2. Samples: 1898590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:02:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:02:41,491][60934] Updated weights for policy 1, policy_version 3690 (0.0010) [2023-10-13 21:02:41,851][60934] Updated weights for policy 1, policy_version 3700 (0.0009) [2023-10-13 21:02:42,223][60934] Updated weights for policy 1, policy_version 3710 (0.0010) [2023-10-13 21:02:42,854][60935] Updated weights for policy 0, policy_version 3690 (0.0010) [2023-10-13 21:02:43,229][60935] Updated weights for policy 0, policy_version 3700 (0.0009) [2023-10-13 21:02:43,596][60935] Updated weights for policy 0, policy_version 3710 (0.0008) [2023-10-13 21:02:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 7602176. Throughput: 0: 1651.2, 1: 1691.4. Samples: 1907542. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:02:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:02:46,511][60934] Updated weights for policy 1, policy_version 3720 (0.0008) [2023-10-13 21:02:46,896][60934] Updated weights for policy 1, policy_version 3730 (0.0009) [2023-10-13 21:02:47,256][60934] Updated weights for policy 1, policy_version 3740 (0.0009) [2023-10-13 21:02:47,810][60935] Updated weights for policy 0, policy_version 3720 (0.0010) [2023-10-13 21:02:48,188][60935] Updated weights for policy 0, policy_version 3730 (0.0008) [2023-10-13 21:02:48,552][60935] Updated weights for policy 0, policy_version 3740 (0.0008) [2023-10-13 21:02:51,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 7667712. Throughput: 0: 1663.1, 1: 1676.0. Samples: 1927530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:02:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:02:51,420][60934] Updated weights for policy 1, policy_version 3750 (0.0007) [2023-10-13 21:02:51,796][60934] Updated weights for policy 1, policy_version 3760 (0.0009) [2023-10-13 21:02:52,166][60934] Updated weights for policy 1, policy_version 3770 (0.0008) [2023-10-13 21:02:52,527][60935] Updated weights for policy 0, policy_version 3750 (0.0007) [2023-10-13 21:02:52,895][60935] Updated weights for policy 0, policy_version 3760 (0.0008) [2023-10-13 21:02:53,285][60935] Updated weights for policy 0, policy_version 3770 (0.0011) [2023-10-13 21:02:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 7733248. Throughput: 0: 1661.4, 1: 1674.4. Samples: 1948182. Policy #0 lag: (min: 16.0, avg: 37.2, max: 48.0) [2023-10-13 21:02:56,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 21:02:56,260][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000003776_3866624.pth... [2023-10-13 21:02:56,294][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000002240_2293760.pth [2023-10-13 21:02:56,307][60934] Updated weights for policy 1, policy_version 3780 (0.0007) [2023-10-13 21:02:56,689][60934] Updated weights for policy 1, policy_version 3790 (0.0008) [2023-10-13 21:02:57,054][60934] Updated weights for policy 1, policy_version 3800 (0.0007) [2023-10-13 21:02:57,347][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000003808_3899392.pth... [2023-10-13 21:02:57,379][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000002240_2293760.pth [2023-10-13 21:02:57,402][60935] Updated weights for policy 0, policy_version 3780 (0.0009) [2023-10-13 21:02:57,766][60935] Updated weights for policy 0, policy_version 3790 (0.0009) [2023-10-13 21:02:58,140][60935] Updated weights for policy 0, policy_version 3800 (0.0008) [2023-10-13 21:03:00,970][60934] Updated weights for policy 1, policy_version 3810 (0.0008) [2023-10-13 21:03:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 7798784. Throughput: 0: 1657.3, 1: 1675.9. Samples: 1957356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:03:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:03:01,335][60934] Updated weights for policy 1, policy_version 3820 (0.0010) [2023-10-13 21:03:01,706][60934] Updated weights for policy 1, policy_version 3830 (0.0010) [2023-10-13 21:03:02,067][60935] Updated weights for policy 0, policy_version 3810 (0.0008) [2023-10-13 21:03:02,072][60934] Updated weights for policy 1, policy_version 3840 (0.0010) [2023-10-13 21:03:02,447][60935] Updated weights for policy 0, policy_version 3820 (0.0010) [2023-10-13 21:03:02,806][60935] Updated weights for policy 0, policy_version 3830 (0.0009) [2023-10-13 21:03:03,178][60935] Updated weights for policy 0, policy_version 3840 (0.0009) [2023-10-13 21:03:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 7864320. Throughput: 0: 1666.2, 1: 1673.3. Samples: 1977980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:03:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:03:06,264][60934] Updated weights for policy 1, policy_version 3850 (0.0008) [2023-10-13 21:03:06,631][60934] Updated weights for policy 1, policy_version 3860 (0.0008) [2023-10-13 21:03:06,997][60934] Updated weights for policy 1, policy_version 3870 (0.0009) [2023-10-13 21:03:07,311][60935] Updated weights for policy 0, policy_version 3850 (0.0008) [2023-10-13 21:03:07,691][60935] Updated weights for policy 0, policy_version 3860 (0.0009) [2023-10-13 21:03:08,064][60935] Updated weights for policy 0, policy_version 3870 (0.0009) [2023-10-13 21:03:11,067][60934] Updated weights for policy 1, policy_version 3880 (0.0009) [2023-10-13 21:03:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 7929856. Throughput: 0: 1662.4, 1: 1673.9. Samples: 1998346. Policy #0 lag: (min: 17.0, avg: 27.9, max: 49.0) [2023-10-13 21:03:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:03:11,438][60934] Updated weights for policy 1, policy_version 3890 (0.0007) [2023-10-13 21:03:11,809][60934] Updated weights for policy 1, policy_version 3900 (0.0008) [2023-10-13 21:03:12,329][60935] Updated weights for policy 0, policy_version 3880 (0.0010) [2023-10-13 21:03:12,699][60935] Updated weights for policy 0, policy_version 3890 (0.0010) [2023-10-13 21:03:13,081][60935] Updated weights for policy 0, policy_version 3900 (0.0009) [2023-10-13 21:03:15,715][60934] Updated weights for policy 1, policy_version 3910 (0.0008) [2023-10-13 21:03:16,081][60934] Updated weights for policy 1, policy_version 3920 (0.0011) [2023-10-13 21:03:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7995392. Throughput: 0: 1657.7, 1: 1675.2. Samples: 2007360. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-13 21:03:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:03:16,457][60934] Updated weights for policy 1, policy_version 3930 (0.0009) [2023-10-13 21:03:17,120][60935] Updated weights for policy 0, policy_version 3910 (0.0009) [2023-10-13 21:03:17,485][60935] Updated weights for policy 0, policy_version 3920 (0.0008) [2023-10-13 21:03:17,848][60935] Updated weights for policy 0, policy_version 3930 (0.0009) [2023-10-13 21:03:20,384][60934] Updated weights for policy 1, policy_version 3940 (0.0009) [2023-10-13 21:03:20,748][60934] Updated weights for policy 1, policy_version 3950 (0.0011) [2023-10-13 21:03:21,192][60934] Updated weights for policy 1, policy_version 3962 (0.0007) [2023-10-13 21:03:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 8060928. Throughput: 0: 1663.3, 1: 1676.3. Samples: 2028150. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-13 21:03:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:03:22,028][60935] Updated weights for policy 0, policy_version 3940 (0.0008) [2023-10-13 21:03:22,412][60935] Updated weights for policy 0, policy_version 3950 (0.0007) [2023-10-13 21:03:22,785][60935] Updated weights for policy 0, policy_version 3960 (0.0008) [2023-10-13 21:03:25,324][60934] Updated weights for policy 1, policy_version 3972 (0.0007) [2023-10-13 21:03:25,700][60934] Updated weights for policy 1, policy_version 3982 (0.0008) [2023-10-13 21:03:26,069][60934] Updated weights for policy 1, policy_version 3992 (0.0010) [2023-10-13 21:03:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 8126464. Throughput: 0: 1667.1, 1: 1668.9. Samples: 2048710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:03:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:03:26,882][60935] Updated weights for policy 0, policy_version 3970 (0.0008) [2023-10-13 21:03:27,292][60935] Updated weights for policy 0, policy_version 3980 (0.0009) [2023-10-13 21:03:27,660][60935] Updated weights for policy 0, policy_version 3990 (0.0009) [2023-10-13 21:03:28,039][60935] Updated weights for policy 0, policy_version 4000 (0.0010) [2023-10-13 21:03:30,129][60934] Updated weights for policy 1, policy_version 4002 (0.0007) [2023-10-13 21:03:30,494][60934] Updated weights for policy 1, policy_version 4012 (0.0008) [2023-10-13 21:03:30,856][60934] Updated weights for policy 1, policy_version 4022 (0.0009) [2023-10-13 21:03:31,227][60934] Updated weights for policy 1, policy_version 4032 (0.0008) [2023-10-13 21:03:31,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 8224768. Throughput: 0: 1667.7, 1: 1678.2. Samples: 2058108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:03:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:03:32,033][60935] Updated weights for policy 0, policy_version 4010 (0.0009) [2023-10-13 21:03:32,405][60935] Updated weights for policy 0, policy_version 4020 (0.0011) [2023-10-13 21:03:32,777][60935] Updated weights for policy 0, policy_version 4030 (0.0010) [2023-10-13 21:03:35,350][60934] Updated weights for policy 1, policy_version 4042 (0.0007) [2023-10-13 21:03:35,719][60934] Updated weights for policy 1, policy_version 4052 (0.0008) [2023-10-13 21:03:36,092][60934] Updated weights for policy 1, policy_version 4062 (0.0009) [2023-10-13 21:03:36,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 8290304. Throughput: 0: 1669.3, 1: 1691.5. Samples: 2078766. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-13 21:03:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:03:36,847][60935] Updated weights for policy 0, policy_version 4040 (0.0009) [2023-10-13 21:03:37,219][60935] Updated weights for policy 0, policy_version 4050 (0.0010) [2023-10-13 21:03:37,594][60935] Updated weights for policy 0, policy_version 4060 (0.0010) [2023-10-13 21:03:40,144][60934] Updated weights for policy 1, policy_version 4072 (0.0010) [2023-10-13 21:03:40,515][60934] Updated weights for policy 1, policy_version 4082 (0.0009) [2023-10-13 21:03:40,891][60934] Updated weights for policy 1, policy_version 4092 (0.0010) [2023-10-13 21:03:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 8355840. Throughput: 0: 1669.8, 1: 1671.1. Samples: 2098524. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-13 21:03:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:03:41,668][60935] Updated weights for policy 0, policy_version 4070 (0.0010) [2023-10-13 21:03:42,034][60935] Updated weights for policy 0, policy_version 4080 (0.0010) [2023-10-13 21:03:42,406][60935] Updated weights for policy 0, policy_version 4090 (0.0008) [2023-10-13 21:03:44,920][60934] Updated weights for policy 1, policy_version 4102 (0.0010) [2023-10-13 21:03:45,279][60934] Updated weights for policy 1, policy_version 4112 (0.0007) [2023-10-13 21:03:45,651][60934] Updated weights for policy 1, policy_version 4122 (0.0009) [2023-10-13 21:03:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 8421376. Throughput: 0: 1668.4, 1: 1683.8. Samples: 2108204. Policy #0 lag: (min: 2.0, avg: 9.5, max: 34.0) [2023-10-13 21:03:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:03:46,512][60935] Updated weights for policy 0, policy_version 4100 (0.0009) [2023-10-13 21:03:46,883][60935] Updated weights for policy 0, policy_version 4110 (0.0008) [2023-10-13 21:03:47,257][60935] Updated weights for policy 0, policy_version 4120 (0.0008) [2023-10-13 21:03:49,756][60934] Updated weights for policy 1, policy_version 4132 (0.0009) [2023-10-13 21:03:50,124][60934] Updated weights for policy 1, policy_version 4142 (0.0008) [2023-10-13 21:03:50,492][60934] Updated weights for policy 1, policy_version 4152 (0.0009) [2023-10-13 21:03:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 8486912. Throughput: 0: 1662.7, 1: 1687.1. Samples: 2128718. Policy #0 lag: (min: 7.0, avg: 12.7, max: 39.0) [2023-10-13 21:03:51,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 21:03:51,475][60935] Updated weights for policy 0, policy_version 4130 (0.0009) [2023-10-13 21:03:51,852][60935] Updated weights for policy 0, policy_version 4140 (0.0008) [2023-10-13 21:03:52,216][60935] Updated weights for policy 0, policy_version 4150 (0.0010) [2023-10-13 21:03:52,594][60935] Updated weights for policy 0, policy_version 4160 (0.0010) [2023-10-13 21:03:54,411][60934] Updated weights for policy 1, policy_version 4162 (0.0009) [2023-10-13 21:03:54,780][60934] Updated weights for policy 1, policy_version 4172 (0.0010) [2023-10-13 21:03:55,145][60934] Updated weights for policy 1, policy_version 4182 (0.0008) [2023-10-13 21:03:55,515][60934] Updated weights for policy 1, policy_version 4192 (0.0010) [2023-10-13 21:03:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 8552448. Throughput: 0: 1671.1, 1: 1661.9. Samples: 2148332. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-13 21:03:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:03:56,762][60935] Updated weights for policy 0, policy_version 4170 (0.0009) [2023-10-13 21:03:57,126][60935] Updated weights for policy 0, policy_version 4180 (0.0011) [2023-10-13 21:03:57,500][60935] Updated weights for policy 0, policy_version 4190 (0.0008) [2023-10-13 21:03:59,668][60934] Updated weights for policy 1, policy_version 4202 (0.0007) [2023-10-13 21:04:00,035][60934] Updated weights for policy 1, policy_version 4212 (0.0007) [2023-10-13 21:04:00,397][60934] Updated weights for policy 1, policy_version 4222 (0.0009) [2023-10-13 21:04:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 8617984. Throughput: 0: 1670.4, 1: 1688.0. Samples: 2158488. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-13 21:04:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:04:01,517][60935] Updated weights for policy 0, policy_version 4200 (0.0009) [2023-10-13 21:04:01,880][60935] Updated weights for policy 0, policy_version 4210 (0.0008) [2023-10-13 21:04:02,249][60935] Updated weights for policy 0, policy_version 4220 (0.0008) [2023-10-13 21:04:04,548][60934] Updated weights for policy 1, policy_version 4232 (0.0008) [2023-10-13 21:04:04,924][60934] Updated weights for policy 1, policy_version 4242 (0.0007) [2023-10-13 21:04:05,296][60934] Updated weights for policy 1, policy_version 4252 (0.0007) [2023-10-13 21:04:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 8683520. Throughput: 0: 1669.9, 1: 1671.3. Samples: 2178506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:04:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:04:06,516][60935] Updated weights for policy 0, policy_version 4230 (0.0009) [2023-10-13 21:04:06,889][60935] Updated weights for policy 0, policy_version 4240 (0.0010) [2023-10-13 21:04:07,253][60935] Updated weights for policy 0, policy_version 4250 (0.0007) [2023-10-13 21:04:09,295][60934] Updated weights for policy 1, policy_version 4262 (0.0009) [2023-10-13 21:04:09,655][60934] Updated weights for policy 1, policy_version 4272 (0.0009) [2023-10-13 21:04:10,031][60934] Updated weights for policy 1, policy_version 4282 (0.0008) [2023-10-13 21:04:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 8749056. Throughput: 0: 1663.5, 1: 1655.8. Samples: 2198076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:04:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:04:11,353][60935] Updated weights for policy 0, policy_version 4260 (0.0008) [2023-10-13 21:04:11,734][60935] Updated weights for policy 0, policy_version 4270 (0.0010) [2023-10-13 21:04:12,111][60935] Updated weights for policy 0, policy_version 4280 (0.0009) [2023-10-13 21:04:14,233][60934] Updated weights for policy 1, policy_version 4292 (0.0008) [2023-10-13 21:04:14,603][60934] Updated weights for policy 1, policy_version 4302 (0.0009) [2023-10-13 21:04:14,974][60934] Updated weights for policy 1, policy_version 4312 (0.0007) [2023-10-13 21:04:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 8814592. Throughput: 0: 1665.1, 1: 1671.6. Samples: 2208258. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 21:04:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:04:16,298][60935] Updated weights for policy 0, policy_version 4290 (0.0009) [2023-10-13 21:04:16,709][60935] Updated weights for policy 0, policy_version 4300 (0.0009) [2023-10-13 21:04:17,073][60935] Updated weights for policy 0, policy_version 4310 (0.0010) [2023-10-13 21:04:17,443][60935] Updated weights for policy 0, policy_version 4320 (0.0008) [2023-10-13 21:04:19,064][60934] Updated weights for policy 1, policy_version 4322 (0.0009) [2023-10-13 21:04:19,432][60934] Updated weights for policy 1, policy_version 4332 (0.0008) [2023-10-13 21:04:19,802][60934] Updated weights for policy 1, policy_version 4342 (0.0010) [2023-10-13 21:04:20,168][60934] Updated weights for policy 1, policy_version 4352 (0.0009) [2023-10-13 21:04:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 8880128. Throughput: 0: 1660.5, 1: 1658.1. Samples: 2228104. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 21:04:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:04:21,598][60935] Updated weights for policy 0, policy_version 4330 (0.0008) [2023-10-13 21:04:21,976][60935] Updated weights for policy 0, policy_version 4340 (0.0009) [2023-10-13 21:04:22,339][60935] Updated weights for policy 0, policy_version 4350 (0.0007) [2023-10-13 21:04:24,381][60934] Updated weights for policy 1, policy_version 4362 (0.0009) [2023-10-13 21:04:24,755][60934] Updated weights for policy 1, policy_version 4372 (0.0010) [2023-10-13 21:04:25,119][60934] Updated weights for policy 1, policy_version 4382 (0.0011) [2023-10-13 21:04:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 8945664. Throughput: 0: 1660.0, 1: 1657.3. Samples: 2247800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:04:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:04:26,397][60935] Updated weights for policy 0, policy_version 4360 (0.0007) [2023-10-13 21:04:26,778][60935] Updated weights for policy 0, policy_version 4370 (0.0008) [2023-10-13 21:04:27,150][60935] Updated weights for policy 0, policy_version 4380 (0.0008) [2023-10-13 21:04:29,319][60934] Updated weights for policy 1, policy_version 4392 (0.0010) [2023-10-13 21:04:29,690][60934] Updated weights for policy 1, policy_version 4402 (0.0010) [2023-10-13 21:04:30,063][60934] Updated weights for policy 1, policy_version 4412 (0.0008) [2023-10-13 21:04:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 9011200. Throughput: 0: 1658.3, 1: 1674.0. Samples: 2258154. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:04:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:04:31,269][60935] Updated weights for policy 0, policy_version 4390 (0.0010) [2023-10-13 21:04:31,645][60935] Updated weights for policy 0, policy_version 4400 (0.0009) [2023-10-13 21:04:32,011][60935] Updated weights for policy 0, policy_version 4410 (0.0010) [2023-10-13 21:04:34,134][60934] Updated weights for policy 1, policy_version 4422 (0.0007) [2023-10-13 21:04:34,497][60934] Updated weights for policy 1, policy_version 4432 (0.0008) [2023-10-13 21:04:34,861][60934] Updated weights for policy 1, policy_version 4442 (0.0009) [2023-10-13 21:04:36,127][60935] Updated weights for policy 0, policy_version 4420 (0.0010) [2023-10-13 21:04:36,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 9076736. Throughput: 0: 1664.2, 1: 1657.3. Samples: 2278184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:04:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:04:36,512][60935] Updated weights for policy 0, policy_version 4430 (0.0009) [2023-10-13 21:04:36,878][60935] Updated weights for policy 0, policy_version 4440 (0.0008) [2023-10-13 21:04:38,923][60934] Updated weights for policy 1, policy_version 4452 (0.0007) [2023-10-13 21:04:39,282][60934] Updated weights for policy 1, policy_version 4462 (0.0010) [2023-10-13 21:04:39,646][60934] Updated weights for policy 1, policy_version 4472 (0.0007) [2023-10-13 21:04:41,000][60935] Updated weights for policy 0, policy_version 4450 (0.0008) [2023-10-13 21:04:41,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 9142272. Throughput: 0: 1659.5, 1: 1668.1. Samples: 2298072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:04:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:04:41,379][60935] Updated weights for policy 0, policy_version 4460 (0.0007) [2023-10-13 21:04:41,756][60935] Updated weights for policy 0, policy_version 4470 (0.0008) [2023-10-13 21:04:42,118][60935] Updated weights for policy 0, policy_version 4480 (0.0011) [2023-10-13 21:04:43,710][60934] Updated weights for policy 1, policy_version 4482 (0.0007) [2023-10-13 21:04:44,089][60934] Updated weights for policy 1, policy_version 4492 (0.0008) [2023-10-13 21:04:44,457][60934] Updated weights for policy 1, policy_version 4502 (0.0008) [2023-10-13 21:04:44,820][60934] Updated weights for policy 1, policy_version 4512 (0.0009) [2023-10-13 21:04:46,103][60935] Updated weights for policy 0, policy_version 4490 (0.0007) [2023-10-13 21:04:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 9207808. Throughput: 0: 1663.1, 1: 1669.7. Samples: 2308464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:04:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:04:46,477][60935] Updated weights for policy 0, policy_version 4500 (0.0007) [2023-10-13 21:04:46,844][60935] Updated weights for policy 0, policy_version 4510 (0.0008) [2023-10-13 21:04:48,967][60934] Updated weights for policy 1, policy_version 4522 (0.0007) [2023-10-13 21:04:49,337][60934] Updated weights for policy 1, policy_version 4532 (0.0008) [2023-10-13 21:04:49,702][60934] Updated weights for policy 1, policy_version 4542 (0.0009) [2023-10-13 21:04:51,095][60935] Updated weights for policy 0, policy_version 4520 (0.0010) [2023-10-13 21:04:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 9273344. Throughput: 0: 1662.1, 1: 1657.5. Samples: 2327886. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:04:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:04:51,466][60935] Updated weights for policy 0, policy_version 4530 (0.0007) [2023-10-13 21:04:51,846][60935] Updated weights for policy 0, policy_version 4540 (0.0009) [2023-10-13 21:04:53,792][60934] Updated weights for policy 1, policy_version 4552 (0.0008) [2023-10-13 21:04:54,160][60934] Updated weights for policy 1, policy_version 4562 (0.0008) [2023-10-13 21:04:54,530][60934] Updated weights for policy 1, policy_version 4572 (0.0008) [2023-10-13 21:04:55,884][60935] Updated weights for policy 0, policy_version 4550 (0.0010) [2023-10-13 21:04:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 9338880. Throughput: 0: 1659.8, 1: 1672.7. Samples: 2348040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:04:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:04:56,258][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000004576_4685824.pth... [2023-10-13 21:04:56,267][60935] Updated weights for policy 0, policy_version 4560 (0.0009) [2023-10-13 21:04:56,300][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000003008_3080192.pth [2023-10-13 21:04:56,644][60935] Updated weights for policy 0, policy_version 4570 (0.0009) [2023-10-13 21:04:56,858][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000004576_4685824.pth... [2023-10-13 21:04:56,896][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000003008_3080192.pth [2023-10-13 21:04:58,661][60934] Updated weights for policy 1, policy_version 4582 (0.0008) [2023-10-13 21:04:59,025][60934] Updated weights for policy 1, policy_version 4592 (0.0008) [2023-10-13 21:04:59,400][60934] Updated weights for policy 1, policy_version 4602 (0.0007) [2023-10-13 21:05:00,689][60935] Updated weights for policy 0, policy_version 4580 (0.0008) [2023-10-13 21:05:01,056][60935] Updated weights for policy 0, policy_version 4590 (0.0008) [2023-10-13 21:05:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 9404416. Throughput: 0: 1665.7, 1: 1673.2. Samples: 2358508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:05:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:05:01,427][60935] Updated weights for policy 0, policy_version 4600 (0.0007) [2023-10-13 21:05:03,393][60934] Updated weights for policy 1, policy_version 4612 (0.0008) [2023-10-13 21:05:03,755][60934] Updated weights for policy 1, policy_version 4622 (0.0008) [2023-10-13 21:05:04,129][60934] Updated weights for policy 1, policy_version 4632 (0.0009) [2023-10-13 21:05:05,541][60935] Updated weights for policy 0, policy_version 4610 (0.0009) [2023-10-13 21:05:05,922][60935] Updated weights for policy 0, policy_version 4620 (0.0008) [2023-10-13 21:05:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 9469952. Throughput: 0: 1671.9, 1: 1661.0. Samples: 2378086. Policy #0 lag: (min: 10.0, avg: 19.6, max: 42.0) [2023-10-13 21:05:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:05:06,295][60935] Updated weights for policy 0, policy_version 4630 (0.0007) [2023-10-13 21:05:06,666][60935] Updated weights for policy 0, policy_version 4640 (0.0008) [2023-10-13 21:05:08,267][60934] Updated weights for policy 1, policy_version 4642 (0.0009) [2023-10-13 21:05:08,648][60934] Updated weights for policy 1, policy_version 4652 (0.0007) [2023-10-13 21:05:09,015][60934] Updated weights for policy 1, policy_version 4662 (0.0007) [2023-10-13 21:05:09,379][60934] Updated weights for policy 1, policy_version 4672 (0.0007) [2023-10-13 21:05:10,668][60935] Updated weights for policy 0, policy_version 4650 (0.0008) [2023-10-13 21:05:11,042][60935] Updated weights for policy 0, policy_version 4660 (0.0007) [2023-10-13 21:05:11,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 9535488. Throughput: 0: 1660.9, 1: 1680.1. Samples: 2398148. Policy #0 lag: (min: 10.0, avg: 19.6, max: 42.0) [2023-10-13 21:05:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:05:11,412][60935] Updated weights for policy 0, policy_version 4670 (0.0007) [2023-10-13 21:05:13,747][60934] Updated weights for policy 1, policy_version 4682 (0.0007) [2023-10-13 21:05:14,132][60934] Updated weights for policy 1, policy_version 4692 (0.0010) [2023-10-13 21:05:14,504][60934] Updated weights for policy 1, policy_version 4702 (0.0008) [2023-10-13 21:05:15,419][60935] Updated weights for policy 0, policy_version 4680 (0.0009) [2023-10-13 21:05:15,788][60935] Updated weights for policy 0, policy_version 4690 (0.0009) [2023-10-13 21:05:16,162][60935] Updated weights for policy 0, policy_version 4700 (0.0009) [2023-10-13 21:05:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 9601024. Throughput: 0: 1675.1, 1: 1671.8. Samples: 2408764. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-13 21:05:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:05:18,446][60934] Updated weights for policy 1, policy_version 4712 (0.0007) [2023-10-13 21:05:18,817][60934] Updated weights for policy 1, policy_version 4722 (0.0007) [2023-10-13 21:05:19,190][60934] Updated weights for policy 1, policy_version 4732 (0.0007) [2023-10-13 21:05:20,175][60935] Updated weights for policy 0, policy_version 4710 (0.0009) [2023-10-13 21:05:20,540][60935] Updated weights for policy 0, policy_version 4720 (0.0008) [2023-10-13 21:05:20,919][60935] Updated weights for policy 0, policy_version 4730 (0.0009) [2023-10-13 21:05:21,248][59943] Fps is (10 sec: 16384.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 9699328. Throughput: 0: 1678.1, 1: 1663.1. Samples: 2428536. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-13 21:05:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:05:23,195][60934] Updated weights for policy 1, policy_version 4742 (0.0008) [2023-10-13 21:05:23,560][60934] Updated weights for policy 1, policy_version 4752 (0.0008) [2023-10-13 21:05:23,920][60934] Updated weights for policy 1, policy_version 4762 (0.0008) [2023-10-13 21:05:24,983][60935] Updated weights for policy 0, policy_version 4740 (0.0009) [2023-10-13 21:05:25,358][60935] Updated weights for policy 0, policy_version 4750 (0.0007) [2023-10-13 21:05:25,733][60935] Updated weights for policy 0, policy_version 4760 (0.0007) [2023-10-13 21:05:26,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 9764864. Throughput: 0: 1658.0, 1: 1679.7. Samples: 2448270. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-13 21:05:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:05:28,155][60934] Updated weights for policy 1, policy_version 4772 (0.0010) [2023-10-13 21:05:28,532][60934] Updated weights for policy 1, policy_version 4782 (0.0008) [2023-10-13 21:05:28,894][60934] Updated weights for policy 1, policy_version 4792 (0.0009) [2023-10-13 21:05:29,779][60935] Updated weights for policy 0, policy_version 4770 (0.0008) [2023-10-13 21:05:30,145][60935] Updated weights for policy 0, policy_version 4780 (0.0008) [2023-10-13 21:05:30,516][60935] Updated weights for policy 0, policy_version 4790 (0.0008) [2023-10-13 21:05:30,887][60935] Updated weights for policy 0, policy_version 4800 (0.0009) [2023-10-13 21:05:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 9830400. Throughput: 0: 1681.8, 1: 1662.2. Samples: 2458944. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 21:05:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:05:32,937][60934] Updated weights for policy 1, policy_version 4802 (0.0010) [2023-10-13 21:05:33,309][60934] Updated weights for policy 1, policy_version 4812 (0.0009) [2023-10-13 21:05:33,690][60934] Updated weights for policy 1, policy_version 4822 (0.0010) [2023-10-13 21:05:34,050][60934] Updated weights for policy 1, policy_version 4832 (0.0010) [2023-10-13 21:05:34,974][60935] Updated weights for policy 0, policy_version 4810 (0.0008) [2023-10-13 21:05:35,349][60935] Updated weights for policy 0, policy_version 4820 (0.0007) [2023-10-13 21:05:35,728][60935] Updated weights for policy 0, policy_version 4830 (0.0009) [2023-10-13 21:05:36,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 9895936. Throughput: 0: 1680.4, 1: 1668.0. Samples: 2478564. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 21:05:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:05:38,056][60934] Updated weights for policy 1, policy_version 4842 (0.0009) [2023-10-13 21:05:38,432][60934] Updated weights for policy 1, policy_version 4852 (0.0011) [2023-10-13 21:05:38,797][60934] Updated weights for policy 1, policy_version 4862 (0.0010) [2023-10-13 21:05:39,738][60935] Updated weights for policy 0, policy_version 4840 (0.0011) [2023-10-13 21:05:40,101][60935] Updated weights for policy 0, policy_version 4850 (0.0010) [2023-10-13 21:05:40,469][60935] Updated weights for policy 0, policy_version 4860 (0.0008) [2023-10-13 21:05:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 9961472. Throughput: 0: 1662.9, 1: 1680.1. Samples: 2498474. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 21:05:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:05:42,817][60934] Updated weights for policy 1, policy_version 4872 (0.0008) [2023-10-13 21:05:43,182][60934] Updated weights for policy 1, policy_version 4882 (0.0008) [2023-10-13 21:05:43,550][60934] Updated weights for policy 1, policy_version 4892 (0.0008) [2023-10-13 21:05:44,612][60935] Updated weights for policy 0, policy_version 4870 (0.0010) [2023-10-13 21:05:44,984][60935] Updated weights for policy 0, policy_version 4880 (0.0010) [2023-10-13 21:05:45,349][60935] Updated weights for policy 0, policy_version 4890 (0.0011) [2023-10-13 21:05:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 10027008. Throughput: 0: 1685.7, 1: 1659.3. Samples: 2509034. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 21:05:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:05:47,632][60934] Updated weights for policy 1, policy_version 4902 (0.0008) [2023-10-13 21:05:47,991][60934] Updated weights for policy 1, policy_version 4912 (0.0008) [2023-10-13 21:05:48,366][60934] Updated weights for policy 1, policy_version 4922 (0.0011) [2023-10-13 21:05:49,626][60935] Updated weights for policy 0, policy_version 4900 (0.0010) [2023-10-13 21:05:49,997][60935] Updated weights for policy 0, policy_version 4910 (0.0010) [2023-10-13 21:05:50,373][60935] Updated weights for policy 0, policy_version 4920 (0.0008) [2023-10-13 21:05:51,248][59943] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 10092544. Throughput: 0: 1668.3, 1: 1679.6. Samples: 2528742. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 21:05:51,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:05:52,354][60934] Updated weights for policy 1, policy_version 4932 (0.0008) [2023-10-13 21:05:52,722][60934] Updated weights for policy 1, policy_version 4942 (0.0007) [2023-10-13 21:05:53,089][60934] Updated weights for policy 1, policy_version 4952 (0.0008) [2023-10-13 21:05:54,500][60935] Updated weights for policy 0, policy_version 4930 (0.0008) [2023-10-13 21:05:54,886][60935] Updated weights for policy 0, policy_version 4940 (0.0007) [2023-10-13 21:05:55,257][60935] Updated weights for policy 0, policy_version 4950 (0.0009) [2023-10-13 21:05:55,623][60935] Updated weights for policy 0, policy_version 4960 (0.0008) [2023-10-13 21:05:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 10158080. Throughput: 0: 1659.2, 1: 1681.1. Samples: 2548464. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 21:05:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:05:57,227][60934] Updated weights for policy 1, policy_version 4962 (0.0007) [2023-10-13 21:05:57,596][60934] Updated weights for policy 1, policy_version 4972 (0.0009) [2023-10-13 21:05:57,976][60934] Updated weights for policy 1, policy_version 4982 (0.0009) [2023-10-13 21:05:58,344][60934] Updated weights for policy 1, policy_version 4992 (0.0009) [2023-10-13 21:05:59,771][60935] Updated weights for policy 0, policy_version 4970 (0.0007) [2023-10-13 21:06:00,138][60935] Updated weights for policy 0, policy_version 4980 (0.0011) [2023-10-13 21:06:00,513][60935] Updated weights for policy 0, policy_version 4990 (0.0010) [2023-10-13 21:06:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 10223616. Throughput: 0: 1675.0, 1: 1656.8. Samples: 2558694. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-13 21:06:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:06:02,287][60934] Updated weights for policy 1, policy_version 5002 (0.0010) [2023-10-13 21:06:02,663][60934] Updated weights for policy 1, policy_version 5012 (0.0010) [2023-10-13 21:06:03,031][60934] Updated weights for policy 1, policy_version 5022 (0.0009) [2023-10-13 21:06:04,538][60935] Updated weights for policy 0, policy_version 5000 (0.0009) [2023-10-13 21:06:04,903][60935] Updated weights for policy 0, policy_version 5010 (0.0009) [2023-10-13 21:06:05,268][60935] Updated weights for policy 0, policy_version 5020 (0.0008) [2023-10-13 21:06:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 10289152. Throughput: 0: 1658.7, 1: 1685.2. Samples: 2579014. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-13 21:06:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:06:07,012][60934] Updated weights for policy 1, policy_version 5032 (0.0009) [2023-10-13 21:06:07,375][60934] Updated weights for policy 1, policy_version 5042 (0.0008) [2023-10-13 21:06:07,744][60934] Updated weights for policy 1, policy_version 5052 (0.0008) [2023-10-13 21:06:09,254][60935] Updated weights for policy 0, policy_version 5030 (0.0007) [2023-10-13 21:06:09,624][60935] Updated weights for policy 0, policy_version 5040 (0.0009) [2023-10-13 21:06:10,001][60935] Updated weights for policy 0, policy_version 5050 (0.0010) [2023-10-13 21:06:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 10354688. Throughput: 0: 1669.3, 1: 1681.3. Samples: 2599046. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-13 21:06:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:06:11,846][60934] Updated weights for policy 1, policy_version 5062 (0.0008) [2023-10-13 21:06:12,211][60934] Updated weights for policy 1, policy_version 5072 (0.0009) [2023-10-13 21:06:12,579][60934] Updated weights for policy 1, policy_version 5082 (0.0009) [2023-10-13 21:06:13,870][60935] Updated weights for policy 0, policy_version 5060 (0.0008) [2023-10-13 21:06:14,237][60935] Updated weights for policy 0, policy_version 5070 (0.0008) [2023-10-13 21:06:14,602][60935] Updated weights for policy 0, policy_version 5080 (0.0009) [2023-10-13 21:06:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 10420224. Throughput: 0: 1673.1, 1: 1669.5. Samples: 2609362. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-13 21:06:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:06:16,664][60934] Updated weights for policy 1, policy_version 5092 (0.0007) [2023-10-13 21:06:17,032][60934] Updated weights for policy 1, policy_version 5102 (0.0007) [2023-10-13 21:06:17,392][60934] Updated weights for policy 1, policy_version 5112 (0.0007) [2023-10-13 21:06:18,638][60935] Updated weights for policy 0, policy_version 5090 (0.0009) [2023-10-13 21:06:19,009][60935] Updated weights for policy 0, policy_version 5100 (0.0008) [2023-10-13 21:06:19,384][60935] Updated weights for policy 0, policy_version 5110 (0.0009) [2023-10-13 21:06:19,761][60935] Updated weights for policy 0, policy_version 5120 (0.0009) [2023-10-13 21:06:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 10485760. Throughput: 0: 1651.6, 1: 1694.3. Samples: 2629130. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 21:06:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:06:21,507][60934] Updated weights for policy 1, policy_version 5122 (0.0007) [2023-10-13 21:06:21,869][60934] Updated weights for policy 1, policy_version 5132 (0.0009) [2023-10-13 21:06:22,231][60934] Updated weights for policy 1, policy_version 5142 (0.0009) [2023-10-13 21:06:22,596][60934] Updated weights for policy 1, policy_version 5152 (0.0010) [2023-10-13 21:06:23,783][60935] Updated weights for policy 0, policy_version 5130 (0.0010) [2023-10-13 21:06:24,156][60935] Updated weights for policy 0, policy_version 5140 (0.0010) [2023-10-13 21:06:24,531][60935] Updated weights for policy 0, policy_version 5150 (0.0011) [2023-10-13 21:06:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 10551296. Throughput: 0: 1675.3, 1: 1683.2. Samples: 2649608. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 21:06:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:06:26,837][60934] Updated weights for policy 1, policy_version 5162 (0.0009) [2023-10-13 21:06:27,203][60934] Updated weights for policy 1, policy_version 5172 (0.0008) [2023-10-13 21:06:27,578][60934] Updated weights for policy 1, policy_version 5182 (0.0009) [2023-10-13 21:06:28,495][60935] Updated weights for policy 0, policy_version 5160 (0.0011) [2023-10-13 21:06:28,871][60935] Updated weights for policy 0, policy_version 5170 (0.0008) [2023-10-13 21:06:29,233][60935] Updated weights for policy 0, policy_version 5180 (0.0007) [2023-10-13 21:06:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 10616832. Throughput: 0: 1661.7, 1: 1678.6. Samples: 2659350. Policy #0 lag: (min: 26.0, avg: 32.5, max: 58.0) [2023-10-13 21:06:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:06:31,593][60934] Updated weights for policy 1, policy_version 5192 (0.0007) [2023-10-13 21:06:31,961][60934] Updated weights for policy 1, policy_version 5202 (0.0008) [2023-10-13 21:06:32,332][60934] Updated weights for policy 1, policy_version 5212 (0.0009) [2023-10-13 21:06:33,479][60935] Updated weights for policy 0, policy_version 5190 (0.0010) [2023-10-13 21:06:33,848][60935] Updated weights for policy 0, policy_version 5200 (0.0010) [2023-10-13 21:06:34,215][60935] Updated weights for policy 0, policy_version 5210 (0.0007) [2023-10-13 21:06:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 10682368. Throughput: 0: 1661.1, 1: 1687.7. Samples: 2679440. Policy #0 lag: (min: 26.0, avg: 32.5, max: 58.0) [2023-10-13 21:06:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:06:36,314][60934] Updated weights for policy 1, policy_version 5222 (0.0010) [2023-10-13 21:06:36,684][60934] Updated weights for policy 1, policy_version 5232 (0.0008) [2023-10-13 21:06:37,053][60934] Updated weights for policy 1, policy_version 5242 (0.0008) [2023-10-13 21:06:38,394][60935] Updated weights for policy 0, policy_version 5220 (0.0008) [2023-10-13 21:06:38,763][60935] Updated weights for policy 0, policy_version 5230 (0.0007) [2023-10-13 21:06:39,140][60935] Updated weights for policy 0, policy_version 5240 (0.0007) [2023-10-13 21:06:41,184][60934] Updated weights for policy 1, policy_version 5252 (0.0009) [2023-10-13 21:06:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 10747904. Throughput: 0: 1680.9, 1: 1686.4. Samples: 2699990. Policy #0 lag: (min: 1.0, avg: 17.1, max: 33.0) [2023-10-13 21:06:41,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:06:41,557][60934] Updated weights for policy 1, policy_version 5262 (0.0007) [2023-10-13 21:06:41,918][60934] Updated weights for policy 1, policy_version 5272 (0.0009) [2023-10-13 21:06:43,415][60935] Updated weights for policy 0, policy_version 5250 (0.0008) [2023-10-13 21:06:43,831][60935] Updated weights for policy 0, policy_version 5260 (0.0011) [2023-10-13 21:06:44,185][60935] Updated weights for policy 0, policy_version 5270 (0.0010) [2023-10-13 21:06:44,552][60935] Updated weights for policy 0, policy_version 5280 (0.0010) [2023-10-13 21:06:45,990][60934] Updated weights for policy 1, policy_version 5282 (0.0009) [2023-10-13 21:06:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 10813440. Throughput: 0: 1665.1, 1: 1690.0. Samples: 2709672. Policy #0 lag: (min: 1.0, avg: 17.1, max: 33.0) [2023-10-13 21:06:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:06:46,361][60934] Updated weights for policy 1, policy_version 5292 (0.0007) [2023-10-13 21:06:46,722][60934] Updated weights for policy 1, policy_version 5302 (0.0008) [2023-10-13 21:06:47,096][60934] Updated weights for policy 1, policy_version 5312 (0.0008) [2023-10-13 21:06:48,814][60935] Updated weights for policy 0, policy_version 5290 (0.0008) [2023-10-13 21:06:49,188][60935] Updated weights for policy 0, policy_version 5300 (0.0009) [2023-10-13 21:06:49,570][60935] Updated weights for policy 0, policy_version 5310 (0.0007) [2023-10-13 21:06:51,096][60934] Updated weights for policy 1, policy_version 5322 (0.0008) [2023-10-13 21:06:51,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 10878976. Throughput: 0: 1652.2, 1: 1682.9. Samples: 2729094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:06:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:06:51,468][60934] Updated weights for policy 1, policy_version 5332 (0.0009) [2023-10-13 21:06:51,835][60934] Updated weights for policy 1, policy_version 5342 (0.0009) [2023-10-13 21:06:53,659][60935] Updated weights for policy 0, policy_version 5320 (0.0010) [2023-10-13 21:06:54,042][60935] Updated weights for policy 0, policy_version 5330 (0.0011) [2023-10-13 21:06:54,408][60935] Updated weights for policy 0, policy_version 5340 (0.0009) [2023-10-13 21:06:56,063][60934] Updated weights for policy 1, policy_version 5352 (0.0008) [2023-10-13 21:06:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 10944512. Throughput: 0: 1661.5, 1: 1688.1. Samples: 2749780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:06:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:06:56,259][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000005344_5472256.pth... [2023-10-13 21:06:56,293][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000003776_3866624.pth [2023-10-13 21:06:56,445][60934] Updated weights for policy 1, policy_version 5362 (0.0007) [2023-10-13 21:06:56,805][60934] Updated weights for policy 1, policy_version 5372 (0.0007) [2023-10-13 21:06:56,951][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000005376_5505024.pth... [2023-10-13 21:06:56,991][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000003808_3899392.pth [2023-10-13 21:06:58,540][60935] Updated weights for policy 0, policy_version 5350 (0.0010) [2023-10-13 21:06:58,915][60935] Updated weights for policy 0, policy_version 5360 (0.0008) [2023-10-13 21:06:59,290][60935] Updated weights for policy 0, policy_version 5370 (0.0009) [2023-10-13 21:07:00,858][60934] Updated weights for policy 1, policy_version 5382 (0.0007) [2023-10-13 21:07:01,221][60934] Updated weights for policy 1, policy_version 5392 (0.0007) [2023-10-13 21:07:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 11010048. Throughput: 0: 1648.3, 1: 1683.6. Samples: 2759294. Policy #0 lag: (min: 4.0, avg: 4.0, max: 7.0) [2023-10-13 21:07:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:07:01,594][60934] Updated weights for policy 1, policy_version 5402 (0.0007) [2023-10-13 21:07:03,302][60935] Updated weights for policy 0, policy_version 5380 (0.0008) [2023-10-13 21:07:03,681][60935] Updated weights for policy 0, policy_version 5390 (0.0007) [2023-10-13 21:07:04,057][60935] Updated weights for policy 0, policy_version 5400 (0.0008) [2023-10-13 21:07:05,804][60934] Updated weights for policy 1, policy_version 5412 (0.0010) [2023-10-13 21:07:06,184][60934] Updated weights for policy 1, policy_version 5422 (0.0008) [2023-10-13 21:07:06,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 11075584. Throughput: 0: 1657.6, 1: 1676.9. Samples: 2779184. Policy #0 lag: (min: 4.0, avg: 4.0, max: 7.0) [2023-10-13 21:07:06,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:07:06,555][60934] Updated weights for policy 1, policy_version 5432 (0.0008) [2023-10-13 21:07:08,158][60935] Updated weights for policy 0, policy_version 5410 (0.0007) [2023-10-13 21:07:08,533][60935] Updated weights for policy 0, policy_version 5420 (0.0009) [2023-10-13 21:07:08,902][60935] Updated weights for policy 0, policy_version 5430 (0.0008) [2023-10-13 21:07:09,286][60935] Updated weights for policy 0, policy_version 5440 (0.0011) [2023-10-13 21:07:10,410][60934] Updated weights for policy 1, policy_version 5442 (0.0011) [2023-10-13 21:07:10,778][60934] Updated weights for policy 1, policy_version 5452 (0.0007) [2023-10-13 21:07:11,155][60934] Updated weights for policy 1, policy_version 5462 (0.0007) [2023-10-13 21:07:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 11141120. Throughput: 0: 1657.0, 1: 1675.2. Samples: 2799558. Policy #0 lag: (min: 2.0, avg: 4.6, max: 34.0) [2023-10-13 21:07:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:07:11,526][60934] Updated weights for policy 1, policy_version 5472 (0.0008) [2023-10-13 21:07:13,381][60935] Updated weights for policy 0, policy_version 5450 (0.0009) [2023-10-13 21:07:13,742][60935] Updated weights for policy 0, policy_version 5460 (0.0009) [2023-10-13 21:07:14,113][60935] Updated weights for policy 0, policy_version 5470 (0.0008) [2023-10-13 21:07:15,664][60934] Updated weights for policy 1, policy_version 5482 (0.0008) [2023-10-13 21:07:16,044][60934] Updated weights for policy 1, policy_version 5492 (0.0009) [2023-10-13 21:07:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 11206656. Throughput: 0: 1651.6, 1: 1678.6. Samples: 2809210. Policy #0 lag: (min: 2.0, avg: 4.6, max: 34.0) [2023-10-13 21:07:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:07:16,412][60934] Updated weights for policy 1, policy_version 5502 (0.0009) [2023-10-13 21:07:18,194][60935] Updated weights for policy 0, policy_version 5480 (0.0008) [2023-10-13 21:07:18,572][60935] Updated weights for policy 0, policy_version 5490 (0.0009) [2023-10-13 21:07:18,954][60935] Updated weights for policy 0, policy_version 5500 (0.0009) [2023-10-13 21:07:20,395][60934] Updated weights for policy 1, policy_version 5512 (0.0007) [2023-10-13 21:07:20,760][60934] Updated weights for policy 1, policy_version 5522 (0.0008) [2023-10-13 21:07:21,132][60934] Updated weights for policy 1, policy_version 5532 (0.0009) [2023-10-13 21:07:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 11272192. Throughput: 0: 1661.0, 1: 1676.5. Samples: 2829624. Policy #0 lag: (min: 17.0, avg: 23.8, max: 49.0) [2023-10-13 21:07:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:07:22,963][60935] Updated weights for policy 0, policy_version 5510 (0.0009) [2023-10-13 21:07:23,331][60935] Updated weights for policy 0, policy_version 5520 (0.0008) [2023-10-13 21:07:23,708][60935] Updated weights for policy 0, policy_version 5530 (0.0007) [2023-10-13 21:07:25,359][60934] Updated weights for policy 1, policy_version 5542 (0.0007) [2023-10-13 21:07:25,725][60934] Updated weights for policy 1, policy_version 5552 (0.0007) [2023-10-13 21:07:26,094][60934] Updated weights for policy 1, policy_version 5562 (0.0007) [2023-10-13 21:07:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 11337728. Throughput: 0: 1665.1, 1: 1667.3. Samples: 2849950. Policy #0 lag: (min: 17.0, avg: 23.8, max: 49.0) [2023-10-13 21:07:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:07:27,621][60935] Updated weights for policy 0, policy_version 5540 (0.0010) [2023-10-13 21:07:27,982][60935] Updated weights for policy 0, policy_version 5550 (0.0009) [2023-10-13 21:07:28,359][60935] Updated weights for policy 0, policy_version 5560 (0.0009) [2023-10-13 21:07:30,148][60934] Updated weights for policy 1, policy_version 5572 (0.0008) [2023-10-13 21:07:30,518][60934] Updated weights for policy 1, policy_version 5582 (0.0008) [2023-10-13 21:07:30,891][60934] Updated weights for policy 1, policy_version 5592 (0.0008) [2023-10-13 21:07:31,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 11436032. Throughput: 0: 1656.0, 1: 1675.6. Samples: 2859598. Policy #0 lag: (min: 17.0, avg: 32.7, max: 49.0) [2023-10-13 21:07:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:07:32,367][60935] Updated weights for policy 0, policy_version 5570 (0.0008) [2023-10-13 21:07:32,732][60935] Updated weights for policy 0, policy_version 5580 (0.0008) [2023-10-13 21:07:33,105][60935] Updated weights for policy 0, policy_version 5590 (0.0009) [2023-10-13 21:07:33,482][60935] Updated weights for policy 0, policy_version 5600 (0.0010) [2023-10-13 21:07:34,808][60934] Updated weights for policy 1, policy_version 5602 (0.0010) [2023-10-13 21:07:35,173][60934] Updated weights for policy 1, policy_version 5612 (0.0010) [2023-10-13 21:07:35,539][60934] Updated weights for policy 1, policy_version 5622 (0.0010) [2023-10-13 21:07:35,908][60934] Updated weights for policy 1, policy_version 5632 (0.0009) [2023-10-13 21:07:36,248][59943] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 11501568. Throughput: 0: 1679.8, 1: 1684.2. Samples: 2880472. Policy #0 lag: (min: 17.0, avg: 32.7, max: 49.0) [2023-10-13 21:07:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:07:37,681][60935] Updated weights for policy 0, policy_version 5610 (0.0010) [2023-10-13 21:07:38,056][60935] Updated weights for policy 0, policy_version 5620 (0.0010) [2023-10-13 21:07:38,423][60935] Updated weights for policy 0, policy_version 5630 (0.0011) [2023-10-13 21:07:39,887][60934] Updated weights for policy 1, policy_version 5642 (0.0008) [2023-10-13 21:07:40,252][60934] Updated weights for policy 1, policy_version 5652 (0.0010) [2023-10-13 21:07:40,616][60934] Updated weights for policy 1, policy_version 5662 (0.0011) [2023-10-13 21:07:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 11567104. Throughput: 0: 1681.1, 1: 1660.6. Samples: 2900154. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:07:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:07:42,538][60935] Updated weights for policy 0, policy_version 5640 (0.0008) [2023-10-13 21:07:42,901][60935] Updated weights for policy 0, policy_version 5650 (0.0007) [2023-10-13 21:07:43,280][60935] Updated weights for policy 0, policy_version 5660 (0.0008) [2023-10-13 21:07:44,765][60934] Updated weights for policy 1, policy_version 5672 (0.0009) [2023-10-13 21:07:45,139][60934] Updated weights for policy 1, policy_version 5682 (0.0009) [2023-10-13 21:07:45,512][60934] Updated weights for policy 1, policy_version 5692 (0.0010) [2023-10-13 21:07:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 11632640. Throughput: 0: 1665.0, 1: 1689.2. Samples: 2910230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:07:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:07:47,282][60935] Updated weights for policy 0, policy_version 5670 (0.0010) [2023-10-13 21:07:47,646][60935] Updated weights for policy 0, policy_version 5680 (0.0009) [2023-10-13 21:07:48,025][60935] Updated weights for policy 0, policy_version 5690 (0.0008) [2023-10-13 21:07:49,508][60934] Updated weights for policy 1, policy_version 5702 (0.0009) [2023-10-13 21:07:49,882][60934] Updated weights for policy 1, policy_version 5712 (0.0007) [2023-10-13 21:07:50,250][60934] Updated weights for policy 1, policy_version 5722 (0.0007) [2023-10-13 21:07:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 11698176. Throughput: 0: 1683.5, 1: 1681.7. Samples: 2930618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:07:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:07:52,147][60935] Updated weights for policy 0, policy_version 5700 (0.0009) [2023-10-13 21:07:52,528][60935] Updated weights for policy 0, policy_version 5710 (0.0010) [2023-10-13 21:07:52,904][60935] Updated weights for policy 0, policy_version 5720 (0.0010) [2023-10-13 21:07:54,377][60934] Updated weights for policy 1, policy_version 5732 (0.0007) [2023-10-13 21:07:54,747][60934] Updated weights for policy 1, policy_version 5742 (0.0009) [2023-10-13 21:07:55,119][60934] Updated weights for policy 1, policy_version 5752 (0.0010) [2023-10-13 21:07:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 11763712. Throughput: 0: 1675.3, 1: 1667.6. Samples: 2949990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:07:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:07:57,161][60935] Updated weights for policy 0, policy_version 5730 (0.0010) [2023-10-13 21:07:57,542][60935] Updated weights for policy 0, policy_version 5740 (0.0009) [2023-10-13 21:07:57,904][60935] Updated weights for policy 0, policy_version 5750 (0.0008) [2023-10-13 21:07:58,278][60935] Updated weights for policy 0, policy_version 5760 (0.0010) [2023-10-13 21:07:59,109][60934] Updated weights for policy 1, policy_version 5762 (0.0008) [2023-10-13 21:07:59,467][60934] Updated weights for policy 1, policy_version 5772 (0.0009) [2023-10-13 21:07:59,836][60934] Updated weights for policy 1, policy_version 5782 (0.0009) [2023-10-13 21:08:00,208][60934] Updated weights for policy 1, policy_version 5792 (0.0009) [2023-10-13 21:08:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 11829248. Throughput: 0: 1664.9, 1: 1690.0. Samples: 2960182. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 21:08:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:08:02,311][60935] Updated weights for policy 0, policy_version 5770 (0.0009) [2023-10-13 21:08:02,688][60935] Updated weights for policy 0, policy_version 5780 (0.0007) [2023-10-13 21:08:03,060][60935] Updated weights for policy 0, policy_version 5790 (0.0008) [2023-10-13 21:08:04,173][60934] Updated weights for policy 1, policy_version 5802 (0.0008) [2023-10-13 21:08:04,538][60934] Updated weights for policy 1, policy_version 5812 (0.0010) [2023-10-13 21:08:04,899][60934] Updated weights for policy 1, policy_version 5822 (0.0011) [2023-10-13 21:08:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 11894784. Throughput: 0: 1679.6, 1: 1672.1. Samples: 2980452. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 21:08:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:08:06,991][60935] Updated weights for policy 0, policy_version 5800 (0.0011) [2023-10-13 21:08:07,350][60935] Updated weights for policy 0, policy_version 5810 (0.0009) [2023-10-13 21:08:07,728][60935] Updated weights for policy 0, policy_version 5820 (0.0009) [2023-10-13 21:08:09,173][60934] Updated weights for policy 1, policy_version 5832 (0.0010) [2023-10-13 21:08:09,536][60934] Updated weights for policy 1, policy_version 5842 (0.0009) [2023-10-13 21:08:09,898][60934] Updated weights for policy 1, policy_version 5852 (0.0007) [2023-10-13 21:08:11,249][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.2, 300 sec: 13440.4). Total num frames: 11960320. Throughput: 0: 1674.2, 1: 1675.3. Samples: 3000678. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-13 21:08:11,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:08:11,851][60935] Updated weights for policy 0, policy_version 5830 (0.0008) [2023-10-13 21:08:12,217][60935] Updated weights for policy 0, policy_version 5840 (0.0008) [2023-10-13 21:08:12,589][60935] Updated weights for policy 0, policy_version 5850 (0.0007) [2023-10-13 21:08:13,977][60934] Updated weights for policy 1, policy_version 5862 (0.0008) [2023-10-13 21:08:14,347][60934] Updated weights for policy 1, policy_version 5872 (0.0008) [2023-10-13 21:08:14,716][60934] Updated weights for policy 1, policy_version 5882 (0.0008) [2023-10-13 21:08:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 12025856. Throughput: 0: 1672.5, 1: 1694.3. Samples: 3011104. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-13 21:08:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:08:16,615][60935] Updated weights for policy 0, policy_version 5860 (0.0007) [2023-10-13 21:08:16,988][60935] Updated weights for policy 0, policy_version 5870 (0.0008) [2023-10-13 21:08:17,355][60935] Updated weights for policy 0, policy_version 5880 (0.0008) [2023-10-13 21:08:18,985][60934] Updated weights for policy 1, policy_version 5892 (0.0008) [2023-10-13 21:08:19,351][60934] Updated weights for policy 1, policy_version 5902 (0.0007) [2023-10-13 21:08:19,726][60934] Updated weights for policy 1, policy_version 5912 (0.0007) [2023-10-13 21:08:21,248][59943] Fps is (10 sec: 13107.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 12091392. Throughput: 0: 1676.8, 1: 1669.9. Samples: 3031076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:08:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:08:21,552][60935] Updated weights for policy 0, policy_version 5890 (0.0009) [2023-10-13 21:08:21,921][60935] Updated weights for policy 0, policy_version 5900 (0.0009) [2023-10-13 21:08:22,302][60935] Updated weights for policy 0, policy_version 5910 (0.0009) [2023-10-13 21:08:22,669][60935] Updated weights for policy 0, policy_version 5920 (0.0008) [2023-10-13 21:08:23,606][60934] Updated weights for policy 1, policy_version 5922 (0.0007) [2023-10-13 21:08:23,982][60934] Updated weights for policy 1, policy_version 5932 (0.0008) [2023-10-13 21:08:24,342][60934] Updated weights for policy 1, policy_version 5942 (0.0008) [2023-10-13 21:08:24,714][60934] Updated weights for policy 1, policy_version 5952 (0.0009) [2023-10-13 21:08:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 12156928. Throughput: 0: 1676.7, 1: 1679.6. Samples: 3051188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:08:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:08:26,928][60935] Updated weights for policy 0, policy_version 5930 (0.0010) [2023-10-13 21:08:27,295][60935] Updated weights for policy 0, policy_version 5940 (0.0010) [2023-10-13 21:08:27,666][60935] Updated weights for policy 0, policy_version 5950 (0.0010) [2023-10-13 21:08:28,920][60934] Updated weights for policy 1, policy_version 5962 (0.0007) [2023-10-13 21:08:29,292][60934] Updated weights for policy 1, policy_version 5972 (0.0007) [2023-10-13 21:08:29,668][60934] Updated weights for policy 1, policy_version 5982 (0.0007) [2023-10-13 21:08:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12222464. Throughput: 0: 1672.7, 1: 1683.3. Samples: 3061248. Policy #0 lag: (min: 17.0, avg: 29.4, max: 49.0) [2023-10-13 21:08:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:08:31,782][60935] Updated weights for policy 0, policy_version 5960 (0.0009) [2023-10-13 21:08:32,154][60935] Updated weights for policy 0, policy_version 5970 (0.0013) [2023-10-13 21:08:32,524][60935] Updated weights for policy 0, policy_version 5980 (0.0011) [2023-10-13 21:08:33,564][60934] Updated weights for policy 1, policy_version 5992 (0.0009) [2023-10-13 21:08:33,938][60934] Updated weights for policy 1, policy_version 6002 (0.0008) [2023-10-13 21:08:34,308][60934] Updated weights for policy 1, policy_version 6012 (0.0007) [2023-10-13 21:08:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12288000. Throughput: 0: 1667.5, 1: 1663.7. Samples: 3080522. Policy #0 lag: (min: 17.0, avg: 29.4, max: 49.0) [2023-10-13 21:08:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:08:36,922][60935] Updated weights for policy 0, policy_version 5990 (0.0010) [2023-10-13 21:08:37,297][60935] Updated weights for policy 0, policy_version 6000 (0.0007) [2023-10-13 21:08:37,671][60935] Updated weights for policy 0, policy_version 6010 (0.0007) [2023-10-13 21:08:38,569][60934] Updated weights for policy 1, policy_version 6022 (0.0009) [2023-10-13 21:08:38,959][60934] Updated weights for policy 1, policy_version 6032 (0.0009) [2023-10-13 21:08:39,324][60934] Updated weights for policy 1, policy_version 6042 (0.0008) [2023-10-13 21:08:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 12353536. Throughput: 0: 1672.2, 1: 1681.1. Samples: 3100888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:08:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:08:41,611][60935] Updated weights for policy 0, policy_version 6020 (0.0009) [2023-10-13 21:08:41,985][60935] Updated weights for policy 0, policy_version 6030 (0.0009) [2023-10-13 21:08:42,351][60935] Updated weights for policy 0, policy_version 6040 (0.0012) [2023-10-13 21:08:43,160][60934] Updated weights for policy 1, policy_version 6052 (0.0008) [2023-10-13 21:08:43,526][60934] Updated weights for policy 1, policy_version 6062 (0.0007) [2023-10-13 21:08:43,896][60934] Updated weights for policy 1, policy_version 6072 (0.0008) [2023-10-13 21:08:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12419072. Throughput: 0: 1676.3, 1: 1674.5. Samples: 3110970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:08:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:08:46,525][60935] Updated weights for policy 0, policy_version 6050 (0.0007) [2023-10-13 21:08:46,893][60935] Updated weights for policy 0, policy_version 6060 (0.0008) [2023-10-13 21:08:47,263][60935] Updated weights for policy 0, policy_version 6070 (0.0009) [2023-10-13 21:08:47,627][60935] Updated weights for policy 0, policy_version 6080 (0.0010) [2023-10-13 21:08:48,081][60934] Updated weights for policy 1, policy_version 6082 (0.0009) [2023-10-13 21:08:48,447][60934] Updated weights for policy 1, policy_version 6092 (0.0010) [2023-10-13 21:08:48,815][60934] Updated weights for policy 1, policy_version 6102 (0.0010) [2023-10-13 21:08:49,178][60934] Updated weights for policy 1, policy_version 6112 (0.0011) [2023-10-13 21:08:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12484608. Throughput: 0: 1668.3, 1: 1670.9. Samples: 3130718. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-13 21:08:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:08:51,556][60935] Updated weights for policy 0, policy_version 6090 (0.0010) [2023-10-13 21:08:51,928][60935] Updated weights for policy 0, policy_version 6100 (0.0010) [2023-10-13 21:08:52,289][60935] Updated weights for policy 0, policy_version 6110 (0.0010) [2023-10-13 21:08:53,321][60934] Updated weights for policy 1, policy_version 6122 (0.0007) [2023-10-13 21:08:53,699][60934] Updated weights for policy 1, policy_version 6132 (0.0010) [2023-10-13 21:08:54,065][60934] Updated weights for policy 1, policy_version 6142 (0.0008) [2023-10-13 21:08:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 12550144. Throughput: 0: 1674.2, 1: 1676.9. Samples: 3151474. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-13 21:08:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:08:56,257][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000006144_6291456.pth... [2023-10-13 21:08:56,296][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000004576_4685824.pth [2023-10-13 21:08:56,328][60935] Updated weights for policy 0, policy_version 6120 (0.0009) [2023-10-13 21:08:56,698][60935] Updated weights for policy 0, policy_version 6130 (0.0009) [2023-10-13 21:08:57,064][60935] Updated weights for policy 0, policy_version 6140 (0.0008) [2023-10-13 21:08:57,211][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000006144_6291456.pth... [2023-10-13 21:08:57,241][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000004576_4685824.pth [2023-10-13 21:08:58,108][60934] Updated weights for policy 1, policy_version 6152 (0.0009) [2023-10-13 21:08:58,484][60934] Updated weights for policy 1, policy_version 6162 (0.0007) [2023-10-13 21:08:58,859][60934] Updated weights for policy 1, policy_version 6172 (0.0008) [2023-10-13 21:09:01,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12615680. Throughput: 0: 1671.2, 1: 1661.7. Samples: 3161086. Policy #0 lag: (min: 10.0, avg: 12.3, max: 42.0) [2023-10-13 21:09:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:09:01,258][60935] Updated weights for policy 0, policy_version 6150 (0.0009) [2023-10-13 21:09:01,644][60935] Updated weights for policy 0, policy_version 6160 (0.0009) [2023-10-13 21:09:02,016][60935] Updated weights for policy 0, policy_version 6170 (0.0008) [2023-10-13 21:09:02,818][60934] Updated weights for policy 1, policy_version 6182 (0.0010) [2023-10-13 21:09:03,182][60934] Updated weights for policy 1, policy_version 6192 (0.0009) [2023-10-13 21:09:03,550][60934] Updated weights for policy 1, policy_version 6202 (0.0007) [2023-10-13 21:09:06,103][60935] Updated weights for policy 0, policy_version 6180 (0.0009) [2023-10-13 21:09:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12681216. Throughput: 0: 1666.8, 1: 1670.8. Samples: 3181270. Policy #0 lag: (min: 10.0, avg: 12.3, max: 42.0) [2023-10-13 21:09:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:09:06,465][60935] Updated weights for policy 0, policy_version 6190 (0.0007) [2023-10-13 21:09:06,852][60935] Updated weights for policy 0, policy_version 6200 (0.0009) [2023-10-13 21:09:07,474][60934] Updated weights for policy 1, policy_version 6212 (0.0007) [2023-10-13 21:09:07,850][60934] Updated weights for policy 1, policy_version 6222 (0.0008) [2023-10-13 21:09:08,220][60934] Updated weights for policy 1, policy_version 6232 (0.0009) [2023-10-13 21:09:11,125][60935] Updated weights for policy 0, policy_version 6210 (0.0012) [2023-10-13 21:09:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.3). Total num frames: 12746752. Throughput: 0: 1661.5, 1: 1682.0. Samples: 3201646. Policy #0 lag: (min: 30.0, avg: 31.9, max: 60.0) [2023-10-13 21:09:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:09:11,526][60935] Updated weights for policy 0, policy_version 6220 (0.0008) [2023-10-13 21:09:11,893][60935] Updated weights for policy 0, policy_version 6230 (0.0011) [2023-10-13 21:09:12,228][60934] Updated weights for policy 1, policy_version 6242 (0.0009) [2023-10-13 21:09:12,259][60935] Updated weights for policy 0, policy_version 6240 (0.0010) [2023-10-13 21:09:12,596][60934] Updated weights for policy 1, policy_version 6252 (0.0009) [2023-10-13 21:09:12,965][60934] Updated weights for policy 1, policy_version 6262 (0.0009) [2023-10-13 21:09:13,330][60934] Updated weights for policy 1, policy_version 6272 (0.0009) [2023-10-13 21:09:16,185][60935] Updated weights for policy 0, policy_version 6250 (0.0009) [2023-10-13 21:09:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12812288. Throughput: 0: 1665.7, 1: 1651.9. Samples: 3210544. Policy #0 lag: (min: 30.0, avg: 31.9, max: 60.0) [2023-10-13 21:09:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:09:16,549][60935] Updated weights for policy 0, policy_version 6260 (0.0007) [2023-10-13 21:09:16,915][60935] Updated weights for policy 0, policy_version 6270 (0.0010) [2023-10-13 21:09:17,528][60934] Updated weights for policy 1, policy_version 6282 (0.0008) [2023-10-13 21:09:17,901][60934] Updated weights for policy 1, policy_version 6292 (0.0009) [2023-10-13 21:09:18,272][60934] Updated weights for policy 1, policy_version 6302 (0.0008) [2023-10-13 21:09:20,912][60935] Updated weights for policy 0, policy_version 6280 (0.0010) [2023-10-13 21:09:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12877824. Throughput: 0: 1669.3, 1: 1681.6. Samples: 3231308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:09:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:09:21,265][60935] Updated weights for policy 0, policy_version 6290 (0.0009) [2023-10-13 21:09:21,638][60935] Updated weights for policy 0, policy_version 6300 (0.0008) [2023-10-13 21:09:22,376][60934] Updated weights for policy 1, policy_version 6312 (0.0009) [2023-10-13 21:09:22,753][60934] Updated weights for policy 1, policy_version 6322 (0.0010) [2023-10-13 21:09:23,121][60934] Updated weights for policy 1, policy_version 6332 (0.0008) [2023-10-13 21:09:25,716][60935] Updated weights for policy 0, policy_version 6310 (0.0009) [2023-10-13 21:09:26,089][60935] Updated weights for policy 0, policy_version 6320 (0.0009) [2023-10-13 21:09:26,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12943360. Throughput: 0: 1664.5, 1: 1684.1. Samples: 3251576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:09:26,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:09:26,464][60935] Updated weights for policy 0, policy_version 6330 (0.0010) [2023-10-13 21:09:27,394][60934] Updated weights for policy 1, policy_version 6342 (0.0010) [2023-10-13 21:09:27,770][60934] Updated weights for policy 1, policy_version 6352 (0.0010) [2023-10-13 21:09:28,133][60934] Updated weights for policy 1, policy_version 6362 (0.0010) [2023-10-13 21:09:30,614][60935] Updated weights for policy 0, policy_version 6340 (0.0009) [2023-10-13 21:09:30,990][60935] Updated weights for policy 0, policy_version 6350 (0.0009) [2023-10-13 21:09:31,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13008896. Throughput: 0: 1670.0, 1: 1659.5. Samples: 3260798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:09:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:09:31,359][60935] Updated weights for policy 0, policy_version 6360 (0.0009) [2023-10-13 21:09:31,869][60934] Updated weights for policy 1, policy_version 6372 (0.0008) [2023-10-13 21:09:32,233][60934] Updated weights for policy 1, policy_version 6382 (0.0010) [2023-10-13 21:09:32,604][60934] Updated weights for policy 1, policy_version 6392 (0.0010) [2023-10-13 21:09:35,570][60935] Updated weights for policy 0, policy_version 6370 (0.0008) [2023-10-13 21:09:35,943][60935] Updated weights for policy 0, policy_version 6380 (0.0008) [2023-10-13 21:09:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13074432. Throughput: 0: 1667.5, 1: 1685.5. Samples: 3281602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:09:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:09:36,308][60935] Updated weights for policy 0, policy_version 6390 (0.0008) [2023-10-13 21:09:36,544][60934] Updated weights for policy 1, policy_version 6402 (0.0009) [2023-10-13 21:09:36,674][60935] Updated weights for policy 0, policy_version 6400 (0.0007) [2023-10-13 21:09:36,907][60934] Updated weights for policy 1, policy_version 6412 (0.0008) [2023-10-13 21:09:37,274][60934] Updated weights for policy 1, policy_version 6422 (0.0010) [2023-10-13 21:09:37,639][60934] Updated weights for policy 1, policy_version 6432 (0.0008) [2023-10-13 21:09:40,674][60935] Updated weights for policy 0, policy_version 6410 (0.0010) [2023-10-13 21:09:41,046][60935] Updated weights for policy 0, policy_version 6420 (0.0009) [2023-10-13 21:09:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 13139968. Throughput: 0: 1651.4, 1: 1690.7. Samples: 3301868. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-13 21:09:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:09:41,413][60935] Updated weights for policy 0, policy_version 6430 (0.0007) [2023-10-13 21:09:41,659][60934] Updated weights for policy 1, policy_version 6442 (0.0009) [2023-10-13 21:09:42,033][60934] Updated weights for policy 1, policy_version 6452 (0.0008) [2023-10-13 21:09:42,394][60934] Updated weights for policy 1, policy_version 6462 (0.0007) [2023-10-13 21:09:45,647][60935] Updated weights for policy 0, policy_version 6440 (0.0010) [2023-10-13 21:09:46,006][60935] Updated weights for policy 0, policy_version 6450 (0.0009) [2023-10-13 21:09:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13205504. Throughput: 0: 1661.3, 1: 1677.7. Samples: 3311342. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-13 21:09:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:09:46,380][60935] Updated weights for policy 0, policy_version 6460 (0.0008) [2023-10-13 21:09:46,393][60934] Updated weights for policy 1, policy_version 6472 (0.0007) [2023-10-13 21:09:46,765][60934] Updated weights for policy 1, policy_version 6482 (0.0008) [2023-10-13 21:09:47,124][60934] Updated weights for policy 1, policy_version 6492 (0.0009) [2023-10-13 21:09:50,434][60935] Updated weights for policy 0, policy_version 6470 (0.0008) [2023-10-13 21:09:50,795][60935] Updated weights for policy 0, policy_version 6480 (0.0009) [2023-10-13 21:09:51,174][60935] Updated weights for policy 0, policy_version 6490 (0.0008) [2023-10-13 21:09:51,232][60934] Updated weights for policy 1, policy_version 6502 (0.0007) [2023-10-13 21:09:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13271040. Throughput: 0: 1660.3, 1: 1692.7. Samples: 3332154. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:09:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:09:51,607][60934] Updated weights for policy 1, policy_version 6512 (0.0007) [2023-10-13 21:09:51,976][60934] Updated weights for policy 1, policy_version 6522 (0.0008) [2023-10-13 21:09:55,332][60935] Updated weights for policy 0, policy_version 6500 (0.0009) [2023-10-13 21:09:55,698][60935] Updated weights for policy 0, policy_version 6510 (0.0007) [2023-10-13 21:09:56,066][60935] Updated weights for policy 0, policy_version 6520 (0.0008) [2023-10-13 21:09:56,110][60934] Updated weights for policy 1, policy_version 6532 (0.0009) [2023-10-13 21:09:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13336576. Throughput: 0: 1650.5, 1: 1692.2. Samples: 3352068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:09:56,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:09:56,486][60934] Updated weights for policy 1, policy_version 6542 (0.0010) [2023-10-13 21:09:56,855][60934] Updated weights for policy 1, policy_version 6552 (0.0010) [2023-10-13 21:10:00,163][60935] Updated weights for policy 0, policy_version 6530 (0.0007) [2023-10-13 21:10:00,565][60935] Updated weights for policy 0, policy_version 6540 (0.0011) [2023-10-13 21:10:00,895][60934] Updated weights for policy 1, policy_version 6562 (0.0007) [2023-10-13 21:10:00,926][60935] Updated weights for policy 0, policy_version 6550 (0.0009) [2023-10-13 21:10:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13402112. Throughput: 0: 1665.4, 1: 1694.7. Samples: 3361746. Policy #0 lag: (min: 0.0, avg: 26.9, max: 32.0) [2023-10-13 21:10:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:10:01,264][60934] Updated weights for policy 1, policy_version 6572 (0.0008) [2023-10-13 21:10:01,298][60935] Updated weights for policy 0, policy_version 6560 (0.0008) [2023-10-13 21:10:01,631][60934] Updated weights for policy 1, policy_version 6582 (0.0007) [2023-10-13 21:10:02,000][60934] Updated weights for policy 1, policy_version 6592 (0.0007) [2023-10-13 21:10:05,271][60935] Updated weights for policy 0, policy_version 6570 (0.0008) [2023-10-13 21:10:05,648][60935] Updated weights for policy 0, policy_version 6580 (0.0009) [2023-10-13 21:10:06,015][60935] Updated weights for policy 0, policy_version 6590 (0.0008) [2023-10-13 21:10:06,027][60934] Updated weights for policy 1, policy_version 6602 (0.0007) [2023-10-13 21:10:06,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 13500416. Throughput: 0: 1666.3, 1: 1693.7. Samples: 3382508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:10:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 21:10:06,395][60934] Updated weights for policy 1, policy_version 6612 (0.0007) [2023-10-13 21:10:06,770][60934] Updated weights for policy 1, policy_version 6622 (0.0010) [2023-10-13 21:10:10,246][60935] Updated weights for policy 0, policy_version 6600 (0.0010) [2023-10-13 21:10:10,625][60935] Updated weights for policy 0, policy_version 6610 (0.0008) [2023-10-13 21:10:10,796][60934] Updated weights for policy 1, policy_version 6632 (0.0008) [2023-10-13 21:10:10,990][60935] Updated weights for policy 0, policy_version 6620 (0.0010) [2023-10-13 21:10:11,165][60934] Updated weights for policy 1, policy_version 6642 (0.0009) [2023-10-13 21:10:11,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 13565952. Throughput: 0: 1648.7, 1: 1694.4. Samples: 3402012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:10:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 21:10:11,542][60934] Updated weights for policy 1, policy_version 6652 (0.0007) [2023-10-13 21:10:15,061][60935] Updated weights for policy 0, policy_version 6630 (0.0007) [2023-10-13 21:10:15,432][60935] Updated weights for policy 0, policy_version 6640 (0.0007) [2023-10-13 21:10:15,638][60934] Updated weights for policy 1, policy_version 6662 (0.0008) [2023-10-13 21:10:15,794][60935] Updated weights for policy 0, policy_version 6650 (0.0010) [2023-10-13 21:10:16,030][60934] Updated weights for policy 1, policy_version 6672 (0.0008) [2023-10-13 21:10:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 13631488. Throughput: 0: 1664.6, 1: 1701.9. Samples: 3412288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:10:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 21:10:16,397][60934] Updated weights for policy 1, policy_version 6682 (0.0007) [2023-10-13 21:10:19,959][60935] Updated weights for policy 0, policy_version 6660 (0.0008) [2023-10-13 21:10:20,328][60935] Updated weights for policy 0, policy_version 6670 (0.0009) [2023-10-13 21:10:20,453][60934] Updated weights for policy 1, policy_version 6692 (0.0007) [2023-10-13 21:10:20,702][60935] Updated weights for policy 0, policy_version 6680 (0.0008) [2023-10-13 21:10:20,812][60934] Updated weights for policy 1, policy_version 6702 (0.0007) [2023-10-13 21:10:21,175][60934] Updated weights for policy 1, policy_version 6712 (0.0008) [2023-10-13 21:10:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 13697024. Throughput: 0: 1664.5, 1: 1695.5. Samples: 3432800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:10:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 21:10:24,965][60935] Updated weights for policy 0, policy_version 6690 (0.0009) [2023-10-13 21:10:25,168][60934] Updated weights for policy 1, policy_version 6722 (0.0009) [2023-10-13 21:10:25,326][60935] Updated weights for policy 0, policy_version 6700 (0.0010) [2023-10-13 21:10:25,532][60934] Updated weights for policy 1, policy_version 6732 (0.0009) [2023-10-13 21:10:25,702][60935] Updated weights for policy 0, policy_version 6710 (0.0010) [2023-10-13 21:10:25,896][60934] Updated weights for policy 1, policy_version 6742 (0.0008) [2023-10-13 21:10:26,066][60935] Updated weights for policy 0, policy_version 6720 (0.0009) [2023-10-13 21:10:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 13762560. Throughput: 0: 1649.4, 1: 1686.1. Samples: 3451962. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-13 21:10:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:10:26,266][60934] Updated weights for policy 1, policy_version 6752 (0.0008) [2023-10-13 21:10:30,191][60935] Updated weights for policy 0, policy_version 6730 (0.0011) [2023-10-13 21:10:30,333][60934] Updated weights for policy 1, policy_version 6762 (0.0010) [2023-10-13 21:10:30,555][60935] Updated weights for policy 0, policy_version 6740 (0.0009) [2023-10-13 21:10:30,702][60934] Updated weights for policy 1, policy_version 6772 (0.0008) [2023-10-13 21:10:30,924][60935] Updated weights for policy 0, policy_version 6750 (0.0009) [2023-10-13 21:10:31,071][60934] Updated weights for policy 1, policy_version 6782 (0.0008) [2023-10-13 21:10:31,248][59943] Fps is (10 sec: 16383.6, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 13860864. Throughput: 0: 1660.3, 1: 1698.6. Samples: 3462494. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-13 21:10:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:10:35,175][60935] Updated weights for policy 0, policy_version 6760 (0.0008) [2023-10-13 21:10:35,261][60934] Updated weights for policy 1, policy_version 6792 (0.0008) [2023-10-13 21:10:35,547][60935] Updated weights for policy 0, policy_version 6770 (0.0007) [2023-10-13 21:10:35,636][60934] Updated weights for policy 1, policy_version 6802 (0.0009) [2023-10-13 21:10:35,908][60935] Updated weights for policy 0, policy_version 6780 (0.0007) [2023-10-13 21:10:35,992][60934] Updated weights for policy 1, policy_version 6812 (0.0008) [2023-10-13 21:10:36,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 13926400. Throughput: 0: 1664.3, 1: 1687.6. Samples: 3482988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:10:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:10:40,106][60935] Updated weights for policy 0, policy_version 6790 (0.0007) [2023-10-13 21:10:40,125][60934] Updated weights for policy 1, policy_version 6822 (0.0009) [2023-10-13 21:10:40,474][60935] Updated weights for policy 0, policy_version 6800 (0.0010) [2023-10-13 21:10:40,488][60934] Updated weights for policy 1, policy_version 6832 (0.0007) [2023-10-13 21:10:40,849][60935] Updated weights for policy 0, policy_version 6810 (0.0010) [2023-10-13 21:10:40,854][60934] Updated weights for policy 1, policy_version 6842 (0.0007) [2023-10-13 21:10:41,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 13991936. Throughput: 0: 1657.9, 1: 1668.1. Samples: 3501736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:10:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:10:44,875][60934] Updated weights for policy 1, policy_version 6852 (0.0008) [2023-10-13 21:10:44,919][60935] Updated weights for policy 0, policy_version 6820 (0.0010) [2023-10-13 21:10:45,246][60934] Updated weights for policy 1, policy_version 6862 (0.0008) [2023-10-13 21:10:45,295][60935] Updated weights for policy 0, policy_version 6830 (0.0010) [2023-10-13 21:10:45,620][60934] Updated weights for policy 1, policy_version 6872 (0.0008) [2023-10-13 21:10:45,667][60935] Updated weights for policy 0, policy_version 6840 (0.0008) [2023-10-13 21:10:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 14057472. Throughput: 0: 1664.9, 1: 1685.4. Samples: 3512512. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-13 21:10:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:10:49,579][60935] Updated weights for policy 0, policy_version 6850 (0.0007) [2023-10-13 21:10:49,845][60934] Updated weights for policy 1, policy_version 6882 (0.0008) [2023-10-13 21:10:49,960][60935] Updated weights for policy 0, policy_version 6860 (0.0007) [2023-10-13 21:10:50,216][60934] Updated weights for policy 1, policy_version 6892 (0.0007) [2023-10-13 21:10:50,329][60935] Updated weights for policy 0, policy_version 6870 (0.0010) [2023-10-13 21:10:50,585][60934] Updated weights for policy 1, policy_version 6902 (0.0007) [2023-10-13 21:10:50,706][60935] Updated weights for policy 0, policy_version 6880 (0.0009) [2023-10-13 21:10:50,948][60934] Updated weights for policy 1, policy_version 6912 (0.0007) [2023-10-13 21:10:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 14123008. Throughput: 0: 1655.4, 1: 1683.6. Samples: 3532764. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-13 21:10:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:10:54,759][60935] Updated weights for policy 0, policy_version 6890 (0.0007) [2023-10-13 21:10:54,993][60934] Updated weights for policy 1, policy_version 6922 (0.0008) [2023-10-13 21:10:55,122][60935] Updated weights for policy 0, policy_version 6900 (0.0008) [2023-10-13 21:10:55,357][60934] Updated weights for policy 1, policy_version 6932 (0.0007) [2023-10-13 21:10:55,496][60935] Updated weights for policy 0, policy_version 6910 (0.0009) [2023-10-13 21:10:55,729][60934] Updated weights for policy 1, policy_version 6942 (0.0007) [2023-10-13 21:10:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 14188544. Throughput: 0: 1661.4, 1: 1666.5. Samples: 3551766. Policy #0 lag: (min: 27.0, avg: 53.2, max: 56.0) [2023-10-13 21:10:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:10:56,263][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000006912_7077888.pth... [2023-10-13 21:10:56,263][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000006944_7110656.pth... [2023-10-13 21:10:56,296][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000005344_5472256.pth [2023-10-13 21:10:56,304][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000005376_5505024.pth [2023-10-13 21:10:59,708][60935] Updated weights for policy 0, policy_version 6920 (0.0008) [2023-10-13 21:10:59,781][60934] Updated weights for policy 1, policy_version 6952 (0.0008) [2023-10-13 21:11:00,083][60935] Updated weights for policy 0, policy_version 6930 (0.0009) [2023-10-13 21:11:00,150][60934] Updated weights for policy 1, policy_version 6962 (0.0009) [2023-10-13 21:11:00,453][60935] Updated weights for policy 0, policy_version 6940 (0.0009) [2023-10-13 21:11:00,520][60934] Updated weights for policy 1, policy_version 6972 (0.0009) [2023-10-13 21:11:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 14254080. Throughput: 0: 1663.6, 1: 1684.9. Samples: 3562974. Policy #0 lag: (min: 27.0, avg: 53.2, max: 56.0) [2023-10-13 21:11:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:11:04,623][60934] Updated weights for policy 1, policy_version 6982 (0.0007) [2023-10-13 21:11:04,694][60935] Updated weights for policy 0, policy_version 6950 (0.0008) [2023-10-13 21:11:04,997][60934] Updated weights for policy 1, policy_version 6992 (0.0007) [2023-10-13 21:11:05,075][60935] Updated weights for policy 0, policy_version 6960 (0.0007) [2023-10-13 21:11:05,361][60934] Updated weights for policy 1, policy_version 7002 (0.0010) [2023-10-13 21:11:05,438][60935] Updated weights for policy 0, policy_version 6970 (0.0010) [2023-10-13 21:11:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 14319616. Throughput: 0: 1653.1, 1: 1678.0. Samples: 3582698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:11:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:11:09,340][60935] Updated weights for policy 0, policy_version 6980 (0.0009) [2023-10-13 21:11:09,462][60934] Updated weights for policy 1, policy_version 7012 (0.0010) [2023-10-13 21:11:09,716][60935] Updated weights for policy 0, policy_version 6990 (0.0007) [2023-10-13 21:11:09,838][60934] Updated weights for policy 1, policy_version 7022 (0.0008) [2023-10-13 21:11:10,087][60935] Updated weights for policy 0, policy_version 7000 (0.0007) [2023-10-13 21:11:10,203][60934] Updated weights for policy 1, policy_version 7032 (0.0007) [2023-10-13 21:11:11,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 14385152. Throughput: 0: 1658.6, 1: 1660.5. Samples: 3601324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:11:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:11:14,166][60935] Updated weights for policy 0, policy_version 7010 (0.0010) [2023-10-13 21:11:14,493][60934] Updated weights for policy 1, policy_version 7042 (0.0008) [2023-10-13 21:11:14,537][60935] Updated weights for policy 0, policy_version 7020 (0.0009) [2023-10-13 21:11:14,862][60934] Updated weights for policy 1, policy_version 7052 (0.0008) [2023-10-13 21:11:14,908][60935] Updated weights for policy 0, policy_version 7030 (0.0009) [2023-10-13 21:11:15,230][60934] Updated weights for policy 1, policy_version 7062 (0.0010) [2023-10-13 21:11:15,269][60935] Updated weights for policy 0, policy_version 7040 (0.0009) [2023-10-13 21:11:15,597][60934] Updated weights for policy 1, policy_version 7072 (0.0010) [2023-10-13 21:11:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 14450688. Throughput: 0: 1667.1, 1: 1676.2. Samples: 3612944. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-13 21:11:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:11:19,359][60935] Updated weights for policy 0, policy_version 7050 (0.0009) [2023-10-13 21:11:19,593][60934] Updated weights for policy 1, policy_version 7082 (0.0009) [2023-10-13 21:11:19,737][60935] Updated weights for policy 0, policy_version 7060 (0.0008) [2023-10-13 21:11:19,961][60934] Updated weights for policy 1, policy_version 7092 (0.0008) [2023-10-13 21:11:20,097][60935] Updated weights for policy 0, policy_version 7070 (0.0010) [2023-10-13 21:11:20,322][60934] Updated weights for policy 1, policy_version 7102 (0.0010) [2023-10-13 21:11:21,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 14516224. Throughput: 0: 1648.5, 1: 1674.4. Samples: 3632520. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-13 21:11:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:11:24,381][60935] Updated weights for policy 0, policy_version 7080 (0.0008) [2023-10-13 21:11:24,382][60934] Updated weights for policy 1, policy_version 7112 (0.0010) [2023-10-13 21:11:24,751][60935] Updated weights for policy 0, policy_version 7090 (0.0010) [2023-10-13 21:11:24,755][60934] Updated weights for policy 1, policy_version 7122 (0.0009) [2023-10-13 21:11:25,119][60935] Updated weights for policy 0, policy_version 7100 (0.0007) [2023-10-13 21:11:25,123][60934] Updated weights for policy 1, policy_version 7132 (0.0008) [2023-10-13 21:11:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 14581760. Throughput: 0: 1659.2, 1: 1670.4. Samples: 3651568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:11:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:11:29,264][60934] Updated weights for policy 1, policy_version 7142 (0.0008) [2023-10-13 21:11:29,359][60935] Updated weights for policy 0, policy_version 7110 (0.0008) [2023-10-13 21:11:29,633][60934] Updated weights for policy 1, policy_version 7152 (0.0007) [2023-10-13 21:11:29,723][60935] Updated weights for policy 0, policy_version 7120 (0.0007) [2023-10-13 21:11:29,994][60934] Updated weights for policy 1, policy_version 7162 (0.0007) [2023-10-13 21:11:30,095][60935] Updated weights for policy 0, policy_version 7130 (0.0008) [2023-10-13 21:11:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 14647296. Throughput: 0: 1664.8, 1: 1682.0. Samples: 3663118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:11:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:11:34,049][60934] Updated weights for policy 1, policy_version 7172 (0.0007) [2023-10-13 21:11:34,220][60935] Updated weights for policy 0, policy_version 7140 (0.0010) [2023-10-13 21:11:34,422][60934] Updated weights for policy 1, policy_version 7182 (0.0009) [2023-10-13 21:11:34,608][60935] Updated weights for policy 0, policy_version 7150 (0.0009) [2023-10-13 21:11:34,784][60934] Updated weights for policy 1, policy_version 7192 (0.0008) [2023-10-13 21:11:34,969][60935] Updated weights for policy 0, policy_version 7160 (0.0008) [2023-10-13 21:11:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 14712832. Throughput: 0: 1656.2, 1: 1667.2. Samples: 3682316. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 21:11:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:11:38,690][60934] Updated weights for policy 1, policy_version 7202 (0.0009) [2023-10-13 21:11:39,055][60935] Updated weights for policy 0, policy_version 7170 (0.0011) [2023-10-13 21:11:39,057][60934] Updated weights for policy 1, policy_version 7212 (0.0008) [2023-10-13 21:11:39,426][60934] Updated weights for policy 1, policy_version 7222 (0.0008) [2023-10-13 21:11:39,427][60935] Updated weights for policy 0, policy_version 7180 (0.0008) [2023-10-13 21:11:39,802][60935] Updated weights for policy 0, policy_version 7190 (0.0008) [2023-10-13 21:11:39,806][60934] Updated weights for policy 1, policy_version 7232 (0.0007) [2023-10-13 21:11:40,176][60935] Updated weights for policy 0, policy_version 7200 (0.0009) [2023-10-13 21:11:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 14778368. Throughput: 0: 1658.9, 1: 1673.1. Samples: 3701708. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 21:11:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:11:43,885][60934] Updated weights for policy 1, policy_version 7242 (0.0007) [2023-10-13 21:11:44,245][60934] Updated weights for policy 1, policy_version 7252 (0.0008) [2023-10-13 21:11:44,409][60935] Updated weights for policy 0, policy_version 7210 (0.0010) [2023-10-13 21:11:44,613][60934] Updated weights for policy 1, policy_version 7262 (0.0008) [2023-10-13 21:11:44,780][60935] Updated weights for policy 0, policy_version 7220 (0.0009) [2023-10-13 21:11:45,162][60935] Updated weights for policy 0, policy_version 7230 (0.0009) [2023-10-13 21:11:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 14843904. Throughput: 0: 1656.6, 1: 1676.9. Samples: 3712980. Policy #0 lag: (min: 29.0, avg: 35.9, max: 61.0) [2023-10-13 21:11:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:11:48,788][60934] Updated weights for policy 1, policy_version 7272 (0.0008) [2023-10-13 21:11:49,156][60934] Updated weights for policy 1, policy_version 7282 (0.0008) [2023-10-13 21:11:49,454][60935] Updated weights for policy 0, policy_version 7240 (0.0010) [2023-10-13 21:11:49,512][60934] Updated weights for policy 1, policy_version 7292 (0.0008) [2023-10-13 21:11:49,816][60935] Updated weights for policy 0, policy_version 7250 (0.0009) [2023-10-13 21:11:50,186][60935] Updated weights for policy 0, policy_version 7260 (0.0011) [2023-10-13 21:11:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 14909440. Throughput: 0: 1646.8, 1: 1657.2. Samples: 3731376. Policy #0 lag: (min: 29.0, avg: 35.9, max: 61.0) [2023-10-13 21:11:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:11:53,569][60934] Updated weights for policy 1, policy_version 7302 (0.0009) [2023-10-13 21:11:53,967][60934] Updated weights for policy 1, policy_version 7312 (0.0009) [2023-10-13 21:11:54,331][60935] Updated weights for policy 0, policy_version 7270 (0.0009) [2023-10-13 21:11:54,335][60934] Updated weights for policy 1, policy_version 7322 (0.0007) [2023-10-13 21:11:54,698][60935] Updated weights for policy 0, policy_version 7280 (0.0009) [2023-10-13 21:11:55,068][60935] Updated weights for policy 0, policy_version 7290 (0.0010) [2023-10-13 21:11:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 14974976. Throughput: 0: 1648.7, 1: 1676.5. Samples: 3750958. Policy #0 lag: (min: 1.0, avg: 12.4, max: 33.0) [2023-10-13 21:11:56,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:11:58,391][60934] Updated weights for policy 1, policy_version 7332 (0.0009) [2023-10-13 21:11:58,762][60934] Updated weights for policy 1, policy_version 7342 (0.0009) [2023-10-13 21:11:59,130][60934] Updated weights for policy 1, policy_version 7352 (0.0007) [2023-10-13 21:11:59,225][60935] Updated weights for policy 0, policy_version 7300 (0.0009) [2023-10-13 21:11:59,591][60935] Updated weights for policy 0, policy_version 7310 (0.0008) [2023-10-13 21:11:59,964][60935] Updated weights for policy 0, policy_version 7320 (0.0008) [2023-10-13 21:12:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 15040512. Throughput: 0: 1646.0, 1: 1665.6. Samples: 3761968. Policy #0 lag: (min: 1.0, avg: 12.4, max: 33.0) [2023-10-13 21:12:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:12:03,448][60934] Updated weights for policy 1, policy_version 7362 (0.0008) [2023-10-13 21:12:03,819][60934] Updated weights for policy 1, policy_version 7372 (0.0007) [2023-10-13 21:12:04,078][60935] Updated weights for policy 0, policy_version 7330 (0.0009) [2023-10-13 21:12:04,178][60934] Updated weights for policy 1, policy_version 7382 (0.0008) [2023-10-13 21:12:04,450][60935] Updated weights for policy 0, policy_version 7340 (0.0008) [2023-10-13 21:12:04,551][60934] Updated weights for policy 1, policy_version 7392 (0.0009) [2023-10-13 21:12:04,824][60935] Updated weights for policy 0, policy_version 7350 (0.0010) [2023-10-13 21:12:05,190][60935] Updated weights for policy 0, policy_version 7360 (0.0011) [2023-10-13 21:12:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 15106048. Throughput: 0: 1643.9, 1: 1647.4. Samples: 3780626. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 21:12:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:12:08,602][60934] Updated weights for policy 1, policy_version 7402 (0.0009) [2023-10-13 21:12:08,965][60934] Updated weights for policy 1, policy_version 7412 (0.0008) [2023-10-13 21:12:09,208][60935] Updated weights for policy 0, policy_version 7370 (0.0007) [2023-10-13 21:12:09,335][60934] Updated weights for policy 1, policy_version 7422 (0.0008) [2023-10-13 21:12:09,573][60935] Updated weights for policy 0, policy_version 7380 (0.0008) [2023-10-13 21:12:09,956][60935] Updated weights for policy 0, policy_version 7390 (0.0008) [2023-10-13 21:12:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 15171584. Throughput: 0: 1646.6, 1: 1671.1. Samples: 3800864. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 21:12:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:12:13,444][60934] Updated weights for policy 1, policy_version 7432 (0.0009) [2023-10-13 21:12:13,809][60934] Updated weights for policy 1, policy_version 7442 (0.0007) [2023-10-13 21:12:14,135][60935] Updated weights for policy 0, policy_version 7400 (0.0008) [2023-10-13 21:12:14,186][60934] Updated weights for policy 1, policy_version 7452 (0.0008) [2023-10-13 21:12:14,501][60935] Updated weights for policy 0, policy_version 7410 (0.0009) [2023-10-13 21:12:14,872][60935] Updated weights for policy 0, policy_version 7420 (0.0009) [2023-10-13 21:12:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 15237120. Throughput: 0: 1648.3, 1: 1659.6. Samples: 3811970. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 21:12:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:12:18,368][60934] Updated weights for policy 1, policy_version 7462 (0.0009) [2023-10-13 21:12:18,748][60934] Updated weights for policy 1, policy_version 7472 (0.0010) [2023-10-13 21:12:18,901][60935] Updated weights for policy 0, policy_version 7430 (0.0008) [2023-10-13 21:12:19,119][60934] Updated weights for policy 1, policy_version 7482 (0.0007) [2023-10-13 21:12:19,271][60935] Updated weights for policy 0, policy_version 7440 (0.0007) [2023-10-13 21:12:19,650][60935] Updated weights for policy 0, policy_version 7450 (0.0009) [2023-10-13 21:12:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 15302656. Throughput: 0: 1638.5, 1: 1656.3. Samples: 3830582. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 21:12:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:12:23,154][60934] Updated weights for policy 1, policy_version 7492 (0.0008) [2023-10-13 21:12:23,521][60934] Updated weights for policy 1, policy_version 7502 (0.0009) [2023-10-13 21:12:23,892][60934] Updated weights for policy 1, policy_version 7512 (0.0007) [2023-10-13 21:12:23,953][60935] Updated weights for policy 0, policy_version 7460 (0.0007) [2023-10-13 21:12:24,332][60935] Updated weights for policy 0, policy_version 7470 (0.0008) [2023-10-13 21:12:24,708][60935] Updated weights for policy 0, policy_version 7480 (0.0008) [2023-10-13 21:12:26,249][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 15368192. Throughput: 0: 1646.9, 1: 1666.5. Samples: 3850812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:12:26,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:12:27,933][60934] Updated weights for policy 1, policy_version 7522 (0.0008) [2023-10-13 21:12:28,299][60934] Updated weights for policy 1, policy_version 7532 (0.0009) [2023-10-13 21:12:28,668][60934] Updated weights for policy 1, policy_version 7542 (0.0009) [2023-10-13 21:12:28,769][60935] Updated weights for policy 0, policy_version 7490 (0.0009) [2023-10-13 21:12:29,033][60934] Updated weights for policy 1, policy_version 7552 (0.0010) [2023-10-13 21:12:29,138][60935] Updated weights for policy 0, policy_version 7500 (0.0009) [2023-10-13 21:12:29,516][60935] Updated weights for policy 0, policy_version 7510 (0.0009) [2023-10-13 21:12:29,895][60935] Updated weights for policy 0, policy_version 7520 (0.0009) [2023-10-13 21:12:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 15433728. Throughput: 0: 1643.4, 1: 1653.5. Samples: 3861340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:12:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:12:33,189][60934] Updated weights for policy 1, policy_version 7562 (0.0007) [2023-10-13 21:12:33,552][60934] Updated weights for policy 1, policy_version 7572 (0.0009) [2023-10-13 21:12:33,914][60934] Updated weights for policy 1, policy_version 7582 (0.0007) [2023-10-13 21:12:33,999][60935] Updated weights for policy 0, policy_version 7530 (0.0009) [2023-10-13 21:12:34,373][60935] Updated weights for policy 0, policy_version 7540 (0.0009) [2023-10-13 21:12:34,736][60935] Updated weights for policy 0, policy_version 7550 (0.0007) [2023-10-13 21:12:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 15499264. Throughput: 0: 1640.5, 1: 1671.1. Samples: 3880400. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-13 21:12:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:12:37,928][60934] Updated weights for policy 1, policy_version 7592 (0.0007) [2023-10-13 21:12:38,301][60934] Updated weights for policy 1, policy_version 7602 (0.0007) [2023-10-13 21:12:38,671][60934] Updated weights for policy 1, policy_version 7612 (0.0008) [2023-10-13 21:12:39,074][60935] Updated weights for policy 0, policy_version 7560 (0.0010) [2023-10-13 21:12:39,453][60935] Updated weights for policy 0, policy_version 7570 (0.0010) [2023-10-13 21:12:39,826][60935] Updated weights for policy 0, policy_version 7580 (0.0009) [2023-10-13 21:12:41,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 15564800. Throughput: 0: 1656.4, 1: 1679.0. Samples: 3901052. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-13 21:12:41,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:12:42,710][60934] Updated weights for policy 1, policy_version 7622 (0.0009) [2023-10-13 21:12:43,101][60934] Updated weights for policy 1, policy_version 7632 (0.0010) [2023-10-13 21:12:43,475][60934] Updated weights for policy 1, policy_version 7642 (0.0008) [2023-10-13 21:12:43,933][60935] Updated weights for policy 0, policy_version 7590 (0.0008) [2023-10-13 21:12:44,308][60935] Updated weights for policy 0, policy_version 7600 (0.0008) [2023-10-13 21:12:44,677][60935] Updated weights for policy 0, policy_version 7610 (0.0011) [2023-10-13 21:12:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 15630336. Throughput: 0: 1657.5, 1: 1664.3. Samples: 3911446. Policy #0 lag: (min: 28.0, avg: 35.7, max: 60.0) [2023-10-13 21:12:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:12:47,460][60934] Updated weights for policy 1, policy_version 7652 (0.0008) [2023-10-13 21:12:47,833][60934] Updated weights for policy 1, policy_version 7662 (0.0009) [2023-10-13 21:12:48,195][60934] Updated weights for policy 1, policy_version 7672 (0.0009) [2023-10-13 21:12:48,798][60935] Updated weights for policy 0, policy_version 7620 (0.0007) [2023-10-13 21:12:49,167][60935] Updated weights for policy 0, policy_version 7630 (0.0009) [2023-10-13 21:12:49,551][60935] Updated weights for policy 0, policy_version 7640 (0.0009) [2023-10-13 21:12:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 15695872. Throughput: 0: 1651.3, 1: 1684.8. Samples: 3930746. Policy #0 lag: (min: 28.0, avg: 35.7, max: 60.0) [2023-10-13 21:12:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:12:52,313][60934] Updated weights for policy 1, policy_version 7682 (0.0009) [2023-10-13 21:12:52,682][60934] Updated weights for policy 1, policy_version 7692 (0.0010) [2023-10-13 21:12:53,046][60934] Updated weights for policy 1, policy_version 7702 (0.0011) [2023-10-13 21:12:53,417][60934] Updated weights for policy 1, policy_version 7712 (0.0010) [2023-10-13 21:12:53,508][60935] Updated weights for policy 0, policy_version 7650 (0.0009) [2023-10-13 21:12:53,881][60935] Updated weights for policy 0, policy_version 7660 (0.0008) [2023-10-13 21:12:54,255][60935] Updated weights for policy 0, policy_version 7670 (0.0008) [2023-10-13 21:12:54,618][60935] Updated weights for policy 0, policy_version 7680 (0.0011) [2023-10-13 21:12:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 15761408. Throughput: 0: 1659.2, 1: 1686.1. Samples: 3951404. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-10-13 21:12:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:12:56,260][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000007680_7864320.pth... [2023-10-13 21:12:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000007712_7897088.pth... [2023-10-13 21:12:56,292][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000006144_6291456.pth [2023-10-13 21:12:56,296][60695] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p0/milestones/checkpoint_000007680_7864320.pth [2023-10-13 21:12:56,298][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000006144_6291456.pth [2023-10-13 21:12:56,304][60828] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p1/milestones/checkpoint_000007712_7897088.pth [2023-10-13 21:12:57,491][60934] Updated weights for policy 1, policy_version 7722 (0.0010) [2023-10-13 21:12:57,854][60934] Updated weights for policy 1, policy_version 7732 (0.0010) [2023-10-13 21:12:58,229][60934] Updated weights for policy 1, policy_version 7742 (0.0008) [2023-10-13 21:12:58,696][60935] Updated weights for policy 0, policy_version 7690 (0.0009) [2023-10-13 21:12:59,068][60935] Updated weights for policy 0, policy_version 7700 (0.0009) [2023-10-13 21:12:59,451][60935] Updated weights for policy 0, policy_version 7710 (0.0010) [2023-10-13 21:13:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 15826944. Throughput: 0: 1649.0, 1: 1668.4. Samples: 3961252. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-10-13 21:13:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:13:02,132][60934] Updated weights for policy 1, policy_version 7752 (0.0009) [2023-10-13 21:13:02,502][60934] Updated weights for policy 1, policy_version 7762 (0.0008) [2023-10-13 21:13:02,863][60934] Updated weights for policy 1, policy_version 7772 (0.0009) [2023-10-13 21:13:03,310][60935] Updated weights for policy 0, policy_version 7720 (0.0009) [2023-10-13 21:13:03,694][60935] Updated weights for policy 0, policy_version 7730 (0.0011) [2023-10-13 21:13:04,063][60935] Updated weights for policy 0, policy_version 7740 (0.0008) [2023-10-13 21:13:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 15892480. Throughput: 0: 1660.7, 1: 1690.6. Samples: 3981390. Policy #0 lag: (min: 31.0, avg: 32.9, max: 55.0) [2023-10-13 21:13:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:13:07,076][60934] Updated weights for policy 1, policy_version 7782 (0.0007) [2023-10-13 21:13:07,440][60934] Updated weights for policy 1, policy_version 7792 (0.0007) [2023-10-13 21:13:07,809][60934] Updated weights for policy 1, policy_version 7802 (0.0007) [2023-10-13 21:13:08,229][60935] Updated weights for policy 0, policy_version 7750 (0.0008) [2023-10-13 21:13:08,610][60935] Updated weights for policy 0, policy_version 7760 (0.0007) [2023-10-13 21:13:08,980][60935] Updated weights for policy 0, policy_version 7770 (0.0010) [2023-10-13 21:13:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 15958016. Throughput: 0: 1663.9, 1: 1691.7. Samples: 4001810. Policy #0 lag: (min: 31.0, avg: 32.9, max: 55.0) [2023-10-13 21:13:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:13:11,752][60934] Updated weights for policy 1, policy_version 7812 (0.0008) [2023-10-13 21:13:12,119][60934] Updated weights for policy 1, policy_version 7822 (0.0008) [2023-10-13 21:13:12,493][60934] Updated weights for policy 1, policy_version 7832 (0.0008) [2023-10-13 21:13:13,072][60935] Updated weights for policy 0, policy_version 7780 (0.0010) [2023-10-13 21:13:13,443][60935] Updated weights for policy 0, policy_version 7790 (0.0008) [2023-10-13 21:13:13,804][60935] Updated weights for policy 0, policy_version 7800 (0.0009) [2023-10-13 21:13:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16023552. Throughput: 0: 1652.9, 1: 1678.9. Samples: 4011270. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-13 21:13:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:13:16,593][60934] Updated weights for policy 1, policy_version 7842 (0.0008) [2023-10-13 21:13:16,965][60934] Updated weights for policy 1, policy_version 7852 (0.0009) [2023-10-13 21:13:17,339][60934] Updated weights for policy 1, policy_version 7862 (0.0008) [2023-10-13 21:13:17,708][60934] Updated weights for policy 1, policy_version 7872 (0.0008) [2023-10-13 21:13:17,848][60935] Updated weights for policy 0, policy_version 7810 (0.0009) [2023-10-13 21:13:18,216][60935] Updated weights for policy 0, policy_version 7820 (0.0011) [2023-10-13 21:13:18,589][60935] Updated weights for policy 0, policy_version 7830 (0.0008) [2023-10-13 21:13:18,960][60935] Updated weights for policy 0, policy_version 7840 (0.0007) [2023-10-13 21:13:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16089088. Throughput: 0: 1671.0, 1: 1695.5. Samples: 4031892. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-13 21:13:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:13:21,774][60934] Updated weights for policy 1, policy_version 7882 (0.0009) [2023-10-13 21:13:22,142][60934] Updated weights for policy 1, policy_version 7892 (0.0007) [2023-10-13 21:13:22,505][60934] Updated weights for policy 1, policy_version 7902 (0.0007) [2023-10-13 21:13:23,217][60935] Updated weights for policy 0, policy_version 7850 (0.0011) [2023-10-13 21:13:23,592][60935] Updated weights for policy 0, policy_version 7860 (0.0007) [2023-10-13 21:13:23,960][60935] Updated weights for policy 0, policy_version 7870 (0.0008) [2023-10-13 21:13:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.3, 300 sec: 13329.3). Total num frames: 16154624. Throughput: 0: 1670.7, 1: 1690.6. Samples: 4052310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:13:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:13:26,626][60934] Updated weights for policy 1, policy_version 7912 (0.0007) [2023-10-13 21:13:26,993][60934] Updated weights for policy 1, policy_version 7922 (0.0009) [2023-10-13 21:13:27,358][60934] Updated weights for policy 1, policy_version 7932 (0.0009) [2023-10-13 21:13:28,161][60935] Updated weights for policy 0, policy_version 7880 (0.0009) [2023-10-13 21:13:28,541][60935] Updated weights for policy 0, policy_version 7890 (0.0009) [2023-10-13 21:13:28,901][60935] Updated weights for policy 0, policy_version 7900 (0.0009) [2023-10-13 21:13:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 16220160. Throughput: 0: 1649.2, 1: 1689.8. Samples: 4061700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:13:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:13:31,477][60934] Updated weights for policy 1, policy_version 7942 (0.0009) [2023-10-13 21:13:31,863][60934] Updated weights for policy 1, policy_version 7952 (0.0008) [2023-10-13 21:13:32,225][60934] Updated weights for policy 1, policy_version 7962 (0.0009) [2023-10-13 21:13:32,945][60935] Updated weights for policy 0, policy_version 7910 (0.0009) [2023-10-13 21:13:33,304][60935] Updated weights for policy 0, policy_version 7920 (0.0009) [2023-10-13 21:13:33,672][60935] Updated weights for policy 0, policy_version 7930 (0.0011) [2023-10-13 21:13:36,241][60934] Updated weights for policy 1, policy_version 7972 (0.0008) [2023-10-13 21:13:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16285696. Throughput: 0: 1670.6, 1: 1694.0. Samples: 4082156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:13:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:13:36,604][60934] Updated weights for policy 1, policy_version 7982 (0.0008) [2023-10-13 21:13:36,978][60934] Updated weights for policy 1, policy_version 7992 (0.0009) [2023-10-13 21:13:37,718][60935] Updated weights for policy 0, policy_version 7940 (0.0009) [2023-10-13 21:13:38,089][60935] Updated weights for policy 0, policy_version 7950 (0.0009) [2023-10-13 21:13:38,470][60935] Updated weights for policy 0, policy_version 7960 (0.0011) [2023-10-13 21:13:40,955][60934] Updated weights for policy 1, policy_version 8002 (0.0008) [2023-10-13 21:13:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16351232. Throughput: 0: 1675.5, 1: 1692.2. Samples: 4102952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:13:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:13:41,317][60934] Updated weights for policy 1, policy_version 8012 (0.0007) [2023-10-13 21:13:41,687][60934] Updated weights for policy 1, policy_version 8022 (0.0009) [2023-10-13 21:13:42,053][60934] Updated weights for policy 1, policy_version 8032 (0.0009) [2023-10-13 21:13:42,586][60935] Updated weights for policy 0, policy_version 7970 (0.0008) [2023-10-13 21:13:42,957][60935] Updated weights for policy 0, policy_version 7980 (0.0008) [2023-10-13 21:13:43,321][60935] Updated weights for policy 0, policy_version 7990 (0.0010) [2023-10-13 21:13:43,687][60935] Updated weights for policy 0, policy_version 8000 (0.0009) [2023-10-13 21:13:45,983][60934] Updated weights for policy 1, policy_version 8042 (0.0009) [2023-10-13 21:13:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16416768. Throughput: 0: 1656.7, 1: 1694.8. Samples: 4112070. Policy #0 lag: (min: 1.0, avg: 16.3, max: 33.0) [2023-10-13 21:13:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:13:46,355][60934] Updated weights for policy 1, policy_version 8052 (0.0009) [2023-10-13 21:13:46,728][60934] Updated weights for policy 1, policy_version 8062 (0.0009) [2023-10-13 21:13:47,623][60935] Updated weights for policy 0, policy_version 8010 (0.0010) [2023-10-13 21:13:48,001][60935] Updated weights for policy 0, policy_version 8020 (0.0010) [2023-10-13 21:13:48,363][60935] Updated weights for policy 0, policy_version 8030 (0.0010) [2023-10-13 21:13:50,738][60934] Updated weights for policy 1, policy_version 8072 (0.0007) [2023-10-13 21:13:51,112][60934] Updated weights for policy 1, policy_version 8082 (0.0008) [2023-10-13 21:13:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16482304. Throughput: 0: 1675.1, 1: 1692.3. Samples: 4132920. Policy #0 lag: (min: 1.0, avg: 16.3, max: 33.0) [2023-10-13 21:13:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:13:51,487][60934] Updated weights for policy 1, policy_version 8092 (0.0008) [2023-10-13 21:13:52,511][60935] Updated weights for policy 0, policy_version 8040 (0.0008) [2023-10-13 21:13:52,893][60935] Updated weights for policy 0, policy_version 8050 (0.0007) [2023-10-13 21:13:53,265][60935] Updated weights for policy 0, policy_version 8060 (0.0007) [2023-10-13 21:13:55,500][60934] Updated weights for policy 1, policy_version 8102 (0.0008) [2023-10-13 21:13:55,871][60934] Updated weights for policy 1, policy_version 8112 (0.0010) [2023-10-13 21:13:56,239][60934] Updated weights for policy 1, policy_version 8122 (0.0010) [2023-10-13 21:13:56,249][59943] Fps is (10 sec: 13106.5, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 16547840. Throughput: 0: 1680.5, 1: 1689.9. Samples: 4153480. Policy #0 lag: (min: 21.0, avg: 30.5, max: 53.0) [2023-10-13 21:13:56,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:13:57,382][60935] Updated weights for policy 0, policy_version 8070 (0.0012) [2023-10-13 21:13:57,773][60935] Updated weights for policy 0, policy_version 8080 (0.0010) [2023-10-13 21:13:58,144][60935] Updated weights for policy 0, policy_version 8090 (0.0009) [2023-10-13 21:14:00,391][60934] Updated weights for policy 1, policy_version 8132 (0.0009) [2023-10-13 21:14:00,750][60934] Updated weights for policy 1, policy_version 8142 (0.0011) [2023-10-13 21:14:01,115][60934] Updated weights for policy 1, policy_version 8152 (0.0011) [2023-10-13 21:14:01,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 16613376. Throughput: 0: 1671.0, 1: 1693.1. Samples: 4162654. Policy #0 lag: (min: 21.0, avg: 30.5, max: 53.0) [2023-10-13 21:14:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:14:02,159][60935] Updated weights for policy 0, policy_version 8100 (0.0008) [2023-10-13 21:14:02,527][60935] Updated weights for policy 0, policy_version 8110 (0.0009) [2023-10-13 21:14:02,891][60935] Updated weights for policy 0, policy_version 8120 (0.0007) [2023-10-13 21:14:05,252][60934] Updated weights for policy 1, policy_version 8162 (0.0008) [2023-10-13 21:14:05,611][60934] Updated weights for policy 1, policy_version 8172 (0.0007) [2023-10-13 21:14:05,980][60934] Updated weights for policy 1, policy_version 8182 (0.0007) [2023-10-13 21:14:06,248][59943] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16678912. Throughput: 0: 1680.9, 1: 1679.9. Samples: 4183126. Policy #0 lag: (min: 2.0, avg: 6.7, max: 34.0) [2023-10-13 21:14:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:14:06,339][60934] Updated weights for policy 1, policy_version 8192 (0.0007) [2023-10-13 21:14:06,905][60935] Updated weights for policy 0, policy_version 8130 (0.0008) [2023-10-13 21:14:07,278][60935] Updated weights for policy 0, policy_version 8140 (0.0009) [2023-10-13 21:14:07,638][60935] Updated weights for policy 0, policy_version 8150 (0.0011) [2023-10-13 21:14:08,005][60935] Updated weights for policy 0, policy_version 8160 (0.0010) [2023-10-13 21:14:10,305][60934] Updated weights for policy 1, policy_version 8202 (0.0008) [2023-10-13 21:14:10,677][60934] Updated weights for policy 1, policy_version 8212 (0.0009) [2023-10-13 21:14:11,040][60934] Updated weights for policy 1, policy_version 8222 (0.0008) [2023-10-13 21:14:11,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 16777216. Throughput: 0: 1685.8, 1: 1672.3. Samples: 4203424. Policy #0 lag: (min: 2.0, avg: 6.7, max: 34.0) [2023-10-13 21:14:11,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:14:12,144][60935] Updated weights for policy 0, policy_version 8170 (0.0008) [2023-10-13 21:14:12,528][60935] Updated weights for policy 0, policy_version 8180 (0.0008) [2023-10-13 21:14:12,896][60935] Updated weights for policy 0, policy_version 8190 (0.0010) [2023-10-13 21:14:15,159][60934] Updated weights for policy 1, policy_version 8232 (0.0008) [2023-10-13 21:14:15,518][60934] Updated weights for policy 1, policy_version 8242 (0.0009) [2023-10-13 21:14:15,879][60934] Updated weights for policy 1, policy_version 8252 (0.0007) [2023-10-13 21:14:16,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 16842752. Throughput: 0: 1678.3, 1: 1686.3. Samples: 4213108. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-13 21:14:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:14:16,871][60935] Updated weights for policy 0, policy_version 8200 (0.0008) [2023-10-13 21:14:17,250][60935] Updated weights for policy 0, policy_version 8210 (0.0008) [2023-10-13 21:14:17,622][60935] Updated weights for policy 0, policy_version 8220 (0.0008) [2023-10-13 21:14:19,929][60934] Updated weights for policy 1, policy_version 8262 (0.0007) [2023-10-13 21:14:20,321][60934] Updated weights for policy 1, policy_version 8272 (0.0008) [2023-10-13 21:14:20,691][60934] Updated weights for policy 1, policy_version 8282 (0.0007) [2023-10-13 21:14:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 16908288. Throughput: 0: 1684.2, 1: 1685.6. Samples: 4233796. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-13 21:14:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:14:21,675][60935] Updated weights for policy 0, policy_version 8230 (0.0008) [2023-10-13 21:14:22,046][60935] Updated weights for policy 0, policy_version 8240 (0.0011) [2023-10-13 21:14:22,421][60935] Updated weights for policy 0, policy_version 8250 (0.0008) [2023-10-13 21:14:24,743][60934] Updated weights for policy 1, policy_version 8292 (0.0009) [2023-10-13 21:14:25,118][60934] Updated weights for policy 1, policy_version 8302 (0.0010) [2023-10-13 21:14:25,485][60934] Updated weights for policy 1, policy_version 8312 (0.0009) [2023-10-13 21:14:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 16973824. Throughput: 0: 1686.1, 1: 1658.3. Samples: 4253450. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-13 21:14:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:14:26,447][60935] Updated weights for policy 0, policy_version 8260 (0.0007) [2023-10-13 21:14:26,817][60935] Updated weights for policy 0, policy_version 8270 (0.0007) [2023-10-13 21:14:27,194][60935] Updated weights for policy 0, policy_version 8280 (0.0008) [2023-10-13 21:14:29,507][60934] Updated weights for policy 1, policy_version 8322 (0.0007) [2023-10-13 21:14:29,879][60934] Updated weights for policy 1, policy_version 8332 (0.0007) [2023-10-13 21:14:30,251][60934] Updated weights for policy 1, policy_version 8342 (0.0008) [2023-10-13 21:14:30,609][60934] Updated weights for policy 1, policy_version 8352 (0.0007) [2023-10-13 21:14:31,200][60935] Updated weights for policy 0, policy_version 8290 (0.0007) [2023-10-13 21:14:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 17039360. Throughput: 0: 1684.6, 1: 1679.9. Samples: 4263474. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-13 21:14:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:14:31,569][60935] Updated weights for policy 0, policy_version 8300 (0.0008) [2023-10-13 21:14:31,941][60935] Updated weights for policy 0, policy_version 8310 (0.0009) [2023-10-13 21:14:32,309][60935] Updated weights for policy 0, policy_version 8320 (0.0008) [2023-10-13 21:14:34,627][60934] Updated weights for policy 1, policy_version 8362 (0.0008) [2023-10-13 21:14:34,993][60934] Updated weights for policy 1, policy_version 8372 (0.0009) [2023-10-13 21:14:35,360][60934] Updated weights for policy 1, policy_version 8382 (0.0008) [2023-10-13 21:14:36,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 17104896. Throughput: 0: 1682.0, 1: 1672.4. Samples: 4283866. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-10-13 21:14:36,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:14:36,271][60935] Updated weights for policy 0, policy_version 8330 (0.0008) [2023-10-13 21:14:36,647][60935] Updated weights for policy 0, policy_version 8340 (0.0009) [2023-10-13 21:14:37,017][60935] Updated weights for policy 0, policy_version 8350 (0.0008) [2023-10-13 21:14:39,464][60934] Updated weights for policy 1, policy_version 8392 (0.0008) [2023-10-13 21:14:39,834][60934] Updated weights for policy 1, policy_version 8402 (0.0010) [2023-10-13 21:14:40,212][60934] Updated weights for policy 1, policy_version 8412 (0.0009) [2023-10-13 21:14:41,143][60935] Updated weights for policy 0, policy_version 8360 (0.0008) [2023-10-13 21:14:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 17170432. Throughput: 0: 1680.0, 1: 1659.3. Samples: 4303748. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-10-13 21:14:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:14:41,512][60935] Updated weights for policy 0, policy_version 8370 (0.0007) [2023-10-13 21:14:41,891][60935] Updated weights for policy 0, policy_version 8380 (0.0010) [2023-10-13 21:14:44,184][60934] Updated weights for policy 1, policy_version 8422 (0.0008) [2023-10-13 21:14:44,564][60934] Updated weights for policy 1, policy_version 8432 (0.0009) [2023-10-13 21:14:44,932][60934] Updated weights for policy 1, policy_version 8442 (0.0008) [2023-10-13 21:14:46,077][60935] Updated weights for policy 0, policy_version 8390 (0.0009) [2023-10-13 21:14:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 17235968. Throughput: 0: 1679.4, 1: 1685.3. Samples: 4314066. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 21:14:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:14:46,449][60935] Updated weights for policy 0, policy_version 8400 (0.0008) [2023-10-13 21:14:46,821][60935] Updated weights for policy 0, policy_version 8410 (0.0008) [2023-10-13 21:14:49,059][60934] Updated weights for policy 1, policy_version 8452 (0.0008) [2023-10-13 21:14:49,428][60934] Updated weights for policy 1, policy_version 8462 (0.0007) [2023-10-13 21:14:49,793][60934] Updated weights for policy 1, policy_version 8472 (0.0007) [2023-10-13 21:14:50,984][60935] Updated weights for policy 0, policy_version 8420 (0.0009) [2023-10-13 21:14:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 17301504. Throughput: 0: 1674.4, 1: 1677.0. Samples: 4333936. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 21:14:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:14:51,353][60935] Updated weights for policy 0, policy_version 8430 (0.0010) [2023-10-13 21:14:51,738][60935] Updated weights for policy 0, policy_version 8440 (0.0007) [2023-10-13 21:14:53,825][60934] Updated weights for policy 1, policy_version 8482 (0.0007) [2023-10-13 21:14:54,195][60934] Updated weights for policy 1, policy_version 8492 (0.0008) [2023-10-13 21:14:54,561][60934] Updated weights for policy 1, policy_version 8502 (0.0008) [2023-10-13 21:14:54,936][60934] Updated weights for policy 1, policy_version 8512 (0.0009) [2023-10-13 21:14:55,676][60935] Updated weights for policy 0, policy_version 8450 (0.0009) [2023-10-13 21:14:56,041][60935] Updated weights for policy 0, policy_version 8460 (0.0007) [2023-10-13 21:14:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 17367040. Throughput: 0: 1673.4, 1: 1674.7. Samples: 4354088. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 21:14:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:14:56,256][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000008512_8716288.pth... [2023-10-13 21:14:56,296][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000006944_7110656.pth [2023-10-13 21:14:56,418][60935] Updated weights for policy 0, policy_version 8470 (0.0007) [2023-10-13 21:14:56,782][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000008480_8683520.pth... [2023-10-13 21:14:56,786][60935] Updated weights for policy 0, policy_version 8480 (0.0009) [2023-10-13 21:14:56,822][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000006912_7077888.pth [2023-10-13 21:14:59,108][60934] Updated weights for policy 1, policy_version 8522 (0.0009) [2023-10-13 21:14:59,474][60934] Updated weights for policy 1, policy_version 8532 (0.0009) [2023-10-13 21:14:59,852][60934] Updated weights for policy 1, policy_version 8542 (0.0009) [2023-10-13 21:15:00,919][60935] Updated weights for policy 0, policy_version 8490 (0.0008) [2023-10-13 21:15:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 17432576. Throughput: 0: 1681.0, 1: 1685.0. Samples: 4364578. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 21:15:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:15:01,286][60935] Updated weights for policy 0, policy_version 8500 (0.0009) [2023-10-13 21:15:01,657][60935] Updated weights for policy 0, policy_version 8510 (0.0011) [2023-10-13 21:15:03,907][60934] Updated weights for policy 1, policy_version 8552 (0.0007) [2023-10-13 21:15:04,282][60934] Updated weights for policy 1, policy_version 8562 (0.0009) [2023-10-13 21:15:04,647][60934] Updated weights for policy 1, policy_version 8572 (0.0010) [2023-10-13 21:15:05,692][60935] Updated weights for policy 0, policy_version 8520 (0.0009) [2023-10-13 21:15:06,053][60935] Updated weights for policy 0, policy_version 8530 (0.0009) [2023-10-13 21:15:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 17498112. Throughput: 0: 1682.4, 1: 1664.8. Samples: 4384422. Policy #0 lag: (min: 26.0, avg: 26.7, max: 40.0) [2023-10-13 21:15:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:15:06,437][60935] Updated weights for policy 0, policy_version 8540 (0.0008) [2023-10-13 21:15:08,662][60934] Updated weights for policy 1, policy_version 8582 (0.0009) [2023-10-13 21:15:09,044][60934] Updated weights for policy 1, policy_version 8592 (0.0007) [2023-10-13 21:15:09,412][60934] Updated weights for policy 1, policy_version 8602 (0.0008) [2023-10-13 21:15:10,638][60935] Updated weights for policy 0, policy_version 8550 (0.0008) [2023-10-13 21:15:11,010][60935] Updated weights for policy 0, policy_version 8560 (0.0008) [2023-10-13 21:15:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17563648. Throughput: 0: 1666.1, 1: 1684.0. Samples: 4404206. Policy #0 lag: (min: 26.0, avg: 26.7, max: 40.0) [2023-10-13 21:15:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:15:11,383][60935] Updated weights for policy 0, policy_version 8570 (0.0007) [2023-10-13 21:15:13,478][60934] Updated weights for policy 1, policy_version 8612 (0.0010) [2023-10-13 21:15:13,845][60934] Updated weights for policy 1, policy_version 8622 (0.0008) [2023-10-13 21:15:14,206][60934] Updated weights for policy 1, policy_version 8632 (0.0007) [2023-10-13 21:15:15,451][60935] Updated weights for policy 0, policy_version 8580 (0.0009) [2023-10-13 21:15:15,821][60935] Updated weights for policy 0, policy_version 8590 (0.0010) [2023-10-13 21:15:16,202][60935] Updated weights for policy 0, policy_version 8600 (0.0007) [2023-10-13 21:15:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 17629184. Throughput: 0: 1678.9, 1: 1684.7. Samples: 4414838. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-13 21:15:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:15:18,246][60934] Updated weights for policy 1, policy_version 8642 (0.0008) [2023-10-13 21:15:18,617][60934] Updated weights for policy 1, policy_version 8652 (0.0010) [2023-10-13 21:15:18,988][60934] Updated weights for policy 1, policy_version 8662 (0.0009) [2023-10-13 21:15:19,359][60934] Updated weights for policy 1, policy_version 8672 (0.0009) [2023-10-13 21:15:20,245][60935] Updated weights for policy 0, policy_version 8610 (0.0008) [2023-10-13 21:15:20,622][60935] Updated weights for policy 0, policy_version 8620 (0.0009) [2023-10-13 21:15:21,003][60935] Updated weights for policy 0, policy_version 8630 (0.0009) [2023-10-13 21:15:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17694720. Throughput: 0: 1679.4, 1: 1668.0. Samples: 4434500. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-13 21:15:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:15:21,374][60935] Updated weights for policy 0, policy_version 8640 (0.0009) [2023-10-13 21:15:23,432][60934] Updated weights for policy 1, policy_version 8682 (0.0010) [2023-10-13 21:15:23,806][60934] Updated weights for policy 1, policy_version 8692 (0.0010) [2023-10-13 21:15:24,185][60934] Updated weights for policy 1, policy_version 8702 (0.0009) [2023-10-13 21:15:25,484][60935] Updated weights for policy 0, policy_version 8650 (0.0009) [2023-10-13 21:15:25,858][60935] Updated weights for policy 0, policy_version 8660 (0.0009) [2023-10-13 21:15:26,223][60935] Updated weights for policy 0, policy_version 8670 (0.0008) [2023-10-13 21:15:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 17760256. Throughput: 0: 1661.0, 1: 1687.4. Samples: 4454426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:15:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:15:28,289][60934] Updated weights for policy 1, policy_version 8712 (0.0008) [2023-10-13 21:15:28,671][60934] Updated weights for policy 1, policy_version 8722 (0.0010) [2023-10-13 21:15:29,038][60934] Updated weights for policy 1, policy_version 8732 (0.0009) [2023-10-13 21:15:30,375][60935] Updated weights for policy 0, policy_version 8680 (0.0009) [2023-10-13 21:15:30,752][60935] Updated weights for policy 0, policy_version 8690 (0.0007) [2023-10-13 21:15:31,121][60935] Updated weights for policy 0, policy_version 8700 (0.0011) [2023-10-13 21:15:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 17825792. Throughput: 0: 1677.7, 1: 1674.6. Samples: 4464920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:15:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:15:32,913][60934] Updated weights for policy 1, policy_version 8742 (0.0009) [2023-10-13 21:15:33,284][60934] Updated weights for policy 1, policy_version 8752 (0.0009) [2023-10-13 21:15:33,646][60934] Updated weights for policy 1, policy_version 8762 (0.0009) [2023-10-13 21:15:35,070][60935] Updated weights for policy 0, policy_version 8710 (0.0009) [2023-10-13 21:15:35,437][60935] Updated weights for policy 0, policy_version 8720 (0.0008) [2023-10-13 21:15:35,814][60935] Updated weights for policy 0, policy_version 8730 (0.0008) [2023-10-13 21:15:36,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 17924096. Throughput: 0: 1684.2, 1: 1674.8. Samples: 4485090. Policy #0 lag: (min: 15.0, avg: 22.6, max: 47.0) [2023-10-13 21:15:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:15:37,837][60934] Updated weights for policy 1, policy_version 8772 (0.0008) [2023-10-13 21:15:38,210][60934] Updated weights for policy 1, policy_version 8782 (0.0008) [2023-10-13 21:15:38,569][60934] Updated weights for policy 1, policy_version 8792 (0.0007) [2023-10-13 21:15:39,882][60935] Updated weights for policy 0, policy_version 8740 (0.0008) [2023-10-13 21:15:40,242][60935] Updated weights for policy 0, policy_version 8750 (0.0008) [2023-10-13 21:15:40,611][60935] Updated weights for policy 0, policy_version 8760 (0.0009) [2023-10-13 21:15:41,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 17989632. Throughput: 0: 1656.6, 1: 1686.4. Samples: 4504524. Policy #0 lag: (min: 9.0, avg: 14.4, max: 41.0) [2023-10-13 21:15:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:15:42,555][60934] Updated weights for policy 1, policy_version 8802 (0.0008) [2023-10-13 21:15:42,924][60934] Updated weights for policy 1, policy_version 8812 (0.0008) [2023-10-13 21:15:43,287][60934] Updated weights for policy 1, policy_version 8822 (0.0007) [2023-10-13 21:15:43,660][60934] Updated weights for policy 1, policy_version 8832 (0.0007) [2023-10-13 21:15:44,586][60935] Updated weights for policy 0, policy_version 8770 (0.0008) [2023-10-13 21:15:44,966][60935] Updated weights for policy 0, policy_version 8780 (0.0008) [2023-10-13 21:15:45,333][60935] Updated weights for policy 0, policy_version 8790 (0.0008) [2023-10-13 21:15:45,697][60935] Updated weights for policy 0, policy_version 8800 (0.0007) [2023-10-13 21:15:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 18055168. Throughput: 0: 1679.7, 1: 1665.7. Samples: 4515120. Policy #0 lag: (min: 9.0, avg: 14.4, max: 41.0) [2023-10-13 21:15:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:15:47,846][60934] Updated weights for policy 1, policy_version 8842 (0.0011) [2023-10-13 21:15:48,216][60934] Updated weights for policy 1, policy_version 8852 (0.0009) [2023-10-13 21:15:48,585][60934] Updated weights for policy 1, policy_version 8862 (0.0008) [2023-10-13 21:15:49,808][60935] Updated weights for policy 0, policy_version 8810 (0.0008) [2023-10-13 21:15:50,182][60935] Updated weights for policy 0, policy_version 8820 (0.0008) [2023-10-13 21:15:50,557][60935] Updated weights for policy 0, policy_version 8830 (0.0009) [2023-10-13 21:15:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 18120704. Throughput: 0: 1668.3, 1: 1686.3. Samples: 4535382. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-13 21:15:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:15:52,490][60934] Updated weights for policy 1, policy_version 8872 (0.0008) [2023-10-13 21:15:52,865][60934] Updated weights for policy 1, policy_version 8882 (0.0009) [2023-10-13 21:15:53,230][60934] Updated weights for policy 1, policy_version 8892 (0.0009) [2023-10-13 21:15:54,434][60935] Updated weights for policy 0, policy_version 8840 (0.0008) [2023-10-13 21:15:54,808][60935] Updated weights for policy 0, policy_version 8850 (0.0010) [2023-10-13 21:15:55,170][60935] Updated weights for policy 0, policy_version 8860 (0.0007) [2023-10-13 21:15:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 18186240. Throughput: 0: 1667.4, 1: 1702.8. Samples: 4555862. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-13 21:15:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:15:57,243][60934] Updated weights for policy 1, policy_version 8902 (0.0008) [2023-10-13 21:15:57,625][60934] Updated weights for policy 1, policy_version 8912 (0.0010) [2023-10-13 21:15:57,986][60934] Updated weights for policy 1, policy_version 8922 (0.0007) [2023-10-13 21:15:59,405][60935] Updated weights for policy 0, policy_version 8870 (0.0008) [2023-10-13 21:15:59,776][60935] Updated weights for policy 0, policy_version 8880 (0.0007) [2023-10-13 21:16:00,146][60935] Updated weights for policy 0, policy_version 8890 (0.0007) [2023-10-13 21:16:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 18251776. Throughput: 0: 1685.5, 1: 1677.6. Samples: 4566176. Policy #0 lag: (min: 17.0, avg: 25.1, max: 49.0) [2023-10-13 21:16:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:16:01,912][60934] Updated weights for policy 1, policy_version 8932 (0.0007) [2023-10-13 21:16:02,286][60934] Updated weights for policy 1, policy_version 8942 (0.0007) [2023-10-13 21:16:02,654][60934] Updated weights for policy 1, policy_version 8952 (0.0007) [2023-10-13 21:16:04,190][60935] Updated weights for policy 0, policy_version 8900 (0.0008) [2023-10-13 21:16:04,553][60935] Updated weights for policy 0, policy_version 8910 (0.0009) [2023-10-13 21:16:04,923][60935] Updated weights for policy 0, policy_version 8920 (0.0008) [2023-10-13 21:16:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 18317312. Throughput: 0: 1670.0, 1: 1700.2. Samples: 4586158. Policy #0 lag: (min: 17.0, avg: 25.1, max: 49.0) [2023-10-13 21:16:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:16:06,755][60934] Updated weights for policy 1, policy_version 8962 (0.0008) [2023-10-13 21:16:07,125][60934] Updated weights for policy 1, policy_version 8972 (0.0010) [2023-10-13 21:16:07,494][60934] Updated weights for policy 1, policy_version 8982 (0.0010) [2023-10-13 21:16:07,859][60934] Updated weights for policy 1, policy_version 8992 (0.0008) [2023-10-13 21:16:08,912][60935] Updated weights for policy 0, policy_version 8930 (0.0009) [2023-10-13 21:16:09,279][60935] Updated weights for policy 0, policy_version 8940 (0.0008) [2023-10-13 21:16:09,649][60935] Updated weights for policy 0, policy_version 8950 (0.0009) [2023-10-13 21:16:10,017][60935] Updated weights for policy 0, policy_version 8960 (0.0009) [2023-10-13 21:16:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 18382848. Throughput: 0: 1676.8, 1: 1697.2. Samples: 4606254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:16:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:16:11,952][60934] Updated weights for policy 1, policy_version 9002 (0.0010) [2023-10-13 21:16:12,317][60934] Updated weights for policy 1, policy_version 9012 (0.0009) [2023-10-13 21:16:12,670][60934] Updated weights for policy 1, policy_version 9022 (0.0008) [2023-10-13 21:16:14,295][60935] Updated weights for policy 0, policy_version 8970 (0.0011) [2023-10-13 21:16:14,667][60935] Updated weights for policy 0, policy_version 8980 (0.0009) [2023-10-13 21:16:15,045][60935] Updated weights for policy 0, policy_version 8990 (0.0010) [2023-10-13 21:16:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 18448384. Throughput: 0: 1692.9, 1: 1681.5. Samples: 4616768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:16:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:16:16,801][60934] Updated weights for policy 1, policy_version 9032 (0.0010) [2023-10-13 21:16:17,177][60934] Updated weights for policy 1, policy_version 9042 (0.0008) [2023-10-13 21:16:17,535][60934] Updated weights for policy 1, policy_version 9052 (0.0007) [2023-10-13 21:16:19,300][60935] Updated weights for policy 0, policy_version 9000 (0.0008) [2023-10-13 21:16:19,684][60935] Updated weights for policy 0, policy_version 9010 (0.0009) [2023-10-13 21:16:20,047][60935] Updated weights for policy 0, policy_version 9020 (0.0008) [2023-10-13 21:16:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 18513920. Throughput: 0: 1665.3, 1: 1696.5. Samples: 4636374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:16:21,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:16:21,540][60934] Updated weights for policy 1, policy_version 9062 (0.0010) [2023-10-13 21:16:21,906][60934] Updated weights for policy 1, policy_version 9072 (0.0011) [2023-10-13 21:16:22,281][60934] Updated weights for policy 1, policy_version 9082 (0.0010) [2023-10-13 21:16:23,852][60935] Updated weights for policy 0, policy_version 9030 (0.0009) [2023-10-13 21:16:24,221][60935] Updated weights for policy 0, policy_version 9040 (0.0008) [2023-10-13 21:16:24,588][60935] Updated weights for policy 0, policy_version 9050 (0.0010) [2023-10-13 21:16:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 18579456. Throughput: 0: 1690.1, 1: 1697.5. Samples: 4656962. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:16:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:16:26,303][60934] Updated weights for policy 1, policy_version 9092 (0.0007) [2023-10-13 21:16:26,668][60934] Updated weights for policy 1, policy_version 9102 (0.0007) [2023-10-13 21:16:27,040][60934] Updated weights for policy 1, policy_version 9112 (0.0007) [2023-10-13 21:16:28,626][60935] Updated weights for policy 0, policy_version 9060 (0.0009) [2023-10-13 21:16:29,000][60935] Updated weights for policy 0, policy_version 9070 (0.0009) [2023-10-13 21:16:29,368][60935] Updated weights for policy 0, policy_version 9080 (0.0010) [2023-10-13 21:16:30,960][60934] Updated weights for policy 1, policy_version 9122 (0.0009) [2023-10-13 21:16:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 18644992. Throughput: 0: 1685.2, 1: 1691.3. Samples: 4667062. Policy #0 lag: (min: 26.0, avg: 34.0, max: 58.0) [2023-10-13 21:16:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:16:31,333][60934] Updated weights for policy 1, policy_version 9132 (0.0007) [2023-10-13 21:16:31,692][60934] Updated weights for policy 1, policy_version 9142 (0.0007) [2023-10-13 21:16:32,058][60934] Updated weights for policy 1, policy_version 9152 (0.0011) [2023-10-13 21:16:33,566][60935] Updated weights for policy 0, policy_version 9090 (0.0008) [2023-10-13 21:16:33,940][60935] Updated weights for policy 0, policy_version 9100 (0.0008) [2023-10-13 21:16:34,311][60935] Updated weights for policy 0, policy_version 9110 (0.0008) [2023-10-13 21:16:34,678][60935] Updated weights for policy 0, policy_version 9120 (0.0009) [2023-10-13 21:16:36,008][60934] Updated weights for policy 1, policy_version 9162 (0.0009) [2023-10-13 21:16:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 18710528. Throughput: 0: 1667.1, 1: 1705.2. Samples: 4687134. Policy #0 lag: (min: 26.0, avg: 34.0, max: 58.0) [2023-10-13 21:16:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:16:36,374][60934] Updated weights for policy 1, policy_version 9172 (0.0009) [2023-10-13 21:16:36,745][60934] Updated weights for policy 1, policy_version 9182 (0.0011) [2023-10-13 21:16:38,596][60935] Updated weights for policy 0, policy_version 9130 (0.0009) [2023-10-13 21:16:38,970][60935] Updated weights for policy 0, policy_version 9140 (0.0008) [2023-10-13 21:16:39,341][60935] Updated weights for policy 0, policy_version 9150 (0.0007) [2023-10-13 21:16:40,737][60934] Updated weights for policy 1, policy_version 9192 (0.0010) [2023-10-13 21:16:41,120][60934] Updated weights for policy 1, policy_version 9202 (0.0007) [2023-10-13 21:16:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 18776064. Throughput: 0: 1681.1, 1: 1697.9. Samples: 4707916. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-13 21:16:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:16:41,492][60934] Updated weights for policy 1, policy_version 9212 (0.0009) [2023-10-13 21:16:43,554][60935] Updated weights for policy 0, policy_version 9160 (0.0011) [2023-10-13 21:16:43,922][60935] Updated weights for policy 0, policy_version 9170 (0.0007) [2023-10-13 21:16:44,285][60935] Updated weights for policy 0, policy_version 9180 (0.0007) [2023-10-13 21:16:45,578][60934] Updated weights for policy 1, policy_version 9222 (0.0008) [2023-10-13 21:16:45,963][60934] Updated weights for policy 1, policy_version 9232 (0.0008) [2023-10-13 21:16:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 18841600. Throughput: 0: 1668.7, 1: 1700.3. Samples: 4717784. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-13 21:16:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:16:46,343][60934] Updated weights for policy 1, policy_version 9242 (0.0010) [2023-10-13 21:16:48,461][60935] Updated weights for policy 0, policy_version 9190 (0.0009) [2023-10-13 21:16:48,821][60935] Updated weights for policy 0, policy_version 9200 (0.0008) [2023-10-13 21:16:49,193][60935] Updated weights for policy 0, policy_version 9210 (0.0008) [2023-10-13 21:16:50,346][60934] Updated weights for policy 1, policy_version 9252 (0.0009) [2023-10-13 21:16:50,715][60934] Updated weights for policy 1, policy_version 9262 (0.0008) [2023-10-13 21:16:51,087][60934] Updated weights for policy 1, policy_version 9272 (0.0008) [2023-10-13 21:16:51,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 18907136. Throughput: 0: 1662.5, 1: 1701.0. Samples: 4737516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:16:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:16:53,346][60935] Updated weights for policy 0, policy_version 9220 (0.0010) [2023-10-13 21:16:53,726][60935] Updated weights for policy 0, policy_version 9230 (0.0008) [2023-10-13 21:16:54,098][60935] Updated weights for policy 0, policy_version 9240 (0.0008) [2023-10-13 21:16:55,169][60934] Updated weights for policy 1, policy_version 9282 (0.0008) [2023-10-13 21:16:55,533][60934] Updated weights for policy 1, policy_version 9292 (0.0009) [2023-10-13 21:16:55,902][60934] Updated weights for policy 1, policy_version 9302 (0.0007) [2023-10-13 21:16:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 18972672. Throughput: 0: 1675.6, 1: 1691.3. Samples: 4757764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:16:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:16:56,257][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000009248_9469952.pth... [2023-10-13 21:16:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000009312_9535488.pth... [2023-10-13 21:16:56,264][60934] Updated weights for policy 1, policy_version 9312 (0.0008) [2023-10-13 21:16:56,286][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000007680_7864320.pth [2023-10-13 21:16:56,289][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000007712_7897088.pth [2023-10-13 21:16:58,173][60935] Updated weights for policy 0, policy_version 9250 (0.0008) [2023-10-13 21:16:58,547][60935] Updated weights for policy 0, policy_version 9260 (0.0010) [2023-10-13 21:16:58,926][60935] Updated weights for policy 0, policy_version 9270 (0.0009) [2023-10-13 21:16:59,291][60935] Updated weights for policy 0, policy_version 9280 (0.0008) [2023-10-13 21:17:00,344][60934] Updated weights for policy 1, policy_version 9322 (0.0009) [2023-10-13 21:17:00,708][60934] Updated weights for policy 1, policy_version 9332 (0.0010) [2023-10-13 21:17:01,085][60934] Updated weights for policy 1, policy_version 9342 (0.0007) [2023-10-13 21:17:01,248][59943] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 19070976. Throughput: 0: 1653.5, 1: 1701.6. Samples: 4767748. Policy #0 lag: (min: 31.0, avg: 33.2, max: 61.0) [2023-10-13 21:17:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:17:03,394][60935] Updated weights for policy 0, policy_version 9290 (0.0007) [2023-10-13 21:17:03,759][60935] Updated weights for policy 0, policy_version 9300 (0.0007) [2023-10-13 21:17:04,122][60935] Updated weights for policy 0, policy_version 9310 (0.0008) [2023-10-13 21:17:05,200][60934] Updated weights for policy 1, policy_version 9352 (0.0007) [2023-10-13 21:17:05,569][60934] Updated weights for policy 1, policy_version 9362 (0.0008) [2023-10-13 21:17:05,935][60934] Updated weights for policy 1, policy_version 9372 (0.0007) [2023-10-13 21:17:06,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 19136512. Throughput: 0: 1669.0, 1: 1700.6. Samples: 4788006. Policy #0 lag: (min: 31.0, avg: 33.2, max: 61.0) [2023-10-13 21:17:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:17:08,118][60935] Updated weights for policy 0, policy_version 9320 (0.0010) [2023-10-13 21:17:08,489][60935] Updated weights for policy 0, policy_version 9330 (0.0007) [2023-10-13 21:17:08,852][60935] Updated weights for policy 0, policy_version 9340 (0.0008) [2023-10-13 21:17:09,902][60934] Updated weights for policy 1, policy_version 9382 (0.0007) [2023-10-13 21:17:10,269][60934] Updated weights for policy 1, policy_version 9392 (0.0007) [2023-10-13 21:17:10,632][60934] Updated weights for policy 1, policy_version 9402 (0.0007) [2023-10-13 21:17:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 19202048. Throughput: 0: 1673.7, 1: 1681.3. Samples: 4807936. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 21:17:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:17:12,921][60935] Updated weights for policy 0, policy_version 9350 (0.0010) [2023-10-13 21:17:13,298][60935] Updated weights for policy 0, policy_version 9360 (0.0009) [2023-10-13 21:17:13,667][60935] Updated weights for policy 0, policy_version 9370 (0.0009) [2023-10-13 21:17:14,645][60934] Updated weights for policy 1, policy_version 9412 (0.0007) [2023-10-13 21:17:15,016][60934] Updated weights for policy 1, policy_version 9422 (0.0007) [2023-10-13 21:17:15,377][60934] Updated weights for policy 1, policy_version 9432 (0.0011) [2023-10-13 21:17:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 19267584. Throughput: 0: 1650.4, 1: 1702.6. Samples: 4817950. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 21:17:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:17:17,578][60935] Updated weights for policy 0, policy_version 9380 (0.0009) [2023-10-13 21:17:17,949][60935] Updated weights for policy 0, policy_version 9390 (0.0010) [2023-10-13 21:17:18,320][60935] Updated weights for policy 0, policy_version 9400 (0.0008) [2023-10-13 21:17:19,434][60934] Updated weights for policy 1, policy_version 9442 (0.0009) [2023-10-13 21:17:19,800][60934] Updated weights for policy 1, policy_version 9452 (0.0010) [2023-10-13 21:17:20,160][60934] Updated weights for policy 1, policy_version 9462 (0.0008) [2023-10-13 21:17:20,530][60934] Updated weights for policy 1, policy_version 9472 (0.0010) [2023-10-13 21:17:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.5). Total num frames: 19333120. Throughput: 0: 1680.8, 1: 1688.1. Samples: 4838734. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 21:17:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:17:22,416][60935] Updated weights for policy 0, policy_version 9410 (0.0007) [2023-10-13 21:17:22,792][60935] Updated weights for policy 0, policy_version 9420 (0.0008) [2023-10-13 21:17:23,171][60935] Updated weights for policy 0, policy_version 9430 (0.0009) [2023-10-13 21:17:23,537][60935] Updated weights for policy 0, policy_version 9440 (0.0007) [2023-10-13 21:17:24,462][60934] Updated weights for policy 1, policy_version 9482 (0.0008) [2023-10-13 21:17:24,829][60934] Updated weights for policy 1, policy_version 9492 (0.0007) [2023-10-13 21:17:25,193][60934] Updated weights for policy 1, policy_version 9502 (0.0009) [2023-10-13 21:17:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 19398656. Throughput: 0: 1682.5, 1: 1667.5. Samples: 4858670. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 21:17:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:17:27,450][60935] Updated weights for policy 0, policy_version 9450 (0.0009) [2023-10-13 21:17:27,816][60935] Updated weights for policy 0, policy_version 9460 (0.0007) [2023-10-13 21:17:28,201][60935] Updated weights for policy 0, policy_version 9470 (0.0008) [2023-10-13 21:17:29,300][60934] Updated weights for policy 1, policy_version 9512 (0.0009) [2023-10-13 21:17:29,661][60934] Updated weights for policy 1, policy_version 9522 (0.0010) [2023-10-13 21:17:30,029][60934] Updated weights for policy 1, policy_version 9532 (0.0008) [2023-10-13 21:17:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 19464192. Throughput: 0: 1663.5, 1: 1696.7. Samples: 4868992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:17:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:17:32,315][60935] Updated weights for policy 0, policy_version 9480 (0.0007) [2023-10-13 21:17:32,682][60935] Updated weights for policy 0, policy_version 9490 (0.0007) [2023-10-13 21:17:33,058][60935] Updated weights for policy 0, policy_version 9500 (0.0008) [2023-10-13 21:17:33,917][60934] Updated weights for policy 1, policy_version 9542 (0.0009) [2023-10-13 21:17:34,284][60934] Updated weights for policy 1, policy_version 9552 (0.0008) [2023-10-13 21:17:34,657][60934] Updated weights for policy 1, policy_version 9562 (0.0010) [2023-10-13 21:17:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 19529728. Throughput: 0: 1690.1, 1: 1679.1. Samples: 4889130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:17:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:17:37,155][60935] Updated weights for policy 0, policy_version 9510 (0.0008) [2023-10-13 21:17:37,536][60935] Updated weights for policy 0, policy_version 9520 (0.0010) [2023-10-13 21:17:37,898][60935] Updated weights for policy 0, policy_version 9530 (0.0008) [2023-10-13 21:17:38,750][60934] Updated weights for policy 1, policy_version 9572 (0.0007) [2023-10-13 21:17:39,151][60934] Updated weights for policy 1, policy_version 9582 (0.0007) [2023-10-13 21:17:39,520][60934] Updated weights for policy 1, policy_version 9592 (0.0007) [2023-10-13 21:17:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 19595264. Throughput: 0: 1684.1, 1: 1679.6. Samples: 4909130. Policy #0 lag: (min: 9.0, avg: 24.1, max: 41.0) [2023-10-13 21:17:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:17:42,022][60935] Updated weights for policy 0, policy_version 9540 (0.0009) [2023-10-13 21:17:42,403][60935] Updated weights for policy 0, policy_version 9550 (0.0010) [2023-10-13 21:17:42,769][60935] Updated weights for policy 0, policy_version 9560 (0.0008) [2023-10-13 21:17:43,542][60934] Updated weights for policy 1, policy_version 9602 (0.0009) [2023-10-13 21:17:43,917][60934] Updated weights for policy 1, policy_version 9612 (0.0007) [2023-10-13 21:17:44,288][60934] Updated weights for policy 1, policy_version 9622 (0.0008) [2023-10-13 21:17:44,656][60934] Updated weights for policy 1, policy_version 9632 (0.0009) [2023-10-13 21:17:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 19660800. Throughput: 0: 1674.0, 1: 1695.9. Samples: 4919392. Policy #0 lag: (min: 9.0, avg: 24.1, max: 41.0) [2023-10-13 21:17:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:17:46,970][60935] Updated weights for policy 0, policy_version 9570 (0.0009) [2023-10-13 21:17:47,335][60935] Updated weights for policy 0, policy_version 9580 (0.0011) [2023-10-13 21:17:47,701][60935] Updated weights for policy 0, policy_version 9590 (0.0008) [2023-10-13 21:17:48,077][60935] Updated weights for policy 0, policy_version 9600 (0.0008) [2023-10-13 21:17:48,782][60934] Updated weights for policy 1, policy_version 9642 (0.0009) [2023-10-13 21:17:49,153][60934] Updated weights for policy 1, policy_version 9652 (0.0009) [2023-10-13 21:17:49,523][60934] Updated weights for policy 1, policy_version 9662 (0.0011) [2023-10-13 21:17:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 19726336. Throughput: 0: 1684.5, 1: 1671.8. Samples: 4939038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:17:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:17:52,231][60935] Updated weights for policy 0, policy_version 9610 (0.0010) [2023-10-13 21:17:52,599][60935] Updated weights for policy 0, policy_version 9620 (0.0011) [2023-10-13 21:17:52,974][60935] Updated weights for policy 0, policy_version 9630 (0.0011) [2023-10-13 21:17:53,784][60934] Updated weights for policy 1, policy_version 9672 (0.0010) [2023-10-13 21:17:54,141][60934] Updated weights for policy 1, policy_version 9682 (0.0009) [2023-10-13 21:17:54,520][60934] Updated weights for policy 1, policy_version 9692 (0.0007) [2023-10-13 21:17:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 19791872. Throughput: 0: 1681.0, 1: 1686.3. Samples: 4959464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:17:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:17:56,835][60935] Updated weights for policy 0, policy_version 9640 (0.0011) [2023-10-13 21:17:57,218][60935] Updated weights for policy 0, policy_version 9650 (0.0010) [2023-10-13 21:17:57,590][60935] Updated weights for policy 0, policy_version 9660 (0.0010) [2023-10-13 21:17:58,600][60934] Updated weights for policy 1, policy_version 9702 (0.0007) [2023-10-13 21:17:58,967][60934] Updated weights for policy 1, policy_version 9712 (0.0008) [2023-10-13 21:17:59,337][60934] Updated weights for policy 1, policy_version 9722 (0.0008) [2023-10-13 21:18:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 19857408. Throughput: 0: 1678.7, 1: 1691.6. Samples: 4969616. Policy #0 lag: (min: 25.0, avg: 32.2, max: 57.0) [2023-10-13 21:18:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:18:01,816][60935] Updated weights for policy 0, policy_version 9670 (0.0009) [2023-10-13 21:18:02,191][60935] Updated weights for policy 0, policy_version 9680 (0.0008) [2023-10-13 21:18:02,566][60935] Updated weights for policy 0, policy_version 9690 (0.0007) [2023-10-13 21:18:03,265][60934] Updated weights for policy 1, policy_version 9732 (0.0009) [2023-10-13 21:18:03,635][60934] Updated weights for policy 1, policy_version 9742 (0.0007) [2023-10-13 21:18:03,999][60934] Updated weights for policy 1, policy_version 9752 (0.0007) [2023-10-13 21:18:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 19922944. Throughput: 0: 1677.7, 1: 1670.8. Samples: 4989416. Policy #0 lag: (min: 25.0, avg: 32.2, max: 57.0) [2023-10-13 21:18:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:18:06,532][60935] Updated weights for policy 0, policy_version 9700 (0.0008) [2023-10-13 21:18:06,897][60935] Updated weights for policy 0, policy_version 9710 (0.0010) [2023-10-13 21:18:07,265][60935] Updated weights for policy 0, policy_version 9720 (0.0009) [2023-10-13 21:18:07,981][60934] Updated weights for policy 1, policy_version 9762 (0.0009) [2023-10-13 21:18:08,345][60934] Updated weights for policy 1, policy_version 9772 (0.0007) [2023-10-13 21:18:08,706][60934] Updated weights for policy 1, policy_version 9782 (0.0007) [2023-10-13 21:18:09,073][60934] Updated weights for policy 1, policy_version 9792 (0.0008) [2023-10-13 21:18:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 19988480. Throughput: 0: 1672.8, 1: 1693.1. Samples: 5010136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:18:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:18:11,387][60935] Updated weights for policy 0, policy_version 9730 (0.0010) [2023-10-13 21:18:11,748][60935] Updated weights for policy 0, policy_version 9740 (0.0011) [2023-10-13 21:18:12,119][60935] Updated weights for policy 0, policy_version 9750 (0.0007) [2023-10-13 21:18:12,499][60935] Updated weights for policy 0, policy_version 9760 (0.0007) [2023-10-13 21:18:13,158][60934] Updated weights for policy 1, policy_version 9802 (0.0009) [2023-10-13 21:18:13,526][60934] Updated weights for policy 1, policy_version 9812 (0.0007) [2023-10-13 21:18:13,900][60934] Updated weights for policy 1, policy_version 9822 (0.0009) [2023-10-13 21:18:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 20054016. Throughput: 0: 1679.0, 1: 1675.3. Samples: 5019936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:18:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:18:16,421][60935] Updated weights for policy 0, policy_version 9770 (0.0009) [2023-10-13 21:18:16,781][60935] Updated weights for policy 0, policy_version 9780 (0.0009) [2023-10-13 21:18:17,153][60935] Updated weights for policy 0, policy_version 9790 (0.0010) [2023-10-13 21:18:17,822][60934] Updated weights for policy 1, policy_version 9832 (0.0010) [2023-10-13 21:18:18,188][60934] Updated weights for policy 1, policy_version 9842 (0.0011) [2023-10-13 21:18:18,560][60934] Updated weights for policy 1, policy_version 9852 (0.0009) [2023-10-13 21:18:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 20119552. Throughput: 0: 1673.2, 1: 1683.0. Samples: 5040160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:18:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:18:21,272][60935] Updated weights for policy 0, policy_version 9800 (0.0010) [2023-10-13 21:18:21,639][60935] Updated weights for policy 0, policy_version 9810 (0.0008) [2023-10-13 21:18:22,006][60935] Updated weights for policy 0, policy_version 9820 (0.0009) [2023-10-13 21:18:22,694][60934] Updated weights for policy 1, policy_version 9862 (0.0007) [2023-10-13 21:18:23,061][60934] Updated weights for policy 1, policy_version 9872 (0.0008) [2023-10-13 21:18:23,433][60934] Updated weights for policy 1, policy_version 9882 (0.0008) [2023-10-13 21:18:26,137][60935] Updated weights for policy 0, policy_version 9830 (0.0009) [2023-10-13 21:18:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 20185088. Throughput: 0: 1677.8, 1: 1696.4. Samples: 5060970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:18:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:18:26,508][60935] Updated weights for policy 0, policy_version 9840 (0.0008) [2023-10-13 21:18:26,877][60935] Updated weights for policy 0, policy_version 9850 (0.0010) [2023-10-13 21:18:27,378][60934] Updated weights for policy 1, policy_version 9892 (0.0009) [2023-10-13 21:18:27,781][60934] Updated weights for policy 1, policy_version 9902 (0.0007) [2023-10-13 21:18:28,148][60934] Updated weights for policy 1, policy_version 9912 (0.0008) [2023-10-13 21:18:30,953][60935] Updated weights for policy 0, policy_version 9860 (0.0010) [2023-10-13 21:18:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 20250624. Throughput: 0: 1677.7, 1: 1667.4. Samples: 5069922. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-13 21:18:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:18:31,318][60935] Updated weights for policy 0, policy_version 9870 (0.0009) [2023-10-13 21:18:31,686][60935] Updated weights for policy 0, policy_version 9880 (0.0010) [2023-10-13 21:18:31,929][60934] Updated weights for policy 1, policy_version 9922 (0.0008) [2023-10-13 21:18:32,293][60934] Updated weights for policy 1, policy_version 9932 (0.0009) [2023-10-13 21:18:32,666][60934] Updated weights for policy 1, policy_version 9942 (0.0011) [2023-10-13 21:18:33,035][60934] Updated weights for policy 1, policy_version 9952 (0.0007) [2023-10-13 21:18:35,808][60935] Updated weights for policy 0, policy_version 9890 (0.0010) [2023-10-13 21:18:36,173][60935] Updated weights for policy 0, policy_version 9900 (0.0007) [2023-10-13 21:18:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 20316160. Throughput: 0: 1680.5, 1: 1700.3. Samples: 5091174. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-13 21:18:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:18:36,555][60935] Updated weights for policy 0, policy_version 9910 (0.0008) [2023-10-13 21:18:36,920][60935] Updated weights for policy 0, policy_version 9920 (0.0008) [2023-10-13 21:18:37,053][60934] Updated weights for policy 1, policy_version 9962 (0.0008) [2023-10-13 21:18:37,422][60934] Updated weights for policy 1, policy_version 9972 (0.0009) [2023-10-13 21:18:37,787][60934] Updated weights for policy 1, policy_version 9982 (0.0009) [2023-10-13 21:18:41,037][60935] Updated weights for policy 0, policy_version 9930 (0.0009) [2023-10-13 21:18:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 20381696. Throughput: 0: 1675.3, 1: 1707.1. Samples: 5111674. Policy #0 lag: (min: 24.0, avg: 45.1, max: 56.0) [2023-10-13 21:18:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:18:41,406][60935] Updated weights for policy 0, policy_version 9940 (0.0008) [2023-10-13 21:18:41,774][60935] Updated weights for policy 0, policy_version 9950 (0.0007) [2023-10-13 21:18:41,937][60934] Updated weights for policy 1, policy_version 9992 (0.0008) [2023-10-13 21:18:42,299][60934] Updated weights for policy 1, policy_version 10002 (0.0007) [2023-10-13 21:18:42,678][60934] Updated weights for policy 1, policy_version 10012 (0.0007) [2023-10-13 21:18:45,854][60935] Updated weights for policy 0, policy_version 9960 (0.0010) [2023-10-13 21:18:46,226][60935] Updated weights for policy 0, policy_version 9970 (0.0007) [2023-10-13 21:18:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 20447232. Throughput: 0: 1681.0, 1: 1679.5. Samples: 5120838. Policy #0 lag: (min: 24.0, avg: 45.1, max: 56.0) [2023-10-13 21:18:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:18:46,594][60935] Updated weights for policy 0, policy_version 9980 (0.0007) [2023-10-13 21:18:46,747][60934] Updated weights for policy 1, policy_version 10022 (0.0007) [2023-10-13 21:18:47,128][60934] Updated weights for policy 1, policy_version 10032 (0.0009) [2023-10-13 21:18:47,496][60934] Updated weights for policy 1, policy_version 10042 (0.0009) [2023-10-13 21:18:50,753][60935] Updated weights for policy 0, policy_version 9990 (0.0010) [2023-10-13 21:18:51,121][60935] Updated weights for policy 0, policy_version 10000 (0.0010) [2023-10-13 21:18:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.5). Total num frames: 20512768. Throughput: 0: 1683.2, 1: 1702.0. Samples: 5141748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:18:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:18:51,493][60935] Updated weights for policy 0, policy_version 10010 (0.0010) [2023-10-13 21:18:51,636][60934] Updated weights for policy 1, policy_version 10052 (0.0007) [2023-10-13 21:18:51,996][60934] Updated weights for policy 1, policy_version 10062 (0.0007) [2023-10-13 21:18:52,374][60934] Updated weights for policy 1, policy_version 10072 (0.0008) [2023-10-13 21:18:55,403][60935] Updated weights for policy 0, policy_version 10020 (0.0010) [2023-10-13 21:18:55,772][60935] Updated weights for policy 0, policy_version 10030 (0.0009) [2023-10-13 21:18:56,151][60935] Updated weights for policy 0, policy_version 10040 (0.0008) [2023-10-13 21:18:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 20578304. Throughput: 0: 1672.1, 1: 1703.1. Samples: 5162018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:18:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:18:56,274][60934] Updated weights for policy 1, policy_version 10082 (0.0008) [2023-10-13 21:18:56,447][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000010048_10289152.pth... [2023-10-13 21:18:56,480][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000008480_8683520.pth [2023-10-13 21:18:56,645][60934] Updated weights for policy 1, policy_version 10092 (0.0009) [2023-10-13 21:18:57,017][60934] Updated weights for policy 1, policy_version 10102 (0.0009) [2023-10-13 21:18:57,383][60934] Updated weights for policy 1, policy_version 10112 (0.0007) [2023-10-13 21:18:57,383][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000010112_10354688.pth... [2023-10-13 21:18:57,421][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000008512_8716288.pth [2023-10-13 21:19:00,044][60935] Updated weights for policy 0, policy_version 10050 (0.0008) [2023-10-13 21:19:00,429][60935] Updated weights for policy 0, policy_version 10060 (0.0008) [2023-10-13 21:19:00,795][60935] Updated weights for policy 0, policy_version 10070 (0.0008) [2023-10-13 21:19:01,167][60935] Updated weights for policy 0, policy_version 10080 (0.0008) [2023-10-13 21:19:01,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 20676608. Throughput: 0: 1681.0, 1: 1695.2. Samples: 5171864. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-13 21:19:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:19:01,301][60934] Updated weights for policy 1, policy_version 10122 (0.0007) [2023-10-13 21:19:01,657][60934] Updated weights for policy 1, policy_version 10132 (0.0008) [2023-10-13 21:19:02,031][60934] Updated weights for policy 1, policy_version 10142 (0.0007) [2023-10-13 21:19:05,315][60935] Updated weights for policy 0, policy_version 10090 (0.0008) [2023-10-13 21:19:05,676][60935] Updated weights for policy 0, policy_version 10100 (0.0009) [2023-10-13 21:19:05,903][60934] Updated weights for policy 1, policy_version 10152 (0.0009) [2023-10-13 21:19:06,050][60935] Updated weights for policy 0, policy_version 10110 (0.0008) [2023-10-13 21:19:06,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 20742144. Throughput: 0: 1682.2, 1: 1714.6. Samples: 5193016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:19:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:19:06,273][60934] Updated weights for policy 1, policy_version 10162 (0.0009) [2023-10-13 21:19:06,646][60934] Updated weights for policy 1, policy_version 10172 (0.0008) [2023-10-13 21:19:10,029][60935] Updated weights for policy 0, policy_version 10120 (0.0012) [2023-10-13 21:19:10,406][60935] Updated weights for policy 0, policy_version 10130 (0.0010) [2023-10-13 21:19:10,696][60934] Updated weights for policy 1, policy_version 10182 (0.0008) [2023-10-13 21:19:10,769][60935] Updated weights for policy 0, policy_version 10140 (0.0007) [2023-10-13 21:19:11,067][60934] Updated weights for policy 1, policy_version 10192 (0.0007) [2023-10-13 21:19:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 20807680. Throughput: 0: 1656.4, 1: 1715.3. Samples: 5212698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:19:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:19:11,434][60934] Updated weights for policy 1, policy_version 10202 (0.0008) [2023-10-13 21:19:14,873][60935] Updated weights for policy 0, policy_version 10150 (0.0009) [2023-10-13 21:19:15,238][60935] Updated weights for policy 0, policy_version 10160 (0.0011) [2023-10-13 21:19:15,403][60934] Updated weights for policy 1, policy_version 10212 (0.0007) [2023-10-13 21:19:15,601][60935] Updated weights for policy 0, policy_version 10170 (0.0007) [2023-10-13 21:19:15,780][60934] Updated weights for policy 1, policy_version 10222 (0.0007) [2023-10-13 21:19:16,157][60934] Updated weights for policy 1, policy_version 10232 (0.0008) [2023-10-13 21:19:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 20873216. Throughput: 0: 1682.4, 1: 1721.2. Samples: 5223082. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-13 21:19:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:19:19,806][60935] Updated weights for policy 0, policy_version 10180 (0.0008) [2023-10-13 21:19:20,088][60934] Updated weights for policy 1, policy_version 10242 (0.0007) [2023-10-13 21:19:20,180][60935] Updated weights for policy 0, policy_version 10190 (0.0009) [2023-10-13 21:19:20,463][60934] Updated weights for policy 1, policy_version 10252 (0.0010) [2023-10-13 21:19:20,546][60935] Updated weights for policy 0, policy_version 10200 (0.0010) [2023-10-13 21:19:20,829][60934] Updated weights for policy 1, policy_version 10262 (0.0008) [2023-10-13 21:19:21,190][60934] Updated weights for policy 1, policy_version 10272 (0.0010) [2023-10-13 21:19:21,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 20971520. Throughput: 0: 1672.3, 1: 1715.9. Samples: 5243644. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-13 21:19:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:19:24,644][60935] Updated weights for policy 0, policy_version 10210 (0.0008) [2023-10-13 21:19:25,008][60935] Updated weights for policy 0, policy_version 10220 (0.0007) [2023-10-13 21:19:25,301][60934] Updated weights for policy 1, policy_version 10282 (0.0009) [2023-10-13 21:19:25,386][60935] Updated weights for policy 0, policy_version 10230 (0.0008) [2023-10-13 21:19:25,669][60934] Updated weights for policy 1, policy_version 10292 (0.0007) [2023-10-13 21:19:25,748][60935] Updated weights for policy 0, policy_version 10240 (0.0008) [2023-10-13 21:19:26,036][60934] Updated weights for policy 1, policy_version 10302 (0.0008) [2023-10-13 21:19:26,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 21037056. Throughput: 0: 1656.7, 1: 1700.0. Samples: 5262726. Policy #0 lag: (min: 5.0, avg: 8.8, max: 37.0) [2023-10-13 21:19:26,250][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:19:29,955][60935] Updated weights for policy 0, policy_version 10250 (0.0010) [2023-10-13 21:19:30,184][60934] Updated weights for policy 1, policy_version 10312 (0.0007) [2023-10-13 21:19:30,326][60935] Updated weights for policy 0, policy_version 10260 (0.0008) [2023-10-13 21:19:30,552][60934] Updated weights for policy 1, policy_version 10322 (0.0008) [2023-10-13 21:19:30,701][60935] Updated weights for policy 0, policy_version 10270 (0.0008) [2023-10-13 21:19:30,912][60934] Updated weights for policy 1, policy_version 10332 (0.0008) [2023-10-13 21:19:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 21102592. Throughput: 0: 1684.7, 1: 1716.8. Samples: 5273906. Policy #0 lag: (min: 5.0, avg: 8.8, max: 37.0) [2023-10-13 21:19:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:19:34,696][60935] Updated weights for policy 0, policy_version 10280 (0.0010) [2023-10-13 21:19:34,930][60934] Updated weights for policy 1, policy_version 10342 (0.0007) [2023-10-13 21:19:35,075][60935] Updated weights for policy 0, policy_version 10290 (0.0007) [2023-10-13 21:19:35,300][60934] Updated weights for policy 1, policy_version 10352 (0.0007) [2023-10-13 21:19:35,449][60935] Updated weights for policy 0, policy_version 10300 (0.0009) [2023-10-13 21:19:35,674][60934] Updated weights for policy 1, policy_version 10362 (0.0008) [2023-10-13 21:19:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 21168128. Throughput: 0: 1666.6, 1: 1719.5. Samples: 5294126. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 21:19:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.900')] [2023-10-13 21:19:39,573][60935] Updated weights for policy 0, policy_version 10310 (0.0008) [2023-10-13 21:19:39,820][60934] Updated weights for policy 1, policy_version 10372 (0.0009) [2023-10-13 21:19:39,949][60935] Updated weights for policy 0, policy_version 10320 (0.0009) [2023-10-13 21:19:40,186][60934] Updated weights for policy 1, policy_version 10382 (0.0008) [2023-10-13 21:19:40,324][60935] Updated weights for policy 0, policy_version 10330 (0.0009) [2023-10-13 21:19:40,557][60934] Updated weights for policy 1, policy_version 10392 (0.0009) [2023-10-13 21:19:41,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 21233664. Throughput: 0: 1660.9, 1: 1697.1. Samples: 5313128. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 21:19:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.900')] [2023-10-13 21:19:44,451][60935] Updated weights for policy 0, policy_version 10340 (0.0008) [2023-10-13 21:19:44,593][60934] Updated weights for policy 1, policy_version 10402 (0.0007) [2023-10-13 21:19:44,825][60935] Updated weights for policy 0, policy_version 10350 (0.0009) [2023-10-13 21:19:44,957][60934] Updated weights for policy 1, policy_version 10412 (0.0010) [2023-10-13 21:19:45,204][60935] Updated weights for policy 0, policy_version 10360 (0.0009) [2023-10-13 21:19:45,318][60934] Updated weights for policy 1, policy_version 10422 (0.0007) [2023-10-13 21:19:45,695][60934] Updated weights for policy 1, policy_version 10432 (0.0007) [2023-10-13 21:19:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 21299200. Throughput: 0: 1678.9, 1: 1711.6. Samples: 5324436. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-13 21:19:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.900')] [2023-10-13 21:19:49,303][60935] Updated weights for policy 0, policy_version 10370 (0.0008) [2023-10-13 21:19:49,663][60935] Updated weights for policy 0, policy_version 10380 (0.0009) [2023-10-13 21:19:49,773][60934] Updated weights for policy 1, policy_version 10442 (0.0008) [2023-10-13 21:19:50,039][60935] Updated weights for policy 0, policy_version 10390 (0.0010) [2023-10-13 21:19:50,141][60934] Updated weights for policy 1, policy_version 10452 (0.0008) [2023-10-13 21:19:50,395][60935] Updated weights for policy 0, policy_version 10400 (0.0010) [2023-10-13 21:19:50,503][60934] Updated weights for policy 1, policy_version 10462 (0.0008) [2023-10-13 21:19:51,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 21364736. Throughput: 0: 1658.0, 1: 1703.0. Samples: 5344260. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-13 21:19:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.900')] [2023-10-13 21:19:54,372][60935] Updated weights for policy 0, policy_version 10410 (0.0007) [2023-10-13 21:19:54,531][60934] Updated weights for policy 1, policy_version 10472 (0.0008) [2023-10-13 21:19:54,740][60935] Updated weights for policy 0, policy_version 10420 (0.0008) [2023-10-13 21:19:54,901][60934] Updated weights for policy 1, policy_version 10482 (0.0008) [2023-10-13 21:19:55,118][60935] Updated weights for policy 0, policy_version 10430 (0.0008) [2023-10-13 21:19:55,267][60934] Updated weights for policy 1, policy_version 10492 (0.0008) [2023-10-13 21:19:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 21430272. Throughput: 0: 1673.0, 1: 1674.5. Samples: 5363336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:19:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.900')] [2023-10-13 21:19:59,174][60935] Updated weights for policy 0, policy_version 10440 (0.0008) [2023-10-13 21:19:59,200][60934] Updated weights for policy 1, policy_version 10502 (0.0007) [2023-10-13 21:19:59,530][60935] Updated weights for policy 0, policy_version 10450 (0.0009) [2023-10-13 21:19:59,573][60934] Updated weights for policy 1, policy_version 10512 (0.0008) [2023-10-13 21:19:59,908][60935] Updated weights for policy 0, policy_version 10460 (0.0007) [2023-10-13 21:19:59,946][60934] Updated weights for policy 1, policy_version 10522 (0.0007) [2023-10-13 21:20:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 21495808. Throughput: 0: 1677.1, 1: 1702.5. Samples: 5375164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:20:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.900')] [2023-10-13 21:20:03,873][60934] Updated weights for policy 1, policy_version 10532 (0.0010) [2023-10-13 21:20:04,068][60935] Updated weights for policy 0, policy_version 10470 (0.0009) [2023-10-13 21:20:04,248][60934] Updated weights for policy 1, policy_version 10542 (0.0007) [2023-10-13 21:20:04,441][60935] Updated weights for policy 0, policy_version 10480 (0.0008) [2023-10-13 21:20:04,610][60934] Updated weights for policy 1, policy_version 10552 (0.0008) [2023-10-13 21:20:04,809][60935] Updated weights for policy 0, policy_version 10490 (0.0008) [2023-10-13 21:20:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 21561344. Throughput: 0: 1660.8, 1: 1684.0. Samples: 5394160. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 21:20:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.900')] [2023-10-13 21:20:08,674][60934] Updated weights for policy 1, policy_version 10562 (0.0009) [2023-10-13 21:20:08,930][60935] Updated weights for policy 0, policy_version 10500 (0.0010) [2023-10-13 21:20:09,081][60934] Updated weights for policy 1, policy_version 10572 (0.0007) [2023-10-13 21:20:09,299][60935] Updated weights for policy 0, policy_version 10510 (0.0009) [2023-10-13 21:20:09,452][60934] Updated weights for policy 1, policy_version 10582 (0.0008) [2023-10-13 21:20:09,669][60935] Updated weights for policy 0, policy_version 10520 (0.0008) [2023-10-13 21:20:09,814][60934] Updated weights for policy 1, policy_version 10592 (0.0009) [2023-10-13 21:20:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 21626880. Throughput: 0: 1674.8, 1: 1687.1. Samples: 5414008. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 21:20:11,250][59943] Avg episode reward: [(0, '0.000'), (1, '-0.900')] [2023-10-13 21:20:13,748][60935] Updated weights for policy 0, policy_version 10530 (0.0009) [2023-10-13 21:20:13,867][60934] Updated weights for policy 1, policy_version 10602 (0.0008) [2023-10-13 21:20:14,123][60935] Updated weights for policy 0, policy_version 10540 (0.0008) [2023-10-13 21:20:14,235][60934] Updated weights for policy 1, policy_version 10612 (0.0008) [2023-10-13 21:20:14,484][60935] Updated weights for policy 0, policy_version 10550 (0.0008) [2023-10-13 21:20:14,598][60934] Updated weights for policy 1, policy_version 10622 (0.0008) [2023-10-13 21:20:14,863][60935] Updated weights for policy 0, policy_version 10560 (0.0011) [2023-10-13 21:20:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 21692416. Throughput: 0: 1666.3, 1: 1697.0. Samples: 5425254. Policy #0 lag: (min: 25.0, avg: 32.8, max: 57.0) [2023-10-13 21:20:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.890')] [2023-10-13 21:20:18,440][60934] Updated weights for policy 1, policy_version 10632 (0.0008) [2023-10-13 21:20:18,815][60934] Updated weights for policy 1, policy_version 10642 (0.0008) [2023-10-13 21:20:19,133][60935] Updated weights for policy 0, policy_version 10570 (0.0009) [2023-10-13 21:20:19,182][60934] Updated weights for policy 1, policy_version 10652 (0.0007) [2023-10-13 21:20:19,498][60935] Updated weights for policy 0, policy_version 10580 (0.0007) [2023-10-13 21:20:19,871][60935] Updated weights for policy 0, policy_version 10590 (0.0007) [2023-10-13 21:20:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 21757952. Throughput: 0: 1654.4, 1: 1670.4. Samples: 5443746. Policy #0 lag: (min: 25.0, avg: 32.8, max: 57.0) [2023-10-13 21:20:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:20:23,218][60934] Updated weights for policy 1, policy_version 10662 (0.0008) [2023-10-13 21:20:23,593][60934] Updated weights for policy 1, policy_version 10672 (0.0007) [2023-10-13 21:20:23,814][60935] Updated weights for policy 0, policy_version 10600 (0.0008) [2023-10-13 21:20:23,952][60934] Updated weights for policy 1, policy_version 10682 (0.0008) [2023-10-13 21:20:24,189][60935] Updated weights for policy 0, policy_version 10610 (0.0008) [2023-10-13 21:20:24,557][60935] Updated weights for policy 0, policy_version 10620 (0.0011) [2023-10-13 21:20:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 21823488. Throughput: 0: 1672.4, 1: 1697.0. Samples: 5464752. Policy #0 lag: (min: 30.0, avg: 37.9, max: 62.0) [2023-10-13 21:20:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:20:27,941][60934] Updated weights for policy 1, policy_version 10692 (0.0009) [2023-10-13 21:20:28,320][60934] Updated weights for policy 1, policy_version 10702 (0.0010) [2023-10-13 21:20:28,592][60935] Updated weights for policy 0, policy_version 10630 (0.0009) [2023-10-13 21:20:28,693][60934] Updated weights for policy 1, policy_version 10712 (0.0008) [2023-10-13 21:20:28,967][60935] Updated weights for policy 0, policy_version 10640 (0.0008) [2023-10-13 21:20:29,336][60935] Updated weights for policy 0, policy_version 10650 (0.0009) [2023-10-13 21:20:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 21889024. Throughput: 0: 1659.8, 1: 1687.7. Samples: 5475074. Policy #0 lag: (min: 30.0, avg: 37.9, max: 62.0) [2023-10-13 21:20:31,249][59943] Avg episode reward: [(0, '-0.070'), (1, '0.000')] [2023-10-13 21:20:32,748][60934] Updated weights for policy 1, policy_version 10722 (0.0008) [2023-10-13 21:20:33,123][60934] Updated weights for policy 1, policy_version 10732 (0.0009) [2023-10-13 21:20:33,502][60934] Updated weights for policy 1, policy_version 10742 (0.0008) [2023-10-13 21:20:33,525][60935] Updated weights for policy 0, policy_version 10660 (0.0009) [2023-10-13 21:20:33,861][60934] Updated weights for policy 1, policy_version 10752 (0.0008) [2023-10-13 21:20:33,902][60935] Updated weights for policy 0, policy_version 10670 (0.0007) [2023-10-13 21:20:34,268][60935] Updated weights for policy 0, policy_version 10680 (0.0009) [2023-10-13 21:20:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 21954560. Throughput: 0: 1658.0, 1: 1679.0. Samples: 5494426. Policy #0 lag: (min: 27.0, avg: 32.4, max: 59.0) [2023-10-13 21:20:36,249][59943] Avg episode reward: [(0, '-0.070'), (1, '0.000')] [2023-10-13 21:20:37,835][60934] Updated weights for policy 1, policy_version 10762 (0.0007) [2023-10-13 21:20:38,212][60934] Updated weights for policy 1, policy_version 10772 (0.0007) [2023-10-13 21:20:38,454][60935] Updated weights for policy 0, policy_version 10690 (0.0009) [2023-10-13 21:20:38,570][60934] Updated weights for policy 1, policy_version 10782 (0.0009) [2023-10-13 21:20:38,813][60935] Updated weights for policy 0, policy_version 10700 (0.0009) [2023-10-13 21:20:39,191][60935] Updated weights for policy 0, policy_version 10710 (0.0011) [2023-10-13 21:20:39,557][60935] Updated weights for policy 0, policy_version 10720 (0.0009) [2023-10-13 21:20:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 22020096. Throughput: 0: 1664.0, 1: 1708.7. Samples: 5515110. Policy #0 lag: (min: 27.0, avg: 32.4, max: 59.0) [2023-10-13 21:20:41,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:20:42,523][60934] Updated weights for policy 1, policy_version 10792 (0.0008) [2023-10-13 21:20:42,892][60934] Updated weights for policy 1, policy_version 10802 (0.0009) [2023-10-13 21:20:43,259][60934] Updated weights for policy 1, policy_version 10812 (0.0010) [2023-10-13 21:20:43,849][60935] Updated weights for policy 0, policy_version 10730 (0.0007) [2023-10-13 21:20:44,216][60935] Updated weights for policy 0, policy_version 10740 (0.0010) [2023-10-13 21:20:44,585][60935] Updated weights for policy 0, policy_version 10750 (0.0011) [2023-10-13 21:20:46,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 22085632. Throughput: 0: 1651.4, 1: 1677.6. Samples: 5524968. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-13 21:20:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:20:47,187][60934] Updated weights for policy 1, policy_version 10822 (0.0008) [2023-10-13 21:20:47,554][60934] Updated weights for policy 1, policy_version 10832 (0.0009) [2023-10-13 21:20:47,924][60934] Updated weights for policy 1, policy_version 10842 (0.0008) [2023-10-13 21:20:48,593][60935] Updated weights for policy 0, policy_version 10760 (0.0009) [2023-10-13 21:20:48,968][60935] Updated weights for policy 0, policy_version 10770 (0.0010) [2023-10-13 21:20:49,344][60935] Updated weights for policy 0, policy_version 10780 (0.0009) [2023-10-13 21:20:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 22151168. Throughput: 0: 1652.9, 1: 1702.6. Samples: 5545158. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-13 21:20:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:20:51,875][60934] Updated weights for policy 1, policy_version 10852 (0.0009) [2023-10-13 21:20:52,250][60934] Updated weights for policy 1, policy_version 10862 (0.0009) [2023-10-13 21:20:52,614][60934] Updated weights for policy 1, policy_version 10872 (0.0007) [2023-10-13 21:20:53,512][60935] Updated weights for policy 0, policy_version 10790 (0.0009) [2023-10-13 21:20:53,873][60935] Updated weights for policy 0, policy_version 10800 (0.0009) [2023-10-13 21:20:54,253][60935] Updated weights for policy 0, policy_version 10810 (0.0011) [2023-10-13 21:20:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 22216704. Throughput: 0: 1661.1, 1: 1714.4. Samples: 5565906. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-13 21:20:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:20:56,258][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000010816_11075584.pth... [2023-10-13 21:20:56,258][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000010880_11141120.pth... [2023-10-13 21:20:56,298][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000009248_9469952.pth [2023-10-13 21:20:56,299][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000009312_9535488.pth [2023-10-13 21:20:56,668][60934] Updated weights for policy 1, policy_version 10882 (0.0010) [2023-10-13 21:20:57,027][60934] Updated weights for policy 1, policy_version 10892 (0.0008) [2023-10-13 21:20:57,403][60934] Updated weights for policy 1, policy_version 10902 (0.0009) [2023-10-13 21:20:57,757][60934] Updated weights for policy 1, policy_version 10912 (0.0010) [2023-10-13 21:20:58,375][60935] Updated weights for policy 0, policy_version 10820 (0.0009) [2023-10-13 21:20:58,744][60935] Updated weights for policy 0, policy_version 10830 (0.0007) [2023-10-13 21:20:59,113][60935] Updated weights for policy 0, policy_version 10840 (0.0007) [2023-10-13 21:21:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 22282240. Throughput: 0: 1653.4, 1: 1688.0. Samples: 5575618. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-13 21:21:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:21:01,741][60934] Updated weights for policy 1, policy_version 10922 (0.0010) [2023-10-13 21:21:02,112][60934] Updated weights for policy 1, policy_version 10932 (0.0009) [2023-10-13 21:21:02,476][60934] Updated weights for policy 1, policy_version 10942 (0.0008) [2023-10-13 21:21:03,220][60935] Updated weights for policy 0, policy_version 10850 (0.0007) [2023-10-13 21:21:03,591][60935] Updated weights for policy 0, policy_version 10860 (0.0009) [2023-10-13 21:21:03,961][60935] Updated weights for policy 0, policy_version 10870 (0.0007) [2023-10-13 21:21:04,332][60935] Updated weights for policy 0, policy_version 10880 (0.0008) [2023-10-13 21:21:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 22347776. Throughput: 0: 1662.5, 1: 1716.8. Samples: 5595812. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-13 21:21:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:21:06,530][60934] Updated weights for policy 1, policy_version 10952 (0.0008) [2023-10-13 21:21:06,908][60934] Updated weights for policy 1, policy_version 10962 (0.0009) [2023-10-13 21:21:07,270][60934] Updated weights for policy 1, policy_version 10972 (0.0008) [2023-10-13 21:21:08,529][60935] Updated weights for policy 0, policy_version 10890 (0.0009) [2023-10-13 21:21:08,904][60935] Updated weights for policy 0, policy_version 10900 (0.0009) [2023-10-13 21:21:09,281][60935] Updated weights for policy 0, policy_version 10910 (0.0010) [2023-10-13 21:21:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 22413312. Throughput: 0: 1657.6, 1: 1709.5. Samples: 5616270. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-13 21:21:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:21:11,307][60934] Updated weights for policy 1, policy_version 10982 (0.0007) [2023-10-13 21:21:11,676][60934] Updated weights for policy 1, policy_version 10992 (0.0007) [2023-10-13 21:21:12,041][60934] Updated weights for policy 1, policy_version 11002 (0.0009) [2023-10-13 21:21:13,606][60935] Updated weights for policy 0, policy_version 10920 (0.0008) [2023-10-13 21:21:13,989][60935] Updated weights for policy 0, policy_version 10930 (0.0007) [2023-10-13 21:21:14,357][60935] Updated weights for policy 0, policy_version 10940 (0.0008) [2023-10-13 21:21:16,139][60934] Updated weights for policy 1, policy_version 11012 (0.0008) [2023-10-13 21:21:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 22478848. Throughput: 0: 1656.7, 1: 1700.8. Samples: 5626162. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-13 21:21:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:21:16,503][60934] Updated weights for policy 1, policy_version 11022 (0.0009) [2023-10-13 21:21:16,881][60934] Updated weights for policy 1, policy_version 11032 (0.0011) [2023-10-13 21:21:18,363][60935] Updated weights for policy 0, policy_version 10950 (0.0008) [2023-10-13 21:21:18,731][60935] Updated weights for policy 0, policy_version 10960 (0.0008) [2023-10-13 21:21:19,098][60935] Updated weights for policy 0, policy_version 10970 (0.0008) [2023-10-13 21:21:20,898][60934] Updated weights for policy 1, policy_version 11042 (0.0010) [2023-10-13 21:21:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 22544384. Throughput: 0: 1660.0, 1: 1709.9. Samples: 5646072. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-13 21:21:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:21:21,269][60934] Updated weights for policy 1, policy_version 11052 (0.0009) [2023-10-13 21:21:21,644][60934] Updated weights for policy 1, policy_version 11062 (0.0009) [2023-10-13 21:21:22,012][60934] Updated weights for policy 1, policy_version 11072 (0.0009) [2023-10-13 21:21:23,055][60935] Updated weights for policy 0, policy_version 10980 (0.0007) [2023-10-13 21:21:23,418][60935] Updated weights for policy 0, policy_version 10990 (0.0009) [2023-10-13 21:21:23,787][60935] Updated weights for policy 0, policy_version 11000 (0.0009) [2023-10-13 21:21:26,162][60934] Updated weights for policy 1, policy_version 11082 (0.0007) [2023-10-13 21:21:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 22609920. Throughput: 0: 1666.0, 1: 1703.3. Samples: 5666728. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-13 21:21:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:21:26,526][60934] Updated weights for policy 1, policy_version 11092 (0.0007) [2023-10-13 21:21:26,893][60934] Updated weights for policy 1, policy_version 11102 (0.0007) [2023-10-13 21:21:27,707][60935] Updated weights for policy 0, policy_version 11010 (0.0009) [2023-10-13 21:21:28,084][60935] Updated weights for policy 0, policy_version 11020 (0.0009) [2023-10-13 21:21:28,455][60935] Updated weights for policy 0, policy_version 11030 (0.0008) [2023-10-13 21:21:28,826][60935] Updated weights for policy 0, policy_version 11040 (0.0009) [2023-10-13 21:21:30,822][60934] Updated weights for policy 1, policy_version 11112 (0.0010) [2023-10-13 21:21:31,186][60934] Updated weights for policy 1, policy_version 11122 (0.0010) [2023-10-13 21:21:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 22675456. Throughput: 0: 1652.8, 1: 1703.1. Samples: 5675982. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-13 21:21:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:21:31,556][60934] Updated weights for policy 1, policy_version 11132 (0.0007) [2023-10-13 21:21:32,863][60935] Updated weights for policy 0, policy_version 11050 (0.0007) [2023-10-13 21:21:33,235][60935] Updated weights for policy 0, policy_version 11060 (0.0010) [2023-10-13 21:21:33,605][60935] Updated weights for policy 0, policy_version 11070 (0.0010) [2023-10-13 21:21:35,646][60934] Updated weights for policy 1, policy_version 11142 (0.0009) [2023-10-13 21:21:36,015][60934] Updated weights for policy 1, policy_version 11152 (0.0008) [2023-10-13 21:21:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 22740992. Throughput: 0: 1673.3, 1: 1693.7. Samples: 5696674. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-10-13 21:21:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:21:36,385][60934] Updated weights for policy 1, policy_version 11162 (0.0008) [2023-10-13 21:21:37,728][60935] Updated weights for policy 0, policy_version 11080 (0.0009) [2023-10-13 21:21:38,097][60935] Updated weights for policy 0, policy_version 11090 (0.0009) [2023-10-13 21:21:38,471][60935] Updated weights for policy 0, policy_version 11100 (0.0008) [2023-10-13 21:21:40,465][60934] Updated weights for policy 1, policy_version 11172 (0.0009) [2023-10-13 21:21:40,842][60934] Updated weights for policy 1, policy_version 11182 (0.0009) [2023-10-13 21:21:41,218][60934] Updated weights for policy 1, policy_version 11192 (0.0008) [2023-10-13 21:21:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 22806528. Throughput: 0: 1672.6, 1: 1692.6. Samples: 5717340. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-10-13 21:21:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:21:42,572][60935] Updated weights for policy 0, policy_version 11110 (0.0008) [2023-10-13 21:21:42,945][60935] Updated weights for policy 0, policy_version 11120 (0.0007) [2023-10-13 21:21:43,325][60935] Updated weights for policy 0, policy_version 11130 (0.0008) [2023-10-13 21:21:45,168][60934] Updated weights for policy 1, policy_version 11202 (0.0007) [2023-10-13 21:21:45,590][60934] Updated weights for policy 1, policy_version 11212 (0.0009) [2023-10-13 21:21:45,960][60934] Updated weights for policy 1, policy_version 11222 (0.0010) [2023-10-13 21:21:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 22872064. Throughput: 0: 1658.0, 1: 1701.4. Samples: 5726792. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 21:21:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:21:46,316][60934] Updated weights for policy 1, policy_version 11232 (0.0011) [2023-10-13 21:21:47,229][60935] Updated weights for policy 0, policy_version 11140 (0.0008) [2023-10-13 21:21:47,598][60935] Updated weights for policy 0, policy_version 11150 (0.0008) [2023-10-13 21:21:47,978][60935] Updated weights for policy 0, policy_version 11160 (0.0007) [2023-10-13 21:21:50,365][60934] Updated weights for policy 1, policy_version 11242 (0.0007) [2023-10-13 21:21:50,726][60934] Updated weights for policy 1, policy_version 11252 (0.0008) [2023-10-13 21:21:51,101][60934] Updated weights for policy 1, policy_version 11262 (0.0008) [2023-10-13 21:21:51,248][59943] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 22970368. Throughput: 0: 1674.1, 1: 1691.7. Samples: 5747276. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 21:21:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:21:52,129][60935] Updated weights for policy 0, policy_version 11170 (0.0009) [2023-10-13 21:21:52,490][60935] Updated weights for policy 0, policy_version 11180 (0.0010) [2023-10-13 21:21:52,860][60935] Updated weights for policy 0, policy_version 11190 (0.0009) [2023-10-13 21:21:53,230][60935] Updated weights for policy 0, policy_version 11200 (0.0011) [2023-10-13 21:21:55,191][60934] Updated weights for policy 1, policy_version 11272 (0.0008) [2023-10-13 21:21:55,555][60934] Updated weights for policy 1, policy_version 11282 (0.0011) [2023-10-13 21:21:55,921][60934] Updated weights for policy 1, policy_version 11292 (0.0008) [2023-10-13 21:21:56,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 23035904. Throughput: 0: 1681.7, 1: 1681.8. Samples: 5767630. Policy #0 lag: (min: 10.0, avg: 10.4, max: 24.0) [2023-10-13 21:21:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:21:57,369][60935] Updated weights for policy 0, policy_version 11210 (0.0009) [2023-10-13 21:21:57,738][60935] Updated weights for policy 0, policy_version 11220 (0.0010) [2023-10-13 21:21:58,117][60935] Updated weights for policy 0, policy_version 11230 (0.0008) [2023-10-13 21:21:59,891][60934] Updated weights for policy 1, policy_version 11302 (0.0010) [2023-10-13 21:22:00,253][60934] Updated weights for policy 1, policy_version 11312 (0.0011) [2023-10-13 21:22:00,625][60934] Updated weights for policy 1, policy_version 11322 (0.0008) [2023-10-13 21:22:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 23101440. Throughput: 0: 1662.8, 1: 1696.4. Samples: 5777326. Policy #0 lag: (min: 10.0, avg: 10.4, max: 24.0) [2023-10-13 21:22:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:22:02,225][60935] Updated weights for policy 0, policy_version 11240 (0.0011) [2023-10-13 21:22:02,588][60935] Updated weights for policy 0, policy_version 11250 (0.0010) [2023-10-13 21:22:02,963][60935] Updated weights for policy 0, policy_version 11260 (0.0011) [2023-10-13 21:22:04,704][60934] Updated weights for policy 1, policy_version 11332 (0.0009) [2023-10-13 21:22:05,065][60934] Updated weights for policy 1, policy_version 11342 (0.0009) [2023-10-13 21:22:05,438][60934] Updated weights for policy 1, policy_version 11352 (0.0008) [2023-10-13 21:22:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 23166976. Throughput: 0: 1681.6, 1: 1691.9. Samples: 5797880. Policy #0 lag: (min: 28.0, avg: 33.4, max: 60.0) [2023-10-13 21:22:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:22:07,019][60935] Updated weights for policy 0, policy_version 11270 (0.0007) [2023-10-13 21:22:07,380][60935] Updated weights for policy 0, policy_version 11280 (0.0011) [2023-10-13 21:22:07,754][60935] Updated weights for policy 0, policy_version 11290 (0.0011) [2023-10-13 21:22:09,390][60934] Updated weights for policy 1, policy_version 11362 (0.0008) [2023-10-13 21:22:09,757][60934] Updated weights for policy 1, policy_version 11372 (0.0007) [2023-10-13 21:22:10,138][60934] Updated weights for policy 1, policy_version 11382 (0.0009) [2023-10-13 21:22:10,501][60934] Updated weights for policy 1, policy_version 11392 (0.0010) [2023-10-13 21:22:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 23232512. Throughput: 0: 1686.0, 1: 1668.9. Samples: 5817698. Policy #0 lag: (min: 28.0, avg: 33.4, max: 60.0) [2023-10-13 21:22:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:22:11,843][60935] Updated weights for policy 0, policy_version 11300 (0.0009) [2023-10-13 21:22:12,206][60935] Updated weights for policy 0, policy_version 11310 (0.0009) [2023-10-13 21:22:12,588][60935] Updated weights for policy 0, policy_version 11320 (0.0007) [2023-10-13 21:22:14,438][60934] Updated weights for policy 1, policy_version 11402 (0.0010) [2023-10-13 21:22:14,797][60934] Updated weights for policy 1, policy_version 11412 (0.0008) [2023-10-13 21:22:15,170][60934] Updated weights for policy 1, policy_version 11422 (0.0009) [2023-10-13 21:22:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 23298048. Throughput: 0: 1678.6, 1: 1705.3. Samples: 5828258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:22:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:22:16,637][60935] Updated weights for policy 0, policy_version 11330 (0.0009) [2023-10-13 21:22:17,007][60935] Updated weights for policy 0, policy_version 11340 (0.0008) [2023-10-13 21:22:17,367][60935] Updated weights for policy 0, policy_version 11350 (0.0010) [2023-10-13 21:22:17,741][60935] Updated weights for policy 0, policy_version 11360 (0.0011) [2023-10-13 21:22:19,407][60934] Updated weights for policy 1, policy_version 11432 (0.0007) [2023-10-13 21:22:19,772][60934] Updated weights for policy 1, policy_version 11442 (0.0007) [2023-10-13 21:22:20,140][60934] Updated weights for policy 1, policy_version 11452 (0.0008) [2023-10-13 21:22:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 23363584. Throughput: 0: 1682.9, 1: 1693.5. Samples: 5848612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:22:21,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:22:21,718][60935] Updated weights for policy 0, policy_version 11370 (0.0012) [2023-10-13 21:22:22,084][60935] Updated weights for policy 0, policy_version 11380 (0.0008) [2023-10-13 21:22:22,465][60935] Updated weights for policy 0, policy_version 11390 (0.0008) [2023-10-13 21:22:24,127][60934] Updated weights for policy 1, policy_version 11462 (0.0009) [2023-10-13 21:22:24,506][60934] Updated weights for policy 1, policy_version 11472 (0.0010) [2023-10-13 21:22:24,878][60934] Updated weights for policy 1, policy_version 11482 (0.0011) [2023-10-13 21:22:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 23429120. Throughput: 0: 1691.6, 1: 1676.9. Samples: 5868924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:22:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:22:26,445][60935] Updated weights for policy 0, policy_version 11400 (0.0008) [2023-10-13 21:22:26,816][60935] Updated weights for policy 0, policy_version 11410 (0.0007) [2023-10-13 21:22:27,184][60935] Updated weights for policy 0, policy_version 11420 (0.0008) [2023-10-13 21:22:28,884][60934] Updated weights for policy 1, policy_version 11492 (0.0010) [2023-10-13 21:22:29,256][60934] Updated weights for policy 1, policy_version 11502 (0.0008) [2023-10-13 21:22:29,628][60934] Updated weights for policy 1, policy_version 11512 (0.0008) [2023-10-13 21:22:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 23494656. Throughput: 0: 1691.0, 1: 1696.5. Samples: 5879232. Policy #0 lag: (min: 24.0, avg: 44.0, max: 56.0) [2023-10-13 21:22:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:22:31,349][60935] Updated weights for policy 0, policy_version 11430 (0.0011) [2023-10-13 21:22:31,721][60935] Updated weights for policy 0, policy_version 11440 (0.0010) [2023-10-13 21:22:32,099][60935] Updated weights for policy 0, policy_version 11450 (0.0009) [2023-10-13 21:22:33,601][60934] Updated weights for policy 1, policy_version 11522 (0.0008) [2023-10-13 21:22:33,972][60934] Updated weights for policy 1, policy_version 11532 (0.0009) [2023-10-13 21:22:34,348][60934] Updated weights for policy 1, policy_version 11542 (0.0009) [2023-10-13 21:22:34,710][60934] Updated weights for policy 1, policy_version 11552 (0.0007) [2023-10-13 21:22:36,082][60935] Updated weights for policy 0, policy_version 11460 (0.0009) [2023-10-13 21:22:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 23560192. Throughput: 0: 1696.3, 1: 1678.9. Samples: 5899160. Policy #0 lag: (min: 24.0, avg: 44.0, max: 56.0) [2023-10-13 21:22:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:22:36,454][60935] Updated weights for policy 0, policy_version 11470 (0.0011) [2023-10-13 21:22:36,817][60935] Updated weights for policy 0, policy_version 11480 (0.0009) [2023-10-13 21:22:38,935][60934] Updated weights for policy 1, policy_version 11562 (0.0007) [2023-10-13 21:22:39,316][60934] Updated weights for policy 1, policy_version 11572 (0.0007) [2023-10-13 21:22:39,683][60934] Updated weights for policy 1, policy_version 11582 (0.0008) [2023-10-13 21:22:40,886][60935] Updated weights for policy 0, policy_version 11490 (0.0010) [2023-10-13 21:22:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 23625728. Throughput: 0: 1689.8, 1: 1681.0. Samples: 5919314. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-13 21:22:41,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:22:41,260][60935] Updated weights for policy 0, policy_version 11500 (0.0009) [2023-10-13 21:22:41,628][60935] Updated weights for policy 0, policy_version 11510 (0.0007) [2023-10-13 21:22:42,004][60935] Updated weights for policy 0, policy_version 11520 (0.0008) [2023-10-13 21:22:43,598][60934] Updated weights for policy 1, policy_version 11592 (0.0007) [2023-10-13 21:22:43,972][60934] Updated weights for policy 1, policy_version 11602 (0.0007) [2023-10-13 21:22:44,329][60934] Updated weights for policy 1, policy_version 11612 (0.0010) [2023-10-13 21:22:46,249][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 23691264. Throughput: 0: 1691.5, 1: 1688.5. Samples: 5929428. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-13 21:22:46,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:22:46,276][60935] Updated weights for policy 0, policy_version 11530 (0.0007) [2023-10-13 21:22:46,651][60935] Updated weights for policy 0, policy_version 11540 (0.0009) [2023-10-13 21:22:47,021][60935] Updated weights for policy 0, policy_version 11550 (0.0008) [2023-10-13 21:22:48,364][60934] Updated weights for policy 1, policy_version 11622 (0.0008) [2023-10-13 21:22:48,727][60934] Updated weights for policy 1, policy_version 11632 (0.0011) [2023-10-13 21:22:49,087][60934] Updated weights for policy 1, policy_version 11642 (0.0010) [2023-10-13 21:22:51,111][60935] Updated weights for policy 0, policy_version 11560 (0.0007) [2023-10-13 21:22:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 23756800. Throughput: 0: 1688.4, 1: 1670.0. Samples: 5949008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:22:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:22:51,479][60935] Updated weights for policy 0, policy_version 11570 (0.0007) [2023-10-13 21:22:51,853][60935] Updated weights for policy 0, policy_version 11580 (0.0009) [2023-10-13 21:22:53,144][60934] Updated weights for policy 1, policy_version 11652 (0.0010) [2023-10-13 21:22:53,518][60934] Updated weights for policy 1, policy_version 11662 (0.0007) [2023-10-13 21:22:53,886][60934] Updated weights for policy 1, policy_version 11672 (0.0008) [2023-10-13 21:22:55,838][60935] Updated weights for policy 0, policy_version 11590 (0.0010) [2023-10-13 21:22:56,210][60935] Updated weights for policy 0, policy_version 11600 (0.0007) [2023-10-13 21:22:56,248][59943] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 23822336. Throughput: 0: 1678.8, 1: 1696.5. Samples: 5969586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:22:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:22:56,258][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000011680_11960320.pth... [2023-10-13 21:22:56,293][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000010112_10354688.pth [2023-10-13 21:22:56,586][60935] Updated weights for policy 0, policy_version 11610 (0.0009) [2023-10-13 21:22:56,796][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000011616_11894784.pth... [2023-10-13 21:22:56,827][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000010048_10289152.pth [2023-10-13 21:22:57,811][60934] Updated weights for policy 1, policy_version 11682 (0.0008) [2023-10-13 21:22:58,177][60934] Updated weights for policy 1, policy_version 11692 (0.0009) [2023-10-13 21:22:58,544][60934] Updated weights for policy 1, policy_version 11702 (0.0009) [2023-10-13 21:22:58,912][60934] Updated weights for policy 1, policy_version 11712 (0.0009) [2023-10-13 21:23:00,731][60935] Updated weights for policy 0, policy_version 11620 (0.0010) [2023-10-13 21:23:01,107][60935] Updated weights for policy 0, policy_version 11630 (0.0009) [2023-10-13 21:23:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 23887872. Throughput: 0: 1688.4, 1: 1673.6. Samples: 5979548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:23:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:23:01,475][60935] Updated weights for policy 0, policy_version 11640 (0.0011) [2023-10-13 21:23:02,827][60934] Updated weights for policy 1, policy_version 11722 (0.0008) [2023-10-13 21:23:03,206][60934] Updated weights for policy 1, policy_version 11732 (0.0011) [2023-10-13 21:23:03,560][60934] Updated weights for policy 1, policy_version 11742 (0.0011) [2023-10-13 21:23:05,615][60935] Updated weights for policy 0, policy_version 11650 (0.0009) [2023-10-13 21:23:05,994][60935] Updated weights for policy 0, policy_version 11660 (0.0007) [2023-10-13 21:23:06,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 23953408. Throughput: 0: 1682.7, 1: 1680.5. Samples: 5999958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:23:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:23:06,371][60935] Updated weights for policy 0, policy_version 11670 (0.0011) [2023-10-13 21:23:06,737][60935] Updated weights for policy 0, policy_version 11680 (0.0010) [2023-10-13 21:23:07,716][60934] Updated weights for policy 1, policy_version 11752 (0.0009) [2023-10-13 21:23:08,086][60934] Updated weights for policy 1, policy_version 11762 (0.0010) [2023-10-13 21:23:08,452][60934] Updated weights for policy 1, policy_version 11772 (0.0011) [2023-10-13 21:23:10,820][60935] Updated weights for policy 0, policy_version 11690 (0.0009) [2023-10-13 21:23:11,186][60935] Updated weights for policy 0, policy_version 11700 (0.0008) [2023-10-13 21:23:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 24018944. Throughput: 0: 1664.2, 1: 1699.0. Samples: 6020266. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:23:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:23:11,568][60935] Updated weights for policy 0, policy_version 11710 (0.0008) [2023-10-13 21:23:12,535][60934] Updated weights for policy 1, policy_version 11782 (0.0010) [2023-10-13 21:23:12,905][60934] Updated weights for policy 1, policy_version 11792 (0.0011) [2023-10-13 21:23:13,288][60934] Updated weights for policy 1, policy_version 11802 (0.0010) [2023-10-13 21:23:15,637][60935] Updated weights for policy 0, policy_version 11720 (0.0012) [2023-10-13 21:23:16,018][60935] Updated weights for policy 0, policy_version 11730 (0.0007) [2023-10-13 21:23:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 24084480. Throughput: 0: 1673.6, 1: 1674.9. Samples: 6029916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:23:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:23:16,391][60935] Updated weights for policy 0, policy_version 11740 (0.0007) [2023-10-13 21:23:17,371][60934] Updated weights for policy 1, policy_version 11812 (0.0008) [2023-10-13 21:23:17,743][60934] Updated weights for policy 1, policy_version 11822 (0.0009) [2023-10-13 21:23:18,112][60934] Updated weights for policy 1, policy_version 11832 (0.0008) [2023-10-13 21:23:20,400][60935] Updated weights for policy 0, policy_version 11750 (0.0008) [2023-10-13 21:23:20,766][60935] Updated weights for policy 0, policy_version 11760 (0.0010) [2023-10-13 21:23:21,138][60935] Updated weights for policy 0, policy_version 11770 (0.0009) [2023-10-13 21:23:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 24150016. Throughput: 0: 1666.6, 1: 1695.1. Samples: 6050434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:23:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:23:22,129][60934] Updated weights for policy 1, policy_version 11842 (0.0009) [2023-10-13 21:23:22,503][60934] Updated weights for policy 1, policy_version 11852 (0.0007) [2023-10-13 21:23:22,876][60934] Updated weights for policy 1, policy_version 11862 (0.0007) [2023-10-13 21:23:23,244][60934] Updated weights for policy 1, policy_version 11872 (0.0007) [2023-10-13 21:23:25,329][60935] Updated weights for policy 0, policy_version 11780 (0.0011) [2023-10-13 21:23:25,700][60935] Updated weights for policy 0, policy_version 11790 (0.0008) [2023-10-13 21:23:26,072][60935] Updated weights for policy 0, policy_version 11800 (0.0009) [2023-10-13 21:23:26,249][59943] Fps is (10 sec: 13106.6, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 24215552. Throughput: 0: 1656.1, 1: 1707.1. Samples: 6070660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:23:26,250][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 21:23:27,258][60934] Updated weights for policy 1, policy_version 11882 (0.0010) [2023-10-13 21:23:27,641][60934] Updated weights for policy 1, policy_version 11892 (0.0010) [2023-10-13 21:23:28,010][60934] Updated weights for policy 1, policy_version 11902 (0.0010) [2023-10-13 21:23:29,990][60935] Updated weights for policy 0, policy_version 11810 (0.0009) [2023-10-13 21:23:30,396][60935] Updated weights for policy 0, policy_version 11820 (0.0009) [2023-10-13 21:23:30,754][60935] Updated weights for policy 0, policy_version 11830 (0.0008) [2023-10-13 21:23:31,123][60935] Updated weights for policy 0, policy_version 11840 (0.0010) [2023-10-13 21:23:31,248][59943] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 24313856. Throughput: 0: 1670.2, 1: 1683.5. Samples: 6080342. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-13 21:23:31,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 21:23:31,941][60934] Updated weights for policy 1, policy_version 11912 (0.0010) [2023-10-13 21:23:32,312][60934] Updated weights for policy 1, policy_version 11922 (0.0007) [2023-10-13 21:23:32,675][60934] Updated weights for policy 1, policy_version 11932 (0.0007) [2023-10-13 21:23:35,296][60935] Updated weights for policy 0, policy_version 11850 (0.0009) [2023-10-13 21:23:35,677][60935] Updated weights for policy 0, policy_version 11860 (0.0009) [2023-10-13 21:23:36,041][60935] Updated weights for policy 0, policy_version 11870 (0.0008) [2023-10-13 21:23:36,248][59943] Fps is (10 sec: 16384.6, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 24379392. Throughput: 0: 1669.3, 1: 1716.7. Samples: 6101378. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-13 21:23:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:23:36,624][60934] Updated weights for policy 1, policy_version 11942 (0.0009) [2023-10-13 21:23:36,999][60934] Updated weights for policy 1, policy_version 11952 (0.0008) [2023-10-13 21:23:37,358][60934] Updated weights for policy 1, policy_version 11962 (0.0008) [2023-10-13 21:23:40,047][60935] Updated weights for policy 0, policy_version 11880 (0.0008) [2023-10-13 21:23:40,416][60935] Updated weights for policy 0, policy_version 11890 (0.0011) [2023-10-13 21:23:40,799][60935] Updated weights for policy 0, policy_version 11900 (0.0011) [2023-10-13 21:23:41,221][60934] Updated weights for policy 1, policy_version 11972 (0.0009) [2023-10-13 21:23:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 24444928. Throughput: 0: 1653.3, 1: 1717.6. Samples: 6121276. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-13 21:23:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:23:41,602][60934] Updated weights for policy 1, policy_version 11982 (0.0008) [2023-10-13 21:23:41,968][60934] Updated weights for policy 1, policy_version 11992 (0.0009) [2023-10-13 21:23:45,041][60935] Updated weights for policy 0, policy_version 11910 (0.0010) [2023-10-13 21:23:45,424][60935] Updated weights for policy 0, policy_version 11920 (0.0010) [2023-10-13 21:23:45,798][60935] Updated weights for policy 0, policy_version 11930 (0.0010) [2023-10-13 21:23:46,066][60934] Updated weights for policy 1, policy_version 12002 (0.0009) [2023-10-13 21:23:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 24510464. Throughput: 0: 1670.6, 1: 1700.4. Samples: 6131244. Policy #0 lag: (min: 1.0, avg: 7.7, max: 33.0) [2023-10-13 21:23:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:23:46,439][60934] Updated weights for policy 1, policy_version 12012 (0.0007) [2023-10-13 21:23:46,810][60934] Updated weights for policy 1, policy_version 12022 (0.0007) [2023-10-13 21:23:47,178][60934] Updated weights for policy 1, policy_version 12032 (0.0008) [2023-10-13 21:23:49,801][60935] Updated weights for policy 0, policy_version 11940 (0.0008) [2023-10-13 21:23:50,179][60935] Updated weights for policy 0, policy_version 11950 (0.0009) [2023-10-13 21:23:50,543][60935] Updated weights for policy 0, policy_version 11960 (0.0007) [2023-10-13 21:23:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 24576000. Throughput: 0: 1667.3, 1: 1705.2. Samples: 6151720. Policy #0 lag: (min: 1.0, avg: 7.7, max: 33.0) [2023-10-13 21:23:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-13 21:23:51,267][60934] Updated weights for policy 1, policy_version 12042 (0.0010) [2023-10-13 21:23:51,637][60934] Updated weights for policy 1, policy_version 12052 (0.0009) [2023-10-13 21:23:52,008][60934] Updated weights for policy 1, policy_version 12062 (0.0009) [2023-10-13 21:23:54,530][60935] Updated weights for policy 0, policy_version 11970 (0.0010) [2023-10-13 21:23:54,897][60935] Updated weights for policy 0, policy_version 11980 (0.0010) [2023-10-13 21:23:55,283][60935] Updated weights for policy 0, policy_version 11990 (0.0007) [2023-10-13 21:23:55,654][60935] Updated weights for policy 0, policy_version 12000 (0.0007) [2023-10-13 21:23:56,061][60934] Updated weights for policy 1, policy_version 12072 (0.0008) [2023-10-13 21:23:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 24641536. Throughput: 0: 1655.9, 1: 1704.8. Samples: 6171498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:23:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-13 21:23:56,435][60934] Updated weights for policy 1, policy_version 12082 (0.0008) [2023-10-13 21:23:56,799][60934] Updated weights for policy 1, policy_version 12092 (0.0008) [2023-10-13 21:23:59,716][60935] Updated weights for policy 0, policy_version 12010 (0.0009) [2023-10-13 21:24:00,100][60935] Updated weights for policy 0, policy_version 12020 (0.0007) [2023-10-13 21:24:00,470][60935] Updated weights for policy 0, policy_version 12030 (0.0008) [2023-10-13 21:24:00,840][60934] Updated weights for policy 1, policy_version 12102 (0.0008) [2023-10-13 21:24:01,213][60934] Updated weights for policy 1, policy_version 12112 (0.0008) [2023-10-13 21:24:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 24707072. Throughput: 0: 1675.1, 1: 1703.4. Samples: 6181946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:24:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-13 21:24:01,584][60934] Updated weights for policy 1, policy_version 12122 (0.0007) [2023-10-13 21:24:04,444][60935] Updated weights for policy 0, policy_version 12040 (0.0008) [2023-10-13 21:24:04,824][60935] Updated weights for policy 0, policy_version 12050 (0.0009) [2023-10-13 21:24:05,192][60935] Updated weights for policy 0, policy_version 12060 (0.0007) [2023-10-13 21:24:05,762][60934] Updated weights for policy 1, policy_version 12132 (0.0008) [2023-10-13 21:24:06,139][60934] Updated weights for policy 1, policy_version 12142 (0.0007) [2023-10-13 21:24:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 24772608. Throughput: 0: 1662.8, 1: 1706.4. Samples: 6202050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:24:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-13 21:24:06,504][60934] Updated weights for policy 1, policy_version 12152 (0.0009) [2023-10-13 21:24:09,210][60935] Updated weights for policy 0, policy_version 12070 (0.0008) [2023-10-13 21:24:09,579][60935] Updated weights for policy 0, policy_version 12080 (0.0008) [2023-10-13 21:24:09,949][60935] Updated weights for policy 0, policy_version 12090 (0.0010) [2023-10-13 21:24:10,443][60934] Updated weights for policy 1, policy_version 12162 (0.0008) [2023-10-13 21:24:10,806][60934] Updated weights for policy 1, policy_version 12172 (0.0009) [2023-10-13 21:24:11,178][60934] Updated weights for policy 1, policy_version 12182 (0.0009) [2023-10-13 21:24:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 24838144. Throughput: 0: 1664.3, 1: 1702.6. Samples: 6222168. Policy #0 lag: (min: 22.0, avg: 23.5, max: 48.0) [2023-10-13 21:24:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-13 21:24:11,548][60934] Updated weights for policy 1, policy_version 12192 (0.0007) [2023-10-13 21:24:14,178][60935] Updated weights for policy 0, policy_version 12100 (0.0008) [2023-10-13 21:24:14,551][60935] Updated weights for policy 0, policy_version 12110 (0.0009) [2023-10-13 21:24:14,912][60935] Updated weights for policy 0, policy_version 12120 (0.0008) [2023-10-13 21:24:15,656][60934] Updated weights for policy 1, policy_version 12202 (0.0010) [2023-10-13 21:24:16,039][60934] Updated weights for policy 1, policy_version 12212 (0.0007) [2023-10-13 21:24:16,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 24903680. Throughput: 0: 1679.6, 1: 1705.1. Samples: 6232654. Policy #0 lag: (min: 22.0, avg: 23.5, max: 48.0) [2023-10-13 21:24:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-13 21:24:16,416][60934] Updated weights for policy 1, policy_version 12222 (0.0009) [2023-10-13 21:24:19,069][60935] Updated weights for policy 0, policy_version 12130 (0.0010) [2023-10-13 21:24:19,473][60935] Updated weights for policy 0, policy_version 12140 (0.0008) [2023-10-13 21:24:19,839][60935] Updated weights for policy 0, policy_version 12150 (0.0009) [2023-10-13 21:24:20,206][60935] Updated weights for policy 0, policy_version 12160 (0.0009) [2023-10-13 21:24:20,398][60934] Updated weights for policy 1, policy_version 12232 (0.0010) [2023-10-13 21:24:20,767][60934] Updated weights for policy 1, policy_version 12242 (0.0010) [2023-10-13 21:24:21,135][60934] Updated weights for policy 1, policy_version 12252 (0.0010) [2023-10-13 21:24:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 24969216. Throughput: 0: 1659.2, 1: 1693.8. Samples: 6252262. Policy #0 lag: (min: 22.0, avg: 24.9, max: 54.0) [2023-10-13 21:24:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:24:24,434][60935] Updated weights for policy 0, policy_version 12170 (0.0009) [2023-10-13 21:24:24,802][60935] Updated weights for policy 0, policy_version 12180 (0.0008) [2023-10-13 21:24:25,032][60934] Updated weights for policy 1, policy_version 12262 (0.0008) [2023-10-13 21:24:25,172][60935] Updated weights for policy 0, policy_version 12190 (0.0007) [2023-10-13 21:24:25,398][60934] Updated weights for policy 1, policy_version 12272 (0.0010) [2023-10-13 21:24:25,770][60934] Updated weights for policy 1, policy_version 12282 (0.0009) [2023-10-13 21:24:26,248][59943] Fps is (10 sec: 16383.6, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 25067520. Throughput: 0: 1665.8, 1: 1680.7. Samples: 6271866. Policy #0 lag: (min: 22.0, avg: 24.9, max: 54.0) [2023-10-13 21:24:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:24:29,425][60935] Updated weights for policy 0, policy_version 12200 (0.0008) [2023-10-13 21:24:29,803][60935] Updated weights for policy 0, policy_version 12210 (0.0007) [2023-10-13 21:24:29,803][60934] Updated weights for policy 1, policy_version 12292 (0.0008) [2023-10-13 21:24:30,170][60934] Updated weights for policy 1, policy_version 12302 (0.0008) [2023-10-13 21:24:30,171][60935] Updated weights for policy 0, policy_version 12220 (0.0010) [2023-10-13 21:24:30,546][60934] Updated weights for policy 1, policy_version 12312 (0.0010) [2023-10-13 21:24:31,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 25133056. Throughput: 0: 1670.6, 1: 1699.9. Samples: 6282918. Policy #0 lag: (min: 5.0, avg: 10.1, max: 37.0) [2023-10-13 21:24:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:24:34,086][60935] Updated weights for policy 0, policy_version 12230 (0.0009) [2023-10-13 21:24:34,381][60934] Updated weights for policy 1, policy_version 12322 (0.0008) [2023-10-13 21:24:34,461][60935] Updated weights for policy 0, policy_version 12240 (0.0008) [2023-10-13 21:24:34,758][60934] Updated weights for policy 1, policy_version 12332 (0.0009) [2023-10-13 21:24:34,830][60935] Updated weights for policy 0, policy_version 12250 (0.0008) [2023-10-13 21:24:35,121][60934] Updated weights for policy 1, policy_version 12342 (0.0008) [2023-10-13 21:24:35,489][60934] Updated weights for policy 1, policy_version 12352 (0.0009) [2023-10-13 21:24:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 25198592. Throughput: 0: 1654.0, 1: 1701.8. Samples: 6302734. Policy #0 lag: (min: 5.0, avg: 10.1, max: 37.0) [2023-10-13 21:24:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:24:38,950][60935] Updated weights for policy 0, policy_version 12260 (0.0007) [2023-10-13 21:24:39,327][60935] Updated weights for policy 0, policy_version 12270 (0.0007) [2023-10-13 21:24:39,530][60934] Updated weights for policy 1, policy_version 12362 (0.0008) [2023-10-13 21:24:39,693][60935] Updated weights for policy 0, policy_version 12280 (0.0007) [2023-10-13 21:24:39,897][60934] Updated weights for policy 1, policy_version 12372 (0.0008) [2023-10-13 21:24:40,260][60934] Updated weights for policy 1, policy_version 12382 (0.0008) [2023-10-13 21:24:41,249][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 25264128. Throughput: 0: 1672.0, 1: 1678.4. Samples: 6322268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:24:41,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:24:43,724][60935] Updated weights for policy 0, policy_version 12290 (0.0009) [2023-10-13 21:24:44,093][60935] Updated weights for policy 0, policy_version 12300 (0.0009) [2023-10-13 21:24:44,434][60934] Updated weights for policy 1, policy_version 12392 (0.0008) [2023-10-13 21:24:44,467][60935] Updated weights for policy 0, policy_version 12310 (0.0009) [2023-10-13 21:24:44,808][60934] Updated weights for policy 1, policy_version 12402 (0.0008) [2023-10-13 21:24:44,831][60935] Updated weights for policy 0, policy_version 12320 (0.0008) [2023-10-13 21:24:45,167][60934] Updated weights for policy 1, policy_version 12412 (0.0010) [2023-10-13 21:24:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 25329664. Throughput: 0: 1667.2, 1: 1705.5. Samples: 6333714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:24:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:24:49,038][60935] Updated weights for policy 0, policy_version 12330 (0.0008) [2023-10-13 21:24:49,191][60934] Updated weights for policy 1, policy_version 12422 (0.0009) [2023-10-13 21:24:49,405][60935] Updated weights for policy 0, policy_version 12340 (0.0007) [2023-10-13 21:24:49,561][60934] Updated weights for policy 1, policy_version 12432 (0.0009) [2023-10-13 21:24:49,775][60935] Updated weights for policy 0, policy_version 12350 (0.0007) [2023-10-13 21:24:49,929][60934] Updated weights for policy 1, policy_version 12442 (0.0009) [2023-10-13 21:24:51,248][59943] Fps is (10 sec: 13107.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 25395200. Throughput: 0: 1654.9, 1: 1692.2. Samples: 6352666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:24:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:24:53,890][60935] Updated weights for policy 0, policy_version 12360 (0.0009) [2023-10-13 21:24:54,025][60934] Updated weights for policy 1, policy_version 12452 (0.0007) [2023-10-13 21:24:54,253][60935] Updated weights for policy 0, policy_version 12370 (0.0008) [2023-10-13 21:24:54,395][60934] Updated weights for policy 1, policy_version 12462 (0.0007) [2023-10-13 21:24:54,625][60935] Updated weights for policy 0, policy_version 12380 (0.0008) [2023-10-13 21:24:54,761][60934] Updated weights for policy 1, policy_version 12472 (0.0009) [2023-10-13 21:24:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 25460736. Throughput: 0: 1666.5, 1: 1674.1. Samples: 6372494. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-13 21:24:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:24:56,260][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000012384_12681216.pth... [2023-10-13 21:24:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000012480_12779520.pth... [2023-10-13 21:24:56,290][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000010816_11075584.pth [2023-10-13 21:24:56,300][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000010880_11141120.pth [2023-10-13 21:24:58,645][60934] Updated weights for policy 1, policy_version 12482 (0.0009) [2023-10-13 21:24:58,722][60935] Updated weights for policy 0, policy_version 12390 (0.0009) [2023-10-13 21:24:59,019][60934] Updated weights for policy 1, policy_version 12492 (0.0007) [2023-10-13 21:24:59,088][60935] Updated weights for policy 0, policy_version 12400 (0.0008) [2023-10-13 21:24:59,387][60934] Updated weights for policy 1, policy_version 12502 (0.0008) [2023-10-13 21:24:59,464][60935] Updated weights for policy 0, policy_version 12410 (0.0007) [2023-10-13 21:24:59,747][60934] Updated weights for policy 1, policy_version 12512 (0.0008) [2023-10-13 21:25:01,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 25526272. Throughput: 0: 1656.6, 1: 1705.8. Samples: 6383960. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-13 21:25:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:25:03,670][60935] Updated weights for policy 0, policy_version 12420 (0.0007) [2023-10-13 21:25:03,868][60934] Updated weights for policy 1, policy_version 12522 (0.0008) [2023-10-13 21:25:04,032][60935] Updated weights for policy 0, policy_version 12430 (0.0009) [2023-10-13 21:25:04,240][60934] Updated weights for policy 1, policy_version 12532 (0.0007) [2023-10-13 21:25:04,396][60935] Updated weights for policy 0, policy_version 12440 (0.0008) [2023-10-13 21:25:04,602][60934] Updated weights for policy 1, policy_version 12542 (0.0009) [2023-10-13 21:25:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 25591808. Throughput: 0: 1656.5, 1: 1681.9. Samples: 6402490. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-13 21:25:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:25:08,456][60935] Updated weights for policy 0, policy_version 12450 (0.0008) [2023-10-13 21:25:08,812][60934] Updated weights for policy 1, policy_version 12552 (0.0009) [2023-10-13 21:25:08,852][60935] Updated weights for policy 0, policy_version 12460 (0.0008) [2023-10-13 21:25:09,195][60934] Updated weights for policy 1, policy_version 12562 (0.0008) [2023-10-13 21:25:09,223][60935] Updated weights for policy 0, policy_version 12470 (0.0009) [2023-10-13 21:25:09,563][60934] Updated weights for policy 1, policy_version 12572 (0.0008) [2023-10-13 21:25:09,595][60935] Updated weights for policy 0, policy_version 12480 (0.0009) [2023-10-13 21:25:11,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 25657344. Throughput: 0: 1664.9, 1: 1685.0. Samples: 6422610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:25:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:25:13,482][60934] Updated weights for policy 1, policy_version 12582 (0.0009) [2023-10-13 21:25:13,748][60935] Updated weights for policy 0, policy_version 12490 (0.0007) [2023-10-13 21:25:13,858][60934] Updated weights for policy 1, policy_version 12592 (0.0007) [2023-10-13 21:25:14,116][60935] Updated weights for policy 0, policy_version 12500 (0.0007) [2023-10-13 21:25:14,220][60934] Updated weights for policy 1, policy_version 12602 (0.0007) [2023-10-13 21:25:14,492][60935] Updated weights for policy 0, policy_version 12510 (0.0009) [2023-10-13 21:25:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 25722880. Throughput: 0: 1652.0, 1: 1689.9. Samples: 6433302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:25:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:25:18,206][60934] Updated weights for policy 1, policy_version 12612 (0.0007) [2023-10-13 21:25:18,562][60934] Updated weights for policy 1, policy_version 12622 (0.0009) [2023-10-13 21:25:18,699][60935] Updated weights for policy 0, policy_version 12520 (0.0007) [2023-10-13 21:25:18,937][60934] Updated weights for policy 1, policy_version 12632 (0.0007) [2023-10-13 21:25:19,070][60935] Updated weights for policy 0, policy_version 12530 (0.0007) [2023-10-13 21:25:19,442][60935] Updated weights for policy 0, policy_version 12540 (0.0007) [2023-10-13 21:25:21,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 25788416. Throughput: 0: 1651.3, 1: 1671.9. Samples: 6452278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:25:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:25:23,106][60934] Updated weights for policy 1, policy_version 12642 (0.0007) [2023-10-13 21:25:23,471][60934] Updated weights for policy 1, policy_version 12652 (0.0008) [2023-10-13 21:25:23,511][60935] Updated weights for policy 0, policy_version 12550 (0.0007) [2023-10-13 21:25:23,842][60934] Updated weights for policy 1, policy_version 12662 (0.0007) [2023-10-13 21:25:23,876][60935] Updated weights for policy 0, policy_version 12560 (0.0008) [2023-10-13 21:25:24,212][60934] Updated weights for policy 1, policy_version 12672 (0.0008) [2023-10-13 21:25:24,257][60935] Updated weights for policy 0, policy_version 12570 (0.0010) [2023-10-13 21:25:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 25853952. Throughput: 0: 1656.3, 1: 1689.9. Samples: 6472844. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 21:25:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:25:28,219][60935] Updated weights for policy 0, policy_version 12580 (0.0009) [2023-10-13 21:25:28,372][60934] Updated weights for policy 1, policy_version 12682 (0.0009) [2023-10-13 21:25:28,594][60935] Updated weights for policy 0, policy_version 12590 (0.0008) [2023-10-13 21:25:28,744][60934] Updated weights for policy 1, policy_version 12692 (0.0010) [2023-10-13 21:25:28,967][60935] Updated weights for policy 0, policy_version 12600 (0.0009) [2023-10-13 21:25:29,122][60934] Updated weights for policy 1, policy_version 12702 (0.0009) [2023-10-13 21:25:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 25919488. Throughput: 0: 1644.3, 1: 1675.9. Samples: 6483124. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 21:25:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:25:33,179][60935] Updated weights for policy 0, policy_version 12610 (0.0010) [2023-10-13 21:25:33,273][60934] Updated weights for policy 1, policy_version 12712 (0.0007) [2023-10-13 21:25:33,556][60935] Updated weights for policy 0, policy_version 12620 (0.0007) [2023-10-13 21:25:33,643][60934] Updated weights for policy 1, policy_version 12722 (0.0007) [2023-10-13 21:25:33,928][60935] Updated weights for policy 0, policy_version 12630 (0.0009) [2023-10-13 21:25:34,011][60934] Updated weights for policy 1, policy_version 12732 (0.0007) [2023-10-13 21:25:34,300][60935] Updated weights for policy 0, policy_version 12640 (0.0008) [2023-10-13 21:25:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 25985024. Throughput: 0: 1653.2, 1: 1672.1. Samples: 6502306. Policy #0 lag: (min: 19.0, avg: 26.4, max: 51.0) [2023-10-13 21:25:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:25:37,881][60934] Updated weights for policy 1, policy_version 12742 (0.0008) [2023-10-13 21:25:38,240][60934] Updated weights for policy 1, policy_version 12752 (0.0010) [2023-10-13 21:25:38,553][60935] Updated weights for policy 0, policy_version 12650 (0.0009) [2023-10-13 21:25:38,603][60934] Updated weights for policy 1, policy_version 12762 (0.0007) [2023-10-13 21:25:38,930][60935] Updated weights for policy 0, policy_version 12660 (0.0009) [2023-10-13 21:25:39,298][60935] Updated weights for policy 0, policy_version 12670 (0.0007) [2023-10-13 21:25:41,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 26050560. Throughput: 0: 1648.5, 1: 1698.5. Samples: 6523106. Policy #0 lag: (min: 19.0, avg: 26.4, max: 51.0) [2023-10-13 21:25:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:25:42,467][60934] Updated weights for policy 1, policy_version 12772 (0.0008) [2023-10-13 21:25:42,834][60934] Updated weights for policy 1, policy_version 12782 (0.0008) [2023-10-13 21:25:43,192][60934] Updated weights for policy 1, policy_version 12792 (0.0007) [2023-10-13 21:25:43,652][60935] Updated weights for policy 0, policy_version 12680 (0.0007) [2023-10-13 21:25:44,019][60935] Updated weights for policy 0, policy_version 12690 (0.0008) [2023-10-13 21:25:44,399][60935] Updated weights for policy 0, policy_version 12700 (0.0008) [2023-10-13 21:25:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 26116096. Throughput: 0: 1644.5, 1: 1664.4. Samples: 6532860. Policy #0 lag: (min: 24.0, avg: 45.4, max: 56.0) [2023-10-13 21:25:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:25:47,274][60934] Updated weights for policy 1, policy_version 12802 (0.0007) [2023-10-13 21:25:47,641][60934] Updated weights for policy 1, policy_version 12812 (0.0010) [2023-10-13 21:25:48,016][60934] Updated weights for policy 1, policy_version 12822 (0.0009) [2023-10-13 21:25:48,359][60935] Updated weights for policy 0, policy_version 12710 (0.0010) [2023-10-13 21:25:48,380][60934] Updated weights for policy 1, policy_version 12832 (0.0009) [2023-10-13 21:25:48,717][60935] Updated weights for policy 0, policy_version 12720 (0.0009) [2023-10-13 21:25:49,084][60935] Updated weights for policy 0, policy_version 12730 (0.0008) [2023-10-13 21:25:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 26181632. Throughput: 0: 1647.5, 1: 1698.8. Samples: 6553072. Policy #0 lag: (min: 24.0, avg: 45.4, max: 56.0) [2023-10-13 21:25:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:25:52,316][60934] Updated weights for policy 1, policy_version 12842 (0.0008) [2023-10-13 21:25:52,680][60934] Updated weights for policy 1, policy_version 12852 (0.0009) [2023-10-13 21:25:53,047][60934] Updated weights for policy 1, policy_version 12862 (0.0009) [2023-10-13 21:25:53,244][60935] Updated weights for policy 0, policy_version 12740 (0.0008) [2023-10-13 21:25:53,638][60935] Updated weights for policy 0, policy_version 12750 (0.0009) [2023-10-13 21:25:54,006][60935] Updated weights for policy 0, policy_version 12760 (0.0007) [2023-10-13 21:25:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 26247168. Throughput: 0: 1648.9, 1: 1703.2. Samples: 6573456. Policy #0 lag: (min: 24.0, avg: 45.4, max: 56.0) [2023-10-13 21:25:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:25:57,080][60934] Updated weights for policy 1, policy_version 12872 (0.0008) [2023-10-13 21:25:57,446][60934] Updated weights for policy 1, policy_version 12882 (0.0008) [2023-10-13 21:25:57,821][60934] Updated weights for policy 1, policy_version 12892 (0.0007) [2023-10-13 21:25:57,940][60935] Updated weights for policy 0, policy_version 12770 (0.0009) [2023-10-13 21:25:58,315][60935] Updated weights for policy 0, policy_version 12780 (0.0011) [2023-10-13 21:25:58,685][60935] Updated weights for policy 0, policy_version 12790 (0.0009) [2023-10-13 21:25:59,059][60935] Updated weights for policy 0, policy_version 12800 (0.0007) [2023-10-13 21:26:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 26312704. Throughput: 0: 1643.1, 1: 1682.7. Samples: 6582962. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-13 21:26:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:26:01,718][60934] Updated weights for policy 1, policy_version 12902 (0.0007) [2023-10-13 21:26:02,088][60934] Updated weights for policy 1, policy_version 12912 (0.0009) [2023-10-13 21:26:02,452][60934] Updated weights for policy 1, policy_version 12922 (0.0008) [2023-10-13 21:26:03,298][60935] Updated weights for policy 0, policy_version 12810 (0.0007) [2023-10-13 21:26:03,673][60935] Updated weights for policy 0, policy_version 12820 (0.0007) [2023-10-13 21:26:04,051][60935] Updated weights for policy 0, policy_version 12830 (0.0008) [2023-10-13 21:26:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 26378240. Throughput: 0: 1655.6, 1: 1706.7. Samples: 6603578. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-13 21:26:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:26:06,287][60934] Updated weights for policy 1, policy_version 12932 (0.0007) [2023-10-13 21:26:06,661][60934] Updated weights for policy 1, policy_version 12942 (0.0007) [2023-10-13 21:26:07,032][60934] Updated weights for policy 1, policy_version 12952 (0.0007) [2023-10-13 21:26:08,153][60935] Updated weights for policy 0, policy_version 12840 (0.0007) [2023-10-13 21:26:08,531][60935] Updated weights for policy 0, policy_version 12850 (0.0008) [2023-10-13 21:26:08,904][60935] Updated weights for policy 0, policy_version 12860 (0.0008) [2023-10-13 21:26:11,132][60934] Updated weights for policy 1, policy_version 12962 (0.0008) [2023-10-13 21:26:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 26443776. Throughput: 0: 1652.4, 1: 1718.9. Samples: 6624554. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-13 21:26:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:26:11,515][60934] Updated weights for policy 1, policy_version 12972 (0.0011) [2023-10-13 21:26:11,877][60934] Updated weights for policy 1, policy_version 12982 (0.0010) [2023-10-13 21:26:12,250][60934] Updated weights for policy 1, policy_version 12992 (0.0008) [2023-10-13 21:26:13,063][60935] Updated weights for policy 0, policy_version 12870 (0.0007) [2023-10-13 21:26:13,437][60935] Updated weights for policy 0, policy_version 12880 (0.0007) [2023-10-13 21:26:13,810][60935] Updated weights for policy 0, policy_version 12890 (0.0010) [2023-10-13 21:26:16,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 26509312. Throughput: 0: 1646.3, 1: 1704.4. Samples: 6633904. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-13 21:26:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:26:16,313][60934] Updated weights for policy 1, policy_version 13002 (0.0008) [2023-10-13 21:26:16,683][60934] Updated weights for policy 1, policy_version 13012 (0.0007) [2023-10-13 21:26:17,046][60934] Updated weights for policy 1, policy_version 13022 (0.0009) [2023-10-13 21:26:17,932][60935] Updated weights for policy 0, policy_version 12900 (0.0008) [2023-10-13 21:26:18,307][60935] Updated weights for policy 0, policy_version 12910 (0.0010) [2023-10-13 21:26:18,669][60935] Updated weights for policy 0, policy_version 12920 (0.0009) [2023-10-13 21:26:21,096][60934] Updated weights for policy 1, policy_version 13032 (0.0007) [2023-10-13 21:26:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 26574848. Throughput: 0: 1653.1, 1: 1723.6. Samples: 6654254. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-13 21:26:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:26:21,462][60934] Updated weights for policy 1, policy_version 13042 (0.0007) [2023-10-13 21:26:21,834][60934] Updated weights for policy 1, policy_version 13052 (0.0008) [2023-10-13 21:26:22,678][60935] Updated weights for policy 0, policy_version 12930 (0.0009) [2023-10-13 21:26:23,043][60935] Updated weights for policy 0, policy_version 12940 (0.0009) [2023-10-13 21:26:23,424][60935] Updated weights for policy 0, policy_version 12950 (0.0010) [2023-10-13 21:26:23,785][60935] Updated weights for policy 0, policy_version 12960 (0.0010) [2023-10-13 21:26:25,850][60934] Updated weights for policy 1, policy_version 13062 (0.0009) [2023-10-13 21:26:26,221][60934] Updated weights for policy 1, policy_version 13072 (0.0008) [2023-10-13 21:26:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 26640384. Throughput: 0: 1656.3, 1: 1714.0. Samples: 6674772. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-13 21:26:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:26:26,586][60934] Updated weights for policy 1, policy_version 13082 (0.0007) [2023-10-13 21:26:28,039][60935] Updated weights for policy 0, policy_version 12970 (0.0009) [2023-10-13 21:26:28,416][60935] Updated weights for policy 0, policy_version 12980 (0.0011) [2023-10-13 21:26:28,778][60935] Updated weights for policy 0, policy_version 12990 (0.0007) [2023-10-13 21:26:30,631][60934] Updated weights for policy 1, policy_version 13092 (0.0008) [2023-10-13 21:26:30,989][60934] Updated weights for policy 1, policy_version 13102 (0.0009) [2023-10-13 21:26:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 26705920. Throughput: 0: 1643.1, 1: 1715.4. Samples: 6683990. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-13 21:26:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:26:31,352][60934] Updated weights for policy 1, policy_version 13112 (0.0009) [2023-10-13 21:26:32,905][60935] Updated weights for policy 0, policy_version 13000 (0.0008) [2023-10-13 21:26:33,273][60935] Updated weights for policy 0, policy_version 13010 (0.0007) [2023-10-13 21:26:33,642][60935] Updated weights for policy 0, policy_version 13020 (0.0007) [2023-10-13 21:26:35,349][60934] Updated weights for policy 1, policy_version 13122 (0.0011) [2023-10-13 21:26:35,714][60934] Updated weights for policy 1, policy_version 13132 (0.0010) [2023-10-13 21:26:36,086][60934] Updated weights for policy 1, policy_version 13142 (0.0009) [2023-10-13 21:26:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 26771456. Throughput: 0: 1662.6, 1: 1706.5. Samples: 6704682. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-13 21:26:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:26:36,454][60934] Updated weights for policy 1, policy_version 13152 (0.0010) [2023-10-13 21:26:37,762][60935] Updated weights for policy 0, policy_version 13030 (0.0009) [2023-10-13 21:26:38,133][60935] Updated weights for policy 0, policy_version 13040 (0.0012) [2023-10-13 21:26:38,506][60935] Updated weights for policy 0, policy_version 13050 (0.0008) [2023-10-13 21:26:40,439][60934] Updated weights for policy 1, policy_version 13162 (0.0008) [2023-10-13 21:26:40,817][60934] Updated weights for policy 1, policy_version 13172 (0.0009) [2023-10-13 21:26:41,182][60934] Updated weights for policy 1, policy_version 13182 (0.0008) [2023-10-13 21:26:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 26836992. Throughput: 0: 1667.7, 1: 1702.0. Samples: 6725094. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-13 21:26:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:26:42,628][60935] Updated weights for policy 0, policy_version 13060 (0.0009) [2023-10-13 21:26:43,020][60935] Updated weights for policy 0, policy_version 13070 (0.0007) [2023-10-13 21:26:43,382][60935] Updated weights for policy 0, policy_version 13080 (0.0008) [2023-10-13 21:26:45,239][60934] Updated weights for policy 1, policy_version 13192 (0.0009) [2023-10-13 21:26:45,610][60934] Updated weights for policy 1, policy_version 13202 (0.0008) [2023-10-13 21:26:45,983][60934] Updated weights for policy 1, policy_version 13212 (0.0007) [2023-10-13 21:26:46,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 26935296. Throughput: 0: 1657.0, 1: 1719.2. Samples: 6734888. Policy #0 lag: (min: 5.0, avg: 12.0, max: 37.0) [2023-10-13 21:26:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:26:47,253][60935] Updated weights for policy 0, policy_version 13090 (0.0007) [2023-10-13 21:26:47,625][60935] Updated weights for policy 0, policy_version 13100 (0.0008) [2023-10-13 21:26:47,986][60935] Updated weights for policy 0, policy_version 13110 (0.0008) [2023-10-13 21:26:48,362][60935] Updated weights for policy 0, policy_version 13120 (0.0010) [2023-10-13 21:26:49,897][60934] Updated weights for policy 1, policy_version 13222 (0.0007) [2023-10-13 21:26:50,264][60934] Updated weights for policy 1, policy_version 13232 (0.0008) [2023-10-13 21:26:50,641][60934] Updated weights for policy 1, policy_version 13242 (0.0008) [2023-10-13 21:26:51,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 27000832. Throughput: 0: 1668.7, 1: 1712.3. Samples: 6755720. Policy #0 lag: (min: 5.0, avg: 12.0, max: 37.0) [2023-10-13 21:26:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:26:52,574][60935] Updated weights for policy 0, policy_version 13130 (0.0007) [2023-10-13 21:26:52,947][60935] Updated weights for policy 0, policy_version 13140 (0.0010) [2023-10-13 21:26:53,330][60935] Updated weights for policy 0, policy_version 13150 (0.0009) [2023-10-13 21:26:54,702][60934] Updated weights for policy 1, policy_version 13252 (0.0008) [2023-10-13 21:26:55,068][60934] Updated weights for policy 1, policy_version 13262 (0.0009) [2023-10-13 21:26:55,440][60934] Updated weights for policy 1, policy_version 13272 (0.0009) [2023-10-13 21:26:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 27066368. Throughput: 0: 1670.9, 1: 1683.1. Samples: 6775484. Policy #0 lag: (min: 5.0, avg: 12.0, max: 37.0) [2023-10-13 21:26:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:26:56,260][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000013152_13467648.pth... [2023-10-13 21:26:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000013280_13598720.pth... [2023-10-13 21:26:56,300][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000011680_11960320.pth [2023-10-13 21:26:56,301][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000011616_11894784.pth [2023-10-13 21:26:57,404][60935] Updated weights for policy 0, policy_version 13160 (0.0010) [2023-10-13 21:26:57,786][60935] Updated weights for policy 0, policy_version 13170 (0.0010) [2023-10-13 21:26:58,154][60935] Updated weights for policy 0, policy_version 13180 (0.0007) [2023-10-13 21:26:59,404][60934] Updated weights for policy 1, policy_version 13282 (0.0010) [2023-10-13 21:26:59,784][60934] Updated weights for policy 1, policy_version 13292 (0.0008) [2023-10-13 21:27:00,156][60934] Updated weights for policy 1, policy_version 13302 (0.0007) [2023-10-13 21:27:00,524][60934] Updated weights for policy 1, policy_version 13312 (0.0009) [2023-10-13 21:27:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 27131904. Throughput: 0: 1664.4, 1: 1703.8. Samples: 6785476. Policy #0 lag: (min: 9.0, avg: 23.9, max: 41.0) [2023-10-13 21:27:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:27:02,192][60935] Updated weights for policy 0, policy_version 13190 (0.0008) [2023-10-13 21:27:02,561][60935] Updated weights for policy 0, policy_version 13200 (0.0007) [2023-10-13 21:27:02,936][60935] Updated weights for policy 0, policy_version 13210 (0.0007) [2023-10-13 21:27:04,448][60934] Updated weights for policy 1, policy_version 13322 (0.0010) [2023-10-13 21:27:04,810][60934] Updated weights for policy 1, policy_version 13332 (0.0010) [2023-10-13 21:27:05,189][60934] Updated weights for policy 1, policy_version 13342 (0.0009) [2023-10-13 21:27:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 27197440. Throughput: 0: 1677.9, 1: 1692.9. Samples: 6805940. Policy #0 lag: (min: 9.0, avg: 23.9, max: 41.0) [2023-10-13 21:27:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:27:06,970][60935] Updated weights for policy 0, policy_version 13220 (0.0008) [2023-10-13 21:27:07,343][60935] Updated weights for policy 0, policy_version 13230 (0.0009) [2023-10-13 21:27:07,721][60935] Updated weights for policy 0, policy_version 13240 (0.0011) [2023-10-13 21:27:09,143][60934] Updated weights for policy 1, policy_version 13352 (0.0010) [2023-10-13 21:27:09,516][60934] Updated weights for policy 1, policy_version 13362 (0.0007) [2023-10-13 21:27:09,883][60934] Updated weights for policy 1, policy_version 13372 (0.0007) [2023-10-13 21:27:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 27262976. Throughput: 0: 1680.3, 1: 1676.7. Samples: 6825836. Policy #0 lag: (min: 9.0, avg: 23.9, max: 41.0) [2023-10-13 21:27:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:27:11,878][60935] Updated weights for policy 0, policy_version 13250 (0.0010) [2023-10-13 21:27:12,242][60935] Updated weights for policy 0, policy_version 13260 (0.0012) [2023-10-13 21:27:12,607][60935] Updated weights for policy 0, policy_version 13270 (0.0009) [2023-10-13 21:27:12,973][60935] Updated weights for policy 0, policy_version 13280 (0.0008) [2023-10-13 21:27:13,997][60934] Updated weights for policy 1, policy_version 13382 (0.0008) [2023-10-13 21:27:14,359][60934] Updated weights for policy 1, policy_version 13392 (0.0008) [2023-10-13 21:27:14,739][60934] Updated weights for policy 1, policy_version 13402 (0.0010) [2023-10-13 21:27:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 27328512. Throughput: 0: 1677.6, 1: 1706.3. Samples: 6836266. Policy #0 lag: (min: 11.0, avg: 34.5, max: 40.0) [2023-10-13 21:27:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:27:17,191][60935] Updated weights for policy 0, policy_version 13290 (0.0008) [2023-10-13 21:27:17,554][60935] Updated weights for policy 0, policy_version 13300 (0.0008) [2023-10-13 21:27:17,918][60935] Updated weights for policy 0, policy_version 13310 (0.0010) [2023-10-13 21:27:18,755][60934] Updated weights for policy 1, policy_version 13412 (0.0007) [2023-10-13 21:27:19,127][60934] Updated weights for policy 1, policy_version 13422 (0.0008) [2023-10-13 21:27:19,502][60934] Updated weights for policy 1, policy_version 13432 (0.0007) [2023-10-13 21:27:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 27394048. Throughput: 0: 1678.9, 1: 1685.4. Samples: 6856076. Policy #0 lag: (min: 11.0, avg: 34.5, max: 40.0) [2023-10-13 21:27:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:27:22,023][60935] Updated weights for policy 0, policy_version 13320 (0.0010) [2023-10-13 21:27:22,384][60935] Updated weights for policy 0, policy_version 13330 (0.0008) [2023-10-13 21:27:22,757][60935] Updated weights for policy 0, policy_version 13340 (0.0007) [2023-10-13 21:27:23,462][60934] Updated weights for policy 1, policy_version 13442 (0.0009) [2023-10-13 21:27:23,836][60934] Updated weights for policy 1, policy_version 13452 (0.0007) [2023-10-13 21:27:24,211][60934] Updated weights for policy 1, policy_version 13462 (0.0007) [2023-10-13 21:27:24,577][60934] Updated weights for policy 1, policy_version 13472 (0.0008) [2023-10-13 21:27:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 27459584. Throughput: 0: 1681.4, 1: 1688.3. Samples: 6876732. Policy #0 lag: (min: 11.0, avg: 34.5, max: 40.0) [2023-10-13 21:27:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:27:26,752][60935] Updated weights for policy 0, policy_version 13350 (0.0010) [2023-10-13 21:27:27,129][60935] Updated weights for policy 0, policy_version 13360 (0.0011) [2023-10-13 21:27:27,504][60935] Updated weights for policy 0, policy_version 13370 (0.0011) [2023-10-13 21:27:28,648][60934] Updated weights for policy 1, policy_version 13482 (0.0009) [2023-10-13 21:27:29,015][60934] Updated weights for policy 1, policy_version 13492 (0.0007) [2023-10-13 21:27:29,388][60934] Updated weights for policy 1, policy_version 13502 (0.0007) [2023-10-13 21:27:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 27525120. Throughput: 0: 1683.9, 1: 1694.6. Samples: 6886920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:27:31,249][59943] Avg episode reward: [(0, '-0.070'), (1, '-0.710')] [2023-10-13 21:27:31,511][60935] Updated weights for policy 0, policy_version 13380 (0.0009) [2023-10-13 21:27:31,876][60935] Updated weights for policy 0, policy_version 13390 (0.0008) [2023-10-13 21:27:32,254][60935] Updated weights for policy 0, policy_version 13400 (0.0009) [2023-10-13 21:27:33,497][60934] Updated weights for policy 1, policy_version 13512 (0.0009) [2023-10-13 21:27:33,863][60934] Updated weights for policy 1, policy_version 13522 (0.0009) [2023-10-13 21:27:34,232][60934] Updated weights for policy 1, policy_version 13532 (0.0007) [2023-10-13 21:27:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 27590656. Throughput: 0: 1683.1, 1: 1673.0. Samples: 6906746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:27:36,249][59943] Avg episode reward: [(0, '-0.070'), (1, '-0.710')] [2023-10-13 21:27:36,320][60935] Updated weights for policy 0, policy_version 13410 (0.0009) [2023-10-13 21:27:36,692][60935] Updated weights for policy 0, policy_version 13420 (0.0010) [2023-10-13 21:27:37,057][60935] Updated weights for policy 0, policy_version 13430 (0.0007) [2023-10-13 21:27:37,422][60935] Updated weights for policy 0, policy_version 13440 (0.0010) [2023-10-13 21:27:38,250][60934] Updated weights for policy 1, policy_version 13542 (0.0008) [2023-10-13 21:27:38,632][60934] Updated weights for policy 1, policy_version 13552 (0.0008) [2023-10-13 21:27:39,007][60934] Updated weights for policy 1, policy_version 13562 (0.0008) [2023-10-13 21:27:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 27656192. Throughput: 0: 1683.6, 1: 1699.5. Samples: 6927726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:27:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.710')] [2023-10-13 21:27:41,500][60935] Updated weights for policy 0, policy_version 13450 (0.0009) [2023-10-13 21:27:41,874][60935] Updated weights for policy 0, policy_version 13460 (0.0009) [2023-10-13 21:27:42,247][60935] Updated weights for policy 0, policy_version 13470 (0.0008) [2023-10-13 21:27:43,017][60934] Updated weights for policy 1, policy_version 13572 (0.0008) [2023-10-13 21:27:43,389][60934] Updated weights for policy 1, policy_version 13582 (0.0007) [2023-10-13 21:27:43,751][60934] Updated weights for policy 1, policy_version 13592 (0.0007) [2023-10-13 21:27:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 27721728. Throughput: 0: 1683.4, 1: 1693.9. Samples: 6937452. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-13 21:27:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.710')] [2023-10-13 21:27:46,291][60935] Updated weights for policy 0, policy_version 13480 (0.0007) [2023-10-13 21:27:46,659][60935] Updated weights for policy 0, policy_version 13490 (0.0007) [2023-10-13 21:27:47,023][60935] Updated weights for policy 0, policy_version 13500 (0.0009) [2023-10-13 21:27:47,702][60934] Updated weights for policy 1, policy_version 13602 (0.0007) [2023-10-13 21:27:48,075][60934] Updated weights for policy 1, policy_version 13612 (0.0009) [2023-10-13 21:27:48,439][60934] Updated weights for policy 1, policy_version 13622 (0.0009) [2023-10-13 21:27:48,813][60934] Updated weights for policy 1, policy_version 13632 (0.0008) [2023-10-13 21:27:51,172][60935] Updated weights for policy 0, policy_version 13510 (0.0009) [2023-10-13 21:27:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 27787264. Throughput: 0: 1678.1, 1: 1697.2. Samples: 6957830. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-13 21:27:51,249][59943] Avg episode reward: [(0, '-0.040'), (1, '-0.710')] [2023-10-13 21:27:51,542][60935] Updated weights for policy 0, policy_version 13520 (0.0010) [2023-10-13 21:27:51,917][60935] Updated weights for policy 0, policy_version 13530 (0.0009) [2023-10-13 21:27:52,586][60934] Updated weights for policy 1, policy_version 13642 (0.0007) [2023-10-13 21:27:52,950][60934] Updated weights for policy 1, policy_version 13652 (0.0007) [2023-10-13 21:27:53,319][60934] Updated weights for policy 1, policy_version 13662 (0.0008) [2023-10-13 21:27:55,959][60935] Updated weights for policy 0, policy_version 13540 (0.0007) [2023-10-13 21:27:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 27852800. Throughput: 0: 1681.6, 1: 1720.0. Samples: 6978908. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-13 21:27:56,249][59943] Avg episode reward: [(0, '-0.040'), (1, '-0.710')] [2023-10-13 21:27:56,339][60935] Updated weights for policy 0, policy_version 13550 (0.0007) [2023-10-13 21:27:56,722][60935] Updated weights for policy 0, policy_version 13560 (0.0010) [2023-10-13 21:27:57,417][60934] Updated weights for policy 1, policy_version 13672 (0.0009) [2023-10-13 21:27:57,786][60934] Updated weights for policy 1, policy_version 13682 (0.0009) [2023-10-13 21:27:58,163][60934] Updated weights for policy 1, policy_version 13692 (0.0008) [2023-10-13 21:28:00,669][60935] Updated weights for policy 0, policy_version 13570 (0.0008) [2023-10-13 21:28:01,041][60935] Updated weights for policy 0, policy_version 13580 (0.0008) [2023-10-13 21:28:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 27918336. Throughput: 0: 1683.1, 1: 1689.0. Samples: 6988010. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-13 21:28:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.710')] [2023-10-13 21:28:01,414][60935] Updated weights for policy 0, policy_version 13590 (0.0008) [2023-10-13 21:28:01,789][60935] Updated weights for policy 0, policy_version 13600 (0.0010) [2023-10-13 21:28:02,049][60934] Updated weights for policy 1, policy_version 13702 (0.0010) [2023-10-13 21:28:02,416][60934] Updated weights for policy 1, policy_version 13712 (0.0010) [2023-10-13 21:28:02,779][60934] Updated weights for policy 1, policy_version 13722 (0.0007) [2023-10-13 21:28:05,907][60935] Updated weights for policy 0, policy_version 13610 (0.0007) [2023-10-13 21:28:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 27983872. Throughput: 0: 1685.2, 1: 1714.3. Samples: 7009050. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-13 21:28:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:28:06,281][60935] Updated weights for policy 0, policy_version 13620 (0.0007) [2023-10-13 21:28:06,656][60935] Updated weights for policy 0, policy_version 13630 (0.0007) [2023-10-13 21:28:06,971][60934] Updated weights for policy 1, policy_version 13732 (0.0010) [2023-10-13 21:28:07,338][60934] Updated weights for policy 1, policy_version 13742 (0.0008) [2023-10-13 21:28:07,713][60934] Updated weights for policy 1, policy_version 13752 (0.0008) [2023-10-13 21:28:10,721][60935] Updated weights for policy 0, policy_version 13640 (0.0008) [2023-10-13 21:28:11,092][60935] Updated weights for policy 0, policy_version 13650 (0.0008) [2023-10-13 21:28:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 28049408. Throughput: 0: 1672.6, 1: 1716.4. Samples: 7029238. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-13 21:28:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:28:11,465][60935] Updated weights for policy 0, policy_version 13660 (0.0007) [2023-10-13 21:28:11,764][60934] Updated weights for policy 1, policy_version 13762 (0.0009) [2023-10-13 21:28:12,129][60934] Updated weights for policy 1, policy_version 13772 (0.0009) [2023-10-13 21:28:12,502][60934] Updated weights for policy 1, policy_version 13782 (0.0011) [2023-10-13 21:28:12,875][60934] Updated weights for policy 1, policy_version 13792 (0.0009) [2023-10-13 21:28:15,439][60935] Updated weights for policy 0, policy_version 13670 (0.0010) [2023-10-13 21:28:15,810][60935] Updated weights for policy 0, policy_version 13680 (0.0011) [2023-10-13 21:28:16,181][60935] Updated weights for policy 0, policy_version 13690 (0.0009) [2023-10-13 21:28:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 28114944. Throughput: 0: 1683.6, 1: 1692.8. Samples: 7038858. Policy #0 lag: (min: 27.0, avg: 27.3, max: 38.0) [2023-10-13 21:28:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:28:16,767][60934] Updated weights for policy 1, policy_version 13802 (0.0010) [2023-10-13 21:28:17,140][60934] Updated weights for policy 1, policy_version 13812 (0.0009) [2023-10-13 21:28:17,502][60934] Updated weights for policy 1, policy_version 13822 (0.0007) [2023-10-13 21:28:20,321][60935] Updated weights for policy 0, policy_version 13700 (0.0009) [2023-10-13 21:28:20,722][60935] Updated weights for policy 0, policy_version 13710 (0.0009) [2023-10-13 21:28:21,089][60935] Updated weights for policy 0, policy_version 13720 (0.0008) [2023-10-13 21:28:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.5). Total num frames: 28180480. Throughput: 0: 1683.3, 1: 1722.0. Samples: 7059982. Policy #0 lag: (min: 27.0, avg: 27.3, max: 38.0) [2023-10-13 21:28:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:28:21,421][60934] Updated weights for policy 1, policy_version 13832 (0.0009) [2023-10-13 21:28:21,790][60934] Updated weights for policy 1, policy_version 13842 (0.0009) [2023-10-13 21:28:22,158][60934] Updated weights for policy 1, policy_version 13852 (0.0011) [2023-10-13 21:28:25,246][60935] Updated weights for policy 0, policy_version 13730 (0.0009) [2023-10-13 21:28:25,622][60935] Updated weights for policy 0, policy_version 13740 (0.0009) [2023-10-13 21:28:25,995][60935] Updated weights for policy 0, policy_version 13750 (0.0009) [2023-10-13 21:28:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 28246016. Throughput: 0: 1666.0, 1: 1720.1. Samples: 7080102. Policy #0 lag: (min: 27.0, avg: 27.3, max: 38.0) [2023-10-13 21:28:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:28:26,271][60934] Updated weights for policy 1, policy_version 13862 (0.0007) [2023-10-13 21:28:26,357][60935] Updated weights for policy 0, policy_version 13760 (0.0008) [2023-10-13 21:28:26,655][60934] Updated weights for policy 1, policy_version 13872 (0.0008) [2023-10-13 21:28:27,031][60934] Updated weights for policy 1, policy_version 13882 (0.0008) [2023-10-13 21:28:30,468][60935] Updated weights for policy 0, policy_version 13770 (0.0016) [2023-10-13 21:28:30,842][60935] Updated weights for policy 0, policy_version 13780 (0.0008) [2023-10-13 21:28:31,043][60934] Updated weights for policy 1, policy_version 13892 (0.0008) [2023-10-13 21:28:31,210][60935] Updated weights for policy 0, policy_version 13790 (0.0010) [2023-10-13 21:28:31,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 28311552. Throughput: 0: 1676.8, 1: 1702.2. Samples: 7089506. Policy #0 lag: (min: 12.0, avg: 28.6, max: 44.0) [2023-10-13 21:28:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:28:31,424][60934] Updated weights for policy 1, policy_version 13902 (0.0009) [2023-10-13 21:28:31,786][60934] Updated weights for policy 1, policy_version 13912 (0.0008) [2023-10-13 21:28:35,292][60935] Updated weights for policy 0, policy_version 13800 (0.0009) [2023-10-13 21:28:35,668][60935] Updated weights for policy 0, policy_version 13810 (0.0010) [2023-10-13 21:28:35,697][60934] Updated weights for policy 1, policy_version 13922 (0.0008) [2023-10-13 21:28:36,029][60935] Updated weights for policy 0, policy_version 13820 (0.0008) [2023-10-13 21:28:36,068][60934] Updated weights for policy 1, policy_version 13932 (0.0008) [2023-10-13 21:28:36,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 28409856. Throughput: 0: 1677.3, 1: 1709.9. Samples: 7110258. Policy #0 lag: (min: 12.0, avg: 28.6, max: 44.0) [2023-10-13 21:28:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:28:36,427][60934] Updated weights for policy 1, policy_version 13942 (0.0007) [2023-10-13 21:28:36,793][60934] Updated weights for policy 1, policy_version 13952 (0.0008) [2023-10-13 21:28:40,139][60935] Updated weights for policy 0, policy_version 13830 (0.0010) [2023-10-13 21:28:40,512][60935] Updated weights for policy 0, policy_version 13840 (0.0008) [2023-10-13 21:28:40,754][60934] Updated weights for policy 1, policy_version 13962 (0.0009) [2023-10-13 21:28:40,882][60935] Updated weights for policy 0, policy_version 13850 (0.0008) [2023-10-13 21:28:41,126][60934] Updated weights for policy 1, policy_version 13972 (0.0008) [2023-10-13 21:28:41,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 28475392. Throughput: 0: 1651.3, 1: 1701.7. Samples: 7129794. Policy #0 lag: (min: 6.0, avg: 6.0, max: 10.0) [2023-10-13 21:28:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:28:41,491][60934] Updated weights for policy 1, policy_version 13982 (0.0009) [2023-10-13 21:28:45,074][60935] Updated weights for policy 0, policy_version 13860 (0.0009) [2023-10-13 21:28:45,451][60935] Updated weights for policy 0, policy_version 13870 (0.0008) [2023-10-13 21:28:45,605][60934] Updated weights for policy 1, policy_version 13992 (0.0009) [2023-10-13 21:28:45,814][60935] Updated weights for policy 0, policy_version 13880 (0.0009) [2023-10-13 21:28:45,963][60934] Updated weights for policy 1, policy_version 14002 (0.0009) [2023-10-13 21:28:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 28540928. Throughput: 0: 1669.5, 1: 1704.9. Samples: 7139860. Policy #0 lag: (min: 6.0, avg: 6.0, max: 10.0) [2023-10-13 21:28:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:28:46,337][60934] Updated weights for policy 1, policy_version 14012 (0.0009) [2023-10-13 21:28:49,949][60935] Updated weights for policy 0, policy_version 13890 (0.0009) [2023-10-13 21:28:50,321][60935] Updated weights for policy 0, policy_version 13900 (0.0008) [2023-10-13 21:28:50,509][60934] Updated weights for policy 1, policy_version 14022 (0.0008) [2023-10-13 21:28:50,694][60935] Updated weights for policy 0, policy_version 13910 (0.0007) [2023-10-13 21:28:50,878][60934] Updated weights for policy 1, policy_version 14032 (0.0007) [2023-10-13 21:28:51,067][60935] Updated weights for policy 0, policy_version 13920 (0.0009) [2023-10-13 21:28:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 28606464. Throughput: 0: 1670.0, 1: 1697.0. Samples: 7160566. Policy #0 lag: (min: 6.0, avg: 6.0, max: 10.0) [2023-10-13 21:28:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:28:51,250][60934] Updated weights for policy 1, policy_version 14042 (0.0010) [2023-10-13 21:28:55,076][60934] Updated weights for policy 1, policy_version 14052 (0.0007) [2023-10-13 21:28:55,116][60935] Updated weights for policy 0, policy_version 13930 (0.0008) [2023-10-13 21:28:55,444][60934] Updated weights for policy 1, policy_version 14062 (0.0008) [2023-10-13 21:28:55,497][60935] Updated weights for policy 0, policy_version 13940 (0.0008) [2023-10-13 21:28:55,810][60934] Updated weights for policy 1, policy_version 14072 (0.0009) [2023-10-13 21:28:55,867][60935] Updated weights for policy 0, policy_version 13950 (0.0011) [2023-10-13 21:28:56,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 28704768. Throughput: 0: 1655.4, 1: 1690.4. Samples: 7179798. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-13 21:28:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:28:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000013952_14286848.pth... [2023-10-13 21:28:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000014080_14417920.pth... [2023-10-13 21:28:56,297][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000012480_12779520.pth [2023-10-13 21:28:56,298][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000012384_12681216.pth [2023-10-13 21:29:00,009][60934] Updated weights for policy 1, policy_version 14082 (0.0007) [2023-10-13 21:29:00,042][60935] Updated weights for policy 0, policy_version 13960 (0.0007) [2023-10-13 21:29:00,372][60934] Updated weights for policy 1, policy_version 14092 (0.0009) [2023-10-13 21:29:00,407][60935] Updated weights for policy 0, policy_version 13970 (0.0008) [2023-10-13 21:29:00,744][60934] Updated weights for policy 1, policy_version 14102 (0.0007) [2023-10-13 21:29:00,781][60935] Updated weights for policy 0, policy_version 13980 (0.0009) [2023-10-13 21:29:01,104][60934] Updated weights for policy 1, policy_version 14112 (0.0008) [2023-10-13 21:29:01,248][59943] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 28770304. Throughput: 0: 1663.7, 1: 1700.7. Samples: 7190256. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-13 21:29:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:29:04,677][60935] Updated weights for policy 0, policy_version 13990 (0.0008) [2023-10-13 21:29:05,051][60935] Updated weights for policy 0, policy_version 14000 (0.0010) [2023-10-13 21:29:05,113][60934] Updated weights for policy 1, policy_version 14122 (0.0008) [2023-10-13 21:29:05,427][60935] Updated weights for policy 0, policy_version 14010 (0.0009) [2023-10-13 21:29:05,485][60934] Updated weights for policy 1, policy_version 14132 (0.0007) [2023-10-13 21:29:05,854][60934] Updated weights for policy 1, policy_version 14142 (0.0007) [2023-10-13 21:29:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 28835840. Throughput: 0: 1653.2, 1: 1693.2. Samples: 7210568. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-13 21:29:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:29:09,721][60935] Updated weights for policy 0, policy_version 14020 (0.0007) [2023-10-13 21:29:10,001][60934] Updated weights for policy 1, policy_version 14152 (0.0008) [2023-10-13 21:29:10,109][60935] Updated weights for policy 0, policy_version 14030 (0.0008) [2023-10-13 21:29:10,363][60934] Updated weights for policy 1, policy_version 14162 (0.0009) [2023-10-13 21:29:10,478][60935] Updated weights for policy 0, policy_version 14040 (0.0007) [2023-10-13 21:29:10,735][60934] Updated weights for policy 1, policy_version 14172 (0.0007) [2023-10-13 21:29:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 28901376. Throughput: 0: 1643.5, 1: 1670.0. Samples: 7229210. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 21:29:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:29:14,506][60935] Updated weights for policy 0, policy_version 14050 (0.0010) [2023-10-13 21:29:14,656][60934] Updated weights for policy 1, policy_version 14182 (0.0008) [2023-10-13 21:29:14,884][60935] Updated weights for policy 0, policy_version 14060 (0.0008) [2023-10-13 21:29:15,044][60934] Updated weights for policy 1, policy_version 14192 (0.0010) [2023-10-13 21:29:15,261][60935] Updated weights for policy 0, policy_version 14070 (0.0008) [2023-10-13 21:29:15,416][60934] Updated weights for policy 1, policy_version 14202 (0.0009) [2023-10-13 21:29:15,623][60935] Updated weights for policy 0, policy_version 14080 (0.0008) [2023-10-13 21:29:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 28966912. Throughput: 0: 1661.7, 1: 1693.6. Samples: 7240494. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 21:29:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:29:19,529][60934] Updated weights for policy 1, policy_version 14212 (0.0008) [2023-10-13 21:29:19,880][60935] Updated weights for policy 0, policy_version 14090 (0.0008) [2023-10-13 21:29:19,896][60934] Updated weights for policy 1, policy_version 14222 (0.0007) [2023-10-13 21:29:20,251][60935] Updated weights for policy 0, policy_version 14100 (0.0008) [2023-10-13 21:29:20,257][60934] Updated weights for policy 1, policy_version 14232 (0.0008) [2023-10-13 21:29:20,620][60935] Updated weights for policy 0, policy_version 14110 (0.0009) [2023-10-13 21:29:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 29032448. Throughput: 0: 1654.5, 1: 1684.0. Samples: 7260490. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 21:29:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:29:24,246][60934] Updated weights for policy 1, policy_version 14242 (0.0008) [2023-10-13 21:29:24,614][60934] Updated weights for policy 1, policy_version 14252 (0.0009) [2023-10-13 21:29:24,704][60935] Updated weights for policy 0, policy_version 14120 (0.0008) [2023-10-13 21:29:24,978][60934] Updated weights for policy 1, policy_version 14262 (0.0008) [2023-10-13 21:29:25,069][60935] Updated weights for policy 0, policy_version 14130 (0.0009) [2023-10-13 21:29:25,343][60934] Updated weights for policy 1, policy_version 14272 (0.0007) [2023-10-13 21:29:25,446][60935] Updated weights for policy 0, policy_version 14140 (0.0008) [2023-10-13 21:29:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 29097984. Throughput: 0: 1657.4, 1: 1665.6. Samples: 7279330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:29:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:29:29,308][60934] Updated weights for policy 1, policy_version 14282 (0.0009) [2023-10-13 21:29:29,622][60935] Updated weights for policy 0, policy_version 14150 (0.0008) [2023-10-13 21:29:29,671][60934] Updated weights for policy 1, policy_version 14292 (0.0008) [2023-10-13 21:29:29,984][60935] Updated weights for policy 0, policy_version 14160 (0.0007) [2023-10-13 21:29:30,035][60934] Updated weights for policy 1, policy_version 14302 (0.0010) [2023-10-13 21:29:30,358][60935] Updated weights for policy 0, policy_version 14170 (0.0008) [2023-10-13 21:29:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 29163520. Throughput: 0: 1664.2, 1: 1693.5. Samples: 7290958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:29:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:29:34,246][60934] Updated weights for policy 1, policy_version 14312 (0.0008) [2023-10-13 21:29:34,455][60935] Updated weights for policy 0, policy_version 14180 (0.0007) [2023-10-13 21:29:34,611][60934] Updated weights for policy 1, policy_version 14322 (0.0008) [2023-10-13 21:29:34,825][60935] Updated weights for policy 0, policy_version 14190 (0.0007) [2023-10-13 21:29:34,976][60934] Updated weights for policy 1, policy_version 14332 (0.0007) [2023-10-13 21:29:35,204][60935] Updated weights for policy 0, policy_version 14200 (0.0007) [2023-10-13 21:29:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.5). Total num frames: 29229056. Throughput: 0: 1650.9, 1: 1679.3. Samples: 7310424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:29:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:29:38,866][60934] Updated weights for policy 1, policy_version 14342 (0.0009) [2023-10-13 21:29:39,155][60935] Updated weights for policy 0, policy_version 14210 (0.0009) [2023-10-13 21:29:39,233][60934] Updated weights for policy 1, policy_version 14352 (0.0007) [2023-10-13 21:29:39,524][60935] Updated weights for policy 0, policy_version 14220 (0.0008) [2023-10-13 21:29:39,603][60934] Updated weights for policy 1, policy_version 14362 (0.0007) [2023-10-13 21:29:39,902][60935] Updated weights for policy 0, policy_version 14230 (0.0007) [2023-10-13 21:29:40,272][60935] Updated weights for policy 0, policy_version 14240 (0.0008) [2023-10-13 21:29:41,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 29294592. Throughput: 0: 1656.9, 1: 1677.3. Samples: 7329840. Policy #0 lag: (min: 2.0, avg: 3.1, max: 23.0) [2023-10-13 21:29:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:29:43,715][60934] Updated weights for policy 1, policy_version 14372 (0.0007) [2023-10-13 21:29:44,084][60934] Updated weights for policy 1, policy_version 14382 (0.0007) [2023-10-13 21:29:44,413][60935] Updated weights for policy 0, policy_version 14250 (0.0007) [2023-10-13 21:29:44,449][60934] Updated weights for policy 1, policy_version 14392 (0.0007) [2023-10-13 21:29:44,772][60935] Updated weights for policy 0, policy_version 14260 (0.0008) [2023-10-13 21:29:45,138][60935] Updated weights for policy 0, policy_version 14270 (0.0007) [2023-10-13 21:29:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 29360128. Throughput: 0: 1665.3, 1: 1692.6. Samples: 7341358. Policy #0 lag: (min: 2.0, avg: 3.1, max: 23.0) [2023-10-13 21:29:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:29:48,561][60934] Updated weights for policy 1, policy_version 14402 (0.0008) [2023-10-13 21:29:48,935][60934] Updated weights for policy 1, policy_version 14412 (0.0008) [2023-10-13 21:29:49,299][60934] Updated weights for policy 1, policy_version 14422 (0.0007) [2023-10-13 21:29:49,307][60935] Updated weights for policy 0, policy_version 14280 (0.0009) [2023-10-13 21:29:49,664][60934] Updated weights for policy 1, policy_version 14432 (0.0009) [2023-10-13 21:29:49,673][60935] Updated weights for policy 0, policy_version 14290 (0.0010) [2023-10-13 21:29:50,053][60935] Updated weights for policy 0, policy_version 14300 (0.0008) [2023-10-13 21:29:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.5). Total num frames: 29425664. Throughput: 0: 1650.6, 1: 1666.3. Samples: 7359828. Policy #0 lag: (min: 2.0, avg: 3.1, max: 23.0) [2023-10-13 21:29:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:29:53,942][60934] Updated weights for policy 1, policy_version 14442 (0.0008) [2023-10-13 21:29:54,250][60935] Updated weights for policy 0, policy_version 14310 (0.0009) [2023-10-13 21:29:54,323][60934] Updated weights for policy 1, policy_version 14452 (0.0009) [2023-10-13 21:29:54,631][60935] Updated weights for policy 0, policy_version 14320 (0.0009) [2023-10-13 21:29:54,692][60934] Updated weights for policy 1, policy_version 14462 (0.0008) [2023-10-13 21:29:54,991][60935] Updated weights for policy 0, policy_version 14330 (0.0009) [2023-10-13 21:29:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 29491200. Throughput: 0: 1666.9, 1: 1680.1. Samples: 7379824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:29:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:29:58,645][60934] Updated weights for policy 1, policy_version 14472 (0.0008) [2023-10-13 21:29:59,014][60934] Updated weights for policy 1, policy_version 14482 (0.0009) [2023-10-13 21:29:59,223][60935] Updated weights for policy 0, policy_version 14340 (0.0007) [2023-10-13 21:29:59,388][60934] Updated weights for policy 1, policy_version 14492 (0.0009) [2023-10-13 21:29:59,588][60935] Updated weights for policy 0, policy_version 14350 (0.0008) [2023-10-13 21:29:59,955][60935] Updated weights for policy 0, policy_version 14360 (0.0011) [2023-10-13 21:30:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 29556736. Throughput: 0: 1663.7, 1: 1685.1. Samples: 7391188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:30:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:30:03,508][60934] Updated weights for policy 1, policy_version 14502 (0.0008) [2023-10-13 21:30:03,881][60934] Updated weights for policy 1, policy_version 14512 (0.0011) [2023-10-13 21:30:04,088][60935] Updated weights for policy 0, policy_version 14370 (0.0010) [2023-10-13 21:30:04,250][60934] Updated weights for policy 1, policy_version 14522 (0.0007) [2023-10-13 21:30:04,460][60935] Updated weights for policy 0, policy_version 14380 (0.0008) [2023-10-13 21:30:04,822][60935] Updated weights for policy 0, policy_version 14390 (0.0011) [2023-10-13 21:30:05,194][60935] Updated weights for policy 0, policy_version 14400 (0.0009) [2023-10-13 21:30:06,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 29622272. Throughput: 0: 1647.6, 1: 1662.9. Samples: 7409464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:30:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:30:08,475][60934] Updated weights for policy 1, policy_version 14532 (0.0009) [2023-10-13 21:30:08,853][60934] Updated weights for policy 1, policy_version 14542 (0.0007) [2023-10-13 21:30:09,226][60934] Updated weights for policy 1, policy_version 14552 (0.0009) [2023-10-13 21:30:09,289][60935] Updated weights for policy 0, policy_version 14410 (0.0007) [2023-10-13 21:30:09,664][60935] Updated weights for policy 0, policy_version 14420 (0.0008) [2023-10-13 21:30:10,030][60935] Updated weights for policy 0, policy_version 14430 (0.0007) [2023-10-13 21:30:11,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 29687808. Throughput: 0: 1660.1, 1: 1682.7. Samples: 7429756. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-13 21:30:11,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:30:13,130][60934] Updated weights for policy 1, policy_version 14562 (0.0008) [2023-10-13 21:30:13,493][60934] Updated weights for policy 1, policy_version 14572 (0.0009) [2023-10-13 21:30:13,867][60934] Updated weights for policy 1, policy_version 14582 (0.0008) [2023-10-13 21:30:13,910][60935] Updated weights for policy 0, policy_version 14440 (0.0008) [2023-10-13 21:30:14,231][60934] Updated weights for policy 1, policy_version 14592 (0.0008) [2023-10-13 21:30:14,286][60935] Updated weights for policy 0, policy_version 14450 (0.0008) [2023-10-13 21:30:14,645][60935] Updated weights for policy 0, policy_version 14460 (0.0009) [2023-10-13 21:30:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 29753344. Throughput: 0: 1663.1, 1: 1669.0. Samples: 7440902. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-13 21:30:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:30:18,263][60934] Updated weights for policy 1, policy_version 14602 (0.0008) [2023-10-13 21:30:18,634][60934] Updated weights for policy 1, policy_version 14612 (0.0007) [2023-10-13 21:30:18,817][60935] Updated weights for policy 0, policy_version 14470 (0.0007) [2023-10-13 21:30:18,998][60934] Updated weights for policy 1, policy_version 14622 (0.0008) [2023-10-13 21:30:19,181][60935] Updated weights for policy 0, policy_version 14480 (0.0007) [2023-10-13 21:30:19,552][60935] Updated weights for policy 0, policy_version 14490 (0.0007) [2023-10-13 21:30:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 29818880. Throughput: 0: 1645.8, 1: 1670.7. Samples: 7459664. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-13 21:30:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:30:22,981][60934] Updated weights for policy 1, policy_version 14632 (0.0008) [2023-10-13 21:30:23,346][60934] Updated weights for policy 1, policy_version 14642 (0.0008) [2023-10-13 21:30:23,711][60934] Updated weights for policy 1, policy_version 14652 (0.0008) [2023-10-13 21:30:23,963][60935] Updated weights for policy 0, policy_version 14500 (0.0009) [2023-10-13 21:30:24,335][60935] Updated weights for policy 0, policy_version 14510 (0.0009) [2023-10-13 21:30:24,697][60935] Updated weights for policy 0, policy_version 14520 (0.0007) [2023-10-13 21:30:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 29884416. Throughput: 0: 1654.1, 1: 1684.7. Samples: 7480090. Policy #0 lag: (min: 24.0, avg: 44.7, max: 56.0) [2023-10-13 21:30:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:30:27,835][60934] Updated weights for policy 1, policy_version 14662 (0.0007) [2023-10-13 21:30:28,211][60934] Updated weights for policy 1, policy_version 14672 (0.0008) [2023-10-13 21:30:28,579][60934] Updated weights for policy 1, policy_version 14682 (0.0008) [2023-10-13 21:30:28,773][60935] Updated weights for policy 0, policy_version 14530 (0.0007) [2023-10-13 21:30:29,142][60935] Updated weights for policy 0, policy_version 14540 (0.0007) [2023-10-13 21:30:29,515][60935] Updated weights for policy 0, policy_version 14550 (0.0007) [2023-10-13 21:30:29,884][60935] Updated weights for policy 0, policy_version 14560 (0.0009) [2023-10-13 21:30:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 29949952. Throughput: 0: 1653.9, 1: 1663.6. Samples: 7490642. Policy #0 lag: (min: 24.0, avg: 44.7, max: 56.0) [2023-10-13 21:30:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:30:32,658][60934] Updated weights for policy 1, policy_version 14692 (0.0008) [2023-10-13 21:30:33,037][60934] Updated weights for policy 1, policy_version 14702 (0.0007) [2023-10-13 21:30:33,401][60934] Updated weights for policy 1, policy_version 14712 (0.0008) [2023-10-13 21:30:33,868][60935] Updated weights for policy 0, policy_version 14570 (0.0007) [2023-10-13 21:30:34,246][60935] Updated weights for policy 0, policy_version 14580 (0.0010) [2023-10-13 21:30:34,605][60935] Updated weights for policy 0, policy_version 14590 (0.0010) [2023-10-13 21:30:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 30015488. Throughput: 0: 1654.3, 1: 1682.0. Samples: 7509962. Policy #0 lag: (min: 24.0, avg: 44.7, max: 56.0) [2023-10-13 21:30:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:30:37,420][60934] Updated weights for policy 1, policy_version 14722 (0.0009) [2023-10-13 21:30:37,788][60934] Updated weights for policy 1, policy_version 14732 (0.0008) [2023-10-13 21:30:38,160][60934] Updated weights for policy 1, policy_version 14742 (0.0007) [2023-10-13 21:30:38,526][60934] Updated weights for policy 1, policy_version 14752 (0.0008) [2023-10-13 21:30:38,646][60935] Updated weights for policy 0, policy_version 14600 (0.0008) [2023-10-13 21:30:39,022][60935] Updated weights for policy 0, policy_version 14610 (0.0009) [2023-10-13 21:30:39,395][60935] Updated weights for policy 0, policy_version 14620 (0.0009) [2023-10-13 21:30:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 30081024. Throughput: 0: 1664.8, 1: 1688.5. Samples: 7530722. Policy #0 lag: (min: 10.0, avg: 18.8, max: 42.0) [2023-10-13 21:30:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:30:42,603][60934] Updated weights for policy 1, policy_version 14762 (0.0007) [2023-10-13 21:30:42,974][60934] Updated weights for policy 1, policy_version 14772 (0.0007) [2023-10-13 21:30:43,345][60934] Updated weights for policy 1, policy_version 14782 (0.0007) [2023-10-13 21:30:43,606][60935] Updated weights for policy 0, policy_version 14630 (0.0009) [2023-10-13 21:30:43,987][60935] Updated weights for policy 0, policy_version 14640 (0.0008) [2023-10-13 21:30:44,358][60935] Updated weights for policy 0, policy_version 14650 (0.0010) [2023-10-13 21:30:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 30146560. Throughput: 0: 1656.5, 1: 1659.4. Samples: 7540406. Policy #0 lag: (min: 10.0, avg: 18.8, max: 42.0) [2023-10-13 21:30:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:30:47,245][60934] Updated weights for policy 1, policy_version 14792 (0.0009) [2023-10-13 21:30:47,618][60934] Updated weights for policy 1, policy_version 14802 (0.0011) [2023-10-13 21:30:47,995][60934] Updated weights for policy 1, policy_version 14812 (0.0009) [2023-10-13 21:30:48,269][60935] Updated weights for policy 0, policy_version 14660 (0.0010) [2023-10-13 21:30:48,642][60935] Updated weights for policy 0, policy_version 14670 (0.0009) [2023-10-13 21:30:49,022][60935] Updated weights for policy 0, policy_version 14680 (0.0008) [2023-10-13 21:30:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 30212096. Throughput: 0: 1665.2, 1: 1690.7. Samples: 7560480. Policy #0 lag: (min: 10.0, avg: 18.8, max: 42.0) [2023-10-13 21:30:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:30:52,152][60934] Updated weights for policy 1, policy_version 14822 (0.0010) [2023-10-13 21:30:52,522][60934] Updated weights for policy 1, policy_version 14832 (0.0007) [2023-10-13 21:30:52,887][60934] Updated weights for policy 1, policy_version 14842 (0.0007) [2023-10-13 21:30:53,153][60935] Updated weights for policy 0, policy_version 14690 (0.0009) [2023-10-13 21:30:53,534][60935] Updated weights for policy 0, policy_version 14700 (0.0009) [2023-10-13 21:30:53,888][60935] Updated weights for policy 0, policy_version 14710 (0.0009) [2023-10-13 21:30:54,255][60935] Updated weights for policy 0, policy_version 14720 (0.0009) [2023-10-13 21:30:56,248][59943] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 30277632. Throughput: 0: 1670.6, 1: 1694.4. Samples: 7581184. Policy #0 lag: (min: 2.0, avg: 2.1, max: 9.0) [2023-10-13 21:30:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:30:56,262][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000014848_15204352.pth... [2023-10-13 21:30:56,262][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000014720_15073280.pth... [2023-10-13 21:30:56,298][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000013152_13467648.pth [2023-10-13 21:30:56,303][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000013280_13598720.pth [2023-10-13 21:30:56,962][60934] Updated weights for policy 1, policy_version 14852 (0.0007) [2023-10-13 21:30:57,379][60934] Updated weights for policy 1, policy_version 14862 (0.0008) [2023-10-13 21:30:57,738][60934] Updated weights for policy 1, policy_version 14872 (0.0007) [2023-10-13 21:30:58,285][60935] Updated weights for policy 0, policy_version 14730 (0.0008) [2023-10-13 21:30:58,651][60935] Updated weights for policy 0, policy_version 14740 (0.0008) [2023-10-13 21:30:59,021][60935] Updated weights for policy 0, policy_version 14750 (0.0008) [2023-10-13 21:31:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 30343168. Throughput: 0: 1652.7, 1: 1671.7. Samples: 7590500. Policy #0 lag: (min: 2.0, avg: 2.1, max: 9.0) [2023-10-13 21:31:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:31:01,861][60934] Updated weights for policy 1, policy_version 14882 (0.0008) [2023-10-13 21:31:02,229][60934] Updated weights for policy 1, policy_version 14892 (0.0009) [2023-10-13 21:31:02,599][60934] Updated weights for policy 1, policy_version 14902 (0.0008) [2023-10-13 21:31:02,960][60934] Updated weights for policy 1, policy_version 14912 (0.0009) [2023-10-13 21:31:03,129][60935] Updated weights for policy 0, policy_version 14760 (0.0008) [2023-10-13 21:31:03,500][60935] Updated weights for policy 0, policy_version 14770 (0.0007) [2023-10-13 21:31:03,870][60935] Updated weights for policy 0, policy_version 14780 (0.0010) [2023-10-13 21:31:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 30408704. Throughput: 0: 1671.1, 1: 1687.0. Samples: 7610778. Policy #0 lag: (min: 2.0, avg: 2.1, max: 9.0) [2023-10-13 21:31:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:31:07,082][60934] Updated weights for policy 1, policy_version 14922 (0.0007) [2023-10-13 21:31:07,448][60934] Updated weights for policy 1, policy_version 14932 (0.0008) [2023-10-13 21:31:07,815][60934] Updated weights for policy 1, policy_version 14942 (0.0007) [2023-10-13 21:31:07,918][60935] Updated weights for policy 0, policy_version 14790 (0.0008) [2023-10-13 21:31:08,291][60935] Updated weights for policy 0, policy_version 14800 (0.0010) [2023-10-13 21:31:08,654][60935] Updated weights for policy 0, policy_version 14810 (0.0008) [2023-10-13 21:31:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 30474240. Throughput: 0: 1681.2, 1: 1684.4. Samples: 7631546. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-10-13 21:31:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:31:11,780][60934] Updated weights for policy 1, policy_version 14952 (0.0009) [2023-10-13 21:31:12,151][60934] Updated weights for policy 1, policy_version 14962 (0.0009) [2023-10-13 21:31:12,520][60934] Updated weights for policy 1, policy_version 14972 (0.0009) [2023-10-13 21:31:12,685][60935] Updated weights for policy 0, policy_version 14820 (0.0008) [2023-10-13 21:31:13,060][60935] Updated weights for policy 0, policy_version 14830 (0.0008) [2023-10-13 21:31:13,428][60935] Updated weights for policy 0, policy_version 14840 (0.0008) [2023-10-13 21:31:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 30539776. Throughput: 0: 1655.0, 1: 1682.3. Samples: 7640818. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-10-13 21:31:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:31:16,409][60934] Updated weights for policy 1, policy_version 14982 (0.0008) [2023-10-13 21:31:16,780][60934] Updated weights for policy 1, policy_version 14992 (0.0007) [2023-10-13 21:31:17,143][60934] Updated weights for policy 1, policy_version 15002 (0.0009) [2023-10-13 21:31:17,499][60935] Updated weights for policy 0, policy_version 14850 (0.0010) [2023-10-13 21:31:17,877][60935] Updated weights for policy 0, policy_version 14860 (0.0008) [2023-10-13 21:31:18,248][60935] Updated weights for policy 0, policy_version 14870 (0.0010) [2023-10-13 21:31:18,616][60935] Updated weights for policy 0, policy_version 14880 (0.0009) [2023-10-13 21:31:21,161][60934] Updated weights for policy 1, policy_version 15012 (0.0007) [2023-10-13 21:31:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 30605312. Throughput: 0: 1686.9, 1: 1694.1. Samples: 7662110. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-10-13 21:31:21,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:31:21,531][60934] Updated weights for policy 1, policy_version 15022 (0.0009) [2023-10-13 21:31:21,892][60934] Updated weights for policy 1, policy_version 15032 (0.0010) [2023-10-13 21:31:22,689][60935] Updated weights for policy 0, policy_version 14890 (0.0010) [2023-10-13 21:31:23,063][60935] Updated weights for policy 0, policy_version 14900 (0.0007) [2023-10-13 21:31:23,429][60935] Updated weights for policy 0, policy_version 14910 (0.0009) [2023-10-13 21:31:25,902][60934] Updated weights for policy 1, policy_version 15042 (0.0008) [2023-10-13 21:31:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 30670848. Throughput: 0: 1685.2, 1: 1702.0. Samples: 7683146. Policy #0 lag: (min: 26.0, avg: 32.7, max: 58.0) [2023-10-13 21:31:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:31:26,276][60934] Updated weights for policy 1, policy_version 15052 (0.0009) [2023-10-13 21:31:26,648][60934] Updated weights for policy 1, policy_version 15062 (0.0009) [2023-10-13 21:31:27,007][60934] Updated weights for policy 1, policy_version 15072 (0.0010) [2023-10-13 21:31:27,468][60935] Updated weights for policy 0, policy_version 14920 (0.0010) [2023-10-13 21:31:27,842][60935] Updated weights for policy 0, policy_version 14930 (0.0008) [2023-10-13 21:31:28,213][60935] Updated weights for policy 0, policy_version 14940 (0.0009) [2023-10-13 21:31:31,074][60934] Updated weights for policy 1, policy_version 15082 (0.0008) [2023-10-13 21:31:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 30736384. Throughput: 0: 1668.1, 1: 1703.9. Samples: 7692146. Policy #0 lag: (min: 26.0, avg: 32.7, max: 58.0) [2023-10-13 21:31:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:31:31,436][60934] Updated weights for policy 1, policy_version 15092 (0.0011) [2023-10-13 21:31:31,802][60934] Updated weights for policy 1, policy_version 15102 (0.0010) [2023-10-13 21:31:32,176][60935] Updated weights for policy 0, policy_version 14950 (0.0010) [2023-10-13 21:31:32,553][60935] Updated weights for policy 0, policy_version 14960 (0.0008) [2023-10-13 21:31:32,925][60935] Updated weights for policy 0, policy_version 14970 (0.0008) [2023-10-13 21:31:35,942][60934] Updated weights for policy 1, policy_version 15112 (0.0007) [2023-10-13 21:31:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 30801920. Throughput: 0: 1685.7, 1: 1698.0. Samples: 7712746. Policy #0 lag: (min: 26.0, avg: 32.7, max: 58.0) [2023-10-13 21:31:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:31:36,301][60934] Updated weights for policy 1, policy_version 15122 (0.0009) [2023-10-13 21:31:36,671][60934] Updated weights for policy 1, policy_version 15132 (0.0008) [2023-10-13 21:31:37,091][60935] Updated weights for policy 0, policy_version 14980 (0.0009) [2023-10-13 21:31:37,478][60935] Updated weights for policy 0, policy_version 14990 (0.0008) [2023-10-13 21:31:37,848][60935] Updated weights for policy 0, policy_version 15000 (0.0007) [2023-10-13 21:31:40,742][60934] Updated weights for policy 1, policy_version 15142 (0.0008) [2023-10-13 21:31:41,104][60934] Updated weights for policy 1, policy_version 15152 (0.0008) [2023-10-13 21:31:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 30867456. Throughput: 0: 1682.3, 1: 1695.3. Samples: 7733176. Policy #0 lag: (min: 3.0, avg: 10.6, max: 35.0) [2023-10-13 21:31:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:31:41,480][60934] Updated weights for policy 1, policy_version 15162 (0.0008) [2023-10-13 21:31:41,852][60935] Updated weights for policy 0, policy_version 15010 (0.0010) [2023-10-13 21:31:42,226][60935] Updated weights for policy 0, policy_version 15020 (0.0010) [2023-10-13 21:31:42,608][60935] Updated weights for policy 0, policy_version 15030 (0.0011) [2023-10-13 21:31:42,970][60935] Updated weights for policy 0, policy_version 15040 (0.0009) [2023-10-13 21:31:45,611][60934] Updated weights for policy 1, policy_version 15172 (0.0009) [2023-10-13 21:31:46,007][60934] Updated weights for policy 1, policy_version 15182 (0.0008) [2023-10-13 21:31:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 30932992. Throughput: 0: 1670.5, 1: 1704.5. Samples: 7742376. Policy #0 lag: (min: 3.0, avg: 10.6, max: 35.0) [2023-10-13 21:31:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:31:46,375][60934] Updated weights for policy 1, policy_version 15192 (0.0009) [2023-10-13 21:31:47,074][60935] Updated weights for policy 0, policy_version 15050 (0.0007) [2023-10-13 21:31:47,445][60935] Updated weights for policy 0, policy_version 15060 (0.0009) [2023-10-13 21:31:47,824][60935] Updated weights for policy 0, policy_version 15070 (0.0009) [2023-10-13 21:31:50,448][60934] Updated weights for policy 1, policy_version 15202 (0.0009) [2023-10-13 21:31:50,822][60934] Updated weights for policy 1, policy_version 15212 (0.0009) [2023-10-13 21:31:51,192][60934] Updated weights for policy 1, policy_version 15222 (0.0007) [2023-10-13 21:31:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30998528. Throughput: 0: 1679.6, 1: 1699.1. Samples: 7762818. Policy #0 lag: (min: 3.0, avg: 10.6, max: 35.0) [2023-10-13 21:31:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:31:51,563][60934] Updated weights for policy 1, policy_version 15232 (0.0008) [2023-10-13 21:31:51,967][60935] Updated weights for policy 0, policy_version 15080 (0.0008) [2023-10-13 21:31:52,347][60935] Updated weights for policy 0, policy_version 15090 (0.0008) [2023-10-13 21:31:52,722][60935] Updated weights for policy 0, policy_version 15100 (0.0007) [2023-10-13 21:31:55,568][60934] Updated weights for policy 1, policy_version 15242 (0.0008) [2023-10-13 21:31:55,935][60934] Updated weights for policy 1, policy_version 15252 (0.0009) [2023-10-13 21:31:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 31064064. Throughput: 0: 1683.4, 1: 1689.1. Samples: 7783308. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-13 21:31:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:31:56,310][60934] Updated weights for policy 1, policy_version 15262 (0.0008) [2023-10-13 21:31:56,928][60935] Updated weights for policy 0, policy_version 15110 (0.0007) [2023-10-13 21:31:57,299][60935] Updated weights for policy 0, policy_version 15120 (0.0010) [2023-10-13 21:31:57,673][60935] Updated weights for policy 0, policy_version 15130 (0.0008) [2023-10-13 21:32:00,350][60934] Updated weights for policy 1, policy_version 15272 (0.0009) [2023-10-13 21:32:00,713][60934] Updated weights for policy 1, policy_version 15282 (0.0010) [2023-10-13 21:32:01,081][60934] Updated weights for policy 1, policy_version 15292 (0.0009) [2023-10-13 21:32:01,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 31162368. Throughput: 0: 1680.0, 1: 1697.4. Samples: 7792802. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-13 21:32:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:32:01,782][60935] Updated weights for policy 0, policy_version 15140 (0.0009) [2023-10-13 21:32:02,151][60935] Updated weights for policy 0, policy_version 15150 (0.0011) [2023-10-13 21:32:02,522][60935] Updated weights for policy 0, policy_version 15160 (0.0010) [2023-10-13 21:32:05,317][60934] Updated weights for policy 1, policy_version 15302 (0.0009) [2023-10-13 21:32:05,688][60934] Updated weights for policy 1, policy_version 15312 (0.0007) [2023-10-13 21:32:06,049][60934] Updated weights for policy 1, policy_version 15322 (0.0007) [2023-10-13 21:32:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 31195136. Throughput: 0: 1670.7, 1: 1686.2. Samples: 7813170. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-13 21:32:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:32:06,287][60935] Updated weights for policy 0, policy_version 15170 (0.0009) [2023-10-13 21:32:06,662][60935] Updated weights for policy 0, policy_version 15180 (0.0008) [2023-10-13 21:32:07,036][60935] Updated weights for policy 0, policy_version 15190 (0.0008) [2023-10-13 21:32:07,412][60935] Updated weights for policy 0, policy_version 15200 (0.0009) [2023-10-13 21:32:09,978][60934] Updated weights for policy 1, policy_version 15332 (0.0008) [2023-10-13 21:32:10,344][60934] Updated weights for policy 1, policy_version 15342 (0.0008) [2023-10-13 21:32:10,704][60934] Updated weights for policy 1, policy_version 15352 (0.0009) [2023-10-13 21:32:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 31293440. Throughput: 0: 1678.5, 1: 1664.2. Samples: 7833566. Policy #0 lag: (min: 11.0, avg: 17.0, max: 43.0) [2023-10-13 21:32:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:32:11,586][60935] Updated weights for policy 0, policy_version 15210 (0.0010) [2023-10-13 21:32:11,959][60935] Updated weights for policy 0, policy_version 15220 (0.0009) [2023-10-13 21:32:12,335][60935] Updated weights for policy 0, policy_version 15230 (0.0010) [2023-10-13 21:32:14,791][60934] Updated weights for policy 1, policy_version 15362 (0.0008) [2023-10-13 21:32:15,158][60934] Updated weights for policy 1, policy_version 15372 (0.0009) [2023-10-13 21:32:15,522][60934] Updated weights for policy 1, policy_version 15382 (0.0008) [2023-10-13 21:32:15,889][60934] Updated weights for policy 1, policy_version 15392 (0.0009) [2023-10-13 21:32:16,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 31358976. Throughput: 0: 1681.0, 1: 1685.1. Samples: 7843620. Policy #0 lag: (min: 11.0, avg: 17.0, max: 43.0) [2023-10-13 21:32:16,249][59943] Avg episode reward: [(0, '-0.020'), (1, '0.000')] [2023-10-13 21:32:16,343][60935] Updated weights for policy 0, policy_version 15240 (0.0010) [2023-10-13 21:32:16,710][60935] Updated weights for policy 0, policy_version 15250 (0.0011) [2023-10-13 21:32:17,080][60935] Updated weights for policy 0, policy_version 15260 (0.0011) [2023-10-13 21:32:20,028][60934] Updated weights for policy 1, policy_version 15402 (0.0007) [2023-10-13 21:32:20,382][60934] Updated weights for policy 1, policy_version 15412 (0.0008) [2023-10-13 21:32:20,750][60934] Updated weights for policy 1, policy_version 15422 (0.0008) [2023-10-13 21:32:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 31424512. Throughput: 0: 1678.4, 1: 1685.9. Samples: 7864140. Policy #0 lag: (min: 11.0, avg: 17.0, max: 43.0) [2023-10-13 21:32:21,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 21:32:21,260][60935] Updated weights for policy 0, policy_version 15270 (0.0010) [2023-10-13 21:32:21,643][60935] Updated weights for policy 0, policy_version 15280 (0.0010) [2023-10-13 21:32:22,007][60935] Updated weights for policy 0, policy_version 15290 (0.0010) [2023-10-13 21:32:24,668][60934] Updated weights for policy 1, policy_version 15432 (0.0008) [2023-10-13 21:32:25,048][60934] Updated weights for policy 1, policy_version 15442 (0.0009) [2023-10-13 21:32:25,415][60934] Updated weights for policy 1, policy_version 15452 (0.0009) [2023-10-13 21:32:26,098][60935] Updated weights for policy 0, policy_version 15300 (0.0007) [2023-10-13 21:32:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 31490048. Throughput: 0: 1683.8, 1: 1661.9. Samples: 7883730. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-13 21:32:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:32:26,490][60935] Updated weights for policy 0, policy_version 15310 (0.0010) [2023-10-13 21:32:26,860][60935] Updated weights for policy 0, policy_version 15320 (0.0008) [2023-10-13 21:32:29,394][60934] Updated weights for policy 1, policy_version 15462 (0.0008) [2023-10-13 21:32:29,765][60934] Updated weights for policy 1, policy_version 15472 (0.0009) [2023-10-13 21:32:30,130][60934] Updated weights for policy 1, policy_version 15482 (0.0010) [2023-10-13 21:32:30,875][60935] Updated weights for policy 0, policy_version 15330 (0.0008) [2023-10-13 21:32:31,236][60935] Updated weights for policy 0, policy_version 15340 (0.0008) [2023-10-13 21:32:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 31555584. Throughput: 0: 1680.3, 1: 1684.5. Samples: 7893792. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-13 21:32:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:32:31,609][60935] Updated weights for policy 0, policy_version 15350 (0.0008) [2023-10-13 21:32:31,981][60935] Updated weights for policy 0, policy_version 15360 (0.0009) [2023-10-13 21:32:34,164][60934] Updated weights for policy 1, policy_version 15492 (0.0009) [2023-10-13 21:32:34,567][60934] Updated weights for policy 1, policy_version 15502 (0.0009) [2023-10-13 21:32:34,933][60934] Updated weights for policy 1, policy_version 15512 (0.0007) [2023-10-13 21:32:36,075][60935] Updated weights for policy 0, policy_version 15370 (0.0010) [2023-10-13 21:32:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 31621120. Throughput: 0: 1681.7, 1: 1676.1. Samples: 7913920. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-13 21:32:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:32:36,448][60935] Updated weights for policy 0, policy_version 15380 (0.0009) [2023-10-13 21:32:36,828][60935] Updated weights for policy 0, policy_version 15390 (0.0010) [2023-10-13 21:32:39,081][60934] Updated weights for policy 1, policy_version 15522 (0.0009) [2023-10-13 21:32:39,451][60934] Updated weights for policy 1, policy_version 15532 (0.0010) [2023-10-13 21:32:39,815][60934] Updated weights for policy 1, policy_version 15542 (0.0007) [2023-10-13 21:32:40,187][60934] Updated weights for policy 1, policy_version 15552 (0.0009) [2023-10-13 21:32:41,065][60935] Updated weights for policy 0, policy_version 15400 (0.0008) [2023-10-13 21:32:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 31686656. Throughput: 0: 1671.2, 1: 1667.0. Samples: 7933526. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-13 21:32:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:32:41,428][60935] Updated weights for policy 0, policy_version 15410 (0.0007) [2023-10-13 21:32:41,792][60935] Updated weights for policy 0, policy_version 15420 (0.0009) [2023-10-13 21:32:44,176][60934] Updated weights for policy 1, policy_version 15562 (0.0010) [2023-10-13 21:32:44,547][60934] Updated weights for policy 1, policy_version 15572 (0.0008) [2023-10-13 21:32:44,913][60934] Updated weights for policy 1, policy_version 15582 (0.0007) [2023-10-13 21:32:46,006][60935] Updated weights for policy 0, policy_version 15430 (0.0009) [2023-10-13 21:32:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 31752192. Throughput: 0: 1676.1, 1: 1685.6. Samples: 7944080. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-13 21:32:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:32:46,383][60935] Updated weights for policy 0, policy_version 15440 (0.0007) [2023-10-13 21:32:46,760][60935] Updated weights for policy 0, policy_version 15450 (0.0008) [2023-10-13 21:32:49,193][60934] Updated weights for policy 1, policy_version 15592 (0.0008) [2023-10-13 21:32:49,568][60934] Updated weights for policy 1, policy_version 15602 (0.0008) [2023-10-13 21:32:49,944][60934] Updated weights for policy 1, policy_version 15612 (0.0008) [2023-10-13 21:32:50,832][60935] Updated weights for policy 0, policy_version 15460 (0.0008) [2023-10-13 21:32:51,207][60935] Updated weights for policy 0, policy_version 15470 (0.0009) [2023-10-13 21:32:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 31817728. Throughput: 0: 1677.2, 1: 1670.4. Samples: 7963814. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-13 21:32:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-3.360')] [2023-10-13 21:32:51,586][60935] Updated weights for policy 0, policy_version 15480 (0.0007) [2023-10-13 21:32:53,781][60934] Updated weights for policy 1, policy_version 15622 (0.0008) [2023-10-13 21:32:54,149][60934] Updated weights for policy 1, policy_version 15632 (0.0007) [2023-10-13 21:32:54,519][60934] Updated weights for policy 1, policy_version 15642 (0.0008) [2023-10-13 21:32:55,792][60935] Updated weights for policy 0, policy_version 15490 (0.0009) [2023-10-13 21:32:56,170][60935] Updated weights for policy 0, policy_version 15500 (0.0007) [2023-10-13 21:32:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 31883264. Throughput: 0: 1663.2, 1: 1674.3. Samples: 7983752. Policy #0 lag: (min: 9.0, avg: 17.9, max: 41.0) [2023-10-13 21:32:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-3.360')] [2023-10-13 21:32:56,255][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000015648_16023552.pth... [2023-10-13 21:32:56,285][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000014080_14417920.pth [2023-10-13 21:32:56,289][60828] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p1/milestones/checkpoint_000015648_16023552.pth [2023-10-13 21:32:56,550][60935] Updated weights for policy 0, policy_version 15510 (0.0012) [2023-10-13 21:32:56,911][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000015520_15892480.pth... [2023-10-13 21:32:56,916][60935] Updated weights for policy 0, policy_version 15520 (0.0009) [2023-10-13 21:32:56,945][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000013952_14286848.pth [2023-10-13 21:32:56,948][60695] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p0/milestones/checkpoint_000015520_15892480.pth [2023-10-13 21:32:58,634][60934] Updated weights for policy 1, policy_version 15652 (0.0007) [2023-10-13 21:32:59,017][60934] Updated weights for policy 1, policy_version 15662 (0.0007) [2023-10-13 21:32:59,392][60934] Updated weights for policy 1, policy_version 15672 (0.0007) [2023-10-13 21:33:00,937][60935] Updated weights for policy 0, policy_version 15530 (0.0008) [2023-10-13 21:33:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 31948800. Throughput: 0: 1662.5, 1: 1679.3. Samples: 7994002. Policy #0 lag: (min: 9.0, avg: 17.9, max: 41.0) [2023-10-13 21:33:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-3.360')] [2023-10-13 21:33:01,301][60935] Updated weights for policy 0, policy_version 15540 (0.0010) [2023-10-13 21:33:01,669][60935] Updated weights for policy 0, policy_version 15550 (0.0009) [2023-10-13 21:33:03,495][60934] Updated weights for policy 1, policy_version 15682 (0.0008) [2023-10-13 21:33:03,861][60934] Updated weights for policy 1, policy_version 15692 (0.0008) [2023-10-13 21:33:04,234][60934] Updated weights for policy 1, policy_version 15702 (0.0008) [2023-10-13 21:33:04,599][60934] Updated weights for policy 1, policy_version 15712 (0.0009) [2023-10-13 21:33:05,843][60935] Updated weights for policy 0, policy_version 15560 (0.0008) [2023-10-13 21:33:06,219][60935] Updated weights for policy 0, policy_version 15570 (0.0009) [2023-10-13 21:33:06,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 32014336. Throughput: 0: 1659.6, 1: 1656.0. Samples: 8013342. Policy #0 lag: (min: 9.0, avg: 17.9, max: 41.0) [2023-10-13 21:33:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:33:06,594][60935] Updated weights for policy 0, policy_version 15580 (0.0008) [2023-10-13 21:33:08,433][60934] Updated weights for policy 1, policy_version 15722 (0.0008) [2023-10-13 21:33:08,805][60934] Updated weights for policy 1, policy_version 15732 (0.0007) [2023-10-13 21:33:09,166][60934] Updated weights for policy 1, policy_version 15742 (0.0008) [2023-10-13 21:33:10,896][60935] Updated weights for policy 0, policy_version 15590 (0.0009) [2023-10-13 21:33:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 32079872. Throughput: 0: 1651.6, 1: 1687.4. Samples: 8033986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:33:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:33:11,275][60935] Updated weights for policy 0, policy_version 15600 (0.0009) [2023-10-13 21:33:11,653][60935] Updated weights for policy 0, policy_version 15610 (0.0008) [2023-10-13 21:33:13,254][60934] Updated weights for policy 1, policy_version 15752 (0.0008) [2023-10-13 21:33:13,611][60934] Updated weights for policy 1, policy_version 15762 (0.0008) [2023-10-13 21:33:13,979][60934] Updated weights for policy 1, policy_version 15772 (0.0009) [2023-10-13 21:33:15,778][60935] Updated weights for policy 0, policy_version 15620 (0.0009) [2023-10-13 21:33:16,146][60935] Updated weights for policy 0, policy_version 15630 (0.0008) [2023-10-13 21:33:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 32145408. Throughput: 0: 1656.4, 1: 1676.0. Samples: 8043752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:33:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.130')] [2023-10-13 21:33:16,519][60935] Updated weights for policy 0, policy_version 15640 (0.0008) [2023-10-13 21:33:18,124][60934] Updated weights for policy 1, policy_version 15782 (0.0009) [2023-10-13 21:33:18,498][60934] Updated weights for policy 1, policy_version 15792 (0.0010) [2023-10-13 21:33:18,863][60934] Updated weights for policy 1, policy_version 15802 (0.0010) [2023-10-13 21:33:20,548][60935] Updated weights for policy 0, policy_version 15650 (0.0008) [2023-10-13 21:33:20,924][60935] Updated weights for policy 0, policy_version 15660 (0.0008) [2023-10-13 21:33:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 32210944. Throughput: 0: 1655.8, 1: 1672.3. Samples: 8063684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:33:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.130')] [2023-10-13 21:33:21,303][60935] Updated weights for policy 0, policy_version 15670 (0.0007) [2023-10-13 21:33:21,672][60935] Updated weights for policy 0, policy_version 15680 (0.0009) [2023-10-13 21:33:23,148][60934] Updated weights for policy 1, policy_version 15812 (0.0010) [2023-10-13 21:33:23,549][60934] Updated weights for policy 1, policy_version 15822 (0.0007) [2023-10-13 21:33:23,909][60934] Updated weights for policy 1, policy_version 15832 (0.0007) [2023-10-13 21:33:25,873][60935] Updated weights for policy 0, policy_version 15690 (0.0009) [2023-10-13 21:33:26,238][60935] Updated weights for policy 0, policy_version 15700 (0.0010) [2023-10-13 21:33:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 32276480. Throughput: 0: 1651.5, 1: 1685.3. Samples: 8083684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:33:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.130')] [2023-10-13 21:33:26,612][60935] Updated weights for policy 0, policy_version 15710 (0.0011) [2023-10-13 21:33:27,954][60934] Updated weights for policy 1, policy_version 15842 (0.0008) [2023-10-13 21:33:28,330][60934] Updated weights for policy 1, policy_version 15852 (0.0011) [2023-10-13 21:33:28,707][60934] Updated weights for policy 1, policy_version 15862 (0.0011) [2023-10-13 21:33:29,072][60934] Updated weights for policy 1, policy_version 15872 (0.0011) [2023-10-13 21:33:30,756][60935] Updated weights for policy 0, policy_version 15720 (0.0008) [2023-10-13 21:33:31,126][60935] Updated weights for policy 0, policy_version 15730 (0.0009) [2023-10-13 21:33:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32342016. Throughput: 0: 1653.1, 1: 1670.1. Samples: 8093626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:33:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.130')] [2023-10-13 21:33:31,491][60935] Updated weights for policy 0, policy_version 15740 (0.0007) [2023-10-13 21:33:33,233][60934] Updated weights for policy 1, policy_version 15882 (0.0007) [2023-10-13 21:33:33,595][60934] Updated weights for policy 1, policy_version 15892 (0.0009) [2023-10-13 21:33:33,974][60934] Updated weights for policy 1, policy_version 15902 (0.0008) [2023-10-13 21:33:35,680][60935] Updated weights for policy 0, policy_version 15750 (0.0007) [2023-10-13 21:33:36,049][60935] Updated weights for policy 0, policy_version 15760 (0.0007) [2023-10-13 21:33:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 32407552. Throughput: 0: 1653.2, 1: 1672.2. Samples: 8113456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:33:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-13 21:33:36,426][60935] Updated weights for policy 0, policy_version 15770 (0.0009) [2023-10-13 21:33:38,078][60934] Updated weights for policy 1, policy_version 15912 (0.0009) [2023-10-13 21:33:38,451][60934] Updated weights for policy 1, policy_version 15922 (0.0009) [2023-10-13 21:33:38,826][60934] Updated weights for policy 1, policy_version 15932 (0.0008) [2023-10-13 21:33:40,446][60935] Updated weights for policy 0, policy_version 15780 (0.0008) [2023-10-13 21:33:40,811][60935] Updated weights for policy 0, policy_version 15790 (0.0008) [2023-10-13 21:33:41,180][60935] Updated weights for policy 0, policy_version 15800 (0.0010) [2023-10-13 21:33:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 32473088. Throughput: 0: 1645.1, 1: 1680.3. Samples: 8133398. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 21:33:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-13 21:33:42,873][60934] Updated weights for policy 1, policy_version 15942 (0.0007) [2023-10-13 21:33:43,253][60934] Updated weights for policy 1, policy_version 15952 (0.0008) [2023-10-13 21:33:43,616][60934] Updated weights for policy 1, policy_version 15962 (0.0008) [2023-10-13 21:33:45,300][60935] Updated weights for policy 0, policy_version 15810 (0.0009) [2023-10-13 21:33:45,661][60935] Updated weights for policy 0, policy_version 15820 (0.0009) [2023-10-13 21:33:46,041][60935] Updated weights for policy 0, policy_version 15830 (0.0011) [2023-10-13 21:33:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32538624. Throughput: 0: 1654.9, 1: 1664.5. Samples: 8143376. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 21:33:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-13 21:33:46,405][60935] Updated weights for policy 0, policy_version 15840 (0.0011) [2023-10-13 21:33:47,519][60934] Updated weights for policy 1, policy_version 15972 (0.0009) [2023-10-13 21:33:47,884][60934] Updated weights for policy 1, policy_version 15982 (0.0007) [2023-10-13 21:33:48,258][60934] Updated weights for policy 1, policy_version 15992 (0.0008) [2023-10-13 21:33:50,560][60935] Updated weights for policy 0, policy_version 15850 (0.0007) [2023-10-13 21:33:50,943][60935] Updated weights for policy 0, policy_version 15860 (0.0008) [2023-10-13 21:33:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 32604160. Throughput: 0: 1655.2, 1: 1688.1. Samples: 8163786. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 21:33:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:33:51,324][60935] Updated weights for policy 0, policy_version 15870 (0.0009) [2023-10-13 21:33:52,149][60934] Updated weights for policy 1, policy_version 16002 (0.0007) [2023-10-13 21:33:52,528][60934] Updated weights for policy 1, policy_version 16012 (0.0008) [2023-10-13 21:33:52,898][60934] Updated weights for policy 1, policy_version 16022 (0.0009) [2023-10-13 21:33:53,260][60934] Updated weights for policy 1, policy_version 16032 (0.0008) [2023-10-13 21:33:55,239][60935] Updated weights for policy 0, policy_version 15880 (0.0008) [2023-10-13 21:33:55,613][60935] Updated weights for policy 0, policy_version 15890 (0.0008) [2023-10-13 21:33:55,980][60935] Updated weights for policy 0, policy_version 15900 (0.0009) [2023-10-13 21:33:56,248][59943] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 32702464. Throughput: 0: 1642.1, 1: 1682.5. Samples: 8183596. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 21:33:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:33:57,330][60934] Updated weights for policy 1, policy_version 16042 (0.0009) [2023-10-13 21:33:57,705][60934] Updated weights for policy 1, policy_version 16052 (0.0011) [2023-10-13 21:33:58,070][60934] Updated weights for policy 1, policy_version 16062 (0.0007) [2023-10-13 21:34:00,260][60935] Updated weights for policy 0, policy_version 15910 (0.0010) [2023-10-13 21:34:00,625][60935] Updated weights for policy 0, policy_version 15920 (0.0010) [2023-10-13 21:34:00,998][60935] Updated weights for policy 0, policy_version 15930 (0.0010) [2023-10-13 21:34:01,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 32768000. Throughput: 0: 1660.3, 1: 1667.6. Samples: 8193508. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 21:34:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:34:02,226][60934] Updated weights for policy 1, policy_version 16072 (0.0008) [2023-10-13 21:34:02,605][60934] Updated weights for policy 1, policy_version 16082 (0.0008) [2023-10-13 21:34:02,981][60934] Updated weights for policy 1, policy_version 16092 (0.0008) [2023-10-13 21:34:05,118][60935] Updated weights for policy 0, policy_version 15940 (0.0008) [2023-10-13 21:34:05,490][60935] Updated weights for policy 0, policy_version 15950 (0.0009) [2023-10-13 21:34:05,862][60935] Updated weights for policy 0, policy_version 15960 (0.0009) [2023-10-13 21:34:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 32833536. Throughput: 0: 1662.1, 1: 1688.9. Samples: 8214480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:34:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:34:06,949][60934] Updated weights for policy 1, policy_version 16102 (0.0009) [2023-10-13 21:34:07,311][60934] Updated weights for policy 1, policy_version 16112 (0.0007) [2023-10-13 21:34:07,684][60934] Updated weights for policy 1, policy_version 16122 (0.0007) [2023-10-13 21:34:09,870][60935] Updated weights for policy 0, policy_version 15970 (0.0007) [2023-10-13 21:34:10,245][60935] Updated weights for policy 0, policy_version 15980 (0.0008) [2023-10-13 21:34:10,607][60935] Updated weights for policy 0, policy_version 15990 (0.0010) [2023-10-13 21:34:10,977][60935] Updated weights for policy 0, policy_version 16000 (0.0011) [2023-10-13 21:34:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 32899072. Throughput: 0: 1646.3, 1: 1700.0. Samples: 8234266. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:34:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:34:11,683][60934] Updated weights for policy 1, policy_version 16132 (0.0007) [2023-10-13 21:34:12,085][60934] Updated weights for policy 1, policy_version 16142 (0.0007) [2023-10-13 21:34:12,449][60934] Updated weights for policy 1, policy_version 16152 (0.0007) [2023-10-13 21:34:15,115][60935] Updated weights for policy 0, policy_version 16010 (0.0009) [2023-10-13 21:34:15,490][60935] Updated weights for policy 0, policy_version 16020 (0.0008) [2023-10-13 21:34:15,856][60935] Updated weights for policy 0, policy_version 16030 (0.0009) [2023-10-13 21:34:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 32964608. Throughput: 0: 1668.2, 1: 1680.2. Samples: 8244302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:34:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.130')] [2023-10-13 21:34:16,456][60934] Updated weights for policy 1, policy_version 16162 (0.0009) [2023-10-13 21:34:16,827][60934] Updated weights for policy 1, policy_version 16172 (0.0009) [2023-10-13 21:34:17,200][60934] Updated weights for policy 1, policy_version 16182 (0.0009) [2023-10-13 21:34:17,574][60934] Updated weights for policy 1, policy_version 16192 (0.0010) [2023-10-13 21:34:19,829][60935] Updated weights for policy 0, policy_version 16040 (0.0011) [2023-10-13 21:34:20,202][60935] Updated weights for policy 0, policy_version 16050 (0.0008) [2023-10-13 21:34:20,574][60935] Updated weights for policy 0, policy_version 16060 (0.0007) [2023-10-13 21:34:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 33030144. Throughput: 0: 1662.2, 1: 1703.4. Samples: 8264908. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-13 21:34:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.130')] [2023-10-13 21:34:21,466][60934] Updated weights for policy 1, policy_version 16202 (0.0008) [2023-10-13 21:34:21,835][60934] Updated weights for policy 1, policy_version 16212 (0.0007) [2023-10-13 21:34:22,201][60934] Updated weights for policy 1, policy_version 16222 (0.0008) [2023-10-13 21:34:24,538][60935] Updated weights for policy 0, policy_version 16070 (0.0008) [2023-10-13 21:34:24,903][60935] Updated weights for policy 0, policy_version 16080 (0.0007) [2023-10-13 21:34:25,280][60935] Updated weights for policy 0, policy_version 16090 (0.0007) [2023-10-13 21:34:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 33095680. Throughput: 0: 1662.8, 1: 1703.6. Samples: 8284884. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-13 21:34:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.130')] [2023-10-13 21:34:26,330][60934] Updated weights for policy 1, policy_version 16232 (0.0008) [2023-10-13 21:34:26,709][60934] Updated weights for policy 1, policy_version 16242 (0.0009) [2023-10-13 21:34:27,071][60934] Updated weights for policy 1, policy_version 16252 (0.0008) [2023-10-13 21:34:29,243][60935] Updated weights for policy 0, policy_version 16100 (0.0011) [2023-10-13 21:34:29,622][60935] Updated weights for policy 0, policy_version 16110 (0.0010) [2023-10-13 21:34:29,997][60935] Updated weights for policy 0, policy_version 16120 (0.0010) [2023-10-13 21:34:31,073][60934] Updated weights for policy 1, policy_version 16262 (0.0009) [2023-10-13 21:34:31,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 33161216. Throughput: 0: 1681.2, 1: 1694.2. Samples: 8295270. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-13 21:34:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.130')] [2023-10-13 21:34:31,447][60934] Updated weights for policy 1, policy_version 16272 (0.0009) [2023-10-13 21:34:31,818][60934] Updated weights for policy 1, policy_version 16282 (0.0008) [2023-10-13 21:34:34,295][60935] Updated weights for policy 0, policy_version 16130 (0.0008) [2023-10-13 21:34:34,670][60935] Updated weights for policy 0, policy_version 16140 (0.0009) [2023-10-13 21:34:35,037][60935] Updated weights for policy 0, policy_version 16150 (0.0009) [2023-10-13 21:34:35,414][60935] Updated weights for policy 0, policy_version 16160 (0.0009) [2023-10-13 21:34:35,712][60934] Updated weights for policy 1, policy_version 16292 (0.0008) [2023-10-13 21:34:36,076][60934] Updated weights for policy 1, policy_version 16302 (0.0008) [2023-10-13 21:34:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 33226752. Throughput: 0: 1664.0, 1: 1704.7. Samples: 8315376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:34:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.130')] [2023-10-13 21:34:36,450][60934] Updated weights for policy 1, policy_version 16312 (0.0007) [2023-10-13 21:34:39,586][60935] Updated weights for policy 0, policy_version 16170 (0.0009) [2023-10-13 21:34:39,960][60935] Updated weights for policy 0, policy_version 16180 (0.0009) [2023-10-13 21:34:40,325][60935] Updated weights for policy 0, policy_version 16190 (0.0010) [2023-10-13 21:34:40,569][60934] Updated weights for policy 1, policy_version 16322 (0.0007) [2023-10-13 21:34:40,949][60934] Updated weights for policy 1, policy_version 16332 (0.0007) [2023-10-13 21:34:41,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 33292288. Throughput: 0: 1664.4, 1: 1702.4. Samples: 8335102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:34:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:34:41,314][60934] Updated weights for policy 1, policy_version 16342 (0.0007) [2023-10-13 21:34:41,681][60934] Updated weights for policy 1, policy_version 16352 (0.0009) [2023-10-13 21:34:44,430][60935] Updated weights for policy 0, policy_version 16200 (0.0008) [2023-10-13 21:34:44,816][60935] Updated weights for policy 0, policy_version 16210 (0.0010) [2023-10-13 21:34:45,189][60935] Updated weights for policy 0, policy_version 16220 (0.0008) [2023-10-13 21:34:45,677][60934] Updated weights for policy 1, policy_version 16362 (0.0009) [2023-10-13 21:34:46,044][60934] Updated weights for policy 1, policy_version 16372 (0.0009) [2023-10-13 21:34:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 33357824. Throughput: 0: 1674.0, 1: 1704.9. Samples: 8345562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:34:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:34:46,425][60934] Updated weights for policy 1, policy_version 16382 (0.0009) [2023-10-13 21:34:49,318][60935] Updated weights for policy 0, policy_version 16230 (0.0008) [2023-10-13 21:34:49,693][60935] Updated weights for policy 0, policy_version 16240 (0.0007) [2023-10-13 21:34:50,063][60935] Updated weights for policy 0, policy_version 16250 (0.0009) [2023-10-13 21:34:50,566][60934] Updated weights for policy 1, policy_version 16392 (0.0008) [2023-10-13 21:34:50,939][60934] Updated weights for policy 1, policy_version 16402 (0.0008) [2023-10-13 21:34:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 33423360. Throughput: 0: 1648.6, 1: 1698.0. Samples: 8365078. Policy #0 lag: (min: 7.0, avg: 10.1, max: 39.0) [2023-10-13 21:34:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:34:51,297][60934] Updated weights for policy 1, policy_version 16412 (0.0009) [2023-10-13 21:34:54,204][60935] Updated weights for policy 0, policy_version 16260 (0.0008) [2023-10-13 21:34:54,566][60935] Updated weights for policy 0, policy_version 16270 (0.0008) [2023-10-13 21:34:54,943][60935] Updated weights for policy 0, policy_version 16280 (0.0009) [2023-10-13 21:34:55,391][60934] Updated weights for policy 1, policy_version 16422 (0.0008) [2023-10-13 21:34:55,758][60934] Updated weights for policy 1, policy_version 16432 (0.0008) [2023-10-13 21:34:56,132][60934] Updated weights for policy 1, policy_version 16442 (0.0008) [2023-10-13 21:34:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 33488896. Throughput: 0: 1661.9, 1: 1686.1. Samples: 8384930. Policy #0 lag: (min: 7.0, avg: 10.1, max: 39.0) [2023-10-13 21:34:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:34:56,262][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000016288_16678912.pth... [2023-10-13 21:34:56,291][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000014720_15073280.pth [2023-10-13 21:34:56,352][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000016448_16842752.pth... [2023-10-13 21:34:56,392][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000014848_15204352.pth [2023-10-13 21:34:59,109][60935] Updated weights for policy 0, policy_version 16290 (0.0008) [2023-10-13 21:34:59,477][60935] Updated weights for policy 0, policy_version 16300 (0.0008) [2023-10-13 21:34:59,846][60935] Updated weights for policy 0, policy_version 16310 (0.0008) [2023-10-13 21:35:00,213][60935] Updated weights for policy 0, policy_version 16320 (0.0008) [2023-10-13 21:35:00,214][60934] Updated weights for policy 1, policy_version 16452 (0.0008) [2023-10-13 21:35:00,622][60934] Updated weights for policy 1, policy_version 16462 (0.0009) [2023-10-13 21:35:00,987][60934] Updated weights for policy 1, policy_version 16472 (0.0007) [2023-10-13 21:35:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 33554432. Throughput: 0: 1665.9, 1: 1698.5. Samples: 8395698. Policy #0 lag: (min: 7.0, avg: 10.1, max: 39.0) [2023-10-13 21:35:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:35:04,317][60935] Updated weights for policy 0, policy_version 16330 (0.0009) [2023-10-13 21:35:04,684][60935] Updated weights for policy 0, policy_version 16340 (0.0009) [2023-10-13 21:35:05,056][60935] Updated weights for policy 0, policy_version 16350 (0.0010) [2023-10-13 21:35:05,067][60934] Updated weights for policy 1, policy_version 16482 (0.0007) [2023-10-13 21:35:05,444][60934] Updated weights for policy 1, policy_version 16492 (0.0008) [2023-10-13 21:35:05,809][60934] Updated weights for policy 1, policy_version 16502 (0.0009) [2023-10-13 21:35:06,173][60934] Updated weights for policy 1, policy_version 16512 (0.0009) [2023-10-13 21:35:06,248][59943] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 33652736. Throughput: 0: 1653.6, 1: 1688.6. Samples: 8415308. Policy #0 lag: (min: 17.0, avg: 26.5, max: 49.0) [2023-10-13 21:35:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:35:09,238][60935] Updated weights for policy 0, policy_version 16360 (0.0009) [2023-10-13 21:35:09,610][60935] Updated weights for policy 0, policy_version 16370 (0.0008) [2023-10-13 21:35:09,980][60935] Updated weights for policy 0, policy_version 16380 (0.0008) [2023-10-13 21:35:10,115][60934] Updated weights for policy 1, policy_version 16522 (0.0007) [2023-10-13 21:35:10,482][60934] Updated weights for policy 1, policy_version 16532 (0.0010) [2023-10-13 21:35:10,849][60934] Updated weights for policy 1, policy_version 16542 (0.0010) [2023-10-13 21:35:11,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 33718272. Throughput: 0: 1655.2, 1: 1677.6. Samples: 8434858. Policy #0 lag: (min: 17.0, avg: 26.5, max: 49.0) [2023-10-13 21:35:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:35:14,088][60935] Updated weights for policy 0, policy_version 16390 (0.0008) [2023-10-13 21:35:14,464][60935] Updated weights for policy 0, policy_version 16400 (0.0009) [2023-10-13 21:35:14,833][60935] Updated weights for policy 0, policy_version 16410 (0.0008) [2023-10-13 21:35:15,018][60934] Updated weights for policy 1, policy_version 16552 (0.0007) [2023-10-13 21:35:15,397][60934] Updated weights for policy 1, policy_version 16562 (0.0007) [2023-10-13 21:35:15,765][60934] Updated weights for policy 1, policy_version 16572 (0.0007) [2023-10-13 21:35:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 33783808. Throughput: 0: 1652.4, 1: 1693.1. Samples: 8445818. Policy #0 lag: (min: 17.0, avg: 26.5, max: 49.0) [2023-10-13 21:35:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:35:18,924][60935] Updated weights for policy 0, policy_version 16420 (0.0010) [2023-10-13 21:35:19,302][60935] Updated weights for policy 0, policy_version 16430 (0.0008) [2023-10-13 21:35:19,671][60935] Updated weights for policy 0, policy_version 16440 (0.0008) [2023-10-13 21:35:19,730][60934] Updated weights for policy 1, policy_version 16582 (0.0008) [2023-10-13 21:35:20,093][60934] Updated weights for policy 1, policy_version 16592 (0.0008) [2023-10-13 21:35:20,458][60934] Updated weights for policy 1, policy_version 16602 (0.0008) [2023-10-13 21:35:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 33849344. Throughput: 0: 1650.8, 1: 1684.1. Samples: 8465446. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-13 21:35:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:35:23,885][60935] Updated weights for policy 0, policy_version 16450 (0.0008) [2023-10-13 21:35:24,251][60935] Updated weights for policy 0, policy_version 16460 (0.0008) [2023-10-13 21:35:24,595][60934] Updated weights for policy 1, policy_version 16612 (0.0010) [2023-10-13 21:35:24,621][60935] Updated weights for policy 0, policy_version 16470 (0.0008) [2023-10-13 21:35:24,960][60934] Updated weights for policy 1, policy_version 16622 (0.0008) [2023-10-13 21:35:24,988][60935] Updated weights for policy 0, policy_version 16480 (0.0007) [2023-10-13 21:35:25,331][60934] Updated weights for policy 1, policy_version 16632 (0.0009) [2023-10-13 21:35:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 33914880. Throughput: 0: 1661.2, 1: 1661.7. Samples: 8484632. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-13 21:35:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:35:29,151][60935] Updated weights for policy 0, policy_version 16490 (0.0007) [2023-10-13 21:35:29,268][60934] Updated weights for policy 1, policy_version 16642 (0.0010) [2023-10-13 21:35:29,530][60935] Updated weights for policy 0, policy_version 16500 (0.0007) [2023-10-13 21:35:29,637][60934] Updated weights for policy 1, policy_version 16652 (0.0008) [2023-10-13 21:35:29,902][60935] Updated weights for policy 0, policy_version 16510 (0.0008) [2023-10-13 21:35:30,011][60934] Updated weights for policy 1, policy_version 16662 (0.0008) [2023-10-13 21:35:30,390][60934] Updated weights for policy 1, policy_version 16672 (0.0008) [2023-10-13 21:35:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 33980416. Throughput: 0: 1659.8, 1: 1688.3. Samples: 8496224. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-13 21:35:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:35:33,857][60935] Updated weights for policy 0, policy_version 16520 (0.0009) [2023-10-13 21:35:34,227][60935] Updated weights for policy 0, policy_version 16530 (0.0011) [2023-10-13 21:35:34,298][60934] Updated weights for policy 1, policy_version 16682 (0.0007) [2023-10-13 21:35:34,607][60935] Updated weights for policy 0, policy_version 16540 (0.0008) [2023-10-13 21:35:34,660][60934] Updated weights for policy 1, policy_version 16692 (0.0008) [2023-10-13 21:35:35,033][60934] Updated weights for policy 1, policy_version 16702 (0.0009) [2023-10-13 21:35:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 34045952. Throughput: 0: 1653.5, 1: 1681.6. Samples: 8515160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:35:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:35:38,912][60935] Updated weights for policy 0, policy_version 16550 (0.0009) [2023-10-13 21:35:39,081][60934] Updated weights for policy 1, policy_version 16712 (0.0007) [2023-10-13 21:35:39,289][60935] Updated weights for policy 0, policy_version 16560 (0.0008) [2023-10-13 21:35:39,455][60934] Updated weights for policy 1, policy_version 16722 (0.0008) [2023-10-13 21:35:39,656][60935] Updated weights for policy 0, policy_version 16570 (0.0007) [2023-10-13 21:35:39,823][60934] Updated weights for policy 1, policy_version 16732 (0.0009) [2023-10-13 21:35:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 34111488. Throughput: 0: 1664.0, 1: 1672.1. Samples: 8535054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:35:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:35:43,654][60935] Updated weights for policy 0, policy_version 16580 (0.0007) [2023-10-13 21:35:43,907][60934] Updated weights for policy 1, policy_version 16742 (0.0009) [2023-10-13 21:35:44,028][60935] Updated weights for policy 0, policy_version 16590 (0.0007) [2023-10-13 21:35:44,276][60934] Updated weights for policy 1, policy_version 16752 (0.0010) [2023-10-13 21:35:44,389][60935] Updated weights for policy 0, policy_version 16600 (0.0008) [2023-10-13 21:35:44,646][60934] Updated weights for policy 1, policy_version 16762 (0.0009) [2023-10-13 21:35:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 34177024. Throughput: 0: 1648.9, 1: 1691.5. Samples: 8546014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:35:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:35:48,545][60935] Updated weights for policy 0, policy_version 16610 (0.0008) [2023-10-13 21:35:48,896][60934] Updated weights for policy 1, policy_version 16772 (0.0008) [2023-10-13 21:35:48,924][60935] Updated weights for policy 0, policy_version 16620 (0.0008) [2023-10-13 21:35:49,304][60935] Updated weights for policy 0, policy_version 16630 (0.0009) [2023-10-13 21:35:49,308][60934] Updated weights for policy 1, policy_version 16782 (0.0007) [2023-10-13 21:35:49,668][60935] Updated weights for policy 0, policy_version 16640 (0.0010) [2023-10-13 21:35:49,672][60934] Updated weights for policy 1, policy_version 16792 (0.0008) [2023-10-13 21:35:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 34242560. Throughput: 0: 1641.1, 1: 1672.4. Samples: 8564414. Policy #0 lag: (min: 16.0, avg: 43.4, max: 48.0) [2023-10-13 21:35:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:35:53,739][60934] Updated weights for policy 1, policy_version 16802 (0.0009) [2023-10-13 21:35:53,944][60935] Updated weights for policy 0, policy_version 16650 (0.0009) [2023-10-13 21:35:54,113][60934] Updated weights for policy 1, policy_version 16812 (0.0007) [2023-10-13 21:35:54,312][60935] Updated weights for policy 0, policy_version 16660 (0.0008) [2023-10-13 21:35:54,476][60934] Updated weights for policy 1, policy_version 16822 (0.0007) [2023-10-13 21:35:54,686][60935] Updated weights for policy 0, policy_version 16670 (0.0008) [2023-10-13 21:35:54,841][60934] Updated weights for policy 1, policy_version 16832 (0.0008) [2023-10-13 21:35:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 34308096. Throughput: 0: 1650.9, 1: 1673.6. Samples: 8584462. Policy #0 lag: (min: 16.0, avg: 43.4, max: 48.0) [2023-10-13 21:35:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:35:58,904][60935] Updated weights for policy 0, policy_version 16680 (0.0009) [2023-10-13 21:35:58,931][60934] Updated weights for policy 1, policy_version 16842 (0.0009) [2023-10-13 21:35:59,276][60935] Updated weights for policy 0, policy_version 16690 (0.0008) [2023-10-13 21:35:59,295][60934] Updated weights for policy 1, policy_version 16852 (0.0008) [2023-10-13 21:35:59,646][60935] Updated weights for policy 0, policy_version 16700 (0.0008) [2023-10-13 21:35:59,668][60934] Updated weights for policy 1, policy_version 16862 (0.0009) [2023-10-13 21:36:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 34373632. Throughput: 0: 1641.5, 1: 1688.7. Samples: 8595678. Policy #0 lag: (min: 16.0, avg: 43.4, max: 48.0) [2023-10-13 21:36:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:36:03,718][60934] Updated weights for policy 1, policy_version 16872 (0.0008) [2023-10-13 21:36:03,828][60935] Updated weights for policy 0, policy_version 16710 (0.0009) [2023-10-13 21:36:04,082][60934] Updated weights for policy 1, policy_version 16882 (0.0008) [2023-10-13 21:36:04,201][60935] Updated weights for policy 0, policy_version 16720 (0.0008) [2023-10-13 21:36:04,460][60934] Updated weights for policy 1, policy_version 16892 (0.0009) [2023-10-13 21:36:04,572][60935] Updated weights for policy 0, policy_version 16730 (0.0007) [2023-10-13 21:36:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 34439168. Throughput: 0: 1642.4, 1: 1662.4. Samples: 8614162. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-13 21:36:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:36:08,553][60934] Updated weights for policy 1, policy_version 16902 (0.0009) [2023-10-13 21:36:08,680][60935] Updated weights for policy 0, policy_version 16740 (0.0010) [2023-10-13 21:36:08,921][60934] Updated weights for policy 1, policy_version 16912 (0.0009) [2023-10-13 21:36:09,058][60935] Updated weights for policy 0, policy_version 16750 (0.0008) [2023-10-13 21:36:09,285][60934] Updated weights for policy 1, policy_version 16922 (0.0008) [2023-10-13 21:36:09,433][60935] Updated weights for policy 0, policy_version 16760 (0.0008) [2023-10-13 21:36:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 34504704. Throughput: 0: 1649.2, 1: 1683.6. Samples: 8634610. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-13 21:36:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:36:13,312][60934] Updated weights for policy 1, policy_version 16932 (0.0009) [2023-10-13 21:36:13,574][60935] Updated weights for policy 0, policy_version 16770 (0.0010) [2023-10-13 21:36:13,686][60934] Updated weights for policy 1, policy_version 16942 (0.0008) [2023-10-13 21:36:13,941][60935] Updated weights for policy 0, policy_version 16780 (0.0008) [2023-10-13 21:36:14,045][60934] Updated weights for policy 1, policy_version 16952 (0.0008) [2023-10-13 21:36:14,310][60935] Updated weights for policy 0, policy_version 16790 (0.0008) [2023-10-13 21:36:14,679][60935] Updated weights for policy 0, policy_version 16800 (0.0007) [2023-10-13 21:36:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 34570240. Throughput: 0: 1639.8, 1: 1672.1. Samples: 8645260. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-13 21:36:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:36:18,076][60934] Updated weights for policy 1, policy_version 16962 (0.0009) [2023-10-13 21:36:18,437][60934] Updated weights for policy 1, policy_version 16972 (0.0009) [2023-10-13 21:36:18,803][60934] Updated weights for policy 1, policy_version 16982 (0.0008) [2023-10-13 21:36:18,910][60935] Updated weights for policy 0, policy_version 16810 (0.0007) [2023-10-13 21:36:19,170][60934] Updated weights for policy 1, policy_version 16992 (0.0007) [2023-10-13 21:36:19,286][60935] Updated weights for policy 0, policy_version 16820 (0.0009) [2023-10-13 21:36:19,649][60935] Updated weights for policy 0, policy_version 16830 (0.0009) [2023-10-13 21:36:21,249][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 34635776. Throughput: 0: 1645.0, 1: 1667.8. Samples: 8664238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:36:21,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:36:23,381][60934] Updated weights for policy 1, policy_version 17002 (0.0007) [2023-10-13 21:36:23,747][60934] Updated weights for policy 1, policy_version 17012 (0.0007) [2023-10-13 21:36:23,910][60935] Updated weights for policy 0, policy_version 16840 (0.0009) [2023-10-13 21:36:24,107][60934] Updated weights for policy 1, policy_version 17022 (0.0008) [2023-10-13 21:36:24,286][60935] Updated weights for policy 0, policy_version 16850 (0.0008) [2023-10-13 21:36:24,651][60935] Updated weights for policy 0, policy_version 16860 (0.0009) [2023-10-13 21:36:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 34701312. Throughput: 0: 1643.0, 1: 1683.1. Samples: 8684728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:36:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:36:28,251][60934] Updated weights for policy 1, policy_version 17032 (0.0008) [2023-10-13 21:36:28,611][60934] Updated weights for policy 1, policy_version 17042 (0.0007) [2023-10-13 21:36:28,703][60935] Updated weights for policy 0, policy_version 16870 (0.0009) [2023-10-13 21:36:28,981][60934] Updated weights for policy 1, policy_version 17052 (0.0007) [2023-10-13 21:36:29,069][60935] Updated weights for policy 0, policy_version 16880 (0.0012) [2023-10-13 21:36:29,432][60935] Updated weights for policy 0, policy_version 16890 (0.0012) [2023-10-13 21:36:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 34766848. Throughput: 0: 1648.2, 1: 1671.5. Samples: 8695404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:36:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:36:32,905][60934] Updated weights for policy 1, policy_version 17062 (0.0008) [2023-10-13 21:36:33,275][60934] Updated weights for policy 1, policy_version 17072 (0.0007) [2023-10-13 21:36:33,566][60935] Updated weights for policy 0, policy_version 16900 (0.0009) [2023-10-13 21:36:33,639][60934] Updated weights for policy 1, policy_version 17082 (0.0007) [2023-10-13 21:36:33,939][60935] Updated weights for policy 0, policy_version 16910 (0.0010) [2023-10-13 21:36:34,304][60935] Updated weights for policy 0, policy_version 16920 (0.0008) [2023-10-13 21:36:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 34832384. Throughput: 0: 1650.8, 1: 1683.2. Samples: 8714444. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-13 21:36:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:36:37,702][60934] Updated weights for policy 1, policy_version 17092 (0.0009) [2023-10-13 21:36:38,094][60934] Updated weights for policy 1, policy_version 17102 (0.0008) [2023-10-13 21:36:38,249][60935] Updated weights for policy 0, policy_version 16930 (0.0010) [2023-10-13 21:36:38,463][60934] Updated weights for policy 1, policy_version 17112 (0.0008) [2023-10-13 21:36:38,623][60935] Updated weights for policy 0, policy_version 16940 (0.0009) [2023-10-13 21:36:38,987][60935] Updated weights for policy 0, policy_version 16950 (0.0009) [2023-10-13 21:36:39,353][60935] Updated weights for policy 0, policy_version 16960 (0.0010) [2023-10-13 21:36:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 34897920. Throughput: 0: 1657.4, 1: 1690.0. Samples: 8735096. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-13 21:36:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:36:42,390][60934] Updated weights for policy 1, policy_version 17122 (0.0007) [2023-10-13 21:36:42,753][60934] Updated weights for policy 1, policy_version 17132 (0.0007) [2023-10-13 21:36:43,118][60934] Updated weights for policy 1, policy_version 17142 (0.0009) [2023-10-13 21:36:43,480][60934] Updated weights for policy 1, policy_version 17152 (0.0008) [2023-10-13 21:36:43,601][60935] Updated weights for policy 0, policy_version 16970 (0.0007) [2023-10-13 21:36:43,984][60935] Updated weights for policy 0, policy_version 16980 (0.0009) [2023-10-13 21:36:44,352][60935] Updated weights for policy 0, policy_version 16990 (0.0009) [2023-10-13 21:36:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 34963456. Throughput: 0: 1654.2, 1: 1661.1. Samples: 8744866. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-13 21:36:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:36:47,500][60934] Updated weights for policy 1, policy_version 17162 (0.0007) [2023-10-13 21:36:47,871][60934] Updated weights for policy 1, policy_version 17172 (0.0008) [2023-10-13 21:36:48,236][60934] Updated weights for policy 1, policy_version 17182 (0.0011) [2023-10-13 21:36:48,435][60935] Updated weights for policy 0, policy_version 17000 (0.0010) [2023-10-13 21:36:48,812][60935] Updated weights for policy 0, policy_version 17010 (0.0009) [2023-10-13 21:36:49,183][60935] Updated weights for policy 0, policy_version 17020 (0.0011) [2023-10-13 21:36:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 35028992. Throughput: 0: 1655.3, 1: 1691.4. Samples: 8764764. Policy #0 lag: (min: 7.0, avg: 15.0, max: 39.0) [2023-10-13 21:36:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:36:52,114][60934] Updated weights for policy 1, policy_version 17192 (0.0008) [2023-10-13 21:36:52,490][60934] Updated weights for policy 1, policy_version 17202 (0.0008) [2023-10-13 21:36:52,856][60934] Updated weights for policy 1, policy_version 17212 (0.0007) [2023-10-13 21:36:53,371][60935] Updated weights for policy 0, policy_version 17030 (0.0010) [2023-10-13 21:36:53,738][60935] Updated weights for policy 0, policy_version 17040 (0.0010) [2023-10-13 21:36:54,105][60935] Updated weights for policy 0, policy_version 17050 (0.0007) [2023-10-13 21:36:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 35094528. Throughput: 0: 1657.6, 1: 1693.6. Samples: 8785410. Policy #0 lag: (min: 7.0, avg: 15.0, max: 39.0) [2023-10-13 21:36:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:36:56,257][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000017056_17465344.pth... [2023-10-13 21:36:56,258][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000017216_17629184.pth... [2023-10-13 21:36:56,288][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000015520_15892480.pth [2023-10-13 21:36:56,295][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000015648_16023552.pth [2023-10-13 21:36:56,905][60934] Updated weights for policy 1, policy_version 17222 (0.0007) [2023-10-13 21:36:57,272][60934] Updated weights for policy 1, policy_version 17232 (0.0009) [2023-10-13 21:36:57,635][60934] Updated weights for policy 1, policy_version 17242 (0.0008) [2023-10-13 21:36:58,105][60935] Updated weights for policy 0, policy_version 17060 (0.0008) [2023-10-13 21:36:58,472][60935] Updated weights for policy 0, policy_version 17070 (0.0009) [2023-10-13 21:36:58,847][60935] Updated weights for policy 0, policy_version 17080 (0.0007) [2023-10-13 21:37:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 35160064. Throughput: 0: 1649.2, 1: 1675.6. Samples: 8794878. Policy #0 lag: (min: 7.0, avg: 15.0, max: 39.0) [2023-10-13 21:37:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:37:01,624][60934] Updated weights for policy 1, policy_version 17252 (0.0008) [2023-10-13 21:37:01,992][60934] Updated weights for policy 1, policy_version 17262 (0.0010) [2023-10-13 21:37:02,362][60934] Updated weights for policy 1, policy_version 17272 (0.0010) [2023-10-13 21:37:02,901][60935] Updated weights for policy 0, policy_version 17090 (0.0009) [2023-10-13 21:37:03,271][60935] Updated weights for policy 0, policy_version 17100 (0.0008) [2023-10-13 21:37:03,646][60935] Updated weights for policy 0, policy_version 17110 (0.0008) [2023-10-13 21:37:04,026][60935] Updated weights for policy 0, policy_version 17120 (0.0008) [2023-10-13 21:37:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 35225600. Throughput: 0: 1662.9, 1: 1693.0. Samples: 8815256. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 21:37:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:37:06,375][60934] Updated weights for policy 1, policy_version 17282 (0.0010) [2023-10-13 21:37:06,751][60934] Updated weights for policy 1, policy_version 17292 (0.0009) [2023-10-13 21:37:07,133][60934] Updated weights for policy 1, policy_version 17302 (0.0009) [2023-10-13 21:37:07,490][60934] Updated weights for policy 1, policy_version 17312 (0.0008) [2023-10-13 21:37:08,169][60935] Updated weights for policy 0, policy_version 17130 (0.0010) [2023-10-13 21:37:08,554][60935] Updated weights for policy 0, policy_version 17140 (0.0009) [2023-10-13 21:37:08,924][60935] Updated weights for policy 0, policy_version 17150 (0.0007) [2023-10-13 21:37:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 35291136. Throughput: 0: 1661.7, 1: 1696.0. Samples: 8835828. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 21:37:11,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:37:11,608][60934] Updated weights for policy 1, policy_version 17322 (0.0007) [2023-10-13 21:37:11,980][60934] Updated weights for policy 1, policy_version 17332 (0.0008) [2023-10-13 21:37:12,347][60934] Updated weights for policy 1, policy_version 17342 (0.0007) [2023-10-13 21:37:13,101][60935] Updated weights for policy 0, policy_version 17160 (0.0008) [2023-10-13 21:37:13,485][60935] Updated weights for policy 0, policy_version 17170 (0.0009) [2023-10-13 21:37:13,861][60935] Updated weights for policy 0, policy_version 17180 (0.0009) [2023-10-13 21:37:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 35356672. Throughput: 0: 1644.8, 1: 1685.0. Samples: 8845244. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 21:37:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:37:16,452][60934] Updated weights for policy 1, policy_version 17352 (0.0009) [2023-10-13 21:37:16,819][60934] Updated weights for policy 1, policy_version 17362 (0.0008) [2023-10-13 21:37:17,195][60934] Updated weights for policy 1, policy_version 17372 (0.0008) [2023-10-13 21:37:17,786][60935] Updated weights for policy 0, policy_version 17190 (0.0008) [2023-10-13 21:37:18,162][60935] Updated weights for policy 0, policy_version 17200 (0.0008) [2023-10-13 21:37:18,531][60935] Updated weights for policy 0, policy_version 17210 (0.0007) [2023-10-13 21:37:21,134][60934] Updated weights for policy 1, policy_version 17382 (0.0008) [2023-10-13 21:37:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 35422208. Throughput: 0: 1666.8, 1: 1702.7. Samples: 8866072. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 21:37:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:37:21,504][60934] Updated weights for policy 1, policy_version 17392 (0.0008) [2023-10-13 21:37:21,882][60934] Updated weights for policy 1, policy_version 17402 (0.0008) [2023-10-13 21:37:22,601][60935] Updated weights for policy 0, policy_version 17220 (0.0008) [2023-10-13 21:37:22,979][60935] Updated weights for policy 0, policy_version 17230 (0.0009) [2023-10-13 21:37:23,351][60935] Updated weights for policy 0, policy_version 17240 (0.0009) [2023-10-13 21:37:25,823][60934] Updated weights for policy 1, policy_version 17412 (0.0008) [2023-10-13 21:37:26,231][60934] Updated weights for policy 1, policy_version 17422 (0.0008) [2023-10-13 21:37:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 35487744. Throughput: 0: 1662.3, 1: 1716.9. Samples: 8887160. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 21:37:26,249][59943] Avg episode reward: [(0, '-0.070'), (1, '0.000')] [2023-10-13 21:37:26,606][60934] Updated weights for policy 1, policy_version 17432 (0.0008) [2023-10-13 21:37:27,593][60935] Updated weights for policy 0, policy_version 17250 (0.0008) [2023-10-13 21:37:27,964][60935] Updated weights for policy 0, policy_version 17260 (0.0009) [2023-10-13 21:37:28,331][60935] Updated weights for policy 0, policy_version 17270 (0.0008) [2023-10-13 21:37:28,698][60935] Updated weights for policy 0, policy_version 17280 (0.0008) [2023-10-13 21:37:30,622][60934] Updated weights for policy 1, policy_version 17442 (0.0007) [2023-10-13 21:37:30,992][60934] Updated weights for policy 1, policy_version 17452 (0.0010) [2023-10-13 21:37:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 35553280. Throughput: 0: 1647.2, 1: 1712.8. Samples: 8896068. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 21:37:31,249][59943] Avg episode reward: [(0, '-0.020'), (1, '0.000')] [2023-10-13 21:37:31,358][60934] Updated weights for policy 1, policy_version 17462 (0.0009) [2023-10-13 21:37:31,728][60934] Updated weights for policy 1, policy_version 17472 (0.0011) [2023-10-13 21:37:32,876][60935] Updated weights for policy 0, policy_version 17290 (0.0009) [2023-10-13 21:37:33,240][60935] Updated weights for policy 0, policy_version 17300 (0.0008) [2023-10-13 21:37:33,609][60935] Updated weights for policy 0, policy_version 17310 (0.0008) [2023-10-13 21:37:35,838][60934] Updated weights for policy 1, policy_version 17482 (0.0010) [2023-10-13 21:37:36,199][60934] Updated weights for policy 1, policy_version 17492 (0.0010) [2023-10-13 21:37:36,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 35618816. Throughput: 0: 1666.1, 1: 1709.8. Samples: 8916678. Policy #0 lag: (min: 10.0, avg: 10.1, max: 17.0) [2023-10-13 21:37:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:37:36,562][60934] Updated weights for policy 1, policy_version 17502 (0.0009) [2023-10-13 21:37:37,631][60935] Updated weights for policy 0, policy_version 17320 (0.0010) [2023-10-13 21:37:37,997][60935] Updated weights for policy 0, policy_version 17330 (0.0010) [2023-10-13 21:37:38,368][60935] Updated weights for policy 0, policy_version 17340 (0.0009) [2023-10-13 21:37:40,475][60934] Updated weights for policy 1, policy_version 17512 (0.0010) [2023-10-13 21:37:40,844][60934] Updated weights for policy 1, policy_version 17522 (0.0008) [2023-10-13 21:37:41,209][60934] Updated weights for policy 1, policy_version 17532 (0.0011) [2023-10-13 21:37:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 35684352. Throughput: 0: 1668.3, 1: 1702.3. Samples: 8937086. Policy #0 lag: (min: 10.0, avg: 10.1, max: 17.0) [2023-10-13 21:37:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:37:42,284][60935] Updated weights for policy 0, policy_version 17350 (0.0010) [2023-10-13 21:37:42,659][60935] Updated weights for policy 0, policy_version 17360 (0.0009) [2023-10-13 21:37:43,038][60935] Updated weights for policy 0, policy_version 17370 (0.0010) [2023-10-13 21:37:45,396][60934] Updated weights for policy 1, policy_version 17542 (0.0011) [2023-10-13 21:37:45,776][60934] Updated weights for policy 1, policy_version 17552 (0.0008) [2023-10-13 21:37:46,150][60934] Updated weights for policy 1, policy_version 17562 (0.0008) [2023-10-13 21:37:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 35749888. Throughput: 0: 1657.4, 1: 1713.7. Samples: 8946578. Policy #0 lag: (min: 10.0, avg: 10.1, max: 17.0) [2023-10-13 21:37:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:37:47,157][60935] Updated weights for policy 0, policy_version 17380 (0.0010) [2023-10-13 21:37:47,524][60935] Updated weights for policy 0, policy_version 17390 (0.0008) [2023-10-13 21:37:47,882][60935] Updated weights for policy 0, policy_version 17400 (0.0007) [2023-10-13 21:37:50,224][60934] Updated weights for policy 1, policy_version 17572 (0.0008) [2023-10-13 21:37:50,592][60934] Updated weights for policy 1, policy_version 17582 (0.0010) [2023-10-13 21:37:50,965][60934] Updated weights for policy 1, policy_version 17592 (0.0010) [2023-10-13 21:37:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 35815424. Throughput: 0: 1667.7, 1: 1709.5. Samples: 8967232. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-13 21:37:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:37:52,151][60935] Updated weights for policy 0, policy_version 17410 (0.0008) [2023-10-13 21:37:52,522][60935] Updated weights for policy 0, policy_version 17420 (0.0008) [2023-10-13 21:37:52,900][60935] Updated weights for policy 0, policy_version 17430 (0.0008) [2023-10-13 21:37:53,273][60935] Updated weights for policy 0, policy_version 17440 (0.0008) [2023-10-13 21:37:55,081][60934] Updated weights for policy 1, policy_version 17602 (0.0007) [2023-10-13 21:37:55,449][60934] Updated weights for policy 1, policy_version 17612 (0.0008) [2023-10-13 21:37:55,811][60934] Updated weights for policy 1, policy_version 17622 (0.0008) [2023-10-13 21:37:56,181][60934] Updated weights for policy 1, policy_version 17632 (0.0007) [2023-10-13 21:37:56,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 35913728. Throughput: 0: 1674.1, 1: 1692.7. Samples: 8987330. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-13 21:37:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:37:57,348][60935] Updated weights for policy 0, policy_version 17450 (0.0011) [2023-10-13 21:37:57,725][60935] Updated weights for policy 0, policy_version 17460 (0.0010) [2023-10-13 21:37:58,103][60935] Updated weights for policy 0, policy_version 17470 (0.0009) [2023-10-13 21:38:00,217][60934] Updated weights for policy 1, policy_version 17642 (0.0007) [2023-10-13 21:38:00,575][60934] Updated weights for policy 1, policy_version 17652 (0.0010) [2023-10-13 21:38:00,953][60934] Updated weights for policy 1, policy_version 17662 (0.0007) [2023-10-13 21:38:01,248][59943] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 35979264. Throughput: 0: 1667.1, 1: 1702.3. Samples: 8996868. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-13 21:38:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:38:02,020][60935] Updated weights for policy 0, policy_version 17480 (0.0009) [2023-10-13 21:38:02,396][60935] Updated weights for policy 0, policy_version 17490 (0.0010) [2023-10-13 21:38:02,765][60935] Updated weights for policy 0, policy_version 17500 (0.0009) [2023-10-13 21:38:04,854][60934] Updated weights for policy 1, policy_version 17672 (0.0008) [2023-10-13 21:38:05,217][60934] Updated weights for policy 1, policy_version 17682 (0.0009) [2023-10-13 21:38:05,586][60934] Updated weights for policy 1, policy_version 17692 (0.0007) [2023-10-13 21:38:06,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 36044800. Throughput: 0: 1674.4, 1: 1700.3. Samples: 9017932. Policy #0 lag: (min: 9.0, avg: 20.7, max: 41.0) [2023-10-13 21:38:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:38:06,900][60935] Updated weights for policy 0, policy_version 17510 (0.0009) [2023-10-13 21:38:07,277][60935] Updated weights for policy 0, policy_version 17520 (0.0009) [2023-10-13 21:38:07,645][60935] Updated weights for policy 0, policy_version 17530 (0.0010) [2023-10-13 21:38:09,563][60934] Updated weights for policy 1, policy_version 17702 (0.0007) [2023-10-13 21:38:09,936][60934] Updated weights for policy 1, policy_version 17712 (0.0009) [2023-10-13 21:38:10,296][60934] Updated weights for policy 1, policy_version 17722 (0.0011) [2023-10-13 21:38:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 36110336. Throughput: 0: 1670.0, 1: 1669.6. Samples: 9037442. Policy #0 lag: (min: 9.0, avg: 20.7, max: 41.0) [2023-10-13 21:38:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:38:11,953][60935] Updated weights for policy 0, policy_version 17540 (0.0010) [2023-10-13 21:38:12,325][60935] Updated weights for policy 0, policy_version 17550 (0.0008) [2023-10-13 21:38:12,690][60935] Updated weights for policy 0, policy_version 17560 (0.0008) [2023-10-13 21:38:14,194][60934] Updated weights for policy 1, policy_version 17732 (0.0008) [2023-10-13 21:38:14,590][60934] Updated weights for policy 1, policy_version 17742 (0.0009) [2023-10-13 21:38:14,954][60934] Updated weights for policy 1, policy_version 17752 (0.0009) [2023-10-13 21:38:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 36175872. Throughput: 0: 1668.2, 1: 1698.7. Samples: 9047580. Policy #0 lag: (min: 9.0, avg: 20.7, max: 41.0) [2023-10-13 21:38:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:38:16,823][60935] Updated weights for policy 0, policy_version 17570 (0.0008) [2023-10-13 21:38:17,201][60935] Updated weights for policy 0, policy_version 17580 (0.0008) [2023-10-13 21:38:17,569][60935] Updated weights for policy 0, policy_version 17590 (0.0008) [2023-10-13 21:38:17,942][60935] Updated weights for policy 0, policy_version 17600 (0.0010) [2023-10-13 21:38:19,113][60934] Updated weights for policy 1, policy_version 17762 (0.0010) [2023-10-13 21:38:19,484][60934] Updated weights for policy 1, policy_version 17772 (0.0008) [2023-10-13 21:38:19,860][60934] Updated weights for policy 1, policy_version 17782 (0.0008) [2023-10-13 21:38:20,231][60934] Updated weights for policy 1, policy_version 17792 (0.0009) [2023-10-13 21:38:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 36241408. Throughput: 0: 1667.2, 1: 1688.3. Samples: 9067680. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-13 21:38:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:38:22,002][60935] Updated weights for policy 0, policy_version 17610 (0.0011) [2023-10-13 21:38:22,356][60935] Updated weights for policy 0, policy_version 17620 (0.0011) [2023-10-13 21:38:22,733][60935] Updated weights for policy 0, policy_version 17630 (0.0009) [2023-10-13 21:38:24,244][60934] Updated weights for policy 1, policy_version 17802 (0.0007) [2023-10-13 21:38:24,613][60934] Updated weights for policy 1, policy_version 17812 (0.0007) [2023-10-13 21:38:24,977][60934] Updated weights for policy 1, policy_version 17822 (0.0007) [2023-10-13 21:38:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 36306944. Throughput: 0: 1667.8, 1: 1681.2. Samples: 9087790. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-13 21:38:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:38:26,817][60935] Updated weights for policy 0, policy_version 17640 (0.0009) [2023-10-13 21:38:27,197][60935] Updated weights for policy 0, policy_version 17650 (0.0008) [2023-10-13 21:38:27,560][60935] Updated weights for policy 0, policy_version 17660 (0.0009) [2023-10-13 21:38:28,861][60934] Updated weights for policy 1, policy_version 17832 (0.0007) [2023-10-13 21:38:29,224][60934] Updated weights for policy 1, policy_version 17842 (0.0007) [2023-10-13 21:38:29,585][60934] Updated weights for policy 1, policy_version 17852 (0.0007) [2023-10-13 21:38:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 36372480. Throughput: 0: 1668.4, 1: 1701.7. Samples: 9098234. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-13 21:38:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:38:31,663][60935] Updated weights for policy 0, policy_version 17670 (0.0008) [2023-10-13 21:38:32,031][60935] Updated weights for policy 0, policy_version 17680 (0.0010) [2023-10-13 21:38:32,400][60935] Updated weights for policy 0, policy_version 17690 (0.0010) [2023-10-13 21:38:33,753][60934] Updated weights for policy 1, policy_version 17862 (0.0007) [2023-10-13 21:38:34,111][60934] Updated weights for policy 1, policy_version 17872 (0.0007) [2023-10-13 21:38:34,485][60934] Updated weights for policy 1, policy_version 17882 (0.0009) [2023-10-13 21:38:36,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 36438016. Throughput: 0: 1670.5, 1: 1676.5. Samples: 9117850. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 21:38:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:38:36,495][60935] Updated weights for policy 0, policy_version 17700 (0.0008) [2023-10-13 21:38:36,862][60935] Updated weights for policy 0, policy_version 17710 (0.0010) [2023-10-13 21:38:37,232][60935] Updated weights for policy 0, policy_version 17720 (0.0009) [2023-10-13 21:38:38,575][60934] Updated weights for policy 1, policy_version 17892 (0.0008) [2023-10-13 21:38:38,938][60934] Updated weights for policy 1, policy_version 17902 (0.0008) [2023-10-13 21:38:39,313][60934] Updated weights for policy 1, policy_version 17912 (0.0008) [2023-10-13 21:38:41,214][60935] Updated weights for policy 0, policy_version 17730 (0.0008) [2023-10-13 21:38:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 36503552. Throughput: 0: 1673.7, 1: 1683.7. Samples: 9138414. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 21:38:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:38:41,576][60935] Updated weights for policy 0, policy_version 17740 (0.0008) [2023-10-13 21:38:41,948][60935] Updated weights for policy 0, policy_version 17750 (0.0008) [2023-10-13 21:38:42,312][60935] Updated weights for policy 0, policy_version 17760 (0.0011) [2023-10-13 21:38:43,503][60934] Updated weights for policy 1, policy_version 17922 (0.0008) [2023-10-13 21:38:43,869][60934] Updated weights for policy 1, policy_version 17932 (0.0009) [2023-10-13 21:38:44,237][60934] Updated weights for policy 1, policy_version 17942 (0.0007) [2023-10-13 21:38:44,612][60934] Updated weights for policy 1, policy_version 17952 (0.0010) [2023-10-13 21:38:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 36569088. Throughput: 0: 1677.0, 1: 1695.3. Samples: 9148624. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 21:38:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:38:46,472][60935] Updated weights for policy 0, policy_version 17770 (0.0010) [2023-10-13 21:38:46,837][60935] Updated weights for policy 0, policy_version 17780 (0.0010) [2023-10-13 21:38:47,201][60935] Updated weights for policy 0, policy_version 17790 (0.0010) [2023-10-13 21:38:48,606][60934] Updated weights for policy 1, policy_version 17962 (0.0009) [2023-10-13 21:38:48,975][60934] Updated weights for policy 1, policy_version 17972 (0.0009) [2023-10-13 21:38:49,349][60934] Updated weights for policy 1, policy_version 17982 (0.0009) [2023-10-13 21:38:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 36634624. Throughput: 0: 1670.5, 1: 1670.4. Samples: 9168270. Policy #0 lag: (min: 10.0, avg: 11.6, max: 31.0) [2023-10-13 21:38:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:38:51,503][60935] Updated weights for policy 0, policy_version 17800 (0.0009) [2023-10-13 21:38:51,875][60935] Updated weights for policy 0, policy_version 17810 (0.0008) [2023-10-13 21:38:52,244][60935] Updated weights for policy 0, policy_version 17820 (0.0008) [2023-10-13 21:38:53,402][60934] Updated weights for policy 1, policy_version 17992 (0.0007) [2023-10-13 21:38:53,772][60934] Updated weights for policy 1, policy_version 18002 (0.0007) [2023-10-13 21:38:54,148][60934] Updated weights for policy 1, policy_version 18012 (0.0007) [2023-10-13 21:38:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 36700160. Throughput: 0: 1669.6, 1: 1692.4. Samples: 9188732. Policy #0 lag: (min: 10.0, avg: 11.6, max: 31.0) [2023-10-13 21:38:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:38:56,258][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000018016_18448384.pth... [2023-10-13 21:38:56,293][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000016448_16842752.pth [2023-10-13 21:38:56,438][60935] Updated weights for policy 0, policy_version 17830 (0.0008) [2023-10-13 21:38:56,816][60935] Updated weights for policy 0, policy_version 17840 (0.0009) [2023-10-13 21:38:57,193][60935] Updated weights for policy 0, policy_version 17850 (0.0009) [2023-10-13 21:38:57,405][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000017856_18284544.pth... [2023-10-13 21:38:57,442][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000016288_16678912.pth [2023-10-13 21:38:58,151][60934] Updated weights for policy 1, policy_version 18022 (0.0010) [2023-10-13 21:38:58,526][60934] Updated weights for policy 1, policy_version 18032 (0.0009) [2023-10-13 21:38:58,884][60934] Updated weights for policy 1, policy_version 18042 (0.0007) [2023-10-13 21:39:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 36765696. Throughput: 0: 1671.3, 1: 1680.9. Samples: 9198430. Policy #0 lag: (min: 10.0, avg: 11.6, max: 31.0) [2023-10-13 21:39:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:39:01,350][60935] Updated weights for policy 0, policy_version 17860 (0.0007) [2023-10-13 21:39:01,726][60935] Updated weights for policy 0, policy_version 17870 (0.0009) [2023-10-13 21:39:02,094][60935] Updated weights for policy 0, policy_version 17880 (0.0009) [2023-10-13 21:39:02,975][60934] Updated weights for policy 1, policy_version 18052 (0.0007) [2023-10-13 21:39:03,335][60934] Updated weights for policy 1, policy_version 18062 (0.0007) [2023-10-13 21:39:03,708][60934] Updated weights for policy 1, policy_version 18072 (0.0008) [2023-10-13 21:39:06,234][60935] Updated weights for policy 0, policy_version 17890 (0.0008) [2023-10-13 21:39:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 36831232. Throughput: 0: 1670.6, 1: 1675.9. Samples: 9218274. Policy #0 lag: (min: 13.0, avg: 16.1, max: 45.0) [2023-10-13 21:39:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:39:06,615][60935] Updated weights for policy 0, policy_version 17900 (0.0011) [2023-10-13 21:39:06,975][60935] Updated weights for policy 0, policy_version 17910 (0.0011) [2023-10-13 21:39:07,347][60935] Updated weights for policy 0, policy_version 17920 (0.0010) [2023-10-13 21:39:07,709][60934] Updated weights for policy 1, policy_version 18082 (0.0008) [2023-10-13 21:39:08,133][60934] Updated weights for policy 1, policy_version 18092 (0.0007) [2023-10-13 21:39:08,506][60934] Updated weights for policy 1, policy_version 18102 (0.0008) [2023-10-13 21:39:08,882][60934] Updated weights for policy 1, policy_version 18112 (0.0009) [2023-10-13 21:39:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 36896768. Throughput: 0: 1663.2, 1: 1689.1. Samples: 9238642. Policy #0 lag: (min: 13.0, avg: 16.1, max: 45.0) [2023-10-13 21:39:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:39:11,512][60935] Updated weights for policy 0, policy_version 17930 (0.0008) [2023-10-13 21:39:11,882][60935] Updated weights for policy 0, policy_version 17940 (0.0009) [2023-10-13 21:39:12,253][60935] Updated weights for policy 0, policy_version 17950 (0.0008) [2023-10-13 21:39:13,015][60934] Updated weights for policy 1, policy_version 18122 (0.0008) [2023-10-13 21:39:13,395][60934] Updated weights for policy 1, policy_version 18132 (0.0008) [2023-10-13 21:39:13,769][60934] Updated weights for policy 1, policy_version 18142 (0.0008) [2023-10-13 21:39:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 36962304. Throughput: 0: 1665.0, 1: 1665.2. Samples: 9248094. Policy #0 lag: (min: 13.0, avg: 16.1, max: 45.0) [2023-10-13 21:39:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:39:16,249][60935] Updated weights for policy 0, policy_version 17960 (0.0010) [2023-10-13 21:39:16,615][60935] Updated weights for policy 0, policy_version 17970 (0.0008) [2023-10-13 21:39:16,978][60935] Updated weights for policy 0, policy_version 17980 (0.0009) [2023-10-13 21:39:17,746][60934] Updated weights for policy 1, policy_version 18152 (0.0010) [2023-10-13 21:39:18,123][60934] Updated weights for policy 1, policy_version 18162 (0.0008) [2023-10-13 21:39:18,490][60934] Updated weights for policy 1, policy_version 18172 (0.0009) [2023-10-13 21:39:21,101][60935] Updated weights for policy 0, policy_version 17990 (0.0009) [2023-10-13 21:39:21,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 37027840. Throughput: 0: 1665.8, 1: 1684.5. Samples: 9268612. Policy #0 lag: (min: 31.0, avg: 39.6, max: 63.0) [2023-10-13 21:39:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:39:21,475][60935] Updated weights for policy 0, policy_version 18000 (0.0007) [2023-10-13 21:39:21,843][60935] Updated weights for policy 0, policy_version 18010 (0.0009) [2023-10-13 21:39:22,593][60934] Updated weights for policy 1, policy_version 18182 (0.0010) [2023-10-13 21:39:22,971][60934] Updated weights for policy 1, policy_version 18192 (0.0010) [2023-10-13 21:39:23,332][60934] Updated weights for policy 1, policy_version 18202 (0.0011) [2023-10-13 21:39:25,942][60935] Updated weights for policy 0, policy_version 18020 (0.0009) [2023-10-13 21:39:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 37093376. Throughput: 0: 1658.6, 1: 1686.1. Samples: 9288928. Policy #0 lag: (min: 31.0, avg: 39.6, max: 63.0) [2023-10-13 21:39:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:39:26,319][60935] Updated weights for policy 0, policy_version 18030 (0.0008) [2023-10-13 21:39:26,688][60935] Updated weights for policy 0, policy_version 18040 (0.0008) [2023-10-13 21:39:27,432][60934] Updated weights for policy 1, policy_version 18212 (0.0009) [2023-10-13 21:39:27,800][60934] Updated weights for policy 1, policy_version 18222 (0.0007) [2023-10-13 21:39:28,167][60934] Updated weights for policy 1, policy_version 18232 (0.0007) [2023-10-13 21:39:30,918][60935] Updated weights for policy 0, policy_version 18050 (0.0008) [2023-10-13 21:39:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 37158912. Throughput: 0: 1659.0, 1: 1660.9. Samples: 9298022. Policy #0 lag: (min: 31.0, avg: 39.6, max: 63.0) [2023-10-13 21:39:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:39:31,296][60935] Updated weights for policy 0, policy_version 18060 (0.0009) [2023-10-13 21:39:31,663][60935] Updated weights for policy 0, policy_version 18070 (0.0007) [2023-10-13 21:39:32,035][60935] Updated weights for policy 0, policy_version 18080 (0.0009) [2023-10-13 21:39:32,119][60934] Updated weights for policy 1, policy_version 18242 (0.0007) [2023-10-13 21:39:32,485][60934] Updated weights for policy 1, policy_version 18252 (0.0008) [2023-10-13 21:39:32,859][60934] Updated weights for policy 1, policy_version 18262 (0.0007) [2023-10-13 21:39:33,222][60934] Updated weights for policy 1, policy_version 18272 (0.0007) [2023-10-13 21:39:35,962][60935] Updated weights for policy 0, policy_version 18090 (0.0009) [2023-10-13 21:39:36,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 37224448. Throughput: 0: 1659.5, 1: 1692.6. Samples: 9319114. Policy #0 lag: (min: 2.0, avg: 3.9, max: 31.0) [2023-10-13 21:39:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:39:36,339][60935] Updated weights for policy 0, policy_version 18100 (0.0007) [2023-10-13 21:39:36,707][60935] Updated weights for policy 0, policy_version 18110 (0.0008) [2023-10-13 21:39:37,037][60934] Updated weights for policy 1, policy_version 18282 (0.0010) [2023-10-13 21:39:37,403][60934] Updated weights for policy 1, policy_version 18292 (0.0009) [2023-10-13 21:39:37,775][60934] Updated weights for policy 1, policy_version 18302 (0.0008) [2023-10-13 21:39:40,911][60935] Updated weights for policy 0, policy_version 18120 (0.0009) [2023-10-13 21:39:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 37289984. Throughput: 0: 1654.5, 1: 1693.1. Samples: 9339374. Policy #0 lag: (min: 2.0, avg: 3.9, max: 31.0) [2023-10-13 21:39:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:39:41,284][60935] Updated weights for policy 0, policy_version 18130 (0.0009) [2023-10-13 21:39:41,658][60935] Updated weights for policy 0, policy_version 18140 (0.0008) [2023-10-13 21:39:41,835][60934] Updated weights for policy 1, policy_version 18312 (0.0008) [2023-10-13 21:39:42,207][60934] Updated weights for policy 1, policy_version 18322 (0.0008) [2023-10-13 21:39:42,583][60934] Updated weights for policy 1, policy_version 18332 (0.0007) [2023-10-13 21:39:45,754][60935] Updated weights for policy 0, policy_version 18150 (0.0008) [2023-10-13 21:39:46,124][60935] Updated weights for policy 0, policy_version 18160 (0.0009) [2023-10-13 21:39:46,248][59943] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 37355520. Throughput: 0: 1660.1, 1: 1681.9. Samples: 9348820. Policy #0 lag: (min: 2.0, avg: 3.9, max: 31.0) [2023-10-13 21:39:46,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:39:46,499][60934] Updated weights for policy 1, policy_version 18342 (0.0008) [2023-10-13 21:39:46,503][60935] Updated weights for policy 0, policy_version 18170 (0.0009) [2023-10-13 21:39:46,857][60934] Updated weights for policy 1, policy_version 18352 (0.0009) [2023-10-13 21:39:47,233][60934] Updated weights for policy 1, policy_version 18362 (0.0008) [2023-10-13 21:39:50,605][60935] Updated weights for policy 0, policy_version 18180 (0.0008) [2023-10-13 21:39:50,977][60935] Updated weights for policy 0, policy_version 18190 (0.0010) [2023-10-13 21:39:51,240][60934] Updated weights for policy 1, policy_version 18372 (0.0010) [2023-10-13 21:39:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 37421056. Throughput: 0: 1661.3, 1: 1703.6. Samples: 9369698. Policy #0 lag: (min: 2.0, avg: 2.7, max: 17.0) [2023-10-13 21:39:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:39:51,347][60935] Updated weights for policy 0, policy_version 18200 (0.0007) [2023-10-13 21:39:51,605][60934] Updated weights for policy 1, policy_version 18382 (0.0009) [2023-10-13 21:39:51,971][60934] Updated weights for policy 1, policy_version 18392 (0.0011) [2023-10-13 21:39:55,397][60935] Updated weights for policy 0, policy_version 18210 (0.0007) [2023-10-13 21:39:55,762][60935] Updated weights for policy 0, policy_version 18220 (0.0007) [2023-10-13 21:39:56,108][60934] Updated weights for policy 1, policy_version 18402 (0.0009) [2023-10-13 21:39:56,132][60935] Updated weights for policy 0, policy_version 18230 (0.0007) [2023-10-13 21:39:56,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 37486592. Throughput: 0: 1655.6, 1: 1701.2. Samples: 9389696. Policy #0 lag: (min: 2.0, avg: 2.7, max: 17.0) [2023-10-13 21:39:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:39:56,506][60935] Updated weights for policy 0, policy_version 18240 (0.0008) [2023-10-13 21:39:56,526][60934] Updated weights for policy 1, policy_version 18412 (0.0007) [2023-10-13 21:39:56,902][60934] Updated weights for policy 1, policy_version 18422 (0.0011) [2023-10-13 21:39:57,271][60934] Updated weights for policy 1, policy_version 18432 (0.0010) [2023-10-13 21:40:00,540][60935] Updated weights for policy 0, policy_version 18250 (0.0008) [2023-10-13 21:40:00,915][60935] Updated weights for policy 0, policy_version 18260 (0.0009) [2023-10-13 21:40:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 37552128. Throughput: 0: 1668.1, 1: 1690.4. Samples: 9399228. Policy #0 lag: (min: 2.0, avg: 2.7, max: 17.0) [2023-10-13 21:40:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:40:01,270][60934] Updated weights for policy 1, policy_version 18442 (0.0008) [2023-10-13 21:40:01,290][60935] Updated weights for policy 0, policy_version 18270 (0.0007) [2023-10-13 21:40:01,646][60934] Updated weights for policy 1, policy_version 18452 (0.0008) [2023-10-13 21:40:02,014][60934] Updated weights for policy 1, policy_version 18462 (0.0009) [2023-10-13 21:40:05,356][60935] Updated weights for policy 0, policy_version 18280 (0.0007) [2023-10-13 21:40:05,716][60935] Updated weights for policy 0, policy_version 18290 (0.0008) [2023-10-13 21:40:05,920][60934] Updated weights for policy 1, policy_version 18472 (0.0007) [2023-10-13 21:40:06,079][60935] Updated weights for policy 0, policy_version 18300 (0.0008) [2023-10-13 21:40:06,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 37650432. Throughput: 0: 1667.7, 1: 1703.9. Samples: 9420332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:40:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:40:06,291][60934] Updated weights for policy 1, policy_version 18482 (0.0008) [2023-10-13 21:40:06,658][60934] Updated weights for policy 1, policy_version 18492 (0.0009) [2023-10-13 21:40:10,316][60935] Updated weights for policy 0, policy_version 18310 (0.0008) [2023-10-13 21:40:10,693][60935] Updated weights for policy 0, policy_version 18320 (0.0008) [2023-10-13 21:40:10,727][60934] Updated weights for policy 1, policy_version 18502 (0.0008) [2023-10-13 21:40:11,063][60935] Updated weights for policy 0, policy_version 18330 (0.0008) [2023-10-13 21:40:11,091][60934] Updated weights for policy 1, policy_version 18512 (0.0009) [2023-10-13 21:40:11,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 37683200. Throughput: 0: 1649.0, 1: 1709.7. Samples: 9440072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:40:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:40:11,458][60934] Updated weights for policy 1, policy_version 18522 (0.0011) [2023-10-13 21:40:15,242][60935] Updated weights for policy 0, policy_version 18340 (0.0008) [2023-10-13 21:40:15,439][60934] Updated weights for policy 1, policy_version 18532 (0.0010) [2023-10-13 21:40:15,611][60935] Updated weights for policy 0, policy_version 18350 (0.0008) [2023-10-13 21:40:15,807][60934] Updated weights for policy 1, policy_version 18542 (0.0008) [2023-10-13 21:40:15,976][60935] Updated weights for policy 0, policy_version 18360 (0.0008) [2023-10-13 21:40:16,180][60934] Updated weights for policy 1, policy_version 18552 (0.0007) [2023-10-13 21:40:16,248][59943] Fps is (10 sec: 9830.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 37748736. Throughput: 0: 1665.0, 1: 1708.4. Samples: 9449826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:40:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:40:19,991][60935] Updated weights for policy 0, policy_version 18370 (0.0007) [2023-10-13 21:40:20,237][60934] Updated weights for policy 1, policy_version 18562 (0.0007) [2023-10-13 21:40:20,356][60935] Updated weights for policy 0, policy_version 18380 (0.0008) [2023-10-13 21:40:20,601][60934] Updated weights for policy 1, policy_version 18572 (0.0009) [2023-10-13 21:40:20,731][60935] Updated weights for policy 0, policy_version 18390 (0.0007) [2023-10-13 21:40:20,971][60934] Updated weights for policy 1, policy_version 18582 (0.0008) [2023-10-13 21:40:21,115][60935] Updated weights for policy 0, policy_version 18400 (0.0007) [2023-10-13 21:40:21,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 37847040. Throughput: 0: 1664.4, 1: 1700.3. Samples: 9470526. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 21:40:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:40:21,336][60934] Updated weights for policy 1, policy_version 18592 (0.0008) [2023-10-13 21:40:25,304][60935] Updated weights for policy 0, policy_version 18410 (0.0009) [2023-10-13 21:40:25,397][60934] Updated weights for policy 1, policy_version 18602 (0.0007) [2023-10-13 21:40:25,672][60935] Updated weights for policy 0, policy_version 18420 (0.0009) [2023-10-13 21:40:25,751][60934] Updated weights for policy 1, policy_version 18612 (0.0009) [2023-10-13 21:40:26,030][60935] Updated weights for policy 0, policy_version 18430 (0.0010) [2023-10-13 21:40:26,126][60934] Updated weights for policy 1, policy_version 18622 (0.0009) [2023-10-13 21:40:26,249][59943] Fps is (10 sec: 19660.4, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 37945344. Throughput: 0: 1655.9, 1: 1685.2. Samples: 9489722. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 21:40:26,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:40:30,182][60934] Updated weights for policy 1, policy_version 18632 (0.0009) [2023-10-13 21:40:30,356][60935] Updated weights for policy 0, policy_version 18440 (0.0010) [2023-10-13 21:40:30,542][60934] Updated weights for policy 1, policy_version 18642 (0.0008) [2023-10-13 21:40:30,724][60935] Updated weights for policy 0, policy_version 18450 (0.0010) [2023-10-13 21:40:30,909][60934] Updated weights for policy 1, policy_version 18652 (0.0008) [2023-10-13 21:40:31,102][60935] Updated weights for policy 0, policy_version 18460 (0.0009) [2023-10-13 21:40:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 37978112. Throughput: 0: 1667.7, 1: 1693.8. Samples: 9500086. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 21:40:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:40:34,859][60934] Updated weights for policy 1, policy_version 18662 (0.0008) [2023-10-13 21:40:35,219][60934] Updated weights for policy 1, policy_version 18672 (0.0007) [2023-10-13 21:40:35,231][60935] Updated weights for policy 0, policy_version 18470 (0.0007) [2023-10-13 21:40:35,588][60934] Updated weights for policy 1, policy_version 18682 (0.0007) [2023-10-13 21:40:35,601][60935] Updated weights for policy 0, policy_version 18480 (0.0008) [2023-10-13 21:40:35,977][60935] Updated weights for policy 0, policy_version 18490 (0.0008) [2023-10-13 21:40:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 38076416. Throughput: 0: 1665.5, 1: 1695.5. Samples: 9520942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:40:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:40:39,730][60934] Updated weights for policy 1, policy_version 18692 (0.0008) [2023-10-13 21:40:40,052][60935] Updated weights for policy 0, policy_version 18500 (0.0009) [2023-10-13 21:40:40,090][60934] Updated weights for policy 1, policy_version 18702 (0.0008) [2023-10-13 21:40:40,421][60935] Updated weights for policy 0, policy_version 18510 (0.0010) [2023-10-13 21:40:40,457][60934] Updated weights for policy 1, policy_version 18712 (0.0009) [2023-10-13 21:40:40,789][60935] Updated weights for policy 0, policy_version 18520 (0.0009) [2023-10-13 21:40:41,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 38141952. Throughput: 0: 1656.9, 1: 1674.8. Samples: 9539622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:40:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:40:44,598][60934] Updated weights for policy 1, policy_version 18722 (0.0010) [2023-10-13 21:40:44,766][60935] Updated weights for policy 0, policy_version 18530 (0.0009) [2023-10-13 21:40:44,959][60934] Updated weights for policy 1, policy_version 18732 (0.0008) [2023-10-13 21:40:45,143][60935] Updated weights for policy 0, policy_version 18540 (0.0009) [2023-10-13 21:40:45,329][60934] Updated weights for policy 1, policy_version 18742 (0.0007) [2023-10-13 21:40:45,516][60935] Updated weights for policy 0, policy_version 18550 (0.0009) [2023-10-13 21:40:45,706][60934] Updated weights for policy 1, policy_version 18752 (0.0009) [2023-10-13 21:40:45,884][60935] Updated weights for policy 0, policy_version 18560 (0.0008) [2023-10-13 21:40:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 38207488. Throughput: 0: 1664.0, 1: 1699.4. Samples: 9550580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:40:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:40:49,782][60934] Updated weights for policy 1, policy_version 18762 (0.0009) [2023-10-13 21:40:49,954][60935] Updated weights for policy 0, policy_version 18570 (0.0007) [2023-10-13 21:40:50,143][60934] Updated weights for policy 1, policy_version 18772 (0.0009) [2023-10-13 21:40:50,326][60935] Updated weights for policy 0, policy_version 18580 (0.0009) [2023-10-13 21:40:50,506][60934] Updated weights for policy 1, policy_version 18782 (0.0009) [2023-10-13 21:40:50,695][60935] Updated weights for policy 0, policy_version 18590 (0.0007) [2023-10-13 21:40:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 38273024. Throughput: 0: 1656.0, 1: 1684.7. Samples: 9570664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:40:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:40:54,525][60934] Updated weights for policy 1, policy_version 18792 (0.0008) [2023-10-13 21:40:54,733][60935] Updated weights for policy 0, policy_version 18600 (0.0007) [2023-10-13 21:40:54,893][60934] Updated weights for policy 1, policy_version 18802 (0.0007) [2023-10-13 21:40:55,101][60935] Updated weights for policy 0, policy_version 18610 (0.0007) [2023-10-13 21:40:55,259][60934] Updated weights for policy 1, policy_version 18812 (0.0008) [2023-10-13 21:40:55,480][60935] Updated weights for policy 0, policy_version 18620 (0.0009) [2023-10-13 21:40:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 38338560. Throughput: 0: 1653.2, 1: 1662.9. Samples: 9589298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:40:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:40:56,258][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000018624_19070976.pth... [2023-10-13 21:40:56,258][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000018816_19267584.pth... [2023-10-13 21:40:56,291][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000017056_17465344.pth [2023-10-13 21:40:56,298][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000017216_17629184.pth [2023-10-13 21:40:59,370][60934] Updated weights for policy 1, policy_version 18822 (0.0008) [2023-10-13 21:40:59,573][60935] Updated weights for policy 0, policy_version 18630 (0.0009) [2023-10-13 21:40:59,743][60934] Updated weights for policy 1, policy_version 18832 (0.0007) [2023-10-13 21:40:59,929][60935] Updated weights for policy 0, policy_version 18640 (0.0009) [2023-10-13 21:41:00,104][60934] Updated weights for policy 1, policy_version 18842 (0.0008) [2023-10-13 21:41:00,304][60935] Updated weights for policy 0, policy_version 18650 (0.0008) [2023-10-13 21:41:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 38404096. Throughput: 0: 1668.1, 1: 1693.9. Samples: 9601116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:41:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:41:04,235][60934] Updated weights for policy 1, policy_version 18852 (0.0009) [2023-10-13 21:41:04,598][60935] Updated weights for policy 0, policy_version 18660 (0.0007) [2023-10-13 21:41:04,606][60934] Updated weights for policy 1, policy_version 18862 (0.0008) [2023-10-13 21:41:04,970][60935] Updated weights for policy 0, policy_version 18670 (0.0007) [2023-10-13 21:41:04,984][60934] Updated weights for policy 1, policy_version 18872 (0.0008) [2023-10-13 21:41:05,354][60935] Updated weights for policy 0, policy_version 18680 (0.0010) [2023-10-13 21:41:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 38469632. Throughput: 0: 1656.9, 1: 1681.6. Samples: 9620760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:41:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:41:08,950][60934] Updated weights for policy 1, policy_version 18882 (0.0008) [2023-10-13 21:41:09,310][60934] Updated weights for policy 1, policy_version 18892 (0.0008) [2023-10-13 21:41:09,364][60935] Updated weights for policy 0, policy_version 18690 (0.0008) [2023-10-13 21:41:09,673][60934] Updated weights for policy 1, policy_version 18902 (0.0009) [2023-10-13 21:41:09,740][60935] Updated weights for policy 0, policy_version 18700 (0.0008) [2023-10-13 21:41:10,041][60934] Updated weights for policy 1, policy_version 18912 (0.0009) [2023-10-13 21:41:10,117][60935] Updated weights for policy 0, policy_version 18710 (0.0009) [2023-10-13 21:41:10,484][60935] Updated weights for policy 0, policy_version 18720 (0.0010) [2023-10-13 21:41:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 38535168. Throughput: 0: 1655.1, 1: 1681.3. Samples: 9639858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:41:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:41:14,145][60934] Updated weights for policy 1, policy_version 18922 (0.0010) [2023-10-13 21:41:14,516][60934] Updated weights for policy 1, policy_version 18932 (0.0008) [2023-10-13 21:41:14,714][60935] Updated weights for policy 0, policy_version 18730 (0.0009) [2023-10-13 21:41:14,873][60934] Updated weights for policy 1, policy_version 18942 (0.0008) [2023-10-13 21:41:15,099][60935] Updated weights for policy 0, policy_version 18740 (0.0009) [2023-10-13 21:41:15,478][60935] Updated weights for policy 0, policy_version 18750 (0.0010) [2023-10-13 21:41:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 38600704. Throughput: 0: 1666.4, 1: 1698.8. Samples: 9651522. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 21:41:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:41:18,789][60934] Updated weights for policy 1, policy_version 18952 (0.0009) [2023-10-13 21:41:19,148][60934] Updated weights for policy 1, policy_version 18962 (0.0008) [2023-10-13 21:41:19,520][60934] Updated weights for policy 1, policy_version 18972 (0.0010) [2023-10-13 21:41:19,677][60935] Updated weights for policy 0, policy_version 18760 (0.0009) [2023-10-13 21:41:20,055][60935] Updated weights for policy 0, policy_version 18770 (0.0008) [2023-10-13 21:41:20,416][60935] Updated weights for policy 0, policy_version 18780 (0.0009) [2023-10-13 21:41:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 38666240. Throughput: 0: 1653.2, 1: 1670.8. Samples: 9670520. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 21:41:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 21:41:23,567][60934] Updated weights for policy 1, policy_version 18982 (0.0007) [2023-10-13 21:41:23,939][60934] Updated weights for policy 1, policy_version 18992 (0.0009) [2023-10-13 21:41:24,307][60934] Updated weights for policy 1, policy_version 19002 (0.0007) [2023-10-13 21:41:24,466][60935] Updated weights for policy 0, policy_version 18790 (0.0009) [2023-10-13 21:41:24,838][60935] Updated weights for policy 0, policy_version 18800 (0.0007) [2023-10-13 21:41:25,208][60935] Updated weights for policy 0, policy_version 18810 (0.0008) [2023-10-13 21:41:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 38731776. Throughput: 0: 1656.1, 1: 1690.6. Samples: 9690224. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 21:41:26,250][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 21:41:28,330][60934] Updated weights for policy 1, policy_version 19012 (0.0009) [2023-10-13 21:41:28,707][60934] Updated weights for policy 1, policy_version 19022 (0.0010) [2023-10-13 21:41:29,066][60934] Updated weights for policy 1, policy_version 19032 (0.0011) [2023-10-13 21:41:29,424][60935] Updated weights for policy 0, policy_version 18820 (0.0009) [2023-10-13 21:41:29,799][60935] Updated weights for policy 0, policy_version 18830 (0.0010) [2023-10-13 21:41:30,170][60935] Updated weights for policy 0, policy_version 18840 (0.0008) [2023-10-13 21:41:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 38797312. Throughput: 0: 1662.2, 1: 1686.6. Samples: 9701274. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-13 21:41:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 21:41:33,158][60934] Updated weights for policy 1, policy_version 19042 (0.0009) [2023-10-13 21:41:33,531][60934] Updated weights for policy 1, policy_version 19052 (0.0009) [2023-10-13 21:41:33,902][60934] Updated weights for policy 1, policy_version 19062 (0.0010) [2023-10-13 21:41:34,112][60935] Updated weights for policy 0, policy_version 18850 (0.0007) [2023-10-13 21:41:34,263][60934] Updated weights for policy 1, policy_version 19072 (0.0009) [2023-10-13 21:41:34,474][60935] Updated weights for policy 0, policy_version 18860 (0.0010) [2023-10-13 21:41:34,851][60935] Updated weights for policy 0, policy_version 18870 (0.0013) [2023-10-13 21:41:35,225][60935] Updated weights for policy 0, policy_version 18880 (0.0011) [2023-10-13 21:41:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 38862848. Throughput: 0: 1650.3, 1: 1676.7. Samples: 9720376. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-13 21:41:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.170')] [2023-10-13 21:41:38,290][60934] Updated weights for policy 1, policy_version 19082 (0.0009) [2023-10-13 21:41:38,662][60934] Updated weights for policy 1, policy_version 19092 (0.0008) [2023-10-13 21:41:39,036][60934] Updated weights for policy 1, policy_version 19102 (0.0007) [2023-10-13 21:41:39,245][60935] Updated weights for policy 0, policy_version 18890 (0.0009) [2023-10-13 21:41:39,618][60935] Updated weights for policy 0, policy_version 18900 (0.0010) [2023-10-13 21:41:39,996][60935] Updated weights for policy 0, policy_version 18910 (0.0011) [2023-10-13 21:41:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 38928384. Throughput: 0: 1663.5, 1: 1700.0. Samples: 9740656. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-13 21:41:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.170')] [2023-10-13 21:41:43,091][60934] Updated weights for policy 1, policy_version 19112 (0.0007) [2023-10-13 21:41:43,447][60934] Updated weights for policy 1, policy_version 19122 (0.0007) [2023-10-13 21:41:43,811][60934] Updated weights for policy 1, policy_version 19132 (0.0009) [2023-10-13 21:41:44,071][60935] Updated weights for policy 0, policy_version 18920 (0.0009) [2023-10-13 21:41:44,442][60935] Updated weights for policy 0, policy_version 18930 (0.0008) [2023-10-13 21:41:44,823][60935] Updated weights for policy 0, policy_version 18940 (0.0009) [2023-10-13 21:41:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 38993920. Throughput: 0: 1658.7, 1: 1682.1. Samples: 9751456. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 21:41:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.170')] [2023-10-13 21:41:48,051][60934] Updated weights for policy 1, policy_version 19142 (0.0008) [2023-10-13 21:41:48,425][60934] Updated weights for policy 1, policy_version 19152 (0.0009) [2023-10-13 21:41:48,784][60934] Updated weights for policy 1, policy_version 19162 (0.0009) [2023-10-13 21:41:48,859][60935] Updated weights for policy 0, policy_version 18950 (0.0010) [2023-10-13 21:41:49,239][60935] Updated weights for policy 0, policy_version 18960 (0.0009) [2023-10-13 21:41:49,623][60935] Updated weights for policy 0, policy_version 18970 (0.0007) [2023-10-13 21:41:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 39059456. Throughput: 0: 1647.5, 1: 1681.7. Samples: 9770574. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 21:41:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:41:52,728][60934] Updated weights for policy 1, policy_version 19172 (0.0011) [2023-10-13 21:41:53,097][60934] Updated weights for policy 1, policy_version 19182 (0.0008) [2023-10-13 21:41:53,477][60934] Updated weights for policy 1, policy_version 19192 (0.0009) [2023-10-13 21:41:53,662][60935] Updated weights for policy 0, policy_version 18980 (0.0007) [2023-10-13 21:41:54,029][60935] Updated weights for policy 0, policy_version 18990 (0.0008) [2023-10-13 21:41:54,395][60935] Updated weights for policy 0, policy_version 19000 (0.0009) [2023-10-13 21:41:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 39124992. Throughput: 0: 1669.1, 1: 1697.3. Samples: 9791346. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 21:41:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:41:57,597][60934] Updated weights for policy 1, policy_version 19202 (0.0007) [2023-10-13 21:41:57,957][60934] Updated weights for policy 1, policy_version 19212 (0.0007) [2023-10-13 21:41:58,328][60934] Updated weights for policy 1, policy_version 19222 (0.0009) [2023-10-13 21:41:58,589][60935] Updated weights for policy 0, policy_version 19010 (0.0009) [2023-10-13 21:41:58,691][60934] Updated weights for policy 1, policy_version 19232 (0.0007) [2023-10-13 21:41:58,957][60935] Updated weights for policy 0, policy_version 19020 (0.0009) [2023-10-13 21:41:59,341][60935] Updated weights for policy 0, policy_version 19030 (0.0009) [2023-10-13 21:41:59,717][60935] Updated weights for policy 0, policy_version 19040 (0.0010) [2023-10-13 21:42:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 39190528. Throughput: 0: 1660.6, 1: 1670.8. Samples: 9801436. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-13 21:42:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:42:02,727][60934] Updated weights for policy 1, policy_version 19242 (0.0007) [2023-10-13 21:42:03,105][60934] Updated weights for policy 1, policy_version 19252 (0.0008) [2023-10-13 21:42:03,478][60934] Updated weights for policy 1, policy_version 19262 (0.0009) [2023-10-13 21:42:03,732][60935] Updated weights for policy 0, policy_version 19050 (0.0009) [2023-10-13 21:42:04,102][60935] Updated weights for policy 0, policy_version 19060 (0.0009) [2023-10-13 21:42:04,466][60935] Updated weights for policy 0, policy_version 19070 (0.0010) [2023-10-13 21:42:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 39256064. Throughput: 0: 1655.6, 1: 1689.4. Samples: 9821044. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-13 21:42:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:42:07,456][60934] Updated weights for policy 1, policy_version 19272 (0.0009) [2023-10-13 21:42:07,817][60934] Updated weights for policy 1, policy_version 19282 (0.0008) [2023-10-13 21:42:08,190][60934] Updated weights for policy 1, policy_version 19292 (0.0008) [2023-10-13 21:42:08,545][60935] Updated weights for policy 0, policy_version 19080 (0.0008) [2023-10-13 21:42:08,921][60935] Updated weights for policy 0, policy_version 19090 (0.0007) [2023-10-13 21:42:09,292][60935] Updated weights for policy 0, policy_version 19100 (0.0007) [2023-10-13 21:42:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 39321600. Throughput: 0: 1675.9, 1: 1696.1. Samples: 9841962. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-13 21:42:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:42:12,229][60934] Updated weights for policy 1, policy_version 19302 (0.0007) [2023-10-13 21:42:12,600][60934] Updated weights for policy 1, policy_version 19312 (0.0007) [2023-10-13 21:42:12,979][60934] Updated weights for policy 1, policy_version 19322 (0.0007) [2023-10-13 21:42:13,226][60935] Updated weights for policy 0, policy_version 19110 (0.0009) [2023-10-13 21:42:13,597][60935] Updated weights for policy 0, policy_version 19120 (0.0010) [2023-10-13 21:42:13,973][60935] Updated weights for policy 0, policy_version 19130 (0.0008) [2023-10-13 21:42:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 39387136. Throughput: 0: 1663.3, 1: 1678.4. Samples: 9851650. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) [2023-10-13 21:42:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:42:17,029][60934] Updated weights for policy 1, policy_version 19332 (0.0008) [2023-10-13 21:42:17,403][60934] Updated weights for policy 1, policy_version 19342 (0.0008) [2023-10-13 21:42:17,778][60934] Updated weights for policy 1, policy_version 19352 (0.0008) [2023-10-13 21:42:18,043][60935] Updated weights for policy 0, policy_version 19140 (0.0009) [2023-10-13 21:42:18,401][60935] Updated weights for policy 0, policy_version 19150 (0.0011) [2023-10-13 21:42:18,772][60935] Updated weights for policy 0, policy_version 19160 (0.0012) [2023-10-13 21:42:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 39452672. Throughput: 0: 1668.9, 1: 1697.4. Samples: 9871860. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) [2023-10-13 21:42:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:42:21,884][60934] Updated weights for policy 1, policy_version 19362 (0.0008) [2023-10-13 21:42:22,258][60934] Updated weights for policy 1, policy_version 19372 (0.0010) [2023-10-13 21:42:22,616][60934] Updated weights for policy 1, policy_version 19382 (0.0009) [2023-10-13 21:42:22,993][60934] Updated weights for policy 1, policy_version 19392 (0.0008) [2023-10-13 21:42:23,078][60935] Updated weights for policy 0, policy_version 19170 (0.0010) [2023-10-13 21:42:23,443][60935] Updated weights for policy 0, policy_version 19180 (0.0011) [2023-10-13 21:42:23,810][60935] Updated weights for policy 0, policy_version 19190 (0.0009) [2023-10-13 21:42:24,182][60935] Updated weights for policy 0, policy_version 19200 (0.0009) [2023-10-13 21:42:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 39518208. Throughput: 0: 1675.7, 1: 1693.4. Samples: 9892268. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) [2023-10-13 21:42:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:42:27,135][60934] Updated weights for policy 1, policy_version 19402 (0.0009) [2023-10-13 21:42:27,508][60934] Updated weights for policy 1, policy_version 19412 (0.0010) [2023-10-13 21:42:27,877][60934] Updated weights for policy 1, policy_version 19422 (0.0009) [2023-10-13 21:42:28,204][60935] Updated weights for policy 0, policy_version 19210 (0.0007) [2023-10-13 21:42:28,580][60935] Updated weights for policy 0, policy_version 19220 (0.0008) [2023-10-13 21:42:28,953][60935] Updated weights for policy 0, policy_version 19230 (0.0009) [2023-10-13 21:42:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 39583744. Throughput: 0: 1658.1, 1: 1677.7. Samples: 9901566. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-10-13 21:42:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:42:31,794][60934] Updated weights for policy 1, policy_version 19432 (0.0010) [2023-10-13 21:42:32,160][60934] Updated weights for policy 1, policy_version 19442 (0.0009) [2023-10-13 21:42:32,536][60934] Updated weights for policy 1, policy_version 19452 (0.0008) [2023-10-13 21:42:32,896][60935] Updated weights for policy 0, policy_version 19240 (0.0009) [2023-10-13 21:42:33,269][60935] Updated weights for policy 0, policy_version 19250 (0.0009) [2023-10-13 21:42:33,642][60935] Updated weights for policy 0, policy_version 19260 (0.0009) [2023-10-13 21:42:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 39649280. Throughput: 0: 1678.4, 1: 1693.4. Samples: 9922306. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-10-13 21:42:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:42:36,384][60934] Updated weights for policy 1, policy_version 19462 (0.0007) [2023-10-13 21:42:36,757][60934] Updated weights for policy 1, policy_version 19472 (0.0009) [2023-10-13 21:42:37,120][60934] Updated weights for policy 1, policy_version 19482 (0.0008) [2023-10-13 21:42:37,813][60935] Updated weights for policy 0, policy_version 19270 (0.0008) [2023-10-13 21:42:38,179][60935] Updated weights for policy 0, policy_version 19280 (0.0009) [2023-10-13 21:42:38,558][60935] Updated weights for policy 0, policy_version 19290 (0.0010) [2023-10-13 21:42:41,195][60934] Updated weights for policy 1, policy_version 19492 (0.0007) [2023-10-13 21:42:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 39714816. Throughput: 0: 1673.6, 1: 1692.2. Samples: 9942804. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-10-13 21:42:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:42:41,567][60934] Updated weights for policy 1, policy_version 19502 (0.0007) [2023-10-13 21:42:41,939][60934] Updated weights for policy 1, policy_version 19512 (0.0007) [2023-10-13 21:42:42,795][60935] Updated weights for policy 0, policy_version 19300 (0.0010) [2023-10-13 21:42:43,163][60935] Updated weights for policy 0, policy_version 19310 (0.0009) [2023-10-13 21:42:43,525][60935] Updated weights for policy 0, policy_version 19320 (0.0007) [2023-10-13 21:42:46,003][60934] Updated weights for policy 1, policy_version 19522 (0.0008) [2023-10-13 21:42:46,249][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 39780352. Throughput: 0: 1651.8, 1: 1691.4. Samples: 9951882. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-13 21:42:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:42:46,370][60934] Updated weights for policy 1, policy_version 19532 (0.0007) [2023-10-13 21:42:46,735][60934] Updated weights for policy 1, policy_version 19542 (0.0008) [2023-10-13 21:42:47,100][60934] Updated weights for policy 1, policy_version 19552 (0.0009) [2023-10-13 21:42:47,722][60935] Updated weights for policy 0, policy_version 19330 (0.0007) [2023-10-13 21:42:48,090][60935] Updated weights for policy 0, policy_version 19340 (0.0008) [2023-10-13 21:42:48,455][60935] Updated weights for policy 0, policy_version 19350 (0.0009) [2023-10-13 21:42:48,827][60935] Updated weights for policy 0, policy_version 19360 (0.0010) [2023-10-13 21:42:51,160][60934] Updated weights for policy 1, policy_version 19562 (0.0009) [2023-10-13 21:42:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 39845888. Throughput: 0: 1673.3, 1: 1690.5. Samples: 9972414. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-13 21:42:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:42:51,532][60934] Updated weights for policy 1, policy_version 19572 (0.0009) [2023-10-13 21:42:51,900][60934] Updated weights for policy 1, policy_version 19582 (0.0008) [2023-10-13 21:42:52,861][60935] Updated weights for policy 0, policy_version 19370 (0.0008) [2023-10-13 21:42:53,237][60935] Updated weights for policy 0, policy_version 19380 (0.0007) [2023-10-13 21:42:53,618][60935] Updated weights for policy 0, policy_version 19390 (0.0011) [2023-10-13 21:42:56,000][60934] Updated weights for policy 1, policy_version 19592 (0.0009) [2023-10-13 21:42:56,248][59943] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 39911424. Throughput: 0: 1669.5, 1: 1691.1. Samples: 9993188. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-13 21:42:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:42:56,258][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000019392_19857408.pth... [2023-10-13 21:42:56,294][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000017856_18284544.pth [2023-10-13 21:42:56,374][60934] Updated weights for policy 1, policy_version 19602 (0.0009) [2023-10-13 21:42:56,743][60934] Updated weights for policy 1, policy_version 19612 (0.0009) [2023-10-13 21:42:56,883][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000019616_20086784.pth... [2023-10-13 21:42:56,921][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000018016_18448384.pth [2023-10-13 21:42:57,704][60935] Updated weights for policy 0, policy_version 19400 (0.0008) [2023-10-13 21:42:58,072][60935] Updated weights for policy 0, policy_version 19410 (0.0010) [2023-10-13 21:42:58,456][60935] Updated weights for policy 0, policy_version 19420 (0.0011) [2023-10-13 21:43:00,690][60934] Updated weights for policy 1, policy_version 19622 (0.0008) [2023-10-13 21:43:01,051][60934] Updated weights for policy 1, policy_version 19632 (0.0009) [2023-10-13 21:43:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 39976960. Throughput: 0: 1655.4, 1: 1693.7. Samples: 10002362. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-10-13 21:43:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:43:01,427][60934] Updated weights for policy 1, policy_version 19642 (0.0008) [2023-10-13 21:43:02,596][60935] Updated weights for policy 0, policy_version 19430 (0.0009) [2023-10-13 21:43:02,977][60935] Updated weights for policy 0, policy_version 19440 (0.0010) [2023-10-13 21:43:03,354][60935] Updated weights for policy 0, policy_version 19450 (0.0009) [2023-10-13 21:43:05,543][60934] Updated weights for policy 1, policy_version 19652 (0.0008) [2023-10-13 21:43:05,909][60934] Updated weights for policy 1, policy_version 19662 (0.0007) [2023-10-13 21:43:06,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 40042496. Throughput: 0: 1663.4, 1: 1690.9. Samples: 10022806. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-10-13 21:43:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:43:06,272][60934] Updated weights for policy 1, policy_version 19672 (0.0007) [2023-10-13 21:43:07,449][60935] Updated weights for policy 0, policy_version 19460 (0.0008) [2023-10-13 21:43:07,823][60935] Updated weights for policy 0, policy_version 19470 (0.0008) [2023-10-13 21:43:08,188][60935] Updated weights for policy 0, policy_version 19480 (0.0007) [2023-10-13 21:43:10,229][60934] Updated weights for policy 1, policy_version 19682 (0.0008) [2023-10-13 21:43:10,590][60934] Updated weights for policy 1, policy_version 19692 (0.0007) [2023-10-13 21:43:10,952][60934] Updated weights for policy 1, policy_version 19702 (0.0009) [2023-10-13 21:43:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 40108032. Throughput: 0: 1666.9, 1: 1686.8. Samples: 10043184. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-10-13 21:43:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:43:11,322][60934] Updated weights for policy 1, policy_version 19712 (0.0010) [2023-10-13 21:43:12,396][60935] Updated weights for policy 0, policy_version 19490 (0.0008) [2023-10-13 21:43:12,759][60935] Updated weights for policy 0, policy_version 19500 (0.0008) [2023-10-13 21:43:13,133][60935] Updated weights for policy 0, policy_version 19510 (0.0007) [2023-10-13 21:43:13,506][60935] Updated weights for policy 0, policy_version 19520 (0.0008) [2023-10-13 21:43:15,473][60934] Updated weights for policy 1, policy_version 19722 (0.0008) [2023-10-13 21:43:15,849][60934] Updated weights for policy 1, policy_version 19732 (0.0008) [2023-10-13 21:43:16,212][60934] Updated weights for policy 1, policy_version 19742 (0.0007) [2023-10-13 21:43:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 40173568. Throughput: 0: 1657.6, 1: 1700.3. Samples: 10052670. Policy #0 lag: (min: 26.0, avg: 30.7, max: 58.0) [2023-10-13 21:43:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:43:17,549][60935] Updated weights for policy 0, policy_version 19530 (0.0009) [2023-10-13 21:43:17,916][60935] Updated weights for policy 0, policy_version 19540 (0.0008) [2023-10-13 21:43:18,286][60935] Updated weights for policy 0, policy_version 19550 (0.0008) [2023-10-13 21:43:20,357][60934] Updated weights for policy 1, policy_version 19752 (0.0008) [2023-10-13 21:43:20,723][60934] Updated weights for policy 1, policy_version 19762 (0.0010) [2023-10-13 21:43:21,089][60934] Updated weights for policy 1, policy_version 19772 (0.0009) [2023-10-13 21:43:21,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 40271872. Throughput: 0: 1660.8, 1: 1694.3. Samples: 10073286. Policy #0 lag: (min: 26.0, avg: 30.7, max: 58.0) [2023-10-13 21:43:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:43:22,320][60935] Updated weights for policy 0, policy_version 19560 (0.0008) [2023-10-13 21:43:22,701][60935] Updated weights for policy 0, policy_version 19570 (0.0010) [2023-10-13 21:43:23,071][60935] Updated weights for policy 0, policy_version 19580 (0.0011) [2023-10-13 21:43:24,971][60934] Updated weights for policy 1, policy_version 19782 (0.0011) [2023-10-13 21:43:25,341][60934] Updated weights for policy 1, policy_version 19792 (0.0010) [2023-10-13 21:43:25,705][60934] Updated weights for policy 1, policy_version 19802 (0.0010) [2023-10-13 21:43:26,248][59943] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 40337408. Throughput: 0: 1667.6, 1: 1677.7. Samples: 10093344. Policy #0 lag: (min: 26.0, avg: 30.7, max: 58.0) [2023-10-13 21:43:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:43:27,001][60935] Updated weights for policy 0, policy_version 19590 (0.0008) [2023-10-13 21:43:27,368][60935] Updated weights for policy 0, policy_version 19600 (0.0011) [2023-10-13 21:43:27,746][60935] Updated weights for policy 0, policy_version 19610 (0.0010) [2023-10-13 21:43:29,786][60934] Updated weights for policy 1, policy_version 19812 (0.0010) [2023-10-13 21:43:30,153][60934] Updated weights for policy 1, policy_version 19822 (0.0008) [2023-10-13 21:43:30,520][60934] Updated weights for policy 1, policy_version 19832 (0.0009) [2023-10-13 21:43:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 40402944. Throughput: 0: 1670.1, 1: 1694.0. Samples: 10103266. Policy #0 lag: (min: 24.0, avg: 49.1, max: 56.0) [2023-10-13 21:43:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:43:31,618][60935] Updated weights for policy 0, policy_version 19620 (0.0009) [2023-10-13 21:43:31,990][60935] Updated weights for policy 0, policy_version 19630 (0.0009) [2023-10-13 21:43:32,356][60935] Updated weights for policy 0, policy_version 19640 (0.0008) [2023-10-13 21:43:34,516][60934] Updated weights for policy 1, policy_version 19842 (0.0009) [2023-10-13 21:43:34,883][60934] Updated weights for policy 1, policy_version 19852 (0.0008) [2023-10-13 21:43:35,257][60934] Updated weights for policy 1, policy_version 19862 (0.0009) [2023-10-13 21:43:35,629][60934] Updated weights for policy 1, policy_version 19872 (0.0010) [2023-10-13 21:43:36,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 40468480. Throughput: 0: 1677.9, 1: 1698.3. Samples: 10124340. Policy #0 lag: (min: 24.0, avg: 49.1, max: 56.0) [2023-10-13 21:43:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:43:36,445][60935] Updated weights for policy 0, policy_version 19650 (0.0007) [2023-10-13 21:43:36,814][60935] Updated weights for policy 0, policy_version 19660 (0.0009) [2023-10-13 21:43:37,182][60935] Updated weights for policy 0, policy_version 19670 (0.0010) [2023-10-13 21:43:37,559][60935] Updated weights for policy 0, policy_version 19680 (0.0009) [2023-10-13 21:43:39,448][60934] Updated weights for policy 1, policy_version 19882 (0.0008) [2023-10-13 21:43:39,824][60934] Updated weights for policy 1, policy_version 19892 (0.0010) [2023-10-13 21:43:40,195][60934] Updated weights for policy 1, policy_version 19902 (0.0009) [2023-10-13 21:43:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 40534016. Throughput: 0: 1676.3, 1: 1674.9. Samples: 10143994. Policy #0 lag: (min: 24.0, avg: 49.1, max: 56.0) [2023-10-13 21:43:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:43:41,956][60935] Updated weights for policy 0, policy_version 19690 (0.0008) [2023-10-13 21:43:42,321][60935] Updated weights for policy 0, policy_version 19700 (0.0008) [2023-10-13 21:43:42,695][60935] Updated weights for policy 0, policy_version 19710 (0.0008) [2023-10-13 21:43:44,298][60934] Updated weights for policy 1, policy_version 19912 (0.0007) [2023-10-13 21:43:44,659][60934] Updated weights for policy 1, policy_version 19922 (0.0009) [2023-10-13 21:43:45,036][60934] Updated weights for policy 1, policy_version 19932 (0.0009) [2023-10-13 21:43:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 40599552. Throughput: 0: 1674.8, 1: 1701.5. Samples: 10154294. Policy #0 lag: (min: 2.0, avg: 12.8, max: 34.0) [2023-10-13 21:43:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:43:46,776][60935] Updated weights for policy 0, policy_version 19720 (0.0009) [2023-10-13 21:43:47,149][60935] Updated weights for policy 0, policy_version 19730 (0.0009) [2023-10-13 21:43:47,520][60935] Updated weights for policy 0, policy_version 19740 (0.0009) [2023-10-13 21:43:49,143][60934] Updated weights for policy 1, policy_version 19942 (0.0007) [2023-10-13 21:43:49,523][60934] Updated weights for policy 1, policy_version 19952 (0.0007) [2023-10-13 21:43:49,885][60934] Updated weights for policy 1, policy_version 19962 (0.0007) [2023-10-13 21:43:51,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 40665088. Throughput: 0: 1677.5, 1: 1685.6. Samples: 10174148. Policy #0 lag: (min: 2.0, avg: 12.8, max: 34.0) [2023-10-13 21:43:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:43:51,628][60935] Updated weights for policy 0, policy_version 19750 (0.0009) [2023-10-13 21:43:51,993][60935] Updated weights for policy 0, policy_version 19760 (0.0009) [2023-10-13 21:43:52,383][60935] Updated weights for policy 0, policy_version 19770 (0.0010) [2023-10-13 21:43:53,876][60934] Updated weights for policy 1, policy_version 19972 (0.0007) [2023-10-13 21:43:54,239][60934] Updated weights for policy 1, policy_version 19982 (0.0008) [2023-10-13 21:43:54,605][60934] Updated weights for policy 1, policy_version 19992 (0.0009) [2023-10-13 21:43:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 40730624. Throughput: 0: 1678.4, 1: 1680.6. Samples: 10194342. Policy #0 lag: (min: 2.0, avg: 12.8, max: 34.0) [2023-10-13 21:43:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:43:56,397][60935] Updated weights for policy 0, policy_version 19780 (0.0010) [2023-10-13 21:43:56,768][60935] Updated weights for policy 0, policy_version 19790 (0.0011) [2023-10-13 21:43:57,143][60935] Updated weights for policy 0, policy_version 19800 (0.0011) [2023-10-13 21:43:58,712][60934] Updated weights for policy 1, policy_version 20002 (0.0008) [2023-10-13 21:43:59,086][60934] Updated weights for policy 1, policy_version 20012 (0.0010) [2023-10-13 21:43:59,451][60934] Updated weights for policy 1, policy_version 20022 (0.0010) [2023-10-13 21:43:59,816][60934] Updated weights for policy 1, policy_version 20032 (0.0011) [2023-10-13 21:44:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 40796160. Throughput: 0: 1677.6, 1: 1702.8. Samples: 10204792. Policy #0 lag: (min: 29.0, avg: 38.7, max: 61.0) [2023-10-13 21:44:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:44:01,256][60935] Updated weights for policy 0, policy_version 19810 (0.0009) [2023-10-13 21:44:01,627][60935] Updated weights for policy 0, policy_version 19820 (0.0007) [2023-10-13 21:44:01,994][60935] Updated weights for policy 0, policy_version 19830 (0.0008) [2023-10-13 21:44:02,359][60935] Updated weights for policy 0, policy_version 19840 (0.0007) [2023-10-13 21:44:03,758][60934] Updated weights for policy 1, policy_version 20042 (0.0007) [2023-10-13 21:44:04,131][60934] Updated weights for policy 1, policy_version 20052 (0.0007) [2023-10-13 21:44:04,498][60934] Updated weights for policy 1, policy_version 20062 (0.0009) [2023-10-13 21:44:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 40861696. Throughput: 0: 1686.3, 1: 1682.2. Samples: 10224868. Policy #0 lag: (min: 29.0, avg: 38.7, max: 61.0) [2023-10-13 21:44:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:44:06,436][60935] Updated weights for policy 0, policy_version 19850 (0.0010) [2023-10-13 21:44:06,806][60935] Updated weights for policy 0, policy_version 19860 (0.0009) [2023-10-13 21:44:07,176][60935] Updated weights for policy 0, policy_version 19870 (0.0009) [2023-10-13 21:44:08,478][60934] Updated weights for policy 1, policy_version 20072 (0.0009) [2023-10-13 21:44:08,854][60934] Updated weights for policy 1, policy_version 20082 (0.0010) [2023-10-13 21:44:09,234][60934] Updated weights for policy 1, policy_version 20092 (0.0010) [2023-10-13 21:44:11,156][60935] Updated weights for policy 0, policy_version 19880 (0.0009) [2023-10-13 21:44:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 40927232. Throughput: 0: 1684.4, 1: 1697.0. Samples: 10245510. Policy #0 lag: (min: 29.0, avg: 38.7, max: 61.0) [2023-10-13 21:44:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:44:11,523][60935] Updated weights for policy 0, policy_version 19890 (0.0008) [2023-10-13 21:44:11,889][60935] Updated weights for policy 0, policy_version 19900 (0.0007) [2023-10-13 21:44:13,224][60934] Updated weights for policy 1, policy_version 20102 (0.0007) [2023-10-13 21:44:13,592][60934] Updated weights for policy 1, policy_version 20112 (0.0007) [2023-10-13 21:44:13,967][60934] Updated weights for policy 1, policy_version 20122 (0.0008) [2023-10-13 21:44:16,000][60935] Updated weights for policy 0, policy_version 19910 (0.0010) [2023-10-13 21:44:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 40992768. Throughput: 0: 1686.2, 1: 1696.1. Samples: 10255468. Policy #0 lag: (min: 29.0, avg: 38.7, max: 61.0) [2023-10-13 21:44:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:44:16,375][60935] Updated weights for policy 0, policy_version 19920 (0.0010) [2023-10-13 21:44:16,750][60935] Updated weights for policy 0, policy_version 19930 (0.0009) [2023-10-13 21:44:18,080][60934] Updated weights for policy 1, policy_version 20132 (0.0009) [2023-10-13 21:44:18,448][60934] Updated weights for policy 1, policy_version 20142 (0.0010) [2023-10-13 21:44:18,807][60934] Updated weights for policy 1, policy_version 20152 (0.0010) [2023-10-13 21:44:20,763][60935] Updated weights for policy 0, policy_version 19940 (0.0008) [2023-10-13 21:44:21,128][60935] Updated weights for policy 0, policy_version 19950 (0.0009) [2023-10-13 21:44:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 41058304. Throughput: 0: 1679.1, 1: 1679.7. Samples: 10275482. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-13 21:44:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:44:21,504][60935] Updated weights for policy 0, policy_version 19960 (0.0008) [2023-10-13 21:44:22,823][60934] Updated weights for policy 1, policy_version 20162 (0.0010) [2023-10-13 21:44:23,190][60934] Updated weights for policy 1, policy_version 20172 (0.0009) [2023-10-13 21:44:23,567][60934] Updated weights for policy 1, policy_version 20182 (0.0010) [2023-10-13 21:44:23,928][60934] Updated weights for policy 1, policy_version 20192 (0.0008) [2023-10-13 21:44:25,567][60935] Updated weights for policy 0, policy_version 19970 (0.0009) [2023-10-13 21:44:25,931][60935] Updated weights for policy 0, policy_version 19980 (0.0009) [2023-10-13 21:44:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 41123840. Throughput: 0: 1674.8, 1: 1702.2. Samples: 10295962. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-13 21:44:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:44:26,304][60935] Updated weights for policy 0, policy_version 19990 (0.0008) [2023-10-13 21:44:26,668][60935] Updated weights for policy 0, policy_version 20000 (0.0008) [2023-10-13 21:44:27,889][60934] Updated weights for policy 1, policy_version 20202 (0.0009) [2023-10-13 21:44:28,251][60934] Updated weights for policy 1, policy_version 20212 (0.0008) [2023-10-13 21:44:28,622][60934] Updated weights for policy 1, policy_version 20222 (0.0007) [2023-10-13 21:44:30,991][60935] Updated weights for policy 0, policy_version 20010 (0.0010) [2023-10-13 21:44:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 41189376. Throughput: 0: 1683.8, 1: 1676.2. Samples: 10305492. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-13 21:44:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:44:31,363][60935] Updated weights for policy 0, policy_version 20020 (0.0007) [2023-10-13 21:44:31,727][60935] Updated weights for policy 0, policy_version 20030 (0.0008) [2023-10-13 21:44:32,610][60934] Updated weights for policy 1, policy_version 20232 (0.0007) [2023-10-13 21:44:32,987][60934] Updated weights for policy 1, policy_version 20242 (0.0008) [2023-10-13 21:44:33,363][60934] Updated weights for policy 1, policy_version 20252 (0.0009) [2023-10-13 21:44:35,678][60935] Updated weights for policy 0, policy_version 20040 (0.0008) [2023-10-13 21:44:36,048][60935] Updated weights for policy 0, policy_version 20050 (0.0008) [2023-10-13 21:44:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 41254912. Throughput: 0: 1679.3, 1: 1691.2. Samples: 10325820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:44:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:44:36,424][60935] Updated weights for policy 0, policy_version 20060 (0.0009) [2023-10-13 21:44:37,288][60934] Updated weights for policy 1, policy_version 20262 (0.0010) [2023-10-13 21:44:37,649][60934] Updated weights for policy 1, policy_version 20272 (0.0008) [2023-10-13 21:44:38,014][60934] Updated weights for policy 1, policy_version 20282 (0.0008) [2023-10-13 21:44:40,576][60935] Updated weights for policy 0, policy_version 20070 (0.0007) [2023-10-13 21:44:40,952][60935] Updated weights for policy 0, policy_version 20080 (0.0008) [2023-10-13 21:44:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 41320448. Throughput: 0: 1670.3, 1: 1710.9. Samples: 10346496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:44:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:44:41,330][60935] Updated weights for policy 0, policy_version 20090 (0.0010) [2023-10-13 21:44:42,088][60934] Updated weights for policy 1, policy_version 20292 (0.0009) [2023-10-13 21:44:42,464][60934] Updated weights for policy 1, policy_version 20302 (0.0007) [2023-10-13 21:44:42,832][60934] Updated weights for policy 1, policy_version 20312 (0.0007) [2023-10-13 21:44:45,475][60935] Updated weights for policy 0, policy_version 20100 (0.0009) [2023-10-13 21:44:45,842][60935] Updated weights for policy 0, policy_version 20110 (0.0010) [2023-10-13 21:44:46,213][60935] Updated weights for policy 0, policy_version 20120 (0.0010) [2023-10-13 21:44:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 41385984. Throughput: 0: 1682.4, 1: 1680.8. Samples: 10356134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:44:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:44:46,706][60934] Updated weights for policy 1, policy_version 20322 (0.0008) [2023-10-13 21:44:47,080][60934] Updated weights for policy 1, policy_version 20332 (0.0010) [2023-10-13 21:44:47,453][60934] Updated weights for policy 1, policy_version 20342 (0.0009) [2023-10-13 21:44:47,825][60934] Updated weights for policy 1, policy_version 20352 (0.0008) [2023-10-13 21:44:50,252][60935] Updated weights for policy 0, policy_version 20130 (0.0008) [2023-10-13 21:44:50,622][60935] Updated weights for policy 0, policy_version 20140 (0.0007) [2023-10-13 21:44:50,985][60935] Updated weights for policy 0, policy_version 20150 (0.0011) [2023-10-13 21:44:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 41451520. Throughput: 0: 1667.2, 1: 1704.8. Samples: 10376604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:44:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:44:51,362][60935] Updated weights for policy 0, policy_version 20160 (0.0009) [2023-10-13 21:44:51,854][60934] Updated weights for policy 1, policy_version 20362 (0.0008) [2023-10-13 21:44:52,218][60934] Updated weights for policy 1, policy_version 20372 (0.0010) [2023-10-13 21:44:52,586][60934] Updated weights for policy 1, policy_version 20382 (0.0007) [2023-10-13 21:44:55,572][60935] Updated weights for policy 0, policy_version 20170 (0.0011) [2023-10-13 21:44:55,946][60935] Updated weights for policy 0, policy_version 20180 (0.0010) [2023-10-13 21:44:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 41517056. Throughput: 0: 1648.5, 1: 1711.7. Samples: 10396718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:44:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:44:56,318][60935] Updated weights for policy 0, policy_version 20190 (0.0009) [2023-10-13 21:44:56,384][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000020192_20676608.pth... [2023-10-13 21:44:56,412][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000018624_19070976.pth [2023-10-13 21:44:56,684][60934] Updated weights for policy 1, policy_version 20392 (0.0007) [2023-10-13 21:44:57,071][60934] Updated weights for policy 1, policy_version 20402 (0.0010) [2023-10-13 21:44:57,439][60934] Updated weights for policy 1, policy_version 20412 (0.0010) [2023-10-13 21:44:57,582][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000020416_20905984.pth... [2023-10-13 21:44:57,611][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000018816_19267584.pth [2023-10-13 21:45:00,434][60935] Updated weights for policy 0, policy_version 20200 (0.0010) [2023-10-13 21:45:00,800][60935] Updated weights for policy 0, policy_version 20210 (0.0010) [2023-10-13 21:45:01,176][60935] Updated weights for policy 0, policy_version 20220 (0.0009) [2023-10-13 21:45:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 41582592. Throughput: 0: 1661.1, 1: 1691.1. Samples: 10406318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:45:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:45:01,580][60934] Updated weights for policy 1, policy_version 20422 (0.0008) [2023-10-13 21:45:01,948][60934] Updated weights for policy 1, policy_version 20432 (0.0009) [2023-10-13 21:45:02,311][60934] Updated weights for policy 1, policy_version 20442 (0.0007) [2023-10-13 21:45:05,395][60935] Updated weights for policy 0, policy_version 20230 (0.0008) [2023-10-13 21:45:05,771][60935] Updated weights for policy 0, policy_version 20240 (0.0008) [2023-10-13 21:45:06,146][60935] Updated weights for policy 0, policy_version 20250 (0.0009) [2023-10-13 21:45:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 41648128. Throughput: 0: 1660.2, 1: 1708.1. Samples: 10427058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:45:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:45:06,272][60934] Updated weights for policy 1, policy_version 20452 (0.0008) [2023-10-13 21:45:06,645][60934] Updated weights for policy 1, policy_version 20462 (0.0008) [2023-10-13 21:45:07,017][60934] Updated weights for policy 1, policy_version 20472 (0.0007) [2023-10-13 21:45:10,119][60935] Updated weights for policy 0, policy_version 20260 (0.0008) [2023-10-13 21:45:10,496][60935] Updated weights for policy 0, policy_version 20270 (0.0009) [2023-10-13 21:45:10,866][60935] Updated weights for policy 0, policy_version 20280 (0.0009) [2023-10-13 21:45:11,034][60934] Updated weights for policy 1, policy_version 20482 (0.0009) [2023-10-13 21:45:11,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 41746432. Throughput: 0: 1646.8, 1: 1706.2. Samples: 10446848. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-13 21:45:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:45:11,403][60934] Updated weights for policy 1, policy_version 20492 (0.0007) [2023-10-13 21:45:11,772][60934] Updated weights for policy 1, policy_version 20502 (0.0008) [2023-10-13 21:45:12,138][60934] Updated weights for policy 1, policy_version 20512 (0.0009) [2023-10-13 21:45:14,938][60935] Updated weights for policy 0, policy_version 20290 (0.0007) [2023-10-13 21:45:15,311][60935] Updated weights for policy 0, policy_version 20300 (0.0009) [2023-10-13 21:45:15,686][60935] Updated weights for policy 0, policy_version 20310 (0.0007) [2023-10-13 21:45:16,056][60935] Updated weights for policy 0, policy_version 20320 (0.0008) [2023-10-13 21:45:16,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 41811968. Throughput: 0: 1658.4, 1: 1706.2. Samples: 10456896. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-13 21:45:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:45:16,271][60934] Updated weights for policy 1, policy_version 20522 (0.0008) [2023-10-13 21:45:16,646][60934] Updated weights for policy 1, policy_version 20532 (0.0009) [2023-10-13 21:45:17,015][60934] Updated weights for policy 1, policy_version 20542 (0.0008) [2023-10-13 21:45:20,180][60935] Updated weights for policy 0, policy_version 20330 (0.0008) [2023-10-13 21:45:20,552][60935] Updated weights for policy 0, policy_version 20340 (0.0010) [2023-10-13 21:45:20,908][60935] Updated weights for policy 0, policy_version 20350 (0.0009) [2023-10-13 21:45:20,943][60934] Updated weights for policy 1, policy_version 20552 (0.0009) [2023-10-13 21:45:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 41877504. Throughput: 0: 1669.1, 1: 1707.6. Samples: 10477772. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-13 21:45:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:45:21,313][60934] Updated weights for policy 1, policy_version 20562 (0.0008) [2023-10-13 21:45:21,682][60934] Updated weights for policy 1, policy_version 20572 (0.0008) [2023-10-13 21:45:24,905][60935] Updated weights for policy 0, policy_version 20360 (0.0009) [2023-10-13 21:45:25,277][60935] Updated weights for policy 0, policy_version 20370 (0.0012) [2023-10-13 21:45:25,642][60935] Updated weights for policy 0, policy_version 20380 (0.0009) [2023-10-13 21:45:25,692][60934] Updated weights for policy 1, policy_version 20582 (0.0009) [2023-10-13 21:45:26,066][60934] Updated weights for policy 1, policy_version 20592 (0.0010) [2023-10-13 21:45:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 41943040. Throughput: 0: 1650.1, 1: 1694.9. Samples: 10497024. Policy #0 lag: (min: 18.0, avg: 19.6, max: 47.0) [2023-10-13 21:45:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:45:26,427][60934] Updated weights for policy 1, policy_version 20602 (0.0007) [2023-10-13 21:45:29,765][60935] Updated weights for policy 0, policy_version 20390 (0.0009) [2023-10-13 21:45:30,127][60935] Updated weights for policy 0, policy_version 20400 (0.0009) [2023-10-13 21:45:30,505][60935] Updated weights for policy 0, policy_version 20410 (0.0009) [2023-10-13 21:45:30,631][60934] Updated weights for policy 1, policy_version 20612 (0.0009) [2023-10-13 21:45:31,001][60934] Updated weights for policy 1, policy_version 20622 (0.0009) [2023-10-13 21:45:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 42008576. Throughput: 0: 1666.5, 1: 1692.3. Samples: 10507280. Policy #0 lag: (min: 18.0, avg: 19.6, max: 47.0) [2023-10-13 21:45:31,248][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 21:45:31,362][60934] Updated weights for policy 1, policy_version 20632 (0.0010) [2023-10-13 21:45:34,564][60935] Updated weights for policy 0, policy_version 20420 (0.0008) [2023-10-13 21:45:34,936][60935] Updated weights for policy 0, policy_version 20430 (0.0009) [2023-10-13 21:45:35,310][60935] Updated weights for policy 0, policy_version 20440 (0.0008) [2023-10-13 21:45:35,328][60934] Updated weights for policy 1, policy_version 20642 (0.0009) [2023-10-13 21:45:35,693][60934] Updated weights for policy 1, policy_version 20652 (0.0008) [2023-10-13 21:45:36,060][60934] Updated weights for policy 1, policy_version 20662 (0.0008) [2023-10-13 21:45:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 42074112. Throughput: 0: 1664.1, 1: 1692.0. Samples: 10527628. Policy #0 lag: (min: 18.0, avg: 19.6, max: 47.0) [2023-10-13 21:45:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 21:45:36,433][60934] Updated weights for policy 1, policy_version 20672 (0.0008) [2023-10-13 21:45:39,470][60935] Updated weights for policy 0, policy_version 20450 (0.0008) [2023-10-13 21:45:39,841][60935] Updated weights for policy 0, policy_version 20460 (0.0007) [2023-10-13 21:45:40,215][60935] Updated weights for policy 0, policy_version 20470 (0.0007) [2023-10-13 21:45:40,507][60934] Updated weights for policy 1, policy_version 20682 (0.0009) [2023-10-13 21:45:40,579][60935] Updated weights for policy 0, policy_version 20480 (0.0007) [2023-10-13 21:45:40,882][60934] Updated weights for policy 1, policy_version 20692 (0.0010) [2023-10-13 21:45:41,248][60934] Updated weights for policy 1, policy_version 20702 (0.0010) [2023-10-13 21:45:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 42139648. Throughput: 0: 1661.4, 1: 1676.8. Samples: 10546940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:45:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 21:45:44,806][60935] Updated weights for policy 0, policy_version 20490 (0.0009) [2023-10-13 21:45:45,176][60935] Updated weights for policy 0, policy_version 20500 (0.0008) [2023-10-13 21:45:45,328][60934] Updated weights for policy 1, policy_version 20712 (0.0009) [2023-10-13 21:45:45,547][60935] Updated weights for policy 0, policy_version 20510 (0.0008) [2023-10-13 21:45:45,698][60934] Updated weights for policy 1, policy_version 20722 (0.0009) [2023-10-13 21:45:46,073][60934] Updated weights for policy 1, policy_version 20732 (0.0010) [2023-10-13 21:45:46,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 42237952. Throughput: 0: 1674.2, 1: 1690.8. Samples: 10557746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:45:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 21:45:49,552][60935] Updated weights for policy 0, policy_version 20520 (0.0008) [2023-10-13 21:45:49,910][60935] Updated weights for policy 0, policy_version 20530 (0.0007) [2023-10-13 21:45:50,060][60934] Updated weights for policy 1, policy_version 20742 (0.0008) [2023-10-13 21:45:50,287][60935] Updated weights for policy 0, policy_version 20540 (0.0007) [2023-10-13 21:45:50,432][60934] Updated weights for policy 1, policy_version 20752 (0.0009) [2023-10-13 21:45:50,791][60934] Updated weights for policy 1, policy_version 20762 (0.0007) [2023-10-13 21:45:51,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 42303488. Throughput: 0: 1657.7, 1: 1688.1. Samples: 10577620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:45:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:45:54,342][60935] Updated weights for policy 0, policy_version 20550 (0.0008) [2023-10-13 21:45:54,726][60935] Updated weights for policy 0, policy_version 20560 (0.0008) [2023-10-13 21:45:54,758][60934] Updated weights for policy 1, policy_version 20772 (0.0008) [2023-10-13 21:45:55,099][60935] Updated weights for policy 0, policy_version 20570 (0.0009) [2023-10-13 21:45:55,133][60934] Updated weights for policy 1, policy_version 20782 (0.0007) [2023-10-13 21:45:55,497][60934] Updated weights for policy 1, policy_version 20792 (0.0008) [2023-10-13 21:45:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 42369024. Throughput: 0: 1664.1, 1: 1667.7. Samples: 10596780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:45:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:45:58,986][60935] Updated weights for policy 0, policy_version 20580 (0.0009) [2023-10-13 21:45:59,355][60935] Updated weights for policy 0, policy_version 20590 (0.0008) [2023-10-13 21:45:59,470][60934] Updated weights for policy 1, policy_version 20802 (0.0007) [2023-10-13 21:45:59,725][60935] Updated weights for policy 0, policy_version 20600 (0.0010) [2023-10-13 21:45:59,843][60934] Updated weights for policy 1, policy_version 20812 (0.0008) [2023-10-13 21:46:00,204][60934] Updated weights for policy 1, policy_version 20822 (0.0008) [2023-10-13 21:46:00,572][60934] Updated weights for policy 1, policy_version 20832 (0.0008) [2023-10-13 21:46:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 42434560. Throughput: 0: 1676.3, 1: 1687.9. Samples: 10608284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:46:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:46:04,090][60935] Updated weights for policy 0, policy_version 20610 (0.0009) [2023-10-13 21:46:04,490][60935] Updated weights for policy 0, policy_version 20620 (0.0008) [2023-10-13 21:46:04,582][60934] Updated weights for policy 1, policy_version 20842 (0.0010) [2023-10-13 21:46:04,851][60935] Updated weights for policy 0, policy_version 20630 (0.0009) [2023-10-13 21:46:04,945][60934] Updated weights for policy 1, policy_version 20852 (0.0007) [2023-10-13 21:46:05,222][60935] Updated weights for policy 0, policy_version 20640 (0.0007) [2023-10-13 21:46:05,305][60934] Updated weights for policy 1, policy_version 20862 (0.0009) [2023-10-13 21:46:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 42500096. Throughput: 0: 1646.3, 1: 1687.1. Samples: 10627776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:46:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:46:09,370][60935] Updated weights for policy 0, policy_version 20650 (0.0008) [2023-10-13 21:46:09,446][60934] Updated weights for policy 1, policy_version 20872 (0.0009) [2023-10-13 21:46:09,749][60935] Updated weights for policy 0, policy_version 20660 (0.0009) [2023-10-13 21:46:09,805][60934] Updated weights for policy 1, policy_version 20882 (0.0008) [2023-10-13 21:46:10,116][60935] Updated weights for policy 0, policy_version 20670 (0.0008) [2023-10-13 21:46:10,181][60934] Updated weights for policy 1, policy_version 20892 (0.0009) [2023-10-13 21:46:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 42565632. Throughput: 0: 1659.8, 1: 1668.7. Samples: 10646808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:46:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:46:14,168][60935] Updated weights for policy 0, policy_version 20680 (0.0008) [2023-10-13 21:46:14,301][60934] Updated weights for policy 1, policy_version 20902 (0.0008) [2023-10-13 21:46:14,540][60935] Updated weights for policy 0, policy_version 20690 (0.0008) [2023-10-13 21:46:14,658][60934] Updated weights for policy 1, policy_version 20912 (0.0009) [2023-10-13 21:46:14,915][60935] Updated weights for policy 0, policy_version 20700 (0.0008) [2023-10-13 21:46:15,036][60934] Updated weights for policy 1, policy_version 20922 (0.0008) [2023-10-13 21:46:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 42631168. Throughput: 0: 1660.9, 1: 1696.4. Samples: 10658360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:46:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.080')] [2023-10-13 21:46:19,092][60935] Updated weights for policy 0, policy_version 20710 (0.0008) [2023-10-13 21:46:19,169][60934] Updated weights for policy 1, policy_version 20932 (0.0008) [2023-10-13 21:46:19,460][60935] Updated weights for policy 0, policy_version 20720 (0.0008) [2023-10-13 21:46:19,532][60934] Updated weights for policy 1, policy_version 20942 (0.0007) [2023-10-13 21:46:19,828][60935] Updated weights for policy 0, policy_version 20730 (0.0009) [2023-10-13 21:46:19,905][60934] Updated weights for policy 1, policy_version 20952 (0.0008) [2023-10-13 21:46:21,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 42696704. Throughput: 0: 1646.0, 1: 1685.4. Samples: 10677540. Policy #0 lag: (min: 0.0, avg: 26.8, max: 32.0) [2023-10-13 21:46:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.080')] [2023-10-13 21:46:23,835][60934] Updated weights for policy 1, policy_version 20962 (0.0008) [2023-10-13 21:46:23,882][60935] Updated weights for policy 0, policy_version 20740 (0.0010) [2023-10-13 21:46:24,206][60934] Updated weights for policy 1, policy_version 20972 (0.0008) [2023-10-13 21:46:24,246][60935] Updated weights for policy 0, policy_version 20750 (0.0010) [2023-10-13 21:46:24,564][60934] Updated weights for policy 1, policy_version 20982 (0.0009) [2023-10-13 21:46:24,617][60935] Updated weights for policy 0, policy_version 20760 (0.0010) [2023-10-13 21:46:24,935][60934] Updated weights for policy 1, policy_version 20992 (0.0007) [2023-10-13 21:46:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 42762240. Throughput: 0: 1659.5, 1: 1681.7. Samples: 10697294. Policy #0 lag: (min: 0.0, avg: 26.8, max: 32.0) [2023-10-13 21:46:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.080')] [2023-10-13 21:46:28,863][60935] Updated weights for policy 0, policy_version 20770 (0.0010) [2023-10-13 21:46:29,141][60934] Updated weights for policy 1, policy_version 21002 (0.0008) [2023-10-13 21:46:29,233][60935] Updated weights for policy 0, policy_version 20780 (0.0008) [2023-10-13 21:46:29,511][60934] Updated weights for policy 1, policy_version 21012 (0.0008) [2023-10-13 21:46:29,605][60935] Updated weights for policy 0, policy_version 20790 (0.0008) [2023-10-13 21:46:29,885][60934] Updated weights for policy 1, policy_version 21022 (0.0007) [2023-10-13 21:46:29,967][60935] Updated weights for policy 0, policy_version 20800 (0.0009) [2023-10-13 21:46:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 42827776. Throughput: 0: 1659.8, 1: 1694.2. Samples: 10708678. Policy #0 lag: (min: 0.0, avg: 26.8, max: 32.0) [2023-10-13 21:46:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.080')] [2023-10-13 21:46:33,916][60934] Updated weights for policy 1, policy_version 21032 (0.0007) [2023-10-13 21:46:34,171][60935] Updated weights for policy 0, policy_version 20810 (0.0009) [2023-10-13 21:46:34,288][60934] Updated weights for policy 1, policy_version 21042 (0.0008) [2023-10-13 21:46:34,543][60935] Updated weights for policy 0, policy_version 20820 (0.0008) [2023-10-13 21:46:34,657][60934] Updated weights for policy 1, policy_version 21052 (0.0010) [2023-10-13 21:46:34,911][60935] Updated weights for policy 0, policy_version 20830 (0.0007) [2023-10-13 21:46:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 42893312. Throughput: 0: 1652.9, 1: 1675.2. Samples: 10727382. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-10-13 21:46:36,249][59943] Avg episode reward: [(0, '-0.020'), (1, '0.000')] [2023-10-13 21:46:38,813][60935] Updated weights for policy 0, policy_version 20840 (0.0009) [2023-10-13 21:46:38,829][60934] Updated weights for policy 1, policy_version 21062 (0.0008) [2023-10-13 21:46:39,187][60935] Updated weights for policy 0, policy_version 20850 (0.0010) [2023-10-13 21:46:39,203][60934] Updated weights for policy 1, policy_version 21072 (0.0008) [2023-10-13 21:46:39,565][60935] Updated weights for policy 0, policy_version 20860 (0.0008) [2023-10-13 21:46:39,567][60934] Updated weights for policy 1, policy_version 21082 (0.0008) [2023-10-13 21:46:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 42958848. Throughput: 0: 1665.7, 1: 1683.3. Samples: 10747482. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-10-13 21:46:41,249][59943] Avg episode reward: [(0, '-0.020'), (1, '0.000')] [2023-10-13 21:46:43,583][60934] Updated weights for policy 1, policy_version 21092 (0.0009) [2023-10-13 21:46:43,685][60935] Updated weights for policy 0, policy_version 20870 (0.0008) [2023-10-13 21:46:43,945][60934] Updated weights for policy 1, policy_version 21102 (0.0008) [2023-10-13 21:46:44,053][60935] Updated weights for policy 0, policy_version 20880 (0.0008) [2023-10-13 21:46:44,316][60934] Updated weights for policy 1, policy_version 21112 (0.0008) [2023-10-13 21:46:44,420][60935] Updated weights for policy 0, policy_version 20890 (0.0009) [2023-10-13 21:46:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 43024384. Throughput: 0: 1656.9, 1: 1686.8. Samples: 10758750. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-10-13 21:46:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:46:48,479][60934] Updated weights for policy 1, policy_version 21122 (0.0008) [2023-10-13 21:46:48,664][60935] Updated weights for policy 0, policy_version 20900 (0.0008) [2023-10-13 21:46:48,852][60934] Updated weights for policy 1, policy_version 21132 (0.0007) [2023-10-13 21:46:49,027][60935] Updated weights for policy 0, policy_version 20910 (0.0008) [2023-10-13 21:46:49,218][60934] Updated weights for policy 1, policy_version 21142 (0.0009) [2023-10-13 21:46:49,410][60935] Updated weights for policy 0, policy_version 20920 (0.0008) [2023-10-13 21:46:49,583][60934] Updated weights for policy 1, policy_version 21152 (0.0008) [2023-10-13 21:46:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 43089920. Throughput: 0: 1657.4, 1: 1662.2. Samples: 10777158. Policy #0 lag: (min: 20.0, avg: 27.0, max: 52.0) [2023-10-13 21:46:51,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 21:46:53,481][60935] Updated weights for policy 0, policy_version 20930 (0.0008) [2023-10-13 21:46:53,647][60934] Updated weights for policy 1, policy_version 21162 (0.0008) [2023-10-13 21:46:53,881][60935] Updated weights for policy 0, policy_version 20940 (0.0009) [2023-10-13 21:46:54,011][60934] Updated weights for policy 1, policy_version 21172 (0.0008) [2023-10-13 21:46:54,249][60935] Updated weights for policy 0, policy_version 20950 (0.0009) [2023-10-13 21:46:54,386][60934] Updated weights for policy 1, policy_version 21182 (0.0008) [2023-10-13 21:46:54,622][60935] Updated weights for policy 0, policy_version 20960 (0.0009) [2023-10-13 21:46:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 43155456. Throughput: 0: 1669.8, 1: 1679.6. Samples: 10797530. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-13 21:46:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:46:56,258][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000021184_21692416.pth... [2023-10-13 21:46:56,258][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000020960_21463040.pth... [2023-10-13 21:46:56,299][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000019616_20086784.pth [2023-10-13 21:46:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000019392_19857408.pth [2023-10-13 21:46:58,476][60934] Updated weights for policy 1, policy_version 21192 (0.0010) [2023-10-13 21:46:58,708][60935] Updated weights for policy 0, policy_version 20970 (0.0009) [2023-10-13 21:46:58,851][60934] Updated weights for policy 1, policy_version 21202 (0.0010) [2023-10-13 21:46:59,076][60935] Updated weights for policy 0, policy_version 20980 (0.0008) [2023-10-13 21:46:59,219][60934] Updated weights for policy 1, policy_version 21212 (0.0008) [2023-10-13 21:46:59,453][60935] Updated weights for policy 0, policy_version 20990 (0.0008) [2023-10-13 21:47:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 43220992. Throughput: 0: 1657.8, 1: 1671.0. Samples: 10808156. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-13 21:47:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:47:03,256][60934] Updated weights for policy 1, policy_version 21222 (0.0009) [2023-10-13 21:47:03,565][60935] Updated weights for policy 0, policy_version 21000 (0.0008) [2023-10-13 21:47:03,622][60934] Updated weights for policy 1, policy_version 21232 (0.0009) [2023-10-13 21:47:03,930][60935] Updated weights for policy 0, policy_version 21010 (0.0008) [2023-10-13 21:47:03,986][60934] Updated weights for policy 1, policy_version 21242 (0.0009) [2023-10-13 21:47:04,311][60935] Updated weights for policy 0, policy_version 21020 (0.0007) [2023-10-13 21:47:06,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 43286528. Throughput: 0: 1660.0, 1: 1660.8. Samples: 10826974. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-13 21:47:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:47:08,019][60934] Updated weights for policy 1, policy_version 21252 (0.0008) [2023-10-13 21:47:08,332][60935] Updated weights for policy 0, policy_version 21030 (0.0008) [2023-10-13 21:47:08,378][60934] Updated weights for policy 1, policy_version 21262 (0.0009) [2023-10-13 21:47:08,698][60935] Updated weights for policy 0, policy_version 21040 (0.0010) [2023-10-13 21:47:08,744][60934] Updated weights for policy 1, policy_version 21272 (0.0008) [2023-10-13 21:47:09,072][60935] Updated weights for policy 0, policy_version 21050 (0.0008) [2023-10-13 21:47:11,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 43352064. Throughput: 0: 1663.6, 1: 1673.0. Samples: 10847442. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-13 21:47:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:47:12,870][60934] Updated weights for policy 1, policy_version 21282 (0.0008) [2023-10-13 21:47:13,227][60934] Updated weights for policy 1, policy_version 21292 (0.0010) [2023-10-13 21:47:13,275][60935] Updated weights for policy 0, policy_version 21060 (0.0008) [2023-10-13 21:47:13,599][60934] Updated weights for policy 1, policy_version 21302 (0.0007) [2023-10-13 21:47:13,647][60935] Updated weights for policy 0, policy_version 21070 (0.0007) [2023-10-13 21:47:13,967][60934] Updated weights for policy 1, policy_version 21312 (0.0009) [2023-10-13 21:47:14,013][60935] Updated weights for policy 0, policy_version 21080 (0.0008) [2023-10-13 21:47:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 43417600. Throughput: 0: 1645.3, 1: 1659.2. Samples: 10857382. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-13 21:47:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:47:18,132][60934] Updated weights for policy 1, policy_version 21322 (0.0008) [2023-10-13 21:47:18,288][60935] Updated weights for policy 0, policy_version 21090 (0.0009) [2023-10-13 21:47:18,502][60934] Updated weights for policy 1, policy_version 21332 (0.0008) [2023-10-13 21:47:18,643][60935] Updated weights for policy 0, policy_version 21100 (0.0009) [2023-10-13 21:47:18,876][60934] Updated weights for policy 1, policy_version 21342 (0.0007) [2023-10-13 21:47:19,016][60935] Updated weights for policy 0, policy_version 21110 (0.0009) [2023-10-13 21:47:19,389][60935] Updated weights for policy 0, policy_version 21120 (0.0010) [2023-10-13 21:47:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 43483136. Throughput: 0: 1649.6, 1: 1669.2. Samples: 10876730. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-13 21:47:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:47:23,034][60934] Updated weights for policy 1, policy_version 21352 (0.0007) [2023-10-13 21:47:23,394][60934] Updated weights for policy 1, policy_version 21362 (0.0007) [2023-10-13 21:47:23,505][60935] Updated weights for policy 0, policy_version 21130 (0.0007) [2023-10-13 21:47:23,768][60934] Updated weights for policy 1, policy_version 21372 (0.0008) [2023-10-13 21:47:23,874][60935] Updated weights for policy 0, policy_version 21140 (0.0008) [2023-10-13 21:47:24,242][60935] Updated weights for policy 0, policy_version 21150 (0.0007) [2023-10-13 21:47:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 43548672. Throughput: 0: 1651.9, 1: 1678.0. Samples: 10897328. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-13 21:47:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:47:28,013][60934] Updated weights for policy 1, policy_version 21382 (0.0009) [2023-10-13 21:47:28,311][60935] Updated weights for policy 0, policy_version 21160 (0.0009) [2023-10-13 21:47:28,399][60934] Updated weights for policy 1, policy_version 21392 (0.0008) [2023-10-13 21:47:28,672][60935] Updated weights for policy 0, policy_version 21170 (0.0007) [2023-10-13 21:47:28,770][60934] Updated weights for policy 1, policy_version 21402 (0.0008) [2023-10-13 21:47:29,038][60935] Updated weights for policy 0, policy_version 21180 (0.0007) [2023-10-13 21:47:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 43614208. Throughput: 0: 1638.5, 1: 1656.6. Samples: 10907030. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-13 21:47:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:47:32,891][60934] Updated weights for policy 1, policy_version 21412 (0.0010) [2023-10-13 21:47:33,232][60935] Updated weights for policy 0, policy_version 21190 (0.0009) [2023-10-13 21:47:33,257][60934] Updated weights for policy 1, policy_version 21422 (0.0009) [2023-10-13 21:47:33,608][60935] Updated weights for policy 0, policy_version 21200 (0.0008) [2023-10-13 21:47:33,624][60934] Updated weights for policy 1, policy_version 21432 (0.0007) [2023-10-13 21:47:33,982][60935] Updated weights for policy 0, policy_version 21210 (0.0009) [2023-10-13 21:47:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 43679744. Throughput: 0: 1652.7, 1: 1671.2. Samples: 10926734. Policy #0 lag: (min: 11.0, avg: 21.4, max: 43.0) [2023-10-13 21:47:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:47:37,675][60934] Updated weights for policy 1, policy_version 21442 (0.0008) [2023-10-13 21:47:38,042][60934] Updated weights for policy 1, policy_version 21452 (0.0008) [2023-10-13 21:47:38,094][60935] Updated weights for policy 0, policy_version 21220 (0.0012) [2023-10-13 21:47:38,410][60934] Updated weights for policy 1, policy_version 21462 (0.0008) [2023-10-13 21:47:38,456][60935] Updated weights for policy 0, policy_version 21230 (0.0008) [2023-10-13 21:47:38,775][60934] Updated weights for policy 1, policy_version 21472 (0.0011) [2023-10-13 21:47:38,825][60935] Updated weights for policy 0, policy_version 21240 (0.0007) [2023-10-13 21:47:41,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.5). Total num frames: 43745280. Throughput: 0: 1653.7, 1: 1674.1. Samples: 10947284. Policy #0 lag: (min: 11.0, avg: 21.4, max: 43.0) [2023-10-13 21:47:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:47:42,729][60934] Updated weights for policy 1, policy_version 21482 (0.0007) [2023-10-13 21:47:43,011][60935] Updated weights for policy 0, policy_version 21250 (0.0008) [2023-10-13 21:47:43,104][60934] Updated weights for policy 1, policy_version 21492 (0.0008) [2023-10-13 21:47:43,410][60935] Updated weights for policy 0, policy_version 21260 (0.0008) [2023-10-13 21:47:43,458][60934] Updated weights for policy 1, policy_version 21502 (0.0008) [2023-10-13 21:47:43,777][60935] Updated weights for policy 0, policy_version 21270 (0.0008) [2023-10-13 21:47:44,147][60935] Updated weights for policy 0, policy_version 21280 (0.0008) [2023-10-13 21:47:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 43810816. Throughput: 0: 1642.5, 1: 1658.2. Samples: 10956688. Policy #0 lag: (min: 11.0, avg: 21.4, max: 43.0) [2023-10-13 21:47:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:47:47,523][60934] Updated weights for policy 1, policy_version 21512 (0.0008) [2023-10-13 21:47:47,896][60934] Updated weights for policy 1, policy_version 21522 (0.0009) [2023-10-13 21:47:48,250][60935] Updated weights for policy 0, policy_version 21290 (0.0008) [2023-10-13 21:47:48,258][60934] Updated weights for policy 1, policy_version 21532 (0.0009) [2023-10-13 21:47:48,617][60935] Updated weights for policy 0, policy_version 21300 (0.0009) [2023-10-13 21:47:48,989][60935] Updated weights for policy 0, policy_version 21310 (0.0009) [2023-10-13 21:47:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 43876352. Throughput: 0: 1654.6, 1: 1679.2. Samples: 10976994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:47:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:47:52,202][60934] Updated weights for policy 1, policy_version 21542 (0.0008) [2023-10-13 21:47:52,562][60934] Updated weights for policy 1, policy_version 21552 (0.0007) [2023-10-13 21:47:52,938][60934] Updated weights for policy 1, policy_version 21562 (0.0007) [2023-10-13 21:47:53,126][60935] Updated weights for policy 0, policy_version 21320 (0.0008) [2023-10-13 21:47:53,484][60935] Updated weights for policy 0, policy_version 21330 (0.0010) [2023-10-13 21:47:53,849][60935] Updated weights for policy 0, policy_version 21340 (0.0009) [2023-10-13 21:47:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 43941888. Throughput: 0: 1659.3, 1: 1683.5. Samples: 10997868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:47:56,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:47:57,055][60934] Updated weights for policy 1, policy_version 21572 (0.0007) [2023-10-13 21:47:57,411][60934] Updated weights for policy 1, policy_version 21582 (0.0009) [2023-10-13 21:47:57,779][60934] Updated weights for policy 1, policy_version 21592 (0.0009) [2023-10-13 21:47:58,056][60935] Updated weights for policy 0, policy_version 21350 (0.0008) [2023-10-13 21:47:58,424][60935] Updated weights for policy 0, policy_version 21360 (0.0010) [2023-10-13 21:47:58,797][60935] Updated weights for policy 0, policy_version 21370 (0.0009) [2023-10-13 21:48:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 44007424. Throughput: 0: 1656.8, 1: 1671.6. Samples: 11007162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:48:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:48:01,759][60934] Updated weights for policy 1, policy_version 21602 (0.0009) [2023-10-13 21:48:02,136][60934] Updated weights for policy 1, policy_version 21612 (0.0009) [2023-10-13 21:48:02,513][60934] Updated weights for policy 1, policy_version 21622 (0.0010) [2023-10-13 21:48:02,882][60934] Updated weights for policy 1, policy_version 21632 (0.0010) [2023-10-13 21:48:02,921][60935] Updated weights for policy 0, policy_version 21380 (0.0009) [2023-10-13 21:48:03,274][60935] Updated weights for policy 0, policy_version 21390 (0.0010) [2023-10-13 21:48:03,651][60935] Updated weights for policy 0, policy_version 21400 (0.0009) [2023-10-13 21:48:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 44072960. Throughput: 0: 1673.7, 1: 1681.5. Samples: 11027716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:48:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:48:06,849][60934] Updated weights for policy 1, policy_version 21642 (0.0008) [2023-10-13 21:48:07,222][60934] Updated weights for policy 1, policy_version 21652 (0.0008) [2023-10-13 21:48:07,572][60935] Updated weights for policy 0, policy_version 21410 (0.0008) [2023-10-13 21:48:07,588][60934] Updated weights for policy 1, policy_version 21662 (0.0010) [2023-10-13 21:48:07,947][60935] Updated weights for policy 0, policy_version 21420 (0.0011) [2023-10-13 21:48:08,317][60935] Updated weights for policy 0, policy_version 21430 (0.0009) [2023-10-13 21:48:08,686][60935] Updated weights for policy 0, policy_version 21440 (0.0009) [2023-10-13 21:48:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 44138496. Throughput: 0: 1673.2, 1: 1685.5. Samples: 11048468. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-13 21:48:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:48:11,643][60934] Updated weights for policy 1, policy_version 21672 (0.0007) [2023-10-13 21:48:12,001][60934] Updated weights for policy 1, policy_version 21682 (0.0007) [2023-10-13 21:48:12,373][60934] Updated weights for policy 1, policy_version 21692 (0.0008) [2023-10-13 21:48:12,756][60935] Updated weights for policy 0, policy_version 21450 (0.0009) [2023-10-13 21:48:13,123][60935] Updated weights for policy 0, policy_version 21460 (0.0010) [2023-10-13 21:48:13,491][60935] Updated weights for policy 0, policy_version 21470 (0.0011) [2023-10-13 21:48:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44204032. Throughput: 0: 1662.8, 1: 1682.9. Samples: 11057586. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-13 21:48:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:48:16,541][60934] Updated weights for policy 1, policy_version 21702 (0.0007) [2023-10-13 21:48:16,924][60934] Updated weights for policy 1, policy_version 21712 (0.0009) [2023-10-13 21:48:17,299][60934] Updated weights for policy 1, policy_version 21722 (0.0009) [2023-10-13 21:48:17,722][60935] Updated weights for policy 0, policy_version 21480 (0.0008) [2023-10-13 21:48:18,089][60935] Updated weights for policy 0, policy_version 21490 (0.0008) [2023-10-13 21:48:18,452][60935] Updated weights for policy 0, policy_version 21500 (0.0011) [2023-10-13 21:48:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44269568. Throughput: 0: 1668.3, 1: 1694.9. Samples: 11078078. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-13 21:48:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:48:21,336][60934] Updated weights for policy 1, policy_version 21732 (0.0007) [2023-10-13 21:48:21,715][60934] Updated weights for policy 1, policy_version 21742 (0.0008) [2023-10-13 21:48:22,085][60934] Updated weights for policy 1, policy_version 21752 (0.0009) [2023-10-13 21:48:22,579][60935] Updated weights for policy 0, policy_version 21510 (0.0008) [2023-10-13 21:48:22,953][60935] Updated weights for policy 0, policy_version 21520 (0.0008) [2023-10-13 21:48:23,319][60935] Updated weights for policy 0, policy_version 21530 (0.0007) [2023-10-13 21:48:26,045][60934] Updated weights for policy 1, policy_version 21762 (0.0008) [2023-10-13 21:48:26,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44335104. Throughput: 0: 1670.6, 1: 1696.0. Samples: 11098778. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-13 21:48:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:48:26,415][60934] Updated weights for policy 1, policy_version 21772 (0.0007) [2023-10-13 21:48:26,789][60934] Updated weights for policy 1, policy_version 21782 (0.0008) [2023-10-13 21:48:27,154][60934] Updated weights for policy 1, policy_version 21792 (0.0008) [2023-10-13 21:48:27,354][60935] Updated weights for policy 0, policy_version 21540 (0.0008) [2023-10-13 21:48:27,739][60935] Updated weights for policy 0, policy_version 21550 (0.0009) [2023-10-13 21:48:28,112][60935] Updated weights for policy 0, policy_version 21560 (0.0007) [2023-10-13 21:48:31,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44400640. Throughput: 0: 1664.3, 1: 1693.4. Samples: 11107786. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-13 21:48:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:48:31,255][60934] Updated weights for policy 1, policy_version 21802 (0.0007) [2023-10-13 21:48:31,622][60934] Updated weights for policy 1, policy_version 21812 (0.0008) [2023-10-13 21:48:31,985][60934] Updated weights for policy 1, policy_version 21822 (0.0009) [2023-10-13 21:48:32,034][60935] Updated weights for policy 0, policy_version 21570 (0.0010) [2023-10-13 21:48:32,408][60935] Updated weights for policy 0, policy_version 21580 (0.0008) [2023-10-13 21:48:32,784][60935] Updated weights for policy 0, policy_version 21590 (0.0009) [2023-10-13 21:48:33,160][60935] Updated weights for policy 0, policy_version 21600 (0.0009) [2023-10-13 21:48:35,880][60934] Updated weights for policy 1, policy_version 21832 (0.0010) [2023-10-13 21:48:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44466176. Throughput: 0: 1673.9, 1: 1694.9. Samples: 11128592. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-13 21:48:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:48:36,254][60934] Updated weights for policy 1, policy_version 21842 (0.0009) [2023-10-13 21:48:36,625][60934] Updated weights for policy 1, policy_version 21852 (0.0007) [2023-10-13 21:48:37,336][60935] Updated weights for policy 0, policy_version 21610 (0.0009) [2023-10-13 21:48:37,698][60935] Updated weights for policy 0, policy_version 21620 (0.0008) [2023-10-13 21:48:38,070][60935] Updated weights for policy 0, policy_version 21630 (0.0008) [2023-10-13 21:48:40,734][60934] Updated weights for policy 1, policy_version 21862 (0.0007) [2023-10-13 21:48:41,109][60934] Updated weights for policy 1, policy_version 21872 (0.0007) [2023-10-13 21:48:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 44531712. Throughput: 0: 1675.2, 1: 1690.5. Samples: 11149328. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-13 21:48:41,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:48:41,485][60934] Updated weights for policy 1, policy_version 21882 (0.0009) [2023-10-13 21:48:42,021][60935] Updated weights for policy 0, policy_version 21640 (0.0009) [2023-10-13 21:48:42,399][60935] Updated weights for policy 0, policy_version 21650 (0.0011) [2023-10-13 21:48:42,765][60935] Updated weights for policy 0, policy_version 21660 (0.0009) [2023-10-13 21:48:45,408][60934] Updated weights for policy 1, policy_version 21892 (0.0010) [2023-10-13 21:48:45,772][60934] Updated weights for policy 1, policy_version 21902 (0.0010) [2023-10-13 21:48:46,139][60934] Updated weights for policy 1, policy_version 21912 (0.0010) [2023-10-13 21:48:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44597248. Throughput: 0: 1671.6, 1: 1696.1. Samples: 11158706. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-13 21:48:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:48:46,780][60935] Updated weights for policy 0, policy_version 21670 (0.0008) [2023-10-13 21:48:47,147][60935] Updated weights for policy 0, policy_version 21680 (0.0008) [2023-10-13 21:48:47,516][60935] Updated weights for policy 0, policy_version 21690 (0.0009) [2023-10-13 21:48:50,119][60934] Updated weights for policy 1, policy_version 21922 (0.0010) [2023-10-13 21:48:50,491][60934] Updated weights for policy 1, policy_version 21932 (0.0008) [2023-10-13 21:48:50,860][60934] Updated weights for policy 1, policy_version 21942 (0.0007) [2023-10-13 21:48:51,236][60934] Updated weights for policy 1, policy_version 21952 (0.0009) [2023-10-13 21:48:51,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 44695552. Throughput: 0: 1673.0, 1: 1695.1. Samples: 11179280. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-13 21:48:51,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:48:51,621][60935] Updated weights for policy 0, policy_version 21700 (0.0009) [2023-10-13 21:48:51,990][60935] Updated weights for policy 0, policy_version 21710 (0.0008) [2023-10-13 21:48:52,355][60935] Updated weights for policy 0, policy_version 21720 (0.0008) [2023-10-13 21:48:55,254][60934] Updated weights for policy 1, policy_version 21962 (0.0009) [2023-10-13 21:48:55,626][60934] Updated weights for policy 1, policy_version 21972 (0.0008) [2023-10-13 21:48:55,998][60934] Updated weights for policy 1, policy_version 21982 (0.0007) [2023-10-13 21:48:56,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 44761088. Throughput: 0: 1677.6, 1: 1683.5. Samples: 11199718. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-13 21:48:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:48:56,257][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000021984_22511616.pth... [2023-10-13 21:48:56,290][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000020416_20905984.pth [2023-10-13 21:48:56,374][60935] Updated weights for policy 0, policy_version 21730 (0.0007) [2023-10-13 21:48:56,745][60935] Updated weights for policy 0, policy_version 21740 (0.0008) [2023-10-13 21:48:57,125][60935] Updated weights for policy 0, policy_version 21750 (0.0010) [2023-10-13 21:48:57,487][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000021760_22282240.pth... [2023-10-13 21:48:57,491][60935] Updated weights for policy 0, policy_version 21760 (0.0010) [2023-10-13 21:48:57,517][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000020192_20676608.pth [2023-10-13 21:49:00,255][60934] Updated weights for policy 1, policy_version 21992 (0.0008) [2023-10-13 21:49:00,632][60934] Updated weights for policy 1, policy_version 22002 (0.0008) [2023-10-13 21:49:01,002][60934] Updated weights for policy 1, policy_version 22012 (0.0007) [2023-10-13 21:49:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 44826624. Throughput: 0: 1675.6, 1: 1697.7. Samples: 11209384. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-13 21:49:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:49:01,883][60935] Updated weights for policy 0, policy_version 21770 (0.0009) [2023-10-13 21:49:02,244][60935] Updated weights for policy 0, policy_version 21780 (0.0011) [2023-10-13 21:49:02,608][60935] Updated weights for policy 0, policy_version 21790 (0.0008) [2023-10-13 21:49:05,104][60934] Updated weights for policy 1, policy_version 22022 (0.0007) [2023-10-13 21:49:05,492][60934] Updated weights for policy 1, policy_version 22032 (0.0008) [2023-10-13 21:49:05,861][60934] Updated weights for policy 1, policy_version 22042 (0.0007) [2023-10-13 21:49:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 44892160. Throughput: 0: 1680.7, 1: 1698.5. Samples: 11230144. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-13 21:49:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:49:06,597][60935] Updated weights for policy 0, policy_version 21800 (0.0009) [2023-10-13 21:49:06,963][60935] Updated weights for policy 0, policy_version 21810 (0.0011) [2023-10-13 21:49:07,338][60935] Updated weights for policy 0, policy_version 21820 (0.0009) [2023-10-13 21:49:09,912][60934] Updated weights for policy 1, policy_version 22052 (0.0008) [2023-10-13 21:49:10,280][60934] Updated weights for policy 1, policy_version 22062 (0.0007) [2023-10-13 21:49:10,646][60934] Updated weights for policy 1, policy_version 22072 (0.0007) [2023-10-13 21:49:11,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 44957696. Throughput: 0: 1680.2, 1: 1678.6. Samples: 11249924. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 21:49:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:49:11,430][60935] Updated weights for policy 0, policy_version 21830 (0.0008) [2023-10-13 21:49:11,794][60935] Updated weights for policy 0, policy_version 21840 (0.0010) [2023-10-13 21:49:12,173][60935] Updated weights for policy 0, policy_version 21850 (0.0008) [2023-10-13 21:49:14,573][60934] Updated weights for policy 1, policy_version 22082 (0.0007) [2023-10-13 21:49:14,946][60934] Updated weights for policy 1, policy_version 22092 (0.0007) [2023-10-13 21:49:15,306][60934] Updated weights for policy 1, policy_version 22102 (0.0008) [2023-10-13 21:49:15,682][60934] Updated weights for policy 1, policy_version 22112 (0.0008) [2023-10-13 21:49:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 45023232. Throughput: 0: 1679.4, 1: 1700.5. Samples: 11259880. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 21:49:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:49:16,331][60935] Updated weights for policy 0, policy_version 21860 (0.0010) [2023-10-13 21:49:16,713][60935] Updated weights for policy 0, policy_version 21870 (0.0007) [2023-10-13 21:49:17,081][60935] Updated weights for policy 0, policy_version 21880 (0.0007) [2023-10-13 21:49:19,663][60934] Updated weights for policy 1, policy_version 22122 (0.0007) [2023-10-13 21:49:20,028][60934] Updated weights for policy 1, policy_version 22132 (0.0008) [2023-10-13 21:49:20,397][60934] Updated weights for policy 1, policy_version 22142 (0.0009) [2023-10-13 21:49:21,142][60935] Updated weights for policy 0, policy_version 21890 (0.0009) [2023-10-13 21:49:21,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 45088768. Throughput: 0: 1678.7, 1: 1696.7. Samples: 11280486. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 21:49:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:49:21,515][60935] Updated weights for policy 0, policy_version 21900 (0.0009) [2023-10-13 21:49:21,880][60935] Updated weights for policy 0, policy_version 21910 (0.0010) [2023-10-13 21:49:22,246][60935] Updated weights for policy 0, policy_version 21920 (0.0010) [2023-10-13 21:49:24,422][60934] Updated weights for policy 1, policy_version 22152 (0.0009) [2023-10-13 21:49:24,800][60934] Updated weights for policy 1, policy_version 22162 (0.0010) [2023-10-13 21:49:25,165][60934] Updated weights for policy 1, policy_version 22172 (0.0009) [2023-10-13 21:49:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 45154304. Throughput: 0: 1674.3, 1: 1674.4. Samples: 11300020. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 21:49:26,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:49:26,433][60935] Updated weights for policy 0, policy_version 21930 (0.0009) [2023-10-13 21:49:26,802][60935] Updated weights for policy 0, policy_version 21940 (0.0008) [2023-10-13 21:49:27,162][60935] Updated weights for policy 0, policy_version 21950 (0.0009) [2023-10-13 21:49:29,168][60934] Updated weights for policy 1, policy_version 22182 (0.0007) [2023-10-13 21:49:29,535][60934] Updated weights for policy 1, policy_version 22192 (0.0007) [2023-10-13 21:49:29,906][60934] Updated weights for policy 1, policy_version 22202 (0.0007) [2023-10-13 21:49:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 45219840. Throughput: 0: 1668.9, 1: 1703.9. Samples: 11310480. Policy #0 lag: (min: 1.0, avg: 7.6, max: 33.0) [2023-10-13 21:49:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:49:31,489][60935] Updated weights for policy 0, policy_version 21960 (0.0009) [2023-10-13 21:49:31,861][60935] Updated weights for policy 0, policy_version 21970 (0.0009) [2023-10-13 21:49:32,243][60935] Updated weights for policy 0, policy_version 21980 (0.0008) [2023-10-13 21:49:33,892][60934] Updated weights for policy 1, policy_version 22212 (0.0008) [2023-10-13 21:49:34,266][60934] Updated weights for policy 1, policy_version 22222 (0.0007) [2023-10-13 21:49:34,637][60934] Updated weights for policy 1, policy_version 22232 (0.0008) [2023-10-13 21:49:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 45285376. Throughput: 0: 1668.8, 1: 1686.8. Samples: 11330282. Policy #0 lag: (min: 1.0, avg: 7.6, max: 33.0) [2023-10-13 21:49:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:49:36,368][60935] Updated weights for policy 0, policy_version 21990 (0.0009) [2023-10-13 21:49:36,730][60935] Updated weights for policy 0, policy_version 22000 (0.0010) [2023-10-13 21:49:37,105][60935] Updated weights for policy 0, policy_version 22010 (0.0009) [2023-10-13 21:49:38,635][60934] Updated weights for policy 1, policy_version 22242 (0.0009) [2023-10-13 21:49:39,005][60934] Updated weights for policy 1, policy_version 22252 (0.0008) [2023-10-13 21:49:39,366][60934] Updated weights for policy 1, policy_version 22262 (0.0009) [2023-10-13 21:49:39,734][60934] Updated weights for policy 1, policy_version 22272 (0.0009) [2023-10-13 21:49:40,914][60935] Updated weights for policy 0, policy_version 22020 (0.0009) [2023-10-13 21:49:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 45350912. Throughput: 0: 1665.9, 1: 1687.1. Samples: 11350600. Policy #0 lag: (min: 1.0, avg: 7.6, max: 33.0) [2023-10-13 21:49:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:49:41,285][60935] Updated weights for policy 0, policy_version 22030 (0.0008) [2023-10-13 21:49:41,657][60935] Updated weights for policy 0, policy_version 22040 (0.0008) [2023-10-13 21:49:43,831][60934] Updated weights for policy 1, policy_version 22282 (0.0010) [2023-10-13 21:49:44,196][60934] Updated weights for policy 1, policy_version 22292 (0.0010) [2023-10-13 21:49:44,569][60934] Updated weights for policy 1, policy_version 22302 (0.0009) [2023-10-13 21:49:45,591][60935] Updated weights for policy 0, policy_version 22050 (0.0009) [2023-10-13 21:49:45,964][60935] Updated weights for policy 0, policy_version 22060 (0.0007) [2023-10-13 21:49:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 45416448. Throughput: 0: 1675.8, 1: 1698.5. Samples: 11361228. Policy #0 lag: (min: 1.0, avg: 7.6, max: 33.0) [2023-10-13 21:49:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:49:46,331][60935] Updated weights for policy 0, policy_version 22070 (0.0008) [2023-10-13 21:49:46,703][60935] Updated weights for policy 0, policy_version 22080 (0.0007) [2023-10-13 21:49:48,565][60934] Updated weights for policy 1, policy_version 22312 (0.0007) [2023-10-13 21:49:48,938][60934] Updated weights for policy 1, policy_version 22322 (0.0007) [2023-10-13 21:49:49,314][60934] Updated weights for policy 1, policy_version 22332 (0.0009) [2023-10-13 21:49:50,899][60935] Updated weights for policy 0, policy_version 22090 (0.0008) [2023-10-13 21:49:51,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 45481984. Throughput: 0: 1675.5, 1: 1672.0. Samples: 11380784. Policy #0 lag: (min: 11.0, avg: 16.1, max: 43.0) [2023-10-13 21:49:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:49:51,266][60935] Updated weights for policy 0, policy_version 22100 (0.0009) [2023-10-13 21:49:51,640][60935] Updated weights for policy 0, policy_version 22110 (0.0008) [2023-10-13 21:49:53,367][60934] Updated weights for policy 1, policy_version 22342 (0.0009) [2023-10-13 21:49:53,748][60934] Updated weights for policy 1, policy_version 22352 (0.0007) [2023-10-13 21:49:54,113][60934] Updated weights for policy 1, policy_version 22362 (0.0007) [2023-10-13 21:49:55,636][60935] Updated weights for policy 0, policy_version 22120 (0.0010) [2023-10-13 21:49:56,011][60935] Updated weights for policy 0, policy_version 22130 (0.0011) [2023-10-13 21:49:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 45547520. Throughput: 0: 1663.6, 1: 1692.2. Samples: 11400936. Policy #0 lag: (min: 11.0, avg: 16.1, max: 43.0) [2023-10-13 21:49:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:49:56,383][60935] Updated weights for policy 0, policy_version 22140 (0.0009) [2023-10-13 21:49:58,116][60934] Updated weights for policy 1, policy_version 22372 (0.0009) [2023-10-13 21:49:58,483][60934] Updated weights for policy 1, policy_version 22382 (0.0011) [2023-10-13 21:49:58,854][60934] Updated weights for policy 1, policy_version 22392 (0.0010) [2023-10-13 21:50:00,442][60935] Updated weights for policy 0, policy_version 22150 (0.0008) [2023-10-13 21:50:00,810][60935] Updated weights for policy 0, policy_version 22160 (0.0009) [2023-10-13 21:50:01,188][60935] Updated weights for policy 0, policy_version 22170 (0.0008) [2023-10-13 21:50:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 45613056. Throughput: 0: 1676.4, 1: 1688.1. Samples: 11411282. Policy #0 lag: (min: 11.0, avg: 16.1, max: 43.0) [2023-10-13 21:50:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:50:02,868][60934] Updated weights for policy 1, policy_version 22402 (0.0009) [2023-10-13 21:50:03,249][60934] Updated weights for policy 1, policy_version 22412 (0.0007) [2023-10-13 21:50:03,622][60934] Updated weights for policy 1, policy_version 22422 (0.0008) [2023-10-13 21:50:03,987][60934] Updated weights for policy 1, policy_version 22432 (0.0007) [2023-10-13 21:50:05,219][60935] Updated weights for policy 0, policy_version 22180 (0.0009) [2023-10-13 21:50:05,605][60935] Updated weights for policy 0, policy_version 22190 (0.0009) [2023-10-13 21:50:05,962][60935] Updated weights for policy 0, policy_version 22200 (0.0007) [2023-10-13 21:50:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45678592. Throughput: 0: 1682.0, 1: 1677.3. Samples: 11431654. Policy #0 lag: (min: 11.0, avg: 16.1, max: 43.0) [2023-10-13 21:50:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:50:07,901][60934] Updated weights for policy 1, policy_version 22442 (0.0008) [2023-10-13 21:50:08,268][60934] Updated weights for policy 1, policy_version 22452 (0.0009) [2023-10-13 21:50:08,635][60934] Updated weights for policy 1, policy_version 22462 (0.0010) [2023-10-13 21:50:09,944][60935] Updated weights for policy 0, policy_version 22210 (0.0008) [2023-10-13 21:50:10,314][60935] Updated weights for policy 0, policy_version 22220 (0.0008) [2023-10-13 21:50:10,687][60935] Updated weights for policy 0, policy_version 22230 (0.0010) [2023-10-13 21:50:11,052][60935] Updated weights for policy 0, policy_version 22240 (0.0010) [2023-10-13 21:50:11,248][59943] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 45776896. Throughput: 0: 1660.5, 1: 1704.5. Samples: 11451446. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-13 21:50:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:50:12,650][60934] Updated weights for policy 1, policy_version 22472 (0.0009) [2023-10-13 21:50:13,017][60934] Updated weights for policy 1, policy_version 22482 (0.0007) [2023-10-13 21:50:13,381][60934] Updated weights for policy 1, policy_version 22492 (0.0009) [2023-10-13 21:50:15,198][60935] Updated weights for policy 0, policy_version 22250 (0.0011) [2023-10-13 21:50:15,572][60935] Updated weights for policy 0, policy_version 22260 (0.0011) [2023-10-13 21:50:15,935][60935] Updated weights for policy 0, policy_version 22270 (0.0011) [2023-10-13 21:50:16,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 45842432. Throughput: 0: 1683.7, 1: 1674.7. Samples: 11461610. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-13 21:50:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:50:17,511][60934] Updated weights for policy 1, policy_version 22502 (0.0009) [2023-10-13 21:50:17,888][60934] Updated weights for policy 1, policy_version 22512 (0.0010) [2023-10-13 21:50:18,246][60934] Updated weights for policy 1, policy_version 22522 (0.0009) [2023-10-13 21:50:20,127][60935] Updated weights for policy 0, policy_version 22280 (0.0008) [2023-10-13 21:50:20,492][60935] Updated weights for policy 0, policy_version 22290 (0.0009) [2023-10-13 21:50:20,860][60935] Updated weights for policy 0, policy_version 22300 (0.0008) [2023-10-13 21:50:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 45907968. Throughput: 0: 1685.9, 1: 1688.6. Samples: 11482134. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-13 21:50:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:50:22,287][60934] Updated weights for policy 1, policy_version 22532 (0.0007) [2023-10-13 21:50:22,659][60934] Updated weights for policy 1, policy_version 22542 (0.0007) [2023-10-13 21:50:23,023][60934] Updated weights for policy 1, policy_version 22552 (0.0008) [2023-10-13 21:50:24,955][60935] Updated weights for policy 0, policy_version 22310 (0.0010) [2023-10-13 21:50:25,326][60935] Updated weights for policy 0, policy_version 22320 (0.0008) [2023-10-13 21:50:25,691][60935] Updated weights for policy 0, policy_version 22330 (0.0010) [2023-10-13 21:50:26,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 45973504. Throughput: 0: 1659.3, 1: 1700.0. Samples: 11501772. Policy #0 lag: (min: 26.0, avg: 37.0, max: 58.0) [2023-10-13 21:50:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-13 21:50:27,193][60934] Updated weights for policy 1, policy_version 22562 (0.0008) [2023-10-13 21:50:27,556][60934] Updated weights for policy 1, policy_version 22572 (0.0009) [2023-10-13 21:50:27,923][60934] Updated weights for policy 1, policy_version 22582 (0.0007) [2023-10-13 21:50:28,297][60934] Updated weights for policy 1, policy_version 22592 (0.0010) [2023-10-13 21:50:29,705][60935] Updated weights for policy 0, policy_version 22340 (0.0007) [2023-10-13 21:50:30,079][60935] Updated weights for policy 0, policy_version 22350 (0.0007) [2023-10-13 21:50:30,445][60935] Updated weights for policy 0, policy_version 22360 (0.0008) [2023-10-13 21:50:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 46039040. Throughput: 0: 1678.5, 1: 1669.1. Samples: 11511872. Policy #0 lag: (min: 26.0, avg: 37.0, max: 58.0) [2023-10-13 21:50:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-13 21:50:32,385][60934] Updated weights for policy 1, policy_version 22602 (0.0010) [2023-10-13 21:50:32,742][60934] Updated weights for policy 1, policy_version 22612 (0.0009) [2023-10-13 21:50:33,109][60934] Updated weights for policy 1, policy_version 22622 (0.0010) [2023-10-13 21:50:34,320][60935] Updated weights for policy 0, policy_version 22370 (0.0009) [2023-10-13 21:50:34,700][60935] Updated weights for policy 0, policy_version 22380 (0.0009) [2023-10-13 21:50:35,067][60935] Updated weights for policy 0, policy_version 22390 (0.0010) [2023-10-13 21:50:35,444][60935] Updated weights for policy 0, policy_version 22400 (0.0011) [2023-10-13 21:50:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 46104576. Throughput: 0: 1671.6, 1: 1693.3. Samples: 11532206. Policy #0 lag: (min: 26.0, avg: 37.0, max: 58.0) [2023-10-13 21:50:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-13 21:50:37,192][60934] Updated weights for policy 1, policy_version 22632 (0.0008) [2023-10-13 21:50:37,556][60934] Updated weights for policy 1, policy_version 22642 (0.0010) [2023-10-13 21:50:37,932][60934] Updated weights for policy 1, policy_version 22652 (0.0007) [2023-10-13 21:50:39,658][60935] Updated weights for policy 0, policy_version 22410 (0.0008) [2023-10-13 21:50:40,032][60935] Updated weights for policy 0, policy_version 22420 (0.0009) [2023-10-13 21:50:40,405][60935] Updated weights for policy 0, policy_version 22430 (0.0009) [2023-10-13 21:50:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 46170112. Throughput: 0: 1663.6, 1: 1697.1. Samples: 11552172. Policy #0 lag: (min: 26.0, avg: 37.0, max: 58.0) [2023-10-13 21:50:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:50:42,079][60934] Updated weights for policy 1, policy_version 22662 (0.0009) [2023-10-13 21:50:42,462][60934] Updated weights for policy 1, policy_version 22672 (0.0009) [2023-10-13 21:50:42,831][60934] Updated weights for policy 1, policy_version 22682 (0.0011) [2023-10-13 21:50:44,250][60935] Updated weights for policy 0, policy_version 22440 (0.0008) [2023-10-13 21:50:44,626][60935] Updated weights for policy 0, policy_version 22450 (0.0010) [2023-10-13 21:50:44,995][60935] Updated weights for policy 0, policy_version 22460 (0.0008) [2023-10-13 21:50:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 46235648. Throughput: 0: 1688.6, 1: 1675.8. Samples: 11562682. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-13 21:50:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:50:46,895][60934] Updated weights for policy 1, policy_version 22692 (0.0010) [2023-10-13 21:50:47,265][60934] Updated weights for policy 1, policy_version 22702 (0.0011) [2023-10-13 21:50:47,641][60934] Updated weights for policy 1, policy_version 22712 (0.0009) [2023-10-13 21:50:49,080][60935] Updated weights for policy 0, policy_version 22470 (0.0010) [2023-10-13 21:50:49,452][60935] Updated weights for policy 0, policy_version 22480 (0.0009) [2023-10-13 21:50:49,828][60935] Updated weights for policy 0, policy_version 22490 (0.0011) [2023-10-13 21:50:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 46301184. Throughput: 0: 1661.2, 1: 1689.0. Samples: 11582416. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-13 21:50:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:50:51,609][60934] Updated weights for policy 1, policy_version 22722 (0.0010) [2023-10-13 21:50:51,977][60934] Updated weights for policy 1, policy_version 22732 (0.0009) [2023-10-13 21:50:52,342][60934] Updated weights for policy 1, policy_version 22742 (0.0010) [2023-10-13 21:50:52,713][60934] Updated weights for policy 1, policy_version 22752 (0.0010) [2023-10-13 21:50:53,998][60935] Updated weights for policy 0, policy_version 22500 (0.0009) [2023-10-13 21:50:54,384][60935] Updated weights for policy 0, policy_version 22510 (0.0009) [2023-10-13 21:50:54,759][60935] Updated weights for policy 0, policy_version 22520 (0.0010) [2023-10-13 21:50:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 46366720. Throughput: 0: 1679.5, 1: 1684.2. Samples: 11602812. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-13 21:50:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:50:56,256][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000022528_23068672.pth... [2023-10-13 21:50:56,288][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000020960_21463040.pth [2023-10-13 21:50:56,691][60934] Updated weights for policy 1, policy_version 22762 (0.0007) [2023-10-13 21:50:57,061][60934] Updated weights for policy 1, policy_version 22772 (0.0007) [2023-10-13 21:50:57,435][60934] Updated weights for policy 1, policy_version 22782 (0.0007) [2023-10-13 21:50:57,501][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000022784_23330816.pth... [2023-10-13 21:50:57,542][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000021184_21692416.pth [2023-10-13 21:50:58,861][60935] Updated weights for policy 0, policy_version 22530 (0.0009) [2023-10-13 21:50:59,230][60935] Updated weights for policy 0, policy_version 22540 (0.0009) [2023-10-13 21:50:59,608][60935] Updated weights for policy 0, policy_version 22550 (0.0008) [2023-10-13 21:50:59,975][60935] Updated weights for policy 0, policy_version 22560 (0.0012) [2023-10-13 21:51:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 46432256. Throughput: 0: 1686.3, 1: 1682.4. Samples: 11613200. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-13 21:51:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:51:01,545][60934] Updated weights for policy 1, policy_version 22792 (0.0010) [2023-10-13 21:51:01,911][60934] Updated weights for policy 1, policy_version 22802 (0.0011) [2023-10-13 21:51:02,274][60934] Updated weights for policy 1, policy_version 22812 (0.0009) [2023-10-13 21:51:04,182][60935] Updated weights for policy 0, policy_version 22570 (0.0010) [2023-10-13 21:51:04,557][60935] Updated weights for policy 0, policy_version 22580 (0.0009) [2023-10-13 21:51:04,920][60935] Updated weights for policy 0, policy_version 22590 (0.0007) [2023-10-13 21:51:06,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 46497792. Throughput: 0: 1658.5, 1: 1686.9. Samples: 11632676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:51:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:51:06,382][60934] Updated weights for policy 1, policy_version 22822 (0.0008) [2023-10-13 21:51:06,739][60934] Updated weights for policy 1, policy_version 22832 (0.0011) [2023-10-13 21:51:07,106][60934] Updated weights for policy 1, policy_version 22842 (0.0010) [2023-10-13 21:51:08,879][60935] Updated weights for policy 0, policy_version 22600 (0.0008) [2023-10-13 21:51:09,249][60935] Updated weights for policy 0, policy_version 22610 (0.0007) [2023-10-13 21:51:09,623][60935] Updated weights for policy 0, policy_version 22620 (0.0007) [2023-10-13 21:51:11,169][60934] Updated weights for policy 1, policy_version 22852 (0.0008) [2023-10-13 21:51:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 46563328. Throughput: 0: 1679.9, 1: 1688.6. Samples: 11653352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:51:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:51:11,536][60934] Updated weights for policy 1, policy_version 22862 (0.0007) [2023-10-13 21:51:11,911][60934] Updated weights for policy 1, policy_version 22872 (0.0009) [2023-10-13 21:51:13,872][60935] Updated weights for policy 0, policy_version 22630 (0.0008) [2023-10-13 21:51:14,243][60935] Updated weights for policy 0, policy_version 22640 (0.0008) [2023-10-13 21:51:14,617][60935] Updated weights for policy 0, policy_version 22650 (0.0009) [2023-10-13 21:51:15,703][60934] Updated weights for policy 1, policy_version 22882 (0.0009) [2023-10-13 21:51:16,068][60934] Updated weights for policy 1, policy_version 22892 (0.0010) [2023-10-13 21:51:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 46628864. Throughput: 0: 1676.8, 1: 1696.8. Samples: 11663686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:51:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:51:16,438][60934] Updated weights for policy 1, policy_version 22902 (0.0008) [2023-10-13 21:51:16,806][60934] Updated weights for policy 1, policy_version 22912 (0.0009) [2023-10-13 21:51:18,831][60935] Updated weights for policy 0, policy_version 22660 (0.0010) [2023-10-13 21:51:19,203][60935] Updated weights for policy 0, policy_version 22670 (0.0011) [2023-10-13 21:51:19,570][60935] Updated weights for policy 0, policy_version 22680 (0.0011) [2023-10-13 21:51:20,725][60934] Updated weights for policy 1, policy_version 22922 (0.0011) [2023-10-13 21:51:21,101][60934] Updated weights for policy 1, policy_version 22932 (0.0009) [2023-10-13 21:51:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 46694400. Throughput: 0: 1655.8, 1: 1701.6. Samples: 11683292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:51:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:51:21,466][60934] Updated weights for policy 1, policy_version 22942 (0.0011) [2023-10-13 21:51:23,614][60935] Updated weights for policy 0, policy_version 22690 (0.0009) [2023-10-13 21:51:23,987][60935] Updated weights for policy 0, policy_version 22700 (0.0007) [2023-10-13 21:51:24,361][60935] Updated weights for policy 0, policy_version 22710 (0.0011) [2023-10-13 21:51:24,728][60935] Updated weights for policy 0, policy_version 22720 (0.0009) [2023-10-13 21:51:25,599][60934] Updated weights for policy 1, policy_version 22952 (0.0009) [2023-10-13 21:51:25,961][60934] Updated weights for policy 1, policy_version 22962 (0.0009) [2023-10-13 21:51:26,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 46759936. Throughput: 0: 1673.8, 1: 1693.7. Samples: 11703706. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-13 21:51:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:51:26,334][60934] Updated weights for policy 1, policy_version 22972 (0.0008) [2023-10-13 21:51:28,670][60935] Updated weights for policy 0, policy_version 22730 (0.0007) [2023-10-13 21:51:29,045][60935] Updated weights for policy 0, policy_version 22740 (0.0007) [2023-10-13 21:51:29,406][60935] Updated weights for policy 0, policy_version 22750 (0.0008) [2023-10-13 21:51:30,315][60934] Updated weights for policy 1, policy_version 22982 (0.0008) [2023-10-13 21:51:30,684][60934] Updated weights for policy 1, policy_version 22992 (0.0007) [2023-10-13 21:51:31,050][60934] Updated weights for policy 1, policy_version 23002 (0.0008) [2023-10-13 21:51:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 46825472. Throughput: 0: 1655.6, 1: 1706.9. Samples: 11713994. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-13 21:51:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:51:33,537][60935] Updated weights for policy 0, policy_version 22760 (0.0010) [2023-10-13 21:51:33,904][60935] Updated weights for policy 0, policy_version 22770 (0.0011) [2023-10-13 21:51:34,276][60935] Updated weights for policy 0, policy_version 22780 (0.0008) [2023-10-13 21:51:35,040][60934] Updated weights for policy 1, policy_version 23012 (0.0009) [2023-10-13 21:51:35,402][60934] Updated weights for policy 1, policy_version 23022 (0.0008) [2023-10-13 21:51:35,780][60934] Updated weights for policy 1, policy_version 23032 (0.0010) [2023-10-13 21:51:36,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 46923776. Throughput: 0: 1661.2, 1: 1711.1. Samples: 11734172. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-13 21:51:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:51:38,460][60935] Updated weights for policy 0, policy_version 22790 (0.0008) [2023-10-13 21:51:38,831][60935] Updated weights for policy 0, policy_version 22800 (0.0008) [2023-10-13 21:51:39,202][60935] Updated weights for policy 0, policy_version 22810 (0.0007) [2023-10-13 21:51:39,868][60934] Updated weights for policy 1, policy_version 23042 (0.0010) [2023-10-13 21:51:40,238][60934] Updated weights for policy 1, policy_version 23052 (0.0007) [2023-10-13 21:51:40,601][60934] Updated weights for policy 1, policy_version 23062 (0.0010) [2023-10-13 21:51:40,966][60934] Updated weights for policy 1, policy_version 23072 (0.0007) [2023-10-13 21:51:41,248][59943] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 46989312. Throughput: 0: 1667.4, 1: 1692.6. Samples: 11754012. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-13 21:51:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:51:43,211][60935] Updated weights for policy 0, policy_version 22820 (0.0008) [2023-10-13 21:51:43,587][60935] Updated weights for policy 0, policy_version 22830 (0.0009) [2023-10-13 21:51:43,957][60935] Updated weights for policy 0, policy_version 22840 (0.0007) [2023-10-13 21:51:44,800][60934] Updated weights for policy 1, policy_version 23082 (0.0010) [2023-10-13 21:51:45,176][60934] Updated weights for policy 1, policy_version 23092 (0.0008) [2023-10-13 21:51:45,546][60934] Updated weights for policy 1, policy_version 23102 (0.0008) [2023-10-13 21:51:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 47054848. Throughput: 0: 1652.8, 1: 1710.7. Samples: 11764556. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-10-13 21:51:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:51:48,084][60935] Updated weights for policy 0, policy_version 22850 (0.0008) [2023-10-13 21:51:48,435][60935] Updated weights for policy 0, policy_version 22860 (0.0010) [2023-10-13 21:51:48,816][60935] Updated weights for policy 0, policy_version 22870 (0.0009) [2023-10-13 21:51:49,185][60935] Updated weights for policy 0, policy_version 22880 (0.0008) [2023-10-13 21:51:49,589][60934] Updated weights for policy 1, policy_version 23112 (0.0009) [2023-10-13 21:51:49,955][60934] Updated weights for policy 1, policy_version 23122 (0.0008) [2023-10-13 21:51:50,327][60934] Updated weights for policy 1, policy_version 23132 (0.0008) [2023-10-13 21:51:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 47120384. Throughput: 0: 1667.8, 1: 1704.1. Samples: 11784410. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-10-13 21:51:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:51:53,294][60935] Updated weights for policy 0, policy_version 22890 (0.0011) [2023-10-13 21:51:53,654][60935] Updated weights for policy 0, policy_version 22900 (0.0009) [2023-10-13 21:51:54,032][60935] Updated weights for policy 0, policy_version 22910 (0.0011) [2023-10-13 21:51:54,489][60934] Updated weights for policy 1, policy_version 23142 (0.0007) [2023-10-13 21:51:54,854][60934] Updated weights for policy 1, policy_version 23152 (0.0007) [2023-10-13 21:51:55,222][60934] Updated weights for policy 1, policy_version 23162 (0.0007) [2023-10-13 21:51:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 47185920. Throughput: 0: 1671.0, 1: 1679.9. Samples: 11804142. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-10-13 21:51:56,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:51:58,120][60935] Updated weights for policy 0, policy_version 22920 (0.0009) [2023-10-13 21:51:58,484][60935] Updated weights for policy 0, policy_version 22930 (0.0008) [2023-10-13 21:51:58,854][60935] Updated weights for policy 0, policy_version 22940 (0.0008) [2023-10-13 21:51:59,205][60934] Updated weights for policy 1, policy_version 23172 (0.0009) [2023-10-13 21:51:59,570][60934] Updated weights for policy 1, policy_version 23182 (0.0009) [2023-10-13 21:51:59,940][60934] Updated weights for policy 1, policy_version 23192 (0.0009) [2023-10-13 21:52:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 47251456. Throughput: 0: 1652.3, 1: 1703.2. Samples: 11814682. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-10-13 21:52:01,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 21:52:02,783][60935] Updated weights for policy 0, policy_version 22950 (0.0008) [2023-10-13 21:52:03,155][60935] Updated weights for policy 0, policy_version 22960 (0.0011) [2023-10-13 21:52:03,520][60935] Updated weights for policy 0, policy_version 22970 (0.0011) [2023-10-13 21:52:04,046][60934] Updated weights for policy 1, policy_version 23202 (0.0008) [2023-10-13 21:52:04,411][60934] Updated weights for policy 1, policy_version 23212 (0.0008) [2023-10-13 21:52:04,778][60934] Updated weights for policy 1, policy_version 23222 (0.0010) [2023-10-13 21:52:05,150][60934] Updated weights for policy 1, policy_version 23232 (0.0009) [2023-10-13 21:52:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 47316992. Throughput: 0: 1677.4, 1: 1685.2. Samples: 11834610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:52:06,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 21:52:07,555][60935] Updated weights for policy 0, policy_version 22980 (0.0009) [2023-10-13 21:52:07,935][60935] Updated weights for policy 0, policy_version 22990 (0.0009) [2023-10-13 21:52:08,297][60935] Updated weights for policy 0, policy_version 23000 (0.0008) [2023-10-13 21:52:09,145][60934] Updated weights for policy 1, policy_version 23242 (0.0007) [2023-10-13 21:52:09,524][60934] Updated weights for policy 1, policy_version 23252 (0.0008) [2023-10-13 21:52:09,900][60934] Updated weights for policy 1, policy_version 23262 (0.0007) [2023-10-13 21:52:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 47382528. Throughput: 0: 1679.7, 1: 1676.8. Samples: 11854752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:52:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:52:12,559][60935] Updated weights for policy 0, policy_version 23010 (0.0007) [2023-10-13 21:52:12,921][60935] Updated weights for policy 0, policy_version 23020 (0.0010) [2023-10-13 21:52:13,291][60935] Updated weights for policy 0, policy_version 23030 (0.0008) [2023-10-13 21:52:13,653][60935] Updated weights for policy 0, policy_version 23040 (0.0007) [2023-10-13 21:52:13,822][60934] Updated weights for policy 1, policy_version 23272 (0.0007) [2023-10-13 21:52:14,199][60934] Updated weights for policy 1, policy_version 23282 (0.0007) [2023-10-13 21:52:14,560][60934] Updated weights for policy 1, policy_version 23292 (0.0009) [2023-10-13 21:52:16,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 47448064. Throughput: 0: 1660.2, 1: 1700.6. Samples: 11865228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:52:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:52:17,763][60935] Updated weights for policy 0, policy_version 23050 (0.0007) [2023-10-13 21:52:18,143][60935] Updated weights for policy 0, policy_version 23060 (0.0008) [2023-10-13 21:52:18,505][60935] Updated weights for policy 0, policy_version 23070 (0.0010) [2023-10-13 21:52:18,679][60934] Updated weights for policy 1, policy_version 23302 (0.0007) [2023-10-13 21:52:19,048][60934] Updated weights for policy 1, policy_version 23312 (0.0008) [2023-10-13 21:52:19,422][60934] Updated weights for policy 1, policy_version 23322 (0.0010) [2023-10-13 21:52:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 47513600. Throughput: 0: 1677.1, 1: 1668.8. Samples: 11884738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:52:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:52:22,595][60935] Updated weights for policy 0, policy_version 23080 (0.0008) [2023-10-13 21:52:22,973][60935] Updated weights for policy 0, policy_version 23090 (0.0008) [2023-10-13 21:52:23,349][60935] Updated weights for policy 0, policy_version 23100 (0.0009) [2023-10-13 21:52:23,598][60934] Updated weights for policy 1, policy_version 23332 (0.0009) [2023-10-13 21:52:23,963][60934] Updated weights for policy 1, policy_version 23342 (0.0009) [2023-10-13 21:52:24,342][60934] Updated weights for policy 1, policy_version 23352 (0.0007) [2023-10-13 21:52:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 47579136. Throughput: 0: 1677.3, 1: 1687.0. Samples: 11905406. Policy #0 lag: (min: 14.0, avg: 21.8, max: 46.0) [2023-10-13 21:52:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:52:27,481][60935] Updated weights for policy 0, policy_version 23110 (0.0009) [2023-10-13 21:52:27,863][60935] Updated weights for policy 0, policy_version 23120 (0.0009) [2023-10-13 21:52:28,228][60935] Updated weights for policy 0, policy_version 23130 (0.0009) [2023-10-13 21:52:28,230][60934] Updated weights for policy 1, policy_version 23362 (0.0011) [2023-10-13 21:52:28,598][60934] Updated weights for policy 1, policy_version 23372 (0.0008) [2023-10-13 21:52:28,971][60934] Updated weights for policy 1, policy_version 23382 (0.0007) [2023-10-13 21:52:29,340][60934] Updated weights for policy 1, policy_version 23392 (0.0009) [2023-10-13 21:52:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 47644672. Throughput: 0: 1661.6, 1: 1687.2. Samples: 11915252. Policy #0 lag: (min: 14.0, avg: 21.8, max: 46.0) [2023-10-13 21:52:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:52:32,421][60935] Updated weights for policy 0, policy_version 23140 (0.0009) [2023-10-13 21:52:32,786][60935] Updated weights for policy 0, policy_version 23150 (0.0008) [2023-10-13 21:52:33,161][60935] Updated weights for policy 0, policy_version 23160 (0.0007) [2023-10-13 21:52:33,374][60934] Updated weights for policy 1, policy_version 23402 (0.0008) [2023-10-13 21:52:33,746][60934] Updated weights for policy 1, policy_version 23412 (0.0009) [2023-10-13 21:52:34,116][60934] Updated weights for policy 1, policy_version 23422 (0.0009) [2023-10-13 21:52:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 47710208. Throughput: 0: 1670.8, 1: 1676.1. Samples: 11935020. Policy #0 lag: (min: 14.0, avg: 21.8, max: 46.0) [2023-10-13 21:52:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:52:37,015][60935] Updated weights for policy 0, policy_version 23170 (0.0008) [2023-10-13 21:52:37,377][60935] Updated weights for policy 0, policy_version 23180 (0.0009) [2023-10-13 21:52:37,752][60935] Updated weights for policy 0, policy_version 23190 (0.0008) [2023-10-13 21:52:38,118][60935] Updated weights for policy 0, policy_version 23200 (0.0008) [2023-10-13 21:52:38,242][60934] Updated weights for policy 1, policy_version 23432 (0.0010) [2023-10-13 21:52:38,616][60934] Updated weights for policy 1, policy_version 23442 (0.0011) [2023-10-13 21:52:38,984][60934] Updated weights for policy 1, policy_version 23452 (0.0011) [2023-10-13 21:52:41,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 47775744. Throughput: 0: 1677.6, 1: 1695.3. Samples: 11955924. Policy #0 lag: (min: 14.0, avg: 21.8, max: 46.0) [2023-10-13 21:52:41,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 21:52:42,234][60935] Updated weights for policy 0, policy_version 23210 (0.0008) [2023-10-13 21:52:42,606][60935] Updated weights for policy 0, policy_version 23220 (0.0007) [2023-10-13 21:52:42,972][60935] Updated weights for policy 0, policy_version 23230 (0.0009) [2023-10-13 21:52:43,060][60934] Updated weights for policy 1, policy_version 23462 (0.0010) [2023-10-13 21:52:43,435][60934] Updated weights for policy 1, policy_version 23472 (0.0010) [2023-10-13 21:52:43,806][60934] Updated weights for policy 1, policy_version 23482 (0.0010) [2023-10-13 21:52:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 47841280. Throughput: 0: 1672.9, 1: 1681.0. Samples: 11965606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:52:46,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 21:52:47,107][60935] Updated weights for policy 0, policy_version 23240 (0.0008) [2023-10-13 21:52:47,480][60935] Updated weights for policy 0, policy_version 23250 (0.0009) [2023-10-13 21:52:47,844][60935] Updated weights for policy 0, policy_version 23260 (0.0010) [2023-10-13 21:52:47,925][60934] Updated weights for policy 1, policy_version 23492 (0.0008) [2023-10-13 21:52:48,291][60934] Updated weights for policy 1, policy_version 23502 (0.0009) [2023-10-13 21:52:48,656][60934] Updated weights for policy 1, policy_version 23512 (0.0009) [2023-10-13 21:52:51,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 47906816. Throughput: 0: 1674.4, 1: 1684.5. Samples: 11985764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:52:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:52:51,922][60935] Updated weights for policy 0, policy_version 23270 (0.0010) [2023-10-13 21:52:52,293][60935] Updated weights for policy 0, policy_version 23280 (0.0008) [2023-10-13 21:52:52,539][60934] Updated weights for policy 1, policy_version 23522 (0.0009) [2023-10-13 21:52:52,663][60935] Updated weights for policy 0, policy_version 23290 (0.0007) [2023-10-13 21:52:52,908][60934] Updated weights for policy 1, policy_version 23532 (0.0008) [2023-10-13 21:52:53,269][60934] Updated weights for policy 1, policy_version 23542 (0.0010) [2023-10-13 21:52:53,642][60934] Updated weights for policy 1, policy_version 23552 (0.0009) [2023-10-13 21:52:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 47972352. Throughput: 0: 1681.0, 1: 1696.4. Samples: 12006734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:52:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:52:56,259][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000023296_23855104.pth... [2023-10-13 21:52:56,259][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000023552_24117248.pth... [2023-10-13 21:52:56,299][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000021984_22511616.pth [2023-10-13 21:52:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000021760_22282240.pth [2023-10-13 21:52:56,305][60828] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p1/milestones/checkpoint_000023552_24117248.pth [2023-10-13 21:52:56,306][60695] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p0/milestones/checkpoint_000023296_23855104.pth [2023-10-13 21:52:56,614][60935] Updated weights for policy 0, policy_version 23300 (0.0008) [2023-10-13 21:52:56,984][60935] Updated weights for policy 0, policy_version 23310 (0.0008) [2023-10-13 21:52:57,355][60935] Updated weights for policy 0, policy_version 23320 (0.0009) [2023-10-13 21:52:57,786][60934] Updated weights for policy 1, policy_version 23562 (0.0009) [2023-10-13 21:52:58,147][60934] Updated weights for policy 1, policy_version 23572 (0.0007) [2023-10-13 21:52:58,516][60934] Updated weights for policy 1, policy_version 23582 (0.0010) [2023-10-13 21:53:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 48037888. Throughput: 0: 1682.0, 1: 1667.7. Samples: 12015964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:53:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:53:01,303][60935] Updated weights for policy 0, policy_version 23330 (0.0008) [2023-10-13 21:53:01,678][60935] Updated weights for policy 0, policy_version 23340 (0.0009) [2023-10-13 21:53:02,052][60935] Updated weights for policy 0, policy_version 23350 (0.0011) [2023-10-13 21:53:02,411][60935] Updated weights for policy 0, policy_version 23360 (0.0009) [2023-10-13 21:53:02,474][60934] Updated weights for policy 1, policy_version 23592 (0.0008) [2023-10-13 21:53:02,847][60934] Updated weights for policy 1, policy_version 23602 (0.0007) [2023-10-13 21:53:03,227][60934] Updated weights for policy 1, policy_version 23612 (0.0009) [2023-10-13 21:53:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 48103424. Throughput: 0: 1678.1, 1: 1691.8. Samples: 12036386. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:53:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:53:06,516][60935] Updated weights for policy 0, policy_version 23370 (0.0007) [2023-10-13 21:53:06,881][60935] Updated weights for policy 0, policy_version 23380 (0.0008) [2023-10-13 21:53:07,253][60935] Updated weights for policy 0, policy_version 23390 (0.0008) [2023-10-13 21:53:07,398][60934] Updated weights for policy 1, policy_version 23622 (0.0007) [2023-10-13 21:53:07,776][60934] Updated weights for policy 1, policy_version 23632 (0.0007) [2023-10-13 21:53:08,142][60934] Updated weights for policy 1, policy_version 23642 (0.0008) [2023-10-13 21:53:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 48168960. Throughput: 0: 1680.3, 1: 1692.1. Samples: 12057162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:53:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:53:11,297][60935] Updated weights for policy 0, policy_version 23400 (0.0008) [2023-10-13 21:53:11,667][60935] Updated weights for policy 0, policy_version 23410 (0.0009) [2023-10-13 21:53:12,031][60935] Updated weights for policy 0, policy_version 23420 (0.0010) [2023-10-13 21:53:12,113][60934] Updated weights for policy 1, policy_version 23652 (0.0008) [2023-10-13 21:53:12,491][60934] Updated weights for policy 1, policy_version 23662 (0.0009) [2023-10-13 21:53:12,851][60934] Updated weights for policy 1, policy_version 23672 (0.0008) [2023-10-13 21:53:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 48234496. Throughput: 0: 1684.4, 1: 1674.6. Samples: 12066408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:53:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:53:16,352][60935] Updated weights for policy 0, policy_version 23430 (0.0010) [2023-10-13 21:53:16,624][60934] Updated weights for policy 1, policy_version 23682 (0.0007) [2023-10-13 21:53:16,739][60935] Updated weights for policy 0, policy_version 23440 (0.0009) [2023-10-13 21:53:16,991][60934] Updated weights for policy 1, policy_version 23692 (0.0009) [2023-10-13 21:53:17,107][60935] Updated weights for policy 0, policy_version 23450 (0.0008) [2023-10-13 21:53:17,362][60934] Updated weights for policy 1, policy_version 23702 (0.0009) [2023-10-13 21:53:17,727][60934] Updated weights for policy 1, policy_version 23712 (0.0008) [2023-10-13 21:53:21,022][60935] Updated weights for policy 0, policy_version 23460 (0.0008) [2023-10-13 21:53:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 48300032. Throughput: 0: 1683.5, 1: 1696.8. Samples: 12087136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:53:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:53:21,400][60935] Updated weights for policy 0, policy_version 23470 (0.0008) [2023-10-13 21:53:21,770][60935] Updated weights for policy 0, policy_version 23480 (0.0008) [2023-10-13 21:53:21,802][60934] Updated weights for policy 1, policy_version 23722 (0.0009) [2023-10-13 21:53:22,165][60934] Updated weights for policy 1, policy_version 23732 (0.0007) [2023-10-13 21:53:22,533][60934] Updated weights for policy 1, policy_version 23742 (0.0008) [2023-10-13 21:53:25,942][60935] Updated weights for policy 0, policy_version 23490 (0.0009) [2023-10-13 21:53:26,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 48365568. Throughput: 0: 1676.5, 1: 1702.8. Samples: 12107992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:53:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:53:26,311][60935] Updated weights for policy 0, policy_version 23500 (0.0007) [2023-10-13 21:53:26,630][60934] Updated weights for policy 1, policy_version 23752 (0.0008) [2023-10-13 21:53:26,684][60935] Updated weights for policy 0, policy_version 23510 (0.0007) [2023-10-13 21:53:26,995][60934] Updated weights for policy 1, policy_version 23762 (0.0007) [2023-10-13 21:53:27,042][60935] Updated weights for policy 0, policy_version 23520 (0.0010) [2023-10-13 21:53:27,353][60934] Updated weights for policy 1, policy_version 23772 (0.0007) [2023-10-13 21:53:31,158][60935] Updated weights for policy 0, policy_version 23530 (0.0007) [2023-10-13 21:53:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 48431104. Throughput: 0: 1676.7, 1: 1690.3. Samples: 12117118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:53:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:53:31,529][60935] Updated weights for policy 0, policy_version 23540 (0.0007) [2023-10-13 21:53:31,550][60934] Updated weights for policy 1, policy_version 23782 (0.0007) [2023-10-13 21:53:31,900][60935] Updated weights for policy 0, policy_version 23550 (0.0008) [2023-10-13 21:53:31,913][60934] Updated weights for policy 1, policy_version 23792 (0.0009) [2023-10-13 21:53:32,274][60934] Updated weights for policy 1, policy_version 23802 (0.0010) [2023-10-13 21:53:36,154][60934] Updated weights for policy 1, policy_version 23812 (0.0008) [2023-10-13 21:53:36,168][60935] Updated weights for policy 0, policy_version 23560 (0.0009) [2023-10-13 21:53:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 48496640. Throughput: 0: 1677.7, 1: 1702.5. Samples: 12137870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:53:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:53:36,525][60934] Updated weights for policy 1, policy_version 23822 (0.0007) [2023-10-13 21:53:36,543][60935] Updated weights for policy 0, policy_version 23570 (0.0010) [2023-10-13 21:53:36,893][60934] Updated weights for policy 1, policy_version 23832 (0.0009) [2023-10-13 21:53:36,911][60935] Updated weights for policy 0, policy_version 23580 (0.0010) [2023-10-13 21:53:40,971][60934] Updated weights for policy 1, policy_version 23842 (0.0009) [2023-10-13 21:53:41,062][60935] Updated weights for policy 0, policy_version 23590 (0.0007) [2023-10-13 21:53:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 48562176. Throughput: 0: 1672.8, 1: 1704.1. Samples: 12158694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:53:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:53:41,339][60934] Updated weights for policy 1, policy_version 23852 (0.0007) [2023-10-13 21:53:41,443][60935] Updated weights for policy 0, policy_version 23600 (0.0007) [2023-10-13 21:53:41,703][60934] Updated weights for policy 1, policy_version 23862 (0.0009) [2023-10-13 21:53:41,814][60935] Updated weights for policy 0, policy_version 23610 (0.0007) [2023-10-13 21:53:42,066][60934] Updated weights for policy 1, policy_version 23872 (0.0009) [2023-10-13 21:53:45,816][60935] Updated weights for policy 0, policy_version 23620 (0.0009) [2023-10-13 21:53:46,042][60934] Updated weights for policy 1, policy_version 23882 (0.0009) [2023-10-13 21:53:46,195][60935] Updated weights for policy 0, policy_version 23630 (0.0009) [2023-10-13 21:53:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48627712. Throughput: 0: 1675.4, 1: 1702.2. Samples: 12167956. Policy #0 lag: (min: 0.0, avg: 18.8, max: 32.0) [2023-10-13 21:53:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:53:46,403][60934] Updated weights for policy 1, policy_version 23892 (0.0007) [2023-10-13 21:53:46,566][60935] Updated weights for policy 0, policy_version 23640 (0.0014) [2023-10-13 21:53:46,778][60934] Updated weights for policy 1, policy_version 23902 (0.0010) [2023-10-13 21:53:50,709][60935] Updated weights for policy 0, policy_version 23650 (0.0008) [2023-10-13 21:53:50,933][60934] Updated weights for policy 1, policy_version 23912 (0.0008) [2023-10-13 21:53:51,088][60935] Updated weights for policy 0, policy_version 23660 (0.0008) [2023-10-13 21:53:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48693248. Throughput: 0: 1674.6, 1: 1700.0. Samples: 12188244. Policy #0 lag: (min: 0.0, avg: 18.8, max: 32.0) [2023-10-13 21:53:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:53:51,298][60934] Updated weights for policy 1, policy_version 23922 (0.0008) [2023-10-13 21:53:51,466][60935] Updated weights for policy 0, policy_version 23670 (0.0009) [2023-10-13 21:53:51,657][60934] Updated weights for policy 1, policy_version 23932 (0.0008) [2023-10-13 21:53:51,826][60935] Updated weights for policy 0, policy_version 23680 (0.0010) [2023-10-13 21:53:55,757][60934] Updated weights for policy 1, policy_version 23942 (0.0008) [2023-10-13 21:53:55,845][60935] Updated weights for policy 0, policy_version 23690 (0.0008) [2023-10-13 21:53:56,119][60934] Updated weights for policy 1, policy_version 23952 (0.0008) [2023-10-13 21:53:56,225][60935] Updated weights for policy 0, policy_version 23700 (0.0010) [2023-10-13 21:53:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48758784. Throughput: 0: 1667.0, 1: 1694.8. Samples: 12208440. Policy #0 lag: (min: 0.0, avg: 18.8, max: 32.0) [2023-10-13 21:53:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:53:56,494][60934] Updated weights for policy 1, policy_version 23962 (0.0007) [2023-10-13 21:53:56,601][60935] Updated weights for policy 0, policy_version 23710 (0.0010) [2023-10-13 21:54:00,679][60935] Updated weights for policy 0, policy_version 23720 (0.0008) [2023-10-13 21:54:00,743][60934] Updated weights for policy 1, policy_version 23972 (0.0008) [2023-10-13 21:54:01,051][60935] Updated weights for policy 0, policy_version 23730 (0.0008) [2023-10-13 21:54:01,110][60934] Updated weights for policy 1, policy_version 23982 (0.0007) [2023-10-13 21:54:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48824320. Throughput: 0: 1674.3, 1: 1691.5. Samples: 12217870. Policy #0 lag: (min: 0.0, avg: 18.8, max: 32.0) [2023-10-13 21:54:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:54:01,424][60935] Updated weights for policy 0, policy_version 23740 (0.0009) [2023-10-13 21:54:01,472][60934] Updated weights for policy 1, policy_version 23992 (0.0008) [2023-10-13 21:54:05,507][60935] Updated weights for policy 0, policy_version 23750 (0.0010) [2023-10-13 21:54:05,539][60934] Updated weights for policy 1, policy_version 24002 (0.0009) [2023-10-13 21:54:05,869][60935] Updated weights for policy 0, policy_version 23760 (0.0009) [2023-10-13 21:54:05,895][60934] Updated weights for policy 1, policy_version 24012 (0.0009) [2023-10-13 21:54:06,238][60935] Updated weights for policy 0, policy_version 23770 (0.0008) [2023-10-13 21:54:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48889856. Throughput: 0: 1675.3, 1: 1683.9. Samples: 12238300. Policy #0 lag: (min: 14.0, avg: 16.3, max: 46.0) [2023-10-13 21:54:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:54:06,270][60934] Updated weights for policy 1, policy_version 24022 (0.0007) [2023-10-13 21:54:06,636][60934] Updated weights for policy 1, policy_version 24032 (0.0010) [2023-10-13 21:54:10,411][60935] Updated weights for policy 0, policy_version 23780 (0.0009) [2023-10-13 21:54:10,778][60934] Updated weights for policy 1, policy_version 24042 (0.0008) [2023-10-13 21:54:10,779][60935] Updated weights for policy 0, policy_version 23790 (0.0009) [2023-10-13 21:54:11,142][60935] Updated weights for policy 0, policy_version 23800 (0.0008) [2023-10-13 21:54:11,156][60934] Updated weights for policy 1, policy_version 24052 (0.0009) [2023-10-13 21:54:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48955392. Throughput: 0: 1658.8, 1: 1679.0. Samples: 12258196. Policy #0 lag: (min: 14.0, avg: 16.3, max: 46.0) [2023-10-13 21:54:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:54:11,515][60934] Updated weights for policy 1, policy_version 24062 (0.0008) [2023-10-13 21:54:14,978][60935] Updated weights for policy 0, policy_version 23810 (0.0008) [2023-10-13 21:54:15,348][60935] Updated weights for policy 0, policy_version 23820 (0.0010) [2023-10-13 21:54:15,491][60934] Updated weights for policy 1, policy_version 24072 (0.0008) [2023-10-13 21:54:15,724][60935] Updated weights for policy 0, policy_version 23830 (0.0009) [2023-10-13 21:54:15,859][60934] Updated weights for policy 1, policy_version 24082 (0.0008) [2023-10-13 21:54:16,087][60935] Updated weights for policy 0, policy_version 23840 (0.0009) [2023-10-13 21:54:16,223][60934] Updated weights for policy 1, policy_version 24092 (0.0007) [2023-10-13 21:54:16,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 49053696. Throughput: 0: 1676.9, 1: 1679.9. Samples: 12268172. Policy #0 lag: (min: 14.0, avg: 16.3, max: 46.0) [2023-10-13 21:54:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:54:20,120][60935] Updated weights for policy 0, policy_version 23850 (0.0008) [2023-10-13 21:54:20,488][60935] Updated weights for policy 0, policy_version 23860 (0.0007) [2023-10-13 21:54:20,495][60934] Updated weights for policy 1, policy_version 24102 (0.0009) [2023-10-13 21:54:20,859][60934] Updated weights for policy 1, policy_version 24112 (0.0009) [2023-10-13 21:54:20,860][60935] Updated weights for policy 0, policy_version 23870 (0.0007) [2023-10-13 21:54:21,219][60934] Updated weights for policy 1, policy_version 24122 (0.0008) [2023-10-13 21:54:21,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 49119232. Throughput: 0: 1676.8, 1: 1678.3. Samples: 12288850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:54:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:54:24,872][60935] Updated weights for policy 0, policy_version 23880 (0.0010) [2023-10-13 21:54:25,170][60934] Updated weights for policy 1, policy_version 24132 (0.0009) [2023-10-13 21:54:25,245][60935] Updated weights for policy 0, policy_version 23890 (0.0009) [2023-10-13 21:54:25,537][60934] Updated weights for policy 1, policy_version 24142 (0.0007) [2023-10-13 21:54:25,602][60935] Updated weights for policy 0, policy_version 23900 (0.0010) [2023-10-13 21:54:25,906][60934] Updated weights for policy 1, policy_version 24152 (0.0008) [2023-10-13 21:54:26,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 49217536. Throughput: 0: 1650.2, 1: 1668.8. Samples: 12308048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:54:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:54:29,749][60935] Updated weights for policy 0, policy_version 23910 (0.0010) [2023-10-13 21:54:29,930][60934] Updated weights for policy 1, policy_version 24162 (0.0007) [2023-10-13 21:54:30,115][60935] Updated weights for policy 0, policy_version 23920 (0.0009) [2023-10-13 21:54:30,294][60934] Updated weights for policy 1, policy_version 24172 (0.0009) [2023-10-13 21:54:30,497][60935] Updated weights for policy 0, policy_version 23930 (0.0009) [2023-10-13 21:54:30,667][60934] Updated weights for policy 1, policy_version 24182 (0.0008) [2023-10-13 21:54:31,037][60934] Updated weights for policy 1, policy_version 24192 (0.0010) [2023-10-13 21:54:31,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 49283072. Throughput: 0: 1677.1, 1: 1679.8. Samples: 12319018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:54:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:54:34,625][60935] Updated weights for policy 0, policy_version 23940 (0.0010) [2023-10-13 21:54:34,999][60935] Updated weights for policy 0, policy_version 23950 (0.0009) [2023-10-13 21:54:35,053][60934] Updated weights for policy 1, policy_version 24202 (0.0009) [2023-10-13 21:54:35,365][60935] Updated weights for policy 0, policy_version 23960 (0.0009) [2023-10-13 21:54:35,422][60934] Updated weights for policy 1, policy_version 24212 (0.0007) [2023-10-13 21:54:35,787][60934] Updated weights for policy 1, policy_version 24222 (0.0008) [2023-10-13 21:54:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 49348608. Throughput: 0: 1673.8, 1: 1683.7. Samples: 12339332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:54:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:54:39,402][60935] Updated weights for policy 0, policy_version 23970 (0.0009) [2023-10-13 21:54:39,769][60935] Updated weights for policy 0, policy_version 23980 (0.0010) [2023-10-13 21:54:39,862][60934] Updated weights for policy 1, policy_version 24232 (0.0008) [2023-10-13 21:54:40,124][60935] Updated weights for policy 0, policy_version 23990 (0.0009) [2023-10-13 21:54:40,227][60934] Updated weights for policy 1, policy_version 24242 (0.0008) [2023-10-13 21:54:40,493][60935] Updated weights for policy 0, policy_version 24000 (0.0007) [2023-10-13 21:54:40,593][60934] Updated weights for policy 1, policy_version 24252 (0.0009) [2023-10-13 21:54:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 49414144. Throughput: 0: 1662.5, 1: 1667.2. Samples: 12358278. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 21:54:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:54:44,653][60934] Updated weights for policy 1, policy_version 24262 (0.0007) [2023-10-13 21:54:44,711][60935] Updated weights for policy 0, policy_version 24010 (0.0009) [2023-10-13 21:54:45,026][60934] Updated weights for policy 1, policy_version 24272 (0.0007) [2023-10-13 21:54:45,087][60935] Updated weights for policy 0, policy_version 24020 (0.0008) [2023-10-13 21:54:45,391][60934] Updated weights for policy 1, policy_version 24282 (0.0007) [2023-10-13 21:54:45,469][60935] Updated weights for policy 0, policy_version 24030 (0.0010) [2023-10-13 21:54:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 49479680. Throughput: 0: 1682.0, 1: 1694.5. Samples: 12369816. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 21:54:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:54:49,543][60934] Updated weights for policy 1, policy_version 24292 (0.0009) [2023-10-13 21:54:49,575][60935] Updated weights for policy 0, policy_version 24040 (0.0008) [2023-10-13 21:54:49,908][60934] Updated weights for policy 1, policy_version 24302 (0.0007) [2023-10-13 21:54:49,948][60935] Updated weights for policy 0, policy_version 24050 (0.0007) [2023-10-13 21:54:50,288][60934] Updated weights for policy 1, policy_version 24312 (0.0008) [2023-10-13 21:54:50,315][60935] Updated weights for policy 0, policy_version 24060 (0.0010) [2023-10-13 21:54:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 49545216. Throughput: 0: 1669.5, 1: 1689.9. Samples: 12389472. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 21:54:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:54:54,219][60934] Updated weights for policy 1, policy_version 24322 (0.0007) [2023-10-13 21:54:54,471][60935] Updated weights for policy 0, policy_version 24070 (0.0008) [2023-10-13 21:54:54,585][60934] Updated weights for policy 1, policy_version 24332 (0.0007) [2023-10-13 21:54:54,842][60935] Updated weights for policy 0, policy_version 24080 (0.0008) [2023-10-13 21:54:54,946][60934] Updated weights for policy 1, policy_version 24342 (0.0008) [2023-10-13 21:54:55,209][60935] Updated weights for policy 0, policy_version 24090 (0.0008) [2023-10-13 21:54:55,315][60934] Updated weights for policy 1, policy_version 24352 (0.0009) [2023-10-13 21:54:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 49610752. Throughput: 0: 1666.0, 1: 1665.0. Samples: 12408092. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 21:54:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:54:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000024352_24936448.pth... [2023-10-13 21:54:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000024096_24674304.pth... [2023-10-13 21:54:56,300][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000022784_23330816.pth [2023-10-13 21:54:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000022528_23068672.pth [2023-10-13 21:54:59,253][60935] Updated weights for policy 0, policy_version 24100 (0.0010) [2023-10-13 21:54:59,294][60934] Updated weights for policy 1, policy_version 24362 (0.0008) [2023-10-13 21:54:59,633][60935] Updated weights for policy 0, policy_version 24110 (0.0008) [2023-10-13 21:54:59,655][60934] Updated weights for policy 1, policy_version 24372 (0.0008) [2023-10-13 21:55:00,000][60935] Updated weights for policy 0, policy_version 24120 (0.0010) [2023-10-13 21:55:00,018][60934] Updated weights for policy 1, policy_version 24382 (0.0007) [2023-10-13 21:55:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 49676288. Throughput: 0: 1680.8, 1: 1688.7. Samples: 12419800. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 21:55:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:55:04,058][60934] Updated weights for policy 1, policy_version 24392 (0.0008) [2023-10-13 21:55:04,211][60935] Updated weights for policy 0, policy_version 24130 (0.0011) [2023-10-13 21:55:04,415][60934] Updated weights for policy 1, policy_version 24402 (0.0007) [2023-10-13 21:55:04,582][60935] Updated weights for policy 0, policy_version 24140 (0.0011) [2023-10-13 21:55:04,787][60934] Updated weights for policy 1, policy_version 24412 (0.0008) [2023-10-13 21:55:04,951][60935] Updated weights for policy 0, policy_version 24150 (0.0011) [2023-10-13 21:55:05,309][60935] Updated weights for policy 0, policy_version 24160 (0.0010) [2023-10-13 21:55:06,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 49741824. Throughput: 0: 1659.7, 1: 1672.9. Samples: 12438818. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 21:55:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:55:08,810][60934] Updated weights for policy 1, policy_version 24422 (0.0009) [2023-10-13 21:55:09,177][60934] Updated weights for policy 1, policy_version 24432 (0.0008) [2023-10-13 21:55:09,472][60935] Updated weights for policy 0, policy_version 24170 (0.0007) [2023-10-13 21:55:09,537][60934] Updated weights for policy 1, policy_version 24442 (0.0008) [2023-10-13 21:55:09,846][60935] Updated weights for policy 0, policy_version 24180 (0.0008) [2023-10-13 21:55:10,221][60935] Updated weights for policy 0, policy_version 24190 (0.0008) [2023-10-13 21:55:11,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 49807360. Throughput: 0: 1670.5, 1: 1674.0. Samples: 12458550. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 21:55:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:55:13,574][60934] Updated weights for policy 1, policy_version 24452 (0.0010) [2023-10-13 21:55:13,933][60934] Updated weights for policy 1, policy_version 24462 (0.0010) [2023-10-13 21:55:14,188][60935] Updated weights for policy 0, policy_version 24200 (0.0009) [2023-10-13 21:55:14,301][60934] Updated weights for policy 1, policy_version 24472 (0.0008) [2023-10-13 21:55:14,548][60935] Updated weights for policy 0, policy_version 24210 (0.0009) [2023-10-13 21:55:14,912][60935] Updated weights for policy 0, policy_version 24220 (0.0010) [2023-10-13 21:55:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 49872896. Throughput: 0: 1677.2, 1: 1689.2. Samples: 12470504. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 21:55:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:55:18,394][60934] Updated weights for policy 1, policy_version 24482 (0.0007) [2023-10-13 21:55:18,767][60934] Updated weights for policy 1, policy_version 24492 (0.0007) [2023-10-13 21:55:19,102][60935] Updated weights for policy 0, policy_version 24230 (0.0009) [2023-10-13 21:55:19,133][60934] Updated weights for policy 1, policy_version 24502 (0.0007) [2023-10-13 21:55:19,472][60935] Updated weights for policy 0, policy_version 24240 (0.0008) [2023-10-13 21:55:19,509][60934] Updated weights for policy 1, policy_version 24512 (0.0007) [2023-10-13 21:55:19,835][60935] Updated weights for policy 0, policy_version 24250 (0.0008) [2023-10-13 21:55:21,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 49938432. Throughput: 0: 1663.6, 1: 1665.2. Samples: 12489130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:55:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:55:23,466][60934] Updated weights for policy 1, policy_version 24522 (0.0007) [2023-10-13 21:55:23,847][60934] Updated weights for policy 1, policy_version 24532 (0.0009) [2023-10-13 21:55:23,857][60935] Updated weights for policy 0, policy_version 24260 (0.0008) [2023-10-13 21:55:24,208][60934] Updated weights for policy 1, policy_version 24542 (0.0008) [2023-10-13 21:55:24,232][60935] Updated weights for policy 0, policy_version 24270 (0.0007) [2023-10-13 21:55:24,596][60935] Updated weights for policy 0, policy_version 24280 (0.0010) [2023-10-13 21:55:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 50003968. Throughput: 0: 1676.2, 1: 1686.7. Samples: 12509608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:55:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:55:28,138][60934] Updated weights for policy 1, policy_version 24552 (0.0009) [2023-10-13 21:55:28,492][60935] Updated weights for policy 0, policy_version 24290 (0.0009) [2023-10-13 21:55:28,514][60934] Updated weights for policy 1, policy_version 24562 (0.0007) [2023-10-13 21:55:28,856][60935] Updated weights for policy 0, policy_version 24300 (0.0008) [2023-10-13 21:55:28,870][60934] Updated weights for policy 1, policy_version 24572 (0.0008) [2023-10-13 21:55:29,217][60935] Updated weights for policy 0, policy_version 24310 (0.0010) [2023-10-13 21:55:29,591][60935] Updated weights for policy 0, policy_version 24320 (0.0008) [2023-10-13 21:55:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 50069504. Throughput: 0: 1668.4, 1: 1673.0. Samples: 12520178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:55:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:55:32,905][60934] Updated weights for policy 1, policy_version 24582 (0.0008) [2023-10-13 21:55:33,270][60934] Updated weights for policy 1, policy_version 24592 (0.0009) [2023-10-13 21:55:33,627][60934] Updated weights for policy 1, policy_version 24602 (0.0007) [2023-10-13 21:55:33,788][60935] Updated weights for policy 0, policy_version 24330 (0.0008) [2023-10-13 21:55:34,160][60935] Updated weights for policy 0, policy_version 24340 (0.0009) [2023-10-13 21:55:34,523][60935] Updated weights for policy 0, policy_version 24350 (0.0009) [2023-10-13 21:55:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 50135040. Throughput: 0: 1664.6, 1: 1675.9. Samples: 12539796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:55:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:55:37,791][60934] Updated weights for policy 1, policy_version 24612 (0.0008) [2023-10-13 21:55:38,171][60934] Updated weights for policy 1, policy_version 24622 (0.0010) [2023-10-13 21:55:38,535][60934] Updated weights for policy 1, policy_version 24632 (0.0009) [2023-10-13 21:55:38,639][60935] Updated weights for policy 0, policy_version 24360 (0.0008) [2023-10-13 21:55:39,006][60935] Updated weights for policy 0, policy_version 24370 (0.0011) [2023-10-13 21:55:39,376][60935] Updated weights for policy 0, policy_version 24380 (0.0007) [2023-10-13 21:55:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 50200576. Throughput: 0: 1686.1, 1: 1705.7. Samples: 12560722. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-13 21:55:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:55:42,445][60934] Updated weights for policy 1, policy_version 24642 (0.0008) [2023-10-13 21:55:42,820][60934] Updated weights for policy 1, policy_version 24652 (0.0008) [2023-10-13 21:55:43,193][60934] Updated weights for policy 1, policy_version 24662 (0.0007) [2023-10-13 21:55:43,367][60935] Updated weights for policy 0, policy_version 24390 (0.0009) [2023-10-13 21:55:43,568][60934] Updated weights for policy 1, policy_version 24672 (0.0007) [2023-10-13 21:55:43,725][60935] Updated weights for policy 0, policy_version 24400 (0.0008) [2023-10-13 21:55:44,101][60935] Updated weights for policy 0, policy_version 24410 (0.0008) [2023-10-13 21:55:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 50266112. Throughput: 0: 1668.4, 1: 1681.4. Samples: 12570544. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-13 21:55:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:55:47,515][60934] Updated weights for policy 1, policy_version 24682 (0.0009) [2023-10-13 21:55:47,878][60934] Updated weights for policy 1, policy_version 24692 (0.0009) [2023-10-13 21:55:48,110][60935] Updated weights for policy 0, policy_version 24420 (0.0008) [2023-10-13 21:55:48,259][60934] Updated weights for policy 1, policy_version 24702 (0.0007) [2023-10-13 21:55:48,484][60935] Updated weights for policy 0, policy_version 24430 (0.0010) [2023-10-13 21:55:48,850][60935] Updated weights for policy 0, policy_version 24440 (0.0009) [2023-10-13 21:55:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 50331648. Throughput: 0: 1674.9, 1: 1702.7. Samples: 12590810. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-13 21:55:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:55:52,161][60934] Updated weights for policy 1, policy_version 24712 (0.0009) [2023-10-13 21:55:52,524][60934] Updated weights for policy 1, policy_version 24722 (0.0007) [2023-10-13 21:55:52,902][60934] Updated weights for policy 1, policy_version 24732 (0.0007) [2023-10-13 21:55:53,063][60935] Updated weights for policy 0, policy_version 24450 (0.0007) [2023-10-13 21:55:53,430][60935] Updated weights for policy 0, policy_version 24460 (0.0007) [2023-10-13 21:55:53,814][60935] Updated weights for policy 0, policy_version 24470 (0.0007) [2023-10-13 21:55:54,177][60935] Updated weights for policy 0, policy_version 24480 (0.0007) [2023-10-13 21:55:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 50397184. Throughput: 0: 1686.3, 1: 1719.6. Samples: 12611816. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-13 21:55:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:55:56,916][60934] Updated weights for policy 1, policy_version 24742 (0.0008) [2023-10-13 21:55:57,292][60934] Updated weights for policy 1, policy_version 24752 (0.0009) [2023-10-13 21:55:57,658][60934] Updated weights for policy 1, policy_version 24762 (0.0011) [2023-10-13 21:55:58,188][60935] Updated weights for policy 0, policy_version 24490 (0.0010) [2023-10-13 21:55:58,560][60935] Updated weights for policy 0, policy_version 24500 (0.0007) [2023-10-13 21:55:58,938][60935] Updated weights for policy 0, policy_version 24510 (0.0009) [2023-10-13 21:56:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 50462720. Throughput: 0: 1657.6, 1: 1690.2. Samples: 12621152. Policy #0 lag: (min: 20.0, avg: 21.5, max: 48.0) [2023-10-13 21:56:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:56:01,543][60934] Updated weights for policy 1, policy_version 24772 (0.0008) [2023-10-13 21:56:01,908][60934] Updated weights for policy 1, policy_version 24782 (0.0009) [2023-10-13 21:56:02,281][60934] Updated weights for policy 1, policy_version 24792 (0.0010) [2023-10-13 21:56:02,995][60935] Updated weights for policy 0, policy_version 24520 (0.0008) [2023-10-13 21:56:03,366][60935] Updated weights for policy 0, policy_version 24530 (0.0008) [2023-10-13 21:56:03,732][60935] Updated weights for policy 0, policy_version 24540 (0.0008) [2023-10-13 21:56:06,144][60934] Updated weights for policy 1, policy_version 24802 (0.0008) [2023-10-13 21:56:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 50528256. Throughput: 0: 1672.7, 1: 1725.3. Samples: 12642040. Policy #0 lag: (min: 20.0, avg: 21.5, max: 48.0) [2023-10-13 21:56:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:56:06,518][60934] Updated weights for policy 1, policy_version 24812 (0.0009) [2023-10-13 21:56:06,876][60934] Updated weights for policy 1, policy_version 24822 (0.0008) [2023-10-13 21:56:07,249][60934] Updated weights for policy 1, policy_version 24832 (0.0008) [2023-10-13 21:56:07,754][60935] Updated weights for policy 0, policy_version 24550 (0.0009) [2023-10-13 21:56:08,125][60935] Updated weights for policy 0, policy_version 24560 (0.0010) [2023-10-13 21:56:08,499][60935] Updated weights for policy 0, policy_version 24570 (0.0009) [2023-10-13 21:56:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 50593792. Throughput: 0: 1678.1, 1: 1727.7. Samples: 12662870. Policy #0 lag: (min: 20.0, avg: 21.5, max: 48.0) [2023-10-13 21:56:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:56:11,322][60934] Updated weights for policy 1, policy_version 24842 (0.0008) [2023-10-13 21:56:11,697][60934] Updated weights for policy 1, policy_version 24852 (0.0008) [2023-10-13 21:56:12,054][60934] Updated weights for policy 1, policy_version 24862 (0.0009) [2023-10-13 21:56:12,633][60935] Updated weights for policy 0, policy_version 24580 (0.0008) [2023-10-13 21:56:13,013][60935] Updated weights for policy 0, policy_version 24590 (0.0009) [2023-10-13 21:56:13,370][60935] Updated weights for policy 0, policy_version 24600 (0.0010) [2023-10-13 21:56:15,931][60934] Updated weights for policy 1, policy_version 24872 (0.0007) [2023-10-13 21:56:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 50659328. Throughput: 0: 1655.9, 1: 1719.0. Samples: 12672048. Policy #0 lag: (min: 20.0, avg: 21.5, max: 48.0) [2023-10-13 21:56:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:56:16,299][60934] Updated weights for policy 1, policy_version 24882 (0.0007) [2023-10-13 21:56:16,662][60934] Updated weights for policy 1, policy_version 24892 (0.0010) [2023-10-13 21:56:17,515][60935] Updated weights for policy 0, policy_version 24610 (0.0007) [2023-10-13 21:56:17,892][60935] Updated weights for policy 0, policy_version 24620 (0.0011) [2023-10-13 21:56:18,261][60935] Updated weights for policy 0, policy_version 24630 (0.0010) [2023-10-13 21:56:18,636][60935] Updated weights for policy 0, policy_version 24640 (0.0007) [2023-10-13 21:56:20,730][60934] Updated weights for policy 1, policy_version 24902 (0.0010) [2023-10-13 21:56:21,092][60934] Updated weights for policy 1, policy_version 24912 (0.0008) [2023-10-13 21:56:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 50724864. Throughput: 0: 1676.5, 1: 1728.2. Samples: 12693008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:56:21,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:56:21,463][60934] Updated weights for policy 1, policy_version 24922 (0.0007) [2023-10-13 21:56:22,653][60935] Updated weights for policy 0, policy_version 24650 (0.0009) [2023-10-13 21:56:23,029][60935] Updated weights for policy 0, policy_version 24660 (0.0008) [2023-10-13 21:56:23,410][60935] Updated weights for policy 0, policy_version 24670 (0.0011) [2023-10-13 21:56:25,615][60934] Updated weights for policy 1, policy_version 24932 (0.0008) [2023-10-13 21:56:26,014][60934] Updated weights for policy 1, policy_version 24942 (0.0008) [2023-10-13 21:56:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 50790400. Throughput: 0: 1673.2, 1: 1719.6. Samples: 12713400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:56:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:56:26,384][60934] Updated weights for policy 1, policy_version 24952 (0.0008) [2023-10-13 21:56:27,494][60935] Updated weights for policy 0, policy_version 24680 (0.0009) [2023-10-13 21:56:27,872][60935] Updated weights for policy 0, policy_version 24690 (0.0007) [2023-10-13 21:56:28,249][60935] Updated weights for policy 0, policy_version 24700 (0.0010) [2023-10-13 21:56:30,386][60934] Updated weights for policy 1, policy_version 24962 (0.0008) [2023-10-13 21:56:30,759][60934] Updated weights for policy 1, policy_version 24972 (0.0010) [2023-10-13 21:56:31,130][60934] Updated weights for policy 1, policy_version 24982 (0.0009) [2023-10-13 21:56:31,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 50855936. Throughput: 0: 1657.8, 1: 1717.2. Samples: 12722420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:56:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:56:31,498][60934] Updated weights for policy 1, policy_version 24992 (0.0009) [2023-10-13 21:56:32,242][60935] Updated weights for policy 0, policy_version 24710 (0.0007) [2023-10-13 21:56:32,614][60935] Updated weights for policy 0, policy_version 24720 (0.0007) [2023-10-13 21:56:32,987][60935] Updated weights for policy 0, policy_version 24730 (0.0007) [2023-10-13 21:56:35,567][60934] Updated weights for policy 1, policy_version 25002 (0.0010) [2023-10-13 21:56:35,938][60934] Updated weights for policy 1, policy_version 25012 (0.0007) [2023-10-13 21:56:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 50921472. Throughput: 0: 1673.1, 1: 1717.4. Samples: 12743384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:56:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 21:56:36,310][60934] Updated weights for policy 1, policy_version 25022 (0.0009) [2023-10-13 21:56:37,037][60935] Updated weights for policy 0, policy_version 24740 (0.0008) [2023-10-13 21:56:37,404][60935] Updated weights for policy 0, policy_version 24750 (0.0008) [2023-10-13 21:56:37,779][60935] Updated weights for policy 0, policy_version 24760 (0.0007) [2023-10-13 21:56:40,125][60934] Updated weights for policy 1, policy_version 25032 (0.0010) [2023-10-13 21:56:40,500][60934] Updated weights for policy 1, policy_version 25042 (0.0008) [2023-10-13 21:56:40,863][60934] Updated weights for policy 1, policy_version 25052 (0.0008) [2023-10-13 21:56:41,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 51019776. Throughput: 0: 1681.4, 1: 1693.5. Samples: 12763686. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-13 21:56:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.130')] [2023-10-13 21:56:41,709][60935] Updated weights for policy 0, policy_version 24770 (0.0008) [2023-10-13 21:56:42,085][60935] Updated weights for policy 0, policy_version 24780 (0.0008) [2023-10-13 21:56:42,450][60935] Updated weights for policy 0, policy_version 24790 (0.0008) [2023-10-13 21:56:42,822][60935] Updated weights for policy 0, policy_version 24800 (0.0007) [2023-10-13 21:56:44,947][60934] Updated weights for policy 1, policy_version 25062 (0.0008) [2023-10-13 21:56:45,306][60934] Updated weights for policy 1, policy_version 25072 (0.0008) [2023-10-13 21:56:45,685][60934] Updated weights for policy 1, policy_version 25082 (0.0009) [2023-10-13 21:56:46,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 51085312. Throughput: 0: 1677.6, 1: 1710.3. Samples: 12773612. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-13 21:56:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.130')] [2023-10-13 21:56:46,789][60935] Updated weights for policy 0, policy_version 24810 (0.0008) [2023-10-13 21:56:47,158][60935] Updated weights for policy 0, policy_version 24820 (0.0009) [2023-10-13 21:56:47,523][60935] Updated weights for policy 0, policy_version 24830 (0.0011) [2023-10-13 21:56:49,706][60934] Updated weights for policy 1, policy_version 25092 (0.0009) [2023-10-13 21:56:50,069][60934] Updated weights for policy 1, policy_version 25102 (0.0007) [2023-10-13 21:56:50,432][60934] Updated weights for policy 1, policy_version 25112 (0.0009) [2023-10-13 21:56:51,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 51150848. Throughput: 0: 1688.3, 1: 1701.9. Samples: 12794598. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-13 21:56:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.130')] [2023-10-13 21:56:51,783][60935] Updated weights for policy 0, policy_version 24840 (0.0008) [2023-10-13 21:56:52,150][60935] Updated weights for policy 0, policy_version 24850 (0.0008) [2023-10-13 21:56:52,530][60935] Updated weights for policy 0, policy_version 24860 (0.0009) [2023-10-13 21:56:54,532][60934] Updated weights for policy 1, policy_version 25122 (0.0008) [2023-10-13 21:56:54,898][60934] Updated weights for policy 1, policy_version 25132 (0.0007) [2023-10-13 21:56:55,265][60934] Updated weights for policy 1, policy_version 25142 (0.0007) [2023-10-13 21:56:55,631][60934] Updated weights for policy 1, policy_version 25152 (0.0008) [2023-10-13 21:56:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 51216384. Throughput: 0: 1690.0, 1: 1683.9. Samples: 12814696. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-13 21:56:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.130')] [2023-10-13 21:56:56,257][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000025152_25755648.pth... [2023-10-13 21:56:56,293][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000023552_24117248.pth [2023-10-13 21:56:56,532][60935] Updated weights for policy 0, policy_version 24870 (0.0009) [2023-10-13 21:56:56,903][60935] Updated weights for policy 0, policy_version 24880 (0.0009) [2023-10-13 21:56:57,281][60935] Updated weights for policy 0, policy_version 24890 (0.0010) [2023-10-13 21:56:57,494][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000024896_25493504.pth... [2023-10-13 21:56:57,531][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000023296_23855104.pth [2023-10-13 21:56:59,609][60934] Updated weights for policy 1, policy_version 25162 (0.0008) [2023-10-13 21:56:59,982][60934] Updated weights for policy 1, policy_version 25172 (0.0008) [2023-10-13 21:57:00,344][60934] Updated weights for policy 1, policy_version 25182 (0.0008) [2023-10-13 21:57:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 51281920. Throughput: 0: 1690.4, 1: 1706.0. Samples: 12824888. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 21:57:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:57:01,347][60935] Updated weights for policy 0, policy_version 24900 (0.0010) [2023-10-13 21:57:01,716][60935] Updated weights for policy 0, policy_version 24910 (0.0009) [2023-10-13 21:57:02,098][60935] Updated weights for policy 0, policy_version 24920 (0.0009) [2023-10-13 21:57:04,232][60934] Updated weights for policy 1, policy_version 25192 (0.0007) [2023-10-13 21:57:04,601][60934] Updated weights for policy 1, policy_version 25202 (0.0008) [2023-10-13 21:57:04,972][60934] Updated weights for policy 1, policy_version 25212 (0.0007) [2023-10-13 21:57:06,091][60935] Updated weights for policy 0, policy_version 24930 (0.0007) [2023-10-13 21:57:06,249][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.2, 300 sec: 13440.4). Total num frames: 51347456. Throughput: 0: 1694.8, 1: 1691.4. Samples: 12845390. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 21:57:06,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:57:06,464][60935] Updated weights for policy 0, policy_version 24940 (0.0008) [2023-10-13 21:57:06,847][60935] Updated weights for policy 0, policy_version 24950 (0.0009) [2023-10-13 21:57:07,216][60935] Updated weights for policy 0, policy_version 24960 (0.0008) [2023-10-13 21:57:08,895][60934] Updated weights for policy 1, policy_version 25222 (0.0007) [2023-10-13 21:57:09,273][60934] Updated weights for policy 1, policy_version 25232 (0.0008) [2023-10-13 21:57:09,649][60934] Updated weights for policy 1, policy_version 25242 (0.0009) [2023-10-13 21:57:11,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 51412992. Throughput: 0: 1701.3, 1: 1685.2. Samples: 12865794. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 21:57:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:57:11,335][60935] Updated weights for policy 0, policy_version 24970 (0.0010) [2023-10-13 21:57:11,698][60935] Updated weights for policy 0, policy_version 24980 (0.0009) [2023-10-13 21:57:12,077][60935] Updated weights for policy 0, policy_version 24990 (0.0011) [2023-10-13 21:57:13,859][60934] Updated weights for policy 1, policy_version 25252 (0.0008) [2023-10-13 21:57:14,259][60934] Updated weights for policy 1, policy_version 25262 (0.0007) [2023-10-13 21:57:14,630][60934] Updated weights for policy 1, policy_version 25272 (0.0007) [2023-10-13 21:57:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 51478528. Throughput: 0: 1702.7, 1: 1715.5. Samples: 12876236. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 21:57:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:57:16,257][60935] Updated weights for policy 0, policy_version 25000 (0.0009) [2023-10-13 21:57:16,641][60935] Updated weights for policy 0, policy_version 25010 (0.0009) [2023-10-13 21:57:17,010][60935] Updated weights for policy 0, policy_version 25020 (0.0008) [2023-10-13 21:57:18,455][60934] Updated weights for policy 1, policy_version 25282 (0.0007) [2023-10-13 21:57:18,823][60934] Updated weights for policy 1, policy_version 25292 (0.0008) [2023-10-13 21:57:19,194][60934] Updated weights for policy 1, policy_version 25302 (0.0007) [2023-10-13 21:57:19,564][60934] Updated weights for policy 1, policy_version 25312 (0.0007) [2023-10-13 21:57:21,064][60935] Updated weights for policy 0, policy_version 25030 (0.0011) [2023-10-13 21:57:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 51544064. Throughput: 0: 1699.3, 1: 1686.1. Samples: 12895728. Policy #0 lag: (min: 13.0, avg: 22.5, max: 45.0) [2023-10-13 21:57:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:57:21,430][60935] Updated weights for policy 0, policy_version 25040 (0.0009) [2023-10-13 21:57:21,803][60935] Updated weights for policy 0, policy_version 25050 (0.0008) [2023-10-13 21:57:23,622][60934] Updated weights for policy 1, policy_version 25322 (0.0010) [2023-10-13 21:57:23,990][60934] Updated weights for policy 1, policy_version 25332 (0.0008) [2023-10-13 21:57:24,362][60934] Updated weights for policy 1, policy_version 25342 (0.0007) [2023-10-13 21:57:25,690][60935] Updated weights for policy 0, policy_version 25060 (0.0007) [2023-10-13 21:57:26,068][60935] Updated weights for policy 0, policy_version 25070 (0.0007) [2023-10-13 21:57:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 51609600. Throughput: 0: 1683.6, 1: 1704.4. Samples: 12916146. Policy #0 lag: (min: 13.0, avg: 22.5, max: 45.0) [2023-10-13 21:57:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:57:26,436][60935] Updated weights for policy 0, policy_version 25080 (0.0009) [2023-10-13 21:57:28,324][60934] Updated weights for policy 1, policy_version 25352 (0.0008) [2023-10-13 21:57:28,702][60934] Updated weights for policy 1, policy_version 25362 (0.0008) [2023-10-13 21:57:29,069][60934] Updated weights for policy 1, policy_version 25372 (0.0008) [2023-10-13 21:57:30,522][60935] Updated weights for policy 0, policy_version 25090 (0.0010) [2023-10-13 21:57:30,885][60935] Updated weights for policy 0, policy_version 25100 (0.0009) [2023-10-13 21:57:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 51675136. Throughput: 0: 1686.5, 1: 1705.6. Samples: 12926256. Policy #0 lag: (min: 13.0, avg: 22.5, max: 45.0) [2023-10-13 21:57:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:57:31,264][60935] Updated weights for policy 0, policy_version 25110 (0.0009) [2023-10-13 21:57:31,639][60935] Updated weights for policy 0, policy_version 25120 (0.0008) [2023-10-13 21:57:33,025][60934] Updated weights for policy 1, policy_version 25382 (0.0010) [2023-10-13 21:57:33,392][60934] Updated weights for policy 1, policy_version 25392 (0.0010) [2023-10-13 21:57:33,761][60934] Updated weights for policy 1, policy_version 25402 (0.0010) [2023-10-13 21:57:35,800][60935] Updated weights for policy 0, policy_version 25130 (0.0008) [2023-10-13 21:57:36,167][60935] Updated weights for policy 0, policy_version 25140 (0.0009) [2023-10-13 21:57:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 51740672. Throughput: 0: 1686.0, 1: 1683.8. Samples: 12946242. Policy #0 lag: (min: 13.0, avg: 22.5, max: 45.0) [2023-10-13 21:57:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:57:36,538][60935] Updated weights for policy 0, policy_version 25150 (0.0010) [2023-10-13 21:57:37,718][60934] Updated weights for policy 1, policy_version 25412 (0.0010) [2023-10-13 21:57:38,089][60934] Updated weights for policy 1, policy_version 25422 (0.0008) [2023-10-13 21:57:38,457][60934] Updated weights for policy 1, policy_version 25432 (0.0008) [2023-10-13 21:57:40,532][60935] Updated weights for policy 0, policy_version 25160 (0.0010) [2023-10-13 21:57:40,912][60935] Updated weights for policy 0, policy_version 25170 (0.0007) [2023-10-13 21:57:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 51806208. Throughput: 0: 1670.2, 1: 1701.9. Samples: 12966442. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 21:57:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:57:41,289][60935] Updated weights for policy 0, policy_version 25180 (0.0007) [2023-10-13 21:57:42,507][60934] Updated weights for policy 1, policy_version 25442 (0.0008) [2023-10-13 21:57:42,874][60934] Updated weights for policy 1, policy_version 25452 (0.0010) [2023-10-13 21:57:43,241][60934] Updated weights for policy 1, policy_version 25462 (0.0009) [2023-10-13 21:57:43,612][60934] Updated weights for policy 1, policy_version 25472 (0.0009) [2023-10-13 21:57:45,266][60935] Updated weights for policy 0, policy_version 25190 (0.0008) [2023-10-13 21:57:45,634][60935] Updated weights for policy 0, policy_version 25200 (0.0008) [2023-10-13 21:57:46,014][60935] Updated weights for policy 0, policy_version 25210 (0.0008) [2023-10-13 21:57:46,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 51904512. Throughput: 0: 1686.7, 1: 1676.0. Samples: 12976210. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 21:57:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:57:47,885][60934] Updated weights for policy 1, policy_version 25482 (0.0007) [2023-10-13 21:57:48,249][60934] Updated weights for policy 1, policy_version 25492 (0.0009) [2023-10-13 21:57:48,616][60934] Updated weights for policy 1, policy_version 25502 (0.0009) [2023-10-13 21:57:50,195][60935] Updated weights for policy 0, policy_version 25220 (0.0009) [2023-10-13 21:57:50,563][60935] Updated weights for policy 0, policy_version 25230 (0.0011) [2023-10-13 21:57:50,934][60935] Updated weights for policy 0, policy_version 25240 (0.0011) [2023-10-13 21:57:51,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 51970048. Throughput: 0: 1678.1, 1: 1678.1. Samples: 12996418. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 21:57:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:57:52,617][60934] Updated weights for policy 1, policy_version 25512 (0.0010) [2023-10-13 21:57:52,982][60934] Updated weights for policy 1, policy_version 25522 (0.0010) [2023-10-13 21:57:53,348][60934] Updated weights for policy 1, policy_version 25532 (0.0009) [2023-10-13 21:57:54,919][60935] Updated weights for policy 0, policy_version 25250 (0.0010) [2023-10-13 21:57:55,294][60935] Updated weights for policy 0, policy_version 25260 (0.0008) [2023-10-13 21:57:55,651][60935] Updated weights for policy 0, policy_version 25270 (0.0010) [2023-10-13 21:57:56,024][60935] Updated weights for policy 0, policy_version 25280 (0.0008) [2023-10-13 21:57:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 52035584. Throughput: 0: 1652.6, 1: 1700.0. Samples: 13016662. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 21:57:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:57:57,342][60934] Updated weights for policy 1, policy_version 25542 (0.0009) [2023-10-13 21:57:57,707][60934] Updated weights for policy 1, policy_version 25552 (0.0009) [2023-10-13 21:57:58,082][60934] Updated weights for policy 1, policy_version 25562 (0.0007) [2023-10-13 21:58:00,109][60935] Updated weights for policy 0, policy_version 25290 (0.0008) [2023-10-13 21:58:00,478][60935] Updated weights for policy 0, policy_version 25300 (0.0009) [2023-10-13 21:58:00,850][60935] Updated weights for policy 0, policy_version 25310 (0.0011) [2023-10-13 21:58:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 52101120. Throughput: 0: 1677.9, 1: 1668.8. Samples: 13026836. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 21:58:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:58:02,250][60934] Updated weights for policy 1, policy_version 25572 (0.0009) [2023-10-13 21:58:02,627][60934] Updated weights for policy 1, policy_version 25582 (0.0010) [2023-10-13 21:58:02,999][60934] Updated weights for policy 1, policy_version 25592 (0.0010) [2023-10-13 21:58:04,975][60935] Updated weights for policy 0, policy_version 25320 (0.0010) [2023-10-13 21:58:05,344][60935] Updated weights for policy 0, policy_version 25330 (0.0008) [2023-10-13 21:58:05,705][60935] Updated weights for policy 0, policy_version 25340 (0.0007) [2023-10-13 21:58:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 52166656. Throughput: 0: 1674.7, 1: 1693.6. Samples: 13047300. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 21:58:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:58:07,126][60934] Updated weights for policy 1, policy_version 25602 (0.0008) [2023-10-13 21:58:07,545][60934] Updated weights for policy 1, policy_version 25612 (0.0008) [2023-10-13 21:58:07,909][60934] Updated weights for policy 1, policy_version 25622 (0.0008) [2023-10-13 21:58:08,278][60934] Updated weights for policy 1, policy_version 25632 (0.0009) [2023-10-13 21:58:09,742][60935] Updated weights for policy 0, policy_version 25350 (0.0008) [2023-10-13 21:58:10,111][60935] Updated weights for policy 0, policy_version 25360 (0.0009) [2023-10-13 21:58:10,479][60935] Updated weights for policy 0, policy_version 25370 (0.0008) [2023-10-13 21:58:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 52232192. Throughput: 0: 1661.2, 1: 1688.7. Samples: 13066888. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 21:58:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:58:12,211][60934] Updated weights for policy 1, policy_version 25642 (0.0010) [2023-10-13 21:58:12,563][60934] Updated weights for policy 1, policy_version 25652 (0.0011) [2023-10-13 21:58:12,929][60934] Updated weights for policy 1, policy_version 25662 (0.0010) [2023-10-13 21:58:14,686][60935] Updated weights for policy 0, policy_version 25380 (0.0009) [2023-10-13 21:58:15,064][60935] Updated weights for policy 0, policy_version 25390 (0.0009) [2023-10-13 21:58:15,427][60935] Updated weights for policy 0, policy_version 25400 (0.0009) [2023-10-13 21:58:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 52297728. Throughput: 0: 1680.7, 1: 1676.7. Samples: 13077336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:58:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:58:16,748][60934] Updated weights for policy 1, policy_version 25672 (0.0008) [2023-10-13 21:58:17,119][60934] Updated weights for policy 1, policy_version 25682 (0.0009) [2023-10-13 21:58:17,480][60934] Updated weights for policy 1, policy_version 25692 (0.0010) [2023-10-13 21:58:19,559][60935] Updated weights for policy 0, policy_version 25410 (0.0008) [2023-10-13 21:58:19,937][60935] Updated weights for policy 0, policy_version 25420 (0.0010) [2023-10-13 21:58:20,300][60935] Updated weights for policy 0, policy_version 25430 (0.0009) [2023-10-13 21:58:20,669][60935] Updated weights for policy 0, policy_version 25440 (0.0008) [2023-10-13 21:58:21,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 52363264. Throughput: 0: 1668.4, 1: 1698.8. Samples: 13097764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:58:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:58:21,459][60934] Updated weights for policy 1, policy_version 25702 (0.0009) [2023-10-13 21:58:21,827][60934] Updated weights for policy 1, policy_version 25712 (0.0008) [2023-10-13 21:58:22,203][60934] Updated weights for policy 1, policy_version 25722 (0.0007) [2023-10-13 21:58:24,525][60935] Updated weights for policy 0, policy_version 25450 (0.0011) [2023-10-13 21:58:24,901][60935] Updated weights for policy 0, policy_version 25460 (0.0008) [2023-10-13 21:58:25,264][60935] Updated weights for policy 0, policy_version 25470 (0.0008) [2023-10-13 21:58:26,190][60934] Updated weights for policy 1, policy_version 25732 (0.0009) [2023-10-13 21:58:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 52428800. Throughput: 0: 1665.6, 1: 1702.2. Samples: 13117990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:58:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:58:26,559][60934] Updated weights for policy 1, policy_version 25742 (0.0009) [2023-10-13 21:58:26,923][60934] Updated weights for policy 1, policy_version 25752 (0.0009) [2023-10-13 21:58:29,310][60935] Updated weights for policy 0, policy_version 25480 (0.0009) [2023-10-13 21:58:29,689][60935] Updated weights for policy 0, policy_version 25490 (0.0009) [2023-10-13 21:58:30,057][60935] Updated weights for policy 0, policy_version 25500 (0.0007) [2023-10-13 21:58:31,092][60934] Updated weights for policy 1, policy_version 25762 (0.0010) [2023-10-13 21:58:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 52494336. Throughput: 0: 1679.3, 1: 1705.8. Samples: 13128540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:58:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:58:31,457][60934] Updated weights for policy 1, policy_version 25772 (0.0008) [2023-10-13 21:58:31,818][60934] Updated weights for policy 1, policy_version 25782 (0.0008) [2023-10-13 21:58:32,184][60934] Updated weights for policy 1, policy_version 25792 (0.0010) [2023-10-13 21:58:34,221][60935] Updated weights for policy 0, policy_version 25510 (0.0010) [2023-10-13 21:58:34,597][60935] Updated weights for policy 0, policy_version 25520 (0.0011) [2023-10-13 21:58:34,977][60935] Updated weights for policy 0, policy_version 25530 (0.0010) [2023-10-13 21:58:36,105][60934] Updated weights for policy 1, policy_version 25802 (0.0011) [2023-10-13 21:58:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 52559872. Throughput: 0: 1664.6, 1: 1717.4. Samples: 13148608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:58:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:58:36,461][60934] Updated weights for policy 1, policy_version 25812 (0.0011) [2023-10-13 21:58:36,825][60934] Updated weights for policy 1, policy_version 25822 (0.0011) [2023-10-13 21:58:38,836][60935] Updated weights for policy 0, policy_version 25540 (0.0009) [2023-10-13 21:58:39,210][60935] Updated weights for policy 0, policy_version 25550 (0.0007) [2023-10-13 21:58:39,579][60935] Updated weights for policy 0, policy_version 25560 (0.0007) [2023-10-13 21:58:40,964][60934] Updated weights for policy 1, policy_version 25832 (0.0009) [2023-10-13 21:58:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 52625408. Throughput: 0: 1682.8, 1: 1706.2. Samples: 13169164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:58:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:58:41,330][60934] Updated weights for policy 1, policy_version 25842 (0.0009) [2023-10-13 21:58:41,691][60934] Updated weights for policy 1, policy_version 25852 (0.0009) [2023-10-13 21:58:43,658][60935] Updated weights for policy 0, policy_version 25570 (0.0010) [2023-10-13 21:58:44,031][60935] Updated weights for policy 0, policy_version 25580 (0.0007) [2023-10-13 21:58:44,387][60935] Updated weights for policy 0, policy_version 25590 (0.0007) [2023-10-13 21:58:44,752][60935] Updated weights for policy 0, policy_version 25600 (0.0011) [2023-10-13 21:58:45,742][60934] Updated weights for policy 1, policy_version 25862 (0.0007) [2023-10-13 21:58:46,105][60934] Updated weights for policy 1, policy_version 25872 (0.0010) [2023-10-13 21:58:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 52690944. Throughput: 0: 1681.8, 1: 1708.0. Samples: 13179376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:58:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:58:46,480][60934] Updated weights for policy 1, policy_version 25882 (0.0010) [2023-10-13 21:58:48,848][60935] Updated weights for policy 0, policy_version 25610 (0.0008) [2023-10-13 21:58:49,224][60935] Updated weights for policy 0, policy_version 25620 (0.0011) [2023-10-13 21:58:49,599][60935] Updated weights for policy 0, policy_version 25630 (0.0007) [2023-10-13 21:58:50,644][60934] Updated weights for policy 1, policy_version 25892 (0.0007) [2023-10-13 21:58:51,007][60934] Updated weights for policy 1, policy_version 25902 (0.0008) [2023-10-13 21:58:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 52756480. Throughput: 0: 1664.1, 1: 1713.3. Samples: 13199286. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:58:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:58:51,383][60934] Updated weights for policy 1, policy_version 25912 (0.0008) [2023-10-13 21:58:53,618][60935] Updated weights for policy 0, policy_version 25640 (0.0010) [2023-10-13 21:58:53,983][60935] Updated weights for policy 0, policy_version 25650 (0.0010) [2023-10-13 21:58:54,359][60935] Updated weights for policy 0, policy_version 25660 (0.0008) [2023-10-13 21:58:55,233][60934] Updated weights for policy 1, policy_version 25922 (0.0008) [2023-10-13 21:58:55,633][60934] Updated weights for policy 1, policy_version 25932 (0.0008) [2023-10-13 21:58:55,997][60934] Updated weights for policy 1, policy_version 25942 (0.0008) [2023-10-13 21:58:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 52822016. Throughput: 0: 1687.2, 1: 1709.9. Samples: 13219758. Policy #0 lag: (min: 9.0, avg: 16.7, max: 41.0) [2023-10-13 21:58:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:58:56,258][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000025664_26279936.pth... [2023-10-13 21:58:56,298][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000024096_24674304.pth [2023-10-13 21:58:56,362][60934] Updated weights for policy 1, policy_version 25952 (0.0009) [2023-10-13 21:58:56,364][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000025952_26574848.pth... [2023-10-13 21:58:56,400][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000024352_24936448.pth [2023-10-13 21:58:58,461][60935] Updated weights for policy 0, policy_version 25670 (0.0009) [2023-10-13 21:58:58,834][60935] Updated weights for policy 0, policy_version 25680 (0.0011) [2023-10-13 21:58:59,205][60935] Updated weights for policy 0, policy_version 25690 (0.0010) [2023-10-13 21:59:00,196][60934] Updated weights for policy 1, policy_version 25962 (0.0007) [2023-10-13 21:59:00,571][60934] Updated weights for policy 1, policy_version 25972 (0.0007) [2023-10-13 21:59:00,939][60934] Updated weights for policy 1, policy_version 25982 (0.0007) [2023-10-13 21:59:01,248][59943] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 52920320. Throughput: 0: 1675.9, 1: 1714.6. Samples: 13229908. Policy #0 lag: (min: 9.0, avg: 16.7, max: 41.0) [2023-10-13 21:59:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:59:03,280][60935] Updated weights for policy 0, policy_version 25700 (0.0009) [2023-10-13 21:59:03,647][60935] Updated weights for policy 0, policy_version 25710 (0.0011) [2023-10-13 21:59:04,011][60935] Updated weights for policy 0, policy_version 25720 (0.0009) [2023-10-13 21:59:05,041][60934] Updated weights for policy 1, policy_version 25992 (0.0008) [2023-10-13 21:59:05,404][60934] Updated weights for policy 1, policy_version 26002 (0.0009) [2023-10-13 21:59:05,768][60934] Updated weights for policy 1, policy_version 26012 (0.0009) [2023-10-13 21:59:06,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 52985856. Throughput: 0: 1668.6, 1: 1716.3. Samples: 13250084. Policy #0 lag: (min: 9.0, avg: 16.7, max: 41.0) [2023-10-13 21:59:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:59:07,981][60935] Updated weights for policy 0, policy_version 25730 (0.0010) [2023-10-13 21:59:08,357][60935] Updated weights for policy 0, policy_version 25740 (0.0010) [2023-10-13 21:59:08,725][60935] Updated weights for policy 0, policy_version 25750 (0.0011) [2023-10-13 21:59:09,100][60935] Updated weights for policy 0, policy_version 25760 (0.0007) [2023-10-13 21:59:09,529][60934] Updated weights for policy 1, policy_version 26022 (0.0008) [2023-10-13 21:59:09,898][60934] Updated weights for policy 1, policy_version 26032 (0.0008) [2023-10-13 21:59:10,267][60934] Updated weights for policy 1, policy_version 26042 (0.0010) [2023-10-13 21:59:11,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 53051392. Throughput: 0: 1691.7, 1: 1690.6. Samples: 13270192. Policy #0 lag: (min: 9.0, avg: 16.7, max: 41.0) [2023-10-13 21:59:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:59:13,236][60935] Updated weights for policy 0, policy_version 25770 (0.0008) [2023-10-13 21:59:13,613][60935] Updated weights for policy 0, policy_version 25780 (0.0010) [2023-10-13 21:59:13,984][60935] Updated weights for policy 0, policy_version 25790 (0.0009) [2023-10-13 21:59:14,454][60934] Updated weights for policy 1, policy_version 26052 (0.0009) [2023-10-13 21:59:14,832][60934] Updated weights for policy 1, policy_version 26062 (0.0008) [2023-10-13 21:59:15,198][60934] Updated weights for policy 1, policy_version 26072 (0.0009) [2023-10-13 21:59:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 53116928. Throughput: 0: 1669.8, 1: 1716.2. Samples: 13280910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:59:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:59:18,066][60935] Updated weights for policy 0, policy_version 25800 (0.0008) [2023-10-13 21:59:18,429][60935] Updated weights for policy 0, policy_version 25810 (0.0008) [2023-10-13 21:59:18,799][60935] Updated weights for policy 0, policy_version 25820 (0.0008) [2023-10-13 21:59:19,173][60934] Updated weights for policy 1, policy_version 26082 (0.0009) [2023-10-13 21:59:19,549][60934] Updated weights for policy 1, policy_version 26092 (0.0008) [2023-10-13 21:59:19,913][60934] Updated weights for policy 1, policy_version 26102 (0.0008) [2023-10-13 21:59:20,280][60934] Updated weights for policy 1, policy_version 26112 (0.0009) [2023-10-13 21:59:21,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 53182464. Throughput: 0: 1680.4, 1: 1704.2. Samples: 13300914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:59:21,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:59:22,969][60935] Updated weights for policy 0, policy_version 25830 (0.0010) [2023-10-13 21:59:23,342][60935] Updated weights for policy 0, policy_version 25840 (0.0010) [2023-10-13 21:59:23,707][60935] Updated weights for policy 0, policy_version 25850 (0.0008) [2023-10-13 21:59:24,243][60934] Updated weights for policy 1, policy_version 26122 (0.0007) [2023-10-13 21:59:24,615][60934] Updated weights for policy 1, policy_version 26132 (0.0008) [2023-10-13 21:59:24,974][60934] Updated weights for policy 1, policy_version 26142 (0.0007) [2023-10-13 21:59:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 53248000. Throughput: 0: 1679.6, 1: 1693.6. Samples: 13320960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:59:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:59:27,817][60935] Updated weights for policy 0, policy_version 25860 (0.0008) [2023-10-13 21:59:28,201][60935] Updated weights for policy 0, policy_version 25870 (0.0009) [2023-10-13 21:59:28,571][60935] Updated weights for policy 0, policy_version 25880 (0.0009) [2023-10-13 21:59:29,035][60934] Updated weights for policy 1, policy_version 26152 (0.0008) [2023-10-13 21:59:29,403][60934] Updated weights for policy 1, policy_version 26162 (0.0007) [2023-10-13 21:59:29,770][60934] Updated weights for policy 1, policy_version 26172 (0.0009) [2023-10-13 21:59:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 53313536. Throughput: 0: 1658.5, 1: 1722.3. Samples: 13331514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 21:59:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 21:59:32,596][60935] Updated weights for policy 0, policy_version 25890 (0.0009) [2023-10-13 21:59:32,971][60935] Updated weights for policy 0, policy_version 25900 (0.0009) [2023-10-13 21:59:33,346][60935] Updated weights for policy 0, policy_version 25910 (0.0009) [2023-10-13 21:59:33,636][60934] Updated weights for policy 1, policy_version 26182 (0.0010) [2023-10-13 21:59:33,709][60935] Updated weights for policy 0, policy_version 25920 (0.0009) [2023-10-13 21:59:33,995][60934] Updated weights for policy 1, policy_version 26192 (0.0009) [2023-10-13 21:59:34,372][60934] Updated weights for policy 1, policy_version 26202 (0.0007) [2023-10-13 21:59:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 53379072. Throughput: 0: 1681.0, 1: 1695.6. Samples: 13351232. Policy #0 lag: (min: 9.0, avg: 21.5, max: 41.0) [2023-10-13 21:59:36,249][59943] Avg episode reward: [(0, '-0.140'), (1, '0.000')] [2023-10-13 21:59:37,783][60935] Updated weights for policy 0, policy_version 25930 (0.0007) [2023-10-13 21:59:38,160][60935] Updated weights for policy 0, policy_version 25940 (0.0007) [2023-10-13 21:59:38,395][60934] Updated weights for policy 1, policy_version 26212 (0.0008) [2023-10-13 21:59:38,529][60935] Updated weights for policy 0, policy_version 25950 (0.0008) [2023-10-13 21:59:38,760][60934] Updated weights for policy 1, policy_version 26222 (0.0007) [2023-10-13 21:59:39,133][60934] Updated weights for policy 1, policy_version 26232 (0.0008) [2023-10-13 21:59:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 53444608. Throughput: 0: 1677.5, 1: 1705.3. Samples: 13371984. Policy #0 lag: (min: 9.0, avg: 21.5, max: 41.0) [2023-10-13 21:59:41,249][59943] Avg episode reward: [(0, '-0.280'), (1, '0.000')] [2023-10-13 21:59:42,729][60935] Updated weights for policy 0, policy_version 25960 (0.0010) [2023-10-13 21:59:43,053][60934] Updated weights for policy 1, policy_version 26242 (0.0008) [2023-10-13 21:59:43,108][60935] Updated weights for policy 0, policy_version 25970 (0.0007) [2023-10-13 21:59:43,460][60934] Updated weights for policy 1, policy_version 26252 (0.0008) [2023-10-13 21:59:43,479][60935] Updated weights for policy 0, policy_version 25980 (0.0007) [2023-10-13 21:59:43,833][60934] Updated weights for policy 1, policy_version 26262 (0.0007) [2023-10-13 21:59:44,197][60934] Updated weights for policy 1, policy_version 26272 (0.0009) [2023-10-13 21:59:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 53510144. Throughput: 0: 1663.1, 1: 1713.3. Samples: 13381848. Policy #0 lag: (min: 9.0, avg: 21.5, max: 41.0) [2023-10-13 21:59:46,249][59943] Avg episode reward: [(0, '-0.140'), (1, '-0.010')] [2023-10-13 21:59:47,484][60935] Updated weights for policy 0, policy_version 25990 (0.0010) [2023-10-13 21:59:47,865][60935] Updated weights for policy 0, policy_version 26000 (0.0008) [2023-10-13 21:59:48,006][60934] Updated weights for policy 1, policy_version 26282 (0.0009) [2023-10-13 21:59:48,228][60935] Updated weights for policy 0, policy_version 26010 (0.0010) [2023-10-13 21:59:48,378][60934] Updated weights for policy 1, policy_version 26292 (0.0007) [2023-10-13 21:59:48,743][60934] Updated weights for policy 1, policy_version 26302 (0.0010) [2023-10-13 21:59:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 53575680. Throughput: 0: 1681.5, 1: 1692.0. Samples: 13401890. Policy #0 lag: (min: 9.0, avg: 21.5, max: 41.0) [2023-10-13 21:59:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 21:59:52,159][60935] Updated weights for policy 0, policy_version 26020 (0.0007) [2023-10-13 21:59:52,534][60935] Updated weights for policy 0, policy_version 26030 (0.0009) [2023-10-13 21:59:52,894][60934] Updated weights for policy 1, policy_version 26312 (0.0008) [2023-10-13 21:59:52,909][60935] Updated weights for policy 0, policy_version 26040 (0.0008) [2023-10-13 21:59:53,261][60934] Updated weights for policy 1, policy_version 26322 (0.0007) [2023-10-13 21:59:53,633][60934] Updated weights for policy 1, policy_version 26332 (0.0007) [2023-10-13 21:59:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 53641216. Throughput: 0: 1674.3, 1: 1717.3. Samples: 13422814. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-13 21:59:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.150')] [2023-10-13 21:59:56,999][60935] Updated weights for policy 0, policy_version 26050 (0.0008) [2023-10-13 21:59:57,363][60935] Updated weights for policy 0, policy_version 26060 (0.0010) [2023-10-13 21:59:57,450][60934] Updated weights for policy 1, policy_version 26342 (0.0008) [2023-10-13 21:59:57,733][60935] Updated weights for policy 0, policy_version 26070 (0.0007) [2023-10-13 21:59:57,811][60934] Updated weights for policy 1, policy_version 26352 (0.0008) [2023-10-13 21:59:58,100][60935] Updated weights for policy 0, policy_version 26080 (0.0008) [2023-10-13 21:59:58,181][60934] Updated weights for policy 1, policy_version 26362 (0.0007) [2023-10-13 22:00:01,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 53706752. Throughput: 0: 1667.9, 1: 1690.4. Samples: 13432034. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-13 22:00:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.150')] [2023-10-13 22:00:02,121][60935] Updated weights for policy 0, policy_version 26090 (0.0009) [2023-10-13 22:00:02,484][60935] Updated weights for policy 0, policy_version 26100 (0.0007) [2023-10-13 22:00:02,522][60934] Updated weights for policy 1, policy_version 26372 (0.0011) [2023-10-13 22:00:02,858][60935] Updated weights for policy 0, policy_version 26110 (0.0009) [2023-10-13 22:00:02,889][60934] Updated weights for policy 1, policy_version 26382 (0.0008) [2023-10-13 22:00:03,262][60934] Updated weights for policy 1, policy_version 26392 (0.0007) [2023-10-13 22:00:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 53772288. Throughput: 0: 1677.4, 1: 1698.0. Samples: 13452804. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-13 22:00:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.150')] [2023-10-13 22:00:06,975][60935] Updated weights for policy 0, policy_version 26120 (0.0011) [2023-10-13 22:00:07,303][60934] Updated weights for policy 1, policy_version 26402 (0.0007) [2023-10-13 22:00:07,329][60935] Updated weights for policy 0, policy_version 26130 (0.0009) [2023-10-13 22:00:07,665][60934] Updated weights for policy 1, policy_version 26412 (0.0008) [2023-10-13 22:00:07,707][60935] Updated weights for policy 0, policy_version 26140 (0.0010) [2023-10-13 22:00:08,037][60934] Updated weights for policy 1, policy_version 26422 (0.0007) [2023-10-13 22:00:08,405][60934] Updated weights for policy 1, policy_version 26432 (0.0010) [2023-10-13 22:00:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 53837824. Throughput: 0: 1676.4, 1: 1720.1. Samples: 13473802. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-13 22:00:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.150')] [2023-10-13 22:00:11,838][60935] Updated weights for policy 0, policy_version 26150 (0.0008) [2023-10-13 22:00:12,202][60935] Updated weights for policy 0, policy_version 26160 (0.0007) [2023-10-13 22:00:12,304][60934] Updated weights for policy 1, policy_version 26442 (0.0007) [2023-10-13 22:00:12,565][60935] Updated weights for policy 0, policy_version 26170 (0.0009) [2023-10-13 22:00:12,678][60934] Updated weights for policy 1, policy_version 26452 (0.0007) [2023-10-13 22:00:13,043][60934] Updated weights for policy 1, policy_version 26462 (0.0007) [2023-10-13 22:00:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 53903360. Throughput: 0: 1673.6, 1: 1690.1. Samples: 13482882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:00:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.150')] [2023-10-13 22:00:16,666][60935] Updated weights for policy 0, policy_version 26180 (0.0008) [2023-10-13 22:00:17,027][60935] Updated weights for policy 0, policy_version 26190 (0.0009) [2023-10-13 22:00:17,041][60934] Updated weights for policy 1, policy_version 26472 (0.0008) [2023-10-13 22:00:17,399][60935] Updated weights for policy 0, policy_version 26200 (0.0008) [2023-10-13 22:00:17,404][60934] Updated weights for policy 1, policy_version 26482 (0.0007) [2023-10-13 22:00:17,772][60934] Updated weights for policy 1, policy_version 26492 (0.0008) [2023-10-13 22:00:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 53968896. Throughput: 0: 1675.6, 1: 1713.6. Samples: 13503748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:00:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.150')] [2023-10-13 22:00:21,572][60935] Updated weights for policy 0, policy_version 26210 (0.0009) [2023-10-13 22:00:21,667][60934] Updated weights for policy 1, policy_version 26502 (0.0008) [2023-10-13 22:00:21,944][60935] Updated weights for policy 0, policy_version 26220 (0.0008) [2023-10-13 22:00:22,036][60934] Updated weights for policy 1, policy_version 26512 (0.0008) [2023-10-13 22:00:22,300][60935] Updated weights for policy 0, policy_version 26230 (0.0008) [2023-10-13 22:00:22,404][60934] Updated weights for policy 1, policy_version 26522 (0.0009) [2023-10-13 22:00:22,676][60935] Updated weights for policy 0, policy_version 26240 (0.0008) [2023-10-13 22:00:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 54034432. Throughput: 0: 1675.4, 1: 1714.2. Samples: 13524516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:00:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 22:00:26,299][60934] Updated weights for policy 1, policy_version 26532 (0.0009) [2023-10-13 22:00:26,660][60934] Updated weights for policy 1, policy_version 26542 (0.0009) [2023-10-13 22:00:26,866][60935] Updated weights for policy 0, policy_version 26250 (0.0009) [2023-10-13 22:00:27,029][60934] Updated weights for policy 1, policy_version 26552 (0.0011) [2023-10-13 22:00:27,239][60935] Updated weights for policy 0, policy_version 26260 (0.0009) [2023-10-13 22:00:27,599][60935] Updated weights for policy 0, policy_version 26270 (0.0010) [2023-10-13 22:00:31,223][60934] Updated weights for policy 1, policy_version 26562 (0.0009) [2023-10-13 22:00:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 54099968. Throughput: 0: 1672.9, 1: 1696.3. Samples: 13533462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:00:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 22:00:31,613][60934] Updated weights for policy 1, policy_version 26572 (0.0009) [2023-10-13 22:00:31,723][60935] Updated weights for policy 0, policy_version 26280 (0.0009) [2023-10-13 22:00:31,985][60934] Updated weights for policy 1, policy_version 26582 (0.0009) [2023-10-13 22:00:32,094][60935] Updated weights for policy 0, policy_version 26290 (0.0008) [2023-10-13 22:00:32,358][60934] Updated weights for policy 1, policy_version 26592 (0.0008) [2023-10-13 22:00:32,470][60935] Updated weights for policy 0, policy_version 26300 (0.0008) [2023-10-13 22:00:36,227][60934] Updated weights for policy 1, policy_version 26602 (0.0008) [2023-10-13 22:00:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 54165504. Throughput: 0: 1672.6, 1: 1714.8. Samples: 13554320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:00:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 22:00:36,509][60935] Updated weights for policy 0, policy_version 26310 (0.0008) [2023-10-13 22:00:36,591][60934] Updated weights for policy 1, policy_version 26612 (0.0007) [2023-10-13 22:00:36,883][60935] Updated weights for policy 0, policy_version 26320 (0.0010) [2023-10-13 22:00:36,960][60934] Updated weights for policy 1, policy_version 26622 (0.0010) [2023-10-13 22:00:37,250][60935] Updated weights for policy 0, policy_version 26330 (0.0009) [2023-10-13 22:00:40,982][60934] Updated weights for policy 1, policy_version 26632 (0.0009) [2023-10-13 22:00:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 54231040. Throughput: 0: 1668.8, 1: 1713.1. Samples: 13574998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:00:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 22:00:41,340][60934] Updated weights for policy 1, policy_version 26642 (0.0007) [2023-10-13 22:00:41,389][60935] Updated weights for policy 0, policy_version 26340 (0.0010) [2023-10-13 22:00:41,707][60934] Updated weights for policy 1, policy_version 26652 (0.0008) [2023-10-13 22:00:41,760][60935] Updated weights for policy 0, policy_version 26350 (0.0007) [2023-10-13 22:00:42,140][60935] Updated weights for policy 0, policy_version 26360 (0.0010) [2023-10-13 22:00:45,604][60934] Updated weights for policy 1, policy_version 26662 (0.0008) [2023-10-13 22:00:45,969][60934] Updated weights for policy 1, policy_version 26672 (0.0010) [2023-10-13 22:00:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 54296576. Throughput: 0: 1665.2, 1: 1714.3. Samples: 13584114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:00:46,248][60935] Updated weights for policy 0, policy_version 26370 (0.0010) [2023-10-13 22:00:46,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 22:00:46,337][60934] Updated weights for policy 1, policy_version 26682 (0.0008) [2023-10-13 22:00:46,615][60935] Updated weights for policy 0, policy_version 26380 (0.0008) [2023-10-13 22:00:46,983][60935] Updated weights for policy 0, policy_version 26390 (0.0007) [2023-10-13 22:00:47,344][60935] Updated weights for policy 0, policy_version 26400 (0.0011) [2023-10-13 22:00:50,479][60934] Updated weights for policy 1, policy_version 26692 (0.0009) [2023-10-13 22:00:50,850][60934] Updated weights for policy 1, policy_version 26702 (0.0009) [2023-10-13 22:00:51,208][60934] Updated weights for policy 1, policy_version 26712 (0.0007) [2023-10-13 22:00:51,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 54362112. Throughput: 0: 1663.7, 1: 1714.9. Samples: 13604842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:00:51,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 22:00:51,539][60935] Updated weights for policy 0, policy_version 26410 (0.0007) [2023-10-13 22:00:51,906][60935] Updated weights for policy 0, policy_version 26420 (0.0008) [2023-10-13 22:00:52,274][60935] Updated weights for policy 0, policy_version 26430 (0.0009) [2023-10-13 22:00:55,137][60934] Updated weights for policy 1, policy_version 26722 (0.0007) [2023-10-13 22:00:55,496][60934] Updated weights for policy 1, policy_version 26732 (0.0010) [2023-10-13 22:00:55,870][60934] Updated weights for policy 1, policy_version 26742 (0.0009) [2023-10-13 22:00:56,232][60934] Updated weights for policy 1, policy_version 26752 (0.0007) [2023-10-13 22:00:56,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 54460416. Throughput: 0: 1668.1, 1: 1696.2. Samples: 13625196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-13 22:00:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:00:56,256][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000026752_27394048.pth... [2023-10-13 22:00:56,295][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000025152_25755648.pth [2023-10-13 22:00:56,351][60935] Updated weights for policy 0, policy_version 26440 (0.0008) [2023-10-13 22:00:56,720][60935] Updated weights for policy 0, policy_version 26450 (0.0008) [2023-10-13 22:00:57,096][60935] Updated weights for policy 0, policy_version 26460 (0.0008) [2023-10-13 22:00:57,236][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000026464_27099136.pth... [2023-10-13 22:00:57,274][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000024896_25493504.pth [2023-10-13 22:01:00,201][60934] Updated weights for policy 1, policy_version 26762 (0.0008) [2023-10-13 22:01:00,568][60934] Updated weights for policy 1, policy_version 26772 (0.0008) [2023-10-13 22:01:00,925][60934] Updated weights for policy 1, policy_version 26782 (0.0008) [2023-10-13 22:01:01,120][60935] Updated weights for policy 0, policy_version 26470 (0.0008) [2023-10-13 22:01:01,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 54525952. Throughput: 0: 1668.4, 1: 1710.6. Samples: 13634936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-13 22:01:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:01:01,484][60935] Updated weights for policy 0, policy_version 26480 (0.0009) [2023-10-13 22:01:01,865][60935] Updated weights for policy 0, policy_version 26490 (0.0011) [2023-10-13 22:01:04,985][60934] Updated weights for policy 1, policy_version 26792 (0.0009) [2023-10-13 22:01:05,355][60934] Updated weights for policy 1, policy_version 26802 (0.0010) [2023-10-13 22:01:05,719][60934] Updated weights for policy 1, policy_version 26812 (0.0007) [2023-10-13 22:01:06,007][60935] Updated weights for policy 0, policy_version 26500 (0.0008) [2023-10-13 22:01:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 54591488. Throughput: 0: 1663.6, 1: 1711.7. Samples: 13655640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-13 22:01:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:01:06,381][60935] Updated weights for policy 0, policy_version 26510 (0.0007) [2023-10-13 22:01:06,751][60935] Updated weights for policy 0, policy_version 26520 (0.0008) [2023-10-13 22:01:09,768][60934] Updated weights for policy 1, policy_version 26822 (0.0008) [2023-10-13 22:01:10,126][60934] Updated weights for policy 1, policy_version 26832 (0.0009) [2023-10-13 22:01:10,492][60934] Updated weights for policy 1, policy_version 26842 (0.0010) [2023-10-13 22:01:10,694][60935] Updated weights for policy 0, policy_version 26530 (0.0010) [2023-10-13 22:01:11,074][60935] Updated weights for policy 0, policy_version 26540 (0.0008) [2023-10-13 22:01:11,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 54657024. Throughput: 0: 1670.1, 1: 1683.5. Samples: 13675426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-13 22:01:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:01:11,452][60935] Updated weights for policy 0, policy_version 26550 (0.0008) [2023-10-13 22:01:11,826][60935] Updated weights for policy 0, policy_version 26560 (0.0008) [2023-10-13 22:01:14,536][60934] Updated weights for policy 1, policy_version 26852 (0.0009) [2023-10-13 22:01:14,902][60934] Updated weights for policy 1, policy_version 26862 (0.0009) [2023-10-13 22:01:15,273][60934] Updated weights for policy 1, policy_version 26872 (0.0007) [2023-10-13 22:01:16,036][60935] Updated weights for policy 0, policy_version 26570 (0.0010) [2023-10-13 22:01:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 54722560. Throughput: 0: 1677.9, 1: 1708.8. Samples: 13685864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:01:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:01:16,414][60935] Updated weights for policy 0, policy_version 26580 (0.0011) [2023-10-13 22:01:16,779][60935] Updated weights for policy 0, policy_version 26590 (0.0009) [2023-10-13 22:01:19,319][60934] Updated weights for policy 1, policy_version 26882 (0.0009) [2023-10-13 22:01:19,696][60934] Updated weights for policy 1, policy_version 26892 (0.0009) [2023-10-13 22:01:20,059][60934] Updated weights for policy 1, policy_version 26902 (0.0009) [2023-10-13 22:01:20,430][60934] Updated weights for policy 1, policy_version 26912 (0.0010) [2023-10-13 22:01:20,992][60935] Updated weights for policy 0, policy_version 26600 (0.0008) [2023-10-13 22:01:21,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 54788096. Throughput: 0: 1671.5, 1: 1701.3. Samples: 13706096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:01:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:01:21,364][60935] Updated weights for policy 0, policy_version 26610 (0.0008) [2023-10-13 22:01:21,741][60935] Updated weights for policy 0, policy_version 26620 (0.0010) [2023-10-13 22:01:24,303][60934] Updated weights for policy 1, policy_version 26922 (0.0008) [2023-10-13 22:01:24,671][60934] Updated weights for policy 1, policy_version 26932 (0.0008) [2023-10-13 22:01:25,041][60934] Updated weights for policy 1, policy_version 26942 (0.0007) [2023-10-13 22:01:25,744][60935] Updated weights for policy 0, policy_version 26630 (0.0008) [2023-10-13 22:01:26,120][60935] Updated weights for policy 0, policy_version 26640 (0.0009) [2023-10-13 22:01:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 54853632. Throughput: 0: 1667.0, 1: 1685.5. Samples: 13725860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:01:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:01:26,492][60935] Updated weights for policy 0, policy_version 26650 (0.0008) [2023-10-13 22:01:29,104][60934] Updated weights for policy 1, policy_version 26952 (0.0007) [2023-10-13 22:01:29,478][60934] Updated weights for policy 1, policy_version 26962 (0.0008) [2023-10-13 22:01:29,848][60934] Updated weights for policy 1, policy_version 26972 (0.0008) [2023-10-13 22:01:30,701][60935] Updated weights for policy 0, policy_version 26660 (0.0011) [2023-10-13 22:01:31,064][60935] Updated weights for policy 0, policy_version 26670 (0.0012) [2023-10-13 22:01:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 54919168. Throughput: 0: 1673.5, 1: 1714.1. Samples: 13736556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:01:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:01:31,438][60935] Updated weights for policy 0, policy_version 26680 (0.0010) [2023-10-13 22:01:33,819][60934] Updated weights for policy 1, policy_version 26982 (0.0009) [2023-10-13 22:01:34,179][60934] Updated weights for policy 1, policy_version 26992 (0.0009) [2023-10-13 22:01:34,548][60934] Updated weights for policy 1, policy_version 27002 (0.0009) [2023-10-13 22:01:35,268][60935] Updated weights for policy 0, policy_version 26690 (0.0011) [2023-10-13 22:01:35,627][60935] Updated weights for policy 0, policy_version 26700 (0.0008) [2023-10-13 22:01:35,992][60935] Updated weights for policy 0, policy_version 26710 (0.0008) [2023-10-13 22:01:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 54984704. Throughput: 0: 1679.4, 1: 1695.6. Samples: 13756718. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-13 22:01:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:01:36,361][60935] Updated weights for policy 0, policy_version 26720 (0.0011) [2023-10-13 22:01:38,426][60934] Updated weights for policy 1, policy_version 27012 (0.0009) [2023-10-13 22:01:38,796][60934] Updated weights for policy 1, policy_version 27022 (0.0011) [2023-10-13 22:01:39,165][60934] Updated weights for policy 1, policy_version 27032 (0.0011) [2023-10-13 22:01:40,257][60935] Updated weights for policy 0, policy_version 26730 (0.0010) [2023-10-13 22:01:40,627][60935] Updated weights for policy 0, policy_version 26740 (0.0009) [2023-10-13 22:01:41,005][60935] Updated weights for policy 0, policy_version 26750 (0.0009) [2023-10-13 22:01:41,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 55083008. Throughput: 0: 1663.3, 1: 1699.3. Samples: 13776514. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-13 22:01:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:01:43,243][60934] Updated weights for policy 1, policy_version 27042 (0.0010) [2023-10-13 22:01:43,611][60934] Updated weights for policy 1, policy_version 27052 (0.0008) [2023-10-13 22:01:43,986][60934] Updated weights for policy 1, policy_version 27062 (0.0009) [2023-10-13 22:01:44,352][60934] Updated weights for policy 1, policy_version 27072 (0.0008) [2023-10-13 22:01:45,003][60935] Updated weights for policy 0, policy_version 26760 (0.0008) [2023-10-13 22:01:45,372][60935] Updated weights for policy 0, policy_version 26770 (0.0009) [2023-10-13 22:01:45,740][60935] Updated weights for policy 0, policy_version 26780 (0.0008) [2023-10-13 22:01:46,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 55148544. Throughput: 0: 1685.3, 1: 1706.5. Samples: 13787568. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-13 22:01:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:01:48,357][60934] Updated weights for policy 1, policy_version 27082 (0.0009) [2023-10-13 22:01:48,722][60934] Updated weights for policy 1, policy_version 27092 (0.0008) [2023-10-13 22:01:49,091][60934] Updated weights for policy 1, policy_version 27102 (0.0010) [2023-10-13 22:01:50,023][60935] Updated weights for policy 0, policy_version 26790 (0.0008) [2023-10-13 22:01:50,386][60935] Updated weights for policy 0, policy_version 26800 (0.0009) [2023-10-13 22:01:50,759][60935] Updated weights for policy 0, policy_version 26810 (0.0011) [2023-10-13 22:01:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 55214080. Throughput: 0: 1686.3, 1: 1687.3. Samples: 13807450. Policy #0 lag: (min: 23.0, avg: 25.7, max: 55.0) [2023-10-13 22:01:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:01:53,022][60934] Updated weights for policy 1, policy_version 27112 (0.0007) [2023-10-13 22:01:53,385][60934] Updated weights for policy 1, policy_version 27122 (0.0008) [2023-10-13 22:01:53,753][60934] Updated weights for policy 1, policy_version 27132 (0.0007) [2023-10-13 22:01:54,758][60935] Updated weights for policy 0, policy_version 26820 (0.0008) [2023-10-13 22:01:55,119][60935] Updated weights for policy 0, policy_version 26830 (0.0007) [2023-10-13 22:01:55,494][60935] Updated weights for policy 0, policy_version 26840 (0.0008) [2023-10-13 22:01:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 55279616. Throughput: 0: 1656.3, 1: 1720.1. Samples: 13827364. Policy #0 lag: (min: 23.0, avg: 25.7, max: 55.0) [2023-10-13 22:01:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:01:57,596][60934] Updated weights for policy 1, policy_version 27142 (0.0008) [2023-10-13 22:01:57,964][60934] Updated weights for policy 1, policy_version 27152 (0.0008) [2023-10-13 22:01:58,328][60934] Updated weights for policy 1, policy_version 27162 (0.0007) [2023-10-13 22:01:59,608][60935] Updated weights for policy 0, policy_version 26850 (0.0009) [2023-10-13 22:01:59,977][60935] Updated weights for policy 0, policy_version 26860 (0.0010) [2023-10-13 22:02:00,349][60935] Updated weights for policy 0, policy_version 26870 (0.0009) [2023-10-13 22:02:00,719][60935] Updated weights for policy 0, policy_version 26880 (0.0007) [2023-10-13 22:02:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 55345152. Throughput: 0: 1678.2, 1: 1700.9. Samples: 13837922. Policy #0 lag: (min: 23.0, avg: 25.7, max: 55.0) [2023-10-13 22:02:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:02:02,190][60934] Updated weights for policy 1, policy_version 27172 (0.0008) [2023-10-13 22:02:02,558][60934] Updated weights for policy 1, policy_version 27182 (0.0010) [2023-10-13 22:02:02,927][60934] Updated weights for policy 1, policy_version 27192 (0.0009) [2023-10-13 22:02:04,974][60935] Updated weights for policy 0, policy_version 26890 (0.0008) [2023-10-13 22:02:05,344][60935] Updated weights for policy 0, policy_version 26900 (0.0009) [2023-10-13 22:02:05,722][60935] Updated weights for policy 0, policy_version 26910 (0.0009) [2023-10-13 22:02:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 55410688. Throughput: 0: 1676.2, 1: 1713.6. Samples: 13858634. Policy #0 lag: (min: 23.0, avg: 25.7, max: 55.0) [2023-10-13 22:02:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:02:06,911][60934] Updated weights for policy 1, policy_version 27202 (0.0009) [2023-10-13 22:02:07,314][60934] Updated weights for policy 1, policy_version 27212 (0.0008) [2023-10-13 22:02:07,682][60934] Updated weights for policy 1, policy_version 27222 (0.0008) [2023-10-13 22:02:08,052][60934] Updated weights for policy 1, policy_version 27232 (0.0007) [2023-10-13 22:02:09,969][60935] Updated weights for policy 0, policy_version 26920 (0.0009) [2023-10-13 22:02:10,347][60935] Updated weights for policy 0, policy_version 26930 (0.0007) [2023-10-13 22:02:10,721][60935] Updated weights for policy 0, policy_version 26940 (0.0009) [2023-10-13 22:02:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 55476224. Throughput: 0: 1659.7, 1: 1730.3. Samples: 13878414. Policy #0 lag: (min: 5.0, avg: 12.8, max: 37.0) [2023-10-13 22:02:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:02:11,968][60934] Updated weights for policy 1, policy_version 27242 (0.0009) [2023-10-13 22:02:12,330][60934] Updated weights for policy 1, policy_version 27252 (0.0007) [2023-10-13 22:02:12,696][60934] Updated weights for policy 1, policy_version 27262 (0.0008) [2023-10-13 22:02:14,796][60935] Updated weights for policy 0, policy_version 26950 (0.0010) [2023-10-13 22:02:15,172][60935] Updated weights for policy 0, policy_version 26960 (0.0008) [2023-10-13 22:02:15,542][60935] Updated weights for policy 0, policy_version 26970 (0.0008) [2023-10-13 22:02:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 55541760. Throughput: 0: 1680.9, 1: 1701.7. Samples: 13888774. Policy #0 lag: (min: 5.0, avg: 12.8, max: 37.0) [2023-10-13 22:02:16,252][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:02:16,816][60934] Updated weights for policy 1, policy_version 27272 (0.0008) [2023-10-13 22:02:17,174][60934] Updated weights for policy 1, policy_version 27282 (0.0011) [2023-10-13 22:02:17,539][60934] Updated weights for policy 1, policy_version 27292 (0.0010) [2023-10-13 22:02:19,787][60935] Updated weights for policy 0, policy_version 26980 (0.0009) [2023-10-13 22:02:20,150][60935] Updated weights for policy 0, policy_version 26990 (0.0009) [2023-10-13 22:02:20,518][60935] Updated weights for policy 0, policy_version 27000 (0.0010) [2023-10-13 22:02:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 55607296. Throughput: 0: 1662.3, 1: 1723.0. Samples: 13909056. Policy #0 lag: (min: 5.0, avg: 12.8, max: 37.0) [2023-10-13 22:02:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:02:21,384][60934] Updated weights for policy 1, policy_version 27302 (0.0009) [2023-10-13 22:02:21,752][60934] Updated weights for policy 1, policy_version 27312 (0.0009) [2023-10-13 22:02:22,123][60934] Updated weights for policy 1, policy_version 27322 (0.0007) [2023-10-13 22:02:24,603][60935] Updated weights for policy 0, policy_version 27010 (0.0009) [2023-10-13 22:02:24,968][60935] Updated weights for policy 0, policy_version 27020 (0.0011) [2023-10-13 22:02:25,339][60935] Updated weights for policy 0, policy_version 27030 (0.0009) [2023-10-13 22:02:25,707][60935] Updated weights for policy 0, policy_version 27040 (0.0010) [2023-10-13 22:02:26,039][60934] Updated weights for policy 1, policy_version 27332 (0.0008) [2023-10-13 22:02:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 55672832. Throughput: 0: 1650.0, 1: 1735.4. Samples: 13928858. Policy #0 lag: (min: 5.0, avg: 12.8, max: 37.0) [2023-10-13 22:02:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:02:26,412][60934] Updated weights for policy 1, policy_version 27342 (0.0008) [2023-10-13 22:02:26,779][60934] Updated weights for policy 1, policy_version 27352 (0.0008) [2023-10-13 22:02:29,798][60935] Updated weights for policy 0, policy_version 27050 (0.0007) [2023-10-13 22:02:30,173][60935] Updated weights for policy 0, policy_version 27060 (0.0008) [2023-10-13 22:02:30,544][60935] Updated weights for policy 0, policy_version 27070 (0.0010) [2023-10-13 22:02:30,835][60934] Updated weights for policy 1, policy_version 27362 (0.0007) [2023-10-13 22:02:31,197][60934] Updated weights for policy 1, policy_version 27372 (0.0010) [2023-10-13 22:02:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 55738368. Throughput: 0: 1658.5, 1: 1715.4. Samples: 13939394. Policy #0 lag: (min: 12.0, avg: 12.6, max: 30.0) [2023-10-13 22:02:31,249][59943] Avg episode reward: [(0, '-0.010'), (1, '-0.030')] [2023-10-13 22:02:31,563][60934] Updated weights for policy 1, policy_version 27382 (0.0007) [2023-10-13 22:02:31,935][60934] Updated weights for policy 1, policy_version 27392 (0.0010) [2023-10-13 22:02:34,673][60935] Updated weights for policy 0, policy_version 27080 (0.0009) [2023-10-13 22:02:35,037][60935] Updated weights for policy 0, policy_version 27090 (0.0011) [2023-10-13 22:02:35,403][60935] Updated weights for policy 0, policy_version 27100 (0.0008) [2023-10-13 22:02:36,060][60934] Updated weights for policy 1, policy_version 27402 (0.0008) [2023-10-13 22:02:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 55803904. Throughput: 0: 1648.3, 1: 1730.0. Samples: 13959474. Policy #0 lag: (min: 12.0, avg: 12.6, max: 30.0) [2023-10-13 22:02:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:02:36,419][60934] Updated weights for policy 1, policy_version 27412 (0.0012) [2023-10-13 22:02:36,783][60934] Updated weights for policy 1, policy_version 27422 (0.0010) [2023-10-13 22:02:39,602][60935] Updated weights for policy 0, policy_version 27110 (0.0009) [2023-10-13 22:02:39,977][60935] Updated weights for policy 0, policy_version 27120 (0.0008) [2023-10-13 22:02:40,343][60935] Updated weights for policy 0, policy_version 27130 (0.0008) [2023-10-13 22:02:40,800][60934] Updated weights for policy 1, policy_version 27432 (0.0007) [2023-10-13 22:02:41,168][60934] Updated weights for policy 1, policy_version 27442 (0.0009) [2023-10-13 22:02:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 55869440. Throughput: 0: 1658.1, 1: 1718.2. Samples: 13979298. Policy #0 lag: (min: 12.0, avg: 12.6, max: 30.0) [2023-10-13 22:02:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:02:41,532][60934] Updated weights for policy 1, policy_version 27452 (0.0008) [2023-10-13 22:02:44,265][60935] Updated weights for policy 0, policy_version 27140 (0.0009) [2023-10-13 22:02:44,641][60935] Updated weights for policy 0, policy_version 27150 (0.0008) [2023-10-13 22:02:45,013][60935] Updated weights for policy 0, policy_version 27160 (0.0010) [2023-10-13 22:02:45,597][60934] Updated weights for policy 1, policy_version 27462 (0.0008) [2023-10-13 22:02:45,962][60934] Updated weights for policy 1, policy_version 27472 (0.0007) [2023-10-13 22:02:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 55934976. Throughput: 0: 1660.3, 1: 1713.5. Samples: 13989744. Policy #0 lag: (min: 12.0, avg: 12.6, max: 30.0) [2023-10-13 22:02:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:02:46,333][60934] Updated weights for policy 1, policy_version 27482 (0.0007) [2023-10-13 22:02:49,114][60935] Updated weights for policy 0, policy_version 27170 (0.0009) [2023-10-13 22:02:49,482][60935] Updated weights for policy 0, policy_version 27180 (0.0008) [2023-10-13 22:02:49,862][60935] Updated weights for policy 0, policy_version 27190 (0.0007) [2023-10-13 22:02:50,225][60935] Updated weights for policy 0, policy_version 27200 (0.0007) [2023-10-13 22:02:50,446][60934] Updated weights for policy 1, policy_version 27492 (0.0007) [2023-10-13 22:02:50,822][60934] Updated weights for policy 1, policy_version 27502 (0.0009) [2023-10-13 22:02:51,183][60934] Updated weights for policy 1, policy_version 27512 (0.0007) [2023-10-13 22:02:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 56000512. Throughput: 0: 1651.4, 1: 1705.7. Samples: 14009706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:02:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:02:54,275][60935] Updated weights for policy 0, policy_version 27210 (0.0010) [2023-10-13 22:02:54,647][60935] Updated weights for policy 0, policy_version 27220 (0.0009) [2023-10-13 22:02:55,015][60935] Updated weights for policy 0, policy_version 27230 (0.0007) [2023-10-13 22:02:55,063][60934] Updated weights for policy 1, policy_version 27522 (0.0007) [2023-10-13 22:02:55,433][60934] Updated weights for policy 1, policy_version 27532 (0.0010) [2023-10-13 22:02:55,800][60934] Updated weights for policy 1, policy_version 27542 (0.0009) [2023-10-13 22:02:56,157][60934] Updated weights for policy 1, policy_version 27552 (0.0009) [2023-10-13 22:02:56,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 56098816. Throughput: 0: 1666.7, 1: 1695.7. Samples: 14029722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:02:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:02:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000027232_27885568.pth... [2023-10-13 22:02:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000027552_28213248.pth... [2023-10-13 22:02:56,298][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000025952_26574848.pth [2023-10-13 22:02:56,298][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000025664_26279936.pth [2023-10-13 22:02:59,143][60935] Updated weights for policy 0, policy_version 27240 (0.0011) [2023-10-13 22:02:59,520][60935] Updated weights for policy 0, policy_version 27250 (0.0010) [2023-10-13 22:02:59,893][60935] Updated weights for policy 0, policy_version 27260 (0.0010) [2023-10-13 22:03:00,069][60934] Updated weights for policy 1, policy_version 27562 (0.0009) [2023-10-13 22:03:00,442][60934] Updated weights for policy 1, policy_version 27572 (0.0008) [2023-10-13 22:03:00,804][60934] Updated weights for policy 1, policy_version 27582 (0.0007) [2023-10-13 22:03:01,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 56164352. Throughput: 0: 1670.7, 1: 1708.6. Samples: 14040842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:03:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:03:03,922][60935] Updated weights for policy 0, policy_version 27270 (0.0008) [2023-10-13 22:03:04,292][60935] Updated weights for policy 0, policy_version 27280 (0.0009) [2023-10-13 22:03:04,666][60935] Updated weights for policy 0, policy_version 27290 (0.0009) [2023-10-13 22:03:04,873][60934] Updated weights for policy 1, policy_version 27592 (0.0008) [2023-10-13 22:03:05,251][60934] Updated weights for policy 1, policy_version 27602 (0.0009) [2023-10-13 22:03:05,616][60934] Updated weights for policy 1, policy_version 27612 (0.0009) [2023-10-13 22:03:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 56229888. Throughput: 0: 1658.4, 1: 1711.9. Samples: 14060722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:03:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:03:08,803][60935] Updated weights for policy 0, policy_version 27300 (0.0008) [2023-10-13 22:03:09,171][60935] Updated weights for policy 0, policy_version 27310 (0.0008) [2023-10-13 22:03:09,538][60935] Updated weights for policy 0, policy_version 27320 (0.0010) [2023-10-13 22:03:09,626][60934] Updated weights for policy 1, policy_version 27622 (0.0009) [2023-10-13 22:03:09,998][60934] Updated weights for policy 1, policy_version 27632 (0.0009) [2023-10-13 22:03:10,359][60934] Updated weights for policy 1, policy_version 27642 (0.0008) [2023-10-13 22:03:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 56295424. Throughput: 0: 1682.6, 1: 1679.9. Samples: 14080172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:03:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:03:13,635][60935] Updated weights for policy 0, policy_version 27330 (0.0008) [2023-10-13 22:03:14,012][60935] Updated weights for policy 0, policy_version 27340 (0.0010) [2023-10-13 22:03:14,215][60934] Updated weights for policy 1, policy_version 27652 (0.0007) [2023-10-13 22:03:14,381][60935] Updated weights for policy 0, policy_version 27350 (0.0008) [2023-10-13 22:03:14,581][60934] Updated weights for policy 1, policy_version 27662 (0.0007) [2023-10-13 22:03:14,753][60935] Updated weights for policy 0, policy_version 27360 (0.0008) [2023-10-13 22:03:14,937][60934] Updated weights for policy 1, policy_version 27672 (0.0009) [2023-10-13 22:03:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 56360960. Throughput: 0: 1672.6, 1: 1712.8. Samples: 14091736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:03:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:03:18,835][60935] Updated weights for policy 0, policy_version 27370 (0.0008) [2023-10-13 22:03:19,019][60934] Updated weights for policy 1, policy_version 27682 (0.0009) [2023-10-13 22:03:19,200][60935] Updated weights for policy 0, policy_version 27380 (0.0008) [2023-10-13 22:03:19,377][60934] Updated weights for policy 1, policy_version 27692 (0.0007) [2023-10-13 22:03:19,565][60935] Updated weights for policy 0, policy_version 27390 (0.0009) [2023-10-13 22:03:19,747][60934] Updated weights for policy 1, policy_version 27702 (0.0008) [2023-10-13 22:03:20,120][60934] Updated weights for policy 1, policy_version 27712 (0.0008) [2023-10-13 22:03:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 56426496. Throughput: 0: 1661.0, 1: 1702.7. Samples: 14110840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:03:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:03:23,787][60935] Updated weights for policy 0, policy_version 27400 (0.0008) [2023-10-13 22:03:24,111][60934] Updated weights for policy 1, policy_version 27722 (0.0007) [2023-10-13 22:03:24,150][60935] Updated weights for policy 0, policy_version 27410 (0.0008) [2023-10-13 22:03:24,472][60934] Updated weights for policy 1, policy_version 27732 (0.0007) [2023-10-13 22:03:24,532][60935] Updated weights for policy 0, policy_version 27420 (0.0008) [2023-10-13 22:03:24,828][60934] Updated weights for policy 1, policy_version 27742 (0.0009) [2023-10-13 22:03:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 56492032. Throughput: 0: 1675.4, 1: 1693.4. Samples: 14130894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:03:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:03:28,677][60935] Updated weights for policy 0, policy_version 27430 (0.0009) [2023-10-13 22:03:28,860][60934] Updated weights for policy 1, policy_version 27752 (0.0009) [2023-10-13 22:03:29,043][60935] Updated weights for policy 0, policy_version 27440 (0.0009) [2023-10-13 22:03:29,228][60934] Updated weights for policy 1, policy_version 27762 (0.0007) [2023-10-13 22:03:29,425][60935] Updated weights for policy 0, policy_version 27450 (0.0007) [2023-10-13 22:03:29,597][60934] Updated weights for policy 1, policy_version 27772 (0.0007) [2023-10-13 22:03:31,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 56557568. Throughput: 0: 1664.3, 1: 1723.4. Samples: 14142190. Policy #0 lag: (min: 17.0, avg: 26.2, max: 49.0) [2023-10-13 22:03:31,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:03:33,433][60935] Updated weights for policy 0, policy_version 27460 (0.0009) [2023-10-13 22:03:33,730][60934] Updated weights for policy 1, policy_version 27782 (0.0007) [2023-10-13 22:03:33,807][60935] Updated weights for policy 0, policy_version 27470 (0.0010) [2023-10-13 22:03:34,102][60934] Updated weights for policy 1, policy_version 27792 (0.0007) [2023-10-13 22:03:34,186][60935] Updated weights for policy 0, policy_version 27480 (0.0009) [2023-10-13 22:03:34,468][60934] Updated weights for policy 1, policy_version 27802 (0.0008) [2023-10-13 22:03:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 56623104. Throughput: 0: 1655.6, 1: 1699.6. Samples: 14160690. Policy #0 lag: (min: 17.0, avg: 26.2, max: 49.0) [2023-10-13 22:03:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:03:38,256][60935] Updated weights for policy 0, policy_version 27490 (0.0009) [2023-10-13 22:03:38,363][60934] Updated weights for policy 1, policy_version 27812 (0.0009) [2023-10-13 22:03:38,629][60935] Updated weights for policy 0, policy_version 27500 (0.0008) [2023-10-13 22:03:38,726][60934] Updated weights for policy 1, policy_version 27822 (0.0008) [2023-10-13 22:03:39,000][60935] Updated weights for policy 0, policy_version 27510 (0.0007) [2023-10-13 22:03:39,083][60934] Updated weights for policy 1, policy_version 27832 (0.0007) [2023-10-13 22:03:39,371][60935] Updated weights for policy 0, policy_version 27520 (0.0009) [2023-10-13 22:03:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 56688640. Throughput: 0: 1664.5, 1: 1703.8. Samples: 14181294. Policy #0 lag: (min: 17.0, avg: 26.2, max: 49.0) [2023-10-13 22:03:41,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:03:43,090][60934] Updated weights for policy 1, policy_version 27842 (0.0007) [2023-10-13 22:03:43,456][60934] Updated weights for policy 1, policy_version 27852 (0.0007) [2023-10-13 22:03:43,543][60935] Updated weights for policy 0, policy_version 27530 (0.0008) [2023-10-13 22:03:43,826][60934] Updated weights for policy 1, policy_version 27862 (0.0007) [2023-10-13 22:03:43,914][60935] Updated weights for policy 0, policy_version 27540 (0.0008) [2023-10-13 22:03:44,190][60934] Updated weights for policy 1, policy_version 27872 (0.0008) [2023-10-13 22:03:44,283][60935] Updated weights for policy 0, policy_version 27550 (0.0007) [2023-10-13 22:03:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 56754176. Throughput: 0: 1645.6, 1: 1708.3. Samples: 14191768. Policy #0 lag: (min: 17.0, avg: 26.2, max: 49.0) [2023-10-13 22:03:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:03:48,172][60934] Updated weights for policy 1, policy_version 27882 (0.0009) [2023-10-13 22:03:48,222][60935] Updated weights for policy 0, policy_version 27560 (0.0009) [2023-10-13 22:03:48,536][60934] Updated weights for policy 1, policy_version 27892 (0.0008) [2023-10-13 22:03:48,597][60935] Updated weights for policy 0, policy_version 27570 (0.0008) [2023-10-13 22:03:48,903][60934] Updated weights for policy 1, policy_version 27902 (0.0009) [2023-10-13 22:03:48,968][60935] Updated weights for policy 0, policy_version 27580 (0.0009) [2023-10-13 22:03:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 56819712. Throughput: 0: 1657.8, 1: 1685.1. Samples: 14211152. Policy #0 lag: (min: 9.0, avg: 10.1, max: 29.0) [2023-10-13 22:03:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:03:53,036][60934] Updated weights for policy 1, policy_version 27912 (0.0010) [2023-10-13 22:03:53,188][60935] Updated weights for policy 0, policy_version 27590 (0.0008) [2023-10-13 22:03:53,400][60934] Updated weights for policy 1, policy_version 27922 (0.0007) [2023-10-13 22:03:53,563][60935] Updated weights for policy 0, policy_version 27600 (0.0008) [2023-10-13 22:03:53,770][60934] Updated weights for policy 1, policy_version 27932 (0.0007) [2023-10-13 22:03:53,932][60935] Updated weights for policy 0, policy_version 27610 (0.0010) [2023-10-13 22:03:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 56885248. Throughput: 0: 1658.8, 1: 1712.6. Samples: 14231884. Policy #0 lag: (min: 9.0, avg: 10.1, max: 29.0) [2023-10-13 22:03:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:03:57,818][60934] Updated weights for policy 1, policy_version 27942 (0.0008) [2023-10-13 22:03:58,034][60935] Updated weights for policy 0, policy_version 27620 (0.0009) [2023-10-13 22:03:58,182][60934] Updated weights for policy 1, policy_version 27952 (0.0008) [2023-10-13 22:03:58,398][60935] Updated weights for policy 0, policy_version 27630 (0.0008) [2023-10-13 22:03:58,537][60934] Updated weights for policy 1, policy_version 27962 (0.0007) [2023-10-13 22:03:58,767][60935] Updated weights for policy 0, policy_version 27640 (0.0008) [2023-10-13 22:04:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 56950784. Throughput: 0: 1646.7, 1: 1684.9. Samples: 14241658. Policy #0 lag: (min: 9.0, avg: 10.1, max: 29.0) [2023-10-13 22:04:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:04:02,543][60934] Updated weights for policy 1, policy_version 27972 (0.0008) [2023-10-13 22:04:02,850][60935] Updated weights for policy 0, policy_version 27650 (0.0010) [2023-10-13 22:04:02,917][60934] Updated weights for policy 1, policy_version 27982 (0.0007) [2023-10-13 22:04:03,210][60935] Updated weights for policy 0, policy_version 27660 (0.0008) [2023-10-13 22:04:03,278][60934] Updated weights for policy 1, policy_version 27992 (0.0009) [2023-10-13 22:04:03,576][60935] Updated weights for policy 0, policy_version 27670 (0.0009) [2023-10-13 22:04:03,956][60935] Updated weights for policy 0, policy_version 27680 (0.0008) [2023-10-13 22:04:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 57016320. Throughput: 0: 1663.5, 1: 1690.3. Samples: 14261762. Policy #0 lag: (min: 9.0, avg: 10.1, max: 29.0) [2023-10-13 22:04:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:04:07,384][60934] Updated weights for policy 1, policy_version 28002 (0.0008) [2023-10-13 22:04:07,753][60934] Updated weights for policy 1, policy_version 28012 (0.0008) [2023-10-13 22:04:08,091][60935] Updated weights for policy 0, policy_version 27690 (0.0009) [2023-10-13 22:04:08,118][60934] Updated weights for policy 1, policy_version 28022 (0.0008) [2023-10-13 22:04:08,455][60935] Updated weights for policy 0, policy_version 27700 (0.0009) [2023-10-13 22:04:08,476][60934] Updated weights for policy 1, policy_version 28032 (0.0008) [2023-10-13 22:04:08,832][60935] Updated weights for policy 0, policy_version 27710 (0.0010) [2023-10-13 22:04:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 57081856. Throughput: 0: 1658.8, 1: 1702.5. Samples: 14282154. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-13 22:04:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:04:12,379][60934] Updated weights for policy 1, policy_version 28042 (0.0008) [2023-10-13 22:04:12,749][60934] Updated weights for policy 1, policy_version 28052 (0.0007) [2023-10-13 22:04:13,020][60935] Updated weights for policy 0, policy_version 27720 (0.0008) [2023-10-13 22:04:13,123][60934] Updated weights for policy 1, policy_version 28062 (0.0007) [2023-10-13 22:04:13,385][60935] Updated weights for policy 0, policy_version 27730 (0.0007) [2023-10-13 22:04:13,766][60935] Updated weights for policy 0, policy_version 27740 (0.0010) [2023-10-13 22:04:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 57147392. Throughput: 0: 1642.8, 1: 1673.3. Samples: 14291416. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-13 22:04:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:04:16,935][60934] Updated weights for policy 1, policy_version 28072 (0.0008) [2023-10-13 22:04:17,303][60934] Updated weights for policy 1, policy_version 28082 (0.0009) [2023-10-13 22:04:17,665][60934] Updated weights for policy 1, policy_version 28092 (0.0010) [2023-10-13 22:04:17,938][60935] Updated weights for policy 0, policy_version 27750 (0.0008) [2023-10-13 22:04:18,296][60935] Updated weights for policy 0, policy_version 27760 (0.0009) [2023-10-13 22:04:18,678][60935] Updated weights for policy 0, policy_version 27770 (0.0009) [2023-10-13 22:04:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 57212928. Throughput: 0: 1661.9, 1: 1709.2. Samples: 14312392. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-13 22:04:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:04:21,723][60934] Updated weights for policy 1, policy_version 28102 (0.0009) [2023-10-13 22:04:22,098][60934] Updated weights for policy 1, policy_version 28112 (0.0007) [2023-10-13 22:04:22,455][60934] Updated weights for policy 1, policy_version 28122 (0.0008) [2023-10-13 22:04:22,745][60935] Updated weights for policy 0, policy_version 27780 (0.0007) [2023-10-13 22:04:23,116][60935] Updated weights for policy 0, policy_version 27790 (0.0009) [2023-10-13 22:04:23,493][60935] Updated weights for policy 0, policy_version 27800 (0.0007) [2023-10-13 22:04:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 57278464. Throughput: 0: 1658.1, 1: 1713.9. Samples: 14333032. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-13 22:04:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:04:26,478][60934] Updated weights for policy 1, policy_version 28132 (0.0007) [2023-10-13 22:04:26,848][60934] Updated weights for policy 1, policy_version 28142 (0.0008) [2023-10-13 22:04:27,212][60934] Updated weights for policy 1, policy_version 28152 (0.0008) [2023-10-13 22:04:27,500][60935] Updated weights for policy 0, policy_version 27810 (0.0011) [2023-10-13 22:04:27,874][60935] Updated weights for policy 0, policy_version 27820 (0.0009) [2023-10-13 22:04:28,240][60935] Updated weights for policy 0, policy_version 27830 (0.0010) [2023-10-13 22:04:28,608][60935] Updated weights for policy 0, policy_version 27840 (0.0009) [2023-10-13 22:04:31,207][60934] Updated weights for policy 1, policy_version 28162 (0.0009) [2023-10-13 22:04:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 57344000. Throughput: 0: 1650.8, 1: 1695.2. Samples: 14342336. Policy #0 lag: (min: 4.0, avg: 5.6, max: 32.0) [2023-10-13 22:04:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:04:31,584][60934] Updated weights for policy 1, policy_version 28172 (0.0008) [2023-10-13 22:04:31,955][60934] Updated weights for policy 1, policy_version 28182 (0.0007) [2023-10-13 22:04:32,324][60934] Updated weights for policy 1, policy_version 28192 (0.0007) [2023-10-13 22:04:32,819][60935] Updated weights for policy 0, policy_version 27850 (0.0007) [2023-10-13 22:04:33,190][60935] Updated weights for policy 0, policy_version 27860 (0.0008) [2023-10-13 22:04:33,565][60935] Updated weights for policy 0, policy_version 27870 (0.0010) [2023-10-13 22:04:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 57409536. Throughput: 0: 1661.6, 1: 1718.8. Samples: 14363268. Policy #0 lag: (min: 4.0, avg: 5.6, max: 32.0) [2023-10-13 22:04:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:04:36,293][60934] Updated weights for policy 1, policy_version 28202 (0.0010) [2023-10-13 22:04:36,659][60934] Updated weights for policy 1, policy_version 28212 (0.0010) [2023-10-13 22:04:37,030][60934] Updated weights for policy 1, policy_version 28222 (0.0007) [2023-10-13 22:04:37,690][60935] Updated weights for policy 0, policy_version 27880 (0.0010) [2023-10-13 22:04:38,064][60935] Updated weights for policy 0, policy_version 27890 (0.0009) [2023-10-13 22:04:38,434][60935] Updated weights for policy 0, policy_version 27900 (0.0007) [2023-10-13 22:04:40,964][60934] Updated weights for policy 1, policy_version 28232 (0.0008) [2023-10-13 22:04:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 57475072. Throughput: 0: 1662.8, 1: 1723.1. Samples: 14384252. Policy #0 lag: (min: 4.0, avg: 5.6, max: 32.0) [2023-10-13 22:04:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:04:41,334][60934] Updated weights for policy 1, policy_version 28242 (0.0009) [2023-10-13 22:04:41,710][60934] Updated weights for policy 1, policy_version 28252 (0.0007) [2023-10-13 22:04:42,507][60935] Updated weights for policy 0, policy_version 27910 (0.0009) [2023-10-13 22:04:42,880][60935] Updated weights for policy 0, policy_version 27920 (0.0010) [2023-10-13 22:04:43,254][60935] Updated weights for policy 0, policy_version 27930 (0.0008) [2023-10-13 22:04:45,632][60934] Updated weights for policy 1, policy_version 28262 (0.0009) [2023-10-13 22:04:46,001][60934] Updated weights for policy 1, policy_version 28272 (0.0009) [2023-10-13 22:04:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 57540608. Throughput: 0: 1650.2, 1: 1718.5. Samples: 14393250. Policy #0 lag: (min: 4.0, avg: 5.6, max: 32.0) [2023-10-13 22:04:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:04:46,367][60934] Updated weights for policy 1, policy_version 28282 (0.0007) [2023-10-13 22:04:47,291][60935] Updated weights for policy 0, policy_version 27940 (0.0008) [2023-10-13 22:04:47,660][60935] Updated weights for policy 0, policy_version 27950 (0.0008) [2023-10-13 22:04:48,031][60935] Updated weights for policy 0, policy_version 27960 (0.0008) [2023-10-13 22:04:50,199][60934] Updated weights for policy 1, policy_version 28292 (0.0010) [2023-10-13 22:04:50,573][60934] Updated weights for policy 1, policy_version 28302 (0.0010) [2023-10-13 22:04:50,937][60934] Updated weights for policy 1, policy_version 28312 (0.0010) [2023-10-13 22:04:51,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 57638912. Throughput: 0: 1659.7, 1: 1728.5. Samples: 14414232. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-13 22:04:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:04:52,090][60935] Updated weights for policy 0, policy_version 27970 (0.0011) [2023-10-13 22:04:52,469][60935] Updated weights for policy 0, policy_version 27980 (0.0009) [2023-10-13 22:04:52,837][60935] Updated weights for policy 0, policy_version 27990 (0.0009) [2023-10-13 22:04:53,212][60935] Updated weights for policy 0, policy_version 28000 (0.0009) [2023-10-13 22:04:54,978][60934] Updated weights for policy 1, policy_version 28322 (0.0010) [2023-10-13 22:04:55,339][60934] Updated weights for policy 1, policy_version 28332 (0.0008) [2023-10-13 22:04:55,717][60934] Updated weights for policy 1, policy_version 28342 (0.0009) [2023-10-13 22:04:56,079][60934] Updated weights for policy 1, policy_version 28352 (0.0007) [2023-10-13 22:04:56,248][59943] Fps is (10 sec: 16383.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 57704448. Throughput: 0: 1670.2, 1: 1714.2. Samples: 14434452. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-13 22:04:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:04:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000028352_29032448.pth... [2023-10-13 22:04:56,260][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000028000_28672000.pth... [2023-10-13 22:04:56,297][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000026752_27394048.pth [2023-10-13 22:04:56,298][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000026464_27099136.pth [2023-10-13 22:04:57,207][60935] Updated weights for policy 0, policy_version 28010 (0.0010) [2023-10-13 22:04:57,581][60935] Updated weights for policy 0, policy_version 28020 (0.0008) [2023-10-13 22:04:57,960][60935] Updated weights for policy 0, policy_version 28030 (0.0009) [2023-10-13 22:05:00,100][60934] Updated weights for policy 1, policy_version 28362 (0.0009) [2023-10-13 22:05:00,468][60934] Updated weights for policy 1, policy_version 28372 (0.0010) [2023-10-13 22:05:00,841][60934] Updated weights for policy 1, policy_version 28382 (0.0008) [2023-10-13 22:05:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 57769984. Throughput: 0: 1669.2, 1: 1731.7. Samples: 14444456. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-13 22:05:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:05:02,093][60935] Updated weights for policy 0, policy_version 28040 (0.0009) [2023-10-13 22:05:02,462][60935] Updated weights for policy 0, policy_version 28050 (0.0009) [2023-10-13 22:05:02,833][60935] Updated weights for policy 0, policy_version 28060 (0.0007) [2023-10-13 22:05:04,829][60934] Updated weights for policy 1, policy_version 28392 (0.0007) [2023-10-13 22:05:05,194][60934] Updated weights for policy 1, policy_version 28402 (0.0008) [2023-10-13 22:05:05,559][60934] Updated weights for policy 1, policy_version 28412 (0.0010) [2023-10-13 22:05:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 57835520. Throughput: 0: 1682.0, 1: 1723.7. Samples: 14465650. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-13 22:05:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:05:06,786][60935] Updated weights for policy 0, policy_version 28070 (0.0009) [2023-10-13 22:05:07,162][60935] Updated weights for policy 0, policy_version 28080 (0.0011) [2023-10-13 22:05:07,528][60935] Updated weights for policy 0, policy_version 28090 (0.0009) [2023-10-13 22:05:09,665][60934] Updated weights for policy 1, policy_version 28422 (0.0011) [2023-10-13 22:05:10,028][60934] Updated weights for policy 1, policy_version 28432 (0.0010) [2023-10-13 22:05:10,406][60934] Updated weights for policy 1, policy_version 28442 (0.0010) [2023-10-13 22:05:11,249][59943] Fps is (10 sec: 13106.6, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 57901056. Throughput: 0: 1690.7, 1: 1694.7. Samples: 14485378. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-13 22:05:11,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:05:11,534][60935] Updated weights for policy 0, policy_version 28100 (0.0008) [2023-10-13 22:05:11,914][60935] Updated weights for policy 0, policy_version 28110 (0.0010) [2023-10-13 22:05:12,274][60935] Updated weights for policy 0, policy_version 28120 (0.0009) [2023-10-13 22:05:14,331][60934] Updated weights for policy 1, policy_version 28452 (0.0008) [2023-10-13 22:05:14,688][60934] Updated weights for policy 1, policy_version 28462 (0.0011) [2023-10-13 22:05:15,055][60934] Updated weights for policy 1, policy_version 28472 (0.0010) [2023-10-13 22:05:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 57966592. Throughput: 0: 1686.2, 1: 1723.3. Samples: 14495764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:05:16,249][59943] Avg episode reward: [(0, '-0.130'), (1, '0.000')] [2023-10-13 22:05:16,273][60935] Updated weights for policy 0, policy_version 28130 (0.0008) [2023-10-13 22:05:16,642][60935] Updated weights for policy 0, policy_version 28140 (0.0007) [2023-10-13 22:05:17,020][60935] Updated weights for policy 0, policy_version 28150 (0.0007) [2023-10-13 22:05:17,382][60935] Updated weights for policy 0, policy_version 28160 (0.0007) [2023-10-13 22:05:19,066][60934] Updated weights for policy 1, policy_version 28482 (0.0009) [2023-10-13 22:05:19,488][60934] Updated weights for policy 1, policy_version 28492 (0.0009) [2023-10-13 22:05:19,859][60934] Updated weights for policy 1, policy_version 28502 (0.0009) [2023-10-13 22:05:20,218][60934] Updated weights for policy 1, policy_version 28512 (0.0007) [2023-10-13 22:05:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 58032128. Throughput: 0: 1692.6, 1: 1704.9. Samples: 14516156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:05:21,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 22:05:21,496][60935] Updated weights for policy 0, policy_version 28170 (0.0010) [2023-10-13 22:05:21,862][60935] Updated weights for policy 0, policy_version 28180 (0.0009) [2023-10-13 22:05:22,225][60935] Updated weights for policy 0, policy_version 28190 (0.0008) [2023-10-13 22:05:24,221][60934] Updated weights for policy 1, policy_version 28522 (0.0009) [2023-10-13 22:05:24,589][60934] Updated weights for policy 1, policy_version 28532 (0.0008) [2023-10-13 22:05:24,949][60934] Updated weights for policy 1, policy_version 28542 (0.0009) [2023-10-13 22:05:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 58097664. Throughput: 0: 1689.4, 1: 1685.2. Samples: 14536108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:05:26,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-13 22:05:26,440][60935] Updated weights for policy 0, policy_version 28200 (0.0010) [2023-10-13 22:05:26,814][60935] Updated weights for policy 0, policy_version 28210 (0.0009) [2023-10-13 22:05:27,180][60935] Updated weights for policy 0, policy_version 28220 (0.0008) [2023-10-13 22:05:28,999][60934] Updated weights for policy 1, policy_version 28552 (0.0008) [2023-10-13 22:05:29,364][60934] Updated weights for policy 1, policy_version 28562 (0.0008) [2023-10-13 22:05:29,728][60934] Updated weights for policy 1, policy_version 28572 (0.0008) [2023-10-13 22:05:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 58163200. Throughput: 0: 1689.5, 1: 1714.6. Samples: 14546436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:05:31,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-13 22:05:31,273][60935] Updated weights for policy 0, policy_version 28230 (0.0010) [2023-10-13 22:05:31,650][60935] Updated weights for policy 0, policy_version 28240 (0.0012) [2023-10-13 22:05:32,021][60935] Updated weights for policy 0, policy_version 28250 (0.0010) [2023-10-13 22:05:33,667][60934] Updated weights for policy 1, policy_version 28582 (0.0010) [2023-10-13 22:05:34,030][60934] Updated weights for policy 1, policy_version 28592 (0.0007) [2023-10-13 22:05:34,399][60934] Updated weights for policy 1, policy_version 28602 (0.0010) [2023-10-13 22:05:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 58228736. Throughput: 0: 1685.3, 1: 1688.0. Samples: 14566032. Policy #0 lag: (min: 26.0, avg: 26.0, max: 27.0) [2023-10-13 22:05:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:05:36,345][60935] Updated weights for policy 0, policy_version 28260 (0.0010) [2023-10-13 22:05:36,710][60935] Updated weights for policy 0, policy_version 28270 (0.0009) [2023-10-13 22:05:37,086][60935] Updated weights for policy 0, policy_version 28280 (0.0008) [2023-10-13 22:05:38,459][60934] Updated weights for policy 1, policy_version 28612 (0.0010) [2023-10-13 22:05:38,825][60934] Updated weights for policy 1, policy_version 28622 (0.0007) [2023-10-13 22:05:39,196][60934] Updated weights for policy 1, policy_version 28632 (0.0007) [2023-10-13 22:05:41,224][60935] Updated weights for policy 0, policy_version 28290 (0.0009) [2023-10-13 22:05:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 58294272. Throughput: 0: 1677.8, 1: 1701.7. Samples: 14586528. Policy #0 lag: (min: 26.0, avg: 26.0, max: 27.0) [2023-10-13 22:05:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:05:41,593][60935] Updated weights for policy 0, policy_version 28300 (0.0010) [2023-10-13 22:05:41,972][60935] Updated weights for policy 0, policy_version 28310 (0.0009) [2023-10-13 22:05:42,337][60935] Updated weights for policy 0, policy_version 28320 (0.0009) [2023-10-13 22:05:43,158][60934] Updated weights for policy 1, policy_version 28642 (0.0008) [2023-10-13 22:05:43,527][60934] Updated weights for policy 1, policy_version 28652 (0.0009) [2023-10-13 22:05:43,887][60934] Updated weights for policy 1, policy_version 28662 (0.0007) [2023-10-13 22:05:44,257][60934] Updated weights for policy 1, policy_version 28672 (0.0008) [2023-10-13 22:05:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 58359808. Throughput: 0: 1677.3, 1: 1700.6. Samples: 14596460. Policy #0 lag: (min: 26.0, avg: 26.0, max: 27.0) [2023-10-13 22:05:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:05:46,454][60935] Updated weights for policy 0, policy_version 28330 (0.0007) [2023-10-13 22:05:46,822][60935] Updated weights for policy 0, policy_version 28340 (0.0008) [2023-10-13 22:05:47,196][60935] Updated weights for policy 0, policy_version 28350 (0.0008) [2023-10-13 22:05:48,154][60934] Updated weights for policy 1, policy_version 28682 (0.0008) [2023-10-13 22:05:48,510][60934] Updated weights for policy 1, policy_version 28692 (0.0010) [2023-10-13 22:05:48,882][60934] Updated weights for policy 1, policy_version 28702 (0.0009) [2023-10-13 22:05:51,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 58425344. Throughput: 0: 1665.9, 1: 1687.1. Samples: 14616532. Policy #0 lag: (min: 26.0, avg: 26.0, max: 27.0) [2023-10-13 22:05:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:05:51,303][60935] Updated weights for policy 0, policy_version 28360 (0.0010) [2023-10-13 22:05:51,658][60935] Updated weights for policy 0, policy_version 28370 (0.0011) [2023-10-13 22:05:52,032][60935] Updated weights for policy 0, policy_version 28380 (0.0007) [2023-10-13 22:05:52,774][60934] Updated weights for policy 1, policy_version 28712 (0.0007) [2023-10-13 22:05:53,136][60934] Updated weights for policy 1, policy_version 28722 (0.0007) [2023-10-13 22:05:53,508][60934] Updated weights for policy 1, policy_version 28732 (0.0007) [2023-10-13 22:05:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 58490880. Throughput: 0: 1658.6, 1: 1718.2. Samples: 14637334. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-13 22:05:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:05:56,272][60935] Updated weights for policy 0, policy_version 28390 (0.0009) [2023-10-13 22:05:56,644][60935] Updated weights for policy 0, policy_version 28400 (0.0011) [2023-10-13 22:05:57,018][60935] Updated weights for policy 0, policy_version 28410 (0.0008) [2023-10-13 22:05:57,609][60934] Updated weights for policy 1, policy_version 28742 (0.0008) [2023-10-13 22:05:57,970][60934] Updated weights for policy 1, policy_version 28752 (0.0008) [2023-10-13 22:05:58,342][60934] Updated weights for policy 1, policy_version 28762 (0.0007) [2023-10-13 22:06:01,025][60935] Updated weights for policy 0, policy_version 28420 (0.0009) [2023-10-13 22:06:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 58556416. Throughput: 0: 1657.3, 1: 1689.8. Samples: 14646382. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-13 22:06:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:06:01,391][60935] Updated weights for policy 0, policy_version 28430 (0.0011) [2023-10-13 22:06:01,758][60935] Updated weights for policy 0, policy_version 28440 (0.0008) [2023-10-13 22:06:02,248][60934] Updated weights for policy 1, policy_version 28772 (0.0009) [2023-10-13 22:06:02,611][60934] Updated weights for policy 1, policy_version 28782 (0.0009) [2023-10-13 22:06:02,972][60934] Updated weights for policy 1, policy_version 28792 (0.0011) [2023-10-13 22:06:05,930][60935] Updated weights for policy 0, policy_version 28450 (0.0008) [2023-10-13 22:06:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 58621952. Throughput: 0: 1653.7, 1: 1702.2. Samples: 14667172. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-13 22:06:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:06:06,290][60935] Updated weights for policy 0, policy_version 28460 (0.0009) [2023-10-13 22:06:06,667][60935] Updated weights for policy 0, policy_version 28470 (0.0009) [2023-10-13 22:06:07,036][60935] Updated weights for policy 0, policy_version 28480 (0.0011) [2023-10-13 22:06:07,099][60934] Updated weights for policy 1, policy_version 28802 (0.0010) [2023-10-13 22:06:07,520][60934] Updated weights for policy 1, policy_version 28812 (0.0008) [2023-10-13 22:06:07,903][60934] Updated weights for policy 1, policy_version 28822 (0.0009) [2023-10-13 22:06:08,266][60934] Updated weights for policy 1, policy_version 28832 (0.0007) [2023-10-13 22:06:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 58687488. Throughput: 0: 1655.3, 1: 1716.9. Samples: 14687858. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-13 22:06:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:06:11,313][60935] Updated weights for policy 0, policy_version 28490 (0.0008) [2023-10-13 22:06:11,686][60935] Updated weights for policy 0, policy_version 28500 (0.0007) [2023-10-13 22:06:12,048][60935] Updated weights for policy 0, policy_version 28510 (0.0010) [2023-10-13 22:06:12,114][60934] Updated weights for policy 1, policy_version 28842 (0.0009) [2023-10-13 22:06:12,477][60934] Updated weights for policy 1, policy_version 28852 (0.0007) [2023-10-13 22:06:12,844][60934] Updated weights for policy 1, policy_version 28862 (0.0007) [2023-10-13 22:06:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 58753024. Throughput: 0: 1655.2, 1: 1685.7. Samples: 14696780. Policy #0 lag: (min: 0.0, avg: 21.1, max: 32.0) [2023-10-13 22:06:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:06:16,256][60935] Updated weights for policy 0, policy_version 28520 (0.0008) [2023-10-13 22:06:16,628][60935] Updated weights for policy 0, policy_version 28530 (0.0007) [2023-10-13 22:06:16,864][60934] Updated weights for policy 1, policy_version 28872 (0.0008) [2023-10-13 22:06:17,004][60935] Updated weights for policy 0, policy_version 28540 (0.0007) [2023-10-13 22:06:17,228][60934] Updated weights for policy 1, policy_version 28882 (0.0010) [2023-10-13 22:06:17,595][60934] Updated weights for policy 1, policy_version 28892 (0.0010) [2023-10-13 22:06:20,999][60935] Updated weights for policy 0, policy_version 28550 (0.0007) [2023-10-13 22:06:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 58818560. Throughput: 0: 1656.8, 1: 1708.9. Samples: 14717486. Policy #0 lag: (min: 0.0, avg: 21.1, max: 32.0) [2023-10-13 22:06:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:06:21,371][60935] Updated weights for policy 0, policy_version 28560 (0.0010) [2023-10-13 22:06:21,658][60934] Updated weights for policy 1, policy_version 28902 (0.0008) [2023-10-13 22:06:21,745][60935] Updated weights for policy 0, policy_version 28570 (0.0007) [2023-10-13 22:06:22,034][60934] Updated weights for policy 1, policy_version 28912 (0.0009) [2023-10-13 22:06:22,394][60934] Updated weights for policy 1, policy_version 28922 (0.0009) [2023-10-13 22:06:25,743][60935] Updated weights for policy 0, policy_version 28580 (0.0010) [2023-10-13 22:06:26,112][60935] Updated weights for policy 0, policy_version 28590 (0.0009) [2023-10-13 22:06:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 58884096. Throughput: 0: 1661.8, 1: 1710.1. Samples: 14738260. Policy #0 lag: (min: 0.0, avg: 21.1, max: 32.0) [2023-10-13 22:06:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:06:26,480][60935] Updated weights for policy 0, policy_version 28600 (0.0008) [2023-10-13 22:06:26,532][60934] Updated weights for policy 1, policy_version 28932 (0.0009) [2023-10-13 22:06:26,897][60934] Updated weights for policy 1, policy_version 28942 (0.0007) [2023-10-13 22:06:27,271][60934] Updated weights for policy 1, policy_version 28952 (0.0008) [2023-10-13 22:06:30,549][60935] Updated weights for policy 0, policy_version 28610 (0.0008) [2023-10-13 22:06:30,921][60935] Updated weights for policy 0, policy_version 28620 (0.0008) [2023-10-13 22:06:31,168][60934] Updated weights for policy 1, policy_version 28962 (0.0008) [2023-10-13 22:06:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 58949632. Throughput: 0: 1664.7, 1: 1695.9. Samples: 14747688. Policy #0 lag: (min: 0.0, avg: 21.1, max: 32.0) [2023-10-13 22:06:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:06:31,292][60935] Updated weights for policy 0, policy_version 28630 (0.0008) [2023-10-13 22:06:31,537][60934] Updated weights for policy 1, policy_version 28972 (0.0007) [2023-10-13 22:06:31,658][60935] Updated weights for policy 0, policy_version 28640 (0.0008) [2023-10-13 22:06:31,901][60934] Updated weights for policy 1, policy_version 28982 (0.0007) [2023-10-13 22:06:32,268][60934] Updated weights for policy 1, policy_version 28992 (0.0009) [2023-10-13 22:06:35,683][60935] Updated weights for policy 0, policy_version 28650 (0.0008) [2023-10-13 22:06:36,065][60935] Updated weights for policy 0, policy_version 28660 (0.0009) [2023-10-13 22:06:36,226][60934] Updated weights for policy 1, policy_version 29002 (0.0008) [2023-10-13 22:06:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 59015168. Throughput: 0: 1669.0, 1: 1711.0. Samples: 14768632. Policy #0 lag: (min: 6.0, avg: 6.7, max: 25.0) [2023-10-13 22:06:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:06:36,433][60935] Updated weights for policy 0, policy_version 28670 (0.0009) [2023-10-13 22:06:36,587][60934] Updated weights for policy 1, policy_version 29012 (0.0009) [2023-10-13 22:06:36,947][60934] Updated weights for policy 1, policy_version 29022 (0.0009) [2023-10-13 22:06:40,470][60935] Updated weights for policy 0, policy_version 28680 (0.0010) [2023-10-13 22:06:40,839][60935] Updated weights for policy 0, policy_version 28690 (0.0008) [2023-10-13 22:06:40,858][60934] Updated weights for policy 1, policy_version 29032 (0.0007) [2023-10-13 22:06:41,212][60935] Updated weights for policy 0, policy_version 28700 (0.0008) [2023-10-13 22:06:41,228][60934] Updated weights for policy 1, policy_version 29042 (0.0008) [2023-10-13 22:06:41,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 59080704. Throughput: 0: 1655.0, 1: 1717.3. Samples: 14789090. Policy #0 lag: (min: 6.0, avg: 6.7, max: 25.0) [2023-10-13 22:06:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:06:41,607][60934] Updated weights for policy 1, policy_version 29052 (0.0008) [2023-10-13 22:06:45,387][60935] Updated weights for policy 0, policy_version 28710 (0.0011) [2023-10-13 22:06:45,548][60934] Updated weights for policy 1, policy_version 29062 (0.0008) [2023-10-13 22:06:45,753][60935] Updated weights for policy 0, policy_version 28720 (0.0008) [2023-10-13 22:06:45,916][60934] Updated weights for policy 1, policy_version 29072 (0.0007) [2023-10-13 22:06:46,109][60935] Updated weights for policy 0, policy_version 28730 (0.0008) [2023-10-13 22:06:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 59146240. Throughput: 0: 1670.1, 1: 1717.4. Samples: 14798822. Policy #0 lag: (min: 6.0, avg: 6.7, max: 25.0) [2023-10-13 22:06:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:06:46,282][60934] Updated weights for policy 1, policy_version 29082 (0.0007) [2023-10-13 22:06:50,193][60935] Updated weights for policy 0, policy_version 28740 (0.0010) [2023-10-13 22:06:50,426][60934] Updated weights for policy 1, policy_version 29092 (0.0008) [2023-10-13 22:06:50,565][60935] Updated weights for policy 0, policy_version 28750 (0.0008) [2023-10-13 22:06:50,789][60934] Updated weights for policy 1, policy_version 29102 (0.0008) [2023-10-13 22:06:50,930][60935] Updated weights for policy 0, policy_version 28760 (0.0008) [2023-10-13 22:06:51,156][60934] Updated weights for policy 1, policy_version 29112 (0.0007) [2023-10-13 22:06:51,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 59244544. Throughput: 0: 1668.9, 1: 1717.6. Samples: 14819566. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-10-13 22:06:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:06:55,048][60935] Updated weights for policy 0, policy_version 28770 (0.0008) [2023-10-13 22:06:55,135][60934] Updated weights for policy 1, policy_version 29122 (0.0009) [2023-10-13 22:06:55,411][60935] Updated weights for policy 0, policy_version 28780 (0.0009) [2023-10-13 22:06:55,555][60934] Updated weights for policy 1, policy_version 29132 (0.0009) [2023-10-13 22:06:55,781][60935] Updated weights for policy 0, policy_version 28790 (0.0009) [2023-10-13 22:06:55,927][60934] Updated weights for policy 1, policy_version 29142 (0.0009) [2023-10-13 22:06:56,158][60935] Updated weights for policy 0, policy_version 28800 (0.0007) [2023-10-13 22:06:56,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 59310080. Throughput: 0: 1653.8, 1: 1702.6. Samples: 14838898. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-10-13 22:06:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:06:56,258][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000028800_29491200.pth... [2023-10-13 22:06:56,286][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000029152_29851648.pth... [2023-10-13 22:06:56,287][60934] Updated weights for policy 1, policy_version 29152 (0.0007) [2023-10-13 22:06:56,294][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000027232_27885568.pth [2023-10-13 22:06:56,315][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000027552_28213248.pth [2023-10-13 22:07:00,217][60935] Updated weights for policy 0, policy_version 28810 (0.0008) [2023-10-13 22:07:00,290][60934] Updated weights for policy 1, policy_version 29162 (0.0008) [2023-10-13 22:07:00,587][60935] Updated weights for policy 0, policy_version 28820 (0.0007) [2023-10-13 22:07:00,661][60934] Updated weights for policy 1, policy_version 29172 (0.0009) [2023-10-13 22:07:00,957][60935] Updated weights for policy 0, policy_version 28830 (0.0008) [2023-10-13 22:07:01,028][60934] Updated weights for policy 1, policy_version 29182 (0.0008) [2023-10-13 22:07:01,248][59943] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 59408384. Throughput: 0: 1676.8, 1: 1715.8. Samples: 14849444. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-10-13 22:07:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:07:05,055][60934] Updated weights for policy 1, policy_version 29192 (0.0010) [2023-10-13 22:07:05,194][60935] Updated weights for policy 0, policy_version 28840 (0.0009) [2023-10-13 22:07:05,419][60934] Updated weights for policy 1, policy_version 29202 (0.0007) [2023-10-13 22:07:05,572][60935] Updated weights for policy 0, policy_version 28850 (0.0008) [2023-10-13 22:07:05,791][60934] Updated weights for policy 1, policy_version 29212 (0.0009) [2023-10-13 22:07:05,942][60935] Updated weights for policy 0, policy_version 28860 (0.0008) [2023-10-13 22:07:06,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 59473920. Throughput: 0: 1673.4, 1: 1716.1. Samples: 14870016. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) [2023-10-13 22:07:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:07:09,654][60934] Updated weights for policy 1, policy_version 29222 (0.0007) [2023-10-13 22:07:10,023][60934] Updated weights for policy 1, policy_version 29232 (0.0007) [2023-10-13 22:07:10,126][60935] Updated weights for policy 0, policy_version 28870 (0.0008) [2023-10-13 22:07:10,394][60934] Updated weights for policy 1, policy_version 29242 (0.0007) [2023-10-13 22:07:10,492][60935] Updated weights for policy 0, policy_version 28880 (0.0009) [2023-10-13 22:07:10,875][60935] Updated weights for policy 0, policy_version 28890 (0.0009) [2023-10-13 22:07:11,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 59539456. Throughput: 0: 1650.1, 1: 1694.4. Samples: 14888760. Policy #0 lag: (min: 30.0, avg: 33.9, max: 62.0) [2023-10-13 22:07:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:07:14,383][60934] Updated weights for policy 1, policy_version 29252 (0.0009) [2023-10-13 22:07:14,738][60934] Updated weights for policy 1, policy_version 29262 (0.0008) [2023-10-13 22:07:14,925][60935] Updated weights for policy 0, policy_version 28900 (0.0008) [2023-10-13 22:07:15,111][60934] Updated weights for policy 1, policy_version 29272 (0.0009) [2023-10-13 22:07:15,295][60935] Updated weights for policy 0, policy_version 28910 (0.0009) [2023-10-13 22:07:15,672][60935] Updated weights for policy 0, policy_version 28920 (0.0009) [2023-10-13 22:07:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 59604992. Throughput: 0: 1672.2, 1: 1719.9. Samples: 14900334. Policy #0 lag: (min: 30.0, avg: 33.9, max: 62.0) [2023-10-13 22:07:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:07:19,056][60934] Updated weights for policy 1, policy_version 29282 (0.0008) [2023-10-13 22:07:19,424][60934] Updated weights for policy 1, policy_version 29292 (0.0007) [2023-10-13 22:07:19,737][60935] Updated weights for policy 0, policy_version 28930 (0.0008) [2023-10-13 22:07:19,786][60934] Updated weights for policy 1, policy_version 29302 (0.0007) [2023-10-13 22:07:20,117][60935] Updated weights for policy 0, policy_version 28940 (0.0007) [2023-10-13 22:07:20,145][60934] Updated weights for policy 1, policy_version 29312 (0.0008) [2023-10-13 22:07:20,489][60935] Updated weights for policy 0, policy_version 28950 (0.0010) [2023-10-13 22:07:20,859][60935] Updated weights for policy 0, policy_version 28960 (0.0008) [2023-10-13 22:07:21,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 59670528. Throughput: 0: 1668.1, 1: 1707.7. Samples: 14920546. Policy #0 lag: (min: 30.0, avg: 33.9, max: 62.0) [2023-10-13 22:07:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:07:24,119][60934] Updated weights for policy 1, policy_version 29322 (0.0009) [2023-10-13 22:07:24,487][60934] Updated weights for policy 1, policy_version 29332 (0.0007) [2023-10-13 22:07:24,853][60934] Updated weights for policy 1, policy_version 29342 (0.0008) [2023-10-13 22:07:24,951][60935] Updated weights for policy 0, policy_version 28970 (0.0008) [2023-10-13 22:07:25,330][60935] Updated weights for policy 0, policy_version 28980 (0.0009) [2023-10-13 22:07:25,705][60935] Updated weights for policy 0, policy_version 28990 (0.0010) [2023-10-13 22:07:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 59736064. Throughput: 0: 1659.3, 1: 1685.5. Samples: 14939610. Policy #0 lag: (min: 30.0, avg: 33.9, max: 62.0) [2023-10-13 22:07:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:07:28,866][60934] Updated weights for policy 1, policy_version 29352 (0.0007) [2023-10-13 22:07:29,240][60934] Updated weights for policy 1, policy_version 29362 (0.0007) [2023-10-13 22:07:29,599][60934] Updated weights for policy 1, policy_version 29372 (0.0007) [2023-10-13 22:07:29,626][60935] Updated weights for policy 0, policy_version 29000 (0.0009) [2023-10-13 22:07:29,987][60935] Updated weights for policy 0, policy_version 29010 (0.0009) [2023-10-13 22:07:30,358][60935] Updated weights for policy 0, policy_version 29020 (0.0008) [2023-10-13 22:07:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 59801600. Throughput: 0: 1674.0, 1: 1712.8. Samples: 14951228. Policy #0 lag: (min: 30.0, avg: 33.9, max: 62.0) [2023-10-13 22:07:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:07:33,648][60934] Updated weights for policy 1, policy_version 29382 (0.0007) [2023-10-13 22:07:34,016][60934] Updated weights for policy 1, policy_version 29392 (0.0007) [2023-10-13 22:07:34,382][60934] Updated weights for policy 1, policy_version 29402 (0.0008) [2023-10-13 22:07:34,477][60935] Updated weights for policy 0, policy_version 29030 (0.0008) [2023-10-13 22:07:34,853][60935] Updated weights for policy 0, policy_version 29040 (0.0010) [2023-10-13 22:07:35,218][60935] Updated weights for policy 0, policy_version 29050 (0.0009) [2023-10-13 22:07:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 59867136. Throughput: 0: 1668.3, 1: 1688.0. Samples: 14970602. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 22:07:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:07:38,414][60934] Updated weights for policy 1, policy_version 29412 (0.0009) [2023-10-13 22:07:38,782][60934] Updated weights for policy 1, policy_version 29422 (0.0010) [2023-10-13 22:07:39,157][60934] Updated weights for policy 1, policy_version 29432 (0.0009) [2023-10-13 22:07:39,255][60935] Updated weights for policy 0, policy_version 29060 (0.0009) [2023-10-13 22:07:39,635][60935] Updated weights for policy 0, policy_version 29070 (0.0008) [2023-10-13 22:07:39,999][60935] Updated weights for policy 0, policy_version 29080 (0.0007) [2023-10-13 22:07:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 59932672. Throughput: 0: 1672.7, 1: 1702.6. Samples: 14990786. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 22:07:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:07:43,286][60934] Updated weights for policy 1, policy_version 29442 (0.0008) [2023-10-13 22:07:43,674][60934] Updated weights for policy 1, policy_version 29452 (0.0010) [2023-10-13 22:07:43,960][60935] Updated weights for policy 0, policy_version 29090 (0.0009) [2023-10-13 22:07:44,037][60934] Updated weights for policy 1, policy_version 29462 (0.0008) [2023-10-13 22:07:44,333][60935] Updated weights for policy 0, policy_version 29100 (0.0007) [2023-10-13 22:07:44,398][60934] Updated weights for policy 1, policy_version 29472 (0.0010) [2023-10-13 22:07:44,691][60935] Updated weights for policy 0, policy_version 29110 (0.0010) [2023-10-13 22:07:45,066][60935] Updated weights for policy 0, policy_version 29120 (0.0008) [2023-10-13 22:07:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 59998208. Throughput: 0: 1681.7, 1: 1705.1. Samples: 15001848. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 22:07:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:07:48,392][60934] Updated weights for policy 1, policy_version 29482 (0.0009) [2023-10-13 22:07:48,753][60934] Updated weights for policy 1, policy_version 29492 (0.0007) [2023-10-13 22:07:48,896][60935] Updated weights for policy 0, policy_version 29130 (0.0007) [2023-10-13 22:07:49,114][60934] Updated weights for policy 1, policy_version 29502 (0.0007) [2023-10-13 22:07:49,261][60935] Updated weights for policy 0, policy_version 29140 (0.0008) [2023-10-13 22:07:49,637][60935] Updated weights for policy 0, policy_version 29150 (0.0009) [2023-10-13 22:07:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 60063744. Throughput: 0: 1660.5, 1: 1685.0. Samples: 15020564. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 22:07:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:07:53,236][60934] Updated weights for policy 1, policy_version 29512 (0.0008) [2023-10-13 22:07:53,607][60934] Updated weights for policy 1, policy_version 29522 (0.0008) [2023-10-13 22:07:53,705][60935] Updated weights for policy 0, policy_version 29160 (0.0009) [2023-10-13 22:07:53,965][60934] Updated weights for policy 1, policy_version 29532 (0.0007) [2023-10-13 22:07:54,078][60935] Updated weights for policy 0, policy_version 29170 (0.0008) [2023-10-13 22:07:54,443][60935] Updated weights for policy 0, policy_version 29180 (0.0009) [2023-10-13 22:07:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 60129280. Throughput: 0: 1685.1, 1: 1708.2. Samples: 15041456. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 22:07:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:07:57,960][60934] Updated weights for policy 1, policy_version 29542 (0.0010) [2023-10-13 22:07:58,329][60934] Updated weights for policy 1, policy_version 29552 (0.0010) [2023-10-13 22:07:58,565][60935] Updated weights for policy 0, policy_version 29190 (0.0008) [2023-10-13 22:07:58,689][60934] Updated weights for policy 1, policy_version 29562 (0.0009) [2023-10-13 22:07:58,929][60935] Updated weights for policy 0, policy_version 29200 (0.0008) [2023-10-13 22:07:59,300][60935] Updated weights for policy 0, policy_version 29210 (0.0008) [2023-10-13 22:08:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 60194816. Throughput: 0: 1677.1, 1: 1689.2. Samples: 15051816. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 22:08:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:08:02,763][60934] Updated weights for policy 1, policy_version 29572 (0.0009) [2023-10-13 22:08:03,129][60934] Updated weights for policy 1, policy_version 29582 (0.0008) [2023-10-13 22:08:03,365][60935] Updated weights for policy 0, policy_version 29220 (0.0009) [2023-10-13 22:08:03,494][60934] Updated weights for policy 1, policy_version 29592 (0.0007) [2023-10-13 22:08:03,738][60935] Updated weights for policy 0, policy_version 29230 (0.0010) [2023-10-13 22:08:04,104][60935] Updated weights for policy 0, policy_version 29240 (0.0008) [2023-10-13 22:08:06,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 60260352. Throughput: 0: 1663.4, 1: 1689.9. Samples: 15071444. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 22:08:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:08:07,337][60934] Updated weights for policy 1, policy_version 29602 (0.0008) [2023-10-13 22:08:07,702][60934] Updated weights for policy 1, policy_version 29612 (0.0007) [2023-10-13 22:08:08,074][60934] Updated weights for policy 1, policy_version 29622 (0.0008) [2023-10-13 22:08:08,304][60935] Updated weights for policy 0, policy_version 29250 (0.0009) [2023-10-13 22:08:08,432][60934] Updated weights for policy 1, policy_version 29632 (0.0008) [2023-10-13 22:08:08,682][60935] Updated weights for policy 0, policy_version 29260 (0.0009) [2023-10-13 22:08:09,052][60935] Updated weights for policy 0, policy_version 29270 (0.0008) [2023-10-13 22:08:09,420][60935] Updated weights for policy 0, policy_version 29280 (0.0009) [2023-10-13 22:08:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 60325888. Throughput: 0: 1686.5, 1: 1709.2. Samples: 15092412. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 22:08:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:08:12,430][60934] Updated weights for policy 1, policy_version 29642 (0.0010) [2023-10-13 22:08:12,800][60934] Updated weights for policy 1, policy_version 29652 (0.0010) [2023-10-13 22:08:13,166][60934] Updated weights for policy 1, policy_version 29662 (0.0007) [2023-10-13 22:08:13,637][60935] Updated weights for policy 0, policy_version 29290 (0.0009) [2023-10-13 22:08:13,999][60935] Updated weights for policy 0, policy_version 29300 (0.0010) [2023-10-13 22:08:14,375][60935] Updated weights for policy 0, policy_version 29310 (0.0010) [2023-10-13 22:08:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 60391424. Throughput: 0: 1674.7, 1: 1681.3. Samples: 15102250. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 22:08:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:08:17,041][60934] Updated weights for policy 1, policy_version 29672 (0.0008) [2023-10-13 22:08:17,407][60934] Updated weights for policy 1, policy_version 29682 (0.0009) [2023-10-13 22:08:17,773][60934] Updated weights for policy 1, policy_version 29692 (0.0008) [2023-10-13 22:08:18,553][60935] Updated weights for policy 0, policy_version 29320 (0.0010) [2023-10-13 22:08:18,926][60935] Updated weights for policy 0, policy_version 29330 (0.0008) [2023-10-13 22:08:19,295][60935] Updated weights for policy 0, policy_version 29340 (0.0008) [2023-10-13 22:08:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 60456960. Throughput: 0: 1665.9, 1: 1714.1. Samples: 15122700. Policy #0 lag: (min: 31.0, avg: 45.5, max: 63.0) [2023-10-13 22:08:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:08:21,763][60934] Updated weights for policy 1, policy_version 29702 (0.0008) [2023-10-13 22:08:22,115][60934] Updated weights for policy 1, policy_version 29712 (0.0008) [2023-10-13 22:08:22,477][60934] Updated weights for policy 1, policy_version 29722 (0.0007) [2023-10-13 22:08:23,224][60935] Updated weights for policy 0, policy_version 29350 (0.0008) [2023-10-13 22:08:23,587][60935] Updated weights for policy 0, policy_version 29360 (0.0008) [2023-10-13 22:08:23,964][60935] Updated weights for policy 0, policy_version 29370 (0.0009) [2023-10-13 22:08:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 60522496. Throughput: 0: 1680.6, 1: 1711.9. Samples: 15143450. Policy #0 lag: (min: 31.0, avg: 45.5, max: 63.0) [2023-10-13 22:08:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:08:26,429][60934] Updated weights for policy 1, policy_version 29732 (0.0008) [2023-10-13 22:08:26,797][60934] Updated weights for policy 1, policy_version 29742 (0.0009) [2023-10-13 22:08:27,155][60934] Updated weights for policy 1, policy_version 29752 (0.0009) [2023-10-13 22:08:28,114][60935] Updated weights for policy 0, policy_version 29380 (0.0008) [2023-10-13 22:08:28,481][60935] Updated weights for policy 0, policy_version 29390 (0.0010) [2023-10-13 22:08:28,854][60935] Updated weights for policy 0, policy_version 29400 (0.0009) [2023-10-13 22:08:31,213][60934] Updated weights for policy 1, policy_version 29762 (0.0009) [2023-10-13 22:08:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 60588032. Throughput: 0: 1662.4, 1: 1699.3. Samples: 15153122. Policy #0 lag: (min: 31.0, avg: 45.5, max: 63.0) [2023-10-13 22:08:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:08:31,597][60934] Updated weights for policy 1, policy_version 29772 (0.0008) [2023-10-13 22:08:31,960][60934] Updated weights for policy 1, policy_version 29782 (0.0010) [2023-10-13 22:08:32,333][60934] Updated weights for policy 1, policy_version 29792 (0.0008) [2023-10-13 22:08:33,017][60935] Updated weights for policy 0, policy_version 29410 (0.0008) [2023-10-13 22:08:33,386][60935] Updated weights for policy 0, policy_version 29420 (0.0007) [2023-10-13 22:08:33,760][60935] Updated weights for policy 0, policy_version 29430 (0.0011) [2023-10-13 22:08:34,121][60935] Updated weights for policy 0, policy_version 29440 (0.0009) [2023-10-13 22:08:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 60653568. Throughput: 0: 1680.7, 1: 1722.0. Samples: 15173686. Policy #0 lag: (min: 31.0, avg: 45.5, max: 63.0) [2023-10-13 22:08:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:08:36,400][60934] Updated weights for policy 1, policy_version 29802 (0.0009) [2023-10-13 22:08:36,767][60934] Updated weights for policy 1, policy_version 29812 (0.0011) [2023-10-13 22:08:37,144][60934] Updated weights for policy 1, policy_version 29822 (0.0009) [2023-10-13 22:08:38,208][60935] Updated weights for policy 0, policy_version 29450 (0.0008) [2023-10-13 22:08:38,582][60935] Updated weights for policy 0, policy_version 29460 (0.0008) [2023-10-13 22:08:38,952][60935] Updated weights for policy 0, policy_version 29470 (0.0010) [2023-10-13 22:08:41,147][60934] Updated weights for policy 1, policy_version 29832 (0.0008) [2023-10-13 22:08:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 60719104. Throughput: 0: 1679.2, 1: 1719.9. Samples: 15194418. Policy #0 lag: (min: 31.0, avg: 45.5, max: 63.0) [2023-10-13 22:08:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:08:41,513][60934] Updated weights for policy 1, policy_version 29842 (0.0007) [2023-10-13 22:08:41,881][60934] Updated weights for policy 1, policy_version 29852 (0.0008) [2023-10-13 22:08:42,978][60935] Updated weights for policy 0, policy_version 29480 (0.0011) [2023-10-13 22:08:43,345][60935] Updated weights for policy 0, policy_version 29490 (0.0010) [2023-10-13 22:08:43,725][60935] Updated weights for policy 0, policy_version 29500 (0.0011) [2023-10-13 22:08:45,719][60934] Updated weights for policy 1, policy_version 29862 (0.0009) [2023-10-13 22:08:46,088][60934] Updated weights for policy 1, policy_version 29872 (0.0008) [2023-10-13 22:08:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 60784640. Throughput: 0: 1664.3, 1: 1713.7. Samples: 15203826. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-13 22:08:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:08:46,455][60934] Updated weights for policy 1, policy_version 29882 (0.0007) [2023-10-13 22:08:47,837][60935] Updated weights for policy 0, policy_version 29510 (0.0010) [2023-10-13 22:08:48,204][60935] Updated weights for policy 0, policy_version 29520 (0.0008) [2023-10-13 22:08:48,573][60935] Updated weights for policy 0, policy_version 29530 (0.0009) [2023-10-13 22:08:50,500][60934] Updated weights for policy 1, policy_version 29892 (0.0008) [2023-10-13 22:08:50,866][60934] Updated weights for policy 1, policy_version 29902 (0.0008) [2023-10-13 22:08:51,237][60934] Updated weights for policy 1, policy_version 29912 (0.0008) [2023-10-13 22:08:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 60850176. Throughput: 0: 1673.4, 1: 1728.6. Samples: 15224532. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-13 22:08:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:08:52,726][60935] Updated weights for policy 0, policy_version 29540 (0.0008) [2023-10-13 22:08:53,097][60935] Updated weights for policy 0, policy_version 29550 (0.0009) [2023-10-13 22:08:53,464][60935] Updated weights for policy 0, policy_version 29560 (0.0009) [2023-10-13 22:08:55,139][60934] Updated weights for policy 1, policy_version 29922 (0.0007) [2023-10-13 22:08:55,505][60934] Updated weights for policy 1, policy_version 29932 (0.0009) [2023-10-13 22:08:55,876][60934] Updated weights for policy 1, policy_version 29942 (0.0009) [2023-10-13 22:08:56,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 60948480. Throughput: 0: 1674.4, 1: 1713.7. Samples: 15244878. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-13 22:08:56,249][60934] Updated weights for policy 1, policy_version 29952 (0.0007) [2023-10-13 22:08:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:08:56,259][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000029952_30670848.pth... [2023-10-13 22:08:56,259][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000029568_30277632.pth... [2023-10-13 22:08:56,290][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000028352_29032448.pth [2023-10-13 22:08:56,290][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000028000_28672000.pth [2023-10-13 22:08:57,596][60935] Updated weights for policy 0, policy_version 29570 (0.0009) [2023-10-13 22:08:57,962][60935] Updated weights for policy 0, policy_version 29580 (0.0008) [2023-10-13 22:08:58,336][60935] Updated weights for policy 0, policy_version 29590 (0.0008) [2023-10-13 22:08:58,704][60935] Updated weights for policy 0, policy_version 29600 (0.0010) [2023-10-13 22:09:00,210][60934] Updated weights for policy 1, policy_version 29962 (0.0007) [2023-10-13 22:09:00,580][60934] Updated weights for policy 1, policy_version 29972 (0.0007) [2023-10-13 22:09:00,950][60934] Updated weights for policy 1, policy_version 29982 (0.0008) [2023-10-13 22:09:01,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 61014016. Throughput: 0: 1658.2, 1: 1729.4. Samples: 15254694. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-13 22:09:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:09:02,813][60935] Updated weights for policy 0, policy_version 29610 (0.0008) [2023-10-13 22:09:03,180][60935] Updated weights for policy 0, policy_version 29620 (0.0009) [2023-10-13 22:09:03,552][60935] Updated weights for policy 0, policy_version 29630 (0.0009) [2023-10-13 22:09:05,039][60934] Updated weights for policy 1, policy_version 29992 (0.0008) [2023-10-13 22:09:05,409][60934] Updated weights for policy 1, policy_version 30002 (0.0009) [2023-10-13 22:09:05,780][60934] Updated weights for policy 1, policy_version 30012 (0.0009) [2023-10-13 22:09:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 61079552. Throughput: 0: 1675.5, 1: 1721.0. Samples: 15275542. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-13 22:09:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:09:07,521][60935] Updated weights for policy 0, policy_version 29640 (0.0009) [2023-10-13 22:09:07,889][60935] Updated weights for policy 0, policy_version 29650 (0.0011) [2023-10-13 22:09:08,261][60935] Updated weights for policy 0, policy_version 29660 (0.0011) [2023-10-13 22:09:09,575][60934] Updated weights for policy 1, policy_version 30022 (0.0009) [2023-10-13 22:09:09,937][60934] Updated weights for policy 1, policy_version 30032 (0.0008) [2023-10-13 22:09:10,311][60934] Updated weights for policy 1, policy_version 30042 (0.0007) [2023-10-13 22:09:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 61145088. Throughput: 0: 1679.6, 1: 1694.9. Samples: 15295302. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-13 22:09:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:09:12,266][60935] Updated weights for policy 0, policy_version 29670 (0.0010) [2023-10-13 22:09:12,646][60935] Updated weights for policy 0, policy_version 29680 (0.0009) [2023-10-13 22:09:13,022][60935] Updated weights for policy 0, policy_version 29690 (0.0009) [2023-10-13 22:09:14,352][60934] Updated weights for policy 1, policy_version 30052 (0.0007) [2023-10-13 22:09:14,719][60934] Updated weights for policy 1, policy_version 30062 (0.0007) [2023-10-13 22:09:15,084][60934] Updated weights for policy 1, policy_version 30072 (0.0009) [2023-10-13 22:09:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 61210624. Throughput: 0: 1668.4, 1: 1725.6. Samples: 15305854. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-13 22:09:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:09:17,130][60935] Updated weights for policy 0, policy_version 29700 (0.0009) [2023-10-13 22:09:17,503][60935] Updated weights for policy 0, policy_version 29710 (0.0008) [2023-10-13 22:09:17,864][60935] Updated weights for policy 0, policy_version 29720 (0.0010) [2023-10-13 22:09:19,022][60934] Updated weights for policy 1, policy_version 30082 (0.0008) [2023-10-13 22:09:19,429][60934] Updated weights for policy 1, policy_version 30092 (0.0009) [2023-10-13 22:09:19,798][60934] Updated weights for policy 1, policy_version 30102 (0.0007) [2023-10-13 22:09:20,167][60934] Updated weights for policy 1, policy_version 30112 (0.0008) [2023-10-13 22:09:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 61276160. Throughput: 0: 1673.9, 1: 1708.0. Samples: 15325870. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-13 22:09:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:09:21,903][60935] Updated weights for policy 0, policy_version 29730 (0.0009) [2023-10-13 22:09:22,270][60935] Updated weights for policy 0, policy_version 29740 (0.0009) [2023-10-13 22:09:22,641][60935] Updated weights for policy 0, policy_version 29750 (0.0009) [2023-10-13 22:09:23,014][60935] Updated weights for policy 0, policy_version 29760 (0.0008) [2023-10-13 22:09:24,325][60934] Updated weights for policy 1, policy_version 30122 (0.0009) [2023-10-13 22:09:24,696][60934] Updated weights for policy 1, policy_version 30132 (0.0008) [2023-10-13 22:09:25,061][60934] Updated weights for policy 1, policy_version 30142 (0.0008) [2023-10-13 22:09:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 61341696. Throughput: 0: 1671.3, 1: 1691.2. Samples: 15345734. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-13 22:09:26,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:09:27,279][60935] Updated weights for policy 0, policy_version 29770 (0.0010) [2023-10-13 22:09:27,659][60935] Updated weights for policy 0, policy_version 29780 (0.0009) [2023-10-13 22:09:28,031][60935] Updated weights for policy 0, policy_version 29790 (0.0008) [2023-10-13 22:09:29,172][60934] Updated weights for policy 1, policy_version 30152 (0.0008) [2023-10-13 22:09:29,530][60934] Updated weights for policy 1, policy_version 30162 (0.0008) [2023-10-13 22:09:29,901][60934] Updated weights for policy 1, policy_version 30172 (0.0010) [2023-10-13 22:09:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 61407232. Throughput: 0: 1667.6, 1: 1716.4. Samples: 15356104. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-13 22:09:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:09:31,964][60935] Updated weights for policy 0, policy_version 29800 (0.0009) [2023-10-13 22:09:32,338][60935] Updated weights for policy 0, policy_version 29810 (0.0007) [2023-10-13 22:09:32,720][60935] Updated weights for policy 0, policy_version 29820 (0.0009) [2023-10-13 22:09:33,930][60934] Updated weights for policy 1, policy_version 30182 (0.0007) [2023-10-13 22:09:34,308][60934] Updated weights for policy 1, policy_version 30192 (0.0010) [2023-10-13 22:09:34,683][60934] Updated weights for policy 1, policy_version 30202 (0.0009) [2023-10-13 22:09:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 61472768. Throughput: 0: 1676.0, 1: 1688.3. Samples: 15375928. Policy #0 lag: (min: 1.0, avg: 2.0, max: 22.0) [2023-10-13 22:09:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:09:36,961][60935] Updated weights for policy 0, policy_version 29830 (0.0008) [2023-10-13 22:09:37,338][60935] Updated weights for policy 0, policy_version 29840 (0.0008) [2023-10-13 22:09:37,706][60935] Updated weights for policy 0, policy_version 29850 (0.0007) [2023-10-13 22:09:38,579][60934] Updated weights for policy 1, policy_version 30212 (0.0009) [2023-10-13 22:09:38,948][60934] Updated weights for policy 1, policy_version 30222 (0.0010) [2023-10-13 22:09:39,323][60934] Updated weights for policy 1, policy_version 30232 (0.0011) [2023-10-13 22:09:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 61538304. Throughput: 0: 1674.0, 1: 1693.5. Samples: 15396418. Policy #0 lag: (min: 1.0, avg: 2.0, max: 22.0) [2023-10-13 22:09:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:09:41,858][60935] Updated weights for policy 0, policy_version 29860 (0.0008) [2023-10-13 22:09:42,236][60935] Updated weights for policy 0, policy_version 29870 (0.0009) [2023-10-13 22:09:42,606][60935] Updated weights for policy 0, policy_version 29880 (0.0009) [2023-10-13 22:09:43,184][60934] Updated weights for policy 1, policy_version 30242 (0.0009) [2023-10-13 22:09:43,554][60934] Updated weights for policy 1, policy_version 30252 (0.0007) [2023-10-13 22:09:43,926][60934] Updated weights for policy 1, policy_version 30262 (0.0007) [2023-10-13 22:09:44,291][60934] Updated weights for policy 1, policy_version 30272 (0.0009) [2023-10-13 22:09:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 61603840. Throughput: 0: 1672.2, 1: 1700.0. Samples: 15406442. Policy #0 lag: (min: 1.0, avg: 2.0, max: 22.0) [2023-10-13 22:09:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:09:46,762][60935] Updated weights for policy 0, policy_version 29890 (0.0009) [2023-10-13 22:09:47,130][60935] Updated weights for policy 0, policy_version 29900 (0.0012) [2023-10-13 22:09:47,501][60935] Updated weights for policy 0, policy_version 29910 (0.0010) [2023-10-13 22:09:47,864][60935] Updated weights for policy 0, policy_version 29920 (0.0008) [2023-10-13 22:09:48,356][60934] Updated weights for policy 1, policy_version 30282 (0.0010) [2023-10-13 22:09:48,723][60934] Updated weights for policy 1, policy_version 30292 (0.0009) [2023-10-13 22:09:49,103][60934] Updated weights for policy 1, policy_version 30302 (0.0008) [2023-10-13 22:09:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 61669376. Throughput: 0: 1670.2, 1: 1681.2. Samples: 15426358. Policy #0 lag: (min: 1.0, avg: 2.0, max: 22.0) [2023-10-13 22:09:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:09:51,788][60935] Updated weights for policy 0, policy_version 29930 (0.0009) [2023-10-13 22:09:52,155][60935] Updated weights for policy 0, policy_version 29940 (0.0007) [2023-10-13 22:09:52,524][60935] Updated weights for policy 0, policy_version 29950 (0.0008) [2023-10-13 22:09:52,973][60934] Updated weights for policy 1, policy_version 30312 (0.0009) [2023-10-13 22:09:53,334][60934] Updated weights for policy 1, policy_version 30322 (0.0007) [2023-10-13 22:09:53,695][60934] Updated weights for policy 1, policy_version 30332 (0.0009) [2023-10-13 22:09:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 61734912. Throughput: 0: 1661.8, 1: 1711.8. Samples: 15447114. Policy #0 lag: (min: 20.0, avg: 27.5, max: 52.0) [2023-10-13 22:09:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:09:56,640][60935] Updated weights for policy 0, policy_version 29960 (0.0008) [2023-10-13 22:09:57,006][60935] Updated weights for policy 0, policy_version 29970 (0.0008) [2023-10-13 22:09:57,378][60935] Updated weights for policy 0, policy_version 29980 (0.0008) [2023-10-13 22:09:57,501][60934] Updated weights for policy 1, policy_version 30342 (0.0009) [2023-10-13 22:09:57,865][60934] Updated weights for policy 1, policy_version 30352 (0.0007) [2023-10-13 22:09:58,236][60934] Updated weights for policy 1, policy_version 30362 (0.0008) [2023-10-13 22:10:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 61800448. Throughput: 0: 1664.0, 1: 1685.3. Samples: 15456574. Policy #0 lag: (min: 20.0, avg: 27.5, max: 52.0) [2023-10-13 22:10:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:10:01,375][60935] Updated weights for policy 0, policy_version 29990 (0.0007) [2023-10-13 22:10:01,751][60935] Updated weights for policy 0, policy_version 30000 (0.0008) [2023-10-13 22:10:02,124][60935] Updated weights for policy 0, policy_version 30010 (0.0008) [2023-10-13 22:10:02,381][60934] Updated weights for policy 1, policy_version 30372 (0.0008) [2023-10-13 22:10:02,757][60934] Updated weights for policy 1, policy_version 30382 (0.0008) [2023-10-13 22:10:03,118][60934] Updated weights for policy 1, policy_version 30392 (0.0009) [2023-10-13 22:10:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.5). Total num frames: 61865984. Throughput: 0: 1662.3, 1: 1697.5. Samples: 15477062. Policy #0 lag: (min: 20.0, avg: 27.5, max: 52.0) [2023-10-13 22:10:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:10:06,329][60935] Updated weights for policy 0, policy_version 30020 (0.0008) [2023-10-13 22:10:06,703][60935] Updated weights for policy 0, policy_version 30030 (0.0008) [2023-10-13 22:10:07,068][60935] Updated weights for policy 0, policy_version 30040 (0.0009) [2023-10-13 22:10:07,184][60934] Updated weights for policy 1, policy_version 30402 (0.0007) [2023-10-13 22:10:07,594][60934] Updated weights for policy 1, policy_version 30412 (0.0007) [2023-10-13 22:10:07,960][60934] Updated weights for policy 1, policy_version 30422 (0.0007) [2023-10-13 22:10:08,329][60934] Updated weights for policy 1, policy_version 30432 (0.0009) [2023-10-13 22:10:11,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 61931520. Throughput: 0: 1665.1, 1: 1712.2. Samples: 15497710. Policy #0 lag: (min: 20.0, avg: 27.5, max: 52.0) [2023-10-13 22:10:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:10:11,319][60935] Updated weights for policy 0, policy_version 30050 (0.0007) [2023-10-13 22:10:11,692][60935] Updated weights for policy 0, policy_version 30060 (0.0007) [2023-10-13 22:10:12,069][60935] Updated weights for policy 0, policy_version 30070 (0.0011) [2023-10-13 22:10:12,354][60934] Updated weights for policy 1, policy_version 30442 (0.0009) [2023-10-13 22:10:12,429][60935] Updated weights for policy 0, policy_version 30080 (0.0009) [2023-10-13 22:10:12,722][60934] Updated weights for policy 1, policy_version 30452 (0.0010) [2023-10-13 22:10:13,090][60934] Updated weights for policy 1, policy_version 30462 (0.0010) [2023-10-13 22:10:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 61997056. Throughput: 0: 1669.2, 1: 1680.9. Samples: 15506858. Policy #0 lag: (min: 20.0, avg: 27.5, max: 52.0) [2023-10-13 22:10:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:10:16,643][60935] Updated weights for policy 0, policy_version 30090 (0.0009) [2023-10-13 22:10:17,018][60935] Updated weights for policy 0, policy_version 30100 (0.0009) [2023-10-13 22:10:17,229][60934] Updated weights for policy 1, policy_version 30472 (0.0008) [2023-10-13 22:10:17,380][60935] Updated weights for policy 0, policy_version 30110 (0.0008) [2023-10-13 22:10:17,598][60934] Updated weights for policy 1, policy_version 30482 (0.0009) [2023-10-13 22:10:17,967][60934] Updated weights for policy 1, policy_version 30492 (0.0008) [2023-10-13 22:10:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 62062592. Throughput: 0: 1669.8, 1: 1701.7. Samples: 15527646. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-13 22:10:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:10:21,256][60935] Updated weights for policy 0, policy_version 30120 (0.0008) [2023-10-13 22:10:21,624][60935] Updated weights for policy 0, policy_version 30130 (0.0009) [2023-10-13 22:10:21,941][60934] Updated weights for policy 1, policy_version 30502 (0.0008) [2023-10-13 22:10:21,992][60935] Updated weights for policy 0, policy_version 30140 (0.0007) [2023-10-13 22:10:22,311][60934] Updated weights for policy 1, policy_version 30512 (0.0008) [2023-10-13 22:10:22,676][60934] Updated weights for policy 1, policy_version 30522 (0.0008) [2023-10-13 22:10:26,160][60935] Updated weights for policy 0, policy_version 30150 (0.0007) [2023-10-13 22:10:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 62128128. Throughput: 0: 1669.5, 1: 1706.8. Samples: 15548354. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-13 22:10:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:10:26,531][60935] Updated weights for policy 0, policy_version 30160 (0.0008) [2023-10-13 22:10:26,708][60934] Updated weights for policy 1, policy_version 30532 (0.0008) [2023-10-13 22:10:26,906][60935] Updated weights for policy 0, policy_version 30170 (0.0010) [2023-10-13 22:10:27,062][60934] Updated weights for policy 1, policy_version 30542 (0.0008) [2023-10-13 22:10:27,431][60934] Updated weights for policy 1, policy_version 30552 (0.0010) [2023-10-13 22:10:30,948][60935] Updated weights for policy 0, policy_version 30180 (0.0008) [2023-10-13 22:10:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 62193664. Throughput: 0: 1670.4, 1: 1684.4. Samples: 15557404. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-13 22:10:31,249][59943] Avg episode reward: [(0, '-0.220'), (1, '-0.010')] [2023-10-13 22:10:31,314][60935] Updated weights for policy 0, policy_version 30190 (0.0009) [2023-10-13 22:10:31,459][60934] Updated weights for policy 1, policy_version 30562 (0.0008) [2023-10-13 22:10:31,683][60935] Updated weights for policy 0, policy_version 30200 (0.0010) [2023-10-13 22:10:31,816][60934] Updated weights for policy 1, policy_version 30572 (0.0008) [2023-10-13 22:10:32,186][60934] Updated weights for policy 1, policy_version 30582 (0.0009) [2023-10-13 22:10:32,557][60934] Updated weights for policy 1, policy_version 30592 (0.0007) [2023-10-13 22:10:35,861][60935] Updated weights for policy 0, policy_version 30210 (0.0010) [2023-10-13 22:10:36,235][60935] Updated weights for policy 0, policy_version 30220 (0.0009) [2023-10-13 22:10:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 62259200. Throughput: 0: 1670.4, 1: 1706.2. Samples: 15578306. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-13 22:10:36,248][59943] Avg episode reward: [(0, '-0.220'), (1, '-0.010')] [2023-10-13 22:10:36,419][60934] Updated weights for policy 1, policy_version 30602 (0.0009) [2023-10-13 22:10:36,608][60935] Updated weights for policy 0, policy_version 30230 (0.0008) [2023-10-13 22:10:36,785][60934] Updated weights for policy 1, policy_version 30612 (0.0009) [2023-10-13 22:10:36,969][60935] Updated weights for policy 0, policy_version 30240 (0.0009) [2023-10-13 22:10:37,150][60934] Updated weights for policy 1, policy_version 30622 (0.0010) [2023-10-13 22:10:41,047][60935] Updated weights for policy 0, policy_version 30250 (0.0008) [2023-10-13 22:10:41,148][60934] Updated weights for policy 1, policy_version 30632 (0.0007) [2023-10-13 22:10:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 62324736. Throughput: 0: 1671.5, 1: 1704.3. Samples: 15599024. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-13 22:10:41,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.010')] [2023-10-13 22:10:41,426][60935] Updated weights for policy 0, policy_version 30260 (0.0008) [2023-10-13 22:10:41,509][60934] Updated weights for policy 1, policy_version 30642 (0.0007) [2023-10-13 22:10:41,799][60935] Updated weights for policy 0, policy_version 30270 (0.0008) [2023-10-13 22:10:41,876][60934] Updated weights for policy 1, policy_version 30652 (0.0009) [2023-10-13 22:10:45,909][60934] Updated weights for policy 1, policy_version 30662 (0.0010) [2023-10-13 22:10:45,910][60935] Updated weights for policy 0, policy_version 30280 (0.0009) [2023-10-13 22:10:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 62390272. Throughput: 0: 1671.4, 1: 1696.8. Samples: 15608142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:10:46,249][59943] Avg episode reward: [(0, '-0.320'), (1, '-0.010')] [2023-10-13 22:10:46,277][60935] Updated weights for policy 0, policy_version 30290 (0.0008) [2023-10-13 22:10:46,277][60934] Updated weights for policy 1, policy_version 30672 (0.0008) [2023-10-13 22:10:46,652][60934] Updated weights for policy 1, policy_version 30682 (0.0008) [2023-10-13 22:10:46,655][60935] Updated weights for policy 0, policy_version 30300 (0.0009) [2023-10-13 22:10:50,687][60935] Updated weights for policy 0, policy_version 30310 (0.0008) [2023-10-13 22:10:50,837][60934] Updated weights for policy 1, policy_version 30692 (0.0009) [2023-10-13 22:10:51,066][60935] Updated weights for policy 0, policy_version 30320 (0.0007) [2023-10-13 22:10:51,211][60934] Updated weights for policy 1, policy_version 30702 (0.0007) [2023-10-13 22:10:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 62455808. Throughput: 0: 1672.2, 1: 1700.7. Samples: 15628840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:10:51,249][59943] Avg episode reward: [(0, '-0.320'), (1, '-0.020')] [2023-10-13 22:10:51,435][60935] Updated weights for policy 0, policy_version 30330 (0.0009) [2023-10-13 22:10:51,567][60934] Updated weights for policy 1, policy_version 30712 (0.0007) [2023-10-13 22:10:55,445][60935] Updated weights for policy 0, policy_version 30340 (0.0009) [2023-10-13 22:10:55,702][60934] Updated weights for policy 1, policy_version 30722 (0.0008) [2023-10-13 22:10:55,808][60935] Updated weights for policy 0, policy_version 30350 (0.0008) [2023-10-13 22:10:56,123][60934] Updated weights for policy 1, policy_version 30732 (0.0009) [2023-10-13 22:10:56,183][60935] Updated weights for policy 0, policy_version 30360 (0.0007) [2023-10-13 22:10:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 62521344. Throughput: 0: 1658.3, 1: 1701.7. Samples: 15648910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:10:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:10:56,474][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000030368_31096832.pth... [2023-10-13 22:10:56,491][60934] Updated weights for policy 1, policy_version 30742 (0.0007) [2023-10-13 22:10:56,502][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000028800_29491200.pth [2023-10-13 22:10:56,856][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000030752_31490048.pth... [2023-10-13 22:10:56,857][60934] Updated weights for policy 1, policy_version 30752 (0.0009) [2023-10-13 22:10:56,886][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000029152_29851648.pth [2023-10-13 22:11:00,370][60935] Updated weights for policy 0, policy_version 30370 (0.0008) [2023-10-13 22:11:00,695][60934] Updated weights for policy 1, policy_version 30762 (0.0009) [2023-10-13 22:11:00,765][60935] Updated weights for policy 0, policy_version 30380 (0.0008) [2023-10-13 22:11:01,058][60934] Updated weights for policy 1, policy_version 30772 (0.0007) [2023-10-13 22:11:01,130][60935] Updated weights for policy 0, policy_version 30390 (0.0007) [2023-10-13 22:11:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 62586880. Throughput: 0: 1668.1, 1: 1701.4. Samples: 15658488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:11:01,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.030')] [2023-10-13 22:11:01,433][60934] Updated weights for policy 1, policy_version 30782 (0.0009) [2023-10-13 22:11:01,512][60935] Updated weights for policy 0, policy_version 30400 (0.0009) [2023-10-13 22:11:05,485][60934] Updated weights for policy 1, policy_version 30792 (0.0008) [2023-10-13 22:11:05,697][60935] Updated weights for policy 0, policy_version 30410 (0.0009) [2023-10-13 22:11:05,850][60934] Updated weights for policy 1, policy_version 30802 (0.0007) [2023-10-13 22:11:06,061][60935] Updated weights for policy 0, policy_version 30420 (0.0010) [2023-10-13 22:11:06,208][60934] Updated weights for policy 1, policy_version 30812 (0.0008) [2023-10-13 22:11:06,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 62652416. Throughput: 0: 1659.5, 1: 1704.9. Samples: 15679046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:11:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:11:06,434][60935] Updated weights for policy 0, policy_version 30430 (0.0008) [2023-10-13 22:11:10,153][60934] Updated weights for policy 1, policy_version 30822 (0.0008) [2023-10-13 22:11:10,515][60934] Updated weights for policy 1, policy_version 30832 (0.0008) [2023-10-13 22:11:10,588][60935] Updated weights for policy 0, policy_version 30440 (0.0009) [2023-10-13 22:11:10,884][60934] Updated weights for policy 1, policy_version 30842 (0.0008) [2023-10-13 22:11:10,956][60935] Updated weights for policy 0, policy_version 30450 (0.0008) [2023-10-13 22:11:11,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 62750720. Throughput: 0: 1647.2, 1: 1688.9. Samples: 15698476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:11:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:11:11,327][60935] Updated weights for policy 0, policy_version 30460 (0.0008) [2023-10-13 22:11:14,965][60934] Updated weights for policy 1, policy_version 30852 (0.0010) [2023-10-13 22:11:15,322][60934] Updated weights for policy 1, policy_version 30862 (0.0010) [2023-10-13 22:11:15,325][60935] Updated weights for policy 0, policy_version 30470 (0.0008) [2023-10-13 22:11:15,687][60935] Updated weights for policy 0, policy_version 30480 (0.0009) [2023-10-13 22:11:15,696][60934] Updated weights for policy 1, policy_version 30872 (0.0008) [2023-10-13 22:11:16,068][60935] Updated weights for policy 0, policy_version 30490 (0.0010) [2023-10-13 22:11:16,248][59943] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 62816256. Throughput: 0: 1664.8, 1: 1704.3. Samples: 15709014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:11:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:11:19,732][60934] Updated weights for policy 1, policy_version 30882 (0.0007) [2023-10-13 22:11:20,043][60935] Updated weights for policy 0, policy_version 30500 (0.0010) [2023-10-13 22:11:20,087][60934] Updated weights for policy 1, policy_version 30892 (0.0008) [2023-10-13 22:11:20,411][60935] Updated weights for policy 0, policy_version 30510 (0.0009) [2023-10-13 22:11:20,456][60934] Updated weights for policy 1, policy_version 30902 (0.0008) [2023-10-13 22:11:20,783][60935] Updated weights for policy 0, policy_version 30520 (0.0008) [2023-10-13 22:11:20,825][60934] Updated weights for policy 1, policy_version 30912 (0.0009) [2023-10-13 22:11:21,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 62914560. Throughput: 0: 1663.3, 1: 1701.9. Samples: 15729740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:11:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 22:11:24,857][60934] Updated weights for policy 1, policy_version 30922 (0.0010) [2023-10-13 22:11:24,862][60935] Updated weights for policy 0, policy_version 30530 (0.0008) [2023-10-13 22:11:25,224][60934] Updated weights for policy 1, policy_version 30932 (0.0009) [2023-10-13 22:11:25,235][60935] Updated weights for policy 0, policy_version 30540 (0.0010) [2023-10-13 22:11:25,592][60934] Updated weights for policy 1, policy_version 30942 (0.0010) [2023-10-13 22:11:25,598][60935] Updated weights for policy 0, policy_version 30550 (0.0007) [2023-10-13 22:11:25,968][60935] Updated weights for policy 0, policy_version 30560 (0.0008) [2023-10-13 22:11:26,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 62980096. Throughput: 0: 1642.6, 1: 1672.4. Samples: 15748198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:11:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-13 22:11:29,617][60934] Updated weights for policy 1, policy_version 30952 (0.0008) [2023-10-13 22:11:29,980][60934] Updated weights for policy 1, policy_version 30962 (0.0008) [2023-10-13 22:11:30,136][60935] Updated weights for policy 0, policy_version 30570 (0.0009) [2023-10-13 22:11:30,347][60934] Updated weights for policy 1, policy_version 30972 (0.0010) [2023-10-13 22:11:30,502][60935] Updated weights for policy 0, policy_version 30580 (0.0010) [2023-10-13 22:11:30,875][60935] Updated weights for policy 0, policy_version 30590 (0.0009) [2023-10-13 22:11:31,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 63045632. Throughput: 0: 1660.6, 1: 1702.0. Samples: 15759462. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-13 22:11:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-13 22:11:34,498][60934] Updated weights for policy 1, policy_version 30982 (0.0009) [2023-10-13 22:11:34,864][60934] Updated weights for policy 1, policy_version 30992 (0.0007) [2023-10-13 22:11:35,093][60935] Updated weights for policy 0, policy_version 30600 (0.0009) [2023-10-13 22:11:35,223][60934] Updated weights for policy 1, policy_version 31002 (0.0007) [2023-10-13 22:11:35,457][60935] Updated weights for policy 0, policy_version 30610 (0.0007) [2023-10-13 22:11:35,833][60935] Updated weights for policy 0, policy_version 30620 (0.0008) [2023-10-13 22:11:36,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 63111168. Throughput: 0: 1666.4, 1: 1689.0. Samples: 15779834. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-13 22:11:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-13 22:11:39,236][60934] Updated weights for policy 1, policy_version 31012 (0.0007) [2023-10-13 22:11:39,597][60934] Updated weights for policy 1, policy_version 31022 (0.0009) [2023-10-13 22:11:39,965][60934] Updated weights for policy 1, policy_version 31032 (0.0008) [2023-10-13 22:11:40,106][60935] Updated weights for policy 0, policy_version 30630 (0.0009) [2023-10-13 22:11:40,474][60935] Updated weights for policy 0, policy_version 30640 (0.0008) [2023-10-13 22:11:40,845][60935] Updated weights for policy 0, policy_version 30650 (0.0010) [2023-10-13 22:11:41,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 63176704. Throughput: 0: 1656.4, 1: 1672.5. Samples: 15798712. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-13 22:11:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-13 22:11:44,122][60934] Updated weights for policy 1, policy_version 31042 (0.0008) [2023-10-13 22:11:44,537][60934] Updated weights for policy 1, policy_version 31052 (0.0009) [2023-10-13 22:11:44,857][60935] Updated weights for policy 0, policy_version 30660 (0.0007) [2023-10-13 22:11:44,904][60934] Updated weights for policy 1, policy_version 31062 (0.0007) [2023-10-13 22:11:45,235][60935] Updated weights for policy 0, policy_version 30670 (0.0009) [2023-10-13 22:11:45,262][60934] Updated weights for policy 1, policy_version 31072 (0.0009) [2023-10-13 22:11:45,595][60935] Updated weights for policy 0, policy_version 30680 (0.0011) [2023-10-13 22:11:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 63242240. Throughput: 0: 1663.7, 1: 1706.1. Samples: 15810128. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-13 22:11:46,249][59943] Avg episode reward: [(0, '-0.180'), (1, '-0.020')] [2023-10-13 22:11:49,224][60934] Updated weights for policy 1, policy_version 31082 (0.0008) [2023-10-13 22:11:49,587][60934] Updated weights for policy 1, policy_version 31092 (0.0009) [2023-10-13 22:11:49,816][60935] Updated weights for policy 0, policy_version 30690 (0.0011) [2023-10-13 22:11:49,956][60934] Updated weights for policy 1, policy_version 31102 (0.0007) [2023-10-13 22:11:50,202][60935] Updated weights for policy 0, policy_version 30700 (0.0010) [2023-10-13 22:11:50,579][60935] Updated weights for policy 0, policy_version 30710 (0.0007) [2023-10-13 22:11:50,949][60935] Updated weights for policy 0, policy_version 30720 (0.0010) [2023-10-13 22:11:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 63307776. Throughput: 0: 1668.1, 1: 1684.3. Samples: 15829902. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-13 22:11:51,249][59943] Avg episode reward: [(0, '-0.180'), (1, '-0.010')] [2023-10-13 22:11:53,960][60934] Updated weights for policy 1, policy_version 31112 (0.0008) [2023-10-13 22:11:54,336][60934] Updated weights for policy 1, policy_version 31122 (0.0010) [2023-10-13 22:11:54,698][60934] Updated weights for policy 1, policy_version 31132 (0.0009) [2023-10-13 22:11:55,106][60935] Updated weights for policy 0, policy_version 30730 (0.0010) [2023-10-13 22:11:55,483][60935] Updated weights for policy 0, policy_version 30740 (0.0009) [2023-10-13 22:11:55,848][60935] Updated weights for policy 0, policy_version 30750 (0.0007) [2023-10-13 22:11:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 63373312. Throughput: 0: 1659.8, 1: 1686.9. Samples: 15849076. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-13 22:11:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:11:58,723][60934] Updated weights for policy 1, policy_version 31142 (0.0008) [2023-10-13 22:11:59,089][60934] Updated weights for policy 1, policy_version 31152 (0.0009) [2023-10-13 22:11:59,454][60934] Updated weights for policy 1, policy_version 31162 (0.0009) [2023-10-13 22:11:59,938][60935] Updated weights for policy 0, policy_version 30760 (0.0009) [2023-10-13 22:12:00,309][60935] Updated weights for policy 0, policy_version 30770 (0.0009) [2023-10-13 22:12:00,679][60935] Updated weights for policy 0, policy_version 30780 (0.0008) [2023-10-13 22:12:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 63438848. Throughput: 0: 1665.2, 1: 1698.7. Samples: 15860390. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-13 22:12:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:12:03,511][60934] Updated weights for policy 1, policy_version 31172 (0.0009) [2023-10-13 22:12:03,881][60934] Updated weights for policy 1, policy_version 31182 (0.0009) [2023-10-13 22:12:04,243][60934] Updated weights for policy 1, policy_version 31192 (0.0009) [2023-10-13 22:12:04,514][60935] Updated weights for policy 0, policy_version 30790 (0.0008) [2023-10-13 22:12:04,893][60935] Updated weights for policy 0, policy_version 30800 (0.0009) [2023-10-13 22:12:05,259][60935] Updated weights for policy 0, policy_version 30810 (0.0008) [2023-10-13 22:12:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 63504384. Throughput: 0: 1658.2, 1: 1670.8. Samples: 15879548. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-13 22:12:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:12:08,257][60934] Updated weights for policy 1, policy_version 31202 (0.0009) [2023-10-13 22:12:08,627][60934] Updated weights for policy 1, policy_version 31212 (0.0008) [2023-10-13 22:12:08,999][60934] Updated weights for policy 1, policy_version 31222 (0.0007) [2023-10-13 22:12:09,369][60934] Updated weights for policy 1, policy_version 31232 (0.0008) [2023-10-13 22:12:09,386][60935] Updated weights for policy 0, policy_version 30820 (0.0009) [2023-10-13 22:12:09,763][60935] Updated weights for policy 0, policy_version 30830 (0.0007) [2023-10-13 22:12:10,140][60935] Updated weights for policy 0, policy_version 30840 (0.0008) [2023-10-13 22:12:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 63569920. Throughput: 0: 1662.2, 1: 1698.3. Samples: 15899418. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-13 22:12:11,249][59943] Avg episode reward: [(0, '-0.120'), (1, '0.000')] [2023-10-13 22:12:13,491][60934] Updated weights for policy 1, policy_version 31242 (0.0009) [2023-10-13 22:12:13,854][60934] Updated weights for policy 1, policy_version 31252 (0.0011) [2023-10-13 22:12:14,233][60934] Updated weights for policy 1, policy_version 31262 (0.0008) [2023-10-13 22:12:14,344][60935] Updated weights for policy 0, policy_version 30850 (0.0009) [2023-10-13 22:12:14,706][60935] Updated weights for policy 0, policy_version 30860 (0.0008) [2023-10-13 22:12:15,074][60935] Updated weights for policy 0, policy_version 30870 (0.0007) [2023-10-13 22:12:15,451][60935] Updated weights for policy 0, policy_version 30880 (0.0009) [2023-10-13 22:12:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 63635456. Throughput: 0: 1671.0, 1: 1685.4. Samples: 15910500. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-13 22:12:16,249][59943] Avg episode reward: [(0, '-0.120'), (1, '0.000')] [2023-10-13 22:12:18,253][60934] Updated weights for policy 1, policy_version 31272 (0.0009) [2023-10-13 22:12:18,626][60934] Updated weights for policy 1, policy_version 31282 (0.0009) [2023-10-13 22:12:18,993][60934] Updated weights for policy 1, policy_version 31292 (0.0008) [2023-10-13 22:12:19,432][60935] Updated weights for policy 0, policy_version 30890 (0.0008) [2023-10-13 22:12:19,804][60935] Updated weights for policy 0, policy_version 30900 (0.0009) [2023-10-13 22:12:20,184][60935] Updated weights for policy 0, policy_version 30910 (0.0009) [2023-10-13 22:12:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 63700992. Throughput: 0: 1652.1, 1: 1681.3. Samples: 15929834. Policy #0 lag: (min: 26.0, avg: 29.0, max: 58.0) [2023-10-13 22:12:21,249][59943] Avg episode reward: [(0, '-0.120'), (1, '0.000')] [2023-10-13 22:12:23,027][60934] Updated weights for policy 1, policy_version 31302 (0.0008) [2023-10-13 22:12:23,390][60934] Updated weights for policy 1, policy_version 31312 (0.0008) [2023-10-13 22:12:23,751][60934] Updated weights for policy 1, policy_version 31322 (0.0008) [2023-10-13 22:12:24,307][60935] Updated weights for policy 0, policy_version 30920 (0.0008) [2023-10-13 22:12:24,672][60935] Updated weights for policy 0, policy_version 30930 (0.0007) [2023-10-13 22:12:25,047][60935] Updated weights for policy 0, policy_version 30940 (0.0009) [2023-10-13 22:12:26,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 63766528. Throughput: 0: 1666.0, 1: 1700.8. Samples: 15950220. Policy #0 lag: (min: 26.0, avg: 29.0, max: 58.0) [2023-10-13 22:12:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:12:27,856][60934] Updated weights for policy 1, policy_version 31332 (0.0010) [2023-10-13 22:12:28,219][60934] Updated weights for policy 1, policy_version 31342 (0.0010) [2023-10-13 22:12:28,582][60934] Updated weights for policy 1, policy_version 31352 (0.0008) [2023-10-13 22:12:29,001][60935] Updated weights for policy 0, policy_version 30950 (0.0010) [2023-10-13 22:12:29,380][60935] Updated weights for policy 0, policy_version 30960 (0.0008) [2023-10-13 22:12:29,756][60935] Updated weights for policy 0, policy_version 30970 (0.0008) [2023-10-13 22:12:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 63832064. Throughput: 0: 1678.1, 1: 1679.5. Samples: 15961218. Policy #0 lag: (min: 26.0, avg: 29.0, max: 58.0) [2023-10-13 22:12:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:12:32,556][60934] Updated weights for policy 1, policy_version 31362 (0.0009) [2023-10-13 22:12:32,926][60934] Updated weights for policy 1, policy_version 31372 (0.0007) [2023-10-13 22:12:33,287][60934] Updated weights for policy 1, policy_version 31382 (0.0007) [2023-10-13 22:12:33,655][60934] Updated weights for policy 1, policy_version 31392 (0.0007) [2023-10-13 22:12:33,804][60935] Updated weights for policy 0, policy_version 30980 (0.0008) [2023-10-13 22:12:34,173][60935] Updated weights for policy 0, policy_version 30990 (0.0007) [2023-10-13 22:12:34,538][60935] Updated weights for policy 0, policy_version 31000 (0.0008) [2023-10-13 22:12:36,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 63897600. Throughput: 0: 1655.6, 1: 1693.0. Samples: 15980590. Policy #0 lag: (min: 26.0, avg: 29.0, max: 58.0) [2023-10-13 22:12:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:12:37,727][60934] Updated weights for policy 1, policy_version 31402 (0.0009) [2023-10-13 22:12:38,102][60934] Updated weights for policy 1, policy_version 31412 (0.0009) [2023-10-13 22:12:38,474][60934] Updated weights for policy 1, policy_version 31422 (0.0009) [2023-10-13 22:12:38,608][60935] Updated weights for policy 0, policy_version 31010 (0.0009) [2023-10-13 22:12:39,016][60935] Updated weights for policy 0, policy_version 31020 (0.0008) [2023-10-13 22:12:39,391][60935] Updated weights for policy 0, policy_version 31030 (0.0008) [2023-10-13 22:12:39,758][60935] Updated weights for policy 0, policy_version 31040 (0.0008) [2023-10-13 22:12:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 63963136. Throughput: 0: 1677.7, 1: 1704.3. Samples: 16001266. Policy #0 lag: (min: 26.0, avg: 29.0, max: 58.0) [2023-10-13 22:12:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:12:42,492][60934] Updated weights for policy 1, policy_version 31432 (0.0009) [2023-10-13 22:12:42,855][60934] Updated weights for policy 1, policy_version 31442 (0.0009) [2023-10-13 22:12:43,224][60934] Updated weights for policy 1, policy_version 31452 (0.0008) [2023-10-13 22:12:43,803][60935] Updated weights for policy 0, policy_version 31050 (0.0007) [2023-10-13 22:12:44,179][60935] Updated weights for policy 0, policy_version 31060 (0.0007) [2023-10-13 22:12:44,546][60935] Updated weights for policy 0, policy_version 31070 (0.0007) [2023-10-13 22:12:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 64028672. Throughput: 0: 1679.2, 1: 1672.7. Samples: 16011224. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-13 22:12:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:12:47,027][60934] Updated weights for policy 1, policy_version 31462 (0.0008) [2023-10-13 22:12:47,388][60934] Updated weights for policy 1, policy_version 31472 (0.0009) [2023-10-13 22:12:47,763][60934] Updated weights for policy 1, policy_version 31482 (0.0010) [2023-10-13 22:12:48,540][60935] Updated weights for policy 0, policy_version 31080 (0.0009) [2023-10-13 22:12:48,922][60935] Updated weights for policy 0, policy_version 31090 (0.0010) [2023-10-13 22:12:49,296][60935] Updated weights for policy 0, policy_version 31100 (0.0007) [2023-10-13 22:12:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 64094208. Throughput: 0: 1667.9, 1: 1705.7. Samples: 16031358. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-13 22:12:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:12:51,827][60934] Updated weights for policy 1, policy_version 31492 (0.0010) [2023-10-13 22:12:52,200][60934] Updated weights for policy 1, policy_version 31502 (0.0010) [2023-10-13 22:12:52,565][60934] Updated weights for policy 1, policy_version 31512 (0.0010) [2023-10-13 22:12:53,383][60935] Updated weights for policy 0, policy_version 31110 (0.0008) [2023-10-13 22:12:53,752][60935] Updated weights for policy 0, policy_version 31120 (0.0007) [2023-10-13 22:12:54,130][60935] Updated weights for policy 0, policy_version 31130 (0.0008) [2023-10-13 22:12:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 64159744. Throughput: 0: 1688.1, 1: 1708.2. Samples: 16052254. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-13 22:12:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:12:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000031136_31883264.pth... [2023-10-13 22:12:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000031520_32276480.pth... [2023-10-13 22:12:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000029568_30277632.pth [2023-10-13 22:12:56,306][60695] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p0/milestones/checkpoint_000031136_31883264.pth [2023-10-13 22:12:56,307][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000029952_30670848.pth [2023-10-13 22:12:56,311][60828] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p1/milestones/checkpoint_000031520_32276480.pth [2023-10-13 22:12:56,528][60934] Updated weights for policy 1, policy_version 31522 (0.0009) [2023-10-13 22:12:56,885][60934] Updated weights for policy 1, policy_version 31532 (0.0010) [2023-10-13 22:12:57,256][60934] Updated weights for policy 1, policy_version 31542 (0.0009) [2023-10-13 22:12:57,615][60934] Updated weights for policy 1, policy_version 31552 (0.0007) [2023-10-13 22:12:58,256][60935] Updated weights for policy 0, policy_version 31140 (0.0011) [2023-10-13 22:12:58,636][60935] Updated weights for policy 0, policy_version 31150 (0.0010) [2023-10-13 22:12:59,011][60935] Updated weights for policy 0, policy_version 31160 (0.0010) [2023-10-13 22:13:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 64225280. Throughput: 0: 1674.0, 1: 1692.9. Samples: 16062008. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-13 22:13:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:13:01,618][60934] Updated weights for policy 1, policy_version 31562 (0.0007) [2023-10-13 22:13:01,980][60934] Updated weights for policy 1, policy_version 31572 (0.0009) [2023-10-13 22:13:02,353][60934] Updated weights for policy 1, policy_version 31582 (0.0008) [2023-10-13 22:13:02,964][60935] Updated weights for policy 0, policy_version 31170 (0.0010) [2023-10-13 22:13:03,330][60935] Updated weights for policy 0, policy_version 31180 (0.0009) [2023-10-13 22:13:03,709][60935] Updated weights for policy 0, policy_version 31190 (0.0007) [2023-10-13 22:13:04,083][60935] Updated weights for policy 0, policy_version 31200 (0.0008) [2023-10-13 22:13:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 64290816. Throughput: 0: 1682.8, 1: 1716.0. Samples: 16082782. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-13 22:13:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:13:06,290][60934] Updated weights for policy 1, policy_version 31592 (0.0007) [2023-10-13 22:13:06,664][60934] Updated weights for policy 1, policy_version 31602 (0.0008) [2023-10-13 22:13:07,027][60934] Updated weights for policy 1, policy_version 31612 (0.0008) [2023-10-13 22:13:08,119][60935] Updated weights for policy 0, policy_version 31210 (0.0009) [2023-10-13 22:13:08,494][60935] Updated weights for policy 0, policy_version 31220 (0.0010) [2023-10-13 22:13:08,870][60935] Updated weights for policy 0, policy_version 31230 (0.0008) [2023-10-13 22:13:10,950][60934] Updated weights for policy 1, policy_version 31622 (0.0010) [2023-10-13 22:13:11,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 64356352. Throughput: 0: 1695.9, 1: 1719.7. Samples: 16103922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:13:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:13:11,313][60934] Updated weights for policy 1, policy_version 31632 (0.0007) [2023-10-13 22:13:11,683][60934] Updated weights for policy 1, policy_version 31642 (0.0009) [2023-10-13 22:13:12,760][60935] Updated weights for policy 0, policy_version 31240 (0.0009) [2023-10-13 22:13:13,136][60935] Updated weights for policy 0, policy_version 31250 (0.0007) [2023-10-13 22:13:13,511][60935] Updated weights for policy 0, policy_version 31260 (0.0007) [2023-10-13 22:13:15,700][60934] Updated weights for policy 1, policy_version 31652 (0.0011) [2023-10-13 22:13:16,074][60934] Updated weights for policy 1, policy_version 31662 (0.0009) [2023-10-13 22:13:16,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 64421888. Throughput: 0: 1666.5, 1: 1710.3. Samples: 16113172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:13:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:13:16,433][60934] Updated weights for policy 1, policy_version 31672 (0.0008) [2023-10-13 22:13:17,439][60935] Updated weights for policy 0, policy_version 31270 (0.0007) [2023-10-13 22:13:17,815][60935] Updated weights for policy 0, policy_version 31280 (0.0008) [2023-10-13 22:13:18,187][60935] Updated weights for policy 0, policy_version 31290 (0.0008) [2023-10-13 22:13:20,444][60934] Updated weights for policy 1, policy_version 31682 (0.0007) [2023-10-13 22:13:20,820][60934] Updated weights for policy 1, policy_version 31692 (0.0007) [2023-10-13 22:13:21,186][60934] Updated weights for policy 1, policy_version 31702 (0.0008) [2023-10-13 22:13:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 64487424. Throughput: 0: 1698.5, 1: 1714.5. Samples: 16134172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:13:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:13:21,550][60934] Updated weights for policy 1, policy_version 31712 (0.0009) [2023-10-13 22:13:22,258][60935] Updated weights for policy 0, policy_version 31300 (0.0008) [2023-10-13 22:13:22,628][60935] Updated weights for policy 0, policy_version 31310 (0.0010) [2023-10-13 22:13:22,992][60935] Updated weights for policy 0, policy_version 31320 (0.0009) [2023-10-13 22:13:25,730][60934] Updated weights for policy 1, policy_version 31722 (0.0011) [2023-10-13 22:13:26,106][60934] Updated weights for policy 1, policy_version 31732 (0.0011) [2023-10-13 22:13:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 64552960. Throughput: 0: 1706.9, 1: 1711.9. Samples: 16155110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:13:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:13:26,472][60934] Updated weights for policy 1, policy_version 31742 (0.0008) [2023-10-13 22:13:26,823][60935] Updated weights for policy 0, policy_version 31330 (0.0008) [2023-10-13 22:13:27,198][60935] Updated weights for policy 0, policy_version 31340 (0.0010) [2023-10-13 22:13:27,571][60935] Updated weights for policy 0, policy_version 31350 (0.0010) [2023-10-13 22:13:27,948][60935] Updated weights for policy 0, policy_version 31360 (0.0010) [2023-10-13 22:13:30,136][60934] Updated weights for policy 1, policy_version 31752 (0.0007) [2023-10-13 22:13:30,509][60934] Updated weights for policy 1, policy_version 31762 (0.0009) [2023-10-13 22:13:30,875][60934] Updated weights for policy 1, policy_version 31772 (0.0010) [2023-10-13 22:13:31,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 64651264. Throughput: 0: 1683.7, 1: 1723.2. Samples: 16164538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:13:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:13:31,859][60935] Updated weights for policy 0, policy_version 31370 (0.0009) [2023-10-13 22:13:32,236][60935] Updated weights for policy 0, policy_version 31380 (0.0008) [2023-10-13 22:13:32,602][60935] Updated weights for policy 0, policy_version 31390 (0.0009) [2023-10-13 22:13:34,949][60934] Updated weights for policy 1, policy_version 31782 (0.0009) [2023-10-13 22:13:35,322][60934] Updated weights for policy 1, policy_version 31792 (0.0009) [2023-10-13 22:13:35,686][60934] Updated weights for policy 1, policy_version 31802 (0.0009) [2023-10-13 22:13:36,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 64716800. Throughput: 0: 1717.6, 1: 1721.1. Samples: 16186096. Policy #0 lag: (min: 18.0, avg: 18.5, max: 33.0) [2023-10-13 22:13:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:13:36,507][60935] Updated weights for policy 0, policy_version 31400 (0.0011) [2023-10-13 22:13:36,885][60935] Updated weights for policy 0, policy_version 31410 (0.0008) [2023-10-13 22:13:37,255][60935] Updated weights for policy 0, policy_version 31420 (0.0009) [2023-10-13 22:13:39,723][60934] Updated weights for policy 1, policy_version 31812 (0.0009) [2023-10-13 22:13:40,090][60934] Updated weights for policy 1, policy_version 31822 (0.0010) [2023-10-13 22:13:40,454][60934] Updated weights for policy 1, policy_version 31832 (0.0010) [2023-10-13 22:13:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 64782336. Throughput: 0: 1724.5, 1: 1694.9. Samples: 16206126. Policy #0 lag: (min: 18.0, avg: 18.5, max: 33.0) [2023-10-13 22:13:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:13:41,287][60935] Updated weights for policy 0, policy_version 31430 (0.0009) [2023-10-13 22:13:41,658][60935] Updated weights for policy 0, policy_version 31440 (0.0007) [2023-10-13 22:13:42,027][60935] Updated weights for policy 0, policy_version 31450 (0.0008) [2023-10-13 22:13:44,282][60934] Updated weights for policy 1, policy_version 31842 (0.0009) [2023-10-13 22:13:44,652][60934] Updated weights for policy 1, policy_version 31852 (0.0008) [2023-10-13 22:13:45,016][60934] Updated weights for policy 1, policy_version 31862 (0.0008) [2023-10-13 22:13:45,379][60934] Updated weights for policy 1, policy_version 31872 (0.0009) [2023-10-13 22:13:46,043][60935] Updated weights for policy 0, policy_version 31460 (0.0009) [2023-10-13 22:13:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 64847872. Throughput: 0: 1714.0, 1: 1718.2. Samples: 16216458. Policy #0 lag: (min: 18.0, avg: 18.5, max: 33.0) [2023-10-13 22:13:46,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.010')] [2023-10-13 22:13:46,421][60935] Updated weights for policy 0, policy_version 31470 (0.0008) [2023-10-13 22:13:46,793][60935] Updated weights for policy 0, policy_version 31480 (0.0009) [2023-10-13 22:13:49,481][60934] Updated weights for policy 1, policy_version 31882 (0.0007) [2023-10-13 22:13:49,839][60934] Updated weights for policy 1, policy_version 31892 (0.0007) [2023-10-13 22:13:50,207][60934] Updated weights for policy 1, policy_version 31902 (0.0009) [2023-10-13 22:13:50,772][60935] Updated weights for policy 0, policy_version 31490 (0.0010) [2023-10-13 22:13:51,138][60935] Updated weights for policy 0, policy_version 31500 (0.0007) [2023-10-13 22:13:51,248][59943] Fps is (10 sec: 13107.7, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 64913408. Throughput: 0: 1727.8, 1: 1700.3. Samples: 16237044. Policy #0 lag: (min: 18.0, avg: 18.5, max: 33.0) [2023-10-13 22:13:51,249][59943] Avg episode reward: [(0, '-0.190'), (1, '-0.010')] [2023-10-13 22:13:51,514][60935] Updated weights for policy 0, policy_version 31510 (0.0008) [2023-10-13 22:13:51,880][60935] Updated weights for policy 0, policy_version 31520 (0.0009) [2023-10-13 22:13:54,325][60934] Updated weights for policy 1, policy_version 31912 (0.0010) [2023-10-13 22:13:54,696][60934] Updated weights for policy 1, policy_version 31922 (0.0010) [2023-10-13 22:13:55,057][60934] Updated weights for policy 1, policy_version 31932 (0.0009) [2023-10-13 22:13:55,823][60935] Updated weights for policy 0, policy_version 31530 (0.0009) [2023-10-13 22:13:56,203][60935] Updated weights for policy 0, policy_version 31540 (0.0008) [2023-10-13 22:13:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 64978944. Throughput: 0: 1719.8, 1: 1680.1. Samples: 16256916. Policy #0 lag: (min: 18.0, avg: 18.5, max: 33.0) [2023-10-13 22:13:56,249][59943] Avg episode reward: [(0, '-0.190'), (1, '-0.010')] [2023-10-13 22:13:56,576][60935] Updated weights for policy 0, policy_version 31550 (0.0009) [2023-10-13 22:13:58,994][60934] Updated weights for policy 1, policy_version 31942 (0.0009) [2023-10-13 22:13:59,365][60934] Updated weights for policy 1, policy_version 31952 (0.0007) [2023-10-13 22:13:59,739][60934] Updated weights for policy 1, policy_version 31962 (0.0009) [2023-10-13 22:14:00,528][60935] Updated weights for policy 0, policy_version 31560 (0.0008) [2023-10-13 22:14:00,899][60935] Updated weights for policy 0, policy_version 31570 (0.0010) [2023-10-13 22:14:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 65044480. Throughput: 0: 1726.4, 1: 1711.8. Samples: 16267892. Policy #0 lag: (min: 14.0, avg: 17.9, max: 46.0) [2023-10-13 22:14:01,249][59943] Avg episode reward: [(0, '-0.210'), (1, '0.000')] [2023-10-13 22:14:01,275][60935] Updated weights for policy 0, policy_version 31580 (0.0008) [2023-10-13 22:14:03,803][60934] Updated weights for policy 1, policy_version 31972 (0.0009) [2023-10-13 22:14:04,170][60934] Updated weights for policy 1, policy_version 31982 (0.0007) [2023-10-13 22:14:04,535][60934] Updated weights for policy 1, policy_version 31992 (0.0008) [2023-10-13 22:14:05,154][60935] Updated weights for policy 0, policy_version 31590 (0.0007) [2023-10-13 22:14:05,533][60935] Updated weights for policy 0, policy_version 31600 (0.0009) [2023-10-13 22:14:05,904][60935] Updated weights for policy 0, policy_version 31610 (0.0009) [2023-10-13 22:14:06,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 65142784. Throughput: 0: 1730.6, 1: 1693.2. Samples: 16288240. Policy #0 lag: (min: 14.0, avg: 17.9, max: 46.0) [2023-10-13 22:14:06,249][59943] Avg episode reward: [(0, '-0.210'), (1, '0.000')] [2023-10-13 22:14:08,575][60934] Updated weights for policy 1, policy_version 32002 (0.0008) [2023-10-13 22:14:08,953][60934] Updated weights for policy 1, policy_version 32012 (0.0008) [2023-10-13 22:14:09,323][60934] Updated weights for policy 1, policy_version 32022 (0.0008) [2023-10-13 22:14:09,683][60934] Updated weights for policy 1, policy_version 32032 (0.0008) [2023-10-13 22:14:09,875][60935] Updated weights for policy 0, policy_version 31620 (0.0007) [2023-10-13 22:14:10,244][60935] Updated weights for policy 0, policy_version 31630 (0.0009) [2023-10-13 22:14:10,618][60935] Updated weights for policy 0, policy_version 31640 (0.0008) [2023-10-13 22:14:11,248][59943] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 65208320. Throughput: 0: 1702.9, 1: 1689.8. Samples: 16307782. Policy #0 lag: (min: 14.0, avg: 17.9, max: 46.0) [2023-10-13 22:14:11,249][59943] Avg episode reward: [(0, '-0.210'), (1, '0.000')] [2023-10-13 22:14:13,679][60934] Updated weights for policy 1, policy_version 32042 (0.0007) [2023-10-13 22:14:14,051][60934] Updated weights for policy 1, policy_version 32052 (0.0007) [2023-10-13 22:14:14,421][60934] Updated weights for policy 1, policy_version 32062 (0.0009) [2023-10-13 22:14:14,776][60935] Updated weights for policy 0, policy_version 31650 (0.0008) [2023-10-13 22:14:15,165][60935] Updated weights for policy 0, policy_version 31660 (0.0007) [2023-10-13 22:14:15,537][60935] Updated weights for policy 0, policy_version 31670 (0.0010) [2023-10-13 22:14:15,919][60935] Updated weights for policy 0, policy_version 31680 (0.0011) [2023-10-13 22:14:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 65273856. Throughput: 0: 1727.9, 1: 1703.3. Samples: 16318942. Policy #0 lag: (min: 14.0, avg: 17.9, max: 46.0) [2023-10-13 22:14:16,249][59943] Avg episode reward: [(0, '-0.440'), (1, '0.000')] [2023-10-13 22:14:18,419][60934] Updated weights for policy 1, policy_version 32072 (0.0008) [2023-10-13 22:14:18,793][60934] Updated weights for policy 1, policy_version 32082 (0.0007) [2023-10-13 22:14:19,166][60934] Updated weights for policy 1, policy_version 32092 (0.0008) [2023-10-13 22:14:19,860][60935] Updated weights for policy 0, policy_version 31690 (0.0007) [2023-10-13 22:14:20,232][60935] Updated weights for policy 0, policy_version 31700 (0.0008) [2023-10-13 22:14:20,602][60935] Updated weights for policy 0, policy_version 31710 (0.0009) [2023-10-13 22:14:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 65339392. Throughput: 0: 1705.5, 1: 1680.2. Samples: 16338454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:14:21,249][59943] Avg episode reward: [(0, '-0.520'), (1, '0.000')] [2023-10-13 22:14:23,076][60934] Updated weights for policy 1, policy_version 32102 (0.0008) [2023-10-13 22:14:23,439][60934] Updated weights for policy 1, policy_version 32112 (0.0007) [2023-10-13 22:14:23,805][60934] Updated weights for policy 1, policy_version 32122 (0.0008) [2023-10-13 22:14:24,596][60935] Updated weights for policy 0, policy_version 31720 (0.0008) [2023-10-13 22:14:24,964][60935] Updated weights for policy 0, policy_version 31730 (0.0008) [2023-10-13 22:14:25,334][60935] Updated weights for policy 0, policy_version 31740 (0.0011) [2023-10-13 22:14:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 65404928. Throughput: 0: 1680.3, 1: 1708.8. Samples: 16358634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:14:26,249][59943] Avg episode reward: [(0, '-0.500'), (1, '0.000')] [2023-10-13 22:14:27,881][60934] Updated weights for policy 1, policy_version 32132 (0.0007) [2023-10-13 22:14:28,253][60934] Updated weights for policy 1, policy_version 32142 (0.0007) [2023-10-13 22:14:28,619][60934] Updated weights for policy 1, policy_version 32152 (0.0008) [2023-10-13 22:14:29,372][60935] Updated weights for policy 0, policy_version 31750 (0.0008) [2023-10-13 22:14:29,759][60935] Updated weights for policy 0, policy_version 31760 (0.0008) [2023-10-13 22:14:30,126][60935] Updated weights for policy 0, policy_version 31770 (0.0007) [2023-10-13 22:14:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 65470464. Throughput: 0: 1706.1, 1: 1693.1. Samples: 16369424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:14:31,249][59943] Avg episode reward: [(0, '-0.350'), (1, '0.000')] [2023-10-13 22:14:32,591][60934] Updated weights for policy 1, policy_version 32162 (0.0008) [2023-10-13 22:14:32,967][60934] Updated weights for policy 1, policy_version 32172 (0.0009) [2023-10-13 22:14:33,328][60934] Updated weights for policy 1, policy_version 32182 (0.0008) [2023-10-13 22:14:33,694][60934] Updated weights for policy 1, policy_version 32192 (0.0007) [2023-10-13 22:14:34,214][60935] Updated weights for policy 0, policy_version 31780 (0.0009) [2023-10-13 22:14:34,583][60935] Updated weights for policy 0, policy_version 31790 (0.0008) [2023-10-13 22:14:34,958][60935] Updated weights for policy 0, policy_version 31800 (0.0009) [2023-10-13 22:14:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 65536000. Throughput: 0: 1685.5, 1: 1696.3. Samples: 16389224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:14:36,249][59943] Avg episode reward: [(0, '-0.410'), (1, '0.000')] [2023-10-13 22:14:37,610][60934] Updated weights for policy 1, policy_version 32202 (0.0008) [2023-10-13 22:14:37,977][60934] Updated weights for policy 1, policy_version 32212 (0.0008) [2023-10-13 22:14:38,348][60934] Updated weights for policy 1, policy_version 32222 (0.0008) [2023-10-13 22:14:38,877][60935] Updated weights for policy 0, policy_version 31810 (0.0009) [2023-10-13 22:14:39,260][60935] Updated weights for policy 0, policy_version 31820 (0.0008) [2023-10-13 22:14:39,641][60935] Updated weights for policy 0, policy_version 31830 (0.0008) [2023-10-13 22:14:40,010][60935] Updated weights for policy 0, policy_version 31840 (0.0009) [2023-10-13 22:14:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 65601536. Throughput: 0: 1684.9, 1: 1716.8. Samples: 16409992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:14:41,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 22:14:42,477][60934] Updated weights for policy 1, policy_version 32232 (0.0008) [2023-10-13 22:14:42,845][60934] Updated weights for policy 1, policy_version 32242 (0.0007) [2023-10-13 22:14:43,217][60934] Updated weights for policy 1, policy_version 32252 (0.0007) [2023-10-13 22:14:44,058][60935] Updated weights for policy 0, policy_version 31850 (0.0009) [2023-10-13 22:14:44,430][60935] Updated weights for policy 0, policy_version 31860 (0.0008) [2023-10-13 22:14:44,794][60935] Updated weights for policy 0, policy_version 31870 (0.0008) [2023-10-13 22:14:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 65667072. Throughput: 0: 1705.6, 1: 1681.9. Samples: 16420330. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 22:14:46,249][59943] Avg episode reward: [(0, '-0.190'), (1, '0.000')] [2023-10-13 22:14:47,224][60934] Updated weights for policy 1, policy_version 32262 (0.0009) [2023-10-13 22:14:47,589][60934] Updated weights for policy 1, policy_version 32272 (0.0008) [2023-10-13 22:14:47,956][60934] Updated weights for policy 1, policy_version 32282 (0.0008) [2023-10-13 22:14:48,799][60935] Updated weights for policy 0, policy_version 31880 (0.0007) [2023-10-13 22:14:49,170][60935] Updated weights for policy 0, policy_version 31890 (0.0013) [2023-10-13 22:14:49,541][60935] Updated weights for policy 0, policy_version 31900 (0.0010) [2023-10-13 22:14:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 65732608. Throughput: 0: 1671.9, 1: 1704.9. Samples: 16440198. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 22:14:51,249][59943] Avg episode reward: [(0, '-0.150'), (1, '0.000')] [2023-10-13 22:14:51,920][60934] Updated weights for policy 1, policy_version 32292 (0.0009) [2023-10-13 22:14:52,294][60934] Updated weights for policy 1, policy_version 32302 (0.0009) [2023-10-13 22:14:52,654][60934] Updated weights for policy 1, policy_version 32312 (0.0009) [2023-10-13 22:14:53,503][60935] Updated weights for policy 0, policy_version 31910 (0.0008) [2023-10-13 22:14:53,860][60935] Updated weights for policy 0, policy_version 31920 (0.0008) [2023-10-13 22:14:54,223][60935] Updated weights for policy 0, policy_version 31930 (0.0009) [2023-10-13 22:14:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 65798144. Throughput: 0: 1699.1, 1: 1713.3. Samples: 16461342. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 22:14:56,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 22:14:56,258][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000031936_32702464.pth... [2023-10-13 22:14:56,258][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000032320_33095680.pth... [2023-10-13 22:14:56,290][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000030752_31490048.pth [2023-10-13 22:14:56,295][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000030368_31096832.pth [2023-10-13 22:14:56,786][60934] Updated weights for policy 1, policy_version 32322 (0.0009) [2023-10-13 22:14:57,156][60934] Updated weights for policy 1, policy_version 32332 (0.0010) [2023-10-13 22:14:57,527][60934] Updated weights for policy 1, policy_version 32342 (0.0009) [2023-10-13 22:14:57,882][60934] Updated weights for policy 1, policy_version 32352 (0.0009) [2023-10-13 22:14:58,182][60935] Updated weights for policy 0, policy_version 31940 (0.0010) [2023-10-13 22:14:58,561][60935] Updated weights for policy 0, policy_version 31950 (0.0009) [2023-10-13 22:14:58,943][60935] Updated weights for policy 0, policy_version 31960 (0.0010) [2023-10-13 22:15:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 65863680. Throughput: 0: 1686.9, 1: 1690.8. Samples: 16470936. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 22:15:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.070')] [2023-10-13 22:15:02,020][60934] Updated weights for policy 1, policy_version 32362 (0.0009) [2023-10-13 22:15:02,386][60934] Updated weights for policy 1, policy_version 32372 (0.0009) [2023-10-13 22:15:02,751][60934] Updated weights for policy 1, policy_version 32382 (0.0008) [2023-10-13 22:15:03,166][60935] Updated weights for policy 0, policy_version 31970 (0.0007) [2023-10-13 22:15:03,540][60935] Updated weights for policy 0, policy_version 31980 (0.0007) [2023-10-13 22:15:03,916][60935] Updated weights for policy 0, policy_version 31990 (0.0008) [2023-10-13 22:15:04,285][60935] Updated weights for policy 0, policy_version 32000 (0.0007) [2023-10-13 22:15:06,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 65929216. Throughput: 0: 1686.0, 1: 1711.2. Samples: 16491332. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 22:15:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.070')] [2023-10-13 22:15:06,797][60934] Updated weights for policy 1, policy_version 32392 (0.0009) [2023-10-13 22:15:07,161][60934] Updated weights for policy 1, policy_version 32402 (0.0009) [2023-10-13 22:15:07,526][60934] Updated weights for policy 1, policy_version 32412 (0.0011) [2023-10-13 22:15:08,146][60935] Updated weights for policy 0, policy_version 32010 (0.0008) [2023-10-13 22:15:08,520][60935] Updated weights for policy 0, policy_version 32020 (0.0008) [2023-10-13 22:15:08,881][60935] Updated weights for policy 0, policy_version 32030 (0.0007) [2023-10-13 22:15:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 65994752. Throughput: 0: 1716.2, 1: 1708.9. Samples: 16512764. Policy #0 lag: (min: 27.0, avg: 54.0, max: 56.0) [2023-10-13 22:15:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.070')] [2023-10-13 22:15:11,465][60934] Updated weights for policy 1, policy_version 32422 (0.0009) [2023-10-13 22:15:11,830][60934] Updated weights for policy 1, policy_version 32432 (0.0010) [2023-10-13 22:15:12,201][60934] Updated weights for policy 1, policy_version 32442 (0.0010) [2023-10-13 22:15:12,794][60935] Updated weights for policy 0, policy_version 32040 (0.0010) [2023-10-13 22:15:13,163][60935] Updated weights for policy 0, policy_version 32050 (0.0010) [2023-10-13 22:15:13,529][60935] Updated weights for policy 0, policy_version 32060 (0.0010) [2023-10-13 22:15:15,956][60934] Updated weights for policy 1, policy_version 32452 (0.0009) [2023-10-13 22:15:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 66060288. Throughput: 0: 1686.1, 1: 1703.8. Samples: 16521970. Policy #0 lag: (min: 27.0, avg: 54.0, max: 56.0) [2023-10-13 22:15:16,249][59943] Avg episode reward: [(0, '-0.070'), (1, '-0.070')] [2023-10-13 22:15:16,325][60934] Updated weights for policy 1, policy_version 32462 (0.0007) [2023-10-13 22:15:16,694][60934] Updated weights for policy 1, policy_version 32472 (0.0008) [2023-10-13 22:15:17,592][60935] Updated weights for policy 0, policy_version 32070 (0.0010) [2023-10-13 22:15:17,959][60935] Updated weights for policy 0, policy_version 32080 (0.0007) [2023-10-13 22:15:18,328][60935] Updated weights for policy 0, policy_version 32090 (0.0009) [2023-10-13 22:15:20,734][60934] Updated weights for policy 1, policy_version 32482 (0.0010) [2023-10-13 22:15:21,097][60934] Updated weights for policy 1, policy_version 32492 (0.0011) [2023-10-13 22:15:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 66125824. Throughput: 0: 1704.1, 1: 1714.3. Samples: 16543052. Policy #0 lag: (min: 27.0, avg: 54.0, max: 56.0) [2023-10-13 22:15:21,249][59943] Avg episode reward: [(0, '-0.070'), (1, '-0.070')] [2023-10-13 22:15:21,467][60934] Updated weights for policy 1, policy_version 32502 (0.0011) [2023-10-13 22:15:21,835][60934] Updated weights for policy 1, policy_version 32512 (0.0010) [2023-10-13 22:15:22,375][60935] Updated weights for policy 0, policy_version 32100 (0.0008) [2023-10-13 22:15:22,751][60935] Updated weights for policy 0, policy_version 32110 (0.0008) [2023-10-13 22:15:23,121][60935] Updated weights for policy 0, policy_version 32120 (0.0011) [2023-10-13 22:15:25,858][60934] Updated weights for policy 1, policy_version 32522 (0.0010) [2023-10-13 22:15:26,229][60934] Updated weights for policy 1, policy_version 32532 (0.0010) [2023-10-13 22:15:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 66191360. Throughput: 0: 1708.7, 1: 1709.9. Samples: 16563830. Policy #0 lag: (min: 27.0, avg: 54.0, max: 56.0) [2023-10-13 22:15:26,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.070')] [2023-10-13 22:15:26,602][60934] Updated weights for policy 1, policy_version 32542 (0.0009) [2023-10-13 22:15:27,049][60935] Updated weights for policy 0, policy_version 32130 (0.0011) [2023-10-13 22:15:27,424][60935] Updated weights for policy 0, policy_version 32140 (0.0010) [2023-10-13 22:15:27,794][60935] Updated weights for policy 0, policy_version 32150 (0.0009) [2023-10-13 22:15:28,159][60935] Updated weights for policy 0, policy_version 32160 (0.0008) [2023-10-13 22:15:30,318][60934] Updated weights for policy 1, policy_version 32552 (0.0009) [2023-10-13 22:15:30,695][60934] Updated weights for policy 1, policy_version 32562 (0.0008) [2023-10-13 22:15:31,051][60934] Updated weights for policy 1, policy_version 32572 (0.0007) [2023-10-13 22:15:31,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 66289664. Throughput: 0: 1681.0, 1: 1717.6. Samples: 16573266. Policy #0 lag: (min: 27.0, avg: 54.0, max: 56.0) [2023-10-13 22:15:31,249][59943] Avg episode reward: [(0, '-0.230'), (1, '-0.070')] [2023-10-13 22:15:32,259][60935] Updated weights for policy 0, policy_version 32170 (0.0008) [2023-10-13 22:15:32,632][60935] Updated weights for policy 0, policy_version 32180 (0.0008) [2023-10-13 22:15:32,998][60935] Updated weights for policy 0, policy_version 32190 (0.0009) [2023-10-13 22:15:35,187][60934] Updated weights for policy 1, policy_version 32582 (0.0008) [2023-10-13 22:15:35,569][60934] Updated weights for policy 1, policy_version 32592 (0.0008) [2023-10-13 22:15:35,932][60934] Updated weights for policy 1, policy_version 32602 (0.0008) [2023-10-13 22:15:36,249][59943] Fps is (10 sec: 16383.4, 60 sec: 13653.2, 300 sec: 13662.6). Total num frames: 66355200. Throughput: 0: 1703.6, 1: 1715.8. Samples: 16594070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:15:36,249][59943] Avg episode reward: [(0, '-0.360'), (1, '-0.070')] [2023-10-13 22:15:37,052][60935] Updated weights for policy 0, policy_version 32200 (0.0009) [2023-10-13 22:15:37,420][60935] Updated weights for policy 0, policy_version 32210 (0.0008) [2023-10-13 22:15:37,800][60935] Updated weights for policy 0, policy_version 32220 (0.0009) [2023-10-13 22:15:39,856][60934] Updated weights for policy 1, policy_version 32612 (0.0008) [2023-10-13 22:15:40,222][60934] Updated weights for policy 1, policy_version 32622 (0.0008) [2023-10-13 22:15:40,586][60934] Updated weights for policy 1, policy_version 32632 (0.0007) [2023-10-13 22:15:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 66420736. Throughput: 0: 1697.6, 1: 1694.0. Samples: 16613962. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:15:41,249][59943] Avg episode reward: [(0, '-0.320'), (1, '-0.070')] [2023-10-13 22:15:42,024][60935] Updated weights for policy 0, policy_version 32230 (0.0010) [2023-10-13 22:15:42,395][60935] Updated weights for policy 0, policy_version 32240 (0.0008) [2023-10-13 22:15:42,761][60935] Updated weights for policy 0, policy_version 32250 (0.0009) [2023-10-13 22:15:44,495][60934] Updated weights for policy 1, policy_version 32642 (0.0008) [2023-10-13 22:15:44,864][60934] Updated weights for policy 1, policy_version 32652 (0.0010) [2023-10-13 22:15:45,231][60934] Updated weights for policy 1, policy_version 32662 (0.0007) [2023-10-13 22:15:45,596][60934] Updated weights for policy 1, policy_version 32672 (0.0009) [2023-10-13 22:15:46,248][59943] Fps is (10 sec: 13108.0, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 66486272. Throughput: 0: 1688.2, 1: 1716.9. Samples: 16624166. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:15:46,249][59943] Avg episode reward: [(0, '-0.130'), (1, '-0.070')] [2023-10-13 22:15:46,690][60935] Updated weights for policy 0, policy_version 32260 (0.0010) [2023-10-13 22:15:47,058][60935] Updated weights for policy 0, policy_version 32270 (0.0008) [2023-10-13 22:15:47,419][60935] Updated weights for policy 0, policy_version 32280 (0.0009) [2023-10-13 22:15:49,889][60934] Updated weights for policy 1, policy_version 32682 (0.0007) [2023-10-13 22:15:50,260][60934] Updated weights for policy 1, policy_version 32692 (0.0008) [2023-10-13 22:15:50,637][60934] Updated weights for policy 1, policy_version 32702 (0.0009) [2023-10-13 22:15:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 66551808. Throughput: 0: 1701.1, 1: 1712.5. Samples: 16644940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:15:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.070')] [2023-10-13 22:15:51,549][60935] Updated weights for policy 0, policy_version 32290 (0.0010) [2023-10-13 22:15:51,949][60935] Updated weights for policy 0, policy_version 32300 (0.0010) [2023-10-13 22:15:52,321][60935] Updated weights for policy 0, policy_version 32310 (0.0007) [2023-10-13 22:15:52,685][60935] Updated weights for policy 0, policy_version 32320 (0.0007) [2023-10-13 22:15:54,622][60934] Updated weights for policy 1, policy_version 32712 (0.0010) [2023-10-13 22:15:54,990][60934] Updated weights for policy 1, policy_version 32722 (0.0008) [2023-10-13 22:15:55,354][60934] Updated weights for policy 1, policy_version 32732 (0.0009) [2023-10-13 22:15:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 66617344. Throughput: 0: 1688.1, 1: 1684.9. Samples: 16664550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:15:56,249][59943] Avg episode reward: [(0, '-0.050'), (1, '0.000')] [2023-10-13 22:15:56,763][60935] Updated weights for policy 0, policy_version 32330 (0.0009) [2023-10-13 22:15:57,134][60935] Updated weights for policy 0, policy_version 32340 (0.0010) [2023-10-13 22:15:57,499][60935] Updated weights for policy 0, policy_version 32350 (0.0007) [2023-10-13 22:15:59,384][60934] Updated weights for policy 1, policy_version 32742 (0.0008) [2023-10-13 22:15:59,760][60934] Updated weights for policy 1, policy_version 32752 (0.0007) [2023-10-13 22:16:00,126][60934] Updated weights for policy 1, policy_version 32762 (0.0008) [2023-10-13 22:16:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 66682880. Throughput: 0: 1691.9, 1: 1712.7. Samples: 16675178. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-13 22:16:01,249][59943] Avg episode reward: [(0, '-0.050'), (1, '0.000')] [2023-10-13 22:16:01,435][60935] Updated weights for policy 0, policy_version 32360 (0.0009) [2023-10-13 22:16:01,805][60935] Updated weights for policy 0, policy_version 32370 (0.0008) [2023-10-13 22:16:02,170][60935] Updated weights for policy 0, policy_version 32380 (0.0009) [2023-10-13 22:16:04,008][60934] Updated weights for policy 1, policy_version 32772 (0.0009) [2023-10-13 22:16:04,382][60934] Updated weights for policy 1, policy_version 32782 (0.0007) [2023-10-13 22:16:04,757][60934] Updated weights for policy 1, policy_version 32792 (0.0009) [2023-10-13 22:16:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 66748416. Throughput: 0: 1691.5, 1: 1695.5. Samples: 16695464. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-13 22:16:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:16:06,264][60935] Updated weights for policy 0, policy_version 32390 (0.0008) [2023-10-13 22:16:06,634][60935] Updated weights for policy 0, policy_version 32400 (0.0007) [2023-10-13 22:16:07,003][60935] Updated weights for policy 0, policy_version 32410 (0.0010) [2023-10-13 22:16:08,650][60934] Updated weights for policy 1, policy_version 32802 (0.0009) [2023-10-13 22:16:09,017][60934] Updated weights for policy 1, policy_version 32812 (0.0008) [2023-10-13 22:16:09,385][60934] Updated weights for policy 1, policy_version 32822 (0.0007) [2023-10-13 22:16:09,755][60934] Updated weights for policy 1, policy_version 32832 (0.0009) [2023-10-13 22:16:10,940][60935] Updated weights for policy 0, policy_version 32420 (0.0009) [2023-10-13 22:16:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 66813952. Throughput: 0: 1696.1, 1: 1685.6. Samples: 16716006. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-13 22:16:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:16:11,316][60935] Updated weights for policy 0, policy_version 32430 (0.0007) [2023-10-13 22:16:11,685][60935] Updated weights for policy 0, policy_version 32440 (0.0007) [2023-10-13 22:16:13,785][60934] Updated weights for policy 1, policy_version 32842 (0.0010) [2023-10-13 22:16:14,155][60934] Updated weights for policy 1, policy_version 32852 (0.0009) [2023-10-13 22:16:14,532][60934] Updated weights for policy 1, policy_version 32862 (0.0009) [2023-10-13 22:16:15,753][60935] Updated weights for policy 0, policy_version 32450 (0.0011) [2023-10-13 22:16:16,129][60935] Updated weights for policy 0, policy_version 32460 (0.0010) [2023-10-13 22:16:16,249][59943] Fps is (10 sec: 13106.7, 60 sec: 13653.2, 300 sec: 13440.4). Total num frames: 66879488. Throughput: 0: 1696.1, 1: 1709.6. Samples: 16726526. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-13 22:16:16,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:16:16,500][60935] Updated weights for policy 0, policy_version 32470 (0.0008) [2023-10-13 22:16:16,877][60935] Updated weights for policy 0, policy_version 32480 (0.0007) [2023-10-13 22:16:18,467][60934] Updated weights for policy 1, policy_version 32872 (0.0008) [2023-10-13 22:16:18,831][60934] Updated weights for policy 1, policy_version 32882 (0.0009) [2023-10-13 22:16:19,196][60934] Updated weights for policy 1, policy_version 32892 (0.0011) [2023-10-13 22:16:20,940][60935] Updated weights for policy 0, policy_version 32490 (0.0009) [2023-10-13 22:16:21,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 66945024. Throughput: 0: 1699.1, 1: 1680.5. Samples: 16746152. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-13 22:16:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:16:21,312][60935] Updated weights for policy 0, policy_version 32500 (0.0011) [2023-10-13 22:16:21,688][60935] Updated weights for policy 0, policy_version 32510 (0.0009) [2023-10-13 22:16:23,233][60934] Updated weights for policy 1, policy_version 32902 (0.0009) [2023-10-13 22:16:23,605][60934] Updated weights for policy 1, policy_version 32912 (0.0007) [2023-10-13 22:16:23,981][60934] Updated weights for policy 1, policy_version 32922 (0.0008) [2023-10-13 22:16:25,650][60935] Updated weights for policy 0, policy_version 32520 (0.0009) [2023-10-13 22:16:26,025][60935] Updated weights for policy 0, policy_version 32530 (0.0008) [2023-10-13 22:16:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 67010560. Throughput: 0: 1691.7, 1: 1703.1. Samples: 16766728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:16:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:16:26,399][60935] Updated weights for policy 0, policy_version 32540 (0.0009) [2023-10-13 22:16:27,991][60934] Updated weights for policy 1, policy_version 32932 (0.0008) [2023-10-13 22:16:28,361][60934] Updated weights for policy 1, policy_version 32942 (0.0008) [2023-10-13 22:16:28,719][60934] Updated weights for policy 1, policy_version 32952 (0.0008) [2023-10-13 22:16:30,478][60935] Updated weights for policy 0, policy_version 32550 (0.0008) [2023-10-13 22:16:30,853][60935] Updated weights for policy 0, policy_version 32560 (0.0009) [2023-10-13 22:16:31,234][60935] Updated weights for policy 0, policy_version 32570 (0.0009) [2023-10-13 22:16:31,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 67076096. Throughput: 0: 1698.9, 1: 1694.7. Samples: 16776878. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:16:31,249][59943] Avg episode reward: [(0, '-0.070'), (1, '0.000')] [2023-10-13 22:16:32,849][60934] Updated weights for policy 1, policy_version 32962 (0.0008) [2023-10-13 22:16:33,215][60934] Updated weights for policy 1, policy_version 32972 (0.0009) [2023-10-13 22:16:33,579][60934] Updated weights for policy 1, policy_version 32982 (0.0008) [2023-10-13 22:16:33,941][60934] Updated weights for policy 1, policy_version 32992 (0.0008) [2023-10-13 22:16:35,360][60935] Updated weights for policy 0, policy_version 32580 (0.0008) [2023-10-13 22:16:35,722][60935] Updated weights for policy 0, policy_version 32590 (0.0009) [2023-10-13 22:16:36,098][60935] Updated weights for policy 0, policy_version 32600 (0.0010) [2023-10-13 22:16:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 67141632. Throughput: 0: 1692.4, 1: 1688.0. Samples: 16797060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:16:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:16:37,966][60934] Updated weights for policy 1, policy_version 33002 (0.0007) [2023-10-13 22:16:38,346][60934] Updated weights for policy 1, policy_version 33012 (0.0008) [2023-10-13 22:16:38,717][60934] Updated weights for policy 1, policy_version 33022 (0.0010) [2023-10-13 22:16:40,277][60935] Updated weights for policy 0, policy_version 32610 (0.0011) [2023-10-13 22:16:40,670][60935] Updated weights for policy 0, policy_version 32620 (0.0009) [2023-10-13 22:16:41,042][60935] Updated weights for policy 0, policy_version 32630 (0.0007) [2023-10-13 22:16:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 67207168. Throughput: 0: 1677.6, 1: 1710.4. Samples: 16817008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:16:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:16:41,410][60935] Updated weights for policy 0, policy_version 32640 (0.0008) [2023-10-13 22:16:42,754][60934] Updated weights for policy 1, policy_version 33032 (0.0010) [2023-10-13 22:16:43,113][60934] Updated weights for policy 1, policy_version 33042 (0.0007) [2023-10-13 22:16:43,485][60934] Updated weights for policy 1, policy_version 33052 (0.0007) [2023-10-13 22:16:45,329][60935] Updated weights for policy 0, policy_version 32650 (0.0008) [2023-10-13 22:16:45,701][60935] Updated weights for policy 0, policy_version 32660 (0.0010) [2023-10-13 22:16:46,076][60935] Updated weights for policy 0, policy_version 32670 (0.0009) [2023-10-13 22:16:46,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 67305472. Throughput: 0: 1688.4, 1: 1685.0. Samples: 16826982. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 22:16:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:16:47,422][60934] Updated weights for policy 1, policy_version 33062 (0.0008) [2023-10-13 22:16:47,782][60934] Updated weights for policy 1, policy_version 33072 (0.0009) [2023-10-13 22:16:48,151][60934] Updated weights for policy 1, policy_version 33082 (0.0007) [2023-10-13 22:16:50,275][60935] Updated weights for policy 0, policy_version 32680 (0.0008) [2023-10-13 22:16:50,648][60935] Updated weights for policy 0, policy_version 32690 (0.0008) [2023-10-13 22:16:51,020][60935] Updated weights for policy 0, policy_version 32700 (0.0007) [2023-10-13 22:16:51,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 67371008. Throughput: 0: 1686.6, 1: 1704.2. Samples: 16848050. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 22:16:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:16:51,930][60934] Updated weights for policy 1, policy_version 33092 (0.0007) [2023-10-13 22:16:52,291][60934] Updated weights for policy 1, policy_version 33102 (0.0008) [2023-10-13 22:16:52,665][60934] Updated weights for policy 1, policy_version 33112 (0.0009) [2023-10-13 22:16:54,874][60935] Updated weights for policy 0, policy_version 32710 (0.0007) [2023-10-13 22:16:55,237][60935] Updated weights for policy 0, policy_version 32720 (0.0010) [2023-10-13 22:16:55,605][60935] Updated weights for policy 0, policy_version 32730 (0.0009) [2023-10-13 22:16:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 67436544. Throughput: 0: 1660.6, 1: 1719.4. Samples: 16868104. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 22:16:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:16:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000033120_33914880.pth... [2023-10-13 22:16:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000032736_33521664.pth... [2023-10-13 22:16:56,298][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000031136_31883264.pth [2023-10-13 22:16:56,299][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000031520_32276480.pth [2023-10-13 22:16:56,702][60934] Updated weights for policy 1, policy_version 33122 (0.0008) [2023-10-13 22:16:57,075][60934] Updated weights for policy 1, policy_version 33132 (0.0009) [2023-10-13 22:16:57,434][60934] Updated weights for policy 1, policy_version 33142 (0.0009) [2023-10-13 22:16:57,801][60934] Updated weights for policy 1, policy_version 33152 (0.0010) [2023-10-13 22:16:59,772][60935] Updated weights for policy 0, policy_version 32740 (0.0009) [2023-10-13 22:17:00,140][60935] Updated weights for policy 0, policy_version 32750 (0.0009) [2023-10-13 22:17:00,519][60935] Updated weights for policy 0, policy_version 32760 (0.0008) [2023-10-13 22:17:01,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 67502080. Throughput: 0: 1688.1, 1: 1690.1. Samples: 16878542. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 22:17:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:17:01,752][60934] Updated weights for policy 1, policy_version 33162 (0.0008) [2023-10-13 22:17:02,124][60934] Updated weights for policy 1, policy_version 33172 (0.0009) [2023-10-13 22:17:02,492][60934] Updated weights for policy 1, policy_version 33182 (0.0007) [2023-10-13 22:17:04,633][60935] Updated weights for policy 0, policy_version 32770 (0.0011) [2023-10-13 22:17:05,003][60935] Updated weights for policy 0, policy_version 32780 (0.0008) [2023-10-13 22:17:05,374][60935] Updated weights for policy 0, policy_version 32790 (0.0008) [2023-10-13 22:17:05,746][60935] Updated weights for policy 0, policy_version 32800 (0.0007) [2023-10-13 22:17:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 67567616. Throughput: 0: 1679.2, 1: 1726.5. Samples: 16899408. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 22:17:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:17:06,385][60934] Updated weights for policy 1, policy_version 33192 (0.0007) [2023-10-13 22:17:06,759][60934] Updated weights for policy 1, policy_version 33202 (0.0007) [2023-10-13 22:17:07,123][60934] Updated weights for policy 1, policy_version 33212 (0.0009) [2023-10-13 22:17:09,660][60935] Updated weights for policy 0, policy_version 32810 (0.0010) [2023-10-13 22:17:10,023][60935] Updated weights for policy 0, policy_version 32820 (0.0008) [2023-10-13 22:17:10,386][60935] Updated weights for policy 0, policy_version 32830 (0.0012) [2023-10-13 22:17:11,060][60934] Updated weights for policy 1, policy_version 33222 (0.0010) [2023-10-13 22:17:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 67633152. Throughput: 0: 1668.9, 1: 1731.4. Samples: 16919744. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-10-13 22:17:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:17:11,436][60934] Updated weights for policy 1, policy_version 33232 (0.0011) [2023-10-13 22:17:11,805][60934] Updated weights for policy 1, policy_version 33242 (0.0010) [2023-10-13 22:17:14,277][60935] Updated weights for policy 0, policy_version 32840 (0.0010) [2023-10-13 22:17:14,642][60935] Updated weights for policy 0, policy_version 32850 (0.0010) [2023-10-13 22:17:15,007][60935] Updated weights for policy 0, policy_version 32860 (0.0011) [2023-10-13 22:17:15,949][60934] Updated weights for policy 1, policy_version 33252 (0.0009) [2023-10-13 22:17:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 67698688. Throughput: 0: 1691.4, 1: 1715.6. Samples: 16930196. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-10-13 22:17:16,249][59943] Avg episode reward: [(0, '-0.020'), (1, '0.000')] [2023-10-13 22:17:16,317][60934] Updated weights for policy 1, policy_version 33262 (0.0008) [2023-10-13 22:17:16,675][60934] Updated weights for policy 1, policy_version 33272 (0.0010) [2023-10-13 22:17:19,162][60935] Updated weights for policy 0, policy_version 32870 (0.0011) [2023-10-13 22:17:19,533][60935] Updated weights for policy 0, policy_version 32880 (0.0011) [2023-10-13 22:17:19,904][60935] Updated weights for policy 0, policy_version 32890 (0.0010) [2023-10-13 22:17:20,711][60934] Updated weights for policy 1, policy_version 33282 (0.0011) [2023-10-13 22:17:21,075][60934] Updated weights for policy 1, policy_version 33292 (0.0010) [2023-10-13 22:17:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 67764224. Throughput: 0: 1672.6, 1: 1728.8. Samples: 16950124. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-10-13 22:17:21,249][59943] Avg episode reward: [(0, '-0.020'), (1, '0.000')] [2023-10-13 22:17:21,446][60934] Updated weights for policy 1, policy_version 33302 (0.0010) [2023-10-13 22:17:21,806][60934] Updated weights for policy 1, policy_version 33312 (0.0011) [2023-10-13 22:17:23,991][60935] Updated weights for policy 0, policy_version 32900 (0.0009) [2023-10-13 22:17:24,359][60935] Updated weights for policy 0, policy_version 32910 (0.0009) [2023-10-13 22:17:24,728][60935] Updated weights for policy 0, policy_version 32920 (0.0009) [2023-10-13 22:17:25,751][60934] Updated weights for policy 1, policy_version 33322 (0.0008) [2023-10-13 22:17:26,120][60934] Updated weights for policy 1, policy_version 33332 (0.0008) [2023-10-13 22:17:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 67829760. Throughput: 0: 1679.3, 1: 1732.2. Samples: 16970524. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-10-13 22:17:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:17:26,487][60934] Updated weights for policy 1, policy_version 33342 (0.0008) [2023-10-13 22:17:28,909][60935] Updated weights for policy 0, policy_version 32930 (0.0009) [2023-10-13 22:17:29,328][60935] Updated weights for policy 0, policy_version 32940 (0.0008) [2023-10-13 22:17:29,688][60935] Updated weights for policy 0, policy_version 32950 (0.0008) [2023-10-13 22:17:30,058][60935] Updated weights for policy 0, policy_version 32960 (0.0007) [2023-10-13 22:17:30,469][60934] Updated weights for policy 1, policy_version 33352 (0.0009) [2023-10-13 22:17:30,830][60934] Updated weights for policy 1, policy_version 33362 (0.0009) [2023-10-13 22:17:31,199][60934] Updated weights for policy 1, policy_version 33372 (0.0008) [2023-10-13 22:17:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 67895296. Throughput: 0: 1691.6, 1: 1729.2. Samples: 16980914. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-10-13 22:17:31,249][59943] Avg episode reward: [(0, '-0.060'), (1, '0.000')] [2023-10-13 22:17:34,065][60935] Updated weights for policy 0, policy_version 32970 (0.0011) [2023-10-13 22:17:34,429][60935] Updated weights for policy 0, policy_version 32980 (0.0011) [2023-10-13 22:17:34,799][60935] Updated weights for policy 0, policy_version 32990 (0.0008) [2023-10-13 22:17:35,301][60934] Updated weights for policy 1, policy_version 33382 (0.0009) [2023-10-13 22:17:35,660][60934] Updated weights for policy 1, policy_version 33392 (0.0010) [2023-10-13 22:17:36,030][60934] Updated weights for policy 1, policy_version 33402 (0.0008) [2023-10-13 22:17:36,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 67993600. Throughput: 0: 1663.5, 1: 1726.8. Samples: 17000614. Policy #0 lag: (min: 14.0, avg: 17.1, max: 46.0) [2023-10-13 22:17:36,249][59943] Avg episode reward: [(0, '-0.060'), (1, '0.000')] [2023-10-13 22:17:38,958][60935] Updated weights for policy 0, policy_version 33000 (0.0008) [2023-10-13 22:17:39,326][60935] Updated weights for policy 0, policy_version 33010 (0.0008) [2023-10-13 22:17:39,701][60935] Updated weights for policy 0, policy_version 33020 (0.0009) [2023-10-13 22:17:40,041][60934] Updated weights for policy 1, policy_version 33412 (0.0009) [2023-10-13 22:17:40,399][60934] Updated weights for policy 1, policy_version 33422 (0.0008) [2023-10-13 22:17:40,762][60934] Updated weights for policy 1, policy_version 33432 (0.0007) [2023-10-13 22:17:41,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 68059136. Throughput: 0: 1683.2, 1: 1705.3. Samples: 17020586. Policy #0 lag: (min: 14.0, avg: 17.1, max: 46.0) [2023-10-13 22:17:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:17:43,805][60935] Updated weights for policy 0, policy_version 33030 (0.0008) [2023-10-13 22:17:44,170][60935] Updated weights for policy 0, policy_version 33040 (0.0008) [2023-10-13 22:17:44,535][60935] Updated weights for policy 0, policy_version 33050 (0.0008) [2023-10-13 22:17:44,690][60934] Updated weights for policy 1, policy_version 33442 (0.0007) [2023-10-13 22:17:45,050][60934] Updated weights for policy 1, policy_version 33452 (0.0008) [2023-10-13 22:17:45,420][60934] Updated weights for policy 1, policy_version 33462 (0.0008) [2023-10-13 22:17:45,786][60934] Updated weights for policy 1, policy_version 33472 (0.0008) [2023-10-13 22:17:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 68124672. Throughput: 0: 1674.3, 1: 1723.2. Samples: 17031430. Policy #0 lag: (min: 14.0, avg: 17.1, max: 46.0) [2023-10-13 22:17:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:17:48,523][60935] Updated weights for policy 0, policy_version 33060 (0.0009) [2023-10-13 22:17:48,893][60935] Updated weights for policy 0, policy_version 33070 (0.0008) [2023-10-13 22:17:49,262][60935] Updated weights for policy 0, policy_version 33080 (0.0008) [2023-10-13 22:17:49,963][60934] Updated weights for policy 1, policy_version 33482 (0.0008) [2023-10-13 22:17:50,327][60934] Updated weights for policy 1, policy_version 33492 (0.0007) [2023-10-13 22:17:50,702][60934] Updated weights for policy 1, policy_version 33502 (0.0009) [2023-10-13 22:17:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 68190208. Throughput: 0: 1658.4, 1: 1715.4. Samples: 17051232. Policy #0 lag: (min: 14.0, avg: 17.1, max: 46.0) [2023-10-13 22:17:51,249][59943] Avg episode reward: [(0, '-0.260'), (1, '0.000')] [2023-10-13 22:17:53,346][60935] Updated weights for policy 0, policy_version 33090 (0.0009) [2023-10-13 22:17:53,712][60935] Updated weights for policy 0, policy_version 33100 (0.0008) [2023-10-13 22:17:54,076][60935] Updated weights for policy 0, policy_version 33110 (0.0008) [2023-10-13 22:17:54,444][60935] Updated weights for policy 0, policy_version 33120 (0.0008) [2023-10-13 22:17:54,587][60934] Updated weights for policy 1, policy_version 33512 (0.0010) [2023-10-13 22:17:54,956][60934] Updated weights for policy 1, policy_version 33522 (0.0011) [2023-10-13 22:17:55,319][60934] Updated weights for policy 1, policy_version 33532 (0.0010) [2023-10-13 22:17:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 68255744. Throughput: 0: 1675.5, 1: 1677.2. Samples: 17070612. Policy #0 lag: (min: 14.0, avg: 17.1, max: 46.0) [2023-10-13 22:17:56,249][59943] Avg episode reward: [(0, '-0.350'), (1, '0.000')] [2023-10-13 22:17:58,582][60935] Updated weights for policy 0, policy_version 33130 (0.0010) [2023-10-13 22:17:58,962][60935] Updated weights for policy 0, policy_version 33140 (0.0007) [2023-10-13 22:17:59,246][60934] Updated weights for policy 1, policy_version 33542 (0.0009) [2023-10-13 22:17:59,323][60935] Updated weights for policy 0, policy_version 33150 (0.0009) [2023-10-13 22:17:59,611][60934] Updated weights for policy 1, policy_version 33552 (0.0007) [2023-10-13 22:17:59,984][60934] Updated weights for policy 1, policy_version 33562 (0.0007) [2023-10-13 22:18:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 68321280. Throughput: 0: 1656.5, 1: 1714.1. Samples: 17081872. Policy #0 lag: (min: 9.0, avg: 12.8, max: 41.0) [2023-10-13 22:18:01,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 22:18:03,384][60935] Updated weights for policy 0, policy_version 33160 (0.0009) [2023-10-13 22:18:03,753][60935] Updated weights for policy 0, policy_version 33170 (0.0009) [2023-10-13 22:18:03,836][60934] Updated weights for policy 1, policy_version 33572 (0.0007) [2023-10-13 22:18:04,120][60935] Updated weights for policy 0, policy_version 33180 (0.0007) [2023-10-13 22:18:04,203][60934] Updated weights for policy 1, policy_version 33582 (0.0008) [2023-10-13 22:18:04,578][60934] Updated weights for policy 1, policy_version 33592 (0.0011) [2023-10-13 22:18:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 68386816. Throughput: 0: 1663.7, 1: 1695.9. Samples: 17101306. Policy #0 lag: (min: 9.0, avg: 12.8, max: 41.0) [2023-10-13 22:18:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:18:08,243][60935] Updated weights for policy 0, policy_version 33190 (0.0008) [2023-10-13 22:18:08,605][60934] Updated weights for policy 1, policy_version 33602 (0.0009) [2023-10-13 22:18:08,618][60935] Updated weights for policy 0, policy_version 33200 (0.0009) [2023-10-13 22:18:08,968][60934] Updated weights for policy 1, policy_version 33612 (0.0009) [2023-10-13 22:18:08,977][60935] Updated weights for policy 0, policy_version 33210 (0.0009) [2023-10-13 22:18:09,337][60934] Updated weights for policy 1, policy_version 33622 (0.0007) [2023-10-13 22:18:09,700][60934] Updated weights for policy 1, policy_version 33632 (0.0007) [2023-10-13 22:18:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 68452352. Throughput: 0: 1669.8, 1: 1688.5. Samples: 17121648. Policy #0 lag: (min: 9.0, avg: 12.8, max: 41.0) [2023-10-13 22:18:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:18:13,067][60935] Updated weights for policy 0, policy_version 33220 (0.0010) [2023-10-13 22:18:13,432][60935] Updated weights for policy 0, policy_version 33230 (0.0009) [2023-10-13 22:18:13,740][60934] Updated weights for policy 1, policy_version 33642 (0.0008) [2023-10-13 22:18:13,801][60935] Updated weights for policy 0, policy_version 33240 (0.0007) [2023-10-13 22:18:14,109][60934] Updated weights for policy 1, policy_version 33652 (0.0008) [2023-10-13 22:18:14,472][60934] Updated weights for policy 1, policy_version 33662 (0.0007) [2023-10-13 22:18:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 68517888. Throughput: 0: 1650.8, 1: 1707.6. Samples: 17132044. Policy #0 lag: (min: 9.0, avg: 12.8, max: 41.0) [2023-10-13 22:18:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:18:17,790][60935] Updated weights for policy 0, policy_version 33250 (0.0008) [2023-10-13 22:18:18,169][60935] Updated weights for policy 0, policy_version 33260 (0.0009) [2023-10-13 22:18:18,532][60935] Updated weights for policy 0, policy_version 33270 (0.0008) [2023-10-13 22:18:18,557][60934] Updated weights for policy 1, policy_version 33672 (0.0008) [2023-10-13 22:18:18,904][60935] Updated weights for policy 0, policy_version 33280 (0.0007) [2023-10-13 22:18:18,932][60934] Updated weights for policy 1, policy_version 33682 (0.0008) [2023-10-13 22:18:19,309][60934] Updated weights for policy 1, policy_version 33692 (0.0010) [2023-10-13 22:18:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 68583424. Throughput: 0: 1673.6, 1: 1677.3. Samples: 17151404. Policy #0 lag: (min: 9.0, avg: 12.8, max: 41.0) [2023-10-13 22:18:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:18:23,128][60935] Updated weights for policy 0, policy_version 33290 (0.0008) [2023-10-13 22:18:23,345][60934] Updated weights for policy 1, policy_version 33702 (0.0008) [2023-10-13 22:18:23,502][60935] Updated weights for policy 0, policy_version 33300 (0.0010) [2023-10-13 22:18:23,709][60934] Updated weights for policy 1, policy_version 33712 (0.0009) [2023-10-13 22:18:23,874][60935] Updated weights for policy 0, policy_version 33310 (0.0009) [2023-10-13 22:18:24,076][60934] Updated weights for policy 1, policy_version 33722 (0.0008) [2023-10-13 22:18:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 68648960. Throughput: 0: 1671.4, 1: 1695.4. Samples: 17172092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:18:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:18:28,006][60935] Updated weights for policy 0, policy_version 33320 (0.0007) [2023-10-13 22:18:28,115][60934] Updated weights for policy 1, policy_version 33732 (0.0009) [2023-10-13 22:18:28,375][60935] Updated weights for policy 0, policy_version 33330 (0.0009) [2023-10-13 22:18:28,483][60934] Updated weights for policy 1, policy_version 33742 (0.0008) [2023-10-13 22:18:28,744][60935] Updated weights for policy 0, policy_version 33340 (0.0007) [2023-10-13 22:18:28,843][60934] Updated weights for policy 1, policy_version 33752 (0.0007) [2023-10-13 22:18:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 68714496. Throughput: 0: 1653.6, 1: 1693.8. Samples: 17182062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:18:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:18:32,783][60935] Updated weights for policy 0, policy_version 33350 (0.0008) [2023-10-13 22:18:32,909][60934] Updated weights for policy 1, policy_version 33762 (0.0009) [2023-10-13 22:18:33,150][60935] Updated weights for policy 0, policy_version 33360 (0.0007) [2023-10-13 22:18:33,275][60934] Updated weights for policy 1, policy_version 33772 (0.0007) [2023-10-13 22:18:33,526][60935] Updated weights for policy 0, policy_version 33370 (0.0008) [2023-10-13 22:18:33,637][60934] Updated weights for policy 1, policy_version 33782 (0.0007) [2023-10-13 22:18:34,010][60934] Updated weights for policy 1, policy_version 33792 (0.0008) [2023-10-13 22:18:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 68780032. Throughput: 0: 1672.8, 1: 1677.9. Samples: 17202014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:18:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:18:37,571][60935] Updated weights for policy 0, policy_version 33380 (0.0009) [2023-10-13 22:18:37,941][60935] Updated weights for policy 0, policy_version 33390 (0.0010) [2023-10-13 22:18:38,205][60934] Updated weights for policy 1, policy_version 33802 (0.0008) [2023-10-13 22:18:38,312][60935] Updated weights for policy 0, policy_version 33400 (0.0008) [2023-10-13 22:18:38,571][60934] Updated weights for policy 1, policy_version 33812 (0.0007) [2023-10-13 22:18:38,932][60934] Updated weights for policy 1, policy_version 33822 (0.0007) [2023-10-13 22:18:41,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 68845568. Throughput: 0: 1672.0, 1: 1705.0. Samples: 17222576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:18:41,249][59943] Avg episode reward: [(0, '-0.050'), (1, '0.000')] [2023-10-13 22:18:42,624][60935] Updated weights for policy 0, policy_version 33410 (0.0008) [2023-10-13 22:18:42,992][60935] Updated weights for policy 0, policy_version 33420 (0.0008) [2023-10-13 22:18:42,997][60934] Updated weights for policy 1, policy_version 33832 (0.0009) [2023-10-13 22:18:43,360][60935] Updated weights for policy 0, policy_version 33430 (0.0008) [2023-10-13 22:18:43,369][60934] Updated weights for policy 1, policy_version 33842 (0.0008) [2023-10-13 22:18:43,725][60935] Updated weights for policy 0, policy_version 33440 (0.0010) [2023-10-13 22:18:43,728][60934] Updated weights for policy 1, policy_version 33852 (0.0007) [2023-10-13 22:18:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 68911104. Throughput: 0: 1657.2, 1: 1679.3. Samples: 17232016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:18:46,249][59943] Avg episode reward: [(0, '-0.050'), (1, '0.000')] [2023-10-13 22:18:47,809][60934] Updated weights for policy 1, policy_version 33862 (0.0007) [2023-10-13 22:18:47,825][60935] Updated weights for policy 0, policy_version 33450 (0.0008) [2023-10-13 22:18:48,174][60934] Updated weights for policy 1, policy_version 33872 (0.0007) [2023-10-13 22:18:48,181][60935] Updated weights for policy 0, policy_version 33460 (0.0007) [2023-10-13 22:18:48,537][60934] Updated weights for policy 1, policy_version 33882 (0.0008) [2023-10-13 22:18:48,553][60935] Updated weights for policy 0, policy_version 33470 (0.0009) [2023-10-13 22:18:51,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 68976640. Throughput: 0: 1672.3, 1: 1684.2. Samples: 17252350. Policy #0 lag: (min: 14.0, avg: 19.0, max: 46.0) [2023-10-13 22:18:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:18:52,644][60935] Updated weights for policy 0, policy_version 33480 (0.0008) [2023-10-13 22:18:52,653][60934] Updated weights for policy 1, policy_version 33892 (0.0007) [2023-10-13 22:18:53,016][60935] Updated weights for policy 0, policy_version 33490 (0.0008) [2023-10-13 22:18:53,020][60934] Updated weights for policy 1, policy_version 33902 (0.0008) [2023-10-13 22:18:53,379][60935] Updated weights for policy 0, policy_version 33500 (0.0008) [2023-10-13 22:18:53,390][60934] Updated weights for policy 1, policy_version 33912 (0.0007) [2023-10-13 22:18:56,248][59943] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 69042176. Throughput: 0: 1674.3, 1: 1689.0. Samples: 17272994. Policy #0 lag: (min: 14.0, avg: 19.0, max: 46.0) [2023-10-13 22:18:56,249][59943] Avg episode reward: [(0, '-0.170'), (1, '0.000')] [2023-10-13 22:18:56,262][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000033504_34308096.pth... [2023-10-13 22:18:56,262][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000033920_34734080.pth... [2023-10-13 22:18:56,297][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000032320_33095680.pth [2023-10-13 22:18:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000031936_32702464.pth [2023-10-13 22:18:57,500][60935] Updated weights for policy 0, policy_version 33510 (0.0007) [2023-10-13 22:18:57,510][60934] Updated weights for policy 1, policy_version 33922 (0.0009) [2023-10-13 22:18:57,862][60935] Updated weights for policy 0, policy_version 33520 (0.0008) [2023-10-13 22:18:57,876][60934] Updated weights for policy 1, policy_version 33932 (0.0008) [2023-10-13 22:18:58,237][60935] Updated weights for policy 0, policy_version 33530 (0.0008) [2023-10-13 22:18:58,248][60934] Updated weights for policy 1, policy_version 33942 (0.0007) [2023-10-13 22:18:58,611][60934] Updated weights for policy 1, policy_version 33952 (0.0010) [2023-10-13 22:19:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 69107712. Throughput: 0: 1664.7, 1: 1668.4. Samples: 17282032. Policy #0 lag: (min: 14.0, avg: 19.0, max: 46.0) [2023-10-13 22:19:01,249][59943] Avg episode reward: [(0, '-0.170'), (1, '0.000')] [2023-10-13 22:19:02,361][60935] Updated weights for policy 0, policy_version 33540 (0.0009) [2023-10-13 22:19:02,606][60934] Updated weights for policy 1, policy_version 33962 (0.0007) [2023-10-13 22:19:02,731][60935] Updated weights for policy 0, policy_version 33550 (0.0010) [2023-10-13 22:19:02,973][60934] Updated weights for policy 1, policy_version 33972 (0.0007) [2023-10-13 22:19:03,097][60935] Updated weights for policy 0, policy_version 33560 (0.0010) [2023-10-13 22:19:03,352][60934] Updated weights for policy 1, policy_version 33982 (0.0008) [2023-10-13 22:19:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 69173248. Throughput: 0: 1667.6, 1: 1694.8. Samples: 17302710. Policy #0 lag: (min: 14.0, avg: 19.0, max: 46.0) [2023-10-13 22:19:06,249][59943] Avg episode reward: [(0, '-0.170'), (1, '-0.010')] [2023-10-13 22:19:07,378][60935] Updated weights for policy 0, policy_version 33570 (0.0010) [2023-10-13 22:19:07,542][60934] Updated weights for policy 1, policy_version 33992 (0.0009) [2023-10-13 22:19:07,781][60935] Updated weights for policy 0, policy_version 33580 (0.0008) [2023-10-13 22:19:07,914][60934] Updated weights for policy 1, policy_version 34002 (0.0009) [2023-10-13 22:19:08,149][60935] Updated weights for policy 0, policy_version 33590 (0.0007) [2023-10-13 22:19:08,275][60934] Updated weights for policy 1, policy_version 34012 (0.0011) [2023-10-13 22:19:08,518][60935] Updated weights for policy 0, policy_version 33600 (0.0008) [2023-10-13 22:19:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 69238784. Throughput: 0: 1666.7, 1: 1694.4. Samples: 17323344. Policy #0 lag: (min: 14.0, avg: 19.0, max: 46.0) [2023-10-13 22:19:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:19:12,243][60934] Updated weights for policy 1, policy_version 34022 (0.0009) [2023-10-13 22:19:12,519][60935] Updated weights for policy 0, policy_version 33610 (0.0008) [2023-10-13 22:19:12,610][60934] Updated weights for policy 1, policy_version 34032 (0.0007) [2023-10-13 22:19:12,889][60935] Updated weights for policy 0, policy_version 33620 (0.0008) [2023-10-13 22:19:12,980][60934] Updated weights for policy 1, policy_version 34042 (0.0007) [2023-10-13 22:19:13,273][60935] Updated weights for policy 0, policy_version 33630 (0.0007) [2023-10-13 22:19:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 69304320. Throughput: 0: 1664.6, 1: 1678.9. Samples: 17332520. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) [2023-10-13 22:19:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:19:17,024][60934] Updated weights for policy 1, policy_version 34052 (0.0007) [2023-10-13 22:19:17,246][60935] Updated weights for policy 0, policy_version 33640 (0.0008) [2023-10-13 22:19:17,389][60934] Updated weights for policy 1, policy_version 34062 (0.0009) [2023-10-13 22:19:17,613][60935] Updated weights for policy 0, policy_version 33650 (0.0008) [2023-10-13 22:19:17,752][60934] Updated weights for policy 1, policy_version 34072 (0.0008) [2023-10-13 22:19:17,976][60935] Updated weights for policy 0, policy_version 33660 (0.0007) [2023-10-13 22:19:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 69369856. Throughput: 0: 1667.8, 1: 1688.6. Samples: 17353052. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) [2023-10-13 22:19:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:19:21,780][60934] Updated weights for policy 1, policy_version 34082 (0.0008) [2023-10-13 22:19:22,143][60934] Updated weights for policy 1, policy_version 34092 (0.0009) [2023-10-13 22:19:22,164][60935] Updated weights for policy 0, policy_version 33670 (0.0009) [2023-10-13 22:19:22,509][60934] Updated weights for policy 1, policy_version 34102 (0.0009) [2023-10-13 22:19:22,536][60935] Updated weights for policy 0, policy_version 33680 (0.0007) [2023-10-13 22:19:22,878][60934] Updated weights for policy 1, policy_version 34112 (0.0008) [2023-10-13 22:19:22,904][60935] Updated weights for policy 0, policy_version 33690 (0.0008) [2023-10-13 22:19:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 69435392. Throughput: 0: 1669.2, 1: 1696.8. Samples: 17374046. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) [2023-10-13 22:19:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:19:26,904][60934] Updated weights for policy 1, policy_version 34122 (0.0008) [2023-10-13 22:19:26,977][60935] Updated weights for policy 0, policy_version 33700 (0.0008) [2023-10-13 22:19:27,266][60934] Updated weights for policy 1, policy_version 34132 (0.0010) [2023-10-13 22:19:27,340][60935] Updated weights for policy 0, policy_version 33710 (0.0008) [2023-10-13 22:19:27,631][60934] Updated weights for policy 1, policy_version 34142 (0.0008) [2023-10-13 22:19:27,703][60935] Updated weights for policy 0, policy_version 33720 (0.0009) [2023-10-13 22:19:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 69500928. Throughput: 0: 1670.0, 1: 1688.6. Samples: 17383152. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) [2023-10-13 22:19:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:19:31,566][60934] Updated weights for policy 1, policy_version 34152 (0.0007) [2023-10-13 22:19:31,828][60935] Updated weights for policy 0, policy_version 33730 (0.0010) [2023-10-13 22:19:31,938][60934] Updated weights for policy 1, policy_version 34162 (0.0008) [2023-10-13 22:19:32,197][60935] Updated weights for policy 0, policy_version 33740 (0.0009) [2023-10-13 22:19:32,301][60934] Updated weights for policy 1, policy_version 34172 (0.0010) [2023-10-13 22:19:32,581][60935] Updated weights for policy 0, policy_version 33750 (0.0009) [2023-10-13 22:19:32,942][60935] Updated weights for policy 0, policy_version 33760 (0.0009) [2023-10-13 22:19:36,079][60934] Updated weights for policy 1, policy_version 34182 (0.0008) [2023-10-13 22:19:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 69566464. Throughput: 0: 1669.7, 1: 1703.6. Samples: 17404144. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) [2023-10-13 22:19:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:19:36,438][60934] Updated weights for policy 1, policy_version 34192 (0.0008) [2023-10-13 22:19:36,805][60934] Updated weights for policy 1, policy_version 34202 (0.0007) [2023-10-13 22:19:36,914][60935] Updated weights for policy 0, policy_version 33770 (0.0011) [2023-10-13 22:19:37,292][60935] Updated weights for policy 0, policy_version 33780 (0.0008) [2023-10-13 22:19:37,662][60935] Updated weights for policy 0, policy_version 33790 (0.0008) [2023-10-13 22:19:40,887][60934] Updated weights for policy 1, policy_version 34212 (0.0009) [2023-10-13 22:19:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 69632000. Throughput: 0: 1672.3, 1: 1707.1. Samples: 17425066. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 22:19:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:19:41,260][60934] Updated weights for policy 1, policy_version 34222 (0.0007) [2023-10-13 22:19:41,628][60934] Updated weights for policy 1, policy_version 34232 (0.0007) [2023-10-13 22:19:41,845][60935] Updated weights for policy 0, policy_version 33800 (0.0008) [2023-10-13 22:19:42,208][60935] Updated weights for policy 0, policy_version 33810 (0.0008) [2023-10-13 22:19:42,586][60935] Updated weights for policy 0, policy_version 33820 (0.0008) [2023-10-13 22:19:45,660][60934] Updated weights for policy 1, policy_version 34242 (0.0008) [2023-10-13 22:19:46,026][60934] Updated weights for policy 1, policy_version 34252 (0.0009) [2023-10-13 22:19:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 69697536. Throughput: 0: 1674.5, 1: 1706.1. Samples: 17434160. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 22:19:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:19:46,390][60934] Updated weights for policy 1, policy_version 34262 (0.0009) [2023-10-13 22:19:46,651][60935] Updated weights for policy 0, policy_version 33830 (0.0007) [2023-10-13 22:19:46,765][60934] Updated weights for policy 1, policy_version 34272 (0.0008) [2023-10-13 22:19:47,010][60935] Updated weights for policy 0, policy_version 33840 (0.0007) [2023-10-13 22:19:47,385][60935] Updated weights for policy 0, policy_version 33850 (0.0008) [2023-10-13 22:19:50,833][60934] Updated weights for policy 1, policy_version 34282 (0.0010) [2023-10-13 22:19:51,203][60934] Updated weights for policy 1, policy_version 34292 (0.0008) [2023-10-13 22:19:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 69763072. Throughput: 0: 1675.9, 1: 1702.7. Samples: 17454744. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 22:19:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:19:51,475][60935] Updated weights for policy 0, policy_version 33860 (0.0007) [2023-10-13 22:19:51,563][60934] Updated weights for policy 1, policy_version 34302 (0.0008) [2023-10-13 22:19:51,875][60935] Updated weights for policy 0, policy_version 33870 (0.0009) [2023-10-13 22:19:52,247][60935] Updated weights for policy 0, policy_version 33880 (0.0007) [2023-10-13 22:19:55,657][60934] Updated weights for policy 1, policy_version 34312 (0.0008) [2023-10-13 22:19:56,032][60934] Updated weights for policy 1, policy_version 34322 (0.0007) [2023-10-13 22:19:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 69828608. Throughput: 0: 1683.1, 1: 1699.3. Samples: 17475552. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 22:19:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:19:56,279][60935] Updated weights for policy 0, policy_version 33890 (0.0007) [2023-10-13 22:19:56,405][60934] Updated weights for policy 1, policy_version 34332 (0.0007) [2023-10-13 22:19:56,646][60935] Updated weights for policy 0, policy_version 33900 (0.0008) [2023-10-13 22:19:57,018][60935] Updated weights for policy 0, policy_version 33910 (0.0008) [2023-10-13 22:19:57,385][60935] Updated weights for policy 0, policy_version 33920 (0.0008) [2023-10-13 22:20:00,387][60934] Updated weights for policy 1, policy_version 34342 (0.0007) [2023-10-13 22:20:00,755][60934] Updated weights for policy 1, policy_version 34352 (0.0010) [2023-10-13 22:20:01,116][60934] Updated weights for policy 1, policy_version 34362 (0.0008) [2023-10-13 22:20:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 69894144. Throughput: 0: 1683.0, 1: 1703.5. Samples: 17484910. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 22:20:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:20:01,467][60935] Updated weights for policy 0, policy_version 33930 (0.0007) [2023-10-13 22:20:01,842][60935] Updated weights for policy 0, policy_version 33940 (0.0010) [2023-10-13 22:20:02,213][60935] Updated weights for policy 0, policy_version 33950 (0.0009) [2023-10-13 22:20:05,053][60934] Updated weights for policy 1, policy_version 34372 (0.0010) [2023-10-13 22:20:05,425][60934] Updated weights for policy 1, policy_version 34382 (0.0009) [2023-10-13 22:20:05,785][60934] Updated weights for policy 1, policy_version 34392 (0.0009) [2023-10-13 22:20:06,118][60935] Updated weights for policy 0, policy_version 33960 (0.0010) [2023-10-13 22:20:06,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 69992448. Throughput: 0: 1680.7, 1: 1711.6. Samples: 17505702. Policy #0 lag: (min: 5.0, avg: 28.8, max: 32.0) [2023-10-13 22:20:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:20:06,497][60935] Updated weights for policy 0, policy_version 33970 (0.0009) [2023-10-13 22:20:06,865][60935] Updated weights for policy 0, policy_version 33980 (0.0008) [2023-10-13 22:20:09,777][60934] Updated weights for policy 1, policy_version 34402 (0.0008) [2023-10-13 22:20:10,151][60934] Updated weights for policy 1, policy_version 34412 (0.0007) [2023-10-13 22:20:10,514][60934] Updated weights for policy 1, policy_version 34422 (0.0009) [2023-10-13 22:20:10,884][60934] Updated weights for policy 1, policy_version 34432 (0.0009) [2023-10-13 22:20:10,994][60935] Updated weights for policy 0, policy_version 33990 (0.0009) [2023-10-13 22:20:11,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 70057984. Throughput: 0: 1678.2, 1: 1687.9. Samples: 17525522. Policy #0 lag: (min: 5.0, avg: 28.8, max: 32.0) [2023-10-13 22:20:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:20:11,369][60935] Updated weights for policy 0, policy_version 34000 (0.0007) [2023-10-13 22:20:11,729][60935] Updated weights for policy 0, policy_version 34010 (0.0007) [2023-10-13 22:20:14,750][60934] Updated weights for policy 1, policy_version 34442 (0.0011) [2023-10-13 22:20:15,114][60934] Updated weights for policy 1, policy_version 34452 (0.0009) [2023-10-13 22:20:15,478][60934] Updated weights for policy 1, policy_version 34462 (0.0009) [2023-10-13 22:20:15,838][60935] Updated weights for policy 0, policy_version 34020 (0.0009) [2023-10-13 22:20:16,202][60935] Updated weights for policy 0, policy_version 34030 (0.0009) [2023-10-13 22:20:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 70123520. Throughput: 0: 1678.6, 1: 1711.1. Samples: 17535686. Policy #0 lag: (min: 5.0, avg: 28.8, max: 32.0) [2023-10-13 22:20:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:20:16,582][60935] Updated weights for policy 0, policy_version 34040 (0.0007) [2023-10-13 22:20:19,544][60934] Updated weights for policy 1, policy_version 34472 (0.0007) [2023-10-13 22:20:19,922][60934] Updated weights for policy 1, policy_version 34482 (0.0008) [2023-10-13 22:20:20,296][60934] Updated weights for policy 1, policy_version 34492 (0.0009) [2023-10-13 22:20:20,624][60935] Updated weights for policy 0, policy_version 34050 (0.0008) [2023-10-13 22:20:20,994][60935] Updated weights for policy 0, policy_version 34060 (0.0007) [2023-10-13 22:20:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 70189056. Throughput: 0: 1680.5, 1: 1700.2. Samples: 17556278. Policy #0 lag: (min: 5.0, avg: 28.8, max: 32.0) [2023-10-13 22:20:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:20:21,368][60935] Updated weights for policy 0, policy_version 34070 (0.0008) [2023-10-13 22:20:21,734][60935] Updated weights for policy 0, policy_version 34080 (0.0008) [2023-10-13 22:20:24,206][60934] Updated weights for policy 1, policy_version 34502 (0.0008) [2023-10-13 22:20:24,574][60934] Updated weights for policy 1, policy_version 34512 (0.0008) [2023-10-13 22:20:24,952][60934] Updated weights for policy 1, policy_version 34522 (0.0007) [2023-10-13 22:20:25,853][60935] Updated weights for policy 0, policy_version 34090 (0.0010) [2023-10-13 22:20:26,234][60935] Updated weights for policy 0, policy_version 34100 (0.0009) [2023-10-13 22:20:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 70254592. Throughput: 0: 1672.8, 1: 1682.3. Samples: 17576046. Policy #0 lag: (min: 5.0, avg: 28.8, max: 32.0) [2023-10-13 22:20:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:20:26,606][60935] Updated weights for policy 0, policy_version 34110 (0.0009) [2023-10-13 22:20:28,968][60934] Updated weights for policy 1, policy_version 34532 (0.0009) [2023-10-13 22:20:29,344][60934] Updated weights for policy 1, policy_version 34542 (0.0009) [2023-10-13 22:20:29,711][60934] Updated weights for policy 1, policy_version 34552 (0.0007) [2023-10-13 22:20:30,617][60935] Updated weights for policy 0, policy_version 34120 (0.0010) [2023-10-13 22:20:30,979][60935] Updated weights for policy 0, policy_version 34130 (0.0010) [2023-10-13 22:20:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.5). Total num frames: 70320128. Throughput: 0: 1679.2, 1: 1714.4. Samples: 17586876. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-13 22:20:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:20:31,349][60935] Updated weights for policy 0, policy_version 34140 (0.0009) [2023-10-13 22:20:33,591][60934] Updated weights for policy 1, policy_version 34562 (0.0007) [2023-10-13 22:20:33,954][60934] Updated weights for policy 1, policy_version 34572 (0.0007) [2023-10-13 22:20:34,327][60934] Updated weights for policy 1, policy_version 34582 (0.0007) [2023-10-13 22:20:34,688][60934] Updated weights for policy 1, policy_version 34592 (0.0009) [2023-10-13 22:20:35,375][60935] Updated weights for policy 0, policy_version 34150 (0.0010) [2023-10-13 22:20:35,745][60935] Updated weights for policy 0, policy_version 34160 (0.0009) [2023-10-13 22:20:36,113][60935] Updated weights for policy 0, policy_version 34170 (0.0009) [2023-10-13 22:20:36,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 70385664. Throughput: 0: 1682.7, 1: 1698.1. Samples: 17606882. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-13 22:20:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:20:38,720][60934] Updated weights for policy 1, policy_version 34602 (0.0007) [2023-10-13 22:20:39,087][60934] Updated weights for policy 1, policy_version 34612 (0.0008) [2023-10-13 22:20:39,457][60934] Updated weights for policy 1, policy_version 34622 (0.0009) [2023-10-13 22:20:40,072][60935] Updated weights for policy 0, policy_version 34180 (0.0010) [2023-10-13 22:20:40,462][60935] Updated weights for policy 0, policy_version 34190 (0.0009) [2023-10-13 22:20:40,835][60935] Updated weights for policy 0, policy_version 34200 (0.0010) [2023-10-13 22:20:41,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 70483968. Throughput: 0: 1664.0, 1: 1695.2. Samples: 17626714. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-13 22:20:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:20:43,590][60934] Updated weights for policy 1, policy_version 34632 (0.0009) [2023-10-13 22:20:43,985][60934] Updated weights for policy 1, policy_version 34642 (0.0009) [2023-10-13 22:20:44,356][60934] Updated weights for policy 1, policy_version 34652 (0.0010) [2023-10-13 22:20:44,729][60935] Updated weights for policy 0, policy_version 34210 (0.0009) [2023-10-13 22:20:45,106][60935] Updated weights for policy 0, policy_version 34220 (0.0009) [2023-10-13 22:20:45,467][60935] Updated weights for policy 0, policy_version 34230 (0.0009) [2023-10-13 22:20:45,842][60935] Updated weights for policy 0, policy_version 34240 (0.0007) [2023-10-13 22:20:46,248][59943] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 70549504. Throughput: 0: 1684.0, 1: 1711.2. Samples: 17637692. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-13 22:20:46,249][59943] Avg episode reward: [(0, '-0.060'), (1, '0.000')] [2023-10-13 22:20:48,585][60934] Updated weights for policy 1, policy_version 34662 (0.0008) [2023-10-13 22:20:48,947][60934] Updated weights for policy 1, policy_version 34672 (0.0007) [2023-10-13 22:20:49,315][60934] Updated weights for policy 1, policy_version 34682 (0.0007) [2023-10-13 22:20:49,915][60935] Updated weights for policy 0, policy_version 34250 (0.0007) [2023-10-13 22:20:50,277][60935] Updated weights for policy 0, policy_version 34260 (0.0007) [2023-10-13 22:20:50,646][60935] Updated weights for policy 0, policy_version 34270 (0.0007) [2023-10-13 22:20:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 70615040. Throughput: 0: 1678.1, 1: 1684.2. Samples: 17657006. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 22:20:51,249][59943] Avg episode reward: [(0, '-0.060'), (1, '0.000')] [2023-10-13 22:20:53,313][60934] Updated weights for policy 1, policy_version 34692 (0.0007) [2023-10-13 22:20:53,693][60934] Updated weights for policy 1, policy_version 34702 (0.0008) [2023-10-13 22:20:54,056][60934] Updated weights for policy 1, policy_version 34712 (0.0010) [2023-10-13 22:20:54,871][60935] Updated weights for policy 0, policy_version 34280 (0.0009) [2023-10-13 22:20:55,232][60935] Updated weights for policy 0, policy_version 34290 (0.0011) [2023-10-13 22:20:55,604][60935] Updated weights for policy 0, policy_version 34300 (0.0010) [2023-10-13 22:20:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 70680576. Throughput: 0: 1659.1, 1: 1701.1. Samples: 17676732. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 22:20:56,249][59943] Avg episode reward: [(0, '-0.060'), (1, '0.000')] [2023-10-13 22:20:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000034720_35553280.pth... [2023-10-13 22:20:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000034304_35127296.pth... [2023-10-13 22:20:56,309][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000032736_33521664.pth [2023-10-13 22:20:56,310][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000033120_33914880.pth [2023-10-13 22:20:57,970][60934] Updated weights for policy 1, policy_version 34722 (0.0010) [2023-10-13 22:20:58,337][60934] Updated weights for policy 1, policy_version 34732 (0.0009) [2023-10-13 22:20:58,709][60934] Updated weights for policy 1, policy_version 34742 (0.0009) [2023-10-13 22:20:59,081][60934] Updated weights for policy 1, policy_version 34752 (0.0008) [2023-10-13 22:20:59,544][60935] Updated weights for policy 0, policy_version 34310 (0.0008) [2023-10-13 22:20:59,915][60935] Updated weights for policy 0, policy_version 34320 (0.0008) [2023-10-13 22:21:00,285][60935] Updated weights for policy 0, policy_version 34330 (0.0009) [2023-10-13 22:21:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 70746112. Throughput: 0: 1690.1, 1: 1691.3. Samples: 17687852. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 22:21:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:21:03,139][60934] Updated weights for policy 1, policy_version 34762 (0.0010) [2023-10-13 22:21:03,514][60934] Updated weights for policy 1, policy_version 34772 (0.0009) [2023-10-13 22:21:03,871][60934] Updated weights for policy 1, policy_version 34782 (0.0008) [2023-10-13 22:21:04,405][60935] Updated weights for policy 0, policy_version 34340 (0.0009) [2023-10-13 22:21:04,779][60935] Updated weights for policy 0, policy_version 34350 (0.0009) [2023-10-13 22:21:05,148][60935] Updated weights for policy 0, policy_version 34360 (0.0009) [2023-10-13 22:21:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 70811648. Throughput: 0: 1677.6, 1: 1685.4. Samples: 17707614. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 22:21:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:21:07,898][60934] Updated weights for policy 1, policy_version 34792 (0.0009) [2023-10-13 22:21:08,266][60934] Updated weights for policy 1, policy_version 34802 (0.0009) [2023-10-13 22:21:08,639][60934] Updated weights for policy 1, policy_version 34812 (0.0008) [2023-10-13 22:21:09,206][60935] Updated weights for policy 0, policy_version 34370 (0.0009) [2023-10-13 22:21:09,580][60935] Updated weights for policy 0, policy_version 34380 (0.0008) [2023-10-13 22:21:09,950][60935] Updated weights for policy 0, policy_version 34390 (0.0009) [2023-10-13 22:21:10,311][60935] Updated weights for policy 0, policy_version 34400 (0.0008) [2023-10-13 22:21:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 70877184. Throughput: 0: 1666.8, 1: 1704.0. Samples: 17727736. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 22:21:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:21:12,652][60934] Updated weights for policy 1, policy_version 34822 (0.0011) [2023-10-13 22:21:13,016][60934] Updated weights for policy 1, policy_version 34832 (0.0010) [2023-10-13 22:21:13,380][60934] Updated weights for policy 1, policy_version 34842 (0.0009) [2023-10-13 22:21:14,344][60935] Updated weights for policy 0, policy_version 34410 (0.0007) [2023-10-13 22:21:14,718][60935] Updated weights for policy 0, policy_version 34420 (0.0010) [2023-10-13 22:21:15,092][60935] Updated weights for policy 0, policy_version 34430 (0.0009) [2023-10-13 22:21:16,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 70942720. Throughput: 0: 1693.2, 1: 1674.0. Samples: 17738400. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 22:21:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:21:17,459][60934] Updated weights for policy 1, policy_version 34852 (0.0009) [2023-10-13 22:21:17,821][60934] Updated weights for policy 1, policy_version 34862 (0.0007) [2023-10-13 22:21:18,188][60934] Updated weights for policy 1, policy_version 34872 (0.0007) [2023-10-13 22:21:19,055][60935] Updated weights for policy 0, policy_version 34440 (0.0010) [2023-10-13 22:21:19,432][60935] Updated weights for policy 0, policy_version 34450 (0.0008) [2023-10-13 22:21:19,808][60935] Updated weights for policy 0, policy_version 34460 (0.0008) [2023-10-13 22:21:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 71008256. Throughput: 0: 1668.7, 1: 1694.7. Samples: 17758232. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 22:21:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:21:22,232][60934] Updated weights for policy 1, policy_version 34882 (0.0008) [2023-10-13 22:21:22,609][60934] Updated weights for policy 1, policy_version 34892 (0.0010) [2023-10-13 22:21:22,981][60934] Updated weights for policy 1, policy_version 34902 (0.0010) [2023-10-13 22:21:23,354][60934] Updated weights for policy 1, policy_version 34912 (0.0010) [2023-10-13 22:21:23,966][60935] Updated weights for policy 0, policy_version 34470 (0.0010) [2023-10-13 22:21:24,328][60935] Updated weights for policy 0, policy_version 34480 (0.0009) [2023-10-13 22:21:24,702][60935] Updated weights for policy 0, policy_version 34490 (0.0008) [2023-10-13 22:21:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 71073792. Throughput: 0: 1681.4, 1: 1698.4. Samples: 17778806. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 22:21:26,249][59943] Avg episode reward: [(0, '-0.050'), (1, '0.000')] [2023-10-13 22:21:27,347][60934] Updated weights for policy 1, policy_version 34922 (0.0008) [2023-10-13 22:21:27,717][60934] Updated weights for policy 1, policy_version 34932 (0.0007) [2023-10-13 22:21:28,086][60934] Updated weights for policy 1, policy_version 34942 (0.0008) [2023-10-13 22:21:28,811][60935] Updated weights for policy 0, policy_version 34500 (0.0008) [2023-10-13 22:21:29,202][60935] Updated weights for policy 0, policy_version 34510 (0.0009) [2023-10-13 22:21:29,572][60935] Updated weights for policy 0, policy_version 34520 (0.0008) [2023-10-13 22:21:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 71139328. Throughput: 0: 1689.1, 1: 1675.8. Samples: 17789110. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 22:21:31,249][59943] Avg episode reward: [(0, '-0.050'), (1, '0.000')] [2023-10-13 22:21:32,314][60934] Updated weights for policy 1, policy_version 34952 (0.0010) [2023-10-13 22:21:32,696][60934] Updated weights for policy 1, policy_version 34962 (0.0009) [2023-10-13 22:21:33,067][60934] Updated weights for policy 1, policy_version 34972 (0.0010) [2023-10-13 22:21:33,657][60935] Updated weights for policy 0, policy_version 34530 (0.0009) [2023-10-13 22:21:34,028][60935] Updated weights for policy 0, policy_version 34540 (0.0009) [2023-10-13 22:21:34,402][60935] Updated weights for policy 0, policy_version 34550 (0.0008) [2023-10-13 22:21:34,763][60935] Updated weights for policy 0, policy_version 34560 (0.0009) [2023-10-13 22:21:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 71204864. Throughput: 0: 1670.8, 1: 1698.9. Samples: 17808644. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 22:21:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:21:37,059][60934] Updated weights for policy 1, policy_version 34982 (0.0009) [2023-10-13 22:21:37,427][60934] Updated weights for policy 1, policy_version 34992 (0.0009) [2023-10-13 22:21:37,799][60934] Updated weights for policy 1, policy_version 35002 (0.0008) [2023-10-13 22:21:38,802][60935] Updated weights for policy 0, policy_version 34570 (0.0008) [2023-10-13 22:21:39,176][60935] Updated weights for policy 0, policy_version 34580 (0.0010) [2023-10-13 22:21:39,546][60935] Updated weights for policy 0, policy_version 34590 (0.0009) [2023-10-13 22:21:41,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 71270400. Throughput: 0: 1693.3, 1: 1704.2. Samples: 17829620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:21:41,249][59943] Avg episode reward: [(0, '-0.170'), (1, '0.000')] [2023-10-13 22:21:41,621][60934] Updated weights for policy 1, policy_version 35012 (0.0009) [2023-10-13 22:21:41,992][60934] Updated weights for policy 1, policy_version 35022 (0.0007) [2023-10-13 22:21:42,361][60934] Updated weights for policy 1, policy_version 35032 (0.0007) [2023-10-13 22:21:43,680][60935] Updated weights for policy 0, policy_version 34600 (0.0010) [2023-10-13 22:21:44,049][60935] Updated weights for policy 0, policy_version 34610 (0.0009) [2023-10-13 22:21:44,408][60935] Updated weights for policy 0, policy_version 34620 (0.0010) [2023-10-13 22:21:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 71335936. Throughput: 0: 1682.2, 1: 1693.5. Samples: 17839758. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:21:46,249][59943] Avg episode reward: [(0, '-0.170'), (1, '0.000')] [2023-10-13 22:21:46,321][60934] Updated weights for policy 1, policy_version 35042 (0.0007) [2023-10-13 22:21:46,695][60934] Updated weights for policy 1, policy_version 35052 (0.0007) [2023-10-13 22:21:47,068][60934] Updated weights for policy 1, policy_version 35062 (0.0007) [2023-10-13 22:21:47,431][60934] Updated weights for policy 1, policy_version 35072 (0.0008) [2023-10-13 22:21:48,490][60935] Updated weights for policy 0, policy_version 34630 (0.0011) [2023-10-13 22:21:48,862][60935] Updated weights for policy 0, policy_version 34640 (0.0008) [2023-10-13 22:21:49,228][60935] Updated weights for policy 0, policy_version 34650 (0.0009) [2023-10-13 22:21:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 71401472. Throughput: 0: 1671.8, 1: 1705.9. Samples: 17859610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:21:51,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:21:51,438][60934] Updated weights for policy 1, policy_version 35082 (0.0008) [2023-10-13 22:21:51,807][60934] Updated weights for policy 1, policy_version 35092 (0.0009) [2023-10-13 22:21:52,171][60934] Updated weights for policy 1, policy_version 35102 (0.0009) [2023-10-13 22:21:53,244][60935] Updated weights for policy 0, policy_version 34660 (0.0010) [2023-10-13 22:21:53,608][60935] Updated weights for policy 0, policy_version 34670 (0.0009) [2023-10-13 22:21:53,987][60935] Updated weights for policy 0, policy_version 34680 (0.0010) [2023-10-13 22:21:56,023][60934] Updated weights for policy 1, policy_version 35112 (0.0009) [2023-10-13 22:21:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 71467008. Throughput: 0: 1693.3, 1: 1706.0. Samples: 17880704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:21:56,249][59943] Avg episode reward: [(0, '-0.130'), (1, '0.000')] [2023-10-13 22:21:56,394][60934] Updated weights for policy 1, policy_version 35122 (0.0008) [2023-10-13 22:21:56,770][60934] Updated weights for policy 1, policy_version 35132 (0.0007) [2023-10-13 22:21:57,971][60935] Updated weights for policy 0, policy_version 34690 (0.0008) [2023-10-13 22:21:58,343][60935] Updated weights for policy 0, policy_version 34700 (0.0008) [2023-10-13 22:21:58,708][60935] Updated weights for policy 0, policy_version 34710 (0.0007) [2023-10-13 22:21:59,073][60935] Updated weights for policy 0, policy_version 34720 (0.0008) [2023-10-13 22:22:00,794][60934] Updated weights for policy 1, policy_version 35142 (0.0009) [2023-10-13 22:22:01,163][60934] Updated weights for policy 1, policy_version 35152 (0.0009) [2023-10-13 22:22:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 71532544. Throughput: 0: 1664.7, 1: 1705.3. Samples: 17890052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:22:01,249][59943] Avg episode reward: [(0, '-0.130'), (1, '0.000')] [2023-10-13 22:22:01,536][60934] Updated weights for policy 1, policy_version 35162 (0.0011) [2023-10-13 22:22:03,147][60935] Updated weights for policy 0, policy_version 34730 (0.0007) [2023-10-13 22:22:03,517][60935] Updated weights for policy 0, policy_version 34740 (0.0010) [2023-10-13 22:22:03,893][60935] Updated weights for policy 0, policy_version 34750 (0.0009) [2023-10-13 22:22:05,359][60934] Updated weights for policy 1, policy_version 35172 (0.0010) [2023-10-13 22:22:05,731][60934] Updated weights for policy 1, policy_version 35182 (0.0009) [2023-10-13 22:22:06,105][60934] Updated weights for policy 1, policy_version 35192 (0.0009) [2023-10-13 22:22:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 71598080. Throughput: 0: 1678.1, 1: 1712.3. Samples: 17910802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:22:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:22:08,114][60935] Updated weights for policy 0, policy_version 34760 (0.0008) [2023-10-13 22:22:08,475][60935] Updated weights for policy 0, policy_version 34770 (0.0012) [2023-10-13 22:22:08,846][60935] Updated weights for policy 0, policy_version 34780 (0.0010) [2023-10-13 22:22:10,161][60934] Updated weights for policy 1, policy_version 35202 (0.0008) [2023-10-13 22:22:10,520][60934] Updated weights for policy 1, policy_version 35212 (0.0007) [2023-10-13 22:22:10,886][60934] Updated weights for policy 1, policy_version 35222 (0.0007) [2023-10-13 22:22:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 71663616. Throughput: 0: 1678.7, 1: 1704.7. Samples: 17931058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:22:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:22:11,260][60934] Updated weights for policy 1, policy_version 35232 (0.0007) [2023-10-13 22:22:13,014][60935] Updated weights for policy 0, policy_version 34790 (0.0012) [2023-10-13 22:22:13,374][60935] Updated weights for policy 0, policy_version 34800 (0.0011) [2023-10-13 22:22:13,750][60935] Updated weights for policy 0, policy_version 34810 (0.0009) [2023-10-13 22:22:15,120][60934] Updated weights for policy 1, policy_version 35242 (0.0008) [2023-10-13 22:22:15,487][60934] Updated weights for policy 1, policy_version 35252 (0.0007) [2023-10-13 22:22:15,857][60934] Updated weights for policy 1, policy_version 35262 (0.0008) [2023-10-13 22:22:16,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 71761920. Throughput: 0: 1652.4, 1: 1722.4. Samples: 17940972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:22:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:22:17,870][60935] Updated weights for policy 0, policy_version 34820 (0.0008) [2023-10-13 22:22:18,251][60935] Updated weights for policy 0, policy_version 34830 (0.0009) [2023-10-13 22:22:18,618][60935] Updated weights for policy 0, policy_version 34840 (0.0010) [2023-10-13 22:22:20,037][60934] Updated weights for policy 1, policy_version 35272 (0.0009) [2023-10-13 22:22:20,422][60934] Updated weights for policy 1, policy_version 35282 (0.0007) [2023-10-13 22:22:20,788][60934] Updated weights for policy 1, policy_version 35292 (0.0008) [2023-10-13 22:22:21,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 71827456. Throughput: 0: 1671.4, 1: 1723.1. Samples: 17961396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:22:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:22:22,726][60935] Updated weights for policy 0, policy_version 34850 (0.0009) [2023-10-13 22:22:23,098][60935] Updated weights for policy 0, policy_version 34860 (0.0009) [2023-10-13 22:22:23,485][60935] Updated weights for policy 0, policy_version 34870 (0.0011) [2023-10-13 22:22:23,855][60935] Updated weights for policy 0, policy_version 34880 (0.0010) [2023-10-13 22:22:24,791][60934] Updated weights for policy 1, policy_version 35302 (0.0009) [2023-10-13 22:22:25,164][60934] Updated weights for policy 1, policy_version 35312 (0.0009) [2023-10-13 22:22:25,533][60934] Updated weights for policy 1, policy_version 35322 (0.0011) [2023-10-13 22:22:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 71892992. Throughput: 0: 1669.2, 1: 1693.7. Samples: 17980952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:22:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:22:28,133][60935] Updated weights for policy 0, policy_version 34890 (0.0008) [2023-10-13 22:22:28,506][60935] Updated weights for policy 0, policy_version 34900 (0.0008) [2023-10-13 22:22:28,880][60935] Updated weights for policy 0, policy_version 34910 (0.0009) [2023-10-13 22:22:29,629][60934] Updated weights for policy 1, policy_version 35332 (0.0008) [2023-10-13 22:22:29,994][60934] Updated weights for policy 1, policy_version 35342 (0.0008) [2023-10-13 22:22:30,364][60934] Updated weights for policy 1, policy_version 35352 (0.0009) [2023-10-13 22:22:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 71958528. Throughput: 0: 1651.3, 1: 1715.1. Samples: 17991246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:22:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:22:32,695][60935] Updated weights for policy 0, policy_version 34920 (0.0009) [2023-10-13 22:22:33,062][60935] Updated weights for policy 0, policy_version 34930 (0.0007) [2023-10-13 22:22:33,429][60935] Updated weights for policy 0, policy_version 34940 (0.0008) [2023-10-13 22:22:34,461][60934] Updated weights for policy 1, policy_version 35362 (0.0008) [2023-10-13 22:22:34,830][60934] Updated weights for policy 1, policy_version 35372 (0.0007) [2023-10-13 22:22:35,206][60934] Updated weights for policy 1, policy_version 35382 (0.0009) [2023-10-13 22:22:35,570][60934] Updated weights for policy 1, policy_version 35392 (0.0007) [2023-10-13 22:22:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 72024064. Throughput: 0: 1671.4, 1: 1710.6. Samples: 18011802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:22:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:22:37,586][60935] Updated weights for policy 0, policy_version 34950 (0.0008) [2023-10-13 22:22:37,953][60935] Updated weights for policy 0, policy_version 34960 (0.0008) [2023-10-13 22:22:38,330][60935] Updated weights for policy 0, policy_version 34970 (0.0008) [2023-10-13 22:22:39,556][60934] Updated weights for policy 1, policy_version 35402 (0.0008) [2023-10-13 22:22:39,926][60934] Updated weights for policy 1, policy_version 35412 (0.0008) [2023-10-13 22:22:40,292][60934] Updated weights for policy 1, policy_version 35422 (0.0009) [2023-10-13 22:22:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 72089600. Throughput: 0: 1668.2, 1: 1681.9. Samples: 18031462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:22:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:22:42,579][60935] Updated weights for policy 0, policy_version 34980 (0.0009) [2023-10-13 22:22:42,938][60935] Updated weights for policy 0, policy_version 34990 (0.0010) [2023-10-13 22:22:43,308][60935] Updated weights for policy 0, policy_version 35000 (0.0010) [2023-10-13 22:22:44,182][60934] Updated weights for policy 1, policy_version 35432 (0.0009) [2023-10-13 22:22:44,548][60934] Updated weights for policy 1, policy_version 35442 (0.0008) [2023-10-13 22:22:44,924][60934] Updated weights for policy 1, policy_version 35452 (0.0010) [2023-10-13 22:22:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 72155136. Throughput: 0: 1658.5, 1: 1714.8. Samples: 18041852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:22:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:22:47,431][60935] Updated weights for policy 0, policy_version 35010 (0.0011) [2023-10-13 22:22:47,805][60935] Updated weights for policy 0, policy_version 35020 (0.0008) [2023-10-13 22:22:48,173][60935] Updated weights for policy 0, policy_version 35030 (0.0010) [2023-10-13 22:22:48,538][60935] Updated weights for policy 0, policy_version 35040 (0.0012) [2023-10-13 22:22:49,063][60934] Updated weights for policy 1, policy_version 35462 (0.0007) [2023-10-13 22:22:49,427][60934] Updated weights for policy 1, policy_version 35472 (0.0007) [2023-10-13 22:22:49,791][60934] Updated weights for policy 1, policy_version 35482 (0.0008) [2023-10-13 22:22:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 72220672. Throughput: 0: 1669.0, 1: 1688.3. Samples: 18061882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:22:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:22:52,630][60935] Updated weights for policy 0, policy_version 35050 (0.0009) [2023-10-13 22:22:52,994][60935] Updated weights for policy 0, policy_version 35060 (0.0007) [2023-10-13 22:22:53,363][60935] Updated weights for policy 0, policy_version 35070 (0.0007) [2023-10-13 22:22:53,818][60934] Updated weights for policy 1, policy_version 35492 (0.0007) [2023-10-13 22:22:54,180][60934] Updated weights for policy 1, policy_version 35502 (0.0007) [2023-10-13 22:22:54,549][60934] Updated weights for policy 1, policy_version 35512 (0.0008) [2023-10-13 22:22:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 72286208. Throughput: 0: 1675.4, 1: 1689.4. Samples: 18082472. Policy #0 lag: (min: 12.0, avg: 12.8, max: 32.0) [2023-10-13 22:22:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:22:56,263][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000035520_36372480.pth... [2023-10-13 22:22:56,263][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000035072_35913728.pth... [2023-10-13 22:22:56,301][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000033504_34308096.pth [2023-10-13 22:22:56,302][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000033920_34734080.pth [2023-10-13 22:22:57,161][60935] Updated weights for policy 0, policy_version 35080 (0.0007) [2023-10-13 22:22:57,521][60935] Updated weights for policy 0, policy_version 35090 (0.0010) [2023-10-13 22:22:57,896][60935] Updated weights for policy 0, policy_version 35100 (0.0011) [2023-10-13 22:22:58,589][60934] Updated weights for policy 1, policy_version 35522 (0.0009) [2023-10-13 22:22:58,967][60934] Updated weights for policy 1, policy_version 35532 (0.0008) [2023-10-13 22:22:59,338][60934] Updated weights for policy 1, policy_version 35542 (0.0009) [2023-10-13 22:22:59,712][60934] Updated weights for policy 1, policy_version 35552 (0.0009) [2023-10-13 22:23:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 72351744. Throughput: 0: 1674.7, 1: 1700.6. Samples: 18092862. Policy #0 lag: (min: 12.0, avg: 12.8, max: 32.0) [2023-10-13 22:23:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:23:02,095][60935] Updated weights for policy 0, policy_version 35110 (0.0011) [2023-10-13 22:23:02,466][60935] Updated weights for policy 0, policy_version 35120 (0.0009) [2023-10-13 22:23:02,836][60935] Updated weights for policy 0, policy_version 35130 (0.0008) [2023-10-13 22:23:03,691][60934] Updated weights for policy 1, policy_version 35562 (0.0009) [2023-10-13 22:23:04,065][60934] Updated weights for policy 1, policy_version 35572 (0.0007) [2023-10-13 22:23:04,433][60934] Updated weights for policy 1, policy_version 35582 (0.0009) [2023-10-13 22:23:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 72417280. Throughput: 0: 1682.4, 1: 1677.5. Samples: 18112592. Policy #0 lag: (min: 12.0, avg: 12.8, max: 32.0) [2023-10-13 22:23:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:23:06,904][60935] Updated weights for policy 0, policy_version 35140 (0.0010) [2023-10-13 22:23:07,278][60935] Updated weights for policy 0, policy_version 35150 (0.0009) [2023-10-13 22:23:07,649][60935] Updated weights for policy 0, policy_version 35160 (0.0010) [2023-10-13 22:23:08,577][60934] Updated weights for policy 1, policy_version 35592 (0.0008) [2023-10-13 22:23:08,944][60934] Updated weights for policy 1, policy_version 35602 (0.0007) [2023-10-13 22:23:09,321][60934] Updated weights for policy 1, policy_version 35612 (0.0008) [2023-10-13 22:23:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 72482816. Throughput: 0: 1683.5, 1: 1701.5. Samples: 18133278. Policy #0 lag: (min: 12.0, avg: 12.8, max: 32.0) [2023-10-13 22:23:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:23:11,699][60935] Updated weights for policy 0, policy_version 35170 (0.0010) [2023-10-13 22:23:12,079][60935] Updated weights for policy 0, policy_version 35180 (0.0009) [2023-10-13 22:23:12,440][60935] Updated weights for policy 0, policy_version 35190 (0.0009) [2023-10-13 22:23:12,809][60935] Updated weights for policy 0, policy_version 35200 (0.0008) [2023-10-13 22:23:13,155][60934] Updated weights for policy 1, policy_version 35622 (0.0008) [2023-10-13 22:23:13,522][60934] Updated weights for policy 1, policy_version 35632 (0.0008) [2023-10-13 22:23:13,889][60934] Updated weights for policy 1, policy_version 35642 (0.0009) [2023-10-13 22:23:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 72548352. Throughput: 0: 1683.9, 1: 1691.6. Samples: 18143140. Policy #0 lag: (min: 12.0, avg: 12.8, max: 32.0) [2023-10-13 22:23:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:23:16,731][60935] Updated weights for policy 0, policy_version 35210 (0.0008) [2023-10-13 22:23:17,111][60935] Updated weights for policy 0, policy_version 35220 (0.0008) [2023-10-13 22:23:17,483][60935] Updated weights for policy 0, policy_version 35230 (0.0009) [2023-10-13 22:23:17,987][60934] Updated weights for policy 1, policy_version 35652 (0.0007) [2023-10-13 22:23:18,348][60934] Updated weights for policy 1, policy_version 35662 (0.0009) [2023-10-13 22:23:18,721][60934] Updated weights for policy 1, policy_version 35672 (0.0008) [2023-10-13 22:23:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 72613888. Throughput: 0: 1690.9, 1: 1683.2. Samples: 18163636. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-13 22:23:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:23:21,499][60935] Updated weights for policy 0, policy_version 35240 (0.0008) [2023-10-13 22:23:21,871][60935] Updated weights for policy 0, policy_version 35250 (0.0010) [2023-10-13 22:23:22,232][60935] Updated weights for policy 0, policy_version 35260 (0.0007) [2023-10-13 22:23:22,660][60934] Updated weights for policy 1, policy_version 35682 (0.0007) [2023-10-13 22:23:23,033][60934] Updated weights for policy 1, policy_version 35692 (0.0009) [2023-10-13 22:23:23,403][60934] Updated weights for policy 1, policy_version 35702 (0.0007) [2023-10-13 22:23:23,766][60934] Updated weights for policy 1, policy_version 35712 (0.0007) [2023-10-13 22:23:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 72679424. Throughput: 0: 1688.7, 1: 1712.4. Samples: 18184512. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-13 22:23:26,249][59943] Avg episode reward: [(0, '-0.050'), (1, '-0.010')] [2023-10-13 22:23:26,272][60935] Updated weights for policy 0, policy_version 35270 (0.0011) [2023-10-13 22:23:26,644][60935] Updated weights for policy 0, policy_version 35280 (0.0010) [2023-10-13 22:23:27,010][60935] Updated weights for policy 0, policy_version 35290 (0.0008) [2023-10-13 22:23:27,660][60934] Updated weights for policy 1, policy_version 35722 (0.0008) [2023-10-13 22:23:28,031][60934] Updated weights for policy 1, policy_version 35732 (0.0008) [2023-10-13 22:23:28,406][60934] Updated weights for policy 1, policy_version 35742 (0.0010) [2023-10-13 22:23:31,050][60935] Updated weights for policy 0, policy_version 35300 (0.0010) [2023-10-13 22:23:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 72744960. Throughput: 0: 1694.6, 1: 1678.7. Samples: 18193648. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-13 22:23:31,249][59943] Avg episode reward: [(0, '-0.050'), (1, '-0.010')] [2023-10-13 22:23:31,419][60935] Updated weights for policy 0, policy_version 35310 (0.0011) [2023-10-13 22:23:31,796][60935] Updated weights for policy 0, policy_version 35320 (0.0010) [2023-10-13 22:23:32,307][60934] Updated weights for policy 1, policy_version 35752 (0.0008) [2023-10-13 22:23:32,682][60934] Updated weights for policy 1, policy_version 35762 (0.0010) [2023-10-13 22:23:33,040][60934] Updated weights for policy 1, policy_version 35772 (0.0009) [2023-10-13 22:23:35,846][60935] Updated weights for policy 0, policy_version 35330 (0.0009) [2023-10-13 22:23:36,214][60935] Updated weights for policy 0, policy_version 35340 (0.0010) [2023-10-13 22:23:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 72810496. Throughput: 0: 1690.5, 1: 1702.8. Samples: 18214580. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-13 22:23:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:23:36,588][60935] Updated weights for policy 0, policy_version 35350 (0.0009) [2023-10-13 22:23:36,955][60935] Updated weights for policy 0, policy_version 35360 (0.0009) [2023-10-13 22:23:37,001][60934] Updated weights for policy 1, policy_version 35782 (0.0007) [2023-10-13 22:23:37,368][60934] Updated weights for policy 1, policy_version 35792 (0.0008) [2023-10-13 22:23:37,737][60934] Updated weights for policy 1, policy_version 35802 (0.0008) [2023-10-13 22:23:41,155][60935] Updated weights for policy 0, policy_version 35370 (0.0009) [2023-10-13 22:23:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 72876032. Throughput: 0: 1683.8, 1: 1717.1. Samples: 18235514. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-13 22:23:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:23:41,522][60935] Updated weights for policy 0, policy_version 35380 (0.0011) [2023-10-13 22:23:41,535][60934] Updated weights for policy 1, policy_version 35812 (0.0008) [2023-10-13 22:23:41,895][60935] Updated weights for policy 0, policy_version 35390 (0.0008) [2023-10-13 22:23:41,899][60934] Updated weights for policy 1, policy_version 35822 (0.0009) [2023-10-13 22:23:42,271][60934] Updated weights for policy 1, policy_version 35832 (0.0010) [2023-10-13 22:23:45,782][60935] Updated weights for policy 0, policy_version 35400 (0.0009) [2023-10-13 22:23:46,149][60935] Updated weights for policy 0, policy_version 35410 (0.0009) [2023-10-13 22:23:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 72941568. Throughput: 0: 1685.5, 1: 1693.6. Samples: 18244920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:23:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:23:46,334][60934] Updated weights for policy 1, policy_version 35842 (0.0009) [2023-10-13 22:23:46,522][60935] Updated weights for policy 0, policy_version 35420 (0.0009) [2023-10-13 22:23:46,699][60934] Updated weights for policy 1, policy_version 35852 (0.0008) [2023-10-13 22:23:47,076][60934] Updated weights for policy 1, policy_version 35862 (0.0009) [2023-10-13 22:23:47,435][60934] Updated weights for policy 1, policy_version 35872 (0.0009) [2023-10-13 22:23:50,713][60935] Updated weights for policy 0, policy_version 35430 (0.0009) [2023-10-13 22:23:51,087][60935] Updated weights for policy 0, policy_version 35440 (0.0008) [2023-10-13 22:23:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 73007104. Throughput: 0: 1689.3, 1: 1718.7. Samples: 18265950. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:23:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:23:51,450][60935] Updated weights for policy 0, policy_version 35450 (0.0007) [2023-10-13 22:23:51,456][60934] Updated weights for policy 1, policy_version 35882 (0.0008) [2023-10-13 22:23:51,817][60934] Updated weights for policy 1, policy_version 35892 (0.0010) [2023-10-13 22:23:52,185][60934] Updated weights for policy 1, policy_version 35902 (0.0007) [2023-10-13 22:23:55,411][60935] Updated weights for policy 0, policy_version 35460 (0.0008) [2023-10-13 22:23:55,798][60935] Updated weights for policy 0, policy_version 35470 (0.0010) [2023-10-13 22:23:56,004][60934] Updated weights for policy 1, policy_version 35912 (0.0007) [2023-10-13 22:23:56,162][60935] Updated weights for policy 0, policy_version 35480 (0.0009) [2023-10-13 22:23:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 73072640. Throughput: 0: 1674.3, 1: 1733.9. Samples: 18286646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:23:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:23:56,375][60934] Updated weights for policy 1, policy_version 35922 (0.0007) [2023-10-13 22:23:56,742][60934] Updated weights for policy 1, policy_version 35932 (0.0008) [2023-10-13 22:24:00,197][60935] Updated weights for policy 0, policy_version 35490 (0.0009) [2023-10-13 22:24:00,568][60935] Updated weights for policy 0, policy_version 35500 (0.0007) [2023-10-13 22:24:00,685][60934] Updated weights for policy 1, policy_version 35942 (0.0007) [2023-10-13 22:24:00,933][60935] Updated weights for policy 0, policy_version 35510 (0.0007) [2023-10-13 22:24:01,053][60934] Updated weights for policy 1, policy_version 35952 (0.0008) [2023-10-13 22:24:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 73138176. Throughput: 0: 1685.7, 1: 1720.9. Samples: 18296436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:24:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:24:01,305][60935] Updated weights for policy 0, policy_version 35520 (0.0010) [2023-10-13 22:24:01,425][60934] Updated weights for policy 1, policy_version 35962 (0.0008) [2023-10-13 22:24:05,402][60934] Updated weights for policy 1, policy_version 35972 (0.0008) [2023-10-13 22:24:05,450][60935] Updated weights for policy 0, policy_version 35530 (0.0008) [2023-10-13 22:24:05,773][60934] Updated weights for policy 1, policy_version 35982 (0.0007) [2023-10-13 22:24:05,811][60935] Updated weights for policy 0, policy_version 35540 (0.0009) [2023-10-13 22:24:06,133][60934] Updated weights for policy 1, policy_version 35992 (0.0008) [2023-10-13 22:24:06,179][60935] Updated weights for policy 0, policy_version 35550 (0.0009) [2023-10-13 22:24:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 73203712. Throughput: 0: 1679.6, 1: 1735.0. Samples: 18317294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:24:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:24:10,089][60934] Updated weights for policy 1, policy_version 36002 (0.0008) [2023-10-13 22:24:10,361][60935] Updated weights for policy 0, policy_version 35560 (0.0008) [2023-10-13 22:24:10,466][60934] Updated weights for policy 1, policy_version 36012 (0.0010) [2023-10-13 22:24:10,724][60935] Updated weights for policy 0, policy_version 35570 (0.0009) [2023-10-13 22:24:10,826][60934] Updated weights for policy 1, policy_version 36022 (0.0008) [2023-10-13 22:24:11,097][60935] Updated weights for policy 0, policy_version 35580 (0.0007) [2023-10-13 22:24:11,196][60934] Updated weights for policy 1, policy_version 36032 (0.0008) [2023-10-13 22:24:11,248][59943] Fps is (10 sec: 19660.9, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 73334784. Throughput: 0: 1660.5, 1: 1725.3. Samples: 18336870. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 22:24:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:24:15,163][60934] Updated weights for policy 1, policy_version 36042 (0.0007) [2023-10-13 22:24:15,259][60935] Updated weights for policy 0, policy_version 35590 (0.0008) [2023-10-13 22:24:15,530][60934] Updated weights for policy 1, policy_version 36052 (0.0009) [2023-10-13 22:24:15,632][60935] Updated weights for policy 0, policy_version 35600 (0.0007) [2023-10-13 22:24:15,905][60934] Updated weights for policy 1, policy_version 36062 (0.0008) [2023-10-13 22:24:15,997][60935] Updated weights for policy 0, policy_version 35610 (0.0008) [2023-10-13 22:24:16,248][59943] Fps is (10 sec: 19660.9, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 73400320. Throughput: 0: 1675.9, 1: 1739.4. Samples: 18347336. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 22:24:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:24:20,090][60934] Updated weights for policy 1, policy_version 36072 (0.0009) [2023-10-13 22:24:20,096][60935] Updated weights for policy 0, policy_version 35620 (0.0009) [2023-10-13 22:24:20,445][60934] Updated weights for policy 1, policy_version 36082 (0.0007) [2023-10-13 22:24:20,456][60935] Updated weights for policy 0, policy_version 35630 (0.0010) [2023-10-13 22:24:20,820][60934] Updated weights for policy 1, policy_version 36092 (0.0007) [2023-10-13 22:24:20,834][60935] Updated weights for policy 0, policy_version 35640 (0.0008) [2023-10-13 22:24:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 73465856. Throughput: 0: 1674.8, 1: 1736.0. Samples: 18368066. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 22:24:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:24:24,684][60934] Updated weights for policy 1, policy_version 36102 (0.0008) [2023-10-13 22:24:25,048][60934] Updated weights for policy 1, policy_version 36112 (0.0009) [2023-10-13 22:24:25,168][60935] Updated weights for policy 0, policy_version 35650 (0.0008) [2023-10-13 22:24:25,417][60934] Updated weights for policy 1, policy_version 36122 (0.0008) [2023-10-13 22:24:25,529][60935] Updated weights for policy 0, policy_version 35660 (0.0009) [2023-10-13 22:24:25,902][60935] Updated weights for policy 0, policy_version 35670 (0.0009) [2023-10-13 22:24:26,248][59943] Fps is (10 sec: 9830.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 73498624. Throughput: 0: 1655.8, 1: 1704.1. Samples: 18386708. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 22:24:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:24:26,275][60935] Updated weights for policy 0, policy_version 35680 (0.0009) [2023-10-13 22:24:29,489][60934] Updated weights for policy 1, policy_version 36132 (0.0007) [2023-10-13 22:24:29,853][60934] Updated weights for policy 1, policy_version 36142 (0.0010) [2023-10-13 22:24:30,216][60934] Updated weights for policy 1, policy_version 36152 (0.0007) [2023-10-13 22:24:30,327][60935] Updated weights for policy 0, policy_version 35690 (0.0009) [2023-10-13 22:24:30,683][60935] Updated weights for policy 0, policy_version 35700 (0.0010) [2023-10-13 22:24:31,068][60935] Updated weights for policy 0, policy_version 35710 (0.0007) [2023-10-13 22:24:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 73596928. Throughput: 0: 1670.2, 1: 1730.1. Samples: 18397932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:24:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:24:34,023][60934] Updated weights for policy 1, policy_version 36162 (0.0007) [2023-10-13 22:24:34,384][60934] Updated weights for policy 1, policy_version 36172 (0.0009) [2023-10-13 22:24:34,749][60934] Updated weights for policy 1, policy_version 36182 (0.0009) [2023-10-13 22:24:35,113][60934] Updated weights for policy 1, policy_version 36192 (0.0008) [2023-10-13 22:24:35,247][60935] Updated weights for policy 0, policy_version 35720 (0.0008) [2023-10-13 22:24:35,615][60935] Updated weights for policy 0, policy_version 35730 (0.0010) [2023-10-13 22:24:35,982][60935] Updated weights for policy 0, policy_version 35740 (0.0007) [2023-10-13 22:24:36,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 73662464. Throughput: 0: 1662.1, 1: 1715.5. Samples: 18417944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:24:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:24:39,198][60934] Updated weights for policy 1, policy_version 36202 (0.0007) [2023-10-13 22:24:39,567][60934] Updated weights for policy 1, policy_version 36212 (0.0009) [2023-10-13 22:24:39,939][60934] Updated weights for policy 1, policy_version 36222 (0.0008) [2023-10-13 22:24:40,036][60935] Updated weights for policy 0, policy_version 35750 (0.0007) [2023-10-13 22:24:40,405][60935] Updated weights for policy 0, policy_version 35760 (0.0009) [2023-10-13 22:24:40,781][60935] Updated weights for policy 0, policy_version 35770 (0.0007) [2023-10-13 22:24:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 73728000. Throughput: 0: 1651.7, 1: 1686.8. Samples: 18436880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:24:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:24:44,026][60934] Updated weights for policy 1, policy_version 36232 (0.0009) [2023-10-13 22:24:44,391][60934] Updated weights for policy 1, policy_version 36242 (0.0007) [2023-10-13 22:24:44,764][60934] Updated weights for policy 1, policy_version 36252 (0.0009) [2023-10-13 22:24:44,814][60935] Updated weights for policy 0, policy_version 35780 (0.0008) [2023-10-13 22:24:45,199][60935] Updated weights for policy 0, policy_version 35790 (0.0009) [2023-10-13 22:24:45,561][60935] Updated weights for policy 0, policy_version 35800 (0.0009) [2023-10-13 22:24:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 73793536. Throughput: 0: 1664.0, 1: 1714.9. Samples: 18448484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:24:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:24:48,898][60934] Updated weights for policy 1, policy_version 36262 (0.0007) [2023-10-13 22:24:49,269][60934] Updated weights for policy 1, policy_version 36272 (0.0008) [2023-10-13 22:24:49,633][60934] Updated weights for policy 1, policy_version 36282 (0.0009) [2023-10-13 22:24:49,689][60935] Updated weights for policy 0, policy_version 35810 (0.0008) [2023-10-13 22:24:50,058][60935] Updated weights for policy 0, policy_version 35820 (0.0009) [2023-10-13 22:24:50,426][60935] Updated weights for policy 0, policy_version 35830 (0.0010) [2023-10-13 22:24:50,798][60935] Updated weights for policy 0, policy_version 35840 (0.0010) [2023-10-13 22:24:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 73859072. Throughput: 0: 1658.7, 1: 1685.1. Samples: 18467768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:24:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:24:53,637][60934] Updated weights for policy 1, policy_version 36292 (0.0007) [2023-10-13 22:24:53,993][60934] Updated weights for policy 1, policy_version 36302 (0.0008) [2023-10-13 22:24:54,356][60934] Updated weights for policy 1, policy_version 36312 (0.0009) [2023-10-13 22:24:54,791][60935] Updated weights for policy 0, policy_version 35850 (0.0009) [2023-10-13 22:24:55,169][60935] Updated weights for policy 0, policy_version 35860 (0.0009) [2023-10-13 22:24:55,533][60935] Updated weights for policy 0, policy_version 35870 (0.0009) [2023-10-13 22:24:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 73924608. Throughput: 0: 1656.9, 1: 1686.3. Samples: 18487316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:24:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:24:56,259][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000035872_36732928.pth... [2023-10-13 22:24:56,259][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000036320_37191680.pth... [2023-10-13 22:24:56,296][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000034304_35127296.pth [2023-10-13 22:24:56,297][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000034720_35553280.pth [2023-10-13 22:24:58,411][60934] Updated weights for policy 1, policy_version 36322 (0.0008) [2023-10-13 22:24:58,774][60934] Updated weights for policy 1, policy_version 36332 (0.0008) [2023-10-13 22:24:59,153][60934] Updated weights for policy 1, policy_version 36342 (0.0009) [2023-10-13 22:24:59,455][60935] Updated weights for policy 0, policy_version 35880 (0.0008) [2023-10-13 22:24:59,517][60934] Updated weights for policy 1, policy_version 36352 (0.0009) [2023-10-13 22:24:59,824][60935] Updated weights for policy 0, policy_version 35890 (0.0010) [2023-10-13 22:25:00,197][60935] Updated weights for policy 0, policy_version 35900 (0.0007) [2023-10-13 22:25:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 73990144. Throughput: 0: 1671.1, 1: 1698.2. Samples: 18498954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:25:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:25:03,478][60934] Updated weights for policy 1, policy_version 36362 (0.0008) [2023-10-13 22:25:03,845][60934] Updated weights for policy 1, policy_version 36372 (0.0009) [2023-10-13 22:25:04,208][60934] Updated weights for policy 1, policy_version 36382 (0.0007) [2023-10-13 22:25:04,467][60935] Updated weights for policy 0, policy_version 35910 (0.0008) [2023-10-13 22:25:04,837][60935] Updated weights for policy 0, policy_version 35920 (0.0009) [2023-10-13 22:25:05,208][60935] Updated weights for policy 0, policy_version 35930 (0.0009) [2023-10-13 22:25:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 74055680. Throughput: 0: 1660.2, 1: 1678.4. Samples: 18518300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:25:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:25:08,287][60934] Updated weights for policy 1, policy_version 36392 (0.0008) [2023-10-13 22:25:08,657][60934] Updated weights for policy 1, policy_version 36402 (0.0009) [2023-10-13 22:25:09,028][60934] Updated weights for policy 1, policy_version 36412 (0.0010) [2023-10-13 22:25:09,159][60935] Updated weights for policy 0, policy_version 35940 (0.0009) [2023-10-13 22:25:09,534][60935] Updated weights for policy 0, policy_version 35950 (0.0009) [2023-10-13 22:25:09,916][60935] Updated weights for policy 0, policy_version 35960 (0.0007) [2023-10-13 22:25:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 74121216. Throughput: 0: 1670.5, 1: 1701.9. Samples: 18538466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:25:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:25:12,990][60934] Updated weights for policy 1, policy_version 36422 (0.0008) [2023-10-13 22:25:13,350][60934] Updated weights for policy 1, policy_version 36432 (0.0010) [2023-10-13 22:25:13,725][60934] Updated weights for policy 1, policy_version 36442 (0.0007) [2023-10-13 22:25:14,151][60935] Updated weights for policy 0, policy_version 35970 (0.0008) [2023-10-13 22:25:14,508][60935] Updated weights for policy 0, policy_version 35980 (0.0008) [2023-10-13 22:25:14,889][60935] Updated weights for policy 0, policy_version 35990 (0.0011) [2023-10-13 22:25:15,261][60935] Updated weights for policy 0, policy_version 36000 (0.0010) [2023-10-13 22:25:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 74186752. Throughput: 0: 1683.6, 1: 1685.1. Samples: 18549522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:25:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:25:17,790][60934] Updated weights for policy 1, policy_version 36452 (0.0010) [2023-10-13 22:25:18,165][60934] Updated weights for policy 1, policy_version 36462 (0.0007) [2023-10-13 22:25:18,536][60934] Updated weights for policy 1, policy_version 36472 (0.0009) [2023-10-13 22:25:19,255][60935] Updated weights for policy 0, policy_version 36010 (0.0007) [2023-10-13 22:25:19,632][60935] Updated weights for policy 0, policy_version 36020 (0.0009) [2023-10-13 22:25:19,997][60935] Updated weights for policy 0, policy_version 36030 (0.0008) [2023-10-13 22:25:21,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 74252288. Throughput: 0: 1665.4, 1: 1687.3. Samples: 18568816. Policy #0 lag: (min: 43.0, avg: 55.6, max: 56.0) [2023-10-13 22:25:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:25:22,517][60934] Updated weights for policy 1, policy_version 36482 (0.0011) [2023-10-13 22:25:22,892][60934] Updated weights for policy 1, policy_version 36492 (0.0008) [2023-10-13 22:25:23,263][60934] Updated weights for policy 1, policy_version 36502 (0.0008) [2023-10-13 22:25:23,618][60934] Updated weights for policy 1, policy_version 36512 (0.0008) [2023-10-13 22:25:24,086][60935] Updated weights for policy 0, policy_version 36040 (0.0009) [2023-10-13 22:25:24,451][60935] Updated weights for policy 0, policy_version 36050 (0.0008) [2023-10-13 22:25:24,821][60935] Updated weights for policy 0, policy_version 36060 (0.0008) [2023-10-13 22:25:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 74317824. Throughput: 0: 1680.8, 1: 1710.6. Samples: 18589492. Policy #0 lag: (min: 43.0, avg: 55.6, max: 56.0) [2023-10-13 22:25:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:25:27,670][60934] Updated weights for policy 1, policy_version 36522 (0.0009) [2023-10-13 22:25:28,040][60934] Updated weights for policy 1, policy_version 36532 (0.0009) [2023-10-13 22:25:28,407][60934] Updated weights for policy 1, policy_version 36542 (0.0007) [2023-10-13 22:25:28,922][60935] Updated weights for policy 0, policy_version 36070 (0.0009) [2023-10-13 22:25:29,294][60935] Updated weights for policy 0, policy_version 36080 (0.0007) [2023-10-13 22:25:29,676][60935] Updated weights for policy 0, policy_version 36090 (0.0008) [2023-10-13 22:25:31,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 74383360. Throughput: 0: 1679.2, 1: 1678.6. Samples: 18599584. Policy #0 lag: (min: 43.0, avg: 55.6, max: 56.0) [2023-10-13 22:25:31,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:25:32,475][60934] Updated weights for policy 1, policy_version 36552 (0.0008) [2023-10-13 22:25:32,845][60934] Updated weights for policy 1, policy_version 36562 (0.0008) [2023-10-13 22:25:33,214][60934] Updated weights for policy 1, policy_version 36572 (0.0009) [2023-10-13 22:25:33,634][60935] Updated weights for policy 0, policy_version 36100 (0.0007) [2023-10-13 22:25:34,013][60935] Updated weights for policy 0, policy_version 36110 (0.0009) [2023-10-13 22:25:34,385][60935] Updated weights for policy 0, policy_version 36120 (0.0008) [2023-10-13 22:25:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 74448896. Throughput: 0: 1660.2, 1: 1711.0. Samples: 18619472. Policy #0 lag: (min: 43.0, avg: 55.6, max: 56.0) [2023-10-13 22:25:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:25:37,035][60934] Updated weights for policy 1, policy_version 36582 (0.0009) [2023-10-13 22:25:37,393][60934] Updated weights for policy 1, policy_version 36592 (0.0009) [2023-10-13 22:25:37,764][60934] Updated weights for policy 1, policy_version 36602 (0.0008) [2023-10-13 22:25:38,525][60935] Updated weights for policy 0, policy_version 36130 (0.0009) [2023-10-13 22:25:38,911][60935] Updated weights for policy 0, policy_version 36140 (0.0007) [2023-10-13 22:25:39,278][60935] Updated weights for policy 0, policy_version 36150 (0.0010) [2023-10-13 22:25:39,650][60935] Updated weights for policy 0, policy_version 36160 (0.0008) [2023-10-13 22:25:41,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 74514432. Throughput: 0: 1674.1, 1: 1723.6. Samples: 18640214. Policy #0 lag: (min: 43.0, avg: 55.6, max: 56.0) [2023-10-13 22:25:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:25:41,636][60934] Updated weights for policy 1, policy_version 36612 (0.0008) [2023-10-13 22:25:41,999][60934] Updated weights for policy 1, policy_version 36622 (0.0010) [2023-10-13 22:25:42,371][60934] Updated weights for policy 1, policy_version 36632 (0.0007) [2023-10-13 22:25:43,556][60935] Updated weights for policy 0, policy_version 36170 (0.0010) [2023-10-13 22:25:43,927][60935] Updated weights for policy 0, policy_version 36180 (0.0008) [2023-10-13 22:25:44,297][60935] Updated weights for policy 0, policy_version 36190 (0.0008) [2023-10-13 22:25:46,238][60934] Updated weights for policy 1, policy_version 36642 (0.0007) [2023-10-13 22:25:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 74579968. Throughput: 0: 1661.5, 1: 1701.4. Samples: 18650282. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-13 22:25:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:25:46,600][60934] Updated weights for policy 1, policy_version 36652 (0.0007) [2023-10-13 22:25:46,971][60934] Updated weights for policy 1, policy_version 36662 (0.0009) [2023-10-13 22:25:47,344][60934] Updated weights for policy 1, policy_version 36672 (0.0010) [2023-10-13 22:25:48,359][60935] Updated weights for policy 0, policy_version 36200 (0.0009) [2023-10-13 22:25:48,726][60935] Updated weights for policy 0, policy_version 36210 (0.0010) [2023-10-13 22:25:49,097][60935] Updated weights for policy 0, policy_version 36220 (0.0009) [2023-10-13 22:25:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13440.5). Total num frames: 74645504. Throughput: 0: 1661.0, 1: 1721.1. Samples: 18670494. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-13 22:25:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:25:51,414][60934] Updated weights for policy 1, policy_version 36682 (0.0009) [2023-10-13 22:25:51,778][60934] Updated weights for policy 1, policy_version 36692 (0.0008) [2023-10-13 22:25:52,156][60934] Updated weights for policy 1, policy_version 36702 (0.0010) [2023-10-13 22:25:53,181][60935] Updated weights for policy 0, policy_version 36230 (0.0009) [2023-10-13 22:25:53,544][60935] Updated weights for policy 0, policy_version 36240 (0.0009) [2023-10-13 22:25:53,915][60935] Updated weights for policy 0, policy_version 36250 (0.0007) [2023-10-13 22:25:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 74711040. Throughput: 0: 1675.8, 1: 1725.1. Samples: 18691506. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-13 22:25:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:25:56,309][60934] Updated weights for policy 1, policy_version 36712 (0.0010) [2023-10-13 22:25:56,683][60934] Updated weights for policy 1, policy_version 36722 (0.0007) [2023-10-13 22:25:57,044][60934] Updated weights for policy 1, policy_version 36732 (0.0008) [2023-10-13 22:25:58,135][60935] Updated weights for policy 0, policy_version 36260 (0.0008) [2023-10-13 22:25:58,496][60935] Updated weights for policy 0, policy_version 36270 (0.0008) [2023-10-13 22:25:58,869][60935] Updated weights for policy 0, policy_version 36280 (0.0008) [2023-10-13 22:26:00,893][60934] Updated weights for policy 1, policy_version 36742 (0.0009) [2023-10-13 22:26:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 74776576. Throughput: 0: 1652.7, 1: 1713.3. Samples: 18700992. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-13 22:26:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:26:01,265][60934] Updated weights for policy 1, policy_version 36752 (0.0007) [2023-10-13 22:26:01,639][60934] Updated weights for policy 1, policy_version 36762 (0.0007) [2023-10-13 22:26:02,879][60935] Updated weights for policy 0, policy_version 36290 (0.0008) [2023-10-13 22:26:03,242][60935] Updated weights for policy 0, policy_version 36300 (0.0008) [2023-10-13 22:26:03,620][60935] Updated weights for policy 0, policy_version 36310 (0.0008) [2023-10-13 22:26:03,994][60935] Updated weights for policy 0, policy_version 36320 (0.0009) [2023-10-13 22:26:05,790][60934] Updated weights for policy 1, policy_version 36772 (0.0008) [2023-10-13 22:26:06,150][60934] Updated weights for policy 1, policy_version 36782 (0.0007) [2023-10-13 22:26:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 74842112. Throughput: 0: 1664.4, 1: 1727.9. Samples: 18721466. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-13 22:26:06,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:26:06,524][60934] Updated weights for policy 1, policy_version 36792 (0.0008) [2023-10-13 22:26:08,046][60935] Updated weights for policy 0, policy_version 36330 (0.0009) [2023-10-13 22:26:08,404][60935] Updated weights for policy 0, policy_version 36340 (0.0009) [2023-10-13 22:26:08,771][60935] Updated weights for policy 0, policy_version 36350 (0.0008) [2023-10-13 22:26:10,107][60934] Updated weights for policy 1, policy_version 36802 (0.0008) [2023-10-13 22:26:10,481][60934] Updated weights for policy 1, policy_version 36812 (0.0007) [2023-10-13 22:26:10,846][60934] Updated weights for policy 1, policy_version 36822 (0.0007) [2023-10-13 22:26:11,207][60934] Updated weights for policy 1, policy_version 36832 (0.0010) [2023-10-13 22:26:11,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 74940416. Throughput: 0: 1674.6, 1: 1715.6. Samples: 18742054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:26:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:26:12,906][60935] Updated weights for policy 0, policy_version 36360 (0.0009) [2023-10-13 22:26:13,281][60935] Updated weights for policy 0, policy_version 36370 (0.0008) [2023-10-13 22:26:13,649][60935] Updated weights for policy 0, policy_version 36380 (0.0007) [2023-10-13 22:26:15,414][60934] Updated weights for policy 1, policy_version 36842 (0.0009) [2023-10-13 22:26:15,780][60934] Updated weights for policy 1, policy_version 36852 (0.0009) [2023-10-13 22:26:16,156][60934] Updated weights for policy 1, policy_version 36862 (0.0008) [2023-10-13 22:26:16,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 75005952. Throughput: 0: 1650.1, 1: 1728.2. Samples: 18751608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:26:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:26:17,908][60935] Updated weights for policy 0, policy_version 36390 (0.0008) [2023-10-13 22:26:18,281][60935] Updated weights for policy 0, policy_version 36400 (0.0010) [2023-10-13 22:26:18,650][60935] Updated weights for policy 0, policy_version 36410 (0.0009) [2023-10-13 22:26:20,256][60934] Updated weights for policy 1, policy_version 36872 (0.0011) [2023-10-13 22:26:20,634][60934] Updated weights for policy 1, policy_version 36882 (0.0007) [2023-10-13 22:26:20,994][60934] Updated weights for policy 1, policy_version 36892 (0.0007) [2023-10-13 22:26:21,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 75071488. Throughput: 0: 1673.6, 1: 1724.3. Samples: 18772378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:26:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:26:22,830][60935] Updated weights for policy 0, policy_version 36420 (0.0008) [2023-10-13 22:26:23,212][60935] Updated weights for policy 0, policy_version 36430 (0.0010) [2023-10-13 22:26:23,584][60935] Updated weights for policy 0, policy_version 36440 (0.0010) [2023-10-13 22:26:24,874][60934] Updated weights for policy 1, policy_version 36902 (0.0009) [2023-10-13 22:26:25,239][60934] Updated weights for policy 1, policy_version 36912 (0.0010) [2023-10-13 22:26:25,602][60934] Updated weights for policy 1, policy_version 36922 (0.0010) [2023-10-13 22:26:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 75137024. Throughput: 0: 1680.4, 1: 1695.2. Samples: 18792116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:26:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:26:27,734][60935] Updated weights for policy 0, policy_version 36450 (0.0007) [2023-10-13 22:26:28,128][60935] Updated weights for policy 0, policy_version 36460 (0.0008) [2023-10-13 22:26:28,497][60935] Updated weights for policy 0, policy_version 36470 (0.0007) [2023-10-13 22:26:28,869][60935] Updated weights for policy 0, policy_version 36480 (0.0009) [2023-10-13 22:26:29,547][60934] Updated weights for policy 1, policy_version 36932 (0.0009) [2023-10-13 22:26:29,917][60934] Updated weights for policy 1, policy_version 36942 (0.0009) [2023-10-13 22:26:30,277][60934] Updated weights for policy 1, policy_version 36952 (0.0007) [2023-10-13 22:26:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 75202560. Throughput: 0: 1664.1, 1: 1709.7. Samples: 18802104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:26:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:26:32,734][60935] Updated weights for policy 0, policy_version 36490 (0.0009) [2023-10-13 22:26:33,110][60935] Updated weights for policy 0, policy_version 36500 (0.0010) [2023-10-13 22:26:33,474][60935] Updated weights for policy 0, policy_version 36510 (0.0007) [2023-10-13 22:26:34,374][60934] Updated weights for policy 1, policy_version 36962 (0.0009) [2023-10-13 22:26:34,735][60934] Updated weights for policy 1, policy_version 36972 (0.0010) [2023-10-13 22:26:35,097][60934] Updated weights for policy 1, policy_version 36982 (0.0009) [2023-10-13 22:26:35,463][60934] Updated weights for policy 1, policy_version 36992 (0.0008) [2023-10-13 22:26:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 75268096. Throughput: 0: 1678.0, 1: 1706.3. Samples: 18822786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:26:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:26:37,599][60935] Updated weights for policy 0, policy_version 36520 (0.0011) [2023-10-13 22:26:37,971][60935] Updated weights for policy 0, policy_version 36530 (0.0009) [2023-10-13 22:26:38,337][60935] Updated weights for policy 0, policy_version 36540 (0.0010) [2023-10-13 22:26:39,492][60934] Updated weights for policy 1, policy_version 37002 (0.0009) [2023-10-13 22:26:39,858][60934] Updated weights for policy 1, policy_version 37012 (0.0009) [2023-10-13 22:26:40,218][60934] Updated weights for policy 1, policy_version 37022 (0.0010) [2023-10-13 22:26:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 75333632. Throughput: 0: 1683.0, 1: 1681.1. Samples: 18842890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:26:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:26:42,242][60935] Updated weights for policy 0, policy_version 36550 (0.0011) [2023-10-13 22:26:42,619][60935] Updated weights for policy 0, policy_version 36560 (0.0008) [2023-10-13 22:26:42,997][60935] Updated weights for policy 0, policy_version 36570 (0.0008) [2023-10-13 22:26:44,265][60934] Updated weights for policy 1, policy_version 37032 (0.0008) [2023-10-13 22:26:44,631][60934] Updated weights for policy 1, policy_version 37042 (0.0008) [2023-10-13 22:26:45,006][60934] Updated weights for policy 1, policy_version 37052 (0.0008) [2023-10-13 22:26:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 75399168. Throughput: 0: 1678.0, 1: 1709.3. Samples: 18853422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:26:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:26:47,150][60935] Updated weights for policy 0, policy_version 36580 (0.0010) [2023-10-13 22:26:47,530][60935] Updated weights for policy 0, policy_version 36590 (0.0011) [2023-10-13 22:26:47,900][60935] Updated weights for policy 0, policy_version 36600 (0.0010) [2023-10-13 22:26:49,127][60934] Updated weights for policy 1, policy_version 37062 (0.0007) [2023-10-13 22:26:49,489][60934] Updated weights for policy 1, policy_version 37072 (0.0010) [2023-10-13 22:26:49,850][60934] Updated weights for policy 1, policy_version 37082 (0.0007) [2023-10-13 22:26:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 75464704. Throughput: 0: 1688.1, 1: 1688.8. Samples: 18873426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:26:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:26:52,017][60935] Updated weights for policy 0, policy_version 36610 (0.0010) [2023-10-13 22:26:52,388][60935] Updated weights for policy 0, policy_version 36620 (0.0009) [2023-10-13 22:26:52,749][60935] Updated weights for policy 0, policy_version 36630 (0.0008) [2023-10-13 22:26:53,120][60935] Updated weights for policy 0, policy_version 36640 (0.0009) [2023-10-13 22:26:53,704][60934] Updated weights for policy 1, policy_version 37092 (0.0008) [2023-10-13 22:26:54,076][60934] Updated weights for policy 1, policy_version 37102 (0.0007) [2023-10-13 22:26:54,440][60934] Updated weights for policy 1, policy_version 37112 (0.0009) [2023-10-13 22:26:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 75530240. Throughput: 0: 1689.9, 1: 1685.8. Samples: 18893960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:26:56,249][59943] Avg episode reward: [(0, '-0.160'), (1, '0.000')] [2023-10-13 22:26:56,258][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000037120_38010880.pth... [2023-10-13 22:26:56,258][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000036640_37519360.pth... [2023-10-13 22:26:56,297][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000035072_35913728.pth [2023-10-13 22:26:56,300][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000035520_36372480.pth [2023-10-13 22:26:57,149][60935] Updated weights for policy 0, policy_version 36650 (0.0008) [2023-10-13 22:26:57,520][60935] Updated weights for policy 0, policy_version 36660 (0.0008) [2023-10-13 22:26:57,890][60935] Updated weights for policy 0, policy_version 36670 (0.0009) [2023-10-13 22:26:58,359][60934] Updated weights for policy 1, policy_version 37122 (0.0010) [2023-10-13 22:26:58,731][60934] Updated weights for policy 1, policy_version 37132 (0.0010) [2023-10-13 22:26:59,101][60934] Updated weights for policy 1, policy_version 37142 (0.0008) [2023-10-13 22:26:59,462][60934] Updated weights for policy 1, policy_version 37152 (0.0008) [2023-10-13 22:27:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 75595776. Throughput: 0: 1690.5, 1: 1697.2. Samples: 18904056. Policy #0 lag: (min: 1.0, avg: 12.2, max: 33.0) [2023-10-13 22:27:01,248][59943] Avg episode reward: [(0, '-0.160'), (1, '0.000')] [2023-10-13 22:27:01,898][60935] Updated weights for policy 0, policy_version 36680 (0.0008) [2023-10-13 22:27:02,259][60935] Updated weights for policy 0, policy_version 36690 (0.0010) [2023-10-13 22:27:02,628][60935] Updated weights for policy 0, policy_version 36700 (0.0009) [2023-10-13 22:27:03,540][60934] Updated weights for policy 1, policy_version 37162 (0.0008) [2023-10-13 22:27:03,909][60934] Updated weights for policy 1, policy_version 37172 (0.0009) [2023-10-13 22:27:04,272][60934] Updated weights for policy 1, policy_version 37182 (0.0008) [2023-10-13 22:27:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 75661312. Throughput: 0: 1692.5, 1: 1674.5. Samples: 18923894. Policy #0 lag: (min: 1.0, avg: 12.2, max: 33.0) [2023-10-13 22:27:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:27:06,627][60935] Updated weights for policy 0, policy_version 36710 (0.0008) [2023-10-13 22:27:06,997][60935] Updated weights for policy 0, policy_version 36720 (0.0008) [2023-10-13 22:27:07,365][60935] Updated weights for policy 0, policy_version 36730 (0.0007) [2023-10-13 22:27:08,424][60934] Updated weights for policy 1, policy_version 37192 (0.0008) [2023-10-13 22:27:08,809][60934] Updated weights for policy 1, policy_version 37202 (0.0008) [2023-10-13 22:27:09,174][60934] Updated weights for policy 1, policy_version 37212 (0.0009) [2023-10-13 22:27:11,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 75726848. Throughput: 0: 1695.9, 1: 1699.5. Samples: 18944910. Policy #0 lag: (min: 1.0, avg: 12.2, max: 33.0) [2023-10-13 22:27:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:27:11,352][60935] Updated weights for policy 0, policy_version 36740 (0.0008) [2023-10-13 22:27:11,719][60935] Updated weights for policy 0, policy_version 36750 (0.0008) [2023-10-13 22:27:12,091][60935] Updated weights for policy 0, policy_version 36760 (0.0010) [2023-10-13 22:27:13,148][60934] Updated weights for policy 1, policy_version 37222 (0.0009) [2023-10-13 22:27:13,518][60934] Updated weights for policy 1, policy_version 37232 (0.0009) [2023-10-13 22:27:13,887][60934] Updated weights for policy 1, policy_version 37242 (0.0010) [2023-10-13 22:27:16,099][60935] Updated weights for policy 0, policy_version 36770 (0.0010) [2023-10-13 22:27:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 75792384. Throughput: 0: 1699.0, 1: 1695.3. Samples: 18954846. Policy #0 lag: (min: 1.0, avg: 12.2, max: 33.0) [2023-10-13 22:27:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:27:16,493][60935] Updated weights for policy 0, policy_version 36780 (0.0008) [2023-10-13 22:27:16,862][60935] Updated weights for policy 0, policy_version 36790 (0.0008) [2023-10-13 22:27:17,226][60935] Updated weights for policy 0, policy_version 36800 (0.0009) [2023-10-13 22:27:17,770][60934] Updated weights for policy 1, policy_version 37252 (0.0010) [2023-10-13 22:27:18,131][60934] Updated weights for policy 1, policy_version 37262 (0.0009) [2023-10-13 22:27:18,501][60934] Updated weights for policy 1, policy_version 37272 (0.0009) [2023-10-13 22:27:21,122][60935] Updated weights for policy 0, policy_version 36810 (0.0008) [2023-10-13 22:27:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 75857920. Throughput: 0: 1699.4, 1: 1682.3. Samples: 18974960. Policy #0 lag: (min: 1.0, avg: 12.2, max: 33.0) [2023-10-13 22:27:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:27:21,486][60935] Updated weights for policy 0, policy_version 36820 (0.0007) [2023-10-13 22:27:21,853][60935] Updated weights for policy 0, policy_version 36830 (0.0008) [2023-10-13 22:27:22,566][60934] Updated weights for policy 1, policy_version 37282 (0.0009) [2023-10-13 22:27:22,921][60934] Updated weights for policy 1, policy_version 37292 (0.0010) [2023-10-13 22:27:23,285][60934] Updated weights for policy 1, policy_version 37302 (0.0009) [2023-10-13 22:27:23,653][60934] Updated weights for policy 1, policy_version 37312 (0.0010) [2023-10-13 22:27:25,954][60935] Updated weights for policy 0, policy_version 36840 (0.0009) [2023-10-13 22:27:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 75923456. Throughput: 0: 1691.3, 1: 1704.5. Samples: 18995702. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 22:27:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:27:26,319][60935] Updated weights for policy 0, policy_version 36850 (0.0010) [2023-10-13 22:27:26,688][60935] Updated weights for policy 0, policy_version 36860 (0.0008) [2023-10-13 22:27:27,764][60934] Updated weights for policy 1, policy_version 37322 (0.0007) [2023-10-13 22:27:28,144][60934] Updated weights for policy 1, policy_version 37332 (0.0009) [2023-10-13 22:27:28,514][60934] Updated weights for policy 1, policy_version 37342 (0.0008) [2023-10-13 22:27:30,918][60935] Updated weights for policy 0, policy_version 36870 (0.0008) [2023-10-13 22:27:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 75988992. Throughput: 0: 1693.5, 1: 1676.8. Samples: 19005088. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 22:27:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:27:31,284][60935] Updated weights for policy 0, policy_version 36880 (0.0008) [2023-10-13 22:27:31,646][60935] Updated weights for policy 0, policy_version 36890 (0.0011) [2023-10-13 22:27:32,407][60934] Updated weights for policy 1, policy_version 37352 (0.0009) [2023-10-13 22:27:32,773][60934] Updated weights for policy 1, policy_version 37362 (0.0007) [2023-10-13 22:27:33,136][60934] Updated weights for policy 1, policy_version 37372 (0.0008) [2023-10-13 22:27:35,646][60935] Updated weights for policy 0, policy_version 36900 (0.0009) [2023-10-13 22:27:36,026][60935] Updated weights for policy 0, policy_version 36910 (0.0008) [2023-10-13 22:27:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 76054528. Throughput: 0: 1695.6, 1: 1696.8. Samples: 19026084. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 22:27:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:27:36,402][60935] Updated weights for policy 0, policy_version 36920 (0.0008) [2023-10-13 22:27:37,234][60934] Updated weights for policy 1, policy_version 37382 (0.0009) [2023-10-13 22:27:37,599][60934] Updated weights for policy 1, policy_version 37392 (0.0010) [2023-10-13 22:27:37,971][60934] Updated weights for policy 1, policy_version 37402 (0.0010) [2023-10-13 22:27:40,253][60935] Updated weights for policy 0, policy_version 36930 (0.0010) [2023-10-13 22:27:40,625][60935] Updated weights for policy 0, policy_version 36940 (0.0008) [2023-10-13 22:27:40,993][60935] Updated weights for policy 0, policy_version 36950 (0.0008) [2023-10-13 22:27:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 76120064. Throughput: 0: 1680.7, 1: 1703.2. Samples: 19046238. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 22:27:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:27:41,361][60935] Updated weights for policy 0, policy_version 36960 (0.0008) [2023-10-13 22:27:41,998][60934] Updated weights for policy 1, policy_version 37412 (0.0008) [2023-10-13 22:27:42,357][60934] Updated weights for policy 1, policy_version 37422 (0.0008) [2023-10-13 22:27:42,717][60934] Updated weights for policy 1, policy_version 37432 (0.0011) [2023-10-13 22:27:45,466][60935] Updated weights for policy 0, policy_version 36970 (0.0008) [2023-10-13 22:27:45,845][60935] Updated weights for policy 0, policy_version 36980 (0.0008) [2023-10-13 22:27:46,215][60935] Updated weights for policy 0, policy_version 36990 (0.0009) [2023-10-13 22:27:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 76185600. Throughput: 0: 1698.3, 1: 1681.5. Samples: 19056146. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 22:27:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:27:46,785][60934] Updated weights for policy 1, policy_version 37442 (0.0010) [2023-10-13 22:27:47,150][60934] Updated weights for policy 1, policy_version 37452 (0.0010) [2023-10-13 22:27:47,519][60934] Updated weights for policy 1, policy_version 37462 (0.0010) [2023-10-13 22:27:47,883][60934] Updated weights for policy 1, policy_version 37472 (0.0008) [2023-10-13 22:27:50,258][60935] Updated weights for policy 0, policy_version 37000 (0.0008) [2023-10-13 22:27:50,637][60935] Updated weights for policy 0, policy_version 37010 (0.0007) [2023-10-13 22:27:51,006][60935] Updated weights for policy 0, policy_version 37020 (0.0007) [2023-10-13 22:27:51,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 76283904. Throughput: 0: 1700.5, 1: 1707.2. Samples: 19077238. Policy #0 lag: (min: 2.0, avg: 3.6, max: 22.0) [2023-10-13 22:27:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:27:51,911][60934] Updated weights for policy 1, policy_version 37482 (0.0009) [2023-10-13 22:27:52,281][60934] Updated weights for policy 1, policy_version 37492 (0.0008) [2023-10-13 22:27:52,646][60934] Updated weights for policy 1, policy_version 37502 (0.0008) [2023-10-13 22:27:54,879][60935] Updated weights for policy 0, policy_version 37030 (0.0008) [2023-10-13 22:27:55,253][60935] Updated weights for policy 0, policy_version 37040 (0.0009) [2023-10-13 22:27:55,627][60935] Updated weights for policy 0, policy_version 37050 (0.0010) [2023-10-13 22:27:56,248][59943] Fps is (10 sec: 16383.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 76349440. Throughput: 0: 1672.3, 1: 1713.1. Samples: 19097256. Policy #0 lag: (min: 2.0, avg: 3.6, max: 22.0) [2023-10-13 22:27:56,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:27:56,739][60934] Updated weights for policy 1, policy_version 37512 (0.0009) [2023-10-13 22:27:57,107][60934] Updated weights for policy 1, policy_version 37522 (0.0011) [2023-10-13 22:27:57,473][60934] Updated weights for policy 1, policy_version 37532 (0.0011) [2023-10-13 22:27:59,615][60935] Updated weights for policy 0, policy_version 37060 (0.0009) [2023-10-13 22:27:59,977][60935] Updated weights for policy 0, policy_version 37070 (0.0008) [2023-10-13 22:28:00,351][60935] Updated weights for policy 0, policy_version 37080 (0.0010) [2023-10-13 22:28:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 76414976. Throughput: 0: 1695.2, 1: 1694.6. Samples: 19107386. Policy #0 lag: (min: 2.0, avg: 3.6, max: 22.0) [2023-10-13 22:28:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:28:01,350][60934] Updated weights for policy 1, policy_version 37542 (0.0008) [2023-10-13 22:28:01,720][60934] Updated weights for policy 1, policy_version 37552 (0.0009) [2023-10-13 22:28:02,085][60934] Updated weights for policy 1, policy_version 37562 (0.0009) [2023-10-13 22:28:04,482][60935] Updated weights for policy 0, policy_version 37090 (0.0008) [2023-10-13 22:28:04,873][60935] Updated weights for policy 0, policy_version 37100 (0.0011) [2023-10-13 22:28:05,245][60935] Updated weights for policy 0, policy_version 37110 (0.0010) [2023-10-13 22:28:05,620][60935] Updated weights for policy 0, policy_version 37120 (0.0008) [2023-10-13 22:28:06,093][60934] Updated weights for policy 1, policy_version 37572 (0.0007) [2023-10-13 22:28:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 76480512. Throughput: 0: 1688.2, 1: 1715.9. Samples: 19128142. Policy #0 lag: (min: 2.0, avg: 3.6, max: 22.0) [2023-10-13 22:28:06,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:28:06,457][60934] Updated weights for policy 1, policy_version 37582 (0.0008) [2023-10-13 22:28:06,821][60934] Updated weights for policy 1, policy_version 37592 (0.0007) [2023-10-13 22:28:09,551][60935] Updated weights for policy 0, policy_version 37130 (0.0009) [2023-10-13 22:28:09,921][60935] Updated weights for policy 0, policy_version 37140 (0.0008) [2023-10-13 22:28:10,286][60935] Updated weights for policy 0, policy_version 37150 (0.0007) [2023-10-13 22:28:10,605][60934] Updated weights for policy 1, policy_version 37602 (0.0010) [2023-10-13 22:28:10,967][60934] Updated weights for policy 1, policy_version 37612 (0.0007) [2023-10-13 22:28:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 76546048. Throughput: 0: 1674.9, 1: 1725.8. Samples: 19148730. Policy #0 lag: (min: 2.0, avg: 3.6, max: 22.0) [2023-10-13 22:28:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:28:11,333][60934] Updated weights for policy 1, policy_version 37622 (0.0007) [2023-10-13 22:28:11,692][60934] Updated weights for policy 1, policy_version 37632 (0.0007) [2023-10-13 22:28:14,282][60935] Updated weights for policy 0, policy_version 37160 (0.0008) [2023-10-13 22:28:14,655][60935] Updated weights for policy 0, policy_version 37170 (0.0011) [2023-10-13 22:28:15,012][60935] Updated weights for policy 0, policy_version 37180 (0.0010) [2023-10-13 22:28:15,619][60934] Updated weights for policy 1, policy_version 37642 (0.0010) [2023-10-13 22:28:15,988][60934] Updated weights for policy 1, policy_version 37652 (0.0009) [2023-10-13 22:28:16,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 76611584. Throughput: 0: 1702.8, 1: 1726.1. Samples: 19159388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:28:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:28:16,355][60934] Updated weights for policy 1, policy_version 37662 (0.0008) [2023-10-13 22:28:19,073][60935] Updated weights for policy 0, policy_version 37190 (0.0010) [2023-10-13 22:28:19,449][60935] Updated weights for policy 0, policy_version 37200 (0.0010) [2023-10-13 22:28:19,829][60935] Updated weights for policy 0, policy_version 37210 (0.0008) [2023-10-13 22:28:20,223][60934] Updated weights for policy 1, policy_version 37672 (0.0008) [2023-10-13 22:28:20,592][60934] Updated weights for policy 1, policy_version 37682 (0.0008) [2023-10-13 22:28:20,965][60934] Updated weights for policy 1, policy_version 37692 (0.0007) [2023-10-13 22:28:21,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 76709888. Throughput: 0: 1679.5, 1: 1734.7. Samples: 19179722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:28:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:28:23,890][60935] Updated weights for policy 0, policy_version 37220 (0.0011) [2023-10-13 22:28:24,262][60935] Updated weights for policy 0, policy_version 37230 (0.0007) [2023-10-13 22:28:24,631][60935] Updated weights for policy 0, policy_version 37240 (0.0008) [2023-10-13 22:28:24,982][60934] Updated weights for policy 1, policy_version 37702 (0.0007) [2023-10-13 22:28:25,352][60934] Updated weights for policy 1, policy_version 37712 (0.0010) [2023-10-13 22:28:25,714][60934] Updated weights for policy 1, policy_version 37722 (0.0009) [2023-10-13 22:28:25,934][60828] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000008 [2023-10-13 22:28:26,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 76775424. Throughput: 0: 1691.9, 1: 1719.2. Samples: 19199740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:28:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:28:28,652][60935] Updated weights for policy 0, policy_version 37250 (0.0008) [2023-10-13 22:28:29,018][60935] Updated weights for policy 0, policy_version 37260 (0.0009) [2023-10-13 22:28:29,393][60935] Updated weights for policy 0, policy_version 37270 (0.0008) [2023-10-13 22:28:29,712][60934] Updated weights for policy 1, policy_version 37732 (0.0007) [2023-10-13 22:28:29,763][60935] Updated weights for policy 0, policy_version 37280 (0.0008) [2023-10-13 22:28:30,084][60934] Updated weights for policy 1, policy_version 37742 (0.0008) [2023-10-13 22:28:30,450][60934] Updated weights for policy 1, policy_version 37752 (0.0009) [2023-10-13 22:28:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 76840960. Throughput: 0: 1695.5, 1: 1738.0. Samples: 19210656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:28:31,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-13 22:28:33,855][60935] Updated weights for policy 0, policy_version 37290 (0.0007) [2023-10-13 22:28:34,215][60935] Updated weights for policy 0, policy_version 37300 (0.0008) [2023-10-13 22:28:34,523][60934] Updated weights for policy 1, policy_version 37762 (0.0009) [2023-10-13 22:28:34,587][60935] Updated weights for policy 0, policy_version 37310 (0.0009) [2023-10-13 22:28:34,885][60934] Updated weights for policy 1, policy_version 37772 (0.0009) [2023-10-13 22:28:35,260][60934] Updated weights for policy 1, policy_version 37782 (0.0008) [2023-10-13 22:28:35,628][60934] Updated weights for policy 1, policy_version 37792 (0.0007) [2023-10-13 22:28:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 76906496. Throughput: 0: 1666.2, 1: 1732.1. Samples: 19230162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:28:36,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-13 22:28:38,788][60935] Updated weights for policy 0, policy_version 37320 (0.0009) [2023-10-13 22:28:39,159][60935] Updated weights for policy 0, policy_version 37330 (0.0010) [2023-10-13 22:28:39,525][60935] Updated weights for policy 0, policy_version 37340 (0.0007) [2023-10-13 22:28:39,606][60934] Updated weights for policy 1, policy_version 37802 (0.0007) [2023-10-13 22:28:39,964][60934] Updated weights for policy 1, policy_version 37812 (0.0007) [2023-10-13 22:28:40,341][60934] Updated weights for policy 1, policy_version 37822 (0.0008) [2023-10-13 22:28:41,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 76972032. Throughput: 0: 1691.0, 1: 1698.5. Samples: 19249782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:28:41,249][59943] Avg episode reward: [(0, '-0.140'), (1, '0.000')] [2023-10-13 22:28:43,576][60935] Updated weights for policy 0, policy_version 37350 (0.0007) [2023-10-13 22:28:43,947][60935] Updated weights for policy 0, policy_version 37360 (0.0007) [2023-10-13 22:28:44,262][60934] Updated weights for policy 1, policy_version 37832 (0.0009) [2023-10-13 22:28:44,316][60935] Updated weights for policy 0, policy_version 37370 (0.0009) [2023-10-13 22:28:44,631][60934] Updated weights for policy 1, policy_version 37842 (0.0008) [2023-10-13 22:28:44,984][60934] Updated weights for policy 1, policy_version 37852 (0.0010) [2023-10-13 22:28:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 77037568. Throughput: 0: 1680.7, 1: 1735.8. Samples: 19261130. Policy #0 lag: (min: 12.0, avg: 20.1, max: 44.0) [2023-10-13 22:28:46,249][59943] Avg episode reward: [(0, '-0.140'), (1, '0.000')] [2023-10-13 22:28:48,346][60935] Updated weights for policy 0, policy_version 37380 (0.0008) [2023-10-13 22:28:48,720][60935] Updated weights for policy 0, policy_version 37390 (0.0009) [2023-10-13 22:28:49,090][60935] Updated weights for policy 0, policy_version 37400 (0.0009) [2023-10-13 22:28:49,100][60934] Updated weights for policy 1, policy_version 37862 (0.0010) [2023-10-13 22:28:49,467][60934] Updated weights for policy 1, policy_version 37872 (0.0008) [2023-10-13 22:28:49,836][60934] Updated weights for policy 1, policy_version 37882 (0.0007) [2023-10-13 22:28:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 77103104. Throughput: 0: 1672.0, 1: 1713.8. Samples: 19280504. Policy #0 lag: (min: 12.0, avg: 20.1, max: 44.0) [2023-10-13 22:28:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:28:53,235][60935] Updated weights for policy 0, policy_version 37410 (0.0009) [2023-10-13 22:28:53,613][60935] Updated weights for policy 0, policy_version 37420 (0.0010) [2023-10-13 22:28:53,656][60934] Updated weights for policy 1, policy_version 37892 (0.0007) [2023-10-13 22:28:53,985][60935] Updated weights for policy 0, policy_version 37430 (0.0009) [2023-10-13 22:28:54,025][60934] Updated weights for policy 1, policy_version 37902 (0.0008) [2023-10-13 22:28:54,094][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-13 22:28:54,352][60935] Updated weights for policy 0, policy_version 37440 (0.0008) [2023-10-13 22:28:56,249][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 77168640. Throughput: 0: 1687.2, 1: 1710.4. Samples: 19301624. Policy #0 lag: (min: 12.0, avg: 20.1, max: 44.0) [2023-10-13 22:28:56,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:28:56,259][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000037904_38830080.pth... [2023-10-13 22:28:56,259][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000037440_38338560.pth... [2023-10-13 22:28:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000035872_36732928.pth [2023-10-13 22:28:56,300][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000036320_37191680.pth [2023-10-13 22:28:58,257][60934] Updated weights for policy 1, policy_version 37912 (0.0009) [2023-10-13 22:28:58,430][60935] Updated weights for policy 0, policy_version 37450 (0.0008) [2023-10-13 22:28:58,540][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000007 [2023-10-13 22:28:58,801][60935] Updated weights for policy 0, policy_version 37460 (0.0007) [2023-10-13 22:28:59,173][60935] Updated weights for policy 0, policy_version 37470 (0.0007) [2023-10-13 22:29:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 77234176. Throughput: 0: 1663.4, 1: 1723.1. Samples: 19311778. Policy #0 lag: (min: 12.0, avg: 20.1, max: 44.0) [2023-10-13 22:29:01,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:29:02,680][60934] Updated weights for policy 1, policy_version 37922 (0.0008) [2023-10-13 22:29:03,053][60934] Updated weights for policy 1, policy_version 37932 (0.0007) [2023-10-13 22:29:03,144][60935] Updated weights for policy 0, policy_version 37480 (0.0008) [2023-10-13 22:29:03,190][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-13 22:29:03,513][60935] Updated weights for policy 0, policy_version 37490 (0.0007) [2023-10-13 22:29:03,886][60935] Updated weights for policy 0, policy_version 37500 (0.0007) [2023-10-13 22:29:06,248][59943] Fps is (10 sec: 13107.7, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 77299712. Throughput: 0: 1678.0, 1: 1726.4. Samples: 19332924. Policy #0 lag: (min: 12.0, avg: 20.1, max: 44.0) [2023-10-13 22:29:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:29:07,249][60934] Updated weights for policy 1, policy_version 37942 (0.0009) [2023-10-13 22:29:07,618][60934] Updated weights for policy 1, policy_version 37952 (0.0009) [2023-10-13 22:29:07,978][60935] Updated weights for policy 0, policy_version 37510 (0.0008) [2023-10-13 22:29:07,987][60934] Updated weights for policy 1, policy_version 37962 (0.0008) [2023-10-13 22:29:08,348][60935] Updated weights for policy 0, policy_version 37520 (0.0010) [2023-10-13 22:29:08,719][60935] Updated weights for policy 0, policy_version 37530 (0.0009) [2023-10-13 22:29:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 77365248. Throughput: 0: 1683.7, 1: 1745.7. Samples: 19354062. Policy #0 lag: (min: 29.0, avg: 53.6, max: 56.0) [2023-10-13 22:29:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:29:11,853][60934] Updated weights for policy 1, policy_version 37972 (0.0008) [2023-10-13 22:29:12,221][60934] Updated weights for policy 1, policy_version 37982 (0.0008) [2023-10-13 22:29:12,589][60934] Updated weights for policy 1, policy_version 37992 (0.0009) [2023-10-13 22:29:12,779][60935] Updated weights for policy 0, policy_version 37540 (0.0008) [2023-10-13 22:29:13,151][60935] Updated weights for policy 0, policy_version 37550 (0.0007) [2023-10-13 22:29:13,521][60935] Updated weights for policy 0, policy_version 37560 (0.0008) [2023-10-13 22:29:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 77430784. Throughput: 0: 1662.2, 1: 1727.9. Samples: 19363210. Policy #0 lag: (min: 29.0, avg: 53.6, max: 56.0) [2023-10-13 22:29:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:29:16,455][60934] Updated weights for policy 1, policy_version 38002 (0.0008) [2023-10-13 22:29:16,817][60934] Updated weights for policy 1, policy_version 38012 (0.0011) [2023-10-13 22:29:17,181][60934] Updated weights for policy 1, policy_version 38022 (0.0010) [2023-10-13 22:29:17,554][60934] Updated weights for policy 1, policy_version 38032 (0.0009) [2023-10-13 22:29:17,616][60935] Updated weights for policy 0, policy_version 37570 (0.0008) [2023-10-13 22:29:17,990][60935] Updated weights for policy 0, policy_version 37580 (0.0010) [2023-10-13 22:29:18,359][60935] Updated weights for policy 0, policy_version 37590 (0.0008) [2023-10-13 22:29:18,726][60935] Updated weights for policy 0, policy_version 37600 (0.0008) [2023-10-13 22:29:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 77496320. Throughput: 0: 1687.5, 1: 1734.1. Samples: 19384134. Policy #0 lag: (min: 29.0, avg: 53.6, max: 56.0) [2023-10-13 22:29:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:29:21,573][60934] Updated weights for policy 1, policy_version 38042 (0.0009) [2023-10-13 22:29:21,936][60934] Updated weights for policy 1, policy_version 38052 (0.0009) [2023-10-13 22:29:22,311][60934] Updated weights for policy 1, policy_version 38062 (0.0009) [2023-10-13 22:29:22,778][60935] Updated weights for policy 0, policy_version 37610 (0.0009) [2023-10-13 22:29:23,147][60935] Updated weights for policy 0, policy_version 37620 (0.0011) [2023-10-13 22:29:23,519][60935] Updated weights for policy 0, policy_version 37630 (0.0008) [2023-10-13 22:29:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 77561856. Throughput: 0: 1692.3, 1: 1762.7. Samples: 19405254. Policy #0 lag: (min: 29.0, avg: 53.6, max: 56.0) [2023-10-13 22:29:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:29:26,348][60934] Updated weights for policy 1, policy_version 38072 (0.0007) [2023-10-13 22:29:26,708][60934] Updated weights for policy 1, policy_version 38082 (0.0010) [2023-10-13 22:29:27,078][60934] Updated weights for policy 1, policy_version 38092 (0.0010) [2023-10-13 22:29:27,462][60935] Updated weights for policy 0, policy_version 37640 (0.0009) [2023-10-13 22:29:27,835][60935] Updated weights for policy 0, policy_version 37650 (0.0007) [2023-10-13 22:29:28,207][60935] Updated weights for policy 0, policy_version 37660 (0.0008) [2023-10-13 22:29:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 77627392. Throughput: 0: 1675.2, 1: 1730.6. Samples: 19414392. Policy #0 lag: (min: 29.0, avg: 53.6, max: 56.0) [2023-10-13 22:29:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:29:31,341][60934] Updated weights for policy 1, policy_version 38102 (0.0009) [2023-10-13 22:29:31,705][60934] Updated weights for policy 1, policy_version 38112 (0.0008) [2023-10-13 22:29:32,078][60934] Updated weights for policy 1, policy_version 38122 (0.0008) [2023-10-13 22:29:32,345][60935] Updated weights for policy 0, policy_version 37670 (0.0007) [2023-10-13 22:29:32,710][60935] Updated weights for policy 0, policy_version 37680 (0.0008) [2023-10-13 22:29:33,085][60935] Updated weights for policy 0, policy_version 37690 (0.0010) [2023-10-13 22:29:35,978][60934] Updated weights for policy 1, policy_version 38132 (0.0007) [2023-10-13 22:29:36,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 77692928. Throughput: 0: 1689.0, 1: 1745.2. Samples: 19435044. Policy #0 lag: (min: 29.0, avg: 53.6, max: 56.0) [2023-10-13 22:29:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:29:36,341][60934] Updated weights for policy 1, policy_version 38142 (0.0009) [2023-10-13 22:29:36,714][60934] Updated weights for policy 1, policy_version 38152 (0.0009) [2023-10-13 22:29:36,985][60935] Updated weights for policy 0, policy_version 37700 (0.0010) [2023-10-13 22:29:37,354][60935] Updated weights for policy 0, policy_version 37710 (0.0009) [2023-10-13 22:29:37,721][60935] Updated weights for policy 0, policy_version 37720 (0.0009) [2023-10-13 22:29:40,717][60934] Updated weights for policy 1, policy_version 38162 (0.0009) [2023-10-13 22:29:41,081][60934] Updated weights for policy 1, policy_version 38172 (0.0007) [2023-10-13 22:29:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 77758464. Throughput: 0: 1690.9, 1: 1744.0. Samples: 19456194. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 22:29:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:29:41,447][60934] Updated weights for policy 1, policy_version 38182 (0.0007) [2023-10-13 22:29:41,810][60934] Updated weights for policy 1, policy_version 38192 (0.0008) [2023-10-13 22:29:41,858][60935] Updated weights for policy 0, policy_version 37730 (0.0008) [2023-10-13 22:29:42,265][60935] Updated weights for policy 0, policy_version 37740 (0.0009) [2023-10-13 22:29:42,641][60935] Updated weights for policy 0, policy_version 37750 (0.0008) [2023-10-13 22:29:43,010][60935] Updated weights for policy 0, policy_version 37760 (0.0009) [2023-10-13 22:29:45,822][60934] Updated weights for policy 1, policy_version 38202 (0.0007) [2023-10-13 22:29:46,194][60934] Updated weights for policy 1, policy_version 38212 (0.0008) [2023-10-13 22:29:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 77824000. Throughput: 0: 1676.7, 1: 1729.4. Samples: 19465052. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 22:29:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:29:46,564][60934] Updated weights for policy 1, policy_version 38222 (0.0009) [2023-10-13 22:29:46,998][60935] Updated weights for policy 0, policy_version 37770 (0.0008) [2023-10-13 22:29:47,363][60935] Updated weights for policy 0, policy_version 37780 (0.0009) [2023-10-13 22:29:47,734][60935] Updated weights for policy 0, policy_version 37790 (0.0011) [2023-10-13 22:29:50,624][60934] Updated weights for policy 1, policy_version 38232 (0.0008) [2023-10-13 22:29:50,995][60934] Updated weights for policy 1, policy_version 38242 (0.0008) [2023-10-13 22:29:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 77889536. Throughput: 0: 1686.6, 1: 1713.4. Samples: 19485924. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 22:29:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:29:51,365][60934] Updated weights for policy 1, policy_version 38252 (0.0010) [2023-10-13 22:29:51,833][60935] Updated weights for policy 0, policy_version 37800 (0.0010) [2023-10-13 22:29:52,202][60935] Updated weights for policy 0, policy_version 37810 (0.0008) [2023-10-13 22:29:52,565][60935] Updated weights for policy 0, policy_version 37820 (0.0007) [2023-10-13 22:29:55,313][60934] Updated weights for policy 1, policy_version 38262 (0.0009) [2023-10-13 22:29:55,672][60934] Updated weights for policy 1, policy_version 38272 (0.0010) [2023-10-13 22:29:56,049][60934] Updated weights for policy 1, policy_version 38282 (0.0011) [2023-10-13 22:29:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 77955072. Throughput: 0: 1682.7, 1: 1700.1. Samples: 19506288. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 22:29:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:29:56,707][60935] Updated weights for policy 0, policy_version 37830 (0.0009) [2023-10-13 22:29:57,077][60935] Updated weights for policy 0, policy_version 37840 (0.0010) [2023-10-13 22:29:57,441][60935] Updated weights for policy 0, policy_version 37850 (0.0009) [2023-10-13 22:30:00,096][60934] Updated weights for policy 1, policy_version 38292 (0.0009) [2023-10-13 22:30:00,472][60934] Updated weights for policy 1, policy_version 38302 (0.0007) [2023-10-13 22:30:00,833][60934] Updated weights for policy 1, policy_version 38312 (0.0007) [2023-10-13 22:30:01,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 78053376. Throughput: 0: 1679.9, 1: 1708.3. Samples: 19515678. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 22:30:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:30:01,607][60935] Updated weights for policy 0, policy_version 37860 (0.0010) [2023-10-13 22:30:01,974][60935] Updated weights for policy 0, policy_version 37870 (0.0010) [2023-10-13 22:30:02,343][60935] Updated weights for policy 0, policy_version 37880 (0.0008) [2023-10-13 22:30:04,768][60934] Updated weights for policy 1, policy_version 38322 (0.0008) [2023-10-13 22:30:05,140][60934] Updated weights for policy 1, policy_version 38332 (0.0008) [2023-10-13 22:30:05,508][60934] Updated weights for policy 1, policy_version 38342 (0.0011) [2023-10-13 22:30:05,866][60934] Updated weights for policy 1, policy_version 38352 (0.0009) [2023-10-13 22:30:06,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 78118912. Throughput: 0: 1678.9, 1: 1711.2. Samples: 19536690. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-13 22:30:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:30:06,585][60935] Updated weights for policy 0, policy_version 37890 (0.0009) [2023-10-13 22:30:06,949][60935] Updated weights for policy 0, policy_version 37900 (0.0011) [2023-10-13 22:30:07,330][60935] Updated weights for policy 0, policy_version 37910 (0.0010) [2023-10-13 22:30:07,694][60935] Updated weights for policy 0, policy_version 37920 (0.0011) [2023-10-13 22:30:09,975][60934] Updated weights for policy 1, policy_version 38362 (0.0007) [2023-10-13 22:30:10,349][60934] Updated weights for policy 1, policy_version 38372 (0.0008) [2023-10-13 22:30:10,723][60934] Updated weights for policy 1, policy_version 38382 (0.0008) [2023-10-13 22:30:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 78184448. Throughput: 0: 1683.1, 1: 1682.7. Samples: 19556714. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-13 22:30:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:30:11,641][60935] Updated weights for policy 0, policy_version 37930 (0.0009) [2023-10-13 22:30:12,017][60935] Updated weights for policy 0, policy_version 37940 (0.0008) [2023-10-13 22:30:12,384][60935] Updated weights for policy 0, policy_version 37950 (0.0008) [2023-10-13 22:30:14,691][60934] Updated weights for policy 1, policy_version 38392 (0.0008) [2023-10-13 22:30:15,052][60934] Updated weights for policy 1, policy_version 38402 (0.0008) [2023-10-13 22:30:15,427][60934] Updated weights for policy 1, policy_version 38412 (0.0009) [2023-10-13 22:30:16,150][60935] Updated weights for policy 0, policy_version 37960 (0.0009) [2023-10-13 22:30:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 78249984. Throughput: 0: 1685.6, 1: 1707.5. Samples: 19567080. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-13 22:30:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:30:16,516][60935] Updated weights for policy 0, policy_version 37970 (0.0009) [2023-10-13 22:30:16,888][60935] Updated weights for policy 0, policy_version 37980 (0.0008) [2023-10-13 22:30:19,559][60934] Updated weights for policy 1, policy_version 38422 (0.0009) [2023-10-13 22:30:19,953][60934] Updated weights for policy 1, policy_version 38432 (0.0008) [2023-10-13 22:30:20,316][60934] Updated weights for policy 1, policy_version 38442 (0.0008) [2023-10-13 22:30:20,782][60935] Updated weights for policy 0, policy_version 37990 (0.0009) [2023-10-13 22:30:21,153][60935] Updated weights for policy 0, policy_version 38000 (0.0009) [2023-10-13 22:30:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 78315520. Throughput: 0: 1690.1, 1: 1702.2. Samples: 19587696. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-13 22:30:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:30:21,524][60935] Updated weights for policy 0, policy_version 38010 (0.0009) [2023-10-13 22:30:24,393][60934] Updated weights for policy 1, policy_version 38452 (0.0010) [2023-10-13 22:30:24,764][60934] Updated weights for policy 1, policy_version 38462 (0.0009) [2023-10-13 22:30:25,136][60934] Updated weights for policy 1, policy_version 38472 (0.0008) [2023-10-13 22:30:25,654][60935] Updated weights for policy 0, policy_version 38020 (0.0008) [2023-10-13 22:30:26,026][60935] Updated weights for policy 0, policy_version 38030 (0.0008) [2023-10-13 22:30:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 78381056. Throughput: 0: 1682.2, 1: 1669.3. Samples: 19607010. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-13 22:30:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:30:26,405][60935] Updated weights for policy 0, policy_version 38040 (0.0010) [2023-10-13 22:30:29,101][60934] Updated weights for policy 1, policy_version 38482 (0.0010) [2023-10-13 22:30:29,465][60934] Updated weights for policy 1, policy_version 38492 (0.0009) [2023-10-13 22:30:29,828][60934] Updated weights for policy 1, policy_version 38502 (0.0008) [2023-10-13 22:30:30,192][60934] Updated weights for policy 1, policy_version 38512 (0.0008) [2023-10-13 22:30:30,473][60935] Updated weights for policy 0, policy_version 38050 (0.0008) [2023-10-13 22:30:30,839][60935] Updated weights for policy 0, policy_version 38060 (0.0009) [2023-10-13 22:30:31,214][60935] Updated weights for policy 0, policy_version 38070 (0.0010) [2023-10-13 22:30:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 78446592. Throughput: 0: 1692.0, 1: 1697.8. Samples: 19617590. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-13 22:30:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:30:31,589][60935] Updated weights for policy 0, policy_version 38080 (0.0008) [2023-10-13 22:30:34,182][60934] Updated weights for policy 1, policy_version 38522 (0.0008) [2023-10-13 22:30:34,544][60934] Updated weights for policy 1, policy_version 38532 (0.0009) [2023-10-13 22:30:34,916][60934] Updated weights for policy 1, policy_version 38542 (0.0007) [2023-10-13 22:30:35,802][60935] Updated weights for policy 0, policy_version 38090 (0.0010) [2023-10-13 22:30:36,173][60935] Updated weights for policy 0, policy_version 38100 (0.0009) [2023-10-13 22:30:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 78512128. Throughput: 0: 1689.0, 1: 1682.6. Samples: 19637646. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-13 22:30:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:30:36,538][60935] Updated weights for policy 0, policy_version 38110 (0.0007) [2023-10-13 22:30:38,921][60934] Updated weights for policy 1, policy_version 38552 (0.0010) [2023-10-13 22:30:39,287][60934] Updated weights for policy 1, policy_version 38562 (0.0009) [2023-10-13 22:30:39,661][60934] Updated weights for policy 1, policy_version 38572 (0.0010) [2023-10-13 22:30:40,442][60935] Updated weights for policy 0, policy_version 38120 (0.0010) [2023-10-13 22:30:40,808][60935] Updated weights for policy 0, policy_version 38130 (0.0010) [2023-10-13 22:30:41,182][60935] Updated weights for policy 0, policy_version 38140 (0.0010) [2023-10-13 22:30:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 78577664. Throughput: 0: 1674.4, 1: 1681.4. Samples: 19657298. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-13 22:30:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:30:43,686][60934] Updated weights for policy 1, policy_version 38582 (0.0007) [2023-10-13 22:30:44,052][60934] Updated weights for policy 1, policy_version 38592 (0.0009) [2023-10-13 22:30:44,435][60934] Updated weights for policy 1, policy_version 38602 (0.0009) [2023-10-13 22:30:45,434][60935] Updated weights for policy 0, policy_version 38150 (0.0011) [2023-10-13 22:30:45,812][60935] Updated weights for policy 0, policy_version 38160 (0.0010) [2023-10-13 22:30:46,188][60935] Updated weights for policy 0, policy_version 38170 (0.0008) [2023-10-13 22:30:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 78643200. Throughput: 0: 1691.2, 1: 1696.2. Samples: 19668110. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-13 22:30:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:30:48,266][60934] Updated weights for policy 1, policy_version 38612 (0.0010) [2023-10-13 22:30:48,628][60934] Updated weights for policy 1, policy_version 38622 (0.0008) [2023-10-13 22:30:49,004][60934] Updated weights for policy 1, policy_version 38632 (0.0009) [2023-10-13 22:30:50,331][60935] Updated weights for policy 0, policy_version 38180 (0.0009) [2023-10-13 22:30:50,707][60935] Updated weights for policy 0, policy_version 38190 (0.0009) [2023-10-13 22:30:51,071][60935] Updated weights for policy 0, policy_version 38200 (0.0008) [2023-10-13 22:30:51,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 78708736. Throughput: 0: 1688.8, 1: 1668.4. Samples: 19687760. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-13 22:30:51,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-13 22:30:53,020][60934] Updated weights for policy 1, policy_version 38642 (0.0007) [2023-10-13 22:30:53,380][60934] Updated weights for policy 1, policy_version 38652 (0.0007) [2023-10-13 22:30:53,745][60934] Updated weights for policy 1, policy_version 38662 (0.0007) [2023-10-13 22:30:54,111][60934] Updated weights for policy 1, policy_version 38672 (0.0007) [2023-10-13 22:30:55,117][60935] Updated weights for policy 0, policy_version 38210 (0.0008) [2023-10-13 22:30:55,490][60935] Updated weights for policy 0, policy_version 38220 (0.0010) [2023-10-13 22:30:55,870][60935] Updated weights for policy 0, policy_version 38230 (0.0010) [2023-10-13 22:30:56,242][60935] Updated weights for policy 0, policy_version 38240 (0.0009) [2023-10-13 22:30:56,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 78807040. Throughput: 0: 1667.1, 1: 1694.2. Samples: 19707972. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-13 22:30:56,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 22:30:56,259][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000038672_39649280.pth... [2023-10-13 22:30:56,259][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000038240_39157760.pth... [2023-10-13 22:30:56,299][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000037120_38010880.pth [2023-10-13 22:30:56,301][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000036640_37519360.pth [2023-10-13 22:30:58,181][60934] Updated weights for policy 1, policy_version 38682 (0.0008) [2023-10-13 22:30:58,549][60934] Updated weights for policy 1, policy_version 38692 (0.0008) [2023-10-13 22:30:58,920][60934] Updated weights for policy 1, policy_version 38702 (0.0008) [2023-10-13 22:31:00,384][60935] Updated weights for policy 0, policy_version 38250 (0.0008) [2023-10-13 22:31:00,760][60935] Updated weights for policy 0, policy_version 38260 (0.0008) [2023-10-13 22:31:01,130][60935] Updated weights for policy 0, policy_version 38270 (0.0007) [2023-10-13 22:31:01,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 78872576. Throughput: 0: 1681.6, 1: 1681.6. Samples: 19718424. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-13 22:31:01,249][59943] Avg episode reward: [(0, '-0.070'), (1, '0.000')] [2023-10-13 22:31:02,959][60934] Updated weights for policy 1, policy_version 38712 (0.0009) [2023-10-13 22:31:03,330][60934] Updated weights for policy 1, policy_version 38722 (0.0011) [2023-10-13 22:31:03,700][60934] Updated weights for policy 1, policy_version 38732 (0.0010) [2023-10-13 22:31:05,116][60935] Updated weights for policy 0, policy_version 38280 (0.0007) [2023-10-13 22:31:05,485][60935] Updated weights for policy 0, policy_version 38290 (0.0008) [2023-10-13 22:31:05,865][60935] Updated weights for policy 0, policy_version 38300 (0.0008) [2023-10-13 22:31:06,248][59943] Fps is (10 sec: 13107.7, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 78938112. Throughput: 0: 1684.4, 1: 1679.5. Samples: 19739070. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-13 22:31:06,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:31:07,693][60934] Updated weights for policy 1, policy_version 38742 (0.0008) [2023-10-13 22:31:08,051][60934] Updated weights for policy 1, policy_version 38752 (0.0007) [2023-10-13 22:31:08,420][60934] Updated weights for policy 1, policy_version 38762 (0.0007) [2023-10-13 22:31:09,815][60935] Updated weights for policy 0, policy_version 38310 (0.0009) [2023-10-13 22:31:10,178][60935] Updated weights for policy 0, policy_version 38320 (0.0007) [2023-10-13 22:31:10,552][60935] Updated weights for policy 0, policy_version 38330 (0.0007) [2023-10-13 22:31:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 79003648. Throughput: 0: 1667.7, 1: 1708.7. Samples: 19758950. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-13 22:31:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:31:12,417][60934] Updated weights for policy 1, policy_version 38772 (0.0007) [2023-10-13 22:31:12,790][60934] Updated weights for policy 1, policy_version 38782 (0.0007) [2023-10-13 22:31:13,151][60934] Updated weights for policy 1, policy_version 38792 (0.0007) [2023-10-13 22:31:14,434][60935] Updated weights for policy 0, policy_version 38340 (0.0007) [2023-10-13 22:31:14,800][60935] Updated weights for policy 0, policy_version 38350 (0.0007) [2023-10-13 22:31:15,166][60935] Updated weights for policy 0, policy_version 38360 (0.0008) [2023-10-13 22:31:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 79069184. Throughput: 0: 1698.2, 1: 1679.4. Samples: 19769580. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-13 22:31:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:31:17,114][60934] Updated weights for policy 1, policy_version 38802 (0.0009) [2023-10-13 22:31:17,473][60934] Updated weights for policy 1, policy_version 38812 (0.0009) [2023-10-13 22:31:17,843][60934] Updated weights for policy 1, policy_version 38822 (0.0010) [2023-10-13 22:31:18,205][60934] Updated weights for policy 1, policy_version 38832 (0.0008) [2023-10-13 22:31:19,132][60935] Updated weights for policy 0, policy_version 38370 (0.0010) [2023-10-13 22:31:19,537][60935] Updated weights for policy 0, policy_version 38380 (0.0007) [2023-10-13 22:31:19,913][60935] Updated weights for policy 0, policy_version 38390 (0.0009) [2023-10-13 22:31:20,281][60935] Updated weights for policy 0, policy_version 38400 (0.0009) [2023-10-13 22:31:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 79134720. Throughput: 0: 1684.3, 1: 1702.8. Samples: 19790064. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-13 22:31:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:31:22,187][60934] Updated weights for policy 1, policy_version 38842 (0.0009) [2023-10-13 22:31:22,562][60934] Updated weights for policy 1, policy_version 38852 (0.0007) [2023-10-13 22:31:22,935][60934] Updated weights for policy 1, policy_version 38862 (0.0007) [2023-10-13 22:31:24,353][60935] Updated weights for policy 0, policy_version 38410 (0.0008) [2023-10-13 22:31:24,729][60935] Updated weights for policy 0, policy_version 38420 (0.0008) [2023-10-13 22:31:25,092][60935] Updated weights for policy 0, policy_version 38430 (0.0007) [2023-10-13 22:31:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 79200256. Throughput: 0: 1687.5, 1: 1716.4. Samples: 19810472. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-13 22:31:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:31:27,034][60934] Updated weights for policy 1, policy_version 38872 (0.0008) [2023-10-13 22:31:27,404][60934] Updated weights for policy 1, policy_version 38882 (0.0009) [2023-10-13 22:31:27,771][60934] Updated weights for policy 1, policy_version 38892 (0.0010) [2023-10-13 22:31:29,000][60935] Updated weights for policy 0, policy_version 38440 (0.0008) [2023-10-13 22:31:29,372][60935] Updated weights for policy 0, policy_version 38450 (0.0008) [2023-10-13 22:31:29,749][60935] Updated weights for policy 0, policy_version 38460 (0.0009) [2023-10-13 22:31:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 79265792. Throughput: 0: 1700.2, 1: 1691.2. Samples: 19820722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:31:31,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:31:31,727][60934] Updated weights for policy 1, policy_version 38902 (0.0009) [2023-10-13 22:31:32,100][60934] Updated weights for policy 1, policy_version 38912 (0.0010) [2023-10-13 22:31:32,470][60934] Updated weights for policy 1, policy_version 38922 (0.0009) [2023-10-13 22:31:33,769][60935] Updated weights for policy 0, policy_version 38470 (0.0008) [2023-10-13 22:31:34,138][60935] Updated weights for policy 0, policy_version 38480 (0.0009) [2023-10-13 22:31:34,512][60935] Updated weights for policy 0, policy_version 38490 (0.0008) [2023-10-13 22:31:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 79331328. Throughput: 0: 1679.3, 1: 1718.9. Samples: 19840682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:31:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:31:36,527][60934] Updated weights for policy 1, policy_version 38932 (0.0009) [2023-10-13 22:31:36,895][60934] Updated weights for policy 1, policy_version 38942 (0.0009) [2023-10-13 22:31:37,266][60934] Updated weights for policy 1, policy_version 38952 (0.0008) [2023-10-13 22:31:38,608][60935] Updated weights for policy 0, policy_version 38500 (0.0010) [2023-10-13 22:31:38,969][60935] Updated weights for policy 0, policy_version 38510 (0.0011) [2023-10-13 22:31:39,335][60935] Updated weights for policy 0, policy_version 38520 (0.0011) [2023-10-13 22:31:41,079][60934] Updated weights for policy 1, policy_version 38962 (0.0007) [2023-10-13 22:31:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 79396864. Throughput: 0: 1693.6, 1: 1725.6. Samples: 19861834. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:31:41,249][59943] Avg episode reward: [(0, '-0.070'), (1, '0.000')] [2023-10-13 22:31:41,444][60934] Updated weights for policy 1, policy_version 38972 (0.0007) [2023-10-13 22:31:41,817][60934] Updated weights for policy 1, policy_version 38982 (0.0008) [2023-10-13 22:31:42,180][60934] Updated weights for policy 1, policy_version 38992 (0.0008) [2023-10-13 22:31:43,377][60935] Updated weights for policy 0, policy_version 38530 (0.0010) [2023-10-13 22:31:43,753][60935] Updated weights for policy 0, policy_version 38540 (0.0008) [2023-10-13 22:31:44,122][60935] Updated weights for policy 0, policy_version 38550 (0.0007) [2023-10-13 22:31:44,488][60935] Updated weights for policy 0, policy_version 38560 (0.0008) [2023-10-13 22:31:46,063][60934] Updated weights for policy 1, policy_version 39002 (0.0007) [2023-10-13 22:31:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 79462400. Throughput: 0: 1695.9, 1: 1714.6. Samples: 19871896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:31:46,249][59943] Avg episode reward: [(0, '-0.070'), (1, '0.000')] [2023-10-13 22:31:46,439][60934] Updated weights for policy 1, policy_version 39012 (0.0009) [2023-10-13 22:31:46,812][60934] Updated weights for policy 1, policy_version 39022 (0.0007) [2023-10-13 22:31:48,604][60935] Updated weights for policy 0, policy_version 38570 (0.0007) [2023-10-13 22:31:48,974][60935] Updated weights for policy 0, policy_version 38580 (0.0008) [2023-10-13 22:31:49,342][60935] Updated weights for policy 0, policy_version 38590 (0.0010) [2023-10-13 22:31:50,834][60934] Updated weights for policy 1, policy_version 39032 (0.0009) [2023-10-13 22:31:51,208][60934] Updated weights for policy 1, policy_version 39042 (0.0008) [2023-10-13 22:31:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 79527936. Throughput: 0: 1672.4, 1: 1727.8. Samples: 19892080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:31:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:31:51,564][60934] Updated weights for policy 1, policy_version 39052 (0.0009) [2023-10-13 22:31:53,173][60935] Updated weights for policy 0, policy_version 38600 (0.0008) [2023-10-13 22:31:53,536][60935] Updated weights for policy 0, policy_version 38610 (0.0009) [2023-10-13 22:31:53,904][60935] Updated weights for policy 0, policy_version 38620 (0.0007) [2023-10-13 22:31:55,721][60934] Updated weights for policy 1, policy_version 39062 (0.0009) [2023-10-13 22:31:56,105][60934] Updated weights for policy 1, policy_version 39072 (0.0008) [2023-10-13 22:31:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 79593472. Throughput: 0: 1700.1, 1: 1719.4. Samples: 19912824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:31:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:31:56,482][60934] Updated weights for policy 1, policy_version 39082 (0.0010) [2023-10-13 22:31:57,996][60935] Updated weights for policy 0, policy_version 38630 (0.0008) [2023-10-13 22:31:58,373][60935] Updated weights for policy 0, policy_version 38640 (0.0007) [2023-10-13 22:31:58,744][60935] Updated weights for policy 0, policy_version 38650 (0.0009) [2023-10-13 22:32:00,485][60934] Updated weights for policy 1, policy_version 39092 (0.0010) [2023-10-13 22:32:00,860][60934] Updated weights for policy 1, policy_version 39102 (0.0009) [2023-10-13 22:32:01,216][60934] Updated weights for policy 1, policy_version 39112 (0.0009) [2023-10-13 22:32:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 79659008. Throughput: 0: 1671.4, 1: 1721.4. Samples: 19922254. Policy #0 lag: (min: 1.0, avg: 11.4, max: 33.0) [2023-10-13 22:32:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:32:02,697][60935] Updated weights for policy 0, policy_version 38660 (0.0009) [2023-10-13 22:32:03,067][60935] Updated weights for policy 0, policy_version 38670 (0.0007) [2023-10-13 22:32:03,437][60935] Updated weights for policy 0, policy_version 38680 (0.0010) [2023-10-13 22:32:05,060][60934] Updated weights for policy 1, policy_version 39122 (0.0011) [2023-10-13 22:32:05,426][60934] Updated weights for policy 1, policy_version 39132 (0.0010) [2023-10-13 22:32:05,794][60934] Updated weights for policy 1, policy_version 39142 (0.0010) [2023-10-13 22:32:06,163][60934] Updated weights for policy 1, policy_version 39152 (0.0010) [2023-10-13 22:32:06,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 79757312. Throughput: 0: 1685.2, 1: 1718.7. Samples: 19943238. Policy #0 lag: (min: 1.0, avg: 11.4, max: 33.0) [2023-10-13 22:32:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:32:07,658][60935] Updated weights for policy 0, policy_version 38690 (0.0008) [2023-10-13 22:32:08,069][60935] Updated weights for policy 0, policy_version 38700 (0.0009) [2023-10-13 22:32:08,442][60935] Updated weights for policy 0, policy_version 38710 (0.0007) [2023-10-13 22:32:08,808][60935] Updated weights for policy 0, policy_version 38720 (0.0008) [2023-10-13 22:32:10,142][60934] Updated weights for policy 1, policy_version 39162 (0.0010) [2023-10-13 22:32:10,517][60934] Updated weights for policy 1, policy_version 39172 (0.0009) [2023-10-13 22:32:10,874][60934] Updated weights for policy 1, policy_version 39182 (0.0008) [2023-10-13 22:32:11,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 79822848. Throughput: 0: 1698.4, 1: 1700.9. Samples: 19963438. Policy #0 lag: (min: 1.0, avg: 11.4, max: 33.0) [2023-10-13 22:32:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 22:32:12,835][60935] Updated weights for policy 0, policy_version 38730 (0.0007) [2023-10-13 22:32:13,212][60935] Updated weights for policy 0, policy_version 38740 (0.0008) [2023-10-13 22:32:13,587][60935] Updated weights for policy 0, policy_version 38750 (0.0007) [2023-10-13 22:32:14,873][60934] Updated weights for policy 1, policy_version 39192 (0.0009) [2023-10-13 22:32:15,237][60934] Updated weights for policy 1, policy_version 39202 (0.0009) [2023-10-13 22:32:15,607][60934] Updated weights for policy 1, policy_version 39212 (0.0007) [2023-10-13 22:32:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 79888384. Throughput: 0: 1671.3, 1: 1723.3. Samples: 19973478. Policy #0 lag: (min: 1.0, avg: 11.4, max: 33.0) [2023-10-13 22:32:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 22:32:17,579][60935] Updated weights for policy 0, policy_version 38760 (0.0010) [2023-10-13 22:32:17,951][60935] Updated weights for policy 0, policy_version 38770 (0.0007) [2023-10-13 22:32:18,313][60935] Updated weights for policy 0, policy_version 38780 (0.0008) [2023-10-13 22:32:19,674][60934] Updated weights for policy 1, policy_version 39222 (0.0007) [2023-10-13 22:32:20,046][60934] Updated weights for policy 1, policy_version 39232 (0.0008) [2023-10-13 22:32:20,411][60934] Updated weights for policy 1, policy_version 39242 (0.0008) [2023-10-13 22:32:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 79953920. Throughput: 0: 1703.4, 1: 1715.2. Samples: 19994518. Policy #0 lag: (min: 1.0, avg: 11.4, max: 33.0) [2023-10-13 22:32:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:32:22,332][60935] Updated weights for policy 0, policy_version 38790 (0.0008) [2023-10-13 22:32:22,693][60935] Updated weights for policy 0, policy_version 38800 (0.0009) [2023-10-13 22:32:23,076][60935] Updated weights for policy 0, policy_version 38810 (0.0008) [2023-10-13 22:32:24,368][60934] Updated weights for policy 1, policy_version 39252 (0.0009) [2023-10-13 22:32:24,734][60934] Updated weights for policy 1, policy_version 39262 (0.0010) [2023-10-13 22:32:25,103][60934] Updated weights for policy 1, policy_version 39272 (0.0009) [2023-10-13 22:32:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 80019456. Throughput: 0: 1710.3, 1: 1682.4. Samples: 20014502. Policy #0 lag: (min: 1.0, avg: 11.4, max: 33.0) [2023-10-13 22:32:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:32:26,932][60935] Updated weights for policy 0, policy_version 38820 (0.0012) [2023-10-13 22:32:27,311][60935] Updated weights for policy 0, policy_version 38830 (0.0010) [2023-10-13 22:32:27,668][60935] Updated weights for policy 0, policy_version 38840 (0.0011) [2023-10-13 22:32:29,221][60934] Updated weights for policy 1, policy_version 39282 (0.0010) [2023-10-13 22:32:29,592][60934] Updated weights for policy 1, policy_version 39292 (0.0008) [2023-10-13 22:32:29,957][60934] Updated weights for policy 1, policy_version 39302 (0.0007) [2023-10-13 22:32:30,318][60934] Updated weights for policy 1, policy_version 39312 (0.0009) [2023-10-13 22:32:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 80084992. Throughput: 0: 1691.9, 1: 1708.1. Samples: 20024898. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-13 22:32:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:32:31,485][60935] Updated weights for policy 0, policy_version 38850 (0.0008) [2023-10-13 22:32:31,857][60935] Updated weights for policy 0, policy_version 38860 (0.0008) [2023-10-13 22:32:32,220][60935] Updated weights for policy 0, policy_version 38870 (0.0009) [2023-10-13 22:32:32,592][60935] Updated weights for policy 0, policy_version 38880 (0.0008) [2023-10-13 22:32:34,323][60934] Updated weights for policy 1, policy_version 39322 (0.0009) [2023-10-13 22:32:34,685][60934] Updated weights for policy 1, policy_version 39332 (0.0008) [2023-10-13 22:32:35,061][60934] Updated weights for policy 1, policy_version 39342 (0.0008) [2023-10-13 22:32:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 80150528. Throughput: 0: 1714.0, 1: 1693.2. Samples: 20045404. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-13 22:32:36,249][59943] Avg episode reward: [(0, '-0.170'), (1, '0.000')] [2023-10-13 22:32:36,701][60935] Updated weights for policy 0, policy_version 38890 (0.0009) [2023-10-13 22:32:37,074][60935] Updated weights for policy 0, policy_version 38900 (0.0008) [2023-10-13 22:32:37,432][60935] Updated weights for policy 0, policy_version 38910 (0.0010) [2023-10-13 22:32:39,166][60934] Updated weights for policy 1, policy_version 39352 (0.0008) [2023-10-13 22:32:39,536][60934] Updated weights for policy 1, policy_version 39362 (0.0010) [2023-10-13 22:32:39,903][60934] Updated weights for policy 1, policy_version 39372 (0.0010) [2023-10-13 22:32:41,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 80216064. Throughput: 0: 1714.8, 1: 1681.6. Samples: 20065660. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-13 22:32:41,249][59943] Avg episode reward: [(0, '-0.170'), (1, '0.000')] [2023-10-13 22:32:41,508][60935] Updated weights for policy 0, policy_version 38920 (0.0010) [2023-10-13 22:32:41,884][60935] Updated weights for policy 0, policy_version 38930 (0.0007) [2023-10-13 22:32:42,248][60935] Updated weights for policy 0, policy_version 38940 (0.0009) [2023-10-13 22:32:43,933][60934] Updated weights for policy 1, policy_version 39382 (0.0010) [2023-10-13 22:32:44,324][60934] Updated weights for policy 1, policy_version 39392 (0.0009) [2023-10-13 22:32:44,684][60934] Updated weights for policy 1, policy_version 39402 (0.0009) [2023-10-13 22:32:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 80281600. Throughput: 0: 1708.0, 1: 1711.7. Samples: 20076140. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-13 22:32:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:32:46,280][60935] Updated weights for policy 0, policy_version 38950 (0.0010) [2023-10-13 22:32:46,658][60935] Updated weights for policy 0, policy_version 38960 (0.0009) [2023-10-13 22:32:47,026][60935] Updated weights for policy 0, policy_version 38970 (0.0011) [2023-10-13 22:32:48,689][60934] Updated weights for policy 1, policy_version 39412 (0.0009) [2023-10-13 22:32:49,054][60934] Updated weights for policy 1, policy_version 39422 (0.0008) [2023-10-13 22:32:49,420][60934] Updated weights for policy 1, policy_version 39432 (0.0007) [2023-10-13 22:32:51,084][60935] Updated weights for policy 0, policy_version 38980 (0.0010) [2023-10-13 22:32:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 80347136. Throughput: 0: 1712.5, 1: 1681.2. Samples: 20095950. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-13 22:32:51,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:32:51,463][60935] Updated weights for policy 0, policy_version 38990 (0.0007) [2023-10-13 22:32:51,831][60935] Updated weights for policy 0, policy_version 39000 (0.0010) [2023-10-13 22:32:53,513][60934] Updated weights for policy 1, policy_version 39442 (0.0008) [2023-10-13 22:32:53,876][60934] Updated weights for policy 1, policy_version 39452 (0.0010) [2023-10-13 22:32:54,245][60934] Updated weights for policy 1, policy_version 39462 (0.0010) [2023-10-13 22:32:54,615][60934] Updated weights for policy 1, policy_version 39472 (0.0009) [2023-10-13 22:32:55,752][60935] Updated weights for policy 0, policy_version 39010 (0.0010) [2023-10-13 22:32:56,158][60935] Updated weights for policy 0, policy_version 39020 (0.0009) [2023-10-13 22:32:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 80412672. Throughput: 0: 1703.6, 1: 1691.2. Samples: 20116200. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-13 22:32:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:32:56,256][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000039472_40468480.pth... [2023-10-13 22:32:56,292][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000037904_38830080.pth [2023-10-13 22:32:56,296][60828] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p1/milestones/checkpoint_000039472_40468480.pth [2023-10-13 22:32:56,526][60935] Updated weights for policy 0, policy_version 39030 (0.0008) [2023-10-13 22:32:56,897][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000039040_39976960.pth... [2023-10-13 22:32:56,900][60935] Updated weights for policy 0, policy_version 39040 (0.0008) [2023-10-13 22:32:56,926][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000037440_38338560.pth [2023-10-13 22:32:56,930][60695] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p0/milestones/checkpoint_000039040_39976960.pth [2023-10-13 22:32:58,666][60934] Updated weights for policy 1, policy_version 39482 (0.0011) [2023-10-13 22:32:59,032][60934] Updated weights for policy 1, policy_version 39492 (0.0010) [2023-10-13 22:32:59,403][60934] Updated weights for policy 1, policy_version 39502 (0.0009) [2023-10-13 22:33:00,897][60935] Updated weights for policy 0, policy_version 39050 (0.0009) [2023-10-13 22:33:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 80478208. Throughput: 0: 1707.3, 1: 1693.7. Samples: 20126524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:33:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:33:01,272][60935] Updated weights for policy 0, policy_version 39060 (0.0008) [2023-10-13 22:33:01,641][60935] Updated weights for policy 0, policy_version 39070 (0.0007) [2023-10-13 22:33:03,242][60934] Updated weights for policy 1, policy_version 39512 (0.0008) [2023-10-13 22:33:03,606][60934] Updated weights for policy 1, policy_version 39522 (0.0008) [2023-10-13 22:33:03,967][60934] Updated weights for policy 1, policy_version 39532 (0.0007) [2023-10-13 22:33:05,819][60935] Updated weights for policy 0, policy_version 39080 (0.0008) [2023-10-13 22:33:06,190][60935] Updated weights for policy 0, policy_version 39090 (0.0009) [2023-10-13 22:33:06,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 80543744. Throughput: 0: 1699.9, 1: 1680.4. Samples: 20146630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:33:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:33:06,551][60935] Updated weights for policy 0, policy_version 39100 (0.0009) [2023-10-13 22:33:07,877][60934] Updated weights for policy 1, policy_version 39542 (0.0010) [2023-10-13 22:33:08,250][60934] Updated weights for policy 1, policy_version 39552 (0.0008) [2023-10-13 22:33:08,623][60934] Updated weights for policy 1, policy_version 39562 (0.0009) [2023-10-13 22:33:10,587][60935] Updated weights for policy 0, policy_version 39110 (0.0011) [2023-10-13 22:33:10,965][60935] Updated weights for policy 0, policy_version 39120 (0.0010) [2023-10-13 22:33:11,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 80609280. Throughput: 0: 1682.7, 1: 1708.3. Samples: 20167096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:33:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:33:11,332][60935] Updated weights for policy 0, policy_version 39130 (0.0008) [2023-10-13 22:33:12,666][60934] Updated weights for policy 1, policy_version 39572 (0.0008) [2023-10-13 22:33:13,032][60934] Updated weights for policy 1, policy_version 39582 (0.0008) [2023-10-13 22:33:13,102][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-13 22:33:15,397][60935] Updated weights for policy 0, policy_version 39140 (0.0009) [2023-10-13 22:33:15,771][60935] Updated weights for policy 0, policy_version 39150 (0.0008) [2023-10-13 22:33:16,137][60935] Updated weights for policy 0, policy_version 39160 (0.0008) [2023-10-13 22:33:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 80674816. Throughput: 0: 1690.7, 1: 1697.6. Samples: 20177372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:33:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:33:17,119][60934] Updated weights for policy 1, policy_version 39592 (0.0009) [2023-10-13 22:33:17,408][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-13 22:33:20,154][60935] Updated weights for policy 0, policy_version 39170 (0.0010) [2023-10-13 22:33:20,518][60935] Updated weights for policy 0, policy_version 39180 (0.0009) [2023-10-13 22:33:20,886][60935] Updated weights for policy 0, policy_version 39190 (0.0008) [2023-10-13 22:33:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 80740352. Throughput: 0: 1693.1, 1: 1724.5. Samples: 20199194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:33:21,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:33:21,258][60935] Updated weights for policy 0, policy_version 39200 (0.0007) [2023-10-13 22:33:21,388][60934] Updated weights for policy 1, policy_version 39602 (0.0009) [2023-10-13 22:33:21,752][60934] Updated weights for policy 1, policy_version 39612 (0.0009) [2023-10-13 22:33:22,111][60934] Updated weights for policy 1, policy_version 39622 (0.0009) [2023-10-13 22:33:22,475][60934] Updated weights for policy 1, policy_version 39632 (0.0009) [2023-10-13 22:33:25,587][60935] Updated weights for policy 0, policy_version 39210 (0.0010) [2023-10-13 22:33:25,953][60935] Updated weights for policy 0, policy_version 39220 (0.0008) [2023-10-13 22:33:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 80805888. Throughput: 0: 1669.4, 1: 1743.9. Samples: 20219258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:33:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:33:26,318][60935] Updated weights for policy 0, policy_version 39230 (0.0009) [2023-10-13 22:33:26,531][60934] Updated weights for policy 1, policy_version 39642 (0.0007) [2023-10-13 22:33:26,893][60934] Updated weights for policy 1, policy_version 39652 (0.0007) [2023-10-13 22:33:27,250][60934] Updated weights for policy 1, policy_version 39662 (0.0007) [2023-10-13 22:33:30,250][60935] Updated weights for policy 0, policy_version 39240 (0.0011) [2023-10-13 22:33:30,626][60935] Updated weights for policy 0, policy_version 39250 (0.0010) [2023-10-13 22:33:30,992][60935] Updated weights for policy 0, policy_version 39260 (0.0008) [2023-10-13 22:33:31,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 80904192. Throughput: 0: 1687.0, 1: 1714.9. Samples: 20229226. Policy #0 lag: (min: 23.0, avg: 30.9, max: 55.0) [2023-10-13 22:33:31,249][59943] Avg episode reward: [(0, '-0.280'), (1, '0.000')] [2023-10-13 22:33:31,389][60934] Updated weights for policy 1, policy_version 39672 (0.0008) [2023-10-13 22:33:31,764][60934] Updated weights for policy 1, policy_version 39682 (0.0008) [2023-10-13 22:33:32,125][60934] Updated weights for policy 1, policy_version 39692 (0.0008) [2023-10-13 22:33:35,138][60935] Updated weights for policy 0, policy_version 39270 (0.0009) [2023-10-13 22:33:35,515][60935] Updated weights for policy 0, policy_version 39280 (0.0009) [2023-10-13 22:33:35,879][60935] Updated weights for policy 0, policy_version 39290 (0.0009) [2023-10-13 22:33:36,167][60934] Updated weights for policy 1, policy_version 39702 (0.0008) [2023-10-13 22:33:36,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 80969728. Throughput: 0: 1685.0, 1: 1744.5. Samples: 20250278. Policy #0 lag: (min: 23.0, avg: 30.9, max: 55.0) [2023-10-13 22:33:36,249][59943] Avg episode reward: [(0, '-0.560'), (1, '0.000')] [2023-10-13 22:33:36,534][60934] Updated weights for policy 1, policy_version 39712 (0.0008) [2023-10-13 22:33:36,897][60934] Updated weights for policy 1, policy_version 39722 (0.0009) [2023-10-13 22:33:39,846][60935] Updated weights for policy 0, policy_version 39300 (0.0008) [2023-10-13 22:33:40,209][60935] Updated weights for policy 0, policy_version 39310 (0.0007) [2023-10-13 22:33:40,575][60935] Updated weights for policy 0, policy_version 39320 (0.0010) [2023-10-13 22:33:40,706][60934] Updated weights for policy 1, policy_version 39732 (0.0009) [2023-10-13 22:33:41,073][60934] Updated weights for policy 1, policy_version 39742 (0.0009) [2023-10-13 22:33:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 81035264. Throughput: 0: 1664.4, 1: 1760.8. Samples: 20270338. Policy #0 lag: (min: 23.0, avg: 30.9, max: 55.0) [2023-10-13 22:33:41,249][59943] Avg episode reward: [(0, '-0.280'), (1, '0.000')] [2023-10-13 22:33:41,434][60934] Updated weights for policy 1, policy_version 39752 (0.0011) [2023-10-13 22:33:44,664][60935] Updated weights for policy 0, policy_version 39330 (0.0008) [2023-10-13 22:33:45,051][60935] Updated weights for policy 0, policy_version 39340 (0.0011) [2023-10-13 22:33:45,313][60934] Updated weights for policy 1, policy_version 39762 (0.0009) [2023-10-13 22:33:45,411][60935] Updated weights for policy 0, policy_version 39350 (0.0009) [2023-10-13 22:33:45,681][60934] Updated weights for policy 1, policy_version 39772 (0.0007) [2023-10-13 22:33:45,778][60935] Updated weights for policy 0, policy_version 39360 (0.0008) [2023-10-13 22:33:45,821][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000002 [2023-10-13 22:33:46,248][59943] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 81133568. Throughput: 0: 1687.3, 1: 1740.4. Samples: 20280772. Policy #0 lag: (min: 23.0, avg: 30.9, max: 55.0) [2023-10-13 22:33:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:33:49,937][60934] Updated weights for policy 1, policy_version 39782 (0.0007) [2023-10-13 22:33:49,964][60935] Updated weights for policy 0, policy_version 39370 (0.0009) [2023-10-13 22:33:50,298][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-13 22:33:50,299][60934] Updated weights for policy 1, policy_version 39792 (0.0008) [2023-10-13 22:33:50,342][60935] Updated weights for policy 0, policy_version 39380 (0.0008) [2023-10-13 22:33:50,712][60935] Updated weights for policy 0, policy_version 39390 (0.0010) [2023-10-13 22:33:51,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 81199104. Throughput: 0: 1675.1, 1: 1771.7. Samples: 20301738. Policy #0 lag: (min: 23.0, avg: 30.9, max: 55.0) [2023-10-13 22:33:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:33:54,636][60935] Updated weights for policy 0, policy_version 39400 (0.0008) [2023-10-13 22:33:54,767][60934] Updated weights for policy 1, policy_version 39802 (0.0008) [2023-10-13 22:33:54,984][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-13 22:33:54,999][60935] Updated weights for policy 0, policy_version 39410 (0.0008) [2023-10-13 22:33:55,380][60935] Updated weights for policy 0, policy_version 39420 (0.0010) [2023-10-13 22:33:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 81264640. Throughput: 0: 1661.5, 1: 1764.7. Samples: 20321278. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-13 22:33:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:33:58,923][60934] Updated weights for policy 1, policy_version 39812 (0.0010) [2023-10-13 22:33:59,283][60934] Updated weights for policy 1, policy_version 39822 (0.0008) [2023-10-13 22:33:59,385][60935] Updated weights for policy 0, policy_version 39430 (0.0009) [2023-10-13 22:33:59,651][60934] Updated weights for policy 1, policy_version 39832 (0.0007) [2023-10-13 22:33:59,758][60935] Updated weights for policy 0, policy_version 39440 (0.0009) [2023-10-13 22:34:00,124][60935] Updated weights for policy 0, policy_version 39450 (0.0010) [2023-10-13 22:34:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 81330176. Throughput: 0: 1686.4, 1: 1778.7. Samples: 20333302. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-13 22:34:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:34:03,465][60934] Updated weights for policy 1, policy_version 39842 (0.0010) [2023-10-13 22:34:03,847][60934] Updated weights for policy 1, policy_version 39852 (0.0008) [2023-10-13 22:34:03,988][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000010 [2023-10-13 22:34:04,139][60935] Updated weights for policy 0, policy_version 39460 (0.0008) [2023-10-13 22:34:04,519][60935] Updated weights for policy 0, policy_version 39470 (0.0008) [2023-10-13 22:34:04,887][60935] Updated weights for policy 0, policy_version 39480 (0.0008) [2023-10-13 22:34:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 81395712. Throughput: 0: 1666.3, 1: 1749.6. Samples: 20352908. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-13 22:34:06,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:34:08,206][60934] Updated weights for policy 1, policy_version 39862 (0.0008) [2023-10-13 22:34:08,572][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-13 22:34:08,574][60934] Updated weights for policy 1, policy_version 39872 (0.0010) [2023-10-13 22:34:08,918][60935] Updated weights for policy 0, policy_version 39490 (0.0008) [2023-10-13 22:34:09,290][60935] Updated weights for policy 0, policy_version 39500 (0.0009) [2023-10-13 22:34:09,653][60935] Updated weights for policy 0, policy_version 39510 (0.0007) [2023-10-13 22:34:10,022][60935] Updated weights for policy 0, policy_version 39520 (0.0007) [2023-10-13 22:34:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 81461248. Throughput: 0: 1675.6, 1: 1769.1. Samples: 20374272. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-13 22:34:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:34:12,727][60934] Updated weights for policy 1, policy_version 39882 (0.0007) [2023-10-13 22:34:13,088][60934] Updated weights for policy 1, policy_version 39892 (0.0008) [2023-10-13 22:34:13,457][60934] Updated weights for policy 1, policy_version 39902 (0.0007) [2023-10-13 22:34:14,150][60935] Updated weights for policy 0, policy_version 39530 (0.0007) [2023-10-13 22:34:14,521][60935] Updated weights for policy 0, policy_version 39540 (0.0010) [2023-10-13 22:34:14,895][60935] Updated weights for policy 0, policy_version 39550 (0.0011) [2023-10-13 22:34:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 81526784. Throughput: 0: 1685.3, 1: 1770.4. Samples: 20384730. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-13 22:34:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:34:17,508][60934] Updated weights for policy 1, policy_version 39912 (0.0009) [2023-10-13 22:34:17,885][60934] Updated weights for policy 1, policy_version 39922 (0.0007) [2023-10-13 22:34:18,244][60934] Updated weights for policy 1, policy_version 39932 (0.0010) [2023-10-13 22:34:18,847][60935] Updated weights for policy 0, policy_version 39560 (0.0007) [2023-10-13 22:34:19,221][60935] Updated weights for policy 0, policy_version 39570 (0.0007) [2023-10-13 22:34:19,593][60935] Updated weights for policy 0, policy_version 39580 (0.0009) [2023-10-13 22:34:21,248][59943] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 81592320. Throughput: 0: 1658.9, 1: 1774.7. Samples: 20404792. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-13 22:34:21,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:34:22,175][60934] Updated weights for policy 1, policy_version 39942 (0.0009) [2023-10-13 22:34:22,555][60934] Updated weights for policy 1, policy_version 39952 (0.0010) [2023-10-13 22:34:22,925][60934] Updated weights for policy 1, policy_version 39962 (0.0008) [2023-10-13 22:34:23,638][60935] Updated weights for policy 0, policy_version 39590 (0.0010) [2023-10-13 22:34:24,018][60935] Updated weights for policy 0, policy_version 39600 (0.0008) [2023-10-13 22:34:24,391][60935] Updated weights for policy 0, policy_version 39610 (0.0007) [2023-10-13 22:34:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 81657856. Throughput: 0: 1689.2, 1: 1770.0. Samples: 20426004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:34:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:34:26,771][60934] Updated weights for policy 1, policy_version 39972 (0.0008) [2023-10-13 22:34:27,128][60934] Updated weights for policy 1, policy_version 39982 (0.0008) [2023-10-13 22:34:27,488][60934] Updated weights for policy 1, policy_version 39992 (0.0010) [2023-10-13 22:34:28,525][60935] Updated weights for policy 0, policy_version 39620 (0.0009) [2023-10-13 22:34:28,895][60935] Updated weights for policy 0, policy_version 39630 (0.0009) [2023-10-13 22:34:29,263][60935] Updated weights for policy 0, policy_version 39640 (0.0012) [2023-10-13 22:34:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 81723392. Throughput: 0: 1681.6, 1: 1766.9. Samples: 20435956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:34:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:34:31,461][60934] Updated weights for policy 1, policy_version 40002 (0.0010) [2023-10-13 22:34:31,831][60934] Updated weights for policy 1, policy_version 40012 (0.0008) [2023-10-13 22:34:32,197][60934] Updated weights for policy 1, policy_version 40022 (0.0009) [2023-10-13 22:34:32,569][60934] Updated weights for policy 1, policy_version 40032 (0.0009) [2023-10-13 22:34:33,357][60935] Updated weights for policy 0, policy_version 39650 (0.0010) [2023-10-13 22:34:33,726][60935] Updated weights for policy 0, policy_version 39660 (0.0007) [2023-10-13 22:34:34,097][60935] Updated weights for policy 0, policy_version 39670 (0.0007) [2023-10-13 22:34:34,479][60935] Updated weights for policy 0, policy_version 39680 (0.0008) [2023-10-13 22:34:36,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 81788928. Throughput: 0: 1673.9, 1: 1758.8. Samples: 20456210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:34:36,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:34:36,401][60934] Updated weights for policy 1, policy_version 40042 (0.0010) [2023-10-13 22:34:36,770][60934] Updated weights for policy 1, policy_version 40052 (0.0010) [2023-10-13 22:34:37,141][60934] Updated weights for policy 1, policy_version 40062 (0.0009) [2023-10-13 22:34:38,712][60935] Updated weights for policy 0, policy_version 39690 (0.0008) [2023-10-13 22:34:39,084][60935] Updated weights for policy 0, policy_version 39700 (0.0010) [2023-10-13 22:34:39,455][60935] Updated weights for policy 0, policy_version 39710 (0.0008) [2023-10-13 22:34:41,045][60934] Updated weights for policy 1, policy_version 40072 (0.0008) [2023-10-13 22:34:41,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 81854464. Throughput: 0: 1690.7, 1: 1769.3. Samples: 20476980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:34:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:34:41,339][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000009 [2023-10-13 22:34:43,492][60935] Updated weights for policy 0, policy_version 39720 (0.0007) [2023-10-13 22:34:43,859][60935] Updated weights for policy 0, policy_version 39730 (0.0008) [2023-10-13 22:34:44,236][60935] Updated weights for policy 0, policy_version 39740 (0.0007) [2023-10-13 22:34:45,432][60934] Updated weights for policy 1, policy_version 40082 (0.0009) [2023-10-13 22:34:45,798][60934] Updated weights for policy 1, policy_version 40092 (0.0007) [2023-10-13 22:34:46,164][60934] Updated weights for policy 1, policy_version 40102 (0.0008) [2023-10-13 22:34:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 81920000. Throughput: 0: 1672.0, 1: 1748.9. Samples: 20487244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:34:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:34:46,538][60934] Updated weights for policy 1, policy_version 40112 (0.0008) [2023-10-13 22:34:48,306][60935] Updated weights for policy 0, policy_version 39750 (0.0009) [2023-10-13 22:34:48,662][60935] Updated weights for policy 0, policy_version 39760 (0.0009) [2023-10-13 22:34:49,027][60935] Updated weights for policy 0, policy_version 39770 (0.0008) [2023-10-13 22:34:50,443][60934] Updated weights for policy 1, policy_version 40122 (0.0009) [2023-10-13 22:34:50,661][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-13 22:34:51,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 82018304. Throughput: 0: 1672.7, 1: 1769.5. Samples: 20507806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:34:51,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:34:53,013][60935] Updated weights for policy 0, policy_version 39780 (0.0010) [2023-10-13 22:34:53,387][60935] Updated weights for policy 0, policy_version 39790 (0.0008) [2023-10-13 22:34:53,766][60935] Updated weights for policy 0, policy_version 39800 (0.0010) [2023-10-13 22:34:54,777][60934] Updated weights for policy 1, policy_version 40132 (0.0009) [2023-10-13 22:34:55,143][60934] Updated weights for policy 1, policy_version 40142 (0.0007) [2023-10-13 22:34:55,507][60934] Updated weights for policy 1, policy_version 40152 (0.0008) [2023-10-13 22:34:56,248][59943] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 82083840. Throughput: 0: 1681.0, 1: 1743.1. Samples: 20528356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:34:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:34:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000039808_40763392.pth... [2023-10-13 22:34:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000040160_41320448.pth... [2023-10-13 22:34:56,292][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000038240_39157760.pth [2023-10-13 22:34:56,307][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000038672_39649280.pth [2023-10-13 22:34:57,943][60935] Updated weights for policy 0, policy_version 39810 (0.0009) [2023-10-13 22:34:58,307][60935] Updated weights for policy 0, policy_version 39820 (0.0009) [2023-10-13 22:34:58,670][60935] Updated weights for policy 0, policy_version 39830 (0.0007) [2023-10-13 22:34:59,044][60935] Updated weights for policy 0, policy_version 39840 (0.0007) [2023-10-13 22:34:59,454][60934] Updated weights for policy 1, policy_version 40162 (0.0009) [2023-10-13 22:34:59,818][60934] Updated weights for policy 1, policy_version 40172 (0.0009) [2023-10-13 22:35:00,175][60934] Updated weights for policy 1, policy_version 40182 (0.0008) [2023-10-13 22:35:00,543][60934] Updated weights for policy 1, policy_version 40192 (0.0008) [2023-10-13 22:35:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 82149376. Throughput: 0: 1661.4, 1: 1762.7. Samples: 20538814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:35:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:35:02,966][60935] Updated weights for policy 0, policy_version 39850 (0.0007) [2023-10-13 22:35:03,333][60935] Updated weights for policy 0, policy_version 39860 (0.0007) [2023-10-13 22:35:03,693][60935] Updated weights for policy 0, policy_version 39870 (0.0009) [2023-10-13 22:35:04,516][60934] Updated weights for policy 1, policy_version 40202 (0.0010) [2023-10-13 22:35:04,881][60934] Updated weights for policy 1, policy_version 40212 (0.0009) [2023-10-13 22:35:05,027][60828] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000007 [2023-10-13 22:35:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 82214912. Throughput: 0: 1677.9, 1: 1747.9. Samples: 20558952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:35:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:35:07,918][60935] Updated weights for policy 0, policy_version 39880 (0.0008) [2023-10-13 22:35:08,287][60935] Updated weights for policy 0, policy_version 39890 (0.0009) [2023-10-13 22:35:08,650][60935] Updated weights for policy 0, policy_version 39900 (0.0010) [2023-10-13 22:35:08,958][60934] Updated weights for policy 1, policy_version 40222 (0.0009) [2023-10-13 22:35:09,323][60934] Updated weights for policy 1, policy_version 40232 (0.0010) [2023-10-13 22:35:09,693][60934] Updated weights for policy 1, policy_version 40242 (0.0010) [2023-10-13 22:35:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 82280448. Throughput: 0: 1674.9, 1: 1731.0. Samples: 20579270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:35:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:35:12,623][60935] Updated weights for policy 0, policy_version 39910 (0.0008) [2023-10-13 22:35:12,989][60935] Updated weights for policy 0, policy_version 39920 (0.0007) [2023-10-13 22:35:13,372][60935] Updated weights for policy 0, policy_version 39930 (0.0008) [2023-10-13 22:35:13,617][60934] Updated weights for policy 1, policy_version 40252 (0.0007) [2023-10-13 22:35:13,975][60934] Updated weights for policy 1, policy_version 40262 (0.0010) [2023-10-13 22:35:14,045][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-13 22:35:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 82345984. Throughput: 0: 1656.2, 1: 1762.4. Samples: 20589794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:35:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:35:17,436][60935] Updated weights for policy 0, policy_version 39940 (0.0008) [2023-10-13 22:35:17,798][60935] Updated weights for policy 0, policy_version 39950 (0.0009) [2023-10-13 22:35:18,170][60935] Updated weights for policy 0, policy_version 39960 (0.0007) [2023-10-13 22:35:18,285][60934] Updated weights for policy 1, policy_version 40272 (0.0008) [2023-10-13 22:35:18,579][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-13 22:35:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 82411520. Throughput: 0: 1673.6, 1: 1759.3. Samples: 20610692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:35:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:35:22,254][60935] Updated weights for policy 0, policy_version 39970 (0.0007) [2023-10-13 22:35:22,534][60934] Updated weights for policy 1, policy_version 40282 (0.0008) [2023-10-13 22:35:22,625][60935] Updated weights for policy 0, policy_version 39980 (0.0008) [2023-10-13 22:35:22,900][60934] Updated weights for policy 1, policy_version 40292 (0.0007) [2023-10-13 22:35:23,003][60935] Updated weights for policy 0, policy_version 39990 (0.0007) [2023-10-13 22:35:23,046][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-13 22:35:23,366][60935] Updated weights for policy 0, policy_version 40000 (0.0009) [2023-10-13 22:35:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 82477056. Throughput: 0: 1680.4, 1: 1769.1. Samples: 20632208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:35:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:35:27,203][60934] Updated weights for policy 1, policy_version 40302 (0.0009) [2023-10-13 22:35:27,552][60935] Updated weights for policy 0, policy_version 40010 (0.0009) [2023-10-13 22:35:27,571][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-13 22:35:27,577][60934] Updated weights for policy 1, policy_version 40312 (0.0007) [2023-10-13 22:35:27,934][60935] Updated weights for policy 0, policy_version 40020 (0.0009) [2023-10-13 22:35:28,307][60935] Updated weights for policy 0, policy_version 40030 (0.0010) [2023-10-13 22:35:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 82542592. Throughput: 0: 1661.7, 1: 1773.9. Samples: 20641848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:35:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:35:31,866][60934] Updated weights for policy 1, policy_version 40322 (0.0009) [2023-10-13 22:35:32,086][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-13 22:35:32,411][60935] Updated weights for policy 0, policy_version 40040 (0.0008) [2023-10-13 22:35:32,782][60935] Updated weights for policy 0, policy_version 40050 (0.0009) [2023-10-13 22:35:33,158][60935] Updated weights for policy 0, policy_version 40060 (0.0009) [2023-10-13 22:35:36,105][60934] Updated weights for policy 1, policy_version 40332 (0.0010) [2023-10-13 22:35:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 82608128. Throughput: 0: 1671.8, 1: 1781.4. Samples: 20663198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:35:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:35:36,461][60934] Updated weights for policy 1, policy_version 40342 (0.0010) [2023-10-13 22:35:36,533][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-13 22:35:37,368][60935] Updated weights for policy 0, policy_version 40070 (0.0008) [2023-10-13 22:35:37,736][60935] Updated weights for policy 0, policy_version 40080 (0.0008) [2023-10-13 22:35:38,104][60935] Updated weights for policy 0, policy_version 40090 (0.0008) [2023-10-13 22:35:40,824][60934] Updated weights for policy 1, policy_version 40352 (0.0009) [2023-10-13 22:35:41,191][60934] Updated weights for policy 1, policy_version 40362 (0.0007) [2023-10-13 22:35:41,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 82673664. Throughput: 0: 1670.8, 1: 1801.7. Samples: 20684616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:35:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:35:41,564][60934] Updated weights for policy 1, policy_version 40372 (0.0008) [2023-10-13 22:35:42,239][60935] Updated weights for policy 0, policy_version 40100 (0.0007) [2023-10-13 22:35:42,612][60935] Updated weights for policy 0, policy_version 40110 (0.0010) [2023-10-13 22:35:42,981][60935] Updated weights for policy 0, policy_version 40120 (0.0009) [2023-10-13 22:35:45,536][60934] Updated weights for policy 1, policy_version 40382 (0.0009) [2023-10-13 22:35:45,892][60934] Updated weights for policy 1, policy_version 40392 (0.0007) [2023-10-13 22:35:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 82739200. Throughput: 0: 1663.4, 1: 1781.1. Samples: 20693816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:35:46,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:35:46,261][60934] Updated weights for policy 1, policy_version 40402 (0.0011) [2023-10-13 22:35:46,788][60935] Updated weights for policy 0, policy_version 40130 (0.0008) [2023-10-13 22:35:47,162][60935] Updated weights for policy 0, policy_version 40140 (0.0009) [2023-10-13 22:35:47,522][60935] Updated weights for policy 0, policy_version 40150 (0.0010) [2023-10-13 22:35:47,892][60935] Updated weights for policy 0, policy_version 40160 (0.0010) [2023-10-13 22:35:50,233][60934] Updated weights for policy 1, policy_version 40412 (0.0007) [2023-10-13 22:35:50,632][60934] Updated weights for policy 1, policy_version 40422 (0.0008) [2023-10-13 22:35:51,003][60934] Updated weights for policy 1, policy_version 40432 (0.0008) [2023-10-13 22:35:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 82804736. Throughput: 0: 1675.6, 1: 1795.2. Samples: 20715136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:35:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:35:51,858][60935] Updated weights for policy 0, policy_version 40170 (0.0008) [2023-10-13 22:35:52,231][60935] Updated weights for policy 0, policy_version 40180 (0.0009) [2023-10-13 22:35:52,612][60935] Updated weights for policy 0, policy_version 40190 (0.0010) [2023-10-13 22:35:54,914][60934] Updated weights for policy 1, policy_version 40442 (0.0009) [2023-10-13 22:35:55,283][60934] Updated weights for policy 1, policy_version 40452 (0.0008) [2023-10-13 22:35:55,644][60934] Updated weights for policy 1, policy_version 40462 (0.0007) [2023-10-13 22:35:56,016][60934] Updated weights for policy 1, policy_version 40472 (0.0008) [2023-10-13 22:35:56,248][59943] Fps is (10 sec: 16383.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 82903040. Throughput: 0: 1685.0, 1: 1788.4. Samples: 20735574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:35:56,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:35:56,619][60935] Updated weights for policy 0, policy_version 40200 (0.0008) [2023-10-13 22:35:56,979][60935] Updated weights for policy 0, policy_version 40210 (0.0009) [2023-10-13 22:35:57,352][60935] Updated weights for policy 0, policy_version 40220 (0.0011) [2023-10-13 22:35:59,995][60934] Updated weights for policy 1, policy_version 40482 (0.0007) [2023-10-13 22:36:00,356][60934] Updated weights for policy 1, policy_version 40492 (0.0008) [2023-10-13 22:36:00,716][60934] Updated weights for policy 1, policy_version 40502 (0.0007) [2023-10-13 22:36:01,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 82968576. Throughput: 0: 1684.6, 1: 1775.0. Samples: 20745478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:36:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:36:01,369][60935] Updated weights for policy 0, policy_version 40230 (0.0008) [2023-10-13 22:36:01,746][60935] Updated weights for policy 0, policy_version 40240 (0.0008) [2023-10-13 22:36:02,111][60935] Updated weights for policy 0, policy_version 40250 (0.0009) [2023-10-13 22:36:04,897][60934] Updated weights for policy 1, policy_version 40512 (0.0007) [2023-10-13 22:36:05,274][60934] Updated weights for policy 1, policy_version 40522 (0.0007) [2023-10-13 22:36:05,636][60934] Updated weights for policy 1, policy_version 40532 (0.0009) [2023-10-13 22:36:06,195][60935] Updated weights for policy 0, policy_version 40260 (0.0009) [2023-10-13 22:36:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 83034112. Throughput: 0: 1688.5, 1: 1767.7. Samples: 20766224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:36:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:36:06,576][60935] Updated weights for policy 0, policy_version 40270 (0.0010) [2023-10-13 22:36:06,948][60935] Updated weights for policy 0, policy_version 40280 (0.0010) [2023-10-13 22:36:09,435][60934] Updated weights for policy 1, policy_version 40542 (0.0008) [2023-10-13 22:36:09,802][60934] Updated weights for policy 1, policy_version 40552 (0.0009) [2023-10-13 22:36:10,161][60934] Updated weights for policy 1, policy_version 40562 (0.0007) [2023-10-13 22:36:10,715][60935] Updated weights for policy 0, policy_version 40290 (0.0010) [2023-10-13 22:36:11,097][60935] Updated weights for policy 0, policy_version 40300 (0.0010) [2023-10-13 22:36:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 83099648. Throughput: 0: 1692.9, 1: 1728.0. Samples: 20786150. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:36:11,249][59943] Avg episode reward: [(0, '-0.120'), (1, '0.000')] [2023-10-13 22:36:11,457][60935] Updated weights for policy 0, policy_version 40310 (0.0010) [2023-10-13 22:36:11,828][60935] Updated weights for policy 0, policy_version 40320 (0.0010) [2023-10-13 22:36:14,035][60934] Updated weights for policy 1, policy_version 40572 (0.0010) [2023-10-13 22:36:14,405][60934] Updated weights for policy 1, policy_version 40582 (0.0008) [2023-10-13 22:36:14,760][60934] Updated weights for policy 1, policy_version 40592 (0.0009) [2023-10-13 22:36:15,960][60935] Updated weights for policy 0, policy_version 40330 (0.0009) [2023-10-13 22:36:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 83165184. Throughput: 0: 1700.5, 1: 1747.7. Samples: 20797018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:36:16,249][59943] Avg episode reward: [(0, '-0.120'), (1, '0.000')] [2023-10-13 22:36:16,337][60935] Updated weights for policy 0, policy_version 40340 (0.0007) [2023-10-13 22:36:16,714][60935] Updated weights for policy 0, policy_version 40350 (0.0008) [2023-10-13 22:36:18,767][60934] Updated weights for policy 1, policy_version 40602 (0.0009) [2023-10-13 22:36:19,129][60934] Updated weights for policy 1, policy_version 40612 (0.0009) [2023-10-13 22:36:19,505][60934] Updated weights for policy 1, policy_version 40622 (0.0008) [2023-10-13 22:36:19,863][60934] Updated weights for policy 1, policy_version 40632 (0.0007) [2023-10-13 22:36:20,749][60935] Updated weights for policy 0, policy_version 40360 (0.0007) [2023-10-13 22:36:21,124][60935] Updated weights for policy 0, policy_version 40370 (0.0008) [2023-10-13 22:36:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 83230720. Throughput: 0: 1703.7, 1: 1715.7. Samples: 20817074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:36:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:36:21,499][60935] Updated weights for policy 0, policy_version 40380 (0.0007) [2023-10-13 22:36:23,701][60934] Updated weights for policy 1, policy_version 40642 (0.0009) [2023-10-13 22:36:24,068][60934] Updated weights for policy 1, policy_version 40652 (0.0009) [2023-10-13 22:36:24,432][60934] Updated weights for policy 1, policy_version 40662 (0.0007) [2023-10-13 22:36:25,672][60935] Updated weights for policy 0, policy_version 40390 (0.0008) [2023-10-13 22:36:26,045][60935] Updated weights for policy 0, policy_version 40400 (0.0009) [2023-10-13 22:36:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 83296256. Throughput: 0: 1695.2, 1: 1695.9. Samples: 20837214. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-13 22:36:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:36:26,417][60935] Updated weights for policy 0, policy_version 40410 (0.0008) [2023-10-13 22:36:28,567][60934] Updated weights for policy 1, policy_version 40672 (0.0008) [2023-10-13 22:36:28,929][60934] Updated weights for policy 1, policy_version 40682 (0.0010) [2023-10-13 22:36:29,293][60934] Updated weights for policy 1, policy_version 40692 (0.0010) [2023-10-13 22:36:30,510][60935] Updated weights for policy 0, policy_version 40420 (0.0009) [2023-10-13 22:36:30,878][60935] Updated weights for policy 0, policy_version 40430 (0.0009) [2023-10-13 22:36:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 83361792. Throughput: 0: 1703.9, 1: 1716.2. Samples: 20847720. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-13 22:36:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:36:31,258][60935] Updated weights for policy 0, policy_version 40440 (0.0010) [2023-10-13 22:36:33,231][60934] Updated weights for policy 1, policy_version 40702 (0.0007) [2023-10-13 22:36:33,600][60934] Updated weights for policy 1, policy_version 40712 (0.0008) [2023-10-13 22:36:33,961][60934] Updated weights for policy 1, policy_version 40722 (0.0008) [2023-10-13 22:36:35,376][60935] Updated weights for policy 0, policy_version 40450 (0.0011) [2023-10-13 22:36:35,737][60935] Updated weights for policy 0, policy_version 40460 (0.0008) [2023-10-13 22:36:36,101][60935] Updated weights for policy 0, policy_version 40470 (0.0009) [2023-10-13 22:36:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 83427328. Throughput: 0: 1703.6, 1: 1691.5. Samples: 20867918. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-13 22:36:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:36:36,469][60935] Updated weights for policy 0, policy_version 40480 (0.0009) [2023-10-13 22:36:38,050][60934] Updated weights for policy 1, policy_version 40732 (0.0010) [2023-10-13 22:36:38,434][60934] Updated weights for policy 1, policy_version 40742 (0.0010) [2023-10-13 22:36:38,799][60934] Updated weights for policy 1, policy_version 40752 (0.0008) [2023-10-13 22:36:40,390][60935] Updated weights for policy 0, policy_version 40490 (0.0007) [2023-10-13 22:36:40,767][60935] Updated weights for policy 0, policy_version 40500 (0.0008) [2023-10-13 22:36:41,139][60935] Updated weights for policy 0, policy_version 40510 (0.0009) [2023-10-13 22:36:41,248][59943] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 83525632. Throughput: 0: 1679.0, 1: 1707.6. Samples: 20887970. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-13 22:36:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:36:42,780][60934] Updated weights for policy 1, policy_version 40762 (0.0008) [2023-10-13 22:36:43,146][60934] Updated weights for policy 1, policy_version 40772 (0.0009) [2023-10-13 22:36:43,516][60934] Updated weights for policy 1, policy_version 40782 (0.0008) [2023-10-13 22:36:43,882][60934] Updated weights for policy 1, policy_version 40792 (0.0007) [2023-10-13 22:36:45,110][60935] Updated weights for policy 0, policy_version 40520 (0.0009) [2023-10-13 22:36:45,479][60935] Updated weights for policy 0, policy_version 40530 (0.0008) [2023-10-13 22:36:45,857][60935] Updated weights for policy 0, policy_version 40540 (0.0009) [2023-10-13 22:36:46,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 83591168. Throughput: 0: 1698.8, 1: 1698.4. Samples: 20898348. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-13 22:36:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:36:47,992][60934] Updated weights for policy 1, policy_version 40802 (0.0008) [2023-10-13 22:36:48,356][60934] Updated weights for policy 1, policy_version 40812 (0.0008) [2023-10-13 22:36:48,725][60934] Updated weights for policy 1, policy_version 40822 (0.0009) [2023-10-13 22:36:50,025][60935] Updated weights for policy 0, policy_version 40550 (0.0008) [2023-10-13 22:36:50,397][60935] Updated weights for policy 0, policy_version 40560 (0.0007) [2023-10-13 22:36:50,763][60935] Updated weights for policy 0, policy_version 40570 (0.0007) [2023-10-13 22:36:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 83656704. Throughput: 0: 1693.4, 1: 1696.8. Samples: 20918780. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-13 22:36:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:36:52,684][60934] Updated weights for policy 1, policy_version 40832 (0.0010) [2023-10-13 22:36:53,052][60934] Updated weights for policy 1, policy_version 40842 (0.0008) [2023-10-13 22:36:53,411][60934] Updated weights for policy 1, policy_version 40852 (0.0007) [2023-10-13 22:36:54,830][60935] Updated weights for policy 0, policy_version 40580 (0.0009) [2023-10-13 22:36:55,205][60935] Updated weights for policy 0, policy_version 40590 (0.0009) [2023-10-13 22:36:55,577][60935] Updated weights for policy 0, policy_version 40600 (0.0009) [2023-10-13 22:36:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 83722240. Throughput: 0: 1667.2, 1: 1722.2. Samples: 20938676. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-13 22:36:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:36:56,258][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000040856_42139648.pth... [2023-10-13 22:36:56,258][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000040608_41582592.pth... [2023-10-13 22:36:56,287][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000039472_40468480.pth [2023-10-13 22:36:56,290][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000039040_39976960.pth [2023-10-13 22:36:57,237][60934] Updated weights for policy 1, policy_version 40862 (0.0008) [2023-10-13 22:36:57,598][60934] Updated weights for policy 1, policy_version 40872 (0.0008) [2023-10-13 22:36:57,972][60934] Updated weights for policy 1, policy_version 40882 (0.0011) [2023-10-13 22:36:59,596][60935] Updated weights for policy 0, policy_version 40610 (0.0008) [2023-10-13 22:36:59,962][60935] Updated weights for policy 0, policy_version 40620 (0.0009) [2023-10-13 22:37:00,338][60935] Updated weights for policy 0, policy_version 40630 (0.0008) [2023-10-13 22:37:00,709][60935] Updated weights for policy 0, policy_version 40640 (0.0009) [2023-10-13 22:37:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 83787776. Throughput: 0: 1691.2, 1: 1686.5. Samples: 20949018. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-13 22:37:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:37:02,042][60934] Updated weights for policy 1, policy_version 40892 (0.0007) [2023-10-13 22:37:02,416][60934] Updated weights for policy 1, policy_version 40902 (0.0009) [2023-10-13 22:37:02,787][60934] Updated weights for policy 1, policy_version 40912 (0.0008) [2023-10-13 22:37:04,980][60935] Updated weights for policy 0, policy_version 40650 (0.0010) [2023-10-13 22:37:05,348][60935] Updated weights for policy 0, policy_version 40660 (0.0008) [2023-10-13 22:37:05,715][60935] Updated weights for policy 0, policy_version 40670 (0.0011) [2023-10-13 22:37:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 83853312. Throughput: 0: 1682.7, 1: 1704.9. Samples: 20969516. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-13 22:37:06,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:37:06,724][60934] Updated weights for policy 1, policy_version 40922 (0.0009) [2023-10-13 22:37:07,087][60934] Updated weights for policy 1, policy_version 40932 (0.0007) [2023-10-13 22:37:07,451][60934] Updated weights for policy 1, policy_version 40942 (0.0007) [2023-10-13 22:37:07,820][60934] Updated weights for policy 1, policy_version 40952 (0.0007) [2023-10-13 22:37:09,791][60935] Updated weights for policy 0, policy_version 40680 (0.0008) [2023-10-13 22:37:10,162][60935] Updated weights for policy 0, policy_version 40690 (0.0007) [2023-10-13 22:37:10,534][60935] Updated weights for policy 0, policy_version 40700 (0.0008) [2023-10-13 22:37:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 83918848. Throughput: 0: 1661.6, 1: 1721.4. Samples: 20989448. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-13 22:37:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:37:11,797][60934] Updated weights for policy 1, policy_version 40962 (0.0008) [2023-10-13 22:37:12,160][60934] Updated weights for policy 1, policy_version 40972 (0.0009) [2023-10-13 22:37:12,526][60934] Updated weights for policy 1, policy_version 40982 (0.0010) [2023-10-13 22:37:14,569][60935] Updated weights for policy 0, policy_version 40710 (0.0008) [2023-10-13 22:37:14,941][60935] Updated weights for policy 0, policy_version 40720 (0.0007) [2023-10-13 22:37:15,323][60935] Updated weights for policy 0, policy_version 40730 (0.0010) [2023-10-13 22:37:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 83984384. Throughput: 0: 1685.6, 1: 1694.8. Samples: 20999840. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-13 22:37:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:37:16,551][60934] Updated weights for policy 1, policy_version 40992 (0.0009) [2023-10-13 22:37:16,924][60934] Updated weights for policy 1, policy_version 41002 (0.0007) [2023-10-13 22:37:17,288][60934] Updated weights for policy 1, policy_version 41012 (0.0008) [2023-10-13 22:37:19,456][60935] Updated weights for policy 0, policy_version 40740 (0.0007) [2023-10-13 22:37:19,824][60935] Updated weights for policy 0, policy_version 40750 (0.0008) [2023-10-13 22:37:20,197][60935] Updated weights for policy 0, policy_version 40760 (0.0008) [2023-10-13 22:37:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 84049920. Throughput: 0: 1670.2, 1: 1712.2. Samples: 21020124. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-13 22:37:21,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:37:21,442][60934] Updated weights for policy 1, policy_version 41022 (0.0010) [2023-10-13 22:37:21,803][60934] Updated weights for policy 1, policy_version 41032 (0.0008) [2023-10-13 22:37:22,175][60934] Updated weights for policy 1, policy_version 41042 (0.0010) [2023-10-13 22:37:24,210][60935] Updated weights for policy 0, policy_version 40770 (0.0010) [2023-10-13 22:37:24,584][60935] Updated weights for policy 0, policy_version 40780 (0.0009) [2023-10-13 22:37:24,951][60935] Updated weights for policy 0, policy_version 40790 (0.0009) [2023-10-13 22:37:25,318][60935] Updated weights for policy 0, policy_version 40800 (0.0007) [2023-10-13 22:37:26,200][60934] Updated weights for policy 1, policy_version 41052 (0.0009) [2023-10-13 22:37:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 84115456. Throughput: 0: 1663.9, 1: 1715.5. Samples: 21040046. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-13 22:37:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:37:26,596][60934] Updated weights for policy 1, policy_version 41062 (0.0010) [2023-10-13 22:37:26,958][60934] Updated weights for policy 1, policy_version 41072 (0.0008) [2023-10-13 22:37:29,295][60935] Updated weights for policy 0, policy_version 40810 (0.0007) [2023-10-13 22:37:29,670][60935] Updated weights for policy 0, policy_version 40820 (0.0007) [2023-10-13 22:37:30,049][60935] Updated weights for policy 0, policy_version 40830 (0.0008) [2023-10-13 22:37:30,824][60934] Updated weights for policy 1, policy_version 41082 (0.0007) [2023-10-13 22:37:31,191][60934] Updated weights for policy 1, policy_version 41092 (0.0008) [2023-10-13 22:37:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 84180992. Throughput: 0: 1676.7, 1: 1703.0. Samples: 21050434. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-13 22:37:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:37:31,550][60934] Updated weights for policy 1, policy_version 41102 (0.0007) [2023-10-13 22:37:31,919][60934] Updated weights for policy 1, policy_version 41112 (0.0009) [2023-10-13 22:37:34,033][60935] Updated weights for policy 0, policy_version 40840 (0.0008) [2023-10-13 22:37:34,414][60935] Updated weights for policy 0, policy_version 40850 (0.0009) [2023-10-13 22:37:34,778][60935] Updated weights for policy 0, policy_version 40860 (0.0009) [2023-10-13 22:37:35,936][60934] Updated weights for policy 1, policy_version 41122 (0.0009) [2023-10-13 22:37:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 84246528. Throughput: 0: 1656.0, 1: 1712.2. Samples: 21070350. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-13 22:37:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:37:36,299][60934] Updated weights for policy 1, policy_version 41132 (0.0008) [2023-10-13 22:37:36,681][60934] Updated weights for policy 1, policy_version 41142 (0.0008) [2023-10-13 22:37:38,954][60935] Updated weights for policy 0, policy_version 40870 (0.0007) [2023-10-13 22:37:39,323][60935] Updated weights for policy 0, policy_version 40880 (0.0008) [2023-10-13 22:37:39,686][60935] Updated weights for policy 0, policy_version 40890 (0.0009) [2023-10-13 22:37:40,477][60934] Updated weights for policy 1, policy_version 41152 (0.0009) [2023-10-13 22:37:40,844][60934] Updated weights for policy 1, policy_version 41162 (0.0007) [2023-10-13 22:37:41,216][60934] Updated weights for policy 1, policy_version 41172 (0.0007) [2023-10-13 22:37:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 84312064. Throughput: 0: 1675.0, 1: 1707.3. Samples: 21090880. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-13 22:37:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:37:43,620][60935] Updated weights for policy 0, policy_version 40900 (0.0008) [2023-10-13 22:37:43,983][60935] Updated weights for policy 0, policy_version 40910 (0.0009) [2023-10-13 22:37:44,349][60935] Updated weights for policy 0, policy_version 40920 (0.0008) [2023-10-13 22:37:45,290][60934] Updated weights for policy 1, policy_version 41182 (0.0008) [2023-10-13 22:37:45,655][60934] Updated weights for policy 1, policy_version 41192 (0.0008) [2023-10-13 22:37:46,030][60934] Updated weights for policy 1, policy_version 41202 (0.0008) [2023-10-13 22:37:46,248][59943] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 84410368. Throughput: 0: 1666.1, 1: 1715.1. Samples: 21101174. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-13 22:37:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:37:48,619][60935] Updated weights for policy 0, policy_version 40930 (0.0009) [2023-10-13 22:37:48,994][60935] Updated weights for policy 0, policy_version 40940 (0.0009) [2023-10-13 22:37:49,364][60935] Updated weights for policy 0, policy_version 40950 (0.0008) [2023-10-13 22:37:49,745][60935] Updated weights for policy 0, policy_version 40960 (0.0007) [2023-10-13 22:37:50,058][60934] Updated weights for policy 1, policy_version 41212 (0.0009) [2023-10-13 22:37:50,422][60934] Updated weights for policy 1, policy_version 41222 (0.0007) [2023-10-13 22:37:50,776][60934] Updated weights for policy 1, policy_version 41232 (0.0008) [2023-10-13 22:37:51,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 84475904. Throughput: 0: 1649.7, 1: 1718.9. Samples: 21121106. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 22:37:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:37:53,887][60935] Updated weights for policy 0, policy_version 40970 (0.0009) [2023-10-13 22:37:54,269][60935] Updated weights for policy 0, policy_version 40980 (0.0009) [2023-10-13 22:37:54,638][60935] Updated weights for policy 0, policy_version 40990 (0.0009) [2023-10-13 22:37:54,822][60934] Updated weights for policy 1, policy_version 41242 (0.0008) [2023-10-13 22:37:55,197][60934] Updated weights for policy 1, policy_version 41252 (0.0008) [2023-10-13 22:37:55,563][60934] Updated weights for policy 1, policy_version 41262 (0.0008) [2023-10-13 22:37:55,933][60934] Updated weights for policy 1, policy_version 41272 (0.0008) [2023-10-13 22:37:56,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 84541440. Throughput: 0: 1678.5, 1: 1693.1. Samples: 21141168. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 22:37:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:37:58,619][60935] Updated weights for policy 0, policy_version 41000 (0.0008) [2023-10-13 22:37:58,987][60935] Updated weights for policy 0, policy_version 41010 (0.0009) [2023-10-13 22:37:59,362][60935] Updated weights for policy 0, policy_version 41020 (0.0010) [2023-10-13 22:37:59,945][60934] Updated weights for policy 1, policy_version 41282 (0.0009) [2023-10-13 22:38:00,321][60934] Updated weights for policy 1, policy_version 41292 (0.0008) [2023-10-13 22:38:00,693][60934] Updated weights for policy 1, policy_version 41302 (0.0007) [2023-10-13 22:38:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 84606976. Throughput: 0: 1663.0, 1: 1713.6. Samples: 21151786. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 22:38:01,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:38:03,430][60935] Updated weights for policy 0, policy_version 41030 (0.0011) [2023-10-13 22:38:03,792][60935] Updated weights for policy 0, policy_version 41040 (0.0010) [2023-10-13 22:38:04,167][60935] Updated weights for policy 0, policy_version 41050 (0.0010) [2023-10-13 22:38:04,699][60934] Updated weights for policy 1, policy_version 41312 (0.0009) [2023-10-13 22:38:05,064][60934] Updated weights for policy 1, policy_version 41322 (0.0008) [2023-10-13 22:38:05,439][60934] Updated weights for policy 1, policy_version 41332 (0.0008) [2023-10-13 22:38:06,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 84672512. Throughput: 0: 1657.4, 1: 1711.3. Samples: 21171718. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 22:38:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:38:08,429][60935] Updated weights for policy 0, policy_version 41060 (0.0008) [2023-10-13 22:38:08,799][60935] Updated weights for policy 0, policy_version 41070 (0.0007) [2023-10-13 22:38:09,156][60935] Updated weights for policy 0, policy_version 41080 (0.0007) [2023-10-13 22:38:09,455][60934] Updated weights for policy 1, policy_version 41342 (0.0011) [2023-10-13 22:38:09,817][60934] Updated weights for policy 1, policy_version 41352 (0.0009) [2023-10-13 22:38:10,177][60934] Updated weights for policy 1, policy_version 41362 (0.0008) [2023-10-13 22:38:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 84738048. Throughput: 0: 1679.3, 1: 1684.3. Samples: 21191410. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 22:38:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:38:13,150][60935] Updated weights for policy 0, policy_version 41090 (0.0009) [2023-10-13 22:38:13,519][60935] Updated weights for policy 0, policy_version 41100 (0.0009) [2023-10-13 22:38:13,893][60935] Updated weights for policy 0, policy_version 41110 (0.0008) [2023-10-13 22:38:14,263][60935] Updated weights for policy 0, policy_version 41120 (0.0008) [2023-10-13 22:38:14,385][60934] Updated weights for policy 1, policy_version 41372 (0.0010) [2023-10-13 22:38:14,791][60934] Updated weights for policy 1, policy_version 41382 (0.0010) [2023-10-13 22:38:15,157][60934] Updated weights for policy 1, policy_version 41392 (0.0008) [2023-10-13 22:38:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 84803584. Throughput: 0: 1661.0, 1: 1716.7. Samples: 21202430. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 22:38:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:38:18,380][60935] Updated weights for policy 0, policy_version 41130 (0.0008) [2023-10-13 22:38:18,747][60935] Updated weights for policy 0, policy_version 41140 (0.0010) [2023-10-13 22:38:19,050][60934] Updated weights for policy 1, policy_version 41402 (0.0007) [2023-10-13 22:38:19,118][60935] Updated weights for policy 0, policy_version 41150 (0.0008) [2023-10-13 22:38:19,415][60934] Updated weights for policy 1, policy_version 41412 (0.0008) [2023-10-13 22:38:19,780][60934] Updated weights for policy 1, policy_version 41422 (0.0008) [2023-10-13 22:38:20,156][60934] Updated weights for policy 1, policy_version 41432 (0.0008) [2023-10-13 22:38:21,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 84869120. Throughput: 0: 1672.7, 1: 1697.5. Samples: 21222008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:38:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:38:23,086][60935] Updated weights for policy 0, policy_version 41160 (0.0010) [2023-10-13 22:38:23,447][60935] Updated weights for policy 0, policy_version 41170 (0.0010) [2023-10-13 22:38:23,823][60935] Updated weights for policy 0, policy_version 41180 (0.0012) [2023-10-13 22:38:24,199][60934] Updated weights for policy 1, policy_version 41442 (0.0010) [2023-10-13 22:38:24,568][60934] Updated weights for policy 1, policy_version 41452 (0.0007) [2023-10-13 22:38:24,937][60934] Updated weights for policy 1, policy_version 41462 (0.0007) [2023-10-13 22:38:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 84934656. Throughput: 0: 1682.0, 1: 1681.3. Samples: 21242230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:38:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:38:27,845][60935] Updated weights for policy 0, policy_version 41190 (0.0008) [2023-10-13 22:38:28,212][60935] Updated weights for policy 0, policy_version 41200 (0.0009) [2023-10-13 22:38:28,581][60935] Updated weights for policy 0, policy_version 41210 (0.0008) [2023-10-13 22:38:28,877][60934] Updated weights for policy 1, policy_version 41472 (0.0008) [2023-10-13 22:38:29,241][60934] Updated weights for policy 1, policy_version 41482 (0.0008) [2023-10-13 22:38:29,603][60934] Updated weights for policy 1, policy_version 41492 (0.0008) [2023-10-13 22:38:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 85000192. Throughput: 0: 1662.9, 1: 1706.2. Samples: 21252782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:38:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:38:32,649][60935] Updated weights for policy 0, policy_version 41220 (0.0007) [2023-10-13 22:38:33,014][60935] Updated weights for policy 0, policy_version 41230 (0.0010) [2023-10-13 22:38:33,387][60935] Updated weights for policy 0, policy_version 41240 (0.0007) [2023-10-13 22:38:33,793][60934] Updated weights for policy 1, policy_version 41502 (0.0008) [2023-10-13 22:38:34,164][60934] Updated weights for policy 1, policy_version 41512 (0.0008) [2023-10-13 22:38:34,531][60934] Updated weights for policy 1, policy_version 41522 (0.0008) [2023-10-13 22:38:36,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 85065728. Throughput: 0: 1690.9, 1: 1679.3. Samples: 21272766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:38:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:38:37,498][60935] Updated weights for policy 0, policy_version 41250 (0.0007) [2023-10-13 22:38:37,857][60935] Updated weights for policy 0, policy_version 41260 (0.0008) [2023-10-13 22:38:38,225][60935] Updated weights for policy 0, policy_version 41270 (0.0009) [2023-10-13 22:38:38,487][60934] Updated weights for policy 1, policy_version 41532 (0.0008) [2023-10-13 22:38:38,596][60935] Updated weights for policy 0, policy_version 41280 (0.0010) [2023-10-13 22:38:38,851][60934] Updated weights for policy 1, policy_version 41542 (0.0008) [2023-10-13 22:38:39,218][60934] Updated weights for policy 1, policy_version 41552 (0.0007) [2023-10-13 22:38:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 85131264. Throughput: 0: 1692.1, 1: 1689.6. Samples: 21293346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:38:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:38:42,745][60935] Updated weights for policy 0, policy_version 41290 (0.0009) [2023-10-13 22:38:43,118][60935] Updated weights for policy 0, policy_version 41300 (0.0008) [2023-10-13 22:38:43,335][60934] Updated weights for policy 1, policy_version 41562 (0.0007) [2023-10-13 22:38:43,484][60935] Updated weights for policy 0, policy_version 41310 (0.0008) [2023-10-13 22:38:43,697][60934] Updated weights for policy 1, policy_version 41572 (0.0008) [2023-10-13 22:38:44,077][60934] Updated weights for policy 1, policy_version 41582 (0.0009) [2023-10-13 22:38:44,440][60934] Updated weights for policy 1, policy_version 41592 (0.0010) [2023-10-13 22:38:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 85196800. Throughput: 0: 1670.8, 1: 1692.3. Samples: 21303128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:38:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:38:47,561][60935] Updated weights for policy 0, policy_version 41320 (0.0007) [2023-10-13 22:38:47,934][60935] Updated weights for policy 0, policy_version 41330 (0.0007) [2023-10-13 22:38:48,302][60935] Updated weights for policy 0, policy_version 41340 (0.0010) [2023-10-13 22:38:48,343][60934] Updated weights for policy 1, policy_version 41602 (0.0009) [2023-10-13 22:38:48,713][60934] Updated weights for policy 1, policy_version 41612 (0.0009) [2023-10-13 22:38:49,077][60934] Updated weights for policy 1, policy_version 41622 (0.0008) [2023-10-13 22:38:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 85262336. Throughput: 0: 1686.9, 1: 1676.6. Samples: 21323078. Policy #0 lag: (min: 11.0, avg: 12.2, max: 35.0) [2023-10-13 22:38:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:38:52,328][60935] Updated weights for policy 0, policy_version 41350 (0.0010) [2023-10-13 22:38:52,691][60935] Updated weights for policy 0, policy_version 41360 (0.0007) [2023-10-13 22:38:53,057][60934] Updated weights for policy 1, policy_version 41632 (0.0008) [2023-10-13 22:38:53,071][60935] Updated weights for policy 0, policy_version 41370 (0.0008) [2023-10-13 22:38:53,428][60934] Updated weights for policy 1, policy_version 41642 (0.0009) [2023-10-13 22:38:53,793][60934] Updated weights for policy 1, policy_version 41652 (0.0010) [2023-10-13 22:38:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13551.5). Total num frames: 85327872. Throughput: 0: 1684.4, 1: 1704.4. Samples: 21343908. Policy #0 lag: (min: 11.0, avg: 12.2, max: 35.0) [2023-10-13 22:38:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:38:56,259][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000041376_42369024.pth... [2023-10-13 22:38:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000041656_42958848.pth... [2023-10-13 22:38:56,297][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000040160_41320448.pth [2023-10-13 22:38:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000039808_40763392.pth [2023-10-13 22:38:57,427][60935] Updated weights for policy 0, policy_version 41380 (0.0009) [2023-10-13 22:38:57,798][60935] Updated weights for policy 0, policy_version 41390 (0.0007) [2023-10-13 22:38:57,929][60934] Updated weights for policy 1, policy_version 41662 (0.0009) [2023-10-13 22:38:58,169][60935] Updated weights for policy 0, policy_version 41400 (0.0007) [2023-10-13 22:38:58,297][60934] Updated weights for policy 1, policy_version 41672 (0.0007) [2023-10-13 22:38:58,667][60934] Updated weights for policy 1, policy_version 41682 (0.0007) [2023-10-13 22:39:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 85393408. Throughput: 0: 1669.0, 1: 1684.6. Samples: 21353340. Policy #0 lag: (min: 11.0, avg: 12.2, max: 35.0) [2023-10-13 22:39:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:39:02,323][60935] Updated weights for policy 0, policy_version 41410 (0.0008) [2023-10-13 22:39:02,529][60934] Updated weights for policy 1, policy_version 41692 (0.0007) [2023-10-13 22:39:02,681][60935] Updated weights for policy 0, policy_version 41420 (0.0009) [2023-10-13 22:39:02,900][60934] Updated weights for policy 1, policy_version 41702 (0.0008) [2023-10-13 22:39:03,058][60935] Updated weights for policy 0, policy_version 41430 (0.0009) [2023-10-13 22:39:03,269][60934] Updated weights for policy 1, policy_version 41712 (0.0008) [2023-10-13 22:39:03,419][60935] Updated weights for policy 0, policy_version 41440 (0.0008) [2023-10-13 22:39:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 85458944. Throughput: 0: 1677.9, 1: 1692.1. Samples: 21373658. Policy #0 lag: (min: 11.0, avg: 12.2, max: 35.0) [2023-10-13 22:39:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:39:07,249][60934] Updated weights for policy 1, policy_version 41722 (0.0009) [2023-10-13 22:39:07,656][60935] Updated weights for policy 0, policy_version 41450 (0.0007) [2023-10-13 22:39:07,667][60934] Updated weights for policy 1, policy_version 41732 (0.0007) [2023-10-13 22:39:08,022][60935] Updated weights for policy 0, policy_version 41460 (0.0009) [2023-10-13 22:39:08,025][60934] Updated weights for policy 1, policy_version 41742 (0.0008) [2023-10-13 22:39:08,389][60934] Updated weights for policy 1, policy_version 41752 (0.0008) [2023-10-13 22:39:08,393][60935] Updated weights for policy 0, policy_version 41470 (0.0008) [2023-10-13 22:39:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 85524480. Throughput: 0: 1671.7, 1: 1717.4. Samples: 21394742. Policy #0 lag: (min: 11.0, avg: 12.2, max: 35.0) [2023-10-13 22:39:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:39:12,369][60935] Updated weights for policy 0, policy_version 41480 (0.0007) [2023-10-13 22:39:12,423][60934] Updated weights for policy 1, policy_version 41762 (0.0008) [2023-10-13 22:39:12,739][60935] Updated weights for policy 0, policy_version 41490 (0.0007) [2023-10-13 22:39:12,800][60934] Updated weights for policy 1, policy_version 41772 (0.0007) [2023-10-13 22:39:13,106][60935] Updated weights for policy 0, policy_version 41500 (0.0007) [2023-10-13 22:39:13,160][60934] Updated weights for policy 1, policy_version 41782 (0.0008) [2023-10-13 22:39:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 85590016. Throughput: 0: 1669.1, 1: 1683.2. Samples: 21403634. Policy #0 lag: (min: 11.0, avg: 12.2, max: 35.0) [2023-10-13 22:39:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:39:17,160][60934] Updated weights for policy 1, policy_version 41792 (0.0008) [2023-10-13 22:39:17,256][60935] Updated weights for policy 0, policy_version 41510 (0.0009) [2023-10-13 22:39:17,531][60934] Updated weights for policy 1, policy_version 41802 (0.0007) [2023-10-13 22:39:17,624][60935] Updated weights for policy 0, policy_version 41520 (0.0008) [2023-10-13 22:39:17,892][60934] Updated weights for policy 1, policy_version 41812 (0.0010) [2023-10-13 22:39:17,991][60935] Updated weights for policy 0, policy_version 41530 (0.0008) [2023-10-13 22:39:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 85655552. Throughput: 0: 1665.8, 1: 1703.7. Samples: 21424394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:39:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:39:21,898][60934] Updated weights for policy 1, policy_version 41822 (0.0008) [2023-10-13 22:39:22,105][60935] Updated weights for policy 0, policy_version 41540 (0.0009) [2023-10-13 22:39:22,264][60934] Updated weights for policy 1, policy_version 41832 (0.0009) [2023-10-13 22:39:22,475][60935] Updated weights for policy 0, policy_version 41550 (0.0008) [2023-10-13 22:39:22,628][60934] Updated weights for policy 1, policy_version 41842 (0.0010) [2023-10-13 22:39:22,833][60935] Updated weights for policy 0, policy_version 41560 (0.0009) [2023-10-13 22:39:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 85721088. Throughput: 0: 1666.0, 1: 1712.7. Samples: 21445388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:39:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:39:26,535][60934] Updated weights for policy 1, policy_version 41852 (0.0007) [2023-10-13 22:39:26,899][60934] Updated weights for policy 1, policy_version 41862 (0.0008) [2023-10-13 22:39:26,920][60935] Updated weights for policy 0, policy_version 41570 (0.0008) [2023-10-13 22:39:27,269][60934] Updated weights for policy 1, policy_version 41872 (0.0007) [2023-10-13 22:39:27,285][60935] Updated weights for policy 0, policy_version 41580 (0.0008) [2023-10-13 22:39:27,656][60935] Updated weights for policy 0, policy_version 41590 (0.0009) [2023-10-13 22:39:28,021][60935] Updated weights for policy 0, policy_version 41600 (0.0009) [2023-10-13 22:39:31,131][60934] Updated weights for policy 1, policy_version 41882 (0.0008) [2023-10-13 22:39:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 85786624. Throughput: 0: 1669.7, 1: 1689.2. Samples: 21454282. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:39:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:39:31,495][60934] Updated weights for policy 1, policy_version 41892 (0.0009) [2023-10-13 22:39:31,860][60934] Updated weights for policy 1, policy_version 41902 (0.0008) [2023-10-13 22:39:32,084][60935] Updated weights for policy 0, policy_version 41610 (0.0010) [2023-10-13 22:39:32,228][60934] Updated weights for policy 1, policy_version 41912 (0.0008) [2023-10-13 22:39:32,445][60935] Updated weights for policy 0, policy_version 41620 (0.0008) [2023-10-13 22:39:32,826][60935] Updated weights for policy 0, policy_version 41630 (0.0008) [2023-10-13 22:39:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 85852160. Throughput: 0: 1670.1, 1: 1711.2. Samples: 21475236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:39:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:39:36,360][60934] Updated weights for policy 1, policy_version 41922 (0.0007) [2023-10-13 22:39:36,721][60934] Updated weights for policy 1, policy_version 41932 (0.0008) [2023-10-13 22:39:36,938][60935] Updated weights for policy 0, policy_version 41640 (0.0008) [2023-10-13 22:39:37,086][60934] Updated weights for policy 1, policy_version 41942 (0.0008) [2023-10-13 22:39:37,309][60935] Updated weights for policy 0, policy_version 41650 (0.0009) [2023-10-13 22:39:37,675][60935] Updated weights for policy 0, policy_version 41660 (0.0009) [2023-10-13 22:39:41,222][60934] Updated weights for policy 1, policy_version 41952 (0.0008) [2023-10-13 22:39:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 85917696. Throughput: 0: 1670.8, 1: 1705.7. Samples: 21495850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:39:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:39:41,576][60934] Updated weights for policy 1, policy_version 41962 (0.0008) [2023-10-13 22:39:41,668][60935] Updated weights for policy 0, policy_version 41670 (0.0008) [2023-10-13 22:39:41,945][60934] Updated weights for policy 1, policy_version 41972 (0.0008) [2023-10-13 22:39:42,028][60935] Updated weights for policy 0, policy_version 41680 (0.0011) [2023-10-13 22:39:42,400][60935] Updated weights for policy 0, policy_version 41690 (0.0011) [2023-10-13 22:39:45,962][60934] Updated weights for policy 1, policy_version 41982 (0.0009) [2023-10-13 22:39:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 85983232. Throughput: 0: 1674.7, 1: 1700.3. Samples: 21505212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:39:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:39:46,340][60934] Updated weights for policy 1, policy_version 41992 (0.0008) [2023-10-13 22:39:46,348][60935] Updated weights for policy 0, policy_version 41700 (0.0008) [2023-10-13 22:39:46,698][60934] Updated weights for policy 1, policy_version 42002 (0.0007) [2023-10-13 22:39:46,706][60935] Updated weights for policy 0, policy_version 41710 (0.0008) [2023-10-13 22:39:47,075][60935] Updated weights for policy 0, policy_version 41720 (0.0010) [2023-10-13 22:39:50,806][60934] Updated weights for policy 1, policy_version 42012 (0.0007) [2023-10-13 22:39:51,086][60935] Updated weights for policy 0, policy_version 41730 (0.0011) [2023-10-13 22:39:51,161][60934] Updated weights for policy 1, policy_version 42022 (0.0007) [2023-10-13 22:39:51,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 86048768. Throughput: 0: 1678.4, 1: 1709.6. Samples: 21526118. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-13 22:39:51,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:39:51,463][60935] Updated weights for policy 0, policy_version 41740 (0.0010) [2023-10-13 22:39:51,528][60934] Updated weights for policy 1, policy_version 42032 (0.0007) [2023-10-13 22:39:51,823][60935] Updated weights for policy 0, policy_version 41750 (0.0009) [2023-10-13 22:39:52,192][60935] Updated weights for policy 0, policy_version 41760 (0.0010) [2023-10-13 22:39:55,762][60934] Updated weights for policy 1, policy_version 42042 (0.0008) [2023-10-13 22:39:56,170][60934] Updated weights for policy 1, policy_version 42052 (0.0007) [2023-10-13 22:39:56,189][60935] Updated weights for policy 0, policy_version 41770 (0.0007) [2023-10-13 22:39:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 86114304. Throughput: 0: 1686.4, 1: 1704.9. Samples: 21547352. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-13 22:39:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:39:56,534][60934] Updated weights for policy 1, policy_version 42062 (0.0008) [2023-10-13 22:39:56,556][60935] Updated weights for policy 0, policy_version 41780 (0.0008) [2023-10-13 22:39:56,900][60934] Updated weights for policy 1, policy_version 42072 (0.0009) [2023-10-13 22:39:56,927][60935] Updated weights for policy 0, policy_version 41790 (0.0008) [2023-10-13 22:40:00,925][60935] Updated weights for policy 0, policy_version 41800 (0.0008) [2023-10-13 22:40:01,024][60934] Updated weights for policy 1, policy_version 42082 (0.0010) [2023-10-13 22:40:01,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 86179840. Throughput: 0: 1689.1, 1: 1700.6. Samples: 21556168. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-13 22:40:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:40:01,300][60935] Updated weights for policy 0, policy_version 41810 (0.0009) [2023-10-13 22:40:01,386][60934] Updated weights for policy 1, policy_version 42092 (0.0009) [2023-10-13 22:40:01,675][60935] Updated weights for policy 0, policy_version 41820 (0.0008) [2023-10-13 22:40:01,758][60934] Updated weights for policy 1, policy_version 42102 (0.0009) [2023-10-13 22:40:05,740][60935] Updated weights for policy 0, policy_version 41830 (0.0008) [2023-10-13 22:40:05,802][60934] Updated weights for policy 1, policy_version 42112 (0.0007) [2023-10-13 22:40:06,106][60935] Updated weights for policy 0, policy_version 41840 (0.0007) [2023-10-13 22:40:06,167][60934] Updated weights for policy 1, policy_version 42122 (0.0009) [2023-10-13 22:40:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 86245376. Throughput: 0: 1692.5, 1: 1699.9. Samples: 21577050. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-13 22:40:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:40:06,467][60935] Updated weights for policy 0, policy_version 41850 (0.0008) [2023-10-13 22:40:06,530][60934] Updated weights for policy 1, policy_version 42132 (0.0009) [2023-10-13 22:40:10,529][60934] Updated weights for policy 1, policy_version 42142 (0.0009) [2023-10-13 22:40:10,696][60935] Updated weights for policy 0, policy_version 41860 (0.0009) [2023-10-13 22:40:10,890][60934] Updated weights for policy 1, policy_version 42152 (0.0007) [2023-10-13 22:40:11,064][60935] Updated weights for policy 0, policy_version 41870 (0.0009) [2023-10-13 22:40:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 86310912. Throughput: 0: 1680.0, 1: 1688.2. Samples: 21596958. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-13 22:40:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:40:11,255][60934] Updated weights for policy 1, policy_version 42162 (0.0007) [2023-10-13 22:40:11,438][60935] Updated weights for policy 0, policy_version 41880 (0.0009) [2023-10-13 22:40:15,199][60934] Updated weights for policy 1, policy_version 42172 (0.0008) [2023-10-13 22:40:15,398][60935] Updated weights for policy 0, policy_version 41890 (0.0010) [2023-10-13 22:40:15,563][60934] Updated weights for policy 1, policy_version 42182 (0.0008) [2023-10-13 22:40:15,764][60935] Updated weights for policy 0, policy_version 41900 (0.0009) [2023-10-13 22:40:15,931][60934] Updated weights for policy 1, policy_version 42192 (0.0008) [2023-10-13 22:40:16,133][60935] Updated weights for policy 0, policy_version 41910 (0.0008) [2023-10-13 22:40:16,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 86409216. Throughput: 0: 1687.9, 1: 1698.0. Samples: 21606644. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-13 22:40:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:40:16,499][60935] Updated weights for policy 0, policy_version 41920 (0.0009) [2023-10-13 22:40:19,849][60934] Updated weights for policy 1, policy_version 42202 (0.0008) [2023-10-13 22:40:20,209][60934] Updated weights for policy 1, policy_version 42212 (0.0008) [2023-10-13 22:40:20,565][60934] Updated weights for policy 1, policy_version 42222 (0.0008) [2023-10-13 22:40:20,705][60935] Updated weights for policy 0, policy_version 41930 (0.0007) [2023-10-13 22:40:20,936][60934] Updated weights for policy 1, policy_version 42232 (0.0008) [2023-10-13 22:40:21,068][60935] Updated weights for policy 0, policy_version 41940 (0.0008) [2023-10-13 22:40:21,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 86474752. Throughput: 0: 1684.3, 1: 1700.4. Samples: 21627548. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 22:40:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:40:21,440][60935] Updated weights for policy 0, policy_version 41950 (0.0009) [2023-10-13 22:40:24,972][60934] Updated weights for policy 1, policy_version 42242 (0.0007) [2023-10-13 22:40:25,338][60934] Updated weights for policy 1, policy_version 42252 (0.0010) [2023-10-13 22:40:25,503][60935] Updated weights for policy 0, policy_version 41960 (0.0009) [2023-10-13 22:40:25,703][60934] Updated weights for policy 1, policy_version 42262 (0.0007) [2023-10-13 22:40:25,869][60935] Updated weights for policy 0, policy_version 41970 (0.0009) [2023-10-13 22:40:26,239][60935] Updated weights for policy 0, policy_version 41980 (0.0007) [2023-10-13 22:40:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 86540288. Throughput: 0: 1667.2, 1: 1680.4. Samples: 21646496. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 22:40:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 22:40:29,710][60934] Updated weights for policy 1, policy_version 42272 (0.0008) [2023-10-13 22:40:30,077][60934] Updated weights for policy 1, policy_version 42282 (0.0011) [2023-10-13 22:40:30,222][60935] Updated weights for policy 0, policy_version 41990 (0.0008) [2023-10-13 22:40:30,440][60934] Updated weights for policy 1, policy_version 42292 (0.0010) [2023-10-13 22:40:30,581][60935] Updated weights for policy 0, policy_version 42000 (0.0010) [2023-10-13 22:40:30,963][60935] Updated weights for policy 0, policy_version 42010 (0.0007) [2023-10-13 22:40:31,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 86638592. Throughput: 0: 1681.3, 1: 1699.5. Samples: 21657346. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 22:40:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 22:40:34,575][60934] Updated weights for policy 1, policy_version 42302 (0.0009) [2023-10-13 22:40:34,939][60934] Updated weights for policy 1, policy_version 42312 (0.0009) [2023-10-13 22:40:35,021][60935] Updated weights for policy 0, policy_version 42020 (0.0009) [2023-10-13 22:40:35,301][60934] Updated weights for policy 1, policy_version 42322 (0.0007) [2023-10-13 22:40:35,394][60935] Updated weights for policy 0, policy_version 42030 (0.0008) [2023-10-13 22:40:35,753][60935] Updated weights for policy 0, policy_version 42040 (0.0009) [2023-10-13 22:40:36,248][59943] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 86704128. Throughput: 0: 1681.6, 1: 1691.4. Samples: 21677900. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 22:40:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 22:40:39,283][60934] Updated weights for policy 1, policy_version 42332 (0.0007) [2023-10-13 22:40:39,644][60934] Updated weights for policy 1, policy_version 42342 (0.0008) [2023-10-13 22:40:39,939][60935] Updated weights for policy 0, policy_version 42050 (0.0009) [2023-10-13 22:40:40,015][60934] Updated weights for policy 1, policy_version 42352 (0.0009) [2023-10-13 22:40:40,312][60935] Updated weights for policy 0, policy_version 42060 (0.0009) [2023-10-13 22:40:40,686][60935] Updated weights for policy 0, policy_version 42070 (0.0009) [2023-10-13 22:40:41,045][60935] Updated weights for policy 0, policy_version 42080 (0.0010) [2023-10-13 22:40:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 86769664. Throughput: 0: 1649.7, 1: 1663.8. Samples: 21696460. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 22:40:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 22:40:44,087][60934] Updated weights for policy 1, policy_version 42362 (0.0007) [2023-10-13 22:40:44,493][60934] Updated weights for policy 1, policy_version 42372 (0.0008) [2023-10-13 22:40:44,854][60934] Updated weights for policy 1, policy_version 42382 (0.0007) [2023-10-13 22:40:45,225][60934] Updated weights for policy 1, policy_version 42392 (0.0007) [2023-10-13 22:40:45,241][60935] Updated weights for policy 0, policy_version 42090 (0.0007) [2023-10-13 22:40:45,613][60935] Updated weights for policy 0, policy_version 42100 (0.0008) [2023-10-13 22:40:45,988][60935] Updated weights for policy 0, policy_version 42110 (0.0008) [2023-10-13 22:40:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 86835200. Throughput: 0: 1667.6, 1: 1702.0. Samples: 21707798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:40:46,248][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 22:40:49,286][60934] Updated weights for policy 1, policy_version 42402 (0.0007) [2023-10-13 22:40:49,657][60934] Updated weights for policy 1, policy_version 42412 (0.0007) [2023-10-13 22:40:50,023][60934] Updated weights for policy 1, policy_version 42422 (0.0008) [2023-10-13 22:40:50,064][60935] Updated weights for policy 0, policy_version 42120 (0.0007) [2023-10-13 22:40:50,435][60935] Updated weights for policy 0, policy_version 42130 (0.0008) [2023-10-13 22:40:50,807][60935] Updated weights for policy 0, policy_version 42140 (0.0010) [2023-10-13 22:40:51,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 86900736. Throughput: 0: 1660.4, 1: 1685.9. Samples: 21727636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:40:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 22:40:54,065][60934] Updated weights for policy 1, policy_version 42432 (0.0008) [2023-10-13 22:40:54,431][60934] Updated weights for policy 1, policy_version 42442 (0.0007) [2023-10-13 22:40:54,803][60934] Updated weights for policy 1, policy_version 42452 (0.0009) [2023-10-13 22:40:54,847][60935] Updated weights for policy 0, policy_version 42150 (0.0009) [2023-10-13 22:40:55,217][60935] Updated weights for policy 0, policy_version 42160 (0.0009) [2023-10-13 22:40:55,582][60935] Updated weights for policy 0, policy_version 42170 (0.0009) [2023-10-13 22:40:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 86966272. Throughput: 0: 1651.3, 1: 1681.5. Samples: 21746934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:40:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 22:40:56,257][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000042176_43188224.pth... [2023-10-13 22:40:56,258][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000042456_43778048.pth... [2023-10-13 22:40:56,294][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000040856_42139648.pth [2023-10-13 22:40:56,298][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000040608_41582592.pth [2023-10-13 22:40:58,685][60934] Updated weights for policy 1, policy_version 42462 (0.0009) [2023-10-13 22:40:59,048][60934] Updated weights for policy 1, policy_version 42472 (0.0008) [2023-10-13 22:40:59,414][60934] Updated weights for policy 1, policy_version 42482 (0.0008) [2023-10-13 22:40:59,726][60935] Updated weights for policy 0, policy_version 42180 (0.0010) [2023-10-13 22:41:00,099][60935] Updated weights for policy 0, policy_version 42190 (0.0008) [2023-10-13 22:41:00,471][60935] Updated weights for policy 0, policy_version 42200 (0.0009) [2023-10-13 22:41:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 87031808. Throughput: 0: 1669.9, 1: 1704.1. Samples: 21758474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:41:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.140')] [2023-10-13 22:41:03,365][60934] Updated weights for policy 1, policy_version 42492 (0.0008) [2023-10-13 22:41:03,732][60934] Updated weights for policy 1, policy_version 42502 (0.0008) [2023-10-13 22:41:04,096][60934] Updated weights for policy 1, policy_version 42512 (0.0009) [2023-10-13 22:41:04,702][60935] Updated weights for policy 0, policy_version 42210 (0.0008) [2023-10-13 22:41:05,107][60935] Updated weights for policy 0, policy_version 42220 (0.0007) [2023-10-13 22:41:05,481][60935] Updated weights for policy 0, policy_version 42230 (0.0009) [2023-10-13 22:41:05,838][60935] Updated weights for policy 0, policy_version 42240 (0.0011) [2023-10-13 22:41:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 87097344. Throughput: 0: 1666.2, 1: 1676.6. Samples: 21777974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:41:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:41:08,152][60934] Updated weights for policy 1, policy_version 42522 (0.0008) [2023-10-13 22:41:08,521][60934] Updated weights for policy 1, policy_version 42532 (0.0007) [2023-10-13 22:41:08,892][60934] Updated weights for policy 1, policy_version 42542 (0.0008) [2023-10-13 22:41:09,256][60934] Updated weights for policy 1, policy_version 42552 (0.0007) [2023-10-13 22:41:09,823][60935] Updated weights for policy 0, policy_version 42250 (0.0009) [2023-10-13 22:41:10,198][60935] Updated weights for policy 0, policy_version 42260 (0.0008) [2023-10-13 22:41:10,562][60935] Updated weights for policy 0, policy_version 42270 (0.0009) [2023-10-13 22:41:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 87162880. Throughput: 0: 1662.0, 1: 1707.8. Samples: 21798136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:41:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:41:13,215][60934] Updated weights for policy 1, policy_version 42562 (0.0008) [2023-10-13 22:41:13,574][60934] Updated weights for policy 1, policy_version 42572 (0.0009) [2023-10-13 22:41:13,946][60934] Updated weights for policy 1, policy_version 42582 (0.0008) [2023-10-13 22:41:14,512][60935] Updated weights for policy 0, policy_version 42280 (0.0009) [2023-10-13 22:41:14,886][60935] Updated weights for policy 0, policy_version 42290 (0.0008) [2023-10-13 22:41:15,261][60935] Updated weights for policy 0, policy_version 42300 (0.0011) [2023-10-13 22:41:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 87228416. Throughput: 0: 1673.8, 1: 1695.8. Samples: 21808980. Policy #0 lag: (min: 9.0, avg: 24.1, max: 41.0) [2023-10-13 22:41:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:41:17,883][60934] Updated weights for policy 1, policy_version 42592 (0.0010) [2023-10-13 22:41:18,247][60934] Updated weights for policy 1, policy_version 42602 (0.0010) [2023-10-13 22:41:18,610][60934] Updated weights for policy 1, policy_version 42612 (0.0009) [2023-10-13 22:41:19,389][60935] Updated weights for policy 0, policy_version 42310 (0.0007) [2023-10-13 22:41:19,758][60935] Updated weights for policy 0, policy_version 42320 (0.0007) [2023-10-13 22:41:20,128][60935] Updated weights for policy 0, policy_version 42330 (0.0008) [2023-10-13 22:41:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 87293952. Throughput: 0: 1656.7, 1: 1693.9. Samples: 21828680. Policy #0 lag: (min: 9.0, avg: 24.1, max: 41.0) [2023-10-13 22:41:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:41:22,532][60934] Updated weights for policy 1, policy_version 42622 (0.0008) [2023-10-13 22:41:22,894][60934] Updated weights for policy 1, policy_version 42632 (0.0008) [2023-10-13 22:41:23,262][60934] Updated weights for policy 1, policy_version 42642 (0.0009) [2023-10-13 22:41:23,979][60935] Updated weights for policy 0, policy_version 42340 (0.0008) [2023-10-13 22:41:24,356][60935] Updated weights for policy 0, policy_version 42350 (0.0007) [2023-10-13 22:41:24,719][60935] Updated weights for policy 0, policy_version 42360 (0.0009) [2023-10-13 22:41:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 87359488. Throughput: 0: 1673.1, 1: 1723.5. Samples: 21849308. Policy #0 lag: (min: 9.0, avg: 24.1, max: 41.0) [2023-10-13 22:41:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:41:27,170][60934] Updated weights for policy 1, policy_version 42652 (0.0008) [2023-10-13 22:41:27,526][60934] Updated weights for policy 1, policy_version 42662 (0.0008) [2023-10-13 22:41:27,894][60934] Updated weights for policy 1, policy_version 42672 (0.0010) [2023-10-13 22:41:28,810][60935] Updated weights for policy 0, policy_version 42370 (0.0009) [2023-10-13 22:41:29,173][60935] Updated weights for policy 0, policy_version 42380 (0.0009) [2023-10-13 22:41:29,548][60935] Updated weights for policy 0, policy_version 42390 (0.0007) [2023-10-13 22:41:29,927][60935] Updated weights for policy 0, policy_version 42400 (0.0010) [2023-10-13 22:41:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 87425024. Throughput: 0: 1682.1, 1: 1690.1. Samples: 21859548. Policy #0 lag: (min: 9.0, avg: 24.1, max: 41.0) [2023-10-13 22:41:31,249][59943] Avg episode reward: [(0, '-0.070'), (1, '0.000')] [2023-10-13 22:41:32,054][60934] Updated weights for policy 1, policy_version 42682 (0.0009) [2023-10-13 22:41:32,426][60934] Updated weights for policy 1, policy_version 42692 (0.0007) [2023-10-13 22:41:32,796][60934] Updated weights for policy 1, policy_version 42702 (0.0007) [2023-10-13 22:41:33,168][60934] Updated weights for policy 1, policy_version 42712 (0.0009) [2023-10-13 22:41:34,128][60935] Updated weights for policy 0, policy_version 42410 (0.0009) [2023-10-13 22:41:34,497][60935] Updated weights for policy 0, policy_version 42420 (0.0008) [2023-10-13 22:41:34,869][60935] Updated weights for policy 0, policy_version 42430 (0.0007) [2023-10-13 22:41:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 87490560. Throughput: 0: 1660.4, 1: 1711.8. Samples: 21879388. Policy #0 lag: (min: 9.0, avg: 24.1, max: 41.0) [2023-10-13 22:41:36,249][59943] Avg episode reward: [(0, '-0.070'), (1, '0.000')] [2023-10-13 22:41:37,078][60934] Updated weights for policy 1, policy_version 42722 (0.0008) [2023-10-13 22:41:37,432][60934] Updated weights for policy 1, policy_version 42732 (0.0008) [2023-10-13 22:41:37,805][60934] Updated weights for policy 1, policy_version 42742 (0.0008) [2023-10-13 22:41:38,869][60935] Updated weights for policy 0, policy_version 42440 (0.0008) [2023-10-13 22:41:39,244][60935] Updated weights for policy 0, policy_version 42450 (0.0011) [2023-10-13 22:41:39,617][60935] Updated weights for policy 0, policy_version 42460 (0.0007) [2023-10-13 22:41:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 87556096. Throughput: 0: 1685.5, 1: 1727.6. Samples: 21900524. Policy #0 lag: (min: 9.0, avg: 24.1, max: 41.0) [2023-10-13 22:41:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:41:41,912][60934] Updated weights for policy 1, policy_version 42752 (0.0008) [2023-10-13 22:41:42,271][60934] Updated weights for policy 1, policy_version 42762 (0.0009) [2023-10-13 22:41:42,646][60934] Updated weights for policy 1, policy_version 42772 (0.0009) [2023-10-13 22:41:43,626][60935] Updated weights for policy 0, policy_version 42470 (0.0007) [2023-10-13 22:41:44,004][60935] Updated weights for policy 0, policy_version 42480 (0.0008) [2023-10-13 22:41:44,370][60935] Updated weights for policy 0, policy_version 42490 (0.0007) [2023-10-13 22:41:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 87621632. Throughput: 0: 1684.4, 1: 1698.7. Samples: 21910710. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-13 22:41:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:41:46,587][60934] Updated weights for policy 1, policy_version 42782 (0.0011) [2023-10-13 22:41:46,957][60934] Updated weights for policy 1, policy_version 42792 (0.0007) [2023-10-13 22:41:47,317][60934] Updated weights for policy 1, policy_version 42802 (0.0007) [2023-10-13 22:41:48,446][60935] Updated weights for policy 0, policy_version 42500 (0.0010) [2023-10-13 22:41:48,825][60935] Updated weights for policy 0, policy_version 42510 (0.0009) [2023-10-13 22:41:49,181][60935] Updated weights for policy 0, policy_version 42520 (0.0008) [2023-10-13 22:41:51,207][60934] Updated weights for policy 1, policy_version 42812 (0.0009) [2023-10-13 22:41:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 87687168. Throughput: 0: 1673.9, 1: 1722.0. Samples: 21930790. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-13 22:41:51,248][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 22:41:51,569][60934] Updated weights for policy 1, policy_version 42822 (0.0009) [2023-10-13 22:41:51,936][60934] Updated weights for policy 1, policy_version 42832 (0.0008) [2023-10-13 22:41:53,274][60935] Updated weights for policy 0, policy_version 42530 (0.0009) [2023-10-13 22:41:53,680][60935] Updated weights for policy 0, policy_version 42540 (0.0010) [2023-10-13 22:41:54,049][60935] Updated weights for policy 0, policy_version 42550 (0.0010) [2023-10-13 22:41:54,417][60935] Updated weights for policy 0, policy_version 42560 (0.0008) [2023-10-13 22:41:55,880][60934] Updated weights for policy 1, policy_version 42842 (0.0007) [2023-10-13 22:41:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 87752704. Throughput: 0: 1696.8, 1: 1721.6. Samples: 21951962. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-13 22:41:56,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 22:41:56,255][60934] Updated weights for policy 1, policy_version 42852 (0.0008) [2023-10-13 22:41:56,615][60934] Updated weights for policy 1, policy_version 42862 (0.0008) [2023-10-13 22:41:56,976][60934] Updated weights for policy 1, policy_version 42872 (0.0007) [2023-10-13 22:41:58,541][60935] Updated weights for policy 0, policy_version 42570 (0.0009) [2023-10-13 22:41:58,898][60935] Updated weights for policy 0, policy_version 42580 (0.0009) [2023-10-13 22:41:59,268][60935] Updated weights for policy 0, policy_version 42590 (0.0010) [2023-10-13 22:42:00,895][60934] Updated weights for policy 1, policy_version 42882 (0.0008) [2023-10-13 22:42:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 87818240. Throughput: 0: 1680.7, 1: 1715.1. Samples: 21961790. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-13 22:42:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:42:01,266][60934] Updated weights for policy 1, policy_version 42892 (0.0007) [2023-10-13 22:42:01,618][60934] Updated weights for policy 1, policy_version 42902 (0.0007) [2023-10-13 22:42:03,244][60935] Updated weights for policy 0, policy_version 42600 (0.0010) [2023-10-13 22:42:03,617][60935] Updated weights for policy 0, policy_version 42610 (0.0009) [2023-10-13 22:42:03,989][60935] Updated weights for policy 0, policy_version 42620 (0.0008) [2023-10-13 22:42:05,528][60934] Updated weights for policy 1, policy_version 42912 (0.0008) [2023-10-13 22:42:05,896][60934] Updated weights for policy 1, policy_version 42922 (0.0011) [2023-10-13 22:42:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 87883776. Throughput: 0: 1686.3, 1: 1729.5. Samples: 21982392. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-13 22:42:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:42:06,268][60934] Updated weights for policy 1, policy_version 42932 (0.0009) [2023-10-13 22:42:08,010][60935] Updated weights for policy 0, policy_version 42630 (0.0009) [2023-10-13 22:42:08,375][60935] Updated weights for policy 0, policy_version 42640 (0.0008) [2023-10-13 22:42:08,745][60935] Updated weights for policy 0, policy_version 42650 (0.0010) [2023-10-13 22:42:10,211][60934] Updated weights for policy 1, policy_version 42942 (0.0008) [2023-10-13 22:42:10,579][60934] Updated weights for policy 1, policy_version 42952 (0.0010) [2023-10-13 22:42:10,948][60934] Updated weights for policy 1, policy_version 42962 (0.0011) [2023-10-13 22:42:11,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 87982080. Throughput: 0: 1695.1, 1: 1718.2. Samples: 22002908. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-13 22:42:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:42:12,843][60935] Updated weights for policy 0, policy_version 42660 (0.0010) [2023-10-13 22:42:13,209][60935] Updated weights for policy 0, policy_version 42670 (0.0009) [2023-10-13 22:42:13,580][60935] Updated weights for policy 0, policy_version 42680 (0.0007) [2023-10-13 22:42:14,849][60934] Updated weights for policy 1, policy_version 42972 (0.0009) [2023-10-13 22:42:15,210][60934] Updated weights for policy 1, policy_version 42982 (0.0008) [2023-10-13 22:42:15,580][60934] Updated weights for policy 1, policy_version 42992 (0.0008) [2023-10-13 22:42:16,248][59943] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 88047616. Throughput: 0: 1672.6, 1: 1736.8. Samples: 22012972. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 22:42:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:42:17,427][60935] Updated weights for policy 0, policy_version 42690 (0.0008) [2023-10-13 22:42:17,783][60935] Updated weights for policy 0, policy_version 42700 (0.0009) [2023-10-13 22:42:18,150][60935] Updated weights for policy 0, policy_version 42710 (0.0008) [2023-10-13 22:42:18,523][60935] Updated weights for policy 0, policy_version 42720 (0.0010) [2023-10-13 22:42:19,657][60934] Updated weights for policy 1, policy_version 43002 (0.0009) [2023-10-13 22:42:20,025][60934] Updated weights for policy 1, policy_version 43012 (0.0008) [2023-10-13 22:42:20,387][60934] Updated weights for policy 1, policy_version 43022 (0.0008) [2023-10-13 22:42:20,746][60934] Updated weights for policy 1, policy_version 43032 (0.0008) [2023-10-13 22:42:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 88113152. Throughput: 0: 1696.9, 1: 1734.4. Samples: 22033794. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 22:42:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:42:22,624][60935] Updated weights for policy 0, policy_version 42730 (0.0011) [2023-10-13 22:42:22,990][60935] Updated weights for policy 0, policy_version 42740 (0.0009) [2023-10-13 22:42:23,359][60935] Updated weights for policy 0, policy_version 42750 (0.0007) [2023-10-13 22:42:24,853][60934] Updated weights for policy 1, policy_version 43042 (0.0011) [2023-10-13 22:42:25,215][60934] Updated weights for policy 1, policy_version 43052 (0.0009) [2023-10-13 22:42:25,581][60934] Updated weights for policy 1, policy_version 43062 (0.0009) [2023-10-13 22:42:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 88178688. Throughput: 0: 1686.0, 1: 1703.5. Samples: 22053050. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 22:42:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:42:27,794][60935] Updated weights for policy 0, policy_version 42760 (0.0008) [2023-10-13 22:42:28,157][60935] Updated weights for policy 0, policy_version 42770 (0.0007) [2023-10-13 22:42:28,521][60935] Updated weights for policy 0, policy_version 42780 (0.0009) [2023-10-13 22:42:29,474][60934] Updated weights for policy 1, policy_version 43072 (0.0008) [2023-10-13 22:42:29,839][60934] Updated weights for policy 1, policy_version 43082 (0.0010) [2023-10-13 22:42:30,200][60934] Updated weights for policy 1, policy_version 43092 (0.0010) [2023-10-13 22:42:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 88244224. Throughput: 0: 1657.1, 1: 1733.0. Samples: 22063264. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 22:42:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:42:32,523][60935] Updated weights for policy 0, policy_version 42790 (0.0008) [2023-10-13 22:42:32,896][60935] Updated weights for policy 0, policy_version 42800 (0.0012) [2023-10-13 22:42:33,279][60935] Updated weights for policy 0, policy_version 42810 (0.0011) [2023-10-13 22:42:34,111][60934] Updated weights for policy 1, policy_version 43102 (0.0007) [2023-10-13 22:42:34,468][60934] Updated weights for policy 1, policy_version 43112 (0.0007) [2023-10-13 22:42:34,839][60934] Updated weights for policy 1, policy_version 43122 (0.0007) [2023-10-13 22:42:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 88309760. Throughput: 0: 1680.5, 1: 1718.3. Samples: 22083738. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 22:42:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:42:37,333][60935] Updated weights for policy 0, policy_version 42820 (0.0010) [2023-10-13 22:42:37,707][60935] Updated weights for policy 0, policy_version 42830 (0.0011) [2023-10-13 22:42:38,086][60935] Updated weights for policy 0, policy_version 42840 (0.0009) [2023-10-13 22:42:38,996][60934] Updated weights for policy 1, policy_version 43132 (0.0008) [2023-10-13 22:42:39,367][60934] Updated weights for policy 1, policy_version 43142 (0.0011) [2023-10-13 22:42:39,734][60934] Updated weights for policy 1, policy_version 43152 (0.0008) [2023-10-13 22:42:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 88375296. Throughput: 0: 1675.4, 1: 1700.9. Samples: 22103894. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-13 22:42:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:42:42,335][60935] Updated weights for policy 0, policy_version 42850 (0.0009) [2023-10-13 22:42:42,738][60935] Updated weights for policy 0, policy_version 42860 (0.0008) [2023-10-13 22:42:43,111][60935] Updated weights for policy 0, policy_version 42870 (0.0009) [2023-10-13 22:42:43,490][60935] Updated weights for policy 0, policy_version 42880 (0.0009) [2023-10-13 22:42:43,686][60934] Updated weights for policy 1, policy_version 43162 (0.0008) [2023-10-13 22:42:44,054][60934] Updated weights for policy 1, policy_version 43172 (0.0007) [2023-10-13 22:42:44,427][60934] Updated weights for policy 1, policy_version 43182 (0.0009) [2023-10-13 22:42:44,795][60934] Updated weights for policy 1, policy_version 43192 (0.0009) [2023-10-13 22:42:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 88440832. Throughput: 0: 1661.1, 1: 1727.1. Samples: 22114262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:42:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:42:47,199][60935] Updated weights for policy 0, policy_version 42890 (0.0010) [2023-10-13 22:42:47,563][60935] Updated weights for policy 0, policy_version 42900 (0.0009) [2023-10-13 22:42:47,936][60935] Updated weights for policy 0, policy_version 42910 (0.0008) [2023-10-13 22:42:48,804][60934] Updated weights for policy 1, policy_version 43202 (0.0009) [2023-10-13 22:42:49,173][60934] Updated weights for policy 1, policy_version 43212 (0.0007) [2023-10-13 22:42:49,535][60934] Updated weights for policy 1, policy_version 43222 (0.0007) [2023-10-13 22:42:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 88506368. Throughput: 0: 1676.4, 1: 1695.6. Samples: 22134128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:42:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:42:52,098][60935] Updated weights for policy 0, policy_version 42920 (0.0009) [2023-10-13 22:42:52,471][60935] Updated weights for policy 0, policy_version 42930 (0.0009) [2023-10-13 22:42:52,840][60935] Updated weights for policy 0, policy_version 42940 (0.0007) [2023-10-13 22:42:53,289][60934] Updated weights for policy 1, policy_version 43232 (0.0008) [2023-10-13 22:42:53,666][60934] Updated weights for policy 1, policy_version 43242 (0.0008) [2023-10-13 22:42:54,018][60934] Updated weights for policy 1, policy_version 43252 (0.0009) [2023-10-13 22:42:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 88571904. Throughput: 0: 1673.3, 1: 1704.1. Samples: 22154892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:42:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:42:56,259][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000042944_43974656.pth... [2023-10-13 22:42:56,259][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000043256_44597248.pth... [2023-10-13 22:42:56,289][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000041376_42369024.pth [2023-10-13 22:42:56,295][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000041656_42958848.pth [2023-10-13 22:42:56,812][60935] Updated weights for policy 0, policy_version 42950 (0.0009) [2023-10-13 22:42:57,179][60935] Updated weights for policy 0, policy_version 42960 (0.0012) [2023-10-13 22:42:57,549][60935] Updated weights for policy 0, policy_version 42970 (0.0008) [2023-10-13 22:42:57,958][60934] Updated weights for policy 1, policy_version 43262 (0.0009) [2023-10-13 22:42:58,322][60934] Updated weights for policy 1, policy_version 43272 (0.0011) [2023-10-13 22:42:58,687][60934] Updated weights for policy 1, policy_version 43282 (0.0010) [2023-10-13 22:43:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 88637440. Throughput: 0: 1668.9, 1: 1698.6. Samples: 22164512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:43:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:43:01,710][60935] Updated weights for policy 0, policy_version 42980 (0.0008) [2023-10-13 22:43:02,076][60935] Updated weights for policy 0, policy_version 42990 (0.0009) [2023-10-13 22:43:02,444][60935] Updated weights for policy 0, policy_version 43000 (0.0012) [2023-10-13 22:43:02,773][60934] Updated weights for policy 1, policy_version 43292 (0.0008) [2023-10-13 22:43:03,144][60934] Updated weights for policy 1, policy_version 43302 (0.0009) [2023-10-13 22:43:03,508][60934] Updated weights for policy 1, policy_version 43312 (0.0009) [2023-10-13 22:43:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 88702976. Throughput: 0: 1671.9, 1: 1689.9. Samples: 22185072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:43:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:43:06,534][60935] Updated weights for policy 0, policy_version 43010 (0.0009) [2023-10-13 22:43:06,897][60935] Updated weights for policy 0, policy_version 43020 (0.0010) [2023-10-13 22:43:07,265][60935] Updated weights for policy 0, policy_version 43030 (0.0009) [2023-10-13 22:43:07,565][60934] Updated weights for policy 1, policy_version 43322 (0.0007) [2023-10-13 22:43:07,646][60935] Updated weights for policy 0, policy_version 43040 (0.0010) [2023-10-13 22:43:07,923][60934] Updated weights for policy 1, policy_version 43332 (0.0008) [2023-10-13 22:43:08,285][60934] Updated weights for policy 1, policy_version 43342 (0.0008) [2023-10-13 22:43:08,654][60934] Updated weights for policy 1, policy_version 43352 (0.0008) [2023-10-13 22:43:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 88768512. Throughput: 0: 1679.7, 1: 1722.5. Samples: 22206152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:43:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:43:11,915][60935] Updated weights for policy 0, policy_version 43050 (0.0011) [2023-10-13 22:43:12,285][60935] Updated weights for policy 0, policy_version 43060 (0.0008) [2023-10-13 22:43:12,645][60935] Updated weights for policy 0, policy_version 43070 (0.0007) [2023-10-13 22:43:12,664][60934] Updated weights for policy 1, policy_version 43362 (0.0008) [2023-10-13 22:43:13,049][60934] Updated weights for policy 1, policy_version 43372 (0.0010) [2023-10-13 22:43:13,422][60934] Updated weights for policy 1, policy_version 43382 (0.0010) [2023-10-13 22:43:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 88834048. Throughput: 0: 1684.4, 1: 1693.9. Samples: 22215290. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 22:43:16,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 22:43:16,584][60935] Updated weights for policy 0, policy_version 43080 (0.0007) [2023-10-13 22:43:16,957][60935] Updated weights for policy 0, policy_version 43090 (0.0008) [2023-10-13 22:43:17,329][60935] Updated weights for policy 0, policy_version 43100 (0.0009) [2023-10-13 22:43:17,364][60934] Updated weights for policy 1, policy_version 43392 (0.0008) [2023-10-13 22:43:17,729][60934] Updated weights for policy 1, policy_version 43402 (0.0007) [2023-10-13 22:43:18,099][60934] Updated weights for policy 1, policy_version 43412 (0.0008) [2023-10-13 22:43:21,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 88899584. Throughput: 0: 1682.9, 1: 1710.7. Samples: 22236448. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 22:43:21,248][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 22:43:21,389][60935] Updated weights for policy 0, policy_version 43110 (0.0010) [2023-10-13 22:43:21,769][60935] Updated weights for policy 0, policy_version 43120 (0.0010) [2023-10-13 22:43:21,866][60934] Updated weights for policy 1, policy_version 43422 (0.0007) [2023-10-13 22:43:22,136][60935] Updated weights for policy 0, policy_version 43130 (0.0008) [2023-10-13 22:43:22,229][60934] Updated weights for policy 1, policy_version 43432 (0.0008) [2023-10-13 22:43:22,606][60934] Updated weights for policy 1, policy_version 43442 (0.0008) [2023-10-13 22:43:26,202][60935] Updated weights for policy 0, policy_version 43140 (0.0009) [2023-10-13 22:43:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 88965120. Throughput: 0: 1685.8, 1: 1724.3. Samples: 22257350. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 22:43:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:43:26,567][60935] Updated weights for policy 0, policy_version 43150 (0.0009) [2023-10-13 22:43:26,625][60934] Updated weights for policy 1, policy_version 43452 (0.0007) [2023-10-13 22:43:26,945][60935] Updated weights for policy 0, policy_version 43160 (0.0008) [2023-10-13 22:43:26,992][60934] Updated weights for policy 1, policy_version 43462 (0.0007) [2023-10-13 22:43:27,357][60934] Updated weights for policy 1, policy_version 43472 (0.0007) [2023-10-13 22:43:30,996][60935] Updated weights for policy 0, policy_version 43170 (0.0009) [2023-10-13 22:43:31,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 89030656. Throughput: 0: 1687.7, 1: 1695.7. Samples: 22266518. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 22:43:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:43:31,414][60935] Updated weights for policy 0, policy_version 43180 (0.0010) [2023-10-13 22:43:31,448][60934] Updated weights for policy 1, policy_version 43482 (0.0010) [2023-10-13 22:43:31,784][60935] Updated weights for policy 0, policy_version 43190 (0.0009) [2023-10-13 22:43:31,808][60934] Updated weights for policy 1, policy_version 43492 (0.0009) [2023-10-13 22:43:32,157][60935] Updated weights for policy 0, policy_version 43200 (0.0009) [2023-10-13 22:43:32,179][60934] Updated weights for policy 1, policy_version 43502 (0.0010) [2023-10-13 22:43:32,542][60934] Updated weights for policy 1, policy_version 43512 (0.0008) [2023-10-13 22:43:36,174][60935] Updated weights for policy 0, policy_version 43210 (0.0008) [2023-10-13 22:43:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 89096192. Throughput: 0: 1678.7, 1: 1719.9. Samples: 22287062. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 22:43:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:43:36,526][60934] Updated weights for policy 1, policy_version 43522 (0.0007) [2023-10-13 22:43:36,539][60935] Updated weights for policy 0, policy_version 43220 (0.0007) [2023-10-13 22:43:36,880][60934] Updated weights for policy 1, policy_version 43532 (0.0007) [2023-10-13 22:43:36,911][60935] Updated weights for policy 0, policy_version 43230 (0.0008) [2023-10-13 22:43:37,248][60934] Updated weights for policy 1, policy_version 43542 (0.0009) [2023-10-13 22:43:40,860][60935] Updated weights for policy 0, policy_version 43240 (0.0009) [2023-10-13 22:43:41,231][60935] Updated weights for policy 0, policy_version 43250 (0.0010) [2023-10-13 22:43:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 89161728. Throughput: 0: 1679.3, 1: 1720.8. Samples: 22307898. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 22:43:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:43:41,289][60934] Updated weights for policy 1, policy_version 43552 (0.0008) [2023-10-13 22:43:41,607][60935] Updated weights for policy 0, policy_version 43260 (0.0008) [2023-10-13 22:43:41,660][60934] Updated weights for policy 1, policy_version 43562 (0.0007) [2023-10-13 22:43:42,027][60934] Updated weights for policy 1, policy_version 43572 (0.0008) [2023-10-13 22:43:45,667][60935] Updated weights for policy 0, policy_version 43270 (0.0010) [2023-10-13 22:43:46,035][60935] Updated weights for policy 0, policy_version 43280 (0.0010) [2023-10-13 22:43:46,055][60934] Updated weights for policy 1, policy_version 43582 (0.0009) [2023-10-13 22:43:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 89227264. Throughput: 0: 1687.5, 1: 1707.6. Samples: 22317294. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-13 22:43:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:43:46,395][60935] Updated weights for policy 0, policy_version 43290 (0.0009) [2023-10-13 22:43:46,422][60934] Updated weights for policy 1, policy_version 43592 (0.0007) [2023-10-13 22:43:46,793][60934] Updated weights for policy 1, policy_version 43602 (0.0007) [2023-10-13 22:43:50,574][60935] Updated weights for policy 0, policy_version 43300 (0.0010) [2023-10-13 22:43:50,840][60934] Updated weights for policy 1, policy_version 43612 (0.0008) [2023-10-13 22:43:50,936][60935] Updated weights for policy 0, policy_version 43310 (0.0008) [2023-10-13 22:43:51,206][60934] Updated weights for policy 1, policy_version 43622 (0.0008) [2023-10-13 22:43:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 89292800. Throughput: 0: 1681.2, 1: 1718.3. Samples: 22338052. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-13 22:43:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:43:51,306][60935] Updated weights for policy 0, policy_version 43320 (0.0007) [2023-10-13 22:43:51,577][60934] Updated weights for policy 1, policy_version 43632 (0.0008) [2023-10-13 22:43:55,562][60934] Updated weights for policy 1, policy_version 43642 (0.0009) [2023-10-13 22:43:55,582][60935] Updated weights for policy 0, policy_version 43330 (0.0007) [2023-10-13 22:43:55,932][60934] Updated weights for policy 1, policy_version 43652 (0.0009) [2023-10-13 22:43:55,951][60935] Updated weights for policy 0, policy_version 43340 (0.0008) [2023-10-13 22:43:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 89358336. Throughput: 0: 1668.5, 1: 1715.1. Samples: 22358416. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-13 22:43:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:43:56,300][60934] Updated weights for policy 1, policy_version 43662 (0.0008) [2023-10-13 22:43:56,319][60935] Updated weights for policy 0, policy_version 43350 (0.0009) [2023-10-13 22:43:56,659][60934] Updated weights for policy 1, policy_version 43672 (0.0009) [2023-10-13 22:43:56,691][60935] Updated weights for policy 0, policy_version 43360 (0.0008) [2023-10-13 22:44:00,564][60934] Updated weights for policy 1, policy_version 43682 (0.0008) [2023-10-13 22:44:00,841][60935] Updated weights for policy 0, policy_version 43370 (0.0008) [2023-10-13 22:44:00,937][60934] Updated weights for policy 1, policy_version 43692 (0.0008) [2023-10-13 22:44:01,216][60935] Updated weights for policy 0, policy_version 43380 (0.0008) [2023-10-13 22:44:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 89423872. Throughput: 0: 1675.0, 1: 1717.1. Samples: 22367934. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-13 22:44:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:44:01,299][60934] Updated weights for policy 1, policy_version 43702 (0.0008) [2023-10-13 22:44:01,577][60935] Updated weights for policy 0, policy_version 43390 (0.0008) [2023-10-13 22:44:05,324][60934] Updated weights for policy 1, policy_version 43712 (0.0008) [2023-10-13 22:44:05,581][60935] Updated weights for policy 0, policy_version 43400 (0.0007) [2023-10-13 22:44:05,690][60934] Updated weights for policy 1, policy_version 43722 (0.0009) [2023-10-13 22:44:05,958][60935] Updated weights for policy 0, policy_version 43410 (0.0008) [2023-10-13 22:44:06,049][60934] Updated weights for policy 1, policy_version 43732 (0.0008) [2023-10-13 22:44:06,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 89522176. Throughput: 0: 1678.0, 1: 1715.3. Samples: 22389146. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-13 22:44:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:44:06,318][60935] Updated weights for policy 0, policy_version 43420 (0.0007) [2023-10-13 22:44:10,015][60934] Updated weights for policy 1, policy_version 43742 (0.0009) [2023-10-13 22:44:10,368][60934] Updated weights for policy 1, policy_version 43752 (0.0008) [2023-10-13 22:44:10,368][60935] Updated weights for policy 0, policy_version 43430 (0.0008) [2023-10-13 22:44:10,735][60935] Updated weights for policy 0, policy_version 43440 (0.0007) [2023-10-13 22:44:10,740][60934] Updated weights for policy 1, policy_version 43762 (0.0009) [2023-10-13 22:44:11,103][60935] Updated weights for policy 0, policy_version 43450 (0.0009) [2023-10-13 22:44:11,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 89587712. Throughput: 0: 1663.8, 1: 1693.9. Samples: 22408446. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-13 22:44:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:44:14,817][60934] Updated weights for policy 1, policy_version 43772 (0.0009) [2023-10-13 22:44:15,012][60935] Updated weights for policy 0, policy_version 43460 (0.0007) [2023-10-13 22:44:15,185][60934] Updated weights for policy 1, policy_version 43782 (0.0008) [2023-10-13 22:44:15,378][60935] Updated weights for policy 0, policy_version 43470 (0.0008) [2023-10-13 22:44:15,551][60934] Updated weights for policy 1, policy_version 43792 (0.0008) [2023-10-13 22:44:15,747][60935] Updated weights for policy 0, policy_version 43480 (0.0008) [2023-10-13 22:44:16,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 89686016. Throughput: 0: 1682.7, 1: 1711.0. Samples: 22419232. Policy #0 lag: (min: 26.0, avg: 26.5, max: 41.0) [2023-10-13 22:44:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:44:19,628][60934] Updated weights for policy 1, policy_version 43802 (0.0008) [2023-10-13 22:44:19,836][60935] Updated weights for policy 0, policy_version 43490 (0.0009) [2023-10-13 22:44:20,001][60934] Updated weights for policy 1, policy_version 43812 (0.0007) [2023-10-13 22:44:20,227][60935] Updated weights for policy 0, policy_version 43500 (0.0008) [2023-10-13 22:44:20,375][60934] Updated weights for policy 1, policy_version 43822 (0.0008) [2023-10-13 22:44:20,600][60935] Updated weights for policy 0, policy_version 43510 (0.0008) [2023-10-13 22:44:20,734][60934] Updated weights for policy 1, policy_version 43832 (0.0009) [2023-10-13 22:44:20,956][60935] Updated weights for policy 0, policy_version 43520 (0.0009) [2023-10-13 22:44:21,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 89751552. Throughput: 0: 1686.4, 1: 1709.1. Samples: 22439858. Policy #0 lag: (min: 26.0, avg: 26.5, max: 41.0) [2023-10-13 22:44:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:44:24,816][60934] Updated weights for policy 1, policy_version 43842 (0.0009) [2023-10-13 22:44:24,929][60935] Updated weights for policy 0, policy_version 43530 (0.0007) [2023-10-13 22:44:25,197][60934] Updated weights for policy 1, policy_version 43852 (0.0008) [2023-10-13 22:44:25,289][60935] Updated weights for policy 0, policy_version 43540 (0.0007) [2023-10-13 22:44:25,554][60934] Updated weights for policy 1, policy_version 43862 (0.0007) [2023-10-13 22:44:25,659][60935] Updated weights for policy 0, policy_version 43550 (0.0007) [2023-10-13 22:44:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 89817088. Throughput: 0: 1660.4, 1: 1682.4. Samples: 22458322. Policy #0 lag: (min: 26.0, avg: 26.5, max: 41.0) [2023-10-13 22:44:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:44:29,547][60934] Updated weights for policy 1, policy_version 43872 (0.0009) [2023-10-13 22:44:29,838][60935] Updated weights for policy 0, policy_version 43560 (0.0010) [2023-10-13 22:44:29,919][60934] Updated weights for policy 1, policy_version 43882 (0.0008) [2023-10-13 22:44:30,207][60935] Updated weights for policy 0, policy_version 43570 (0.0008) [2023-10-13 22:44:30,289][60934] Updated weights for policy 1, policy_version 43892 (0.0007) [2023-10-13 22:44:30,577][60935] Updated weights for policy 0, policy_version 43580 (0.0008) [2023-10-13 22:44:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 89882624. Throughput: 0: 1680.3, 1: 1710.0. Samples: 22469856. Policy #0 lag: (min: 26.0, avg: 26.5, max: 41.0) [2023-10-13 22:44:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:44:34,138][60934] Updated weights for policy 1, policy_version 43902 (0.0009) [2023-10-13 22:44:34,508][60934] Updated weights for policy 1, policy_version 43912 (0.0009) [2023-10-13 22:44:34,647][60935] Updated weights for policy 0, policy_version 43590 (0.0008) [2023-10-13 22:44:34,874][60934] Updated weights for policy 1, policy_version 43922 (0.0008) [2023-10-13 22:44:35,019][60935] Updated weights for policy 0, policy_version 43600 (0.0007) [2023-10-13 22:44:35,383][60935] Updated weights for policy 0, policy_version 43610 (0.0009) [2023-10-13 22:44:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 89948160. Throughput: 0: 1675.2, 1: 1695.6. Samples: 22489740. Policy #0 lag: (min: 26.0, avg: 26.5, max: 41.0) [2023-10-13 22:44:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:44:38,914][60934] Updated weights for policy 1, policy_version 43932 (0.0007) [2023-10-13 22:44:39,287][60934] Updated weights for policy 1, policy_version 43942 (0.0010) [2023-10-13 22:44:39,503][60935] Updated weights for policy 0, policy_version 43620 (0.0009) [2023-10-13 22:44:39,654][60934] Updated weights for policy 1, policy_version 43952 (0.0008) [2023-10-13 22:44:39,886][60935] Updated weights for policy 0, policy_version 43630 (0.0008) [2023-10-13 22:44:40,255][60935] Updated weights for policy 0, policy_version 43640 (0.0008) [2023-10-13 22:44:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 90013696. Throughput: 0: 1669.9, 1: 1680.3. Samples: 22509172. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-10-13 22:44:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:44:43,658][60934] Updated weights for policy 1, policy_version 43962 (0.0008) [2023-10-13 22:44:44,026][60934] Updated weights for policy 1, policy_version 43972 (0.0007) [2023-10-13 22:44:44,364][60935] Updated weights for policy 0, policy_version 43650 (0.0011) [2023-10-13 22:44:44,391][60934] Updated weights for policy 1, policy_version 43982 (0.0007) [2023-10-13 22:44:44,730][60935] Updated weights for policy 0, policy_version 43660 (0.0009) [2023-10-13 22:44:44,749][60934] Updated weights for policy 1, policy_version 43992 (0.0007) [2023-10-13 22:44:45,092][60935] Updated weights for policy 0, policy_version 43670 (0.0009) [2023-10-13 22:44:45,461][60935] Updated weights for policy 0, policy_version 43680 (0.0007) [2023-10-13 22:44:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 90079232. Throughput: 0: 1692.1, 1: 1709.3. Samples: 22520998. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-10-13 22:44:46,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:44:48,950][60934] Updated weights for policy 1, policy_version 44002 (0.0008) [2023-10-13 22:44:49,314][60934] Updated weights for policy 1, policy_version 44012 (0.0009) [2023-10-13 22:44:49,454][60935] Updated weights for policy 0, policy_version 43690 (0.0009) [2023-10-13 22:44:49,678][60934] Updated weights for policy 1, policy_version 44022 (0.0008) [2023-10-13 22:44:49,821][60935] Updated weights for policy 0, policy_version 43700 (0.0008) [2023-10-13 22:44:50,189][60935] Updated weights for policy 0, policy_version 43710 (0.0009) [2023-10-13 22:44:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 90144768. Throughput: 0: 1668.0, 1: 1684.1. Samples: 22539990. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-10-13 22:44:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:44:53,624][60934] Updated weights for policy 1, policy_version 44032 (0.0008) [2023-10-13 22:44:53,994][60934] Updated weights for policy 1, policy_version 44042 (0.0007) [2023-10-13 22:44:54,255][60935] Updated weights for policy 0, policy_version 43720 (0.0008) [2023-10-13 22:44:54,362][60934] Updated weights for policy 1, policy_version 44052 (0.0007) [2023-10-13 22:44:54,626][60935] Updated weights for policy 0, policy_version 43730 (0.0008) [2023-10-13 22:44:55,006][60935] Updated weights for policy 0, policy_version 43740 (0.0008) [2023-10-13 22:44:56,248][59943] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 90210304. Throughput: 0: 1673.9, 1: 1700.3. Samples: 22560286. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-10-13 22:44:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:44:56,263][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000043744_44793856.pth... [2023-10-13 22:44:56,263][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000044056_45416448.pth... [2023-10-13 22:44:56,302][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000042176_43188224.pth [2023-10-13 22:44:56,306][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000042456_43778048.pth [2023-10-13 22:44:58,288][60934] Updated weights for policy 1, policy_version 44062 (0.0007) [2023-10-13 22:44:58,655][60934] Updated weights for policy 1, policy_version 44072 (0.0008) [2023-10-13 22:44:58,962][60935] Updated weights for policy 0, policy_version 43750 (0.0008) [2023-10-13 22:44:59,011][60934] Updated weights for policy 1, policy_version 44082 (0.0007) [2023-10-13 22:44:59,342][60935] Updated weights for policy 0, policy_version 43760 (0.0007) [2023-10-13 22:44:59,710][60935] Updated weights for policy 0, policy_version 43770 (0.0011) [2023-10-13 22:45:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 90275840. Throughput: 0: 1684.1, 1: 1700.3. Samples: 22571530. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-10-13 22:45:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:45:03,108][60934] Updated weights for policy 1, policy_version 44092 (0.0007) [2023-10-13 22:45:03,481][60934] Updated weights for policy 1, policy_version 44102 (0.0009) [2023-10-13 22:45:03,713][60935] Updated weights for policy 0, policy_version 43780 (0.0008) [2023-10-13 22:45:03,838][60934] Updated weights for policy 1, policy_version 44112 (0.0009) [2023-10-13 22:45:04,081][60935] Updated weights for policy 0, policy_version 43790 (0.0008) [2023-10-13 22:45:04,447][60935] Updated weights for policy 0, policy_version 43800 (0.0008) [2023-10-13 22:45:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 90341376. Throughput: 0: 1656.8, 1: 1687.4. Samples: 22590346. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-10-13 22:45:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:45:07,664][60934] Updated weights for policy 1, policy_version 44122 (0.0008) [2023-10-13 22:45:08,027][60934] Updated weights for policy 1, policy_version 44132 (0.0010) [2023-10-13 22:45:08,393][60934] Updated weights for policy 1, policy_version 44142 (0.0010) [2023-10-13 22:45:08,569][60935] Updated weights for policy 0, policy_version 43810 (0.0008) [2023-10-13 22:45:08,761][60934] Updated weights for policy 1, policy_version 44152 (0.0007) [2023-10-13 22:45:08,943][60935] Updated weights for policy 0, policy_version 43820 (0.0009) [2023-10-13 22:45:09,310][60935] Updated weights for policy 0, policy_version 43830 (0.0010) [2023-10-13 22:45:09,682][60935] Updated weights for policy 0, policy_version 43840 (0.0009) [2023-10-13 22:45:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 90406912. Throughput: 0: 1678.4, 1: 1718.6. Samples: 22611186. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-13 22:45:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:45:12,649][60934] Updated weights for policy 1, policy_version 44162 (0.0008) [2023-10-13 22:45:13,021][60934] Updated weights for policy 1, policy_version 44172 (0.0010) [2023-10-13 22:45:13,382][60934] Updated weights for policy 1, policy_version 44182 (0.0009) [2023-10-13 22:45:13,721][60935] Updated weights for policy 0, policy_version 43850 (0.0008) [2023-10-13 22:45:14,087][60935] Updated weights for policy 0, policy_version 43860 (0.0007) [2023-10-13 22:45:14,461][60935] Updated weights for policy 0, policy_version 43870 (0.0008) [2023-10-13 22:45:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 90472448. Throughput: 0: 1670.2, 1: 1690.1. Samples: 22621068. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-13 22:45:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:45:17,212][60934] Updated weights for policy 1, policy_version 44192 (0.0007) [2023-10-13 22:45:17,584][60934] Updated weights for policy 1, policy_version 44202 (0.0008) [2023-10-13 22:45:17,951][60934] Updated weights for policy 1, policy_version 44212 (0.0008) [2023-10-13 22:45:18,576][60935] Updated weights for policy 0, policy_version 43880 (0.0011) [2023-10-13 22:45:18,933][60935] Updated weights for policy 0, policy_version 43890 (0.0010) [2023-10-13 22:45:19,297][60935] Updated weights for policy 0, policy_version 43900 (0.0011) [2023-10-13 22:45:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 90537984. Throughput: 0: 1659.4, 1: 1712.2. Samples: 22641464. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-13 22:45:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:45:22,022][60934] Updated weights for policy 1, policy_version 44222 (0.0009) [2023-10-13 22:45:22,390][60934] Updated weights for policy 1, policy_version 44232 (0.0007) [2023-10-13 22:45:22,747][60934] Updated weights for policy 1, policy_version 44242 (0.0009) [2023-10-13 22:45:23,351][60935] Updated weights for policy 0, policy_version 43910 (0.0009) [2023-10-13 22:45:23,718][60935] Updated weights for policy 0, policy_version 43920 (0.0011) [2023-10-13 22:45:24,081][60935] Updated weights for policy 0, policy_version 43930 (0.0010) [2023-10-13 22:45:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 90603520. Throughput: 0: 1681.1, 1: 1727.7. Samples: 22662568. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-13 22:45:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:45:26,876][60934] Updated weights for policy 1, policy_version 44252 (0.0010) [2023-10-13 22:45:27,244][60934] Updated weights for policy 1, policy_version 44262 (0.0007) [2023-10-13 22:45:27,610][60934] Updated weights for policy 1, policy_version 44272 (0.0009) [2023-10-13 22:45:28,329][60935] Updated weights for policy 0, policy_version 43940 (0.0009) [2023-10-13 22:45:28,698][60935] Updated weights for policy 0, policy_version 43950 (0.0010) [2023-10-13 22:45:29,078][60935] Updated weights for policy 0, policy_version 43960 (0.0010) [2023-10-13 22:45:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 90669056. Throughput: 0: 1664.1, 1: 1697.1. Samples: 22672252. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-13 22:45:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:45:31,567][60934] Updated weights for policy 1, policy_version 44282 (0.0009) [2023-10-13 22:45:31,922][60934] Updated weights for policy 1, policy_version 44292 (0.0008) [2023-10-13 22:45:32,286][60934] Updated weights for policy 1, policy_version 44302 (0.0008) [2023-10-13 22:45:32,648][60934] Updated weights for policy 1, policy_version 44312 (0.0007) [2023-10-13 22:45:33,124][60935] Updated weights for policy 0, policy_version 43970 (0.0009) [2023-10-13 22:45:33,506][60935] Updated weights for policy 0, policy_version 43980 (0.0008) [2023-10-13 22:45:33,879][60935] Updated weights for policy 0, policy_version 43990 (0.0009) [2023-10-13 22:45:34,240][60935] Updated weights for policy 0, policy_version 44000 (0.0009) [2023-10-13 22:45:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 90734592. Throughput: 0: 1672.5, 1: 1723.7. Samples: 22692820. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-13 22:45:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:45:36,742][60934] Updated weights for policy 1, policy_version 44322 (0.0010) [2023-10-13 22:45:37,111][60934] Updated weights for policy 1, policy_version 44332 (0.0010) [2023-10-13 22:45:37,474][60934] Updated weights for policy 1, policy_version 44342 (0.0009) [2023-10-13 22:45:38,273][60935] Updated weights for policy 0, policy_version 44010 (0.0012) [2023-10-13 22:45:38,640][60935] Updated weights for policy 0, policy_version 44020 (0.0008) [2023-10-13 22:45:39,011][60935] Updated weights for policy 0, policy_version 44030 (0.0008) [2023-10-13 22:45:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 90800128. Throughput: 0: 1681.8, 1: 1725.6. Samples: 22713620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:45:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:45:41,432][60934] Updated weights for policy 1, policy_version 44352 (0.0008) [2023-10-13 22:45:41,800][60934] Updated weights for policy 1, policy_version 44362 (0.0009) [2023-10-13 22:45:42,159][60934] Updated weights for policy 1, policy_version 44372 (0.0007) [2023-10-13 22:45:43,285][60935] Updated weights for policy 0, policy_version 44040 (0.0010) [2023-10-13 22:45:43,655][60935] Updated weights for policy 0, policy_version 44050 (0.0009) [2023-10-13 22:45:44,026][60935] Updated weights for policy 0, policy_version 44060 (0.0009) [2023-10-13 22:45:46,171][60934] Updated weights for policy 1, policy_version 44382 (0.0010) [2023-10-13 22:45:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 90865664. Throughput: 0: 1663.7, 1: 1704.9. Samples: 22723116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:45:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:45:46,528][60934] Updated weights for policy 1, policy_version 44392 (0.0009) [2023-10-13 22:45:46,892][60934] Updated weights for policy 1, policy_version 44402 (0.0011) [2023-10-13 22:45:47,887][60935] Updated weights for policy 0, policy_version 44070 (0.0009) [2023-10-13 22:45:48,256][60935] Updated weights for policy 0, policy_version 44080 (0.0012) [2023-10-13 22:45:48,629][60935] Updated weights for policy 0, policy_version 44090 (0.0011) [2023-10-13 22:45:50,892][60934] Updated weights for policy 1, policy_version 44412 (0.0007) [2023-10-13 22:45:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 90931200. Throughput: 0: 1686.4, 1: 1725.8. Samples: 22743894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:45:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:45:51,260][60934] Updated weights for policy 1, policy_version 44422 (0.0007) [2023-10-13 22:45:51,624][60934] Updated weights for policy 1, policy_version 44432 (0.0007) [2023-10-13 22:45:52,658][60935] Updated weights for policy 0, policy_version 44100 (0.0010) [2023-10-13 22:45:53,043][60935] Updated weights for policy 0, policy_version 44110 (0.0009) [2023-10-13 22:45:53,407][60935] Updated weights for policy 0, policy_version 44120 (0.0009) [2023-10-13 22:45:55,442][60934] Updated weights for policy 1, policy_version 44442 (0.0009) [2023-10-13 22:45:55,808][60934] Updated weights for policy 1, policy_version 44452 (0.0009) [2023-10-13 22:45:56,177][60934] Updated weights for policy 1, policy_version 44462 (0.0008) [2023-10-13 22:45:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 90996736. Throughput: 0: 1697.8, 1: 1721.3. Samples: 22765048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:45:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:45:56,541][60934] Updated weights for policy 1, policy_version 44472 (0.0009) [2023-10-13 22:45:57,438][60935] Updated weights for policy 0, policy_version 44130 (0.0007) [2023-10-13 22:45:57,841][60935] Updated weights for policy 0, policy_version 44140 (0.0010) [2023-10-13 22:45:58,213][60935] Updated weights for policy 0, policy_version 44150 (0.0011) [2023-10-13 22:45:58,579][60935] Updated weights for policy 0, policy_version 44160 (0.0008) [2023-10-13 22:46:00,463][60934] Updated weights for policy 1, policy_version 44482 (0.0009) [2023-10-13 22:46:00,832][60934] Updated weights for policy 1, policy_version 44492 (0.0007) [2023-10-13 22:46:01,213][60934] Updated weights for policy 1, policy_version 44502 (0.0008) [2023-10-13 22:46:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 91062272. Throughput: 0: 1675.0, 1: 1729.6. Samples: 22774274. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:46:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:46:02,550][60935] Updated weights for policy 0, policy_version 44170 (0.0008) [2023-10-13 22:46:02,925][60935] Updated weights for policy 0, policy_version 44180 (0.0007) [2023-10-13 22:46:03,292][60935] Updated weights for policy 0, policy_version 44190 (0.0010) [2023-10-13 22:46:05,276][60934] Updated weights for policy 1, policy_version 44512 (0.0008) [2023-10-13 22:46:05,640][60934] Updated weights for policy 1, policy_version 44522 (0.0007) [2023-10-13 22:46:06,007][60934] Updated weights for policy 1, policy_version 44532 (0.0008) [2023-10-13 22:46:06,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 91160576. Throughput: 0: 1695.8, 1: 1722.1. Samples: 22795270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:46:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:46:07,369][60935] Updated weights for policy 0, policy_version 44200 (0.0010) [2023-10-13 22:46:07,744][60935] Updated weights for policy 0, policy_version 44210 (0.0011) [2023-10-13 22:46:08,111][60935] Updated weights for policy 0, policy_version 44220 (0.0010) [2023-10-13 22:46:10,004][60934] Updated weights for policy 1, policy_version 44542 (0.0009) [2023-10-13 22:46:10,372][60934] Updated weights for policy 1, policy_version 44552 (0.0008) [2023-10-13 22:46:10,734][60934] Updated weights for policy 1, policy_version 44562 (0.0008) [2023-10-13 22:46:11,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 91226112. Throughput: 0: 1693.2, 1: 1708.5. Samples: 22815642. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-13 22:46:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:46:12,139][60935] Updated weights for policy 0, policy_version 44230 (0.0009) [2023-10-13 22:46:12,507][60935] Updated weights for policy 0, policy_version 44240 (0.0010) [2023-10-13 22:46:12,889][60935] Updated weights for policy 0, policy_version 44250 (0.0009) [2023-10-13 22:46:14,589][60934] Updated weights for policy 1, policy_version 44572 (0.0007) [2023-10-13 22:46:14,961][60934] Updated weights for policy 1, policy_version 44582 (0.0009) [2023-10-13 22:46:15,326][60934] Updated weights for policy 1, policy_version 44592 (0.0007) [2023-10-13 22:46:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 91291648. Throughput: 0: 1682.5, 1: 1728.5. Samples: 22825748. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-13 22:46:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:46:16,906][60935] Updated weights for policy 0, policy_version 44260 (0.0010) [2023-10-13 22:46:17,272][60935] Updated weights for policy 0, policy_version 44270 (0.0010) [2023-10-13 22:46:17,637][60935] Updated weights for policy 0, policy_version 44280 (0.0010) [2023-10-13 22:46:19,381][60934] Updated weights for policy 1, policy_version 44602 (0.0008) [2023-10-13 22:46:19,739][60934] Updated weights for policy 1, policy_version 44612 (0.0007) [2023-10-13 22:46:20,114][60934] Updated weights for policy 1, policy_version 44622 (0.0010) [2023-10-13 22:46:20,484][60934] Updated weights for policy 1, policy_version 44632 (0.0009) [2023-10-13 22:46:21,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 91357184. Throughput: 0: 1698.1, 1: 1719.3. Samples: 22846602. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-13 22:46:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:46:21,604][60935] Updated weights for policy 0, policy_version 44290 (0.0008) [2023-10-13 22:46:21,977][60935] Updated weights for policy 0, policy_version 44300 (0.0010) [2023-10-13 22:46:22,344][60935] Updated weights for policy 0, policy_version 44310 (0.0008) [2023-10-13 22:46:22,716][60935] Updated weights for policy 0, policy_version 44320 (0.0009) [2023-10-13 22:46:24,564][60934] Updated weights for policy 1, policy_version 44642 (0.0009) [2023-10-13 22:46:24,937][60934] Updated weights for policy 1, policy_version 44652 (0.0009) [2023-10-13 22:46:25,303][60934] Updated weights for policy 1, policy_version 44662 (0.0008) [2023-10-13 22:46:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 91422720. Throughput: 0: 1698.9, 1: 1692.8. Samples: 22866244. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-13 22:46:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:46:26,822][60935] Updated weights for policy 0, policy_version 44330 (0.0012) [2023-10-13 22:46:27,184][60935] Updated weights for policy 0, policy_version 44340 (0.0009) [2023-10-13 22:46:27,563][60935] Updated weights for policy 0, policy_version 44350 (0.0008) [2023-10-13 22:46:29,192][60934] Updated weights for policy 1, policy_version 44672 (0.0010) [2023-10-13 22:46:29,558][60934] Updated weights for policy 1, policy_version 44682 (0.0010) [2023-10-13 22:46:29,928][60934] Updated weights for policy 1, policy_version 44692 (0.0008) [2023-10-13 22:46:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 91488256. Throughput: 0: 1690.7, 1: 1725.3. Samples: 22876836. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-13 22:46:31,248][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:46:31,340][60935] Updated weights for policy 0, policy_version 44360 (0.0009) [2023-10-13 22:46:31,705][60935] Updated weights for policy 0, policy_version 44370 (0.0008) [2023-10-13 22:46:32,079][60935] Updated weights for policy 0, policy_version 44380 (0.0008) [2023-10-13 22:46:33,959][60934] Updated weights for policy 1, policy_version 44702 (0.0010) [2023-10-13 22:46:34,324][60934] Updated weights for policy 1, policy_version 44712 (0.0007) [2023-10-13 22:46:34,691][60934] Updated weights for policy 1, policy_version 44722 (0.0008) [2023-10-13 22:46:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 91553792. Throughput: 0: 1704.1, 1: 1701.6. Samples: 22897152. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-13 22:46:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:46:36,310][60935] Updated weights for policy 0, policy_version 44390 (0.0007) [2023-10-13 22:46:36,665][60935] Updated weights for policy 0, policy_version 44400 (0.0007) [2023-10-13 22:46:37,041][60935] Updated weights for policy 0, policy_version 44410 (0.0007) [2023-10-13 22:46:38,555][60934] Updated weights for policy 1, policy_version 44732 (0.0007) [2023-10-13 22:46:38,929][60934] Updated weights for policy 1, policy_version 44742 (0.0008) [2023-10-13 22:46:39,290][60934] Updated weights for policy 1, policy_version 44752 (0.0007) [2023-10-13 22:46:41,103][60935] Updated weights for policy 0, policy_version 44420 (0.0008) [2023-10-13 22:46:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 91619328. Throughput: 0: 1695.2, 1: 1697.8. Samples: 22917730. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-13 22:46:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:46:41,472][60935] Updated weights for policy 0, policy_version 44430 (0.0010) [2023-10-13 22:46:41,842][60935] Updated weights for policy 0, policy_version 44440 (0.0009) [2023-10-13 22:46:43,312][60934] Updated weights for policy 1, policy_version 44762 (0.0007) [2023-10-13 22:46:43,677][60934] Updated weights for policy 1, policy_version 44772 (0.0007) [2023-10-13 22:46:44,046][60934] Updated weights for policy 1, policy_version 44782 (0.0007) [2023-10-13 22:46:44,416][60934] Updated weights for policy 1, policy_version 44792 (0.0009) [2023-10-13 22:46:45,718][60935] Updated weights for policy 0, policy_version 44450 (0.0009) [2023-10-13 22:46:46,124][60935] Updated weights for policy 0, policy_version 44460 (0.0010) [2023-10-13 22:46:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 91684864. Throughput: 0: 1699.7, 1: 1715.3. Samples: 22927950. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-13 22:46:46,250][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:46:46,492][60935] Updated weights for policy 0, policy_version 44470 (0.0007) [2023-10-13 22:46:46,857][60935] Updated weights for policy 0, policy_version 44480 (0.0008) [2023-10-13 22:46:48,387][60934] Updated weights for policy 1, policy_version 44802 (0.0009) [2023-10-13 22:46:48,748][60934] Updated weights for policy 1, policy_version 44812 (0.0007) [2023-10-13 22:46:49,113][60934] Updated weights for policy 1, policy_version 44822 (0.0007) [2023-10-13 22:46:50,902][60935] Updated weights for policy 0, policy_version 44490 (0.0009) [2023-10-13 22:46:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 91750400. Throughput: 0: 1702.7, 1: 1693.5. Samples: 22948100. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-13 22:46:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:46:51,269][60935] Updated weights for policy 0, policy_version 44500 (0.0007) [2023-10-13 22:46:51,634][60935] Updated weights for policy 0, policy_version 44510 (0.0008) [2023-10-13 22:46:53,088][60934] Updated weights for policy 1, policy_version 44832 (0.0007) [2023-10-13 22:46:53,451][60934] Updated weights for policy 1, policy_version 44842 (0.0007) [2023-10-13 22:46:53,814][60934] Updated weights for policy 1, policy_version 44852 (0.0007) [2023-10-13 22:46:55,668][60935] Updated weights for policy 0, policy_version 44520 (0.0007) [2023-10-13 22:46:56,027][60935] Updated weights for policy 0, policy_version 44530 (0.0008) [2023-10-13 22:46:56,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 91815936. Throughput: 0: 1691.6, 1: 1707.7. Samples: 22968612. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-13 22:46:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:46:56,257][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000044856_46235648.pth... [2023-10-13 22:46:56,291][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000043256_44597248.pth [2023-10-13 22:46:56,404][60935] Updated weights for policy 0, policy_version 44540 (0.0008) [2023-10-13 22:46:56,556][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000044544_45613056.pth... [2023-10-13 22:46:56,585][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000042944_43974656.pth [2023-10-13 22:46:57,761][60934] Updated weights for policy 1, policy_version 44862 (0.0007) [2023-10-13 22:46:58,131][60934] Updated weights for policy 1, policy_version 44872 (0.0008) [2023-10-13 22:46:58,501][60934] Updated weights for policy 1, policy_version 44882 (0.0008) [2023-10-13 22:47:00,530][60935] Updated weights for policy 0, policy_version 44550 (0.0010) [2023-10-13 22:47:00,901][60935] Updated weights for policy 0, policy_version 44560 (0.0010) [2023-10-13 22:47:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 91881472. Throughput: 0: 1701.2, 1: 1695.5. Samples: 22978598. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-13 22:47:01,248][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:47:01,268][60935] Updated weights for policy 0, policy_version 44570 (0.0009) [2023-10-13 22:47:02,503][60934] Updated weights for policy 1, policy_version 44892 (0.0009) [2023-10-13 22:47:02,861][60934] Updated weights for policy 1, policy_version 44902 (0.0007) [2023-10-13 22:47:03,230][60934] Updated weights for policy 1, policy_version 44912 (0.0007) [2023-10-13 22:47:05,230][60935] Updated weights for policy 0, policy_version 44580 (0.0008) [2023-10-13 22:47:05,597][60935] Updated weights for policy 0, policy_version 44590 (0.0007) [2023-10-13 22:47:05,969][60935] Updated weights for policy 0, policy_version 44600 (0.0007) [2023-10-13 22:47:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 91947008. Throughput: 0: 1696.9, 1: 1698.4. Samples: 22999388. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-13 22:47:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:47:07,131][60934] Updated weights for policy 1, policy_version 44922 (0.0008) [2023-10-13 22:47:07,503][60934] Updated weights for policy 1, policy_version 44932 (0.0009) [2023-10-13 22:47:07,870][60934] Updated weights for policy 1, policy_version 44942 (0.0007) [2023-10-13 22:47:08,232][60934] Updated weights for policy 1, policy_version 44952 (0.0009) [2023-10-13 22:47:10,169][60935] Updated weights for policy 0, policy_version 44610 (0.0008) [2023-10-13 22:47:10,533][60935] Updated weights for policy 0, policy_version 44620 (0.0008) [2023-10-13 22:47:10,899][60935] Updated weights for policy 0, policy_version 44630 (0.0007) [2023-10-13 22:47:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 92012544. Throughput: 0: 1681.8, 1: 1729.4. Samples: 23019748. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-13 22:47:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:47:11,265][60935] Updated weights for policy 0, policy_version 44640 (0.0008) [2023-10-13 22:47:12,400][60934] Updated weights for policy 1, policy_version 44962 (0.0009) [2023-10-13 22:47:12,779][60934] Updated weights for policy 1, policy_version 44972 (0.0007) [2023-10-13 22:47:13,148][60934] Updated weights for policy 1, policy_version 44982 (0.0007) [2023-10-13 22:47:15,326][60935] Updated weights for policy 0, policy_version 44650 (0.0009) [2023-10-13 22:47:15,699][60935] Updated weights for policy 0, policy_version 44660 (0.0008) [2023-10-13 22:47:16,068][60935] Updated weights for policy 0, policy_version 44670 (0.0007) [2023-10-13 22:47:16,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 92110848. Throughput: 0: 1699.1, 1: 1699.9. Samples: 23029788. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 22:47:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:47:17,015][60934] Updated weights for policy 1, policy_version 44992 (0.0009) [2023-10-13 22:47:17,375][60934] Updated weights for policy 1, policy_version 45002 (0.0010) [2023-10-13 22:47:17,753][60934] Updated weights for policy 1, policy_version 45012 (0.0010) [2023-10-13 22:47:20,135][60935] Updated weights for policy 0, policy_version 44680 (0.0009) [2023-10-13 22:47:20,496][60935] Updated weights for policy 0, policy_version 44690 (0.0009) [2023-10-13 22:47:20,871][60935] Updated weights for policy 0, policy_version 44700 (0.0007) [2023-10-13 22:47:21,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 92176384. Throughput: 0: 1690.6, 1: 1723.2. Samples: 23050772. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 22:47:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:47:21,682][60934] Updated weights for policy 1, policy_version 45022 (0.0008) [2023-10-13 22:47:22,040][60934] Updated weights for policy 1, policy_version 45032 (0.0008) [2023-10-13 22:47:22,420][60934] Updated weights for policy 1, policy_version 45042 (0.0009) [2023-10-13 22:47:24,711][60935] Updated weights for policy 0, policy_version 44710 (0.0009) [2023-10-13 22:47:25,077][60935] Updated weights for policy 0, policy_version 44720 (0.0009) [2023-10-13 22:47:25,449][60935] Updated weights for policy 0, policy_version 44730 (0.0008) [2023-10-13 22:47:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 92241920. Throughput: 0: 1670.5, 1: 1734.8. Samples: 23070968. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 22:47:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:47:26,343][60934] Updated weights for policy 1, policy_version 45052 (0.0009) [2023-10-13 22:47:26,706][60934] Updated weights for policy 1, policy_version 45062 (0.0009) [2023-10-13 22:47:27,073][60934] Updated weights for policy 1, policy_version 45072 (0.0009) [2023-10-13 22:47:29,585][60935] Updated weights for policy 0, policy_version 44740 (0.0008) [2023-10-13 22:47:29,956][60935] Updated weights for policy 0, policy_version 44750 (0.0011) [2023-10-13 22:47:30,328][60935] Updated weights for policy 0, policy_version 44760 (0.0009) [2023-10-13 22:47:31,109][60934] Updated weights for policy 1, policy_version 45082 (0.0009) [2023-10-13 22:47:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 92307456. Throughput: 0: 1697.7, 1: 1709.9. Samples: 23081288. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 22:47:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:47:31,467][60934] Updated weights for policy 1, policy_version 45092 (0.0009) [2023-10-13 22:47:31,837][60934] Updated weights for policy 1, policy_version 45102 (0.0009) [2023-10-13 22:47:32,201][60934] Updated weights for policy 1, policy_version 45112 (0.0010) [2023-10-13 22:47:34,579][60935] Updated weights for policy 0, policy_version 44770 (0.0008) [2023-10-13 22:47:34,964][60935] Updated weights for policy 0, policy_version 44780 (0.0009) [2023-10-13 22:47:35,331][60935] Updated weights for policy 0, policy_version 44790 (0.0007) [2023-10-13 22:47:35,692][60935] Updated weights for policy 0, policy_version 44800 (0.0008) [2023-10-13 22:47:36,058][60934] Updated weights for policy 1, policy_version 45122 (0.0008) [2023-10-13 22:47:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 92372992. Throughput: 0: 1678.3, 1: 1735.1. Samples: 23101706. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-13 22:47:36,250][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:47:36,420][60934] Updated weights for policy 1, policy_version 45132 (0.0008) [2023-10-13 22:47:36,793][60934] Updated weights for policy 1, policy_version 45142 (0.0008) [2023-10-13 22:47:39,735][60935] Updated weights for policy 0, policy_version 44810 (0.0010) [2023-10-13 22:47:40,104][60935] Updated weights for policy 0, policy_version 44820 (0.0009) [2023-10-13 22:47:40,471][60935] Updated weights for policy 0, policy_version 44830 (0.0009) [2023-10-13 22:47:40,750][60934] Updated weights for policy 1, policy_version 45152 (0.0009) [2023-10-13 22:47:41,113][60934] Updated weights for policy 1, policy_version 45162 (0.0011) [2023-10-13 22:47:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 92438528. Throughput: 0: 1669.6, 1: 1731.8. Samples: 23121676. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-13 22:47:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:47:41,492][60934] Updated weights for policy 1, policy_version 45172 (0.0009) [2023-10-13 22:47:44,499][60935] Updated weights for policy 0, policy_version 44840 (0.0009) [2023-10-13 22:47:44,868][60935] Updated weights for policy 0, policy_version 44850 (0.0011) [2023-10-13 22:47:45,233][60935] Updated weights for policy 0, policy_version 44860 (0.0009) [2023-10-13 22:47:45,360][60934] Updated weights for policy 1, policy_version 45182 (0.0007) [2023-10-13 22:47:45,728][60934] Updated weights for policy 1, policy_version 45192 (0.0007) [2023-10-13 22:47:46,099][60934] Updated weights for policy 1, policy_version 45202 (0.0008) [2023-10-13 22:47:46,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 92504064. Throughput: 0: 1687.4, 1: 1725.8. Samples: 23132194. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-13 22:47:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:47:49,378][60935] Updated weights for policy 0, policy_version 44870 (0.0010) [2023-10-13 22:47:49,762][60935] Updated weights for policy 0, policy_version 44880 (0.0010) [2023-10-13 22:47:50,125][60935] Updated weights for policy 0, policy_version 44890 (0.0011) [2023-10-13 22:47:50,207][60934] Updated weights for policy 1, policy_version 45212 (0.0007) [2023-10-13 22:47:50,574][60934] Updated weights for policy 1, policy_version 45222 (0.0008) [2023-10-13 22:47:50,945][60934] Updated weights for policy 1, policy_version 45232 (0.0008) [2023-10-13 22:47:51,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 92602368. Throughput: 0: 1666.2, 1: 1735.6. Samples: 23152470. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-13 22:47:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:47:54,078][60935] Updated weights for policy 0, policy_version 44900 (0.0008) [2023-10-13 22:47:54,447][60935] Updated weights for policy 0, policy_version 44910 (0.0008) [2023-10-13 22:47:54,819][60935] Updated weights for policy 0, policy_version 44920 (0.0007) [2023-10-13 22:47:54,877][60934] Updated weights for policy 1, policy_version 45242 (0.0007) [2023-10-13 22:47:55,240][60934] Updated weights for policy 1, policy_version 45252 (0.0008) [2023-10-13 22:47:55,608][60934] Updated weights for policy 1, policy_version 45262 (0.0007) [2023-10-13 22:47:55,968][60934] Updated weights for policy 1, policy_version 45272 (0.0007) [2023-10-13 22:47:56,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 92667904. Throughput: 0: 1675.9, 1: 1714.2. Samples: 23172306. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-13 22:47:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:47:58,849][60935] Updated weights for policy 0, policy_version 44930 (0.0009) [2023-10-13 22:47:59,218][60935] Updated weights for policy 0, policy_version 44940 (0.0009) [2023-10-13 22:47:59,593][60935] Updated weights for policy 0, policy_version 44950 (0.0008) [2023-10-13 22:47:59,961][60935] Updated weights for policy 0, policy_version 44960 (0.0010) [2023-10-13 22:48:00,084][60934] Updated weights for policy 1, policy_version 45282 (0.0009) [2023-10-13 22:48:00,459][60934] Updated weights for policy 1, policy_version 45292 (0.0008) [2023-10-13 22:48:00,837][60934] Updated weights for policy 1, policy_version 45302 (0.0011) [2023-10-13 22:48:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 92733440. Throughput: 0: 1682.8, 1: 1730.8. Samples: 23183404. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-13 22:48:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:48:03,972][60935] Updated weights for policy 0, policy_version 44970 (0.0010) [2023-10-13 22:48:04,353][60935] Updated weights for policy 0, policy_version 44980 (0.0010) [2023-10-13 22:48:04,624][60934] Updated weights for policy 1, policy_version 45312 (0.0007) [2023-10-13 22:48:04,713][60935] Updated weights for policy 0, policy_version 44990 (0.0008) [2023-10-13 22:48:04,995][60934] Updated weights for policy 1, policy_version 45322 (0.0008) [2023-10-13 22:48:05,352][60934] Updated weights for policy 1, policy_version 45332 (0.0009) [2023-10-13 22:48:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 92798976. Throughput: 0: 1660.5, 1: 1722.3. Samples: 23203000. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-13 22:48:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:48:08,850][60935] Updated weights for policy 0, policy_version 45000 (0.0008) [2023-10-13 22:48:09,196][60934] Updated weights for policy 1, policy_version 45342 (0.0010) [2023-10-13 22:48:09,220][60935] Updated weights for policy 0, policy_version 45010 (0.0007) [2023-10-13 22:48:09,563][60934] Updated weights for policy 1, policy_version 45352 (0.0007) [2023-10-13 22:48:09,584][60935] Updated weights for policy 0, policy_version 45020 (0.0008) [2023-10-13 22:48:09,915][60934] Updated weights for policy 1, policy_version 45362 (0.0008) [2023-10-13 22:48:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 92864512. Throughput: 0: 1683.1, 1: 1696.9. Samples: 23223068. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-13 22:48:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:48:13,714][60935] Updated weights for policy 0, policy_version 45030 (0.0008) [2023-10-13 22:48:13,896][60934] Updated weights for policy 1, policy_version 45372 (0.0007) [2023-10-13 22:48:14,089][60935] Updated weights for policy 0, policy_version 45040 (0.0009) [2023-10-13 22:48:14,258][60934] Updated weights for policy 1, policy_version 45382 (0.0007) [2023-10-13 22:48:14,474][60935] Updated weights for policy 0, policy_version 45050 (0.0008) [2023-10-13 22:48:14,621][60934] Updated weights for policy 1, policy_version 45392 (0.0008) [2023-10-13 22:48:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 92930048. Throughput: 0: 1673.5, 1: 1731.3. Samples: 23234504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:48:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:48:18,592][60934] Updated weights for policy 1, policy_version 45402 (0.0007) [2023-10-13 22:48:18,612][60935] Updated weights for policy 0, policy_version 45060 (0.0007) [2023-10-13 22:48:18,954][60934] Updated weights for policy 1, policy_version 45412 (0.0007) [2023-10-13 22:48:18,988][60935] Updated weights for policy 0, policy_version 45070 (0.0009) [2023-10-13 22:48:19,310][60934] Updated weights for policy 1, policy_version 45422 (0.0010) [2023-10-13 22:48:19,358][60935] Updated weights for policy 0, policy_version 45080 (0.0009) [2023-10-13 22:48:19,675][60934] Updated weights for policy 1, policy_version 45432 (0.0007) [2023-10-13 22:48:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 92995584. Throughput: 0: 1666.8, 1: 1704.9. Samples: 23253432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:48:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:48:23,389][60935] Updated weights for policy 0, policy_version 45090 (0.0007) [2023-10-13 22:48:23,627][60934] Updated weights for policy 1, policy_version 45442 (0.0008) [2023-10-13 22:48:23,783][60935] Updated weights for policy 0, policy_version 45100 (0.0009) [2023-10-13 22:48:23,988][60934] Updated weights for policy 1, policy_version 45452 (0.0007) [2023-10-13 22:48:24,152][60935] Updated weights for policy 0, policy_version 45110 (0.0010) [2023-10-13 22:48:24,358][60934] Updated weights for policy 1, policy_version 45462 (0.0008) [2023-10-13 22:48:24,529][60935] Updated weights for policy 0, policy_version 45120 (0.0009) [2023-10-13 22:48:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 93061120. Throughput: 0: 1689.6, 1: 1705.1. Samples: 23274440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:48:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:48:28,302][60935] Updated weights for policy 0, policy_version 45130 (0.0010) [2023-10-13 22:48:28,450][60934] Updated weights for policy 1, policy_version 45472 (0.0007) [2023-10-13 22:48:28,678][60935] Updated weights for policy 0, policy_version 45140 (0.0008) [2023-10-13 22:48:28,808][60934] Updated weights for policy 1, policy_version 45482 (0.0007) [2023-10-13 22:48:29,041][60935] Updated weights for policy 0, policy_version 45150 (0.0008) [2023-10-13 22:48:29,181][60934] Updated weights for policy 1, policy_version 45492 (0.0008) [2023-10-13 22:48:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 93126656. Throughput: 0: 1669.9, 1: 1720.0. Samples: 23284738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:48:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:48:33,107][60935] Updated weights for policy 0, policy_version 45160 (0.0009) [2023-10-13 22:48:33,344][60934] Updated weights for policy 1, policy_version 45502 (0.0010) [2023-10-13 22:48:33,486][60935] Updated weights for policy 0, policy_version 45170 (0.0008) [2023-10-13 22:48:33,714][60934] Updated weights for policy 1, policy_version 45512 (0.0008) [2023-10-13 22:48:33,853][60935] Updated weights for policy 0, policy_version 45180 (0.0008) [2023-10-13 22:48:34,084][60934] Updated weights for policy 1, policy_version 45522 (0.0007) [2023-10-13 22:48:36,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 93192192. Throughput: 0: 1680.0, 1: 1691.6. Samples: 23304196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:48:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:48:37,951][60935] Updated weights for policy 0, policy_version 45190 (0.0007) [2023-10-13 22:48:38,167][60934] Updated weights for policy 1, policy_version 45532 (0.0008) [2023-10-13 22:48:38,310][60935] Updated weights for policy 0, policy_version 45200 (0.0008) [2023-10-13 22:48:38,524][60934] Updated weights for policy 1, policy_version 45542 (0.0007) [2023-10-13 22:48:38,686][60935] Updated weights for policy 0, policy_version 45210 (0.0008) [2023-10-13 22:48:38,892][60934] Updated weights for policy 1, policy_version 45552 (0.0007) [2023-10-13 22:48:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 93257728. Throughput: 0: 1685.5, 1: 1705.5. Samples: 23324900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:48:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:48:42,783][60935] Updated weights for policy 0, policy_version 45220 (0.0009) [2023-10-13 22:48:42,869][60934] Updated weights for policy 1, policy_version 45562 (0.0008) [2023-10-13 22:48:43,151][60935] Updated weights for policy 0, policy_version 45230 (0.0009) [2023-10-13 22:48:43,229][60934] Updated weights for policy 1, policy_version 45572 (0.0008) [2023-10-13 22:48:43,520][60935] Updated weights for policy 0, policy_version 45240 (0.0010) [2023-10-13 22:48:43,591][60934] Updated weights for policy 1, policy_version 45582 (0.0008) [2023-10-13 22:48:43,960][60934] Updated weights for policy 1, policy_version 45592 (0.0009) [2023-10-13 22:48:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 93323264. Throughput: 0: 1660.5, 1: 1700.4. Samples: 23334646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:48:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:48:47,723][60935] Updated weights for policy 0, policy_version 45250 (0.0009) [2023-10-13 22:48:48,016][60934] Updated weights for policy 1, policy_version 45602 (0.0007) [2023-10-13 22:48:48,099][60935] Updated weights for policy 0, policy_version 45260 (0.0008) [2023-10-13 22:48:48,386][60934] Updated weights for policy 1, policy_version 45612 (0.0009) [2023-10-13 22:48:48,464][60935] Updated weights for policy 0, policy_version 45270 (0.0009) [2023-10-13 22:48:48,755][60934] Updated weights for policy 1, policy_version 45622 (0.0007) [2023-10-13 22:48:48,833][60935] Updated weights for policy 0, policy_version 45280 (0.0007) [2023-10-13 22:48:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 93388800. Throughput: 0: 1681.5, 1: 1690.6. Samples: 23354744. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) [2023-10-13 22:48:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:48:52,705][60934] Updated weights for policy 1, policy_version 45632 (0.0009) [2023-10-13 22:48:52,791][60935] Updated weights for policy 0, policy_version 45290 (0.0007) [2023-10-13 22:48:53,071][60934] Updated weights for policy 1, policy_version 45642 (0.0009) [2023-10-13 22:48:53,159][60935] Updated weights for policy 0, policy_version 45300 (0.0008) [2023-10-13 22:48:53,426][60934] Updated weights for policy 1, policy_version 45652 (0.0009) [2023-10-13 22:48:53,520][60935] Updated weights for policy 0, policy_version 45310 (0.0007) [2023-10-13 22:48:56,249][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 93454336. Throughput: 0: 1680.3, 1: 1711.6. Samples: 23375702. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) [2023-10-13 22:48:56,250][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:48:56,263][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000045656_47054848.pth... [2023-10-13 22:48:56,263][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000045312_46399488.pth... [2023-10-13 22:48:56,297][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000044056_45416448.pth [2023-10-13 22:48:56,303][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000043744_44793856.pth [2023-10-13 22:48:57,424][60934] Updated weights for policy 1, policy_version 45662 (0.0009) [2023-10-13 22:48:57,689][60935] Updated weights for policy 0, policy_version 45320 (0.0008) [2023-10-13 22:48:57,791][60934] Updated weights for policy 1, policy_version 45672 (0.0007) [2023-10-13 22:48:58,056][60935] Updated weights for policy 0, policy_version 45330 (0.0008) [2023-10-13 22:48:58,146][60934] Updated weights for policy 1, policy_version 45682 (0.0009) [2023-10-13 22:48:58,414][60935] Updated weights for policy 0, policy_version 45340 (0.0010) [2023-10-13 22:49:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 93519872. Throughput: 0: 1660.6, 1: 1678.7. Samples: 23384774. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) [2023-10-13 22:49:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:49:02,281][60934] Updated weights for policy 1, policy_version 45692 (0.0009) [2023-10-13 22:49:02,608][60935] Updated weights for policy 0, policy_version 45350 (0.0007) [2023-10-13 22:49:02,645][60934] Updated weights for policy 1, policy_version 45702 (0.0008) [2023-10-13 22:49:02,981][60935] Updated weights for policy 0, policy_version 45360 (0.0007) [2023-10-13 22:49:03,010][60934] Updated weights for policy 1, policy_version 45712 (0.0007) [2023-10-13 22:49:03,352][60935] Updated weights for policy 0, policy_version 45370 (0.0008) [2023-10-13 22:49:06,248][59943] Fps is (10 sec: 13107.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 93585408. Throughput: 0: 1675.6, 1: 1702.0. Samples: 23405424. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) [2023-10-13 22:49:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:49:07,009][60934] Updated weights for policy 1, policy_version 45722 (0.0007) [2023-10-13 22:49:07,372][60934] Updated weights for policy 1, policy_version 45732 (0.0010) [2023-10-13 22:49:07,458][60935] Updated weights for policy 0, policy_version 45380 (0.0008) [2023-10-13 22:49:07,738][60934] Updated weights for policy 1, policy_version 45742 (0.0008) [2023-10-13 22:49:07,836][60935] Updated weights for policy 0, policy_version 45390 (0.0008) [2023-10-13 22:49:08,106][60934] Updated weights for policy 1, policy_version 45752 (0.0008) [2023-10-13 22:49:08,205][60935] Updated weights for policy 0, policy_version 45400 (0.0009) [2023-10-13 22:49:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 93650944. Throughput: 0: 1675.2, 1: 1702.0. Samples: 23426414. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) [2023-10-13 22:49:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:49:12,087][60934] Updated weights for policy 1, policy_version 45762 (0.0008) [2023-10-13 22:49:12,234][60935] Updated weights for policy 0, policy_version 45410 (0.0009) [2023-10-13 22:49:12,450][60934] Updated weights for policy 1, policy_version 45772 (0.0007) [2023-10-13 22:49:12,617][60935] Updated weights for policy 0, policy_version 45420 (0.0008) [2023-10-13 22:49:12,809][60934] Updated weights for policy 1, policy_version 45782 (0.0008) [2023-10-13 22:49:12,984][60935] Updated weights for policy 0, policy_version 45430 (0.0010) [2023-10-13 22:49:13,346][60935] Updated weights for policy 0, policy_version 45440 (0.0011) [2023-10-13 22:49:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 93716480. Throughput: 0: 1666.1, 1: 1686.2. Samples: 23435594. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) [2023-10-13 22:49:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:49:16,776][60934] Updated weights for policy 1, policy_version 45792 (0.0008) [2023-10-13 22:49:17,137][60934] Updated weights for policy 1, policy_version 45802 (0.0008) [2023-10-13 22:49:17,367][60935] Updated weights for policy 0, policy_version 45450 (0.0007) [2023-10-13 22:49:17,510][60934] Updated weights for policy 1, policy_version 45812 (0.0008) [2023-10-13 22:49:17,732][60935] Updated weights for policy 0, policy_version 45460 (0.0009) [2023-10-13 22:49:18,103][60935] Updated weights for policy 0, policy_version 45470 (0.0008) [2023-10-13 22:49:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 93782016. Throughput: 0: 1677.0, 1: 1710.5. Samples: 23456636. Policy #0 lag: (min: 8.0, avg: 34.2, max: 40.0) [2023-10-13 22:49:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:49:21,458][60934] Updated weights for policy 1, policy_version 45822 (0.0009) [2023-10-13 22:49:21,832][60934] Updated weights for policy 1, policy_version 45832 (0.0010) [2023-10-13 22:49:22,074][60935] Updated weights for policy 0, policy_version 45480 (0.0008) [2023-10-13 22:49:22,201][60934] Updated weights for policy 1, policy_version 45842 (0.0009) [2023-10-13 22:49:22,429][60935] Updated weights for policy 0, policy_version 45490 (0.0009) [2023-10-13 22:49:22,801][60935] Updated weights for policy 0, policy_version 45500 (0.0010) [2023-10-13 22:49:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 93847552. Throughput: 0: 1677.9, 1: 1715.9. Samples: 23477620. Policy #0 lag: (min: 8.0, avg: 34.2, max: 40.0) [2023-10-13 22:49:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:49:26,263][60934] Updated weights for policy 1, policy_version 45852 (0.0009) [2023-10-13 22:49:26,632][60934] Updated weights for policy 1, policy_version 45862 (0.0008) [2023-10-13 22:49:26,848][60935] Updated weights for policy 0, policy_version 45510 (0.0009) [2023-10-13 22:49:26,996][60934] Updated weights for policy 1, policy_version 45872 (0.0009) [2023-10-13 22:49:27,227][60935] Updated weights for policy 0, policy_version 45520 (0.0008) [2023-10-13 22:49:27,602][60935] Updated weights for policy 0, policy_version 45530 (0.0009) [2023-10-13 22:49:30,866][60934] Updated weights for policy 1, policy_version 45882 (0.0009) [2023-10-13 22:49:31,241][60934] Updated weights for policy 1, policy_version 45892 (0.0009) [2023-10-13 22:49:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 93913088. Throughput: 0: 1676.7, 1: 1703.4. Samples: 23486750. Policy #0 lag: (min: 8.0, avg: 34.2, max: 40.0) [2023-10-13 22:49:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:49:31,604][60934] Updated weights for policy 1, policy_version 45902 (0.0009) [2023-10-13 22:49:31,702][60935] Updated weights for policy 0, policy_version 45540 (0.0008) [2023-10-13 22:49:31,966][60934] Updated weights for policy 1, policy_version 45912 (0.0009) [2023-10-13 22:49:32,073][60935] Updated weights for policy 0, policy_version 45550 (0.0009) [2023-10-13 22:49:32,447][60935] Updated weights for policy 0, policy_version 45560 (0.0010) [2023-10-13 22:49:36,191][60934] Updated weights for policy 1, policy_version 45922 (0.0007) [2023-10-13 22:49:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 93978624. Throughput: 0: 1682.3, 1: 1721.1. Samples: 23507896. Policy #0 lag: (min: 8.0, avg: 34.2, max: 40.0) [2023-10-13 22:49:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:49:36,502][60935] Updated weights for policy 0, policy_version 45570 (0.0009) [2023-10-13 22:49:36,564][60934] Updated weights for policy 1, policy_version 45932 (0.0007) [2023-10-13 22:49:36,874][60935] Updated weights for policy 0, policy_version 45580 (0.0008) [2023-10-13 22:49:36,933][60934] Updated weights for policy 1, policy_version 45942 (0.0008) [2023-10-13 22:49:37,250][60935] Updated weights for policy 0, policy_version 45590 (0.0009) [2023-10-13 22:49:37,614][60935] Updated weights for policy 0, policy_version 45600 (0.0009) [2023-10-13 22:49:40,845][60934] Updated weights for policy 1, policy_version 45952 (0.0009) [2023-10-13 22:49:41,217][60934] Updated weights for policy 1, policy_version 45962 (0.0008) [2023-10-13 22:49:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 94044160. Throughput: 0: 1687.9, 1: 1713.6. Samples: 23528768. Policy #0 lag: (min: 8.0, avg: 34.2, max: 40.0) [2023-10-13 22:49:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:49:41,572][60934] Updated weights for policy 1, policy_version 45972 (0.0008) [2023-10-13 22:49:41,606][60935] Updated weights for policy 0, policy_version 45610 (0.0008) [2023-10-13 22:49:41,964][60935] Updated weights for policy 0, policy_version 45620 (0.0008) [2023-10-13 22:49:42,332][60935] Updated weights for policy 0, policy_version 45630 (0.0007) [2023-10-13 22:49:45,605][60934] Updated weights for policy 1, policy_version 45982 (0.0009) [2023-10-13 22:49:45,970][60934] Updated weights for policy 1, policy_version 45992 (0.0009) [2023-10-13 22:49:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 94109696. Throughput: 0: 1692.2, 1: 1716.6. Samples: 23538170. Policy #0 lag: (min: 8.0, avg: 34.2, max: 40.0) [2023-10-13 22:49:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:49:46,282][60935] Updated weights for policy 0, policy_version 45640 (0.0009) [2023-10-13 22:49:46,334][60934] Updated weights for policy 1, policy_version 46002 (0.0008) [2023-10-13 22:49:46,645][60935] Updated weights for policy 0, policy_version 45650 (0.0010) [2023-10-13 22:49:47,012][60935] Updated weights for policy 0, policy_version 45660 (0.0009) [2023-10-13 22:49:50,258][60934] Updated weights for policy 1, policy_version 46012 (0.0008) [2023-10-13 22:49:50,626][60934] Updated weights for policy 1, policy_version 46022 (0.0010) [2023-10-13 22:49:50,986][60934] Updated weights for policy 1, policy_version 46032 (0.0009) [2023-10-13 22:49:51,022][60935] Updated weights for policy 0, policy_version 45670 (0.0007) [2023-10-13 22:49:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 94175232. Throughput: 0: 1698.7, 1: 1716.7. Samples: 23559118. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 22:49:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:49:51,399][60935] Updated weights for policy 0, policy_version 45680 (0.0009) [2023-10-13 22:49:51,766][60935] Updated weights for policy 0, policy_version 45690 (0.0007) [2023-10-13 22:49:54,950][60934] Updated weights for policy 1, policy_version 46042 (0.0010) [2023-10-13 22:49:55,316][60934] Updated weights for policy 1, policy_version 46052 (0.0009) [2023-10-13 22:49:55,689][60934] Updated weights for policy 1, policy_version 46062 (0.0009) [2023-10-13 22:49:55,800][60935] Updated weights for policy 0, policy_version 45700 (0.0009) [2023-10-13 22:49:56,055][60934] Updated weights for policy 1, policy_version 46072 (0.0008) [2023-10-13 22:49:56,171][60935] Updated weights for policy 0, policy_version 45710 (0.0007) [2023-10-13 22:49:56,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 94273536. Throughput: 0: 1694.8, 1: 1698.4. Samples: 23579108. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 22:49:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:49:56,547][60935] Updated weights for policy 0, policy_version 45720 (0.0007) [2023-10-13 22:49:59,985][60934] Updated weights for policy 1, policy_version 46082 (0.0011) [2023-10-13 22:50:00,350][60934] Updated weights for policy 1, policy_version 46092 (0.0007) [2023-10-13 22:50:00,717][60934] Updated weights for policy 1, policy_version 46102 (0.0007) [2023-10-13 22:50:00,782][60935] Updated weights for policy 0, policy_version 45730 (0.0008) [2023-10-13 22:50:01,181][60935] Updated weights for policy 0, policy_version 45740 (0.0008) [2023-10-13 22:50:01,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 94339072. Throughput: 0: 1697.9, 1: 1715.0. Samples: 23589174. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 22:50:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:50:01,565][60935] Updated weights for policy 0, policy_version 45750 (0.0009) [2023-10-13 22:50:01,932][60935] Updated weights for policy 0, policy_version 45760 (0.0010) [2023-10-13 22:50:04,717][60934] Updated weights for policy 1, policy_version 46112 (0.0008) [2023-10-13 22:50:05,092][60934] Updated weights for policy 1, policy_version 46122 (0.0007) [2023-10-13 22:50:05,453][60934] Updated weights for policy 1, policy_version 46132 (0.0007) [2023-10-13 22:50:06,028][60935] Updated weights for policy 0, policy_version 45770 (0.0011) [2023-10-13 22:50:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 94404608. Throughput: 0: 1695.7, 1: 1712.8. Samples: 23610020. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 22:50:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:50:06,391][60935] Updated weights for policy 0, policy_version 45780 (0.0011) [2023-10-13 22:50:06,762][60935] Updated weights for policy 0, policy_version 45790 (0.0011) [2023-10-13 22:50:09,647][60934] Updated weights for policy 1, policy_version 46142 (0.0008) [2023-10-13 22:50:10,011][60934] Updated weights for policy 1, policy_version 46152 (0.0007) [2023-10-13 22:50:10,388][60934] Updated weights for policy 1, policy_version 46162 (0.0009) [2023-10-13 22:50:10,782][60935] Updated weights for policy 0, policy_version 45800 (0.0009) [2023-10-13 22:50:11,163][60935] Updated weights for policy 0, policy_version 45810 (0.0011) [2023-10-13 22:50:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 94470144. Throughput: 0: 1691.8, 1: 1690.3. Samples: 23629812. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 22:50:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:50:11,542][60935] Updated weights for policy 0, policy_version 45820 (0.0011) [2023-10-13 22:50:14,312][60934] Updated weights for policy 1, policy_version 46172 (0.0008) [2023-10-13 22:50:14,673][60934] Updated weights for policy 1, policy_version 46182 (0.0008) [2023-10-13 22:50:15,038][60934] Updated weights for policy 1, policy_version 46192 (0.0007) [2023-10-13 22:50:15,568][60935] Updated weights for policy 0, policy_version 45830 (0.0009) [2023-10-13 22:50:15,939][60935] Updated weights for policy 0, policy_version 45840 (0.0009) [2023-10-13 22:50:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 94535680. Throughput: 0: 1698.0, 1: 1719.7. Samples: 23640546. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 22:50:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:50:16,311][60935] Updated weights for policy 0, policy_version 45850 (0.0010) [2023-10-13 22:50:18,973][60934] Updated weights for policy 1, policy_version 46202 (0.0009) [2023-10-13 22:50:19,333][60934] Updated weights for policy 1, policy_version 46212 (0.0007) [2023-10-13 22:50:19,703][60934] Updated weights for policy 1, policy_version 46222 (0.0007) [2023-10-13 22:50:20,069][60934] Updated weights for policy 1, policy_version 46232 (0.0009) [2023-10-13 22:50:20,302][60935] Updated weights for policy 0, policy_version 45860 (0.0010) [2023-10-13 22:50:20,668][60935] Updated weights for policy 0, policy_version 45870 (0.0008) [2023-10-13 22:50:21,039][60935] Updated weights for policy 0, policy_version 45880 (0.0007) [2023-10-13 22:50:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 94601216. Throughput: 0: 1699.5, 1: 1704.6. Samples: 23661080. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 22:50:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:50:23,971][60934] Updated weights for policy 1, policy_version 46242 (0.0007) [2023-10-13 22:50:24,338][60934] Updated weights for policy 1, policy_version 46252 (0.0008) [2023-10-13 22:50:24,699][60934] Updated weights for policy 1, policy_version 46262 (0.0007) [2023-10-13 22:50:25,025][60935] Updated weights for policy 0, policy_version 45890 (0.0009) [2023-10-13 22:50:25,391][60935] Updated weights for policy 0, policy_version 45900 (0.0008) [2023-10-13 22:50:25,769][60935] Updated weights for policy 0, policy_version 45910 (0.0009) [2023-10-13 22:50:26,137][60935] Updated weights for policy 0, policy_version 45920 (0.0008) [2023-10-13 22:50:26,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 94699520. Throughput: 0: 1679.0, 1: 1697.9. Samples: 23680730. Policy #0 lag: (min: 2.0, avg: 4.6, max: 30.0) [2023-10-13 22:50:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:50:28,728][60934] Updated weights for policy 1, policy_version 46272 (0.0007) [2023-10-13 22:50:29,085][60934] Updated weights for policy 1, policy_version 46282 (0.0007) [2023-10-13 22:50:29,464][60934] Updated weights for policy 1, policy_version 46292 (0.0009) [2023-10-13 22:50:30,329][60935] Updated weights for policy 0, policy_version 45930 (0.0009) [2023-10-13 22:50:30,699][60935] Updated weights for policy 0, policy_version 45940 (0.0011) [2023-10-13 22:50:31,069][60935] Updated weights for policy 0, policy_version 45950 (0.0010) [2023-10-13 22:50:31,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 94765056. Throughput: 0: 1695.1, 1: 1719.8. Samples: 23691842. Policy #0 lag: (min: 2.0, avg: 4.6, max: 30.0) [2023-10-13 22:50:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:50:33,300][60934] Updated weights for policy 1, policy_version 46302 (0.0009) [2023-10-13 22:50:33,671][60934] Updated weights for policy 1, policy_version 46312 (0.0007) [2023-10-13 22:50:34,038][60934] Updated weights for policy 1, policy_version 46322 (0.0009) [2023-10-13 22:50:35,151][60935] Updated weights for policy 0, policy_version 45960 (0.0007) [2023-10-13 22:50:35,524][60935] Updated weights for policy 0, policy_version 45970 (0.0008) [2023-10-13 22:50:35,887][60935] Updated weights for policy 0, policy_version 45980 (0.0007) [2023-10-13 22:50:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 94830592. Throughput: 0: 1694.6, 1: 1691.6. Samples: 23711496. Policy #0 lag: (min: 2.0, avg: 4.6, max: 30.0) [2023-10-13 22:50:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:50:38,034][60934] Updated weights for policy 1, policy_version 46332 (0.0007) [2023-10-13 22:50:38,397][60934] Updated weights for policy 1, policy_version 46342 (0.0009) [2023-10-13 22:50:38,760][60934] Updated weights for policy 1, policy_version 46352 (0.0008) [2023-10-13 22:50:39,761][60935] Updated weights for policy 0, policy_version 45990 (0.0012) [2023-10-13 22:50:40,120][60935] Updated weights for policy 0, policy_version 46000 (0.0011) [2023-10-13 22:50:40,487][60935] Updated weights for policy 0, policy_version 46010 (0.0010) [2023-10-13 22:50:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 94896128. Throughput: 0: 1670.0, 1: 1713.3. Samples: 23731354. Policy #0 lag: (min: 2.0, avg: 4.6, max: 30.0) [2023-10-13 22:50:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:50:42,835][60934] Updated weights for policy 1, policy_version 46362 (0.0007) [2023-10-13 22:50:43,206][60934] Updated weights for policy 1, policy_version 46372 (0.0008) [2023-10-13 22:50:43,577][60934] Updated weights for policy 1, policy_version 46382 (0.0007) [2023-10-13 22:50:43,954][60934] Updated weights for policy 1, policy_version 46392 (0.0008) [2023-10-13 22:50:44,517][60935] Updated weights for policy 0, policy_version 46020 (0.0010) [2023-10-13 22:50:44,887][60935] Updated weights for policy 0, policy_version 46030 (0.0008) [2023-10-13 22:50:45,247][60935] Updated weights for policy 0, policy_version 46040 (0.0010) [2023-10-13 22:50:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 94961664. Throughput: 0: 1696.4, 1: 1708.0. Samples: 23742374. Policy #0 lag: (min: 2.0, avg: 4.6, max: 30.0) [2023-10-13 22:50:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:50:47,896][60934] Updated weights for policy 1, policy_version 46402 (0.0007) [2023-10-13 22:50:48,266][60934] Updated weights for policy 1, policy_version 46412 (0.0009) [2023-10-13 22:50:48,639][60934] Updated weights for policy 1, policy_version 46422 (0.0009) [2023-10-13 22:50:49,338][60935] Updated weights for policy 0, policy_version 46050 (0.0012) [2023-10-13 22:50:49,740][60935] Updated weights for policy 0, policy_version 46060 (0.0010) [2023-10-13 22:50:50,109][60935] Updated weights for policy 0, policy_version 46070 (0.0007) [2023-10-13 22:50:50,485][60935] Updated weights for policy 0, policy_version 46080 (0.0010) [2023-10-13 22:50:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 95027200. Throughput: 0: 1687.6, 1: 1699.6. Samples: 23762446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-13 22:50:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:50:52,625][60934] Updated weights for policy 1, policy_version 46432 (0.0008) [2023-10-13 22:50:52,989][60934] Updated weights for policy 1, policy_version 46442 (0.0010) [2023-10-13 22:50:53,359][60934] Updated weights for policy 1, policy_version 46452 (0.0011) [2023-10-13 22:50:54,347][60935] Updated weights for policy 0, policy_version 46090 (0.0009) [2023-10-13 22:50:54,724][60935] Updated weights for policy 0, policy_version 46100 (0.0011) [2023-10-13 22:50:55,088][60935] Updated weights for policy 0, policy_version 46110 (0.0011) [2023-10-13 22:50:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 95092736. Throughput: 0: 1679.0, 1: 1726.3. Samples: 23783050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-13 22:50:56,249][59943] Avg episode reward: [(0, '-0.240'), (1, '-0.280')] [2023-10-13 22:50:56,257][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000046112_47218688.pth... [2023-10-13 22:50:56,257][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000046456_47874048.pth... [2023-10-13 22:50:56,298][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000044544_45613056.pth [2023-10-13 22:50:56,299][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000044856_46235648.pth [2023-10-13 22:50:57,373][60934] Updated weights for policy 1, policy_version 46462 (0.0009) [2023-10-13 22:50:57,736][60934] Updated weights for policy 1, policy_version 46472 (0.0008) [2023-10-13 22:50:58,102][60934] Updated weights for policy 1, policy_version 46482 (0.0008) [2023-10-13 22:50:59,063][60935] Updated weights for policy 0, policy_version 46120 (0.0008) [2023-10-13 22:50:59,430][60935] Updated weights for policy 0, policy_version 46130 (0.0010) [2023-10-13 22:50:59,795][60935] Updated weights for policy 0, policy_version 46140 (0.0010) [2023-10-13 22:51:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 95158272. Throughput: 0: 1702.7, 1: 1697.2. Samples: 23793542. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-13 22:51:01,248][59943] Avg episode reward: [(0, '-0.240'), (1, '-0.280')] [2023-10-13 22:51:02,181][60934] Updated weights for policy 1, policy_version 46492 (0.0008) [2023-10-13 22:51:02,541][60934] Updated weights for policy 1, policy_version 46502 (0.0010) [2023-10-13 22:51:02,907][60934] Updated weights for policy 1, policy_version 46512 (0.0007) [2023-10-13 22:51:04,017][60935] Updated weights for policy 0, policy_version 46150 (0.0011) [2023-10-13 22:51:04,395][60935] Updated weights for policy 0, policy_version 46160 (0.0009) [2023-10-13 22:51:04,764][60935] Updated weights for policy 0, policy_version 46170 (0.0009) [2023-10-13 22:51:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 95223808. Throughput: 0: 1670.9, 1: 1709.7. Samples: 23813208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-13 22:51:06,249][59943] Avg episode reward: [(0, '-0.240'), (1, '-0.280')] [2023-10-13 22:51:06,882][60934] Updated weights for policy 1, policy_version 46522 (0.0008) [2023-10-13 22:51:07,242][60934] Updated weights for policy 1, policy_version 46532 (0.0008) [2023-10-13 22:51:07,601][60934] Updated weights for policy 1, policy_version 46542 (0.0009) [2023-10-13 22:51:07,970][60934] Updated weights for policy 1, policy_version 46552 (0.0007) [2023-10-13 22:51:08,765][60935] Updated weights for policy 0, policy_version 46180 (0.0008) [2023-10-13 22:51:09,127][60935] Updated weights for policy 0, policy_version 46190 (0.0007) [2023-10-13 22:51:09,500][60935] Updated weights for policy 0, policy_version 46200 (0.0007) [2023-10-13 22:51:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 95289344. Throughput: 0: 1681.0, 1: 1724.4. Samples: 23833976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-13 22:51:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:51:11,882][60934] Updated weights for policy 1, policy_version 46562 (0.0007) [2023-10-13 22:51:12,249][60934] Updated weights for policy 1, policy_version 46572 (0.0007) [2023-10-13 22:51:12,612][60934] Updated weights for policy 1, policy_version 46582 (0.0009) [2023-10-13 22:51:13,530][60935] Updated weights for policy 0, policy_version 46210 (0.0008) [2023-10-13 22:51:13,906][60935] Updated weights for policy 0, policy_version 46220 (0.0011) [2023-10-13 22:51:14,274][60935] Updated weights for policy 0, policy_version 46230 (0.0011) [2023-10-13 22:51:14,646][60935] Updated weights for policy 0, policy_version 46240 (0.0011) [2023-10-13 22:51:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 95354880. Throughput: 0: 1685.7, 1: 1698.4. Samples: 23844124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-13 22:51:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:51:16,420][60934] Updated weights for policy 1, policy_version 46592 (0.0008) [2023-10-13 22:51:16,781][60934] Updated weights for policy 1, policy_version 46602 (0.0008) [2023-10-13 22:51:17,140][60934] Updated weights for policy 1, policy_version 46612 (0.0008) [2023-10-13 22:51:18,908][60935] Updated weights for policy 0, policy_version 46250 (0.0007) [2023-10-13 22:51:19,284][60935] Updated weights for policy 0, policy_version 46260 (0.0008) [2023-10-13 22:51:19,656][60935] Updated weights for policy 0, policy_version 46270 (0.0007) [2023-10-13 22:51:20,979][60934] Updated weights for policy 1, policy_version 46622 (0.0008) [2023-10-13 22:51:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 95420416. Throughput: 0: 1664.4, 1: 1734.3. Samples: 23864438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-13 22:51:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:51:21,340][60934] Updated weights for policy 1, policy_version 46632 (0.0007) [2023-10-13 22:51:21,711][60934] Updated weights for policy 1, policy_version 46642 (0.0008) [2023-10-13 22:51:23,538][60935] Updated weights for policy 0, policy_version 46280 (0.0009) [2023-10-13 22:51:23,906][60935] Updated weights for policy 0, policy_version 46290 (0.0008) [2023-10-13 22:51:24,268][60935] Updated weights for policy 0, policy_version 46300 (0.0009) [2023-10-13 22:51:25,622][60934] Updated weights for policy 1, policy_version 46652 (0.0008) [2023-10-13 22:51:25,989][60934] Updated weights for policy 1, policy_version 46662 (0.0008) [2023-10-13 22:51:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 95485952. Throughput: 0: 1697.1, 1: 1731.7. Samples: 23885648. Policy #0 lag: (min: 25.0, avg: 39.0, max: 40.0) [2023-10-13 22:51:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:51:26,348][60934] Updated weights for policy 1, policy_version 46672 (0.0008) [2023-10-13 22:51:28,193][60935] Updated weights for policy 0, policy_version 46310 (0.0010) [2023-10-13 22:51:28,565][60935] Updated weights for policy 0, policy_version 46320 (0.0011) [2023-10-13 22:51:28,937][60935] Updated weights for policy 0, policy_version 46330 (0.0010) [2023-10-13 22:51:30,382][60934] Updated weights for policy 1, policy_version 46682 (0.0009) [2023-10-13 22:51:30,744][60934] Updated weights for policy 1, policy_version 46692 (0.0009) [2023-10-13 22:51:31,108][60934] Updated weights for policy 1, policy_version 46702 (0.0010) [2023-10-13 22:51:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 95551488. Throughput: 0: 1676.4, 1: 1721.6. Samples: 23895282. Policy #0 lag: (min: 25.0, avg: 39.0, max: 40.0) [2023-10-13 22:51:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:51:31,481][60934] Updated weights for policy 1, policy_version 46712 (0.0009) [2023-10-13 22:51:32,809][60935] Updated weights for policy 0, policy_version 46340 (0.0009) [2023-10-13 22:51:33,171][60935] Updated weights for policy 0, policy_version 46350 (0.0007) [2023-10-13 22:51:33,536][60935] Updated weights for policy 0, policy_version 46360 (0.0009) [2023-10-13 22:51:35,664][60934] Updated weights for policy 1, policy_version 46722 (0.0008) [2023-10-13 22:51:36,015][60934] Updated weights for policy 1, policy_version 46732 (0.0011) [2023-10-13 22:51:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 95617024. Throughput: 0: 1682.0, 1: 1727.9. Samples: 23915892. Policy #0 lag: (min: 25.0, avg: 39.0, max: 40.0) [2023-10-13 22:51:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:51:36,378][60934] Updated weights for policy 1, policy_version 46742 (0.0009) [2023-10-13 22:51:37,589][60935] Updated weights for policy 0, policy_version 46370 (0.0010) [2023-10-13 22:51:37,969][60935] Updated weights for policy 0, policy_version 46380 (0.0008) [2023-10-13 22:51:38,328][60935] Updated weights for policy 0, policy_version 46390 (0.0010) [2023-10-13 22:51:38,694][60935] Updated weights for policy 0, policy_version 46400 (0.0008) [2023-10-13 22:51:40,328][60934] Updated weights for policy 1, policy_version 46752 (0.0008) [2023-10-13 22:51:40,697][60934] Updated weights for policy 1, policy_version 46762 (0.0009) [2023-10-13 22:51:41,068][60934] Updated weights for policy 1, policy_version 46772 (0.0008) [2023-10-13 22:51:41,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 95715328. Throughput: 0: 1702.4, 1: 1710.0. Samples: 23936604. Policy #0 lag: (min: 25.0, avg: 39.0, max: 40.0) [2023-10-13 22:51:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:51:42,784][60935] Updated weights for policy 0, policy_version 46410 (0.0007) [2023-10-13 22:51:43,150][60935] Updated weights for policy 0, policy_version 46420 (0.0007) [2023-10-13 22:51:43,516][60935] Updated weights for policy 0, policy_version 46430 (0.0008) [2023-10-13 22:51:45,093][60934] Updated weights for policy 1, policy_version 46782 (0.0009) [2023-10-13 22:51:45,456][60934] Updated weights for policy 1, policy_version 46792 (0.0010) [2023-10-13 22:51:45,831][60934] Updated weights for policy 1, policy_version 46802 (0.0010) [2023-10-13 22:51:46,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 95780864. Throughput: 0: 1670.9, 1: 1726.5. Samples: 23946426. Policy #0 lag: (min: 25.0, avg: 39.0, max: 40.0) [2023-10-13 22:51:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:51:47,621][60935] Updated weights for policy 0, policy_version 46440 (0.0011) [2023-10-13 22:51:48,001][60935] Updated weights for policy 0, policy_version 46450 (0.0010) [2023-10-13 22:51:48,369][60935] Updated weights for policy 0, policy_version 46460 (0.0011) [2023-10-13 22:51:49,836][60934] Updated weights for policy 1, policy_version 46812 (0.0010) [2023-10-13 22:51:50,206][60934] Updated weights for policy 1, policy_version 46822 (0.0010) [2023-10-13 22:51:50,574][60934] Updated weights for policy 1, policy_version 46832 (0.0007) [2023-10-13 22:51:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 95846400. Throughput: 0: 1701.9, 1: 1727.3. Samples: 23967522. Policy #0 lag: (min: 25.0, avg: 39.0, max: 40.0) [2023-10-13 22:51:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:51:52,324][60935] Updated weights for policy 0, policy_version 46470 (0.0010) [2023-10-13 22:51:52,698][60935] Updated weights for policy 0, policy_version 46480 (0.0011) [2023-10-13 22:51:53,061][60935] Updated weights for policy 0, policy_version 46490 (0.0008) [2023-10-13 22:51:54,533][60934] Updated weights for policy 1, policy_version 46842 (0.0007) [2023-10-13 22:51:54,903][60934] Updated weights for policy 1, policy_version 46852 (0.0007) [2023-10-13 22:51:55,269][60934] Updated weights for policy 1, policy_version 46862 (0.0009) [2023-10-13 22:51:55,638][60934] Updated weights for policy 1, policy_version 46872 (0.0009) [2023-10-13 22:51:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 95911936. Throughput: 0: 1714.0, 1: 1693.9. Samples: 23987336. Policy #0 lag: (min: 25.0, avg: 39.0, max: 40.0) [2023-10-13 22:51:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:51:57,098][60935] Updated weights for policy 0, policy_version 46500 (0.0008) [2023-10-13 22:51:57,475][60935] Updated weights for policy 0, policy_version 46510 (0.0010) [2023-10-13 22:51:57,845][60935] Updated weights for policy 0, policy_version 46520 (0.0007) [2023-10-13 22:51:59,600][60934] Updated weights for policy 1, policy_version 46882 (0.0008) [2023-10-13 22:51:59,963][60934] Updated weights for policy 1, policy_version 46892 (0.0008) [2023-10-13 22:52:00,325][60934] Updated weights for policy 1, policy_version 46902 (0.0008) [2023-10-13 22:52:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 95977472. Throughput: 0: 1690.8, 1: 1720.9. Samples: 23997648. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 22:52:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:52:01,828][60935] Updated weights for policy 0, policy_version 46530 (0.0009) [2023-10-13 22:52:02,197][60935] Updated weights for policy 0, policy_version 46540 (0.0008) [2023-10-13 22:52:02,576][60935] Updated weights for policy 0, policy_version 46550 (0.0008) [2023-10-13 22:52:02,943][60935] Updated weights for policy 0, policy_version 46560 (0.0009) [2023-10-13 22:52:04,318][60934] Updated weights for policy 1, policy_version 46912 (0.0010) [2023-10-13 22:52:04,682][60934] Updated weights for policy 1, policy_version 46922 (0.0007) [2023-10-13 22:52:05,041][60934] Updated weights for policy 1, policy_version 46932 (0.0008) [2023-10-13 22:52:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 96043008. Throughput: 0: 1718.2, 1: 1697.0. Samples: 24018120. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 22:52:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:52:06,822][60935] Updated weights for policy 0, policy_version 46570 (0.0009) [2023-10-13 22:52:07,183][60935] Updated weights for policy 0, policy_version 46580 (0.0008) [2023-10-13 22:52:07,545][60935] Updated weights for policy 0, policy_version 46590 (0.0009) [2023-10-13 22:52:09,128][60934] Updated weights for policy 1, policy_version 46942 (0.0007) [2023-10-13 22:52:09,499][60934] Updated weights for policy 1, policy_version 46952 (0.0007) [2023-10-13 22:52:09,860][60934] Updated weights for policy 1, policy_version 46962 (0.0008) [2023-10-13 22:52:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 96108544. Throughput: 0: 1719.3, 1: 1681.6. Samples: 24038690. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 22:52:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.280')] [2023-10-13 22:52:11,433][60935] Updated weights for policy 0, policy_version 46600 (0.0009) [2023-10-13 22:52:11,801][60935] Updated weights for policy 0, policy_version 46610 (0.0009) [2023-10-13 22:52:12,168][60935] Updated weights for policy 0, policy_version 46620 (0.0008) [2023-10-13 22:52:13,818][60934] Updated weights for policy 1, policy_version 46972 (0.0008) [2023-10-13 22:52:14,179][60934] Updated weights for policy 1, policy_version 46982 (0.0007) [2023-10-13 22:52:14,554][60934] Updated weights for policy 1, policy_version 46992 (0.0008) [2023-10-13 22:52:16,152][60935] Updated weights for policy 0, policy_version 46630 (0.0007) [2023-10-13 22:52:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 96174080. Throughput: 0: 1709.3, 1: 1712.6. Samples: 24049266. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 22:52:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:52:16,526][60935] Updated weights for policy 0, policy_version 46640 (0.0008) [2023-10-13 22:52:16,890][60935] Updated weights for policy 0, policy_version 46650 (0.0008) [2023-10-13 22:52:18,405][60934] Updated weights for policy 1, policy_version 47002 (0.0007) [2023-10-13 22:52:18,782][60934] Updated weights for policy 1, policy_version 47012 (0.0008) [2023-10-13 22:52:19,157][60934] Updated weights for policy 1, policy_version 47022 (0.0009) [2023-10-13 22:52:19,524][60934] Updated weights for policy 1, policy_version 47032 (0.0010) [2023-10-13 22:52:20,817][60935] Updated weights for policy 0, policy_version 46660 (0.0009) [2023-10-13 22:52:21,186][60935] Updated weights for policy 0, policy_version 46670 (0.0009) [2023-10-13 22:52:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 96239616. Throughput: 0: 1719.1, 1: 1691.4. Samples: 24069362. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 22:52:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:52:21,561][60935] Updated weights for policy 0, policy_version 46680 (0.0011) [2023-10-13 22:52:23,547][60934] Updated weights for policy 1, policy_version 47042 (0.0009) [2023-10-13 22:52:23,902][60934] Updated weights for policy 1, policy_version 47052 (0.0009) [2023-10-13 22:52:24,277][60934] Updated weights for policy 1, policy_version 47062 (0.0008) [2023-10-13 22:52:25,487][60935] Updated weights for policy 0, policy_version 46690 (0.0010) [2023-10-13 22:52:25,889][60935] Updated weights for policy 0, policy_version 46700 (0.0008) [2023-10-13 22:52:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 96305152. Throughput: 0: 1706.4, 1: 1703.7. Samples: 24090060. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 22:52:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:52:26,263][60935] Updated weights for policy 0, policy_version 46710 (0.0010) [2023-10-13 22:52:26,624][60935] Updated weights for policy 0, policy_version 46720 (0.0010) [2023-10-13 22:52:28,322][60934] Updated weights for policy 1, policy_version 47072 (0.0008) [2023-10-13 22:52:28,679][60934] Updated weights for policy 1, policy_version 47082 (0.0008) [2023-10-13 22:52:29,050][60934] Updated weights for policy 1, policy_version 47092 (0.0007) [2023-10-13 22:52:30,504][60935] Updated weights for policy 0, policy_version 46730 (0.0012) [2023-10-13 22:52:30,872][60935] Updated weights for policy 0, policy_version 46740 (0.0010) [2023-10-13 22:52:31,244][60935] Updated weights for policy 0, policy_version 46750 (0.0007) [2023-10-13 22:52:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 96370688. Throughput: 0: 1718.4, 1: 1702.6. Samples: 24100372. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 22:52:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:52:33,146][60934] Updated weights for policy 1, policy_version 47102 (0.0007) [2023-10-13 22:52:33,515][60934] Updated weights for policy 1, policy_version 47112 (0.0008) [2023-10-13 22:52:33,895][60934] Updated weights for policy 1, policy_version 47122 (0.0008) [2023-10-13 22:52:35,436][60935] Updated weights for policy 0, policy_version 46760 (0.0009) [2023-10-13 22:52:35,806][60935] Updated weights for policy 0, policy_version 46770 (0.0009) [2023-10-13 22:52:36,171][60935] Updated weights for policy 0, policy_version 46780 (0.0009) [2023-10-13 22:52:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 96436224. Throughput: 0: 1714.1, 1: 1686.9. Samples: 24120562. Policy #0 lag: (min: 3.0, avg: 8.2, max: 35.0) [2023-10-13 22:52:36,248][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:52:37,841][60934] Updated weights for policy 1, policy_version 47132 (0.0008) [2023-10-13 22:52:38,209][60934] Updated weights for policy 1, policy_version 47142 (0.0008) [2023-10-13 22:52:38,584][60934] Updated weights for policy 1, policy_version 47152 (0.0011) [2023-10-13 22:52:40,283][60935] Updated weights for policy 0, policy_version 46790 (0.0008) [2023-10-13 22:52:40,647][60935] Updated weights for policy 0, policy_version 46800 (0.0010) [2023-10-13 22:52:41,023][60935] Updated weights for policy 0, policy_version 46810 (0.0011) [2023-10-13 22:52:41,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 96534528. Throughput: 0: 1696.9, 1: 1718.1. Samples: 24141010. Policy #0 lag: (min: 3.0, avg: 8.2, max: 35.0) [2023-10-13 22:52:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:52:42,527][60934] Updated weights for policy 1, policy_version 47162 (0.0007) [2023-10-13 22:52:42,886][60934] Updated weights for policy 1, policy_version 47172 (0.0009) [2023-10-13 22:52:43,254][60934] Updated weights for policy 1, policy_version 47182 (0.0008) [2023-10-13 22:52:43,620][60934] Updated weights for policy 1, policy_version 47192 (0.0009) [2023-10-13 22:52:45,090][60935] Updated weights for policy 0, policy_version 46820 (0.0012) [2023-10-13 22:52:45,468][60935] Updated weights for policy 0, policy_version 46830 (0.0011) [2023-10-13 22:52:45,825][60935] Updated weights for policy 0, policy_version 46840 (0.0008) [2023-10-13 22:52:46,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 96600064. Throughput: 0: 1714.8, 1: 1697.6. Samples: 24151206. Policy #0 lag: (min: 3.0, avg: 8.2, max: 35.0) [2023-10-13 22:52:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:52:47,768][60934] Updated weights for policy 1, policy_version 47202 (0.0008) [2023-10-13 22:52:48,136][60934] Updated weights for policy 1, policy_version 47212 (0.0008) [2023-10-13 22:52:48,493][60934] Updated weights for policy 1, policy_version 47222 (0.0009) [2023-10-13 22:52:49,839][60935] Updated weights for policy 0, policy_version 46850 (0.0009) [2023-10-13 22:52:50,196][60935] Updated weights for policy 0, policy_version 46860 (0.0010) [2023-10-13 22:52:50,563][60935] Updated weights for policy 0, policy_version 46870 (0.0010) [2023-10-13 22:52:50,933][60935] Updated weights for policy 0, policy_version 46880 (0.0011) [2023-10-13 22:52:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 96665600. Throughput: 0: 1712.0, 1: 1704.9. Samples: 24171880. Policy #0 lag: (min: 3.0, avg: 8.2, max: 35.0) [2023-10-13 22:52:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:52:52,444][60934] Updated weights for policy 1, policy_version 47232 (0.0007) [2023-10-13 22:52:52,807][60934] Updated weights for policy 1, policy_version 47242 (0.0009) [2023-10-13 22:52:53,178][60934] Updated weights for policy 1, policy_version 47252 (0.0007) [2023-10-13 22:52:54,810][60935] Updated weights for policy 0, policy_version 46890 (0.0008) [2023-10-13 22:52:55,177][60935] Updated weights for policy 0, policy_version 46900 (0.0009) [2023-10-13 22:52:55,551][60935] Updated weights for policy 0, policy_version 46910 (0.0009) [2023-10-13 22:52:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 96731136. Throughput: 0: 1679.1, 1: 1729.0. Samples: 24192054. Policy #0 lag: (min: 3.0, avg: 8.2, max: 35.0) [2023-10-13 22:52:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:52:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000047256_48693248.pth... [2023-10-13 22:52:56,260][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000046912_48037888.pth... [2023-10-13 22:52:56,295][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000045312_46399488.pth [2023-10-13 22:52:56,299][60695] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p0/milestones/checkpoint_000046912_48037888.pth [2023-10-13 22:52:56,301][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000045656_47054848.pth [2023-10-13 22:52:56,307][60828] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p1/milestones/checkpoint_000047256_48693248.pth [2023-10-13 22:52:57,093][60934] Updated weights for policy 1, policy_version 47262 (0.0007) [2023-10-13 22:52:57,469][60934] Updated weights for policy 1, policy_version 47272 (0.0009) [2023-10-13 22:52:57,847][60934] Updated weights for policy 1, policy_version 47282 (0.0007) [2023-10-13 22:52:59,736][60935] Updated weights for policy 0, policy_version 46920 (0.0008) [2023-10-13 22:53:00,103][60935] Updated weights for policy 0, policy_version 46930 (0.0008) [2023-10-13 22:53:00,473][60935] Updated weights for policy 0, policy_version 46940 (0.0009) [2023-10-13 22:53:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 96796672. Throughput: 0: 1712.5, 1: 1695.8. Samples: 24202640. Policy #0 lag: (min: 3.0, avg: 8.2, max: 35.0) [2023-10-13 22:53:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:53:01,597][60934] Updated weights for policy 1, policy_version 47292 (0.0007) [2023-10-13 22:53:01,958][60934] Updated weights for policy 1, policy_version 47302 (0.0009) [2023-10-13 22:53:02,326][60934] Updated weights for policy 1, policy_version 47312 (0.0011) [2023-10-13 22:53:04,563][60935] Updated weights for policy 0, policy_version 46950 (0.0009) [2023-10-13 22:53:04,928][60935] Updated weights for policy 0, policy_version 46960 (0.0012) [2023-10-13 22:53:05,300][60935] Updated weights for policy 0, policy_version 46970 (0.0011) [2023-10-13 22:53:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 96862208. Throughput: 0: 1689.5, 1: 1721.5. Samples: 24222860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:53:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:53:06,350][60934] Updated weights for policy 1, policy_version 47322 (0.0010) [2023-10-13 22:53:06,719][60934] Updated weights for policy 1, policy_version 47332 (0.0007) [2023-10-13 22:53:07,094][60934] Updated weights for policy 1, policy_version 47342 (0.0008) [2023-10-13 22:53:07,463][60934] Updated weights for policy 1, policy_version 47352 (0.0009) [2023-10-13 22:53:09,413][60935] Updated weights for policy 0, policy_version 46980 (0.0009) [2023-10-13 22:53:09,781][60935] Updated weights for policy 0, policy_version 46990 (0.0007) [2023-10-13 22:53:10,150][60935] Updated weights for policy 0, policy_version 47000 (0.0008) [2023-10-13 22:53:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 96927744. Throughput: 0: 1675.2, 1: 1728.6. Samples: 24243230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:53:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:53:11,492][60934] Updated weights for policy 1, policy_version 47362 (0.0009) [2023-10-13 22:53:11,853][60934] Updated weights for policy 1, policy_version 47372 (0.0010) [2023-10-13 22:53:12,226][60934] Updated weights for policy 1, policy_version 47382 (0.0010) [2023-10-13 22:53:14,050][60935] Updated weights for policy 0, policy_version 47010 (0.0010) [2023-10-13 22:53:14,411][60935] Updated weights for policy 0, policy_version 47020 (0.0008) [2023-10-13 22:53:14,784][60935] Updated weights for policy 0, policy_version 47030 (0.0008) [2023-10-13 22:53:15,142][60935] Updated weights for policy 0, policy_version 47040 (0.0008) [2023-10-13 22:53:16,087][60934] Updated weights for policy 1, policy_version 47392 (0.0008) [2023-10-13 22:53:16,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 96993280. Throughput: 0: 1697.3, 1: 1714.0. Samples: 24253880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:53:16,248][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:53:16,457][60934] Updated weights for policy 1, policy_version 47402 (0.0009) [2023-10-13 22:53:16,817][60934] Updated weights for policy 1, policy_version 47412 (0.0009) [2023-10-13 22:53:19,068][60935] Updated weights for policy 0, policy_version 47050 (0.0008) [2023-10-13 22:53:19,435][60935] Updated weights for policy 0, policy_version 47060 (0.0009) [2023-10-13 22:53:19,800][60935] Updated weights for policy 0, policy_version 47070 (0.0008) [2023-10-13 22:53:20,904][60934] Updated weights for policy 1, policy_version 47422 (0.0008) [2023-10-13 22:53:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 97058816. Throughput: 0: 1676.7, 1: 1729.1. Samples: 24273826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:53:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:53:21,271][60934] Updated weights for policy 1, policy_version 47432 (0.0008) [2023-10-13 22:53:21,647][60934] Updated weights for policy 1, policy_version 47442 (0.0009) [2023-10-13 22:53:23,700][60935] Updated weights for policy 0, policy_version 47080 (0.0010) [2023-10-13 22:53:24,075][60935] Updated weights for policy 0, policy_version 47090 (0.0009) [2023-10-13 22:53:24,447][60935] Updated weights for policy 0, policy_version 47100 (0.0008) [2023-10-13 22:53:25,573][60934] Updated weights for policy 1, policy_version 47452 (0.0008) [2023-10-13 22:53:25,934][60934] Updated weights for policy 1, policy_version 47462 (0.0008) [2023-10-13 22:53:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 97124352. Throughput: 0: 1694.4, 1: 1725.8. Samples: 24294916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:53:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:53:26,297][60934] Updated weights for policy 1, policy_version 47472 (0.0009) [2023-10-13 22:53:28,436][60935] Updated weights for policy 0, policy_version 47110 (0.0010) [2023-10-13 22:53:28,808][60935] Updated weights for policy 0, policy_version 47120 (0.0008) [2023-10-13 22:53:29,182][60935] Updated weights for policy 0, policy_version 47130 (0.0007) [2023-10-13 22:53:30,376][60934] Updated weights for policy 1, policy_version 47482 (0.0009) [2023-10-13 22:53:30,739][60934] Updated weights for policy 1, policy_version 47492 (0.0010) [2023-10-13 22:53:31,114][60934] Updated weights for policy 1, policy_version 47502 (0.0007) [2023-10-13 22:53:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 97189888. Throughput: 0: 1692.3, 1: 1723.4. Samples: 24304912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:53:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:53:31,472][60934] Updated weights for policy 1, policy_version 47512 (0.0011) [2023-10-13 22:53:33,232][60935] Updated weights for policy 0, policy_version 47140 (0.0009) [2023-10-13 22:53:33,602][60935] Updated weights for policy 0, policy_version 47150 (0.0008) [2023-10-13 22:53:33,980][60935] Updated weights for policy 0, policy_version 47160 (0.0009) [2023-10-13 22:53:35,477][60934] Updated weights for policy 1, policy_version 47522 (0.0009) [2023-10-13 22:53:35,847][60934] Updated weights for policy 1, policy_version 47532 (0.0010) [2023-10-13 22:53:36,211][60934] Updated weights for policy 1, policy_version 47542 (0.0011) [2023-10-13 22:53:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 97255424. Throughput: 0: 1679.1, 1: 1732.6. Samples: 24325404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:53:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:53:38,007][60935] Updated weights for policy 0, policy_version 47170 (0.0007) [2023-10-13 22:53:38,381][60935] Updated weights for policy 0, policy_version 47180 (0.0007) [2023-10-13 22:53:38,749][60935] Updated weights for policy 0, policy_version 47190 (0.0008) [2023-10-13 22:53:39,122][60935] Updated weights for policy 0, policy_version 47200 (0.0008) [2023-10-13 22:53:40,226][60934] Updated weights for policy 1, policy_version 47552 (0.0008) [2023-10-13 22:53:40,596][60934] Updated weights for policy 1, policy_version 47562 (0.0009) [2023-10-13 22:53:40,961][60934] Updated weights for policy 1, policy_version 47572 (0.0008) [2023-10-13 22:53:41,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 97353728. Throughput: 0: 1704.4, 1: 1707.7. Samples: 24345600. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-13 22:53:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:53:43,180][60935] Updated weights for policy 0, policy_version 47210 (0.0007) [2023-10-13 22:53:43,563][60935] Updated weights for policy 0, policy_version 47220 (0.0008) [2023-10-13 22:53:43,931][60935] Updated weights for policy 0, policy_version 47230 (0.0007) [2023-10-13 22:53:44,932][60934] Updated weights for policy 1, policy_version 47582 (0.0007) [2023-10-13 22:53:45,289][60934] Updated weights for policy 1, policy_version 47592 (0.0008) [2023-10-13 22:53:45,658][60934] Updated weights for policy 1, policy_version 47602 (0.0011) [2023-10-13 22:53:46,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 97419264. Throughput: 0: 1676.8, 1: 1724.5. Samples: 24355700. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-13 22:53:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:53:47,941][60935] Updated weights for policy 0, policy_version 47240 (0.0008) [2023-10-13 22:53:48,306][60935] Updated weights for policy 0, policy_version 47250 (0.0009) [2023-10-13 22:53:48,674][60935] Updated weights for policy 0, policy_version 47260 (0.0009) [2023-10-13 22:53:49,650][60934] Updated weights for policy 1, policy_version 47612 (0.0008) [2023-10-13 22:53:50,023][60934] Updated weights for policy 1, policy_version 47622 (0.0007) [2023-10-13 22:53:50,385][60934] Updated weights for policy 1, policy_version 47632 (0.0009) [2023-10-13 22:53:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 97484800. Throughput: 0: 1691.5, 1: 1721.9. Samples: 24376462. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-13 22:53:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.390')] [2023-10-13 22:53:52,770][60935] Updated weights for policy 0, policy_version 47270 (0.0008) [2023-10-13 22:53:53,139][60935] Updated weights for policy 0, policy_version 47280 (0.0008) [2023-10-13 22:53:53,516][60935] Updated weights for policy 0, policy_version 47290 (0.0009) [2023-10-13 22:53:54,418][60934] Updated weights for policy 1, policy_version 47642 (0.0009) [2023-10-13 22:53:54,786][60934] Updated weights for policy 1, policy_version 47652 (0.0008) [2023-10-13 22:53:55,157][60934] Updated weights for policy 1, policy_version 47662 (0.0008) [2023-10-13 22:53:55,510][60934] Updated weights for policy 1, policy_version 47672 (0.0008) [2023-10-13 22:53:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 97550336. Throughput: 0: 1710.8, 1: 1687.9. Samples: 24396168. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-13 22:53:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.400')] [2023-10-13 22:53:57,716][60935] Updated weights for policy 0, policy_version 47300 (0.0010) [2023-10-13 22:53:58,085][60935] Updated weights for policy 0, policy_version 47310 (0.0009) [2023-10-13 22:53:58,455][60935] Updated weights for policy 0, policy_version 47320 (0.0007) [2023-10-13 22:53:59,406][60934] Updated weights for policy 1, policy_version 47682 (0.0007) [2023-10-13 22:53:59,778][60934] Updated weights for policy 1, policy_version 47692 (0.0008) [2023-10-13 22:54:00,139][60934] Updated weights for policy 1, policy_version 47702 (0.0007) [2023-10-13 22:54:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 97615872. Throughput: 0: 1675.4, 1: 1720.2. Samples: 24406682. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-13 22:54:01,248][59943] Avg episode reward: [(0, '0.000'), (1, '-0.400')] [2023-10-13 22:54:02,420][60935] Updated weights for policy 0, policy_version 47330 (0.0008) [2023-10-13 22:54:02,796][60935] Updated weights for policy 0, policy_version 47340 (0.0007) [2023-10-13 22:54:03,154][60935] Updated weights for policy 0, policy_version 47350 (0.0007) [2023-10-13 22:54:03,527][60935] Updated weights for policy 0, policy_version 47360 (0.0007) [2023-10-13 22:54:04,071][60934] Updated weights for policy 1, policy_version 47712 (0.0009) [2023-10-13 22:54:04,441][60934] Updated weights for policy 1, policy_version 47722 (0.0007) [2023-10-13 22:54:04,804][60934] Updated weights for policy 1, policy_version 47732 (0.0008) [2023-10-13 22:54:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 97681408. Throughput: 0: 1705.4, 1: 1700.2. Samples: 24427078. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-13 22:54:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.400')] [2023-10-13 22:54:07,418][60935] Updated weights for policy 0, policy_version 47370 (0.0009) [2023-10-13 22:54:07,785][60935] Updated weights for policy 0, policy_version 47380 (0.0008) [2023-10-13 22:54:08,161][60935] Updated weights for policy 0, policy_version 47390 (0.0011) [2023-10-13 22:54:08,990][60934] Updated weights for policy 1, policy_version 47742 (0.0009) [2023-10-13 22:54:09,347][60934] Updated weights for policy 1, policy_version 47752 (0.0008) [2023-10-13 22:54:09,725][60934] Updated weights for policy 1, policy_version 47762 (0.0009) [2023-10-13 22:54:11,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 97746944. Throughput: 0: 1701.4, 1: 1687.7. Samples: 24447428. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-13 22:54:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.400')] [2023-10-13 22:54:12,136][60935] Updated weights for policy 0, policy_version 47400 (0.0011) [2023-10-13 22:54:12,514][60935] Updated weights for policy 0, policy_version 47410 (0.0009) [2023-10-13 22:54:12,878][60935] Updated weights for policy 0, policy_version 47420 (0.0009) [2023-10-13 22:54:13,859][60934] Updated weights for policy 1, policy_version 47772 (0.0008) [2023-10-13 22:54:14,233][60934] Updated weights for policy 1, policy_version 47782 (0.0009) [2023-10-13 22:54:14,595][60934] Updated weights for policy 1, policy_version 47792 (0.0011) [2023-10-13 22:54:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 97812480. Throughput: 0: 1686.6, 1: 1711.5. Samples: 24457826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:54:16,249][59943] Avg episode reward: [(0, '-0.460'), (1, '-0.400')] [2023-10-13 22:54:16,904][60935] Updated weights for policy 0, policy_version 47430 (0.0008) [2023-10-13 22:54:17,268][60935] Updated weights for policy 0, policy_version 47440 (0.0008) [2023-10-13 22:54:17,641][60935] Updated weights for policy 0, policy_version 47450 (0.0008) [2023-10-13 22:54:18,496][60934] Updated weights for policy 1, policy_version 47802 (0.0007) [2023-10-13 22:54:18,862][60934] Updated weights for policy 1, policy_version 47812 (0.0008) [2023-10-13 22:54:19,223][60934] Updated weights for policy 1, policy_version 47822 (0.0010) [2023-10-13 22:54:19,584][60934] Updated weights for policy 1, policy_version 47832 (0.0010) [2023-10-13 22:54:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 97878016. Throughput: 0: 1700.6, 1: 1684.1. Samples: 24477714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:54:21,249][59943] Avg episode reward: [(0, '-0.460'), (1, '-0.400')] [2023-10-13 22:54:21,812][60935] Updated weights for policy 0, policy_version 47460 (0.0008) [2023-10-13 22:54:22,177][60935] Updated weights for policy 0, policy_version 47470 (0.0009) [2023-10-13 22:54:22,547][60935] Updated weights for policy 0, policy_version 47480 (0.0008) [2023-10-13 22:54:23,557][60934] Updated weights for policy 1, policy_version 47842 (0.0008) [2023-10-13 22:54:23,933][60934] Updated weights for policy 1, policy_version 47852 (0.0010) [2023-10-13 22:54:24,300][60934] Updated weights for policy 1, policy_version 47862 (0.0007) [2023-10-13 22:54:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 97943552. Throughput: 0: 1699.3, 1: 1697.2. Samples: 24498444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:54:26,249][59943] Avg episode reward: [(0, '-0.460'), (1, '-0.400')] [2023-10-13 22:54:26,624][60935] Updated weights for policy 0, policy_version 47490 (0.0008) [2023-10-13 22:54:27,000][60935] Updated weights for policy 0, policy_version 47500 (0.0010) [2023-10-13 22:54:27,376][60935] Updated weights for policy 0, policy_version 47510 (0.0008) [2023-10-13 22:54:27,746][60935] Updated weights for policy 0, policy_version 47520 (0.0009) [2023-10-13 22:54:28,180][60934] Updated weights for policy 1, policy_version 47872 (0.0007) [2023-10-13 22:54:28,550][60934] Updated weights for policy 1, policy_version 47882 (0.0007) [2023-10-13 22:54:28,917][60934] Updated weights for policy 1, policy_version 47892 (0.0007) [2023-10-13 22:54:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 98009088. Throughput: 0: 1693.5, 1: 1696.2. Samples: 24508238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:54:31,249][59943] Avg episode reward: [(0, '-0.460'), (1, '-0.400')] [2023-10-13 22:54:31,686][60935] Updated weights for policy 0, policy_version 47530 (0.0008) [2023-10-13 22:54:32,053][60935] Updated weights for policy 0, policy_version 47540 (0.0010) [2023-10-13 22:54:32,429][60935] Updated weights for policy 0, policy_version 47550 (0.0010) [2023-10-13 22:54:32,940][60934] Updated weights for policy 1, policy_version 47902 (0.0008) [2023-10-13 22:54:33,307][60934] Updated weights for policy 1, policy_version 47912 (0.0007) [2023-10-13 22:54:33,681][60934] Updated weights for policy 1, policy_version 47922 (0.0008) [2023-10-13 22:54:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 98074624. Throughput: 0: 1699.2, 1: 1680.0. Samples: 24528526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:54:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.400')] [2023-10-13 22:54:36,506][60935] Updated weights for policy 0, policy_version 47560 (0.0007) [2023-10-13 22:54:36,880][60935] Updated weights for policy 0, policy_version 47570 (0.0009) [2023-10-13 22:54:37,245][60935] Updated weights for policy 0, policy_version 47580 (0.0009) [2023-10-13 22:54:37,504][60934] Updated weights for policy 1, policy_version 47932 (0.0008) [2023-10-13 22:54:37,870][60934] Updated weights for policy 1, policy_version 47942 (0.0009) [2023-10-13 22:54:38,239][60934] Updated weights for policy 1, policy_version 47952 (0.0010) [2023-10-13 22:54:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 98140160. Throughput: 0: 1696.2, 1: 1710.9. Samples: 24549488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:54:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.400')] [2023-10-13 22:54:41,434][60935] Updated weights for policy 0, policy_version 47590 (0.0008) [2023-10-13 22:54:41,801][60935] Updated weights for policy 0, policy_version 47600 (0.0008) [2023-10-13 22:54:42,156][60934] Updated weights for policy 1, policy_version 47962 (0.0008) [2023-10-13 22:54:42,168][60935] Updated weights for policy 0, policy_version 47610 (0.0008) [2023-10-13 22:54:42,523][60934] Updated weights for policy 1, policy_version 47972 (0.0008) [2023-10-13 22:54:42,890][60934] Updated weights for policy 1, policy_version 47982 (0.0007) [2023-10-13 22:54:43,258][60934] Updated weights for policy 1, policy_version 47992 (0.0007) [2023-10-13 22:54:46,146][60935] Updated weights for policy 0, policy_version 47620 (0.0008) [2023-10-13 22:54:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 98205696. Throughput: 0: 1703.2, 1: 1681.3. Samples: 24558986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:54:46,249][59943] Avg episode reward: [(0, '-0.810'), (1, '-0.400')] [2023-10-13 22:54:46,520][60935] Updated weights for policy 0, policy_version 47630 (0.0007) [2023-10-13 22:54:46,882][60935] Updated weights for policy 0, policy_version 47640 (0.0008) [2023-10-13 22:54:47,397][60934] Updated weights for policy 1, policy_version 48002 (0.0008) [2023-10-13 22:54:47,773][60934] Updated weights for policy 1, policy_version 48012 (0.0009) [2023-10-13 22:54:48,130][60934] Updated weights for policy 1, policy_version 48022 (0.0008) [2023-10-13 22:54:50,778][60935] Updated weights for policy 0, policy_version 47650 (0.0008) [2023-10-13 22:54:51,141][60935] Updated weights for policy 0, policy_version 47660 (0.0007) [2023-10-13 22:54:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 98271232. Throughput: 0: 1700.7, 1: 1695.0. Samples: 24579884. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-13 22:54:51,249][59943] Avg episode reward: [(0, '-0.810'), (1, '-0.400')] [2023-10-13 22:54:51,513][60935] Updated weights for policy 0, policy_version 47670 (0.0007) [2023-10-13 22:54:51,882][60935] Updated weights for policy 0, policy_version 47680 (0.0007) [2023-10-13 22:54:52,194][60934] Updated weights for policy 1, policy_version 48032 (0.0008) [2023-10-13 22:54:52,565][60934] Updated weights for policy 1, policy_version 48042 (0.0007) [2023-10-13 22:54:52,939][60934] Updated weights for policy 1, policy_version 48052 (0.0008) [2023-10-13 22:54:55,904][60935] Updated weights for policy 0, policy_version 47690 (0.0008) [2023-10-13 22:54:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 98336768. Throughput: 0: 1695.9, 1: 1710.1. Samples: 24600698. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-13 22:54:56,249][59943] Avg episode reward: [(0, '-0.810'), (1, '-0.400')] [2023-10-13 22:54:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000048056_49512448.pth... [2023-10-13 22:54:56,265][60935] Updated weights for policy 0, policy_version 47700 (0.0007) [2023-10-13 22:54:56,300][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000046456_47874048.pth [2023-10-13 22:54:56,633][60935] Updated weights for policy 0, policy_version 47710 (0.0008) [2023-10-13 22:54:56,706][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000047712_48857088.pth... [2023-10-13 22:54:56,744][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000046112_47218688.pth [2023-10-13 22:54:57,084][60934] Updated weights for policy 1, policy_version 48062 (0.0009) [2023-10-13 22:54:57,448][60934] Updated weights for policy 1, policy_version 48072 (0.0008) [2023-10-13 22:54:57,811][60934] Updated weights for policy 1, policy_version 48082 (0.0010) [2023-10-13 22:55:00,643][60935] Updated weights for policy 0, policy_version 47720 (0.0011) [2023-10-13 22:55:01,009][60935] Updated weights for policy 0, policy_version 47730 (0.0010) [2023-10-13 22:55:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 98402304. Throughput: 0: 1703.5, 1: 1682.7. Samples: 24610204. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-13 22:55:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.400')] [2023-10-13 22:55:01,385][60935] Updated weights for policy 0, policy_version 47740 (0.0008) [2023-10-13 22:55:01,990][60934] Updated weights for policy 1, policy_version 48092 (0.0008) [2023-10-13 22:55:02,362][60934] Updated weights for policy 1, policy_version 48102 (0.0008) [2023-10-13 22:55:02,727][60934] Updated weights for policy 1, policy_version 48112 (0.0007) [2023-10-13 22:55:05,301][60935] Updated weights for policy 0, policy_version 47750 (0.0009) [2023-10-13 22:55:05,674][60935] Updated weights for policy 0, policy_version 47760 (0.0010) [2023-10-13 22:55:06,035][60935] Updated weights for policy 0, policy_version 47770 (0.0007) [2023-10-13 22:55:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 98467840. Throughput: 0: 1703.5, 1: 1706.2. Samples: 24631148. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-13 22:55:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.400')] [2023-10-13 22:55:06,659][60934] Updated weights for policy 1, policy_version 48122 (0.0007) [2023-10-13 22:55:07,025][60934] Updated weights for policy 1, policy_version 48132 (0.0008) [2023-10-13 22:55:07,388][60934] Updated weights for policy 1, policy_version 48142 (0.0007) [2023-10-13 22:55:07,756][60934] Updated weights for policy 1, policy_version 48152 (0.0007) [2023-10-13 22:55:10,042][60935] Updated weights for policy 0, policy_version 47780 (0.0010) [2023-10-13 22:55:10,412][60935] Updated weights for policy 0, policy_version 47790 (0.0008) [2023-10-13 22:55:10,774][60935] Updated weights for policy 0, policy_version 47800 (0.0008) [2023-10-13 22:55:11,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 98566144. Throughput: 0: 1677.4, 1: 1714.8. Samples: 24651092. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-13 22:55:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.400')] [2023-10-13 22:55:11,763][60934] Updated weights for policy 1, policy_version 48162 (0.0007) [2023-10-13 22:55:12,133][60934] Updated weights for policy 1, policy_version 48172 (0.0008) [2023-10-13 22:55:12,502][60934] Updated weights for policy 1, policy_version 48182 (0.0008) [2023-10-13 22:55:14,824][60935] Updated weights for policy 0, policy_version 47810 (0.0009) [2023-10-13 22:55:15,204][60935] Updated weights for policy 0, policy_version 47820 (0.0007) [2023-10-13 22:55:15,564][60935] Updated weights for policy 0, policy_version 47830 (0.0010) [2023-10-13 22:55:15,933][60935] Updated weights for policy 0, policy_version 47840 (0.0008) [2023-10-13 22:55:16,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 98631680. Throughput: 0: 1703.0, 1: 1701.1. Samples: 24661424. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-13 22:55:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.400')] [2023-10-13 22:55:16,381][60934] Updated weights for policy 1, policy_version 48192 (0.0007) [2023-10-13 22:55:16,742][60934] Updated weights for policy 1, policy_version 48202 (0.0007) [2023-10-13 22:55:17,114][60934] Updated weights for policy 1, policy_version 48212 (0.0009) [2023-10-13 22:55:19,959][60935] Updated weights for policy 0, policy_version 47850 (0.0010) [2023-10-13 22:55:20,338][60935] Updated weights for policy 0, policy_version 47860 (0.0009) [2023-10-13 22:55:20,706][60935] Updated weights for policy 0, policy_version 47870 (0.0008) [2023-10-13 22:55:21,114][60934] Updated weights for policy 1, policy_version 48222 (0.0009) [2023-10-13 22:55:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 98697216. Throughput: 0: 1698.1, 1: 1718.4. Samples: 24682266. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:55:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:55:21,481][60934] Updated weights for policy 1, policy_version 48232 (0.0010) [2023-10-13 22:55:21,856][60934] Updated weights for policy 1, policy_version 48242 (0.0009) [2023-10-13 22:55:24,464][60935] Updated weights for policy 0, policy_version 47880 (0.0009) [2023-10-13 22:55:24,821][60935] Updated weights for policy 0, policy_version 47890 (0.0009) [2023-10-13 22:55:25,189][60935] Updated weights for policy 0, policy_version 47900 (0.0008) [2023-10-13 22:55:25,870][60934] Updated weights for policy 1, policy_version 48252 (0.0009) [2023-10-13 22:55:26,234][60934] Updated weights for policy 1, policy_version 48262 (0.0009) [2023-10-13 22:55:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 98762752. Throughput: 0: 1685.5, 1: 1714.4. Samples: 24702484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:55:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:55:26,606][60934] Updated weights for policy 1, policy_version 48272 (0.0008) [2023-10-13 22:55:29,263][60935] Updated weights for policy 0, policy_version 47910 (0.0007) [2023-10-13 22:55:29,635][60935] Updated weights for policy 0, policy_version 47920 (0.0008) [2023-10-13 22:55:30,001][60935] Updated weights for policy 0, policy_version 47930 (0.0008) [2023-10-13 22:55:30,420][60934] Updated weights for policy 1, policy_version 48282 (0.0009) [2023-10-13 22:55:30,785][60934] Updated weights for policy 1, policy_version 48292 (0.0009) [2023-10-13 22:55:31,158][60934] Updated weights for policy 1, policy_version 48302 (0.0009) [2023-10-13 22:55:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 98828288. Throughput: 0: 1706.9, 1: 1712.8. Samples: 24712870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:55:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:55:31,523][60934] Updated weights for policy 1, policy_version 48312 (0.0008) [2023-10-13 22:55:34,122][60935] Updated weights for policy 0, policy_version 47940 (0.0009) [2023-10-13 22:55:34,489][60935] Updated weights for policy 0, policy_version 47950 (0.0008) [2023-10-13 22:55:34,857][60935] Updated weights for policy 0, policy_version 47960 (0.0008) [2023-10-13 22:55:35,594][60934] Updated weights for policy 1, policy_version 48322 (0.0011) [2023-10-13 22:55:35,963][60934] Updated weights for policy 1, policy_version 48332 (0.0007) [2023-10-13 22:55:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 98893824. Throughput: 0: 1682.6, 1: 1722.7. Samples: 24733122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:55:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:55:36,322][60934] Updated weights for policy 1, policy_version 48342 (0.0007) [2023-10-13 22:55:39,006][60935] Updated weights for policy 0, policy_version 47970 (0.0009) [2023-10-13 22:55:39,364][60935] Updated weights for policy 0, policy_version 47980 (0.0012) [2023-10-13 22:55:39,729][60935] Updated weights for policy 0, policy_version 47990 (0.0011) [2023-10-13 22:55:40,093][60935] Updated weights for policy 0, policy_version 48000 (0.0009) [2023-10-13 22:55:40,266][60934] Updated weights for policy 1, policy_version 48352 (0.0008) [2023-10-13 22:55:40,640][60934] Updated weights for policy 1, policy_version 48362 (0.0011) [2023-10-13 22:55:41,000][60934] Updated weights for policy 1, policy_version 48372 (0.0010) [2023-10-13 22:55:41,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 98992128. Throughput: 0: 1674.9, 1: 1707.2. Samples: 24752892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:55:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:55:44,160][60935] Updated weights for policy 0, policy_version 48010 (0.0010) [2023-10-13 22:55:44,530][60935] Updated weights for policy 0, policy_version 48020 (0.0009) [2023-10-13 22:55:44,901][60935] Updated weights for policy 0, policy_version 48030 (0.0008) [2023-10-13 22:55:44,908][60934] Updated weights for policy 1, policy_version 48382 (0.0009) [2023-10-13 22:55:45,280][60934] Updated weights for policy 1, policy_version 48392 (0.0009) [2023-10-13 22:55:45,649][60934] Updated weights for policy 1, policy_version 48402 (0.0009) [2023-10-13 22:55:46,248][59943] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 99057664. Throughput: 0: 1695.1, 1: 1722.7. Samples: 24764006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:55:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:55:48,900][60935] Updated weights for policy 0, policy_version 48040 (0.0007) [2023-10-13 22:55:49,258][60935] Updated weights for policy 0, policy_version 48050 (0.0008) [2023-10-13 22:55:49,627][60935] Updated weights for policy 0, policy_version 48060 (0.0009) [2023-10-13 22:55:49,724][60934] Updated weights for policy 1, policy_version 48412 (0.0008) [2023-10-13 22:55:50,083][60934] Updated weights for policy 1, policy_version 48422 (0.0009) [2023-10-13 22:55:50,445][60934] Updated weights for policy 1, policy_version 48432 (0.0009) [2023-10-13 22:55:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 99123200. Throughput: 0: 1669.5, 1: 1722.4. Samples: 24783786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:55:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:55:53,684][60935] Updated weights for policy 0, policy_version 48070 (0.0007) [2023-10-13 22:55:54,060][60935] Updated weights for policy 0, policy_version 48080 (0.0011) [2023-10-13 22:55:54,434][60935] Updated weights for policy 0, policy_version 48090 (0.0007) [2023-10-13 22:55:54,488][60934] Updated weights for policy 1, policy_version 48442 (0.0008) [2023-10-13 22:55:54,858][60934] Updated weights for policy 1, policy_version 48452 (0.0007) [2023-10-13 22:55:55,222][60934] Updated weights for policy 1, policy_version 48462 (0.0007) [2023-10-13 22:55:55,592][60934] Updated weights for policy 1, policy_version 48472 (0.0008) [2023-10-13 22:55:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 99188736. Throughput: 0: 1696.8, 1: 1687.0. Samples: 24803364. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 22:55:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:55:58,479][60935] Updated weights for policy 0, policy_version 48100 (0.0008) [2023-10-13 22:55:58,865][60935] Updated weights for policy 0, policy_version 48110 (0.0008) [2023-10-13 22:55:59,226][60935] Updated weights for policy 0, policy_version 48120 (0.0009) [2023-10-13 22:55:59,600][60934] Updated weights for policy 1, policy_version 48482 (0.0010) [2023-10-13 22:55:59,978][60934] Updated weights for policy 1, policy_version 48492 (0.0009) [2023-10-13 22:56:00,348][60934] Updated weights for policy 1, policy_version 48502 (0.0009) [2023-10-13 22:56:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 99254272. Throughput: 0: 1690.0, 1: 1714.3. Samples: 24814618. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 22:56:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:56:03,297][60935] Updated weights for policy 0, policy_version 48130 (0.0008) [2023-10-13 22:56:03,664][60935] Updated weights for policy 0, policy_version 48140 (0.0009) [2023-10-13 22:56:04,034][60935] Updated weights for policy 0, policy_version 48150 (0.0009) [2023-10-13 22:56:04,261][60934] Updated weights for policy 1, policy_version 48512 (0.0008) [2023-10-13 22:56:04,393][60935] Updated weights for policy 0, policy_version 48160 (0.0010) [2023-10-13 22:56:04,628][60934] Updated weights for policy 1, policy_version 48522 (0.0007) [2023-10-13 22:56:04,995][60934] Updated weights for policy 1, policy_version 48532 (0.0007) [2023-10-13 22:56:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 99319808. Throughput: 0: 1674.1, 1: 1698.3. Samples: 24834026. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 22:56:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:56:08,478][60935] Updated weights for policy 0, policy_version 48170 (0.0011) [2023-10-13 22:56:08,842][60935] Updated weights for policy 0, policy_version 48180 (0.0011) [2023-10-13 22:56:09,156][60934] Updated weights for policy 1, policy_version 48542 (0.0007) [2023-10-13 22:56:09,216][60935] Updated weights for policy 0, policy_version 48190 (0.0008) [2023-10-13 22:56:09,526][60934] Updated weights for policy 1, policy_version 48552 (0.0007) [2023-10-13 22:56:09,886][60934] Updated weights for policy 1, policy_version 48562 (0.0008) [2023-10-13 22:56:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 99385344. Throughput: 0: 1689.0, 1: 1682.1. Samples: 24854184. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 22:56:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:56:13,380][60935] Updated weights for policy 0, policy_version 48200 (0.0009) [2023-10-13 22:56:13,752][60935] Updated weights for policy 0, policy_version 48210 (0.0010) [2023-10-13 22:56:13,842][60934] Updated weights for policy 1, policy_version 48572 (0.0008) [2023-10-13 22:56:14,120][60935] Updated weights for policy 0, policy_version 48220 (0.0009) [2023-10-13 22:56:14,206][60934] Updated weights for policy 1, policy_version 48582 (0.0007) [2023-10-13 22:56:14,567][60934] Updated weights for policy 1, policy_version 48592 (0.0007) [2023-10-13 22:56:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 99450880. Throughput: 0: 1674.0, 1: 1711.8. Samples: 24865230. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 22:56:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:56:18,290][60935] Updated weights for policy 0, policy_version 48230 (0.0008) [2023-10-13 22:56:18,547][60934] Updated weights for policy 1, policy_version 48602 (0.0008) [2023-10-13 22:56:18,662][60935] Updated weights for policy 0, policy_version 48240 (0.0008) [2023-10-13 22:56:18,920][60934] Updated weights for policy 1, policy_version 48612 (0.0007) [2023-10-13 22:56:19,024][60935] Updated weights for policy 0, policy_version 48250 (0.0009) [2023-10-13 22:56:19,291][60934] Updated weights for policy 1, policy_version 48622 (0.0008) [2023-10-13 22:56:19,656][60934] Updated weights for policy 1, policy_version 48632 (0.0010) [2023-10-13 22:56:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 99516416. Throughput: 0: 1682.4, 1: 1683.6. Samples: 24884590. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 22:56:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:56:22,815][60935] Updated weights for policy 0, policy_version 48260 (0.0009) [2023-10-13 22:56:23,192][60935] Updated weights for policy 0, policy_version 48270 (0.0011) [2023-10-13 22:56:23,570][60935] Updated weights for policy 0, policy_version 48280 (0.0008) [2023-10-13 22:56:23,823][60934] Updated weights for policy 1, policy_version 48642 (0.0007) [2023-10-13 22:56:24,194][60934] Updated weights for policy 1, policy_version 48652 (0.0007) [2023-10-13 22:56:24,553][60934] Updated weights for policy 1, policy_version 48662 (0.0008) [2023-10-13 22:56:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 99581952. Throughput: 0: 1698.0, 1: 1686.0. Samples: 24905170. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-13 22:56:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:56:27,330][60935] Updated weights for policy 0, policy_version 48290 (0.0010) [2023-10-13 22:56:27,708][60935] Updated weights for policy 0, policy_version 48300 (0.0008) [2023-10-13 22:56:28,070][60935] Updated weights for policy 0, policy_version 48310 (0.0009) [2023-10-13 22:56:28,449][60935] Updated weights for policy 0, policy_version 48320 (0.0009) [2023-10-13 22:56:28,650][60934] Updated weights for policy 1, policy_version 48672 (0.0010) [2023-10-13 22:56:29,016][60934] Updated weights for policy 1, policy_version 48682 (0.0007) [2023-10-13 22:56:29,380][60934] Updated weights for policy 1, policy_version 48692 (0.0007) [2023-10-13 22:56:31,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 99647488. Throughput: 0: 1668.8, 1: 1693.4. Samples: 24915306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:56:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:56:32,461][60935] Updated weights for policy 0, policy_version 48330 (0.0007) [2023-10-13 22:56:32,818][60935] Updated weights for policy 0, policy_version 48340 (0.0009) [2023-10-13 22:56:33,185][60935] Updated weights for policy 0, policy_version 48350 (0.0008) [2023-10-13 22:56:33,498][60934] Updated weights for policy 1, policy_version 48702 (0.0007) [2023-10-13 22:56:33,872][60934] Updated weights for policy 1, policy_version 48712 (0.0007) [2023-10-13 22:56:34,243][60934] Updated weights for policy 1, policy_version 48722 (0.0007) [2023-10-13 22:56:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 99713024. Throughput: 0: 1695.8, 1: 1666.3. Samples: 24935078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:56:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:56:37,414][60935] Updated weights for policy 0, policy_version 48360 (0.0009) [2023-10-13 22:56:37,791][60935] Updated weights for policy 0, policy_version 48370 (0.0009) [2023-10-13 22:56:38,167][60935] Updated weights for policy 0, policy_version 48380 (0.0009) [2023-10-13 22:56:38,251][60934] Updated weights for policy 1, policy_version 48732 (0.0009) [2023-10-13 22:56:38,627][60934] Updated weights for policy 1, policy_version 48742 (0.0010) [2023-10-13 22:56:38,997][60934] Updated weights for policy 1, policy_version 48752 (0.0009) [2023-10-13 22:56:41,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 99778560. Throughput: 0: 1695.7, 1: 1697.5. Samples: 24956058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:56:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:56:42,174][60935] Updated weights for policy 0, policy_version 48390 (0.0008) [2023-10-13 22:56:42,548][60935] Updated weights for policy 0, policy_version 48400 (0.0007) [2023-10-13 22:56:42,921][60935] Updated weights for policy 0, policy_version 48410 (0.0007) [2023-10-13 22:56:42,991][60934] Updated weights for policy 1, policy_version 48762 (0.0008) [2023-10-13 22:56:43,369][60934] Updated weights for policy 1, policy_version 48772 (0.0007) [2023-10-13 22:56:43,731][60934] Updated weights for policy 1, policy_version 48782 (0.0009) [2023-10-13 22:56:44,096][60934] Updated weights for policy 1, policy_version 48792 (0.0007) [2023-10-13 22:56:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 99844096. Throughput: 0: 1680.0, 1: 1680.8. Samples: 24965854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:56:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:56:46,856][60935] Updated weights for policy 0, policy_version 48420 (0.0009) [2023-10-13 22:56:47,226][60935] Updated weights for policy 0, policy_version 48430 (0.0009) [2023-10-13 22:56:47,603][60935] Updated weights for policy 0, policy_version 48440 (0.0009) [2023-10-13 22:56:48,131][60934] Updated weights for policy 1, policy_version 48802 (0.0009) [2023-10-13 22:56:48,502][60934] Updated weights for policy 1, policy_version 48812 (0.0010) [2023-10-13 22:56:48,877][60934] Updated weights for policy 1, policy_version 48822 (0.0009) [2023-10-13 22:56:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 99909632. Throughput: 0: 1701.7, 1: 1682.5. Samples: 24986314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:56:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:56:51,633][60935] Updated weights for policy 0, policy_version 48450 (0.0008) [2023-10-13 22:56:52,006][60935] Updated weights for policy 0, policy_version 48460 (0.0008) [2023-10-13 22:56:52,374][60935] Updated weights for policy 0, policy_version 48470 (0.0007) [2023-10-13 22:56:52,750][60935] Updated weights for policy 0, policy_version 48480 (0.0007) [2023-10-13 22:56:52,927][60934] Updated weights for policy 1, policy_version 48832 (0.0010) [2023-10-13 22:56:53,300][60934] Updated weights for policy 1, policy_version 48842 (0.0009) [2023-10-13 22:56:53,662][60934] Updated weights for policy 1, policy_version 48852 (0.0008) [2023-10-13 22:56:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 99975168. Throughput: 0: 1705.9, 1: 1695.4. Samples: 25007240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:56:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:56:56,260][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000048480_49643520.pth... [2023-10-13 22:56:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000048856_50331648.pth... [2023-10-13 22:56:56,290][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000046912_48037888.pth [2023-10-13 22:56:56,300][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000047256_48693248.pth [2023-10-13 22:56:56,824][60935] Updated weights for policy 0, policy_version 48490 (0.0009) [2023-10-13 22:56:57,190][60935] Updated weights for policy 0, policy_version 48500 (0.0012) [2023-10-13 22:56:57,554][60935] Updated weights for policy 0, policy_version 48510 (0.0010) [2023-10-13 22:56:57,660][60934] Updated weights for policy 1, policy_version 48862 (0.0009) [2023-10-13 22:56:58,025][60934] Updated weights for policy 1, policy_version 48872 (0.0008) [2023-10-13 22:56:58,398][60934] Updated weights for policy 1, policy_version 48882 (0.0007) [2023-10-13 22:57:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 100040704. Throughput: 0: 1693.6, 1: 1667.0. Samples: 25016456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:57:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:57:01,618][60935] Updated weights for policy 0, policy_version 48520 (0.0008) [2023-10-13 22:57:01,993][60935] Updated weights for policy 0, policy_version 48530 (0.0009) [2023-10-13 22:57:02,338][60934] Updated weights for policy 1, policy_version 48892 (0.0009) [2023-10-13 22:57:02,355][60935] Updated weights for policy 0, policy_version 48540 (0.0008) [2023-10-13 22:57:02,707][60934] Updated weights for policy 1, policy_version 48902 (0.0007) [2023-10-13 22:57:03,066][60934] Updated weights for policy 1, policy_version 48912 (0.0010) [2023-10-13 22:57:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 100106240. Throughput: 0: 1705.4, 1: 1696.2. Samples: 25037662. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 22:57:06,248][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:57:06,494][60935] Updated weights for policy 0, policy_version 48550 (0.0009) [2023-10-13 22:57:06,854][60935] Updated weights for policy 0, policy_version 48560 (0.0009) [2023-10-13 22:57:06,914][60934] Updated weights for policy 1, policy_version 48922 (0.0007) [2023-10-13 22:57:07,233][60935] Updated weights for policy 0, policy_version 48570 (0.0008) [2023-10-13 22:57:07,272][60934] Updated weights for policy 1, policy_version 48932 (0.0007) [2023-10-13 22:57:07,646][60934] Updated weights for policy 1, policy_version 48942 (0.0007) [2023-10-13 22:57:08,012][60934] Updated weights for policy 1, policy_version 48952 (0.0007) [2023-10-13 22:57:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 100171776. Throughput: 0: 1704.0, 1: 1715.3. Samples: 25059040. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 22:57:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:57:11,286][60935] Updated weights for policy 0, policy_version 48580 (0.0008) [2023-10-13 22:57:11,646][60935] Updated weights for policy 0, policy_version 48590 (0.0008) [2023-10-13 22:57:11,830][60934] Updated weights for policy 1, policy_version 48962 (0.0008) [2023-10-13 22:57:12,003][60935] Updated weights for policy 0, policy_version 48600 (0.0010) [2023-10-13 22:57:12,190][60934] Updated weights for policy 1, policy_version 48972 (0.0008) [2023-10-13 22:57:12,553][60934] Updated weights for policy 1, policy_version 48982 (0.0011) [2023-10-13 22:57:16,004][60935] Updated weights for policy 0, policy_version 48610 (0.0008) [2023-10-13 22:57:16,248][59943] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13551.5). Total num frames: 100237312. Throughput: 0: 1702.0, 1: 1694.8. Samples: 25068164. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 22:57:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:57:16,376][60935] Updated weights for policy 0, policy_version 48620 (0.0009) [2023-10-13 22:57:16,628][60934] Updated weights for policy 1, policy_version 48992 (0.0009) [2023-10-13 22:57:16,736][60935] Updated weights for policy 0, policy_version 48630 (0.0008) [2023-10-13 22:57:17,001][60934] Updated weights for policy 1, policy_version 49002 (0.0007) [2023-10-13 22:57:17,111][60935] Updated weights for policy 0, policy_version 48640 (0.0010) [2023-10-13 22:57:17,364][60934] Updated weights for policy 1, policy_version 49012 (0.0009) [2023-10-13 22:57:21,132][60935] Updated weights for policy 0, policy_version 48650 (0.0010) [2023-10-13 22:57:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 100302848. Throughput: 0: 1696.7, 1: 1721.4. Samples: 25088892. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 22:57:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:57:21,465][60934] Updated weights for policy 1, policy_version 49022 (0.0009) [2023-10-13 22:57:21,509][60935] Updated weights for policy 0, policy_version 48660 (0.0010) [2023-10-13 22:57:21,821][60934] Updated weights for policy 1, policy_version 49032 (0.0007) [2023-10-13 22:57:21,874][60935] Updated weights for policy 0, policy_version 48670 (0.0007) [2023-10-13 22:57:22,186][60934] Updated weights for policy 1, policy_version 49042 (0.0009) [2023-10-13 22:57:26,098][60935] Updated weights for policy 0, policy_version 48680 (0.0008) [2023-10-13 22:57:26,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 100368384. Throughput: 0: 1691.2, 1: 1715.0. Samples: 25109340. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 22:57:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:57:26,316][60934] Updated weights for policy 1, policy_version 49052 (0.0007) [2023-10-13 22:57:26,463][60935] Updated weights for policy 0, policy_version 48690 (0.0007) [2023-10-13 22:57:26,682][60934] Updated weights for policy 1, policy_version 49062 (0.0007) [2023-10-13 22:57:26,828][60935] Updated weights for policy 0, policy_version 48700 (0.0009) [2023-10-13 22:57:27,043][60934] Updated weights for policy 1, policy_version 49072 (0.0010) [2023-10-13 22:57:30,767][60935] Updated weights for policy 0, policy_version 48710 (0.0009) [2023-10-13 22:57:31,113][60934] Updated weights for policy 1, policy_version 49082 (0.0008) [2023-10-13 22:57:31,142][60935] Updated weights for policy 0, policy_version 48720 (0.0008) [2023-10-13 22:57:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 100433920. Throughput: 0: 1691.6, 1: 1701.2. Samples: 25118534. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 22:57:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:57:31,479][60934] Updated weights for policy 1, policy_version 49092 (0.0007) [2023-10-13 22:57:31,511][60935] Updated weights for policy 0, policy_version 48730 (0.0008) [2023-10-13 22:57:31,844][60934] Updated weights for policy 1, policy_version 49102 (0.0007) [2023-10-13 22:57:32,201][60934] Updated weights for policy 1, policy_version 49112 (0.0009) [2023-10-13 22:57:35,835][60935] Updated weights for policy 0, policy_version 48740 (0.0009) [2023-10-13 22:57:36,203][60935] Updated weights for policy 0, policy_version 48750 (0.0007) [2023-10-13 22:57:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 100499456. Throughput: 0: 1688.0, 1: 1714.1. Samples: 25139410. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-13 22:57:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:57:36,356][60934] Updated weights for policy 1, policy_version 49122 (0.0007) [2023-10-13 22:57:36,561][60935] Updated weights for policy 0, policy_version 48760 (0.0009) [2023-10-13 22:57:36,737][60934] Updated weights for policy 1, policy_version 49132 (0.0009) [2023-10-13 22:57:37,097][60934] Updated weights for policy 1, policy_version 49142 (0.0010) [2023-10-13 22:57:40,476][60935] Updated weights for policy 0, policy_version 48770 (0.0009) [2023-10-13 22:57:40,846][60935] Updated weights for policy 0, policy_version 48780 (0.0007) [2023-10-13 22:57:40,958][60934] Updated weights for policy 1, policy_version 49152 (0.0008) [2023-10-13 22:57:41,225][60935] Updated weights for policy 0, policy_version 48790 (0.0008) [2023-10-13 22:57:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 100564992. Throughput: 0: 1674.2, 1: 1716.1. Samples: 25159806. Policy #0 lag: (min: 29.0, avg: 30.3, max: 55.0) [2023-10-13 22:57:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-13 22:57:41,334][60934] Updated weights for policy 1, policy_version 49162 (0.0009) [2023-10-13 22:57:41,594][60935] Updated weights for policy 0, policy_version 48800 (0.0008) [2023-10-13 22:57:41,693][60934] Updated weights for policy 1, policy_version 49172 (0.0009) [2023-10-13 22:57:45,643][60935] Updated weights for policy 0, policy_version 48810 (0.0008) [2023-10-13 22:57:45,754][60934] Updated weights for policy 1, policy_version 49182 (0.0010) [2023-10-13 22:57:46,004][60935] Updated weights for policy 0, policy_version 48820 (0.0008) [2023-10-13 22:57:46,133][60934] Updated weights for policy 1, policy_version 49192 (0.0008) [2023-10-13 22:57:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 100630528. Throughput: 0: 1686.1, 1: 1710.7. Samples: 25169312. Policy #0 lag: (min: 29.0, avg: 30.3, max: 55.0) [2023-10-13 22:57:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-13 22:57:46,374][60935] Updated weights for policy 0, policy_version 48830 (0.0008) [2023-10-13 22:57:46,500][60934] Updated weights for policy 1, policy_version 49202 (0.0008) [2023-10-13 22:57:50,457][60935] Updated weights for policy 0, policy_version 48840 (0.0009) [2023-10-13 22:57:50,515][60934] Updated weights for policy 1, policy_version 49212 (0.0009) [2023-10-13 22:57:50,821][60935] Updated weights for policy 0, policy_version 48850 (0.0010) [2023-10-13 22:57:50,889][60934] Updated weights for policy 1, policy_version 49222 (0.0009) [2023-10-13 22:57:51,190][60935] Updated weights for policy 0, policy_version 48860 (0.0010) [2023-10-13 22:57:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 100696064. Throughput: 0: 1684.1, 1: 1702.0. Samples: 25190038. Policy #0 lag: (min: 29.0, avg: 30.3, max: 55.0) [2023-10-13 22:57:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:57:51,252][60934] Updated weights for policy 1, policy_version 49232 (0.0008) [2023-10-13 22:57:55,392][60934] Updated weights for policy 1, policy_version 49242 (0.0008) [2023-10-13 22:57:55,410][60935] Updated weights for policy 0, policy_version 48870 (0.0009) [2023-10-13 22:57:55,750][60934] Updated weights for policy 1, policy_version 49252 (0.0007) [2023-10-13 22:57:55,762][60935] Updated weights for policy 0, policy_version 48880 (0.0007) [2023-10-13 22:57:56,111][60934] Updated weights for policy 1, policy_version 49262 (0.0007) [2023-10-13 22:57:56,129][60935] Updated weights for policy 0, policy_version 48890 (0.0008) [2023-10-13 22:57:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 100761600. Throughput: 0: 1663.9, 1: 1692.4. Samples: 25210072. Policy #0 lag: (min: 29.0, avg: 30.3, max: 55.0) [2023-10-13 22:57:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:57:56,470][60934] Updated weights for policy 1, policy_version 49272 (0.0007) [2023-10-13 22:58:00,228][60935] Updated weights for policy 0, policy_version 48900 (0.0008) [2023-10-13 22:58:00,343][60934] Updated weights for policy 1, policy_version 49282 (0.0007) [2023-10-13 22:58:00,609][60935] Updated weights for policy 0, policy_version 48910 (0.0008) [2023-10-13 22:58:00,713][60934] Updated weights for policy 1, policy_version 49292 (0.0008) [2023-10-13 22:58:00,970][60935] Updated weights for policy 0, policy_version 48920 (0.0008) [2023-10-13 22:58:01,091][60934] Updated weights for policy 1, policy_version 49302 (0.0009) [2023-10-13 22:58:01,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 100859904. Throughput: 0: 1679.5, 1: 1698.6. Samples: 25220180. Policy #0 lag: (min: 29.0, avg: 30.3, max: 55.0) [2023-10-13 22:58:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:58:05,044][60935] Updated weights for policy 0, policy_version 48930 (0.0010) [2023-10-13 22:58:05,144][60934] Updated weights for policy 1, policy_version 49312 (0.0008) [2023-10-13 22:58:05,418][60935] Updated weights for policy 0, policy_version 48940 (0.0009) [2023-10-13 22:58:05,505][60934] Updated weights for policy 1, policy_version 49322 (0.0010) [2023-10-13 22:58:05,777][60935] Updated weights for policy 0, policy_version 48950 (0.0010) [2023-10-13 22:58:05,876][60934] Updated weights for policy 1, policy_version 49332 (0.0008) [2023-10-13 22:58:06,138][60935] Updated weights for policy 0, policy_version 48960 (0.0011) [2023-10-13 22:58:06,248][59943] Fps is (10 sec: 19660.5, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 100958208. Throughput: 0: 1684.5, 1: 1701.9. Samples: 25241280. Policy #0 lag: (min: 29.0, avg: 30.3, max: 55.0) [2023-10-13 22:58:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:58:09,638][60934] Updated weights for policy 1, policy_version 49342 (0.0008) [2023-10-13 22:58:09,999][60934] Updated weights for policy 1, policy_version 49352 (0.0007) [2023-10-13 22:58:10,307][60935] Updated weights for policy 0, policy_version 48970 (0.0009) [2023-10-13 22:58:10,372][60934] Updated weights for policy 1, policy_version 49362 (0.0009) [2023-10-13 22:58:10,681][60935] Updated weights for policy 0, policy_version 48980 (0.0009) [2023-10-13 22:58:11,049][60935] Updated weights for policy 0, policy_version 48990 (0.0011) [2023-10-13 22:58:11,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 101023744. Throughput: 0: 1662.4, 1: 1690.3. Samples: 25260214. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 22:58:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 22:58:14,278][60934] Updated weights for policy 1, policy_version 49372 (0.0008) [2023-10-13 22:58:14,649][60934] Updated weights for policy 1, policy_version 49382 (0.0008) [2023-10-13 22:58:15,019][60934] Updated weights for policy 1, policy_version 49392 (0.0007) [2023-10-13 22:58:15,209][60935] Updated weights for policy 0, policy_version 49000 (0.0009) [2023-10-13 22:58:15,586][60935] Updated weights for policy 0, policy_version 49010 (0.0011) [2023-10-13 22:58:15,952][60935] Updated weights for policy 0, policy_version 49020 (0.0008) [2023-10-13 22:58:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 101089280. Throughput: 0: 1682.5, 1: 1723.8. Samples: 25271816. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 22:58:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:58:18,992][60934] Updated weights for policy 1, policy_version 49402 (0.0009) [2023-10-13 22:58:19,349][60934] Updated weights for policy 1, policy_version 49412 (0.0011) [2023-10-13 22:58:19,720][60934] Updated weights for policy 1, policy_version 49422 (0.0010) [2023-10-13 22:58:19,939][60935] Updated weights for policy 0, policy_version 49030 (0.0010) [2023-10-13 22:58:20,083][60934] Updated weights for policy 1, policy_version 49432 (0.0008) [2023-10-13 22:58:20,309][60935] Updated weights for policy 0, policy_version 49040 (0.0009) [2023-10-13 22:58:20,677][60935] Updated weights for policy 0, policy_version 49050 (0.0009) [2023-10-13 22:58:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 101154816. Throughput: 0: 1678.9, 1: 1709.6. Samples: 25291894. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 22:58:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:58:24,068][60934] Updated weights for policy 1, policy_version 49442 (0.0007) [2023-10-13 22:58:24,436][60934] Updated weights for policy 1, policy_version 49452 (0.0009) [2023-10-13 22:58:24,526][60935] Updated weights for policy 0, policy_version 49060 (0.0009) [2023-10-13 22:58:24,806][60934] Updated weights for policy 1, policy_version 49462 (0.0008) [2023-10-13 22:58:24,888][60935] Updated weights for policy 0, policy_version 49070 (0.0008) [2023-10-13 22:58:25,258][60935] Updated weights for policy 0, policy_version 49080 (0.0011) [2023-10-13 22:58:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 101220352. Throughput: 0: 1668.3, 1: 1695.4. Samples: 25311174. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 22:58:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:58:28,927][60934] Updated weights for policy 1, policy_version 49472 (0.0008) [2023-10-13 22:58:29,296][60934] Updated weights for policy 1, policy_version 49482 (0.0010) [2023-10-13 22:58:29,453][60935] Updated weights for policy 0, policy_version 49090 (0.0009) [2023-10-13 22:58:29,668][60934] Updated weights for policy 1, policy_version 49492 (0.0009) [2023-10-13 22:58:29,810][60935] Updated weights for policy 0, policy_version 49100 (0.0008) [2023-10-13 22:58:30,186][60935] Updated weights for policy 0, policy_version 49110 (0.0008) [2023-10-13 22:58:30,555][60935] Updated weights for policy 0, policy_version 49120 (0.0010) [2023-10-13 22:58:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 101285888. Throughput: 0: 1684.8, 1: 1723.1. Samples: 25322664. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 22:58:31,249][59943] Avg episode reward: [(0, '-0.020'), (1, '-0.010')] [2023-10-13 22:58:33,563][60934] Updated weights for policy 1, policy_version 49502 (0.0008) [2023-10-13 22:58:33,936][60934] Updated weights for policy 1, policy_version 49512 (0.0007) [2023-10-13 22:58:34,296][60934] Updated weights for policy 1, policy_version 49522 (0.0009) [2023-10-13 22:58:34,675][60935] Updated weights for policy 0, policy_version 49130 (0.0009) [2023-10-13 22:58:35,053][60935] Updated weights for policy 0, policy_version 49140 (0.0007) [2023-10-13 22:58:35,413][60935] Updated weights for policy 0, policy_version 49150 (0.0008) [2023-10-13 22:58:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 101351424. Throughput: 0: 1671.8, 1: 1702.0. Samples: 25341860. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 22:58:36,249][59943] Avg episode reward: [(0, '-0.020'), (1, '-0.010')] [2023-10-13 22:58:38,176][60934] Updated weights for policy 1, policy_version 49532 (0.0009) [2023-10-13 22:58:38,548][60934] Updated weights for policy 1, policy_version 49542 (0.0008) [2023-10-13 22:58:38,924][60934] Updated weights for policy 1, policy_version 49552 (0.0008) [2023-10-13 22:58:39,402][60935] Updated weights for policy 0, policy_version 49160 (0.0008) [2023-10-13 22:58:39,774][60935] Updated weights for policy 0, policy_version 49170 (0.0009) [2023-10-13 22:58:40,156][60935] Updated weights for policy 0, policy_version 49180 (0.0011) [2023-10-13 22:58:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 101416960. Throughput: 0: 1672.4, 1: 1702.7. Samples: 25361950. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 22:58:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:58:42,933][60934] Updated weights for policy 1, policy_version 49562 (0.0009) [2023-10-13 22:58:43,310][60934] Updated weights for policy 1, policy_version 49572 (0.0007) [2023-10-13 22:58:43,676][60934] Updated weights for policy 1, policy_version 49582 (0.0007) [2023-10-13 22:58:44,037][60934] Updated weights for policy 1, policy_version 49592 (0.0008) [2023-10-13 22:58:44,171][60935] Updated weights for policy 0, policy_version 49190 (0.0010) [2023-10-13 22:58:44,538][60935] Updated weights for policy 0, policy_version 49200 (0.0010) [2023-10-13 22:58:44,911][60935] Updated weights for policy 0, policy_version 49210 (0.0009) [2023-10-13 22:58:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 101482496. Throughput: 0: 1686.5, 1: 1709.6. Samples: 25373008. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-13 22:58:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:58:47,976][60934] Updated weights for policy 1, policy_version 49602 (0.0008) [2023-10-13 22:58:48,344][60934] Updated weights for policy 1, policy_version 49612 (0.0009) [2023-10-13 22:58:48,717][60934] Updated weights for policy 1, policy_version 49622 (0.0009) [2023-10-13 22:58:48,816][60935] Updated weights for policy 0, policy_version 49220 (0.0008) [2023-10-13 22:58:49,179][60935] Updated weights for policy 0, policy_version 49230 (0.0010) [2023-10-13 22:58:49,546][60935] Updated weights for policy 0, policy_version 49240 (0.0011) [2023-10-13 22:58:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 101548032. Throughput: 0: 1656.6, 1: 1698.3. Samples: 25392250. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-13 22:58:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:58:52,588][60934] Updated weights for policy 1, policy_version 49632 (0.0010) [2023-10-13 22:58:52,949][60934] Updated weights for policy 1, policy_version 49642 (0.0009) [2023-10-13 22:58:53,317][60934] Updated weights for policy 1, policy_version 49652 (0.0010) [2023-10-13 22:58:53,533][60935] Updated weights for policy 0, policy_version 49250 (0.0009) [2023-10-13 22:58:53,899][60935] Updated weights for policy 0, policy_version 49260 (0.0009) [2023-10-13 22:58:54,273][60935] Updated weights for policy 0, policy_version 49270 (0.0008) [2023-10-13 22:58:54,635][60935] Updated weights for policy 0, policy_version 49280 (0.0010) [2023-10-13 22:58:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 101613568. Throughput: 0: 1681.7, 1: 1719.6. Samples: 25413272. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-13 22:58:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:58:56,259][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000049280_50462720.pth... [2023-10-13 22:58:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000049656_51150848.pth... [2023-10-13 22:58:56,296][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000048056_49512448.pth [2023-10-13 22:58:56,299][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000047712_48857088.pth [2023-10-13 22:58:57,427][60934] Updated weights for policy 1, policy_version 49662 (0.0007) [2023-10-13 22:58:57,795][60934] Updated weights for policy 1, policy_version 49672 (0.0008) [2023-10-13 22:58:58,157][60934] Updated weights for policy 1, policy_version 49682 (0.0011) [2023-10-13 22:58:58,820][60935] Updated weights for policy 0, policy_version 49290 (0.0009) [2023-10-13 22:58:59,188][60935] Updated weights for policy 0, policy_version 49300 (0.0012) [2023-10-13 22:58:59,556][60935] Updated weights for policy 0, policy_version 49310 (0.0009) [2023-10-13 22:59:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 101679104. Throughput: 0: 1679.8, 1: 1686.0. Samples: 25423278. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-13 22:59:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:59:02,224][60934] Updated weights for policy 1, policy_version 49692 (0.0009) [2023-10-13 22:59:02,596][60934] Updated weights for policy 1, policy_version 49702 (0.0009) [2023-10-13 22:59:02,957][60934] Updated weights for policy 1, policy_version 49712 (0.0007) [2023-10-13 22:59:03,667][60935] Updated weights for policy 0, policy_version 49320 (0.0010) [2023-10-13 22:59:04,035][60935] Updated weights for policy 0, policy_version 49330 (0.0009) [2023-10-13 22:59:04,404][60935] Updated weights for policy 0, policy_version 49340 (0.0010) [2023-10-13 22:59:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 101744640. Throughput: 0: 1661.5, 1: 1700.5. Samples: 25443184. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-13 22:59:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:59:07,158][60934] Updated weights for policy 1, policy_version 49722 (0.0008) [2023-10-13 22:59:07,520][60934] Updated weights for policy 1, policy_version 49732 (0.0009) [2023-10-13 22:59:07,884][60934] Updated weights for policy 1, policy_version 49742 (0.0009) [2023-10-13 22:59:08,245][60934] Updated weights for policy 1, policy_version 49752 (0.0008) [2023-10-13 22:59:08,298][60935] Updated weights for policy 0, policy_version 49350 (0.0009) [2023-10-13 22:59:08,661][60935] Updated weights for policy 0, policy_version 49360 (0.0010) [2023-10-13 22:59:09,031][60935] Updated weights for policy 0, policy_version 49370 (0.0010) [2023-10-13 22:59:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 101810176. Throughput: 0: 1686.1, 1: 1712.5. Samples: 25464112. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-13 22:59:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:59:12,310][60934] Updated weights for policy 1, policy_version 49762 (0.0007) [2023-10-13 22:59:12,686][60934] Updated weights for policy 1, policy_version 49772 (0.0007) [2023-10-13 22:59:13,054][60934] Updated weights for policy 1, policy_version 49782 (0.0007) [2023-10-13 22:59:13,354][60935] Updated weights for policy 0, policy_version 49380 (0.0010) [2023-10-13 22:59:13,716][60935] Updated weights for policy 0, policy_version 49390 (0.0008) [2023-10-13 22:59:14,093][60935] Updated weights for policy 0, policy_version 49400 (0.0007) [2023-10-13 22:59:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 101875712. Throughput: 0: 1672.4, 1: 1685.5. Samples: 25473770. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-13 22:59:16,249][59943] Avg episode reward: [(0, '-0.020'), (1, '-0.010')] [2023-10-13 22:59:16,889][60934] Updated weights for policy 1, policy_version 49792 (0.0010) [2023-10-13 22:59:17,255][60934] Updated weights for policy 1, policy_version 49802 (0.0008) [2023-10-13 22:59:17,624][60934] Updated weights for policy 1, policy_version 49812 (0.0010) [2023-10-13 22:59:18,023][60935] Updated weights for policy 0, policy_version 49410 (0.0008) [2023-10-13 22:59:18,393][60935] Updated weights for policy 0, policy_version 49420 (0.0009) [2023-10-13 22:59:18,768][60935] Updated weights for policy 0, policy_version 49430 (0.0010) [2023-10-13 22:59:19,140][60935] Updated weights for policy 0, policy_version 49440 (0.0011) [2023-10-13 22:59:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 101941248. Throughput: 0: 1672.1, 1: 1713.1. Samples: 25494190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:59:21,249][59943] Avg episode reward: [(0, '-0.020'), (1, '-0.010')] [2023-10-13 22:59:21,659][60934] Updated weights for policy 1, policy_version 49822 (0.0008) [2023-10-13 22:59:22,026][60934] Updated weights for policy 1, policy_version 49832 (0.0008) [2023-10-13 22:59:22,384][60934] Updated weights for policy 1, policy_version 49842 (0.0007) [2023-10-13 22:59:23,185][60935] Updated weights for policy 0, policy_version 49450 (0.0008) [2023-10-13 22:59:23,555][60935] Updated weights for policy 0, policy_version 49460 (0.0008) [2023-10-13 22:59:23,929][60935] Updated weights for policy 0, policy_version 49470 (0.0007) [2023-10-13 22:59:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 102006784. Throughput: 0: 1692.5, 1: 1720.9. Samples: 25515554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:59:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 22:59:26,370][60934] Updated weights for policy 1, policy_version 49852 (0.0008) [2023-10-13 22:59:26,745][60934] Updated weights for policy 1, policy_version 49862 (0.0009) [2023-10-13 22:59:27,101][60934] Updated weights for policy 1, policy_version 49872 (0.0008) [2023-10-13 22:59:27,927][60935] Updated weights for policy 0, policy_version 49480 (0.0009) [2023-10-13 22:59:28,304][60935] Updated weights for policy 0, policy_version 49490 (0.0009) [2023-10-13 22:59:28,669][60935] Updated weights for policy 0, policy_version 49500 (0.0009) [2023-10-13 22:59:31,103][60934] Updated weights for policy 1, policy_version 49882 (0.0009) [2023-10-13 22:59:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 102072320. Throughput: 0: 1665.7, 1: 1706.7. Samples: 25524764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:59:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:59:31,473][60934] Updated weights for policy 1, policy_version 49892 (0.0007) [2023-10-13 22:59:31,835][60934] Updated weights for policy 1, policy_version 49902 (0.0008) [2023-10-13 22:59:32,209][60934] Updated weights for policy 1, policy_version 49912 (0.0008) [2023-10-13 22:59:32,745][60935] Updated weights for policy 0, policy_version 49510 (0.0007) [2023-10-13 22:59:33,101][60935] Updated weights for policy 0, policy_version 49520 (0.0007) [2023-10-13 22:59:33,470][60935] Updated weights for policy 0, policy_version 49530 (0.0007) [2023-10-13 22:59:36,203][60934] Updated weights for policy 1, policy_version 49922 (0.0008) [2023-10-13 22:59:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 102137856. Throughput: 0: 1689.7, 1: 1717.3. Samples: 25545566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:59:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:59:36,568][60934] Updated weights for policy 1, policy_version 49932 (0.0007) [2023-10-13 22:59:36,940][60934] Updated weights for policy 1, policy_version 49942 (0.0008) [2023-10-13 22:59:37,542][60935] Updated weights for policy 0, policy_version 49540 (0.0008) [2023-10-13 22:59:37,926][60935] Updated weights for policy 0, policy_version 49550 (0.0009) [2023-10-13 22:59:38,299][60935] Updated weights for policy 0, policy_version 49560 (0.0010) [2023-10-13 22:59:40,896][60934] Updated weights for policy 1, policy_version 49952 (0.0010) [2023-10-13 22:59:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 102203392. Throughput: 0: 1692.8, 1: 1714.2. Samples: 25566586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:59:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:59:41,270][60934] Updated weights for policy 1, policy_version 49962 (0.0011) [2023-10-13 22:59:41,628][60934] Updated weights for policy 1, policy_version 49972 (0.0010) [2023-10-13 22:59:42,281][60935] Updated weights for policy 0, policy_version 49570 (0.0007) [2023-10-13 22:59:42,639][60935] Updated weights for policy 0, policy_version 49580 (0.0009) [2023-10-13 22:59:43,010][60935] Updated weights for policy 0, policy_version 49590 (0.0008) [2023-10-13 22:59:43,380][60935] Updated weights for policy 0, policy_version 49600 (0.0009) [2023-10-13 22:59:45,738][60934] Updated weights for policy 1, policy_version 49982 (0.0009) [2023-10-13 22:59:46,103][60934] Updated weights for policy 1, policy_version 49992 (0.0008) [2023-10-13 22:59:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 102268928. Throughput: 0: 1672.9, 1: 1715.4. Samples: 25575750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:59:46,249][59943] Avg episode reward: [(0, '-0.070'), (1, '0.000')] [2023-10-13 22:59:46,466][60934] Updated weights for policy 1, policy_version 50002 (0.0007) [2023-10-13 22:59:47,411][60935] Updated weights for policy 0, policy_version 49610 (0.0010) [2023-10-13 22:59:47,791][60935] Updated weights for policy 0, policy_version 49620 (0.0010) [2023-10-13 22:59:48,165][60935] Updated weights for policy 0, policy_version 49630 (0.0009) [2023-10-13 22:59:50,433][60934] Updated weights for policy 1, policy_version 50012 (0.0007) [2023-10-13 22:59:50,798][60934] Updated weights for policy 1, policy_version 50022 (0.0008) [2023-10-13 22:59:51,164][60934] Updated weights for policy 1, policy_version 50032 (0.0011) [2023-10-13 22:59:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 102334464. Throughput: 0: 1696.8, 1: 1716.6. Samples: 25596790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 22:59:51,249][59943] Avg episode reward: [(0, '-0.070'), (1, '0.000')] [2023-10-13 22:59:52,027][60935] Updated weights for policy 0, policy_version 49640 (0.0009) [2023-10-13 22:59:52,404][60935] Updated weights for policy 0, policy_version 49650 (0.0009) [2023-10-13 22:59:52,773][60935] Updated weights for policy 0, policy_version 49660 (0.0010) [2023-10-13 22:59:55,241][60934] Updated weights for policy 1, policy_version 50042 (0.0010) [2023-10-13 22:59:55,606][60934] Updated weights for policy 1, policy_version 50052 (0.0009) [2023-10-13 22:59:55,985][60934] Updated weights for policy 1, policy_version 50062 (0.0008) [2023-10-13 22:59:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 102400000. Throughput: 0: 1697.3, 1: 1707.4. Samples: 25617324. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-13 22:59:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 22:59:56,343][60934] Updated weights for policy 1, policy_version 50072 (0.0008) [2023-10-13 22:59:56,978][60935] Updated weights for policy 0, policy_version 49670 (0.0010) [2023-10-13 22:59:57,360][60935] Updated weights for policy 0, policy_version 49680 (0.0008) [2023-10-13 22:59:57,731][60935] Updated weights for policy 0, policy_version 49690 (0.0010) [2023-10-13 23:00:00,303][60934] Updated weights for policy 1, policy_version 50082 (0.0008) [2023-10-13 23:00:00,673][60934] Updated weights for policy 1, policy_version 50092 (0.0008) [2023-10-13 23:00:01,048][60934] Updated weights for policy 1, policy_version 50102 (0.0008) [2023-10-13 23:00:01,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 102498304. Throughput: 0: 1683.8, 1: 1721.7. Samples: 25627016. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-13 23:00:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:00:01,841][60935] Updated weights for policy 0, policy_version 49700 (0.0008) [2023-10-13 23:00:02,215][60935] Updated weights for policy 0, policy_version 49710 (0.0010) [2023-10-13 23:00:02,573][60935] Updated weights for policy 0, policy_version 49720 (0.0009) [2023-10-13 23:00:05,164][60934] Updated weights for policy 1, policy_version 50112 (0.0008) [2023-10-13 23:00:05,526][60934] Updated weights for policy 1, policy_version 50122 (0.0010) [2023-10-13 23:00:05,903][60934] Updated weights for policy 1, policy_version 50132 (0.0010) [2023-10-13 23:00:06,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 102563840. Throughput: 0: 1695.7, 1: 1714.3. Samples: 25647640. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-13 23:00:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:00:06,581][60935] Updated weights for policy 0, policy_version 49730 (0.0010) [2023-10-13 23:00:06,947][60935] Updated weights for policy 0, policy_version 49740 (0.0010) [2023-10-13 23:00:07,320][60935] Updated weights for policy 0, policy_version 49750 (0.0011) [2023-10-13 23:00:07,683][60935] Updated weights for policy 0, policy_version 49760 (0.0008) [2023-10-13 23:00:09,963][60934] Updated weights for policy 1, policy_version 50142 (0.0010) [2023-10-13 23:00:10,335][60934] Updated weights for policy 1, policy_version 50152 (0.0010) [2023-10-13 23:00:10,697][60934] Updated weights for policy 1, policy_version 50162 (0.0010) [2023-10-13 23:00:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 102629376. Throughput: 0: 1696.9, 1: 1685.4. Samples: 25667754. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-13 23:00:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:00:11,516][60935] Updated weights for policy 0, policy_version 49770 (0.0009) [2023-10-13 23:00:11,887][60935] Updated weights for policy 0, policy_version 49780 (0.0010) [2023-10-13 23:00:12,256][60935] Updated weights for policy 0, policy_version 49790 (0.0008) [2023-10-13 23:00:14,478][60934] Updated weights for policy 1, policy_version 50172 (0.0009) [2023-10-13 23:00:14,842][60934] Updated weights for policy 1, policy_version 50182 (0.0011) [2023-10-13 23:00:15,210][60934] Updated weights for policy 1, policy_version 50192 (0.0009) [2023-10-13 23:00:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 102694912. Throughput: 0: 1696.8, 1: 1709.4. Samples: 25678046. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-13 23:00:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:00:16,346][60935] Updated weights for policy 0, policy_version 49800 (0.0008) [2023-10-13 23:00:16,711][60935] Updated weights for policy 0, policy_version 49810 (0.0011) [2023-10-13 23:00:17,077][60935] Updated weights for policy 0, policy_version 49820 (0.0010) [2023-10-13 23:00:19,180][60934] Updated weights for policy 1, policy_version 50202 (0.0008) [2023-10-13 23:00:19,551][60934] Updated weights for policy 1, policy_version 50212 (0.0010) [2023-10-13 23:00:19,922][60934] Updated weights for policy 1, policy_version 50222 (0.0010) [2023-10-13 23:00:20,286][60934] Updated weights for policy 1, policy_version 50232 (0.0007) [2023-10-13 23:00:21,010][60935] Updated weights for policy 0, policy_version 49830 (0.0008) [2023-10-13 23:00:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 102760448. Throughput: 0: 1703.1, 1: 1702.5. Samples: 25698818. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-13 23:00:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:00:21,384][60935] Updated weights for policy 0, policy_version 49840 (0.0007) [2023-10-13 23:00:21,744][60935] Updated weights for policy 0, policy_version 49850 (0.0011) [2023-10-13 23:00:24,188][60934] Updated weights for policy 1, policy_version 50242 (0.0009) [2023-10-13 23:00:24,549][60934] Updated weights for policy 1, policy_version 50252 (0.0007) [2023-10-13 23:00:24,924][60934] Updated weights for policy 1, policy_version 50262 (0.0008) [2023-10-13 23:00:25,916][60935] Updated weights for policy 0, policy_version 49860 (0.0008) [2023-10-13 23:00:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 102825984. Throughput: 0: 1698.1, 1: 1688.4. Samples: 25718978. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-13 23:00:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:00:26,280][60935] Updated weights for policy 0, policy_version 49870 (0.0007) [2023-10-13 23:00:26,646][60935] Updated weights for policy 0, policy_version 49880 (0.0009) [2023-10-13 23:00:29,014][60934] Updated weights for policy 1, policy_version 50272 (0.0008) [2023-10-13 23:00:29,379][60934] Updated weights for policy 1, policy_version 50282 (0.0009) [2023-10-13 23:00:29,745][60934] Updated weights for policy 1, policy_version 50292 (0.0007) [2023-10-13 23:00:30,661][60935] Updated weights for policy 0, policy_version 49890 (0.0010) [2023-10-13 23:00:31,028][60935] Updated weights for policy 0, policy_version 49900 (0.0009) [2023-10-13 23:00:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 102891520. Throughput: 0: 1701.5, 1: 1719.2. Samples: 25729678. Policy #0 lag: (min: 24.0, avg: 48.8, max: 56.0) [2023-10-13 23:00:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:00:31,409][60935] Updated weights for policy 0, policy_version 49910 (0.0009) [2023-10-13 23:00:31,775][60935] Updated weights for policy 0, policy_version 49920 (0.0008) [2023-10-13 23:00:33,746][60934] Updated weights for policy 1, policy_version 50302 (0.0009) [2023-10-13 23:00:34,102][60934] Updated weights for policy 1, policy_version 50312 (0.0009) [2023-10-13 23:00:34,469][60934] Updated weights for policy 1, policy_version 50322 (0.0009) [2023-10-13 23:00:36,004][60935] Updated weights for policy 0, policy_version 49930 (0.0007) [2023-10-13 23:00:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 102957056. Throughput: 0: 1693.8, 1: 1692.8. Samples: 25749184. Policy #0 lag: (min: 24.0, avg: 48.8, max: 56.0) [2023-10-13 23:00:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:00:36,376][60935] Updated weights for policy 0, policy_version 49940 (0.0009) [2023-10-13 23:00:36,755][60935] Updated weights for policy 0, policy_version 49950 (0.0011) [2023-10-13 23:00:38,537][60934] Updated weights for policy 1, policy_version 50332 (0.0008) [2023-10-13 23:00:38,913][60934] Updated weights for policy 1, policy_version 50342 (0.0007) [2023-10-13 23:00:39,269][60934] Updated weights for policy 1, policy_version 50352 (0.0009) [2023-10-13 23:00:40,675][60935] Updated weights for policy 0, policy_version 49960 (0.0009) [2023-10-13 23:00:41,035][60935] Updated weights for policy 0, policy_version 49970 (0.0008) [2023-10-13 23:00:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 103022592. Throughput: 0: 1682.4, 1: 1696.9. Samples: 25769394. Policy #0 lag: (min: 24.0, avg: 48.8, max: 56.0) [2023-10-13 23:00:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:00:41,408][60935] Updated weights for policy 0, policy_version 49980 (0.0010) [2023-10-13 23:00:43,368][60934] Updated weights for policy 1, policy_version 50362 (0.0009) [2023-10-13 23:00:43,733][60934] Updated weights for policy 1, policy_version 50372 (0.0011) [2023-10-13 23:00:44,100][60934] Updated weights for policy 1, policy_version 50382 (0.0010) [2023-10-13 23:00:44,468][60934] Updated weights for policy 1, policy_version 50392 (0.0007) [2023-10-13 23:00:45,420][60935] Updated weights for policy 0, policy_version 49990 (0.0008) [2023-10-13 23:00:45,800][60935] Updated weights for policy 0, policy_version 50000 (0.0009) [2023-10-13 23:00:46,172][60935] Updated weights for policy 0, policy_version 50010 (0.0010) [2023-10-13 23:00:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 103088128. Throughput: 0: 1694.5, 1: 1709.2. Samples: 25780182. Policy #0 lag: (min: 24.0, avg: 48.8, max: 56.0) [2023-10-13 23:00:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:00:48,529][60934] Updated weights for policy 1, policy_version 50402 (0.0009) [2023-10-13 23:00:48,884][60934] Updated weights for policy 1, policy_version 50412 (0.0009) [2023-10-13 23:00:49,250][60934] Updated weights for policy 1, policy_version 50422 (0.0011) [2023-10-13 23:00:50,357][60935] Updated weights for policy 0, policy_version 50020 (0.0009) [2023-10-13 23:00:50,720][60935] Updated weights for policy 0, policy_version 50030 (0.0009) [2023-10-13 23:00:51,091][60935] Updated weights for policy 0, policy_version 50040 (0.0009) [2023-10-13 23:00:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 103153664. Throughput: 0: 1693.6, 1: 1690.7. Samples: 25799934. Policy #0 lag: (min: 24.0, avg: 48.8, max: 56.0) [2023-10-13 23:00:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:00:53,129][60934] Updated weights for policy 1, policy_version 50432 (0.0007) [2023-10-13 23:00:53,491][60934] Updated weights for policy 1, policy_version 50442 (0.0008) [2023-10-13 23:00:53,873][60934] Updated weights for policy 1, policy_version 50452 (0.0009) [2023-10-13 23:00:55,028][60935] Updated weights for policy 0, policy_version 50050 (0.0008) [2023-10-13 23:00:55,397][60935] Updated weights for policy 0, policy_version 50060 (0.0010) [2023-10-13 23:00:55,767][60935] Updated weights for policy 0, policy_version 50070 (0.0010) [2023-10-13 23:00:56,130][60935] Updated weights for policy 0, policy_version 50080 (0.0012) [2023-10-13 23:00:56,248][59943] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 103251968. Throughput: 0: 1671.6, 1: 1716.7. Samples: 25820230. Policy #0 lag: (min: 24.0, avg: 48.8, max: 56.0) [2023-10-13 23:00:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:00:56,258][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000050456_51970048.pth... [2023-10-13 23:00:56,258][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000050080_51281920.pth... [2023-10-13 23:00:56,296][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000048856_50331648.pth [2023-10-13 23:00:56,298][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000048480_49643520.pth [2023-10-13 23:00:57,789][60934] Updated weights for policy 1, policy_version 50462 (0.0008) [2023-10-13 23:00:58,162][60934] Updated weights for policy 1, policy_version 50472 (0.0008) [2023-10-13 23:00:58,522][60934] Updated weights for policy 1, policy_version 50482 (0.0010) [2023-10-13 23:01:00,138][60935] Updated weights for policy 0, policy_version 50090 (0.0009) [2023-10-13 23:01:00,511][60935] Updated weights for policy 0, policy_version 50100 (0.0009) [2023-10-13 23:01:00,879][60935] Updated weights for policy 0, policy_version 50110 (0.0009) [2023-10-13 23:01:01,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 103317504. Throughput: 0: 1691.7, 1: 1698.4. Samples: 25830600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:01:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:01:02,562][60934] Updated weights for policy 1, policy_version 50492 (0.0010) [2023-10-13 23:01:02,935][60934] Updated weights for policy 1, policy_version 50502 (0.0010) [2023-10-13 23:01:03,306][60934] Updated weights for policy 1, policy_version 50512 (0.0008) [2023-10-13 23:01:05,015][60935] Updated weights for policy 0, policy_version 50120 (0.0009) [2023-10-13 23:01:05,389][60935] Updated weights for policy 0, policy_version 50130 (0.0010) [2023-10-13 23:01:05,757][60935] Updated weights for policy 0, policy_version 50140 (0.0009) [2023-10-13 23:01:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 103383040. Throughput: 0: 1684.1, 1: 1701.2. Samples: 25851158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:01:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:01:07,144][60934] Updated weights for policy 1, policy_version 50522 (0.0008) [2023-10-13 23:01:07,512][60934] Updated weights for policy 1, policy_version 50532 (0.0007) [2023-10-13 23:01:07,873][60934] Updated weights for policy 1, policy_version 50542 (0.0007) [2023-10-13 23:01:08,245][60934] Updated weights for policy 1, policy_version 50552 (0.0007) [2023-10-13 23:01:09,754][60935] Updated weights for policy 0, policy_version 50150 (0.0007) [2023-10-13 23:01:10,123][60935] Updated weights for policy 0, policy_version 50160 (0.0011) [2023-10-13 23:01:10,497][60935] Updated weights for policy 0, policy_version 50170 (0.0008) [2023-10-13 23:01:11,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 103448576. Throughput: 0: 1663.7, 1: 1719.7. Samples: 25871232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:01:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:01:12,367][60934] Updated weights for policy 1, policy_version 50562 (0.0009) [2023-10-13 23:01:12,736][60934] Updated weights for policy 1, policy_version 50572 (0.0009) [2023-10-13 23:01:13,103][60934] Updated weights for policy 1, policy_version 50582 (0.0007) [2023-10-13 23:01:14,480][60935] Updated weights for policy 0, policy_version 50180 (0.0009) [2023-10-13 23:01:14,865][60935] Updated weights for policy 0, policy_version 50190 (0.0009) [2023-10-13 23:01:15,239][60935] Updated weights for policy 0, policy_version 50200 (0.0008) [2023-10-13 23:01:16,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 103514112. Throughput: 0: 1690.4, 1: 1685.3. Samples: 25881584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:01:16,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:01:16,994][60934] Updated weights for policy 1, policy_version 50592 (0.0008) [2023-10-13 23:01:17,359][60934] Updated weights for policy 1, policy_version 50602 (0.0008) [2023-10-13 23:01:17,729][60934] Updated weights for policy 1, policy_version 50612 (0.0007) [2023-10-13 23:01:19,127][60935] Updated weights for policy 0, policy_version 50210 (0.0010) [2023-10-13 23:01:19,495][60935] Updated weights for policy 0, policy_version 50220 (0.0007) [2023-10-13 23:01:19,863][60935] Updated weights for policy 0, policy_version 50230 (0.0007) [2023-10-13 23:01:20,233][60935] Updated weights for policy 0, policy_version 50240 (0.0009) [2023-10-13 23:01:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 103579648. Throughput: 0: 1678.0, 1: 1717.1. Samples: 25901966. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:01:21,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:01:21,589][60934] Updated weights for policy 1, policy_version 50622 (0.0009) [2023-10-13 23:01:21,950][60934] Updated weights for policy 1, policy_version 50632 (0.0008) [2023-10-13 23:01:22,315][60934] Updated weights for policy 1, policy_version 50642 (0.0007) [2023-10-13 23:01:24,265][60935] Updated weights for policy 0, policy_version 50250 (0.0007) [2023-10-13 23:01:24,628][60935] Updated weights for policy 0, policy_version 50260 (0.0008) [2023-10-13 23:01:24,995][60935] Updated weights for policy 0, policy_version 50270 (0.0009) [2023-10-13 23:01:26,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 103645184. Throughput: 0: 1676.8, 1: 1727.3. Samples: 25922578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:01:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:01:26,267][60934] Updated weights for policy 1, policy_version 50652 (0.0008) [2023-10-13 23:01:26,640][60934] Updated weights for policy 1, policy_version 50662 (0.0007) [2023-10-13 23:01:27,006][60934] Updated weights for policy 1, policy_version 50672 (0.0008) [2023-10-13 23:01:28,975][60935] Updated weights for policy 0, policy_version 50280 (0.0008) [2023-10-13 23:01:29,339][60935] Updated weights for policy 0, policy_version 50290 (0.0007) [2023-10-13 23:01:29,712][60935] Updated weights for policy 0, policy_version 50300 (0.0008) [2023-10-13 23:01:30,881][60934] Updated weights for policy 1, policy_version 50682 (0.0007) [2023-10-13 23:01:31,249][60934] Updated weights for policy 1, policy_version 50692 (0.0009) [2023-10-13 23:01:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 103710720. Throughput: 0: 1694.6, 1: 1701.2. Samples: 25932994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:01:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:01:31,624][60934] Updated weights for policy 1, policy_version 50702 (0.0008) [2023-10-13 23:01:31,985][60934] Updated weights for policy 1, policy_version 50712 (0.0009) [2023-10-13 23:01:33,671][60935] Updated weights for policy 0, policy_version 50310 (0.0010) [2023-10-13 23:01:34,041][60935] Updated weights for policy 0, policy_version 50320 (0.0010) [2023-10-13 23:01:34,409][60935] Updated weights for policy 0, policy_version 50330 (0.0007) [2023-10-13 23:01:35,994][60934] Updated weights for policy 1, policy_version 50722 (0.0007) [2023-10-13 23:01:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 103776256. Throughput: 0: 1676.0, 1: 1729.4. Samples: 25953176. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 23:01:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:01:36,357][60934] Updated weights for policy 1, policy_version 50732 (0.0007) [2023-10-13 23:01:36,734][60934] Updated weights for policy 1, policy_version 50742 (0.0008) [2023-10-13 23:01:38,591][60935] Updated weights for policy 0, policy_version 50340 (0.0008) [2023-10-13 23:01:38,987][60935] Updated weights for policy 0, policy_version 50350 (0.0009) [2023-10-13 23:01:39,358][60935] Updated weights for policy 0, policy_version 50360 (0.0008) [2023-10-13 23:01:40,739][60934] Updated weights for policy 1, policy_version 50752 (0.0009) [2023-10-13 23:01:41,110][60934] Updated weights for policy 1, policy_version 50762 (0.0009) [2023-10-13 23:01:41,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 103841792. Throughput: 0: 1692.5, 1: 1718.9. Samples: 25973744. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 23:01:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:01:41,481][60934] Updated weights for policy 1, policy_version 50772 (0.0007) [2023-10-13 23:01:43,413][60935] Updated weights for policy 0, policy_version 50370 (0.0009) [2023-10-13 23:01:43,778][60935] Updated weights for policy 0, policy_version 50380 (0.0007) [2023-10-13 23:01:44,151][60935] Updated weights for policy 0, policy_version 50390 (0.0007) [2023-10-13 23:01:44,517][60935] Updated weights for policy 0, policy_version 50400 (0.0010) [2023-10-13 23:01:45,566][60934] Updated weights for policy 1, policy_version 50782 (0.0008) [2023-10-13 23:01:45,939][60934] Updated weights for policy 1, policy_version 50792 (0.0009) [2023-10-13 23:01:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 103907328. Throughput: 0: 1692.9, 1: 1714.1. Samples: 25983916. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 23:01:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:01:46,311][60934] Updated weights for policy 1, policy_version 50802 (0.0008) [2023-10-13 23:01:48,629][60935] Updated weights for policy 0, policy_version 50410 (0.0010) [2023-10-13 23:01:48,994][60935] Updated weights for policy 0, policy_version 50420 (0.0009) [2023-10-13 23:01:49,367][60935] Updated weights for policy 0, policy_version 50430 (0.0007) [2023-10-13 23:01:50,274][60934] Updated weights for policy 1, policy_version 50812 (0.0008) [2023-10-13 23:01:50,642][60934] Updated weights for policy 1, policy_version 50822 (0.0007) [2023-10-13 23:01:51,009][60934] Updated weights for policy 1, policy_version 50832 (0.0008) [2023-10-13 23:01:51,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 103972864. Throughput: 0: 1680.3, 1: 1721.6. Samples: 26004244. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 23:01:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:01:53,165][60935] Updated weights for policy 0, policy_version 50440 (0.0008) [2023-10-13 23:01:53,535][60935] Updated weights for policy 0, policy_version 50450 (0.0007) [2023-10-13 23:01:53,901][60935] Updated weights for policy 0, policy_version 50460 (0.0007) [2023-10-13 23:01:54,903][60934] Updated weights for policy 1, policy_version 50842 (0.0010) [2023-10-13 23:01:55,278][60934] Updated weights for policy 1, policy_version 50852 (0.0009) [2023-10-13 23:01:55,640][60934] Updated weights for policy 1, policy_version 50862 (0.0008) [2023-10-13 23:01:56,005][60934] Updated weights for policy 1, policy_version 50872 (0.0010) [2023-10-13 23:01:56,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 104071168. Throughput: 0: 1714.3, 1: 1701.8. Samples: 26024956. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 23:01:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:01:57,799][60935] Updated weights for policy 0, policy_version 50470 (0.0007) [2023-10-13 23:01:58,177][60935] Updated weights for policy 0, policy_version 50480 (0.0007) [2023-10-13 23:01:58,550][60935] Updated weights for policy 0, policy_version 50490 (0.0010) [2023-10-13 23:02:00,098][60934] Updated weights for policy 1, policy_version 50882 (0.0008) [2023-10-13 23:02:00,466][60934] Updated weights for policy 1, policy_version 50892 (0.0009) [2023-10-13 23:02:00,826][60934] Updated weights for policy 1, policy_version 50902 (0.0008) [2023-10-13 23:02:01,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 104136704. Throughput: 0: 1684.1, 1: 1721.6. Samples: 26034838. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 23:02:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:02:02,474][60935] Updated weights for policy 0, policy_version 50500 (0.0008) [2023-10-13 23:02:02,843][60935] Updated weights for policy 0, policy_version 50510 (0.0008) [2023-10-13 23:02:03,219][60935] Updated weights for policy 0, policy_version 50520 (0.0009) [2023-10-13 23:02:04,759][60934] Updated weights for policy 1, policy_version 50912 (0.0009) [2023-10-13 23:02:05,125][60934] Updated weights for policy 1, policy_version 50922 (0.0008) [2023-10-13 23:02:05,491][60934] Updated weights for policy 1, policy_version 50932 (0.0008) [2023-10-13 23:02:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 104202240. Throughput: 0: 1704.0, 1: 1717.6. Samples: 26055938. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 23:02:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:02:07,382][60935] Updated weights for policy 0, policy_version 50530 (0.0009) [2023-10-13 23:02:07,764][60935] Updated weights for policy 0, policy_version 50540 (0.0009) [2023-10-13 23:02:08,128][60935] Updated weights for policy 0, policy_version 50550 (0.0007) [2023-10-13 23:02:08,504][60935] Updated weights for policy 0, policy_version 50560 (0.0008) [2023-10-13 23:02:09,427][60934] Updated weights for policy 1, policy_version 50942 (0.0007) [2023-10-13 23:02:09,796][60934] Updated weights for policy 1, policy_version 50952 (0.0010) [2023-10-13 23:02:10,165][60934] Updated weights for policy 1, policy_version 50962 (0.0007) [2023-10-13 23:02:11,249][59943] Fps is (10 sec: 13106.6, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 104267776. Throughput: 0: 1715.2, 1: 1693.4. Samples: 26075966. Policy #0 lag: (min: 10.0, avg: 16.0, max: 42.0) [2023-10-13 23:02:11,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:02:12,637][60935] Updated weights for policy 0, policy_version 50570 (0.0009) [2023-10-13 23:02:13,008][60935] Updated weights for policy 0, policy_version 50580 (0.0010) [2023-10-13 23:02:13,380][60935] Updated weights for policy 0, policy_version 50590 (0.0010) [2023-10-13 23:02:14,018][60934] Updated weights for policy 1, policy_version 50972 (0.0008) [2023-10-13 23:02:14,374][60934] Updated weights for policy 1, policy_version 50982 (0.0008) [2023-10-13 23:02:14,745][60934] Updated weights for policy 1, policy_version 50992 (0.0009) [2023-10-13 23:02:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 104333312. Throughput: 0: 1684.0, 1: 1727.8. Samples: 26086528. Policy #0 lag: (min: 10.0, avg: 16.0, max: 42.0) [2023-10-13 23:02:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:02:17,365][60935] Updated weights for policy 0, policy_version 50600 (0.0012) [2023-10-13 23:02:17,743][60935] Updated weights for policy 0, policy_version 50610 (0.0011) [2023-10-13 23:02:18,106][60935] Updated weights for policy 0, policy_version 50620 (0.0009) [2023-10-13 23:02:18,666][60934] Updated weights for policy 1, policy_version 51002 (0.0007) [2023-10-13 23:02:19,027][60934] Updated weights for policy 1, policy_version 51012 (0.0010) [2023-10-13 23:02:19,385][60934] Updated weights for policy 1, policy_version 51022 (0.0007) [2023-10-13 23:02:19,757][60934] Updated weights for policy 1, policy_version 51032 (0.0008) [2023-10-13 23:02:21,248][59943] Fps is (10 sec: 13107.8, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 104398848. Throughput: 0: 1710.1, 1: 1701.5. Samples: 26106700. Policy #0 lag: (min: 10.0, avg: 16.0, max: 42.0) [2023-10-13 23:02:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:02:22,091][60935] Updated weights for policy 0, policy_version 50630 (0.0011) [2023-10-13 23:02:22,464][60935] Updated weights for policy 0, policy_version 50640 (0.0009) [2023-10-13 23:02:22,826][60935] Updated weights for policy 0, policy_version 50650 (0.0009) [2023-10-13 23:02:23,994][60934] Updated weights for policy 1, policy_version 51042 (0.0008) [2023-10-13 23:02:24,361][60934] Updated weights for policy 1, policy_version 51052 (0.0009) [2023-10-13 23:02:24,720][60934] Updated weights for policy 1, policy_version 51062 (0.0009) [2023-10-13 23:02:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 104464384. Throughput: 0: 1711.9, 1: 1694.3. Samples: 26127022. Policy #0 lag: (min: 10.0, avg: 16.0, max: 42.0) [2023-10-13 23:02:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:02:26,982][60935] Updated weights for policy 0, policy_version 50660 (0.0008) [2023-10-13 23:02:27,364][60935] Updated weights for policy 0, policy_version 50670 (0.0011) [2023-10-13 23:02:27,739][60935] Updated weights for policy 0, policy_version 50680 (0.0009) [2023-10-13 23:02:28,757][60934] Updated weights for policy 1, policy_version 51072 (0.0010) [2023-10-13 23:02:29,124][60934] Updated weights for policy 1, policy_version 51082 (0.0008) [2023-10-13 23:02:29,489][60934] Updated weights for policy 1, policy_version 51092 (0.0008) [2023-10-13 23:02:31,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 104529920. Throughput: 0: 1687.6, 1: 1721.0. Samples: 26137306. Policy #0 lag: (min: 10.0, avg: 16.0, max: 42.0) [2023-10-13 23:02:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:02:31,561][60935] Updated weights for policy 0, policy_version 50690 (0.0008) [2023-10-13 23:02:31,932][60935] Updated weights for policy 0, policy_version 50700 (0.0008) [2023-10-13 23:02:32,294][60935] Updated weights for policy 0, policy_version 50710 (0.0009) [2023-10-13 23:02:32,667][60935] Updated weights for policy 0, policy_version 50720 (0.0007) [2023-10-13 23:02:33,512][60934] Updated weights for policy 1, policy_version 51102 (0.0009) [2023-10-13 23:02:33,881][60934] Updated weights for policy 1, policy_version 51112 (0.0007) [2023-10-13 23:02:34,254][60934] Updated weights for policy 1, policy_version 51122 (0.0008) [2023-10-13 23:02:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 104595456. Throughput: 0: 1717.9, 1: 1690.5. Samples: 26157620. Policy #0 lag: (min: 10.0, avg: 16.0, max: 42.0) [2023-10-13 23:02:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:02:36,540][60935] Updated weights for policy 0, policy_version 50730 (0.0007) [2023-10-13 23:02:36,905][60935] Updated weights for policy 0, policy_version 50740 (0.0009) [2023-10-13 23:02:37,273][60935] Updated weights for policy 0, policy_version 50750 (0.0010) [2023-10-13 23:02:38,285][60934] Updated weights for policy 1, policy_version 51132 (0.0010) [2023-10-13 23:02:38,642][60934] Updated weights for policy 1, policy_version 51142 (0.0009) [2023-10-13 23:02:39,010][60934] Updated weights for policy 1, policy_version 51152 (0.0007) [2023-10-13 23:02:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 104660992. Throughput: 0: 1708.6, 1: 1705.6. Samples: 26178594. Policy #0 lag: (min: 10.0, avg: 16.0, max: 42.0) [2023-10-13 23:02:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:02:41,347][60935] Updated weights for policy 0, policy_version 50760 (0.0010) [2023-10-13 23:02:41,719][60935] Updated weights for policy 0, policy_version 50770 (0.0008) [2023-10-13 23:02:42,086][60935] Updated weights for policy 0, policy_version 50780 (0.0008) [2023-10-13 23:02:42,979][60934] Updated weights for policy 1, policy_version 51162 (0.0008) [2023-10-13 23:02:43,349][60934] Updated weights for policy 1, policy_version 51172 (0.0009) [2023-10-13 23:02:43,717][60934] Updated weights for policy 1, policy_version 51182 (0.0011) [2023-10-13 23:02:44,086][60934] Updated weights for policy 1, policy_version 51192 (0.0008) [2023-10-13 23:02:46,067][60935] Updated weights for policy 0, policy_version 50790 (0.0008) [2023-10-13 23:02:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 104726528. Throughput: 0: 1710.4, 1: 1701.1. Samples: 26188358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:02:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:02:46,436][60935] Updated weights for policy 0, policy_version 50800 (0.0008) [2023-10-13 23:02:46,800][60935] Updated weights for policy 0, policy_version 50810 (0.0008) [2023-10-13 23:02:48,116][60934] Updated weights for policy 1, policy_version 51202 (0.0008) [2023-10-13 23:02:48,488][60934] Updated weights for policy 1, policy_version 51212 (0.0009) [2023-10-13 23:02:48,857][60934] Updated weights for policy 1, policy_version 51222 (0.0008) [2023-10-13 23:02:50,849][60935] Updated weights for policy 0, policy_version 50820 (0.0008) [2023-10-13 23:02:51,217][60935] Updated weights for policy 0, policy_version 50830 (0.0010) [2023-10-13 23:02:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 104792064. Throughput: 0: 1712.3, 1: 1683.0. Samples: 26208726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:02:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:02:51,586][60935] Updated weights for policy 0, policy_version 50840 (0.0008) [2023-10-13 23:02:52,942][60934] Updated weights for policy 1, policy_version 51232 (0.0010) [2023-10-13 23:02:53,311][60934] Updated weights for policy 1, policy_version 51242 (0.0010) [2023-10-13 23:02:53,686][60934] Updated weights for policy 1, policy_version 51252 (0.0008) [2023-10-13 23:02:55,391][60935] Updated weights for policy 0, policy_version 50850 (0.0009) [2023-10-13 23:02:55,764][60935] Updated weights for policy 0, policy_version 50860 (0.0008) [2023-10-13 23:02:56,131][60935] Updated weights for policy 0, policy_version 50870 (0.0008) [2023-10-13 23:02:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 104857600. Throughput: 0: 1706.7, 1: 1707.7. Samples: 26229610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:02:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:02:56,259][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000051256_52789248.pth... [2023-10-13 23:02:56,297][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000049656_51150848.pth [2023-10-13 23:02:56,498][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000050880_52101120.pth... [2023-10-13 23:02:56,498][60935] Updated weights for policy 0, policy_version 50880 (0.0009) [2023-10-13 23:02:56,535][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000049280_50462720.pth [2023-10-13 23:02:57,698][60934] Updated weights for policy 1, policy_version 51262 (0.0007) [2023-10-13 23:02:58,059][60934] Updated weights for policy 1, policy_version 51272 (0.0009) [2023-10-13 23:02:58,426][60934] Updated weights for policy 1, policy_version 51282 (0.0009) [2023-10-13 23:03:00,348][60935] Updated weights for policy 0, policy_version 50890 (0.0009) [2023-10-13 23:03:00,716][60935] Updated weights for policy 0, policy_version 50900 (0.0009) [2023-10-13 23:03:01,082][60935] Updated weights for policy 0, policy_version 50910 (0.0009) [2023-10-13 23:03:01,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 104955904. Throughput: 0: 1725.1, 1: 1679.9. Samples: 26239752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:03:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:03:02,489][60934] Updated weights for policy 1, policy_version 51292 (0.0009) [2023-10-13 23:03:02,859][60934] Updated weights for policy 1, policy_version 51302 (0.0007) [2023-10-13 23:03:03,218][60934] Updated weights for policy 1, policy_version 51312 (0.0008) [2023-10-13 23:03:05,138][60935] Updated weights for policy 0, policy_version 50920 (0.0008) [2023-10-13 23:03:05,515][60935] Updated weights for policy 0, policy_version 50930 (0.0010) [2023-10-13 23:03:05,877][60935] Updated weights for policy 0, policy_version 50940 (0.0009) [2023-10-13 23:03:06,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 105021440. Throughput: 0: 1724.9, 1: 1700.4. Samples: 26260838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:03:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:03:06,998][60934] Updated weights for policy 1, policy_version 51322 (0.0008) [2023-10-13 23:03:07,361][60934] Updated weights for policy 1, policy_version 51332 (0.0008) [2023-10-13 23:03:07,735][60934] Updated weights for policy 1, policy_version 51342 (0.0007) [2023-10-13 23:03:08,097][60934] Updated weights for policy 1, policy_version 51352 (0.0007) [2023-10-13 23:03:09,657][60935] Updated weights for policy 0, policy_version 50950 (0.0008) [2023-10-13 23:03:10,026][60935] Updated weights for policy 0, policy_version 50960 (0.0008) [2023-10-13 23:03:10,397][60935] Updated weights for policy 0, policy_version 50970 (0.0009) [2023-10-13 23:03:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 105086976. Throughput: 0: 1700.2, 1: 1725.2. Samples: 26281162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:03:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:03:12,036][60934] Updated weights for policy 1, policy_version 51362 (0.0009) [2023-10-13 23:03:12,403][60934] Updated weights for policy 1, policy_version 51372 (0.0008) [2023-10-13 23:03:12,772][60934] Updated weights for policy 1, policy_version 51382 (0.0008) [2023-10-13 23:03:14,450][60935] Updated weights for policy 0, policy_version 50980 (0.0009) [2023-10-13 23:03:14,846][60935] Updated weights for policy 0, policy_version 50990 (0.0009) [2023-10-13 23:03:15,217][60935] Updated weights for policy 0, policy_version 51000 (0.0009) [2023-10-13 23:03:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 105152512. Throughput: 0: 1738.8, 1: 1697.3. Samples: 26291934. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 23:03:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:03:16,798][60934] Updated weights for policy 1, policy_version 51392 (0.0011) [2023-10-13 23:03:17,176][60934] Updated weights for policy 1, policy_version 51402 (0.0008) [2023-10-13 23:03:17,539][60934] Updated weights for policy 1, policy_version 51412 (0.0009) [2023-10-13 23:03:19,150][60935] Updated weights for policy 0, policy_version 51010 (0.0008) [2023-10-13 23:03:19,518][60935] Updated weights for policy 0, policy_version 51020 (0.0011) [2023-10-13 23:03:19,889][60935] Updated weights for policy 0, policy_version 51030 (0.0010) [2023-10-13 23:03:20,263][60935] Updated weights for policy 0, policy_version 51040 (0.0009) [2023-10-13 23:03:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 105218048. Throughput: 0: 1713.2, 1: 1724.2. Samples: 26312302. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 23:03:21,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:03:21,326][60934] Updated weights for policy 1, policy_version 51422 (0.0009) [2023-10-13 23:03:21,695][60934] Updated weights for policy 1, policy_version 51432 (0.0010) [2023-10-13 23:03:22,055][60934] Updated weights for policy 1, policy_version 51442 (0.0011) [2023-10-13 23:03:24,231][60935] Updated weights for policy 0, policy_version 51050 (0.0008) [2023-10-13 23:03:24,600][60935] Updated weights for policy 0, policy_version 51060 (0.0008) [2023-10-13 23:03:24,964][60935] Updated weights for policy 0, policy_version 51070 (0.0008) [2023-10-13 23:03:26,050][60934] Updated weights for policy 1, policy_version 51452 (0.0008) [2023-10-13 23:03:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 105283584. Throughput: 0: 1708.0, 1: 1732.0. Samples: 26333392. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 23:03:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:03:26,410][60934] Updated weights for policy 1, policy_version 51462 (0.0007) [2023-10-13 23:03:26,775][60934] Updated weights for policy 1, policy_version 51472 (0.0007) [2023-10-13 23:03:28,886][60935] Updated weights for policy 0, policy_version 51080 (0.0010) [2023-10-13 23:03:29,251][60935] Updated weights for policy 0, policy_version 51090 (0.0009) [2023-10-13 23:03:29,614][60935] Updated weights for policy 0, policy_version 51100 (0.0012) [2023-10-13 23:03:30,785][60934] Updated weights for policy 1, policy_version 51482 (0.0008) [2023-10-13 23:03:31,151][60934] Updated weights for policy 1, policy_version 51492 (0.0007) [2023-10-13 23:03:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 105349120. Throughput: 0: 1732.6, 1: 1721.8. Samples: 26343806. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 23:03:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:03:31,525][60934] Updated weights for policy 1, policy_version 51502 (0.0008) [2023-10-13 23:03:31,886][60934] Updated weights for policy 1, policy_version 51512 (0.0010) [2023-10-13 23:03:33,508][60935] Updated weights for policy 0, policy_version 51110 (0.0009) [2023-10-13 23:03:33,879][60935] Updated weights for policy 0, policy_version 51120 (0.0010) [2023-10-13 23:03:34,263][60935] Updated weights for policy 0, policy_version 51130 (0.0007) [2023-10-13 23:03:35,923][60934] Updated weights for policy 1, policy_version 51522 (0.0008) [2023-10-13 23:03:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 105414656. Throughput: 0: 1707.9, 1: 1738.8. Samples: 26363824. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 23:03:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:03:36,291][60934] Updated weights for policy 1, policy_version 51532 (0.0008) [2023-10-13 23:03:36,664][60934] Updated weights for policy 1, policy_version 51542 (0.0010) [2023-10-13 23:03:38,185][60935] Updated weights for policy 0, policy_version 51140 (0.0008) [2023-10-13 23:03:38,557][60935] Updated weights for policy 0, policy_version 51150 (0.0008) [2023-10-13 23:03:38,926][60935] Updated weights for policy 0, policy_version 51160 (0.0009) [2023-10-13 23:03:40,540][60934] Updated weights for policy 1, policy_version 51552 (0.0009) [2023-10-13 23:03:40,906][60934] Updated weights for policy 1, policy_version 51562 (0.0011) [2023-10-13 23:03:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 105480192. Throughput: 0: 1719.2, 1: 1731.9. Samples: 26384908. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 23:03:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:03:41,267][60934] Updated weights for policy 1, policy_version 51572 (0.0009) [2023-10-13 23:03:42,815][60935] Updated weights for policy 0, policy_version 51170 (0.0007) [2023-10-13 23:03:43,182][60935] Updated weights for policy 0, policy_version 51180 (0.0008) [2023-10-13 23:03:43,553][60935] Updated weights for policy 0, policy_version 51190 (0.0009) [2023-10-13 23:03:43,921][60935] Updated weights for policy 0, policy_version 51200 (0.0009) [2023-10-13 23:03:45,099][60934] Updated weights for policy 1, policy_version 51582 (0.0009) [2023-10-13 23:03:45,466][60934] Updated weights for policy 1, policy_version 51592 (0.0008) [2023-10-13 23:03:45,828][60934] Updated weights for policy 1, policy_version 51602 (0.0007) [2023-10-13 23:03:46,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 105578496. Throughput: 0: 1709.7, 1: 1736.7. Samples: 26394840. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 23:03:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:03:47,934][60935] Updated weights for policy 0, policy_version 51210 (0.0007) [2023-10-13 23:03:48,311][60935] Updated weights for policy 0, policy_version 51220 (0.0008) [2023-10-13 23:03:48,677][60935] Updated weights for policy 0, policy_version 51230 (0.0009) [2023-10-13 23:03:49,887][60934] Updated weights for policy 1, policy_version 51612 (0.0007) [2023-10-13 23:03:50,262][60934] Updated weights for policy 1, policy_version 51622 (0.0009) [2023-10-13 23:03:50,632][60934] Updated weights for policy 1, policy_version 51632 (0.0008) [2023-10-13 23:03:51,248][59943] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 105644032. Throughput: 0: 1706.8, 1: 1737.4. Samples: 26415830. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-10-13 23:03:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:03:52,649][60935] Updated weights for policy 0, policy_version 51240 (0.0010) [2023-10-13 23:03:53,009][60935] Updated weights for policy 0, policy_version 51250 (0.0008) [2023-10-13 23:03:53,374][60935] Updated weights for policy 0, policy_version 51260 (0.0007) [2023-10-13 23:03:54,371][60934] Updated weights for policy 1, policy_version 51642 (0.0007) [2023-10-13 23:03:54,737][60934] Updated weights for policy 1, policy_version 51652 (0.0011) [2023-10-13 23:03:55,111][60934] Updated weights for policy 1, policy_version 51662 (0.0010) [2023-10-13 23:03:55,474][60934] Updated weights for policy 1, policy_version 51672 (0.0010) [2023-10-13 23:03:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 105709568. Throughput: 0: 1736.0, 1: 1697.8. Samples: 26435686. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-10-13 23:03:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:03:57,437][60935] Updated weights for policy 0, policy_version 51270 (0.0009) [2023-10-13 23:03:57,822][60935] Updated weights for policy 0, policy_version 51280 (0.0009) [2023-10-13 23:03:58,194][60935] Updated weights for policy 0, policy_version 51290 (0.0008) [2023-10-13 23:03:59,601][60934] Updated weights for policy 1, policy_version 51682 (0.0007) [2023-10-13 23:03:59,974][60934] Updated weights for policy 1, policy_version 51692 (0.0010) [2023-10-13 23:04:00,344][60934] Updated weights for policy 1, policy_version 51702 (0.0010) [2023-10-13 23:04:01,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 105775104. Throughput: 0: 1701.9, 1: 1728.4. Samples: 26446296. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-10-13 23:04:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:04:02,142][60935] Updated weights for policy 0, policy_version 51300 (0.0008) [2023-10-13 23:04:02,504][60935] Updated weights for policy 0, policy_version 51310 (0.0009) [2023-10-13 23:04:02,869][60935] Updated weights for policy 0, policy_version 51320 (0.0008) [2023-10-13 23:04:04,339][60934] Updated weights for policy 1, policy_version 51712 (0.0008) [2023-10-13 23:04:04,707][60934] Updated weights for policy 1, policy_version 51722 (0.0007) [2023-10-13 23:04:05,065][60934] Updated weights for policy 1, policy_version 51732 (0.0010) [2023-10-13 23:04:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 105840640. Throughput: 0: 1722.2, 1: 1714.7. Samples: 26466964. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-10-13 23:04:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:04:06,974][60935] Updated weights for policy 0, policy_version 51330 (0.0007) [2023-10-13 23:04:07,342][60935] Updated weights for policy 0, policy_version 51340 (0.0008) [2023-10-13 23:04:07,720][60935] Updated weights for policy 0, policy_version 51350 (0.0009) [2023-10-13 23:04:08,086][60935] Updated weights for policy 0, policy_version 51360 (0.0010) [2023-10-13 23:04:09,081][60934] Updated weights for policy 1, policy_version 51742 (0.0008) [2023-10-13 23:04:09,449][60934] Updated weights for policy 1, policy_version 51752 (0.0008) [2023-10-13 23:04:09,825][60934] Updated weights for policy 1, policy_version 51762 (0.0007) [2023-10-13 23:04:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 105906176. Throughput: 0: 1726.8, 1: 1692.7. Samples: 26487270. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-10-13 23:04:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:04:12,029][60935] Updated weights for policy 0, policy_version 51370 (0.0009) [2023-10-13 23:04:12,392][60935] Updated weights for policy 0, policy_version 51380 (0.0009) [2023-10-13 23:04:12,771][60935] Updated weights for policy 0, policy_version 51390 (0.0009) [2023-10-13 23:04:13,960][60934] Updated weights for policy 1, policy_version 51772 (0.0009) [2023-10-13 23:04:14,327][60934] Updated weights for policy 1, policy_version 51782 (0.0007) [2023-10-13 23:04:14,687][60934] Updated weights for policy 1, policy_version 51792 (0.0007) [2023-10-13 23:04:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 105971712. Throughput: 0: 1704.7, 1: 1720.1. Samples: 26497922. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-10-13 23:04:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:04:16,476][60935] Updated weights for policy 0, policy_version 51400 (0.0008) [2023-10-13 23:04:16,849][60935] Updated weights for policy 0, policy_version 51410 (0.0009) [2023-10-13 23:04:17,218][60935] Updated weights for policy 0, policy_version 51420 (0.0011) [2023-10-13 23:04:18,729][60934] Updated weights for policy 1, policy_version 51802 (0.0008) [2023-10-13 23:04:19,093][60934] Updated weights for policy 1, policy_version 51812 (0.0008) [2023-10-13 23:04:19,460][60934] Updated weights for policy 1, policy_version 51822 (0.0008) [2023-10-13 23:04:19,821][60934] Updated weights for policy 1, policy_version 51832 (0.0007) [2023-10-13 23:04:21,136][60935] Updated weights for policy 0, policy_version 51430 (0.0008) [2023-10-13 23:04:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 106037248. Throughput: 0: 1734.8, 1: 1694.4. Samples: 26518140. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-10-13 23:04:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:04:21,508][60935] Updated weights for policy 0, policy_version 51440 (0.0008) [2023-10-13 23:04:21,889][60935] Updated weights for policy 0, policy_version 51450 (0.0009) [2023-10-13 23:04:24,092][60934] Updated weights for policy 1, policy_version 51842 (0.0007) [2023-10-13 23:04:24,452][60934] Updated weights for policy 1, policy_version 51852 (0.0007) [2023-10-13 23:04:24,813][60934] Updated weights for policy 1, policy_version 51862 (0.0007) [2023-10-13 23:04:25,830][60935] Updated weights for policy 0, policy_version 51460 (0.0009) [2023-10-13 23:04:26,205][60935] Updated weights for policy 0, policy_version 51470 (0.0009) [2023-10-13 23:04:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 106102784. Throughput: 0: 1728.9, 1: 1687.4. Samples: 26538640. Policy #0 lag: (min: 21.0, avg: 22.3, max: 45.0) [2023-10-13 23:04:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:04:26,586][60935] Updated weights for policy 0, policy_version 51480 (0.0008) [2023-10-13 23:04:28,816][60934] Updated weights for policy 1, policy_version 51872 (0.0008) [2023-10-13 23:04:29,178][60934] Updated weights for policy 1, policy_version 51882 (0.0007) [2023-10-13 23:04:29,544][60934] Updated weights for policy 1, policy_version 51892 (0.0007) [2023-10-13 23:04:30,503][60935] Updated weights for policy 0, policy_version 51490 (0.0010) [2023-10-13 23:04:30,872][60935] Updated weights for policy 0, policy_version 51500 (0.0009) [2023-10-13 23:04:31,236][60935] Updated weights for policy 0, policy_version 51510 (0.0009) [2023-10-13 23:04:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 106168320. Throughput: 0: 1726.0, 1: 1705.3. Samples: 26549252. Policy #0 lag: (min: 21.0, avg: 22.3, max: 45.0) [2023-10-13 23:04:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:04:31,600][60935] Updated weights for policy 0, policy_version 51520 (0.0008) [2023-10-13 23:04:33,466][60934] Updated weights for policy 1, policy_version 51902 (0.0008) [2023-10-13 23:04:33,830][60934] Updated weights for policy 1, policy_version 51912 (0.0009) [2023-10-13 23:04:34,201][60934] Updated weights for policy 1, policy_version 51922 (0.0008) [2023-10-13 23:04:35,821][60935] Updated weights for policy 0, policy_version 51530 (0.0010) [2023-10-13 23:04:36,198][60935] Updated weights for policy 0, policy_version 51540 (0.0010) [2023-10-13 23:04:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 106233856. Throughput: 0: 1725.7, 1: 1680.6. Samples: 26569114. Policy #0 lag: (min: 21.0, avg: 22.3, max: 45.0) [2023-10-13 23:04:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:04:36,559][60935] Updated weights for policy 0, policy_version 51550 (0.0011) [2023-10-13 23:04:38,116][60934] Updated weights for policy 1, policy_version 51932 (0.0009) [2023-10-13 23:04:38,487][60934] Updated weights for policy 1, policy_version 51942 (0.0007) [2023-10-13 23:04:38,853][60934] Updated weights for policy 1, policy_version 51952 (0.0007) [2023-10-13 23:04:40,491][60935] Updated weights for policy 0, policy_version 51560 (0.0011) [2023-10-13 23:04:40,869][60935] Updated weights for policy 0, policy_version 51570 (0.0008) [2023-10-13 23:04:41,242][60935] Updated weights for policy 0, policy_version 51580 (0.0008) [2023-10-13 23:04:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 106299392. Throughput: 0: 1709.1, 1: 1710.7. Samples: 26589574. Policy #0 lag: (min: 21.0, avg: 22.3, max: 45.0) [2023-10-13 23:04:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:04:42,718][60934] Updated weights for policy 1, policy_version 51962 (0.0007) [2023-10-13 23:04:43,086][60934] Updated weights for policy 1, policy_version 51972 (0.0008) [2023-10-13 23:04:43,452][60934] Updated weights for policy 1, policy_version 51982 (0.0009) [2023-10-13 23:04:43,814][60934] Updated weights for policy 1, policy_version 51992 (0.0009) [2023-10-13 23:04:45,215][60935] Updated weights for policy 0, policy_version 51590 (0.0008) [2023-10-13 23:04:45,580][60935] Updated weights for policy 0, policy_version 51600 (0.0008) [2023-10-13 23:04:45,953][60935] Updated weights for policy 0, policy_version 51610 (0.0009) [2023-10-13 23:04:46,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 106397696. Throughput: 0: 1725.3, 1: 1684.7. Samples: 26599746. Policy #0 lag: (min: 21.0, avg: 22.3, max: 45.0) [2023-10-13 23:04:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:04:47,944][60934] Updated weights for policy 1, policy_version 52002 (0.0008) [2023-10-13 23:04:48,317][60934] Updated weights for policy 1, policy_version 52012 (0.0011) [2023-10-13 23:04:48,691][60934] Updated weights for policy 1, policy_version 52022 (0.0008) [2023-10-13 23:04:49,976][60935] Updated weights for policy 0, policy_version 51620 (0.0009) [2023-10-13 23:04:50,348][60935] Updated weights for policy 0, policy_version 51630 (0.0012) [2023-10-13 23:04:50,713][60935] Updated weights for policy 0, policy_version 51640 (0.0010) [2023-10-13 23:04:51,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 106463232. Throughput: 0: 1715.4, 1: 1688.4. Samples: 26620138. Policy #0 lag: (min: 21.0, avg: 22.3, max: 45.0) [2023-10-13 23:04:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:04:52,494][60934] Updated weights for policy 1, policy_version 52032 (0.0010) [2023-10-13 23:04:52,861][60934] Updated weights for policy 1, policy_version 52042 (0.0007) [2023-10-13 23:04:53,216][60934] Updated weights for policy 1, policy_version 52052 (0.0007) [2023-10-13 23:04:54,706][60935] Updated weights for policy 0, policy_version 51650 (0.0008) [2023-10-13 23:04:55,078][60935] Updated weights for policy 0, policy_version 51660 (0.0009) [2023-10-13 23:04:55,445][60935] Updated weights for policy 0, policy_version 51670 (0.0008) [2023-10-13 23:04:55,813][60935] Updated weights for policy 0, policy_version 51680 (0.0007) [2023-10-13 23:04:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 106528768. Throughput: 0: 1691.0, 1: 1706.7. Samples: 26640166. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-13 23:04:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:04:56,260][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000051680_52920320.pth... [2023-10-13 23:04:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000052056_53608448.pth... [2023-10-13 23:04:56,312][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000050456_51970048.pth [2023-10-13 23:04:56,313][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000050080_51281920.pth [2023-10-13 23:04:57,257][60934] Updated weights for policy 1, policy_version 52062 (0.0007) [2023-10-13 23:04:57,617][60934] Updated weights for policy 1, policy_version 52072 (0.0008) [2023-10-13 23:04:57,993][60934] Updated weights for policy 1, policy_version 52082 (0.0007) [2023-10-13 23:04:59,778][60935] Updated weights for policy 0, policy_version 51690 (0.0010) [2023-10-13 23:05:00,149][60935] Updated weights for policy 0, policy_version 51700 (0.0010) [2023-10-13 23:05:00,507][60935] Updated weights for policy 0, policy_version 51710 (0.0009) [2023-10-13 23:05:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 106594304. Throughput: 0: 1716.8, 1: 1681.0. Samples: 26650820. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-13 23:05:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:05:01,785][60934] Updated weights for policy 1, policy_version 52092 (0.0008) [2023-10-13 23:05:02,142][60934] Updated weights for policy 1, policy_version 52102 (0.0009) [2023-10-13 23:05:02,516][60934] Updated weights for policy 1, policy_version 52112 (0.0009) [2023-10-13 23:05:04,406][60935] Updated weights for policy 0, policy_version 51720 (0.0007) [2023-10-13 23:05:04,775][60935] Updated weights for policy 0, policy_version 51730 (0.0008) [2023-10-13 23:05:05,149][60935] Updated weights for policy 0, policy_version 51740 (0.0008) [2023-10-13 23:05:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 106659840. Throughput: 0: 1693.9, 1: 1713.2. Samples: 26671458. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-13 23:05:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:05:06,354][60934] Updated weights for policy 1, policy_version 52122 (0.0007) [2023-10-13 23:05:06,723][60934] Updated weights for policy 1, policy_version 52132 (0.0008) [2023-10-13 23:05:07,082][60934] Updated weights for policy 1, policy_version 52142 (0.0007) [2023-10-13 23:05:07,454][60934] Updated weights for policy 1, policy_version 52152 (0.0008) [2023-10-13 23:05:09,367][60935] Updated weights for policy 0, policy_version 51750 (0.0008) [2023-10-13 23:05:09,745][60935] Updated weights for policy 0, policy_version 51760 (0.0008) [2023-10-13 23:05:10,119][60935] Updated weights for policy 0, policy_version 51770 (0.0008) [2023-10-13 23:05:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 106725376. Throughput: 0: 1678.1, 1: 1735.0. Samples: 26692228. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-13 23:05:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:05:11,441][60934] Updated weights for policy 1, policy_version 52162 (0.0008) [2023-10-13 23:05:11,806][60934] Updated weights for policy 1, policy_version 52172 (0.0009) [2023-10-13 23:05:12,169][60934] Updated weights for policy 1, policy_version 52182 (0.0008) [2023-10-13 23:05:14,124][60935] Updated weights for policy 0, policy_version 51780 (0.0009) [2023-10-13 23:05:14,490][60935] Updated weights for policy 0, policy_version 51790 (0.0008) [2023-10-13 23:05:14,855][60935] Updated weights for policy 0, policy_version 51800 (0.0009) [2023-10-13 23:05:16,170][60934] Updated weights for policy 1, policy_version 52192 (0.0008) [2023-10-13 23:05:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 106790912. Throughput: 0: 1706.7, 1: 1707.9. Samples: 26702908. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-13 23:05:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:05:16,538][60934] Updated weights for policy 1, policy_version 52202 (0.0008) [2023-10-13 23:05:16,907][60934] Updated weights for policy 1, policy_version 52212 (0.0008) [2023-10-13 23:05:19,001][60935] Updated weights for policy 0, policy_version 51810 (0.0009) [2023-10-13 23:05:19,372][60935] Updated weights for policy 0, policy_version 51820 (0.0008) [2023-10-13 23:05:19,743][60935] Updated weights for policy 0, policy_version 51830 (0.0010) [2023-10-13 23:05:20,124][60935] Updated weights for policy 0, policy_version 51840 (0.0010) [2023-10-13 23:05:20,888][60934] Updated weights for policy 1, policy_version 52222 (0.0008) [2023-10-13 23:05:21,245][60934] Updated weights for policy 1, policy_version 52232 (0.0008) [2023-10-13 23:05:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 106856448. Throughput: 0: 1684.9, 1: 1731.8. Samples: 26722866. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-13 23:05:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:05:21,617][60934] Updated weights for policy 1, policy_version 52242 (0.0008) [2023-10-13 23:05:24,248][60935] Updated weights for policy 0, policy_version 51850 (0.0008) [2023-10-13 23:05:24,613][60935] Updated weights for policy 0, policy_version 51860 (0.0009) [2023-10-13 23:05:24,988][60935] Updated weights for policy 0, policy_version 51870 (0.0008) [2023-10-13 23:05:25,691][60934] Updated weights for policy 1, policy_version 52252 (0.0009) [2023-10-13 23:05:26,048][60934] Updated weights for policy 1, policy_version 52262 (0.0008) [2023-10-13 23:05:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 106921984. Throughput: 0: 1689.1, 1: 1730.6. Samples: 26743460. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-13 23:05:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:05:26,408][60934] Updated weights for policy 1, policy_version 52272 (0.0010) [2023-10-13 23:05:29,073][60935] Updated weights for policy 0, policy_version 51880 (0.0010) [2023-10-13 23:05:29,448][60935] Updated weights for policy 0, policy_version 51890 (0.0009) [2023-10-13 23:05:29,825][60935] Updated weights for policy 0, policy_version 51900 (0.0011) [2023-10-13 23:05:30,428][60934] Updated weights for policy 1, policy_version 52282 (0.0007) [2023-10-13 23:05:30,785][60934] Updated weights for policy 1, policy_version 52292 (0.0008) [2023-10-13 23:05:31,161][60934] Updated weights for policy 1, policy_version 52302 (0.0008) [2023-10-13 23:05:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 106987520. Throughput: 0: 1697.3, 1: 1729.8. Samples: 26753968. Policy #0 lag: (min: 6.0, avg: 20.9, max: 38.0) [2023-10-13 23:05:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:05:31,522][60934] Updated weights for policy 1, policy_version 52312 (0.0007) [2023-10-13 23:05:33,859][60935] Updated weights for policy 0, policy_version 51910 (0.0009) [2023-10-13 23:05:34,226][60935] Updated weights for policy 0, policy_version 51920 (0.0010) [2023-10-13 23:05:34,601][60935] Updated weights for policy 0, policy_version 51930 (0.0009) [2023-10-13 23:05:35,413][60934] Updated weights for policy 1, policy_version 52322 (0.0009) [2023-10-13 23:05:35,778][60934] Updated weights for policy 1, policy_version 52332 (0.0008) [2023-10-13 23:05:36,143][60934] Updated weights for policy 1, policy_version 52342 (0.0007) [2023-10-13 23:05:36,248][59943] Fps is (10 sec: 16383.4, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 107085824. Throughput: 0: 1674.0, 1: 1749.1. Samples: 26774180. Policy #0 lag: (min: 6.0, avg: 20.9, max: 38.0) [2023-10-13 23:05:36,249][59943] Avg episode reward: [(0, '-0.410'), (1, '0.000')] [2023-10-13 23:05:38,781][60935] Updated weights for policy 0, policy_version 51940 (0.0007) [2023-10-13 23:05:39,143][60935] Updated weights for policy 0, policy_version 51950 (0.0007) [2023-10-13 23:05:39,510][60935] Updated weights for policy 0, policy_version 51960 (0.0007) [2023-10-13 23:05:40,122][60934] Updated weights for policy 1, policy_version 52352 (0.0008) [2023-10-13 23:05:40,496][60934] Updated weights for policy 1, policy_version 52362 (0.0008) [2023-10-13 23:05:40,859][60934] Updated weights for policy 1, policy_version 52372 (0.0008) [2023-10-13 23:05:41,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 107151360. Throughput: 0: 1694.0, 1: 1729.9. Samples: 26794242. Policy #0 lag: (min: 6.0, avg: 20.9, max: 38.0) [2023-10-13 23:05:41,249][59943] Avg episode reward: [(0, '-0.410'), (1, '0.000')] [2023-10-13 23:05:43,484][60935] Updated weights for policy 0, policy_version 51970 (0.0007) [2023-10-13 23:05:43,853][60935] Updated weights for policy 0, policy_version 51980 (0.0007) [2023-10-13 23:05:44,225][60935] Updated weights for policy 0, policy_version 51990 (0.0008) [2023-10-13 23:05:44,588][60935] Updated weights for policy 0, policy_version 52000 (0.0009) [2023-10-13 23:05:44,740][60934] Updated weights for policy 1, policy_version 52382 (0.0009) [2023-10-13 23:05:45,110][60934] Updated weights for policy 1, policy_version 52392 (0.0008) [2023-10-13 23:05:45,469][60934] Updated weights for policy 1, policy_version 52402 (0.0008) [2023-10-13 23:05:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 107216896. Throughput: 0: 1681.6, 1: 1743.7. Samples: 26804958. Policy #0 lag: (min: 6.0, avg: 20.9, max: 38.0) [2023-10-13 23:05:46,249][59943] Avg episode reward: [(0, '-0.410'), (1, '0.000')] [2023-10-13 23:05:48,615][60935] Updated weights for policy 0, policy_version 52010 (0.0011) [2023-10-13 23:05:48,990][60935] Updated weights for policy 0, policy_version 52020 (0.0008) [2023-10-13 23:05:49,358][60935] Updated weights for policy 0, policy_version 52030 (0.0009) [2023-10-13 23:05:49,479][60934] Updated weights for policy 1, policy_version 52412 (0.0009) [2023-10-13 23:05:49,847][60934] Updated weights for policy 1, policy_version 52422 (0.0009) [2023-10-13 23:05:50,218][60934] Updated weights for policy 1, policy_version 52432 (0.0009) [2023-10-13 23:05:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 107282432. Throughput: 0: 1680.4, 1: 1731.2. Samples: 26824980. Policy #0 lag: (min: 6.0, avg: 20.9, max: 38.0) [2023-10-13 23:05:51,249][59943] Avg episode reward: [(0, '-0.410'), (1, '0.000')] [2023-10-13 23:05:53,543][60935] Updated weights for policy 0, policy_version 52040 (0.0009) [2023-10-13 23:05:53,917][60935] Updated weights for policy 0, policy_version 52050 (0.0007) [2023-10-13 23:05:54,248][60934] Updated weights for policy 1, policy_version 52442 (0.0008) [2023-10-13 23:05:54,295][60935] Updated weights for policy 0, policy_version 52060 (0.0009) [2023-10-13 23:05:54,614][60934] Updated weights for policy 1, policy_version 52452 (0.0008) [2023-10-13 23:05:54,980][60934] Updated weights for policy 1, policy_version 52462 (0.0011) [2023-10-13 23:05:55,344][60934] Updated weights for policy 1, policy_version 52472 (0.0011) [2023-10-13 23:05:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 107347968. Throughput: 0: 1691.7, 1: 1696.8. Samples: 26844710. Policy #0 lag: (min: 6.0, avg: 20.9, max: 38.0) [2023-10-13 23:05:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:05:58,382][60935] Updated weights for policy 0, policy_version 52070 (0.0010) [2023-10-13 23:05:58,747][60935] Updated weights for policy 0, policy_version 52080 (0.0008) [2023-10-13 23:05:59,116][60935] Updated weights for policy 0, policy_version 52090 (0.0007) [2023-10-13 23:05:59,264][60934] Updated weights for policy 1, policy_version 52482 (0.0007) [2023-10-13 23:05:59,634][60934] Updated weights for policy 1, policy_version 52492 (0.0010) [2023-10-13 23:06:00,002][60934] Updated weights for policy 1, policy_version 52502 (0.0008) [2023-10-13 23:06:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 107413504. Throughput: 0: 1670.9, 1: 1729.7. Samples: 26855932. Policy #0 lag: (min: 6.0, avg: 20.9, max: 38.0) [2023-10-13 23:06:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:06:03,283][60935] Updated weights for policy 0, policy_version 52100 (0.0010) [2023-10-13 23:06:03,648][60935] Updated weights for policy 0, policy_version 52110 (0.0007) [2023-10-13 23:06:03,919][60934] Updated weights for policy 1, policy_version 52512 (0.0008) [2023-10-13 23:06:04,019][60935] Updated weights for policy 0, policy_version 52120 (0.0007) [2023-10-13 23:06:04,294][60934] Updated weights for policy 1, policy_version 52522 (0.0009) [2023-10-13 23:06:04,658][60934] Updated weights for policy 1, policy_version 52532 (0.0010) [2023-10-13 23:06:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 107479040. Throughput: 0: 1672.8, 1: 1711.6. Samples: 26875160. Policy #0 lag: (min: 22.0, avg: 23.8, max: 50.0) [2023-10-13 23:06:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:06:07,861][60935] Updated weights for policy 0, policy_version 52130 (0.0007) [2023-10-13 23:06:08,234][60935] Updated weights for policy 0, policy_version 52140 (0.0009) [2023-10-13 23:06:08,605][60935] Updated weights for policy 0, policy_version 52150 (0.0009) [2023-10-13 23:06:08,715][60934] Updated weights for policy 1, policy_version 52542 (0.0008) [2023-10-13 23:06:08,974][60935] Updated weights for policy 0, policy_version 52160 (0.0010) [2023-10-13 23:06:09,084][60934] Updated weights for policy 1, policy_version 52552 (0.0007) [2023-10-13 23:06:09,461][60934] Updated weights for policy 1, policy_version 52562 (0.0007) [2023-10-13 23:06:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 107544576. Throughput: 0: 1687.6, 1: 1704.3. Samples: 26896098. Policy #0 lag: (min: 22.0, avg: 23.8, max: 50.0) [2023-10-13 23:06:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:06:12,998][60935] Updated weights for policy 0, policy_version 52170 (0.0008) [2023-10-13 23:06:13,363][60935] Updated weights for policy 0, policy_version 52180 (0.0009) [2023-10-13 23:06:13,411][60934] Updated weights for policy 1, policy_version 52572 (0.0008) [2023-10-13 23:06:13,733][60935] Updated weights for policy 0, policy_version 52190 (0.0009) [2023-10-13 23:06:13,782][60934] Updated weights for policy 1, policy_version 52582 (0.0008) [2023-10-13 23:06:14,154][60934] Updated weights for policy 1, policy_version 52592 (0.0007) [2023-10-13 23:06:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 107610112. Throughput: 0: 1660.5, 1: 1724.1. Samples: 26906276. Policy #0 lag: (min: 22.0, avg: 23.8, max: 50.0) [2023-10-13 23:06:16,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 23:06:17,628][60935] Updated weights for policy 0, policy_version 52200 (0.0009) [2023-10-13 23:06:17,998][60935] Updated weights for policy 0, policy_version 52210 (0.0010) [2023-10-13 23:06:18,166][60934] Updated weights for policy 1, policy_version 52602 (0.0007) [2023-10-13 23:06:18,364][60935] Updated weights for policy 0, policy_version 52220 (0.0008) [2023-10-13 23:06:18,523][60934] Updated weights for policy 1, policy_version 52612 (0.0007) [2023-10-13 23:06:18,891][60934] Updated weights for policy 1, policy_version 52622 (0.0007) [2023-10-13 23:06:19,255][60934] Updated weights for policy 1, policy_version 52632 (0.0009) [2023-10-13 23:06:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 107675648. Throughput: 0: 1689.2, 1: 1691.3. Samples: 26926300. Policy #0 lag: (min: 22.0, avg: 23.8, max: 50.0) [2023-10-13 23:06:21,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 23:06:22,460][60935] Updated weights for policy 0, policy_version 52230 (0.0009) [2023-10-13 23:06:22,825][60935] Updated weights for policy 0, policy_version 52240 (0.0010) [2023-10-13 23:06:23,205][60935] Updated weights for policy 0, policy_version 52250 (0.0009) [2023-10-13 23:06:23,326][60934] Updated weights for policy 1, policy_version 52642 (0.0008) [2023-10-13 23:06:23,717][60934] Updated weights for policy 1, policy_version 52652 (0.0007) [2023-10-13 23:06:24,087][60934] Updated weights for policy 1, policy_version 52662 (0.0008) [2023-10-13 23:06:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 107741184. Throughput: 0: 1698.5, 1: 1708.3. Samples: 26947548. Policy #0 lag: (min: 22.0, avg: 23.8, max: 50.0) [2023-10-13 23:06:26,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-13 23:06:27,343][60935] Updated weights for policy 0, policy_version 52260 (0.0008) [2023-10-13 23:06:27,741][60935] Updated weights for policy 0, policy_version 52270 (0.0008) [2023-10-13 23:06:28,072][60934] Updated weights for policy 1, policy_version 52672 (0.0009) [2023-10-13 23:06:28,110][60935] Updated weights for policy 0, policy_version 52280 (0.0008) [2023-10-13 23:06:28,446][60934] Updated weights for policy 1, policy_version 52682 (0.0007) [2023-10-13 23:06:28,804][60934] Updated weights for policy 1, policy_version 52692 (0.0007) [2023-10-13 23:06:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 107806720. Throughput: 0: 1678.3, 1: 1702.4. Samples: 26957090. Policy #0 lag: (min: 22.0, avg: 23.8, max: 50.0) [2023-10-13 23:06:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:06:32,010][60935] Updated weights for policy 0, policy_version 52290 (0.0010) [2023-10-13 23:06:32,383][60935] Updated weights for policy 0, policy_version 52300 (0.0010) [2023-10-13 23:06:32,738][60935] Updated weights for policy 0, policy_version 52310 (0.0009) [2023-10-13 23:06:32,745][60934] Updated weights for policy 1, policy_version 52702 (0.0008) [2023-10-13 23:06:33,105][60935] Updated weights for policy 0, policy_version 52320 (0.0008) [2023-10-13 23:06:33,107][60934] Updated weights for policy 1, policy_version 52712 (0.0008) [2023-10-13 23:06:33,470][60934] Updated weights for policy 1, policy_version 52722 (0.0010) [2023-10-13 23:06:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 107872256. Throughput: 0: 1698.6, 1: 1693.7. Samples: 26977634. Policy #0 lag: (min: 22.0, avg: 23.8, max: 50.0) [2023-10-13 23:06:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:06:37,200][60935] Updated weights for policy 0, policy_version 52330 (0.0007) [2023-10-13 23:06:37,548][60934] Updated weights for policy 1, policy_version 52732 (0.0009) [2023-10-13 23:06:37,574][60935] Updated weights for policy 0, policy_version 52340 (0.0009) [2023-10-13 23:06:37,913][60934] Updated weights for policy 1, policy_version 52742 (0.0008) [2023-10-13 23:06:37,940][60935] Updated weights for policy 0, policy_version 52350 (0.0009) [2023-10-13 23:06:38,277][60934] Updated weights for policy 1, policy_version 52752 (0.0009) [2023-10-13 23:06:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 107937792. Throughput: 0: 1696.6, 1: 1714.6. Samples: 26998214. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 23:06:41,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:06:42,079][60935] Updated weights for policy 0, policy_version 52360 (0.0008) [2023-10-13 23:06:42,381][60934] Updated weights for policy 1, policy_version 52762 (0.0011) [2023-10-13 23:06:42,439][60935] Updated weights for policy 0, policy_version 52370 (0.0009) [2023-10-13 23:06:42,746][60934] Updated weights for policy 1, policy_version 52772 (0.0008) [2023-10-13 23:06:42,810][60935] Updated weights for policy 0, policy_version 52380 (0.0009) [2023-10-13 23:06:43,109][60934] Updated weights for policy 1, policy_version 52782 (0.0007) [2023-10-13 23:06:43,474][60934] Updated weights for policy 1, policy_version 52792 (0.0009) [2023-10-13 23:06:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 108003328. Throughput: 0: 1686.4, 1: 1682.4. Samples: 27007528. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 23:06:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:06:46,691][60935] Updated weights for policy 0, policy_version 52390 (0.0008) [2023-10-13 23:06:47,070][60935] Updated weights for policy 0, policy_version 52400 (0.0007) [2023-10-13 23:06:47,442][60935] Updated weights for policy 0, policy_version 52410 (0.0008) [2023-10-13 23:06:47,516][60934] Updated weights for policy 1, policy_version 52802 (0.0009) [2023-10-13 23:06:47,874][60934] Updated weights for policy 1, policy_version 52812 (0.0009) [2023-10-13 23:06:48,239][60934] Updated weights for policy 1, policy_version 52822 (0.0008) [2023-10-13 23:06:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 108068864. Throughput: 0: 1706.8, 1: 1703.5. Samples: 27028626. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 23:06:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:06:51,478][60935] Updated weights for policy 0, policy_version 52420 (0.0008) [2023-10-13 23:06:51,850][60935] Updated weights for policy 0, policy_version 52430 (0.0008) [2023-10-13 23:06:51,941][60934] Updated weights for policy 1, policy_version 52832 (0.0008) [2023-10-13 23:06:52,217][60935] Updated weights for policy 0, policy_version 52440 (0.0009) [2023-10-13 23:06:52,297][60934] Updated weights for policy 1, policy_version 52842 (0.0008) [2023-10-13 23:06:52,664][60934] Updated weights for policy 1, policy_version 52852 (0.0009) [2023-10-13 23:06:56,206][60935] Updated weights for policy 0, policy_version 52450 (0.0008) [2023-10-13 23:06:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 108134400. Throughput: 0: 1701.7, 1: 1714.5. Samples: 27049826. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 23:06:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:06:56,259][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000052856_54427648.pth... [2023-10-13 23:06:56,299][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000051256_52789248.pth [2023-10-13 23:06:56,574][60935] Updated weights for policy 0, policy_version 52460 (0.0007) [2023-10-13 23:06:56,754][60934] Updated weights for policy 1, policy_version 52862 (0.0009) [2023-10-13 23:06:56,940][60935] Updated weights for policy 0, policy_version 52470 (0.0009) [2023-10-13 23:06:57,119][60934] Updated weights for policy 1, policy_version 52872 (0.0008) [2023-10-13 23:06:57,298][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000052480_53739520.pth... [2023-10-13 23:06:57,302][60935] Updated weights for policy 0, policy_version 52480 (0.0008) [2023-10-13 23:06:57,331][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000050880_52101120.pth [2023-10-13 23:06:57,489][60934] Updated weights for policy 1, policy_version 52882 (0.0007) [2023-10-13 23:07:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 108199936. Throughput: 0: 1704.3, 1: 1693.5. Samples: 27059178. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 23:07:01,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:07:01,372][60935] Updated weights for policy 0, policy_version 52490 (0.0007) [2023-10-13 23:07:01,393][60934] Updated weights for policy 1, policy_version 52892 (0.0008) [2023-10-13 23:07:01,737][60935] Updated weights for policy 0, policy_version 52500 (0.0008) [2023-10-13 23:07:01,758][60934] Updated weights for policy 1, policy_version 52902 (0.0007) [2023-10-13 23:07:02,101][60935] Updated weights for policy 0, policy_version 52510 (0.0008) [2023-10-13 23:07:02,121][60934] Updated weights for policy 1, policy_version 52912 (0.0010) [2023-10-13 23:07:05,968][60935] Updated weights for policy 0, policy_version 52520 (0.0009) [2023-10-13 23:07:06,145][60934] Updated weights for policy 1, policy_version 52922 (0.0009) [2023-10-13 23:07:06,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 108265472. Throughput: 0: 1706.2, 1: 1720.9. Samples: 27080522. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 23:07:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:07:06,332][60935] Updated weights for policy 0, policy_version 52530 (0.0008) [2023-10-13 23:07:06,517][60934] Updated weights for policy 1, policy_version 52932 (0.0008) [2023-10-13 23:07:06,697][60935] Updated weights for policy 0, policy_version 52540 (0.0007) [2023-10-13 23:07:06,881][60934] Updated weights for policy 1, policy_version 52942 (0.0008) [2023-10-13 23:07:07,248][60934] Updated weights for policy 1, policy_version 52952 (0.0009) [2023-10-13 23:07:10,784][60935] Updated weights for policy 0, policy_version 52550 (0.0007) [2023-10-13 23:07:11,152][60935] Updated weights for policy 0, policy_version 52560 (0.0008) [2023-10-13 23:07:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 108331008. Throughput: 0: 1695.2, 1: 1720.9. Samples: 27101272. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 23:07:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:07:11,324][60934] Updated weights for policy 1, policy_version 52962 (0.0008) [2023-10-13 23:07:11,517][60935] Updated weights for policy 0, policy_version 52570 (0.0008) [2023-10-13 23:07:11,696][60934] Updated weights for policy 1, policy_version 52972 (0.0008) [2023-10-13 23:07:12,053][60934] Updated weights for policy 1, policy_version 52982 (0.0010) [2023-10-13 23:07:15,465][60935] Updated weights for policy 0, policy_version 52580 (0.0007) [2023-10-13 23:07:15,843][60935] Updated weights for policy 0, policy_version 52590 (0.0010) [2023-10-13 23:07:15,954][60934] Updated weights for policy 1, policy_version 52992 (0.0008) [2023-10-13 23:07:16,221][60935] Updated weights for policy 0, policy_version 52600 (0.0008) [2023-10-13 23:07:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 108396544. Throughput: 0: 1709.0, 1: 1705.1. Samples: 27110726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:07:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:07:16,316][60934] Updated weights for policy 1, policy_version 53002 (0.0008) [2023-10-13 23:07:16,678][60934] Updated weights for policy 1, policy_version 53012 (0.0007) [2023-10-13 23:07:20,352][60935] Updated weights for policy 0, policy_version 52610 (0.0009) [2023-10-13 23:07:20,622][60934] Updated weights for policy 1, policy_version 53022 (0.0008) [2023-10-13 23:07:20,728][60935] Updated weights for policy 0, policy_version 52620 (0.0009) [2023-10-13 23:07:20,990][60934] Updated weights for policy 1, policy_version 53032 (0.0010) [2023-10-13 23:07:21,088][60935] Updated weights for policy 0, policy_version 52630 (0.0008) [2023-10-13 23:07:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 108462080. Throughput: 0: 1695.8, 1: 1724.4. Samples: 27131540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:07:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:07:21,357][60934] Updated weights for policy 1, policy_version 53042 (0.0007) [2023-10-13 23:07:21,463][60935] Updated weights for policy 0, policy_version 52640 (0.0007) [2023-10-13 23:07:25,308][60934] Updated weights for policy 1, policy_version 53052 (0.0008) [2023-10-13 23:07:25,410][60935] Updated weights for policy 0, policy_version 52650 (0.0008) [2023-10-13 23:07:25,675][60934] Updated weights for policy 1, policy_version 53062 (0.0007) [2023-10-13 23:07:25,783][60935] Updated weights for policy 0, policy_version 52660 (0.0008) [2023-10-13 23:07:26,042][60934] Updated weights for policy 1, policy_version 53072 (0.0008) [2023-10-13 23:07:26,154][60935] Updated weights for policy 0, policy_version 52670 (0.0008) [2023-10-13 23:07:26,248][59943] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 108560384. Throughput: 0: 1686.0, 1: 1719.7. Samples: 27151470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:07:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:07:30,053][60934] Updated weights for policy 1, policy_version 53082 (0.0010) [2023-10-13 23:07:30,266][60935] Updated weights for policy 0, policy_version 52680 (0.0008) [2023-10-13 23:07:30,418][60934] Updated weights for policy 1, policy_version 53092 (0.0008) [2023-10-13 23:07:30,630][60935] Updated weights for policy 0, policy_version 52690 (0.0009) [2023-10-13 23:07:30,782][60934] Updated weights for policy 1, policy_version 53102 (0.0007) [2023-10-13 23:07:31,005][60935] Updated weights for policy 0, policy_version 52700 (0.0008) [2023-10-13 23:07:31,147][60934] Updated weights for policy 1, policy_version 53112 (0.0008) [2023-10-13 23:07:31,248][59943] Fps is (10 sec: 19660.8, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 108658688. Throughput: 0: 1697.4, 1: 1727.0. Samples: 27161624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:07:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:07:35,082][60935] Updated weights for policy 0, policy_version 52710 (0.0009) [2023-10-13 23:07:35,184][60934] Updated weights for policy 1, policy_version 53122 (0.0009) [2023-10-13 23:07:35,457][60935] Updated weights for policy 0, policy_version 52720 (0.0009) [2023-10-13 23:07:35,547][60934] Updated weights for policy 1, policy_version 53132 (0.0009) [2023-10-13 23:07:35,828][60935] Updated weights for policy 0, policy_version 52730 (0.0008) [2023-10-13 23:07:35,919][60934] Updated weights for policy 1, policy_version 53142 (0.0008) [2023-10-13 23:07:36,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 108724224. Throughput: 0: 1697.8, 1: 1725.7. Samples: 27182682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:07:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:07:39,770][60935] Updated weights for policy 0, policy_version 52740 (0.0009) [2023-10-13 23:07:39,989][60934] Updated weights for policy 1, policy_version 53152 (0.0008) [2023-10-13 23:07:40,135][60935] Updated weights for policy 0, policy_version 52750 (0.0009) [2023-10-13 23:07:40,349][60934] Updated weights for policy 1, policy_version 53162 (0.0008) [2023-10-13 23:07:40,509][60935] Updated weights for policy 0, policy_version 52760 (0.0009) [2023-10-13 23:07:40,719][60934] Updated weights for policy 1, policy_version 53172 (0.0009) [2023-10-13 23:07:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 108789760. Throughput: 0: 1667.2, 1: 1702.5. Samples: 27201462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:07:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:07:44,550][60934] Updated weights for policy 1, policy_version 53182 (0.0009) [2023-10-13 23:07:44,701][60935] Updated weights for policy 0, policy_version 52770 (0.0010) [2023-10-13 23:07:44,919][60934] Updated weights for policy 1, policy_version 53192 (0.0008) [2023-10-13 23:07:45,074][60935] Updated weights for policy 0, policy_version 52780 (0.0009) [2023-10-13 23:07:45,283][60934] Updated weights for policy 1, policy_version 53202 (0.0007) [2023-10-13 23:07:45,439][60935] Updated weights for policy 0, policy_version 52790 (0.0009) [2023-10-13 23:07:45,796][60935] Updated weights for policy 0, policy_version 52800 (0.0008) [2023-10-13 23:07:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 108855296. Throughput: 0: 1689.3, 1: 1721.7. Samples: 27212674. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 23:07:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:07:49,255][60934] Updated weights for policy 1, policy_version 53212 (0.0009) [2023-10-13 23:07:49,611][60934] Updated weights for policy 1, policy_version 53222 (0.0009) [2023-10-13 23:07:49,933][60935] Updated weights for policy 0, policy_version 52810 (0.0008) [2023-10-13 23:07:49,974][60934] Updated weights for policy 1, policy_version 53232 (0.0008) [2023-10-13 23:07:50,305][60935] Updated weights for policy 0, policy_version 52820 (0.0007) [2023-10-13 23:07:50,660][60935] Updated weights for policy 0, policy_version 52830 (0.0008) [2023-10-13 23:07:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 108920832. Throughput: 0: 1676.8, 1: 1700.9. Samples: 27232516. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 23:07:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:07:53,900][60934] Updated weights for policy 1, policy_version 53242 (0.0009) [2023-10-13 23:07:54,278][60934] Updated weights for policy 1, policy_version 53252 (0.0008) [2023-10-13 23:07:54,640][60934] Updated weights for policy 1, policy_version 53262 (0.0007) [2023-10-13 23:07:54,747][60935] Updated weights for policy 0, policy_version 52840 (0.0008) [2023-10-13 23:07:55,002][60934] Updated weights for policy 1, policy_version 53272 (0.0007) [2023-10-13 23:07:55,121][60935] Updated weights for policy 0, policy_version 52850 (0.0007) [2023-10-13 23:07:55,490][60935] Updated weights for policy 0, policy_version 52860 (0.0009) [2023-10-13 23:07:56,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 108986368. Throughput: 0: 1658.4, 1: 1685.5. Samples: 27251744. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 23:07:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:07:59,110][60934] Updated weights for policy 1, policy_version 53282 (0.0009) [2023-10-13 23:07:59,470][60935] Updated weights for policy 0, policy_version 52870 (0.0009) [2023-10-13 23:07:59,479][60934] Updated weights for policy 1, policy_version 53292 (0.0007) [2023-10-13 23:07:59,842][60934] Updated weights for policy 1, policy_version 53302 (0.0008) [2023-10-13 23:07:59,843][60935] Updated weights for policy 0, policy_version 52880 (0.0009) [2023-10-13 23:08:00,215][60935] Updated weights for policy 0, policy_version 52890 (0.0010) [2023-10-13 23:08:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 109051904. Throughput: 0: 1680.8, 1: 1718.8. Samples: 27263708. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 23:08:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:08:03,856][60934] Updated weights for policy 1, policy_version 53312 (0.0008) [2023-10-13 23:08:04,214][60934] Updated weights for policy 1, policy_version 53322 (0.0009) [2023-10-13 23:08:04,325][60935] Updated weights for policy 0, policy_version 52900 (0.0008) [2023-10-13 23:08:04,587][60934] Updated weights for policy 1, policy_version 53332 (0.0009) [2023-10-13 23:08:04,710][60935] Updated weights for policy 0, policy_version 52910 (0.0008) [2023-10-13 23:08:05,074][60935] Updated weights for policy 0, policy_version 52920 (0.0008) [2023-10-13 23:08:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 109117440. Throughput: 0: 1670.3, 1: 1689.0. Samples: 27282710. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 23:08:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:08:08,676][60934] Updated weights for policy 1, policy_version 53342 (0.0009) [2023-10-13 23:08:09,046][60934] Updated weights for policy 1, policy_version 53352 (0.0007) [2023-10-13 23:08:09,048][60935] Updated weights for policy 0, policy_version 52930 (0.0009) [2023-10-13 23:08:09,409][60934] Updated weights for policy 1, policy_version 53362 (0.0009) [2023-10-13 23:08:09,417][60935] Updated weights for policy 0, policy_version 52940 (0.0009) [2023-10-13 23:08:09,779][60935] Updated weights for policy 0, policy_version 52950 (0.0010) [2023-10-13 23:08:10,147][60935] Updated weights for policy 0, policy_version 52960 (0.0011) [2023-10-13 23:08:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 109182976. Throughput: 0: 1670.1, 1: 1692.1. Samples: 27302772. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 23:08:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:08:13,500][60934] Updated weights for policy 1, policy_version 53372 (0.0009) [2023-10-13 23:08:13,866][60934] Updated weights for policy 1, policy_version 53382 (0.0007) [2023-10-13 23:08:14,174][60935] Updated weights for policy 0, policy_version 52970 (0.0008) [2023-10-13 23:08:14,247][60934] Updated weights for policy 1, policy_version 53392 (0.0007) [2023-10-13 23:08:14,540][60935] Updated weights for policy 0, policy_version 52980 (0.0008) [2023-10-13 23:08:14,923][60935] Updated weights for policy 0, policy_version 52990 (0.0009) [2023-10-13 23:08:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 109248512. Throughput: 0: 1684.1, 1: 1705.5. Samples: 27314156. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 23:08:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:08:18,375][60934] Updated weights for policy 1, policy_version 53402 (0.0007) [2023-10-13 23:08:18,745][60934] Updated weights for policy 1, policy_version 53412 (0.0007) [2023-10-13 23:08:18,935][60935] Updated weights for policy 0, policy_version 53000 (0.0008) [2023-10-13 23:08:19,112][60934] Updated weights for policy 1, policy_version 53422 (0.0007) [2023-10-13 23:08:19,304][60935] Updated weights for policy 0, policy_version 53010 (0.0007) [2023-10-13 23:08:19,470][60934] Updated weights for policy 1, policy_version 53432 (0.0009) [2023-10-13 23:08:19,681][60935] Updated weights for policy 0, policy_version 53020 (0.0007) [2023-10-13 23:08:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 109314048. Throughput: 0: 1660.7, 1: 1677.4. Samples: 27332896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:08:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:08:23,481][60934] Updated weights for policy 1, policy_version 53442 (0.0009) [2023-10-13 23:08:23,694][60935] Updated weights for policy 0, policy_version 53030 (0.0008) [2023-10-13 23:08:23,846][60934] Updated weights for policy 1, policy_version 53452 (0.0008) [2023-10-13 23:08:24,073][60935] Updated weights for policy 0, policy_version 53040 (0.0008) [2023-10-13 23:08:24,213][60934] Updated weights for policy 1, policy_version 53462 (0.0009) [2023-10-13 23:08:24,439][60935] Updated weights for policy 0, policy_version 53050 (0.0008) [2023-10-13 23:08:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 109379584. Throughput: 0: 1688.4, 1: 1697.6. Samples: 27353836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:08:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:08:28,203][60934] Updated weights for policy 1, policy_version 53472 (0.0009) [2023-10-13 23:08:28,545][60935] Updated weights for policy 0, policy_version 53060 (0.0010) [2023-10-13 23:08:28,564][60934] Updated weights for policy 1, policy_version 53482 (0.0007) [2023-10-13 23:08:28,918][60935] Updated weights for policy 0, policy_version 53070 (0.0008) [2023-10-13 23:08:28,935][60934] Updated weights for policy 1, policy_version 53492 (0.0007) [2023-10-13 23:08:29,284][60935] Updated weights for policy 0, policy_version 53080 (0.0008) [2023-10-13 23:08:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 109445120. Throughput: 0: 1683.2, 1: 1688.5. Samples: 27364400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:08:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:08:32,988][60934] Updated weights for policy 1, policy_version 53502 (0.0009) [2023-10-13 23:08:33,249][60935] Updated weights for policy 0, policy_version 53090 (0.0007) [2023-10-13 23:08:33,345][60934] Updated weights for policy 1, policy_version 53512 (0.0007) [2023-10-13 23:08:33,617][60935] Updated weights for policy 0, policy_version 53100 (0.0009) [2023-10-13 23:08:33,706][60934] Updated weights for policy 1, policy_version 53522 (0.0008) [2023-10-13 23:08:33,989][60935] Updated weights for policy 0, policy_version 53110 (0.0008) [2023-10-13 23:08:34,349][60935] Updated weights for policy 0, policy_version 53120 (0.0009) [2023-10-13 23:08:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 109510656. Throughput: 0: 1670.1, 1: 1691.2. Samples: 27383772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:08:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:08:37,586][60934] Updated weights for policy 1, policy_version 53532 (0.0008) [2023-10-13 23:08:37,950][60934] Updated weights for policy 1, policy_version 53542 (0.0011) [2023-10-13 23:08:38,319][60934] Updated weights for policy 1, policy_version 53552 (0.0007) [2023-10-13 23:08:38,399][60935] Updated weights for policy 0, policy_version 53130 (0.0009) [2023-10-13 23:08:38,765][60935] Updated weights for policy 0, policy_version 53140 (0.0010) [2023-10-13 23:08:39,141][60935] Updated weights for policy 0, policy_version 53150 (0.0008) [2023-10-13 23:08:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 109576192. Throughput: 0: 1697.3, 1: 1704.4. Samples: 27404822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:08:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:08:42,374][60934] Updated weights for policy 1, policy_version 53562 (0.0008) [2023-10-13 23:08:42,748][60934] Updated weights for policy 1, policy_version 53572 (0.0008) [2023-10-13 23:08:43,109][60934] Updated weights for policy 1, policy_version 53582 (0.0007) [2023-10-13 23:08:43,215][60935] Updated weights for policy 0, policy_version 53160 (0.0007) [2023-10-13 23:08:43,475][60934] Updated weights for policy 1, policy_version 53592 (0.0008) [2023-10-13 23:08:43,587][60935] Updated weights for policy 0, policy_version 53170 (0.0007) [2023-10-13 23:08:43,954][60935] Updated weights for policy 0, policy_version 53180 (0.0008) [2023-10-13 23:08:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 109641728. Throughput: 0: 1672.3, 1: 1675.6. Samples: 27414366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:08:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:08:47,720][60934] Updated weights for policy 1, policy_version 53602 (0.0008) [2023-10-13 23:08:47,831][60935] Updated weights for policy 0, policy_version 53190 (0.0008) [2023-10-13 23:08:48,084][60934] Updated weights for policy 1, policy_version 53612 (0.0009) [2023-10-13 23:08:48,192][60935] Updated weights for policy 0, policy_version 53200 (0.0009) [2023-10-13 23:08:48,461][60934] Updated weights for policy 1, policy_version 53622 (0.0008) [2023-10-13 23:08:48,555][60935] Updated weights for policy 0, policy_version 53210 (0.0009) [2023-10-13 23:08:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 109707264. Throughput: 0: 1691.8, 1: 1693.3. Samples: 27435040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:08:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:08:52,279][60934] Updated weights for policy 1, policy_version 53632 (0.0009) [2023-10-13 23:08:52,513][60935] Updated weights for policy 0, policy_version 53220 (0.0008) [2023-10-13 23:08:52,645][60934] Updated weights for policy 1, policy_version 53642 (0.0009) [2023-10-13 23:08:52,896][60935] Updated weights for policy 0, policy_version 53230 (0.0009) [2023-10-13 23:08:53,014][60934] Updated weights for policy 1, policy_version 53652 (0.0009) [2023-10-13 23:08:53,261][60935] Updated weights for policy 0, policy_version 53240 (0.0009) [2023-10-13 23:08:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13551.5). Total num frames: 109772800. Throughput: 0: 1705.0, 1: 1694.1. Samples: 27455730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:08:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:08:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000053656_55246848.pth... [2023-10-13 23:08:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000053248_54525952.pth... [2023-10-13 23:08:56,301][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000052056_53608448.pth [2023-10-13 23:08:56,305][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000051680_52920320.pth [2023-10-13 23:08:57,032][60934] Updated weights for policy 1, policy_version 53662 (0.0009) [2023-10-13 23:08:57,367][60935] Updated weights for policy 0, policy_version 53250 (0.0009) [2023-10-13 23:08:57,405][60934] Updated weights for policy 1, policy_version 53672 (0.0008) [2023-10-13 23:08:57,741][60935] Updated weights for policy 0, policy_version 53260 (0.0007) [2023-10-13 23:08:57,762][60934] Updated weights for policy 1, policy_version 53682 (0.0010) [2023-10-13 23:08:58,119][60935] Updated weights for policy 0, policy_version 53270 (0.0008) [2023-10-13 23:08:58,484][60935] Updated weights for policy 0, policy_version 53280 (0.0007) [2023-10-13 23:09:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 109838336. Throughput: 0: 1679.6, 1: 1673.2. Samples: 27465028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:09:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:09:01,821][60934] Updated weights for policy 1, policy_version 53692 (0.0009) [2023-10-13 23:09:02,186][60934] Updated weights for policy 1, policy_version 53702 (0.0008) [2023-10-13 23:09:02,523][60935] Updated weights for policy 0, policy_version 53290 (0.0009) [2023-10-13 23:09:02,551][60934] Updated weights for policy 1, policy_version 53712 (0.0009) [2023-10-13 23:09:02,889][60935] Updated weights for policy 0, policy_version 53300 (0.0007) [2023-10-13 23:09:03,251][60935] Updated weights for policy 0, policy_version 53310 (0.0011) [2023-10-13 23:09:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 109903872. Throughput: 0: 1698.2, 1: 1700.8. Samples: 27485850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:09:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:09:06,434][60934] Updated weights for policy 1, policy_version 53722 (0.0007) [2023-10-13 23:09:06,792][60934] Updated weights for policy 1, policy_version 53732 (0.0008) [2023-10-13 23:09:07,155][60934] Updated weights for policy 1, policy_version 53742 (0.0007) [2023-10-13 23:09:07,384][60935] Updated weights for policy 0, policy_version 53320 (0.0009) [2023-10-13 23:09:07,516][60934] Updated weights for policy 1, policy_version 53752 (0.0008) [2023-10-13 23:09:07,750][60935] Updated weights for policy 0, policy_version 53330 (0.0008) [2023-10-13 23:09:08,125][60935] Updated weights for policy 0, policy_version 53340 (0.0009) [2023-10-13 23:09:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 109969408. Throughput: 0: 1697.8, 1: 1701.7. Samples: 27506816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:09:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:09:11,602][60934] Updated weights for policy 1, policy_version 53762 (0.0007) [2023-10-13 23:09:11,964][60934] Updated weights for policy 1, policy_version 53772 (0.0009) [2023-10-13 23:09:12,253][60935] Updated weights for policy 0, policy_version 53350 (0.0010) [2023-10-13 23:09:12,335][60934] Updated weights for policy 1, policy_version 53782 (0.0008) [2023-10-13 23:09:12,623][60935] Updated weights for policy 0, policy_version 53360 (0.0008) [2023-10-13 23:09:12,988][60935] Updated weights for policy 0, policy_version 53370 (0.0008) [2023-10-13 23:09:16,243][60934] Updated weights for policy 1, policy_version 53792 (0.0010) [2023-10-13 23:09:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 110034944. Throughput: 0: 1680.8, 1: 1693.3. Samples: 27516230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:09:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:09:16,608][60934] Updated weights for policy 1, policy_version 53802 (0.0010) [2023-10-13 23:09:16,982][60934] Updated weights for policy 1, policy_version 53812 (0.0009) [2023-10-13 23:09:17,021][60935] Updated weights for policy 0, policy_version 53380 (0.0008) [2023-10-13 23:09:17,383][60935] Updated weights for policy 0, policy_version 53390 (0.0010) [2023-10-13 23:09:17,748][60935] Updated weights for policy 0, policy_version 53400 (0.0011) [2023-10-13 23:09:21,007][60934] Updated weights for policy 1, policy_version 53822 (0.0009) [2023-10-13 23:09:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 110100480. Throughput: 0: 1707.1, 1: 1703.8. Samples: 27537260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:09:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:09:21,376][60934] Updated weights for policy 1, policy_version 53832 (0.0007) [2023-10-13 23:09:21,699][60935] Updated weights for policy 0, policy_version 53410 (0.0008) [2023-10-13 23:09:21,746][60934] Updated weights for policy 1, policy_version 53842 (0.0009) [2023-10-13 23:09:22,063][60935] Updated weights for policy 0, policy_version 53420 (0.0009) [2023-10-13 23:09:22,432][60935] Updated weights for policy 0, policy_version 53430 (0.0009) [2023-10-13 23:09:22,807][60935] Updated weights for policy 0, policy_version 53440 (0.0008) [2023-10-13 23:09:25,736][60934] Updated weights for policy 1, policy_version 53852 (0.0007) [2023-10-13 23:09:26,096][60934] Updated weights for policy 1, policy_version 53862 (0.0008) [2023-10-13 23:09:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 110166016. Throughput: 0: 1703.7, 1: 1706.6. Samples: 27558288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:09:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:09:26,472][60934] Updated weights for policy 1, policy_version 53872 (0.0010) [2023-10-13 23:09:26,838][60935] Updated weights for policy 0, policy_version 53450 (0.0007) [2023-10-13 23:09:27,205][60935] Updated weights for policy 0, policy_version 53460 (0.0008) [2023-10-13 23:09:27,576][60935] Updated weights for policy 0, policy_version 53470 (0.0009) [2023-10-13 23:09:30,369][60934] Updated weights for policy 1, policy_version 53882 (0.0007) [2023-10-13 23:09:30,743][60934] Updated weights for policy 1, policy_version 53892 (0.0008) [2023-10-13 23:09:31,105][60934] Updated weights for policy 1, policy_version 53902 (0.0007) [2023-10-13 23:09:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 110231552. Throughput: 0: 1694.9, 1: 1706.3. Samples: 27567418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:09:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:09:31,468][60934] Updated weights for policy 1, policy_version 53912 (0.0008) [2023-10-13 23:09:31,650][60935] Updated weights for policy 0, policy_version 53480 (0.0009) [2023-10-13 23:09:32,016][60935] Updated weights for policy 0, policy_version 53490 (0.0008) [2023-10-13 23:09:32,381][60935] Updated weights for policy 0, policy_version 53500 (0.0007) [2023-10-13 23:09:35,454][60934] Updated weights for policy 1, policy_version 53922 (0.0008) [2023-10-13 23:09:35,813][60934] Updated weights for policy 1, policy_version 53932 (0.0010) [2023-10-13 23:09:36,175][60934] Updated weights for policy 1, policy_version 53942 (0.0010) [2023-10-13 23:09:36,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 110297088. Throughput: 0: 1696.5, 1: 1718.4. Samples: 27588708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:09:36,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:09:36,252][60935] Updated weights for policy 0, policy_version 53510 (0.0009) [2023-10-13 23:09:36,614][60935] Updated weights for policy 0, policy_version 53520 (0.0007) [2023-10-13 23:09:36,983][60935] Updated weights for policy 0, policy_version 53530 (0.0008) [2023-10-13 23:09:40,062][60934] Updated weights for policy 1, policy_version 53952 (0.0009) [2023-10-13 23:09:40,435][60934] Updated weights for policy 1, policy_version 53962 (0.0010) [2023-10-13 23:09:40,797][60934] Updated weights for policy 1, policy_version 53972 (0.0009) [2023-10-13 23:09:41,108][60935] Updated weights for policy 0, policy_version 53540 (0.0009) [2023-10-13 23:09:41,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 110395392. Throughput: 0: 1700.8, 1: 1705.2. Samples: 27608996. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-13 23:09:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:09:41,488][60935] Updated weights for policy 0, policy_version 53550 (0.0008) [2023-10-13 23:09:41,848][60935] Updated weights for policy 0, policy_version 53560 (0.0009) [2023-10-13 23:09:44,875][60934] Updated weights for policy 1, policy_version 53982 (0.0010) [2023-10-13 23:09:45,247][60934] Updated weights for policy 1, policy_version 53992 (0.0009) [2023-10-13 23:09:45,613][60934] Updated weights for policy 1, policy_version 54002 (0.0009) [2023-10-13 23:09:45,767][60935] Updated weights for policy 0, policy_version 53570 (0.0009) [2023-10-13 23:09:46,131][60935] Updated weights for policy 0, policy_version 53580 (0.0009) [2023-10-13 23:09:46,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 110460928. Throughput: 0: 1699.0, 1: 1723.8. Samples: 27619052. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-13 23:09:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:09:46,490][60935] Updated weights for policy 0, policy_version 53590 (0.0009) [2023-10-13 23:09:46,859][60935] Updated weights for policy 0, policy_version 53600 (0.0008) [2023-10-13 23:09:49,645][60934] Updated weights for policy 1, policy_version 54012 (0.0007) [2023-10-13 23:09:50,018][60934] Updated weights for policy 1, policy_version 54022 (0.0009) [2023-10-13 23:09:50,389][60934] Updated weights for policy 1, policy_version 54032 (0.0008) [2023-10-13 23:09:50,838][60935] Updated weights for policy 0, policy_version 53610 (0.0008) [2023-10-13 23:09:51,204][60935] Updated weights for policy 0, policy_version 53620 (0.0009) [2023-10-13 23:09:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 110526464. Throughput: 0: 1707.3, 1: 1723.0. Samples: 27640216. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-13 23:09:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:09:51,582][60935] Updated weights for policy 0, policy_version 53630 (0.0008) [2023-10-13 23:09:54,284][60934] Updated weights for policy 1, policy_version 54042 (0.0008) [2023-10-13 23:09:54,654][60934] Updated weights for policy 1, policy_version 54052 (0.0008) [2023-10-13 23:09:55,019][60934] Updated weights for policy 1, policy_version 54062 (0.0008) [2023-10-13 23:09:55,383][60934] Updated weights for policy 1, policy_version 54072 (0.0007) [2023-10-13 23:09:55,650][60935] Updated weights for policy 0, policy_version 53640 (0.0009) [2023-10-13 23:09:56,022][60935] Updated weights for policy 0, policy_version 53650 (0.0007) [2023-10-13 23:09:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 110592000. Throughput: 0: 1698.0, 1: 1699.5. Samples: 27659702. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-13 23:09:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:09:56,396][60935] Updated weights for policy 0, policy_version 53660 (0.0007) [2023-10-13 23:09:59,348][60934] Updated weights for policy 1, policy_version 54082 (0.0007) [2023-10-13 23:09:59,717][60934] Updated weights for policy 1, policy_version 54092 (0.0007) [2023-10-13 23:10:00,092][60934] Updated weights for policy 1, policy_version 54102 (0.0008) [2023-10-13 23:10:00,384][60935] Updated weights for policy 0, policy_version 53670 (0.0008) [2023-10-13 23:10:00,757][60935] Updated weights for policy 0, policy_version 53680 (0.0007) [2023-10-13 23:10:01,132][60935] Updated weights for policy 0, policy_version 53690 (0.0008) [2023-10-13 23:10:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 110657536. Throughput: 0: 1705.4, 1: 1728.2. Samples: 27670740. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-13 23:10:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:10:04,183][60934] Updated weights for policy 1, policy_version 54112 (0.0007) [2023-10-13 23:10:04,541][60934] Updated weights for policy 1, policy_version 54122 (0.0009) [2023-10-13 23:10:04,913][60934] Updated weights for policy 1, policy_version 54132 (0.0010) [2023-10-13 23:10:05,184][60935] Updated weights for policy 0, policy_version 53700 (0.0008) [2023-10-13 23:10:05,559][60935] Updated weights for policy 0, policy_version 53710 (0.0008) [2023-10-13 23:10:05,929][60935] Updated weights for policy 0, policy_version 53720 (0.0007) [2023-10-13 23:10:06,248][59943] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 110755840. Throughput: 0: 1699.2, 1: 1713.5. Samples: 27690832. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-13 23:10:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:10:08,878][60934] Updated weights for policy 1, policy_version 54142 (0.0008) [2023-10-13 23:10:09,241][60934] Updated weights for policy 1, policy_version 54152 (0.0008) [2023-10-13 23:10:09,614][60934] Updated weights for policy 1, policy_version 54162 (0.0009) [2023-10-13 23:10:09,873][60935] Updated weights for policy 0, policy_version 53730 (0.0011) [2023-10-13 23:10:10,248][60935] Updated weights for policy 0, policy_version 53740 (0.0010) [2023-10-13 23:10:10,616][60935] Updated weights for policy 0, policy_version 53750 (0.0009) [2023-10-13 23:10:10,996][60935] Updated weights for policy 0, policy_version 53760 (0.0008) [2023-10-13 23:10:11,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 110821376. Throughput: 0: 1682.3, 1: 1698.2. Samples: 27710408. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-13 23:10:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:10:13,493][60934] Updated weights for policy 1, policy_version 54172 (0.0009) [2023-10-13 23:10:13,851][60934] Updated weights for policy 1, policy_version 54182 (0.0008) [2023-10-13 23:10:14,215][60934] Updated weights for policy 1, policy_version 54192 (0.0008) [2023-10-13 23:10:14,925][60935] Updated weights for policy 0, policy_version 53770 (0.0008) [2023-10-13 23:10:15,286][60935] Updated weights for policy 0, policy_version 53780 (0.0007) [2023-10-13 23:10:15,665][60935] Updated weights for policy 0, policy_version 53790 (0.0007) [2023-10-13 23:10:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 110886912. Throughput: 0: 1709.5, 1: 1724.5. Samples: 27721950. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:10:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:10:18,285][60934] Updated weights for policy 1, policy_version 54202 (0.0009) [2023-10-13 23:10:18,657][60934] Updated weights for policy 1, policy_version 54212 (0.0008) [2023-10-13 23:10:19,020][60934] Updated weights for policy 1, policy_version 54222 (0.0008) [2023-10-13 23:10:19,388][60934] Updated weights for policy 1, policy_version 54232 (0.0008) [2023-10-13 23:10:19,869][60935] Updated weights for policy 0, policy_version 53800 (0.0008) [2023-10-13 23:10:20,251][60935] Updated weights for policy 0, policy_version 53810 (0.0009) [2023-10-13 23:10:20,621][60935] Updated weights for policy 0, policy_version 53820 (0.0008) [2023-10-13 23:10:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 110952448. Throughput: 0: 1701.4, 1: 1692.3. Samples: 27741422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:10:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:10:23,476][60934] Updated weights for policy 1, policy_version 54242 (0.0009) [2023-10-13 23:10:23,843][60934] Updated weights for policy 1, policy_version 54252 (0.0007) [2023-10-13 23:10:24,218][60934] Updated weights for policy 1, policy_version 54262 (0.0009) [2023-10-13 23:10:24,544][60935] Updated weights for policy 0, policy_version 53830 (0.0008) [2023-10-13 23:10:24,915][60935] Updated weights for policy 0, policy_version 53840 (0.0009) [2023-10-13 23:10:25,282][60935] Updated weights for policy 0, policy_version 53850 (0.0008) [2023-10-13 23:10:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 111017984. Throughput: 0: 1678.3, 1: 1708.3. Samples: 27761394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:10:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:10:28,239][60934] Updated weights for policy 1, policy_version 54272 (0.0009) [2023-10-13 23:10:28,605][60934] Updated weights for policy 1, policy_version 54282 (0.0009) [2023-10-13 23:10:28,978][60934] Updated weights for policy 1, policy_version 54292 (0.0010) [2023-10-13 23:10:29,441][60935] Updated weights for policy 0, policy_version 53860 (0.0007) [2023-10-13 23:10:29,829][60935] Updated weights for policy 0, policy_version 53870 (0.0008) [2023-10-13 23:10:30,196][60935] Updated weights for policy 0, policy_version 53880 (0.0008) [2023-10-13 23:10:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 111083520. Throughput: 0: 1703.7, 1: 1705.9. Samples: 27772482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:10:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:10:32,917][60934] Updated weights for policy 1, policy_version 54302 (0.0010) [2023-10-13 23:10:33,285][60934] Updated weights for policy 1, policy_version 54312 (0.0011) [2023-10-13 23:10:33,662][60934] Updated weights for policy 1, policy_version 54322 (0.0009) [2023-10-13 23:10:34,341][60935] Updated weights for policy 0, policy_version 53890 (0.0009) [2023-10-13 23:10:34,718][60935] Updated weights for policy 0, policy_version 53900 (0.0008) [2023-10-13 23:10:35,078][60935] Updated weights for policy 0, policy_version 53910 (0.0010) [2023-10-13 23:10:35,445][60935] Updated weights for policy 0, policy_version 53920 (0.0011) [2023-10-13 23:10:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 111149056. Throughput: 0: 1685.0, 1: 1691.9. Samples: 27792176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:10:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:10:37,613][60934] Updated weights for policy 1, policy_version 54332 (0.0009) [2023-10-13 23:10:37,976][60934] Updated weights for policy 1, policy_version 54342 (0.0008) [2023-10-13 23:10:38,342][60934] Updated weights for policy 1, policy_version 54352 (0.0008) [2023-10-13 23:10:39,536][60935] Updated weights for policy 0, policy_version 53930 (0.0010) [2023-10-13 23:10:39,905][60935] Updated weights for policy 0, policy_version 53940 (0.0007) [2023-10-13 23:10:40,265][60935] Updated weights for policy 0, policy_version 53950 (0.0011) [2023-10-13 23:10:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 111214592. Throughput: 0: 1676.9, 1: 1720.9. Samples: 27812602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:10:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:10:42,382][60934] Updated weights for policy 1, policy_version 54362 (0.0007) [2023-10-13 23:10:42,745][60934] Updated weights for policy 1, policy_version 54372 (0.0007) [2023-10-13 23:10:43,111][60934] Updated weights for policy 1, policy_version 54382 (0.0007) [2023-10-13 23:10:43,477][60934] Updated weights for policy 1, policy_version 54392 (0.0009) [2023-10-13 23:10:44,093][60935] Updated weights for policy 0, policy_version 53960 (0.0008) [2023-10-13 23:10:44,461][60935] Updated weights for policy 0, policy_version 53970 (0.0008) [2023-10-13 23:10:44,823][60935] Updated weights for policy 0, policy_version 53980 (0.0008) [2023-10-13 23:10:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 111280128. Throughput: 0: 1702.4, 1: 1686.3. Samples: 27823228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:10:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:10:47,482][60934] Updated weights for policy 1, policy_version 54402 (0.0009) [2023-10-13 23:10:47,856][60934] Updated weights for policy 1, policy_version 54412 (0.0008) [2023-10-13 23:10:48,225][60934] Updated weights for policy 1, policy_version 54422 (0.0008) [2023-10-13 23:10:48,808][60935] Updated weights for policy 0, policy_version 53990 (0.0009) [2023-10-13 23:10:49,178][60935] Updated weights for policy 0, policy_version 54000 (0.0009) [2023-10-13 23:10:49,546][60935] Updated weights for policy 0, policy_version 54010 (0.0007) [2023-10-13 23:10:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 111345664. Throughput: 0: 1683.2, 1: 1701.3. Samples: 27843138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:10:51,248][59943] Avg episode reward: [(0, '-0.070'), (1, '0.000')] [2023-10-13 23:10:52,260][60934] Updated weights for policy 1, policy_version 54432 (0.0010) [2023-10-13 23:10:52,631][60934] Updated weights for policy 1, policy_version 54442 (0.0009) [2023-10-13 23:10:53,000][60934] Updated weights for policy 1, policy_version 54452 (0.0010) [2023-10-13 23:10:53,599][60935] Updated weights for policy 0, policy_version 54020 (0.0008) [2023-10-13 23:10:53,972][60935] Updated weights for policy 0, policy_version 54030 (0.0007) [2023-10-13 23:10:54,337][60935] Updated weights for policy 0, policy_version 54040 (0.0007) [2023-10-13 23:10:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 111411200. Throughput: 0: 1701.8, 1: 1711.9. Samples: 27864022. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:10:56,250][59943] Avg episode reward: [(0, '-0.070'), (1, '0.000')] [2023-10-13 23:10:56,262][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000054456_56066048.pth... [2023-10-13 23:10:56,262][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000054048_55345152.pth... [2023-10-13 23:10:56,295][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000052856_54427648.pth [2023-10-13 23:10:56,298][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000052480_53739520.pth [2023-10-13 23:10:57,078][60934] Updated weights for policy 1, policy_version 54462 (0.0008) [2023-10-13 23:10:57,434][60934] Updated weights for policy 1, policy_version 54472 (0.0009) [2023-10-13 23:10:57,803][60934] Updated weights for policy 1, policy_version 54482 (0.0009) [2023-10-13 23:10:58,328][60935] Updated weights for policy 0, policy_version 54050 (0.0007) [2023-10-13 23:10:58,698][60935] Updated weights for policy 0, policy_version 54060 (0.0008) [2023-10-13 23:10:59,071][60935] Updated weights for policy 0, policy_version 54070 (0.0007) [2023-10-13 23:10:59,449][60935] Updated weights for policy 0, policy_version 54080 (0.0009) [2023-10-13 23:11:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 111476736. Throughput: 0: 1689.5, 1: 1688.4. Samples: 27873952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:11:01,249][59943] Avg episode reward: [(0, '-0.070'), (1, '0.000')] [2023-10-13 23:11:01,660][60934] Updated weights for policy 1, policy_version 54492 (0.0009) [2023-10-13 23:11:02,029][60934] Updated weights for policy 1, policy_version 54502 (0.0008) [2023-10-13 23:11:02,397][60934] Updated weights for policy 1, policy_version 54512 (0.0009) [2023-10-13 23:11:03,273][60935] Updated weights for policy 0, policy_version 54090 (0.0007) [2023-10-13 23:11:03,640][60935] Updated weights for policy 0, policy_version 54100 (0.0008) [2023-10-13 23:11:04,005][60935] Updated weights for policy 0, policy_version 54110 (0.0011) [2023-10-13 23:11:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 111542272. Throughput: 0: 1680.8, 1: 1721.5. Samples: 27894522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:11:06,249][59943] Avg episode reward: [(0, '-0.070'), (1, '0.000')] [2023-10-13 23:11:06,455][60934] Updated weights for policy 1, policy_version 54522 (0.0008) [2023-10-13 23:11:06,814][60934] Updated weights for policy 1, policy_version 54532 (0.0008) [2023-10-13 23:11:07,181][60934] Updated weights for policy 1, policy_version 54542 (0.0010) [2023-10-13 23:11:07,550][60934] Updated weights for policy 1, policy_version 54552 (0.0010) [2023-10-13 23:11:08,075][60935] Updated weights for policy 0, policy_version 54120 (0.0010) [2023-10-13 23:11:08,448][60935] Updated weights for policy 0, policy_version 54130 (0.0008) [2023-10-13 23:11:08,826][60935] Updated weights for policy 0, policy_version 54140 (0.0008) [2023-10-13 23:11:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 111607808. Throughput: 0: 1702.8, 1: 1719.9. Samples: 27915416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:11:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:11:11,649][60934] Updated weights for policy 1, policy_version 54562 (0.0008) [2023-10-13 23:11:12,014][60934] Updated weights for policy 1, policy_version 54572 (0.0009) [2023-10-13 23:11:12,379][60934] Updated weights for policy 1, policy_version 54582 (0.0011) [2023-10-13 23:11:12,815][60935] Updated weights for policy 0, policy_version 54150 (0.0009) [2023-10-13 23:11:13,182][60935] Updated weights for policy 0, policy_version 54160 (0.0010) [2023-10-13 23:11:13,546][60935] Updated weights for policy 0, policy_version 54170 (0.0010) [2023-10-13 23:11:16,152][60934] Updated weights for policy 1, policy_version 54592 (0.0007) [2023-10-13 23:11:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 111673344. Throughput: 0: 1680.3, 1: 1703.2. Samples: 27924738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:11:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:11:16,522][60934] Updated weights for policy 1, policy_version 54602 (0.0008) [2023-10-13 23:11:16,886][60934] Updated weights for policy 1, policy_version 54612 (0.0010) [2023-10-13 23:11:17,878][60935] Updated weights for policy 0, policy_version 54180 (0.0007) [2023-10-13 23:11:18,259][60935] Updated weights for policy 0, policy_version 54190 (0.0009) [2023-10-13 23:11:18,630][60935] Updated weights for policy 0, policy_version 54200 (0.0009) [2023-10-13 23:11:20,893][60934] Updated weights for policy 1, policy_version 54622 (0.0008) [2023-10-13 23:11:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 111738880. Throughput: 0: 1685.8, 1: 1723.0. Samples: 27945574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:11:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:11:21,261][60934] Updated weights for policy 1, policy_version 54632 (0.0008) [2023-10-13 23:11:21,622][60934] Updated weights for policy 1, policy_version 54642 (0.0007) [2023-10-13 23:11:22,699][60935] Updated weights for policy 0, policy_version 54210 (0.0008) [2023-10-13 23:11:23,075][60935] Updated weights for policy 0, policy_version 54220 (0.0008) [2023-10-13 23:11:23,450][60935] Updated weights for policy 0, policy_version 54230 (0.0009) [2023-10-13 23:11:23,810][60935] Updated weights for policy 0, policy_version 54240 (0.0008) [2023-10-13 23:11:25,581][60934] Updated weights for policy 1, policy_version 54652 (0.0008) [2023-10-13 23:11:25,941][60934] Updated weights for policy 1, policy_version 54662 (0.0008) [2023-10-13 23:11:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 111804416. Throughput: 0: 1704.2, 1: 1715.7. Samples: 27966496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:11:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:11:26,312][60934] Updated weights for policy 1, policy_version 54672 (0.0008) [2023-10-13 23:11:27,668][60935] Updated weights for policy 0, policy_version 54250 (0.0011) [2023-10-13 23:11:28,041][60935] Updated weights for policy 0, policy_version 54260 (0.0010) [2023-10-13 23:11:28,416][60935] Updated weights for policy 0, policy_version 54270 (0.0010) [2023-10-13 23:11:30,366][60934] Updated weights for policy 1, policy_version 54682 (0.0007) [2023-10-13 23:11:30,725][60934] Updated weights for policy 1, policy_version 54692 (0.0009) [2023-10-13 23:11:31,107][60934] Updated weights for policy 1, policy_version 54702 (0.0009) [2023-10-13 23:11:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 111869952. Throughput: 0: 1670.2, 1: 1720.7. Samples: 27975818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:11:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:11:31,467][60934] Updated weights for policy 1, policy_version 54712 (0.0009) [2023-10-13 23:11:32,422][60935] Updated weights for policy 0, policy_version 54280 (0.0008) [2023-10-13 23:11:32,785][60935] Updated weights for policy 0, policy_version 54290 (0.0010) [2023-10-13 23:11:33,152][60935] Updated weights for policy 0, policy_version 54300 (0.0010) [2023-10-13 23:11:35,564][60934] Updated weights for policy 1, policy_version 54722 (0.0010) [2023-10-13 23:11:35,925][60934] Updated weights for policy 1, policy_version 54732 (0.0008) [2023-10-13 23:11:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 111935488. Throughput: 0: 1690.3, 1: 1722.9. Samples: 27996730. Policy #0 lag: (min: 19.0, avg: 21.7, max: 51.0) [2023-10-13 23:11:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:11:36,296][60934] Updated weights for policy 1, policy_version 54742 (0.0009) [2023-10-13 23:11:37,245][60935] Updated weights for policy 0, policy_version 54310 (0.0009) [2023-10-13 23:11:37,611][60935] Updated weights for policy 0, policy_version 54320 (0.0009) [2023-10-13 23:11:37,984][60935] Updated weights for policy 0, policy_version 54330 (0.0010) [2023-10-13 23:11:40,301][60934] Updated weights for policy 1, policy_version 54752 (0.0009) [2023-10-13 23:11:40,665][60934] Updated weights for policy 1, policy_version 54762 (0.0011) [2023-10-13 23:11:41,028][60934] Updated weights for policy 1, policy_version 54772 (0.0010) [2023-10-13 23:11:41,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 112033792. Throughput: 0: 1693.7, 1: 1708.0. Samples: 28017094. Policy #0 lag: (min: 19.0, avg: 21.7, max: 51.0) [2023-10-13 23:11:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 23:11:42,111][60935] Updated weights for policy 0, policy_version 54340 (0.0009) [2023-10-13 23:11:42,488][60935] Updated weights for policy 0, policy_version 54350 (0.0009) [2023-10-13 23:11:42,850][60935] Updated weights for policy 0, policy_version 54360 (0.0008) [2023-10-13 23:11:45,020][60934] Updated weights for policy 1, policy_version 54782 (0.0008) [2023-10-13 23:11:45,385][60934] Updated weights for policy 1, policy_version 54792 (0.0010) [2023-10-13 23:11:45,757][60934] Updated weights for policy 1, policy_version 54802 (0.0009) [2023-10-13 23:11:46,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 112099328. Throughput: 0: 1678.2, 1: 1720.2. Samples: 28026880. Policy #0 lag: (min: 19.0, avg: 21.7, max: 51.0) [2023-10-13 23:11:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 23:11:46,849][60935] Updated weights for policy 0, policy_version 54370 (0.0007) [2023-10-13 23:11:47,218][60935] Updated weights for policy 0, policy_version 54380 (0.0008) [2023-10-13 23:11:47,589][60935] Updated weights for policy 0, policy_version 54390 (0.0010) [2023-10-13 23:11:47,958][60935] Updated weights for policy 0, policy_version 54400 (0.0008) [2023-10-13 23:11:49,740][60934] Updated weights for policy 1, policy_version 54812 (0.0008) [2023-10-13 23:11:50,115][60934] Updated weights for policy 1, policy_version 54822 (0.0008) [2023-10-13 23:11:50,471][60934] Updated weights for policy 1, policy_version 54832 (0.0008) [2023-10-13 23:11:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 112164864. Throughput: 0: 1698.0, 1: 1710.9. Samples: 28047922. Policy #0 lag: (min: 19.0, avg: 21.7, max: 51.0) [2023-10-13 23:11:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-13 23:11:52,001][60935] Updated weights for policy 0, policy_version 54410 (0.0009) [2023-10-13 23:11:52,364][60935] Updated weights for policy 0, policy_version 54420 (0.0012) [2023-10-13 23:11:52,740][60935] Updated weights for policy 0, policy_version 54430 (0.0008) [2023-10-13 23:11:54,497][60934] Updated weights for policy 1, policy_version 54842 (0.0007) [2023-10-13 23:11:54,863][60934] Updated weights for policy 1, policy_version 54852 (0.0010) [2023-10-13 23:11:55,236][60934] Updated weights for policy 1, policy_version 54862 (0.0009) [2023-10-13 23:11:55,603][60934] Updated weights for policy 1, policy_version 54872 (0.0008) [2023-10-13 23:11:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 112230400. Throughput: 0: 1693.0, 1: 1688.1. Samples: 28067566. Policy #0 lag: (min: 19.0, avg: 21.7, max: 51.0) [2023-10-13 23:11:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 23:11:56,729][60935] Updated weights for policy 0, policy_version 54440 (0.0007) [2023-10-13 23:11:57,096][60935] Updated weights for policy 0, policy_version 54450 (0.0008) [2023-10-13 23:11:57,458][60935] Updated weights for policy 0, policy_version 54460 (0.0007) [2023-10-13 23:11:59,733][60934] Updated weights for policy 1, policy_version 54882 (0.0007) [2023-10-13 23:12:00,107][60934] Updated weights for policy 1, policy_version 54892 (0.0010) [2023-10-13 23:12:00,465][60934] Updated weights for policy 1, policy_version 54902 (0.0011) [2023-10-13 23:12:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 112295936. Throughput: 0: 1693.4, 1: 1714.1. Samples: 28078076. Policy #0 lag: (min: 19.0, avg: 21.7, max: 51.0) [2023-10-13 23:12:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 23:12:01,567][60935] Updated weights for policy 0, policy_version 54470 (0.0007) [2023-10-13 23:12:01,933][60935] Updated weights for policy 0, policy_version 54480 (0.0008) [2023-10-13 23:12:02,311][60935] Updated weights for policy 0, policy_version 54490 (0.0009) [2023-10-13 23:12:04,409][60934] Updated weights for policy 1, policy_version 54912 (0.0008) [2023-10-13 23:12:04,772][60934] Updated weights for policy 1, policy_version 54922 (0.0009) [2023-10-13 23:12:05,137][60934] Updated weights for policy 1, policy_version 54932 (0.0007) [2023-10-13 23:12:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 112361472. Throughput: 0: 1707.1, 1: 1692.1. Samples: 28098538. Policy #0 lag: (min: 19.0, avg: 21.7, max: 51.0) [2023-10-13 23:12:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 23:12:06,354][60935] Updated weights for policy 0, policy_version 54500 (0.0007) [2023-10-13 23:12:06,750][60935] Updated weights for policy 0, policy_version 54510 (0.0009) [2023-10-13 23:12:07,118][60935] Updated weights for policy 0, policy_version 54520 (0.0011) [2023-10-13 23:12:09,251][60934] Updated weights for policy 1, policy_version 54942 (0.0008) [2023-10-13 23:12:09,610][60934] Updated weights for policy 1, policy_version 54952 (0.0009) [2023-10-13 23:12:09,981][60934] Updated weights for policy 1, policy_version 54962 (0.0009) [2023-10-13 23:12:11,151][60935] Updated weights for policy 0, policy_version 54530 (0.0010) [2023-10-13 23:12:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 112427008. Throughput: 0: 1703.1, 1: 1673.8. Samples: 28118456. Policy #0 lag: (min: 19.0, avg: 21.7, max: 51.0) [2023-10-13 23:12:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 23:12:11,511][60935] Updated weights for policy 0, policy_version 54540 (0.0007) [2023-10-13 23:12:11,889][60935] Updated weights for policy 0, policy_version 54550 (0.0008) [2023-10-13 23:12:12,250][60935] Updated weights for policy 0, policy_version 54560 (0.0009) [2023-10-13 23:12:13,979][60934] Updated weights for policy 1, policy_version 54972 (0.0008) [2023-10-13 23:12:14,346][60934] Updated weights for policy 1, policy_version 54982 (0.0009) [2023-10-13 23:12:14,712][60934] Updated weights for policy 1, policy_version 54992 (0.0009) [2023-10-13 23:12:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 112492544. Throughput: 0: 1703.9, 1: 1700.3. Samples: 28129006. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-13 23:12:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-13 23:12:16,262][60935] Updated weights for policy 0, policy_version 54570 (0.0008) [2023-10-13 23:12:16,620][60935] Updated weights for policy 0, policy_version 54580 (0.0008) [2023-10-13 23:12:16,988][60935] Updated weights for policy 0, policy_version 54590 (0.0009) [2023-10-13 23:12:18,726][60934] Updated weights for policy 1, policy_version 55002 (0.0008) [2023-10-13 23:12:19,092][60934] Updated weights for policy 1, policy_version 55012 (0.0008) [2023-10-13 23:12:19,459][60934] Updated weights for policy 1, policy_version 55022 (0.0009) [2023-10-13 23:12:19,824][60934] Updated weights for policy 1, policy_version 55032 (0.0007) [2023-10-13 23:12:20,932][60935] Updated weights for policy 0, policy_version 54600 (0.0009) [2023-10-13 23:12:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 112558080. Throughput: 0: 1708.9, 1: 1678.5. Samples: 28149164. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-13 23:12:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-13 23:12:21,304][60935] Updated weights for policy 0, policy_version 54610 (0.0007) [2023-10-13 23:12:21,678][60935] Updated weights for policy 0, policy_version 54620 (0.0008) [2023-10-13 23:12:23,686][60934] Updated weights for policy 1, policy_version 55042 (0.0007) [2023-10-13 23:12:24,055][60934] Updated weights for policy 1, policy_version 55052 (0.0009) [2023-10-13 23:12:24,430][60934] Updated weights for policy 1, policy_version 55062 (0.0010) [2023-10-13 23:12:25,492][60935] Updated weights for policy 0, policy_version 54630 (0.0010) [2023-10-13 23:12:25,862][60935] Updated weights for policy 0, policy_version 54640 (0.0009) [2023-10-13 23:12:26,234][60935] Updated weights for policy 0, policy_version 54650 (0.0011) [2023-10-13 23:12:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 112623616. Throughput: 0: 1694.4, 1: 1688.5. Samples: 28169322. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-13 23:12:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-13 23:12:28,495][60934] Updated weights for policy 1, policy_version 55072 (0.0010) [2023-10-13 23:12:28,863][60934] Updated weights for policy 1, policy_version 55082 (0.0010) [2023-10-13 23:12:29,229][60934] Updated weights for policy 1, policy_version 55092 (0.0009) [2023-10-13 23:12:30,343][60935] Updated weights for policy 0, policy_version 54660 (0.0009) [2023-10-13 23:12:30,717][60935] Updated weights for policy 0, policy_version 54670 (0.0007) [2023-10-13 23:12:31,079][60935] Updated weights for policy 0, policy_version 54680 (0.0007) [2023-10-13 23:12:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 112689152. Throughput: 0: 1707.5, 1: 1694.0. Samples: 28179950. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-13 23:12:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-13 23:12:33,342][60934] Updated weights for policy 1, policy_version 55102 (0.0008) [2023-10-13 23:12:33,709][60934] Updated weights for policy 1, policy_version 55112 (0.0008) [2023-10-13 23:12:34,078][60934] Updated weights for policy 1, policy_version 55122 (0.0009) [2023-10-13 23:12:35,180][60935] Updated weights for policy 0, policy_version 54690 (0.0008) [2023-10-13 23:12:35,554][60935] Updated weights for policy 0, policy_version 54700 (0.0007) [2023-10-13 23:12:35,921][60935] Updated weights for policy 0, policy_version 54710 (0.0008) [2023-10-13 23:12:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 112754688. Throughput: 0: 1698.9, 1: 1676.0. Samples: 28199794. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-13 23:12:36,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:12:36,291][60935] Updated weights for policy 0, policy_version 54720 (0.0007) [2023-10-13 23:12:38,370][60934] Updated weights for policy 1, policy_version 55132 (0.0009) [2023-10-13 23:12:38,735][60934] Updated weights for policy 1, policy_version 55142 (0.0009) [2023-10-13 23:12:39,097][60934] Updated weights for policy 1, policy_version 55152 (0.0007) [2023-10-13 23:12:40,329][60935] Updated weights for policy 0, policy_version 54730 (0.0007) [2023-10-13 23:12:40,688][60935] Updated weights for policy 0, policy_version 54740 (0.0007) [2023-10-13 23:12:41,057][60935] Updated weights for policy 0, policy_version 54750 (0.0011) [2023-10-13 23:12:41,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 112852992. Throughput: 0: 1683.8, 1: 1700.9. Samples: 28219876. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-13 23:12:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:12:43,128][60934] Updated weights for policy 1, policy_version 55162 (0.0009) [2023-10-13 23:12:43,497][60934] Updated weights for policy 1, policy_version 55172 (0.0007) [2023-10-13 23:12:43,867][60934] Updated weights for policy 1, policy_version 55182 (0.0011) [2023-10-13 23:12:44,233][60934] Updated weights for policy 1, policy_version 55192 (0.0011) [2023-10-13 23:12:45,288][60935] Updated weights for policy 0, policy_version 54760 (0.0011) [2023-10-13 23:12:45,651][60935] Updated weights for policy 0, policy_version 54770 (0.0009) [2023-10-13 23:12:46,023][60935] Updated weights for policy 0, policy_version 54780 (0.0007) [2023-10-13 23:12:46,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 112918528. Throughput: 0: 1698.0, 1: 1690.8. Samples: 28230570. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-13 23:12:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:12:48,327][60934] Updated weights for policy 1, policy_version 55202 (0.0010) [2023-10-13 23:12:48,696][60934] Updated weights for policy 1, policy_version 55212 (0.0007) [2023-10-13 23:12:49,054][60934] Updated weights for policy 1, policy_version 55222 (0.0007) [2023-10-13 23:12:50,009][60935] Updated weights for policy 0, policy_version 54790 (0.0009) [2023-10-13 23:12:50,384][60935] Updated weights for policy 0, policy_version 54800 (0.0007) [2023-10-13 23:12:50,751][60935] Updated weights for policy 0, policy_version 54810 (0.0009) [2023-10-13 23:12:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 112984064. Throughput: 0: 1690.3, 1: 1688.0. Samples: 28250564. Policy #0 lag: (min: 29.0, avg: 30.5, max: 55.0) [2023-10-13 23:12:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:12:53,104][60934] Updated weights for policy 1, policy_version 55232 (0.0007) [2023-10-13 23:12:53,471][60934] Updated weights for policy 1, policy_version 55242 (0.0007) [2023-10-13 23:12:53,843][60934] Updated weights for policy 1, policy_version 55252 (0.0007) [2023-10-13 23:12:54,877][60935] Updated weights for policy 0, policy_version 54820 (0.0008) [2023-10-13 23:12:55,269][60935] Updated weights for policy 0, policy_version 54830 (0.0009) [2023-10-13 23:12:55,637][60935] Updated weights for policy 0, policy_version 54840 (0.0008) [2023-10-13 23:12:56,249][59943] Fps is (10 sec: 13106.5, 60 sec: 13653.2, 300 sec: 13551.5). Total num frames: 113049600. Throughput: 0: 1667.0, 1: 1705.6. Samples: 28270226. Policy #0 lag: (min: 29.0, avg: 30.5, max: 55.0) [2023-10-13 23:12:56,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:12:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000055256_56885248.pth... [2023-10-13 23:12:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000054848_56164352.pth... [2023-10-13 23:12:56,298][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000053248_54525952.pth [2023-10-13 23:12:56,302][60695] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p0/milestones/checkpoint_000054848_56164352.pth [2023-10-13 23:12:56,306][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000053656_55246848.pth [2023-10-13 23:12:56,312][60828] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p1/milestones/checkpoint_000055256_56885248.pth [2023-10-13 23:12:57,820][60934] Updated weights for policy 1, policy_version 55262 (0.0007) [2023-10-13 23:12:58,176][60934] Updated weights for policy 1, policy_version 55272 (0.0008) [2023-10-13 23:12:58,535][60934] Updated weights for policy 1, policy_version 55282 (0.0011) [2023-10-13 23:12:59,747][60935] Updated weights for policy 0, policy_version 54850 (0.0009) [2023-10-13 23:13:00,114][60935] Updated weights for policy 0, policy_version 54860 (0.0010) [2023-10-13 23:13:00,485][60935] Updated weights for policy 0, policy_version 54870 (0.0009) [2023-10-13 23:13:00,857][60935] Updated weights for policy 0, policy_version 54880 (0.0010) [2023-10-13 23:13:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 113115136. Throughput: 0: 1689.6, 1: 1680.1. Samples: 28280642. Policy #0 lag: (min: 29.0, avg: 30.5, max: 55.0) [2023-10-13 23:13:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:13:02,637][60934] Updated weights for policy 1, policy_version 55292 (0.0008) [2023-10-13 23:13:03,005][60934] Updated weights for policy 1, policy_version 55302 (0.0007) [2023-10-13 23:13:03,364][60934] Updated weights for policy 1, policy_version 55312 (0.0007) [2023-10-13 23:13:04,807][60935] Updated weights for policy 0, policy_version 54890 (0.0009) [2023-10-13 23:13:05,187][60935] Updated weights for policy 0, policy_version 54900 (0.0009) [2023-10-13 23:13:05,551][60935] Updated weights for policy 0, policy_version 54910 (0.0009) [2023-10-13 23:13:06,248][59943] Fps is (10 sec: 13107.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 113180672. Throughput: 0: 1676.8, 1: 1696.0. Samples: 28300938. Policy #0 lag: (min: 29.0, avg: 30.5, max: 55.0) [2023-10-13 23:13:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:13:07,328][60934] Updated weights for policy 1, policy_version 55322 (0.0007) [2023-10-13 23:13:07,687][60934] Updated weights for policy 1, policy_version 55332 (0.0007) [2023-10-13 23:13:08,050][60934] Updated weights for policy 1, policy_version 55342 (0.0008) [2023-10-13 23:13:08,420][60934] Updated weights for policy 1, policy_version 55352 (0.0009) [2023-10-13 23:13:09,519][60935] Updated weights for policy 0, policy_version 54920 (0.0008) [2023-10-13 23:13:09,876][60935] Updated weights for policy 0, policy_version 54930 (0.0009) [2023-10-13 23:13:10,245][60935] Updated weights for policy 0, policy_version 54940 (0.0009) [2023-10-13 23:13:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 113246208. Throughput: 0: 1669.1, 1: 1700.7. Samples: 28320962. Policy #0 lag: (min: 29.0, avg: 30.5, max: 55.0) [2023-10-13 23:13:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:13:12,453][60934] Updated weights for policy 1, policy_version 55362 (0.0011) [2023-10-13 23:13:12,819][60934] Updated weights for policy 1, policy_version 55372 (0.0008) [2023-10-13 23:13:13,188][60934] Updated weights for policy 1, policy_version 55382 (0.0008) [2023-10-13 23:13:14,434][60935] Updated weights for policy 0, policy_version 54950 (0.0010) [2023-10-13 23:13:14,794][60935] Updated weights for policy 0, policy_version 54960 (0.0008) [2023-10-13 23:13:15,160][60935] Updated weights for policy 0, policy_version 54970 (0.0008) [2023-10-13 23:13:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 113311744. Throughput: 0: 1684.5, 1: 1679.5. Samples: 28331332. Policy #0 lag: (min: 29.0, avg: 30.5, max: 55.0) [2023-10-13 23:13:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:13:17,233][60934] Updated weights for policy 1, policy_version 55392 (0.0009) [2023-10-13 23:13:17,607][60934] Updated weights for policy 1, policy_version 55402 (0.0010) [2023-10-13 23:13:17,971][60934] Updated weights for policy 1, policy_version 55412 (0.0007) [2023-10-13 23:13:19,056][60935] Updated weights for policy 0, policy_version 54980 (0.0007) [2023-10-13 23:13:19,420][60935] Updated weights for policy 0, policy_version 54990 (0.0008) [2023-10-13 23:13:19,791][60935] Updated weights for policy 0, policy_version 55000 (0.0009) [2023-10-13 23:13:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 113377280. Throughput: 0: 1671.2, 1: 1698.6. Samples: 28351434. Policy #0 lag: (min: 29.0, avg: 30.5, max: 55.0) [2023-10-13 23:13:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:13:22,030][60934] Updated weights for policy 1, policy_version 55422 (0.0009) [2023-10-13 23:13:22,407][60934] Updated weights for policy 1, policy_version 55432 (0.0008) [2023-10-13 23:13:22,784][60934] Updated weights for policy 1, policy_version 55442 (0.0008) [2023-10-13 23:13:23,908][60935] Updated weights for policy 0, policy_version 55010 (0.0008) [2023-10-13 23:13:24,270][60935] Updated weights for policy 0, policy_version 55020 (0.0009) [2023-10-13 23:13:24,647][60935] Updated weights for policy 0, policy_version 55030 (0.0007) [2023-10-13 23:13:25,017][60935] Updated weights for policy 0, policy_version 55040 (0.0008) [2023-10-13 23:13:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 113442816. Throughput: 0: 1682.6, 1: 1697.5. Samples: 28371982. Policy #0 lag: (min: 29.0, avg: 30.5, max: 55.0) [2023-10-13 23:13:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:13:26,709][60934] Updated weights for policy 1, policy_version 55452 (0.0007) [2023-10-13 23:13:27,083][60934] Updated weights for policy 1, policy_version 55462 (0.0008) [2023-10-13 23:13:27,449][60934] Updated weights for policy 1, policy_version 55472 (0.0010) [2023-10-13 23:13:29,057][60935] Updated weights for policy 0, policy_version 55050 (0.0009) [2023-10-13 23:13:29,430][60935] Updated weights for policy 0, policy_version 55060 (0.0009) [2023-10-13 23:13:29,794][60935] Updated weights for policy 0, policy_version 55070 (0.0009) [2023-10-13 23:13:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 113508352. Throughput: 0: 1691.6, 1: 1679.1. Samples: 28382248. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 23:13:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:13:31,452][60934] Updated weights for policy 1, policy_version 55482 (0.0009) [2023-10-13 23:13:31,818][60934] Updated weights for policy 1, policy_version 55492 (0.0008) [2023-10-13 23:13:32,192][60934] Updated weights for policy 1, policy_version 55502 (0.0008) [2023-10-13 23:13:32,558][60934] Updated weights for policy 1, policy_version 55512 (0.0010) [2023-10-13 23:13:33,860][60935] Updated weights for policy 0, policy_version 55080 (0.0008) [2023-10-13 23:13:34,232][60935] Updated weights for policy 0, policy_version 55090 (0.0010) [2023-10-13 23:13:34,609][60935] Updated weights for policy 0, policy_version 55100 (0.0009) [2023-10-13 23:13:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 113573888. Throughput: 0: 1668.4, 1: 1700.9. Samples: 28402184. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 23:13:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:13:36,573][60934] Updated weights for policy 1, policy_version 55522 (0.0007) [2023-10-13 23:13:36,945][60934] Updated weights for policy 1, policy_version 55532 (0.0011) [2023-10-13 23:13:37,318][60934] Updated weights for policy 1, policy_version 55542 (0.0008) [2023-10-13 23:13:38,497][60935] Updated weights for policy 0, policy_version 55110 (0.0007) [2023-10-13 23:13:38,867][60935] Updated weights for policy 0, policy_version 55120 (0.0008) [2023-10-13 23:13:39,237][60935] Updated weights for policy 0, policy_version 55130 (0.0011) [2023-10-13 23:13:41,244][60934] Updated weights for policy 1, policy_version 55552 (0.0007) [2023-10-13 23:13:41,249][59943] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13551.5). Total num frames: 113639424. Throughput: 0: 1693.8, 1: 1696.3. Samples: 28422778. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 23:13:41,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:13:41,606][60934] Updated weights for policy 1, policy_version 55562 (0.0008) [2023-10-13 23:13:41,974][60934] Updated weights for policy 1, policy_version 55572 (0.0008) [2023-10-13 23:13:43,365][60935] Updated weights for policy 0, policy_version 55140 (0.0010) [2023-10-13 23:13:43,738][60935] Updated weights for policy 0, policy_version 55150 (0.0011) [2023-10-13 23:13:44,107][60935] Updated weights for policy 0, policy_version 55160 (0.0010) [2023-10-13 23:13:45,986][60934] Updated weights for policy 1, policy_version 55582 (0.0009) [2023-10-13 23:13:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 113704960. Throughput: 0: 1682.5, 1: 1689.3. Samples: 28432374. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 23:13:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:13:46,366][60934] Updated weights for policy 1, policy_version 55592 (0.0011) [2023-10-13 23:13:46,728][60934] Updated weights for policy 1, policy_version 55602 (0.0009) [2023-10-13 23:13:48,223][60935] Updated weights for policy 0, policy_version 55170 (0.0009) [2023-10-13 23:13:48,588][60935] Updated weights for policy 0, policy_version 55180 (0.0011) [2023-10-13 23:13:48,957][60935] Updated weights for policy 0, policy_version 55190 (0.0012) [2023-10-13 23:13:49,336][60935] Updated weights for policy 0, policy_version 55200 (0.0010) [2023-10-13 23:13:50,822][60934] Updated weights for policy 1, policy_version 55612 (0.0009) [2023-10-13 23:13:51,190][60934] Updated weights for policy 1, policy_version 55622 (0.0007) [2023-10-13 23:13:51,248][59943] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 113770496. Throughput: 0: 1672.2, 1: 1694.3. Samples: 28452428. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 23:13:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:13:51,558][60934] Updated weights for policy 1, policy_version 55632 (0.0008) [2023-10-13 23:13:53,346][60935] Updated weights for policy 0, policy_version 55210 (0.0010) [2023-10-13 23:13:53,705][60935] Updated weights for policy 0, policy_version 55220 (0.0009) [2023-10-13 23:13:54,076][60935] Updated weights for policy 0, policy_version 55230 (0.0008) [2023-10-13 23:13:55,489][60934] Updated weights for policy 1, policy_version 55642 (0.0008) [2023-10-13 23:13:55,865][60934] Updated weights for policy 1, policy_version 55652 (0.0009) [2023-10-13 23:13:56,226][60934] Updated weights for policy 1, policy_version 55662 (0.0008) [2023-10-13 23:13:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 113836032. Throughput: 0: 1690.4, 1: 1694.6. Samples: 28473288. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 23:13:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:13:56,590][60934] Updated weights for policy 1, policy_version 55672 (0.0008) [2023-10-13 23:13:58,104][60935] Updated weights for policy 0, policy_version 55240 (0.0007) [2023-10-13 23:13:58,478][60935] Updated weights for policy 0, policy_version 55250 (0.0008) [2023-10-13 23:13:58,858][60935] Updated weights for policy 0, policy_version 55260 (0.0007) [2023-10-13 23:14:00,739][60934] Updated weights for policy 1, policy_version 55682 (0.0008) [2023-10-13 23:14:01,102][60934] Updated weights for policy 1, policy_version 55692 (0.0007) [2023-10-13 23:14:01,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 113901568. Throughput: 0: 1670.5, 1: 1695.2. Samples: 28482790. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 23:14:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:14:01,468][60934] Updated weights for policy 1, policy_version 55702 (0.0008) [2023-10-13 23:14:02,924][60935] Updated weights for policy 0, policy_version 55270 (0.0007) [2023-10-13 23:14:03,304][60935] Updated weights for policy 0, policy_version 55280 (0.0008) [2023-10-13 23:14:03,677][60935] Updated weights for policy 0, policy_version 55290 (0.0011) [2023-10-13 23:14:05,417][60934] Updated weights for policy 1, policy_version 55712 (0.0009) [2023-10-13 23:14:05,784][60934] Updated weights for policy 1, policy_version 55722 (0.0010) [2023-10-13 23:14:06,146][60934] Updated weights for policy 1, policy_version 55732 (0.0011) [2023-10-13 23:14:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 113967104. Throughput: 0: 1684.1, 1: 1699.0. Samples: 28503674. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-13 23:14:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:14:07,613][60935] Updated weights for policy 0, policy_version 55300 (0.0009) [2023-10-13 23:14:07,983][60935] Updated weights for policy 0, policy_version 55310 (0.0011) [2023-10-13 23:14:08,358][60935] Updated weights for policy 0, policy_version 55320 (0.0011) [2023-10-13 23:14:10,211][60934] Updated weights for policy 1, policy_version 55742 (0.0008) [2023-10-13 23:14:10,589][60934] Updated weights for policy 1, policy_version 55752 (0.0010) [2023-10-13 23:14:10,950][60934] Updated weights for policy 1, policy_version 55762 (0.0009) [2023-10-13 23:14:11,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 114065408. Throughput: 0: 1691.7, 1: 1689.0. Samples: 28524114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:14:11,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:14:12,367][60935] Updated weights for policy 0, policy_version 55330 (0.0009) [2023-10-13 23:14:12,738][60935] Updated weights for policy 0, policy_version 55340 (0.0009) [2023-10-13 23:14:13,111][60935] Updated weights for policy 0, policy_version 55350 (0.0008) [2023-10-13 23:14:13,480][60935] Updated weights for policy 0, policy_version 55360 (0.0007) [2023-10-13 23:14:14,828][60934] Updated weights for policy 1, policy_version 55772 (0.0008) [2023-10-13 23:14:15,202][60934] Updated weights for policy 1, policy_version 55782 (0.0009) [2023-10-13 23:14:15,574][60934] Updated weights for policy 1, policy_version 55792 (0.0008) [2023-10-13 23:14:16,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 114130944. Throughput: 0: 1666.0, 1: 1708.4. Samples: 28534096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:14:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:14:17,629][60935] Updated weights for policy 0, policy_version 55370 (0.0008) [2023-10-13 23:14:17,985][60935] Updated weights for policy 0, policy_version 55380 (0.0012) [2023-10-13 23:14:18,357][60935] Updated weights for policy 0, policy_version 55390 (0.0011) [2023-10-13 23:14:19,583][60934] Updated weights for policy 1, policy_version 55802 (0.0008) [2023-10-13 23:14:19,941][60934] Updated weights for policy 1, policy_version 55812 (0.0009) [2023-10-13 23:14:20,316][60934] Updated weights for policy 1, policy_version 55822 (0.0011) [2023-10-13 23:14:20,680][60934] Updated weights for policy 1, policy_version 55832 (0.0011) [2023-10-13 23:14:21,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 114196480. Throughput: 0: 1689.6, 1: 1708.8. Samples: 28555114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:14:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:14:22,516][60935] Updated weights for policy 0, policy_version 55400 (0.0010) [2023-10-13 23:14:22,879][60935] Updated weights for policy 0, policy_version 55410 (0.0008) [2023-10-13 23:14:23,254][60935] Updated weights for policy 0, policy_version 55420 (0.0009) [2023-10-13 23:14:24,698][60934] Updated weights for policy 1, policy_version 55842 (0.0008) [2023-10-13 23:14:25,059][60934] Updated weights for policy 1, policy_version 55852 (0.0007) [2023-10-13 23:14:25,429][60934] Updated weights for policy 1, policy_version 55862 (0.0007) [2023-10-13 23:14:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 114262016. Throughput: 0: 1696.1, 1: 1688.2. Samples: 28575070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:14:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:14:27,123][60935] Updated weights for policy 0, policy_version 55430 (0.0009) [2023-10-13 23:14:27,491][60935] Updated weights for policy 0, policy_version 55440 (0.0011) [2023-10-13 23:14:27,855][60935] Updated weights for policy 0, policy_version 55450 (0.0011) [2023-10-13 23:14:29,337][60934] Updated weights for policy 1, policy_version 55872 (0.0009) [2023-10-13 23:14:29,711][60934] Updated weights for policy 1, policy_version 55882 (0.0009) [2023-10-13 23:14:30,080][60934] Updated weights for policy 1, policy_version 55892 (0.0009) [2023-10-13 23:14:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 114327552. Throughput: 0: 1684.9, 1: 1722.7. Samples: 28585716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:14:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:14:31,873][60935] Updated weights for policy 0, policy_version 55460 (0.0008) [2023-10-13 23:14:32,235][60935] Updated weights for policy 0, policy_version 55470 (0.0010) [2023-10-13 23:14:32,599][60935] Updated weights for policy 0, policy_version 55480 (0.0009) [2023-10-13 23:14:34,114][60934] Updated weights for policy 1, policy_version 55902 (0.0010) [2023-10-13 23:14:34,474][60934] Updated weights for policy 1, policy_version 55912 (0.0008) [2023-10-13 23:14:34,846][60934] Updated weights for policy 1, policy_version 55922 (0.0007) [2023-10-13 23:14:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 114393088. Throughput: 0: 1707.0, 1: 1706.1. Samples: 28606018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:14:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:14:36,738][60935] Updated weights for policy 0, policy_version 55490 (0.0011) [2023-10-13 23:14:37,103][60935] Updated weights for policy 0, policy_version 55500 (0.0007) [2023-10-13 23:14:37,474][60935] Updated weights for policy 0, policy_version 55510 (0.0008) [2023-10-13 23:14:37,835][60935] Updated weights for policy 0, policy_version 55520 (0.0007) [2023-10-13 23:14:38,818][60934] Updated weights for policy 1, policy_version 55932 (0.0008) [2023-10-13 23:14:39,185][60934] Updated weights for policy 1, policy_version 55942 (0.0008) [2023-10-13 23:14:39,537][60934] Updated weights for policy 1, policy_version 55952 (0.0008) [2023-10-13 23:14:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 114458624. Throughput: 0: 1705.6, 1: 1690.9. Samples: 28626132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:14:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:14:41,830][60935] Updated weights for policy 0, policy_version 55530 (0.0009) [2023-10-13 23:14:42,198][60935] Updated weights for policy 0, policy_version 55540 (0.0007) [2023-10-13 23:14:42,567][60935] Updated weights for policy 0, policy_version 55550 (0.0007) [2023-10-13 23:14:43,683][60934] Updated weights for policy 1, policy_version 55962 (0.0008) [2023-10-13 23:14:44,057][60934] Updated weights for policy 1, policy_version 55972 (0.0009) [2023-10-13 23:14:44,419][60934] Updated weights for policy 1, policy_version 55982 (0.0011) [2023-10-13 23:14:44,782][60934] Updated weights for policy 1, policy_version 55992 (0.0009) [2023-10-13 23:14:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 114524160. Throughput: 0: 1698.2, 1: 1720.3. Samples: 28636624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:14:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:14:46,573][60935] Updated weights for policy 0, policy_version 55560 (0.0007) [2023-10-13 23:14:46,943][60935] Updated weights for policy 0, policy_version 55570 (0.0010) [2023-10-13 23:14:47,327][60935] Updated weights for policy 0, policy_version 55580 (0.0007) [2023-10-13 23:14:48,794][60934] Updated weights for policy 1, policy_version 56002 (0.0008) [2023-10-13 23:14:49,158][60934] Updated weights for policy 1, policy_version 56012 (0.0008) [2023-10-13 23:14:49,529][60934] Updated weights for policy 1, policy_version 56022 (0.0008) [2023-10-13 23:14:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 114589696. Throughput: 0: 1700.7, 1: 1693.0. Samples: 28656392. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:14:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.430')] [2023-10-13 23:14:51,318][60935] Updated weights for policy 0, policy_version 55590 (0.0009) [2023-10-13 23:14:51,691][60935] Updated weights for policy 0, policy_version 55600 (0.0009) [2023-10-13 23:14:52,059][60935] Updated weights for policy 0, policy_version 55610 (0.0008) [2023-10-13 23:14:53,382][60934] Updated weights for policy 1, policy_version 56032 (0.0008) [2023-10-13 23:14:53,755][60934] Updated weights for policy 1, policy_version 56042 (0.0007) [2023-10-13 23:14:54,125][60934] Updated weights for policy 1, policy_version 56052 (0.0009) [2023-10-13 23:14:56,088][60935] Updated weights for policy 0, policy_version 55620 (0.0011) [2023-10-13 23:14:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 114655232. Throughput: 0: 1704.7, 1: 1708.0. Samples: 28677686. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:14:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.430')] [2023-10-13 23:14:56,259][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000056056_57704448.pth... [2023-10-13 23:14:56,299][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000054456_56066048.pth [2023-10-13 23:14:56,454][60935] Updated weights for policy 0, policy_version 55630 (0.0007) [2023-10-13 23:14:56,818][60935] Updated weights for policy 0, policy_version 55640 (0.0010) [2023-10-13 23:14:57,123][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000055648_56983552.pth... [2023-10-13 23:14:57,164][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000054048_55345152.pth [2023-10-13 23:14:58,032][60934] Updated weights for policy 1, policy_version 56062 (0.0007) [2023-10-13 23:14:58,400][60934] Updated weights for policy 1, policy_version 56072 (0.0007) [2023-10-13 23:14:58,778][60934] Updated weights for policy 1, policy_version 56082 (0.0008) [2023-10-13 23:15:00,669][60935] Updated weights for policy 0, policy_version 55650 (0.0009) [2023-10-13 23:15:01,044][60935] Updated weights for policy 0, policy_version 55660 (0.0008) [2023-10-13 23:15:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 114720768. Throughput: 0: 1706.4, 1: 1705.4. Samples: 28687628. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:15:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.430')] [2023-10-13 23:15:01,404][60935] Updated weights for policy 0, policy_version 55670 (0.0011) [2023-10-13 23:15:01,774][60935] Updated weights for policy 0, policy_version 55680 (0.0008) [2023-10-13 23:15:02,688][60934] Updated weights for policy 1, policy_version 56092 (0.0008) [2023-10-13 23:15:03,043][60934] Updated weights for policy 1, policy_version 56102 (0.0009) [2023-10-13 23:15:03,415][60934] Updated weights for policy 1, policy_version 56112 (0.0007) [2023-10-13 23:15:05,795][60935] Updated weights for policy 0, policy_version 55690 (0.0009) [2023-10-13 23:15:06,153][60935] Updated weights for policy 0, policy_version 55700 (0.0008) [2023-10-13 23:15:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 114786304. Throughput: 0: 1713.7, 1: 1690.8. Samples: 28708318. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:15:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.430')] [2023-10-13 23:15:06,534][60935] Updated weights for policy 0, policy_version 55710 (0.0009) [2023-10-13 23:15:07,458][60934] Updated weights for policy 1, policy_version 56122 (0.0008) [2023-10-13 23:15:07,823][60934] Updated weights for policy 1, policy_version 56132 (0.0009) [2023-10-13 23:15:08,196][60934] Updated weights for policy 1, policy_version 56142 (0.0008) [2023-10-13 23:15:08,556][60934] Updated weights for policy 1, policy_version 56152 (0.0009) [2023-10-13 23:15:10,523][60935] Updated weights for policy 0, policy_version 55720 (0.0008) [2023-10-13 23:15:10,896][60935] Updated weights for policy 0, policy_version 55730 (0.0008) [2023-10-13 23:15:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 114851840. Throughput: 0: 1696.6, 1: 1715.9. Samples: 28728630. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:15:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.430')] [2023-10-13 23:15:11,268][60935] Updated weights for policy 0, policy_version 55740 (0.0007) [2023-10-13 23:15:12,761][60934] Updated weights for policy 1, policy_version 56162 (0.0008) [2023-10-13 23:15:13,140][60934] Updated weights for policy 1, policy_version 56172 (0.0007) [2023-10-13 23:15:13,521][60934] Updated weights for policy 1, policy_version 56182 (0.0007) [2023-10-13 23:15:15,220][60935] Updated weights for policy 0, policy_version 55750 (0.0008) [2023-10-13 23:15:15,595][60935] Updated weights for policy 0, policy_version 55760 (0.0010) [2023-10-13 23:15:15,969][60935] Updated weights for policy 0, policy_version 55770 (0.0007) [2023-10-13 23:15:16,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 114950144. Throughput: 0: 1714.6, 1: 1683.1. Samples: 28738612. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:15:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.430')] [2023-10-13 23:15:17,386][60934] Updated weights for policy 1, policy_version 56192 (0.0009) [2023-10-13 23:15:17,746][60934] Updated weights for policy 1, policy_version 56202 (0.0010) [2023-10-13 23:15:18,126][60934] Updated weights for policy 1, policy_version 56212 (0.0009) [2023-10-13 23:15:19,925][60935] Updated weights for policy 0, policy_version 55780 (0.0008) [2023-10-13 23:15:20,316][60935] Updated weights for policy 0, policy_version 55790 (0.0008) [2023-10-13 23:15:20,678][60935] Updated weights for policy 0, policy_version 55800 (0.0007) [2023-10-13 23:15:21,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 115015680. Throughput: 0: 1714.0, 1: 1694.8. Samples: 28759412. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:15:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.430')] [2023-10-13 23:15:22,180][60934] Updated weights for policy 1, policy_version 56222 (0.0008) [2023-10-13 23:15:22,559][60934] Updated weights for policy 1, policy_version 56232 (0.0008) [2023-10-13 23:15:22,935][60934] Updated weights for policy 1, policy_version 56242 (0.0009) [2023-10-13 23:15:24,528][60935] Updated weights for policy 0, policy_version 55810 (0.0008) [2023-10-13 23:15:24,895][60935] Updated weights for policy 0, policy_version 55820 (0.0011) [2023-10-13 23:15:25,270][60935] Updated weights for policy 0, policy_version 55830 (0.0011) [2023-10-13 23:15:25,639][60935] Updated weights for policy 0, policy_version 55840 (0.0011) [2023-10-13 23:15:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 115081216. Throughput: 0: 1695.0, 1: 1716.8. Samples: 28779662. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 23:15:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.430')] [2023-10-13 23:15:26,959][60934] Updated weights for policy 1, policy_version 56252 (0.0007) [2023-10-13 23:15:27,324][60934] Updated weights for policy 1, policy_version 56262 (0.0008) [2023-10-13 23:15:27,678][60934] Updated weights for policy 1, policy_version 56272 (0.0010) [2023-10-13 23:15:29,660][60935] Updated weights for policy 0, policy_version 55850 (0.0007) [2023-10-13 23:15:30,030][60935] Updated weights for policy 0, policy_version 55860 (0.0007) [2023-10-13 23:15:30,400][60935] Updated weights for policy 0, policy_version 55870 (0.0010) [2023-10-13 23:15:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 115146752. Throughput: 0: 1726.9, 1: 1682.7. Samples: 28790056. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 23:15:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:15:31,717][60934] Updated weights for policy 1, policy_version 56282 (0.0009) [2023-10-13 23:15:32,079][60934] Updated weights for policy 1, policy_version 56292 (0.0007) [2023-10-13 23:15:32,442][60934] Updated weights for policy 1, policy_version 56302 (0.0007) [2023-10-13 23:15:32,807][60934] Updated weights for policy 1, policy_version 56312 (0.0009) [2023-10-13 23:15:34,518][60935] Updated weights for policy 0, policy_version 55880 (0.0008) [2023-10-13 23:15:34,891][60935] Updated weights for policy 0, policy_version 55890 (0.0008) [2023-10-13 23:15:35,262][60935] Updated weights for policy 0, policy_version 55900 (0.0010) [2023-10-13 23:15:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 115212288. Throughput: 0: 1717.5, 1: 1708.5. Samples: 28810562. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 23:15:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:15:36,843][60934] Updated weights for policy 1, policy_version 56322 (0.0008) [2023-10-13 23:15:37,200][60934] Updated weights for policy 1, policy_version 56332 (0.0007) [2023-10-13 23:15:37,572][60934] Updated weights for policy 1, policy_version 56342 (0.0007) [2023-10-13 23:15:39,215][60935] Updated weights for policy 0, policy_version 55910 (0.0008) [2023-10-13 23:15:39,581][60935] Updated weights for policy 0, policy_version 55920 (0.0008) [2023-10-13 23:15:39,949][60935] Updated weights for policy 0, policy_version 55930 (0.0007) [2023-10-13 23:15:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 115277824. Throughput: 0: 1698.4, 1: 1706.5. Samples: 28830910. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 23:15:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:15:41,638][60934] Updated weights for policy 1, policy_version 56352 (0.0007) [2023-10-13 23:15:41,996][60934] Updated weights for policy 1, policy_version 56362 (0.0010) [2023-10-13 23:15:42,370][60934] Updated weights for policy 1, policy_version 56372 (0.0008) [2023-10-13 23:15:43,834][60935] Updated weights for policy 0, policy_version 55940 (0.0008) [2023-10-13 23:15:44,199][60935] Updated weights for policy 0, policy_version 55950 (0.0008) [2023-10-13 23:15:44,565][60935] Updated weights for policy 0, policy_version 55960 (0.0009) [2023-10-13 23:15:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 115343360. Throughput: 0: 1723.6, 1: 1690.9. Samples: 28841284. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 23:15:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:15:46,265][60934] Updated weights for policy 1, policy_version 56382 (0.0008) [2023-10-13 23:15:46,630][60934] Updated weights for policy 1, policy_version 56392 (0.0008) [2023-10-13 23:15:46,991][60934] Updated weights for policy 1, policy_version 56402 (0.0008) [2023-10-13 23:15:48,629][60935] Updated weights for policy 0, policy_version 55970 (0.0009) [2023-10-13 23:15:48,991][60935] Updated weights for policy 0, policy_version 55980 (0.0007) [2023-10-13 23:15:49,361][60935] Updated weights for policy 0, policy_version 55990 (0.0008) [2023-10-13 23:15:49,726][60935] Updated weights for policy 0, policy_version 56000 (0.0007) [2023-10-13 23:15:51,114][60934] Updated weights for policy 1, policy_version 56412 (0.0009) [2023-10-13 23:15:51,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 115408896. Throughput: 0: 1691.7, 1: 1704.9. Samples: 28861164. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 23:15:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:15:51,483][60934] Updated weights for policy 1, policy_version 56422 (0.0007) [2023-10-13 23:15:51,839][60934] Updated weights for policy 1, policy_version 56432 (0.0009) [2023-10-13 23:15:53,677][60935] Updated weights for policy 0, policy_version 56010 (0.0008) [2023-10-13 23:15:54,052][60935] Updated weights for policy 0, policy_version 56020 (0.0008) [2023-10-13 23:15:54,432][60935] Updated weights for policy 0, policy_version 56030 (0.0009) [2023-10-13 23:15:55,811][60934] Updated weights for policy 1, policy_version 56442 (0.0008) [2023-10-13 23:15:56,168][60934] Updated weights for policy 1, policy_version 56452 (0.0007) [2023-10-13 23:15:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 115474432. Throughput: 0: 1714.0, 1: 1705.5. Samples: 28882506. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 23:15:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:15:56,530][60934] Updated weights for policy 1, policy_version 56462 (0.0011) [2023-10-13 23:15:56,896][60934] Updated weights for policy 1, policy_version 56472 (0.0009) [2023-10-13 23:15:58,324][60935] Updated weights for policy 0, policy_version 56040 (0.0009) [2023-10-13 23:15:58,693][60935] Updated weights for policy 0, policy_version 56050 (0.0009) [2023-10-13 23:15:59,059][60935] Updated weights for policy 0, policy_version 56060 (0.0008) [2023-10-13 23:16:01,063][60934] Updated weights for policy 1, policy_version 56482 (0.0009) [2023-10-13 23:16:01,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 115539968. Throughput: 0: 1704.5, 1: 1702.5. Samples: 28891930. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-13 23:16:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:16:01,439][60934] Updated weights for policy 1, policy_version 56492 (0.0010) [2023-10-13 23:16:01,806][60934] Updated weights for policy 1, policy_version 56502 (0.0011) [2023-10-13 23:16:03,130][60935] Updated weights for policy 0, policy_version 56070 (0.0009) [2023-10-13 23:16:03,505][60935] Updated weights for policy 0, policy_version 56080 (0.0007) [2023-10-13 23:16:03,864][60935] Updated weights for policy 0, policy_version 56090 (0.0007) [2023-10-13 23:16:05,806][60934] Updated weights for policy 1, policy_version 56512 (0.0009) [2023-10-13 23:16:06,168][60934] Updated weights for policy 1, policy_version 56522 (0.0007) [2023-10-13 23:16:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 115605504. Throughput: 0: 1693.8, 1: 1708.7. Samples: 28912526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:16:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:16:06,540][60934] Updated weights for policy 1, policy_version 56532 (0.0007) [2023-10-13 23:16:07,821][60935] Updated weights for policy 0, policy_version 56100 (0.0007) [2023-10-13 23:16:08,205][60935] Updated weights for policy 0, policy_version 56110 (0.0009) [2023-10-13 23:16:08,584][60935] Updated weights for policy 0, policy_version 56120 (0.0007) [2023-10-13 23:16:10,600][60934] Updated weights for policy 1, policy_version 56542 (0.0008) [2023-10-13 23:16:10,964][60934] Updated weights for policy 1, policy_version 56552 (0.0008) [2023-10-13 23:16:11,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 115671040. Throughput: 0: 1713.6, 1: 1697.8. Samples: 28933176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:16:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:16:11,330][60934] Updated weights for policy 1, policy_version 56562 (0.0008) [2023-10-13 23:16:12,730][60935] Updated weights for policy 0, policy_version 56130 (0.0008) [2023-10-13 23:16:13,115][60935] Updated weights for policy 0, policy_version 56140 (0.0009) [2023-10-13 23:16:13,475][60935] Updated weights for policy 0, policy_version 56150 (0.0008) [2023-10-13 23:16:13,849][60935] Updated weights for policy 0, policy_version 56160 (0.0010) [2023-10-13 23:16:15,173][60934] Updated weights for policy 1, policy_version 56572 (0.0008) [2023-10-13 23:16:15,545][60934] Updated weights for policy 1, policy_version 56582 (0.0010) [2023-10-13 23:16:15,905][60934] Updated weights for policy 1, policy_version 56592 (0.0008) [2023-10-13 23:16:16,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 115769344. Throughput: 0: 1682.5, 1: 1706.8. Samples: 28942574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:16:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:16:17,820][60935] Updated weights for policy 0, policy_version 56170 (0.0010) [2023-10-13 23:16:18,189][60935] Updated weights for policy 0, policy_version 56180 (0.0010) [2023-10-13 23:16:18,557][60935] Updated weights for policy 0, policy_version 56190 (0.0008) [2023-10-13 23:16:19,994][60934] Updated weights for policy 1, policy_version 56602 (0.0008) [2023-10-13 23:16:20,367][60934] Updated weights for policy 1, policy_version 56612 (0.0008) [2023-10-13 23:16:20,724][60934] Updated weights for policy 1, policy_version 56622 (0.0009) [2023-10-13 23:16:21,095][60934] Updated weights for policy 1, policy_version 56632 (0.0008) [2023-10-13 23:16:21,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 115834880. Throughput: 0: 1690.0, 1: 1706.4. Samples: 28963400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:16:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:16:22,499][60935] Updated weights for policy 0, policy_version 56200 (0.0009) [2023-10-13 23:16:22,873][60935] Updated weights for policy 0, policy_version 56210 (0.0009) [2023-10-13 23:16:23,242][60935] Updated weights for policy 0, policy_version 56220 (0.0008) [2023-10-13 23:16:25,076][60934] Updated weights for policy 1, policy_version 56642 (0.0008) [2023-10-13 23:16:25,451][60934] Updated weights for policy 1, policy_version 56652 (0.0007) [2023-10-13 23:16:25,811][60934] Updated weights for policy 1, policy_version 56662 (0.0009) [2023-10-13 23:16:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 115900416. Throughput: 0: 1710.9, 1: 1684.7. Samples: 28983710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:16:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:16:27,233][60935] Updated weights for policy 0, policy_version 56230 (0.0008) [2023-10-13 23:16:27,601][60935] Updated weights for policy 0, policy_version 56240 (0.0007) [2023-10-13 23:16:27,973][60935] Updated weights for policy 0, policy_version 56250 (0.0008) [2023-10-13 23:16:29,911][60934] Updated weights for policy 1, policy_version 56672 (0.0007) [2023-10-13 23:16:30,270][60934] Updated weights for policy 1, policy_version 56682 (0.0008) [2023-10-13 23:16:30,638][60934] Updated weights for policy 1, policy_version 56692 (0.0008) [2023-10-13 23:16:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 115965952. Throughput: 0: 1686.2, 1: 1704.1. Samples: 28993848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:16:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:16:31,990][60935] Updated weights for policy 0, policy_version 56260 (0.0009) [2023-10-13 23:16:32,348][60935] Updated weights for policy 0, policy_version 56270 (0.0008) [2023-10-13 23:16:32,717][60935] Updated weights for policy 0, policy_version 56280 (0.0010) [2023-10-13 23:16:34,591][60934] Updated weights for policy 1, policy_version 56702 (0.0009) [2023-10-13 23:16:34,953][60934] Updated weights for policy 1, policy_version 56712 (0.0007) [2023-10-13 23:16:35,325][60934] Updated weights for policy 1, policy_version 56722 (0.0009) [2023-10-13 23:16:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 116031488. Throughput: 0: 1712.8, 1: 1697.2. Samples: 29014614. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:16:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:16:36,588][60935] Updated weights for policy 0, policy_version 56290 (0.0009) [2023-10-13 23:16:36,958][60935] Updated weights for policy 0, policy_version 56300 (0.0009) [2023-10-13 23:16:37,326][60935] Updated weights for policy 0, policy_version 56310 (0.0009) [2023-10-13 23:16:37,696][60935] Updated weights for policy 0, policy_version 56320 (0.0009) [2023-10-13 23:16:39,277][60934] Updated weights for policy 1, policy_version 56732 (0.0009) [2023-10-13 23:16:39,641][60934] Updated weights for policy 1, policy_version 56742 (0.0009) [2023-10-13 23:16:40,020][60934] Updated weights for policy 1, policy_version 56752 (0.0009) [2023-10-13 23:16:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 116097024. Throughput: 0: 1710.6, 1: 1676.3. Samples: 29034916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:16:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:16:41,655][60935] Updated weights for policy 0, policy_version 56330 (0.0009) [2023-10-13 23:16:42,025][60935] Updated weights for policy 0, policy_version 56340 (0.0011) [2023-10-13 23:16:42,396][60935] Updated weights for policy 0, policy_version 56350 (0.0010) [2023-10-13 23:16:44,010][60934] Updated weights for policy 1, policy_version 56762 (0.0008) [2023-10-13 23:16:44,368][60934] Updated weights for policy 1, policy_version 56772 (0.0008) [2023-10-13 23:16:44,738][60934] Updated weights for policy 1, policy_version 56782 (0.0010) [2023-10-13 23:16:45,111][60934] Updated weights for policy 1, policy_version 56792 (0.0007) [2023-10-13 23:16:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 116162560. Throughput: 0: 1699.3, 1: 1712.8. Samples: 29045476. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-13 23:16:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:16:46,318][60935] Updated weights for policy 0, policy_version 56360 (0.0007) [2023-10-13 23:16:46,684][60935] Updated weights for policy 0, policy_version 56370 (0.0008) [2023-10-13 23:16:47,060][60935] Updated weights for policy 0, policy_version 56380 (0.0009) [2023-10-13 23:16:49,127][60934] Updated weights for policy 1, policy_version 56802 (0.0007) [2023-10-13 23:16:49,500][60934] Updated weights for policy 1, policy_version 56812 (0.0007) [2023-10-13 23:16:49,871][60934] Updated weights for policy 1, policy_version 56822 (0.0007) [2023-10-13 23:16:51,079][60935] Updated weights for policy 0, policy_version 56390 (0.0012) [2023-10-13 23:16:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 116228096. Throughput: 0: 1713.1, 1: 1695.0. Samples: 29065890. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-13 23:16:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:16:51,450][60935] Updated weights for policy 0, policy_version 56400 (0.0010) [2023-10-13 23:16:51,823][60935] Updated weights for policy 0, policy_version 56410 (0.0010) [2023-10-13 23:16:53,939][60934] Updated weights for policy 1, policy_version 56832 (0.0008) [2023-10-13 23:16:54,305][60934] Updated weights for policy 1, policy_version 56842 (0.0008) [2023-10-13 23:16:54,665][60934] Updated weights for policy 1, policy_version 56852 (0.0007) [2023-10-13 23:16:55,853][60935] Updated weights for policy 0, policy_version 56420 (0.0010) [2023-10-13 23:16:56,242][60935] Updated weights for policy 0, policy_version 56430 (0.0010) [2023-10-13 23:16:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 116293632. Throughput: 0: 1709.6, 1: 1688.6. Samples: 29086098. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-13 23:16:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:16:56,256][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000056856_58523648.pth... [2023-10-13 23:16:56,287][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000055256_56885248.pth [2023-10-13 23:16:56,609][60935] Updated weights for policy 0, policy_version 56440 (0.0007) [2023-10-13 23:16:56,895][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000056448_57802752.pth... [2023-10-13 23:16:56,924][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000054848_56164352.pth [2023-10-13 23:16:58,592][60934] Updated weights for policy 1, policy_version 56862 (0.0008) [2023-10-13 23:16:58,962][60934] Updated weights for policy 1, policy_version 56872 (0.0007) [2023-10-13 23:16:59,322][60934] Updated weights for policy 1, policy_version 56882 (0.0007) [2023-10-13 23:17:00,633][60935] Updated weights for policy 0, policy_version 56450 (0.0010) [2023-10-13 23:17:01,005][60935] Updated weights for policy 0, policy_version 56460 (0.0008) [2023-10-13 23:17:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 116359168. Throughput: 0: 1709.3, 1: 1709.7. Samples: 29096428. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-13 23:17:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:17:01,375][60935] Updated weights for policy 0, policy_version 56470 (0.0007) [2023-10-13 23:17:01,740][60935] Updated weights for policy 0, policy_version 56480 (0.0009) [2023-10-13 23:17:03,421][60934] Updated weights for policy 1, policy_version 56892 (0.0008) [2023-10-13 23:17:03,786][60934] Updated weights for policy 1, policy_version 56902 (0.0009) [2023-10-13 23:17:04,156][60934] Updated weights for policy 1, policy_version 56912 (0.0008) [2023-10-13 23:17:05,637][60935] Updated weights for policy 0, policy_version 56490 (0.0009) [2023-10-13 23:17:06,003][60935] Updated weights for policy 0, policy_version 56500 (0.0010) [2023-10-13 23:17:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 116424704. Throughput: 0: 1718.6, 1: 1682.0. Samples: 29116426. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-13 23:17:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:17:06,376][60935] Updated weights for policy 0, policy_version 56510 (0.0008) [2023-10-13 23:17:08,040][60934] Updated weights for policy 1, policy_version 56922 (0.0008) [2023-10-13 23:17:08,404][60934] Updated weights for policy 1, policy_version 56932 (0.0007) [2023-10-13 23:17:08,765][60934] Updated weights for policy 1, policy_version 56942 (0.0007) [2023-10-13 23:17:09,136][60934] Updated weights for policy 1, policy_version 56952 (0.0007) [2023-10-13 23:17:10,594][60935] Updated weights for policy 0, policy_version 56520 (0.0009) [2023-10-13 23:17:10,965][60935] Updated weights for policy 0, policy_version 56530 (0.0008) [2023-10-13 23:17:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 116490240. Throughput: 0: 1697.6, 1: 1708.7. Samples: 29136990. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-13 23:17:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:17:11,333][60935] Updated weights for policy 0, policy_version 56540 (0.0007) [2023-10-13 23:17:13,066][60934] Updated weights for policy 1, policy_version 56962 (0.0011) [2023-10-13 23:17:13,437][60934] Updated weights for policy 1, policy_version 56972 (0.0010) [2023-10-13 23:17:13,801][60934] Updated weights for policy 1, policy_version 56982 (0.0010) [2023-10-13 23:17:15,263][60935] Updated weights for policy 0, policy_version 56550 (0.0009) [2023-10-13 23:17:15,638][60935] Updated weights for policy 0, policy_version 56560 (0.0007) [2023-10-13 23:17:16,007][60935] Updated weights for policy 0, policy_version 56570 (0.0007) [2023-10-13 23:17:16,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 116588544. Throughput: 0: 1705.9, 1: 1701.5. Samples: 29147182. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-13 23:17:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:17:17,763][60934] Updated weights for policy 1, policy_version 56992 (0.0009) [2023-10-13 23:17:18,128][60934] Updated weights for policy 1, policy_version 57002 (0.0009) [2023-10-13 23:17:18,491][60934] Updated weights for policy 1, policy_version 57012 (0.0007) [2023-10-13 23:17:20,029][60935] Updated weights for policy 0, policy_version 56580 (0.0009) [2023-10-13 23:17:20,402][60935] Updated weights for policy 0, policy_version 56590 (0.0009) [2023-10-13 23:17:20,772][60935] Updated weights for policy 0, policy_version 56600 (0.0010) [2023-10-13 23:17:21,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 116654080. Throughput: 0: 1714.4, 1: 1694.6. Samples: 29168020. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 23:17:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:17:22,595][60934] Updated weights for policy 1, policy_version 57022 (0.0008) [2023-10-13 23:17:22,963][60934] Updated weights for policy 1, policy_version 57032 (0.0007) [2023-10-13 23:17:23,332][60934] Updated weights for policy 1, policy_version 57042 (0.0007) [2023-10-13 23:17:24,833][60935] Updated weights for policy 0, policy_version 56610 (0.0011) [2023-10-13 23:17:25,201][60935] Updated weights for policy 0, policy_version 56620 (0.0008) [2023-10-13 23:17:25,579][60935] Updated weights for policy 0, policy_version 56630 (0.0008) [2023-10-13 23:17:25,948][60935] Updated weights for policy 0, policy_version 56640 (0.0009) [2023-10-13 23:17:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 116719616. Throughput: 0: 1683.7, 1: 1716.3. Samples: 29187916. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 23:17:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:17:27,396][60934] Updated weights for policy 1, policy_version 57052 (0.0009) [2023-10-13 23:17:27,764][60934] Updated weights for policy 1, policy_version 57062 (0.0008) [2023-10-13 23:17:28,135][60934] Updated weights for policy 1, policy_version 57072 (0.0008) [2023-10-13 23:17:29,936][60935] Updated weights for policy 0, policy_version 56650 (0.0008) [2023-10-13 23:17:30,310][60935] Updated weights for policy 0, policy_version 56660 (0.0008) [2023-10-13 23:17:30,682][60935] Updated weights for policy 0, policy_version 56670 (0.0010) [2023-10-13 23:17:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 116785152. Throughput: 0: 1713.0, 1: 1681.5. Samples: 29198228. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 23:17:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:17:32,226][60934] Updated weights for policy 1, policy_version 57082 (0.0008) [2023-10-13 23:17:32,591][60934] Updated weights for policy 1, policy_version 57092 (0.0008) [2023-10-13 23:17:32,965][60934] Updated weights for policy 1, policy_version 57102 (0.0010) [2023-10-13 23:17:33,326][60934] Updated weights for policy 1, policy_version 57112 (0.0009) [2023-10-13 23:17:34,440][60935] Updated weights for policy 0, policy_version 56680 (0.0009) [2023-10-13 23:17:34,806][60935] Updated weights for policy 0, policy_version 56690 (0.0011) [2023-10-13 23:17:35,176][60935] Updated weights for policy 0, policy_version 56700 (0.0009) [2023-10-13 23:17:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 116850688. Throughput: 0: 1695.6, 1: 1696.9. Samples: 29218554. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 23:17:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:17:37,495][60934] Updated weights for policy 1, policy_version 57122 (0.0008) [2023-10-13 23:17:37,864][60934] Updated weights for policy 1, policy_version 57132 (0.0008) [2023-10-13 23:17:38,233][60934] Updated weights for policy 1, policy_version 57142 (0.0009) [2023-10-13 23:17:39,260][60935] Updated weights for policy 0, policy_version 56710 (0.0009) [2023-10-13 23:17:39,630][60935] Updated weights for policy 0, policy_version 56720 (0.0010) [2023-10-13 23:17:40,005][60935] Updated weights for policy 0, policy_version 56730 (0.0007) [2023-10-13 23:17:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 116916224. Throughput: 0: 1687.9, 1: 1711.9. Samples: 29239088. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 23:17:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:17:42,053][60934] Updated weights for policy 1, policy_version 57152 (0.0008) [2023-10-13 23:17:42,426][60934] Updated weights for policy 1, policy_version 57162 (0.0008) [2023-10-13 23:17:42,795][60934] Updated weights for policy 1, policy_version 57172 (0.0007) [2023-10-13 23:17:44,020][60935] Updated weights for policy 0, policy_version 56740 (0.0008) [2023-10-13 23:17:44,386][60935] Updated weights for policy 0, policy_version 56750 (0.0007) [2023-10-13 23:17:44,750][60935] Updated weights for policy 0, policy_version 56760 (0.0007) [2023-10-13 23:17:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 116981760. Throughput: 0: 1719.9, 1: 1683.1. Samples: 29249560. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 23:17:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:17:46,800][60934] Updated weights for policy 1, policy_version 57182 (0.0009) [2023-10-13 23:17:47,170][60934] Updated weights for policy 1, policy_version 57192 (0.0008) [2023-10-13 23:17:47,540][60934] Updated weights for policy 1, policy_version 57202 (0.0009) [2023-10-13 23:17:48,584][60935] Updated weights for policy 0, policy_version 56770 (0.0010) [2023-10-13 23:17:48,947][60935] Updated weights for policy 0, policy_version 56780 (0.0011) [2023-10-13 23:17:49,320][60935] Updated weights for policy 0, policy_version 56790 (0.0009) [2023-10-13 23:17:49,693][60935] Updated weights for policy 0, policy_version 56800 (0.0008) [2023-10-13 23:17:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 117047296. Throughput: 0: 1692.1, 1: 1710.2. Samples: 29269528. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 23:17:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:17:51,650][60934] Updated weights for policy 1, policy_version 57212 (0.0007) [2023-10-13 23:17:52,013][60934] Updated weights for policy 1, policy_version 57222 (0.0010) [2023-10-13 23:17:52,391][60934] Updated weights for policy 1, policy_version 57232 (0.0010) [2023-10-13 23:17:53,608][60935] Updated weights for policy 0, policy_version 56810 (0.0010) [2023-10-13 23:17:53,984][60935] Updated weights for policy 0, policy_version 56820 (0.0011) [2023-10-13 23:17:54,352][60935] Updated weights for policy 0, policy_version 56830 (0.0008) [2023-10-13 23:17:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 117112832. Throughput: 0: 1710.1, 1: 1698.4. Samples: 29290370. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-13 23:17:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:17:56,466][60934] Updated weights for policy 1, policy_version 57242 (0.0008) [2023-10-13 23:17:56,833][60934] Updated weights for policy 1, policy_version 57252 (0.0008) [2023-10-13 23:17:57,198][60934] Updated weights for policy 1, policy_version 57262 (0.0008) [2023-10-13 23:17:57,557][60934] Updated weights for policy 1, policy_version 57272 (0.0009) [2023-10-13 23:17:58,399][60935] Updated weights for policy 0, policy_version 56840 (0.0009) [2023-10-13 23:17:58,780][60935] Updated weights for policy 0, policy_version 56850 (0.0011) [2023-10-13 23:17:59,138][60935] Updated weights for policy 0, policy_version 56860 (0.0010) [2023-10-13 23:18:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 117178368. Throughput: 0: 1710.9, 1: 1686.8. Samples: 29300076. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-10-13 23:18:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:18:01,525][60934] Updated weights for policy 1, policy_version 57282 (0.0007) [2023-10-13 23:18:01,893][60934] Updated weights for policy 1, policy_version 57292 (0.0009) [2023-10-13 23:18:02,262][60934] Updated weights for policy 1, policy_version 57302 (0.0008) [2023-10-13 23:18:03,292][60935] Updated weights for policy 0, policy_version 56870 (0.0008) [2023-10-13 23:18:03,663][60935] Updated weights for policy 0, policy_version 56880 (0.0008) [2023-10-13 23:18:04,034][60935] Updated weights for policy 0, policy_version 56890 (0.0008) [2023-10-13 23:18:06,184][60934] Updated weights for policy 1, policy_version 57312 (0.0009) [2023-10-13 23:18:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 117243904. Throughput: 0: 1691.6, 1: 1702.5. Samples: 29320758. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-10-13 23:18:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:18:06,555][60934] Updated weights for policy 1, policy_version 57322 (0.0011) [2023-10-13 23:18:06,919][60934] Updated weights for policy 1, policy_version 57332 (0.0009) [2023-10-13 23:18:07,967][60935] Updated weights for policy 0, policy_version 56900 (0.0007) [2023-10-13 23:18:08,331][60935] Updated weights for policy 0, policy_version 56910 (0.0010) [2023-10-13 23:18:08,701][60935] Updated weights for policy 0, policy_version 56920 (0.0010) [2023-10-13 23:18:10,992][60934] Updated weights for policy 1, policy_version 57342 (0.0008) [2023-10-13 23:18:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 117309440. Throughput: 0: 1720.6, 1: 1701.4. Samples: 29341906. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-10-13 23:18:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:18:11,368][60934] Updated weights for policy 1, policy_version 57352 (0.0008) [2023-10-13 23:18:11,733][60934] Updated weights for policy 1, policy_version 57362 (0.0009) [2023-10-13 23:18:12,660][60935] Updated weights for policy 0, policy_version 56930 (0.0007) [2023-10-13 23:18:13,028][60935] Updated weights for policy 0, policy_version 56940 (0.0008) [2023-10-13 23:18:13,393][60935] Updated weights for policy 0, policy_version 56950 (0.0008) [2023-10-13 23:18:13,763][60935] Updated weights for policy 0, policy_version 56960 (0.0008) [2023-10-13 23:18:15,799][60934] Updated weights for policy 1, policy_version 57372 (0.0010) [2023-10-13 23:18:16,163][60934] Updated weights for policy 1, policy_version 57382 (0.0008) [2023-10-13 23:18:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 117374976. Throughput: 0: 1694.6, 1: 1703.0. Samples: 29351122. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-10-13 23:18:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:18:16,532][60934] Updated weights for policy 1, policy_version 57392 (0.0009) [2023-10-13 23:18:17,743][60935] Updated weights for policy 0, policy_version 56970 (0.0008) [2023-10-13 23:18:18,101][60935] Updated weights for policy 0, policy_version 56980 (0.0009) [2023-10-13 23:18:18,480][60935] Updated weights for policy 0, policy_version 56990 (0.0009) [2023-10-13 23:18:20,750][60934] Updated weights for policy 1, policy_version 57402 (0.0008) [2023-10-13 23:18:21,115][60934] Updated weights for policy 1, policy_version 57412 (0.0010) [2023-10-13 23:18:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 117440512. Throughput: 0: 1711.5, 1: 1701.0. Samples: 29372114. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-10-13 23:18:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:18:21,480][60934] Updated weights for policy 1, policy_version 57422 (0.0007) [2023-10-13 23:18:21,846][60934] Updated weights for policy 1, policy_version 57432 (0.0009) [2023-10-13 23:18:22,545][60935] Updated weights for policy 0, policy_version 57000 (0.0010) [2023-10-13 23:18:22,916][60935] Updated weights for policy 0, policy_version 57010 (0.0011) [2023-10-13 23:18:23,287][60935] Updated weights for policy 0, policy_version 57020 (0.0010) [2023-10-13 23:18:25,973][60934] Updated weights for policy 1, policy_version 57442 (0.0009) [2023-10-13 23:18:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 117506048. Throughput: 0: 1721.9, 1: 1693.5. Samples: 29392782. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-10-13 23:18:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:18:26,361][60934] Updated weights for policy 1, policy_version 57452 (0.0008) [2023-10-13 23:18:26,727][60934] Updated weights for policy 1, policy_version 57462 (0.0010) [2023-10-13 23:18:27,258][60935] Updated weights for policy 0, policy_version 57030 (0.0008) [2023-10-13 23:18:27,626][60935] Updated weights for policy 0, policy_version 57040 (0.0008) [2023-10-13 23:18:27,992][60935] Updated weights for policy 0, policy_version 57050 (0.0007) [2023-10-13 23:18:30,529][60934] Updated weights for policy 1, policy_version 57472 (0.0008) [2023-10-13 23:18:30,906][60934] Updated weights for policy 1, policy_version 57482 (0.0008) [2023-10-13 23:18:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 117571584. Throughput: 0: 1694.4, 1: 1696.8. Samples: 29402160. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-10-13 23:18:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:18:31,283][60934] Updated weights for policy 1, policy_version 57492 (0.0008) [2023-10-13 23:18:31,888][60935] Updated weights for policy 0, policy_version 57060 (0.0008) [2023-10-13 23:18:32,256][60935] Updated weights for policy 0, policy_version 57070 (0.0010) [2023-10-13 23:18:32,632][60935] Updated weights for policy 0, policy_version 57080 (0.0010) [2023-10-13 23:18:35,193][60934] Updated weights for policy 1, policy_version 57502 (0.0009) [2023-10-13 23:18:35,564][60934] Updated weights for policy 1, policy_version 57512 (0.0008) [2023-10-13 23:18:35,924][60934] Updated weights for policy 1, policy_version 57522 (0.0007) [2023-10-13 23:18:36,249][59943] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 117669888. Throughput: 0: 1720.4, 1: 1702.3. Samples: 29423548. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-10-13 23:18:36,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:18:36,649][60935] Updated weights for policy 0, policy_version 57090 (0.0009) [2023-10-13 23:18:37,056][60935] Updated weights for policy 0, policy_version 57100 (0.0008) [2023-10-13 23:18:37,440][60935] Updated weights for policy 0, policy_version 57110 (0.0009) [2023-10-13 23:18:37,815][60935] Updated weights for policy 0, policy_version 57120 (0.0009) [2023-10-13 23:18:39,917][60934] Updated weights for policy 1, policy_version 57532 (0.0008) [2023-10-13 23:18:40,282][60934] Updated weights for policy 1, policy_version 57542 (0.0008) [2023-10-13 23:18:40,651][60934] Updated weights for policy 1, policy_version 57552 (0.0008) [2023-10-13 23:18:41,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 117735424. Throughput: 0: 1718.0, 1: 1690.6. Samples: 29443760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:18:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:18:41,781][60935] Updated weights for policy 0, policy_version 57130 (0.0009) [2023-10-13 23:18:42,149][60935] Updated weights for policy 0, policy_version 57140 (0.0009) [2023-10-13 23:18:42,523][60935] Updated weights for policy 0, policy_version 57150 (0.0009) [2023-10-13 23:18:44,758][60934] Updated weights for policy 1, policy_version 57562 (0.0009) [2023-10-13 23:18:45,118][60934] Updated weights for policy 1, policy_version 57572 (0.0009) [2023-10-13 23:18:45,474][60934] Updated weights for policy 1, policy_version 57582 (0.0008) [2023-10-13 23:18:45,849][60934] Updated weights for policy 1, policy_version 57592 (0.0007) [2023-10-13 23:18:46,248][59943] Fps is (10 sec: 13107.8, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 117800960. Throughput: 0: 1707.1, 1: 1709.0. Samples: 29453800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:18:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:18:46,623][60935] Updated weights for policy 0, policy_version 57160 (0.0007) [2023-10-13 23:18:46,986][60935] Updated weights for policy 0, policy_version 57170 (0.0011) [2023-10-13 23:18:47,350][60935] Updated weights for policy 0, policy_version 57180 (0.0009) [2023-10-13 23:18:49,868][60934] Updated weights for policy 1, policy_version 57602 (0.0009) [2023-10-13 23:18:50,241][60934] Updated weights for policy 1, policy_version 57612 (0.0008) [2023-10-13 23:18:50,613][60934] Updated weights for policy 1, policy_version 57622 (0.0009) [2023-10-13 23:18:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 117866496. Throughput: 0: 1717.6, 1: 1700.6. Samples: 29474580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:18:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:18:51,274][60935] Updated weights for policy 0, policy_version 57190 (0.0009) [2023-10-13 23:18:51,642][60935] Updated weights for policy 0, policy_version 57200 (0.0008) [2023-10-13 23:18:52,006][60935] Updated weights for policy 0, policy_version 57210 (0.0009) [2023-10-13 23:18:54,576][60934] Updated weights for policy 1, policy_version 57632 (0.0009) [2023-10-13 23:18:54,942][60934] Updated weights for policy 1, policy_version 57642 (0.0009) [2023-10-13 23:18:55,313][60934] Updated weights for policy 1, policy_version 57652 (0.0009) [2023-10-13 23:18:56,179][60935] Updated weights for policy 0, policy_version 57220 (0.0009) [2023-10-13 23:18:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 117932032. Throughput: 0: 1713.4, 1: 1672.9. Samples: 29494290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:18:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:18:56,257][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000057656_59342848.pth... [2023-10-13 23:18:56,288][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000056056_57704448.pth [2023-10-13 23:18:56,543][60935] Updated weights for policy 0, policy_version 57230 (0.0007) [2023-10-13 23:18:56,914][60935] Updated weights for policy 0, policy_version 57240 (0.0009) [2023-10-13 23:18:57,210][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000057248_58621952.pth... [2023-10-13 23:18:57,247][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000055648_56983552.pth [2023-10-13 23:18:59,194][60934] Updated weights for policy 1, policy_version 57662 (0.0008) [2023-10-13 23:18:59,566][60934] Updated weights for policy 1, policy_version 57672 (0.0007) [2023-10-13 23:18:59,934][60934] Updated weights for policy 1, policy_version 57682 (0.0009) [2023-10-13 23:19:00,972][60935] Updated weights for policy 0, policy_version 57250 (0.0008) [2023-10-13 23:19:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 117997568. Throughput: 0: 1709.4, 1: 1706.4. Samples: 29504830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:19:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:19:01,342][60935] Updated weights for policy 0, policy_version 57260 (0.0009) [2023-10-13 23:19:01,716][60935] Updated weights for policy 0, policy_version 57270 (0.0011) [2023-10-13 23:19:02,077][60935] Updated weights for policy 0, policy_version 57280 (0.0011) [2023-10-13 23:19:03,935][60934] Updated weights for policy 1, policy_version 57692 (0.0008) [2023-10-13 23:19:04,306][60934] Updated weights for policy 1, policy_version 57702 (0.0008) [2023-10-13 23:19:04,674][60934] Updated weights for policy 1, policy_version 57712 (0.0008) [2023-10-13 23:19:06,032][60935] Updated weights for policy 0, policy_version 57290 (0.0007) [2023-10-13 23:19:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 118063104. Throughput: 0: 1709.8, 1: 1695.6. Samples: 29525354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:19:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:19:06,411][60935] Updated weights for policy 0, policy_version 57300 (0.0008) [2023-10-13 23:19:06,777][60935] Updated weights for policy 0, policy_version 57310 (0.0007) [2023-10-13 23:19:08,626][60934] Updated weights for policy 1, policy_version 57722 (0.0008) [2023-10-13 23:19:08,993][60934] Updated weights for policy 1, policy_version 57732 (0.0007) [2023-10-13 23:19:09,358][60934] Updated weights for policy 1, policy_version 57742 (0.0007) [2023-10-13 23:19:09,720][60934] Updated weights for policy 1, policy_version 57752 (0.0009) [2023-10-13 23:19:10,729][60935] Updated weights for policy 0, policy_version 57320 (0.0009) [2023-10-13 23:19:11,108][60935] Updated weights for policy 0, policy_version 57330 (0.0008) [2023-10-13 23:19:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 118128640. Throughput: 0: 1704.7, 1: 1690.7. Samples: 29545574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:19:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:19:11,471][60935] Updated weights for policy 0, policy_version 57340 (0.0010) [2023-10-13 23:19:13,988][60934] Updated weights for policy 1, policy_version 57762 (0.0011) [2023-10-13 23:19:14,358][60934] Updated weights for policy 1, policy_version 57772 (0.0009) [2023-10-13 23:19:14,724][60934] Updated weights for policy 1, policy_version 57782 (0.0009) [2023-10-13 23:19:15,562][60935] Updated weights for policy 0, policy_version 57350 (0.0007) [2023-10-13 23:19:15,928][60935] Updated weights for policy 0, policy_version 57360 (0.0010) [2023-10-13 23:19:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 118194176. Throughput: 0: 1707.0, 1: 1722.0. Samples: 29556468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:19:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:19:16,294][60935] Updated weights for policy 0, policy_version 57370 (0.0010) [2023-10-13 23:19:18,700][60934] Updated weights for policy 1, policy_version 57792 (0.0011) [2023-10-13 23:19:19,062][60934] Updated weights for policy 1, policy_version 57802 (0.0010) [2023-10-13 23:19:19,435][60934] Updated weights for policy 1, policy_version 57812 (0.0009) [2023-10-13 23:19:20,138][60935] Updated weights for policy 0, policy_version 57380 (0.0010) [2023-10-13 23:19:20,504][60935] Updated weights for policy 0, policy_version 57390 (0.0007) [2023-10-13 23:19:20,878][60935] Updated weights for policy 0, policy_version 57400 (0.0007) [2023-10-13 23:19:21,248][59943] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 118292480. Throughput: 0: 1705.4, 1: 1688.3. Samples: 29576264. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-13 23:19:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:19:23,414][60934] Updated weights for policy 1, policy_version 57822 (0.0008) [2023-10-13 23:19:23,792][60934] Updated weights for policy 1, policy_version 57832 (0.0007) [2023-10-13 23:19:24,166][60934] Updated weights for policy 1, policy_version 57842 (0.0008) [2023-10-13 23:19:24,927][60935] Updated weights for policy 0, policy_version 57410 (0.0008) [2023-10-13 23:19:25,297][60935] Updated weights for policy 0, policy_version 57420 (0.0008) [2023-10-13 23:19:25,664][60935] Updated weights for policy 0, policy_version 57430 (0.0007) [2023-10-13 23:19:26,028][60935] Updated weights for policy 0, policy_version 57440 (0.0008) [2023-10-13 23:19:26,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 118358016. Throughput: 0: 1684.7, 1: 1706.0. Samples: 29596342. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-13 23:19:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:19:28,197][60934] Updated weights for policy 1, policy_version 57852 (0.0007) [2023-10-13 23:19:28,567][60934] Updated weights for policy 1, policy_version 57862 (0.0009) [2023-10-13 23:19:28,933][60934] Updated weights for policy 1, policy_version 57872 (0.0008) [2023-10-13 23:19:30,055][60935] Updated weights for policy 0, policy_version 57450 (0.0009) [2023-10-13 23:19:30,414][60935] Updated weights for policy 0, policy_version 57460 (0.0008) [2023-10-13 23:19:30,781][60935] Updated weights for policy 0, policy_version 57470 (0.0010) [2023-10-13 23:19:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 118423552. Throughput: 0: 1707.1, 1: 1703.2. Samples: 29607264. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-13 23:19:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:19:32,691][60934] Updated weights for policy 1, policy_version 57882 (0.0007) [2023-10-13 23:19:33,061][60934] Updated weights for policy 1, policy_version 57892 (0.0007) [2023-10-13 23:19:33,421][60934] Updated weights for policy 1, policy_version 57902 (0.0007) [2023-10-13 23:19:33,788][60934] Updated weights for policy 1, policy_version 57912 (0.0009) [2023-10-13 23:19:34,849][60935] Updated weights for policy 0, policy_version 57480 (0.0009) [2023-10-13 23:19:35,220][60935] Updated weights for policy 0, policy_version 57490 (0.0009) [2023-10-13 23:19:35,589][60935] Updated weights for policy 0, policy_version 57500 (0.0008) [2023-10-13 23:19:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 118489088. Throughput: 0: 1702.4, 1: 1693.7. Samples: 29627406. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-13 23:19:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:19:37,757][60934] Updated weights for policy 1, policy_version 57922 (0.0009) [2023-10-13 23:19:38,122][60934] Updated weights for policy 1, policy_version 57932 (0.0011) [2023-10-13 23:19:38,489][60934] Updated weights for policy 1, policy_version 57942 (0.0008) [2023-10-13 23:19:39,602][60935] Updated weights for policy 0, policy_version 57510 (0.0008) [2023-10-13 23:19:39,971][60935] Updated weights for policy 0, policy_version 57520 (0.0007) [2023-10-13 23:19:40,349][60935] Updated weights for policy 0, policy_version 57530 (0.0007) [2023-10-13 23:19:41,249][59943] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 118554624. Throughput: 0: 1681.5, 1: 1719.7. Samples: 29647344. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-13 23:19:41,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:19:42,457][60934] Updated weights for policy 1, policy_version 57952 (0.0007) [2023-10-13 23:19:42,825][60934] Updated weights for policy 1, policy_version 57962 (0.0007) [2023-10-13 23:19:43,182][60934] Updated weights for policy 1, policy_version 57972 (0.0007) [2023-10-13 23:19:44,188][60935] Updated weights for policy 0, policy_version 57540 (0.0010) [2023-10-13 23:19:44,556][60935] Updated weights for policy 0, policy_version 57550 (0.0011) [2023-10-13 23:19:44,926][60935] Updated weights for policy 0, policy_version 57560 (0.0010) [2023-10-13 23:19:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 118620160. Throughput: 0: 1717.6, 1: 1687.2. Samples: 29658048. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-13 23:19:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:19:46,920][60934] Updated weights for policy 1, policy_version 57982 (0.0008) [2023-10-13 23:19:47,283][60934] Updated weights for policy 1, policy_version 57992 (0.0009) [2023-10-13 23:19:47,661][60934] Updated weights for policy 1, policy_version 58002 (0.0010) [2023-10-13 23:19:48,863][60935] Updated weights for policy 0, policy_version 57570 (0.0009) [2023-10-13 23:19:49,237][60935] Updated weights for policy 0, policy_version 57580 (0.0009) [2023-10-13 23:19:49,609][60935] Updated weights for policy 0, policy_version 57590 (0.0010) [2023-10-13 23:19:49,978][60935] Updated weights for policy 0, policy_version 57600 (0.0009) [2023-10-13 23:19:51,248][59943] Fps is (10 sec: 13107.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 118685696. Throughput: 0: 1689.9, 1: 1707.3. Samples: 29678226. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-13 23:19:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:19:51,709][60934] Updated weights for policy 1, policy_version 58012 (0.0008) [2023-10-13 23:19:52,079][60934] Updated weights for policy 1, policy_version 58022 (0.0009) [2023-10-13 23:19:52,447][60934] Updated weights for policy 1, policy_version 58032 (0.0007) [2023-10-13 23:19:53,898][60935] Updated weights for policy 0, policy_version 57610 (0.0009) [2023-10-13 23:19:54,262][60935] Updated weights for policy 0, policy_version 57620 (0.0009) [2023-10-13 23:19:54,642][60935] Updated weights for policy 0, policy_version 57630 (0.0010) [2023-10-13 23:19:56,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 118751232. Throughput: 0: 1694.7, 1: 1724.8. Samples: 29699452. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-13 23:19:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:19:56,400][60934] Updated weights for policy 1, policy_version 58042 (0.0009) [2023-10-13 23:19:56,769][60934] Updated weights for policy 1, policy_version 58052 (0.0009) [2023-10-13 23:19:57,134][60934] Updated weights for policy 1, policy_version 58062 (0.0009) [2023-10-13 23:19:57,492][60934] Updated weights for policy 1, policy_version 58072 (0.0008) [2023-10-13 23:19:58,650][60935] Updated weights for policy 0, policy_version 57640 (0.0008) [2023-10-13 23:19:59,026][60935] Updated weights for policy 0, policy_version 57650 (0.0009) [2023-10-13 23:19:59,389][60935] Updated weights for policy 0, policy_version 57660 (0.0010) [2023-10-13 23:20:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 118816768. Throughput: 0: 1705.5, 1: 1693.6. Samples: 29709426. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-13 23:20:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:20:01,605][60934] Updated weights for policy 1, policy_version 58082 (0.0007) [2023-10-13 23:20:01,972][60934] Updated weights for policy 1, policy_version 58092 (0.0007) [2023-10-13 23:20:02,343][60934] Updated weights for policy 1, policy_version 58102 (0.0007) [2023-10-13 23:20:03,487][60935] Updated weights for policy 0, policy_version 57670 (0.0009) [2023-10-13 23:20:03,847][60935] Updated weights for policy 0, policy_version 57680 (0.0009) [2023-10-13 23:20:04,213][60935] Updated weights for policy 0, policy_version 57690 (0.0010) [2023-10-13 23:20:06,222][60934] Updated weights for policy 1, policy_version 58112 (0.0008) [2023-10-13 23:20:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 118882304. Throughput: 0: 1684.2, 1: 1720.0. Samples: 29729454. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-13 23:20:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:20:06,585][60934] Updated weights for policy 1, policy_version 58122 (0.0007) [2023-10-13 23:20:06,955][60934] Updated weights for policy 1, policy_version 58132 (0.0009) [2023-10-13 23:20:08,272][60935] Updated weights for policy 0, policy_version 57700 (0.0009) [2023-10-13 23:20:08,637][60935] Updated weights for policy 0, policy_version 57710 (0.0011) [2023-10-13 23:20:09,012][60935] Updated weights for policy 0, policy_version 57720 (0.0010) [2023-10-13 23:20:10,866][60934] Updated weights for policy 1, policy_version 58142 (0.0010) [2023-10-13 23:20:11,222][60934] Updated weights for policy 1, policy_version 58152 (0.0010) [2023-10-13 23:20:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 118947840. Throughput: 0: 1703.9, 1: 1722.0. Samples: 29750508. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-13 23:20:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:20:11,591][60934] Updated weights for policy 1, policy_version 58162 (0.0010) [2023-10-13 23:20:13,022][60935] Updated weights for policy 0, policy_version 57730 (0.0008) [2023-10-13 23:20:13,405][60935] Updated weights for policy 0, policy_version 57740 (0.0007) [2023-10-13 23:20:13,774][60935] Updated weights for policy 0, policy_version 57750 (0.0007) [2023-10-13 23:20:14,141][60935] Updated weights for policy 0, policy_version 57760 (0.0010) [2023-10-13 23:20:15,748][60934] Updated weights for policy 1, policy_version 58172 (0.0010) [2023-10-13 23:20:16,115][60934] Updated weights for policy 1, policy_version 58182 (0.0008) [2023-10-13 23:20:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 119013376. Throughput: 0: 1690.0, 1: 1704.2. Samples: 29760002. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-13 23:20:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:20:16,483][60934] Updated weights for policy 1, policy_version 58192 (0.0008) [2023-10-13 23:20:18,024][60935] Updated weights for policy 0, policy_version 57770 (0.0011) [2023-10-13 23:20:18,393][60935] Updated weights for policy 0, policy_version 57780 (0.0010) [2023-10-13 23:20:18,761][60935] Updated weights for policy 0, policy_version 57790 (0.0009) [2023-10-13 23:20:20,868][60934] Updated weights for policy 1, policy_version 58202 (0.0010) [2023-10-13 23:20:21,238][60934] Updated weights for policy 1, policy_version 58212 (0.0010) [2023-10-13 23:20:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 119078912. Throughput: 0: 1691.7, 1: 1710.8. Samples: 29780516. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-13 23:20:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:20:21,606][60934] Updated weights for policy 1, policy_version 58222 (0.0008) [2023-10-13 23:20:21,978][60934] Updated weights for policy 1, policy_version 58232 (0.0010) [2023-10-13 23:20:22,733][60935] Updated weights for policy 0, policy_version 57800 (0.0010) [2023-10-13 23:20:23,103][60935] Updated weights for policy 0, policy_version 57810 (0.0007) [2023-10-13 23:20:23,467][60935] Updated weights for policy 0, policy_version 57820 (0.0008) [2023-10-13 23:20:25,948][60934] Updated weights for policy 1, policy_version 58242 (0.0008) [2023-10-13 23:20:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 119144448. Throughput: 0: 1714.5, 1: 1710.6. Samples: 29801472. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-13 23:20:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:20:26,321][60934] Updated weights for policy 1, policy_version 58252 (0.0007) [2023-10-13 23:20:26,688][60934] Updated weights for policy 1, policy_version 58262 (0.0009) [2023-10-13 23:20:27,496][60935] Updated weights for policy 0, policy_version 57830 (0.0007) [2023-10-13 23:20:27,868][60935] Updated weights for policy 0, policy_version 57840 (0.0009) [2023-10-13 23:20:28,238][60935] Updated weights for policy 0, policy_version 57850 (0.0009) [2023-10-13 23:20:30,889][60934] Updated weights for policy 1, policy_version 58272 (0.0008) [2023-10-13 23:20:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 119209984. Throughput: 0: 1681.5, 1: 1709.7. Samples: 29810652. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-13 23:20:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:20:31,263][60934] Updated weights for policy 1, policy_version 58282 (0.0009) [2023-10-13 23:20:31,629][60934] Updated weights for policy 1, policy_version 58292 (0.0007) [2023-10-13 23:20:32,175][60935] Updated weights for policy 0, policy_version 57860 (0.0008) [2023-10-13 23:20:32,557][60935] Updated weights for policy 0, policy_version 57870 (0.0009) [2023-10-13 23:20:32,933][60935] Updated weights for policy 0, policy_version 57880 (0.0008) [2023-10-13 23:20:35,679][60934] Updated weights for policy 1, policy_version 58302 (0.0009) [2023-10-13 23:20:36,048][60934] Updated weights for policy 1, policy_version 58312 (0.0007) [2023-10-13 23:20:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 119275520. Throughput: 0: 1707.6, 1: 1706.5. Samples: 29831864. Policy #0 lag: (min: 14.0, avg: 23.2, max: 46.0) [2023-10-13 23:20:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:20:36,414][60934] Updated weights for policy 1, policy_version 58322 (0.0007) [2023-10-13 23:20:36,896][60935] Updated weights for policy 0, policy_version 57890 (0.0008) [2023-10-13 23:20:37,280][60935] Updated weights for policy 0, policy_version 57900 (0.0007) [2023-10-13 23:20:37,643][60935] Updated weights for policy 0, policy_version 57910 (0.0009) [2023-10-13 23:20:38,017][60935] Updated weights for policy 0, policy_version 57920 (0.0009) [2023-10-13 23:20:40,430][60934] Updated weights for policy 1, policy_version 58332 (0.0009) [2023-10-13 23:20:40,792][60934] Updated weights for policy 1, policy_version 58342 (0.0009) [2023-10-13 23:20:41,161][60934] Updated weights for policy 1, policy_version 58352 (0.0009) [2023-10-13 23:20:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 119341056. Throughput: 0: 1708.1, 1: 1689.3. Samples: 29852336. Policy #0 lag: (min: 14.0, avg: 23.2, max: 46.0) [2023-10-13 23:20:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:20:42,044][60935] Updated weights for policy 0, policy_version 57930 (0.0010) [2023-10-13 23:20:42,410][60935] Updated weights for policy 0, policy_version 57940 (0.0008) [2023-10-13 23:20:42,774][60935] Updated weights for policy 0, policy_version 57950 (0.0008) [2023-10-13 23:20:45,151][60934] Updated weights for policy 1, policy_version 58362 (0.0007) [2023-10-13 23:20:45,510][60934] Updated weights for policy 1, policy_version 58372 (0.0008) [2023-10-13 23:20:45,884][60934] Updated weights for policy 1, policy_version 58382 (0.0008) [2023-10-13 23:20:46,243][60934] Updated weights for policy 1, policy_version 58392 (0.0007) [2023-10-13 23:20:46,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 119439360. Throughput: 0: 1693.2, 1: 1696.2. Samples: 29861950. Policy #0 lag: (min: 14.0, avg: 23.2, max: 46.0) [2023-10-13 23:20:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:20:46,768][60935] Updated weights for policy 0, policy_version 57960 (0.0008) [2023-10-13 23:20:47,140][60935] Updated weights for policy 0, policy_version 57970 (0.0009) [2023-10-13 23:20:47,511][60935] Updated weights for policy 0, policy_version 57980 (0.0009) [2023-10-13 23:20:50,189][60934] Updated weights for policy 1, policy_version 58402 (0.0010) [2023-10-13 23:20:50,562][60934] Updated weights for policy 1, policy_version 58412 (0.0008) [2023-10-13 23:20:50,926][60934] Updated weights for policy 1, policy_version 58422 (0.0009) [2023-10-13 23:20:51,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 119504896. Throughput: 0: 1708.7, 1: 1703.2. Samples: 29882988. Policy #0 lag: (min: 14.0, avg: 23.2, max: 46.0) [2023-10-13 23:20:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:20:51,513][60935] Updated weights for policy 0, policy_version 57990 (0.0010) [2023-10-13 23:20:51,874][60935] Updated weights for policy 0, policy_version 58000 (0.0009) [2023-10-13 23:20:52,252][60935] Updated weights for policy 0, policy_version 58010 (0.0008) [2023-10-13 23:20:54,875][60934] Updated weights for policy 1, policy_version 58432 (0.0011) [2023-10-13 23:20:55,241][60934] Updated weights for policy 1, policy_version 58442 (0.0008) [2023-10-13 23:20:55,616][60934] Updated weights for policy 1, policy_version 58452 (0.0010) [2023-10-13 23:20:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 119570432. Throughput: 0: 1714.6, 1: 1681.1. Samples: 29903314. Policy #0 lag: (min: 14.0, avg: 23.2, max: 46.0) [2023-10-13 23:20:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:20:56,257][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000058456_60162048.pth... [2023-10-13 23:20:56,268][60935] Updated weights for policy 0, policy_version 58020 (0.0007) [2023-10-13 23:20:56,294][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000056856_58523648.pth [2023-10-13 23:20:56,643][60935] Updated weights for policy 0, policy_version 58030 (0.0008) [2023-10-13 23:20:57,013][60935] Updated weights for policy 0, policy_version 58040 (0.0011) [2023-10-13 23:20:57,298][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000058048_59441152.pth... [2023-10-13 23:20:57,327][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000056448_57802752.pth [2023-10-13 23:20:59,619][60934] Updated weights for policy 1, policy_version 58462 (0.0007) [2023-10-13 23:20:59,996][60934] Updated weights for policy 1, policy_version 58472 (0.0009) [2023-10-13 23:21:00,363][60934] Updated weights for policy 1, policy_version 58482 (0.0008) [2023-10-13 23:21:01,043][60935] Updated weights for policy 0, policy_version 58050 (0.0009) [2023-10-13 23:21:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 119635968. Throughput: 0: 1705.2, 1: 1707.7. Samples: 29913584. Policy #0 lag: (min: 14.0, avg: 23.2, max: 46.0) [2023-10-13 23:21:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:21:01,439][60935] Updated weights for policy 0, policy_version 58060 (0.0009) [2023-10-13 23:21:01,796][60935] Updated weights for policy 0, policy_version 58070 (0.0010) [2023-10-13 23:21:02,160][60935] Updated weights for policy 0, policy_version 58080 (0.0008) [2023-10-13 23:21:04,393][60934] Updated weights for policy 1, policy_version 58492 (0.0009) [2023-10-13 23:21:04,760][60934] Updated weights for policy 1, policy_version 58502 (0.0007) [2023-10-13 23:21:05,119][60934] Updated weights for policy 1, policy_version 58512 (0.0007) [2023-10-13 23:21:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 119701504. Throughput: 0: 1709.3, 1: 1704.8. Samples: 29934152. Policy #0 lag: (min: 14.0, avg: 23.2, max: 46.0) [2023-10-13 23:21:06,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:21:06,261][60935] Updated weights for policy 0, policy_version 58090 (0.0009) [2023-10-13 23:21:06,627][60935] Updated weights for policy 0, policy_version 58100 (0.0009) [2023-10-13 23:21:06,998][60935] Updated weights for policy 0, policy_version 58110 (0.0007) [2023-10-13 23:21:09,050][60934] Updated weights for policy 1, policy_version 58522 (0.0009) [2023-10-13 23:21:09,429][60934] Updated weights for policy 1, policy_version 58532 (0.0008) [2023-10-13 23:21:09,796][60934] Updated weights for policy 1, policy_version 58542 (0.0008) [2023-10-13 23:21:10,167][60934] Updated weights for policy 1, policy_version 58552 (0.0007) [2023-10-13 23:21:11,219][60935] Updated weights for policy 0, policy_version 58120 (0.0007) [2023-10-13 23:21:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 119767040. Throughput: 0: 1702.1, 1: 1683.9. Samples: 29953842. Policy #0 lag: (min: 14.0, avg: 23.2, max: 46.0) [2023-10-13 23:21:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:21:11,601][60935] Updated weights for policy 0, policy_version 58130 (0.0009) [2023-10-13 23:21:11,968][60935] Updated weights for policy 0, policy_version 58140 (0.0009) [2023-10-13 23:21:14,112][60934] Updated weights for policy 1, policy_version 58562 (0.0009) [2023-10-13 23:21:14,485][60934] Updated weights for policy 1, policy_version 58572 (0.0008) [2023-10-13 23:21:14,853][60934] Updated weights for policy 1, policy_version 58582 (0.0009) [2023-10-13 23:21:15,916][60935] Updated weights for policy 0, policy_version 58150 (0.0011) [2023-10-13 23:21:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 119832576. Throughput: 0: 1701.8, 1: 1716.4. Samples: 29964468. Policy #0 lag: (min: 16.0, avg: 34.5, max: 48.0) [2023-10-13 23:21:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:21:16,282][60935] Updated weights for policy 0, policy_version 58160 (0.0010) [2023-10-13 23:21:16,658][60935] Updated weights for policy 0, policy_version 58170 (0.0008) [2023-10-13 23:21:18,693][60934] Updated weights for policy 1, policy_version 58592 (0.0009) [2023-10-13 23:21:19,063][60934] Updated weights for policy 1, policy_version 58602 (0.0008) [2023-10-13 23:21:19,422][60934] Updated weights for policy 1, policy_version 58612 (0.0007) [2023-10-13 23:21:20,668][60935] Updated weights for policy 0, policy_version 58180 (0.0009) [2023-10-13 23:21:21,037][60935] Updated weights for policy 0, policy_version 58190 (0.0008) [2023-10-13 23:21:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 119898112. Throughput: 0: 1700.4, 1: 1687.4. Samples: 29984312. Policy #0 lag: (min: 16.0, avg: 34.5, max: 48.0) [2023-10-13 23:21:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:21:21,399][60935] Updated weights for policy 0, policy_version 58200 (0.0008) [2023-10-13 23:21:23,561][60934] Updated weights for policy 1, policy_version 58622 (0.0009) [2023-10-13 23:21:23,922][60934] Updated weights for policy 1, policy_version 58632 (0.0007) [2023-10-13 23:21:24,297][60934] Updated weights for policy 1, policy_version 58642 (0.0008) [2023-10-13 23:21:25,263][60935] Updated weights for policy 0, policy_version 58210 (0.0010) [2023-10-13 23:21:25,628][60935] Updated weights for policy 0, policy_version 58220 (0.0010) [2023-10-13 23:21:26,000][60935] Updated weights for policy 0, policy_version 58230 (0.0010) [2023-10-13 23:21:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 119963648. Throughput: 0: 1688.4, 1: 1696.3. Samples: 30004644. Policy #0 lag: (min: 16.0, avg: 34.5, max: 48.0) [2023-10-13 23:21:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:21:26,374][60935] Updated weights for policy 0, policy_version 58240 (0.0009) [2023-10-13 23:21:28,204][60934] Updated weights for policy 1, policy_version 58652 (0.0008) [2023-10-13 23:21:28,572][60934] Updated weights for policy 1, policy_version 58662 (0.0009) [2023-10-13 23:21:28,934][60934] Updated weights for policy 1, policy_version 58672 (0.0010) [2023-10-13 23:21:30,365][60935] Updated weights for policy 0, policy_version 58250 (0.0009) [2023-10-13 23:21:30,734][60935] Updated weights for policy 0, policy_version 58260 (0.0011) [2023-10-13 23:21:31,101][60935] Updated weights for policy 0, policy_version 58270 (0.0011) [2023-10-13 23:21:31,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 120061952. Throughput: 0: 1701.2, 1: 1707.3. Samples: 30015332. Policy #0 lag: (min: 16.0, avg: 34.5, max: 48.0) [2023-10-13 23:21:31,249][59943] Avg episode reward: [(0, '-0.120'), (1, '0.000')] [2023-10-13 23:21:33,043][60934] Updated weights for policy 1, policy_version 58682 (0.0009) [2023-10-13 23:21:33,416][60934] Updated weights for policy 1, policy_version 58692 (0.0008) [2023-10-13 23:21:33,784][60934] Updated weights for policy 1, policy_version 58702 (0.0008) [2023-10-13 23:21:34,151][60934] Updated weights for policy 1, policy_version 58712 (0.0008) [2023-10-13 23:21:35,029][60935] Updated weights for policy 0, policy_version 58280 (0.0010) [2023-10-13 23:21:35,396][60935] Updated weights for policy 0, policy_version 58290 (0.0008) [2023-10-13 23:21:35,765][60935] Updated weights for policy 0, policy_version 58300 (0.0009) [2023-10-13 23:21:36,248][59943] Fps is (10 sec: 16384.2, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 120127488. Throughput: 0: 1702.6, 1: 1682.3. Samples: 30035308. Policy #0 lag: (min: 16.0, avg: 34.5, max: 48.0) [2023-10-13 23:21:36,249][59943] Avg episode reward: [(0, '-0.120'), (1, '0.000')] [2023-10-13 23:21:38,351][60934] Updated weights for policy 1, policy_version 58722 (0.0010) [2023-10-13 23:21:38,718][60934] Updated weights for policy 1, policy_version 58732 (0.0009) [2023-10-13 23:21:39,097][60934] Updated weights for policy 1, policy_version 58742 (0.0007) [2023-10-13 23:21:39,857][60935] Updated weights for policy 0, policy_version 58310 (0.0009) [2023-10-13 23:21:40,227][60935] Updated weights for policy 0, policy_version 58320 (0.0010) [2023-10-13 23:21:40,598][60935] Updated weights for policy 0, policy_version 58330 (0.0011) [2023-10-13 23:21:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 120193024. Throughput: 0: 1673.8, 1: 1696.3. Samples: 30054968. Policy #0 lag: (min: 16.0, avg: 34.5, max: 48.0) [2023-10-13 23:21:41,249][59943] Avg episode reward: [(0, '-0.120'), (1, '0.000')] [2023-10-13 23:21:43,066][60934] Updated weights for policy 1, policy_version 58752 (0.0007) [2023-10-13 23:21:43,426][60934] Updated weights for policy 1, policy_version 58762 (0.0007) [2023-10-13 23:21:43,792][60934] Updated weights for policy 1, policy_version 58772 (0.0008) [2023-10-13 23:21:44,560][60935] Updated weights for policy 0, policy_version 58340 (0.0010) [2023-10-13 23:21:44,921][60935] Updated weights for policy 0, policy_version 58350 (0.0010) [2023-10-13 23:21:45,288][60935] Updated weights for policy 0, policy_version 58360 (0.0012) [2023-10-13 23:21:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 120258560. Throughput: 0: 1711.7, 1: 1681.7. Samples: 30066286. Policy #0 lag: (min: 16.0, avg: 34.5, max: 48.0) [2023-10-13 23:21:46,249][59943] Avg episode reward: [(0, '-0.120'), (1, '0.000')] [2023-10-13 23:21:47,735][60934] Updated weights for policy 1, policy_version 58782 (0.0008) [2023-10-13 23:21:48,100][60934] Updated weights for policy 1, policy_version 58792 (0.0010) [2023-10-13 23:21:48,466][60934] Updated weights for policy 1, policy_version 58802 (0.0010) [2023-10-13 23:21:49,392][60935] Updated weights for policy 0, policy_version 58370 (0.0011) [2023-10-13 23:21:49,795][60935] Updated weights for policy 0, policy_version 58380 (0.0008) [2023-10-13 23:21:50,162][60935] Updated weights for policy 0, policy_version 58390 (0.0007) [2023-10-13 23:21:50,531][60935] Updated weights for policy 0, policy_version 58400 (0.0008) [2023-10-13 23:21:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 120324096. Throughput: 0: 1691.3, 1: 1679.8. Samples: 30085852. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 23:21:51,249][59943] Avg episode reward: [(0, '-0.120'), (1, '0.000')] [2023-10-13 23:21:52,498][60934] Updated weights for policy 1, policy_version 58812 (0.0007) [2023-10-13 23:21:52,858][60934] Updated weights for policy 1, policy_version 58822 (0.0007) [2023-10-13 23:21:53,229][60934] Updated weights for policy 1, policy_version 58832 (0.0009) [2023-10-13 23:21:54,563][60935] Updated weights for policy 0, policy_version 58410 (0.0010) [2023-10-13 23:21:54,945][60935] Updated weights for policy 0, policy_version 58420 (0.0011) [2023-10-13 23:21:55,316][60935] Updated weights for policy 0, policy_version 58430 (0.0010) [2023-10-13 23:21:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 120389632. Throughput: 0: 1675.7, 1: 1705.3. Samples: 30105984. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 23:21:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:21:57,417][60934] Updated weights for policy 1, policy_version 58842 (0.0008) [2023-10-13 23:21:57,783][60934] Updated weights for policy 1, policy_version 58852 (0.0009) [2023-10-13 23:21:58,143][60934] Updated weights for policy 1, policy_version 58862 (0.0009) [2023-10-13 23:21:58,519][60934] Updated weights for policy 1, policy_version 58872 (0.0009) [2023-10-13 23:21:59,531][60935] Updated weights for policy 0, policy_version 58440 (0.0007) [2023-10-13 23:21:59,898][60935] Updated weights for policy 0, policy_version 58450 (0.0007) [2023-10-13 23:22:00,269][60935] Updated weights for policy 0, policy_version 58460 (0.0008) [2023-10-13 23:22:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 120455168. Throughput: 0: 1704.4, 1: 1672.3. Samples: 30116418. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 23:22:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:22:02,353][60934] Updated weights for policy 1, policy_version 58882 (0.0008) [2023-10-13 23:22:02,723][60934] Updated weights for policy 1, policy_version 58892 (0.0009) [2023-10-13 23:22:03,086][60934] Updated weights for policy 1, policy_version 58902 (0.0009) [2023-10-13 23:22:04,379][60935] Updated weights for policy 0, policy_version 58470 (0.0008) [2023-10-13 23:22:04,739][60935] Updated weights for policy 0, policy_version 58480 (0.0007) [2023-10-13 23:22:05,113][60935] Updated weights for policy 0, policy_version 58490 (0.0008) [2023-10-13 23:22:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 120520704. Throughput: 0: 1687.2, 1: 1700.7. Samples: 30136766. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 23:22:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:22:07,151][60934] Updated weights for policy 1, policy_version 58912 (0.0009) [2023-10-13 23:22:07,529][60934] Updated weights for policy 1, policy_version 58922 (0.0009) [2023-10-13 23:22:07,890][60934] Updated weights for policy 1, policy_version 58932 (0.0008) [2023-10-13 23:22:09,133][60935] Updated weights for policy 0, policy_version 58500 (0.0009) [2023-10-13 23:22:09,500][60935] Updated weights for policy 0, policy_version 58510 (0.0008) [2023-10-13 23:22:09,871][60935] Updated weights for policy 0, policy_version 58520 (0.0007) [2023-10-13 23:22:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 120586240. Throughput: 0: 1689.9, 1: 1700.3. Samples: 30157204. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 23:22:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:22:11,947][60934] Updated weights for policy 1, policy_version 58942 (0.0009) [2023-10-13 23:22:12,303][60934] Updated weights for policy 1, policy_version 58952 (0.0010) [2023-10-13 23:22:12,671][60934] Updated weights for policy 1, policy_version 58962 (0.0010) [2023-10-13 23:22:13,903][60935] Updated weights for policy 0, policy_version 58530 (0.0008) [2023-10-13 23:22:14,267][60935] Updated weights for policy 0, policy_version 58540 (0.0008) [2023-10-13 23:22:14,645][60935] Updated weights for policy 0, policy_version 58550 (0.0007) [2023-10-13 23:22:15,018][60935] Updated weights for policy 0, policy_version 58560 (0.0008) [2023-10-13 23:22:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 120651776. Throughput: 0: 1700.5, 1: 1684.3. Samples: 30167648. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 23:22:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:22:16,772][60934] Updated weights for policy 1, policy_version 58972 (0.0010) [2023-10-13 23:22:17,141][60934] Updated weights for policy 1, policy_version 58982 (0.0009) [2023-10-13 23:22:17,514][60934] Updated weights for policy 1, policy_version 58992 (0.0009) [2023-10-13 23:22:18,909][60935] Updated weights for policy 0, policy_version 58570 (0.0007) [2023-10-13 23:22:19,283][60935] Updated weights for policy 0, policy_version 58580 (0.0010) [2023-10-13 23:22:19,647][60935] Updated weights for policy 0, policy_version 58590 (0.0008) [2023-10-13 23:22:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 120717312. Throughput: 0: 1674.0, 1: 1706.6. Samples: 30187434. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 23:22:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:22:21,580][60934] Updated weights for policy 1, policy_version 59002 (0.0007) [2023-10-13 23:22:21,943][60934] Updated weights for policy 1, policy_version 59012 (0.0009) [2023-10-13 23:22:22,309][60934] Updated weights for policy 1, policy_version 59022 (0.0009) [2023-10-13 23:22:22,679][60934] Updated weights for policy 1, policy_version 59032 (0.0009) [2023-10-13 23:22:23,679][60935] Updated weights for policy 0, policy_version 58600 (0.0007) [2023-10-13 23:22:24,052][60935] Updated weights for policy 0, policy_version 58610 (0.0008) [2023-10-13 23:22:24,423][60935] Updated weights for policy 0, policy_version 58620 (0.0010) [2023-10-13 23:22:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 120782848. Throughput: 0: 1698.8, 1: 1705.7. Samples: 30208170. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 23:22:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:22:26,870][60934] Updated weights for policy 1, policy_version 59042 (0.0009) [2023-10-13 23:22:27,234][60934] Updated weights for policy 1, policy_version 59052 (0.0010) [2023-10-13 23:22:27,603][60934] Updated weights for policy 1, policy_version 59062 (0.0007) [2023-10-13 23:22:28,249][60935] Updated weights for policy 0, policy_version 58630 (0.0010) [2023-10-13 23:22:28,620][60935] Updated weights for policy 0, policy_version 58640 (0.0011) [2023-10-13 23:22:28,998][60935] Updated weights for policy 0, policy_version 58650 (0.0010) [2023-10-13 23:22:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 120848384. Throughput: 0: 1674.8, 1: 1690.4. Samples: 30217716. Policy #0 lag: (min: 26.0, avg: 31.9, max: 58.0) [2023-10-13 23:22:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:22:31,623][60934] Updated weights for policy 1, policy_version 59072 (0.0009) [2023-10-13 23:22:31,987][60934] Updated weights for policy 1, policy_version 59082 (0.0008) [2023-10-13 23:22:32,361][60934] Updated weights for policy 1, policy_version 59092 (0.0010) [2023-10-13 23:22:32,955][60935] Updated weights for policy 0, policy_version 58660 (0.0009) [2023-10-13 23:22:33,323][60935] Updated weights for policy 0, policy_version 58670 (0.0007) [2023-10-13 23:22:33,689][60935] Updated weights for policy 0, policy_version 58680 (0.0007) [2023-10-13 23:22:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 120913920. Throughput: 0: 1689.5, 1: 1699.1. Samples: 30238340. Policy #0 lag: (min: 26.0, avg: 31.9, max: 58.0) [2023-10-13 23:22:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:22:36,355][60934] Updated weights for policy 1, policy_version 59102 (0.0008) [2023-10-13 23:22:36,715][60934] Updated weights for policy 1, policy_version 59112 (0.0009) [2023-10-13 23:22:37,084][60934] Updated weights for policy 1, policy_version 59122 (0.0010) [2023-10-13 23:22:37,651][60935] Updated weights for policy 0, policy_version 58690 (0.0008) [2023-10-13 23:22:38,043][60935] Updated weights for policy 0, policy_version 58700 (0.0008) [2023-10-13 23:22:38,406][60935] Updated weights for policy 0, policy_version 58710 (0.0011) [2023-10-13 23:22:38,787][60935] Updated weights for policy 0, policy_version 58720 (0.0009) [2023-10-13 23:22:41,018][60934] Updated weights for policy 1, policy_version 59132 (0.0008) [2023-10-13 23:22:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 120979456. Throughput: 0: 1708.8, 1: 1698.3. Samples: 30259306. Policy #0 lag: (min: 26.0, avg: 31.9, max: 58.0) [2023-10-13 23:22:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:22:41,390][60934] Updated weights for policy 1, policy_version 59142 (0.0008) [2023-10-13 23:22:41,756][60934] Updated weights for policy 1, policy_version 59152 (0.0008) [2023-10-13 23:22:42,785][60935] Updated weights for policy 0, policy_version 58730 (0.0007) [2023-10-13 23:22:43,155][60935] Updated weights for policy 0, policy_version 58740 (0.0009) [2023-10-13 23:22:43,525][60935] Updated weights for policy 0, policy_version 58750 (0.0009) [2023-10-13 23:22:45,991][60934] Updated weights for policy 1, policy_version 59162 (0.0011) [2023-10-13 23:22:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 121044992. Throughput: 0: 1680.4, 1: 1698.1. Samples: 30268450. Policy #0 lag: (min: 26.0, avg: 31.9, max: 58.0) [2023-10-13 23:22:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:22:46,359][60934] Updated weights for policy 1, policy_version 59172 (0.0007) [2023-10-13 23:22:46,730][60934] Updated weights for policy 1, policy_version 59182 (0.0008) [2023-10-13 23:22:47,096][60934] Updated weights for policy 1, policy_version 59192 (0.0008) [2023-10-13 23:22:47,701][60935] Updated weights for policy 0, policy_version 58760 (0.0010) [2023-10-13 23:22:48,074][60935] Updated weights for policy 0, policy_version 58770 (0.0011) [2023-10-13 23:22:48,436][60935] Updated weights for policy 0, policy_version 58780 (0.0011) [2023-10-13 23:22:50,970][60934] Updated weights for policy 1, policy_version 59202 (0.0008) [2023-10-13 23:22:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 121110528. Throughput: 0: 1694.6, 1: 1693.2. Samples: 30289218. Policy #0 lag: (min: 26.0, avg: 31.9, max: 58.0) [2023-10-13 23:22:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:22:51,334][60934] Updated weights for policy 1, policy_version 59212 (0.0008) [2023-10-13 23:22:51,709][60934] Updated weights for policy 1, policy_version 59222 (0.0007) [2023-10-13 23:22:52,461][60935] Updated weights for policy 0, policy_version 58790 (0.0011) [2023-10-13 23:22:52,837][60935] Updated weights for policy 0, policy_version 58800 (0.0008) [2023-10-13 23:22:53,199][60935] Updated weights for policy 0, policy_version 58810 (0.0010) [2023-10-13 23:22:55,632][60934] Updated weights for policy 1, policy_version 59232 (0.0008) [2023-10-13 23:22:56,007][60934] Updated weights for policy 1, policy_version 59242 (0.0007) [2023-10-13 23:22:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 121176064. Throughput: 0: 1709.5, 1: 1690.0. Samples: 30310182. Policy #0 lag: (min: 26.0, avg: 31.9, max: 58.0) [2023-10-13 23:22:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:22:56,258][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000058816_60227584.pth... [2023-10-13 23:22:56,295][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000057248_58621952.pth [2023-10-13 23:22:56,380][60934] Updated weights for policy 1, policy_version 59252 (0.0008) [2023-10-13 23:22:56,521][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000059256_60981248.pth... [2023-10-13 23:22:56,561][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000057656_59342848.pth [2023-10-13 23:22:57,230][60935] Updated weights for policy 0, policy_version 58820 (0.0009) [2023-10-13 23:22:57,600][60935] Updated weights for policy 0, policy_version 58830 (0.0011) [2023-10-13 23:22:57,982][60935] Updated weights for policy 0, policy_version 58840 (0.0008) [2023-10-13 23:23:00,433][60934] Updated weights for policy 1, policy_version 59262 (0.0008) [2023-10-13 23:23:00,791][60934] Updated weights for policy 1, policy_version 59272 (0.0008) [2023-10-13 23:23:01,165][60934] Updated weights for policy 1, policy_version 59282 (0.0007) [2023-10-13 23:23:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 121241600. Throughput: 0: 1680.8, 1: 1695.0. Samples: 30319562. Policy #0 lag: (min: 26.0, avg: 31.9, max: 58.0) [2023-10-13 23:23:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:23:01,927][60935] Updated weights for policy 0, policy_version 58850 (0.0008) [2023-10-13 23:23:02,296][60935] Updated weights for policy 0, policy_version 58860 (0.0008) [2023-10-13 23:23:02,669][60935] Updated weights for policy 0, policy_version 58870 (0.0008) [2023-10-13 23:23:03,052][60935] Updated weights for policy 0, policy_version 58880 (0.0010) [2023-10-13 23:23:05,204][60934] Updated weights for policy 1, policy_version 59292 (0.0007) [2023-10-13 23:23:05,565][60934] Updated weights for policy 1, policy_version 59302 (0.0009) [2023-10-13 23:23:05,935][60934] Updated weights for policy 1, policy_version 59312 (0.0008) [2023-10-13 23:23:06,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 121339904. Throughput: 0: 1708.7, 1: 1694.2. Samples: 30340566. Policy #0 lag: (min: 26.0, avg: 31.9, max: 58.0) [2023-10-13 23:23:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:23:07,151][60935] Updated weights for policy 0, policy_version 58890 (0.0008) [2023-10-13 23:23:07,521][60935] Updated weights for policy 0, policy_version 58900 (0.0008) [2023-10-13 23:23:07,897][60935] Updated weights for policy 0, policy_version 58910 (0.0009) [2023-10-13 23:23:09,931][60934] Updated weights for policy 1, policy_version 59322 (0.0009) [2023-10-13 23:23:10,293][60934] Updated weights for policy 1, policy_version 59332 (0.0008) [2023-10-13 23:23:10,670][60934] Updated weights for policy 1, policy_version 59342 (0.0010) [2023-10-13 23:23:11,028][60934] Updated weights for policy 1, policy_version 59352 (0.0010) [2023-10-13 23:23:11,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 121405440. Throughput: 0: 1708.6, 1: 1681.0. Samples: 30360702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:23:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:23:11,788][60935] Updated weights for policy 0, policy_version 58920 (0.0010) [2023-10-13 23:23:12,158][60935] Updated weights for policy 0, policy_version 58930 (0.0010) [2023-10-13 23:23:12,532][60935] Updated weights for policy 0, policy_version 58940 (0.0009) [2023-10-13 23:23:15,269][60934] Updated weights for policy 1, policy_version 59362 (0.0007) [2023-10-13 23:23:15,637][60934] Updated weights for policy 1, policy_version 59372 (0.0008) [2023-10-13 23:23:15,999][60934] Updated weights for policy 1, policy_version 59382 (0.0010) [2023-10-13 23:23:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 121470976. Throughput: 0: 1695.6, 1: 1697.4. Samples: 30370402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:23:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:23:16,486][60935] Updated weights for policy 0, policy_version 58950 (0.0007) [2023-10-13 23:23:16,842][60935] Updated weights for policy 0, policy_version 58960 (0.0010) [2023-10-13 23:23:17,225][60935] Updated weights for policy 0, policy_version 58970 (0.0008) [2023-10-13 23:23:20,116][60934] Updated weights for policy 1, policy_version 59392 (0.0008) [2023-10-13 23:23:20,481][60934] Updated weights for policy 1, policy_version 59402 (0.0008) [2023-10-13 23:23:20,845][60934] Updated weights for policy 1, policy_version 59412 (0.0007) [2023-10-13 23:23:21,151][60935] Updated weights for policy 0, policy_version 58980 (0.0008) [2023-10-13 23:23:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 121536512. Throughput: 0: 1705.9, 1: 1689.4. Samples: 30391130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:23:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:23:21,530][60935] Updated weights for policy 0, policy_version 58990 (0.0009) [2023-10-13 23:23:21,899][60935] Updated weights for policy 0, policy_version 59000 (0.0010) [2023-10-13 23:23:24,792][60934] Updated weights for policy 1, policy_version 59422 (0.0008) [2023-10-13 23:23:25,156][60934] Updated weights for policy 1, policy_version 59432 (0.0007) [2023-10-13 23:23:25,518][60934] Updated weights for policy 1, policy_version 59442 (0.0009) [2023-10-13 23:23:26,008][60935] Updated weights for policy 0, policy_version 59010 (0.0008) [2023-10-13 23:23:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 121602048. Throughput: 0: 1710.8, 1: 1667.5. Samples: 30411330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:23:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:23:26,398][60935] Updated weights for policy 0, policy_version 59020 (0.0008) [2023-10-13 23:23:26,775][60935] Updated weights for policy 0, policy_version 59030 (0.0008) [2023-10-13 23:23:27,138][60935] Updated weights for policy 0, policy_version 59040 (0.0010) [2023-10-13 23:23:29,516][60934] Updated weights for policy 1, policy_version 59452 (0.0009) [2023-10-13 23:23:29,881][60934] Updated weights for policy 1, policy_version 59462 (0.0007) [2023-10-13 23:23:30,248][60934] Updated weights for policy 1, policy_version 59472 (0.0007) [2023-10-13 23:23:31,222][60935] Updated weights for policy 0, policy_version 59050 (0.0007) [2023-10-13 23:23:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 121667584. Throughput: 0: 1705.5, 1: 1690.0. Samples: 30421250. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:23:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:23:31,586][60935] Updated weights for policy 0, policy_version 59060 (0.0008) [2023-10-13 23:23:31,944][60935] Updated weights for policy 0, policy_version 59070 (0.0010) [2023-10-13 23:23:34,370][60934] Updated weights for policy 1, policy_version 59482 (0.0007) [2023-10-13 23:23:34,738][60934] Updated weights for policy 1, policy_version 59492 (0.0009) [2023-10-13 23:23:35,114][60934] Updated weights for policy 1, policy_version 59502 (0.0007) [2023-10-13 23:23:35,476][60934] Updated weights for policy 1, policy_version 59512 (0.0010) [2023-10-13 23:23:35,909][60935] Updated weights for policy 0, policy_version 59080 (0.0008) [2023-10-13 23:23:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 121733120. Throughput: 0: 1709.9, 1: 1683.3. Samples: 30441914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:23:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:23:36,288][60935] Updated weights for policy 0, policy_version 59090 (0.0007) [2023-10-13 23:23:36,662][60935] Updated weights for policy 0, policy_version 59100 (0.0008) [2023-10-13 23:23:39,539][60934] Updated weights for policy 1, policy_version 59522 (0.0010) [2023-10-13 23:23:39,910][60934] Updated weights for policy 1, policy_version 59532 (0.0008) [2023-10-13 23:23:40,277][60934] Updated weights for policy 1, policy_version 59542 (0.0008) [2023-10-13 23:23:40,669][60935] Updated weights for policy 0, policy_version 59110 (0.0010) [2023-10-13 23:23:41,025][60935] Updated weights for policy 0, policy_version 59120 (0.0007) [2023-10-13 23:23:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 121798656. Throughput: 0: 1696.0, 1: 1660.9. Samples: 30461242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:23:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:23:41,394][60935] Updated weights for policy 0, policy_version 59130 (0.0008) [2023-10-13 23:23:44,471][60934] Updated weights for policy 1, policy_version 59552 (0.0008) [2023-10-13 23:23:44,838][60934] Updated weights for policy 1, policy_version 59562 (0.0008) [2023-10-13 23:23:45,205][60934] Updated weights for policy 1, policy_version 59572 (0.0007) [2023-10-13 23:23:45,289][60935] Updated weights for policy 0, policy_version 59140 (0.0009) [2023-10-13 23:23:45,653][60935] Updated weights for policy 0, policy_version 59150 (0.0008) [2023-10-13 23:23:46,017][60935] Updated weights for policy 0, policy_version 59160 (0.0008) [2023-10-13 23:23:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 121864192. Throughput: 0: 1711.3, 1: 1679.8. Samples: 30472164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:23:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:23:49,212][60934] Updated weights for policy 1, policy_version 59582 (0.0010) [2023-10-13 23:23:49,578][60934] Updated weights for policy 1, policy_version 59592 (0.0008) [2023-10-13 23:23:49,945][60934] Updated weights for policy 1, policy_version 59602 (0.0007) [2023-10-13 23:23:50,065][60935] Updated weights for policy 0, policy_version 59170 (0.0008) [2023-10-13 23:23:50,433][60935] Updated weights for policy 0, policy_version 59180 (0.0008) [2023-10-13 23:23:50,796][60935] Updated weights for policy 0, policy_version 59190 (0.0008) [2023-10-13 23:23:51,171][60935] Updated weights for policy 0, policy_version 59200 (0.0007) [2023-10-13 23:23:51,248][59943] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 121962496. Throughput: 0: 1711.4, 1: 1666.3. Samples: 30492564. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-10-13 23:23:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:23:54,069][60934] Updated weights for policy 1, policy_version 59612 (0.0007) [2023-10-13 23:23:54,433][60934] Updated weights for policy 1, policy_version 59622 (0.0008) [2023-10-13 23:23:54,803][60934] Updated weights for policy 1, policy_version 59632 (0.0008) [2023-10-13 23:23:55,212][60935] Updated weights for policy 0, policy_version 59210 (0.0007) [2023-10-13 23:23:55,589][60935] Updated weights for policy 0, policy_version 59220 (0.0009) [2023-10-13 23:23:55,969][60935] Updated weights for policy 0, policy_version 59230 (0.0009) [2023-10-13 23:23:56,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 122028032. Throughput: 0: 1687.0, 1: 1667.3. Samples: 30511648. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-10-13 23:23:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:23:58,857][60934] Updated weights for policy 1, policy_version 59642 (0.0008) [2023-10-13 23:23:59,231][60934] Updated weights for policy 1, policy_version 59652 (0.0007) [2023-10-13 23:23:59,594][60934] Updated weights for policy 1, policy_version 59662 (0.0007) [2023-10-13 23:23:59,937][60935] Updated weights for policy 0, policy_version 59240 (0.0010) [2023-10-13 23:23:59,958][60934] Updated weights for policy 1, policy_version 59672 (0.0007) [2023-10-13 23:24:00,306][60935] Updated weights for policy 0, policy_version 59250 (0.0009) [2023-10-13 23:24:00,668][60935] Updated weights for policy 0, policy_version 59260 (0.0009) [2023-10-13 23:24:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 122093568. Throughput: 0: 1708.2, 1: 1687.1. Samples: 30523190. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-10-13 23:24:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:24:03,959][60934] Updated weights for policy 1, policy_version 59682 (0.0007) [2023-10-13 23:24:04,330][60934] Updated weights for policy 1, policy_version 59692 (0.0007) [2023-10-13 23:24:04,645][60935] Updated weights for policy 0, policy_version 59270 (0.0009) [2023-10-13 23:24:04,693][60934] Updated weights for policy 1, policy_version 59702 (0.0008) [2023-10-13 23:24:05,011][60935] Updated weights for policy 0, policy_version 59280 (0.0007) [2023-10-13 23:24:05,372][60935] Updated weights for policy 0, policy_version 59290 (0.0009) [2023-10-13 23:24:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 122159104. Throughput: 0: 1697.3, 1: 1676.6. Samples: 30542958. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-10-13 23:24:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:24:08,647][60934] Updated weights for policy 1, policy_version 59712 (0.0007) [2023-10-13 23:24:09,015][60934] Updated weights for policy 1, policy_version 59722 (0.0008) [2023-10-13 23:24:09,386][60934] Updated weights for policy 1, policy_version 59732 (0.0009) [2023-10-13 23:24:09,392][60935] Updated weights for policy 0, policy_version 59300 (0.0007) [2023-10-13 23:24:09,754][60935] Updated weights for policy 0, policy_version 59310 (0.0007) [2023-10-13 23:24:10,133][60935] Updated weights for policy 0, policy_version 59320 (0.0010) [2023-10-13 23:24:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 122224640. Throughput: 0: 1676.8, 1: 1692.4. Samples: 30562946. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-10-13 23:24:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:24:13,306][60934] Updated weights for policy 1, policy_version 59742 (0.0009) [2023-10-13 23:24:13,663][60934] Updated weights for policy 1, policy_version 59752 (0.0010) [2023-10-13 23:24:14,039][60934] Updated weights for policy 1, policy_version 59762 (0.0007) [2023-10-13 23:24:14,079][60935] Updated weights for policy 0, policy_version 59330 (0.0009) [2023-10-13 23:24:14,448][60935] Updated weights for policy 0, policy_version 59340 (0.0008) [2023-10-13 23:24:14,825][60935] Updated weights for policy 0, policy_version 59350 (0.0010) [2023-10-13 23:24:15,191][60935] Updated weights for policy 0, policy_version 59360 (0.0007) [2023-10-13 23:24:16,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 122290176. Throughput: 0: 1711.0, 1: 1687.6. Samples: 30574190. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-10-13 23:24:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:24:18,176][60934] Updated weights for policy 1, policy_version 59772 (0.0009) [2023-10-13 23:24:18,555][60934] Updated weights for policy 1, policy_version 59782 (0.0011) [2023-10-13 23:24:18,917][60934] Updated weights for policy 1, policy_version 59792 (0.0010) [2023-10-13 23:24:19,325][60935] Updated weights for policy 0, policy_version 59370 (0.0009) [2023-10-13 23:24:19,697][60935] Updated weights for policy 0, policy_version 59380 (0.0009) [2023-10-13 23:24:20,068][60935] Updated weights for policy 0, policy_version 59390 (0.0009) [2023-10-13 23:24:21,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 122355712. Throughput: 0: 1688.9, 1: 1676.1. Samples: 30593338. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-10-13 23:24:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:24:23,222][60934] Updated weights for policy 1, policy_version 59802 (0.0009) [2023-10-13 23:24:23,598][60934] Updated weights for policy 1, policy_version 59812 (0.0008) [2023-10-13 23:24:23,932][60935] Updated weights for policy 0, policy_version 59400 (0.0007) [2023-10-13 23:24:23,956][60934] Updated weights for policy 1, policy_version 59822 (0.0008) [2023-10-13 23:24:24,299][60935] Updated weights for policy 0, policy_version 59410 (0.0008) [2023-10-13 23:24:24,327][60934] Updated weights for policy 1, policy_version 59832 (0.0008) [2023-10-13 23:24:24,667][60935] Updated weights for policy 0, policy_version 59420 (0.0007) [2023-10-13 23:24:26,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 122421248. Throughput: 0: 1693.0, 1: 1696.8. Samples: 30613782. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-10-13 23:24:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:24:28,333][60934] Updated weights for policy 1, policy_version 59842 (0.0010) [2023-10-13 23:24:28,663][60935] Updated weights for policy 0, policy_version 59430 (0.0009) [2023-10-13 23:24:28,703][60934] Updated weights for policy 1, policy_version 59852 (0.0008) [2023-10-13 23:24:29,026][60935] Updated weights for policy 0, policy_version 59440 (0.0009) [2023-10-13 23:24:29,069][60934] Updated weights for policy 1, policy_version 59862 (0.0007) [2023-10-13 23:24:29,392][60935] Updated weights for policy 0, policy_version 59450 (0.0010) [2023-10-13 23:24:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 122486784. Throughput: 0: 1697.5, 1: 1685.1. Samples: 30624382. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-10-13 23:24:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:24:32,976][60934] Updated weights for policy 1, policy_version 59872 (0.0008) [2023-10-13 23:24:33,341][60934] Updated weights for policy 1, policy_version 59882 (0.0007) [2023-10-13 23:24:33,526][60935] Updated weights for policy 0, policy_version 59460 (0.0009) [2023-10-13 23:24:33,706][60934] Updated weights for policy 1, policy_version 59892 (0.0008) [2023-10-13 23:24:33,900][60935] Updated weights for policy 0, policy_version 59470 (0.0009) [2023-10-13 23:24:34,278][60935] Updated weights for policy 0, policy_version 59480 (0.0010) [2023-10-13 23:24:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 122552320. Throughput: 0: 1677.7, 1: 1685.1. Samples: 30643890. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-10-13 23:24:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:24:37,934][60934] Updated weights for policy 1, policy_version 59902 (0.0007) [2023-10-13 23:24:38,304][60934] Updated weights for policy 1, policy_version 59912 (0.0008) [2023-10-13 23:24:38,371][60935] Updated weights for policy 0, policy_version 59490 (0.0012) [2023-10-13 23:24:38,667][60934] Updated weights for policy 1, policy_version 59922 (0.0007) [2023-10-13 23:24:38,738][60935] Updated weights for policy 0, policy_version 59500 (0.0008) [2023-10-13 23:24:39,104][60935] Updated weights for policy 0, policy_version 59510 (0.0008) [2023-10-13 23:24:39,469][60935] Updated weights for policy 0, policy_version 59520 (0.0009) [2023-10-13 23:24:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 122617856. Throughput: 0: 1705.2, 1: 1699.0. Samples: 30664840. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-10-13 23:24:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:24:42,897][60934] Updated weights for policy 1, policy_version 59932 (0.0008) [2023-10-13 23:24:43,276][60934] Updated weights for policy 1, policy_version 59942 (0.0008) [2023-10-13 23:24:43,474][60935] Updated weights for policy 0, policy_version 59530 (0.0009) [2023-10-13 23:24:43,644][60934] Updated weights for policy 1, policy_version 59952 (0.0008) [2023-10-13 23:24:43,845][60935] Updated weights for policy 0, policy_version 59540 (0.0010) [2023-10-13 23:24:44,214][60935] Updated weights for policy 0, policy_version 59550 (0.0009) [2023-10-13 23:24:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 122683392. Throughput: 0: 1697.6, 1: 1672.4. Samples: 30674844. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-10-13 23:24:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:24:47,689][60934] Updated weights for policy 1, policy_version 59962 (0.0007) [2023-10-13 23:24:48,046][60934] Updated weights for policy 1, policy_version 59972 (0.0009) [2023-10-13 23:24:48,142][60935] Updated weights for policy 0, policy_version 59560 (0.0008) [2023-10-13 23:24:48,413][60934] Updated weights for policy 1, policy_version 59982 (0.0007) [2023-10-13 23:24:48,515][60935] Updated weights for policy 0, policy_version 59570 (0.0008) [2023-10-13 23:24:48,776][60934] Updated weights for policy 1, policy_version 59992 (0.0007) [2023-10-13 23:24:48,883][60935] Updated weights for policy 0, policy_version 59580 (0.0008) [2023-10-13 23:24:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 122748928. Throughput: 0: 1699.1, 1: 1678.8. Samples: 30694964. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-10-13 23:24:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:24:52,822][60935] Updated weights for policy 0, policy_version 59590 (0.0008) [2023-10-13 23:24:52,943][60934] Updated weights for policy 1, policy_version 60002 (0.0008) [2023-10-13 23:24:53,188][60935] Updated weights for policy 0, policy_version 59600 (0.0009) [2023-10-13 23:24:53,315][60934] Updated weights for policy 1, policy_version 60012 (0.0008) [2023-10-13 23:24:53,549][60935] Updated weights for policy 0, policy_version 59610 (0.0009) [2023-10-13 23:24:53,680][60934] Updated weights for policy 1, policy_version 60022 (0.0008) [2023-10-13 23:24:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 122814464. Throughput: 0: 1718.8, 1: 1676.1. Samples: 30715718. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-10-13 23:24:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:24:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000059616_61046784.pth... [2023-10-13 23:24:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000060024_61767680.pth... [2023-10-13 23:24:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000058048_59441152.pth [2023-10-13 23:24:56,305][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000058456_60162048.pth [2023-10-13 23:24:57,437][60934] Updated weights for policy 1, policy_version 60032 (0.0008) [2023-10-13 23:24:57,718][60935] Updated weights for policy 0, policy_version 59620 (0.0009) [2023-10-13 23:24:57,813][60934] Updated weights for policy 1, policy_version 60042 (0.0007) [2023-10-13 23:24:58,080][60935] Updated weights for policy 0, policy_version 59630 (0.0009) [2023-10-13 23:24:58,179][60934] Updated weights for policy 1, policy_version 60052 (0.0008) [2023-10-13 23:24:58,454][60935] Updated weights for policy 0, policy_version 59640 (0.0009) [2023-10-13 23:25:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 122880000. Throughput: 0: 1688.7, 1: 1661.2. Samples: 30724936. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-10-13 23:25:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:25:02,045][60934] Updated weights for policy 1, policy_version 60062 (0.0008) [2023-10-13 23:25:02,404][60934] Updated weights for policy 1, policy_version 60072 (0.0009) [2023-10-13 23:25:02,516][60935] Updated weights for policy 0, policy_version 59650 (0.0009) [2023-10-13 23:25:02,767][60934] Updated weights for policy 1, policy_version 60082 (0.0008) [2023-10-13 23:25:02,880][60935] Updated weights for policy 0, policy_version 59660 (0.0009) [2023-10-13 23:25:03,246][60935] Updated weights for policy 0, policy_version 59670 (0.0008) [2023-10-13 23:25:03,609][60935] Updated weights for policy 0, policy_version 59680 (0.0008) [2023-10-13 23:25:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 122945536. Throughput: 0: 1706.4, 1: 1686.9. Samples: 30746036. Policy #0 lag: (min: 19.0, avg: 19.4, max: 32.0) [2023-10-13 23:25:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:25:06,781][60934] Updated weights for policy 1, policy_version 60092 (0.0008) [2023-10-13 23:25:07,146][60934] Updated weights for policy 1, policy_version 60102 (0.0008) [2023-10-13 23:25:07,506][60934] Updated weights for policy 1, policy_version 60112 (0.0007) [2023-10-13 23:25:07,613][60935] Updated weights for policy 0, policy_version 59690 (0.0008) [2023-10-13 23:25:07,986][60935] Updated weights for policy 0, policy_version 59700 (0.0007) [2023-10-13 23:25:08,354][60935] Updated weights for policy 0, policy_version 59710 (0.0008) [2023-10-13 23:25:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 123011072. Throughput: 0: 1715.0, 1: 1691.0. Samples: 30767054. Policy #0 lag: (min: 19.0, avg: 19.4, max: 32.0) [2023-10-13 23:25:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:25:11,635][60934] Updated weights for policy 1, policy_version 60122 (0.0007) [2023-10-13 23:25:11,991][60934] Updated weights for policy 1, policy_version 60132 (0.0008) [2023-10-13 23:25:12,328][60935] Updated weights for policy 0, policy_version 59720 (0.0007) [2023-10-13 23:25:12,362][60934] Updated weights for policy 1, policy_version 60142 (0.0009) [2023-10-13 23:25:12,692][60935] Updated weights for policy 0, policy_version 59730 (0.0010) [2023-10-13 23:25:12,727][60934] Updated weights for policy 1, policy_version 60152 (0.0009) [2023-10-13 23:25:13,065][60935] Updated weights for policy 0, policy_version 59740 (0.0011) [2023-10-13 23:25:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 123076608. Throughput: 0: 1695.3, 1: 1676.2. Samples: 30776100. Policy #0 lag: (min: 19.0, avg: 19.4, max: 32.0) [2023-10-13 23:25:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:25:16,760][60934] Updated weights for policy 1, policy_version 60162 (0.0008) [2023-10-13 23:25:17,072][60935] Updated weights for policy 0, policy_version 59750 (0.0009) [2023-10-13 23:25:17,127][60934] Updated weights for policy 1, policy_version 60172 (0.0009) [2023-10-13 23:25:17,430][60935] Updated weights for policy 0, policy_version 59760 (0.0010) [2023-10-13 23:25:17,490][60934] Updated weights for policy 1, policy_version 60182 (0.0008) [2023-10-13 23:25:17,802][60935] Updated weights for policy 0, policy_version 59770 (0.0008) [2023-10-13 23:25:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 123142144. Throughput: 0: 1715.5, 1: 1688.0. Samples: 30797046. Policy #0 lag: (min: 19.0, avg: 19.4, max: 32.0) [2023-10-13 23:25:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:25:21,542][60934] Updated weights for policy 1, policy_version 60192 (0.0008) [2023-10-13 23:25:21,912][60934] Updated weights for policy 1, policy_version 60202 (0.0008) [2023-10-13 23:25:21,924][60935] Updated weights for policy 0, policy_version 59780 (0.0007) [2023-10-13 23:25:22,268][60934] Updated weights for policy 1, policy_version 60212 (0.0008) [2023-10-13 23:25:22,288][60935] Updated weights for policy 0, policy_version 59790 (0.0009) [2023-10-13 23:25:22,665][60935] Updated weights for policy 0, policy_version 59800 (0.0007) [2023-10-13 23:25:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 123207680. Throughput: 0: 1711.2, 1: 1687.8. Samples: 30817794. Policy #0 lag: (min: 19.0, avg: 19.4, max: 32.0) [2023-10-13 23:25:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:25:26,412][60934] Updated weights for policy 1, policy_version 60222 (0.0009) [2023-10-13 23:25:26,738][60935] Updated weights for policy 0, policy_version 59810 (0.0008) [2023-10-13 23:25:26,785][60934] Updated weights for policy 1, policy_version 60232 (0.0008) [2023-10-13 23:25:27,113][60935] Updated weights for policy 0, policy_version 59820 (0.0008) [2023-10-13 23:25:27,151][60934] Updated weights for policy 1, policy_version 60242 (0.0010) [2023-10-13 23:25:27,479][60935] Updated weights for policy 0, policy_version 59830 (0.0009) [2023-10-13 23:25:27,850][60935] Updated weights for policy 0, policy_version 59840 (0.0009) [2023-10-13 23:25:31,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 123273216. Throughput: 0: 1697.3, 1: 1684.5. Samples: 30827024. Policy #0 lag: (min: 19.0, avg: 19.4, max: 32.0) [2023-10-13 23:25:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:25:31,316][60934] Updated weights for policy 1, policy_version 60252 (0.0008) [2023-10-13 23:25:31,678][60934] Updated weights for policy 1, policy_version 60262 (0.0007) [2023-10-13 23:25:31,840][60935] Updated weights for policy 0, policy_version 59850 (0.0008) [2023-10-13 23:25:32,049][60934] Updated weights for policy 1, policy_version 60272 (0.0008) [2023-10-13 23:25:32,210][60935] Updated weights for policy 0, policy_version 59860 (0.0009) [2023-10-13 23:25:32,592][60935] Updated weights for policy 0, policy_version 59870 (0.0009) [2023-10-13 23:25:36,153][60934] Updated weights for policy 1, policy_version 60282 (0.0008) [2023-10-13 23:25:36,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 123338752. Throughput: 0: 1702.9, 1: 1694.4. Samples: 30847840. Policy #0 lag: (min: 19.0, avg: 19.4, max: 32.0) [2023-10-13 23:25:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:25:36,525][60934] Updated weights for policy 1, policy_version 60292 (0.0009) [2023-10-13 23:25:36,552][60935] Updated weights for policy 0, policy_version 59880 (0.0009) [2023-10-13 23:25:36,880][60934] Updated weights for policy 1, policy_version 60302 (0.0009) [2023-10-13 23:25:36,914][60935] Updated weights for policy 0, policy_version 59890 (0.0007) [2023-10-13 23:25:37,241][60934] Updated weights for policy 1, policy_version 60312 (0.0010) [2023-10-13 23:25:37,282][60935] Updated weights for policy 0, policy_version 59900 (0.0008) [2023-10-13 23:25:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 123404288. Throughput: 0: 1702.6, 1: 1698.9. Samples: 30868788. Policy #0 lag: (min: 19.0, avg: 19.4, max: 32.0) [2023-10-13 23:25:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:25:41,306][60935] Updated weights for policy 0, policy_version 59910 (0.0008) [2023-10-13 23:25:41,478][60934] Updated weights for policy 1, policy_version 60322 (0.0007) [2023-10-13 23:25:41,670][60935] Updated weights for policy 0, policy_version 59920 (0.0010) [2023-10-13 23:25:41,854][60934] Updated weights for policy 1, policy_version 60332 (0.0008) [2023-10-13 23:25:42,038][60935] Updated weights for policy 0, policy_version 59930 (0.0008) [2023-10-13 23:25:42,216][60934] Updated weights for policy 1, policy_version 60342 (0.0010) [2023-10-13 23:25:45,955][60935] Updated weights for policy 0, policy_version 59940 (0.0009) [2023-10-13 23:25:46,065][60934] Updated weights for policy 1, policy_version 60352 (0.0008) [2023-10-13 23:25:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 123469824. Throughput: 0: 1703.1, 1: 1693.8. Samples: 30877794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:25:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:25:46,326][60935] Updated weights for policy 0, policy_version 59950 (0.0008) [2023-10-13 23:25:46,429][60934] Updated weights for policy 1, policy_version 60362 (0.0008) [2023-10-13 23:25:46,695][60935] Updated weights for policy 0, policy_version 59960 (0.0007) [2023-10-13 23:25:46,796][60934] Updated weights for policy 1, policy_version 60372 (0.0009) [2023-10-13 23:25:50,684][60935] Updated weights for policy 0, policy_version 59970 (0.0009) [2023-10-13 23:25:50,928][60934] Updated weights for policy 1, policy_version 60382 (0.0008) [2023-10-13 23:25:51,059][60935] Updated weights for policy 0, policy_version 59980 (0.0007) [2023-10-13 23:25:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 123535360. Throughput: 0: 1705.4, 1: 1686.0. Samples: 30898652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:25:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:25:51,290][60934] Updated weights for policy 1, policy_version 60392 (0.0007) [2023-10-13 23:25:51,422][60935] Updated weights for policy 0, policy_version 59990 (0.0008) [2023-10-13 23:25:51,660][60934] Updated weights for policy 1, policy_version 60402 (0.0007) [2023-10-13 23:25:51,782][60935] Updated weights for policy 0, policy_version 60000 (0.0009) [2023-10-13 23:25:55,465][60934] Updated weights for policy 1, policy_version 60412 (0.0008) [2023-10-13 23:25:55,839][60934] Updated weights for policy 1, policy_version 60422 (0.0008) [2023-10-13 23:25:55,949][60935] Updated weights for policy 0, policy_version 60010 (0.0008) [2023-10-13 23:25:56,198][60934] Updated weights for policy 1, policy_version 60432 (0.0008) [2023-10-13 23:25:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 123600896. Throughput: 0: 1695.5, 1: 1685.1. Samples: 30919180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:25:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:25:56,313][60935] Updated weights for policy 0, policy_version 60020 (0.0007) [2023-10-13 23:25:56,678][60935] Updated weights for policy 0, policy_version 60030 (0.0009) [2023-10-13 23:26:00,066][60934] Updated weights for policy 1, policy_version 60442 (0.0008) [2023-10-13 23:26:00,442][60934] Updated weights for policy 1, policy_version 60452 (0.0008) [2023-10-13 23:26:00,507][60935] Updated weights for policy 0, policy_version 60040 (0.0008) [2023-10-13 23:26:00,809][60934] Updated weights for policy 1, policy_version 60462 (0.0007) [2023-10-13 23:26:00,884][60935] Updated weights for policy 0, policy_version 60050 (0.0008) [2023-10-13 23:26:01,184][60934] Updated weights for policy 1, policy_version 60472 (0.0008) [2023-10-13 23:26:01,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 123699200. Throughput: 0: 1704.5, 1: 1691.9. Samples: 30928938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:26:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:26:01,251][60935] Updated weights for policy 0, policy_version 60060 (0.0009) [2023-10-13 23:26:05,139][60935] Updated weights for policy 0, policy_version 60070 (0.0009) [2023-10-13 23:26:05,445][60934] Updated weights for policy 1, policy_version 60482 (0.0008) [2023-10-13 23:26:05,508][60935] Updated weights for policy 0, policy_version 60080 (0.0009) [2023-10-13 23:26:05,811][60934] Updated weights for policy 1, policy_version 60492 (0.0007) [2023-10-13 23:26:05,881][60935] Updated weights for policy 0, policy_version 60090 (0.0008) [2023-10-13 23:26:06,174][60934] Updated weights for policy 1, policy_version 60502 (0.0009) [2023-10-13 23:26:06,248][59943] Fps is (10 sec: 19660.6, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 123797504. Throughput: 0: 1712.5, 1: 1685.7. Samples: 30949966. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:26:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:26:09,729][60935] Updated weights for policy 0, policy_version 60100 (0.0008) [2023-10-13 23:26:10,089][60935] Updated weights for policy 0, policy_version 60110 (0.0009) [2023-10-13 23:26:10,204][60934] Updated weights for policy 1, policy_version 60512 (0.0008) [2023-10-13 23:26:10,458][60935] Updated weights for policy 0, policy_version 60120 (0.0008) [2023-10-13 23:26:10,575][60934] Updated weights for policy 1, policy_version 60522 (0.0008) [2023-10-13 23:26:10,939][60934] Updated weights for policy 1, policy_version 60532 (0.0007) [2023-10-13 23:26:11,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 123863040. Throughput: 0: 1687.5, 1: 1671.6. Samples: 30968952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:26:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:26:14,507][60935] Updated weights for policy 0, policy_version 60130 (0.0009) [2023-10-13 23:26:14,877][60935] Updated weights for policy 0, policy_version 60140 (0.0009) [2023-10-13 23:26:14,880][60934] Updated weights for policy 1, policy_version 60542 (0.0007) [2023-10-13 23:26:15,238][60934] Updated weights for policy 1, policy_version 60552 (0.0008) [2023-10-13 23:26:15,242][60935] Updated weights for policy 0, policy_version 60150 (0.0008) [2023-10-13 23:26:15,599][60934] Updated weights for policy 1, policy_version 60562 (0.0009) [2023-10-13 23:26:15,600][60935] Updated weights for policy 0, policy_version 60160 (0.0008) [2023-10-13 23:26:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 123928576. Throughput: 0: 1718.5, 1: 1684.2. Samples: 30980148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:26:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:26:19,574][60935] Updated weights for policy 0, policy_version 60170 (0.0007) [2023-10-13 23:26:19,695][60934] Updated weights for policy 1, policy_version 60572 (0.0008) [2023-10-13 23:26:19,933][60935] Updated weights for policy 0, policy_version 60180 (0.0008) [2023-10-13 23:26:20,063][60934] Updated weights for policy 1, policy_version 60582 (0.0007) [2023-10-13 23:26:20,312][60935] Updated weights for policy 0, policy_version 60190 (0.0008) [2023-10-13 23:26:20,429][60934] Updated weights for policy 1, policy_version 60592 (0.0010) [2023-10-13 23:26:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 123994112. Throughput: 0: 1703.9, 1: 1689.7. Samples: 31000556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:26:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:26:24,340][60935] Updated weights for policy 0, policy_version 60200 (0.0010) [2023-10-13 23:26:24,600][60934] Updated weights for policy 1, policy_version 60602 (0.0010) [2023-10-13 23:26:24,714][60935] Updated weights for policy 0, policy_version 60210 (0.0009) [2023-10-13 23:26:24,968][60934] Updated weights for policy 1, policy_version 60612 (0.0007) [2023-10-13 23:26:25,085][60935] Updated weights for policy 0, policy_version 60220 (0.0009) [2023-10-13 23:26:25,329][60934] Updated weights for policy 1, policy_version 60622 (0.0007) [2023-10-13 23:26:25,688][60934] Updated weights for policy 1, policy_version 60632 (0.0007) [2023-10-13 23:26:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 124059648. Throughput: 0: 1688.1, 1: 1665.9. Samples: 31019718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:26:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:26:29,043][60935] Updated weights for policy 0, policy_version 60230 (0.0008) [2023-10-13 23:26:29,411][60935] Updated weights for policy 0, policy_version 60240 (0.0010) [2023-10-13 23:26:29,769][60935] Updated weights for policy 0, policy_version 60250 (0.0009) [2023-10-13 23:26:29,887][60934] Updated weights for policy 1, policy_version 60642 (0.0008) [2023-10-13 23:26:30,250][60934] Updated weights for policy 1, policy_version 60652 (0.0007) [2023-10-13 23:26:30,622][60934] Updated weights for policy 1, policy_version 60662 (0.0008) [2023-10-13 23:26:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 124125184. Throughput: 0: 1719.6, 1: 1689.1. Samples: 31031186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:26:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:26:33,733][60935] Updated weights for policy 0, policy_version 60260 (0.0011) [2023-10-13 23:26:34,103][60935] Updated weights for policy 0, policy_version 60270 (0.0008) [2023-10-13 23:26:34,475][60935] Updated weights for policy 0, policy_version 60280 (0.0009) [2023-10-13 23:26:34,715][60934] Updated weights for policy 1, policy_version 60672 (0.0007) [2023-10-13 23:26:35,091][60934] Updated weights for policy 1, policy_version 60682 (0.0009) [2023-10-13 23:26:35,451][60934] Updated weights for policy 1, policy_version 60692 (0.0009) [2023-10-13 23:26:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 124190720. Throughput: 0: 1690.8, 1: 1683.7. Samples: 31050508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:26:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:26:38,477][60935] Updated weights for policy 0, policy_version 60290 (0.0010) [2023-10-13 23:26:38,844][60935] Updated weights for policy 0, policy_version 60300 (0.0010) [2023-10-13 23:26:39,210][60935] Updated weights for policy 0, policy_version 60310 (0.0009) [2023-10-13 23:26:39,475][60934] Updated weights for policy 1, policy_version 60702 (0.0008) [2023-10-13 23:26:39,573][60935] Updated weights for policy 0, policy_version 60320 (0.0008) [2023-10-13 23:26:39,841][60934] Updated weights for policy 1, policy_version 60712 (0.0007) [2023-10-13 23:26:40,206][60934] Updated weights for policy 1, policy_version 60722 (0.0007) [2023-10-13 23:26:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 124256256. Throughput: 0: 1701.3, 1: 1665.2. Samples: 31070672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:26:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:26:43,557][60935] Updated weights for policy 0, policy_version 60330 (0.0007) [2023-10-13 23:26:43,927][60935] Updated weights for policy 0, policy_version 60340 (0.0008) [2023-10-13 23:26:44,293][60935] Updated weights for policy 0, policy_version 60350 (0.0007) [2023-10-13 23:26:44,378][60934] Updated weights for policy 1, policy_version 60732 (0.0007) [2023-10-13 23:26:44,746][60934] Updated weights for policy 1, policy_version 60742 (0.0008) [2023-10-13 23:26:45,113][60934] Updated weights for policy 1, policy_version 60752 (0.0007) [2023-10-13 23:26:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 124321792. Throughput: 0: 1708.6, 1: 1685.2. Samples: 31081658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:26:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:26:48,228][60935] Updated weights for policy 0, policy_version 60360 (0.0011) [2023-10-13 23:26:48,594][60935] Updated weights for policy 0, policy_version 60370 (0.0010) [2023-10-13 23:26:48,975][60935] Updated weights for policy 0, policy_version 60380 (0.0009) [2023-10-13 23:26:49,093][60934] Updated weights for policy 1, policy_version 60762 (0.0007) [2023-10-13 23:26:49,468][60934] Updated weights for policy 1, policy_version 60772 (0.0007) [2023-10-13 23:26:49,827][60934] Updated weights for policy 1, policy_version 60782 (0.0007) [2023-10-13 23:26:50,184][60934] Updated weights for policy 1, policy_version 60792 (0.0007) [2023-10-13 23:26:51,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 124387328. Throughput: 0: 1688.1, 1: 1679.1. Samples: 31101488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:26:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:26:52,932][60935] Updated weights for policy 0, policy_version 60390 (0.0008) [2023-10-13 23:26:53,298][60935] Updated weights for policy 0, policy_version 60400 (0.0008) [2023-10-13 23:26:53,670][60935] Updated weights for policy 0, policy_version 60410 (0.0008) [2023-10-13 23:26:54,271][60934] Updated weights for policy 1, policy_version 60802 (0.0009) [2023-10-13 23:26:54,639][60934] Updated weights for policy 1, policy_version 60812 (0.0010) [2023-10-13 23:26:55,010][60934] Updated weights for policy 1, policy_version 60822 (0.0009) [2023-10-13 23:26:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 124452864. Throughput: 0: 1717.8, 1: 1677.0. Samples: 31121718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:26:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:26:56,260][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000060416_61865984.pth... [2023-10-13 23:26:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000060824_62586880.pth... [2023-10-13 23:26:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000058816_60227584.pth [2023-10-13 23:26:56,300][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000059256_60981248.pth [2023-10-13 23:26:57,680][60935] Updated weights for policy 0, policy_version 60420 (0.0010) [2023-10-13 23:26:58,048][60935] Updated weights for policy 0, policy_version 60430 (0.0008) [2023-10-13 23:26:58,422][60935] Updated weights for policy 0, policy_version 60440 (0.0009) [2023-10-13 23:26:59,165][60934] Updated weights for policy 1, policy_version 60832 (0.0009) [2023-10-13 23:26:59,539][60934] Updated weights for policy 1, policy_version 60842 (0.0009) [2023-10-13 23:26:59,900][60934] Updated weights for policy 1, policy_version 60852 (0.0008) [2023-10-13 23:27:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 124518400. Throughput: 0: 1686.3, 1: 1690.9. Samples: 31132118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:27:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:27:02,405][60935] Updated weights for policy 0, policy_version 60450 (0.0008) [2023-10-13 23:27:02,774][60935] Updated weights for policy 0, policy_version 60460 (0.0009) [2023-10-13 23:27:03,144][60935] Updated weights for policy 0, policy_version 60470 (0.0009) [2023-10-13 23:27:03,502][60935] Updated weights for policy 0, policy_version 60480 (0.0009) [2023-10-13 23:27:03,898][60934] Updated weights for policy 1, policy_version 60862 (0.0009) [2023-10-13 23:27:04,265][60934] Updated weights for policy 1, policy_version 60872 (0.0008) [2023-10-13 23:27:04,634][60934] Updated weights for policy 1, policy_version 60882 (0.0008) [2023-10-13 23:27:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 124583936. Throughput: 0: 1704.4, 1: 1671.2. Samples: 31152458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:27:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:27:07,596][60935] Updated weights for policy 0, policy_version 60490 (0.0011) [2023-10-13 23:27:07,966][60935] Updated weights for policy 0, policy_version 60500 (0.0010) [2023-10-13 23:27:08,324][60935] Updated weights for policy 0, policy_version 60510 (0.0008) [2023-10-13 23:27:08,554][60934] Updated weights for policy 1, policy_version 60892 (0.0007) [2023-10-13 23:27:08,933][60934] Updated weights for policy 1, policy_version 60902 (0.0009) [2023-10-13 23:27:09,300][60934] Updated weights for policy 1, policy_version 60912 (0.0009) [2023-10-13 23:27:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 124649472. Throughput: 0: 1722.8, 1: 1690.5. Samples: 31173316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:27:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:27:12,202][60935] Updated weights for policy 0, policy_version 60520 (0.0010) [2023-10-13 23:27:12,570][60935] Updated weights for policy 0, policy_version 60530 (0.0010) [2023-10-13 23:27:12,941][60935] Updated weights for policy 0, policy_version 60540 (0.0007) [2023-10-13 23:27:13,269][60934] Updated weights for policy 1, policy_version 60922 (0.0008) [2023-10-13 23:27:13,642][60934] Updated weights for policy 1, policy_version 60932 (0.0007) [2023-10-13 23:27:14,009][60934] Updated weights for policy 1, policy_version 60942 (0.0009) [2023-10-13 23:27:14,369][60934] Updated weights for policy 1, policy_version 60952 (0.0007) [2023-10-13 23:27:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 124715008. Throughput: 0: 1694.1, 1: 1690.6. Samples: 31183496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:27:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:27:16,909][60935] Updated weights for policy 0, policy_version 60550 (0.0008) [2023-10-13 23:27:17,282][60935] Updated weights for policy 0, policy_version 60560 (0.0008) [2023-10-13 23:27:17,648][60935] Updated weights for policy 0, policy_version 60570 (0.0008) [2023-10-13 23:27:18,453][60934] Updated weights for policy 1, policy_version 60962 (0.0008) [2023-10-13 23:27:18,820][60934] Updated weights for policy 1, policy_version 60972 (0.0009) [2023-10-13 23:27:19,181][60934] Updated weights for policy 1, policy_version 60982 (0.0007) [2023-10-13 23:27:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 124780544. Throughput: 0: 1725.6, 1: 1676.8. Samples: 31203618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:27:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:27:21,763][60935] Updated weights for policy 0, policy_version 60580 (0.0009) [2023-10-13 23:27:22,129][60935] Updated weights for policy 0, policy_version 60590 (0.0008) [2023-10-13 23:27:22,500][60935] Updated weights for policy 0, policy_version 60600 (0.0008) [2023-10-13 23:27:23,249][60934] Updated weights for policy 1, policy_version 60992 (0.0007) [2023-10-13 23:27:23,612][60934] Updated weights for policy 1, policy_version 61002 (0.0009) [2023-10-13 23:27:23,989][60934] Updated weights for policy 1, policy_version 61012 (0.0008) [2023-10-13 23:27:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 124846080. Throughput: 0: 1720.0, 1: 1695.9. Samples: 31224388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:27:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:27:26,529][60935] Updated weights for policy 0, policy_version 60610 (0.0009) [2023-10-13 23:27:26,893][60935] Updated weights for policy 0, policy_version 60620 (0.0008) [2023-10-13 23:27:27,267][60935] Updated weights for policy 0, policy_version 60630 (0.0009) [2023-10-13 23:27:27,650][60935] Updated weights for policy 0, policy_version 60640 (0.0008) [2023-10-13 23:27:28,102][60934] Updated weights for policy 1, policy_version 61022 (0.0007) [2023-10-13 23:27:28,465][60934] Updated weights for policy 1, policy_version 61032 (0.0008) [2023-10-13 23:27:28,830][60934] Updated weights for policy 1, policy_version 61042 (0.0008) [2023-10-13 23:27:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 124911616. Throughput: 0: 1703.2, 1: 1682.7. Samples: 31234020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:27:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:27:31,745][60935] Updated weights for policy 0, policy_version 60650 (0.0010) [2023-10-13 23:27:32,120][60935] Updated weights for policy 0, policy_version 60660 (0.0008) [2023-10-13 23:27:32,479][60935] Updated weights for policy 0, policy_version 60670 (0.0008) [2023-10-13 23:27:32,704][60934] Updated weights for policy 1, policy_version 61052 (0.0009) [2023-10-13 23:27:33,074][60934] Updated weights for policy 1, policy_version 61062 (0.0009) [2023-10-13 23:27:33,436][60934] Updated weights for policy 1, policy_version 61072 (0.0007) [2023-10-13 23:27:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 124977152. Throughput: 0: 1716.2, 1: 1685.1. Samples: 31254544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:27:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:27:36,408][60935] Updated weights for policy 0, policy_version 60680 (0.0010) [2023-10-13 23:27:36,786][60935] Updated weights for policy 0, policy_version 60690 (0.0008) [2023-10-13 23:27:37,153][60935] Updated weights for policy 0, policy_version 60700 (0.0009) [2023-10-13 23:27:37,620][60934] Updated weights for policy 1, policy_version 61082 (0.0008) [2023-10-13 23:27:37,989][60934] Updated weights for policy 1, policy_version 61092 (0.0008) [2023-10-13 23:27:38,359][60934] Updated weights for policy 1, policy_version 61102 (0.0010) [2023-10-13 23:27:38,719][60934] Updated weights for policy 1, policy_version 61112 (0.0008) [2023-10-13 23:27:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 125042688. Throughput: 0: 1708.3, 1: 1703.6. Samples: 31275254. Policy #0 lag: (min: 3.0, avg: 4.8, max: 32.0) [2023-10-13 23:27:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:27:41,403][60935] Updated weights for policy 0, policy_version 60710 (0.0009) [2023-10-13 23:27:41,771][60935] Updated weights for policy 0, policy_version 60720 (0.0009) [2023-10-13 23:27:42,138][60935] Updated weights for policy 0, policy_version 60730 (0.0009) [2023-10-13 23:27:42,694][60934] Updated weights for policy 1, policy_version 61122 (0.0008) [2023-10-13 23:27:43,063][60934] Updated weights for policy 1, policy_version 61132 (0.0009) [2023-10-13 23:27:43,430][60934] Updated weights for policy 1, policy_version 61142 (0.0010) [2023-10-13 23:27:46,109][60935] Updated weights for policy 0, policy_version 60740 (0.0009) [2023-10-13 23:27:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 125108224. Throughput: 0: 1709.6, 1: 1677.3. Samples: 31284526. Policy #0 lag: (min: 3.0, avg: 4.8, max: 32.0) [2023-10-13 23:27:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:27:46,477][60935] Updated weights for policy 0, policy_version 60750 (0.0010) [2023-10-13 23:27:46,849][60935] Updated weights for policy 0, policy_version 60760 (0.0010) [2023-10-13 23:27:47,420][60934] Updated weights for policy 1, policy_version 61152 (0.0008) [2023-10-13 23:27:47,785][60934] Updated weights for policy 1, policy_version 61162 (0.0007) [2023-10-13 23:27:48,155][60934] Updated weights for policy 1, policy_version 61172 (0.0007) [2023-10-13 23:27:50,883][60935] Updated weights for policy 0, policy_version 60770 (0.0009) [2023-10-13 23:27:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 125173760. Throughput: 0: 1705.5, 1: 1694.9. Samples: 31305476. Policy #0 lag: (min: 3.0, avg: 4.8, max: 32.0) [2023-10-13 23:27:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:27:51,254][60935] Updated weights for policy 0, policy_version 60780 (0.0008) [2023-10-13 23:27:51,622][60935] Updated weights for policy 0, policy_version 60790 (0.0007) [2023-10-13 23:27:51,982][60935] Updated weights for policy 0, policy_version 60800 (0.0008) [2023-10-13 23:27:52,225][60934] Updated weights for policy 1, policy_version 61182 (0.0010) [2023-10-13 23:27:52,591][60934] Updated weights for policy 1, policy_version 61192 (0.0010) [2023-10-13 23:27:52,973][60934] Updated weights for policy 1, policy_version 61202 (0.0009) [2023-10-13 23:27:55,833][60935] Updated weights for policy 0, policy_version 60810 (0.0011) [2023-10-13 23:27:56,205][60935] Updated weights for policy 0, policy_version 60820 (0.0010) [2023-10-13 23:27:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 125239296. Throughput: 0: 1696.8, 1: 1699.5. Samples: 31326148. Policy #0 lag: (min: 3.0, avg: 4.8, max: 32.0) [2023-10-13 23:27:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:27:56,580][60935] Updated weights for policy 0, policy_version 60830 (0.0010) [2023-10-13 23:27:57,001][60934] Updated weights for policy 1, policy_version 61212 (0.0010) [2023-10-13 23:27:57,374][60934] Updated weights for policy 1, policy_version 61222 (0.0008) [2023-10-13 23:27:57,732][60934] Updated weights for policy 1, policy_version 61232 (0.0009) [2023-10-13 23:28:00,612][60935] Updated weights for policy 0, policy_version 60840 (0.0009) [2023-10-13 23:28:00,983][60935] Updated weights for policy 0, policy_version 60850 (0.0010) [2023-10-13 23:28:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 125304832. Throughput: 0: 1705.7, 1: 1677.8. Samples: 31335756. Policy #0 lag: (min: 3.0, avg: 4.8, max: 32.0) [2023-10-13 23:28:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:28:01,355][60935] Updated weights for policy 0, policy_version 60860 (0.0008) [2023-10-13 23:28:01,890][60934] Updated weights for policy 1, policy_version 61242 (0.0008) [2023-10-13 23:28:02,256][60934] Updated weights for policy 1, policy_version 61252 (0.0011) [2023-10-13 23:28:02,629][60934] Updated weights for policy 1, policy_version 61262 (0.0010) [2023-10-13 23:28:02,993][60934] Updated weights for policy 1, policy_version 61272 (0.0011) [2023-10-13 23:28:05,293][60935] Updated weights for policy 0, policy_version 60870 (0.0008) [2023-10-13 23:28:05,671][60935] Updated weights for policy 0, policy_version 60880 (0.0012) [2023-10-13 23:28:06,032][60935] Updated weights for policy 0, policy_version 60890 (0.0009) [2023-10-13 23:28:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 125370368. Throughput: 0: 1704.0, 1: 1695.8. Samples: 31356606. Policy #0 lag: (min: 3.0, avg: 4.8, max: 32.0) [2023-10-13 23:28:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:28:07,137][60934] Updated weights for policy 1, policy_version 61282 (0.0010) [2023-10-13 23:28:07,502][60934] Updated weights for policy 1, policy_version 61292 (0.0011) [2023-10-13 23:28:07,882][60934] Updated weights for policy 1, policy_version 61302 (0.0011) [2023-10-13 23:28:10,013][60935] Updated weights for policy 0, policy_version 60900 (0.0007) [2023-10-13 23:28:10,377][60935] Updated weights for policy 0, policy_version 60910 (0.0011) [2023-10-13 23:28:10,756][60935] Updated weights for policy 0, policy_version 60920 (0.0007) [2023-10-13 23:28:11,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 125468672. Throughput: 0: 1687.6, 1: 1690.4. Samples: 31376396. Policy #0 lag: (min: 3.0, avg: 4.8, max: 32.0) [2023-10-13 23:28:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:28:11,946][60934] Updated weights for policy 1, policy_version 61312 (0.0010) [2023-10-13 23:28:12,308][60934] Updated weights for policy 1, policy_version 61322 (0.0007) [2023-10-13 23:28:12,679][60934] Updated weights for policy 1, policy_version 61332 (0.0009) [2023-10-13 23:28:14,689][60935] Updated weights for policy 0, policy_version 60930 (0.0008) [2023-10-13 23:28:15,059][60935] Updated weights for policy 0, policy_version 60940 (0.0008) [2023-10-13 23:28:15,419][60935] Updated weights for policy 0, policy_version 60950 (0.0007) [2023-10-13 23:28:15,785][60935] Updated weights for policy 0, policy_version 60960 (0.0007) [2023-10-13 23:28:16,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 125534208. Throughput: 0: 1715.7, 1: 1679.0. Samples: 31386780. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:28:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:28:16,627][60934] Updated weights for policy 1, policy_version 61342 (0.0009) [2023-10-13 23:28:16,990][60934] Updated weights for policy 1, policy_version 61352 (0.0009) [2023-10-13 23:28:17,359][60934] Updated weights for policy 1, policy_version 61362 (0.0009) [2023-10-13 23:28:19,859][60935] Updated weights for policy 0, policy_version 60970 (0.0008) [2023-10-13 23:28:20,219][60935] Updated weights for policy 0, policy_version 60980 (0.0008) [2023-10-13 23:28:20,598][60935] Updated weights for policy 0, policy_version 60990 (0.0008) [2023-10-13 23:28:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 125599744. Throughput: 0: 1708.4, 1: 1684.9. Samples: 31407242. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:28:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:28:21,585][60934] Updated weights for policy 1, policy_version 61372 (0.0008) [2023-10-13 23:28:21,954][60934] Updated weights for policy 1, policy_version 61382 (0.0010) [2023-10-13 23:28:22,326][60934] Updated weights for policy 1, policy_version 61392 (0.0010) [2023-10-13 23:28:24,485][60935] Updated weights for policy 0, policy_version 61000 (0.0007) [2023-10-13 23:28:24,848][60935] Updated weights for policy 0, policy_version 61010 (0.0007) [2023-10-13 23:28:25,218][60935] Updated weights for policy 0, policy_version 61020 (0.0008) [2023-10-13 23:28:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 125665280. Throughput: 0: 1697.2, 1: 1683.9. Samples: 31427400. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:28:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:28:26,340][60934] Updated weights for policy 1, policy_version 61402 (0.0009) [2023-10-13 23:28:26,705][60934] Updated weights for policy 1, policy_version 61412 (0.0009) [2023-10-13 23:28:27,082][60934] Updated weights for policy 1, policy_version 61422 (0.0007) [2023-10-13 23:28:27,449][60934] Updated weights for policy 1, policy_version 61432 (0.0011) [2023-10-13 23:28:29,182][60935] Updated weights for policy 0, policy_version 61030 (0.0009) [2023-10-13 23:28:29,551][60935] Updated weights for policy 0, policy_version 61040 (0.0010) [2023-10-13 23:28:29,926][60935] Updated weights for policy 0, policy_version 61050 (0.0011) [2023-10-13 23:28:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 125730816. Throughput: 0: 1724.4, 1: 1680.6. Samples: 31437750. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:28:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:28:31,569][60934] Updated weights for policy 1, policy_version 61442 (0.0007) [2023-10-13 23:28:31,937][60934] Updated weights for policy 1, policy_version 61452 (0.0009) [2023-10-13 23:28:32,302][60934] Updated weights for policy 1, policy_version 61462 (0.0009) [2023-10-13 23:28:34,064][60935] Updated weights for policy 0, policy_version 61060 (0.0010) [2023-10-13 23:28:34,432][60935] Updated weights for policy 0, policy_version 61070 (0.0009) [2023-10-13 23:28:34,798][60935] Updated weights for policy 0, policy_version 61080 (0.0008) [2023-10-13 23:28:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 125796352. Throughput: 0: 1702.0, 1: 1683.6. Samples: 31457826. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:28:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:28:36,262][60934] Updated weights for policy 1, policy_version 61472 (0.0007) [2023-10-13 23:28:36,631][60934] Updated weights for policy 1, policy_version 61482 (0.0007) [2023-10-13 23:28:36,998][60934] Updated weights for policy 1, policy_version 61492 (0.0008) [2023-10-13 23:28:38,603][60935] Updated weights for policy 0, policy_version 61090 (0.0007) [2023-10-13 23:28:38,976][60935] Updated weights for policy 0, policy_version 61100 (0.0008) [2023-10-13 23:28:39,354][60935] Updated weights for policy 0, policy_version 61110 (0.0007) [2023-10-13 23:28:39,722][60935] Updated weights for policy 0, policy_version 61120 (0.0007) [2023-10-13 23:28:40,969][60934] Updated weights for policy 1, policy_version 61502 (0.0010) [2023-10-13 23:28:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 125861888. Throughput: 0: 1706.7, 1: 1685.3. Samples: 31478786. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:28:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:28:41,336][60934] Updated weights for policy 1, policy_version 61512 (0.0010) [2023-10-13 23:28:41,697][60934] Updated weights for policy 1, policy_version 61522 (0.0010) [2023-10-13 23:28:43,595][60935] Updated weights for policy 0, policy_version 61130 (0.0008) [2023-10-13 23:28:43,959][60935] Updated weights for policy 0, policy_version 61140 (0.0008) [2023-10-13 23:28:44,327][60935] Updated weights for policy 0, policy_version 61150 (0.0008) [2023-10-13 23:28:45,660][60934] Updated weights for policy 1, policy_version 61532 (0.0008) [2023-10-13 23:28:46,041][60934] Updated weights for policy 1, policy_version 61542 (0.0008) [2023-10-13 23:28:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 125927424. Throughput: 0: 1714.2, 1: 1685.7. Samples: 31488750. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:28:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:28:46,410][60934] Updated weights for policy 1, policy_version 61552 (0.0010) [2023-10-13 23:28:48,295][60935] Updated weights for policy 0, policy_version 61160 (0.0009) [2023-10-13 23:28:48,668][60935] Updated weights for policy 0, policy_version 61170 (0.0008) [2023-10-13 23:28:49,030][60935] Updated weights for policy 0, policy_version 61180 (0.0011) [2023-10-13 23:28:50,369][60934] Updated weights for policy 1, policy_version 61562 (0.0009) [2023-10-13 23:28:50,737][60934] Updated weights for policy 1, policy_version 61572 (0.0009) [2023-10-13 23:28:51,100][60934] Updated weights for policy 1, policy_version 61582 (0.0007) [2023-10-13 23:28:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 125992960. Throughput: 0: 1697.0, 1: 1694.2. Samples: 31509208. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:28:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:28:51,462][60934] Updated weights for policy 1, policy_version 61592 (0.0007) [2023-10-13 23:28:53,173][60935] Updated weights for policy 0, policy_version 61190 (0.0010) [2023-10-13 23:28:53,549][60935] Updated weights for policy 0, policy_version 61200 (0.0008) [2023-10-13 23:28:53,910][60935] Updated weights for policy 0, policy_version 61210 (0.0009) [2023-10-13 23:28:55,560][60934] Updated weights for policy 1, policy_version 61602 (0.0009) [2023-10-13 23:28:55,935][60934] Updated weights for policy 1, policy_version 61612 (0.0008) [2023-10-13 23:28:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 126058496. Throughput: 0: 1717.9, 1: 1689.9. Samples: 31529746. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:28:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:28:56,255][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000061216_62685184.pth... [2023-10-13 23:28:56,284][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000059616_61046784.pth [2023-10-13 23:28:56,303][60934] Updated weights for policy 1, policy_version 61622 (0.0007) [2023-10-13 23:28:56,374][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000061624_63406080.pth... [2023-10-13 23:28:56,403][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000060024_61767680.pth [2023-10-13 23:28:57,893][60935] Updated weights for policy 0, policy_version 61220 (0.0010) [2023-10-13 23:28:58,267][60935] Updated weights for policy 0, policy_version 61230 (0.0008) [2023-10-13 23:28:58,627][60935] Updated weights for policy 0, policy_version 61240 (0.0009) [2023-10-13 23:29:00,305][60934] Updated weights for policy 1, policy_version 61632 (0.0010) [2023-10-13 23:29:00,668][60934] Updated weights for policy 1, policy_version 61642 (0.0009) [2023-10-13 23:29:01,038][60934] Updated weights for policy 1, policy_version 61652 (0.0009) [2023-10-13 23:29:01,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 126156800. Throughput: 0: 1695.6, 1: 1697.1. Samples: 31539450. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-13 23:29:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:29:02,559][60935] Updated weights for policy 0, policy_version 61250 (0.0011) [2023-10-13 23:29:02,920][60935] Updated weights for policy 0, policy_version 61260 (0.0010) [2023-10-13 23:29:03,292][60935] Updated weights for policy 0, policy_version 61270 (0.0008) [2023-10-13 23:29:03,661][60935] Updated weights for policy 0, policy_version 61280 (0.0008) [2023-10-13 23:29:04,975][60934] Updated weights for policy 1, policy_version 61662 (0.0009) [2023-10-13 23:29:05,343][60934] Updated weights for policy 1, policy_version 61672 (0.0009) [2023-10-13 23:29:05,711][60934] Updated weights for policy 1, policy_version 61682 (0.0008) [2023-10-13 23:29:06,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 126222336. Throughput: 0: 1700.7, 1: 1701.3. Samples: 31560332. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-13 23:29:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:29:07,583][60935] Updated weights for policy 0, policy_version 61290 (0.0008) [2023-10-13 23:29:07,959][60935] Updated weights for policy 0, policy_version 61300 (0.0010) [2023-10-13 23:29:08,325][60935] Updated weights for policy 0, policy_version 61310 (0.0008) [2023-10-13 23:29:09,836][60934] Updated weights for policy 1, policy_version 61692 (0.0007) [2023-10-13 23:29:10,200][60934] Updated weights for policy 1, policy_version 61702 (0.0007) [2023-10-13 23:29:10,566][60934] Updated weights for policy 1, policy_version 61712 (0.0010) [2023-10-13 23:29:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 126287872. Throughput: 0: 1720.7, 1: 1684.2. Samples: 31580620. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-13 23:29:11,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:29:12,289][60935] Updated weights for policy 0, policy_version 61320 (0.0010) [2023-10-13 23:29:12,650][60935] Updated weights for policy 0, policy_version 61330 (0.0010) [2023-10-13 23:29:13,026][60935] Updated weights for policy 0, policy_version 61340 (0.0010) [2023-10-13 23:29:14,444][60934] Updated weights for policy 1, policy_version 61722 (0.0008) [2023-10-13 23:29:14,802][60934] Updated weights for policy 1, policy_version 61732 (0.0007) [2023-10-13 23:29:15,174][60934] Updated weights for policy 1, policy_version 61742 (0.0007) [2023-10-13 23:29:15,546][60934] Updated weights for policy 1, policy_version 61752 (0.0010) [2023-10-13 23:29:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 126353408. Throughput: 0: 1690.7, 1: 1708.3. Samples: 31590710. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-13 23:29:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:29:17,174][60935] Updated weights for policy 0, policy_version 61350 (0.0008) [2023-10-13 23:29:17,550][60935] Updated weights for policy 0, policy_version 61360 (0.0008) [2023-10-13 23:29:17,928][60935] Updated weights for policy 0, policy_version 61370 (0.0007) [2023-10-13 23:29:19,642][60934] Updated weights for policy 1, policy_version 61762 (0.0007) [2023-10-13 23:29:20,005][60934] Updated weights for policy 1, policy_version 61772 (0.0008) [2023-10-13 23:29:20,374][60934] Updated weights for policy 1, policy_version 61782 (0.0007) [2023-10-13 23:29:21,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 126418944. Throughput: 0: 1715.3, 1: 1697.6. Samples: 31611410. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-13 23:29:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:29:21,901][60935] Updated weights for policy 0, policy_version 61380 (0.0009) [2023-10-13 23:29:22,278][60935] Updated weights for policy 0, policy_version 61390 (0.0010) [2023-10-13 23:29:22,651][60935] Updated weights for policy 0, policy_version 61400 (0.0010) [2023-10-13 23:29:24,412][60934] Updated weights for policy 1, policy_version 61792 (0.0008) [2023-10-13 23:29:24,772][60934] Updated weights for policy 1, policy_version 61802 (0.0007) [2023-10-13 23:29:25,144][60934] Updated weights for policy 1, policy_version 61812 (0.0008) [2023-10-13 23:29:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 126484480. Throughput: 0: 1715.2, 1: 1675.9. Samples: 31631386. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-13 23:29:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:29:26,562][60935] Updated weights for policy 0, policy_version 61410 (0.0010) [2023-10-13 23:29:26,931][60935] Updated weights for policy 0, policy_version 61420 (0.0008) [2023-10-13 23:29:27,293][60935] Updated weights for policy 0, policy_version 61430 (0.0008) [2023-10-13 23:29:27,662][60935] Updated weights for policy 0, policy_version 61440 (0.0010) [2023-10-13 23:29:29,280][60934] Updated weights for policy 1, policy_version 61822 (0.0009) [2023-10-13 23:29:29,650][60934] Updated weights for policy 1, policy_version 61832 (0.0007) [2023-10-13 23:29:30,018][60934] Updated weights for policy 1, policy_version 61842 (0.0007) [2023-10-13 23:29:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 126550016. Throughput: 0: 1698.4, 1: 1707.4. Samples: 31642010. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-13 23:29:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:29:31,675][60935] Updated weights for policy 0, policy_version 61450 (0.0010) [2023-10-13 23:29:32,053][60935] Updated weights for policy 0, policy_version 61460 (0.0009) [2023-10-13 23:29:32,424][60935] Updated weights for policy 0, policy_version 61470 (0.0007) [2023-10-13 23:29:33,922][60934] Updated weights for policy 1, policy_version 61852 (0.0008) [2023-10-13 23:29:34,282][60934] Updated weights for policy 1, policy_version 61862 (0.0007) [2023-10-13 23:29:34,647][60934] Updated weights for policy 1, policy_version 61872 (0.0010) [2023-10-13 23:29:36,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 126615552. Throughput: 0: 1718.4, 1: 1686.0. Samples: 31662408. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-13 23:29:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:29:36,289][60935] Updated weights for policy 0, policy_version 61480 (0.0009) [2023-10-13 23:29:36,660][60935] Updated weights for policy 0, policy_version 61490 (0.0009) [2023-10-13 23:29:37,034][60935] Updated weights for policy 0, policy_version 61500 (0.0007) [2023-10-13 23:29:38,698][60934] Updated weights for policy 1, policy_version 61882 (0.0011) [2023-10-13 23:29:39,064][60934] Updated weights for policy 1, policy_version 61892 (0.0011) [2023-10-13 23:29:39,433][60934] Updated weights for policy 1, policy_version 61902 (0.0009) [2023-10-13 23:29:39,795][60934] Updated weights for policy 1, policy_version 61912 (0.0008) [2023-10-13 23:29:40,921][60935] Updated weights for policy 0, policy_version 61510 (0.0009) [2023-10-13 23:29:41,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 126681088. Throughput: 0: 1721.2, 1: 1682.3. Samples: 31682902. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-13 23:29:41,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:29:41,292][60935] Updated weights for policy 0, policy_version 61520 (0.0008) [2023-10-13 23:29:41,666][60935] Updated weights for policy 0, policy_version 61530 (0.0008) [2023-10-13 23:29:44,014][60934] Updated weights for policy 1, policy_version 61922 (0.0009) [2023-10-13 23:29:44,376][60934] Updated weights for policy 1, policy_version 61932 (0.0007) [2023-10-13 23:29:44,734][60934] Updated weights for policy 1, policy_version 61942 (0.0008) [2023-10-13 23:29:45,529][60935] Updated weights for policy 0, policy_version 61540 (0.0009) [2023-10-13 23:29:45,891][60935] Updated weights for policy 0, policy_version 61550 (0.0008) [2023-10-13 23:29:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 126746624. Throughput: 0: 1719.8, 1: 1701.1. Samples: 31693388. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-13 23:29:46,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:29:46,262][60935] Updated weights for policy 0, policy_version 61560 (0.0008) [2023-10-13 23:29:48,841][60934] Updated weights for policy 1, policy_version 61952 (0.0008) [2023-10-13 23:29:49,206][60934] Updated weights for policy 1, policy_version 61962 (0.0009) [2023-10-13 23:29:49,573][60934] Updated weights for policy 1, policy_version 61972 (0.0007) [2023-10-13 23:29:50,216][60935] Updated weights for policy 0, policy_version 61570 (0.0008) [2023-10-13 23:29:50,585][60935] Updated weights for policy 0, policy_version 61580 (0.0009) [2023-10-13 23:29:50,952][60935] Updated weights for policy 0, policy_version 61590 (0.0008) [2023-10-13 23:29:51,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 126812160. Throughput: 0: 1725.2, 1: 1672.5. Samples: 31713232. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-13 23:29:51,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:29:51,318][60935] Updated weights for policy 0, policy_version 61600 (0.0007) [2023-10-13 23:29:53,567][60934] Updated weights for policy 1, policy_version 61982 (0.0008) [2023-10-13 23:29:53,938][60934] Updated weights for policy 1, policy_version 61992 (0.0009) [2023-10-13 23:29:54,300][60934] Updated weights for policy 1, policy_version 62002 (0.0010) [2023-10-13 23:29:55,399][60935] Updated weights for policy 0, policy_version 61610 (0.0007) [2023-10-13 23:29:55,780][60935] Updated weights for policy 0, policy_version 61620 (0.0007) [2023-10-13 23:29:56,155][60935] Updated weights for policy 0, policy_version 61630 (0.0007) [2023-10-13 23:29:56,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 126910464. Throughput: 0: 1703.2, 1: 1687.8. Samples: 31733218. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-13 23:29:56,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:29:58,316][60934] Updated weights for policy 1, policy_version 62012 (0.0008) [2023-10-13 23:29:58,680][60934] Updated weights for policy 1, policy_version 62022 (0.0007) [2023-10-13 23:29:59,049][60934] Updated weights for policy 1, policy_version 62032 (0.0007) [2023-10-13 23:29:59,986][60935] Updated weights for policy 0, policy_version 61640 (0.0008) [2023-10-13 23:30:00,354][60935] Updated weights for policy 0, policy_version 61650 (0.0008) [2023-10-13 23:30:00,717][60935] Updated weights for policy 0, policy_version 61660 (0.0008) [2023-10-13 23:30:01,248][59943] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 126976000. Throughput: 0: 1725.3, 1: 1685.3. Samples: 31744184. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-13 23:30:01,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:30:03,088][60934] Updated weights for policy 1, policy_version 62042 (0.0007) [2023-10-13 23:30:03,458][60934] Updated weights for policy 1, policy_version 62052 (0.0010) [2023-10-13 23:30:03,817][60934] Updated weights for policy 1, policy_version 62062 (0.0010) [2023-10-13 23:30:04,179][60934] Updated weights for policy 1, policy_version 62072 (0.0008) [2023-10-13 23:30:04,713][60935] Updated weights for policy 0, policy_version 61670 (0.0010) [2023-10-13 23:30:05,081][60935] Updated weights for policy 0, policy_version 61680 (0.0010) [2023-10-13 23:30:05,438][60935] Updated weights for policy 0, policy_version 61690 (0.0010) [2023-10-13 23:30:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 127041536. Throughput: 0: 1722.5, 1: 1673.0. Samples: 31764208. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-13 23:30:06,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:30:08,451][60934] Updated weights for policy 1, policy_version 62082 (0.0008) [2023-10-13 23:30:08,822][60934] Updated weights for policy 1, policy_version 62092 (0.0010) [2023-10-13 23:30:09,191][60934] Updated weights for policy 1, policy_version 62102 (0.0008) [2023-10-13 23:30:09,368][60935] Updated weights for policy 0, policy_version 61700 (0.0009) [2023-10-13 23:30:09,729][60935] Updated weights for policy 0, policy_version 61710 (0.0007) [2023-10-13 23:30:10,092][60935] Updated weights for policy 0, policy_version 61720 (0.0010) [2023-10-13 23:30:11,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 127107072. Throughput: 0: 1701.6, 1: 1689.7. Samples: 31783996. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-13 23:30:11,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:30:13,262][60934] Updated weights for policy 1, policy_version 62112 (0.0010) [2023-10-13 23:30:13,623][60934] Updated weights for policy 1, policy_version 62122 (0.0011) [2023-10-13 23:30:13,967][60935] Updated weights for policy 0, policy_version 61730 (0.0009) [2023-10-13 23:30:13,986][60934] Updated weights for policy 1, policy_version 62132 (0.0009) [2023-10-13 23:30:14,338][60935] Updated weights for policy 0, policy_version 61740 (0.0009) [2023-10-13 23:30:14,708][60935] Updated weights for policy 0, policy_version 61750 (0.0008) [2023-10-13 23:30:15,070][60935] Updated weights for policy 0, policy_version 61760 (0.0010) [2023-10-13 23:30:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 127172608. Throughput: 0: 1737.1, 1: 1669.6. Samples: 31795310. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-13 23:30:16,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:30:18,083][60934] Updated weights for policy 1, policy_version 62142 (0.0008) [2023-10-13 23:30:18,449][60934] Updated weights for policy 1, policy_version 62152 (0.0007) [2023-10-13 23:30:18,805][60934] Updated weights for policy 1, policy_version 62162 (0.0007) [2023-10-13 23:30:18,962][60935] Updated weights for policy 0, policy_version 61770 (0.0009) [2023-10-13 23:30:19,327][60935] Updated weights for policy 0, policy_version 61780 (0.0010) [2023-10-13 23:30:19,702][60935] Updated weights for policy 0, policy_version 61790 (0.0009) [2023-10-13 23:30:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 127238144. Throughput: 0: 1708.0, 1: 1670.9. Samples: 31814460. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-13 23:30:21,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:30:22,824][60934] Updated weights for policy 1, policy_version 62172 (0.0009) [2023-10-13 23:30:23,191][60934] Updated weights for policy 1, policy_version 62182 (0.0011) [2023-10-13 23:30:23,558][60934] Updated weights for policy 1, policy_version 62192 (0.0007) [2023-10-13 23:30:23,715][60935] Updated weights for policy 0, policy_version 61800 (0.0010) [2023-10-13 23:30:24,077][60935] Updated weights for policy 0, policy_version 61810 (0.0007) [2023-10-13 23:30:24,439][60935] Updated weights for policy 0, policy_version 61820 (0.0009) [2023-10-13 23:30:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 127303680. Throughput: 0: 1704.7, 1: 1683.9. Samples: 31835388. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-13 23:30:26,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:30:27,687][60934] Updated weights for policy 1, policy_version 62202 (0.0008) [2023-10-13 23:30:28,052][60934] Updated weights for policy 1, policy_version 62212 (0.0010) [2023-10-13 23:30:28,410][60934] Updated weights for policy 1, policy_version 62222 (0.0008) [2023-10-13 23:30:28,475][60935] Updated weights for policy 0, policy_version 61830 (0.0008) [2023-10-13 23:30:28,779][60934] Updated weights for policy 1, policy_version 62232 (0.0008) [2023-10-13 23:30:28,844][60935] Updated weights for policy 0, policy_version 61840 (0.0007) [2023-10-13 23:30:29,214][60935] Updated weights for policy 0, policy_version 61850 (0.0010) [2023-10-13 23:30:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 127369216. Throughput: 0: 1718.1, 1: 1659.0. Samples: 31845360. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-13 23:30:31,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:30:32,882][60934] Updated weights for policy 1, policy_version 62242 (0.0009) [2023-10-13 23:30:33,253][60934] Updated weights for policy 1, policy_version 62252 (0.0010) [2023-10-13 23:30:33,286][60935] Updated weights for policy 0, policy_version 61860 (0.0010) [2023-10-13 23:30:33,619][60934] Updated weights for policy 1, policy_version 62262 (0.0008) [2023-10-13 23:30:33,657][60935] Updated weights for policy 0, policy_version 61870 (0.0009) [2023-10-13 23:30:34,023][60935] Updated weights for policy 0, policy_version 61880 (0.0011) [2023-10-13 23:30:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 127434752. Throughput: 0: 1698.0, 1: 1677.5. Samples: 31865128. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-13 23:30:36,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:30:37,735][60934] Updated weights for policy 1, policy_version 62272 (0.0008) [2023-10-13 23:30:38,072][60935] Updated weights for policy 0, policy_version 61890 (0.0009) [2023-10-13 23:30:38,115][60934] Updated weights for policy 1, policy_version 62282 (0.0008) [2023-10-13 23:30:38,437][60935] Updated weights for policy 0, policy_version 61900 (0.0008) [2023-10-13 23:30:38,481][60934] Updated weights for policy 1, policy_version 62292 (0.0008) [2023-10-13 23:30:38,811][60935] Updated weights for policy 0, policy_version 61910 (0.0009) [2023-10-13 23:30:39,173][60935] Updated weights for policy 0, policy_version 61920 (0.0010) [2023-10-13 23:30:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 127500288. Throughput: 0: 1716.2, 1: 1682.8. Samples: 31886172. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-13 23:30:41,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:30:42,511][60934] Updated weights for policy 1, policy_version 62302 (0.0010) [2023-10-13 23:30:42,876][60934] Updated weights for policy 1, policy_version 62312 (0.0008) [2023-10-13 23:30:43,248][60934] Updated weights for policy 1, policy_version 62322 (0.0008) [2023-10-13 23:30:43,260][60935] Updated weights for policy 0, policy_version 61930 (0.0008) [2023-10-13 23:30:43,626][60935] Updated weights for policy 0, policy_version 61940 (0.0008) [2023-10-13 23:30:44,000][60935] Updated weights for policy 0, policy_version 61950 (0.0009) [2023-10-13 23:30:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 127565824. Throughput: 0: 1700.8, 1: 1661.4. Samples: 31895484. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-13 23:30:46,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:30:47,098][60934] Updated weights for policy 1, policy_version 62332 (0.0008) [2023-10-13 23:30:47,462][60934] Updated weights for policy 1, policy_version 62342 (0.0007) [2023-10-13 23:30:47,834][60934] Updated weights for policy 1, policy_version 62352 (0.0008) [2023-10-13 23:30:47,996][60935] Updated weights for policy 0, policy_version 61960 (0.0007) [2023-10-13 23:30:48,362][60935] Updated weights for policy 0, policy_version 61970 (0.0009) [2023-10-13 23:30:48,733][60935] Updated weights for policy 0, policy_version 61980 (0.0009) [2023-10-13 23:30:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 127631360. Throughput: 0: 1696.8, 1: 1687.2. Samples: 31916484. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-13 23:30:51,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:30:51,834][60934] Updated weights for policy 1, policy_version 62362 (0.0008) [2023-10-13 23:30:52,200][60934] Updated weights for policy 1, policy_version 62372 (0.0010) [2023-10-13 23:30:52,576][60934] Updated weights for policy 1, policy_version 62382 (0.0008) [2023-10-13 23:30:52,616][60935] Updated weights for policy 0, policy_version 61990 (0.0007) [2023-10-13 23:30:52,937][60934] Updated weights for policy 1, policy_version 62392 (0.0008) [2023-10-13 23:30:52,981][60935] Updated weights for policy 0, policy_version 62000 (0.0007) [2023-10-13 23:30:53,351][60935] Updated weights for policy 0, policy_version 62010 (0.0007) [2023-10-13 23:30:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 127696896. Throughput: 0: 1719.8, 1: 1694.4. Samples: 31937632. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-13 23:30:56,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:30:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000062392_64192512.pth... [2023-10-13 23:30:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000062016_63504384.pth... [2023-10-13 23:30:56,300][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000060824_62586880.pth [2023-10-13 23:30:56,306][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000060416_61865984.pth [2023-10-13 23:30:56,942][60934] Updated weights for policy 1, policy_version 62402 (0.0009) [2023-10-13 23:30:57,312][60934] Updated weights for policy 1, policy_version 62412 (0.0007) [2023-10-13 23:30:57,456][60935] Updated weights for policy 0, policy_version 62020 (0.0008) [2023-10-13 23:30:57,682][60934] Updated weights for policy 1, policy_version 62422 (0.0009) [2023-10-13 23:30:57,834][60935] Updated weights for policy 0, policy_version 62030 (0.0009) [2023-10-13 23:30:58,201][60935] Updated weights for policy 0, policy_version 62040 (0.0012) [2023-10-13 23:31:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 127762432. Throughput: 0: 1679.7, 1: 1685.6. Samples: 31946748. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-13 23:31:01,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:31:01,700][60934] Updated weights for policy 1, policy_version 62432 (0.0009) [2023-10-13 23:31:02,066][60934] Updated weights for policy 1, policy_version 62442 (0.0011) [2023-10-13 23:31:02,221][60935] Updated weights for policy 0, policy_version 62050 (0.0008) [2023-10-13 23:31:02,432][60934] Updated weights for policy 1, policy_version 62452 (0.0007) [2023-10-13 23:31:02,582][60935] Updated weights for policy 0, policy_version 62060 (0.0008) [2023-10-13 23:31:02,951][60935] Updated weights for policy 0, policy_version 62070 (0.0010) [2023-10-13 23:31:03,328][60935] Updated weights for policy 0, policy_version 62080 (0.0009) [2023-10-13 23:31:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 127827968. Throughput: 0: 1700.4, 1: 1701.6. Samples: 31967548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:31:06,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:31:06,576][60934] Updated weights for policy 1, policy_version 62462 (0.0010) [2023-10-13 23:31:06,943][60934] Updated weights for policy 1, policy_version 62472 (0.0009) [2023-10-13 23:31:07,312][60934] Updated weights for policy 1, policy_version 62482 (0.0008) [2023-10-13 23:31:07,356][60935] Updated weights for policy 0, policy_version 62090 (0.0009) [2023-10-13 23:31:07,723][60935] Updated weights for policy 0, policy_version 62100 (0.0007) [2023-10-13 23:31:08,086][60935] Updated weights for policy 0, policy_version 62110 (0.0010) [2023-10-13 23:31:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 127893504. Throughput: 0: 1696.6, 1: 1703.1. Samples: 31988372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:31:11,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:31:11,414][60934] Updated weights for policy 1, policy_version 62492 (0.0008) [2023-10-13 23:31:11,773][60934] Updated weights for policy 1, policy_version 62502 (0.0008) [2023-10-13 23:31:12,137][60934] Updated weights for policy 1, policy_version 62512 (0.0009) [2023-10-13 23:31:12,330][60935] Updated weights for policy 0, policy_version 62120 (0.0009) [2023-10-13 23:31:12,694][60935] Updated weights for policy 0, policy_version 62130 (0.0010) [2023-10-13 23:31:13,055][60935] Updated weights for policy 0, policy_version 62140 (0.0011) [2023-10-13 23:31:16,201][60934] Updated weights for policy 1, policy_version 62522 (0.0009) [2023-10-13 23:31:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 127959040. Throughput: 0: 1680.4, 1: 1700.1. Samples: 31997482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:31:16,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:31:16,574][60934] Updated weights for policy 1, policy_version 62532 (0.0007) [2023-10-13 23:31:16,928][60934] Updated weights for policy 1, policy_version 62542 (0.0009) [2023-10-13 23:31:16,957][60935] Updated weights for policy 0, policy_version 62150 (0.0008) [2023-10-13 23:31:17,302][60934] Updated weights for policy 1, policy_version 62552 (0.0009) [2023-10-13 23:31:17,320][60935] Updated weights for policy 0, policy_version 62160 (0.0008) [2023-10-13 23:31:17,697][60935] Updated weights for policy 0, policy_version 62170 (0.0011) [2023-10-13 23:31:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 128024576. Throughput: 0: 1703.7, 1: 1706.3. Samples: 32018576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:31:21,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.120')] [2023-10-13 23:31:21,357][60934] Updated weights for policy 1, policy_version 62562 (0.0007) [2023-10-13 23:31:21,721][60934] Updated weights for policy 1, policy_version 62572 (0.0007) [2023-10-13 23:31:21,722][60935] Updated weights for policy 0, policy_version 62180 (0.0010) [2023-10-13 23:31:22,079][60935] Updated weights for policy 0, policy_version 62190 (0.0008) [2023-10-13 23:31:22,089][60934] Updated weights for policy 1, policy_version 62582 (0.0008) [2023-10-13 23:31:22,452][60935] Updated weights for policy 0, policy_version 62200 (0.0010) [2023-10-13 23:31:26,125][60934] Updated weights for policy 1, policy_version 62592 (0.0010) [2023-10-13 23:31:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 128090112. Throughput: 0: 1705.6, 1: 1710.0. Samples: 32039872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:31:26,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.120')] [2023-10-13 23:31:26,468][60935] Updated weights for policy 0, policy_version 62210 (0.0008) [2023-10-13 23:31:26,507][60934] Updated weights for policy 1, policy_version 62602 (0.0010) [2023-10-13 23:31:26,838][60935] Updated weights for policy 0, policy_version 62220 (0.0008) [2023-10-13 23:31:26,866][60934] Updated weights for policy 1, policy_version 62612 (0.0010) [2023-10-13 23:31:27,201][60935] Updated weights for policy 0, policy_version 62230 (0.0008) [2023-10-13 23:31:27,578][60935] Updated weights for policy 0, policy_version 62240 (0.0007) [2023-10-13 23:31:30,889][60934] Updated weights for policy 1, policy_version 62622 (0.0008) [2023-10-13 23:31:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 128155648. Throughput: 0: 1704.1, 1: 1703.7. Samples: 32048838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:31:31,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.190')] [2023-10-13 23:31:31,258][60934] Updated weights for policy 1, policy_version 62632 (0.0007) [2023-10-13 23:31:31,546][60935] Updated weights for policy 0, policy_version 62250 (0.0010) [2023-10-13 23:31:31,626][60934] Updated weights for policy 1, policy_version 62642 (0.0008) [2023-10-13 23:31:31,917][60935] Updated weights for policy 0, policy_version 62260 (0.0009) [2023-10-13 23:31:32,292][60935] Updated weights for policy 0, policy_version 62270 (0.0010) [2023-10-13 23:31:35,826][60934] Updated weights for policy 1, policy_version 62652 (0.0009) [2023-10-13 23:31:36,193][60934] Updated weights for policy 1, policy_version 62662 (0.0007) [2023-10-13 23:31:36,237][60935] Updated weights for policy 0, policy_version 62280 (0.0009) [2023-10-13 23:31:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 128221184. Throughput: 0: 1714.4, 1: 1691.8. Samples: 32069762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:31:36,249][59943] Avg episode reward: [(0, '-0.180'), (1, '-0.210')] [2023-10-13 23:31:36,557][60934] Updated weights for policy 1, policy_version 62672 (0.0007) [2023-10-13 23:31:36,600][60935] Updated weights for policy 0, policy_version 62290 (0.0007) [2023-10-13 23:31:36,968][60935] Updated weights for policy 0, policy_version 62300 (0.0009) [2023-10-13 23:31:40,808][60934] Updated weights for policy 1, policy_version 62682 (0.0007) [2023-10-13 23:31:40,824][60935] Updated weights for policy 0, policy_version 62310 (0.0009) [2023-10-13 23:31:41,168][60934] Updated weights for policy 1, policy_version 62692 (0.0007) [2023-10-13 23:31:41,190][60935] Updated weights for policy 0, policy_version 62320 (0.0007) [2023-10-13 23:31:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 128286720. Throughput: 0: 1710.2, 1: 1684.0. Samples: 32090374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:31:41,249][59943] Avg episode reward: [(0, '-0.180'), (1, '-0.210')] [2023-10-13 23:31:41,533][60934] Updated weights for policy 1, policy_version 62702 (0.0009) [2023-10-13 23:31:41,562][60935] Updated weights for policy 0, policy_version 62330 (0.0008) [2023-10-13 23:31:41,901][60934] Updated weights for policy 1, policy_version 62712 (0.0008) [2023-10-13 23:31:45,565][60935] Updated weights for policy 0, policy_version 62340 (0.0008) [2023-10-13 23:31:45,806][60934] Updated weights for policy 1, policy_version 62722 (0.0008) [2023-10-13 23:31:45,935][60935] Updated weights for policy 0, policy_version 62350 (0.0007) [2023-10-13 23:31:46,174][60934] Updated weights for policy 1, policy_version 62732 (0.0009) [2023-10-13 23:31:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 128352256. Throughput: 0: 1718.0, 1: 1681.8. Samples: 32099740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:31:46,249][59943] Avg episode reward: [(0, '-0.180'), (1, '-0.210')] [2023-10-13 23:31:46,308][60935] Updated weights for policy 0, policy_version 62360 (0.0007) [2023-10-13 23:31:46,537][60934] Updated weights for policy 1, policy_version 62742 (0.0009) [2023-10-13 23:31:50,264][60935] Updated weights for policy 0, policy_version 62370 (0.0007) [2023-10-13 23:31:50,571][60934] Updated weights for policy 1, policy_version 62752 (0.0007) [2023-10-13 23:31:50,640][60935] Updated weights for policy 0, policy_version 62380 (0.0007) [2023-10-13 23:31:50,943][60934] Updated weights for policy 1, policy_version 62762 (0.0008) [2023-10-13 23:31:51,011][60935] Updated weights for policy 0, policy_version 62390 (0.0009) [2023-10-13 23:31:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 128417792. Throughput: 0: 1722.5, 1: 1681.0. Samples: 32120708. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:31:51,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:31:51,308][60934] Updated weights for policy 1, policy_version 62772 (0.0009) [2023-10-13 23:31:51,365][60935] Updated weights for policy 0, policy_version 62400 (0.0010) [2023-10-13 23:31:55,306][60934] Updated weights for policy 1, policy_version 62782 (0.0009) [2023-10-13 23:31:55,462][60935] Updated weights for policy 0, policy_version 62410 (0.0008) [2023-10-13 23:31:55,676][60934] Updated weights for policy 1, policy_version 62792 (0.0008) [2023-10-13 23:31:55,838][60935] Updated weights for policy 0, policy_version 62420 (0.0009) [2023-10-13 23:31:56,045][60934] Updated weights for policy 1, policy_version 62802 (0.0009) [2023-10-13 23:31:56,213][60935] Updated weights for policy 0, policy_version 62430 (0.0007) [2023-10-13 23:31:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 128483328. Throughput: 0: 1711.9, 1: 1667.7. Samples: 32140456. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:31:56,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:32:00,091][60934] Updated weights for policy 1, policy_version 62812 (0.0007) [2023-10-13 23:32:00,265][60935] Updated weights for policy 0, policy_version 62440 (0.0007) [2023-10-13 23:32:00,463][60934] Updated weights for policy 1, policy_version 62822 (0.0008) [2023-10-13 23:32:00,627][60935] Updated weights for policy 0, policy_version 62450 (0.0007) [2023-10-13 23:32:00,817][60934] Updated weights for policy 1, policy_version 62832 (0.0007) [2023-10-13 23:32:00,998][60935] Updated weights for policy 0, policy_version 62460 (0.0009) [2023-10-13 23:32:01,248][59943] Fps is (10 sec: 19660.7, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 128614400. Throughput: 0: 1731.8, 1: 1676.0. Samples: 32150834. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:32:01,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:32:04,831][60934] Updated weights for policy 1, policy_version 62842 (0.0007) [2023-10-13 23:32:04,876][60935] Updated weights for policy 0, policy_version 62470 (0.0009) [2023-10-13 23:32:05,184][60934] Updated weights for policy 1, policy_version 62852 (0.0007) [2023-10-13 23:32:05,240][60935] Updated weights for policy 0, policy_version 62480 (0.0007) [2023-10-13 23:32:05,547][60934] Updated weights for policy 1, policy_version 62862 (0.0008) [2023-10-13 23:32:05,601][60935] Updated weights for policy 0, policy_version 62490 (0.0009) [2023-10-13 23:32:05,924][60934] Updated weights for policy 1, policy_version 62872 (0.0007) [2023-10-13 23:32:06,248][59943] Fps is (10 sec: 19660.5, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 128679936. Throughput: 0: 1722.4, 1: 1679.6. Samples: 32171668. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:32:06,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:32:09,639][60935] Updated weights for policy 0, policy_version 62500 (0.0008) [2023-10-13 23:32:10,013][60935] Updated weights for policy 0, policy_version 62510 (0.0007) [2023-10-13 23:32:10,051][60934] Updated weights for policy 1, policy_version 62882 (0.0007) [2023-10-13 23:32:10,374][60935] Updated weights for policy 0, policy_version 62520 (0.0008) [2023-10-13 23:32:10,419][60934] Updated weights for policy 1, policy_version 62892 (0.0008) [2023-10-13 23:32:10,793][60934] Updated weights for policy 1, policy_version 62902 (0.0007) [2023-10-13 23:32:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 128745472. Throughput: 0: 1692.7, 1: 1651.4. Samples: 32190356. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:32:11,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:32:14,408][60935] Updated weights for policy 0, policy_version 62530 (0.0009) [2023-10-13 23:32:14,778][60935] Updated weights for policy 0, policy_version 62540 (0.0008) [2023-10-13 23:32:14,827][60934] Updated weights for policy 1, policy_version 62912 (0.0007) [2023-10-13 23:32:15,141][60935] Updated weights for policy 0, policy_version 62550 (0.0009) [2023-10-13 23:32:15,198][60934] Updated weights for policy 1, policy_version 62922 (0.0008) [2023-10-13 23:32:15,502][60935] Updated weights for policy 0, policy_version 62560 (0.0008) [2023-10-13 23:32:15,567][60934] Updated weights for policy 1, policy_version 62932 (0.0007) [2023-10-13 23:32:16,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 128811008. Throughput: 0: 1718.2, 1: 1682.9. Samples: 32201890. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:32:16,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:32:19,623][60935] Updated weights for policy 0, policy_version 62570 (0.0008) [2023-10-13 23:32:19,683][60934] Updated weights for policy 1, policy_version 62942 (0.0009) [2023-10-13 23:32:19,991][60935] Updated weights for policy 0, policy_version 62580 (0.0008) [2023-10-13 23:32:20,053][60934] Updated weights for policy 1, policy_version 62952 (0.0008) [2023-10-13 23:32:20,364][60935] Updated weights for policy 0, policy_version 62590 (0.0009) [2023-10-13 23:32:20,416][60934] Updated weights for policy 1, policy_version 62962 (0.0007) [2023-10-13 23:32:21,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 128876544. Throughput: 0: 1696.6, 1: 1680.4. Samples: 32221730. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:32:21,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:32:24,240][60935] Updated weights for policy 0, policy_version 62600 (0.0010) [2023-10-13 23:32:24,448][60934] Updated weights for policy 1, policy_version 62972 (0.0007) [2023-10-13 23:32:24,614][60935] Updated weights for policy 0, policy_version 62610 (0.0011) [2023-10-13 23:32:24,808][60934] Updated weights for policy 1, policy_version 62982 (0.0009) [2023-10-13 23:32:24,979][60935] Updated weights for policy 0, policy_version 62620 (0.0008) [2023-10-13 23:32:25,176][60934] Updated weights for policy 1, policy_version 62992 (0.0009) [2023-10-13 23:32:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 128942080. Throughput: 0: 1681.3, 1: 1663.5. Samples: 32240890. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-13 23:32:26,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:32:29,086][60934] Updated weights for policy 1, policy_version 63002 (0.0010) [2023-10-13 23:32:29,111][60935] Updated weights for policy 0, policy_version 62630 (0.0009) [2023-10-13 23:32:29,455][60934] Updated weights for policy 1, policy_version 63012 (0.0007) [2023-10-13 23:32:29,473][60935] Updated weights for policy 0, policy_version 62640 (0.0009) [2023-10-13 23:32:29,817][60934] Updated weights for policy 1, policy_version 63022 (0.0009) [2023-10-13 23:32:29,849][60935] Updated weights for policy 0, policy_version 62650 (0.0009) [2023-10-13 23:32:30,186][60934] Updated weights for policy 1, policy_version 63032 (0.0008) [2023-10-13 23:32:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 129007616. Throughput: 0: 1705.2, 1: 1693.9. Samples: 32252704. Policy #0 lag: (min: 6.0, avg: 6.0, max: 8.0) [2023-10-13 23:32:31,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:32:33,681][60935] Updated weights for policy 0, policy_version 62660 (0.0010) [2023-10-13 23:32:34,049][60935] Updated weights for policy 0, policy_version 62670 (0.0008) [2023-10-13 23:32:34,310][60934] Updated weights for policy 1, policy_version 63042 (0.0009) [2023-10-13 23:32:34,421][60935] Updated weights for policy 0, policy_version 62680 (0.0007) [2023-10-13 23:32:34,678][60934] Updated weights for policy 1, policy_version 63052 (0.0007) [2023-10-13 23:32:35,042][60934] Updated weights for policy 1, policy_version 63062 (0.0009) [2023-10-13 23:32:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 129073152. Throughput: 0: 1676.8, 1: 1680.4. Samples: 32271782. Policy #0 lag: (min: 6.0, avg: 6.0, max: 8.0) [2023-10-13 23:32:36,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:32:38,565][60935] Updated weights for policy 0, policy_version 62690 (0.0008) [2023-10-13 23:32:38,940][60935] Updated weights for policy 0, policy_version 62700 (0.0011) [2023-10-13 23:32:39,078][60934] Updated weights for policy 1, policy_version 63072 (0.0010) [2023-10-13 23:32:39,304][60935] Updated weights for policy 0, policy_version 62710 (0.0009) [2023-10-13 23:32:39,445][60934] Updated weights for policy 1, policy_version 63082 (0.0009) [2023-10-13 23:32:39,667][60935] Updated weights for policy 0, policy_version 62720 (0.0008) [2023-10-13 23:32:39,811][60934] Updated weights for policy 1, policy_version 63092 (0.0010) [2023-10-13 23:32:41,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 129138688. Throughput: 0: 1691.2, 1: 1680.5. Samples: 32292184. Policy #0 lag: (min: 6.0, avg: 6.0, max: 8.0) [2023-10-13 23:32:41,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:32:43,631][60935] Updated weights for policy 0, policy_version 62730 (0.0010) [2023-10-13 23:32:43,887][60934] Updated weights for policy 1, policy_version 63102 (0.0008) [2023-10-13 23:32:43,995][60935] Updated weights for policy 0, policy_version 62740 (0.0008) [2023-10-13 23:32:44,241][60934] Updated weights for policy 1, policy_version 63112 (0.0008) [2023-10-13 23:32:44,364][60935] Updated weights for policy 0, policy_version 62750 (0.0008) [2023-10-13 23:32:44,612][60934] Updated weights for policy 1, policy_version 63122 (0.0008) [2023-10-13 23:32:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 129204224. Throughput: 0: 1691.9, 1: 1698.6. Samples: 32303406. Policy #0 lag: (min: 6.0, avg: 6.0, max: 8.0) [2023-10-13 23:32:46,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:32:48,429][60935] Updated weights for policy 0, policy_version 62760 (0.0009) [2023-10-13 23:32:48,795][60935] Updated weights for policy 0, policy_version 62770 (0.0011) [2023-10-13 23:32:48,820][60934] Updated weights for policy 1, policy_version 63132 (0.0008) [2023-10-13 23:32:49,154][60935] Updated weights for policy 0, policy_version 62780 (0.0010) [2023-10-13 23:32:49,176][60934] Updated weights for policy 1, policy_version 63142 (0.0007) [2023-10-13 23:32:49,552][60934] Updated weights for policy 1, policy_version 63152 (0.0007) [2023-10-13 23:32:51,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 129269760. Throughput: 0: 1680.7, 1: 1676.7. Samples: 32322750. Policy #0 lag: (min: 6.0, avg: 6.0, max: 8.0) [2023-10-13 23:32:51,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:32:53,266][60935] Updated weights for policy 0, policy_version 62790 (0.0009) [2023-10-13 23:32:53,559][60934] Updated weights for policy 1, policy_version 63162 (0.0007) [2023-10-13 23:32:53,643][60935] Updated weights for policy 0, policy_version 62800 (0.0008) [2023-10-13 23:32:53,922][60934] Updated weights for policy 1, policy_version 63172 (0.0008) [2023-10-13 23:32:54,003][60935] Updated weights for policy 0, policy_version 62810 (0.0007) [2023-10-13 23:32:54,292][60934] Updated weights for policy 1, policy_version 63182 (0.0008) [2023-10-13 23:32:54,646][60934] Updated weights for policy 1, policy_version 63192 (0.0011) [2023-10-13 23:32:56,249][59943] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 129335296. Throughput: 0: 1707.9, 1: 1690.1. Samples: 32343268. Policy #0 lag: (min: 6.0, avg: 6.0, max: 8.0) [2023-10-13 23:32:56,250][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:32:56,259][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000062816_64323584.pth... [2023-10-13 23:32:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000063192_65011712.pth... [2023-10-13 23:32:56,295][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000061624_63406080.pth [2023-10-13 23:32:56,299][60828] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p1/milestones/checkpoint_000063192_65011712.pth [2023-10-13 23:32:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000061216_62685184.pth [2023-10-13 23:32:56,306][60695] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p0/milestones/checkpoint_000062816_64323584.pth [2023-10-13 23:32:57,959][60935] Updated weights for policy 0, policy_version 62820 (0.0008) [2023-10-13 23:32:58,331][60935] Updated weights for policy 0, policy_version 62830 (0.0008) [2023-10-13 23:32:58,696][60935] Updated weights for policy 0, policy_version 62840 (0.0007) [2023-10-13 23:32:58,791][60934] Updated weights for policy 1, policy_version 63202 (0.0007) [2023-10-13 23:32:59,159][60934] Updated weights for policy 1, policy_version 63212 (0.0008) [2023-10-13 23:32:59,528][60934] Updated weights for policy 1, policy_version 63222 (0.0007) [2023-10-13 23:33:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 129400832. Throughput: 0: 1687.1, 1: 1690.4. Samples: 32353876. Policy #0 lag: (min: 6.0, avg: 6.0, max: 8.0) [2023-10-13 23:33:01,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:33:02,532][60935] Updated weights for policy 0, policy_version 62850 (0.0008) [2023-10-13 23:33:02,894][60935] Updated weights for policy 0, policy_version 62860 (0.0007) [2023-10-13 23:33:03,270][60935] Updated weights for policy 0, policy_version 62870 (0.0009) [2023-10-13 23:33:03,512][60934] Updated weights for policy 1, policy_version 63232 (0.0009) [2023-10-13 23:33:03,631][60935] Updated weights for policy 0, policy_version 62880 (0.0007) [2023-10-13 23:33:03,872][60934] Updated weights for policy 1, policy_version 63242 (0.0010) [2023-10-13 23:33:04,237][60934] Updated weights for policy 1, policy_version 63252 (0.0010) [2023-10-13 23:33:06,248][59943] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 129466368. Throughput: 0: 1702.8, 1: 1671.6. Samples: 32373582. Policy #0 lag: (min: 6.0, avg: 6.0, max: 8.0) [2023-10-13 23:33:06,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:33:07,833][60935] Updated weights for policy 0, policy_version 62890 (0.0009) [2023-10-13 23:33:08,196][60935] Updated weights for policy 0, policy_version 62900 (0.0009) [2023-10-13 23:33:08,198][60934] Updated weights for policy 1, policy_version 63262 (0.0010) [2023-10-13 23:33:08,560][60935] Updated weights for policy 0, policy_version 62910 (0.0008) [2023-10-13 23:33:08,566][60934] Updated weights for policy 1, policy_version 63272 (0.0007) [2023-10-13 23:33:08,935][60934] Updated weights for policy 1, policy_version 63282 (0.0010) [2023-10-13 23:33:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 129531904. Throughput: 0: 1716.5, 1: 1693.6. Samples: 32394344. Policy #0 lag: (min: 6.0, avg: 6.0, max: 8.0) [2023-10-13 23:33:11,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:33:12,584][60935] Updated weights for policy 0, policy_version 62920 (0.0011) [2023-10-13 23:33:12,900][60934] Updated weights for policy 1, policy_version 63292 (0.0008) [2023-10-13 23:33:12,961][60935] Updated weights for policy 0, policy_version 62930 (0.0009) [2023-10-13 23:33:13,270][60934] Updated weights for policy 1, policy_version 63302 (0.0007) [2023-10-13 23:33:13,326][60935] Updated weights for policy 0, policy_version 62940 (0.0007) [2023-10-13 23:33:13,634][60934] Updated weights for policy 1, policy_version 63312 (0.0007) [2023-10-13 23:33:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 129597440. Throughput: 0: 1687.1, 1: 1674.6. Samples: 32403978. Policy #0 lag: (min: 21.0, avg: 23.0, max: 51.0) [2023-10-13 23:33:16,248][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:33:16,992][60935] Updated weights for policy 0, policy_version 62950 (0.0007) [2023-10-13 23:33:17,360][60935] Updated weights for policy 0, policy_version 62960 (0.0009) [2023-10-13 23:33:17,672][60934] Updated weights for policy 1, policy_version 63322 (0.0008) [2023-10-13 23:33:17,737][60935] Updated weights for policy 0, policy_version 62970 (0.0007) [2023-10-13 23:33:18,032][60934] Updated weights for policy 1, policy_version 63332 (0.0008) [2023-10-13 23:33:18,406][60934] Updated weights for policy 1, policy_version 63342 (0.0009) [2023-10-13 23:33:18,774][60934] Updated weights for policy 1, policy_version 63352 (0.0010) [2023-10-13 23:33:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 129662976. Throughput: 0: 1727.5, 1: 1675.7. Samples: 32424926. Policy #0 lag: (min: 21.0, avg: 23.0, max: 51.0) [2023-10-13 23:33:21,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:33:21,664][60935] Updated weights for policy 0, policy_version 62980 (0.0008) [2023-10-13 23:33:22,029][60935] Updated weights for policy 0, policy_version 62990 (0.0009) [2023-10-13 23:33:22,406][60935] Updated weights for policy 0, policy_version 63000 (0.0009) [2023-10-13 23:33:22,923][60934] Updated weights for policy 1, policy_version 63362 (0.0009) [2023-10-13 23:33:23,292][60934] Updated weights for policy 1, policy_version 63372 (0.0007) [2023-10-13 23:33:23,657][60934] Updated weights for policy 1, policy_version 63382 (0.0008) [2023-10-13 23:33:26,198][60935] Updated weights for policy 0, policy_version 63010 (0.0010) [2023-10-13 23:33:26,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 129728512. Throughput: 0: 1731.2, 1: 1691.6. Samples: 32446210. Policy #0 lag: (min: 21.0, avg: 23.0, max: 51.0) [2023-10-13 23:33:26,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:33:26,558][60935] Updated weights for policy 0, policy_version 63020 (0.0009) [2023-10-13 23:33:26,923][60935] Updated weights for policy 0, policy_version 63030 (0.0009) [2023-10-13 23:33:27,298][60935] Updated weights for policy 0, policy_version 63040 (0.0010) [2023-10-13 23:33:27,649][60934] Updated weights for policy 1, policy_version 63392 (0.0008) [2023-10-13 23:33:28,020][60934] Updated weights for policy 1, policy_version 63402 (0.0007) [2023-10-13 23:33:28,387][60934] Updated weights for policy 1, policy_version 63412 (0.0008) [2023-10-13 23:33:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 129794048. Throughput: 0: 1711.8, 1: 1666.5. Samples: 32455426. Policy #0 lag: (min: 21.0, avg: 23.0, max: 51.0) [2023-10-13 23:33:31,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:33:31,348][60935] Updated weights for policy 0, policy_version 63050 (0.0007) [2023-10-13 23:33:31,724][60935] Updated weights for policy 0, policy_version 63060 (0.0007) [2023-10-13 23:33:32,093][60935] Updated weights for policy 0, policy_version 63070 (0.0009) [2023-10-13 23:33:32,493][60934] Updated weights for policy 1, policy_version 63422 (0.0007) [2023-10-13 23:33:32,856][60934] Updated weights for policy 1, policy_version 63432 (0.0007) [2023-10-13 23:33:33,218][60934] Updated weights for policy 1, policy_version 63442 (0.0008) [2023-10-13 23:33:35,961][60935] Updated weights for policy 0, policy_version 63080 (0.0008) [2023-10-13 23:33:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 129859584. Throughput: 0: 1731.6, 1: 1690.7. Samples: 32476756. Policy #0 lag: (min: 21.0, avg: 23.0, max: 51.0) [2023-10-13 23:33:36,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:33:36,338][60935] Updated weights for policy 0, policy_version 63090 (0.0008) [2023-10-13 23:33:36,713][60935] Updated weights for policy 0, policy_version 63100 (0.0009) [2023-10-13 23:33:37,119][60934] Updated weights for policy 1, policy_version 63452 (0.0009) [2023-10-13 23:33:37,483][60934] Updated weights for policy 1, policy_version 63462 (0.0010) [2023-10-13 23:33:37,862][60934] Updated weights for policy 1, policy_version 63472 (0.0009) [2023-10-13 23:33:40,677][60935] Updated weights for policy 0, policy_version 63110 (0.0009) [2023-10-13 23:33:41,046][60935] Updated weights for policy 0, policy_version 63120 (0.0009) [2023-10-13 23:33:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 129925120. Throughput: 0: 1726.0, 1: 1701.8. Samples: 32497518. Policy #0 lag: (min: 21.0, avg: 23.0, max: 51.0) [2023-10-13 23:33:41,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:33:41,412][60935] Updated weights for policy 0, policy_version 63130 (0.0009) [2023-10-13 23:33:41,828][60934] Updated weights for policy 1, policy_version 63482 (0.0009) [2023-10-13 23:33:42,189][60934] Updated weights for policy 1, policy_version 63492 (0.0007) [2023-10-13 23:33:42,555][60934] Updated weights for policy 1, policy_version 63502 (0.0007) [2023-10-13 23:33:42,926][60934] Updated weights for policy 1, policy_version 63512 (0.0007) [2023-10-13 23:33:45,196][60935] Updated weights for policy 0, policy_version 63140 (0.0009) [2023-10-13 23:33:45,553][60935] Updated weights for policy 0, policy_version 63150 (0.0010) [2023-10-13 23:33:45,925][60935] Updated weights for policy 0, policy_version 63160 (0.0011) [2023-10-13 23:33:46,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 130023424. Throughput: 0: 1731.2, 1: 1677.6. Samples: 32507276. Policy #0 lag: (min: 21.0, avg: 23.0, max: 51.0) [2023-10-13 23:33:46,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:33:47,023][60934] Updated weights for policy 1, policy_version 63522 (0.0007) [2023-10-13 23:33:47,389][60934] Updated weights for policy 1, policy_version 63532 (0.0007) [2023-10-13 23:33:47,756][60934] Updated weights for policy 1, policy_version 63542 (0.0007) [2023-10-13 23:33:49,895][60935] Updated weights for policy 0, policy_version 63170 (0.0010) [2023-10-13 23:33:50,265][60935] Updated weights for policy 0, policy_version 63180 (0.0007) [2023-10-13 23:33:50,638][60935] Updated weights for policy 0, policy_version 63190 (0.0007) [2023-10-13 23:33:51,017][60935] Updated weights for policy 0, policy_version 63200 (0.0010) [2023-10-13 23:33:51,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 130088960. Throughput: 0: 1734.9, 1: 1700.2. Samples: 32528160. Policy #0 lag: (min: 21.0, avg: 23.0, max: 51.0) [2023-10-13 23:33:51,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:33:51,921][60934] Updated weights for policy 1, policy_version 63552 (0.0009) [2023-10-13 23:33:52,301][60934] Updated weights for policy 1, policy_version 63562 (0.0009) [2023-10-13 23:33:52,670][60934] Updated weights for policy 1, policy_version 63572 (0.0008) [2023-10-13 23:33:55,051][60935] Updated weights for policy 0, policy_version 63210 (0.0010) [2023-10-13 23:33:55,429][60935] Updated weights for policy 0, policy_version 63220 (0.0008) [2023-10-13 23:33:55,791][60935] Updated weights for policy 0, policy_version 63230 (0.0008) [2023-10-13 23:33:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 130154496. Throughput: 0: 1712.8, 1: 1694.4. Samples: 32547670. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:33:56,249][59943] Avg episode reward: [(0, '-0.180'), (1, '0.000')] [2023-10-13 23:33:56,753][60934] Updated weights for policy 1, policy_version 63582 (0.0007) [2023-10-13 23:33:57,118][60934] Updated weights for policy 1, policy_version 63592 (0.0010) [2023-10-13 23:33:57,482][60934] Updated weights for policy 1, policy_version 63602 (0.0008) [2023-10-13 23:33:59,800][60935] Updated weights for policy 0, policy_version 63240 (0.0008) [2023-10-13 23:34:00,174][60935] Updated weights for policy 0, policy_version 63250 (0.0009) [2023-10-13 23:34:00,537][60935] Updated weights for policy 0, policy_version 63260 (0.0009) [2023-10-13 23:34:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 130220032. Throughput: 0: 1743.0, 1: 1683.9. Samples: 32558188. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:34:01,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:34:01,560][60934] Updated weights for policy 1, policy_version 63612 (0.0008) [2023-10-13 23:34:01,923][60934] Updated weights for policy 1, policy_version 63622 (0.0009) [2023-10-13 23:34:02,292][60934] Updated weights for policy 1, policy_version 63632 (0.0009) [2023-10-13 23:34:04,448][60935] Updated weights for policy 0, policy_version 63270 (0.0009) [2023-10-13 23:34:04,807][60935] Updated weights for policy 0, policy_version 63280 (0.0010) [2023-10-13 23:34:05,179][60935] Updated weights for policy 0, policy_version 63290 (0.0010) [2023-10-13 23:34:06,215][60934] Updated weights for policy 1, policy_version 63642 (0.0007) [2023-10-13 23:34:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 130285568. Throughput: 0: 1718.8, 1: 1698.6. Samples: 32578706. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:34:06,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:34:06,578][60934] Updated weights for policy 1, policy_version 63652 (0.0011) [2023-10-13 23:34:06,942][60934] Updated weights for policy 1, policy_version 63662 (0.0009) [2023-10-13 23:34:07,313][60934] Updated weights for policy 1, policy_version 63672 (0.0008) [2023-10-13 23:34:09,185][60935] Updated weights for policy 0, policy_version 63300 (0.0009) [2023-10-13 23:34:09,561][60935] Updated weights for policy 0, policy_version 63310 (0.0010) [2023-10-13 23:34:09,925][60935] Updated weights for policy 0, policy_version 63320 (0.0009) [2023-10-13 23:34:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 130351104. Throughput: 0: 1702.3, 1: 1692.0. Samples: 32598950. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:34:11,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:34:11,430][60934] Updated weights for policy 1, policy_version 63682 (0.0010) [2023-10-13 23:34:11,795][60934] Updated weights for policy 1, policy_version 63692 (0.0011) [2023-10-13 23:34:12,173][60934] Updated weights for policy 1, policy_version 63702 (0.0008) [2023-10-13 23:34:13,863][60935] Updated weights for policy 0, policy_version 63330 (0.0008) [2023-10-13 23:34:14,230][60935] Updated weights for policy 0, policy_version 63340 (0.0010) [2023-10-13 23:34:14,597][60935] Updated weights for policy 0, policy_version 63350 (0.0008) [2023-10-13 23:34:14,967][60935] Updated weights for policy 0, policy_version 63360 (0.0008) [2023-10-13 23:34:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 130416640. Throughput: 0: 1730.4, 1: 1690.0. Samples: 32609342. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:34:16,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:34:16,277][60934] Updated weights for policy 1, policy_version 63712 (0.0010) [2023-10-13 23:34:16,644][60934] Updated weights for policy 1, policy_version 63722 (0.0008) [2023-10-13 23:34:17,003][60934] Updated weights for policy 1, policy_version 63732 (0.0008) [2023-10-13 23:34:18,995][60935] Updated weights for policy 0, policy_version 63370 (0.0009) [2023-10-13 23:34:19,358][60935] Updated weights for policy 0, policy_version 63380 (0.0007) [2023-10-13 23:34:19,733][60935] Updated weights for policy 0, policy_version 63390 (0.0008) [2023-10-13 23:34:21,037][60934] Updated weights for policy 1, policy_version 63742 (0.0009) [2023-10-13 23:34:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 130482176. Throughput: 0: 1698.5, 1: 1687.8. Samples: 32629140. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:34:21,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:34:21,410][60934] Updated weights for policy 1, policy_version 63752 (0.0007) [2023-10-13 23:34:21,776][60934] Updated weights for policy 1, policy_version 63762 (0.0010) [2023-10-13 23:34:23,703][60935] Updated weights for policy 0, policy_version 63400 (0.0010) [2023-10-13 23:34:24,079][60935] Updated weights for policy 0, policy_version 63410 (0.0008) [2023-10-13 23:34:24,449][60935] Updated weights for policy 0, policy_version 63420 (0.0009) [2023-10-13 23:34:25,953][60934] Updated weights for policy 1, policy_version 63772 (0.0009) [2023-10-13 23:34:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 130547712. Throughput: 0: 1708.6, 1: 1688.6. Samples: 32650392. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:34:26,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:34:26,318][60934] Updated weights for policy 1, policy_version 63782 (0.0009) [2023-10-13 23:34:26,686][60934] Updated weights for policy 1, policy_version 63792 (0.0007) [2023-10-13 23:34:28,346][60935] Updated weights for policy 0, policy_version 63430 (0.0008) [2023-10-13 23:34:28,721][60935] Updated weights for policy 0, policy_version 63440 (0.0008) [2023-10-13 23:34:29,088][60935] Updated weights for policy 0, policy_version 63450 (0.0008) [2023-10-13 23:34:30,715][60934] Updated weights for policy 1, policy_version 63802 (0.0009) [2023-10-13 23:34:31,081][60934] Updated weights for policy 1, policy_version 63812 (0.0011) [2023-10-13 23:34:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 130613248. Throughput: 0: 1711.7, 1: 1685.6. Samples: 32660158. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:34:31,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:34:31,457][60934] Updated weights for policy 1, policy_version 63822 (0.0009) [2023-10-13 23:34:31,809][60934] Updated weights for policy 1, policy_version 63832 (0.0009) [2023-10-13 23:34:33,002][60935] Updated weights for policy 0, policy_version 63460 (0.0008) [2023-10-13 23:34:33,383][60935] Updated weights for policy 0, policy_version 63470 (0.0009) [2023-10-13 23:34:33,747][60935] Updated weights for policy 0, policy_version 63480 (0.0008) [2023-10-13 23:34:35,861][60934] Updated weights for policy 1, policy_version 63842 (0.0008) [2023-10-13 23:34:36,222][60934] Updated weights for policy 1, policy_version 63852 (0.0010) [2023-10-13 23:34:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 130678784. Throughput: 0: 1697.5, 1: 1692.6. Samples: 32680714. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:34:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:34:36,588][60934] Updated weights for policy 1, policy_version 63862 (0.0010) [2023-10-13 23:34:37,758][60935] Updated weights for policy 0, policy_version 63490 (0.0008) [2023-10-13 23:34:38,135][60935] Updated weights for policy 0, policy_version 63500 (0.0010) [2023-10-13 23:34:38,495][60935] Updated weights for policy 0, policy_version 63510 (0.0010) [2023-10-13 23:34:38,858][60935] Updated weights for policy 0, policy_version 63520 (0.0009) [2023-10-13 23:34:40,592][60934] Updated weights for policy 1, policy_version 63872 (0.0009) [2023-10-13 23:34:40,958][60934] Updated weights for policy 1, policy_version 63882 (0.0007) [2023-10-13 23:34:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 130744320. Throughput: 0: 1730.0, 1: 1692.7. Samples: 32701690. Policy #0 lag: (min: 0.0, avg: 23.2, max: 32.0) [2023-10-13 23:34:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:34:41,324][60934] Updated weights for policy 1, policy_version 63892 (0.0007) [2023-10-13 23:34:42,906][60935] Updated weights for policy 0, policy_version 63530 (0.0007) [2023-10-13 23:34:43,272][60935] Updated weights for policy 0, policy_version 63540 (0.0007) [2023-10-13 23:34:43,650][60935] Updated weights for policy 0, policy_version 63550 (0.0007) [2023-10-13 23:34:45,302][60934] Updated weights for policy 1, policy_version 63902 (0.0007) [2023-10-13 23:34:45,669][60934] Updated weights for policy 1, policy_version 63912 (0.0009) [2023-10-13 23:34:46,035][60934] Updated weights for policy 1, policy_version 63922 (0.0009) [2023-10-13 23:34:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 130809856. Throughput: 0: 1697.7, 1: 1700.3. Samples: 32711098. Policy #0 lag: (min: 0.0, avg: 23.2, max: 32.0) [2023-10-13 23:34:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:34:47,507][60935] Updated weights for policy 0, policy_version 63560 (0.0010) [2023-10-13 23:34:47,890][60935] Updated weights for policy 0, policy_version 63570 (0.0009) [2023-10-13 23:34:48,244][60935] Updated weights for policy 0, policy_version 63580 (0.0010) [2023-10-13 23:34:49,873][60934] Updated weights for policy 1, policy_version 63932 (0.0008) [2023-10-13 23:34:50,242][60934] Updated weights for policy 1, policy_version 63942 (0.0008) [2023-10-13 23:34:50,607][60934] Updated weights for policy 1, policy_version 63952 (0.0008) [2023-10-13 23:34:51,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 130908160. Throughput: 0: 1716.9, 1: 1700.2. Samples: 32732476. Policy #0 lag: (min: 0.0, avg: 23.2, max: 32.0) [2023-10-13 23:34:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:34:52,210][60935] Updated weights for policy 0, policy_version 63590 (0.0009) [2023-10-13 23:34:52,572][60935] Updated weights for policy 0, policy_version 63600 (0.0011) [2023-10-13 23:34:52,940][60935] Updated weights for policy 0, policy_version 63610 (0.0010) [2023-10-13 23:34:54,715][60934] Updated weights for policy 1, policy_version 63962 (0.0008) [2023-10-13 23:34:55,093][60934] Updated weights for policy 1, policy_version 63972 (0.0008) [2023-10-13 23:34:55,455][60934] Updated weights for policy 1, policy_version 63982 (0.0010) [2023-10-13 23:34:55,828][60934] Updated weights for policy 1, policy_version 63992 (0.0009) [2023-10-13 23:34:56,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 130973696. Throughput: 0: 1734.0, 1: 1684.7. Samples: 32752792. Policy #0 lag: (min: 0.0, avg: 23.2, max: 32.0) [2023-10-13 23:34:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:34:56,258][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000063992_65830912.pth... [2023-10-13 23:34:56,258][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000063616_65142784.pth... [2023-10-13 23:34:56,299][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000062392_64192512.pth [2023-10-13 23:34:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000062016_63504384.pth [2023-10-13 23:34:56,867][60935] Updated weights for policy 0, policy_version 63620 (0.0008) [2023-10-13 23:34:57,223][60935] Updated weights for policy 0, policy_version 63630 (0.0009) [2023-10-13 23:34:57,595][60935] Updated weights for policy 0, policy_version 63640 (0.0010) [2023-10-13 23:34:59,849][60934] Updated weights for policy 1, policy_version 64002 (0.0011) [2023-10-13 23:35:00,218][60934] Updated weights for policy 1, policy_version 64012 (0.0008) [2023-10-13 23:35:00,589][60934] Updated weights for policy 1, policy_version 64022 (0.0009) [2023-10-13 23:35:01,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 131039232. Throughput: 0: 1707.7, 1: 1709.1. Samples: 32763100. Policy #0 lag: (min: 0.0, avg: 23.2, max: 32.0) [2023-10-13 23:35:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:35:01,628][60935] Updated weights for policy 0, policy_version 63650 (0.0009) [2023-10-13 23:35:01,994][60935] Updated weights for policy 0, policy_version 63660 (0.0009) [2023-10-13 23:35:02,362][60935] Updated weights for policy 0, policy_version 63670 (0.0009) [2023-10-13 23:35:02,730][60935] Updated weights for policy 0, policy_version 63680 (0.0007) [2023-10-13 23:35:04,527][60934] Updated weights for policy 1, policy_version 64032 (0.0007) [2023-10-13 23:35:04,893][60934] Updated weights for policy 1, policy_version 64042 (0.0008) [2023-10-13 23:35:05,271][60934] Updated weights for policy 1, policy_version 64052 (0.0007) [2023-10-13 23:35:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 131104768. Throughput: 0: 1740.6, 1: 1704.4. Samples: 32784164. Policy #0 lag: (min: 0.0, avg: 23.2, max: 32.0) [2023-10-13 23:35:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:35:06,568][60935] Updated weights for policy 0, policy_version 63690 (0.0007) [2023-10-13 23:35:06,941][60935] Updated weights for policy 0, policy_version 63700 (0.0009) [2023-10-13 23:35:07,310][60935] Updated weights for policy 0, policy_version 63710 (0.0008) [2023-10-13 23:35:09,291][60934] Updated weights for policy 1, policy_version 64062 (0.0010) [2023-10-13 23:35:09,654][60934] Updated weights for policy 1, policy_version 64072 (0.0010) [2023-10-13 23:35:10,029][60934] Updated weights for policy 1, policy_version 64082 (0.0011) [2023-10-13 23:35:11,215][60935] Updated weights for policy 0, policy_version 63720 (0.0010) [2023-10-13 23:35:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 131170304. Throughput: 0: 1741.7, 1: 1682.4. Samples: 32804474. Policy #0 lag: (min: 0.0, avg: 23.2, max: 32.0) [2023-10-13 23:35:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:35:11,585][60935] Updated weights for policy 0, policy_version 63730 (0.0011) [2023-10-13 23:35:11,949][60935] Updated weights for policy 0, policy_version 63740 (0.0010) [2023-10-13 23:35:14,111][60934] Updated weights for policy 1, policy_version 64092 (0.0008) [2023-10-13 23:35:14,478][60934] Updated weights for policy 1, policy_version 64102 (0.0008) [2023-10-13 23:35:14,847][60934] Updated weights for policy 1, policy_version 64112 (0.0008) [2023-10-13 23:35:15,901][60935] Updated weights for policy 0, policy_version 63750 (0.0009) [2023-10-13 23:35:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 131235840. Throughput: 0: 1728.2, 1: 1715.6. Samples: 32815126. Policy #0 lag: (min: 0.0, avg: 23.2, max: 32.0) [2023-10-13 23:35:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:35:16,266][60935] Updated weights for policy 0, policy_version 63760 (0.0008) [2023-10-13 23:35:16,649][60935] Updated weights for policy 0, policy_version 63770 (0.0008) [2023-10-13 23:35:18,664][60934] Updated weights for policy 1, policy_version 64122 (0.0009) [2023-10-13 23:35:19,038][60934] Updated weights for policy 1, policy_version 64132 (0.0007) [2023-10-13 23:35:19,400][60934] Updated weights for policy 1, policy_version 64142 (0.0007) [2023-10-13 23:35:19,759][60934] Updated weights for policy 1, policy_version 64152 (0.0007) [2023-10-13 23:35:20,607][60935] Updated weights for policy 0, policy_version 63780 (0.0007) [2023-10-13 23:35:20,975][60935] Updated weights for policy 0, policy_version 63790 (0.0011) [2023-10-13 23:35:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 131301376. Throughput: 0: 1739.2, 1: 1692.8. Samples: 32835154. Policy #0 lag: (min: 0.0, avg: 23.2, max: 32.0) [2023-10-13 23:35:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:35:21,345][60935] Updated weights for policy 0, policy_version 63800 (0.0010) [2023-10-13 23:35:23,778][60934] Updated weights for policy 1, policy_version 64162 (0.0008) [2023-10-13 23:35:24,139][60934] Updated weights for policy 1, policy_version 64172 (0.0011) [2023-10-13 23:35:24,501][60934] Updated weights for policy 1, policy_version 64182 (0.0010) [2023-10-13 23:35:25,314][60935] Updated weights for policy 0, policy_version 63810 (0.0008) [2023-10-13 23:35:25,698][60935] Updated weights for policy 0, policy_version 63820 (0.0011) [2023-10-13 23:35:26,076][60935] Updated weights for policy 0, policy_version 63830 (0.0012) [2023-10-13 23:35:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 131366912. Throughput: 0: 1721.1, 1: 1692.8. Samples: 32855312. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 23:35:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:35:26,441][60935] Updated weights for policy 0, policy_version 63840 (0.0010) [2023-10-13 23:35:28,750][60934] Updated weights for policy 1, policy_version 64192 (0.0011) [2023-10-13 23:35:29,120][60934] Updated weights for policy 1, policy_version 64202 (0.0011) [2023-10-13 23:35:29,487][60934] Updated weights for policy 1, policy_version 64212 (0.0010) [2023-10-13 23:35:30,613][60935] Updated weights for policy 0, policy_version 63850 (0.0008) [2023-10-13 23:35:30,985][60935] Updated weights for policy 0, policy_version 63860 (0.0008) [2023-10-13 23:35:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 131432448. Throughput: 0: 1738.1, 1: 1707.6. Samples: 32866154. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 23:35:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:35:31,356][60935] Updated weights for policy 0, policy_version 63870 (0.0007) [2023-10-13 23:35:33,561][60934] Updated weights for policy 1, policy_version 64222 (0.0008) [2023-10-13 23:35:33,932][60934] Updated weights for policy 1, policy_version 64232 (0.0008) [2023-10-13 23:35:34,301][60934] Updated weights for policy 1, policy_version 64242 (0.0008) [2023-10-13 23:35:35,286][60935] Updated weights for policy 0, policy_version 63880 (0.0010) [2023-10-13 23:35:35,651][60935] Updated weights for policy 0, policy_version 63890 (0.0009) [2023-10-13 23:35:36,034][60935] Updated weights for policy 0, policy_version 63900 (0.0011) [2023-10-13 23:35:36,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 131530752. Throughput: 0: 1728.1, 1: 1680.7. Samples: 32885872. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 23:35:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:35:38,272][60934] Updated weights for policy 1, policy_version 64252 (0.0008) [2023-10-13 23:35:38,635][60934] Updated weights for policy 1, policy_version 64262 (0.0009) [2023-10-13 23:35:39,010][60934] Updated weights for policy 1, policy_version 64272 (0.0008) [2023-10-13 23:35:40,184][60935] Updated weights for policy 0, policy_version 63910 (0.0008) [2023-10-13 23:35:40,543][60935] Updated weights for policy 0, policy_version 63920 (0.0009) [2023-10-13 23:35:40,915][60935] Updated weights for policy 0, policy_version 63930 (0.0008) [2023-10-13 23:35:41,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 131596288. Throughput: 0: 1702.4, 1: 1701.2. Samples: 32905954. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 23:35:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:35:43,000][60934] Updated weights for policy 1, policy_version 64282 (0.0009) [2023-10-13 23:35:43,376][60934] Updated weights for policy 1, policy_version 64292 (0.0007) [2023-10-13 23:35:43,742][60934] Updated weights for policy 1, policy_version 64302 (0.0009) [2023-10-13 23:35:44,119][60934] Updated weights for policy 1, policy_version 64312 (0.0011) [2023-10-13 23:35:44,976][60935] Updated weights for policy 0, policy_version 63940 (0.0009) [2023-10-13 23:35:45,355][60935] Updated weights for policy 0, policy_version 63950 (0.0009) [2023-10-13 23:35:45,725][60935] Updated weights for policy 0, policy_version 63960 (0.0010) [2023-10-13 23:35:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 131661824. Throughput: 0: 1722.0, 1: 1692.9. Samples: 32916768. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 23:35:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:35:48,149][60934] Updated weights for policy 1, policy_version 64322 (0.0008) [2023-10-13 23:35:48,506][60934] Updated weights for policy 1, policy_version 64332 (0.0010) [2023-10-13 23:35:48,876][60934] Updated weights for policy 1, policy_version 64342 (0.0008) [2023-10-13 23:35:49,583][60935] Updated weights for policy 0, policy_version 63970 (0.0007) [2023-10-13 23:35:49,960][60935] Updated weights for policy 0, policy_version 63980 (0.0007) [2023-10-13 23:35:50,334][60935] Updated weights for policy 0, policy_version 63990 (0.0009) [2023-10-13 23:35:50,694][60935] Updated weights for policy 0, policy_version 64000 (0.0008) [2023-10-13 23:35:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 131727360. Throughput: 0: 1710.9, 1: 1679.2. Samples: 32936718. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 23:35:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:35:53,073][60934] Updated weights for policy 1, policy_version 64352 (0.0009) [2023-10-13 23:35:53,432][60934] Updated weights for policy 1, policy_version 64362 (0.0011) [2023-10-13 23:35:53,792][60934] Updated weights for policy 1, policy_version 64372 (0.0010) [2023-10-13 23:35:54,389][60935] Updated weights for policy 0, policy_version 64010 (0.0007) [2023-10-13 23:35:54,766][60935] Updated weights for policy 0, policy_version 64020 (0.0007) [2023-10-13 23:35:55,131][60935] Updated weights for policy 0, policy_version 64030 (0.0007) [2023-10-13 23:35:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 131792896. Throughput: 0: 1689.7, 1: 1696.7. Samples: 32956862. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 23:35:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:35:57,740][60934] Updated weights for policy 1, policy_version 64382 (0.0007) [2023-10-13 23:35:58,113][60934] Updated weights for policy 1, policy_version 64392 (0.0007) [2023-10-13 23:35:58,471][60934] Updated weights for policy 1, policy_version 64402 (0.0009) [2023-10-13 23:35:59,087][60935] Updated weights for policy 0, policy_version 64040 (0.0008) [2023-10-13 23:35:59,463][60935] Updated weights for policy 0, policy_version 64050 (0.0010) [2023-10-13 23:35:59,826][60935] Updated weights for policy 0, policy_version 64060 (0.0007) [2023-10-13 23:36:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 131858432. Throughput: 0: 1718.1, 1: 1673.2. Samples: 32967736. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-13 23:36:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:36:02,590][60934] Updated weights for policy 1, policy_version 64412 (0.0008) [2023-10-13 23:36:02,957][60934] Updated weights for policy 1, policy_version 64422 (0.0011) [2023-10-13 23:36:03,325][60934] Updated weights for policy 1, policy_version 64432 (0.0010) [2023-10-13 23:36:03,773][60935] Updated weights for policy 0, policy_version 64070 (0.0010) [2023-10-13 23:36:04,149][60935] Updated weights for policy 0, policy_version 64080 (0.0009) [2023-10-13 23:36:04,521][60935] Updated weights for policy 0, policy_version 64090 (0.0009) [2023-10-13 23:36:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 131923968. Throughput: 0: 1694.3, 1: 1688.3. Samples: 32987372. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:36:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:36:07,374][60934] Updated weights for policy 1, policy_version 64442 (0.0010) [2023-10-13 23:36:07,745][60934] Updated weights for policy 1, policy_version 64452 (0.0008) [2023-10-13 23:36:08,111][60934] Updated weights for policy 1, policy_version 64462 (0.0009) [2023-10-13 23:36:08,478][60934] Updated weights for policy 1, policy_version 64472 (0.0010) [2023-10-13 23:36:08,543][60935] Updated weights for policy 0, policy_version 64100 (0.0008) [2023-10-13 23:36:08,912][60935] Updated weights for policy 0, policy_version 64110 (0.0009) [2023-10-13 23:36:09,280][60935] Updated weights for policy 0, policy_version 64120 (0.0008) [2023-10-13 23:36:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 131989504. Throughput: 0: 1707.2, 1: 1697.0. Samples: 33008504. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:36:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:36:12,518][60934] Updated weights for policy 1, policy_version 64482 (0.0007) [2023-10-13 23:36:12,888][60934] Updated weights for policy 1, policy_version 64492 (0.0009) [2023-10-13 23:36:13,253][60934] Updated weights for policy 1, policy_version 64502 (0.0008) [2023-10-13 23:36:13,338][60935] Updated weights for policy 0, policy_version 64130 (0.0008) [2023-10-13 23:36:13,704][60935] Updated weights for policy 0, policy_version 64140 (0.0007) [2023-10-13 23:36:14,078][60935] Updated weights for policy 0, policy_version 64150 (0.0008) [2023-10-13 23:36:14,443][60935] Updated weights for policy 0, policy_version 64160 (0.0007) [2023-10-13 23:36:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 132055040. Throughput: 0: 1710.8, 1: 1671.9. Samples: 33018378. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:36:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:36:17,358][60934] Updated weights for policy 1, policy_version 64512 (0.0008) [2023-10-13 23:36:17,738][60934] Updated weights for policy 1, policy_version 64522 (0.0007) [2023-10-13 23:36:18,095][60934] Updated weights for policy 1, policy_version 64532 (0.0007) [2023-10-13 23:36:18,519][60935] Updated weights for policy 0, policy_version 64170 (0.0007) [2023-10-13 23:36:18,881][60935] Updated weights for policy 0, policy_version 64180 (0.0007) [2023-10-13 23:36:19,249][60935] Updated weights for policy 0, policy_version 64190 (0.0009) [2023-10-13 23:36:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 132120576. Throughput: 0: 1693.2, 1: 1694.2. Samples: 33038304. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:36:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:36:22,091][60934] Updated weights for policy 1, policy_version 64542 (0.0008) [2023-10-13 23:36:22,457][60934] Updated weights for policy 1, policy_version 64552 (0.0008) [2023-10-13 23:36:22,822][60934] Updated weights for policy 1, policy_version 64562 (0.0009) [2023-10-13 23:36:23,292][60935] Updated weights for policy 0, policy_version 64200 (0.0010) [2023-10-13 23:36:23,662][60935] Updated weights for policy 0, policy_version 64210 (0.0009) [2023-10-13 23:36:24,028][60935] Updated weights for policy 0, policy_version 64220 (0.0008) [2023-10-13 23:36:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 132186112. Throughput: 0: 1713.0, 1: 1695.8. Samples: 33059350. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:36:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:36:26,651][60934] Updated weights for policy 1, policy_version 64572 (0.0007) [2023-10-13 23:36:27,025][60934] Updated weights for policy 1, policy_version 64582 (0.0008) [2023-10-13 23:36:27,394][60934] Updated weights for policy 1, policy_version 64592 (0.0009) [2023-10-13 23:36:27,988][60935] Updated weights for policy 0, policy_version 64230 (0.0010) [2023-10-13 23:36:28,363][60935] Updated weights for policy 0, policy_version 64240 (0.0010) [2023-10-13 23:36:28,731][60935] Updated weights for policy 0, policy_version 64250 (0.0010) [2023-10-13 23:36:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 132251648. Throughput: 0: 1695.4, 1: 1681.1. Samples: 33068710. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:36:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:36:31,307][60934] Updated weights for policy 1, policy_version 64602 (0.0008) [2023-10-13 23:36:31,681][60934] Updated weights for policy 1, policy_version 64612 (0.0007) [2023-10-13 23:36:32,041][60934] Updated weights for policy 1, policy_version 64622 (0.0008) [2023-10-13 23:36:32,409][60934] Updated weights for policy 1, policy_version 64632 (0.0009) [2023-10-13 23:36:32,648][60935] Updated weights for policy 0, policy_version 64260 (0.0007) [2023-10-13 23:36:33,009][60935] Updated weights for policy 0, policy_version 64270 (0.0007) [2023-10-13 23:36:33,386][60935] Updated weights for policy 0, policy_version 64280 (0.0009) [2023-10-13 23:36:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 132317184. Throughput: 0: 1699.6, 1: 1699.3. Samples: 33089666. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:36:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:36:36,444][60934] Updated weights for policy 1, policy_version 64642 (0.0008) [2023-10-13 23:36:36,810][60934] Updated weights for policy 1, policy_version 64652 (0.0007) [2023-10-13 23:36:37,149][60935] Updated weights for policy 0, policy_version 64290 (0.0008) [2023-10-13 23:36:37,178][60934] Updated weights for policy 1, policy_version 64662 (0.0008) [2023-10-13 23:36:37,518][60935] Updated weights for policy 0, policy_version 64300 (0.0010) [2023-10-13 23:36:37,897][60935] Updated weights for policy 0, policy_version 64310 (0.0008) [2023-10-13 23:36:38,254][60935] Updated weights for policy 0, policy_version 64320 (0.0009) [2023-10-13 23:36:41,245][60934] Updated weights for policy 1, policy_version 64672 (0.0008) [2023-10-13 23:36:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 132382720. Throughput: 0: 1716.1, 1: 1705.7. Samples: 33110838. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:36:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:36:41,616][60934] Updated weights for policy 1, policy_version 64682 (0.0007) [2023-10-13 23:36:41,987][60934] Updated weights for policy 1, policy_version 64692 (0.0010) [2023-10-13 23:36:42,293][60935] Updated weights for policy 0, policy_version 64330 (0.0010) [2023-10-13 23:36:42,666][60935] Updated weights for policy 0, policy_version 64340 (0.0010) [2023-10-13 23:36:43,032][60935] Updated weights for policy 0, policy_version 64350 (0.0007) [2023-10-13 23:36:45,962][60934] Updated weights for policy 1, policy_version 64702 (0.0007) [2023-10-13 23:36:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 132448256. Throughput: 0: 1687.1, 1: 1698.9. Samples: 33120110. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:36:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:36:46,330][60934] Updated weights for policy 1, policy_version 64712 (0.0007) [2023-10-13 23:36:46,689][60934] Updated weights for policy 1, policy_version 64722 (0.0008) [2023-10-13 23:36:46,906][60935] Updated weights for policy 0, policy_version 64360 (0.0010) [2023-10-13 23:36:47,271][60935] Updated weights for policy 0, policy_version 64370 (0.0008) [2023-10-13 23:36:47,641][60935] Updated weights for policy 0, policy_version 64380 (0.0009) [2023-10-13 23:36:50,576][60934] Updated weights for policy 1, policy_version 64732 (0.0008) [2023-10-13 23:36:50,940][60934] Updated weights for policy 1, policy_version 64742 (0.0009) [2023-10-13 23:36:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 132513792. Throughput: 0: 1710.1, 1: 1703.2. Samples: 33140970. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-13 23:36:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:36:51,312][60934] Updated weights for policy 1, policy_version 64752 (0.0009) [2023-10-13 23:36:51,744][60935] Updated weights for policy 0, policy_version 64390 (0.0010) [2023-10-13 23:36:52,105][60935] Updated weights for policy 0, policy_version 64400 (0.0009) [2023-10-13 23:36:52,476][60935] Updated weights for policy 0, policy_version 64410 (0.0010) [2023-10-13 23:36:55,429][60934] Updated weights for policy 1, policy_version 64762 (0.0007) [2023-10-13 23:36:55,794][60934] Updated weights for policy 1, policy_version 64772 (0.0007) [2023-10-13 23:36:56,154][60934] Updated weights for policy 1, policy_version 64782 (0.0009) [2023-10-13 23:36:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 132579328. Throughput: 0: 1711.5, 1: 1695.4. Samples: 33161812. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-13 23:36:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:36:56,423][60935] Updated weights for policy 0, policy_version 64420 (0.0008) [2023-10-13 23:36:56,516][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000064792_66650112.pth... [2023-10-13 23:36:56,520][60934] Updated weights for policy 1, policy_version 64792 (0.0007) [2023-10-13 23:36:56,550][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000063192_65011712.pth [2023-10-13 23:36:56,792][60935] Updated weights for policy 0, policy_version 64430 (0.0009) [2023-10-13 23:36:57,172][60935] Updated weights for policy 0, policy_version 64440 (0.0008) [2023-10-13 23:36:57,457][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000064448_65994752.pth... [2023-10-13 23:36:57,492][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000062816_64323584.pth [2023-10-13 23:37:00,530][60934] Updated weights for policy 1, policy_version 64802 (0.0009) [2023-10-13 23:37:00,893][60934] Updated weights for policy 1, policy_version 64812 (0.0009) [2023-10-13 23:37:01,177][60935] Updated weights for policy 0, policy_version 64450 (0.0008) [2023-10-13 23:37:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 132644864. Throughput: 0: 1696.8, 1: 1705.7. Samples: 33171490. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-13 23:37:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:37:01,262][60934] Updated weights for policy 1, policy_version 64822 (0.0009) [2023-10-13 23:37:01,551][60935] Updated weights for policy 0, policy_version 64460 (0.0009) [2023-10-13 23:37:01,919][60935] Updated weights for policy 0, policy_version 64470 (0.0009) [2023-10-13 23:37:02,288][60935] Updated weights for policy 0, policy_version 64480 (0.0010) [2023-10-13 23:37:05,502][60934] Updated weights for policy 1, policy_version 64832 (0.0009) [2023-10-13 23:37:05,873][60934] Updated weights for policy 1, policy_version 64842 (0.0008) [2023-10-13 23:37:06,236][60934] Updated weights for policy 1, policy_version 64852 (0.0008) [2023-10-13 23:37:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 132710400. Throughput: 0: 1717.7, 1: 1710.4. Samples: 33192568. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-13 23:37:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:37:06,300][60935] Updated weights for policy 0, policy_version 64490 (0.0008) [2023-10-13 23:37:06,671][60935] Updated weights for policy 0, policy_version 64500 (0.0008) [2023-10-13 23:37:07,052][60935] Updated weights for policy 0, policy_version 64510 (0.0008) [2023-10-13 23:37:10,148][60934] Updated weights for policy 1, policy_version 64862 (0.0009) [2023-10-13 23:37:10,515][60934] Updated weights for policy 1, policy_version 64872 (0.0011) [2023-10-13 23:37:10,880][60934] Updated weights for policy 1, policy_version 64882 (0.0009) [2023-10-13 23:37:10,960][60935] Updated weights for policy 0, policy_version 64520 (0.0008) [2023-10-13 23:37:11,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 132808704. Throughput: 0: 1716.8, 1: 1688.7. Samples: 33212596. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-13 23:37:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:37:11,328][60935] Updated weights for policy 0, policy_version 64530 (0.0007) [2023-10-13 23:37:11,703][60935] Updated weights for policy 0, policy_version 64540 (0.0007) [2023-10-13 23:37:14,956][60934] Updated weights for policy 1, policy_version 64892 (0.0008) [2023-10-13 23:37:15,339][60934] Updated weights for policy 1, policy_version 64902 (0.0009) [2023-10-13 23:37:15,701][60934] Updated weights for policy 1, policy_version 64912 (0.0009) [2023-10-13 23:37:15,750][60935] Updated weights for policy 0, policy_version 64550 (0.0008) [2023-10-13 23:37:16,115][60935] Updated weights for policy 0, policy_version 64560 (0.0010) [2023-10-13 23:37:16,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 132874240. Throughput: 0: 1718.8, 1: 1702.8. Samples: 33222680. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-13 23:37:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:37:16,486][60935] Updated weights for policy 0, policy_version 64570 (0.0009) [2023-10-13 23:37:19,706][60934] Updated weights for policy 1, policy_version 64922 (0.0007) [2023-10-13 23:37:20,087][60934] Updated weights for policy 1, policy_version 64932 (0.0009) [2023-10-13 23:37:20,455][60934] Updated weights for policy 1, policy_version 64942 (0.0008) [2023-10-13 23:37:20,531][60935] Updated weights for policy 0, policy_version 64580 (0.0009) [2023-10-13 23:37:20,826][60934] Updated weights for policy 1, policy_version 64952 (0.0007) [2023-10-13 23:37:20,899][60935] Updated weights for policy 0, policy_version 64590 (0.0008) [2023-10-13 23:37:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 132939776. Throughput: 0: 1720.0, 1: 1702.8. Samples: 33243696. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-13 23:37:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:37:21,274][60935] Updated weights for policy 0, policy_version 64600 (0.0011) [2023-10-13 23:37:24,789][60934] Updated weights for policy 1, policy_version 64962 (0.0007) [2023-10-13 23:37:25,159][60934] Updated weights for policy 1, policy_version 64972 (0.0008) [2023-10-13 23:37:25,261][60935] Updated weights for policy 0, policy_version 64610 (0.0009) [2023-10-13 23:37:25,516][60934] Updated weights for policy 1, policy_version 64982 (0.0007) [2023-10-13 23:37:25,622][60935] Updated weights for policy 0, policy_version 64620 (0.0009) [2023-10-13 23:37:25,988][60935] Updated weights for policy 0, policy_version 64630 (0.0010) [2023-10-13 23:37:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 133005312. Throughput: 0: 1708.6, 1: 1670.4. Samples: 33262892. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-13 23:37:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:37:26,359][60935] Updated weights for policy 0, policy_version 64640 (0.0010) [2023-10-13 23:37:29,567][60934] Updated weights for policy 1, policy_version 64992 (0.0007) [2023-10-13 23:37:29,932][60934] Updated weights for policy 1, policy_version 65002 (0.0009) [2023-10-13 23:37:30,300][60934] Updated weights for policy 1, policy_version 65012 (0.0009) [2023-10-13 23:37:30,394][60935] Updated weights for policy 0, policy_version 64650 (0.0008) [2023-10-13 23:37:30,763][60935] Updated weights for policy 0, policy_version 64660 (0.0007) [2023-10-13 23:37:31,141][60935] Updated weights for policy 0, policy_version 64670 (0.0007) [2023-10-13 23:37:31,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 133103616. Throughput: 0: 1721.5, 1: 1697.1. Samples: 33273944. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-13 23:37:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:37:34,400][60934] Updated weights for policy 1, policy_version 65022 (0.0008) [2023-10-13 23:37:34,775][60934] Updated weights for policy 1, policy_version 65032 (0.0008) [2023-10-13 23:37:35,001][60935] Updated weights for policy 0, policy_version 64680 (0.0008) [2023-10-13 23:37:35,152][60934] Updated weights for policy 1, policy_version 65042 (0.0009) [2023-10-13 23:37:35,369][60935] Updated weights for policy 0, policy_version 64690 (0.0007) [2023-10-13 23:37:35,736][60935] Updated weights for policy 0, policy_version 64700 (0.0009) [2023-10-13 23:37:36,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 133169152. Throughput: 0: 1722.7, 1: 1689.3. Samples: 33294510. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-13 23:37:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:37:39,182][60934] Updated weights for policy 1, policy_version 65052 (0.0009) [2023-10-13 23:37:39,546][60934] Updated weights for policy 1, policy_version 65062 (0.0008) [2023-10-13 23:37:39,673][60935] Updated weights for policy 0, policy_version 64710 (0.0007) [2023-10-13 23:37:39,913][60934] Updated weights for policy 1, policy_version 65072 (0.0007) [2023-10-13 23:37:40,044][60935] Updated weights for policy 0, policy_version 64720 (0.0008) [2023-10-13 23:37:40,405][60935] Updated weights for policy 0, policy_version 64730 (0.0008) [2023-10-13 23:37:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 133234688. Throughput: 0: 1695.6, 1: 1674.1. Samples: 33313448. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-13 23:37:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:37:43,970][60934] Updated weights for policy 1, policy_version 65082 (0.0008) [2023-10-13 23:37:44,339][60934] Updated weights for policy 1, policy_version 65092 (0.0008) [2023-10-13 23:37:44,451][60935] Updated weights for policy 0, policy_version 64740 (0.0008) [2023-10-13 23:37:44,701][60934] Updated weights for policy 1, policy_version 65102 (0.0009) [2023-10-13 23:37:44,811][60935] Updated weights for policy 0, policy_version 64750 (0.0009) [2023-10-13 23:37:45,063][60934] Updated weights for policy 1, policy_version 65112 (0.0007) [2023-10-13 23:37:45,177][60935] Updated weights for policy 0, policy_version 64760 (0.0009) [2023-10-13 23:37:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 133300224. Throughput: 0: 1720.8, 1: 1691.4. Samples: 33325038. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-13 23:37:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:37:49,125][60934] Updated weights for policy 1, policy_version 65122 (0.0007) [2023-10-13 23:37:49,287][60935] Updated weights for policy 0, policy_version 64770 (0.0008) [2023-10-13 23:37:49,489][60934] Updated weights for policy 1, policy_version 65132 (0.0008) [2023-10-13 23:37:49,656][60935] Updated weights for policy 0, policy_version 64780 (0.0008) [2023-10-13 23:37:49,852][60934] Updated weights for policy 1, policy_version 65142 (0.0008) [2023-10-13 23:37:50,027][60935] Updated weights for policy 0, policy_version 64790 (0.0007) [2023-10-13 23:37:50,390][60935] Updated weights for policy 0, policy_version 64800 (0.0007) [2023-10-13 23:37:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 133365760. Throughput: 0: 1702.4, 1: 1675.1. Samples: 33344554. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-13 23:37:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:37:53,913][60934] Updated weights for policy 1, policy_version 65152 (0.0010) [2023-10-13 23:37:54,276][60934] Updated weights for policy 1, policy_version 65162 (0.0008) [2023-10-13 23:37:54,402][60935] Updated weights for policy 0, policy_version 64810 (0.0008) [2023-10-13 23:37:54,645][60934] Updated weights for policy 1, policy_version 65172 (0.0008) [2023-10-13 23:37:54,779][60935] Updated weights for policy 0, policy_version 64820 (0.0010) [2023-10-13 23:37:55,135][60935] Updated weights for policy 0, policy_version 64830 (0.0010) [2023-10-13 23:37:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 133431296. Throughput: 0: 1692.0, 1: 1684.4. Samples: 33364534. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-13 23:37:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:37:58,805][60934] Updated weights for policy 1, policy_version 65182 (0.0010) [2023-10-13 23:37:59,176][60934] Updated weights for policy 1, policy_version 65192 (0.0009) [2023-10-13 23:37:59,190][60935] Updated weights for policy 0, policy_version 64840 (0.0008) [2023-10-13 23:37:59,541][60934] Updated weights for policy 1, policy_version 65202 (0.0008) [2023-10-13 23:37:59,545][60935] Updated weights for policy 0, policy_version 64850 (0.0007) [2023-10-13 23:37:59,921][60935] Updated weights for policy 0, policy_version 64860 (0.0010) [2023-10-13 23:38:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 133496832. Throughput: 0: 1720.3, 1: 1697.3. Samples: 33376472. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-13 23:38:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.070')] [2023-10-13 23:38:03,502][60934] Updated weights for policy 1, policy_version 65212 (0.0008) [2023-10-13 23:38:03,875][60934] Updated weights for policy 1, policy_version 65222 (0.0008) [2023-10-13 23:38:03,966][60935] Updated weights for policy 0, policy_version 64870 (0.0008) [2023-10-13 23:38:04,238][60934] Updated weights for policy 1, policy_version 65232 (0.0009) [2023-10-13 23:38:04,339][60935] Updated weights for policy 0, policy_version 64880 (0.0007) [2023-10-13 23:38:04,706][60935] Updated weights for policy 0, policy_version 64890 (0.0009) [2023-10-13 23:38:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 133562368. Throughput: 0: 1693.3, 1: 1669.6. Samples: 33395028. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-13 23:38:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.070')] [2023-10-13 23:38:08,196][60934] Updated weights for policy 1, policy_version 65242 (0.0008) [2023-10-13 23:38:08,560][60934] Updated weights for policy 1, policy_version 65252 (0.0008) [2023-10-13 23:38:08,699][60935] Updated weights for policy 0, policy_version 64900 (0.0008) [2023-10-13 23:38:08,924][60934] Updated weights for policy 1, policy_version 65262 (0.0008) [2023-10-13 23:38:09,071][60935] Updated weights for policy 0, policy_version 64910 (0.0008) [2023-10-13 23:38:09,286][60934] Updated weights for policy 1, policy_version 65272 (0.0009) [2023-10-13 23:38:09,445][60935] Updated weights for policy 0, policy_version 64920 (0.0009) [2023-10-13 23:38:11,249][59943] Fps is (10 sec: 13106.6, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 133627904. Throughput: 0: 1699.5, 1: 1697.2. Samples: 33415748. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-13 23:38:11,250][59943] Avg episode reward: [(0, '0.000'), (1, '-0.070')] [2023-10-13 23:38:13,387][60935] Updated weights for policy 0, policy_version 64930 (0.0007) [2023-10-13 23:38:13,388][60934] Updated weights for policy 1, policy_version 65282 (0.0009) [2023-10-13 23:38:13,750][60934] Updated weights for policy 1, policy_version 65292 (0.0008) [2023-10-13 23:38:13,751][60935] Updated weights for policy 0, policy_version 64940 (0.0008) [2023-10-13 23:38:14,106][60934] Updated weights for policy 1, policy_version 65302 (0.0007) [2023-10-13 23:38:14,108][60935] Updated weights for policy 0, policy_version 64950 (0.0008) [2023-10-13 23:38:14,481][60935] Updated weights for policy 0, policy_version 64960 (0.0008) [2023-10-13 23:38:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 133693440. Throughput: 0: 1704.3, 1: 1686.2. Samples: 33426516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:38:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:38:18,281][60934] Updated weights for policy 1, policy_version 65312 (0.0007) [2023-10-13 23:38:18,416][60935] Updated weights for policy 0, policy_version 64970 (0.0007) [2023-10-13 23:38:18,645][60934] Updated weights for policy 1, policy_version 65322 (0.0008) [2023-10-13 23:38:18,791][60935] Updated weights for policy 0, policy_version 64980 (0.0007) [2023-10-13 23:38:19,013][60934] Updated weights for policy 1, policy_version 65332 (0.0007) [2023-10-13 23:38:19,164][60935] Updated weights for policy 0, policy_version 64990 (0.0007) [2023-10-13 23:38:21,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 133758976. Throughput: 0: 1690.3, 1: 1672.8. Samples: 33445848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:38:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:38:22,964][60934] Updated weights for policy 1, policy_version 65342 (0.0007) [2023-10-13 23:38:23,165][60935] Updated weights for policy 0, policy_version 65000 (0.0008) [2023-10-13 23:38:23,323][60934] Updated weights for policy 1, policy_version 65352 (0.0008) [2023-10-13 23:38:23,527][60935] Updated weights for policy 0, policy_version 65010 (0.0007) [2023-10-13 23:38:23,693][60934] Updated weights for policy 1, policy_version 65362 (0.0007) [2023-10-13 23:38:23,889][60935] Updated weights for policy 0, policy_version 65020 (0.0007) [2023-10-13 23:38:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 133824512. Throughput: 0: 1715.2, 1: 1692.8. Samples: 33466810. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:38:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:38:27,713][60935] Updated weights for policy 0, policy_version 65030 (0.0009) [2023-10-13 23:38:27,777][60934] Updated weights for policy 1, policy_version 65372 (0.0007) [2023-10-13 23:38:28,091][60935] Updated weights for policy 0, policy_version 65040 (0.0008) [2023-10-13 23:38:28,147][60934] Updated weights for policy 1, policy_version 65382 (0.0008) [2023-10-13 23:38:28,457][60935] Updated weights for policy 0, policy_version 65050 (0.0011) [2023-10-13 23:38:28,515][60934] Updated weights for policy 1, policy_version 65392 (0.0009) [2023-10-13 23:38:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 133890048. Throughput: 0: 1690.3, 1: 1673.8. Samples: 33476422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:38:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:38:32,336][60935] Updated weights for policy 0, policy_version 65060 (0.0009) [2023-10-13 23:38:32,545][60934] Updated weights for policy 1, policy_version 65402 (0.0008) [2023-10-13 23:38:32,707][60935] Updated weights for policy 0, policy_version 65070 (0.0009) [2023-10-13 23:38:32,907][60934] Updated weights for policy 1, policy_version 65412 (0.0009) [2023-10-13 23:38:33,076][60935] Updated weights for policy 0, policy_version 65080 (0.0008) [2023-10-13 23:38:33,264][60934] Updated weights for policy 1, policy_version 65422 (0.0007) [2023-10-13 23:38:33,628][60934] Updated weights for policy 1, policy_version 65432 (0.0007) [2023-10-13 23:38:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 133955584. Throughput: 0: 1711.5, 1: 1679.0. Samples: 33497126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:38:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:38:37,045][60935] Updated weights for policy 0, policy_version 65090 (0.0008) [2023-10-13 23:38:37,421][60935] Updated weights for policy 0, policy_version 65100 (0.0008) [2023-10-13 23:38:37,795][60935] Updated weights for policy 0, policy_version 65110 (0.0009) [2023-10-13 23:38:37,883][60934] Updated weights for policy 1, policy_version 65442 (0.0008) [2023-10-13 23:38:38,159][60935] Updated weights for policy 0, policy_version 65120 (0.0009) [2023-10-13 23:38:38,252][60934] Updated weights for policy 1, policy_version 65452 (0.0007) [2023-10-13 23:38:38,622][60934] Updated weights for policy 1, policy_version 65462 (0.0007) [2023-10-13 23:38:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 134021120. Throughput: 0: 1730.6, 1: 1684.0. Samples: 33518190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:38:41,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:38:42,149][60935] Updated weights for policy 0, policy_version 65130 (0.0008) [2023-10-13 23:38:42,524][60935] Updated weights for policy 0, policy_version 65140 (0.0007) [2023-10-13 23:38:42,774][60934] Updated weights for policy 1, policy_version 65472 (0.0009) [2023-10-13 23:38:42,897][60935] Updated weights for policy 0, policy_version 65150 (0.0008) [2023-10-13 23:38:43,147][60934] Updated weights for policy 1, policy_version 65482 (0.0009) [2023-10-13 23:38:43,513][60934] Updated weights for policy 1, policy_version 65492 (0.0008) [2023-10-13 23:38:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 134086656. Throughput: 0: 1693.1, 1: 1656.8. Samples: 33527220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:38:46,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:38:46,729][60935] Updated weights for policy 0, policy_version 65160 (0.0008) [2023-10-13 23:38:47,100][60935] Updated weights for policy 0, policy_version 65170 (0.0007) [2023-10-13 23:38:47,469][60935] Updated weights for policy 0, policy_version 65180 (0.0007) [2023-10-13 23:38:47,604][60934] Updated weights for policy 1, policy_version 65502 (0.0008) [2023-10-13 23:38:47,978][60934] Updated weights for policy 1, policy_version 65512 (0.0010) [2023-10-13 23:38:48,338][60934] Updated weights for policy 1, policy_version 65522 (0.0008) [2023-10-13 23:38:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 134152192. Throughput: 0: 1723.6, 1: 1678.9. Samples: 33548142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:38:51,250][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:38:51,516][60935] Updated weights for policy 0, policy_version 65190 (0.0009) [2023-10-13 23:38:51,886][60935] Updated weights for policy 0, policy_version 65200 (0.0008) [2023-10-13 23:38:52,251][60935] Updated weights for policy 0, policy_version 65210 (0.0011) [2023-10-13 23:38:52,422][60934] Updated weights for policy 1, policy_version 65532 (0.0009) [2023-10-13 23:38:52,791][60934] Updated weights for policy 1, policy_version 65542 (0.0008) [2023-10-13 23:38:53,170][60934] Updated weights for policy 1, policy_version 65552 (0.0009) [2023-10-13 23:38:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 134217728. Throughput: 0: 1728.3, 1: 1674.0. Samples: 33568854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:38:56,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:38:56,259][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000065560_67436544.pth... [2023-10-13 23:38:56,293][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000063992_65830912.pth [2023-10-13 23:38:56,427][60935] Updated weights for policy 0, policy_version 65220 (0.0011) [2023-10-13 23:38:56,793][60935] Updated weights for policy 0, policy_version 65230 (0.0010) [2023-10-13 23:38:57,174][60935] Updated weights for policy 0, policy_version 65240 (0.0011) [2023-10-13 23:38:57,339][60934] Updated weights for policy 1, policy_version 65562 (0.0009) [2023-10-13 23:38:57,472][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000065248_66813952.pth... [2023-10-13 23:38:57,501][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000063616_65142784.pth [2023-10-13 23:38:57,700][60934] Updated weights for policy 1, policy_version 65572 (0.0010) [2023-10-13 23:38:58,077][60934] Updated weights for policy 1, policy_version 65582 (0.0010) [2023-10-13 23:38:58,438][60934] Updated weights for policy 1, policy_version 65592 (0.0009) [2023-10-13 23:39:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 134283264. Throughput: 0: 1710.2, 1: 1658.6. Samples: 33578112. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 23:39:01,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:39:01,249][60935] Updated weights for policy 0, policy_version 65250 (0.0008) [2023-10-13 23:39:01,614][60935] Updated weights for policy 0, policy_version 65260 (0.0012) [2023-10-13 23:39:01,972][60935] Updated weights for policy 0, policy_version 65270 (0.0010) [2023-10-13 23:39:02,338][60935] Updated weights for policy 0, policy_version 65280 (0.0008) [2023-10-13 23:39:02,509][60934] Updated weights for policy 1, policy_version 65602 (0.0007) [2023-10-13 23:39:02,874][60934] Updated weights for policy 1, policy_version 65612 (0.0007) [2023-10-13 23:39:03,245][60934] Updated weights for policy 1, policy_version 65622 (0.0008) [2023-10-13 23:39:06,035][60935] Updated weights for policy 0, policy_version 65290 (0.0007) [2023-10-13 23:39:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 134348800. Throughput: 0: 1726.9, 1: 1681.9. Samples: 33599244. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 23:39:06,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:39:06,402][60935] Updated weights for policy 0, policy_version 65300 (0.0010) [2023-10-13 23:39:06,773][60935] Updated weights for policy 0, policy_version 65310 (0.0009) [2023-10-13 23:39:07,343][60934] Updated weights for policy 1, policy_version 65632 (0.0009) [2023-10-13 23:39:07,703][60934] Updated weights for policy 1, policy_version 65642 (0.0009) [2023-10-13 23:39:08,066][60934] Updated weights for policy 1, policy_version 65652 (0.0009) [2023-10-13 23:39:10,732][60935] Updated weights for policy 0, policy_version 65320 (0.0008) [2023-10-13 23:39:11,101][60935] Updated weights for policy 0, policy_version 65330 (0.0008) [2023-10-13 23:39:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 134414336. Throughput: 0: 1719.0, 1: 1680.8. Samples: 33619802. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 23:39:11,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:39:11,465][60935] Updated weights for policy 0, policy_version 65340 (0.0007) [2023-10-13 23:39:12,096][60934] Updated weights for policy 1, policy_version 65662 (0.0008) [2023-10-13 23:39:12,464][60934] Updated weights for policy 1, policy_version 65672 (0.0010) [2023-10-13 23:39:12,813][60934] Updated weights for policy 1, policy_version 65682 (0.0008) [2023-10-13 23:39:15,246][60935] Updated weights for policy 0, policy_version 65350 (0.0008) [2023-10-13 23:39:15,618][60935] Updated weights for policy 0, policy_version 65360 (0.0007) [2023-10-13 23:39:15,992][60935] Updated weights for policy 0, policy_version 65370 (0.0008) [2023-10-13 23:39:16,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 134512640. Throughput: 0: 1728.1, 1: 1675.6. Samples: 33629592. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 23:39:16,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:39:16,770][60934] Updated weights for policy 1, policy_version 65692 (0.0007) [2023-10-13 23:39:17,144][60934] Updated weights for policy 1, policy_version 65702 (0.0008) [2023-10-13 23:39:17,508][60934] Updated weights for policy 1, policy_version 65712 (0.0010) [2023-10-13 23:39:20,078][60935] Updated weights for policy 0, policy_version 65380 (0.0010) [2023-10-13 23:39:20,448][60935] Updated weights for policy 0, policy_version 65390 (0.0010) [2023-10-13 23:39:20,826][60935] Updated weights for policy 0, policy_version 65400 (0.0009) [2023-10-13 23:39:21,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 134578176. Throughput: 0: 1728.8, 1: 1688.7. Samples: 33650910. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 23:39:21,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:39:21,486][60934] Updated weights for policy 1, policy_version 65722 (0.0009) [2023-10-13 23:39:21,856][60934] Updated weights for policy 1, policy_version 65732 (0.0009) [2023-10-13 23:39:22,212][60934] Updated weights for policy 1, policy_version 65742 (0.0008) [2023-10-13 23:39:22,577][60934] Updated weights for policy 1, policy_version 65752 (0.0010) [2023-10-13 23:39:24,758][60935] Updated weights for policy 0, policy_version 65410 (0.0007) [2023-10-13 23:39:25,121][60935] Updated weights for policy 0, policy_version 65420 (0.0012) [2023-10-13 23:39:25,495][60935] Updated weights for policy 0, policy_version 65430 (0.0007) [2023-10-13 23:39:25,865][60935] Updated weights for policy 0, policy_version 65440 (0.0007) [2023-10-13 23:39:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 134643712. Throughput: 0: 1694.1, 1: 1695.1. Samples: 33670704. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 23:39:26,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:39:26,554][60934] Updated weights for policy 1, policy_version 65762 (0.0008) [2023-10-13 23:39:26,919][60934] Updated weights for policy 1, policy_version 65772 (0.0010) [2023-10-13 23:39:27,294][60934] Updated weights for policy 1, policy_version 65782 (0.0008) [2023-10-13 23:39:29,901][60935] Updated weights for policy 0, policy_version 65450 (0.0009) [2023-10-13 23:39:30,278][60935] Updated weights for policy 0, policy_version 65460 (0.0008) [2023-10-13 23:39:30,644][60935] Updated weights for policy 0, policy_version 65470 (0.0008) [2023-10-13 23:39:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 134709248. Throughput: 0: 1728.0, 1: 1690.2. Samples: 33681040. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 23:39:31,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:39:31,462][60934] Updated weights for policy 1, policy_version 65792 (0.0009) [2023-10-13 23:39:31,833][60934] Updated weights for policy 1, policy_version 65802 (0.0009) [2023-10-13 23:39:32,202][60934] Updated weights for policy 1, policy_version 65812 (0.0008) [2023-10-13 23:39:34,641][60935] Updated weights for policy 0, policy_version 65480 (0.0009) [2023-10-13 23:39:35,004][60935] Updated weights for policy 0, policy_version 65490 (0.0009) [2023-10-13 23:39:35,380][60935] Updated weights for policy 0, policy_version 65500 (0.0009) [2023-10-13 23:39:36,243][60934] Updated weights for policy 1, policy_version 65822 (0.0010) [2023-10-13 23:39:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 134774784. Throughput: 0: 1709.4, 1: 1695.7. Samples: 33701370. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-13 23:39:36,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:39:36,617][60934] Updated weights for policy 1, policy_version 65832 (0.0010) [2023-10-13 23:39:36,992][60934] Updated weights for policy 1, policy_version 65842 (0.0007) [2023-10-13 23:39:39,466][60935] Updated weights for policy 0, policy_version 65510 (0.0010) [2023-10-13 23:39:39,841][60935] Updated weights for policy 0, policy_version 65520 (0.0010) [2023-10-13 23:39:40,206][60935] Updated weights for policy 0, policy_version 65530 (0.0009) [2023-10-13 23:39:41,051][60934] Updated weights for policy 1, policy_version 65852 (0.0008) [2023-10-13 23:39:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 134840320. Throughput: 0: 1687.6, 1: 1699.4. Samples: 33721268. Policy #0 lag: (min: 5.0, avg: 5.7, max: 24.0) [2023-10-13 23:39:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:39:41,427][60934] Updated weights for policy 1, policy_version 65862 (0.0008) [2023-10-13 23:39:41,796][60934] Updated weights for policy 1, policy_version 65872 (0.0008) [2023-10-13 23:39:44,436][60935] Updated weights for policy 0, policy_version 65540 (0.0009) [2023-10-13 23:39:44,816][60935] Updated weights for policy 0, policy_version 65550 (0.0007) [2023-10-13 23:39:45,182][60935] Updated weights for policy 0, policy_version 65560 (0.0009) [2023-10-13 23:39:45,797][60934] Updated weights for policy 1, policy_version 65882 (0.0009) [2023-10-13 23:39:46,160][60934] Updated weights for policy 1, policy_version 65892 (0.0009) [2023-10-13 23:39:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 134905856. Throughput: 0: 1710.2, 1: 1698.3. Samples: 33731494. Policy #0 lag: (min: 5.0, avg: 5.7, max: 24.0) [2023-10-13 23:39:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:39:46,528][60934] Updated weights for policy 1, policy_version 65902 (0.0008) [2023-10-13 23:39:46,883][60934] Updated weights for policy 1, policy_version 65912 (0.0007) [2023-10-13 23:39:49,093][60935] Updated weights for policy 0, policy_version 65570 (0.0008) [2023-10-13 23:39:49,464][60935] Updated weights for policy 0, policy_version 65580 (0.0008) [2023-10-13 23:39:49,833][60935] Updated weights for policy 0, policy_version 65590 (0.0007) [2023-10-13 23:39:50,202][60935] Updated weights for policy 0, policy_version 65600 (0.0009) [2023-10-13 23:39:50,989][60934] Updated weights for policy 1, policy_version 65922 (0.0008) [2023-10-13 23:39:51,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 134971392. Throughput: 0: 1689.5, 1: 1695.2. Samples: 33751552. Policy #0 lag: (min: 5.0, avg: 5.7, max: 24.0) [2023-10-13 23:39:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:39:51,365][60934] Updated weights for policy 1, policy_version 65932 (0.0009) [2023-10-13 23:39:51,729][60934] Updated weights for policy 1, policy_version 65942 (0.0007) [2023-10-13 23:39:54,188][60935] Updated weights for policy 0, policy_version 65610 (0.0010) [2023-10-13 23:39:54,558][60935] Updated weights for policy 0, policy_version 65620 (0.0011) [2023-10-13 23:39:54,913][60935] Updated weights for policy 0, policy_version 65630 (0.0009) [2023-10-13 23:39:55,834][60934] Updated weights for policy 1, policy_version 65952 (0.0008) [2023-10-13 23:39:56,209][60934] Updated weights for policy 1, policy_version 65962 (0.0010) [2023-10-13 23:39:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 135036928. Throughput: 0: 1689.9, 1: 1690.7. Samples: 33771928. Policy #0 lag: (min: 5.0, avg: 5.7, max: 24.0) [2023-10-13 23:39:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:39:56,573][60934] Updated weights for policy 1, policy_version 65972 (0.0011) [2023-10-13 23:39:58,904][60935] Updated weights for policy 0, policy_version 65640 (0.0008) [2023-10-13 23:39:59,279][60935] Updated weights for policy 0, policy_version 65650 (0.0008) [2023-10-13 23:39:59,647][60935] Updated weights for policy 0, policy_version 65660 (0.0009) [2023-10-13 23:40:00,650][60934] Updated weights for policy 1, policy_version 65982 (0.0008) [2023-10-13 23:40:01,024][60934] Updated weights for policy 1, policy_version 65992 (0.0008) [2023-10-13 23:40:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 135102464. Throughput: 0: 1700.2, 1: 1687.7. Samples: 33782044. Policy #0 lag: (min: 5.0, avg: 5.7, max: 24.0) [2023-10-13 23:40:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:40:01,391][60934] Updated weights for policy 1, policy_version 66002 (0.0007) [2023-10-13 23:40:03,688][60935] Updated weights for policy 0, policy_version 65670 (0.0008) [2023-10-13 23:40:04,057][60935] Updated weights for policy 0, policy_version 65680 (0.0010) [2023-10-13 23:40:04,429][60935] Updated weights for policy 0, policy_version 65690 (0.0009) [2023-10-13 23:40:05,524][60934] Updated weights for policy 1, policy_version 66012 (0.0008) [2023-10-13 23:40:05,894][60934] Updated weights for policy 1, policy_version 66022 (0.0008) [2023-10-13 23:40:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 135168000. Throughput: 0: 1668.9, 1: 1682.0. Samples: 33801700. Policy #0 lag: (min: 5.0, avg: 5.7, max: 24.0) [2023-10-13 23:40:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:40:06,269][60934] Updated weights for policy 1, policy_version 66032 (0.0009) [2023-10-13 23:40:08,472][60935] Updated weights for policy 0, policy_version 65700 (0.0008) [2023-10-13 23:40:08,833][60935] Updated weights for policy 0, policy_version 65710 (0.0007) [2023-10-13 23:40:09,215][60935] Updated weights for policy 0, policy_version 65720 (0.0008) [2023-10-13 23:40:10,278][60934] Updated weights for policy 1, policy_version 66042 (0.0008) [2023-10-13 23:40:10,656][60934] Updated weights for policy 1, policy_version 66052 (0.0009) [2023-10-13 23:40:11,029][60934] Updated weights for policy 1, policy_version 66062 (0.0009) [2023-10-13 23:40:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 135233536. Throughput: 0: 1700.2, 1: 1667.6. Samples: 33822252. Policy #0 lag: (min: 5.0, avg: 5.7, max: 24.0) [2023-10-13 23:40:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:40:11,391][60934] Updated weights for policy 1, policy_version 66072 (0.0009) [2023-10-13 23:40:13,094][60935] Updated weights for policy 0, policy_version 65730 (0.0008) [2023-10-13 23:40:13,467][60935] Updated weights for policy 0, policy_version 65740 (0.0007) [2023-10-13 23:40:13,846][60935] Updated weights for policy 0, policy_version 65750 (0.0009) [2023-10-13 23:40:14,212][60935] Updated weights for policy 0, policy_version 65760 (0.0009) [2023-10-13 23:40:15,414][60934] Updated weights for policy 1, policy_version 66082 (0.0008) [2023-10-13 23:40:15,790][60934] Updated weights for policy 1, policy_version 66092 (0.0007) [2023-10-13 23:40:16,153][60934] Updated weights for policy 1, policy_version 66102 (0.0008) [2023-10-13 23:40:16,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 135331840. Throughput: 0: 1685.0, 1: 1681.3. Samples: 33832526. Policy #0 lag: (min: 5.0, avg: 5.7, max: 24.0) [2023-10-13 23:40:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:40:18,223][60935] Updated weights for policy 0, policy_version 65770 (0.0011) [2023-10-13 23:40:18,604][60935] Updated weights for policy 0, policy_version 65780 (0.0009) [2023-10-13 23:40:18,969][60935] Updated weights for policy 0, policy_version 65790 (0.0008) [2023-10-13 23:40:20,320][60934] Updated weights for policy 1, policy_version 66112 (0.0009) [2023-10-13 23:40:20,692][60934] Updated weights for policy 1, policy_version 66122 (0.0007) [2023-10-13 23:40:21,057][60934] Updated weights for policy 1, policy_version 66132 (0.0009) [2023-10-13 23:40:21,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 135397376. Throughput: 0: 1689.6, 1: 1680.3. Samples: 33853014. Policy #0 lag: (min: 5.0, avg: 5.7, max: 24.0) [2023-10-13 23:40:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:40:22,871][60935] Updated weights for policy 0, policy_version 65800 (0.0008) [2023-10-13 23:40:23,236][60935] Updated weights for policy 0, policy_version 65810 (0.0007) [2023-10-13 23:40:23,616][60935] Updated weights for policy 0, policy_version 65820 (0.0009) [2023-10-13 23:40:25,164][60934] Updated weights for policy 1, policy_version 66142 (0.0007) [2023-10-13 23:40:25,529][60934] Updated weights for policy 1, policy_version 66152 (0.0008) [2023-10-13 23:40:25,893][60934] Updated weights for policy 1, policy_version 66162 (0.0009) [2023-10-13 23:40:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 135462912. Throughput: 0: 1713.9, 1: 1669.4. Samples: 33873520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:40:26,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:40:27,767][60935] Updated weights for policy 0, policy_version 65830 (0.0009) [2023-10-13 23:40:28,125][60935] Updated weights for policy 0, policy_version 65840 (0.0009) [2023-10-13 23:40:28,496][60935] Updated weights for policy 0, policy_version 65850 (0.0010) [2023-10-13 23:40:30,241][60934] Updated weights for policy 1, policy_version 66172 (0.0009) [2023-10-13 23:40:30,614][60934] Updated weights for policy 1, policy_version 66182 (0.0009) [2023-10-13 23:40:30,973][60934] Updated weights for policy 1, policy_version 66192 (0.0010) [2023-10-13 23:40:31,248][59943] Fps is (10 sec: 9830.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 135495680. Throughput: 0: 1684.9, 1: 1679.0. Samples: 33882870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:40:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:40:32,968][60935] Updated weights for policy 0, policy_version 65860 (0.0009) [2023-10-13 23:40:33,336][60935] Updated weights for policy 0, policy_version 65870 (0.0008) [2023-10-13 23:40:33,707][60935] Updated weights for policy 0, policy_version 65880 (0.0009) [2023-10-13 23:40:35,253][60934] Updated weights for policy 1, policy_version 66202 (0.0008) [2023-10-13 23:40:35,618][60934] Updated weights for policy 1, policy_version 66212 (0.0010) [2023-10-13 23:40:35,984][60934] Updated weights for policy 1, policy_version 66222 (0.0010) [2023-10-13 23:40:36,248][59943] Fps is (10 sec: 9830.7, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 135561216. Throughput: 0: 1685.3, 1: 1666.1. Samples: 33902366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:40:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:40:36,351][60934] Updated weights for policy 1, policy_version 66232 (0.0012) [2023-10-13 23:40:37,961][60935] Updated weights for policy 0, policy_version 65890 (0.0008) [2023-10-13 23:40:38,319][60935] Updated weights for policy 0, policy_version 65900 (0.0009) [2023-10-13 23:40:38,687][60935] Updated weights for policy 0, policy_version 65910 (0.0010) [2023-10-13 23:40:39,051][60935] Updated weights for policy 0, policy_version 65920 (0.0010) [2023-10-13 23:40:40,648][60934] Updated weights for policy 1, policy_version 66242 (0.0010) [2023-10-13 23:40:41,010][60934] Updated weights for policy 1, policy_version 66252 (0.0010) [2023-10-13 23:40:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 135626752. Throughput: 0: 1675.6, 1: 1648.3. Samples: 33921504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:40:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:40:41,372][60934] Updated weights for policy 1, policy_version 66262 (0.0010) [2023-10-13 23:40:43,594][60935] Updated weights for policy 0, policy_version 65930 (0.0010) [2023-10-13 23:40:43,955][60935] Updated weights for policy 0, policy_version 65940 (0.0011) [2023-10-13 23:40:44,322][60935] Updated weights for policy 0, policy_version 65950 (0.0010) [2023-10-13 23:40:45,997][60934] Updated weights for policy 1, policy_version 66272 (0.0009) [2023-10-13 23:40:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 135692288. Throughput: 0: 1656.3, 1: 1645.0. Samples: 33930602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:40:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:40:46,371][60934] Updated weights for policy 1, policy_version 66282 (0.0011) [2023-10-13 23:40:46,732][60934] Updated weights for policy 1, policy_version 66292 (0.0010) [2023-10-13 23:40:48,820][60935] Updated weights for policy 0, policy_version 65960 (0.0009) [2023-10-13 23:40:49,181][60935] Updated weights for policy 0, policy_version 65970 (0.0010) [2023-10-13 23:40:49,555][60935] Updated weights for policy 0, policy_version 65980 (0.0011) [2023-10-13 23:40:51,235][60934] Updated weights for policy 1, policy_version 66302 (0.0009) [2023-10-13 23:40:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 135757824. Throughput: 0: 1647.7, 1: 1621.0. Samples: 33948790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:40:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:40:51,608][60934] Updated weights for policy 1, policy_version 66312 (0.0008) [2023-10-13 23:40:51,962][60934] Updated weights for policy 1, policy_version 66322 (0.0009) [2023-10-13 23:40:53,758][60935] Updated weights for policy 0, policy_version 65990 (0.0010) [2023-10-13 23:40:54,115][60935] Updated weights for policy 0, policy_version 66000 (0.0009) [2023-10-13 23:40:54,495][60935] Updated weights for policy 0, policy_version 66010 (0.0008) [2023-10-13 23:40:56,087][60934] Updated weights for policy 1, policy_version 66332 (0.0010) [2023-10-13 23:40:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 135823360. Throughput: 0: 1633.8, 1: 1618.4. Samples: 33968602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:40:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:40:56,257][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000066016_67600384.pth... [2023-10-13 23:40:56,287][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000064448_65994752.pth [2023-10-13 23:40:56,452][60934] Updated weights for policy 1, policy_version 66342 (0.0007) [2023-10-13 23:40:56,816][60934] Updated weights for policy 1, policy_version 66352 (0.0008) [2023-10-13 23:40:57,108][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000066360_68255744.pth... [2023-10-13 23:40:57,148][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000064792_66650112.pth [2023-10-13 23:40:58,829][60935] Updated weights for policy 0, policy_version 66020 (0.0011) [2023-10-13 23:40:59,193][60935] Updated weights for policy 0, policy_version 66030 (0.0009) [2023-10-13 23:40:59,559][60935] Updated weights for policy 0, policy_version 66040 (0.0011) [2023-10-13 23:41:01,112][60934] Updated weights for policy 1, policy_version 66362 (0.0008) [2023-10-13 23:41:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 135888896. Throughput: 0: 1634.5, 1: 1601.7. Samples: 33978156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:41:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:41:01,485][60934] Updated weights for policy 1, policy_version 66372 (0.0009) [2023-10-13 23:41:01,844][60934] Updated weights for policy 1, policy_version 66382 (0.0009) [2023-10-13 23:41:02,211][60934] Updated weights for policy 1, policy_version 66392 (0.0009) [2023-10-13 23:41:03,847][60935] Updated weights for policy 0, policy_version 66050 (0.0009) [2023-10-13 23:41:04,224][60935] Updated weights for policy 0, policy_version 66060 (0.0009) [2023-10-13 23:41:04,582][60935] Updated weights for policy 0, policy_version 66070 (0.0011) [2023-10-13 23:41:04,951][60935] Updated weights for policy 0, policy_version 66080 (0.0010) [2023-10-13 23:41:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 135954432. Throughput: 0: 1606.0, 1: 1583.9. Samples: 33996558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:41:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:41:06,631][60934] Updated weights for policy 1, policy_version 66402 (0.0007) [2023-10-13 23:41:06,994][60934] Updated weights for policy 1, policy_version 66412 (0.0008) [2023-10-13 23:41:07,363][60934] Updated weights for policy 1, policy_version 66422 (0.0008) [2023-10-13 23:41:09,086][60935] Updated weights for policy 0, policy_version 66090 (0.0009) [2023-10-13 23:41:09,460][60935] Updated weights for policy 0, policy_version 66100 (0.0010) [2023-10-13 23:41:09,825][60935] Updated weights for policy 0, policy_version 66110 (0.0008) [2023-10-13 23:41:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 136019968. Throughput: 0: 1598.0, 1: 1590.5. Samples: 34017004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:41:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:41:11,649][60934] Updated weights for policy 1, policy_version 66432 (0.0009) [2023-10-13 23:41:12,013][60934] Updated weights for policy 1, policy_version 66442 (0.0011) [2023-10-13 23:41:12,389][60934] Updated weights for policy 1, policy_version 66452 (0.0010) [2023-10-13 23:41:13,744][60935] Updated weights for policy 0, policy_version 66120 (0.0007) [2023-10-13 23:41:14,118][60935] Updated weights for policy 0, policy_version 66130 (0.0008) [2023-10-13 23:41:14,486][60935] Updated weights for policy 0, policy_version 66140 (0.0009) [2023-10-13 23:41:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 12561.0, 300 sec: 13440.4). Total num frames: 136085504. Throughput: 0: 1625.8, 1: 1578.3. Samples: 34027056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:41:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:41:16,439][60934] Updated weights for policy 1, policy_version 66462 (0.0008) [2023-10-13 23:41:16,799][60934] Updated weights for policy 1, policy_version 66472 (0.0007) [2023-10-13 23:41:17,176][60934] Updated weights for policy 1, policy_version 66482 (0.0009) [2023-10-13 23:41:18,402][60935] Updated weights for policy 0, policy_version 66150 (0.0009) [2023-10-13 23:41:18,763][60935] Updated weights for policy 0, policy_version 66160 (0.0007) [2023-10-13 23:41:19,139][60935] Updated weights for policy 0, policy_version 66170 (0.0008) [2023-10-13 23:41:21,224][60934] Updated weights for policy 1, policy_version 66492 (0.0009) [2023-10-13 23:41:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 12561.1, 300 sec: 13440.4). Total num frames: 136151040. Throughput: 0: 1624.6, 1: 1593.7. Samples: 34047190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:41:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:41:21,586][60934] Updated weights for policy 1, policy_version 66502 (0.0008) [2023-10-13 23:41:21,944][60934] Updated weights for policy 1, policy_version 66512 (0.0007) [2023-10-13 23:41:22,964][60935] Updated weights for policy 0, policy_version 66180 (0.0008) [2023-10-13 23:41:23,320][60935] Updated weights for policy 0, policy_version 66190 (0.0007) [2023-10-13 23:41:23,696][60935] Updated weights for policy 0, policy_version 66200 (0.0008) [2023-10-13 23:41:25,894][60934] Updated weights for policy 1, policy_version 66522 (0.0008) [2023-10-13 23:41:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 12561.1, 300 sec: 13440.4). Total num frames: 136216576. Throughput: 0: 1642.9, 1: 1622.0. Samples: 34068422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:41:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:41:26,261][60934] Updated weights for policy 1, policy_version 66532 (0.0008) [2023-10-13 23:41:26,618][60934] Updated weights for policy 1, policy_version 66542 (0.0008) [2023-10-13 23:41:26,988][60934] Updated weights for policy 1, policy_version 66552 (0.0009) [2023-10-13 23:41:27,705][60935] Updated weights for policy 0, policy_version 66210 (0.0009) [2023-10-13 23:41:28,083][60935] Updated weights for policy 0, policy_version 66220 (0.0008) [2023-10-13 23:41:28,451][60935] Updated weights for policy 0, policy_version 66230 (0.0008) [2023-10-13 23:41:28,817][60935] Updated weights for policy 0, policy_version 66240 (0.0010) [2023-10-13 23:41:30,986][60934] Updated weights for policy 1, policy_version 66562 (0.0007) [2023-10-13 23:41:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 136282112. Throughput: 0: 1643.3, 1: 1627.4. Samples: 34077782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:41:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:41:31,345][60934] Updated weights for policy 1, policy_version 66572 (0.0007) [2023-10-13 23:41:31,708][60934] Updated weights for policy 1, policy_version 66582 (0.0007) [2023-10-13 23:41:32,802][60935] Updated weights for policy 0, policy_version 66250 (0.0007) [2023-10-13 23:41:33,164][60935] Updated weights for policy 0, policy_version 66260 (0.0007) [2023-10-13 23:41:33,526][60935] Updated weights for policy 0, policy_version 66270 (0.0007) [2023-10-13 23:41:35,618][60934] Updated weights for policy 1, policy_version 66592 (0.0008) [2023-10-13 23:41:35,976][60934] Updated weights for policy 1, policy_version 66602 (0.0007) [2023-10-13 23:41:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 136347648. Throughput: 0: 1682.5, 1: 1651.6. Samples: 34098822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:41:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:41:36,341][60934] Updated weights for policy 1, policy_version 66612 (0.0009) [2023-10-13 23:41:37,426][60935] Updated weights for policy 0, policy_version 66280 (0.0009) [2023-10-13 23:41:37,794][60935] Updated weights for policy 0, policy_version 66290 (0.0008) [2023-10-13 23:41:38,174][60935] Updated weights for policy 0, policy_version 66300 (0.0008) [2023-10-13 23:41:40,417][60934] Updated weights for policy 1, policy_version 66622 (0.0007) [2023-10-13 23:41:40,780][60934] Updated weights for policy 1, policy_version 66632 (0.0009) [2023-10-13 23:41:41,143][60934] Updated weights for policy 1, policy_version 66642 (0.0009) [2023-10-13 23:41:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 136413184. Throughput: 0: 1697.3, 1: 1656.7. Samples: 34119532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:41:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:41:42,098][60935] Updated weights for policy 0, policy_version 66310 (0.0010) [2023-10-13 23:41:42,474][60935] Updated weights for policy 0, policy_version 66320 (0.0009) [2023-10-13 23:41:42,839][60935] Updated weights for policy 0, policy_version 66330 (0.0009) [2023-10-13 23:41:45,241][60934] Updated weights for policy 1, policy_version 66652 (0.0011) [2023-10-13 23:41:45,606][60934] Updated weights for policy 1, policy_version 66662 (0.0010) [2023-10-13 23:41:45,967][60934] Updated weights for policy 1, policy_version 66672 (0.0011) [2023-10-13 23:41:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 136478720. Throughput: 0: 1681.6, 1: 1670.7. Samples: 34129008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:41:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:41:46,923][60935] Updated weights for policy 0, policy_version 66340 (0.0009) [2023-10-13 23:41:47,303][60935] Updated weights for policy 0, policy_version 66350 (0.0009) [2023-10-13 23:41:47,670][60935] Updated weights for policy 0, policy_version 66360 (0.0009) [2023-10-13 23:41:50,034][60934] Updated weights for policy 1, policy_version 66682 (0.0011) [2023-10-13 23:41:50,397][60934] Updated weights for policy 1, policy_version 66692 (0.0009) [2023-10-13 23:41:50,775][60934] Updated weights for policy 1, policy_version 66702 (0.0008) [2023-10-13 23:41:51,142][60934] Updated weights for policy 1, policy_version 66712 (0.0008) [2023-10-13 23:41:51,248][59943] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 136577024. Throughput: 0: 1715.4, 1: 1686.2. Samples: 34149628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:41:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:41:51,709][60935] Updated weights for policy 0, policy_version 66370 (0.0008) [2023-10-13 23:41:52,104][60935] Updated weights for policy 0, policy_version 66380 (0.0009) [2023-10-13 23:41:52,482][60935] Updated weights for policy 0, policy_version 66390 (0.0009) [2023-10-13 23:41:52,852][60935] Updated weights for policy 0, policy_version 66400 (0.0008) [2023-10-13 23:41:55,167][60934] Updated weights for policy 1, policy_version 66722 (0.0010) [2023-10-13 23:41:55,538][60934] Updated weights for policy 1, policy_version 66732 (0.0011) [2023-10-13 23:41:55,903][60934] Updated weights for policy 1, policy_version 66742 (0.0007) [2023-10-13 23:41:56,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 136642560. Throughput: 0: 1725.3, 1: 1672.3. Samples: 34169894. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-13 23:41:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:41:56,820][60935] Updated weights for policy 0, policy_version 66410 (0.0009) [2023-10-13 23:41:57,191][60935] Updated weights for policy 0, policy_version 66420 (0.0007) [2023-10-13 23:41:57,553][60935] Updated weights for policy 0, policy_version 66430 (0.0010) [2023-10-13 23:42:00,065][60934] Updated weights for policy 1, policy_version 66752 (0.0009) [2023-10-13 23:42:00,428][60934] Updated weights for policy 1, policy_version 66762 (0.0011) [2023-10-13 23:42:00,795][60934] Updated weights for policy 1, policy_version 66772 (0.0010) [2023-10-13 23:42:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 136708096. Throughput: 0: 1702.5, 1: 1692.6. Samples: 34179834. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-13 23:42:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:42:01,496][60935] Updated weights for policy 0, policy_version 66440 (0.0007) [2023-10-13 23:42:01,853][60935] Updated weights for policy 0, policy_version 66450 (0.0010) [2023-10-13 23:42:02,229][60935] Updated weights for policy 0, policy_version 66460 (0.0008) [2023-10-13 23:42:04,876][60934] Updated weights for policy 1, policy_version 66782 (0.0009) [2023-10-13 23:42:05,240][60934] Updated weights for policy 1, policy_version 66792 (0.0007) [2023-10-13 23:42:05,604][60934] Updated weights for policy 1, policy_version 66802 (0.0007) [2023-10-13 23:42:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 136773632. Throughput: 0: 1720.1, 1: 1693.8. Samples: 34200816. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-13 23:42:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:42:06,347][60935] Updated weights for policy 0, policy_version 66470 (0.0008) [2023-10-13 23:42:06,715][60935] Updated weights for policy 0, policy_version 66480 (0.0010) [2023-10-13 23:42:07,082][60935] Updated weights for policy 0, policy_version 66490 (0.0008) [2023-10-13 23:42:09,523][60934] Updated weights for policy 1, policy_version 66812 (0.0009) [2023-10-13 23:42:09,903][60934] Updated weights for policy 1, policy_version 66822 (0.0011) [2023-10-13 23:42:10,276][60934] Updated weights for policy 1, policy_version 66832 (0.0011) [2023-10-13 23:42:10,844][60935] Updated weights for policy 0, policy_version 66500 (0.0008) [2023-10-13 23:42:11,214][60935] Updated weights for policy 0, policy_version 66510 (0.0007) [2023-10-13 23:42:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 136839168. Throughput: 0: 1725.0, 1: 1666.0. Samples: 34221018. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-13 23:42:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:42:11,575][60935] Updated weights for policy 0, policy_version 66520 (0.0010) [2023-10-13 23:42:14,244][60934] Updated weights for policy 1, policy_version 66842 (0.0008) [2023-10-13 23:42:14,611][60934] Updated weights for policy 1, policy_version 66852 (0.0008) [2023-10-13 23:42:14,981][60934] Updated weights for policy 1, policy_version 66862 (0.0008) [2023-10-13 23:42:15,345][60934] Updated weights for policy 1, policy_version 66872 (0.0008) [2023-10-13 23:42:15,539][60935] Updated weights for policy 0, policy_version 66530 (0.0011) [2023-10-13 23:42:15,907][60935] Updated weights for policy 0, policy_version 66540 (0.0007) [2023-10-13 23:42:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 136904704. Throughput: 0: 1725.2, 1: 1692.3. Samples: 34231566. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-13 23:42:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:42:16,269][60935] Updated weights for policy 0, policy_version 66550 (0.0010) [2023-10-13 23:42:16,638][60935] Updated weights for policy 0, policy_version 66560 (0.0009) [2023-10-13 23:42:19,532][60934] Updated weights for policy 1, policy_version 66882 (0.0007) [2023-10-13 23:42:19,901][60934] Updated weights for policy 1, policy_version 66892 (0.0008) [2023-10-13 23:42:20,270][60934] Updated weights for policy 1, policy_version 66902 (0.0008) [2023-10-13 23:42:20,523][60935] Updated weights for policy 0, policy_version 66570 (0.0010) [2023-10-13 23:42:20,896][60935] Updated weights for policy 0, policy_version 66580 (0.0009) [2023-10-13 23:42:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 136970240. Throughput: 0: 1730.4, 1: 1688.4. Samples: 34252670. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-13 23:42:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:42:21,267][60935] Updated weights for policy 0, policy_version 66590 (0.0010) [2023-10-13 23:42:24,153][60934] Updated weights for policy 1, policy_version 66912 (0.0007) [2023-10-13 23:42:24,510][60934] Updated weights for policy 1, policy_version 66922 (0.0009) [2023-10-13 23:42:24,883][60934] Updated weights for policy 1, policy_version 66932 (0.0009) [2023-10-13 23:42:25,307][60935] Updated weights for policy 0, policy_version 66600 (0.0008) [2023-10-13 23:42:25,681][60935] Updated weights for policy 0, policy_version 66610 (0.0010) [2023-10-13 23:42:26,054][60935] Updated weights for policy 0, policy_version 66620 (0.0010) [2023-10-13 23:42:26,248][59943] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 137068544. Throughput: 0: 1706.9, 1: 1684.1. Samples: 34272128. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-13 23:42:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:42:28,987][60934] Updated weights for policy 1, policy_version 66942 (0.0009) [2023-10-13 23:42:29,357][60934] Updated weights for policy 1, policy_version 66952 (0.0007) [2023-10-13 23:42:29,719][60934] Updated weights for policy 1, policy_version 66962 (0.0007) [2023-10-13 23:42:30,162][60935] Updated weights for policy 0, policy_version 66630 (0.0009) [2023-10-13 23:42:30,534][60935] Updated weights for policy 0, policy_version 66640 (0.0008) [2023-10-13 23:42:30,897][60935] Updated weights for policy 0, policy_version 66650 (0.0009) [2023-10-13 23:42:31,248][59943] Fps is (10 sec: 16383.5, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 137134080. Throughput: 0: 1724.0, 1: 1701.5. Samples: 34283156. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-13 23:42:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:42:33,606][60934] Updated weights for policy 1, policy_version 66972 (0.0008) [2023-10-13 23:42:33,963][60934] Updated weights for policy 1, policy_version 66982 (0.0008) [2023-10-13 23:42:34,331][60934] Updated weights for policy 1, policy_version 66992 (0.0008) [2023-10-13 23:42:34,914][60935] Updated weights for policy 0, policy_version 66660 (0.0009) [2023-10-13 23:42:35,286][60935] Updated weights for policy 0, policy_version 66670 (0.0008) [2023-10-13 23:42:35,653][60935] Updated weights for policy 0, policy_version 66680 (0.0008) [2023-10-13 23:42:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 137199616. Throughput: 0: 1729.0, 1: 1685.2. Samples: 34303266. Policy #0 lag: (min: 22.0, avg: 24.8, max: 54.0) [2023-10-13 23:42:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:42:38,213][60934] Updated weights for policy 1, policy_version 67002 (0.0008) [2023-10-13 23:42:38,586][60934] Updated weights for policy 1, policy_version 67012 (0.0007) [2023-10-13 23:42:38,959][60934] Updated weights for policy 1, policy_version 67022 (0.0008) [2023-10-13 23:42:39,321][60934] Updated weights for policy 1, policy_version 67032 (0.0007) [2023-10-13 23:42:39,619][60935] Updated weights for policy 0, policy_version 66690 (0.0009) [2023-10-13 23:42:39,993][60935] Updated weights for policy 0, policy_version 66700 (0.0008) [2023-10-13 23:42:40,364][60935] Updated weights for policy 0, policy_version 66710 (0.0008) [2023-10-13 23:42:40,722][60935] Updated weights for policy 0, policy_version 66720 (0.0009) [2023-10-13 23:42:41,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 137265152. Throughput: 0: 1698.5, 1: 1702.4. Samples: 34322934. Policy #0 lag: (min: 22.0, avg: 24.8, max: 54.0) [2023-10-13 23:42:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:42:43,526][60934] Updated weights for policy 1, policy_version 67042 (0.0007) [2023-10-13 23:42:43,912][60934] Updated weights for policy 1, policy_version 67052 (0.0007) [2023-10-13 23:42:44,280][60934] Updated weights for policy 1, policy_version 67062 (0.0007) [2023-10-13 23:42:44,660][60935] Updated weights for policy 0, policy_version 66730 (0.0008) [2023-10-13 23:42:45,025][60935] Updated weights for policy 0, policy_version 66740 (0.0008) [2023-10-13 23:42:45,400][60935] Updated weights for policy 0, policy_version 66750 (0.0009) [2023-10-13 23:42:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 137330688. Throughput: 0: 1729.6, 1: 1701.6. Samples: 34334240. Policy #0 lag: (min: 22.0, avg: 24.8, max: 54.0) [2023-10-13 23:42:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:42:48,225][60934] Updated weights for policy 1, policy_version 67072 (0.0007) [2023-10-13 23:42:48,591][60934] Updated weights for policy 1, policy_version 67082 (0.0008) [2023-10-13 23:42:48,956][60934] Updated weights for policy 1, policy_version 67092 (0.0010) [2023-10-13 23:42:49,400][60935] Updated weights for policy 0, policy_version 66760 (0.0010) [2023-10-13 23:42:49,772][60935] Updated weights for policy 0, policy_version 66770 (0.0009) [2023-10-13 23:42:50,144][60935] Updated weights for policy 0, policy_version 66780 (0.0011) [2023-10-13 23:42:51,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 137396224. Throughput: 0: 1715.0, 1: 1682.1. Samples: 34353686. Policy #0 lag: (min: 22.0, avg: 24.8, max: 54.0) [2023-10-13 23:42:51,250][59943] Avg episode reward: [(0, '0.000'), (1, '-0.120')] [2023-10-13 23:42:53,104][60934] Updated weights for policy 1, policy_version 67102 (0.0010) [2023-10-13 23:42:53,477][60934] Updated weights for policy 1, policy_version 67112 (0.0008) [2023-10-13 23:42:53,844][60934] Updated weights for policy 1, policy_version 67122 (0.0008) [2023-10-13 23:42:53,849][60935] Updated weights for policy 0, policy_version 66790 (0.0008) [2023-10-13 23:42:54,223][60935] Updated weights for policy 0, policy_version 66800 (0.0008) [2023-10-13 23:42:54,585][60935] Updated weights for policy 0, policy_version 66810 (0.0008) [2023-10-13 23:42:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 137461760. Throughput: 0: 1701.1, 1: 1705.8. Samples: 34374330. Policy #0 lag: (min: 22.0, avg: 24.8, max: 54.0) [2023-10-13 23:42:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.240')] [2023-10-13 23:42:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000067128_69042176.pth... [2023-10-13 23:42:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000066816_68419584.pth... [2023-10-13 23:42:56,297][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000065560_67436544.pth [2023-10-13 23:42:56,298][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000065248_66813952.pth [2023-10-13 23:42:57,893][60934] Updated weights for policy 1, policy_version 67132 (0.0010) [2023-10-13 23:42:58,257][60934] Updated weights for policy 1, policy_version 67142 (0.0009) [2023-10-13 23:42:58,588][60935] Updated weights for policy 0, policy_version 66820 (0.0008) [2023-10-13 23:42:58,628][60934] Updated weights for policy 1, policy_version 67152 (0.0007) [2023-10-13 23:42:58,950][60935] Updated weights for policy 0, policy_version 66830 (0.0008) [2023-10-13 23:42:59,308][60935] Updated weights for policy 0, policy_version 66840 (0.0009) [2023-10-13 23:43:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 137527296. Throughput: 0: 1718.6, 1: 1687.3. Samples: 34384830. Policy #0 lag: (min: 22.0, avg: 24.8, max: 54.0) [2023-10-13 23:43:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.240')] [2023-10-13 23:43:02,646][60934] Updated weights for policy 1, policy_version 67162 (0.0007) [2023-10-13 23:43:03,012][60934] Updated weights for policy 1, policy_version 67172 (0.0008) [2023-10-13 23:43:03,267][60935] Updated weights for policy 0, policy_version 66850 (0.0008) [2023-10-13 23:43:03,377][60934] Updated weights for policy 1, policy_version 67182 (0.0008) [2023-10-13 23:43:03,625][60935] Updated weights for policy 0, policy_version 66860 (0.0009) [2023-10-13 23:43:03,734][60934] Updated weights for policy 1, policy_version 67192 (0.0009) [2023-10-13 23:43:03,999][60935] Updated weights for policy 0, policy_version 66870 (0.0009) [2023-10-13 23:43:04,360][60935] Updated weights for policy 0, policy_version 66880 (0.0010) [2023-10-13 23:43:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 137592832. Throughput: 0: 1694.0, 1: 1683.5. Samples: 34404660. Policy #0 lag: (min: 22.0, avg: 24.8, max: 54.0) [2023-10-13 23:43:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.240')] [2023-10-13 23:43:07,740][60934] Updated weights for policy 1, policy_version 67202 (0.0009) [2023-10-13 23:43:08,107][60934] Updated weights for policy 1, policy_version 67212 (0.0007) [2023-10-13 23:43:08,350][60935] Updated weights for policy 0, policy_version 66890 (0.0009) [2023-10-13 23:43:08,473][60934] Updated weights for policy 1, policy_version 67222 (0.0007) [2023-10-13 23:43:08,717][60935] Updated weights for policy 0, policy_version 66900 (0.0009) [2023-10-13 23:43:09,089][60935] Updated weights for policy 0, policy_version 66910 (0.0008) [2023-10-13 23:43:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 137658368. Throughput: 0: 1720.0, 1: 1697.7. Samples: 34425924. Policy #0 lag: (min: 22.0, avg: 24.8, max: 54.0) [2023-10-13 23:43:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:43:12,436][60934] Updated weights for policy 1, policy_version 67232 (0.0007) [2023-10-13 23:43:12,802][60934] Updated weights for policy 1, policy_version 67242 (0.0009) [2023-10-13 23:43:13,015][60935] Updated weights for policy 0, policy_version 66920 (0.0009) [2023-10-13 23:43:13,162][60934] Updated weights for policy 1, policy_version 67252 (0.0007) [2023-10-13 23:43:13,379][60935] Updated weights for policy 0, policy_version 66930 (0.0010) [2023-10-13 23:43:13,745][60935] Updated weights for policy 0, policy_version 66940 (0.0010) [2023-10-13 23:43:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 137723904. Throughput: 0: 1706.7, 1: 1675.3. Samples: 34435346. Policy #0 lag: (min: 22.0, avg: 24.8, max: 54.0) [2023-10-13 23:43:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:43:17,250][60934] Updated weights for policy 1, policy_version 67262 (0.0009) [2023-10-13 23:43:17,627][60934] Updated weights for policy 1, policy_version 67272 (0.0007) [2023-10-13 23:43:17,831][60935] Updated weights for policy 0, policy_version 66950 (0.0009) [2023-10-13 23:43:17,994][60934] Updated weights for policy 1, policy_version 67282 (0.0007) [2023-10-13 23:43:18,201][60935] Updated weights for policy 0, policy_version 66960 (0.0008) [2023-10-13 23:43:18,568][60935] Updated weights for policy 0, policy_version 66970 (0.0011) [2023-10-13 23:43:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 137789440. Throughput: 0: 1702.0, 1: 1700.6. Samples: 34456382. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-13 23:43:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:43:22,020][60934] Updated weights for policy 1, policy_version 67292 (0.0007) [2023-10-13 23:43:22,377][60934] Updated weights for policy 1, policy_version 67302 (0.0008) [2023-10-13 23:43:22,491][60935] Updated weights for policy 0, policy_version 66980 (0.0009) [2023-10-13 23:43:22,749][60934] Updated weights for policy 1, policy_version 67312 (0.0010) [2023-10-13 23:43:22,862][60935] Updated weights for policy 0, policy_version 66990 (0.0008) [2023-10-13 23:43:23,236][60935] Updated weights for policy 0, policy_version 67000 (0.0008) [2023-10-13 23:43:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 137854976. Throughput: 0: 1728.8, 1: 1706.0. Samples: 34477504. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-13 23:43:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:43:26,724][60934] Updated weights for policy 1, policy_version 67322 (0.0008) [2023-10-13 23:43:27,090][60934] Updated weights for policy 1, policy_version 67332 (0.0007) [2023-10-13 23:43:27,233][60935] Updated weights for policy 0, policy_version 67010 (0.0009) [2023-10-13 23:43:27,447][60934] Updated weights for policy 1, policy_version 67342 (0.0007) [2023-10-13 23:43:27,639][60935] Updated weights for policy 0, policy_version 67020 (0.0008) [2023-10-13 23:43:27,820][60934] Updated weights for policy 1, policy_version 67352 (0.0007) [2023-10-13 23:43:28,014][60935] Updated weights for policy 0, policy_version 67030 (0.0009) [2023-10-13 23:43:28,377][60935] Updated weights for policy 0, policy_version 67040 (0.0010) [2023-10-13 23:43:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 137920512. Throughput: 0: 1698.8, 1: 1691.2. Samples: 34486790. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-13 23:43:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:43:31,885][60934] Updated weights for policy 1, policy_version 67362 (0.0008) [2023-10-13 23:43:32,216][60935] Updated weights for policy 0, policy_version 67050 (0.0009) [2023-10-13 23:43:32,252][60934] Updated weights for policy 1, policy_version 67372 (0.0009) [2023-10-13 23:43:32,580][60935] Updated weights for policy 0, policy_version 67060 (0.0008) [2023-10-13 23:43:32,622][60934] Updated weights for policy 1, policy_version 67382 (0.0007) [2023-10-13 23:43:32,951][60935] Updated weights for policy 0, policy_version 67070 (0.0007) [2023-10-13 23:43:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 137986048. Throughput: 0: 1720.8, 1: 1707.3. Samples: 34507950. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-13 23:43:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:43:36,587][60934] Updated weights for policy 1, policy_version 67392 (0.0008) [2023-10-13 23:43:36,825][60935] Updated weights for policy 0, policy_version 67080 (0.0008) [2023-10-13 23:43:36,945][60934] Updated weights for policy 1, policy_version 67402 (0.0008) [2023-10-13 23:43:37,191][60935] Updated weights for policy 0, policy_version 67090 (0.0009) [2023-10-13 23:43:37,310][60934] Updated weights for policy 1, policy_version 67412 (0.0007) [2023-10-13 23:43:37,556][60935] Updated weights for policy 0, policy_version 67100 (0.0010) [2023-10-13 23:43:41,248][59943] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 138051584. Throughput: 0: 1730.6, 1: 1709.4. Samples: 34529132. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-13 23:43:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:43:41,388][60934] Updated weights for policy 1, policy_version 67422 (0.0007) [2023-10-13 23:43:41,606][60935] Updated weights for policy 0, policy_version 67110 (0.0007) [2023-10-13 23:43:41,759][60934] Updated weights for policy 1, policy_version 67432 (0.0008) [2023-10-13 23:43:41,962][60935] Updated weights for policy 0, policy_version 67120 (0.0008) [2023-10-13 23:43:42,125][60934] Updated weights for policy 1, policy_version 67442 (0.0007) [2023-10-13 23:43:42,331][60935] Updated weights for policy 0, policy_version 67130 (0.0008) [2023-10-13 23:43:46,234][60935] Updated weights for policy 0, policy_version 67140 (0.0011) [2023-10-13 23:43:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 138117120. Throughput: 0: 1711.4, 1: 1700.9. Samples: 34538384. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-13 23:43:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:43:46,249][60934] Updated weights for policy 1, policy_version 67452 (0.0009) [2023-10-13 23:43:46,600][60935] Updated weights for policy 0, policy_version 67150 (0.0007) [2023-10-13 23:43:46,603][60934] Updated weights for policy 1, policy_version 67462 (0.0007) [2023-10-13 23:43:46,964][60935] Updated weights for policy 0, policy_version 67160 (0.0008) [2023-10-13 23:43:46,965][60934] Updated weights for policy 1, policy_version 67472 (0.0008) [2023-10-13 23:43:50,942][60934] Updated weights for policy 1, policy_version 67482 (0.0008) [2023-10-13 23:43:51,030][60935] Updated weights for policy 0, policy_version 67170 (0.0008) [2023-10-13 23:43:51,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 138182656. Throughput: 0: 1729.1, 1: 1711.5. Samples: 34559486. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-13 23:43:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:43:51,311][60934] Updated weights for policy 1, policy_version 67492 (0.0007) [2023-10-13 23:43:51,394][60935] Updated weights for policy 0, policy_version 67180 (0.0007) [2023-10-13 23:43:51,674][60934] Updated weights for policy 1, policy_version 67502 (0.0007) [2023-10-13 23:43:51,759][60935] Updated weights for policy 0, policy_version 67190 (0.0008) [2023-10-13 23:43:52,033][60934] Updated weights for policy 1, policy_version 67512 (0.0009) [2023-10-13 23:43:52,119][60935] Updated weights for policy 0, policy_version 67200 (0.0008) [2023-10-13 23:43:56,023][60934] Updated weights for policy 1, policy_version 67522 (0.0007) [2023-10-13 23:43:56,144][60935] Updated weights for policy 0, policy_version 67210 (0.0009) [2023-10-13 23:43:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 138248192. Throughput: 0: 1721.8, 1: 1705.5. Samples: 34580154. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-13 23:43:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:43:56,386][60934] Updated weights for policy 1, policy_version 67532 (0.0007) [2023-10-13 23:43:56,510][60935] Updated weights for policy 0, policy_version 67220 (0.0007) [2023-10-13 23:43:56,754][60934] Updated weights for policy 1, policy_version 67542 (0.0007) [2023-10-13 23:43:56,882][60935] Updated weights for policy 0, policy_version 67230 (0.0008) [2023-10-13 23:44:00,793][60934] Updated weights for policy 1, policy_version 67552 (0.0009) [2023-10-13 23:44:00,905][60935] Updated weights for policy 0, policy_version 67240 (0.0008) [2023-10-13 23:44:01,158][60934] Updated weights for policy 1, policy_version 67562 (0.0008) [2023-10-13 23:44:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 138313728. Throughput: 0: 1718.2, 1: 1704.6. Samples: 34589370. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-13 23:44:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:44:01,269][60935] Updated weights for policy 0, policy_version 67250 (0.0009) [2023-10-13 23:44:01,525][60934] Updated weights for policy 1, policy_version 67572 (0.0008) [2023-10-13 23:44:01,629][60935] Updated weights for policy 0, policy_version 67260 (0.0011) [2023-10-13 23:44:05,633][60934] Updated weights for policy 1, policy_version 67582 (0.0008) [2023-10-13 23:44:05,688][60935] Updated weights for policy 0, policy_version 67270 (0.0009) [2023-10-13 23:44:05,996][60934] Updated weights for policy 1, policy_version 67592 (0.0007) [2023-10-13 23:44:06,052][60935] Updated weights for policy 0, policy_version 67280 (0.0010) [2023-10-13 23:44:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 138379264. Throughput: 0: 1724.8, 1: 1693.0. Samples: 34610186. Policy #0 lag: (min: 2.0, avg: 5.0, max: 34.0) [2023-10-13 23:44:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:44:06,373][60934] Updated weights for policy 1, policy_version 67602 (0.0008) [2023-10-13 23:44:06,422][60935] Updated weights for policy 0, policy_version 67290 (0.0009) [2023-10-13 23:44:10,402][60935] Updated weights for policy 0, policy_version 67300 (0.0010) [2023-10-13 23:44:10,407][60934] Updated weights for policy 1, policy_version 67612 (0.0008) [2023-10-13 23:44:10,763][60935] Updated weights for policy 0, policy_version 67310 (0.0007) [2023-10-13 23:44:10,777][60934] Updated weights for policy 1, policy_version 67622 (0.0008) [2023-10-13 23:44:11,130][60935] Updated weights for policy 0, policy_version 67320 (0.0008) [2023-10-13 23:44:11,141][60934] Updated weights for policy 1, policy_version 67632 (0.0008) [2023-10-13 23:44:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138444800. Throughput: 0: 1707.7, 1: 1685.5. Samples: 34630200. Policy #0 lag: (min: 2.0, avg: 5.0, max: 34.0) [2023-10-13 23:44:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:44:15,270][60934] Updated weights for policy 1, policy_version 67642 (0.0008) [2023-10-13 23:44:15,322][60935] Updated weights for policy 0, policy_version 67330 (0.0008) [2023-10-13 23:44:15,633][60934] Updated weights for policy 1, policy_version 67652 (0.0008) [2023-10-13 23:44:15,723][60935] Updated weights for policy 0, policy_version 67340 (0.0009) [2023-10-13 23:44:15,999][60934] Updated weights for policy 1, policy_version 67662 (0.0007) [2023-10-13 23:44:16,085][60935] Updated weights for policy 0, policy_version 67350 (0.0008) [2023-10-13 23:44:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138510336. Throughput: 0: 1718.9, 1: 1692.3. Samples: 34640296. Policy #0 lag: (min: 2.0, avg: 5.0, max: 34.0) [2023-10-13 23:44:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:44:16,356][60934] Updated weights for policy 1, policy_version 67672 (0.0008) [2023-10-13 23:44:16,451][60935] Updated weights for policy 0, policy_version 67360 (0.0007) [2023-10-13 23:44:20,311][60934] Updated weights for policy 1, policy_version 67682 (0.0008) [2023-10-13 23:44:20,419][60935] Updated weights for policy 0, policy_version 67370 (0.0008) [2023-10-13 23:44:20,680][60934] Updated weights for policy 1, policy_version 67692 (0.0009) [2023-10-13 23:44:20,790][60935] Updated weights for policy 0, policy_version 67380 (0.0010) [2023-10-13 23:44:21,047][60934] Updated weights for policy 1, policy_version 67702 (0.0009) [2023-10-13 23:44:21,155][60935] Updated weights for policy 0, policy_version 67390 (0.0010) [2023-10-13 23:44:21,248][59943] Fps is (10 sec: 19661.1, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 138641408. Throughput: 0: 1711.4, 1: 1693.6. Samples: 34661178. Policy #0 lag: (min: 2.0, avg: 5.0, max: 34.0) [2023-10-13 23:44:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:44:25,131][60935] Updated weights for policy 0, policy_version 67400 (0.0009) [2023-10-13 23:44:25,204][60934] Updated weights for policy 1, policy_version 67712 (0.0007) [2023-10-13 23:44:25,498][60935] Updated weights for policy 0, policy_version 67410 (0.0008) [2023-10-13 23:44:25,578][60934] Updated weights for policy 1, policy_version 67722 (0.0008) [2023-10-13 23:44:25,867][60935] Updated weights for policy 0, policy_version 67420 (0.0009) [2023-10-13 23:44:25,938][60934] Updated weights for policy 1, policy_version 67732 (0.0008) [2023-10-13 23:44:26,248][59943] Fps is (10 sec: 19660.9, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 138706944. Throughput: 0: 1686.9, 1: 1674.6. Samples: 34680400. Policy #0 lag: (min: 2.0, avg: 5.0, max: 34.0) [2023-10-13 23:44:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:44:29,866][60935] Updated weights for policy 0, policy_version 67430 (0.0009) [2023-10-13 23:44:29,957][60934] Updated weights for policy 1, policy_version 67742 (0.0009) [2023-10-13 23:44:30,231][60935] Updated weights for policy 0, policy_version 67440 (0.0008) [2023-10-13 23:44:30,325][60934] Updated weights for policy 1, policy_version 67752 (0.0009) [2023-10-13 23:44:30,595][60935] Updated weights for policy 0, policy_version 67450 (0.0007) [2023-10-13 23:44:30,683][60934] Updated weights for policy 1, policy_version 67762 (0.0008) [2023-10-13 23:44:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 138772480. Throughput: 0: 1708.3, 1: 1689.7. Samples: 34691296. Policy #0 lag: (min: 2.0, avg: 5.0, max: 34.0) [2023-10-13 23:44:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:44:34,695][60935] Updated weights for policy 0, policy_version 67460 (0.0009) [2023-10-13 23:44:34,730][60934] Updated weights for policy 1, policy_version 67772 (0.0007) [2023-10-13 23:44:35,062][60935] Updated weights for policy 0, policy_version 67470 (0.0009) [2023-10-13 23:44:35,100][60934] Updated weights for policy 1, policy_version 67782 (0.0007) [2023-10-13 23:44:35,430][60935] Updated weights for policy 0, policy_version 67480 (0.0010) [2023-10-13 23:44:35,460][60934] Updated weights for policy 1, policy_version 67792 (0.0007) [2023-10-13 23:44:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 138838016. Throughput: 0: 1698.7, 1: 1688.8. Samples: 34711922. Policy #0 lag: (min: 2.0, avg: 5.0, max: 34.0) [2023-10-13 23:44:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:44:39,388][60935] Updated weights for policy 0, policy_version 67490 (0.0007) [2023-10-13 23:44:39,457][60934] Updated weights for policy 1, policy_version 67802 (0.0007) [2023-10-13 23:44:39,755][60935] Updated weights for policy 0, policy_version 67500 (0.0007) [2023-10-13 23:44:39,829][60934] Updated weights for policy 1, policy_version 67812 (0.0007) [2023-10-13 23:44:40,130][60935] Updated weights for policy 0, policy_version 67510 (0.0008) [2023-10-13 23:44:40,197][60934] Updated weights for policy 1, policy_version 67822 (0.0008) [2023-10-13 23:44:40,492][60935] Updated weights for policy 0, policy_version 67520 (0.0008) [2023-10-13 23:44:40,563][60934] Updated weights for policy 1, policy_version 67832 (0.0008) [2023-10-13 23:44:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 138903552. Throughput: 0: 1677.7, 1: 1669.5. Samples: 34730778. Policy #0 lag: (min: 2.0, avg: 5.0, max: 34.0) [2023-10-13 23:44:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:44:44,435][60935] Updated weights for policy 0, policy_version 67530 (0.0008) [2023-10-13 23:44:44,695][60934] Updated weights for policy 1, policy_version 67842 (0.0008) [2023-10-13 23:44:44,805][60935] Updated weights for policy 0, policy_version 67540 (0.0009) [2023-10-13 23:44:45,065][60934] Updated weights for policy 1, policy_version 67852 (0.0008) [2023-10-13 23:44:45,178][60935] Updated weights for policy 0, policy_version 67550 (0.0008) [2023-10-13 23:44:45,428][60934] Updated weights for policy 1, policy_version 67862 (0.0009) [2023-10-13 23:44:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 138969088. Throughput: 0: 1707.2, 1: 1697.7. Samples: 34742590. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-13 23:44:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-13 23:44:49,296][60935] Updated weights for policy 0, policy_version 67560 (0.0008) [2023-10-13 23:44:49,305][60934] Updated weights for policy 1, policy_version 67872 (0.0010) [2023-10-13 23:44:49,665][60935] Updated weights for policy 0, policy_version 67570 (0.0009) [2023-10-13 23:44:49,673][60934] Updated weights for policy 1, policy_version 67882 (0.0008) [2023-10-13 23:44:50,039][60934] Updated weights for policy 1, policy_version 67892 (0.0008) [2023-10-13 23:44:50,043][60935] Updated weights for policy 0, policy_version 67580 (0.0010) [2023-10-13 23:44:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 139034624. Throughput: 0: 1683.8, 1: 1688.9. Samples: 34761958. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-13 23:44:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-13 23:44:54,033][60935] Updated weights for policy 0, policy_version 67590 (0.0008) [2023-10-13 23:44:54,274][60934] Updated weights for policy 1, policy_version 67902 (0.0007) [2023-10-13 23:44:54,396][60935] Updated weights for policy 0, policy_version 67600 (0.0009) [2023-10-13 23:44:54,643][60934] Updated weights for policy 1, policy_version 67912 (0.0009) [2023-10-13 23:44:54,773][60935] Updated weights for policy 0, policy_version 67610 (0.0007) [2023-10-13 23:44:55,018][60934] Updated weights for policy 1, policy_version 67922 (0.0009) [2023-10-13 23:44:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 139100160. Throughput: 0: 1686.9, 1: 1677.5. Samples: 34781596. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-13 23:44:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-13 23:44:56,262][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000067928_69861376.pth... [2023-10-13 23:44:56,262][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000067616_69238784.pth... [2023-10-13 23:44:56,300][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000066360_68255744.pth [2023-10-13 23:44:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000066016_67600384.pth [2023-10-13 23:44:58,821][60935] Updated weights for policy 0, policy_version 67620 (0.0009) [2023-10-13 23:44:59,051][60934] Updated weights for policy 1, policy_version 67932 (0.0008) [2023-10-13 23:44:59,199][60935] Updated weights for policy 0, policy_version 67630 (0.0008) [2023-10-13 23:44:59,417][60934] Updated weights for policy 1, policy_version 67942 (0.0009) [2023-10-13 23:44:59,566][60935] Updated weights for policy 0, policy_version 67640 (0.0009) [2023-10-13 23:44:59,784][60934] Updated weights for policy 1, policy_version 67952 (0.0008) [2023-10-13 23:45:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 139165696. Throughput: 0: 1700.3, 1: 1699.1. Samples: 34793266. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-13 23:45:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:45:03,712][60935] Updated weights for policy 0, policy_version 67650 (0.0008) [2023-10-13 23:45:03,770][60934] Updated weights for policy 1, policy_version 67962 (0.0011) [2023-10-13 23:45:04,086][60935] Updated weights for policy 0, policy_version 67660 (0.0007) [2023-10-13 23:45:04,136][60934] Updated weights for policy 1, policy_version 67972 (0.0008) [2023-10-13 23:45:04,467][60935] Updated weights for policy 0, policy_version 67670 (0.0008) [2023-10-13 23:45:04,503][60934] Updated weights for policy 1, policy_version 67982 (0.0009) [2023-10-13 23:45:04,829][60935] Updated weights for policy 0, policy_version 67680 (0.0009) [2023-10-13 23:45:04,864][60934] Updated weights for policy 1, policy_version 67992 (0.0010) [2023-10-13 23:45:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 139231232. Throughput: 0: 1673.7, 1: 1681.3. Samples: 34812156. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-13 23:45:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:45:08,833][60934] Updated weights for policy 1, policy_version 68002 (0.0008) [2023-10-13 23:45:08,849][60935] Updated weights for policy 0, policy_version 67690 (0.0010) [2023-10-13 23:45:09,203][60934] Updated weights for policy 1, policy_version 68012 (0.0008) [2023-10-13 23:45:09,210][60935] Updated weights for policy 0, policy_version 67700 (0.0008) [2023-10-13 23:45:09,575][60934] Updated weights for policy 1, policy_version 68022 (0.0008) [2023-10-13 23:45:09,581][60935] Updated weights for policy 0, policy_version 67710 (0.0007) [2023-10-13 23:45:11,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 139296768. Throughput: 0: 1695.7, 1: 1685.7. Samples: 34832564. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-13 23:45:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:45:13,551][60934] Updated weights for policy 1, policy_version 68032 (0.0008) [2023-10-13 23:45:13,554][60935] Updated weights for policy 0, policy_version 67720 (0.0008) [2023-10-13 23:45:13,913][60934] Updated weights for policy 1, policy_version 68042 (0.0009) [2023-10-13 23:45:13,921][60935] Updated weights for policy 0, policy_version 67730 (0.0009) [2023-10-13 23:45:14,279][60934] Updated weights for policy 1, policy_version 68052 (0.0008) [2023-10-13 23:45:14,286][60935] Updated weights for policy 0, policy_version 67740 (0.0009) [2023-10-13 23:45:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 139362304. Throughput: 0: 1686.7, 1: 1693.2. Samples: 34843390. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-13 23:45:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:45:18,306][60935] Updated weights for policy 0, policy_version 67750 (0.0008) [2023-10-13 23:45:18,372][60934] Updated weights for policy 1, policy_version 68062 (0.0007) [2023-10-13 23:45:18,670][60935] Updated weights for policy 0, policy_version 67760 (0.0007) [2023-10-13 23:45:18,734][60934] Updated weights for policy 1, policy_version 68072 (0.0009) [2023-10-13 23:45:19,047][60935] Updated weights for policy 0, policy_version 67770 (0.0008) [2023-10-13 23:45:19,104][60934] Updated weights for policy 1, policy_version 68082 (0.0008) [2023-10-13 23:45:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 139427840. Throughput: 0: 1680.2, 1: 1672.0. Samples: 34862768. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-13 23:45:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:45:22,889][60934] Updated weights for policy 1, policy_version 68092 (0.0008) [2023-10-13 23:45:22,980][60935] Updated weights for policy 0, policy_version 67780 (0.0009) [2023-10-13 23:45:23,254][60934] Updated weights for policy 1, policy_version 68102 (0.0007) [2023-10-13 23:45:23,338][60935] Updated weights for policy 0, policy_version 67790 (0.0009) [2023-10-13 23:45:23,618][60934] Updated weights for policy 1, policy_version 68112 (0.0007) [2023-10-13 23:45:23,715][60935] Updated weights for policy 0, policy_version 67800 (0.0007) [2023-10-13 23:45:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 139493376. Throughput: 0: 1700.3, 1: 1704.3. Samples: 34883984. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-13 23:45:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:45:27,770][60934] Updated weights for policy 1, policy_version 68122 (0.0009) [2023-10-13 23:45:27,791][60935] Updated weights for policy 0, policy_version 67810 (0.0009) [2023-10-13 23:45:28,135][60934] Updated weights for policy 1, policy_version 68132 (0.0007) [2023-10-13 23:45:28,164][60935] Updated weights for policy 0, policy_version 67820 (0.0009) [2023-10-13 23:45:28,496][60934] Updated weights for policy 1, policy_version 68142 (0.0007) [2023-10-13 23:45:28,525][60935] Updated weights for policy 0, policy_version 67830 (0.0009) [2023-10-13 23:45:28,858][60934] Updated weights for policy 1, policy_version 68152 (0.0007) [2023-10-13 23:45:28,890][60935] Updated weights for policy 0, policy_version 67840 (0.0010) [2023-10-13 23:45:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 139558912. Throughput: 0: 1674.6, 1: 1685.4. Samples: 34893790. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:45:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:45:32,987][60935] Updated weights for policy 0, policy_version 67850 (0.0007) [2023-10-13 23:45:33,022][60934] Updated weights for policy 1, policy_version 68162 (0.0007) [2023-10-13 23:45:33,359][60935] Updated weights for policy 0, policy_version 67860 (0.0009) [2023-10-13 23:45:33,379][60934] Updated weights for policy 1, policy_version 68172 (0.0008) [2023-10-13 23:45:33,719][60935] Updated weights for policy 0, policy_version 67870 (0.0009) [2023-10-13 23:45:33,746][60934] Updated weights for policy 1, policy_version 68182 (0.0007) [2023-10-13 23:45:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 139624448. Throughput: 0: 1688.4, 1: 1691.6. Samples: 34914056. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:45:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:45:37,714][60934] Updated weights for policy 1, policy_version 68192 (0.0008) [2023-10-13 23:45:37,844][60935] Updated weights for policy 0, policy_version 67880 (0.0007) [2023-10-13 23:45:38,073][60934] Updated weights for policy 1, policy_version 68202 (0.0008) [2023-10-13 23:45:38,208][60935] Updated weights for policy 0, policy_version 67890 (0.0008) [2023-10-13 23:45:38,443][60934] Updated weights for policy 1, policy_version 68212 (0.0009) [2023-10-13 23:45:38,583][60935] Updated weights for policy 0, policy_version 67900 (0.0009) [2023-10-13 23:45:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 139689984. Throughput: 0: 1700.1, 1: 1712.4. Samples: 34935158. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:45:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:45:42,338][60934] Updated weights for policy 1, policy_version 68222 (0.0009) [2023-10-13 23:45:42,556][60935] Updated weights for policy 0, policy_version 67910 (0.0008) [2023-10-13 23:45:42,701][60934] Updated weights for policy 1, policy_version 68232 (0.0008) [2023-10-13 23:45:42,925][60935] Updated weights for policy 0, policy_version 67920 (0.0008) [2023-10-13 23:45:43,074][60934] Updated weights for policy 1, policy_version 68242 (0.0007) [2023-10-13 23:45:43,280][60935] Updated weights for policy 0, policy_version 67930 (0.0009) [2023-10-13 23:45:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 139755520. Throughput: 0: 1675.2, 1: 1685.7. Samples: 34944510. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:45:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:45:47,042][60934] Updated weights for policy 1, policy_version 68252 (0.0009) [2023-10-13 23:45:47,289][60935] Updated weights for policy 0, policy_version 67940 (0.0009) [2023-10-13 23:45:47,409][60934] Updated weights for policy 1, policy_version 68262 (0.0010) [2023-10-13 23:45:47,655][60935] Updated weights for policy 0, policy_version 67950 (0.0009) [2023-10-13 23:45:47,770][60934] Updated weights for policy 1, policy_version 68272 (0.0009) [2023-10-13 23:45:48,023][60935] Updated weights for policy 0, policy_version 67960 (0.0009) [2023-10-13 23:45:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 139821056. Throughput: 0: 1704.3, 1: 1705.6. Samples: 34965600. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:45:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:45:51,778][60934] Updated weights for policy 1, policy_version 68282 (0.0008) [2023-10-13 23:45:51,901][60935] Updated weights for policy 0, policy_version 67970 (0.0009) [2023-10-13 23:45:52,146][60934] Updated weights for policy 1, policy_version 68292 (0.0007) [2023-10-13 23:45:52,269][60935] Updated weights for policy 0, policy_version 67980 (0.0010) [2023-10-13 23:45:52,503][60934] Updated weights for policy 1, policy_version 68302 (0.0007) [2023-10-13 23:45:52,634][60935] Updated weights for policy 0, policy_version 67990 (0.0007) [2023-10-13 23:45:52,870][60934] Updated weights for policy 1, policy_version 68312 (0.0007) [2023-10-13 23:45:53,001][60935] Updated weights for policy 0, policy_version 68000 (0.0008) [2023-10-13 23:45:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 139886592. Throughput: 0: 1706.3, 1: 1716.1. Samples: 34986570. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:45:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:45:57,086][60934] Updated weights for policy 1, policy_version 68322 (0.0008) [2023-10-13 23:45:57,273][60935] Updated weights for policy 0, policy_version 68010 (0.0008) [2023-10-13 23:45:57,456][60934] Updated weights for policy 1, policy_version 68332 (0.0008) [2023-10-13 23:45:57,635][60935] Updated weights for policy 0, policy_version 68020 (0.0008) [2023-10-13 23:45:57,822][60934] Updated weights for policy 1, policy_version 68342 (0.0009) [2023-10-13 23:45:58,002][60935] Updated weights for policy 0, policy_version 68030 (0.0009) [2023-10-13 23:46:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 139952128. Throughput: 0: 1688.2, 1: 1689.6. Samples: 34995392. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:46:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:46:01,831][60935] Updated weights for policy 0, policy_version 68040 (0.0009) [2023-10-13 23:46:01,892][60934] Updated weights for policy 1, policy_version 68352 (0.0007) [2023-10-13 23:46:02,194][60935] Updated weights for policy 0, policy_version 68050 (0.0010) [2023-10-13 23:46:02,255][60934] Updated weights for policy 1, policy_version 68362 (0.0007) [2023-10-13 23:46:02,561][60935] Updated weights for policy 0, policy_version 68060 (0.0010) [2023-10-13 23:46:02,623][60934] Updated weights for policy 1, policy_version 68372 (0.0007) [2023-10-13 23:46:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 140017664. Throughput: 0: 1702.9, 1: 1708.8. Samples: 35016294. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:46:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:46:06,515][60935] Updated weights for policy 0, policy_version 68070 (0.0008) [2023-10-13 23:46:06,605][60934] Updated weights for policy 1, policy_version 68382 (0.0007) [2023-10-13 23:46:06,868][60935] Updated weights for policy 0, policy_version 68080 (0.0009) [2023-10-13 23:46:06,965][60934] Updated weights for policy 1, policy_version 68392 (0.0007) [2023-10-13 23:46:07,237][60935] Updated weights for policy 0, policy_version 68090 (0.0009) [2023-10-13 23:46:07,327][60934] Updated weights for policy 1, policy_version 68402 (0.0008) [2023-10-13 23:46:11,157][60935] Updated weights for policy 0, policy_version 68100 (0.0008) [2023-10-13 23:46:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 140083200. Throughput: 0: 1704.9, 1: 1708.5. Samples: 35037590. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-13 23:46:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:46:11,397][60934] Updated weights for policy 1, policy_version 68412 (0.0008) [2023-10-13 23:46:11,525][60935] Updated weights for policy 0, policy_version 68110 (0.0007) [2023-10-13 23:46:11,764][60934] Updated weights for policy 1, policy_version 68422 (0.0007) [2023-10-13 23:46:11,895][60935] Updated weights for policy 0, policy_version 68120 (0.0007) [2023-10-13 23:46:12,121][60934] Updated weights for policy 1, policy_version 68432 (0.0009) [2023-10-13 23:46:15,757][60935] Updated weights for policy 0, policy_version 68130 (0.0009) [2023-10-13 23:46:16,116][60935] Updated weights for policy 0, policy_version 68140 (0.0009) [2023-10-13 23:46:16,179][60934] Updated weights for policy 1, policy_version 68442 (0.0009) [2023-10-13 23:46:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 140148736. Throughput: 0: 1705.1, 1: 1697.5. Samples: 35046906. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 23:46:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:46:16,490][60935] Updated weights for policy 0, policy_version 68150 (0.0008) [2023-10-13 23:46:16,551][60934] Updated weights for policy 1, policy_version 68452 (0.0008) [2023-10-13 23:46:16,851][60935] Updated weights for policy 0, policy_version 68160 (0.0008) [2023-10-13 23:46:16,912][60934] Updated weights for policy 1, policy_version 68462 (0.0009) [2023-10-13 23:46:17,276][60934] Updated weights for policy 1, policy_version 68472 (0.0007) [2023-10-13 23:46:20,890][60935] Updated weights for policy 0, policy_version 68170 (0.0009) [2023-10-13 23:46:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 140214272. Throughput: 0: 1716.4, 1: 1710.5. Samples: 35068264. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 23:46:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:46:21,257][60935] Updated weights for policy 0, policy_version 68180 (0.0007) [2023-10-13 23:46:21,330][60934] Updated weights for policy 1, policy_version 68482 (0.0007) [2023-10-13 23:46:21,619][60935] Updated weights for policy 0, policy_version 68190 (0.0007) [2023-10-13 23:46:21,688][60934] Updated weights for policy 1, policy_version 68492 (0.0008) [2023-10-13 23:46:22,057][60934] Updated weights for policy 1, policy_version 68502 (0.0008) [2023-10-13 23:46:25,691][60935] Updated weights for policy 0, policy_version 68200 (0.0009) [2023-10-13 23:46:26,063][60934] Updated weights for policy 1, policy_version 68512 (0.0008) [2023-10-13 23:46:26,064][60935] Updated weights for policy 0, policy_version 68210 (0.0010) [2023-10-13 23:46:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 140279808. Throughput: 0: 1708.0, 1: 1705.9. Samples: 35088780. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 23:46:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:46:26,432][60934] Updated weights for policy 1, policy_version 68522 (0.0008) [2023-10-13 23:46:26,433][60935] Updated weights for policy 0, policy_version 68220 (0.0008) [2023-10-13 23:46:26,796][60934] Updated weights for policy 1, policy_version 68532 (0.0008) [2023-10-13 23:46:30,332][60935] Updated weights for policy 0, policy_version 68230 (0.0008) [2023-10-13 23:46:30,692][60935] Updated weights for policy 0, policy_version 68240 (0.0007) [2023-10-13 23:46:30,895][60934] Updated weights for policy 1, policy_version 68542 (0.0009) [2023-10-13 23:46:31,059][60935] Updated weights for policy 0, policy_version 68250 (0.0009) [2023-10-13 23:46:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 140345344. Throughput: 0: 1716.2, 1: 1700.4. Samples: 35098258. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 23:46:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:46:31,262][60934] Updated weights for policy 1, policy_version 68552 (0.0008) [2023-10-13 23:46:31,627][60934] Updated weights for policy 1, policy_version 68562 (0.0008) [2023-10-13 23:46:35,161][60935] Updated weights for policy 0, policy_version 68260 (0.0007) [2023-10-13 23:46:35,527][60935] Updated weights for policy 0, policy_version 68270 (0.0008) [2023-10-13 23:46:35,599][60934] Updated weights for policy 1, policy_version 68572 (0.0008) [2023-10-13 23:46:35,894][60935] Updated weights for policy 0, policy_version 68280 (0.0008) [2023-10-13 23:46:35,970][60934] Updated weights for policy 1, policy_version 68582 (0.0007) [2023-10-13 23:46:36,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 140443648. Throughput: 0: 1718.3, 1: 1698.6. Samples: 35119360. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 23:46:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:46:36,333][60934] Updated weights for policy 1, policy_version 68592 (0.0007) [2023-10-13 23:46:39,790][60935] Updated weights for policy 0, policy_version 68290 (0.0008) [2023-10-13 23:46:40,159][60935] Updated weights for policy 0, policy_version 68300 (0.0010) [2023-10-13 23:46:40,250][60934] Updated weights for policy 1, policy_version 68602 (0.0008) [2023-10-13 23:46:40,524][60935] Updated weights for policy 0, policy_version 68310 (0.0008) [2023-10-13 23:46:40,626][60934] Updated weights for policy 1, policy_version 68612 (0.0007) [2023-10-13 23:46:40,896][60935] Updated weights for policy 0, policy_version 68320 (0.0007) [2023-10-13 23:46:40,992][60934] Updated weights for policy 1, policy_version 68622 (0.0008) [2023-10-13 23:46:41,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 140509184. Throughput: 0: 1691.8, 1: 1693.7. Samples: 35138916. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 23:46:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:46:41,361][60934] Updated weights for policy 1, policy_version 68632 (0.0008) [2023-10-13 23:46:44,915][60935] Updated weights for policy 0, policy_version 68330 (0.0010) [2023-10-13 23:46:45,274][60935] Updated weights for policy 0, policy_version 68340 (0.0009) [2023-10-13 23:46:45,326][60934] Updated weights for policy 1, policy_version 68642 (0.0007) [2023-10-13 23:46:45,645][60935] Updated weights for policy 0, policy_version 68350 (0.0007) [2023-10-13 23:46:45,689][60934] Updated weights for policy 1, policy_version 68652 (0.0009) [2023-10-13 23:46:46,053][60934] Updated weights for policy 1, policy_version 68662 (0.0007) [2023-10-13 23:46:46,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 140607488. Throughput: 0: 1725.2, 1: 1710.3. Samples: 35149990. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 23:46:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:46:49,619][60935] Updated weights for policy 0, policy_version 68360 (0.0007) [2023-10-13 23:46:49,979][60935] Updated weights for policy 0, policy_version 68370 (0.0007) [2023-10-13 23:46:50,032][60934] Updated weights for policy 1, policy_version 68672 (0.0008) [2023-10-13 23:46:50,349][60935] Updated weights for policy 0, policy_version 68380 (0.0008) [2023-10-13 23:46:50,403][60934] Updated weights for policy 1, policy_version 68682 (0.0009) [2023-10-13 23:46:50,760][60934] Updated weights for policy 1, policy_version 68692 (0.0008) [2023-10-13 23:46:51,248][59943] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 140673024. Throughput: 0: 1711.9, 1: 1710.5. Samples: 35170300. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-13 23:46:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:46:54,292][60935] Updated weights for policy 0, policy_version 68390 (0.0008) [2023-10-13 23:46:54,654][60935] Updated weights for policy 0, policy_version 68400 (0.0008) [2023-10-13 23:46:54,754][60934] Updated weights for policy 1, policy_version 68702 (0.0007) [2023-10-13 23:46:55,023][60935] Updated weights for policy 0, policy_version 68410 (0.0007) [2023-10-13 23:46:55,113][60934] Updated weights for policy 1, policy_version 68712 (0.0007) [2023-10-13 23:46:55,476][60934] Updated weights for policy 1, policy_version 68722 (0.0009) [2023-10-13 23:46:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 140738560. Throughput: 0: 1699.2, 1: 1683.9. Samples: 35189832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:46:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:46:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000068728_70680576.pth... [2023-10-13 23:46:56,260][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000068416_70057984.pth... [2023-10-13 23:46:56,292][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000066816_68419584.pth [2023-10-13 23:46:56,300][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000067128_69042176.pth [2023-10-13 23:46:59,112][60935] Updated weights for policy 0, policy_version 68420 (0.0008) [2023-10-13 23:46:59,484][60935] Updated weights for policy 0, policy_version 68430 (0.0009) [2023-10-13 23:46:59,488][60934] Updated weights for policy 1, policy_version 68732 (0.0010) [2023-10-13 23:46:59,851][60935] Updated weights for policy 0, policy_version 68440 (0.0008) [2023-10-13 23:46:59,856][60934] Updated weights for policy 1, policy_version 68742 (0.0008) [2023-10-13 23:47:00,221][60934] Updated weights for policy 1, policy_version 68752 (0.0010) [2023-10-13 23:47:01,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 140804096. Throughput: 0: 1724.3, 1: 1708.5. Samples: 35201382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:47:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:47:03,694][60935] Updated weights for policy 0, policy_version 68450 (0.0007) [2023-10-13 23:47:04,060][60935] Updated weights for policy 0, policy_version 68460 (0.0007) [2023-10-13 23:47:04,143][60934] Updated weights for policy 1, policy_version 68762 (0.0011) [2023-10-13 23:47:04,430][60935] Updated weights for policy 0, policy_version 68470 (0.0009) [2023-10-13 23:47:04,508][60934] Updated weights for policy 1, policy_version 68772 (0.0008) [2023-10-13 23:47:04,792][60935] Updated weights for policy 0, policy_version 68480 (0.0009) [2023-10-13 23:47:04,878][60934] Updated weights for policy 1, policy_version 68782 (0.0007) [2023-10-13 23:47:05,242][60934] Updated weights for policy 1, policy_version 68792 (0.0009) [2023-10-13 23:47:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 140869632. Throughput: 0: 1692.9, 1: 1694.3. Samples: 35220690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:47:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:47:08,894][60935] Updated weights for policy 0, policy_version 68490 (0.0008) [2023-10-13 23:47:09,247][60934] Updated weights for policy 1, policy_version 68802 (0.0007) [2023-10-13 23:47:09,266][60935] Updated weights for policy 0, policy_version 68500 (0.0009) [2023-10-13 23:47:09,604][60934] Updated weights for policy 1, policy_version 68812 (0.0012) [2023-10-13 23:47:09,628][60935] Updated weights for policy 0, policy_version 68510 (0.0007) [2023-10-13 23:47:09,973][60934] Updated weights for policy 1, policy_version 68822 (0.0008) [2023-10-13 23:47:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 140935168. Throughput: 0: 1702.8, 1: 1678.4. Samples: 35240932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:47:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:47:13,579][60935] Updated weights for policy 0, policy_version 68520 (0.0009) [2023-10-13 23:47:13,936][60934] Updated weights for policy 1, policy_version 68832 (0.0008) [2023-10-13 23:47:13,944][60935] Updated weights for policy 0, policy_version 68530 (0.0009) [2023-10-13 23:47:14,297][60934] Updated weights for policy 1, policy_version 68842 (0.0007) [2023-10-13 23:47:14,316][60935] Updated weights for policy 0, policy_version 68540 (0.0009) [2023-10-13 23:47:14,666][60934] Updated weights for policy 1, policy_version 68852 (0.0008) [2023-10-13 23:47:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 141000704. Throughput: 0: 1711.8, 1: 1713.9. Samples: 35252412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:47:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:47:18,280][60935] Updated weights for policy 0, policy_version 68550 (0.0010) [2023-10-13 23:47:18,657][60935] Updated weights for policy 0, policy_version 68560 (0.0008) [2023-10-13 23:47:18,736][60934] Updated weights for policy 1, policy_version 68862 (0.0007) [2023-10-13 23:47:19,022][60935] Updated weights for policy 0, policy_version 68570 (0.0008) [2023-10-13 23:47:19,102][60934] Updated weights for policy 1, policy_version 68872 (0.0007) [2023-10-13 23:47:19,464][60934] Updated weights for policy 1, policy_version 68882 (0.0008) [2023-10-13 23:47:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 141066240. Throughput: 0: 1696.3, 1: 1692.0. Samples: 35271830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:47:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:47:22,995][60935] Updated weights for policy 0, policy_version 68580 (0.0008) [2023-10-13 23:47:23,369][60935] Updated weights for policy 0, policy_version 68590 (0.0008) [2023-10-13 23:47:23,607][60934] Updated weights for policy 1, policy_version 68892 (0.0009) [2023-10-13 23:47:23,741][60935] Updated weights for policy 0, policy_version 68600 (0.0008) [2023-10-13 23:47:23,975][60934] Updated weights for policy 1, policy_version 68902 (0.0010) [2023-10-13 23:47:24,333][60934] Updated weights for policy 1, policy_version 68912 (0.0010) [2023-10-13 23:47:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 141131776. Throughput: 0: 1719.0, 1: 1690.5. Samples: 35292344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:47:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:47:27,914][60935] Updated weights for policy 0, policy_version 68610 (0.0008) [2023-10-13 23:47:28,279][60935] Updated weights for policy 0, policy_version 68620 (0.0008) [2023-10-13 23:47:28,342][60934] Updated weights for policy 1, policy_version 68922 (0.0010) [2023-10-13 23:47:28,646][60935] Updated weights for policy 0, policy_version 68630 (0.0009) [2023-10-13 23:47:28,716][60934] Updated weights for policy 1, policy_version 68932 (0.0008) [2023-10-13 23:47:29,021][60935] Updated weights for policy 0, policy_version 68640 (0.0010) [2023-10-13 23:47:29,082][60934] Updated weights for policy 1, policy_version 68942 (0.0007) [2023-10-13 23:47:29,448][60934] Updated weights for policy 1, policy_version 68952 (0.0007) [2023-10-13 23:47:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 141197312. Throughput: 0: 1691.6, 1: 1701.5. Samples: 35302682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:47:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:47:33,149][60935] Updated weights for policy 0, policy_version 68650 (0.0010) [2023-10-13 23:47:33,414][60934] Updated weights for policy 1, policy_version 68962 (0.0007) [2023-10-13 23:47:33,512][60935] Updated weights for policy 0, policy_version 68660 (0.0007) [2023-10-13 23:47:33,780][60934] Updated weights for policy 1, policy_version 68972 (0.0007) [2023-10-13 23:47:33,881][60935] Updated weights for policy 0, policy_version 68670 (0.0008) [2023-10-13 23:47:34,152][60934] Updated weights for policy 1, policy_version 68982 (0.0008) [2023-10-13 23:47:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 141262848. Throughput: 0: 1697.3, 1: 1685.2. Samples: 35322514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:47:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:47:38,017][60935] Updated weights for policy 0, policy_version 68680 (0.0009) [2023-10-13 23:47:38,181][60934] Updated weights for policy 1, policy_version 68992 (0.0009) [2023-10-13 23:47:38,386][60935] Updated weights for policy 0, policy_version 68690 (0.0009) [2023-10-13 23:47:38,547][60934] Updated weights for policy 1, policy_version 69002 (0.0009) [2023-10-13 23:47:38,751][60935] Updated weights for policy 0, policy_version 68700 (0.0008) [2023-10-13 23:47:38,914][60934] Updated weights for policy 1, policy_version 69012 (0.0008) [2023-10-13 23:47:41,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 141328384. Throughput: 0: 1703.7, 1: 1703.8. Samples: 35343170. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-13 23:47:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:47:42,761][60935] Updated weights for policy 0, policy_version 68710 (0.0009) [2023-10-13 23:47:42,996][60934] Updated weights for policy 1, policy_version 69022 (0.0009) [2023-10-13 23:47:43,139][60935] Updated weights for policy 0, policy_version 68720 (0.0009) [2023-10-13 23:47:43,365][60934] Updated weights for policy 1, policy_version 69032 (0.0008) [2023-10-13 23:47:43,510][60935] Updated weights for policy 0, policy_version 68730 (0.0009) [2023-10-13 23:47:43,729][60934] Updated weights for policy 1, policy_version 69042 (0.0009) [2023-10-13 23:47:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 141393920. Throughput: 0: 1676.4, 1: 1689.2. Samples: 35352834. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-13 23:47:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:47:47,261][60935] Updated weights for policy 0, policy_version 68740 (0.0010) [2023-10-13 23:47:47,631][60935] Updated weights for policy 0, policy_version 68750 (0.0009) [2023-10-13 23:47:47,784][60934] Updated weights for policy 1, policy_version 69052 (0.0007) [2023-10-13 23:47:48,005][60935] Updated weights for policy 0, policy_version 68760 (0.0007) [2023-10-13 23:47:48,144][60934] Updated weights for policy 1, policy_version 69062 (0.0007) [2023-10-13 23:47:48,512][60934] Updated weights for policy 1, policy_version 69072 (0.0011) [2023-10-13 23:47:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 141459456. Throughput: 0: 1708.6, 1: 1689.3. Samples: 35373592. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-13 23:47:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:47:51,670][60935] Updated weights for policy 0, policy_version 68770 (0.0009) [2023-10-13 23:47:52,030][60935] Updated weights for policy 0, policy_version 68780 (0.0009) [2023-10-13 23:47:52,402][60935] Updated weights for policy 0, policy_version 68790 (0.0009) [2023-10-13 23:47:52,622][60934] Updated weights for policy 1, policy_version 69082 (0.0009) [2023-10-13 23:47:52,766][60935] Updated weights for policy 0, policy_version 68800 (0.0008) [2023-10-13 23:47:52,997][60934] Updated weights for policy 1, policy_version 69092 (0.0008) [2023-10-13 23:47:53,370][60934] Updated weights for policy 1, policy_version 69102 (0.0008) [2023-10-13 23:47:53,733][60934] Updated weights for policy 1, policy_version 69112 (0.0009) [2023-10-13 23:47:56,249][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 141524992. Throughput: 0: 1714.3, 1: 1706.6. Samples: 35394876. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-13 23:47:56,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:47:56,813][60935] Updated weights for policy 0, policy_version 68810 (0.0010) [2023-10-13 23:47:57,191][60935] Updated weights for policy 0, policy_version 68820 (0.0010) [2023-10-13 23:47:57,552][60935] Updated weights for policy 0, policy_version 68830 (0.0008) [2023-10-13 23:47:57,694][60934] Updated weights for policy 1, policy_version 69122 (0.0007) [2023-10-13 23:47:58,067][60934] Updated weights for policy 1, policy_version 69132 (0.0007) [2023-10-13 23:47:58,439][60934] Updated weights for policy 1, policy_version 69142 (0.0007) [2023-10-13 23:48:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 141590528. Throughput: 0: 1697.9, 1: 1675.2. Samples: 35404200. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-13 23:48:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:48:01,507][60935] Updated weights for policy 0, policy_version 68840 (0.0008) [2023-10-13 23:48:01,871][60935] Updated weights for policy 0, policy_version 68850 (0.0010) [2023-10-13 23:48:02,247][60935] Updated weights for policy 0, policy_version 68860 (0.0008) [2023-10-13 23:48:02,361][60934] Updated weights for policy 1, policy_version 69152 (0.0009) [2023-10-13 23:48:02,732][60934] Updated weights for policy 1, policy_version 69162 (0.0008) [2023-10-13 23:48:03,090][60934] Updated weights for policy 1, policy_version 69172 (0.0009) [2023-10-13 23:48:06,243][60935] Updated weights for policy 0, policy_version 68870 (0.0009) [2023-10-13 23:48:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 141656064. Throughput: 0: 1711.5, 1: 1700.3. Samples: 35425364. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-13 23:48:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:48:06,621][60935] Updated weights for policy 0, policy_version 68880 (0.0011) [2023-10-13 23:48:06,987][60935] Updated weights for policy 0, policy_version 68890 (0.0009) [2023-10-13 23:48:07,071][60934] Updated weights for policy 1, policy_version 69182 (0.0010) [2023-10-13 23:48:07,443][60934] Updated weights for policy 1, policy_version 69192 (0.0008) [2023-10-13 23:48:07,803][60934] Updated weights for policy 1, policy_version 69202 (0.0007) [2023-10-13 23:48:10,973][60935] Updated weights for policy 0, policy_version 68900 (0.0007) [2023-10-13 23:48:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 141721600. Throughput: 0: 1716.0, 1: 1710.8. Samples: 35446550. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-13 23:48:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:48:11,345][60935] Updated weights for policy 0, policy_version 68910 (0.0008) [2023-10-13 23:48:11,718][60935] Updated weights for policy 0, policy_version 68920 (0.0008) [2023-10-13 23:48:11,800][60934] Updated weights for policy 1, policy_version 69212 (0.0008) [2023-10-13 23:48:12,166][60934] Updated weights for policy 1, policy_version 69222 (0.0007) [2023-10-13 23:48:12,524][60934] Updated weights for policy 1, policy_version 69232 (0.0007) [2023-10-13 23:48:15,712][60935] Updated weights for policy 0, policy_version 68930 (0.0008) [2023-10-13 23:48:16,078][60935] Updated weights for policy 0, policy_version 68940 (0.0008) [2023-10-13 23:48:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 141787136. Throughput: 0: 1716.4, 1: 1691.1. Samples: 35456022. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-13 23:48:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:48:16,446][60935] Updated weights for policy 0, policy_version 68950 (0.0009) [2023-10-13 23:48:16,648][60934] Updated weights for policy 1, policy_version 69242 (0.0008) [2023-10-13 23:48:16,810][60935] Updated weights for policy 0, policy_version 68960 (0.0009) [2023-10-13 23:48:17,012][60934] Updated weights for policy 1, policy_version 69252 (0.0009) [2023-10-13 23:48:17,377][60934] Updated weights for policy 1, policy_version 69262 (0.0008) [2023-10-13 23:48:17,749][60934] Updated weights for policy 1, policy_version 69272 (0.0007) [2023-10-13 23:48:20,770][60935] Updated weights for policy 0, policy_version 68970 (0.0007) [2023-10-13 23:48:21,143][60935] Updated weights for policy 0, policy_version 68980 (0.0008) [2023-10-13 23:48:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 141852672. Throughput: 0: 1729.4, 1: 1706.0. Samples: 35477106. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-13 23:48:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:48:21,504][60935] Updated weights for policy 0, policy_version 68990 (0.0009) [2023-10-13 23:48:21,735][60934] Updated weights for policy 1, policy_version 69282 (0.0009) [2023-10-13 23:48:22,101][60934] Updated weights for policy 1, policy_version 69292 (0.0010) [2023-10-13 23:48:22,471][60934] Updated weights for policy 1, policy_version 69302 (0.0010) [2023-10-13 23:48:25,484][60935] Updated weights for policy 0, policy_version 69000 (0.0008) [2023-10-13 23:48:25,863][60935] Updated weights for policy 0, policy_version 69010 (0.0008) [2023-10-13 23:48:26,243][60935] Updated weights for policy 0, policy_version 69020 (0.0009) [2023-10-13 23:48:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 141918208. Throughput: 0: 1721.6, 1: 1713.2. Samples: 35497736. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-13 23:48:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:48:26,344][60934] Updated weights for policy 1, policy_version 69312 (0.0008) [2023-10-13 23:48:26,710][60934] Updated weights for policy 1, policy_version 69322 (0.0009) [2023-10-13 23:48:27,079][60934] Updated weights for policy 1, policy_version 69332 (0.0009) [2023-10-13 23:48:30,203][60935] Updated weights for policy 0, policy_version 69030 (0.0008) [2023-10-13 23:48:30,568][60935] Updated weights for policy 0, policy_version 69040 (0.0008) [2023-10-13 23:48:30,932][60935] Updated weights for policy 0, policy_version 69050 (0.0007) [2023-10-13 23:48:31,158][60934] Updated weights for policy 1, policy_version 69342 (0.0008) [2023-10-13 23:48:31,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 142016512. Throughput: 0: 1736.6, 1: 1706.4. Samples: 35507768. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-13 23:48:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:48:31,521][60934] Updated weights for policy 1, policy_version 69352 (0.0010) [2023-10-13 23:48:31,886][60934] Updated weights for policy 1, policy_version 69362 (0.0009) [2023-10-13 23:48:35,080][60935] Updated weights for policy 0, policy_version 69060 (0.0008) [2023-10-13 23:48:35,441][60935] Updated weights for policy 0, policy_version 69070 (0.0008) [2023-10-13 23:48:35,811][60935] Updated weights for policy 0, policy_version 69080 (0.0007) [2023-10-13 23:48:35,934][60934] Updated weights for policy 1, policy_version 69372 (0.0009) [2023-10-13 23:48:36,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 142082048. Throughput: 0: 1727.0, 1: 1712.7. Samples: 35528378. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-13 23:48:36,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:48:36,298][60934] Updated weights for policy 1, policy_version 69382 (0.0008) [2023-10-13 23:48:36,664][60934] Updated weights for policy 1, policy_version 69392 (0.0008) [2023-10-13 23:48:39,827][60935] Updated weights for policy 0, policy_version 69090 (0.0007) [2023-10-13 23:48:40,186][60935] Updated weights for policy 0, policy_version 69100 (0.0009) [2023-10-13 23:48:40,563][60935] Updated weights for policy 0, policy_version 69110 (0.0009) [2023-10-13 23:48:40,572][60934] Updated weights for policy 1, policy_version 69402 (0.0007) [2023-10-13 23:48:40,922][60934] Updated weights for policy 1, policy_version 69412 (0.0008) [2023-10-13 23:48:40,926][60935] Updated weights for policy 0, policy_version 69120 (0.0008) [2023-10-13 23:48:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 142147584. Throughput: 0: 1695.0, 1: 1714.0. Samples: 35548280. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-13 23:48:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:48:41,294][60934] Updated weights for policy 1, policy_version 69422 (0.0008) [2023-10-13 23:48:41,663][60934] Updated weights for policy 1, policy_version 69432 (0.0008) [2023-10-13 23:48:44,848][60935] Updated weights for policy 0, policy_version 69130 (0.0011) [2023-10-13 23:48:45,213][60935] Updated weights for policy 0, policy_version 69140 (0.0008) [2023-10-13 23:48:45,581][60935] Updated weights for policy 0, policy_version 69150 (0.0010) [2023-10-13 23:48:45,689][60934] Updated weights for policy 1, policy_version 69442 (0.0007) [2023-10-13 23:48:46,055][60934] Updated weights for policy 1, policy_version 69452 (0.0007) [2023-10-13 23:48:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 142213120. Throughput: 0: 1724.7, 1: 1711.1. Samples: 35558808. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-13 23:48:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:48:46,426][60934] Updated weights for policy 1, policy_version 69462 (0.0007) [2023-10-13 23:48:49,478][60935] Updated weights for policy 0, policy_version 69160 (0.0009) [2023-10-13 23:48:49,858][60935] Updated weights for policy 0, policy_version 69170 (0.0011) [2023-10-13 23:48:50,228][60935] Updated weights for policy 0, policy_version 69180 (0.0009) [2023-10-13 23:48:50,456][60934] Updated weights for policy 1, policy_version 69472 (0.0007) [2023-10-13 23:48:50,830][60934] Updated weights for policy 1, policy_version 69482 (0.0007) [2023-10-13 23:48:51,205][60934] Updated weights for policy 1, policy_version 69492 (0.0010) [2023-10-13 23:48:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 142278656. Throughput: 0: 1712.0, 1: 1707.3. Samples: 35579232. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-13 23:48:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:48:54,082][60935] Updated weights for policy 0, policy_version 69190 (0.0010) [2023-10-13 23:48:54,453][60935] Updated weights for policy 0, policy_version 69200 (0.0012) [2023-10-13 23:48:54,826][60935] Updated weights for policy 0, policy_version 69210 (0.0008) [2023-10-13 23:48:55,075][60934] Updated weights for policy 1, policy_version 69502 (0.0008) [2023-10-13 23:48:55,446][60934] Updated weights for policy 1, policy_version 69512 (0.0008) [2023-10-13 23:48:55,804][60934] Updated weights for policy 1, policy_version 69522 (0.0009) [2023-10-13 23:48:56,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 142376960. Throughput: 0: 1700.2, 1: 1689.6. Samples: 35599092. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-13 23:48:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:48:56,262][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000069216_70877184.pth... [2023-10-13 23:48:56,262][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000069528_71499776.pth... [2023-10-13 23:48:56,300][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000067928_69861376.pth [2023-10-13 23:48:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000067616_69238784.pth [2023-10-13 23:48:58,861][60935] Updated weights for policy 0, policy_version 69220 (0.0009) [2023-10-13 23:48:59,221][60935] Updated weights for policy 0, policy_version 69230 (0.0008) [2023-10-13 23:48:59,584][60935] Updated weights for policy 0, policy_version 69240 (0.0007) [2023-10-13 23:48:59,902][60934] Updated weights for policy 1, policy_version 69532 (0.0008) [2023-10-13 23:49:00,265][60934] Updated weights for policy 1, policy_version 69542 (0.0008) [2023-10-13 23:49:00,633][60934] Updated weights for policy 1, policy_version 69552 (0.0008) [2023-10-13 23:49:01,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 142442496. Throughput: 0: 1722.7, 1: 1702.8. Samples: 35610168. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-13 23:49:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:49:03,496][60935] Updated weights for policy 0, policy_version 69250 (0.0009) [2023-10-13 23:49:03,861][60935] Updated weights for policy 0, policy_version 69260 (0.0008) [2023-10-13 23:49:04,235][60935] Updated weights for policy 0, policy_version 69270 (0.0008) [2023-10-13 23:49:04,582][60934] Updated weights for policy 1, policy_version 69562 (0.0008) [2023-10-13 23:49:04,591][60935] Updated weights for policy 0, policy_version 69280 (0.0008) [2023-10-13 23:49:04,942][60934] Updated weights for policy 1, policy_version 69572 (0.0009) [2023-10-13 23:49:05,303][60934] Updated weights for policy 1, policy_version 69582 (0.0007) [2023-10-13 23:49:05,672][60934] Updated weights for policy 1, policy_version 69592 (0.0008) [2023-10-13 23:49:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 142508032. Throughput: 0: 1693.7, 1: 1704.0. Samples: 35630002. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-13 23:49:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:49:08,549][60935] Updated weights for policy 0, policy_version 69290 (0.0010) [2023-10-13 23:49:08,914][60935] Updated weights for policy 0, policy_version 69300 (0.0011) [2023-10-13 23:49:09,286][60935] Updated weights for policy 0, policy_version 69310 (0.0008) [2023-10-13 23:49:09,758][60934] Updated weights for policy 1, policy_version 69602 (0.0008) [2023-10-13 23:49:10,128][60934] Updated weights for policy 1, policy_version 69612 (0.0008) [2023-10-13 23:49:10,492][60934] Updated weights for policy 1, policy_version 69622 (0.0009) [2023-10-13 23:49:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 142573568. Throughput: 0: 1712.8, 1: 1669.6. Samples: 35649944. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:49:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:49:13,394][60935] Updated weights for policy 0, policy_version 69320 (0.0007) [2023-10-13 23:49:13,765][60935] Updated weights for policy 0, policy_version 69330 (0.0007) [2023-10-13 23:49:14,129][60935] Updated weights for policy 0, policy_version 69340 (0.0008) [2023-10-13 23:49:14,456][60934] Updated weights for policy 1, policy_version 69632 (0.0010) [2023-10-13 23:49:14,819][60934] Updated weights for policy 1, policy_version 69642 (0.0007) [2023-10-13 23:49:15,178][60934] Updated weights for policy 1, policy_version 69652 (0.0011) [2023-10-13 23:49:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 142639104. Throughput: 0: 1703.1, 1: 1695.5. Samples: 35660702. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:49:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:49:18,182][60935] Updated weights for policy 0, policy_version 69350 (0.0009) [2023-10-13 23:49:18,547][60935] Updated weights for policy 0, policy_version 69360 (0.0008) [2023-10-13 23:49:18,919][60935] Updated weights for policy 0, policy_version 69370 (0.0008) [2023-10-13 23:49:19,414][60934] Updated weights for policy 1, policy_version 69662 (0.0011) [2023-10-13 23:49:19,778][60934] Updated weights for policy 1, policy_version 69672 (0.0010) [2023-10-13 23:49:20,148][60934] Updated weights for policy 1, policy_version 69682 (0.0010) [2023-10-13 23:49:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 142704640. Throughput: 0: 1698.3, 1: 1690.5. Samples: 35680876. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:49:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:49:22,771][60935] Updated weights for policy 0, policy_version 69380 (0.0009) [2023-10-13 23:49:23,131][60935] Updated weights for policy 0, policy_version 69390 (0.0007) [2023-10-13 23:49:23,503][60935] Updated weights for policy 0, policy_version 69400 (0.0010) [2023-10-13 23:49:24,150][60934] Updated weights for policy 1, policy_version 69692 (0.0007) [2023-10-13 23:49:24,517][60934] Updated weights for policy 1, policy_version 69702 (0.0008) [2023-10-13 23:49:24,883][60934] Updated weights for policy 1, policy_version 69712 (0.0008) [2023-10-13 23:49:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 142770176. Throughput: 0: 1730.3, 1: 1668.7. Samples: 35701234. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:49:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:49:27,418][60935] Updated weights for policy 0, policy_version 69410 (0.0009) [2023-10-13 23:49:27,785][60935] Updated weights for policy 0, policy_version 69420 (0.0008) [2023-10-13 23:49:28,162][60935] Updated weights for policy 0, policy_version 69430 (0.0009) [2023-10-13 23:49:28,522][60935] Updated weights for policy 0, policy_version 69440 (0.0009) [2023-10-13 23:49:28,918][60934] Updated weights for policy 1, policy_version 69722 (0.0010) [2023-10-13 23:49:29,285][60934] Updated weights for policy 1, policy_version 69732 (0.0009) [2023-10-13 23:49:29,646][60934] Updated weights for policy 1, policy_version 69742 (0.0008) [2023-10-13 23:49:30,008][60934] Updated weights for policy 1, policy_version 69752 (0.0008) [2023-10-13 23:49:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 142835712. Throughput: 0: 1701.8, 1: 1700.6. Samples: 35711918. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:49:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-13 23:49:32,596][60935] Updated weights for policy 0, policy_version 69450 (0.0008) [2023-10-13 23:49:32,965][60935] Updated weights for policy 0, policy_version 69460 (0.0011) [2023-10-13 23:49:33,331][60935] Updated weights for policy 0, policy_version 69470 (0.0007) [2023-10-13 23:49:33,983][60934] Updated weights for policy 1, policy_version 69762 (0.0007) [2023-10-13 23:49:34,345][60934] Updated weights for policy 1, policy_version 69772 (0.0008) [2023-10-13 23:49:34,714][60934] Updated weights for policy 1, policy_version 69782 (0.0007) [2023-10-13 23:49:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 142901248. Throughput: 0: 1711.2, 1: 1680.2. Samples: 35731846. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:49:36,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:49:37,278][60935] Updated weights for policy 0, policy_version 69480 (0.0010) [2023-10-13 23:49:37,647][60935] Updated weights for policy 0, policy_version 69490 (0.0009) [2023-10-13 23:49:38,003][60935] Updated weights for policy 0, policy_version 69500 (0.0008) [2023-10-13 23:49:38,687][60934] Updated weights for policy 1, policy_version 69792 (0.0009) [2023-10-13 23:49:39,053][60934] Updated weights for policy 1, policy_version 69802 (0.0008) [2023-10-13 23:49:39,420][60934] Updated weights for policy 1, policy_version 69812 (0.0010) [2023-10-13 23:49:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 142966784. Throughput: 0: 1724.7, 1: 1690.4. Samples: 35752774. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:49:41,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:49:41,914][60935] Updated weights for policy 0, policy_version 69510 (0.0009) [2023-10-13 23:49:42,282][60935] Updated weights for policy 0, policy_version 69520 (0.0008) [2023-10-13 23:49:42,635][60935] Updated weights for policy 0, policy_version 69530 (0.0007) [2023-10-13 23:49:43,561][60934] Updated weights for policy 1, policy_version 69822 (0.0008) [2023-10-13 23:49:43,931][60934] Updated weights for policy 1, policy_version 69832 (0.0009) [2023-10-13 23:49:44,291][60934] Updated weights for policy 1, policy_version 69842 (0.0009) [2023-10-13 23:49:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 143032320. Throughput: 0: 1699.7, 1: 1695.9. Samples: 35762968. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:49:46,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:49:46,614][60935] Updated weights for policy 0, policy_version 69540 (0.0008) [2023-10-13 23:49:46,967][60935] Updated weights for policy 0, policy_version 69550 (0.0009) [2023-10-13 23:49:47,332][60935] Updated weights for policy 0, policy_version 69560 (0.0009) [2023-10-13 23:49:48,432][60934] Updated weights for policy 1, policy_version 69852 (0.0007) [2023-10-13 23:49:48,796][60934] Updated weights for policy 1, policy_version 69862 (0.0008) [2023-10-13 23:49:49,167][60934] Updated weights for policy 1, policy_version 69872 (0.0009) [2023-10-13 23:49:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 143097856. Throughput: 0: 1725.4, 1: 1674.3. Samples: 35782990. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:49:51,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:49:51,467][60935] Updated weights for policy 0, policy_version 69570 (0.0010) [2023-10-13 23:49:51,840][60935] Updated weights for policy 0, policy_version 69580 (0.0008) [2023-10-13 23:49:52,206][60935] Updated weights for policy 0, policy_version 69590 (0.0009) [2023-10-13 23:49:52,562][60935] Updated weights for policy 0, policy_version 69600 (0.0008) [2023-10-13 23:49:53,026][60934] Updated weights for policy 1, policy_version 69882 (0.0008) [2023-10-13 23:49:53,389][60934] Updated weights for policy 1, policy_version 69892 (0.0008) [2023-10-13 23:49:53,753][60934] Updated weights for policy 1, policy_version 69902 (0.0007) [2023-10-13 23:49:54,118][60934] Updated weights for policy 1, policy_version 69912 (0.0008) [2023-10-13 23:49:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 143163392. Throughput: 0: 1725.9, 1: 1707.2. Samples: 35804432. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-13 23:49:56,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:49:56,474][60935] Updated weights for policy 0, policy_version 69610 (0.0010) [2023-10-13 23:49:56,835][60935] Updated weights for policy 0, policy_version 69620 (0.0010) [2023-10-13 23:49:57,203][60935] Updated weights for policy 0, policy_version 69630 (0.0010) [2023-10-13 23:49:58,198][60934] Updated weights for policy 1, policy_version 69922 (0.0008) [2023-10-13 23:49:58,558][60934] Updated weights for policy 1, policy_version 69932 (0.0007) [2023-10-13 23:49:58,932][60934] Updated weights for policy 1, policy_version 69942 (0.0009) [2023-10-13 23:50:01,206][60935] Updated weights for policy 0, policy_version 69640 (0.0011) [2023-10-13 23:50:01,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 143228928. Throughput: 0: 1721.6, 1: 1689.1. Samples: 35814184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:50:01,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:50:01,579][60935] Updated weights for policy 0, policy_version 69650 (0.0008) [2023-10-13 23:50:01,944][60935] Updated weights for policy 0, policy_version 69660 (0.0010) [2023-10-13 23:50:03,002][60934] Updated weights for policy 1, policy_version 69952 (0.0008) [2023-10-13 23:50:03,370][60934] Updated weights for policy 1, policy_version 69962 (0.0010) [2023-10-13 23:50:03,743][60934] Updated weights for policy 1, policy_version 69972 (0.0009) [2023-10-13 23:50:06,047][60935] Updated weights for policy 0, policy_version 69670 (0.0008) [2023-10-13 23:50:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 143294464. Throughput: 0: 1729.6, 1: 1685.2. Samples: 35834544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:50:06,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:50:06,403][60935] Updated weights for policy 0, policy_version 69680 (0.0009) [2023-10-13 23:50:06,771][60935] Updated weights for policy 0, policy_version 69690 (0.0010) [2023-10-13 23:50:07,831][60934] Updated weights for policy 1, policy_version 69982 (0.0008) [2023-10-13 23:50:08,194][60934] Updated weights for policy 1, policy_version 69992 (0.0007) [2023-10-13 23:50:08,561][60934] Updated weights for policy 1, policy_version 70002 (0.0008) [2023-10-13 23:50:10,615][60935] Updated weights for policy 0, policy_version 69700 (0.0007) [2023-10-13 23:50:10,988][60935] Updated weights for policy 0, policy_version 69710 (0.0007) [2023-10-13 23:50:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 143360000. Throughput: 0: 1717.0, 1: 1706.3. Samples: 35855284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:50:11,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:50:11,363][60935] Updated weights for policy 0, policy_version 69720 (0.0007) [2023-10-13 23:50:12,581][60934] Updated weights for policy 1, policy_version 70012 (0.0009) [2023-10-13 23:50:12,942][60934] Updated weights for policy 1, policy_version 70022 (0.0008) [2023-10-13 23:50:13,301][60934] Updated weights for policy 1, policy_version 70032 (0.0007) [2023-10-13 23:50:15,092][60935] Updated weights for policy 0, policy_version 69730 (0.0008) [2023-10-13 23:50:15,455][60935] Updated weights for policy 0, policy_version 69740 (0.0009) [2023-10-13 23:50:15,825][60935] Updated weights for policy 0, policy_version 69750 (0.0011) [2023-10-13 23:50:16,192][60935] Updated weights for policy 0, policy_version 69760 (0.0009) [2023-10-13 23:50:16,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 143458304. Throughput: 0: 1726.9, 1: 1676.0. Samples: 35865052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:50:16,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:50:17,394][60934] Updated weights for policy 1, policy_version 70042 (0.0007) [2023-10-13 23:50:17,762][60934] Updated weights for policy 1, policy_version 70052 (0.0009) [2023-10-13 23:50:18,124][60934] Updated weights for policy 1, policy_version 70062 (0.0010) [2023-10-13 23:50:18,498][60934] Updated weights for policy 1, policy_version 70072 (0.0009) [2023-10-13 23:50:20,332][60935] Updated weights for policy 0, policy_version 69770 (0.0008) [2023-10-13 23:50:20,688][60935] Updated weights for policy 0, policy_version 69780 (0.0008) [2023-10-13 23:50:21,052][60935] Updated weights for policy 0, policy_version 69790 (0.0008) [2023-10-13 23:50:21,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 143523840. Throughput: 0: 1724.6, 1: 1693.7. Samples: 35885670. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:50:21,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:50:22,408][60934] Updated weights for policy 1, policy_version 70082 (0.0008) [2023-10-13 23:50:22,776][60934] Updated weights for policy 1, policy_version 70092 (0.0008) [2023-10-13 23:50:23,134][60934] Updated weights for policy 1, policy_version 70102 (0.0007) [2023-10-13 23:50:24,880][60935] Updated weights for policy 0, policy_version 69800 (0.0008) [2023-10-13 23:50:25,240][60935] Updated weights for policy 0, policy_version 69810 (0.0011) [2023-10-13 23:50:25,608][60935] Updated weights for policy 0, policy_version 69820 (0.0008) [2023-10-13 23:50:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 143589376. Throughput: 0: 1694.9, 1: 1701.0. Samples: 35905588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:50:26,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:50:27,274][60934] Updated weights for policy 1, policy_version 70112 (0.0007) [2023-10-13 23:50:27,645][60934] Updated weights for policy 1, policy_version 70122 (0.0009) [2023-10-13 23:50:28,019][60934] Updated weights for policy 1, policy_version 70132 (0.0009) [2023-10-13 23:50:29,682][60935] Updated weights for policy 0, policy_version 69830 (0.0007) [2023-10-13 23:50:30,052][60935] Updated weights for policy 0, policy_version 69840 (0.0009) [2023-10-13 23:50:30,413][60935] Updated weights for policy 0, policy_version 69850 (0.0009) [2023-10-13 23:50:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 143654912. Throughput: 0: 1726.4, 1: 1676.8. Samples: 35916110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:50:31,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:50:32,253][60934] Updated weights for policy 1, policy_version 70142 (0.0007) [2023-10-13 23:50:32,620][60934] Updated weights for policy 1, policy_version 70152 (0.0007) [2023-10-13 23:50:32,980][60934] Updated weights for policy 1, policy_version 70162 (0.0009) [2023-10-13 23:50:34,291][60935] Updated weights for policy 0, policy_version 69860 (0.0008) [2023-10-13 23:50:34,660][60935] Updated weights for policy 0, policy_version 69870 (0.0008) [2023-10-13 23:50:35,028][60935] Updated weights for policy 0, policy_version 69880 (0.0007) [2023-10-13 23:50:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 143720448. Throughput: 0: 1713.0, 1: 1697.4. Samples: 35936456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:50:36,249][59943] Avg episode reward: [(0, '-0.080'), (1, '0.000')] [2023-10-13 23:50:37,026][60934] Updated weights for policy 1, policy_version 70172 (0.0009) [2023-10-13 23:50:37,390][60934] Updated weights for policy 1, policy_version 70182 (0.0010) [2023-10-13 23:50:37,760][60934] Updated weights for policy 1, policy_version 70192 (0.0009) [2023-10-13 23:50:39,002][60935] Updated weights for policy 0, policy_version 69890 (0.0008) [2023-10-13 23:50:39,371][60935] Updated weights for policy 0, policy_version 69900 (0.0009) [2023-10-13 23:50:39,733][60935] Updated weights for policy 0, policy_version 69910 (0.0010) [2023-10-13 23:50:40,102][60935] Updated weights for policy 0, policy_version 69920 (0.0010) [2023-10-13 23:50:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 143785984. Throughput: 0: 1696.5, 1: 1694.9. Samples: 35957048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:50:41,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:50:41,727][60934] Updated weights for policy 1, policy_version 70202 (0.0009) [2023-10-13 23:50:42,088][60934] Updated weights for policy 1, policy_version 70212 (0.0009) [2023-10-13 23:50:42,458][60934] Updated weights for policy 1, policy_version 70222 (0.0009) [2023-10-13 23:50:42,816][60934] Updated weights for policy 1, policy_version 70232 (0.0008) [2023-10-13 23:50:44,206][60935] Updated weights for policy 0, policy_version 69930 (0.0008) [2023-10-13 23:50:44,583][60935] Updated weights for policy 0, policy_version 69940 (0.0008) [2023-10-13 23:50:44,955][60935] Updated weights for policy 0, policy_version 69950 (0.0008) [2023-10-13 23:50:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 143851520. Throughput: 0: 1721.9, 1: 1686.8. Samples: 35967576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-13 23:50:46,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:50:46,999][60934] Updated weights for policy 1, policy_version 70242 (0.0009) [2023-10-13 23:50:47,376][60934] Updated weights for policy 1, policy_version 70252 (0.0007) [2023-10-13 23:50:47,736][60934] Updated weights for policy 1, policy_version 70262 (0.0008) [2023-10-13 23:50:48,989][60935] Updated weights for policy 0, policy_version 69960 (0.0011) [2023-10-13 23:50:49,366][60935] Updated weights for policy 0, policy_version 69970 (0.0007) [2023-10-13 23:50:49,739][60935] Updated weights for policy 0, policy_version 69980 (0.0008) [2023-10-13 23:50:51,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 143917056. Throughput: 0: 1697.6, 1: 1694.3. Samples: 35987180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-13 23:50:51,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:50:51,619][60934] Updated weights for policy 1, policy_version 70272 (0.0009) [2023-10-13 23:50:51,988][60934] Updated weights for policy 1, policy_version 70282 (0.0008) [2023-10-13 23:50:52,355][60934] Updated weights for policy 1, policy_version 70292 (0.0007) [2023-10-13 23:50:53,691][60935] Updated weights for policy 0, policy_version 69990 (0.0009) [2023-10-13 23:50:54,061][60935] Updated weights for policy 0, policy_version 70000 (0.0010) [2023-10-13 23:50:54,431][60935] Updated weights for policy 0, policy_version 70010 (0.0011) [2023-10-13 23:50:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 143982592. Throughput: 0: 1703.8, 1: 1702.5. Samples: 36008570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-13 23:50:56,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:50:56,256][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000070016_71696384.pth... [2023-10-13 23:50:56,290][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000068416_70057984.pth [2023-10-13 23:50:56,350][60934] Updated weights for policy 1, policy_version 70302 (0.0008) [2023-10-13 23:50:56,713][60934] Updated weights for policy 1, policy_version 70312 (0.0009) [2023-10-13 23:50:57,083][60934] Updated weights for policy 1, policy_version 70322 (0.0009) [2023-10-13 23:50:57,292][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000070328_72318976.pth... [2023-10-13 23:50:57,321][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000068728_70680576.pth [2023-10-13 23:50:58,318][60935] Updated weights for policy 0, policy_version 70020 (0.0009) [2023-10-13 23:50:58,692][60935] Updated weights for policy 0, policy_version 70030 (0.0007) [2023-10-13 23:50:59,060][60935] Updated weights for policy 0, policy_version 70040 (0.0007) [2023-10-13 23:51:01,016][60934] Updated weights for policy 1, policy_version 70332 (0.0008) [2023-10-13 23:51:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 144048128. Throughput: 0: 1705.2, 1: 1703.0. Samples: 36018420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-13 23:51:01,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:51:01,385][60934] Updated weights for policy 1, policy_version 70342 (0.0009) [2023-10-13 23:51:01,751][60934] Updated weights for policy 1, policy_version 70352 (0.0008) [2023-10-13 23:51:03,070][60935] Updated weights for policy 0, policy_version 70050 (0.0009) [2023-10-13 23:51:03,438][60935] Updated weights for policy 0, policy_version 70060 (0.0010) [2023-10-13 23:51:03,813][60935] Updated weights for policy 0, policy_version 70070 (0.0011) [2023-10-13 23:51:04,172][60935] Updated weights for policy 0, policy_version 70080 (0.0008) [2023-10-13 23:51:05,665][60934] Updated weights for policy 1, policy_version 70362 (0.0007) [2023-10-13 23:51:06,034][60934] Updated weights for policy 1, policy_version 70372 (0.0007) [2023-10-13 23:51:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 144113664. Throughput: 0: 1696.8, 1: 1714.4. Samples: 36039172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-13 23:51:06,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:51:06,396][60934] Updated weights for policy 1, policy_version 70382 (0.0007) [2023-10-13 23:51:06,764][60934] Updated weights for policy 1, policy_version 70392 (0.0007) [2023-10-13 23:51:08,253][60935] Updated weights for policy 0, policy_version 70090 (0.0010) [2023-10-13 23:51:08,629][60935] Updated weights for policy 0, policy_version 70100 (0.0010) [2023-10-13 23:51:09,003][60935] Updated weights for policy 0, policy_version 70110 (0.0010) [2023-10-13 23:51:10,866][60934] Updated weights for policy 1, policy_version 70402 (0.0008) [2023-10-13 23:51:11,231][60934] Updated weights for policy 1, policy_version 70412 (0.0008) [2023-10-13 23:51:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 144179200. Throughput: 0: 1724.4, 1: 1710.0. Samples: 36060136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-13 23:51:11,249][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:51:11,593][60934] Updated weights for policy 1, policy_version 70422 (0.0007) [2023-10-13 23:51:12,772][60935] Updated weights for policy 0, policy_version 70120 (0.0009) [2023-10-13 23:51:13,136][60935] Updated weights for policy 0, policy_version 70130 (0.0007) [2023-10-13 23:51:13,507][60935] Updated weights for policy 0, policy_version 70140 (0.0012) [2023-10-13 23:51:15,520][60934] Updated weights for policy 1, policy_version 70432 (0.0010) [2023-10-13 23:51:15,886][60934] Updated weights for policy 1, policy_version 70442 (0.0008) [2023-10-13 23:51:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13662.6). Total num frames: 144244736. Throughput: 0: 1697.7, 1: 1713.2. Samples: 36069598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-13 23:51:16,248][59943] Avg episode reward: [(0, '-0.090'), (1, '0.000')] [2023-10-13 23:51:16,256][60934] Updated weights for policy 1, policy_version 70452 (0.0009) [2023-10-13 23:51:17,425][60935] Updated weights for policy 0, policy_version 70150 (0.0011) [2023-10-13 23:51:17,780][60935] Updated weights for policy 0, policy_version 70160 (0.0008) [2023-10-13 23:51:18,153][60935] Updated weights for policy 0, policy_version 70170 (0.0007) [2023-10-13 23:51:20,226][60934] Updated weights for policy 1, policy_version 70462 (0.0009) [2023-10-13 23:51:20,587][60934] Updated weights for policy 1, policy_version 70472 (0.0007) [2023-10-13 23:51:20,952][60934] Updated weights for policy 1, policy_version 70482 (0.0009) [2023-10-13 23:51:21,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 144343040. Throughput: 0: 1711.6, 1: 1718.2. Samples: 36090794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-13 23:51:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:51:22,221][60935] Updated weights for policy 0, policy_version 70180 (0.0009) [2023-10-13 23:51:22,598][60935] Updated weights for policy 0, policy_version 70190 (0.0008) [2023-10-13 23:51:22,968][60935] Updated weights for policy 0, policy_version 70200 (0.0009) [2023-10-13 23:51:24,922][60934] Updated weights for policy 1, policy_version 70492 (0.0008) [2023-10-13 23:51:25,292][60934] Updated weights for policy 1, policy_version 70502 (0.0009) [2023-10-13 23:51:25,663][60934] Updated weights for policy 1, policy_version 70512 (0.0009) [2023-10-13 23:51:26,248][59943] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 144408576. Throughput: 0: 1732.0, 1: 1698.6. Samples: 36111426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-13 23:51:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:51:26,781][60935] Updated weights for policy 0, policy_version 70210 (0.0010) [2023-10-13 23:51:27,150][60935] Updated weights for policy 0, policy_version 70220 (0.0008) [2023-10-13 23:51:27,508][60935] Updated weights for policy 0, policy_version 70230 (0.0008) [2023-10-13 23:51:27,877][60935] Updated weights for policy 0, policy_version 70240 (0.0008) [2023-10-13 23:51:29,752][60934] Updated weights for policy 1, policy_version 70522 (0.0009) [2023-10-13 23:51:30,125][60934] Updated weights for policy 1, policy_version 70532 (0.0007) [2023-10-13 23:51:30,498][60934] Updated weights for policy 1, policy_version 70542 (0.0008) [2023-10-13 23:51:30,859][60934] Updated weights for policy 1, policy_version 70552 (0.0007) [2023-10-13 23:51:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 144474112. Throughput: 0: 1704.9, 1: 1713.2. Samples: 36121390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-13 23:51:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:51:31,855][60935] Updated weights for policy 0, policy_version 70250 (0.0008) [2023-10-13 23:51:32,214][60935] Updated weights for policy 0, policy_version 70260 (0.0010) [2023-10-13 23:51:32,586][60935] Updated weights for policy 0, policy_version 70270 (0.0011) [2023-10-13 23:51:35,070][60934] Updated weights for policy 1, policy_version 70562 (0.0007) [2023-10-13 23:51:35,430][60934] Updated weights for policy 1, policy_version 70572 (0.0007) [2023-10-13 23:51:35,795][60934] Updated weights for policy 1, policy_version 70582 (0.0007) [2023-10-13 23:51:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 144539648. Throughput: 0: 1728.9, 1: 1716.6. Samples: 36142230. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-13 23:51:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:51:36,720][60935] Updated weights for policy 0, policy_version 70280 (0.0009) [2023-10-13 23:51:37,100][60935] Updated weights for policy 0, policy_version 70290 (0.0008) [2023-10-13 23:51:37,471][60935] Updated weights for policy 0, policy_version 70300 (0.0008) [2023-10-13 23:51:39,672][60934] Updated weights for policy 1, policy_version 70592 (0.0007) [2023-10-13 23:51:40,030][60934] Updated weights for policy 1, policy_version 70602 (0.0008) [2023-10-13 23:51:40,397][60934] Updated weights for policy 1, policy_version 70612 (0.0009) [2023-10-13 23:51:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 144605184. Throughput: 0: 1730.3, 1: 1677.8. Samples: 36161932. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-13 23:51:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:51:41,568][60935] Updated weights for policy 0, policy_version 70310 (0.0008) [2023-10-13 23:51:41,942][60935] Updated weights for policy 0, policy_version 70320 (0.0009) [2023-10-13 23:51:42,308][60935] Updated weights for policy 0, policy_version 70330 (0.0009) [2023-10-13 23:51:44,360][60934] Updated weights for policy 1, policy_version 70622 (0.0007) [2023-10-13 23:51:44,734][60934] Updated weights for policy 1, policy_version 70632 (0.0007) [2023-10-13 23:51:45,093][60934] Updated weights for policy 1, policy_version 70642 (0.0008) [2023-10-13 23:51:46,181][60935] Updated weights for policy 0, policy_version 70340 (0.0008) [2023-10-13 23:51:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 144670720. Throughput: 0: 1720.1, 1: 1708.2. Samples: 36172696. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-13 23:51:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:51:46,545][60935] Updated weights for policy 0, policy_version 70350 (0.0007) [2023-10-13 23:51:46,914][60935] Updated weights for policy 0, policy_version 70360 (0.0008) [2023-10-13 23:51:49,084][60934] Updated weights for policy 1, policy_version 70652 (0.0009) [2023-10-13 23:51:49,450][60934] Updated weights for policy 1, policy_version 70662 (0.0008) [2023-10-13 23:51:49,820][60934] Updated weights for policy 1, policy_version 70672 (0.0007) [2023-10-13 23:51:50,929][60935] Updated weights for policy 0, policy_version 70370 (0.0008) [2023-10-13 23:51:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 144736256. Throughput: 0: 1731.4, 1: 1686.0. Samples: 36192956. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-13 23:51:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:51:51,309][60935] Updated weights for policy 0, policy_version 70380 (0.0008) [2023-10-13 23:51:51,678][60935] Updated weights for policy 0, policy_version 70390 (0.0008) [2023-10-13 23:51:52,055][60935] Updated weights for policy 0, policy_version 70400 (0.0009) [2023-10-13 23:51:53,770][60934] Updated weights for policy 1, policy_version 70682 (0.0008) [2023-10-13 23:51:54,141][60934] Updated weights for policy 1, policy_version 70692 (0.0009) [2023-10-13 23:51:54,497][60934] Updated weights for policy 1, policy_version 70702 (0.0007) [2023-10-13 23:51:54,863][60934] Updated weights for policy 1, policy_version 70712 (0.0009) [2023-10-13 23:51:55,750][60935] Updated weights for policy 0, policy_version 70410 (0.0011) [2023-10-13 23:51:56,111][60935] Updated weights for policy 0, policy_version 70420 (0.0010) [2023-10-13 23:51:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 144801792. Throughput: 0: 1721.8, 1: 1675.0. Samples: 36212994. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-13 23:51:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:51:56,482][60935] Updated weights for policy 0, policy_version 70430 (0.0010) [2023-10-13 23:51:58,920][60934] Updated weights for policy 1, policy_version 70722 (0.0007) [2023-10-13 23:51:59,282][60934] Updated weights for policy 1, policy_version 70732 (0.0009) [2023-10-13 23:51:59,656][60934] Updated weights for policy 1, policy_version 70742 (0.0009) [2023-10-13 23:52:00,436][60935] Updated weights for policy 0, policy_version 70440 (0.0008) [2023-10-13 23:52:00,817][60935] Updated weights for policy 0, policy_version 70450 (0.0009) [2023-10-13 23:52:01,180][60935] Updated weights for policy 0, policy_version 70460 (0.0008) [2023-10-13 23:52:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 144867328. Throughput: 0: 1726.5, 1: 1707.7. Samples: 36224138. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-13 23:52:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:52:03,755][60934] Updated weights for policy 1, policy_version 70752 (0.0008) [2023-10-13 23:52:04,126][60934] Updated weights for policy 1, policy_version 70762 (0.0007) [2023-10-13 23:52:04,487][60934] Updated weights for policy 1, policy_version 70772 (0.0010) [2023-10-13 23:52:05,198][60935] Updated weights for policy 0, policy_version 70470 (0.0009) [2023-10-13 23:52:05,572][60935] Updated weights for policy 0, policy_version 70480 (0.0007) [2023-10-13 23:52:05,934][60935] Updated weights for policy 0, policy_version 70490 (0.0007) [2023-10-13 23:52:06,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 144965632. Throughput: 0: 1729.7, 1: 1682.0. Samples: 36244324. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-13 23:52:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.040')] [2023-10-13 23:52:08,525][60934] Updated weights for policy 1, policy_version 70782 (0.0007) [2023-10-13 23:52:08,899][60934] Updated weights for policy 1, policy_version 70792 (0.0008) [2023-10-13 23:52:09,264][60934] Updated weights for policy 1, policy_version 70802 (0.0009) [2023-10-13 23:52:09,954][60935] Updated weights for policy 0, policy_version 70500 (0.0007) [2023-10-13 23:52:10,325][60935] Updated weights for policy 0, policy_version 70510 (0.0009) [2023-10-13 23:52:10,688][60935] Updated weights for policy 0, policy_version 70520 (0.0009) [2023-10-13 23:52:11,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 145031168. Throughput: 0: 1694.8, 1: 1695.1. Samples: 36263974. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-13 23:52:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.040')] [2023-10-13 23:52:13,319][60934] Updated weights for policy 1, policy_version 70812 (0.0008) [2023-10-13 23:52:13,684][60934] Updated weights for policy 1, policy_version 70822 (0.0008) [2023-10-13 23:52:14,054][60934] Updated weights for policy 1, policy_version 70832 (0.0009) [2023-10-13 23:52:14,561][60935] Updated weights for policy 0, policy_version 70530 (0.0009) [2023-10-13 23:52:14,924][60935] Updated weights for policy 0, policy_version 70540 (0.0011) [2023-10-13 23:52:15,297][60935] Updated weights for policy 0, policy_version 70550 (0.0010) [2023-10-13 23:52:15,655][60935] Updated weights for policy 0, policy_version 70560 (0.0010) [2023-10-13 23:52:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 145096704. Throughput: 0: 1722.4, 1: 1697.9. Samples: 36275304. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-13 23:52:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.040')] [2023-10-13 23:52:18,038][60934] Updated weights for policy 1, policy_version 70842 (0.0008) [2023-10-13 23:52:18,393][60934] Updated weights for policy 1, policy_version 70852 (0.0008) [2023-10-13 23:52:18,756][60934] Updated weights for policy 1, policy_version 70862 (0.0009) [2023-10-13 23:52:19,130][60934] Updated weights for policy 1, policy_version 70872 (0.0008) [2023-10-13 23:52:19,586][60935] Updated weights for policy 0, policy_version 70570 (0.0007) [2023-10-13 23:52:19,953][60935] Updated weights for policy 0, policy_version 70580 (0.0010) [2023-10-13 23:52:20,336][60935] Updated weights for policy 0, policy_version 70590 (0.0007) [2023-10-13 23:52:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 145162240. Throughput: 0: 1716.9, 1: 1678.2. Samples: 36295010. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 23:52:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.040')] [2023-10-13 23:52:23,143][60934] Updated weights for policy 1, policy_version 70882 (0.0007) [2023-10-13 23:52:23,517][60934] Updated weights for policy 1, policy_version 70892 (0.0008) [2023-10-13 23:52:23,878][60934] Updated weights for policy 1, policy_version 70902 (0.0010) [2023-10-13 23:52:24,293][60935] Updated weights for policy 0, policy_version 70600 (0.0009) [2023-10-13 23:52:24,659][60935] Updated weights for policy 0, policy_version 70610 (0.0008) [2023-10-13 23:52:25,037][60935] Updated weights for policy 0, policy_version 70620 (0.0009) [2023-10-13 23:52:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 145227776. Throughput: 0: 1704.1, 1: 1714.4. Samples: 36315766. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 23:52:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:52:27,921][60934] Updated weights for policy 1, policy_version 70912 (0.0010) [2023-10-13 23:52:28,287][60934] Updated weights for policy 1, policy_version 70922 (0.0010) [2023-10-13 23:52:28,658][60934] Updated weights for policy 1, policy_version 70932 (0.0007) [2023-10-13 23:52:28,949][60935] Updated weights for policy 0, policy_version 70630 (0.0009) [2023-10-13 23:52:29,317][60935] Updated weights for policy 0, policy_version 70640 (0.0007) [2023-10-13 23:52:29,688][60935] Updated weights for policy 0, policy_version 70650 (0.0007) [2023-10-13 23:52:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 145293312. Throughput: 0: 1726.6, 1: 1687.7. Samples: 36326340. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 23:52:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:52:32,744][60934] Updated weights for policy 1, policy_version 70942 (0.0008) [2023-10-13 23:52:33,102][60934] Updated weights for policy 1, policy_version 70952 (0.0008) [2023-10-13 23:52:33,472][60934] Updated weights for policy 1, policy_version 70962 (0.0011) [2023-10-13 23:52:33,702][60935] Updated weights for policy 0, policy_version 70660 (0.0008) [2023-10-13 23:52:34,081][60935] Updated weights for policy 0, policy_version 70670 (0.0007) [2023-10-13 23:52:34,449][60935] Updated weights for policy 0, policy_version 70680 (0.0007) [2023-10-13 23:52:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 145358848. Throughput: 0: 1700.7, 1: 1688.8. Samples: 36345482. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 23:52:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:52:37,576][60934] Updated weights for policy 1, policy_version 70972 (0.0008) [2023-10-13 23:52:37,948][60934] Updated weights for policy 1, policy_version 70982 (0.0009) [2023-10-13 23:52:38,313][60934] Updated weights for policy 1, policy_version 70992 (0.0010) [2023-10-13 23:52:38,328][60935] Updated weights for policy 0, policy_version 70690 (0.0009) [2023-10-13 23:52:38,696][60935] Updated weights for policy 0, policy_version 70700 (0.0009) [2023-10-13 23:52:39,076][60935] Updated weights for policy 0, policy_version 70710 (0.0009) [2023-10-13 23:52:39,445][60935] Updated weights for policy 0, policy_version 70720 (0.0010) [2023-10-13 23:52:41,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 145424384. Throughput: 0: 1711.4, 1: 1706.8. Samples: 36366810. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 23:52:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:52:42,100][60934] Updated weights for policy 1, policy_version 71002 (0.0009) [2023-10-13 23:52:42,464][60934] Updated weights for policy 1, policy_version 71012 (0.0009) [2023-10-13 23:52:42,833][60934] Updated weights for policy 1, policy_version 71022 (0.0009) [2023-10-13 23:52:43,204][60934] Updated weights for policy 1, policy_version 71032 (0.0009) [2023-10-13 23:52:43,553][60935] Updated weights for policy 0, policy_version 70730 (0.0007) [2023-10-13 23:52:43,917][60935] Updated weights for policy 0, policy_version 70740 (0.0009) [2023-10-13 23:52:44,291][60935] Updated weights for policy 0, policy_version 70750 (0.0010) [2023-10-13 23:52:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 145489920. Throughput: 0: 1715.2, 1: 1676.2. Samples: 36376752. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 23:52:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:52:47,243][60934] Updated weights for policy 1, policy_version 71042 (0.0008) [2023-10-13 23:52:47,606][60934] Updated weights for policy 1, policy_version 71052 (0.0008) [2023-10-13 23:52:47,960][60934] Updated weights for policy 1, policy_version 71062 (0.0012) [2023-10-13 23:52:48,264][60935] Updated weights for policy 0, policy_version 70760 (0.0010) [2023-10-13 23:52:48,634][60935] Updated weights for policy 0, policy_version 70770 (0.0007) [2023-10-13 23:52:49,000][60935] Updated weights for policy 0, policy_version 70780 (0.0007) [2023-10-13 23:52:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 145555456. Throughput: 0: 1700.0, 1: 1700.6. Samples: 36397354. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 23:52:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:52:52,006][60934] Updated weights for policy 1, policy_version 71072 (0.0009) [2023-10-13 23:52:52,370][60934] Updated weights for policy 1, policy_version 71082 (0.0008) [2023-10-13 23:52:52,748][60934] Updated weights for policy 1, policy_version 71092 (0.0007) [2023-10-13 23:52:52,845][60935] Updated weights for policy 0, policy_version 70790 (0.0008) [2023-10-13 23:52:53,214][60935] Updated weights for policy 0, policy_version 70800 (0.0007) [2023-10-13 23:52:53,576][60935] Updated weights for policy 0, policy_version 70810 (0.0009) [2023-10-13 23:52:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 145620992. Throughput: 0: 1728.6, 1: 1705.2. Samples: 36418494. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 23:52:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:52:56,259][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000071096_73105408.pth... [2023-10-13 23:52:56,259][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000070816_72515584.pth... [2023-10-13 23:52:56,296][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000069528_71499776.pth [2023-10-13 23:52:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000069216_70877184.pth [2023-10-13 23:52:56,302][60828] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p1/milestones/checkpoint_000071096_73105408.pth [2023-10-13 23:52:56,305][60695] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p0/milestones/checkpoint_000070816_72515584.pth [2023-10-13 23:52:56,772][60934] Updated weights for policy 1, policy_version 71102 (0.0008) [2023-10-13 23:52:57,139][60934] Updated weights for policy 1, policy_version 71112 (0.0010) [2023-10-13 23:52:57,508][60934] Updated weights for policy 1, policy_version 71122 (0.0009) [2023-10-13 23:52:57,610][60935] Updated weights for policy 0, policy_version 70820 (0.0009) [2023-10-13 23:52:57,984][60935] Updated weights for policy 0, policy_version 70830 (0.0009) [2023-10-13 23:52:58,357][60935] Updated weights for policy 0, policy_version 70840 (0.0011) [2023-10-13 23:53:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 145686528. Throughput: 0: 1699.1, 1: 1686.0. Samples: 36427634. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 23:53:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:53:01,616][60934] Updated weights for policy 1, policy_version 71132 (0.0009) [2023-10-13 23:53:01,975][60934] Updated weights for policy 1, policy_version 71142 (0.0010) [2023-10-13 23:53:02,279][60935] Updated weights for policy 0, policy_version 70850 (0.0007) [2023-10-13 23:53:02,340][60934] Updated weights for policy 1, policy_version 71152 (0.0007) [2023-10-13 23:53:02,652][60935] Updated weights for policy 0, policy_version 70860 (0.0008) [2023-10-13 23:53:03,021][60935] Updated weights for policy 0, policy_version 70870 (0.0008) [2023-10-13 23:53:03,391][60935] Updated weights for policy 0, policy_version 70880 (0.0009) [2023-10-13 23:53:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 145752064. Throughput: 0: 1710.7, 1: 1704.0. Samples: 36448674. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-13 23:53:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:53:06,363][60934] Updated weights for policy 1, policy_version 71162 (0.0007) [2023-10-13 23:53:06,723][60934] Updated weights for policy 1, policy_version 71172 (0.0008) [2023-10-13 23:53:07,091][60934] Updated weights for policy 1, policy_version 71182 (0.0008) [2023-10-13 23:53:07,450][60934] Updated weights for policy 1, policy_version 71192 (0.0007) [2023-10-13 23:53:07,468][60935] Updated weights for policy 0, policy_version 70890 (0.0007) [2023-10-13 23:53:07,842][60935] Updated weights for policy 0, policy_version 70900 (0.0008) [2023-10-13 23:53:08,216][60935] Updated weights for policy 0, policy_version 70910 (0.0007) [2023-10-13 23:53:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 145817600. Throughput: 0: 1727.7, 1: 1697.7. Samples: 36469910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:53:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:53:11,528][60934] Updated weights for policy 1, policy_version 71202 (0.0008) [2023-10-13 23:53:11,896][60934] Updated weights for policy 1, policy_version 71212 (0.0010) [2023-10-13 23:53:12,149][60935] Updated weights for policy 0, policy_version 70920 (0.0010) [2023-10-13 23:53:12,264][60934] Updated weights for policy 1, policy_version 71222 (0.0008) [2023-10-13 23:53:12,520][60935] Updated weights for policy 0, policy_version 70930 (0.0010) [2023-10-13 23:53:12,893][60935] Updated weights for policy 0, policy_version 70940 (0.0009) [2023-10-13 23:53:16,245][60934] Updated weights for policy 1, policy_version 71232 (0.0007) [2023-10-13 23:53:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 145883136. Throughput: 0: 1699.5, 1: 1692.2. Samples: 36478968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:53:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:53:16,612][60934] Updated weights for policy 1, policy_version 71242 (0.0007) [2023-10-13 23:53:16,973][60934] Updated weights for policy 1, policy_version 71252 (0.0010) [2023-10-13 23:53:17,003][60935] Updated weights for policy 0, policy_version 70950 (0.0010) [2023-10-13 23:53:17,365][60935] Updated weights for policy 0, policy_version 70960 (0.0010) [2023-10-13 23:53:17,738][60935] Updated weights for policy 0, policy_version 70970 (0.0008) [2023-10-13 23:53:20,914][60934] Updated weights for policy 1, policy_version 71262 (0.0008) [2023-10-13 23:53:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 145948672. Throughput: 0: 1725.2, 1: 1711.4. Samples: 36500130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:53:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.010')] [2023-10-13 23:53:21,279][60934] Updated weights for policy 1, policy_version 71272 (0.0009) [2023-10-13 23:53:21,529][60935] Updated weights for policy 0, policy_version 70980 (0.0008) [2023-10-13 23:53:21,646][60934] Updated weights for policy 1, policy_version 71282 (0.0008) [2023-10-13 23:53:21,892][60935] Updated weights for policy 0, policy_version 70990 (0.0008) [2023-10-13 23:53:22,257][60935] Updated weights for policy 0, policy_version 71000 (0.0007) [2023-10-13 23:53:25,758][60934] Updated weights for policy 1, policy_version 71292 (0.0008) [2023-10-13 23:53:26,068][60935] Updated weights for policy 0, policy_version 71010 (0.0008) [2023-10-13 23:53:26,124][60934] Updated weights for policy 1, policy_version 71302 (0.0008) [2023-10-13 23:53:26,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 146014208. Throughput: 0: 1733.2, 1: 1703.7. Samples: 36521470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:53:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.010')] [2023-10-13 23:53:26,428][60935] Updated weights for policy 0, policy_version 71020 (0.0007) [2023-10-13 23:53:26,482][60934] Updated weights for policy 1, policy_version 71312 (0.0007) [2023-10-13 23:53:26,803][60935] Updated weights for policy 0, policy_version 71030 (0.0007) [2023-10-13 23:53:27,167][60935] Updated weights for policy 0, policy_version 71040 (0.0007) [2023-10-13 23:53:30,312][60934] Updated weights for policy 1, policy_version 71322 (0.0008) [2023-10-13 23:53:30,674][60934] Updated weights for policy 1, policy_version 71332 (0.0010) [2023-10-13 23:53:30,950][60935] Updated weights for policy 0, policy_version 71050 (0.0009) [2023-10-13 23:53:31,049][60934] Updated weights for policy 1, policy_version 71342 (0.0009) [2023-10-13 23:53:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 146079744. Throughput: 0: 1721.8, 1: 1703.5. Samples: 36530890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:53:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.010')] [2023-10-13 23:53:31,313][60935] Updated weights for policy 0, policy_version 71060 (0.0008) [2023-10-13 23:53:31,412][60934] Updated weights for policy 1, policy_version 71352 (0.0007) [2023-10-13 23:53:31,682][60935] Updated weights for policy 0, policy_version 71070 (0.0007) [2023-10-13 23:53:35,603][60934] Updated weights for policy 1, policy_version 71362 (0.0007) [2023-10-13 23:53:35,627][60935] Updated weights for policy 0, policy_version 71080 (0.0008) [2023-10-13 23:53:35,966][60934] Updated weights for policy 1, policy_version 71372 (0.0007) [2023-10-13 23:53:35,988][60935] Updated weights for policy 0, policy_version 71090 (0.0007) [2023-10-13 23:53:36,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13551.5). Total num frames: 146145280. Throughput: 0: 1731.5, 1: 1707.5. Samples: 36552112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:53:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.010')] [2023-10-13 23:53:36,338][60934] Updated weights for policy 1, policy_version 71382 (0.0008) [2023-10-13 23:53:36,347][60935] Updated weights for policy 0, policy_version 71100 (0.0009) [2023-10-13 23:53:40,278][60934] Updated weights for policy 1, policy_version 71392 (0.0007) [2023-10-13 23:53:40,555][60935] Updated weights for policy 0, policy_version 71110 (0.0010) [2023-10-13 23:53:40,650][60934] Updated weights for policy 1, policy_version 71402 (0.0008) [2023-10-13 23:53:40,922][60935] Updated weights for policy 0, policy_version 71120 (0.0009) [2023-10-13 23:53:41,016][60934] Updated weights for policy 1, policy_version 71412 (0.0008) [2023-10-13 23:53:41,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 146243584. Throughput: 0: 1714.6, 1: 1693.3. Samples: 36571852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:53:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:53:41,291][60935] Updated weights for policy 0, policy_version 71130 (0.0010) [2023-10-13 23:53:44,960][60934] Updated weights for policy 1, policy_version 71422 (0.0008) [2023-10-13 23:53:45,318][60935] Updated weights for policy 0, policy_version 71140 (0.0009) [2023-10-13 23:53:45,328][60934] Updated weights for policy 1, policy_version 71432 (0.0008) [2023-10-13 23:53:45,687][60935] Updated weights for policy 0, policy_version 71150 (0.0008) [2023-10-13 23:53:45,698][60934] Updated weights for policy 1, policy_version 71442 (0.0008) [2023-10-13 23:53:46,043][60935] Updated weights for policy 0, policy_version 71160 (0.0009) [2023-10-13 23:53:46,248][59943] Fps is (10 sec: 16384.5, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 146309120. Throughput: 0: 1729.3, 1: 1709.5. Samples: 36582382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:53:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:53:49,844][60934] Updated weights for policy 1, policy_version 71452 (0.0008) [2023-10-13 23:53:50,055][60935] Updated weights for policy 0, policy_version 71170 (0.0010) [2023-10-13 23:53:50,219][60934] Updated weights for policy 1, policy_version 71462 (0.0007) [2023-10-13 23:53:50,424][60935] Updated weights for policy 0, policy_version 71180 (0.0008) [2023-10-13 23:53:50,576][60934] Updated weights for policy 1, policy_version 71472 (0.0008) [2023-10-13 23:53:50,787][60935] Updated weights for policy 0, policy_version 71190 (0.0011) [2023-10-13 23:53:51,163][60935] Updated weights for policy 0, policy_version 71200 (0.0007) [2023-10-13 23:53:51,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 146407424. Throughput: 0: 1728.2, 1: 1711.7. Samples: 36603470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:53:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:53:54,666][60934] Updated weights for policy 1, policy_version 71482 (0.0007) [2023-10-13 23:53:55,040][60934] Updated weights for policy 1, policy_version 71492 (0.0007) [2023-10-13 23:53:55,078][60935] Updated weights for policy 0, policy_version 71210 (0.0007) [2023-10-13 23:53:55,407][60934] Updated weights for policy 1, policy_version 71502 (0.0009) [2023-10-13 23:53:55,451][60935] Updated weights for policy 0, policy_version 71220 (0.0008) [2023-10-13 23:53:55,773][60934] Updated weights for policy 1, policy_version 71512 (0.0009) [2023-10-13 23:53:55,821][60935] Updated weights for policy 0, policy_version 71230 (0.0009) [2023-10-13 23:53:56,248][59943] Fps is (10 sec: 16383.6, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 146472960. Throughput: 0: 1699.2, 1: 1689.7. Samples: 36622412. Policy #0 lag: (min: 26.0, avg: 30.2, max: 58.0) [2023-10-13 23:53:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:53:59,871][60935] Updated weights for policy 0, policy_version 71240 (0.0009) [2023-10-13 23:53:59,905][60934] Updated weights for policy 1, policy_version 71522 (0.0007) [2023-10-13 23:54:00,235][60935] Updated weights for policy 0, policy_version 71250 (0.0008) [2023-10-13 23:54:00,264][60934] Updated weights for policy 1, policy_version 71532 (0.0008) [2023-10-13 23:54:00,608][60935] Updated weights for policy 0, policy_version 71260 (0.0008) [2023-10-13 23:54:00,627][60934] Updated weights for policy 1, policy_version 71542 (0.0007) [2023-10-13 23:54:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 146538496. Throughput: 0: 1729.4, 1: 1710.1. Samples: 36633742. Policy #0 lag: (min: 26.0, avg: 30.2, max: 58.0) [2023-10-13 23:54:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:54:04,535][60934] Updated weights for policy 1, policy_version 71552 (0.0009) [2023-10-13 23:54:04,560][60935] Updated weights for policy 0, policy_version 71270 (0.0010) [2023-10-13 23:54:04,895][60934] Updated weights for policy 1, policy_version 71562 (0.0008) [2023-10-13 23:54:04,916][60935] Updated weights for policy 0, policy_version 71280 (0.0009) [2023-10-13 23:54:05,257][60934] Updated weights for policy 1, policy_version 71572 (0.0008) [2023-10-13 23:54:05,280][60935] Updated weights for policy 0, policy_version 71290 (0.0007) [2023-10-13 23:54:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 146604032. Throughput: 0: 1717.9, 1: 1692.7. Samples: 36653608. Policy #0 lag: (min: 26.0, avg: 30.2, max: 58.0) [2023-10-13 23:54:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:54:09,224][60935] Updated weights for policy 0, policy_version 71300 (0.0007) [2023-10-13 23:54:09,300][60934] Updated weights for policy 1, policy_version 71582 (0.0009) [2023-10-13 23:54:09,598][60935] Updated weights for policy 0, policy_version 71310 (0.0009) [2023-10-13 23:54:09,663][60934] Updated weights for policy 1, policy_version 71592 (0.0008) [2023-10-13 23:54:09,968][60935] Updated weights for policy 0, policy_version 71320 (0.0007) [2023-10-13 23:54:10,018][60934] Updated weights for policy 1, policy_version 71602 (0.0008) [2023-10-13 23:54:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 146669568. Throughput: 0: 1693.9, 1: 1674.8. Samples: 36673062. Policy #0 lag: (min: 26.0, avg: 30.2, max: 58.0) [2023-10-13 23:54:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:54:13,887][60935] Updated weights for policy 0, policy_version 71330 (0.0009) [2023-10-13 23:54:14,146][60934] Updated weights for policy 1, policy_version 71612 (0.0007) [2023-10-13 23:54:14,250][60935] Updated weights for policy 0, policy_version 71340 (0.0007) [2023-10-13 23:54:14,507][60934] Updated weights for policy 1, policy_version 71622 (0.0008) [2023-10-13 23:54:14,609][60935] Updated weights for policy 0, policy_version 71350 (0.0008) [2023-10-13 23:54:14,873][60934] Updated weights for policy 1, policy_version 71632 (0.0010) [2023-10-13 23:54:14,968][60935] Updated weights for policy 0, policy_version 71360 (0.0007) [2023-10-13 23:54:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 146735104. Throughput: 0: 1721.9, 1: 1698.0. Samples: 36684784. Policy #0 lag: (min: 26.0, avg: 30.2, max: 58.0) [2023-10-13 23:54:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:54:18,892][60934] Updated weights for policy 1, policy_version 71642 (0.0009) [2023-10-13 23:54:18,952][60935] Updated weights for policy 0, policy_version 71370 (0.0007) [2023-10-13 23:54:19,252][60934] Updated weights for policy 1, policy_version 71652 (0.0007) [2023-10-13 23:54:19,316][60935] Updated weights for policy 0, policy_version 71380 (0.0007) [2023-10-13 23:54:19,625][60934] Updated weights for policy 1, policy_version 71662 (0.0007) [2023-10-13 23:54:19,677][60935] Updated weights for policy 0, policy_version 71390 (0.0008) [2023-10-13 23:54:19,994][60934] Updated weights for policy 1, policy_version 71672 (0.0007) [2023-10-13 23:54:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 146800640. Throughput: 0: 1696.6, 1: 1674.0. Samples: 36703788. Policy #0 lag: (min: 26.0, avg: 30.2, max: 58.0) [2023-10-13 23:54:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:54:23,448][60935] Updated weights for policy 0, policy_version 71400 (0.0009) [2023-10-13 23:54:23,831][60935] Updated weights for policy 0, policy_version 71410 (0.0009) [2023-10-13 23:54:24,194][60935] Updated weights for policy 0, policy_version 71420 (0.0009) [2023-10-13 23:54:24,239][60934] Updated weights for policy 1, policy_version 71682 (0.0007) [2023-10-13 23:54:24,600][60934] Updated weights for policy 1, policy_version 71692 (0.0008) [2023-10-13 23:54:24,967][60934] Updated weights for policy 1, policy_version 71702 (0.0009) [2023-10-13 23:54:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 146866176. Throughput: 0: 1715.7, 1: 1671.1. Samples: 36724260. Policy #0 lag: (min: 26.0, avg: 30.2, max: 58.0) [2023-10-13 23:54:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:54:28,293][60935] Updated weights for policy 0, policy_version 71430 (0.0008) [2023-10-13 23:54:28,667][60935] Updated weights for policy 0, policy_version 71440 (0.0009) [2023-10-13 23:54:29,032][60935] Updated weights for policy 0, policy_version 71450 (0.0009) [2023-10-13 23:54:29,073][60934] Updated weights for policy 1, policy_version 71712 (0.0008) [2023-10-13 23:54:29,448][60934] Updated weights for policy 1, policy_version 71722 (0.0009) [2023-10-13 23:54:29,820][60934] Updated weights for policy 1, policy_version 71732 (0.0009) [2023-10-13 23:54:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 146931712. Throughput: 0: 1714.0, 1: 1681.3. Samples: 36735170. Policy #0 lag: (min: 26.0, avg: 30.2, max: 58.0) [2023-10-13 23:54:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:54:32,948][60935] Updated weights for policy 0, policy_version 71460 (0.0008) [2023-10-13 23:54:33,320][60935] Updated weights for policy 0, policy_version 71470 (0.0008) [2023-10-13 23:54:33,676][60935] Updated weights for policy 0, policy_version 71480 (0.0008) [2023-10-13 23:54:33,969][60934] Updated weights for policy 1, policy_version 71742 (0.0009) [2023-10-13 23:54:34,336][60934] Updated weights for policy 1, policy_version 71752 (0.0008) [2023-10-13 23:54:34,714][60934] Updated weights for policy 1, policy_version 71762 (0.0008) [2023-10-13 23:54:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 146997248. Throughput: 0: 1703.9, 1: 1663.6. Samples: 36755010. Policy #0 lag: (min: 26.0, avg: 30.2, max: 58.0) [2023-10-13 23:54:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:54:37,723][60935] Updated weights for policy 0, policy_version 71490 (0.0008) [2023-10-13 23:54:38,092][60935] Updated weights for policy 0, policy_version 71500 (0.0009) [2023-10-13 23:54:38,464][60935] Updated weights for policy 0, policy_version 71510 (0.0011) [2023-10-13 23:54:38,741][60934] Updated weights for policy 1, policy_version 71772 (0.0007) [2023-10-13 23:54:38,833][60935] Updated weights for policy 0, policy_version 71520 (0.0008) [2023-10-13 23:54:39,110][60934] Updated weights for policy 1, policy_version 71782 (0.0010) [2023-10-13 23:54:39,473][60934] Updated weights for policy 1, policy_version 71792 (0.0010) [2023-10-13 23:54:41,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 147062784. Throughput: 0: 1728.9, 1: 1675.4. Samples: 36775606. Policy #0 lag: (min: 26.0, avg: 30.2, max: 58.0) [2023-10-13 23:54:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:54:42,891][60935] Updated weights for policy 0, policy_version 71530 (0.0008) [2023-10-13 23:54:43,259][60935] Updated weights for policy 0, policy_version 71540 (0.0009) [2023-10-13 23:54:43,622][60934] Updated weights for policy 1, policy_version 71802 (0.0008) [2023-10-13 23:54:43,627][60935] Updated weights for policy 0, policy_version 71550 (0.0010) [2023-10-13 23:54:44,030][60934] Updated weights for policy 1, policy_version 71812 (0.0010) [2023-10-13 23:54:44,399][60934] Updated weights for policy 1, policy_version 71822 (0.0007) [2023-10-13 23:54:44,768][60934] Updated weights for policy 1, policy_version 71832 (0.0008) [2023-10-13 23:54:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 147128320. Throughput: 0: 1700.4, 1: 1682.4. Samples: 36785970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:54:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:54:47,742][60935] Updated weights for policy 0, policy_version 71560 (0.0009) [2023-10-13 23:54:48,124][60935] Updated weights for policy 0, policy_version 71570 (0.0008) [2023-10-13 23:54:48,440][60934] Updated weights for policy 1, policy_version 71842 (0.0009) [2023-10-13 23:54:48,485][60935] Updated weights for policy 0, policy_version 71580 (0.0008) [2023-10-13 23:54:48,808][60934] Updated weights for policy 1, policy_version 71852 (0.0007) [2023-10-13 23:54:49,183][60934] Updated weights for policy 1, policy_version 71862 (0.0008) [2023-10-13 23:54:51,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 147193856. Throughput: 0: 1715.9, 1: 1665.6. Samples: 36805772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:54:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:54:52,250][60935] Updated weights for policy 0, policy_version 71590 (0.0008) [2023-10-13 23:54:52,620][60935] Updated weights for policy 0, policy_version 71600 (0.0007) [2023-10-13 23:54:52,984][60935] Updated weights for policy 0, policy_version 71610 (0.0007) [2023-10-13 23:54:53,253][60934] Updated weights for policy 1, policy_version 71872 (0.0009) [2023-10-13 23:54:53,622][60934] Updated weights for policy 1, policy_version 71882 (0.0007) [2023-10-13 23:54:53,985][60934] Updated weights for policy 1, policy_version 71892 (0.0007) [2023-10-13 23:54:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 147259392. Throughput: 0: 1727.2, 1: 1691.6. Samples: 36826910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:54:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:54:56,263][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000071616_73334784.pth... [2023-10-13 23:54:56,263][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000071896_73924608.pth... [2023-10-13 23:54:56,293][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000070016_71696384.pth [2023-10-13 23:54:56,298][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000070328_72318976.pth [2023-10-13 23:54:57,115][60935] Updated weights for policy 0, policy_version 71620 (0.0008) [2023-10-13 23:54:57,485][60935] Updated weights for policy 0, policy_version 71630 (0.0010) [2023-10-13 23:54:57,841][60935] Updated weights for policy 0, policy_version 71640 (0.0007) [2023-10-13 23:54:58,031][60934] Updated weights for policy 1, policy_version 71902 (0.0007) [2023-10-13 23:54:58,402][60934] Updated weights for policy 1, policy_version 71912 (0.0009) [2023-10-13 23:54:58,777][60934] Updated weights for policy 1, policy_version 71922 (0.0008) [2023-10-13 23:55:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 147324928. Throughput: 0: 1696.7, 1: 1675.6. Samples: 36836534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:55:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:55:01,826][60935] Updated weights for policy 0, policy_version 71650 (0.0007) [2023-10-13 23:55:02,199][60935] Updated weights for policy 0, policy_version 71660 (0.0008) [2023-10-13 23:55:02,561][60935] Updated weights for policy 0, policy_version 71670 (0.0007) [2023-10-13 23:55:02,634][60934] Updated weights for policy 1, policy_version 71932 (0.0007) [2023-10-13 23:55:02,930][60935] Updated weights for policy 0, policy_version 71680 (0.0008) [2023-10-13 23:55:03,008][60934] Updated weights for policy 1, policy_version 71942 (0.0010) [2023-10-13 23:55:03,374][60934] Updated weights for policy 1, policy_version 71952 (0.0009) [2023-10-13 23:55:06,249][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13662.6). Total num frames: 147390464. Throughput: 0: 1722.0, 1: 1689.4. Samples: 36857302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:55:06,250][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:55:06,877][60935] Updated weights for policy 0, policy_version 71690 (0.0010) [2023-10-13 23:55:07,250][60935] Updated weights for policy 0, policy_version 71700 (0.0008) [2023-10-13 23:55:07,522][60934] Updated weights for policy 1, policy_version 71962 (0.0008) [2023-10-13 23:55:07,612][60935] Updated weights for policy 0, policy_version 71710 (0.0007) [2023-10-13 23:55:07,884][60934] Updated weights for policy 1, policy_version 71972 (0.0009) [2023-10-13 23:55:08,247][60934] Updated weights for policy 1, policy_version 71982 (0.0007) [2023-10-13 23:55:08,614][60934] Updated weights for policy 1, policy_version 71992 (0.0007) [2023-10-13 23:55:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 147456000. Throughput: 0: 1723.3, 1: 1702.7. Samples: 36878428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:55:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-1.160')] [2023-10-13 23:55:11,492][60935] Updated weights for policy 0, policy_version 71720 (0.0010) [2023-10-13 23:55:11,868][60935] Updated weights for policy 0, policy_version 71730 (0.0009) [2023-10-13 23:55:12,226][60935] Updated weights for policy 0, policy_version 71740 (0.0008) [2023-10-13 23:55:12,642][60934] Updated weights for policy 1, policy_version 72002 (0.0007) [2023-10-13 23:55:13,006][60934] Updated weights for policy 1, policy_version 72012 (0.0008) [2023-10-13 23:55:13,369][60934] Updated weights for policy 1, policy_version 72022 (0.0007) [2023-10-13 23:55:16,248][59943] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 147521536. Throughput: 0: 1709.7, 1: 1673.8. Samples: 36887428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:55:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-1.160')] [2023-10-13 23:55:16,446][60935] Updated weights for policy 0, policy_version 71750 (0.0010) [2023-10-13 23:55:16,808][60935] Updated weights for policy 0, policy_version 71760 (0.0008) [2023-10-13 23:55:17,176][60935] Updated weights for policy 0, policy_version 71770 (0.0008) [2023-10-13 23:55:17,198][60934] Updated weights for policy 1, policy_version 72032 (0.0009) [2023-10-13 23:55:17,576][60934] Updated weights for policy 1, policy_version 72042 (0.0009) [2023-10-13 23:55:17,946][60934] Updated weights for policy 1, policy_version 72052 (0.0009) [2023-10-13 23:55:21,066][60935] Updated weights for policy 0, policy_version 71780 (0.0008) [2023-10-13 23:55:21,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 147587072. Throughput: 0: 1716.7, 1: 1697.8. Samples: 36908662. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:55:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-1.160')] [2023-10-13 23:55:21,437][60935] Updated weights for policy 0, policy_version 71790 (0.0009) [2023-10-13 23:55:21,801][60935] Updated weights for policy 0, policy_version 71800 (0.0010) [2023-10-13 23:55:21,935][60934] Updated weights for policy 1, policy_version 72062 (0.0010) [2023-10-13 23:55:22,301][60934] Updated weights for policy 1, policy_version 72072 (0.0008) [2023-10-13 23:55:22,668][60934] Updated weights for policy 1, policy_version 72082 (0.0008) [2023-10-13 23:55:25,825][60935] Updated weights for policy 0, policy_version 71810 (0.0009) [2023-10-13 23:55:26,197][60935] Updated weights for policy 0, policy_version 71820 (0.0008) [2023-10-13 23:55:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 147652608. Throughput: 0: 1713.9, 1: 1702.1. Samples: 36929326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:55:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-1.160')] [2023-10-13 23:55:26,561][60935] Updated weights for policy 0, policy_version 71830 (0.0007) [2023-10-13 23:55:26,854][60934] Updated weights for policy 1, policy_version 72092 (0.0009) [2023-10-13 23:55:26,918][60935] Updated weights for policy 0, policy_version 71840 (0.0008) [2023-10-13 23:55:27,218][60934] Updated weights for policy 1, policy_version 72102 (0.0009) [2023-10-13 23:55:27,588][60934] Updated weights for policy 1, policy_version 72112 (0.0009) [2023-10-13 23:55:31,002][60935] Updated weights for policy 0, policy_version 71850 (0.0012) [2023-10-13 23:55:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 147718144. Throughput: 0: 1717.8, 1: 1674.0. Samples: 36938604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:55:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:55:31,385][60935] Updated weights for policy 0, policy_version 71860 (0.0011) [2023-10-13 23:55:31,650][60934] Updated weights for policy 1, policy_version 72122 (0.0007) [2023-10-13 23:55:31,748][60935] Updated weights for policy 0, policy_version 71870 (0.0009) [2023-10-13 23:55:32,013][60934] Updated weights for policy 1, policy_version 72132 (0.0008) [2023-10-13 23:55:32,386][60934] Updated weights for policy 1, policy_version 72142 (0.0010) [2023-10-13 23:55:32,754][60934] Updated weights for policy 1, policy_version 72152 (0.0009) [2023-10-13 23:55:35,788][60935] Updated weights for policy 0, policy_version 71880 (0.0008) [2023-10-13 23:55:36,160][60935] Updated weights for policy 0, policy_version 71890 (0.0010) [2023-10-13 23:55:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 147783680. Throughput: 0: 1714.7, 1: 1700.3. Samples: 36959446. Policy #0 lag: (min: 26.0, avg: 29.2, max: 56.0) [2023-10-13 23:55:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:55:36,524][60935] Updated weights for policy 0, policy_version 71900 (0.0007) [2023-10-13 23:55:36,627][60934] Updated weights for policy 1, policy_version 72162 (0.0008) [2023-10-13 23:55:37,003][60934] Updated weights for policy 1, policy_version 72172 (0.0008) [2023-10-13 23:55:37,360][60934] Updated weights for policy 1, policy_version 72182 (0.0008) [2023-10-13 23:55:40,438][60935] Updated weights for policy 0, policy_version 71910 (0.0009) [2023-10-13 23:55:40,797][60935] Updated weights for policy 0, policy_version 71920 (0.0007) [2023-10-13 23:55:41,172][60935] Updated weights for policy 0, policy_version 71930 (0.0009) [2023-10-13 23:55:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 147849216. Throughput: 0: 1703.8, 1: 1692.6. Samples: 36979748. Policy #0 lag: (min: 26.0, avg: 29.2, max: 56.0) [2023-10-13 23:55:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:55:41,563][60934] Updated weights for policy 1, policy_version 72192 (0.0008) [2023-10-13 23:55:41,924][60934] Updated weights for policy 1, policy_version 72202 (0.0009) [2023-10-13 23:55:42,286][60934] Updated weights for policy 1, policy_version 72212 (0.0008) [2023-10-13 23:55:45,052][60935] Updated weights for policy 0, policy_version 71940 (0.0008) [2023-10-13 23:55:45,420][60935] Updated weights for policy 0, policy_version 71950 (0.0009) [2023-10-13 23:55:45,795][60935] Updated weights for policy 0, policy_version 71960 (0.0008) [2023-10-13 23:55:46,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 147947520. Throughput: 0: 1723.7, 1: 1684.4. Samples: 36989900. Policy #0 lag: (min: 26.0, avg: 29.2, max: 56.0) [2023-10-13 23:55:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:55:46,521][60934] Updated weights for policy 1, policy_version 72222 (0.0009) [2023-10-13 23:55:46,882][60934] Updated weights for policy 1, policy_version 72232 (0.0009) [2023-10-13 23:55:47,245][60934] Updated weights for policy 1, policy_version 72242 (0.0010) [2023-10-13 23:55:49,651][60935] Updated weights for policy 0, policy_version 71970 (0.0008) [2023-10-13 23:55:50,019][60935] Updated weights for policy 0, policy_version 71980 (0.0008) [2023-10-13 23:55:50,389][60935] Updated weights for policy 0, policy_version 71990 (0.0009) [2023-10-13 23:55:50,750][60935] Updated weights for policy 0, policy_version 72000 (0.0009) [2023-10-13 23:55:51,248][59943] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 148013056. Throughput: 0: 1719.5, 1: 1686.9. Samples: 37010588. Policy #0 lag: (min: 26.0, avg: 29.2, max: 56.0) [2023-10-13 23:55:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:55:51,410][60934] Updated weights for policy 1, policy_version 72252 (0.0011) [2023-10-13 23:55:51,785][60934] Updated weights for policy 1, policy_version 72262 (0.0008) [2023-10-13 23:55:52,149][60934] Updated weights for policy 1, policy_version 72272 (0.0008) [2023-10-13 23:55:54,669][60935] Updated weights for policy 0, policy_version 72010 (0.0007) [2023-10-13 23:55:55,035][60935] Updated weights for policy 0, policy_version 72020 (0.0007) [2023-10-13 23:55:55,397][60935] Updated weights for policy 0, policy_version 72030 (0.0007) [2023-10-13 23:55:56,248][60934] Updated weights for policy 1, policy_version 72282 (0.0007) [2023-10-13 23:55:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 148078592. Throughput: 0: 1692.1, 1: 1687.5. Samples: 37030510. Policy #0 lag: (min: 26.0, avg: 29.2, max: 56.0) [2023-10-13 23:55:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:55:56,611][60934] Updated weights for policy 1, policy_version 72292 (0.0007) [2023-10-13 23:55:56,983][60934] Updated weights for policy 1, policy_version 72302 (0.0009) [2023-10-13 23:55:57,344][60934] Updated weights for policy 1, policy_version 72312 (0.0007) [2023-10-13 23:55:59,433][60935] Updated weights for policy 0, policy_version 72040 (0.0011) [2023-10-13 23:55:59,795][60935] Updated weights for policy 0, policy_version 72050 (0.0009) [2023-10-13 23:56:00,160][60935] Updated weights for policy 0, policy_version 72060 (0.0010) [2023-10-13 23:56:01,225][60934] Updated weights for policy 1, policy_version 72322 (0.0008) [2023-10-13 23:56:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 148144128. Throughput: 0: 1725.3, 1: 1692.3. Samples: 37041220. Policy #0 lag: (min: 26.0, avg: 29.2, max: 56.0) [2023-10-13 23:56:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:56:01,600][60934] Updated weights for policy 1, policy_version 72332 (0.0009) [2023-10-13 23:56:01,968][60934] Updated weights for policy 1, policy_version 72342 (0.0010) [2023-10-13 23:56:04,060][60935] Updated weights for policy 0, policy_version 72070 (0.0010) [2023-10-13 23:56:04,413][60935] Updated weights for policy 0, policy_version 72080 (0.0010) [2023-10-13 23:56:04,791][60935] Updated weights for policy 0, policy_version 72090 (0.0008) [2023-10-13 23:56:06,059][60934] Updated weights for policy 1, policy_version 72352 (0.0008) [2023-10-13 23:56:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 148209664. Throughput: 0: 1702.7, 1: 1690.3. Samples: 37061344. Policy #0 lag: (min: 26.0, avg: 29.2, max: 56.0) [2023-10-13 23:56:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:56:06,424][60934] Updated weights for policy 1, policy_version 72362 (0.0007) [2023-10-13 23:56:06,797][60934] Updated weights for policy 1, policy_version 72372 (0.0007) [2023-10-13 23:56:08,749][60935] Updated weights for policy 0, policy_version 72100 (0.0009) [2023-10-13 23:56:09,113][60935] Updated weights for policy 0, policy_version 72110 (0.0009) [2023-10-13 23:56:09,481][60935] Updated weights for policy 0, policy_version 72120 (0.0009) [2023-10-13 23:56:10,591][60934] Updated weights for policy 1, policy_version 72382 (0.0009) [2023-10-13 23:56:10,941][60934] Updated weights for policy 1, policy_version 72392 (0.0008) [2023-10-13 23:56:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 148275200. Throughput: 0: 1704.3, 1: 1692.7. Samples: 37082190. Policy #0 lag: (min: 26.0, avg: 29.2, max: 56.0) [2023-10-13 23:56:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:56:11,311][60934] Updated weights for policy 1, policy_version 72402 (0.0007) [2023-10-13 23:56:13,559][60935] Updated weights for policy 0, policy_version 72130 (0.0008) [2023-10-13 23:56:13,937][60935] Updated weights for policy 0, policy_version 72140 (0.0007) [2023-10-13 23:56:14,305][60935] Updated weights for policy 0, policy_version 72150 (0.0009) [2023-10-13 23:56:14,670][60935] Updated weights for policy 0, policy_version 72160 (0.0007) [2023-10-13 23:56:15,383][60934] Updated weights for policy 1, policy_version 72412 (0.0007) [2023-10-13 23:56:15,742][60934] Updated weights for policy 1, policy_version 72422 (0.0007) [2023-10-13 23:56:16,096][60934] Updated weights for policy 1, policy_version 72432 (0.0007) [2023-10-13 23:56:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 148340736. Throughput: 0: 1722.0, 1: 1699.6. Samples: 37092574. Policy #0 lag: (min: 26.0, avg: 29.2, max: 56.0) [2023-10-13 23:56:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:56:18,590][60935] Updated weights for policy 0, policy_version 72170 (0.0009) [2023-10-13 23:56:18,962][60935] Updated weights for policy 0, policy_version 72180 (0.0009) [2023-10-13 23:56:19,334][60935] Updated weights for policy 0, policy_version 72190 (0.0009) [2023-10-13 23:56:20,192][60934] Updated weights for policy 1, policy_version 72442 (0.0008) [2023-10-13 23:56:20,603][60934] Updated weights for policy 1, policy_version 72452 (0.0009) [2023-10-13 23:56:20,970][60934] Updated weights for policy 1, policy_version 72462 (0.0007) [2023-10-13 23:56:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 148406272. Throughput: 0: 1702.0, 1: 1707.9. Samples: 37112890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:56:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:56:21,330][60934] Updated weights for policy 1, policy_version 72472 (0.0009) [2023-10-13 23:56:23,415][60935] Updated weights for policy 0, policy_version 72200 (0.0008) [2023-10-13 23:56:23,794][60935] Updated weights for policy 0, policy_version 72210 (0.0007) [2023-10-13 23:56:24,171][60935] Updated weights for policy 0, policy_version 72220 (0.0007) [2023-10-13 23:56:25,420][60934] Updated weights for policy 1, policy_version 72482 (0.0009) [2023-10-13 23:56:25,782][60934] Updated weights for policy 1, policy_version 72492 (0.0007) [2023-10-13 23:56:26,151][60934] Updated weights for policy 1, policy_version 72502 (0.0008) [2023-10-13 23:56:26,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 148504576. Throughput: 0: 1710.1, 1: 1694.6. Samples: 37132958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:56:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:56:27,890][60935] Updated weights for policy 0, policy_version 72230 (0.0010) [2023-10-13 23:56:28,260][60935] Updated weights for policy 0, policy_version 72240 (0.0010) [2023-10-13 23:56:28,630][60935] Updated weights for policy 0, policy_version 72250 (0.0011) [2023-10-13 23:56:30,095][60934] Updated weights for policy 1, policy_version 72512 (0.0009) [2023-10-13 23:56:30,463][60934] Updated weights for policy 1, policy_version 72522 (0.0010) [2023-10-13 23:56:30,835][60934] Updated weights for policy 1, policy_version 72532 (0.0009) [2023-10-13 23:56:31,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 148570112. Throughput: 0: 1696.1, 1: 1703.3. Samples: 37142872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:56:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:56:32,667][60935] Updated weights for policy 0, policy_version 72260 (0.0009) [2023-10-13 23:56:33,029][60935] Updated weights for policy 0, policy_version 72270 (0.0008) [2023-10-13 23:56:33,406][60935] Updated weights for policy 0, policy_version 72280 (0.0009) [2023-10-13 23:56:34,788][60934] Updated weights for policy 1, policy_version 72542 (0.0008) [2023-10-13 23:56:35,147][60934] Updated weights for policy 1, policy_version 72552 (0.0008) [2023-10-13 23:56:35,512][60934] Updated weights for policy 1, policy_version 72562 (0.0007) [2023-10-13 23:56:36,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 148635648. Throughput: 0: 1698.3, 1: 1706.5. Samples: 37163806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:56:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:56:37,251][60935] Updated weights for policy 0, policy_version 72290 (0.0010) [2023-10-13 23:56:37,617][60935] Updated weights for policy 0, policy_version 72300 (0.0010) [2023-10-13 23:56:37,983][60935] Updated weights for policy 0, policy_version 72310 (0.0010) [2023-10-13 23:56:38,349][60935] Updated weights for policy 0, policy_version 72320 (0.0011) [2023-10-13 23:56:39,585][60934] Updated weights for policy 1, policy_version 72572 (0.0007) [2023-10-13 23:56:39,949][60934] Updated weights for policy 1, policy_version 72582 (0.0009) [2023-10-13 23:56:40,304][60934] Updated weights for policy 1, policy_version 72592 (0.0008) [2023-10-13 23:56:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 148701184. Throughput: 0: 1721.7, 1: 1685.0. Samples: 37183810. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:56:41,250][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:56:42,611][60935] Updated weights for policy 0, policy_version 72330 (0.0008) [2023-10-13 23:56:42,968][60935] Updated weights for policy 0, policy_version 72340 (0.0008) [2023-10-13 23:56:43,338][60935] Updated weights for policy 0, policy_version 72350 (0.0009) [2023-10-13 23:56:44,273][60934] Updated weights for policy 1, policy_version 72602 (0.0011) [2023-10-13 23:56:44,649][60934] Updated weights for policy 1, policy_version 72612 (0.0009) [2023-10-13 23:56:45,009][60934] Updated weights for policy 1, policy_version 72622 (0.0010) [2023-10-13 23:56:45,381][60934] Updated weights for policy 1, policy_version 72632 (0.0009) [2023-10-13 23:56:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 148766720. Throughput: 0: 1687.1, 1: 1711.3. Samples: 37194150. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:56:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:56:47,258][60935] Updated weights for policy 0, policy_version 72360 (0.0007) [2023-10-13 23:56:47,617][60935] Updated weights for policy 0, policy_version 72370 (0.0008) [2023-10-13 23:56:47,989][60935] Updated weights for policy 0, policy_version 72380 (0.0009) [2023-10-13 23:56:49,519][60934] Updated weights for policy 1, policy_version 72642 (0.0007) [2023-10-13 23:56:49,884][60934] Updated weights for policy 1, policy_version 72652 (0.0007) [2023-10-13 23:56:50,258][60934] Updated weights for policy 1, policy_version 72662 (0.0008) [2023-10-13 23:56:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 148832256. Throughput: 0: 1710.1, 1: 1690.3. Samples: 37214360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:56:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:56:51,967][60935] Updated weights for policy 0, policy_version 72390 (0.0009) [2023-10-13 23:56:52,338][60935] Updated weights for policy 0, policy_version 72400 (0.0007) [2023-10-13 23:56:52,705][60935] Updated weights for policy 0, policy_version 72410 (0.0007) [2023-10-13 23:56:54,287][60934] Updated weights for policy 1, policy_version 72672 (0.0008) [2023-10-13 23:56:54,650][60934] Updated weights for policy 1, policy_version 72682 (0.0008) [2023-10-13 23:56:55,013][60934] Updated weights for policy 1, policy_version 72692 (0.0008) [2023-10-13 23:56:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 148897792. Throughput: 0: 1712.1, 1: 1675.7. Samples: 37234642. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:56:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:56:56,262][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000072416_74153984.pth... [2023-10-13 23:56:56,262][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000072696_74743808.pth... [2023-10-13 23:56:56,298][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000070816_72515584.pth [2023-10-13 23:56:56,301][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000071096_73105408.pth [2023-10-13 23:56:56,808][60935] Updated weights for policy 0, policy_version 72420 (0.0010) [2023-10-13 23:56:57,180][60935] Updated weights for policy 0, policy_version 72430 (0.0010) [2023-10-13 23:56:57,544][60935] Updated weights for policy 0, policy_version 72440 (0.0008) [2023-10-13 23:56:59,111][60934] Updated weights for policy 1, policy_version 72702 (0.0009) [2023-10-13 23:56:59,481][60934] Updated weights for policy 1, policy_version 72712 (0.0011) [2023-10-13 23:56:59,861][60934] Updated weights for policy 1, policy_version 72722 (0.0010) [2023-10-13 23:57:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 148963328. Throughput: 0: 1691.1, 1: 1699.9. Samples: 37245166. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:57:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:57:01,414][60935] Updated weights for policy 0, policy_version 72450 (0.0008) [2023-10-13 23:57:01,778][60935] Updated weights for policy 0, policy_version 72460 (0.0008) [2023-10-13 23:57:02,145][60935] Updated weights for policy 0, policy_version 72470 (0.0008) [2023-10-13 23:57:02,506][60935] Updated weights for policy 0, policy_version 72480 (0.0008) [2023-10-13 23:57:03,772][60934] Updated weights for policy 1, policy_version 72732 (0.0009) [2023-10-13 23:57:04,142][60934] Updated weights for policy 1, policy_version 72742 (0.0008) [2023-10-13 23:57:04,499][60934] Updated weights for policy 1, policy_version 72752 (0.0008) [2023-10-13 23:57:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 149028864. Throughput: 0: 1715.0, 1: 1672.9. Samples: 37265346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 23:57:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:57:06,518][60935] Updated weights for policy 0, policy_version 72490 (0.0008) [2023-10-13 23:57:06,889][60935] Updated weights for policy 0, policy_version 72500 (0.0009) [2023-10-13 23:57:07,257][60935] Updated weights for policy 0, policy_version 72510 (0.0009) [2023-10-13 23:57:08,666][60934] Updated weights for policy 1, policy_version 72762 (0.0010) [2023-10-13 23:57:09,068][60934] Updated weights for policy 1, policy_version 72772 (0.0008) [2023-10-13 23:57:09,428][60934] Updated weights for policy 1, policy_version 72782 (0.0007) [2023-10-13 23:57:09,798][60934] Updated weights for policy 1, policy_version 72792 (0.0007) [2023-10-13 23:57:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 149094400. Throughput: 0: 1724.0, 1: 1677.2. Samples: 37286016. Policy #0 lag: (min: 29.0, avg: 36.4, max: 61.0) [2023-10-13 23:57:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:57:11,318][60935] Updated weights for policy 0, policy_version 72520 (0.0010) [2023-10-13 23:57:11,684][60935] Updated weights for policy 0, policy_version 72530 (0.0009) [2023-10-13 23:57:12,047][60935] Updated weights for policy 0, policy_version 72540 (0.0009) [2023-10-13 23:57:13,730][60934] Updated weights for policy 1, policy_version 72802 (0.0007) [2023-10-13 23:57:14,100][60934] Updated weights for policy 1, policy_version 72812 (0.0008) [2023-10-13 23:57:14,462][60934] Updated weights for policy 1, policy_version 72822 (0.0007) [2023-10-13 23:57:16,127][60935] Updated weights for policy 0, policy_version 72550 (0.0008) [2023-10-13 23:57:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 149159936. Throughput: 0: 1715.4, 1: 1694.3. Samples: 37296308. Policy #0 lag: (min: 29.0, avg: 36.4, max: 61.0) [2023-10-13 23:57:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:57:16,505][60935] Updated weights for policy 0, policy_version 72560 (0.0009) [2023-10-13 23:57:16,877][60935] Updated weights for policy 0, policy_version 72570 (0.0008) [2023-10-13 23:57:18,559][60934] Updated weights for policy 1, policy_version 72832 (0.0009) [2023-10-13 23:57:18,931][60934] Updated weights for policy 1, policy_version 72842 (0.0007) [2023-10-13 23:57:19,293][60934] Updated weights for policy 1, policy_version 72852 (0.0008) [2023-10-13 23:57:20,828][60935] Updated weights for policy 0, policy_version 72580 (0.0010) [2023-10-13 23:57:21,196][60935] Updated weights for policy 0, policy_version 72590 (0.0010) [2023-10-13 23:57:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 149225472. Throughput: 0: 1713.2, 1: 1669.5. Samples: 37316028. Policy #0 lag: (min: 29.0, avg: 36.4, max: 61.0) [2023-10-13 23:57:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:57:21,560][60935] Updated weights for policy 0, policy_version 72600 (0.0007) [2023-10-13 23:57:23,450][60934] Updated weights for policy 1, policy_version 72862 (0.0008) [2023-10-13 23:57:23,822][60934] Updated weights for policy 1, policy_version 72872 (0.0009) [2023-10-13 23:57:24,201][60934] Updated weights for policy 1, policy_version 72882 (0.0009) [2023-10-13 23:57:25,573][60935] Updated weights for policy 0, policy_version 72610 (0.0008) [2023-10-13 23:57:25,948][60935] Updated weights for policy 0, policy_version 72620 (0.0009) [2023-10-13 23:57:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13551.5). Total num frames: 149291008. Throughput: 0: 1706.3, 1: 1684.3. Samples: 37336386. Policy #0 lag: (min: 29.0, avg: 36.4, max: 61.0) [2023-10-13 23:57:26,250][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:57:26,316][60935] Updated weights for policy 0, policy_version 72630 (0.0008) [2023-10-13 23:57:26,690][60935] Updated weights for policy 0, policy_version 72640 (0.0007) [2023-10-13 23:57:28,421][60934] Updated weights for policy 1, policy_version 72892 (0.0008) [2023-10-13 23:57:28,777][60934] Updated weights for policy 1, policy_version 72902 (0.0009) [2023-10-13 23:57:29,138][60934] Updated weights for policy 1, policy_version 72912 (0.0009) [2023-10-13 23:57:30,629][60935] Updated weights for policy 0, policy_version 72650 (0.0008) [2023-10-13 23:57:31,001][60935] Updated weights for policy 0, policy_version 72660 (0.0009) [2023-10-13 23:57:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 149356544. Throughput: 0: 1718.3, 1: 1675.8. Samples: 37346884. Policy #0 lag: (min: 29.0, avg: 36.4, max: 61.0) [2023-10-13 23:57:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:57:31,375][60935] Updated weights for policy 0, policy_version 72670 (0.0007) [2023-10-13 23:57:33,130][60934] Updated weights for policy 1, policy_version 72922 (0.0008) [2023-10-13 23:57:33,493][60934] Updated weights for policy 1, policy_version 72932 (0.0007) [2023-10-13 23:57:33,859][60934] Updated weights for policy 1, policy_version 72942 (0.0008) [2023-10-13 23:57:34,230][60934] Updated weights for policy 1, policy_version 72952 (0.0008) [2023-10-13 23:57:35,273][60935] Updated weights for policy 0, policy_version 72680 (0.0007) [2023-10-13 23:57:35,630][60935] Updated weights for policy 0, policy_version 72690 (0.0010) [2023-10-13 23:57:36,003][60935] Updated weights for policy 0, policy_version 72700 (0.0009) [2023-10-13 23:57:36,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 149454848. Throughput: 0: 1728.5, 1: 1675.2. Samples: 37367524. Policy #0 lag: (min: 29.0, avg: 36.4, max: 61.0) [2023-10-13 23:57:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:57:38,334][60934] Updated weights for policy 1, policy_version 72962 (0.0008) [2023-10-13 23:57:38,699][60934] Updated weights for policy 1, policy_version 72972 (0.0009) [2023-10-13 23:57:39,066][60934] Updated weights for policy 1, policy_version 72982 (0.0007) [2023-10-13 23:57:39,873][60935] Updated weights for policy 0, policy_version 72710 (0.0010) [2023-10-13 23:57:40,239][60935] Updated weights for policy 0, policy_version 72720 (0.0007) [2023-10-13 23:57:40,605][60935] Updated weights for policy 0, policy_version 72730 (0.0009) [2023-10-13 23:57:41,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 149520384. Throughput: 0: 1703.6, 1: 1694.8. Samples: 37387566. Policy #0 lag: (min: 29.0, avg: 36.4, max: 61.0) [2023-10-13 23:57:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:57:43,008][60934] Updated weights for policy 1, policy_version 72992 (0.0008) [2023-10-13 23:57:43,383][60934] Updated weights for policy 1, policy_version 73002 (0.0008) [2023-10-13 23:57:43,741][60934] Updated weights for policy 1, policy_version 73012 (0.0010) [2023-10-13 23:57:44,661][60935] Updated weights for policy 0, policy_version 72740 (0.0009) [2023-10-13 23:57:45,035][60935] Updated weights for policy 0, policy_version 72750 (0.0009) [2023-10-13 23:57:45,396][60935] Updated weights for policy 0, policy_version 72760 (0.0009) [2023-10-13 23:57:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 149585920. Throughput: 0: 1732.3, 1: 1673.5. Samples: 37398424. Policy #0 lag: (min: 29.0, avg: 36.4, max: 61.0) [2023-10-13 23:57:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:57:47,724][60934] Updated weights for policy 1, policy_version 73022 (0.0008) [2023-10-13 23:57:48,097][60934] Updated weights for policy 1, policy_version 73032 (0.0007) [2023-10-13 23:57:48,454][60934] Updated weights for policy 1, policy_version 73042 (0.0011) [2023-10-13 23:57:49,366][60935] Updated weights for policy 0, policy_version 72770 (0.0009) [2023-10-13 23:57:49,740][60935] Updated weights for policy 0, policy_version 72780 (0.0009) [2023-10-13 23:57:50,101][60935] Updated weights for policy 0, policy_version 72790 (0.0008) [2023-10-13 23:57:50,464][60935] Updated weights for policy 0, policy_version 72800 (0.0010) [2023-10-13 23:57:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 149651456. Throughput: 0: 1719.3, 1: 1683.1. Samples: 37418454. Policy #0 lag: (min: 29.0, avg: 36.4, max: 61.0) [2023-10-13 23:57:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:57:52,590][60934] Updated weights for policy 1, policy_version 73052 (0.0010) [2023-10-13 23:57:52,955][60934] Updated weights for policy 1, policy_version 73062 (0.0009) [2023-10-13 23:57:53,328][60934] Updated weights for policy 1, policy_version 73072 (0.0010) [2023-10-13 23:57:54,342][60935] Updated weights for policy 0, policy_version 72810 (0.0011) [2023-10-13 23:57:54,704][60935] Updated weights for policy 0, policy_version 72820 (0.0012) [2023-10-13 23:57:55,078][60935] Updated weights for policy 0, policy_version 72830 (0.0007) [2023-10-13 23:57:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 149716992. Throughput: 0: 1697.5, 1: 1695.6. Samples: 37438706. Policy #0 lag: (min: 9.0, avg: 24.8, max: 41.0) [2023-10-13 23:57:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:57:57,611][60934] Updated weights for policy 1, policy_version 73082 (0.0010) [2023-10-13 23:57:58,026][60934] Updated weights for policy 1, policy_version 73092 (0.0009) [2023-10-13 23:57:58,384][60934] Updated weights for policy 1, policy_version 73102 (0.0008) [2023-10-13 23:57:58,753][60934] Updated weights for policy 1, policy_version 73112 (0.0009) [2023-10-13 23:57:59,167][60935] Updated weights for policy 0, policy_version 72840 (0.0009) [2023-10-13 23:57:59,541][60935] Updated weights for policy 0, policy_version 72850 (0.0008) [2023-10-13 23:57:59,911][60935] Updated weights for policy 0, policy_version 72860 (0.0009) [2023-10-13 23:58:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 149782528. Throughput: 0: 1731.9, 1: 1669.2. Samples: 37449358. Policy #0 lag: (min: 9.0, avg: 24.8, max: 41.0) [2023-10-13 23:58:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:58:02,760][60934] Updated weights for policy 1, policy_version 73122 (0.0008) [2023-10-13 23:58:03,138][60934] Updated weights for policy 1, policy_version 73132 (0.0008) [2023-10-13 23:58:03,498][60934] Updated weights for policy 1, policy_version 73142 (0.0007) [2023-10-13 23:58:03,968][60935] Updated weights for policy 0, policy_version 72870 (0.0008) [2023-10-13 23:58:04,336][60935] Updated weights for policy 0, policy_version 72880 (0.0007) [2023-10-13 23:58:04,699][60935] Updated weights for policy 0, policy_version 72890 (0.0009) [2023-10-13 23:58:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 149848064. Throughput: 0: 1710.6, 1: 1690.7. Samples: 37469086. Policy #0 lag: (min: 9.0, avg: 24.8, max: 41.0) [2023-10-13 23:58:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:58:07,280][60934] Updated weights for policy 1, policy_version 73152 (0.0007) [2023-10-13 23:58:07,648][60934] Updated weights for policy 1, policy_version 73162 (0.0008) [2023-10-13 23:58:08,015][60934] Updated weights for policy 1, policy_version 73172 (0.0008) [2023-10-13 23:58:08,730][60935] Updated weights for policy 0, policy_version 72900 (0.0008) [2023-10-13 23:58:09,094][60935] Updated weights for policy 0, policy_version 72910 (0.0008) [2023-10-13 23:58:09,475][60935] Updated weights for policy 0, policy_version 72920 (0.0011) [2023-10-13 23:58:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 149913600. Throughput: 0: 1715.9, 1: 1703.8. Samples: 37490272. Policy #0 lag: (min: 9.0, avg: 24.8, max: 41.0) [2023-10-13 23:58:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:58:11,984][60934] Updated weights for policy 1, policy_version 73182 (0.0008) [2023-10-13 23:58:12,346][60934] Updated weights for policy 1, policy_version 73192 (0.0008) [2023-10-13 23:58:12,721][60934] Updated weights for policy 1, policy_version 73202 (0.0008) [2023-10-13 23:58:13,286][60935] Updated weights for policy 0, policy_version 72930 (0.0008) [2023-10-13 23:58:13,649][60935] Updated weights for policy 0, policy_version 72940 (0.0009) [2023-10-13 23:58:14,020][60935] Updated weights for policy 0, policy_version 72950 (0.0007) [2023-10-13 23:58:14,381][60935] Updated weights for policy 0, policy_version 72960 (0.0009) [2023-10-13 23:58:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 149979136. Throughput: 0: 1723.9, 1: 1685.0. Samples: 37500288. Policy #0 lag: (min: 9.0, avg: 24.8, max: 41.0) [2023-10-13 23:58:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:58:16,729][60934] Updated weights for policy 1, policy_version 73212 (0.0008) [2023-10-13 23:58:17,094][60934] Updated weights for policy 1, policy_version 73222 (0.0009) [2023-10-13 23:58:17,461][60934] Updated weights for policy 1, policy_version 73232 (0.0008) [2023-10-13 23:58:18,350][60935] Updated weights for policy 0, policy_version 72970 (0.0007) [2023-10-13 23:58:18,717][60935] Updated weights for policy 0, policy_version 72980 (0.0009) [2023-10-13 23:58:19,088][60935] Updated weights for policy 0, policy_version 72990 (0.0010) [2023-10-13 23:58:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 150044672. Throughput: 0: 1693.6, 1: 1706.7. Samples: 37520542. Policy #0 lag: (min: 9.0, avg: 24.8, max: 41.0) [2023-10-13 23:58:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:58:21,540][60934] Updated weights for policy 1, policy_version 73242 (0.0008) [2023-10-13 23:58:21,896][60934] Updated weights for policy 1, policy_version 73252 (0.0011) [2023-10-13 23:58:22,262][60934] Updated weights for policy 1, policy_version 73262 (0.0011) [2023-10-13 23:58:22,640][60934] Updated weights for policy 1, policy_version 73272 (0.0009) [2023-10-13 23:58:23,026][60935] Updated weights for policy 0, policy_version 73000 (0.0009) [2023-10-13 23:58:23,390][60935] Updated weights for policy 0, policy_version 73010 (0.0009) [2023-10-13 23:58:23,766][60935] Updated weights for policy 0, policy_version 73020 (0.0009) [2023-10-13 23:58:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 150110208. Throughput: 0: 1719.1, 1: 1705.2. Samples: 37541660. Policy #0 lag: (min: 9.0, avg: 24.8, max: 41.0) [2023-10-13 23:58:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:58:26,451][60934] Updated weights for policy 1, policy_version 73282 (0.0011) [2023-10-13 23:58:26,820][60934] Updated weights for policy 1, policy_version 73292 (0.0010) [2023-10-13 23:58:27,186][60934] Updated weights for policy 1, policy_version 73302 (0.0010) [2023-10-13 23:58:27,578][60935] Updated weights for policy 0, policy_version 73030 (0.0008) [2023-10-13 23:58:27,938][60935] Updated weights for policy 0, policy_version 73040 (0.0009) [2023-10-13 23:58:28,313][60935] Updated weights for policy 0, policy_version 73050 (0.0008) [2023-10-13 23:58:31,174][60934] Updated weights for policy 1, policy_version 73312 (0.0007) [2023-10-13 23:58:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 150175744. Throughput: 0: 1692.4, 1: 1699.4. Samples: 37551056. Policy #0 lag: (min: 9.0, avg: 24.8, max: 41.0) [2023-10-13 23:58:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:58:31,538][60934] Updated weights for policy 1, policy_version 73322 (0.0007) [2023-10-13 23:58:31,901][60934] Updated weights for policy 1, policy_version 73332 (0.0009) [2023-10-13 23:58:32,311][60935] Updated weights for policy 0, policy_version 73060 (0.0010) [2023-10-13 23:58:32,680][60935] Updated weights for policy 0, policy_version 73070 (0.0010) [2023-10-13 23:58:33,052][60935] Updated weights for policy 0, policy_version 73080 (0.0009) [2023-10-13 23:58:35,881][60934] Updated weights for policy 1, policy_version 73342 (0.0008) [2023-10-13 23:58:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 150241280. Throughput: 0: 1704.8, 1: 1707.8. Samples: 37572022. Policy #0 lag: (min: 9.0, avg: 24.8, max: 41.0) [2023-10-13 23:58:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:58:36,252][60934] Updated weights for policy 1, policy_version 73352 (0.0007) [2023-10-13 23:58:36,612][60934] Updated weights for policy 1, policy_version 73362 (0.0011) [2023-10-13 23:58:36,927][60935] Updated weights for policy 0, policy_version 73090 (0.0008) [2023-10-13 23:58:37,295][60935] Updated weights for policy 0, policy_version 73100 (0.0008) [2023-10-13 23:58:37,662][60935] Updated weights for policy 0, policy_version 73110 (0.0009) [2023-10-13 23:58:38,032][60935] Updated weights for policy 0, policy_version 73120 (0.0008) [2023-10-13 23:58:40,783][60934] Updated weights for policy 1, policy_version 73372 (0.0009) [2023-10-13 23:58:41,152][60934] Updated weights for policy 1, policy_version 73382 (0.0008) [2023-10-13 23:58:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 150306816. Throughput: 0: 1720.5, 1: 1706.0. Samples: 37592898. Policy #0 lag: (min: 9.0, avg: 24.8, max: 41.0) [2023-10-13 23:58:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:58:41,513][60934] Updated weights for policy 1, policy_version 73392 (0.0010) [2023-10-13 23:58:42,012][60935] Updated weights for policy 0, policy_version 73130 (0.0010) [2023-10-13 23:58:42,386][60935] Updated weights for policy 0, policy_version 73140 (0.0011) [2023-10-13 23:58:42,752][60935] Updated weights for policy 0, policy_version 73150 (0.0010) [2023-10-13 23:58:45,429][60934] Updated weights for policy 1, policy_version 73402 (0.0008) [2023-10-13 23:58:45,836][60934] Updated weights for policy 1, policy_version 73412 (0.0007) [2023-10-13 23:58:46,205][60934] Updated weights for policy 1, policy_version 73422 (0.0008) [2023-10-13 23:58:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 150372352. Throughput: 0: 1690.2, 1: 1706.8. Samples: 37602224. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-13 23:58:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:58:46,581][60934] Updated weights for policy 1, policy_version 73432 (0.0007) [2023-10-13 23:58:46,973][60935] Updated weights for policy 0, policy_version 73160 (0.0007) [2023-10-13 23:58:47,345][60935] Updated weights for policy 0, policy_version 73170 (0.0008) [2023-10-13 23:58:47,706][60935] Updated weights for policy 0, policy_version 73180 (0.0010) [2023-10-13 23:58:50,497][60934] Updated weights for policy 1, policy_version 73442 (0.0009) [2023-10-13 23:58:50,863][60934] Updated weights for policy 1, policy_version 73452 (0.0009) [2023-10-13 23:58:51,225][60934] Updated weights for policy 1, policy_version 73462 (0.0009) [2023-10-13 23:58:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 150437888. Throughput: 0: 1712.1, 1: 1712.0. Samples: 37623172. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-13 23:58:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:58:51,823][60935] Updated weights for policy 0, policy_version 73190 (0.0009) [2023-10-13 23:58:52,189][60935] Updated weights for policy 0, policy_version 73200 (0.0009) [2023-10-13 23:58:52,562][60935] Updated weights for policy 0, policy_version 73210 (0.0008) [2023-10-13 23:58:55,349][60934] Updated weights for policy 1, policy_version 73472 (0.0008) [2023-10-13 23:58:55,716][60934] Updated weights for policy 1, policy_version 73482 (0.0007) [2023-10-13 23:58:56,083][60934] Updated weights for policy 1, policy_version 73492 (0.0009) [2023-10-13 23:58:56,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 150536192. Throughput: 0: 1713.1, 1: 1697.0. Samples: 37643728. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-13 23:58:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:58:56,259][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000073496_75563008.pth... [2023-10-13 23:58:56,298][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000071896_73924608.pth [2023-10-13 23:58:56,458][60935] Updated weights for policy 0, policy_version 73220 (0.0010) [2023-10-13 23:58:56,835][60935] Updated weights for policy 0, policy_version 73230 (0.0008) [2023-10-13 23:58:57,197][60935] Updated weights for policy 0, policy_version 73240 (0.0011) [2023-10-13 23:58:57,491][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000073248_75005952.pth... [2023-10-13 23:58:57,520][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000071616_73334784.pth [2023-10-13 23:59:00,048][60934] Updated weights for policy 1, policy_version 73502 (0.0010) [2023-10-13 23:59:00,414][60934] Updated weights for policy 1, policy_version 73512 (0.0010) [2023-10-13 23:59:00,784][60934] Updated weights for policy 1, policy_version 73522 (0.0008) [2023-10-13 23:59:01,244][60935] Updated weights for policy 0, policy_version 73250 (0.0007) [2023-10-13 23:59:01,248][59943] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 150601728. Throughput: 0: 1697.4, 1: 1708.3. Samples: 37653542. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-13 23:59:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:59:01,611][60935] Updated weights for policy 0, policy_version 73260 (0.0007) [2023-10-13 23:59:01,973][60935] Updated weights for policy 0, policy_version 73270 (0.0010) [2023-10-13 23:59:02,338][60935] Updated weights for policy 0, policy_version 73280 (0.0011) [2023-10-13 23:59:04,961][60934] Updated weights for policy 1, policy_version 73532 (0.0007) [2023-10-13 23:59:05,338][60934] Updated weights for policy 1, policy_version 73542 (0.0007) [2023-10-13 23:59:05,705][60934] Updated weights for policy 1, policy_version 73552 (0.0007) [2023-10-13 23:59:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 150667264. Throughput: 0: 1716.9, 1: 1704.8. Samples: 37674516. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-13 23:59:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:59:06,258][60935] Updated weights for policy 0, policy_version 73290 (0.0009) [2023-10-13 23:59:06,633][60935] Updated weights for policy 0, policy_version 73300 (0.0011) [2023-10-13 23:59:07,000][60935] Updated weights for policy 0, policy_version 73310 (0.0011) [2023-10-13 23:59:09,617][60934] Updated weights for policy 1, policy_version 73562 (0.0008) [2023-10-13 23:59:09,983][60934] Updated weights for policy 1, policy_version 73572 (0.0008) [2023-10-13 23:59:10,346][60934] Updated weights for policy 1, policy_version 73582 (0.0007) [2023-10-13 23:59:10,715][60934] Updated weights for policy 1, policy_version 73592 (0.0007) [2023-10-13 23:59:10,980][60935] Updated weights for policy 0, policy_version 73320 (0.0009) [2023-10-13 23:59:11,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 150732800. Throughput: 0: 1715.1, 1: 1682.9. Samples: 37694570. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-13 23:59:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:59:11,353][60935] Updated weights for policy 0, policy_version 73330 (0.0008) [2023-10-13 23:59:11,719][60935] Updated weights for policy 0, policy_version 73340 (0.0009) [2023-10-13 23:59:14,876][60934] Updated weights for policy 1, policy_version 73602 (0.0009) [2023-10-13 23:59:15,257][60934] Updated weights for policy 1, policy_version 73612 (0.0010) [2023-10-13 23:59:15,618][60934] Updated weights for policy 1, policy_version 73622 (0.0009) [2023-10-13 23:59:15,693][60935] Updated weights for policy 0, policy_version 73350 (0.0008) [2023-10-13 23:59:16,054][60935] Updated weights for policy 0, policy_version 73360 (0.0007) [2023-10-13 23:59:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 150798336. Throughput: 0: 1720.4, 1: 1703.7. Samples: 37705142. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-13 23:59:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:59:16,427][60935] Updated weights for policy 0, policy_version 73370 (0.0009) [2023-10-13 23:59:19,589][60934] Updated weights for policy 1, policy_version 73632 (0.0008) [2023-10-13 23:59:19,951][60934] Updated weights for policy 1, policy_version 73642 (0.0009) [2023-10-13 23:59:20,295][60935] Updated weights for policy 0, policy_version 73380 (0.0008) [2023-10-13 23:59:20,322][60934] Updated weights for policy 1, policy_version 73652 (0.0008) [2023-10-13 23:59:20,668][60935] Updated weights for policy 0, policy_version 73390 (0.0008) [2023-10-13 23:59:21,026][60935] Updated weights for policy 0, policy_version 73400 (0.0007) [2023-10-13 23:59:21,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 150863872. Throughput: 0: 1722.3, 1: 1704.1. Samples: 37726212. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-13 23:59:21,250][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:59:24,417][60934] Updated weights for policy 1, policy_version 73662 (0.0007) [2023-10-13 23:59:24,780][60934] Updated weights for policy 1, policy_version 73672 (0.0007) [2023-10-13 23:59:25,141][60934] Updated weights for policy 1, policy_version 73682 (0.0009) [2023-10-13 23:59:25,179][60935] Updated weights for policy 0, policy_version 73410 (0.0010) [2023-10-13 23:59:25,536][60935] Updated weights for policy 0, policy_version 73420 (0.0007) [2023-10-13 23:59:25,897][60935] Updated weights for policy 0, policy_version 73430 (0.0007) [2023-10-13 23:59:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 150929408. Throughput: 0: 1704.6, 1: 1683.4. Samples: 37745360. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-13 23:59:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:59:26,264][60935] Updated weights for policy 0, policy_version 73440 (0.0007) [2023-10-13 23:59:29,226][60934] Updated weights for policy 1, policy_version 73692 (0.0008) [2023-10-13 23:59:29,587][60934] Updated weights for policy 1, policy_version 73702 (0.0009) [2023-10-13 23:59:29,958][60934] Updated weights for policy 1, policy_version 73712 (0.0008) [2023-10-13 23:59:30,157][60935] Updated weights for policy 0, policy_version 73450 (0.0009) [2023-10-13 23:59:30,523][60935] Updated weights for policy 0, policy_version 73460 (0.0008) [2023-10-13 23:59:30,898][60935] Updated weights for policy 0, policy_version 73470 (0.0009) [2023-10-13 23:59:31,248][59943] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 151027712. Throughput: 0: 1723.6, 1: 1708.2. Samples: 37756652. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-13 23:59:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:59:34,125][60934] Updated weights for policy 1, policy_version 73722 (0.0008) [2023-10-13 23:59:34,532][60934] Updated weights for policy 1, policy_version 73732 (0.0008) [2023-10-13 23:59:34,720][60935] Updated weights for policy 0, policy_version 73480 (0.0010) [2023-10-13 23:59:34,897][60934] Updated weights for policy 1, policy_version 73742 (0.0007) [2023-10-13 23:59:35,086][60935] Updated weights for policy 0, policy_version 73490 (0.0010) [2023-10-13 23:59:35,256][60934] Updated weights for policy 1, policy_version 73752 (0.0007) [2023-10-13 23:59:35,453][60935] Updated weights for policy 0, policy_version 73500 (0.0010) [2023-10-13 23:59:36,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 151093248. Throughput: 0: 1722.1, 1: 1690.8. Samples: 37776752. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-13 23:59:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:59:39,115][60934] Updated weights for policy 1, policy_version 73762 (0.0008) [2023-10-13 23:59:39,475][60934] Updated weights for policy 1, policy_version 73772 (0.0008) [2023-10-13 23:59:39,566][60935] Updated weights for policy 0, policy_version 73510 (0.0009) [2023-10-13 23:59:39,839][60934] Updated weights for policy 1, policy_version 73782 (0.0009) [2023-10-13 23:59:39,935][60935] Updated weights for policy 0, policy_version 73520 (0.0007) [2023-10-13 23:59:40,302][60935] Updated weights for policy 0, policy_version 73530 (0.0009) [2023-10-13 23:59:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 151158784. Throughput: 0: 1702.4, 1: 1686.1. Samples: 37796210. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-13 23:59:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:59:43,958][60934] Updated weights for policy 1, policy_version 73792 (0.0008) [2023-10-13 23:59:44,322][60934] Updated weights for policy 1, policy_version 73802 (0.0008) [2023-10-13 23:59:44,358][60935] Updated weights for policy 0, policy_version 73540 (0.0008) [2023-10-13 23:59:44,687][60934] Updated weights for policy 1, policy_version 73812 (0.0007) [2023-10-13 23:59:44,720][60935] Updated weights for policy 0, policy_version 73550 (0.0010) [2023-10-13 23:59:45,085][60935] Updated weights for policy 0, policy_version 73560 (0.0008) [2023-10-13 23:59:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 151224320. Throughput: 0: 1728.6, 1: 1700.8. Samples: 37807866. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-13 23:59:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:59:48,609][60934] Updated weights for policy 1, policy_version 73822 (0.0009) [2023-10-13 23:59:48,947][60935] Updated weights for policy 0, policy_version 73570 (0.0007) [2023-10-13 23:59:48,984][60934] Updated weights for policy 1, policy_version 73832 (0.0009) [2023-10-13 23:59:49,324][60935] Updated weights for policy 0, policy_version 73580 (0.0008) [2023-10-13 23:59:49,346][60934] Updated weights for policy 1, policy_version 73842 (0.0008) [2023-10-13 23:59:49,683][60935] Updated weights for policy 0, policy_version 73590 (0.0007) [2023-10-13 23:59:50,058][60935] Updated weights for policy 0, policy_version 73600 (0.0009) [2023-10-13 23:59:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 151289856. Throughput: 0: 1709.0, 1: 1675.6. Samples: 37826826. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-13 23:59:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:59:53,386][60934] Updated weights for policy 1, policy_version 73852 (0.0010) [2023-10-13 23:59:53,746][60934] Updated weights for policy 1, policy_version 73862 (0.0010) [2023-10-13 23:59:54,074][60935] Updated weights for policy 0, policy_version 73610 (0.0009) [2023-10-13 23:59:54,111][60934] Updated weights for policy 1, policy_version 73872 (0.0008) [2023-10-13 23:59:54,441][60935] Updated weights for policy 0, policy_version 73620 (0.0009) [2023-10-13 23:59:54,806][60935] Updated weights for policy 0, policy_version 73630 (0.0010) [2023-10-13 23:59:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 151355392. Throughput: 0: 1704.2, 1: 1692.5. Samples: 37847422. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-13 23:59:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-13 23:59:57,999][60934] Updated weights for policy 1, policy_version 73882 (0.0008) [2023-10-13 23:59:58,360][60934] Updated weights for policy 1, policy_version 73892 (0.0008) [2023-10-13 23:59:58,669][60935] Updated weights for policy 0, policy_version 73640 (0.0008) [2023-10-13 23:59:58,723][60934] Updated weights for policy 1, policy_version 73902 (0.0009) [2023-10-13 23:59:59,032][60935] Updated weights for policy 0, policy_version 73650 (0.0008) [2023-10-13 23:59:59,087][60934] Updated weights for policy 1, policy_version 73912 (0.0007) [2023-10-13 23:59:59,403][60935] Updated weights for policy 0, policy_version 73660 (0.0011) [2023-10-14 00:00:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 151420928. Throughput: 0: 1719.1, 1: 1687.4. Samples: 37858434. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-14 00:00:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:00:03,170][60934] Updated weights for policy 1, policy_version 73922 (0.0010) [2023-10-14 00:00:03,526][60934] Updated weights for policy 1, policy_version 73932 (0.0007) [2023-10-14 00:00:03,535][60935] Updated weights for policy 0, policy_version 73670 (0.0010) [2023-10-14 00:00:03,886][60934] Updated weights for policy 1, policy_version 73942 (0.0007) [2023-10-14 00:00:03,902][60935] Updated weights for policy 0, policy_version 73680 (0.0008) [2023-10-14 00:00:04,269][60935] Updated weights for policy 0, policy_version 73690 (0.0008) [2023-10-14 00:00:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 151486464. Throughput: 0: 1695.3, 1: 1672.0. Samples: 37877740. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-14 00:00:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:00:07,873][60934] Updated weights for policy 1, policy_version 73952 (0.0008) [2023-10-14 00:00:08,209][60935] Updated weights for policy 0, policy_version 73700 (0.0009) [2023-10-14 00:00:08,238][60934] Updated weights for policy 1, policy_version 73962 (0.0009) [2023-10-14 00:00:08,579][60935] Updated weights for policy 0, policy_version 73710 (0.0007) [2023-10-14 00:00:08,601][60934] Updated weights for policy 1, policy_version 73972 (0.0008) [2023-10-14 00:00:08,946][60935] Updated weights for policy 0, policy_version 73720 (0.0008) [2023-10-14 00:00:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 151552000. Throughput: 0: 1715.7, 1: 1696.8. Samples: 37898922. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-14 00:00:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:00:12,680][60934] Updated weights for policy 1, policy_version 73982 (0.0008) [2023-10-14 00:00:12,942][60935] Updated weights for policy 0, policy_version 73730 (0.0008) [2023-10-14 00:00:13,043][60934] Updated weights for policy 1, policy_version 73992 (0.0008) [2023-10-14 00:00:13,312][60935] Updated weights for policy 0, policy_version 73740 (0.0007) [2023-10-14 00:00:13,409][60934] Updated weights for policy 1, policy_version 74002 (0.0007) [2023-10-14 00:00:13,674][60935] Updated weights for policy 0, policy_version 73750 (0.0008) [2023-10-14 00:00:14,052][60935] Updated weights for policy 0, policy_version 73760 (0.0011) [2023-10-14 00:00:16,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 151617536. Throughput: 0: 1705.0, 1: 1674.5. Samples: 37908726. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-14 00:00:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:00:17,341][60934] Updated weights for policy 1, policy_version 74012 (0.0008) [2023-10-14 00:00:17,707][60934] Updated weights for policy 1, policy_version 74022 (0.0007) [2023-10-14 00:00:18,063][60934] Updated weights for policy 1, policy_version 74032 (0.0007) [2023-10-14 00:00:18,143][60935] Updated weights for policy 0, policy_version 73770 (0.0010) [2023-10-14 00:00:18,505][60935] Updated weights for policy 0, policy_version 73780 (0.0011) [2023-10-14 00:00:18,878][60935] Updated weights for policy 0, policy_version 73790 (0.0008) [2023-10-14 00:00:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 151683072. Throughput: 0: 1699.8, 1: 1686.5. Samples: 37929138. Policy #0 lag: (min: 22.0, avg: 26.7, max: 54.0) [2023-10-14 00:00:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:00:21,944][60934] Updated weights for policy 1, policy_version 74042 (0.0008) [2023-10-14 00:00:22,305][60934] Updated weights for policy 1, policy_version 74052 (0.0010) [2023-10-14 00:00:22,672][60934] Updated weights for policy 1, policy_version 74062 (0.0008) [2023-10-14 00:00:22,921][60935] Updated weights for policy 0, policy_version 73800 (0.0008) [2023-10-14 00:00:23,030][60934] Updated weights for policy 1, policy_version 74072 (0.0008) [2023-10-14 00:00:23,299][60935] Updated weights for policy 0, policy_version 73810 (0.0009) [2023-10-14 00:00:23,670][60935] Updated weights for policy 0, policy_version 73820 (0.0010) [2023-10-14 00:00:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 151748608. Throughput: 0: 1710.8, 1: 1701.9. Samples: 37949778. Policy #0 lag: (min: 22.0, avg: 26.7, max: 54.0) [2023-10-14 00:00:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:00:27,357][60934] Updated weights for policy 1, policy_version 74082 (0.0007) [2023-10-14 00:00:27,688][60935] Updated weights for policy 0, policy_version 73830 (0.0009) [2023-10-14 00:00:27,720][60934] Updated weights for policy 1, policy_version 74092 (0.0008) [2023-10-14 00:00:28,061][60935] Updated weights for policy 0, policy_version 73840 (0.0008) [2023-10-14 00:00:28,087][60934] Updated weights for policy 1, policy_version 74102 (0.0007) [2023-10-14 00:00:28,414][60935] Updated weights for policy 0, policy_version 73850 (0.0009) [2023-10-14 00:00:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 151814144. Throughput: 0: 1682.0, 1: 1671.1. Samples: 37958754. Policy #0 lag: (min: 22.0, avg: 26.7, max: 54.0) [2023-10-14 00:00:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:00:32,391][60934] Updated weights for policy 1, policy_version 74112 (0.0009) [2023-10-14 00:00:32,568][60935] Updated weights for policy 0, policy_version 73860 (0.0008) [2023-10-14 00:00:32,762][60934] Updated weights for policy 1, policy_version 74122 (0.0009) [2023-10-14 00:00:32,931][60935] Updated weights for policy 0, policy_version 73870 (0.0009) [2023-10-14 00:00:33,120][60934] Updated weights for policy 1, policy_version 74132 (0.0008) [2023-10-14 00:00:33,294][60935] Updated weights for policy 0, policy_version 73880 (0.0007) [2023-10-14 00:00:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 151879680. Throughput: 0: 1699.4, 1: 1688.3. Samples: 37979272. Policy #0 lag: (min: 22.0, avg: 26.7, max: 54.0) [2023-10-14 00:00:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:00:37,197][60934] Updated weights for policy 1, policy_version 74142 (0.0011) [2023-10-14 00:00:37,200][60935] Updated weights for policy 0, policy_version 73890 (0.0008) [2023-10-14 00:00:37,562][60934] Updated weights for policy 1, policy_version 74152 (0.0009) [2023-10-14 00:00:37,567][60935] Updated weights for policy 0, policy_version 73900 (0.0010) [2023-10-14 00:00:37,930][60935] Updated weights for policy 0, policy_version 73910 (0.0009) [2023-10-14 00:00:37,932][60934] Updated weights for policy 1, policy_version 74162 (0.0007) [2023-10-14 00:00:38,296][60935] Updated weights for policy 0, policy_version 73920 (0.0008) [2023-10-14 00:00:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 151945216. Throughput: 0: 1711.3, 1: 1687.9. Samples: 38000388. Policy #0 lag: (min: 22.0, avg: 26.7, max: 54.0) [2023-10-14 00:00:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:00:42,111][60934] Updated weights for policy 1, policy_version 74172 (0.0008) [2023-10-14 00:00:42,207][60935] Updated weights for policy 0, policy_version 73930 (0.0008) [2023-10-14 00:00:42,466][60934] Updated weights for policy 1, policy_version 74182 (0.0007) [2023-10-14 00:00:42,587][60935] Updated weights for policy 0, policy_version 73940 (0.0008) [2023-10-14 00:00:42,828][60934] Updated weights for policy 1, policy_version 74192 (0.0008) [2023-10-14 00:00:42,958][60935] Updated weights for policy 0, policy_version 73950 (0.0008) [2023-10-14 00:00:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 152010752. Throughput: 0: 1689.9, 1: 1668.3. Samples: 38009554. Policy #0 lag: (min: 22.0, avg: 26.7, max: 54.0) [2023-10-14 00:00:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:00:46,769][60934] Updated weights for policy 1, policy_version 74202 (0.0010) [2023-10-14 00:00:46,953][60935] Updated weights for policy 0, policy_version 73960 (0.0007) [2023-10-14 00:00:47,142][60934] Updated weights for policy 1, policy_version 74212 (0.0009) [2023-10-14 00:00:47,322][60935] Updated weights for policy 0, policy_version 73970 (0.0009) [2023-10-14 00:00:47,507][60934] Updated weights for policy 1, policy_version 74222 (0.0009) [2023-10-14 00:00:47,686][60935] Updated weights for policy 0, policy_version 73980 (0.0009) [2023-10-14 00:00:47,866][60934] Updated weights for policy 1, policy_version 74232 (0.0008) [2023-10-14 00:00:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 152076288. Throughput: 0: 1714.0, 1: 1683.0. Samples: 38030604. Policy #0 lag: (min: 22.0, avg: 26.7, max: 54.0) [2023-10-14 00:00:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:00:51,556][60935] Updated weights for policy 0, policy_version 73990 (0.0009) [2023-10-14 00:00:51,899][60934] Updated weights for policy 1, policy_version 74242 (0.0008) [2023-10-14 00:00:51,916][60935] Updated weights for policy 0, policy_version 74000 (0.0008) [2023-10-14 00:00:52,264][60934] Updated weights for policy 1, policy_version 74252 (0.0009) [2023-10-14 00:00:52,291][60935] Updated weights for policy 0, policy_version 74010 (0.0007) [2023-10-14 00:00:52,634][60934] Updated weights for policy 1, policy_version 74262 (0.0009) [2023-10-14 00:00:56,214][60935] Updated weights for policy 0, policy_version 74020 (0.0009) [2023-10-14 00:00:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 152141824. Throughput: 0: 1717.1, 1: 1683.6. Samples: 38051952. Policy #0 lag: (min: 22.0, avg: 26.7, max: 54.0) [2023-10-14 00:00:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:00:56,554][60934] Updated weights for policy 1, policy_version 74272 (0.0007) [2023-10-14 00:00:56,588][60935] Updated weights for policy 0, policy_version 74030 (0.0008) [2023-10-14 00:00:56,919][60934] Updated weights for policy 1, policy_version 74282 (0.0009) [2023-10-14 00:00:56,951][60935] Updated weights for policy 0, policy_version 74040 (0.0008) [2023-10-14 00:00:57,246][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000074048_75825152.pth... [2023-10-14 00:00:57,286][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000072416_74153984.pth [2023-10-14 00:00:57,289][60934] Updated weights for policy 1, policy_version 74292 (0.0008) [2023-10-14 00:00:57,426][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000074296_76382208.pth... [2023-10-14 00:00:57,454][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000072696_74743808.pth [2023-10-14 00:01:00,812][60935] Updated weights for policy 0, policy_version 74050 (0.0008) [2023-10-14 00:01:01,172][60935] Updated weights for policy 0, policy_version 74060 (0.0009) [2023-10-14 00:01:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 152207360. Throughput: 0: 1709.2, 1: 1682.2. Samples: 38061338. Policy #0 lag: (min: 22.0, avg: 26.7, max: 54.0) [2023-10-14 00:01:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:01:01,275][60934] Updated weights for policy 1, policy_version 74302 (0.0009) [2023-10-14 00:01:01,539][60935] Updated weights for policy 0, policy_version 74070 (0.0008) [2023-10-14 00:01:01,641][60934] Updated weights for policy 1, policy_version 74312 (0.0008) [2023-10-14 00:01:01,901][60935] Updated weights for policy 0, policy_version 74080 (0.0010) [2023-10-14 00:01:02,015][60934] Updated weights for policy 1, policy_version 74322 (0.0008) [2023-10-14 00:01:05,812][60935] Updated weights for policy 0, policy_version 74090 (0.0008) [2023-10-14 00:01:06,034][60934] Updated weights for policy 1, policy_version 74332 (0.0009) [2023-10-14 00:01:06,177][60935] Updated weights for policy 0, policy_version 74100 (0.0008) [2023-10-14 00:01:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 152272896. Throughput: 0: 1723.6, 1: 1687.3. Samples: 38082630. Policy #0 lag: (min: 22.0, avg: 26.7, max: 54.0) [2023-10-14 00:01:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:01:06,405][60934] Updated weights for policy 1, policy_version 74342 (0.0007) [2023-10-14 00:01:06,545][60935] Updated weights for policy 0, policy_version 74110 (0.0007) [2023-10-14 00:01:06,760][60934] Updated weights for policy 1, policy_version 74352 (0.0008) [2023-10-14 00:01:10,600][60935] Updated weights for policy 0, policy_version 74120 (0.0009) [2023-10-14 00:01:10,876][60934] Updated weights for policy 1, policy_version 74362 (0.0010) [2023-10-14 00:01:10,980][60935] Updated weights for policy 0, policy_version 74130 (0.0007) [2023-10-14 00:01:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 152338432. Throughput: 0: 1721.6, 1: 1689.8. Samples: 38103292. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 00:01:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:01:11,263][60934] Updated weights for policy 1, policy_version 74372 (0.0007) [2023-10-14 00:01:11,339][60935] Updated weights for policy 0, policy_version 74140 (0.0008) [2023-10-14 00:01:11,629][60934] Updated weights for policy 1, policy_version 74382 (0.0007) [2023-10-14 00:01:11,983][60934] Updated weights for policy 1, policy_version 74392 (0.0008) [2023-10-14 00:01:15,371][60935] Updated weights for policy 0, policy_version 74150 (0.0009) [2023-10-14 00:01:15,744][60935] Updated weights for policy 0, policy_version 74160 (0.0007) [2023-10-14 00:01:15,989][60934] Updated weights for policy 1, policy_version 74402 (0.0007) [2023-10-14 00:01:16,119][60935] Updated weights for policy 0, policy_version 74170 (0.0009) [2023-10-14 00:01:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 152403968. Throughput: 0: 1734.0, 1: 1691.9. Samples: 38112922. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 00:01:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:01:16,355][60934] Updated weights for policy 1, policy_version 74412 (0.0007) [2023-10-14 00:01:16,730][60934] Updated weights for policy 1, policy_version 74422 (0.0008) [2023-10-14 00:01:20,166][60935] Updated weights for policy 0, policy_version 74180 (0.0009) [2023-10-14 00:01:20,535][60935] Updated weights for policy 0, policy_version 74190 (0.0009) [2023-10-14 00:01:20,742][60934] Updated weights for policy 1, policy_version 74432 (0.0009) [2023-10-14 00:01:20,893][60935] Updated weights for policy 0, policy_version 74200 (0.0007) [2023-10-14 00:01:21,110][60934] Updated weights for policy 1, policy_version 74442 (0.0010) [2023-10-14 00:01:21,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 152502272. Throughput: 0: 1736.2, 1: 1704.5. Samples: 38134100. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 00:01:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:01:21,479][60934] Updated weights for policy 1, policy_version 74452 (0.0008) [2023-10-14 00:01:24,988][60935] Updated weights for policy 0, policy_version 74210 (0.0008) [2023-10-14 00:01:25,346][60935] Updated weights for policy 0, policy_version 74220 (0.0009) [2023-10-14 00:01:25,500][60934] Updated weights for policy 1, policy_version 74462 (0.0008) [2023-10-14 00:01:25,711][60935] Updated weights for policy 0, policy_version 74230 (0.0009) [2023-10-14 00:01:25,862][60934] Updated weights for policy 1, policy_version 74472 (0.0008) [2023-10-14 00:01:26,083][60935] Updated weights for policy 0, policy_version 74240 (0.0007) [2023-10-14 00:01:26,233][60934] Updated weights for policy 1, policy_version 74482 (0.0007) [2023-10-14 00:01:26,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 152567808. Throughput: 0: 1704.0, 1: 1698.2. Samples: 38153486. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 00:01:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:01:30,078][60935] Updated weights for policy 0, policy_version 74250 (0.0009) [2023-10-14 00:01:30,296][60934] Updated weights for policy 1, policy_version 74492 (0.0008) [2023-10-14 00:01:30,440][60935] Updated weights for policy 0, policy_version 74260 (0.0008) [2023-10-14 00:01:30,656][60934] Updated weights for policy 1, policy_version 74502 (0.0008) [2023-10-14 00:01:30,813][60935] Updated weights for policy 0, policy_version 74270 (0.0010) [2023-10-14 00:01:31,024][60934] Updated weights for policy 1, policy_version 74512 (0.0008) [2023-10-14 00:01:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 152633344. Throughput: 0: 1724.0, 1: 1708.2. Samples: 38164006. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 00:01:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:01:34,745][60935] Updated weights for policy 0, policy_version 74280 (0.0009) [2023-10-14 00:01:34,979][60934] Updated weights for policy 1, policy_version 74522 (0.0007) [2023-10-14 00:01:35,113][60935] Updated weights for policy 0, policy_version 74290 (0.0009) [2023-10-14 00:01:35,345][60934] Updated weights for policy 1, policy_version 74532 (0.0009) [2023-10-14 00:01:35,480][60935] Updated weights for policy 0, policy_version 74300 (0.0008) [2023-10-14 00:01:35,705][60934] Updated weights for policy 1, policy_version 74542 (0.0007) [2023-10-14 00:01:36,070][60934] Updated weights for policy 1, policy_version 74552 (0.0009) [2023-10-14 00:01:36,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 152731648. Throughput: 0: 1711.5, 1: 1713.0. Samples: 38184706. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 00:01:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:01:39,359][60935] Updated weights for policy 0, policy_version 74310 (0.0008) [2023-10-14 00:01:39,727][60935] Updated weights for policy 0, policy_version 74320 (0.0008) [2023-10-14 00:01:40,079][60934] Updated weights for policy 1, policy_version 74562 (0.0007) [2023-10-14 00:01:40,104][60935] Updated weights for policy 0, policy_version 74330 (0.0008) [2023-10-14 00:01:40,439][60934] Updated weights for policy 1, policy_version 74572 (0.0008) [2023-10-14 00:01:40,797][60934] Updated weights for policy 1, policy_version 74582 (0.0008) [2023-10-14 00:01:41,248][59943] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 152797184. Throughput: 0: 1689.8, 1: 1694.9. Samples: 38204264. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 00:01:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:01:44,081][60935] Updated weights for policy 0, policy_version 74340 (0.0010) [2023-10-14 00:01:44,449][60935] Updated weights for policy 0, policy_version 74350 (0.0009) [2023-10-14 00:01:44,573][60934] Updated weights for policy 1, policy_version 74592 (0.0010) [2023-10-14 00:01:44,814][60935] Updated weights for policy 0, policy_version 74360 (0.0008) [2023-10-14 00:01:44,932][60934] Updated weights for policy 1, policy_version 74602 (0.0008) [2023-10-14 00:01:45,296][60934] Updated weights for policy 1, policy_version 74612 (0.0008) [2023-10-14 00:01:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 152862720. Throughput: 0: 1717.2, 1: 1717.4. Samples: 38215892. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 00:01:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:01:48,970][60935] Updated weights for policy 0, policy_version 74370 (0.0008) [2023-10-14 00:01:49,340][60935] Updated weights for policy 0, policy_version 74380 (0.0008) [2023-10-14 00:01:49,406][60934] Updated weights for policy 1, policy_version 74622 (0.0008) [2023-10-14 00:01:49,702][60935] Updated weights for policy 0, policy_version 74390 (0.0009) [2023-10-14 00:01:49,780][60934] Updated weights for policy 1, policy_version 74632 (0.0009) [2023-10-14 00:01:50,073][60935] Updated weights for policy 0, policy_version 74400 (0.0008) [2023-10-14 00:01:50,140][60934] Updated weights for policy 1, policy_version 74642 (0.0009) [2023-10-14 00:01:51,248][59943] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 152928256. Throughput: 0: 1691.1, 1: 1706.3. Samples: 38235514. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 00:01:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:01:54,069][60935] Updated weights for policy 0, policy_version 74410 (0.0008) [2023-10-14 00:01:54,088][60934] Updated weights for policy 1, policy_version 74652 (0.0009) [2023-10-14 00:01:54,433][60935] Updated weights for policy 0, policy_version 74420 (0.0008) [2023-10-14 00:01:54,462][60934] Updated weights for policy 1, policy_version 74662 (0.0007) [2023-10-14 00:01:54,795][60935] Updated weights for policy 0, policy_version 74430 (0.0007) [2023-10-14 00:01:54,825][60934] Updated weights for policy 1, policy_version 74672 (0.0008) [2023-10-14 00:01:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 152993792. Throughput: 0: 1692.9, 1: 1684.2. Samples: 38255262. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-14 00:01:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:01:58,810][60934] Updated weights for policy 1, policy_version 74682 (0.0009) [2023-10-14 00:01:58,975][60935] Updated weights for policy 0, policy_version 74440 (0.0008) [2023-10-14 00:01:59,163][60934] Updated weights for policy 1, policy_version 74692 (0.0007) [2023-10-14 00:01:59,350][60935] Updated weights for policy 0, policy_version 74450 (0.0007) [2023-10-14 00:01:59,535][60934] Updated weights for policy 1, policy_version 74702 (0.0007) [2023-10-14 00:01:59,706][60935] Updated weights for policy 0, policy_version 74460 (0.0007) [2023-10-14 00:01:59,893][60934] Updated weights for policy 1, policy_version 74712 (0.0009) [2023-10-14 00:02:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 153059328. Throughput: 0: 1701.7, 1: 1715.2. Samples: 38266682. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-14 00:02:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:02:03,567][60935] Updated weights for policy 0, policy_version 74470 (0.0010) [2023-10-14 00:02:03,910][60934] Updated weights for policy 1, policy_version 74722 (0.0011) [2023-10-14 00:02:03,935][60935] Updated weights for policy 0, policy_version 74480 (0.0008) [2023-10-14 00:02:04,271][60934] Updated weights for policy 1, policy_version 74732 (0.0008) [2023-10-14 00:02:04,301][60935] Updated weights for policy 0, policy_version 74490 (0.0008) [2023-10-14 00:02:04,634][60934] Updated weights for policy 1, policy_version 74742 (0.0009) [2023-10-14 00:02:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 153124864. Throughput: 0: 1676.7, 1: 1685.6. Samples: 38285402. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-14 00:02:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:02:08,317][60935] Updated weights for policy 0, policy_version 74500 (0.0008) [2023-10-14 00:02:08,693][60935] Updated weights for policy 0, policy_version 74510 (0.0009) [2023-10-14 00:02:08,865][60934] Updated weights for policy 1, policy_version 74752 (0.0008) [2023-10-14 00:02:09,060][60935] Updated weights for policy 0, policy_version 74520 (0.0010) [2023-10-14 00:02:09,233][60934] Updated weights for policy 1, policy_version 74762 (0.0007) [2023-10-14 00:02:09,594][60934] Updated weights for policy 1, policy_version 74772 (0.0007) [2023-10-14 00:02:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 153190400. Throughput: 0: 1703.5, 1: 1689.8. Samples: 38306186. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-14 00:02:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:02:13,214][60935] Updated weights for policy 0, policy_version 74530 (0.0009) [2023-10-14 00:02:13,572][60934] Updated weights for policy 1, policy_version 74782 (0.0007) [2023-10-14 00:02:13,575][60935] Updated weights for policy 0, policy_version 74540 (0.0008) [2023-10-14 00:02:13,943][60935] Updated weights for policy 0, policy_version 74550 (0.0009) [2023-10-14 00:02:13,944][60934] Updated weights for policy 1, policy_version 74792 (0.0007) [2023-10-14 00:02:14,308][60934] Updated weights for policy 1, policy_version 74802 (0.0008) [2023-10-14 00:02:14,310][60935] Updated weights for policy 0, policy_version 74560 (0.0007) [2023-10-14 00:02:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 153255936. Throughput: 0: 1693.6, 1: 1704.7. Samples: 38316930. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-14 00:02:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:02:18,054][60935] Updated weights for policy 0, policy_version 74570 (0.0008) [2023-10-14 00:02:18,287][60934] Updated weights for policy 1, policy_version 74812 (0.0008) [2023-10-14 00:02:18,423][60935] Updated weights for policy 0, policy_version 74580 (0.0007) [2023-10-14 00:02:18,649][60934] Updated weights for policy 1, policy_version 74822 (0.0008) [2023-10-14 00:02:18,784][60935] Updated weights for policy 0, policy_version 74590 (0.0008) [2023-10-14 00:02:19,015][60934] Updated weights for policy 1, policy_version 74832 (0.0010) [2023-10-14 00:02:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 153321472. Throughput: 0: 1692.6, 1: 1678.4. Samples: 38336398. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-14 00:02:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:02:22,792][60935] Updated weights for policy 0, policy_version 74600 (0.0008) [2023-10-14 00:02:23,153][60935] Updated weights for policy 0, policy_version 74610 (0.0008) [2023-10-14 00:02:23,248][60934] Updated weights for policy 1, policy_version 74842 (0.0009) [2023-10-14 00:02:23,516][60935] Updated weights for policy 0, policy_version 74620 (0.0011) [2023-10-14 00:02:23,617][60934] Updated weights for policy 1, policy_version 74852 (0.0008) [2023-10-14 00:02:23,990][60934] Updated weights for policy 1, policy_version 74862 (0.0007) [2023-10-14 00:02:24,358][60934] Updated weights for policy 1, policy_version 74872 (0.0007) [2023-10-14 00:02:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 153387008. Throughput: 0: 1708.1, 1: 1685.9. Samples: 38356996. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-14 00:02:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:02:27,392][60935] Updated weights for policy 0, policy_version 74630 (0.0008) [2023-10-14 00:02:27,753][60935] Updated weights for policy 0, policy_version 74640 (0.0007) [2023-10-14 00:02:28,128][60935] Updated weights for policy 0, policy_version 74650 (0.0009) [2023-10-14 00:02:28,439][60934] Updated weights for policy 1, policy_version 74882 (0.0008) [2023-10-14 00:02:28,805][60934] Updated weights for policy 1, policy_version 74892 (0.0007) [2023-10-14 00:02:29,174][60934] Updated weights for policy 1, policy_version 74902 (0.0010) [2023-10-14 00:02:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 153452544. Throughput: 0: 1680.7, 1: 1675.0. Samples: 38366898. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-14 00:02:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:02:32,143][60935] Updated weights for policy 0, policy_version 74660 (0.0010) [2023-10-14 00:02:32,515][60935] Updated weights for policy 0, policy_version 74670 (0.0007) [2023-10-14 00:02:32,879][60935] Updated weights for policy 0, policy_version 74680 (0.0007) [2023-10-14 00:02:33,376][60934] Updated weights for policy 1, policy_version 74912 (0.0008) [2023-10-14 00:02:33,754][60934] Updated weights for policy 1, policy_version 74922 (0.0010) [2023-10-14 00:02:34,131][60934] Updated weights for policy 1, policy_version 74932 (0.0008) [2023-10-14 00:02:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 153518080. Throughput: 0: 1706.1, 1: 1664.9. Samples: 38387210. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-14 00:02:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:02:36,914][60935] Updated weights for policy 0, policy_version 74690 (0.0008) [2023-10-14 00:02:37,284][60935] Updated weights for policy 0, policy_version 74700 (0.0011) [2023-10-14 00:02:37,651][60935] Updated weights for policy 0, policy_version 74710 (0.0011) [2023-10-14 00:02:38,020][60935] Updated weights for policy 0, policy_version 74720 (0.0009) [2023-10-14 00:02:38,207][60934] Updated weights for policy 1, policy_version 74942 (0.0010) [2023-10-14 00:02:38,577][60934] Updated weights for policy 1, policy_version 74952 (0.0010) [2023-10-14 00:02:38,940][60934] Updated weights for policy 1, policy_version 74962 (0.0009) [2023-10-14 00:02:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 153583616. Throughput: 0: 1716.3, 1: 1683.7. Samples: 38408260. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-14 00:02:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:02:41,910][60935] Updated weights for policy 0, policy_version 74730 (0.0009) [2023-10-14 00:02:42,270][60935] Updated weights for policy 0, policy_version 74740 (0.0008) [2023-10-14 00:02:42,645][60935] Updated weights for policy 0, policy_version 74750 (0.0008) [2023-10-14 00:02:42,874][60934] Updated weights for policy 1, policy_version 74972 (0.0009) [2023-10-14 00:02:43,246][60934] Updated weights for policy 1, policy_version 74982 (0.0009) [2023-10-14 00:02:43,611][60934] Updated weights for policy 1, policy_version 74992 (0.0009) [2023-10-14 00:02:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 153649152. Throughput: 0: 1695.3, 1: 1666.4. Samples: 38417960. Policy #0 lag: (min: 24.0, avg: 51.6, max: 56.0) [2023-10-14 00:02:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:02:46,691][60935] Updated weights for policy 0, policy_version 74760 (0.0008) [2023-10-14 00:02:47,069][60935] Updated weights for policy 0, policy_version 74770 (0.0009) [2023-10-14 00:02:47,440][60935] Updated weights for policy 0, policy_version 74780 (0.0009) [2023-10-14 00:02:47,618][60934] Updated weights for policy 1, policy_version 75002 (0.0007) [2023-10-14 00:02:48,016][60934] Updated weights for policy 1, policy_version 75012 (0.0008) [2023-10-14 00:02:48,386][60934] Updated weights for policy 1, policy_version 75022 (0.0010) [2023-10-14 00:02:48,746][60934] Updated weights for policy 1, policy_version 75032 (0.0009) [2023-10-14 00:02:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 153714688. Throughput: 0: 1719.5, 1: 1685.8. Samples: 38438638. Policy #0 lag: (min: 24.0, avg: 51.6, max: 56.0) [2023-10-14 00:02:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:02:51,408][60935] Updated weights for policy 0, policy_version 74790 (0.0008) [2023-10-14 00:02:51,787][60935] Updated weights for policy 0, policy_version 74800 (0.0008) [2023-10-14 00:02:52,147][60935] Updated weights for policy 0, policy_version 74810 (0.0008) [2023-10-14 00:02:52,712][60934] Updated weights for policy 1, policy_version 75042 (0.0007) [2023-10-14 00:02:53,088][60934] Updated weights for policy 1, policy_version 75052 (0.0009) [2023-10-14 00:02:53,444][60934] Updated weights for policy 1, policy_version 75062 (0.0008) [2023-10-14 00:02:56,081][60935] Updated weights for policy 0, policy_version 74820 (0.0008) [2023-10-14 00:02:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13551.5). Total num frames: 153780224. Throughput: 0: 1722.5, 1: 1692.2. Samples: 38459848. Policy #0 lag: (min: 24.0, avg: 51.6, max: 56.0) [2023-10-14 00:02:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:02:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000075064_77168640.pth... [2023-10-14 00:02:56,296][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000073496_75563008.pth [2023-10-14 00:02:56,455][60935] Updated weights for policy 0, policy_version 74830 (0.0007) [2023-10-14 00:02:56,818][60935] Updated weights for policy 0, policy_version 74840 (0.0009) [2023-10-14 00:02:57,102][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000074848_76644352.pth... [2023-10-14 00:02:57,140][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000073248_75005952.pth [2023-10-14 00:02:57,451][60934] Updated weights for policy 1, policy_version 75072 (0.0009) [2023-10-14 00:02:57,817][60934] Updated weights for policy 1, policy_version 75082 (0.0008) [2023-10-14 00:02:58,187][60934] Updated weights for policy 1, policy_version 75092 (0.0009) [2023-10-14 00:03:00,885][60935] Updated weights for policy 0, policy_version 74850 (0.0009) [2023-10-14 00:03:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 153845760. Throughput: 0: 1712.5, 1: 1669.9. Samples: 38469138. Policy #0 lag: (min: 24.0, avg: 51.6, max: 56.0) [2023-10-14 00:03:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:03:01,252][60935] Updated weights for policy 0, policy_version 74860 (0.0010) [2023-10-14 00:03:01,628][60935] Updated weights for policy 0, policy_version 74870 (0.0011) [2023-10-14 00:03:01,997][60935] Updated weights for policy 0, policy_version 74880 (0.0010) [2023-10-14 00:03:02,252][60934] Updated weights for policy 1, policy_version 75102 (0.0008) [2023-10-14 00:03:02,622][60934] Updated weights for policy 1, policy_version 75112 (0.0007) [2023-10-14 00:03:02,984][60934] Updated weights for policy 1, policy_version 75122 (0.0008) [2023-10-14 00:03:05,804][60935] Updated weights for policy 0, policy_version 74890 (0.0008) [2023-10-14 00:03:06,171][60935] Updated weights for policy 0, policy_version 74900 (0.0009) [2023-10-14 00:03:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 153911296. Throughput: 0: 1728.4, 1: 1695.7. Samples: 38490484. Policy #0 lag: (min: 24.0, avg: 51.6, max: 56.0) [2023-10-14 00:03:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:03:06,535][60935] Updated weights for policy 0, policy_version 74910 (0.0008) [2023-10-14 00:03:07,021][60934] Updated weights for policy 1, policy_version 75132 (0.0008) [2023-10-14 00:03:07,388][60934] Updated weights for policy 1, policy_version 75142 (0.0011) [2023-10-14 00:03:07,765][60934] Updated weights for policy 1, policy_version 75152 (0.0010) [2023-10-14 00:03:10,476][60935] Updated weights for policy 0, policy_version 74920 (0.0010) [2023-10-14 00:03:10,851][60935] Updated weights for policy 0, policy_version 74930 (0.0009) [2023-10-14 00:03:11,226][60935] Updated weights for policy 0, policy_version 74940 (0.0007) [2023-10-14 00:03:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 153976832. Throughput: 0: 1715.4, 1: 1699.9. Samples: 38510686. Policy #0 lag: (min: 24.0, avg: 51.6, max: 56.0) [2023-10-14 00:03:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:03:11,828][60934] Updated weights for policy 1, policy_version 75162 (0.0010) [2023-10-14 00:03:12,195][60934] Updated weights for policy 1, policy_version 75172 (0.0011) [2023-10-14 00:03:12,564][60934] Updated weights for policy 1, policy_version 75182 (0.0007) [2023-10-14 00:03:12,928][60934] Updated weights for policy 1, policy_version 75192 (0.0007) [2023-10-14 00:03:15,203][60935] Updated weights for policy 0, policy_version 74950 (0.0008) [2023-10-14 00:03:15,561][60935] Updated weights for policy 0, policy_version 74960 (0.0008) [2023-10-14 00:03:15,922][60935] Updated weights for policy 0, policy_version 74970 (0.0008) [2023-10-14 00:03:16,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 154075136. Throughput: 0: 1733.5, 1: 1683.9. Samples: 38520682. Policy #0 lag: (min: 24.0, avg: 51.6, max: 56.0) [2023-10-14 00:03:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:03:16,859][60934] Updated weights for policy 1, policy_version 75202 (0.0009) [2023-10-14 00:03:17,231][60934] Updated weights for policy 1, policy_version 75212 (0.0008) [2023-10-14 00:03:17,597][60934] Updated weights for policy 1, policy_version 75222 (0.0008) [2023-10-14 00:03:19,918][60935] Updated weights for policy 0, policy_version 74980 (0.0008) [2023-10-14 00:03:20,278][60935] Updated weights for policy 0, policy_version 74990 (0.0011) [2023-10-14 00:03:20,651][60935] Updated weights for policy 0, policy_version 75000 (0.0011) [2023-10-14 00:03:21,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 154140672. Throughput: 0: 1728.7, 1: 1705.0. Samples: 38541726. Policy #0 lag: (min: 24.0, avg: 51.6, max: 56.0) [2023-10-14 00:03:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:03:21,760][60934] Updated weights for policy 1, policy_version 75232 (0.0010) [2023-10-14 00:03:22,131][60934] Updated weights for policy 1, policy_version 75242 (0.0010) [2023-10-14 00:03:22,498][60934] Updated weights for policy 1, policy_version 75252 (0.0011) [2023-10-14 00:03:24,544][60935] Updated weights for policy 0, policy_version 75010 (0.0010) [2023-10-14 00:03:24,921][60935] Updated weights for policy 0, policy_version 75020 (0.0007) [2023-10-14 00:03:25,288][60935] Updated weights for policy 0, policy_version 75030 (0.0009) [2023-10-14 00:03:25,655][60935] Updated weights for policy 0, policy_version 75040 (0.0009) [2023-10-14 00:03:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 154206208. Throughput: 0: 1703.6, 1: 1706.3. Samples: 38561702. Policy #0 lag: (min: 24.0, avg: 51.6, max: 56.0) [2023-10-14 00:03:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:03:26,651][60934] Updated weights for policy 1, policy_version 75262 (0.0010) [2023-10-14 00:03:27,015][60934] Updated weights for policy 1, policy_version 75272 (0.0008) [2023-10-14 00:03:27,381][60934] Updated weights for policy 1, policy_version 75282 (0.0009) [2023-10-14 00:03:29,540][60935] Updated weights for policy 0, policy_version 75050 (0.0008) [2023-10-14 00:03:29,913][60935] Updated weights for policy 0, policy_version 75060 (0.0007) [2023-10-14 00:03:30,278][60935] Updated weights for policy 0, policy_version 75070 (0.0009) [2023-10-14 00:03:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 154271744. Throughput: 0: 1738.1, 1: 1691.4. Samples: 38572288. Policy #0 lag: (min: 18.0, avg: 18.2, max: 27.0) [2023-10-14 00:03:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:03:31,530][60934] Updated weights for policy 1, policy_version 75292 (0.0010) [2023-10-14 00:03:31,894][60934] Updated weights for policy 1, policy_version 75302 (0.0010) [2023-10-14 00:03:32,263][60934] Updated weights for policy 1, policy_version 75312 (0.0008) [2023-10-14 00:03:34,319][60935] Updated weights for policy 0, policy_version 75080 (0.0010) [2023-10-14 00:03:34,679][60935] Updated weights for policy 0, policy_version 75090 (0.0011) [2023-10-14 00:03:35,046][60935] Updated weights for policy 0, policy_version 75100 (0.0010) [2023-10-14 00:03:36,100][60934] Updated weights for policy 1, policy_version 75322 (0.0009) [2023-10-14 00:03:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 154337280. Throughput: 0: 1723.0, 1: 1697.1. Samples: 38592540. Policy #0 lag: (min: 18.0, avg: 18.2, max: 27.0) [2023-10-14 00:03:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:03:36,527][60934] Updated weights for policy 1, policy_version 75332 (0.0007) [2023-10-14 00:03:36,891][60934] Updated weights for policy 1, policy_version 75342 (0.0008) [2023-10-14 00:03:37,250][60934] Updated weights for policy 1, policy_version 75352 (0.0008) [2023-10-14 00:03:39,069][60935] Updated weights for policy 0, policy_version 75110 (0.0008) [2023-10-14 00:03:39,439][60935] Updated weights for policy 0, policy_version 75120 (0.0009) [2023-10-14 00:03:39,820][60935] Updated weights for policy 0, policy_version 75130 (0.0007) [2023-10-14 00:03:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 154402816. Throughput: 0: 1706.6, 1: 1694.0. Samples: 38612874. Policy #0 lag: (min: 18.0, avg: 18.2, max: 27.0) [2023-10-14 00:03:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:03:41,356][60934] Updated weights for policy 1, policy_version 75362 (0.0008) [2023-10-14 00:03:41,725][60934] Updated weights for policy 1, policy_version 75372 (0.0008) [2023-10-14 00:03:42,094][60934] Updated weights for policy 1, policy_version 75382 (0.0007) [2023-10-14 00:03:43,744][60935] Updated weights for policy 0, policy_version 75140 (0.0008) [2023-10-14 00:03:44,120][60935] Updated weights for policy 0, policy_version 75150 (0.0007) [2023-10-14 00:03:44,492][60935] Updated weights for policy 0, policy_version 75160 (0.0010) [2023-10-14 00:03:46,025][60934] Updated weights for policy 1, policy_version 75392 (0.0007) [2023-10-14 00:03:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 154468352. Throughput: 0: 1733.1, 1: 1691.2. Samples: 38623232. Policy #0 lag: (min: 18.0, avg: 18.2, max: 27.0) [2023-10-14 00:03:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:03:46,399][60934] Updated weights for policy 1, policy_version 75402 (0.0009) [2023-10-14 00:03:46,763][60934] Updated weights for policy 1, policy_version 75412 (0.0010) [2023-10-14 00:03:48,565][60935] Updated weights for policy 0, policy_version 75170 (0.0010) [2023-10-14 00:03:48,935][60935] Updated weights for policy 0, policy_version 75180 (0.0009) [2023-10-14 00:03:49,300][60935] Updated weights for policy 0, policy_version 75190 (0.0007) [2023-10-14 00:03:49,672][60935] Updated weights for policy 0, policy_version 75200 (0.0009) [2023-10-14 00:03:50,862][60934] Updated weights for policy 1, policy_version 75422 (0.0011) [2023-10-14 00:03:51,225][60934] Updated weights for policy 1, policy_version 75432 (0.0009) [2023-10-14 00:03:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 154533888. Throughput: 0: 1700.1, 1: 1690.6. Samples: 38643068. Policy #0 lag: (min: 18.0, avg: 18.2, max: 27.0) [2023-10-14 00:03:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:03:51,588][60934] Updated weights for policy 1, policy_version 75442 (0.0011) [2023-10-14 00:03:53,730][60935] Updated weights for policy 0, policy_version 75210 (0.0008) [2023-10-14 00:03:54,097][60935] Updated weights for policy 0, policy_version 75220 (0.0012) [2023-10-14 00:03:54,467][60935] Updated weights for policy 0, policy_version 75230 (0.0007) [2023-10-14 00:03:55,635][60934] Updated weights for policy 1, policy_version 75452 (0.0010) [2023-10-14 00:03:55,994][60934] Updated weights for policy 1, policy_version 75462 (0.0010) [2023-10-14 00:03:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 154599424. Throughput: 0: 1714.1, 1: 1690.8. Samples: 38663908. Policy #0 lag: (min: 18.0, avg: 18.2, max: 27.0) [2023-10-14 00:03:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:03:56,354][60934] Updated weights for policy 1, policy_version 75472 (0.0009) [2023-10-14 00:03:58,540][60935] Updated weights for policy 0, policy_version 75240 (0.0008) [2023-10-14 00:03:58,901][60935] Updated weights for policy 0, policy_version 75250 (0.0008) [2023-10-14 00:03:59,277][60935] Updated weights for policy 0, policy_version 75260 (0.0007) [2023-10-14 00:04:00,357][60934] Updated weights for policy 1, policy_version 75482 (0.0010) [2023-10-14 00:04:00,714][60934] Updated weights for policy 1, policy_version 75492 (0.0009) [2023-10-14 00:04:01,084][60934] Updated weights for policy 1, policy_version 75502 (0.0011) [2023-10-14 00:04:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 154664960. Throughput: 0: 1708.8, 1: 1695.1. Samples: 38673856. Policy #0 lag: (min: 18.0, avg: 18.2, max: 27.0) [2023-10-14 00:04:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:04:01,451][60934] Updated weights for policy 1, policy_version 75512 (0.0008) [2023-10-14 00:04:03,147][60935] Updated weights for policy 0, policy_version 75270 (0.0009) [2023-10-14 00:04:03,517][60935] Updated weights for policy 0, policy_version 75280 (0.0009) [2023-10-14 00:04:03,889][60935] Updated weights for policy 0, policy_version 75290 (0.0008) [2023-10-14 00:04:05,471][60934] Updated weights for policy 1, policy_version 75522 (0.0011) [2023-10-14 00:04:05,832][60934] Updated weights for policy 1, policy_version 75532 (0.0008) [2023-10-14 00:04:06,205][60934] Updated weights for policy 1, policy_version 75542 (0.0009) [2023-10-14 00:04:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 154730496. Throughput: 0: 1693.8, 1: 1698.3. Samples: 38694372. Policy #0 lag: (min: 18.0, avg: 18.2, max: 27.0) [2023-10-14 00:04:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:04:07,723][60935] Updated weights for policy 0, policy_version 75300 (0.0008) [2023-10-14 00:04:08,095][60935] Updated weights for policy 0, policy_version 75310 (0.0009) [2023-10-14 00:04:08,463][60935] Updated weights for policy 0, policy_version 75320 (0.0009) [2023-10-14 00:04:10,190][60934] Updated weights for policy 1, policy_version 75552 (0.0010) [2023-10-14 00:04:10,556][60934] Updated weights for policy 1, policy_version 75562 (0.0010) [2023-10-14 00:04:10,924][60934] Updated weights for policy 1, policy_version 75572 (0.0010) [2023-10-14 00:04:11,248][59943] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 154828800. Throughput: 0: 1725.0, 1: 1676.4. Samples: 38714764. Policy #0 lag: (min: 18.0, avg: 18.2, max: 27.0) [2023-10-14 00:04:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:04:12,321][60935] Updated weights for policy 0, policy_version 75330 (0.0009) [2023-10-14 00:04:12,682][60935] Updated weights for policy 0, policy_version 75340 (0.0010) [2023-10-14 00:04:13,050][60935] Updated weights for policy 0, policy_version 75350 (0.0009) [2023-10-14 00:04:13,423][60935] Updated weights for policy 0, policy_version 75360 (0.0007) [2023-10-14 00:04:14,857][60934] Updated weights for policy 1, policy_version 75582 (0.0008) [2023-10-14 00:04:15,221][60934] Updated weights for policy 1, policy_version 75592 (0.0007) [2023-10-14 00:04:15,593][60934] Updated weights for policy 1, policy_version 75602 (0.0008) [2023-10-14 00:04:16,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 154894336. Throughput: 0: 1690.6, 1: 1699.1. Samples: 38724828. Policy #0 lag: (min: 18.0, avg: 18.2, max: 27.0) [2023-10-14 00:04:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:04:17,348][60935] Updated weights for policy 0, policy_version 75370 (0.0008) [2023-10-14 00:04:17,720][60935] Updated weights for policy 0, policy_version 75380 (0.0008) [2023-10-14 00:04:18,074][60935] Updated weights for policy 0, policy_version 75390 (0.0008) [2023-10-14 00:04:19,608][60934] Updated weights for policy 1, policy_version 75612 (0.0008) [2023-10-14 00:04:19,979][60934] Updated weights for policy 1, policy_version 75622 (0.0007) [2023-10-14 00:04:20,335][60934] Updated weights for policy 1, policy_version 75632 (0.0009) [2023-10-14 00:04:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 154959872. Throughput: 0: 1710.4, 1: 1701.6. Samples: 38746084. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 00:04:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:04:22,047][60935] Updated weights for policy 0, policy_version 75400 (0.0011) [2023-10-14 00:04:22,413][60935] Updated weights for policy 0, policy_version 75410 (0.0010) [2023-10-14 00:04:22,779][60935] Updated weights for policy 0, policy_version 75420 (0.0010) [2023-10-14 00:04:24,341][60934] Updated weights for policy 1, policy_version 75642 (0.0010) [2023-10-14 00:04:24,758][60934] Updated weights for policy 1, policy_version 75652 (0.0010) [2023-10-14 00:04:25,132][60934] Updated weights for policy 1, policy_version 75662 (0.0009) [2023-10-14 00:04:25,497][60934] Updated weights for policy 1, policy_version 75672 (0.0009) [2023-10-14 00:04:26,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 155025408. Throughput: 0: 1724.0, 1: 1673.6. Samples: 38765766. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 00:04:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:04:26,863][60935] Updated weights for policy 0, policy_version 75430 (0.0008) [2023-10-14 00:04:27,232][60935] Updated weights for policy 0, policy_version 75440 (0.0008) [2023-10-14 00:04:27,596][60935] Updated weights for policy 0, policy_version 75450 (0.0008) [2023-10-14 00:04:29,572][60934] Updated weights for policy 1, policy_version 75682 (0.0011) [2023-10-14 00:04:29,940][60934] Updated weights for policy 1, policy_version 75692 (0.0010) [2023-10-14 00:04:30,302][60934] Updated weights for policy 1, policy_version 75702 (0.0010) [2023-10-14 00:04:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 155090944. Throughput: 0: 1696.8, 1: 1705.1. Samples: 38776318. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 00:04:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:04:31,514][60935] Updated weights for policy 0, policy_version 75460 (0.0008) [2023-10-14 00:04:31,876][60935] Updated weights for policy 0, policy_version 75470 (0.0008) [2023-10-14 00:04:32,243][60935] Updated weights for policy 0, policy_version 75480 (0.0008) [2023-10-14 00:04:34,553][60934] Updated weights for policy 1, policy_version 75712 (0.0010) [2023-10-14 00:04:34,919][60934] Updated weights for policy 1, policy_version 75722 (0.0008) [2023-10-14 00:04:35,288][60934] Updated weights for policy 1, policy_version 75732 (0.0008) [2023-10-14 00:04:36,157][60935] Updated weights for policy 0, policy_version 75490 (0.0008) [2023-10-14 00:04:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 155156480. Throughput: 0: 1723.8, 1: 1694.0. Samples: 38796868. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 00:04:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:04:36,522][60935] Updated weights for policy 0, policy_version 75500 (0.0008) [2023-10-14 00:04:36,885][60935] Updated weights for policy 0, policy_version 75510 (0.0010) [2023-10-14 00:04:37,251][60935] Updated weights for policy 0, policy_version 75520 (0.0009) [2023-10-14 00:04:39,351][60934] Updated weights for policy 1, policy_version 75742 (0.0007) [2023-10-14 00:04:39,709][60934] Updated weights for policy 1, policy_version 75752 (0.0008) [2023-10-14 00:04:40,072][60934] Updated weights for policy 1, policy_version 75762 (0.0008) [2023-10-14 00:04:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 155222016. Throughput: 0: 1726.8, 1: 1680.4. Samples: 38817230. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 00:04:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:04:41,278][60935] Updated weights for policy 0, policy_version 75530 (0.0009) [2023-10-14 00:04:41,650][60935] Updated weights for policy 0, policy_version 75540 (0.0007) [2023-10-14 00:04:42,023][60935] Updated weights for policy 0, policy_version 75550 (0.0010) [2023-10-14 00:04:44,156][60934] Updated weights for policy 1, policy_version 75772 (0.0008) [2023-10-14 00:04:44,518][60934] Updated weights for policy 1, policy_version 75782 (0.0007) [2023-10-14 00:04:44,874][60934] Updated weights for policy 1, policy_version 75792 (0.0007) [2023-10-14 00:04:46,056][60935] Updated weights for policy 0, policy_version 75560 (0.0008) [2023-10-14 00:04:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 155287552. Throughput: 0: 1713.2, 1: 1706.0. Samples: 38827720. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 00:04:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:04:46,427][60935] Updated weights for policy 0, policy_version 75570 (0.0008) [2023-10-14 00:04:46,787][60935] Updated weights for policy 0, policy_version 75580 (0.0007) [2023-10-14 00:04:48,623][60934] Updated weights for policy 1, policy_version 75802 (0.0007) [2023-10-14 00:04:48,989][60934] Updated weights for policy 1, policy_version 75812 (0.0008) [2023-10-14 00:04:49,359][60934] Updated weights for policy 1, policy_version 75822 (0.0008) [2023-10-14 00:04:49,717][60934] Updated weights for policy 1, policy_version 75832 (0.0008) [2023-10-14 00:04:50,820][60935] Updated weights for policy 0, policy_version 75590 (0.0008) [2023-10-14 00:04:51,189][60935] Updated weights for policy 0, policy_version 75600 (0.0007) [2023-10-14 00:04:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 155353088. Throughput: 0: 1728.4, 1: 1686.5. Samples: 38848042. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 00:04:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:04:51,556][60935] Updated weights for policy 0, policy_version 75610 (0.0007) [2023-10-14 00:04:53,639][60934] Updated weights for policy 1, policy_version 75842 (0.0007) [2023-10-14 00:04:54,003][60934] Updated weights for policy 1, policy_version 75852 (0.0007) [2023-10-14 00:04:54,369][60934] Updated weights for policy 1, policy_version 75862 (0.0008) [2023-10-14 00:04:55,543][60935] Updated weights for policy 0, policy_version 75620 (0.0009) [2023-10-14 00:04:55,918][60935] Updated weights for policy 0, policy_version 75630 (0.0010) [2023-10-14 00:04:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 155418624. Throughput: 0: 1708.7, 1: 1706.9. Samples: 38868464. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 00:04:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:04:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000075864_77987840.pth... [2023-10-14 00:04:56,281][60935] Updated weights for policy 0, policy_version 75640 (0.0008) [2023-10-14 00:04:56,301][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000074296_76382208.pth [2023-10-14 00:04:56,574][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000075648_77463552.pth... [2023-10-14 00:04:56,613][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000074048_75825152.pth [2023-10-14 00:04:58,373][60934] Updated weights for policy 1, policy_version 75872 (0.0007) [2023-10-14 00:04:58,741][60934] Updated weights for policy 1, policy_version 75882 (0.0007) [2023-10-14 00:04:59,105][60934] Updated weights for policy 1, policy_version 75892 (0.0008) [2023-10-14 00:05:00,272][60935] Updated weights for policy 0, policy_version 75650 (0.0009) [2023-10-14 00:05:00,629][60935] Updated weights for policy 0, policy_version 75660 (0.0009) [2023-10-14 00:05:01,008][60935] Updated weights for policy 0, policy_version 75670 (0.0009) [2023-10-14 00:05:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 155484160. Throughput: 0: 1723.4, 1: 1707.9. Samples: 38879236. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 00:05:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:05:01,380][60935] Updated weights for policy 0, policy_version 75680 (0.0007) [2023-10-14 00:05:02,955][60934] Updated weights for policy 1, policy_version 75902 (0.0008) [2023-10-14 00:05:03,324][60934] Updated weights for policy 1, policy_version 75912 (0.0008) [2023-10-14 00:05:03,690][60934] Updated weights for policy 1, policy_version 75922 (0.0008) [2023-10-14 00:05:05,475][60935] Updated weights for policy 0, policy_version 75690 (0.0009) [2023-10-14 00:05:05,846][60935] Updated weights for policy 0, policy_version 75700 (0.0009) [2023-10-14 00:05:06,213][60935] Updated weights for policy 0, policy_version 75710 (0.0009) [2023-10-14 00:05:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 155549696. Throughput: 0: 1725.8, 1: 1691.3. Samples: 38899856. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 00:05:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:05:07,547][60934] Updated weights for policy 1, policy_version 75932 (0.0008) [2023-10-14 00:05:07,917][60934] Updated weights for policy 1, policy_version 75942 (0.0009) [2023-10-14 00:05:08,283][60934] Updated weights for policy 1, policy_version 75952 (0.0009) [2023-10-14 00:05:10,193][60935] Updated weights for policy 0, policy_version 75720 (0.0009) [2023-10-14 00:05:10,556][60935] Updated weights for policy 0, policy_version 75730 (0.0008) [2023-10-14 00:05:10,927][60935] Updated weights for policy 0, policy_version 75740 (0.0007) [2023-10-14 00:05:11,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 155648000. Throughput: 0: 1704.2, 1: 1724.2. Samples: 38920044. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-14 00:05:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:05:12,430][60934] Updated weights for policy 1, policy_version 75962 (0.0008) [2023-10-14 00:05:12,832][60934] Updated weights for policy 1, policy_version 75972 (0.0008) [2023-10-14 00:05:13,197][60934] Updated weights for policy 1, policy_version 75982 (0.0008) [2023-10-14 00:05:13,566][60934] Updated weights for policy 1, policy_version 75992 (0.0008) [2023-10-14 00:05:14,741][60935] Updated weights for policy 0, policy_version 75750 (0.0009) [2023-10-14 00:05:15,115][60935] Updated weights for policy 0, policy_version 75760 (0.0011) [2023-10-14 00:05:15,485][60935] Updated weights for policy 0, policy_version 75770 (0.0008) [2023-10-14 00:05:16,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 155713536. Throughput: 0: 1727.9, 1: 1691.2. Samples: 38930178. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-14 00:05:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:05:17,591][60934] Updated weights for policy 1, policy_version 76002 (0.0007) [2023-10-14 00:05:17,945][60934] Updated weights for policy 1, policy_version 76012 (0.0008) [2023-10-14 00:05:18,314][60934] Updated weights for policy 1, policy_version 76022 (0.0008) [2023-10-14 00:05:19,490][60935] Updated weights for policy 0, policy_version 75780 (0.0008) [2023-10-14 00:05:19,865][60935] Updated weights for policy 0, policy_version 75790 (0.0009) [2023-10-14 00:05:20,221][60935] Updated weights for policy 0, policy_version 75800 (0.0009) [2023-10-14 00:05:21,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 155779072. Throughput: 0: 1717.4, 1: 1705.8. Samples: 38950914. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-14 00:05:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:05:22,190][60934] Updated weights for policy 1, policy_version 76032 (0.0008) [2023-10-14 00:05:22,553][60934] Updated weights for policy 1, policy_version 76042 (0.0010) [2023-10-14 00:05:22,917][60934] Updated weights for policy 1, policy_version 76052 (0.0009) [2023-10-14 00:05:24,257][60935] Updated weights for policy 0, policy_version 75810 (0.0010) [2023-10-14 00:05:24,637][60935] Updated weights for policy 0, policy_version 75820 (0.0007) [2023-10-14 00:05:25,003][60935] Updated weights for policy 0, policy_version 75830 (0.0008) [2023-10-14 00:05:25,366][60935] Updated weights for policy 0, policy_version 75840 (0.0009) [2023-10-14 00:05:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 155844608. Throughput: 0: 1698.0, 1: 1729.6. Samples: 38971470. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-14 00:05:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:05:26,998][60934] Updated weights for policy 1, policy_version 76062 (0.0008) [2023-10-14 00:05:27,361][60934] Updated weights for policy 1, policy_version 76072 (0.0009) [2023-10-14 00:05:27,720][60934] Updated weights for policy 1, policy_version 76082 (0.0009) [2023-10-14 00:05:29,300][60935] Updated weights for policy 0, policy_version 75850 (0.0007) [2023-10-14 00:05:29,664][60935] Updated weights for policy 0, policy_version 75860 (0.0009) [2023-10-14 00:05:30,024][60935] Updated weights for policy 0, policy_version 75870 (0.0008) [2023-10-14 00:05:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 155910144. Throughput: 0: 1729.2, 1: 1701.2. Samples: 38982092. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-14 00:05:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:05:31,712][60934] Updated weights for policy 1, policy_version 76092 (0.0009) [2023-10-14 00:05:32,082][60934] Updated weights for policy 1, policy_version 76102 (0.0009) [2023-10-14 00:05:32,443][60934] Updated weights for policy 1, policy_version 76112 (0.0008) [2023-10-14 00:05:33,956][60935] Updated weights for policy 0, policy_version 75880 (0.0009) [2023-10-14 00:05:34,325][60935] Updated weights for policy 0, policy_version 75890 (0.0010) [2023-10-14 00:05:34,698][60935] Updated weights for policy 0, policy_version 75900 (0.0011) [2023-10-14 00:05:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 155975680. Throughput: 0: 1702.4, 1: 1722.0. Samples: 39002140. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-14 00:05:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:05:36,332][60934] Updated weights for policy 1, policy_version 76122 (0.0007) [2023-10-14 00:05:36,689][60934] Updated weights for policy 1, policy_version 76132 (0.0008) [2023-10-14 00:05:37,054][60934] Updated weights for policy 1, policy_version 76142 (0.0009) [2023-10-14 00:05:37,420][60828] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000008 [2023-10-14 00:05:37,424][60934] Updated weights for policy 1, policy_version 76152 (0.0009) [2023-10-14 00:05:38,692][60935] Updated weights for policy 0, policy_version 75910 (0.0011) [2023-10-14 00:05:39,058][60935] Updated weights for policy 0, policy_version 75920 (0.0007) [2023-10-14 00:05:39,435][60935] Updated weights for policy 0, policy_version 75930 (0.0007) [2023-10-14 00:05:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 156041216. Throughput: 0: 1711.7, 1: 1724.8. Samples: 39023108. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-14 00:05:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:05:41,412][60934] Updated weights for policy 1, policy_version 76162 (0.0007) [2023-10-14 00:05:41,784][60934] Updated weights for policy 1, policy_version 76172 (0.0010) [2023-10-14 00:05:41,920][60828] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000005 [2023-10-14 00:05:43,500][60935] Updated weights for policy 0, policy_version 75940 (0.0007) [2023-10-14 00:05:43,870][60935] Updated weights for policy 0, policy_version 75950 (0.0009) [2023-10-14 00:05:44,245][60935] Updated weights for policy 0, policy_version 75960 (0.0009) [2023-10-14 00:05:45,861][60934] Updated weights for policy 1, policy_version 76182 (0.0009) [2023-10-14 00:05:46,229][60934] Updated weights for policy 1, policy_version 76192 (0.0009) [2023-10-14 00:05:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 156106752. Throughput: 0: 1720.4, 1: 1713.6. Samples: 39033766. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-14 00:05:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:05:46,589][60934] Updated weights for policy 1, policy_version 76202 (0.0009) [2023-10-14 00:05:48,215][60935] Updated weights for policy 0, policy_version 75970 (0.0008) [2023-10-14 00:05:48,572][60935] Updated weights for policy 0, policy_version 75980 (0.0011) [2023-10-14 00:05:48,941][60935] Updated weights for policy 0, policy_version 75990 (0.0009) [2023-10-14 00:05:49,298][60935] Updated weights for policy 0, policy_version 76000 (0.0007) [2023-10-14 00:05:50,440][60934] Updated weights for policy 1, policy_version 76212 (0.0007) [2023-10-14 00:05:50,803][60934] Updated weights for policy 1, policy_version 76222 (0.0007) [2023-10-14 00:05:50,868][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:05:51,248][59943] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 156205056. Throughput: 0: 1699.2, 1: 1731.4. Samples: 39054232. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-14 00:05:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:05:53,039][60935] Updated weights for policy 0, policy_version 76010 (0.0010) [2023-10-14 00:05:53,403][60935] Updated weights for policy 0, policy_version 76020 (0.0009) [2023-10-14 00:05:53,765][60935] Updated weights for policy 0, policy_version 76030 (0.0011) [2023-10-14 00:05:55,098][60934] Updated weights for policy 1, policy_version 76232 (0.0008) [2023-10-14 00:05:55,387][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-14 00:05:56,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 156270592. Throughput: 0: 1723.7, 1: 1735.1. Samples: 39075690. Policy #0 lag: (min: 30.0, avg: 30.4, max: 45.0) [2023-10-14 00:05:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:05:57,845][60935] Updated weights for policy 0, policy_version 76040 (0.0010) [2023-10-14 00:05:58,223][60935] Updated weights for policy 0, policy_version 76050 (0.0011) [2023-10-14 00:05:58,589][60935] Updated weights for policy 0, policy_version 76060 (0.0011) [2023-10-14 00:05:59,407][60934] Updated weights for policy 1, policy_version 76242 (0.0010) [2023-10-14 00:05:59,818][60934] Updated weights for policy 1, policy_version 76252 (0.0007) [2023-10-14 00:05:59,964][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-14 00:06:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 156336128. Throughput: 0: 1696.0, 1: 1771.0. Samples: 39086190. Policy #0 lag: (min: 30.0, avg: 30.4, max: 45.0) [2023-10-14 00:06:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:06:02,485][60935] Updated weights for policy 0, policy_version 76070 (0.0010) [2023-10-14 00:06:02,850][60935] Updated weights for policy 0, policy_version 76080 (0.0009) [2023-10-14 00:06:03,224][60935] Updated weights for policy 0, policy_version 76090 (0.0009) [2023-10-14 00:06:04,067][60934] Updated weights for policy 1, policy_version 76262 (0.0009) [2023-10-14 00:06:04,420][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000006 [2023-10-14 00:06:04,426][60934] Updated weights for policy 1, policy_version 76272 (0.0008) [2023-10-14 00:06:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 156401664. Throughput: 0: 1708.4, 1: 1765.5. Samples: 39107238. Policy #0 lag: (min: 30.0, avg: 30.4, max: 45.0) [2023-10-14 00:06:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:06:07,219][60935] Updated weights for policy 0, policy_version 76100 (0.0010) [2023-10-14 00:06:07,591][60935] Updated weights for policy 0, policy_version 76110 (0.0009) [2023-10-14 00:06:07,962][60935] Updated weights for policy 0, policy_version 76120 (0.0008) [2023-10-14 00:06:08,757][60934] Updated weights for policy 1, policy_version 76282 (0.0008) [2023-10-14 00:06:08,970][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-14 00:06:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 156467200. Throughput: 0: 1726.3, 1: 1775.4. Samples: 39129046. Policy #0 lag: (min: 30.0, avg: 30.4, max: 45.0) [2023-10-14 00:06:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '-0.120')] [2023-10-14 00:06:11,890][60935] Updated weights for policy 0, policy_version 76130 (0.0008) [2023-10-14 00:06:12,267][60935] Updated weights for policy 0, policy_version 76140 (0.0008) [2023-10-14 00:06:12,639][60935] Updated weights for policy 0, policy_version 76150 (0.0009) [2023-10-14 00:06:13,003][60934] Updated weights for policy 1, policy_version 76292 (0.0007) [2023-10-14 00:06:13,011][60935] Updated weights for policy 0, policy_version 76160 (0.0007) [2023-10-14 00:06:13,374][60934] Updated weights for policy 1, policy_version 76302 (0.0007) [2023-10-14 00:06:13,740][60934] Updated weights for policy 1, policy_version 76312 (0.0010) [2023-10-14 00:06:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 156532736. Throughput: 0: 1696.7, 1: 1791.7. Samples: 39139070. Policy #0 lag: (min: 30.0, avg: 30.4, max: 45.0) [2023-10-14 00:06:16,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:06:16,923][60935] Updated weights for policy 0, policy_version 76170 (0.0009) [2023-10-14 00:06:17,286][60935] Updated weights for policy 0, policy_version 76180 (0.0009) [2023-10-14 00:06:17,664][60935] Updated weights for policy 0, policy_version 76190 (0.0011) [2023-10-14 00:06:17,716][60934] Updated weights for policy 1, policy_version 76322 (0.0009) [2023-10-14 00:06:18,073][60934] Updated weights for policy 1, policy_version 76332 (0.0008) [2023-10-14 00:06:18,444][60934] Updated weights for policy 1, policy_version 76342 (0.0007) [2023-10-14 00:06:18,806][60934] Updated weights for policy 1, policy_version 76352 (0.0007) [2023-10-14 00:06:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 156598272. Throughput: 0: 1728.0, 1: 1774.9. Samples: 39159772. Policy #0 lag: (min: 30.0, avg: 30.4, max: 45.0) [2023-10-14 00:06:21,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:06:21,616][60935] Updated weights for policy 0, policy_version 76200 (0.0009) [2023-10-14 00:06:21,992][60935] Updated weights for policy 0, policy_version 76210 (0.0010) [2023-10-14 00:06:22,351][60935] Updated weights for policy 0, policy_version 76220 (0.0010) [2023-10-14 00:06:22,825][60934] Updated weights for policy 1, policy_version 76362 (0.0009) [2023-10-14 00:06:23,181][60934] Updated weights for policy 1, policy_version 76372 (0.0007) [2023-10-14 00:06:23,549][60934] Updated weights for policy 1, policy_version 76382 (0.0007) [2023-10-14 00:06:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 156663808. Throughput: 0: 1732.8, 1: 1775.3. Samples: 39180974. Policy #0 lag: (min: 30.0, avg: 30.4, max: 45.0) [2023-10-14 00:06:26,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:06:26,351][60935] Updated weights for policy 0, policy_version 76230 (0.0009) [2023-10-14 00:06:26,718][60935] Updated weights for policy 0, policy_version 76240 (0.0008) [2023-10-14 00:06:27,087][60935] Updated weights for policy 0, policy_version 76250 (0.0008) [2023-10-14 00:06:27,440][60934] Updated weights for policy 1, policy_version 76392 (0.0009) [2023-10-14 00:06:27,796][60934] Updated weights for policy 1, policy_version 76402 (0.0009) [2023-10-14 00:06:28,159][60934] Updated weights for policy 1, policy_version 76412 (0.0009) [2023-10-14 00:06:31,022][60935] Updated weights for policy 0, policy_version 76260 (0.0008) [2023-10-14 00:06:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 156729344. Throughput: 0: 1711.9, 1: 1767.6. Samples: 39190344. Policy #0 lag: (min: 30.0, avg: 30.4, max: 45.0) [2023-10-14 00:06:31,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:06:31,384][60935] Updated weights for policy 0, policy_version 76270 (0.0010) [2023-10-14 00:06:31,755][60935] Updated weights for policy 0, policy_version 76280 (0.0008) [2023-10-14 00:06:31,995][60934] Updated weights for policy 1, policy_version 76422 (0.0008) [2023-10-14 00:06:32,364][60934] Updated weights for policy 1, policy_version 76432 (0.0008) [2023-10-14 00:06:32,735][60934] Updated weights for policy 1, policy_version 76442 (0.0007) [2023-10-14 00:06:35,846][60935] Updated weights for policy 0, policy_version 76290 (0.0008) [2023-10-14 00:06:36,211][60935] Updated weights for policy 0, policy_version 76300 (0.0011) [2023-10-14 00:06:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 156794880. Throughput: 0: 1727.6, 1: 1773.8. Samples: 39211794. Policy #0 lag: (min: 30.0, avg: 30.4, max: 45.0) [2023-10-14 00:06:36,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:06:36,580][60935] Updated weights for policy 0, policy_version 76310 (0.0010) [2023-10-14 00:06:36,803][60934] Updated weights for policy 1, policy_version 76452 (0.0008) [2023-10-14 00:06:36,947][60935] Updated weights for policy 0, policy_version 76320 (0.0009) [2023-10-14 00:06:37,171][60934] Updated weights for policy 1, policy_version 76462 (0.0007) [2023-10-14 00:06:37,537][60934] Updated weights for policy 1, policy_version 76472 (0.0009) [2023-10-14 00:06:37,825][60828] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000003 [2023-10-14 00:06:40,820][60935] Updated weights for policy 0, policy_version 76330 (0.0009) [2023-10-14 00:06:41,197][60935] Updated weights for policy 0, policy_version 76340 (0.0008) [2023-10-14 00:06:41,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 156860416. Throughput: 0: 1716.3, 1: 1766.4. Samples: 39232410. Policy #0 lag: (min: 30.0, avg: 30.4, max: 45.0) [2023-10-14 00:06:41,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:06:41,485][60934] Updated weights for policy 1, policy_version 76482 (0.0010) [2023-10-14 00:06:41,567][60935] Updated weights for policy 0, policy_version 76350 (0.0008) [2023-10-14 00:06:41,853][60934] Updated weights for policy 1, policy_version 76492 (0.0010) [2023-10-14 00:06:42,211][60934] Updated weights for policy 1, policy_version 76502 (0.0007) [2023-10-14 00:06:42,582][60934] Updated weights for policy 1, policy_version 76512 (0.0009) [2023-10-14 00:06:45,568][60935] Updated weights for policy 0, policy_version 76360 (0.0009) [2023-10-14 00:06:45,943][60935] Updated weights for policy 0, policy_version 76370 (0.0007) [2023-10-14 00:06:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 156925952. Throughput: 0: 1731.9, 1: 1737.6. Samples: 39242314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:06:46,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:06:46,311][60935] Updated weights for policy 0, policy_version 76380 (0.0007) [2023-10-14 00:06:46,494][60934] Updated weights for policy 1, policy_version 76522 (0.0009) [2023-10-14 00:06:46,870][60934] Updated weights for policy 1, policy_version 76532 (0.0007) [2023-10-14 00:06:47,227][60934] Updated weights for policy 1, policy_version 76542 (0.0007) [2023-10-14 00:06:50,345][60935] Updated weights for policy 0, policy_version 76390 (0.0010) [2023-10-14 00:06:50,721][60935] Updated weights for policy 0, policy_version 76400 (0.0010) [2023-10-14 00:06:50,964][60934] Updated weights for policy 1, policy_version 76552 (0.0007) [2023-10-14 00:06:51,082][60935] Updated weights for policy 0, policy_version 76410 (0.0010) [2023-10-14 00:06:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 156991488. Throughput: 0: 1726.7, 1: 1745.6. Samples: 39263492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:06:51,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:06:51,332][60934] Updated weights for policy 1, policy_version 76562 (0.0007) [2023-10-14 00:06:51,691][60934] Updated weights for policy 1, policy_version 76572 (0.0007) [2023-10-14 00:06:55,156][60935] Updated weights for policy 0, policy_version 76420 (0.0008) [2023-10-14 00:06:55,527][60935] Updated weights for policy 0, policy_version 76430 (0.0008) [2023-10-14 00:06:55,896][60935] Updated weights for policy 0, policy_version 76440 (0.0007) [2023-10-14 00:06:55,926][60934] Updated weights for policy 1, policy_version 76582 (0.0008) [2023-10-14 00:06:56,249][59943] Fps is (10 sec: 16383.0, 60 sec: 13653.2, 300 sec: 13662.6). Total num frames: 157089792. Throughput: 0: 1706.8, 1: 1733.2. Samples: 39283850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:06:56,250][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:06:56,260][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000076448_78282752.pth... [2023-10-14 00:06:56,290][60934] Updated weights for policy 1, policy_version 76592 (0.0007) [2023-10-14 00:06:56,297][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000074848_76644352.pth [2023-10-14 00:06:56,660][60934] Updated weights for policy 1, policy_version 76602 (0.0007) [2023-10-14 00:06:56,876][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000076608_78839808.pth... [2023-10-14 00:06:56,916][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000075064_77168640.pth [2023-10-14 00:06:59,894][60935] Updated weights for policy 0, policy_version 76450 (0.0009) [2023-10-14 00:07:00,258][60935] Updated weights for policy 0, policy_version 76460 (0.0009) [2023-10-14 00:07:00,620][60934] Updated weights for policy 1, policy_version 76612 (0.0009) [2023-10-14 00:07:00,626][60935] Updated weights for policy 0, policy_version 76470 (0.0012) [2023-10-14 00:07:00,984][60934] Updated weights for policy 1, policy_version 76622 (0.0007) [2023-10-14 00:07:00,991][60935] Updated weights for policy 0, policy_version 76480 (0.0010) [2023-10-14 00:07:01,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 157155328. Throughput: 0: 1722.8, 1: 1713.6. Samples: 39293712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:07:01,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:07:01,362][60934] Updated weights for policy 1, policy_version 76632 (0.0008) [2023-10-14 00:07:04,862][60935] Updated weights for policy 0, policy_version 76490 (0.0007) [2023-10-14 00:07:05,220][60935] Updated weights for policy 0, policy_version 76500 (0.0007) [2023-10-14 00:07:05,257][60934] Updated weights for policy 1, policy_version 76642 (0.0009) [2023-10-14 00:07:05,584][60935] Updated weights for policy 0, policy_version 76510 (0.0009) [2023-10-14 00:07:05,619][60934] Updated weights for policy 1, policy_version 76652 (0.0008) [2023-10-14 00:07:05,996][60934] Updated weights for policy 1, policy_version 76662 (0.0010) [2023-10-14 00:07:06,248][59943] Fps is (10 sec: 13108.0, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 157220864. Throughput: 0: 1718.9, 1: 1728.5. Samples: 39314904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:07:06,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:07:06,374][60934] Updated weights for policy 1, policy_version 76672 (0.0009) [2023-10-14 00:07:09,442][60935] Updated weights for policy 0, policy_version 76520 (0.0008) [2023-10-14 00:07:09,811][60935] Updated weights for policy 0, policy_version 76530 (0.0008) [2023-10-14 00:07:10,179][60935] Updated weights for policy 0, policy_version 76540 (0.0010) [2023-10-14 00:07:10,319][60934] Updated weights for policy 1, policy_version 76682 (0.0009) [2023-10-14 00:07:10,684][60934] Updated weights for policy 1, policy_version 76692 (0.0009) [2023-10-14 00:07:11,043][60934] Updated weights for policy 1, policy_version 76702 (0.0009) [2023-10-14 00:07:11,248][59943] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 157319168. Throughput: 0: 1698.3, 1: 1716.7. Samples: 39334646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:07:11,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:07:14,043][60935] Updated weights for policy 0, policy_version 76550 (0.0008) [2023-10-14 00:07:14,412][60935] Updated weights for policy 0, policy_version 76560 (0.0009) [2023-10-14 00:07:14,779][60935] Updated weights for policy 0, policy_version 76570 (0.0008) [2023-10-14 00:07:14,936][60934] Updated weights for policy 1, policy_version 76712 (0.0008) [2023-10-14 00:07:15,297][60934] Updated weights for policy 1, policy_version 76722 (0.0008) [2023-10-14 00:07:15,668][60934] Updated weights for policy 1, policy_version 76732 (0.0010) [2023-10-14 00:07:16,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 157384704. Throughput: 0: 1726.1, 1: 1730.6. Samples: 39345898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:07:16,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:07:18,828][60935] Updated weights for policy 0, policy_version 76580 (0.0008) [2023-10-14 00:07:19,203][60935] Updated weights for policy 0, policy_version 76590 (0.0009) [2023-10-14 00:07:19,541][60934] Updated weights for policy 1, policy_version 76742 (0.0009) [2023-10-14 00:07:19,564][60935] Updated weights for policy 0, policy_version 76600 (0.0007) [2023-10-14 00:07:19,921][60934] Updated weights for policy 1, policy_version 76752 (0.0009) [2023-10-14 00:07:20,285][60934] Updated weights for policy 1, policy_version 76762 (0.0010) [2023-10-14 00:07:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 157450240. Throughput: 0: 1699.5, 1: 1722.1. Samples: 39365768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:07:21,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:07:23,546][60935] Updated weights for policy 0, policy_version 76610 (0.0008) [2023-10-14 00:07:23,916][60935] Updated weights for policy 0, policy_version 76620 (0.0011) [2023-10-14 00:07:24,285][60935] Updated weights for policy 0, policy_version 76630 (0.0008) [2023-10-14 00:07:24,364][60934] Updated weights for policy 1, policy_version 76772 (0.0009) [2023-10-14 00:07:24,653][60935] Updated weights for policy 0, policy_version 76640 (0.0010) [2023-10-14 00:07:24,737][60934] Updated weights for policy 1, policy_version 76782 (0.0009) [2023-10-14 00:07:25,090][60934] Updated weights for policy 1, policy_version 76792 (0.0010) [2023-10-14 00:07:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 157515776. Throughput: 0: 1701.6, 1: 1695.8. Samples: 39385296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:07:26,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:07:28,621][60935] Updated weights for policy 0, policy_version 76650 (0.0010) [2023-10-14 00:07:28,983][60935] Updated weights for policy 0, policy_version 76660 (0.0010) [2023-10-14 00:07:29,298][60934] Updated weights for policy 1, policy_version 76802 (0.0007) [2023-10-14 00:07:29,347][60935] Updated weights for policy 0, policy_version 76670 (0.0007) [2023-10-14 00:07:29,654][60934] Updated weights for policy 1, policy_version 76812 (0.0008) [2023-10-14 00:07:30,022][60934] Updated weights for policy 1, policy_version 76822 (0.0007) [2023-10-14 00:07:30,388][60934] Updated weights for policy 1, policy_version 76832 (0.0008) [2023-10-14 00:07:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 157581312. Throughput: 0: 1706.4, 1: 1718.8. Samples: 39396450. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-14 00:07:31,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:07:33,406][60935] Updated weights for policy 0, policy_version 76680 (0.0008) [2023-10-14 00:07:33,779][60935] Updated weights for policy 0, policy_version 76690 (0.0011) [2023-10-14 00:07:34,139][60935] Updated weights for policy 0, policy_version 76700 (0.0008) [2023-10-14 00:07:34,344][60934] Updated weights for policy 1, policy_version 76842 (0.0008) [2023-10-14 00:07:34,711][60934] Updated weights for policy 1, policy_version 76852 (0.0007) [2023-10-14 00:07:35,077][60934] Updated weights for policy 1, policy_version 76862 (0.0010) [2023-10-14 00:07:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 157646848. Throughput: 0: 1692.4, 1: 1702.8. Samples: 39416278. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-14 00:07:36,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:07:38,388][60935] Updated weights for policy 0, policy_version 76710 (0.0007) [2023-10-14 00:07:38,783][60935] Updated weights for policy 0, policy_version 76720 (0.0009) [2023-10-14 00:07:39,007][60934] Updated weights for policy 1, policy_version 76872 (0.0007) [2023-10-14 00:07:39,141][60935] Updated weights for policy 0, policy_version 76730 (0.0007) [2023-10-14 00:07:39,370][60934] Updated weights for policy 1, policy_version 76882 (0.0007) [2023-10-14 00:07:39,736][60934] Updated weights for policy 1, policy_version 76892 (0.0008) [2023-10-14 00:07:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 157712384. Throughput: 0: 1703.8, 1: 1688.6. Samples: 39436510. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-14 00:07:41,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:07:43,172][60935] Updated weights for policy 0, policy_version 76740 (0.0009) [2023-10-14 00:07:43,533][60935] Updated weights for policy 0, policy_version 76750 (0.0009) [2023-10-14 00:07:43,546][60934] Updated weights for policy 1, policy_version 76902 (0.0008) [2023-10-14 00:07:43,895][60935] Updated weights for policy 0, policy_version 76760 (0.0009) [2023-10-14 00:07:43,921][60934] Updated weights for policy 1, policy_version 76912 (0.0007) [2023-10-14 00:07:44,280][60934] Updated weights for policy 1, policy_version 76922 (0.0008) [2023-10-14 00:07:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 157777920. Throughput: 0: 1696.3, 1: 1719.2. Samples: 39447408. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-14 00:07:46,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:07:47,891][60935] Updated weights for policy 0, policy_version 76770 (0.0009) [2023-10-14 00:07:48,253][60935] Updated weights for policy 0, policy_version 76780 (0.0009) [2023-10-14 00:07:48,343][60934] Updated weights for policy 1, policy_version 76932 (0.0009) [2023-10-14 00:07:48,627][60935] Updated weights for policy 0, policy_version 76790 (0.0007) [2023-10-14 00:07:48,716][60934] Updated weights for policy 1, policy_version 76942 (0.0007) [2023-10-14 00:07:48,989][60935] Updated weights for policy 0, policy_version 76800 (0.0008) [2023-10-14 00:07:49,085][60934] Updated weights for policy 1, policy_version 76952 (0.0007) [2023-10-14 00:07:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 157843456. Throughput: 0: 1687.6, 1: 1694.2. Samples: 39467084. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-14 00:07:51,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:07:52,836][60935] Updated weights for policy 0, policy_version 76810 (0.0009) [2023-10-14 00:07:53,201][60935] Updated weights for policy 0, policy_version 76820 (0.0008) [2023-10-14 00:07:53,235][60934] Updated weights for policy 1, policy_version 76962 (0.0008) [2023-10-14 00:07:53,566][60935] Updated weights for policy 0, policy_version 76830 (0.0008) [2023-10-14 00:07:53,604][60934] Updated weights for policy 1, policy_version 76972 (0.0009) [2023-10-14 00:07:53,969][60934] Updated weights for policy 1, policy_version 76982 (0.0008) [2023-10-14 00:07:54,327][60934] Updated weights for policy 1, policy_version 76992 (0.0008) [2023-10-14 00:07:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 157908992. Throughput: 0: 1705.2, 1: 1703.2. Samples: 39488022. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-14 00:07:56,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:07:57,676][60935] Updated weights for policy 0, policy_version 76840 (0.0008) [2023-10-14 00:07:58,043][60935] Updated weights for policy 0, policy_version 76850 (0.0008) [2023-10-14 00:07:58,236][60934] Updated weights for policy 1, policy_version 77002 (0.0008) [2023-10-14 00:07:58,418][60935] Updated weights for policy 0, policy_version 76860 (0.0008) [2023-10-14 00:07:58,600][60934] Updated weights for policy 1, policy_version 77012 (0.0010) [2023-10-14 00:07:58,954][60934] Updated weights for policy 1, policy_version 77022 (0.0008) [2023-10-14 00:08:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 157974528. Throughput: 0: 1675.1, 1: 1701.3. Samples: 39497838. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-14 00:08:01,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:08:02,310][60935] Updated weights for policy 0, policy_version 76870 (0.0009) [2023-10-14 00:08:02,683][60935] Updated weights for policy 0, policy_version 76880 (0.0009) [2023-10-14 00:08:02,961][60934] Updated weights for policy 1, policy_version 77032 (0.0007) [2023-10-14 00:08:03,048][60935] Updated weights for policy 0, policy_version 76890 (0.0007) [2023-10-14 00:08:03,330][60934] Updated weights for policy 1, policy_version 77042 (0.0007) [2023-10-14 00:08:03,713][60934] Updated weights for policy 1, policy_version 77052 (0.0010) [2023-10-14 00:08:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 158040064. Throughput: 0: 1704.0, 1: 1689.5. Samples: 39518476. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-14 00:08:06,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:08:07,031][60935] Updated weights for policy 0, policy_version 76900 (0.0009) [2023-10-14 00:08:07,398][60935] Updated weights for policy 0, policy_version 76910 (0.0009) [2023-10-14 00:08:07,608][60934] Updated weights for policy 1, policy_version 77062 (0.0007) [2023-10-14 00:08:07,768][60935] Updated weights for policy 0, policy_version 76920 (0.0007) [2023-10-14 00:08:07,970][60934] Updated weights for policy 1, policy_version 77072 (0.0008) [2023-10-14 00:08:08,333][60934] Updated weights for policy 1, policy_version 77082 (0.0007) [2023-10-14 00:08:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 158105600. Throughput: 0: 1710.4, 1: 1726.5. Samples: 39539954. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-14 00:08:11,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:08:11,807][60935] Updated weights for policy 0, policy_version 76930 (0.0007) [2023-10-14 00:08:12,177][60935] Updated weights for policy 0, policy_version 76940 (0.0011) [2023-10-14 00:08:12,343][60934] Updated weights for policy 1, policy_version 77092 (0.0010) [2023-10-14 00:08:12,545][60935] Updated weights for policy 0, policy_version 76950 (0.0007) [2023-10-14 00:08:12,705][60934] Updated weights for policy 1, policy_version 77102 (0.0007) [2023-10-14 00:08:12,905][60935] Updated weights for policy 0, policy_version 76960 (0.0008) [2023-10-14 00:08:13,074][60934] Updated weights for policy 1, policy_version 77112 (0.0007) [2023-10-14 00:08:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 158171136. Throughput: 0: 1694.2, 1: 1701.6. Samples: 39549260. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-14 00:08:16,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:08:16,769][60935] Updated weights for policy 0, policy_version 76970 (0.0008) [2023-10-14 00:08:16,879][60934] Updated weights for policy 1, policy_version 77122 (0.0007) [2023-10-14 00:08:17,139][60935] Updated weights for policy 0, policy_version 76980 (0.0009) [2023-10-14 00:08:17,243][60934] Updated weights for policy 1, policy_version 77132 (0.0008) [2023-10-14 00:08:17,508][60935] Updated weights for policy 0, policy_version 76990 (0.0009) [2023-10-14 00:08:17,604][60934] Updated weights for policy 1, policy_version 77142 (0.0009) [2023-10-14 00:08:17,977][60934] Updated weights for policy 1, policy_version 77152 (0.0007) [2023-10-14 00:08:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 158236672. Throughput: 0: 1708.1, 1: 1719.9. Samples: 39570538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:08:21,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:08:21,607][60935] Updated weights for policy 0, policy_version 77000 (0.0009) [2023-10-14 00:08:21,971][60935] Updated weights for policy 0, policy_version 77010 (0.0009) [2023-10-14 00:08:22,079][60934] Updated weights for policy 1, policy_version 77162 (0.0008) [2023-10-14 00:08:22,344][60935] Updated weights for policy 0, policy_version 77020 (0.0008) [2023-10-14 00:08:22,446][60934] Updated weights for policy 1, policy_version 77172 (0.0008) [2023-10-14 00:08:22,814][60934] Updated weights for policy 1, policy_version 77182 (0.0010) [2023-10-14 00:08:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13662.6). Total num frames: 158302208. Throughput: 0: 1714.2, 1: 1730.8. Samples: 39591538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:08:26,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:08:26,368][60935] Updated weights for policy 0, policy_version 77030 (0.0009) [2023-10-14 00:08:26,749][60935] Updated weights for policy 0, policy_version 77040 (0.0009) [2023-10-14 00:08:26,810][60934] Updated weights for policy 1, policy_version 77192 (0.0008) [2023-10-14 00:08:27,110][60935] Updated weights for policy 0, policy_version 77050 (0.0008) [2023-10-14 00:08:27,179][60934] Updated weights for policy 1, policy_version 77202 (0.0008) [2023-10-14 00:08:27,542][60934] Updated weights for policy 1, policy_version 77212 (0.0008) [2023-10-14 00:08:31,022][60935] Updated weights for policy 0, policy_version 77060 (0.0010) [2023-10-14 00:08:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 158367744. Throughput: 0: 1702.2, 1: 1704.4. Samples: 39600704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:08:31,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:08:31,401][60935] Updated weights for policy 0, policy_version 77070 (0.0011) [2023-10-14 00:08:31,669][60934] Updated weights for policy 1, policy_version 77222 (0.0009) [2023-10-14 00:08:31,760][60935] Updated weights for policy 0, policy_version 77080 (0.0008) [2023-10-14 00:08:32,031][60934] Updated weights for policy 1, policy_version 77232 (0.0009) [2023-10-14 00:08:32,397][60934] Updated weights for policy 1, policy_version 77242 (0.0008) [2023-10-14 00:08:35,889][60935] Updated weights for policy 0, policy_version 77090 (0.0009) [2023-10-14 00:08:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 158433280. Throughput: 0: 1705.6, 1: 1718.1. Samples: 39621150. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:08:36,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:08:36,252][60935] Updated weights for policy 0, policy_version 77100 (0.0008) [2023-10-14 00:08:36,509][60934] Updated weights for policy 1, policy_version 77252 (0.0010) [2023-10-14 00:08:36,624][60935] Updated weights for policy 0, policy_version 77110 (0.0009) [2023-10-14 00:08:36,871][60934] Updated weights for policy 1, policy_version 77262 (0.0008) [2023-10-14 00:08:36,993][60935] Updated weights for policy 0, policy_version 77120 (0.0008) [2023-10-14 00:08:37,235][60934] Updated weights for policy 1, policy_version 77272 (0.0007) [2023-10-14 00:08:41,006][60935] Updated weights for policy 0, policy_version 77130 (0.0009) [2023-10-14 00:08:41,085][60934] Updated weights for policy 1, policy_version 77282 (0.0007) [2023-10-14 00:08:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 158498816. Throughput: 0: 1703.2, 1: 1720.7. Samples: 39642094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:08:41,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:08:41,381][60935] Updated weights for policy 0, policy_version 77140 (0.0008) [2023-10-14 00:08:41,445][60934] Updated weights for policy 1, policy_version 77292 (0.0008) [2023-10-14 00:08:41,745][60935] Updated weights for policy 0, policy_version 77150 (0.0008) [2023-10-14 00:08:41,809][60934] Updated weights for policy 1, policy_version 77302 (0.0008) [2023-10-14 00:08:42,170][60934] Updated weights for policy 1, policy_version 77312 (0.0009) [2023-10-14 00:08:45,764][60935] Updated weights for policy 0, policy_version 77160 (0.0007) [2023-10-14 00:08:46,138][60935] Updated weights for policy 0, policy_version 77170 (0.0008) [2023-10-14 00:08:46,221][60934] Updated weights for policy 1, policy_version 77322 (0.0007) [2023-10-14 00:08:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 158564352. Throughput: 0: 1709.7, 1: 1712.0. Samples: 39651814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:08:46,249][59943] Avg episode reward: [(0, '-0.110'), (1, '-0.120')] [2023-10-14 00:08:46,510][60935] Updated weights for policy 0, policy_version 77180 (0.0007) [2023-10-14 00:08:46,592][60934] Updated weights for policy 1, policy_version 77332 (0.0008) [2023-10-14 00:08:46,946][60934] Updated weights for policy 1, policy_version 77342 (0.0009) [2023-10-14 00:08:50,504][60935] Updated weights for policy 0, policy_version 77190 (0.0009) [2023-10-14 00:08:50,872][60935] Updated weights for policy 0, policy_version 77200 (0.0009) [2023-10-14 00:08:50,957][60934] Updated weights for policy 1, policy_version 77352 (0.0007) [2023-10-14 00:08:51,245][60935] Updated weights for policy 0, policy_version 77210 (0.0008) [2023-10-14 00:08:51,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 158629888. Throughput: 0: 1706.6, 1: 1719.6. Samples: 39672654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:08:51,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:08:51,319][60934] Updated weights for policy 1, policy_version 77362 (0.0007) [2023-10-14 00:08:51,674][60934] Updated weights for policy 1, policy_version 77372 (0.0008) [2023-10-14 00:08:55,234][60935] Updated weights for policy 0, policy_version 77220 (0.0008) [2023-10-14 00:08:55,615][60935] Updated weights for policy 0, policy_version 77230 (0.0009) [2023-10-14 00:08:55,842][60934] Updated weights for policy 1, policy_version 77382 (0.0007) [2023-10-14 00:08:55,983][60935] Updated weights for policy 0, policy_version 77240 (0.0008) [2023-10-14 00:08:56,213][60934] Updated weights for policy 1, policy_version 77392 (0.0008) [2023-10-14 00:08:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 158695424. Throughput: 0: 1685.3, 1: 1708.6. Samples: 39692680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:08:56,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:08:56,270][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000077248_79101952.pth... [2023-10-14 00:08:56,305][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000075648_77463552.pth [2023-10-14 00:08:56,576][60934] Updated weights for policy 1, policy_version 77402 (0.0007) [2023-10-14 00:08:56,788][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000077408_79659008.pth... [2023-10-14 00:08:56,817][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000075864_77987840.pth [2023-10-14 00:08:59,825][60935] Updated weights for policy 0, policy_version 77250 (0.0009) [2023-10-14 00:09:00,199][60935] Updated weights for policy 0, policy_version 77260 (0.0007) [2023-10-14 00:09:00,560][60935] Updated weights for policy 0, policy_version 77270 (0.0008) [2023-10-14 00:09:00,753][60934] Updated weights for policy 1, policy_version 77412 (0.0009) [2023-10-14 00:09:00,926][60935] Updated weights for policy 0, policy_version 77280 (0.0007) [2023-10-14 00:09:01,127][60934] Updated weights for policy 1, policy_version 77422 (0.0009) [2023-10-14 00:09:01,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 158793728. Throughput: 0: 1705.8, 1: 1703.4. Samples: 39702674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:09:01,249][59943] Avg episode reward: [(0, '-0.030'), (1, '0.000')] [2023-10-14 00:09:01,485][60934] Updated weights for policy 1, policy_version 77432 (0.0011) [2023-10-14 00:09:05,084][60935] Updated weights for policy 0, policy_version 77290 (0.0009) [2023-10-14 00:09:05,420][60934] Updated weights for policy 1, policy_version 77442 (0.0009) [2023-10-14 00:09:05,452][60935] Updated weights for policy 0, policy_version 77300 (0.0009) [2023-10-14 00:09:05,788][60934] Updated weights for policy 1, policy_version 77452 (0.0009) [2023-10-14 00:09:05,827][60935] Updated weights for policy 0, policy_version 77310 (0.0007) [2023-10-14 00:09:06,150][60934] Updated weights for policy 1, policy_version 77462 (0.0008) [2023-10-14 00:09:06,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 158859264. Throughput: 0: 1701.9, 1: 1696.6. Samples: 39723470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:09:06,249][59943] Avg episode reward: [(0, '-0.030'), (1, '0.000')] [2023-10-14 00:09:06,511][60934] Updated weights for policy 1, policy_version 77472 (0.0008) [2023-10-14 00:09:09,709][60935] Updated weights for policy 0, policy_version 77320 (0.0008) [2023-10-14 00:09:10,079][60935] Updated weights for policy 0, policy_version 77330 (0.0010) [2023-10-14 00:09:10,443][60935] Updated weights for policy 0, policy_version 77340 (0.0009) [2023-10-14 00:09:10,499][60934] Updated weights for policy 1, policy_version 77482 (0.0009) [2023-10-14 00:09:10,866][60934] Updated weights for policy 1, policy_version 77492 (0.0007) [2023-10-14 00:09:11,225][60934] Updated weights for policy 1, policy_version 77502 (0.0008) [2023-10-14 00:09:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 158924800. Throughput: 0: 1681.0, 1: 1685.0. Samples: 39743008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:09:11,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:09:14,482][60935] Updated weights for policy 0, policy_version 77350 (0.0008) [2023-10-14 00:09:14,865][60935] Updated weights for policy 0, policy_version 77360 (0.0008) [2023-10-14 00:09:15,241][60935] Updated weights for policy 0, policy_version 77370 (0.0009) [2023-10-14 00:09:15,271][60934] Updated weights for policy 1, policy_version 77512 (0.0008) [2023-10-14 00:09:15,635][60934] Updated weights for policy 1, policy_version 77522 (0.0008) [2023-10-14 00:09:16,002][60934] Updated weights for policy 1, policy_version 77532 (0.0009) [2023-10-14 00:09:16,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 159023104. Throughput: 0: 1716.1, 1: 1697.6. Samples: 39754322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:09:16,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:09:19,377][60935] Updated weights for policy 0, policy_version 77380 (0.0009) [2023-10-14 00:09:19,745][60935] Updated weights for policy 0, policy_version 77390 (0.0008) [2023-10-14 00:09:19,983][60934] Updated weights for policy 1, policy_version 77542 (0.0007) [2023-10-14 00:09:20,121][60935] Updated weights for policy 0, policy_version 77400 (0.0008) [2023-10-14 00:09:20,351][60934] Updated weights for policy 1, policy_version 77552 (0.0009) [2023-10-14 00:09:20,712][60934] Updated weights for policy 1, policy_version 77562 (0.0009) [2023-10-14 00:09:21,248][59943] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 159088640. Throughput: 0: 1701.3, 1: 1707.7. Samples: 39774558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:09:21,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:09:24,054][60935] Updated weights for policy 0, policy_version 77410 (0.0010) [2023-10-14 00:09:24,424][60935] Updated weights for policy 0, policy_version 77420 (0.0008) [2023-10-14 00:09:24,787][60935] Updated weights for policy 0, policy_version 77430 (0.0008) [2023-10-14 00:09:24,798][60934] Updated weights for policy 1, policy_version 77572 (0.0008) [2023-10-14 00:09:25,154][60935] Updated weights for policy 0, policy_version 77440 (0.0007) [2023-10-14 00:09:25,164][60934] Updated weights for policy 1, policy_version 77582 (0.0008) [2023-10-14 00:09:25,527][60934] Updated weights for policy 1, policy_version 77592 (0.0008) [2023-10-14 00:09:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 159154176. Throughput: 0: 1692.5, 1: 1683.9. Samples: 39794032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:09:26,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:09:29,003][60935] Updated weights for policy 0, policy_version 77450 (0.0007) [2023-10-14 00:09:29,369][60935] Updated weights for policy 0, policy_version 77460 (0.0008) [2023-10-14 00:09:29,561][60934] Updated weights for policy 1, policy_version 77602 (0.0009) [2023-10-14 00:09:29,735][60935] Updated weights for policy 0, policy_version 77470 (0.0009) [2023-10-14 00:09:29,932][60934] Updated weights for policy 1, policy_version 77612 (0.0009) [2023-10-14 00:09:30,299][60934] Updated weights for policy 1, policy_version 77622 (0.0009) [2023-10-14 00:09:30,668][60934] Updated weights for policy 1, policy_version 77632 (0.0008) [2023-10-14 00:09:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 159219712. Throughput: 0: 1716.0, 1: 1697.0. Samples: 39805400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:09:31,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:09:33,745][60935] Updated weights for policy 0, policy_version 77480 (0.0008) [2023-10-14 00:09:34,105][60935] Updated weights for policy 0, policy_version 77490 (0.0009) [2023-10-14 00:09:34,468][60935] Updated weights for policy 0, policy_version 77500 (0.0009) [2023-10-14 00:09:34,737][60934] Updated weights for policy 1, policy_version 77642 (0.0007) [2023-10-14 00:09:35,097][60934] Updated weights for policy 1, policy_version 77652 (0.0007) [2023-10-14 00:09:35,463][60934] Updated weights for policy 1, policy_version 77662 (0.0007) [2023-10-14 00:09:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 159285248. Throughput: 0: 1690.3, 1: 1694.2. Samples: 39824956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:09:36,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:09:38,524][60935] Updated weights for policy 0, policy_version 77510 (0.0010) [2023-10-14 00:09:38,896][60935] Updated weights for policy 0, policy_version 77520 (0.0008) [2023-10-14 00:09:39,275][60935] Updated weights for policy 0, policy_version 77530 (0.0009) [2023-10-14 00:09:39,387][60934] Updated weights for policy 1, policy_version 77672 (0.0007) [2023-10-14 00:09:39,750][60934] Updated weights for policy 1, policy_version 77682 (0.0007) [2023-10-14 00:09:40,123][60934] Updated weights for policy 1, policy_version 77692 (0.0010) [2023-10-14 00:09:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 159350784. Throughput: 0: 1708.3, 1: 1672.0. Samples: 39844796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:09:41,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:09:43,237][60935] Updated weights for policy 0, policy_version 77540 (0.0007) [2023-10-14 00:09:43,603][60935] Updated weights for policy 0, policy_version 77550 (0.0007) [2023-10-14 00:09:43,965][60935] Updated weights for policy 0, policy_version 77560 (0.0007) [2023-10-14 00:09:44,206][60934] Updated weights for policy 1, policy_version 77702 (0.0010) [2023-10-14 00:09:44,571][60934] Updated weights for policy 1, policy_version 77712 (0.0009) [2023-10-14 00:09:44,943][60934] Updated weights for policy 1, policy_version 77722 (0.0007) [2023-10-14 00:09:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 159416320. Throughput: 0: 1696.5, 1: 1705.2. Samples: 39855748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:09:46,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:09:47,940][60935] Updated weights for policy 0, policy_version 77570 (0.0009) [2023-10-14 00:09:48,308][60935] Updated weights for policy 0, policy_version 77580 (0.0011) [2023-10-14 00:09:48,684][60935] Updated weights for policy 0, policy_version 77590 (0.0009) [2023-10-14 00:09:48,994][60934] Updated weights for policy 1, policy_version 77732 (0.0007) [2023-10-14 00:09:49,051][60935] Updated weights for policy 0, policy_version 77600 (0.0009) [2023-10-14 00:09:49,354][60934] Updated weights for policy 1, policy_version 77742 (0.0009) [2023-10-14 00:09:49,735][60934] Updated weights for policy 1, policy_version 77752 (0.0010) [2023-10-14 00:09:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 159481856. Throughput: 0: 1694.3, 1: 1685.9. Samples: 39875582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:09:51,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:09:52,905][60935] Updated weights for policy 0, policy_version 77610 (0.0011) [2023-10-14 00:09:53,283][60935] Updated weights for policy 0, policy_version 77620 (0.0011) [2023-10-14 00:09:53,640][60934] Updated weights for policy 1, policy_version 77762 (0.0009) [2023-10-14 00:09:53,649][60935] Updated weights for policy 0, policy_version 77630 (0.0009) [2023-10-14 00:09:54,013][60934] Updated weights for policy 1, policy_version 77772 (0.0007) [2023-10-14 00:09:54,369][60934] Updated weights for policy 1, policy_version 77782 (0.0008) [2023-10-14 00:09:54,735][60934] Updated weights for policy 1, policy_version 77792 (0.0008) [2023-10-14 00:09:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 159547392. Throughput: 0: 1720.5, 1: 1683.5. Samples: 39896190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:09:56,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:09:57,666][60935] Updated weights for policy 0, policy_version 77640 (0.0008) [2023-10-14 00:09:58,036][60935] Updated weights for policy 0, policy_version 77650 (0.0008) [2023-10-14 00:09:58,408][60935] Updated weights for policy 0, policy_version 77660 (0.0009) [2023-10-14 00:09:58,853][60934] Updated weights for policy 1, policy_version 77802 (0.0009) [2023-10-14 00:09:59,222][60934] Updated weights for policy 1, policy_version 77812 (0.0011) [2023-10-14 00:09:59,592][60934] Updated weights for policy 1, policy_version 77822 (0.0011) [2023-10-14 00:10:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 159612928. Throughput: 0: 1684.8, 1: 1699.0. Samples: 39906592. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-14 00:10:01,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:10:02,420][60935] Updated weights for policy 0, policy_version 77670 (0.0009) [2023-10-14 00:10:02,792][60935] Updated weights for policy 0, policy_version 77680 (0.0010) [2023-10-14 00:10:03,158][60935] Updated weights for policy 0, policy_version 77690 (0.0007) [2023-10-14 00:10:03,457][60934] Updated weights for policy 1, policy_version 77832 (0.0008) [2023-10-14 00:10:03,821][60934] Updated weights for policy 1, policy_version 77842 (0.0007) [2023-10-14 00:10:04,193][60934] Updated weights for policy 1, policy_version 77852 (0.0009) [2023-10-14 00:10:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 159678464. Throughput: 0: 1708.5, 1: 1671.3. Samples: 39926650. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-14 00:10:06,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:10:07,173][60935] Updated weights for policy 0, policy_version 77700 (0.0009) [2023-10-14 00:10:07,573][60935] Updated weights for policy 0, policy_version 77710 (0.0011) [2023-10-14 00:10:07,934][60935] Updated weights for policy 0, policy_version 77720 (0.0011) [2023-10-14 00:10:08,072][60934] Updated weights for policy 1, policy_version 77862 (0.0008) [2023-10-14 00:10:08,441][60934] Updated weights for policy 1, policy_version 77872 (0.0007) [2023-10-14 00:10:08,798][60934] Updated weights for policy 1, policy_version 77882 (0.0007) [2023-10-14 00:10:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 159744000. Throughput: 0: 1722.2, 1: 1698.1. Samples: 39947942. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-14 00:10:11,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:10:11,753][60935] Updated weights for policy 0, policy_version 77730 (0.0008) [2023-10-14 00:10:12,111][60935] Updated weights for policy 0, policy_version 77740 (0.0008) [2023-10-14 00:10:12,488][60935] Updated weights for policy 0, policy_version 77750 (0.0010) [2023-10-14 00:10:12,849][60935] Updated weights for policy 0, policy_version 77760 (0.0009) [2023-10-14 00:10:12,942][60934] Updated weights for policy 1, policy_version 77892 (0.0007) [2023-10-14 00:10:13,304][60934] Updated weights for policy 1, policy_version 77902 (0.0010) [2023-10-14 00:10:13,665][60934] Updated weights for policy 1, policy_version 77912 (0.0010) [2023-10-14 00:10:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 159809536. Throughput: 0: 1694.3, 1: 1688.1. Samples: 39957610. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-14 00:10:16,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:10:16,764][60935] Updated weights for policy 0, policy_version 77770 (0.0008) [2023-10-14 00:10:17,123][60935] Updated weights for policy 0, policy_version 77780 (0.0008) [2023-10-14 00:10:17,497][60935] Updated weights for policy 0, policy_version 77790 (0.0009) [2023-10-14 00:10:17,580][60934] Updated weights for policy 1, policy_version 77922 (0.0008) [2023-10-14 00:10:17,948][60934] Updated weights for policy 1, policy_version 77932 (0.0010) [2023-10-14 00:10:18,311][60934] Updated weights for policy 1, policy_version 77942 (0.0011) [2023-10-14 00:10:18,671][60934] Updated weights for policy 1, policy_version 77952 (0.0009) [2023-10-14 00:10:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 159875072. Throughput: 0: 1728.7, 1: 1690.1. Samples: 39978800. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-14 00:10:21,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:10:21,461][60935] Updated weights for policy 0, policy_version 77800 (0.0007) [2023-10-14 00:10:21,818][60935] Updated weights for policy 0, policy_version 77810 (0.0008) [2023-10-14 00:10:22,182][60935] Updated weights for policy 0, policy_version 77820 (0.0008) [2023-10-14 00:10:22,817][60934] Updated weights for policy 1, policy_version 77962 (0.0010) [2023-10-14 00:10:23,035][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000002 [2023-10-14 00:10:26,209][60935] Updated weights for policy 0, policy_version 77830 (0.0008) [2023-10-14 00:10:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 159940608. Throughput: 0: 1733.2, 1: 1729.6. Samples: 40000626. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-14 00:10:26,249][59943] Avg episode reward: [(0, '-0.110'), (1, '0.000')] [2023-10-14 00:10:26,578][60935] Updated weights for policy 0, policy_version 77840 (0.0010) [2023-10-14 00:10:26,945][60935] Updated weights for policy 0, policy_version 77850 (0.0008) [2023-10-14 00:10:27,054][60934] Updated weights for policy 1, policy_version 77972 (0.0009) [2023-10-14 00:10:27,413][60934] Updated weights for policy 1, policy_version 77982 (0.0009) [2023-10-14 00:10:27,479][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000002 [2023-10-14 00:10:30,836][60935] Updated weights for policy 0, policy_version 77860 (0.0008) [2023-10-14 00:10:31,207][60935] Updated weights for policy 0, policy_version 77870 (0.0008) [2023-10-14 00:10:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 160006144. Throughput: 0: 1723.7, 1: 1714.6. Samples: 40010472. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-14 00:10:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:10:31,567][60935] Updated weights for policy 0, policy_version 77880 (0.0007) [2023-10-14 00:10:31,706][60934] Updated weights for policy 1, policy_version 77992 (0.0007) [2023-10-14 00:10:31,991][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:10:35,645][60935] Updated weights for policy 0, policy_version 77890 (0.0010) [2023-10-14 00:10:36,003][60935] Updated weights for policy 0, policy_version 77900 (0.0008) [2023-10-14 00:10:36,137][60934] Updated weights for policy 1, policy_version 78002 (0.0008) [2023-10-14 00:10:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13662.6). Total num frames: 160071680. Throughput: 0: 1729.4, 1: 1745.0. Samples: 40031930. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-14 00:10:36,248][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:10:36,364][60935] Updated weights for policy 0, policy_version 77910 (0.0008) [2023-10-14 00:10:36,498][60934] Updated weights for policy 1, policy_version 78012 (0.0008) [2023-10-14 00:10:36,633][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:10:36,730][60935] Updated weights for policy 0, policy_version 77920 (0.0010) [2023-10-14 00:10:40,669][60935] Updated weights for policy 0, policy_version 77930 (0.0008) [2023-10-14 00:10:40,746][60934] Updated weights for policy 1, policy_version 78022 (0.0008) [2023-10-14 00:10:41,026][60935] Updated weights for policy 0, policy_version 77940 (0.0009) [2023-10-14 00:10:41,108][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:10:41,110][60934] Updated weights for policy 1, policy_version 78032 (0.0008) [2023-10-14 00:10:41,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 160169984. Throughput: 0: 1715.4, 1: 1770.0. Samples: 40053030. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-14 00:10:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:10:41,400][60935] Updated weights for policy 0, policy_version 77950 (0.0009) [2023-10-14 00:10:45,223][60934] Updated weights for policy 1, policy_version 78042 (0.0008) [2023-10-14 00:10:45,430][60935] Updated weights for policy 0, policy_version 77960 (0.0009) [2023-10-14 00:10:45,442][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:10:45,795][60935] Updated weights for policy 0, policy_version 77970 (0.0009) [2023-10-14 00:10:46,170][60935] Updated weights for policy 0, policy_version 77980 (0.0009) [2023-10-14 00:10:46,248][59943] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 160235520. Throughput: 0: 1729.7, 1: 1762.4. Samples: 40063738. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-14 00:10:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:10:49,356][60934] Updated weights for policy 1, policy_version 78052 (0.0008) [2023-10-14 00:10:49,723][60934] Updated weights for policy 1, policy_version 78062 (0.0008) [2023-10-14 00:10:49,789][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:10:50,130][60935] Updated weights for policy 0, policy_version 77990 (0.0008) [2023-10-14 00:10:50,498][60935] Updated weights for policy 0, policy_version 78000 (0.0009) [2023-10-14 00:10:50,874][60935] Updated weights for policy 0, policy_version 78010 (0.0007) [2023-10-14 00:10:51,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 160333824. Throughput: 0: 1732.3, 1: 1794.3. Samples: 40085346. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-14 00:10:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:10:54,073][60934] Updated weights for policy 1, policy_version 78072 (0.0008) [2023-10-14 00:10:54,435][60934] Updated weights for policy 1, policy_version 78082 (0.0008) [2023-10-14 00:10:54,800][60934] Updated weights for policy 1, policy_version 78092 (0.0008) [2023-10-14 00:10:54,845][60935] Updated weights for policy 0, policy_version 78020 (0.0009) [2023-10-14 00:10:55,237][60935] Updated weights for policy 0, policy_version 78030 (0.0009) [2023-10-14 00:10:55,605][60935] Updated weights for policy 0, policy_version 78040 (0.0008) [2023-10-14 00:10:56,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 160399360. Throughput: 0: 1703.9, 1: 1788.1. Samples: 40105080. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-14 00:10:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:10:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000078096_80478208.pth... [2023-10-14 00:10:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000078048_79921152.pth... [2023-10-14 00:10:56,298][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000076448_78282752.pth [2023-10-14 00:10:56,299][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000076608_78839808.pth [2023-10-14 00:10:58,846][60934] Updated weights for policy 1, policy_version 78102 (0.0009) [2023-10-14 00:10:59,203][60934] Updated weights for policy 1, policy_version 78112 (0.0009) [2023-10-14 00:10:59,566][60934] Updated weights for policy 1, policy_version 78122 (0.0008) [2023-10-14 00:10:59,646][60935] Updated weights for policy 0, policy_version 78050 (0.0008) [2023-10-14 00:11:00,008][60935] Updated weights for policy 0, policy_version 78060 (0.0009) [2023-10-14 00:11:00,374][60935] Updated weights for policy 0, policy_version 78070 (0.0008) [2023-10-14 00:11:00,739][60935] Updated weights for policy 0, policy_version 78080 (0.0009) [2023-10-14 00:11:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 160464896. Throughput: 0: 1728.4, 1: 1812.2. Samples: 40116934. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-14 00:11:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:11:03,459][60934] Updated weights for policy 1, policy_version 78132 (0.0008) [2023-10-14 00:11:03,832][60934] Updated weights for policy 1, policy_version 78142 (0.0007) [2023-10-14 00:11:04,201][60934] Updated weights for policy 1, policy_version 78152 (0.0009) [2023-10-14 00:11:04,740][60935] Updated weights for policy 0, policy_version 78090 (0.0008) [2023-10-14 00:11:05,106][60935] Updated weights for policy 0, policy_version 78100 (0.0008) [2023-10-14 00:11:05,479][60935] Updated weights for policy 0, policy_version 78110 (0.0007) [2023-10-14 00:11:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 160530432. Throughput: 0: 1711.4, 1: 1791.9. Samples: 40136446. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-14 00:11:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:11:08,074][60934] Updated weights for policy 1, policy_version 78162 (0.0010) [2023-10-14 00:11:08,440][60934] Updated weights for policy 1, policy_version 78172 (0.0010) [2023-10-14 00:11:08,581][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:11:09,277][60935] Updated weights for policy 0, policy_version 78120 (0.0007) [2023-10-14 00:11:09,645][60935] Updated weights for policy 0, policy_version 78130 (0.0008) [2023-10-14 00:11:10,013][60935] Updated weights for policy 0, policy_version 78140 (0.0008) [2023-10-14 00:11:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 160595968. Throughput: 0: 1693.2, 1: 1796.9. Samples: 40157682. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-14 00:11:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:11:12,794][60934] Updated weights for policy 1, policy_version 78182 (0.0007) [2023-10-14 00:11:13,143][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:11:13,149][60934] Updated weights for policy 1, policy_version 78192 (0.0007) [2023-10-14 00:11:14,184][60935] Updated weights for policy 0, policy_version 78150 (0.0007) [2023-10-14 00:11:14,551][60935] Updated weights for policy 0, policy_version 78160 (0.0007) [2023-10-14 00:11:14,923][60935] Updated weights for policy 0, policy_version 78170 (0.0010) [2023-10-14 00:11:16,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 160661504. Throughput: 0: 1722.0, 1: 1795.4. Samples: 40168756. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-14 00:11:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:11:17,259][60934] Updated weights for policy 1, policy_version 78202 (0.0007) [2023-10-14 00:11:17,479][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:11:18,987][60935] Updated weights for policy 0, policy_version 78180 (0.0010) [2023-10-14 00:11:19,362][60935] Updated weights for policy 0, policy_version 78190 (0.0008) [2023-10-14 00:11:19,724][60935] Updated weights for policy 0, policy_version 78200 (0.0007) [2023-10-14 00:11:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 160727040. Throughput: 0: 1698.2, 1: 1802.3. Samples: 40189452. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-14 00:11:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:11:21,348][60934] Updated weights for policy 1, policy_version 78212 (0.0010) [2023-10-14 00:11:21,721][60934] Updated weights for policy 1, policy_version 78222 (0.0009) [2023-10-14 00:11:21,786][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:11:23,676][60935] Updated weights for policy 0, policy_version 78210 (0.0008) [2023-10-14 00:11:24,043][60935] Updated weights for policy 0, policy_version 78220 (0.0009) [2023-10-14 00:11:24,413][60935] Updated weights for policy 0, policy_version 78230 (0.0008) [2023-10-14 00:11:24,767][60935] Updated weights for policy 0, policy_version 78240 (0.0010) [2023-10-14 00:11:26,047][60934] Updated weights for policy 1, policy_version 78232 (0.0010) [2023-10-14 00:11:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 160792576. Throughput: 0: 1703.7, 1: 1809.2. Samples: 40211114. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-14 00:11:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:11:26,336][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:11:28,685][60935] Updated weights for policy 0, policy_version 78250 (0.0011) [2023-10-14 00:11:29,057][60935] Updated weights for policy 0, policy_version 78260 (0.0009) [2023-10-14 00:11:29,416][60935] Updated weights for policy 0, policy_version 78270 (0.0008) [2023-10-14 00:11:30,458][60934] Updated weights for policy 1, policy_version 78242 (0.0008) [2023-10-14 00:11:30,876][60934] Updated weights for policy 1, policy_version 78252 (0.0007) [2023-10-14 00:11:31,023][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:11:31,248][59943] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 13884.7). Total num frames: 160890880. Throughput: 0: 1710.5, 1: 1802.8. Samples: 40221838. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-14 00:11:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:11:33,316][60935] Updated weights for policy 0, policy_version 78280 (0.0010) [2023-10-14 00:11:33,679][60935] Updated weights for policy 0, policy_version 78290 (0.0011) [2023-10-14 00:11:34,047][60935] Updated weights for policy 0, policy_version 78300 (0.0011) [2023-10-14 00:11:35,261][60934] Updated weights for policy 1, policy_version 78262 (0.0007) [2023-10-14 00:11:35,613][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:11:35,619][60934] Updated weights for policy 1, policy_version 78272 (0.0009) [2023-10-14 00:11:36,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 13884.7). Total num frames: 160956416. Throughput: 0: 1689.4, 1: 1803.8. Samples: 40242540. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-14 00:11:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:11:38,030][60935] Updated weights for policy 0, policy_version 78310 (0.0008) [2023-10-14 00:11:38,406][60935] Updated weights for policy 0, policy_version 78320 (0.0011) [2023-10-14 00:11:38,777][60935] Updated weights for policy 0, policy_version 78330 (0.0011) [2023-10-14 00:11:39,923][60934] Updated weights for policy 1, policy_version 78282 (0.0008) [2023-10-14 00:11:40,143][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:11:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 161021952. Throughput: 0: 1717.4, 1: 1807.3. Samples: 40263694. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-14 00:11:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:11:42,953][60935] Updated weights for policy 0, policy_version 78340 (0.0007) [2023-10-14 00:11:43,339][60935] Updated weights for policy 0, policy_version 78350 (0.0007) [2023-10-14 00:11:43,705][60935] Updated weights for policy 0, policy_version 78360 (0.0007) [2023-10-14 00:11:44,148][60934] Updated weights for policy 1, policy_version 78292 (0.0007) [2023-10-14 00:11:44,508][60934] Updated weights for policy 1, policy_version 78302 (0.0007) [2023-10-14 00:11:44,579][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:11:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 161087488. Throughput: 0: 1694.8, 1: 1806.0. Samples: 40274474. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:11:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:11:47,594][60935] Updated weights for policy 0, policy_version 78370 (0.0008) [2023-10-14 00:11:47,958][60935] Updated weights for policy 0, policy_version 78380 (0.0010) [2023-10-14 00:11:48,325][60935] Updated weights for policy 0, policy_version 78390 (0.0008) [2023-10-14 00:11:48,646][60934] Updated weights for policy 1, policy_version 78312 (0.0007) [2023-10-14 00:11:48,690][60935] Updated weights for policy 0, policy_version 78400 (0.0008) [2023-10-14 00:11:48,938][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:11:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 161153024. Throughput: 0: 1702.1, 1: 1831.5. Samples: 40295458. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:11:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:11:52,667][60935] Updated weights for policy 0, policy_version 78410 (0.0007) [2023-10-14 00:11:52,877][60934] Updated weights for policy 1, policy_version 78322 (0.0008) [2023-10-14 00:11:53,037][60935] Updated weights for policy 0, policy_version 78420 (0.0008) [2023-10-14 00:11:53,248][60934] Updated weights for policy 1, policy_version 78332 (0.0008) [2023-10-14 00:11:53,386][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:11:53,404][60935] Updated weights for policy 0, policy_version 78430 (0.0008) [2023-10-14 00:11:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 161218560. Throughput: 0: 1717.6, 1: 1827.3. Samples: 40317200. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:11:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:11:57,329][60935] Updated weights for policy 0, policy_version 78440 (0.0010) [2023-10-14 00:11:57,508][60934] Updated weights for policy 1, policy_version 78342 (0.0007) [2023-10-14 00:11:57,695][60935] Updated weights for policy 0, policy_version 78450 (0.0008) [2023-10-14 00:11:57,865][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:11:57,867][60934] Updated weights for policy 1, policy_version 78352 (0.0007) [2023-10-14 00:11:58,066][60935] Updated weights for policy 0, policy_version 78460 (0.0008) [2023-10-14 00:12:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 161284096. Throughput: 0: 1688.5, 1: 1827.3. Samples: 40326970. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:12:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:12:01,867][60935] Updated weights for policy 0, policy_version 78470 (0.0009) [2023-10-14 00:12:02,120][60934] Updated weights for policy 1, policy_version 78362 (0.0008) [2023-10-14 00:12:02,233][60935] Updated weights for policy 0, policy_version 78480 (0.0008) [2023-10-14 00:12:02,330][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:12:02,607][60935] Updated weights for policy 0, policy_version 78490 (0.0008) [2023-10-14 00:12:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 161349632. Throughput: 0: 1719.0, 1: 1826.5. Samples: 40349000. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:12:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:12:06,488][60934] Updated weights for policy 1, policy_version 78372 (0.0008) [2023-10-14 00:12:06,820][60935] Updated weights for policy 0, policy_version 78500 (0.0009) [2023-10-14 00:12:06,855][60934] Updated weights for policy 1, policy_version 78382 (0.0010) [2023-10-14 00:12:06,926][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:12:07,188][60935] Updated weights for policy 0, policy_version 78510 (0.0009) [2023-10-14 00:12:07,550][60935] Updated weights for policy 0, policy_version 78520 (0.0010) [2023-10-14 00:12:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 161415168. Throughput: 0: 1724.8, 1: 1818.6. Samples: 40370566. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:12:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:12:11,268][60934] Updated weights for policy 1, policy_version 78392 (0.0010) [2023-10-14 00:12:11,427][60935] Updated weights for policy 0, policy_version 78530 (0.0010) [2023-10-14 00:12:11,636][60934] Updated weights for policy 1, policy_version 78402 (0.0008) [2023-10-14 00:12:11,789][60935] Updated weights for policy 0, policy_version 78540 (0.0008) [2023-10-14 00:12:11,996][60934] Updated weights for policy 1, policy_version 78412 (0.0007) [2023-10-14 00:12:12,156][60935] Updated weights for policy 0, policy_version 78550 (0.0009) [2023-10-14 00:12:12,529][60935] Updated weights for policy 0, policy_version 78560 (0.0010) [2023-10-14 00:12:16,010][60934] Updated weights for policy 1, policy_version 78422 (0.0007) [2023-10-14 00:12:16,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 161480704. Throughput: 0: 1708.6, 1: 1806.4. Samples: 40380012. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:12:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:12:16,372][60935] Updated weights for policy 0, policy_version 78570 (0.0008) [2023-10-14 00:12:16,403][60934] Updated weights for policy 1, policy_version 78432 (0.0007) [2023-10-14 00:12:16,732][60935] Updated weights for policy 0, policy_version 78580 (0.0008) [2023-10-14 00:12:16,769][60934] Updated weights for policy 1, policy_version 78442 (0.0008) [2023-10-14 00:12:17,110][60935] Updated weights for policy 0, policy_version 78590 (0.0010) [2023-10-14 00:12:20,831][60934] Updated weights for policy 1, policy_version 78452 (0.0008) [2023-10-14 00:12:21,172][60935] Updated weights for policy 0, policy_version 78600 (0.0007) [2023-10-14 00:12:21,188][60934] Updated weights for policy 1, policy_version 78462 (0.0007) [2023-10-14 00:12:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 161546240. Throughput: 0: 1720.7, 1: 1792.4. Samples: 40400630. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:12:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:12:21,531][60935] Updated weights for policy 0, policy_version 78610 (0.0007) [2023-10-14 00:12:21,548][60828] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000001 [2023-10-14 00:12:21,551][60934] Updated weights for policy 1, policy_version 78472 (0.0007) [2023-10-14 00:12:21,905][60935] Updated weights for policy 0, policy_version 78620 (0.0010) [2023-10-14 00:12:25,474][60934] Updated weights for policy 1, policy_version 78482 (0.0007) [2023-10-14 00:12:25,834][60934] Updated weights for policy 1, policy_version 78492 (0.0007) [2023-10-14 00:12:25,938][60935] Updated weights for policy 0, policy_version 78630 (0.0009) [2023-10-14 00:12:26,206][60934] Updated weights for policy 1, policy_version 78502 (0.0007) [2023-10-14 00:12:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 161611776. Throughput: 0: 1715.3, 1: 1792.0. Samples: 40421522. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:12:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:12:26,305][60935] Updated weights for policy 0, policy_version 78640 (0.0008) [2023-10-14 00:12:26,661][60935] Updated weights for policy 0, policy_version 78650 (0.0009) [2023-10-14 00:12:30,297][60934] Updated weights for policy 1, policy_version 78512 (0.0009) [2023-10-14 00:12:30,592][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:12:30,808][60935] Updated weights for policy 0, policy_version 78660 (0.0009) [2023-10-14 00:12:31,201][60935] Updated weights for policy 0, policy_version 78670 (0.0008) [2023-10-14 00:12:31,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 161710080. Throughput: 0: 1713.6, 1: 1773.1. Samples: 40431376. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:12:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:12:31,581][60935] Updated weights for policy 0, policy_version 78680 (0.0008) [2023-10-14 00:12:34,486][60934] Updated weights for policy 1, policy_version 78522 (0.0008) [2023-10-14 00:12:34,858][60934] Updated weights for policy 1, policy_version 78532 (0.0009) [2023-10-14 00:12:35,000][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-14 00:12:35,460][60935] Updated weights for policy 0, policy_version 78690 (0.0008) [2023-10-14 00:12:35,831][60935] Updated weights for policy 0, policy_version 78700 (0.0008) [2023-10-14 00:12:36,192][60935] Updated weights for policy 0, policy_version 78710 (0.0007) [2023-10-14 00:12:36,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 161775616. Throughput: 0: 1712.5, 1: 1788.4. Samples: 40453000. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:12:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:12:36,562][60935] Updated weights for policy 0, policy_version 78720 (0.0007) [2023-10-14 00:12:38,900][60934] Updated weights for policy 1, policy_version 78542 (0.0007) [2023-10-14 00:12:39,258][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-14 00:12:39,263][60934] Updated weights for policy 1, policy_version 78552 (0.0007) [2023-10-14 00:12:40,605][60935] Updated weights for policy 0, policy_version 78730 (0.0010) [2023-10-14 00:12:40,972][60935] Updated weights for policy 0, policy_version 78740 (0.0009) [2023-10-14 00:12:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 161841152. Throughput: 0: 1699.6, 1: 1777.6. Samples: 40473672. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 00:12:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:12:41,333][60935] Updated weights for policy 0, policy_version 78750 (0.0010) [2023-10-14 00:12:43,539][60934] Updated weights for policy 1, policy_version 78562 (0.0009) [2023-10-14 00:12:43,751][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-14 00:12:45,488][60935] Updated weights for policy 0, policy_version 78760 (0.0010) [2023-10-14 00:12:45,852][60935] Updated weights for policy 0, policy_version 78770 (0.0011) [2023-10-14 00:12:46,231][60935] Updated weights for policy 0, policy_version 78780 (0.0009) [2023-10-14 00:12:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 161906688. Throughput: 0: 1712.8, 1: 1785.5. Samples: 40484396. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 00:12:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:12:47,905][60934] Updated weights for policy 1, policy_version 78572 (0.0009) [2023-10-14 00:12:48,271][60934] Updated weights for policy 1, policy_version 78582 (0.0009) [2023-10-14 00:12:48,637][60934] Updated weights for policy 1, policy_version 78592 (0.0007) [2023-10-14 00:12:50,343][60935] Updated weights for policy 0, policy_version 78790 (0.0008) [2023-10-14 00:12:50,720][60935] Updated weights for policy 0, policy_version 78800 (0.0011) [2023-10-14 00:12:51,074][60935] Updated weights for policy 0, policy_version 78810 (0.0011) [2023-10-14 00:12:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 161972224. Throughput: 0: 1707.3, 1: 1759.2. Samples: 40504992. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 00:12:51,250][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:12:52,655][60934] Updated weights for policy 1, policy_version 78602 (0.0008) [2023-10-14 00:12:53,015][60934] Updated weights for policy 1, policy_version 78612 (0.0009) [2023-10-14 00:12:53,386][60934] Updated weights for policy 1, policy_version 78622 (0.0010) [2023-10-14 00:12:53,759][60934] Updated weights for policy 1, policy_version 78632 (0.0007) [2023-10-14 00:12:55,055][60935] Updated weights for policy 0, policy_version 78820 (0.0010) [2023-10-14 00:12:55,430][60935] Updated weights for policy 0, policy_version 78830 (0.0008) [2023-10-14 00:12:55,795][60935] Updated weights for policy 0, policy_version 78840 (0.0009) [2023-10-14 00:12:56,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 162070528. Throughput: 0: 1686.3, 1: 1752.5. Samples: 40525310. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 00:12:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:12:56,257][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000078632_81330176.pth... [2023-10-14 00:12:56,257][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000078848_80740352.pth... [2023-10-14 00:12:56,294][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000077408_79659008.pth [2023-10-14 00:12:56,299][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000077248_79101952.pth [2023-10-14 00:12:56,300][60828] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p1/milestones/checkpoint_000078632_81330176.pth [2023-10-14 00:12:56,305][60695] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p0/milestones/checkpoint_000078848_80740352.pth [2023-10-14 00:12:57,598][60934] Updated weights for policy 1, policy_version 78642 (0.0008) [2023-10-14 00:12:57,967][60934] Updated weights for policy 1, policy_version 78652 (0.0008) [2023-10-14 00:12:58,327][60934] Updated weights for policy 1, policy_version 78662 (0.0009) [2023-10-14 00:12:59,758][60935] Updated weights for policy 0, policy_version 78850 (0.0009) [2023-10-14 00:13:00,132][60935] Updated weights for policy 0, policy_version 78860 (0.0009) [2023-10-14 00:13:00,496][60935] Updated weights for policy 0, policy_version 78870 (0.0012) [2023-10-14 00:13:00,857][60935] Updated weights for policy 0, policy_version 78880 (0.0011) [2023-10-14 00:13:01,248][59943] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 162136064. Throughput: 0: 1703.4, 1: 1754.2. Samples: 40535604. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 00:13:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:13:02,268][60934] Updated weights for policy 1, policy_version 78672 (0.0009) [2023-10-14 00:13:02,639][60934] Updated weights for policy 1, policy_version 78682 (0.0009) [2023-10-14 00:13:02,996][60934] Updated weights for policy 1, policy_version 78692 (0.0007) [2023-10-14 00:13:04,930][60935] Updated weights for policy 0, policy_version 78890 (0.0008) [2023-10-14 00:13:05,296][60935] Updated weights for policy 0, policy_version 78900 (0.0009) [2023-10-14 00:13:05,676][60935] Updated weights for policy 0, policy_version 78910 (0.0009) [2023-10-14 00:13:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 162201600. Throughput: 0: 1694.6, 1: 1765.1. Samples: 40556316. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 00:13:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:13:06,881][60934] Updated weights for policy 1, policy_version 78702 (0.0008) [2023-10-14 00:13:07,262][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-14 00:13:07,268][60934] Updated weights for policy 1, policy_version 78712 (0.0008) [2023-10-14 00:13:09,677][60935] Updated weights for policy 0, policy_version 78920 (0.0009) [2023-10-14 00:13:10,046][60935] Updated weights for policy 0, policy_version 78930 (0.0009) [2023-10-14 00:13:10,403][60935] Updated weights for policy 0, policy_version 78940 (0.0009) [2023-10-14 00:13:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 162267136. Throughput: 0: 1676.8, 1: 1783.7. Samples: 40577248. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 00:13:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:13:11,405][60934] Updated weights for policy 1, policy_version 78722 (0.0008) [2023-10-14 00:13:11,772][60934] Updated weights for policy 1, policy_version 78732 (0.0007) [2023-10-14 00:13:12,132][60934] Updated weights for policy 1, policy_version 78742 (0.0009) [2023-10-14 00:13:14,251][60935] Updated weights for policy 0, policy_version 78950 (0.0007) [2023-10-14 00:13:14,623][60935] Updated weights for policy 0, policy_version 78960 (0.0007) [2023-10-14 00:13:14,987][60935] Updated weights for policy 0, policy_version 78970 (0.0008) [2023-10-14 00:13:16,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 162332672. Throughput: 0: 1710.3, 1: 1773.8. Samples: 40588158. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 00:13:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:13:16,315][60934] Updated weights for policy 1, policy_version 78752 (0.0008) [2023-10-14 00:13:16,674][60934] Updated weights for policy 1, policy_version 78762 (0.0008) [2023-10-14 00:13:17,035][60934] Updated weights for policy 1, policy_version 78772 (0.0007) [2023-10-14 00:13:19,025][60935] Updated weights for policy 0, policy_version 78980 (0.0008) [2023-10-14 00:13:19,414][60935] Updated weights for policy 0, policy_version 78990 (0.0010) [2023-10-14 00:13:19,788][60935] Updated weights for policy 0, policy_version 79000 (0.0010) [2023-10-14 00:13:20,947][60934] Updated weights for policy 1, policy_version 78782 (0.0007) [2023-10-14 00:13:21,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13884.7). Total num frames: 162398208. Throughput: 0: 1687.5, 1: 1763.0. Samples: 40608270. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 00:13:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:13:21,312][60934] Updated weights for policy 1, policy_version 78792 (0.0007) [2023-10-14 00:13:21,679][60934] Updated weights for policy 1, policy_version 78802 (0.0007) [2023-10-14 00:13:23,772][60935] Updated weights for policy 0, policy_version 79010 (0.0008) [2023-10-14 00:13:24,138][60935] Updated weights for policy 0, policy_version 79020 (0.0007) [2023-10-14 00:13:24,501][60935] Updated weights for policy 0, policy_version 79030 (0.0009) [2023-10-14 00:13:24,869][60935] Updated weights for policy 0, policy_version 79040 (0.0007) [2023-10-14 00:13:25,502][60934] Updated weights for policy 1, policy_version 78812 (0.0008) [2023-10-14 00:13:25,860][60934] Updated weights for policy 1, policy_version 78822 (0.0009) [2023-10-14 00:13:26,223][60934] Updated weights for policy 1, policy_version 78832 (0.0008) [2023-10-14 00:13:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 162463744. Throughput: 0: 1693.4, 1: 1755.2. Samples: 40628858. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 00:13:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:13:29,013][60935] Updated weights for policy 0, policy_version 79050 (0.0009) [2023-10-14 00:13:29,393][60935] Updated weights for policy 0, policy_version 79060 (0.0011) [2023-10-14 00:13:29,765][60935] Updated weights for policy 0, policy_version 79070 (0.0010) [2023-10-14 00:13:30,116][60934] Updated weights for policy 1, policy_version 78842 (0.0007) [2023-10-14 00:13:30,481][60934] Updated weights for policy 1, policy_version 78852 (0.0010) [2023-10-14 00:13:30,833][60934] Updated weights for policy 1, policy_version 78862 (0.0007) [2023-10-14 00:13:31,207][60934] Updated weights for policy 1, policy_version 78872 (0.0007) [2023-10-14 00:13:31,248][59943] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 162562048. Throughput: 0: 1701.5, 1: 1737.4. Samples: 40639144. Policy #0 lag: (min: 18.0, avg: 21.1, max: 50.0) [2023-10-14 00:13:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:13:33,905][60935] Updated weights for policy 0, policy_version 79080 (0.0009) [2023-10-14 00:13:34,270][60935] Updated weights for policy 0, policy_version 79090 (0.0012) [2023-10-14 00:13:34,633][60935] Updated weights for policy 0, policy_version 79100 (0.0010) [2023-10-14 00:13:35,201][60934] Updated weights for policy 1, policy_version 78882 (0.0008) [2023-10-14 00:13:35,564][60934] Updated weights for policy 1, policy_version 78892 (0.0009) [2023-10-14 00:13:35,936][60934] Updated weights for policy 1, policy_version 78902 (0.0010) [2023-10-14 00:13:36,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13995.8). Total num frames: 162627584. Throughput: 0: 1674.8, 1: 1752.8. Samples: 40659234. Policy #0 lag: (min: 18.0, avg: 21.1, max: 50.0) [2023-10-14 00:13:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:13:38,501][60935] Updated weights for policy 0, policy_version 79110 (0.0007) [2023-10-14 00:13:38,872][60935] Updated weights for policy 0, policy_version 79120 (0.0011) [2023-10-14 00:13:39,236][60935] Updated weights for policy 0, policy_version 79130 (0.0008) [2023-10-14 00:13:40,035][60934] Updated weights for policy 1, policy_version 78912 (0.0009) [2023-10-14 00:13:40,404][60934] Updated weights for policy 1, policy_version 78922 (0.0009) [2023-10-14 00:13:40,770][60934] Updated weights for policy 1, policy_version 78932 (0.0007) [2023-10-14 00:13:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 162693120. Throughput: 0: 1700.3, 1: 1730.9. Samples: 40679712. Policy #0 lag: (min: 18.0, avg: 21.1, max: 50.0) [2023-10-14 00:13:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:13:43,297][60935] Updated weights for policy 0, policy_version 79140 (0.0008) [2023-10-14 00:13:43,661][60935] Updated weights for policy 0, policy_version 79150 (0.0007) [2023-10-14 00:13:44,027][60935] Updated weights for policy 0, policy_version 79160 (0.0008) [2023-10-14 00:13:44,559][60934] Updated weights for policy 1, policy_version 78942 (0.0008) [2023-10-14 00:13:44,926][60934] Updated weights for policy 1, policy_version 78952 (0.0010) [2023-10-14 00:13:45,284][60934] Updated weights for policy 1, policy_version 78962 (0.0009) [2023-10-14 00:13:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 162758656. Throughput: 0: 1693.6, 1: 1748.0. Samples: 40690474. Policy #0 lag: (min: 18.0, avg: 21.1, max: 50.0) [2023-10-14 00:13:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:13:47,964][60935] Updated weights for policy 0, policy_version 79170 (0.0009) [2023-10-14 00:13:48,332][60935] Updated weights for policy 0, policy_version 79180 (0.0010) [2023-10-14 00:13:48,695][60935] Updated weights for policy 0, policy_version 79190 (0.0007) [2023-10-14 00:13:49,068][60935] Updated weights for policy 0, policy_version 79200 (0.0007) [2023-10-14 00:13:49,463][60934] Updated weights for policy 1, policy_version 78972 (0.0010) [2023-10-14 00:13:49,824][60934] Updated weights for policy 1, policy_version 78982 (0.0010) [2023-10-14 00:13:50,188][60934] Updated weights for policy 1, policy_version 78992 (0.0009) [2023-10-14 00:13:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 162824192. Throughput: 0: 1693.6, 1: 1740.2. Samples: 40710838. Policy #0 lag: (min: 18.0, avg: 21.1, max: 50.0) [2023-10-14 00:13:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:13:52,952][60935] Updated weights for policy 0, policy_version 79210 (0.0011) [2023-10-14 00:13:53,321][60935] Updated weights for policy 0, policy_version 79220 (0.0008) [2023-10-14 00:13:53,686][60935] Updated weights for policy 0, policy_version 79230 (0.0007) [2023-10-14 00:13:54,268][60934] Updated weights for policy 1, policy_version 79002 (0.0010) [2023-10-14 00:13:54,681][60934] Updated weights for policy 1, policy_version 79012 (0.0008) [2023-10-14 00:13:55,048][60934] Updated weights for policy 1, policy_version 79022 (0.0007) [2023-10-14 00:13:55,411][60934] Updated weights for policy 1, policy_version 79032 (0.0011) [2023-10-14 00:13:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 162889728. Throughput: 0: 1716.0, 1: 1698.9. Samples: 40730918. Policy #0 lag: (min: 18.0, avg: 21.1, max: 50.0) [2023-10-14 00:13:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:13:57,656][60935] Updated weights for policy 0, policy_version 79240 (0.0008) [2023-10-14 00:13:58,023][60935] Updated weights for policy 0, policy_version 79250 (0.0008) [2023-10-14 00:13:58,396][60935] Updated weights for policy 0, policy_version 79260 (0.0009) [2023-10-14 00:13:59,422][60934] Updated weights for policy 1, policy_version 79042 (0.0007) [2023-10-14 00:13:59,794][60934] Updated weights for policy 1, policy_version 79052 (0.0007) [2023-10-14 00:14:00,166][60934] Updated weights for policy 1, policy_version 79062 (0.0008) [2023-10-14 00:14:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 162955264. Throughput: 0: 1680.5, 1: 1724.9. Samples: 40741402. Policy #0 lag: (min: 18.0, avg: 21.1, max: 50.0) [2023-10-14 00:14:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:14:02,412][60935] Updated weights for policy 0, policy_version 79270 (0.0008) [2023-10-14 00:14:02,780][60935] Updated weights for policy 0, policy_version 79280 (0.0007) [2023-10-14 00:14:03,146][60935] Updated weights for policy 0, policy_version 79290 (0.0007) [2023-10-14 00:14:04,099][60934] Updated weights for policy 1, policy_version 79072 (0.0007) [2023-10-14 00:14:04,458][60934] Updated weights for policy 1, policy_version 79082 (0.0007) [2023-10-14 00:14:04,823][60934] Updated weights for policy 1, policy_version 79092 (0.0009) [2023-10-14 00:14:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 163020800. Throughput: 0: 1707.3, 1: 1706.6. Samples: 40761896. Policy #0 lag: (min: 18.0, avg: 21.1, max: 50.0) [2023-10-14 00:14:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:14:07,246][60935] Updated weights for policy 0, policy_version 79300 (0.0008) [2023-10-14 00:14:07,630][60935] Updated weights for policy 0, policy_version 79310 (0.0010) [2023-10-14 00:14:08,002][60935] Updated weights for policy 0, policy_version 79320 (0.0008) [2023-10-14 00:14:08,883][60934] Updated weights for policy 1, policy_version 79102 (0.0010) [2023-10-14 00:14:09,240][60934] Updated weights for policy 1, policy_version 79112 (0.0009) [2023-10-14 00:14:09,612][60934] Updated weights for policy 1, policy_version 79122 (0.0007) [2023-10-14 00:14:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 163086336. Throughput: 0: 1711.1, 1: 1699.2. Samples: 40782320. Policy #0 lag: (min: 18.0, avg: 21.1, max: 50.0) [2023-10-14 00:14:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:14:12,021][60935] Updated weights for policy 0, policy_version 79330 (0.0009) [2023-10-14 00:14:12,393][60935] Updated weights for policy 0, policy_version 79340 (0.0008) [2023-10-14 00:14:12,761][60935] Updated weights for policy 0, policy_version 79350 (0.0010) [2023-10-14 00:14:13,134][60935] Updated weights for policy 0, policy_version 79360 (0.0007) [2023-10-14 00:14:13,547][60934] Updated weights for policy 1, policy_version 79132 (0.0008) [2023-10-14 00:14:13,909][60934] Updated weights for policy 1, policy_version 79142 (0.0007) [2023-10-14 00:14:14,271][60934] Updated weights for policy 1, policy_version 79152 (0.0007) [2023-10-14 00:14:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 163151872. Throughput: 0: 1691.5, 1: 1724.8. Samples: 40792874. Policy #0 lag: (min: 18.0, avg: 21.1, max: 50.0) [2023-10-14 00:14:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:14:17,045][60935] Updated weights for policy 0, policy_version 79370 (0.0012) [2023-10-14 00:14:17,415][60935] Updated weights for policy 0, policy_version 79380 (0.0010) [2023-10-14 00:14:17,775][60935] Updated weights for policy 0, policy_version 79390 (0.0010) [2023-10-14 00:14:18,391][60934] Updated weights for policy 1, policy_version 79162 (0.0007) [2023-10-14 00:14:18,759][60934] Updated weights for policy 1, policy_version 79172 (0.0007) [2023-10-14 00:14:19,118][60934] Updated weights for policy 1, policy_version 79182 (0.0008) [2023-10-14 00:14:19,480][60828] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000005 [2023-10-14 00:14:19,481][60934] Updated weights for policy 1, policy_version 79192 (0.0010) [2023-10-14 00:14:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 163217408. Throughput: 0: 1720.8, 1: 1692.0. Samples: 40812808. Policy #0 lag: (min: 18.0, avg: 21.1, max: 50.0) [2023-10-14 00:14:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:14:21,772][60935] Updated weights for policy 0, policy_version 79400 (0.0010) [2023-10-14 00:14:22,147][60935] Updated weights for policy 0, policy_version 79410 (0.0008) [2023-10-14 00:14:22,515][60935] Updated weights for policy 0, policy_version 79420 (0.0007) [2023-10-14 00:14:23,304][60934] Updated weights for policy 1, policy_version 79202 (0.0008) [2023-10-14 00:14:23,663][60934] Updated weights for policy 1, policy_version 79212 (0.0008) [2023-10-14 00:14:24,039][60934] Updated weights for policy 1, policy_version 79222 (0.0009) [2023-10-14 00:14:24,110][60828] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000006 [2023-10-14 00:14:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 163282944. Throughput: 0: 1717.3, 1: 1712.6. Samples: 40834058. Policy #0 lag: (min: 18.0, avg: 20.6, max: 50.0) [2023-10-14 00:14:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:14:26,438][60935] Updated weights for policy 0, policy_version 79430 (0.0007) [2023-10-14 00:14:26,815][60935] Updated weights for policy 0, policy_version 79440 (0.0009) [2023-10-14 00:14:27,171][60935] Updated weights for policy 0, policy_version 79450 (0.0009) [2023-10-14 00:14:28,069][60934] Updated weights for policy 1, policy_version 79232 (0.0008) [2023-10-14 00:14:28,439][60934] Updated weights for policy 1, policy_version 79242 (0.0007) [2023-10-14 00:14:28,804][60934] Updated weights for policy 1, policy_version 79252 (0.0007) [2023-10-14 00:14:31,126][60935] Updated weights for policy 0, policy_version 79460 (0.0009) [2023-10-14 00:14:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13773.7). Total num frames: 163348480. Throughput: 0: 1705.6, 1: 1704.7. Samples: 40843938. Policy #0 lag: (min: 18.0, avg: 20.6, max: 50.0) [2023-10-14 00:14:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:14:31,503][60935] Updated weights for policy 0, policy_version 79470 (0.0009) [2023-10-14 00:14:31,861][60935] Updated weights for policy 0, policy_version 79480 (0.0009) [2023-10-14 00:14:32,772][60934] Updated weights for policy 1, policy_version 79262 (0.0007) [2023-10-14 00:14:33,141][60934] Updated weights for policy 1, policy_version 79272 (0.0007) [2023-10-14 00:14:33,504][60934] Updated weights for policy 1, policy_version 79282 (0.0007) [2023-10-14 00:14:33,725][60828] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000008 [2023-10-14 00:14:35,884][60935] Updated weights for policy 0, policy_version 79490 (0.0008) [2023-10-14 00:14:36,248][60935] Updated weights for policy 0, policy_version 79500 (0.0008) [2023-10-14 00:14:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13773.7). Total num frames: 163414016. Throughput: 0: 1718.3, 1: 1698.7. Samples: 40864600. Policy #0 lag: (min: 18.0, avg: 20.6, max: 50.0) [2023-10-14 00:14:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:14:36,620][60935] Updated weights for policy 0, policy_version 79510 (0.0010) [2023-10-14 00:14:36,981][60935] Updated weights for policy 0, policy_version 79520 (0.0011) [2023-10-14 00:14:37,305][60934] Updated weights for policy 1, policy_version 79292 (0.0007) [2023-10-14 00:14:37,675][60934] Updated weights for policy 1, policy_version 79302 (0.0008) [2023-10-14 00:14:38,037][60934] Updated weights for policy 1, policy_version 79312 (0.0010) [2023-10-14 00:14:40,979][60935] Updated weights for policy 0, policy_version 79530 (0.0009) [2023-10-14 00:14:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13773.7). Total num frames: 163479552. Throughput: 0: 1712.4, 1: 1724.8. Samples: 40885596. Policy #0 lag: (min: 18.0, avg: 20.6, max: 50.0) [2023-10-14 00:14:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:14:41,336][60935] Updated weights for policy 0, policy_version 79540 (0.0010) [2023-10-14 00:14:41,701][60935] Updated weights for policy 0, policy_version 79550 (0.0012) [2023-10-14 00:14:42,087][60934] Updated weights for policy 1, policy_version 79322 (0.0009) [2023-10-14 00:14:42,490][60934] Updated weights for policy 1, policy_version 79332 (0.0007) [2023-10-14 00:14:42,850][60934] Updated weights for policy 1, policy_version 79342 (0.0009) [2023-10-14 00:14:43,220][60934] Updated weights for policy 1, policy_version 79352 (0.0010) [2023-10-14 00:14:45,820][60935] Updated weights for policy 0, policy_version 79560 (0.0009) [2023-10-14 00:14:46,195][60935] Updated weights for policy 0, policy_version 79570 (0.0010) [2023-10-14 00:14:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13773.7). Total num frames: 163545088. Throughput: 0: 1716.8, 1: 1694.0. Samples: 40894888. Policy #0 lag: (min: 18.0, avg: 20.6, max: 50.0) [2023-10-14 00:14:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:14:46,568][60935] Updated weights for policy 0, policy_version 79580 (0.0008) [2023-10-14 00:14:47,052][60934] Updated weights for policy 1, policy_version 79362 (0.0008) [2023-10-14 00:14:47,416][60934] Updated weights for policy 1, policy_version 79372 (0.0009) [2023-10-14 00:14:47,563][60828] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000005 [2023-10-14 00:14:50,460][60935] Updated weights for policy 0, policy_version 79590 (0.0008) [2023-10-14 00:14:50,828][60935] Updated weights for policy 0, policy_version 79600 (0.0010) [2023-10-14 00:14:51,201][60935] Updated weights for policy 0, policy_version 79610 (0.0009) [2023-10-14 00:14:51,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13773.7). Total num frames: 163610624. Throughput: 0: 1712.4, 1: 1715.4. Samples: 40916148. Policy #0 lag: (min: 18.0, avg: 20.6, max: 50.0) [2023-10-14 00:14:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:14:51,667][60934] Updated weights for policy 1, policy_version 79382 (0.0008) [2023-10-14 00:14:52,023][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000002 [2023-10-14 00:14:52,029][60934] Updated weights for policy 1, policy_version 79392 (0.0009) [2023-10-14 00:14:55,216][60935] Updated weights for policy 0, policy_version 79620 (0.0008) [2023-10-14 00:14:55,608][60935] Updated weights for policy 0, policy_version 79630 (0.0010) [2023-10-14 00:14:55,980][60935] Updated weights for policy 0, policy_version 79640 (0.0009) [2023-10-14 00:14:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13773.7). Total num frames: 163676160. Throughput: 0: 1697.4, 1: 1744.2. Samples: 40937192. Policy #0 lag: (min: 18.0, avg: 20.6, max: 50.0) [2023-10-14 00:14:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:14:56,276][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000079648_81559552.pth... [2023-10-14 00:14:56,310][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000078048_79921152.pth [2023-10-14 00:14:56,321][60934] Updated weights for policy 1, policy_version 79402 (0.0007) [2023-10-14 00:14:56,536][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-14 00:14:56,538][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000079408_82182144.pth... [2023-10-14 00:14:56,576][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000078096_80478208.pth [2023-10-14 00:14:59,845][60935] Updated weights for policy 0, policy_version 79650 (0.0009) [2023-10-14 00:15:00,215][60935] Updated weights for policy 0, policy_version 79660 (0.0010) [2023-10-14 00:15:00,584][60935] Updated weights for policy 0, policy_version 79670 (0.0008) [2023-10-14 00:15:00,629][60934] Updated weights for policy 1, policy_version 79412 (0.0008) [2023-10-14 00:15:00,955][60935] Updated weights for policy 0, policy_version 79680 (0.0008) [2023-10-14 00:15:00,998][60934] Updated weights for policy 1, policy_version 79422 (0.0008) [2023-10-14 00:15:01,063][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000002 [2023-10-14 00:15:01,248][59943] Fps is (10 sec: 19660.7, 60 sec: 14199.4, 300 sec: 13995.8). Total num frames: 163807232. Throughput: 0: 1715.5, 1: 1729.0. Samples: 40947878. Policy #0 lag: (min: 18.0, avg: 20.6, max: 50.0) [2023-10-14 00:15:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:15:05,014][60935] Updated weights for policy 0, policy_version 79690 (0.0010) [2023-10-14 00:15:05,192][60934] Updated weights for policy 1, policy_version 79432 (0.0008) [2023-10-14 00:15:05,386][60935] Updated weights for policy 0, policy_version 79700 (0.0009) [2023-10-14 00:15:05,473][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-14 00:15:05,751][60935] Updated weights for policy 0, policy_version 79710 (0.0009) [2023-10-14 00:15:06,248][59943] Fps is (10 sec: 19660.7, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 163872768. Throughput: 0: 1712.4, 1: 1767.4. Samples: 40969396. Policy #0 lag: (min: 18.0, avg: 20.6, max: 50.0) [2023-10-14 00:15:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:15:09,450][60934] Updated weights for policy 1, policy_version 79442 (0.0007) [2023-10-14 00:15:09,700][60935] Updated weights for policy 0, policy_version 79720 (0.0007) [2023-10-14 00:15:09,809][60934] Updated weights for policy 1, policy_version 79452 (0.0007) [2023-10-14 00:15:09,953][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:15:10,078][60935] Updated weights for policy 0, policy_version 79730 (0.0010) [2023-10-14 00:15:10,449][60935] Updated weights for policy 0, policy_version 79740 (0.0009) [2023-10-14 00:15:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13995.8). Total num frames: 163938304. Throughput: 0: 1683.3, 1: 1769.5. Samples: 40989434. Policy #0 lag: (min: 18.0, avg: 20.6, max: 50.0) [2023-10-14 00:15:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:15:14,090][60934] Updated weights for policy 1, policy_version 79462 (0.0009) [2023-10-14 00:15:14,447][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-14 00:15:14,452][60934] Updated weights for policy 1, policy_version 79472 (0.0009) [2023-10-14 00:15:14,626][60935] Updated weights for policy 0, policy_version 79750 (0.0008) [2023-10-14 00:15:14,990][60935] Updated weights for policy 0, policy_version 79760 (0.0009) [2023-10-14 00:15:15,359][60935] Updated weights for policy 0, policy_version 79770 (0.0008) [2023-10-14 00:15:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 164003840. Throughput: 0: 1709.5, 1: 1783.6. Samples: 41001126. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-14 00:15:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:15:18,803][60934] Updated weights for policy 1, policy_version 79482 (0.0007) [2023-10-14 00:15:19,018][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:15:19,396][60935] Updated weights for policy 0, policy_version 79780 (0.0010) [2023-10-14 00:15:19,762][60935] Updated weights for policy 0, policy_version 79790 (0.0007) [2023-10-14 00:15:20,135][60935] Updated weights for policy 0, policy_version 79800 (0.0008) [2023-10-14 00:15:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13995.8). Total num frames: 164069376. Throughput: 0: 1697.3, 1: 1793.1. Samples: 41021666. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-14 00:15:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:15:22,942][60934] Updated weights for policy 1, policy_version 79492 (0.0009) [2023-10-14 00:15:23,308][60934] Updated weights for policy 1, policy_version 79502 (0.0008) [2023-10-14 00:15:23,375][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000002 [2023-10-14 00:15:24,022][60935] Updated weights for policy 0, policy_version 79810 (0.0009) [2023-10-14 00:15:24,383][60935] Updated weights for policy 0, policy_version 79820 (0.0009) [2023-10-14 00:15:24,757][60935] Updated weights for policy 0, policy_version 79830 (0.0009) [2023-10-14 00:15:25,110][60935] Updated weights for policy 0, policy_version 79840 (0.0008) [2023-10-14 00:15:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 164134912. Throughput: 0: 1692.1, 1: 1806.7. Samples: 41043042. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-14 00:15:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:15:27,577][60934] Updated weights for policy 1, policy_version 79512 (0.0008) [2023-10-14 00:15:27,884][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000002 [2023-10-14 00:15:29,277][60935] Updated weights for policy 0, policy_version 79850 (0.0011) [2023-10-14 00:15:29,636][60935] Updated weights for policy 0, policy_version 79860 (0.0011) [2023-10-14 00:15:29,995][60935] Updated weights for policy 0, policy_version 79870 (0.0010) [2023-10-14 00:15:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 164200448. Throughput: 0: 1714.0, 1: 1820.0. Samples: 41053918. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-14 00:15:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:15:31,820][60934] Updated weights for policy 1, policy_version 79522 (0.0007) [2023-10-14 00:15:32,197][60934] Updated weights for policy 1, policy_version 79532 (0.0009) [2023-10-14 00:15:32,335][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:15:33,933][60935] Updated weights for policy 0, policy_version 79880 (0.0008) [2023-10-14 00:15:34,296][60935] Updated weights for policy 0, policy_version 79890 (0.0008) [2023-10-14 00:15:34,662][60935] Updated weights for policy 0, policy_version 79900 (0.0008) [2023-10-14 00:15:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 164265984. Throughput: 0: 1688.5, 1: 1830.8. Samples: 41074516. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-14 00:15:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:15:36,457][60934] Updated weights for policy 1, policy_version 79542 (0.0010) [2023-10-14 00:15:36,818][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:15:36,822][60934] Updated weights for policy 1, policy_version 79552 (0.0009) [2023-10-14 00:15:38,538][60935] Updated weights for policy 0, policy_version 79910 (0.0009) [2023-10-14 00:15:38,908][60935] Updated weights for policy 0, policy_version 79920 (0.0008) [2023-10-14 00:15:39,285][60935] Updated weights for policy 0, policy_version 79930 (0.0009) [2023-10-14 00:15:41,239][60934] Updated weights for policy 1, policy_version 79562 (0.0008) [2023-10-14 00:15:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 164331520. Throughput: 0: 1707.1, 1: 1824.6. Samples: 41096118. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-14 00:15:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:15:41,455][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000005 [2023-10-14 00:15:43,332][60935] Updated weights for policy 0, policy_version 79940 (0.0009) [2023-10-14 00:15:43,728][60935] Updated weights for policy 0, policy_version 79950 (0.0010) [2023-10-14 00:15:44,112][60935] Updated weights for policy 0, policy_version 79960 (0.0009) [2023-10-14 00:15:45,506][60934] Updated weights for policy 1, policy_version 79572 (0.0008) [2023-10-14 00:15:45,875][60934] Updated weights for policy 1, policy_version 79582 (0.0008) [2023-10-14 00:15:46,227][60828] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000001 [2023-10-14 00:15:46,232][60934] Updated weights for policy 1, policy_version 79592 (0.0007) [2023-10-14 00:15:46,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 13884.7). Total num frames: 164429824. Throughput: 0: 1700.8, 1: 1824.7. Samples: 41106524. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-14 00:15:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:15:48,125][60935] Updated weights for policy 0, policy_version 79970 (0.0009) [2023-10-14 00:15:48,500][60935] Updated weights for policy 0, policy_version 79980 (0.0009) [2023-10-14 00:15:48,870][60935] Updated weights for policy 0, policy_version 79990 (0.0008) [2023-10-14 00:15:49,233][60935] Updated weights for policy 0, policy_version 80000 (0.0009) [2023-10-14 00:15:50,381][60934] Updated weights for policy 1, policy_version 79602 (0.0009) [2023-10-14 00:15:50,754][60934] Updated weights for policy 1, policy_version 79612 (0.0011) [2023-10-14 00:15:50,893][60828] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000001 [2023-10-14 00:15:51,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 13884.8). Total num frames: 164495360. Throughput: 0: 1686.0, 1: 1825.4. Samples: 41127408. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-14 00:15:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:15:53,125][60935] Updated weights for policy 0, policy_version 80010 (0.0008) [2023-10-14 00:15:53,490][60935] Updated weights for policy 0, policy_version 80020 (0.0010) [2023-10-14 00:15:53,866][60935] Updated weights for policy 0, policy_version 80030 (0.0009) [2023-10-14 00:15:54,937][60934] Updated weights for policy 1, policy_version 79622 (0.0009) [2023-10-14 00:15:55,297][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:15:55,303][60934] Updated weights for policy 1, policy_version 79632 (0.0009) [2023-10-14 00:15:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 13884.7). Total num frames: 164560896. Throughput: 0: 1717.6, 1: 1811.2. Samples: 41148232. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-14 00:15:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:15:57,888][60935] Updated weights for policy 0, policy_version 80040 (0.0008) [2023-10-14 00:15:58,258][60935] Updated weights for policy 0, policy_version 80050 (0.0010) [2023-10-14 00:15:58,622][60935] Updated weights for policy 0, policy_version 80060 (0.0008) [2023-10-14 00:15:59,495][60934] Updated weights for policy 1, policy_version 79642 (0.0009) [2023-10-14 00:15:59,868][60934] Updated weights for policy 1, policy_version 79652 (0.0009) [2023-10-14 00:16:00,231][60934] Updated weights for policy 1, policy_version 79662 (0.0008) [2023-10-14 00:16:01,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 164626432. Throughput: 0: 1688.9, 1: 1817.1. Samples: 41158894. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-14 00:16:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:16:02,679][60935] Updated weights for policy 0, policy_version 80070 (0.0010) [2023-10-14 00:16:03,054][60935] Updated weights for policy 0, policy_version 80080 (0.0008) [2023-10-14 00:16:03,421][60935] Updated weights for policy 0, policy_version 80090 (0.0011) [2023-10-14 00:16:04,162][60934] Updated weights for policy 1, policy_version 79672 (0.0007) [2023-10-14 00:16:04,533][60934] Updated weights for policy 1, policy_version 79682 (0.0008) [2023-10-14 00:16:04,892][60934] Updated weights for policy 1, policy_version 79692 (0.0011) [2023-10-14 00:16:05,037][60828] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000008 [2023-10-14 00:16:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13884.8). Total num frames: 164691968. Throughput: 0: 1704.1, 1: 1804.7. Samples: 41179564. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-14 00:16:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:16:07,301][60935] Updated weights for policy 0, policy_version 80100 (0.0010) [2023-10-14 00:16:07,675][60935] Updated weights for policy 0, policy_version 80110 (0.0007) [2023-10-14 00:16:08,053][60935] Updated weights for policy 0, policy_version 80120 (0.0007) [2023-10-14 00:16:08,784][60934] Updated weights for policy 1, policy_version 79702 (0.0008) [2023-10-14 00:16:09,149][60934] Updated weights for policy 1, policy_version 79712 (0.0009) [2023-10-14 00:16:09,516][60934] Updated weights for policy 1, policy_version 79722 (0.0009) [2023-10-14 00:16:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 164757504. Throughput: 0: 1724.7, 1: 1773.1. Samples: 41200442. Policy #0 lag: (min: 18.0, avg: 18.9, max: 38.0) [2023-10-14 00:16:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:16:11,771][60935] Updated weights for policy 0, policy_version 80130 (0.0009) [2023-10-14 00:16:12,133][60935] Updated weights for policy 0, policy_version 80140 (0.0009) [2023-10-14 00:16:12,506][60935] Updated weights for policy 0, policy_version 80150 (0.0008) [2023-10-14 00:16:12,874][60935] Updated weights for policy 0, policy_version 80160 (0.0010) [2023-10-14 00:16:13,620][60934] Updated weights for policy 1, policy_version 79732 (0.0008) [2023-10-14 00:16:14,029][60934] Updated weights for policy 1, policy_version 79742 (0.0009) [2023-10-14 00:16:14,394][60934] Updated weights for policy 1, policy_version 79752 (0.0009) [2023-10-14 00:16:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 164823040. Throughput: 0: 1700.8, 1: 1789.6. Samples: 41210986. Policy #0 lag: (min: 18.0, avg: 18.9, max: 38.0) [2023-10-14 00:16:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:16:16,963][60935] Updated weights for policy 0, policy_version 80170 (0.0008) [2023-10-14 00:16:17,328][60935] Updated weights for policy 0, policy_version 80180 (0.0011) [2023-10-14 00:16:17,704][60935] Updated weights for policy 0, policy_version 80190 (0.0009) [2023-10-14 00:16:18,305][60934] Updated weights for policy 1, policy_version 79762 (0.0010) [2023-10-14 00:16:18,671][60934] Updated weights for policy 1, policy_version 79772 (0.0009) [2023-10-14 00:16:19,037][60934] Updated weights for policy 1, policy_version 79782 (0.0007) [2023-10-14 00:16:19,403][60934] Updated weights for policy 1, policy_version 79792 (0.0007) [2023-10-14 00:16:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 164888576. Throughput: 0: 1735.1, 1: 1743.3. Samples: 41231042. Policy #0 lag: (min: 18.0, avg: 18.9, max: 38.0) [2023-10-14 00:16:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:16:21,543][60935] Updated weights for policy 0, policy_version 80200 (0.0010) [2023-10-14 00:16:21,899][60935] Updated weights for policy 0, policy_version 80210 (0.0008) [2023-10-14 00:16:22,276][60935] Updated weights for policy 0, policy_version 80220 (0.0007) [2023-10-14 00:16:23,477][60934] Updated weights for policy 1, policy_version 79802 (0.0010) [2023-10-14 00:16:23,832][60934] Updated weights for policy 1, policy_version 79812 (0.0009) [2023-10-14 00:16:24,203][60934] Updated weights for policy 1, policy_version 79822 (0.0008) [2023-10-14 00:16:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 164954112. Throughput: 0: 1737.0, 1: 1727.2. Samples: 41252008. Policy #0 lag: (min: 18.0, avg: 18.9, max: 38.0) [2023-10-14 00:16:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:16:26,284][60935] Updated weights for policy 0, policy_version 80230 (0.0009) [2023-10-14 00:16:26,651][60935] Updated weights for policy 0, policy_version 80240 (0.0008) [2023-10-14 00:16:27,023][60935] Updated weights for policy 0, policy_version 80250 (0.0008) [2023-10-14 00:16:28,187][60934] Updated weights for policy 1, policy_version 79832 (0.0008) [2023-10-14 00:16:28,545][60934] Updated weights for policy 1, policy_version 79842 (0.0010) [2023-10-14 00:16:28,901][60934] Updated weights for policy 1, policy_version 79852 (0.0009) [2023-10-14 00:16:30,981][60935] Updated weights for policy 0, policy_version 80260 (0.0008) [2023-10-14 00:16:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 165019648. Throughput: 0: 1724.7, 1: 1728.7. Samples: 41261926. Policy #0 lag: (min: 18.0, avg: 18.9, max: 38.0) [2023-10-14 00:16:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:16:31,372][60935] Updated weights for policy 0, policy_version 80270 (0.0009) [2023-10-14 00:16:31,734][60935] Updated weights for policy 0, policy_version 80280 (0.0011) [2023-10-14 00:16:32,817][60934] Updated weights for policy 1, policy_version 79862 (0.0012) [2023-10-14 00:16:33,176][60934] Updated weights for policy 1, policy_version 79872 (0.0010) [2023-10-14 00:16:33,545][60934] Updated weights for policy 1, policy_version 79882 (0.0010) [2023-10-14 00:16:35,799][60935] Updated weights for policy 0, policy_version 80290 (0.0010) [2023-10-14 00:16:36,171][60935] Updated weights for policy 0, policy_version 80300 (0.0007) [2023-10-14 00:16:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 165085184. Throughput: 0: 1740.6, 1: 1705.5. Samples: 41282482. Policy #0 lag: (min: 18.0, avg: 18.9, max: 38.0) [2023-10-14 00:16:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:16:36,546][60935] Updated weights for policy 0, policy_version 80310 (0.0009) [2023-10-14 00:16:36,917][60935] Updated weights for policy 0, policy_version 80320 (0.0010) [2023-10-14 00:16:37,613][60934] Updated weights for policy 1, policy_version 79892 (0.0008) [2023-10-14 00:16:37,985][60934] Updated weights for policy 1, policy_version 79902 (0.0008) [2023-10-14 00:16:38,344][60934] Updated weights for policy 1, policy_version 79912 (0.0008) [2023-10-14 00:16:40,954][60935] Updated weights for policy 0, policy_version 80330 (0.0007) [2023-10-14 00:16:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 165150720. Throughput: 0: 1729.6, 1: 1710.9. Samples: 41303054. Policy #0 lag: (min: 18.0, avg: 18.9, max: 38.0) [2023-10-14 00:16:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:16:41,308][60935] Updated weights for policy 0, policy_version 80340 (0.0008) [2023-10-14 00:16:41,677][60935] Updated weights for policy 0, policy_version 80350 (0.0008) [2023-10-14 00:16:42,400][60934] Updated weights for policy 1, policy_version 79922 (0.0009) [2023-10-14 00:16:42,753][60934] Updated weights for policy 1, policy_version 79932 (0.0008) [2023-10-14 00:16:43,119][60934] Updated weights for policy 1, policy_version 79942 (0.0007) [2023-10-14 00:16:43,487][60934] Updated weights for policy 1, policy_version 79952 (0.0007) [2023-10-14 00:16:45,660][60935] Updated weights for policy 0, policy_version 80360 (0.0010) [2023-10-14 00:16:46,032][60935] Updated weights for policy 0, policy_version 80370 (0.0008) [2023-10-14 00:16:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13773.7). Total num frames: 165216256. Throughput: 0: 1738.6, 1: 1677.6. Samples: 41312622. Policy #0 lag: (min: 18.0, avg: 18.9, max: 38.0) [2023-10-14 00:16:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:16:46,402][60935] Updated weights for policy 0, policy_version 80380 (0.0010) [2023-10-14 00:16:47,499][60934] Updated weights for policy 1, policy_version 79962 (0.0007) [2023-10-14 00:16:47,860][60934] Updated weights for policy 1, policy_version 79972 (0.0008) [2023-10-14 00:16:48,223][60934] Updated weights for policy 1, policy_version 79982 (0.0008) [2023-10-14 00:16:50,348][60935] Updated weights for policy 0, policy_version 80390 (0.0010) [2023-10-14 00:16:50,718][60935] Updated weights for policy 0, policy_version 80400 (0.0008) [2023-10-14 00:16:51,087][60935] Updated weights for policy 0, policy_version 80410 (0.0011) [2023-10-14 00:16:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13773.7). Total num frames: 165281792. Throughput: 0: 1734.8, 1: 1692.9. Samples: 41333810. Policy #0 lag: (min: 18.0, avg: 18.9, max: 38.0) [2023-10-14 00:16:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:16:52,330][60934] Updated weights for policy 1, policy_version 79992 (0.0008) [2023-10-14 00:16:52,690][60934] Updated weights for policy 1, policy_version 80002 (0.0009) [2023-10-14 00:16:53,052][60934] Updated weights for policy 1, policy_version 80012 (0.0008) [2023-10-14 00:16:55,038][60935] Updated weights for policy 0, policy_version 80420 (0.0009) [2023-10-14 00:16:55,411][60935] Updated weights for policy 0, policy_version 80430 (0.0008) [2023-10-14 00:16:55,781][60935] Updated weights for policy 0, policy_version 80440 (0.0009) [2023-10-14 00:16:56,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 165380096. Throughput: 0: 1704.8, 1: 1711.4. Samples: 41354170. Policy #0 lag: (min: 18.0, avg: 18.9, max: 38.0) [2023-10-14 00:16:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:16:56,256][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000080016_83001344.pth... [2023-10-14 00:16:56,256][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000080448_82378752.pth... [2023-10-14 00:16:56,286][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000078632_81330176.pth [2023-10-14 00:16:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000078848_80740352.pth [2023-10-14 00:16:56,964][60934] Updated weights for policy 1, policy_version 80022 (0.0008) [2023-10-14 00:16:57,327][60934] Updated weights for policy 1, policy_version 80032 (0.0010) [2023-10-14 00:16:57,697][60934] Updated weights for policy 1, policy_version 80042 (0.0008) [2023-10-14 00:16:59,670][60935] Updated weights for policy 0, policy_version 80450 (0.0009) [2023-10-14 00:17:00,038][60935] Updated weights for policy 0, policy_version 80460 (0.0007) [2023-10-14 00:17:00,404][60935] Updated weights for policy 0, policy_version 80470 (0.0008) [2023-10-14 00:17:00,770][60935] Updated weights for policy 0, policy_version 80480 (0.0009) [2023-10-14 00:17:01,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 165445632. Throughput: 0: 1726.3, 1: 1687.1. Samples: 41364586. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:17:01,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:17:01,844][60934] Updated weights for policy 1, policy_version 80052 (0.0008) [2023-10-14 00:17:02,233][60934] Updated weights for policy 1, policy_version 80062 (0.0008) [2023-10-14 00:17:02,591][60934] Updated weights for policy 1, policy_version 80072 (0.0007) [2023-10-14 00:17:04,767][60935] Updated weights for policy 0, policy_version 80490 (0.0010) [2023-10-14 00:17:05,139][60935] Updated weights for policy 0, policy_version 80500 (0.0007) [2023-10-14 00:17:05,518][60935] Updated weights for policy 0, policy_version 80510 (0.0008) [2023-10-14 00:17:06,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 165511168. Throughput: 0: 1712.0, 1: 1716.3. Samples: 41385314. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:17:06,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:17:06,425][60934] Updated weights for policy 1, policy_version 80082 (0.0008) [2023-10-14 00:17:06,787][60934] Updated weights for policy 1, policy_version 80092 (0.0009) [2023-10-14 00:17:07,161][60934] Updated weights for policy 1, policy_version 80102 (0.0009) [2023-10-14 00:17:07,522][60934] Updated weights for policy 1, policy_version 80112 (0.0010) [2023-10-14 00:17:09,404][60935] Updated weights for policy 0, policy_version 80520 (0.0009) [2023-10-14 00:17:09,772][60935] Updated weights for policy 0, policy_version 80530 (0.0009) [2023-10-14 00:17:10,139][60935] Updated weights for policy 0, policy_version 80540 (0.0009) [2023-10-14 00:17:11,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 165576704. Throughput: 0: 1691.7, 1: 1722.8. Samples: 41405662. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:17:11,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:17:11,549][60934] Updated weights for policy 1, policy_version 80122 (0.0009) [2023-10-14 00:17:11,910][60934] Updated weights for policy 1, policy_version 80132 (0.0009) [2023-10-14 00:17:12,278][60934] Updated weights for policy 1, policy_version 80142 (0.0009) [2023-10-14 00:17:14,182][60935] Updated weights for policy 0, policy_version 80550 (0.0009) [2023-10-14 00:17:14,542][60935] Updated weights for policy 0, policy_version 80560 (0.0009) [2023-10-14 00:17:14,910][60935] Updated weights for policy 0, policy_version 80570 (0.0010) [2023-10-14 00:17:16,107][60934] Updated weights for policy 1, policy_version 80152 (0.0007) [2023-10-14 00:17:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 165642240. Throughput: 0: 1719.5, 1: 1708.4. Samples: 41416180. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:17:16,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:17:16,469][60934] Updated weights for policy 1, policy_version 80162 (0.0008) [2023-10-14 00:17:16,845][60934] Updated weights for policy 1, policy_version 80172 (0.0010) [2023-10-14 00:17:18,851][60935] Updated weights for policy 0, policy_version 80580 (0.0007) [2023-10-14 00:17:19,241][60935] Updated weights for policy 0, policy_version 80590 (0.0009) [2023-10-14 00:17:19,607][60935] Updated weights for policy 0, policy_version 80600 (0.0010) [2023-10-14 00:17:20,582][60934] Updated weights for policy 1, policy_version 80182 (0.0010) [2023-10-14 00:17:20,958][60934] Updated weights for policy 1, policy_version 80192 (0.0010) [2023-10-14 00:17:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 165707776. Throughput: 0: 1697.3, 1: 1725.6. Samples: 41436514. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:17:21,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:17:21,325][60934] Updated weights for policy 1, policy_version 80202 (0.0009) [2023-10-14 00:17:23,418][60935] Updated weights for policy 0, policy_version 80610 (0.0008) [2023-10-14 00:17:23,787][60935] Updated weights for policy 0, policy_version 80620 (0.0008) [2023-10-14 00:17:24,147][60935] Updated weights for policy 0, policy_version 80630 (0.0007) [2023-10-14 00:17:24,512][60935] Updated weights for policy 0, policy_version 80640 (0.0008) [2023-10-14 00:17:25,441][60934] Updated weights for policy 1, policy_version 80212 (0.0007) [2023-10-14 00:17:25,800][60934] Updated weights for policy 1, policy_version 80222 (0.0008) [2023-10-14 00:17:26,166][60934] Updated weights for policy 1, policy_version 80232 (0.0007) [2023-10-14 00:17:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 165773312. Throughput: 0: 1705.1, 1: 1730.3. Samples: 41457644. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:17:26,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:17:28,592][60935] Updated weights for policy 0, policy_version 80650 (0.0009) [2023-10-14 00:17:28,943][60935] Updated weights for policy 0, policy_version 80660 (0.0010) [2023-10-14 00:17:29,306][60935] Updated weights for policy 0, policy_version 80670 (0.0009) [2023-10-14 00:17:30,351][60934] Updated weights for policy 1, policy_version 80242 (0.0008) [2023-10-14 00:17:30,716][60934] Updated weights for policy 1, policy_version 80252 (0.0008) [2023-10-14 00:17:31,070][60934] Updated weights for policy 1, policy_version 80262 (0.0007) [2023-10-14 00:17:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 165838848. Throughput: 0: 1710.8, 1: 1735.2. Samples: 41467690. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:17:31,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:17:31,440][60934] Updated weights for policy 1, policy_version 80272 (0.0008) [2023-10-14 00:17:33,236][60935] Updated weights for policy 0, policy_version 80680 (0.0008) [2023-10-14 00:17:33,604][60935] Updated weights for policy 0, policy_version 80690 (0.0008) [2023-10-14 00:17:33,967][60935] Updated weights for policy 0, policy_version 80700 (0.0007) [2023-10-14 00:17:35,350][60934] Updated weights for policy 1, policy_version 80282 (0.0010) [2023-10-14 00:17:35,717][60934] Updated weights for policy 1, policy_version 80292 (0.0008) [2023-10-14 00:17:36,081][60934] Updated weights for policy 1, policy_version 80302 (0.0008) [2023-10-14 00:17:36,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 165937152. Throughput: 0: 1700.1, 1: 1733.5. Samples: 41488322. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:17:36,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:17:37,789][60935] Updated weights for policy 0, policy_version 80710 (0.0008) [2023-10-14 00:17:38,153][60935] Updated weights for policy 0, policy_version 80720 (0.0009) [2023-10-14 00:17:38,524][60935] Updated weights for policy 0, policy_version 80730 (0.0008) [2023-10-14 00:17:39,965][60934] Updated weights for policy 1, policy_version 80312 (0.0007) [2023-10-14 00:17:40,338][60934] Updated weights for policy 1, policy_version 80322 (0.0010) [2023-10-14 00:17:40,700][60934] Updated weights for policy 1, policy_version 80332 (0.0007) [2023-10-14 00:17:41,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 166002688. Throughput: 0: 1724.8, 1: 1710.8. Samples: 41508768. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:17:41,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:17:42,529][60935] Updated weights for policy 0, policy_version 80740 (0.0008) [2023-10-14 00:17:42,901][60935] Updated weights for policy 0, policy_version 80750 (0.0007) [2023-10-14 00:17:43,273][60935] Updated weights for policy 0, policy_version 80760 (0.0007) [2023-10-14 00:17:44,586][60934] Updated weights for policy 1, policy_version 80342 (0.0009) [2023-10-14 00:17:44,947][60934] Updated weights for policy 1, policy_version 80352 (0.0008) [2023-10-14 00:17:45,311][60934] Updated weights for policy 1, policy_version 80362 (0.0010) [2023-10-14 00:17:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 166068224. Throughput: 0: 1701.7, 1: 1733.6. Samples: 41519170. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:17:46,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:17:47,261][60935] Updated weights for policy 0, policy_version 80770 (0.0008) [2023-10-14 00:17:47,627][60935] Updated weights for policy 0, policy_version 80780 (0.0008) [2023-10-14 00:17:47,997][60935] Updated weights for policy 0, policy_version 80790 (0.0007) [2023-10-14 00:17:48,363][60935] Updated weights for policy 0, policy_version 80800 (0.0009) [2023-10-14 00:17:49,445][60934] Updated weights for policy 1, policy_version 80372 (0.0009) [2023-10-14 00:17:49,842][60934] Updated weights for policy 1, policy_version 80382 (0.0007) [2023-10-14 00:17:50,205][60934] Updated weights for policy 1, policy_version 80392 (0.0009) [2023-10-14 00:17:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 166133760. Throughput: 0: 1714.5, 1: 1719.6. Samples: 41539850. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:17:51,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:17:52,190][60935] Updated weights for policy 0, policy_version 80810 (0.0009) [2023-10-14 00:17:52,549][60935] Updated weights for policy 0, policy_version 80820 (0.0011) [2023-10-14 00:17:52,925][60935] Updated weights for policy 0, policy_version 80830 (0.0009) [2023-10-14 00:17:54,231][60934] Updated weights for policy 1, policy_version 80402 (0.0011) [2023-10-14 00:17:54,601][60934] Updated weights for policy 1, policy_version 80412 (0.0010) [2023-10-14 00:17:54,959][60934] Updated weights for policy 1, policy_version 80422 (0.0009) [2023-10-14 00:17:55,329][60934] Updated weights for policy 1, policy_version 80432 (0.0010) [2023-10-14 00:17:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 166199296. Throughput: 0: 1735.8, 1: 1695.5. Samples: 41560072. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 00:17:56,249][59943] Avg episode reward: [(0, '-0.100'), (1, '0.000')] [2023-10-14 00:17:56,987][60935] Updated weights for policy 0, policy_version 80840 (0.0007) [2023-10-14 00:17:57,358][60935] Updated weights for policy 0, policy_version 80850 (0.0008) [2023-10-14 00:17:57,732][60935] Updated weights for policy 0, policy_version 80860 (0.0008) [2023-10-14 00:17:59,188][60934] Updated weights for policy 1, policy_version 80442 (0.0010) [2023-10-14 00:17:59,560][60934] Updated weights for policy 1, policy_version 80452 (0.0007) [2023-10-14 00:17:59,924][60934] Updated weights for policy 1, policy_version 80462 (0.0009) [2023-10-14 00:18:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 166264832. Throughput: 0: 1705.9, 1: 1727.6. Samples: 41570686. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 00:18:01,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:18:01,601][60935] Updated weights for policy 0, policy_version 80870 (0.0008) [2023-10-14 00:18:01,963][60935] Updated weights for policy 0, policy_version 80880 (0.0008) [2023-10-14 00:18:02,340][60935] Updated weights for policy 0, policy_version 80890 (0.0009) [2023-10-14 00:18:03,911][60934] Updated weights for policy 1, policy_version 80472 (0.0009) [2023-10-14 00:18:04,272][60934] Updated weights for policy 1, policy_version 80482 (0.0009) [2023-10-14 00:18:04,638][60934] Updated weights for policy 1, policy_version 80492 (0.0008) [2023-10-14 00:18:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 166330368. Throughput: 0: 1736.0, 1: 1698.2. Samples: 41591056. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 00:18:06,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:18:06,276][60935] Updated weights for policy 0, policy_version 80900 (0.0011) [2023-10-14 00:18:06,662][60935] Updated weights for policy 0, policy_version 80910 (0.0010) [2023-10-14 00:18:07,037][60935] Updated weights for policy 0, policy_version 80920 (0.0011) [2023-10-14 00:18:08,635][60934] Updated weights for policy 1, policy_version 80502 (0.0008) [2023-10-14 00:18:09,006][60934] Updated weights for policy 1, policy_version 80512 (0.0007) [2023-10-14 00:18:09,375][60934] Updated weights for policy 1, policy_version 80522 (0.0007) [2023-10-14 00:18:11,118][60935] Updated weights for policy 0, policy_version 80930 (0.0012) [2023-10-14 00:18:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 166395904. Throughput: 0: 1737.9, 1: 1696.8. Samples: 41612204. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 00:18:11,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:18:11,488][60935] Updated weights for policy 0, policy_version 80940 (0.0007) [2023-10-14 00:18:11,856][60935] Updated weights for policy 0, policy_version 80950 (0.0008) [2023-10-14 00:18:12,225][60935] Updated weights for policy 0, policy_version 80960 (0.0009) [2023-10-14 00:18:13,317][60934] Updated weights for policy 1, policy_version 80532 (0.0007) [2023-10-14 00:18:13,680][60934] Updated weights for policy 1, policy_version 80542 (0.0009) [2023-10-14 00:18:14,049][60934] Updated weights for policy 1, policy_version 80552 (0.0007) [2023-10-14 00:18:16,085][60935] Updated weights for policy 0, policy_version 80970 (0.0007) [2023-10-14 00:18:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 166461440. Throughput: 0: 1723.3, 1: 1715.6. Samples: 41622442. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 00:18:16,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:18:16,444][60935] Updated weights for policy 0, policy_version 80980 (0.0007) [2023-10-14 00:18:16,815][60935] Updated weights for policy 0, policy_version 80990 (0.0007) [2023-10-14 00:18:18,021][60934] Updated weights for policy 1, policy_version 80562 (0.0008) [2023-10-14 00:18:18,395][60934] Updated weights for policy 1, policy_version 80572 (0.0008) [2023-10-14 00:18:18,756][60934] Updated weights for policy 1, policy_version 80582 (0.0007) [2023-10-14 00:18:19,122][60934] Updated weights for policy 1, policy_version 80592 (0.0009) [2023-10-14 00:18:20,850][60935] Updated weights for policy 0, policy_version 81000 (0.0010) [2023-10-14 00:18:21,207][60935] Updated weights for policy 0, policy_version 81010 (0.0011) [2023-10-14 00:18:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 166526976. Throughput: 0: 1733.0, 1: 1697.0. Samples: 41642672. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 00:18:21,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:18:21,577][60935] Updated weights for policy 0, policy_version 81020 (0.0009) [2023-10-14 00:18:23,079][60934] Updated weights for policy 1, policy_version 80602 (0.0007) [2023-10-14 00:18:23,448][60934] Updated weights for policy 1, policy_version 80612 (0.0007) [2023-10-14 00:18:23,809][60934] Updated weights for policy 1, policy_version 80622 (0.0010) [2023-10-14 00:18:25,289][60935] Updated weights for policy 0, policy_version 81030 (0.0008) [2023-10-14 00:18:25,654][60935] Updated weights for policy 0, policy_version 81040 (0.0008) [2023-10-14 00:18:26,031][60935] Updated weights for policy 0, policy_version 81050 (0.0008) [2023-10-14 00:18:26,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 166625280. Throughput: 0: 1719.3, 1: 1718.0. Samples: 41663444. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 00:18:26,250][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:18:27,795][60934] Updated weights for policy 1, policy_version 80632 (0.0008) [2023-10-14 00:18:28,159][60934] Updated weights for policy 1, policy_version 80642 (0.0008) [2023-10-14 00:18:28,519][60934] Updated weights for policy 1, policy_version 80652 (0.0009) [2023-10-14 00:18:29,978][60935] Updated weights for policy 0, policy_version 81060 (0.0008) [2023-10-14 00:18:30,345][60935] Updated weights for policy 0, policy_version 81070 (0.0007) [2023-10-14 00:18:30,702][60935] Updated weights for policy 0, policy_version 81080 (0.0007) [2023-10-14 00:18:31,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 166690816. Throughput: 0: 1739.0, 1: 1694.5. Samples: 41673678. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 00:18:31,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:18:32,488][60934] Updated weights for policy 1, policy_version 80662 (0.0009) [2023-10-14 00:18:32,856][60934] Updated weights for policy 1, policy_version 80672 (0.0009) [2023-10-14 00:18:33,214][60934] Updated weights for policy 1, policy_version 80682 (0.0010) [2023-10-14 00:18:34,856][60935] Updated weights for policy 0, policy_version 81090 (0.0009) [2023-10-14 00:18:35,210][60935] Updated weights for policy 0, policy_version 81100 (0.0008) [2023-10-14 00:18:35,579][60935] Updated weights for policy 0, policy_version 81110 (0.0009) [2023-10-14 00:18:35,948][60935] Updated weights for policy 0, policy_version 81120 (0.0009) [2023-10-14 00:18:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 166756352. Throughput: 0: 1732.6, 1: 1708.5. Samples: 41694702. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 00:18:36,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:18:37,151][60934] Updated weights for policy 1, policy_version 80692 (0.0009) [2023-10-14 00:18:37,557][60934] Updated weights for policy 1, policy_version 80702 (0.0010) [2023-10-14 00:18:37,922][60934] Updated weights for policy 1, policy_version 80712 (0.0008) [2023-10-14 00:18:39,965][60935] Updated weights for policy 0, policy_version 81130 (0.0011) [2023-10-14 00:18:40,330][60935] Updated weights for policy 0, policy_version 81140 (0.0010) [2023-10-14 00:18:40,696][60935] Updated weights for policy 0, policy_version 81150 (0.0010) [2023-10-14 00:18:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 166821888. Throughput: 0: 1698.6, 1: 1730.9. Samples: 41714398. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 00:18:41,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:18:41,866][60934] Updated weights for policy 1, policy_version 80722 (0.0007) [2023-10-14 00:18:42,228][60934] Updated weights for policy 1, policy_version 80732 (0.0007) [2023-10-14 00:18:42,592][60934] Updated weights for policy 1, policy_version 80742 (0.0007) [2023-10-14 00:18:42,957][60934] Updated weights for policy 1, policy_version 80752 (0.0007) [2023-10-14 00:18:44,614][60935] Updated weights for policy 0, policy_version 81160 (0.0009) [2023-10-14 00:18:44,972][60935] Updated weights for policy 0, policy_version 81170 (0.0011) [2023-10-14 00:18:45,347][60935] Updated weights for policy 0, policy_version 81180 (0.0011) [2023-10-14 00:18:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 166887424. Throughput: 0: 1732.5, 1: 1698.8. Samples: 41725092. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-14 00:18:46,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:18:47,191][60934] Updated weights for policy 1, policy_version 80762 (0.0009) [2023-10-14 00:18:47,567][60934] Updated weights for policy 1, policy_version 80772 (0.0008) [2023-10-14 00:18:47,924][60934] Updated weights for policy 1, policy_version 80782 (0.0008) [2023-10-14 00:18:49,495][60935] Updated weights for policy 0, policy_version 81190 (0.0010) [2023-10-14 00:18:49,870][60935] Updated weights for policy 0, policy_version 81200 (0.0009) [2023-10-14 00:18:50,231][60935] Updated weights for policy 0, policy_version 81210 (0.0010) [2023-10-14 00:18:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 166952960. Throughput: 0: 1709.3, 1: 1719.3. Samples: 41745342. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-14 00:18:51,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:18:51,673][60934] Updated weights for policy 1, policy_version 80792 (0.0008) [2023-10-14 00:18:52,035][60934] Updated weights for policy 1, policy_version 80802 (0.0010) [2023-10-14 00:18:52,418][60934] Updated weights for policy 1, policy_version 80812 (0.0007) [2023-10-14 00:18:54,260][60935] Updated weights for policy 0, policy_version 81220 (0.0009) [2023-10-14 00:18:54,647][60935] Updated weights for policy 0, policy_version 81230 (0.0009) [2023-10-14 00:18:55,006][60935] Updated weights for policy 0, policy_version 81240 (0.0011) [2023-10-14 00:18:56,204][60934] Updated weights for policy 1, policy_version 80822 (0.0008) [2023-10-14 00:18:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 167018496. Throughput: 0: 1688.2, 1: 1727.6. Samples: 41765914. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-14 00:18:56,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:18:56,259][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000081248_83197952.pth... [2023-10-14 00:18:56,292][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000079648_81559552.pth [2023-10-14 00:18:56,571][60934] Updated weights for policy 1, policy_version 80832 (0.0008) [2023-10-14 00:18:56,938][60934] Updated weights for policy 1, policy_version 80842 (0.0009) [2023-10-14 00:18:57,152][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000080848_83853312.pth... [2023-10-14 00:18:57,184][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000079408_82182144.pth [2023-10-14 00:18:58,981][60935] Updated weights for policy 0, policy_version 81250 (0.0008) [2023-10-14 00:18:59,347][60935] Updated weights for policy 0, policy_version 81260 (0.0008) [2023-10-14 00:18:59,730][60935] Updated weights for policy 0, policy_version 81270 (0.0011) [2023-10-14 00:19:00,092][60935] Updated weights for policy 0, policy_version 81280 (0.0010) [2023-10-14 00:19:00,958][60934] Updated weights for policy 1, policy_version 80852 (0.0009) [2023-10-14 00:19:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 167084032. Throughput: 0: 1718.8, 1: 1704.7. Samples: 41776502. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-14 00:19:01,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:19:01,325][60934] Updated weights for policy 1, policy_version 80862 (0.0008) [2023-10-14 00:19:01,692][60934] Updated weights for policy 1, policy_version 80872 (0.0007) [2023-10-14 00:19:03,927][60935] Updated weights for policy 0, policy_version 81290 (0.0008) [2023-10-14 00:19:04,291][60935] Updated weights for policy 0, policy_version 81300 (0.0008) [2023-10-14 00:19:04,659][60935] Updated weights for policy 0, policy_version 81310 (0.0009) [2023-10-14 00:19:05,771][60934] Updated weights for policy 1, policy_version 80882 (0.0009) [2023-10-14 00:19:06,135][60934] Updated weights for policy 1, policy_version 80892 (0.0007) [2023-10-14 00:19:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 167149568. Throughput: 0: 1694.8, 1: 1721.1. Samples: 41796386. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-14 00:19:06,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:19:06,494][60934] Updated weights for policy 1, policy_version 80902 (0.0007) [2023-10-14 00:19:06,860][60934] Updated weights for policy 1, policy_version 80912 (0.0007) [2023-10-14 00:19:08,484][60935] Updated weights for policy 0, policy_version 81320 (0.0009) [2023-10-14 00:19:08,848][60935] Updated weights for policy 0, policy_version 81330 (0.0009) [2023-10-14 00:19:09,221][60935] Updated weights for policy 0, policy_version 81340 (0.0009) [2023-10-14 00:19:10,596][60934] Updated weights for policy 1, policy_version 80922 (0.0009) [2023-10-14 00:19:10,960][60934] Updated weights for policy 1, policy_version 80932 (0.0007) [2023-10-14 00:19:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 167215104. Throughput: 0: 1708.9, 1: 1716.9. Samples: 41817606. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-14 00:19:11,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:19:11,323][60934] Updated weights for policy 1, policy_version 80942 (0.0007) [2023-10-14 00:19:13,246][60935] Updated weights for policy 0, policy_version 81350 (0.0008) [2023-10-14 00:19:13,625][60935] Updated weights for policy 0, policy_version 81360 (0.0008) [2023-10-14 00:19:13,995][60935] Updated weights for policy 0, policy_version 81370 (0.0009) [2023-10-14 00:19:15,355][60934] Updated weights for policy 1, policy_version 80952 (0.0009) [2023-10-14 00:19:15,720][60934] Updated weights for policy 1, policy_version 80962 (0.0011) [2023-10-14 00:19:16,093][60934] Updated weights for policy 1, policy_version 80972 (0.0011) [2023-10-14 00:19:16,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13884.7). Total num frames: 167313408. Throughput: 0: 1700.8, 1: 1725.5. Samples: 41827862. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-14 00:19:16,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:19:17,847][60935] Updated weights for policy 0, policy_version 81380 (0.0010) [2023-10-14 00:19:18,217][60935] Updated weights for policy 0, policy_version 81390 (0.0010) [2023-10-14 00:19:18,586][60935] Updated weights for policy 0, policy_version 81400 (0.0010) [2023-10-14 00:19:20,080][60934] Updated weights for policy 1, policy_version 80982 (0.0009) [2023-10-14 00:19:20,447][60934] Updated weights for policy 1, policy_version 80992 (0.0007) [2023-10-14 00:19:20,814][60934] Updated weights for policy 1, policy_version 81002 (0.0008) [2023-10-14 00:19:21,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 167378944. Throughput: 0: 1694.3, 1: 1725.0. Samples: 41848570. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-14 00:19:21,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:19:22,531][60935] Updated weights for policy 0, policy_version 81410 (0.0010) [2023-10-14 00:19:22,902][60935] Updated weights for policy 0, policy_version 81420 (0.0010) [2023-10-14 00:19:23,268][60935] Updated weights for policy 0, policy_version 81430 (0.0009) [2023-10-14 00:19:23,635][60935] Updated weights for policy 0, policy_version 81440 (0.0008) [2023-10-14 00:19:24,892][60934] Updated weights for policy 1, policy_version 81012 (0.0010) [2023-10-14 00:19:25,293][60934] Updated weights for policy 1, policy_version 81022 (0.0009) [2023-10-14 00:19:25,353][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:19:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 167444480. Throughput: 0: 1730.1, 1: 1708.3. Samples: 41869126. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-14 00:19:26,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:19:27,506][60935] Updated weights for policy 0, policy_version 81450 (0.0010) [2023-10-14 00:19:27,870][60935] Updated weights for policy 0, policy_version 81460 (0.0010) [2023-10-14 00:19:28,239][60935] Updated weights for policy 0, policy_version 81470 (0.0010) [2023-10-14 00:19:29,657][60934] Updated weights for policy 1, policy_version 81032 (0.0008) [2023-10-14 00:19:30,026][60934] Updated weights for policy 1, policy_version 81042 (0.0007) [2023-10-14 00:19:30,393][60934] Updated weights for policy 1, policy_version 81052 (0.0009) [2023-10-14 00:19:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 167510016. Throughput: 0: 1699.8, 1: 1741.6. Samples: 41879956. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-14 00:19:31,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:19:32,160][60935] Updated weights for policy 0, policy_version 81480 (0.0010) [2023-10-14 00:19:32,537][60935] Updated weights for policy 0, policy_version 81490 (0.0011) [2023-10-14 00:19:32,902][60935] Updated weights for policy 0, policy_version 81500 (0.0011) [2023-10-14 00:19:34,298][60934] Updated weights for policy 1, policy_version 81062 (0.0007) [2023-10-14 00:19:34,660][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-14 00:19:34,666][60934] Updated weights for policy 1, policy_version 81072 (0.0008) [2023-10-14 00:19:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 167575552. Throughput: 0: 1720.0, 1: 1734.3. Samples: 41900788. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-14 00:19:36,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:19:36,940][60935] Updated weights for policy 0, policy_version 81510 (0.0008) [2023-10-14 00:19:37,316][60935] Updated weights for policy 0, policy_version 81520 (0.0008) [2023-10-14 00:19:37,688][60935] Updated weights for policy 0, policy_version 81530 (0.0009) [2023-10-14 00:19:38,961][60934] Updated weights for policy 1, policy_version 81082 (0.0009) [2023-10-14 00:19:39,328][60934] Updated weights for policy 1, policy_version 81092 (0.0008) [2023-10-14 00:19:39,695][60934] Updated weights for policy 1, policy_version 81102 (0.0010) [2023-10-14 00:19:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 167641088. Throughput: 0: 1745.2, 1: 1722.4. Samples: 41921958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:19:41,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:19:41,765][60935] Updated weights for policy 0, policy_version 81540 (0.0008) [2023-10-14 00:19:42,163][60935] Updated weights for policy 0, policy_version 81550 (0.0008) [2023-10-14 00:19:42,536][60935] Updated weights for policy 0, policy_version 81560 (0.0009) [2023-10-14 00:19:43,630][60934] Updated weights for policy 1, policy_version 81112 (0.0008) [2023-10-14 00:19:43,991][60934] Updated weights for policy 1, policy_version 81122 (0.0007) [2023-10-14 00:19:44,359][60934] Updated weights for policy 1, policy_version 81132 (0.0008) [2023-10-14 00:19:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13884.8). Total num frames: 167706624. Throughput: 0: 1714.9, 1: 1745.9. Samples: 41932236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:19:46,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:19:46,461][60935] Updated weights for policy 0, policy_version 81570 (0.0008) [2023-10-14 00:19:46,841][60935] Updated weights for policy 0, policy_version 81580 (0.0007) [2023-10-14 00:19:47,209][60935] Updated weights for policy 0, policy_version 81590 (0.0009) [2023-10-14 00:19:47,570][60935] Updated weights for policy 0, policy_version 81600 (0.0007) [2023-10-14 00:19:48,292][60934] Updated weights for policy 1, policy_version 81142 (0.0010) [2023-10-14 00:19:48,659][60934] Updated weights for policy 1, policy_version 81152 (0.0008) [2023-10-14 00:19:49,029][60934] Updated weights for policy 1, policy_version 81162 (0.0007) [2023-10-14 00:19:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 167772160. Throughput: 0: 1740.5, 1: 1729.9. Samples: 41952554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:19:51,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:19:51,559][60935] Updated weights for policy 0, policy_version 81610 (0.0007) [2023-10-14 00:19:51,923][60935] Updated weights for policy 0, policy_version 81620 (0.0009) [2023-10-14 00:19:52,282][60935] Updated weights for policy 0, policy_version 81630 (0.0009) [2023-10-14 00:19:53,006][60934] Updated weights for policy 1, policy_version 81172 (0.0009) [2023-10-14 00:19:53,376][60934] Updated weights for policy 1, policy_version 81182 (0.0007) [2023-10-14 00:19:53,737][60934] Updated weights for policy 1, policy_version 81192 (0.0007) [2023-10-14 00:19:56,138][60935] Updated weights for policy 0, policy_version 81640 (0.0008) [2023-10-14 00:19:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 167837696. Throughput: 0: 1743.5, 1: 1735.6. Samples: 41974166. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:19:56,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:19:56,513][60935] Updated weights for policy 0, policy_version 81650 (0.0008) [2023-10-14 00:19:56,880][60935] Updated weights for policy 0, policy_version 81660 (0.0010) [2023-10-14 00:19:57,609][60934] Updated weights for policy 1, policy_version 81202 (0.0008) [2023-10-14 00:19:57,977][60934] Updated weights for policy 1, policy_version 81212 (0.0007) [2023-10-14 00:19:58,340][60934] Updated weights for policy 1, policy_version 81222 (0.0009) [2023-10-14 00:19:58,709][60934] Updated weights for policy 1, policy_version 81232 (0.0009) [2023-10-14 00:20:00,778][60935] Updated weights for policy 0, policy_version 81670 (0.0009) [2023-10-14 00:20:01,145][60935] Updated weights for policy 0, policy_version 81680 (0.0009) [2023-10-14 00:20:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 167903232. Throughput: 0: 1730.6, 1: 1731.2. Samples: 41983646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:20:01,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:20:01,514][60935] Updated weights for policy 0, policy_version 81690 (0.0008) [2023-10-14 00:20:02,795][60934] Updated weights for policy 1, policy_version 81242 (0.0007) [2023-10-14 00:20:03,165][60934] Updated weights for policy 1, policy_version 81252 (0.0009) [2023-10-14 00:20:03,538][60934] Updated weights for policy 1, policy_version 81262 (0.0009) [2023-10-14 00:20:05,525][60935] Updated weights for policy 0, policy_version 81700 (0.0009) [2023-10-14 00:20:05,880][60935] Updated weights for policy 0, policy_version 81710 (0.0012) [2023-10-14 00:20:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 167968768. Throughput: 0: 1739.6, 1: 1724.0. Samples: 42004434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:20:06,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:20:06,249][60935] Updated weights for policy 0, policy_version 81720 (0.0011) [2023-10-14 00:20:07,420][60934] Updated weights for policy 1, policy_version 81272 (0.0008) [2023-10-14 00:20:07,786][60934] Updated weights for policy 1, policy_version 81282 (0.0008) [2023-10-14 00:20:08,161][60934] Updated weights for policy 1, policy_version 81292 (0.0008) [2023-10-14 00:20:10,099][60935] Updated weights for policy 0, policy_version 81730 (0.0011) [2023-10-14 00:20:10,462][60935] Updated weights for policy 0, policy_version 81740 (0.0007) [2023-10-14 00:20:10,828][60935] Updated weights for policy 0, policy_version 81750 (0.0007) [2023-10-14 00:20:11,196][60935] Updated weights for policy 0, policy_version 81760 (0.0007) [2023-10-14 00:20:11,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 168067072. Throughput: 0: 1720.9, 1: 1746.4. Samples: 42025158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:20:11,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:20:12,176][60934] Updated weights for policy 1, policy_version 81302 (0.0008) [2023-10-14 00:20:12,566][60934] Updated weights for policy 1, policy_version 81312 (0.0007) [2023-10-14 00:20:12,944][60934] Updated weights for policy 1, policy_version 81322 (0.0009) [2023-10-14 00:20:14,980][60935] Updated weights for policy 0, policy_version 81770 (0.0011) [2023-10-14 00:20:15,346][60935] Updated weights for policy 0, policy_version 81780 (0.0008) [2023-10-14 00:20:15,719][60935] Updated weights for policy 0, policy_version 81790 (0.0008) [2023-10-14 00:20:16,248][59943] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 168132608. Throughput: 0: 1741.9, 1: 1711.1. Samples: 42035340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:20:16,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:20:16,876][60934] Updated weights for policy 1, policy_version 81332 (0.0007) [2023-10-14 00:20:17,244][60934] Updated weights for policy 1, policy_version 81342 (0.0007) [2023-10-14 00:20:17,602][60934] Updated weights for policy 1, policy_version 81352 (0.0008) [2023-10-14 00:20:19,602][60935] Updated weights for policy 0, policy_version 81800 (0.0010) [2023-10-14 00:20:19,966][60935] Updated weights for policy 0, policy_version 81810 (0.0010) [2023-10-14 00:20:20,340][60935] Updated weights for policy 0, policy_version 81820 (0.0009) [2023-10-14 00:20:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 168198144. Throughput: 0: 1726.7, 1: 1716.1. Samples: 42055714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:20:21,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:20:21,606][60934] Updated weights for policy 1, policy_version 81362 (0.0008) [2023-10-14 00:20:21,974][60934] Updated weights for policy 1, policy_version 81372 (0.0009) [2023-10-14 00:20:22,348][60934] Updated weights for policy 1, policy_version 81382 (0.0011) [2023-10-14 00:20:22,713][60934] Updated weights for policy 1, policy_version 81392 (0.0009) [2023-10-14 00:20:24,346][60935] Updated weights for policy 0, policy_version 81830 (0.0008) [2023-10-14 00:20:24,711][60935] Updated weights for policy 0, policy_version 81840 (0.0011) [2023-10-14 00:20:25,080][60935] Updated weights for policy 0, policy_version 81850 (0.0011) [2023-10-14 00:20:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 168263680. Throughput: 0: 1705.4, 1: 1723.1. Samples: 42076242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:20:26,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:20:26,802][60934] Updated weights for policy 1, policy_version 81402 (0.0009) [2023-10-14 00:20:27,172][60934] Updated weights for policy 1, policy_version 81412 (0.0008) [2023-10-14 00:20:27,536][60934] Updated weights for policy 1, policy_version 81422 (0.0008) [2023-10-14 00:20:29,001][60935] Updated weights for policy 0, policy_version 81860 (0.0009) [2023-10-14 00:20:29,390][60935] Updated weights for policy 0, policy_version 81870 (0.0011) [2023-10-14 00:20:29,766][60935] Updated weights for policy 0, policy_version 81880 (0.0010) [2023-10-14 00:20:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 168329216. Throughput: 0: 1740.0, 1: 1699.3. Samples: 42087008. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 00:20:31,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:20:31,610][60934] Updated weights for policy 1, policy_version 81432 (0.0011) [2023-10-14 00:20:31,978][60934] Updated weights for policy 1, policy_version 81442 (0.0007) [2023-10-14 00:20:32,351][60934] Updated weights for policy 1, policy_version 81452 (0.0007) [2023-10-14 00:20:33,766][60935] Updated weights for policy 0, policy_version 81890 (0.0010) [2023-10-14 00:20:34,136][60935] Updated weights for policy 0, policy_version 81900 (0.0008) [2023-10-14 00:20:34,506][60935] Updated weights for policy 0, policy_version 81910 (0.0009) [2023-10-14 00:20:34,868][60935] Updated weights for policy 0, policy_version 81920 (0.0009) [2023-10-14 00:20:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 168394752. Throughput: 0: 1709.3, 1: 1721.6. Samples: 42106944. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 00:20:36,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:20:36,377][60934] Updated weights for policy 1, policy_version 81462 (0.0008) [2023-10-14 00:20:36,743][60934] Updated weights for policy 1, policy_version 81472 (0.0008) [2023-10-14 00:20:37,113][60934] Updated weights for policy 1, policy_version 81482 (0.0010) [2023-10-14 00:20:38,843][60935] Updated weights for policy 0, policy_version 81930 (0.0008) [2023-10-14 00:20:39,220][60935] Updated weights for policy 0, policy_version 81940 (0.0007) [2023-10-14 00:20:39,586][60935] Updated weights for policy 0, policy_version 81950 (0.0008) [2023-10-14 00:20:41,069][60934] Updated weights for policy 1, policy_version 81492 (0.0009) [2023-10-14 00:20:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 168460288. Throughput: 0: 1702.4, 1: 1719.7. Samples: 42128160. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 00:20:41,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:20:41,442][60934] Updated weights for policy 1, policy_version 81502 (0.0008) [2023-10-14 00:20:41,807][60934] Updated weights for policy 1, policy_version 81512 (0.0007) [2023-10-14 00:20:43,627][60935] Updated weights for policy 0, policy_version 81960 (0.0008) [2023-10-14 00:20:43,997][60935] Updated weights for policy 0, policy_version 81970 (0.0008) [2023-10-14 00:20:44,361][60935] Updated weights for policy 0, policy_version 81980 (0.0008) [2023-10-14 00:20:45,813][60934] Updated weights for policy 1, policy_version 81522 (0.0008) [2023-10-14 00:20:46,178][60934] Updated weights for policy 1, policy_version 81532 (0.0009) [2023-10-14 00:20:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 168525824. Throughput: 0: 1719.8, 1: 1711.5. Samples: 42138056. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 00:20:46,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:20:46,543][60934] Updated weights for policy 1, policy_version 81542 (0.0010) [2023-10-14 00:20:46,910][60934] Updated weights for policy 1, policy_version 81552 (0.0008) [2023-10-14 00:20:48,471][60935] Updated weights for policy 0, policy_version 81990 (0.0009) [2023-10-14 00:20:48,837][60935] Updated weights for policy 0, policy_version 82000 (0.0010) [2023-10-14 00:20:49,198][60935] Updated weights for policy 0, policy_version 82010 (0.0009) [2023-10-14 00:20:50,894][60934] Updated weights for policy 1, policy_version 81562 (0.0008) [2023-10-14 00:20:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 168591360. Throughput: 0: 1700.7, 1: 1716.3. Samples: 42158196. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 00:20:51,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:20:51,265][60934] Updated weights for policy 1, policy_version 81572 (0.0008) [2023-10-14 00:20:51,618][60934] Updated weights for policy 1, policy_version 81582 (0.0009) [2023-10-14 00:20:53,136][60935] Updated weights for policy 0, policy_version 82020 (0.0009) [2023-10-14 00:20:53,502][60935] Updated weights for policy 0, policy_version 82030 (0.0007) [2023-10-14 00:20:53,873][60935] Updated weights for policy 0, policy_version 82040 (0.0008) [2023-10-14 00:20:55,613][60934] Updated weights for policy 1, policy_version 81592 (0.0009) [2023-10-14 00:20:55,975][60934] Updated weights for policy 1, policy_version 81602 (0.0007) [2023-10-14 00:20:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 168656896. Throughput: 0: 1712.6, 1: 1702.6. Samples: 42178844. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 00:20:56,249][59943] Avg episode reward: [(0, '-0.090'), (1, '-0.090')] [2023-10-14 00:20:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000082048_84017152.pth... [2023-10-14 00:20:56,297][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000080448_82378752.pth [2023-10-14 00:20:56,335][60934] Updated weights for policy 1, policy_version 81612 (0.0009) [2023-10-14 00:20:56,472][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000081616_84672512.pth... [2023-10-14 00:20:56,501][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000080016_83001344.pth [2023-10-14 00:20:57,827][60935] Updated weights for policy 0, policy_version 82050 (0.0009) [2023-10-14 00:20:58,201][60935] Updated weights for policy 0, policy_version 82060 (0.0008) [2023-10-14 00:20:58,564][60935] Updated weights for policy 0, policy_version 82070 (0.0009) [2023-10-14 00:20:58,931][60935] Updated weights for policy 0, policy_version 82080 (0.0008) [2023-10-14 00:21:00,497][60934] Updated weights for policy 1, policy_version 81622 (0.0008) [2023-10-14 00:21:00,897][60934] Updated weights for policy 1, policy_version 81632 (0.0009) [2023-10-14 00:21:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 168722432. Throughput: 0: 1695.0, 1: 1711.7. Samples: 42188644. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 00:21:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:21:01,256][60934] Updated weights for policy 1, policy_version 81642 (0.0009) [2023-10-14 00:21:02,962][60935] Updated weights for policy 0, policy_version 82090 (0.0010) [2023-10-14 00:21:03,333][60935] Updated weights for policy 0, policy_version 82100 (0.0011) [2023-10-14 00:21:03,697][60935] Updated weights for policy 0, policy_version 82110 (0.0009) [2023-10-14 00:21:05,153][60934] Updated weights for policy 1, policy_version 81652 (0.0010) [2023-10-14 00:21:05,522][60934] Updated weights for policy 1, policy_version 81662 (0.0010) [2023-10-14 00:21:05,875][60934] Updated weights for policy 1, policy_version 81672 (0.0008) [2023-10-14 00:21:06,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 168820736. Throughput: 0: 1700.8, 1: 1712.3. Samples: 42209300. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 00:21:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:21:07,630][60935] Updated weights for policy 0, policy_version 82120 (0.0010) [2023-10-14 00:21:07,990][60935] Updated weights for policy 0, policy_version 82130 (0.0007) [2023-10-14 00:21:08,363][60935] Updated weights for policy 0, policy_version 82140 (0.0008) [2023-10-14 00:21:09,855][60934] Updated weights for policy 1, policy_version 81682 (0.0009) [2023-10-14 00:21:10,228][60934] Updated weights for policy 1, policy_version 81692 (0.0008) [2023-10-14 00:21:10,593][60934] Updated weights for policy 1, policy_version 81702 (0.0009) [2023-10-14 00:21:10,959][60934] Updated weights for policy 1, policy_version 81712 (0.0009) [2023-10-14 00:21:11,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 168886272. Throughput: 0: 1721.2, 1: 1695.2. Samples: 42229976. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 00:21:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:21:12,384][60935] Updated weights for policy 0, policy_version 82150 (0.0010) [2023-10-14 00:21:12,766][60935] Updated weights for policy 0, policy_version 82160 (0.0010) [2023-10-14 00:21:13,129][60935] Updated weights for policy 0, policy_version 82170 (0.0008) [2023-10-14 00:21:14,590][60934] Updated weights for policy 1, policy_version 81722 (0.0007) [2023-10-14 00:21:14,952][60934] Updated weights for policy 1, policy_version 81732 (0.0008) [2023-10-14 00:21:15,327][60934] Updated weights for policy 1, policy_version 81742 (0.0009) [2023-10-14 00:21:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 168951808. Throughput: 0: 1683.2, 1: 1720.6. Samples: 42240178. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 00:21:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:21:17,281][60935] Updated weights for policy 0, policy_version 82180 (0.0008) [2023-10-14 00:21:17,661][60935] Updated weights for policy 0, policy_version 82190 (0.0009) [2023-10-14 00:21:18,036][60935] Updated weights for policy 0, policy_version 82200 (0.0009) [2023-10-14 00:21:19,414][60934] Updated weights for policy 1, policy_version 81752 (0.0008) [2023-10-14 00:21:19,789][60934] Updated weights for policy 1, policy_version 81762 (0.0007) [2023-10-14 00:21:20,166][60934] Updated weights for policy 1, policy_version 81772 (0.0009) [2023-10-14 00:21:21,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 169017344. Throughput: 0: 1706.1, 1: 1706.3. Samples: 42260502. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 00:21:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:21:22,024][60935] Updated weights for policy 0, policy_version 82210 (0.0008) [2023-10-14 00:21:22,397][60935] Updated weights for policy 0, policy_version 82220 (0.0008) [2023-10-14 00:21:22,767][60935] Updated weights for policy 0, policy_version 82230 (0.0008) [2023-10-14 00:21:23,135][60935] Updated weights for policy 0, policy_version 82240 (0.0007) [2023-10-14 00:21:24,236][60934] Updated weights for policy 1, policy_version 81782 (0.0007) [2023-10-14 00:21:24,606][60934] Updated weights for policy 1, policy_version 81792 (0.0008) [2023-10-14 00:21:24,970][60934] Updated weights for policy 1, policy_version 81802 (0.0010) [2023-10-14 00:21:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 169082880. Throughput: 0: 1711.9, 1: 1681.8. Samples: 42280876. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 00:21:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:21:27,000][60935] Updated weights for policy 0, policy_version 82250 (0.0011) [2023-10-14 00:21:27,373][60935] Updated weights for policy 0, policy_version 82260 (0.0011) [2023-10-14 00:21:27,739][60935] Updated weights for policy 0, policy_version 82270 (0.0010) [2023-10-14 00:21:29,019][60934] Updated weights for policy 1, policy_version 81812 (0.0007) [2023-10-14 00:21:29,388][60934] Updated weights for policy 1, policy_version 81822 (0.0008) [2023-10-14 00:21:29,749][60934] Updated weights for policy 1, policy_version 81832 (0.0009) [2023-10-14 00:21:31,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 169148416. Throughput: 0: 1698.7, 1: 1712.2. Samples: 42291550. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 00:21:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:21:31,782][60935] Updated weights for policy 0, policy_version 82280 (0.0009) [2023-10-14 00:21:32,158][60935] Updated weights for policy 0, policy_version 82290 (0.0009) [2023-10-14 00:21:32,522][60935] Updated weights for policy 0, policy_version 82300 (0.0010) [2023-10-14 00:21:33,759][60934] Updated weights for policy 1, policy_version 81842 (0.0009) [2023-10-14 00:21:34,122][60934] Updated weights for policy 1, policy_version 81852 (0.0010) [2023-10-14 00:21:34,484][60934] Updated weights for policy 1, policy_version 81862 (0.0011) [2023-10-14 00:21:34,851][60934] Updated weights for policy 1, policy_version 81872 (0.0009) [2023-10-14 00:21:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 169213952. Throughput: 0: 1722.2, 1: 1692.0. Samples: 42311836. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 00:21:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:21:36,427][60935] Updated weights for policy 0, policy_version 82310 (0.0008) [2023-10-14 00:21:36,796][60935] Updated weights for policy 0, policy_version 82320 (0.0009) [2023-10-14 00:21:37,158][60935] Updated weights for policy 0, policy_version 82330 (0.0009) [2023-10-14 00:21:38,927][60934] Updated weights for policy 1, policy_version 81882 (0.0007) [2023-10-14 00:21:39,286][60934] Updated weights for policy 1, policy_version 81892 (0.0008) [2023-10-14 00:21:39,662][60934] Updated weights for policy 1, policy_version 81902 (0.0009) [2023-10-14 00:21:41,003][60935] Updated weights for policy 0, policy_version 82340 (0.0011) [2023-10-14 00:21:41,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 169279488. Throughput: 0: 1725.6, 1: 1692.1. Samples: 42332638. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 00:21:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:21:41,372][60935] Updated weights for policy 0, policy_version 82350 (0.0011) [2023-10-14 00:21:41,735][60935] Updated weights for policy 0, policy_version 82360 (0.0008) [2023-10-14 00:21:43,661][60934] Updated weights for policy 1, policy_version 81912 (0.0008) [2023-10-14 00:21:44,020][60934] Updated weights for policy 1, policy_version 81922 (0.0007) [2023-10-14 00:21:44,395][60934] Updated weights for policy 1, policy_version 81932 (0.0007) [2023-10-14 00:21:45,761][60935] Updated weights for policy 0, policy_version 82370 (0.0010) [2023-10-14 00:21:46,123][60935] Updated weights for policy 0, policy_version 82380 (0.0008) [2023-10-14 00:21:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 169345024. Throughput: 0: 1717.7, 1: 1710.8. Samples: 42342928. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 00:21:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:21:46,484][60935] Updated weights for policy 0, policy_version 82390 (0.0008) [2023-10-14 00:21:46,856][60935] Updated weights for policy 0, policy_version 82400 (0.0010) [2023-10-14 00:21:48,483][60934] Updated weights for policy 1, policy_version 81942 (0.0009) [2023-10-14 00:21:48,859][60934] Updated weights for policy 1, policy_version 81952 (0.0007) [2023-10-14 00:21:49,227][60934] Updated weights for policy 1, policy_version 81962 (0.0007) [2023-10-14 00:21:50,931][60935] Updated weights for policy 0, policy_version 82410 (0.0007) [2023-10-14 00:21:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 169410560. Throughput: 0: 1722.5, 1: 1685.7. Samples: 42362668. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 00:21:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:21:51,291][60935] Updated weights for policy 0, policy_version 82420 (0.0007) [2023-10-14 00:21:51,666][60935] Updated weights for policy 0, policy_version 82430 (0.0007) [2023-10-14 00:21:53,191][60934] Updated weights for policy 1, policy_version 81972 (0.0007) [2023-10-14 00:21:53,553][60934] Updated weights for policy 1, policy_version 81982 (0.0007) [2023-10-14 00:21:53,916][60934] Updated weights for policy 1, policy_version 81992 (0.0007) [2023-10-14 00:21:55,618][60935] Updated weights for policy 0, policy_version 82440 (0.0008) [2023-10-14 00:21:55,991][60935] Updated weights for policy 0, policy_version 82450 (0.0007) [2023-10-14 00:21:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 169476096. Throughput: 0: 1705.1, 1: 1702.0. Samples: 42383298. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 00:21:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:21:56,366][60935] Updated weights for policy 0, policy_version 82460 (0.0008) [2023-10-14 00:21:57,826][60934] Updated weights for policy 1, policy_version 82002 (0.0008) [2023-10-14 00:21:58,196][60934] Updated weights for policy 1, policy_version 82012 (0.0007) [2023-10-14 00:21:58,559][60934] Updated weights for policy 1, policy_version 82022 (0.0008) [2023-10-14 00:21:58,925][60934] Updated weights for policy 1, policy_version 82032 (0.0010) [2023-10-14 00:22:00,375][60935] Updated weights for policy 0, policy_version 82470 (0.0012) [2023-10-14 00:22:00,746][60935] Updated weights for policy 0, policy_version 82480 (0.0008) [2023-10-14 00:22:01,112][60935] Updated weights for policy 0, policy_version 82490 (0.0008) [2023-10-14 00:22:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 169541632. Throughput: 0: 1720.3, 1: 1689.3. Samples: 42393610. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 00:22:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:22:02,941][60934] Updated weights for policy 1, policy_version 82042 (0.0007) [2023-10-14 00:22:03,309][60934] Updated weights for policy 1, policy_version 82052 (0.0009) [2023-10-14 00:22:03,676][60934] Updated weights for policy 1, policy_version 82062 (0.0008) [2023-10-14 00:22:05,160][60935] Updated weights for policy 0, policy_version 82500 (0.0009) [2023-10-14 00:22:05,533][60935] Updated weights for policy 0, policy_version 82510 (0.0010) [2023-10-14 00:22:05,899][60935] Updated weights for policy 0, policy_version 82520 (0.0011) [2023-10-14 00:22:06,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 169639936. Throughput: 0: 1729.0, 1: 1692.2. Samples: 42414458. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 00:22:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:22:07,526][60934] Updated weights for policy 1, policy_version 82072 (0.0008) [2023-10-14 00:22:07,886][60934] Updated weights for policy 1, policy_version 82082 (0.0008) [2023-10-14 00:22:08,253][60934] Updated weights for policy 1, policy_version 82092 (0.0008) [2023-10-14 00:22:09,851][60935] Updated weights for policy 0, policy_version 82530 (0.0011) [2023-10-14 00:22:10,216][60935] Updated weights for policy 0, policy_version 82540 (0.0010) [2023-10-14 00:22:10,581][60935] Updated weights for policy 0, policy_version 82550 (0.0010) [2023-10-14 00:22:10,952][60935] Updated weights for policy 0, policy_version 82560 (0.0009) [2023-10-14 00:22:11,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 169705472. Throughput: 0: 1694.4, 1: 1722.4. Samples: 42434632. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 00:22:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:22:12,239][60934] Updated weights for policy 1, policy_version 82102 (0.0010) [2023-10-14 00:22:12,598][60934] Updated weights for policy 1, policy_version 82112 (0.0009) [2023-10-14 00:22:12,972][60934] Updated weights for policy 1, policy_version 82122 (0.0007) [2023-10-14 00:22:14,840][60935] Updated weights for policy 0, policy_version 82570 (0.0010) [2023-10-14 00:22:15,216][60935] Updated weights for policy 0, policy_version 82580 (0.0010) [2023-10-14 00:22:15,596][60935] Updated weights for policy 0, policy_version 82590 (0.0008) [2023-10-14 00:22:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 169771008. Throughput: 0: 1720.5, 1: 1694.3. Samples: 42445216. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 00:22:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:22:16,853][60934] Updated weights for policy 1, policy_version 82132 (0.0008) [2023-10-14 00:22:17,213][60934] Updated weights for policy 1, policy_version 82142 (0.0009) [2023-10-14 00:22:17,578][60934] Updated weights for policy 1, policy_version 82152 (0.0007) [2023-10-14 00:22:19,657][60935] Updated weights for policy 0, policy_version 82600 (0.0008) [2023-10-14 00:22:20,025][60935] Updated weights for policy 0, policy_version 82610 (0.0008) [2023-10-14 00:22:20,388][60935] Updated weights for policy 0, policy_version 82620 (0.0008) [2023-10-14 00:22:21,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 169836544. Throughput: 0: 1709.0, 1: 1720.0. Samples: 42466144. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 00:22:21,250][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:22:21,791][60934] Updated weights for policy 1, policy_version 82162 (0.0008) [2023-10-14 00:22:22,156][60934] Updated weights for policy 1, policy_version 82172 (0.0010) [2023-10-14 00:22:22,531][60934] Updated weights for policy 1, policy_version 82182 (0.0008) [2023-10-14 00:22:22,890][60934] Updated weights for policy 1, policy_version 82192 (0.0007) [2023-10-14 00:22:24,441][60935] Updated weights for policy 0, policy_version 82630 (0.0008) [2023-10-14 00:22:24,811][60935] Updated weights for policy 0, policy_version 82640 (0.0008) [2023-10-14 00:22:25,181][60935] Updated weights for policy 0, policy_version 82650 (0.0008) [2023-10-14 00:22:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 169902080. Throughput: 0: 1690.8, 1: 1726.7. Samples: 42486426. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 00:22:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.090')] [2023-10-14 00:22:26,803][60934] Updated weights for policy 1, policy_version 82202 (0.0008) [2023-10-14 00:22:27,172][60934] Updated weights for policy 1, policy_version 82212 (0.0007) [2023-10-14 00:22:27,535][60934] Updated weights for policy 1, policy_version 82222 (0.0011) [2023-10-14 00:22:29,126][60935] Updated weights for policy 0, policy_version 82660 (0.0009) [2023-10-14 00:22:29,494][60935] Updated weights for policy 0, policy_version 82670 (0.0007) [2023-10-14 00:22:29,865][60935] Updated weights for policy 0, policy_version 82680 (0.0008) [2023-10-14 00:22:31,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 169967616. Throughput: 0: 1727.2, 1: 1699.7. Samples: 42497140. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 00:22:31,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.090')] [2023-10-14 00:22:31,547][60934] Updated weights for policy 1, policy_version 82232 (0.0010) [2023-10-14 00:22:31,909][60934] Updated weights for policy 1, policy_version 82242 (0.0009) [2023-10-14 00:22:32,272][60934] Updated weights for policy 1, policy_version 82252 (0.0010) [2023-10-14 00:22:33,771][60935] Updated weights for policy 0, policy_version 82690 (0.0009) [2023-10-14 00:22:34,138][60935] Updated weights for policy 0, policy_version 82700 (0.0007) [2023-10-14 00:22:34,500][60935] Updated weights for policy 0, policy_version 82710 (0.0010) [2023-10-14 00:22:34,864][60935] Updated weights for policy 0, policy_version 82720 (0.0009) [2023-10-14 00:22:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 170033152. Throughput: 0: 1702.7, 1: 1730.7. Samples: 42517170. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 00:22:36,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.090')] [2023-10-14 00:22:36,349][60934] Updated weights for policy 1, policy_version 82262 (0.0008) [2023-10-14 00:22:36,734][60934] Updated weights for policy 1, policy_version 82272 (0.0008) [2023-10-14 00:22:37,098][60934] Updated weights for policy 1, policy_version 82282 (0.0010) [2023-10-14 00:22:38,837][60935] Updated weights for policy 0, policy_version 82730 (0.0010) [2023-10-14 00:22:39,194][60935] Updated weights for policy 0, policy_version 82740 (0.0011) [2023-10-14 00:22:39,564][60935] Updated weights for policy 0, policy_version 82750 (0.0011) [2023-10-14 00:22:40,982][60934] Updated weights for policy 1, policy_version 82292 (0.0009) [2023-10-14 00:22:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 170098688. Throughput: 0: 1711.5, 1: 1726.2. Samples: 42537992. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 00:22:41,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.090')] [2023-10-14 00:22:41,349][60934] Updated weights for policy 1, policy_version 82302 (0.0008) [2023-10-14 00:22:41,711][60934] Updated weights for policy 1, policy_version 82312 (0.0008) [2023-10-14 00:22:43,504][60935] Updated weights for policy 0, policy_version 82760 (0.0010) [2023-10-14 00:22:43,875][60935] Updated weights for policy 0, policy_version 82770 (0.0011) [2023-10-14 00:22:44,248][60935] Updated weights for policy 0, policy_version 82780 (0.0009) [2023-10-14 00:22:45,530][60934] Updated weights for policy 1, policy_version 82322 (0.0010) [2023-10-14 00:22:45,899][60934] Updated weights for policy 1, policy_version 82332 (0.0009) [2023-10-14 00:22:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 170164224. Throughput: 0: 1716.3, 1: 1717.7. Samples: 42548140. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 00:22:46,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.090')] [2023-10-14 00:22:46,259][60934] Updated weights for policy 1, policy_version 82342 (0.0010) [2023-10-14 00:22:46,632][60934] Updated weights for policy 1, policy_version 82352 (0.0009) [2023-10-14 00:22:48,225][60935] Updated weights for policy 0, policy_version 82790 (0.0009) [2023-10-14 00:22:48,589][60935] Updated weights for policy 0, policy_version 82800 (0.0010) [2023-10-14 00:22:48,968][60935] Updated weights for policy 0, policy_version 82810 (0.0007) [2023-10-14 00:22:50,654][60934] Updated weights for policy 1, policy_version 82362 (0.0009) [2023-10-14 00:22:51,013][60934] Updated weights for policy 1, policy_version 82372 (0.0009) [2023-10-14 00:22:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 170229760. Throughput: 0: 1697.1, 1: 1730.2. Samples: 42568684. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 00:22:51,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.090')] [2023-10-14 00:22:51,379][60934] Updated weights for policy 1, policy_version 82382 (0.0007) [2023-10-14 00:22:53,009][60935] Updated weights for policy 0, policy_version 82820 (0.0009) [2023-10-14 00:22:53,396][60935] Updated weights for policy 0, policy_version 82830 (0.0011) [2023-10-14 00:22:53,766][60935] Updated weights for policy 0, policy_version 82840 (0.0011) [2023-10-14 00:22:55,258][60934] Updated weights for policy 1, policy_version 82392 (0.0010) [2023-10-14 00:22:55,627][60934] Updated weights for policy 1, policy_version 82402 (0.0008) [2023-10-14 00:22:55,991][60934] Updated weights for policy 1, policy_version 82412 (0.0008) [2023-10-14 00:22:56,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 170328064. Throughput: 0: 1724.7, 1: 1712.6. Samples: 42589314. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 00:22:56,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.090')] [2023-10-14 00:22:56,260][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000082848_84836352.pth... [2023-10-14 00:22:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000082416_85491712.pth... [2023-10-14 00:22:56,294][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000081248_83197952.pth [2023-10-14 00:22:56,301][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000080848_83853312.pth [2023-10-14 00:22:57,873][60935] Updated weights for policy 0, policy_version 82850 (0.0011) [2023-10-14 00:22:58,243][60935] Updated weights for policy 0, policy_version 82860 (0.0009) [2023-10-14 00:22:58,603][60935] Updated weights for policy 0, policy_version 82870 (0.0007) [2023-10-14 00:22:58,973][60935] Updated weights for policy 0, policy_version 82880 (0.0009) [2023-10-14 00:23:00,003][60934] Updated weights for policy 1, policy_version 82422 (0.0010) [2023-10-14 00:23:00,373][60934] Updated weights for policy 1, policy_version 82432 (0.0008) [2023-10-14 00:23:00,741][60934] Updated weights for policy 1, policy_version 82442 (0.0008) [2023-10-14 00:23:01,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 170393600. Throughput: 0: 1699.5, 1: 1725.2. Samples: 42599330. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 00:23:01,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.090')] [2023-10-14 00:23:02,986][60935] Updated weights for policy 0, policy_version 82890 (0.0009) [2023-10-14 00:23:03,363][60935] Updated weights for policy 0, policy_version 82900 (0.0009) [2023-10-14 00:23:03,738][60935] Updated weights for policy 0, policy_version 82910 (0.0008) [2023-10-14 00:23:04,803][60934] Updated weights for policy 1, policy_version 82452 (0.0008) [2023-10-14 00:23:05,171][60934] Updated weights for policy 1, policy_version 82462 (0.0007) [2023-10-14 00:23:05,543][60934] Updated weights for policy 1, policy_version 82472 (0.0011) [2023-10-14 00:23:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 170459136. Throughput: 0: 1706.9, 1: 1721.8. Samples: 42620436. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 00:23:06,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.090')] [2023-10-14 00:23:07,682][60935] Updated weights for policy 0, policy_version 82920 (0.0007) [2023-10-14 00:23:08,055][60935] Updated weights for policy 0, policy_version 82930 (0.0009) [2023-10-14 00:23:08,422][60935] Updated weights for policy 0, policy_version 82940 (0.0008) [2023-10-14 00:23:09,492][60934] Updated weights for policy 1, policy_version 82482 (0.0009) [2023-10-14 00:23:09,864][60934] Updated weights for policy 1, policy_version 82492 (0.0008) [2023-10-14 00:23:10,240][60934] Updated weights for policy 1, policy_version 82502 (0.0009) [2023-10-14 00:23:10,611][60934] Updated weights for policy 1, policy_version 82512 (0.0008) [2023-10-14 00:23:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 170524672. Throughput: 0: 1728.0, 1: 1699.0. Samples: 42640638. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-14 00:23:11,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.090')] [2023-10-14 00:23:12,311][60935] Updated weights for policy 0, policy_version 82950 (0.0009) [2023-10-14 00:23:12,681][60935] Updated weights for policy 0, policy_version 82960 (0.0007) [2023-10-14 00:23:13,042][60935] Updated weights for policy 0, policy_version 82970 (0.0009) [2023-10-14 00:23:14,512][60934] Updated weights for policy 1, policy_version 82522 (0.0008) [2023-10-14 00:23:14,881][60934] Updated weights for policy 1, policy_version 82532 (0.0008) [2023-10-14 00:23:15,244][60934] Updated weights for policy 1, policy_version 82542 (0.0009) [2023-10-14 00:23:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 170590208. Throughput: 0: 1690.7, 1: 1727.4. Samples: 42650954. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-14 00:23:16,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.090')] [2023-10-14 00:23:17,141][60935] Updated weights for policy 0, policy_version 82980 (0.0012) [2023-10-14 00:23:17,506][60935] Updated weights for policy 0, policy_version 82990 (0.0009) [2023-10-14 00:23:17,882][60935] Updated weights for policy 0, policy_version 83000 (0.0010) [2023-10-14 00:23:19,316][60934] Updated weights for policy 1, policy_version 82552 (0.0010) [2023-10-14 00:23:19,687][60934] Updated weights for policy 1, policy_version 82562 (0.0010) [2023-10-14 00:23:20,052][60934] Updated weights for policy 1, policy_version 82572 (0.0011) [2023-10-14 00:23:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 170655744. Throughput: 0: 1720.9, 1: 1713.6. Samples: 42671724. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-14 00:23:21,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.100')] [2023-10-14 00:23:21,591][60935] Updated weights for policy 0, policy_version 83010 (0.0008) [2023-10-14 00:23:21,963][60935] Updated weights for policy 0, policy_version 83020 (0.0010) [2023-10-14 00:23:22,330][60935] Updated weights for policy 0, policy_version 83030 (0.0008) [2023-10-14 00:23:22,697][60935] Updated weights for policy 0, policy_version 83040 (0.0008) [2023-10-14 00:23:24,195][60934] Updated weights for policy 1, policy_version 82582 (0.0008) [2023-10-14 00:23:24,583][60934] Updated weights for policy 1, policy_version 82592 (0.0007) [2023-10-14 00:23:24,949][60934] Updated weights for policy 1, policy_version 82602 (0.0007) [2023-10-14 00:23:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 170721280. Throughput: 0: 1728.5, 1: 1694.4. Samples: 42692020. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-14 00:23:26,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.100')] [2023-10-14 00:23:26,574][60935] Updated weights for policy 0, policy_version 83050 (0.0008) [2023-10-14 00:23:26,941][60935] Updated weights for policy 0, policy_version 83060 (0.0009) [2023-10-14 00:23:27,306][60935] Updated weights for policy 0, policy_version 83070 (0.0009) [2023-10-14 00:23:28,945][60934] Updated weights for policy 1, policy_version 82612 (0.0008) [2023-10-14 00:23:29,313][60934] Updated weights for policy 1, policy_version 82622 (0.0008) [2023-10-14 00:23:29,685][60934] Updated weights for policy 1, policy_version 82632 (0.0010) [2023-10-14 00:23:31,023][60935] Updated weights for policy 0, policy_version 83080 (0.0008) [2023-10-14 00:23:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 170786816. Throughput: 0: 1711.6, 1: 1722.9. Samples: 42702692. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-14 00:23:31,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.100')] [2023-10-14 00:23:31,394][60935] Updated weights for policy 0, policy_version 83090 (0.0008) [2023-10-14 00:23:31,754][60935] Updated weights for policy 0, policy_version 83100 (0.0009) [2023-10-14 00:23:33,622][60934] Updated weights for policy 1, policy_version 82642 (0.0010) [2023-10-14 00:23:33,989][60934] Updated weights for policy 1, policy_version 82652 (0.0008) [2023-10-14 00:23:34,357][60934] Updated weights for policy 1, policy_version 82662 (0.0009) [2023-10-14 00:23:34,718][60934] Updated weights for policy 1, policy_version 82672 (0.0009) [2023-10-14 00:23:35,722][60935] Updated weights for policy 0, policy_version 83110 (0.0010) [2023-10-14 00:23:36,094][60935] Updated weights for policy 0, policy_version 83120 (0.0009) [2023-10-14 00:23:36,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 170852352. Throughput: 0: 1735.5, 1: 1691.7. Samples: 42722910. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-14 00:23:36,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.110')] [2023-10-14 00:23:36,457][60935] Updated weights for policy 0, policy_version 83130 (0.0008) [2023-10-14 00:23:38,678][60934] Updated weights for policy 1, policy_version 82682 (0.0008) [2023-10-14 00:23:39,048][60934] Updated weights for policy 1, policy_version 82692 (0.0009) [2023-10-14 00:23:39,423][60934] Updated weights for policy 1, policy_version 82702 (0.0009) [2023-10-14 00:23:40,674][60935] Updated weights for policy 0, policy_version 83140 (0.0008) [2023-10-14 00:23:41,052][60935] Updated weights for policy 0, policy_version 83150 (0.0010) [2023-10-14 00:23:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 170917888. Throughput: 0: 1723.7, 1: 1697.4. Samples: 42743264. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-14 00:23:41,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.110')] [2023-10-14 00:23:41,424][60935] Updated weights for policy 0, policy_version 83160 (0.0010) [2023-10-14 00:23:43,373][60934] Updated weights for policy 1, policy_version 82712 (0.0010) [2023-10-14 00:23:43,737][60934] Updated weights for policy 1, policy_version 82722 (0.0010) [2023-10-14 00:23:44,109][60934] Updated weights for policy 1, policy_version 82732 (0.0008) [2023-10-14 00:23:45,350][60935] Updated weights for policy 0, policy_version 83170 (0.0009) [2023-10-14 00:23:45,711][60935] Updated weights for policy 0, policy_version 83180 (0.0007) [2023-10-14 00:23:46,063][60935] Updated weights for policy 0, policy_version 83190 (0.0007) [2023-10-14 00:23:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 170983424. Throughput: 0: 1726.6, 1: 1705.3. Samples: 42753764. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-14 00:23:46,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.110')] [2023-10-14 00:23:46,426][60935] Updated weights for policy 0, policy_version 83200 (0.0010) [2023-10-14 00:23:48,234][60934] Updated weights for policy 1, policy_version 82742 (0.0008) [2023-10-14 00:23:48,599][60934] Updated weights for policy 1, policy_version 82752 (0.0008) [2023-10-14 00:23:48,967][60934] Updated weights for policy 1, policy_version 82762 (0.0008) [2023-10-14 00:23:50,389][60935] Updated weights for policy 0, policy_version 83210 (0.0007) [2023-10-14 00:23:50,754][60935] Updated weights for policy 0, policy_version 83220 (0.0007) [2023-10-14 00:23:51,114][60935] Updated weights for policy 0, policy_version 83230 (0.0010) [2023-10-14 00:23:51,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 171081728. Throughput: 0: 1728.9, 1: 1689.0. Samples: 42774242. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-14 00:23:51,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.110')] [2023-10-14 00:23:52,783][60934] Updated weights for policy 1, policy_version 82772 (0.0007) [2023-10-14 00:23:53,148][60934] Updated weights for policy 1, policy_version 82782 (0.0010) [2023-10-14 00:23:53,512][60934] Updated weights for policy 1, policy_version 82792 (0.0008) [2023-10-14 00:23:55,346][60935] Updated weights for policy 0, policy_version 83240 (0.0011) [2023-10-14 00:23:55,709][60935] Updated weights for policy 0, policy_version 83250 (0.0010) [2023-10-14 00:23:56,079][60935] Updated weights for policy 0, policy_version 83260 (0.0011) [2023-10-14 00:23:56,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 171147264. Throughput: 0: 1703.1, 1: 1720.1. Samples: 42794680. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-14 00:23:56,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.110')] [2023-10-14 00:23:57,268][60934] Updated weights for policy 1, policy_version 82802 (0.0008) [2023-10-14 00:23:57,624][60934] Updated weights for policy 1, policy_version 82812 (0.0011) [2023-10-14 00:23:57,986][60934] Updated weights for policy 1, policy_version 82822 (0.0008) [2023-10-14 00:23:58,349][60934] Updated weights for policy 1, policy_version 82832 (0.0008) [2023-10-14 00:23:59,964][60935] Updated weights for policy 0, policy_version 83270 (0.0009) [2023-10-14 00:24:00,334][60935] Updated weights for policy 0, policy_version 83280 (0.0008) [2023-10-14 00:24:00,713][60935] Updated weights for policy 0, policy_version 83290 (0.0009) [2023-10-14 00:24:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 171212800. Throughput: 0: 1724.2, 1: 1696.9. Samples: 42804902. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 00:24:01,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.110')] [2023-10-14 00:24:02,297][60934] Updated weights for policy 1, policy_version 82842 (0.0008) [2023-10-14 00:24:02,669][60934] Updated weights for policy 1, policy_version 82852 (0.0009) [2023-10-14 00:24:03,033][60934] Updated weights for policy 1, policy_version 82862 (0.0008) [2023-10-14 00:24:04,651][60935] Updated weights for policy 0, policy_version 83300 (0.0008) [2023-10-14 00:24:05,016][60935] Updated weights for policy 0, policy_version 83310 (0.0009) [2023-10-14 00:24:05,384][60935] Updated weights for policy 0, policy_version 83320 (0.0009) [2023-10-14 00:24:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 171278336. Throughput: 0: 1716.6, 1: 1708.9. Samples: 42825872. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 00:24:06,249][59943] Avg episode reward: [(0, '-0.060'), (1, '-0.110')] [2023-10-14 00:24:06,973][60934] Updated weights for policy 1, policy_version 82872 (0.0009) [2023-10-14 00:24:07,344][60934] Updated weights for policy 1, policy_version 82882 (0.0008) [2023-10-14 00:24:07,718][60934] Updated weights for policy 1, policy_version 82892 (0.0007) [2023-10-14 00:24:09,314][60935] Updated weights for policy 0, policy_version 83330 (0.0009) [2023-10-14 00:24:09,686][60935] Updated weights for policy 0, policy_version 83340 (0.0009) [2023-10-14 00:24:10,049][60935] Updated weights for policy 0, policy_version 83350 (0.0008) [2023-10-14 00:24:10,425][60935] Updated weights for policy 0, policy_version 83360 (0.0008) [2023-10-14 00:24:11,249][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 171343872. Throughput: 0: 1691.9, 1: 1738.6. Samples: 42846394. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 00:24:11,250][59943] Avg episode reward: [(0, '0.000'), (1, '-0.110')] [2023-10-14 00:24:11,712][60934] Updated weights for policy 1, policy_version 82902 (0.0009) [2023-10-14 00:24:12,101][60934] Updated weights for policy 1, policy_version 82912 (0.0010) [2023-10-14 00:24:12,479][60934] Updated weights for policy 1, policy_version 82922 (0.0009) [2023-10-14 00:24:14,369][60935] Updated weights for policy 0, policy_version 83370 (0.0009) [2023-10-14 00:24:14,741][60935] Updated weights for policy 0, policy_version 83380 (0.0008) [2023-10-14 00:24:15,118][60935] Updated weights for policy 0, policy_version 83390 (0.0009) [2023-10-14 00:24:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 171409408. Throughput: 0: 1722.8, 1: 1701.3. Samples: 42856776. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 00:24:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:24:16,309][60934] Updated weights for policy 1, policy_version 82932 (0.0007) [2023-10-14 00:24:16,673][60934] Updated weights for policy 1, policy_version 82942 (0.0007) [2023-10-14 00:24:17,037][60934] Updated weights for policy 1, policy_version 82952 (0.0007) [2023-10-14 00:24:19,144][60935] Updated weights for policy 0, policy_version 83400 (0.0009) [2023-10-14 00:24:19,510][60935] Updated weights for policy 0, policy_version 83410 (0.0009) [2023-10-14 00:24:19,876][60935] Updated weights for policy 0, policy_version 83420 (0.0007) [2023-10-14 00:24:20,859][60934] Updated weights for policy 1, policy_version 82962 (0.0009) [2023-10-14 00:24:21,218][60934] Updated weights for policy 1, policy_version 82972 (0.0008) [2023-10-14 00:24:21,248][59943] Fps is (10 sec: 13107.8, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 171474944. Throughput: 0: 1691.3, 1: 1735.8. Samples: 42877126. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 00:24:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:24:21,588][60934] Updated weights for policy 1, policy_version 82982 (0.0007) [2023-10-14 00:24:21,953][60934] Updated weights for policy 1, policy_version 82992 (0.0008) [2023-10-14 00:24:23,851][60935] Updated weights for policy 0, policy_version 83430 (0.0008) [2023-10-14 00:24:24,238][60935] Updated weights for policy 0, policy_version 83440 (0.0011) [2023-10-14 00:24:24,600][60935] Updated weights for policy 0, policy_version 83450 (0.0010) [2023-10-14 00:24:25,971][60934] Updated weights for policy 1, policy_version 83002 (0.0007) [2023-10-14 00:24:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 171540480. Throughput: 0: 1695.1, 1: 1744.7. Samples: 42898054. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 00:24:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:24:26,330][60934] Updated weights for policy 1, policy_version 83012 (0.0007) [2023-10-14 00:24:26,705][60934] Updated weights for policy 1, policy_version 83022 (0.0008) [2023-10-14 00:24:28,659][60935] Updated weights for policy 0, policy_version 83460 (0.0010) [2023-10-14 00:24:29,051][60935] Updated weights for policy 0, policy_version 83470 (0.0009) [2023-10-14 00:24:29,421][60935] Updated weights for policy 0, policy_version 83480 (0.0009) [2023-10-14 00:24:30,574][60934] Updated weights for policy 1, policy_version 83032 (0.0008) [2023-10-14 00:24:30,942][60934] Updated weights for policy 1, policy_version 83042 (0.0008) [2023-10-14 00:24:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 171606016. Throughput: 0: 1706.1, 1: 1723.2. Samples: 42908082. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 00:24:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:24:31,305][60934] Updated weights for policy 1, policy_version 83052 (0.0009) [2023-10-14 00:24:33,531][60935] Updated weights for policy 0, policy_version 83490 (0.0010) [2023-10-14 00:24:33,914][60935] Updated weights for policy 0, policy_version 83500 (0.0007) [2023-10-14 00:24:34,273][60935] Updated weights for policy 0, policy_version 83510 (0.0007) [2023-10-14 00:24:34,641][60935] Updated weights for policy 0, policy_version 83520 (0.0008) [2023-10-14 00:24:35,233][60934] Updated weights for policy 1, policy_version 83062 (0.0007) [2023-10-14 00:24:35,591][60934] Updated weights for policy 1, policy_version 83072 (0.0009) [2023-10-14 00:24:35,961][60934] Updated weights for policy 1, policy_version 83082 (0.0007) [2023-10-14 00:24:36,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 171704320. Throughput: 0: 1679.4, 1: 1740.9. Samples: 42928156. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 00:24:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:24:38,591][60935] Updated weights for policy 0, policy_version 83530 (0.0012) [2023-10-14 00:24:38,975][60935] Updated weights for policy 0, policy_version 83540 (0.0010) [2023-10-14 00:24:39,336][60935] Updated weights for policy 0, policy_version 83550 (0.0010) [2023-10-14 00:24:39,970][60934] Updated weights for policy 1, policy_version 83092 (0.0008) [2023-10-14 00:24:40,333][60934] Updated weights for policy 1, policy_version 83102 (0.0009) [2023-10-14 00:24:40,693][60934] Updated weights for policy 1, policy_version 83112 (0.0009) [2023-10-14 00:24:41,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 171769856. Throughput: 0: 1708.2, 1: 1720.3. Samples: 42948960. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 00:24:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:24:43,183][60935] Updated weights for policy 0, policy_version 83560 (0.0008) [2023-10-14 00:24:43,562][60935] Updated weights for policy 0, policy_version 83570 (0.0011) [2023-10-14 00:24:43,916][60935] Updated weights for policy 0, policy_version 83580 (0.0011) [2023-10-14 00:24:44,829][60934] Updated weights for policy 1, policy_version 83122 (0.0010) [2023-10-14 00:24:45,196][60934] Updated weights for policy 1, policy_version 83132 (0.0007) [2023-10-14 00:24:45,567][60934] Updated weights for policy 1, policy_version 83142 (0.0009) [2023-10-14 00:24:45,933][60934] Updated weights for policy 1, policy_version 83152 (0.0011) [2023-10-14 00:24:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 171835392. Throughput: 0: 1699.9, 1: 1733.3. Samples: 42959394. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 00:24:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:24:47,597][60935] Updated weights for policy 0, policy_version 83590 (0.0008) [2023-10-14 00:24:47,958][60935] Updated weights for policy 0, policy_version 83600 (0.0008) [2023-10-14 00:24:48,324][60935] Updated weights for policy 0, policy_version 83610 (0.0011) [2023-10-14 00:24:49,869][60934] Updated weights for policy 1, policy_version 83162 (0.0008) [2023-10-14 00:24:50,232][60934] Updated weights for policy 1, policy_version 83172 (0.0008) [2023-10-14 00:24:50,594][60934] Updated weights for policy 1, policy_version 83182 (0.0008) [2023-10-14 00:24:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 171900928. Throughput: 0: 1704.9, 1: 1730.7. Samples: 42980476. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 00:24:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:24:52,353][60935] Updated weights for policy 0, policy_version 83620 (0.0009) [2023-10-14 00:24:52,727][60935] Updated weights for policy 0, policy_version 83630 (0.0008) [2023-10-14 00:24:53,102][60935] Updated weights for policy 0, policy_version 83640 (0.0008) [2023-10-14 00:24:54,501][60934] Updated weights for policy 1, policy_version 83192 (0.0008) [2023-10-14 00:24:54,876][60934] Updated weights for policy 1, policy_version 83202 (0.0008) [2023-10-14 00:24:55,255][60934] Updated weights for policy 1, policy_version 83212 (0.0007) [2023-10-14 00:24:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 171966464. Throughput: 0: 1725.1, 1: 1698.2. Samples: 43000442. Policy #0 lag: (min: 9.0, avg: 10.2, max: 27.0) [2023-10-14 00:24:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:24:56,263][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000083648_85655552.pth... [2023-10-14 00:24:56,264][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000083216_86310912.pth... [2023-10-14 00:24:56,301][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000082048_84017152.pth [2023-10-14 00:24:56,303][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000081616_84672512.pth [2023-10-14 00:24:57,169][60935] Updated weights for policy 0, policy_version 83650 (0.0007) [2023-10-14 00:24:57,529][60935] Updated weights for policy 0, policy_version 83660 (0.0008) [2023-10-14 00:24:57,898][60935] Updated weights for policy 0, policy_version 83670 (0.0008) [2023-10-14 00:24:58,263][60935] Updated weights for policy 0, policy_version 83680 (0.0009) [2023-10-14 00:24:59,348][60934] Updated weights for policy 1, policy_version 83222 (0.0008) [2023-10-14 00:24:59,731][60934] Updated weights for policy 1, policy_version 83232 (0.0008) [2023-10-14 00:25:00,088][60934] Updated weights for policy 1, policy_version 83242 (0.0009) [2023-10-14 00:25:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 172032000. Throughput: 0: 1692.0, 1: 1730.5. Samples: 43010786. Policy #0 lag: (min: 9.0, avg: 10.2, max: 27.0) [2023-10-14 00:25:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:25:02,305][60935] Updated weights for policy 0, policy_version 83690 (0.0010) [2023-10-14 00:25:02,666][60935] Updated weights for policy 0, policy_version 83700 (0.0009) [2023-10-14 00:25:03,033][60935] Updated weights for policy 0, policy_version 83710 (0.0009) [2023-10-14 00:25:03,996][60934] Updated weights for policy 1, policy_version 83252 (0.0009) [2023-10-14 00:25:04,368][60934] Updated weights for policy 1, policy_version 83262 (0.0009) [2023-10-14 00:25:04,730][60934] Updated weights for policy 1, policy_version 83272 (0.0008) [2023-10-14 00:25:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 172097536. Throughput: 0: 1721.0, 1: 1706.1. Samples: 43031348. Policy #0 lag: (min: 9.0, avg: 10.2, max: 27.0) [2023-10-14 00:25:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:25:07,108][60935] Updated weights for policy 0, policy_version 83720 (0.0008) [2023-10-14 00:25:07,475][60935] Updated weights for policy 0, policy_version 83730 (0.0010) [2023-10-14 00:25:07,847][60935] Updated weights for policy 0, policy_version 83740 (0.0008) [2023-10-14 00:25:08,694][60934] Updated weights for policy 1, policy_version 83282 (0.0010) [2023-10-14 00:25:09,062][60934] Updated weights for policy 1, policy_version 83292 (0.0011) [2023-10-14 00:25:09,432][60934] Updated weights for policy 1, policy_version 83302 (0.0009) [2023-10-14 00:25:09,796][60934] Updated weights for policy 1, policy_version 83312 (0.0008) [2023-10-14 00:25:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 172163072. Throughput: 0: 1729.5, 1: 1690.3. Samples: 43051946. Policy #0 lag: (min: 9.0, avg: 10.2, max: 27.0) [2023-10-14 00:25:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:25:11,776][60935] Updated weights for policy 0, policy_version 83750 (0.0011) [2023-10-14 00:25:12,148][60935] Updated weights for policy 0, policy_version 83760 (0.0009) [2023-10-14 00:25:12,514][60935] Updated weights for policy 0, policy_version 83770 (0.0008) [2023-10-14 00:25:13,793][60934] Updated weights for policy 1, policy_version 83322 (0.0009) [2023-10-14 00:25:14,157][60934] Updated weights for policy 1, policy_version 83332 (0.0009) [2023-10-14 00:25:14,527][60934] Updated weights for policy 1, policy_version 83342 (0.0008) [2023-10-14 00:25:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 172228608. Throughput: 0: 1710.4, 1: 1720.0. Samples: 43062450. Policy #0 lag: (min: 9.0, avg: 10.2, max: 27.0) [2023-10-14 00:25:16,249][59943] Avg episode reward: [(0, '-0.010'), (1, '-0.020')] [2023-10-14 00:25:16,507][60935] Updated weights for policy 0, policy_version 83780 (0.0008) [2023-10-14 00:25:16,890][60935] Updated weights for policy 0, policy_version 83790 (0.0009) [2023-10-14 00:25:17,264][60935] Updated weights for policy 0, policy_version 83800 (0.0008) [2023-10-14 00:25:18,626][60934] Updated weights for policy 1, policy_version 83352 (0.0009) [2023-10-14 00:25:18,985][60934] Updated weights for policy 1, policy_version 83362 (0.0007) [2023-10-14 00:25:19,353][60934] Updated weights for policy 1, policy_version 83372 (0.0008) [2023-10-14 00:25:21,199][60935] Updated weights for policy 0, policy_version 83810 (0.0007) [2023-10-14 00:25:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 172294144. Throughput: 0: 1737.0, 1: 1688.8. Samples: 43082314. Policy #0 lag: (min: 9.0, avg: 10.2, max: 27.0) [2023-10-14 00:25:21,249][59943] Avg episode reward: [(0, '-0.010'), (1, '-0.020')] [2023-10-14 00:25:21,563][60935] Updated weights for policy 0, policy_version 83820 (0.0007) [2023-10-14 00:25:21,940][60935] Updated weights for policy 0, policy_version 83830 (0.0010) [2023-10-14 00:25:22,307][60935] Updated weights for policy 0, policy_version 83840 (0.0008) [2023-10-14 00:25:23,308][60934] Updated weights for policy 1, policy_version 83382 (0.0009) [2023-10-14 00:25:23,677][60934] Updated weights for policy 1, policy_version 83392 (0.0008) [2023-10-14 00:25:24,043][60934] Updated weights for policy 1, policy_version 83402 (0.0007) [2023-10-14 00:25:26,232][60935] Updated weights for policy 0, policy_version 83850 (0.0011) [2023-10-14 00:25:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 172359680. Throughput: 0: 1730.4, 1: 1710.0. Samples: 43103774. Policy #0 lag: (min: 9.0, avg: 10.2, max: 27.0) [2023-10-14 00:25:26,249][59943] Avg episode reward: [(0, '-0.020'), (1, '-0.020')] [2023-10-14 00:25:26,598][60935] Updated weights for policy 0, policy_version 83860 (0.0010) [2023-10-14 00:25:26,972][60935] Updated weights for policy 0, policy_version 83870 (0.0007) [2023-10-14 00:25:28,032][60934] Updated weights for policy 1, policy_version 83412 (0.0009) [2023-10-14 00:25:28,402][60934] Updated weights for policy 1, policy_version 83422 (0.0009) [2023-10-14 00:25:28,769][60934] Updated weights for policy 1, policy_version 83432 (0.0007) [2023-10-14 00:25:30,952][60935] Updated weights for policy 0, policy_version 83880 (0.0008) [2023-10-14 00:25:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 172425216. Throughput: 0: 1721.3, 1: 1708.8. Samples: 43113752. Policy #0 lag: (min: 9.0, avg: 10.2, max: 27.0) [2023-10-14 00:25:31,249][59943] Avg episode reward: [(0, '-0.020'), (1, '-0.020')] [2023-10-14 00:25:31,321][60935] Updated weights for policy 0, policy_version 83890 (0.0008) [2023-10-14 00:25:31,684][60935] Updated weights for policy 0, policy_version 83900 (0.0008) [2023-10-14 00:25:32,686][60934] Updated weights for policy 1, policy_version 83442 (0.0009) [2023-10-14 00:25:33,052][60934] Updated weights for policy 1, policy_version 83452 (0.0008) [2023-10-14 00:25:33,422][60934] Updated weights for policy 1, policy_version 83462 (0.0008) [2023-10-14 00:25:33,788][60934] Updated weights for policy 1, policy_version 83472 (0.0009) [2023-10-14 00:25:35,723][60935] Updated weights for policy 0, policy_version 83910 (0.0008) [2023-10-14 00:25:36,094][60935] Updated weights for policy 0, policy_version 83920 (0.0007) [2023-10-14 00:25:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 172490752. Throughput: 0: 1716.5, 1: 1703.7. Samples: 43134386. Policy #0 lag: (min: 9.0, avg: 10.2, max: 27.0) [2023-10-14 00:25:36,249][59943] Avg episode reward: [(0, '-0.020'), (1, '-0.020')] [2023-10-14 00:25:36,469][60935] Updated weights for policy 0, policy_version 83930 (0.0009) [2023-10-14 00:25:37,776][60934] Updated weights for policy 1, policy_version 83482 (0.0009) [2023-10-14 00:25:38,138][60934] Updated weights for policy 1, policy_version 83492 (0.0009) [2023-10-14 00:25:38,496][60934] Updated weights for policy 1, policy_version 83502 (0.0010) [2023-10-14 00:25:40,455][60935] Updated weights for policy 0, policy_version 83940 (0.0010) [2023-10-14 00:25:40,832][60935] Updated weights for policy 0, policy_version 83950 (0.0009) [2023-10-14 00:25:41,207][60935] Updated weights for policy 0, policy_version 83960 (0.0008) [2023-10-14 00:25:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 172556288. Throughput: 0: 1703.7, 1: 1729.2. Samples: 43154922. Policy #0 lag: (min: 9.0, avg: 10.2, max: 27.0) [2023-10-14 00:25:41,249][59943] Avg episode reward: [(0, '-0.020'), (1, '-0.020')] [2023-10-14 00:25:42,497][60934] Updated weights for policy 1, policy_version 83512 (0.0009) [2023-10-14 00:25:42,861][60934] Updated weights for policy 1, policy_version 83522 (0.0009) [2023-10-14 00:25:43,241][60934] Updated weights for policy 1, policy_version 83532 (0.0007) [2023-10-14 00:25:45,348][60935] Updated weights for policy 0, policy_version 83970 (0.0010) [2023-10-14 00:25:45,715][60935] Updated weights for policy 0, policy_version 83980 (0.0011) [2023-10-14 00:25:46,089][60935] Updated weights for policy 0, policy_version 83990 (0.0009) [2023-10-14 00:25:46,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13662.6). Total num frames: 172621824. Throughput: 0: 1716.7, 1: 1702.2. Samples: 43164634. Policy #0 lag: (min: 9.0, avg: 10.2, max: 27.0) [2023-10-14 00:25:46,249][59943] Avg episode reward: [(0, '-0.020'), (1, '-0.020')] [2023-10-14 00:25:46,446][60935] Updated weights for policy 0, policy_version 84000 (0.0011) [2023-10-14 00:25:47,236][60934] Updated weights for policy 1, policy_version 83542 (0.0009) [2023-10-14 00:25:47,625][60934] Updated weights for policy 1, policy_version 83552 (0.0010) [2023-10-14 00:25:47,990][60934] Updated weights for policy 1, policy_version 83562 (0.0009) [2023-10-14 00:25:50,534][60935] Updated weights for policy 0, policy_version 84010 (0.0009) [2023-10-14 00:25:50,899][60935] Updated weights for policy 0, policy_version 84020 (0.0008) [2023-10-14 00:25:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 172687360. Throughput: 0: 1711.7, 1: 1712.7. Samples: 43185446. Policy #0 lag: (min: 13.0, avg: 36.3, max: 40.0) [2023-10-14 00:25:51,249][59943] Avg episode reward: [(0, '-0.020'), (1, '-0.020')] [2023-10-14 00:25:51,275][60935] Updated weights for policy 0, policy_version 84030 (0.0007) [2023-10-14 00:25:52,039][60934] Updated weights for policy 1, policy_version 83572 (0.0010) [2023-10-14 00:25:52,407][60934] Updated weights for policy 1, policy_version 83582 (0.0009) [2023-10-14 00:25:52,776][60934] Updated weights for policy 1, policy_version 83592 (0.0009) [2023-10-14 00:25:55,215][60935] Updated weights for policy 0, policy_version 84040 (0.0008) [2023-10-14 00:25:55,589][60935] Updated weights for policy 0, policy_version 84050 (0.0009) [2023-10-14 00:25:55,955][60935] Updated weights for policy 0, policy_version 84060 (0.0009) [2023-10-14 00:25:56,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 172785664. Throughput: 0: 1692.0, 1: 1724.4. Samples: 43205686. Policy #0 lag: (min: 13.0, avg: 36.3, max: 40.0) [2023-10-14 00:25:56,249][59943] Avg episode reward: [(0, '-0.010'), (1, '-0.020')] [2023-10-14 00:25:56,702][60934] Updated weights for policy 1, policy_version 83602 (0.0008) [2023-10-14 00:25:57,071][60934] Updated weights for policy 1, policy_version 83612 (0.0008) [2023-10-14 00:25:57,442][60934] Updated weights for policy 1, policy_version 83622 (0.0007) [2023-10-14 00:25:57,813][60934] Updated weights for policy 1, policy_version 83632 (0.0008) [2023-10-14 00:25:59,967][60935] Updated weights for policy 0, policy_version 84070 (0.0009) [2023-10-14 00:26:00,337][60935] Updated weights for policy 0, policy_version 84080 (0.0007) [2023-10-14 00:26:00,713][60935] Updated weights for policy 0, policy_version 84090 (0.0008) [2023-10-14 00:26:01,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 172851200. Throughput: 0: 1710.5, 1: 1697.9. Samples: 43215828. Policy #0 lag: (min: 13.0, avg: 36.3, max: 40.0) [2023-10-14 00:26:01,249][59943] Avg episode reward: [(0, '-0.010'), (1, '-0.020')] [2023-10-14 00:26:01,803][60934] Updated weights for policy 1, policy_version 83642 (0.0008) [2023-10-14 00:26:02,162][60934] Updated weights for policy 1, policy_version 83652 (0.0009) [2023-10-14 00:26:02,534][60934] Updated weights for policy 1, policy_version 83662 (0.0010) [2023-10-14 00:26:04,772][60935] Updated weights for policy 0, policy_version 84100 (0.0008) [2023-10-14 00:26:05,164][60935] Updated weights for policy 0, policy_version 84110 (0.0007) [2023-10-14 00:26:05,530][60935] Updated weights for policy 0, policy_version 84120 (0.0008) [2023-10-14 00:26:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 172916736. Throughput: 0: 1701.9, 1: 1729.1. Samples: 43236706. Policy #0 lag: (min: 13.0, avg: 36.3, max: 40.0) [2023-10-14 00:26:06,249][59943] Avg episode reward: [(0, '-0.010'), (1, '-0.020')] [2023-10-14 00:26:06,510][60934] Updated weights for policy 1, policy_version 83672 (0.0008) [2023-10-14 00:26:06,877][60934] Updated weights for policy 1, policy_version 83682 (0.0008) [2023-10-14 00:26:07,245][60934] Updated weights for policy 1, policy_version 83692 (0.0007) [2023-10-14 00:26:09,379][60935] Updated weights for policy 0, policy_version 84130 (0.0008) [2023-10-14 00:26:09,752][60935] Updated weights for policy 0, policy_version 84140 (0.0009) [2023-10-14 00:26:10,123][60935] Updated weights for policy 0, policy_version 84150 (0.0007) [2023-10-14 00:26:10,489][60935] Updated weights for policy 0, policy_version 84160 (0.0008) [2023-10-14 00:26:11,029][60934] Updated weights for policy 1, policy_version 83702 (0.0008) [2023-10-14 00:26:11,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 172982272. Throughput: 0: 1677.1, 1: 1733.0. Samples: 43257226. Policy #0 lag: (min: 13.0, avg: 36.3, max: 40.0) [2023-10-14 00:26:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:26:11,397][60934] Updated weights for policy 1, policy_version 83712 (0.0009) [2023-10-14 00:26:11,767][60934] Updated weights for policy 1, policy_version 83722 (0.0009) [2023-10-14 00:26:14,288][60935] Updated weights for policy 0, policy_version 84170 (0.0009) [2023-10-14 00:26:14,665][60935] Updated weights for policy 0, policy_version 84180 (0.0007) [2023-10-14 00:26:15,030][60935] Updated weights for policy 0, policy_version 84190 (0.0009) [2023-10-14 00:26:15,698][60934] Updated weights for policy 1, policy_version 83732 (0.0007) [2023-10-14 00:26:16,059][60934] Updated weights for policy 1, policy_version 83742 (0.0009) [2023-10-14 00:26:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 173047808. Throughput: 0: 1710.6, 1: 1719.3. Samples: 43268098. Policy #0 lag: (min: 13.0, avg: 36.3, max: 40.0) [2023-10-14 00:26:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:26:16,431][60934] Updated weights for policy 1, policy_version 83752 (0.0011) [2023-10-14 00:26:19,058][60935] Updated weights for policy 0, policy_version 84200 (0.0008) [2023-10-14 00:26:19,418][60935] Updated weights for policy 0, policy_version 84210 (0.0007) [2023-10-14 00:26:19,793][60935] Updated weights for policy 0, policy_version 84220 (0.0007) [2023-10-14 00:26:20,272][60934] Updated weights for policy 1, policy_version 83762 (0.0011) [2023-10-14 00:26:20,626][60934] Updated weights for policy 1, policy_version 83772 (0.0008) [2023-10-14 00:26:20,987][60934] Updated weights for policy 1, policy_version 83782 (0.0007) [2023-10-14 00:26:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 173113344. Throughput: 0: 1687.6, 1: 1731.2. Samples: 43288234. Policy #0 lag: (min: 13.0, avg: 36.3, max: 40.0) [2023-10-14 00:26:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:26:21,350][60934] Updated weights for policy 1, policy_version 83792 (0.0008) [2023-10-14 00:26:23,924][60935] Updated weights for policy 0, policy_version 84230 (0.0007) [2023-10-14 00:26:24,291][60935] Updated weights for policy 0, policy_version 84240 (0.0009) [2023-10-14 00:26:24,660][60935] Updated weights for policy 0, policy_version 84250 (0.0007) [2023-10-14 00:26:25,260][60934] Updated weights for policy 1, policy_version 83802 (0.0007) [2023-10-14 00:26:25,626][60934] Updated weights for policy 1, policy_version 83812 (0.0007) [2023-10-14 00:26:25,995][60934] Updated weights for policy 1, policy_version 83822 (0.0007) [2023-10-14 00:26:26,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 173211648. Throughput: 0: 1689.5, 1: 1721.0. Samples: 43308398. Policy #0 lag: (min: 13.0, avg: 36.3, max: 40.0) [2023-10-14 00:26:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:26:28,841][60935] Updated weights for policy 0, policy_version 84260 (0.0008) [2023-10-14 00:26:29,207][60935] Updated weights for policy 0, policy_version 84270 (0.0010) [2023-10-14 00:26:29,573][60935] Updated weights for policy 0, policy_version 84280 (0.0009) [2023-10-14 00:26:29,882][60934] Updated weights for policy 1, policy_version 83832 (0.0007) [2023-10-14 00:26:30,250][60934] Updated weights for policy 1, policy_version 83842 (0.0007) [2023-10-14 00:26:30,623][60934] Updated weights for policy 1, policy_version 83852 (0.0007) [2023-10-14 00:26:31,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 173277184. Throughput: 0: 1700.3, 1: 1741.3. Samples: 43319504. Policy #0 lag: (min: 13.0, avg: 36.3, max: 40.0) [2023-10-14 00:26:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:26:33,620][60935] Updated weights for policy 0, policy_version 84290 (0.0008) [2023-10-14 00:26:33,986][60935] Updated weights for policy 0, policy_version 84300 (0.0010) [2023-10-14 00:26:34,346][60935] Updated weights for policy 0, policy_version 84310 (0.0007) [2023-10-14 00:26:34,598][60934] Updated weights for policy 1, policy_version 83862 (0.0008) [2023-10-14 00:26:34,717][60935] Updated weights for policy 0, policy_version 84320 (0.0009) [2023-10-14 00:26:34,987][60934] Updated weights for policy 1, policy_version 83872 (0.0009) [2023-10-14 00:26:35,350][60934] Updated weights for policy 1, policy_version 83882 (0.0011) [2023-10-14 00:26:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 173342720. Throughput: 0: 1676.5, 1: 1742.1. Samples: 43339282. Policy #0 lag: (min: 13.0, avg: 36.3, max: 40.0) [2023-10-14 00:26:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:26:38,762][60935] Updated weights for policy 0, policy_version 84330 (0.0009) [2023-10-14 00:26:39,146][60935] Updated weights for policy 0, policy_version 84340 (0.0010) [2023-10-14 00:26:39,176][60934] Updated weights for policy 1, policy_version 83892 (0.0009) [2023-10-14 00:26:39,511][60935] Updated weights for policy 0, policy_version 84350 (0.0009) [2023-10-14 00:26:39,551][60934] Updated weights for policy 1, policy_version 83902 (0.0007) [2023-10-14 00:26:39,916][60934] Updated weights for policy 1, policy_version 83912 (0.0008) [2023-10-14 00:26:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 173408256. Throughput: 0: 1693.0, 1: 1721.9. Samples: 43359356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:26:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:26:43,641][60935] Updated weights for policy 0, policy_version 84360 (0.0008) [2023-10-14 00:26:43,852][60934] Updated weights for policy 1, policy_version 83922 (0.0008) [2023-10-14 00:26:44,009][60935] Updated weights for policy 0, policy_version 84370 (0.0007) [2023-10-14 00:26:44,214][60934] Updated weights for policy 1, policy_version 83932 (0.0008) [2023-10-14 00:26:44,382][60935] Updated weights for policy 0, policy_version 84380 (0.0008) [2023-10-14 00:26:44,581][60934] Updated weights for policy 1, policy_version 83942 (0.0008) [2023-10-14 00:26:44,942][60934] Updated weights for policy 1, policy_version 83952 (0.0007) [2023-10-14 00:26:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 173473792. Throughput: 0: 1689.5, 1: 1747.8. Samples: 43370504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:26:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:26:48,372][60935] Updated weights for policy 0, policy_version 84390 (0.0009) [2023-10-14 00:26:48,742][60935] Updated weights for policy 0, policy_version 84400 (0.0009) [2023-10-14 00:26:48,998][60934] Updated weights for policy 1, policy_version 83962 (0.0009) [2023-10-14 00:26:49,114][60935] Updated weights for policy 0, policy_version 84410 (0.0007) [2023-10-14 00:26:49,360][60934] Updated weights for policy 1, policy_version 83972 (0.0008) [2023-10-14 00:26:49,728][60934] Updated weights for policy 1, policy_version 83982 (0.0008) [2023-10-14 00:26:51,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 173539328. Throughput: 0: 1676.4, 1: 1724.7. Samples: 43389752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:26:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:26:53,120][60935] Updated weights for policy 0, policy_version 84420 (0.0010) [2023-10-14 00:26:53,501][60935] Updated weights for policy 0, policy_version 84430 (0.0008) [2023-10-14 00:26:53,721][60934] Updated weights for policy 1, policy_version 83992 (0.0007) [2023-10-14 00:26:53,872][60935] Updated weights for policy 0, policy_version 84440 (0.0007) [2023-10-14 00:26:54,092][60934] Updated weights for policy 1, policy_version 84002 (0.0007) [2023-10-14 00:26:54,466][60934] Updated weights for policy 1, policy_version 84012 (0.0009) [2023-10-14 00:26:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 173604864. Throughput: 0: 1699.9, 1: 1704.5. Samples: 43410422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:26:56,250][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:26:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000084016_87130112.pth... [2023-10-14 00:26:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000084448_86474752.pth... [2023-10-14 00:26:56,291][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000082416_85491712.pth [2023-10-14 00:26:56,295][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000082848_84836352.pth [2023-10-14 00:26:57,954][60935] Updated weights for policy 0, policy_version 84450 (0.0007) [2023-10-14 00:26:58,331][60935] Updated weights for policy 0, policy_version 84460 (0.0007) [2023-10-14 00:26:58,536][60934] Updated weights for policy 1, policy_version 84022 (0.0009) [2023-10-14 00:26:58,697][60935] Updated weights for policy 0, policy_version 84470 (0.0007) [2023-10-14 00:26:58,909][60934] Updated weights for policy 1, policy_version 84032 (0.0009) [2023-10-14 00:26:59,060][60935] Updated weights for policy 0, policy_version 84480 (0.0008) [2023-10-14 00:26:59,264][60934] Updated weights for policy 1, policy_version 84042 (0.0009) [2023-10-14 00:27:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 173670400. Throughput: 0: 1670.1, 1: 1726.9. Samples: 43420966. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:27:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-14 00:27:03,105][60935] Updated weights for policy 0, policy_version 84490 (0.0008) [2023-10-14 00:27:03,425][60934] Updated weights for policy 1, policy_version 84052 (0.0009) [2023-10-14 00:27:03,482][60935] Updated weights for policy 0, policy_version 84500 (0.0008) [2023-10-14 00:27:03,795][60934] Updated weights for policy 1, policy_version 84062 (0.0009) [2023-10-14 00:27:03,841][60935] Updated weights for policy 0, policy_version 84510 (0.0008) [2023-10-14 00:27:04,164][60934] Updated weights for policy 1, policy_version 84072 (0.0007) [2023-10-14 00:27:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 173735936. Throughput: 0: 1689.6, 1: 1698.1. Samples: 43440680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:27:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-14 00:27:07,785][60935] Updated weights for policy 0, policy_version 84520 (0.0007) [2023-10-14 00:27:08,024][60934] Updated weights for policy 1, policy_version 84082 (0.0007) [2023-10-14 00:27:08,151][60935] Updated weights for policy 0, policy_version 84530 (0.0007) [2023-10-14 00:27:08,381][60934] Updated weights for policy 1, policy_version 84092 (0.0009) [2023-10-14 00:27:08,518][60935] Updated weights for policy 0, policy_version 84540 (0.0009) [2023-10-14 00:27:08,743][60934] Updated weights for policy 1, policy_version 84102 (0.0008) [2023-10-14 00:27:09,109][60934] Updated weights for policy 1, policy_version 84112 (0.0009) [2023-10-14 00:27:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 173801472. Throughput: 0: 1700.6, 1: 1713.2. Samples: 43462018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:27:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.010')] [2023-10-14 00:27:12,500][60935] Updated weights for policy 0, policy_version 84550 (0.0009) [2023-10-14 00:27:12,867][60935] Updated weights for policy 0, policy_version 84560 (0.0009) [2023-10-14 00:27:13,074][60934] Updated weights for policy 1, policy_version 84122 (0.0008) [2023-10-14 00:27:13,242][60935] Updated weights for policy 0, policy_version 84570 (0.0008) [2023-10-14 00:27:13,440][60934] Updated weights for policy 1, policy_version 84132 (0.0007) [2023-10-14 00:27:13,802][60934] Updated weights for policy 1, policy_version 84142 (0.0008) [2023-10-14 00:27:16,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 173867008. Throughput: 0: 1678.4, 1: 1700.9. Samples: 43471572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:27:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:27:17,158][60935] Updated weights for policy 0, policy_version 84580 (0.0009) [2023-10-14 00:27:17,535][60935] Updated weights for policy 0, policy_version 84590 (0.0008) [2023-10-14 00:27:17,905][60935] Updated weights for policy 0, policy_version 84600 (0.0007) [2023-10-14 00:27:17,978][60934] Updated weights for policy 1, policy_version 84152 (0.0007) [2023-10-14 00:27:18,348][60934] Updated weights for policy 1, policy_version 84162 (0.0007) [2023-10-14 00:27:18,709][60934] Updated weights for policy 1, policy_version 84172 (0.0009) [2023-10-14 00:27:21,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 173932544. Throughput: 0: 1700.8, 1: 1694.8. Samples: 43492086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:27:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:27:21,989][60935] Updated weights for policy 0, policy_version 84610 (0.0008) [2023-10-14 00:27:22,350][60935] Updated weights for policy 0, policy_version 84620 (0.0008) [2023-10-14 00:27:22,620][60934] Updated weights for policy 1, policy_version 84182 (0.0008) [2023-10-14 00:27:22,715][60935] Updated weights for policy 0, policy_version 84630 (0.0007) [2023-10-14 00:27:23,000][60934] Updated weights for policy 1, policy_version 84192 (0.0008) [2023-10-14 00:27:23,082][60935] Updated weights for policy 0, policy_version 84640 (0.0007) [2023-10-14 00:27:23,372][60934] Updated weights for policy 1, policy_version 84202 (0.0008) [2023-10-14 00:27:26,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 173998080. Throughput: 0: 1702.4, 1: 1714.6. Samples: 43513124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:27:26,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:27:27,012][60935] Updated weights for policy 0, policy_version 84650 (0.0010) [2023-10-14 00:27:27,378][60935] Updated weights for policy 0, policy_version 84660 (0.0008) [2023-10-14 00:27:27,452][60934] Updated weights for policy 1, policy_version 84212 (0.0009) [2023-10-14 00:27:27,747][60935] Updated weights for policy 0, policy_version 84670 (0.0008) [2023-10-14 00:27:27,818][60934] Updated weights for policy 1, policy_version 84222 (0.0010) [2023-10-14 00:27:28,187][60934] Updated weights for policy 1, policy_version 84232 (0.0008) [2023-10-14 00:27:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 174063616. Throughput: 0: 1690.9, 1: 1685.8. Samples: 43522458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:27:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:27:31,750][60935] Updated weights for policy 0, policy_version 84680 (0.0009) [2023-10-14 00:27:32,111][60935] Updated weights for policy 0, policy_version 84690 (0.0008) [2023-10-14 00:27:32,186][60934] Updated weights for policy 1, policy_version 84242 (0.0009) [2023-10-14 00:27:32,481][60935] Updated weights for policy 0, policy_version 84700 (0.0007) [2023-10-14 00:27:32,548][60934] Updated weights for policy 1, policy_version 84252 (0.0008) [2023-10-14 00:27:32,912][60934] Updated weights for policy 1, policy_version 84262 (0.0008) [2023-10-14 00:27:33,277][60934] Updated weights for policy 1, policy_version 84272 (0.0010) [2023-10-14 00:27:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13662.6). Total num frames: 174129152. Throughput: 0: 1712.0, 1: 1706.0. Samples: 43543562. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:27:36,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:27:36,523][60935] Updated weights for policy 0, policy_version 84710 (0.0008) [2023-10-14 00:27:36,895][60935] Updated weights for policy 0, policy_version 84720 (0.0009) [2023-10-14 00:27:37,239][60934] Updated weights for policy 1, policy_version 84282 (0.0007) [2023-10-14 00:27:37,254][60935] Updated weights for policy 0, policy_version 84730 (0.0008) [2023-10-14 00:27:37,604][60934] Updated weights for policy 1, policy_version 84292 (0.0009) [2023-10-14 00:27:37,969][60934] Updated weights for policy 1, policy_version 84302 (0.0009) [2023-10-14 00:27:41,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 174194688. Throughput: 0: 1713.6, 1: 1717.9. Samples: 43564836. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:27:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:27:41,278][60935] Updated weights for policy 0, policy_version 84740 (0.0008) [2023-10-14 00:27:41,656][60935] Updated weights for policy 0, policy_version 84750 (0.0007) [2023-10-14 00:27:41,838][60934] Updated weights for policy 1, policy_version 84312 (0.0008) [2023-10-14 00:27:42,022][60935] Updated weights for policy 0, policy_version 84760 (0.0010) [2023-10-14 00:27:42,206][60934] Updated weights for policy 1, policy_version 84322 (0.0007) [2023-10-14 00:27:42,573][60934] Updated weights for policy 1, policy_version 84332 (0.0007) [2023-10-14 00:27:45,927][60935] Updated weights for policy 0, policy_version 84770 (0.0009) [2023-10-14 00:27:46,248][59943] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 174260224. Throughput: 0: 1708.6, 1: 1694.8. Samples: 43574120. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:27:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:27:46,291][60935] Updated weights for policy 0, policy_version 84780 (0.0010) [2023-10-14 00:27:46,618][60934] Updated weights for policy 1, policy_version 84342 (0.0007) [2023-10-14 00:27:46,661][60935] Updated weights for policy 0, policy_version 84790 (0.0007) [2023-10-14 00:27:46,980][60934] Updated weights for policy 1, policy_version 84352 (0.0007) [2023-10-14 00:27:47,024][60935] Updated weights for policy 0, policy_version 84800 (0.0008) [2023-10-14 00:27:47,344][60934] Updated weights for policy 1, policy_version 84362 (0.0009) [2023-10-14 00:27:51,085][60935] Updated weights for policy 0, policy_version 84810 (0.0008) [2023-10-14 00:27:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 174325760. Throughput: 0: 1713.8, 1: 1717.2. Samples: 43595076. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:27:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:27:51,267][60934] Updated weights for policy 1, policy_version 84372 (0.0009) [2023-10-14 00:27:51,448][60935] Updated weights for policy 0, policy_version 84820 (0.0008) [2023-10-14 00:27:51,639][60934] Updated weights for policy 1, policy_version 84382 (0.0009) [2023-10-14 00:27:51,818][60935] Updated weights for policy 0, policy_version 84830 (0.0008) [2023-10-14 00:27:51,997][60934] Updated weights for policy 1, policy_version 84392 (0.0009) [2023-10-14 00:27:55,741][60935] Updated weights for policy 0, policy_version 84840 (0.0009) [2023-10-14 00:27:56,089][60934] Updated weights for policy 1, policy_version 84402 (0.0007) [2023-10-14 00:27:56,120][60935] Updated weights for policy 0, policy_version 84850 (0.0010) [2023-10-14 00:27:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 174391296. Throughput: 0: 1708.8, 1: 1710.8. Samples: 43615902. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:27:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:27:56,449][60934] Updated weights for policy 1, policy_version 84412 (0.0008) [2023-10-14 00:27:56,485][60935] Updated weights for policy 0, policy_version 84860 (0.0007) [2023-10-14 00:27:56,820][60934] Updated weights for policy 1, policy_version 84422 (0.0008) [2023-10-14 00:27:57,181][60934] Updated weights for policy 1, policy_version 84432 (0.0009) [2023-10-14 00:28:00,492][60935] Updated weights for policy 0, policy_version 84870 (0.0008) [2023-10-14 00:28:00,863][60935] Updated weights for policy 0, policy_version 84880 (0.0007) [2023-10-14 00:28:01,236][60935] Updated weights for policy 0, policy_version 84890 (0.0008) [2023-10-14 00:28:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 174456832. Throughput: 0: 1719.7, 1: 1702.0. Samples: 43625552. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:28:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:28:01,292][60934] Updated weights for policy 1, policy_version 84442 (0.0007) [2023-10-14 00:28:01,652][60934] Updated weights for policy 1, policy_version 84452 (0.0008) [2023-10-14 00:28:02,018][60934] Updated weights for policy 1, policy_version 84462 (0.0010) [2023-10-14 00:28:05,121][60935] Updated weights for policy 0, policy_version 84900 (0.0008) [2023-10-14 00:28:05,494][60935] Updated weights for policy 0, policy_version 84910 (0.0008) [2023-10-14 00:28:05,855][60935] Updated weights for policy 0, policy_version 84920 (0.0008) [2023-10-14 00:28:06,042][60934] Updated weights for policy 1, policy_version 84472 (0.0007) [2023-10-14 00:28:06,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 174555136. Throughput: 0: 1718.4, 1: 1711.4. Samples: 43646426. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:28:06,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:28:06,406][60934] Updated weights for policy 1, policy_version 84482 (0.0007) [2023-10-14 00:28:06,774][60934] Updated weights for policy 1, policy_version 84492 (0.0007) [2023-10-14 00:28:09,854][60935] Updated weights for policy 0, policy_version 84930 (0.0008) [2023-10-14 00:28:10,224][60935] Updated weights for policy 0, policy_version 84940 (0.0009) [2023-10-14 00:28:10,596][60935] Updated weights for policy 0, policy_version 84950 (0.0010) [2023-10-14 00:28:10,767][60934] Updated weights for policy 1, policy_version 84502 (0.0008) [2023-10-14 00:28:10,967][60935] Updated weights for policy 0, policy_version 84960 (0.0007) [2023-10-14 00:28:11,164][60934] Updated weights for policy 1, policy_version 84512 (0.0007) [2023-10-14 00:28:11,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 174620672. Throughput: 0: 1693.5, 1: 1714.0. Samples: 43666460. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:28:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:28:11,535][60934] Updated weights for policy 1, policy_version 84522 (0.0009) [2023-10-14 00:28:14,842][60935] Updated weights for policy 0, policy_version 84970 (0.0008) [2023-10-14 00:28:15,213][60935] Updated weights for policy 0, policy_version 84980 (0.0008) [2023-10-14 00:28:15,557][60934] Updated weights for policy 1, policy_version 84532 (0.0009) [2023-10-14 00:28:15,575][60935] Updated weights for policy 0, policy_version 84990 (0.0009) [2023-10-14 00:28:15,912][60934] Updated weights for policy 1, policy_version 84542 (0.0008) [2023-10-14 00:28:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 174686208. Throughput: 0: 1718.5, 1: 1713.7. Samples: 43676908. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:28:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:28:16,273][60934] Updated weights for policy 1, policy_version 84552 (0.0009) [2023-10-14 00:28:19,556][60935] Updated weights for policy 0, policy_version 85000 (0.0011) [2023-10-14 00:28:19,935][60935] Updated weights for policy 0, policy_version 85010 (0.0008) [2023-10-14 00:28:20,255][60934] Updated weights for policy 1, policy_version 84562 (0.0009) [2023-10-14 00:28:20,303][60935] Updated weights for policy 0, policy_version 85020 (0.0007) [2023-10-14 00:28:20,626][60934] Updated weights for policy 1, policy_version 84572 (0.0009) [2023-10-14 00:28:20,993][60934] Updated weights for policy 1, policy_version 84582 (0.0007) [2023-10-14 00:28:21,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 174751744. Throughput: 0: 1705.3, 1: 1714.9. Samples: 43697470. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 00:28:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:28:21,351][60934] Updated weights for policy 1, policy_version 84592 (0.0007) [2023-10-14 00:28:24,304][60935] Updated weights for policy 0, policy_version 85030 (0.0009) [2023-10-14 00:28:24,682][60935] Updated weights for policy 0, policy_version 85040 (0.0009) [2023-10-14 00:28:25,055][60935] Updated weights for policy 0, policy_version 85050 (0.0008) [2023-10-14 00:28:25,106][60934] Updated weights for policy 1, policy_version 84602 (0.0009) [2023-10-14 00:28:25,481][60934] Updated weights for policy 1, policy_version 84612 (0.0009) [2023-10-14 00:28:25,840][60934] Updated weights for policy 1, policy_version 84622 (0.0010) [2023-10-14 00:28:26,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 174850048. Throughput: 0: 1692.8, 1: 1695.1. Samples: 43717288. Policy #0 lag: (min: 16.0, avg: 40.2, max: 48.0) [2023-10-14 00:28:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:28:29,024][60935] Updated weights for policy 0, policy_version 85060 (0.0009) [2023-10-14 00:28:29,409][60935] Updated weights for policy 0, policy_version 85070 (0.0008) [2023-10-14 00:28:29,774][60934] Updated weights for policy 1, policy_version 84632 (0.0008) [2023-10-14 00:28:29,777][60935] Updated weights for policy 0, policy_version 85080 (0.0008) [2023-10-14 00:28:30,137][60934] Updated weights for policy 1, policy_version 84642 (0.0009) [2023-10-14 00:28:30,506][60934] Updated weights for policy 1, policy_version 84652 (0.0008) [2023-10-14 00:28:31,248][59943] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 174915584. Throughput: 0: 1720.1, 1: 1711.2. Samples: 43728528. Policy #0 lag: (min: 16.0, avg: 40.2, max: 48.0) [2023-10-14 00:28:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:28:33,770][60935] Updated weights for policy 0, policy_version 85090 (0.0010) [2023-10-14 00:28:34,130][60935] Updated weights for policy 0, policy_version 85100 (0.0007) [2023-10-14 00:28:34,450][60934] Updated weights for policy 1, policy_version 84662 (0.0008) [2023-10-14 00:28:34,503][60935] Updated weights for policy 0, policy_version 85110 (0.0008) [2023-10-14 00:28:34,822][60934] Updated weights for policy 1, policy_version 84672 (0.0008) [2023-10-14 00:28:34,866][60935] Updated weights for policy 0, policy_version 85120 (0.0008) [2023-10-14 00:28:35,182][60934] Updated weights for policy 1, policy_version 84682 (0.0007) [2023-10-14 00:28:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 174981120. Throughput: 0: 1695.7, 1: 1711.6. Samples: 43748404. Policy #0 lag: (min: 16.0, avg: 40.2, max: 48.0) [2023-10-14 00:28:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:28:38,863][60935] Updated weights for policy 0, policy_version 85130 (0.0009) [2023-10-14 00:28:39,122][60934] Updated weights for policy 1, policy_version 84692 (0.0007) [2023-10-14 00:28:39,229][60935] Updated weights for policy 0, policy_version 85140 (0.0009) [2023-10-14 00:28:39,483][60934] Updated weights for policy 1, policy_version 84702 (0.0007) [2023-10-14 00:28:39,596][60935] Updated weights for policy 0, policy_version 85150 (0.0008) [2023-10-14 00:28:39,850][60934] Updated weights for policy 1, policy_version 84712 (0.0008) [2023-10-14 00:28:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 175046656. Throughput: 0: 1699.4, 1: 1693.6. Samples: 43768588. Policy #0 lag: (min: 16.0, avg: 40.2, max: 48.0) [2023-10-14 00:28:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:28:43,527][60935] Updated weights for policy 0, policy_version 85160 (0.0007) [2023-10-14 00:28:43,791][60934] Updated weights for policy 1, policy_version 84722 (0.0009) [2023-10-14 00:28:43,890][60935] Updated weights for policy 0, policy_version 85170 (0.0007) [2023-10-14 00:28:44,164][60934] Updated weights for policy 1, policy_version 84732 (0.0007) [2023-10-14 00:28:44,260][60935] Updated weights for policy 0, policy_version 85180 (0.0008) [2023-10-14 00:28:44,540][60934] Updated weights for policy 1, policy_version 84742 (0.0008) [2023-10-14 00:28:44,900][60934] Updated weights for policy 1, policy_version 84752 (0.0009) [2023-10-14 00:28:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 175112192. Throughput: 0: 1708.3, 1: 1724.1. Samples: 43780010. Policy #0 lag: (min: 16.0, avg: 40.2, max: 48.0) [2023-10-14 00:28:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:28:48,201][60935] Updated weights for policy 0, policy_version 85190 (0.0008) [2023-10-14 00:28:48,565][60935] Updated weights for policy 0, policy_version 85200 (0.0009) [2023-10-14 00:28:48,931][60935] Updated weights for policy 0, policy_version 85210 (0.0008) [2023-10-14 00:28:48,993][60934] Updated weights for policy 1, policy_version 84762 (0.0008) [2023-10-14 00:28:49,364][60934] Updated weights for policy 1, policy_version 84772 (0.0007) [2023-10-14 00:28:49,730][60934] Updated weights for policy 1, policy_version 84782 (0.0007) [2023-10-14 00:28:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 175177728. Throughput: 0: 1697.5, 1: 1700.4. Samples: 43799330. Policy #0 lag: (min: 16.0, avg: 40.2, max: 48.0) [2023-10-14 00:28:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:28:52,925][60935] Updated weights for policy 0, policy_version 85220 (0.0008) [2023-10-14 00:28:53,285][60935] Updated weights for policy 0, policy_version 85230 (0.0008) [2023-10-14 00:28:53,506][60934] Updated weights for policy 1, policy_version 84792 (0.0008) [2023-10-14 00:28:53,646][60935] Updated weights for policy 0, policy_version 85240 (0.0008) [2023-10-14 00:28:53,874][60934] Updated weights for policy 1, policy_version 84802 (0.0007) [2023-10-14 00:28:54,242][60934] Updated weights for policy 1, policy_version 84812 (0.0009) [2023-10-14 00:28:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 175243264. Throughput: 0: 1726.2, 1: 1696.7. Samples: 43820492. Policy #0 lag: (min: 16.0, avg: 40.2, max: 48.0) [2023-10-14 00:28:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:28:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000085248_87293952.pth... [2023-10-14 00:28:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000084816_87949312.pth... [2023-10-14 00:28:56,297][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000083216_86310912.pth [2023-10-14 00:28:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000083648_85655552.pth [2023-10-14 00:28:57,675][60935] Updated weights for policy 0, policy_version 85250 (0.0008) [2023-10-14 00:28:58,039][60935] Updated weights for policy 0, policy_version 85260 (0.0008) [2023-10-14 00:28:58,383][60934] Updated weights for policy 1, policy_version 84822 (0.0009) [2023-10-14 00:28:58,400][60935] Updated weights for policy 0, policy_version 85270 (0.0009) [2023-10-14 00:28:58,764][60934] Updated weights for policy 1, policy_version 84832 (0.0009) [2023-10-14 00:28:58,779][60935] Updated weights for policy 0, policy_version 85280 (0.0008) [2023-10-14 00:28:59,135][60934] Updated weights for policy 1, policy_version 84842 (0.0009) [2023-10-14 00:29:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 175308800. Throughput: 0: 1697.2, 1: 1713.4. Samples: 43830384. Policy #0 lag: (min: 16.0, avg: 40.2, max: 48.0) [2023-10-14 00:29:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:29:02,958][60935] Updated weights for policy 0, policy_version 85290 (0.0009) [2023-10-14 00:29:03,140][60934] Updated weights for policy 1, policy_version 84852 (0.0009) [2023-10-14 00:29:03,317][60935] Updated weights for policy 0, policy_version 85300 (0.0009) [2023-10-14 00:29:03,511][60934] Updated weights for policy 1, policy_version 84862 (0.0009) [2023-10-14 00:29:03,689][60935] Updated weights for policy 0, policy_version 85310 (0.0009) [2023-10-14 00:29:03,876][60934] Updated weights for policy 1, policy_version 84872 (0.0008) [2023-10-14 00:29:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 175374336. Throughput: 0: 1705.0, 1: 1694.7. Samples: 43850456. Policy #0 lag: (min: 16.0, avg: 40.2, max: 48.0) [2023-10-14 00:29:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:29:07,663][60935] Updated weights for policy 0, policy_version 85320 (0.0009) [2023-10-14 00:29:07,978][60934] Updated weights for policy 1, policy_version 84882 (0.0007) [2023-10-14 00:29:08,027][60935] Updated weights for policy 0, policy_version 85330 (0.0009) [2023-10-14 00:29:08,339][60934] Updated weights for policy 1, policy_version 84892 (0.0008) [2023-10-14 00:29:08,391][60935] Updated weights for policy 0, policy_version 85340 (0.0009) [2023-10-14 00:29:08,709][60934] Updated weights for policy 1, policy_version 84902 (0.0010) [2023-10-14 00:29:09,072][60934] Updated weights for policy 1, policy_version 84912 (0.0008) [2023-10-14 00:29:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 175439872. Throughput: 0: 1712.1, 1: 1713.9. Samples: 43871458. Policy #0 lag: (min: 16.0, avg: 40.2, max: 48.0) [2023-10-14 00:29:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:29:12,450][60935] Updated weights for policy 0, policy_version 85350 (0.0008) [2023-10-14 00:29:12,809][60935] Updated weights for policy 0, policy_version 85360 (0.0008) [2023-10-14 00:29:13,083][60934] Updated weights for policy 1, policy_version 84922 (0.0009) [2023-10-14 00:29:13,183][60935] Updated weights for policy 0, policy_version 85370 (0.0007) [2023-10-14 00:29:13,453][60934] Updated weights for policy 1, policy_version 84932 (0.0009) [2023-10-14 00:29:13,805][60934] Updated weights for policy 1, policy_version 84942 (0.0009) [2023-10-14 00:29:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 175505408. Throughput: 0: 1689.5, 1: 1705.3. Samples: 43881294. Policy #0 lag: (min: 16.0, avg: 40.2, max: 48.0) [2023-10-14 00:29:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:29:17,001][60935] Updated weights for policy 0, policy_version 85380 (0.0008) [2023-10-14 00:29:17,395][60935] Updated weights for policy 0, policy_version 85390 (0.0008) [2023-10-14 00:29:17,736][60934] Updated weights for policy 1, policy_version 84952 (0.0008) [2023-10-14 00:29:17,760][60935] Updated weights for policy 0, policy_version 85400 (0.0009) [2023-10-14 00:29:18,102][60934] Updated weights for policy 1, policy_version 84962 (0.0008) [2023-10-14 00:29:18,458][60934] Updated weights for policy 1, policy_version 84972 (0.0010) [2023-10-14 00:29:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 175570944. Throughput: 0: 1713.4, 1: 1698.2. Samples: 43901924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:29:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:29:21,642][60935] Updated weights for policy 0, policy_version 85410 (0.0010) [2023-10-14 00:29:22,017][60935] Updated weights for policy 0, policy_version 85420 (0.0010) [2023-10-14 00:29:22,383][60935] Updated weights for policy 0, policy_version 85430 (0.0009) [2023-10-14 00:29:22,474][60934] Updated weights for policy 1, policy_version 84982 (0.0009) [2023-10-14 00:29:22,747][60935] Updated weights for policy 0, policy_version 85440 (0.0010) [2023-10-14 00:29:22,840][60934] Updated weights for policy 1, policy_version 84992 (0.0007) [2023-10-14 00:29:23,211][60934] Updated weights for policy 1, policy_version 85002 (0.0008) [2023-10-14 00:29:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 175636480. Throughput: 0: 1715.7, 1: 1717.0. Samples: 43923060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:29:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:29:26,711][60935] Updated weights for policy 0, policy_version 85450 (0.0011) [2023-10-14 00:29:27,072][60935] Updated weights for policy 0, policy_version 85460 (0.0008) [2023-10-14 00:29:27,282][60934] Updated weights for policy 1, policy_version 85012 (0.0009) [2023-10-14 00:29:27,443][60935] Updated weights for policy 0, policy_version 85470 (0.0008) [2023-10-14 00:29:27,646][60934] Updated weights for policy 1, policy_version 85022 (0.0010) [2023-10-14 00:29:28,019][60934] Updated weights for policy 1, policy_version 85032 (0.0008) [2023-10-14 00:29:31,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13551.5). Total num frames: 175702016. Throughput: 0: 1699.3, 1: 1686.7. Samples: 43932378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:29:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:29:31,526][60935] Updated weights for policy 0, policy_version 85480 (0.0007) [2023-10-14 00:29:31,897][60935] Updated weights for policy 0, policy_version 85490 (0.0009) [2023-10-14 00:29:32,036][60934] Updated weights for policy 1, policy_version 85042 (0.0009) [2023-10-14 00:29:32,264][60935] Updated weights for policy 0, policy_version 85500 (0.0009) [2023-10-14 00:29:32,404][60934] Updated weights for policy 1, policy_version 85052 (0.0007) [2023-10-14 00:29:32,762][60934] Updated weights for policy 1, policy_version 85062 (0.0009) [2023-10-14 00:29:33,132][60934] Updated weights for policy 1, policy_version 85072 (0.0009) [2023-10-14 00:29:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 175767552. Throughput: 0: 1710.5, 1: 1709.1. Samples: 43953212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:29:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:29:36,309][60935] Updated weights for policy 0, policy_version 85510 (0.0007) [2023-10-14 00:29:36,677][60935] Updated weights for policy 0, policy_version 85520 (0.0007) [2023-10-14 00:29:37,046][60935] Updated weights for policy 0, policy_version 85530 (0.0007) [2023-10-14 00:29:37,114][60934] Updated weights for policy 1, policy_version 85082 (0.0009) [2023-10-14 00:29:37,481][60934] Updated weights for policy 1, policy_version 85092 (0.0008) [2023-10-14 00:29:37,860][60934] Updated weights for policy 1, policy_version 85102 (0.0008) [2023-10-14 00:29:40,867][60935] Updated weights for policy 0, policy_version 85540 (0.0009) [2023-10-14 00:29:41,237][60935] Updated weights for policy 0, policy_version 85550 (0.0007) [2023-10-14 00:29:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13551.5). Total num frames: 175833088. Throughput: 0: 1710.3, 1: 1707.9. Samples: 43974310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:29:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:29:41,604][60935] Updated weights for policy 0, policy_version 85560 (0.0009) [2023-10-14 00:29:41,904][60934] Updated weights for policy 1, policy_version 85112 (0.0009) [2023-10-14 00:29:42,261][60934] Updated weights for policy 1, policy_version 85122 (0.0009) [2023-10-14 00:29:42,622][60934] Updated weights for policy 1, policy_version 85132 (0.0011) [2023-10-14 00:29:45,705][60935] Updated weights for policy 0, policy_version 85570 (0.0008) [2023-10-14 00:29:46,081][60935] Updated weights for policy 0, policy_version 85580 (0.0007) [2023-10-14 00:29:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 175898624. Throughput: 0: 1715.5, 1: 1689.6. Samples: 43983612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:29:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:29:46,461][60935] Updated weights for policy 0, policy_version 85590 (0.0007) [2023-10-14 00:29:46,752][60934] Updated weights for policy 1, policy_version 85142 (0.0009) [2023-10-14 00:29:46,822][60935] Updated weights for policy 0, policy_version 85600 (0.0007) [2023-10-14 00:29:47,139][60934] Updated weights for policy 1, policy_version 85152 (0.0011) [2023-10-14 00:29:47,501][60934] Updated weights for policy 1, policy_version 85162 (0.0008) [2023-10-14 00:29:50,842][60935] Updated weights for policy 0, policy_version 85610 (0.0008) [2023-10-14 00:29:51,211][60935] Updated weights for policy 0, policy_version 85620 (0.0007) [2023-10-14 00:29:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 175964160. Throughput: 0: 1718.8, 1: 1701.3. Samples: 44004358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:29:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:29:51,469][60934] Updated weights for policy 1, policy_version 85172 (0.0007) [2023-10-14 00:29:51,568][60935] Updated weights for policy 0, policy_version 85630 (0.0009) [2023-10-14 00:29:51,836][60934] Updated weights for policy 1, policy_version 85182 (0.0007) [2023-10-14 00:29:52,198][60934] Updated weights for policy 1, policy_version 85192 (0.0008) [2023-10-14 00:29:55,561][60935] Updated weights for policy 0, policy_version 85640 (0.0008) [2023-10-14 00:29:55,922][60935] Updated weights for policy 0, policy_version 85650 (0.0009) [2023-10-14 00:29:56,224][60934] Updated weights for policy 1, policy_version 85202 (0.0010) [2023-10-14 00:29:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 176029696. Throughput: 0: 1710.7, 1: 1702.4. Samples: 44025046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:29:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:29:56,297][60935] Updated weights for policy 0, policy_version 85660 (0.0009) [2023-10-14 00:29:56,590][60934] Updated weights for policy 1, policy_version 85212 (0.0007) [2023-10-14 00:29:56,946][60934] Updated weights for policy 1, policy_version 85222 (0.0008) [2023-10-14 00:29:57,314][60934] Updated weights for policy 1, policy_version 85232 (0.0007) [2023-10-14 00:30:00,402][60935] Updated weights for policy 0, policy_version 85670 (0.0007) [2023-10-14 00:30:00,772][60935] Updated weights for policy 0, policy_version 85680 (0.0007) [2023-10-14 00:30:01,142][60935] Updated weights for policy 0, policy_version 85690 (0.0007) [2023-10-14 00:30:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 176095232. Throughput: 0: 1718.8, 1: 1696.3. Samples: 44034970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:30:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:30:01,399][60934] Updated weights for policy 1, policy_version 85242 (0.0008) [2023-10-14 00:30:01,768][60934] Updated weights for policy 1, policy_version 85252 (0.0008) [2023-10-14 00:30:02,130][60934] Updated weights for policy 1, policy_version 85262 (0.0010) [2023-10-14 00:30:05,227][60935] Updated weights for policy 0, policy_version 85700 (0.0009) [2023-10-14 00:30:05,613][60935] Updated weights for policy 0, policy_version 85710 (0.0007) [2023-10-14 00:30:05,979][60935] Updated weights for policy 0, policy_version 85720 (0.0009) [2023-10-14 00:30:06,055][60934] Updated weights for policy 1, policy_version 85272 (0.0008) [2023-10-14 00:30:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 176160768. Throughput: 0: 1717.9, 1: 1711.1. Samples: 44056226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:30:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:30:06,424][60934] Updated weights for policy 1, policy_version 85282 (0.0008) [2023-10-14 00:30:06,787][60934] Updated weights for policy 1, policy_version 85292 (0.0008) [2023-10-14 00:30:09,985][60935] Updated weights for policy 0, policy_version 85730 (0.0007) [2023-10-14 00:30:10,362][60935] Updated weights for policy 0, policy_version 85740 (0.0009) [2023-10-14 00:30:10,729][60935] Updated weights for policy 0, policy_version 85750 (0.0010) [2023-10-14 00:30:10,746][60934] Updated weights for policy 1, policy_version 85302 (0.0008) [2023-10-14 00:30:11,105][60935] Updated weights for policy 0, policy_version 85760 (0.0009) [2023-10-14 00:30:11,106][60934] Updated weights for policy 1, policy_version 85312 (0.0007) [2023-10-14 00:30:11,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 176259072. Throughput: 0: 1694.1, 1: 1714.9. Samples: 44076464. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) [2023-10-14 00:30:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:30:11,469][60934] Updated weights for policy 1, policy_version 85322 (0.0007) [2023-10-14 00:30:15,035][60935] Updated weights for policy 0, policy_version 85770 (0.0009) [2023-10-14 00:30:15,378][60934] Updated weights for policy 1, policy_version 85332 (0.0007) [2023-10-14 00:30:15,410][60935] Updated weights for policy 0, policy_version 85780 (0.0009) [2023-10-14 00:30:15,743][60934] Updated weights for policy 1, policy_version 85342 (0.0007) [2023-10-14 00:30:15,776][60935] Updated weights for policy 0, policy_version 85790 (0.0009) [2023-10-14 00:30:16,114][60934] Updated weights for policy 1, policy_version 85352 (0.0009) [2023-10-14 00:30:16,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 176324608. Throughput: 0: 1714.1, 1: 1716.3. Samples: 44086742. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) [2023-10-14 00:30:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:30:19,756][60935] Updated weights for policy 0, policy_version 85800 (0.0009) [2023-10-14 00:30:19,906][60934] Updated weights for policy 1, policy_version 85362 (0.0008) [2023-10-14 00:30:20,125][60935] Updated weights for policy 0, policy_version 85810 (0.0010) [2023-10-14 00:30:20,266][60934] Updated weights for policy 1, policy_version 85372 (0.0009) [2023-10-14 00:30:20,492][60935] Updated weights for policy 0, policy_version 85820 (0.0007) [2023-10-14 00:30:20,632][60934] Updated weights for policy 1, policy_version 85382 (0.0009) [2023-10-14 00:30:20,999][60934] Updated weights for policy 1, policy_version 85392 (0.0007) [2023-10-14 00:30:21,248][59943] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 176422912. Throughput: 0: 1708.6, 1: 1726.6. Samples: 44107794. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) [2023-10-14 00:30:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:30:24,395][60935] Updated weights for policy 0, policy_version 85830 (0.0009) [2023-10-14 00:30:24,767][60935] Updated weights for policy 0, policy_version 85840 (0.0009) [2023-10-14 00:30:25,035][60934] Updated weights for policy 1, policy_version 85402 (0.0008) [2023-10-14 00:30:25,133][60935] Updated weights for policy 0, policy_version 85850 (0.0008) [2023-10-14 00:30:25,389][60934] Updated weights for policy 1, policy_version 85412 (0.0008) [2023-10-14 00:30:25,757][60934] Updated weights for policy 1, policy_version 85422 (0.0009) [2023-10-14 00:30:26,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 176488448. Throughput: 0: 1688.8, 1: 1703.7. Samples: 44126972. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) [2023-10-14 00:30:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:30:29,084][60935] Updated weights for policy 0, policy_version 85860 (0.0008) [2023-10-14 00:30:29,447][60935] Updated weights for policy 0, policy_version 85870 (0.0007) [2023-10-14 00:30:29,820][60935] Updated weights for policy 0, policy_version 85880 (0.0007) [2023-10-14 00:30:29,823][60934] Updated weights for policy 1, policy_version 85432 (0.0008) [2023-10-14 00:30:30,185][60934] Updated weights for policy 1, policy_version 85442 (0.0009) [2023-10-14 00:30:30,551][60934] Updated weights for policy 1, policy_version 85452 (0.0007) [2023-10-14 00:30:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 176553984. Throughput: 0: 1715.8, 1: 1727.4. Samples: 44138556. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) [2023-10-14 00:30:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:30:33,733][60935] Updated weights for policy 0, policy_version 85890 (0.0008) [2023-10-14 00:30:34,101][60935] Updated weights for policy 0, policy_version 85900 (0.0011) [2023-10-14 00:30:34,473][60935] Updated weights for policy 0, policy_version 85910 (0.0009) [2023-10-14 00:30:34,673][60934] Updated weights for policy 1, policy_version 85462 (0.0008) [2023-10-14 00:30:34,832][60935] Updated weights for policy 0, policy_version 85920 (0.0009) [2023-10-14 00:30:35,058][60934] Updated weights for policy 1, policy_version 85472 (0.0009) [2023-10-14 00:30:35,418][60934] Updated weights for policy 1, policy_version 85482 (0.0010) [2023-10-14 00:30:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 176619520. Throughput: 0: 1690.1, 1: 1726.4. Samples: 44158100. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) [2023-10-14 00:30:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:30:38,910][60935] Updated weights for policy 0, policy_version 85930 (0.0010) [2023-10-14 00:30:39,268][60935] Updated weights for policy 0, policy_version 85940 (0.0010) [2023-10-14 00:30:39,387][60934] Updated weights for policy 1, policy_version 85492 (0.0009) [2023-10-14 00:30:39,644][60935] Updated weights for policy 0, policy_version 85950 (0.0009) [2023-10-14 00:30:39,753][60934] Updated weights for policy 1, policy_version 85502 (0.0008) [2023-10-14 00:30:40,122][60934] Updated weights for policy 1, policy_version 85512 (0.0007) [2023-10-14 00:30:41,248][59943] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 176685056. Throughput: 0: 1695.4, 1: 1696.3. Samples: 44177672. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) [2023-10-14 00:30:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:30:43,597][60935] Updated weights for policy 0, policy_version 85960 (0.0010) [2023-10-14 00:30:43,933][60934] Updated weights for policy 1, policy_version 85522 (0.0007) [2023-10-14 00:30:43,960][60935] Updated weights for policy 0, policy_version 85970 (0.0008) [2023-10-14 00:30:44,295][60934] Updated weights for policy 1, policy_version 85532 (0.0008) [2023-10-14 00:30:44,328][60935] Updated weights for policy 0, policy_version 85980 (0.0009) [2023-10-14 00:30:44,661][60934] Updated weights for policy 1, policy_version 85542 (0.0009) [2023-10-14 00:30:45,031][60934] Updated weights for policy 1, policy_version 85552 (0.0008) [2023-10-14 00:30:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 176750592. Throughput: 0: 1701.7, 1: 1728.0. Samples: 44189304. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) [2023-10-14 00:30:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:30:48,175][60935] Updated weights for policy 0, policy_version 85990 (0.0009) [2023-10-14 00:30:48,541][60935] Updated weights for policy 0, policy_version 86000 (0.0008) [2023-10-14 00:30:48,911][60935] Updated weights for policy 0, policy_version 86010 (0.0009) [2023-10-14 00:30:49,142][60934] Updated weights for policy 1, policy_version 85562 (0.0007) [2023-10-14 00:30:49,495][60934] Updated weights for policy 1, policy_version 85572 (0.0007) [2023-10-14 00:30:49,860][60934] Updated weights for policy 1, policy_version 85582 (0.0007) [2023-10-14 00:30:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 176816128. Throughput: 0: 1691.1, 1: 1703.0. Samples: 44208960. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) [2023-10-14 00:30:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:30:52,975][60935] Updated weights for policy 0, policy_version 86020 (0.0009) [2023-10-14 00:30:53,362][60935] Updated weights for policy 0, policy_version 86030 (0.0010) [2023-10-14 00:30:53,733][60935] Updated weights for policy 0, policy_version 86040 (0.0010) [2023-10-14 00:30:53,862][60934] Updated weights for policy 1, policy_version 85592 (0.0007) [2023-10-14 00:30:54,228][60934] Updated weights for policy 1, policy_version 85602 (0.0011) [2023-10-14 00:30:54,583][60934] Updated weights for policy 1, policy_version 85612 (0.0009) [2023-10-14 00:30:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 176881664. Throughput: 0: 1708.5, 1: 1688.6. Samples: 44229334. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) [2023-10-14 00:30:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:30:56,257][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000086048_88113152.pth... [2023-10-14 00:30:56,257][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000085616_88768512.pth... [2023-10-14 00:30:56,294][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000084016_87130112.pth [2023-10-14 00:30:56,296][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000084448_86474752.pth [2023-10-14 00:30:57,740][60935] Updated weights for policy 0, policy_version 86050 (0.0009) [2023-10-14 00:30:58,103][60935] Updated weights for policy 0, policy_version 86060 (0.0009) [2023-10-14 00:30:58,466][60935] Updated weights for policy 0, policy_version 86070 (0.0011) [2023-10-14 00:30:58,541][60934] Updated weights for policy 1, policy_version 85622 (0.0007) [2023-10-14 00:30:58,838][60935] Updated weights for policy 0, policy_version 86080 (0.0010) [2023-10-14 00:30:58,905][60934] Updated weights for policy 1, policy_version 85632 (0.0007) [2023-10-14 00:30:59,270][60934] Updated weights for policy 1, policy_version 85642 (0.0008) [2023-10-14 00:31:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 176947200. Throughput: 0: 1688.0, 1: 1713.7. Samples: 44239816. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) [2023-10-14 00:31:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:31:02,739][60935] Updated weights for policy 0, policy_version 86090 (0.0009) [2023-10-14 00:31:03,100][60935] Updated weights for policy 0, policy_version 86100 (0.0007) [2023-10-14 00:31:03,179][60934] Updated weights for policy 1, policy_version 85652 (0.0010) [2023-10-14 00:31:03,470][60935] Updated weights for policy 0, policy_version 86110 (0.0007) [2023-10-14 00:31:03,551][60934] Updated weights for policy 1, policy_version 85662 (0.0010) [2023-10-14 00:31:03,921][60934] Updated weights for policy 1, policy_version 85672 (0.0007) [2023-10-14 00:31:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 177012736. Throughput: 0: 1698.9, 1: 1685.9. Samples: 44260108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:31:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:31:07,530][60935] Updated weights for policy 0, policy_version 86120 (0.0009) [2023-10-14 00:31:07,775][60934] Updated weights for policy 1, policy_version 85682 (0.0008) [2023-10-14 00:31:07,903][60935] Updated weights for policy 0, policy_version 86130 (0.0008) [2023-10-14 00:31:08,143][60934] Updated weights for policy 1, policy_version 85692 (0.0008) [2023-10-14 00:31:08,263][60935] Updated weights for policy 0, policy_version 86140 (0.0009) [2023-10-14 00:31:08,509][60934] Updated weights for policy 1, policy_version 85702 (0.0008) [2023-10-14 00:31:08,872][60934] Updated weights for policy 1, policy_version 85712 (0.0008) [2023-10-14 00:31:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 177078272. Throughput: 0: 1712.4, 1: 1712.5. Samples: 44281096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:31:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:31:12,356][60935] Updated weights for policy 0, policy_version 86150 (0.0011) [2023-10-14 00:31:12,720][60935] Updated weights for policy 0, policy_version 86160 (0.0009) [2023-10-14 00:31:12,908][60934] Updated weights for policy 1, policy_version 85722 (0.0008) [2023-10-14 00:31:13,081][60935] Updated weights for policy 0, policy_version 86170 (0.0009) [2023-10-14 00:31:13,266][60934] Updated weights for policy 1, policy_version 85732 (0.0007) [2023-10-14 00:31:13,633][60934] Updated weights for policy 1, policy_version 85742 (0.0008) [2023-10-14 00:31:16,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 177143808. Throughput: 0: 1678.9, 1: 1699.1. Samples: 44290566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:31:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:31:17,132][60935] Updated weights for policy 0, policy_version 86180 (0.0010) [2023-10-14 00:31:17,494][60935] Updated weights for policy 0, policy_version 86190 (0.0007) [2023-10-14 00:31:17,538][60934] Updated weights for policy 1, policy_version 85752 (0.0007) [2023-10-14 00:31:17,862][60935] Updated weights for policy 0, policy_version 86200 (0.0007) [2023-10-14 00:31:17,904][60934] Updated weights for policy 1, policy_version 85762 (0.0007) [2023-10-14 00:31:18,121][60828] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000009 [2023-10-14 00:31:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 177209344. Throughput: 0: 1702.0, 1: 1710.2. Samples: 44311646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:31:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:31:22,010][60935] Updated weights for policy 0, policy_version 86210 (0.0007) [2023-10-14 00:31:22,065][60934] Updated weights for policy 1, policy_version 85772 (0.0008) [2023-10-14 00:31:22,382][60935] Updated weights for policy 0, policy_version 86220 (0.0008) [2023-10-14 00:31:22,463][60934] Updated weights for policy 1, policy_version 85782 (0.0007) [2023-10-14 00:31:22,739][60935] Updated weights for policy 0, policy_version 86230 (0.0007) [2023-10-14 00:31:22,822][60828] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000009 [2023-10-14 00:31:22,826][60934] Updated weights for policy 1, policy_version 85792 (0.0007) [2023-10-14 00:31:23,107][60935] Updated weights for policy 0, policy_version 86240 (0.0010) [2023-10-14 00:31:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 177274880. Throughput: 0: 1710.0, 1: 1748.5. Samples: 44333302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:31:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:31:26,829][60934] Updated weights for policy 1, policy_version 85802 (0.0007) [2023-10-14 00:31:27,045][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000002 [2023-10-14 00:31:27,100][60935] Updated weights for policy 0, policy_version 86250 (0.0009) [2023-10-14 00:31:27,475][60935] Updated weights for policy 0, policy_version 86260 (0.0011) [2023-10-14 00:31:27,834][60935] Updated weights for policy 0, policy_version 86270 (0.0010) [2023-10-14 00:31:31,159][60934] Updated weights for policy 1, policy_version 85812 (0.0007) [2023-10-14 00:31:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 177340416. Throughput: 0: 1693.4, 1: 1732.9. Samples: 44343488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:31:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:31:31,531][60934] Updated weights for policy 1, policy_version 85822 (0.0007) [2023-10-14 00:31:31,774][60935] Updated weights for policy 0, policy_version 86280 (0.0010) [2023-10-14 00:31:31,894][60934] Updated weights for policy 1, policy_version 85832 (0.0009) [2023-10-14 00:31:32,137][60935] Updated weights for policy 0, policy_version 86290 (0.0009) [2023-10-14 00:31:32,514][60935] Updated weights for policy 0, policy_version 86300 (0.0007) [2023-10-14 00:31:35,936][60934] Updated weights for policy 1, policy_version 85842 (0.0010) [2023-10-14 00:31:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 177405952. Throughput: 0: 1707.0, 1: 1749.2. Samples: 44364492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:31:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:31:36,306][60934] Updated weights for policy 1, policy_version 85852 (0.0008) [2023-10-14 00:31:36,429][60935] Updated weights for policy 0, policy_version 86310 (0.0008) [2023-10-14 00:31:36,668][60934] Updated weights for policy 1, policy_version 85862 (0.0010) [2023-10-14 00:31:36,800][60935] Updated weights for policy 0, policy_version 86320 (0.0009) [2023-10-14 00:31:37,030][60934] Updated weights for policy 1, policy_version 85872 (0.0009) [2023-10-14 00:31:37,173][60935] Updated weights for policy 0, policy_version 86330 (0.0009) [2023-10-14 00:31:40,887][60934] Updated weights for policy 1, policy_version 85882 (0.0008) [2023-10-14 00:31:41,198][60935] Updated weights for policy 0, policy_version 86340 (0.0008) [2023-10-14 00:31:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 177471488. Throughput: 0: 1709.3, 1: 1766.6. Samples: 44385750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:31:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:31:41,256][60934] Updated weights for policy 1, policy_version 85892 (0.0008) [2023-10-14 00:31:41,573][60935] Updated weights for policy 0, policy_version 86350 (0.0007) [2023-10-14 00:31:41,615][60934] Updated weights for policy 1, policy_version 85902 (0.0009) [2023-10-14 00:31:41,938][60935] Updated weights for policy 0, policy_version 86360 (0.0007) [2023-10-14 00:31:45,594][60934] Updated weights for policy 1, policy_version 85912 (0.0007) [2023-10-14 00:31:45,928][60935] Updated weights for policy 0, policy_version 86370 (0.0007) [2023-10-14 00:31:45,957][60934] Updated weights for policy 1, policy_version 85922 (0.0007) [2023-10-14 00:31:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 177537024. Throughput: 0: 1708.7, 1: 1740.1. Samples: 44395010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:31:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:31:46,290][60935] Updated weights for policy 0, policy_version 86380 (0.0007) [2023-10-14 00:31:46,322][60934] Updated weights for policy 1, policy_version 85932 (0.0007) [2023-10-14 00:31:46,668][60935] Updated weights for policy 0, policy_version 86390 (0.0008) [2023-10-14 00:31:47,035][60935] Updated weights for policy 0, policy_version 86400 (0.0009) [2023-10-14 00:31:50,348][60934] Updated weights for policy 1, policy_version 85942 (0.0007) [2023-10-14 00:31:50,717][60934] Updated weights for policy 1, policy_version 85952 (0.0007) [2023-10-14 00:31:51,020][60935] Updated weights for policy 0, policy_version 86410 (0.0008) [2023-10-14 00:31:51,077][60934] Updated weights for policy 1, policy_version 85962 (0.0008) [2023-10-14 00:31:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 177602560. Throughput: 0: 1706.4, 1: 1763.8. Samples: 44416264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:31:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:31:51,379][60935] Updated weights for policy 0, policy_version 86420 (0.0008) [2023-10-14 00:31:51,743][60935] Updated weights for policy 0, policy_version 86430 (0.0009) [2023-10-14 00:31:55,028][60934] Updated weights for policy 1, policy_version 85972 (0.0007) [2023-10-14 00:31:55,401][60934] Updated weights for policy 1, policy_version 85982 (0.0008) [2023-10-14 00:31:55,759][60934] Updated weights for policy 1, policy_version 85992 (0.0008) [2023-10-14 00:31:55,789][60935] Updated weights for policy 0, policy_version 86440 (0.0009) [2023-10-14 00:31:56,154][60935] Updated weights for policy 0, policy_version 86450 (0.0007) [2023-10-14 00:31:56,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 177700864. Throughput: 0: 1704.2, 1: 1749.8. Samples: 44436528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:31:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:31:56,523][60935] Updated weights for policy 0, policy_version 86460 (0.0009) [2023-10-14 00:31:59,710][60934] Updated weights for policy 1, policy_version 86002 (0.0008) [2023-10-14 00:32:00,082][60934] Updated weights for policy 1, policy_version 86012 (0.0010) [2023-10-14 00:32:00,436][60934] Updated weights for policy 1, policy_version 86022 (0.0010) [2023-10-14 00:32:00,572][60935] Updated weights for policy 0, policy_version 86470 (0.0007) [2023-10-14 00:32:00,804][60934] Updated weights for policy 1, policy_version 86032 (0.0007) [2023-10-14 00:32:00,937][60935] Updated weights for policy 0, policy_version 86480 (0.0007) [2023-10-14 00:32:01,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 177766400. Throughput: 0: 1714.3, 1: 1759.2. Samples: 44446876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:32:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:32:01,305][60935] Updated weights for policy 0, policy_version 86490 (0.0007) [2023-10-14 00:32:04,916][60934] Updated weights for policy 1, policy_version 86042 (0.0010) [2023-10-14 00:32:05,277][60934] Updated weights for policy 1, policy_version 86052 (0.0009) [2023-10-14 00:32:05,437][60935] Updated weights for policy 0, policy_version 86500 (0.0010) [2023-10-14 00:32:05,652][60934] Updated weights for policy 1, policy_version 86062 (0.0009) [2023-10-14 00:32:05,808][60935] Updated weights for policy 0, policy_version 86510 (0.0009) [2023-10-14 00:32:06,176][60935] Updated weights for policy 0, policy_version 86520 (0.0010) [2023-10-14 00:32:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 177831936. Throughput: 0: 1712.4, 1: 1752.8. Samples: 44467584. Policy #0 lag: (min: 7.0, avg: 11.1, max: 39.0) [2023-10-14 00:32:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:32:09,477][60934] Updated weights for policy 1, policy_version 86072 (0.0009) [2023-10-14 00:32:09,851][60934] Updated weights for policy 1, policy_version 86082 (0.0007) [2023-10-14 00:32:10,226][60934] Updated weights for policy 1, policy_version 86092 (0.0007) [2023-10-14 00:32:10,271][60935] Updated weights for policy 0, policy_version 86530 (0.0010) [2023-10-14 00:32:10,637][60935] Updated weights for policy 0, policy_version 86540 (0.0010) [2023-10-14 00:32:11,005][60935] Updated weights for policy 0, policy_version 86550 (0.0010) [2023-10-14 00:32:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 177897472. Throughput: 0: 1696.4, 1: 1713.0. Samples: 44486724. Policy #0 lag: (min: 7.0, avg: 11.1, max: 39.0) [2023-10-14 00:32:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:32:11,376][60935] Updated weights for policy 0, policy_version 86560 (0.0011) [2023-10-14 00:32:14,155][60934] Updated weights for policy 1, policy_version 86102 (0.0007) [2023-10-14 00:32:14,523][60934] Updated weights for policy 1, policy_version 86112 (0.0009) [2023-10-14 00:32:14,898][60934] Updated weights for policy 1, policy_version 86122 (0.0008) [2023-10-14 00:32:15,279][60935] Updated weights for policy 0, policy_version 86570 (0.0007) [2023-10-14 00:32:15,653][60935] Updated weights for policy 0, policy_version 86580 (0.0007) [2023-10-14 00:32:16,016][60935] Updated weights for policy 0, policy_version 86590 (0.0009) [2023-10-14 00:32:16,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 177995776. Throughput: 0: 1713.2, 1: 1725.4. Samples: 44498226. Policy #0 lag: (min: 7.0, avg: 11.1, max: 39.0) [2023-10-14 00:32:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:32:18,877][60934] Updated weights for policy 1, policy_version 86132 (0.0008) [2023-10-14 00:32:19,243][60934] Updated weights for policy 1, policy_version 86142 (0.0009) [2023-10-14 00:32:19,606][60934] Updated weights for policy 1, policy_version 86152 (0.0007) [2023-10-14 00:32:20,085][60935] Updated weights for policy 0, policy_version 86600 (0.0008) [2023-10-14 00:32:20,458][60935] Updated weights for policy 0, policy_version 86610 (0.0007) [2023-10-14 00:32:20,822][60935] Updated weights for policy 0, policy_version 86620 (0.0008) [2023-10-14 00:32:21,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 178061312. Throughput: 0: 1708.6, 1: 1710.3. Samples: 44518342. Policy #0 lag: (min: 7.0, avg: 11.1, max: 39.0) [2023-10-14 00:32:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:32:23,556][60934] Updated weights for policy 1, policy_version 86162 (0.0008) [2023-10-14 00:32:23,926][60934] Updated weights for policy 1, policy_version 86172 (0.0008) [2023-10-14 00:32:24,298][60934] Updated weights for policy 1, policy_version 86182 (0.0008) [2023-10-14 00:32:24,661][60934] Updated weights for policy 1, policy_version 86192 (0.0008) [2023-10-14 00:32:24,875][60935] Updated weights for policy 0, policy_version 86630 (0.0008) [2023-10-14 00:32:25,250][60935] Updated weights for policy 0, policy_version 86640 (0.0009) [2023-10-14 00:32:25,616][60935] Updated weights for policy 0, policy_version 86650 (0.0008) [2023-10-14 00:32:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 178126848. Throughput: 0: 1684.5, 1: 1694.2. Samples: 44537792. Policy #0 lag: (min: 7.0, avg: 11.1, max: 39.0) [2023-10-14 00:32:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:32:28,638][60934] Updated weights for policy 1, policy_version 86202 (0.0009) [2023-10-14 00:32:29,006][60934] Updated weights for policy 1, policy_version 86212 (0.0009) [2023-10-14 00:32:29,374][60934] Updated weights for policy 1, policy_version 86222 (0.0008) [2023-10-14 00:32:29,721][60935] Updated weights for policy 0, policy_version 86660 (0.0008) [2023-10-14 00:32:30,117][60935] Updated weights for policy 0, policy_version 86670 (0.0007) [2023-10-14 00:32:30,492][60935] Updated weights for policy 0, policy_version 86680 (0.0008) [2023-10-14 00:32:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 178192384. Throughput: 0: 1710.7, 1: 1720.3. Samples: 44549410. Policy #0 lag: (min: 7.0, avg: 11.1, max: 39.0) [2023-10-14 00:32:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:32:33,240][60934] Updated weights for policy 1, policy_version 86232 (0.0008) [2023-10-14 00:32:33,610][60934] Updated weights for policy 1, policy_version 86242 (0.0008) [2023-10-14 00:32:33,982][60934] Updated weights for policy 1, policy_version 86252 (0.0009) [2023-10-14 00:32:34,511][60935] Updated weights for policy 0, policy_version 86690 (0.0008) [2023-10-14 00:32:34,877][60935] Updated weights for policy 0, policy_version 86700 (0.0010) [2023-10-14 00:32:35,244][60935] Updated weights for policy 0, policy_version 86710 (0.0011) [2023-10-14 00:32:35,612][60935] Updated weights for policy 0, policy_version 86720 (0.0011) [2023-10-14 00:32:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 178257920. Throughput: 0: 1699.0, 1: 1693.4. Samples: 44568920. Policy #0 lag: (min: 7.0, avg: 11.1, max: 39.0) [2023-10-14 00:32:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:32:37,988][60934] Updated weights for policy 1, policy_version 86262 (0.0007) [2023-10-14 00:32:38,348][60934] Updated weights for policy 1, policy_version 86272 (0.0010) [2023-10-14 00:32:38,710][60934] Updated weights for policy 1, policy_version 86282 (0.0008) [2023-10-14 00:32:39,676][60935] Updated weights for policy 0, policy_version 86730 (0.0007) [2023-10-14 00:32:40,049][60935] Updated weights for policy 0, policy_version 86740 (0.0007) [2023-10-14 00:32:40,408][60935] Updated weights for policy 0, policy_version 86750 (0.0009) [2023-10-14 00:32:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 178323456. Throughput: 0: 1683.4, 1: 1710.0. Samples: 44589232. Policy #0 lag: (min: 7.0, avg: 11.1, max: 39.0) [2023-10-14 00:32:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:32:42,721][60934] Updated weights for policy 1, policy_version 86292 (0.0008) [2023-10-14 00:32:43,093][60934] Updated weights for policy 1, policy_version 86302 (0.0007) [2023-10-14 00:32:43,460][60934] Updated weights for policy 1, policy_version 86312 (0.0010) [2023-10-14 00:32:44,242][60935] Updated weights for policy 0, policy_version 86760 (0.0008) [2023-10-14 00:32:44,611][60935] Updated weights for policy 0, policy_version 86770 (0.0008) [2023-10-14 00:32:44,980][60935] Updated weights for policy 0, policy_version 86780 (0.0007) [2023-10-14 00:32:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 178388992. Throughput: 0: 1705.8, 1: 1698.0. Samples: 44600050. Policy #0 lag: (min: 7.0, avg: 11.1, max: 39.0) [2023-10-14 00:32:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:32:47,446][60934] Updated weights for policy 1, policy_version 86322 (0.0008) [2023-10-14 00:32:47,809][60934] Updated weights for policy 1, policy_version 86332 (0.0007) [2023-10-14 00:32:48,178][60934] Updated weights for policy 1, policy_version 86342 (0.0008) [2023-10-14 00:32:48,538][60934] Updated weights for policy 1, policy_version 86352 (0.0009) [2023-10-14 00:32:49,058][60935] Updated weights for policy 0, policy_version 86790 (0.0008) [2023-10-14 00:32:49,433][60935] Updated weights for policy 0, policy_version 86800 (0.0010) [2023-10-14 00:32:49,796][60935] Updated weights for policy 0, policy_version 86810 (0.0010) [2023-10-14 00:32:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 178454528. Throughput: 0: 1682.7, 1: 1697.9. Samples: 44619714. Policy #0 lag: (min: 7.0, avg: 11.1, max: 39.0) [2023-10-14 00:32:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:32:52,588][60934] Updated weights for policy 1, policy_version 86362 (0.0009) [2023-10-14 00:32:52,950][60934] Updated weights for policy 1, policy_version 86372 (0.0008) [2023-10-14 00:32:53,318][60934] Updated weights for policy 1, policy_version 86382 (0.0010) [2023-10-14 00:32:53,812][60935] Updated weights for policy 0, policy_version 86820 (0.0011) [2023-10-14 00:32:54,181][60935] Updated weights for policy 0, policy_version 86830 (0.0009) [2023-10-14 00:32:54,548][60935] Updated weights for policy 0, policy_version 86840 (0.0008) [2023-10-14 00:32:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 178520064. Throughput: 0: 1692.5, 1: 1729.8. Samples: 44640730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:32:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:32:56,260][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000086848_88932352.pth... [2023-10-14 00:32:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000086384_89587712.pth... [2023-10-14 00:32:56,298][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000084816_87949312.pth [2023-10-14 00:32:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000085248_87293952.pth [2023-10-14 00:32:56,302][60828] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p1/milestones/checkpoint_000086384_89587712.pth [2023-10-14 00:32:56,306][60695] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p0/milestones/checkpoint_000086848_88932352.pth [2023-10-14 00:32:57,291][60934] Updated weights for policy 1, policy_version 86392 (0.0010) [2023-10-14 00:32:57,595][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:32:58,554][60935] Updated weights for policy 0, policy_version 86850 (0.0011) [2023-10-14 00:32:58,928][60935] Updated weights for policy 0, policy_version 86860 (0.0010) [2023-10-14 00:32:59,299][60935] Updated weights for policy 0, policy_version 86870 (0.0010) [2023-10-14 00:32:59,662][60935] Updated weights for policy 0, policy_version 86880 (0.0011) [2023-10-14 00:33:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 178585600. Throughput: 0: 1694.0, 1: 1709.7. Samples: 44651396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:33:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:33:01,594][60934] Updated weights for policy 1, policy_version 86402 (0.0009) [2023-10-14 00:33:01,963][60934] Updated weights for policy 1, policy_version 86412 (0.0009) [2023-10-14 00:33:02,326][60934] Updated weights for policy 1, policy_version 86422 (0.0007) [2023-10-14 00:33:02,692][60934] Updated weights for policy 1, policy_version 86432 (0.0008) [2023-10-14 00:33:03,718][60935] Updated weights for policy 0, policy_version 86890 (0.0007) [2023-10-14 00:33:04,093][60935] Updated weights for policy 0, policy_version 86900 (0.0008) [2023-10-14 00:33:04,452][60935] Updated weights for policy 0, policy_version 86910 (0.0007) [2023-10-14 00:33:06,248][59943] Fps is (10 sec: 13107.7, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 178651136. Throughput: 0: 1674.7, 1: 1730.5. Samples: 44671574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:33:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:33:06,783][60934] Updated weights for policy 1, policy_version 86442 (0.0009) [2023-10-14 00:33:07,142][60934] Updated weights for policy 1, policy_version 86452 (0.0010) [2023-10-14 00:33:07,506][60934] Updated weights for policy 1, policy_version 86462 (0.0008) [2023-10-14 00:33:08,354][60935] Updated weights for policy 0, policy_version 86920 (0.0008) [2023-10-14 00:33:08,727][60935] Updated weights for policy 0, policy_version 86930 (0.0008) [2023-10-14 00:33:09,098][60935] Updated weights for policy 0, policy_version 86940 (0.0007) [2023-10-14 00:33:11,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 178716672. Throughput: 0: 1704.5, 1: 1739.0. Samples: 44692746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:33:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:33:11,472][60934] Updated weights for policy 1, policy_version 86472 (0.0008) [2023-10-14 00:33:11,837][60934] Updated weights for policy 1, policy_version 86482 (0.0008) [2023-10-14 00:33:12,201][60934] Updated weights for policy 1, policy_version 86492 (0.0009) [2023-10-14 00:33:12,987][60935] Updated weights for policy 0, policy_version 86950 (0.0009) [2023-10-14 00:33:13,355][60935] Updated weights for policy 0, policy_version 86960 (0.0011) [2023-10-14 00:33:13,722][60935] Updated weights for policy 0, policy_version 86970 (0.0011) [2023-10-14 00:33:16,053][60934] Updated weights for policy 1, policy_version 86502 (0.0008) [2023-10-14 00:33:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 178782208. Throughput: 0: 1679.4, 1: 1717.2. Samples: 44702254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:33:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:33:16,415][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:33:16,419][60934] Updated weights for policy 1, policy_version 86512 (0.0007) [2023-10-14 00:33:17,815][60935] Updated weights for policy 0, policy_version 86980 (0.0008) [2023-10-14 00:33:18,203][60935] Updated weights for policy 0, policy_version 86990 (0.0007) [2023-10-14 00:33:18,571][60935] Updated weights for policy 0, policy_version 87000 (0.0010) [2023-10-14 00:33:20,668][60934] Updated weights for policy 1, policy_version 86522 (0.0007) [2023-10-14 00:33:20,879][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:33:21,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 178880512. Throughput: 0: 1685.4, 1: 1756.1. Samples: 44723788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:33:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:33:22,739][60935] Updated weights for policy 0, policy_version 87010 (0.0010) [2023-10-14 00:33:23,107][60935] Updated weights for policy 0, policy_version 87020 (0.0009) [2023-10-14 00:33:23,483][60935] Updated weights for policy 0, policy_version 87030 (0.0008) [2023-10-14 00:33:23,852][60935] Updated weights for policy 0, policy_version 87040 (0.0007) [2023-10-14 00:33:24,954][60934] Updated weights for policy 1, policy_version 86532 (0.0009) [2023-10-14 00:33:25,314][60934] Updated weights for policy 1, policy_version 86542 (0.0007) [2023-10-14 00:33:25,682][60934] Updated weights for policy 1, policy_version 86552 (0.0008) [2023-10-14 00:33:26,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 178946048. Throughput: 0: 1706.0, 1: 1748.1. Samples: 44744666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:33:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:33:27,857][60935] Updated weights for policy 0, policy_version 87050 (0.0008) [2023-10-14 00:33:28,232][60935] Updated weights for policy 0, policy_version 87060 (0.0008) [2023-10-14 00:33:28,607][60935] Updated weights for policy 0, policy_version 87070 (0.0007) [2023-10-14 00:33:29,606][60934] Updated weights for policy 1, policy_version 86562 (0.0011) [2023-10-14 00:33:29,976][60934] Updated weights for policy 1, policy_version 86572 (0.0010) [2023-10-14 00:33:30,342][60934] Updated weights for policy 1, policy_version 86582 (0.0010) [2023-10-14 00:33:30,710][60934] Updated weights for policy 1, policy_version 86592 (0.0009) [2023-10-14 00:33:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 179011584. Throughput: 0: 1675.0, 1: 1764.6. Samples: 44754832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:33:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:33:32,536][60935] Updated weights for policy 0, policy_version 87080 (0.0010) [2023-10-14 00:33:32,909][60935] Updated weights for policy 0, policy_version 87090 (0.0009) [2023-10-14 00:33:33,270][60935] Updated weights for policy 0, policy_version 87100 (0.0007) [2023-10-14 00:33:34,685][60934] Updated weights for policy 1, policy_version 86602 (0.0007) [2023-10-14 00:33:35,054][60934] Updated weights for policy 1, policy_version 86612 (0.0008) [2023-10-14 00:33:35,427][60934] Updated weights for policy 1, policy_version 86622 (0.0008) [2023-10-14 00:33:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 179077120. Throughput: 0: 1700.8, 1: 1763.7. Samples: 44775616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:33:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:33:37,202][60935] Updated weights for policy 0, policy_version 87110 (0.0009) [2023-10-14 00:33:37,572][60935] Updated weights for policy 0, policy_version 87120 (0.0008) [2023-10-14 00:33:37,933][60935] Updated weights for policy 0, policy_version 87130 (0.0008) [2023-10-14 00:33:39,443][60934] Updated weights for policy 1, policy_version 86632 (0.0008) [2023-10-14 00:33:39,808][60934] Updated weights for policy 1, policy_version 86642 (0.0009) [2023-10-14 00:33:40,179][60934] Updated weights for policy 1, policy_version 86652 (0.0008) [2023-10-14 00:33:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 179142656. Throughput: 0: 1706.2, 1: 1736.7. Samples: 44795662. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:33:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:33:41,779][60935] Updated weights for policy 0, policy_version 87140 (0.0009) [2023-10-14 00:33:42,152][60935] Updated weights for policy 0, policy_version 87150 (0.0010) [2023-10-14 00:33:42,521][60935] Updated weights for policy 0, policy_version 87160 (0.0012) [2023-10-14 00:33:44,283][60934] Updated weights for policy 1, policy_version 86662 (0.0008) [2023-10-14 00:33:44,669][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:33:44,675][60934] Updated weights for policy 1, policy_version 86672 (0.0008) [2023-10-14 00:33:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 179208192. Throughput: 0: 1686.5, 1: 1756.8. Samples: 44806346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:33:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:33:46,510][60935] Updated weights for policy 0, policy_version 87170 (0.0010) [2023-10-14 00:33:46,879][60935] Updated weights for policy 0, policy_version 87180 (0.0008) [2023-10-14 00:33:47,238][60935] Updated weights for policy 0, policy_version 87190 (0.0009) [2023-10-14 00:33:47,603][60935] Updated weights for policy 0, policy_version 87200 (0.0010) [2023-10-14 00:33:48,762][60934] Updated weights for policy 1, policy_version 86682 (0.0009) [2023-10-14 00:33:48,979][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:33:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 179273728. Throughput: 0: 1715.3, 1: 1747.5. Samples: 44827400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:33:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:33:51,566][60935] Updated weights for policy 0, policy_version 87210 (0.0010) [2023-10-14 00:33:51,941][60935] Updated weights for policy 0, policy_version 87220 (0.0011) [2023-10-14 00:33:52,302][60935] Updated weights for policy 0, policy_version 87230 (0.0010) [2023-10-14 00:33:52,956][60934] Updated weights for policy 1, policy_version 86692 (0.0009) [2023-10-14 00:33:53,326][60934] Updated weights for policy 1, policy_version 86702 (0.0009) [2023-10-14 00:33:53,393][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:33:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 179339264. Throughput: 0: 1708.8, 1: 1764.0. Samples: 44849020. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:33:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:33:56,397][60935] Updated weights for policy 0, policy_version 87240 (0.0009) [2023-10-14 00:33:56,763][60935] Updated weights for policy 0, policy_version 87250 (0.0010) [2023-10-14 00:33:57,131][60935] Updated weights for policy 0, policy_version 87260 (0.0010) [2023-10-14 00:33:57,588][60934] Updated weights for policy 1, policy_version 86712 (0.0010) [2023-10-14 00:33:57,943][60934] Updated weights for policy 1, policy_version 86722 (0.0010) [2023-10-14 00:33:58,309][60934] Updated weights for policy 1, policy_version 86732 (0.0009) [2023-10-14 00:34:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 179404800. Throughput: 0: 1705.8, 1: 1761.4. Samples: 44858280. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:34:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:34:01,250][60935] Updated weights for policy 0, policy_version 87270 (0.0008) [2023-10-14 00:34:01,623][60935] Updated weights for policy 0, policy_version 87280 (0.0007) [2023-10-14 00:34:01,980][60935] Updated weights for policy 0, policy_version 87290 (0.0007) [2023-10-14 00:34:02,475][60934] Updated weights for policy 1, policy_version 86742 (0.0008) [2023-10-14 00:34:02,829][60934] Updated weights for policy 1, policy_version 86752 (0.0009) [2023-10-14 00:34:03,201][60934] Updated weights for policy 1, policy_version 86762 (0.0007) [2023-10-14 00:34:05,909][60935] Updated weights for policy 0, policy_version 87300 (0.0008) [2023-10-14 00:34:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 179470336. Throughput: 0: 1712.4, 1: 1745.7. Samples: 44879400. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:34:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:34:06,281][60935] Updated weights for policy 0, policy_version 87310 (0.0007) [2023-10-14 00:34:06,645][60935] Updated weights for policy 0, policy_version 87320 (0.0009) [2023-10-14 00:34:07,047][60934] Updated weights for policy 1, policy_version 86772 (0.0009) [2023-10-14 00:34:07,415][60934] Updated weights for policy 1, policy_version 86782 (0.0010) [2023-10-14 00:34:07,475][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000007 [2023-10-14 00:34:10,735][60935] Updated weights for policy 0, policy_version 87330 (0.0010) [2023-10-14 00:34:11,096][60935] Updated weights for policy 0, policy_version 87340 (0.0008) [2023-10-14 00:34:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 179535872. Throughput: 0: 1705.5, 1: 1765.2. Samples: 44900846. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:34:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:34:11,468][60935] Updated weights for policy 0, policy_version 87350 (0.0009) [2023-10-14 00:34:11,693][60934] Updated weights for policy 1, policy_version 86792 (0.0008) [2023-10-14 00:34:11,828][60935] Updated weights for policy 0, policy_version 87360 (0.0007) [2023-10-14 00:34:11,984][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-14 00:34:15,656][60935] Updated weights for policy 0, policy_version 87370 (0.0009) [2023-10-14 00:34:15,972][60934] Updated weights for policy 1, policy_version 86802 (0.0009) [2023-10-14 00:34:16,023][60935] Updated weights for policy 0, policy_version 87380 (0.0009) [2023-10-14 00:34:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 179601408. Throughput: 0: 1713.8, 1: 1757.6. Samples: 44911044. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:34:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:34:16,342][60934] Updated weights for policy 1, policy_version 86812 (0.0008) [2023-10-14 00:34:16,394][60935] Updated weights for policy 0, policy_version 87390 (0.0008) [2023-10-14 00:34:16,487][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:34:20,303][60935] Updated weights for policy 0, policy_version 87400 (0.0008) [2023-10-14 00:34:20,561][60934] Updated weights for policy 1, policy_version 86822 (0.0007) [2023-10-14 00:34:20,667][60935] Updated weights for policy 0, policy_version 87410 (0.0007) [2023-10-14 00:34:20,923][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:34:20,925][60934] Updated weights for policy 1, policy_version 86832 (0.0008) [2023-10-14 00:34:21,030][60935] Updated weights for policy 0, policy_version 87420 (0.0008) [2023-10-14 00:34:21,248][59943] Fps is (10 sec: 19660.4, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 179732480. Throughput: 0: 1718.1, 1: 1778.9. Samples: 44932982. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:34:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:34:25,094][60934] Updated weights for policy 1, policy_version 86842 (0.0008) [2023-10-14 00:34:25,144][60935] Updated weights for policy 0, policy_version 87430 (0.0008) [2023-10-14 00:34:25,301][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:34:25,516][60935] Updated weights for policy 0, policy_version 87440 (0.0007) [2023-10-14 00:34:25,885][60935] Updated weights for policy 0, policy_version 87450 (0.0009) [2023-10-14 00:34:26,248][59943] Fps is (10 sec: 19661.1, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 179798016. Throughput: 0: 1694.5, 1: 1804.4. Samples: 44953116. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:34:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:34:29,369][60934] Updated weights for policy 1, policy_version 86852 (0.0009) [2023-10-14 00:34:29,776][60934] Updated weights for policy 1, policy_version 86862 (0.0008) [2023-10-14 00:34:29,844][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:34:29,873][60935] Updated weights for policy 0, policy_version 87460 (0.0008) [2023-10-14 00:34:30,244][60935] Updated weights for policy 0, policy_version 87470 (0.0010) [2023-10-14 00:34:30,605][60935] Updated weights for policy 0, policy_version 87480 (0.0010) [2023-10-14 00:34:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 179863552. Throughput: 0: 1715.4, 1: 1806.1. Samples: 44964814. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:34:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:34:34,003][60934] Updated weights for policy 1, policy_version 86872 (0.0008) [2023-10-14 00:34:34,364][60934] Updated weights for policy 1, policy_version 86882 (0.0007) [2023-10-14 00:34:34,703][60935] Updated weights for policy 0, policy_version 87490 (0.0009) [2023-10-14 00:34:34,725][60934] Updated weights for policy 1, policy_version 86892 (0.0008) [2023-10-14 00:34:35,077][60935] Updated weights for policy 0, policy_version 87500 (0.0009) [2023-10-14 00:34:35,448][60935] Updated weights for policy 0, policy_version 87510 (0.0010) [2023-10-14 00:34:35,815][60935] Updated weights for policy 0, policy_version 87520 (0.0008) [2023-10-14 00:34:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 179929088. Throughput: 0: 1702.7, 1: 1801.7. Samples: 44985096. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:34:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:34:38,798][60934] Updated weights for policy 1, policy_version 86902 (0.0007) [2023-10-14 00:34:39,174][60934] Updated weights for policy 1, policy_version 86912 (0.0008) [2023-10-14 00:34:39,550][60934] Updated weights for policy 1, policy_version 86922 (0.0009) [2023-10-14 00:34:39,833][60935] Updated weights for policy 0, policy_version 87530 (0.0008) [2023-10-14 00:34:40,197][60935] Updated weights for policy 0, policy_version 87540 (0.0009) [2023-10-14 00:34:40,559][60935] Updated weights for policy 0, policy_version 87550 (0.0007) [2023-10-14 00:34:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 179994624. Throughput: 0: 1682.0, 1: 1777.3. Samples: 45004690. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:34:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:34:43,470][60934] Updated weights for policy 1, policy_version 86932 (0.0007) [2023-10-14 00:34:43,840][60934] Updated weights for policy 1, policy_version 86942 (0.0007) [2023-10-14 00:34:43,906][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:34:44,716][60935] Updated weights for policy 0, policy_version 87560 (0.0008) [2023-10-14 00:34:45,090][60935] Updated weights for policy 0, policy_version 87570 (0.0007) [2023-10-14 00:34:45,451][60935] Updated weights for policy 0, policy_version 87580 (0.0008) [2023-10-14 00:34:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13884.7). Total num frames: 180060160. Throughput: 0: 1708.4, 1: 1799.1. Samples: 45016120. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:34:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:34:48,048][60934] Updated weights for policy 1, policy_version 86952 (0.0009) [2023-10-14 00:34:48,329][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:34:49,494][60935] Updated weights for policy 0, policy_version 87590 (0.0009) [2023-10-14 00:34:49,871][60935] Updated weights for policy 0, policy_version 87600 (0.0009) [2023-10-14 00:34:50,233][60935] Updated weights for policy 0, policy_version 87610 (0.0010) [2023-10-14 00:34:51,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13884.7). Total num frames: 180125696. Throughput: 0: 1694.2, 1: 1811.3. Samples: 45037150. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-14 00:34:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:34:52,129][60934] Updated weights for policy 1, policy_version 86962 (0.0010) [2023-10-14 00:34:52,495][60934] Updated weights for policy 1, policy_version 86972 (0.0007) [2023-10-14 00:34:52,632][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:34:54,229][60935] Updated weights for policy 0, policy_version 87620 (0.0008) [2023-10-14 00:34:54,615][60935] Updated weights for policy 0, policy_version 87630 (0.0008) [2023-10-14 00:34:54,993][60935] Updated weights for policy 0, policy_version 87640 (0.0009) [2023-10-14 00:34:56,249][59943] Fps is (10 sec: 13106.7, 60 sec: 14199.3, 300 sec: 13884.7). Total num frames: 180191232. Throughput: 0: 1681.1, 1: 1815.4. Samples: 45058192. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-14 00:34:56,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:34:56,264][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000086976_90439680.pth... [2023-10-14 00:34:56,264][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000087648_89751552.pth... [2023-10-14 00:34:56,304][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000085616_88768512.pth [2023-10-14 00:34:56,309][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000086048_88113152.pth [2023-10-14 00:34:56,928][60934] Updated weights for policy 1, policy_version 86982 (0.0009) [2023-10-14 00:34:57,284][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:34:57,290][60934] Updated weights for policy 1, policy_version 86992 (0.0009) [2023-10-14 00:34:59,189][60935] Updated weights for policy 0, policy_version 87650 (0.0008) [2023-10-14 00:34:59,554][60935] Updated weights for policy 0, policy_version 87660 (0.0010) [2023-10-14 00:34:59,936][60935] Updated weights for policy 0, policy_version 87670 (0.0008) [2023-10-14 00:35:00,302][60935] Updated weights for policy 0, policy_version 87680 (0.0008) [2023-10-14 00:35:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 180256768. Throughput: 0: 1699.6, 1: 1813.0. Samples: 45069108. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-14 00:35:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:35:01,530][60934] Updated weights for policy 1, policy_version 87002 (0.0009) [2023-10-14 00:35:01,893][60934] Updated weights for policy 1, policy_version 87012 (0.0009) [2023-10-14 00:35:02,264][60934] Updated weights for policy 1, policy_version 87022 (0.0007) [2023-10-14 00:35:04,299][60935] Updated weights for policy 0, policy_version 87690 (0.0011) [2023-10-14 00:35:04,666][60935] Updated weights for policy 0, policy_version 87700 (0.0009) [2023-10-14 00:35:05,030][60935] Updated weights for policy 0, policy_version 87710 (0.0011) [2023-10-14 00:35:06,128][60934] Updated weights for policy 1, policy_version 87032 (0.0009) [2023-10-14 00:35:06,248][59943] Fps is (10 sec: 13108.0, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 180322304. Throughput: 0: 1673.8, 1: 1801.0. Samples: 45089348. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-14 00:35:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:35:06,495][60934] Updated weights for policy 1, policy_version 87042 (0.0010) [2023-10-14 00:35:06,861][60934] Updated weights for policy 1, policy_version 87052 (0.0009) [2023-10-14 00:35:08,981][60935] Updated weights for policy 0, policy_version 87720 (0.0009) [2023-10-14 00:35:09,352][60935] Updated weights for policy 0, policy_version 87730 (0.0008) [2023-10-14 00:35:09,724][60935] Updated weights for policy 0, policy_version 87740 (0.0007) [2023-10-14 00:35:10,864][60934] Updated weights for policy 1, policy_version 87062 (0.0008) [2023-10-14 00:35:11,232][60934] Updated weights for policy 1, policy_version 87072 (0.0007) [2023-10-14 00:35:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 180387840. Throughput: 0: 1691.2, 1: 1800.4. Samples: 45110242. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-14 00:35:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:35:11,594][60934] Updated weights for policy 1, policy_version 87082 (0.0010) [2023-10-14 00:35:13,698][60935] Updated weights for policy 0, policy_version 87750 (0.0007) [2023-10-14 00:35:14,073][60935] Updated weights for policy 0, policy_version 87760 (0.0009) [2023-10-14 00:35:14,438][60935] Updated weights for policy 0, policy_version 87770 (0.0010) [2023-10-14 00:35:15,476][60934] Updated weights for policy 1, policy_version 87092 (0.0009) [2023-10-14 00:35:15,875][60934] Updated weights for policy 1, policy_version 87102 (0.0011) [2023-10-14 00:35:15,940][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:35:16,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 13773.7). Total num frames: 180486144. Throughput: 0: 1691.8, 1: 1770.7. Samples: 45120626. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-14 00:35:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:35:18,650][60935] Updated weights for policy 0, policy_version 87780 (0.0010) [2023-10-14 00:35:19,002][60935] Updated weights for policy 0, policy_version 87790 (0.0010) [2023-10-14 00:35:19,365][60935] Updated weights for policy 0, policy_version 87800 (0.0007) [2023-10-14 00:35:20,192][60934] Updated weights for policy 1, policy_version 87112 (0.0008) [2023-10-14 00:35:20,479][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:35:21,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 180551680. Throughput: 0: 1672.1, 1: 1796.4. Samples: 45141182. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-14 00:35:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:35:23,440][60935] Updated weights for policy 0, policy_version 87810 (0.0008) [2023-10-14 00:35:23,803][60935] Updated weights for policy 0, policy_version 87820 (0.0008) [2023-10-14 00:35:24,173][60935] Updated weights for policy 0, policy_version 87830 (0.0008) [2023-10-14 00:35:24,442][60934] Updated weights for policy 1, policy_version 87122 (0.0009) [2023-10-14 00:35:24,541][60935] Updated weights for policy 0, policy_version 87840 (0.0009) [2023-10-14 00:35:24,808][60934] Updated weights for policy 1, policy_version 87132 (0.0008) [2023-10-14 00:35:24,953][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:35:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 180617216. Throughput: 0: 1695.7, 1: 1803.5. Samples: 45162154. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-14 00:35:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:35:28,483][60935] Updated weights for policy 0, policy_version 87850 (0.0008) [2023-10-14 00:35:28,850][60935] Updated weights for policy 0, policy_version 87860 (0.0008) [2023-10-14 00:35:29,112][60934] Updated weights for policy 1, policy_version 87142 (0.0007) [2023-10-14 00:35:29,225][60935] Updated weights for policy 0, policy_version 87870 (0.0008) [2023-10-14 00:35:29,476][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:35:29,477][60934] Updated weights for policy 1, policy_version 87152 (0.0007) [2023-10-14 00:35:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 180682752. Throughput: 0: 1680.9, 1: 1807.2. Samples: 45173082. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-14 00:35:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:35:33,224][60935] Updated weights for policy 0, policy_version 87880 (0.0010) [2023-10-14 00:35:33,593][60934] Updated weights for policy 1, policy_version 87162 (0.0010) [2023-10-14 00:35:33,594][60935] Updated weights for policy 0, policy_version 87890 (0.0009) [2023-10-14 00:35:33,811][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:35:33,962][60935] Updated weights for policy 0, policy_version 87900 (0.0010) [2023-10-14 00:35:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 180748288. Throughput: 0: 1675.1, 1: 1797.0. Samples: 45193396. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-14 00:35:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:35:37,982][60934] Updated weights for policy 1, policy_version 87172 (0.0008) [2023-10-14 00:35:38,137][60935] Updated weights for policy 0, policy_version 87910 (0.0011) [2023-10-14 00:35:38,340][60934] Updated weights for policy 1, policy_version 87182 (0.0010) [2023-10-14 00:35:38,498][60935] Updated weights for policy 0, policy_version 87920 (0.0009) [2023-10-14 00:35:38,712][60934] Updated weights for policy 1, policy_version 87192 (0.0009) [2023-10-14 00:35:38,861][60935] Updated weights for policy 0, policy_version 87930 (0.0010) [2023-10-14 00:35:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 180813824. Throughput: 0: 1691.2, 1: 1776.0. Samples: 45214214. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-14 00:35:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:35:42,741][60934] Updated weights for policy 1, policy_version 87202 (0.0007) [2023-10-14 00:35:43,033][60935] Updated weights for policy 0, policy_version 87940 (0.0008) [2023-10-14 00:35:43,113][60934] Updated weights for policy 1, policy_version 87212 (0.0007) [2023-10-14 00:35:43,428][60935] Updated weights for policy 0, policy_version 87950 (0.0007) [2023-10-14 00:35:43,468][60934] Updated weights for policy 1, policy_version 87222 (0.0007) [2023-10-14 00:35:43,798][60935] Updated weights for policy 0, policy_version 87960 (0.0008) [2023-10-14 00:35:43,837][60934] Updated weights for policy 1, policy_version 87232 (0.0008) [2023-10-14 00:35:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 180879360. Throughput: 0: 1669.9, 1: 1772.4. Samples: 45224014. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-14 00:35:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:35:47,795][60935] Updated weights for policy 0, policy_version 87970 (0.0009) [2023-10-14 00:35:47,816][60934] Updated weights for policy 1, policy_version 87242 (0.0009) [2023-10-14 00:35:48,154][60935] Updated weights for policy 0, policy_version 87980 (0.0010) [2023-10-14 00:35:48,180][60934] Updated weights for policy 1, policy_version 87252 (0.0008) [2023-10-14 00:35:48,532][60935] Updated weights for policy 0, policy_version 87990 (0.0008) [2023-10-14 00:35:48,546][60934] Updated weights for policy 1, policy_version 87262 (0.0007) [2023-10-14 00:35:48,894][60935] Updated weights for policy 0, policy_version 88000 (0.0009) [2023-10-14 00:35:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 180944896. Throughput: 0: 1684.4, 1: 1760.1. Samples: 45244348. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 00:35:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:35:52,567][60934] Updated weights for policy 1, policy_version 87272 (0.0008) [2023-10-14 00:35:52,860][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000002 [2023-10-14 00:35:52,906][60935] Updated weights for policy 0, policy_version 88010 (0.0008) [2023-10-14 00:35:53,270][60935] Updated weights for policy 0, policy_version 88020 (0.0008) [2023-10-14 00:35:53,637][60935] Updated weights for policy 0, policy_version 88030 (0.0007) [2023-10-14 00:35:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 181010432. Throughput: 0: 1689.3, 1: 1770.3. Samples: 45265924. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 00:35:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:35:56,843][60934] Updated weights for policy 1, policy_version 87282 (0.0007) [2023-10-14 00:35:57,202][60934] Updated weights for policy 1, policy_version 87292 (0.0008) [2023-10-14 00:35:57,567][60934] Updated weights for policy 1, policy_version 87302 (0.0008) [2023-10-14 00:35:57,749][60935] Updated weights for policy 0, policy_version 88040 (0.0009) [2023-10-14 00:35:57,934][60934] Updated weights for policy 1, policy_version 87312 (0.0009) [2023-10-14 00:35:58,120][60935] Updated weights for policy 0, policy_version 88050 (0.0011) [2023-10-14 00:35:58,488][60935] Updated weights for policy 0, policy_version 88060 (0.0008) [2023-10-14 00:36:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 181075968. Throughput: 0: 1665.6, 1: 1770.1. Samples: 45275230. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 00:36:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:36:02,035][60934] Updated weights for policy 1, policy_version 87322 (0.0009) [2023-10-14 00:36:02,248][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:36:02,466][60935] Updated weights for policy 0, policy_version 88070 (0.0007) [2023-10-14 00:36:02,835][60935] Updated weights for policy 0, policy_version 88080 (0.0009) [2023-10-14 00:36:03,203][60935] Updated weights for policy 0, policy_version 88090 (0.0007) [2023-10-14 00:36:06,208][60934] Updated weights for policy 1, policy_version 87332 (0.0010) [2023-10-14 00:36:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 181141504. Throughput: 0: 1690.2, 1: 1767.0. Samples: 45296758. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 00:36:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:36:06,577][60934] Updated weights for policy 1, policy_version 87342 (0.0009) [2023-10-14 00:36:06,640][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:36:07,011][60935] Updated weights for policy 0, policy_version 88100 (0.0008) [2023-10-14 00:36:07,374][60935] Updated weights for policy 0, policy_version 88110 (0.0009) [2023-10-14 00:36:07,741][60935] Updated weights for policy 0, policy_version 88120 (0.0008) [2023-10-14 00:36:10,865][60934] Updated weights for policy 1, policy_version 87352 (0.0008) [2023-10-14 00:36:11,161][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:36:11,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 13884.7). Total num frames: 181239808. Throughput: 0: 1689.6, 1: 1781.2. Samples: 45318340. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 00:36:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:36:11,815][60935] Updated weights for policy 0, policy_version 88130 (0.0010) [2023-10-14 00:36:12,187][60935] Updated weights for policy 0, policy_version 88140 (0.0008) [2023-10-14 00:36:12,546][60935] Updated weights for policy 0, policy_version 88150 (0.0008) [2023-10-14 00:36:12,924][60935] Updated weights for policy 0, policy_version 88160 (0.0010) [2023-10-14 00:36:15,097][60934] Updated weights for policy 1, policy_version 87362 (0.0009) [2023-10-14 00:36:15,465][60934] Updated weights for policy 1, policy_version 87372 (0.0008) [2023-10-14 00:36:15,820][60934] Updated weights for policy 1, policy_version 87382 (0.0009) [2023-10-14 00:36:16,181][60934] Updated weights for policy 1, policy_version 87392 (0.0009) [2023-10-14 00:36:16,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 181305344. Throughput: 0: 1679.5, 1: 1765.2. Samples: 45328096. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 00:36:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:36:16,915][60935] Updated weights for policy 0, policy_version 88170 (0.0007) [2023-10-14 00:36:17,276][60935] Updated weights for policy 0, policy_version 88180 (0.0007) [2023-10-14 00:36:17,647][60935] Updated weights for policy 0, policy_version 88190 (0.0007) [2023-10-14 00:36:20,224][60934] Updated weights for policy 1, policy_version 87402 (0.0009) [2023-10-14 00:36:20,587][60934] Updated weights for policy 1, policy_version 87412 (0.0008) [2023-10-14 00:36:20,957][60934] Updated weights for policy 1, policy_version 87422 (0.0007) [2023-10-14 00:36:21,248][59943] Fps is (10 sec: 13107.7, 60 sec: 13653.4, 300 sec: 13884.8). Total num frames: 181370880. Throughput: 0: 1698.9, 1: 1766.2. Samples: 45349328. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 00:36:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:36:21,629][60935] Updated weights for policy 0, policy_version 88200 (0.0008) [2023-10-14 00:36:21,991][60935] Updated weights for policy 0, policy_version 88210 (0.0008) [2023-10-14 00:36:22,352][60935] Updated weights for policy 0, policy_version 88220 (0.0009) [2023-10-14 00:36:24,934][60934] Updated weights for policy 1, policy_version 87432 (0.0007) [2023-10-14 00:36:25,297][60934] Updated weights for policy 1, policy_version 87442 (0.0009) [2023-10-14 00:36:25,671][60934] Updated weights for policy 1, policy_version 87452 (0.0007) [2023-10-14 00:36:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 181436416. Throughput: 0: 1701.9, 1: 1748.9. Samples: 45369498. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 00:36:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:36:26,405][60935] Updated weights for policy 0, policy_version 88230 (0.0009) [2023-10-14 00:36:26,770][60935] Updated weights for policy 0, policy_version 88240 (0.0009) [2023-10-14 00:36:27,146][60935] Updated weights for policy 0, policy_version 88250 (0.0007) [2023-10-14 00:36:29,575][60934] Updated weights for policy 1, policy_version 87462 (0.0008) [2023-10-14 00:36:29,928][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:36:29,932][60934] Updated weights for policy 1, policy_version 87472 (0.0009) [2023-10-14 00:36:31,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 181501952. Throughput: 0: 1693.5, 1: 1764.9. Samples: 45379640. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 00:36:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:36:31,284][60935] Updated weights for policy 0, policy_version 88260 (0.0011) [2023-10-14 00:36:31,651][60935] Updated weights for policy 0, policy_version 88270 (0.0011) [2023-10-14 00:36:32,022][60935] Updated weights for policy 0, policy_version 88280 (0.0011) [2023-10-14 00:36:34,045][60934] Updated weights for policy 1, policy_version 87482 (0.0009) [2023-10-14 00:36:34,414][60934] Updated weights for policy 1, policy_version 87492 (0.0010) [2023-10-14 00:36:34,782][60934] Updated weights for policy 1, policy_version 87502 (0.0007) [2023-10-14 00:36:36,091][60935] Updated weights for policy 0, policy_version 88290 (0.0008) [2023-10-14 00:36:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 181567488. Throughput: 0: 1699.2, 1: 1767.4. Samples: 45400344. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 00:36:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:36:36,455][60935] Updated weights for policy 0, policy_version 88300 (0.0011) [2023-10-14 00:36:36,819][60935] Updated weights for policy 0, policy_version 88310 (0.0009) [2023-10-14 00:36:37,191][60935] Updated weights for policy 0, policy_version 88320 (0.0008) [2023-10-14 00:36:38,747][60934] Updated weights for policy 1, policy_version 87512 (0.0008) [2023-10-14 00:36:39,113][60934] Updated weights for policy 1, policy_version 87522 (0.0008) [2023-10-14 00:36:39,490][60934] Updated weights for policy 1, policy_version 87532 (0.0007) [2023-10-14 00:36:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 181633024. Throughput: 0: 1695.6, 1: 1756.3. Samples: 45421256. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 00:36:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:36:41,316][60935] Updated weights for policy 0, policy_version 88330 (0.0007) [2023-10-14 00:36:41,686][60935] Updated weights for policy 0, policy_version 88340 (0.0007) [2023-10-14 00:36:42,057][60935] Updated weights for policy 0, policy_version 88350 (0.0010) [2023-10-14 00:36:43,184][60934] Updated weights for policy 1, policy_version 87542 (0.0008) [2023-10-14 00:36:43,560][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-14 00:36:43,566][60934] Updated weights for policy 1, policy_version 87552 (0.0010) [2023-10-14 00:36:46,147][60935] Updated weights for policy 0, policy_version 88360 (0.0010) [2023-10-14 00:36:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 181698560. Throughput: 0: 1697.5, 1: 1771.6. Samples: 45431340. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 00:36:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:36:46,523][60935] Updated weights for policy 0, policy_version 88370 (0.0009) [2023-10-14 00:36:46,886][60935] Updated weights for policy 0, policy_version 88380 (0.0009) [2023-10-14 00:36:47,938][60934] Updated weights for policy 1, policy_version 87562 (0.0007) [2023-10-14 00:36:48,163][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-14 00:36:50,872][60935] Updated weights for policy 0, policy_version 88390 (0.0008) [2023-10-14 00:36:51,231][60935] Updated weights for policy 0, policy_version 88400 (0.0009) [2023-10-14 00:36:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 181764096. Throughput: 0: 1692.3, 1: 1770.8. Samples: 45452594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:36:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:36:51,601][60935] Updated weights for policy 0, policy_version 88410 (0.0007) [2023-10-14 00:36:52,052][60934] Updated weights for policy 1, policy_version 87572 (0.0008) [2023-10-14 00:36:52,403][60934] Updated weights for policy 1, policy_version 87582 (0.0009) [2023-10-14 00:36:52,770][60934] Updated weights for policy 1, policy_version 87592 (0.0007) [2023-10-14 00:36:55,657][60935] Updated weights for policy 0, policy_version 88420 (0.0008) [2023-10-14 00:36:56,020][60935] Updated weights for policy 0, policy_version 88430 (0.0008) [2023-10-14 00:36:56,248][59943] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 181829632. Throughput: 0: 1682.6, 1: 1765.0. Samples: 45473482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:36:56,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:36:56,262][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000087600_91291648.pth... [2023-10-14 00:36:56,295][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000086384_89587712.pth [2023-10-14 00:36:56,393][60935] Updated weights for policy 0, policy_version 88440 (0.0011) [2023-10-14 00:36:56,687][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000088448_90570752.pth... [2023-10-14 00:36:56,719][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000086848_88932352.pth [2023-10-14 00:36:56,748][60934] Updated weights for policy 1, policy_version 87602 (0.0008) [2023-10-14 00:36:57,107][60934] Updated weights for policy 1, policy_version 87612 (0.0010) [2023-10-14 00:36:57,248][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:37:00,357][60935] Updated weights for policy 0, policy_version 88450 (0.0009) [2023-10-14 00:37:00,727][60935] Updated weights for policy 0, policy_version 88460 (0.0010) [2023-10-14 00:37:01,093][60935] Updated weights for policy 0, policy_version 88470 (0.0008) [2023-10-14 00:37:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 181895168. Throughput: 0: 1691.6, 1: 1770.4. Samples: 45483882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:37:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:37:01,424][60934] Updated weights for policy 1, policy_version 87622 (0.0008) [2023-10-14 00:37:01,456][60935] Updated weights for policy 0, policy_version 88480 (0.0009) [2023-10-14 00:37:01,782][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:37:01,784][60934] Updated weights for policy 1, policy_version 87632 (0.0009) [2023-10-14 00:37:05,493][60935] Updated weights for policy 0, policy_version 88490 (0.0009) [2023-10-14 00:37:05,862][60935] Updated weights for policy 0, policy_version 88500 (0.0007) [2023-10-14 00:37:06,041][60934] Updated weights for policy 1, policy_version 87642 (0.0007) [2023-10-14 00:37:06,226][60935] Updated weights for policy 0, policy_version 88510 (0.0009) [2023-10-14 00:37:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 181960704. Throughput: 0: 1696.9, 1: 1778.9. Samples: 45505740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:37:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:37:06,403][60934] Updated weights for policy 1, policy_version 87652 (0.0008) [2023-10-14 00:37:06,766][60934] Updated weights for policy 1, policy_version 87662 (0.0010) [2023-10-14 00:37:10,296][60935] Updated weights for policy 0, policy_version 88520 (0.0010) [2023-10-14 00:37:10,657][60935] Updated weights for policy 0, policy_version 88530 (0.0008) [2023-10-14 00:37:10,682][60934] Updated weights for policy 1, policy_version 87672 (0.0008) [2023-10-14 00:37:11,022][60935] Updated weights for policy 0, policy_version 88540 (0.0008) [2023-10-14 00:37:11,052][60934] Updated weights for policy 1, policy_version 87682 (0.0008) [2023-10-14 00:37:11,248][59943] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 182059008. Throughput: 0: 1675.6, 1: 1799.1. Samples: 45525860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:37:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:37:11,412][60934] Updated weights for policy 1, policy_version 87692 (0.0009) [2023-10-14 00:37:15,082][60935] Updated weights for policy 0, policy_version 88550 (0.0007) [2023-10-14 00:37:15,404][60934] Updated weights for policy 1, policy_version 87702 (0.0008) [2023-10-14 00:37:15,452][60935] Updated weights for policy 0, policy_version 88560 (0.0008) [2023-10-14 00:37:15,761][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:37:15,768][60934] Updated weights for policy 1, policy_version 87712 (0.0007) [2023-10-14 00:37:15,817][60935] Updated weights for policy 0, policy_version 88570 (0.0008) [2023-10-14 00:37:16,248][59943] Fps is (10 sec: 19660.9, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 182157312. Throughput: 0: 1699.1, 1: 1780.5. Samples: 45536220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:37:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:37:19,821][60935] Updated weights for policy 0, policy_version 88580 (0.0008) [2023-10-14 00:37:20,058][60934] Updated weights for policy 1, policy_version 87722 (0.0008) [2023-10-14 00:37:20,212][60935] Updated weights for policy 0, policy_version 88590 (0.0007) [2023-10-14 00:37:20,274][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-14 00:37:20,570][60935] Updated weights for policy 0, policy_version 88600 (0.0011) [2023-10-14 00:37:21,248][59943] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 182222848. Throughput: 0: 1704.1, 1: 1796.5. Samples: 45557870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:37:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:37:24,305][60934] Updated weights for policy 1, policy_version 87732 (0.0008) [2023-10-14 00:37:24,398][60935] Updated weights for policy 0, policy_version 88610 (0.0009) [2023-10-14 00:37:24,670][60934] Updated weights for policy 1, policy_version 87742 (0.0010) [2023-10-14 00:37:24,733][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000003 [2023-10-14 00:37:24,762][60935] Updated weights for policy 0, policy_version 88620 (0.0008) [2023-10-14 00:37:25,126][60935] Updated weights for policy 0, policy_version 88630 (0.0009) [2023-10-14 00:37:25,487][60935] Updated weights for policy 0, policy_version 88640 (0.0010) [2023-10-14 00:37:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13884.7). Total num frames: 182288384. Throughput: 0: 1685.9, 1: 1796.5. Samples: 45577966. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:37:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:37:28,842][60934] Updated weights for policy 1, policy_version 87752 (0.0008) [2023-10-14 00:37:29,203][60934] Updated weights for policy 1, policy_version 87762 (0.0009) [2023-10-14 00:37:29,514][60935] Updated weights for policy 0, policy_version 88650 (0.0008) [2023-10-14 00:37:29,565][60934] Updated weights for policy 1, policy_version 87772 (0.0007) [2023-10-14 00:37:29,882][60935] Updated weights for policy 0, policy_version 88660 (0.0010) [2023-10-14 00:37:30,259][60935] Updated weights for policy 0, policy_version 88670 (0.0010) [2023-10-14 00:37:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 182353920. Throughput: 0: 1714.8, 1: 1808.5. Samples: 45589890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:37:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:37:33,549][60934] Updated weights for policy 1, policy_version 87782 (0.0010) [2023-10-14 00:37:33,921][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:37:33,926][60934] Updated weights for policy 1, policy_version 87792 (0.0007) [2023-10-14 00:37:34,212][60935] Updated weights for policy 0, policy_version 88680 (0.0008) [2023-10-14 00:37:34,585][60935] Updated weights for policy 0, policy_version 88690 (0.0007) [2023-10-14 00:37:34,957][60935] Updated weights for policy 0, policy_version 88700 (0.0009) [2023-10-14 00:37:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 182419456. Throughput: 0: 1703.0, 1: 1785.8. Samples: 45609590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:37:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:37:38,167][60934] Updated weights for policy 1, policy_version 87802 (0.0010) [2023-10-14 00:37:38,392][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:37:39,000][60935] Updated weights for policy 0, policy_version 88710 (0.0010) [2023-10-14 00:37:39,368][60935] Updated weights for policy 0, policy_version 88720 (0.0010) [2023-10-14 00:37:39,738][60935] Updated weights for policy 0, policy_version 88730 (0.0010) [2023-10-14 00:37:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13884.7). Total num frames: 182484992. Throughput: 0: 1703.5, 1: 1793.6. Samples: 45630850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:37:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:37:42,404][60934] Updated weights for policy 1, policy_version 87812 (0.0009) [2023-10-14 00:37:42,762][60934] Updated weights for policy 1, policy_version 87822 (0.0010) [2023-10-14 00:37:42,831][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-10-14 00:37:43,612][60935] Updated weights for policy 0, policy_version 88740 (0.0010) [2023-10-14 00:37:43,977][60935] Updated weights for policy 0, policy_version 88750 (0.0007) [2023-10-14 00:37:44,345][60935] Updated weights for policy 0, policy_version 88760 (0.0009) [2023-10-14 00:37:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13884.7). Total num frames: 182550528. Throughput: 0: 1718.8, 1: 1788.8. Samples: 45641726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:37:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:37:46,951][60934] Updated weights for policy 1, policy_version 87832 (0.0007) [2023-10-14 00:37:47,240][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000002 [2023-10-14 00:37:48,557][60935] Updated weights for policy 0, policy_version 88770 (0.0008) [2023-10-14 00:37:48,919][60935] Updated weights for policy 0, policy_version 88780 (0.0009) [2023-10-14 00:37:49,297][60935] Updated weights for policy 0, policy_version 88790 (0.0009) [2023-10-14 00:37:49,665][60935] Updated weights for policy 0, policy_version 88800 (0.0008) [2023-10-14 00:37:51,133][60934] Updated weights for policy 1, policy_version 87842 (0.0010) [2023-10-14 00:37:51,248][59943] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 182616064. Throughput: 0: 1688.4, 1: 1796.3. Samples: 45662550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:37:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:37:51,496][60934] Updated weights for policy 1, policy_version 87852 (0.0008) [2023-10-14 00:37:51,639][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:37:53,710][60935] Updated weights for policy 0, policy_version 88810 (0.0008) [2023-10-14 00:37:54,081][60935] Updated weights for policy 0, policy_version 88820 (0.0008) [2023-10-14 00:37:54,440][60935] Updated weights for policy 0, policy_version 88830 (0.0009) [2023-10-14 00:37:55,692][60934] Updated weights for policy 1, policy_version 87862 (0.0007) [2023-10-14 00:37:56,048][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:37:56,053][60934] Updated weights for policy 1, policy_version 87872 (0.0008) [2023-10-14 00:37:56,248][59943] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 13995.8). Total num frames: 182714368. Throughput: 0: 1709.9, 1: 1813.5. Samples: 45684414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:37:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:37:58,481][60935] Updated weights for policy 0, policy_version 88840 (0.0008) [2023-10-14 00:37:58,859][60935] Updated weights for policy 0, policy_version 88850 (0.0007) [2023-10-14 00:37:59,229][60935] Updated weights for policy 0, policy_version 88860 (0.0008) [2023-10-14 00:38:00,404][60934] Updated weights for policy 1, policy_version 87882 (0.0007) [2023-10-14 00:38:00,779][60934] Updated weights for policy 1, policy_version 87892 (0.0008) [2023-10-14 00:38:01,136][60934] Updated weights for policy 1, policy_version 87902 (0.0008) [2023-10-14 00:38:01,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 13995.8). Total num frames: 182779904. Throughput: 0: 1703.4, 1: 1823.5. Samples: 45694930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:38:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:38:03,207][60935] Updated weights for policy 0, policy_version 88870 (0.0009) [2023-10-14 00:38:03,579][60935] Updated weights for policy 0, policy_version 88880 (0.0008) [2023-10-14 00:38:03,962][60935] Updated weights for policy 0, policy_version 88890 (0.0009) [2023-10-14 00:38:05,148][60934] Updated weights for policy 1, policy_version 87912 (0.0008) [2023-10-14 00:38:05,509][60934] Updated weights for policy 1, policy_version 87922 (0.0007) [2023-10-14 00:38:05,870][60934] Updated weights for policy 1, policy_version 87932 (0.0009) [2023-10-14 00:38:06,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 13995.8). Total num frames: 182845440. Throughput: 0: 1683.8, 1: 1813.8. Samples: 45715262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:38:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:38:07,869][60935] Updated weights for policy 0, policy_version 88900 (0.0008) [2023-10-14 00:38:08,241][60935] Updated weights for policy 0, policy_version 88910 (0.0009) [2023-10-14 00:38:08,608][60935] Updated weights for policy 0, policy_version 88920 (0.0008) [2023-10-14 00:38:09,874][60934] Updated weights for policy 1, policy_version 87942 (0.0009) [2023-10-14 00:38:10,230][60934] Updated weights for policy 1, policy_version 87952 (0.0009) [2023-10-14 00:38:10,600][60934] Updated weights for policy 1, policy_version 87962 (0.0008) [2023-10-14 00:38:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 182910976. Throughput: 0: 1716.9, 1: 1790.5. Samples: 45735798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:38:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:38:12,553][60935] Updated weights for policy 0, policy_version 88930 (0.0009) [2023-10-14 00:38:12,922][60935] Updated weights for policy 0, policy_version 88940 (0.0007) [2023-10-14 00:38:13,298][60935] Updated weights for policy 0, policy_version 88950 (0.0007) [2023-10-14 00:38:13,661][60935] Updated weights for policy 0, policy_version 88960 (0.0007) [2023-10-14 00:38:14,555][60934] Updated weights for policy 1, policy_version 87972 (0.0008) [2023-10-14 00:38:14,924][60934] Updated weights for policy 1, policy_version 87982 (0.0009) [2023-10-14 00:38:15,284][60828] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000003 [2023-10-14 00:38:15,290][60934] Updated weights for policy 1, policy_version 87992 (0.0009) [2023-10-14 00:38:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 182976512. Throughput: 0: 1685.2, 1: 1783.6. Samples: 45745988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:38:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:38:17,710][60935] Updated weights for policy 0, policy_version 88970 (0.0007) [2023-10-14 00:38:18,085][60935] Updated weights for policy 0, policy_version 88980 (0.0007) [2023-10-14 00:38:18,456][60935] Updated weights for policy 0, policy_version 88990 (0.0010) [2023-10-14 00:38:19,484][60934] Updated weights for policy 1, policy_version 88002 (0.0007) [2023-10-14 00:38:19,850][60934] Updated weights for policy 1, policy_version 88012 (0.0007) [2023-10-14 00:38:20,205][60934] Updated weights for policy 1, policy_version 88022 (0.0007) [2023-10-14 00:38:20,275][60828] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000010 [2023-10-14 00:38:21,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 183042048. Throughput: 0: 1704.0, 1: 1789.6. Samples: 45766804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:38:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:38:22,466][60935] Updated weights for policy 0, policy_version 89000 (0.0010) [2023-10-14 00:38:22,839][60935] Updated weights for policy 0, policy_version 89010 (0.0007) [2023-10-14 00:38:23,209][60935] Updated weights for policy 0, policy_version 89020 (0.0007) [2023-10-14 00:38:24,297][60934] Updated weights for policy 1, policy_version 88032 (0.0007) [2023-10-14 00:38:24,589][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000005 [2023-10-14 00:38:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 183107584. Throughput: 0: 1712.4, 1: 1766.7. Samples: 45787406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:38:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:38:27,286][60935] Updated weights for policy 0, policy_version 89030 (0.0009) [2023-10-14 00:38:27,662][60935] Updated weights for policy 0, policy_version 89040 (0.0011) [2023-10-14 00:38:28,025][60935] Updated weights for policy 0, policy_version 89050 (0.0011) [2023-10-14 00:38:28,696][60934] Updated weights for policy 1, policy_version 88042 (0.0010) [2023-10-14 00:38:29,073][60934] Updated weights for policy 1, policy_version 88052 (0.0008) [2023-10-14 00:38:29,215][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000006 [2023-10-14 00:38:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13884.8). Total num frames: 183173120. Throughput: 0: 1684.7, 1: 1784.1. Samples: 45797820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:38:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:38:32,148][60935] Updated weights for policy 0, policy_version 89060 (0.0009) [2023-10-14 00:38:32,526][60935] Updated weights for policy 0, policy_version 89070 (0.0008) [2023-10-14 00:38:32,899][60935] Updated weights for policy 0, policy_version 89080 (0.0011) [2023-10-14 00:38:33,581][60934] Updated weights for policy 1, policy_version 88062 (0.0010) [2023-10-14 00:38:33,937][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:38:33,942][60934] Updated weights for policy 1, policy_version 88072 (0.0008) [2023-10-14 00:38:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 183238656. Throughput: 0: 1707.3, 1: 1755.3. Samples: 45818366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:38:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:38:36,937][60935] Updated weights for policy 0, policy_version 89090 (0.0009) [2023-10-14 00:38:37,310][60935] Updated weights for policy 0, policy_version 89100 (0.0008) [2023-10-14 00:38:37,678][60935] Updated weights for policy 0, policy_version 89110 (0.0009) [2023-10-14 00:38:38,048][60935] Updated weights for policy 0, policy_version 89120 (0.0010) [2023-10-14 00:38:38,301][60934] Updated weights for policy 1, policy_version 88082 (0.0009) [2023-10-14 00:38:38,677][60934] Updated weights for policy 1, policy_version 88092 (0.0008) [2023-10-14 00:38:39,042][60934] Updated weights for policy 1, policy_version 88102 (0.0008) [2023-10-14 00:38:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 183304192. Throughput: 0: 1708.3, 1: 1730.8. Samples: 45839176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:38:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:38:41,921][60935] Updated weights for policy 0, policy_version 89130 (0.0011) [2023-10-14 00:38:42,297][60935] Updated weights for policy 0, policy_version 89140 (0.0008) [2023-10-14 00:38:42,659][60935] Updated weights for policy 0, policy_version 89150 (0.0007) [2023-10-14 00:38:42,927][60934] Updated weights for policy 1, policy_version 88112 (0.0008) [2023-10-14 00:38:43,296][60934] Updated weights for policy 1, policy_version 88122 (0.0008) [2023-10-14 00:38:43,655][60934] Updated weights for policy 1, policy_version 88132 (0.0009) [2023-10-14 00:38:46,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 183369728. Throughput: 0: 1696.4, 1: 1727.3. Samples: 45848998. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 00:38:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:38:46,750][60935] Updated weights for policy 0, policy_version 89160 (0.0009) [2023-10-14 00:38:47,117][60935] Updated weights for policy 0, policy_version 89170 (0.0009) [2023-10-14 00:38:47,486][60935] Updated weights for policy 0, policy_version 89180 (0.0010) [2023-10-14 00:38:47,701][60934] Updated weights for policy 1, policy_version 88142 (0.0009) [2023-10-14 00:38:48,065][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000009 [2023-10-14 00:38:48,071][60934] Updated weights for policy 1, policy_version 88152 (0.0009) [2023-10-14 00:38:51,167][60935] Updated weights for policy 0, policy_version 89190 (0.0008) [2023-10-14 00:38:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 183435264. Throughput: 0: 1716.7, 1: 1732.2. Samples: 45870462. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 00:38:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:38:51,529][60935] Updated weights for policy 0, policy_version 89200 (0.0007) [2023-10-14 00:38:51,896][60935] Updated weights for policy 0, policy_version 89210 (0.0007) [2023-10-14 00:38:52,335][60934] Updated weights for policy 1, policy_version 88162 (0.0010) [2023-10-14 00:38:52,548][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000007 [2023-10-14 00:38:55,909][60935] Updated weights for policy 0, policy_version 89220 (0.0009) [2023-10-14 00:38:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13884.7). Total num frames: 183500800. Throughput: 0: 1707.7, 1: 1775.3. Samples: 45892532. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 00:38:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:38:56,295][60935] Updated weights for policy 0, policy_version 89230 (0.0007) [2023-10-14 00:38:56,385][60934] Updated weights for policy 1, policy_version 88172 (0.0009) [2023-10-14 00:38:56,658][60935] Updated weights for policy 0, policy_version 89240 (0.0007) [2023-10-14 00:38:56,759][60934] Updated weights for policy 1, policy_version 88182 (0.0009) [2023-10-14 00:38:56,946][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000089248_91389952.pth... [2023-10-14 00:38:56,974][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000087648_89751552.pth [2023-10-14 00:38:57,124][60934] Updated weights for policy 1, policy_version 88192 (0.0009) [2023-10-14 00:38:57,408][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000088200_92176384.pth... [2023-10-14 00:38:57,445][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000086976_90439680.pth [2023-10-14 00:39:00,802][60935] Updated weights for policy 0, policy_version 89250 (0.0008) [2023-10-14 00:39:01,075][60934] Updated weights for policy 1, policy_version 88202 (0.0007) [2023-10-14 00:39:01,169][60935] Updated weights for policy 0, policy_version 89260 (0.0007) [2023-10-14 00:39:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13884.7). Total num frames: 183566336. Throughput: 0: 1708.9, 1: 1751.3. Samples: 45901698. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 00:39:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:39:01,437][60934] Updated weights for policy 1, policy_version 88212 (0.0008) [2023-10-14 00:39:01,538][60935] Updated weights for policy 0, policy_version 89270 (0.0007) [2023-10-14 00:39:01,791][60934] Updated weights for policy 1, policy_version 88222 (0.0008) [2023-10-14 00:39:01,893][60935] Updated weights for policy 0, policy_version 89280 (0.0009) [2023-10-14 00:39:02,152][60934] Updated weights for policy 1, policy_version 88232 (0.0010) [2023-10-14 00:39:05,896][60935] Updated weights for policy 0, policy_version 89290 (0.0008) [2023-10-14 00:39:06,162][60934] Updated weights for policy 1, policy_version 88242 (0.0007) [2023-10-14 00:39:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13884.7). Total num frames: 183631872. Throughput: 0: 1708.4, 1: 1757.7. Samples: 45922776. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 00:39:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:39:06,267][60935] Updated weights for policy 0, policy_version 89300 (0.0008) [2023-10-14 00:39:06,379][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:39:06,633][60935] Updated weights for policy 0, policy_version 89310 (0.0009) [2023-10-14 00:39:10,304][60934] Updated weights for policy 1, policy_version 88252 (0.0009) [2023-10-14 00:39:10,565][60935] Updated weights for policy 0, policy_version 89320 (0.0008) [2023-10-14 00:39:10,703][60934] Updated weights for policy 1, policy_version 88262 (0.0008) [2023-10-14 00:39:10,947][60935] Updated weights for policy 0, policy_version 89330 (0.0008) [2023-10-14 00:39:11,071][60934] Updated weights for policy 1, policy_version 88272 (0.0007) [2023-10-14 00:39:11,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13884.7). Total num frames: 183697408. Throughput: 0: 1693.7, 1: 1774.8. Samples: 45943488. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 00:39:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:39:11,315][60935] Updated weights for policy 0, policy_version 89340 (0.0009) [2023-10-14 00:39:14,892][60934] Updated weights for policy 1, policy_version 88282 (0.0007) [2023-10-14 00:39:15,257][60934] Updated weights for policy 1, policy_version 88292 (0.0009) [2023-10-14 00:39:15,462][60935] Updated weights for policy 0, policy_version 89350 (0.0009) [2023-10-14 00:39:15,620][60934] Updated weights for policy 1, policy_version 88302 (0.0009) [2023-10-14 00:39:15,828][60935] Updated weights for policy 0, policy_version 89360 (0.0008) [2023-10-14 00:39:15,991][60934] Updated weights for policy 1, policy_version 88312 (0.0008) [2023-10-14 00:39:16,201][60935] Updated weights for policy 0, policy_version 89370 (0.0007) [2023-10-14 00:39:16,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 183795712. Throughput: 0: 1709.8, 1: 1756.2. Samples: 45953792. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 00:39:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:39:20,074][60934] Updated weights for policy 1, policy_version 88322 (0.0007) [2023-10-14 00:39:20,128][60935] Updated weights for policy 0, policy_version 89380 (0.0009) [2023-10-14 00:39:20,290][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:39:20,490][60935] Updated weights for policy 0, policy_version 89390 (0.0008) [2023-10-14 00:39:20,862][60935] Updated weights for policy 0, policy_version 89400 (0.0010) [2023-10-14 00:39:21,248][59943] Fps is (10 sec: 19661.3, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 183894016. Throughput: 0: 1708.8, 1: 1765.2. Samples: 45974698. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 00:39:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:39:24,451][60934] Updated weights for policy 1, policy_version 88332 (0.0007) [2023-10-14 00:39:24,821][60934] Updated weights for policy 1, policy_version 88342 (0.0009) [2023-10-14 00:39:24,885][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:39:25,070][60935] Updated weights for policy 0, policy_version 89410 (0.0009) [2023-10-14 00:39:25,437][60935] Updated weights for policy 0, policy_version 89420 (0.0009) [2023-10-14 00:39:25,818][60935] Updated weights for policy 0, policy_version 89430 (0.0011) [2023-10-14 00:39:26,191][60935] Updated weights for policy 0, policy_version 89440 (0.0007) [2023-10-14 00:39:26,248][59943] Fps is (10 sec: 16383.6, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 183959552. Throughput: 0: 1685.8, 1: 1765.5. Samples: 45994484. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 00:39:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:39:29,114][60934] Updated weights for policy 1, policy_version 88352 (0.0008) [2023-10-14 00:39:29,402][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:39:30,069][60935] Updated weights for policy 0, policy_version 89450 (0.0009) [2023-10-14 00:39:30,433][60935] Updated weights for policy 0, policy_version 89460 (0.0010) [2023-10-14 00:39:30,803][60935] Updated weights for policy 0, policy_version 89470 (0.0008) [2023-10-14 00:39:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13884.7). Total num frames: 184025088. Throughput: 0: 1703.8, 1: 1783.5. Samples: 46005928. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 00:39:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:39:33,366][60934] Updated weights for policy 1, policy_version 88362 (0.0008) [2023-10-14 00:39:33,732][60934] Updated weights for policy 1, policy_version 88372 (0.0009) [2023-10-14 00:39:33,876][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:39:34,623][60935] Updated weights for policy 0, policy_version 89480 (0.0008) [2023-10-14 00:39:34,985][60935] Updated weights for policy 0, policy_version 89490 (0.0008) [2023-10-14 00:39:35,347][60935] Updated weights for policy 0, policy_version 89500 (0.0008) [2023-10-14 00:39:36,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13884.7). Total num frames: 184090624. Throughput: 0: 1694.6, 1: 1778.3. Samples: 46026742. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 00:39:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:39:37,873][60934] Updated weights for policy 1, policy_version 88382 (0.0008) [2023-10-14 00:39:38,225][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:39:38,226][60934] Updated weights for policy 1, policy_version 88392 (0.0007) [2023-10-14 00:39:39,367][60935] Updated weights for policy 0, policy_version 89510 (0.0008) [2023-10-14 00:39:39,737][60935] Updated weights for policy 0, policy_version 89520 (0.0007) [2023-10-14 00:39:40,099][60935] Updated weights for policy 0, policy_version 89530 (0.0008) [2023-10-14 00:39:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 184156160. Throughput: 0: 1677.6, 1: 1773.3. Samples: 46047822. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 00:39:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:39:42,579][60934] Updated weights for policy 1, policy_version 88402 (0.0007) [2023-10-14 00:39:42,939][60934] Updated weights for policy 1, policy_version 88412 (0.0008) [2023-10-14 00:39:43,305][60934] Updated weights for policy 1, policy_version 88422 (0.0007) [2023-10-14 00:39:44,114][60935] Updated weights for policy 0, policy_version 89540 (0.0007) [2023-10-14 00:39:44,505][60935] Updated weights for policy 0, policy_version 89550 (0.0010) [2023-10-14 00:39:44,870][60935] Updated weights for policy 0, policy_version 89560 (0.0007) [2023-10-14 00:39:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 184221696. Throughput: 0: 1712.9, 1: 1771.9. Samples: 46058518. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 00:39:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:39:47,421][60934] Updated weights for policy 1, policy_version 88432 (0.0007) [2023-10-14 00:39:47,784][60934] Updated weights for policy 1, policy_version 88442 (0.0008) [2023-10-14 00:39:48,140][60934] Updated weights for policy 1, policy_version 88452 (0.0008) [2023-10-14 00:39:49,096][60935] Updated weights for policy 0, policy_version 89570 (0.0007) [2023-10-14 00:39:49,476][60935] Updated weights for policy 0, policy_version 89580 (0.0009) [2023-10-14 00:39:49,840][60935] Updated weights for policy 0, policy_version 89590 (0.0009) [2023-10-14 00:39:50,205][60935] Updated weights for policy 0, policy_version 89600 (0.0010) [2023-10-14 00:39:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 184287232. Throughput: 0: 1688.0, 1: 1775.5. Samples: 46078632. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 00:39:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:39:52,022][60934] Updated weights for policy 1, policy_version 88462 (0.0009) [2023-10-14 00:39:52,378][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:39:52,384][60934] Updated weights for policy 1, policy_version 88472 (0.0008) [2023-10-14 00:39:54,096][60935] Updated weights for policy 0, policy_version 89610 (0.0008) [2023-10-14 00:39:54,474][60935] Updated weights for policy 0, policy_version 89620 (0.0008) [2023-10-14 00:39:54,836][60935] Updated weights for policy 0, policy_version 89630 (0.0009) [2023-10-14 00:39:56,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 184352768. Throughput: 0: 1694.2, 1: 1785.5. Samples: 46100072. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 00:39:56,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-14 00:39:56,711][60934] Updated weights for policy 1, policy_version 88482 (0.0009) [2023-10-14 00:39:56,935][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:39:58,856][60935] Updated weights for policy 0, policy_version 89640 (0.0011) [2023-10-14 00:39:59,233][60935] Updated weights for policy 0, policy_version 89650 (0.0008) [2023-10-14 00:39:59,611][60935] Updated weights for policy 0, policy_version 89660 (0.0008) [2023-10-14 00:40:01,039][60934] Updated weights for policy 1, policy_version 88492 (0.0008) [2023-10-14 00:40:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 184418304. Throughput: 0: 1702.5, 1: 1781.0. Samples: 46110548. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 00:40:01,249][59943] Avg episode reward: [(0, '-0.150'), (1, '0.000')] [2023-10-14 00:40:01,403][60934] Updated weights for policy 1, policy_version 88502 (0.0011) [2023-10-14 00:40:01,769][60934] Updated weights for policy 1, policy_version 88512 (0.0008) [2023-10-14 00:40:03,479][60935] Updated weights for policy 0, policy_version 89670 (0.0009) [2023-10-14 00:40:03,837][60935] Updated weights for policy 0, policy_version 89680 (0.0007) [2023-10-14 00:40:04,207][60935] Updated weights for policy 0, policy_version 89690 (0.0008) [2023-10-14 00:40:05,776][60934] Updated weights for policy 1, policy_version 88522 (0.0008) [2023-10-14 00:40:06,130][60934] Updated weights for policy 1, policy_version 88532 (0.0009) [2023-10-14 00:40:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 184483840. Throughput: 0: 1683.2, 1: 1785.7. Samples: 46130800. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 00:40:06,249][59943] Avg episode reward: [(0, '-0.150'), (1, '0.000')] [2023-10-14 00:40:06,491][60934] Updated weights for policy 1, policy_version 88542 (0.0009) [2023-10-14 00:40:06,861][60934] Updated weights for policy 1, policy_version 88552 (0.0009) [2023-10-14 00:40:08,155][60935] Updated weights for policy 0, policy_version 89700 (0.0008) [2023-10-14 00:40:08,521][60935] Updated weights for policy 0, policy_version 89710 (0.0011) [2023-10-14 00:40:08,888][60935] Updated weights for policy 0, policy_version 89720 (0.0009) [2023-10-14 00:40:10,889][60934] Updated weights for policy 1, policy_version 88562 (0.0008) [2023-10-14 00:40:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 184549376. Throughput: 0: 1706.4, 1: 1797.5. Samples: 46152156. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 00:40:11,249][59943] Avg episode reward: [(0, '-0.150'), (1, '0.000')] [2023-10-14 00:40:11,252][60934] Updated weights for policy 1, policy_version 88572 (0.0008) [2023-10-14 00:40:11,623][60934] Updated weights for policy 1, policy_version 88582 (0.0007) [2023-10-14 00:40:13,049][60935] Updated weights for policy 0, policy_version 89730 (0.0008) [2023-10-14 00:40:13,414][60935] Updated weights for policy 0, policy_version 89740 (0.0009) [2023-10-14 00:40:13,793][60935] Updated weights for policy 0, policy_version 89750 (0.0008) [2023-10-14 00:40:14,158][60935] Updated weights for policy 0, policy_version 89760 (0.0009) [2023-10-14 00:40:15,323][60934] Updated weights for policy 1, policy_version 88592 (0.0008) [2023-10-14 00:40:15,689][60934] Updated weights for policy 1, policy_version 88602 (0.0009) [2023-10-14 00:40:16,058][60934] Updated weights for policy 1, policy_version 88612 (0.0009) [2023-10-14 00:40:16,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 184647680. Throughput: 0: 1696.2, 1: 1770.9. Samples: 46161948. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 00:40:16,249][59943] Avg episode reward: [(0, '-0.270'), (1, '0.000')] [2023-10-14 00:40:18,064][60935] Updated weights for policy 0, policy_version 89770 (0.0008) [2023-10-14 00:40:18,431][60935] Updated weights for policy 0, policy_version 89780 (0.0009) [2023-10-14 00:40:18,811][60935] Updated weights for policy 0, policy_version 89790 (0.0007) [2023-10-14 00:40:20,017][60934] Updated weights for policy 1, policy_version 88622 (0.0008) [2023-10-14 00:40:20,381][60934] Updated weights for policy 1, policy_version 88632 (0.0007) [2023-10-14 00:40:20,749][60934] Updated weights for policy 1, policy_version 88642 (0.0007) [2023-10-14 00:40:21,248][59943] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13884.8). Total num frames: 184713216. Throughput: 0: 1692.6, 1: 1774.4. Samples: 46182758. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 00:40:21,249][59943] Avg episode reward: [(0, '-0.270'), (1, '0.000')] [2023-10-14 00:40:22,973][60935] Updated weights for policy 0, policy_version 89800 (0.0008) [2023-10-14 00:40:23,344][60935] Updated weights for policy 0, policy_version 89810 (0.0010) [2023-10-14 00:40:23,722][60935] Updated weights for policy 0, policy_version 89820 (0.0008) [2023-10-14 00:40:24,716][60934] Updated weights for policy 1, policy_version 88652 (0.0008) [2023-10-14 00:40:25,089][60934] Updated weights for policy 1, policy_version 88662 (0.0009) [2023-10-14 00:40:25,455][60934] Updated weights for policy 1, policy_version 88672 (0.0008) [2023-10-14 00:40:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 184778752. Throughput: 0: 1708.0, 1: 1738.9. Samples: 46202934. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 00:40:26,249][59943] Avg episode reward: [(0, '-0.130'), (1, '0.000')] [2023-10-14 00:40:27,668][60935] Updated weights for policy 0, policy_version 89830 (0.0008) [2023-10-14 00:40:28,041][60935] Updated weights for policy 0, policy_version 89840 (0.0010) [2023-10-14 00:40:28,415][60935] Updated weights for policy 0, policy_version 89850 (0.0009) [2023-10-14 00:40:29,390][60934] Updated weights for policy 1, policy_version 88682 (0.0007) [2023-10-14 00:40:29,760][60934] Updated weights for policy 1, policy_version 88692 (0.0008) [2023-10-14 00:40:30,126][60934] Updated weights for policy 1, policy_version 88702 (0.0008) [2023-10-14 00:40:30,499][60934] Updated weights for policy 1, policy_version 88712 (0.0011) [2023-10-14 00:40:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 184844288. Throughput: 0: 1672.3, 1: 1764.9. Samples: 46213192. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 00:40:31,249][59943] Avg episode reward: [(0, '-0.130'), (1, '0.000')] [2023-10-14 00:40:32,350][60935] Updated weights for policy 0, policy_version 89860 (0.0010) [2023-10-14 00:40:32,720][60935] Updated weights for policy 0, policy_version 89870 (0.0010) [2023-10-14 00:40:33,094][60935] Updated weights for policy 0, policy_version 89880 (0.0009) [2023-10-14 00:40:34,400][60934] Updated weights for policy 1, policy_version 88722 (0.0010) [2023-10-14 00:40:34,760][60934] Updated weights for policy 1, policy_version 88732 (0.0008) [2023-10-14 00:40:35,121][60934] Updated weights for policy 1, policy_version 88742 (0.0010) [2023-10-14 00:40:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 184909824. Throughput: 0: 1705.7, 1: 1748.6. Samples: 46234074. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 00:40:36,250][59943] Avg episode reward: [(0, '-0.130'), (1, '0.000')] [2023-10-14 00:40:37,134][60935] Updated weights for policy 0, policy_version 89890 (0.0009) [2023-10-14 00:40:37,529][60935] Updated weights for policy 0, policy_version 89900 (0.0010) [2023-10-14 00:40:37,896][60935] Updated weights for policy 0, policy_version 89910 (0.0007) [2023-10-14 00:40:38,274][60935] Updated weights for policy 0, policy_version 89920 (0.0009) [2023-10-14 00:40:39,134][60934] Updated weights for policy 1, policy_version 88752 (0.0010) [2023-10-14 00:40:39,512][60934] Updated weights for policy 1, policy_version 88762 (0.0009) [2023-10-14 00:40:39,883][60934] Updated weights for policy 1, policy_version 88772 (0.0008) [2023-10-14 00:40:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13884.8). Total num frames: 184975360. Throughput: 0: 1714.2, 1: 1710.9. Samples: 46254200. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-14 00:40:41,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-14 00:40:42,094][60935] Updated weights for policy 0, policy_version 89930 (0.0009) [2023-10-14 00:40:42,451][60935] Updated weights for policy 0, policy_version 89940 (0.0010) [2023-10-14 00:40:42,827][60935] Updated weights for policy 0, policy_version 89950 (0.0009) [2023-10-14 00:40:43,885][60934] Updated weights for policy 1, policy_version 88782 (0.0008) [2023-10-14 00:40:44,255][60934] Updated weights for policy 1, policy_version 88792 (0.0009) [2023-10-14 00:40:44,627][60934] Updated weights for policy 1, policy_version 88802 (0.0007) [2023-10-14 00:40:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 185040896. Throughput: 0: 1692.0, 1: 1736.5. Samples: 46264832. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-14 00:40:46,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-14 00:40:46,891][60935] Updated weights for policy 0, policy_version 89960 (0.0009) [2023-10-14 00:40:47,254][60935] Updated weights for policy 0, policy_version 89970 (0.0008) [2023-10-14 00:40:47,624][60935] Updated weights for policy 0, policy_version 89980 (0.0007) [2023-10-14 00:40:48,607][60934] Updated weights for policy 1, policy_version 88812 (0.0008) [2023-10-14 00:40:48,972][60934] Updated weights for policy 1, policy_version 88822 (0.0007) [2023-10-14 00:40:49,338][60934] Updated weights for policy 1, policy_version 88832 (0.0008) [2023-10-14 00:40:51,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 185106432. Throughput: 0: 1711.7, 1: 1703.7. Samples: 46284492. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-14 00:40:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:40:51,649][60935] Updated weights for policy 0, policy_version 89990 (0.0008) [2023-10-14 00:40:52,022][60935] Updated weights for policy 0, policy_version 90000 (0.0007) [2023-10-14 00:40:52,391][60935] Updated weights for policy 0, policy_version 90010 (0.0008) [2023-10-14 00:40:53,406][60934] Updated weights for policy 1, policy_version 88842 (0.0008) [2023-10-14 00:40:53,773][60934] Updated weights for policy 1, policy_version 88852 (0.0007) [2023-10-14 00:40:54,142][60934] Updated weights for policy 1, policy_version 88862 (0.0008) [2023-10-14 00:40:54,515][60934] Updated weights for policy 1, policy_version 88872 (0.0009) [2023-10-14 00:40:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 185171968. Throughput: 0: 1712.8, 1: 1692.2. Samples: 46305382. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-14 00:40:56,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-14 00:40:56,256][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000088872_92995584.pth... [2023-10-14 00:40:56,288][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000087600_91291648.pth [2023-10-14 00:40:56,324][60935] Updated weights for policy 0, policy_version 90020 (0.0010) [2023-10-14 00:40:56,682][60935] Updated weights for policy 0, policy_version 90030 (0.0010) [2023-10-14 00:40:57,060][60935] Updated weights for policy 0, policy_version 90040 (0.0008) [2023-10-14 00:40:57,344][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000090048_92209152.pth... [2023-10-14 00:40:57,382][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000088448_90570752.pth [2023-10-14 00:40:58,375][60934] Updated weights for policy 1, policy_version 88882 (0.0009) [2023-10-14 00:40:58,752][60934] Updated weights for policy 1, policy_version 88892 (0.0009) [2023-10-14 00:40:59,119][60934] Updated weights for policy 1, policy_version 88902 (0.0008) [2023-10-14 00:41:00,936][60935] Updated weights for policy 0, policy_version 90050 (0.0008) [2023-10-14 00:41:01,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 185237504. Throughput: 0: 1705.5, 1: 1709.7. Samples: 46315632. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-14 00:41:01,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-14 00:41:01,310][60935] Updated weights for policy 0, policy_version 90060 (0.0008) [2023-10-14 00:41:01,680][60935] Updated weights for policy 0, policy_version 90070 (0.0009) [2023-10-14 00:41:02,041][60935] Updated weights for policy 0, policy_version 90080 (0.0010) [2023-10-14 00:41:03,170][60934] Updated weights for policy 1, policy_version 88912 (0.0009) [2023-10-14 00:41:03,542][60934] Updated weights for policy 1, policy_version 88922 (0.0009) [2023-10-14 00:41:03,915][60934] Updated weights for policy 1, policy_version 88932 (0.0007) [2023-10-14 00:41:05,919][60935] Updated weights for policy 0, policy_version 90090 (0.0010) [2023-10-14 00:41:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 185303040. Throughput: 0: 1724.5, 1: 1690.9. Samples: 46336452. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-14 00:41:06,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-14 00:41:06,291][60935] Updated weights for policy 0, policy_version 90100 (0.0009) [2023-10-14 00:41:06,665][60935] Updated weights for policy 0, policy_version 90110 (0.0009) [2023-10-14 00:41:07,970][60934] Updated weights for policy 1, policy_version 88942 (0.0008) [2023-10-14 00:41:08,334][60934] Updated weights for policy 1, policy_version 88952 (0.0008) [2023-10-14 00:41:08,711][60934] Updated weights for policy 1, policy_version 88962 (0.0009) [2023-10-14 00:41:10,752][60935] Updated weights for policy 0, policy_version 90120 (0.0009) [2023-10-14 00:41:11,107][60935] Updated weights for policy 0, policy_version 90130 (0.0010) [2023-10-14 00:41:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 185368576. Throughput: 0: 1712.5, 1: 1711.2. Samples: 46356998. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-14 00:41:11,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-14 00:41:11,478][60935] Updated weights for policy 0, policy_version 90140 (0.0010) [2023-10-14 00:41:12,612][60934] Updated weights for policy 1, policy_version 88972 (0.0007) [2023-10-14 00:41:12,977][60934] Updated weights for policy 1, policy_version 88982 (0.0007) [2023-10-14 00:41:13,350][60934] Updated weights for policy 1, policy_version 88992 (0.0008) [2023-10-14 00:41:15,441][60935] Updated weights for policy 0, policy_version 90150 (0.0009) [2023-10-14 00:41:15,810][60935] Updated weights for policy 0, policy_version 90160 (0.0008) [2023-10-14 00:41:16,172][60935] Updated weights for policy 0, policy_version 90170 (0.0007) [2023-10-14 00:41:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13773.7). Total num frames: 185434112. Throughput: 0: 1727.5, 1: 1691.9. Samples: 46367068. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-14 00:41:16,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-14 00:41:17,239][60934] Updated weights for policy 1, policy_version 89002 (0.0007) [2023-10-14 00:41:17,613][60934] Updated weights for policy 1, policy_version 89012 (0.0008) [2023-10-14 00:41:17,967][60934] Updated weights for policy 1, policy_version 89022 (0.0008) [2023-10-14 00:41:18,328][60934] Updated weights for policy 1, policy_version 89032 (0.0009) [2023-10-14 00:41:20,095][60935] Updated weights for policy 0, policy_version 90180 (0.0007) [2023-10-14 00:41:20,471][60935] Updated weights for policy 0, policy_version 90190 (0.0008) [2023-10-14 00:41:20,844][60935] Updated weights for policy 0, policy_version 90200 (0.0008) [2023-10-14 00:41:21,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13884.8). Total num frames: 185532416. Throughput: 0: 1725.4, 1: 1698.2. Samples: 46388136. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-14 00:41:21,249][59943] Avg episode reward: [(0, '-0.010'), (1, '0.000')] [2023-10-14 00:41:22,508][60934] Updated weights for policy 1, policy_version 89042 (0.0007) [2023-10-14 00:41:22,863][60934] Updated weights for policy 1, policy_version 89052 (0.0007) [2023-10-14 00:41:23,235][60934] Updated weights for policy 1, policy_version 89062 (0.0007) [2023-10-14 00:41:24,837][60935] Updated weights for policy 0, policy_version 90210 (0.0011) [2023-10-14 00:41:25,228][60935] Updated weights for policy 0, policy_version 90220 (0.0008) [2023-10-14 00:41:25,606][60935] Updated weights for policy 0, policy_version 90230 (0.0009) [2023-10-14 00:41:25,972][60935] Updated weights for policy 0, policy_version 90240 (0.0007) [2023-10-14 00:41:26,248][59943] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13884.8). Total num frames: 185597952. Throughput: 0: 1699.2, 1: 1716.2. Samples: 46407892. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-14 00:41:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:41:27,204][60934] Updated weights for policy 1, policy_version 89072 (0.0007) [2023-10-14 00:41:27,569][60934] Updated weights for policy 1, policy_version 89082 (0.0009) [2023-10-14 00:41:27,933][60934] Updated weights for policy 1, policy_version 89092 (0.0009) [2023-10-14 00:41:29,982][60935] Updated weights for policy 0, policy_version 90250 (0.0008) [2023-10-14 00:41:30,357][60935] Updated weights for policy 0, policy_version 90260 (0.0009) [2023-10-14 00:41:30,728][60935] Updated weights for policy 0, policy_version 90270 (0.0008) [2023-10-14 00:41:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13884.8). Total num frames: 185663488. Throughput: 0: 1727.0, 1: 1682.1. Samples: 46418240. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-14 00:41:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:41:31,981][60934] Updated weights for policy 1, policy_version 89102 (0.0009) [2023-10-14 00:41:32,348][60934] Updated weights for policy 1, policy_version 89112 (0.0009) [2023-10-14 00:41:32,723][60934] Updated weights for policy 1, policy_version 89122 (0.0009) [2023-10-14 00:41:34,787][60935] Updated weights for policy 0, policy_version 90280 (0.0009) [2023-10-14 00:41:35,155][60935] Updated weights for policy 0, policy_version 90290 (0.0007) [2023-10-14 00:41:35,531][60935] Updated weights for policy 0, policy_version 90300 (0.0008) [2023-10-14 00:41:36,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 185729024. Throughput: 0: 1719.4, 1: 1712.6. Samples: 46438932. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-14 00:41:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:41:36,750][60934] Updated weights for policy 1, policy_version 89132 (0.0009) [2023-10-14 00:41:37,121][60934] Updated weights for policy 1, policy_version 89142 (0.0010) [2023-10-14 00:41:37,490][60934] Updated weights for policy 1, policy_version 89152 (0.0011) [2023-10-14 00:41:39,463][60935] Updated weights for policy 0, policy_version 90310 (0.0010) [2023-10-14 00:41:39,837][60935] Updated weights for policy 0, policy_version 90320 (0.0011) [2023-10-14 00:41:40,200][60935] Updated weights for policy 0, policy_version 90330 (0.0009) [2023-10-14 00:41:41,248][59943] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 185794560. Throughput: 0: 1698.8, 1: 1715.5. Samples: 46459026. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-14 00:41:41,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:41:41,443][60934] Updated weights for policy 1, policy_version 89162 (0.0010) [2023-10-14 00:41:41,805][60934] Updated weights for policy 1, policy_version 89172 (0.0009) [2023-10-14 00:41:42,165][60934] Updated weights for policy 1, policy_version 89182 (0.0010) [2023-10-14 00:41:42,526][60934] Updated weights for policy 1, policy_version 89192 (0.0011) [2023-10-14 00:41:44,100][60935] Updated weights for policy 0, policy_version 90340 (0.0009) [2023-10-14 00:41:44,471][60935] Updated weights for policy 0, policy_version 90350 (0.0007) [2023-10-14 00:41:44,833][60935] Updated weights for policy 0, policy_version 90360 (0.0007) [2023-10-14 00:41:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 185860096. Throughput: 0: 1727.3, 1: 1696.4. Samples: 46469700. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-14 00:41:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:41:46,480][60934] Updated weights for policy 1, policy_version 89202 (0.0010) [2023-10-14 00:41:46,846][60934] Updated weights for policy 1, policy_version 89212 (0.0012) [2023-10-14 00:41:47,215][60934] Updated weights for policy 1, policy_version 89222 (0.0009) [2023-10-14 00:41:48,890][60935] Updated weights for policy 0, policy_version 90370 (0.0010) [2023-10-14 00:41:49,251][60935] Updated weights for policy 0, policy_version 90380 (0.0007) [2023-10-14 00:41:49,620][60935] Updated weights for policy 0, policy_version 90390 (0.0008) [2023-10-14 00:41:49,983][60935] Updated weights for policy 0, policy_version 90400 (0.0008) [2023-10-14 00:41:51,219][60934] Updated weights for policy 1, policy_version 89232 (0.0009) [2023-10-14 00:41:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13884.8). Total num frames: 185925632. Throughput: 0: 1690.3, 1: 1712.3. Samples: 46489568. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-14 00:41:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:41:51,576][60934] Updated weights for policy 1, policy_version 89242 (0.0008) [2023-10-14 00:41:51,941][60934] Updated weights for policy 1, policy_version 89252 (0.0009) [2023-10-14 00:41:53,877][60935] Updated weights for policy 0, policy_version 90410 (0.0009) [2023-10-14 00:41:54,246][60935] Updated weights for policy 0, policy_version 90420 (0.0008) [2023-10-14 00:41:54,617][60935] Updated weights for policy 0, policy_version 90430 (0.0009) [2023-10-14 00:41:55,808][60934] Updated weights for policy 1, policy_version 89262 (0.0008) [2023-10-14 00:41:56,177][60934] Updated weights for policy 1, policy_version 89272 (0.0009) [2023-10-14 00:41:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13884.7). Total num frames: 185991168. Throughput: 0: 1694.8, 1: 1721.0. Samples: 46510708. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-14 00:41:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:41:56,543][60934] Updated weights for policy 1, policy_version 89282 (0.0008) [2023-10-14 00:41:58,849][60935] Updated weights for policy 0, policy_version 90440 (0.0009) [2023-10-14 00:41:59,221][60935] Updated weights for policy 0, policy_version 90450 (0.0009) [2023-10-14 00:41:59,582][60935] Updated weights for policy 0, policy_version 90460 (0.0009) [2023-10-14 00:42:00,497][60934] Updated weights for policy 1, policy_version 89292 (0.0007) [2023-10-14 00:42:00,850][60934] Updated weights for policy 1, policy_version 89302 (0.0007) [2023-10-14 00:42:00,920][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000003 [2023-10-14 00:42:01,248][59943] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 186089472. Throughput: 0: 1703.2, 1: 1713.7. Samples: 46520830. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-14 00:42:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:42:03,743][60935] Updated weights for policy 0, policy_version 90470 (0.0010) [2023-10-14 00:42:04,112][60935] Updated weights for policy 0, policy_version 90480 (0.0010) [2023-10-14 00:42:04,477][60935] Updated weights for policy 0, policy_version 90490 (0.0009) [2023-10-14 00:42:05,098][60934] Updated weights for policy 1, policy_version 89312 (0.0010) [2023-10-14 00:42:05,464][60934] Updated weights for policy 1, policy_version 89322 (0.0009) [2023-10-14 00:42:05,837][60934] Updated weights for policy 1, policy_version 89332 (0.0007) [2023-10-14 00:42:06,248][59943] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 186155008. Throughput: 0: 1671.9, 1: 1733.8. Samples: 46541392. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-14 00:42:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:42:08,416][60935] Updated weights for policy 0, policy_version 90500 (0.0009) [2023-10-14 00:42:08,785][60935] Updated weights for policy 0, policy_version 90510 (0.0009) [2023-10-14 00:42:09,155][60935] Updated weights for policy 0, policy_version 90520 (0.0010) [2023-10-14 00:42:09,806][60934] Updated weights for policy 1, policy_version 89342 (0.0008) [2023-10-14 00:42:10,175][60934] Updated weights for policy 1, policy_version 89352 (0.0009) [2023-10-14 00:42:10,537][60934] Updated weights for policy 1, policy_version 89362 (0.0008) [2023-10-14 00:42:11,249][59943] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 13773.6). Total num frames: 186220544. Throughput: 0: 1701.2, 1: 1716.6. Samples: 46561696. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-14 00:42:11,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:42:13,117][60935] Updated weights for policy 0, policy_version 90530 (0.0009) [2023-10-14 00:42:13,525][60935] Updated weights for policy 0, policy_version 90540 (0.0007) [2023-10-14 00:42:13,891][60935] Updated weights for policy 0, policy_version 90550 (0.0009) [2023-10-14 00:42:14,259][60935] Updated weights for policy 0, policy_version 90560 (0.0009) [2023-10-14 00:42:14,522][60934] Updated weights for policy 1, policy_version 89372 (0.0008) [2023-10-14 00:42:14,885][60934] Updated weights for policy 1, policy_version 89382 (0.0007) [2023-10-14 00:42:15,246][60934] Updated weights for policy 1, policy_version 89392 (0.0007) [2023-10-14 00:42:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 186286080. Throughput: 0: 1683.1, 1: 1743.3. Samples: 46572428. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-14 00:42:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:42:18,279][60935] Updated weights for policy 0, policy_version 90570 (0.0008) [2023-10-14 00:42:18,650][60935] Updated weights for policy 0, policy_version 90580 (0.0007) [2023-10-14 00:42:19,026][60935] Updated weights for policy 0, policy_version 90590 (0.0007) [2023-10-14 00:42:19,194][60934] Updated weights for policy 1, policy_version 89402 (0.0010) [2023-10-14 00:42:19,606][60934] Updated weights for policy 1, policy_version 89412 (0.0008) [2023-10-14 00:42:19,970][60934] Updated weights for policy 1, policy_version 89422 (0.0007) [2023-10-14 00:42:20,334][60934] Updated weights for policy 1, policy_version 89432 (0.0008) [2023-10-14 00:42:21,248][59943] Fps is (10 sec: 13107.8, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 186351616. Throughput: 0: 1682.4, 1: 1735.4. Samples: 46592734. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-14 00:42:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:42:22,951][60935] Updated weights for policy 0, policy_version 90600 (0.0008) [2023-10-14 00:42:23,319][60935] Updated weights for policy 0, policy_version 90610 (0.0007) [2023-10-14 00:42:23,679][60935] Updated weights for policy 0, policy_version 90620 (0.0008) [2023-10-14 00:42:24,315][60934] Updated weights for policy 1, policy_version 89442 (0.0008) [2023-10-14 00:42:24,681][60934] Updated weights for policy 1, policy_version 89452 (0.0008) [2023-10-14 00:42:25,049][60934] Updated weights for policy 1, policy_version 89462 (0.0007) [2023-10-14 00:42:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 186417152. Throughput: 0: 1702.6, 1: 1720.0. Samples: 46613046. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-14 00:42:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:42:27,594][60935] Updated weights for policy 0, policy_version 90630 (0.0008) [2023-10-14 00:42:27,958][60935] Updated weights for policy 0, policy_version 90640 (0.0008) [2023-10-14 00:42:28,323][60935] Updated weights for policy 0, policy_version 90650 (0.0010) [2023-10-14 00:42:29,055][60934] Updated weights for policy 1, policy_version 89472 (0.0010) [2023-10-14 00:42:29,345][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:42:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 186482688. Throughput: 0: 1670.5, 1: 1749.2. Samples: 46623584. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-14 00:42:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:42:32,322][60935] Updated weights for policy 0, policy_version 90660 (0.0009) [2023-10-14 00:42:32,694][60935] Updated weights for policy 0, policy_version 90670 (0.0008) [2023-10-14 00:42:33,067][60935] Updated weights for policy 0, policy_version 90680 (0.0008) [2023-10-14 00:42:33,217][60934] Updated weights for policy 1, policy_version 89482 (0.0010) [2023-10-14 00:42:33,587][60934] Updated weights for policy 1, policy_version 89492 (0.0008) [2023-10-14 00:42:33,724][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:42:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 186548224. Throughput: 0: 1701.0, 1: 1747.8. Samples: 46644764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:42:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:42:37,125][60935] Updated weights for policy 0, policy_version 90690 (0.0009) [2023-10-14 00:42:37,491][60935] Updated weights for policy 0, policy_version 90700 (0.0012) [2023-10-14 00:42:37,858][60935] Updated weights for policy 0, policy_version 90710 (0.0011) [2023-10-14 00:42:37,945][60934] Updated weights for policy 1, policy_version 89502 (0.0009) [2023-10-14 00:42:38,226][60935] Updated weights for policy 0, policy_version 90720 (0.0009) [2023-10-14 00:42:38,309][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:42:38,310][60934] Updated weights for policy 1, policy_version 89512 (0.0009) [2023-10-14 00:42:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 186613760. Throughput: 0: 1705.6, 1: 1755.4. Samples: 46666456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:42:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:42:42,316][60935] Updated weights for policy 0, policy_version 90730 (0.0009) [2023-10-14 00:42:42,539][60934] Updated weights for policy 1, policy_version 89522 (0.0008) [2023-10-14 00:42:42,679][60935] Updated weights for policy 0, policy_version 90740 (0.0008) [2023-10-14 00:42:42,903][60934] Updated weights for policy 1, policy_version 89532 (0.0008) [2023-10-14 00:42:43,057][60935] Updated weights for policy 0, policy_version 90750 (0.0009) [2023-10-14 00:42:43,276][60934] Updated weights for policy 1, policy_version 89542 (0.0010) [2023-10-14 00:42:46,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 186679296. Throughput: 0: 1683.3, 1: 1755.4. Samples: 46675570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:42:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:42:46,994][60935] Updated weights for policy 0, policy_version 90760 (0.0009) [2023-10-14 00:42:47,330][60934] Updated weights for policy 1, policy_version 89552 (0.0007) [2023-10-14 00:42:47,352][60935] Updated weights for policy 0, policy_version 90770 (0.0008) [2023-10-14 00:42:47,687][60934] Updated weights for policy 1, policy_version 89562 (0.0007) [2023-10-14 00:42:47,722][60935] Updated weights for policy 0, policy_version 90780 (0.0007) [2023-10-14 00:42:48,049][60934] Updated weights for policy 1, policy_version 89572 (0.0009) [2023-10-14 00:42:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 186744832. Throughput: 0: 1710.4, 1: 1744.7. Samples: 46696872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:42:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:42:51,855][60935] Updated weights for policy 0, policy_version 90790 (0.0009) [2023-10-14 00:42:52,015][60934] Updated weights for policy 1, policy_version 89582 (0.0009) [2023-10-14 00:42:52,222][60935] Updated weights for policy 0, policy_version 90800 (0.0008) [2023-10-14 00:42:52,380][60934] Updated weights for policy 1, policy_version 89592 (0.0008) [2023-10-14 00:42:52,593][60935] Updated weights for policy 0, policy_version 90810 (0.0008) [2023-10-14 00:42:52,743][60934] Updated weights for policy 1, policy_version 89602 (0.0007) [2023-10-14 00:42:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 186810368. Throughput: 0: 1701.9, 1: 1767.1. Samples: 46717802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:42:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:42:56,255][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000089608_93814784.pth... [2023-10-14 00:42:56,255][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000090816_92995584.pth... [2023-10-14 00:42:56,290][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000088200_92176384.pth [2023-10-14 00:42:56,297][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000089248_91389952.pth [2023-10-14 00:42:56,583][60935] Updated weights for policy 0, policy_version 90820 (0.0007) [2023-10-14 00:42:56,681][60934] Updated weights for policy 1, policy_version 89612 (0.0007) [2023-10-14 00:42:56,947][60935] Updated weights for policy 0, policy_version 90830 (0.0009) [2023-10-14 00:42:57,039][60934] Updated weights for policy 1, policy_version 89622 (0.0007) [2023-10-14 00:42:57,312][60935] Updated weights for policy 0, policy_version 90840 (0.0008) [2023-10-14 00:42:57,395][60934] Updated weights for policy 1, policy_version 89632 (0.0008) [2023-10-14 00:43:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 186875904. Throughput: 0: 1692.2, 1: 1744.4. Samples: 46727076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:43:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:43:01,299][60934] Updated weights for policy 1, policy_version 89642 (0.0008) [2023-10-14 00:43:01,394][60935] Updated weights for policy 0, policy_version 90850 (0.0008) [2023-10-14 00:43:01,679][60934] Updated weights for policy 1, policy_version 89652 (0.0008) [2023-10-14 00:43:01,774][60935] Updated weights for policy 0, policy_version 90860 (0.0007) [2023-10-14 00:43:01,816][60828] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-10-14 00:43:02,136][60935] Updated weights for policy 0, policy_version 90870 (0.0009) [2023-10-14 00:43:02,500][60935] Updated weights for policy 0, policy_version 90880 (0.0010) [2023-10-14 00:43:05,945][60934] Updated weights for policy 1, policy_version 89662 (0.0009) [2023-10-14 00:43:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 186941440. Throughput: 0: 1701.0, 1: 1766.2. Samples: 46748758. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:43:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:43:06,310][60934] Updated weights for policy 1, policy_version 89672 (0.0008) [2023-10-14 00:43:06,514][60935] Updated weights for policy 0, policy_version 90890 (0.0007) [2023-10-14 00:43:06,674][60934] Updated weights for policy 1, policy_version 89682 (0.0008) [2023-10-14 00:43:06,874][60935] Updated weights for policy 0, policy_version 90900 (0.0009) [2023-10-14 00:43:07,249][60935] Updated weights for policy 0, policy_version 90910 (0.0008) [2023-10-14 00:43:10,656][60934] Updated weights for policy 1, policy_version 89692 (0.0007) [2023-10-14 00:43:11,015][60934] Updated weights for policy 1, policy_version 89702 (0.0008) [2023-10-14 00:43:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13662.6). Total num frames: 187006976. Throughput: 0: 1701.5, 1: 1782.2. Samples: 46769814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:43:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:43:11,278][60935] Updated weights for policy 0, policy_version 90920 (0.0009) [2023-10-14 00:43:11,384][60934] Updated weights for policy 1, policy_version 89712 (0.0007) [2023-10-14 00:43:11,643][60935] Updated weights for policy 0, policy_version 90930 (0.0010) [2023-10-14 00:43:12,013][60935] Updated weights for policy 0, policy_version 90940 (0.0011) [2023-10-14 00:43:15,303][60934] Updated weights for policy 1, policy_version 89722 (0.0008) [2023-10-14 00:43:15,664][60934] Updated weights for policy 1, policy_version 89732 (0.0007) [2023-10-14 00:43:16,028][60934] Updated weights for policy 1, policy_version 89742 (0.0008) [2023-10-14 00:43:16,157][60935] Updated weights for policy 0, policy_version 90950 (0.0010) [2023-10-14 00:43:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 187072512. Throughput: 0: 1699.8, 1: 1754.1. Samples: 46779010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:43:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:43:16,395][60934] Updated weights for policy 1, policy_version 89752 (0.0008) [2023-10-14 00:43:16,523][60935] Updated weights for policy 0, policy_version 90960 (0.0009) [2023-10-14 00:43:16,894][60935] Updated weights for policy 0, policy_version 90970 (0.0011) [2023-10-14 00:43:20,389][60934] Updated weights for policy 1, policy_version 89762 (0.0009) [2023-10-14 00:43:20,749][60934] Updated weights for policy 1, policy_version 89772 (0.0007) [2023-10-14 00:43:20,935][60935] Updated weights for policy 0, policy_version 90980 (0.0009) [2023-10-14 00:43:21,107][60934] Updated weights for policy 1, policy_version 89782 (0.0007) [2023-10-14 00:43:21,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 187170816. Throughput: 0: 1691.2, 1: 1757.6. Samples: 46799956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:43:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:43:21,291][60935] Updated weights for policy 0, policy_version 90990 (0.0009) [2023-10-14 00:43:21,656][60935] Updated weights for policy 0, policy_version 91000 (0.0012) [2023-10-14 00:43:25,246][60934] Updated weights for policy 1, policy_version 89792 (0.0007) [2023-10-14 00:43:25,614][60934] Updated weights for policy 1, policy_version 89802 (0.0009) [2023-10-14 00:43:25,803][60935] Updated weights for policy 0, policy_version 91010 (0.0010) [2023-10-14 00:43:25,982][60934] Updated weights for policy 1, policy_version 89812 (0.0007) [2023-10-14 00:43:26,161][60935] Updated weights for policy 0, policy_version 91020 (0.0008) [2023-10-14 00:43:26,248][59943] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 187236352. Throughput: 0: 1689.2, 1: 1725.0. Samples: 46820094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:43:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:43:26,536][60935] Updated weights for policy 0, policy_version 91030 (0.0008) [2023-10-14 00:43:26,901][60935] Updated weights for policy 0, policy_version 91040 (0.0009) [2023-10-14 00:43:29,880][60934] Updated weights for policy 1, policy_version 89822 (0.0008) [2023-10-14 00:43:30,242][60934] Updated weights for policy 1, policy_version 89832 (0.0009) [2023-10-14 00:43:30,607][60934] Updated weights for policy 1, policy_version 89842 (0.0009) [2023-10-14 00:43:30,934][60935] Updated weights for policy 0, policy_version 91050 (0.0009) [2023-10-14 00:43:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 187301888. Throughput: 0: 1691.8, 1: 1741.8. Samples: 46830082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:43:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:43:31,302][60935] Updated weights for policy 0, policy_version 91060 (0.0007) [2023-10-14 00:43:31,679][60935] Updated weights for policy 0, policy_version 91070 (0.0008) [2023-10-14 00:43:34,762][60934] Updated weights for policy 1, policy_version 89852 (0.0008) [2023-10-14 00:43:35,134][60934] Updated weights for policy 1, policy_version 89862 (0.0009) [2023-10-14 00:43:35,493][60934] Updated weights for policy 1, policy_version 89872 (0.0008) [2023-10-14 00:43:35,861][60935] Updated weights for policy 0, policy_version 91080 (0.0010) [2023-10-14 00:43:36,227][60935] Updated weights for policy 0, policy_version 91090 (0.0010) [2023-10-14 00:43:36,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.4, 300 sec: 13773.7). Total num frames: 187367424. Throughput: 0: 1686.3, 1: 1736.0. Samples: 46850878. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-14 00:43:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:43:36,594][60935] Updated weights for policy 0, policy_version 91100 (0.0011) [2023-10-14 00:43:39,368][60934] Updated weights for policy 1, policy_version 89882 (0.0009) [2023-10-14 00:43:39,727][60934] Updated weights for policy 1, policy_version 89892 (0.0010) [2023-10-14 00:43:40,098][60934] Updated weights for policy 1, policy_version 89902 (0.0007) [2023-10-14 00:43:40,461][60934] Updated weights for policy 1, policy_version 89912 (0.0009) [2023-10-14 00:43:40,695][60935] Updated weights for policy 0, policy_version 91110 (0.0009) [2023-10-14 00:43:41,063][60935] Updated weights for policy 0, policy_version 91120 (0.0008) [2023-10-14 00:43:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 187432960. Throughput: 0: 1682.4, 1: 1706.3. Samples: 46870296. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-14 00:43:41,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:43:41,428][60935] Updated weights for policy 0, policy_version 91130 (0.0008) [2023-10-14 00:43:44,533][60934] Updated weights for policy 1, policy_version 89922 (0.0007) [2023-10-14 00:43:44,911][60934] Updated weights for policy 1, policy_version 89932 (0.0007) [2023-10-14 00:43:45,270][60934] Updated weights for policy 1, policy_version 89942 (0.0007) [2023-10-14 00:43:45,394][60935] Updated weights for policy 0, policy_version 91140 (0.0009) [2023-10-14 00:43:45,772][60935] Updated weights for policy 0, policy_version 91150 (0.0009) [2023-10-14 00:43:46,137][60935] Updated weights for policy 0, policy_version 91160 (0.0010) [2023-10-14 00:43:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 187498496. Throughput: 0: 1690.0, 1: 1732.1. Samples: 46881068. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-14 00:43:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:43:49,345][60934] Updated weights for policy 1, policy_version 89952 (0.0009) [2023-10-14 00:43:49,709][60934] Updated weights for policy 1, policy_version 89962 (0.0009) [2023-10-14 00:43:50,076][60934] Updated weights for policy 1, policy_version 89972 (0.0009) [2023-10-14 00:43:50,299][60935] Updated weights for policy 0, policy_version 91170 (0.0009) [2023-10-14 00:43:50,672][60935] Updated weights for policy 0, policy_version 91180 (0.0009) [2023-10-14 00:43:51,045][60935] Updated weights for policy 0, policy_version 91190 (0.0010) [2023-10-14 00:43:51,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13773.7). Total num frames: 187564032. Throughput: 0: 1695.6, 1: 1702.9. Samples: 46901692. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-14 00:43:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:43:51,422][60935] Updated weights for policy 0, policy_version 91200 (0.0008) [2023-10-14 00:43:54,017][60934] Updated weights for policy 1, policy_version 89982 (0.0008) [2023-10-14 00:43:54,400][60934] Updated weights for policy 1, policy_version 89992 (0.0009) [2023-10-14 00:43:54,762][60934] Updated weights for policy 1, policy_version 90002 (0.0009) [2023-10-14 00:43:55,218][60935] Updated weights for policy 0, policy_version 91210 (0.0008) [2023-10-14 00:43:55,593][60935] Updated weights for policy 0, policy_version 91220 (0.0009) [2023-10-14 00:43:55,954][60935] Updated weights for policy 0, policy_version 91230 (0.0011) [2023-10-14 00:43:56,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 187662336. Throughput: 0: 1667.2, 1: 1685.4. Samples: 46920680. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-14 00:43:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:43:58,831][60934] Updated weights for policy 1, policy_version 90012 (0.0008) [2023-10-14 00:43:59,200][60934] Updated weights for policy 1, policy_version 90022 (0.0010) [2023-10-14 00:43:59,560][60934] Updated weights for policy 1, policy_version 90032 (0.0009) [2023-10-14 00:44:00,102][60935] Updated weights for policy 0, policy_version 91240 (0.0010) [2023-10-14 00:44:00,474][60935] Updated weights for policy 0, policy_version 91250 (0.0009) [2023-10-14 00:44:00,839][60935] Updated weights for policy 0, policy_version 91260 (0.0009) [2023-10-14 00:44:01,248][59943] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 187727872. Throughput: 0: 1690.8, 1: 1710.3. Samples: 46932060. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-14 00:44:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:44:03,491][60934] Updated weights for policy 1, policy_version 90042 (0.0010) [2023-10-14 00:44:03,851][60934] Updated weights for policy 1, policy_version 90052 (0.0008) [2023-10-14 00:44:04,220][60934] Updated weights for policy 1, policy_version 90062 (0.0008) [2023-10-14 00:44:04,584][60934] Updated weights for policy 1, policy_version 90072 (0.0009) [2023-10-14 00:44:04,887][60935] Updated weights for policy 0, policy_version 91270 (0.0008) [2023-10-14 00:44:05,250][60935] Updated weights for policy 0, policy_version 91280 (0.0009) [2023-10-14 00:44:05,617][60935] Updated weights for policy 0, policy_version 91290 (0.0008) [2023-10-14 00:44:06,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13884.7). Total num frames: 187793408. Throughput: 0: 1693.1, 1: 1682.2. Samples: 46951848. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-14 00:44:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:44:08,427][60934] Updated weights for policy 1, policy_version 90082 (0.0008) [2023-10-14 00:44:08,794][60934] Updated weights for policy 1, policy_version 90092 (0.0010) [2023-10-14 00:44:09,154][60934] Updated weights for policy 1, policy_version 90102 (0.0007) [2023-10-14 00:44:09,532][60935] Updated weights for policy 0, policy_version 91300 (0.0009) [2023-10-14 00:44:09,902][60935] Updated weights for policy 0, policy_version 91310 (0.0009) [2023-10-14 00:44:10,275][60935] Updated weights for policy 0, policy_version 91320 (0.0008) [2023-10-14 00:44:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 187858944. Throughput: 0: 1673.0, 1: 1703.9. Samples: 46972052. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-14 00:44:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:44:13,116][60934] Updated weights for policy 1, policy_version 90112 (0.0010) [2023-10-14 00:44:13,488][60934] Updated weights for policy 1, policy_version 90122 (0.0008) [2023-10-14 00:44:13,849][60934] Updated weights for policy 1, policy_version 90132 (0.0008) [2023-10-14 00:44:14,248][60935] Updated weights for policy 0, policy_version 91330 (0.0008) [2023-10-14 00:44:14,620][60935] Updated weights for policy 0, policy_version 91340 (0.0007) [2023-10-14 00:44:14,986][60935] Updated weights for policy 0, policy_version 91350 (0.0007) [2023-10-14 00:44:15,344][60935] Updated weights for policy 0, policy_version 91360 (0.0009) [2023-10-14 00:44:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 187924480. Throughput: 0: 1704.5, 1: 1699.6. Samples: 46983270. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-14 00:44:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:44:17,836][60934] Updated weights for policy 1, policy_version 90142 (0.0009) [2023-10-14 00:44:18,193][60934] Updated weights for policy 1, policy_version 90152 (0.0010) [2023-10-14 00:44:18,553][60934] Updated weights for policy 1, policy_version 90162 (0.0010) [2023-10-14 00:44:19,292][60935] Updated weights for policy 0, policy_version 91370 (0.0010) [2023-10-14 00:44:19,664][60935] Updated weights for policy 0, policy_version 91380 (0.0008) [2023-10-14 00:44:20,043][60935] Updated weights for policy 0, policy_version 91390 (0.0007) [2023-10-14 00:44:21,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 187990016. Throughput: 0: 1688.0, 1: 1694.0. Samples: 47003066. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-14 00:44:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:44:22,442][60934] Updated weights for policy 1, policy_version 90172 (0.0010) [2023-10-14 00:44:22,806][60934] Updated weights for policy 1, policy_version 90182 (0.0010) [2023-10-14 00:44:23,160][60934] Updated weights for policy 1, policy_version 90192 (0.0008) [2023-10-14 00:44:24,000][60935] Updated weights for policy 0, policy_version 91400 (0.0007) [2023-10-14 00:44:24,373][60935] Updated weights for policy 0, policy_version 91410 (0.0008) [2023-10-14 00:44:24,737][60935] Updated weights for policy 0, policy_version 91420 (0.0011) [2023-10-14 00:44:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 188055552. Throughput: 0: 1687.7, 1: 1721.3. Samples: 47023698. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-14 00:44:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:44:27,194][60934] Updated weights for policy 1, policy_version 90202 (0.0008) [2023-10-14 00:44:27,562][60934] Updated weights for policy 1, policy_version 90212 (0.0008) [2023-10-14 00:44:27,932][60934] Updated weights for policy 1, policy_version 90222 (0.0008) [2023-10-14 00:44:28,295][60934] Updated weights for policy 1, policy_version 90232 (0.0008) [2023-10-14 00:44:28,950][60935] Updated weights for policy 0, policy_version 91430 (0.0009) [2023-10-14 00:44:29,326][60935] Updated weights for policy 0, policy_version 91440 (0.0008) [2023-10-14 00:44:29,686][60935] Updated weights for policy 0, policy_version 91450 (0.0010) [2023-10-14 00:44:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 188121088. Throughput: 0: 1707.7, 1: 1691.8. Samples: 47034046. Policy #0 lag: (min: 10.0, avg: 15.9, max: 42.0) [2023-10-14 00:44:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:44:32,337][60934] Updated weights for policy 1, policy_version 90242 (0.0008) [2023-10-14 00:44:32,705][60934] Updated weights for policy 1, policy_version 90252 (0.0010) [2023-10-14 00:44:33,072][60934] Updated weights for policy 1, policy_version 90262 (0.0007) [2023-10-14 00:44:33,806][60935] Updated weights for policy 0, policy_version 91460 (0.0009) [2023-10-14 00:44:34,181][60935] Updated weights for policy 0, policy_version 91470 (0.0009) [2023-10-14 00:44:34,548][60935] Updated weights for policy 0, policy_version 91480 (0.0009) [2023-10-14 00:44:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 188186624. Throughput: 0: 1676.8, 1: 1706.7. Samples: 47053948. Policy #0 lag: (min: 10.0, avg: 15.9, max: 42.0) [2023-10-14 00:44:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:44:36,937][60934] Updated weights for policy 1, policy_version 90272 (0.0009) [2023-10-14 00:44:37,309][60934] Updated weights for policy 1, policy_version 90282 (0.0010) [2023-10-14 00:44:37,671][60934] Updated weights for policy 1, policy_version 90292 (0.0008) [2023-10-14 00:44:38,516][60935] Updated weights for policy 0, policy_version 91490 (0.0009) [2023-10-14 00:44:38,906][60935] Updated weights for policy 0, policy_version 91500 (0.0012) [2023-10-14 00:44:39,278][60935] Updated weights for policy 0, policy_version 91510 (0.0009) [2023-10-14 00:44:39,651][60935] Updated weights for policy 0, policy_version 91520 (0.0007) [2023-10-14 00:44:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 188252160. Throughput: 0: 1699.2, 1: 1731.3. Samples: 47075052. Policy #0 lag: (min: 10.0, avg: 15.9, max: 42.0) [2023-10-14 00:44:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:44:41,661][60934] Updated weights for policy 1, policy_version 90302 (0.0008) [2023-10-14 00:44:42,046][60934] Updated weights for policy 1, policy_version 90312 (0.0008) [2023-10-14 00:44:42,409][60934] Updated weights for policy 1, policy_version 90322 (0.0011) [2023-10-14 00:44:43,749][60935] Updated weights for policy 0, policy_version 91530 (0.0008) [2023-10-14 00:44:44,114][60935] Updated weights for policy 0, policy_version 91540 (0.0008) [2023-10-14 00:44:44,477][60935] Updated weights for policy 0, policy_version 91550 (0.0009) [2023-10-14 00:44:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 188317696. Throughput: 0: 1694.7, 1: 1701.1. Samples: 47084872. Policy #0 lag: (min: 10.0, avg: 15.9, max: 42.0) [2023-10-14 00:44:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:44:46,437][60934] Updated weights for policy 1, policy_version 90332 (0.0008) [2023-10-14 00:44:46,802][60934] Updated weights for policy 1, policy_version 90342 (0.0011) [2023-10-14 00:44:47,170][60934] Updated weights for policy 1, policy_version 90352 (0.0010) [2023-10-14 00:44:48,404][60935] Updated weights for policy 0, policy_version 91560 (0.0009) [2023-10-14 00:44:48,773][60935] Updated weights for policy 0, policy_version 91570 (0.0009) [2023-10-14 00:44:49,139][60935] Updated weights for policy 0, policy_version 91580 (0.0008) [2023-10-14 00:44:51,123][60934] Updated weights for policy 1, policy_version 90362 (0.0009) [2023-10-14 00:44:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 188383232. Throughput: 0: 1676.1, 1: 1731.0. Samples: 47105170. Policy #0 lag: (min: 10.0, avg: 15.9, max: 42.0) [2023-10-14 00:44:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:44:51,484][60934] Updated weights for policy 1, policy_version 90372 (0.0008) [2023-10-14 00:44:51,849][60934] Updated weights for policy 1, policy_version 90382 (0.0010) [2023-10-14 00:44:52,223][60934] Updated weights for policy 1, policy_version 90392 (0.0008) [2023-10-14 00:44:53,305][60935] Updated weights for policy 0, policy_version 91590 (0.0008) [2023-10-14 00:44:53,677][60935] Updated weights for policy 0, policy_version 91600 (0.0008) [2023-10-14 00:44:54,037][60935] Updated weights for policy 0, policy_version 91610 (0.0008) [2023-10-14 00:44:55,995][60934] Updated weights for policy 1, policy_version 90402 (0.0010) [2023-10-14 00:44:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 188448768. Throughput: 0: 1699.6, 1: 1741.8. Samples: 47126912. Policy #0 lag: (min: 10.0, avg: 15.9, max: 42.0) [2023-10-14 00:44:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:44:56,256][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000091616_93814784.pth... [2023-10-14 00:44:56,296][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000090048_92209152.pth [2023-10-14 00:44:56,361][60934] Updated weights for policy 1, policy_version 90412 (0.0009) [2023-10-14 00:44:56,732][60934] Updated weights for policy 1, policy_version 90422 (0.0009) [2023-10-14 00:44:56,805][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000090424_94666752.pth... [2023-10-14 00:44:56,834][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000088872_92995584.pth [2023-10-14 00:44:58,019][60935] Updated weights for policy 0, policy_version 91620 (0.0007) [2023-10-14 00:44:58,385][60935] Updated weights for policy 0, policy_version 91630 (0.0009) [2023-10-14 00:44:58,753][60935] Updated weights for policy 0, policy_version 91640 (0.0009) [2023-10-14 00:45:00,784][60934] Updated weights for policy 1, policy_version 90432 (0.0009) [2023-10-14 00:45:01,149][60934] Updated weights for policy 1, policy_version 90442 (0.0007) [2023-10-14 00:45:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 188514304. Throughput: 0: 1670.9, 1: 1730.5. Samples: 47136330. Policy #0 lag: (min: 10.0, avg: 15.9, max: 42.0) [2023-10-14 00:45:01,248][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:45:01,510][60934] Updated weights for policy 1, policy_version 90452 (0.0008) [2023-10-14 00:45:02,733][60935] Updated weights for policy 0, policy_version 91650 (0.0009) [2023-10-14 00:45:03,098][60935] Updated weights for policy 0, policy_version 91660 (0.0009) [2023-10-14 00:45:03,464][60935] Updated weights for policy 0, policy_version 91670 (0.0009) [2023-10-14 00:45:03,829][60935] Updated weights for policy 0, policy_version 91680 (0.0009) [2023-10-14 00:45:05,588][60934] Updated weights for policy 1, policy_version 90462 (0.0009) [2023-10-14 00:45:05,952][60934] Updated weights for policy 1, policy_version 90472 (0.0007) [2023-10-14 00:45:06,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 188579840. Throughput: 0: 1686.6, 1: 1736.4. Samples: 47157102. Policy #0 lag: (min: 10.0, avg: 15.9, max: 42.0) [2023-10-14 00:45:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:45:06,325][60934] Updated weights for policy 1, policy_version 90482 (0.0007) [2023-10-14 00:45:07,842][60935] Updated weights for policy 0, policy_version 91690 (0.0008) [2023-10-14 00:45:08,207][60935] Updated weights for policy 0, policy_version 91700 (0.0009) [2023-10-14 00:45:08,569][60935] Updated weights for policy 0, policy_version 91710 (0.0008) [2023-10-14 00:45:10,347][60934] Updated weights for policy 1, policy_version 90492 (0.0009) [2023-10-14 00:45:10,720][60934] Updated weights for policy 1, policy_version 90502 (0.0010) [2023-10-14 00:45:11,084][60934] Updated weights for policy 1, policy_version 90512 (0.0009) [2023-10-14 00:45:11,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13551.5). Total num frames: 188645376. Throughput: 0: 1696.7, 1: 1721.3. Samples: 47177512. Policy #0 lag: (min: 10.0, avg: 15.9, max: 42.0) [2023-10-14 00:45:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:45:12,640][60935] Updated weights for policy 0, policy_version 91720 (0.0010) [2023-10-14 00:45:13,004][60935] Updated weights for policy 0, policy_version 91730 (0.0012) [2023-10-14 00:45:13,382][60935] Updated weights for policy 0, policy_version 91740 (0.0011) [2023-10-14 00:45:15,005][60934] Updated weights for policy 1, policy_version 90522 (0.0008) [2023-10-14 00:45:15,371][60934] Updated weights for policy 1, policy_version 90532 (0.0007) [2023-10-14 00:45:15,732][60934] Updated weights for policy 1, policy_version 90542 (0.0008) [2023-10-14 00:45:16,097][60934] Updated weights for policy 1, policy_version 90552 (0.0008) [2023-10-14 00:45:16,248][59943] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 188743680. Throughput: 0: 1670.5, 1: 1733.2. Samples: 47187210. Policy #0 lag: (min: 10.0, avg: 15.9, max: 42.0) [2023-10-14 00:45:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:45:17,398][60935] Updated weights for policy 0, policy_version 91750 (0.0008) [2023-10-14 00:45:17,762][60935] Updated weights for policy 0, policy_version 91760 (0.0008) [2023-10-14 00:45:18,132][60935] Updated weights for policy 0, policy_version 91770 (0.0007) [2023-10-14 00:45:20,128][60934] Updated weights for policy 1, policy_version 90562 (0.0007) [2023-10-14 00:45:20,487][60934] Updated weights for policy 1, policy_version 90572 (0.0008) [2023-10-14 00:45:20,853][60934] Updated weights for policy 1, policy_version 90582 (0.0009) [2023-10-14 00:45:21,248][59943] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 188809216. Throughput: 0: 1699.5, 1: 1728.5. Samples: 47208204. Policy #0 lag: (min: 10.0, avg: 15.9, max: 42.0) [2023-10-14 00:45:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:45:21,992][60935] Updated weights for policy 0, policy_version 91780 (0.0008) [2023-10-14 00:45:22,359][60935] Updated weights for policy 0, policy_version 91790 (0.0010) [2023-10-14 00:45:22,736][60935] Updated weights for policy 0, policy_version 91800 (0.0011) [2023-10-14 00:45:24,864][60934] Updated weights for policy 1, policy_version 90592 (0.0008) [2023-10-14 00:45:25,236][60934] Updated weights for policy 1, policy_version 90602 (0.0008) [2023-10-14 00:45:25,607][60934] Updated weights for policy 1, policy_version 90612 (0.0010) [2023-10-14 00:45:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 188874752. Throughput: 0: 1705.6, 1: 1699.8. Samples: 47228298. Policy #0 lag: (min: 10.0, avg: 15.9, max: 42.0) [2023-10-14 00:45:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:45:26,746][60935] Updated weights for policy 0, policy_version 91810 (0.0008) [2023-10-14 00:45:27,122][60935] Updated weights for policy 0, policy_version 91820 (0.0010) [2023-10-14 00:45:27,488][60935] Updated weights for policy 0, policy_version 91830 (0.0009) [2023-10-14 00:45:27,858][60935] Updated weights for policy 0, policy_version 91840 (0.0007) [2023-10-14 00:45:29,636][60934] Updated weights for policy 1, policy_version 90622 (0.0007) [2023-10-14 00:45:30,009][60934] Updated weights for policy 1, policy_version 90632 (0.0007) [2023-10-14 00:45:30,375][60934] Updated weights for policy 1, policy_version 90642 (0.0008) [2023-10-14 00:45:31,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 188940288. Throughput: 0: 1686.8, 1: 1723.7. Samples: 47238342. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-14 00:45:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:45:32,057][60935] Updated weights for policy 0, policy_version 91850 (0.0008) [2023-10-14 00:45:32,417][60935] Updated weights for policy 0, policy_version 91860 (0.0008) [2023-10-14 00:45:32,786][60935] Updated weights for policy 0, policy_version 91870 (0.0008) [2023-10-14 00:45:34,338][60934] Updated weights for policy 1, policy_version 90652 (0.0009) [2023-10-14 00:45:34,702][60934] Updated weights for policy 1, policy_version 90662 (0.0011) [2023-10-14 00:45:35,067][60934] Updated weights for policy 1, policy_version 90672 (0.0008) [2023-10-14 00:45:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 189005824. Throughput: 0: 1709.9, 1: 1708.0. Samples: 47258976. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-14 00:45:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:45:36,809][60935] Updated weights for policy 0, policy_version 91880 (0.0007) [2023-10-14 00:45:37,172][60935] Updated weights for policy 0, policy_version 91890 (0.0010) [2023-10-14 00:45:37,536][60935] Updated weights for policy 0, policy_version 91900 (0.0010) [2023-10-14 00:45:38,968][60934] Updated weights for policy 1, policy_version 90682 (0.0010) [2023-10-14 00:45:39,339][60934] Updated weights for policy 1, policy_version 90692 (0.0009) [2023-10-14 00:45:39,711][60934] Updated weights for policy 1, policy_version 90702 (0.0008) [2023-10-14 00:45:40,082][60934] Updated weights for policy 1, policy_version 90712 (0.0008) [2023-10-14 00:45:41,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 189071360. Throughput: 0: 1709.7, 1: 1672.0. Samples: 47279090. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-14 00:45:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:45:41,549][60935] Updated weights for policy 0, policy_version 91910 (0.0008) [2023-10-14 00:45:41,922][60935] Updated weights for policy 0, policy_version 91920 (0.0008) [2023-10-14 00:45:42,290][60935] Updated weights for policy 0, policy_version 91930 (0.0010) [2023-10-14 00:45:44,069][60934] Updated weights for policy 1, policy_version 90722 (0.0007) [2023-10-14 00:45:44,430][60934] Updated weights for policy 1, policy_version 90732 (0.0009) [2023-10-14 00:45:44,790][60934] Updated weights for policy 1, policy_version 90742 (0.0010) [2023-10-14 00:45:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 189136896. Throughput: 0: 1703.9, 1: 1701.4. Samples: 47289568. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-14 00:45:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:45:46,251][60935] Updated weights for policy 0, policy_version 91940 (0.0010) [2023-10-14 00:45:46,624][60935] Updated weights for policy 0, policy_version 91950 (0.0009) [2023-10-14 00:45:46,989][60935] Updated weights for policy 0, policy_version 91960 (0.0007) [2023-10-14 00:45:48,797][60934] Updated weights for policy 1, policy_version 90752 (0.0008) [2023-10-14 00:45:49,167][60934] Updated weights for policy 1, policy_version 90762 (0.0008) [2023-10-14 00:45:49,525][60934] Updated weights for policy 1, policy_version 90772 (0.0007) [2023-10-14 00:45:50,940][60935] Updated weights for policy 0, policy_version 91970 (0.0008) [2023-10-14 00:45:51,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 189202432. Throughput: 0: 1712.0, 1: 1679.9. Samples: 47309736. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-14 00:45:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:45:51,310][60935] Updated weights for policy 0, policy_version 91980 (0.0008) [2023-10-14 00:45:51,675][60935] Updated weights for policy 0, policy_version 91990 (0.0007) [2023-10-14 00:45:52,026][60935] Updated weights for policy 0, policy_version 92000 (0.0008) [2023-10-14 00:45:53,565][60934] Updated weights for policy 1, policy_version 90782 (0.0007) [2023-10-14 00:45:53,932][60934] Updated weights for policy 1, policy_version 90792 (0.0008) [2023-10-14 00:45:54,301][60934] Updated weights for policy 1, policy_version 90802 (0.0008) [2023-10-14 00:45:55,977][60935] Updated weights for policy 0, policy_version 92010 (0.0007) [2023-10-14 00:45:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 189267968. Throughput: 0: 1705.4, 1: 1689.3. Samples: 47330274. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-14 00:45:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:45:56,354][60935] Updated weights for policy 0, policy_version 92020 (0.0007) [2023-10-14 00:45:56,731][60935] Updated weights for policy 0, policy_version 92030 (0.0009) [2023-10-14 00:45:58,216][60934] Updated weights for policy 1, policy_version 90812 (0.0009) [2023-10-14 00:45:58,575][60934] Updated weights for policy 1, policy_version 90822 (0.0009) [2023-10-14 00:45:58,936][60934] Updated weights for policy 1, policy_version 90832 (0.0008) [2023-10-14 00:46:00,812][60935] Updated weights for policy 0, policy_version 92040 (0.0008) [2023-10-14 00:46:01,181][60935] Updated weights for policy 0, policy_version 92050 (0.0007) [2023-10-14 00:46:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 189333504. Throughput: 0: 1708.2, 1: 1696.8. Samples: 47340436. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-14 00:46:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:46:01,548][60935] Updated weights for policy 0, policy_version 92060 (0.0008) [2023-10-14 00:46:02,844][60934] Updated weights for policy 1, policy_version 90842 (0.0008) [2023-10-14 00:46:03,206][60934] Updated weights for policy 1, policy_version 90852 (0.0009) [2023-10-14 00:46:03,579][60934] Updated weights for policy 1, policy_version 90862 (0.0010) [2023-10-14 00:46:03,945][60934] Updated weights for policy 1, policy_version 90872 (0.0008) [2023-10-14 00:46:05,588][60935] Updated weights for policy 0, policy_version 92070 (0.0009) [2023-10-14 00:46:05,964][60935] Updated weights for policy 0, policy_version 92080 (0.0009) [2023-10-14 00:46:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 189399040. Throughput: 0: 1703.8, 1: 1687.6. Samples: 47360820. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-14 00:46:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:46:06,337][60935] Updated weights for policy 0, policy_version 92090 (0.0009) [2023-10-14 00:46:07,810][60934] Updated weights for policy 1, policy_version 90882 (0.0007) [2023-10-14 00:46:08,182][60934] Updated weights for policy 1, policy_version 90892 (0.0007) [2023-10-14 00:46:08,559][60934] Updated weights for policy 1, policy_version 90902 (0.0007) [2023-10-14 00:46:10,409][60935] Updated weights for policy 0, policy_version 92100 (0.0011) [2023-10-14 00:46:10,779][60935] Updated weights for policy 0, policy_version 92110 (0.0010) [2023-10-14 00:46:11,147][60935] Updated weights for policy 0, policy_version 92120 (0.0009) [2023-10-14 00:46:11,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 189464576. Throughput: 0: 1688.1, 1: 1717.0. Samples: 47381530. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-14 00:46:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:46:12,445][60934] Updated weights for policy 1, policy_version 90912 (0.0007) [2023-10-14 00:46:12,816][60934] Updated weights for policy 1, policy_version 90922 (0.0009) [2023-10-14 00:46:13,186][60934] Updated weights for policy 1, policy_version 90932 (0.0010) [2023-10-14 00:46:15,178][60935] Updated weights for policy 0, policy_version 92130 (0.0009) [2023-10-14 00:46:15,598][60935] Updated weights for policy 0, policy_version 92140 (0.0008) [2023-10-14 00:46:15,977][60935] Updated weights for policy 0, policy_version 92150 (0.0010) [2023-10-14 00:46:16,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 189530112. Throughput: 0: 1707.5, 1: 1697.0. Samples: 47391544. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-14 00:46:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:46:16,347][60935] Updated weights for policy 0, policy_version 92160 (0.0008) [2023-10-14 00:46:17,308][60934] Updated weights for policy 1, policy_version 90942 (0.0008) [2023-10-14 00:46:17,688][60934] Updated weights for policy 1, policy_version 90952 (0.0007) [2023-10-14 00:46:18,056][60934] Updated weights for policy 1, policy_version 90962 (0.0010) [2023-10-14 00:46:20,275][60935] Updated weights for policy 0, policy_version 92170 (0.0008) [2023-10-14 00:46:20,638][60935] Updated weights for policy 0, policy_version 92180 (0.0008) [2023-10-14 00:46:21,011][60935] Updated weights for policy 0, policy_version 92190 (0.0007) [2023-10-14 00:46:21,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 189628416. Throughput: 0: 1705.8, 1: 1708.0. Samples: 47412598. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-14 00:46:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:46:22,007][60934] Updated weights for policy 1, policy_version 90972 (0.0010) [2023-10-14 00:46:22,377][60934] Updated weights for policy 1, policy_version 90982 (0.0007) [2023-10-14 00:46:22,740][60934] Updated weights for policy 1, policy_version 90992 (0.0007) [2023-10-14 00:46:24,917][60935] Updated weights for policy 0, policy_version 92200 (0.0009) [2023-10-14 00:46:25,279][60935] Updated weights for policy 0, policy_version 92210 (0.0010) [2023-10-14 00:46:25,660][60935] Updated weights for policy 0, policy_version 92220 (0.0010) [2023-10-14 00:46:26,248][59943] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 189693952. Throughput: 0: 1679.7, 1: 1729.9. Samples: 47432522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 00:46:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:46:26,649][60934] Updated weights for policy 1, policy_version 91002 (0.0008) [2023-10-14 00:46:27,008][60934] Updated weights for policy 1, policy_version 91012 (0.0009) [2023-10-14 00:46:27,370][60934] Updated weights for policy 1, policy_version 91022 (0.0011) [2023-10-14 00:46:27,737][60934] Updated weights for policy 1, policy_version 91032 (0.0008) [2023-10-14 00:46:29,746][60935] Updated weights for policy 0, policy_version 92230 (0.0008) [2023-10-14 00:46:30,115][60935] Updated weights for policy 0, policy_version 92240 (0.0008) [2023-10-14 00:46:30,476][60935] Updated weights for policy 0, policy_version 92250 (0.0009) [2023-10-14 00:46:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 189759488. Throughput: 0: 1707.6, 1: 1701.3. Samples: 47442970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 00:46:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:46:31,823][60934] Updated weights for policy 1, policy_version 91042 (0.0009) [2023-10-14 00:46:32,183][60934] Updated weights for policy 1, policy_version 91052 (0.0008) [2023-10-14 00:46:32,552][60934] Updated weights for policy 1, policy_version 91062 (0.0009) [2023-10-14 00:46:34,437][60935] Updated weights for policy 0, policy_version 92260 (0.0011) [2023-10-14 00:46:34,800][60935] Updated weights for policy 0, policy_version 92270 (0.0010) [2023-10-14 00:46:35,170][60935] Updated weights for policy 0, policy_version 92280 (0.0009) [2023-10-14 00:46:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 189825024. Throughput: 0: 1688.6, 1: 1722.1. Samples: 47463218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 00:46:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:46:36,519][60934] Updated weights for policy 1, policy_version 91072 (0.0009) [2023-10-14 00:46:36,875][60934] Updated weights for policy 1, policy_version 91082 (0.0010) [2023-10-14 00:46:37,234][60934] Updated weights for policy 1, policy_version 91092 (0.0009) [2023-10-14 00:46:39,256][60935] Updated weights for policy 0, policy_version 92290 (0.0009) [2023-10-14 00:46:39,617][60935] Updated weights for policy 0, policy_version 92300 (0.0007) [2023-10-14 00:46:39,982][60935] Updated weights for policy 0, policy_version 92310 (0.0008) [2023-10-14 00:46:40,355][60935] Updated weights for policy 0, policy_version 92320 (0.0010) [2023-10-14 00:46:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 189890560. Throughput: 0: 1680.8, 1: 1726.7. Samples: 47483614. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 00:46:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:46:41,322][60934] Updated weights for policy 1, policy_version 91102 (0.0008) [2023-10-14 00:46:41,685][60934] Updated weights for policy 1, policy_version 91112 (0.0008) [2023-10-14 00:46:42,045][60934] Updated weights for policy 1, policy_version 91122 (0.0008) [2023-10-14 00:46:44,216][60935] Updated weights for policy 0, policy_version 92330 (0.0009) [2023-10-14 00:46:44,578][60935] Updated weights for policy 0, policy_version 92340 (0.0008) [2023-10-14 00:46:44,939][60935] Updated weights for policy 0, policy_version 92350 (0.0008) [2023-10-14 00:46:46,197][60934] Updated weights for policy 1, policy_version 91132 (0.0008) [2023-10-14 00:46:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 189956096. Throughput: 0: 1711.7, 1: 1707.2. Samples: 47494288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 00:46:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:46:46,559][60934] Updated weights for policy 1, policy_version 91142 (0.0010) [2023-10-14 00:46:46,936][60934] Updated weights for policy 1, policy_version 91152 (0.0008) [2023-10-14 00:46:48,776][60935] Updated weights for policy 0, policy_version 92360 (0.0010) [2023-10-14 00:46:49,138][60935] Updated weights for policy 0, policy_version 92370 (0.0010) [2023-10-14 00:46:49,507][60935] Updated weights for policy 0, policy_version 92380 (0.0012) [2023-10-14 00:46:50,793][60934] Updated weights for policy 1, policy_version 91162 (0.0008) [2023-10-14 00:46:51,163][60934] Updated weights for policy 1, policy_version 91172 (0.0008) [2023-10-14 00:46:51,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 190021632. Throughput: 0: 1689.3, 1: 1725.0. Samples: 47514464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 00:46:51,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:46:51,538][60934] Updated weights for policy 1, policy_version 91182 (0.0007) [2023-10-14 00:46:51,896][60934] Updated weights for policy 1, policy_version 91192 (0.0007) [2023-10-14 00:46:53,534][60935] Updated weights for policy 0, policy_version 92390 (0.0007) [2023-10-14 00:46:53,897][60935] Updated weights for policy 0, policy_version 92400 (0.0007) [2023-10-14 00:46:54,274][60935] Updated weights for policy 0, policy_version 92410 (0.0008) [2023-10-14 00:46:55,882][60934] Updated weights for policy 1, policy_version 91202 (0.0009) [2023-10-14 00:46:56,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 190087168. Throughput: 0: 1705.3, 1: 1717.3. Samples: 47535548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 00:46:56,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:46:56,259][60934] Updated weights for policy 1, policy_version 91212 (0.0008) [2023-10-14 00:46:56,259][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000092416_94633984.pth... [2023-10-14 00:46:56,294][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000090816_92995584.pth [2023-10-14 00:46:56,627][60934] Updated weights for policy 1, policy_version 91222 (0.0007) [2023-10-14 00:46:56,695][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000091224_95485952.pth... [2023-10-14 00:46:56,724][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000089608_93814784.pth [2023-10-14 00:46:58,344][60935] Updated weights for policy 0, policy_version 92420 (0.0009) [2023-10-14 00:46:58,708][60935] Updated weights for policy 0, policy_version 92430 (0.0009) [2023-10-14 00:46:59,081][60935] Updated weights for policy 0, policy_version 92440 (0.0008) [2023-10-14 00:47:00,567][60934] Updated weights for policy 1, policy_version 91232 (0.0010) [2023-10-14 00:47:00,923][60934] Updated weights for policy 1, policy_version 91242 (0.0009) [2023-10-14 00:47:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 190152704. Throughput: 0: 1700.4, 1: 1720.1. Samples: 47545462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 00:47:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:47:01,288][60934] Updated weights for policy 1, policy_version 91252 (0.0011) [2023-10-14 00:47:03,038][60935] Updated weights for policy 0, policy_version 92450 (0.0007) [2023-10-14 00:47:03,413][60935] Updated weights for policy 0, policy_version 92460 (0.0007) [2023-10-14 00:47:03,769][60935] Updated weights for policy 0, policy_version 92470 (0.0009) [2023-10-14 00:47:04,139][60935] Updated weights for policy 0, policy_version 92480 (0.0010) [2023-10-14 00:47:05,352][60934] Updated weights for policy 1, policy_version 91262 (0.0009) [2023-10-14 00:47:05,725][60934] Updated weights for policy 1, policy_version 91272 (0.0009) [2023-10-14 00:47:06,099][60934] Updated weights for policy 1, policy_version 91282 (0.0009) [2023-10-14 00:47:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 190218240. Throughput: 0: 1689.4, 1: 1721.4. Samples: 47566084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 00:47:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:47:08,290][60935] Updated weights for policy 0, policy_version 92490 (0.0009) [2023-10-14 00:47:08,654][60935] Updated weights for policy 0, policy_version 92500 (0.0010) [2023-10-14 00:47:09,020][60935] Updated weights for policy 0, policy_version 92510 (0.0010) [2023-10-14 00:47:10,197][60934] Updated weights for policy 1, policy_version 91292 (0.0008) [2023-10-14 00:47:10,563][60934] Updated weights for policy 1, policy_version 91302 (0.0009) [2023-10-14 00:47:10,932][60934] Updated weights for policy 1, policy_version 91312 (0.0011) [2023-10-14 00:47:11,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 190316544. Throughput: 0: 1720.6, 1: 1700.5. Samples: 47586474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 00:47:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:47:12,943][60935] Updated weights for policy 0, policy_version 92520 (0.0007) [2023-10-14 00:47:13,323][60935] Updated weights for policy 0, policy_version 92530 (0.0007) [2023-10-14 00:47:13,687][60935] Updated weights for policy 0, policy_version 92540 (0.0007) [2023-10-14 00:47:14,979][60934] Updated weights for policy 1, policy_version 91322 (0.0008) [2023-10-14 00:47:15,355][60934] Updated weights for policy 1, policy_version 91332 (0.0009) [2023-10-14 00:47:15,723][60934] Updated weights for policy 1, policy_version 91342 (0.0007) [2023-10-14 00:47:16,079][60934] Updated weights for policy 1, policy_version 91352 (0.0008) [2023-10-14 00:47:16,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 190382080. Throughput: 0: 1693.2, 1: 1713.9. Samples: 47596288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 00:47:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:47:17,600][60935] Updated weights for policy 0, policy_version 92550 (0.0008) [2023-10-14 00:47:17,965][60935] Updated weights for policy 0, policy_version 92560 (0.0008) [2023-10-14 00:47:18,335][60935] Updated weights for policy 0, policy_version 92570 (0.0010) [2023-10-14 00:47:19,945][60934] Updated weights for policy 1, policy_version 91362 (0.0009) [2023-10-14 00:47:20,311][60934] Updated weights for policy 1, policy_version 91372 (0.0008) [2023-10-14 00:47:20,676][60934] Updated weights for policy 1, policy_version 91382 (0.0009) [2023-10-14 00:47:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 190447616. Throughput: 0: 1712.9, 1: 1721.0. Samples: 47617742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 00:47:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:47:22,281][60935] Updated weights for policy 0, policy_version 92580 (0.0008) [2023-10-14 00:47:22,657][60935] Updated weights for policy 0, policy_version 92590 (0.0008) [2023-10-14 00:47:23,029][60935] Updated weights for policy 0, policy_version 92600 (0.0008) [2023-10-14 00:47:24,521][60934] Updated weights for policy 1, policy_version 91392 (0.0009) [2023-10-14 00:47:24,888][60934] Updated weights for policy 1, policy_version 91402 (0.0008) [2023-10-14 00:47:25,258][60934] Updated weights for policy 1, policy_version 91412 (0.0009) [2023-10-14 00:47:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 190513152. Throughput: 0: 1730.3, 1: 1695.0. Samples: 47637752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:47:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:47:26,967][60935] Updated weights for policy 0, policy_version 92610 (0.0008) [2023-10-14 00:47:27,324][60935] Updated weights for policy 0, policy_version 92620 (0.0011) [2023-10-14 00:47:27,690][60935] Updated weights for policy 0, policy_version 92630 (0.0010) [2023-10-14 00:47:28,066][60935] Updated weights for policy 0, policy_version 92640 (0.0010) [2023-10-14 00:47:29,193][60934] Updated weights for policy 1, policy_version 91422 (0.0008) [2023-10-14 00:47:29,562][60934] Updated weights for policy 1, policy_version 91432 (0.0008) [2023-10-14 00:47:29,926][60934] Updated weights for policy 1, policy_version 91442 (0.0009) [2023-10-14 00:47:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 190578688. Throughput: 0: 1692.5, 1: 1731.8. Samples: 47648384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:47:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:47:32,314][60935] Updated weights for policy 0, policy_version 92650 (0.0010) [2023-10-14 00:47:32,681][60935] Updated weights for policy 0, policy_version 92660 (0.0010) [2023-10-14 00:47:33,056][60935] Updated weights for policy 0, policy_version 92670 (0.0007) [2023-10-14 00:47:33,929][60934] Updated weights for policy 1, policy_version 91452 (0.0008) [2023-10-14 00:47:34,289][60934] Updated weights for policy 1, policy_version 91462 (0.0011) [2023-10-14 00:47:34,664][60934] Updated weights for policy 1, policy_version 91472 (0.0009) [2023-10-14 00:47:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 190644224. Throughput: 0: 1715.7, 1: 1710.6. Samples: 47668648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:47:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:47:36,912][60935] Updated weights for policy 0, policy_version 92680 (0.0010) [2023-10-14 00:47:37,283][60935] Updated weights for policy 0, policy_version 92690 (0.0008) [2023-10-14 00:47:37,651][60935] Updated weights for policy 0, policy_version 92700 (0.0009) [2023-10-14 00:47:38,680][60934] Updated weights for policy 1, policy_version 91482 (0.0009) [2023-10-14 00:47:39,050][60934] Updated weights for policy 1, policy_version 91492 (0.0008) [2023-10-14 00:47:39,416][60934] Updated weights for policy 1, policy_version 91502 (0.0008) [2023-10-14 00:47:39,783][60934] Updated weights for policy 1, policy_version 91512 (0.0009) [2023-10-14 00:47:41,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 190709760. Throughput: 0: 1713.3, 1: 1700.6. Samples: 47689172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:47:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:47:41,744][60935] Updated weights for policy 0, policy_version 92710 (0.0010) [2023-10-14 00:47:42,117][60935] Updated weights for policy 0, policy_version 92720 (0.0009) [2023-10-14 00:47:42,480][60935] Updated weights for policy 0, policy_version 92730 (0.0007) [2023-10-14 00:47:43,868][60934] Updated weights for policy 1, policy_version 91522 (0.0007) [2023-10-14 00:47:44,229][60934] Updated weights for policy 1, policy_version 91532 (0.0009) [2023-10-14 00:47:44,601][60934] Updated weights for policy 1, policy_version 91542 (0.0007) [2023-10-14 00:47:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 190775296. Throughput: 0: 1697.1, 1: 1726.2. Samples: 47699514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:47:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:47:46,531][60935] Updated weights for policy 0, policy_version 92740 (0.0009) [2023-10-14 00:47:46,901][60935] Updated weights for policy 0, policy_version 92750 (0.0008) [2023-10-14 00:47:47,275][60935] Updated weights for policy 0, policy_version 92760 (0.0008) [2023-10-14 00:47:48,466][60934] Updated weights for policy 1, policy_version 91552 (0.0009) [2023-10-14 00:47:48,842][60934] Updated weights for policy 1, policy_version 91562 (0.0008) [2023-10-14 00:47:49,197][60934] Updated weights for policy 1, policy_version 91572 (0.0007) [2023-10-14 00:47:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 190840832. Throughput: 0: 1701.4, 1: 1698.6. Samples: 47719084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:47:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:47:51,416][60935] Updated weights for policy 0, policy_version 92770 (0.0008) [2023-10-14 00:47:51,783][60935] Updated weights for policy 0, policy_version 92780 (0.0010) [2023-10-14 00:47:52,156][60935] Updated weights for policy 0, policy_version 92790 (0.0010) [2023-10-14 00:47:52,521][60935] Updated weights for policy 0, policy_version 92800 (0.0009) [2023-10-14 00:47:53,068][60934] Updated weights for policy 1, policy_version 91582 (0.0009) [2023-10-14 00:47:53,450][60934] Updated weights for policy 1, policy_version 91592 (0.0007) [2023-10-14 00:47:53,814][60934] Updated weights for policy 1, policy_version 91602 (0.0007) [2023-10-14 00:47:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 190906368. Throughput: 0: 1693.9, 1: 1720.0. Samples: 47740100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:47:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:47:56,793][60935] Updated weights for policy 0, policy_version 92810 (0.0010) [2023-10-14 00:47:57,160][60935] Updated weights for policy 0, policy_version 92820 (0.0007) [2023-10-14 00:47:57,528][60935] Updated weights for policy 0, policy_version 92830 (0.0008) [2023-10-14 00:47:57,695][60934] Updated weights for policy 1, policy_version 91612 (0.0008) [2023-10-14 00:47:58,055][60934] Updated weights for policy 1, policy_version 91622 (0.0010) [2023-10-14 00:47:58,423][60934] Updated weights for policy 1, policy_version 91632 (0.0007) [2023-10-14 00:48:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 190971904. Throughput: 0: 1689.5, 1: 1713.1. Samples: 47749402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:48:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:48:01,422][60935] Updated weights for policy 0, policy_version 92840 (0.0008) [2023-10-14 00:48:01,793][60935] Updated weights for policy 0, policy_version 92850 (0.0008) [2023-10-14 00:48:02,157][60935] Updated weights for policy 0, policy_version 92860 (0.0010) [2023-10-14 00:48:02,487][60934] Updated weights for policy 1, policy_version 91642 (0.0008) [2023-10-14 00:48:02,856][60934] Updated weights for policy 1, policy_version 91652 (0.0009) [2023-10-14 00:48:03,214][60934] Updated weights for policy 1, policy_version 91662 (0.0009) [2023-10-14 00:48:03,582][60934] Updated weights for policy 1, policy_version 91672 (0.0009) [2023-10-14 00:48:06,179][60935] Updated weights for policy 0, policy_version 92870 (0.0008) [2023-10-14 00:48:06,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 191037440. Throughput: 0: 1690.4, 1: 1697.4. Samples: 47770194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:48:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:48:06,547][60935] Updated weights for policy 0, policy_version 92880 (0.0008) [2023-10-14 00:48:06,913][60935] Updated weights for policy 0, policy_version 92890 (0.0010) [2023-10-14 00:48:07,696][60934] Updated weights for policy 1, policy_version 91682 (0.0007) [2023-10-14 00:48:08,055][60934] Updated weights for policy 1, policy_version 91692 (0.0008) [2023-10-14 00:48:08,424][60934] Updated weights for policy 1, policy_version 91702 (0.0010) [2023-10-14 00:48:10,808][60935] Updated weights for policy 0, policy_version 92900 (0.0011) [2023-10-14 00:48:11,181][60935] Updated weights for policy 0, policy_version 92910 (0.0011) [2023-10-14 00:48:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 191102976. Throughput: 0: 1682.1, 1: 1723.2. Samples: 47790990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:48:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:48:11,552][60935] Updated weights for policy 0, policy_version 92920 (0.0010) [2023-10-14 00:48:12,362][60934] Updated weights for policy 1, policy_version 91712 (0.0010) [2023-10-14 00:48:12,731][60934] Updated weights for policy 1, policy_version 91722 (0.0008) [2023-10-14 00:48:13,092][60934] Updated weights for policy 1, policy_version 91732 (0.0008) [2023-10-14 00:48:15,757][60935] Updated weights for policy 0, policy_version 92930 (0.0009) [2023-10-14 00:48:16,129][60935] Updated weights for policy 0, policy_version 92940 (0.0007) [2023-10-14 00:48:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 191168512. Throughput: 0: 1687.3, 1: 1688.6. Samples: 47800302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:48:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:48:16,495][60935] Updated weights for policy 0, policy_version 92950 (0.0010) [2023-10-14 00:48:16,858][60935] Updated weights for policy 0, policy_version 92960 (0.0008) [2023-10-14 00:48:16,981][60934] Updated weights for policy 1, policy_version 91742 (0.0007) [2023-10-14 00:48:17,336][60934] Updated weights for policy 1, policy_version 91752 (0.0008) [2023-10-14 00:48:17,701][60934] Updated weights for policy 1, policy_version 91762 (0.0010) [2023-10-14 00:48:20,918][60935] Updated weights for policy 0, policy_version 92970 (0.0007) [2023-10-14 00:48:21,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 191234048. Throughput: 0: 1691.3, 1: 1704.5. Samples: 47821458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:48:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:48:21,292][60935] Updated weights for policy 0, policy_version 92980 (0.0009) [2023-10-14 00:48:21,666][60935] Updated weights for policy 0, policy_version 92990 (0.0009) [2023-10-14 00:48:21,851][60934] Updated weights for policy 1, policy_version 91772 (0.0008) [2023-10-14 00:48:22,209][60934] Updated weights for policy 1, policy_version 91782 (0.0008) [2023-10-14 00:48:22,576][60934] Updated weights for policy 1, policy_version 91792 (0.0009) [2023-10-14 00:48:25,519][60935] Updated weights for policy 0, policy_version 93000 (0.0009) [2023-10-14 00:48:25,886][60935] Updated weights for policy 0, policy_version 93010 (0.0009) [2023-10-14 00:48:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 191299584. Throughput: 0: 1678.9, 1: 1715.4. Samples: 47841914. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:48:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:48:26,263][60935] Updated weights for policy 0, policy_version 93020 (0.0008) [2023-10-14 00:48:26,479][60934] Updated weights for policy 1, policy_version 91802 (0.0011) [2023-10-14 00:48:26,843][60934] Updated weights for policy 1, policy_version 91812 (0.0009) [2023-10-14 00:48:27,208][60934] Updated weights for policy 1, policy_version 91822 (0.0008) [2023-10-14 00:48:27,574][60934] Updated weights for policy 1, policy_version 91832 (0.0007) [2023-10-14 00:48:30,271][60935] Updated weights for policy 0, policy_version 93030 (0.0009) [2023-10-14 00:48:30,644][60935] Updated weights for policy 0, policy_version 93040 (0.0008) [2023-10-14 00:48:31,011][60935] Updated weights for policy 0, policy_version 93050 (0.0008) [2023-10-14 00:48:31,248][59943] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 191397888. Throughput: 0: 1697.6, 1: 1689.0. Samples: 47851910. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:48:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:48:31,510][60934] Updated weights for policy 1, policy_version 91842 (0.0008) [2023-10-14 00:48:31,879][60934] Updated weights for policy 1, policy_version 91852 (0.0008) [2023-10-14 00:48:32,238][60934] Updated weights for policy 1, policy_version 91862 (0.0007) [2023-10-14 00:48:35,014][60935] Updated weights for policy 0, policy_version 93060 (0.0007) [2023-10-14 00:48:35,378][60935] Updated weights for policy 0, policy_version 93070 (0.0007) [2023-10-14 00:48:35,750][60935] Updated weights for policy 0, policy_version 93080 (0.0009) [2023-10-14 00:48:36,191][60934] Updated weights for policy 1, policy_version 91872 (0.0007) [2023-10-14 00:48:36,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 191463424. Throughput: 0: 1706.3, 1: 1719.8. Samples: 47873256. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:48:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:48:36,553][60934] Updated weights for policy 1, policy_version 91882 (0.0009) [2023-10-14 00:48:36,921][60934] Updated weights for policy 1, policy_version 91892 (0.0008) [2023-10-14 00:48:39,789][60935] Updated weights for policy 0, policy_version 93090 (0.0007) [2023-10-14 00:48:40,152][60935] Updated weights for policy 0, policy_version 93100 (0.0007) [2023-10-14 00:48:40,526][60935] Updated weights for policy 0, policy_version 93110 (0.0008) [2023-10-14 00:48:40,895][60935] Updated weights for policy 0, policy_version 93120 (0.0009) [2023-10-14 00:48:40,961][60934] Updated weights for policy 1, policy_version 91902 (0.0008) [2023-10-14 00:48:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 191528960. Throughput: 0: 1686.8, 1: 1718.5. Samples: 47893336. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:48:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:48:41,339][60934] Updated weights for policy 1, policy_version 91912 (0.0007) [2023-10-14 00:48:41,710][60934] Updated weights for policy 1, policy_version 91922 (0.0009) [2023-10-14 00:48:44,879][60935] Updated weights for policy 0, policy_version 93130 (0.0009) [2023-10-14 00:48:45,238][60935] Updated weights for policy 0, policy_version 93140 (0.0009) [2023-10-14 00:48:45,605][60935] Updated weights for policy 0, policy_version 93150 (0.0009) [2023-10-14 00:48:45,864][60934] Updated weights for policy 1, policy_version 91932 (0.0008) [2023-10-14 00:48:46,225][60934] Updated weights for policy 1, policy_version 91942 (0.0007) [2023-10-14 00:48:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 191594496. Throughput: 0: 1716.4, 1: 1711.0. Samples: 47903636. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:48:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:48:46,588][60934] Updated weights for policy 1, policy_version 91952 (0.0007) [2023-10-14 00:48:49,729][60935] Updated weights for policy 0, policy_version 93160 (0.0008) [2023-10-14 00:48:50,103][60935] Updated weights for policy 0, policy_version 93170 (0.0008) [2023-10-14 00:48:50,446][60934] Updated weights for policy 1, policy_version 91962 (0.0007) [2023-10-14 00:48:50,465][60935] Updated weights for policy 0, policy_version 93180 (0.0009) [2023-10-14 00:48:50,812][60934] Updated weights for policy 1, policy_version 91972 (0.0009) [2023-10-14 00:48:51,174][60934] Updated weights for policy 1, policy_version 91982 (0.0010) [2023-10-14 00:48:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 191660032. Throughput: 0: 1695.2, 1: 1721.8. Samples: 47923962. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:48:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:48:51,541][60934] Updated weights for policy 1, policy_version 91992 (0.0008) [2023-10-14 00:48:54,464][60935] Updated weights for policy 0, policy_version 93190 (0.0008) [2023-10-14 00:48:54,835][60935] Updated weights for policy 0, policy_version 93200 (0.0011) [2023-10-14 00:48:55,200][60935] Updated weights for policy 0, policy_version 93210 (0.0009) [2023-10-14 00:48:55,446][60934] Updated weights for policy 1, policy_version 92002 (0.0008) [2023-10-14 00:48:55,818][60934] Updated weights for policy 1, policy_version 92012 (0.0007) [2023-10-14 00:48:56,180][60934] Updated weights for policy 1, policy_version 92022 (0.0007) [2023-10-14 00:48:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 191725568. Throughput: 0: 1675.2, 1: 1714.9. Samples: 47943542. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:48:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:48:56,262][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000092024_96305152.pth... [2023-10-14 00:48:56,262][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000093216_95453184.pth... [2023-10-14 00:48:56,299][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000091616_93814784.pth [2023-10-14 00:48:56,303][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000090424_94666752.pth [2023-10-14 00:48:59,183][60935] Updated weights for policy 0, policy_version 93220 (0.0009) [2023-10-14 00:48:59,555][60935] Updated weights for policy 0, policy_version 93230 (0.0008) [2023-10-14 00:48:59,932][60935] Updated weights for policy 0, policy_version 93240 (0.0007) [2023-10-14 00:49:00,274][60934] Updated weights for policy 1, policy_version 92032 (0.0009) [2023-10-14 00:49:00,641][60934] Updated weights for policy 1, policy_version 92042 (0.0007) [2023-10-14 00:49:01,006][60934] Updated weights for policy 1, policy_version 92052 (0.0008) [2023-10-14 00:49:01,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 191823872. Throughput: 0: 1703.4, 1: 1725.3. Samples: 47954594. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:49:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:49:04,090][60935] Updated weights for policy 0, policy_version 93250 (0.0009) [2023-10-14 00:49:04,456][60935] Updated weights for policy 0, policy_version 93260 (0.0010) [2023-10-14 00:49:04,832][60935] Updated weights for policy 0, policy_version 93270 (0.0009) [2023-10-14 00:49:05,197][60934] Updated weights for policy 1, policy_version 92062 (0.0007) [2023-10-14 00:49:05,199][60935] Updated weights for policy 0, policy_version 93280 (0.0010) [2023-10-14 00:49:05,558][60934] Updated weights for policy 1, policy_version 92072 (0.0007) [2023-10-14 00:49:05,920][60934] Updated weights for policy 1, policy_version 92082 (0.0007) [2023-10-14 00:49:06,248][59943] Fps is (10 sec: 16384.2, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 191889408. Throughput: 0: 1681.7, 1: 1722.6. Samples: 47974652. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:49:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:49:09,255][60935] Updated weights for policy 0, policy_version 93290 (0.0008) [2023-10-14 00:49:09,626][60935] Updated weights for policy 0, policy_version 93300 (0.0009) [2023-10-14 00:49:09,787][60934] Updated weights for policy 1, policy_version 92092 (0.0007) [2023-10-14 00:49:09,991][60935] Updated weights for policy 0, policy_version 93310 (0.0007) [2023-10-14 00:49:10,144][60934] Updated weights for policy 1, policy_version 92102 (0.0009) [2023-10-14 00:49:10,511][60934] Updated weights for policy 1, policy_version 92112 (0.0007) [2023-10-14 00:49:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 191954944. Throughput: 0: 1685.2, 1: 1704.2. Samples: 47994438. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:49:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:49:14,083][60935] Updated weights for policy 0, policy_version 93320 (0.0008) [2023-10-14 00:49:14,418][60934] Updated weights for policy 1, policy_version 92122 (0.0008) [2023-10-14 00:49:14,445][60935] Updated weights for policy 0, policy_version 93330 (0.0007) [2023-10-14 00:49:14,772][60934] Updated weights for policy 1, policy_version 92132 (0.0008) [2023-10-14 00:49:14,805][60935] Updated weights for policy 0, policy_version 93340 (0.0009) [2023-10-14 00:49:15,133][60934] Updated weights for policy 1, policy_version 92142 (0.0007) [2023-10-14 00:49:15,499][60934] Updated weights for policy 1, policy_version 92152 (0.0009) [2023-10-14 00:49:16,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 192020480. Throughput: 0: 1700.0, 1: 1724.5. Samples: 48006010. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 00:49:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:49:18,762][60935] Updated weights for policy 0, policy_version 93350 (0.0009) [2023-10-14 00:49:19,122][60935] Updated weights for policy 0, policy_version 93360 (0.0010) [2023-10-14 00:49:19,494][60935] Updated weights for policy 0, policy_version 93370 (0.0008) [2023-10-14 00:49:19,545][60934] Updated weights for policy 1, policy_version 92162 (0.0008) [2023-10-14 00:49:19,914][60934] Updated weights for policy 1, policy_version 92172 (0.0008) [2023-10-14 00:49:20,281][60934] Updated weights for policy 1, policy_version 92182 (0.0008) [2023-10-14 00:49:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 192086016. Throughput: 0: 1665.9, 1: 1712.6. Samples: 48025290. Policy #0 lag: (min: 3.0, avg: 3.8, max: 18.0) [2023-10-14 00:49:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:49:23,626][60935] Updated weights for policy 0, policy_version 93380 (0.0007) [2023-10-14 00:49:23,996][60935] Updated weights for policy 0, policy_version 93390 (0.0008) [2023-10-14 00:49:24,177][60934] Updated weights for policy 1, policy_version 92192 (0.0009) [2023-10-14 00:49:24,360][60935] Updated weights for policy 0, policy_version 93400 (0.0007) [2023-10-14 00:49:24,539][60934] Updated weights for policy 1, policy_version 92202 (0.0008) [2023-10-14 00:49:24,899][60934] Updated weights for policy 1, policy_version 92212 (0.0010) [2023-10-14 00:49:26,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 192151552. Throughput: 0: 1691.9, 1: 1691.6. Samples: 48045594. Policy #0 lag: (min: 3.0, avg: 3.8, max: 18.0) [2023-10-14 00:49:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:49:28,063][60935] Updated weights for policy 0, policy_version 93410 (0.0008) [2023-10-14 00:49:28,429][60935] Updated weights for policy 0, policy_version 93420 (0.0008) [2023-10-14 00:49:28,792][60935] Updated weights for policy 0, policy_version 93430 (0.0008) [2023-10-14 00:49:28,895][60934] Updated weights for policy 1, policy_version 92222 (0.0007) [2023-10-14 00:49:29,162][60935] Updated weights for policy 0, policy_version 93440 (0.0008) [2023-10-14 00:49:29,264][60934] Updated weights for policy 1, policy_version 92232 (0.0008) [2023-10-14 00:49:29,625][60934] Updated weights for policy 1, policy_version 92242 (0.0007) [2023-10-14 00:49:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 192217088. Throughput: 0: 1678.4, 1: 1724.0. Samples: 48056744. Policy #0 lag: (min: 3.0, avg: 3.8, max: 18.0) [2023-10-14 00:49:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:49:33,238][60935] Updated weights for policy 0, policy_version 93450 (0.0008) [2023-10-14 00:49:33,600][60935] Updated weights for policy 0, policy_version 93460 (0.0007) [2023-10-14 00:49:33,856][60934] Updated weights for policy 1, policy_version 92252 (0.0008) [2023-10-14 00:49:33,975][60935] Updated weights for policy 0, policy_version 93470 (0.0008) [2023-10-14 00:49:34,222][60934] Updated weights for policy 1, policy_version 92262 (0.0011) [2023-10-14 00:49:34,583][60934] Updated weights for policy 1, policy_version 92272 (0.0009) [2023-10-14 00:49:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 192282624. Throughput: 0: 1686.0, 1: 1696.6. Samples: 48076178. Policy #0 lag: (min: 3.0, avg: 3.8, max: 18.0) [2023-10-14 00:49:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:49:38,069][60935] Updated weights for policy 0, policy_version 93480 (0.0010) [2023-10-14 00:49:38,443][60935] Updated weights for policy 0, policy_version 93490 (0.0009) [2023-10-14 00:49:38,513][60934] Updated weights for policy 1, policy_version 92282 (0.0009) [2023-10-14 00:49:38,803][60935] Updated weights for policy 0, policy_version 93500 (0.0009) [2023-10-14 00:49:38,880][60934] Updated weights for policy 1, policy_version 92292 (0.0008) [2023-10-14 00:49:39,238][60934] Updated weights for policy 1, policy_version 92302 (0.0010) [2023-10-14 00:49:39,604][60934] Updated weights for policy 1, policy_version 92312 (0.0010) [2023-10-14 00:49:41,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 192348160. Throughput: 0: 1706.8, 1: 1693.3. Samples: 48096544. Policy #0 lag: (min: 3.0, avg: 3.8, max: 18.0) [2023-10-14 00:49:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:49:42,839][60935] Updated weights for policy 0, policy_version 93510 (0.0009) [2023-10-14 00:49:43,203][60935] Updated weights for policy 0, policy_version 93520 (0.0010) [2023-10-14 00:49:43,576][60935] Updated weights for policy 0, policy_version 93530 (0.0008) [2023-10-14 00:49:43,704][60934] Updated weights for policy 1, policy_version 92322 (0.0008) [2023-10-14 00:49:44,074][60934] Updated weights for policy 1, policy_version 92332 (0.0007) [2023-10-14 00:49:44,434][60934] Updated weights for policy 1, policy_version 92342 (0.0009) [2023-10-14 00:49:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 192413696. Throughput: 0: 1677.8, 1: 1705.2. Samples: 48106826. Policy #0 lag: (min: 3.0, avg: 3.8, max: 18.0) [2023-10-14 00:49:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:49:47,579][60935] Updated weights for policy 0, policy_version 93540 (0.0007) [2023-10-14 00:49:47,943][60935] Updated weights for policy 0, policy_version 93550 (0.0010) [2023-10-14 00:49:48,307][60935] Updated weights for policy 0, policy_version 93560 (0.0009) [2023-10-14 00:49:48,550][60934] Updated weights for policy 1, policy_version 92352 (0.0007) [2023-10-14 00:49:48,909][60934] Updated weights for policy 1, policy_version 92362 (0.0007) [2023-10-14 00:49:49,272][60934] Updated weights for policy 1, policy_version 92372 (0.0008) [2023-10-14 00:49:51,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 192479232. Throughput: 0: 1696.8, 1: 1680.4. Samples: 48126624. Policy #0 lag: (min: 3.0, avg: 3.8, max: 18.0) [2023-10-14 00:49:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:49:52,486][60935] Updated weights for policy 0, policy_version 93570 (0.0010) [2023-10-14 00:49:52,853][60935] Updated weights for policy 0, policy_version 93580 (0.0009) [2023-10-14 00:49:53,178][60934] Updated weights for policy 1, policy_version 92382 (0.0010) [2023-10-14 00:49:53,222][60935] Updated weights for policy 0, policy_version 93590 (0.0009) [2023-10-14 00:49:53,538][60934] Updated weights for policy 1, policy_version 92392 (0.0008) [2023-10-14 00:49:53,579][60935] Updated weights for policy 0, policy_version 93600 (0.0009) [2023-10-14 00:49:53,917][60934] Updated weights for policy 1, policy_version 92402 (0.0009) [2023-10-14 00:49:56,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 192544768. Throughput: 0: 1702.3, 1: 1700.5. Samples: 48147564. Policy #0 lag: (min: 3.0, avg: 3.8, max: 18.0) [2023-10-14 00:49:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.110')] [2023-10-14 00:49:57,511][60935] Updated weights for policy 0, policy_version 93610 (0.0010) [2023-10-14 00:49:57,877][60935] Updated weights for policy 0, policy_version 93620 (0.0010) [2023-10-14 00:49:58,018][60934] Updated weights for policy 1, policy_version 92412 (0.0008) [2023-10-14 00:49:58,246][60935] Updated weights for policy 0, policy_version 93630 (0.0008) [2023-10-14 00:49:58,389][60934] Updated weights for policy 1, policy_version 92422 (0.0008) [2023-10-14 00:49:58,749][60934] Updated weights for policy 1, policy_version 92432 (0.0007) [2023-10-14 00:50:01,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 192610304. Throughput: 0: 1676.1, 1: 1690.3. Samples: 48157502. Policy #0 lag: (min: 3.0, avg: 3.8, max: 18.0) [2023-10-14 00:50:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.110')] [2023-10-14 00:50:02,412][60935] Updated weights for policy 0, policy_version 93640 (0.0008) [2023-10-14 00:50:02,774][60935] Updated weights for policy 0, policy_version 93650 (0.0008) [2023-10-14 00:50:02,825][60934] Updated weights for policy 1, policy_version 92442 (0.0009) [2023-10-14 00:50:03,153][60935] Updated weights for policy 0, policy_version 93660 (0.0009) [2023-10-14 00:50:03,192][60934] Updated weights for policy 1, policy_version 92452 (0.0008) [2023-10-14 00:50:03,558][60934] Updated weights for policy 1, policy_version 92462 (0.0007) [2023-10-14 00:50:03,921][60934] Updated weights for policy 1, policy_version 92472 (0.0009) [2023-10-14 00:50:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13662.6). Total num frames: 192675840. Throughput: 0: 1704.8, 1: 1683.4. Samples: 48177760. Policy #0 lag: (min: 3.0, avg: 3.8, max: 18.0) [2023-10-14 00:50:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.110')] [2023-10-14 00:50:07,341][60935] Updated weights for policy 0, policy_version 93670 (0.0007) [2023-10-14 00:50:07,707][60935] Updated weights for policy 0, policy_version 93680 (0.0009) [2023-10-14 00:50:07,933][60934] Updated weights for policy 1, policy_version 92482 (0.0008) [2023-10-14 00:50:08,080][60935] Updated weights for policy 0, policy_version 93690 (0.0008) [2023-10-14 00:50:08,299][60934] Updated weights for policy 1, policy_version 92492 (0.0008) [2023-10-14 00:50:08,670][60934] Updated weights for policy 1, policy_version 92502 (0.0008) [2023-10-14 00:50:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 192741376. Throughput: 0: 1699.4, 1: 1700.1. Samples: 48198572. Policy #0 lag: (min: 3.0, avg: 3.8, max: 18.0) [2023-10-14 00:50:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.110')] [2023-10-14 00:50:11,850][60935] Updated weights for policy 0, policy_version 93700 (0.0010) [2023-10-14 00:50:12,216][60935] Updated weights for policy 0, policy_version 93710 (0.0010) [2023-10-14 00:50:12,579][60935] Updated weights for policy 0, policy_version 93720 (0.0009) [2023-10-14 00:50:12,743][60934] Updated weights for policy 1, policy_version 92512 (0.0009) [2023-10-14 00:50:13,107][60934] Updated weights for policy 1, policy_version 92522 (0.0008) [2023-10-14 00:50:13,468][60934] Updated weights for policy 1, policy_version 92532 (0.0008) [2023-10-14 00:50:16,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 192806912. Throughput: 0: 1689.8, 1: 1670.9. Samples: 48207976. Policy #0 lag: (min: 3.0, avg: 3.8, max: 18.0) [2023-10-14 00:50:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:50:16,597][60935] Updated weights for policy 0, policy_version 93730 (0.0009) [2023-10-14 00:50:16,961][60935] Updated weights for policy 0, policy_version 93740 (0.0010) [2023-10-14 00:50:17,332][60935] Updated weights for policy 0, policy_version 93750 (0.0009) [2023-10-14 00:50:17,597][60934] Updated weights for policy 1, policy_version 92542 (0.0008) [2023-10-14 00:50:17,700][60935] Updated weights for policy 0, policy_version 93760 (0.0008) [2023-10-14 00:50:17,973][60934] Updated weights for policy 1, policy_version 92552 (0.0009) [2023-10-14 00:50:18,343][60934] Updated weights for policy 1, policy_version 92562 (0.0007) [2023-10-14 00:50:21,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 192872448. Throughput: 0: 1698.4, 1: 1691.8. Samples: 48228740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:50:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:50:21,714][60935] Updated weights for policy 0, policy_version 93770 (0.0009) [2023-10-14 00:50:22,049][60934] Updated weights for policy 1, policy_version 92572 (0.0008) [2023-10-14 00:50:22,072][60935] Updated weights for policy 0, policy_version 93780 (0.0009) [2023-10-14 00:50:22,406][60934] Updated weights for policy 1, policy_version 92582 (0.0009) [2023-10-14 00:50:22,446][60935] Updated weights for policy 0, policy_version 93790 (0.0009) [2023-10-14 00:50:22,769][60934] Updated weights for policy 1, policy_version 92592 (0.0007) [2023-10-14 00:50:26,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 192937984. Throughput: 0: 1701.3, 1: 1708.0. Samples: 48249962. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:50:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.240')] [2023-10-14 00:50:26,434][60935] Updated weights for policy 0, policy_version 93800 (0.0008) [2023-10-14 00:50:26,799][60935] Updated weights for policy 0, policy_version 93810 (0.0009) [2023-10-14 00:50:26,960][60934] Updated weights for policy 1, policy_version 92602 (0.0008) [2023-10-14 00:50:27,170][60935] Updated weights for policy 0, policy_version 93820 (0.0008) [2023-10-14 00:50:27,328][60934] Updated weights for policy 1, policy_version 92612 (0.0007) [2023-10-14 00:50:27,689][60934] Updated weights for policy 1, policy_version 92622 (0.0009) [2023-10-14 00:50:28,061][60934] Updated weights for policy 1, policy_version 92632 (0.0010) [2023-10-14 00:50:31,077][60935] Updated weights for policy 0, policy_version 93830 (0.0008) [2023-10-14 00:50:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 193003520. Throughput: 0: 1700.2, 1: 1684.3. Samples: 48259130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:50:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.250')] [2023-10-14 00:50:31,451][60935] Updated weights for policy 0, policy_version 93840 (0.0010) [2023-10-14 00:50:31,803][60935] Updated weights for policy 0, policy_version 93850 (0.0010) [2023-10-14 00:50:32,020][60934] Updated weights for policy 1, policy_version 92642 (0.0008) [2023-10-14 00:50:32,389][60934] Updated weights for policy 1, policy_version 92652 (0.0008) [2023-10-14 00:50:32,759][60934] Updated weights for policy 1, policy_version 92662 (0.0008) [2023-10-14 00:50:35,957][60935] Updated weights for policy 0, policy_version 93860 (0.0008) [2023-10-14 00:50:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 193069056. Throughput: 0: 1697.5, 1: 1713.0. Samples: 48280094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:50:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.250')] [2023-10-14 00:50:36,324][60935] Updated weights for policy 0, policy_version 93870 (0.0009) [2023-10-14 00:50:36,685][60935] Updated weights for policy 0, policy_version 93880 (0.0008) [2023-10-14 00:50:36,817][60934] Updated weights for policy 1, policy_version 92672 (0.0008) [2023-10-14 00:50:37,188][60934] Updated weights for policy 1, policy_version 92682 (0.0010) [2023-10-14 00:50:37,551][60934] Updated weights for policy 1, policy_version 92692 (0.0011) [2023-10-14 00:50:40,813][60935] Updated weights for policy 0, policy_version 93890 (0.0010) [2023-10-14 00:50:41,189][60935] Updated weights for policy 0, policy_version 93900 (0.0009) [2023-10-14 00:50:41,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13551.5). Total num frames: 193134592. Throughput: 0: 1701.1, 1: 1706.8. Samples: 48300920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:50:41,250][59943] Avg episode reward: [(0, '0.000'), (1, '-0.250')] [2023-10-14 00:50:41,550][60935] Updated weights for policy 0, policy_version 93910 (0.0007) [2023-10-14 00:50:41,557][60934] Updated weights for policy 1, policy_version 92702 (0.0009) [2023-10-14 00:50:41,925][60935] Updated weights for policy 0, policy_version 93920 (0.0007) [2023-10-14 00:50:41,927][60934] Updated weights for policy 1, policy_version 92712 (0.0007) [2023-10-14 00:50:42,291][60934] Updated weights for policy 1, policy_version 92722 (0.0010) [2023-10-14 00:50:45,830][60935] Updated weights for policy 0, policy_version 93930 (0.0011) [2023-10-14 00:50:46,199][60935] Updated weights for policy 0, policy_version 93940 (0.0011) [2023-10-14 00:50:46,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 193200128. Throughput: 0: 1698.9, 1: 1691.4. Samples: 48310062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:50:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.250')] [2023-10-14 00:50:46,363][60934] Updated weights for policy 1, policy_version 92732 (0.0009) [2023-10-14 00:50:46,561][60935] Updated weights for policy 0, policy_version 93950 (0.0010) [2023-10-14 00:50:46,720][60934] Updated weights for policy 1, policy_version 92742 (0.0008) [2023-10-14 00:50:47,081][60934] Updated weights for policy 1, policy_version 92752 (0.0008) [2023-10-14 00:50:50,554][60935] Updated weights for policy 0, policy_version 93960 (0.0009) [2023-10-14 00:50:50,918][60935] Updated weights for policy 0, policy_version 93970 (0.0008) [2023-10-14 00:50:51,169][60934] Updated weights for policy 1, policy_version 92762 (0.0008) [2023-10-14 00:50:51,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 193265664. Throughput: 0: 1705.0, 1: 1698.7. Samples: 48330924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:50:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.270')] [2023-10-14 00:50:51,287][60935] Updated weights for policy 0, policy_version 93980 (0.0008) [2023-10-14 00:50:51,535][60934] Updated weights for policy 1, policy_version 92772 (0.0008) [2023-10-14 00:50:51,897][60934] Updated weights for policy 1, policy_version 92782 (0.0009) [2023-10-14 00:50:52,261][60934] Updated weights for policy 1, policy_version 92792 (0.0009) [2023-10-14 00:50:55,255][60935] Updated weights for policy 0, policy_version 93990 (0.0009) [2023-10-14 00:50:55,628][60935] Updated weights for policy 0, policy_version 94000 (0.0008) [2023-10-14 00:50:55,996][60935] Updated weights for policy 0, policy_version 94010 (0.0007) [2023-10-14 00:50:56,138][60934] Updated weights for policy 1, policy_version 92802 (0.0007) [2023-10-14 00:50:56,248][59943] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 193363968. Throughput: 0: 1689.0, 1: 1706.5. Samples: 48351370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:50:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.030')] [2023-10-14 00:50:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000094016_96272384.pth... [2023-10-14 00:50:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000092416_94633984.pth [2023-10-14 00:50:56,502][60934] Updated weights for policy 1, policy_version 92812 (0.0008) [2023-10-14 00:50:56,867][60934] Updated weights for policy 1, policy_version 92822 (0.0008) [2023-10-14 00:50:56,939][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000092824_97124352.pth... [2023-10-14 00:50:56,968][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000091224_95485952.pth [2023-10-14 00:51:00,041][60935] Updated weights for policy 0, policy_version 94020 (0.0007) [2023-10-14 00:51:00,412][60935] Updated weights for policy 0, policy_version 94030 (0.0010) [2023-10-14 00:51:00,770][60935] Updated weights for policy 0, policy_version 94040 (0.0007) [2023-10-14 00:51:00,927][60934] Updated weights for policy 1, policy_version 92832 (0.0008) [2023-10-14 00:51:01,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 193429504. Throughput: 0: 1706.4, 1: 1703.0. Samples: 48361396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:51:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:51:01,291][60934] Updated weights for policy 1, policy_version 92842 (0.0008) [2023-10-14 00:51:01,662][60934] Updated weights for policy 1, policy_version 92852 (0.0009) [2023-10-14 00:51:04,990][60935] Updated weights for policy 0, policy_version 94050 (0.0009) [2023-10-14 00:51:05,364][60935] Updated weights for policy 0, policy_version 94060 (0.0008) [2023-10-14 00:51:05,737][60935] Updated weights for policy 0, policy_version 94070 (0.0011) [2023-10-14 00:51:05,774][60934] Updated weights for policy 1, policy_version 92862 (0.0009) [2023-10-14 00:51:06,102][60935] Updated weights for policy 0, policy_version 94080 (0.0008) [2023-10-14 00:51:06,156][60934] Updated weights for policy 1, policy_version 92872 (0.0008) [2023-10-14 00:51:06,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 193495040. Throughput: 0: 1697.3, 1: 1711.6. Samples: 48382144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:51:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:51:06,510][60934] Updated weights for policy 1, policy_version 92882 (0.0007) [2023-10-14 00:51:10,108][60935] Updated weights for policy 0, policy_version 94090 (0.0009) [2023-10-14 00:51:10,377][60934] Updated weights for policy 1, policy_version 92892 (0.0007) [2023-10-14 00:51:10,467][60935] Updated weights for policy 0, policy_version 94100 (0.0010) [2023-10-14 00:51:10,742][60934] Updated weights for policy 1, policy_version 92902 (0.0008) [2023-10-14 00:51:10,833][60935] Updated weights for policy 0, policy_version 94110 (0.0010) [2023-10-14 00:51:11,108][60934] Updated weights for policy 1, policy_version 92912 (0.0010) [2023-10-14 00:51:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 193560576. Throughput: 0: 1674.1, 1: 1696.9. Samples: 48401658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:51:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:51:14,839][60935] Updated weights for policy 0, policy_version 94120 (0.0010) [2023-10-14 00:51:15,185][60934] Updated weights for policy 1, policy_version 92922 (0.0010) [2023-10-14 00:51:15,205][60935] Updated weights for policy 0, policy_version 94130 (0.0007) [2023-10-14 00:51:15,558][60934] Updated weights for policy 1, policy_version 92932 (0.0008) [2023-10-14 00:51:15,570][60935] Updated weights for policy 0, policy_version 94140 (0.0008) [2023-10-14 00:51:15,920][60934] Updated weights for policy 1, policy_version 92942 (0.0007) [2023-10-14 00:51:16,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 193626112. Throughput: 0: 1699.6, 1: 1705.1. Samples: 48412344. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) [2023-10-14 00:51:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:51:16,289][60934] Updated weights for policy 1, policy_version 92952 (0.0007) [2023-10-14 00:51:19,745][60935] Updated weights for policy 0, policy_version 94150 (0.0009) [2023-10-14 00:51:20,115][60935] Updated weights for policy 0, policy_version 94160 (0.0008) [2023-10-14 00:51:20,375][60934] Updated weights for policy 1, policy_version 92962 (0.0009) [2023-10-14 00:51:20,475][60935] Updated weights for policy 0, policy_version 94170 (0.0009) [2023-10-14 00:51:20,739][60934] Updated weights for policy 1, policy_version 92972 (0.0007) [2023-10-14 00:51:21,099][60934] Updated weights for policy 1, policy_version 92982 (0.0007) [2023-10-14 00:51:21,248][59943] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 193724416. Throughput: 0: 1696.9, 1: 1700.1. Samples: 48432960. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) [2023-10-14 00:51:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.020')] [2023-10-14 00:51:24,386][60935] Updated weights for policy 0, policy_version 94180 (0.0009) [2023-10-14 00:51:24,750][60935] Updated weights for policy 0, policy_version 94190 (0.0010) [2023-10-14 00:51:25,046][60934] Updated weights for policy 1, policy_version 92992 (0.0010) [2023-10-14 00:51:25,125][60935] Updated weights for policy 0, policy_version 94200 (0.0008) [2023-10-14 00:51:25,408][60934] Updated weights for policy 1, policy_version 93002 (0.0009) [2023-10-14 00:51:25,771][60934] Updated weights for policy 1, policy_version 93012 (0.0010) [2023-10-14 00:51:26,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 193789952. Throughput: 0: 1675.5, 1: 1686.7. Samples: 48452218. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) [2023-10-14 00:51:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:51:29,159][60935] Updated weights for policy 0, policy_version 94210 (0.0007) [2023-10-14 00:51:29,536][60935] Updated weights for policy 0, policy_version 94220 (0.0008) [2023-10-14 00:51:29,773][60934] Updated weights for policy 1, policy_version 93022 (0.0009) [2023-10-14 00:51:29,904][60935] Updated weights for policy 0, policy_version 94230 (0.0007) [2023-10-14 00:51:30,139][60934] Updated weights for policy 1, policy_version 93032 (0.0008) [2023-10-14 00:51:30,275][60935] Updated weights for policy 0, policy_version 94240 (0.0007) [2023-10-14 00:51:30,490][60934] Updated weights for policy 1, policy_version 93042 (0.0009) [2023-10-14 00:51:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 193855488. Throughput: 0: 1703.2, 1: 1707.9. Samples: 48463560. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) [2023-10-14 00:51:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:51:34,334][60935] Updated weights for policy 0, policy_version 94250 (0.0009) [2023-10-14 00:51:34,508][60934] Updated weights for policy 1, policy_version 93052 (0.0008) [2023-10-14 00:51:34,694][60935] Updated weights for policy 0, policy_version 94260 (0.0007) [2023-10-14 00:51:34,881][60934] Updated weights for policy 1, policy_version 93062 (0.0007) [2023-10-14 00:51:35,052][60935] Updated weights for policy 0, policy_version 94270 (0.0009) [2023-10-14 00:51:35,254][60934] Updated weights for policy 1, policy_version 93072 (0.0008) [2023-10-14 00:51:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 193921024. Throughput: 0: 1682.3, 1: 1706.3. Samples: 48483416. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) [2023-10-14 00:51:36,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:51:39,106][60935] Updated weights for policy 0, policy_version 94280 (0.0010) [2023-10-14 00:51:39,260][60934] Updated weights for policy 1, policy_version 93082 (0.0008) [2023-10-14 00:51:39,476][60935] Updated weights for policy 0, policy_version 94290 (0.0008) [2023-10-14 00:51:39,634][60934] Updated weights for policy 1, policy_version 93092 (0.0007) [2023-10-14 00:51:39,844][60935] Updated weights for policy 0, policy_version 94300 (0.0007) [2023-10-14 00:51:39,997][60934] Updated weights for policy 1, policy_version 93102 (0.0010) [2023-10-14 00:51:40,365][60934] Updated weights for policy 1, policy_version 93112 (0.0010) [2023-10-14 00:51:41,248][59943] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 193986560. Throughput: 0: 1687.6, 1: 1677.0. Samples: 48502780. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) [2023-10-14 00:51:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-2.590')] [2023-10-14 00:51:43,972][60935] Updated weights for policy 0, policy_version 94310 (0.0008) [2023-10-14 00:51:44,275][60934] Updated weights for policy 1, policy_version 93122 (0.0007) [2023-10-14 00:51:44,341][60935] Updated weights for policy 0, policy_version 94320 (0.0007) [2023-10-14 00:51:44,644][60934] Updated weights for policy 1, policy_version 93132 (0.0009) [2023-10-14 00:51:44,705][60935] Updated weights for policy 0, policy_version 94330 (0.0009) [2023-10-14 00:51:45,011][60934] Updated weights for policy 1, policy_version 93142 (0.0008) [2023-10-14 00:51:46,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 194052096. Throughput: 0: 1690.3, 1: 1709.9. Samples: 48514406. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) [2023-10-14 00:51:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-2.590')] [2023-10-14 00:51:48,768][60935] Updated weights for policy 0, policy_version 94340 (0.0009) [2023-10-14 00:51:49,134][60935] Updated weights for policy 0, policy_version 94350 (0.0008) [2023-10-14 00:51:49,151][60934] Updated weights for policy 1, policy_version 93152 (0.0007) [2023-10-14 00:51:49,496][60935] Updated weights for policy 0, policy_version 94360 (0.0007) [2023-10-14 00:51:49,514][60934] Updated weights for policy 1, policy_version 93162 (0.0007) [2023-10-14 00:51:49,869][60934] Updated weights for policy 1, policy_version 93172 (0.0009) [2023-10-14 00:51:51,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 194117632. Throughput: 0: 1670.5, 1: 1685.9. Samples: 48533184. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) [2023-10-14 00:51:51,250][59943] Avg episode reward: [(0, '0.000'), (1, '-2.590')] [2023-10-14 00:51:53,711][60935] Updated weights for policy 0, policy_version 94370 (0.0007) [2023-10-14 00:51:53,984][60934] Updated weights for policy 1, policy_version 93182 (0.0009) [2023-10-14 00:51:54,086][60935] Updated weights for policy 0, policy_version 94380 (0.0007) [2023-10-14 00:51:54,374][60934] Updated weights for policy 1, policy_version 93192 (0.0009) [2023-10-14 00:51:54,460][60935] Updated weights for policy 0, policy_version 94390 (0.0011) [2023-10-14 00:51:54,746][60934] Updated weights for policy 1, policy_version 93202 (0.0007) [2023-10-14 00:51:54,829][60935] Updated weights for policy 0, policy_version 94400 (0.0009) [2023-10-14 00:51:56,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 194183168. Throughput: 0: 1693.4, 1: 1677.2. Samples: 48553334. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) [2023-10-14 00:51:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-2.590')] [2023-10-14 00:51:58,636][60934] Updated weights for policy 1, policy_version 93212 (0.0010) [2023-10-14 00:51:58,822][60935] Updated weights for policy 0, policy_version 94410 (0.0008) [2023-10-14 00:51:58,999][60934] Updated weights for policy 1, policy_version 93222 (0.0008) [2023-10-14 00:51:59,201][60935] Updated weights for policy 0, policy_version 94420 (0.0008) [2023-10-14 00:51:59,364][60934] Updated weights for policy 1, policy_version 93232 (0.0007) [2023-10-14 00:51:59,558][60935] Updated weights for policy 0, policy_version 94430 (0.0008) [2023-10-14 00:52:01,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 194248704. Throughput: 0: 1685.6, 1: 1699.4. Samples: 48564668. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) [2023-10-14 00:52:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-2.590')] [2023-10-14 00:52:03,379][60934] Updated weights for policy 1, policy_version 93242 (0.0007) [2023-10-14 00:52:03,624][60935] Updated weights for policy 0, policy_version 94440 (0.0010) [2023-10-14 00:52:03,739][60934] Updated weights for policy 1, policy_version 93252 (0.0010) [2023-10-14 00:52:04,002][60935] Updated weights for policy 0, policy_version 94450 (0.0008) [2023-10-14 00:52:04,109][60934] Updated weights for policy 1, policy_version 93262 (0.0008) [2023-10-14 00:52:04,366][60935] Updated weights for policy 0, policy_version 94460 (0.0007) [2023-10-14 00:52:04,465][60934] Updated weights for policy 1, policy_version 93272 (0.0009) [2023-10-14 00:52:06,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 194314240. Throughput: 0: 1665.1, 1: 1675.4. Samples: 48583284. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) [2023-10-14 00:52:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-2.590')] [2023-10-14 00:52:08,448][60935] Updated weights for policy 0, policy_version 94470 (0.0007) [2023-10-14 00:52:08,527][60934] Updated weights for policy 1, policy_version 93282 (0.0007) [2023-10-14 00:52:08,834][60935] Updated weights for policy 0, policy_version 94480 (0.0009) [2023-10-14 00:52:08,891][60934] Updated weights for policy 1, policy_version 93292 (0.0007) [2023-10-14 00:52:09,199][60935] Updated weights for policy 0, policy_version 94490 (0.0009) [2023-10-14 00:52:09,263][60934] Updated weights for policy 1, policy_version 93302 (0.0007) [2023-10-14 00:52:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 194379776. Throughput: 0: 1684.1, 1: 1694.7. Samples: 48604264. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) [2023-10-14 00:52:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-2.590')] [2023-10-14 00:52:13,161][60935] Updated weights for policy 0, policy_version 94500 (0.0008) [2023-10-14 00:52:13,421][60934] Updated weights for policy 1, policy_version 93312 (0.0009) [2023-10-14 00:52:13,528][60935] Updated weights for policy 0, policy_version 94510 (0.0007) [2023-10-14 00:52:13,788][60934] Updated weights for policy 1, policy_version 93322 (0.0008) [2023-10-14 00:52:13,889][60935] Updated weights for policy 0, policy_version 94520 (0.0007) [2023-10-14 00:52:14,146][60934] Updated weights for policy 1, policy_version 93332 (0.0009) [2023-10-14 00:52:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 194445312. Throughput: 0: 1665.5, 1: 1692.0. Samples: 48614652. Policy #0 lag: (min: 29.0, avg: 29.6, max: 45.0) [2023-10-14 00:52:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-2.590')] [2023-10-14 00:52:17,905][60935] Updated weights for policy 0, policy_version 94530 (0.0008) [2023-10-14 00:52:18,117][60934] Updated weights for policy 1, policy_version 93342 (0.0008) [2023-10-14 00:52:18,273][60935] Updated weights for policy 0, policy_version 94540 (0.0008) [2023-10-14 00:52:18,484][60934] Updated weights for policy 1, policy_version 93352 (0.0008) [2023-10-14 00:52:18,641][60935] Updated weights for policy 0, policy_version 94550 (0.0007) [2023-10-14 00:52:18,849][60934] Updated weights for policy 1, policy_version 93362 (0.0007) [2023-10-14 00:52:19,012][60935] Updated weights for policy 0, policy_version 94560 (0.0007) [2023-10-14 00:52:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 194510848. Throughput: 0: 1674.6, 1: 1677.4. Samples: 48634256. Policy #0 lag: (min: 29.0, avg: 29.6, max: 45.0) [2023-10-14 00:52:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-2.590')] [2023-10-14 00:52:22,846][60934] Updated weights for policy 1, policy_version 93372 (0.0009) [2023-10-14 00:52:23,028][60935] Updated weights for policy 0, policy_version 94570 (0.0009) [2023-10-14 00:52:23,216][60934] Updated weights for policy 1, policy_version 93382 (0.0008) [2023-10-14 00:52:23,397][60935] Updated weights for policy 0, policy_version 94580 (0.0007) [2023-10-14 00:52:23,578][60934] Updated weights for policy 1, policy_version 93392 (0.0010) [2023-10-14 00:52:23,760][60935] Updated weights for policy 0, policy_version 94590 (0.0009) [2023-10-14 00:52:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 194576384. Throughput: 0: 1682.5, 1: 1700.5. Samples: 48655014. Policy #0 lag: (min: 29.0, avg: 29.6, max: 45.0) [2023-10-14 00:52:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:52:27,639][60934] Updated weights for policy 1, policy_version 93402 (0.0008) [2023-10-14 00:52:27,875][60935] Updated weights for policy 0, policy_version 94600 (0.0007) [2023-10-14 00:52:28,005][60934] Updated weights for policy 1, policy_version 93412 (0.0008) [2023-10-14 00:52:28,240][60935] Updated weights for policy 0, policy_version 94610 (0.0008) [2023-10-14 00:52:28,367][60934] Updated weights for policy 1, policy_version 93422 (0.0007) [2023-10-14 00:52:28,613][60935] Updated weights for policy 0, policy_version 94620 (0.0008) [2023-10-14 00:52:28,731][60934] Updated weights for policy 1, policy_version 93432 (0.0009) [2023-10-14 00:52:31,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 194641920. Throughput: 0: 1658.6, 1: 1674.3. Samples: 48664388. Policy #0 lag: (min: 29.0, avg: 29.6, max: 45.0) [2023-10-14 00:52:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:52:32,664][60935] Updated weights for policy 0, policy_version 94630 (0.0009) [2023-10-14 00:52:32,760][60934] Updated weights for policy 1, policy_version 93442 (0.0007) [2023-10-14 00:52:33,031][60935] Updated weights for policy 0, policy_version 94640 (0.0008) [2023-10-14 00:52:33,122][60934] Updated weights for policy 1, policy_version 93452 (0.0007) [2023-10-14 00:52:33,397][60935] Updated weights for policy 0, policy_version 94650 (0.0009) [2023-10-14 00:52:33,489][60934] Updated weights for policy 1, policy_version 93462 (0.0008) [2023-10-14 00:52:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 194707456. Throughput: 0: 1682.8, 1: 1685.6. Samples: 48684760. Policy #0 lag: (min: 29.0, avg: 29.6, max: 45.0) [2023-10-14 00:52:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:52:37,415][60935] Updated weights for policy 0, policy_version 94660 (0.0008) [2023-10-14 00:52:37,572][60934] Updated weights for policy 1, policy_version 93472 (0.0008) [2023-10-14 00:52:37,782][60935] Updated weights for policy 0, policy_version 94670 (0.0009) [2023-10-14 00:52:37,935][60934] Updated weights for policy 1, policy_version 93482 (0.0008) [2023-10-14 00:52:38,148][60935] Updated weights for policy 0, policy_version 94680 (0.0009) [2023-10-14 00:52:38,297][60934] Updated weights for policy 1, policy_version 93492 (0.0008) [2023-10-14 00:52:41,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 194772992. Throughput: 0: 1684.4, 1: 1701.7. Samples: 48705708. Policy #0 lag: (min: 29.0, avg: 29.6, max: 45.0) [2023-10-14 00:52:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:52:42,250][60935] Updated weights for policy 0, policy_version 94690 (0.0009) [2023-10-14 00:52:42,555][60934] Updated weights for policy 1, policy_version 93502 (0.0008) [2023-10-14 00:52:42,615][60935] Updated weights for policy 0, policy_version 94700 (0.0008) [2023-10-14 00:52:42,934][60934] Updated weights for policy 1, policy_version 93512 (0.0007) [2023-10-14 00:52:42,992][60935] Updated weights for policy 0, policy_version 94710 (0.0008) [2023-10-14 00:52:43,296][60934] Updated weights for policy 1, policy_version 93522 (0.0009) [2023-10-14 00:52:43,356][60935] Updated weights for policy 0, policy_version 94720 (0.0007) [2023-10-14 00:52:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 194838528. Throughput: 0: 1664.8, 1: 1668.8. Samples: 48714676. Policy #0 lag: (min: 29.0, avg: 29.6, max: 45.0) [2023-10-14 00:52:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:52:47,149][60934] Updated weights for policy 1, policy_version 93532 (0.0008) [2023-10-14 00:52:47,286][60935] Updated weights for policy 0, policy_version 94730 (0.0009) [2023-10-14 00:52:47,522][60934] Updated weights for policy 1, policy_version 93542 (0.0008) [2023-10-14 00:52:47,663][60935] Updated weights for policy 0, policy_version 94740 (0.0010) [2023-10-14 00:52:47,893][60934] Updated weights for policy 1, policy_version 93552 (0.0009) [2023-10-14 00:52:48,035][60935] Updated weights for policy 0, policy_version 94750 (0.0008) [2023-10-14 00:52:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 194904064. Throughput: 0: 1695.8, 1: 1701.9. Samples: 48736180. Policy #0 lag: (min: 29.0, avg: 29.6, max: 45.0) [2023-10-14 00:52:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:52:51,992][60934] Updated weights for policy 1, policy_version 93562 (0.0009) [2023-10-14 00:52:52,033][60935] Updated weights for policy 0, policy_version 94760 (0.0010) [2023-10-14 00:52:52,362][60934] Updated weights for policy 1, policy_version 93572 (0.0008) [2023-10-14 00:52:52,403][60935] Updated weights for policy 0, policy_version 94770 (0.0009) [2023-10-14 00:52:52,726][60934] Updated weights for policy 1, policy_version 93582 (0.0008) [2023-10-14 00:52:52,771][60935] Updated weights for policy 0, policy_version 94780 (0.0008) [2023-10-14 00:52:53,094][60934] Updated weights for policy 1, policy_version 93592 (0.0007) [2023-10-14 00:52:56,249][59943] Fps is (10 sec: 13106.4, 60 sec: 13107.1, 300 sec: 13551.5). Total num frames: 194969600. Throughput: 0: 1695.4, 1: 1698.3. Samples: 48756982. Policy #0 lag: (min: 29.0, avg: 29.6, max: 45.0) [2023-10-14 00:52:56,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:52:56,264][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000093592_97910784.pth... [2023-10-14 00:52:56,264][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000094784_97058816.pth... [2023-10-14 00:52:56,302][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000092024_96305152.pth [2023-10-14 00:52:56,304][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000093216_95453184.pth [2023-10-14 00:52:56,308][60828] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p1/milestones/checkpoint_000093592_97910784.pth [2023-10-14 00:52:56,310][60695] Saving a milestone ./train_atari/atari_pitfall_APPO/checkpoint_p0/milestones/checkpoint_000094784_97058816.pth [2023-10-14 00:52:56,946][60935] Updated weights for policy 0, policy_version 94790 (0.0010) [2023-10-14 00:52:57,227][60934] Updated weights for policy 1, policy_version 93602 (0.0009) [2023-10-14 00:52:57,320][60935] Updated weights for policy 0, policy_version 94800 (0.0008) [2023-10-14 00:52:57,598][60934] Updated weights for policy 1, policy_version 93612 (0.0009) [2023-10-14 00:52:57,693][60935] Updated weights for policy 0, policy_version 94810 (0.0007) [2023-10-14 00:52:57,961][60934] Updated weights for policy 1, policy_version 93622 (0.0008) [2023-10-14 00:53:01,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 195035136. Throughput: 0: 1680.7, 1: 1680.4. Samples: 48765902. Policy #0 lag: (min: 29.0, avg: 29.6, max: 45.0) [2023-10-14 00:53:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:53:01,620][60935] Updated weights for policy 0, policy_version 94820 (0.0008) [2023-10-14 00:53:01,988][60935] Updated weights for policy 0, policy_version 94830 (0.0009) [2023-10-14 00:53:02,037][60934] Updated weights for policy 1, policy_version 93632 (0.0009) [2023-10-14 00:53:02,350][60935] Updated weights for policy 0, policy_version 94840 (0.0008) [2023-10-14 00:53:02,404][60934] Updated weights for policy 1, policy_version 93642 (0.0010) [2023-10-14 00:53:02,767][60934] Updated weights for policy 1, policy_version 93652 (0.0009) [2023-10-14 00:53:06,248][59943] Fps is (10 sec: 13107.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 195100672. Throughput: 0: 1697.9, 1: 1704.1. Samples: 48787344. Policy #0 lag: (min: 29.0, avg: 29.6, max: 45.0) [2023-10-14 00:53:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:53:06,396][60935] Updated weights for policy 0, policy_version 94850 (0.0007) [2023-10-14 00:53:06,715][60934] Updated weights for policy 1, policy_version 93662 (0.0007) [2023-10-14 00:53:06,750][60935] Updated weights for policy 0, policy_version 94860 (0.0008) [2023-10-14 00:53:07,086][60934] Updated weights for policy 1, policy_version 93672 (0.0010) [2023-10-14 00:53:07,113][60935] Updated weights for policy 0, policy_version 94870 (0.0011) [2023-10-14 00:53:07,445][60934] Updated weights for policy 1, policy_version 93682 (0.0010) [2023-10-14 00:53:07,477][60935] Updated weights for policy 0, policy_version 94880 (0.0011) [2023-10-14 00:53:11,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 195166208. Throughput: 0: 1704.5, 1: 1701.4. Samples: 48808280. Policy #0 lag: (min: 29.0, avg: 29.6, max: 45.0) [2023-10-14 00:53:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:53:11,356][60934] Updated weights for policy 1, policy_version 93692 (0.0009) [2023-10-14 00:53:11,482][60935] Updated weights for policy 0, policy_version 94890 (0.0008) [2023-10-14 00:53:11,718][60934] Updated weights for policy 1, policy_version 93702 (0.0007) [2023-10-14 00:53:11,853][60935] Updated weights for policy 0, policy_version 94900 (0.0008) [2023-10-14 00:53:12,084][60934] Updated weights for policy 1, policy_version 93712 (0.0008) [2023-10-14 00:53:12,223][60935] Updated weights for policy 0, policy_version 94910 (0.0009) [2023-10-14 00:53:15,913][60934] Updated weights for policy 1, policy_version 93722 (0.0009) [2023-10-14 00:53:16,064][60935] Updated weights for policy 0, policy_version 94920 (0.0009) [2023-10-14 00:53:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 195231744. Throughput: 0: 1706.5, 1: 1700.0. Samples: 48817680. Policy #0 lag: (min: 29.0, avg: 29.6, max: 45.0) [2023-10-14 00:53:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:53:16,272][60934] Updated weights for policy 1, policy_version 93732 (0.0007) [2023-10-14 00:53:16,430][60935] Updated weights for policy 0, policy_version 94930 (0.0009) [2023-10-14 00:53:16,637][60934] Updated weights for policy 1, policy_version 93742 (0.0007) [2023-10-14 00:53:16,803][60935] Updated weights for policy 0, policy_version 94940 (0.0008) [2023-10-14 00:53:16,999][60934] Updated weights for policy 1, policy_version 93752 (0.0009) [2023-10-14 00:53:20,935][60935] Updated weights for policy 0, policy_version 94950 (0.0008) [2023-10-14 00:53:21,132][60934] Updated weights for policy 1, policy_version 93762 (0.0008) [2023-10-14 00:53:21,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 195297280. Throughput: 0: 1708.5, 1: 1709.6. Samples: 48838574. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 00:53:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:53:21,295][60935] Updated weights for policy 0, policy_version 94960 (0.0008) [2023-10-14 00:53:21,488][60934] Updated weights for policy 1, policy_version 93772 (0.0008) [2023-10-14 00:53:21,664][60935] Updated weights for policy 0, policy_version 94970 (0.0008) [2023-10-14 00:53:21,848][60934] Updated weights for policy 1, policy_version 93782 (0.0008) [2023-10-14 00:53:25,811][60935] Updated weights for policy 0, policy_version 94980 (0.0008) [2023-10-14 00:53:25,951][60934] Updated weights for policy 1, policy_version 93792 (0.0008) [2023-10-14 00:53:26,176][60935] Updated weights for policy 0, policy_version 94990 (0.0008) [2023-10-14 00:53:26,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 195362816. Throughput: 0: 1700.4, 1: 1705.2. Samples: 48858956. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 00:53:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:53:26,319][60934] Updated weights for policy 1, policy_version 93802 (0.0007) [2023-10-14 00:53:26,546][60935] Updated weights for policy 0, policy_version 95000 (0.0007) [2023-10-14 00:53:26,689][60934] Updated weights for policy 1, policy_version 93812 (0.0009) [2023-10-14 00:53:30,789][60935] Updated weights for policy 0, policy_version 95010 (0.0009) [2023-10-14 00:53:30,919][60934] Updated weights for policy 1, policy_version 93822 (0.0008) [2023-10-14 00:53:31,164][60935] Updated weights for policy 0, policy_version 95020 (0.0007) [2023-10-14 00:53:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 195428352. Throughput: 0: 1701.6, 1: 1709.3. Samples: 48868168. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 00:53:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:53:31,302][60934] Updated weights for policy 1, policy_version 93832 (0.0008) [2023-10-14 00:53:31,533][60935] Updated weights for policy 0, policy_version 95030 (0.0008) [2023-10-14 00:53:31,670][60934] Updated weights for policy 1, policy_version 93842 (0.0008) [2023-10-14 00:53:31,894][60935] Updated weights for policy 0, policy_version 95040 (0.0009) [2023-10-14 00:53:35,741][60934] Updated weights for policy 1, policy_version 93852 (0.0009) [2023-10-14 00:53:35,887][60935] Updated weights for policy 0, policy_version 95050 (0.0009) [2023-10-14 00:53:36,107][60934] Updated weights for policy 1, policy_version 93862 (0.0007) [2023-10-14 00:53:36,247][60935] Updated weights for policy 0, policy_version 95060 (0.0009) [2023-10-14 00:53:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 195493888. Throughput: 0: 1691.3, 1: 1699.1. Samples: 48888746. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 00:53:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:53:36,473][60934] Updated weights for policy 1, policy_version 93872 (0.0009) [2023-10-14 00:53:36,617][60935] Updated weights for policy 0, policy_version 95070 (0.0010) [2023-10-14 00:53:40,261][60934] Updated weights for policy 1, policy_version 93882 (0.0008) [2023-10-14 00:53:40,632][60934] Updated weights for policy 1, policy_version 93892 (0.0008) [2023-10-14 00:53:40,922][60935] Updated weights for policy 0, policy_version 95080 (0.0009) [2023-10-14 00:53:41,001][60934] Updated weights for policy 1, policy_version 93902 (0.0009) [2023-10-14 00:53:41,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 195559424. Throughput: 0: 1680.4, 1: 1697.9. Samples: 48909004. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 00:53:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:53:41,290][60935] Updated weights for policy 0, policy_version 95090 (0.0010) [2023-10-14 00:53:41,366][60934] Updated weights for policy 1, policy_version 93912 (0.0009) [2023-10-14 00:53:41,673][60935] Updated weights for policy 0, policy_version 95100 (0.0010) [2023-10-14 00:53:45,462][60934] Updated weights for policy 1, policy_version 93922 (0.0009) [2023-10-14 00:53:45,796][60935] Updated weights for policy 0, policy_version 95110 (0.0008) [2023-10-14 00:53:45,826][60934] Updated weights for policy 1, policy_version 93932 (0.0008) [2023-10-14 00:53:46,151][60935] Updated weights for policy 0, policy_version 95120 (0.0009) [2023-10-14 00:53:46,186][60934] Updated weights for policy 1, policy_version 93942 (0.0007) [2023-10-14 00:53:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 195624960. Throughput: 0: 1686.7, 1: 1706.4. Samples: 48918592. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 00:53:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:53:46,529][60935] Updated weights for policy 0, policy_version 95130 (0.0008) [2023-10-14 00:53:50,236][60934] Updated weights for policy 1, policy_version 93952 (0.0009) [2023-10-14 00:53:50,558][60935] Updated weights for policy 0, policy_version 95140 (0.0008) [2023-10-14 00:53:50,596][60934] Updated weights for policy 1, policy_version 93962 (0.0009) [2023-10-14 00:53:50,930][60935] Updated weights for policy 0, policy_version 95150 (0.0008) [2023-10-14 00:53:50,972][60934] Updated weights for policy 1, policy_version 93972 (0.0008) [2023-10-14 00:53:51,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 195723264. Throughput: 0: 1674.8, 1: 1700.8. Samples: 48939246. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 00:53:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:53:51,289][60935] Updated weights for policy 0, policy_version 95160 (0.0009) [2023-10-14 00:53:54,860][60934] Updated weights for policy 1, policy_version 93982 (0.0008) [2023-10-14 00:53:55,222][60934] Updated weights for policy 1, policy_version 93992 (0.0008) [2023-10-14 00:53:55,450][60935] Updated weights for policy 0, policy_version 95170 (0.0008) [2023-10-14 00:53:55,587][60934] Updated weights for policy 1, policy_version 94002 (0.0008) [2023-10-14 00:53:55,817][60935] Updated weights for policy 0, policy_version 95180 (0.0007) [2023-10-14 00:53:56,184][60935] Updated weights for policy 0, policy_version 95190 (0.0010) [2023-10-14 00:53:56,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.5, 300 sec: 13440.4). Total num frames: 195788800. Throughput: 0: 1658.9, 1: 1685.2. Samples: 48958764. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 00:53:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:53:56,556][60935] Updated weights for policy 0, policy_version 95200 (0.0007) [2023-10-14 00:53:59,616][60934] Updated weights for policy 1, policy_version 94012 (0.0009) [2023-10-14 00:53:59,985][60934] Updated weights for policy 1, policy_version 94022 (0.0008) [2023-10-14 00:54:00,343][60934] Updated weights for policy 1, policy_version 94032 (0.0010) [2023-10-14 00:54:00,518][60935] Updated weights for policy 0, policy_version 95210 (0.0008) [2023-10-14 00:54:00,881][60935] Updated weights for policy 0, policy_version 95220 (0.0008) [2023-10-14 00:54:01,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 195854336. Throughput: 0: 1669.7, 1: 1705.6. Samples: 48969572. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 00:54:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:54:01,252][60935] Updated weights for policy 0, policy_version 95230 (0.0009) [2023-10-14 00:54:04,281][60934] Updated weights for policy 1, policy_version 94042 (0.0007) [2023-10-14 00:54:04,651][60934] Updated weights for policy 1, policy_version 94052 (0.0008) [2023-10-14 00:54:05,014][60934] Updated weights for policy 1, policy_version 94062 (0.0010) [2023-10-14 00:54:05,276][60935] Updated weights for policy 0, policy_version 95240 (0.0008) [2023-10-14 00:54:05,373][60934] Updated weights for policy 1, policy_version 94072 (0.0007) [2023-10-14 00:54:05,648][60935] Updated weights for policy 0, policy_version 95250 (0.0009) [2023-10-14 00:54:06,014][60935] Updated weights for policy 0, policy_version 95260 (0.0007) [2023-10-14 00:54:06,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 195952640. Throughput: 0: 1677.3, 1: 1700.0. Samples: 48990552. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 00:54:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:54:09,479][60934] Updated weights for policy 1, policy_version 94082 (0.0008) [2023-10-14 00:54:09,841][60934] Updated weights for policy 1, policy_version 94092 (0.0008) [2023-10-14 00:54:09,909][60935] Updated weights for policy 0, policy_version 95270 (0.0008) [2023-10-14 00:54:10,216][60934] Updated weights for policy 1, policy_version 94102 (0.0007) [2023-10-14 00:54:10,271][60935] Updated weights for policy 0, policy_version 95280 (0.0009) [2023-10-14 00:54:10,646][60935] Updated weights for policy 0, policy_version 95290 (0.0008) [2023-10-14 00:54:11,248][59943] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 196018176. Throughput: 0: 1661.6, 1: 1678.7. Samples: 49009266. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 00:54:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:54:14,131][60934] Updated weights for policy 1, policy_version 94112 (0.0009) [2023-10-14 00:54:14,499][60934] Updated weights for policy 1, policy_version 94122 (0.0008) [2023-10-14 00:54:14,658][60935] Updated weights for policy 0, policy_version 95300 (0.0008) [2023-10-14 00:54:14,867][60934] Updated weights for policy 1, policy_version 94132 (0.0007) [2023-10-14 00:54:15,018][60935] Updated weights for policy 0, policy_version 95310 (0.0007) [2023-10-14 00:54:15,380][60935] Updated weights for policy 0, policy_version 95320 (0.0009) [2023-10-14 00:54:16,248][59943] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 196083712. Throughput: 0: 1689.7, 1: 1709.2. Samples: 49021118. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-14 00:54:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:54:19,103][60934] Updated weights for policy 1, policy_version 94142 (0.0008) [2023-10-14 00:54:19,481][60934] Updated weights for policy 1, policy_version 94152 (0.0007) [2023-10-14 00:54:19,491][60935] Updated weights for policy 0, policy_version 95330 (0.0009) [2023-10-14 00:54:19,845][60934] Updated weights for policy 1, policy_version 94162 (0.0007) [2023-10-14 00:54:19,855][60935] Updated weights for policy 0, policy_version 95340 (0.0009) [2023-10-14 00:54:20,219][60935] Updated weights for policy 0, policy_version 95350 (0.0008) [2023-10-14 00:54:20,588][60935] Updated weights for policy 0, policy_version 95360 (0.0010) [2023-10-14 00:54:21,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 196149248. Throughput: 0: 1681.2, 1: 1689.3. Samples: 49040420. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-14 00:54:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:54:23,838][60934] Updated weights for policy 1, policy_version 94172 (0.0008) [2023-10-14 00:54:24,204][60934] Updated weights for policy 1, policy_version 94182 (0.0008) [2023-10-14 00:54:24,564][60934] Updated weights for policy 1, policy_version 94192 (0.0008) [2023-10-14 00:54:24,738][60935] Updated weights for policy 0, policy_version 95370 (0.0007) [2023-10-14 00:54:25,108][60935] Updated weights for policy 0, policy_version 95380 (0.0009) [2023-10-14 00:54:25,472][60935] Updated weights for policy 0, policy_version 95390 (0.0010) [2023-10-14 00:54:26,248][59943] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 196214784. Throughput: 0: 1669.5, 1: 1680.3. Samples: 49059742. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-14 00:54:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:54:28,581][60934] Updated weights for policy 1, policy_version 94202 (0.0008) [2023-10-14 00:54:28,949][60934] Updated weights for policy 1, policy_version 94212 (0.0011) [2023-10-14 00:54:29,313][60934] Updated weights for policy 1, policy_version 94222 (0.0009) [2023-10-14 00:54:29,518][60935] Updated weights for policy 0, policy_version 95400 (0.0010) [2023-10-14 00:54:29,675][60934] Updated weights for policy 1, policy_version 94232 (0.0009) [2023-10-14 00:54:29,896][60935] Updated weights for policy 0, policy_version 95410 (0.0008) [2023-10-14 00:54:30,262][60935] Updated weights for policy 0, policy_version 95420 (0.0009) [2023-10-14 00:54:31,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 196280320. Throughput: 0: 1697.0, 1: 1696.4. Samples: 49071296. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-14 00:54:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:54:33,673][60934] Updated weights for policy 1, policy_version 94242 (0.0009) [2023-10-14 00:54:34,034][60934] Updated weights for policy 1, policy_version 94252 (0.0008) [2023-10-14 00:54:34,385][60935] Updated weights for policy 0, policy_version 95430 (0.0008) [2023-10-14 00:54:34,405][60934] Updated weights for policy 1, policy_version 94262 (0.0007) [2023-10-14 00:54:34,770][60935] Updated weights for policy 0, policy_version 95440 (0.0009) [2023-10-14 00:54:35,145][60935] Updated weights for policy 0, policy_version 95450 (0.0010) [2023-10-14 00:54:36,248][59943] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 196345856. Throughput: 0: 1678.3, 1: 1675.3. Samples: 49090160. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-14 00:54:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:54:38,572][60934] Updated weights for policy 1, policy_version 94272 (0.0009) [2023-10-14 00:54:38,936][60934] Updated weights for policy 1, policy_version 94282 (0.0008) [2023-10-14 00:54:39,049][60935] Updated weights for policy 0, policy_version 95460 (0.0010) [2023-10-14 00:54:39,304][60934] Updated weights for policy 1, policy_version 94292 (0.0007) [2023-10-14 00:54:39,422][60935] Updated weights for policy 0, policy_version 95470 (0.0009) [2023-10-14 00:54:39,790][60935] Updated weights for policy 0, policy_version 95480 (0.0009) [2023-10-14 00:54:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 196411392. Throughput: 0: 1678.4, 1: 1693.7. Samples: 49110510. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-14 00:54:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:54:43,319][60934] Updated weights for policy 1, policy_version 94302 (0.0009) [2023-10-14 00:54:43,688][60934] Updated weights for policy 1, policy_version 94312 (0.0007) [2023-10-14 00:54:43,778][60935] Updated weights for policy 0, policy_version 95490 (0.0009) [2023-10-14 00:54:44,046][60934] Updated weights for policy 1, policy_version 94322 (0.0007) [2023-10-14 00:54:44,147][60935] Updated weights for policy 0, policy_version 95500 (0.0008) [2023-10-14 00:54:44,517][60935] Updated weights for policy 0, policy_version 95510 (0.0009) [2023-10-14 00:54:44,885][60935] Updated weights for policy 0, policy_version 95520 (0.0007) [2023-10-14 00:54:46,248][59943] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 196476928. Throughput: 0: 1695.2, 1: 1682.5. Samples: 49121566. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-14 00:54:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:54:48,106][60934] Updated weights for policy 1, policy_version 94332 (0.0007) [2023-10-14 00:54:48,472][60934] Updated weights for policy 1, policy_version 94342 (0.0009) [2023-10-14 00:54:48,824][60934] Updated weights for policy 1, policy_version 94352 (0.0009) [2023-10-14 00:54:49,101][60935] Updated weights for policy 0, policy_version 95530 (0.0007) [2023-10-14 00:54:49,473][60935] Updated weights for policy 0, policy_version 95540 (0.0007) [2023-10-14 00:54:49,840][60935] Updated weights for policy 0, policy_version 95550 (0.0007) [2023-10-14 00:54:51,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 196542464. Throughput: 0: 1662.3, 1: 1670.5. Samples: 49140526. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-14 00:54:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:54:52,896][60934] Updated weights for policy 1, policy_version 94362 (0.0010) [2023-10-14 00:54:53,273][60934] Updated weights for policy 1, policy_version 94372 (0.0008) [2023-10-14 00:54:53,635][60934] Updated weights for policy 1, policy_version 94382 (0.0007) [2023-10-14 00:54:53,908][60935] Updated weights for policy 0, policy_version 95560 (0.0009) [2023-10-14 00:54:53,996][60934] Updated weights for policy 1, policy_version 94392 (0.0008) [2023-10-14 00:54:54,271][60935] Updated weights for policy 0, policy_version 95570 (0.0009) [2023-10-14 00:54:54,641][60935] Updated weights for policy 0, policy_version 95580 (0.0010) [2023-10-14 00:54:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 196608000. Throughput: 0: 1680.8, 1: 1702.9. Samples: 49161532. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-14 00:54:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:54:56,260][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000095584_97878016.pth... [2023-10-14 00:54:56,261][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000094392_98729984.pth... [2023-10-14 00:54:56,300][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000094016_96272384.pth [2023-10-14 00:54:56,301][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000092824_97124352.pth [2023-10-14 00:54:57,796][60934] Updated weights for policy 1, policy_version 94402 (0.0008) [2023-10-14 00:54:58,161][60934] Updated weights for policy 1, policy_version 94412 (0.0008) [2023-10-14 00:54:58,533][60934] Updated weights for policy 1, policy_version 94422 (0.0009) [2023-10-14 00:54:58,555][60935] Updated weights for policy 0, policy_version 95590 (0.0009) [2023-10-14 00:54:58,922][60935] Updated weights for policy 0, policy_version 95600 (0.0009) [2023-10-14 00:54:59,304][60935] Updated weights for policy 0, policy_version 95610 (0.0011) [2023-10-14 00:55:01,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 196673536. Throughput: 0: 1672.2, 1: 1672.2. Samples: 49171616. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-14 00:55:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:55:02,756][60934] Updated weights for policy 1, policy_version 94432 (0.0009) [2023-10-14 00:55:03,123][60934] Updated weights for policy 1, policy_version 94442 (0.0007) [2023-10-14 00:55:03,332][60935] Updated weights for policy 0, policy_version 95620 (0.0007) [2023-10-14 00:55:03,493][60934] Updated weights for policy 1, policy_version 94452 (0.0007) [2023-10-14 00:55:03,695][60935] Updated weights for policy 0, policy_version 95630 (0.0009) [2023-10-14 00:55:04,059][60935] Updated weights for policy 0, policy_version 95640 (0.0009) [2023-10-14 00:55:06,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 196739072. Throughput: 0: 1666.7, 1: 1689.2. Samples: 49191436. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-14 00:55:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:55:07,575][60934] Updated weights for policy 1, policy_version 94462 (0.0008) [2023-10-14 00:55:07,961][60934] Updated weights for policy 1, policy_version 94472 (0.0007) [2023-10-14 00:55:08,025][60935] Updated weights for policy 0, policy_version 95650 (0.0008) [2023-10-14 00:55:08,325][60934] Updated weights for policy 1, policy_version 94482 (0.0008) [2023-10-14 00:55:08,394][60935] Updated weights for policy 0, policy_version 95660 (0.0009) [2023-10-14 00:55:08,756][60935] Updated weights for policy 0, policy_version 95670 (0.0008) [2023-10-14 00:55:09,123][60935] Updated weights for policy 0, policy_version 95680 (0.0008) [2023-10-14 00:55:11,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13551.5). Total num frames: 196804608. Throughput: 0: 1689.1, 1: 1701.9. Samples: 49212342. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-14 00:55:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:55:12,117][60934] Updated weights for policy 1, policy_version 94492 (0.0008) [2023-10-14 00:55:12,491][60934] Updated weights for policy 1, policy_version 94502 (0.0008) [2023-10-14 00:55:12,849][60934] Updated weights for policy 1, policy_version 94512 (0.0007) [2023-10-14 00:55:13,176][60935] Updated weights for policy 0, policy_version 95690 (0.0009) [2023-10-14 00:55:13,542][60935] Updated weights for policy 0, policy_version 95700 (0.0009) [2023-10-14 00:55:13,910][60935] Updated weights for policy 0, policy_version 95710 (0.0008) [2023-10-14 00:55:16,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 196870144. Throughput: 0: 1664.9, 1: 1679.0. Samples: 49221770. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-14 00:55:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:55:16,878][60934] Updated weights for policy 1, policy_version 94522 (0.0007) [2023-10-14 00:55:17,242][60934] Updated weights for policy 1, policy_version 94532 (0.0008) [2023-10-14 00:55:17,605][60934] Updated weights for policy 1, policy_version 94542 (0.0010) [2023-10-14 00:55:17,880][60935] Updated weights for policy 0, policy_version 95720 (0.0009) [2023-10-14 00:55:17,970][60934] Updated weights for policy 1, policy_version 94552 (0.0008) [2023-10-14 00:55:18,244][60935] Updated weights for policy 0, policy_version 95730 (0.0008) [2023-10-14 00:55:18,620][60935] Updated weights for policy 0, policy_version 95740 (0.0008) [2023-10-14 00:55:21,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 196935680. Throughput: 0: 1684.3, 1: 1703.7. Samples: 49242620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:55:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:55:21,993][60934] Updated weights for policy 1, policy_version 94562 (0.0009) [2023-10-14 00:55:22,355][60934] Updated weights for policy 1, policy_version 94572 (0.0009) [2023-10-14 00:55:22,722][60934] Updated weights for policy 1, policy_version 94582 (0.0008) [2023-10-14 00:55:22,811][60935] Updated weights for policy 0, policy_version 95750 (0.0009) [2023-10-14 00:55:23,190][60935] Updated weights for policy 0, policy_version 95760 (0.0011) [2023-10-14 00:55:23,557][60935] Updated weights for policy 0, policy_version 95770 (0.0008) [2023-10-14 00:55:26,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 197001216. Throughput: 0: 1698.1, 1: 1704.8. Samples: 49263642. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:55:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:55:26,749][60934] Updated weights for policy 1, policy_version 94592 (0.0008) [2023-10-14 00:55:27,117][60934] Updated weights for policy 1, policy_version 94602 (0.0007) [2023-10-14 00:55:27,484][60934] Updated weights for policy 1, policy_version 94612 (0.0007) [2023-10-14 00:55:27,612][60935] Updated weights for policy 0, policy_version 95780 (0.0007) [2023-10-14 00:55:27,984][60935] Updated weights for policy 0, policy_version 95790 (0.0009) [2023-10-14 00:55:28,345][60935] Updated weights for policy 0, policy_version 95800 (0.0009) [2023-10-14 00:55:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 197066752. Throughput: 0: 1668.5, 1: 1693.7. Samples: 49272866. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:55:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:55:31,401][60934] Updated weights for policy 1, policy_version 94622 (0.0007) [2023-10-14 00:55:31,761][60934] Updated weights for policy 1, policy_version 94632 (0.0008) [2023-10-14 00:55:32,131][60934] Updated weights for policy 1, policy_version 94642 (0.0008) [2023-10-14 00:55:32,523][60935] Updated weights for policy 0, policy_version 95810 (0.0009) [2023-10-14 00:55:32,884][60935] Updated weights for policy 0, policy_version 95820 (0.0009) [2023-10-14 00:55:33,254][60935] Updated weights for policy 0, policy_version 95830 (0.0007) [2023-10-14 00:55:33,615][60935] Updated weights for policy 0, policy_version 95840 (0.0008) [2023-10-14 00:55:36,071][60934] Updated weights for policy 1, policy_version 94652 (0.0009) [2023-10-14 00:55:36,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 197132288. Throughput: 0: 1695.7, 1: 1716.6. Samples: 49294076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:55:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:55:36,439][60934] Updated weights for policy 1, policy_version 94662 (0.0008) [2023-10-14 00:55:36,807][60934] Updated weights for policy 1, policy_version 94672 (0.0007) [2023-10-14 00:55:37,443][60935] Updated weights for policy 0, policy_version 95850 (0.0010) [2023-10-14 00:55:37,818][60935] Updated weights for policy 0, policy_version 95860 (0.0007) [2023-10-14 00:55:38,191][60935] Updated weights for policy 0, policy_version 95870 (0.0008) [2023-10-14 00:55:40,844][60934] Updated weights for policy 1, policy_version 94682 (0.0008) [2023-10-14 00:55:41,212][60934] Updated weights for policy 1, policy_version 94692 (0.0007) [2023-10-14 00:55:41,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 197197824. Throughput: 0: 1707.3, 1: 1710.6. Samples: 49315336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:55:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:55:41,575][60934] Updated weights for policy 1, policy_version 94702 (0.0007) [2023-10-14 00:55:41,938][60934] Updated weights for policy 1, policy_version 94712 (0.0008) [2023-10-14 00:55:42,077][60935] Updated weights for policy 0, policy_version 95880 (0.0009) [2023-10-14 00:55:42,441][60935] Updated weights for policy 0, policy_version 95890 (0.0009) [2023-10-14 00:55:42,814][60935] Updated weights for policy 0, policy_version 95900 (0.0007) [2023-10-14 00:55:45,971][60934] Updated weights for policy 1, policy_version 94722 (0.0009) [2023-10-14 00:55:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 197263360. Throughput: 0: 1687.8, 1: 1710.9. Samples: 49324560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:55:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:55:46,332][60934] Updated weights for policy 1, policy_version 94732 (0.0008) [2023-10-14 00:55:46,699][60934] Updated weights for policy 1, policy_version 94742 (0.0009) [2023-10-14 00:55:46,881][60935] Updated weights for policy 0, policy_version 95910 (0.0011) [2023-10-14 00:55:47,251][60935] Updated weights for policy 0, policy_version 95920 (0.0010) [2023-10-14 00:55:47,621][60935] Updated weights for policy 0, policy_version 95930 (0.0010) [2023-10-14 00:55:50,685][60934] Updated weights for policy 1, policy_version 94752 (0.0010) [2023-10-14 00:55:51,039][60934] Updated weights for policy 1, policy_version 94762 (0.0010) [2023-10-14 00:55:51,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 197328896. Throughput: 0: 1702.9, 1: 1715.5. Samples: 49345264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:55:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:55:51,400][60934] Updated weights for policy 1, policy_version 94772 (0.0010) [2023-10-14 00:55:51,712][60935] Updated weights for policy 0, policy_version 95940 (0.0010) [2023-10-14 00:55:52,079][60935] Updated weights for policy 0, policy_version 95950 (0.0008) [2023-10-14 00:55:52,449][60935] Updated weights for policy 0, policy_version 95960 (0.0008) [2023-10-14 00:55:55,411][60934] Updated weights for policy 1, policy_version 94782 (0.0008) [2023-10-14 00:55:55,795][60934] Updated weights for policy 1, policy_version 94792 (0.0007) [2023-10-14 00:55:56,153][60934] Updated weights for policy 1, policy_version 94802 (0.0007) [2023-10-14 00:55:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 197394432. Throughput: 0: 1709.2, 1: 1705.6. Samples: 49366006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:55:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:55:56,521][60935] Updated weights for policy 0, policy_version 95970 (0.0008) [2023-10-14 00:55:56,889][60935] Updated weights for policy 0, policy_version 95980 (0.0009) [2023-10-14 00:55:57,251][60935] Updated weights for policy 0, policy_version 95990 (0.0009) [2023-10-14 00:55:57,615][60935] Updated weights for policy 0, policy_version 96000 (0.0009) [2023-10-14 00:56:00,086][60934] Updated weights for policy 1, policy_version 94812 (0.0009) [2023-10-14 00:56:00,452][60934] Updated weights for policy 1, policy_version 94822 (0.0008) [2023-10-14 00:56:00,813][60934] Updated weights for policy 1, policy_version 94832 (0.0008) [2023-10-14 00:56:01,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 197492736. Throughput: 0: 1702.4, 1: 1715.3. Samples: 49375570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:56:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:56:01,592][60935] Updated weights for policy 0, policy_version 96010 (0.0008) [2023-10-14 00:56:01,949][60935] Updated weights for policy 0, policy_version 96020 (0.0008) [2023-10-14 00:56:02,324][60935] Updated weights for policy 0, policy_version 96030 (0.0008) [2023-10-14 00:56:04,908][60934] Updated weights for policy 1, policy_version 94842 (0.0007) [2023-10-14 00:56:05,286][60934] Updated weights for policy 1, policy_version 94852 (0.0008) [2023-10-14 00:56:05,648][60934] Updated weights for policy 1, policy_version 94862 (0.0010) [2023-10-14 00:56:06,014][60934] Updated weights for policy 1, policy_version 94872 (0.0009) [2023-10-14 00:56:06,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 197558272. Throughput: 0: 1706.2, 1: 1718.1. Samples: 49396710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:56:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:56:06,336][60935] Updated weights for policy 0, policy_version 96040 (0.0010) [2023-10-14 00:56:06,703][60935] Updated weights for policy 0, policy_version 96050 (0.0010) [2023-10-14 00:56:07,076][60935] Updated weights for policy 0, policy_version 96060 (0.0008) [2023-10-14 00:56:09,987][60934] Updated weights for policy 1, policy_version 94882 (0.0011) [2023-10-14 00:56:10,345][60934] Updated weights for policy 1, policy_version 94892 (0.0010) [2023-10-14 00:56:10,709][60934] Updated weights for policy 1, policy_version 94902 (0.0007) [2023-10-14 00:56:11,044][60935] Updated weights for policy 0, policy_version 96070 (0.0009) [2023-10-14 00:56:11,249][59943] Fps is (10 sec: 13106.6, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 197623808. Throughput: 0: 1707.6, 1: 1698.0. Samples: 49416898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:56:11,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:56:11,432][60935] Updated weights for policy 0, policy_version 96080 (0.0008) [2023-10-14 00:56:11,808][60935] Updated weights for policy 0, policy_version 96090 (0.0008) [2023-10-14 00:56:14,565][60934] Updated weights for policy 1, policy_version 94912 (0.0008) [2023-10-14 00:56:14,929][60934] Updated weights for policy 1, policy_version 94922 (0.0010) [2023-10-14 00:56:15,297][60934] Updated weights for policy 1, policy_version 94932 (0.0007) [2023-10-14 00:56:15,787][60935] Updated weights for policy 0, policy_version 96100 (0.0008) [2023-10-14 00:56:16,152][60935] Updated weights for policy 0, policy_version 96110 (0.0008) [2023-10-14 00:56:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 197689344. Throughput: 0: 1707.0, 1: 1723.8. Samples: 49427254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:56:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:56:16,512][60935] Updated weights for policy 0, policy_version 96120 (0.0009) [2023-10-14 00:56:19,349][60934] Updated weights for policy 1, policy_version 94942 (0.0008) [2023-10-14 00:56:19,713][60934] Updated weights for policy 1, policy_version 94952 (0.0007) [2023-10-14 00:56:20,076][60934] Updated weights for policy 1, policy_version 94962 (0.0010) [2023-10-14 00:56:20,592][60935] Updated weights for policy 0, policy_version 96130 (0.0008) [2023-10-14 00:56:20,959][60935] Updated weights for policy 0, policy_version 96140 (0.0007) [2023-10-14 00:56:21,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 197754880. Throughput: 0: 1705.1, 1: 1713.3. Samples: 49447902. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:56:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:56:21,322][60935] Updated weights for policy 0, policy_version 96150 (0.0008) [2023-10-14 00:56:21,693][60935] Updated weights for policy 0, policy_version 96160 (0.0012) [2023-10-14 00:56:24,066][60934] Updated weights for policy 1, policy_version 94972 (0.0007) [2023-10-14 00:56:24,433][60934] Updated weights for policy 1, policy_version 94982 (0.0007) [2023-10-14 00:56:24,802][60934] Updated weights for policy 1, policy_version 94992 (0.0007) [2023-10-14 00:56:25,619][60935] Updated weights for policy 0, policy_version 96170 (0.0010) [2023-10-14 00:56:25,976][60935] Updated weights for policy 0, policy_version 96180 (0.0011) [2023-10-14 00:56:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 197820416. Throughput: 0: 1688.6, 1: 1693.9. Samples: 49467548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:56:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:56:26,350][60935] Updated weights for policy 0, policy_version 96190 (0.0011) [2023-10-14 00:56:28,795][60934] Updated weights for policy 1, policy_version 95002 (0.0007) [2023-10-14 00:56:29,151][60934] Updated weights for policy 1, policy_version 95012 (0.0007) [2023-10-14 00:56:29,514][60934] Updated weights for policy 1, policy_version 95022 (0.0007) [2023-10-14 00:56:29,881][60934] Updated weights for policy 1, policy_version 95032 (0.0007) [2023-10-14 00:56:30,494][60935] Updated weights for policy 0, policy_version 96200 (0.0009) [2023-10-14 00:56:30,868][60935] Updated weights for policy 0, policy_version 96210 (0.0009) [2023-10-14 00:56:31,238][60935] Updated weights for policy 0, policy_version 96220 (0.0009) [2023-10-14 00:56:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 197885952. Throughput: 0: 1698.0, 1: 1721.6. Samples: 49478440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:56:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:56:33,976][60934] Updated weights for policy 1, policy_version 95042 (0.0009) [2023-10-14 00:56:34,332][60934] Updated weights for policy 1, policy_version 95052 (0.0009) [2023-10-14 00:56:34,689][60934] Updated weights for policy 1, policy_version 95062 (0.0009) [2023-10-14 00:56:35,119][60935] Updated weights for policy 0, policy_version 96230 (0.0011) [2023-10-14 00:56:35,498][60935] Updated weights for policy 0, policy_version 96240 (0.0011) [2023-10-14 00:56:35,855][60935] Updated weights for policy 0, policy_version 96250 (0.0011) [2023-10-14 00:56:36,248][59943] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 197984256. Throughput: 0: 1710.1, 1: 1701.8. Samples: 49498802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:56:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:56:38,577][60934] Updated weights for policy 1, policy_version 95072 (0.0009) [2023-10-14 00:56:38,954][60934] Updated weights for policy 1, policy_version 95082 (0.0008) [2023-10-14 00:56:39,320][60934] Updated weights for policy 1, policy_version 95092 (0.0009) [2023-10-14 00:56:39,791][60935] Updated weights for policy 0, policy_version 96260 (0.0010) [2023-10-14 00:56:40,161][60935] Updated weights for policy 0, policy_version 96270 (0.0008) [2023-10-14 00:56:40,533][60935] Updated weights for policy 0, policy_version 96280 (0.0009) [2023-10-14 00:56:41,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 198049792. Throughput: 0: 1680.1, 1: 1709.0. Samples: 49518516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:56:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:56:43,422][60934] Updated weights for policy 1, policy_version 95102 (0.0007) [2023-10-14 00:56:43,819][60934] Updated weights for policy 1, policy_version 95112 (0.0007) [2023-10-14 00:56:44,188][60934] Updated weights for policy 1, policy_version 95122 (0.0008) [2023-10-14 00:56:44,622][60935] Updated weights for policy 0, policy_version 96290 (0.0009) [2023-10-14 00:56:44,991][60935] Updated weights for policy 0, policy_version 96300 (0.0007) [2023-10-14 00:56:45,351][60935] Updated weights for policy 0, policy_version 96310 (0.0008) [2023-10-14 00:56:45,726][60935] Updated weights for policy 0, policy_version 96320 (0.0008) [2023-10-14 00:56:46,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 198115328. Throughput: 0: 1710.5, 1: 1715.8. Samples: 49529752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:56:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:56:48,225][60934] Updated weights for policy 1, policy_version 95132 (0.0008) [2023-10-14 00:56:48,574][60934] Updated weights for policy 1, policy_version 95142 (0.0008) [2023-10-14 00:56:48,941][60934] Updated weights for policy 1, policy_version 95152 (0.0009) [2023-10-14 00:56:49,874][60935] Updated weights for policy 0, policy_version 96330 (0.0010) [2023-10-14 00:56:50,239][60935] Updated weights for policy 0, policy_version 96340 (0.0009) [2023-10-14 00:56:50,608][60935] Updated weights for policy 0, policy_version 96350 (0.0009) [2023-10-14 00:56:51,248][59943] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 198180864. Throughput: 0: 1700.7, 1: 1690.7. Samples: 49549322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:56:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:56:52,983][60934] Updated weights for policy 1, policy_version 95162 (0.0007) [2023-10-14 00:56:53,350][60934] Updated weights for policy 1, policy_version 95172 (0.0008) [2023-10-14 00:56:53,705][60934] Updated weights for policy 1, policy_version 95182 (0.0009) [2023-10-14 00:56:54,073][60934] Updated weights for policy 1, policy_version 95192 (0.0008) [2023-10-14 00:56:54,578][60935] Updated weights for policy 0, policy_version 96360 (0.0008) [2023-10-14 00:56:54,951][60935] Updated weights for policy 0, policy_version 96370 (0.0008) [2023-10-14 00:56:55,312][60935] Updated weights for policy 0, policy_version 96380 (0.0008) [2023-10-14 00:56:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 198246400. Throughput: 0: 1679.2, 1: 1710.4. Samples: 49569432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:56:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:56:56,260][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000095192_99549184.pth... [2023-10-14 00:56:56,261][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000096384_98697216.pth... [2023-10-14 00:56:56,295][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000093592_97910784.pth [2023-10-14 00:56:56,308][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000094784_97058816.pth [2023-10-14 00:56:57,929][60934] Updated weights for policy 1, policy_version 95202 (0.0007) [2023-10-14 00:56:58,299][60934] Updated weights for policy 1, policy_version 95212 (0.0008) [2023-10-14 00:56:58,662][60934] Updated weights for policy 1, policy_version 95222 (0.0008) [2023-10-14 00:56:59,426][60935] Updated weights for policy 0, policy_version 96390 (0.0009) [2023-10-14 00:56:59,807][60935] Updated weights for policy 0, policy_version 96400 (0.0010) [2023-10-14 00:57:00,173][60935] Updated weights for policy 0, policy_version 96410 (0.0009) [2023-10-14 00:57:01,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 198311936. Throughput: 0: 1710.7, 1: 1688.9. Samples: 49580238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:57:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:57:02,719][60934] Updated weights for policy 1, policy_version 95232 (0.0010) [2023-10-14 00:57:03,093][60934] Updated weights for policy 1, policy_version 95242 (0.0007) [2023-10-14 00:57:03,462][60934] Updated weights for policy 1, policy_version 95252 (0.0009) [2023-10-14 00:57:04,176][60935] Updated weights for policy 0, policy_version 96420 (0.0008) [2023-10-14 00:57:04,543][60935] Updated weights for policy 0, policy_version 96430 (0.0008) [2023-10-14 00:57:04,912][60935] Updated weights for policy 0, policy_version 96440 (0.0008) [2023-10-14 00:57:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 198377472. Throughput: 0: 1687.9, 1: 1685.5. Samples: 49599704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:57:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:57:07,473][60934] Updated weights for policy 1, policy_version 95262 (0.0008) [2023-10-14 00:57:07,843][60934] Updated weights for policy 1, policy_version 95272 (0.0008) [2023-10-14 00:57:08,208][60934] Updated weights for policy 1, policy_version 95282 (0.0007) [2023-10-14 00:57:08,968][60935] Updated weights for policy 0, policy_version 96450 (0.0007) [2023-10-14 00:57:09,331][60935] Updated weights for policy 0, policy_version 96460 (0.0008) [2023-10-14 00:57:09,698][60935] Updated weights for policy 0, policy_version 96470 (0.0008) [2023-10-14 00:57:10,066][60935] Updated weights for policy 0, policy_version 96480 (0.0007) [2023-10-14 00:57:11,248][59943] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 198443008. Throughput: 0: 1686.1, 1: 1708.7. Samples: 49620312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:57:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:57:12,177][60934] Updated weights for policy 1, policy_version 95292 (0.0009) [2023-10-14 00:57:12,547][60934] Updated weights for policy 1, policy_version 95302 (0.0008) [2023-10-14 00:57:12,907][60934] Updated weights for policy 1, policy_version 95312 (0.0008) [2023-10-14 00:57:14,083][60935] Updated weights for policy 0, policy_version 96490 (0.0007) [2023-10-14 00:57:14,453][60935] Updated weights for policy 0, policy_version 96500 (0.0009) [2023-10-14 00:57:14,820][60935] Updated weights for policy 0, policy_version 96510 (0.0010) [2023-10-14 00:57:16,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 198508544. Throughput: 0: 1703.4, 1: 1677.9. Samples: 49630598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:57:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:57:17,027][60934] Updated weights for policy 1, policy_version 95322 (0.0008) [2023-10-14 00:57:17,394][60934] Updated weights for policy 1, policy_version 95332 (0.0007) [2023-10-14 00:57:17,758][60934] Updated weights for policy 1, policy_version 95342 (0.0007) [2023-10-14 00:57:18,124][60934] Updated weights for policy 1, policy_version 95352 (0.0007) [2023-10-14 00:57:18,735][60935] Updated weights for policy 0, policy_version 96520 (0.0008) [2023-10-14 00:57:19,107][60935] Updated weights for policy 0, policy_version 96530 (0.0008) [2023-10-14 00:57:19,478][60935] Updated weights for policy 0, policy_version 96540 (0.0007) [2023-10-14 00:57:21,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 198574080. Throughput: 0: 1672.4, 1: 1703.9. Samples: 49650736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 00:57:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:57:22,034][60934] Updated weights for policy 1, policy_version 95362 (0.0009) [2023-10-14 00:57:22,405][60934] Updated weights for policy 1, policy_version 95372 (0.0008) [2023-10-14 00:57:22,769][60934] Updated weights for policy 1, policy_version 95382 (0.0009) [2023-10-14 00:57:23,429][60935] Updated weights for policy 0, policy_version 96550 (0.0008) [2023-10-14 00:57:23,797][60935] Updated weights for policy 0, policy_version 96560 (0.0008) [2023-10-14 00:57:24,166][60935] Updated weights for policy 0, policy_version 96570 (0.0008) [2023-10-14 00:57:26,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 198639616. Throughput: 0: 1700.9, 1: 1708.1. Samples: 49671922. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:57:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:57:26,718][60934] Updated weights for policy 1, policy_version 95392 (0.0007) [2023-10-14 00:57:27,083][60934] Updated weights for policy 1, policy_version 95402 (0.0007) [2023-10-14 00:57:27,458][60934] Updated weights for policy 1, policy_version 95412 (0.0008) [2023-10-14 00:57:28,178][60935] Updated weights for policy 0, policy_version 96580 (0.0009) [2023-10-14 00:57:28,542][60935] Updated weights for policy 0, policy_version 96590 (0.0008) [2023-10-14 00:57:28,907][60935] Updated weights for policy 0, policy_version 96600 (0.0009) [2023-10-14 00:57:31,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 198705152. Throughput: 0: 1684.6, 1: 1690.2. Samples: 49681618. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:57:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-14 00:57:31,456][60934] Updated weights for policy 1, policy_version 95422 (0.0010) [2023-10-14 00:57:31,825][60934] Updated weights for policy 1, policy_version 95432 (0.0009) [2023-10-14 00:57:32,194][60934] Updated weights for policy 1, policy_version 95442 (0.0009) [2023-10-14 00:57:32,861][60935] Updated weights for policy 0, policy_version 96610 (0.0008) [2023-10-14 00:57:33,224][60935] Updated weights for policy 0, policy_version 96620 (0.0011) [2023-10-14 00:57:33,602][60935] Updated weights for policy 0, policy_version 96630 (0.0008) [2023-10-14 00:57:33,975][60935] Updated weights for policy 0, policy_version 96640 (0.0008) [2023-10-14 00:57:36,191][60934] Updated weights for policy 1, policy_version 95452 (0.0009) [2023-10-14 00:57:36,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 198770688. Throughput: 0: 1690.2, 1: 1710.3. Samples: 49702346. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:57:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-14 00:57:36,554][60934] Updated weights for policy 1, policy_version 95462 (0.0008) [2023-10-14 00:57:36,928][60934] Updated weights for policy 1, policy_version 95472 (0.0008) [2023-10-14 00:57:37,990][60935] Updated weights for policy 0, policy_version 96650 (0.0008) [2023-10-14 00:57:38,355][60935] Updated weights for policy 0, policy_version 96660 (0.0009) [2023-10-14 00:57:38,722][60935] Updated weights for policy 0, policy_version 96670 (0.0010) [2023-10-14 00:57:40,924][60934] Updated weights for policy 1, policy_version 95482 (0.0008) [2023-10-14 00:57:41,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 198836224. Throughput: 0: 1709.4, 1: 1715.6. Samples: 49723556. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:57:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-14 00:57:41,286][60934] Updated weights for policy 1, policy_version 95492 (0.0007) [2023-10-14 00:57:41,647][60934] Updated weights for policy 1, policy_version 95502 (0.0009) [2023-10-14 00:57:42,007][60934] Updated weights for policy 1, policy_version 95512 (0.0010) [2023-10-14 00:57:42,815][60935] Updated weights for policy 0, policy_version 96680 (0.0009) [2023-10-14 00:57:43,195][60935] Updated weights for policy 0, policy_version 96690 (0.0008) [2023-10-14 00:57:43,562][60935] Updated weights for policy 0, policy_version 96700 (0.0009) [2023-10-14 00:57:45,955][60934] Updated weights for policy 1, policy_version 95522 (0.0009) [2023-10-14 00:57:46,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13551.5). Total num frames: 198901760. Throughput: 0: 1682.0, 1: 1711.8. Samples: 49732958. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:57:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-14 00:57:46,315][60934] Updated weights for policy 1, policy_version 95532 (0.0008) [2023-10-14 00:57:46,675][60934] Updated weights for policy 1, policy_version 95542 (0.0007) [2023-10-14 00:57:47,533][60935] Updated weights for policy 0, policy_version 96710 (0.0009) [2023-10-14 00:57:47,920][60935] Updated weights for policy 0, policy_version 96720 (0.0008) [2023-10-14 00:57:48,289][60935] Updated weights for policy 0, policy_version 96730 (0.0010) [2023-10-14 00:57:50,740][60934] Updated weights for policy 1, policy_version 95552 (0.0008) [2023-10-14 00:57:51,102][60934] Updated weights for policy 1, policy_version 95562 (0.0009) [2023-10-14 00:57:51,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 198967296. Throughput: 0: 1705.1, 1: 1719.5. Samples: 49753810. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:57:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-14 00:57:51,476][60934] Updated weights for policy 1, policy_version 95572 (0.0007) [2023-10-14 00:57:52,185][60935] Updated weights for policy 0, policy_version 96740 (0.0010) [2023-10-14 00:57:52,550][60935] Updated weights for policy 0, policy_version 96750 (0.0009) [2023-10-14 00:57:52,922][60935] Updated weights for policy 0, policy_version 96760 (0.0008) [2023-10-14 00:57:55,412][60934] Updated weights for policy 1, policy_version 95582 (0.0008) [2023-10-14 00:57:55,777][60934] Updated weights for policy 1, policy_version 95592 (0.0009) [2023-10-14 00:57:56,144][60934] Updated weights for policy 1, policy_version 95602 (0.0009) [2023-10-14 00:57:56,248][59943] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 199032832. Throughput: 0: 1721.8, 1: 1710.3. Samples: 49774760. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:57:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-14 00:57:56,993][60935] Updated weights for policy 0, policy_version 96770 (0.0009) [2023-10-14 00:57:57,363][60935] Updated weights for policy 0, policy_version 96780 (0.0009) [2023-10-14 00:57:57,727][60935] Updated weights for policy 0, policy_version 96790 (0.0011) [2023-10-14 00:57:58,103][60935] Updated weights for policy 0, policy_version 96800 (0.0008) [2023-10-14 00:58:00,129][60934] Updated weights for policy 1, policy_version 95612 (0.0007) [2023-10-14 00:58:00,494][60934] Updated weights for policy 1, policy_version 95622 (0.0008) [2023-10-14 00:58:00,866][60934] Updated weights for policy 1, policy_version 95632 (0.0007) [2023-10-14 00:58:01,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 199131136. Throughput: 0: 1695.2, 1: 1720.4. Samples: 49784302. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:58:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-14 00:58:02,111][60935] Updated weights for policy 0, policy_version 96810 (0.0010) [2023-10-14 00:58:02,486][60935] Updated weights for policy 0, policy_version 96820 (0.0010) [2023-10-14 00:58:02,852][60935] Updated weights for policy 0, policy_version 96830 (0.0008) [2023-10-14 00:58:05,175][60934] Updated weights for policy 1, policy_version 95642 (0.0009) [2023-10-14 00:58:05,535][60934] Updated weights for policy 1, policy_version 95652 (0.0009) [2023-10-14 00:58:05,898][60934] Updated weights for policy 1, policy_version 95662 (0.0007) [2023-10-14 00:58:06,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 199163904. Throughput: 0: 1719.3, 1: 1714.8. Samples: 49805270. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:58:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-14 00:58:06,265][60934] Updated weights for policy 1, policy_version 95672 (0.0007) [2023-10-14 00:58:06,831][60935] Updated weights for policy 0, policy_version 96840 (0.0008) [2023-10-14 00:58:07,198][60935] Updated weights for policy 0, policy_version 96850 (0.0010) [2023-10-14 00:58:07,575][60935] Updated weights for policy 0, policy_version 96860 (0.0011) [2023-10-14 00:58:10,347][60934] Updated weights for policy 1, policy_version 95682 (0.0008) [2023-10-14 00:58:10,711][60934] Updated weights for policy 1, policy_version 95692 (0.0009) [2023-10-14 00:58:11,075][60934] Updated weights for policy 1, policy_version 95702 (0.0010) [2023-10-14 00:58:11,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 199262208. Throughput: 0: 1720.0, 1: 1696.2. Samples: 49825650. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:58:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-14 00:58:11,519][60935] Updated weights for policy 0, policy_version 96870 (0.0011) [2023-10-14 00:58:11,878][60935] Updated weights for policy 0, policy_version 96880 (0.0011) [2023-10-14 00:58:12,248][60935] Updated weights for policy 0, policy_version 96890 (0.0011) [2023-10-14 00:58:15,057][60934] Updated weights for policy 1, policy_version 95712 (0.0008) [2023-10-14 00:58:15,418][60934] Updated weights for policy 1, policy_version 95722 (0.0010) [2023-10-14 00:58:15,785][60934] Updated weights for policy 1, policy_version 95732 (0.0008) [2023-10-14 00:58:16,248][59943] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 199327744. Throughput: 0: 1709.7, 1: 1714.5. Samples: 49835702. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:58:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-14 00:58:16,267][60935] Updated weights for policy 0, policy_version 96900 (0.0009) [2023-10-14 00:58:16,637][60935] Updated weights for policy 0, policy_version 96910 (0.0008) [2023-10-14 00:58:17,008][60935] Updated weights for policy 0, policy_version 96920 (0.0008) [2023-10-14 00:58:19,958][60934] Updated weights for policy 1, policy_version 95742 (0.0008) [2023-10-14 00:58:20,340][60934] Updated weights for policy 1, policy_version 95752 (0.0009) [2023-10-14 00:58:20,700][60934] Updated weights for policy 1, policy_version 95762 (0.0009) [2023-10-14 00:58:20,880][60935] Updated weights for policy 0, policy_version 96930 (0.0007) [2023-10-14 00:58:21,245][60935] Updated weights for policy 0, policy_version 96940 (0.0011) [2023-10-14 00:58:21,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 199393280. Throughput: 0: 1717.2, 1: 1713.4. Samples: 49856724. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:58:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-14 00:58:21,610][60935] Updated weights for policy 0, policy_version 96950 (0.0010) [2023-10-14 00:58:21,977][60935] Updated weights for policy 0, policy_version 96960 (0.0011) [2023-10-14 00:58:24,613][60934] Updated weights for policy 1, policy_version 95772 (0.0007) [2023-10-14 00:58:24,984][60934] Updated weights for policy 1, policy_version 95782 (0.0008) [2023-10-14 00:58:25,350][60934] Updated weights for policy 1, policy_version 95792 (0.0010) [2023-10-14 00:58:25,963][60935] Updated weights for policy 0, policy_version 96970 (0.0009) [2023-10-14 00:58:26,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 199458816. Throughput: 0: 1709.4, 1: 1682.8. Samples: 49876206. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 00:58:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-14 00:58:26,331][60935] Updated weights for policy 0, policy_version 96980 (0.0009) [2023-10-14 00:58:26,703][60935] Updated weights for policy 0, policy_version 96990 (0.0010) [2023-10-14 00:58:29,134][60934] Updated weights for policy 1, policy_version 95802 (0.0009) [2023-10-14 00:58:29,501][60934] Updated weights for policy 1, policy_version 95812 (0.0007) [2023-10-14 00:58:29,871][60934] Updated weights for policy 1, policy_version 95822 (0.0007) [2023-10-14 00:58:30,227][60934] Updated weights for policy 1, policy_version 95832 (0.0007) [2023-10-14 00:58:30,648][60935] Updated weights for policy 0, policy_version 97000 (0.0008) [2023-10-14 00:58:31,015][60935] Updated weights for policy 0, policy_version 97010 (0.0009) [2023-10-14 00:58:31,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 199524352. Throughput: 0: 1713.4, 1: 1712.4. Samples: 49887118. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-14 00:58:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-14 00:58:31,386][60935] Updated weights for policy 0, policy_version 97020 (0.0010) [2023-10-14 00:58:34,260][60934] Updated weights for policy 1, policy_version 95842 (0.0007) [2023-10-14 00:58:34,626][60934] Updated weights for policy 1, policy_version 95852 (0.0007) [2023-10-14 00:58:34,989][60934] Updated weights for policy 1, policy_version 95862 (0.0007) [2023-10-14 00:58:35,518][60935] Updated weights for policy 0, policy_version 97030 (0.0011) [2023-10-14 00:58:35,899][60935] Updated weights for policy 0, policy_version 97040 (0.0008) [2023-10-14 00:58:36,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 199589888. Throughput: 0: 1715.2, 1: 1699.7. Samples: 49907478. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-14 00:58:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '-0.100')] [2023-10-14 00:58:36,259][60935] Updated weights for policy 0, policy_version 97050 (0.0007) [2023-10-14 00:58:38,856][60934] Updated weights for policy 1, policy_version 95872 (0.0008) [2023-10-14 00:58:39,221][60934] Updated weights for policy 1, policy_version 95882 (0.0008) [2023-10-14 00:58:39,588][60934] Updated weights for policy 1, policy_version 95892 (0.0007) [2023-10-14 00:58:40,279][60935] Updated weights for policy 0, policy_version 97060 (0.0008) [2023-10-14 00:58:40,641][60935] Updated weights for policy 0, policy_version 97070 (0.0008) [2023-10-14 00:58:41,009][60935] Updated weights for policy 0, policy_version 97080 (0.0007) [2023-10-14 00:58:41,248][59943] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13662.6). Total num frames: 199655424. Throughput: 0: 1690.8, 1: 1696.5. Samples: 49927188. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-14 00:58:41,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:58:43,666][60934] Updated weights for policy 1, policy_version 95902 (0.0009) [2023-10-14 00:58:44,039][60934] Updated weights for policy 1, policy_version 95912 (0.0008) [2023-10-14 00:58:44,400][60934] Updated weights for policy 1, policy_version 95922 (0.0007) [2023-10-14 00:58:44,952][60935] Updated weights for policy 0, policy_version 97090 (0.0007) [2023-10-14 00:58:45,313][60935] Updated weights for policy 0, policy_version 97100 (0.0008) [2023-10-14 00:58:45,681][60935] Updated weights for policy 0, policy_version 97110 (0.0009) [2023-10-14 00:58:46,036][60935] Updated weights for policy 0, policy_version 97120 (0.0007) [2023-10-14 00:58:46,248][59943] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 13662.6). Total num frames: 199753728. Throughput: 0: 1707.6, 1: 1717.5. Samples: 49938432. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-14 00:58:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:58:48,125][60934] Updated weights for policy 1, policy_version 95932 (0.0008) [2023-10-14 00:58:48,492][60934] Updated weights for policy 1, policy_version 95942 (0.0012) [2023-10-14 00:58:48,859][60934] Updated weights for policy 1, policy_version 95952 (0.0007) [2023-10-14 00:58:50,052][60935] Updated weights for policy 0, policy_version 97130 (0.0008) [2023-10-14 00:58:50,418][60935] Updated weights for policy 0, policy_version 97140 (0.0009) [2023-10-14 00:58:50,792][60935] Updated weights for policy 0, policy_version 97150 (0.0010) [2023-10-14 00:58:51,248][59943] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 199819264. Throughput: 0: 1709.7, 1: 1694.0. Samples: 49958436. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-14 00:58:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:58:52,883][60934] Updated weights for policy 1, policy_version 95962 (0.0008) [2023-10-14 00:58:53,254][60934] Updated weights for policy 1, policy_version 95972 (0.0010) [2023-10-14 00:58:53,623][60934] Updated weights for policy 1, policy_version 95982 (0.0010) [2023-10-14 00:58:53,979][60934] Updated weights for policy 1, policy_version 95992 (0.0011) [2023-10-14 00:58:54,468][60935] Updated weights for policy 0, policy_version 97160 (0.0008) [2023-10-14 00:58:54,841][60935] Updated weights for policy 0, policy_version 97170 (0.0008) [2023-10-14 00:58:55,206][60935] Updated weights for policy 0, policy_version 97180 (0.0010) [2023-10-14 00:58:56,248][59943] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13662.6). Total num frames: 199884800. Throughput: 0: 1688.5, 1: 1709.0. Samples: 49978538. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-14 00:58:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:58:56,257][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000097184_99516416.pth... [2023-10-14 00:58:56,257][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000095992_100368384.pth... [2023-10-14 00:58:56,288][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000094392_98729984.pth [2023-10-14 00:58:56,295][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000095584_97878016.pth [2023-10-14 00:58:58,110][60934] Updated weights for policy 1, policy_version 96002 (0.0007) [2023-10-14 00:58:58,466][60934] Updated weights for policy 1, policy_version 96012 (0.0009) [2023-10-14 00:58:58,835][60934] Updated weights for policy 1, policy_version 96022 (0.0008) [2023-10-14 00:58:59,258][60935] Updated weights for policy 0, policy_version 97190 (0.0008) [2023-10-14 00:58:59,621][60935] Updated weights for policy 0, policy_version 97200 (0.0007) [2023-10-14 00:58:59,987][60935] Updated weights for policy 0, policy_version 97210 (0.0007) [2023-10-14 00:59:01,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 199950336. Throughput: 0: 1716.6, 1: 1699.7. Samples: 49989434. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-14 00:59:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:59:02,913][60934] Updated weights for policy 1, policy_version 96032 (0.0009) [2023-10-14 00:59:03,269][60934] Updated weights for policy 1, policy_version 96042 (0.0008) [2023-10-14 00:59:03,630][60934] Updated weights for policy 1, policy_version 96052 (0.0008) [2023-10-14 00:59:04,172][60935] Updated weights for policy 0, policy_version 97220 (0.0009) [2023-10-14 00:59:04,532][60935] Updated weights for policy 0, policy_version 97230 (0.0009) [2023-10-14 00:59:04,902][60935] Updated weights for policy 0, policy_version 97240 (0.0009) [2023-10-14 00:59:06,248][59943] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 200015872. Throughput: 0: 1695.2, 1: 1696.0. Samples: 50009326. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-14 00:59:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:59:07,726][60934] Updated weights for policy 1, policy_version 96062 (0.0009) [2023-10-14 00:59:08,118][60934] Updated weights for policy 1, policy_version 96072 (0.0007) [2023-10-14 00:59:08,477][60934] Updated weights for policy 1, policy_version 96082 (0.0008) [2023-10-14 00:59:08,820][60935] Updated weights for policy 0, policy_version 97250 (0.0009) [2023-10-14 00:59:09,181][60935] Updated weights for policy 0, policy_version 97260 (0.0008) [2023-10-14 00:59:09,553][60935] Updated weights for policy 0, policy_version 97270 (0.0007) [2023-10-14 00:59:09,920][60935] Updated weights for policy 0, policy_version 97280 (0.0008) [2023-10-14 00:59:11,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 200081408. Throughput: 0: 1696.0, 1: 1718.4. Samples: 50029858. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-14 00:59:11,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:59:12,309][60934] Updated weights for policy 1, policy_version 96092 (0.0009) [2023-10-14 00:59:12,686][60934] Updated weights for policy 1, policy_version 96102 (0.0008) [2023-10-14 00:59:13,065][60934] Updated weights for policy 1, policy_version 96112 (0.0009) [2023-10-14 00:59:13,870][60935] Updated weights for policy 0, policy_version 97290 (0.0010) [2023-10-14 00:59:14,229][60935] Updated weights for policy 0, policy_version 97300 (0.0009) [2023-10-14 00:59:14,589][60935] Updated weights for policy 0, policy_version 97310 (0.0008) [2023-10-14 00:59:16,248][59943] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 200146944. Throughput: 0: 1714.0, 1: 1686.5. Samples: 50040144. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-14 00:59:16,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:59:16,940][60934] Updated weights for policy 1, policy_version 96122 (0.0009) [2023-10-14 00:59:17,308][60934] Updated weights for policy 1, policy_version 96132 (0.0008) [2023-10-14 00:59:17,671][60934] Updated weights for policy 1, policy_version 96142 (0.0010) [2023-10-14 00:59:18,030][60934] Updated weights for policy 1, policy_version 96152 (0.0009) [2023-10-14 00:59:18,584][60935] Updated weights for policy 0, policy_version 97320 (0.0007) [2023-10-14 00:59:18,958][60935] Updated weights for policy 0, policy_version 97330 (0.0007) [2023-10-14 00:59:19,329][60935] Updated weights for policy 0, policy_version 97340 (0.0008) [2023-10-14 00:59:21,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 200212480. Throughput: 0: 1698.0, 1: 1704.1. Samples: 50060574. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-14 00:59:21,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:59:22,048][60934] Updated weights for policy 1, policy_version 96162 (0.0010) [2023-10-14 00:59:22,412][60934] Updated weights for policy 1, policy_version 96172 (0.0009) [2023-10-14 00:59:22,778][60934] Updated weights for policy 1, policy_version 96182 (0.0007) [2023-10-14 00:59:23,446][60935] Updated weights for policy 0, policy_version 97350 (0.0009) [2023-10-14 00:59:23,824][60935] Updated weights for policy 0, policy_version 97360 (0.0009) [2023-10-14 00:59:24,191][60935] Updated weights for policy 0, policy_version 97370 (0.0009) [2023-10-14 00:59:26,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 200278016. Throughput: 0: 1714.4, 1: 1712.2. Samples: 50081384. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-14 00:59:26,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:59:26,748][60934] Updated weights for policy 1, policy_version 96192 (0.0007) [2023-10-14 00:59:27,103][60934] Updated weights for policy 1, policy_version 96202 (0.0009) [2023-10-14 00:59:27,472][60934] Updated weights for policy 1, policy_version 96212 (0.0007) [2023-10-14 00:59:28,281][60935] Updated weights for policy 0, policy_version 97380 (0.0009) [2023-10-14 00:59:28,648][60935] Updated weights for policy 0, policy_version 97390 (0.0011) [2023-10-14 00:59:29,026][60935] Updated weights for policy 0, policy_version 97400 (0.0009) [2023-10-14 00:59:31,248][59943] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 200343552. Throughput: 0: 1711.8, 1: 1682.8. Samples: 50091190. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 00:59:31,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:59:31,481][60934] Updated weights for policy 1, policy_version 96222 (0.0008) [2023-10-14 00:59:31,852][60934] Updated weights for policy 1, policy_version 96232 (0.0008) [2023-10-14 00:59:32,206][60934] Updated weights for policy 1, policy_version 96242 (0.0008) [2023-10-14 00:59:33,043][60935] Updated weights for policy 0, policy_version 97410 (0.0008) [2023-10-14 00:59:33,412][60935] Updated weights for policy 0, policy_version 97420 (0.0010) [2023-10-14 00:59:33,793][60935] Updated weights for policy 0, policy_version 97430 (0.0011) [2023-10-14 00:59:34,164][60935] Updated weights for policy 0, policy_version 97440 (0.0010) [2023-10-14 00:59:36,248][59943] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 200409088. Throughput: 0: 1695.4, 1: 1708.3. Samples: 50111604. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 00:59:36,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:59:36,378][60934] Updated weights for policy 1, policy_version 96252 (0.0009) [2023-10-14 00:59:36,754][60934] Updated weights for policy 1, policy_version 96262 (0.0008) [2023-10-14 00:59:37,119][60934] Updated weights for policy 1, policy_version 96272 (0.0010) [2023-10-14 00:59:37,939][60935] Updated weights for policy 0, policy_version 97450 (0.0009) [2023-10-14 00:59:38,303][60935] Updated weights for policy 0, policy_version 97460 (0.0009) [2023-10-14 00:59:38,680][60935] Updated weights for policy 0, policy_version 97470 (0.0008) [2023-10-14 00:59:41,249][59943] Fps is (10 sec: 13106.6, 60 sec: 13653.2, 300 sec: 13551.5). Total num frames: 200474624. Throughput: 0: 1718.0, 1: 1709.9. Samples: 50132798. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 00:59:41,250][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:59:41,293][60934] Updated weights for policy 1, policy_version 96282 (0.0008) [2023-10-14 00:59:41,654][60934] Updated weights for policy 1, policy_version 96292 (0.0010) [2023-10-14 00:59:42,016][60934] Updated weights for policy 1, policy_version 96302 (0.0010) [2023-10-14 00:59:42,378][60934] Updated weights for policy 1, policy_version 96312 (0.0010) [2023-10-14 00:59:42,733][60935] Updated weights for policy 0, policy_version 97480 (0.0008) [2023-10-14 00:59:43,109][60935] Updated weights for policy 0, policy_version 97490 (0.0009) [2023-10-14 00:59:43,469][60935] Updated weights for policy 0, policy_version 97500 (0.0008) [2023-10-14 00:59:46,248][59943] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 200540160. Throughput: 0: 1689.1, 1: 1702.6. Samples: 50142062. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 00:59:46,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:59:46,284][60934] Updated weights for policy 1, policy_version 96322 (0.0009) [2023-10-14 00:59:46,643][60934] Updated weights for policy 1, policy_version 96332 (0.0008) [2023-10-14 00:59:47,010][60934] Updated weights for policy 1, policy_version 96342 (0.0008) [2023-10-14 00:59:47,280][60935] Updated weights for policy 0, policy_version 97510 (0.0008) [2023-10-14 00:59:47,646][60935] Updated weights for policy 0, policy_version 97520 (0.0009) [2023-10-14 00:59:48,018][60935] Updated weights for policy 0, policy_version 97530 (0.0007) [2023-10-14 00:59:51,125][60934] Updated weights for policy 1, policy_version 96352 (0.0007) [2023-10-14 00:59:51,248][59943] Fps is (10 sec: 13107.8, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 200605696. Throughput: 0: 1712.8, 1: 1709.2. Samples: 50163314. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 00:59:51,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:59:51,489][60934] Updated weights for policy 1, policy_version 96362 (0.0007) [2023-10-14 00:59:51,864][60934] Updated weights for policy 1, policy_version 96372 (0.0009) [2023-10-14 00:59:52,061][60935] Updated weights for policy 0, policy_version 97540 (0.0010) [2023-10-14 00:59:52,429][60935] Updated weights for policy 0, policy_version 97550 (0.0009) [2023-10-14 00:59:52,799][60935] Updated weights for policy 0, policy_version 97560 (0.0009) [2023-10-14 00:59:55,790][60934] Updated weights for policy 1, policy_version 96382 (0.0007) [2023-10-14 00:59:56,173][60934] Updated weights for policy 1, policy_version 96392 (0.0008) [2023-10-14 00:59:56,248][59943] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 200671232. Throughput: 0: 1720.2, 1: 1715.3. Samples: 50184456. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 00:59:56,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 00:59:56,550][60934] Updated weights for policy 1, policy_version 96402 (0.0008) [2023-10-14 00:59:56,782][60935] Updated weights for policy 0, policy_version 97570 (0.0009) [2023-10-14 00:59:57,143][60935] Updated weights for policy 0, policy_version 97580 (0.0010) [2023-10-14 00:59:57,523][60935] Updated weights for policy 0, policy_version 97590 (0.0007) [2023-10-14 00:59:57,893][60935] Updated weights for policy 0, policy_version 97600 (0.0007) [2023-10-14 01:00:00,372][60934] Updated weights for policy 1, policy_version 96412 (0.0008) [2023-10-14 01:00:00,745][60934] Updated weights for policy 1, policy_version 96422 (0.0010) [2023-10-14 01:00:01,106][60934] Updated weights for policy 1, policy_version 96432 (0.0007) [2023-10-14 01:00:01,248][59943] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13551.5). Total num frames: 200736768. Throughput: 0: 1696.9, 1: 1715.8. Samples: 50193718. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 01:00:01,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 01:00:02,031][60935] Updated weights for policy 0, policy_version 97610 (0.0010) [2023-10-14 01:00:02,409][60935] Updated weights for policy 0, policy_version 97620 (0.0008) [2023-10-14 01:00:02,786][60935] Updated weights for policy 0, policy_version 97630 (0.0009) [2023-10-14 01:00:05,060][60934] Updated weights for policy 1, policy_version 96442 (0.0007) [2023-10-14 01:00:05,429][60934] Updated weights for policy 1, policy_version 96452 (0.0010) [2023-10-14 01:00:05,787][60934] Updated weights for policy 1, policy_version 96462 (0.0010) [2023-10-14 01:00:06,147][60934] Updated weights for policy 1, policy_version 96472 (0.0010) [2023-10-14 01:00:06,248][59943] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13662.6). Total num frames: 200835072. Throughput: 0: 1713.3, 1: 1717.3. Samples: 50214948. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 01:00:06,249][59943] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 01:00:06,712][60935] Updated weights for policy 0, policy_version 97640 (0.0011) [2023-10-14 01:00:07,078][60935] Updated weights for policy 0, policy_version 97650 (0.0009) [2023-10-14 01:00:07,447][60935] Updated weights for policy 0, policy_version 97660 (0.0008) [2023-10-14 01:00:09,879][60934] Updated weights for policy 1, policy_version 96482 (0.0008) [2023-10-14 01:00:10,239][60934] Updated weights for policy 1, policy_version 96492 (0.0008) [2023-10-14 01:00:10,611][60934] Updated weights for policy 1, policy_version 96502 (0.0008) [2023-10-14 01:00:10,675][60984] Stopping RolloutWorker_w8... [2023-10-14 01:00:10,675][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000096504_100892672.pth... [2023-10-14 01:00:10,676][60986] Stopping RolloutWorker_w10... [2023-10-14 01:00:10,676][60974] Stopping RolloutWorker_w5... [2023-10-14 01:00:10,676][60998] Stopping RolloutWorker_w11... [2023-10-14 01:00:10,676][60970] Stopping RolloutWorker_w1... [2023-10-14 01:00:10,676][60984] Loop rollout_proc8_evt_loop terminating... [2023-10-14 01:00:10,676][60971] Stopping RolloutWorker_w2... [2023-10-14 01:00:10,676][60695] Stopping Batcher_0... [2023-10-14 01:00:10,676][60983] Stopping RolloutWorker_w9... [2023-10-14 01:00:10,676][60986] Loop rollout_proc10_evt_loop terminating... [2023-10-14 01:00:10,676][60970] Loop rollout_proc1_evt_loop terminating... [2023-10-14 01:00:10,676][60998] Loop rollout_proc11_evt_loop terminating... [2023-10-14 01:00:10,676][60971] Loop rollout_proc2_evt_loop terminating... [2023-10-14 01:00:10,676][60974] Loop rollout_proc5_evt_loop terminating... [2023-10-14 01:00:10,676][59943] Component RolloutWorker_w8 stopped! [2023-10-14 01:00:10,676][60983] Loop rollout_proc9_evt_loop terminating... [2023-10-14 01:00:10,676][60695] Loop batcher_evt_loop terminating... [2023-10-14 01:00:10,676][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000097664_100007936.pth... [2023-10-14 01:00:10,677][59943] Component RolloutWorker_w10 stopped! [2023-10-14 01:00:10,677][60968] Stopping RolloutWorker_w0... [2023-10-14 01:00:10,677][60975] Stopping RolloutWorker_w6... [2023-10-14 01:00:10,677][60968] Loop rollout_proc0_evt_loop terminating... [2023-10-14 01:00:10,677][59943] Component RolloutWorker_w5 stopped! [2023-10-14 01:00:10,677][60975] Loop rollout_proc6_evt_loop terminating... [2023-10-14 01:00:10,677][59943] Component RolloutWorker_w11 stopped! [2023-10-14 01:00:10,678][60976] Stopping RolloutWorker_w7... [2023-10-14 01:00:10,678][61664] Stopping RolloutWorker_w15... [2023-10-14 01:00:10,678][59943] Component RolloutWorker_w1 stopped! [2023-10-14 01:00:10,678][60976] Loop rollout_proc7_evt_loop terminating... [2023-10-14 01:00:10,678][61664] Loop rollout_proc15_evt_loop terminating... [2023-10-14 01:00:10,678][59943] Component RolloutWorker_w2 stopped! [2023-10-14 01:00:10,679][60973] Stopping RolloutWorker_w4... [2023-10-14 01:00:10,679][59943] Component Batcher_0 stopped! [2023-10-14 01:00:10,679][61663] Stopping RolloutWorker_w14... [2023-10-14 01:00:10,679][60973] Loop rollout_proc4_evt_loop terminating... [2023-10-14 01:00:10,679][61663] Loop rollout_proc14_evt_loop terminating... [2023-10-14 01:00:10,679][59943] Component RolloutWorker_w9 stopped! [2023-10-14 01:00:10,680][59943] Component RolloutWorker_w0 stopped! [2023-10-14 01:00:10,680][59943] Component RolloutWorker_w6 stopped! [2023-10-14 01:00:10,680][59943] Component RolloutWorker_w7 stopped! [2023-10-14 01:00:10,681][59943] Component RolloutWorker_w15 stopped! [2023-10-14 01:00:10,681][59943] Component RolloutWorker_w4 stopped! [2023-10-14 01:00:10,682][59943] Component RolloutWorker_w14 stopped! [2023-10-14 01:00:10,682][60997] Stopping RolloutWorker_w13... [2023-10-14 01:00:10,682][60996] Stopping RolloutWorker_w12... [2023-10-14 01:00:10,682][60997] Loop rollout_proc13_evt_loop terminating... [2023-10-14 01:00:10,682][60972] Stopping RolloutWorker_w3... [2023-10-14 01:00:10,682][59943] Component RolloutWorker_w13 stopped! [2023-10-14 01:00:10,683][60972] Loop rollout_proc3_evt_loop terminating... [2023-10-14 01:00:10,683][60996] Loop rollout_proc12_evt_loop terminating... [2023-10-14 01:00:10,682][59943] Component Batcher_1 stopped! [2023-10-14 01:00:10,683][59943] Component RolloutWorker_w12 stopped! [2023-10-14 01:00:10,683][59943] Component RolloutWorker_w3 stopped! [2023-10-14 01:00:10,696][60935] Weights refcount: 2 0 [2023-10-14 01:00:10,682][60828] Stopping Batcher_1... [2023-10-14 01:00:10,697][60935] Stopping InferenceWorker_p0-w0... [2023-10-14 01:00:10,697][60935] Loop inference_proc0-0_evt_loop terminating... [2023-10-14 01:00:10,697][59943] Component InferenceWorker_p0-w0 stopped! [2023-10-14 01:00:10,703][60934] Weights refcount: 2 0 [2023-10-14 01:00:10,705][60934] Stopping InferenceWorker_p1-w0... [2023-10-14 01:00:10,705][60934] Loop inference_proc1-0_evt_loop terminating... [2023-10-14 01:00:10,705][59943] Component InferenceWorker_p1-w0 stopped! [2023-10-14 01:00:10,710][60828] Loop batcher_evt_loop terminating... [2023-10-14 01:00:10,723][60695] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000096384_98697216.pth [2023-10-14 01:00:10,724][60828] Removing ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000095192_99549184.pth [2023-10-14 01:00:10,729][60695] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p0/checkpoint_000097664_100007936.pth... [2023-10-14 01:00:10,730][60828] Saving ./train_atari/atari_pitfall_APPO/checkpoint_p1/checkpoint_000096504_100892672.pth... [2023-10-14 01:00:10,786][60695] Stopping LearnerWorker_p0... [2023-10-14 01:00:10,787][60695] Loop learner_proc0_evt_loop terminating... [2023-10-14 01:00:10,787][59943] Component LearnerWorker_p0 stopped! [2023-10-14 01:00:10,791][60828] Stopping LearnerWorker_p1... [2023-10-14 01:00:10,791][60828] Loop learner_proc1_evt_loop terminating... [2023-10-14 01:00:10,791][59943] Component LearnerWorker_p1 stopped! [2023-10-14 01:00:10,792][59943] Waiting for process learner_proc0 to stop... [2023-10-14 01:00:11,676][59943] Waiting for process learner_proc1 to stop... [2023-10-14 01:00:11,677][59943] Waiting for process inference_proc0-0 to join... [2023-10-14 01:00:11,678][59943] Waiting for process inference_proc1-0 to join... [2023-10-14 01:00:11,679][59943] Waiting for process rollout_proc0 to join... [2023-10-14 01:00:11,680][59943] Waiting for process rollout_proc1 to join... [2023-10-14 01:00:11,681][59943] Waiting for process rollout_proc2 to join... [2023-10-14 01:00:11,682][59943] Waiting for process rollout_proc3 to join... [2023-10-14 01:00:11,683][59943] Waiting for process rollout_proc4 to join... [2023-10-14 01:00:11,684][59943] Waiting for process rollout_proc5 to join... [2023-10-14 01:00:11,685][59943] Waiting for process rollout_proc6 to join... [2023-10-14 01:00:11,686][59943] Waiting for process rollout_proc7 to join... [2023-10-14 01:00:11,687][59943] Waiting for process rollout_proc8 to join... [2023-10-14 01:00:11,688][59943] Waiting for process rollout_proc9 to join... [2023-10-14 01:00:11,689][59943] Waiting for process rollout_proc10 to join... [2023-10-14 01:00:11,690][59943] Waiting for process rollout_proc11 to join... [2023-10-14 01:00:11,690][59943] Waiting for process rollout_proc12 to join... [2023-10-14 01:00:11,691][59943] Waiting for process rollout_proc13 to join... [2023-10-14 01:00:11,691][59943] Waiting for process rollout_proc14 to join... [2023-10-14 01:00:11,692][59943] Waiting for process rollout_proc15 to join... [2023-10-14 01:00:11,692][59943] Batcher 0 profile tree view: batching: 171.0345, releasing_batches: 0.0923 [2023-10-14 01:00:11,693][59943] Batcher 1 profile tree view: batching: 171.5839, releasing_batches: 0.0917 [2023-10-14 01:00:11,693][59943] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0000 wait_policy_total: 2540.6902 update_model: 206.8619 weight_update: 0.0007 one_step: 0.0019 handle_policy_step: 11378.7289 deserialize: 64.3573, stack: 193.4434, obs_to_device_normalize: 2521.7027, forward: 5141.1749, prepare_outputs: 2490.6052, send_messages: 464.7162 [2023-10-14 01:00:11,693][59943] InferenceWorker_p1-w0 profile tree view: wait_policy: 0.0000 wait_policy_total: 2565.2296 update_model: 206.4922 weight_update: 0.0007 one_step: 0.0024 handle_policy_step: 11342.8612 deserialize: 65.6408, stack: 195.2158, obs_to_device_normalize: 2536.9232, forward: 5124.1922, prepare_outputs: 2448.2630, send_messages: 469.7878 [2023-10-14 01:00:11,694][59943] Learner 0 profile tree view: misc: 0.0204, prepare_batch: 268.9553 train: 3621.7618 epoch_init: 0.1942, minibatch_init: 13.3454, losses_postprocess: 889.9788, kl_divergence: 31.9089, update: 383.3731, after_optimizer: 2118.5724 calculate_losses: 167.3148 losses_init: 0.3880, forward_head: 56.3462, bptt_initial: 1.4555, bptt: 1.8534, tail: 38.3525, advantages_returns: 11.1881, losses: 43.9717 [2023-10-14 01:00:11,694][59943] Learner 1 profile tree view: misc: 0.0185, prepare_batch: 271.5364 train: 3558.4195 epoch_init: 0.1876, minibatch_init: 12.8614, losses_postprocess: 876.3155, kl_divergence: 31.1417, update: 375.9648, after_optimizer: 2081.2576 calculate_losses: 164.2290 losses_init: 0.3965, forward_head: 55.0367, bptt_initial: 1.4293, bptt: 1.9982, tail: 37.7855, advantages_returns: 10.9285, losses: 43.2527 [2023-10-14 01:00:11,694][59943] RolloutWorker_w0 profile tree view: wait_for_trajectories: 1.2335, enqueue_policy_requests: 409.6055, process_policy_outputs: 193.0734, env_step: 7539.7149, finalize_trajectories: 3.4640, complete_rollouts: 2.9244 post_env_step: 376.7189 process_env_step: 85.5315 [2023-10-14 01:00:11,695][59943] RolloutWorker_w15 profile tree view: wait_for_trajectories: 1.2193, enqueue_policy_requests: 411.3923, process_policy_outputs: 192.2521, env_step: 7549.3365, finalize_trajectories: 3.4018, complete_rollouts: 2.9260 post_env_step: 375.9401 process_env_step: 83.6495 [2023-10-14 01:00:11,695][59943] Loop Runner_EvtLoop terminating... [2023-10-14 01:00:11,696][59943] Runner profile tree view: main_loop: 14830.7931 [2023-10-14 01:00:11,697][59943] Collected {0: 100007936, 1: 100892672}, FPS: 13546.2