[2023-10-07 19:37:11,302][66916] Saving configuration to ./train_atari/atari_alien_APPO/config.json... [2023-10-07 19:37:11,619][66916] Rollout worker 0 uses device cpu [2023-10-07 19:37:11,620][66916] Rollout worker 1 uses device cpu [2023-10-07 19:37:11,620][66916] Rollout worker 2 uses device cpu [2023-10-07 19:37:11,621][66916] Rollout worker 3 uses device cpu [2023-10-07 19:37:11,621][66916] Rollout worker 4 uses device cpu [2023-10-07 19:37:11,621][66916] Rollout worker 5 uses device cpu [2023-10-07 19:37:11,622][66916] Rollout worker 6 uses device cpu [2023-10-07 19:37:11,622][66916] Rollout worker 7 uses device cpu [2023-10-07 19:37:11,623][66916] Rollout worker 8 uses device cpu [2023-10-07 19:37:11,623][66916] Rollout worker 9 uses device cpu [2023-10-07 19:37:11,623][66916] Rollout worker 10 uses device cpu [2023-10-07 19:37:11,624][66916] Rollout worker 11 uses device cpu [2023-10-07 19:37:11,624][66916] Rollout worker 12 uses device cpu [2023-10-07 19:37:11,625][66916] Rollout worker 13 uses device cpu [2023-10-07 19:37:11,625][66916] Rollout worker 14 uses device cpu [2023-10-07 19:37:11,626][66916] Rollout worker 15 uses device cpu [2023-10-07 19:37:11,904][66916] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-07 19:37:11,904][66916] InferenceWorker_p0-w0: min num requests: 2 [2023-10-07 19:37:11,907][66916] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-07 19:37:11,907][66916] InferenceWorker_p1-w0: min num requests: 2 [2023-10-07 19:37:11,951][66916] Starting all processes... [2023-10-07 19:37:11,951][66916] Starting process learner_proc0 [2023-10-07 19:37:13,598][66916] Starting process learner_proc1 [2023-10-07 19:37:13,601][67511] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-07 19:37:13,602][67511] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-10-07 19:37:13,620][67511] Num visible devices: 1 [2023-10-07 19:37:13,626][67511] Setting fixed seed 1234 [2023-10-07 19:37:13,627][67511] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-07 19:37:13,627][67511] Initializing actor-critic model on device cuda:0 [2023-10-07 19:37:13,628][67511] RunningMeanStd input shape: (4, 84, 84) [2023-10-07 19:37:13,628][67511] RunningMeanStd input shape: (1,) [2023-10-07 19:37:13,639][67511] ConvEncoder: input_channels=4 [2023-10-07 19:37:13,820][67511] Conv encoder output size: 512 [2023-10-07 19:37:13,823][67511] Created Actor Critic model with architecture: [2023-10-07 19:37:13,823][67511] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=18, bias=True) ) ) [2023-10-07 19:37:14,380][67511] Using optimizer [2023-10-07 19:37:14,381][67511] No checkpoints found [2023-10-07 19:37:14,381][67511] Did not load from checkpoint, starting from scratch! [2023-10-07 19:37:14,381][67511] Initialized policy 0 weights for model version 0 [2023-10-07 19:37:14,382][67511] LearnerWorker_p0 finished initialization! [2023-10-07 19:37:14,383][67511] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-07 19:37:15,365][66916] Starting all processes... [2023-10-07 19:37:15,368][67676] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-07 19:37:15,368][67676] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1 [2023-10-07 19:37:15,374][66916] Starting process inference_proc0-0 [2023-10-07 19:37:15,374][66916] Starting process inference_proc1-0 [2023-10-07 19:37:15,374][66916] Starting process rollout_proc0 [2023-10-07 19:37:15,386][67676] Num visible devices: 1 [2023-10-07 19:37:15,375][66916] Starting process rollout_proc1 [2023-10-07 19:37:15,378][66916] Starting process rollout_proc2 [2023-10-07 19:37:15,409][67676] Setting fixed seed 1234 [2023-10-07 19:37:15,411][67676] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-07 19:37:15,411][67676] Initializing actor-critic model on device cuda:0 [2023-10-07 19:37:15,404][66916] Starting process rollout_proc8 [2023-10-07 19:37:15,412][67676] RunningMeanStd input shape: (4, 84, 84) [2023-10-07 19:37:15,380][66916] Starting process rollout_proc4 [2023-10-07 19:37:15,412][67676] RunningMeanStd input shape: (1,) [2023-10-07 19:37:15,385][66916] Starting process rollout_proc5 [2023-10-07 19:37:15,389][66916] Starting process rollout_proc6 [2023-10-07 19:37:15,389][66916] Starting process rollout_proc7 [2023-10-07 19:37:15,379][66916] Starting process rollout_proc3 [2023-10-07 19:37:15,409][66916] Starting process rollout_proc9 [2023-10-07 19:37:15,409][66916] Starting process rollout_proc10 [2023-10-07 19:37:15,411][66916] Starting process rollout_proc11 [2023-10-07 19:37:15,432][67676] ConvEncoder: input_channels=4 [2023-10-07 19:37:15,411][66916] Starting process rollout_proc12 [2023-10-07 19:37:15,412][66916] Starting process rollout_proc13 [2023-10-07 19:37:15,916][67676] Conv encoder output size: 512 [2023-10-07 19:37:15,919][67676] Created Actor Critic model with architecture: [2023-10-07 19:37:15,920][67676] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=18, bias=True) ) ) [2023-10-07 19:37:16,685][67676] Using optimizer [2023-10-07 19:37:16,686][67676] No checkpoints found [2023-10-07 19:37:16,686][67676] Did not load from checkpoint, starting from scratch! [2023-10-07 19:37:16,686][67676] Initialized policy 1 weights for model version 0 [2023-10-07 19:37:16,688][67676] LearnerWorker_p1 finished initialization! [2023-10-07 19:37:16,688][67676] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-07 19:37:17,525][66916] Starting process rollout_proc14 [2023-10-07 19:37:17,527][66916] Starting process rollout_proc15 [2023-10-07 19:37:17,530][67919] Worker 13 uses CPU cores [26, 27] [2023-10-07 19:37:17,531][67918] Worker 12 uses CPU cores [24, 25] [2023-10-07 19:37:17,544][67875] Worker 8 uses CPU cores [16, 17] [2023-10-07 19:37:17,563][67916] Worker 9 uses CPU cores [18, 19] [2023-10-07 19:37:17,564][67877] Worker 5 uses CPU cores [10, 11] [2023-10-07 19:37:17,658][67887] Worker 3 uses CPU cores [6, 7] [2023-10-07 19:37:17,671][67884] Worker 6 uses CPU cores [12, 13] [2023-10-07 19:37:17,952][67874] Worker 2 uses CPU cores [4, 5] [2023-10-07 19:37:18,005][67871] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-07 19:37:18,005][67871] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1 [2023-10-07 19:37:18,023][67871] Num visible devices: 1 [2023-10-07 19:37:18,024][67876] Worker 4 uses CPU cores [8, 9] [2023-10-07 19:37:18,028][67885] Worker 7 uses CPU cores [14, 15] [2023-10-07 19:37:18,091][67917] Worker 11 uses CPU cores [22, 23] [2023-10-07 19:37:18,128][67915] Worker 10 uses CPU cores [20, 21] [2023-10-07 19:37:18,241][67838] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-07 19:37:18,241][67838] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-10-07 19:37:18,249][67870] Worker 1 uses CPU cores [2, 3] [2023-10-07 19:37:18,259][67838] Num visible devices: 1 [2023-10-07 19:37:18,291][67873] Worker 0 uses CPU cores [0, 1] [2023-10-07 19:37:18,649][67871] RunningMeanStd input shape: (4, 84, 84) [2023-10-07 19:37:18,650][67871] RunningMeanStd input shape: (1,) [2023-10-07 19:37:18,661][67871] ConvEncoder: input_channels=4 [2023-10-07 19:37:18,765][67871] Conv encoder output size: 512 [2023-10-07 19:37:18,848][67838] RunningMeanStd input shape: (4, 84, 84) [2023-10-07 19:37:18,849][67838] RunningMeanStd input shape: (1,) [2023-10-07 19:37:18,861][67838] ConvEncoder: input_channels=4 [2023-10-07 19:37:18,966][67838] Conv encoder output size: 512 [2023-10-07 19:37:19,394][68573] Worker 15 uses CPU cores [30, 31] [2023-10-07 19:37:19,398][66916] Inference worker 1-0 is ready! [2023-10-07 19:37:19,399][66916] Inference worker 0-0 is ready! [2023-10-07 19:37:19,399][68572] Worker 14 uses CPU cores [28, 29] [2023-10-07 19:37:19,400][66916] All inference workers are ready! Signal rollout workers to start! [2023-10-07 19:37:19,401][67885] EnvRunner 7-0 uses policy 1 [2023-10-07 19:37:19,401][67884] EnvRunner 6-0 uses policy 0 [2023-10-07 19:37:19,401][67919] EnvRunner 13-0 uses policy 1 [2023-10-07 19:37:19,401][67877] EnvRunner 5-0 uses policy 1 [2023-10-07 19:37:19,401][67875] EnvRunner 8-0 uses policy 0 [2023-10-07 19:37:19,401][67917] EnvRunner 11-0 uses policy 1 [2023-10-07 19:37:19,401][67918] EnvRunner 12-0 uses policy 0 [2023-10-07 19:37:19,401][66916] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-07 19:37:19,401][67876] EnvRunner 4-0 uses policy 0 [2023-10-07 19:37:19,401][67887] EnvRunner 3-0 uses policy 1 [2023-10-07 19:37:19,402][67874] EnvRunner 2-0 uses policy 0 [2023-10-07 19:37:19,402][67870] EnvRunner 1-0 uses policy 1 [2023-10-07 19:37:19,402][67916] EnvRunner 9-0 uses policy 1 [2023-10-07 19:37:19,402][67873] EnvRunner 0-0 uses policy 0 [2023-10-07 19:37:19,402][67915] EnvRunner 10-0 uses policy 0 [2023-10-07 19:37:19,513][68572] EnvRunner 14-0 uses policy 0 [2023-10-07 19:37:19,605][68573] EnvRunner 15-0 uses policy 1 [2023-10-07 19:37:21,892][66916] Heartbeat connected on Batcher_0 [2023-10-07 19:37:21,894][66916] Heartbeat connected on LearnerWorker_p0 [2023-10-07 19:37:21,897][66916] Heartbeat connected on Batcher_1 [2023-10-07 19:37:21,900][66916] Heartbeat connected on LearnerWorker_p1 [2023-10-07 19:37:21,907][66916] Heartbeat connected on InferenceWorker_p0-w0 [2023-10-07 19:37:21,910][66916] Heartbeat connected on InferenceWorker_p1-w0 [2023-10-07 19:37:21,911][66916] Heartbeat connected on RolloutWorker_w0 [2023-10-07 19:37:21,913][66916] Heartbeat connected on RolloutWorker_w1 [2023-10-07 19:37:21,916][66916] Heartbeat connected on RolloutWorker_w2 [2023-10-07 19:37:21,920][66916] Heartbeat connected on RolloutWorker_w3 [2023-10-07 19:37:21,922][66916] Heartbeat connected on RolloutWorker_w4 [2023-10-07 19:37:21,925][66916] Heartbeat connected on RolloutWorker_w5 [2023-10-07 19:37:21,928][66916] Heartbeat connected on RolloutWorker_w6 [2023-10-07 19:37:21,929][66916] Heartbeat connected on RolloutWorker_w7 [2023-10-07 19:37:21,933][66916] Heartbeat connected on RolloutWorker_w8 [2023-10-07 19:37:21,936][66916] Heartbeat connected on RolloutWorker_w9 [2023-10-07 19:37:21,937][66916] Heartbeat connected on RolloutWorker_w10 [2023-10-07 19:37:21,944][66916] Heartbeat connected on RolloutWorker_w12 [2023-10-07 19:37:21,946][66916] Heartbeat connected on RolloutWorker_w11 [2023-10-07 19:37:21,948][66916] Heartbeat connected on RolloutWorker_w13 [2023-10-07 19:37:21,950][66916] Heartbeat connected on RolloutWorker_w15 [2023-10-07 19:37:21,951][66916] Heartbeat connected on RolloutWorker_w14 [2023-10-07 19:37:22,476][66916] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 539.2, 1: 489.7. Samples: 3164. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-07 19:37:22,477][66916] Avg episode reward: [(0, '7.500'), (1, '7.833')] [2023-10-07 19:37:27,476][66916] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 880.7, 1: 859.9. Samples: 14056. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-07 19:37:27,477][66916] Avg episode reward: [(0, '6.286'), (1, '6.217')] [2023-10-07 19:37:29,647][67871] Updated weights for policy 1, policy_version 10 (0.0008) [2023-10-07 19:37:29,933][67838] Updated weights for policy 0, policy_version 10 (0.0008) [2023-10-07 19:37:30,013][67871] Updated weights for policy 1, policy_version 20 (0.0008) [2023-10-07 19:37:30,303][67838] Updated weights for policy 0, policy_version 20 (0.0008) [2023-10-07 19:37:30,377][67871] Updated weights for policy 1, policy_version 30 (0.0008) [2023-10-07 19:37:30,670][67838] Updated weights for policy 0, policy_version 30 (0.0009) [2023-10-07 19:37:32,477][66916] Fps is (10 sec: 6553.6, 60 sec: 5012.3, 300 sec: 5012.3). Total num frames: 65536. Throughput: 0: 1143.2, 1: 1126.0. Samples: 29670. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 19:37:32,478][66916] Avg episode reward: [(0, '6.421'), (1, '6.667')] [2023-10-07 19:37:33,335][67871] Updated weights for policy 1, policy_version 40 (0.0008) [2023-10-07 19:37:33,409][67838] Updated weights for policy 0, policy_version 40 (0.0009) [2023-10-07 19:37:33,707][67871] Updated weights for policy 1, policy_version 50 (0.0007) [2023-10-07 19:37:33,859][67838] Updated weights for policy 0, policy_version 52 (0.0008) [2023-10-07 19:37:34,071][67871] Updated weights for policy 1, policy_version 60 (0.0008) [2023-10-07 19:37:34,226][67838] Updated weights for policy 0, policy_version 62 (0.0008) [2023-10-07 19:37:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 7251.5, 300 sec: 7251.5). Total num frames: 131072. Throughput: 0: 1370.6, 1: 1361.5. Samples: 49384. Policy #0 lag: (min: 33.0, avg: 33.0, max: 33.0) [2023-10-07 19:37:37,477][66916] Avg episode reward: [(0, '5.890'), (1, '6.760')] [2023-10-07 19:37:37,985][67871] Updated weights for policy 1, policy_version 70 (0.0010) [2023-10-07 19:37:38,206][67838] Updated weights for policy 0, policy_version 72 (0.0007) [2023-10-07 19:37:38,345][67871] Updated weights for policy 1, policy_version 80 (0.0010) [2023-10-07 19:37:38,578][67838] Updated weights for policy 0, policy_version 82 (0.0007) [2023-10-07 19:37:38,727][67871] Updated weights for policy 1, policy_version 90 (0.0007) [2023-10-07 19:37:38,945][67838] Updated weights for policy 0, policy_version 92 (0.0007) [2023-10-07 19:37:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 8520.4, 300 sec: 8520.4). Total num frames: 196608. Throughput: 0: 1264.9, 1: 1254.3. Samples: 58132. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-10-07 19:37:42,477][66916] Avg episode reward: [(0, '5.960'), (1, '7.280')] [2023-10-07 19:37:42,522][67871] Updated weights for policy 1, policy_version 100 (0.0008) [2023-10-07 19:37:42,747][67838] Updated weights for policy 0, policy_version 102 (0.0009) [2023-10-07 19:37:42,894][67871] Updated weights for policy 1, policy_version 110 (0.0008) [2023-10-07 19:37:43,124][67838] Updated weights for policy 0, policy_version 112 (0.0009) [2023-10-07 19:37:43,264][67871] Updated weights for policy 1, policy_version 120 (0.0007) [2023-10-07 19:37:43,507][67838] Updated weights for policy 0, policy_version 122 (0.0008) [2023-10-07 19:37:47,477][66916] Fps is (10 sec: 13107.1, 60 sec: 9337.2, 300 sec: 9337.2). Total num frames: 262144. Throughput: 0: 1393.8, 1: 1378.2. Samples: 77824. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-10-07 19:37:47,478][66916] Avg episode reward: [(0, '6.590'), (1, '7.120')] [2023-10-07 19:37:47,479][67511] Saving new best policy, reward=6.590! [2023-10-07 19:37:47,568][67871] Updated weights for policy 1, policy_version 130 (0.0008) [2023-10-07 19:37:47,925][67871] Updated weights for policy 1, policy_version 140 (0.0009) [2023-10-07 19:37:47,942][67838] Updated weights for policy 0, policy_version 132 (0.0009) [2023-10-07 19:37:48,289][67871] Updated weights for policy 1, policy_version 150 (0.0007) [2023-10-07 19:37:48,316][67838] Updated weights for policy 0, policy_version 142 (0.0009) [2023-10-07 19:37:48,658][67676] Saving new best policy, reward=7.120! [2023-10-07 19:37:48,660][67871] Updated weights for policy 1, policy_version 160 (0.0007) [2023-10-07 19:37:48,689][67838] Updated weights for policy 0, policy_version 152 (0.0008) [2023-10-07 19:37:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 9907.2, 300 sec: 9907.2). Total num frames: 327680. Throughput: 0: 1476.1, 1: 1471.3. Samples: 97486. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 19:37:52,477][66916] Avg episode reward: [(0, '7.160'), (1, '7.180')] [2023-10-07 19:37:52,483][67511] Saving new best policy, reward=7.160! [2023-10-07 19:37:52,484][67676] Saving new best policy, reward=7.180! [2023-10-07 19:37:52,951][67838] Updated weights for policy 0, policy_version 162 (0.0007) [2023-10-07 19:37:53,021][67871] Updated weights for policy 1, policy_version 170 (0.0008) [2023-10-07 19:37:53,310][67838] Updated weights for policy 0, policy_version 172 (0.0007) [2023-10-07 19:37:53,390][67871] Updated weights for policy 1, policy_version 180 (0.0007) [2023-10-07 19:37:53,688][67838] Updated weights for policy 0, policy_version 182 (0.0009) [2023-10-07 19:37:53,754][67871] Updated weights for policy 1, policy_version 190 (0.0009) [2023-10-07 19:37:54,053][67838] Updated weights for policy 0, policy_version 192 (0.0010) [2023-10-07 19:37:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 10327.4, 300 sec: 10327.4). Total num frames: 393216. Throughput: 0: 1393.2, 1: 1392.5. Samples: 106064. Policy #0 lag: (min: 31.0, avg: 46.1, max: 63.0) [2023-10-07 19:37:57,477][66916] Avg episode reward: [(0, '6.940'), (1, '6.780')] [2023-10-07 19:37:57,806][67871] Updated weights for policy 1, policy_version 200 (0.0009) [2023-10-07 19:37:58,178][67871] Updated weights for policy 1, policy_version 210 (0.0007) [2023-10-07 19:37:58,198][67838] Updated weights for policy 0, policy_version 202 (0.0009) [2023-10-07 19:37:58,539][67871] Updated weights for policy 1, policy_version 220 (0.0008) [2023-10-07 19:37:58,564][67838] Updated weights for policy 0, policy_version 212 (0.0007) [2023-10-07 19:37:58,940][67838] Updated weights for policy 0, policy_version 222 (0.0007) [2023-10-07 19:38:02,476][66916] Fps is (10 sec: 13107.1, 60 sec: 10650.1, 300 sec: 10650.1). Total num frames: 458752. Throughput: 0: 1460.8, 1: 1460.3. Samples: 125828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:38:02,477][66916] Avg episode reward: [(0, '7.270'), (1, '7.430')] [2023-10-07 19:38:02,478][67511] Saving new best policy, reward=7.270! [2023-10-07 19:38:02,478][67676] Saving new best policy, reward=7.430! [2023-10-07 19:38:02,831][67871] Updated weights for policy 1, policy_version 230 (0.0009) [2023-10-07 19:38:03,194][67871] Updated weights for policy 1, policy_version 240 (0.0007) [2023-10-07 19:38:03,288][67838] Updated weights for policy 0, policy_version 232 (0.0008) [2023-10-07 19:38:03,571][67871] Updated weights for policy 1, policy_version 250 (0.0008) [2023-10-07 19:38:03,664][67838] Updated weights for policy 0, policy_version 242 (0.0009) [2023-10-07 19:38:04,037][67838] Updated weights for policy 0, policy_version 252 (0.0008) [2023-10-07 19:38:07,476][66916] Fps is (10 sec: 13107.0, 60 sec: 10905.6, 300 sec: 10905.6). Total num frames: 524288. Throughput: 0: 1581.7, 1: 1587.8. Samples: 145794. Policy #0 lag: (min: 4.0, avg: 4.4, max: 18.0) [2023-10-07 19:38:07,477][66916] Avg episode reward: [(0, '7.510'), (1, '8.210')] [2023-10-07 19:38:07,486][67511] Saving new best policy, reward=7.510! [2023-10-07 19:38:07,486][67676] Saving new best policy, reward=8.210! [2023-10-07 19:38:07,803][67871] Updated weights for policy 1, policy_version 260 (0.0008) [2023-10-07 19:38:08,161][67871] Updated weights for policy 1, policy_version 270 (0.0010) [2023-10-07 19:38:08,330][67838] Updated weights for policy 0, policy_version 262 (0.0009) [2023-10-07 19:38:08,523][67871] Updated weights for policy 1, policy_version 280 (0.0008) [2023-10-07 19:38:08,712][67838] Updated weights for policy 0, policy_version 272 (0.0008) [2023-10-07 19:38:09,087][67838] Updated weights for policy 0, policy_version 282 (0.0008) [2023-10-07 19:38:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 11113.0, 300 sec: 11113.0). Total num frames: 589824. Throughput: 0: 1556.8, 1: 1564.8. Samples: 154532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:38:12,477][66916] Avg episode reward: [(0, '7.960'), (1, '8.790')] [2023-10-07 19:38:12,478][67676] Saving new best policy, reward=8.790! [2023-10-07 19:38:12,478][67511] Saving new best policy, reward=7.960! [2023-10-07 19:38:12,806][67871] Updated weights for policy 1, policy_version 290 (0.0010) [2023-10-07 19:38:13,182][67871] Updated weights for policy 1, policy_version 300 (0.0007) [2023-10-07 19:38:13,316][67838] Updated weights for policy 0, policy_version 292 (0.0009) [2023-10-07 19:38:13,554][67871] Updated weights for policy 1, policy_version 310 (0.0007) [2023-10-07 19:38:13,685][67838] Updated weights for policy 0, policy_version 302 (0.0007) [2023-10-07 19:38:13,915][67871] Updated weights for policy 1, policy_version 320 (0.0009) [2023-10-07 19:38:14,063][67838] Updated weights for policy 0, policy_version 312 (0.0009) [2023-10-07 19:38:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 11284.7, 300 sec: 11284.7). Total num frames: 655360. Throughput: 0: 1598.4, 1: 1608.5. Samples: 173980. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-07 19:38:17,477][66916] Avg episode reward: [(0, '8.690'), (1, '9.500')] [2023-10-07 19:38:17,478][67676] Saving new best policy, reward=9.500! [2023-10-07 19:38:17,478][67511] Saving new best policy, reward=8.690! [2023-10-07 19:38:18,171][67871] Updated weights for policy 1, policy_version 330 (0.0008) [2023-10-07 19:38:18,410][67838] Updated weights for policy 0, policy_version 322 (0.0010) [2023-10-07 19:38:18,534][67871] Updated weights for policy 1, policy_version 340 (0.0009) [2023-10-07 19:38:18,818][67838] Updated weights for policy 0, policy_version 332 (0.0007) [2023-10-07 19:38:18,912][67871] Updated weights for policy 1, policy_version 350 (0.0008) [2023-10-07 19:38:19,177][67838] Updated weights for policy 0, policy_version 342 (0.0008) [2023-10-07 19:38:19,546][67838] Updated weights for policy 0, policy_version 352 (0.0007) [2023-10-07 19:38:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 12014.9, 300 sec: 11429.2). Total num frames: 720896. Throughput: 0: 1594.6, 1: 1609.6. Samples: 193572. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-07 19:38:22,477][66916] Avg episode reward: [(0, '8.960'), (1, '9.870')] [2023-10-07 19:38:22,484][67676] Saving new best policy, reward=9.870! [2023-10-07 19:38:22,484][67511] Saving new best policy, reward=8.960! [2023-10-07 19:38:23,143][67871] Updated weights for policy 1, policy_version 360 (0.0008) [2023-10-07 19:38:23,500][67871] Updated weights for policy 1, policy_version 370 (0.0009) [2023-10-07 19:38:23,871][67871] Updated weights for policy 1, policy_version 380 (0.0008) [2023-10-07 19:38:23,877][67838] Updated weights for policy 0, policy_version 362 (0.0008) [2023-10-07 19:38:24,253][67838] Updated weights for policy 0, policy_version 372 (0.0007) [2023-10-07 19:38:24,621][67838] Updated weights for policy 0, policy_version 382 (0.0007) [2023-10-07 19:38:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 11552.4). Total num frames: 786432. Throughput: 0: 1593.4, 1: 1607.6. Samples: 202176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:38:27,478][66916] Avg episode reward: [(0, '9.120'), (1, '9.950')] [2023-10-07 19:38:27,479][67511] Saving new best policy, reward=9.120! [2023-10-07 19:38:27,479][67676] Saving new best policy, reward=9.950! [2023-10-07 19:38:28,365][67871] Updated weights for policy 1, policy_version 390 (0.0007) [2023-10-07 19:38:28,738][67871] Updated weights for policy 1, policy_version 400 (0.0007) [2023-10-07 19:38:28,935][67838] Updated weights for policy 0, policy_version 392 (0.0008) [2023-10-07 19:38:29,111][67871] Updated weights for policy 1, policy_version 410 (0.0009) [2023-10-07 19:38:29,305][67838] Updated weights for policy 0, policy_version 402 (0.0008) [2023-10-07 19:38:29,680][67838] Updated weights for policy 0, policy_version 412 (0.0008) [2023-10-07 19:38:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 11658.8). Total num frames: 851968. Throughput: 0: 1590.3, 1: 1606.3. Samples: 221672. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-10-07 19:38:32,478][66916] Avg episode reward: [(0, '9.410'), (1, '9.600')] [2023-10-07 19:38:32,479][67511] Saving new best policy, reward=9.410! [2023-10-07 19:38:33,396][67871] Updated weights for policy 1, policy_version 420 (0.0007) [2023-10-07 19:38:33,756][67871] Updated weights for policy 1, policy_version 430 (0.0009) [2023-10-07 19:38:34,060][67838] Updated weights for policy 0, policy_version 422 (0.0007) [2023-10-07 19:38:34,119][67871] Updated weights for policy 1, policy_version 440 (0.0009) [2023-10-07 19:38:34,429][67838] Updated weights for policy 0, policy_version 432 (0.0009) [2023-10-07 19:38:34,810][67838] Updated weights for policy 0, policy_version 442 (0.0010) [2023-10-07 19:38:37,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 11751.6). Total num frames: 917504. Throughput: 0: 1591.3, 1: 1604.3. Samples: 241288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:38:37,477][66916] Avg episode reward: [(0, '10.000'), (1, '10.420')] [2023-10-07 19:38:37,486][67511] Saving new best policy, reward=10.000! [2023-10-07 19:38:37,487][67676] Saving new best policy, reward=10.420! [2023-10-07 19:38:38,414][67871] Updated weights for policy 1, policy_version 450 (0.0008) [2023-10-07 19:38:38,779][67871] Updated weights for policy 1, policy_version 460 (0.0009) [2023-10-07 19:38:39,057][67838] Updated weights for policy 0, policy_version 452 (0.0010) [2023-10-07 19:38:39,146][67871] Updated weights for policy 1, policy_version 470 (0.0009) [2023-10-07 19:38:39,420][67838] Updated weights for policy 0, policy_version 462 (0.0008) [2023-10-07 19:38:39,515][67871] Updated weights for policy 1, policy_version 480 (0.0008) [2023-10-07 19:38:39,793][67838] Updated weights for policy 0, policy_version 472 (0.0010) [2023-10-07 19:38:42,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 11833.1). Total num frames: 983040. Throughput: 0: 1594.1, 1: 1605.1. Samples: 250026. Policy #0 lag: (min: 1.0, avg: 9.6, max: 33.0) [2023-10-07 19:38:42,478][66916] Avg episode reward: [(0, '11.000'), (1, '10.570')] [2023-10-07 19:38:42,479][67676] Saving new best policy, reward=10.570! [2023-10-07 19:38:42,479][67511] Saving new best policy, reward=11.000! [2023-10-07 19:38:43,986][67838] Updated weights for policy 0, policy_version 482 (0.0010) [2023-10-07 19:38:44,023][67871] Updated weights for policy 1, policy_version 490 (0.0007) [2023-10-07 19:38:44,349][67838] Updated weights for policy 0, policy_version 492 (0.0009) [2023-10-07 19:38:44,387][67871] Updated weights for policy 1, policy_version 500 (0.0008) [2023-10-07 19:38:44,718][67838] Updated weights for policy 0, policy_version 502 (0.0008) [2023-10-07 19:38:44,760][67871] Updated weights for policy 1, policy_version 510 (0.0008) [2023-10-07 19:38:45,100][67838] Updated weights for policy 0, policy_version 512 (0.0010) [2023-10-07 19:38:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 11905.5). Total num frames: 1048576. Throughput: 0: 1593.4, 1: 1600.8. Samples: 269564. Policy #0 lag: (min: 26.0, avg: 33.9, max: 58.0) [2023-10-07 19:38:47,477][66916] Avg episode reward: [(0, '10.550'), (1, '10.840')] [2023-10-07 19:38:47,478][67676] Saving new best policy, reward=10.840! [2023-10-07 19:38:49,075][67871] Updated weights for policy 1, policy_version 520 (0.0008) [2023-10-07 19:38:49,445][67871] Updated weights for policy 1, policy_version 530 (0.0010) [2023-10-07 19:38:49,484][67838] Updated weights for policy 0, policy_version 522 (0.0008) [2023-10-07 19:38:49,804][67871] Updated weights for policy 1, policy_version 540 (0.0007) [2023-10-07 19:38:49,851][67838] Updated weights for policy 0, policy_version 532 (0.0007) [2023-10-07 19:38:50,224][67838] Updated weights for policy 0, policy_version 542 (0.0011) [2023-10-07 19:38:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 11970.0). Total num frames: 1114112. Throughput: 0: 1591.0, 1: 1597.7. Samples: 289284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:38:52,477][66916] Avg episode reward: [(0, '10.550'), (1, '11.510')] [2023-10-07 19:38:52,486][67676] Saving new best policy, reward=11.510! [2023-10-07 19:38:54,044][67871] Updated weights for policy 1, policy_version 550 (0.0009) [2023-10-07 19:38:54,419][67871] Updated weights for policy 1, policy_version 560 (0.0008) [2023-10-07 19:38:54,609][67838] Updated weights for policy 0, policy_version 552 (0.0008) [2023-10-07 19:38:54,777][67871] Updated weights for policy 1, policy_version 570 (0.0009) [2023-10-07 19:38:54,984][67838] Updated weights for policy 0, policy_version 562 (0.0008) [2023-10-07 19:38:55,360][67838] Updated weights for policy 0, policy_version 572 (0.0009) [2023-10-07 19:38:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12028.0). Total num frames: 1179648. Throughput: 0: 1601.5, 1: 1600.6. Samples: 298624. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-07 19:38:57,477][66916] Avg episode reward: [(0, '11.180'), (1, '11.520')] [2023-10-07 19:38:57,478][67676] Saving new best policy, reward=11.520! [2023-10-07 19:38:57,478][67511] Saving new best policy, reward=11.180! [2023-10-07 19:38:59,251][67871] Updated weights for policy 1, policy_version 580 (0.0008) [2023-10-07 19:38:59,641][67871] Updated weights for policy 1, policy_version 590 (0.0008) [2023-10-07 19:38:59,641][67838] Updated weights for policy 0, policy_version 582 (0.0009) [2023-10-07 19:39:00,013][67838] Updated weights for policy 0, policy_version 592 (0.0008) [2023-10-07 19:39:00,014][67871] Updated weights for policy 1, policy_version 600 (0.0009) [2023-10-07 19:39:00,384][67838] Updated weights for policy 0, policy_version 602 (0.0007) [2023-10-07 19:39:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12080.3). Total num frames: 1245184. Throughput: 0: 1589.5, 1: 1595.8. Samples: 317316. Policy #0 lag: (min: 17.0, avg: 24.6, max: 49.0) [2023-10-07 19:39:02,478][66916] Avg episode reward: [(0, '10.740'), (1, '12.930')] [2023-10-07 19:39:02,479][67676] Saving new best policy, reward=12.930! [2023-10-07 19:39:04,370][67871] Updated weights for policy 1, policy_version 610 (0.0008) [2023-10-07 19:39:04,736][67871] Updated weights for policy 1, policy_version 620 (0.0007) [2023-10-07 19:39:04,784][67838] Updated weights for policy 0, policy_version 612 (0.0008) [2023-10-07 19:39:05,094][67871] Updated weights for policy 1, policy_version 630 (0.0007) [2023-10-07 19:39:05,165][67838] Updated weights for policy 0, policy_version 622 (0.0007) [2023-10-07 19:39:05,469][67871] Updated weights for policy 1, policy_version 640 (0.0009) [2023-10-07 19:39:05,540][67838] Updated weights for policy 0, policy_version 632 (0.0009) [2023-10-07 19:39:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12127.8). Total num frames: 1310720. Throughput: 0: 1590.8, 1: 1590.3. Samples: 336724. Policy #0 lag: (min: 28.0, avg: 35.9, max: 60.0) [2023-10-07 19:39:07,478][66916] Avg episode reward: [(0, '11.460'), (1, '13.000')] [2023-10-07 19:39:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000000640_655360.pth... [2023-10-07 19:39:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000000640_655360.pth... [2023-10-07 19:39:07,523][67511] Saving new best policy, reward=11.460! [2023-10-07 19:39:07,528][67676] Saving new best policy, reward=13.000! [2023-10-07 19:39:09,645][67871] Updated weights for policy 1, policy_version 650 (0.0008) [2023-10-07 19:39:09,937][67838] Updated weights for policy 0, policy_version 642 (0.0010) [2023-10-07 19:39:10,022][67871] Updated weights for policy 1, policy_version 660 (0.0008) [2023-10-07 19:39:10,313][67838] Updated weights for policy 0, policy_version 652 (0.0008) [2023-10-07 19:39:10,381][67871] Updated weights for policy 1, policy_version 670 (0.0009) [2023-10-07 19:39:10,690][67838] Updated weights for policy 0, policy_version 662 (0.0007) [2023-10-07 19:39:11,063][67838] Updated weights for policy 0, policy_version 672 (0.0007) [2023-10-07 19:39:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12171.2). Total num frames: 1376256. Throughput: 0: 1611.8, 1: 1606.6. Samples: 347004. Policy #0 lag: (min: 31.0, avg: 31.5, max: 45.0) [2023-10-07 19:39:12,478][66916] Avg episode reward: [(0, '11.710'), (1, '12.550')] [2023-10-07 19:39:12,479][67511] Saving new best policy, reward=11.710! [2023-10-07 19:39:14,543][67871] Updated weights for policy 1, policy_version 680 (0.0008) [2023-10-07 19:39:14,906][67871] Updated weights for policy 1, policy_version 690 (0.0007) [2023-10-07 19:39:15,271][67871] Updated weights for policy 1, policy_version 700 (0.0007) [2023-10-07 19:39:15,331][67838] Updated weights for policy 0, policy_version 682 (0.0007) [2023-10-07 19:39:15,713][67838] Updated weights for policy 0, policy_version 692 (0.0008) [2023-10-07 19:39:16,086][67838] Updated weights for policy 0, policy_version 702 (0.0008) [2023-10-07 19:39:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 12210.8). Total num frames: 1441792. Throughput: 0: 1595.1, 1: 1598.4. Samples: 365380. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 19:39:17,477][66916] Avg episode reward: [(0, '11.830'), (1, '12.330')] [2023-10-07 19:39:17,478][67511] Saving new best policy, reward=11.830! [2023-10-07 19:39:19,588][67871] Updated weights for policy 1, policy_version 710 (0.0009) [2023-10-07 19:39:19,951][67871] Updated weights for policy 1, policy_version 720 (0.0008) [2023-10-07 19:39:20,270][67838] Updated weights for policy 0, policy_version 712 (0.0008) [2023-10-07 19:39:20,316][67871] Updated weights for policy 1, policy_version 730 (0.0009) [2023-10-07 19:39:20,648][67838] Updated weights for policy 0, policy_version 722 (0.0008) [2023-10-07 19:39:21,004][67838] Updated weights for policy 0, policy_version 732 (0.0009) [2023-10-07 19:39:22,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12247.2). Total num frames: 1507328. Throughput: 0: 1595.6, 1: 1598.4. Samples: 385020. Policy #0 lag: (min: 4.0, avg: 7.1, max: 36.0) [2023-10-07 19:39:22,477][66916] Avg episode reward: [(0, '11.630'), (1, '13.000')] [2023-10-07 19:39:24,651][67871] Updated weights for policy 1, policy_version 740 (0.0010) [2023-10-07 19:39:25,015][67871] Updated weights for policy 1, policy_version 750 (0.0008) [2023-10-07 19:39:25,238][67838] Updated weights for policy 0, policy_version 742 (0.0008) [2023-10-07 19:39:25,375][67871] Updated weights for policy 1, policy_version 760 (0.0007) [2023-10-07 19:39:25,612][67838] Updated weights for policy 0, policy_version 752 (0.0009) [2023-10-07 19:39:25,984][67838] Updated weights for policy 0, policy_version 762 (0.0007) [2023-10-07 19:39:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12280.8). Total num frames: 1572864. Throughput: 0: 1620.1, 1: 1616.3. Samples: 395664. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-07 19:39:27,477][66916] Avg episode reward: [(0, '11.740'), (1, '13.090')] [2023-10-07 19:39:27,478][67676] Saving new best policy, reward=13.090! [2023-10-07 19:39:29,822][67871] Updated weights for policy 1, policy_version 770 (0.0007) [2023-10-07 19:39:30,192][67871] Updated weights for policy 1, policy_version 780 (0.0008) [2023-10-07 19:39:30,354][67838] Updated weights for policy 0, policy_version 772 (0.0008) [2023-10-07 19:39:30,554][67871] Updated weights for policy 1, policy_version 790 (0.0007) [2023-10-07 19:39:30,713][67838] Updated weights for policy 0, policy_version 782 (0.0008) [2023-10-07 19:39:30,921][67871] Updated weights for policy 1, policy_version 800 (0.0007) [2023-10-07 19:39:31,084][67838] Updated weights for policy 0, policy_version 792 (0.0009) [2023-10-07 19:39:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12311.9). Total num frames: 1638400. Throughput: 0: 1600.8, 1: 1603.2. Samples: 413742. Policy #0 lag: (min: 15.0, avg: 16.4, max: 41.0) [2023-10-07 19:39:32,477][66916] Avg episode reward: [(0, '12.240'), (1, '14.400')] [2023-10-07 19:39:32,477][67676] Saving new best policy, reward=14.400! [2023-10-07 19:39:32,477][67511] Saving new best policy, reward=12.240! [2023-10-07 19:39:35,172][67871] Updated weights for policy 1, policy_version 810 (0.0009) [2023-10-07 19:39:35,273][67838] Updated weights for policy 0, policy_version 802 (0.0007) [2023-10-07 19:39:35,540][67871] Updated weights for policy 1, policy_version 820 (0.0008) [2023-10-07 19:39:35,642][67838] Updated weights for policy 0, policy_version 812 (0.0008) [2023-10-07 19:39:35,906][67871] Updated weights for policy 1, policy_version 830 (0.0010) [2023-10-07 19:39:36,013][67838] Updated weights for policy 0, policy_version 822 (0.0007) [2023-10-07 19:39:36,378][67838] Updated weights for policy 0, policy_version 832 (0.0007) [2023-10-07 19:39:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12340.7). Total num frames: 1703936. Throughput: 0: 1594.6, 1: 1599.8. Samples: 433034. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-07 19:39:37,477][66916] Avg episode reward: [(0, '13.010'), (1, '13.570')] [2023-10-07 19:39:37,485][67511] Saving new best policy, reward=13.010! [2023-10-07 19:39:40,016][67871] Updated weights for policy 1, policy_version 840 (0.0007) [2023-10-07 19:39:40,392][67871] Updated weights for policy 1, policy_version 850 (0.0009) [2023-10-07 19:39:40,648][67838] Updated weights for policy 0, policy_version 842 (0.0008) [2023-10-07 19:39:40,749][67871] Updated weights for policy 1, policy_version 860 (0.0008) [2023-10-07 19:39:41,021][67838] Updated weights for policy 0, policy_version 852 (0.0007) [2023-10-07 19:39:41,387][67838] Updated weights for policy 0, policy_version 862 (0.0008) [2023-10-07 19:39:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12367.4). Total num frames: 1769472. Throughput: 0: 1610.1, 1: 1621.3. Samples: 444040. Policy #0 lag: (min: 8.0, avg: 20.5, max: 40.0) [2023-10-07 19:39:42,478][66916] Avg episode reward: [(0, '13.360'), (1, '13.200')] [2023-10-07 19:39:42,479][67511] Saving new best policy, reward=13.360! [2023-10-07 19:39:45,244][67871] Updated weights for policy 1, policy_version 870 (0.0009) [2023-10-07 19:39:45,613][67838] Updated weights for policy 0, policy_version 872 (0.0009) [2023-10-07 19:39:45,619][67871] Updated weights for policy 1, policy_version 880 (0.0008) [2023-10-07 19:39:45,983][67838] Updated weights for policy 0, policy_version 882 (0.0009) [2023-10-07 19:39:45,990][67871] Updated weights for policy 1, policy_version 890 (0.0008) [2023-10-07 19:39:46,346][67838] Updated weights for policy 0, policy_version 892 (0.0007) [2023-10-07 19:39:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12392.4). Total num frames: 1835008. Throughput: 0: 1614.8, 1: 1609.9. Samples: 462426. Policy #0 lag: (min: 16.0, avg: 36.2, max: 48.0) [2023-10-07 19:39:47,478][66916] Avg episode reward: [(0, '13.290'), (1, '14.190')] [2023-10-07 19:39:50,183][67871] Updated weights for policy 1, policy_version 900 (0.0010) [2023-10-07 19:39:50,546][67871] Updated weights for policy 1, policy_version 910 (0.0010) [2023-10-07 19:39:50,714][67838] Updated weights for policy 0, policy_version 902 (0.0008) [2023-10-07 19:39:50,908][67871] Updated weights for policy 1, policy_version 920 (0.0007) [2023-10-07 19:39:51,099][67838] Updated weights for policy 0, policy_version 912 (0.0008) [2023-10-07 19:39:51,463][67838] Updated weights for policy 0, policy_version 922 (0.0009) [2023-10-07 19:39:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12415.8). Total num frames: 1900544. Throughput: 0: 1603.3, 1: 1601.7. Samples: 480948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:39:52,477][66916] Avg episode reward: [(0, '13.150'), (1, '14.370')] [2023-10-07 19:39:55,196][67871] Updated weights for policy 1, policy_version 930 (0.0007) [2023-10-07 19:39:55,565][67871] Updated weights for policy 1, policy_version 940 (0.0010) [2023-10-07 19:39:55,722][67838] Updated weights for policy 0, policy_version 932 (0.0009) [2023-10-07 19:39:55,925][67871] Updated weights for policy 1, policy_version 950 (0.0009) [2023-10-07 19:39:56,093][67838] Updated weights for policy 0, policy_version 942 (0.0011) [2023-10-07 19:39:56,300][67871] Updated weights for policy 1, policy_version 960 (0.0007) [2023-10-07 19:39:56,466][67838] Updated weights for policy 0, policy_version 952 (0.0010) [2023-10-07 19:39:57,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12437.6). Total num frames: 1966080. Throughput: 0: 1609.2, 1: 1612.9. Samples: 492002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:39:57,478][66916] Avg episode reward: [(0, '14.200'), (1, '14.900')] [2023-10-07 19:39:57,479][67676] Saving new best policy, reward=14.900! [2023-10-07 19:39:57,479][67511] Saving new best policy, reward=14.200! [2023-10-07 19:40:00,570][67838] Updated weights for policy 0, policy_version 962 (0.0007) [2023-10-07 19:40:00,607][67871] Updated weights for policy 1, policy_version 970 (0.0008) [2023-10-07 19:40:00,947][67838] Updated weights for policy 0, policy_version 972 (0.0009) [2023-10-07 19:40:00,974][67871] Updated weights for policy 1, policy_version 980 (0.0009) [2023-10-07 19:40:01,317][67838] Updated weights for policy 0, policy_version 982 (0.0007) [2023-10-07 19:40:01,335][67871] Updated weights for policy 1, policy_version 990 (0.0009) [2023-10-07 19:40:01,683][67838] Updated weights for policy 0, policy_version 992 (0.0008) [2023-10-07 19:40:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12458.2). Total num frames: 2031616. Throughput: 0: 1620.1, 1: 1610.7. Samples: 510768. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 19:40:02,478][66916] Avg episode reward: [(0, '13.940'), (1, '14.140')] [2023-10-07 19:40:05,690][67871] Updated weights for policy 1, policy_version 1000 (0.0009) [2023-10-07 19:40:05,932][67838] Updated weights for policy 0, policy_version 1002 (0.0008) [2023-10-07 19:40:06,055][67871] Updated weights for policy 1, policy_version 1010 (0.0008) [2023-10-07 19:40:06,310][67838] Updated weights for policy 0, policy_version 1012 (0.0009) [2023-10-07 19:40:06,419][67871] Updated weights for policy 1, policy_version 1020 (0.0007) [2023-10-07 19:40:06,682][67838] Updated weights for policy 0, policy_version 1022 (0.0009) [2023-10-07 19:40:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12477.5). Total num frames: 2097152. Throughput: 0: 1606.8, 1: 1597.4. Samples: 529212. Policy #0 lag: (min: 13.0, avg: 21.0, max: 45.0) [2023-10-07 19:40:07,478][66916] Avg episode reward: [(0, '14.140'), (1, '14.460')] [2023-10-07 19:40:10,858][67871] Updated weights for policy 1, policy_version 1030 (0.0011) [2023-10-07 19:40:10,995][67838] Updated weights for policy 0, policy_version 1032 (0.0010) [2023-10-07 19:40:11,229][67871] Updated weights for policy 1, policy_version 1040 (0.0009) [2023-10-07 19:40:11,375][67838] Updated weights for policy 0, policy_version 1042 (0.0008) [2023-10-07 19:40:11,605][67871] Updated weights for policy 1, policy_version 1050 (0.0007) [2023-10-07 19:40:11,746][67838] Updated weights for policy 0, policy_version 1052 (0.0008) [2023-10-07 19:40:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12495.7). Total num frames: 2162688. Throughput: 0: 1610.4, 1: 1602.3. Samples: 540238. Policy #0 lag: (min: 3.0, avg: 5.8, max: 35.0) [2023-10-07 19:40:12,478][66916] Avg episode reward: [(0, '13.690'), (1, '15.260')] [2023-10-07 19:40:12,479][67676] Saving new best policy, reward=15.260! [2023-10-07 19:40:15,667][67871] Updated weights for policy 1, policy_version 1060 (0.0007) [2023-10-07 19:40:16,023][67838] Updated weights for policy 0, policy_version 1062 (0.0008) [2023-10-07 19:40:16,046][67871] Updated weights for policy 1, policy_version 1070 (0.0007) [2023-10-07 19:40:16,396][67838] Updated weights for policy 0, policy_version 1072 (0.0007) [2023-10-07 19:40:16,426][67871] Updated weights for policy 1, policy_version 1080 (0.0007) [2023-10-07 19:40:16,775][67838] Updated weights for policy 0, policy_version 1082 (0.0010) [2023-10-07 19:40:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12512.8). Total num frames: 2228224. Throughput: 0: 1622.3, 1: 1614.5. Samples: 559400. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-07 19:40:17,478][66916] Avg episode reward: [(0, '14.400'), (1, '15.070')] [2023-10-07 19:40:17,479][67511] Saving new best policy, reward=14.400! [2023-10-07 19:40:20,884][67871] Updated weights for policy 1, policy_version 1090 (0.0007) [2023-10-07 19:40:21,051][67838] Updated weights for policy 0, policy_version 1092 (0.0009) [2023-10-07 19:40:21,255][67871] Updated weights for policy 1, policy_version 1100 (0.0008) [2023-10-07 19:40:21,426][67838] Updated weights for policy 0, policy_version 1102 (0.0008) [2023-10-07 19:40:21,627][67871] Updated weights for policy 1, policy_version 1110 (0.0010) [2023-10-07 19:40:21,808][67838] Updated weights for policy 0, policy_version 1112 (0.0009) [2023-10-07 19:40:21,993][67871] Updated weights for policy 1, policy_version 1120 (0.0009) [2023-10-07 19:40:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12529.1). Total num frames: 2293760. Throughput: 0: 1609.9, 1: 1599.0. Samples: 577432. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 19:40:22,477][66916] Avg episode reward: [(0, '13.700'), (1, '15.510')] [2023-10-07 19:40:22,486][67676] Saving new best policy, reward=15.510! [2023-10-07 19:40:26,106][67838] Updated weights for policy 0, policy_version 1122 (0.0010) [2023-10-07 19:40:26,446][67871] Updated weights for policy 1, policy_version 1130 (0.0007) [2023-10-07 19:40:26,487][67838] Updated weights for policy 0, policy_version 1132 (0.0007) [2023-10-07 19:40:26,826][67871] Updated weights for policy 1, policy_version 1140 (0.0009) [2023-10-07 19:40:26,859][67838] Updated weights for policy 0, policy_version 1142 (0.0007) [2023-10-07 19:40:27,182][67871] Updated weights for policy 1, policy_version 1150 (0.0007) [2023-10-07 19:40:27,221][67838] Updated weights for policy 0, policy_version 1152 (0.0007) [2023-10-07 19:40:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12544.4). Total num frames: 2359296. Throughput: 0: 1607.7, 1: 1593.5. Samples: 588094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:40:27,477][66916] Avg episode reward: [(0, '13.830'), (1, '15.220')] [2023-10-07 19:40:31,493][67871] Updated weights for policy 1, policy_version 1160 (0.0010) [2023-10-07 19:40:31,499][67838] Updated weights for policy 0, policy_version 1162 (0.0010) [2023-10-07 19:40:31,874][67871] Updated weights for policy 1, policy_version 1170 (0.0008) [2023-10-07 19:40:31,875][67838] Updated weights for policy 0, policy_version 1172 (0.0009) [2023-10-07 19:40:32,246][67838] Updated weights for policy 0, policy_version 1182 (0.0008) [2023-10-07 19:40:32,247][67871] Updated weights for policy 1, policy_version 1180 (0.0008) [2023-10-07 19:40:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12559.0). Total num frames: 2424832. Throughput: 0: 1619.2, 1: 1610.5. Samples: 607760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-07 19:40:32,478][66916] Avg episode reward: [(0, '14.940'), (1, '15.420')] [2023-10-07 19:40:32,479][67511] Saving new best policy, reward=14.940! [2023-10-07 19:40:36,487][67838] Updated weights for policy 0, policy_version 1192 (0.0008) [2023-10-07 19:40:36,620][67871] Updated weights for policy 1, policy_version 1190 (0.0008) [2023-10-07 19:40:36,853][67838] Updated weights for policy 0, policy_version 1202 (0.0008) [2023-10-07 19:40:36,998][67871] Updated weights for policy 1, policy_version 1200 (0.0008) [2023-10-07 19:40:37,214][67838] Updated weights for policy 0, policy_version 1212 (0.0009) [2023-10-07 19:40:37,365][67871] Updated weights for policy 1, policy_version 1210 (0.0007) [2023-10-07 19:40:37,476][66916] Fps is (10 sec: 9830.5, 60 sec: 12561.1, 300 sec: 12407.4). Total num frames: 2457600. Throughput: 0: 1612.7, 1: 1611.2. Samples: 626024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:40:37,477][66916] Avg episode reward: [(0, '15.420'), (1, '15.420')] [2023-10-07 19:40:37,483][67511] Saving new best policy, reward=15.420! [2023-10-07 19:40:41,562][67871] Updated weights for policy 1, policy_version 1220 (0.0008) [2023-10-07 19:40:41,569][67838] Updated weights for policy 0, policy_version 1222 (0.0009) [2023-10-07 19:40:41,930][67871] Updated weights for policy 1, policy_version 1230 (0.0008) [2023-10-07 19:40:41,933][67838] Updated weights for policy 0, policy_version 1232 (0.0007) [2023-10-07 19:40:42,300][67871] Updated weights for policy 1, policy_version 1240 (0.0010) [2023-10-07 19:40:42,305][67838] Updated weights for policy 0, policy_version 1242 (0.0010) [2023-10-07 19:40:42,477][66916] Fps is (10 sec: 6553.6, 60 sec: 12014.9, 300 sec: 12263.3). Total num frames: 2490368. Throughput: 0: 1601.3, 1: 1596.2. Samples: 635890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:40:42,478][66916] Avg episode reward: [(0, '15.810'), (1, '15.250')] [2023-10-07 19:40:42,531][67511] Saving new best policy, reward=15.810! [2023-10-07 19:40:46,640][67871] Updated weights for policy 1, policy_version 1250 (0.0009) [2023-10-07 19:40:46,785][67838] Updated weights for policy 0, policy_version 1252 (0.0008) [2023-10-07 19:40:47,000][67871] Updated weights for policy 1, policy_version 1260 (0.0009) [2023-10-07 19:40:47,157][67838] Updated weights for policy 0, policy_version 1262 (0.0007) [2023-10-07 19:40:47,364][67871] Updated weights for policy 1, policy_version 1270 (0.0007) [2023-10-07 19:40:47,476][66916] Fps is (10 sec: 9830.4, 60 sec: 12015.0, 300 sec: 12283.6). Total num frames: 2555904. Throughput: 0: 1609.9, 1: 1603.7. Samples: 655378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:40:47,477][66916] Avg episode reward: [(0, '15.550'), (1, '15.190')] [2023-10-07 19:40:47,535][67838] Updated weights for policy 0, policy_version 1272 (0.0008) [2023-10-07 19:40:47,731][67871] Updated weights for policy 1, policy_version 1280 (0.0009) [2023-10-07 19:40:51,738][67838] Updated weights for policy 0, policy_version 1282 (0.0009) [2023-10-07 19:40:52,097][67871] Updated weights for policy 1, policy_version 1290 (0.0009) [2023-10-07 19:40:52,112][67838] Updated weights for policy 0, policy_version 1292 (0.0008) [2023-10-07 19:40:52,462][67871] Updated weights for policy 1, policy_version 1300 (0.0007) [2023-10-07 19:40:52,476][67838] Updated weights for policy 0, policy_version 1302 (0.0009) [2023-10-07 19:40:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 12014.9, 300 sec: 12302.9). Total num frames: 2621440. Throughput: 0: 1618.3, 1: 1610.2. Samples: 674496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:40:52,477][66916] Avg episode reward: [(0, '16.340'), (1, '15.930')] [2023-10-07 19:40:52,830][67871] Updated weights for policy 1, policy_version 1310 (0.0007) [2023-10-07 19:40:52,843][67511] Saving new best policy, reward=16.340! [2023-10-07 19:40:52,848][67838] Updated weights for policy 0, policy_version 1312 (0.0009) [2023-10-07 19:40:52,904][67676] Saving new best policy, reward=15.930! [2023-10-07 19:40:57,033][67871] Updated weights for policy 1, policy_version 1320 (0.0007) [2023-10-07 19:40:57,308][67838] Updated weights for policy 0, policy_version 1322 (0.0008) [2023-10-07 19:40:57,403][67871] Updated weights for policy 1, policy_version 1330 (0.0008) [2023-10-07 19:40:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 12014.9, 300 sec: 12321.3). Total num frames: 2686976. Throughput: 0: 1594.4, 1: 1591.8. Samples: 683618. Policy #0 lag: (min: 26.0, avg: 26.4, max: 40.0) [2023-10-07 19:40:57,478][66916] Avg episode reward: [(0, '16.470'), (1, '16.760')] [2023-10-07 19:40:57,688][67838] Updated weights for policy 0, policy_version 1332 (0.0007) [2023-10-07 19:40:57,767][67871] Updated weights for policy 1, policy_version 1340 (0.0008) [2023-10-07 19:40:57,912][67676] Saving new best policy, reward=16.760! [2023-10-07 19:40:58,063][67838] Updated weights for policy 0, policy_version 1342 (0.0008) [2023-10-07 19:40:58,127][67511] Saving new best policy, reward=16.470! [2023-10-07 19:41:02,100][67871] Updated weights for policy 1, policy_version 1350 (0.0008) [2023-10-07 19:41:02,393][67838] Updated weights for policy 0, policy_version 1352 (0.0009) [2023-10-07 19:41:02,474][67871] Updated weights for policy 1, policy_version 1360 (0.0007) [2023-10-07 19:41:02,476][66916] Fps is (10 sec: 13107.1, 60 sec: 12015.0, 300 sec: 12339.0). Total num frames: 2752512. Throughput: 0: 1598.0, 1: 1597.9. Samples: 703216. Policy #0 lag: (min: 28.0, avg: 35.8, max: 60.0) [2023-10-07 19:41:02,477][66916] Avg episode reward: [(0, '15.840'), (1, '16.460')] [2023-10-07 19:41:02,765][67838] Updated weights for policy 0, policy_version 1362 (0.0009) [2023-10-07 19:41:02,834][67871] Updated weights for policy 1, policy_version 1370 (0.0007) [2023-10-07 19:41:03,140][67838] Updated weights for policy 0, policy_version 1372 (0.0008) [2023-10-07 19:41:06,933][67871] Updated weights for policy 1, policy_version 1380 (0.0007) [2023-10-07 19:41:07,182][67838] Updated weights for policy 0, policy_version 1382 (0.0007) [2023-10-07 19:41:07,296][67871] Updated weights for policy 1, policy_version 1390 (0.0008) [2023-10-07 19:41:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 12014.9, 300 sec: 12355.8). Total num frames: 2818048. Throughput: 0: 1618.0, 1: 1620.8. Samples: 723180. Policy #0 lag: (min: 10.0, avg: 17.3, max: 42.0) [2023-10-07 19:41:07,478][66916] Avg episode reward: [(0, '16.210'), (1, '16.270')] [2023-10-07 19:41:07,550][67838] Updated weights for policy 0, policy_version 1392 (0.0007) [2023-10-07 19:41:07,666][67871] Updated weights for policy 1, policy_version 1400 (0.0008) [2023-10-07 19:41:07,924][67838] Updated weights for policy 0, policy_version 1402 (0.0007) [2023-10-07 19:41:07,955][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000001408_1441792.pth... [2023-10-07 19:41:08,148][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000001408_1441792.pth... [2023-10-07 19:41:11,748][67871] Updated weights for policy 1, policy_version 1410 (0.0008) [2023-10-07 19:41:12,128][67871] Updated weights for policy 1, policy_version 1420 (0.0010) [2023-10-07 19:41:12,218][67838] Updated weights for policy 0, policy_version 1412 (0.0009) [2023-10-07 19:41:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 12015.0, 300 sec: 12371.9). Total num frames: 2883584. Throughput: 0: 1597.0, 1: 1605.8. Samples: 732218. Policy #0 lag: (min: 10.0, avg: 18.3, max: 42.0) [2023-10-07 19:41:12,477][66916] Avg episode reward: [(0, '16.330'), (1, '17.280')] [2023-10-07 19:41:12,488][67871] Updated weights for policy 1, policy_version 1430 (0.0009) [2023-10-07 19:41:12,590][67838] Updated weights for policy 0, policy_version 1422 (0.0008) [2023-10-07 19:41:12,851][67676] Saving new best policy, reward=17.280! [2023-10-07 19:41:12,856][67871] Updated weights for policy 1, policy_version 1440 (0.0008) [2023-10-07 19:41:12,965][67838] Updated weights for policy 0, policy_version 1432 (0.0008) [2023-10-07 19:41:17,143][67838] Updated weights for policy 0, policy_version 1442 (0.0008) [2023-10-07 19:41:17,256][67871] Updated weights for policy 1, policy_version 1450 (0.0008) [2023-10-07 19:41:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 12015.0, 300 sec: 12387.4). Total num frames: 2949120. Throughput: 0: 1595.5, 1: 1610.8. Samples: 752042. Policy #0 lag: (min: 15.0, avg: 21.7, max: 47.0) [2023-10-07 19:41:17,477][66916] Avg episode reward: [(0, '17.420'), (1, '17.310')] [2023-10-07 19:41:17,518][67838] Updated weights for policy 0, policy_version 1452 (0.0007) [2023-10-07 19:41:17,624][67871] Updated weights for policy 1, policy_version 1460 (0.0008) [2023-10-07 19:41:17,877][67838] Updated weights for policy 0, policy_version 1462 (0.0007) [2023-10-07 19:41:17,990][67871] Updated weights for policy 1, policy_version 1470 (0.0009) [2023-10-07 19:41:18,065][67676] Saving new best policy, reward=17.310! [2023-10-07 19:41:18,251][67511] Saving new best policy, reward=17.420! [2023-10-07 19:41:18,254][67838] Updated weights for policy 0, policy_version 1472 (0.0009) [2023-10-07 19:41:22,175][67871] Updated weights for policy 1, policy_version 1480 (0.0007) [2023-10-07 19:41:22,476][66916] Fps is (10 sec: 13107.0, 60 sec: 12014.9, 300 sec: 12402.2). Total num frames: 3014656. Throughput: 0: 1616.6, 1: 1621.8. Samples: 771752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:41:22,477][66916] Avg episode reward: [(0, '17.420'), (1, '17.130')] [2023-10-07 19:41:22,532][67871] Updated weights for policy 1, policy_version 1490 (0.0008) [2023-10-07 19:41:22,587][67838] Updated weights for policy 0, policy_version 1482 (0.0009) [2023-10-07 19:41:22,910][67871] Updated weights for policy 1, policy_version 1500 (0.0008) [2023-10-07 19:41:22,969][67838] Updated weights for policy 0, policy_version 1492 (0.0007) [2023-10-07 19:41:23,345][67838] Updated weights for policy 0, policy_version 1502 (0.0008) [2023-10-07 19:41:27,213][67871] Updated weights for policy 1, policy_version 1510 (0.0008) [2023-10-07 19:41:27,477][66916] Fps is (10 sec: 13107.1, 60 sec: 12014.9, 300 sec: 12416.4). Total num frames: 3080192. Throughput: 0: 1598.0, 1: 1611.4. Samples: 780310. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-07 19:41:27,478][66916] Avg episode reward: [(0, '17.830'), (1, '16.760')] [2023-10-07 19:41:27,547][67838] Updated weights for policy 0, policy_version 1512 (0.0008) [2023-10-07 19:41:27,584][67871] Updated weights for policy 1, policy_version 1520 (0.0009) [2023-10-07 19:41:27,925][67838] Updated weights for policy 0, policy_version 1522 (0.0007) [2023-10-07 19:41:27,952][67871] Updated weights for policy 1, policy_version 1530 (0.0008) [2023-10-07 19:41:28,296][67838] Updated weights for policy 0, policy_version 1532 (0.0010) [2023-10-07 19:41:28,435][67511] Saving new best policy, reward=17.830! [2023-10-07 19:41:32,262][67871] Updated weights for policy 1, policy_version 1540 (0.0008) [2023-10-07 19:41:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 12015.0, 300 sec: 12430.0). Total num frames: 3145728. Throughput: 0: 1600.9, 1: 1614.0. Samples: 800048. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-07 19:41:32,477][66916] Avg episode reward: [(0, '17.380'), (1, '17.130')] [2023-10-07 19:41:32,598][67838] Updated weights for policy 0, policy_version 1542 (0.0008) [2023-10-07 19:41:32,630][67871] Updated weights for policy 1, policy_version 1550 (0.0007) [2023-10-07 19:41:32,961][67838] Updated weights for policy 0, policy_version 1552 (0.0009) [2023-10-07 19:41:32,991][67871] Updated weights for policy 1, policy_version 1560 (0.0008) [2023-10-07 19:41:33,345][67838] Updated weights for policy 0, policy_version 1562 (0.0008) [2023-10-07 19:41:37,121][67871] Updated weights for policy 1, policy_version 1570 (0.0009) [2023-10-07 19:41:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 12561.0, 300 sec: 12443.1). Total num frames: 3211264. Throughput: 0: 1606.2, 1: 1622.0. Samples: 819762. Policy #0 lag: (min: 17.0, avg: 17.5, max: 32.0) [2023-10-07 19:41:37,477][66916] Avg episode reward: [(0, '17.000'), (1, '18.130')] [2023-10-07 19:41:37,492][67871] Updated weights for policy 1, policy_version 1580 (0.0009) [2023-10-07 19:41:37,661][67838] Updated weights for policy 0, policy_version 1572 (0.0008) [2023-10-07 19:41:37,866][67871] Updated weights for policy 1, policy_version 1590 (0.0008) [2023-10-07 19:41:38,039][67838] Updated weights for policy 0, policy_version 1582 (0.0008) [2023-10-07 19:41:38,231][67676] Saving new best policy, reward=18.130! [2023-10-07 19:41:38,236][67871] Updated weights for policy 1, policy_version 1600 (0.0008) [2023-10-07 19:41:38,404][67838] Updated weights for policy 0, policy_version 1592 (0.0007) [2023-10-07 19:41:42,461][67871] Updated weights for policy 1, policy_version 1610 (0.0009) [2023-10-07 19:41:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12455.8). Total num frames: 3276800. Throughput: 0: 1604.3, 1: 1616.8. Samples: 828564. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-07 19:41:42,477][66916] Avg episode reward: [(0, '17.590'), (1, '17.890')] [2023-10-07 19:41:42,832][67871] Updated weights for policy 1, policy_version 1620 (0.0008) [2023-10-07 19:41:42,837][67838] Updated weights for policy 0, policy_version 1602 (0.0008) [2023-10-07 19:41:43,201][67871] Updated weights for policy 1, policy_version 1630 (0.0008) [2023-10-07 19:41:43,222][67838] Updated weights for policy 0, policy_version 1612 (0.0008) [2023-10-07 19:41:43,596][67838] Updated weights for policy 0, policy_version 1622 (0.0007) [2023-10-07 19:41:43,966][67838] Updated weights for policy 0, policy_version 1632 (0.0007) [2023-10-07 19:41:47,474][67871] Updated weights for policy 1, policy_version 1640 (0.0009) [2023-10-07 19:41:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12467.9). Total num frames: 3342336. Throughput: 0: 1609.9, 1: 1616.4. Samples: 848398. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 19:41:47,477][66916] Avg episode reward: [(0, '18.020'), (1, '17.960')] [2023-10-07 19:41:47,478][67511] Saving new best policy, reward=18.020! [2023-10-07 19:41:47,843][67871] Updated weights for policy 1, policy_version 1650 (0.0009) [2023-10-07 19:41:48,135][67838] Updated weights for policy 0, policy_version 1642 (0.0007) [2023-10-07 19:41:48,209][67871] Updated weights for policy 1, policy_version 1660 (0.0008) [2023-10-07 19:41:48,501][67838] Updated weights for policy 0, policy_version 1652 (0.0007) [2023-10-07 19:41:48,878][67838] Updated weights for policy 0, policy_version 1662 (0.0007) [2023-10-07 19:41:52,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 12479.6). Total num frames: 3407872. Throughput: 0: 1612.0, 1: 1612.5. Samples: 868282. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-10-07 19:41:52,478][66916] Avg episode reward: [(0, '18.320'), (1, '18.060')] [2023-10-07 19:41:52,487][67511] Saving new best policy, reward=18.320! [2023-10-07 19:41:52,543][67871] Updated weights for policy 1, policy_version 1670 (0.0007) [2023-10-07 19:41:52,918][67871] Updated weights for policy 1, policy_version 1680 (0.0007) [2023-10-07 19:41:53,096][67838] Updated weights for policy 0, policy_version 1672 (0.0007) [2023-10-07 19:41:53,285][67871] Updated weights for policy 1, policy_version 1690 (0.0009) [2023-10-07 19:41:53,464][67838] Updated weights for policy 0, policy_version 1682 (0.0009) [2023-10-07 19:41:53,834][67838] Updated weights for policy 0, policy_version 1692 (0.0009) [2023-10-07 19:41:57,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12490.9). Total num frames: 3473408. Throughput: 0: 1607.9, 1: 1607.5. Samples: 876914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:41:57,478][66916] Avg episode reward: [(0, '18.620'), (1, '18.390')] [2023-10-07 19:41:57,479][67511] Saving new best policy, reward=18.620! [2023-10-07 19:41:57,573][67871] Updated weights for policy 1, policy_version 1700 (0.0009) [2023-10-07 19:41:57,938][67871] Updated weights for policy 1, policy_version 1710 (0.0009) [2023-10-07 19:41:58,079][67838] Updated weights for policy 0, policy_version 1702 (0.0010) [2023-10-07 19:41:58,312][67871] Updated weights for policy 1, policy_version 1720 (0.0007) [2023-10-07 19:41:58,443][67838] Updated weights for policy 0, policy_version 1712 (0.0009) [2023-10-07 19:41:58,597][67676] Saving new best policy, reward=18.390! [2023-10-07 19:41:58,823][67838] Updated weights for policy 0, policy_version 1722 (0.0007) [2023-10-07 19:42:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12501.8). Total num frames: 3538944. Throughput: 0: 1606.4, 1: 1604.8. Samples: 896542. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-07 19:42:02,477][66916] Avg episode reward: [(0, '18.750'), (1, '18.850')] [2023-10-07 19:42:02,478][67511] Saving new best policy, reward=18.750! [2023-10-07 19:42:02,553][67871] Updated weights for policy 1, policy_version 1730 (0.0007) [2023-10-07 19:42:02,935][67871] Updated weights for policy 1, policy_version 1740 (0.0008) [2023-10-07 19:42:03,047][67838] Updated weights for policy 0, policy_version 1732 (0.0007) [2023-10-07 19:42:03,304][67871] Updated weights for policy 1, policy_version 1750 (0.0008) [2023-10-07 19:42:03,420][67838] Updated weights for policy 0, policy_version 1742 (0.0008) [2023-10-07 19:42:03,680][67676] Saving new best policy, reward=18.850! [2023-10-07 19:42:03,682][67871] Updated weights for policy 1, policy_version 1760 (0.0007) [2023-10-07 19:42:03,798][67838] Updated weights for policy 0, policy_version 1752 (0.0009) [2023-10-07 19:42:07,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12512.3). Total num frames: 3604480. Throughput: 0: 1605.5, 1: 1604.8. Samples: 916216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:42:07,477][66916] Avg episode reward: [(0, '18.730'), (1, '18.670')] [2023-10-07 19:42:08,066][67871] Updated weights for policy 1, policy_version 1770 (0.0008) [2023-10-07 19:42:08,262][67838] Updated weights for policy 0, policy_version 1762 (0.0010) [2023-10-07 19:42:08,427][67871] Updated weights for policy 1, policy_version 1780 (0.0007) [2023-10-07 19:42:08,669][67838] Updated weights for policy 0, policy_version 1772 (0.0009) [2023-10-07 19:42:08,800][67871] Updated weights for policy 1, policy_version 1790 (0.0008) [2023-10-07 19:42:09,045][67838] Updated weights for policy 0, policy_version 1782 (0.0010) [2023-10-07 19:42:09,416][67838] Updated weights for policy 0, policy_version 1792 (0.0011) [2023-10-07 19:42:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12522.4). Total num frames: 3670016. Throughput: 0: 1607.3, 1: 1606.1. Samples: 924912. Policy #0 lag: (min: 8.0, avg: 31.7, max: 40.0) [2023-10-07 19:42:12,477][66916] Avg episode reward: [(0, '18.430'), (1, '19.200')] [2023-10-07 19:42:12,478][67676] Saving new best policy, reward=19.200! [2023-10-07 19:42:13,128][67871] Updated weights for policy 1, policy_version 1800 (0.0008) [2023-10-07 19:42:13,491][67871] Updated weights for policy 1, policy_version 1810 (0.0008) [2023-10-07 19:42:13,545][67838] Updated weights for policy 0, policy_version 1802 (0.0007) [2023-10-07 19:42:13,863][67871] Updated weights for policy 1, policy_version 1820 (0.0007) [2023-10-07 19:42:13,912][67838] Updated weights for policy 0, policy_version 1812 (0.0009) [2023-10-07 19:42:14,286][67838] Updated weights for policy 0, policy_version 1822 (0.0009) [2023-10-07 19:42:17,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12662.9). Total num frames: 3735552. Throughput: 0: 1603.5, 1: 1612.7. Samples: 944778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:42:17,478][66916] Avg episode reward: [(0, '18.570'), (1, '19.320')] [2023-10-07 19:42:17,479][67676] Saving new best policy, reward=19.320! [2023-10-07 19:42:17,924][67871] Updated weights for policy 1, policy_version 1830 (0.0009) [2023-10-07 19:42:18,285][67871] Updated weights for policy 1, policy_version 1840 (0.0008) [2023-10-07 19:42:18,549][67838] Updated weights for policy 0, policy_version 1832 (0.0008) [2023-10-07 19:42:18,650][67871] Updated weights for policy 1, policy_version 1850 (0.0007) [2023-10-07 19:42:18,925][67838] Updated weights for policy 0, policy_version 1842 (0.0007) [2023-10-07 19:42:19,294][67838] Updated weights for policy 0, policy_version 1852 (0.0007) [2023-10-07 19:42:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 3801088. Throughput: 0: 1607.2, 1: 1613.3. Samples: 964684. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-07 19:42:22,477][66916] Avg episode reward: [(0, '18.900'), (1, '20.070')] [2023-10-07 19:42:22,486][67511] Saving new best policy, reward=18.900! [2023-10-07 19:42:22,486][67676] Saving new best policy, reward=20.070! [2023-10-07 19:42:22,813][67871] Updated weights for policy 1, policy_version 1860 (0.0008) [2023-10-07 19:42:23,175][67871] Updated weights for policy 1, policy_version 1870 (0.0008) [2023-10-07 19:42:23,329][67838] Updated weights for policy 0, policy_version 1862 (0.0009) [2023-10-07 19:42:23,543][67871] Updated weights for policy 1, policy_version 1880 (0.0009) [2023-10-07 19:42:23,689][67838] Updated weights for policy 0, policy_version 1872 (0.0009) [2023-10-07 19:42:24,061][67838] Updated weights for policy 0, policy_version 1882 (0.0007) [2023-10-07 19:42:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 3866624. Throughput: 0: 1610.0, 1: 1615.0. Samples: 973688. Policy #0 lag: (min: 26.0, avg: 28.9, max: 58.0) [2023-10-07 19:42:27,478][66916] Avg episode reward: [(0, '19.100'), (1, '19.060')] [2023-10-07 19:42:27,479][67511] Saving new best policy, reward=19.100! [2023-10-07 19:42:27,817][67871] Updated weights for policy 1, policy_version 1890 (0.0010) [2023-10-07 19:42:28,182][67871] Updated weights for policy 1, policy_version 1900 (0.0009) [2023-10-07 19:42:28,236][67838] Updated weights for policy 0, policy_version 1892 (0.0008) [2023-10-07 19:42:28,547][67871] Updated weights for policy 1, policy_version 1910 (0.0007) [2023-10-07 19:42:28,606][67838] Updated weights for policy 0, policy_version 1902 (0.0008) [2023-10-07 19:42:28,920][67871] Updated weights for policy 1, policy_version 1920 (0.0009) [2023-10-07 19:42:28,984][67838] Updated weights for policy 0, policy_version 1912 (0.0009) [2023-10-07 19:42:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 3932160. Throughput: 0: 1613.5, 1: 1615.1. Samples: 993682. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 19:42:32,477][66916] Avg episode reward: [(0, '19.610'), (1, '19.900')] [2023-10-07 19:42:32,478][67511] Saving new best policy, reward=19.610! [2023-10-07 19:42:33,111][67871] Updated weights for policy 1, policy_version 1930 (0.0011) [2023-10-07 19:42:33,248][67838] Updated weights for policy 0, policy_version 1922 (0.0010) [2023-10-07 19:42:33,480][67871] Updated weights for policy 1, policy_version 1940 (0.0008) [2023-10-07 19:42:33,616][67838] Updated weights for policy 0, policy_version 1932 (0.0008) [2023-10-07 19:42:33,854][67871] Updated weights for policy 1, policy_version 1950 (0.0008) [2023-10-07 19:42:33,998][67838] Updated weights for policy 0, policy_version 1942 (0.0007) [2023-10-07 19:42:34,378][67838] Updated weights for policy 0, policy_version 1952 (0.0009) [2023-10-07 19:42:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 3997696. Throughput: 0: 1608.5, 1: 1618.5. Samples: 1013496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:42:37,477][66916] Avg episode reward: [(0, '19.570'), (1, '19.640')] [2023-10-07 19:42:38,051][67871] Updated weights for policy 1, policy_version 1960 (0.0008) [2023-10-07 19:42:38,422][67871] Updated weights for policy 1, policy_version 1970 (0.0007) [2023-10-07 19:42:38,681][67838] Updated weights for policy 0, policy_version 1962 (0.0008) [2023-10-07 19:42:38,785][67871] Updated weights for policy 1, policy_version 1980 (0.0008) [2023-10-07 19:42:39,056][67838] Updated weights for policy 0, policy_version 1972 (0.0008) [2023-10-07 19:42:39,427][67838] Updated weights for policy 0, policy_version 1982 (0.0010) [2023-10-07 19:42:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12885.1). Total num frames: 4063232. Throughput: 0: 1609.3, 1: 1620.0. Samples: 1022230. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-07 19:42:42,477][66916] Avg episode reward: [(0, '19.880'), (1, '18.950')] [2023-10-07 19:42:42,478][67511] Saving new best policy, reward=19.880! [2023-10-07 19:42:43,266][67871] Updated weights for policy 1, policy_version 1990 (0.0008) [2023-10-07 19:42:43,637][67871] Updated weights for policy 1, policy_version 2000 (0.0007) [2023-10-07 19:42:43,821][67838] Updated weights for policy 0, policy_version 1992 (0.0009) [2023-10-07 19:42:44,008][67871] Updated weights for policy 1, policy_version 2010 (0.0009) [2023-10-07 19:42:44,201][67838] Updated weights for policy 0, policy_version 2002 (0.0007) [2023-10-07 19:42:44,570][67838] Updated weights for policy 0, policy_version 2012 (0.0007) [2023-10-07 19:42:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 4128768. Throughput: 0: 1609.4, 1: 1618.4. Samples: 1041796. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-10-07 19:42:47,478][66916] Avg episode reward: [(0, '19.990'), (1, '18.820')] [2023-10-07 19:42:47,479][67511] Saving new best policy, reward=19.990! [2023-10-07 19:42:48,302][67871] Updated weights for policy 1, policy_version 2020 (0.0008) [2023-10-07 19:42:48,691][67871] Updated weights for policy 1, policy_version 2030 (0.0011) [2023-10-07 19:42:48,937][67838] Updated weights for policy 0, policy_version 2022 (0.0009) [2023-10-07 19:42:49,052][67871] Updated weights for policy 1, policy_version 2040 (0.0008) [2023-10-07 19:42:49,316][67838] Updated weights for policy 0, policy_version 2032 (0.0008) [2023-10-07 19:42:49,683][67838] Updated weights for policy 0, policy_version 2042 (0.0009) [2023-10-07 19:42:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 12885.0). Total num frames: 4194304. Throughput: 0: 1611.2, 1: 1620.5. Samples: 1061640. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-07 19:42:52,477][66916] Avg episode reward: [(0, '19.680'), (1, '19.730')] [2023-10-07 19:42:53,164][67871] Updated weights for policy 1, policy_version 2050 (0.0010) [2023-10-07 19:42:53,534][67871] Updated weights for policy 1, policy_version 2060 (0.0007) [2023-10-07 19:42:53,905][67871] Updated weights for policy 1, policy_version 2070 (0.0008) [2023-10-07 19:42:53,914][67838] Updated weights for policy 0, policy_version 2052 (0.0009) [2023-10-07 19:42:54,273][67871] Updated weights for policy 1, policy_version 2080 (0.0008) [2023-10-07 19:42:54,306][67838] Updated weights for policy 0, policy_version 2062 (0.0009) [2023-10-07 19:42:54,680][67838] Updated weights for policy 0, policy_version 2072 (0.0007) [2023-10-07 19:42:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 4259840. Throughput: 0: 1612.8, 1: 1621.6. Samples: 1070460. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-07 19:42:57,477][66916] Avg episode reward: [(0, '19.560'), (1, '20.670')] [2023-10-07 19:42:57,478][67676] Saving new best policy, reward=20.670! [2023-10-07 19:42:58,402][67871] Updated weights for policy 1, policy_version 2090 (0.0009) [2023-10-07 19:42:58,771][67871] Updated weights for policy 1, policy_version 2100 (0.0007) [2023-10-07 19:42:58,798][67838] Updated weights for policy 0, policy_version 2082 (0.0008) [2023-10-07 19:42:59,136][67871] Updated weights for policy 1, policy_version 2110 (0.0007) [2023-10-07 19:42:59,174][67838] Updated weights for policy 0, policy_version 2092 (0.0009) [2023-10-07 19:42:59,542][67838] Updated weights for policy 0, policy_version 2102 (0.0009) [2023-10-07 19:42:59,908][67838] Updated weights for policy 0, policy_version 2112 (0.0007) [2023-10-07 19:43:02,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 4325376. Throughput: 0: 1618.0, 1: 1625.2. Samples: 1090720. Policy #0 lag: (min: 26.0, avg: 31.2, max: 58.0) [2023-10-07 19:43:02,477][66916] Avg episode reward: [(0, '19.570'), (1, '21.580')] [2023-10-07 19:43:02,478][67676] Saving new best policy, reward=21.580! [2023-10-07 19:43:03,314][67871] Updated weights for policy 1, policy_version 2120 (0.0008) [2023-10-07 19:43:03,686][67871] Updated weights for policy 1, policy_version 2130 (0.0008) [2023-10-07 19:43:04,055][67871] Updated weights for policy 1, policy_version 2140 (0.0007) [2023-10-07 19:43:04,124][67838] Updated weights for policy 0, policy_version 2122 (0.0009) [2023-10-07 19:43:04,489][67838] Updated weights for policy 0, policy_version 2132 (0.0009) [2023-10-07 19:43:04,864][67838] Updated weights for policy 0, policy_version 2142 (0.0010) [2023-10-07 19:43:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 4390912. Throughput: 0: 1618.0, 1: 1627.5. Samples: 1110732. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-07 19:43:07,478][66916] Avg episode reward: [(0, '19.450'), (1, '21.710')] [2023-10-07 19:43:07,490][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000002144_2195456.pth... [2023-10-07 19:43:07,490][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000002144_2195456.pth... [2023-10-07 19:43:07,520][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000000640_655360.pth [2023-10-07 19:43:07,528][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000000640_655360.pth [2023-10-07 19:43:07,533][67676] Saving new best policy, reward=21.710! [2023-10-07 19:43:08,231][67871] Updated weights for policy 1, policy_version 2150 (0.0009) [2023-10-07 19:43:08,596][67871] Updated weights for policy 1, policy_version 2160 (0.0008) [2023-10-07 19:43:08,958][67871] Updated weights for policy 1, policy_version 2170 (0.0007) [2023-10-07 19:43:08,989][67838] Updated weights for policy 0, policy_version 2152 (0.0007) [2023-10-07 19:43:09,355][67838] Updated weights for policy 0, policy_version 2162 (0.0009) [2023-10-07 19:43:09,726][67838] Updated weights for policy 0, policy_version 2172 (0.0008) [2023-10-07 19:43:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 4456448. Throughput: 0: 1611.8, 1: 1626.0. Samples: 1119388. Policy #0 lag: (min: 31.0, avg: 32.6, max: 58.0) [2023-10-07 19:43:12,477][66916] Avg episode reward: [(0, '19.700'), (1, '20.310')] [2023-10-07 19:43:13,284][67871] Updated weights for policy 1, policy_version 2180 (0.0007) [2023-10-07 19:43:13,648][67871] Updated weights for policy 1, policy_version 2190 (0.0007) [2023-10-07 19:43:14,020][67871] Updated weights for policy 1, policy_version 2200 (0.0008) [2023-10-07 19:43:14,105][67838] Updated weights for policy 0, policy_version 2182 (0.0007) [2023-10-07 19:43:14,478][67838] Updated weights for policy 0, policy_version 2192 (0.0007) [2023-10-07 19:43:14,855][67838] Updated weights for policy 0, policy_version 2202 (0.0008) [2023-10-07 19:43:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 4521984. Throughput: 0: 1611.4, 1: 1623.6. Samples: 1139256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:43:17,478][66916] Avg episode reward: [(0, '20.940'), (1, '21.980')] [2023-10-07 19:43:17,479][67511] Saving new best policy, reward=20.940! [2023-10-07 19:43:17,479][67676] Saving new best policy, reward=21.980! [2023-10-07 19:43:18,208][67871] Updated weights for policy 1, policy_version 2210 (0.0009) [2023-10-07 19:43:18,570][67871] Updated weights for policy 1, policy_version 2220 (0.0007) [2023-10-07 19:43:18,939][67871] Updated weights for policy 1, policy_version 2230 (0.0007) [2023-10-07 19:43:19,024][67838] Updated weights for policy 0, policy_version 2212 (0.0008) [2023-10-07 19:43:19,309][67871] Updated weights for policy 1, policy_version 2240 (0.0007) [2023-10-07 19:43:19,393][67838] Updated weights for policy 0, policy_version 2222 (0.0009) [2023-10-07 19:43:19,770][67838] Updated weights for policy 0, policy_version 2232 (0.0010) [2023-10-07 19:43:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12885.1). Total num frames: 4587520. Throughput: 0: 1615.1, 1: 1620.8. Samples: 1159112. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-07 19:43:22,477][66916] Avg episode reward: [(0, '20.670'), (1, '21.840')] [2023-10-07 19:43:23,566][67871] Updated weights for policy 1, policy_version 2250 (0.0009) [2023-10-07 19:43:23,876][67838] Updated weights for policy 0, policy_version 2242 (0.0010) [2023-10-07 19:43:23,942][67871] Updated weights for policy 1, policy_version 2260 (0.0008) [2023-10-07 19:43:24,249][67838] Updated weights for policy 0, policy_version 2252 (0.0007) [2023-10-07 19:43:24,324][67871] Updated weights for policy 1, policy_version 2270 (0.0007) [2023-10-07 19:43:24,635][67838] Updated weights for policy 0, policy_version 2262 (0.0010) [2023-10-07 19:43:25,002][67838] Updated weights for policy 0, policy_version 2272 (0.0011) [2023-10-07 19:43:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12885.1). Total num frames: 4653056. Throughput: 0: 1620.2, 1: 1619.3. Samples: 1168008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:43:27,477][66916] Avg episode reward: [(0, '20.760'), (1, '20.730')] [2023-10-07 19:43:28,463][67871] Updated weights for policy 1, policy_version 2280 (0.0007) [2023-10-07 19:43:28,832][67871] Updated weights for policy 1, policy_version 2290 (0.0008) [2023-10-07 19:43:29,201][67871] Updated weights for policy 1, policy_version 2300 (0.0009) [2023-10-07 19:43:29,334][67838] Updated weights for policy 0, policy_version 2282 (0.0008) [2023-10-07 19:43:29,705][67838] Updated weights for policy 0, policy_version 2292 (0.0008) [2023-10-07 19:43:30,086][67838] Updated weights for policy 0, policy_version 2302 (0.0009) [2023-10-07 19:43:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 4718592. Throughput: 0: 1619.3, 1: 1626.3. Samples: 1187850. Policy #0 lag: (min: 9.0, avg: 19.2, max: 41.0) [2023-10-07 19:43:32,477][66916] Avg episode reward: [(0, '21.300'), (1, '20.410')] [2023-10-07 19:43:32,478][67511] Saving new best policy, reward=21.300! [2023-10-07 19:43:33,572][67871] Updated weights for policy 1, policy_version 2310 (0.0008) [2023-10-07 19:43:33,963][67871] Updated weights for policy 1, policy_version 2320 (0.0008) [2023-10-07 19:43:34,328][67871] Updated weights for policy 1, policy_version 2330 (0.0008) [2023-10-07 19:43:34,425][67838] Updated weights for policy 0, policy_version 2312 (0.0008) [2023-10-07 19:43:34,797][67838] Updated weights for policy 0, policy_version 2322 (0.0009) [2023-10-07 19:43:35,170][67838] Updated weights for policy 0, policy_version 2332 (0.0009) [2023-10-07 19:43:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 4784128. Throughput: 0: 1618.7, 1: 1623.6. Samples: 1207542. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-07 19:43:37,477][66916] Avg episode reward: [(0, '21.230'), (1, '20.920')] [2023-10-07 19:43:38,621][67871] Updated weights for policy 1, policy_version 2340 (0.0007) [2023-10-07 19:43:38,985][67871] Updated weights for policy 1, policy_version 2350 (0.0007) [2023-10-07 19:43:39,352][67871] Updated weights for policy 1, policy_version 2360 (0.0007) [2023-10-07 19:43:39,497][67838] Updated weights for policy 0, policy_version 2342 (0.0007) [2023-10-07 19:43:39,876][67838] Updated weights for policy 0, policy_version 2352 (0.0008) [2023-10-07 19:43:40,258][67838] Updated weights for policy 0, policy_version 2362 (0.0007) [2023-10-07 19:43:42,477][66916] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 12885.0). Total num frames: 4849664. Throughput: 0: 1626.3, 1: 1622.8. Samples: 1216670. Policy #0 lag: (min: 14.0, avg: 18.7, max: 46.0) [2023-10-07 19:43:42,478][66916] Avg episode reward: [(0, '20.980'), (1, '21.300')] [2023-10-07 19:43:43,638][67871] Updated weights for policy 1, policy_version 2370 (0.0007) [2023-10-07 19:43:44,008][67871] Updated weights for policy 1, policy_version 2380 (0.0009) [2023-10-07 19:43:44,356][67838] Updated weights for policy 0, policy_version 2372 (0.0008) [2023-10-07 19:43:44,386][67871] Updated weights for policy 1, policy_version 2390 (0.0009) [2023-10-07 19:43:44,729][67838] Updated weights for policy 0, policy_version 2382 (0.0009) [2023-10-07 19:43:44,752][67871] Updated weights for policy 1, policy_version 2400 (0.0009) [2023-10-07 19:43:45,099][67838] Updated weights for policy 0, policy_version 2392 (0.0009) [2023-10-07 19:43:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 4915200. Throughput: 0: 1613.4, 1: 1616.4. Samples: 1236064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:43:47,477][66916] Avg episode reward: [(0, '21.680'), (1, '20.500')] [2023-10-07 19:43:47,478][67511] Saving new best policy, reward=21.680! [2023-10-07 19:43:48,944][67871] Updated weights for policy 1, policy_version 2410 (0.0010) [2023-10-07 19:43:49,298][67838] Updated weights for policy 0, policy_version 2402 (0.0008) [2023-10-07 19:43:49,310][67871] Updated weights for policy 1, policy_version 2420 (0.0009) [2023-10-07 19:43:49,668][67838] Updated weights for policy 0, policy_version 2412 (0.0009) [2023-10-07 19:43:49,687][67871] Updated weights for policy 1, policy_version 2430 (0.0008) [2023-10-07 19:43:50,045][67838] Updated weights for policy 0, policy_version 2422 (0.0008) [2023-10-07 19:43:50,419][67838] Updated weights for policy 0, policy_version 2432 (0.0008) [2023-10-07 19:43:52,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 4980736. Throughput: 0: 1614.9, 1: 1614.7. Samples: 1256060. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 19:43:52,477][66916] Avg episode reward: [(0, '21.330'), (1, '19.670')] [2023-10-07 19:43:53,905][67871] Updated weights for policy 1, policy_version 2440 (0.0008) [2023-10-07 19:43:54,279][67871] Updated weights for policy 1, policy_version 2450 (0.0008) [2023-10-07 19:43:54,644][67871] Updated weights for policy 1, policy_version 2460 (0.0008) [2023-10-07 19:43:54,666][67838] Updated weights for policy 0, policy_version 2442 (0.0007) [2023-10-07 19:43:55,050][67838] Updated weights for policy 0, policy_version 2452 (0.0007) [2023-10-07 19:43:55,415][67838] Updated weights for policy 0, policy_version 2462 (0.0008) [2023-10-07 19:43:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 5046272. Throughput: 0: 1627.7, 1: 1616.0. Samples: 1265354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:43:57,478][66916] Avg episode reward: [(0, '22.660'), (1, '21.210')] [2023-10-07 19:43:57,479][67511] Saving new best policy, reward=22.660! [2023-10-07 19:43:59,025][67871] Updated weights for policy 1, policy_version 2470 (0.0007) [2023-10-07 19:43:59,397][67871] Updated weights for policy 1, policy_version 2480 (0.0007) [2023-10-07 19:43:59,726][67838] Updated weights for policy 0, policy_version 2472 (0.0008) [2023-10-07 19:43:59,763][67871] Updated weights for policy 1, policy_version 2490 (0.0008) [2023-10-07 19:44:00,090][67838] Updated weights for policy 0, policy_version 2482 (0.0008) [2023-10-07 19:44:00,468][67838] Updated weights for policy 0, policy_version 2492 (0.0009) [2023-10-07 19:44:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 5111808. Throughput: 0: 1611.6, 1: 1622.1. Samples: 1284772. Policy #0 lag: (min: 21.0, avg: 26.0, max: 53.0) [2023-10-07 19:44:02,478][66916] Avg episode reward: [(0, '23.160'), (1, '20.180')] [2023-10-07 19:44:02,479][67511] Saving new best policy, reward=23.160! [2023-10-07 19:44:03,908][67871] Updated weights for policy 1, policy_version 2500 (0.0007) [2023-10-07 19:44:04,284][67871] Updated weights for policy 1, policy_version 2510 (0.0008) [2023-10-07 19:44:04,653][67871] Updated weights for policy 1, policy_version 2520 (0.0010) [2023-10-07 19:44:04,750][67838] Updated weights for policy 0, policy_version 2502 (0.0007) [2023-10-07 19:44:05,134][67838] Updated weights for policy 0, policy_version 2512 (0.0007) [2023-10-07 19:44:05,514][67838] Updated weights for policy 0, policy_version 2522 (0.0009) [2023-10-07 19:44:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 5177344. Throughput: 0: 1612.8, 1: 1621.7. Samples: 1304662. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-07 19:44:07,477][66916] Avg episode reward: [(0, '23.340'), (1, '22.280')] [2023-10-07 19:44:07,488][67676] Saving new best policy, reward=22.280! [2023-10-07 19:44:07,488][67511] Saving new best policy, reward=23.340! [2023-10-07 19:44:08,694][67871] Updated weights for policy 1, policy_version 2530 (0.0008) [2023-10-07 19:44:09,050][67871] Updated weights for policy 1, policy_version 2540 (0.0010) [2023-10-07 19:44:09,429][67871] Updated weights for policy 1, policy_version 2550 (0.0009) [2023-10-07 19:44:09,783][67838] Updated weights for policy 0, policy_version 2532 (0.0009) [2023-10-07 19:44:09,796][67871] Updated weights for policy 1, policy_version 2560 (0.0010) [2023-10-07 19:44:10,170][67838] Updated weights for policy 0, policy_version 2542 (0.0007) [2023-10-07 19:44:10,545][67838] Updated weights for policy 0, policy_version 2552 (0.0007) [2023-10-07 19:44:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 5242880. Throughput: 0: 1628.2, 1: 1623.4. Samples: 1314332. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-07 19:44:12,477][66916] Avg episode reward: [(0, '22.270'), (1, '22.470')] [2023-10-07 19:44:12,478][67676] Saving new best policy, reward=22.470! [2023-10-07 19:44:13,893][67871] Updated weights for policy 1, policy_version 2570 (0.0009) [2023-10-07 19:44:14,267][67871] Updated weights for policy 1, policy_version 2580 (0.0008) [2023-10-07 19:44:14,639][67871] Updated weights for policy 1, policy_version 2590 (0.0007) [2023-10-07 19:44:14,673][67838] Updated weights for policy 0, policy_version 2562 (0.0007) [2023-10-07 19:44:15,049][67838] Updated weights for policy 0, policy_version 2572 (0.0008) [2023-10-07 19:44:15,412][67838] Updated weights for policy 0, policy_version 2582 (0.0009) [2023-10-07 19:44:15,780][67838] Updated weights for policy 0, policy_version 2592 (0.0008) [2023-10-07 19:44:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 5308416. Throughput: 0: 1617.0, 1: 1624.7. Samples: 1333726. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-10-07 19:44:17,478][66916] Avg episode reward: [(0, '21.640'), (1, '22.920')] [2023-10-07 19:44:17,479][67676] Saving new best policy, reward=22.920! [2023-10-07 19:44:19,037][67871] Updated weights for policy 1, policy_version 2600 (0.0010) [2023-10-07 19:44:19,412][67871] Updated weights for policy 1, policy_version 2610 (0.0010) [2023-10-07 19:44:19,791][67871] Updated weights for policy 1, policy_version 2620 (0.0008) [2023-10-07 19:44:20,188][67838] Updated weights for policy 0, policy_version 2602 (0.0008) [2023-10-07 19:44:20,556][67838] Updated weights for policy 0, policy_version 2612 (0.0008) [2023-10-07 19:44:20,933][67838] Updated weights for policy 0, policy_version 2622 (0.0009) [2023-10-07 19:44:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 12885.0). Total num frames: 5373952. Throughput: 0: 1615.1, 1: 1623.6. Samples: 1353284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:44:22,478][66916] Avg episode reward: [(0, '22.380'), (1, '21.970')] [2023-10-07 19:44:24,016][67871] Updated weights for policy 1, policy_version 2630 (0.0009) [2023-10-07 19:44:24,400][67871] Updated weights for policy 1, policy_version 2640 (0.0010) [2023-10-07 19:44:24,769][67871] Updated weights for policy 1, policy_version 2650 (0.0009) [2023-10-07 19:44:25,046][67838] Updated weights for policy 0, policy_version 2632 (0.0008) [2023-10-07 19:44:25,418][67838] Updated weights for policy 0, policy_version 2642 (0.0010) [2023-10-07 19:44:25,793][67838] Updated weights for policy 0, policy_version 2652 (0.0007) [2023-10-07 19:44:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 5439488. Throughput: 0: 1631.1, 1: 1626.2. Samples: 1363248. Policy #0 lag: (min: 31.0, avg: 32.2, max: 56.0) [2023-10-07 19:44:27,477][66916] Avg episode reward: [(0, '23.020'), (1, '21.870')] [2023-10-07 19:44:29,061][67871] Updated weights for policy 1, policy_version 2660 (0.0008) [2023-10-07 19:44:29,427][67871] Updated weights for policy 1, policy_version 2670 (0.0010) [2023-10-07 19:44:29,799][67871] Updated weights for policy 1, policy_version 2680 (0.0008) [2023-10-07 19:44:29,937][67838] Updated weights for policy 0, policy_version 2662 (0.0007) [2023-10-07 19:44:30,304][67838] Updated weights for policy 0, policy_version 2672 (0.0007) [2023-10-07 19:44:30,676][67838] Updated weights for policy 0, policy_version 2682 (0.0010) [2023-10-07 19:44:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 5505024. Throughput: 0: 1624.8, 1: 1624.8. Samples: 1382298. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-07 19:44:32,477][66916] Avg episode reward: [(0, '22.450'), (1, '21.800')] [2023-10-07 19:44:34,050][67871] Updated weights for policy 1, policy_version 2690 (0.0007) [2023-10-07 19:44:34,422][67871] Updated weights for policy 1, policy_version 2700 (0.0008) [2023-10-07 19:44:34,792][67871] Updated weights for policy 1, policy_version 2710 (0.0008) [2023-10-07 19:44:34,978][67838] Updated weights for policy 0, policy_version 2692 (0.0008) [2023-10-07 19:44:35,156][67871] Updated weights for policy 1, policy_version 2720 (0.0009) [2023-10-07 19:44:35,356][67838] Updated weights for policy 0, policy_version 2702 (0.0008) [2023-10-07 19:44:35,737][67838] Updated weights for policy 0, policy_version 2712 (0.0009) [2023-10-07 19:44:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 5570560. Throughput: 0: 1622.8, 1: 1623.3. Samples: 1402134. Policy #0 lag: (min: 14.0, avg: 16.5, max: 46.0) [2023-10-07 19:44:37,478][66916] Avg episode reward: [(0, '22.690'), (1, '22.390')] [2023-10-07 19:44:39,420][67871] Updated weights for policy 1, policy_version 2730 (0.0009) [2023-10-07 19:44:39,795][67871] Updated weights for policy 1, policy_version 2740 (0.0008) [2023-10-07 19:44:39,816][67838] Updated weights for policy 0, policy_version 2722 (0.0009) [2023-10-07 19:44:40,161][67871] Updated weights for policy 1, policy_version 2750 (0.0008) [2023-10-07 19:44:40,182][67838] Updated weights for policy 0, policy_version 2732 (0.0008) [2023-10-07 19:44:40,566][67838] Updated weights for policy 0, policy_version 2742 (0.0011) [2023-10-07 19:44:40,931][67838] Updated weights for policy 0, policy_version 2752 (0.0010) [2023-10-07 19:44:42,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 12885.1). Total num frames: 5636096. Throughput: 0: 1634.7, 1: 1627.4. Samples: 1412148. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 19:44:42,477][66916] Avg episode reward: [(0, '21.690'), (1, '22.530')] [2023-10-07 19:44:44,475][67871] Updated weights for policy 1, policy_version 2760 (0.0009) [2023-10-07 19:44:44,853][67871] Updated weights for policy 1, policy_version 2770 (0.0008) [2023-10-07 19:44:45,216][67871] Updated weights for policy 1, policy_version 2780 (0.0007) [2023-10-07 19:44:45,257][67838] Updated weights for policy 0, policy_version 2762 (0.0007) [2023-10-07 19:44:45,619][67838] Updated weights for policy 0, policy_version 2772 (0.0010) [2023-10-07 19:44:46,005][67838] Updated weights for policy 0, policy_version 2782 (0.0008) [2023-10-07 19:44:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 5701632. Throughput: 0: 1628.2, 1: 1610.8. Samples: 1430528. Policy #0 lag: (min: 25.0, avg: 37.8, max: 57.0) [2023-10-07 19:44:47,477][66916] Avg episode reward: [(0, '22.440'), (1, '23.530')] [2023-10-07 19:44:47,478][67676] Saving new best policy, reward=23.530! [2023-10-07 19:44:49,449][67871] Updated weights for policy 1, policy_version 2790 (0.0009) [2023-10-07 19:44:49,819][67871] Updated weights for policy 1, policy_version 2800 (0.0010) [2023-10-07 19:44:50,192][67871] Updated weights for policy 1, policy_version 2810 (0.0009) [2023-10-07 19:44:50,321][67838] Updated weights for policy 0, policy_version 2792 (0.0009) [2023-10-07 19:44:50,697][67838] Updated weights for policy 0, policy_version 2802 (0.0009) [2023-10-07 19:44:51,070][67838] Updated weights for policy 0, policy_version 2812 (0.0008) [2023-10-07 19:44:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12885.1). Total num frames: 5767168. Throughput: 0: 1626.0, 1: 1608.8. Samples: 1450232. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-07 19:44:52,477][66916] Avg episode reward: [(0, '21.790'), (1, '22.370')] [2023-10-07 19:44:54,317][67871] Updated weights for policy 1, policy_version 2820 (0.0007) [2023-10-07 19:44:54,682][67871] Updated weights for policy 1, policy_version 2830 (0.0009) [2023-10-07 19:44:55,053][67871] Updated weights for policy 1, policy_version 2840 (0.0007) [2023-10-07 19:44:55,095][67838] Updated weights for policy 0, policy_version 2822 (0.0009) [2023-10-07 19:44:55,469][67838] Updated weights for policy 0, policy_version 2832 (0.0009) [2023-10-07 19:44:55,837][67838] Updated weights for policy 0, policy_version 2842 (0.0010) [2023-10-07 19:44:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 5832704. Throughput: 0: 1631.3, 1: 1624.3. Samples: 1460836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:44:57,478][66916] Avg episode reward: [(0, '23.110'), (1, '23.330')] [2023-10-07 19:44:59,246][67871] Updated weights for policy 1, policy_version 2850 (0.0007) [2023-10-07 19:44:59,613][67871] Updated weights for policy 1, policy_version 2860 (0.0009) [2023-10-07 19:44:59,981][67871] Updated weights for policy 1, policy_version 2870 (0.0008) [2023-10-07 19:44:59,993][67838] Updated weights for policy 0, policy_version 2852 (0.0008) [2023-10-07 19:45:00,352][67871] Updated weights for policy 1, policy_version 2880 (0.0008) [2023-10-07 19:45:00,367][67838] Updated weights for policy 0, policy_version 2862 (0.0009) [2023-10-07 19:45:00,745][67838] Updated weights for policy 0, policy_version 2872 (0.0011) [2023-10-07 19:45:02,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12885.1). Total num frames: 5898240. Throughput: 0: 1624.4, 1: 1606.3. Samples: 1479110. Policy #0 lag: (min: 16.0, avg: 41.2, max: 48.0) [2023-10-07 19:45:02,477][66916] Avg episode reward: [(0, '21.890'), (1, '21.270')] [2023-10-07 19:45:04,781][67871] Updated weights for policy 1, policy_version 2890 (0.0011) [2023-10-07 19:45:05,097][67838] Updated weights for policy 0, policy_version 2882 (0.0009) [2023-10-07 19:45:05,145][67871] Updated weights for policy 1, policy_version 2900 (0.0009) [2023-10-07 19:45:05,461][67838] Updated weights for policy 0, policy_version 2892 (0.0009) [2023-10-07 19:45:05,511][67871] Updated weights for policy 1, policy_version 2910 (0.0010) [2023-10-07 19:45:05,848][67838] Updated weights for policy 0, policy_version 2902 (0.0010) [2023-10-07 19:45:06,214][67838] Updated weights for policy 0, policy_version 2912 (0.0010) [2023-10-07 19:45:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 5963776. Throughput: 0: 1629.6, 1: 1604.9. Samples: 1498834. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-10-07 19:45:07,478][66916] Avg episode reward: [(0, '23.850'), (1, '21.940')] [2023-10-07 19:45:07,486][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000002912_2981888.pth... [2023-10-07 19:45:07,486][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000002912_2981888.pth... [2023-10-07 19:45:07,516][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000001408_1441792.pth [2023-10-07 19:45:07,526][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000001408_1441792.pth [2023-10-07 19:45:07,531][67511] Saving new best policy, reward=23.850! [2023-10-07 19:45:09,802][67871] Updated weights for policy 1, policy_version 2920 (0.0010) [2023-10-07 19:45:10,175][67871] Updated weights for policy 1, policy_version 2930 (0.0009) [2023-10-07 19:45:10,376][67838] Updated weights for policy 0, policy_version 2922 (0.0009) [2023-10-07 19:45:10,536][67871] Updated weights for policy 1, policy_version 2940 (0.0008) [2023-10-07 19:45:10,759][67838] Updated weights for policy 0, policy_version 2932 (0.0008) [2023-10-07 19:45:11,127][67838] Updated weights for policy 0, policy_version 2942 (0.0008) [2023-10-07 19:45:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 6029312. Throughput: 0: 1631.7, 1: 1622.6. Samples: 1509690. Policy #0 lag: (min: 3.0, avg: 9.8, max: 35.0) [2023-10-07 19:45:12,477][66916] Avg episode reward: [(0, '23.880'), (1, '21.870')] [2023-10-07 19:45:12,478][67511] Saving new best policy, reward=23.880! [2023-10-07 19:45:14,888][67871] Updated weights for policy 1, policy_version 2950 (0.0007) [2023-10-07 19:45:15,246][67871] Updated weights for policy 1, policy_version 2960 (0.0008) [2023-10-07 19:45:15,411][67838] Updated weights for policy 0, policy_version 2952 (0.0008) [2023-10-07 19:45:15,611][67871] Updated weights for policy 1, policy_version 2970 (0.0010) [2023-10-07 19:45:15,774][67838] Updated weights for policy 0, policy_version 2962 (0.0010) [2023-10-07 19:45:16,159][67838] Updated weights for policy 0, policy_version 2972 (0.0010) [2023-10-07 19:45:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 6094848. Throughput: 0: 1623.6, 1: 1604.7. Samples: 1527574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:45:17,478][66916] Avg episode reward: [(0, '22.950'), (1, '22.300')] [2023-10-07 19:45:19,846][67871] Updated weights for policy 1, policy_version 2980 (0.0007) [2023-10-07 19:45:20,217][67871] Updated weights for policy 1, policy_version 2990 (0.0009) [2023-10-07 19:45:20,483][67838] Updated weights for policy 0, policy_version 2982 (0.0009) [2023-10-07 19:45:20,588][67871] Updated weights for policy 1, policy_version 3000 (0.0009) [2023-10-07 19:45:20,848][67838] Updated weights for policy 0, policy_version 2992 (0.0010) [2023-10-07 19:45:21,220][67838] Updated weights for policy 0, policy_version 3002 (0.0010) [2023-10-07 19:45:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 6160384. Throughput: 0: 1618.2, 1: 1606.4. Samples: 1547242. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-07 19:45:22,477][66916] Avg episode reward: [(0, '24.080'), (1, '21.240')] [2023-10-07 19:45:22,486][67511] Saving new best policy, reward=24.080! [2023-10-07 19:45:24,901][67871] Updated weights for policy 1, policy_version 3010 (0.0007) [2023-10-07 19:45:25,270][67871] Updated weights for policy 1, policy_version 3020 (0.0007) [2023-10-07 19:45:25,409][67838] Updated weights for policy 0, policy_version 3012 (0.0009) [2023-10-07 19:45:25,636][67871] Updated weights for policy 1, policy_version 3030 (0.0008) [2023-10-07 19:45:25,789][67838] Updated weights for policy 0, policy_version 3022 (0.0008) [2023-10-07 19:45:25,996][67871] Updated weights for policy 1, policy_version 3040 (0.0007) [2023-10-07 19:45:26,156][67838] Updated weights for policy 0, policy_version 3032 (0.0011) [2023-10-07 19:45:27,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 12885.1). Total num frames: 6225920. Throughput: 0: 1620.8, 1: 1626.5. Samples: 1558278. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 19:45:27,477][66916] Avg episode reward: [(0, '22.410'), (1, '22.010')] [2023-10-07 19:45:30,061][67871] Updated weights for policy 1, policy_version 3050 (0.0007) [2023-10-07 19:45:30,294][67838] Updated weights for policy 0, policy_version 3042 (0.0009) [2023-10-07 19:45:30,430][67871] Updated weights for policy 1, policy_version 3060 (0.0009) [2023-10-07 19:45:30,666][67838] Updated weights for policy 0, policy_version 3052 (0.0008) [2023-10-07 19:45:30,801][67871] Updated weights for policy 1, policy_version 3070 (0.0010) [2023-10-07 19:45:31,046][67838] Updated weights for policy 0, policy_version 3062 (0.0008) [2023-10-07 19:45:31,430][67838] Updated weights for policy 0, policy_version 3072 (0.0008) [2023-10-07 19:45:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 6291456. Throughput: 0: 1627.6, 1: 1621.3. Samples: 1576728. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-07 19:45:32,478][66916] Avg episode reward: [(0, '24.040'), (1, '21.390')] [2023-10-07 19:45:35,053][67871] Updated weights for policy 1, policy_version 3080 (0.0010) [2023-10-07 19:45:35,431][67871] Updated weights for policy 1, policy_version 3090 (0.0009) [2023-10-07 19:45:35,678][67838] Updated weights for policy 0, policy_version 3082 (0.0008) [2023-10-07 19:45:35,798][67871] Updated weights for policy 1, policy_version 3100 (0.0010) [2023-10-07 19:45:36,054][67838] Updated weights for policy 0, policy_version 3092 (0.0010) [2023-10-07 19:45:36,423][67838] Updated weights for policy 0, policy_version 3102 (0.0009) [2023-10-07 19:45:37,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 6356992. Throughput: 0: 1618.4, 1: 1619.8. Samples: 1595954. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 19:45:37,477][66916] Avg episode reward: [(0, '23.180'), (1, '22.250')] [2023-10-07 19:45:39,978][67871] Updated weights for policy 1, policy_version 3110 (0.0008) [2023-10-07 19:45:40,350][67871] Updated weights for policy 1, policy_version 3120 (0.0009) [2023-10-07 19:45:40,546][67838] Updated weights for policy 0, policy_version 3112 (0.0008) [2023-10-07 19:45:40,716][67871] Updated weights for policy 1, policy_version 3130 (0.0007) [2023-10-07 19:45:40,912][67838] Updated weights for policy 0, policy_version 3122 (0.0008) [2023-10-07 19:45:41,300][67838] Updated weights for policy 0, policy_version 3132 (0.0009) [2023-10-07 19:45:42,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 6422528. Throughput: 0: 1624.2, 1: 1627.4. Samples: 1607158. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 19:45:42,478][66916] Avg episode reward: [(0, '23.870'), (1, '22.520')] [2023-10-07 19:45:44,966][67871] Updated weights for policy 1, policy_version 3140 (0.0007) [2023-10-07 19:45:45,337][67871] Updated weights for policy 1, policy_version 3150 (0.0009) [2023-10-07 19:45:45,531][67838] Updated weights for policy 0, policy_version 3142 (0.0009) [2023-10-07 19:45:45,711][67871] Updated weights for policy 1, policy_version 3160 (0.0009) [2023-10-07 19:45:45,910][67838] Updated weights for policy 0, policy_version 3152 (0.0009) [2023-10-07 19:45:46,282][67838] Updated weights for policy 0, policy_version 3162 (0.0007) [2023-10-07 19:45:47,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 6488064. Throughput: 0: 1634.4, 1: 1618.0. Samples: 1625466. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-07 19:45:47,478][66916] Avg episode reward: [(0, '22.030'), (1, '22.720')] [2023-10-07 19:45:50,065][67871] Updated weights for policy 1, policy_version 3170 (0.0008) [2023-10-07 19:45:50,448][67871] Updated weights for policy 1, policy_version 3180 (0.0008) [2023-10-07 19:45:50,591][67838] Updated weights for policy 0, policy_version 3172 (0.0010) [2023-10-07 19:45:50,811][67871] Updated weights for policy 1, policy_version 3190 (0.0009) [2023-10-07 19:45:50,960][67838] Updated weights for policy 0, policy_version 3182 (0.0008) [2023-10-07 19:45:51,176][67871] Updated weights for policy 1, policy_version 3200 (0.0008) [2023-10-07 19:45:51,332][67838] Updated weights for policy 0, policy_version 3192 (0.0008) [2023-10-07 19:45:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 6553600. Throughput: 0: 1620.9, 1: 1614.5. Samples: 1644426. Policy #0 lag: (min: 29.0, avg: 31.5, max: 61.0) [2023-10-07 19:45:52,477][66916] Avg episode reward: [(0, '23.800'), (1, '22.580')] [2023-10-07 19:45:55,383][67871] Updated weights for policy 1, policy_version 3210 (0.0008) [2023-10-07 19:45:55,491][67838] Updated weights for policy 0, policy_version 3202 (0.0010) [2023-10-07 19:45:55,764][67871] Updated weights for policy 1, policy_version 3220 (0.0009) [2023-10-07 19:45:55,897][67838] Updated weights for policy 0, policy_version 3212 (0.0008) [2023-10-07 19:45:56,139][67871] Updated weights for policy 1, policy_version 3230 (0.0009) [2023-10-07 19:45:56,268][67838] Updated weights for policy 0, policy_version 3222 (0.0008) [2023-10-07 19:45:56,652][67838] Updated weights for policy 0, policy_version 3232 (0.0009) [2023-10-07 19:45:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 6619136. Throughput: 0: 1626.9, 1: 1622.3. Samples: 1655902. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-07 19:45:57,477][66916] Avg episode reward: [(0, '23.850'), (1, '22.500')] [2023-10-07 19:46:00,433][67871] Updated weights for policy 1, policy_version 3240 (0.0010) [2023-10-07 19:46:00,800][67871] Updated weights for policy 1, policy_version 3250 (0.0007) [2023-10-07 19:46:00,863][67838] Updated weights for policy 0, policy_version 3242 (0.0009) [2023-10-07 19:46:01,171][67871] Updated weights for policy 1, policy_version 3260 (0.0010) [2023-10-07 19:46:01,222][67838] Updated weights for policy 0, policy_version 3252 (0.0009) [2023-10-07 19:46:01,601][67838] Updated weights for policy 0, policy_version 3262 (0.0008) [2023-10-07 19:46:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 6684672. Throughput: 0: 1634.6, 1: 1632.6. Samples: 1674598. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-07 19:46:02,478][66916] Avg episode reward: [(0, '24.760'), (1, '22.380')] [2023-10-07 19:46:02,479][67511] Saving new best policy, reward=24.760! [2023-10-07 19:46:05,427][67871] Updated weights for policy 1, policy_version 3270 (0.0009) [2023-10-07 19:46:05,666][67838] Updated weights for policy 0, policy_version 3272 (0.0010) [2023-10-07 19:46:05,797][67871] Updated weights for policy 1, policy_version 3280 (0.0008) [2023-10-07 19:46:06,041][67838] Updated weights for policy 0, policy_version 3282 (0.0008) [2023-10-07 19:46:06,155][67871] Updated weights for policy 1, policy_version 3290 (0.0008) [2023-10-07 19:46:06,415][67838] Updated weights for policy 0, policy_version 3292 (0.0007) [2023-10-07 19:46:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 6750208. Throughput: 0: 1625.3, 1: 1621.8. Samples: 1693360. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-07 19:46:07,478][66916] Avg episode reward: [(0, '23.210'), (1, '22.470')] [2023-10-07 19:46:10,323][67871] Updated weights for policy 1, policy_version 3300 (0.0007) [2023-10-07 19:46:10,687][67871] Updated weights for policy 1, policy_version 3310 (0.0007) [2023-10-07 19:46:10,709][67838] Updated weights for policy 0, policy_version 3302 (0.0008) [2023-10-07 19:46:11,055][67871] Updated weights for policy 1, policy_version 3320 (0.0007) [2023-10-07 19:46:11,073][67838] Updated weights for policy 0, policy_version 3312 (0.0007) [2023-10-07 19:46:11,451][67838] Updated weights for policy 0, policy_version 3322 (0.0007) [2023-10-07 19:46:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 6815744. Throughput: 0: 1626.3, 1: 1624.7. Samples: 1704574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:46:12,478][66916] Avg episode reward: [(0, '24.250'), (1, '23.070')] [2023-10-07 19:46:15,317][67871] Updated weights for policy 1, policy_version 3330 (0.0008) [2023-10-07 19:46:15,686][67871] Updated weights for policy 1, policy_version 3340 (0.0009) [2023-10-07 19:46:15,775][67838] Updated weights for policy 0, policy_version 3332 (0.0008) [2023-10-07 19:46:16,050][67871] Updated weights for policy 1, policy_version 3350 (0.0009) [2023-10-07 19:46:16,158][67838] Updated weights for policy 0, policy_version 3342 (0.0009) [2023-10-07 19:46:16,420][67871] Updated weights for policy 1, policy_version 3360 (0.0008) [2023-10-07 19:46:16,526][67838] Updated weights for policy 0, policy_version 3352 (0.0007) [2023-10-07 19:46:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 6881280. Throughput: 0: 1631.7, 1: 1632.7. Samples: 1723626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:46:17,478][66916] Avg episode reward: [(0, '23.620'), (1, '23.710')] [2023-10-07 19:46:17,479][67676] Saving new best policy, reward=23.710! [2023-10-07 19:46:20,573][67871] Updated weights for policy 1, policy_version 3370 (0.0008) [2023-10-07 19:46:20,788][67838] Updated weights for policy 0, policy_version 3362 (0.0009) [2023-10-07 19:46:20,945][67871] Updated weights for policy 1, policy_version 3380 (0.0009) [2023-10-07 19:46:21,160][67838] Updated weights for policy 0, policy_version 3372 (0.0007) [2023-10-07 19:46:21,317][67871] Updated weights for policy 1, policy_version 3390 (0.0008) [2023-10-07 19:46:21,543][67838] Updated weights for policy 0, policy_version 3382 (0.0008) [2023-10-07 19:46:21,917][67838] Updated weights for policy 0, policy_version 3392 (0.0009) [2023-10-07 19:46:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 6946816. Throughput: 0: 1625.2, 1: 1628.3. Samples: 1742364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:46:22,477][66916] Avg episode reward: [(0, '24.690'), (1, '23.950')] [2023-10-07 19:46:22,486][67676] Saving new best policy, reward=23.950! [2023-10-07 19:46:25,532][67871] Updated weights for policy 1, policy_version 3400 (0.0008) [2023-10-07 19:46:25,909][67871] Updated weights for policy 1, policy_version 3410 (0.0007) [2023-10-07 19:46:26,117][67838] Updated weights for policy 0, policy_version 3402 (0.0007) [2023-10-07 19:46:26,272][67871] Updated weights for policy 1, policy_version 3420 (0.0007) [2023-10-07 19:46:26,488][67838] Updated weights for policy 0, policy_version 3412 (0.0008) [2023-10-07 19:46:26,866][67838] Updated weights for policy 0, policy_version 3422 (0.0008) [2023-10-07 19:46:27,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 7012352. Throughput: 0: 1622.8, 1: 1628.9. Samples: 1753482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:46:27,478][66916] Avg episode reward: [(0, '23.760'), (1, '24.480')] [2023-10-07 19:46:27,479][67676] Saving new best policy, reward=24.480! [2023-10-07 19:46:30,457][67871] Updated weights for policy 1, policy_version 3430 (0.0008) [2023-10-07 19:46:30,822][67871] Updated weights for policy 1, policy_version 3440 (0.0008) [2023-10-07 19:46:31,186][67838] Updated weights for policy 0, policy_version 3432 (0.0007) [2023-10-07 19:46:31,186][67871] Updated weights for policy 1, policy_version 3450 (0.0008) [2023-10-07 19:46:31,550][67838] Updated weights for policy 0, policy_version 3442 (0.0007) [2023-10-07 19:46:31,930][67838] Updated weights for policy 0, policy_version 3452 (0.0009) [2023-10-07 19:46:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7077888. Throughput: 0: 1633.1, 1: 1641.7. Samples: 1772830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:46:32,477][66916] Avg episode reward: [(0, '23.760'), (1, '23.990')] [2023-10-07 19:46:35,447][67871] Updated weights for policy 1, policy_version 3460 (0.0007) [2023-10-07 19:46:35,852][67871] Updated weights for policy 1, policy_version 3470 (0.0007) [2023-10-07 19:46:36,047][67838] Updated weights for policy 0, policy_version 3462 (0.0008) [2023-10-07 19:46:36,215][67871] Updated weights for policy 1, policy_version 3480 (0.0009) [2023-10-07 19:46:36,427][67838] Updated weights for policy 0, policy_version 3472 (0.0009) [2023-10-07 19:46:36,794][67838] Updated weights for policy 0, policy_version 3482 (0.0007) [2023-10-07 19:46:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7143424. Throughput: 0: 1619.9, 1: 1637.6. Samples: 1791016. Policy #0 lag: (min: 7.0, avg: 14.3, max: 39.0) [2023-10-07 19:46:37,478][66916] Avg episode reward: [(0, '24.180'), (1, '24.400')] [2023-10-07 19:46:40,273][67871] Updated weights for policy 1, policy_version 3490 (0.0009) [2023-10-07 19:46:40,649][67871] Updated weights for policy 1, policy_version 3500 (0.0008) [2023-10-07 19:46:41,011][67871] Updated weights for policy 1, policy_version 3510 (0.0009) [2023-10-07 19:46:41,068][67838] Updated weights for policy 0, policy_version 3492 (0.0008) [2023-10-07 19:46:41,376][67871] Updated weights for policy 1, policy_version 3520 (0.0009) [2023-10-07 19:46:41,453][67838] Updated weights for policy 0, policy_version 3502 (0.0008) [2023-10-07 19:46:41,822][67838] Updated weights for policy 0, policy_version 3512 (0.0007) [2023-10-07 19:46:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7208960. Throughput: 0: 1614.0, 1: 1638.6. Samples: 1802268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:46:42,478][66916] Avg episode reward: [(0, '24.690'), (1, '23.220')] [2023-10-07 19:46:45,625][67871] Updated weights for policy 1, policy_version 3530 (0.0009) [2023-10-07 19:46:45,999][67871] Updated weights for policy 1, policy_version 3540 (0.0008) [2023-10-07 19:46:46,087][67838] Updated weights for policy 0, policy_version 3522 (0.0008) [2023-10-07 19:46:46,368][67871] Updated weights for policy 1, policy_version 3550 (0.0009) [2023-10-07 19:46:46,462][67838] Updated weights for policy 0, policy_version 3532 (0.0008) [2023-10-07 19:46:46,838][67838] Updated weights for policy 0, policy_version 3542 (0.0008) [2023-10-07 19:46:47,209][67838] Updated weights for policy 0, policy_version 3552 (0.0008) [2023-10-07 19:46:47,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7274496. Throughput: 0: 1626.6, 1: 1637.9. Samples: 1821498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:46:47,478][66916] Avg episode reward: [(0, '23.850'), (1, '22.700')] [2023-10-07 19:46:50,616][67871] Updated weights for policy 1, policy_version 3560 (0.0008) [2023-10-07 19:46:50,982][67871] Updated weights for policy 1, policy_version 3570 (0.0007) [2023-10-07 19:46:51,350][67871] Updated weights for policy 1, policy_version 3580 (0.0009) [2023-10-07 19:46:51,495][67838] Updated weights for policy 0, policy_version 3562 (0.0008) [2023-10-07 19:46:51,873][67838] Updated weights for policy 0, policy_version 3572 (0.0010) [2023-10-07 19:46:52,253][67838] Updated weights for policy 0, policy_version 3582 (0.0010) [2023-10-07 19:46:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7340032. Throughput: 0: 1622.2, 1: 1630.6. Samples: 1839736. Policy #0 lag: (min: 31.0, avg: 39.6, max: 63.0) [2023-10-07 19:46:52,477][66916] Avg episode reward: [(0, '24.150'), (1, '22.750')] [2023-10-07 19:46:55,481][67871] Updated weights for policy 1, policy_version 3590 (0.0008) [2023-10-07 19:46:55,861][67871] Updated weights for policy 1, policy_version 3600 (0.0008) [2023-10-07 19:46:56,227][67871] Updated weights for policy 1, policy_version 3610 (0.0009) [2023-10-07 19:46:56,403][67838] Updated weights for policy 0, policy_version 3592 (0.0008) [2023-10-07 19:46:56,774][67838] Updated weights for policy 0, policy_version 3602 (0.0007) [2023-10-07 19:46:57,145][67838] Updated weights for policy 0, policy_version 3612 (0.0007) [2023-10-07 19:46:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7405568. Throughput: 0: 1618.2, 1: 1632.3. Samples: 1850844. Policy #0 lag: (min: 31.0, avg: 39.6, max: 63.0) [2023-10-07 19:46:57,477][66916] Avg episode reward: [(0, '24.810'), (1, '23.560')] [2023-10-07 19:46:57,478][67511] Saving new best policy, reward=24.810! [2023-10-07 19:47:00,333][67871] Updated weights for policy 1, policy_version 3620 (0.0007) [2023-10-07 19:47:00,703][67871] Updated weights for policy 1, policy_version 3630 (0.0008) [2023-10-07 19:47:01,075][67871] Updated weights for policy 1, policy_version 3640 (0.0009) [2023-10-07 19:47:01,431][67838] Updated weights for policy 0, policy_version 3622 (0.0010) [2023-10-07 19:47:01,811][67838] Updated weights for policy 0, policy_version 3632 (0.0009) [2023-10-07 19:47:02,181][67838] Updated weights for policy 0, policy_version 3642 (0.0009) [2023-10-07 19:47:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7471104. Throughput: 0: 1625.0, 1: 1629.9. Samples: 1870096. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-10-07 19:47:02,478][66916] Avg episode reward: [(0, '24.410'), (1, '23.500')] [2023-10-07 19:47:05,180][67871] Updated weights for policy 1, policy_version 3650 (0.0009) [2023-10-07 19:47:05,539][67871] Updated weights for policy 1, policy_version 3660 (0.0008) [2023-10-07 19:47:05,904][67871] Updated weights for policy 1, policy_version 3670 (0.0007) [2023-10-07 19:47:06,250][67838] Updated weights for policy 0, policy_version 3652 (0.0007) [2023-10-07 19:47:06,264][67871] Updated weights for policy 1, policy_version 3680 (0.0008) [2023-10-07 19:47:06,627][67838] Updated weights for policy 0, policy_version 3662 (0.0009) [2023-10-07 19:47:07,022][67838] Updated weights for policy 0, policy_version 3672 (0.0009) [2023-10-07 19:47:07,477][66916] Fps is (10 sec: 13106.6, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 7536640. Throughput: 0: 1625.9, 1: 1632.7. Samples: 1889004. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-10-07 19:47:07,478][66916] Avg episode reward: [(0, '24.590'), (1, '24.250')] [2023-10-07 19:47:07,490][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000003680_3768320.pth... [2023-10-07 19:47:07,491][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000003680_3768320.pth... [2023-10-07 19:47:07,530][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000002144_2195456.pth [2023-10-07 19:47:07,531][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000002144_2195456.pth [2023-10-07 19:47:10,446][67871] Updated weights for policy 1, policy_version 3690 (0.0008) [2023-10-07 19:47:10,815][67871] Updated weights for policy 1, policy_version 3700 (0.0007) [2023-10-07 19:47:11,179][67871] Updated weights for policy 1, policy_version 3710 (0.0008) [2023-10-07 19:47:11,190][67838] Updated weights for policy 0, policy_version 3682 (0.0008) [2023-10-07 19:47:11,563][67838] Updated weights for policy 0, policy_version 3692 (0.0007) [2023-10-07 19:47:11,944][67838] Updated weights for policy 0, policy_version 3702 (0.0008) [2023-10-07 19:47:12,311][67838] Updated weights for policy 0, policy_version 3712 (0.0009) [2023-10-07 19:47:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7602176. Throughput: 0: 1620.0, 1: 1638.6. Samples: 1900122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:47:12,478][66916] Avg episode reward: [(0, '24.350'), (1, '23.900')] [2023-10-07 19:47:15,351][67871] Updated weights for policy 1, policy_version 3720 (0.0010) [2023-10-07 19:47:15,720][67871] Updated weights for policy 1, policy_version 3730 (0.0008) [2023-10-07 19:47:16,087][67871] Updated weights for policy 1, policy_version 3740 (0.0008) [2023-10-07 19:47:16,400][67838] Updated weights for policy 0, policy_version 3722 (0.0009) [2023-10-07 19:47:16,764][67838] Updated weights for policy 0, policy_version 3732 (0.0008) [2023-10-07 19:47:17,135][67838] Updated weights for policy 0, policy_version 3742 (0.0007) [2023-10-07 19:47:17,477][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7667712. Throughput: 0: 1623.8, 1: 1632.3. Samples: 1919356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:47:17,478][66916] Avg episode reward: [(0, '24.880'), (1, '23.700')] [2023-10-07 19:47:17,482][67511] Saving new best policy, reward=24.880! [2023-10-07 19:47:20,497][67871] Updated weights for policy 1, policy_version 3750 (0.0009) [2023-10-07 19:47:20,869][67871] Updated weights for policy 1, policy_version 3760 (0.0009) [2023-10-07 19:47:21,181][67838] Updated weights for policy 0, policy_version 3752 (0.0009) [2023-10-07 19:47:21,247][67871] Updated weights for policy 1, policy_version 3770 (0.0009) [2023-10-07 19:47:21,550][67838] Updated weights for policy 0, policy_version 3762 (0.0009) [2023-10-07 19:47:21,935][67838] Updated weights for policy 0, policy_version 3772 (0.0009) [2023-10-07 19:47:22,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7733248. Throughput: 0: 1632.1, 1: 1632.6. Samples: 1937930. Policy #0 lag: (min: 16.0, avg: 40.2, max: 48.0) [2023-10-07 19:47:22,478][66916] Avg episode reward: [(0, '25.010'), (1, '23.070')] [2023-10-07 19:47:22,488][67511] Saving new best policy, reward=25.010! [2023-10-07 19:47:25,554][67871] Updated weights for policy 1, policy_version 3780 (0.0009) [2023-10-07 19:47:25,910][67871] Updated weights for policy 1, policy_version 3790 (0.0008) [2023-10-07 19:47:26,284][67871] Updated weights for policy 1, policy_version 3800 (0.0008) [2023-10-07 19:47:26,382][67838] Updated weights for policy 0, policy_version 3782 (0.0009) [2023-10-07 19:47:26,764][67838] Updated weights for policy 0, policy_version 3792 (0.0009) [2023-10-07 19:47:27,144][67838] Updated weights for policy 0, policy_version 3802 (0.0008) [2023-10-07 19:47:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7798784. Throughput: 0: 1629.8, 1: 1630.4. Samples: 1948974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:47:27,477][66916] Avg episode reward: [(0, '24.870'), (1, '23.230')] [2023-10-07 19:47:30,509][67871] Updated weights for policy 1, policy_version 3810 (0.0007) [2023-10-07 19:47:30,888][67871] Updated weights for policy 1, policy_version 3820 (0.0007) [2023-10-07 19:47:31,263][67871] Updated weights for policy 1, policy_version 3830 (0.0008) [2023-10-07 19:47:31,267][67838] Updated weights for policy 0, policy_version 3812 (0.0008) [2023-10-07 19:47:31,624][67871] Updated weights for policy 1, policy_version 3840 (0.0008) [2023-10-07 19:47:31,643][67838] Updated weights for policy 0, policy_version 3822 (0.0010) [2023-10-07 19:47:32,018][67838] Updated weights for policy 0, policy_version 3832 (0.0009) [2023-10-07 19:47:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7864320. Throughput: 0: 1630.7, 1: 1634.2. Samples: 1968418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:47:32,477][66916] Avg episode reward: [(0, '24.570'), (1, '25.210')] [2023-10-07 19:47:32,478][67676] Saving new best policy, reward=25.210! [2023-10-07 19:47:35,726][67871] Updated weights for policy 1, policy_version 3850 (0.0009) [2023-10-07 19:47:36,088][67871] Updated weights for policy 1, policy_version 3860 (0.0009) [2023-10-07 19:47:36,302][67838] Updated weights for policy 0, policy_version 3842 (0.0008) [2023-10-07 19:47:36,456][67871] Updated weights for policy 1, policy_version 3870 (0.0008) [2023-10-07 19:47:36,678][67838] Updated weights for policy 0, policy_version 3852 (0.0009) [2023-10-07 19:47:37,051][67838] Updated weights for policy 0, policy_version 3862 (0.0008) [2023-10-07 19:47:37,426][67838] Updated weights for policy 0, policy_version 3872 (0.0009) [2023-10-07 19:47:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7929856. Throughput: 0: 1632.2, 1: 1638.3. Samples: 1986908. Policy #0 lag: (min: 17.0, avg: 21.1, max: 49.0) [2023-10-07 19:47:37,478][66916] Avg episode reward: [(0, '23.510'), (1, '23.810')] [2023-10-07 19:47:40,588][67871] Updated weights for policy 1, policy_version 3880 (0.0009) [2023-10-07 19:47:40,956][67871] Updated weights for policy 1, policy_version 3890 (0.0007) [2023-10-07 19:47:41,318][67871] Updated weights for policy 1, policy_version 3900 (0.0008) [2023-10-07 19:47:41,589][67838] Updated weights for policy 0, policy_version 3882 (0.0007) [2023-10-07 19:47:41,964][67838] Updated weights for policy 0, policy_version 3892 (0.0008) [2023-10-07 19:47:42,332][67838] Updated weights for policy 0, policy_version 3902 (0.0009) [2023-10-07 19:47:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7995392. Throughput: 0: 1629.7, 1: 1634.6. Samples: 1997740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:47:42,477][66916] Avg episode reward: [(0, '25.350'), (1, '23.280')] [2023-10-07 19:47:42,479][67511] Saving new best policy, reward=25.350! [2023-10-07 19:47:45,548][67871] Updated weights for policy 1, policy_version 3910 (0.0010) [2023-10-07 19:47:45,917][67871] Updated weights for policy 1, policy_version 3920 (0.0009) [2023-10-07 19:47:46,288][67871] Updated weights for policy 1, policy_version 3930 (0.0009) [2023-10-07 19:47:46,445][67838] Updated weights for policy 0, policy_version 3912 (0.0009) [2023-10-07 19:47:46,820][67838] Updated weights for policy 0, policy_version 3922 (0.0010) [2023-10-07 19:47:47,200][67838] Updated weights for policy 0, policy_version 3932 (0.0007) [2023-10-07 19:47:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 8060928. Throughput: 0: 1633.3, 1: 1638.0. Samples: 2017304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:47:47,478][66916] Avg episode reward: [(0, '24.990'), (1, '23.630')] [2023-10-07 19:47:50,510][67871] Updated weights for policy 1, policy_version 3940 (0.0008) [2023-10-07 19:47:50,876][67871] Updated weights for policy 1, policy_version 3950 (0.0010) [2023-10-07 19:47:51,243][67871] Updated weights for policy 1, policy_version 3960 (0.0010) [2023-10-07 19:47:51,416][67838] Updated weights for policy 0, policy_version 3942 (0.0007) [2023-10-07 19:47:51,794][67838] Updated weights for policy 0, policy_version 3952 (0.0010) [2023-10-07 19:47:52,159][67838] Updated weights for policy 0, policy_version 3962 (0.0010) [2023-10-07 19:47:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 8126464. Throughput: 0: 1629.5, 1: 1631.6. Samples: 2035754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:47:52,477][66916] Avg episode reward: [(0, '24.940'), (1, '24.360')] [2023-10-07 19:47:55,276][67871] Updated weights for policy 1, policy_version 3970 (0.0007) [2023-10-07 19:47:55,648][67871] Updated weights for policy 1, policy_version 3980 (0.0010) [2023-10-07 19:47:56,025][67871] Updated weights for policy 1, policy_version 3990 (0.0008) [2023-10-07 19:47:56,395][67871] Updated weights for policy 1, policy_version 4000 (0.0008) [2023-10-07 19:47:56,468][67838] Updated weights for policy 0, policy_version 3972 (0.0010) [2023-10-07 19:47:56,857][67838] Updated weights for policy 0, policy_version 3982 (0.0010) [2023-10-07 19:47:57,240][67838] Updated weights for policy 0, policy_version 3992 (0.0010) [2023-10-07 19:47:57,476][66916] Fps is (10 sec: 9830.6, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 8159232. Throughput: 0: 1626.5, 1: 1632.0. Samples: 2046750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:47:57,477][66916] Avg episode reward: [(0, '24.240'), (1, '24.920')] [2023-10-07 19:48:00,664][67871] Updated weights for policy 1, policy_version 4010 (0.0007) [2023-10-07 19:48:01,042][67871] Updated weights for policy 1, policy_version 4020 (0.0008) [2023-10-07 19:48:01,406][67871] Updated weights for policy 1, policy_version 4030 (0.0008) [2023-10-07 19:48:01,425][67838] Updated weights for policy 0, policy_version 4002 (0.0009) [2023-10-07 19:48:01,791][67838] Updated weights for policy 0, policy_version 4012 (0.0007) [2023-10-07 19:48:02,176][67838] Updated weights for policy 0, policy_version 4022 (0.0010) [2023-10-07 19:48:02,477][66916] Fps is (10 sec: 9830.2, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 8224768. Throughput: 0: 1622.8, 1: 1636.6. Samples: 2066026. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:48:02,478][66916] Avg episode reward: [(0, '24.480'), (1, '24.220')] [2023-10-07 19:48:02,554][67838] Updated weights for policy 0, policy_version 4032 (0.0007) [2023-10-07 19:48:05,430][67871] Updated weights for policy 1, policy_version 4040 (0.0008) [2023-10-07 19:48:05,798][67871] Updated weights for policy 1, policy_version 4050 (0.0010) [2023-10-07 19:48:06,172][67871] Updated weights for policy 1, policy_version 4060 (0.0008) [2023-10-07 19:48:06,791][67838] Updated weights for policy 0, policy_version 4042 (0.0007) [2023-10-07 19:48:07,155][67838] Updated weights for policy 0, policy_version 4052 (0.0010) [2023-10-07 19:48:07,476][66916] Fps is (10 sec: 13107.0, 60 sec: 12561.2, 300 sec: 12996.1). Total num frames: 8290304. Throughput: 0: 1627.4, 1: 1638.9. Samples: 2084914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:48:07,477][66916] Avg episode reward: [(0, '24.960'), (1, '24.190')] [2023-10-07 19:48:07,524][67838] Updated weights for policy 0, policy_version 4062 (0.0008) [2023-10-07 19:48:10,557][67871] Updated weights for policy 1, policy_version 4070 (0.0008) [2023-10-07 19:48:10,922][67871] Updated weights for policy 1, policy_version 4080 (0.0007) [2023-10-07 19:48:11,288][67871] Updated weights for policy 1, policy_version 4090 (0.0008) [2023-10-07 19:48:11,784][67838] Updated weights for policy 0, policy_version 4072 (0.0007) [2023-10-07 19:48:12,163][67838] Updated weights for policy 0, policy_version 4082 (0.0008) [2023-10-07 19:48:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 8355840. Throughput: 0: 1622.7, 1: 1640.5. Samples: 2095818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:48:12,477][66916] Avg episode reward: [(0, '25.570'), (1, '23.970')] [2023-10-07 19:48:12,537][67838] Updated weights for policy 0, policy_version 4092 (0.0007) [2023-10-07 19:48:12,687][67511] Saving new best policy, reward=25.570! [2023-10-07 19:48:15,628][67871] Updated weights for policy 1, policy_version 4100 (0.0007) [2023-10-07 19:48:16,006][67871] Updated weights for policy 1, policy_version 4110 (0.0011) [2023-10-07 19:48:16,371][67871] Updated weights for policy 1, policy_version 4120 (0.0009) [2023-10-07 19:48:16,702][67838] Updated weights for policy 0, policy_version 4102 (0.0009) [2023-10-07 19:48:17,079][67838] Updated weights for policy 0, policy_version 4112 (0.0009) [2023-10-07 19:48:17,448][67838] Updated weights for policy 0, policy_version 4122 (0.0008) [2023-10-07 19:48:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 8421376. Throughput: 0: 1623.5, 1: 1642.7. Samples: 2115394. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-07 19:48:17,477][66916] Avg episode reward: [(0, '25.930'), (1, '24.710')] [2023-10-07 19:48:17,673][67511] Saving new best policy, reward=25.930! [2023-10-07 19:48:20,408][67871] Updated weights for policy 1, policy_version 4130 (0.0008) [2023-10-07 19:48:20,769][67871] Updated weights for policy 1, policy_version 4140 (0.0008) [2023-10-07 19:48:21,143][67871] Updated weights for policy 1, policy_version 4150 (0.0010) [2023-10-07 19:48:21,526][67871] Updated weights for policy 1, policy_version 4160 (0.0009) [2023-10-07 19:48:21,771][67838] Updated weights for policy 0, policy_version 4132 (0.0010) [2023-10-07 19:48:22,136][67838] Updated weights for policy 0, policy_version 4142 (0.0010) [2023-10-07 19:48:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 8486912. Throughput: 0: 1632.2, 1: 1642.2. Samples: 2134254. Policy #0 lag: (min: 7.0, avg: 19.3, max: 39.0) [2023-10-07 19:48:22,477][66916] Avg episode reward: [(0, '26.170'), (1, '24.030')] [2023-10-07 19:48:22,511][67838] Updated weights for policy 0, policy_version 4152 (0.0009) [2023-10-07 19:48:22,818][67511] Saving new best policy, reward=26.170! [2023-10-07 19:48:25,548][67871] Updated weights for policy 1, policy_version 4170 (0.0010) [2023-10-07 19:48:25,935][67871] Updated weights for policy 1, policy_version 4180 (0.0010) [2023-10-07 19:48:26,301][67871] Updated weights for policy 1, policy_version 4190 (0.0010) [2023-10-07 19:48:26,981][67838] Updated weights for policy 0, policy_version 4162 (0.0010) [2023-10-07 19:48:27,357][67838] Updated weights for policy 0, policy_version 4172 (0.0008) [2023-10-07 19:48:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 8552448. Throughput: 0: 1618.3, 1: 1647.7. Samples: 2144708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:48:27,477][66916] Avg episode reward: [(0, '25.490'), (1, '24.120')] [2023-10-07 19:48:27,737][67838] Updated weights for policy 0, policy_version 4182 (0.0008) [2023-10-07 19:48:28,103][67838] Updated weights for policy 0, policy_version 4192 (0.0010) [2023-10-07 19:48:30,302][67871] Updated weights for policy 1, policy_version 4200 (0.0009) [2023-10-07 19:48:30,669][67871] Updated weights for policy 1, policy_version 4210 (0.0008) [2023-10-07 19:48:31,044][67871] Updated weights for policy 1, policy_version 4220 (0.0008) [2023-10-07 19:48:32,198][67838] Updated weights for policy 0, policy_version 4202 (0.0007) [2023-10-07 19:48:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 12561.0, 300 sec: 12996.1). Total num frames: 8617984. Throughput: 0: 1620.2, 1: 1646.8. Samples: 2164318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:48:32,478][66916] Avg episode reward: [(0, '25.750'), (1, '25.550')] [2023-10-07 19:48:32,479][67676] Saving new best policy, reward=25.550! [2023-10-07 19:48:32,569][67838] Updated weights for policy 0, policy_version 4212 (0.0007) [2023-10-07 19:48:32,936][67838] Updated weights for policy 0, policy_version 4222 (0.0007) [2023-10-07 19:48:35,198][67871] Updated weights for policy 1, policy_version 4230 (0.0009) [2023-10-07 19:48:35,560][67871] Updated weights for policy 1, policy_version 4240 (0.0009) [2023-10-07 19:48:35,934][67871] Updated weights for policy 1, policy_version 4250 (0.0010) [2023-10-07 19:48:37,073][67838] Updated weights for policy 0, policy_version 4232 (0.0011) [2023-10-07 19:48:37,440][67838] Updated weights for policy 0, policy_version 4242 (0.0011) [2023-10-07 19:48:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 8683520. Throughput: 0: 1640.6, 1: 1650.3. Samples: 2183848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:48:37,477][66916] Avg episode reward: [(0, '25.940'), (1, '24.410')] [2023-10-07 19:48:37,808][67838] Updated weights for policy 0, policy_version 4252 (0.0009) [2023-10-07 19:48:40,115][67871] Updated weights for policy 1, policy_version 4260 (0.0008) [2023-10-07 19:48:40,490][67871] Updated weights for policy 1, policy_version 4270 (0.0008) [2023-10-07 19:48:40,851][67871] Updated weights for policy 1, policy_version 4280 (0.0007) [2023-10-07 19:48:41,911][67838] Updated weights for policy 0, policy_version 4262 (0.0009) [2023-10-07 19:48:42,277][67838] Updated weights for policy 0, policy_version 4272 (0.0009) [2023-10-07 19:48:42,477][66916] Fps is (10 sec: 13107.4, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 8749056. Throughput: 0: 1627.0, 1: 1647.9. Samples: 2194120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:48:42,477][66916] Avg episode reward: [(0, '26.520'), (1, '24.450')] [2023-10-07 19:48:42,655][67838] Updated weights for policy 0, policy_version 4282 (0.0007) [2023-10-07 19:48:42,870][67511] Saving new best policy, reward=26.520! [2023-10-07 19:48:45,060][67871] Updated weights for policy 1, policy_version 4290 (0.0008) [2023-10-07 19:48:45,425][67871] Updated weights for policy 1, policy_version 4300 (0.0010) [2023-10-07 19:48:45,788][67871] Updated weights for policy 1, policy_version 4310 (0.0010) [2023-10-07 19:48:46,164][67871] Updated weights for policy 1, policy_version 4320 (0.0010) [2023-10-07 19:48:46,975][67838] Updated weights for policy 0, policy_version 4292 (0.0008) [2023-10-07 19:48:47,344][67838] Updated weights for policy 0, policy_version 4302 (0.0007) [2023-10-07 19:48:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 8814592. Throughput: 0: 1627.9, 1: 1639.1. Samples: 2213040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:48:47,478][66916] Avg episode reward: [(0, '25.580'), (1, '24.810')] [2023-10-07 19:48:47,723][67838] Updated weights for policy 0, policy_version 4312 (0.0009) [2023-10-07 19:48:50,323][67871] Updated weights for policy 1, policy_version 4330 (0.0011) [2023-10-07 19:48:50,702][67871] Updated weights for policy 1, policy_version 4340 (0.0009) [2023-10-07 19:48:51,075][67871] Updated weights for policy 1, policy_version 4350 (0.0008) [2023-10-07 19:48:51,794][67838] Updated weights for policy 0, policy_version 4322 (0.0009) [2023-10-07 19:48:52,164][67838] Updated weights for policy 0, policy_version 4332 (0.0008) [2023-10-07 19:48:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 12561.0, 300 sec: 12996.1). Total num frames: 8880128. Throughput: 0: 1633.6, 1: 1639.5. Samples: 2232206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:48:52,477][66916] Avg episode reward: [(0, '26.450'), (1, '25.200')] [2023-10-07 19:48:52,545][67838] Updated weights for policy 0, policy_version 4342 (0.0008) [2023-10-07 19:48:52,912][67838] Updated weights for policy 0, policy_version 4352 (0.0007) [2023-10-07 19:48:55,626][67871] Updated weights for policy 1, policy_version 4360 (0.0008) [2023-10-07 19:48:56,006][67871] Updated weights for policy 1, policy_version 4370 (0.0009) [2023-10-07 19:48:56,379][67871] Updated weights for policy 1, policy_version 4380 (0.0010) [2023-10-07 19:48:57,321][67838] Updated weights for policy 0, policy_version 4362 (0.0010) [2023-10-07 19:48:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 12996.1). Total num frames: 8945664. Throughput: 0: 1621.1, 1: 1636.6. Samples: 2242416. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-07 19:48:57,478][66916] Avg episode reward: [(0, '25.340'), (1, '24.550')] [2023-10-07 19:48:57,693][67838] Updated weights for policy 0, policy_version 4372 (0.0010) [2023-10-07 19:48:58,064][67838] Updated weights for policy 0, policy_version 4382 (0.0012) [2023-10-07 19:49:00,769][67871] Updated weights for policy 1, policy_version 4390 (0.0010) [2023-10-07 19:49:01,122][67871] Updated weights for policy 1, policy_version 4400 (0.0011) [2023-10-07 19:49:01,504][67871] Updated weights for policy 1, policy_version 4410 (0.0008) [2023-10-07 19:49:02,345][67838] Updated weights for policy 0, policy_version 4392 (0.0008) [2023-10-07 19:49:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9011200. Throughput: 0: 1622.0, 1: 1632.2. Samples: 2261834. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-07 19:49:02,477][66916] Avg episode reward: [(0, '25.700'), (1, '25.200')] [2023-10-07 19:49:02,725][67838] Updated weights for policy 0, policy_version 4402 (0.0007) [2023-10-07 19:49:03,099][67838] Updated weights for policy 0, policy_version 4412 (0.0007) [2023-10-07 19:49:05,588][67871] Updated weights for policy 1, policy_version 4420 (0.0011) [2023-10-07 19:49:05,954][67871] Updated weights for policy 1, policy_version 4430 (0.0009) [2023-10-07 19:49:06,321][67871] Updated weights for policy 1, policy_version 4440 (0.0008) [2023-10-07 19:49:07,156][67838] Updated weights for policy 0, policy_version 4422 (0.0008) [2023-10-07 19:49:07,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9076736. Throughput: 0: 1632.0, 1: 1625.6. Samples: 2280846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:49:07,477][66916] Avg episode reward: [(0, '26.900'), (1, '24.610')] [2023-10-07 19:49:07,484][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000004448_4554752.pth... [2023-10-07 19:49:07,512][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000002912_2981888.pth [2023-10-07 19:49:07,530][67838] Updated weights for policy 0, policy_version 4432 (0.0008) [2023-10-07 19:49:07,906][67838] Updated weights for policy 0, policy_version 4442 (0.0007) [2023-10-07 19:49:08,127][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000004448_4554752.pth... [2023-10-07 19:49:08,157][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000002912_2981888.pth [2023-10-07 19:49:08,160][67511] Saving new best policy, reward=26.900! [2023-10-07 19:49:10,370][67871] Updated weights for policy 1, policy_version 4450 (0.0007) [2023-10-07 19:49:10,740][67871] Updated weights for policy 1, policy_version 4460 (0.0010) [2023-10-07 19:49:11,111][67871] Updated weights for policy 1, policy_version 4470 (0.0009) [2023-10-07 19:49:11,484][67871] Updated weights for policy 1, policy_version 4480 (0.0009) [2023-10-07 19:49:12,222][67838] Updated weights for policy 0, policy_version 4452 (0.0010) [2023-10-07 19:49:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9142272. Throughput: 0: 1627.7, 1: 1621.8. Samples: 2290938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:49:12,477][66916] Avg episode reward: [(0, '26.730'), (1, '26.540')] [2023-10-07 19:49:12,477][67676] Saving new best policy, reward=26.540! [2023-10-07 19:49:12,602][67838] Updated weights for policy 0, policy_version 4462 (0.0007) [2023-10-07 19:49:12,966][67838] Updated weights for policy 0, policy_version 4472 (0.0008) [2023-10-07 19:49:15,736][67871] Updated weights for policy 1, policy_version 4490 (0.0007) [2023-10-07 19:49:16,111][67871] Updated weights for policy 1, policy_version 4500 (0.0009) [2023-10-07 19:49:16,481][67871] Updated weights for policy 1, policy_version 4510 (0.0008) [2023-10-07 19:49:17,127][67838] Updated weights for policy 0, policy_version 4482 (0.0009) [2023-10-07 19:49:17,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9207808. Throughput: 0: 1624.2, 1: 1626.2. Samples: 2310586. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 19:49:17,477][66916] Avg episode reward: [(0, '26.080'), (1, '25.580')] [2023-10-07 19:49:17,497][67838] Updated weights for policy 0, policy_version 4492 (0.0009) [2023-10-07 19:49:17,860][67838] Updated weights for policy 0, policy_version 4502 (0.0009) [2023-10-07 19:49:18,242][67838] Updated weights for policy 0, policy_version 4512 (0.0010) [2023-10-07 19:49:20,845][67871] Updated weights for policy 1, policy_version 4520 (0.0009) [2023-10-07 19:49:21,217][67871] Updated weights for policy 1, policy_version 4530 (0.0007) [2023-10-07 19:49:21,591][67871] Updated weights for policy 1, policy_version 4540 (0.0009) [2023-10-07 19:49:22,440][67838] Updated weights for policy 0, policy_version 4522 (0.0007) [2023-10-07 19:49:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 12996.1). Total num frames: 9273344. Throughput: 0: 1630.3, 1: 1617.1. Samples: 2329980. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 19:49:22,478][66916] Avg episode reward: [(0, '26.490'), (1, '25.160')] [2023-10-07 19:49:22,805][67838] Updated weights for policy 0, policy_version 4532 (0.0008) [2023-10-07 19:49:23,179][67838] Updated weights for policy 0, policy_version 4542 (0.0008) [2023-10-07 19:49:25,945][67871] Updated weights for policy 1, policy_version 4550 (0.0009) [2023-10-07 19:49:26,316][67871] Updated weights for policy 1, policy_version 4560 (0.0009) [2023-10-07 19:49:26,675][67871] Updated weights for policy 1, policy_version 4570 (0.0008) [2023-10-07 19:49:27,423][67838] Updated weights for policy 0, policy_version 4552 (0.0007) [2023-10-07 19:49:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9338880. Throughput: 0: 1624.0, 1: 1617.2. Samples: 2339972. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-07 19:49:27,477][66916] Avg episode reward: [(0, '25.700'), (1, '24.680')] [2023-10-07 19:49:27,799][67838] Updated weights for policy 0, policy_version 4562 (0.0008) [2023-10-07 19:49:28,167][67838] Updated weights for policy 0, policy_version 4572 (0.0009) [2023-10-07 19:49:30,854][67871] Updated weights for policy 1, policy_version 4580 (0.0010) [2023-10-07 19:49:31,226][67871] Updated weights for policy 1, policy_version 4590 (0.0011) [2023-10-07 19:49:31,589][67871] Updated weights for policy 1, policy_version 4600 (0.0011) [2023-10-07 19:49:32,389][67838] Updated weights for policy 0, policy_version 4582 (0.0008) [2023-10-07 19:49:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 12996.1). Total num frames: 9404416. Throughput: 0: 1630.4, 1: 1633.0. Samples: 2359892. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-07 19:49:32,477][66916] Avg episode reward: [(0, '26.320'), (1, '25.130')] [2023-10-07 19:49:32,768][67838] Updated weights for policy 0, policy_version 4592 (0.0007) [2023-10-07 19:49:33,142][67838] Updated weights for policy 0, policy_version 4602 (0.0007) [2023-10-07 19:49:36,018][67871] Updated weights for policy 1, policy_version 4610 (0.0009) [2023-10-07 19:49:36,390][67871] Updated weights for policy 1, policy_version 4620 (0.0007) [2023-10-07 19:49:36,754][67871] Updated weights for policy 1, policy_version 4630 (0.0011) [2023-10-07 19:49:37,123][67871] Updated weights for policy 1, policy_version 4640 (0.0007) [2023-10-07 19:49:37,132][67838] Updated weights for policy 0, policy_version 4612 (0.0007) [2023-10-07 19:49:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9469952. Throughput: 0: 1638.3, 1: 1623.5. Samples: 2378988. Policy #0 lag: (min: 10.0, avg: 15.3, max: 42.0) [2023-10-07 19:49:37,477][66916] Avg episode reward: [(0, '26.280'), (1, '23.750')] [2023-10-07 19:49:37,505][67838] Updated weights for policy 0, policy_version 4622 (0.0007) [2023-10-07 19:49:37,881][67838] Updated weights for policy 0, policy_version 4632 (0.0007) [2023-10-07 19:49:41,269][67871] Updated weights for policy 1, policy_version 4650 (0.0011) [2023-10-07 19:49:41,637][67871] Updated weights for policy 1, policy_version 4660 (0.0009) [2023-10-07 19:49:41,843][67838] Updated weights for policy 0, policy_version 4642 (0.0007) [2023-10-07 19:49:41,999][67871] Updated weights for policy 1, policy_version 4670 (0.0007) [2023-10-07 19:49:42,238][67838] Updated weights for policy 0, policy_version 4652 (0.0007) [2023-10-07 19:49:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9535488. Throughput: 0: 1636.8, 1: 1619.2. Samples: 2388938. Policy #0 lag: (min: 10.0, avg: 15.3, max: 42.0) [2023-10-07 19:49:42,477][66916] Avg episode reward: [(0, '26.230'), (1, '25.190')] [2023-10-07 19:49:42,614][67838] Updated weights for policy 0, policy_version 4662 (0.0007) [2023-10-07 19:49:42,995][67838] Updated weights for policy 0, policy_version 4672 (0.0008) [2023-10-07 19:49:46,248][67871] Updated weights for policy 1, policy_version 4680 (0.0008) [2023-10-07 19:49:46,616][67871] Updated weights for policy 1, policy_version 4690 (0.0007) [2023-10-07 19:49:46,982][67871] Updated weights for policy 1, policy_version 4700 (0.0010) [2023-10-07 19:49:47,241][67838] Updated weights for policy 0, policy_version 4682 (0.0010) [2023-10-07 19:49:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9601024. Throughput: 0: 1637.1, 1: 1629.1. Samples: 2408814. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-07 19:49:47,477][66916] Avg episode reward: [(0, '26.460'), (1, '26.320')] [2023-10-07 19:49:47,618][67838] Updated weights for policy 0, policy_version 4692 (0.0009) [2023-10-07 19:49:47,994][67838] Updated weights for policy 0, policy_version 4702 (0.0008) [2023-10-07 19:49:51,214][67871] Updated weights for policy 1, policy_version 4710 (0.0008) [2023-10-07 19:49:51,577][67871] Updated weights for policy 1, policy_version 4720 (0.0007) [2023-10-07 19:49:51,945][67871] Updated weights for policy 1, policy_version 4730 (0.0009) [2023-10-07 19:49:52,215][67838] Updated weights for policy 0, policy_version 4712 (0.0007) [2023-10-07 19:49:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9666560. Throughput: 0: 1635.3, 1: 1629.8. Samples: 2427778. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-07 19:49:52,477][66916] Avg episode reward: [(0, '25.660'), (1, '25.630')] [2023-10-07 19:49:52,593][67838] Updated weights for policy 0, policy_version 4722 (0.0008) [2023-10-07 19:49:52,973][67838] Updated weights for policy 0, policy_version 4732 (0.0008) [2023-10-07 19:49:56,171][67871] Updated weights for policy 1, policy_version 4740 (0.0008) [2023-10-07 19:49:56,533][67871] Updated weights for policy 1, policy_version 4750 (0.0009) [2023-10-07 19:49:56,908][67871] Updated weights for policy 1, policy_version 4760 (0.0009) [2023-10-07 19:49:57,168][67838] Updated weights for policy 0, policy_version 4742 (0.0009) [2023-10-07 19:49:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 12996.1). Total num frames: 9732096. Throughput: 0: 1634.2, 1: 1625.2. Samples: 2437612. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 19:49:57,477][66916] Avg episode reward: [(0, '26.360'), (1, '25.570')] [2023-10-07 19:49:57,537][67838] Updated weights for policy 0, policy_version 4752 (0.0007) [2023-10-07 19:49:57,911][67838] Updated weights for policy 0, policy_version 4762 (0.0008) [2023-10-07 19:50:00,983][67871] Updated weights for policy 1, policy_version 4770 (0.0009) [2023-10-07 19:50:01,359][67871] Updated weights for policy 1, policy_version 4780 (0.0010) [2023-10-07 19:50:01,725][67871] Updated weights for policy 1, policy_version 4790 (0.0010) [2023-10-07 19:50:02,092][67871] Updated weights for policy 1, policy_version 4800 (0.0010) [2023-10-07 19:50:02,191][67838] Updated weights for policy 0, policy_version 4772 (0.0007) [2023-10-07 19:50:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9797632. Throughput: 0: 1639.4, 1: 1634.2. Samples: 2457898. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 19:50:02,478][66916] Avg episode reward: [(0, '26.190'), (1, '26.130')] [2023-10-07 19:50:02,564][67838] Updated weights for policy 0, policy_version 4782 (0.0008) [2023-10-07 19:50:02,944][67838] Updated weights for policy 0, policy_version 4792 (0.0008) [2023-10-07 19:50:06,364][67871] Updated weights for policy 1, policy_version 4810 (0.0009) [2023-10-07 19:50:06,721][67871] Updated weights for policy 1, policy_version 4820 (0.0011) [2023-10-07 19:50:07,060][67838] Updated weights for policy 0, policy_version 4802 (0.0009) [2023-10-07 19:50:07,094][67871] Updated weights for policy 1, policy_version 4830 (0.0009) [2023-10-07 19:50:07,435][67838] Updated weights for policy 0, policy_version 4812 (0.0007) [2023-10-07 19:50:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9863168. Throughput: 0: 1634.9, 1: 1634.1. Samples: 2477088. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 19:50:07,477][66916] Avg episode reward: [(0, '26.990'), (1, '25.730')] [2023-10-07 19:50:07,804][67838] Updated weights for policy 0, policy_version 4822 (0.0009) [2023-10-07 19:50:08,176][67511] Saving new best policy, reward=26.990! [2023-10-07 19:50:08,180][67838] Updated weights for policy 0, policy_version 4832 (0.0009) [2023-10-07 19:50:11,239][67871] Updated weights for policy 1, policy_version 4840 (0.0008) [2023-10-07 19:50:11,598][67871] Updated weights for policy 1, policy_version 4850 (0.0010) [2023-10-07 19:50:11,969][67871] Updated weights for policy 1, policy_version 4860 (0.0010) [2023-10-07 19:50:12,106][67838] Updated weights for policy 0, policy_version 4842 (0.0008) [2023-10-07 19:50:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9928704. Throughput: 0: 1640.5, 1: 1626.3. Samples: 2486978. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 19:50:12,477][67838] Updated weights for policy 0, policy_version 4852 (0.0008) [2023-10-07 19:50:12,477][66916] Avg episode reward: [(0, '26.240'), (1, '24.950')] [2023-10-07 19:50:12,852][67838] Updated weights for policy 0, policy_version 4862 (0.0008) [2023-10-07 19:50:16,228][67871] Updated weights for policy 1, policy_version 4870 (0.0011) [2023-10-07 19:50:16,592][67871] Updated weights for policy 1, policy_version 4880 (0.0009) [2023-10-07 19:50:16,968][67871] Updated weights for policy 1, policy_version 4890 (0.0007) [2023-10-07 19:50:17,256][67838] Updated weights for policy 0, policy_version 4872 (0.0010) [2023-10-07 19:50:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9994240. Throughput: 0: 1633.5, 1: 1631.3. Samples: 2506808. Policy #0 lag: (min: 12.0, avg: 20.9, max: 44.0) [2023-10-07 19:50:17,478][66916] Avg episode reward: [(0, '26.640'), (1, '25.660')] [2023-10-07 19:50:17,638][67838] Updated weights for policy 0, policy_version 4882 (0.0009) [2023-10-07 19:50:18,006][67838] Updated weights for policy 0, policy_version 4892 (0.0009) [2023-10-07 19:50:21,130][67871] Updated weights for policy 1, policy_version 4900 (0.0008) [2023-10-07 19:50:21,497][67871] Updated weights for policy 1, policy_version 4910 (0.0011) [2023-10-07 19:50:21,873][67871] Updated weights for policy 1, policy_version 4920 (0.0010) [2023-10-07 19:50:22,222][67838] Updated weights for policy 0, policy_version 4902 (0.0009) [2023-10-07 19:50:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 12996.1). Total num frames: 10059776. Throughput: 0: 1632.5, 1: 1632.0. Samples: 2525894. Policy #0 lag: (min: 12.0, avg: 20.9, max: 44.0) [2023-10-07 19:50:22,477][66916] Avg episode reward: [(0, '26.830'), (1, '25.970')] [2023-10-07 19:50:22,592][67838] Updated weights for policy 0, policy_version 4912 (0.0010) [2023-10-07 19:50:22,966][67838] Updated weights for policy 0, policy_version 4922 (0.0010) [2023-10-07 19:50:26,015][67871] Updated weights for policy 1, policy_version 4930 (0.0008) [2023-10-07 19:50:26,434][67871] Updated weights for policy 1, policy_version 4940 (0.0007) [2023-10-07 19:50:26,812][67871] Updated weights for policy 1, policy_version 4950 (0.0008) [2023-10-07 19:50:27,186][67871] Updated weights for policy 1, policy_version 4960 (0.0007) [2023-10-07 19:50:27,216][67838] Updated weights for policy 0, policy_version 4932 (0.0009) [2023-10-07 19:50:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 10125312. Throughput: 0: 1629.5, 1: 1630.6. Samples: 2535642. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:50:27,477][66916] Avg episode reward: [(0, '27.990'), (1, '25.530')] [2023-10-07 19:50:27,610][67838] Updated weights for policy 0, policy_version 4942 (0.0008) [2023-10-07 19:50:27,986][67838] Updated weights for policy 0, policy_version 4952 (0.0008) [2023-10-07 19:50:28,270][67511] Saving new best policy, reward=27.990! [2023-10-07 19:50:31,368][67871] Updated weights for policy 1, policy_version 4970 (0.0009) [2023-10-07 19:50:31,744][67871] Updated weights for policy 1, policy_version 4980 (0.0009) [2023-10-07 19:50:32,117][67871] Updated weights for policy 1, policy_version 4990 (0.0009) [2023-10-07 19:50:32,333][67838] Updated weights for policy 0, policy_version 4962 (0.0008) [2023-10-07 19:50:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 10190848. Throughput: 0: 1627.2, 1: 1630.2. Samples: 2555396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:50:32,477][66916] Avg episode reward: [(0, '27.140'), (1, '24.880')] [2023-10-07 19:50:32,707][67838] Updated weights for policy 0, policy_version 4972 (0.0008) [2023-10-07 19:50:33,083][67838] Updated weights for policy 0, policy_version 4982 (0.0008) [2023-10-07 19:50:33,466][67838] Updated weights for policy 0, policy_version 4992 (0.0010) [2023-10-07 19:50:36,303][67871] Updated weights for policy 1, policy_version 5000 (0.0008) [2023-10-07 19:50:36,675][67871] Updated weights for policy 1, policy_version 5010 (0.0008) [2023-10-07 19:50:37,044][67871] Updated weights for policy 1, policy_version 5020 (0.0007) [2023-10-07 19:50:37,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 10256384. Throughput: 0: 1627.0, 1: 1630.3. Samples: 2574358. Policy #0 lag: (min: 8.0, avg: 31.9, max: 40.0) [2023-10-07 19:50:37,478][66916] Avg episode reward: [(0, '27.340'), (1, '25.010')] [2023-10-07 19:50:37,642][67838] Updated weights for policy 0, policy_version 5002 (0.0007) [2023-10-07 19:50:38,019][67838] Updated weights for policy 0, policy_version 5012 (0.0008) [2023-10-07 19:50:38,395][67838] Updated weights for policy 0, policy_version 5022 (0.0008) [2023-10-07 19:50:41,250][67871] Updated weights for policy 1, policy_version 5030 (0.0009) [2023-10-07 19:50:41,624][67871] Updated weights for policy 1, policy_version 5040 (0.0009) [2023-10-07 19:50:41,986][67871] Updated weights for policy 1, policy_version 5050 (0.0009) [2023-10-07 19:50:42,413][67838] Updated weights for policy 0, policy_version 5032 (0.0009) [2023-10-07 19:50:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 10321920. Throughput: 0: 1626.2, 1: 1627.8. Samples: 2584042. Policy #0 lag: (min: 8.0, avg: 31.9, max: 40.0) [2023-10-07 19:50:42,478][66916] Avg episode reward: [(0, '26.610'), (1, '25.600')] [2023-10-07 19:50:42,789][67838] Updated weights for policy 0, policy_version 5042 (0.0007) [2023-10-07 19:50:43,161][67838] Updated weights for policy 0, policy_version 5052 (0.0007) [2023-10-07 19:50:46,300][67871] Updated weights for policy 1, policy_version 5060 (0.0008) [2023-10-07 19:50:46,672][67871] Updated weights for policy 1, policy_version 5070 (0.0009) [2023-10-07 19:50:47,040][67871] Updated weights for policy 1, policy_version 5080 (0.0008) [2023-10-07 19:50:47,448][67838] Updated weights for policy 0, policy_version 5062 (0.0008) [2023-10-07 19:50:47,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 10387456. Throughput: 0: 1620.4, 1: 1625.3. Samples: 2603952. Policy #0 lag: (min: 27.0, avg: 30.5, max: 59.0) [2023-10-07 19:50:47,477][66916] Avg episode reward: [(0, '27.780'), (1, '25.410')] [2023-10-07 19:50:47,830][67838] Updated weights for policy 0, policy_version 5072 (0.0010) [2023-10-07 19:50:48,198][67838] Updated weights for policy 0, policy_version 5082 (0.0010) [2023-10-07 19:50:51,290][67871] Updated weights for policy 1, policy_version 5090 (0.0008) [2023-10-07 19:50:51,662][67871] Updated weights for policy 1, policy_version 5100 (0.0008) [2023-10-07 19:50:52,033][67871] Updated weights for policy 1, policy_version 5110 (0.0008) [2023-10-07 19:50:52,310][67838] Updated weights for policy 0, policy_version 5092 (0.0008) [2023-10-07 19:50:52,391][67871] Updated weights for policy 1, policy_version 5120 (0.0007) [2023-10-07 19:50:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 12996.1). Total num frames: 10452992. Throughput: 0: 1623.7, 1: 1627.1. Samples: 2623372. Policy #0 lag: (min: 27.0, avg: 30.5, max: 59.0) [2023-10-07 19:50:52,478][66916] Avg episode reward: [(0, '26.530'), (1, '26.370')] [2023-10-07 19:50:52,685][67838] Updated weights for policy 0, policy_version 5102 (0.0010) [2023-10-07 19:50:53,063][67838] Updated weights for policy 0, policy_version 5112 (0.0007) [2023-10-07 19:50:56,537][67871] Updated weights for policy 1, policy_version 5130 (0.0007) [2023-10-07 19:50:56,917][67871] Updated weights for policy 1, policy_version 5140 (0.0008) [2023-10-07 19:50:57,275][67871] Updated weights for policy 1, policy_version 5150 (0.0010) [2023-10-07 19:50:57,404][67838] Updated weights for policy 0, policy_version 5122 (0.0007) [2023-10-07 19:50:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 10518528. Throughput: 0: 1619.1, 1: 1626.4. Samples: 2633028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:50:57,478][66916] Avg episode reward: [(0, '27.340'), (1, '26.330')] [2023-10-07 19:50:57,785][67838] Updated weights for policy 0, policy_version 5132 (0.0008) [2023-10-07 19:50:58,160][67838] Updated weights for policy 0, policy_version 5142 (0.0008) [2023-10-07 19:50:58,531][67838] Updated weights for policy 0, policy_version 5152 (0.0009) [2023-10-07 19:51:01,417][67871] Updated weights for policy 1, policy_version 5160 (0.0008) [2023-10-07 19:51:01,790][67871] Updated weights for policy 1, policy_version 5170 (0.0009) [2023-10-07 19:51:02,154][67871] Updated weights for policy 1, policy_version 5180 (0.0010) [2023-10-07 19:51:02,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 10584064. Throughput: 0: 1621.9, 1: 1627.4. Samples: 2653028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:51:02,477][66916] Avg episode reward: [(0, '25.860'), (1, '25.590')] [2023-10-07 19:51:02,798][67838] Updated weights for policy 0, policy_version 5162 (0.0007) [2023-10-07 19:51:03,169][67838] Updated weights for policy 0, policy_version 5172 (0.0010) [2023-10-07 19:51:03,542][67838] Updated weights for policy 0, policy_version 5182 (0.0010) [2023-10-07 19:51:06,286][67871] Updated weights for policy 1, policy_version 5190 (0.0009) [2023-10-07 19:51:06,655][67871] Updated weights for policy 1, policy_version 5200 (0.0007) [2023-10-07 19:51:07,018][67871] Updated weights for policy 1, policy_version 5210 (0.0008) [2023-10-07 19:51:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 10649600. Throughput: 0: 1624.5, 1: 1631.9. Samples: 2672430. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-07 19:51:07,478][66916] Avg episode reward: [(0, '26.710'), (1, '26.080')] [2023-10-07 19:51:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000005216_5341184.pth... [2023-10-07 19:51:07,524][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000003680_3768320.pth [2023-10-07 19:51:07,739][67838] Updated weights for policy 0, policy_version 5192 (0.0008) [2023-10-07 19:51:08,101][67838] Updated weights for policy 0, policy_version 5202 (0.0008) [2023-10-07 19:51:08,476][67838] Updated weights for policy 0, policy_version 5212 (0.0008) [2023-10-07 19:51:08,615][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000005216_5341184.pth... [2023-10-07 19:51:08,644][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000003680_3768320.pth [2023-10-07 19:51:11,220][67871] Updated weights for policy 1, policy_version 5220 (0.0008) [2023-10-07 19:51:11,607][67871] Updated weights for policy 1, policy_version 5230 (0.0009) [2023-10-07 19:51:11,978][67871] Updated weights for policy 1, policy_version 5240 (0.0010) [2023-10-07 19:51:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 10715136. Throughput: 0: 1622.7, 1: 1630.3. Samples: 2682026. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-07 19:51:12,478][66916] Avg episode reward: [(0, '28.600'), (1, '25.750')] [2023-10-07 19:51:12,575][67838] Updated weights for policy 0, policy_version 5222 (0.0007) [2023-10-07 19:51:12,949][67838] Updated weights for policy 0, policy_version 5232 (0.0010) [2023-10-07 19:51:13,330][67838] Updated weights for policy 0, policy_version 5242 (0.0008) [2023-10-07 19:51:13,550][67511] Saving new best policy, reward=28.600! [2023-10-07 19:51:16,216][67871] Updated weights for policy 1, policy_version 5250 (0.0009) [2023-10-07 19:51:16,594][67871] Updated weights for policy 1, policy_version 5260 (0.0008) [2023-10-07 19:51:16,958][67871] Updated weights for policy 1, policy_version 5270 (0.0009) [2023-10-07 19:51:17,336][67871] Updated weights for policy 1, policy_version 5280 (0.0008) [2023-10-07 19:51:17,445][67838] Updated weights for policy 0, policy_version 5252 (0.0009) [2023-10-07 19:51:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 10780672. Throughput: 0: 1629.1, 1: 1630.6. Samples: 2702086. Policy #0 lag: (min: 1.0, avg: 7.3, max: 33.0) [2023-10-07 19:51:17,478][66916] Avg episode reward: [(0, '27.700'), (1, '26.090')] [2023-10-07 19:51:17,812][67838] Updated weights for policy 0, policy_version 5262 (0.0009) [2023-10-07 19:51:18,189][67838] Updated weights for policy 0, policy_version 5272 (0.0009) [2023-10-07 19:51:21,490][67871] Updated weights for policy 1, policy_version 5290 (0.0007) [2023-10-07 19:51:21,852][67871] Updated weights for policy 1, policy_version 5300 (0.0009) [2023-10-07 19:51:22,229][67871] Updated weights for policy 1, policy_version 5310 (0.0009) [2023-10-07 19:51:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 10846208. Throughput: 0: 1634.2, 1: 1638.4. Samples: 2721624. Policy #0 lag: (min: 1.0, avg: 7.3, max: 33.0) [2023-10-07 19:51:22,477][66916] Avg episode reward: [(0, '28.500'), (1, '26.020')] [2023-10-07 19:51:22,588][67838] Updated weights for policy 0, policy_version 5282 (0.0008) [2023-10-07 19:51:22,954][67838] Updated weights for policy 0, policy_version 5292 (0.0011) [2023-10-07 19:51:23,328][67838] Updated weights for policy 0, policy_version 5302 (0.0008) [2023-10-07 19:51:23,702][67838] Updated weights for policy 0, policy_version 5312 (0.0008) [2023-10-07 19:51:26,289][67871] Updated weights for policy 1, policy_version 5320 (0.0010) [2023-10-07 19:51:26,660][67871] Updated weights for policy 1, policy_version 5330 (0.0008) [2023-10-07 19:51:27,025][67871] Updated weights for policy 1, policy_version 5340 (0.0008) [2023-10-07 19:51:27,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 10911744. Throughput: 0: 1635.3, 1: 1635.1. Samples: 2731208. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 19:51:27,477][66916] Avg episode reward: [(0, '28.510'), (1, '24.930')] [2023-10-07 19:51:27,883][67838] Updated weights for policy 0, policy_version 5322 (0.0007) [2023-10-07 19:51:28,253][67838] Updated weights for policy 0, policy_version 5332 (0.0009) [2023-10-07 19:51:28,636][67838] Updated weights for policy 0, policy_version 5342 (0.0008) [2023-10-07 19:51:31,206][67871] Updated weights for policy 1, policy_version 5350 (0.0009) [2023-10-07 19:51:31,581][67871] Updated weights for policy 1, policy_version 5360 (0.0008) [2023-10-07 19:51:31,949][67871] Updated weights for policy 1, policy_version 5370 (0.0010) [2023-10-07 19:51:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 10977280. Throughput: 0: 1637.6, 1: 1638.8. Samples: 2751390. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 19:51:32,477][66916] Avg episode reward: [(0, '28.180'), (1, '25.460')] [2023-10-07 19:51:32,835][67838] Updated weights for policy 0, policy_version 5352 (0.0007) [2023-10-07 19:51:33,201][67838] Updated weights for policy 0, policy_version 5362 (0.0008) [2023-10-07 19:51:33,578][67838] Updated weights for policy 0, policy_version 5372 (0.0007) [2023-10-07 19:51:36,181][67871] Updated weights for policy 1, policy_version 5380 (0.0009) [2023-10-07 19:51:36,549][67871] Updated weights for policy 1, policy_version 5390 (0.0007) [2023-10-07 19:51:36,919][67871] Updated weights for policy 1, policy_version 5400 (0.0007) [2023-10-07 19:51:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 12996.1). Total num frames: 11042816. Throughput: 0: 1638.0, 1: 1640.2. Samples: 2770892. Policy #0 lag: (min: 4.0, avg: 9.4, max: 36.0) [2023-10-07 19:51:37,477][66916] Avg episode reward: [(0, '27.760'), (1, '25.710')] [2023-10-07 19:51:37,718][67838] Updated weights for policy 0, policy_version 5382 (0.0008) [2023-10-07 19:51:38,086][67838] Updated weights for policy 0, policy_version 5392 (0.0009) [2023-10-07 19:51:38,451][67838] Updated weights for policy 0, policy_version 5402 (0.0009) [2023-10-07 19:51:41,061][67871] Updated weights for policy 1, policy_version 5410 (0.0008) [2023-10-07 19:51:41,429][67871] Updated weights for policy 1, policy_version 5420 (0.0008) [2023-10-07 19:51:41,802][67871] Updated weights for policy 1, policy_version 5430 (0.0009) [2023-10-07 19:51:42,173][67871] Updated weights for policy 1, policy_version 5440 (0.0008) [2023-10-07 19:51:42,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 11108352. Throughput: 0: 1638.0, 1: 1642.0. Samples: 2780632. Policy #0 lag: (min: 4.0, avg: 9.4, max: 36.0) [2023-10-07 19:51:42,478][66916] Avg episode reward: [(0, '28.180'), (1, '25.520')] [2023-10-07 19:51:42,774][67838] Updated weights for policy 0, policy_version 5412 (0.0008) [2023-10-07 19:51:43,156][67838] Updated weights for policy 0, policy_version 5422 (0.0009) [2023-10-07 19:51:43,533][67838] Updated weights for policy 0, policy_version 5432 (0.0009) [2023-10-07 19:51:46,344][67871] Updated weights for policy 1, policy_version 5450 (0.0008) [2023-10-07 19:51:46,714][67871] Updated weights for policy 1, policy_version 5460 (0.0007) [2023-10-07 19:51:47,091][67871] Updated weights for policy 1, policy_version 5470 (0.0009) [2023-10-07 19:51:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 11173888. Throughput: 0: 1637.9, 1: 1641.3. Samples: 2800594. Policy #0 lag: (min: 2.0, avg: 3.4, max: 23.0) [2023-10-07 19:51:47,477][66916] Avg episode reward: [(0, '27.780'), (1, '26.290')] [2023-10-07 19:51:47,777][67838] Updated weights for policy 0, policy_version 5442 (0.0011) [2023-10-07 19:51:48,149][67838] Updated weights for policy 0, policy_version 5452 (0.0010) [2023-10-07 19:51:48,516][67838] Updated weights for policy 0, policy_version 5462 (0.0009) [2023-10-07 19:51:48,895][67838] Updated weights for policy 0, policy_version 5472 (0.0009) [2023-10-07 19:51:51,276][67871] Updated weights for policy 1, policy_version 5480 (0.0009) [2023-10-07 19:51:51,637][67871] Updated weights for policy 1, policy_version 5490 (0.0011) [2023-10-07 19:51:52,001][67871] Updated weights for policy 1, policy_version 5500 (0.0007) [2023-10-07 19:51:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 12996.1). Total num frames: 11239424. Throughput: 0: 1636.1, 1: 1640.5. Samples: 2819874. Policy #0 lag: (min: 2.0, avg: 3.4, max: 23.0) [2023-10-07 19:51:52,477][66916] Avg episode reward: [(0, '28.510'), (1, '26.390')] [2023-10-07 19:51:53,085][67838] Updated weights for policy 0, policy_version 5482 (0.0007) [2023-10-07 19:51:53,468][67838] Updated weights for policy 0, policy_version 5492 (0.0009) [2023-10-07 19:51:53,834][67838] Updated weights for policy 0, policy_version 5502 (0.0009) [2023-10-07 19:51:56,288][67871] Updated weights for policy 1, policy_version 5510 (0.0008) [2023-10-07 19:51:56,678][67871] Updated weights for policy 1, policy_version 5520 (0.0009) [2023-10-07 19:51:57,041][67871] Updated weights for policy 1, policy_version 5530 (0.0007) [2023-10-07 19:51:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 11304960. Throughput: 0: 1635.9, 1: 1644.5. Samples: 2829644. Policy #0 lag: (min: 17.0, avg: 34.8, max: 49.0) [2023-10-07 19:51:57,478][66916] Avg episode reward: [(0, '27.830'), (1, '26.700')] [2023-10-07 19:51:57,479][67676] Saving new best policy, reward=26.700! [2023-10-07 19:51:58,091][67838] Updated weights for policy 0, policy_version 5512 (0.0007) [2023-10-07 19:51:58,462][67838] Updated weights for policy 0, policy_version 5522 (0.0008) [2023-10-07 19:51:58,838][67838] Updated weights for policy 0, policy_version 5532 (0.0007) [2023-10-07 19:52:01,123][67871] Updated weights for policy 1, policy_version 5540 (0.0008) [2023-10-07 19:52:01,499][67871] Updated weights for policy 1, policy_version 5550 (0.0008) [2023-10-07 19:52:01,860][67871] Updated weights for policy 1, policy_version 5560 (0.0008) [2023-10-07 19:52:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 11370496. Throughput: 0: 1634.7, 1: 1646.1. Samples: 2849724. Policy #0 lag: (min: 17.0, avg: 34.8, max: 49.0) [2023-10-07 19:52:02,478][66916] Avg episode reward: [(0, '28.780'), (1, '26.440')] [2023-10-07 19:52:02,765][67838] Updated weights for policy 0, policy_version 5542 (0.0009) [2023-10-07 19:52:03,141][67838] Updated weights for policy 0, policy_version 5552 (0.0008) [2023-10-07 19:52:03,519][67838] Updated weights for policy 0, policy_version 5562 (0.0008) [2023-10-07 19:52:03,740][67511] Saving new best policy, reward=28.780! [2023-10-07 19:52:06,138][67871] Updated weights for policy 1, policy_version 5570 (0.0009) [2023-10-07 19:52:06,503][67871] Updated weights for policy 1, policy_version 5580 (0.0010) [2023-10-07 19:52:06,880][67871] Updated weights for policy 1, policy_version 5590 (0.0009) [2023-10-07 19:52:07,240][67871] Updated weights for policy 1, policy_version 5600 (0.0011) [2023-10-07 19:52:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 11436032. Throughput: 0: 1637.1, 1: 1644.2. Samples: 2869282. Policy #0 lag: (min: 17.0, avg: 23.7, max: 49.0) [2023-10-07 19:52:07,478][66916] Avg episode reward: [(0, '27.940'), (1, '25.530')] [2023-10-07 19:52:07,826][67838] Updated weights for policy 0, policy_version 5572 (0.0010) [2023-10-07 19:52:08,210][67838] Updated weights for policy 0, policy_version 5582 (0.0008) [2023-10-07 19:52:08,589][67838] Updated weights for policy 0, policy_version 5592 (0.0009) [2023-10-07 19:52:11,499][67871] Updated weights for policy 1, policy_version 5610 (0.0007) [2023-10-07 19:52:11,881][67871] Updated weights for policy 1, policy_version 5620 (0.0009) [2023-10-07 19:52:12,252][67871] Updated weights for policy 1, policy_version 5630 (0.0009) [2023-10-07 19:52:12,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 11501568. Throughput: 0: 1637.5, 1: 1642.3. Samples: 2878798. Policy #0 lag: (min: 17.0, avg: 23.7, max: 49.0) [2023-10-07 19:52:12,478][66916] Avg episode reward: [(0, '28.460'), (1, '27.190')] [2023-10-07 19:52:12,479][67676] Saving new best policy, reward=27.190! [2023-10-07 19:52:12,920][67838] Updated weights for policy 0, policy_version 5602 (0.0008) [2023-10-07 19:52:13,288][67838] Updated weights for policy 0, policy_version 5612 (0.0007) [2023-10-07 19:52:13,665][67838] Updated weights for policy 0, policy_version 5622 (0.0009) [2023-10-07 19:52:14,041][67838] Updated weights for policy 0, policy_version 5632 (0.0009) [2023-10-07 19:52:16,303][67871] Updated weights for policy 1, policy_version 5640 (0.0009) [2023-10-07 19:52:16,669][67871] Updated weights for policy 1, policy_version 5650 (0.0010) [2023-10-07 19:52:17,036][67871] Updated weights for policy 1, policy_version 5660 (0.0010) [2023-10-07 19:52:17,477][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 11567104. Throughput: 0: 1632.0, 1: 1642.7. Samples: 2898752. Policy #0 lag: (min: 31.0, avg: 32.9, max: 55.0) [2023-10-07 19:52:17,478][66916] Avg episode reward: [(0, '28.040'), (1, '27.540')] [2023-10-07 19:52:17,479][67676] Saving new best policy, reward=27.540! [2023-10-07 19:52:18,218][67838] Updated weights for policy 0, policy_version 5642 (0.0009) [2023-10-07 19:52:18,597][67838] Updated weights for policy 0, policy_version 5652 (0.0010) [2023-10-07 19:52:18,973][67838] Updated weights for policy 0, policy_version 5662 (0.0007) [2023-10-07 19:52:21,029][67871] Updated weights for policy 1, policy_version 5670 (0.0008) [2023-10-07 19:52:21,395][67871] Updated weights for policy 1, policy_version 5680 (0.0008) [2023-10-07 19:52:21,768][67871] Updated weights for policy 1, policy_version 5690 (0.0011) [2023-10-07 19:52:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 12996.1). Total num frames: 11632640. Throughput: 0: 1628.9, 1: 1639.6. Samples: 2917976. Policy #0 lag: (min: 31.0, avg: 32.9, max: 55.0) [2023-10-07 19:52:22,478][66916] Avg episode reward: [(0, '28.200'), (1, '27.270')] [2023-10-07 19:52:23,212][67838] Updated weights for policy 0, policy_version 5672 (0.0009) [2023-10-07 19:52:23,588][67838] Updated weights for policy 0, policy_version 5682 (0.0010) [2023-10-07 19:52:23,961][67838] Updated weights for policy 0, policy_version 5692 (0.0008) [2023-10-07 19:52:25,990][67871] Updated weights for policy 1, policy_version 5700 (0.0010) [2023-10-07 19:52:26,368][67871] Updated weights for policy 1, policy_version 5710 (0.0008) [2023-10-07 19:52:26,745][67871] Updated weights for policy 1, policy_version 5720 (0.0007) [2023-10-07 19:52:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 11698176. Throughput: 0: 1628.7, 1: 1642.9. Samples: 2927850. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-07 19:52:27,477][66916] Avg episode reward: [(0, '28.340'), (1, '26.410')] [2023-10-07 19:52:28,039][67838] Updated weights for policy 0, policy_version 5702 (0.0008) [2023-10-07 19:52:28,409][67838] Updated weights for policy 0, policy_version 5712 (0.0010) [2023-10-07 19:52:28,795][67838] Updated weights for policy 0, policy_version 5722 (0.0010) [2023-10-07 19:52:30,920][67871] Updated weights for policy 1, policy_version 5730 (0.0010) [2023-10-07 19:52:31,294][67871] Updated weights for policy 1, policy_version 5740 (0.0011) [2023-10-07 19:52:31,649][67871] Updated weights for policy 1, policy_version 5750 (0.0008) [2023-10-07 19:52:32,017][67871] Updated weights for policy 1, policy_version 5760 (0.0008) [2023-10-07 19:52:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 11763712. Throughput: 0: 1631.3, 1: 1642.5. Samples: 2947916. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-07 19:52:32,477][66916] Avg episode reward: [(0, '28.360'), (1, '26.830')] [2023-10-07 19:52:32,784][67838] Updated weights for policy 0, policy_version 5732 (0.0011) [2023-10-07 19:52:33,154][67838] Updated weights for policy 0, policy_version 5742 (0.0007) [2023-10-07 19:52:33,524][67838] Updated weights for policy 0, policy_version 5752 (0.0007) [2023-10-07 19:52:36,279][67871] Updated weights for policy 1, policy_version 5770 (0.0007) [2023-10-07 19:52:36,648][67871] Updated weights for policy 1, policy_version 5780 (0.0007) [2023-10-07 19:52:37,022][67871] Updated weights for policy 1, policy_version 5790 (0.0007) [2023-10-07 19:52:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 11829248. Throughput: 0: 1634.8, 1: 1644.4. Samples: 2967438. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-07 19:52:37,477][66916] Avg episode reward: [(0, '28.530'), (1, '28.020')] [2023-10-07 19:52:37,486][67676] Saving new best policy, reward=28.020! [2023-10-07 19:52:37,788][67838] Updated weights for policy 0, policy_version 5762 (0.0007) [2023-10-07 19:52:38,165][67838] Updated weights for policy 0, policy_version 5772 (0.0009) [2023-10-07 19:52:38,541][67838] Updated weights for policy 0, policy_version 5782 (0.0007) [2023-10-07 19:52:38,922][67838] Updated weights for policy 0, policy_version 5792 (0.0007) [2023-10-07 19:52:41,203][67871] Updated weights for policy 1, policy_version 5800 (0.0008) [2023-10-07 19:52:41,588][67871] Updated weights for policy 1, policy_version 5810 (0.0011) [2023-10-07 19:52:41,958][67871] Updated weights for policy 1, policy_version 5820 (0.0010) [2023-10-07 19:52:42,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 11894784. Throughput: 0: 1637.9, 1: 1648.2. Samples: 2977516. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-07 19:52:42,478][66916] Avg episode reward: [(0, '27.920'), (1, '27.650')] [2023-10-07 19:52:43,129][67838] Updated weights for policy 0, policy_version 5802 (0.0010) [2023-10-07 19:52:43,512][67838] Updated weights for policy 0, policy_version 5812 (0.0007) [2023-10-07 19:52:43,884][67838] Updated weights for policy 0, policy_version 5822 (0.0009) [2023-10-07 19:52:45,968][67871] Updated weights for policy 1, policy_version 5830 (0.0009) [2023-10-07 19:52:46,334][67871] Updated weights for policy 1, policy_version 5840 (0.0008) [2023-10-07 19:52:46,712][67871] Updated weights for policy 1, policy_version 5850 (0.0009) [2023-10-07 19:52:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 11960320. Throughput: 0: 1632.7, 1: 1648.0. Samples: 2997356. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-07 19:52:47,477][66916] Avg episode reward: [(0, '27.690'), (1, '27.820')] [2023-10-07 19:52:48,185][67838] Updated weights for policy 0, policy_version 5832 (0.0009) [2023-10-07 19:52:48,557][67838] Updated weights for policy 0, policy_version 5842 (0.0010) [2023-10-07 19:52:48,936][67838] Updated weights for policy 0, policy_version 5852 (0.0011) [2023-10-07 19:52:50,973][67871] Updated weights for policy 1, policy_version 5860 (0.0007) [2023-10-07 19:52:51,332][67871] Updated weights for policy 1, policy_version 5870 (0.0007) [2023-10-07 19:52:51,708][67871] Updated weights for policy 1, policy_version 5880 (0.0009) [2023-10-07 19:52:52,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 12025856. Throughput: 0: 1628.1, 1: 1639.2. Samples: 3016312. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-07 19:52:52,478][66916] Avg episode reward: [(0, '28.000'), (1, '27.440')] [2023-10-07 19:52:53,153][67838] Updated weights for policy 0, policy_version 5862 (0.0009) [2023-10-07 19:52:53,523][67838] Updated weights for policy 0, policy_version 5872 (0.0008) [2023-10-07 19:52:53,899][67838] Updated weights for policy 0, policy_version 5882 (0.0007) [2023-10-07 19:52:55,933][67871] Updated weights for policy 1, policy_version 5890 (0.0010) [2023-10-07 19:52:56,314][67871] Updated weights for policy 1, policy_version 5900 (0.0010) [2023-10-07 19:52:56,678][67871] Updated weights for policy 1, policy_version 5910 (0.0011) [2023-10-07 19:52:57,049][67871] Updated weights for policy 1, policy_version 5920 (0.0010) [2023-10-07 19:52:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12091392. Throughput: 0: 1628.8, 1: 1648.2. Samples: 3026266. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:52:57,478][66916] Avg episode reward: [(0, '27.120'), (1, '27.380')] [2023-10-07 19:52:58,036][67838] Updated weights for policy 0, policy_version 5892 (0.0009) [2023-10-07 19:52:58,410][67838] Updated weights for policy 0, policy_version 5902 (0.0007) [2023-10-07 19:52:58,795][67838] Updated weights for policy 0, policy_version 5912 (0.0010) [2023-10-07 19:53:01,252][67871] Updated weights for policy 1, policy_version 5930 (0.0007) [2023-10-07 19:53:01,614][67871] Updated weights for policy 1, policy_version 5940 (0.0011) [2023-10-07 19:53:01,989][67871] Updated weights for policy 1, policy_version 5950 (0.0010) [2023-10-07 19:53:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12156928. Throughput: 0: 1635.6, 1: 1643.0. Samples: 3046292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:53:02,478][66916] Avg episode reward: [(0, '27.840'), (1, '25.990')] [2023-10-07 19:53:02,856][67838] Updated weights for policy 0, policy_version 5922 (0.0007) [2023-10-07 19:53:03,233][67838] Updated weights for policy 0, policy_version 5932 (0.0007) [2023-10-07 19:53:03,601][67838] Updated weights for policy 0, policy_version 5942 (0.0007) [2023-10-07 19:53:03,973][67838] Updated weights for policy 0, policy_version 5952 (0.0010) [2023-10-07 19:53:06,354][67871] Updated weights for policy 1, policy_version 5960 (0.0010) [2023-10-07 19:53:06,727][67871] Updated weights for policy 1, policy_version 5970 (0.0011) [2023-10-07 19:53:07,095][67871] Updated weights for policy 1, policy_version 5980 (0.0008) [2023-10-07 19:53:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 12222464. Throughput: 0: 1642.1, 1: 1640.7. Samples: 3065702. Policy #0 lag: (min: 17.0, avg: 28.1, max: 49.0) [2023-10-07 19:53:07,477][66916] Avg episode reward: [(0, '27.120'), (1, '26.270')] [2023-10-07 19:53:07,485][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000005984_6127616.pth... [2023-10-07 19:53:07,521][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000004448_4554752.pth [2023-10-07 19:53:07,913][67838] Updated weights for policy 0, policy_version 5962 (0.0007) [2023-10-07 19:53:08,285][67838] Updated weights for policy 0, policy_version 5972 (0.0007) [2023-10-07 19:53:08,667][67838] Updated weights for policy 0, policy_version 5982 (0.0007) [2023-10-07 19:53:08,735][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000005984_6127616.pth... [2023-10-07 19:53:08,763][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000004448_4554752.pth [2023-10-07 19:53:11,216][67871] Updated weights for policy 1, policy_version 5990 (0.0008) [2023-10-07 19:53:11,587][67871] Updated weights for policy 1, policy_version 6000 (0.0011) [2023-10-07 19:53:11,962][67871] Updated weights for policy 1, policy_version 6010 (0.0011) [2023-10-07 19:53:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12288000. Throughput: 0: 1643.4, 1: 1633.7. Samples: 3075320. Policy #0 lag: (min: 17.0, avg: 28.1, max: 49.0) [2023-10-07 19:53:12,477][66916] Avg episode reward: [(0, '27.100'), (1, '27.360')] [2023-10-07 19:53:12,878][67838] Updated weights for policy 0, policy_version 5992 (0.0009) [2023-10-07 19:53:13,253][67838] Updated weights for policy 0, policy_version 6002 (0.0010) [2023-10-07 19:53:13,622][67838] Updated weights for policy 0, policy_version 6012 (0.0010) [2023-10-07 19:53:16,269][67871] Updated weights for policy 1, policy_version 6020 (0.0009) [2023-10-07 19:53:16,639][67871] Updated weights for policy 1, policy_version 6030 (0.0007) [2023-10-07 19:53:17,008][67871] Updated weights for policy 1, policy_version 6040 (0.0007) [2023-10-07 19:53:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12353536. Throughput: 0: 1638.7, 1: 1636.5. Samples: 3095298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:53:17,478][66916] Avg episode reward: [(0, '27.740'), (1, '26.790')] [2023-10-07 19:53:17,755][67838] Updated weights for policy 0, policy_version 6022 (0.0009) [2023-10-07 19:53:18,129][67838] Updated weights for policy 0, policy_version 6032 (0.0009) [2023-10-07 19:53:18,505][67838] Updated weights for policy 0, policy_version 6042 (0.0011) [2023-10-07 19:53:21,242][67871] Updated weights for policy 1, policy_version 6050 (0.0007) [2023-10-07 19:53:21,606][67871] Updated weights for policy 1, policy_version 6060 (0.0009) [2023-10-07 19:53:21,971][67871] Updated weights for policy 1, policy_version 6070 (0.0009) [2023-10-07 19:53:22,344][67871] Updated weights for policy 1, policy_version 6080 (0.0010) [2023-10-07 19:53:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 12419072. Throughput: 0: 1633.5, 1: 1638.4. Samples: 3114674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:53:22,477][66916] Avg episode reward: [(0, '27.450'), (1, '27.790')] [2023-10-07 19:53:22,792][67838] Updated weights for policy 0, policy_version 6052 (0.0009) [2023-10-07 19:53:23,170][67838] Updated weights for policy 0, policy_version 6062 (0.0007) [2023-10-07 19:53:23,545][67838] Updated weights for policy 0, policy_version 6072 (0.0008) [2023-10-07 19:53:26,565][67871] Updated weights for policy 1, policy_version 6090 (0.0009) [2023-10-07 19:53:26,934][67871] Updated weights for policy 1, policy_version 6100 (0.0008) [2023-10-07 19:53:27,311][67871] Updated weights for policy 1, policy_version 6110 (0.0008) [2023-10-07 19:53:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12484608. Throughput: 0: 1630.1, 1: 1629.2. Samples: 3124188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:53:27,477][66916] Avg episode reward: [(0, '28.600'), (1, '26.170')] [2023-10-07 19:53:27,789][67838] Updated weights for policy 0, policy_version 6082 (0.0008) [2023-10-07 19:53:28,197][67838] Updated weights for policy 0, policy_version 6092 (0.0010) [2023-10-07 19:53:28,571][67838] Updated weights for policy 0, policy_version 6102 (0.0010) [2023-10-07 19:53:28,940][67838] Updated weights for policy 0, policy_version 6112 (0.0010) [2023-10-07 19:53:31,267][67871] Updated weights for policy 1, policy_version 6120 (0.0009) [2023-10-07 19:53:31,626][67871] Updated weights for policy 1, policy_version 6130 (0.0009) [2023-10-07 19:53:32,005][67871] Updated weights for policy 1, policy_version 6140 (0.0011) [2023-10-07 19:53:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12550144. Throughput: 0: 1635.7, 1: 1632.7. Samples: 3144434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:53:32,478][66916] Avg episode reward: [(0, '28.970'), (1, '28.370')] [2023-10-07 19:53:32,479][67676] Saving new best policy, reward=28.370! [2023-10-07 19:53:32,479][67511] Saving new best policy, reward=28.970! [2023-10-07 19:53:33,149][67838] Updated weights for policy 0, policy_version 6122 (0.0010) [2023-10-07 19:53:33,523][67838] Updated weights for policy 0, policy_version 6132 (0.0009) [2023-10-07 19:53:33,890][67838] Updated weights for policy 0, policy_version 6142 (0.0009) [2023-10-07 19:53:36,275][67871] Updated weights for policy 1, policy_version 6150 (0.0009) [2023-10-07 19:53:36,638][67871] Updated weights for policy 1, policy_version 6160 (0.0008) [2023-10-07 19:53:37,009][67871] Updated weights for policy 1, policy_version 6170 (0.0008) [2023-10-07 19:53:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12615680. Throughput: 0: 1638.0, 1: 1638.4. Samples: 3163748. Policy #0 lag: (min: 10.0, avg: 12.2, max: 42.0) [2023-10-07 19:53:37,477][66916] Avg episode reward: [(0, '28.480'), (1, '28.500')] [2023-10-07 19:53:37,485][67676] Saving new best policy, reward=28.500! [2023-10-07 19:53:38,054][67838] Updated weights for policy 0, policy_version 6152 (0.0008) [2023-10-07 19:53:38,415][67838] Updated weights for policy 0, policy_version 6162 (0.0011) [2023-10-07 19:53:38,799][67838] Updated weights for policy 0, policy_version 6172 (0.0011) [2023-10-07 19:53:41,279][67871] Updated weights for policy 1, policy_version 6180 (0.0008) [2023-10-07 19:53:41,646][67871] Updated weights for policy 1, policy_version 6190 (0.0007) [2023-10-07 19:53:42,021][67871] Updated weights for policy 1, policy_version 6200 (0.0008) [2023-10-07 19:53:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12681216. Throughput: 0: 1634.2, 1: 1631.3. Samples: 3173212. Policy #0 lag: (min: 10.0, avg: 12.2, max: 42.0) [2023-10-07 19:53:42,477][66916] Avg episode reward: [(0, '29.060'), (1, '27.130')] [2023-10-07 19:53:42,478][67511] Saving new best policy, reward=29.060! [2023-10-07 19:53:43,145][67838] Updated weights for policy 0, policy_version 6182 (0.0009) [2023-10-07 19:53:43,514][67838] Updated weights for policy 0, policy_version 6192 (0.0009) [2023-10-07 19:53:43,894][67838] Updated weights for policy 0, policy_version 6202 (0.0008) [2023-10-07 19:53:46,197][67871] Updated weights for policy 1, policy_version 6210 (0.0008) [2023-10-07 19:53:46,561][67871] Updated weights for policy 1, policy_version 6220 (0.0007) [2023-10-07 19:53:46,934][67871] Updated weights for policy 1, policy_version 6230 (0.0009) [2023-10-07 19:53:47,305][67871] Updated weights for policy 1, policy_version 6240 (0.0007) [2023-10-07 19:53:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12746752. Throughput: 0: 1637.1, 1: 1631.4. Samples: 3193372. Policy #0 lag: (min: 30.0, avg: 36.6, max: 62.0) [2023-10-07 19:53:47,477][66916] Avg episode reward: [(0, '29.360'), (1, '26.630')] [2023-10-07 19:53:47,478][67511] Saving new best policy, reward=29.360! [2023-10-07 19:53:47,934][67838] Updated weights for policy 0, policy_version 6212 (0.0009) [2023-10-07 19:53:48,309][67838] Updated weights for policy 0, policy_version 6222 (0.0011) [2023-10-07 19:53:48,683][67838] Updated weights for policy 0, policy_version 6232 (0.0010) [2023-10-07 19:53:51,568][67871] Updated weights for policy 1, policy_version 6250 (0.0009) [2023-10-07 19:53:51,938][67871] Updated weights for policy 1, policy_version 6260 (0.0008) [2023-10-07 19:53:52,316][67871] Updated weights for policy 1, policy_version 6270 (0.0007) [2023-10-07 19:53:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12812288. Throughput: 0: 1634.6, 1: 1638.6. Samples: 3212996. Policy #0 lag: (min: 30.0, avg: 36.6, max: 62.0) [2023-10-07 19:53:52,478][66916] Avg episode reward: [(0, '28.760'), (1, '26.560')] [2023-10-07 19:53:52,835][67838] Updated weights for policy 0, policy_version 6242 (0.0009) [2023-10-07 19:53:53,208][67838] Updated weights for policy 0, policy_version 6252 (0.0007) [2023-10-07 19:53:53,586][67838] Updated weights for policy 0, policy_version 6262 (0.0008) [2023-10-07 19:53:53,960][67838] Updated weights for policy 0, policy_version 6272 (0.0007) [2023-10-07 19:53:56,540][67871] Updated weights for policy 1, policy_version 6280 (0.0009) [2023-10-07 19:53:56,916][67871] Updated weights for policy 1, policy_version 6290 (0.0010) [2023-10-07 19:53:57,284][67871] Updated weights for policy 1, policy_version 6300 (0.0008) [2023-10-07 19:53:57,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12877824. Throughput: 0: 1632.9, 1: 1637.5. Samples: 3222488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:53:57,477][66916] Avg episode reward: [(0, '28.390'), (1, '26.780')] [2023-10-07 19:53:58,146][67838] Updated weights for policy 0, policy_version 6282 (0.0008) [2023-10-07 19:53:58,519][67838] Updated weights for policy 0, policy_version 6292 (0.0007) [2023-10-07 19:53:58,895][67838] Updated weights for policy 0, policy_version 6302 (0.0007) [2023-10-07 19:54:01,303][67871] Updated weights for policy 1, policy_version 6310 (0.0009) [2023-10-07 19:54:01,678][67871] Updated weights for policy 1, policy_version 6320 (0.0008) [2023-10-07 19:54:02,049][67871] Updated weights for policy 1, policy_version 6330 (0.0009) [2023-10-07 19:54:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12943360. Throughput: 0: 1636.2, 1: 1636.3. Samples: 3242562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:54:02,477][66916] Avg episode reward: [(0, '28.410'), (1, '26.910')] [2023-10-07 19:54:02,977][67838] Updated weights for policy 0, policy_version 6312 (0.0009) [2023-10-07 19:54:03,364][67838] Updated weights for policy 0, policy_version 6322 (0.0009) [2023-10-07 19:54:03,739][67838] Updated weights for policy 0, policy_version 6332 (0.0008) [2023-10-07 19:54:06,190][67871] Updated weights for policy 1, policy_version 6340 (0.0010) [2023-10-07 19:54:06,559][67871] Updated weights for policy 1, policy_version 6350 (0.0011) [2023-10-07 19:54:06,929][67871] Updated weights for policy 1, policy_version 6360 (0.0008) [2023-10-07 19:54:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13008896. Throughput: 0: 1643.6, 1: 1632.5. Samples: 3262100. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-07 19:54:07,478][66916] Avg episode reward: [(0, '27.130'), (1, '27.330')] [2023-10-07 19:54:07,740][67838] Updated weights for policy 0, policy_version 6342 (0.0009) [2023-10-07 19:54:08,120][67838] Updated weights for policy 0, policy_version 6352 (0.0008) [2023-10-07 19:54:08,493][67838] Updated weights for policy 0, policy_version 6362 (0.0009) [2023-10-07 19:54:10,932][67871] Updated weights for policy 1, policy_version 6370 (0.0007) [2023-10-07 19:54:11,319][67871] Updated weights for policy 1, policy_version 6380 (0.0008) [2023-10-07 19:54:11,677][67871] Updated weights for policy 1, policy_version 6390 (0.0008) [2023-10-07 19:54:12,048][67871] Updated weights for policy 1, policy_version 6400 (0.0007) [2023-10-07 19:54:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13074432. Throughput: 0: 1646.2, 1: 1635.6. Samples: 3271872. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-07 19:54:12,477][66916] Avg episode reward: [(0, '28.810'), (1, '28.160')] [2023-10-07 19:54:12,688][67838] Updated weights for policy 0, policy_version 6372 (0.0009) [2023-10-07 19:54:13,056][67838] Updated weights for policy 0, policy_version 6382 (0.0009) [2023-10-07 19:54:13,424][67838] Updated weights for policy 0, policy_version 6392 (0.0010) [2023-10-07 19:54:16,151][67871] Updated weights for policy 1, policy_version 6410 (0.0009) [2023-10-07 19:54:16,517][67871] Updated weights for policy 1, policy_version 6420 (0.0008) [2023-10-07 19:54:16,887][67871] Updated weights for policy 1, policy_version 6430 (0.0007) [2023-10-07 19:54:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 13139968. Throughput: 0: 1646.2, 1: 1634.5. Samples: 3292066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:54:17,477][66916] Avg episode reward: [(0, '28.830'), (1, '27.410')] [2023-10-07 19:54:17,701][67838] Updated weights for policy 0, policy_version 6402 (0.0009) [2023-10-07 19:54:18,094][67838] Updated weights for policy 0, policy_version 6412 (0.0009) [2023-10-07 19:54:18,462][67838] Updated weights for policy 0, policy_version 6422 (0.0008) [2023-10-07 19:54:18,835][67838] Updated weights for policy 0, policy_version 6432 (0.0008) [2023-10-07 19:54:21,165][67871] Updated weights for policy 1, policy_version 6440 (0.0007) [2023-10-07 19:54:21,532][67871] Updated weights for policy 1, policy_version 6450 (0.0008) [2023-10-07 19:54:21,890][67871] Updated weights for policy 1, policy_version 6460 (0.0008) [2023-10-07 19:54:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 13205504. Throughput: 0: 1644.8, 1: 1633.8. Samples: 3311284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:54:22,478][66916] Avg episode reward: [(0, '28.520'), (1, '28.470')] [2023-10-07 19:54:22,993][67838] Updated weights for policy 0, policy_version 6442 (0.0008) [2023-10-07 19:54:23,365][67838] Updated weights for policy 0, policy_version 6452 (0.0008) [2023-10-07 19:54:23,748][67838] Updated weights for policy 0, policy_version 6462 (0.0008) [2023-10-07 19:54:26,193][67871] Updated weights for policy 1, policy_version 6470 (0.0010) [2023-10-07 19:54:26,559][67871] Updated weights for policy 1, policy_version 6480 (0.0008) [2023-10-07 19:54:26,940][67871] Updated weights for policy 1, policy_version 6490 (0.0007) [2023-10-07 19:54:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13271040. Throughput: 0: 1645.5, 1: 1639.0. Samples: 3321014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:54:27,477][66916] Avg episode reward: [(0, '28.830'), (1, '27.040')] [2023-10-07 19:54:27,792][67838] Updated weights for policy 0, policy_version 6472 (0.0009) [2023-10-07 19:54:28,173][67838] Updated weights for policy 0, policy_version 6482 (0.0009) [2023-10-07 19:54:28,542][67838] Updated weights for policy 0, policy_version 6492 (0.0009) [2023-10-07 19:54:31,147][67871] Updated weights for policy 1, policy_version 6500 (0.0009) [2023-10-07 19:54:31,512][67871] Updated weights for policy 1, policy_version 6510 (0.0010) [2023-10-07 19:54:31,887][67871] Updated weights for policy 1, policy_version 6520 (0.0010) [2023-10-07 19:54:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13336576. Throughput: 0: 1640.1, 1: 1643.1. Samples: 3341114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:54:32,477][66916] Avg episode reward: [(0, '28.680'), (1, '27.580')] [2023-10-07 19:54:32,748][67838] Updated weights for policy 0, policy_version 6502 (0.0009) [2023-10-07 19:54:33,137][67838] Updated weights for policy 0, policy_version 6512 (0.0009) [2023-10-07 19:54:33,507][67838] Updated weights for policy 0, policy_version 6522 (0.0009) [2023-10-07 19:54:36,130][67871] Updated weights for policy 1, policy_version 6530 (0.0010) [2023-10-07 19:54:36,494][67871] Updated weights for policy 1, policy_version 6540 (0.0008) [2023-10-07 19:54:36,864][67871] Updated weights for policy 1, policy_version 6550 (0.0008) [2023-10-07 19:54:37,235][67871] Updated weights for policy 1, policy_version 6560 (0.0010) [2023-10-07 19:54:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13402112. Throughput: 0: 1635.6, 1: 1639.4. Samples: 3360372. Policy #0 lag: (min: 0.0, avg: 23.2, max: 32.0) [2023-10-07 19:54:37,477][66916] Avg episode reward: [(0, '27.820'), (1, '26.980')] [2023-10-07 19:54:37,750][67838] Updated weights for policy 0, policy_version 6532 (0.0008) [2023-10-07 19:54:38,129][67838] Updated weights for policy 0, policy_version 6542 (0.0009) [2023-10-07 19:54:38,498][67838] Updated weights for policy 0, policy_version 6552 (0.0009) [2023-10-07 19:54:41,567][67871] Updated weights for policy 1, policy_version 6570 (0.0009) [2023-10-07 19:54:41,919][67871] Updated weights for policy 1, policy_version 6580 (0.0008) [2023-10-07 19:54:42,292][67871] Updated weights for policy 1, policy_version 6590 (0.0008) [2023-10-07 19:54:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13467648. Throughput: 0: 1638.5, 1: 1641.9. Samples: 3370106. Policy #0 lag: (min: 0.0, avg: 23.2, max: 32.0) [2023-10-07 19:54:42,477][66916] Avg episode reward: [(0, '28.240'), (1, '26.640')] [2023-10-07 19:54:42,663][67838] Updated weights for policy 0, policy_version 6562 (0.0010) [2023-10-07 19:54:43,040][67838] Updated weights for policy 0, policy_version 6572 (0.0009) [2023-10-07 19:54:43,409][67838] Updated weights for policy 0, policy_version 6582 (0.0008) [2023-10-07 19:54:43,781][67838] Updated weights for policy 0, policy_version 6592 (0.0007) [2023-10-07 19:54:46,305][67871] Updated weights for policy 1, policy_version 6600 (0.0010) [2023-10-07 19:54:46,676][67871] Updated weights for policy 1, policy_version 6610 (0.0009) [2023-10-07 19:54:47,044][67871] Updated weights for policy 1, policy_version 6620 (0.0008) [2023-10-07 19:54:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13533184. Throughput: 0: 1637.6, 1: 1641.0. Samples: 3390096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:54:47,478][66916] Avg episode reward: [(0, '28.510'), (1, '28.030')] [2023-10-07 19:54:47,943][67838] Updated weights for policy 0, policy_version 6602 (0.0010) [2023-10-07 19:54:48,312][67838] Updated weights for policy 0, policy_version 6612 (0.0010) [2023-10-07 19:54:48,686][67838] Updated weights for policy 0, policy_version 6622 (0.0008) [2023-10-07 19:54:51,280][67871] Updated weights for policy 1, policy_version 6630 (0.0010) [2023-10-07 19:54:51,647][67871] Updated weights for policy 1, policy_version 6640 (0.0007) [2023-10-07 19:54:52,018][67871] Updated weights for policy 1, policy_version 6650 (0.0007) [2023-10-07 19:54:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13598720. Throughput: 0: 1637.1, 1: 1638.9. Samples: 3409522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:54:52,477][66916] Avg episode reward: [(0, '29.960'), (1, '27.650')] [2023-10-07 19:54:52,487][67511] Saving new best policy, reward=29.960! [2023-10-07 19:54:52,961][67838] Updated weights for policy 0, policy_version 6632 (0.0009) [2023-10-07 19:54:53,337][67838] Updated weights for policy 0, policy_version 6642 (0.0011) [2023-10-07 19:54:53,703][67838] Updated weights for policy 0, policy_version 6652 (0.0008) [2023-10-07 19:54:56,222][67871] Updated weights for policy 1, policy_version 6660 (0.0009) [2023-10-07 19:54:56,613][67871] Updated weights for policy 1, policy_version 6670 (0.0010) [2023-10-07 19:54:56,980][67871] Updated weights for policy 1, policy_version 6680 (0.0008) [2023-10-07 19:54:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 13664256. Throughput: 0: 1635.4, 1: 1639.6. Samples: 3419246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:54:57,477][66916] Avg episode reward: [(0, '29.050'), (1, '27.080')] [2023-10-07 19:54:57,915][67838] Updated weights for policy 0, policy_version 6662 (0.0008) [2023-10-07 19:54:58,283][67838] Updated weights for policy 0, policy_version 6672 (0.0008) [2023-10-07 19:54:58,659][67838] Updated weights for policy 0, policy_version 6682 (0.0009) [2023-10-07 19:55:01,032][67871] Updated weights for policy 1, policy_version 6690 (0.0008) [2023-10-07 19:55:01,392][67871] Updated weights for policy 1, policy_version 6700 (0.0010) [2023-10-07 19:55:01,759][67871] Updated weights for policy 1, policy_version 6710 (0.0008) [2023-10-07 19:55:02,129][67871] Updated weights for policy 1, policy_version 6720 (0.0008) [2023-10-07 19:55:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13729792. Throughput: 0: 1631.6, 1: 1641.7. Samples: 3439362. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:55:02,477][66916] Avg episode reward: [(0, '28.600'), (1, '27.220')] [2023-10-07 19:55:03,061][67838] Updated weights for policy 0, policy_version 6692 (0.0009) [2023-10-07 19:55:03,462][67838] Updated weights for policy 0, policy_version 6702 (0.0009) [2023-10-07 19:55:03,837][67838] Updated weights for policy 0, policy_version 6712 (0.0009) [2023-10-07 19:55:06,427][67871] Updated weights for policy 1, policy_version 6730 (0.0008) [2023-10-07 19:55:06,795][67871] Updated weights for policy 1, policy_version 6740 (0.0008) [2023-10-07 19:55:07,170][67871] Updated weights for policy 1, policy_version 6750 (0.0009) [2023-10-07 19:55:07,477][66916] Fps is (10 sec: 13106.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13795328. Throughput: 0: 1631.0, 1: 1642.8. Samples: 3458604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:55:07,478][66916] Avg episode reward: [(0, '30.110'), (1, '27.840')] [2023-10-07 19:55:07,492][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000006720_6881280.pth... [2023-10-07 19:55:07,492][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000006752_6914048.pth... [2023-10-07 19:55:07,542][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000005216_5341184.pth [2023-10-07 19:55:07,542][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000005216_5341184.pth [2023-10-07 19:55:07,547][67511] Saving new best policy, reward=30.110! [2023-10-07 19:55:08,085][67838] Updated weights for policy 0, policy_version 6722 (0.0009) [2023-10-07 19:55:08,456][67838] Updated weights for policy 0, policy_version 6732 (0.0009) [2023-10-07 19:55:08,831][67838] Updated weights for policy 0, policy_version 6742 (0.0010) [2023-10-07 19:55:09,205][67838] Updated weights for policy 0, policy_version 6752 (0.0009) [2023-10-07 19:55:11,247][67871] Updated weights for policy 1, policy_version 6760 (0.0010) [2023-10-07 19:55:11,618][67871] Updated weights for policy 1, policy_version 6770 (0.0007) [2023-10-07 19:55:11,981][67871] Updated weights for policy 1, policy_version 6780 (0.0007) [2023-10-07 19:55:12,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13860864. Throughput: 0: 1631.8, 1: 1638.2. Samples: 3468162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:55:12,478][66916] Avg episode reward: [(0, '28.230'), (1, '27.920')] [2023-10-07 19:55:13,240][67838] Updated weights for policy 0, policy_version 6762 (0.0009) [2023-10-07 19:55:13,607][67838] Updated weights for policy 0, policy_version 6772 (0.0008) [2023-10-07 19:55:13,989][67838] Updated weights for policy 0, policy_version 6782 (0.0009) [2023-10-07 19:55:16,275][67871] Updated weights for policy 1, policy_version 6790 (0.0007) [2023-10-07 19:55:16,655][67871] Updated weights for policy 1, policy_version 6800 (0.0007) [2023-10-07 19:55:17,016][67871] Updated weights for policy 1, policy_version 6810 (0.0007) [2023-10-07 19:55:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13926400. Throughput: 0: 1634.5, 1: 1638.6. Samples: 3488404. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-07 19:55:17,477][66916] Avg episode reward: [(0, '29.040'), (1, '28.590')] [2023-10-07 19:55:17,478][67676] Saving new best policy, reward=28.590! [2023-10-07 19:55:17,946][67838] Updated weights for policy 0, policy_version 6792 (0.0008) [2023-10-07 19:55:18,321][67838] Updated weights for policy 0, policy_version 6802 (0.0008) [2023-10-07 19:55:18,693][67838] Updated weights for policy 0, policy_version 6812 (0.0007) [2023-10-07 19:55:21,278][67871] Updated weights for policy 1, policy_version 6820 (0.0007) [2023-10-07 19:55:21,647][67871] Updated weights for policy 1, policy_version 6830 (0.0007) [2023-10-07 19:55:22,023][67871] Updated weights for policy 1, policy_version 6840 (0.0007) [2023-10-07 19:55:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13991936. Throughput: 0: 1635.9, 1: 1639.4. Samples: 3507758. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-07 19:55:22,478][66916] Avg episode reward: [(0, '29.280'), (1, '28.800')] [2023-10-07 19:55:22,490][67676] Saving new best policy, reward=28.800! [2023-10-07 19:55:22,868][67838] Updated weights for policy 0, policy_version 6822 (0.0010) [2023-10-07 19:55:23,247][67838] Updated weights for policy 0, policy_version 6832 (0.0009) [2023-10-07 19:55:23,609][67838] Updated weights for policy 0, policy_version 6842 (0.0009) [2023-10-07 19:55:26,214][67871] Updated weights for policy 1, policy_version 6850 (0.0007) [2023-10-07 19:55:26,587][67871] Updated weights for policy 1, policy_version 6860 (0.0008) [2023-10-07 19:55:26,963][67871] Updated weights for policy 1, policy_version 6870 (0.0009) [2023-10-07 19:55:27,336][67871] Updated weights for policy 1, policy_version 6880 (0.0009) [2023-10-07 19:55:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 14057472. Throughput: 0: 1633.1, 1: 1636.6. Samples: 3517242. Policy #0 lag: (min: 14.0, avg: 18.5, max: 46.0) [2023-10-07 19:55:27,478][66916] Avg episode reward: [(0, '27.640'), (1, '27.490')] [2023-10-07 19:55:27,842][67838] Updated weights for policy 0, policy_version 6852 (0.0010) [2023-10-07 19:55:28,214][67838] Updated weights for policy 0, policy_version 6862 (0.0009) [2023-10-07 19:55:28,593][67838] Updated weights for policy 0, policy_version 6872 (0.0007) [2023-10-07 19:55:31,479][67871] Updated weights for policy 1, policy_version 6890 (0.0011) [2023-10-07 19:55:31,848][67871] Updated weights for policy 1, policy_version 6900 (0.0007) [2023-10-07 19:55:32,224][67871] Updated weights for policy 1, policy_version 6910 (0.0007) [2023-10-07 19:55:32,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 14123008. Throughput: 0: 1634.9, 1: 1638.5. Samples: 3537396. Policy #0 lag: (min: 14.0, avg: 18.5, max: 46.0) [2023-10-07 19:55:32,477][66916] Avg episode reward: [(0, '30.130'), (1, '29.140')] [2023-10-07 19:55:32,477][67676] Saving new best policy, reward=29.140! [2023-10-07 19:55:32,478][67511] Saving new best policy, reward=30.130! [2023-10-07 19:55:32,834][67838] Updated weights for policy 0, policy_version 6882 (0.0009) [2023-10-07 19:55:33,205][67838] Updated weights for policy 0, policy_version 6892 (0.0007) [2023-10-07 19:55:33,579][67838] Updated weights for policy 0, policy_version 6902 (0.0009) [2023-10-07 19:55:33,953][67838] Updated weights for policy 0, policy_version 6912 (0.0012) [2023-10-07 19:55:36,169][67871] Updated weights for policy 1, policy_version 6920 (0.0008) [2023-10-07 19:55:36,543][67871] Updated weights for policy 1, policy_version 6930 (0.0010) [2023-10-07 19:55:36,927][67871] Updated weights for policy 1, policy_version 6940 (0.0011) [2023-10-07 19:55:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 14188544. Throughput: 0: 1629.4, 1: 1641.4. Samples: 3556710. Policy #0 lag: (min: 24.0, avg: 46.4, max: 56.0) [2023-10-07 19:55:37,477][66916] Avg episode reward: [(0, '28.490'), (1, '27.880')] [2023-10-07 19:55:38,207][67838] Updated weights for policy 0, policy_version 6922 (0.0010) [2023-10-07 19:55:38,583][67838] Updated weights for policy 0, policy_version 6932 (0.0010) [2023-10-07 19:55:38,956][67838] Updated weights for policy 0, policy_version 6942 (0.0009) [2023-10-07 19:55:41,170][67871] Updated weights for policy 1, policy_version 6950 (0.0009) [2023-10-07 19:55:41,549][67871] Updated weights for policy 1, policy_version 6960 (0.0008) [2023-10-07 19:55:41,929][67871] Updated weights for policy 1, policy_version 6970 (0.0008) [2023-10-07 19:55:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 14254080. Throughput: 0: 1627.0, 1: 1645.0. Samples: 3566486. Policy #0 lag: (min: 24.0, avg: 46.4, max: 56.0) [2023-10-07 19:55:42,478][66916] Avg episode reward: [(0, '28.940'), (1, '27.780')] [2023-10-07 19:55:43,171][67838] Updated weights for policy 0, policy_version 6952 (0.0007) [2023-10-07 19:55:43,547][67838] Updated weights for policy 0, policy_version 6962 (0.0007) [2023-10-07 19:55:43,921][67838] Updated weights for policy 0, policy_version 6972 (0.0008) [2023-10-07 19:55:45,984][67871] Updated weights for policy 1, policy_version 6980 (0.0009) [2023-10-07 19:55:46,347][67871] Updated weights for policy 1, policy_version 6990 (0.0010) [2023-10-07 19:55:46,712][67871] Updated weights for policy 1, policy_version 7000 (0.0007) [2023-10-07 19:55:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 14319616. Throughput: 0: 1627.7, 1: 1640.3. Samples: 3586422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:55:47,477][66916] Avg episode reward: [(0, '29.030'), (1, '29.030')] [2023-10-07 19:55:48,211][67838] Updated weights for policy 0, policy_version 6982 (0.0008) [2023-10-07 19:55:48,588][67838] Updated weights for policy 0, policy_version 6992 (0.0010) [2023-10-07 19:55:48,960][67838] Updated weights for policy 0, policy_version 7002 (0.0007) [2023-10-07 19:55:50,768][67871] Updated weights for policy 1, policy_version 7010 (0.0007) [2023-10-07 19:55:51,144][67871] Updated weights for policy 1, policy_version 7020 (0.0007) [2023-10-07 19:55:51,505][67871] Updated weights for policy 1, policy_version 7030 (0.0009) [2023-10-07 19:55:51,878][67871] Updated weights for policy 1, policy_version 7040 (0.0010) [2023-10-07 19:55:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 14385152. Throughput: 0: 1630.4, 1: 1639.9. Samples: 3605764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:55:52,477][66916] Avg episode reward: [(0, '29.550'), (1, '28.650')] [2023-10-07 19:55:52,963][67838] Updated weights for policy 0, policy_version 7012 (0.0008) [2023-10-07 19:55:53,331][67838] Updated weights for policy 0, policy_version 7022 (0.0008) [2023-10-07 19:55:53,708][67838] Updated weights for policy 0, policy_version 7032 (0.0008) [2023-10-07 19:55:55,969][67871] Updated weights for policy 1, policy_version 7050 (0.0009) [2023-10-07 19:55:56,341][67871] Updated weights for policy 1, policy_version 7060 (0.0011) [2023-10-07 19:55:56,712][67871] Updated weights for policy 1, policy_version 7070 (0.0010) [2023-10-07 19:55:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 14450688. Throughput: 0: 1633.9, 1: 1651.8. Samples: 3616018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:55:57,478][66916] Avg episode reward: [(0, '29.370'), (1, '27.600')] [2023-10-07 19:55:57,960][67838] Updated weights for policy 0, policy_version 7042 (0.0011) [2023-10-07 19:55:58,330][67838] Updated weights for policy 0, policy_version 7052 (0.0007) [2023-10-07 19:55:58,709][67838] Updated weights for policy 0, policy_version 7062 (0.0009) [2023-10-07 19:55:59,085][67838] Updated weights for policy 0, policy_version 7072 (0.0007) [2023-10-07 19:56:01,086][67871] Updated weights for policy 1, policy_version 7080 (0.0011) [2023-10-07 19:56:01,470][67871] Updated weights for policy 1, policy_version 7090 (0.0011) [2023-10-07 19:56:01,833][67871] Updated weights for policy 1, policy_version 7100 (0.0009) [2023-10-07 19:56:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 14516224. Throughput: 0: 1632.0, 1: 1644.7. Samples: 3635858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:56:02,477][66916] Avg episode reward: [(0, '30.310'), (1, '28.310')] [2023-10-07 19:56:02,478][67511] Saving new best policy, reward=30.310! [2023-10-07 19:56:03,172][67838] Updated weights for policy 0, policy_version 7082 (0.0009) [2023-10-07 19:56:03,548][67838] Updated weights for policy 0, policy_version 7092 (0.0009) [2023-10-07 19:56:03,922][67838] Updated weights for policy 0, policy_version 7102 (0.0008) [2023-10-07 19:56:05,961][67871] Updated weights for policy 1, policy_version 7110 (0.0010) [2023-10-07 19:56:06,331][67871] Updated weights for policy 1, policy_version 7120 (0.0007) [2023-10-07 19:56:06,694][67871] Updated weights for policy 1, policy_version 7130 (0.0007) [2023-10-07 19:56:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 14581760. Throughput: 0: 1631.4, 1: 1645.5. Samples: 3655220. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-07 19:56:07,477][66916] Avg episode reward: [(0, '29.230'), (1, '28.630')] [2023-10-07 19:56:08,249][67838] Updated weights for policy 0, policy_version 7112 (0.0009) [2023-10-07 19:56:08,630][67838] Updated weights for policy 0, policy_version 7122 (0.0007) [2023-10-07 19:56:09,000][67838] Updated weights for policy 0, policy_version 7132 (0.0007) [2023-10-07 19:56:10,984][67871] Updated weights for policy 1, policy_version 7140 (0.0008) [2023-10-07 19:56:11,353][67871] Updated weights for policy 1, policy_version 7150 (0.0007) [2023-10-07 19:56:11,721][67871] Updated weights for policy 1, policy_version 7160 (0.0007) [2023-10-07 19:56:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 14647296. Throughput: 0: 1633.5, 1: 1656.6. Samples: 3665296. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-07 19:56:12,477][66916] Avg episode reward: [(0, '30.200'), (1, '28.180')] [2023-10-07 19:56:13,203][67838] Updated weights for policy 0, policy_version 7142 (0.0007) [2023-10-07 19:56:13,584][67838] Updated weights for policy 0, policy_version 7152 (0.0008) [2023-10-07 19:56:13,952][67838] Updated weights for policy 0, policy_version 7162 (0.0009) [2023-10-07 19:56:15,613][67871] Updated weights for policy 1, policy_version 7170 (0.0008) [2023-10-07 19:56:15,980][67871] Updated weights for policy 1, policy_version 7180 (0.0008) [2023-10-07 19:56:16,345][67871] Updated weights for policy 1, policy_version 7190 (0.0007) [2023-10-07 19:56:16,715][67871] Updated weights for policy 1, policy_version 7200 (0.0008) [2023-10-07 19:56:17,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 14712832. Throughput: 0: 1634.8, 1: 1650.0. Samples: 3685212. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-07 19:56:17,477][66916] Avg episode reward: [(0, '30.200'), (1, '28.760')] [2023-10-07 19:56:18,152][67838] Updated weights for policy 0, policy_version 7172 (0.0009) [2023-10-07 19:56:18,528][67838] Updated weights for policy 0, policy_version 7182 (0.0011) [2023-10-07 19:56:18,902][67838] Updated weights for policy 0, policy_version 7192 (0.0010) [2023-10-07 19:56:20,890][67871] Updated weights for policy 1, policy_version 7210 (0.0011) [2023-10-07 19:56:21,257][67871] Updated weights for policy 1, policy_version 7220 (0.0011) [2023-10-07 19:56:21,625][67871] Updated weights for policy 1, policy_version 7230 (0.0008) [2023-10-07 19:56:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 14778368. Throughput: 0: 1637.7, 1: 1644.6. Samples: 3704414. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-07 19:56:22,477][66916] Avg episode reward: [(0, '29.280'), (1, '29.010')] [2023-10-07 19:56:22,890][67838] Updated weights for policy 0, policy_version 7202 (0.0010) [2023-10-07 19:56:23,274][67838] Updated weights for policy 0, policy_version 7212 (0.0010) [2023-10-07 19:56:23,645][67838] Updated weights for policy 0, policy_version 7222 (0.0008) [2023-10-07 19:56:24,015][67838] Updated weights for policy 0, policy_version 7232 (0.0010) [2023-10-07 19:56:26,074][67871] Updated weights for policy 1, policy_version 7240 (0.0010) [2023-10-07 19:56:26,462][67871] Updated weights for policy 1, policy_version 7250 (0.0007) [2023-10-07 19:56:26,827][67871] Updated weights for policy 1, policy_version 7260 (0.0009) [2023-10-07 19:56:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 14843904. Throughput: 0: 1643.5, 1: 1649.7. Samples: 3714680. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-07 19:56:27,477][66916] Avg episode reward: [(0, '28.850'), (1, '28.430')] [2023-10-07 19:56:28,136][67838] Updated weights for policy 0, policy_version 7242 (0.0008) [2023-10-07 19:56:28,512][67838] Updated weights for policy 0, policy_version 7252 (0.0008) [2023-10-07 19:56:28,884][67838] Updated weights for policy 0, policy_version 7262 (0.0008) [2023-10-07 19:56:30,936][67871] Updated weights for policy 1, policy_version 7270 (0.0007) [2023-10-07 19:56:31,305][67871] Updated weights for policy 1, policy_version 7280 (0.0007) [2023-10-07 19:56:31,665][67871] Updated weights for policy 1, policy_version 7290 (0.0008) [2023-10-07 19:56:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 14909440. Throughput: 0: 1649.7, 1: 1644.8. Samples: 3734674. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-07 19:56:32,477][66916] Avg episode reward: [(0, '29.300'), (1, '27.550')] [2023-10-07 19:56:32,965][67838] Updated weights for policy 0, policy_version 7272 (0.0009) [2023-10-07 19:56:33,351][67838] Updated weights for policy 0, policy_version 7282 (0.0008) [2023-10-07 19:56:33,717][67838] Updated weights for policy 0, policy_version 7292 (0.0007) [2023-10-07 19:56:35,939][67871] Updated weights for policy 1, policy_version 7300 (0.0008) [2023-10-07 19:56:36,295][67871] Updated weights for policy 1, policy_version 7310 (0.0008) [2023-10-07 19:56:36,666][67871] Updated weights for policy 1, policy_version 7320 (0.0007) [2023-10-07 19:56:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 14974976. Throughput: 0: 1648.1, 1: 1642.5. Samples: 3753842. Policy #0 lag: (min: 1.0, avg: 5.0, max: 33.0) [2023-10-07 19:56:37,478][66916] Avg episode reward: [(0, '30.220'), (1, '28.470')] [2023-10-07 19:56:37,910][67838] Updated weights for policy 0, policy_version 7302 (0.0008) [2023-10-07 19:56:38,285][67838] Updated weights for policy 0, policy_version 7312 (0.0008) [2023-10-07 19:56:38,671][67838] Updated weights for policy 0, policy_version 7322 (0.0007) [2023-10-07 19:56:40,805][67871] Updated weights for policy 1, policy_version 7330 (0.0008) [2023-10-07 19:56:41,176][67871] Updated weights for policy 1, policy_version 7340 (0.0007) [2023-10-07 19:56:41,544][67871] Updated weights for policy 1, policy_version 7350 (0.0008) [2023-10-07 19:56:41,910][67871] Updated weights for policy 1, policy_version 7360 (0.0008) [2023-10-07 19:56:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 15040512. Throughput: 0: 1645.6, 1: 1637.5. Samples: 3763758. Policy #0 lag: (min: 1.0, avg: 5.0, max: 33.0) [2023-10-07 19:56:42,478][66916] Avg episode reward: [(0, '28.530'), (1, '29.300')] [2023-10-07 19:56:42,479][67676] Saving new best policy, reward=29.300! [2023-10-07 19:56:42,797][67838] Updated weights for policy 0, policy_version 7332 (0.0009) [2023-10-07 19:56:43,181][67838] Updated weights for policy 0, policy_version 7342 (0.0010) [2023-10-07 19:56:43,562][67838] Updated weights for policy 0, policy_version 7352 (0.0010) [2023-10-07 19:56:46,183][67871] Updated weights for policy 1, policy_version 7370 (0.0008) [2023-10-07 19:56:46,553][67871] Updated weights for policy 1, policy_version 7380 (0.0011) [2023-10-07 19:56:46,920][67871] Updated weights for policy 1, policy_version 7390 (0.0009) [2023-10-07 19:56:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 15106048. Throughput: 0: 1643.4, 1: 1641.7. Samples: 3783686. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 19:56:47,477][66916] Avg episode reward: [(0, '28.820'), (1, '28.160')] [2023-10-07 19:56:47,941][67838] Updated weights for policy 0, policy_version 7362 (0.0009) [2023-10-07 19:56:48,308][67838] Updated weights for policy 0, policy_version 7372 (0.0009) [2023-10-07 19:56:48,676][67838] Updated weights for policy 0, policy_version 7382 (0.0010) [2023-10-07 19:56:49,053][67838] Updated weights for policy 0, policy_version 7392 (0.0009) [2023-10-07 19:56:51,102][67871] Updated weights for policy 1, policy_version 7400 (0.0008) [2023-10-07 19:56:51,469][67871] Updated weights for policy 1, policy_version 7410 (0.0007) [2023-10-07 19:56:51,842][67871] Updated weights for policy 1, policy_version 7420 (0.0010) [2023-10-07 19:56:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 15171584. Throughput: 0: 1640.7, 1: 1635.3. Samples: 3802640. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 19:56:52,477][66916] Avg episode reward: [(0, '28.380'), (1, '30.180')] [2023-10-07 19:56:52,487][67676] Saving new best policy, reward=30.180! [2023-10-07 19:56:53,407][67838] Updated weights for policy 0, policy_version 7402 (0.0011) [2023-10-07 19:56:53,784][67838] Updated weights for policy 0, policy_version 7412 (0.0009) [2023-10-07 19:56:54,161][67838] Updated weights for policy 0, policy_version 7422 (0.0010) [2023-10-07 19:56:55,826][67871] Updated weights for policy 1, policy_version 7430 (0.0010) [2023-10-07 19:56:56,196][67871] Updated weights for policy 1, policy_version 7440 (0.0009) [2023-10-07 19:56:56,571][67871] Updated weights for policy 1, policy_version 7450 (0.0007) [2023-10-07 19:56:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 15237120. Throughput: 0: 1639.6, 1: 1635.8. Samples: 3812688. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 19:56:57,478][66916] Avg episode reward: [(0, '30.940'), (1, '28.900')] [2023-10-07 19:56:57,479][67511] Saving new best policy, reward=30.940! [2023-10-07 19:56:58,245][67838] Updated weights for policy 0, policy_version 7432 (0.0008) [2023-10-07 19:56:58,621][67838] Updated weights for policy 0, policy_version 7442 (0.0009) [2023-10-07 19:56:58,989][67838] Updated weights for policy 0, policy_version 7452 (0.0011) [2023-10-07 19:57:00,772][67871] Updated weights for policy 1, policy_version 7460 (0.0010) [2023-10-07 19:57:01,143][67871] Updated weights for policy 1, policy_version 7470 (0.0009) [2023-10-07 19:57:01,502][67871] Updated weights for policy 1, policy_version 7480 (0.0008) [2023-10-07 19:57:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 15302656. Throughput: 0: 1640.2, 1: 1638.2. Samples: 3832742. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 19:57:02,477][66916] Avg episode reward: [(0, '29.830'), (1, '29.410')] [2023-10-07 19:57:03,234][67838] Updated weights for policy 0, policy_version 7462 (0.0010) [2023-10-07 19:57:03,609][67838] Updated weights for policy 0, policy_version 7472 (0.0008) [2023-10-07 19:57:03,990][67838] Updated weights for policy 0, policy_version 7482 (0.0007) [2023-10-07 19:57:05,633][67871] Updated weights for policy 1, policy_version 7490 (0.0010) [2023-10-07 19:57:06,010][67871] Updated weights for policy 1, policy_version 7500 (0.0008) [2023-10-07 19:57:06,368][67871] Updated weights for policy 1, policy_version 7510 (0.0008) [2023-10-07 19:57:06,742][67871] Updated weights for policy 1, policy_version 7520 (0.0007) [2023-10-07 19:57:07,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 15368192. Throughput: 0: 1638.5, 1: 1640.0. Samples: 3851950. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-07 19:57:07,477][66916] Avg episode reward: [(0, '29.500'), (1, '29.150')] [2023-10-07 19:57:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000007488_7667712.pth... [2023-10-07 19:57:07,490][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000007520_7700480.pth... [2023-10-07 19:57:07,523][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000005984_6127616.pth [2023-10-07 19:57:07,525][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000005984_6127616.pth [2023-10-07 19:57:07,527][67511] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p0/milestones/checkpoint_000007488_7667712.pth [2023-10-07 19:57:07,529][67676] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p1/milestones/checkpoint_000007520_7700480.pth [2023-10-07 19:57:08,020][67838] Updated weights for policy 0, policy_version 7492 (0.0008) [2023-10-07 19:57:08,397][67838] Updated weights for policy 0, policy_version 7502 (0.0008) [2023-10-07 19:57:08,772][67838] Updated weights for policy 0, policy_version 7512 (0.0007) [2023-10-07 19:57:10,870][67871] Updated weights for policy 1, policy_version 7530 (0.0008) [2023-10-07 19:57:11,238][67871] Updated weights for policy 1, policy_version 7540 (0.0007) [2023-10-07 19:57:11,615][67871] Updated weights for policy 1, policy_version 7550 (0.0010) [2023-10-07 19:57:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 15433728. Throughput: 0: 1635.1, 1: 1642.4. Samples: 3862166. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-07 19:57:12,478][66916] Avg episode reward: [(0, '30.520'), (1, '29.150')] [2023-10-07 19:57:12,965][67838] Updated weights for policy 0, policy_version 7522 (0.0010) [2023-10-07 19:57:13,339][67838] Updated weights for policy 0, policy_version 7532 (0.0009) [2023-10-07 19:57:13,712][67838] Updated weights for policy 0, policy_version 7542 (0.0009) [2023-10-07 19:57:14,081][67838] Updated weights for policy 0, policy_version 7552 (0.0009) [2023-10-07 19:57:15,651][67871] Updated weights for policy 1, policy_version 7560 (0.0011) [2023-10-07 19:57:16,015][67871] Updated weights for policy 1, policy_version 7570 (0.0007) [2023-10-07 19:57:16,381][67871] Updated weights for policy 1, policy_version 7580 (0.0007) [2023-10-07 19:57:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 15499264. Throughput: 0: 1632.6, 1: 1639.9. Samples: 3881938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:57:17,478][66916] Avg episode reward: [(0, '29.100'), (1, '28.570')] [2023-10-07 19:57:18,440][67838] Updated weights for policy 0, policy_version 7562 (0.0009) [2023-10-07 19:57:18,810][67838] Updated weights for policy 0, policy_version 7572 (0.0008) [2023-10-07 19:57:19,183][67838] Updated weights for policy 0, policy_version 7582 (0.0010) [2023-10-07 19:57:20,543][67871] Updated weights for policy 1, policy_version 7590 (0.0007) [2023-10-07 19:57:20,908][67871] Updated weights for policy 1, policy_version 7600 (0.0008) [2023-10-07 19:57:21,286][67871] Updated weights for policy 1, policy_version 7610 (0.0011) [2023-10-07 19:57:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 15564800. Throughput: 0: 1633.5, 1: 1649.6. Samples: 3901578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:57:22,477][66916] Avg episode reward: [(0, '30.260'), (1, '28.090')] [2023-10-07 19:57:23,324][67838] Updated weights for policy 0, policy_version 7592 (0.0008) [2023-10-07 19:57:23,700][67838] Updated weights for policy 0, policy_version 7602 (0.0007) [2023-10-07 19:57:24,072][67838] Updated weights for policy 0, policy_version 7612 (0.0007) [2023-10-07 19:57:25,608][67871] Updated weights for policy 1, policy_version 7620 (0.0010) [2023-10-07 19:57:25,975][67871] Updated weights for policy 1, policy_version 7630 (0.0007) [2023-10-07 19:57:26,341][67871] Updated weights for policy 1, policy_version 7640 (0.0009) [2023-10-07 19:57:27,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 15630336. Throughput: 0: 1633.5, 1: 1649.6. Samples: 3911494. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 19:57:27,478][66916] Avg episode reward: [(0, '29.850'), (1, '28.960')] [2023-10-07 19:57:28,096][67838] Updated weights for policy 0, policy_version 7622 (0.0007) [2023-10-07 19:57:28,481][67838] Updated weights for policy 0, policy_version 7632 (0.0007) [2023-10-07 19:57:28,847][67838] Updated weights for policy 0, policy_version 7642 (0.0008) [2023-10-07 19:57:30,454][67871] Updated weights for policy 1, policy_version 7650 (0.0010) [2023-10-07 19:57:30,832][67871] Updated weights for policy 1, policy_version 7660 (0.0010) [2023-10-07 19:57:31,196][67871] Updated weights for policy 1, policy_version 7670 (0.0009) [2023-10-07 19:57:31,559][67871] Updated weights for policy 1, policy_version 7680 (0.0011) [2023-10-07 19:57:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 15695872. Throughput: 0: 1640.2, 1: 1639.8. Samples: 3931286. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 19:57:32,478][66916] Avg episode reward: [(0, '30.460'), (1, '28.920')] [2023-10-07 19:57:32,870][67838] Updated weights for policy 0, policy_version 7652 (0.0008) [2023-10-07 19:57:33,249][67838] Updated weights for policy 0, policy_version 7662 (0.0007) [2023-10-07 19:57:33,614][67838] Updated weights for policy 0, policy_version 7672 (0.0010) [2023-10-07 19:57:35,833][67871] Updated weights for policy 1, policy_version 7690 (0.0009) [2023-10-07 19:57:36,201][67871] Updated weights for policy 1, policy_version 7700 (0.0011) [2023-10-07 19:57:36,570][67871] Updated weights for policy 1, policy_version 7710 (0.0010) [2023-10-07 19:57:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 15761408. Throughput: 0: 1645.9, 1: 1645.6. Samples: 3950754. Policy #0 lag: (min: 25.0, avg: 40.4, max: 57.0) [2023-10-07 19:57:37,477][66916] Avg episode reward: [(0, '30.080'), (1, '28.700')] [2023-10-07 19:57:37,803][67838] Updated weights for policy 0, policy_version 7682 (0.0009) [2023-10-07 19:57:38,182][67838] Updated weights for policy 0, policy_version 7692 (0.0008) [2023-10-07 19:57:38,562][67838] Updated weights for policy 0, policy_version 7702 (0.0009) [2023-10-07 19:57:38,943][67838] Updated weights for policy 0, policy_version 7712 (0.0008) [2023-10-07 19:57:40,747][67871] Updated weights for policy 1, policy_version 7720 (0.0008) [2023-10-07 19:57:41,120][67871] Updated weights for policy 1, policy_version 7730 (0.0007) [2023-10-07 19:57:41,494][67871] Updated weights for policy 1, policy_version 7740 (0.0007) [2023-10-07 19:57:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 15826944. Throughput: 0: 1647.2, 1: 1643.8. Samples: 3960780. Policy #0 lag: (min: 25.0, avg: 40.4, max: 57.0) [2023-10-07 19:57:42,478][66916] Avg episode reward: [(0, '30.040'), (1, '30.440')] [2023-10-07 19:57:42,479][67676] Saving new best policy, reward=30.440! [2023-10-07 19:57:43,069][67838] Updated weights for policy 0, policy_version 7722 (0.0009) [2023-10-07 19:57:43,443][67838] Updated weights for policy 0, policy_version 7732 (0.0009) [2023-10-07 19:57:43,823][67838] Updated weights for policy 0, policy_version 7742 (0.0008) [2023-10-07 19:57:45,655][67871] Updated weights for policy 1, policy_version 7750 (0.0008) [2023-10-07 19:57:46,028][67871] Updated weights for policy 1, policy_version 7760 (0.0009) [2023-10-07 19:57:46,401][67871] Updated weights for policy 1, policy_version 7770 (0.0008) [2023-10-07 19:57:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 15892480. Throughput: 0: 1643.4, 1: 1638.5. Samples: 3980430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:57:47,477][66916] Avg episode reward: [(0, '30.130'), (1, '28.850')] [2023-10-07 19:57:47,877][67838] Updated weights for policy 0, policy_version 7752 (0.0010) [2023-10-07 19:57:48,248][67838] Updated weights for policy 0, policy_version 7762 (0.0010) [2023-10-07 19:57:48,623][67838] Updated weights for policy 0, policy_version 7772 (0.0009) [2023-10-07 19:57:50,635][67871] Updated weights for policy 1, policy_version 7780 (0.0009) [2023-10-07 19:57:51,008][67871] Updated weights for policy 1, policy_version 7790 (0.0010) [2023-10-07 19:57:51,377][67871] Updated weights for policy 1, policy_version 7800 (0.0009) [2023-10-07 19:57:52,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 15958016. Throughput: 0: 1644.3, 1: 1643.6. Samples: 3999908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:57:52,478][66916] Avg episode reward: [(0, '30.440'), (1, '29.480')] [2023-10-07 19:57:52,947][67838] Updated weights for policy 0, policy_version 7782 (0.0007) [2023-10-07 19:57:53,323][67838] Updated weights for policy 0, policy_version 7792 (0.0007) [2023-10-07 19:57:53,700][67838] Updated weights for policy 0, policy_version 7802 (0.0008) [2023-10-07 19:57:55,487][67871] Updated weights for policy 1, policy_version 7810 (0.0010) [2023-10-07 19:57:55,921][67871] Updated weights for policy 1, policy_version 7820 (0.0010) [2023-10-07 19:57:56,288][67871] Updated weights for policy 1, policy_version 7830 (0.0008) [2023-10-07 19:57:56,662][67871] Updated weights for policy 1, policy_version 7840 (0.0007) [2023-10-07 19:57:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 16023552. Throughput: 0: 1643.1, 1: 1645.4. Samples: 4010148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:57:57,477][66916] Avg episode reward: [(0, '30.490'), (1, '30.310')] [2023-10-07 19:57:58,040][67838] Updated weights for policy 0, policy_version 7812 (0.0007) [2023-10-07 19:57:58,411][67838] Updated weights for policy 0, policy_version 7822 (0.0009) [2023-10-07 19:57:58,790][67838] Updated weights for policy 0, policy_version 7832 (0.0008) [2023-10-07 19:58:00,639][67871] Updated weights for policy 1, policy_version 7850 (0.0007) [2023-10-07 19:58:01,000][67871] Updated weights for policy 1, policy_version 7860 (0.0009) [2023-10-07 19:58:01,380][67871] Updated weights for policy 1, policy_version 7870 (0.0008) [2023-10-07 19:58:02,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 16089088. Throughput: 0: 1644.9, 1: 1641.0. Samples: 4029802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:58:02,478][66916] Avg episode reward: [(0, '30.780'), (1, '29.190')] [2023-10-07 19:58:02,672][67838] Updated weights for policy 0, policy_version 7842 (0.0007) [2023-10-07 19:58:03,044][67838] Updated weights for policy 0, policy_version 7852 (0.0009) [2023-10-07 19:58:03,425][67838] Updated weights for policy 0, policy_version 7862 (0.0009) [2023-10-07 19:58:03,796][67838] Updated weights for policy 0, policy_version 7872 (0.0008) [2023-10-07 19:58:05,505][67871] Updated weights for policy 1, policy_version 7880 (0.0008) [2023-10-07 19:58:05,865][67871] Updated weights for policy 1, policy_version 7890 (0.0008) [2023-10-07 19:58:06,232][67871] Updated weights for policy 1, policy_version 7900 (0.0007) [2023-10-07 19:58:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 16154624. Throughput: 0: 1646.8, 1: 1639.2. Samples: 4049446. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-07 19:58:07,477][66916] Avg episode reward: [(0, '28.670'), (1, '29.480')] [2023-10-07 19:58:07,905][67838] Updated weights for policy 0, policy_version 7882 (0.0008) [2023-10-07 19:58:08,287][67838] Updated weights for policy 0, policy_version 7892 (0.0008) [2023-10-07 19:58:08,659][67838] Updated weights for policy 0, policy_version 7902 (0.0007) [2023-10-07 19:58:10,411][67871] Updated weights for policy 1, policy_version 7910 (0.0008) [2023-10-07 19:58:10,779][67871] Updated weights for policy 1, policy_version 7920 (0.0009) [2023-10-07 19:58:11,142][67871] Updated weights for policy 1, policy_version 7930 (0.0009) [2023-10-07 19:58:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 16220160. Throughput: 0: 1647.4, 1: 1641.7. Samples: 4059504. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-07 19:58:12,477][66916] Avg episode reward: [(0, '29.250'), (1, '30.550')] [2023-10-07 19:58:12,478][67676] Saving new best policy, reward=30.550! [2023-10-07 19:58:12,768][67838] Updated weights for policy 0, policy_version 7912 (0.0008) [2023-10-07 19:58:13,138][67838] Updated weights for policy 0, policy_version 7922 (0.0008) [2023-10-07 19:58:13,512][67838] Updated weights for policy 0, policy_version 7932 (0.0007) [2023-10-07 19:58:15,270][67871] Updated weights for policy 1, policy_version 7940 (0.0008) [2023-10-07 19:58:15,635][67871] Updated weights for policy 1, policy_version 7950 (0.0009) [2023-10-07 19:58:16,004][67871] Updated weights for policy 1, policy_version 7960 (0.0007) [2023-10-07 19:58:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 16285696. Throughput: 0: 1642.7, 1: 1641.1. Samples: 4079058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:58:17,477][66916] Avg episode reward: [(0, '28.990'), (1, '29.430')] [2023-10-07 19:58:17,772][67838] Updated weights for policy 0, policy_version 7942 (0.0007) [2023-10-07 19:58:18,148][67838] Updated weights for policy 0, policy_version 7952 (0.0008) [2023-10-07 19:58:18,520][67838] Updated weights for policy 0, policy_version 7962 (0.0008) [2023-10-07 19:58:20,125][67871] Updated weights for policy 1, policy_version 7970 (0.0007) [2023-10-07 19:58:20,502][67871] Updated weights for policy 1, policy_version 7980 (0.0010) [2023-10-07 19:58:20,870][67871] Updated weights for policy 1, policy_version 7990 (0.0010) [2023-10-07 19:58:21,230][67871] Updated weights for policy 1, policy_version 8000 (0.0007) [2023-10-07 19:58:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 16351232. Throughput: 0: 1645.4, 1: 1651.2. Samples: 4099100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:58:22,477][66916] Avg episode reward: [(0, '31.460'), (1, '30.060')] [2023-10-07 19:58:22,696][67838] Updated weights for policy 0, policy_version 7972 (0.0010) [2023-10-07 19:58:23,065][67838] Updated weights for policy 0, policy_version 7982 (0.0008) [2023-10-07 19:58:23,439][67838] Updated weights for policy 0, policy_version 7992 (0.0009) [2023-10-07 19:58:23,736][67511] Saving new best policy, reward=31.460! [2023-10-07 19:58:25,416][67871] Updated weights for policy 1, policy_version 8010 (0.0008) [2023-10-07 19:58:25,784][67871] Updated weights for policy 1, policy_version 8020 (0.0010) [2023-10-07 19:58:26,160][67871] Updated weights for policy 1, policy_version 8030 (0.0010) [2023-10-07 19:58:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 16416768. Throughput: 0: 1643.9, 1: 1656.3. Samples: 4109290. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-10-07 19:58:27,477][66916] Avg episode reward: [(0, '28.960'), (1, '31.050')] [2023-10-07 19:58:27,478][67676] Saving new best policy, reward=31.050! [2023-10-07 19:58:27,655][67838] Updated weights for policy 0, policy_version 8002 (0.0010) [2023-10-07 19:58:28,032][67838] Updated weights for policy 0, policy_version 8012 (0.0008) [2023-10-07 19:58:28,408][67838] Updated weights for policy 0, policy_version 8022 (0.0008) [2023-10-07 19:58:28,788][67838] Updated weights for policy 0, policy_version 8032 (0.0008) [2023-10-07 19:58:30,409][67871] Updated weights for policy 1, policy_version 8040 (0.0009) [2023-10-07 19:58:30,783][67871] Updated weights for policy 1, policy_version 8050 (0.0007) [2023-10-07 19:58:31,148][67871] Updated weights for policy 1, policy_version 8060 (0.0011) [2023-10-07 19:58:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 16482304. Throughput: 0: 1647.8, 1: 1653.2. Samples: 4128976. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-10-07 19:58:32,477][66916] Avg episode reward: [(0, '30.070'), (1, '29.680')] [2023-10-07 19:58:32,874][67838] Updated weights for policy 0, policy_version 8042 (0.0009) [2023-10-07 19:58:33,236][67838] Updated weights for policy 0, policy_version 8052 (0.0007) [2023-10-07 19:58:33,606][67838] Updated weights for policy 0, policy_version 8062 (0.0007) [2023-10-07 19:58:35,119][67871] Updated weights for policy 1, policy_version 8070 (0.0011) [2023-10-07 19:58:35,483][67871] Updated weights for policy 1, policy_version 8080 (0.0010) [2023-10-07 19:58:35,852][67871] Updated weights for policy 1, policy_version 8090 (0.0007) [2023-10-07 19:58:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 16547840. Throughput: 0: 1653.7, 1: 1659.2. Samples: 4148990. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 19:58:37,478][66916] Avg episode reward: [(0, '29.360'), (1, '30.100')] [2023-10-07 19:58:37,563][67838] Updated weights for policy 0, policy_version 8072 (0.0009) [2023-10-07 19:58:37,937][67838] Updated weights for policy 0, policy_version 8082 (0.0009) [2023-10-07 19:58:38,311][67838] Updated weights for policy 0, policy_version 8092 (0.0007) [2023-10-07 19:58:40,050][67871] Updated weights for policy 1, policy_version 8100 (0.0008) [2023-10-07 19:58:40,415][67871] Updated weights for policy 1, policy_version 8110 (0.0008) [2023-10-07 19:58:40,785][67871] Updated weights for policy 1, policy_version 8120 (0.0007) [2023-10-07 19:58:42,400][67838] Updated weights for policy 0, policy_version 8102 (0.0008) [2023-10-07 19:58:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 16613376. Throughput: 0: 1657.0, 1: 1657.5. Samples: 4159300. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 19:58:42,478][66916] Avg episode reward: [(0, '30.740'), (1, '29.980')] [2023-10-07 19:58:42,775][67838] Updated weights for policy 0, policy_version 8112 (0.0007) [2023-10-07 19:58:43,144][67838] Updated weights for policy 0, policy_version 8122 (0.0009) [2023-10-07 19:58:44,830][67871] Updated weights for policy 1, policy_version 8130 (0.0007) [2023-10-07 19:58:45,191][67871] Updated weights for policy 1, policy_version 8140 (0.0009) [2023-10-07 19:58:45,557][67871] Updated weights for policy 1, policy_version 8150 (0.0010) [2023-10-07 19:58:45,925][67871] Updated weights for policy 1, policy_version 8160 (0.0007) [2023-10-07 19:58:47,371][67838] Updated weights for policy 0, policy_version 8132 (0.0010) [2023-10-07 19:58:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 16678912. Throughput: 0: 1657.8, 1: 1650.1. Samples: 4178658. Policy #0 lag: (min: 2.0, avg: 2.5, max: 17.0) [2023-10-07 19:58:47,477][66916] Avg episode reward: [(0, '30.650'), (1, '29.680')] [2023-10-07 19:58:47,751][67838] Updated weights for policy 0, policy_version 8142 (0.0010) [2023-10-07 19:58:48,116][67838] Updated weights for policy 0, policy_version 8152 (0.0008) [2023-10-07 19:58:49,936][67871] Updated weights for policy 1, policy_version 8170 (0.0009) [2023-10-07 19:58:50,307][67871] Updated weights for policy 1, policy_version 8180 (0.0009) [2023-10-07 19:58:50,681][67871] Updated weights for policy 1, policy_version 8190 (0.0007) [2023-10-07 19:58:52,361][67838] Updated weights for policy 0, policy_version 8162 (0.0007) [2023-10-07 19:58:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 16744448. Throughput: 0: 1655.7, 1: 1663.7. Samples: 4198822. Policy #0 lag: (min: 2.0, avg: 2.5, max: 17.0) [2023-10-07 19:58:52,477][66916] Avg episode reward: [(0, '30.250'), (1, '30.480')] [2023-10-07 19:58:52,759][67838] Updated weights for policy 0, policy_version 8172 (0.0009) [2023-10-07 19:58:53,137][67838] Updated weights for policy 0, policy_version 8182 (0.0009) [2023-10-07 19:58:53,513][67838] Updated weights for policy 0, policy_version 8192 (0.0008) [2023-10-07 19:58:54,781][67871] Updated weights for policy 1, policy_version 8200 (0.0008) [2023-10-07 19:58:55,156][67871] Updated weights for policy 1, policy_version 8210 (0.0009) [2023-10-07 19:58:55,516][67871] Updated weights for policy 1, policy_version 8220 (0.0009) [2023-10-07 19:58:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 16809984. Throughput: 0: 1652.3, 1: 1657.0. Samples: 4208422. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-10-07 19:58:57,477][66916] Avg episode reward: [(0, '30.590'), (1, '30.270')] [2023-10-07 19:58:57,734][67838] Updated weights for policy 0, policy_version 8202 (0.0011) [2023-10-07 19:58:58,106][67838] Updated weights for policy 0, policy_version 8212 (0.0010) [2023-10-07 19:58:58,482][67838] Updated weights for policy 0, policy_version 8222 (0.0011) [2023-10-07 19:58:59,768][67871] Updated weights for policy 1, policy_version 8230 (0.0009) [2023-10-07 19:59:00,130][67871] Updated weights for policy 1, policy_version 8240 (0.0009) [2023-10-07 19:59:00,504][67871] Updated weights for policy 1, policy_version 8250 (0.0008) [2023-10-07 19:59:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 16875520. Throughput: 0: 1648.7, 1: 1650.4. Samples: 4227516. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-10-07 19:59:02,477][66916] Avg episode reward: [(0, '30.710'), (1, '29.070')] [2023-10-07 19:59:02,761][67838] Updated weights for policy 0, policy_version 8232 (0.0009) [2023-10-07 19:59:03,125][67838] Updated weights for policy 0, policy_version 8242 (0.0007) [2023-10-07 19:59:03,497][67838] Updated weights for policy 0, policy_version 8252 (0.0008) [2023-10-07 19:59:04,679][67871] Updated weights for policy 1, policy_version 8260 (0.0009) [2023-10-07 19:59:05,050][67871] Updated weights for policy 1, policy_version 8270 (0.0008) [2023-10-07 19:59:05,413][67871] Updated weights for policy 1, policy_version 8280 (0.0010) [2023-10-07 19:59:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 16941056. Throughput: 0: 1648.3, 1: 1656.4. Samples: 4247808. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-07 19:59:07,477][66916] Avg episode reward: [(0, '28.490'), (1, '30.320')] [2023-10-07 19:59:07,484][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000008288_8486912.pth... [2023-10-07 19:59:07,497][67838] Updated weights for policy 0, policy_version 8262 (0.0008) [2023-10-07 19:59:07,519][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000006752_6914048.pth [2023-10-07 19:59:07,868][67838] Updated weights for policy 0, policy_version 8272 (0.0007) [2023-10-07 19:59:08,244][67838] Updated weights for policy 0, policy_version 8282 (0.0008) [2023-10-07 19:59:08,467][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000008288_8486912.pth... [2023-10-07 19:59:08,495][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000006720_6881280.pth [2023-10-07 19:59:09,894][67871] Updated weights for policy 1, policy_version 8290 (0.0008) [2023-10-07 19:59:10,260][67871] Updated weights for policy 1, policy_version 8300 (0.0010) [2023-10-07 19:59:10,624][67871] Updated weights for policy 1, policy_version 8310 (0.0010) [2023-10-07 19:59:10,994][67871] Updated weights for policy 1, policy_version 8320 (0.0009) [2023-10-07 19:59:12,457][67838] Updated weights for policy 0, policy_version 8292 (0.0010) [2023-10-07 19:59:12,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 17006592. Throughput: 0: 1648.0, 1: 1652.7. Samples: 4257820. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-07 19:59:12,477][66916] Avg episode reward: [(0, '30.930'), (1, '30.200')] [2023-10-07 19:59:12,827][67838] Updated weights for policy 0, policy_version 8302 (0.0011) [2023-10-07 19:59:13,206][67838] Updated weights for policy 0, policy_version 8312 (0.0010) [2023-10-07 19:59:15,126][67871] Updated weights for policy 1, policy_version 8330 (0.0007) [2023-10-07 19:59:15,501][67871] Updated weights for policy 1, policy_version 8340 (0.0008) [2023-10-07 19:59:15,873][67871] Updated weights for policy 1, policy_version 8350 (0.0008) [2023-10-07 19:59:17,325][67838] Updated weights for policy 0, policy_version 8322 (0.0008) [2023-10-07 19:59:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 17072128. Throughput: 0: 1648.2, 1: 1645.8. Samples: 4277206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:59:17,477][66916] Avg episode reward: [(0, '30.070'), (1, '30.300')] [2023-10-07 19:59:17,694][67838] Updated weights for policy 0, policy_version 8332 (0.0009) [2023-10-07 19:59:18,069][67838] Updated weights for policy 0, policy_version 8342 (0.0007) [2023-10-07 19:59:18,439][67838] Updated weights for policy 0, policy_version 8352 (0.0008) [2023-10-07 19:59:19,964][67871] Updated weights for policy 1, policy_version 8360 (0.0010) [2023-10-07 19:59:20,336][67871] Updated weights for policy 1, policy_version 8370 (0.0011) [2023-10-07 19:59:20,703][67871] Updated weights for policy 1, policy_version 8380 (0.0011) [2023-10-07 19:59:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 17137664. Throughput: 0: 1644.8, 1: 1652.9. Samples: 4297390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:59:22,478][66916] Avg episode reward: [(0, '30.010'), (1, '29.410')] [2023-10-07 19:59:22,536][67838] Updated weights for policy 0, policy_version 8362 (0.0009) [2023-10-07 19:59:22,924][67838] Updated weights for policy 0, policy_version 8372 (0.0008) [2023-10-07 19:59:23,297][67838] Updated weights for policy 0, policy_version 8382 (0.0009) [2023-10-07 19:59:24,800][67871] Updated weights for policy 1, policy_version 8390 (0.0008) [2023-10-07 19:59:25,170][67871] Updated weights for policy 1, policy_version 8400 (0.0007) [2023-10-07 19:59:25,544][67871] Updated weights for policy 1, policy_version 8410 (0.0009) [2023-10-07 19:59:27,353][67838] Updated weights for policy 0, policy_version 8392 (0.0009) [2023-10-07 19:59:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 17203200. Throughput: 0: 1642.5, 1: 1642.0. Samples: 4307098. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:59:27,477][66916] Avg episode reward: [(0, '29.850'), (1, '30.360')] [2023-10-07 19:59:27,725][67838] Updated weights for policy 0, policy_version 8402 (0.0011) [2023-10-07 19:59:28,109][67838] Updated weights for policy 0, policy_version 8412 (0.0010) [2023-10-07 19:59:29,661][67871] Updated weights for policy 1, policy_version 8420 (0.0008) [2023-10-07 19:59:30,037][67871] Updated weights for policy 1, policy_version 8430 (0.0007) [2023-10-07 19:59:30,404][67871] Updated weights for policy 1, policy_version 8440 (0.0008) [2023-10-07 19:59:32,277][67838] Updated weights for policy 0, policy_version 8422 (0.0010) [2023-10-07 19:59:32,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 17268736. Throughput: 0: 1641.9, 1: 1639.6. Samples: 4326326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:59:32,477][66916] Avg episode reward: [(0, '30.260'), (1, '31.320')] [2023-10-07 19:59:32,478][67676] Saving new best policy, reward=31.320! [2023-10-07 19:59:32,649][67838] Updated weights for policy 0, policy_version 8432 (0.0008) [2023-10-07 19:59:33,033][67838] Updated weights for policy 0, policy_version 8442 (0.0009) [2023-10-07 19:59:34,580][67871] Updated weights for policy 1, policy_version 8450 (0.0009) [2023-10-07 19:59:34,949][67871] Updated weights for policy 1, policy_version 8460 (0.0009) [2023-10-07 19:59:35,321][67871] Updated weights for policy 1, policy_version 8470 (0.0009) [2023-10-07 19:59:35,687][67871] Updated weights for policy 1, policy_version 8480 (0.0007) [2023-10-07 19:59:37,365][67838] Updated weights for policy 0, policy_version 8452 (0.0009) [2023-10-07 19:59:37,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 17334272. Throughput: 0: 1639.3, 1: 1643.6. Samples: 4346552. Policy #0 lag: (min: 21.0, avg: 25.8, max: 53.0) [2023-10-07 19:59:37,478][66916] Avg episode reward: [(0, '29.960'), (1, '30.990')] [2023-10-07 19:59:37,745][67838] Updated weights for policy 0, policy_version 8462 (0.0010) [2023-10-07 19:59:38,114][67838] Updated weights for policy 0, policy_version 8472 (0.0008) [2023-10-07 19:59:39,787][67871] Updated weights for policy 1, policy_version 8490 (0.0008) [2023-10-07 19:59:40,154][67871] Updated weights for policy 1, policy_version 8500 (0.0009) [2023-10-07 19:59:40,526][67871] Updated weights for policy 1, policy_version 8510 (0.0008) [2023-10-07 19:59:42,323][67838] Updated weights for policy 0, policy_version 8482 (0.0008) [2023-10-07 19:59:42,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 17399808. Throughput: 0: 1637.2, 1: 1644.8. Samples: 4356110. Policy #0 lag: (min: 21.0, avg: 25.8, max: 53.0) [2023-10-07 19:59:42,477][66916] Avg episode reward: [(0, '30.700'), (1, '30.870')] [2023-10-07 19:59:42,700][67838] Updated weights for policy 0, policy_version 8492 (0.0008) [2023-10-07 19:59:43,074][67838] Updated weights for policy 0, policy_version 8502 (0.0009) [2023-10-07 19:59:43,450][67838] Updated weights for policy 0, policy_version 8512 (0.0007) [2023-10-07 19:59:44,828][67871] Updated weights for policy 1, policy_version 8520 (0.0008) [2023-10-07 19:59:45,197][67871] Updated weights for policy 1, policy_version 8530 (0.0008) [2023-10-07 19:59:45,560][67871] Updated weights for policy 1, policy_version 8540 (0.0008) [2023-10-07 19:59:47,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 17465344. Throughput: 0: 1646.9, 1: 1644.7. Samples: 4375636. Policy #0 lag: (min: 26.0, avg: 26.0, max: 30.0) [2023-10-07 19:59:47,477][66916] Avg episode reward: [(0, '30.050'), (1, '30.510')] [2023-10-07 19:59:47,614][67838] Updated weights for policy 0, policy_version 8522 (0.0007) [2023-10-07 19:59:47,984][67838] Updated weights for policy 0, policy_version 8532 (0.0008) [2023-10-07 19:59:48,367][67838] Updated weights for policy 0, policy_version 8542 (0.0009) [2023-10-07 19:59:49,501][67871] Updated weights for policy 1, policy_version 8550 (0.0010) [2023-10-07 19:59:49,873][67871] Updated weights for policy 1, policy_version 8560 (0.0009) [2023-10-07 19:59:50,235][67871] Updated weights for policy 1, policy_version 8570 (0.0009) [2023-10-07 19:59:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 17530880. Throughput: 0: 1642.3, 1: 1651.5. Samples: 4396028. Policy #0 lag: (min: 26.0, avg: 26.0, max: 30.0) [2023-10-07 19:59:52,477][66916] Avg episode reward: [(0, '30.560'), (1, '30.840')] [2023-10-07 19:59:52,665][67838] Updated weights for policy 0, policy_version 8552 (0.0007) [2023-10-07 19:59:53,046][67838] Updated weights for policy 0, policy_version 8562 (0.0008) [2023-10-07 19:59:53,419][67838] Updated weights for policy 0, policy_version 8572 (0.0008) [2023-10-07 19:59:54,600][67871] Updated weights for policy 1, policy_version 8580 (0.0010) [2023-10-07 19:59:54,970][67871] Updated weights for policy 1, policy_version 8590 (0.0008) [2023-10-07 19:59:55,346][67871] Updated weights for policy 1, policy_version 8600 (0.0008) [2023-10-07 19:59:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 17596416. Throughput: 0: 1643.5, 1: 1642.8. Samples: 4405702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 19:59:57,477][66916] Avg episode reward: [(0, '31.840'), (1, '30.940')] [2023-10-07 19:59:57,558][67838] Updated weights for policy 0, policy_version 8582 (0.0008) [2023-10-07 19:59:57,925][67838] Updated weights for policy 0, policy_version 8592 (0.0008) [2023-10-07 19:59:58,292][67838] Updated weights for policy 0, policy_version 8602 (0.0009) [2023-10-07 19:59:58,514][67511] Saving new best policy, reward=31.840! [2023-10-07 19:59:59,616][67871] Updated weights for policy 1, policy_version 8610 (0.0009) [2023-10-07 19:59:59,993][67871] Updated weights for policy 1, policy_version 8620 (0.0009) [2023-10-07 20:00:00,360][67871] Updated weights for policy 1, policy_version 8630 (0.0008) [2023-10-07 20:00:00,733][67871] Updated weights for policy 1, policy_version 8640 (0.0008) [2023-10-07 20:00:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 17661952. Throughput: 0: 1637.9, 1: 1642.6. Samples: 4424828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:00:02,477][66916] Avg episode reward: [(0, '31.220'), (1, '31.310')] [2023-10-07 20:00:02,633][67838] Updated weights for policy 0, policy_version 8612 (0.0009) [2023-10-07 20:00:02,999][67838] Updated weights for policy 0, policy_version 8622 (0.0011) [2023-10-07 20:00:03,371][67838] Updated weights for policy 0, policy_version 8632 (0.0010) [2023-10-07 20:00:04,582][67871] Updated weights for policy 1, policy_version 8650 (0.0007) [2023-10-07 20:00:04,958][67871] Updated weights for policy 1, policy_version 8660 (0.0007) [2023-10-07 20:00:05,321][67871] Updated weights for policy 1, policy_version 8670 (0.0011) [2023-10-07 20:00:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 17727488. Throughput: 0: 1634.5, 1: 1652.3. Samples: 4445296. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 20:00:07,478][66916] Avg episode reward: [(0, '29.540'), (1, '30.070')] [2023-10-07 20:00:07,488][67838] Updated weights for policy 0, policy_version 8642 (0.0009) [2023-10-07 20:00:07,855][67838] Updated weights for policy 0, policy_version 8652 (0.0007) [2023-10-07 20:00:08,230][67838] Updated weights for policy 0, policy_version 8662 (0.0010) [2023-10-07 20:00:08,602][67838] Updated weights for policy 0, policy_version 8672 (0.0009) [2023-10-07 20:00:09,545][67871] Updated weights for policy 1, policy_version 8680 (0.0010) [2023-10-07 20:00:09,908][67871] Updated weights for policy 1, policy_version 8690 (0.0009) [2023-10-07 20:00:10,286][67871] Updated weights for policy 1, policy_version 8700 (0.0008) [2023-10-07 20:00:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 17793024. Throughput: 0: 1639.9, 1: 1644.0. Samples: 4454876. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 20:00:12,477][66916] Avg episode reward: [(0, '31.070'), (1, '31.110')] [2023-10-07 20:00:12,532][67838] Updated weights for policy 0, policy_version 8682 (0.0008) [2023-10-07 20:00:12,910][67838] Updated weights for policy 0, policy_version 8692 (0.0008) [2023-10-07 20:00:13,285][67838] Updated weights for policy 0, policy_version 8702 (0.0009) [2023-10-07 20:00:14,556][67871] Updated weights for policy 1, policy_version 8710 (0.0008) [2023-10-07 20:00:14,925][67871] Updated weights for policy 1, policy_version 8720 (0.0007) [2023-10-07 20:00:15,294][67871] Updated weights for policy 1, policy_version 8730 (0.0008) [2023-10-07 20:00:17,388][67838] Updated weights for policy 0, policy_version 8712 (0.0009) [2023-10-07 20:00:17,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 17858560. Throughput: 0: 1640.3, 1: 1654.8. Samples: 4474606. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 20:00:17,477][66916] Avg episode reward: [(0, '29.710'), (1, '31.040')] [2023-10-07 20:00:17,757][67838] Updated weights for policy 0, policy_version 8722 (0.0007) [2023-10-07 20:00:18,133][67838] Updated weights for policy 0, policy_version 8732 (0.0009) [2023-10-07 20:00:19,542][67871] Updated weights for policy 1, policy_version 8740 (0.0009) [2023-10-07 20:00:19,930][67871] Updated weights for policy 1, policy_version 8750 (0.0008) [2023-10-07 20:00:20,294][67871] Updated weights for policy 1, policy_version 8760 (0.0008) [2023-10-07 20:00:22,163][67838] Updated weights for policy 0, policy_version 8742 (0.0011) [2023-10-07 20:00:22,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 17924096. Throughput: 0: 1639.8, 1: 1651.1. Samples: 4494642. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 20:00:22,477][66916] Avg episode reward: [(0, '29.730'), (1, '31.160')] [2023-10-07 20:00:22,546][67838] Updated weights for policy 0, policy_version 8752 (0.0007) [2023-10-07 20:00:22,920][67838] Updated weights for policy 0, policy_version 8762 (0.0007) [2023-10-07 20:00:24,340][67871] Updated weights for policy 1, policy_version 8770 (0.0008) [2023-10-07 20:00:24,701][67871] Updated weights for policy 1, policy_version 8780 (0.0011) [2023-10-07 20:00:25,076][67871] Updated weights for policy 1, policy_version 8790 (0.0007) [2023-10-07 20:00:25,440][67871] Updated weights for policy 1, policy_version 8800 (0.0009) [2023-10-07 20:00:27,328][67838] Updated weights for policy 0, policy_version 8772 (0.0008) [2023-10-07 20:00:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 17989632. Throughput: 0: 1642.3, 1: 1643.2. Samples: 4503958. Policy #0 lag: (min: 9.0, avg: 28.9, max: 41.0) [2023-10-07 20:00:27,478][66916] Avg episode reward: [(0, '28.590'), (1, '31.580')] [2023-10-07 20:00:27,479][67676] Saving new best policy, reward=31.580! [2023-10-07 20:00:27,702][67838] Updated weights for policy 0, policy_version 8782 (0.0007) [2023-10-07 20:00:28,067][67838] Updated weights for policy 0, policy_version 8792 (0.0008) [2023-10-07 20:00:29,574][67871] Updated weights for policy 1, policy_version 8810 (0.0007) [2023-10-07 20:00:29,942][67871] Updated weights for policy 1, policy_version 8820 (0.0007) [2023-10-07 20:00:30,322][67871] Updated weights for policy 1, policy_version 8830 (0.0008) [2023-10-07 20:00:32,249][67838] Updated weights for policy 0, policy_version 8802 (0.0010) [2023-10-07 20:00:32,477][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18055168. Throughput: 0: 1637.8, 1: 1647.0. Samples: 4523454. Policy #0 lag: (min: 9.0, avg: 28.9, max: 41.0) [2023-10-07 20:00:32,477][66916] Avg episode reward: [(0, '30.680'), (1, '30.100')] [2023-10-07 20:00:32,615][67838] Updated weights for policy 0, policy_version 8812 (0.0008) [2023-10-07 20:00:32,998][67838] Updated weights for policy 0, policy_version 8822 (0.0009) [2023-10-07 20:00:33,365][67838] Updated weights for policy 0, policy_version 8832 (0.0008) [2023-10-07 20:00:34,546][67871] Updated weights for policy 1, policy_version 8840 (0.0008) [2023-10-07 20:00:34,912][67871] Updated weights for policy 1, policy_version 8850 (0.0011) [2023-10-07 20:00:35,292][67871] Updated weights for policy 1, policy_version 8860 (0.0011) [2023-10-07 20:00:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18120704. Throughput: 0: 1641.6, 1: 1640.7. Samples: 4543736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:00:37,478][66916] Avg episode reward: [(0, '30.540'), (1, '31.510')] [2023-10-07 20:00:37,508][67838] Updated weights for policy 0, policy_version 8842 (0.0007) [2023-10-07 20:00:37,887][67838] Updated weights for policy 0, policy_version 8852 (0.0008) [2023-10-07 20:00:38,261][67838] Updated weights for policy 0, policy_version 8862 (0.0008) [2023-10-07 20:00:39,605][67871] Updated weights for policy 1, policy_version 8870 (0.0008) [2023-10-07 20:00:39,967][67871] Updated weights for policy 1, policy_version 8880 (0.0007) [2023-10-07 20:00:40,335][67871] Updated weights for policy 1, policy_version 8890 (0.0007) [2023-10-07 20:00:42,407][67838] Updated weights for policy 0, policy_version 8872 (0.0008) [2023-10-07 20:00:42,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18186240. Throughput: 0: 1640.5, 1: 1639.8. Samples: 4553314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:00:42,478][66916] Avg episode reward: [(0, '30.530'), (1, '30.140')] [2023-10-07 20:00:42,781][67838] Updated weights for policy 0, policy_version 8882 (0.0010) [2023-10-07 20:00:43,156][67838] Updated weights for policy 0, policy_version 8892 (0.0008) [2023-10-07 20:00:44,554][67871] Updated weights for policy 1, policy_version 8900 (0.0008) [2023-10-07 20:00:44,921][67871] Updated weights for policy 1, policy_version 8910 (0.0008) [2023-10-07 20:00:45,294][67871] Updated weights for policy 1, policy_version 8920 (0.0008) [2023-10-07 20:00:47,227][67838] Updated weights for policy 0, policy_version 8902 (0.0008) [2023-10-07 20:00:47,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18251776. Throughput: 0: 1648.8, 1: 1640.7. Samples: 4572854. Policy #0 lag: (min: 17.0, avg: 26.9, max: 49.0) [2023-10-07 20:00:47,477][66916] Avg episode reward: [(0, '30.190'), (1, '31.310')] [2023-10-07 20:00:47,599][67838] Updated weights for policy 0, policy_version 8912 (0.0009) [2023-10-07 20:00:47,981][67838] Updated weights for policy 0, policy_version 8922 (0.0008) [2023-10-07 20:00:49,568][67871] Updated weights for policy 1, policy_version 8930 (0.0009) [2023-10-07 20:00:49,949][67871] Updated weights for policy 1, policy_version 8940 (0.0007) [2023-10-07 20:00:50,318][67871] Updated weights for policy 1, policy_version 8950 (0.0008) [2023-10-07 20:00:50,680][67871] Updated weights for policy 1, policy_version 8960 (0.0009) [2023-10-07 20:00:52,177][67838] Updated weights for policy 0, policy_version 8932 (0.0010) [2023-10-07 20:00:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18317312. Throughput: 0: 1652.1, 1: 1634.9. Samples: 4593208. Policy #0 lag: (min: 17.0, avg: 26.9, max: 49.0) [2023-10-07 20:00:52,477][66916] Avg episode reward: [(0, '31.240'), (1, '31.370')] [2023-10-07 20:00:52,551][67838] Updated weights for policy 0, policy_version 8942 (0.0011) [2023-10-07 20:00:52,928][67838] Updated weights for policy 0, policy_version 8952 (0.0010) [2023-10-07 20:00:54,652][67871] Updated weights for policy 1, policy_version 8970 (0.0007) [2023-10-07 20:00:55,028][67871] Updated weights for policy 1, policy_version 8980 (0.0009) [2023-10-07 20:00:55,395][67871] Updated weights for policy 1, policy_version 8990 (0.0008) [2023-10-07 20:00:56,994][67838] Updated weights for policy 0, policy_version 8962 (0.0009) [2023-10-07 20:00:57,357][67838] Updated weights for policy 0, policy_version 8972 (0.0010) [2023-10-07 20:00:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18382848. Throughput: 0: 1650.5, 1: 1640.0. Samples: 4602950. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-07 20:00:57,477][66916] Avg episode reward: [(0, '30.920'), (1, '30.980')] [2023-10-07 20:00:57,733][67838] Updated weights for policy 0, policy_version 8982 (0.0008) [2023-10-07 20:00:58,110][67838] Updated weights for policy 0, policy_version 8992 (0.0007) [2023-10-07 20:00:59,535][67871] Updated weights for policy 1, policy_version 9000 (0.0008) [2023-10-07 20:00:59,908][67871] Updated weights for policy 1, policy_version 9010 (0.0007) [2023-10-07 20:01:00,277][67871] Updated weights for policy 1, policy_version 9020 (0.0009) [2023-10-07 20:01:02,168][67838] Updated weights for policy 0, policy_version 9002 (0.0007) [2023-10-07 20:01:02,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18448384. Throughput: 0: 1657.7, 1: 1639.9. Samples: 4622998. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-07 20:01:02,477][66916] Avg episode reward: [(0, '30.710'), (1, '30.640')] [2023-10-07 20:01:02,548][67838] Updated weights for policy 0, policy_version 9012 (0.0008) [2023-10-07 20:01:02,920][67838] Updated weights for policy 0, policy_version 9022 (0.0007) [2023-10-07 20:01:04,448][67871] Updated weights for policy 1, policy_version 9030 (0.0007) [2023-10-07 20:01:04,820][67871] Updated weights for policy 1, policy_version 9040 (0.0008) [2023-10-07 20:01:05,190][67871] Updated weights for policy 1, policy_version 9050 (0.0008) [2023-10-07 20:01:07,088][67838] Updated weights for policy 0, policy_version 9032 (0.0008) [2023-10-07 20:01:07,465][67838] Updated weights for policy 0, policy_version 9042 (0.0008) [2023-10-07 20:01:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18513920. Throughput: 0: 1656.4, 1: 1640.7. Samples: 4643012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:01:07,478][66916] Avg episode reward: [(0, '30.670'), (1, '32.350')] [2023-10-07 20:01:07,487][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000009056_9273344.pth... [2023-10-07 20:01:07,521][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000007520_7700480.pth [2023-10-07 20:01:07,524][67676] Saving new best policy, reward=32.350! [2023-10-07 20:01:07,843][67838] Updated weights for policy 0, policy_version 9052 (0.0009) [2023-10-07 20:01:07,986][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000009056_9273344.pth... [2023-10-07 20:01:08,028][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000007488_7667712.pth [2023-10-07 20:01:09,383][67871] Updated weights for policy 1, policy_version 9060 (0.0008) [2023-10-07 20:01:09,786][67871] Updated weights for policy 1, policy_version 9070 (0.0009) [2023-10-07 20:01:10,152][67871] Updated weights for policy 1, policy_version 9080 (0.0009) [2023-10-07 20:01:12,080][67838] Updated weights for policy 0, policy_version 9062 (0.0010) [2023-10-07 20:01:12,442][67838] Updated weights for policy 0, policy_version 9072 (0.0008) [2023-10-07 20:01:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18579456. Throughput: 0: 1665.0, 1: 1642.5. Samples: 4652796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:01:12,477][66916] Avg episode reward: [(0, '31.370'), (1, '30.820')] [2023-10-07 20:01:12,826][67838] Updated weights for policy 0, policy_version 9082 (0.0010) [2023-10-07 20:01:14,394][67871] Updated weights for policy 1, policy_version 9090 (0.0008) [2023-10-07 20:01:14,768][67871] Updated weights for policy 1, policy_version 9100 (0.0010) [2023-10-07 20:01:15,141][67871] Updated weights for policy 1, policy_version 9110 (0.0008) [2023-10-07 20:01:15,520][67871] Updated weights for policy 1, policy_version 9120 (0.0009) [2023-10-07 20:01:16,782][67838] Updated weights for policy 0, policy_version 9092 (0.0008) [2023-10-07 20:01:17,161][67838] Updated weights for policy 0, policy_version 9102 (0.0008) [2023-10-07 20:01:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18644992. Throughput: 0: 1665.8, 1: 1643.6. Samples: 4672374. Policy #0 lag: (min: 26.0, avg: 35.6, max: 58.0) [2023-10-07 20:01:17,477][66916] Avg episode reward: [(0, '29.110'), (1, '31.240')] [2023-10-07 20:01:17,528][67838] Updated weights for policy 0, policy_version 9112 (0.0009) [2023-10-07 20:01:19,635][67871] Updated weights for policy 1, policy_version 9130 (0.0009) [2023-10-07 20:01:20,009][67871] Updated weights for policy 1, policy_version 9140 (0.0009) [2023-10-07 20:01:20,391][67871] Updated weights for policy 1, policy_version 9150 (0.0008) [2023-10-07 20:01:21,779][67838] Updated weights for policy 0, policy_version 9122 (0.0009) [2023-10-07 20:01:22,161][67838] Updated weights for policy 0, policy_version 9132 (0.0009) [2023-10-07 20:01:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18710528. Throughput: 0: 1656.1, 1: 1647.5. Samples: 4692396. Policy #0 lag: (min: 26.0, avg: 35.6, max: 58.0) [2023-10-07 20:01:22,477][66916] Avg episode reward: [(0, '32.110'), (1, '32.400')] [2023-10-07 20:01:22,485][67676] Saving new best policy, reward=32.400! [2023-10-07 20:01:22,531][67838] Updated weights for policy 0, policy_version 9142 (0.0009) [2023-10-07 20:01:22,911][67511] Saving new best policy, reward=32.110! [2023-10-07 20:01:22,911][67838] Updated weights for policy 0, policy_version 9152 (0.0010) [2023-10-07 20:01:24,590][67871] Updated weights for policy 1, policy_version 9160 (0.0008) [2023-10-07 20:01:24,962][67871] Updated weights for policy 1, policy_version 9170 (0.0007) [2023-10-07 20:01:25,321][67871] Updated weights for policy 1, policy_version 9180 (0.0009) [2023-10-07 20:01:26,795][67838] Updated weights for policy 0, policy_version 9162 (0.0008) [2023-10-07 20:01:27,174][67838] Updated weights for policy 0, policy_version 9172 (0.0008) [2023-10-07 20:01:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18776064. Throughput: 0: 1666.7, 1: 1648.7. Samples: 4702506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:01:27,477][66916] Avg episode reward: [(0, '31.120'), (1, '31.100')] [2023-10-07 20:01:27,548][67838] Updated weights for policy 0, policy_version 9182 (0.0009) [2023-10-07 20:01:29,431][67871] Updated weights for policy 1, policy_version 9190 (0.0010) [2023-10-07 20:01:29,794][67871] Updated weights for policy 1, policy_version 9200 (0.0010) [2023-10-07 20:01:30,160][67871] Updated weights for policy 1, policy_version 9210 (0.0007) [2023-10-07 20:01:31,849][67838] Updated weights for policy 0, policy_version 9192 (0.0009) [2023-10-07 20:01:32,234][67838] Updated weights for policy 0, policy_version 9202 (0.0007) [2023-10-07 20:01:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18841600. Throughput: 0: 1664.6, 1: 1652.2. Samples: 4722110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:01:32,477][66916] Avg episode reward: [(0, '30.160'), (1, '31.770')] [2023-10-07 20:01:32,601][67838] Updated weights for policy 0, policy_version 9212 (0.0009) [2023-10-07 20:01:34,290][67871] Updated weights for policy 1, policy_version 9220 (0.0010) [2023-10-07 20:01:34,657][67871] Updated weights for policy 1, policy_version 9230 (0.0008) [2023-10-07 20:01:35,027][67871] Updated weights for policy 1, policy_version 9240 (0.0007) [2023-10-07 20:01:36,879][67838] Updated weights for policy 0, policy_version 9222 (0.0010) [2023-10-07 20:01:37,252][67838] Updated weights for policy 0, policy_version 9232 (0.0007) [2023-10-07 20:01:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 18907136. Throughput: 0: 1651.8, 1: 1656.1. Samples: 4742062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:01:37,477][66916] Avg episode reward: [(0, '31.470'), (1, '30.710')] [2023-10-07 20:01:37,638][67838] Updated weights for policy 0, policy_version 9242 (0.0010) [2023-10-07 20:01:39,248][67871] Updated weights for policy 1, policy_version 9250 (0.0008) [2023-10-07 20:01:39,611][67871] Updated weights for policy 1, policy_version 9260 (0.0007) [2023-10-07 20:01:39,985][67871] Updated weights for policy 1, policy_version 9270 (0.0008) [2023-10-07 20:01:40,346][67871] Updated weights for policy 1, policy_version 9280 (0.0008) [2023-10-07 20:01:41,762][67838] Updated weights for policy 0, policy_version 9252 (0.0010) [2023-10-07 20:01:42,145][67838] Updated weights for policy 0, policy_version 9262 (0.0007) [2023-10-07 20:01:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18972672. Throughput: 0: 1659.1, 1: 1649.4. Samples: 4751834. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:01:42,478][66916] Avg episode reward: [(0, '30.910'), (1, '31.760')] [2023-10-07 20:01:42,521][67838] Updated weights for policy 0, policy_version 9272 (0.0008) [2023-10-07 20:01:44,603][67871] Updated weights for policy 1, policy_version 9290 (0.0007) [2023-10-07 20:01:44,965][67871] Updated weights for policy 1, policy_version 9300 (0.0007) [2023-10-07 20:01:45,346][67871] Updated weights for policy 1, policy_version 9310 (0.0009) [2023-10-07 20:01:46,722][67838] Updated weights for policy 0, policy_version 9282 (0.0007) [2023-10-07 20:01:47,090][67838] Updated weights for policy 0, policy_version 9292 (0.0007) [2023-10-07 20:01:47,472][67838] Updated weights for policy 0, policy_version 9302 (0.0008) [2023-10-07 20:01:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 19038208. Throughput: 0: 1646.4, 1: 1648.5. Samples: 4771266. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-10-07 20:01:47,477][66916] Avg episode reward: [(0, '30.780'), (1, '30.900')] [2023-10-07 20:01:47,846][67838] Updated weights for policy 0, policy_version 9312 (0.0008) [2023-10-07 20:01:49,214][67871] Updated weights for policy 1, policy_version 9320 (0.0010) [2023-10-07 20:01:49,590][67871] Updated weights for policy 1, policy_version 9330 (0.0007) [2023-10-07 20:01:49,963][67871] Updated weights for policy 1, policy_version 9340 (0.0007) [2023-10-07 20:01:51,879][67838] Updated weights for policy 0, policy_version 9322 (0.0010) [2023-10-07 20:01:52,252][67838] Updated weights for policy 0, policy_version 9332 (0.0009) [2023-10-07 20:01:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 19103744. Throughput: 0: 1640.3, 1: 1648.0. Samples: 4790986. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-10-07 20:01:52,478][66916] Avg episode reward: [(0, '30.640'), (1, '31.020')] [2023-10-07 20:01:52,644][67838] Updated weights for policy 0, policy_version 9342 (0.0009) [2023-10-07 20:01:54,322][67871] Updated weights for policy 1, policy_version 9350 (0.0007) [2023-10-07 20:01:54,716][67871] Updated weights for policy 1, policy_version 9360 (0.0007) [2023-10-07 20:01:55,079][67871] Updated weights for policy 1, policy_version 9370 (0.0007) [2023-10-07 20:01:56,534][67838] Updated weights for policy 0, policy_version 9352 (0.0007) [2023-10-07 20:01:56,904][67838] Updated weights for policy 0, policy_version 9362 (0.0008) [2023-10-07 20:01:57,286][67838] Updated weights for policy 0, policy_version 9372 (0.0007) [2023-10-07 20:01:57,476][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 19202048. Throughput: 0: 1647.8, 1: 1642.8. Samples: 4800870. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 20:01:57,477][66916] Avg episode reward: [(0, '31.000'), (1, '32.660')] [2023-10-07 20:01:57,478][67676] Saving new best policy, reward=32.660! [2023-10-07 20:01:59,229][67871] Updated weights for policy 1, policy_version 9380 (0.0009) [2023-10-07 20:01:59,591][67871] Updated weights for policy 1, policy_version 9390 (0.0007) [2023-10-07 20:01:59,958][67871] Updated weights for policy 1, policy_version 9400 (0.0009) [2023-10-07 20:02:01,523][67838] Updated weights for policy 0, policy_version 9382 (0.0008) [2023-10-07 20:02:01,903][67838] Updated weights for policy 0, policy_version 9392 (0.0008) [2023-10-07 20:02:02,280][67838] Updated weights for policy 0, policy_version 9402 (0.0009) [2023-10-07 20:02:02,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 19234816. Throughput: 0: 1649.1, 1: 1644.6. Samples: 4820592. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 20:02:02,477][66916] Avg episode reward: [(0, '31.190'), (1, '30.230')] [2023-10-07 20:02:04,166][67871] Updated weights for policy 1, policy_version 9410 (0.0011) [2023-10-07 20:02:04,535][67871] Updated weights for policy 1, policy_version 9420 (0.0010) [2023-10-07 20:02:04,910][67871] Updated weights for policy 1, policy_version 9430 (0.0011) [2023-10-07 20:02:05,277][67871] Updated weights for policy 1, policy_version 9440 (0.0010) [2023-10-07 20:02:06,410][67838] Updated weights for policy 0, policy_version 9412 (0.0009) [2023-10-07 20:02:06,783][67838] Updated weights for policy 0, policy_version 9422 (0.0010) [2023-10-07 20:02:07,156][67838] Updated weights for policy 0, policy_version 9432 (0.0007) [2023-10-07 20:02:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 19333120. Throughput: 0: 1638.3, 1: 1647.3. Samples: 4840248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:02:07,477][66916] Avg episode reward: [(0, '31.620'), (1, '32.160')] [2023-10-07 20:02:09,292][67871] Updated weights for policy 1, policy_version 9450 (0.0008) [2023-10-07 20:02:09,654][67871] Updated weights for policy 1, policy_version 9460 (0.0007) [2023-10-07 20:02:10,024][67871] Updated weights for policy 1, policy_version 9470 (0.0007) [2023-10-07 20:02:11,422][67838] Updated weights for policy 0, policy_version 9442 (0.0007) [2023-10-07 20:02:11,790][67838] Updated weights for policy 0, policy_version 9452 (0.0008) [2023-10-07 20:02:12,174][67838] Updated weights for policy 0, policy_version 9462 (0.0008) [2023-10-07 20:02:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 19365888. Throughput: 0: 1646.6, 1: 1636.5. Samples: 4850244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:02:12,478][66916] Avg episode reward: [(0, '30.850'), (1, '32.350')] [2023-10-07 20:02:12,554][67838] Updated weights for policy 0, policy_version 9472 (0.0007) [2023-10-07 20:02:14,230][67871] Updated weights for policy 1, policy_version 9480 (0.0009) [2023-10-07 20:02:14,598][67871] Updated weights for policy 1, policy_version 9490 (0.0008) [2023-10-07 20:02:14,966][67871] Updated weights for policy 1, policy_version 9500 (0.0008) [2023-10-07 20:02:16,645][67838] Updated weights for policy 0, policy_version 9482 (0.0007) [2023-10-07 20:02:17,020][67838] Updated weights for policy 0, policy_version 9492 (0.0007) [2023-10-07 20:02:17,398][67838] Updated weights for policy 0, policy_version 9502 (0.0009) [2023-10-07 20:02:17,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 19464192. Throughput: 0: 1644.6, 1: 1643.4. Samples: 4870070. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-07 20:02:17,477][66916] Avg episode reward: [(0, '32.620'), (1, '32.290')] [2023-10-07 20:02:17,478][67511] Saving new best policy, reward=32.620! [2023-10-07 20:02:19,164][67871] Updated weights for policy 1, policy_version 9510 (0.0008) [2023-10-07 20:02:19,535][67871] Updated weights for policy 1, policy_version 9520 (0.0009) [2023-10-07 20:02:19,909][67871] Updated weights for policy 1, policy_version 9530 (0.0009) [2023-10-07 20:02:21,707][67838] Updated weights for policy 0, policy_version 9512 (0.0009) [2023-10-07 20:02:22,087][67838] Updated weights for policy 0, policy_version 9522 (0.0010) [2023-10-07 20:02:22,463][67838] Updated weights for policy 0, policy_version 9532 (0.0010) [2023-10-07 20:02:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 19496960. Throughput: 0: 1640.3, 1: 1642.4. Samples: 4889786. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-07 20:02:22,477][66916] Avg episode reward: [(0, '31.800'), (1, '32.090')] [2023-10-07 20:02:23,944][67871] Updated weights for policy 1, policy_version 9540 (0.0007) [2023-10-07 20:02:24,322][67871] Updated weights for policy 1, policy_version 9550 (0.0010) [2023-10-07 20:02:24,680][67871] Updated weights for policy 1, policy_version 9560 (0.0009) [2023-10-07 20:02:26,489][67838] Updated weights for policy 0, policy_version 9542 (0.0009) [2023-10-07 20:02:26,878][67838] Updated weights for policy 0, policy_version 9552 (0.0009) [2023-10-07 20:02:27,261][67838] Updated weights for policy 0, policy_version 9562 (0.0008) [2023-10-07 20:02:27,476][66916] Fps is (10 sec: 9830.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 19562496. Throughput: 0: 1644.8, 1: 1635.3. Samples: 4899436. Policy #0 lag: (min: 9.0, avg: 19.4, max: 41.0) [2023-10-07 20:02:27,477][66916] Avg episode reward: [(0, '31.990'), (1, '31.230')] [2023-10-07 20:02:28,811][67871] Updated weights for policy 1, policy_version 9570 (0.0008) [2023-10-07 20:02:29,184][67871] Updated weights for policy 1, policy_version 9580 (0.0009) [2023-10-07 20:02:29,556][67871] Updated weights for policy 1, policy_version 9590 (0.0009) [2023-10-07 20:02:29,925][67871] Updated weights for policy 1, policy_version 9600 (0.0009) [2023-10-07 20:02:31,269][67838] Updated weights for policy 0, policy_version 9572 (0.0008) [2023-10-07 20:02:31,638][67838] Updated weights for policy 0, policy_version 9582 (0.0008) [2023-10-07 20:02:32,015][67838] Updated weights for policy 0, policy_version 9592 (0.0009) [2023-10-07 20:02:32,476][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 19660800. Throughput: 0: 1649.1, 1: 1653.1. Samples: 4919864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:02:32,477][66916] Avg episode reward: [(0, '31.080'), (1, '31.570')] [2023-10-07 20:02:33,932][67871] Updated weights for policy 1, policy_version 9610 (0.0009) [2023-10-07 20:02:34,306][67871] Updated weights for policy 1, policy_version 9620 (0.0008) [2023-10-07 20:02:34,678][67871] Updated weights for policy 1, policy_version 9630 (0.0009) [2023-10-07 20:02:36,346][67838] Updated weights for policy 0, policy_version 9602 (0.0009) [2023-10-07 20:02:36,720][67838] Updated weights for policy 0, policy_version 9612 (0.0009) [2023-10-07 20:02:37,080][67838] Updated weights for policy 0, policy_version 9622 (0.0009) [2023-10-07 20:02:37,464][67838] Updated weights for policy 0, policy_version 9632 (0.0009) [2023-10-07 20:02:37,476][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 19726336. Throughput: 0: 1640.5, 1: 1663.0. Samples: 4939644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:02:37,477][66916] Avg episode reward: [(0, '30.120'), (1, '32.650')] [2023-10-07 20:02:38,755][67871] Updated weights for policy 1, policy_version 9640 (0.0007) [2023-10-07 20:02:39,119][67871] Updated weights for policy 1, policy_version 9650 (0.0007) [2023-10-07 20:02:39,502][67871] Updated weights for policy 1, policy_version 9660 (0.0010) [2023-10-07 20:02:41,608][67838] Updated weights for policy 0, policy_version 9642 (0.0011) [2023-10-07 20:02:41,970][67838] Updated weights for policy 0, policy_version 9652 (0.0011) [2023-10-07 20:02:42,348][67838] Updated weights for policy 0, policy_version 9662 (0.0008) [2023-10-07 20:02:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 19791872. Throughput: 0: 1644.4, 1: 1653.2. Samples: 4949266. Policy #0 lag: (min: 25.0, avg: 35.2, max: 57.0) [2023-10-07 20:02:42,477][66916] Avg episode reward: [(0, '30.190'), (1, '31.910')] [2023-10-07 20:02:43,598][67871] Updated weights for policy 1, policy_version 9670 (0.0009) [2023-10-07 20:02:43,965][67871] Updated weights for policy 1, policy_version 9680 (0.0009) [2023-10-07 20:02:44,340][67871] Updated weights for policy 1, policy_version 9690 (0.0008) [2023-10-07 20:02:46,550][67838] Updated weights for policy 0, policy_version 9672 (0.0007) [2023-10-07 20:02:46,921][67838] Updated weights for policy 0, policy_version 9682 (0.0008) [2023-10-07 20:02:47,287][67838] Updated weights for policy 0, policy_version 9692 (0.0009) [2023-10-07 20:02:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 19857408. Throughput: 0: 1638.4, 1: 1664.7. Samples: 4969234. Policy #0 lag: (min: 25.0, avg: 35.2, max: 57.0) [2023-10-07 20:02:47,477][66916] Avg episode reward: [(0, '31.400'), (1, '31.510')] [2023-10-07 20:02:48,545][67871] Updated weights for policy 1, policy_version 9700 (0.0009) [2023-10-07 20:02:48,914][67871] Updated weights for policy 1, policy_version 9710 (0.0009) [2023-10-07 20:02:49,287][67871] Updated weights for policy 1, policy_version 9720 (0.0007) [2023-10-07 20:02:51,475][67838] Updated weights for policy 0, policy_version 9702 (0.0009) [2023-10-07 20:02:51,852][67838] Updated weights for policy 0, policy_version 9712 (0.0007) [2023-10-07 20:02:52,221][67838] Updated weights for policy 0, policy_version 9722 (0.0007) [2023-10-07 20:02:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 19922944. Throughput: 0: 1639.2, 1: 1660.2. Samples: 4988720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:02:52,477][66916] Avg episode reward: [(0, '30.990'), (1, '32.030')] [2023-10-07 20:02:53,471][67871] Updated weights for policy 1, policy_version 9730 (0.0008) [2023-10-07 20:02:53,844][67871] Updated weights for policy 1, policy_version 9740 (0.0007) [2023-10-07 20:02:54,202][67871] Updated weights for policy 1, policy_version 9750 (0.0009) [2023-10-07 20:02:54,571][67871] Updated weights for policy 1, policy_version 9760 (0.0008) [2023-10-07 20:02:56,542][67838] Updated weights for policy 0, policy_version 9732 (0.0007) [2023-10-07 20:02:56,924][67838] Updated weights for policy 0, policy_version 9742 (0.0007) [2023-10-07 20:02:57,301][67838] Updated weights for policy 0, policy_version 9752 (0.0007) [2023-10-07 20:02:57,477][66916] Fps is (10 sec: 9830.3, 60 sec: 12561.0, 300 sec: 13107.2). Total num frames: 19955712. Throughput: 0: 1642.3, 1: 1652.0. Samples: 4998488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:02:57,478][66916] Avg episode reward: [(0, '31.440'), (1, '31.930')] [2023-10-07 20:02:58,802][67871] Updated weights for policy 1, policy_version 9770 (0.0007) [2023-10-07 20:02:59,176][67871] Updated weights for policy 1, policy_version 9780 (0.0007) [2023-10-07 20:02:59,548][67871] Updated weights for policy 1, policy_version 9790 (0.0008) [2023-10-07 20:03:01,465][67838] Updated weights for policy 0, policy_version 9762 (0.0007) [2023-10-07 20:03:01,839][67838] Updated weights for policy 0, policy_version 9772 (0.0008) [2023-10-07 20:03:02,216][67838] Updated weights for policy 0, policy_version 9782 (0.0007) [2023-10-07 20:03:02,476][66916] Fps is (10 sec: 9830.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 20021248. Throughput: 0: 1637.9, 1: 1662.1. Samples: 5018570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:03:02,477][66916] Avg episode reward: [(0, '31.780'), (1, '30.250')] [2023-10-07 20:03:02,588][67838] Updated weights for policy 0, policy_version 9792 (0.0008) [2023-10-07 20:03:03,702][67871] Updated weights for policy 1, policy_version 9800 (0.0009) [2023-10-07 20:03:04,069][67871] Updated weights for policy 1, policy_version 9810 (0.0010) [2023-10-07 20:03:04,445][67871] Updated weights for policy 1, policy_version 9820 (0.0010) [2023-10-07 20:03:06,746][67838] Updated weights for policy 0, policy_version 9802 (0.0009) [2023-10-07 20:03:07,129][67838] Updated weights for policy 0, policy_version 9812 (0.0009) [2023-10-07 20:03:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 12561.0, 300 sec: 13107.2). Total num frames: 20086784. Throughput: 0: 1637.8, 1: 1659.5. Samples: 5038164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:03:07,478][66916] Avg episode reward: [(0, '31.190'), (1, '29.650')] [2023-10-07 20:03:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000009824_10059776.pth... [2023-10-07 20:03:07,506][67838] Updated weights for policy 0, policy_version 9822 (0.0009) [2023-10-07 20:03:07,529][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000008288_8486912.pth [2023-10-07 20:03:07,574][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000009824_10059776.pth... [2023-10-07 20:03:07,602][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000008288_8486912.pth [2023-10-07 20:03:08,672][67871] Updated weights for policy 1, policy_version 9830 (0.0009) [2023-10-07 20:03:09,037][67871] Updated weights for policy 1, policy_version 9840 (0.0010) [2023-10-07 20:03:09,415][67871] Updated weights for policy 1, policy_version 9850 (0.0009) [2023-10-07 20:03:11,695][67838] Updated weights for policy 0, policy_version 9832 (0.0008) [2023-10-07 20:03:12,064][67838] Updated weights for policy 0, policy_version 9842 (0.0008) [2023-10-07 20:03:12,437][67838] Updated weights for policy 0, policy_version 9852 (0.0008) [2023-10-07 20:03:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 20152320. Throughput: 0: 1640.4, 1: 1657.0. Samples: 5047820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:03:12,478][66916] Avg episode reward: [(0, '30.660'), (1, '30.830')] [2023-10-07 20:03:13,412][67871] Updated weights for policy 1, policy_version 9860 (0.0010) [2023-10-07 20:03:13,778][67871] Updated weights for policy 1, policy_version 9870 (0.0009) [2023-10-07 20:03:14,150][67871] Updated weights for policy 1, policy_version 9880 (0.0010) [2023-10-07 20:03:16,490][67838] Updated weights for policy 0, policy_version 9862 (0.0009) [2023-10-07 20:03:16,855][67838] Updated weights for policy 0, policy_version 9872 (0.0010) [2023-10-07 20:03:17,235][67838] Updated weights for policy 0, policy_version 9882 (0.0008) [2023-10-07 20:03:17,476][66916] Fps is (10 sec: 16384.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 20250624. Throughput: 0: 1635.8, 1: 1653.6. Samples: 5067886. Policy #0 lag: (min: 3.0, avg: 18.4, max: 35.0) [2023-10-07 20:03:17,477][66916] Avg episode reward: [(0, '31.760'), (1, '31.430')] [2023-10-07 20:03:18,311][67871] Updated weights for policy 1, policy_version 9890 (0.0008) [2023-10-07 20:03:18,687][67871] Updated weights for policy 1, policy_version 9900 (0.0008) [2023-10-07 20:03:19,062][67871] Updated weights for policy 1, policy_version 9910 (0.0008) [2023-10-07 20:03:19,423][67871] Updated weights for policy 1, policy_version 9920 (0.0009) [2023-10-07 20:03:21,427][67838] Updated weights for policy 0, policy_version 9892 (0.0009) [2023-10-07 20:03:21,797][67838] Updated weights for policy 0, policy_version 9902 (0.0010) [2023-10-07 20:03:22,176][67838] Updated weights for policy 0, policy_version 9912 (0.0008) [2023-10-07 20:03:22,477][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 20316160. Throughput: 0: 1642.5, 1: 1646.0. Samples: 5087626. Policy #0 lag: (min: 24.0, avg: 49.4, max: 56.0) [2023-10-07 20:03:22,477][66916] Avg episode reward: [(0, '30.900'), (1, '30.360')] [2023-10-07 20:03:23,582][67871] Updated weights for policy 1, policy_version 9930 (0.0009) [2023-10-07 20:03:23,951][67871] Updated weights for policy 1, policy_version 9940 (0.0010) [2023-10-07 20:03:24,330][67871] Updated weights for policy 1, policy_version 9950 (0.0010) [2023-10-07 20:03:26,343][67838] Updated weights for policy 0, policy_version 9922 (0.0008) [2023-10-07 20:03:26,736][67838] Updated weights for policy 0, policy_version 9932 (0.0007) [2023-10-07 20:03:27,108][67838] Updated weights for policy 0, policy_version 9942 (0.0007) [2023-10-07 20:03:27,477][66916] Fps is (10 sec: 9830.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 20348928. Throughput: 0: 1645.3, 1: 1646.8. Samples: 5097412. Policy #0 lag: (min: 24.0, avg: 49.4, max: 56.0) [2023-10-07 20:03:27,478][66916] Avg episode reward: [(0, '30.870'), (1, '32.080')] [2023-10-07 20:03:27,487][67838] Updated weights for policy 0, policy_version 9952 (0.0009) [2023-10-07 20:03:28,688][67871] Updated weights for policy 1, policy_version 9960 (0.0010) [2023-10-07 20:03:29,054][67871] Updated weights for policy 1, policy_version 9970 (0.0009) [2023-10-07 20:03:29,426][67871] Updated weights for policy 1, policy_version 9980 (0.0009) [2023-10-07 20:03:31,559][67838] Updated weights for policy 0, policy_version 9962 (0.0007) [2023-10-07 20:03:31,932][67838] Updated weights for policy 0, policy_version 9972 (0.0007) [2023-10-07 20:03:32,315][67838] Updated weights for policy 0, policy_version 9982 (0.0008) [2023-10-07 20:03:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 20447232. Throughput: 0: 1646.2, 1: 1641.9. Samples: 5117198. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-07 20:03:32,478][66916] Avg episode reward: [(0, '31.750'), (1, '31.660')] [2023-10-07 20:03:33,516][67871] Updated weights for policy 1, policy_version 9990 (0.0008) [2023-10-07 20:03:33,880][67871] Updated weights for policy 1, policy_version 10000 (0.0008) [2023-10-07 20:03:34,242][67871] Updated weights for policy 1, policy_version 10010 (0.0007) [2023-10-07 20:03:36,303][67838] Updated weights for policy 0, policy_version 9992 (0.0007) [2023-10-07 20:03:36,685][67838] Updated weights for policy 0, policy_version 10002 (0.0008) [2023-10-07 20:03:37,067][67838] Updated weights for policy 0, policy_version 10012 (0.0009) [2023-10-07 20:03:37,477][66916] Fps is (10 sec: 16383.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 20512768. Throughput: 0: 1644.2, 1: 1649.0. Samples: 5136912. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-07 20:03:37,478][66916] Avg episode reward: [(0, '30.970'), (1, '30.110')] [2023-10-07 20:03:38,294][67871] Updated weights for policy 1, policy_version 10020 (0.0007) [2023-10-07 20:03:38,664][67871] Updated weights for policy 1, policy_version 10030 (0.0009) [2023-10-07 20:03:39,028][67871] Updated weights for policy 1, policy_version 10040 (0.0009) [2023-10-07 20:03:41,077][67838] Updated weights for policy 0, policy_version 10022 (0.0008) [2023-10-07 20:03:41,440][67838] Updated weights for policy 0, policy_version 10032 (0.0008) [2023-10-07 20:03:41,813][67838] Updated weights for policy 0, policy_version 10042 (0.0009) [2023-10-07 20:03:42,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 20578304. Throughput: 0: 1647.8, 1: 1646.9. Samples: 5146750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:03:42,478][66916] Avg episode reward: [(0, '32.730'), (1, '31.980')] [2023-10-07 20:03:42,479][67511] Saving new best policy, reward=32.730! [2023-10-07 20:03:43,306][67871] Updated weights for policy 1, policy_version 10050 (0.0008) [2023-10-07 20:03:43,670][67871] Updated weights for policy 1, policy_version 10060 (0.0009) [2023-10-07 20:03:44,038][67871] Updated weights for policy 1, policy_version 10070 (0.0011) [2023-10-07 20:03:44,405][67871] Updated weights for policy 1, policy_version 10080 (0.0008) [2023-10-07 20:03:46,013][67838] Updated weights for policy 0, policy_version 10052 (0.0009) [2023-10-07 20:03:46,385][67838] Updated weights for policy 0, policy_version 10062 (0.0010) [2023-10-07 20:03:46,771][67838] Updated weights for policy 0, policy_version 10072 (0.0009) [2023-10-07 20:03:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 20643840. Throughput: 0: 1645.6, 1: 1645.2. Samples: 5166660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:03:47,477][66916] Avg episode reward: [(0, '32.710'), (1, '32.040')] [2023-10-07 20:03:48,535][67871] Updated weights for policy 1, policy_version 10090 (0.0010) [2023-10-07 20:03:48,908][67871] Updated weights for policy 1, policy_version 10100 (0.0009) [2023-10-07 20:03:49,277][67871] Updated weights for policy 1, policy_version 10110 (0.0009) [2023-10-07 20:03:51,016][67838] Updated weights for policy 0, policy_version 10082 (0.0009) [2023-10-07 20:03:51,396][67838] Updated weights for policy 0, policy_version 10092 (0.0007) [2023-10-07 20:03:51,774][67838] Updated weights for policy 0, policy_version 10102 (0.0007) [2023-10-07 20:03:52,157][67838] Updated weights for policy 0, policy_version 10112 (0.0008) [2023-10-07 20:03:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 20709376. Throughput: 0: 1641.2, 1: 1642.0. Samples: 5185904. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-07 20:03:52,478][66916] Avg episode reward: [(0, '31.440'), (1, '32.380')] [2023-10-07 20:03:53,334][67871] Updated weights for policy 1, policy_version 10120 (0.0008) [2023-10-07 20:03:53,697][67871] Updated weights for policy 1, policy_version 10130 (0.0008) [2023-10-07 20:03:54,069][67871] Updated weights for policy 1, policy_version 10140 (0.0009) [2023-10-07 20:03:56,342][67838] Updated weights for policy 0, policy_version 10122 (0.0008) [2023-10-07 20:03:56,719][67838] Updated weights for policy 0, policy_version 10132 (0.0007) [2023-10-07 20:03:57,095][67838] Updated weights for policy 0, policy_version 10142 (0.0007) [2023-10-07 20:03:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 20774912. Throughput: 0: 1650.4, 1: 1643.2. Samples: 5196032. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-07 20:03:57,478][66916] Avg episode reward: [(0, '32.850'), (1, '31.680')] [2023-10-07 20:03:57,479][67511] Saving new best policy, reward=32.850! [2023-10-07 20:03:58,208][67871] Updated weights for policy 1, policy_version 10150 (0.0008) [2023-10-07 20:03:58,584][67871] Updated weights for policy 1, policy_version 10160 (0.0007) [2023-10-07 20:03:58,957][67871] Updated weights for policy 1, policy_version 10170 (0.0008) [2023-10-07 20:04:01,235][67838] Updated weights for policy 0, policy_version 10152 (0.0008) [2023-10-07 20:04:01,616][67838] Updated weights for policy 0, policy_version 10162 (0.0008) [2023-10-07 20:04:01,991][67838] Updated weights for policy 0, policy_version 10172 (0.0009) [2023-10-07 20:04:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 20840448. Throughput: 0: 1649.3, 1: 1643.7. Samples: 5216072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:04:02,477][66916] Avg episode reward: [(0, '32.120'), (1, '32.960')] [2023-10-07 20:04:02,478][67676] Saving new best policy, reward=32.960! [2023-10-07 20:04:03,063][67871] Updated weights for policy 1, policy_version 10180 (0.0007) [2023-10-07 20:04:03,428][67871] Updated weights for policy 1, policy_version 10190 (0.0011) [2023-10-07 20:04:03,792][67871] Updated weights for policy 1, policy_version 10200 (0.0009) [2023-10-07 20:04:06,196][67838] Updated weights for policy 0, policy_version 10182 (0.0008) [2023-10-07 20:04:06,576][67838] Updated weights for policy 0, policy_version 10192 (0.0008) [2023-10-07 20:04:06,954][67838] Updated weights for policy 0, policy_version 10202 (0.0008) [2023-10-07 20:04:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 20905984. Throughput: 0: 1643.2, 1: 1643.0. Samples: 5235506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:04:07,477][66916] Avg episode reward: [(0, '29.490'), (1, '32.050')] [2023-10-07 20:04:07,941][67871] Updated weights for policy 1, policy_version 10210 (0.0010) [2023-10-07 20:04:08,312][67871] Updated weights for policy 1, policy_version 10220 (0.0007) [2023-10-07 20:04:08,687][67871] Updated weights for policy 1, policy_version 10230 (0.0007) [2023-10-07 20:04:09,056][67871] Updated weights for policy 1, policy_version 10240 (0.0009) [2023-10-07 20:04:11,244][67838] Updated weights for policy 0, policy_version 10212 (0.0008) [2023-10-07 20:04:11,646][67838] Updated weights for policy 0, policy_version 10222 (0.0008) [2023-10-07 20:04:12,021][67838] Updated weights for policy 0, policy_version 10232 (0.0007) [2023-10-07 20:04:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 20971520. Throughput: 0: 1646.3, 1: 1643.0. Samples: 5245430. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-07 20:04:12,477][66916] Avg episode reward: [(0, '32.370'), (1, '31.870')] [2023-10-07 20:04:13,335][67871] Updated weights for policy 1, policy_version 10250 (0.0009) [2023-10-07 20:04:13,696][67871] Updated weights for policy 1, policy_version 10260 (0.0007) [2023-10-07 20:04:14,064][67871] Updated weights for policy 1, policy_version 10270 (0.0009) [2023-10-07 20:04:16,165][67838] Updated weights for policy 0, policy_version 10242 (0.0009) [2023-10-07 20:04:16,535][67838] Updated weights for policy 0, policy_version 10252 (0.0009) [2023-10-07 20:04:16,916][67838] Updated weights for policy 0, policy_version 10262 (0.0007) [2023-10-07 20:04:17,284][67838] Updated weights for policy 0, policy_version 10272 (0.0007) [2023-10-07 20:04:17,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 21037056. Throughput: 0: 1642.6, 1: 1644.3. Samples: 5265110. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-07 20:04:17,477][66916] Avg episode reward: [(0, '31.860'), (1, '32.750')] [2023-10-07 20:04:18,525][67871] Updated weights for policy 1, policy_version 10280 (0.0009) [2023-10-07 20:04:18,890][67871] Updated weights for policy 1, policy_version 10290 (0.0007) [2023-10-07 20:04:19,261][67871] Updated weights for policy 1, policy_version 10300 (0.0008) [2023-10-07 20:04:21,534][67838] Updated weights for policy 0, policy_version 10282 (0.0007) [2023-10-07 20:04:21,908][67838] Updated weights for policy 0, policy_version 10292 (0.0007) [2023-10-07 20:04:22,283][67838] Updated weights for policy 0, policy_version 10302 (0.0008) [2023-10-07 20:04:22,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 21102592. Throughput: 0: 1644.3, 1: 1634.1. Samples: 5284442. Policy #0 lag: (min: 19.0, avg: 19.0, max: 23.0) [2023-10-07 20:04:22,478][66916] Avg episode reward: [(0, '31.380'), (1, '31.360')] [2023-10-07 20:04:23,533][67871] Updated weights for policy 1, policy_version 10310 (0.0008) [2023-10-07 20:04:23,885][67871] Updated weights for policy 1, policy_version 10320 (0.0008) [2023-10-07 20:04:24,249][67871] Updated weights for policy 1, policy_version 10330 (0.0010) [2023-10-07 20:04:26,476][67838] Updated weights for policy 0, policy_version 10312 (0.0010) [2023-10-07 20:04:26,853][67838] Updated weights for policy 0, policy_version 10322 (0.0008) [2023-10-07 20:04:27,224][67838] Updated weights for policy 0, policy_version 10332 (0.0008) [2023-10-07 20:04:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 21168128. Throughput: 0: 1640.0, 1: 1635.9. Samples: 5294164. Policy #0 lag: (min: 19.0, avg: 19.0, max: 23.0) [2023-10-07 20:04:27,478][66916] Avg episode reward: [(0, '31.530'), (1, '32.770')] [2023-10-07 20:04:28,412][67871] Updated weights for policy 1, policy_version 10340 (0.0009) [2023-10-07 20:04:28,781][67871] Updated weights for policy 1, policy_version 10350 (0.0009) [2023-10-07 20:04:29,146][67871] Updated weights for policy 1, policy_version 10360 (0.0007) [2023-10-07 20:04:31,336][67838] Updated weights for policy 0, policy_version 10342 (0.0008) [2023-10-07 20:04:31,699][67838] Updated weights for policy 0, policy_version 10352 (0.0012) [2023-10-07 20:04:32,080][67838] Updated weights for policy 0, policy_version 10362 (0.0011) [2023-10-07 20:04:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 21233664. Throughput: 0: 1644.9, 1: 1639.9. Samples: 5314476. Policy #0 lag: (min: 22.0, avg: 22.5, max: 37.0) [2023-10-07 20:04:32,478][66916] Avg episode reward: [(0, '33.430'), (1, '31.960')] [2023-10-07 20:04:32,479][67511] Saving new best policy, reward=33.430! [2023-10-07 20:04:33,220][67871] Updated weights for policy 1, policy_version 10370 (0.0008) [2023-10-07 20:04:33,590][67871] Updated weights for policy 1, policy_version 10380 (0.0008) [2023-10-07 20:04:33,955][67871] Updated weights for policy 1, policy_version 10390 (0.0009) [2023-10-07 20:04:34,330][67871] Updated weights for policy 1, policy_version 10400 (0.0008) [2023-10-07 20:04:36,055][67838] Updated weights for policy 0, policy_version 10372 (0.0008) [2023-10-07 20:04:36,424][67838] Updated weights for policy 0, policy_version 10382 (0.0007) [2023-10-07 20:04:36,800][67838] Updated weights for policy 0, policy_version 10392 (0.0007) [2023-10-07 20:04:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 21299200. Throughput: 0: 1640.5, 1: 1643.0. Samples: 5333660. Policy #0 lag: (min: 22.0, avg: 22.5, max: 37.0) [2023-10-07 20:04:37,477][66916] Avg episode reward: [(0, '29.700'), (1, '31.500')] [2023-10-07 20:04:38,457][67871] Updated weights for policy 1, policy_version 10410 (0.0007) [2023-10-07 20:04:38,823][67871] Updated weights for policy 1, policy_version 10420 (0.0008) [2023-10-07 20:04:39,186][67871] Updated weights for policy 1, policy_version 10430 (0.0008) [2023-10-07 20:04:40,940][67838] Updated weights for policy 0, policy_version 10402 (0.0007) [2023-10-07 20:04:41,308][67838] Updated weights for policy 0, policy_version 10412 (0.0010) [2023-10-07 20:04:41,686][67838] Updated weights for policy 0, policy_version 10422 (0.0009) [2023-10-07 20:04:42,054][67838] Updated weights for policy 0, policy_version 10432 (0.0010) [2023-10-07 20:04:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 21364736. Throughput: 0: 1640.1, 1: 1643.6. Samples: 5343802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:04:42,478][66916] Avg episode reward: [(0, '32.030'), (1, '31.690')] [2023-10-07 20:04:43,259][67871] Updated weights for policy 1, policy_version 10440 (0.0011) [2023-10-07 20:04:43,625][67871] Updated weights for policy 1, policy_version 10450 (0.0009) [2023-10-07 20:04:43,990][67871] Updated weights for policy 1, policy_version 10460 (0.0010) [2023-10-07 20:04:46,184][67838] Updated weights for policy 0, policy_version 10442 (0.0008) [2023-10-07 20:04:46,556][67838] Updated weights for policy 0, policy_version 10452 (0.0007) [2023-10-07 20:04:46,931][67838] Updated weights for policy 0, policy_version 10462 (0.0007) [2023-10-07 20:04:47,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 21430272. Throughput: 0: 1639.4, 1: 1642.4. Samples: 5363754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:04:47,478][66916] Avg episode reward: [(0, '30.580'), (1, '33.050')] [2023-10-07 20:04:47,479][67676] Saving new best policy, reward=33.050! [2023-10-07 20:04:48,226][67871] Updated weights for policy 1, policy_version 10470 (0.0008) [2023-10-07 20:04:48,593][67871] Updated weights for policy 1, policy_version 10480 (0.0007) [2023-10-07 20:04:48,964][67871] Updated weights for policy 1, policy_version 10490 (0.0008) [2023-10-07 20:04:51,226][67838] Updated weights for policy 0, policy_version 10472 (0.0008) [2023-10-07 20:04:51,603][67838] Updated weights for policy 0, policy_version 10482 (0.0009) [2023-10-07 20:04:51,969][67838] Updated weights for policy 0, policy_version 10492 (0.0007) [2023-10-07 20:04:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 21495808. Throughput: 0: 1641.3, 1: 1640.4. Samples: 5383182. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 20:04:52,477][66916] Avg episode reward: [(0, '32.820'), (1, '33.010')] [2023-10-07 20:04:53,164][67871] Updated weights for policy 1, policy_version 10500 (0.0008) [2023-10-07 20:04:53,530][67871] Updated weights for policy 1, policy_version 10510 (0.0008) [2023-10-07 20:04:53,905][67871] Updated weights for policy 1, policy_version 10520 (0.0007) [2023-10-07 20:04:56,224][67838] Updated weights for policy 0, policy_version 10502 (0.0007) [2023-10-07 20:04:56,608][67838] Updated weights for policy 0, policy_version 10512 (0.0007) [2023-10-07 20:04:56,976][67838] Updated weights for policy 0, policy_version 10522 (0.0007) [2023-10-07 20:04:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 21561344. Throughput: 0: 1644.2, 1: 1644.1. Samples: 5393402. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 20:04:57,477][66916] Avg episode reward: [(0, '32.470'), (1, '31.860')] [2023-10-07 20:04:58,096][67871] Updated weights for policy 1, policy_version 10530 (0.0007) [2023-10-07 20:04:58,519][67871] Updated weights for policy 1, policy_version 10540 (0.0010) [2023-10-07 20:04:58,894][67871] Updated weights for policy 1, policy_version 10550 (0.0010) [2023-10-07 20:04:59,258][67871] Updated weights for policy 1, policy_version 10560 (0.0011) [2023-10-07 20:05:01,009][67838] Updated weights for policy 0, policy_version 10532 (0.0007) [2023-10-07 20:05:01,385][67838] Updated weights for policy 0, policy_version 10542 (0.0007) [2023-10-07 20:05:01,759][67838] Updated weights for policy 0, policy_version 10552 (0.0008) [2023-10-07 20:05:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 21626880. Throughput: 0: 1650.1, 1: 1642.0. Samples: 5413254. Policy #0 lag: (min: 25.0, avg: 48.0, max: 57.0) [2023-10-07 20:05:02,477][66916] Avg episode reward: [(0, '32.620'), (1, '32.340')] [2023-10-07 20:05:03,359][67871] Updated weights for policy 1, policy_version 10570 (0.0008) [2023-10-07 20:05:03,729][67871] Updated weights for policy 1, policy_version 10580 (0.0007) [2023-10-07 20:05:04,106][67871] Updated weights for policy 1, policy_version 10590 (0.0007) [2023-10-07 20:05:05,833][67838] Updated weights for policy 0, policy_version 10562 (0.0008) [2023-10-07 20:05:06,215][67838] Updated weights for policy 0, policy_version 10572 (0.0010) [2023-10-07 20:05:06,595][67838] Updated weights for policy 0, policy_version 10582 (0.0011) [2023-10-07 20:05:06,967][67838] Updated weights for policy 0, policy_version 10592 (0.0010) [2023-10-07 20:05:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 21692416. Throughput: 0: 1639.6, 1: 1656.9. Samples: 5432782. Policy #0 lag: (min: 25.0, avg: 48.0, max: 57.0) [2023-10-07 20:05:07,477][66916] Avg episode reward: [(0, '31.020'), (1, '32.100')] [2023-10-07 20:05:07,487][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000010592_10846208.pth... [2023-10-07 20:05:07,487][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000010592_10846208.pth... [2023-10-07 20:05:07,527][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000009056_9273344.pth [2023-10-07 20:05:07,528][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000009056_9273344.pth [2023-10-07 20:05:08,030][67871] Updated weights for policy 1, policy_version 10600 (0.0010) [2023-10-07 20:05:08,401][67871] Updated weights for policy 1, policy_version 10610 (0.0010) [2023-10-07 20:05:08,768][67871] Updated weights for policy 1, policy_version 10620 (0.0009) [2023-10-07 20:05:11,076][67838] Updated weights for policy 0, policy_version 10602 (0.0007) [2023-10-07 20:05:11,454][67838] Updated weights for policy 0, policy_version 10612 (0.0011) [2023-10-07 20:05:11,823][67838] Updated weights for policy 0, policy_version 10622 (0.0009) [2023-10-07 20:05:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 21757952. Throughput: 0: 1648.1, 1: 1659.7. Samples: 5443016. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-07 20:05:12,477][66916] Avg episode reward: [(0, '31.410'), (1, '32.460')] [2023-10-07 20:05:12,849][67871] Updated weights for policy 1, policy_version 10630 (0.0007) [2023-10-07 20:05:13,218][67871] Updated weights for policy 1, policy_version 10640 (0.0007) [2023-10-07 20:05:13,591][67871] Updated weights for policy 1, policy_version 10650 (0.0007) [2023-10-07 20:05:16,054][67838] Updated weights for policy 0, policy_version 10632 (0.0009) [2023-10-07 20:05:16,421][67838] Updated weights for policy 0, policy_version 10642 (0.0008) [2023-10-07 20:05:16,805][67838] Updated weights for policy 0, policy_version 10652 (0.0008) [2023-10-07 20:05:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 21823488. Throughput: 0: 1644.3, 1: 1657.5. Samples: 5463058. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-07 20:05:17,477][66916] Avg episode reward: [(0, '29.680'), (1, '33.330')] [2023-10-07 20:05:17,478][67676] Saving new best policy, reward=33.330! [2023-10-07 20:05:17,815][67871] Updated weights for policy 1, policy_version 10660 (0.0008) [2023-10-07 20:05:18,181][67871] Updated weights for policy 1, policy_version 10670 (0.0008) [2023-10-07 20:05:18,552][67871] Updated weights for policy 1, policy_version 10680 (0.0008) [2023-10-07 20:05:20,938][67838] Updated weights for policy 0, policy_version 10662 (0.0008) [2023-10-07 20:05:21,318][67838] Updated weights for policy 0, policy_version 10672 (0.0007) [2023-10-07 20:05:21,688][67838] Updated weights for policy 0, policy_version 10682 (0.0007) [2023-10-07 20:05:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 21889024. Throughput: 0: 1645.5, 1: 1660.6. Samples: 5482436. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-10-07 20:05:22,477][66916] Avg episode reward: [(0, '30.640'), (1, '31.850')] [2023-10-07 20:05:22,531][67871] Updated weights for policy 1, policy_version 10690 (0.0008) [2023-10-07 20:05:22,899][67871] Updated weights for policy 1, policy_version 10700 (0.0007) [2023-10-07 20:05:23,269][67871] Updated weights for policy 1, policy_version 10710 (0.0008) [2023-10-07 20:05:23,645][67871] Updated weights for policy 1, policy_version 10720 (0.0007) [2023-10-07 20:05:25,933][67838] Updated weights for policy 0, policy_version 10692 (0.0008) [2023-10-07 20:05:26,298][67838] Updated weights for policy 0, policy_version 10702 (0.0008) [2023-10-07 20:05:26,676][67838] Updated weights for policy 0, policy_version 10712 (0.0009) [2023-10-07 20:05:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 21954560. Throughput: 0: 1649.1, 1: 1657.5. Samples: 5492600. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-10-07 20:05:27,478][66916] Avg episode reward: [(0, '31.620'), (1, '32.980')] [2023-10-07 20:05:27,857][67871] Updated weights for policy 1, policy_version 10730 (0.0010) [2023-10-07 20:05:28,231][67871] Updated weights for policy 1, policy_version 10740 (0.0009) [2023-10-07 20:05:28,602][67871] Updated weights for policy 1, policy_version 10750 (0.0010) [2023-10-07 20:05:30,852][67838] Updated weights for policy 0, policy_version 10722 (0.0008) [2023-10-07 20:05:31,227][67838] Updated weights for policy 0, policy_version 10732 (0.0010) [2023-10-07 20:05:31,591][67838] Updated weights for policy 0, policy_version 10742 (0.0008) [2023-10-07 20:05:31,965][67838] Updated weights for policy 0, policy_version 10752 (0.0010) [2023-10-07 20:05:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22020096. Throughput: 0: 1646.7, 1: 1659.3. Samples: 5512522. Policy #0 lag: (min: 2.0, avg: 5.3, max: 31.0) [2023-10-07 20:05:32,478][66916] Avg episode reward: [(0, '30.420'), (1, '31.580')] [2023-10-07 20:05:32,585][67871] Updated weights for policy 1, policy_version 10760 (0.0008) [2023-10-07 20:05:32,953][67871] Updated weights for policy 1, policy_version 10770 (0.0009) [2023-10-07 20:05:33,315][67871] Updated weights for policy 1, policy_version 10780 (0.0010) [2023-10-07 20:05:36,217][67838] Updated weights for policy 0, policy_version 10762 (0.0010) [2023-10-07 20:05:36,589][67838] Updated weights for policy 0, policy_version 10772 (0.0008) [2023-10-07 20:05:36,958][67838] Updated weights for policy 0, policy_version 10782 (0.0007) [2023-10-07 20:05:37,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22085632. Throughput: 0: 1643.0, 1: 1661.1. Samples: 5531868. Policy #0 lag: (min: 2.0, avg: 5.3, max: 31.0) [2023-10-07 20:05:37,478][66916] Avg episode reward: [(0, '32.090'), (1, '33.250')] [2023-10-07 20:05:37,618][67871] Updated weights for policy 1, policy_version 10790 (0.0008) [2023-10-07 20:05:37,977][67871] Updated weights for policy 1, policy_version 10800 (0.0008) [2023-10-07 20:05:38,347][67871] Updated weights for policy 1, policy_version 10810 (0.0007) [2023-10-07 20:05:41,117][67838] Updated weights for policy 0, policy_version 10792 (0.0008) [2023-10-07 20:05:41,505][67838] Updated weights for policy 0, policy_version 10802 (0.0008) [2023-10-07 20:05:41,873][67838] Updated weights for policy 0, policy_version 10812 (0.0009) [2023-10-07 20:05:42,418][67871] Updated weights for policy 1, policy_version 10820 (0.0007) [2023-10-07 20:05:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22151168. Throughput: 0: 1644.4, 1: 1660.7. Samples: 5542130. Policy #0 lag: (min: 29.0, avg: 31.5, max: 61.0) [2023-10-07 20:05:42,477][66916] Avg episode reward: [(0, '32.540'), (1, '32.570')] [2023-10-07 20:05:42,783][67871] Updated weights for policy 1, policy_version 10830 (0.0008) [2023-10-07 20:05:43,162][67871] Updated weights for policy 1, policy_version 10840 (0.0011) [2023-10-07 20:05:45,951][67838] Updated weights for policy 0, policy_version 10822 (0.0008) [2023-10-07 20:05:46,316][67838] Updated weights for policy 0, policy_version 10832 (0.0009) [2023-10-07 20:05:46,690][67838] Updated weights for policy 0, policy_version 10842 (0.0007) [2023-10-07 20:05:47,181][67871] Updated weights for policy 1, policy_version 10850 (0.0010) [2023-10-07 20:05:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22216704. Throughput: 0: 1634.7, 1: 1668.8. Samples: 5561912. Policy #0 lag: (min: 29.0, avg: 31.5, max: 61.0) [2023-10-07 20:05:47,477][66916] Avg episode reward: [(0, '31.250'), (1, '31.950')] [2023-10-07 20:05:47,554][67871] Updated weights for policy 1, policy_version 10860 (0.0008) [2023-10-07 20:05:47,919][67871] Updated weights for policy 1, policy_version 10870 (0.0009) [2023-10-07 20:05:48,282][67871] Updated weights for policy 1, policy_version 10880 (0.0008) [2023-10-07 20:05:50,721][67838] Updated weights for policy 0, policy_version 10852 (0.0009) [2023-10-07 20:05:51,096][67838] Updated weights for policy 0, policy_version 10862 (0.0010) [2023-10-07 20:05:51,464][67838] Updated weights for policy 0, policy_version 10872 (0.0008) [2023-10-07 20:05:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22282240. Throughput: 0: 1642.1, 1: 1653.4. Samples: 5581078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:05:52,477][66916] Avg episode reward: [(0, '32.290'), (1, '32.130')] [2023-10-07 20:05:52,592][67871] Updated weights for policy 1, policy_version 10890 (0.0008) [2023-10-07 20:05:52,962][67871] Updated weights for policy 1, policy_version 10900 (0.0008) [2023-10-07 20:05:53,327][67871] Updated weights for policy 1, policy_version 10910 (0.0009) [2023-10-07 20:05:55,694][67838] Updated weights for policy 0, policy_version 10882 (0.0010) [2023-10-07 20:05:56,071][67838] Updated weights for policy 0, policy_version 10892 (0.0008) [2023-10-07 20:05:56,447][67838] Updated weights for policy 0, policy_version 10902 (0.0009) [2023-10-07 20:05:56,822][67838] Updated weights for policy 0, policy_version 10912 (0.0008) [2023-10-07 20:05:57,447][67871] Updated weights for policy 1, policy_version 10920 (0.0007) [2023-10-07 20:05:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22347776. Throughput: 0: 1641.6, 1: 1653.2. Samples: 5591278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:05:57,477][66916] Avg episode reward: [(0, '32.580'), (1, '31.940')] [2023-10-07 20:05:57,820][67871] Updated weights for policy 1, policy_version 10930 (0.0007) [2023-10-07 20:05:58,194][67871] Updated weights for policy 1, policy_version 10940 (0.0007) [2023-10-07 20:06:00,727][67838] Updated weights for policy 0, policy_version 10922 (0.0010) [2023-10-07 20:06:01,107][67838] Updated weights for policy 0, policy_version 10932 (0.0009) [2023-10-07 20:06:01,480][67838] Updated weights for policy 0, policy_version 10942 (0.0007) [2023-10-07 20:06:02,395][67871] Updated weights for policy 1, policy_version 10950 (0.0008) [2023-10-07 20:06:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22413312. Throughput: 0: 1636.1, 1: 1652.2. Samples: 5611032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:06:02,477][66916] Avg episode reward: [(0, '32.690'), (1, '33.470')] [2023-10-07 20:06:02,765][67871] Updated weights for policy 1, policy_version 10960 (0.0008) [2023-10-07 20:06:03,128][67871] Updated weights for policy 1, policy_version 10970 (0.0007) [2023-10-07 20:06:03,345][67676] Saving new best policy, reward=33.470! [2023-10-07 20:06:05,345][67838] Updated weights for policy 0, policy_version 10952 (0.0007) [2023-10-07 20:06:05,719][67838] Updated weights for policy 0, policy_version 10962 (0.0008) [2023-10-07 20:06:06,082][67838] Updated weights for policy 0, policy_version 10972 (0.0011) [2023-10-07 20:06:07,358][67871] Updated weights for policy 1, policy_version 10980 (0.0008) [2023-10-07 20:06:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22478848. Throughput: 0: 1656.4, 1: 1647.2. Samples: 5631094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:06:07,477][66916] Avg episode reward: [(0, '31.830'), (1, '31.900')] [2023-10-07 20:06:07,726][67871] Updated weights for policy 1, policy_version 10990 (0.0008) [2023-10-07 20:06:08,100][67871] Updated weights for policy 1, policy_version 11000 (0.0010) [2023-10-07 20:06:10,121][67838] Updated weights for policy 0, policy_version 10982 (0.0008) [2023-10-07 20:06:10,495][67838] Updated weights for policy 0, policy_version 10992 (0.0008) [2023-10-07 20:06:10,871][67838] Updated weights for policy 0, policy_version 11002 (0.0009) [2023-10-07 20:06:12,166][67871] Updated weights for policy 1, policy_version 11010 (0.0010) [2023-10-07 20:06:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22544384. Throughput: 0: 1658.0, 1: 1647.1. Samples: 5641328. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-07 20:06:12,477][66916] Avg episode reward: [(0, '31.920'), (1, '33.110')] [2023-10-07 20:06:12,541][67871] Updated weights for policy 1, policy_version 11020 (0.0010) [2023-10-07 20:06:12,904][67871] Updated weights for policy 1, policy_version 11030 (0.0010) [2023-10-07 20:06:13,278][67871] Updated weights for policy 1, policy_version 11040 (0.0010) [2023-10-07 20:06:15,083][67838] Updated weights for policy 0, policy_version 11012 (0.0011) [2023-10-07 20:06:15,446][67838] Updated weights for policy 0, policy_version 11022 (0.0010) [2023-10-07 20:06:15,824][67838] Updated weights for policy 0, policy_version 11032 (0.0011) [2023-10-07 20:06:17,419][67871] Updated weights for policy 1, policy_version 11050 (0.0010) [2023-10-07 20:06:17,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22609920. Throughput: 0: 1643.9, 1: 1646.8. Samples: 5660602. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-07 20:06:17,477][66916] Avg episode reward: [(0, '31.090'), (1, '33.380')] [2023-10-07 20:06:17,788][67871] Updated weights for policy 1, policy_version 11060 (0.0009) [2023-10-07 20:06:18,155][67871] Updated weights for policy 1, policy_version 11070 (0.0008) [2023-10-07 20:06:20,086][67838] Updated weights for policy 0, policy_version 11042 (0.0009) [2023-10-07 20:06:20,463][67838] Updated weights for policy 0, policy_version 11052 (0.0008) [2023-10-07 20:06:20,834][67838] Updated weights for policy 0, policy_version 11062 (0.0008) [2023-10-07 20:06:21,209][67838] Updated weights for policy 0, policy_version 11072 (0.0010) [2023-10-07 20:06:22,461][67871] Updated weights for policy 1, policy_version 11080 (0.0009) [2023-10-07 20:06:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22675456. Throughput: 0: 1660.4, 1: 1646.3. Samples: 5680666. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-07 20:06:22,477][66916] Avg episode reward: [(0, '31.250'), (1, '33.030')] [2023-10-07 20:06:22,820][67871] Updated weights for policy 1, policy_version 11090 (0.0008) [2023-10-07 20:06:23,197][67871] Updated weights for policy 1, policy_version 11100 (0.0009) [2023-10-07 20:06:25,313][67838] Updated weights for policy 0, policy_version 11082 (0.0008) [2023-10-07 20:06:25,677][67838] Updated weights for policy 0, policy_version 11092 (0.0007) [2023-10-07 20:06:26,058][67838] Updated weights for policy 0, policy_version 11102 (0.0007) [2023-10-07 20:06:27,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22740992. Throughput: 0: 1662.0, 1: 1640.3. Samples: 5690736. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-07 20:06:27,477][66916] Avg episode reward: [(0, '31.430'), (1, '31.840')] [2023-10-07 20:06:27,620][67871] Updated weights for policy 1, policy_version 11110 (0.0008) [2023-10-07 20:06:27,987][67871] Updated weights for policy 1, policy_version 11120 (0.0008) [2023-10-07 20:06:28,364][67871] Updated weights for policy 1, policy_version 11130 (0.0009) [2023-10-07 20:06:30,148][67838] Updated weights for policy 0, policy_version 11112 (0.0011) [2023-10-07 20:06:30,530][67838] Updated weights for policy 0, policy_version 11122 (0.0010) [2023-10-07 20:06:30,897][67838] Updated weights for policy 0, policy_version 11132 (0.0010) [2023-10-07 20:06:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22806528. Throughput: 0: 1647.6, 1: 1634.7. Samples: 5709616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:06:32,477][66916] Avg episode reward: [(0, '30.310'), (1, '33.230')] [2023-10-07 20:06:32,688][67871] Updated weights for policy 1, policy_version 11140 (0.0008) [2023-10-07 20:06:33,081][67871] Updated weights for policy 1, policy_version 11150 (0.0011) [2023-10-07 20:06:33,456][67871] Updated weights for policy 1, policy_version 11160 (0.0010) [2023-10-07 20:06:35,168][67838] Updated weights for policy 0, policy_version 11142 (0.0009) [2023-10-07 20:06:35,545][67838] Updated weights for policy 0, policy_version 11152 (0.0008) [2023-10-07 20:06:35,917][67838] Updated weights for policy 0, policy_version 11162 (0.0009) [2023-10-07 20:06:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 22872064. Throughput: 0: 1669.9, 1: 1637.4. Samples: 5729904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:06:37,477][66916] Avg episode reward: [(0, '31.690'), (1, '33.340')] [2023-10-07 20:06:37,609][67871] Updated weights for policy 1, policy_version 11170 (0.0008) [2023-10-07 20:06:37,972][67871] Updated weights for policy 1, policy_version 11180 (0.0008) [2023-10-07 20:06:38,348][67871] Updated weights for policy 1, policy_version 11190 (0.0007) [2023-10-07 20:06:38,712][67871] Updated weights for policy 1, policy_version 11200 (0.0009) [2023-10-07 20:06:40,112][67838] Updated weights for policy 0, policy_version 11172 (0.0009) [2023-10-07 20:06:40,482][67838] Updated weights for policy 0, policy_version 11182 (0.0010) [2023-10-07 20:06:40,853][67838] Updated weights for policy 0, policy_version 11192 (0.0009) [2023-10-07 20:06:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22937600. Throughput: 0: 1665.4, 1: 1634.8. Samples: 5739788. Policy #0 lag: (min: 10.0, avg: 10.0, max: 11.0) [2023-10-07 20:06:42,477][66916] Avg episode reward: [(0, '31.360'), (1, '32.650')] [2023-10-07 20:06:42,853][67871] Updated weights for policy 1, policy_version 11210 (0.0010) [2023-10-07 20:06:43,219][67871] Updated weights for policy 1, policy_version 11220 (0.0010) [2023-10-07 20:06:43,585][67871] Updated weights for policy 1, policy_version 11230 (0.0010) [2023-10-07 20:06:45,021][67838] Updated weights for policy 0, policy_version 11202 (0.0008) [2023-10-07 20:06:45,395][67838] Updated weights for policy 0, policy_version 11212 (0.0008) [2023-10-07 20:06:45,773][67838] Updated weights for policy 0, policy_version 11222 (0.0010) [2023-10-07 20:06:46,139][67838] Updated weights for policy 0, policy_version 11232 (0.0008) [2023-10-07 20:06:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23003136. Throughput: 0: 1656.9, 1: 1637.6. Samples: 5759282. Policy #0 lag: (min: 10.0, avg: 10.0, max: 11.0) [2023-10-07 20:06:47,477][66916] Avg episode reward: [(0, '31.750'), (1, '33.360')] [2023-10-07 20:06:47,762][67871] Updated weights for policy 1, policy_version 11240 (0.0011) [2023-10-07 20:06:48,134][67871] Updated weights for policy 1, policy_version 11250 (0.0010) [2023-10-07 20:06:48,508][67871] Updated weights for policy 1, policy_version 11260 (0.0010) [2023-10-07 20:06:50,162][67838] Updated weights for policy 0, policy_version 11242 (0.0007) [2023-10-07 20:06:50,528][67838] Updated weights for policy 0, policy_version 11252 (0.0007) [2023-10-07 20:06:50,907][67838] Updated weights for policy 0, policy_version 11262 (0.0009) [2023-10-07 20:06:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 23068672. Throughput: 0: 1657.6, 1: 1638.0. Samples: 5779400. Policy #0 lag: (min: 28.0, avg: 35.7, max: 60.0) [2023-10-07 20:06:52,477][66916] Avg episode reward: [(0, '30.820'), (1, '32.160')] [2023-10-07 20:06:52,625][67871] Updated weights for policy 1, policy_version 11270 (0.0010) [2023-10-07 20:06:52,991][67871] Updated weights for policy 1, policy_version 11280 (0.0009) [2023-10-07 20:06:53,358][67871] Updated weights for policy 1, policy_version 11290 (0.0007) [2023-10-07 20:06:55,058][67838] Updated weights for policy 0, policy_version 11272 (0.0008) [2023-10-07 20:06:55,424][67838] Updated weights for policy 0, policy_version 11282 (0.0008) [2023-10-07 20:06:55,802][67838] Updated weights for policy 0, policy_version 11292 (0.0010) [2023-10-07 20:06:57,470][67871] Updated weights for policy 1, policy_version 11300 (0.0007) [2023-10-07 20:06:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23134208. Throughput: 0: 1650.6, 1: 1638.5. Samples: 5789338. Policy #0 lag: (min: 28.0, avg: 35.7, max: 60.0) [2023-10-07 20:06:57,478][66916] Avg episode reward: [(0, '31.660'), (1, '32.420')] [2023-10-07 20:06:57,839][67871] Updated weights for policy 1, policy_version 11310 (0.0007) [2023-10-07 20:06:58,198][67871] Updated weights for policy 1, policy_version 11320 (0.0008) [2023-10-07 20:07:00,059][67838] Updated weights for policy 0, policy_version 11302 (0.0009) [2023-10-07 20:07:00,432][67838] Updated weights for policy 0, policy_version 11312 (0.0008) [2023-10-07 20:07:00,805][67838] Updated weights for policy 0, policy_version 11322 (0.0010) [2023-10-07 20:07:02,214][67871] Updated weights for policy 1, policy_version 11330 (0.0008) [2023-10-07 20:07:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 23199744. Throughput: 0: 1651.3, 1: 1642.7. Samples: 5808834. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 20:07:02,478][66916] Avg episode reward: [(0, '30.810'), (1, '32.730')] [2023-10-07 20:07:02,589][67871] Updated weights for policy 1, policy_version 11340 (0.0009) [2023-10-07 20:07:02,960][67871] Updated weights for policy 1, policy_version 11350 (0.0008) [2023-10-07 20:07:03,339][67871] Updated weights for policy 1, policy_version 11360 (0.0010) [2023-10-07 20:07:04,807][67838] Updated weights for policy 0, policy_version 11332 (0.0009) [2023-10-07 20:07:05,180][67838] Updated weights for policy 0, policy_version 11342 (0.0010) [2023-10-07 20:07:05,557][67838] Updated weights for policy 0, policy_version 11352 (0.0010) [2023-10-07 20:07:07,433][67871] Updated weights for policy 1, policy_version 11370 (0.0008) [2023-10-07 20:07:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 23265280. Throughput: 0: 1661.3, 1: 1647.0. Samples: 5829540. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 20:07:07,478][66916] Avg episode reward: [(0, '33.520'), (1, '33.400')] [2023-10-07 20:07:07,487][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000011360_11632640.pth... [2023-10-07 20:07:07,519][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000009824_10059776.pth [2023-10-07 20:07:07,523][67511] Saving new best policy, reward=33.520! [2023-10-07 20:07:07,812][67871] Updated weights for policy 1, policy_version 11380 (0.0009) [2023-10-07 20:07:08,178][67871] Updated weights for policy 1, policy_version 11390 (0.0009) [2023-10-07 20:07:08,245][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000011392_11665408.pth... [2023-10-07 20:07:08,274][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000009824_10059776.pth [2023-10-07 20:07:09,627][67838] Updated weights for policy 0, policy_version 11362 (0.0009) [2023-10-07 20:07:09,999][67838] Updated weights for policy 0, policy_version 11372 (0.0008) [2023-10-07 20:07:10,377][67838] Updated weights for policy 0, policy_version 11382 (0.0008) [2023-10-07 20:07:10,754][67838] Updated weights for policy 0, policy_version 11392 (0.0008) [2023-10-07 20:07:12,283][67871] Updated weights for policy 1, policy_version 11400 (0.0010) [2023-10-07 20:07:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 23330816. Throughput: 0: 1651.1, 1: 1649.9. Samples: 5839278. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 20:07:12,478][66916] Avg episode reward: [(0, '31.870'), (1, '32.330')] [2023-10-07 20:07:12,658][67871] Updated weights for policy 1, policy_version 11410 (0.0008) [2023-10-07 20:07:13,032][67871] Updated weights for policy 1, policy_version 11420 (0.0007) [2023-10-07 20:07:14,938][67838] Updated weights for policy 0, policy_version 11402 (0.0007) [2023-10-07 20:07:15,311][67838] Updated weights for policy 0, policy_version 11412 (0.0011) [2023-10-07 20:07:15,679][67838] Updated weights for policy 0, policy_version 11422 (0.0009) [2023-10-07 20:07:17,155][67871] Updated weights for policy 1, policy_version 11430 (0.0007) [2023-10-07 20:07:17,477][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23396352. Throughput: 0: 1657.9, 1: 1658.7. Samples: 5858864. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-10-07 20:07:17,478][66916] Avg episode reward: [(0, '31.780'), (1, '31.790')] [2023-10-07 20:07:17,520][67871] Updated weights for policy 1, policy_version 11440 (0.0007) [2023-10-07 20:07:17,885][67871] Updated weights for policy 1, policy_version 11450 (0.0007) [2023-10-07 20:07:19,721][67838] Updated weights for policy 0, policy_version 11432 (0.0008) [2023-10-07 20:07:20,104][67838] Updated weights for policy 0, policy_version 11442 (0.0007) [2023-10-07 20:07:20,468][67838] Updated weights for policy 0, policy_version 11452 (0.0009) [2023-10-07 20:07:22,075][67871] Updated weights for policy 1, policy_version 11460 (0.0007) [2023-10-07 20:07:22,458][67871] Updated weights for policy 1, policy_version 11470 (0.0009) [2023-10-07 20:07:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23461888. Throughput: 0: 1655.8, 1: 1662.0. Samples: 5879204. Policy #0 lag: (min: 24.0, avg: 51.8, max: 56.0) [2023-10-07 20:07:22,477][66916] Avg episode reward: [(0, '31.550'), (1, '32.160')] [2023-10-07 20:07:22,834][67871] Updated weights for policy 1, policy_version 11480 (0.0009) [2023-10-07 20:07:24,573][67838] Updated weights for policy 0, policy_version 11462 (0.0009) [2023-10-07 20:07:24,938][67838] Updated weights for policy 0, policy_version 11472 (0.0009) [2023-10-07 20:07:25,317][67838] Updated weights for policy 0, policy_version 11482 (0.0009) [2023-10-07 20:07:26,792][67871] Updated weights for policy 1, policy_version 11490 (0.0010) [2023-10-07 20:07:27,160][67871] Updated weights for policy 1, policy_version 11500 (0.0011) [2023-10-07 20:07:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 23527424. Throughput: 0: 1646.2, 1: 1662.1. Samples: 5888662. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-10-07 20:07:27,477][66916] Avg episode reward: [(0, '30.830'), (1, '31.590')] [2023-10-07 20:07:27,537][67871] Updated weights for policy 1, policy_version 11510 (0.0008) [2023-10-07 20:07:27,903][67871] Updated weights for policy 1, policy_version 11520 (0.0009) [2023-10-07 20:07:29,588][67838] Updated weights for policy 0, policy_version 11492 (0.0008) [2023-10-07 20:07:29,969][67838] Updated weights for policy 0, policy_version 11502 (0.0007) [2023-10-07 20:07:30,337][67838] Updated weights for policy 0, policy_version 11512 (0.0010) [2023-10-07 20:07:32,016][67871] Updated weights for policy 1, policy_version 11530 (0.0010) [2023-10-07 20:07:32,387][67871] Updated weights for policy 1, policy_version 11540 (0.0008) [2023-10-07 20:07:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 23592960. Throughput: 0: 1655.3, 1: 1664.1. Samples: 5908654. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-10-07 20:07:32,478][66916] Avg episode reward: [(0, '32.330'), (1, '33.240')] [2023-10-07 20:07:32,761][67871] Updated weights for policy 1, policy_version 11550 (0.0008) [2023-10-07 20:07:34,420][67838] Updated weights for policy 0, policy_version 11522 (0.0009) [2023-10-07 20:07:34,789][67838] Updated weights for policy 0, policy_version 11532 (0.0007) [2023-10-07 20:07:35,159][67838] Updated weights for policy 0, policy_version 11542 (0.0008) [2023-10-07 20:07:35,537][67838] Updated weights for policy 0, policy_version 11552 (0.0008) [2023-10-07 20:07:36,889][67871] Updated weights for policy 1, policy_version 11560 (0.0009) [2023-10-07 20:07:37,253][67871] Updated weights for policy 1, policy_version 11570 (0.0008) [2023-10-07 20:07:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 23658496. Throughput: 0: 1661.8, 1: 1658.7. Samples: 5928824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:07:37,477][66916] Avg episode reward: [(0, '32.700'), (1, '32.500')] [2023-10-07 20:07:37,629][67871] Updated weights for policy 1, policy_version 11580 (0.0011) [2023-10-07 20:07:39,580][67838] Updated weights for policy 0, policy_version 11562 (0.0008) [2023-10-07 20:07:39,954][67838] Updated weights for policy 0, policy_version 11572 (0.0008) [2023-10-07 20:07:40,329][67838] Updated weights for policy 0, policy_version 11582 (0.0008) [2023-10-07 20:07:41,716][67871] Updated weights for policy 1, policy_version 11590 (0.0008) [2023-10-07 20:07:42,089][67871] Updated weights for policy 1, policy_version 11600 (0.0009) [2023-10-07 20:07:42,461][67871] Updated weights for policy 1, policy_version 11610 (0.0007) [2023-10-07 20:07:42,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 23724032. Throughput: 0: 1648.8, 1: 1667.9. Samples: 5938590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:07:42,478][66916] Avg episode reward: [(0, '29.710'), (1, '33.300')] [2023-10-07 20:07:44,589][67838] Updated weights for policy 0, policy_version 11592 (0.0008) [2023-10-07 20:07:44,965][67838] Updated weights for policy 0, policy_version 11602 (0.0010) [2023-10-07 20:07:45,352][67838] Updated weights for policy 0, policy_version 11612 (0.0010) [2023-10-07 20:07:46,681][67871] Updated weights for policy 1, policy_version 11620 (0.0008) [2023-10-07 20:07:47,048][67871] Updated weights for policy 1, policy_version 11630 (0.0010) [2023-10-07 20:07:47,424][67871] Updated weights for policy 1, policy_version 11640 (0.0007) [2023-10-07 20:07:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 23789568. Throughput: 0: 1654.3, 1: 1664.9. Samples: 5958200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:07:47,477][66916] Avg episode reward: [(0, '32.830'), (1, '34.470')] [2023-10-07 20:07:47,711][67676] Saving new best policy, reward=34.470! [2023-10-07 20:07:49,596][67838] Updated weights for policy 0, policy_version 11622 (0.0010) [2023-10-07 20:07:49,960][67838] Updated weights for policy 0, policy_version 11632 (0.0009) [2023-10-07 20:07:50,336][67838] Updated weights for policy 0, policy_version 11642 (0.0009) [2023-10-07 20:07:51,522][67871] Updated weights for policy 1, policy_version 11650 (0.0007) [2023-10-07 20:07:51,901][67871] Updated weights for policy 1, policy_version 11660 (0.0008) [2023-10-07 20:07:52,269][67871] Updated weights for policy 1, policy_version 11670 (0.0010) [2023-10-07 20:07:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23855104. Throughput: 0: 1647.2, 1: 1651.7. Samples: 5977988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:07:52,477][66916] Avg episode reward: [(0, '31.420'), (1, '33.140')] [2023-10-07 20:07:52,636][67871] Updated weights for policy 1, policy_version 11680 (0.0009) [2023-10-07 20:07:54,641][67838] Updated weights for policy 0, policy_version 11652 (0.0009) [2023-10-07 20:07:55,012][67838] Updated weights for policy 0, policy_version 11662 (0.0010) [2023-10-07 20:07:55,379][67838] Updated weights for policy 0, policy_version 11672 (0.0009) [2023-10-07 20:07:56,850][67871] Updated weights for policy 1, policy_version 11690 (0.0007) [2023-10-07 20:07:57,211][67871] Updated weights for policy 1, policy_version 11700 (0.0007) [2023-10-07 20:07:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 23920640. Throughput: 0: 1642.4, 1: 1660.9. Samples: 5987928. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-07 20:07:57,477][66916] Avg episode reward: [(0, '31.620'), (1, '34.130')] [2023-10-07 20:07:57,583][67871] Updated weights for policy 1, policy_version 11710 (0.0008) [2023-10-07 20:07:59,461][67838] Updated weights for policy 0, policy_version 11682 (0.0010) [2023-10-07 20:07:59,835][67838] Updated weights for policy 0, policy_version 11692 (0.0008) [2023-10-07 20:08:00,214][67838] Updated weights for policy 0, policy_version 11702 (0.0010) [2023-10-07 20:08:00,582][67838] Updated weights for policy 0, policy_version 11712 (0.0010) [2023-10-07 20:08:01,803][67871] Updated weights for policy 1, policy_version 11720 (0.0008) [2023-10-07 20:08:02,180][67871] Updated weights for policy 1, policy_version 11730 (0.0007) [2023-10-07 20:08:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23986176. Throughput: 0: 1647.7, 1: 1657.0. Samples: 6007574. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-07 20:08:02,477][66916] Avg episode reward: [(0, '32.110'), (1, '33.370')] [2023-10-07 20:08:02,547][67871] Updated weights for policy 1, policy_version 11740 (0.0007) [2023-10-07 20:08:04,858][67838] Updated weights for policy 0, policy_version 11722 (0.0008) [2023-10-07 20:08:05,227][67838] Updated weights for policy 0, policy_version 11732 (0.0008) [2023-10-07 20:08:05,615][67838] Updated weights for policy 0, policy_version 11742 (0.0008) [2023-10-07 20:08:06,650][67871] Updated weights for policy 1, policy_version 11750 (0.0008) [2023-10-07 20:08:07,039][67871] Updated weights for policy 1, policy_version 11760 (0.0009) [2023-10-07 20:08:07,404][67871] Updated weights for policy 1, policy_version 11770 (0.0009) [2023-10-07 20:08:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 24051712. Throughput: 0: 1649.5, 1: 1645.5. Samples: 6027476. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-07 20:08:07,478][66916] Avg episode reward: [(0, '32.760'), (1, '33.760')] [2023-10-07 20:08:09,541][67838] Updated weights for policy 0, policy_version 11752 (0.0010) [2023-10-07 20:08:09,911][67838] Updated weights for policy 0, policy_version 11762 (0.0008) [2023-10-07 20:08:10,287][67838] Updated weights for policy 0, policy_version 11772 (0.0007) [2023-10-07 20:08:11,489][67871] Updated weights for policy 1, policy_version 11780 (0.0008) [2023-10-07 20:08:11,861][67871] Updated weights for policy 1, policy_version 11790 (0.0007) [2023-10-07 20:08:12,238][67871] Updated weights for policy 1, policy_version 11800 (0.0008) [2023-10-07 20:08:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 24117248. Throughput: 0: 1643.2, 1: 1656.0. Samples: 6037128. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-10-07 20:08:12,477][66916] Avg episode reward: [(0, '31.140'), (1, '33.000')] [2023-10-07 20:08:14,438][67838] Updated weights for policy 0, policy_version 11782 (0.0010) [2023-10-07 20:08:14,804][67838] Updated weights for policy 0, policy_version 11792 (0.0011) [2023-10-07 20:08:15,181][67838] Updated weights for policy 0, policy_version 11802 (0.0009) [2023-10-07 20:08:16,470][67871] Updated weights for policy 1, policy_version 11810 (0.0007) [2023-10-07 20:08:16,840][67871] Updated weights for policy 1, policy_version 11820 (0.0008) [2023-10-07 20:08:17,213][67871] Updated weights for policy 1, policy_version 11830 (0.0008) [2023-10-07 20:08:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 24182784. Throughput: 0: 1646.8, 1: 1652.1. Samples: 6057100. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-10-07 20:08:17,477][66916] Avg episode reward: [(0, '32.360'), (1, '32.320')] [2023-10-07 20:08:17,581][67871] Updated weights for policy 1, policy_version 11840 (0.0008) [2023-10-07 20:08:19,373][67838] Updated weights for policy 0, policy_version 11812 (0.0010) [2023-10-07 20:08:19,746][67838] Updated weights for policy 0, policy_version 11822 (0.0007) [2023-10-07 20:08:20,123][67838] Updated weights for policy 0, policy_version 11832 (0.0007) [2023-10-07 20:08:21,774][67871] Updated weights for policy 1, policy_version 11850 (0.0009) [2023-10-07 20:08:22,149][67871] Updated weights for policy 1, policy_version 11860 (0.0008) [2023-10-07 20:08:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 24248320. Throughput: 0: 1639.3, 1: 1650.9. Samples: 6076886. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:08:22,477][66916] Avg episode reward: [(0, '32.610'), (1, '31.600')] [2023-10-07 20:08:22,511][67871] Updated weights for policy 1, policy_version 11870 (0.0009) [2023-10-07 20:08:24,260][67838] Updated weights for policy 0, policy_version 11842 (0.0010) [2023-10-07 20:08:24,629][67838] Updated weights for policy 0, policy_version 11852 (0.0011) [2023-10-07 20:08:25,001][67838] Updated weights for policy 0, policy_version 11862 (0.0011) [2023-10-07 20:08:25,375][67838] Updated weights for policy 0, policy_version 11872 (0.0008) [2023-10-07 20:08:26,676][67871] Updated weights for policy 1, policy_version 11880 (0.0008) [2023-10-07 20:08:27,036][67871] Updated weights for policy 1, policy_version 11890 (0.0007) [2023-10-07 20:08:27,401][67871] Updated weights for policy 1, policy_version 11900 (0.0007) [2023-10-07 20:08:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 24313856. Throughput: 0: 1638.5, 1: 1652.8. Samples: 6086696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:08:27,478][66916] Avg episode reward: [(0, '32.980'), (1, '31.240')] [2023-10-07 20:08:29,540][67838] Updated weights for policy 0, policy_version 11882 (0.0008) [2023-10-07 20:08:29,907][67838] Updated weights for policy 0, policy_version 11892 (0.0008) [2023-10-07 20:08:30,274][67838] Updated weights for policy 0, policy_version 11902 (0.0010) [2023-10-07 20:08:31,621][67871] Updated weights for policy 1, policy_version 11910 (0.0008) [2023-10-07 20:08:31,986][67871] Updated weights for policy 1, policy_version 11920 (0.0010) [2023-10-07 20:08:32,356][67871] Updated weights for policy 1, policy_version 11930 (0.0009) [2023-10-07 20:08:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 24379392. Throughput: 0: 1650.2, 1: 1648.6. Samples: 6106644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:08:32,477][66916] Avg episode reward: [(0, '30.800'), (1, '31.400')] [2023-10-07 20:08:34,387][67838] Updated weights for policy 0, policy_version 11912 (0.0011) [2023-10-07 20:08:34,756][67838] Updated weights for policy 0, policy_version 11922 (0.0009) [2023-10-07 20:08:35,132][67838] Updated weights for policy 0, policy_version 11932 (0.0011) [2023-10-07 20:08:36,468][67871] Updated weights for policy 1, policy_version 11940 (0.0008) [2023-10-07 20:08:36,846][67871] Updated weights for policy 1, policy_version 11950 (0.0007) [2023-10-07 20:08:37,211][67871] Updated weights for policy 1, policy_version 11960 (0.0010) [2023-10-07 20:08:37,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 24444928. Throughput: 0: 1645.6, 1: 1644.8. Samples: 6126056. Policy #0 lag: (min: 1.0, avg: 12.4, max: 33.0) [2023-10-07 20:08:37,477][66916] Avg episode reward: [(0, '32.990'), (1, '33.020')] [2023-10-07 20:08:39,396][67838] Updated weights for policy 0, policy_version 11942 (0.0012) [2023-10-07 20:08:39,775][67838] Updated weights for policy 0, policy_version 11952 (0.0010) [2023-10-07 20:08:40,147][67838] Updated weights for policy 0, policy_version 11962 (0.0011) [2023-10-07 20:08:41,389][67871] Updated weights for policy 1, policy_version 11970 (0.0010) [2023-10-07 20:08:41,762][67871] Updated weights for policy 1, policy_version 11980 (0.0008) [2023-10-07 20:08:42,143][67871] Updated weights for policy 1, policy_version 11990 (0.0012) [2023-10-07 20:08:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 24510464. Throughput: 0: 1640.7, 1: 1649.9. Samples: 6136002. Policy #0 lag: (min: 1.0, avg: 12.4, max: 33.0) [2023-10-07 20:08:42,477][66916] Avg episode reward: [(0, '30.620'), (1, '32.600')] [2023-10-07 20:08:42,513][67871] Updated weights for policy 1, policy_version 12000 (0.0009) [2023-10-07 20:08:44,339][67838] Updated weights for policy 0, policy_version 11972 (0.0010) [2023-10-07 20:08:44,709][67838] Updated weights for policy 0, policy_version 11982 (0.0010) [2023-10-07 20:08:45,085][67838] Updated weights for policy 0, policy_version 11992 (0.0011) [2023-10-07 20:08:46,605][67871] Updated weights for policy 1, policy_version 12010 (0.0008) [2023-10-07 20:08:46,981][67871] Updated weights for policy 1, policy_version 12020 (0.0008) [2023-10-07 20:08:47,345][67871] Updated weights for policy 1, policy_version 12030 (0.0007) [2023-10-07 20:08:47,476][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 24608768. Throughput: 0: 1640.9, 1: 1652.8. Samples: 6155790. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-07 20:08:47,477][66916] Avg episode reward: [(0, '32.520'), (1, '34.150')] [2023-10-07 20:08:49,088][67838] Updated weights for policy 0, policy_version 12002 (0.0010) [2023-10-07 20:08:49,482][67838] Updated weights for policy 0, policy_version 12012 (0.0009) [2023-10-07 20:08:49,863][67838] Updated weights for policy 0, policy_version 12022 (0.0011) [2023-10-07 20:08:50,230][67838] Updated weights for policy 0, policy_version 12032 (0.0011) [2023-10-07 20:08:51,505][67871] Updated weights for policy 1, policy_version 12040 (0.0009) [2023-10-07 20:08:51,877][67871] Updated weights for policy 1, policy_version 12050 (0.0009) [2023-10-07 20:08:52,257][67871] Updated weights for policy 1, policy_version 12060 (0.0009) [2023-10-07 20:08:52,477][66916] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 24674304. Throughput: 0: 1645.4, 1: 1649.3. Samples: 6175740. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-07 20:08:52,478][66916] Avg episode reward: [(0, '32.850'), (1, '33.320')] [2023-10-07 20:08:54,399][67838] Updated weights for policy 0, policy_version 12042 (0.0010) [2023-10-07 20:08:54,770][67838] Updated weights for policy 0, policy_version 12052 (0.0010) [2023-10-07 20:08:55,149][67838] Updated weights for policy 0, policy_version 12062 (0.0009) [2023-10-07 20:08:56,504][67871] Updated weights for policy 1, policy_version 12070 (0.0010) [2023-10-07 20:08:56,896][67871] Updated weights for policy 1, policy_version 12080 (0.0007) [2023-10-07 20:08:57,266][67871] Updated weights for policy 1, policy_version 12090 (0.0008) [2023-10-07 20:08:57,477][66916] Fps is (10 sec: 9830.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 24707072. Throughput: 0: 1641.9, 1: 1654.6. Samples: 6185472. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-07 20:08:57,478][66916] Avg episode reward: [(0, '31.730'), (1, '32.650')] [2023-10-07 20:08:59,142][67838] Updated weights for policy 0, policy_version 12072 (0.0009) [2023-10-07 20:08:59,520][67838] Updated weights for policy 0, policy_version 12082 (0.0010) [2023-10-07 20:08:59,888][67838] Updated weights for policy 0, policy_version 12092 (0.0008) [2023-10-07 20:09:01,350][67871] Updated weights for policy 1, policy_version 12100 (0.0009) [2023-10-07 20:09:01,717][67871] Updated weights for policy 1, policy_version 12110 (0.0009) [2023-10-07 20:09:02,089][67871] Updated weights for policy 1, policy_version 12120 (0.0008) [2023-10-07 20:09:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 24805376. Throughput: 0: 1647.4, 1: 1653.7. Samples: 6205650. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-07 20:09:02,478][66916] Avg episode reward: [(0, '32.680'), (1, '33.140')] [2023-10-07 20:09:04,026][67838] Updated weights for policy 0, policy_version 12102 (0.0009) [2023-10-07 20:09:04,391][67838] Updated weights for policy 0, policy_version 12112 (0.0011) [2023-10-07 20:09:04,770][67838] Updated weights for policy 0, policy_version 12122 (0.0009) [2023-10-07 20:09:06,181][67871] Updated weights for policy 1, policy_version 12130 (0.0011) [2023-10-07 20:09:06,557][67871] Updated weights for policy 1, policy_version 12140 (0.0010) [2023-10-07 20:09:06,929][67871] Updated weights for policy 1, policy_version 12150 (0.0009) [2023-10-07 20:09:07,295][67871] Updated weights for policy 1, policy_version 12160 (0.0009) [2023-10-07 20:09:07,477][66916] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 24870912. Throughput: 0: 1651.1, 1: 1645.9. Samples: 6225248. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-07 20:09:07,478][66916] Avg episode reward: [(0, '31.480'), (1, '32.300')] [2023-10-07 20:09:07,488][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000012160_12451840.pth... [2023-10-07 20:09:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000012128_12419072.pth... [2023-10-07 20:09:07,529][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000010592_10846208.pth [2023-10-07 20:09:07,530][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000010592_10846208.pth [2023-10-07 20:09:08,998][67838] Updated weights for policy 0, policy_version 12132 (0.0008) [2023-10-07 20:09:09,377][67838] Updated weights for policy 0, policy_version 12142 (0.0010) [2023-10-07 20:09:09,749][67838] Updated weights for policy 0, policy_version 12152 (0.0010) [2023-10-07 20:09:11,413][67871] Updated weights for policy 1, policy_version 12170 (0.0008) [2023-10-07 20:09:11,794][67871] Updated weights for policy 1, policy_version 12180 (0.0010) [2023-10-07 20:09:12,157][67871] Updated weights for policy 1, policy_version 12190 (0.0009) [2023-10-07 20:09:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 24936448. Throughput: 0: 1641.7, 1: 1652.1. Samples: 6234918. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-07 20:09:12,477][66916] Avg episode reward: [(0, '31.300'), (1, '32.280')] [2023-10-07 20:09:13,830][67838] Updated weights for policy 0, policy_version 12162 (0.0008) [2023-10-07 20:09:14,202][67838] Updated weights for policy 0, policy_version 12172 (0.0011) [2023-10-07 20:09:14,569][67838] Updated weights for policy 0, policy_version 12182 (0.0008) [2023-10-07 20:09:14,951][67838] Updated weights for policy 0, policy_version 12192 (0.0008) [2023-10-07 20:09:16,265][67871] Updated weights for policy 1, policy_version 12200 (0.0008) [2023-10-07 20:09:16,641][67871] Updated weights for policy 1, policy_version 12210 (0.0010) [2023-10-07 20:09:16,998][67871] Updated weights for policy 1, policy_version 12220 (0.0010) [2023-10-07 20:09:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 25001984. Throughput: 0: 1645.8, 1: 1658.2. Samples: 6255322. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-07 20:09:17,477][66916] Avg episode reward: [(0, '32.120'), (1, '33.430')] [2023-10-07 20:09:19,007][67838] Updated weights for policy 0, policy_version 12202 (0.0008) [2023-10-07 20:09:19,380][67838] Updated weights for policy 0, policy_version 12212 (0.0008) [2023-10-07 20:09:19,752][67838] Updated weights for policy 0, policy_version 12222 (0.0007) [2023-10-07 20:09:21,110][67871] Updated weights for policy 1, policy_version 12230 (0.0007) [2023-10-07 20:09:21,476][67871] Updated weights for policy 1, policy_version 12240 (0.0007) [2023-10-07 20:09:21,847][67871] Updated weights for policy 1, policy_version 12250 (0.0009) [2023-10-07 20:09:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 25067520. Throughput: 0: 1664.5, 1: 1647.5. Samples: 6275094. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-07 20:09:22,478][66916] Avg episode reward: [(0, '33.360'), (1, '32.490')] [2023-10-07 20:09:23,809][67838] Updated weights for policy 0, policy_version 12232 (0.0008) [2023-10-07 20:09:24,182][67838] Updated weights for policy 0, policy_version 12242 (0.0010) [2023-10-07 20:09:24,562][67838] Updated weights for policy 0, policy_version 12252 (0.0011) [2023-10-07 20:09:26,076][67871] Updated weights for policy 1, policy_version 12260 (0.0009) [2023-10-07 20:09:26,444][67871] Updated weights for policy 1, policy_version 12270 (0.0008) [2023-10-07 20:09:26,815][67871] Updated weights for policy 1, policy_version 12280 (0.0009) [2023-10-07 20:09:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 25133056. Throughput: 0: 1656.4, 1: 1653.1. Samples: 6284930. Policy #0 lag: (min: 8.0, avg: 26.3, max: 40.0) [2023-10-07 20:09:27,477][66916] Avg episode reward: [(0, '31.660'), (1, '33.460')] [2023-10-07 20:09:28,727][67838] Updated weights for policy 0, policy_version 12262 (0.0009) [2023-10-07 20:09:29,104][67838] Updated weights for policy 0, policy_version 12272 (0.0009) [2023-10-07 20:09:29,469][67838] Updated weights for policy 0, policy_version 12282 (0.0009) [2023-10-07 20:09:30,976][67871] Updated weights for policy 1, policy_version 12290 (0.0010) [2023-10-07 20:09:31,351][67871] Updated weights for policy 1, policy_version 12300 (0.0007) [2023-10-07 20:09:31,720][67871] Updated weights for policy 1, policy_version 12310 (0.0007) [2023-10-07 20:09:32,083][67871] Updated weights for policy 1, policy_version 12320 (0.0007) [2023-10-07 20:09:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 25198592. Throughput: 0: 1665.8, 1: 1654.3. Samples: 6305192. Policy #0 lag: (min: 8.0, avg: 26.3, max: 40.0) [2023-10-07 20:09:32,477][66916] Avg episode reward: [(0, '33.200'), (1, '33.260')] [2023-10-07 20:09:33,571][67838] Updated weights for policy 0, policy_version 12292 (0.0008) [2023-10-07 20:09:33,937][67838] Updated weights for policy 0, policy_version 12302 (0.0011) [2023-10-07 20:09:34,308][67838] Updated weights for policy 0, policy_version 12312 (0.0011) [2023-10-07 20:09:36,094][67871] Updated weights for policy 1, policy_version 12330 (0.0010) [2023-10-07 20:09:36,460][67871] Updated weights for policy 1, policy_version 12340 (0.0011) [2023-10-07 20:09:36,824][67871] Updated weights for policy 1, policy_version 12350 (0.0007) [2023-10-07 20:09:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 25264128. Throughput: 0: 1662.8, 1: 1642.6. Samples: 6324480. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 20:09:37,477][66916] Avg episode reward: [(0, '31.840'), (1, '33.200')] [2023-10-07 20:09:38,620][67838] Updated weights for policy 0, policy_version 12322 (0.0011) [2023-10-07 20:09:39,024][67838] Updated weights for policy 0, policy_version 12332 (0.0008) [2023-10-07 20:09:39,397][67838] Updated weights for policy 0, policy_version 12342 (0.0007) [2023-10-07 20:09:39,764][67838] Updated weights for policy 0, policy_version 12352 (0.0007) [2023-10-07 20:09:41,134][67871] Updated weights for policy 1, policy_version 12360 (0.0008) [2023-10-07 20:09:41,509][67871] Updated weights for policy 1, policy_version 12370 (0.0007) [2023-10-07 20:09:41,884][67871] Updated weights for policy 1, policy_version 12380 (0.0009) [2023-10-07 20:09:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 25329664. Throughput: 0: 1657.3, 1: 1656.0. Samples: 6334568. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 20:09:42,477][66916] Avg episode reward: [(0, '32.720'), (1, '33.400')] [2023-10-07 20:09:43,896][67838] Updated weights for policy 0, policy_version 12362 (0.0009) [2023-10-07 20:09:44,269][67838] Updated weights for policy 0, policy_version 12372 (0.0008) [2023-10-07 20:09:44,630][67838] Updated weights for policy 0, policy_version 12382 (0.0007) [2023-10-07 20:09:45,981][67871] Updated weights for policy 1, policy_version 12390 (0.0009) [2023-10-07 20:09:46,356][67871] Updated weights for policy 1, policy_version 12400 (0.0009) [2023-10-07 20:09:46,721][67871] Updated weights for policy 1, policy_version 12410 (0.0007) [2023-10-07 20:09:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 25395200. Throughput: 0: 1662.2, 1: 1651.8. Samples: 6354780. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 20:09:47,477][66916] Avg episode reward: [(0, '32.870'), (1, '34.690')] [2023-10-07 20:09:47,478][67676] Saving new best policy, reward=34.690! [2023-10-07 20:09:48,818][67838] Updated weights for policy 0, policy_version 12392 (0.0009) [2023-10-07 20:09:49,196][67838] Updated weights for policy 0, policy_version 12402 (0.0007) [2023-10-07 20:09:49,576][67838] Updated weights for policy 0, policy_version 12412 (0.0011) [2023-10-07 20:09:50,893][67871] Updated weights for policy 1, policy_version 12420 (0.0009) [2023-10-07 20:09:51,266][67871] Updated weights for policy 1, policy_version 12430 (0.0009) [2023-10-07 20:09:51,633][67871] Updated weights for policy 1, policy_version 12440 (0.0008) [2023-10-07 20:09:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 25460736. Throughput: 0: 1662.5, 1: 1645.0. Samples: 6374082. Policy #0 lag: (min: 25.0, avg: 38.9, max: 57.0) [2023-10-07 20:09:52,477][66916] Avg episode reward: [(0, '30.840'), (1, '33.300')] [2023-10-07 20:09:53,611][67838] Updated weights for policy 0, policy_version 12422 (0.0008) [2023-10-07 20:09:53,985][67838] Updated weights for policy 0, policy_version 12432 (0.0009) [2023-10-07 20:09:54,363][67838] Updated weights for policy 0, policy_version 12442 (0.0010) [2023-10-07 20:09:55,814][67871] Updated weights for policy 1, policy_version 12450 (0.0008) [2023-10-07 20:09:56,191][67871] Updated weights for policy 1, policy_version 12460 (0.0008) [2023-10-07 20:09:56,560][67871] Updated weights for policy 1, policy_version 12470 (0.0007) [2023-10-07 20:09:56,921][67871] Updated weights for policy 1, policy_version 12480 (0.0009) [2023-10-07 20:09:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 25526272. Throughput: 0: 1664.7, 1: 1652.0. Samples: 6384174. Policy #0 lag: (min: 25.0, avg: 38.9, max: 57.0) [2023-10-07 20:09:57,478][66916] Avg episode reward: [(0, '30.760'), (1, '34.070')] [2023-10-07 20:09:58,526][67838] Updated weights for policy 0, policy_version 12452 (0.0007) [2023-10-07 20:09:58,905][67838] Updated weights for policy 0, policy_version 12462 (0.0008) [2023-10-07 20:09:59,275][67838] Updated weights for policy 0, policy_version 12472 (0.0008) [2023-10-07 20:10:00,831][67871] Updated weights for policy 1, policy_version 12490 (0.0008) [2023-10-07 20:10:01,207][67871] Updated weights for policy 1, policy_version 12500 (0.0009) [2023-10-07 20:10:01,570][67871] Updated weights for policy 1, policy_version 12510 (0.0009) [2023-10-07 20:10:02,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 25591808. Throughput: 0: 1662.9, 1: 1640.7. Samples: 6403984. Policy #0 lag: (min: 25.0, avg: 38.9, max: 57.0) [2023-10-07 20:10:02,477][66916] Avg episode reward: [(0, '32.710'), (1, '33.900')] [2023-10-07 20:10:03,424][67838] Updated weights for policy 0, policy_version 12482 (0.0007) [2023-10-07 20:10:03,802][67838] Updated weights for policy 0, policy_version 12492 (0.0008) [2023-10-07 20:10:04,181][67838] Updated weights for policy 0, policy_version 12502 (0.0009) [2023-10-07 20:10:04,553][67838] Updated weights for policy 0, policy_version 12512 (0.0010) [2023-10-07 20:10:05,716][67871] Updated weights for policy 1, policy_version 12520 (0.0009) [2023-10-07 20:10:06,095][67871] Updated weights for policy 1, policy_version 12530 (0.0008) [2023-10-07 20:10:06,470][67871] Updated weights for policy 1, policy_version 12540 (0.0008) [2023-10-07 20:10:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 25657344. Throughput: 0: 1650.8, 1: 1649.8. Samples: 6423622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:10:07,478][66916] Avg episode reward: [(0, '30.320'), (1, '32.950')] [2023-10-07 20:10:08,652][67838] Updated weights for policy 0, policy_version 12522 (0.0008) [2023-10-07 20:10:09,025][67838] Updated weights for policy 0, policy_version 12532 (0.0007) [2023-10-07 20:10:09,395][67838] Updated weights for policy 0, policy_version 12542 (0.0008) [2023-10-07 20:10:10,578][67871] Updated weights for policy 1, policy_version 12550 (0.0009) [2023-10-07 20:10:10,945][67871] Updated weights for policy 1, policy_version 12560 (0.0009) [2023-10-07 20:10:11,316][67871] Updated weights for policy 1, policy_version 12570 (0.0008) [2023-10-07 20:10:12,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 25722880. Throughput: 0: 1648.8, 1: 1658.6. Samples: 6433764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:10:12,478][66916] Avg episode reward: [(0, '34.860'), (1, '32.770')] [2023-10-07 20:10:12,479][67511] Saving new best policy, reward=34.860! [2023-10-07 20:10:13,643][67838] Updated weights for policy 0, policy_version 12552 (0.0007) [2023-10-07 20:10:14,012][67838] Updated weights for policy 0, policy_version 12562 (0.0010) [2023-10-07 20:10:14,394][67838] Updated weights for policy 0, policy_version 12572 (0.0007) [2023-10-07 20:10:15,507][67871] Updated weights for policy 1, policy_version 12580 (0.0009) [2023-10-07 20:10:15,883][67871] Updated weights for policy 1, policy_version 12590 (0.0009) [2023-10-07 20:10:16,250][67871] Updated weights for policy 1, policy_version 12600 (0.0008) [2023-10-07 20:10:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 25788416. Throughput: 0: 1655.8, 1: 1643.9. Samples: 6453678. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:10:17,477][66916] Avg episode reward: [(0, '32.010'), (1, '32.780')] [2023-10-07 20:10:18,197][67838] Updated weights for policy 0, policy_version 12582 (0.0008) [2023-10-07 20:10:18,584][67838] Updated weights for policy 0, policy_version 12592 (0.0008) [2023-10-07 20:10:18,948][67838] Updated weights for policy 0, policy_version 12602 (0.0009) [2023-10-07 20:10:20,318][67871] Updated weights for policy 1, policy_version 12610 (0.0008) [2023-10-07 20:10:20,690][67871] Updated weights for policy 1, policy_version 12620 (0.0008) [2023-10-07 20:10:21,055][67871] Updated weights for policy 1, policy_version 12630 (0.0010) [2023-10-07 20:10:21,424][67871] Updated weights for policy 1, policy_version 12640 (0.0011) [2023-10-07 20:10:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 25853952. Throughput: 0: 1651.0, 1: 1656.1. Samples: 6473300. Policy #0 lag: (min: 9.0, avg: 29.2, max: 41.0) [2023-10-07 20:10:22,477][66916] Avg episode reward: [(0, '32.190'), (1, '32.330')] [2023-10-07 20:10:23,301][67838] Updated weights for policy 0, policy_version 12612 (0.0010) [2023-10-07 20:10:23,693][67838] Updated weights for policy 0, policy_version 12622 (0.0009) [2023-10-07 20:10:24,067][67838] Updated weights for policy 0, policy_version 12632 (0.0009) [2023-10-07 20:10:25,546][67871] Updated weights for policy 1, policy_version 12650 (0.0008) [2023-10-07 20:10:25,917][67871] Updated weights for policy 1, policy_version 12660 (0.0007) [2023-10-07 20:10:26,280][67871] Updated weights for policy 1, policy_version 12670 (0.0008) [2023-10-07 20:10:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 25919488. Throughput: 0: 1649.6, 1: 1657.7. Samples: 6483396. Policy #0 lag: (min: 9.0, avg: 29.2, max: 41.0) [2023-10-07 20:10:27,477][66916] Avg episode reward: [(0, '32.640'), (1, '32.690')] [2023-10-07 20:10:28,222][67838] Updated weights for policy 0, policy_version 12642 (0.0009) [2023-10-07 20:10:28,592][67838] Updated weights for policy 0, policy_version 12652 (0.0008) [2023-10-07 20:10:28,963][67838] Updated weights for policy 0, policy_version 12662 (0.0009) [2023-10-07 20:10:29,337][67838] Updated weights for policy 0, policy_version 12672 (0.0009) [2023-10-07 20:10:30,370][67871] Updated weights for policy 1, policy_version 12680 (0.0008) [2023-10-07 20:10:30,753][67871] Updated weights for policy 1, policy_version 12690 (0.0011) [2023-10-07 20:10:31,112][67871] Updated weights for policy 1, policy_version 12700 (0.0007) [2023-10-07 20:10:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 25985024. Throughput: 0: 1650.4, 1: 1643.6. Samples: 6503006. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-07 20:10:32,477][66916] Avg episode reward: [(0, '30.120'), (1, '32.980')] [2023-10-07 20:10:33,441][67838] Updated weights for policy 0, policy_version 12682 (0.0009) [2023-10-07 20:10:33,809][67838] Updated weights for policy 0, policy_version 12692 (0.0008) [2023-10-07 20:10:34,173][67838] Updated weights for policy 0, policy_version 12702 (0.0009) [2023-10-07 20:10:35,261][67871] Updated weights for policy 1, policy_version 12710 (0.0007) [2023-10-07 20:10:35,628][67871] Updated weights for policy 1, policy_version 12720 (0.0008) [2023-10-07 20:10:36,015][67871] Updated weights for policy 1, policy_version 12730 (0.0007) [2023-10-07 20:10:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 26050560. Throughput: 0: 1652.7, 1: 1662.7. Samples: 6523272. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-07 20:10:37,477][66916] Avg episode reward: [(0, '33.360'), (1, '34.210')] [2023-10-07 20:10:38,386][67838] Updated weights for policy 0, policy_version 12712 (0.0007) [2023-10-07 20:10:38,771][67838] Updated weights for policy 0, policy_version 12722 (0.0008) [2023-10-07 20:10:39,140][67838] Updated weights for policy 0, policy_version 12732 (0.0007) [2023-10-07 20:10:40,121][67871] Updated weights for policy 1, policy_version 12740 (0.0008) [2023-10-07 20:10:40,487][67871] Updated weights for policy 1, policy_version 12750 (0.0008) [2023-10-07 20:10:40,854][67871] Updated weights for policy 1, policy_version 12760 (0.0007) [2023-10-07 20:10:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 26116096. Throughput: 0: 1651.5, 1: 1666.8. Samples: 6533498. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-07 20:10:42,477][66916] Avg episode reward: [(0, '31.830'), (1, '32.300')] [2023-10-07 20:10:43,143][67838] Updated weights for policy 0, policy_version 12742 (0.0008) [2023-10-07 20:10:43,519][67838] Updated weights for policy 0, policy_version 12752 (0.0007) [2023-10-07 20:10:43,890][67838] Updated weights for policy 0, policy_version 12762 (0.0008) [2023-10-07 20:10:45,047][67871] Updated weights for policy 1, policy_version 12770 (0.0008) [2023-10-07 20:10:45,413][67871] Updated weights for policy 1, policy_version 12780 (0.0010) [2023-10-07 20:10:45,781][67871] Updated weights for policy 1, policy_version 12790 (0.0011) [2023-10-07 20:10:46,155][67871] Updated weights for policy 1, policy_version 12800 (0.0009) [2023-10-07 20:10:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 26181632. Throughput: 0: 1655.1, 1: 1661.3. Samples: 6553220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:10:47,477][66916] Avg episode reward: [(0, '31.080'), (1, '33.470')] [2023-10-07 20:10:48,100][67838] Updated weights for policy 0, policy_version 12772 (0.0009) [2023-10-07 20:10:48,476][67838] Updated weights for policy 0, policy_version 12782 (0.0009) [2023-10-07 20:10:48,859][67838] Updated weights for policy 0, policy_version 12792 (0.0008) [2023-10-07 20:10:50,185][67871] Updated weights for policy 1, policy_version 12810 (0.0008) [2023-10-07 20:10:50,550][67871] Updated weights for policy 1, policy_version 12820 (0.0008) [2023-10-07 20:10:50,921][67871] Updated weights for policy 1, policy_version 12830 (0.0008) [2023-10-07 20:10:52,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 26247168. Throughput: 0: 1656.5, 1: 1669.1. Samples: 6573274. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:10:52,477][66916] Avg episode reward: [(0, '33.370'), (1, '34.290')] [2023-10-07 20:10:52,809][67838] Updated weights for policy 0, policy_version 12802 (0.0009) [2023-10-07 20:10:53,192][67838] Updated weights for policy 0, policy_version 12812 (0.0009) [2023-10-07 20:10:53,567][67838] Updated weights for policy 0, policy_version 12822 (0.0009) [2023-10-07 20:10:53,939][67838] Updated weights for policy 0, policy_version 12832 (0.0008) [2023-10-07 20:10:55,008][67871] Updated weights for policy 1, policy_version 12840 (0.0008) [2023-10-07 20:10:55,372][67871] Updated weights for policy 1, policy_version 12850 (0.0007) [2023-10-07 20:10:55,736][67871] Updated weights for policy 1, policy_version 12860 (0.0010) [2023-10-07 20:10:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 26312704. Throughput: 0: 1661.7, 1: 1673.0. Samples: 6583826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:10:57,478][66916] Avg episode reward: [(0, '32.940'), (1, '33.130')] [2023-10-07 20:10:58,200][67838] Updated weights for policy 0, policy_version 12842 (0.0011) [2023-10-07 20:10:58,575][67838] Updated weights for policy 0, policy_version 12852 (0.0008) [2023-10-07 20:10:58,955][67838] Updated weights for policy 0, policy_version 12862 (0.0007) [2023-10-07 20:10:59,955][67871] Updated weights for policy 1, policy_version 12870 (0.0009) [2023-10-07 20:11:00,329][67871] Updated weights for policy 1, policy_version 12880 (0.0008) [2023-10-07 20:11:00,693][67871] Updated weights for policy 1, policy_version 12890 (0.0007) [2023-10-07 20:11:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 26378240. Throughput: 0: 1658.6, 1: 1659.9. Samples: 6603010. Policy #0 lag: (min: 24.0, avg: 47.9, max: 56.0) [2023-10-07 20:11:02,477][66916] Avg episode reward: [(0, '31.120'), (1, '34.690')] [2023-10-07 20:11:03,031][67838] Updated weights for policy 0, policy_version 12872 (0.0011) [2023-10-07 20:11:03,413][67838] Updated weights for policy 0, policy_version 12882 (0.0007) [2023-10-07 20:11:03,793][67838] Updated weights for policy 0, policy_version 12892 (0.0009) [2023-10-07 20:11:04,746][67871] Updated weights for policy 1, policy_version 12900 (0.0007) [2023-10-07 20:11:05,115][67871] Updated weights for policy 1, policy_version 12910 (0.0007) [2023-10-07 20:11:05,488][67871] Updated weights for policy 1, policy_version 12920 (0.0010) [2023-10-07 20:11:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 26443776. Throughput: 0: 1661.5, 1: 1671.0. Samples: 6623264. Policy #0 lag: (min: 24.0, avg: 47.9, max: 56.0) [2023-10-07 20:11:07,477][66916] Avg episode reward: [(0, '32.150'), (1, '33.440')] [2023-10-07 20:11:07,486][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000012928_13238272.pth... [2023-10-07 20:11:07,486][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000012896_13205504.pth... [2023-10-07 20:11:07,526][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000011360_11632640.pth [2023-10-07 20:11:07,526][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000011392_11665408.pth [2023-10-07 20:11:08,032][67838] Updated weights for policy 0, policy_version 12902 (0.0009) [2023-10-07 20:11:08,420][67838] Updated weights for policy 0, policy_version 12912 (0.0008) [2023-10-07 20:11:08,793][67838] Updated weights for policy 0, policy_version 12922 (0.0007) [2023-10-07 20:11:09,672][67871] Updated weights for policy 1, policy_version 12930 (0.0009) [2023-10-07 20:11:10,047][67871] Updated weights for policy 1, policy_version 12940 (0.0007) [2023-10-07 20:11:10,423][67871] Updated weights for policy 1, policy_version 12950 (0.0007) [2023-10-07 20:11:10,790][67871] Updated weights for policy 1, policy_version 12960 (0.0008) [2023-10-07 20:11:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 26509312. Throughput: 0: 1664.3, 1: 1662.1. Samples: 6633084. Policy #0 lag: (min: 24.0, avg: 47.9, max: 56.0) [2023-10-07 20:11:12,477][66916] Avg episode reward: [(0, '32.080'), (1, '34.960')] [2023-10-07 20:11:12,478][67676] Saving new best policy, reward=34.960! [2023-10-07 20:11:12,740][67838] Updated weights for policy 0, policy_version 12932 (0.0008) [2023-10-07 20:11:13,114][67838] Updated weights for policy 0, policy_version 12942 (0.0009) [2023-10-07 20:11:13,474][67838] Updated weights for policy 0, policy_version 12952 (0.0007) [2023-10-07 20:11:14,893][67871] Updated weights for policy 1, policy_version 12970 (0.0008) [2023-10-07 20:11:15,261][67871] Updated weights for policy 1, policy_version 12980 (0.0010) [2023-10-07 20:11:15,626][67871] Updated weights for policy 1, policy_version 12990 (0.0007) [2023-10-07 20:11:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 26574848. Throughput: 0: 1663.0, 1: 1662.6. Samples: 6652658. Policy #0 lag: (min: 1.0, avg: 7.2, max: 33.0) [2023-10-07 20:11:17,478][66916] Avg episode reward: [(0, '32.630'), (1, '33.320')] [2023-10-07 20:11:17,637][67838] Updated weights for policy 0, policy_version 12962 (0.0007) [2023-10-07 20:11:18,014][67838] Updated weights for policy 0, policy_version 12972 (0.0011) [2023-10-07 20:11:18,385][67838] Updated weights for policy 0, policy_version 12982 (0.0009) [2023-10-07 20:11:18,757][67838] Updated weights for policy 0, policy_version 12992 (0.0010) [2023-10-07 20:11:19,772][67871] Updated weights for policy 1, policy_version 13000 (0.0007) [2023-10-07 20:11:20,146][67871] Updated weights for policy 1, policy_version 13010 (0.0009) [2023-10-07 20:11:20,515][67871] Updated weights for policy 1, policy_version 13020 (0.0007) [2023-10-07 20:11:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 26640384. Throughput: 0: 1657.7, 1: 1664.7. Samples: 6672784. Policy #0 lag: (min: 1.0, avg: 7.2, max: 33.0) [2023-10-07 20:11:22,478][66916] Avg episode reward: [(0, '32.200'), (1, '33.150')] [2023-10-07 20:11:22,943][67838] Updated weights for policy 0, policy_version 13002 (0.0008) [2023-10-07 20:11:23,321][67838] Updated weights for policy 0, policy_version 13012 (0.0010) [2023-10-07 20:11:23,702][67838] Updated weights for policy 0, policy_version 13022 (0.0009) [2023-10-07 20:11:24,709][67871] Updated weights for policy 1, policy_version 13030 (0.0008) [2023-10-07 20:11:25,087][67871] Updated weights for policy 1, policy_version 13040 (0.0009) [2023-10-07 20:11:25,459][67871] Updated weights for policy 1, policy_version 13050 (0.0009) [2023-10-07 20:11:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 26705920. Throughput: 0: 1654.8, 1: 1656.4. Samples: 6682506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:11:27,478][66916] Avg episode reward: [(0, '30.520'), (1, '34.150')] [2023-10-07 20:11:27,861][67838] Updated weights for policy 0, policy_version 13032 (0.0008) [2023-10-07 20:11:28,228][67838] Updated weights for policy 0, policy_version 13042 (0.0012) [2023-10-07 20:11:28,607][67838] Updated weights for policy 0, policy_version 13052 (0.0010) [2023-10-07 20:11:29,475][67871] Updated weights for policy 1, policy_version 13060 (0.0009) [2023-10-07 20:11:29,841][67871] Updated weights for policy 1, policy_version 13070 (0.0007) [2023-10-07 20:11:30,213][67871] Updated weights for policy 1, policy_version 13080 (0.0009) [2023-10-07 20:11:32,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 26771456. Throughput: 0: 1652.0, 1: 1652.8. Samples: 6701936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:11:32,477][66916] Avg episode reward: [(0, '32.500'), (1, '33.680')] [2023-10-07 20:11:32,626][67838] Updated weights for policy 0, policy_version 13062 (0.0009) [2023-10-07 20:11:33,005][67838] Updated weights for policy 0, policy_version 13072 (0.0011) [2023-10-07 20:11:33,387][67838] Updated weights for policy 0, policy_version 13082 (0.0008) [2023-10-07 20:11:34,322][67871] Updated weights for policy 1, policy_version 13090 (0.0008) [2023-10-07 20:11:34,690][67871] Updated weights for policy 1, policy_version 13100 (0.0008) [2023-10-07 20:11:35,060][67871] Updated weights for policy 1, policy_version 13110 (0.0010) [2023-10-07 20:11:35,431][67871] Updated weights for policy 1, policy_version 13120 (0.0008) [2023-10-07 20:11:37,414][67838] Updated weights for policy 0, policy_version 13092 (0.0008) [2023-10-07 20:11:37,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 26836992. Throughput: 0: 1658.0, 1: 1661.2. Samples: 6722638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:11:37,477][66916] Avg episode reward: [(0, '30.810'), (1, '34.510')] [2023-10-07 20:11:37,783][67838] Updated weights for policy 0, policy_version 13102 (0.0007) [2023-10-07 20:11:38,160][67838] Updated weights for policy 0, policy_version 13112 (0.0008) [2023-10-07 20:11:39,497][67871] Updated weights for policy 1, policy_version 13130 (0.0007) [2023-10-07 20:11:39,860][67871] Updated weights for policy 1, policy_version 13140 (0.0009) [2023-10-07 20:11:40,236][67871] Updated weights for policy 1, policy_version 13150 (0.0008) [2023-10-07 20:11:42,315][67838] Updated weights for policy 0, policy_version 13122 (0.0007) [2023-10-07 20:11:42,477][66916] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 26902528. Throughput: 0: 1654.1, 1: 1640.3. Samples: 6732074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:11:42,478][66916] Avg episode reward: [(0, '32.690'), (1, '34.880')] [2023-10-07 20:11:42,688][67838] Updated weights for policy 0, policy_version 13132 (0.0011) [2023-10-07 20:11:43,058][67838] Updated weights for policy 0, policy_version 13142 (0.0010) [2023-10-07 20:11:43,436][67838] Updated weights for policy 0, policy_version 13152 (0.0009) [2023-10-07 20:11:44,340][67871] Updated weights for policy 1, policy_version 13160 (0.0007) [2023-10-07 20:11:44,714][67871] Updated weights for policy 1, policy_version 13170 (0.0007) [2023-10-07 20:11:45,084][67871] Updated weights for policy 1, policy_version 13180 (0.0008) [2023-10-07 20:11:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 26968064. Throughput: 0: 1656.2, 1: 1653.8. Samples: 6751958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:11:47,477][66916] Avg episode reward: [(0, '30.600'), (1, '34.240')] [2023-10-07 20:11:47,627][67838] Updated weights for policy 0, policy_version 13162 (0.0009) [2023-10-07 20:11:47,993][67838] Updated weights for policy 0, policy_version 13172 (0.0008) [2023-10-07 20:11:48,377][67838] Updated weights for policy 0, policy_version 13182 (0.0010) [2023-10-07 20:11:49,219][67871] Updated weights for policy 1, policy_version 13190 (0.0009) [2023-10-07 20:11:49,591][67871] Updated weights for policy 1, policy_version 13200 (0.0008) [2023-10-07 20:11:49,954][67871] Updated weights for policy 1, policy_version 13210 (0.0007) [2023-10-07 20:11:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27033600. Throughput: 0: 1651.5, 1: 1656.5. Samples: 6772122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:11:52,477][66916] Avg episode reward: [(0, '32.600'), (1, '34.510')] [2023-10-07 20:11:52,609][67838] Updated weights for policy 0, policy_version 13192 (0.0009) [2023-10-07 20:11:52,991][67838] Updated weights for policy 0, policy_version 13202 (0.0008) [2023-10-07 20:11:53,361][67838] Updated weights for policy 0, policy_version 13212 (0.0010) [2023-10-07 20:11:53,839][67871] Updated weights for policy 1, policy_version 13220 (0.0008) [2023-10-07 20:11:54,214][67871] Updated weights for policy 1, policy_version 13230 (0.0009) [2023-10-07 20:11:54,579][67871] Updated weights for policy 1, policy_version 13240 (0.0007) [2023-10-07 20:11:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27099136. Throughput: 0: 1652.1, 1: 1643.9. Samples: 6781404. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-07 20:11:57,478][66916] Avg episode reward: [(0, '30.470'), (1, '32.970')] [2023-10-07 20:11:57,728][67838] Updated weights for policy 0, policy_version 13222 (0.0008) [2023-10-07 20:11:58,111][67838] Updated weights for policy 0, policy_version 13232 (0.0009) [2023-10-07 20:11:58,505][67838] Updated weights for policy 0, policy_version 13242 (0.0010) [2023-10-07 20:11:58,779][67871] Updated weights for policy 1, policy_version 13250 (0.0008) [2023-10-07 20:11:59,144][67871] Updated weights for policy 1, policy_version 13260 (0.0011) [2023-10-07 20:11:59,519][67871] Updated weights for policy 1, policy_version 13270 (0.0010) [2023-10-07 20:11:59,895][67871] Updated weights for policy 1, policy_version 13280 (0.0008) [2023-10-07 20:12:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27164672. Throughput: 0: 1641.4, 1: 1663.5. Samples: 6801376. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-07 20:12:02,477][66916] Avg episode reward: [(0, '31.330'), (1, '34.750')] [2023-10-07 20:12:02,769][67838] Updated weights for policy 0, policy_version 13252 (0.0010) [2023-10-07 20:12:03,153][67838] Updated weights for policy 0, policy_version 13262 (0.0008) [2023-10-07 20:12:03,516][67838] Updated weights for policy 0, policy_version 13272 (0.0009) [2023-10-07 20:12:03,964][67871] Updated weights for policy 1, policy_version 13290 (0.0008) [2023-10-07 20:12:04,342][67871] Updated weights for policy 1, policy_version 13300 (0.0009) [2023-10-07 20:12:04,703][67871] Updated weights for policy 1, policy_version 13310 (0.0008) [2023-10-07 20:12:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27230208. Throughput: 0: 1644.0, 1: 1669.0. Samples: 6821866. Policy #0 lag: (min: 26.0, avg: 39.5, max: 40.0) [2023-10-07 20:12:07,477][66916] Avg episode reward: [(0, '32.180'), (1, '33.610')] [2023-10-07 20:12:07,623][67838] Updated weights for policy 0, policy_version 13282 (0.0009) [2023-10-07 20:12:07,993][67838] Updated weights for policy 0, policy_version 13292 (0.0009) [2023-10-07 20:12:08,376][67838] Updated weights for policy 0, policy_version 13302 (0.0009) [2023-10-07 20:12:08,741][67838] Updated weights for policy 0, policy_version 13312 (0.0010) [2023-10-07 20:12:08,998][67871] Updated weights for policy 1, policy_version 13320 (0.0010) [2023-10-07 20:12:09,368][67871] Updated weights for policy 1, policy_version 13330 (0.0008) [2023-10-07 20:12:09,740][67871] Updated weights for policy 1, policy_version 13340 (0.0009) [2023-10-07 20:12:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27295744. Throughput: 0: 1647.9, 1: 1651.7. Samples: 6830986. Policy #0 lag: (min: 26.0, avg: 39.5, max: 40.0) [2023-10-07 20:12:12,477][66916] Avg episode reward: [(0, '31.450'), (1, '34.590')] [2023-10-07 20:12:12,873][67838] Updated weights for policy 0, policy_version 13322 (0.0008) [2023-10-07 20:12:13,243][67838] Updated weights for policy 0, policy_version 13332 (0.0007) [2023-10-07 20:12:13,609][67838] Updated weights for policy 0, policy_version 13342 (0.0010) [2023-10-07 20:12:13,771][67871] Updated weights for policy 1, policy_version 13350 (0.0010) [2023-10-07 20:12:14,153][67871] Updated weights for policy 1, policy_version 13360 (0.0010) [2023-10-07 20:12:14,526][67871] Updated weights for policy 1, policy_version 13370 (0.0007) [2023-10-07 20:12:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27361280. Throughput: 0: 1646.1, 1: 1669.6. Samples: 6851144. Policy #0 lag: (min: 26.0, avg: 39.5, max: 40.0) [2023-10-07 20:12:17,477][66916] Avg episode reward: [(0, '33.250'), (1, '33.330')] [2023-10-07 20:12:17,740][67838] Updated weights for policy 0, policy_version 13352 (0.0009) [2023-10-07 20:12:18,113][67838] Updated weights for policy 0, policy_version 13362 (0.0008) [2023-10-07 20:12:18,466][67871] Updated weights for policy 1, policy_version 13380 (0.0008) [2023-10-07 20:12:18,492][67838] Updated weights for policy 0, policy_version 13372 (0.0009) [2023-10-07 20:12:18,845][67871] Updated weights for policy 1, policy_version 13390 (0.0009) [2023-10-07 20:12:19,215][67871] Updated weights for policy 1, policy_version 13400 (0.0008) [2023-10-07 20:12:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 27426816. Throughput: 0: 1637.8, 1: 1670.9. Samples: 6871532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:12:22,477][66916] Avg episode reward: [(0, '33.140'), (1, '33.610')] [2023-10-07 20:12:22,503][67838] Updated weights for policy 0, policy_version 13382 (0.0007) [2023-10-07 20:12:22,868][67838] Updated weights for policy 0, policy_version 13392 (0.0007) [2023-10-07 20:12:23,247][67838] Updated weights for policy 0, policy_version 13402 (0.0007) [2023-10-07 20:12:23,389][67871] Updated weights for policy 1, policy_version 13410 (0.0007) [2023-10-07 20:12:23,761][67871] Updated weights for policy 1, policy_version 13420 (0.0007) [2023-10-07 20:12:24,134][67871] Updated weights for policy 1, policy_version 13430 (0.0007) [2023-10-07 20:12:24,501][67871] Updated weights for policy 1, policy_version 13440 (0.0008) [2023-10-07 20:12:27,476][67838] Updated weights for policy 0, policy_version 13412 (0.0008) [2023-10-07 20:12:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 27492352. Throughput: 0: 1640.3, 1: 1661.0. Samples: 6880634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:12:27,477][66916] Avg episode reward: [(0, '30.970'), (1, '33.410')] [2023-10-07 20:12:27,856][67838] Updated weights for policy 0, policy_version 13422 (0.0009) [2023-10-07 20:12:28,237][67838] Updated weights for policy 0, policy_version 13432 (0.0008) [2023-10-07 20:12:28,565][67871] Updated weights for policy 1, policy_version 13450 (0.0007) [2023-10-07 20:12:28,924][67871] Updated weights for policy 1, policy_version 13460 (0.0009) [2023-10-07 20:12:29,293][67871] Updated weights for policy 1, policy_version 13470 (0.0007) [2023-10-07 20:12:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 27557888. Throughput: 0: 1637.6, 1: 1673.1. Samples: 6900944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:12:32,478][66916] Avg episode reward: [(0, '31.070'), (1, '32.920')] [2023-10-07 20:12:32,511][67838] Updated weights for policy 0, policy_version 13442 (0.0009) [2023-10-07 20:12:32,878][67838] Updated weights for policy 0, policy_version 13452 (0.0007) [2023-10-07 20:12:33,251][67838] Updated weights for policy 0, policy_version 13462 (0.0008) [2023-10-07 20:12:33,627][67838] Updated weights for policy 0, policy_version 13472 (0.0008) [2023-10-07 20:12:33,634][67871] Updated weights for policy 1, policy_version 13480 (0.0007) [2023-10-07 20:12:33,993][67871] Updated weights for policy 1, policy_version 13490 (0.0009) [2023-10-07 20:12:34,360][67871] Updated weights for policy 1, policy_version 13500 (0.0010) [2023-10-07 20:12:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27623424. Throughput: 0: 1641.2, 1: 1672.1. Samples: 6921222. Policy #0 lag: (min: 2.0, avg: 6.9, max: 34.0) [2023-10-07 20:12:37,477][66916] Avg episode reward: [(0, '32.830'), (1, '33.500')] [2023-10-07 20:12:37,722][67838] Updated weights for policy 0, policy_version 13482 (0.0010) [2023-10-07 20:12:38,105][67838] Updated weights for policy 0, policy_version 13492 (0.0008) [2023-10-07 20:12:38,166][67871] Updated weights for policy 1, policy_version 13510 (0.0009) [2023-10-07 20:12:38,478][67838] Updated weights for policy 0, policy_version 13502 (0.0008) [2023-10-07 20:12:38,539][67871] Updated weights for policy 1, policy_version 13520 (0.0008) [2023-10-07 20:12:38,908][67871] Updated weights for policy 1, policy_version 13530 (0.0008) [2023-10-07 20:12:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27688960. Throughput: 0: 1641.0, 1: 1668.3. Samples: 6930322. Policy #0 lag: (min: 2.0, avg: 6.9, max: 34.0) [2023-10-07 20:12:42,478][66916] Avg episode reward: [(0, '31.300'), (1, '33.880')] [2023-10-07 20:12:42,551][67838] Updated weights for policy 0, policy_version 13512 (0.0007) [2023-10-07 20:12:42,924][67838] Updated weights for policy 0, policy_version 13522 (0.0007) [2023-10-07 20:12:43,094][67871] Updated weights for policy 1, policy_version 13540 (0.0008) [2023-10-07 20:12:43,299][67838] Updated weights for policy 0, policy_version 13532 (0.0007) [2023-10-07 20:12:43,465][67871] Updated weights for policy 1, policy_version 13550 (0.0007) [2023-10-07 20:12:43,838][67871] Updated weights for policy 1, policy_version 13560 (0.0008) [2023-10-07 20:12:47,323][67838] Updated weights for policy 0, policy_version 13542 (0.0009) [2023-10-07 20:12:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 27754496. Throughput: 0: 1646.7, 1: 1664.6. Samples: 6950388. Policy #0 lag: (min: 2.0, avg: 6.9, max: 34.0) [2023-10-07 20:12:47,478][66916] Avg episode reward: [(0, '32.620'), (1, '33.800')] [2023-10-07 20:12:47,698][67838] Updated weights for policy 0, policy_version 13552 (0.0011) [2023-10-07 20:12:47,950][67871] Updated weights for policy 1, policy_version 13570 (0.0007) [2023-10-07 20:12:48,071][67838] Updated weights for policy 0, policy_version 13562 (0.0008) [2023-10-07 20:12:48,314][67871] Updated weights for policy 1, policy_version 13580 (0.0008) [2023-10-07 20:12:48,682][67871] Updated weights for policy 1, policy_version 13590 (0.0009) [2023-10-07 20:12:49,067][67871] Updated weights for policy 1, policy_version 13600 (0.0009) [2023-10-07 20:12:52,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27820032. Throughput: 0: 1644.6, 1: 1664.2. Samples: 6970762. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 20:12:52,478][66916] Avg episode reward: [(0, '31.010'), (1, '34.700')] [2023-10-07 20:12:52,492][67838] Updated weights for policy 0, policy_version 13572 (0.0010) [2023-10-07 20:12:52,868][67838] Updated weights for policy 0, policy_version 13582 (0.0008) [2023-10-07 20:12:53,238][67838] Updated weights for policy 0, policy_version 13592 (0.0008) [2023-10-07 20:12:53,253][67871] Updated weights for policy 1, policy_version 13610 (0.0010) [2023-10-07 20:12:53,623][67871] Updated weights for policy 1, policy_version 13620 (0.0009) [2023-10-07 20:12:53,996][67871] Updated weights for policy 1, policy_version 13630 (0.0009) [2023-10-07 20:12:57,392][67838] Updated weights for policy 0, policy_version 13602 (0.0007) [2023-10-07 20:12:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27885568. Throughput: 0: 1641.5, 1: 1665.0. Samples: 6979778. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 20:12:57,477][66916] Avg episode reward: [(0, '32.170'), (1, '33.790')] [2023-10-07 20:12:57,756][67838] Updated weights for policy 0, policy_version 13612 (0.0007) [2023-10-07 20:12:58,134][67838] Updated weights for policy 0, policy_version 13622 (0.0008) [2023-10-07 20:12:58,293][67871] Updated weights for policy 1, policy_version 13640 (0.0008) [2023-10-07 20:12:58,505][67838] Updated weights for policy 0, policy_version 13632 (0.0007) [2023-10-07 20:12:58,674][67871] Updated weights for policy 1, policy_version 13650 (0.0007) [2023-10-07 20:12:59,032][67871] Updated weights for policy 1, policy_version 13660 (0.0010) [2023-10-07 20:13:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27951104. Throughput: 0: 1644.3, 1: 1664.0. Samples: 7000018. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 20:13:02,478][66916] Avg episode reward: [(0, '33.300'), (1, '34.200')] [2023-10-07 20:13:02,716][67838] Updated weights for policy 0, policy_version 13642 (0.0009) [2023-10-07 20:13:03,093][67838] Updated weights for policy 0, policy_version 13652 (0.0010) [2023-10-07 20:13:03,211][67871] Updated weights for policy 1, policy_version 13670 (0.0008) [2023-10-07 20:13:03,472][67838] Updated weights for policy 0, policy_version 13662 (0.0009) [2023-10-07 20:13:03,585][67871] Updated weights for policy 1, policy_version 13680 (0.0008) [2023-10-07 20:13:03,951][67871] Updated weights for policy 1, policy_version 13690 (0.0008) [2023-10-07 20:13:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28016640. Throughput: 0: 1643.6, 1: 1661.7. Samples: 7020268. Policy #0 lag: (min: 27.0, avg: 32.4, max: 59.0) [2023-10-07 20:13:07,477][66916] Avg episode reward: [(0, '31.720'), (1, '34.330')] [2023-10-07 20:13:07,485][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000013696_14024704.pth... [2023-10-07 20:13:07,524][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000012160_12451840.pth [2023-10-07 20:13:07,620][67838] Updated weights for policy 0, policy_version 13672 (0.0008) [2023-10-07 20:13:07,990][67838] Updated weights for policy 0, policy_version 13682 (0.0008) [2023-10-07 20:13:08,054][67871] Updated weights for policy 1, policy_version 13700 (0.0007) [2023-10-07 20:13:08,357][67838] Updated weights for policy 0, policy_version 13692 (0.0010) [2023-10-07 20:13:08,420][67871] Updated weights for policy 1, policy_version 13710 (0.0007) [2023-10-07 20:13:08,501][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000013696_14024704.pth... [2023-10-07 20:13:08,540][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000012128_12419072.pth [2023-10-07 20:13:08,787][67871] Updated weights for policy 1, policy_version 13720 (0.0008) [2023-10-07 20:13:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28082176. Throughput: 0: 1640.6, 1: 1661.0. Samples: 7029204. Policy #0 lag: (min: 27.0, avg: 32.4, max: 59.0) [2023-10-07 20:13:12,477][66916] Avg episode reward: [(0, '31.920'), (1, '33.260')] [2023-10-07 20:13:12,479][67838] Updated weights for policy 0, policy_version 13702 (0.0007) [2023-10-07 20:13:12,857][67838] Updated weights for policy 0, policy_version 13712 (0.0007) [2023-10-07 20:13:13,001][67871] Updated weights for policy 1, policy_version 13730 (0.0009) [2023-10-07 20:13:13,230][67838] Updated weights for policy 0, policy_version 13722 (0.0007) [2023-10-07 20:13:13,375][67871] Updated weights for policy 1, policy_version 13740 (0.0008) [2023-10-07 20:13:13,740][67871] Updated weights for policy 1, policy_version 13750 (0.0008) [2023-10-07 20:13:14,113][67871] Updated weights for policy 1, policy_version 13760 (0.0007) [2023-10-07 20:13:17,382][67838] Updated weights for policy 0, policy_version 13732 (0.0008) [2023-10-07 20:13:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28147712. Throughput: 0: 1640.4, 1: 1658.7. Samples: 7049404. Policy #0 lag: (min: 27.0, avg: 32.4, max: 59.0) [2023-10-07 20:13:17,478][66916] Avg episode reward: [(0, '31.700'), (1, '33.280')] [2023-10-07 20:13:17,753][67838] Updated weights for policy 0, policy_version 13742 (0.0009) [2023-10-07 20:13:18,129][67838] Updated weights for policy 0, policy_version 13752 (0.0009) [2023-10-07 20:13:18,203][67871] Updated weights for policy 1, policy_version 13770 (0.0008) [2023-10-07 20:13:18,569][67871] Updated weights for policy 1, policy_version 13780 (0.0008) [2023-10-07 20:13:18,931][67871] Updated weights for policy 1, policy_version 13790 (0.0009) [2023-10-07 20:13:22,273][67838] Updated weights for policy 0, policy_version 13762 (0.0009) [2023-10-07 20:13:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 28213248. Throughput: 0: 1645.5, 1: 1659.1. Samples: 7069930. Policy #0 lag: (min: 12.0, avg: 15.2, max: 44.0) [2023-10-07 20:13:22,478][66916] Avg episode reward: [(0, '32.600'), (1, '34.810')] [2023-10-07 20:13:22,651][67838] Updated weights for policy 0, policy_version 13772 (0.0008) [2023-10-07 20:13:23,019][67838] Updated weights for policy 0, policy_version 13782 (0.0007) [2023-10-07 20:13:23,060][67871] Updated weights for policy 1, policy_version 13800 (0.0007) [2023-10-07 20:13:23,395][67838] Updated weights for policy 0, policy_version 13792 (0.0008) [2023-10-07 20:13:23,426][67871] Updated weights for policy 1, policy_version 13810 (0.0007) [2023-10-07 20:13:23,794][67871] Updated weights for policy 1, policy_version 13820 (0.0008) [2023-10-07 20:13:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28278784. Throughput: 0: 1644.9, 1: 1656.5. Samples: 7078886. Policy #0 lag: (min: 12.0, avg: 15.2, max: 44.0) [2023-10-07 20:13:27,477][66916] Avg episode reward: [(0, '32.180'), (1, '34.290')] [2023-10-07 20:13:27,630][67838] Updated weights for policy 0, policy_version 13802 (0.0009) [2023-10-07 20:13:27,891][67871] Updated weights for policy 1, policy_version 13830 (0.0008) [2023-10-07 20:13:27,993][67838] Updated weights for policy 0, policy_version 13812 (0.0008) [2023-10-07 20:13:28,268][67871] Updated weights for policy 1, policy_version 13840 (0.0007) [2023-10-07 20:13:28,371][67838] Updated weights for policy 0, policy_version 13822 (0.0008) [2023-10-07 20:13:28,627][67871] Updated weights for policy 1, policy_version 13850 (0.0009) [2023-10-07 20:13:32,458][67838] Updated weights for policy 0, policy_version 13832 (0.0007) [2023-10-07 20:13:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28344320. Throughput: 0: 1649.3, 1: 1657.7. Samples: 7099200. Policy #0 lag: (min: 12.0, avg: 15.2, max: 44.0) [2023-10-07 20:13:32,477][66916] Avg episode reward: [(0, '32.760'), (1, '35.150')] [2023-10-07 20:13:32,478][67676] Saving new best policy, reward=35.150! [2023-10-07 20:13:32,830][67838] Updated weights for policy 0, policy_version 13842 (0.0010) [2023-10-07 20:13:32,979][67871] Updated weights for policy 1, policy_version 13860 (0.0011) [2023-10-07 20:13:33,194][67838] Updated weights for policy 0, policy_version 13852 (0.0008) [2023-10-07 20:13:33,344][67871] Updated weights for policy 1, policy_version 13870 (0.0009) [2023-10-07 20:13:33,714][67871] Updated weights for policy 1, policy_version 13880 (0.0007) [2023-10-07 20:13:37,315][67838] Updated weights for policy 0, policy_version 13862 (0.0010) [2023-10-07 20:13:37,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28409856. Throughput: 0: 1654.8, 1: 1649.6. Samples: 7119456. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 20:13:37,478][66916] Avg episode reward: [(0, '31.910'), (1, '33.960')] [2023-10-07 20:13:37,688][67838] Updated weights for policy 0, policy_version 13872 (0.0009) [2023-10-07 20:13:37,912][67871] Updated weights for policy 1, policy_version 13890 (0.0011) [2023-10-07 20:13:38,052][67838] Updated weights for policy 0, policy_version 13882 (0.0008) [2023-10-07 20:13:38,277][67871] Updated weights for policy 1, policy_version 13900 (0.0008) [2023-10-07 20:13:38,645][67871] Updated weights for policy 1, policy_version 13910 (0.0009) [2023-10-07 20:13:39,020][67871] Updated weights for policy 1, policy_version 13920 (0.0009) [2023-10-07 20:13:42,074][67838] Updated weights for policy 0, policy_version 13892 (0.0010) [2023-10-07 20:13:42,449][67838] Updated weights for policy 0, policy_version 13902 (0.0009) [2023-10-07 20:13:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 28475392. Throughput: 0: 1656.9, 1: 1644.5. Samples: 7128342. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 20:13:42,477][66916] Avg episode reward: [(0, '31.330'), (1, '35.480')] [2023-10-07 20:13:42,478][67676] Saving new best policy, reward=35.480! [2023-10-07 20:13:42,818][67838] Updated weights for policy 0, policy_version 13912 (0.0009) [2023-10-07 20:13:43,040][67871] Updated weights for policy 1, policy_version 13930 (0.0010) [2023-10-07 20:13:43,417][67871] Updated weights for policy 1, policy_version 13940 (0.0009) [2023-10-07 20:13:43,781][67871] Updated weights for policy 1, policy_version 13950 (0.0010) [2023-10-07 20:13:46,993][67838] Updated weights for policy 0, policy_version 13922 (0.0009) [2023-10-07 20:13:47,369][67838] Updated weights for policy 0, policy_version 13932 (0.0008) [2023-10-07 20:13:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 28540928. Throughput: 0: 1655.8, 1: 1645.9. Samples: 7148594. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 20:13:47,477][66916] Avg episode reward: [(0, '33.140'), (1, '34.090')] [2023-10-07 20:13:47,742][67838] Updated weights for policy 0, policy_version 13942 (0.0008) [2023-10-07 20:13:47,871][67871] Updated weights for policy 1, policy_version 13960 (0.0008) [2023-10-07 20:13:48,117][67838] Updated weights for policy 0, policy_version 13952 (0.0008) [2023-10-07 20:13:48,245][67871] Updated weights for policy 1, policy_version 13970 (0.0010) [2023-10-07 20:13:48,602][67871] Updated weights for policy 1, policy_version 13980 (0.0011) [2023-10-07 20:13:52,196][67838] Updated weights for policy 0, policy_version 13962 (0.0007) [2023-10-07 20:13:52,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28606464. Throughput: 0: 1649.6, 1: 1643.6. Samples: 7168464. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 20:13:52,477][66916] Avg episode reward: [(0, '32.600'), (1, '35.390')] [2023-10-07 20:13:52,572][67838] Updated weights for policy 0, policy_version 13972 (0.0008) [2023-10-07 20:13:52,815][67871] Updated weights for policy 1, policy_version 13990 (0.0008) [2023-10-07 20:13:52,941][67838] Updated weights for policy 0, policy_version 13982 (0.0009) [2023-10-07 20:13:53,182][67871] Updated weights for policy 1, policy_version 14000 (0.0008) [2023-10-07 20:13:53,555][67871] Updated weights for policy 1, policy_version 14010 (0.0009) [2023-10-07 20:13:57,032][67838] Updated weights for policy 0, policy_version 13992 (0.0007) [2023-10-07 20:13:57,411][67838] Updated weights for policy 0, policy_version 14002 (0.0007) [2023-10-07 20:13:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 28672000. Throughput: 0: 1657.6, 1: 1644.1. Samples: 7177782. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 20:13:57,477][66916] Avg episode reward: [(0, '32.530'), (1, '35.250')] [2023-10-07 20:13:57,785][67838] Updated weights for policy 0, policy_version 14012 (0.0007) [2023-10-07 20:13:57,908][67871] Updated weights for policy 1, policy_version 14020 (0.0009) [2023-10-07 20:13:58,267][67871] Updated weights for policy 1, policy_version 14030 (0.0009) [2023-10-07 20:13:58,632][67871] Updated weights for policy 1, policy_version 14040 (0.0009) [2023-10-07 20:14:02,015][67838] Updated weights for policy 0, policy_version 14022 (0.0007) [2023-10-07 20:14:02,384][67838] Updated weights for policy 0, policy_version 14032 (0.0010) [2023-10-07 20:14:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 28737536. Throughput: 0: 1657.6, 1: 1642.2. Samples: 7197898. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 20:14:02,478][66916] Avg episode reward: [(0, '32.120'), (1, '34.810')] [2023-10-07 20:14:02,760][67838] Updated weights for policy 0, policy_version 14042 (0.0010) [2023-10-07 20:14:02,789][67871] Updated weights for policy 1, policy_version 14050 (0.0008) [2023-10-07 20:14:03,157][67871] Updated weights for policy 1, policy_version 14060 (0.0007) [2023-10-07 20:14:03,535][67871] Updated weights for policy 1, policy_version 14070 (0.0007) [2023-10-07 20:14:03,900][67871] Updated weights for policy 1, policy_version 14080 (0.0009) [2023-10-07 20:14:06,769][67838] Updated weights for policy 0, policy_version 14052 (0.0008) [2023-10-07 20:14:07,149][67838] Updated weights for policy 0, policy_version 14062 (0.0007) [2023-10-07 20:14:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 28803072. Throughput: 0: 1648.4, 1: 1643.0. Samples: 7218042. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 20:14:07,478][66916] Avg episode reward: [(0, '32.150'), (1, '34.140')] [2023-10-07 20:14:07,532][67838] Updated weights for policy 0, policy_version 14072 (0.0009) [2023-10-07 20:14:08,107][67871] Updated weights for policy 1, policy_version 14090 (0.0008) [2023-10-07 20:14:08,478][67871] Updated weights for policy 1, policy_version 14100 (0.0008) [2023-10-07 20:14:08,846][67871] Updated weights for policy 1, policy_version 14110 (0.0008) [2023-10-07 20:14:11,775][67838] Updated weights for policy 0, policy_version 14082 (0.0007) [2023-10-07 20:14:12,192][67838] Updated weights for policy 0, policy_version 14092 (0.0009) [2023-10-07 20:14:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 28868608. Throughput: 0: 1657.4, 1: 1640.0. Samples: 7227268. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 20:14:12,477][66916] Avg episode reward: [(0, '32.880'), (1, '35.070')] [2023-10-07 20:14:12,575][67838] Updated weights for policy 0, policy_version 14102 (0.0008) [2023-10-07 20:14:12,946][67838] Updated weights for policy 0, policy_version 14112 (0.0009) [2023-10-07 20:14:13,068][67871] Updated weights for policy 1, policy_version 14120 (0.0007) [2023-10-07 20:14:13,442][67871] Updated weights for policy 1, policy_version 14130 (0.0008) [2023-10-07 20:14:13,809][67871] Updated weights for policy 1, policy_version 14140 (0.0008) [2023-10-07 20:14:17,139][67838] Updated weights for policy 0, policy_version 14122 (0.0007) [2023-10-07 20:14:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 28934144. Throughput: 0: 1650.0, 1: 1639.1. Samples: 7247208. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 20:14:17,477][66916] Avg episode reward: [(0, '32.630'), (1, '35.040')] [2023-10-07 20:14:17,528][67838] Updated weights for policy 0, policy_version 14132 (0.0008) [2023-10-07 20:14:17,810][67871] Updated weights for policy 1, policy_version 14150 (0.0008) [2023-10-07 20:14:17,903][67838] Updated weights for policy 0, policy_version 14142 (0.0007) [2023-10-07 20:14:18,175][67871] Updated weights for policy 1, policy_version 14160 (0.0010) [2023-10-07 20:14:18,535][67871] Updated weights for policy 1, policy_version 14170 (0.0007) [2023-10-07 20:14:22,103][67838] Updated weights for policy 0, policy_version 14152 (0.0007) [2023-10-07 20:14:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 28999680. Throughput: 0: 1639.9, 1: 1644.6. Samples: 7267260. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-07 20:14:22,477][66916] Avg episode reward: [(0, '33.690'), (1, '34.570')] [2023-10-07 20:14:22,480][67838] Updated weights for policy 0, policy_version 14162 (0.0008) [2023-10-07 20:14:22,795][67871] Updated weights for policy 1, policy_version 14180 (0.0009) [2023-10-07 20:14:22,849][67838] Updated weights for policy 0, policy_version 14172 (0.0009) [2023-10-07 20:14:23,166][67871] Updated weights for policy 1, policy_version 14190 (0.0007) [2023-10-07 20:14:23,539][67871] Updated weights for policy 1, policy_version 14200 (0.0008) [2023-10-07 20:14:26,949][67838] Updated weights for policy 0, policy_version 14182 (0.0011) [2023-10-07 20:14:27,321][67838] Updated weights for policy 0, policy_version 14192 (0.0010) [2023-10-07 20:14:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 29065216. Throughput: 0: 1645.3, 1: 1647.2. Samples: 7276508. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-07 20:14:27,477][66916] Avg episode reward: [(0, '31.460'), (1, '35.170')] [2023-10-07 20:14:27,704][67838] Updated weights for policy 0, policy_version 14202 (0.0008) [2023-10-07 20:14:27,771][67871] Updated weights for policy 1, policy_version 14210 (0.0008) [2023-10-07 20:14:28,190][67871] Updated weights for policy 1, policy_version 14220 (0.0009) [2023-10-07 20:14:28,561][67871] Updated weights for policy 1, policy_version 14230 (0.0007) [2023-10-07 20:14:28,930][67871] Updated weights for policy 1, policy_version 14240 (0.0008) [2023-10-07 20:14:31,785][67838] Updated weights for policy 0, policy_version 14212 (0.0009) [2023-10-07 20:14:32,150][67838] Updated weights for policy 0, policy_version 14222 (0.0009) [2023-10-07 20:14:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 29130752. Throughput: 0: 1649.5, 1: 1645.4. Samples: 7296866. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-07 20:14:32,478][66916] Avg episode reward: [(0, '32.010'), (1, '34.470')] [2023-10-07 20:14:32,529][67838] Updated weights for policy 0, policy_version 14232 (0.0007) [2023-10-07 20:14:33,034][67871] Updated weights for policy 1, policy_version 14250 (0.0008) [2023-10-07 20:14:33,398][67871] Updated weights for policy 1, policy_version 14260 (0.0009) [2023-10-07 20:14:33,765][67871] Updated weights for policy 1, policy_version 14270 (0.0007) [2023-10-07 20:14:36,680][67838] Updated weights for policy 0, policy_version 14242 (0.0008) [2023-10-07 20:14:37,064][67838] Updated weights for policy 0, policy_version 14252 (0.0007) [2023-10-07 20:14:37,443][67838] Updated weights for policy 0, policy_version 14262 (0.0007) [2023-10-07 20:14:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 29196288. Throughput: 0: 1646.3, 1: 1647.1. Samples: 7316668. Policy #0 lag: (min: 2.0, avg: 13.2, max: 34.0) [2023-10-07 20:14:37,478][66916] Avg episode reward: [(0, '32.690'), (1, '33.870')] [2023-10-07 20:14:37,807][67838] Updated weights for policy 0, policy_version 14272 (0.0007) [2023-10-07 20:14:37,957][67871] Updated weights for policy 1, policy_version 14280 (0.0007) [2023-10-07 20:14:38,321][67871] Updated weights for policy 1, policy_version 14290 (0.0009) [2023-10-07 20:14:38,688][67871] Updated weights for policy 1, policy_version 14300 (0.0010) [2023-10-07 20:14:41,984][67838] Updated weights for policy 0, policy_version 14282 (0.0008) [2023-10-07 20:14:42,365][67838] Updated weights for policy 0, policy_version 14292 (0.0008) [2023-10-07 20:14:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 29261824. Throughput: 0: 1647.5, 1: 1649.7. Samples: 7326156. Policy #0 lag: (min: 2.0, avg: 13.2, max: 34.0) [2023-10-07 20:14:42,477][66916] Avg episode reward: [(0, '32.310'), (1, '33.360')] [2023-10-07 20:14:42,742][67838] Updated weights for policy 0, policy_version 14302 (0.0010) [2023-10-07 20:14:42,846][67871] Updated weights for policy 1, policy_version 14310 (0.0009) [2023-10-07 20:14:43,220][67871] Updated weights for policy 1, policy_version 14320 (0.0008) [2023-10-07 20:14:43,590][67871] Updated weights for policy 1, policy_version 14330 (0.0008) [2023-10-07 20:14:46,958][67838] Updated weights for policy 0, policy_version 14312 (0.0007) [2023-10-07 20:14:47,335][67838] Updated weights for policy 0, policy_version 14322 (0.0008) [2023-10-07 20:14:47,464][67871] Updated weights for policy 1, policy_version 14340 (0.0008) [2023-10-07 20:14:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 29327360. Throughput: 0: 1646.0, 1: 1653.1. Samples: 7346358. Policy #0 lag: (min: 2.0, avg: 13.2, max: 34.0) [2023-10-07 20:14:47,477][66916] Avg episode reward: [(0, '33.600'), (1, '34.820')] [2023-10-07 20:14:47,716][67838] Updated weights for policy 0, policy_version 14332 (0.0008) [2023-10-07 20:14:47,831][67871] Updated weights for policy 1, policy_version 14350 (0.0007) [2023-10-07 20:14:48,200][67871] Updated weights for policy 1, policy_version 14360 (0.0009) [2023-10-07 20:14:51,624][67838] Updated weights for policy 0, policy_version 14342 (0.0009) [2023-10-07 20:14:51,998][67838] Updated weights for policy 0, policy_version 14352 (0.0009) [2023-10-07 20:14:52,376][67838] Updated weights for policy 0, policy_version 14362 (0.0007) [2023-10-07 20:14:52,384][67871] Updated weights for policy 1, policy_version 14370 (0.0008) [2023-10-07 20:14:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 29392896. Throughput: 0: 1641.9, 1: 1652.7. Samples: 7366296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:14:52,478][66916] Avg episode reward: [(0, '31.610'), (1, '33.930')] [2023-10-07 20:14:52,741][67871] Updated weights for policy 1, policy_version 14380 (0.0008) [2023-10-07 20:14:53,104][67871] Updated weights for policy 1, policy_version 14390 (0.0010) [2023-10-07 20:14:53,473][67871] Updated weights for policy 1, policy_version 14400 (0.0008) [2023-10-07 20:14:56,250][67838] Updated weights for policy 0, policy_version 14372 (0.0008) [2023-10-07 20:14:56,635][67838] Updated weights for policy 0, policy_version 14382 (0.0007) [2023-10-07 20:14:56,998][67838] Updated weights for policy 0, policy_version 14392 (0.0007) [2023-10-07 20:14:57,476][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 29491200. Throughput: 0: 1651.4, 1: 1654.0. Samples: 7376012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:14:57,477][66916] Avg episode reward: [(0, '33.860'), (1, '34.790')] [2023-10-07 20:14:57,628][67871] Updated weights for policy 1, policy_version 14410 (0.0009) [2023-10-07 20:14:57,999][67871] Updated weights for policy 1, policy_version 14420 (0.0007) [2023-10-07 20:14:58,376][67871] Updated weights for policy 1, policy_version 14430 (0.0008) [2023-10-07 20:15:01,091][67838] Updated weights for policy 0, policy_version 14402 (0.0009) [2023-10-07 20:15:01,494][67838] Updated weights for policy 0, policy_version 14412 (0.0009) [2023-10-07 20:15:01,866][67838] Updated weights for policy 0, policy_version 14422 (0.0008) [2023-10-07 20:15:02,234][67838] Updated weights for policy 0, policy_version 14432 (0.0009) [2023-10-07 20:15:02,476][66916] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 29556736. Throughput: 0: 1658.3, 1: 1655.2. Samples: 7396312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:15:02,477][66916] Avg episode reward: [(0, '32.000'), (1, '34.410')] [2023-10-07 20:15:02,628][67871] Updated weights for policy 1, policy_version 14440 (0.0010) [2023-10-07 20:15:02,996][67871] Updated weights for policy 1, policy_version 14450 (0.0007) [2023-10-07 20:15:03,372][67871] Updated weights for policy 1, policy_version 14460 (0.0008) [2023-10-07 20:15:06,501][67838] Updated weights for policy 0, policy_version 14442 (0.0008) [2023-10-07 20:15:06,866][67838] Updated weights for policy 0, policy_version 14452 (0.0011) [2023-10-07 20:15:07,237][67838] Updated weights for policy 0, policy_version 14462 (0.0009) [2023-10-07 20:15:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 29622272. Throughput: 0: 1643.8, 1: 1655.5. Samples: 7415732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:15:07,478][66916] Avg episode reward: [(0, '32.800'), (1, '33.600')] [2023-10-07 20:15:07,490][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000014464_14811136.pth... [2023-10-07 20:15:07,519][67871] Updated weights for policy 1, policy_version 14470 (0.0007) [2023-10-07 20:15:07,528][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000012896_13205504.pth [2023-10-07 20:15:07,890][67871] Updated weights for policy 1, policy_version 14480 (0.0007) [2023-10-07 20:15:08,252][67871] Updated weights for policy 1, policy_version 14490 (0.0011) [2023-10-07 20:15:08,471][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000014496_14843904.pth... [2023-10-07 20:15:08,500][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000012928_13238272.pth [2023-10-07 20:15:11,302][67838] Updated weights for policy 0, policy_version 14472 (0.0010) [2023-10-07 20:15:11,671][67838] Updated weights for policy 0, policy_version 14482 (0.0010) [2023-10-07 20:15:12,055][67838] Updated weights for policy 0, policy_version 14492 (0.0011) [2023-10-07 20:15:12,408][67871] Updated weights for policy 1, policy_version 14500 (0.0009) [2023-10-07 20:15:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 29687808. Throughput: 0: 1660.2, 1: 1654.7. Samples: 7425676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:15:12,477][66916] Avg episode reward: [(0, '31.210'), (1, '35.670')] [2023-10-07 20:15:12,781][67871] Updated weights for policy 1, policy_version 14510 (0.0007) [2023-10-07 20:15:13,156][67871] Updated weights for policy 1, policy_version 14520 (0.0007) [2023-10-07 20:15:13,439][67676] Saving new best policy, reward=35.670! [2023-10-07 20:15:16,244][67838] Updated weights for policy 0, policy_version 14502 (0.0011) [2023-10-07 20:15:16,616][67838] Updated weights for policy 0, policy_version 14512 (0.0008) [2023-10-07 20:15:16,995][67838] Updated weights for policy 0, policy_version 14522 (0.0009) [2023-10-07 20:15:17,307][67871] Updated weights for policy 1, policy_version 14530 (0.0007) [2023-10-07 20:15:17,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 29753344. Throughput: 0: 1655.4, 1: 1655.9. Samples: 7445874. Policy #0 lag: (min: 24.0, avg: 48.8, max: 56.0) [2023-10-07 20:15:17,477][66916] Avg episode reward: [(0, '32.720'), (1, '35.190')] [2023-10-07 20:15:17,698][67871] Updated weights for policy 1, policy_version 14540 (0.0007) [2023-10-07 20:15:18,066][67871] Updated weights for policy 1, policy_version 14550 (0.0009) [2023-10-07 20:15:18,428][67871] Updated weights for policy 1, policy_version 14560 (0.0008) [2023-10-07 20:15:21,251][67838] Updated weights for policy 0, policy_version 14532 (0.0009) [2023-10-07 20:15:21,616][67838] Updated weights for policy 0, policy_version 14542 (0.0008) [2023-10-07 20:15:21,988][67838] Updated weights for policy 0, policy_version 14552 (0.0010) [2023-10-07 20:15:22,440][67871] Updated weights for policy 1, policy_version 14570 (0.0009) [2023-10-07 20:15:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 29818880. Throughput: 0: 1639.1, 1: 1653.4. Samples: 7464828. Policy #0 lag: (min: 24.0, avg: 48.8, max: 56.0) [2023-10-07 20:15:22,477][66916] Avg episode reward: [(0, '32.730'), (1, '34.610')] [2023-10-07 20:15:22,803][67871] Updated weights for policy 1, policy_version 14580 (0.0008) [2023-10-07 20:15:23,179][67871] Updated weights for policy 1, policy_version 14590 (0.0007) [2023-10-07 20:15:26,227][67838] Updated weights for policy 0, policy_version 14562 (0.0009) [2023-10-07 20:15:26,608][67838] Updated weights for policy 0, policy_version 14572 (0.0009) [2023-10-07 20:15:26,985][67838] Updated weights for policy 0, policy_version 14582 (0.0010) [2023-10-07 20:15:27,362][67838] Updated weights for policy 0, policy_version 14592 (0.0010) [2023-10-07 20:15:27,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 29884416. Throughput: 0: 1651.7, 1: 1646.5. Samples: 7474576. Policy #0 lag: (min: 24.0, avg: 48.8, max: 56.0) [2023-10-07 20:15:27,478][66916] Avg episode reward: [(0, '32.910'), (1, '34.910')] [2023-10-07 20:15:27,588][67871] Updated weights for policy 1, policy_version 14600 (0.0008) [2023-10-07 20:15:27,960][67871] Updated weights for policy 1, policy_version 14610 (0.0009) [2023-10-07 20:15:28,332][67871] Updated weights for policy 1, policy_version 14620 (0.0009) [2023-10-07 20:15:31,471][67838] Updated weights for policy 0, policy_version 14602 (0.0007) [2023-10-07 20:15:31,838][67838] Updated weights for policy 0, policy_version 14612 (0.0010) [2023-10-07 20:15:32,219][67838] Updated weights for policy 0, policy_version 14622 (0.0009) [2023-10-07 20:15:32,332][67871] Updated weights for policy 1, policy_version 14630 (0.0008) [2023-10-07 20:15:32,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 29949952. Throughput: 0: 1653.4, 1: 1645.5. Samples: 7494806. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-07 20:15:32,478][66916] Avg episode reward: [(0, '31.130'), (1, '33.890')] [2023-10-07 20:15:32,698][67871] Updated weights for policy 1, policy_version 14640 (0.0010) [2023-10-07 20:15:33,075][67871] Updated weights for policy 1, policy_version 14650 (0.0010) [2023-10-07 20:15:36,294][67838] Updated weights for policy 0, policy_version 14632 (0.0007) [2023-10-07 20:15:36,677][67838] Updated weights for policy 0, policy_version 14642 (0.0007) [2023-10-07 20:15:37,056][67838] Updated weights for policy 0, policy_version 14652 (0.0010) [2023-10-07 20:15:37,230][67871] Updated weights for policy 1, policy_version 14660 (0.0007) [2023-10-07 20:15:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 30015488. Throughput: 0: 1641.6, 1: 1640.4. Samples: 7513988. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-07 20:15:37,477][66916] Avg episode reward: [(0, '31.940'), (1, '34.450')] [2023-10-07 20:15:37,597][67871] Updated weights for policy 1, policy_version 14670 (0.0007) [2023-10-07 20:15:37,971][67871] Updated weights for policy 1, policy_version 14680 (0.0007) [2023-10-07 20:15:41,314][67838] Updated weights for policy 0, policy_version 14662 (0.0007) [2023-10-07 20:15:41,675][67838] Updated weights for policy 0, policy_version 14672 (0.0007) [2023-10-07 20:15:42,047][67838] Updated weights for policy 0, policy_version 14682 (0.0008) [2023-10-07 20:15:42,260][67871] Updated weights for policy 1, policy_version 14690 (0.0008) [2023-10-07 20:15:42,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 30081024. Throughput: 0: 1644.8, 1: 1640.4. Samples: 7523846. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-07 20:15:42,478][66916] Avg episode reward: [(0, '31.730'), (1, '34.530')] [2023-10-07 20:15:42,632][67871] Updated weights for policy 1, policy_version 14700 (0.0009) [2023-10-07 20:15:43,006][67871] Updated weights for policy 1, policy_version 14710 (0.0007) [2023-10-07 20:15:43,371][67871] Updated weights for policy 1, policy_version 14720 (0.0007) [2023-10-07 20:15:46,375][67838] Updated weights for policy 0, policy_version 14692 (0.0008) [2023-10-07 20:15:46,745][67838] Updated weights for policy 0, policy_version 14702 (0.0007) [2023-10-07 20:15:47,118][67838] Updated weights for policy 0, policy_version 14712 (0.0007) [2023-10-07 20:15:47,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 30146560. Throughput: 0: 1646.5, 1: 1641.8. Samples: 7544286. Policy #0 lag: (min: 2.0, avg: 2.1, max: 10.0) [2023-10-07 20:15:47,478][66916] Avg episode reward: [(0, '33.800'), (1, '33.820')] [2023-10-07 20:15:47,594][67871] Updated weights for policy 1, policy_version 14730 (0.0010) [2023-10-07 20:15:47,963][67871] Updated weights for policy 1, policy_version 14740 (0.0009) [2023-10-07 20:15:48,337][67871] Updated weights for policy 1, policy_version 14750 (0.0008) [2023-10-07 20:15:51,325][67838] Updated weights for policy 0, policy_version 14722 (0.0009) [2023-10-07 20:15:51,701][67838] Updated weights for policy 0, policy_version 14732 (0.0007) [2023-10-07 20:15:52,088][67838] Updated weights for policy 0, policy_version 14742 (0.0007) [2023-10-07 20:15:52,464][67838] Updated weights for policy 0, policy_version 14752 (0.0007) [2023-10-07 20:15:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 30212096. Throughput: 0: 1648.4, 1: 1642.1. Samples: 7563806. Policy #0 lag: (min: 2.0, avg: 2.1, max: 10.0) [2023-10-07 20:15:52,477][66916] Avg episode reward: [(0, '33.360'), (1, '34.090')] [2023-10-07 20:15:52,546][67871] Updated weights for policy 1, policy_version 14760 (0.0007) [2023-10-07 20:15:52,921][67871] Updated weights for policy 1, policy_version 14770 (0.0009) [2023-10-07 20:15:53,292][67871] Updated weights for policy 1, policy_version 14780 (0.0007) [2023-10-07 20:15:56,597][67838] Updated weights for policy 0, policy_version 14762 (0.0007) [2023-10-07 20:15:56,980][67838] Updated weights for policy 0, policy_version 14772 (0.0007) [2023-10-07 20:15:57,207][67871] Updated weights for policy 1, policy_version 14790 (0.0008) [2023-10-07 20:15:57,352][67838] Updated weights for policy 0, policy_version 14782 (0.0007) [2023-10-07 20:15:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 30277632. Throughput: 0: 1644.1, 1: 1641.7. Samples: 7573540. Policy #0 lag: (min: 2.0, avg: 2.1, max: 10.0) [2023-10-07 20:15:57,477][66916] Avg episode reward: [(0, '32.680'), (1, '32.980')] [2023-10-07 20:15:57,575][67871] Updated weights for policy 1, policy_version 14800 (0.0007) [2023-10-07 20:15:57,945][67871] Updated weights for policy 1, policy_version 14810 (0.0007) [2023-10-07 20:16:01,642][67838] Updated weights for policy 0, policy_version 14792 (0.0007) [2023-10-07 20:16:02,014][67838] Updated weights for policy 0, policy_version 14802 (0.0007) [2023-10-07 20:16:02,030][67871] Updated weights for policy 1, policy_version 14820 (0.0008) [2023-10-07 20:16:02,391][67838] Updated weights for policy 0, policy_version 14812 (0.0009) [2023-10-07 20:16:02,419][67871] Updated weights for policy 1, policy_version 14830 (0.0008) [2023-10-07 20:16:02,477][66916] Fps is (10 sec: 9830.2, 60 sec: 12561.0, 300 sec: 13107.2). Total num frames: 30310400. Throughput: 0: 1644.2, 1: 1648.5. Samples: 7594044. Policy #0 lag: (min: 2.0, avg: 2.1, max: 10.0) [2023-10-07 20:16:02,478][66916] Avg episode reward: [(0, '32.020'), (1, '34.130')] [2023-10-07 20:16:02,789][67871] Updated weights for policy 1, policy_version 14840 (0.0008) [2023-10-07 20:16:06,428][67838] Updated weights for policy 0, policy_version 14822 (0.0009) [2023-10-07 20:16:06,801][67871] Updated weights for policy 1, policy_version 14850 (0.0008) [2023-10-07 20:16:06,808][67838] Updated weights for policy 0, policy_version 14832 (0.0010) [2023-10-07 20:16:07,171][67871] Updated weights for policy 1, policy_version 14860 (0.0008) [2023-10-07 20:16:07,175][67838] Updated weights for policy 0, policy_version 14842 (0.0007) [2023-10-07 20:16:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 30408704. Throughput: 0: 1652.8, 1: 1652.8. Samples: 7613580. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 20:16:07,477][66916] Avg episode reward: [(0, '34.330'), (1, '33.930')] [2023-10-07 20:16:07,535][67871] Updated weights for policy 1, policy_version 14870 (0.0008) [2023-10-07 20:16:07,898][67871] Updated weights for policy 1, policy_version 14880 (0.0008) [2023-10-07 20:16:11,415][67838] Updated weights for policy 0, policy_version 14852 (0.0009) [2023-10-07 20:16:11,802][67838] Updated weights for policy 0, policy_version 14862 (0.0008) [2023-10-07 20:16:11,991][67871] Updated weights for policy 1, policy_version 14890 (0.0009) [2023-10-07 20:16:12,172][67838] Updated weights for policy 0, policy_version 14872 (0.0009) [2023-10-07 20:16:12,363][67871] Updated weights for policy 1, policy_version 14900 (0.0008) [2023-10-07 20:16:12,476][66916] Fps is (10 sec: 16384.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 30474240. Throughput: 0: 1647.2, 1: 1659.2. Samples: 7623360. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 20:16:12,477][66916] Avg episode reward: [(0, '32.700'), (1, '33.480')] [2023-10-07 20:16:12,722][67871] Updated weights for policy 1, policy_version 14910 (0.0008) [2023-10-07 20:16:16,242][67838] Updated weights for policy 0, policy_version 14882 (0.0009) [2023-10-07 20:16:16,618][67838] Updated weights for policy 0, policy_version 14892 (0.0007) [2023-10-07 20:16:16,950][67871] Updated weights for policy 1, policy_version 14920 (0.0008) [2023-10-07 20:16:16,997][67838] Updated weights for policy 0, policy_version 14902 (0.0008) [2023-10-07 20:16:17,322][67871] Updated weights for policy 1, policy_version 14930 (0.0007) [2023-10-07 20:16:17,365][67838] Updated weights for policy 0, policy_version 14912 (0.0008) [2023-10-07 20:16:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 30539776. Throughput: 0: 1646.5, 1: 1660.9. Samples: 7643634. Policy #0 lag: (min: 26.0, avg: 26.6, max: 43.0) [2023-10-07 20:16:17,477][66916] Avg episode reward: [(0, '32.890'), (1, '34.390')] [2023-10-07 20:16:17,693][67871] Updated weights for policy 1, policy_version 14940 (0.0007) [2023-10-07 20:16:21,476][67838] Updated weights for policy 0, policy_version 14922 (0.0009) [2023-10-07 20:16:21,848][67838] Updated weights for policy 0, policy_version 14932 (0.0008) [2023-10-07 20:16:22,014][67871] Updated weights for policy 1, policy_version 14950 (0.0008) [2023-10-07 20:16:22,219][67838] Updated weights for policy 0, policy_version 14942 (0.0008) [2023-10-07 20:16:22,383][67871] Updated weights for policy 1, policy_version 14960 (0.0008) [2023-10-07 20:16:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 30605312. Throughput: 0: 1650.5, 1: 1663.7. Samples: 7663126. Policy #0 lag: (min: 26.0, avg: 26.6, max: 43.0) [2023-10-07 20:16:22,477][66916] Avg episode reward: [(0, '32.860'), (1, '33.540')] [2023-10-07 20:16:22,741][67871] Updated weights for policy 1, policy_version 14970 (0.0008) [2023-10-07 20:16:26,363][67838] Updated weights for policy 0, policy_version 14952 (0.0007) [2023-10-07 20:16:26,736][67838] Updated weights for policy 0, policy_version 14962 (0.0008) [2023-10-07 20:16:26,937][67871] Updated weights for policy 1, policy_version 14980 (0.0009) [2023-10-07 20:16:27,110][67838] Updated weights for policy 0, policy_version 14972 (0.0007) [2023-10-07 20:16:27,308][67871] Updated weights for policy 1, policy_version 14990 (0.0007) [2023-10-07 20:16:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 30670848. Throughput: 0: 1650.6, 1: 1662.1. Samples: 7672916. Policy #0 lag: (min: 26.0, avg: 26.6, max: 43.0) [2023-10-07 20:16:27,478][66916] Avg episode reward: [(0, '32.680'), (1, '34.450')] [2023-10-07 20:16:27,683][67871] Updated weights for policy 1, policy_version 15000 (0.0007) [2023-10-07 20:16:31,171][67838] Updated weights for policy 0, policy_version 14982 (0.0009) [2023-10-07 20:16:31,557][67838] Updated weights for policy 0, policy_version 14992 (0.0007) [2023-10-07 20:16:31,841][67871] Updated weights for policy 1, policy_version 15010 (0.0009) [2023-10-07 20:16:31,920][67838] Updated weights for policy 0, policy_version 15002 (0.0007) [2023-10-07 20:16:32,212][67871] Updated weights for policy 1, policy_version 15020 (0.0007) [2023-10-07 20:16:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 30736384. Throughput: 0: 1644.9, 1: 1656.2. Samples: 7692836. Policy #0 lag: (min: 3.0, avg: 3.0, max: 7.0) [2023-10-07 20:16:32,477][66916] Avg episode reward: [(0, '33.300'), (1, '33.130')] [2023-10-07 20:16:32,573][67871] Updated weights for policy 1, policy_version 15030 (0.0008) [2023-10-07 20:16:32,944][67871] Updated weights for policy 1, policy_version 15040 (0.0008) [2023-10-07 20:16:36,185][67838] Updated weights for policy 0, policy_version 15012 (0.0007) [2023-10-07 20:16:36,559][67838] Updated weights for policy 0, policy_version 15022 (0.0010) [2023-10-07 20:16:36,934][67838] Updated weights for policy 0, policy_version 15032 (0.0008) [2023-10-07 20:16:36,981][67871] Updated weights for policy 1, policy_version 15050 (0.0009) [2023-10-07 20:16:37,343][67871] Updated weights for policy 1, policy_version 15060 (0.0009) [2023-10-07 20:16:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 30801920. Throughput: 0: 1644.8, 1: 1655.6. Samples: 7712324. Policy #0 lag: (min: 3.0, avg: 3.0, max: 7.0) [2023-10-07 20:16:37,477][66916] Avg episode reward: [(0, '31.650'), (1, '35.140')] [2023-10-07 20:16:37,715][67871] Updated weights for policy 1, policy_version 15070 (0.0010) [2023-10-07 20:16:40,999][67838] Updated weights for policy 0, policy_version 15042 (0.0008) [2023-10-07 20:16:41,368][67838] Updated weights for policy 0, policy_version 15052 (0.0008) [2023-10-07 20:16:41,730][67838] Updated weights for policy 0, policy_version 15062 (0.0008) [2023-10-07 20:16:41,919][67871] Updated weights for policy 1, policy_version 15080 (0.0009) [2023-10-07 20:16:42,107][67838] Updated weights for policy 0, policy_version 15072 (0.0009) [2023-10-07 20:16:42,292][67871] Updated weights for policy 1, policy_version 15090 (0.0008) [2023-10-07 20:16:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 30867456. Throughput: 0: 1648.2, 1: 1661.5. Samples: 7722476. Policy #0 lag: (min: 3.0, avg: 3.0, max: 7.0) [2023-10-07 20:16:42,477][66916] Avg episode reward: [(0, '33.130'), (1, '33.680')] [2023-10-07 20:16:42,675][67871] Updated weights for policy 1, policy_version 15100 (0.0009) [2023-10-07 20:16:46,297][67838] Updated weights for policy 0, policy_version 15082 (0.0009) [2023-10-07 20:16:46,669][67838] Updated weights for policy 0, policy_version 15092 (0.0009) [2023-10-07 20:16:46,963][67871] Updated weights for policy 1, policy_version 15110 (0.0008) [2023-10-07 20:16:47,031][67838] Updated weights for policy 0, policy_version 15102 (0.0009) [2023-10-07 20:16:47,353][67871] Updated weights for policy 1, policy_version 15120 (0.0008) [2023-10-07 20:16:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 30932992. Throughput: 0: 1648.1, 1: 1651.8. Samples: 7742540. Policy #0 lag: (min: 10.0, avg: 13.1, max: 39.0) [2023-10-07 20:16:47,477][66916] Avg episode reward: [(0, '32.610'), (1, '34.600')] [2023-10-07 20:16:47,713][67871] Updated weights for policy 1, policy_version 15130 (0.0011) [2023-10-07 20:16:51,287][67838] Updated weights for policy 0, policy_version 15112 (0.0008) [2023-10-07 20:16:51,666][67838] Updated weights for policy 0, policy_version 15122 (0.0007) [2023-10-07 20:16:51,828][67871] Updated weights for policy 1, policy_version 15140 (0.0009) [2023-10-07 20:16:52,038][67838] Updated weights for policy 0, policy_version 15132 (0.0007) [2023-10-07 20:16:52,199][67871] Updated weights for policy 1, policy_version 15150 (0.0010) [2023-10-07 20:16:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 30998528. Throughput: 0: 1643.9, 1: 1644.5. Samples: 7761558. Policy #0 lag: (min: 10.0, avg: 13.1, max: 39.0) [2023-10-07 20:16:52,478][66916] Avg episode reward: [(0, '32.500'), (1, '36.010')] [2023-10-07 20:16:52,561][67871] Updated weights for policy 1, policy_version 15160 (0.0007) [2023-10-07 20:16:52,854][67676] Saving new best policy, reward=36.010! [2023-10-07 20:16:56,099][67838] Updated weights for policy 0, policy_version 15142 (0.0007) [2023-10-07 20:16:56,482][67838] Updated weights for policy 0, policy_version 15152 (0.0007) [2023-10-07 20:16:56,761][67871] Updated weights for policy 1, policy_version 15170 (0.0008) [2023-10-07 20:16:56,849][67838] Updated weights for policy 0, policy_version 15162 (0.0007) [2023-10-07 20:16:57,133][67871] Updated weights for policy 1, policy_version 15180 (0.0007) [2023-10-07 20:16:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 31064064. Throughput: 0: 1651.3, 1: 1642.5. Samples: 7771584. Policy #0 lag: (min: 10.0, avg: 13.1, max: 39.0) [2023-10-07 20:16:57,477][66916] Avg episode reward: [(0, '34.170'), (1, '33.400')] [2023-10-07 20:16:57,508][67871] Updated weights for policy 1, policy_version 15190 (0.0010) [2023-10-07 20:16:57,879][67871] Updated weights for policy 1, policy_version 15200 (0.0010) [2023-10-07 20:17:00,921][67838] Updated weights for policy 0, policy_version 15172 (0.0008) [2023-10-07 20:17:01,298][67838] Updated weights for policy 0, policy_version 15182 (0.0011) [2023-10-07 20:17:01,663][67838] Updated weights for policy 0, policy_version 15192 (0.0010) [2023-10-07 20:17:01,943][67871] Updated weights for policy 1, policy_version 15210 (0.0008) [2023-10-07 20:17:02,316][67871] Updated weights for policy 1, policy_version 15220 (0.0009) [2023-10-07 20:17:02,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 31129600. Throughput: 0: 1649.9, 1: 1643.4. Samples: 7791832. Policy #0 lag: (min: 11.0, avg: 11.7, max: 28.0) [2023-10-07 20:17:02,477][66916] Avg episode reward: [(0, '31.270'), (1, '35.180')] [2023-10-07 20:17:02,687][67871] Updated weights for policy 1, policy_version 15230 (0.0008) [2023-10-07 20:17:05,662][67838] Updated weights for policy 0, policy_version 15202 (0.0009) [2023-10-07 20:17:06,038][67838] Updated weights for policy 0, policy_version 15212 (0.0007) [2023-10-07 20:17:06,417][67838] Updated weights for policy 0, policy_version 15222 (0.0008) [2023-10-07 20:17:06,769][67871] Updated weights for policy 1, policy_version 15240 (0.0008) [2023-10-07 20:17:06,791][67838] Updated weights for policy 0, policy_version 15232 (0.0007) [2023-10-07 20:17:07,135][67871] Updated weights for policy 1, policy_version 15250 (0.0008) [2023-10-07 20:17:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 31195136. Throughput: 0: 1644.8, 1: 1636.5. Samples: 7810786. Policy #0 lag: (min: 11.0, avg: 11.7, max: 28.0) [2023-10-07 20:17:07,477][66916] Avg episode reward: [(0, '32.230'), (1, '35.210')] [2023-10-07 20:17:07,485][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000015232_15597568.pth... [2023-10-07 20:17:07,502][67871] Updated weights for policy 1, policy_version 15260 (0.0011) [2023-10-07 20:17:07,514][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000013696_14024704.pth [2023-10-07 20:17:07,518][67511] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p0/milestones/checkpoint_000015232_15597568.pth [2023-10-07 20:17:07,647][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000015264_15630336.pth... [2023-10-07 20:17:07,675][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000013696_14024704.pth [2023-10-07 20:17:07,678][67676] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p1/milestones/checkpoint_000015264_15630336.pth [2023-10-07 20:17:10,980][67838] Updated weights for policy 0, policy_version 15242 (0.0008) [2023-10-07 20:17:11,346][67838] Updated weights for policy 0, policy_version 15252 (0.0009) [2023-10-07 20:17:11,710][67838] Updated weights for policy 0, policy_version 15262 (0.0008) [2023-10-07 20:17:11,720][67871] Updated weights for policy 1, policy_version 15270 (0.0010) [2023-10-07 20:17:12,103][67871] Updated weights for policy 1, policy_version 15280 (0.0010) [2023-10-07 20:17:12,467][67871] Updated weights for policy 1, policy_version 15290 (0.0010) [2023-10-07 20:17:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 31260672. Throughput: 0: 1652.2, 1: 1645.3. Samples: 7821306. Policy #0 lag: (min: 11.0, avg: 11.7, max: 28.0) [2023-10-07 20:17:12,477][66916] Avg episode reward: [(0, '33.480'), (1, '34.640')] [2023-10-07 20:17:15,838][67838] Updated weights for policy 0, policy_version 15272 (0.0010) [2023-10-07 20:17:16,222][67838] Updated weights for policy 0, policy_version 15282 (0.0009) [2023-10-07 20:17:16,605][67838] Updated weights for policy 0, policy_version 15292 (0.0008) [2023-10-07 20:17:16,692][67871] Updated weights for policy 1, policy_version 15300 (0.0010) [2023-10-07 20:17:17,066][67871] Updated weights for policy 1, policy_version 15310 (0.0008) [2023-10-07 20:17:17,437][67871] Updated weights for policy 1, policy_version 15320 (0.0009) [2023-10-07 20:17:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 31326208. Throughput: 0: 1640.7, 1: 1651.3. Samples: 7840976. Policy #0 lag: (min: 11.0, avg: 18.8, max: 43.0) [2023-10-07 20:17:17,478][66916] Avg episode reward: [(0, '31.890'), (1, '35.360')] [2023-10-07 20:17:20,679][67838] Updated weights for policy 0, policy_version 15302 (0.0008) [2023-10-07 20:17:21,054][67838] Updated weights for policy 0, policy_version 15312 (0.0008) [2023-10-07 20:17:21,435][67838] Updated weights for policy 0, policy_version 15322 (0.0008) [2023-10-07 20:17:21,616][67871] Updated weights for policy 1, policy_version 15330 (0.0007) [2023-10-07 20:17:21,978][67871] Updated weights for policy 1, policy_version 15340 (0.0009) [2023-10-07 20:17:22,352][67871] Updated weights for policy 1, policy_version 15350 (0.0009) [2023-10-07 20:17:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 31391744. Throughput: 0: 1642.9, 1: 1643.2. Samples: 7860202. Policy #0 lag: (min: 11.0, avg: 18.8, max: 43.0) [2023-10-07 20:17:22,478][66916] Avg episode reward: [(0, '33.240'), (1, '35.240')] [2023-10-07 20:17:22,717][67871] Updated weights for policy 1, policy_version 15360 (0.0009) [2023-10-07 20:17:25,534][67838] Updated weights for policy 0, policy_version 15332 (0.0007) [2023-10-07 20:17:25,911][67838] Updated weights for policy 0, policy_version 15342 (0.0007) [2023-10-07 20:17:26,284][67838] Updated weights for policy 0, policy_version 15352 (0.0010) [2023-10-07 20:17:26,853][67871] Updated weights for policy 1, policy_version 15370 (0.0008) [2023-10-07 20:17:27,226][67871] Updated weights for policy 1, policy_version 15380 (0.0007) [2023-10-07 20:17:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 31457280. Throughput: 0: 1650.0, 1: 1643.7. Samples: 7870696. Policy #0 lag: (min: 11.0, avg: 18.8, max: 43.0) [2023-10-07 20:17:27,477][66916] Avg episode reward: [(0, '32.860'), (1, '36.660')] [2023-10-07 20:17:27,597][67871] Updated weights for policy 1, policy_version 15390 (0.0007) [2023-10-07 20:17:27,663][67676] Saving new best policy, reward=36.660! [2023-10-07 20:17:30,387][67838] Updated weights for policy 0, policy_version 15362 (0.0007) [2023-10-07 20:17:30,759][67838] Updated weights for policy 0, policy_version 15372 (0.0007) [2023-10-07 20:17:31,146][67838] Updated weights for policy 0, policy_version 15382 (0.0008) [2023-10-07 20:17:31,516][67838] Updated weights for policy 0, policy_version 15392 (0.0009) [2023-10-07 20:17:31,827][67871] Updated weights for policy 1, policy_version 15400 (0.0010) [2023-10-07 20:17:32,216][67871] Updated weights for policy 1, policy_version 15410 (0.0007) [2023-10-07 20:17:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 31522816. Throughput: 0: 1636.2, 1: 1655.3. Samples: 7890656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:17:32,477][66916] Avg episode reward: [(0, '31.280'), (1, '34.740')] [2023-10-07 20:17:32,575][67871] Updated weights for policy 1, policy_version 15420 (0.0007) [2023-10-07 20:17:35,558][67838] Updated weights for policy 0, policy_version 15402 (0.0008) [2023-10-07 20:17:35,931][67838] Updated weights for policy 0, policy_version 15412 (0.0007) [2023-10-07 20:17:36,298][67838] Updated weights for policy 0, policy_version 15422 (0.0008) [2023-10-07 20:17:36,521][67871] Updated weights for policy 1, policy_version 15430 (0.0008) [2023-10-07 20:17:36,883][67871] Updated weights for policy 1, policy_version 15440 (0.0009) [2023-10-07 20:17:37,256][67871] Updated weights for policy 1, policy_version 15450 (0.0009) [2023-10-07 20:17:37,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 31621120. Throughput: 0: 1650.8, 1: 1647.0. Samples: 7909960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:17:37,478][66916] Avg episode reward: [(0, '32.640'), (1, '34.580')] [2023-10-07 20:17:40,522][67838] Updated weights for policy 0, policy_version 15432 (0.0007) [2023-10-07 20:17:40,901][67838] Updated weights for policy 0, policy_version 15442 (0.0009) [2023-10-07 20:17:41,275][67838] Updated weights for policy 0, policy_version 15452 (0.0007) [2023-10-07 20:17:41,537][67871] Updated weights for policy 1, policy_version 15460 (0.0007) [2023-10-07 20:17:41,901][67871] Updated weights for policy 1, policy_version 15470 (0.0010) [2023-10-07 20:17:42,271][67871] Updated weights for policy 1, policy_version 15480 (0.0011) [2023-10-07 20:17:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 31653888. Throughput: 0: 1655.8, 1: 1655.9. Samples: 7920610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:17:42,477][66916] Avg episode reward: [(0, '33.010'), (1, '34.340')] [2023-10-07 20:17:45,402][67838] Updated weights for policy 0, policy_version 15462 (0.0007) [2023-10-07 20:17:45,777][67838] Updated weights for policy 0, policy_version 15472 (0.0008) [2023-10-07 20:17:46,145][67838] Updated weights for policy 0, policy_version 15482 (0.0007) [2023-10-07 20:17:46,303][67871] Updated weights for policy 1, policy_version 15490 (0.0009) [2023-10-07 20:17:46,674][67871] Updated weights for policy 1, policy_version 15500 (0.0007) [2023-10-07 20:17:47,047][67871] Updated weights for policy 1, policy_version 15510 (0.0007) [2023-10-07 20:17:47,413][67871] Updated weights for policy 1, policy_version 15520 (0.0007) [2023-10-07 20:17:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 31752192. Throughput: 0: 1637.9, 1: 1655.9. Samples: 7940054. Policy #0 lag: (min: 9.0, avg: 19.9, max: 41.0) [2023-10-07 20:17:47,477][66916] Avg episode reward: [(0, '32.230'), (1, '34.020')] [2023-10-07 20:17:50,313][67838] Updated weights for policy 0, policy_version 15492 (0.0008) [2023-10-07 20:17:50,690][67838] Updated weights for policy 0, policy_version 15502 (0.0008) [2023-10-07 20:17:51,062][67838] Updated weights for policy 0, policy_version 15512 (0.0011) [2023-10-07 20:17:51,485][67871] Updated weights for policy 1, policy_version 15530 (0.0009) [2023-10-07 20:17:51,856][67871] Updated weights for policy 1, policy_version 15540 (0.0008) [2023-10-07 20:17:52,227][67871] Updated weights for policy 1, policy_version 15550 (0.0010) [2023-10-07 20:17:52,476][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 31817728. Throughput: 0: 1652.8, 1: 1653.0. Samples: 7959548. Policy #0 lag: (min: 9.0, avg: 19.9, max: 41.0) [2023-10-07 20:17:52,477][66916] Avg episode reward: [(0, '31.740'), (1, '33.410')] [2023-10-07 20:17:55,121][67838] Updated weights for policy 0, policy_version 15522 (0.0007) [2023-10-07 20:17:55,492][67838] Updated weights for policy 0, policy_version 15532 (0.0008) [2023-10-07 20:17:55,861][67838] Updated weights for policy 0, policy_version 15542 (0.0008) [2023-10-07 20:17:56,239][67838] Updated weights for policy 0, policy_version 15552 (0.0007) [2023-10-07 20:17:56,298][67871] Updated weights for policy 1, policy_version 15560 (0.0007) [2023-10-07 20:17:56,663][67871] Updated weights for policy 1, policy_version 15570 (0.0009) [2023-10-07 20:17:57,032][67871] Updated weights for policy 1, policy_version 15580 (0.0009) [2023-10-07 20:17:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 31883264. Throughput: 0: 1652.4, 1: 1663.5. Samples: 7970520. Policy #0 lag: (min: 9.0, avg: 19.9, max: 41.0) [2023-10-07 20:17:57,477][66916] Avg episode reward: [(0, '35.020'), (1, '34.650')] [2023-10-07 20:17:57,478][67511] Saving new best policy, reward=35.020! [2023-10-07 20:18:00,292][67838] Updated weights for policy 0, policy_version 15562 (0.0007) [2023-10-07 20:18:00,657][67838] Updated weights for policy 0, policy_version 15572 (0.0008) [2023-10-07 20:18:01,041][67838] Updated weights for policy 0, policy_version 15582 (0.0009) [2023-10-07 20:18:01,156][67871] Updated weights for policy 1, policy_version 15590 (0.0009) [2023-10-07 20:18:01,530][67871] Updated weights for policy 1, policy_version 15600 (0.0007) [2023-10-07 20:18:01,896][67871] Updated weights for policy 1, policy_version 15610 (0.0007) [2023-10-07 20:18:02,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 31948800. Throughput: 0: 1645.7, 1: 1664.2. Samples: 7989922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:18:02,478][66916] Avg episode reward: [(0, '32.090'), (1, '33.310')] [2023-10-07 20:18:05,374][67838] Updated weights for policy 0, policy_version 15592 (0.0010) [2023-10-07 20:18:05,743][67838] Updated weights for policy 0, policy_version 15602 (0.0009) [2023-10-07 20:18:06,065][67871] Updated weights for policy 1, policy_version 15620 (0.0007) [2023-10-07 20:18:06,120][67838] Updated weights for policy 0, policy_version 15612 (0.0009) [2023-10-07 20:18:06,431][67871] Updated weights for policy 1, policy_version 15630 (0.0007) [2023-10-07 20:18:06,807][67871] Updated weights for policy 1, policy_version 15640 (0.0007) [2023-10-07 20:18:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 32014336. Throughput: 0: 1655.2, 1: 1650.7. Samples: 8008970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:18:07,478][66916] Avg episode reward: [(0, '34.110'), (1, '33.710')] [2023-10-07 20:18:10,258][67838] Updated weights for policy 0, policy_version 15622 (0.0009) [2023-10-07 20:18:10,638][67838] Updated weights for policy 0, policy_version 15632 (0.0009) [2023-10-07 20:18:10,917][67871] Updated weights for policy 1, policy_version 15650 (0.0007) [2023-10-07 20:18:11,005][67838] Updated weights for policy 0, policy_version 15642 (0.0010) [2023-10-07 20:18:11,278][67871] Updated weights for policy 1, policy_version 15660 (0.0008) [2023-10-07 20:18:11,651][67871] Updated weights for policy 1, policy_version 15670 (0.0007) [2023-10-07 20:18:12,017][67871] Updated weights for policy 1, policy_version 15680 (0.0008) [2023-10-07 20:18:12,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 32079872. Throughput: 0: 1655.2, 1: 1663.4. Samples: 8020036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:18:12,477][66916] Avg episode reward: [(0, '31.890'), (1, '34.970')] [2023-10-07 20:18:15,121][67838] Updated weights for policy 0, policy_version 15652 (0.0010) [2023-10-07 20:18:15,490][67838] Updated weights for policy 0, policy_version 15662 (0.0008) [2023-10-07 20:18:15,867][67838] Updated weights for policy 0, policy_version 15672 (0.0008) [2023-10-07 20:18:16,171][67871] Updated weights for policy 1, policy_version 15690 (0.0007) [2023-10-07 20:18:16,536][67871] Updated weights for policy 1, policy_version 15700 (0.0008) [2023-10-07 20:18:16,902][67871] Updated weights for policy 1, policy_version 15710 (0.0009) [2023-10-07 20:18:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 32145408. Throughput: 0: 1650.1, 1: 1658.4. Samples: 8039538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:18:17,477][66916] Avg episode reward: [(0, '33.990'), (1, '35.010')] [2023-10-07 20:18:19,936][67838] Updated weights for policy 0, policy_version 15682 (0.0008) [2023-10-07 20:18:20,303][67838] Updated weights for policy 0, policy_version 15692 (0.0010) [2023-10-07 20:18:20,688][67838] Updated weights for policy 0, policy_version 15702 (0.0008) [2023-10-07 20:18:21,051][67838] Updated weights for policy 0, policy_version 15712 (0.0007) [2023-10-07 20:18:21,152][67871] Updated weights for policy 1, policy_version 15720 (0.0010) [2023-10-07 20:18:21,518][67871] Updated weights for policy 1, policy_version 15730 (0.0009) [2023-10-07 20:18:21,874][67871] Updated weights for policy 1, policy_version 15740 (0.0009) [2023-10-07 20:18:22,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 32210944. Throughput: 0: 1653.9, 1: 1648.9. Samples: 8058584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:18:22,477][66916] Avg episode reward: [(0, '34.060'), (1, '34.750')] [2023-10-07 20:18:25,325][67838] Updated weights for policy 0, policy_version 15722 (0.0008) [2023-10-07 20:18:25,706][67838] Updated weights for policy 0, policy_version 15732 (0.0011) [2023-10-07 20:18:25,899][67871] Updated weights for policy 1, policy_version 15750 (0.0008) [2023-10-07 20:18:26,083][67838] Updated weights for policy 0, policy_version 15742 (0.0009) [2023-10-07 20:18:26,265][67871] Updated weights for policy 1, policy_version 15760 (0.0008) [2023-10-07 20:18:26,630][67871] Updated weights for policy 1, policy_version 15770 (0.0007) [2023-10-07 20:18:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 32276480. Throughput: 0: 1651.9, 1: 1663.8. Samples: 8069818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:18:27,477][66916] Avg episode reward: [(0, '31.030'), (1, '35.480')] [2023-10-07 20:18:30,295][67838] Updated weights for policy 0, policy_version 15752 (0.0008) [2023-10-07 20:18:30,668][67838] Updated weights for policy 0, policy_version 15762 (0.0008) [2023-10-07 20:18:30,729][67871] Updated weights for policy 1, policy_version 15780 (0.0008) [2023-10-07 20:18:31,042][67838] Updated weights for policy 0, policy_version 15772 (0.0007) [2023-10-07 20:18:31,094][67871] Updated weights for policy 1, policy_version 15790 (0.0008) [2023-10-07 20:18:31,451][67871] Updated weights for policy 1, policy_version 15800 (0.0007) [2023-10-07 20:18:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 32342016. Throughput: 0: 1653.6, 1: 1657.1. Samples: 8089036. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-07 20:18:32,478][66916] Avg episode reward: [(0, '33.430'), (1, '33.320')] [2023-10-07 20:18:35,115][67838] Updated weights for policy 0, policy_version 15782 (0.0007) [2023-10-07 20:18:35,480][67838] Updated weights for policy 0, policy_version 15792 (0.0008) [2023-10-07 20:18:35,506][67871] Updated weights for policy 1, policy_version 15810 (0.0008) [2023-10-07 20:18:35,852][67838] Updated weights for policy 0, policy_version 15802 (0.0009) [2023-10-07 20:18:35,881][67871] Updated weights for policy 1, policy_version 15820 (0.0007) [2023-10-07 20:18:36,248][67871] Updated weights for policy 1, policy_version 15830 (0.0008) [2023-10-07 20:18:36,622][67871] Updated weights for policy 1, policy_version 15840 (0.0010) [2023-10-07 20:18:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 32407552. Throughput: 0: 1656.6, 1: 1651.6. Samples: 8108416. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-07 20:18:37,478][66916] Avg episode reward: [(0, '32.430'), (1, '35.540')] [2023-10-07 20:18:39,951][67838] Updated weights for policy 0, policy_version 15812 (0.0010) [2023-10-07 20:18:40,336][67838] Updated weights for policy 0, policy_version 15822 (0.0009) [2023-10-07 20:18:40,704][67838] Updated weights for policy 0, policy_version 15832 (0.0009) [2023-10-07 20:18:40,739][67871] Updated weights for policy 1, policy_version 15850 (0.0007) [2023-10-07 20:18:41,101][67871] Updated weights for policy 1, policy_version 15860 (0.0007) [2023-10-07 20:18:41,478][67871] Updated weights for policy 1, policy_version 15870 (0.0008) [2023-10-07 20:18:42,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 32473088. Throughput: 0: 1650.7, 1: 1662.5. Samples: 8119612. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-07 20:18:42,477][66916] Avg episode reward: [(0, '32.860'), (1, '34.550')] [2023-10-07 20:18:44,937][67838] Updated weights for policy 0, policy_version 15842 (0.0008) [2023-10-07 20:18:45,309][67838] Updated weights for policy 0, policy_version 15852 (0.0009) [2023-10-07 20:18:45,472][67871] Updated weights for policy 1, policy_version 15880 (0.0008) [2023-10-07 20:18:45,680][67838] Updated weights for policy 0, policy_version 15862 (0.0008) [2023-10-07 20:18:45,834][67871] Updated weights for policy 1, policy_version 15890 (0.0008) [2023-10-07 20:18:46,059][67838] Updated weights for policy 0, policy_version 15872 (0.0008) [2023-10-07 20:18:46,207][67871] Updated weights for policy 1, policy_version 15900 (0.0009) [2023-10-07 20:18:47,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32538624. Throughput: 0: 1648.5, 1: 1649.4. Samples: 8138324. Policy #0 lag: (min: 28.0, avg: 30.9, max: 60.0) [2023-10-07 20:18:47,477][66916] Avg episode reward: [(0, '34.150'), (1, '33.970')] [2023-10-07 20:18:50,080][67838] Updated weights for policy 0, policy_version 15882 (0.0010) [2023-10-07 20:18:50,338][67871] Updated weights for policy 1, policy_version 15910 (0.0010) [2023-10-07 20:18:50,458][67838] Updated weights for policy 0, policy_version 15892 (0.0008) [2023-10-07 20:18:50,705][67871] Updated weights for policy 1, policy_version 15920 (0.0009) [2023-10-07 20:18:50,821][67838] Updated weights for policy 0, policy_version 15902 (0.0009) [2023-10-07 20:18:51,062][67871] Updated weights for policy 1, policy_version 15930 (0.0008) [2023-10-07 20:18:52,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32604160. Throughput: 0: 1654.4, 1: 1657.5. Samples: 8158002. Policy #0 lag: (min: 28.0, avg: 30.9, max: 60.0) [2023-10-07 20:18:52,477][66916] Avg episode reward: [(0, '31.450'), (1, '35.490')] [2023-10-07 20:18:54,860][67838] Updated weights for policy 0, policy_version 15912 (0.0007) [2023-10-07 20:18:55,218][67871] Updated weights for policy 1, policy_version 15940 (0.0008) [2023-10-07 20:18:55,232][67838] Updated weights for policy 0, policy_version 15922 (0.0008) [2023-10-07 20:18:55,574][67871] Updated weights for policy 1, policy_version 15950 (0.0010) [2023-10-07 20:18:55,609][67838] Updated weights for policy 0, policy_version 15932 (0.0007) [2023-10-07 20:18:55,952][67871] Updated weights for policy 1, policy_version 15960 (0.0009) [2023-10-07 20:18:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32669696. Throughput: 0: 1643.3, 1: 1667.6. Samples: 8169028. Policy #0 lag: (min: 28.0, avg: 30.9, max: 60.0) [2023-10-07 20:18:57,478][66916] Avg episode reward: [(0, '33.890'), (1, '34.720')] [2023-10-07 20:18:59,777][67838] Updated weights for policy 0, policy_version 15942 (0.0008) [2023-10-07 20:19:00,126][67871] Updated weights for policy 1, policy_version 15970 (0.0008) [2023-10-07 20:19:00,150][67838] Updated weights for policy 0, policy_version 15952 (0.0009) [2023-10-07 20:19:00,497][67871] Updated weights for policy 1, policy_version 15980 (0.0008) [2023-10-07 20:19:00,527][67838] Updated weights for policy 0, policy_version 15962 (0.0007) [2023-10-07 20:19:00,864][67871] Updated weights for policy 1, policy_version 15990 (0.0009) [2023-10-07 20:19:01,227][67871] Updated weights for policy 1, policy_version 16000 (0.0008) [2023-10-07 20:19:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32735232. Throughput: 0: 1647.0, 1: 1648.6. Samples: 8187840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:19:02,478][66916] Avg episode reward: [(0, '33.020'), (1, '35.590')] [2023-10-07 20:19:04,732][67838] Updated weights for policy 0, policy_version 15972 (0.0011) [2023-10-07 20:19:05,110][67838] Updated weights for policy 0, policy_version 15982 (0.0010) [2023-10-07 20:19:05,259][67871] Updated weights for policy 1, policy_version 16010 (0.0008) [2023-10-07 20:19:05,481][67838] Updated weights for policy 0, policy_version 15992 (0.0007) [2023-10-07 20:19:05,628][67871] Updated weights for policy 1, policy_version 16020 (0.0009) [2023-10-07 20:19:05,998][67871] Updated weights for policy 1, policy_version 16030 (0.0009) [2023-10-07 20:19:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32800768. Throughput: 0: 1648.4, 1: 1664.4. Samples: 8207656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:19:07,477][66916] Avg episode reward: [(0, '32.220'), (1, '35.110')] [2023-10-07 20:19:07,488][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000016032_16416768.pth... [2023-10-07 20:19:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000016000_16384000.pth... [2023-10-07 20:19:07,523][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000014464_14811136.pth [2023-10-07 20:19:07,529][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000014496_14843904.pth [2023-10-07 20:19:09,641][67838] Updated weights for policy 0, policy_version 16002 (0.0008) [2023-10-07 20:19:10,023][67838] Updated weights for policy 0, policy_version 16012 (0.0007) [2023-10-07 20:19:10,342][67871] Updated weights for policy 1, policy_version 16040 (0.0009) [2023-10-07 20:19:10,391][67838] Updated weights for policy 0, policy_version 16022 (0.0008) [2023-10-07 20:19:10,709][67871] Updated weights for policy 1, policy_version 16050 (0.0008) [2023-10-07 20:19:10,762][67838] Updated weights for policy 0, policy_version 16032 (0.0008) [2023-10-07 20:19:11,082][67871] Updated weights for policy 1, policy_version 16060 (0.0008) [2023-10-07 20:19:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32866304. Throughput: 0: 1642.0, 1: 1665.6. Samples: 8218660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:19:12,477][66916] Avg episode reward: [(0, '34.810'), (1, '34.440')] [2023-10-07 20:19:14,898][67838] Updated weights for policy 0, policy_version 16042 (0.0009) [2023-10-07 20:19:15,089][67871] Updated weights for policy 1, policy_version 16070 (0.0008) [2023-10-07 20:19:15,280][67838] Updated weights for policy 0, policy_version 16052 (0.0009) [2023-10-07 20:19:15,452][67871] Updated weights for policy 1, policy_version 16080 (0.0009) [2023-10-07 20:19:15,642][67838] Updated weights for policy 0, policy_version 16062 (0.0009) [2023-10-07 20:19:15,820][67871] Updated weights for policy 1, policy_version 16090 (0.0007) [2023-10-07 20:19:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32931840. Throughput: 0: 1645.9, 1: 1645.4. Samples: 8237142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:19:17,477][66916] Avg episode reward: [(0, '32.860'), (1, '35.100')] [2023-10-07 20:19:19,869][67838] Updated weights for policy 0, policy_version 16072 (0.0008) [2023-10-07 20:19:20,141][67871] Updated weights for policy 1, policy_version 16100 (0.0007) [2023-10-07 20:19:20,246][67838] Updated weights for policy 0, policy_version 16082 (0.0009) [2023-10-07 20:19:20,513][67871] Updated weights for policy 1, policy_version 16110 (0.0008) [2023-10-07 20:19:20,615][67838] Updated weights for policy 0, policy_version 16092 (0.0009) [2023-10-07 20:19:20,867][67871] Updated weights for policy 1, policy_version 16120 (0.0008) [2023-10-07 20:19:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32997376. Throughput: 0: 1645.2, 1: 1657.8. Samples: 8257052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:19:22,477][66916] Avg episode reward: [(0, '32.280'), (1, '35.330')] [2023-10-07 20:19:24,873][67838] Updated weights for policy 0, policy_version 16102 (0.0008) [2023-10-07 20:19:25,147][67871] Updated weights for policy 1, policy_version 16130 (0.0010) [2023-10-07 20:19:25,247][67838] Updated weights for policy 0, policy_version 16112 (0.0007) [2023-10-07 20:19:25,516][67871] Updated weights for policy 1, policy_version 16140 (0.0008) [2023-10-07 20:19:25,618][67838] Updated weights for policy 0, policy_version 16122 (0.0009) [2023-10-07 20:19:25,881][67871] Updated weights for policy 1, policy_version 16150 (0.0008) [2023-10-07 20:19:26,248][67871] Updated weights for policy 1, policy_version 16160 (0.0007) [2023-10-07 20:19:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 33062912. Throughput: 0: 1642.7, 1: 1656.3. Samples: 8268070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:19:27,478][66916] Avg episode reward: [(0, '33.410'), (1, '35.020')] [2023-10-07 20:19:29,720][67838] Updated weights for policy 0, policy_version 16132 (0.0009) [2023-10-07 20:19:30,098][67838] Updated weights for policy 0, policy_version 16142 (0.0009) [2023-10-07 20:19:30,355][67871] Updated weights for policy 1, policy_version 16170 (0.0009) [2023-10-07 20:19:30,471][67838] Updated weights for policy 0, policy_version 16152 (0.0008) [2023-10-07 20:19:30,725][67871] Updated weights for policy 1, policy_version 16180 (0.0008) [2023-10-07 20:19:31,100][67871] Updated weights for policy 1, policy_version 16190 (0.0010) [2023-10-07 20:19:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 33128448. Throughput: 0: 1647.6, 1: 1648.9. Samples: 8286666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:19:32,478][66916] Avg episode reward: [(0, '30.530'), (1, '35.310')] [2023-10-07 20:19:34,683][67838] Updated weights for policy 0, policy_version 16162 (0.0009) [2023-10-07 20:19:35,067][67838] Updated weights for policy 0, policy_version 16172 (0.0007) [2023-10-07 20:19:35,106][67871] Updated weights for policy 1, policy_version 16200 (0.0007) [2023-10-07 20:19:35,431][67838] Updated weights for policy 0, policy_version 16182 (0.0009) [2023-10-07 20:19:35,475][67871] Updated weights for policy 1, policy_version 16210 (0.0007) [2023-10-07 20:19:35,801][67838] Updated weights for policy 0, policy_version 16192 (0.0009) [2023-10-07 20:19:35,847][67871] Updated weights for policy 1, policy_version 16220 (0.0007) [2023-10-07 20:19:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 33193984. Throughput: 0: 1648.8, 1: 1653.0. Samples: 8306582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:19:37,478][66916] Avg episode reward: [(0, '33.240'), (1, '35.800')] [2023-10-07 20:19:39,930][67838] Updated weights for policy 0, policy_version 16202 (0.0008) [2023-10-07 20:19:40,184][67871] Updated weights for policy 1, policy_version 16230 (0.0008) [2023-10-07 20:19:40,308][67838] Updated weights for policy 0, policy_version 16212 (0.0008) [2023-10-07 20:19:40,561][67871] Updated weights for policy 1, policy_version 16240 (0.0008) [2023-10-07 20:19:40,674][67838] Updated weights for policy 0, policy_version 16222 (0.0009) [2023-10-07 20:19:40,932][67871] Updated weights for policy 1, policy_version 16250 (0.0010) [2023-10-07 20:19:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 33259520. Throughput: 0: 1645.6, 1: 1651.3. Samples: 8317392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:19:42,478][66916] Avg episode reward: [(0, '34.510'), (1, '34.920')] [2023-10-07 20:19:44,855][67838] Updated weights for policy 0, policy_version 16232 (0.0010) [2023-10-07 20:19:45,071][67871] Updated weights for policy 1, policy_version 16260 (0.0007) [2023-10-07 20:19:45,233][67838] Updated weights for policy 0, policy_version 16242 (0.0010) [2023-10-07 20:19:45,449][67871] Updated weights for policy 1, policy_version 16270 (0.0008) [2023-10-07 20:19:45,602][67838] Updated weights for policy 0, policy_version 16252 (0.0008) [2023-10-07 20:19:45,811][67871] Updated weights for policy 1, policy_version 16280 (0.0009) [2023-10-07 20:19:47,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 33325056. Throughput: 0: 1642.8, 1: 1645.3. Samples: 8335804. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 20:19:47,478][66916] Avg episode reward: [(0, '33.480'), (1, '35.490')] [2023-10-07 20:19:49,674][67838] Updated weights for policy 0, policy_version 16262 (0.0009) [2023-10-07 20:19:49,874][67871] Updated weights for policy 1, policy_version 16290 (0.0010) [2023-10-07 20:19:50,053][67838] Updated weights for policy 0, policy_version 16272 (0.0010) [2023-10-07 20:19:50,250][67871] Updated weights for policy 1, policy_version 16300 (0.0007) [2023-10-07 20:19:50,426][67838] Updated weights for policy 0, policy_version 16282 (0.0008) [2023-10-07 20:19:50,615][67871] Updated weights for policy 1, policy_version 16310 (0.0007) [2023-10-07 20:19:50,977][67871] Updated weights for policy 1, policy_version 16320 (0.0008) [2023-10-07 20:19:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33390592. Throughput: 0: 1648.1, 1: 1647.0. Samples: 8355936. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 20:19:52,478][66916] Avg episode reward: [(0, '32.100'), (1, '35.360')] [2023-10-07 20:19:54,633][67838] Updated weights for policy 0, policy_version 16292 (0.0008) [2023-10-07 20:19:54,956][67871] Updated weights for policy 1, policy_version 16330 (0.0009) [2023-10-07 20:19:55,005][67838] Updated weights for policy 0, policy_version 16302 (0.0009) [2023-10-07 20:19:55,317][67871] Updated weights for policy 1, policy_version 16340 (0.0009) [2023-10-07 20:19:55,380][67838] Updated weights for policy 0, policy_version 16312 (0.0007) [2023-10-07 20:19:55,683][67871] Updated weights for policy 1, policy_version 16350 (0.0008) [2023-10-07 20:19:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33456128. Throughput: 0: 1642.0, 1: 1645.5. Samples: 8366596. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 20:19:57,477][66916] Avg episode reward: [(0, '32.600'), (1, '34.360')] [2023-10-07 20:19:59,519][67838] Updated weights for policy 0, policy_version 16322 (0.0007) [2023-10-07 20:19:59,785][67871] Updated weights for policy 1, policy_version 16360 (0.0009) [2023-10-07 20:19:59,892][67838] Updated weights for policy 0, policy_version 16332 (0.0007) [2023-10-07 20:20:00,161][67871] Updated weights for policy 1, policy_version 16370 (0.0008) [2023-10-07 20:20:00,267][67838] Updated weights for policy 0, policy_version 16342 (0.0008) [2023-10-07 20:20:00,530][67871] Updated weights for policy 1, policy_version 16380 (0.0008) [2023-10-07 20:20:00,638][67838] Updated weights for policy 0, policy_version 16352 (0.0008) [2023-10-07 20:20:02,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33521664. Throughput: 0: 1644.8, 1: 1648.8. Samples: 8385354. Policy #0 lag: (min: 17.0, avg: 24.7, max: 49.0) [2023-10-07 20:20:02,477][66916] Avg episode reward: [(0, '33.780'), (1, '35.410')] [2023-10-07 20:20:04,674][67871] Updated weights for policy 1, policy_version 16390 (0.0010) [2023-10-07 20:20:04,713][67838] Updated weights for policy 0, policy_version 16362 (0.0008) [2023-10-07 20:20:05,036][67871] Updated weights for policy 1, policy_version 16400 (0.0009) [2023-10-07 20:20:05,086][67838] Updated weights for policy 0, policy_version 16372 (0.0008) [2023-10-07 20:20:05,412][67871] Updated weights for policy 1, policy_version 16410 (0.0008) [2023-10-07 20:20:05,462][67838] Updated weights for policy 0, policy_version 16382 (0.0009) [2023-10-07 20:20:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33587200. Throughput: 0: 1649.1, 1: 1655.2. Samples: 8405746. Policy #0 lag: (min: 17.0, avg: 24.7, max: 49.0) [2023-10-07 20:20:07,478][66916] Avg episode reward: [(0, '32.700'), (1, '35.730')] [2023-10-07 20:20:09,573][67838] Updated weights for policy 0, policy_version 16392 (0.0009) [2023-10-07 20:20:09,667][67871] Updated weights for policy 1, policy_version 16420 (0.0008) [2023-10-07 20:20:09,947][67838] Updated weights for policy 0, policy_version 16402 (0.0009) [2023-10-07 20:20:10,031][67871] Updated weights for policy 1, policy_version 16430 (0.0008) [2023-10-07 20:20:10,318][67838] Updated weights for policy 0, policy_version 16412 (0.0008) [2023-10-07 20:20:10,392][67871] Updated weights for policy 1, policy_version 16440 (0.0011) [2023-10-07 20:20:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 33652736. Throughput: 0: 1641.7, 1: 1649.6. Samples: 8416176. Policy #0 lag: (min: 17.0, avg: 24.7, max: 49.0) [2023-10-07 20:20:12,478][66916] Avg episode reward: [(0, '32.020'), (1, '35.110')] [2023-10-07 20:20:14,601][67838] Updated weights for policy 0, policy_version 16422 (0.0008) [2023-10-07 20:20:14,686][67871] Updated weights for policy 1, policy_version 16450 (0.0009) [2023-10-07 20:20:14,976][67838] Updated weights for policy 0, policy_version 16432 (0.0009) [2023-10-07 20:20:15,043][67871] Updated weights for policy 1, policy_version 16460 (0.0007) [2023-10-07 20:20:15,346][67838] Updated weights for policy 0, policy_version 16442 (0.0007) [2023-10-07 20:20:15,420][67871] Updated weights for policy 1, policy_version 16470 (0.0008) [2023-10-07 20:20:15,780][67871] Updated weights for policy 1, policy_version 16480 (0.0009) [2023-10-07 20:20:17,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33718272. Throughput: 0: 1646.7, 1: 1644.4. Samples: 8434766. Policy #0 lag: (min: 9.0, avg: 9.1, max: 14.0) [2023-10-07 20:20:17,477][66916] Avg episode reward: [(0, '32.480'), (1, '34.970')] [2023-10-07 20:20:19,410][67838] Updated weights for policy 0, policy_version 16452 (0.0008) [2023-10-07 20:20:19,779][67838] Updated weights for policy 0, policy_version 16462 (0.0007) [2023-10-07 20:20:19,822][67871] Updated weights for policy 1, policy_version 16490 (0.0009) [2023-10-07 20:20:20,153][67838] Updated weights for policy 0, policy_version 16472 (0.0009) [2023-10-07 20:20:20,187][67871] Updated weights for policy 1, policy_version 16500 (0.0008) [2023-10-07 20:20:20,557][67871] Updated weights for policy 1, policy_version 16510 (0.0008) [2023-10-07 20:20:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33783808. Throughput: 0: 1649.2, 1: 1654.2. Samples: 8455236. Policy #0 lag: (min: 9.0, avg: 9.1, max: 14.0) [2023-10-07 20:20:22,477][66916] Avg episode reward: [(0, '33.270'), (1, '35.490')] [2023-10-07 20:20:24,400][67838] Updated weights for policy 0, policy_version 16482 (0.0007) [2023-10-07 20:20:24,770][67838] Updated weights for policy 0, policy_version 16492 (0.0007) [2023-10-07 20:20:24,774][67871] Updated weights for policy 1, policy_version 16520 (0.0008) [2023-10-07 20:20:25,145][67871] Updated weights for policy 1, policy_version 16530 (0.0008) [2023-10-07 20:20:25,152][67838] Updated weights for policy 0, policy_version 16502 (0.0007) [2023-10-07 20:20:25,518][67871] Updated weights for policy 1, policy_version 16540 (0.0008) [2023-10-07 20:20:25,525][67838] Updated weights for policy 0, policy_version 16512 (0.0008) [2023-10-07 20:20:27,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33849344. Throughput: 0: 1646.1, 1: 1648.1. Samples: 8465630. Policy #0 lag: (min: 9.0, avg: 9.1, max: 14.0) [2023-10-07 20:20:27,478][66916] Avg episode reward: [(0, '32.730'), (1, '34.280')] [2023-10-07 20:20:29,575][67838] Updated weights for policy 0, policy_version 16522 (0.0009) [2023-10-07 20:20:29,718][67871] Updated weights for policy 1, policy_version 16550 (0.0008) [2023-10-07 20:20:29,948][67838] Updated weights for policy 0, policy_version 16532 (0.0010) [2023-10-07 20:20:30,086][67871] Updated weights for policy 1, policy_version 16560 (0.0007) [2023-10-07 20:20:30,321][67838] Updated weights for policy 0, policy_version 16542 (0.0010) [2023-10-07 20:20:30,454][67871] Updated weights for policy 1, policy_version 16570 (0.0009) [2023-10-07 20:20:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 33914880. Throughput: 0: 1654.7, 1: 1652.5. Samples: 8484628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:20:32,477][66916] Avg episode reward: [(0, '33.790'), (1, '34.590')] [2023-10-07 20:20:34,486][67838] Updated weights for policy 0, policy_version 16552 (0.0007) [2023-10-07 20:20:34,526][67871] Updated weights for policy 1, policy_version 16580 (0.0008) [2023-10-07 20:20:34,857][67838] Updated weights for policy 0, policy_version 16562 (0.0007) [2023-10-07 20:20:34,884][67871] Updated weights for policy 1, policy_version 16590 (0.0008) [2023-10-07 20:20:35,221][67838] Updated weights for policy 0, policy_version 16572 (0.0008) [2023-10-07 20:20:35,253][67871] Updated weights for policy 1, policy_version 16600 (0.0007) [2023-10-07 20:20:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33980416. Throughput: 0: 1652.4, 1: 1661.0. Samples: 8505038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:20:37,478][66916] Avg episode reward: [(0, '33.240'), (1, '34.560')] [2023-10-07 20:20:39,357][67838] Updated weights for policy 0, policy_version 16582 (0.0008) [2023-10-07 20:20:39,485][67871] Updated weights for policy 1, policy_version 16610 (0.0008) [2023-10-07 20:20:39,729][67838] Updated weights for policy 0, policy_version 16592 (0.0009) [2023-10-07 20:20:39,894][67871] Updated weights for policy 1, policy_version 16620 (0.0009) [2023-10-07 20:20:40,107][67838] Updated weights for policy 0, policy_version 16602 (0.0008) [2023-10-07 20:20:40,269][67871] Updated weights for policy 1, policy_version 16630 (0.0009) [2023-10-07 20:20:40,635][67871] Updated weights for policy 1, policy_version 16640 (0.0008) [2023-10-07 20:20:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 34045952. Throughput: 0: 1643.8, 1: 1653.1. Samples: 8514956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:20:42,478][66916] Avg episode reward: [(0, '34.750'), (1, '34.530')] [2023-10-07 20:20:44,214][67838] Updated weights for policy 0, policy_version 16612 (0.0009) [2023-10-07 20:20:44,583][67838] Updated weights for policy 0, policy_version 16622 (0.0010) [2023-10-07 20:20:44,666][67871] Updated weights for policy 1, policy_version 16650 (0.0008) [2023-10-07 20:20:44,956][67838] Updated weights for policy 0, policy_version 16632 (0.0008) [2023-10-07 20:20:45,033][67871] Updated weights for policy 1, policy_version 16660 (0.0009) [2023-10-07 20:20:45,398][67871] Updated weights for policy 1, policy_version 16670 (0.0009) [2023-10-07 20:20:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 34111488. Throughput: 0: 1646.7, 1: 1655.7. Samples: 8533966. Policy #0 lag: (min: 16.0, avg: 39.8, max: 48.0) [2023-10-07 20:20:47,478][66916] Avg episode reward: [(0, '33.270'), (1, '34.110')] [2023-10-07 20:20:49,363][67838] Updated weights for policy 0, policy_version 16642 (0.0008) [2023-10-07 20:20:49,662][67871] Updated weights for policy 1, policy_version 16680 (0.0009) [2023-10-07 20:20:49,738][67838] Updated weights for policy 0, policy_version 16652 (0.0010) [2023-10-07 20:20:50,035][67871] Updated weights for policy 1, policy_version 16690 (0.0008) [2023-10-07 20:20:50,118][67838] Updated weights for policy 0, policy_version 16662 (0.0009) [2023-10-07 20:20:50,410][67871] Updated weights for policy 1, policy_version 16700 (0.0009) [2023-10-07 20:20:50,481][67838] Updated weights for policy 0, policy_version 16672 (0.0007) [2023-10-07 20:20:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 34177024. Throughput: 0: 1641.7, 1: 1652.2. Samples: 8553974. Policy #0 lag: (min: 16.0, avg: 39.8, max: 48.0) [2023-10-07 20:20:52,477][66916] Avg episode reward: [(0, '31.090'), (1, '34.870')] [2023-10-07 20:20:54,528][67871] Updated weights for policy 1, policy_version 16710 (0.0011) [2023-10-07 20:20:54,588][67838] Updated weights for policy 0, policy_version 16682 (0.0008) [2023-10-07 20:20:54,891][67871] Updated weights for policy 1, policy_version 16720 (0.0009) [2023-10-07 20:20:54,959][67838] Updated weights for policy 0, policy_version 16692 (0.0010) [2023-10-07 20:20:55,253][67871] Updated weights for policy 1, policy_version 16730 (0.0008) [2023-10-07 20:20:55,337][67838] Updated weights for policy 0, policy_version 16702 (0.0007) [2023-10-07 20:20:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 34242560. Throughput: 0: 1639.3, 1: 1647.1. Samples: 8564064. Policy #0 lag: (min: 16.0, avg: 39.8, max: 48.0) [2023-10-07 20:20:57,477][66916] Avg episode reward: [(0, '34.310'), (1, '34.190')] [2023-10-07 20:20:59,524][67838] Updated weights for policy 0, policy_version 16712 (0.0007) [2023-10-07 20:20:59,535][67871] Updated weights for policy 1, policy_version 16740 (0.0007) [2023-10-07 20:20:59,894][67838] Updated weights for policy 0, policy_version 16722 (0.0007) [2023-10-07 20:20:59,905][67871] Updated weights for policy 1, policy_version 16750 (0.0009) [2023-10-07 20:21:00,263][67871] Updated weights for policy 1, policy_version 16760 (0.0007) [2023-10-07 20:21:00,265][67838] Updated weights for policy 0, policy_version 16732 (0.0007) [2023-10-07 20:21:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 34308096. Throughput: 0: 1645.6, 1: 1652.0. Samples: 8583160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:21:02,478][66916] Avg episode reward: [(0, '31.650'), (1, '33.590')] [2023-10-07 20:21:04,296][67838] Updated weights for policy 0, policy_version 16742 (0.0008) [2023-10-07 20:21:04,352][67871] Updated weights for policy 1, policy_version 16770 (0.0007) [2023-10-07 20:21:04,674][67838] Updated weights for policy 0, policy_version 16752 (0.0009) [2023-10-07 20:21:04,722][67871] Updated weights for policy 1, policy_version 16780 (0.0009) [2023-10-07 20:21:05,048][67838] Updated weights for policy 0, policy_version 16762 (0.0008) [2023-10-07 20:21:05,098][67871] Updated weights for policy 1, policy_version 16790 (0.0009) [2023-10-07 20:21:05,460][67871] Updated weights for policy 1, policy_version 16800 (0.0009) [2023-10-07 20:21:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 34373632. Throughput: 0: 1645.5, 1: 1651.0. Samples: 8603578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:21:07,478][66916] Avg episode reward: [(0, '32.390'), (1, '33.510')] [2023-10-07 20:21:07,487][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000016768_17170432.pth... [2023-10-07 20:21:07,487][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000016800_17203200.pth... [2023-10-07 20:21:07,518][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000015232_15597568.pth [2023-10-07 20:21:07,521][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000015264_15630336.pth [2023-10-07 20:21:09,166][67838] Updated weights for policy 0, policy_version 16772 (0.0008) [2023-10-07 20:21:09,523][67838] Updated weights for policy 0, policy_version 16782 (0.0009) [2023-10-07 20:21:09,661][67871] Updated weights for policy 1, policy_version 16810 (0.0008) [2023-10-07 20:21:09,897][67838] Updated weights for policy 0, policy_version 16792 (0.0009) [2023-10-07 20:21:10,024][67871] Updated weights for policy 1, policy_version 16820 (0.0007) [2023-10-07 20:21:10,390][67871] Updated weights for policy 1, policy_version 16830 (0.0009) [2023-10-07 20:21:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 34439168. Throughput: 0: 1638.4, 1: 1643.2. Samples: 8613302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:21:12,478][66916] Avg episode reward: [(0, '33.580'), (1, '34.020')] [2023-10-07 20:21:13,999][67838] Updated weights for policy 0, policy_version 16802 (0.0008) [2023-10-07 20:21:14,372][67838] Updated weights for policy 0, policy_version 16812 (0.0008) [2023-10-07 20:21:14,663][67871] Updated weights for policy 1, policy_version 16840 (0.0008) [2023-10-07 20:21:14,750][67838] Updated weights for policy 0, policy_version 16822 (0.0008) [2023-10-07 20:21:15,029][67871] Updated weights for policy 1, policy_version 16850 (0.0007) [2023-10-07 20:21:15,118][67838] Updated weights for policy 0, policy_version 16832 (0.0009) [2023-10-07 20:21:15,392][67871] Updated weights for policy 1, policy_version 16860 (0.0008) [2023-10-07 20:21:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 34504704. Throughput: 0: 1644.0, 1: 1643.6. Samples: 8632570. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-07 20:21:17,477][66916] Avg episode reward: [(0, '35.970'), (1, '34.460')] [2023-10-07 20:21:17,478][67511] Saving new best policy, reward=35.970! [2023-10-07 20:21:19,287][67838] Updated weights for policy 0, policy_version 16842 (0.0008) [2023-10-07 20:21:19,414][67871] Updated weights for policy 1, policy_version 16870 (0.0008) [2023-10-07 20:21:19,662][67838] Updated weights for policy 0, policy_version 16852 (0.0007) [2023-10-07 20:21:19,777][67871] Updated weights for policy 1, policy_version 16880 (0.0009) [2023-10-07 20:21:20,033][67838] Updated weights for policy 0, policy_version 16862 (0.0008) [2023-10-07 20:21:20,146][67871] Updated weights for policy 1, policy_version 16890 (0.0009) [2023-10-07 20:21:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 34570240. Throughput: 0: 1644.1, 1: 1648.0. Samples: 8653182. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-07 20:21:22,478][66916] Avg episode reward: [(0, '34.220'), (1, '34.960')] [2023-10-07 20:21:24,099][67871] Updated weights for policy 1, policy_version 16900 (0.0007) [2023-10-07 20:21:24,133][67838] Updated weights for policy 0, policy_version 16872 (0.0008) [2023-10-07 20:21:24,466][67871] Updated weights for policy 1, policy_version 16910 (0.0008) [2023-10-07 20:21:24,495][67838] Updated weights for policy 0, policy_version 16882 (0.0007) [2023-10-07 20:21:24,845][67871] Updated weights for policy 1, policy_version 16920 (0.0007) [2023-10-07 20:21:24,876][67838] Updated weights for policy 0, policy_version 16892 (0.0007) [2023-10-07 20:21:27,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 34635776. Throughput: 0: 1641.8, 1: 1642.3. Samples: 8662740. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-07 20:21:27,478][66916] Avg episode reward: [(0, '33.270'), (1, '34.370')] [2023-10-07 20:21:28,977][67871] Updated weights for policy 1, policy_version 16930 (0.0007) [2023-10-07 20:21:29,058][67838] Updated weights for policy 0, policy_version 16902 (0.0010) [2023-10-07 20:21:29,401][67871] Updated weights for policy 1, policy_version 16940 (0.0008) [2023-10-07 20:21:29,433][67838] Updated weights for policy 0, policy_version 16912 (0.0007) [2023-10-07 20:21:29,774][67871] Updated weights for policy 1, policy_version 16950 (0.0008) [2023-10-07 20:21:29,803][67838] Updated weights for policy 0, policy_version 16922 (0.0009) [2023-10-07 20:21:30,139][67871] Updated weights for policy 1, policy_version 16960 (0.0008) [2023-10-07 20:21:32,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 34701312. Throughput: 0: 1645.5, 1: 1651.5. Samples: 8682330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:21:32,477][66916] Avg episode reward: [(0, '35.280'), (1, '35.360')] [2023-10-07 20:21:34,074][67838] Updated weights for policy 0, policy_version 16932 (0.0010) [2023-10-07 20:21:34,236][67871] Updated weights for policy 1, policy_version 16970 (0.0009) [2023-10-07 20:21:34,455][67838] Updated weights for policy 0, policy_version 16942 (0.0008) [2023-10-07 20:21:34,602][67871] Updated weights for policy 1, policy_version 16980 (0.0010) [2023-10-07 20:21:34,834][67838] Updated weights for policy 0, policy_version 16952 (0.0011) [2023-10-07 20:21:34,968][67871] Updated weights for policy 1, policy_version 16990 (0.0009) [2023-10-07 20:21:37,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 34766848. Throughput: 0: 1651.3, 1: 1656.0. Samples: 8702806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:21:37,478][66916] Avg episode reward: [(0, '33.190'), (1, '34.330')] [2023-10-07 20:21:38,964][67838] Updated weights for policy 0, policy_version 16962 (0.0008) [2023-10-07 20:21:39,080][67871] Updated weights for policy 1, policy_version 17000 (0.0009) [2023-10-07 20:21:39,338][67838] Updated weights for policy 0, policy_version 16972 (0.0007) [2023-10-07 20:21:39,448][67871] Updated weights for policy 1, policy_version 17010 (0.0009) [2023-10-07 20:21:39,711][67838] Updated weights for policy 0, policy_version 16982 (0.0007) [2023-10-07 20:21:39,822][67871] Updated weights for policy 1, policy_version 17020 (0.0008) [2023-10-07 20:21:40,072][67838] Updated weights for policy 0, policy_version 16992 (0.0007) [2023-10-07 20:21:42,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 34832384. Throughput: 0: 1643.4, 1: 1640.4. Samples: 8711836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:21:42,477][66916] Avg episode reward: [(0, '33.580'), (1, '35.650')] [2023-10-07 20:21:43,978][67871] Updated weights for policy 1, policy_version 17030 (0.0008) [2023-10-07 20:21:44,063][67838] Updated weights for policy 0, policy_version 17002 (0.0009) [2023-10-07 20:21:44,347][67871] Updated weights for policy 1, policy_version 17040 (0.0008) [2023-10-07 20:21:44,421][67838] Updated weights for policy 0, policy_version 17012 (0.0008) [2023-10-07 20:21:44,709][67871] Updated weights for policy 1, policy_version 17050 (0.0009) [2023-10-07 20:21:44,802][67838] Updated weights for policy 0, policy_version 17022 (0.0009) [2023-10-07 20:21:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 34897920. Throughput: 0: 1655.2, 1: 1653.5. Samples: 8732054. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 20:21:47,478][66916] Avg episode reward: [(0, '34.000'), (1, '35.260')] [2023-10-07 20:21:48,984][67871] Updated weights for policy 1, policy_version 17060 (0.0007) [2023-10-07 20:21:49,230][67838] Updated weights for policy 0, policy_version 17032 (0.0007) [2023-10-07 20:21:49,350][67871] Updated weights for policy 1, policy_version 17070 (0.0009) [2023-10-07 20:21:49,607][67838] Updated weights for policy 0, policy_version 17042 (0.0007) [2023-10-07 20:21:49,715][67871] Updated weights for policy 1, policy_version 17080 (0.0009) [2023-10-07 20:21:49,980][67838] Updated weights for policy 0, policy_version 17052 (0.0008) [2023-10-07 20:21:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 34963456. Throughput: 0: 1650.8, 1: 1648.3. Samples: 8752036. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 20:21:52,477][66916] Avg episode reward: [(0, '32.790'), (1, '35.170')] [2023-10-07 20:21:53,807][67871] Updated weights for policy 1, policy_version 17090 (0.0007) [2023-10-07 20:21:53,892][67838] Updated weights for policy 0, policy_version 17062 (0.0009) [2023-10-07 20:21:54,170][67871] Updated weights for policy 1, policy_version 17100 (0.0009) [2023-10-07 20:21:54,271][67838] Updated weights for policy 0, policy_version 17072 (0.0008) [2023-10-07 20:21:54,549][67871] Updated weights for policy 1, policy_version 17110 (0.0007) [2023-10-07 20:21:54,637][67838] Updated weights for policy 0, policy_version 17082 (0.0010) [2023-10-07 20:21:54,913][67871] Updated weights for policy 1, policy_version 17120 (0.0009) [2023-10-07 20:21:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 35028992. Throughput: 0: 1647.9, 1: 1644.9. Samples: 8761478. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 20:21:57,478][66916] Avg episode reward: [(0, '32.120'), (1, '34.190')] [2023-10-07 20:21:58,950][67838] Updated weights for policy 0, policy_version 17092 (0.0008) [2023-10-07 20:21:58,988][67871] Updated weights for policy 1, policy_version 17130 (0.0008) [2023-10-07 20:21:59,330][67838] Updated weights for policy 0, policy_version 17102 (0.0010) [2023-10-07 20:21:59,365][67871] Updated weights for policy 1, policy_version 17140 (0.0008) [2023-10-07 20:21:59,700][67838] Updated weights for policy 0, policy_version 17112 (0.0009) [2023-10-07 20:21:59,727][67871] Updated weights for policy 1, policy_version 17150 (0.0009) [2023-10-07 20:22:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 35094528. Throughput: 0: 1652.0, 1: 1659.0. Samples: 8781564. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-07 20:22:02,478][66916] Avg episode reward: [(0, '33.680'), (1, '35.140')] [2023-10-07 20:22:03,841][67838] Updated weights for policy 0, policy_version 17122 (0.0008) [2023-10-07 20:22:03,989][67871] Updated weights for policy 1, policy_version 17160 (0.0008) [2023-10-07 20:22:04,233][67838] Updated weights for policy 0, policy_version 17132 (0.0008) [2023-10-07 20:22:04,359][67871] Updated weights for policy 1, policy_version 17170 (0.0008) [2023-10-07 20:22:04,605][67838] Updated weights for policy 0, policy_version 17142 (0.0007) [2023-10-07 20:22:04,721][67871] Updated weights for policy 1, policy_version 17180 (0.0007) [2023-10-07 20:22:04,975][67838] Updated weights for policy 0, policy_version 17152 (0.0008) [2023-10-07 20:22:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 35160064. Throughput: 0: 1656.2, 1: 1654.0. Samples: 8802144. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-07 20:22:07,478][66916] Avg episode reward: [(0, '32.950'), (1, '34.900')] [2023-10-07 20:22:09,010][67871] Updated weights for policy 1, policy_version 17190 (0.0009) [2023-10-07 20:22:09,046][67838] Updated weights for policy 0, policy_version 17162 (0.0008) [2023-10-07 20:22:09,376][67871] Updated weights for policy 1, policy_version 17200 (0.0008) [2023-10-07 20:22:09,419][67838] Updated weights for policy 0, policy_version 17172 (0.0007) [2023-10-07 20:22:09,746][67871] Updated weights for policy 1, policy_version 17210 (0.0009) [2023-10-07 20:22:09,788][67838] Updated weights for policy 0, policy_version 17182 (0.0008) [2023-10-07 20:22:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 35225600. Throughput: 0: 1649.3, 1: 1646.6. Samples: 8811056. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-07 20:22:12,477][66916] Avg episode reward: [(0, '34.160'), (1, '35.050')] [2023-10-07 20:22:13,825][67838] Updated weights for policy 0, policy_version 17192 (0.0007) [2023-10-07 20:22:13,866][67871] Updated weights for policy 1, policy_version 17220 (0.0008) [2023-10-07 20:22:14,212][67838] Updated weights for policy 0, policy_version 17202 (0.0007) [2023-10-07 20:22:14,230][67871] Updated weights for policy 1, policy_version 17230 (0.0009) [2023-10-07 20:22:14,582][67838] Updated weights for policy 0, policy_version 17212 (0.0008) [2023-10-07 20:22:14,605][67871] Updated weights for policy 1, policy_version 17240 (0.0009) [2023-10-07 20:22:17,477][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 35291136. Throughput: 0: 1656.8, 1: 1650.0. Samples: 8831140. Policy #0 lag: (min: 23.0, avg: 24.2, max: 43.0) [2023-10-07 20:22:17,478][66916] Avg episode reward: [(0, '32.790'), (1, '35.870')] [2023-10-07 20:22:18,707][67871] Updated weights for policy 1, policy_version 17250 (0.0009) [2023-10-07 20:22:18,777][67838] Updated weights for policy 0, policy_version 17222 (0.0007) [2023-10-07 20:22:19,076][67871] Updated weights for policy 1, policy_version 17260 (0.0009) [2023-10-07 20:22:19,150][67838] Updated weights for policy 0, policy_version 17232 (0.0007) [2023-10-07 20:22:19,451][67871] Updated weights for policy 1, policy_version 17270 (0.0007) [2023-10-07 20:22:19,523][67838] Updated weights for policy 0, policy_version 17242 (0.0007) [2023-10-07 20:22:19,815][67871] Updated weights for policy 1, policy_version 17280 (0.0007) [2023-10-07 20:22:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 35356672. Throughput: 0: 1656.5, 1: 1648.5. Samples: 8851530. Policy #0 lag: (min: 23.0, avg: 24.2, max: 43.0) [2023-10-07 20:22:22,477][66916] Avg episode reward: [(0, '34.200'), (1, '34.700')] [2023-10-07 20:22:23,692][67838] Updated weights for policy 0, policy_version 17252 (0.0007) [2023-10-07 20:22:24,007][67871] Updated weights for policy 1, policy_version 17290 (0.0008) [2023-10-07 20:22:24,055][67838] Updated weights for policy 0, policy_version 17262 (0.0007) [2023-10-07 20:22:24,370][67871] Updated weights for policy 1, policy_version 17300 (0.0008) [2023-10-07 20:22:24,423][67838] Updated weights for policy 0, policy_version 17272 (0.0009) [2023-10-07 20:22:24,735][67871] Updated weights for policy 1, policy_version 17310 (0.0009) [2023-10-07 20:22:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 35422208. Throughput: 0: 1651.6, 1: 1651.2. Samples: 8860460. Policy #0 lag: (min: 23.0, avg: 24.2, max: 43.0) [2023-10-07 20:22:27,477][66916] Avg episode reward: [(0, '32.970'), (1, '35.190')] [2023-10-07 20:22:28,670][67838] Updated weights for policy 0, policy_version 17282 (0.0008) [2023-10-07 20:22:28,818][67871] Updated weights for policy 1, policy_version 17320 (0.0007) [2023-10-07 20:22:29,047][67838] Updated weights for policy 0, policy_version 17292 (0.0007) [2023-10-07 20:22:29,187][67871] Updated weights for policy 1, policy_version 17330 (0.0008) [2023-10-07 20:22:29,421][67838] Updated weights for policy 0, policy_version 17302 (0.0007) [2023-10-07 20:22:29,550][67871] Updated weights for policy 1, policy_version 17340 (0.0009) [2023-10-07 20:22:29,792][67838] Updated weights for policy 0, policy_version 17312 (0.0008) [2023-10-07 20:22:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35487744. Throughput: 0: 1648.5, 1: 1656.7. Samples: 8880786. Policy #0 lag: (min: 10.0, avg: 18.4, max: 42.0) [2023-10-07 20:22:32,477][66916] Avg episode reward: [(0, '34.390'), (1, '34.090')] [2023-10-07 20:22:33,739][67871] Updated weights for policy 1, policy_version 17350 (0.0008) [2023-10-07 20:22:33,864][67838] Updated weights for policy 0, policy_version 17322 (0.0007) [2023-10-07 20:22:34,104][67871] Updated weights for policy 1, policy_version 17360 (0.0007) [2023-10-07 20:22:34,246][67838] Updated weights for policy 0, policy_version 17332 (0.0008) [2023-10-07 20:22:34,479][67871] Updated weights for policy 1, policy_version 17370 (0.0007) [2023-10-07 20:22:34,630][67838] Updated weights for policy 0, policy_version 17342 (0.0007) [2023-10-07 20:22:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 35553280. Throughput: 0: 1651.3, 1: 1657.1. Samples: 8900916. Policy #0 lag: (min: 10.0, avg: 18.4, max: 42.0) [2023-10-07 20:22:37,477][66916] Avg episode reward: [(0, '32.840'), (1, '35.380')] [2023-10-07 20:22:38,523][67871] Updated weights for policy 1, policy_version 17380 (0.0007) [2023-10-07 20:22:38,767][67838] Updated weights for policy 0, policy_version 17352 (0.0007) [2023-10-07 20:22:38,892][67871] Updated weights for policy 1, policy_version 17390 (0.0010) [2023-10-07 20:22:39,141][67838] Updated weights for policy 0, policy_version 17362 (0.0007) [2023-10-07 20:22:39,260][67871] Updated weights for policy 1, policy_version 17400 (0.0009) [2023-10-07 20:22:39,527][67838] Updated weights for policy 0, policy_version 17372 (0.0007) [2023-10-07 20:22:42,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35618816. Throughput: 0: 1650.5, 1: 1650.7. Samples: 8910034. Policy #0 lag: (min: 10.0, avg: 18.4, max: 42.0) [2023-10-07 20:22:42,478][66916] Avg episode reward: [(0, '33.260'), (1, '34.640')] [2023-10-07 20:22:43,474][67871] Updated weights for policy 1, policy_version 17410 (0.0007) [2023-10-07 20:22:43,563][67838] Updated weights for policy 0, policy_version 17382 (0.0008) [2023-10-07 20:22:43,845][67871] Updated weights for policy 1, policy_version 17420 (0.0007) [2023-10-07 20:22:43,942][67838] Updated weights for policy 0, policy_version 17392 (0.0008) [2023-10-07 20:22:44,207][67871] Updated weights for policy 1, policy_version 17430 (0.0008) [2023-10-07 20:22:44,310][67838] Updated weights for policy 0, policy_version 17402 (0.0010) [2023-10-07 20:22:44,572][67871] Updated weights for policy 1, policy_version 17440 (0.0008) [2023-10-07 20:22:47,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35684352. Throughput: 0: 1653.9, 1: 1652.6. Samples: 8930354. Policy #0 lag: (min: 30.0, avg: 34.6, max: 62.0) [2023-10-07 20:22:47,478][66916] Avg episode reward: [(0, '35.860'), (1, '34.450')] [2023-10-07 20:22:48,588][67838] Updated weights for policy 0, policy_version 17412 (0.0009) [2023-10-07 20:22:48,903][67871] Updated weights for policy 1, policy_version 17450 (0.0008) [2023-10-07 20:22:48,958][67838] Updated weights for policy 0, policy_version 17422 (0.0007) [2023-10-07 20:22:49,267][67871] Updated weights for policy 1, policy_version 17460 (0.0007) [2023-10-07 20:22:49,337][67838] Updated weights for policy 0, policy_version 17432 (0.0009) [2023-10-07 20:22:49,636][67871] Updated weights for policy 1, policy_version 17470 (0.0008) [2023-10-07 20:22:52,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35749888. Throughput: 0: 1647.8, 1: 1650.8. Samples: 8950580. Policy #0 lag: (min: 30.0, avg: 34.6, max: 62.0) [2023-10-07 20:22:52,478][66916] Avg episode reward: [(0, '33.770'), (1, '35.880')] [2023-10-07 20:22:53,659][67838] Updated weights for policy 0, policy_version 17442 (0.0011) [2023-10-07 20:22:53,671][67871] Updated weights for policy 1, policy_version 17480 (0.0008) [2023-10-07 20:22:54,036][67871] Updated weights for policy 1, policy_version 17490 (0.0007) [2023-10-07 20:22:54,054][67838] Updated weights for policy 0, policy_version 17452 (0.0007) [2023-10-07 20:22:54,395][67871] Updated weights for policy 1, policy_version 17500 (0.0007) [2023-10-07 20:22:54,432][67838] Updated weights for policy 0, policy_version 17462 (0.0008) [2023-10-07 20:22:54,801][67838] Updated weights for policy 0, policy_version 17472 (0.0009) [2023-10-07 20:22:57,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35815424. Throughput: 0: 1649.1, 1: 1649.5. Samples: 8959494. Policy #0 lag: (min: 30.0, avg: 34.6, max: 62.0) [2023-10-07 20:22:57,478][66916] Avg episode reward: [(0, '33.760'), (1, '34.660')] [2023-10-07 20:22:58,471][67871] Updated weights for policy 1, policy_version 17510 (0.0010) [2023-10-07 20:22:58,850][67871] Updated weights for policy 1, policy_version 17520 (0.0009) [2023-10-07 20:22:58,895][67838] Updated weights for policy 0, policy_version 17482 (0.0007) [2023-10-07 20:22:59,226][67871] Updated weights for policy 1, policy_version 17530 (0.0009) [2023-10-07 20:22:59,264][67838] Updated weights for policy 0, policy_version 17492 (0.0007) [2023-10-07 20:22:59,637][67838] Updated weights for policy 0, policy_version 17502 (0.0007) [2023-10-07 20:23:02,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35880960. Throughput: 0: 1644.8, 1: 1656.8. Samples: 8979716. Policy #0 lag: (min: 9.0, avg: 16.1, max: 41.0) [2023-10-07 20:23:02,478][66916] Avg episode reward: [(0, '33.370'), (1, '34.390')] [2023-10-07 20:23:03,453][67871] Updated weights for policy 1, policy_version 17540 (0.0009) [2023-10-07 20:23:03,685][67838] Updated weights for policy 0, policy_version 17512 (0.0008) [2023-10-07 20:23:03,822][67871] Updated weights for policy 1, policy_version 17550 (0.0007) [2023-10-07 20:23:04,063][67838] Updated weights for policy 0, policy_version 17522 (0.0007) [2023-10-07 20:23:04,187][67871] Updated weights for policy 1, policy_version 17560 (0.0008) [2023-10-07 20:23:04,432][67838] Updated weights for policy 0, policy_version 17532 (0.0007) [2023-10-07 20:23:07,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35946496. Throughput: 0: 1649.1, 1: 1653.6. Samples: 9000154. Policy #0 lag: (min: 9.0, avg: 16.1, max: 41.0) [2023-10-07 20:23:07,478][66916] Avg episode reward: [(0, '33.730'), (1, '35.630')] [2023-10-07 20:23:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000017568_17989632.pth... [2023-10-07 20:23:07,490][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000017536_17956864.pth... [2023-10-07 20:23:07,519][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000016032_16416768.pth [2023-10-07 20:23:07,525][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000016000_16384000.pth [2023-10-07 20:23:08,337][67871] Updated weights for policy 1, policy_version 17570 (0.0009) [2023-10-07 20:23:08,619][67838] Updated weights for policy 0, policy_version 17542 (0.0009) [2023-10-07 20:23:08,701][67871] Updated weights for policy 1, policy_version 17580 (0.0008) [2023-10-07 20:23:08,983][67838] Updated weights for policy 0, policy_version 17552 (0.0007) [2023-10-07 20:23:09,066][67871] Updated weights for policy 1, policy_version 17590 (0.0011) [2023-10-07 20:23:09,342][67838] Updated weights for policy 0, policy_version 17562 (0.0007) [2023-10-07 20:23:09,435][67871] Updated weights for policy 1, policy_version 17600 (0.0010) [2023-10-07 20:23:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36012032. Throughput: 0: 1650.9, 1: 1651.2. Samples: 9009058. Policy #0 lag: (min: 9.0, avg: 16.1, max: 41.0) [2023-10-07 20:23:12,477][66916] Avg episode reward: [(0, '30.090'), (1, '34.050')] [2023-10-07 20:23:13,582][67838] Updated weights for policy 0, policy_version 17572 (0.0008) [2023-10-07 20:23:13,684][67871] Updated weights for policy 1, policy_version 17610 (0.0008) [2023-10-07 20:23:13,957][67838] Updated weights for policy 0, policy_version 17582 (0.0009) [2023-10-07 20:23:14,050][67871] Updated weights for policy 1, policy_version 17620 (0.0008) [2023-10-07 20:23:14,317][67838] Updated weights for policy 0, policy_version 17592 (0.0009) [2023-10-07 20:23:14,420][67871] Updated weights for policy 1, policy_version 17630 (0.0007) [2023-10-07 20:23:17,477][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36077568. Throughput: 0: 1648.0, 1: 1652.0. Samples: 9029286. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-07 20:23:17,478][66916] Avg episode reward: [(0, '29.680'), (1, '34.520')] [2023-10-07 20:23:18,230][67871] Updated weights for policy 1, policy_version 17640 (0.0010) [2023-10-07 20:23:18,386][67838] Updated weights for policy 0, policy_version 17602 (0.0009) [2023-10-07 20:23:18,604][67871] Updated weights for policy 1, policy_version 17650 (0.0007) [2023-10-07 20:23:18,767][67838] Updated weights for policy 0, policy_version 17612 (0.0008) [2023-10-07 20:23:18,965][67871] Updated weights for policy 1, policy_version 17660 (0.0007) [2023-10-07 20:23:19,133][67838] Updated weights for policy 0, policy_version 17622 (0.0008) [2023-10-07 20:23:19,506][67838] Updated weights for policy 0, policy_version 17632 (0.0007) [2023-10-07 20:23:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36143104. Throughput: 0: 1646.8, 1: 1657.5. Samples: 9049612. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-07 20:23:22,478][66916] Avg episode reward: [(0, '28.980'), (1, '34.320')] [2023-10-07 20:23:23,214][67871] Updated weights for policy 1, policy_version 17670 (0.0007) [2023-10-07 20:23:23,577][67871] Updated weights for policy 1, policy_version 17680 (0.0008) [2023-10-07 20:23:23,762][67838] Updated weights for policy 0, policy_version 17642 (0.0008) [2023-10-07 20:23:23,950][67871] Updated weights for policy 1, policy_version 17690 (0.0008) [2023-10-07 20:23:24,134][67838] Updated weights for policy 0, policy_version 17652 (0.0007) [2023-10-07 20:23:24,506][67838] Updated weights for policy 0, policy_version 17662 (0.0008) [2023-10-07 20:23:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36208640. Throughput: 0: 1648.2, 1: 1655.9. Samples: 9058720. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-07 20:23:27,477][66916] Avg episode reward: [(0, '30.720'), (1, '33.770')] [2023-10-07 20:23:28,144][67871] Updated weights for policy 1, policy_version 17700 (0.0008) [2023-10-07 20:23:28,513][67838] Updated weights for policy 0, policy_version 17672 (0.0008) [2023-10-07 20:23:28,513][67871] Updated weights for policy 1, policy_version 17710 (0.0008) [2023-10-07 20:23:28,880][67871] Updated weights for policy 1, policy_version 17720 (0.0008) [2023-10-07 20:23:28,890][67838] Updated weights for policy 0, policy_version 17682 (0.0008) [2023-10-07 20:23:29,261][67838] Updated weights for policy 0, policy_version 17692 (0.0009) [2023-10-07 20:23:32,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36274176. Throughput: 0: 1650.3, 1: 1655.2. Samples: 9079104. Policy #0 lag: (min: 12.0, avg: 14.2, max: 44.0) [2023-10-07 20:23:32,478][66916] Avg episode reward: [(0, '32.400'), (1, '35.980')] [2023-10-07 20:23:32,943][67871] Updated weights for policy 1, policy_version 17730 (0.0008) [2023-10-07 20:23:33,314][67871] Updated weights for policy 1, policy_version 17740 (0.0008) [2023-10-07 20:23:33,361][67838] Updated weights for policy 0, policy_version 17702 (0.0009) [2023-10-07 20:23:33,680][67871] Updated weights for policy 1, policy_version 17750 (0.0007) [2023-10-07 20:23:33,738][67838] Updated weights for policy 0, policy_version 17712 (0.0009) [2023-10-07 20:23:34,050][67871] Updated weights for policy 1, policy_version 17760 (0.0008) [2023-10-07 20:23:34,103][67838] Updated weights for policy 0, policy_version 17722 (0.0008) [2023-10-07 20:23:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 36339712. Throughput: 0: 1651.5, 1: 1651.7. Samples: 9099226. Policy #0 lag: (min: 12.0, avg: 14.2, max: 44.0) [2023-10-07 20:23:37,478][66916] Avg episode reward: [(0, '31.720'), (1, '34.300')] [2023-10-07 20:23:38,260][67871] Updated weights for policy 1, policy_version 17770 (0.0009) [2023-10-07 20:23:38,283][67838] Updated weights for policy 0, policy_version 17732 (0.0008) [2023-10-07 20:23:38,623][67871] Updated weights for policy 1, policy_version 17780 (0.0009) [2023-10-07 20:23:38,668][67838] Updated weights for policy 0, policy_version 17742 (0.0008) [2023-10-07 20:23:38,999][67871] Updated weights for policy 1, policy_version 17790 (0.0009) [2023-10-07 20:23:39,033][67838] Updated weights for policy 0, policy_version 17752 (0.0008) [2023-10-07 20:23:42,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 36405248. Throughput: 0: 1652.7, 1: 1650.7. Samples: 9108146. Policy #0 lag: (min: 12.0, avg: 14.2, max: 44.0) [2023-10-07 20:23:42,477][66916] Avg episode reward: [(0, '30.790'), (1, '34.060')] [2023-10-07 20:23:43,130][67838] Updated weights for policy 0, policy_version 17762 (0.0009) [2023-10-07 20:23:43,271][67871] Updated weights for policy 1, policy_version 17800 (0.0007) [2023-10-07 20:23:43,509][67838] Updated weights for policy 0, policy_version 17772 (0.0007) [2023-10-07 20:23:43,631][67871] Updated weights for policy 1, policy_version 17810 (0.0008) [2023-10-07 20:23:43,888][67838] Updated weights for policy 0, policy_version 17782 (0.0008) [2023-10-07 20:23:43,998][67871] Updated weights for policy 1, policy_version 17820 (0.0009) [2023-10-07 20:23:44,258][67838] Updated weights for policy 0, policy_version 17792 (0.0008) [2023-10-07 20:23:47,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36470784. Throughput: 0: 1654.1, 1: 1649.1. Samples: 9128362. Policy #0 lag: (min: 10.0, avg: 10.3, max: 20.0) [2023-10-07 20:23:47,477][66916] Avg episode reward: [(0, '32.900'), (1, '35.320')] [2023-10-07 20:23:48,195][67871] Updated weights for policy 1, policy_version 17830 (0.0008) [2023-10-07 20:23:48,425][67838] Updated weights for policy 0, policy_version 17802 (0.0009) [2023-10-07 20:23:48,573][67871] Updated weights for policy 1, policy_version 17840 (0.0008) [2023-10-07 20:23:48,796][67838] Updated weights for policy 0, policy_version 17812 (0.0008) [2023-10-07 20:23:48,946][67871] Updated weights for policy 1, policy_version 17850 (0.0009) [2023-10-07 20:23:49,161][67838] Updated weights for policy 0, policy_version 17822 (0.0008) [2023-10-07 20:23:52,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36536320. Throughput: 0: 1646.8, 1: 1651.6. Samples: 9148584. Policy #0 lag: (min: 10.0, avg: 10.3, max: 20.0) [2023-10-07 20:23:52,478][66916] Avg episode reward: [(0, '31.770'), (1, '34.320')] [2023-10-07 20:23:53,026][67871] Updated weights for policy 1, policy_version 17860 (0.0008) [2023-10-07 20:23:53,159][67838] Updated weights for policy 0, policy_version 17832 (0.0009) [2023-10-07 20:23:53,390][67871] Updated weights for policy 1, policy_version 17870 (0.0008) [2023-10-07 20:23:53,529][67838] Updated weights for policy 0, policy_version 17842 (0.0008) [2023-10-07 20:23:53,756][67871] Updated weights for policy 1, policy_version 17880 (0.0008) [2023-10-07 20:23:53,901][67838] Updated weights for policy 0, policy_version 17852 (0.0010) [2023-10-07 20:23:57,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36601856. Throughput: 0: 1650.1, 1: 1651.5. Samples: 9157632. Policy #0 lag: (min: 10.0, avg: 10.3, max: 20.0) [2023-10-07 20:23:57,478][66916] Avg episode reward: [(0, '31.770'), (1, '34.830')] [2023-10-07 20:23:57,740][67871] Updated weights for policy 1, policy_version 17890 (0.0010) [2023-10-07 20:23:57,992][67838] Updated weights for policy 0, policy_version 17862 (0.0010) [2023-10-07 20:23:58,109][67871] Updated weights for policy 1, policy_version 17900 (0.0010) [2023-10-07 20:23:58,364][67838] Updated weights for policy 0, policy_version 17872 (0.0010) [2023-10-07 20:23:58,474][67871] Updated weights for policy 1, policy_version 17910 (0.0009) [2023-10-07 20:23:58,740][67838] Updated weights for policy 0, policy_version 17882 (0.0009) [2023-10-07 20:23:58,847][67871] Updated weights for policy 1, policy_version 17920 (0.0009) [2023-10-07 20:24:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36667392. Throughput: 0: 1650.7, 1: 1649.3. Samples: 9177786. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 20:24:02,478][66916] Avg episode reward: [(0, '32.140'), (1, '33.830')] [2023-10-07 20:24:02,906][67838] Updated weights for policy 0, policy_version 17892 (0.0010) [2023-10-07 20:24:03,137][67871] Updated weights for policy 1, policy_version 17930 (0.0008) [2023-10-07 20:24:03,275][67838] Updated weights for policy 0, policy_version 17902 (0.0008) [2023-10-07 20:24:03,503][67871] Updated weights for policy 1, policy_version 17940 (0.0007) [2023-10-07 20:24:03,648][67838] Updated weights for policy 0, policy_version 17912 (0.0008) [2023-10-07 20:24:03,871][67871] Updated weights for policy 1, policy_version 17950 (0.0009) [2023-10-07 20:24:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36732928. Throughput: 0: 1651.4, 1: 1653.2. Samples: 9198322. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 20:24:07,478][66916] Avg episode reward: [(0, '31.910'), (1, '34.890')] [2023-10-07 20:24:07,769][67871] Updated weights for policy 1, policy_version 17960 (0.0008) [2023-10-07 20:24:07,951][67838] Updated weights for policy 0, policy_version 17922 (0.0011) [2023-10-07 20:24:08,131][67871] Updated weights for policy 1, policy_version 17970 (0.0007) [2023-10-07 20:24:08,320][67838] Updated weights for policy 0, policy_version 17932 (0.0011) [2023-10-07 20:24:08,507][67871] Updated weights for policy 1, policy_version 17980 (0.0009) [2023-10-07 20:24:08,687][67838] Updated weights for policy 0, policy_version 17942 (0.0008) [2023-10-07 20:24:09,059][67838] Updated weights for policy 0, policy_version 17952 (0.0009) [2023-10-07 20:24:12,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 36798464. Throughput: 0: 1646.7, 1: 1653.1. Samples: 9207214. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 20:24:12,478][66916] Avg episode reward: [(0, '34.160'), (1, '34.850')] [2023-10-07 20:24:12,852][67871] Updated weights for policy 1, policy_version 17990 (0.0007) [2023-10-07 20:24:13,209][67871] Updated weights for policy 1, policy_version 18000 (0.0007) [2023-10-07 20:24:13,377][67838] Updated weights for policy 0, policy_version 17962 (0.0007) [2023-10-07 20:24:13,578][67871] Updated weights for policy 1, policy_version 18010 (0.0007) [2023-10-07 20:24:13,744][67838] Updated weights for policy 0, policy_version 17972 (0.0011) [2023-10-07 20:24:14,116][67838] Updated weights for policy 0, policy_version 17982 (0.0008) [2023-10-07 20:24:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36864000. Throughput: 0: 1642.4, 1: 1650.2. Samples: 9227270. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-07 20:24:17,478][66916] Avg episode reward: [(0, '31.830'), (1, '34.850')] [2023-10-07 20:24:17,736][67871] Updated weights for policy 1, policy_version 18020 (0.0009) [2023-10-07 20:24:18,118][67871] Updated weights for policy 1, policy_version 18030 (0.0010) [2023-10-07 20:24:18,213][67838] Updated weights for policy 0, policy_version 17992 (0.0008) [2023-10-07 20:24:18,485][67871] Updated weights for policy 1, policy_version 18040 (0.0007) [2023-10-07 20:24:18,585][67838] Updated weights for policy 0, policy_version 18002 (0.0008) [2023-10-07 20:24:18,943][67838] Updated weights for policy 0, policy_version 18012 (0.0008) [2023-10-07 20:24:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36929536. Throughput: 0: 1641.9, 1: 1655.2. Samples: 9247594. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-07 20:24:22,477][66916] Avg episode reward: [(0, '33.160'), (1, '35.830')] [2023-10-07 20:24:22,689][67871] Updated weights for policy 1, policy_version 18050 (0.0007) [2023-10-07 20:24:23,061][67871] Updated weights for policy 1, policy_version 18060 (0.0007) [2023-10-07 20:24:23,125][67838] Updated weights for policy 0, policy_version 18022 (0.0008) [2023-10-07 20:24:23,444][67871] Updated weights for policy 1, policy_version 18070 (0.0007) [2023-10-07 20:24:23,520][67838] Updated weights for policy 0, policy_version 18032 (0.0007) [2023-10-07 20:24:23,814][67871] Updated weights for policy 1, policy_version 18080 (0.0008) [2023-10-07 20:24:23,888][67838] Updated weights for policy 0, policy_version 18042 (0.0007) [2023-10-07 20:24:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36995072. Throughput: 0: 1640.4, 1: 1652.7. Samples: 9256332. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-07 20:24:27,477][66916] Avg episode reward: [(0, '32.760'), (1, '34.630')] [2023-10-07 20:24:27,982][67871] Updated weights for policy 1, policy_version 18090 (0.0007) [2023-10-07 20:24:28,115][67838] Updated weights for policy 0, policy_version 18052 (0.0007) [2023-10-07 20:24:28,346][67871] Updated weights for policy 1, policy_version 18100 (0.0009) [2023-10-07 20:24:28,486][67838] Updated weights for policy 0, policy_version 18062 (0.0008) [2023-10-07 20:24:28,721][67871] Updated weights for policy 1, policy_version 18110 (0.0008) [2023-10-07 20:24:28,862][67838] Updated weights for policy 0, policy_version 18072 (0.0008) [2023-10-07 20:24:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37060608. Throughput: 0: 1639.2, 1: 1656.0. Samples: 9276648. Policy #0 lag: (min: 2.0, avg: 2.1, max: 8.0) [2023-10-07 20:24:32,477][66916] Avg episode reward: [(0, '32.850'), (1, '34.850')] [2023-10-07 20:24:32,706][67871] Updated weights for policy 1, policy_version 18120 (0.0011) [2023-10-07 20:24:32,930][67838] Updated weights for policy 0, policy_version 18082 (0.0009) [2023-10-07 20:24:33,068][67871] Updated weights for policy 1, policy_version 18130 (0.0007) [2023-10-07 20:24:33,308][67838] Updated weights for policy 0, policy_version 18092 (0.0010) [2023-10-07 20:24:33,444][67871] Updated weights for policy 1, policy_version 18140 (0.0008) [2023-10-07 20:24:33,670][67838] Updated weights for policy 0, policy_version 18102 (0.0007) [2023-10-07 20:24:34,040][67838] Updated weights for policy 0, policy_version 18112 (0.0010) [2023-10-07 20:24:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 37126144. Throughput: 0: 1647.0, 1: 1652.4. Samples: 9297056. Policy #0 lag: (min: 2.0, avg: 2.1, max: 8.0) [2023-10-07 20:24:37,477][66916] Avg episode reward: [(0, '33.360'), (1, '34.440')] [2023-10-07 20:24:37,763][67871] Updated weights for policy 1, policy_version 18150 (0.0010) [2023-10-07 20:24:38,138][67871] Updated weights for policy 1, policy_version 18160 (0.0007) [2023-10-07 20:24:38,191][67838] Updated weights for policy 0, policy_version 18122 (0.0009) [2023-10-07 20:24:38,503][67871] Updated weights for policy 1, policy_version 18170 (0.0007) [2023-10-07 20:24:38,561][67838] Updated weights for policy 0, policy_version 18132 (0.0009) [2023-10-07 20:24:38,920][67838] Updated weights for policy 0, policy_version 18142 (0.0010) [2023-10-07 20:24:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37191680. Throughput: 0: 1648.0, 1: 1650.2. Samples: 9306054. Policy #0 lag: (min: 2.0, avg: 2.1, max: 8.0) [2023-10-07 20:24:42,477][66916] Avg episode reward: [(0, '32.920'), (1, '35.640')] [2023-10-07 20:24:42,608][67871] Updated weights for policy 1, policy_version 18180 (0.0007) [2023-10-07 20:24:42,978][67871] Updated weights for policy 1, policy_version 18190 (0.0007) [2023-10-07 20:24:43,091][67838] Updated weights for policy 0, policy_version 18152 (0.0008) [2023-10-07 20:24:43,352][67871] Updated weights for policy 1, policy_version 18200 (0.0008) [2023-10-07 20:24:43,457][67838] Updated weights for policy 0, policy_version 18162 (0.0008) [2023-10-07 20:24:43,832][67838] Updated weights for policy 0, policy_version 18172 (0.0008) [2023-10-07 20:24:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37257216. Throughput: 0: 1647.8, 1: 1648.3. Samples: 9326110. Policy #0 lag: (min: 2.0, avg: 2.4, max: 16.0) [2023-10-07 20:24:47,477][66916] Avg episode reward: [(0, '35.630'), (1, '34.010')] [2023-10-07 20:24:47,532][67871] Updated weights for policy 1, policy_version 18210 (0.0010) [2023-10-07 20:24:47,885][67871] Updated weights for policy 1, policy_version 18220 (0.0010) [2023-10-07 20:24:47,996][67838] Updated weights for policy 0, policy_version 18182 (0.0009) [2023-10-07 20:24:48,254][67871] Updated weights for policy 1, policy_version 18230 (0.0009) [2023-10-07 20:24:48,363][67838] Updated weights for policy 0, policy_version 18192 (0.0011) [2023-10-07 20:24:48,623][67871] Updated weights for policy 1, policy_version 18240 (0.0009) [2023-10-07 20:24:48,741][67838] Updated weights for policy 0, policy_version 18202 (0.0009) [2023-10-07 20:24:52,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37322752. Throughput: 0: 1644.8, 1: 1646.5. Samples: 9346430. Policy #0 lag: (min: 2.0, avg: 2.4, max: 16.0) [2023-10-07 20:24:52,478][66916] Avg episode reward: [(0, '32.560'), (1, '35.100')] [2023-10-07 20:24:52,742][67871] Updated weights for policy 1, policy_version 18250 (0.0008) [2023-10-07 20:24:52,926][67838] Updated weights for policy 0, policy_version 18212 (0.0009) [2023-10-07 20:24:53,109][67871] Updated weights for policy 1, policy_version 18260 (0.0007) [2023-10-07 20:24:53,300][67838] Updated weights for policy 0, policy_version 18222 (0.0007) [2023-10-07 20:24:53,476][67871] Updated weights for policy 1, policy_version 18270 (0.0007) [2023-10-07 20:24:53,675][67838] Updated weights for policy 0, policy_version 18232 (0.0008) [2023-10-07 20:24:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37388288. Throughput: 0: 1644.1, 1: 1645.6. Samples: 9355248. Policy #0 lag: (min: 2.0, avg: 2.4, max: 16.0) [2023-10-07 20:24:57,477][66916] Avg episode reward: [(0, '32.570'), (1, '35.120')] [2023-10-07 20:24:57,816][67871] Updated weights for policy 1, policy_version 18280 (0.0007) [2023-10-07 20:24:57,848][67838] Updated weights for policy 0, policy_version 18242 (0.0009) [2023-10-07 20:24:58,189][67871] Updated weights for policy 1, policy_version 18290 (0.0008) [2023-10-07 20:24:58,217][67838] Updated weights for policy 0, policy_version 18252 (0.0007) [2023-10-07 20:24:58,563][67871] Updated weights for policy 1, policy_version 18300 (0.0008) [2023-10-07 20:24:58,581][67838] Updated weights for policy 0, policy_version 18262 (0.0008) [2023-10-07 20:24:58,957][67838] Updated weights for policy 0, policy_version 18272 (0.0010) [2023-10-07 20:25:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37453824. Throughput: 0: 1648.8, 1: 1649.2. Samples: 9375680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:25:02,478][66916] Avg episode reward: [(0, '33.450'), (1, '34.090')] [2023-10-07 20:25:02,580][67871] Updated weights for policy 1, policy_version 18310 (0.0008) [2023-10-07 20:25:02,951][67871] Updated weights for policy 1, policy_version 18320 (0.0009) [2023-10-07 20:25:03,038][67838] Updated weights for policy 0, policy_version 18282 (0.0008) [2023-10-07 20:25:03,325][67871] Updated weights for policy 1, policy_version 18330 (0.0007) [2023-10-07 20:25:03,407][67838] Updated weights for policy 0, policy_version 18292 (0.0007) [2023-10-07 20:25:03,782][67838] Updated weights for policy 0, policy_version 18302 (0.0009) [2023-10-07 20:25:07,413][67871] Updated weights for policy 1, policy_version 18340 (0.0007) [2023-10-07 20:25:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37519360. Throughput: 0: 1646.5, 1: 1648.2. Samples: 9395858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:25:07,477][66916] Avg episode reward: [(0, '34.180'), (1, '36.020')] [2023-10-07 20:25:07,487][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000018304_18743296.pth... [2023-10-07 20:25:07,527][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000016768_17170432.pth [2023-10-07 20:25:07,784][67871] Updated weights for policy 1, policy_version 18350 (0.0009) [2023-10-07 20:25:08,091][67838] Updated weights for policy 0, policy_version 18312 (0.0008) [2023-10-07 20:25:08,152][67871] Updated weights for policy 1, policy_version 18360 (0.0008) [2023-10-07 20:25:08,449][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000018368_18808832.pth... [2023-10-07 20:25:08,465][67838] Updated weights for policy 0, policy_version 18322 (0.0007) [2023-10-07 20:25:08,484][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000016800_17203200.pth [2023-10-07 20:25:08,840][67838] Updated weights for policy 0, policy_version 18332 (0.0008) [2023-10-07 20:25:12,340][67871] Updated weights for policy 1, policy_version 18370 (0.0010) [2023-10-07 20:25:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 37584896. Throughput: 0: 1648.8, 1: 1648.2. Samples: 9404700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:25:12,477][66916] Avg episode reward: [(0, '34.060'), (1, '35.090')] [2023-10-07 20:25:12,713][67871] Updated weights for policy 1, policy_version 18380 (0.0011) [2023-10-07 20:25:12,980][67838] Updated weights for policy 0, policy_version 18342 (0.0008) [2023-10-07 20:25:13,075][67871] Updated weights for policy 1, policy_version 18390 (0.0008) [2023-10-07 20:25:13,348][67838] Updated weights for policy 0, policy_version 18352 (0.0007) [2023-10-07 20:25:13,445][67871] Updated weights for policy 1, policy_version 18400 (0.0007) [2023-10-07 20:25:13,721][67838] Updated weights for policy 0, policy_version 18362 (0.0008) [2023-10-07 20:25:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37650432. Throughput: 0: 1648.7, 1: 1642.4. Samples: 9424748. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-07 20:25:17,477][66916] Avg episode reward: [(0, '32.130'), (1, '36.690')] [2023-10-07 20:25:17,627][67871] Updated weights for policy 1, policy_version 18410 (0.0009) [2023-10-07 20:25:17,867][67838] Updated weights for policy 0, policy_version 18372 (0.0008) [2023-10-07 20:25:17,992][67871] Updated weights for policy 1, policy_version 18420 (0.0008) [2023-10-07 20:25:18,238][67838] Updated weights for policy 0, policy_version 18382 (0.0008) [2023-10-07 20:25:18,359][67871] Updated weights for policy 1, policy_version 18430 (0.0009) [2023-10-07 20:25:18,426][67676] Saving new best policy, reward=36.690! [2023-10-07 20:25:18,613][67838] Updated weights for policy 0, policy_version 18392 (0.0007) [2023-10-07 20:25:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37715968. Throughput: 0: 1638.0, 1: 1647.4. Samples: 9444900. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-07 20:25:22,477][66916] Avg episode reward: [(0, '32.140'), (1, '34.150')] [2023-10-07 20:25:22,562][67871] Updated weights for policy 1, policy_version 18440 (0.0008) [2023-10-07 20:25:22,902][67838] Updated weights for policy 0, policy_version 18402 (0.0007) [2023-10-07 20:25:22,932][67871] Updated weights for policy 1, policy_version 18450 (0.0008) [2023-10-07 20:25:23,266][67838] Updated weights for policy 0, policy_version 18412 (0.0008) [2023-10-07 20:25:23,298][67871] Updated weights for policy 1, policy_version 18460 (0.0007) [2023-10-07 20:25:23,644][67838] Updated weights for policy 0, policy_version 18422 (0.0009) [2023-10-07 20:25:24,010][67838] Updated weights for policy 0, policy_version 18432 (0.0008) [2023-10-07 20:25:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37781504. Throughput: 0: 1639.3, 1: 1645.9. Samples: 9453888. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-07 20:25:27,478][66916] Avg episode reward: [(0, '33.750'), (1, '34.260')] [2023-10-07 20:25:27,506][67871] Updated weights for policy 1, policy_version 18470 (0.0009) [2023-10-07 20:25:27,872][67871] Updated weights for policy 1, policy_version 18480 (0.0007) [2023-10-07 20:25:28,217][67838] Updated weights for policy 0, policy_version 18442 (0.0007) [2023-10-07 20:25:28,240][67871] Updated weights for policy 1, policy_version 18490 (0.0010) [2023-10-07 20:25:28,591][67838] Updated weights for policy 0, policy_version 18452 (0.0009) [2023-10-07 20:25:28,968][67838] Updated weights for policy 0, policy_version 18462 (0.0009) [2023-10-07 20:25:32,278][67871] Updated weights for policy 1, policy_version 18500 (0.0010) [2023-10-07 20:25:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37847040. Throughput: 0: 1643.0, 1: 1651.5. Samples: 9474360. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 20:25:32,477][66916] Avg episode reward: [(0, '34.210'), (1, '35.190')] [2023-10-07 20:25:32,640][67871] Updated weights for policy 1, policy_version 18510 (0.0008) [2023-10-07 20:25:32,984][67838] Updated weights for policy 0, policy_version 18472 (0.0009) [2023-10-07 20:25:33,022][67871] Updated weights for policy 1, policy_version 18520 (0.0008) [2023-10-07 20:25:33,360][67838] Updated weights for policy 0, policy_version 18482 (0.0008) [2023-10-07 20:25:33,732][67838] Updated weights for policy 0, policy_version 18492 (0.0008) [2023-10-07 20:25:37,090][67871] Updated weights for policy 1, policy_version 18530 (0.0008) [2023-10-07 20:25:37,462][67871] Updated weights for policy 1, policy_version 18540 (0.0010) [2023-10-07 20:25:37,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37912576. Throughput: 0: 1649.4, 1: 1653.7. Samples: 9495066. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 20:25:37,477][66916] Avg episode reward: [(0, '35.510'), (1, '34.860')] [2023-10-07 20:25:37,675][67838] Updated weights for policy 0, policy_version 18502 (0.0010) [2023-10-07 20:25:37,824][67871] Updated weights for policy 1, policy_version 18550 (0.0009) [2023-10-07 20:25:38,048][67838] Updated weights for policy 0, policy_version 18512 (0.0008) [2023-10-07 20:25:38,191][67871] Updated weights for policy 1, policy_version 18560 (0.0010) [2023-10-07 20:25:38,416][67838] Updated weights for policy 0, policy_version 18522 (0.0007) [2023-10-07 20:25:42,441][67871] Updated weights for policy 1, policy_version 18570 (0.0007) [2023-10-07 20:25:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37978112. Throughput: 0: 1654.0, 1: 1651.2. Samples: 9503980. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 20:25:42,477][66916] Avg episode reward: [(0, '32.770'), (1, '35.930')] [2023-10-07 20:25:42,671][67838] Updated weights for policy 0, policy_version 18532 (0.0008) [2023-10-07 20:25:42,806][67871] Updated weights for policy 1, policy_version 18580 (0.0008) [2023-10-07 20:25:43,051][67838] Updated weights for policy 0, policy_version 18542 (0.0007) [2023-10-07 20:25:43,177][67871] Updated weights for policy 1, policy_version 18590 (0.0009) [2023-10-07 20:25:43,427][67838] Updated weights for policy 0, policy_version 18552 (0.0007) [2023-10-07 20:25:47,269][67871] Updated weights for policy 1, policy_version 18600 (0.0008) [2023-10-07 20:25:47,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38043648. Throughput: 0: 1649.6, 1: 1649.7. Samples: 9524152. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 20:25:47,478][66916] Avg episode reward: [(0, '33.580'), (1, '34.260')] [2023-10-07 20:25:47,519][67838] Updated weights for policy 0, policy_version 18562 (0.0008) [2023-10-07 20:25:47,637][67871] Updated weights for policy 1, policy_version 18610 (0.0008) [2023-10-07 20:25:47,894][67838] Updated weights for policy 0, policy_version 18572 (0.0008) [2023-10-07 20:25:47,994][67871] Updated weights for policy 1, policy_version 18620 (0.0007) [2023-10-07 20:25:48,265][67838] Updated weights for policy 0, policy_version 18582 (0.0009) [2023-10-07 20:25:48,637][67838] Updated weights for policy 0, policy_version 18592 (0.0009) [2023-10-07 20:25:52,283][67871] Updated weights for policy 1, policy_version 18630 (0.0009) [2023-10-07 20:25:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38109184. Throughput: 0: 1650.2, 1: 1648.7. Samples: 9544308. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 20:25:52,477][66916] Avg episode reward: [(0, '33.900'), (1, '34.260')] [2023-10-07 20:25:52,649][67871] Updated weights for policy 1, policy_version 18640 (0.0008) [2023-10-07 20:25:52,826][67838] Updated weights for policy 0, policy_version 18602 (0.0008) [2023-10-07 20:25:53,025][67871] Updated weights for policy 1, policy_version 18650 (0.0008) [2023-10-07 20:25:53,201][67838] Updated weights for policy 0, policy_version 18612 (0.0010) [2023-10-07 20:25:53,574][67838] Updated weights for policy 0, policy_version 18622 (0.0007) [2023-10-07 20:25:57,077][67871] Updated weights for policy 1, policy_version 18660 (0.0009) [2023-10-07 20:25:57,455][67871] Updated weights for policy 1, policy_version 18670 (0.0008) [2023-10-07 20:25:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38174720. Throughput: 0: 1652.4, 1: 1648.4. Samples: 9553240. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 20:25:57,478][66916] Avg episode reward: [(0, '34.560'), (1, '34.300')] [2023-10-07 20:25:57,759][67838] Updated weights for policy 0, policy_version 18632 (0.0007) [2023-10-07 20:25:57,823][67871] Updated weights for policy 1, policy_version 18680 (0.0009) [2023-10-07 20:25:58,136][67838] Updated weights for policy 0, policy_version 18642 (0.0007) [2023-10-07 20:25:58,512][67838] Updated weights for policy 0, policy_version 18652 (0.0007) [2023-10-07 20:26:01,948][67871] Updated weights for policy 1, policy_version 18690 (0.0008) [2023-10-07 20:26:02,310][67871] Updated weights for policy 1, policy_version 18700 (0.0009) [2023-10-07 20:26:02,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38240256. Throughput: 0: 1652.7, 1: 1652.8. Samples: 9573492. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-07 20:26:02,477][66916] Avg episode reward: [(0, '34.870'), (1, '35.430')] [2023-10-07 20:26:02,674][67871] Updated weights for policy 1, policy_version 18710 (0.0008) [2023-10-07 20:26:02,800][67838] Updated weights for policy 0, policy_version 18662 (0.0008) [2023-10-07 20:26:03,058][67871] Updated weights for policy 1, policy_version 18720 (0.0008) [2023-10-07 20:26:03,189][67838] Updated weights for policy 0, policy_version 18672 (0.0009) [2023-10-07 20:26:03,568][67838] Updated weights for policy 0, policy_version 18682 (0.0008) [2023-10-07 20:26:07,351][67871] Updated weights for policy 1, policy_version 18730 (0.0007) [2023-10-07 20:26:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38305792. Throughput: 0: 1657.8, 1: 1650.0. Samples: 9593750. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-07 20:26:07,477][66916] Avg episode reward: [(0, '34.540'), (1, '34.840')] [2023-10-07 20:26:07,621][67838] Updated weights for policy 0, policy_version 18692 (0.0009) [2023-10-07 20:26:07,709][67871] Updated weights for policy 1, policy_version 18740 (0.0008) [2023-10-07 20:26:07,986][67838] Updated weights for policy 0, policy_version 18702 (0.0009) [2023-10-07 20:26:08,079][67871] Updated weights for policy 1, policy_version 18750 (0.0007) [2023-10-07 20:26:08,362][67838] Updated weights for policy 0, policy_version 18712 (0.0009) [2023-10-07 20:26:12,149][67871] Updated weights for policy 1, policy_version 18760 (0.0008) [2023-10-07 20:26:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38371328. Throughput: 0: 1652.8, 1: 1655.1. Samples: 9602744. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-07 20:26:12,477][66916] Avg episode reward: [(0, '34.940'), (1, '35.700')] [2023-10-07 20:26:12,519][67871] Updated weights for policy 1, policy_version 18770 (0.0010) [2023-10-07 20:26:12,597][67838] Updated weights for policy 0, policy_version 18722 (0.0010) [2023-10-07 20:26:12,897][67871] Updated weights for policy 1, policy_version 18780 (0.0007) [2023-10-07 20:26:12,965][67838] Updated weights for policy 0, policy_version 18732 (0.0011) [2023-10-07 20:26:13,339][67838] Updated weights for policy 0, policy_version 18742 (0.0008) [2023-10-07 20:26:13,714][67838] Updated weights for policy 0, policy_version 18752 (0.0007) [2023-10-07 20:26:16,914][67871] Updated weights for policy 1, policy_version 18790 (0.0007) [2023-10-07 20:26:17,275][67871] Updated weights for policy 1, policy_version 18800 (0.0007) [2023-10-07 20:26:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38436864. Throughput: 0: 1648.7, 1: 1656.8. Samples: 9623106. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-07 20:26:17,477][66916] Avg episode reward: [(0, '33.120'), (1, '36.350')] [2023-10-07 20:26:17,644][67871] Updated weights for policy 1, policy_version 18810 (0.0007) [2023-10-07 20:26:17,817][67838] Updated weights for policy 0, policy_version 18762 (0.0007) [2023-10-07 20:26:18,186][67838] Updated weights for policy 0, policy_version 18772 (0.0008) [2023-10-07 20:26:18,564][67838] Updated weights for policy 0, policy_version 18782 (0.0007) [2023-10-07 20:26:21,742][67871] Updated weights for policy 1, policy_version 18820 (0.0008) [2023-10-07 20:26:22,097][67871] Updated weights for policy 1, policy_version 18830 (0.0010) [2023-10-07 20:26:22,464][67871] Updated weights for policy 1, policy_version 18840 (0.0007) [2023-10-07 20:26:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38502400. Throughput: 0: 1647.9, 1: 1646.2. Samples: 9643302. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-07 20:26:22,477][66916] Avg episode reward: [(0, '33.420'), (1, '35.460')] [2023-10-07 20:26:22,550][67838] Updated weights for policy 0, policy_version 18792 (0.0008) [2023-10-07 20:26:22,922][67838] Updated weights for policy 0, policy_version 18802 (0.0009) [2023-10-07 20:26:23,297][67838] Updated weights for policy 0, policy_version 18812 (0.0007) [2023-10-07 20:26:26,521][67871] Updated weights for policy 1, policy_version 18850 (0.0007) [2023-10-07 20:26:26,898][67871] Updated weights for policy 1, policy_version 18860 (0.0008) [2023-10-07 20:26:27,265][67871] Updated weights for policy 1, policy_version 18870 (0.0009) [2023-10-07 20:26:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38567936. Throughput: 0: 1647.0, 1: 1655.6. Samples: 9652598. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-07 20:26:27,477][66916] Avg episode reward: [(0, '33.900'), (1, '35.360')] [2023-10-07 20:26:27,477][67838] Updated weights for policy 0, policy_version 18822 (0.0008) [2023-10-07 20:26:27,634][67871] Updated weights for policy 1, policy_version 18880 (0.0007) [2023-10-07 20:26:27,858][67838] Updated weights for policy 0, policy_version 18832 (0.0008) [2023-10-07 20:26:28,237][67838] Updated weights for policy 0, policy_version 18842 (0.0007) [2023-10-07 20:26:31,745][67871] Updated weights for policy 1, policy_version 18890 (0.0007) [2023-10-07 20:26:32,121][67871] Updated weights for policy 1, policy_version 18900 (0.0009) [2023-10-07 20:26:32,266][67838] Updated weights for policy 0, policy_version 18852 (0.0007) [2023-10-07 20:26:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38633472. Throughput: 0: 1647.7, 1: 1662.8. Samples: 9673126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:26:32,477][66916] Avg episode reward: [(0, '33.000'), (1, '34.720')] [2023-10-07 20:26:32,483][67871] Updated weights for policy 1, policy_version 18910 (0.0009) [2023-10-07 20:26:32,641][67838] Updated weights for policy 0, policy_version 18862 (0.0009) [2023-10-07 20:26:33,029][67838] Updated weights for policy 0, policy_version 18872 (0.0009) [2023-10-07 20:26:36,672][67871] Updated weights for policy 1, policy_version 18920 (0.0007) [2023-10-07 20:26:37,034][67838] Updated weights for policy 0, policy_version 18882 (0.0010) [2023-10-07 20:26:37,041][67871] Updated weights for policy 1, policy_version 18930 (0.0007) [2023-10-07 20:26:37,409][67871] Updated weights for policy 1, policy_version 18940 (0.0008) [2023-10-07 20:26:37,415][67838] Updated weights for policy 0, policy_version 18892 (0.0007) [2023-10-07 20:26:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38699008. Throughput: 0: 1651.3, 1: 1655.5. Samples: 9693112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:26:37,477][66916] Avg episode reward: [(0, '34.320'), (1, '33.680')] [2023-10-07 20:26:37,791][67838] Updated weights for policy 0, policy_version 18902 (0.0007) [2023-10-07 20:26:38,167][67838] Updated weights for policy 0, policy_version 18912 (0.0008) [2023-10-07 20:26:41,564][67871] Updated weights for policy 1, policy_version 18950 (0.0008) [2023-10-07 20:26:41,928][67871] Updated weights for policy 1, policy_version 18960 (0.0007) [2023-10-07 20:26:42,300][67871] Updated weights for policy 1, policy_version 18970 (0.0008) [2023-10-07 20:26:42,327][67838] Updated weights for policy 0, policy_version 18922 (0.0009) [2023-10-07 20:26:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38764544. Throughput: 0: 1650.3, 1: 1664.5. Samples: 9702408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:26:42,477][66916] Avg episode reward: [(0, '34.240'), (1, '35.130')] [2023-10-07 20:26:42,707][67838] Updated weights for policy 0, policy_version 18932 (0.0008) [2023-10-07 20:26:43,091][67838] Updated weights for policy 0, policy_version 18942 (0.0010) [2023-10-07 20:26:46,578][67871] Updated weights for policy 1, policy_version 18980 (0.0007) [2023-10-07 20:26:46,943][67871] Updated weights for policy 1, policy_version 18990 (0.0008) [2023-10-07 20:26:47,309][67871] Updated weights for policy 1, policy_version 19000 (0.0008) [2023-10-07 20:26:47,323][67838] Updated weights for policy 0, policy_version 18952 (0.0010) [2023-10-07 20:26:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38830080. Throughput: 0: 1647.3, 1: 1665.7. Samples: 9722578. Policy #0 lag: (min: 19.0, avg: 31.4, max: 51.0) [2023-10-07 20:26:47,477][66916] Avg episode reward: [(0, '32.950'), (1, '34.590')] [2023-10-07 20:26:47,693][67838] Updated weights for policy 0, policy_version 18962 (0.0008) [2023-10-07 20:26:48,064][67838] Updated weights for policy 0, policy_version 18972 (0.0010) [2023-10-07 20:26:51,656][67871] Updated weights for policy 1, policy_version 19010 (0.0009) [2023-10-07 20:26:52,069][67871] Updated weights for policy 1, policy_version 19020 (0.0009) [2023-10-07 20:26:52,117][67838] Updated weights for policy 0, policy_version 18982 (0.0007) [2023-10-07 20:26:52,432][67871] Updated weights for policy 1, policy_version 19030 (0.0008) [2023-10-07 20:26:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38895616. Throughput: 0: 1650.1, 1: 1658.0. Samples: 9742618. Policy #0 lag: (min: 19.0, avg: 31.4, max: 51.0) [2023-10-07 20:26:52,477][66916] Avg episode reward: [(0, '33.090'), (1, '34.210')] [2023-10-07 20:26:52,493][67838] Updated weights for policy 0, policy_version 18992 (0.0009) [2023-10-07 20:26:52,796][67871] Updated weights for policy 1, policy_version 19040 (0.0007) [2023-10-07 20:26:52,850][67838] Updated weights for policy 0, policy_version 19002 (0.0010) [2023-10-07 20:26:56,809][67871] Updated weights for policy 1, policy_version 19050 (0.0008) [2023-10-07 20:26:57,162][67838] Updated weights for policy 0, policy_version 19012 (0.0008) [2023-10-07 20:26:57,183][67871] Updated weights for policy 1, policy_version 19060 (0.0008) [2023-10-07 20:26:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38961152. Throughput: 0: 1654.5, 1: 1661.1. Samples: 9751950. Policy #0 lag: (min: 19.0, avg: 31.4, max: 51.0) [2023-10-07 20:26:57,478][66916] Avg episode reward: [(0, '34.310'), (1, '35.240')] [2023-10-07 20:26:57,535][67838] Updated weights for policy 0, policy_version 19022 (0.0010) [2023-10-07 20:26:57,553][67871] Updated weights for policy 1, policy_version 19070 (0.0009) [2023-10-07 20:26:57,915][67838] Updated weights for policy 0, policy_version 19032 (0.0010) [2023-10-07 20:27:01,768][67871] Updated weights for policy 1, policy_version 19080 (0.0009) [2023-10-07 20:27:01,915][67838] Updated weights for policy 0, policy_version 19042 (0.0007) [2023-10-07 20:27:02,126][67871] Updated weights for policy 1, policy_version 19090 (0.0009) [2023-10-07 20:27:02,283][67838] Updated weights for policy 0, policy_version 19052 (0.0007) [2023-10-07 20:27:02,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39026688. Throughput: 0: 1656.7, 1: 1656.3. Samples: 9772190. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-07 20:27:02,478][66916] Avg episode reward: [(0, '34.040'), (1, '35.100')] [2023-10-07 20:27:02,495][67871] Updated weights for policy 1, policy_version 19100 (0.0008) [2023-10-07 20:27:02,658][67838] Updated weights for policy 0, policy_version 19062 (0.0007) [2023-10-07 20:27:03,028][67838] Updated weights for policy 0, policy_version 19072 (0.0009) [2023-10-07 20:27:06,717][67871] Updated weights for policy 1, policy_version 19110 (0.0009) [2023-10-07 20:27:07,085][67871] Updated weights for policy 1, policy_version 19120 (0.0007) [2023-10-07 20:27:07,449][67838] Updated weights for policy 0, policy_version 19082 (0.0007) [2023-10-07 20:27:07,458][67871] Updated weights for policy 1, policy_version 19130 (0.0007) [2023-10-07 20:27:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 39092224. Throughput: 0: 1653.1, 1: 1654.4. Samples: 9792140. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-07 20:27:07,478][66916] Avg episode reward: [(0, '34.010'), (1, '35.930')] [2023-10-07 20:27:07,673][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000019136_19595264.pth... [2023-10-07 20:27:07,702][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000017568_17989632.pth [2023-10-07 20:27:07,819][67838] Updated weights for policy 0, policy_version 19092 (0.0007) [2023-10-07 20:27:08,190][67838] Updated weights for policy 0, policy_version 19102 (0.0009) [2023-10-07 20:27:08,260][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000019104_19562496.pth... [2023-10-07 20:27:08,297][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000017536_17956864.pth [2023-10-07 20:27:11,608][67871] Updated weights for policy 1, policy_version 19140 (0.0008) [2023-10-07 20:27:11,970][67871] Updated weights for policy 1, policy_version 19150 (0.0010) [2023-10-07 20:27:12,328][67838] Updated weights for policy 0, policy_version 19112 (0.0008) [2023-10-07 20:27:12,341][67871] Updated weights for policy 1, policy_version 19160 (0.0009) [2023-10-07 20:27:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39157760. Throughput: 0: 1653.6, 1: 1657.2. Samples: 9801582. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-07 20:27:12,477][66916] Avg episode reward: [(0, '35.330'), (1, '35.010')] [2023-10-07 20:27:12,705][67838] Updated weights for policy 0, policy_version 19122 (0.0008) [2023-10-07 20:27:13,072][67838] Updated weights for policy 0, policy_version 19132 (0.0008) [2023-10-07 20:27:16,379][67871] Updated weights for policy 1, policy_version 19170 (0.0007) [2023-10-07 20:27:16,743][67871] Updated weights for policy 1, policy_version 19180 (0.0010) [2023-10-07 20:27:17,118][67871] Updated weights for policy 1, policy_version 19190 (0.0009) [2023-10-07 20:27:17,213][67838] Updated weights for policy 0, policy_version 19142 (0.0008) [2023-10-07 20:27:17,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39223296. Throughput: 0: 1654.8, 1: 1653.9. Samples: 9822018. Policy #0 lag: (min: 18.0, avg: 33.5, max: 50.0) [2023-10-07 20:27:17,477][66916] Avg episode reward: [(0, '32.820'), (1, '34.400')] [2023-10-07 20:27:17,495][67871] Updated weights for policy 1, policy_version 19200 (0.0009) [2023-10-07 20:27:17,595][67838] Updated weights for policy 0, policy_version 19152 (0.0008) [2023-10-07 20:27:17,966][67838] Updated weights for policy 0, policy_version 19162 (0.0008) [2023-10-07 20:27:21,548][67871] Updated weights for policy 1, policy_version 19210 (0.0007) [2023-10-07 20:27:21,918][67871] Updated weights for policy 1, policy_version 19220 (0.0007) [2023-10-07 20:27:22,195][67838] Updated weights for policy 0, policy_version 19172 (0.0008) [2023-10-07 20:27:22,277][67871] Updated weights for policy 1, policy_version 19230 (0.0007) [2023-10-07 20:27:22,476][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 39321600. Throughput: 0: 1654.2, 1: 1647.5. Samples: 9841686. Policy #0 lag: (min: 18.0, avg: 33.5, max: 50.0) [2023-10-07 20:27:22,477][66916] Avg episode reward: [(0, '35.850'), (1, '34.620')] [2023-10-07 20:27:22,565][67838] Updated weights for policy 0, policy_version 19182 (0.0009) [2023-10-07 20:27:22,938][67838] Updated weights for policy 0, policy_version 19192 (0.0009) [2023-10-07 20:27:26,501][67871] Updated weights for policy 1, policy_version 19240 (0.0009) [2023-10-07 20:27:26,866][67871] Updated weights for policy 1, policy_version 19250 (0.0008) [2023-10-07 20:27:27,124][67838] Updated weights for policy 0, policy_version 19202 (0.0010) [2023-10-07 20:27:27,233][67871] Updated weights for policy 1, policy_version 19260 (0.0008) [2023-10-07 20:27:27,477][66916] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 39387136. Throughput: 0: 1654.9, 1: 1654.5. Samples: 9851334. Policy #0 lag: (min: 18.0, avg: 33.5, max: 50.0) [2023-10-07 20:27:27,478][66916] Avg episode reward: [(0, '33.770'), (1, '34.770')] [2023-10-07 20:27:27,504][67838] Updated weights for policy 0, policy_version 19212 (0.0008) [2023-10-07 20:27:27,883][67838] Updated weights for policy 0, policy_version 19222 (0.0007) [2023-10-07 20:27:28,252][67838] Updated weights for policy 0, policy_version 19232 (0.0008) [2023-10-07 20:27:31,322][67871] Updated weights for policy 1, policy_version 19270 (0.0010) [2023-10-07 20:27:31,697][67871] Updated weights for policy 1, policy_version 19280 (0.0007) [2023-10-07 20:27:32,064][67871] Updated weights for policy 1, policy_version 19290 (0.0008) [2023-10-07 20:27:32,353][67838] Updated weights for policy 0, policy_version 19242 (0.0007) [2023-10-07 20:27:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 39452672. Throughput: 0: 1657.9, 1: 1650.5. Samples: 9871456. Policy #0 lag: (min: 26.0, avg: 30.6, max: 58.0) [2023-10-07 20:27:32,477][66916] Avg episode reward: [(0, '34.670'), (1, '33.890')] [2023-10-07 20:27:32,734][67838] Updated weights for policy 0, policy_version 19252 (0.0008) [2023-10-07 20:27:33,113][67838] Updated weights for policy 0, policy_version 19262 (0.0010) [2023-10-07 20:27:36,316][67871] Updated weights for policy 1, policy_version 19300 (0.0008) [2023-10-07 20:27:36,684][67871] Updated weights for policy 1, policy_version 19310 (0.0009) [2023-10-07 20:27:37,046][67871] Updated weights for policy 1, policy_version 19320 (0.0010) [2023-10-07 20:27:37,229][67838] Updated weights for policy 0, policy_version 19272 (0.0007) [2023-10-07 20:27:37,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 39518208. Throughput: 0: 1651.7, 1: 1641.5. Samples: 9890810. Policy #0 lag: (min: 26.0, avg: 30.6, max: 58.0) [2023-10-07 20:27:37,477][66916] Avg episode reward: [(0, '34.910'), (1, '34.490')] [2023-10-07 20:27:37,601][67838] Updated weights for policy 0, policy_version 19282 (0.0008) [2023-10-07 20:27:37,979][67838] Updated weights for policy 0, policy_version 19292 (0.0011) [2023-10-07 20:27:41,300][67871] Updated weights for policy 1, policy_version 19330 (0.0008) [2023-10-07 20:27:41,705][67871] Updated weights for policy 1, policy_version 19340 (0.0009) [2023-10-07 20:27:42,073][67871] Updated weights for policy 1, policy_version 19350 (0.0007) [2023-10-07 20:27:42,189][67838] Updated weights for policy 0, policy_version 19302 (0.0009) [2023-10-07 20:27:42,436][67871] Updated weights for policy 1, policy_version 19360 (0.0008) [2023-10-07 20:27:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 39583744. Throughput: 0: 1651.7, 1: 1651.3. Samples: 9900586. Policy #0 lag: (min: 26.0, avg: 30.6, max: 58.0) [2023-10-07 20:27:42,477][66916] Avg episode reward: [(0, '31.920'), (1, '33.500')] [2023-10-07 20:27:42,555][67838] Updated weights for policy 0, policy_version 19312 (0.0010) [2023-10-07 20:27:42,931][67838] Updated weights for policy 0, policy_version 19322 (0.0010) [2023-10-07 20:27:46,662][67871] Updated weights for policy 1, policy_version 19370 (0.0007) [2023-10-07 20:27:47,034][67871] Updated weights for policy 1, policy_version 19380 (0.0008) [2023-10-07 20:27:47,096][67838] Updated weights for policy 0, policy_version 19332 (0.0007) [2023-10-07 20:27:47,394][67871] Updated weights for policy 1, policy_version 19390 (0.0009) [2023-10-07 20:27:47,464][67838] Updated weights for policy 0, policy_version 19342 (0.0009) [2023-10-07 20:27:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 39649280. Throughput: 0: 1651.3, 1: 1651.6. Samples: 9920820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:27:47,477][66916] Avg episode reward: [(0, '33.830'), (1, '34.280')] [2023-10-07 20:27:47,838][67838] Updated weights for policy 0, policy_version 19352 (0.0008) [2023-10-07 20:27:51,661][67871] Updated weights for policy 1, policy_version 19400 (0.0008) [2023-10-07 20:27:51,878][67838] Updated weights for policy 0, policy_version 19362 (0.0009) [2023-10-07 20:27:52,033][67871] Updated weights for policy 1, policy_version 19410 (0.0007) [2023-10-07 20:27:52,241][67838] Updated weights for policy 0, policy_version 19372 (0.0007) [2023-10-07 20:27:52,409][67871] Updated weights for policy 1, policy_version 19420 (0.0008) [2023-10-07 20:27:52,477][66916] Fps is (10 sec: 9830.3, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 39682048. Throughput: 0: 1645.9, 1: 1647.6. Samples: 9940348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:27:52,478][66916] Avg episode reward: [(0, '31.770'), (1, '34.780')] [2023-10-07 20:27:52,614][67838] Updated weights for policy 0, policy_version 19382 (0.0008) [2023-10-07 20:27:52,985][67838] Updated weights for policy 0, policy_version 19392 (0.0007) [2023-10-07 20:27:56,373][67871] Updated weights for policy 1, policy_version 19430 (0.0008) [2023-10-07 20:27:56,737][67871] Updated weights for policy 1, policy_version 19440 (0.0007) [2023-10-07 20:27:57,110][67871] Updated weights for policy 1, policy_version 19450 (0.0008) [2023-10-07 20:27:57,173][67838] Updated weights for policy 0, policy_version 19402 (0.0009) [2023-10-07 20:27:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 39780352. Throughput: 0: 1649.9, 1: 1650.9. Samples: 9950116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:27:57,478][66916] Avg episode reward: [(0, '35.210'), (1, '33.380')] [2023-10-07 20:27:57,547][67838] Updated weights for policy 0, policy_version 19412 (0.0007) [2023-10-07 20:27:57,926][67838] Updated weights for policy 0, policy_version 19422 (0.0007) [2023-10-07 20:28:01,157][67871] Updated weights for policy 1, policy_version 19460 (0.0007) [2023-10-07 20:28:01,531][67871] Updated weights for policy 1, policy_version 19470 (0.0008) [2023-10-07 20:28:01,896][67871] Updated weights for policy 1, policy_version 19480 (0.0009) [2023-10-07 20:28:02,051][67838] Updated weights for policy 0, policy_version 19432 (0.0008) [2023-10-07 20:28:02,438][67838] Updated weights for policy 0, policy_version 19442 (0.0008) [2023-10-07 20:28:02,476][66916] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 39845888. Throughput: 0: 1651.9, 1: 1652.6. Samples: 9970720. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-07 20:28:02,477][66916] Avg episode reward: [(0, '34.020'), (1, '35.220')] [2023-10-07 20:28:02,806][67838] Updated weights for policy 0, policy_version 19452 (0.0008) [2023-10-07 20:28:06,186][67871] Updated weights for policy 1, policy_version 19490 (0.0007) [2023-10-07 20:28:06,570][67871] Updated weights for policy 1, policy_version 19500 (0.0009) [2023-10-07 20:28:06,842][67838] Updated weights for policy 0, policy_version 19462 (0.0008) [2023-10-07 20:28:06,931][67871] Updated weights for policy 1, policy_version 19510 (0.0009) [2023-10-07 20:28:07,220][67838] Updated weights for policy 0, policy_version 19472 (0.0007) [2023-10-07 20:28:07,299][67871] Updated weights for policy 1, policy_version 19520 (0.0008) [2023-10-07 20:28:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 39911424. Throughput: 0: 1642.6, 1: 1648.4. Samples: 9989782. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-07 20:28:07,477][66916] Avg episode reward: [(0, '32.510'), (1, '33.430')] [2023-10-07 20:28:07,592][67838] Updated weights for policy 0, policy_version 19482 (0.0008) [2023-10-07 20:28:11,288][67871] Updated weights for policy 1, policy_version 19530 (0.0007) [2023-10-07 20:28:11,651][67871] Updated weights for policy 1, policy_version 19540 (0.0009) [2023-10-07 20:28:11,761][67838] Updated weights for policy 0, policy_version 19492 (0.0007) [2023-10-07 20:28:12,026][67871] Updated weights for policy 1, policy_version 19550 (0.0008) [2023-10-07 20:28:12,141][67838] Updated weights for policy 0, policy_version 19502 (0.0007) [2023-10-07 20:28:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 39976960. Throughput: 0: 1650.5, 1: 1650.4. Samples: 9999870. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-07 20:28:12,477][66916] Avg episode reward: [(0, '34.420'), (1, '34.430')] [2023-10-07 20:28:12,503][67838] Updated weights for policy 0, policy_version 19512 (0.0010) [2023-10-07 20:28:16,229][67871] Updated weights for policy 1, policy_version 19560 (0.0008) [2023-10-07 20:28:16,592][67871] Updated weights for policy 1, policy_version 19570 (0.0008) [2023-10-07 20:28:16,685][67838] Updated weights for policy 0, policy_version 19522 (0.0010) [2023-10-07 20:28:16,963][67871] Updated weights for policy 1, policy_version 19580 (0.0008) [2023-10-07 20:28:17,055][67838] Updated weights for policy 0, policy_version 19532 (0.0008) [2023-10-07 20:28:17,436][67838] Updated weights for policy 0, policy_version 19542 (0.0009) [2023-10-07 20:28:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 40042496. Throughput: 0: 1652.9, 1: 1652.9. Samples: 10020218. Policy #0 lag: (min: 26.0, avg: 32.7, max: 58.0) [2023-10-07 20:28:17,477][66916] Avg episode reward: [(0, '32.830'), (1, '34.870')] [2023-10-07 20:28:17,807][67838] Updated weights for policy 0, policy_version 19552 (0.0007) [2023-10-07 20:28:21,012][67871] Updated weights for policy 1, policy_version 19590 (0.0009) [2023-10-07 20:28:21,385][67871] Updated weights for policy 1, policy_version 19600 (0.0009) [2023-10-07 20:28:21,751][67871] Updated weights for policy 1, policy_version 19610 (0.0008) [2023-10-07 20:28:22,090][67838] Updated weights for policy 0, policy_version 19562 (0.0010) [2023-10-07 20:28:22,457][67838] Updated weights for policy 0, policy_version 19572 (0.0007) [2023-10-07 20:28:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 40108032. Throughput: 0: 1648.0, 1: 1647.9. Samples: 10039124. Policy #0 lag: (min: 26.0, avg: 32.7, max: 58.0) [2023-10-07 20:28:22,478][66916] Avg episode reward: [(0, '33.560'), (1, '34.500')] [2023-10-07 20:28:22,836][67838] Updated weights for policy 0, policy_version 19582 (0.0009) [2023-10-07 20:28:25,855][67871] Updated weights for policy 1, policy_version 19620 (0.0008) [2023-10-07 20:28:26,239][67871] Updated weights for policy 1, policy_version 19630 (0.0008) [2023-10-07 20:28:26,615][67871] Updated weights for policy 1, policy_version 19640 (0.0010) [2023-10-07 20:28:26,947][67838] Updated weights for policy 0, policy_version 19592 (0.0010) [2023-10-07 20:28:27,316][67838] Updated weights for policy 0, policy_version 19602 (0.0007) [2023-10-07 20:28:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 40173568. Throughput: 0: 1650.4, 1: 1655.8. Samples: 10049366. Policy #0 lag: (min: 26.0, avg: 32.7, max: 58.0) [2023-10-07 20:28:27,478][66916] Avg episode reward: [(0, '32.300'), (1, '33.630')] [2023-10-07 20:28:27,696][67838] Updated weights for policy 0, policy_version 19612 (0.0007) [2023-10-07 20:28:30,796][67871] Updated weights for policy 1, policy_version 19650 (0.0009) [2023-10-07 20:28:31,165][67871] Updated weights for policy 1, policy_version 19660 (0.0008) [2023-10-07 20:28:31,541][67871] Updated weights for policy 1, policy_version 19670 (0.0010) [2023-10-07 20:28:31,908][67871] Updated weights for policy 1, policy_version 19680 (0.0009) [2023-10-07 20:28:31,915][67838] Updated weights for policy 0, policy_version 19622 (0.0009) [2023-10-07 20:28:32,301][67838] Updated weights for policy 0, policy_version 19632 (0.0007) [2023-10-07 20:28:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 40239104. Throughput: 0: 1644.6, 1: 1649.3. Samples: 10069044. Policy #0 lag: (min: 44.0, avg: 55.7, max: 56.0) [2023-10-07 20:28:32,478][66916] Avg episode reward: [(0, '34.950'), (1, '33.950')] [2023-10-07 20:28:32,672][67838] Updated weights for policy 0, policy_version 19642 (0.0007) [2023-10-07 20:28:36,009][67871] Updated weights for policy 1, policy_version 19690 (0.0008) [2023-10-07 20:28:36,375][67871] Updated weights for policy 1, policy_version 19700 (0.0008) [2023-10-07 20:28:36,568][67838] Updated weights for policy 0, policy_version 19652 (0.0009) [2023-10-07 20:28:36,741][67871] Updated weights for policy 1, policy_version 19710 (0.0007) [2023-10-07 20:28:36,947][67838] Updated weights for policy 0, policy_version 19662 (0.0008) [2023-10-07 20:28:37,326][67838] Updated weights for policy 0, policy_version 19672 (0.0008) [2023-10-07 20:28:37,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 40304640. Throughput: 0: 1644.3, 1: 1635.7. Samples: 10087948. Policy #0 lag: (min: 44.0, avg: 55.7, max: 56.0) [2023-10-07 20:28:37,478][66916] Avg episode reward: [(0, '33.630'), (1, '35.110')] [2023-10-07 20:28:40,874][67871] Updated weights for policy 1, policy_version 19720 (0.0009) [2023-10-07 20:28:41,248][67871] Updated weights for policy 1, policy_version 19730 (0.0008) [2023-10-07 20:28:41,423][67838] Updated weights for policy 0, policy_version 19682 (0.0007) [2023-10-07 20:28:41,621][67871] Updated weights for policy 1, policy_version 19740 (0.0010) [2023-10-07 20:28:41,807][67838] Updated weights for policy 0, policy_version 19692 (0.0008) [2023-10-07 20:28:42,173][67838] Updated weights for policy 0, policy_version 19702 (0.0009) [2023-10-07 20:28:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 40370176. Throughput: 0: 1654.0, 1: 1652.3. Samples: 10098898. Policy #0 lag: (min: 44.0, avg: 55.7, max: 56.0) [2023-10-07 20:28:42,477][66916] Avg episode reward: [(0, '32.440'), (1, '35.430')] [2023-10-07 20:28:42,549][67838] Updated weights for policy 0, policy_version 19712 (0.0010) [2023-10-07 20:28:45,712][67871] Updated weights for policy 1, policy_version 19750 (0.0008) [2023-10-07 20:28:46,074][67871] Updated weights for policy 1, policy_version 19760 (0.0007) [2023-10-07 20:28:46,439][67871] Updated weights for policy 1, policy_version 19770 (0.0009) [2023-10-07 20:28:46,707][67838] Updated weights for policy 0, policy_version 19722 (0.0007) [2023-10-07 20:28:47,084][67838] Updated weights for policy 0, policy_version 19732 (0.0007) [2023-10-07 20:28:47,450][67838] Updated weights for policy 0, policy_version 19742 (0.0009) [2023-10-07 20:28:47,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 40435712. Throughput: 0: 1651.7, 1: 1638.6. Samples: 10118784. Policy #0 lag: (min: 2.0, avg: 5.4, max: 34.0) [2023-10-07 20:28:47,478][66916] Avg episode reward: [(0, '34.530'), (1, '35.090')] [2023-10-07 20:28:50,588][67871] Updated weights for policy 1, policy_version 19780 (0.0009) [2023-10-07 20:28:50,964][67871] Updated weights for policy 1, policy_version 19790 (0.0009) [2023-10-07 20:28:51,333][67871] Updated weights for policy 1, policy_version 19800 (0.0008) [2023-10-07 20:28:51,589][67838] Updated weights for policy 0, policy_version 19752 (0.0008) [2023-10-07 20:28:51,969][67838] Updated weights for policy 0, policy_version 19762 (0.0009) [2023-10-07 20:28:52,337][67838] Updated weights for policy 0, policy_version 19772 (0.0007) [2023-10-07 20:28:52,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 40501248. Throughput: 0: 1647.5, 1: 1637.3. Samples: 10137600. Policy #0 lag: (min: 2.0, avg: 5.4, max: 34.0) [2023-10-07 20:28:52,478][66916] Avg episode reward: [(0, '34.580'), (1, '36.100')] [2023-10-07 20:28:55,601][67871] Updated weights for policy 1, policy_version 19810 (0.0008) [2023-10-07 20:28:55,975][67871] Updated weights for policy 1, policy_version 19820 (0.0010) [2023-10-07 20:28:56,338][67871] Updated weights for policy 1, policy_version 19830 (0.0008) [2023-10-07 20:28:56,441][67838] Updated weights for policy 0, policy_version 19782 (0.0008) [2023-10-07 20:28:56,705][67871] Updated weights for policy 1, policy_version 19840 (0.0009) [2023-10-07 20:28:56,814][67838] Updated weights for policy 0, policy_version 19792 (0.0010) [2023-10-07 20:28:57,200][67838] Updated weights for policy 0, policy_version 19802 (0.0009) [2023-10-07 20:28:57,476][66916] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 40599552. Throughput: 0: 1654.7, 1: 1649.5. Samples: 10148560. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) [2023-10-07 20:28:57,477][66916] Avg episode reward: [(0, '34.680'), (1, '35.270')] [2023-10-07 20:29:00,743][67871] Updated weights for policy 1, policy_version 19850 (0.0009) [2023-10-07 20:29:01,111][67871] Updated weights for policy 1, policy_version 19860 (0.0009) [2023-10-07 20:29:01,379][67838] Updated weights for policy 0, policy_version 19812 (0.0008) [2023-10-07 20:29:01,482][67871] Updated weights for policy 1, policy_version 19870 (0.0008) [2023-10-07 20:29:01,741][67838] Updated weights for policy 0, policy_version 19822 (0.0008) [2023-10-07 20:29:02,110][67838] Updated weights for policy 0, policy_version 19832 (0.0007) [2023-10-07 20:29:02,476][66916] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 40665088. Throughput: 0: 1651.2, 1: 1642.1. Samples: 10168418. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) [2023-10-07 20:29:02,477][66916] Avg episode reward: [(0, '34.170'), (1, '36.520')] [2023-10-07 20:29:05,682][67871] Updated weights for policy 1, policy_version 19880 (0.0008) [2023-10-07 20:29:06,057][67871] Updated weights for policy 1, policy_version 19890 (0.0007) [2023-10-07 20:29:06,212][67838] Updated weights for policy 0, policy_version 19842 (0.0008) [2023-10-07 20:29:06,415][67871] Updated weights for policy 1, policy_version 19900 (0.0007) [2023-10-07 20:29:06,601][67838] Updated weights for policy 0, policy_version 19852 (0.0007) [2023-10-07 20:29:06,971][67838] Updated weights for policy 0, policy_version 19862 (0.0007) [2023-10-07 20:29:07,353][67838] Updated weights for policy 0, policy_version 19872 (0.0009) [2023-10-07 20:29:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 40730624. Throughput: 0: 1647.6, 1: 1645.2. Samples: 10187298. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) [2023-10-07 20:29:07,477][66916] Avg episode reward: [(0, '34.200'), (1, '34.550')] [2023-10-07 20:29:07,485][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000019904_20381696.pth... [2023-10-07 20:29:07,485][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000019872_20348928.pth... [2023-10-07 20:29:07,518][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000018368_18808832.pth [2023-10-07 20:29:07,526][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000018304_18743296.pth [2023-10-07 20:29:10,769][67871] Updated weights for policy 1, policy_version 19910 (0.0009) [2023-10-07 20:29:11,153][67871] Updated weights for policy 1, policy_version 19920 (0.0008) [2023-10-07 20:29:11,321][67838] Updated weights for policy 0, policy_version 19882 (0.0007) [2023-10-07 20:29:11,516][67871] Updated weights for policy 1, policy_version 19930 (0.0008) [2023-10-07 20:29:11,702][67838] Updated weights for policy 0, policy_version 19892 (0.0009) [2023-10-07 20:29:12,067][67838] Updated weights for policy 0, policy_version 19902 (0.0007) [2023-10-07 20:29:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 40796160. Throughput: 0: 1664.0, 1: 1650.0. Samples: 10198500. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 20:29:12,477][66916] Avg episode reward: [(0, '35.530'), (1, '36.720')] [2023-10-07 20:29:12,478][67676] Saving new best policy, reward=36.720! [2023-10-07 20:29:15,645][67871] Updated weights for policy 1, policy_version 19940 (0.0009) [2023-10-07 20:29:16,007][67871] Updated weights for policy 1, policy_version 19950 (0.0007) [2023-10-07 20:29:16,345][67838] Updated weights for policy 0, policy_version 19912 (0.0008) [2023-10-07 20:29:16,371][67871] Updated weights for policy 1, policy_version 19960 (0.0008) [2023-10-07 20:29:16,718][67838] Updated weights for policy 0, policy_version 19922 (0.0010) [2023-10-07 20:29:17,089][67838] Updated weights for policy 0, policy_version 19932 (0.0011) [2023-10-07 20:29:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 40861696. Throughput: 0: 1668.1, 1: 1647.8. Samples: 10218256. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 20:29:17,477][66916] Avg episode reward: [(0, '33.820'), (1, '33.470')] [2023-10-07 20:29:20,451][67871] Updated weights for policy 1, policy_version 19970 (0.0009) [2023-10-07 20:29:20,812][67871] Updated weights for policy 1, policy_version 19980 (0.0007) [2023-10-07 20:29:21,064][67838] Updated weights for policy 0, policy_version 19942 (0.0009) [2023-10-07 20:29:21,175][67871] Updated weights for policy 1, policy_version 19990 (0.0009) [2023-10-07 20:29:21,437][67838] Updated weights for policy 0, policy_version 19952 (0.0008) [2023-10-07 20:29:21,544][67871] Updated weights for policy 1, policy_version 20000 (0.0008) [2023-10-07 20:29:21,815][67838] Updated weights for policy 0, policy_version 19962 (0.0010) [2023-10-07 20:29:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 40927232. Throughput: 0: 1647.3, 1: 1650.5. Samples: 10236352. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 20:29:22,477][66916] Avg episode reward: [(0, '34.080'), (1, '34.870')] [2023-10-07 20:29:25,705][67871] Updated weights for policy 1, policy_version 20010 (0.0010) [2023-10-07 20:29:26,076][67871] Updated weights for policy 1, policy_version 20020 (0.0008) [2023-10-07 20:29:26,107][67838] Updated weights for policy 0, policy_version 19972 (0.0011) [2023-10-07 20:29:26,450][67871] Updated weights for policy 1, policy_version 20030 (0.0009) [2023-10-07 20:29:26,476][67838] Updated weights for policy 0, policy_version 19982 (0.0009) [2023-10-07 20:29:26,857][67838] Updated weights for policy 0, policy_version 19992 (0.0008) [2023-10-07 20:29:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 40992768. Throughput: 0: 1657.7, 1: 1651.7. Samples: 10247824. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-07 20:29:27,477][66916] Avg episode reward: [(0, '32.700'), (1, '34.180')] [2023-10-07 20:29:30,590][67871] Updated weights for policy 1, policy_version 20040 (0.0009) [2023-10-07 20:29:30,892][67838] Updated weights for policy 0, policy_version 20002 (0.0008) [2023-10-07 20:29:30,950][67871] Updated weights for policy 1, policy_version 20050 (0.0010) [2023-10-07 20:29:31,264][67838] Updated weights for policy 0, policy_version 20012 (0.0008) [2023-10-07 20:29:31,319][67871] Updated weights for policy 1, policy_version 20060 (0.0008) [2023-10-07 20:29:31,634][67838] Updated weights for policy 0, policy_version 20022 (0.0007) [2023-10-07 20:29:32,014][67838] Updated weights for policy 0, policy_version 20032 (0.0007) [2023-10-07 20:29:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 41058304. Throughput: 0: 1652.9, 1: 1649.5. Samples: 10267394. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-07 20:29:32,477][66916] Avg episode reward: [(0, '35.110'), (1, '32.960')] [2023-10-07 20:29:35,441][67871] Updated weights for policy 1, policy_version 20070 (0.0009) [2023-10-07 20:29:35,809][67871] Updated weights for policy 1, policy_version 20080 (0.0010) [2023-10-07 20:29:36,067][67838] Updated weights for policy 0, policy_version 20042 (0.0008) [2023-10-07 20:29:36,169][67871] Updated weights for policy 1, policy_version 20090 (0.0008) [2023-10-07 20:29:36,440][67838] Updated weights for policy 0, policy_version 20052 (0.0009) [2023-10-07 20:29:36,821][67838] Updated weights for policy 0, policy_version 20062 (0.0009) [2023-10-07 20:29:37,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 41123840. Throughput: 0: 1646.1, 1: 1657.4. Samples: 10286256. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-07 20:29:37,477][66916] Avg episode reward: [(0, '33.460'), (1, '33.760')] [2023-10-07 20:29:40,052][67871] Updated weights for policy 1, policy_version 20100 (0.0008) [2023-10-07 20:29:40,423][67871] Updated weights for policy 1, policy_version 20110 (0.0009) [2023-10-07 20:29:40,794][67871] Updated weights for policy 1, policy_version 20120 (0.0007) [2023-10-07 20:29:40,907][67838] Updated weights for policy 0, policy_version 20072 (0.0009) [2023-10-07 20:29:41,291][67838] Updated weights for policy 0, policy_version 20082 (0.0008) [2023-10-07 20:29:41,665][67838] Updated weights for policy 0, policy_version 20092 (0.0008) [2023-10-07 20:29:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 41189376. Throughput: 0: 1656.7, 1: 1661.9. Samples: 10297898. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-07 20:29:42,477][66916] Avg episode reward: [(0, '37.140'), (1, '34.010')] [2023-10-07 20:29:42,477][67511] Saving new best policy, reward=37.140! [2023-10-07 20:29:45,114][67871] Updated weights for policy 1, policy_version 20130 (0.0009) [2023-10-07 20:29:45,481][67871] Updated weights for policy 1, policy_version 20140 (0.0009) [2023-10-07 20:29:45,749][67838] Updated weights for policy 0, policy_version 20102 (0.0009) [2023-10-07 20:29:45,851][67871] Updated weights for policy 1, policy_version 20150 (0.0009) [2023-10-07 20:29:46,118][67838] Updated weights for policy 0, policy_version 20112 (0.0008) [2023-10-07 20:29:46,224][67871] Updated weights for policy 1, policy_version 20160 (0.0009) [2023-10-07 20:29:46,493][67838] Updated weights for policy 0, policy_version 20122 (0.0009) [2023-10-07 20:29:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 41254912. Throughput: 0: 1652.2, 1: 1648.4. Samples: 10316946. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-10-07 20:29:47,477][66916] Avg episode reward: [(0, '34.880'), (1, '33.710')] [2023-10-07 20:29:50,327][67871] Updated weights for policy 1, policy_version 20170 (0.0009) [2023-10-07 20:29:50,529][67838] Updated weights for policy 0, policy_version 20132 (0.0009) [2023-10-07 20:29:50,693][67871] Updated weights for policy 1, policy_version 20180 (0.0008) [2023-10-07 20:29:50,905][67838] Updated weights for policy 0, policy_version 20142 (0.0009) [2023-10-07 20:29:51,053][67871] Updated weights for policy 1, policy_version 20190 (0.0008) [2023-10-07 20:29:51,270][67838] Updated weights for policy 0, policy_version 20152 (0.0007) [2023-10-07 20:29:52,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 41320448. Throughput: 0: 1648.7, 1: 1659.9. Samples: 10336186. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-10-07 20:29:52,478][66916] Avg episode reward: [(0, '34.670'), (1, '33.990')] [2023-10-07 20:29:55,114][67871] Updated weights for policy 1, policy_version 20200 (0.0008) [2023-10-07 20:29:55,483][67871] Updated weights for policy 1, policy_version 20210 (0.0010) [2023-10-07 20:29:55,506][67838] Updated weights for policy 0, policy_version 20162 (0.0009) [2023-10-07 20:29:55,858][67871] Updated weights for policy 1, policy_version 20220 (0.0008) [2023-10-07 20:29:55,918][67838] Updated weights for policy 0, policy_version 20172 (0.0008) [2023-10-07 20:29:56,291][67838] Updated weights for policy 0, policy_version 20182 (0.0009) [2023-10-07 20:29:56,666][67838] Updated weights for policy 0, policy_version 20192 (0.0007) [2023-10-07 20:29:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41385984. Throughput: 0: 1655.7, 1: 1660.1. Samples: 10347710. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-10-07 20:29:57,477][66916] Avg episode reward: [(0, '37.630'), (1, '35.060')] [2023-10-07 20:29:57,478][67511] Saving new best policy, reward=37.630! [2023-10-07 20:30:00,184][67871] Updated weights for policy 1, policy_version 20230 (0.0009) [2023-10-07 20:30:00,559][67871] Updated weights for policy 1, policy_version 20240 (0.0007) [2023-10-07 20:30:00,797][67838] Updated weights for policy 0, policy_version 20202 (0.0010) [2023-10-07 20:30:00,925][67871] Updated weights for policy 1, policy_version 20250 (0.0007) [2023-10-07 20:30:01,165][67838] Updated weights for policy 0, policy_version 20212 (0.0007) [2023-10-07 20:30:01,541][67838] Updated weights for policy 0, policy_version 20222 (0.0010) [2023-10-07 20:30:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41451520. Throughput: 0: 1637.9, 1: 1651.9. Samples: 10366300. Policy #0 lag: (min: 9.0, avg: 17.7, max: 41.0) [2023-10-07 20:30:02,478][66916] Avg episode reward: [(0, '35.430'), (1, '35.000')] [2023-10-07 20:30:04,967][67871] Updated weights for policy 1, policy_version 20260 (0.0008) [2023-10-07 20:30:05,335][67871] Updated weights for policy 1, policy_version 20270 (0.0008) [2023-10-07 20:30:05,682][67838] Updated weights for policy 0, policy_version 20232 (0.0008) [2023-10-07 20:30:05,695][67871] Updated weights for policy 1, policy_version 20280 (0.0009) [2023-10-07 20:30:06,059][67838] Updated weights for policy 0, policy_version 20242 (0.0010) [2023-10-07 20:30:06,432][67838] Updated weights for policy 0, policy_version 20252 (0.0010) [2023-10-07 20:30:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 41517056. Throughput: 0: 1647.9, 1: 1668.3. Samples: 10385582. Policy #0 lag: (min: 9.0, avg: 17.7, max: 41.0) [2023-10-07 20:30:07,478][66916] Avg episode reward: [(0, '34.870'), (1, '35.040')] [2023-10-07 20:30:09,828][67871] Updated weights for policy 1, policy_version 20290 (0.0008) [2023-10-07 20:30:10,194][67871] Updated weights for policy 1, policy_version 20300 (0.0007) [2023-10-07 20:30:10,567][67871] Updated weights for policy 1, policy_version 20310 (0.0007) [2023-10-07 20:30:10,622][67838] Updated weights for policy 0, policy_version 20262 (0.0008) [2023-10-07 20:30:10,933][67871] Updated weights for policy 1, policy_version 20320 (0.0009) [2023-10-07 20:30:10,992][67838] Updated weights for policy 0, policy_version 20272 (0.0007) [2023-10-07 20:30:11,359][67838] Updated weights for policy 0, policy_version 20282 (0.0010) [2023-10-07 20:30:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41582592. Throughput: 0: 1653.1, 1: 1666.0. Samples: 10397182. Policy #0 lag: (min: 9.0, avg: 17.7, max: 41.0) [2023-10-07 20:30:12,477][66916] Avg episode reward: [(0, '33.990'), (1, '34.210')] [2023-10-07 20:30:14,944][67871] Updated weights for policy 1, policy_version 20330 (0.0008) [2023-10-07 20:30:15,312][67871] Updated weights for policy 1, policy_version 20340 (0.0010) [2023-10-07 20:30:15,512][67838] Updated weights for policy 0, policy_version 20292 (0.0011) [2023-10-07 20:30:15,681][67871] Updated weights for policy 1, policy_version 20350 (0.0008) [2023-10-07 20:30:15,878][67838] Updated weights for policy 0, policy_version 20302 (0.0009) [2023-10-07 20:30:16,254][67838] Updated weights for policy 0, policy_version 20312 (0.0008) [2023-10-07 20:30:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 41648128. Throughput: 0: 1640.2, 1: 1659.0. Samples: 10415858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:30:17,478][66916] Avg episode reward: [(0, '33.410'), (1, '33.820')] [2023-10-07 20:30:19,829][67871] Updated weights for policy 1, policy_version 20360 (0.0008) [2023-10-07 20:30:20,198][67871] Updated weights for policy 1, policy_version 20370 (0.0009) [2023-10-07 20:30:20,520][67838] Updated weights for policy 0, policy_version 20322 (0.0009) [2023-10-07 20:30:20,567][67871] Updated weights for policy 1, policy_version 20380 (0.0007) [2023-10-07 20:30:20,901][67838] Updated weights for policy 0, policy_version 20332 (0.0008) [2023-10-07 20:30:21,274][67838] Updated weights for policy 0, policy_version 20342 (0.0009) [2023-10-07 20:30:21,647][67838] Updated weights for policy 0, policy_version 20352 (0.0008) [2023-10-07 20:30:22,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41713664. Throughput: 0: 1647.3, 1: 1675.0. Samples: 10435758. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:30:22,478][66916] Avg episode reward: [(0, '35.730'), (1, '32.870')] [2023-10-07 20:30:24,601][67871] Updated weights for policy 1, policy_version 20390 (0.0009) [2023-10-07 20:30:24,960][67871] Updated weights for policy 1, policy_version 20400 (0.0007) [2023-10-07 20:30:25,325][67871] Updated weights for policy 1, policy_version 20410 (0.0008) [2023-10-07 20:30:25,706][67838] Updated weights for policy 0, policy_version 20362 (0.0009) [2023-10-07 20:30:26,079][67838] Updated weights for policy 0, policy_version 20372 (0.0010) [2023-10-07 20:30:26,451][67838] Updated weights for policy 0, policy_version 20382 (0.0007) [2023-10-07 20:30:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41779200. Throughput: 0: 1650.5, 1: 1659.4. Samples: 10446844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:30:27,477][66916] Avg episode reward: [(0, '34.390'), (1, '34.820')] [2023-10-07 20:30:29,528][67871] Updated weights for policy 1, policy_version 20420 (0.0009) [2023-10-07 20:30:29,891][67871] Updated weights for policy 1, policy_version 20430 (0.0008) [2023-10-07 20:30:30,262][67871] Updated weights for policy 1, policy_version 20440 (0.0007) [2023-10-07 20:30:30,729][67838] Updated weights for policy 0, policy_version 20392 (0.0009) [2023-10-07 20:30:31,109][67838] Updated weights for policy 0, policy_version 20402 (0.0008) [2023-10-07 20:30:31,485][67838] Updated weights for policy 0, policy_version 20412 (0.0009) [2023-10-07 20:30:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41844736. Throughput: 0: 1638.4, 1: 1660.4. Samples: 10465394. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-07 20:30:32,477][66916] Avg episode reward: [(0, '36.420'), (1, '35.360')] [2023-10-07 20:30:34,279][67871] Updated weights for policy 1, policy_version 20450 (0.0008) [2023-10-07 20:30:34,647][67871] Updated weights for policy 1, policy_version 20460 (0.0007) [2023-10-07 20:30:35,016][67871] Updated weights for policy 1, policy_version 20470 (0.0007) [2023-10-07 20:30:35,383][67871] Updated weights for policy 1, policy_version 20480 (0.0007) [2023-10-07 20:30:35,463][67838] Updated weights for policy 0, policy_version 20422 (0.0008) [2023-10-07 20:30:35,827][67838] Updated weights for policy 0, policy_version 20432 (0.0010) [2023-10-07 20:30:36,201][67838] Updated weights for policy 0, policy_version 20442 (0.0008) [2023-10-07 20:30:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41910272. Throughput: 0: 1648.7, 1: 1670.7. Samples: 10485560. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-07 20:30:37,477][66916] Avg episode reward: [(0, '36.250'), (1, '33.010')] [2023-10-07 20:30:39,610][67871] Updated weights for policy 1, policy_version 20490 (0.0007) [2023-10-07 20:30:39,976][67871] Updated weights for policy 1, policy_version 20500 (0.0007) [2023-10-07 20:30:40,334][67871] Updated weights for policy 1, policy_version 20510 (0.0009) [2023-10-07 20:30:40,425][67838] Updated weights for policy 0, policy_version 20452 (0.0009) [2023-10-07 20:30:40,815][67838] Updated weights for policy 0, policy_version 20462 (0.0009) [2023-10-07 20:30:41,189][67838] Updated weights for policy 0, policy_version 20472 (0.0009) [2023-10-07 20:30:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41975808. Throughput: 0: 1647.8, 1: 1657.0. Samples: 10496424. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-07 20:30:42,477][66916] Avg episode reward: [(0, '32.720'), (1, '34.550')] [2023-10-07 20:30:44,479][67871] Updated weights for policy 1, policy_version 20520 (0.0007) [2023-10-07 20:30:44,847][67871] Updated weights for policy 1, policy_version 20530 (0.0007) [2023-10-07 20:30:45,213][67871] Updated weights for policy 1, policy_version 20540 (0.0008) [2023-10-07 20:30:45,272][67838] Updated weights for policy 0, policy_version 20482 (0.0007) [2023-10-07 20:30:45,651][67838] Updated weights for policy 0, policy_version 20492 (0.0007) [2023-10-07 20:30:46,019][67838] Updated weights for policy 0, policy_version 20502 (0.0009) [2023-10-07 20:30:46,400][67838] Updated weights for policy 0, policy_version 20512 (0.0008) [2023-10-07 20:30:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 42041344. Throughput: 0: 1646.6, 1: 1666.3. Samples: 10515380. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-07 20:30:47,477][66916] Avg episode reward: [(0, '37.420'), (1, '34.220')] [2023-10-07 20:30:49,335][67871] Updated weights for policy 1, policy_version 20550 (0.0009) [2023-10-07 20:30:49,720][67871] Updated weights for policy 1, policy_version 20560 (0.0008) [2023-10-07 20:30:50,087][67871] Updated weights for policy 1, policy_version 20570 (0.0007) [2023-10-07 20:30:50,457][67838] Updated weights for policy 0, policy_version 20522 (0.0007) [2023-10-07 20:30:50,836][67838] Updated weights for policy 0, policy_version 20532 (0.0007) [2023-10-07 20:30:51,198][67838] Updated weights for policy 0, policy_version 20542 (0.0008) [2023-10-07 20:30:52,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 42106880. Throughput: 0: 1656.5, 1: 1666.3. Samples: 10535106. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-07 20:30:52,478][66916] Avg episode reward: [(0, '32.970'), (1, '34.060')] [2023-10-07 20:30:54,173][67871] Updated weights for policy 1, policy_version 20580 (0.0010) [2023-10-07 20:30:54,540][67871] Updated weights for policy 1, policy_version 20590 (0.0010) [2023-10-07 20:30:54,907][67871] Updated weights for policy 1, policy_version 20600 (0.0007) [2023-10-07 20:30:55,243][67838] Updated weights for policy 0, policy_version 20552 (0.0008) [2023-10-07 20:30:55,624][67838] Updated weights for policy 0, policy_version 20562 (0.0009) [2023-10-07 20:30:56,000][67838] Updated weights for policy 0, policy_version 20572 (0.0007) [2023-10-07 20:30:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 42172416. Throughput: 0: 1655.4, 1: 1650.8. Samples: 10545960. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-07 20:30:57,477][66916] Avg episode reward: [(0, '34.480'), (1, '35.180')] [2023-10-07 20:30:58,997][67871] Updated weights for policy 1, policy_version 20610 (0.0007) [2023-10-07 20:30:59,376][67871] Updated weights for policy 1, policy_version 20620 (0.0008) [2023-10-07 20:30:59,742][67871] Updated weights for policy 1, policy_version 20630 (0.0009) [2023-10-07 20:31:00,035][67838] Updated weights for policy 0, policy_version 20582 (0.0008) [2023-10-07 20:31:00,102][67871] Updated weights for policy 1, policy_version 20640 (0.0009) [2023-10-07 20:31:00,411][67838] Updated weights for policy 0, policy_version 20592 (0.0009) [2023-10-07 20:31:00,785][67838] Updated weights for policy 0, policy_version 20602 (0.0008) [2023-10-07 20:31:02,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 42237952. Throughput: 0: 1650.5, 1: 1659.9. Samples: 10564826. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-07 20:31:02,478][66916] Avg episode reward: [(0, '33.500'), (1, '33.920')] [2023-10-07 20:31:04,165][67871] Updated weights for policy 1, policy_version 20650 (0.0009) [2023-10-07 20:31:04,541][67871] Updated weights for policy 1, policy_version 20660 (0.0009) [2023-10-07 20:31:04,857][67838] Updated weights for policy 0, policy_version 20612 (0.0007) [2023-10-07 20:31:04,920][67871] Updated weights for policy 1, policy_version 20670 (0.0009) [2023-10-07 20:31:05,231][67838] Updated weights for policy 0, policy_version 20622 (0.0009) [2023-10-07 20:31:05,599][67838] Updated weights for policy 0, policy_version 20632 (0.0009) [2023-10-07 20:31:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 42303488. Throughput: 0: 1668.6, 1: 1658.1. Samples: 10585460. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-07 20:31:07,478][66916] Avg episode reward: [(0, '33.190'), (1, '35.150')] [2023-10-07 20:31:07,492][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000020640_21135360.pth... [2023-10-07 20:31:07,492][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000020672_21168128.pth... [2023-10-07 20:31:07,525][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000019104_19562496.pth [2023-10-07 20:31:07,528][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000019136_19595264.pth [2023-10-07 20:31:09,107][67871] Updated weights for policy 1, policy_version 20680 (0.0008) [2023-10-07 20:31:09,473][67871] Updated weights for policy 1, policy_version 20690 (0.0007) [2023-10-07 20:31:09,647][67838] Updated weights for policy 0, policy_version 20642 (0.0007) [2023-10-07 20:31:09,839][67871] Updated weights for policy 1, policy_version 20700 (0.0008) [2023-10-07 20:31:10,015][67838] Updated weights for policy 0, policy_version 20652 (0.0010) [2023-10-07 20:31:10,394][67838] Updated weights for policy 0, policy_version 20662 (0.0010) [2023-10-07 20:31:10,759][67838] Updated weights for policy 0, policy_version 20672 (0.0009) [2023-10-07 20:31:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 42369024. Throughput: 0: 1658.4, 1: 1643.9. Samples: 10595446. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-07 20:31:12,477][66916] Avg episode reward: [(0, '34.800'), (1, '33.890')] [2023-10-07 20:31:13,939][67871] Updated weights for policy 1, policy_version 20710 (0.0008) [2023-10-07 20:31:14,308][67871] Updated weights for policy 1, policy_version 20720 (0.0008) [2023-10-07 20:31:14,680][67871] Updated weights for policy 1, policy_version 20730 (0.0008) [2023-10-07 20:31:14,972][67838] Updated weights for policy 0, policy_version 20682 (0.0007) [2023-10-07 20:31:15,338][67838] Updated weights for policy 0, policy_version 20692 (0.0007) [2023-10-07 20:31:15,716][67838] Updated weights for policy 0, policy_version 20702 (0.0010) [2023-10-07 20:31:17,477][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 42434560. Throughput: 0: 1661.0, 1: 1661.4. Samples: 10614904. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-07 20:31:17,478][66916] Avg episode reward: [(0, '32.070'), (1, '34.700')] [2023-10-07 20:31:18,940][67871] Updated weights for policy 1, policy_version 20740 (0.0009) [2023-10-07 20:31:19,302][67871] Updated weights for policy 1, policy_version 20750 (0.0011) [2023-10-07 20:31:19,675][67871] Updated weights for policy 1, policy_version 20760 (0.0009) [2023-10-07 20:31:19,771][67838] Updated weights for policy 0, policy_version 20712 (0.0007) [2023-10-07 20:31:20,145][67838] Updated weights for policy 0, policy_version 20722 (0.0007) [2023-10-07 20:31:20,514][67838] Updated weights for policy 0, policy_version 20732 (0.0008) [2023-10-07 20:31:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 42500096. Throughput: 0: 1671.4, 1: 1659.9. Samples: 10635466. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-07 20:31:22,477][66916] Avg episode reward: [(0, '34.370'), (1, '34.300')] [2023-10-07 20:31:23,994][67871] Updated weights for policy 1, policy_version 20770 (0.0008) [2023-10-07 20:31:24,361][67871] Updated weights for policy 1, policy_version 20780 (0.0009) [2023-10-07 20:31:24,525][67838] Updated weights for policy 0, policy_version 20742 (0.0009) [2023-10-07 20:31:24,733][67871] Updated weights for policy 1, policy_version 20790 (0.0008) [2023-10-07 20:31:24,883][67838] Updated weights for policy 0, policy_version 20752 (0.0009) [2023-10-07 20:31:25,107][67871] Updated weights for policy 1, policy_version 20800 (0.0009) [2023-10-07 20:31:25,259][67838] Updated weights for policy 0, policy_version 20762 (0.0009) [2023-10-07 20:31:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 42565632. Throughput: 0: 1656.7, 1: 1653.8. Samples: 10645394. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) [2023-10-07 20:31:27,477][66916] Avg episode reward: [(0, '33.640'), (1, '33.420')] [2023-10-07 20:31:29,081][67871] Updated weights for policy 1, policy_version 20810 (0.0008) [2023-10-07 20:31:29,449][67871] Updated weights for policy 1, policy_version 20820 (0.0008) [2023-10-07 20:31:29,505][67838] Updated weights for policy 0, policy_version 20772 (0.0008) [2023-10-07 20:31:29,821][67871] Updated weights for policy 1, policy_version 20830 (0.0008) [2023-10-07 20:31:29,893][67838] Updated weights for policy 0, policy_version 20782 (0.0009) [2023-10-07 20:31:30,260][67838] Updated weights for policy 0, policy_version 20792 (0.0009) [2023-10-07 20:31:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 42631168. Throughput: 0: 1662.5, 1: 1656.6. Samples: 10664738. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) [2023-10-07 20:31:32,478][66916] Avg episode reward: [(0, '34.690'), (1, '35.680')] [2023-10-07 20:31:34,099][67871] Updated weights for policy 1, policy_version 20840 (0.0010) [2023-10-07 20:31:34,471][67838] Updated weights for policy 0, policy_version 20802 (0.0011) [2023-10-07 20:31:34,489][67871] Updated weights for policy 1, policy_version 20850 (0.0010) [2023-10-07 20:31:34,849][67838] Updated weights for policy 0, policy_version 20812 (0.0007) [2023-10-07 20:31:34,855][67871] Updated weights for policy 1, policy_version 20860 (0.0008) [2023-10-07 20:31:35,223][67838] Updated weights for policy 0, policy_version 20822 (0.0009) [2023-10-07 20:31:35,596][67838] Updated weights for policy 0, policy_version 20832 (0.0008) [2023-10-07 20:31:37,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 42696704. Throughput: 0: 1673.3, 1: 1662.5. Samples: 10685218. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) [2023-10-07 20:31:37,478][66916] Avg episode reward: [(0, '33.560'), (1, '33.210')] [2023-10-07 20:31:38,906][67871] Updated weights for policy 1, policy_version 20870 (0.0008) [2023-10-07 20:31:39,273][67871] Updated weights for policy 1, policy_version 20880 (0.0011) [2023-10-07 20:31:39,638][67871] Updated weights for policy 1, policy_version 20890 (0.0009) [2023-10-07 20:31:39,723][67838] Updated weights for policy 0, policy_version 20842 (0.0009) [2023-10-07 20:31:40,097][67838] Updated weights for policy 0, policy_version 20852 (0.0007) [2023-10-07 20:31:40,467][67838] Updated weights for policy 0, policy_version 20862 (0.0007) [2023-10-07 20:31:42,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 42762240. Throughput: 0: 1657.8, 1: 1652.8. Samples: 10694938. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) [2023-10-07 20:31:42,477][66916] Avg episode reward: [(0, '36.240'), (1, '33.390')] [2023-10-07 20:31:43,685][67871] Updated weights for policy 1, policy_version 20900 (0.0008) [2023-10-07 20:31:44,050][67871] Updated weights for policy 1, policy_version 20910 (0.0009) [2023-10-07 20:31:44,418][67871] Updated weights for policy 1, policy_version 20920 (0.0011) [2023-10-07 20:31:44,705][67838] Updated weights for policy 0, policy_version 20872 (0.0009) [2023-10-07 20:31:45,066][67838] Updated weights for policy 0, policy_version 20882 (0.0007) [2023-10-07 20:31:45,436][67838] Updated weights for policy 0, policy_version 20892 (0.0009) [2023-10-07 20:31:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 42827776. Throughput: 0: 1665.6, 1: 1664.8. Samples: 10714690. Policy #0 lag: (min: 3.0, avg: 8.2, max: 35.0) [2023-10-07 20:31:47,478][66916] Avg episode reward: [(0, '36.750'), (1, '34.130')] [2023-10-07 20:31:48,626][67871] Updated weights for policy 1, policy_version 20930 (0.0008) [2023-10-07 20:31:48,996][67871] Updated weights for policy 1, policy_version 20940 (0.0010) [2023-10-07 20:31:49,349][67871] Updated weights for policy 1, policy_version 20950 (0.0008) [2023-10-07 20:31:49,524][67838] Updated weights for policy 0, policy_version 20902 (0.0008) [2023-10-07 20:31:49,728][67871] Updated weights for policy 1, policy_version 20960 (0.0007) [2023-10-07 20:31:49,889][67838] Updated weights for policy 0, policy_version 20912 (0.0008) [2023-10-07 20:31:50,267][67838] Updated weights for policy 0, policy_version 20922 (0.0007) [2023-10-07 20:31:52,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 42893312. Throughput: 0: 1663.9, 1: 1660.5. Samples: 10735056. Policy #0 lag: (min: 3.0, avg: 8.2, max: 35.0) [2023-10-07 20:31:52,477][66916] Avg episode reward: [(0, '33.250'), (1, '34.030')] [2023-10-07 20:31:53,849][67871] Updated weights for policy 1, policy_version 20970 (0.0007) [2023-10-07 20:31:54,218][67871] Updated weights for policy 1, policy_version 20980 (0.0007) [2023-10-07 20:31:54,307][67838] Updated weights for policy 0, policy_version 20932 (0.0009) [2023-10-07 20:31:54,576][67871] Updated weights for policy 1, policy_version 20990 (0.0009) [2023-10-07 20:31:54,681][67838] Updated weights for policy 0, policy_version 20942 (0.0008) [2023-10-07 20:31:55,049][67838] Updated weights for policy 0, policy_version 20952 (0.0009) [2023-10-07 20:31:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 42958848. Throughput: 0: 1652.6, 1: 1654.8. Samples: 10744278. Policy #0 lag: (min: 3.0, avg: 8.2, max: 35.0) [2023-10-07 20:31:57,477][66916] Avg episode reward: [(0, '31.490'), (1, '32.810')] [2023-10-07 20:31:58,527][67871] Updated weights for policy 1, policy_version 21000 (0.0008) [2023-10-07 20:31:58,903][67871] Updated weights for policy 1, policy_version 21010 (0.0009) [2023-10-07 20:31:59,197][67838] Updated weights for policy 0, policy_version 20962 (0.0010) [2023-10-07 20:31:59,275][67871] Updated weights for policy 1, policy_version 21020 (0.0009) [2023-10-07 20:31:59,574][67838] Updated weights for policy 0, policy_version 20972 (0.0009) [2023-10-07 20:31:59,939][67838] Updated weights for policy 0, policy_version 20982 (0.0010) [2023-10-07 20:32:00,315][67838] Updated weights for policy 0, policy_version 20992 (0.0010) [2023-10-07 20:32:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 43024384. Throughput: 0: 1656.9, 1: 1663.2. Samples: 10764306. Policy #0 lag: (min: 1.0, avg: 6.6, max: 33.0) [2023-10-07 20:32:02,477][66916] Avg episode reward: [(0, '35.770'), (1, '33.630')] [2023-10-07 20:32:03,402][67871] Updated weights for policy 1, policy_version 21030 (0.0007) [2023-10-07 20:32:03,769][67871] Updated weights for policy 1, policy_version 21040 (0.0007) [2023-10-07 20:32:04,134][67871] Updated weights for policy 1, policy_version 21050 (0.0008) [2023-10-07 20:32:04,517][67838] Updated weights for policy 0, policy_version 21002 (0.0010) [2023-10-07 20:32:04,892][67838] Updated weights for policy 0, policy_version 21012 (0.0008) [2023-10-07 20:32:05,260][67838] Updated weights for policy 0, policy_version 21022 (0.0009) [2023-10-07 20:32:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 43089920. Throughput: 0: 1651.8, 1: 1663.6. Samples: 10784656. Policy #0 lag: (min: 1.0, avg: 6.6, max: 33.0) [2023-10-07 20:32:07,477][66916] Avg episode reward: [(0, '33.820'), (1, '34.020')] [2023-10-07 20:32:08,114][67871] Updated weights for policy 1, policy_version 21060 (0.0007) [2023-10-07 20:32:08,474][67871] Updated weights for policy 1, policy_version 21070 (0.0009) [2023-10-07 20:32:08,847][67871] Updated weights for policy 1, policy_version 21080 (0.0008) [2023-10-07 20:32:09,381][67838] Updated weights for policy 0, policy_version 21032 (0.0008) [2023-10-07 20:32:09,765][67838] Updated weights for policy 0, policy_version 21042 (0.0010) [2023-10-07 20:32:10,152][67838] Updated weights for policy 0, policy_version 21052 (0.0012) [2023-10-07 20:32:12,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 43155456. Throughput: 0: 1647.5, 1: 1657.2. Samples: 10794108. Policy #0 lag: (min: 1.0, avg: 6.6, max: 33.0) [2023-10-07 20:32:12,478][66916] Avg episode reward: [(0, '33.360'), (1, '35.180')] [2023-10-07 20:32:13,162][67871] Updated weights for policy 1, policy_version 21090 (0.0008) [2023-10-07 20:32:13,539][67871] Updated weights for policy 1, policy_version 21100 (0.0007) [2023-10-07 20:32:13,904][67871] Updated weights for policy 1, policy_version 21110 (0.0009) [2023-10-07 20:32:14,272][67871] Updated weights for policy 1, policy_version 21120 (0.0010) [2023-10-07 20:32:14,305][67838] Updated weights for policy 0, policy_version 21062 (0.0009) [2023-10-07 20:32:14,687][67838] Updated weights for policy 0, policy_version 21072 (0.0008) [2023-10-07 20:32:15,055][67838] Updated weights for policy 0, policy_version 21082 (0.0011) [2023-10-07 20:32:17,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43220992. Throughput: 0: 1658.2, 1: 1662.1. Samples: 10814148. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 20:32:17,477][66916] Avg episode reward: [(0, '35.110'), (1, '34.530')] [2023-10-07 20:32:18,266][67871] Updated weights for policy 1, policy_version 21130 (0.0010) [2023-10-07 20:32:18,631][67871] Updated weights for policy 1, policy_version 21140 (0.0012) [2023-10-07 20:32:19,001][67871] Updated weights for policy 1, policy_version 21150 (0.0008) [2023-10-07 20:32:19,140][67838] Updated weights for policy 0, policy_version 21092 (0.0009) [2023-10-07 20:32:19,524][67838] Updated weights for policy 0, policy_version 21102 (0.0010) [2023-10-07 20:32:19,899][67838] Updated weights for policy 0, policy_version 21112 (0.0011) [2023-10-07 20:32:22,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43286528. Throughput: 0: 1649.4, 1: 1661.0. Samples: 10834188. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 20:32:22,478][66916] Avg episode reward: [(0, '33.540'), (1, '34.200')] [2023-10-07 20:32:23,247][67871] Updated weights for policy 1, policy_version 21160 (0.0007) [2023-10-07 20:32:23,618][67871] Updated weights for policy 1, policy_version 21170 (0.0007) [2023-10-07 20:32:23,982][67871] Updated weights for policy 1, policy_version 21180 (0.0009) [2023-10-07 20:32:24,258][67838] Updated weights for policy 0, policy_version 21122 (0.0009) [2023-10-07 20:32:24,638][67838] Updated weights for policy 0, policy_version 21132 (0.0010) [2023-10-07 20:32:25,006][67838] Updated weights for policy 0, policy_version 21142 (0.0010) [2023-10-07 20:32:25,380][67838] Updated weights for policy 0, policy_version 21152 (0.0011) [2023-10-07 20:32:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43352064. Throughput: 0: 1643.6, 1: 1656.8. Samples: 10843454. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 20:32:27,477][66916] Avg episode reward: [(0, '33.660'), (1, '35.240')] [2023-10-07 20:32:27,978][67871] Updated weights for policy 1, policy_version 21190 (0.0009) [2023-10-07 20:32:28,347][67871] Updated weights for policy 1, policy_version 21200 (0.0009) [2023-10-07 20:32:28,714][67871] Updated weights for policy 1, policy_version 21210 (0.0009) [2023-10-07 20:32:29,269][67838] Updated weights for policy 0, policy_version 21162 (0.0008) [2023-10-07 20:32:29,641][67838] Updated weights for policy 0, policy_version 21172 (0.0008) [2023-10-07 20:32:30,011][67838] Updated weights for policy 0, policy_version 21182 (0.0009) [2023-10-07 20:32:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43417600. Throughput: 0: 1649.7, 1: 1659.9. Samples: 10863624. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 20:32:32,477][66916] Avg episode reward: [(0, '35.030'), (1, '33.850')] [2023-10-07 20:32:32,857][67871] Updated weights for policy 1, policy_version 21220 (0.0009) [2023-10-07 20:32:33,218][67871] Updated weights for policy 1, policy_version 21230 (0.0008) [2023-10-07 20:32:33,587][67871] Updated weights for policy 1, policy_version 21240 (0.0009) [2023-10-07 20:32:34,285][67838] Updated weights for policy 0, policy_version 21192 (0.0009) [2023-10-07 20:32:34,654][67838] Updated weights for policy 0, policy_version 21202 (0.0009) [2023-10-07 20:32:35,040][67838] Updated weights for policy 0, policy_version 21212 (0.0008) [2023-10-07 20:32:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43483136. Throughput: 0: 1649.0, 1: 1666.2. Samples: 10884240. Policy #0 lag: (min: 11.0, avg: 11.2, max: 22.0) [2023-10-07 20:32:37,477][66916] Avg episode reward: [(0, '34.240'), (1, '34.390')] [2023-10-07 20:32:37,584][67871] Updated weights for policy 1, policy_version 21250 (0.0009) [2023-10-07 20:32:37,964][67871] Updated weights for policy 1, policy_version 21260 (0.0008) [2023-10-07 20:32:38,326][67871] Updated weights for policy 1, policy_version 21270 (0.0009) [2023-10-07 20:32:38,702][67871] Updated weights for policy 1, policy_version 21280 (0.0009) [2023-10-07 20:32:39,152][67838] Updated weights for policy 0, policy_version 21222 (0.0009) [2023-10-07 20:32:39,538][67838] Updated weights for policy 0, policy_version 21232 (0.0007) [2023-10-07 20:32:39,909][67838] Updated weights for policy 0, policy_version 21242 (0.0007) [2023-10-07 20:32:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43548672. Throughput: 0: 1644.6, 1: 1669.6. Samples: 10893416. Policy #0 lag: (min: 11.0, avg: 11.2, max: 22.0) [2023-10-07 20:32:42,477][66916] Avg episode reward: [(0, '34.560'), (1, '33.120')] [2023-10-07 20:32:42,862][67871] Updated weights for policy 1, policy_version 21290 (0.0008) [2023-10-07 20:32:43,237][67871] Updated weights for policy 1, policy_version 21300 (0.0011) [2023-10-07 20:32:43,602][67871] Updated weights for policy 1, policy_version 21310 (0.0008) [2023-10-07 20:32:43,952][67838] Updated weights for policy 0, policy_version 21252 (0.0007) [2023-10-07 20:32:44,318][67838] Updated weights for policy 0, policy_version 21262 (0.0007) [2023-10-07 20:32:44,701][67838] Updated weights for policy 0, policy_version 21272 (0.0010) [2023-10-07 20:32:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 43614208. Throughput: 0: 1660.4, 1: 1664.8. Samples: 10913940. Policy #0 lag: (min: 11.0, avg: 11.2, max: 22.0) [2023-10-07 20:32:47,477][66916] Avg episode reward: [(0, '34.080'), (1, '33.530')] [2023-10-07 20:32:47,638][67871] Updated weights for policy 1, policy_version 21320 (0.0009) [2023-10-07 20:32:48,003][67871] Updated weights for policy 1, policy_version 21330 (0.0008) [2023-10-07 20:32:48,373][67871] Updated weights for policy 1, policy_version 21340 (0.0009) [2023-10-07 20:32:48,825][67838] Updated weights for policy 0, policy_version 21282 (0.0009) [2023-10-07 20:32:49,193][67838] Updated weights for policy 0, policy_version 21292 (0.0010) [2023-10-07 20:32:49,579][67838] Updated weights for policy 0, policy_version 21302 (0.0011) [2023-10-07 20:32:49,945][67838] Updated weights for policy 0, policy_version 21312 (0.0010) [2023-10-07 20:32:52,447][67871] Updated weights for policy 1, policy_version 21350 (0.0008) [2023-10-07 20:32:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43679744. Throughput: 0: 1660.4, 1: 1669.6. Samples: 10934506. Policy #0 lag: (min: 11.0, avg: 11.2, max: 22.0) [2023-10-07 20:32:52,477][66916] Avg episode reward: [(0, '33.730'), (1, '34.970')] [2023-10-07 20:32:52,813][67871] Updated weights for policy 1, policy_version 21360 (0.0009) [2023-10-07 20:32:53,181][67871] Updated weights for policy 1, policy_version 21370 (0.0007) [2023-10-07 20:32:53,951][67838] Updated weights for policy 0, policy_version 21322 (0.0010) [2023-10-07 20:32:54,323][67838] Updated weights for policy 0, policy_version 21332 (0.0012) [2023-10-07 20:32:54,697][67838] Updated weights for policy 0, policy_version 21342 (0.0011) [2023-10-07 20:32:57,383][67871] Updated weights for policy 1, policy_version 21380 (0.0009) [2023-10-07 20:32:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43745280. Throughput: 0: 1652.3, 1: 1669.0. Samples: 10943566. Policy #0 lag: (min: 27.0, avg: 28.6, max: 54.0) [2023-10-07 20:32:57,478][66916] Avg episode reward: [(0, '34.790'), (1, '34.920')] [2023-10-07 20:32:57,756][67871] Updated weights for policy 1, policy_version 21390 (0.0009) [2023-10-07 20:32:58,126][67871] Updated weights for policy 1, policy_version 21400 (0.0007) [2023-10-07 20:32:58,642][67838] Updated weights for policy 0, policy_version 21352 (0.0009) [2023-10-07 20:32:59,024][67838] Updated weights for policy 0, policy_version 21362 (0.0011) [2023-10-07 20:32:59,397][67838] Updated weights for policy 0, policy_version 21372 (0.0010) [2023-10-07 20:33:02,366][67871] Updated weights for policy 1, policy_version 21410 (0.0008) [2023-10-07 20:33:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43810816. Throughput: 0: 1664.8, 1: 1669.7. Samples: 10964200. Policy #0 lag: (min: 27.0, avg: 28.6, max: 54.0) [2023-10-07 20:33:02,477][66916] Avg episode reward: [(0, '33.060'), (1, '35.080')] [2023-10-07 20:33:02,724][67871] Updated weights for policy 1, policy_version 21420 (0.0010) [2023-10-07 20:33:03,086][67871] Updated weights for policy 1, policy_version 21430 (0.0011) [2023-10-07 20:33:03,297][67838] Updated weights for policy 0, policy_version 21382 (0.0008) [2023-10-07 20:33:03,448][67871] Updated weights for policy 1, policy_version 21440 (0.0009) [2023-10-07 20:33:03,667][67838] Updated weights for policy 0, policy_version 21392 (0.0008) [2023-10-07 20:33:04,046][67838] Updated weights for policy 0, policy_version 21402 (0.0011) [2023-10-07 20:33:07,443][67871] Updated weights for policy 1, policy_version 21450 (0.0008) [2023-10-07 20:33:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43876352. Throughput: 0: 1676.2, 1: 1674.9. Samples: 10984986. Policy #0 lag: (min: 27.0, avg: 28.6, max: 54.0) [2023-10-07 20:33:07,478][66916] Avg episode reward: [(0, '35.330'), (1, '34.880')] [2023-10-07 20:33:07,488][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000021408_21921792.pth... [2023-10-07 20:33:07,522][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000019872_20348928.pth [2023-10-07 20:33:07,815][67871] Updated weights for policy 1, policy_version 21460 (0.0007) [2023-10-07 20:33:08,187][67871] Updated weights for policy 1, policy_version 21470 (0.0009) [2023-10-07 20:33:08,236][67838] Updated weights for policy 0, policy_version 21412 (0.0008) [2023-10-07 20:33:08,253][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000021472_21987328.pth... [2023-10-07 20:33:08,283][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000019904_20381696.pth [2023-10-07 20:33:08,623][67838] Updated weights for policy 0, policy_version 21422 (0.0008) [2023-10-07 20:33:09,004][67838] Updated weights for policy 0, policy_version 21432 (0.0010) [2023-10-07 20:33:12,425][67871] Updated weights for policy 1, policy_version 21480 (0.0007) [2023-10-07 20:33:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 43941888. Throughput: 0: 1664.3, 1: 1673.8. Samples: 10993670. Policy #0 lag: (min: 27.0, avg: 28.6, max: 54.0) [2023-10-07 20:33:12,477][66916] Avg episode reward: [(0, '35.680'), (1, '34.680')] [2023-10-07 20:33:12,799][67871] Updated weights for policy 1, policy_version 21490 (0.0007) [2023-10-07 20:33:13,157][67838] Updated weights for policy 0, policy_version 21442 (0.0010) [2023-10-07 20:33:13,167][67871] Updated weights for policy 1, policy_version 21500 (0.0007) [2023-10-07 20:33:13,543][67838] Updated weights for policy 0, policy_version 21452 (0.0007) [2023-10-07 20:33:13,914][67838] Updated weights for policy 0, policy_version 21462 (0.0008) [2023-10-07 20:33:14,281][67838] Updated weights for policy 0, policy_version 21472 (0.0007) [2023-10-07 20:33:17,307][67871] Updated weights for policy 1, policy_version 21510 (0.0008) [2023-10-07 20:33:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 44007424. Throughput: 0: 1673.0, 1: 1670.2. Samples: 11014068. Policy #0 lag: (min: 17.0, avg: 25.3, max: 49.0) [2023-10-07 20:33:17,477][66916] Avg episode reward: [(0, '35.640'), (1, '34.380')] [2023-10-07 20:33:17,679][67871] Updated weights for policy 1, policy_version 21520 (0.0011) [2023-10-07 20:33:18,056][67871] Updated weights for policy 1, policy_version 21530 (0.0010) [2023-10-07 20:33:18,253][67838] Updated weights for policy 0, policy_version 21482 (0.0007) [2023-10-07 20:33:18,627][67838] Updated weights for policy 0, policy_version 21492 (0.0008) [2023-10-07 20:33:19,000][67838] Updated weights for policy 0, policy_version 21502 (0.0009) [2023-10-07 20:33:22,174][67871] Updated weights for policy 1, policy_version 21540 (0.0007) [2023-10-07 20:33:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 44072960. Throughput: 0: 1676.3, 1: 1663.8. Samples: 11034546. Policy #0 lag: (min: 17.0, avg: 25.3, max: 49.0) [2023-10-07 20:33:22,478][66916] Avg episode reward: [(0, '34.850'), (1, '33.390')] [2023-10-07 20:33:22,551][67871] Updated weights for policy 1, policy_version 21550 (0.0007) [2023-10-07 20:33:22,919][67871] Updated weights for policy 1, policy_version 21560 (0.0008) [2023-10-07 20:33:23,104][67838] Updated weights for policy 0, policy_version 21512 (0.0009) [2023-10-07 20:33:23,481][67838] Updated weights for policy 0, policy_version 21522 (0.0008) [2023-10-07 20:33:23,848][67838] Updated weights for policy 0, policy_version 21532 (0.0009) [2023-10-07 20:33:27,086][67871] Updated weights for policy 1, policy_version 21570 (0.0007) [2023-10-07 20:33:27,461][67871] Updated weights for policy 1, policy_version 21580 (0.0007) [2023-10-07 20:33:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 44138496. Throughput: 0: 1675.0, 1: 1663.7. Samples: 11043658. Policy #0 lag: (min: 17.0, avg: 25.3, max: 49.0) [2023-10-07 20:33:27,477][66916] Avg episode reward: [(0, '34.530'), (1, '33.110')] [2023-10-07 20:33:27,834][67871] Updated weights for policy 1, policy_version 21590 (0.0007) [2023-10-07 20:33:27,923][67838] Updated weights for policy 0, policy_version 21542 (0.0008) [2023-10-07 20:33:28,206][67871] Updated weights for policy 1, policy_version 21600 (0.0009) [2023-10-07 20:33:28,287][67838] Updated weights for policy 0, policy_version 21552 (0.0009) [2023-10-07 20:33:28,660][67838] Updated weights for policy 0, policy_version 21562 (0.0008) [2023-10-07 20:33:32,342][67871] Updated weights for policy 1, policy_version 21610 (0.0008) [2023-10-07 20:33:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 44204032. Throughput: 0: 1672.4, 1: 1658.8. Samples: 11063842. Policy #0 lag: (min: 17.0, avg: 25.3, max: 49.0) [2023-10-07 20:33:32,477][66916] Avg episode reward: [(0, '34.910'), (1, '35.250')] [2023-10-07 20:33:32,702][67871] Updated weights for policy 1, policy_version 21620 (0.0007) [2023-10-07 20:33:32,961][67838] Updated weights for policy 0, policy_version 21572 (0.0007) [2023-10-07 20:33:33,070][67871] Updated weights for policy 1, policy_version 21630 (0.0007) [2023-10-07 20:33:33,338][67838] Updated weights for policy 0, policy_version 21582 (0.0008) [2023-10-07 20:33:33,714][67838] Updated weights for policy 0, policy_version 21592 (0.0007) [2023-10-07 20:33:37,224][67871] Updated weights for policy 1, policy_version 21640 (0.0009) [2023-10-07 20:33:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 44269568. Throughput: 0: 1672.8, 1: 1654.3. Samples: 11084224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:33:37,477][66916] Avg episode reward: [(0, '33.710'), (1, '34.370')] [2023-10-07 20:33:37,605][67871] Updated weights for policy 1, policy_version 21650 (0.0008) [2023-10-07 20:33:37,964][67871] Updated weights for policy 1, policy_version 21660 (0.0009) [2023-10-07 20:33:37,970][67838] Updated weights for policy 0, policy_version 21602 (0.0009) [2023-10-07 20:33:38,331][67838] Updated weights for policy 0, policy_version 21612 (0.0008) [2023-10-07 20:33:38,707][67838] Updated weights for policy 0, policy_version 21622 (0.0009) [2023-10-07 20:33:39,085][67838] Updated weights for policy 0, policy_version 21632 (0.0009) [2023-10-07 20:33:42,100][67871] Updated weights for policy 1, policy_version 21670 (0.0008) [2023-10-07 20:33:42,473][67871] Updated weights for policy 1, policy_version 21680 (0.0007) [2023-10-07 20:33:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 44335104. Throughput: 0: 1676.1, 1: 1652.3. Samples: 11093346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:33:42,478][66916] Avg episode reward: [(0, '35.110'), (1, '35.030')] [2023-10-07 20:33:42,838][67871] Updated weights for policy 1, policy_version 21690 (0.0008) [2023-10-07 20:33:43,101][67838] Updated weights for policy 0, policy_version 21642 (0.0007) [2023-10-07 20:33:43,463][67838] Updated weights for policy 0, policy_version 21652 (0.0008) [2023-10-07 20:33:43,834][67838] Updated weights for policy 0, policy_version 21662 (0.0008) [2023-10-07 20:33:46,894][67871] Updated weights for policy 1, policy_version 21700 (0.0007) [2023-10-07 20:33:47,265][67871] Updated weights for policy 1, policy_version 21710 (0.0010) [2023-10-07 20:33:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 44400640. Throughput: 0: 1675.5, 1: 1657.6. Samples: 11114190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:33:47,477][66916] Avg episode reward: [(0, '35.340'), (1, '34.820')] [2023-10-07 20:33:47,635][67871] Updated weights for policy 1, policy_version 21720 (0.0010) [2023-10-07 20:33:47,926][67838] Updated weights for policy 0, policy_version 21672 (0.0008) [2023-10-07 20:33:48,298][67838] Updated weights for policy 0, policy_version 21682 (0.0011) [2023-10-07 20:33:48,676][67838] Updated weights for policy 0, policy_version 21692 (0.0009) [2023-10-07 20:33:51,749][67871] Updated weights for policy 1, policy_version 21730 (0.0008) [2023-10-07 20:33:52,113][67871] Updated weights for policy 1, policy_version 21740 (0.0009) [2023-10-07 20:33:52,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44466176. Throughput: 0: 1672.8, 1: 1647.9. Samples: 11134414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:33:52,478][66916] Avg episode reward: [(0, '36.190'), (1, '34.700')] [2023-10-07 20:33:52,485][67871] Updated weights for policy 1, policy_version 21750 (0.0008) [2023-10-07 20:33:52,603][67838] Updated weights for policy 0, policy_version 21702 (0.0008) [2023-10-07 20:33:52,850][67871] Updated weights for policy 1, policy_version 21760 (0.0008) [2023-10-07 20:33:52,982][67838] Updated weights for policy 0, policy_version 21712 (0.0010) [2023-10-07 20:33:53,352][67838] Updated weights for policy 0, policy_version 21722 (0.0009) [2023-10-07 20:33:57,001][67871] Updated weights for policy 1, policy_version 21770 (0.0007) [2023-10-07 20:33:57,380][67871] Updated weights for policy 1, policy_version 21780 (0.0007) [2023-10-07 20:33:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44531712. Throughput: 0: 1676.6, 1: 1656.2. Samples: 11143646. Policy #0 lag: (min: 31.0, avg: 41.4, max: 63.0) [2023-10-07 20:33:57,478][66916] Avg episode reward: [(0, '35.840'), (1, '33.570')] [2023-10-07 20:33:57,663][67838] Updated weights for policy 0, policy_version 21732 (0.0008) [2023-10-07 20:33:57,748][67871] Updated weights for policy 1, policy_version 21790 (0.0007) [2023-10-07 20:33:58,051][67838] Updated weights for policy 0, policy_version 21742 (0.0010) [2023-10-07 20:33:58,426][67838] Updated weights for policy 0, policy_version 21752 (0.0007) [2023-10-07 20:34:01,750][67871] Updated weights for policy 1, policy_version 21800 (0.0009) [2023-10-07 20:34:02,121][67871] Updated weights for policy 1, policy_version 21810 (0.0007) [2023-10-07 20:34:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44597248. Throughput: 0: 1671.1, 1: 1658.8. Samples: 11163912. Policy #0 lag: (min: 31.0, avg: 41.4, max: 63.0) [2023-10-07 20:34:02,477][66916] Avg episode reward: [(0, '35.130'), (1, '35.590')] [2023-10-07 20:34:02,486][67871] Updated weights for policy 1, policy_version 21820 (0.0009) [2023-10-07 20:34:02,524][67838] Updated weights for policy 0, policy_version 21762 (0.0009) [2023-10-07 20:34:02,894][67838] Updated weights for policy 0, policy_version 21772 (0.0008) [2023-10-07 20:34:03,263][67838] Updated weights for policy 0, policy_version 21782 (0.0009) [2023-10-07 20:34:03,635][67838] Updated weights for policy 0, policy_version 21792 (0.0008) [2023-10-07 20:34:06,825][67871] Updated weights for policy 1, policy_version 21830 (0.0007) [2023-10-07 20:34:07,200][67871] Updated weights for policy 1, policy_version 21840 (0.0007) [2023-10-07 20:34:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44662784. Throughput: 0: 1666.9, 1: 1656.5. Samples: 11184098. Policy #0 lag: (min: 31.0, avg: 41.4, max: 63.0) [2023-10-07 20:34:07,478][66916] Avg episode reward: [(0, '35.270'), (1, '34.530')] [2023-10-07 20:34:07,550][67838] Updated weights for policy 0, policy_version 21802 (0.0007) [2023-10-07 20:34:07,556][67871] Updated weights for policy 1, policy_version 21850 (0.0007) [2023-10-07 20:34:07,931][67838] Updated weights for policy 0, policy_version 21812 (0.0009) [2023-10-07 20:34:08,308][67838] Updated weights for policy 0, policy_version 21822 (0.0010) [2023-10-07 20:34:11,511][67871] Updated weights for policy 1, policy_version 21860 (0.0007) [2023-10-07 20:34:11,886][67871] Updated weights for policy 1, policy_version 21870 (0.0009) [2023-10-07 20:34:12,256][67871] Updated weights for policy 1, policy_version 21880 (0.0009) [2023-10-07 20:34:12,406][67838] Updated weights for policy 0, policy_version 21832 (0.0008) [2023-10-07 20:34:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44728320. Throughput: 0: 1664.8, 1: 1665.5. Samples: 11193522. Policy #0 lag: (min: 31.0, avg: 41.4, max: 63.0) [2023-10-07 20:34:12,477][66916] Avg episode reward: [(0, '32.540'), (1, '33.990')] [2023-10-07 20:34:12,783][67838] Updated weights for policy 0, policy_version 21842 (0.0007) [2023-10-07 20:34:13,156][67838] Updated weights for policy 0, policy_version 21852 (0.0010) [2023-10-07 20:34:16,407][67871] Updated weights for policy 1, policy_version 21890 (0.0007) [2023-10-07 20:34:16,783][67871] Updated weights for policy 1, policy_version 21900 (0.0010) [2023-10-07 20:34:17,163][67871] Updated weights for policy 1, policy_version 21910 (0.0008) [2023-10-07 20:34:17,415][67838] Updated weights for policy 0, policy_version 21862 (0.0009) [2023-10-07 20:34:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44793856. Throughput: 0: 1664.2, 1: 1667.0. Samples: 11213746. Policy #0 lag: (min: 13.0, avg: 20.1, max: 45.0) [2023-10-07 20:34:17,477][66916] Avg episode reward: [(0, '36.640'), (1, '34.570')] [2023-10-07 20:34:17,527][67871] Updated weights for policy 1, policy_version 21920 (0.0009) [2023-10-07 20:34:17,786][67838] Updated weights for policy 0, policy_version 21872 (0.0010) [2023-10-07 20:34:18,162][67838] Updated weights for policy 0, policy_version 21882 (0.0008) [2023-10-07 20:34:21,813][67871] Updated weights for policy 1, policy_version 21930 (0.0008) [2023-10-07 20:34:22,181][67871] Updated weights for policy 1, policy_version 21940 (0.0007) [2023-10-07 20:34:22,238][67838] Updated weights for policy 0, policy_version 21892 (0.0008) [2023-10-07 20:34:22,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44859392. Throughput: 0: 1667.5, 1: 1652.3. Samples: 11233618. Policy #0 lag: (min: 13.0, avg: 20.1, max: 45.0) [2023-10-07 20:34:22,478][66916] Avg episode reward: [(0, '34.690'), (1, '34.000')] [2023-10-07 20:34:22,548][67871] Updated weights for policy 1, policy_version 21950 (0.0007) [2023-10-07 20:34:22,612][67838] Updated weights for policy 0, policy_version 21902 (0.0007) [2023-10-07 20:34:22,987][67838] Updated weights for policy 0, policy_version 21912 (0.0011) [2023-10-07 20:34:26,664][67871] Updated weights for policy 1, policy_version 21960 (0.0008) [2023-10-07 20:34:27,024][67871] Updated weights for policy 1, policy_version 21970 (0.0007) [2023-10-07 20:34:27,193][67838] Updated weights for policy 0, policy_version 21922 (0.0011) [2023-10-07 20:34:27,403][67871] Updated weights for policy 1, policy_version 21980 (0.0008) [2023-10-07 20:34:27,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44924928. Throughput: 0: 1660.0, 1: 1664.8. Samples: 11242962. Policy #0 lag: (min: 13.0, avg: 20.1, max: 45.0) [2023-10-07 20:34:27,477][66916] Avg episode reward: [(0, '34.650'), (1, '34.340')] [2023-10-07 20:34:27,567][67838] Updated weights for policy 0, policy_version 21932 (0.0007) [2023-10-07 20:34:27,939][67838] Updated weights for policy 0, policy_version 21942 (0.0008) [2023-10-07 20:34:28,312][67838] Updated weights for policy 0, policy_version 21952 (0.0010) [2023-10-07 20:34:31,537][67871] Updated weights for policy 1, policy_version 21990 (0.0008) [2023-10-07 20:34:31,902][67871] Updated weights for policy 1, policy_version 22000 (0.0009) [2023-10-07 20:34:32,266][67871] Updated weights for policy 1, policy_version 22010 (0.0009) [2023-10-07 20:34:32,447][67838] Updated weights for policy 0, policy_version 21962 (0.0009) [2023-10-07 20:34:32,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44990464. Throughput: 0: 1655.8, 1: 1659.7. Samples: 11263390. Policy #0 lag: (min: 13.0, avg: 20.1, max: 45.0) [2023-10-07 20:34:32,477][66916] Avg episode reward: [(0, '34.200'), (1, '36.070')] [2023-10-07 20:34:32,826][67838] Updated weights for policy 0, policy_version 21972 (0.0008) [2023-10-07 20:34:33,194][67838] Updated weights for policy 0, policy_version 21982 (0.0010) [2023-10-07 20:34:36,355][67871] Updated weights for policy 1, policy_version 22020 (0.0008) [2023-10-07 20:34:36,714][67871] Updated weights for policy 1, policy_version 22030 (0.0009) [2023-10-07 20:34:37,082][67871] Updated weights for policy 1, policy_version 22040 (0.0009) [2023-10-07 20:34:37,335][67838] Updated weights for policy 0, policy_version 21992 (0.0008) [2023-10-07 20:34:37,477][66916] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 45088768. Throughput: 0: 1657.8, 1: 1653.3. Samples: 11283414. Policy #0 lag: (min: 18.0, avg: 18.4, max: 32.0) [2023-10-07 20:34:37,478][66916] Avg episode reward: [(0, '34.720'), (1, '32.580')] [2023-10-07 20:34:37,710][67838] Updated weights for policy 0, policy_version 22002 (0.0008) [2023-10-07 20:34:38,079][67838] Updated weights for policy 0, policy_version 22012 (0.0009) [2023-10-07 20:34:41,136][67871] Updated weights for policy 1, policy_version 22050 (0.0009) [2023-10-07 20:34:41,505][67871] Updated weights for policy 1, policy_version 22060 (0.0007) [2023-10-07 20:34:41,874][67871] Updated weights for policy 1, policy_version 22070 (0.0010) [2023-10-07 20:34:42,241][67871] Updated weights for policy 1, policy_version 22080 (0.0008) [2023-10-07 20:34:42,326][67838] Updated weights for policy 0, policy_version 22022 (0.0009) [2023-10-07 20:34:42,476][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 45154304. Throughput: 0: 1656.4, 1: 1661.3. Samples: 11292944. Policy #0 lag: (min: 18.0, avg: 18.4, max: 32.0) [2023-10-07 20:34:42,477][66916] Avg episode reward: [(0, '35.860'), (1, '36.720')] [2023-10-07 20:34:42,704][67838] Updated weights for policy 0, policy_version 22032 (0.0008) [2023-10-07 20:34:43,087][67838] Updated weights for policy 0, policy_version 22042 (0.0011) [2023-10-07 20:34:46,439][67871] Updated weights for policy 1, policy_version 22090 (0.0010) [2023-10-07 20:34:46,800][67871] Updated weights for policy 1, policy_version 22100 (0.0008) [2023-10-07 20:34:47,159][67871] Updated weights for policy 1, policy_version 22110 (0.0009) [2023-10-07 20:34:47,223][67838] Updated weights for policy 0, policy_version 22052 (0.0009) [2023-10-07 20:34:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 45219840. Throughput: 0: 1654.7, 1: 1660.2. Samples: 11313082. Policy #0 lag: (min: 18.0, avg: 18.4, max: 32.0) [2023-10-07 20:34:47,478][66916] Avg episode reward: [(0, '34.030'), (1, '32.570')] [2023-10-07 20:34:47,599][67838] Updated weights for policy 0, policy_version 22062 (0.0008) [2023-10-07 20:34:47,963][67838] Updated weights for policy 0, policy_version 22072 (0.0007) [2023-10-07 20:34:51,291][67871] Updated weights for policy 1, policy_version 22120 (0.0008) [2023-10-07 20:34:51,667][67871] Updated weights for policy 1, policy_version 22130 (0.0009) [2023-10-07 20:34:52,030][67871] Updated weights for policy 1, policy_version 22140 (0.0008) [2023-10-07 20:34:52,172][67838] Updated weights for policy 0, policy_version 22082 (0.0008) [2023-10-07 20:34:52,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 45285376. Throughput: 0: 1653.5, 1: 1644.4. Samples: 11332506. Policy #0 lag: (min: 18.0, avg: 18.4, max: 32.0) [2023-10-07 20:34:52,478][66916] Avg episode reward: [(0, '36.130'), (1, '34.250')] [2023-10-07 20:34:52,552][67838] Updated weights for policy 0, policy_version 22092 (0.0010) [2023-10-07 20:34:52,915][67838] Updated weights for policy 0, policy_version 22102 (0.0009) [2023-10-07 20:34:53,297][67838] Updated weights for policy 0, policy_version 22112 (0.0008) [2023-10-07 20:34:56,100][67871] Updated weights for policy 1, policy_version 22150 (0.0008) [2023-10-07 20:34:56,468][67871] Updated weights for policy 1, policy_version 22160 (0.0011) [2023-10-07 20:34:56,842][67871] Updated weights for policy 1, policy_version 22170 (0.0007) [2023-10-07 20:34:57,439][67838] Updated weights for policy 0, policy_version 22122 (0.0009) [2023-10-07 20:34:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 45350912. Throughput: 0: 1654.1, 1: 1657.2. Samples: 11342532. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-07 20:34:57,477][66916] Avg episode reward: [(0, '35.180'), (1, '35.490')] [2023-10-07 20:34:57,818][67838] Updated weights for policy 0, policy_version 22132 (0.0009) [2023-10-07 20:34:58,182][67838] Updated weights for policy 0, policy_version 22142 (0.0009) [2023-10-07 20:35:00,960][67871] Updated weights for policy 1, policy_version 22180 (0.0010) [2023-10-07 20:35:01,339][67871] Updated weights for policy 1, policy_version 22190 (0.0010) [2023-10-07 20:35:01,698][67871] Updated weights for policy 1, policy_version 22200 (0.0009) [2023-10-07 20:35:02,260][67838] Updated weights for policy 0, policy_version 22152 (0.0010) [2023-10-07 20:35:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 45416448. Throughput: 0: 1651.9, 1: 1659.0. Samples: 11362734. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-07 20:35:02,477][66916] Avg episode reward: [(0, '34.800'), (1, '33.340')] [2023-10-07 20:35:02,630][67838] Updated weights for policy 0, policy_version 22162 (0.0008) [2023-10-07 20:35:03,001][67838] Updated weights for policy 0, policy_version 22172 (0.0008) [2023-10-07 20:35:05,861][67871] Updated weights for policy 1, policy_version 22210 (0.0008) [2023-10-07 20:35:06,232][67871] Updated weights for policy 1, policy_version 22220 (0.0007) [2023-10-07 20:35:06,599][67871] Updated weights for policy 1, policy_version 22230 (0.0007) [2023-10-07 20:35:06,960][67871] Updated weights for policy 1, policy_version 22240 (0.0008) [2023-10-07 20:35:07,201][67838] Updated weights for policy 0, policy_version 22182 (0.0008) [2023-10-07 20:35:07,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 45481984. Throughput: 0: 1646.8, 1: 1652.4. Samples: 11382082. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-07 20:35:07,477][66916] Avg episode reward: [(0, '34.690'), (1, '36.670')] [2023-10-07 20:35:07,488][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000022240_22773760.pth... [2023-10-07 20:35:07,517][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000020672_21168128.pth [2023-10-07 20:35:07,583][67838] Updated weights for policy 0, policy_version 22192 (0.0008) [2023-10-07 20:35:07,962][67838] Updated weights for policy 0, policy_version 22202 (0.0007) [2023-10-07 20:35:08,181][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000022208_22740992.pth... [2023-10-07 20:35:08,220][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000020640_21135360.pth [2023-10-07 20:35:11,113][67871] Updated weights for policy 1, policy_version 22250 (0.0008) [2023-10-07 20:35:11,488][67871] Updated weights for policy 1, policy_version 22260 (0.0009) [2023-10-07 20:35:11,848][67871] Updated weights for policy 1, policy_version 22270 (0.0008) [2023-10-07 20:35:12,049][67838] Updated weights for policy 0, policy_version 22212 (0.0007) [2023-10-07 20:35:12,421][67838] Updated weights for policy 0, policy_version 22222 (0.0008) [2023-10-07 20:35:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 45547520. Throughput: 0: 1651.2, 1: 1664.3. Samples: 11392160. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-07 20:35:12,477][66916] Avg episode reward: [(0, '35.950'), (1, '35.260')] [2023-10-07 20:35:12,792][67838] Updated weights for policy 0, policy_version 22232 (0.0009) [2023-10-07 20:35:15,736][67871] Updated weights for policy 1, policy_version 22280 (0.0009) [2023-10-07 20:35:16,107][67871] Updated weights for policy 1, policy_version 22290 (0.0007) [2023-10-07 20:35:16,483][67871] Updated weights for policy 1, policy_version 22300 (0.0007) [2023-10-07 20:35:16,797][67838] Updated weights for policy 0, policy_version 22242 (0.0008) [2023-10-07 20:35:17,167][67838] Updated weights for policy 0, policy_version 22252 (0.0009) [2023-10-07 20:35:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 45613056. Throughput: 0: 1652.4, 1: 1663.2. Samples: 11412596. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-07 20:35:17,478][66916] Avg episode reward: [(0, '35.340'), (1, '34.050')] [2023-10-07 20:35:17,545][67838] Updated weights for policy 0, policy_version 22262 (0.0009) [2023-10-07 20:35:17,914][67838] Updated weights for policy 0, policy_version 22272 (0.0007) [2023-10-07 20:35:20,696][67871] Updated weights for policy 1, policy_version 22310 (0.0007) [2023-10-07 20:35:21,072][67871] Updated weights for policy 1, policy_version 22320 (0.0007) [2023-10-07 20:35:21,445][67871] Updated weights for policy 1, policy_version 22330 (0.0007) [2023-10-07 20:35:21,907][67838] Updated weights for policy 0, policy_version 22282 (0.0009) [2023-10-07 20:35:22,293][67838] Updated weights for policy 0, policy_version 22292 (0.0008) [2023-10-07 20:35:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 45678592. Throughput: 0: 1639.9, 1: 1655.5. Samples: 11431706. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-07 20:35:22,477][66916] Avg episode reward: [(0, '33.610'), (1, '34.810')] [2023-10-07 20:35:22,664][67838] Updated weights for policy 0, policy_version 22302 (0.0008) [2023-10-07 20:35:25,582][67871] Updated weights for policy 1, policy_version 22340 (0.0009) [2023-10-07 20:35:25,947][67871] Updated weights for policy 1, policy_version 22350 (0.0007) [2023-10-07 20:35:26,313][67871] Updated weights for policy 1, policy_version 22360 (0.0007) [2023-10-07 20:35:26,776][67838] Updated weights for policy 0, policy_version 22312 (0.0009) [2023-10-07 20:35:27,154][67838] Updated weights for policy 0, policy_version 22322 (0.0009) [2023-10-07 20:35:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 45744128. Throughput: 0: 1658.0, 1: 1670.1. Samples: 11442710. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-07 20:35:27,477][66916] Avg episode reward: [(0, '35.800'), (1, '33.390')] [2023-10-07 20:35:27,513][67838] Updated weights for policy 0, policy_version 22332 (0.0010) [2023-10-07 20:35:30,332][67871] Updated weights for policy 1, policy_version 22370 (0.0008) [2023-10-07 20:35:30,702][67871] Updated weights for policy 1, policy_version 22380 (0.0007) [2023-10-07 20:35:31,069][67871] Updated weights for policy 1, policy_version 22390 (0.0008) [2023-10-07 20:35:31,433][67871] Updated weights for policy 1, policy_version 22400 (0.0007) [2023-10-07 20:35:31,718][67838] Updated weights for policy 0, policy_version 22342 (0.0010) [2023-10-07 20:35:32,100][67838] Updated weights for policy 0, policy_version 22352 (0.0009) [2023-10-07 20:35:32,471][67838] Updated weights for policy 0, policy_version 22362 (0.0008) [2023-10-07 20:35:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 45809664. Throughput: 0: 1665.3, 1: 1657.1. Samples: 11462588. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-07 20:35:32,477][66916] Avg episode reward: [(0, '34.670'), (1, '34.530')] [2023-10-07 20:35:35,438][67871] Updated weights for policy 1, policy_version 22410 (0.0009) [2023-10-07 20:35:35,821][67871] Updated weights for policy 1, policy_version 22420 (0.0008) [2023-10-07 20:35:36,187][67871] Updated weights for policy 1, policy_version 22430 (0.0009) [2023-10-07 20:35:36,643][67838] Updated weights for policy 0, policy_version 22372 (0.0007) [2023-10-07 20:35:37,007][67838] Updated weights for policy 0, policy_version 22382 (0.0007) [2023-10-07 20:35:37,386][67838] Updated weights for policy 0, policy_version 22392 (0.0008) [2023-10-07 20:35:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 45875200. Throughput: 0: 1653.6, 1: 1666.2. Samples: 11481896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:35:37,478][66916] Avg episode reward: [(0, '35.720'), (1, '34.120')] [2023-10-07 20:35:40,324][67871] Updated weights for policy 1, policy_version 22440 (0.0010) [2023-10-07 20:35:40,690][67871] Updated weights for policy 1, policy_version 22450 (0.0010) [2023-10-07 20:35:41,065][67871] Updated weights for policy 1, policy_version 22460 (0.0010) [2023-10-07 20:35:41,431][67838] Updated weights for policy 0, policy_version 22402 (0.0008) [2023-10-07 20:35:41,800][67838] Updated weights for policy 0, policy_version 22412 (0.0009) [2023-10-07 20:35:42,177][67838] Updated weights for policy 0, policy_version 22422 (0.0009) [2023-10-07 20:35:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 45940736. Throughput: 0: 1665.2, 1: 1672.4. Samples: 11492724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:35:42,477][66916] Avg episode reward: [(0, '35.130'), (1, '34.580')] [2023-10-07 20:35:42,553][67838] Updated weights for policy 0, policy_version 22432 (0.0008) [2023-10-07 20:35:45,162][67871] Updated weights for policy 1, policy_version 22470 (0.0011) [2023-10-07 20:35:45,527][67871] Updated weights for policy 1, policy_version 22480 (0.0010) [2023-10-07 20:35:45,907][67871] Updated weights for policy 1, policy_version 22490 (0.0010) [2023-10-07 20:35:46,685][67838] Updated weights for policy 0, policy_version 22442 (0.0009) [2023-10-07 20:35:47,062][67838] Updated weights for policy 0, policy_version 22452 (0.0009) [2023-10-07 20:35:47,422][67838] Updated weights for policy 0, policy_version 22462 (0.0008) [2023-10-07 20:35:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 46006272. Throughput: 0: 1668.4, 1: 1652.5. Samples: 11512178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:35:47,477][66916] Avg episode reward: [(0, '36.520'), (1, '35.120')] [2023-10-07 20:35:50,157][67871] Updated weights for policy 1, policy_version 22500 (0.0008) [2023-10-07 20:35:50,521][67871] Updated weights for policy 1, policy_version 22510 (0.0007) [2023-10-07 20:35:50,893][67871] Updated weights for policy 1, policy_version 22520 (0.0009) [2023-10-07 20:35:51,551][67838] Updated weights for policy 0, policy_version 22472 (0.0008) [2023-10-07 20:35:51,928][67838] Updated weights for policy 0, policy_version 22482 (0.0007) [2023-10-07 20:35:52,303][67838] Updated weights for policy 0, policy_version 22492 (0.0008) [2023-10-07 20:35:52,476][66916] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 46104576. Throughput: 0: 1652.8, 1: 1664.8. Samples: 11531374. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-07 20:35:52,477][66916] Avg episode reward: [(0, '35.220'), (1, '34.550')] [2023-10-07 20:35:54,974][67871] Updated weights for policy 1, policy_version 22530 (0.0009) [2023-10-07 20:35:55,335][67871] Updated weights for policy 1, policy_version 22540 (0.0009) [2023-10-07 20:35:55,704][67871] Updated weights for policy 1, policy_version 22550 (0.0010) [2023-10-07 20:35:56,079][67871] Updated weights for policy 1, policy_version 22560 (0.0007) [2023-10-07 20:35:56,385][67838] Updated weights for policy 0, policy_version 22502 (0.0008) [2023-10-07 20:35:56,753][67838] Updated weights for policy 0, policy_version 22512 (0.0007) [2023-10-07 20:35:57,122][67838] Updated weights for policy 0, policy_version 22522 (0.0007) [2023-10-07 20:35:57,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 46170112. Throughput: 0: 1669.3, 1: 1668.3. Samples: 11542350. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-07 20:35:57,478][66916] Avg episode reward: [(0, '35.280'), (1, '33.510')] [2023-10-07 20:36:00,200][67871] Updated weights for policy 1, policy_version 22570 (0.0009) [2023-10-07 20:36:00,571][67871] Updated weights for policy 1, policy_version 22580 (0.0007) [2023-10-07 20:36:00,940][67871] Updated weights for policy 1, policy_version 22590 (0.0009) [2023-10-07 20:36:01,415][67838] Updated weights for policy 0, policy_version 22532 (0.0009) [2023-10-07 20:36:01,793][67838] Updated weights for policy 0, policy_version 22542 (0.0011) [2023-10-07 20:36:02,165][67838] Updated weights for policy 0, policy_version 22552 (0.0008) [2023-10-07 20:36:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 46235648. Throughput: 0: 1663.9, 1: 1650.8. Samples: 11561758. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-07 20:36:02,478][66916] Avg episode reward: [(0, '35.590'), (1, '34.260')] [2023-10-07 20:36:05,104][67871] Updated weights for policy 1, policy_version 22600 (0.0007) [2023-10-07 20:36:05,473][67871] Updated weights for policy 1, policy_version 22610 (0.0010) [2023-10-07 20:36:05,834][67871] Updated weights for policy 1, policy_version 22620 (0.0007) [2023-10-07 20:36:06,158][67838] Updated weights for policy 0, policy_version 22562 (0.0010) [2023-10-07 20:36:06,542][67838] Updated weights for policy 0, policy_version 22572 (0.0009) [2023-10-07 20:36:06,917][67838] Updated weights for policy 0, policy_version 22582 (0.0007) [2023-10-07 20:36:07,296][67838] Updated weights for policy 0, policy_version 22592 (0.0009) [2023-10-07 20:36:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 46301184. Throughput: 0: 1651.8, 1: 1667.6. Samples: 11581080. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-07 20:36:07,477][66916] Avg episode reward: [(0, '35.550'), (1, '34.260')] [2023-10-07 20:36:09,768][67871] Updated weights for policy 1, policy_version 22630 (0.0007) [2023-10-07 20:36:10,138][67871] Updated weights for policy 1, policy_version 22640 (0.0007) [2023-10-07 20:36:10,504][67871] Updated weights for policy 1, policy_version 22650 (0.0010) [2023-10-07 20:36:11,404][67838] Updated weights for policy 0, policy_version 22602 (0.0010) [2023-10-07 20:36:11,784][67838] Updated weights for policy 0, policy_version 22612 (0.0010) [2023-10-07 20:36:12,151][67838] Updated weights for policy 0, policy_version 22622 (0.0008) [2023-10-07 20:36:12,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 46366720. Throughput: 0: 1657.6, 1: 1665.2. Samples: 11592238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:36:12,478][66916] Avg episode reward: [(0, '35.690'), (1, '33.250')] [2023-10-07 20:36:14,644][67871] Updated weights for policy 1, policy_version 22660 (0.0009) [2023-10-07 20:36:15,012][67871] Updated weights for policy 1, policy_version 22670 (0.0007) [2023-10-07 20:36:15,389][67871] Updated weights for policy 1, policy_version 22680 (0.0008) [2023-10-07 20:36:16,413][67838] Updated weights for policy 0, policy_version 22632 (0.0008) [2023-10-07 20:36:16,791][67838] Updated weights for policy 0, policy_version 22642 (0.0007) [2023-10-07 20:36:17,166][67838] Updated weights for policy 0, policy_version 22652 (0.0007) [2023-10-07 20:36:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 46432256. Throughput: 0: 1653.6, 1: 1649.2. Samples: 11611218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:36:17,477][66916] Avg episode reward: [(0, '34.490'), (1, '34.980')] [2023-10-07 20:36:19,674][67871] Updated weights for policy 1, policy_version 22690 (0.0007) [2023-10-07 20:36:20,078][67871] Updated weights for policy 1, policy_version 22700 (0.0007) [2023-10-07 20:36:20,450][67871] Updated weights for policy 1, policy_version 22710 (0.0008) [2023-10-07 20:36:20,817][67871] Updated weights for policy 1, policy_version 22720 (0.0008) [2023-10-07 20:36:21,070][67838] Updated weights for policy 0, policy_version 22662 (0.0008) [2023-10-07 20:36:21,440][67838] Updated weights for policy 0, policy_version 22672 (0.0007) [2023-10-07 20:36:21,810][67838] Updated weights for policy 0, policy_version 22682 (0.0009) [2023-10-07 20:36:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 46497792. Throughput: 0: 1641.6, 1: 1658.9. Samples: 11630414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:36:22,477][66916] Avg episode reward: [(0, '36.360'), (1, '33.690')] [2023-10-07 20:36:24,809][67871] Updated weights for policy 1, policy_version 22730 (0.0008) [2023-10-07 20:36:25,175][67871] Updated weights for policy 1, policy_version 22740 (0.0007) [2023-10-07 20:36:25,539][67871] Updated weights for policy 1, policy_version 22750 (0.0009) [2023-10-07 20:36:26,010][67838] Updated weights for policy 0, policy_version 22692 (0.0008) [2023-10-07 20:36:26,381][67838] Updated weights for policy 0, policy_version 22702 (0.0009) [2023-10-07 20:36:26,749][67838] Updated weights for policy 0, policy_version 22712 (0.0007) [2023-10-07 20:36:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 46563328. Throughput: 0: 1655.5, 1: 1652.7. Samples: 11641594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:36:27,478][66916] Avg episode reward: [(0, '36.580'), (1, '33.690')] [2023-10-07 20:36:29,685][67871] Updated weights for policy 1, policy_version 22760 (0.0009) [2023-10-07 20:36:30,053][67871] Updated weights for policy 1, policy_version 22770 (0.0009) [2023-10-07 20:36:30,411][67871] Updated weights for policy 1, policy_version 22780 (0.0008) [2023-10-07 20:36:30,737][67838] Updated weights for policy 0, policy_version 22722 (0.0010) [2023-10-07 20:36:31,110][67838] Updated weights for policy 0, policy_version 22732 (0.0010) [2023-10-07 20:36:31,491][67838] Updated weights for policy 0, policy_version 22742 (0.0011) [2023-10-07 20:36:31,859][67838] Updated weights for policy 0, policy_version 22752 (0.0010) [2023-10-07 20:36:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 46628864. Throughput: 0: 1647.2, 1: 1659.2. Samples: 11660964. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-07 20:36:32,477][66916] Avg episode reward: [(0, '34.570'), (1, '34.590')] [2023-10-07 20:36:34,474][67871] Updated weights for policy 1, policy_version 22790 (0.0009) [2023-10-07 20:36:34,842][67871] Updated weights for policy 1, policy_version 22800 (0.0009) [2023-10-07 20:36:35,199][67871] Updated weights for policy 1, policy_version 22810 (0.0008) [2023-10-07 20:36:35,896][67838] Updated weights for policy 0, policy_version 22762 (0.0009) [2023-10-07 20:36:36,276][67838] Updated weights for policy 0, policy_version 22772 (0.0009) [2023-10-07 20:36:36,651][67838] Updated weights for policy 0, policy_version 22782 (0.0008) [2023-10-07 20:36:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 46694400. Throughput: 0: 1648.0, 1: 1669.5. Samples: 11680660. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-07 20:36:37,478][66916] Avg episode reward: [(0, '39.940'), (1, '34.340')] [2023-10-07 20:36:37,490][67511] Saving new best policy, reward=39.940! [2023-10-07 20:36:39,359][67871] Updated weights for policy 1, policy_version 22820 (0.0008) [2023-10-07 20:36:39,725][67871] Updated weights for policy 1, policy_version 22830 (0.0010) [2023-10-07 20:36:40,107][67871] Updated weights for policy 1, policy_version 22840 (0.0009) [2023-10-07 20:36:40,737][67838] Updated weights for policy 0, policy_version 22792 (0.0009) [2023-10-07 20:36:41,101][67838] Updated weights for policy 0, policy_version 22802 (0.0009) [2023-10-07 20:36:41,482][67838] Updated weights for policy 0, policy_version 22812 (0.0008) [2023-10-07 20:36:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 46759936. Throughput: 0: 1658.7, 1: 1655.0. Samples: 11691468. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-07 20:36:42,478][66916] Avg episode reward: [(0, '36.510'), (1, '33.860')] [2023-10-07 20:36:44,277][67871] Updated weights for policy 1, policy_version 22850 (0.0008) [2023-10-07 20:36:44,644][67871] Updated weights for policy 1, policy_version 22860 (0.0007) [2023-10-07 20:36:45,002][67871] Updated weights for policy 1, policy_version 22870 (0.0007) [2023-10-07 20:36:45,376][67871] Updated weights for policy 1, policy_version 22880 (0.0008) [2023-10-07 20:36:45,688][67838] Updated weights for policy 0, policy_version 22822 (0.0008) [2023-10-07 20:36:46,061][67838] Updated weights for policy 0, policy_version 22832 (0.0007) [2023-10-07 20:36:46,440][67838] Updated weights for policy 0, policy_version 22842 (0.0007) [2023-10-07 20:36:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 46825472. Throughput: 0: 1646.0, 1: 1663.6. Samples: 11710690. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-07 20:36:47,477][66916] Avg episode reward: [(0, '37.270'), (1, '33.970')] [2023-10-07 20:36:49,388][67871] Updated weights for policy 1, policy_version 22890 (0.0010) [2023-10-07 20:36:49,765][67871] Updated weights for policy 1, policy_version 22900 (0.0010) [2023-10-07 20:36:50,134][67871] Updated weights for policy 1, policy_version 22910 (0.0010) [2023-10-07 20:36:50,646][67838] Updated weights for policy 0, policy_version 22852 (0.0008) [2023-10-07 20:36:51,018][67838] Updated weights for policy 0, policy_version 22862 (0.0008) [2023-10-07 20:36:51,401][67838] Updated weights for policy 0, policy_version 22872 (0.0007) [2023-10-07 20:36:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 46891008. Throughput: 0: 1657.2, 1: 1666.9. Samples: 11730664. Policy #0 lag: (min: 16.0, avg: 38.9, max: 48.0) [2023-10-07 20:36:52,477][66916] Avg episode reward: [(0, '38.600'), (1, '33.570')] [2023-10-07 20:36:54,297][67871] Updated weights for policy 1, policy_version 22920 (0.0010) [2023-10-07 20:36:54,666][67871] Updated weights for policy 1, policy_version 22930 (0.0011) [2023-10-07 20:36:55,031][67871] Updated weights for policy 1, policy_version 22940 (0.0010) [2023-10-07 20:36:55,239][67838] Updated weights for policy 0, policy_version 22882 (0.0007) [2023-10-07 20:36:55,611][67838] Updated weights for policy 0, policy_version 22892 (0.0008) [2023-10-07 20:36:55,994][67838] Updated weights for policy 0, policy_version 22902 (0.0007) [2023-10-07 20:36:56,359][67838] Updated weights for policy 0, policy_version 22912 (0.0010) [2023-10-07 20:36:57,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 46956544. Throughput: 0: 1668.0, 1: 1650.1. Samples: 11741554. Policy #0 lag: (min: 16.0, avg: 38.9, max: 48.0) [2023-10-07 20:36:57,478][66916] Avg episode reward: [(0, '34.810'), (1, '33.740')] [2023-10-07 20:36:59,354][67871] Updated weights for policy 1, policy_version 22950 (0.0011) [2023-10-07 20:36:59,721][67871] Updated weights for policy 1, policy_version 22960 (0.0009) [2023-10-07 20:37:00,085][67871] Updated weights for policy 1, policy_version 22970 (0.0009) [2023-10-07 20:37:00,668][67838] Updated weights for policy 0, policy_version 22922 (0.0008) [2023-10-07 20:37:01,048][67838] Updated weights for policy 0, policy_version 22932 (0.0009) [2023-10-07 20:37:01,413][67838] Updated weights for policy 0, policy_version 22942 (0.0007) [2023-10-07 20:37:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 47022080. Throughput: 0: 1654.8, 1: 1666.8. Samples: 11760686. Policy #0 lag: (min: 16.0, avg: 38.9, max: 48.0) [2023-10-07 20:37:02,477][66916] Avg episode reward: [(0, '36.750'), (1, '35.380')] [2023-10-07 20:37:03,998][67871] Updated weights for policy 1, policy_version 22980 (0.0009) [2023-10-07 20:37:04,370][67871] Updated weights for policy 1, policy_version 22990 (0.0009) [2023-10-07 20:37:04,752][67871] Updated weights for policy 1, policy_version 23000 (0.0008) [2023-10-07 20:37:05,533][67838] Updated weights for policy 0, policy_version 22952 (0.0007) [2023-10-07 20:37:05,912][67838] Updated weights for policy 0, policy_version 22962 (0.0009) [2023-10-07 20:37:06,282][67838] Updated weights for policy 0, policy_version 22972 (0.0008) [2023-10-07 20:37:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 47087616. Throughput: 0: 1666.1, 1: 1672.7. Samples: 11780662. Policy #0 lag: (min: 16.0, avg: 38.9, max: 48.0) [2023-10-07 20:37:07,477][66916] Avg episode reward: [(0, '33.920'), (1, '34.590')] [2023-10-07 20:37:07,485][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000023008_23560192.pth... [2023-10-07 20:37:07,485][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000022976_23527424.pth... [2023-10-07 20:37:07,524][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000021408_21921792.pth [2023-10-07 20:37:07,525][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000021472_21987328.pth [2023-10-07 20:37:07,529][67676] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p1/milestones/checkpoint_000023008_23560192.pth [2023-10-07 20:37:07,530][67511] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p0/milestones/checkpoint_000022976_23527424.pth [2023-10-07 20:37:08,888][67871] Updated weights for policy 1, policy_version 23010 (0.0008) [2023-10-07 20:37:09,294][67871] Updated weights for policy 1, policy_version 23020 (0.0007) [2023-10-07 20:37:09,667][67871] Updated weights for policy 1, policy_version 23030 (0.0007) [2023-10-07 20:37:10,032][67871] Updated weights for policy 1, policy_version 23040 (0.0007) [2023-10-07 20:37:10,368][67838] Updated weights for policy 0, policy_version 22982 (0.0010) [2023-10-07 20:37:10,744][67838] Updated weights for policy 0, policy_version 22992 (0.0008) [2023-10-07 20:37:11,127][67838] Updated weights for policy 0, policy_version 23002 (0.0008) [2023-10-07 20:37:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 47153152. Throughput: 0: 1667.8, 1: 1654.4. Samples: 11791096. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 20:37:12,478][66916] Avg episode reward: [(0, '36.600'), (1, '34.980')] [2023-10-07 20:37:14,078][67871] Updated weights for policy 1, policy_version 23050 (0.0009) [2023-10-07 20:37:14,455][67871] Updated weights for policy 1, policy_version 23060 (0.0008) [2023-10-07 20:37:14,816][67871] Updated weights for policy 1, policy_version 23070 (0.0009) [2023-10-07 20:37:15,270][67838] Updated weights for policy 0, policy_version 23012 (0.0008) [2023-10-07 20:37:15,634][67838] Updated weights for policy 0, policy_version 23022 (0.0008) [2023-10-07 20:37:16,012][67838] Updated weights for policy 0, policy_version 23032 (0.0009) [2023-10-07 20:37:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 47218688. Throughput: 0: 1655.9, 1: 1664.8. Samples: 11810394. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 20:37:17,478][66916] Avg episode reward: [(0, '36.340'), (1, '33.800')] [2023-10-07 20:37:18,999][67871] Updated weights for policy 1, policy_version 23080 (0.0009) [2023-10-07 20:37:19,368][67871] Updated weights for policy 1, policy_version 23090 (0.0008) [2023-10-07 20:37:19,734][67871] Updated weights for policy 1, policy_version 23100 (0.0008) [2023-10-07 20:37:20,042][67838] Updated weights for policy 0, policy_version 23042 (0.0008) [2023-10-07 20:37:20,422][67838] Updated weights for policy 0, policy_version 23052 (0.0009) [2023-10-07 20:37:20,785][67838] Updated weights for policy 0, policy_version 23062 (0.0008) [2023-10-07 20:37:21,160][67838] Updated weights for policy 0, policy_version 23072 (0.0008) [2023-10-07 20:37:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 47284224. Throughput: 0: 1667.8, 1: 1668.1. Samples: 11830776. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 20:37:22,478][66916] Avg episode reward: [(0, '36.240'), (1, '33.850')] [2023-10-07 20:37:23,739][67871] Updated weights for policy 1, policy_version 23110 (0.0008) [2023-10-07 20:37:24,114][67871] Updated weights for policy 1, policy_version 23120 (0.0007) [2023-10-07 20:37:24,482][67871] Updated weights for policy 1, policy_version 23130 (0.0007) [2023-10-07 20:37:25,147][67838] Updated weights for policy 0, policy_version 23082 (0.0010) [2023-10-07 20:37:25,529][67838] Updated weights for policy 0, policy_version 23092 (0.0009) [2023-10-07 20:37:25,896][67838] Updated weights for policy 0, policy_version 23102 (0.0008) [2023-10-07 20:37:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 47349760. Throughput: 0: 1661.3, 1: 1654.2. Samples: 11840666. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 20:37:27,477][66916] Avg episode reward: [(0, '36.880'), (1, '31.850')] [2023-10-07 20:37:28,613][67871] Updated weights for policy 1, policy_version 23140 (0.0008) [2023-10-07 20:37:28,988][67871] Updated weights for policy 1, policy_version 23150 (0.0012) [2023-10-07 20:37:29,350][67871] Updated weights for policy 1, policy_version 23160 (0.0008) [2023-10-07 20:37:29,983][67838] Updated weights for policy 0, policy_version 23112 (0.0008) [2023-10-07 20:37:30,356][67838] Updated weights for policy 0, policy_version 23122 (0.0009) [2023-10-07 20:37:30,732][67838] Updated weights for policy 0, policy_version 23132 (0.0009) [2023-10-07 20:37:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 47415296. Throughput: 0: 1652.3, 1: 1664.4. Samples: 11859942. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-07 20:37:32,477][66916] Avg episode reward: [(0, '35.390'), (1, '35.530')] [2023-10-07 20:37:33,478][67871] Updated weights for policy 1, policy_version 23170 (0.0009) [2023-10-07 20:37:33,847][67871] Updated weights for policy 1, policy_version 23180 (0.0008) [2023-10-07 20:37:34,220][67871] Updated weights for policy 1, policy_version 23190 (0.0007) [2023-10-07 20:37:34,590][67871] Updated weights for policy 1, policy_version 23200 (0.0007) [2023-10-07 20:37:34,877][67838] Updated weights for policy 0, policy_version 23142 (0.0009) [2023-10-07 20:37:35,252][67838] Updated weights for policy 0, policy_version 23152 (0.0008) [2023-10-07 20:37:35,623][67838] Updated weights for policy 0, policy_version 23162 (0.0012) [2023-10-07 20:37:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 47480832. Throughput: 0: 1664.0, 1: 1664.2. Samples: 11880434. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-07 20:37:37,478][66916] Avg episode reward: [(0, '33.990'), (1, '34.020')] [2023-10-07 20:37:38,515][67871] Updated weights for policy 1, policy_version 23210 (0.0008) [2023-10-07 20:37:38,880][67871] Updated weights for policy 1, policy_version 23220 (0.0007) [2023-10-07 20:37:39,256][67871] Updated weights for policy 1, policy_version 23230 (0.0007) [2023-10-07 20:37:39,877][67838] Updated weights for policy 0, policy_version 23172 (0.0009) [2023-10-07 20:37:40,242][67838] Updated weights for policy 0, policy_version 23182 (0.0008) [2023-10-07 20:37:40,621][67838] Updated weights for policy 0, policy_version 23192 (0.0008) [2023-10-07 20:37:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 47546368. Throughput: 0: 1649.4, 1: 1661.0. Samples: 11890524. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-07 20:37:42,478][66916] Avg episode reward: [(0, '35.450'), (1, '36.960')] [2023-10-07 20:37:42,479][67676] Saving new best policy, reward=36.960! [2023-10-07 20:37:43,427][67871] Updated weights for policy 1, policy_version 23240 (0.0007) [2023-10-07 20:37:43,795][67871] Updated weights for policy 1, policy_version 23250 (0.0008) [2023-10-07 20:37:44,161][67871] Updated weights for policy 1, policy_version 23260 (0.0009) [2023-10-07 20:37:44,915][67838] Updated weights for policy 0, policy_version 23202 (0.0009) [2023-10-07 20:37:45,290][67838] Updated weights for policy 0, policy_version 23212 (0.0009) [2023-10-07 20:37:45,657][67838] Updated weights for policy 0, policy_version 23222 (0.0009) [2023-10-07 20:37:46,044][67838] Updated weights for policy 0, policy_version 23232 (0.0010) [2023-10-07 20:37:47,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 47611904. Throughput: 0: 1643.2, 1: 1676.8. Samples: 11910090. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-07 20:37:47,478][66916] Avg episode reward: [(0, '34.360'), (1, '35.350')] [2023-10-07 20:37:48,251][67871] Updated weights for policy 1, policy_version 23270 (0.0009) [2023-10-07 20:37:48,613][67871] Updated weights for policy 1, policy_version 23280 (0.0008) [2023-10-07 20:37:48,978][67871] Updated weights for policy 1, policy_version 23290 (0.0008) [2023-10-07 20:37:50,213][67838] Updated weights for policy 0, policy_version 23242 (0.0008) [2023-10-07 20:37:50,586][67838] Updated weights for policy 0, policy_version 23252 (0.0008) [2023-10-07 20:37:50,954][67838] Updated weights for policy 0, policy_version 23262 (0.0012) [2023-10-07 20:37:52,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 47677440. Throughput: 0: 1651.6, 1: 1677.8. Samples: 11930484. Policy #0 lag: (min: 0.0, avg: 27.0, max: 32.0) [2023-10-07 20:37:52,478][66916] Avg episode reward: [(0, '33.730'), (1, '35.840')] [2023-10-07 20:37:53,185][67871] Updated weights for policy 1, policy_version 23300 (0.0010) [2023-10-07 20:37:53,548][67871] Updated weights for policy 1, policy_version 23310 (0.0007) [2023-10-07 20:37:53,924][67871] Updated weights for policy 1, policy_version 23320 (0.0009) [2023-10-07 20:37:55,066][67838] Updated weights for policy 0, policy_version 23272 (0.0009) [2023-10-07 20:37:55,432][67838] Updated weights for policy 0, policy_version 23282 (0.0008) [2023-10-07 20:37:55,805][67838] Updated weights for policy 0, policy_version 23292 (0.0009) [2023-10-07 20:37:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 47742976. Throughput: 0: 1648.2, 1: 1673.6. Samples: 11940576. Policy #0 lag: (min: 0.0, avg: 27.0, max: 32.0) [2023-10-07 20:37:57,477][66916] Avg episode reward: [(0, '35.930'), (1, '34.140')] [2023-10-07 20:37:58,046][67871] Updated weights for policy 1, policy_version 23330 (0.0009) [2023-10-07 20:37:58,472][67871] Updated weights for policy 1, policy_version 23340 (0.0007) [2023-10-07 20:37:58,837][67871] Updated weights for policy 1, policy_version 23350 (0.0007) [2023-10-07 20:37:59,211][67871] Updated weights for policy 1, policy_version 23360 (0.0007) [2023-10-07 20:37:59,785][67838] Updated weights for policy 0, policy_version 23302 (0.0009) [2023-10-07 20:38:00,166][67838] Updated weights for policy 0, policy_version 23312 (0.0010) [2023-10-07 20:38:00,543][67838] Updated weights for policy 0, policy_version 23322 (0.0010) [2023-10-07 20:38:02,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 47808512. Throughput: 0: 1647.9, 1: 1680.4. Samples: 11960168. Policy #0 lag: (min: 0.0, avg: 27.0, max: 32.0) [2023-10-07 20:38:02,478][66916] Avg episode reward: [(0, '34.910'), (1, '33.770')] [2023-10-07 20:38:03,017][67871] Updated weights for policy 1, policy_version 23370 (0.0011) [2023-10-07 20:38:03,394][67871] Updated weights for policy 1, policy_version 23380 (0.0011) [2023-10-07 20:38:03,764][67871] Updated weights for policy 1, policy_version 23390 (0.0009) [2023-10-07 20:38:04,547][67838] Updated weights for policy 0, policy_version 23332 (0.0010) [2023-10-07 20:38:04,924][67838] Updated weights for policy 0, policy_version 23342 (0.0008) [2023-10-07 20:38:05,297][67838] Updated weights for policy 0, policy_version 23352 (0.0008) [2023-10-07 20:38:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 47874048. Throughput: 0: 1657.0, 1: 1670.6. Samples: 11980520. Policy #0 lag: (min: 0.0, avg: 27.0, max: 32.0) [2023-10-07 20:38:07,478][66916] Avg episode reward: [(0, '34.150'), (1, '35.650')] [2023-10-07 20:38:08,072][67871] Updated weights for policy 1, policy_version 23400 (0.0008) [2023-10-07 20:38:08,435][67871] Updated weights for policy 1, policy_version 23410 (0.0009) [2023-10-07 20:38:08,799][67871] Updated weights for policy 1, policy_version 23420 (0.0011) [2023-10-07 20:38:09,363][67838] Updated weights for policy 0, policy_version 23362 (0.0008) [2023-10-07 20:38:09,734][67838] Updated weights for policy 0, policy_version 23372 (0.0009) [2023-10-07 20:38:10,111][67838] Updated weights for policy 0, policy_version 23382 (0.0007) [2023-10-07 20:38:10,480][67838] Updated weights for policy 0, policy_version 23392 (0.0010) [2023-10-07 20:38:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 47939584. Throughput: 0: 1644.7, 1: 1668.6. Samples: 11989766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:38:12,477][66916] Avg episode reward: [(0, '35.210'), (1, '33.830')] [2023-10-07 20:38:12,876][67871] Updated weights for policy 1, policy_version 23430 (0.0008) [2023-10-07 20:38:13,246][67871] Updated weights for policy 1, policy_version 23440 (0.0009) [2023-10-07 20:38:13,625][67871] Updated weights for policy 1, policy_version 23450 (0.0008) [2023-10-07 20:38:14,596][67838] Updated weights for policy 0, policy_version 23402 (0.0007) [2023-10-07 20:38:14,979][67838] Updated weights for policy 0, policy_version 23412 (0.0007) [2023-10-07 20:38:15,356][67838] Updated weights for policy 0, policy_version 23422 (0.0009) [2023-10-07 20:38:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48005120. Throughput: 0: 1659.9, 1: 1674.3. Samples: 12009978. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:38:17,477][66916] Avg episode reward: [(0, '33.890'), (1, '35.240')] [2023-10-07 20:38:17,658][67871] Updated weights for policy 1, policy_version 23460 (0.0007) [2023-10-07 20:38:18,020][67871] Updated weights for policy 1, policy_version 23470 (0.0010) [2023-10-07 20:38:18,394][67871] Updated weights for policy 1, policy_version 23480 (0.0008) [2023-10-07 20:38:19,401][67838] Updated weights for policy 0, policy_version 23432 (0.0010) [2023-10-07 20:38:19,775][67838] Updated weights for policy 0, policy_version 23442 (0.0010) [2023-10-07 20:38:20,154][67838] Updated weights for policy 0, policy_version 23452 (0.0009) [2023-10-07 20:38:22,428][67871] Updated weights for policy 1, policy_version 23490 (0.0007) [2023-10-07 20:38:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 48070656. Throughput: 0: 1658.0, 1: 1679.2. Samples: 12030608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:38:22,477][66916] Avg episode reward: [(0, '36.110'), (1, '33.950')] [2023-10-07 20:38:22,801][67871] Updated weights for policy 1, policy_version 23500 (0.0009) [2023-10-07 20:38:23,167][67871] Updated weights for policy 1, policy_version 23510 (0.0009) [2023-10-07 20:38:23,544][67871] Updated weights for policy 1, policy_version 23520 (0.0009) [2023-10-07 20:38:24,331][67838] Updated weights for policy 0, policy_version 23462 (0.0007) [2023-10-07 20:38:24,702][67838] Updated weights for policy 0, policy_version 23472 (0.0007) [2023-10-07 20:38:25,080][67838] Updated weights for policy 0, policy_version 23482 (0.0010) [2023-10-07 20:38:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 48136192. Throughput: 0: 1650.5, 1: 1671.9. Samples: 12040032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:38:27,478][66916] Avg episode reward: [(0, '36.850'), (1, '34.220')] [2023-10-07 20:38:27,667][67871] Updated weights for policy 1, policy_version 23530 (0.0010) [2023-10-07 20:38:28,032][67871] Updated weights for policy 1, policy_version 23540 (0.0010) [2023-10-07 20:38:28,418][67871] Updated weights for policy 1, policy_version 23550 (0.0011) [2023-10-07 20:38:29,132][67838] Updated weights for policy 0, policy_version 23492 (0.0008) [2023-10-07 20:38:29,506][67838] Updated weights for policy 0, policy_version 23502 (0.0008) [2023-10-07 20:38:29,884][67838] Updated weights for policy 0, policy_version 23512 (0.0010) [2023-10-07 20:38:32,470][67871] Updated weights for policy 1, policy_version 23560 (0.0008) [2023-10-07 20:38:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48201728. Throughput: 0: 1669.1, 1: 1669.3. Samples: 12060318. Policy #0 lag: (min: 8.0, avg: 23.9, max: 40.0) [2023-10-07 20:38:32,477][66916] Avg episode reward: [(0, '36.210'), (1, '36.050')] [2023-10-07 20:38:32,835][67871] Updated weights for policy 1, policy_version 23570 (0.0007) [2023-10-07 20:38:33,205][67871] Updated weights for policy 1, policy_version 23580 (0.0007) [2023-10-07 20:38:33,923][67838] Updated weights for policy 0, policy_version 23522 (0.0010) [2023-10-07 20:38:34,303][67838] Updated weights for policy 0, policy_version 23532 (0.0009) [2023-10-07 20:38:34,674][67838] Updated weights for policy 0, policy_version 23542 (0.0007) [2023-10-07 20:38:35,048][67838] Updated weights for policy 0, policy_version 23552 (0.0007) [2023-10-07 20:38:37,441][67871] Updated weights for policy 1, policy_version 23590 (0.0008) [2023-10-07 20:38:37,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 48267264. Throughput: 0: 1677.1, 1: 1666.9. Samples: 12080964. Policy #0 lag: (min: 8.0, avg: 23.9, max: 40.0) [2023-10-07 20:38:37,477][66916] Avg episode reward: [(0, '36.000'), (1, '34.040')] [2023-10-07 20:38:37,804][67871] Updated weights for policy 1, policy_version 23600 (0.0009) [2023-10-07 20:38:38,173][67871] Updated weights for policy 1, policy_version 23610 (0.0009) [2023-10-07 20:38:39,282][67838] Updated weights for policy 0, policy_version 23562 (0.0008) [2023-10-07 20:38:39,648][67838] Updated weights for policy 0, policy_version 23572 (0.0008) [2023-10-07 20:38:40,026][67838] Updated weights for policy 0, policy_version 23582 (0.0008) [2023-10-07 20:38:42,310][67871] Updated weights for policy 1, policy_version 23620 (0.0009) [2023-10-07 20:38:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48332800. Throughput: 0: 1653.5, 1: 1672.3. Samples: 12090238. Policy #0 lag: (min: 8.0, avg: 23.9, max: 40.0) [2023-10-07 20:38:42,477][66916] Avg episode reward: [(0, '34.210'), (1, '35.370')] [2023-10-07 20:38:42,688][67871] Updated weights for policy 1, policy_version 23630 (0.0008) [2023-10-07 20:38:43,045][67871] Updated weights for policy 1, policy_version 23640 (0.0008) [2023-10-07 20:38:44,241][67838] Updated weights for policy 0, policy_version 23592 (0.0007) [2023-10-07 20:38:44,603][67838] Updated weights for policy 0, policy_version 23602 (0.0008) [2023-10-07 20:38:44,978][67838] Updated weights for policy 0, policy_version 23612 (0.0008) [2023-10-07 20:38:47,281][67871] Updated weights for policy 1, policy_version 23650 (0.0007) [2023-10-07 20:38:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48398336. Throughput: 0: 1673.5, 1: 1668.9. Samples: 12110576. Policy #0 lag: (min: 8.0, avg: 23.9, max: 40.0) [2023-10-07 20:38:47,478][66916] Avg episode reward: [(0, '35.990'), (1, '35.880')] [2023-10-07 20:38:47,703][67871] Updated weights for policy 1, policy_version 23660 (0.0009) [2023-10-07 20:38:48,061][67871] Updated weights for policy 1, policy_version 23670 (0.0009) [2023-10-07 20:38:48,433][67871] Updated weights for policy 1, policy_version 23680 (0.0011) [2023-10-07 20:38:49,021][67838] Updated weights for policy 0, policy_version 23622 (0.0010) [2023-10-07 20:38:49,397][67838] Updated weights for policy 0, policy_version 23632 (0.0010) [2023-10-07 20:38:49,776][67838] Updated weights for policy 0, policy_version 23642 (0.0010) [2023-10-07 20:38:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48463872. Throughput: 0: 1673.2, 1: 1666.8. Samples: 12130816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:38:52,477][66916] Avg episode reward: [(0, '35.810'), (1, '35.690')] [2023-10-07 20:38:52,486][67871] Updated weights for policy 1, policy_version 23690 (0.0007) [2023-10-07 20:38:52,852][67871] Updated weights for policy 1, policy_version 23700 (0.0007) [2023-10-07 20:38:53,227][67871] Updated weights for policy 1, policy_version 23710 (0.0007) [2023-10-07 20:38:53,774][67838] Updated weights for policy 0, policy_version 23652 (0.0007) [2023-10-07 20:38:54,149][67838] Updated weights for policy 0, policy_version 23662 (0.0007) [2023-10-07 20:38:54,526][67838] Updated weights for policy 0, policy_version 23672 (0.0007) [2023-10-07 20:38:57,261][67871] Updated weights for policy 1, policy_version 23720 (0.0009) [2023-10-07 20:38:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48529408. Throughput: 0: 1665.4, 1: 1671.4. Samples: 12139920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:38:57,477][66916] Avg episode reward: [(0, '34.870'), (1, '34.120')] [2023-10-07 20:38:57,632][67871] Updated weights for policy 1, policy_version 23730 (0.0008) [2023-10-07 20:38:57,999][67871] Updated weights for policy 1, policy_version 23740 (0.0007) [2023-10-07 20:38:58,602][67838] Updated weights for policy 0, policy_version 23682 (0.0007) [2023-10-07 20:38:58,973][67838] Updated weights for policy 0, policy_version 23692 (0.0009) [2023-10-07 20:38:59,350][67838] Updated weights for policy 0, policy_version 23702 (0.0009) [2023-10-07 20:38:59,715][67838] Updated weights for policy 0, policy_version 23712 (0.0009) [2023-10-07 20:39:02,228][67871] Updated weights for policy 1, policy_version 23750 (0.0007) [2023-10-07 20:39:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48594944. Throughput: 0: 1672.4, 1: 1665.2. Samples: 12160174. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:39:02,477][66916] Avg episode reward: [(0, '34.370'), (1, '35.580')] [2023-10-07 20:39:02,600][67871] Updated weights for policy 1, policy_version 23760 (0.0009) [2023-10-07 20:39:02,982][67871] Updated weights for policy 1, policy_version 23770 (0.0007) [2023-10-07 20:39:03,944][67838] Updated weights for policy 0, policy_version 23722 (0.0007) [2023-10-07 20:39:04,327][67838] Updated weights for policy 0, policy_version 23732 (0.0007) [2023-10-07 20:39:04,696][67838] Updated weights for policy 0, policy_version 23742 (0.0007) [2023-10-07 20:39:07,174][67871] Updated weights for policy 1, policy_version 23780 (0.0007) [2023-10-07 20:39:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 48660480. Throughput: 0: 1674.5, 1: 1659.2. Samples: 12180628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:39:07,478][66916] Avg episode reward: [(0, '36.440'), (1, '33.690')] [2023-10-07 20:39:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000023744_24313856.pth... [2023-10-07 20:39:07,527][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000022208_22740992.pth [2023-10-07 20:39:07,541][67871] Updated weights for policy 1, policy_version 23790 (0.0007) [2023-10-07 20:39:07,912][67871] Updated weights for policy 1, policy_version 23800 (0.0009) [2023-10-07 20:39:08,197][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000023808_24379392.pth... [2023-10-07 20:39:08,225][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000022240_22773760.pth [2023-10-07 20:39:08,775][67838] Updated weights for policy 0, policy_version 23752 (0.0008) [2023-10-07 20:39:09,151][67838] Updated weights for policy 0, policy_version 23762 (0.0009) [2023-10-07 20:39:09,531][67838] Updated weights for policy 0, policy_version 23772 (0.0008) [2023-10-07 20:39:12,030][67871] Updated weights for policy 1, policy_version 23810 (0.0008) [2023-10-07 20:39:12,392][67871] Updated weights for policy 1, policy_version 23820 (0.0007) [2023-10-07 20:39:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48726016. Throughput: 0: 1662.8, 1: 1662.9. Samples: 12189686. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 20:39:12,477][66916] Avg episode reward: [(0, '34.320'), (1, '33.300')] [2023-10-07 20:39:12,756][67871] Updated weights for policy 1, policy_version 23830 (0.0010) [2023-10-07 20:39:13,124][67871] Updated weights for policy 1, policy_version 23840 (0.0007) [2023-10-07 20:39:13,525][67838] Updated weights for policy 0, policy_version 23782 (0.0009) [2023-10-07 20:39:13,904][67838] Updated weights for policy 0, policy_version 23792 (0.0009) [2023-10-07 20:39:14,263][67838] Updated weights for policy 0, policy_version 23802 (0.0007) [2023-10-07 20:39:17,328][67871] Updated weights for policy 1, policy_version 23850 (0.0009) [2023-10-07 20:39:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 48791552. Throughput: 0: 1671.1, 1: 1655.2. Samples: 12210002. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 20:39:17,478][66916] Avg episode reward: [(0, '35.710'), (1, '33.690')] [2023-10-07 20:39:17,694][67871] Updated weights for policy 1, policy_version 23860 (0.0009) [2023-10-07 20:39:18,057][67871] Updated weights for policy 1, policy_version 23870 (0.0008) [2023-10-07 20:39:18,273][67838] Updated weights for policy 0, policy_version 23812 (0.0008) [2023-10-07 20:39:18,657][67838] Updated weights for policy 0, policy_version 23822 (0.0010) [2023-10-07 20:39:19,035][67838] Updated weights for policy 0, policy_version 23832 (0.0009) [2023-10-07 20:39:22,186][67871] Updated weights for policy 1, policy_version 23880 (0.0008) [2023-10-07 20:39:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48857088. Throughput: 0: 1668.0, 1: 1654.0. Samples: 12230452. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 20:39:22,477][66916] Avg episode reward: [(0, '35.430'), (1, '32.890')] [2023-10-07 20:39:22,555][67871] Updated weights for policy 1, policy_version 23890 (0.0007) [2023-10-07 20:39:22,913][67871] Updated weights for policy 1, policy_version 23900 (0.0008) [2023-10-07 20:39:23,176][67838] Updated weights for policy 0, policy_version 23842 (0.0007) [2023-10-07 20:39:23,550][67838] Updated weights for policy 0, policy_version 23852 (0.0008) [2023-10-07 20:39:23,927][67838] Updated weights for policy 0, policy_version 23862 (0.0008) [2023-10-07 20:39:24,291][67838] Updated weights for policy 0, policy_version 23872 (0.0009) [2023-10-07 20:39:27,151][67871] Updated weights for policy 1, policy_version 23910 (0.0008) [2023-10-07 20:39:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48922624. Throughput: 0: 1671.2, 1: 1649.3. Samples: 12239660. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 20:39:27,477][66916] Avg episode reward: [(0, '33.250'), (1, '34.600')] [2023-10-07 20:39:27,520][67871] Updated weights for policy 1, policy_version 23920 (0.0007) [2023-10-07 20:39:27,881][67871] Updated weights for policy 1, policy_version 23930 (0.0008) [2023-10-07 20:39:28,375][67838] Updated weights for policy 0, policy_version 23882 (0.0007) [2023-10-07 20:39:28,756][67838] Updated weights for policy 0, policy_version 23892 (0.0010) [2023-10-07 20:39:29,121][67838] Updated weights for policy 0, policy_version 23902 (0.0012) [2023-10-07 20:39:32,257][67871] Updated weights for policy 1, policy_version 23940 (0.0009) [2023-10-07 20:39:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 48988160. Throughput: 0: 1667.6, 1: 1647.0. Samples: 12259730. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 20:39:32,478][66916] Avg episode reward: [(0, '36.340'), (1, '34.020')] [2023-10-07 20:39:32,656][67871] Updated weights for policy 1, policy_version 23950 (0.0009) [2023-10-07 20:39:33,025][67871] Updated weights for policy 1, policy_version 23960 (0.0009) [2023-10-07 20:39:33,365][67838] Updated weights for policy 0, policy_version 23912 (0.0009) [2023-10-07 20:39:33,732][67838] Updated weights for policy 0, policy_version 23922 (0.0008) [2023-10-07 20:39:34,108][67838] Updated weights for policy 0, policy_version 23932 (0.0011) [2023-10-07 20:39:36,953][67871] Updated weights for policy 1, policy_version 23970 (0.0009) [2023-10-07 20:39:37,334][67871] Updated weights for policy 1, policy_version 23980 (0.0009) [2023-10-07 20:39:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 49053696. Throughput: 0: 1665.6, 1: 1648.3. Samples: 12279940. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 20:39:37,477][66916] Avg episode reward: [(0, '33.990'), (1, '33.460')] [2023-10-07 20:39:37,699][67871] Updated weights for policy 1, policy_version 23990 (0.0008) [2023-10-07 20:39:38,061][67871] Updated weights for policy 1, policy_version 24000 (0.0008) [2023-10-07 20:39:38,130][67838] Updated weights for policy 0, policy_version 23942 (0.0010) [2023-10-07 20:39:38,511][67838] Updated weights for policy 0, policy_version 23952 (0.0010) [2023-10-07 20:39:38,878][67838] Updated weights for policy 0, policy_version 23962 (0.0010) [2023-10-07 20:39:42,235][67871] Updated weights for policy 1, policy_version 24010 (0.0009) [2023-10-07 20:39:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 49119232. Throughput: 0: 1663.6, 1: 1646.8. Samples: 12288892. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 20:39:42,478][66916] Avg episode reward: [(0, '34.610'), (1, '34.470')] [2023-10-07 20:39:42,597][67871] Updated weights for policy 1, policy_version 24020 (0.0010) [2023-10-07 20:39:42,970][67838] Updated weights for policy 0, policy_version 23972 (0.0009) [2023-10-07 20:39:42,975][67871] Updated weights for policy 1, policy_version 24030 (0.0008) [2023-10-07 20:39:43,333][67838] Updated weights for policy 0, policy_version 23982 (0.0010) [2023-10-07 20:39:43,703][67838] Updated weights for policy 0, policy_version 23992 (0.0009) [2023-10-07 20:39:47,070][67871] Updated weights for policy 1, policy_version 24040 (0.0008) [2023-10-07 20:39:47,442][67871] Updated weights for policy 1, policy_version 24050 (0.0008) [2023-10-07 20:39:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 49184768. Throughput: 0: 1665.5, 1: 1649.6. Samples: 12309352. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 20:39:47,477][66916] Avg episode reward: [(0, '35.870'), (1, '33.730')] [2023-10-07 20:39:47,805][67871] Updated weights for policy 1, policy_version 24060 (0.0009) [2023-10-07 20:39:47,863][67838] Updated weights for policy 0, policy_version 24002 (0.0008) [2023-10-07 20:39:48,236][67838] Updated weights for policy 0, policy_version 24012 (0.0008) [2023-10-07 20:39:48,614][67838] Updated weights for policy 0, policy_version 24022 (0.0010) [2023-10-07 20:39:48,989][67838] Updated weights for policy 0, policy_version 24032 (0.0010) [2023-10-07 20:39:52,002][67871] Updated weights for policy 1, policy_version 24070 (0.0009) [2023-10-07 20:39:52,376][67871] Updated weights for policy 1, policy_version 24080 (0.0007) [2023-10-07 20:39:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 49250304. Throughput: 0: 1660.4, 1: 1647.0. Samples: 12329462. Policy #0 lag: (min: 21.0, avg: 24.0, max: 53.0) [2023-10-07 20:39:52,477][66916] Avg episode reward: [(0, '35.090'), (1, '33.080')] [2023-10-07 20:39:52,751][67871] Updated weights for policy 1, policy_version 24090 (0.0007) [2023-10-07 20:39:53,145][67838] Updated weights for policy 0, policy_version 24042 (0.0007) [2023-10-07 20:39:53,523][67838] Updated weights for policy 0, policy_version 24052 (0.0008) [2023-10-07 20:39:53,890][67838] Updated weights for policy 0, policy_version 24062 (0.0007) [2023-10-07 20:39:56,657][67871] Updated weights for policy 1, policy_version 24100 (0.0008) [2023-10-07 20:39:57,016][67871] Updated weights for policy 1, policy_version 24110 (0.0007) [2023-10-07 20:39:57,385][67871] Updated weights for policy 1, policy_version 24120 (0.0009) [2023-10-07 20:39:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 49315840. Throughput: 0: 1663.4, 1: 1649.0. Samples: 12338742. Policy #0 lag: (min: 21.0, avg: 24.0, max: 53.0) [2023-10-07 20:39:57,478][66916] Avg episode reward: [(0, '35.330'), (1, '35.740')] [2023-10-07 20:39:58,067][67838] Updated weights for policy 0, policy_version 24072 (0.0010) [2023-10-07 20:39:58,444][67838] Updated weights for policy 0, policy_version 24082 (0.0010) [2023-10-07 20:39:58,824][67838] Updated weights for policy 0, policy_version 24092 (0.0007) [2023-10-07 20:40:01,518][67871] Updated weights for policy 1, policy_version 24130 (0.0008) [2023-10-07 20:40:01,878][67871] Updated weights for policy 1, policy_version 24140 (0.0009) [2023-10-07 20:40:02,253][67871] Updated weights for policy 1, policy_version 24150 (0.0008) [2023-10-07 20:40:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 49381376. Throughput: 0: 1660.3, 1: 1658.1. Samples: 12359330. Policy #0 lag: (min: 21.0, avg: 24.0, max: 53.0) [2023-10-07 20:40:02,477][66916] Avg episode reward: [(0, '35.670'), (1, '32.630')] [2023-10-07 20:40:02,612][67871] Updated weights for policy 1, policy_version 24160 (0.0008) [2023-10-07 20:40:02,740][67838] Updated weights for policy 0, policy_version 24102 (0.0007) [2023-10-07 20:40:03,121][67838] Updated weights for policy 0, policy_version 24112 (0.0009) [2023-10-07 20:40:03,496][67838] Updated weights for policy 0, policy_version 24122 (0.0009) [2023-10-07 20:40:06,677][67871] Updated weights for policy 1, policy_version 24170 (0.0007) [2023-10-07 20:40:07,036][67871] Updated weights for policy 1, policy_version 24180 (0.0007) [2023-10-07 20:40:07,411][67871] Updated weights for policy 1, policy_version 24190 (0.0010) [2023-10-07 20:40:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 49446912. Throughput: 0: 1664.1, 1: 1647.4. Samples: 12379468. Policy #0 lag: (min: 21.0, avg: 24.0, max: 53.0) [2023-10-07 20:40:07,477][66916] Avg episode reward: [(0, '34.710'), (1, '34.950')] [2023-10-07 20:40:07,685][67838] Updated weights for policy 0, policy_version 24132 (0.0008) [2023-10-07 20:40:08,065][67838] Updated weights for policy 0, policy_version 24142 (0.0010) [2023-10-07 20:40:08,436][67838] Updated weights for policy 0, policy_version 24152 (0.0007) [2023-10-07 20:40:11,569][67871] Updated weights for policy 1, policy_version 24200 (0.0008) [2023-10-07 20:40:11,935][67871] Updated weights for policy 1, policy_version 24210 (0.0008) [2023-10-07 20:40:12,304][67871] Updated weights for policy 1, policy_version 24220 (0.0007) [2023-10-07 20:40:12,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 49545216. Throughput: 0: 1658.0, 1: 1658.4. Samples: 12388896. Policy #0 lag: (min: 5.0, avg: 10.8, max: 37.0) [2023-10-07 20:40:12,478][66916] Avg episode reward: [(0, '36.330'), (1, '34.340')] [2023-10-07 20:40:12,690][67838] Updated weights for policy 0, policy_version 24162 (0.0008) [2023-10-07 20:40:13,070][67838] Updated weights for policy 0, policy_version 24172 (0.0007) [2023-10-07 20:40:13,442][67838] Updated weights for policy 0, policy_version 24182 (0.0009) [2023-10-07 20:40:13,814][67838] Updated weights for policy 0, policy_version 24192 (0.0008) [2023-10-07 20:40:16,546][67871] Updated weights for policy 1, policy_version 24230 (0.0008) [2023-10-07 20:40:16,921][67871] Updated weights for policy 1, policy_version 24240 (0.0007) [2023-10-07 20:40:17,290][67871] Updated weights for policy 1, policy_version 24250 (0.0008) [2023-10-07 20:40:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 49577984. Throughput: 0: 1658.4, 1: 1662.6. Samples: 12409176. Policy #0 lag: (min: 5.0, avg: 10.8, max: 37.0) [2023-10-07 20:40:17,477][66916] Avg episode reward: [(0, '34.280'), (1, '33.360')] [2023-10-07 20:40:18,068][67838] Updated weights for policy 0, policy_version 24202 (0.0007) [2023-10-07 20:40:18,446][67838] Updated weights for policy 0, policy_version 24212 (0.0007) [2023-10-07 20:40:18,813][67838] Updated weights for policy 0, policy_version 24222 (0.0009) [2023-10-07 20:40:21,315][67871] Updated weights for policy 1, policy_version 24260 (0.0009) [2023-10-07 20:40:21,707][67871] Updated weights for policy 1, policy_version 24270 (0.0010) [2023-10-07 20:40:22,070][67871] Updated weights for policy 1, policy_version 24280 (0.0007) [2023-10-07 20:40:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 49676288. Throughput: 0: 1657.1, 1: 1653.4. Samples: 12428910. Policy #0 lag: (min: 5.0, avg: 10.8, max: 37.0) [2023-10-07 20:40:22,477][66916] Avg episode reward: [(0, '36.070'), (1, '34.350')] [2023-10-07 20:40:22,970][67838] Updated weights for policy 0, policy_version 24232 (0.0008) [2023-10-07 20:40:23,347][67838] Updated weights for policy 0, policy_version 24242 (0.0007) [2023-10-07 20:40:23,710][67838] Updated weights for policy 0, policy_version 24252 (0.0008) [2023-10-07 20:40:26,108][67871] Updated weights for policy 1, policy_version 24290 (0.0007) [2023-10-07 20:40:26,480][67871] Updated weights for policy 1, policy_version 24300 (0.0007) [2023-10-07 20:40:26,843][67871] Updated weights for policy 1, policy_version 24310 (0.0008) [2023-10-07 20:40:27,217][67871] Updated weights for policy 1, policy_version 24320 (0.0010) [2023-10-07 20:40:27,477][66916] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 49741824. Throughput: 0: 1656.9, 1: 1670.4. Samples: 12438622. Policy #0 lag: (min: 5.0, avg: 10.8, max: 37.0) [2023-10-07 20:40:27,478][66916] Avg episode reward: [(0, '35.370'), (1, '33.840')] [2023-10-07 20:40:27,739][67838] Updated weights for policy 0, policy_version 24262 (0.0009) [2023-10-07 20:40:28,130][67838] Updated weights for policy 0, policy_version 24272 (0.0008) [2023-10-07 20:40:28,495][67838] Updated weights for policy 0, policy_version 24282 (0.0009) [2023-10-07 20:40:31,200][67871] Updated weights for policy 1, policy_version 24330 (0.0009) [2023-10-07 20:40:31,571][67871] Updated weights for policy 1, policy_version 24340 (0.0008) [2023-10-07 20:40:31,951][67871] Updated weights for policy 1, policy_version 24350 (0.0009) [2023-10-07 20:40:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 49807360. Throughput: 0: 1657.4, 1: 1671.0. Samples: 12459132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:40:32,477][66916] Avg episode reward: [(0, '34.190'), (1, '34.090')] [2023-10-07 20:40:32,662][67838] Updated weights for policy 0, policy_version 24292 (0.0007) [2023-10-07 20:40:33,039][67838] Updated weights for policy 0, policy_version 24302 (0.0008) [2023-10-07 20:40:33,411][67838] Updated weights for policy 0, policy_version 24312 (0.0007) [2023-10-07 20:40:36,237][67871] Updated weights for policy 1, policy_version 24360 (0.0010) [2023-10-07 20:40:36,607][67871] Updated weights for policy 1, policy_version 24370 (0.0009) [2023-10-07 20:40:36,982][67871] Updated weights for policy 1, policy_version 24380 (0.0009) [2023-10-07 20:40:37,358][67838] Updated weights for policy 0, policy_version 24322 (0.0008) [2023-10-07 20:40:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 49872896. Throughput: 0: 1663.4, 1: 1649.9. Samples: 12478562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:40:37,477][66916] Avg episode reward: [(0, '34.900'), (1, '35.590')] [2023-10-07 20:40:37,745][67838] Updated weights for policy 0, policy_version 24332 (0.0008) [2023-10-07 20:40:38,115][67838] Updated weights for policy 0, policy_version 24342 (0.0009) [2023-10-07 20:40:38,488][67838] Updated weights for policy 0, policy_version 24352 (0.0007) [2023-10-07 20:40:41,009][67871] Updated weights for policy 1, policy_version 24390 (0.0008) [2023-10-07 20:40:41,380][67871] Updated weights for policy 1, policy_version 24400 (0.0007) [2023-10-07 20:40:41,752][67871] Updated weights for policy 1, policy_version 24410 (0.0007) [2023-10-07 20:40:42,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 49938432. Throughput: 0: 1660.6, 1: 1669.2. Samples: 12488582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:40:42,477][66916] Avg episode reward: [(0, '35.280'), (1, '34.860')] [2023-10-07 20:40:42,638][67838] Updated weights for policy 0, policy_version 24362 (0.0007) [2023-10-07 20:40:43,009][67838] Updated weights for policy 0, policy_version 24372 (0.0007) [2023-10-07 20:40:43,377][67838] Updated weights for policy 0, policy_version 24382 (0.0007) [2023-10-07 20:40:45,939][67871] Updated weights for policy 1, policy_version 24420 (0.0009) [2023-10-07 20:40:46,312][67871] Updated weights for policy 1, policy_version 24430 (0.0008) [2023-10-07 20:40:46,683][67871] Updated weights for policy 1, policy_version 24440 (0.0007) [2023-10-07 20:40:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 50003968. Throughput: 0: 1659.2, 1: 1664.7. Samples: 12508902. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:40:47,477][66916] Avg episode reward: [(0, '35.300'), (1, '35.460')] [2023-10-07 20:40:47,574][67838] Updated weights for policy 0, policy_version 24392 (0.0010) [2023-10-07 20:40:47,956][67838] Updated weights for policy 0, policy_version 24402 (0.0009) [2023-10-07 20:40:48,330][67838] Updated weights for policy 0, policy_version 24412 (0.0009) [2023-10-07 20:40:50,859][67871] Updated weights for policy 1, policy_version 24450 (0.0007) [2023-10-07 20:40:51,235][67871] Updated weights for policy 1, policy_version 24460 (0.0007) [2023-10-07 20:40:51,613][67871] Updated weights for policy 1, policy_version 24470 (0.0009) [2023-10-07 20:40:51,971][67871] Updated weights for policy 1, policy_version 24480 (0.0008) [2023-10-07 20:40:52,371][67838] Updated weights for policy 0, policy_version 24422 (0.0010) [2023-10-07 20:40:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 50069504. Throughput: 0: 1655.3, 1: 1655.4. Samples: 12528448. Policy #0 lag: (min: 16.0, avg: 35.1, max: 48.0) [2023-10-07 20:40:52,478][66916] Avg episode reward: [(0, '37.160'), (1, '35.490')] [2023-10-07 20:40:52,746][67838] Updated weights for policy 0, policy_version 24432 (0.0008) [2023-10-07 20:40:53,119][67838] Updated weights for policy 0, policy_version 24442 (0.0010) [2023-10-07 20:40:55,837][67871] Updated weights for policy 1, policy_version 24490 (0.0008) [2023-10-07 20:40:56,206][67871] Updated weights for policy 1, policy_version 24500 (0.0007) [2023-10-07 20:40:56,572][67871] Updated weights for policy 1, policy_version 24510 (0.0011) [2023-10-07 20:40:57,278][67838] Updated weights for policy 0, policy_version 24452 (0.0009) [2023-10-07 20:40:57,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 50135040. Throughput: 0: 1657.7, 1: 1676.8. Samples: 12538950. Policy #0 lag: (min: 16.0, avg: 35.1, max: 48.0) [2023-10-07 20:40:57,477][66916] Avg episode reward: [(0, '35.140'), (1, '35.200')] [2023-10-07 20:40:57,661][67838] Updated weights for policy 0, policy_version 24462 (0.0010) [2023-10-07 20:40:58,030][67838] Updated weights for policy 0, policy_version 24472 (0.0009) [2023-10-07 20:41:00,499][67871] Updated weights for policy 1, policy_version 24520 (0.0010) [2023-10-07 20:41:00,871][67871] Updated weights for policy 1, policy_version 24530 (0.0011) [2023-10-07 20:41:01,242][67871] Updated weights for policy 1, policy_version 24540 (0.0009) [2023-10-07 20:41:02,137][67838] Updated weights for policy 0, policy_version 24482 (0.0007) [2023-10-07 20:41:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 50200576. Throughput: 0: 1660.1, 1: 1664.4. Samples: 12558778. Policy #0 lag: (min: 16.0, avg: 35.1, max: 48.0) [2023-10-07 20:41:02,477][66916] Avg episode reward: [(0, '37.150'), (1, '35.570')] [2023-10-07 20:41:02,532][67838] Updated weights for policy 0, policy_version 24492 (0.0007) [2023-10-07 20:41:02,909][67838] Updated weights for policy 0, policy_version 24502 (0.0008) [2023-10-07 20:41:03,282][67838] Updated weights for policy 0, policy_version 24512 (0.0008) [2023-10-07 20:41:05,354][67871] Updated weights for policy 1, policy_version 24550 (0.0009) [2023-10-07 20:41:05,722][67871] Updated weights for policy 1, policy_version 24560 (0.0010) [2023-10-07 20:41:06,100][67871] Updated weights for policy 1, policy_version 24570 (0.0010) [2023-10-07 20:41:07,392][67838] Updated weights for policy 0, policy_version 24522 (0.0010) [2023-10-07 20:41:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 50266112. Throughput: 0: 1656.4, 1: 1668.1. Samples: 12578514. Policy #0 lag: (min: 16.0, avg: 35.1, max: 48.0) [2023-10-07 20:41:07,478][66916] Avg episode reward: [(0, '36.240'), (1, '35.290')] [2023-10-07 20:41:07,488][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000024576_25165824.pth... [2023-10-07 20:41:07,524][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000023008_23560192.pth [2023-10-07 20:41:07,759][67838] Updated weights for policy 0, policy_version 24532 (0.0008) [2023-10-07 20:41:08,142][67838] Updated weights for policy 0, policy_version 24542 (0.0008) [2023-10-07 20:41:08,205][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000024544_25133056.pth... [2023-10-07 20:41:08,234][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000022976_23527424.pth [2023-10-07 20:41:10,146][67871] Updated weights for policy 1, policy_version 24580 (0.0008) [2023-10-07 20:41:10,526][67871] Updated weights for policy 1, policy_version 24590 (0.0007) [2023-10-07 20:41:10,898][67871] Updated weights for policy 1, policy_version 24600 (0.0007) [2023-10-07 20:41:12,199][67838] Updated weights for policy 0, policy_version 24552 (0.0007) [2023-10-07 20:41:12,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 50331648. Throughput: 0: 1660.7, 1: 1685.2. Samples: 12589188. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 20:41:12,478][66916] Avg episode reward: [(0, '35.180'), (1, '34.130')] [2023-10-07 20:41:12,582][67838] Updated weights for policy 0, policy_version 24562 (0.0009) [2023-10-07 20:41:12,951][67838] Updated weights for policy 0, policy_version 24572 (0.0007) [2023-10-07 20:41:14,931][67871] Updated weights for policy 1, policy_version 24610 (0.0007) [2023-10-07 20:41:15,308][67871] Updated weights for policy 1, policy_version 24620 (0.0009) [2023-10-07 20:41:15,666][67871] Updated weights for policy 1, policy_version 24630 (0.0009) [2023-10-07 20:41:16,038][67871] Updated weights for policy 1, policy_version 24640 (0.0009) [2023-10-07 20:41:17,076][67838] Updated weights for policy 0, policy_version 24582 (0.0009) [2023-10-07 20:41:17,457][67838] Updated weights for policy 0, policy_version 24592 (0.0007) [2023-10-07 20:41:17,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 50397184. Throughput: 0: 1660.8, 1: 1661.8. Samples: 12608648. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 20:41:17,477][66916] Avg episode reward: [(0, '34.340'), (1, '33.340')] [2023-10-07 20:41:17,827][67838] Updated weights for policy 0, policy_version 24602 (0.0007) [2023-10-07 20:41:20,188][67871] Updated weights for policy 1, policy_version 24650 (0.0008) [2023-10-07 20:41:20,556][67871] Updated weights for policy 1, policy_version 24660 (0.0008) [2023-10-07 20:41:20,928][67871] Updated weights for policy 1, policy_version 24670 (0.0008) [2023-10-07 20:41:21,748][67838] Updated weights for policy 0, policy_version 24612 (0.0008) [2023-10-07 20:41:22,121][67838] Updated weights for policy 0, policy_version 24622 (0.0008) [2023-10-07 20:41:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 50462720. Throughput: 0: 1649.2, 1: 1683.2. Samples: 12628522. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 20:41:22,477][66916] Avg episode reward: [(0, '32.850'), (1, '35.310')] [2023-10-07 20:41:22,491][67838] Updated weights for policy 0, policy_version 24632 (0.0010) [2023-10-07 20:41:24,865][67871] Updated weights for policy 1, policy_version 24680 (0.0009) [2023-10-07 20:41:25,225][67871] Updated weights for policy 1, policy_version 24690 (0.0009) [2023-10-07 20:41:25,589][67871] Updated weights for policy 1, policy_version 24700 (0.0011) [2023-10-07 20:41:26,664][67838] Updated weights for policy 0, policy_version 24642 (0.0010) [2023-10-07 20:41:27,048][67838] Updated weights for policy 0, policy_version 24652 (0.0008) [2023-10-07 20:41:27,417][67838] Updated weights for policy 0, policy_version 24662 (0.0009) [2023-10-07 20:41:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 50528256. Throughput: 0: 1660.1, 1: 1687.2. Samples: 12639210. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 20:41:27,477][66916] Avg episode reward: [(0, '33.370'), (1, '34.570')] [2023-10-07 20:41:27,793][67838] Updated weights for policy 0, policy_version 24672 (0.0008) [2023-10-07 20:41:29,689][67871] Updated weights for policy 1, policy_version 24710 (0.0008) [2023-10-07 20:41:30,046][67871] Updated weights for policy 1, policy_version 24720 (0.0008) [2023-10-07 20:41:30,418][67871] Updated weights for policy 1, policy_version 24730 (0.0007) [2023-10-07 20:41:31,884][67838] Updated weights for policy 0, policy_version 24682 (0.0008) [2023-10-07 20:41:32,258][67838] Updated weights for policy 0, policy_version 24692 (0.0011) [2023-10-07 20:41:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 50593792. Throughput: 0: 1664.0, 1: 1664.4. Samples: 12658680. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 20:41:32,478][66916] Avg episode reward: [(0, '34.000'), (1, '34.420')] [2023-10-07 20:41:32,634][67838] Updated weights for policy 0, policy_version 24702 (0.0008) [2023-10-07 20:41:34,618][67871] Updated weights for policy 1, policy_version 24740 (0.0009) [2023-10-07 20:41:34,989][67871] Updated weights for policy 1, policy_version 24750 (0.0009) [2023-10-07 20:41:35,363][67871] Updated weights for policy 1, policy_version 24760 (0.0008) [2023-10-07 20:41:36,815][67838] Updated weights for policy 0, policy_version 24712 (0.0007) [2023-10-07 20:41:37,193][67838] Updated weights for policy 0, policy_version 24722 (0.0008) [2023-10-07 20:41:37,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 50659328. Throughput: 0: 1653.6, 1: 1683.7. Samples: 12678628. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 20:41:37,477][66916] Avg episode reward: [(0, '36.540'), (1, '34.760')] [2023-10-07 20:41:37,563][67838] Updated weights for policy 0, policy_version 24732 (0.0009) [2023-10-07 20:41:39,308][67871] Updated weights for policy 1, policy_version 24770 (0.0009) [2023-10-07 20:41:39,683][67871] Updated weights for policy 1, policy_version 24780 (0.0007) [2023-10-07 20:41:40,051][67871] Updated weights for policy 1, policy_version 24790 (0.0008) [2023-10-07 20:41:40,417][67871] Updated weights for policy 1, policy_version 24800 (0.0008) [2023-10-07 20:41:41,802][67838] Updated weights for policy 0, policy_version 24742 (0.0007) [2023-10-07 20:41:42,179][67838] Updated weights for policy 0, policy_version 24752 (0.0007) [2023-10-07 20:41:42,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 50724864. Throughput: 0: 1665.5, 1: 1666.0. Samples: 12688866. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 20:41:42,477][66916] Avg episode reward: [(0, '35.640'), (1, '34.030')] [2023-10-07 20:41:42,550][67838] Updated weights for policy 0, policy_version 24762 (0.0008) [2023-10-07 20:41:44,496][67871] Updated weights for policy 1, policy_version 24810 (0.0009) [2023-10-07 20:41:44,864][67871] Updated weights for policy 1, policy_version 24820 (0.0010) [2023-10-07 20:41:45,238][67871] Updated weights for policy 1, policy_version 24830 (0.0008) [2023-10-07 20:41:46,510][67838] Updated weights for policy 0, policy_version 24772 (0.0009) [2023-10-07 20:41:46,895][67838] Updated weights for policy 0, policy_version 24782 (0.0011) [2023-10-07 20:41:47,275][67838] Updated weights for policy 0, policy_version 24792 (0.0007) [2023-10-07 20:41:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 50790400. Throughput: 0: 1667.2, 1: 1666.1. Samples: 12708778. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 20:41:47,477][66916] Avg episode reward: [(0, '35.860'), (1, '35.790')] [2023-10-07 20:41:49,377][67871] Updated weights for policy 1, policy_version 24840 (0.0008) [2023-10-07 20:41:49,744][67871] Updated weights for policy 1, policy_version 24850 (0.0010) [2023-10-07 20:41:50,116][67871] Updated weights for policy 1, policy_version 24860 (0.0009) [2023-10-07 20:41:51,382][67838] Updated weights for policy 0, policy_version 24802 (0.0008) [2023-10-07 20:41:51,781][67838] Updated weights for policy 0, policy_version 24812 (0.0009) [2023-10-07 20:41:52,151][67838] Updated weights for policy 0, policy_version 24822 (0.0010) [2023-10-07 20:41:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 50855936. Throughput: 0: 1652.4, 1: 1674.5. Samples: 12728224. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 20:41:52,477][66916] Avg episode reward: [(0, '37.550'), (1, '33.940')] [2023-10-07 20:41:52,528][67838] Updated weights for policy 0, policy_version 24832 (0.0007) [2023-10-07 20:41:54,297][67871] Updated weights for policy 1, policy_version 24870 (0.0007) [2023-10-07 20:41:54,667][67871] Updated weights for policy 1, policy_version 24880 (0.0010) [2023-10-07 20:41:55,036][67871] Updated weights for policy 1, policy_version 24890 (0.0009) [2023-10-07 20:41:56,699][67838] Updated weights for policy 0, policy_version 24842 (0.0008) [2023-10-07 20:41:57,071][67838] Updated weights for policy 0, policy_version 24852 (0.0008) [2023-10-07 20:41:57,448][67838] Updated weights for policy 0, policy_version 24862 (0.0009) [2023-10-07 20:41:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 50921472. Throughput: 0: 1666.1, 1: 1652.5. Samples: 12738524. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 20:41:57,477][66916] Avg episode reward: [(0, '34.160'), (1, '35.080')] [2023-10-07 20:41:59,426][67871] Updated weights for policy 1, policy_version 24900 (0.0007) [2023-10-07 20:41:59,814][67871] Updated weights for policy 1, policy_version 24910 (0.0009) [2023-10-07 20:42:00,179][67871] Updated weights for policy 1, policy_version 24920 (0.0008) [2023-10-07 20:42:01,726][67838] Updated weights for policy 0, policy_version 24872 (0.0009) [2023-10-07 20:42:02,102][67838] Updated weights for policy 0, policy_version 24882 (0.0008) [2023-10-07 20:42:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 50987008. Throughput: 0: 1664.4, 1: 1657.9. Samples: 12758148. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 20:42:02,477][66916] Avg episode reward: [(0, '36.640'), (1, '34.170')] [2023-10-07 20:42:02,488][67838] Updated weights for policy 0, policy_version 24892 (0.0010) [2023-10-07 20:42:04,345][67871] Updated weights for policy 1, policy_version 24930 (0.0008) [2023-10-07 20:42:04,701][67871] Updated weights for policy 1, policy_version 24940 (0.0008) [2023-10-07 20:42:05,074][67871] Updated weights for policy 1, policy_version 24950 (0.0008) [2023-10-07 20:42:05,440][67871] Updated weights for policy 1, policy_version 24960 (0.0008) [2023-10-07 20:42:06,478][67838] Updated weights for policy 0, policy_version 24902 (0.0010) [2023-10-07 20:42:06,845][67838] Updated weights for policy 0, policy_version 24912 (0.0008) [2023-10-07 20:42:07,222][67838] Updated weights for policy 0, policy_version 24922 (0.0008) [2023-10-07 20:42:07,476][66916] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 51085312. Throughput: 0: 1657.4, 1: 1661.7. Samples: 12777882. Policy #0 lag: (min: 29.0, avg: 39.5, max: 61.0) [2023-10-07 20:42:07,477][66916] Avg episode reward: [(0, '35.870'), (1, '34.310')] [2023-10-07 20:42:09,635][67871] Updated weights for policy 1, policy_version 24970 (0.0007) [2023-10-07 20:42:10,004][67871] Updated weights for policy 1, policy_version 24980 (0.0007) [2023-10-07 20:42:10,379][67871] Updated weights for policy 1, policy_version 24990 (0.0008) [2023-10-07 20:42:11,358][67838] Updated weights for policy 0, policy_version 24932 (0.0007) [2023-10-07 20:42:11,725][67838] Updated weights for policy 0, policy_version 24942 (0.0010) [2023-10-07 20:42:12,096][67838] Updated weights for policy 0, policy_version 24952 (0.0008) [2023-10-07 20:42:12,476][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 51150848. Throughput: 0: 1668.5, 1: 1649.6. Samples: 12788522. Policy #0 lag: (min: 29.0, avg: 39.5, max: 61.0) [2023-10-07 20:42:12,477][66916] Avg episode reward: [(0, '32.890'), (1, '34.160')] [2023-10-07 20:42:14,360][67871] Updated weights for policy 1, policy_version 25000 (0.0007) [2023-10-07 20:42:14,737][67871] Updated weights for policy 1, policy_version 25010 (0.0009) [2023-10-07 20:42:15,100][67871] Updated weights for policy 1, policy_version 25020 (0.0009) [2023-10-07 20:42:16,178][67838] Updated weights for policy 0, policy_version 24962 (0.0008) [2023-10-07 20:42:16,551][67838] Updated weights for policy 0, policy_version 24972 (0.0007) [2023-10-07 20:42:16,921][67838] Updated weights for policy 0, policy_version 24982 (0.0008) [2023-10-07 20:42:17,287][67838] Updated weights for policy 0, policy_version 24992 (0.0007) [2023-10-07 20:42:17,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 51216384. Throughput: 0: 1667.1, 1: 1663.2. Samples: 12808542. Policy #0 lag: (min: 29.0, avg: 39.5, max: 61.0) [2023-10-07 20:42:17,477][66916] Avg episode reward: [(0, '36.770'), (1, '34.460')] [2023-10-07 20:42:19,213][67871] Updated weights for policy 1, policy_version 25030 (0.0009) [2023-10-07 20:42:19,585][67871] Updated weights for policy 1, policy_version 25040 (0.0007) [2023-10-07 20:42:19,951][67871] Updated weights for policy 1, policy_version 25050 (0.0007) [2023-10-07 20:42:21,354][67838] Updated weights for policy 0, policy_version 25002 (0.0010) [2023-10-07 20:42:21,723][67838] Updated weights for policy 0, policy_version 25012 (0.0008) [2023-10-07 20:42:22,091][67838] Updated weights for policy 0, policy_version 25022 (0.0008) [2023-10-07 20:42:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 51281920. Throughput: 0: 1655.8, 1: 1664.4. Samples: 12828036. Policy #0 lag: (min: 29.0, avg: 39.5, max: 61.0) [2023-10-07 20:42:22,478][66916] Avg episode reward: [(0, '38.040'), (1, '33.460')] [2023-10-07 20:42:24,106][67871] Updated weights for policy 1, policy_version 25060 (0.0008) [2023-10-07 20:42:24,473][67871] Updated weights for policy 1, policy_version 25070 (0.0007) [2023-10-07 20:42:24,845][67871] Updated weights for policy 1, policy_version 25080 (0.0007) [2023-10-07 20:42:26,302][67838] Updated weights for policy 0, policy_version 25032 (0.0008) [2023-10-07 20:42:26,690][67838] Updated weights for policy 0, policy_version 25042 (0.0008) [2023-10-07 20:42:27,069][67838] Updated weights for policy 0, policy_version 25052 (0.0009) [2023-10-07 20:42:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 51347456. Throughput: 0: 1665.8, 1: 1659.2. Samples: 12838492. Policy #0 lag: (min: 1.0, avg: 12.7, max: 33.0) [2023-10-07 20:42:27,477][66916] Avg episode reward: [(0, '37.020'), (1, '34.510')] [2023-10-07 20:42:28,850][67871] Updated weights for policy 1, policy_version 25090 (0.0008) [2023-10-07 20:42:29,220][67871] Updated weights for policy 1, policy_version 25100 (0.0011) [2023-10-07 20:42:29,590][67871] Updated weights for policy 1, policy_version 25110 (0.0008) [2023-10-07 20:42:29,960][67871] Updated weights for policy 1, policy_version 25120 (0.0009) [2023-10-07 20:42:30,985][67838] Updated weights for policy 0, policy_version 25062 (0.0010) [2023-10-07 20:42:31,354][67838] Updated weights for policy 0, policy_version 25072 (0.0009) [2023-10-07 20:42:31,724][67838] Updated weights for policy 0, policy_version 25082 (0.0008) [2023-10-07 20:42:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 51412992. Throughput: 0: 1661.4, 1: 1662.4. Samples: 12858350. Policy #0 lag: (min: 1.0, avg: 12.7, max: 33.0) [2023-10-07 20:42:32,477][66916] Avg episode reward: [(0, '36.900'), (1, '34.560')] [2023-10-07 20:42:34,068][67871] Updated weights for policy 1, policy_version 25130 (0.0009) [2023-10-07 20:42:34,438][67871] Updated weights for policy 1, policy_version 25140 (0.0008) [2023-10-07 20:42:34,817][67871] Updated weights for policy 1, policy_version 25150 (0.0007) [2023-10-07 20:42:35,713][67838] Updated weights for policy 0, policy_version 25092 (0.0009) [2023-10-07 20:42:36,086][67838] Updated weights for policy 0, policy_version 25102 (0.0010) [2023-10-07 20:42:36,467][67838] Updated weights for policy 0, policy_version 25112 (0.0009) [2023-10-07 20:42:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 51478528. Throughput: 0: 1661.3, 1: 1669.1. Samples: 12878094. Policy #0 lag: (min: 1.0, avg: 12.7, max: 33.0) [2023-10-07 20:42:37,477][66916] Avg episode reward: [(0, '36.000'), (1, '35.360')] [2023-10-07 20:42:38,781][67871] Updated weights for policy 1, policy_version 25160 (0.0008) [2023-10-07 20:42:39,157][67871] Updated weights for policy 1, policy_version 25170 (0.0008) [2023-10-07 20:42:39,517][67871] Updated weights for policy 1, policy_version 25180 (0.0007) [2023-10-07 20:42:40,760][67838] Updated weights for policy 0, policy_version 25122 (0.0008) [2023-10-07 20:42:41,158][67838] Updated weights for policy 0, policy_version 25132 (0.0008) [2023-10-07 20:42:41,545][67838] Updated weights for policy 0, policy_version 25142 (0.0008) [2023-10-07 20:42:41,910][67838] Updated weights for policy 0, policy_version 25152 (0.0010) [2023-10-07 20:42:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 51544064. Throughput: 0: 1674.2, 1: 1657.7. Samples: 12888458. Policy #0 lag: (min: 1.0, avg: 12.7, max: 33.0) [2023-10-07 20:42:42,477][66916] Avg episode reward: [(0, '36.000'), (1, '34.790')] [2023-10-07 20:42:43,402][67871] Updated weights for policy 1, policy_version 25190 (0.0008) [2023-10-07 20:42:43,774][67871] Updated weights for policy 1, policy_version 25200 (0.0007) [2023-10-07 20:42:44,137][67871] Updated weights for policy 1, policy_version 25210 (0.0007) [2023-10-07 20:42:45,991][67838] Updated weights for policy 0, policy_version 25162 (0.0008) [2023-10-07 20:42:46,372][67838] Updated weights for policy 0, policy_version 25172 (0.0009) [2023-10-07 20:42:46,744][67838] Updated weights for policy 0, policy_version 25182 (0.0009) [2023-10-07 20:42:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 51609600. Throughput: 0: 1666.3, 1: 1678.9. Samples: 12908684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:42:47,477][66916] Avg episode reward: [(0, '34.080'), (1, '34.230')] [2023-10-07 20:42:48,421][67871] Updated weights for policy 1, policy_version 25220 (0.0007) [2023-10-07 20:42:48,813][67871] Updated weights for policy 1, policy_version 25230 (0.0007) [2023-10-07 20:42:49,179][67871] Updated weights for policy 1, policy_version 25240 (0.0007) [2023-10-07 20:42:50,868][67838] Updated weights for policy 0, policy_version 25192 (0.0007) [2023-10-07 20:42:51,246][67838] Updated weights for policy 0, policy_version 25202 (0.0008) [2023-10-07 20:42:51,618][67838] Updated weights for policy 0, policy_version 25212 (0.0010) [2023-10-07 20:42:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 51675136. Throughput: 0: 1657.9, 1: 1675.6. Samples: 12927892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:42:52,478][66916] Avg episode reward: [(0, '35.890'), (1, '34.330')] [2023-10-07 20:42:53,263][67871] Updated weights for policy 1, policy_version 25250 (0.0009) [2023-10-07 20:42:53,622][67871] Updated weights for policy 1, policy_version 25260 (0.0009) [2023-10-07 20:42:53,979][67871] Updated weights for policy 1, policy_version 25270 (0.0010) [2023-10-07 20:42:54,353][67871] Updated weights for policy 1, policy_version 25280 (0.0010) [2023-10-07 20:42:55,641][67838] Updated weights for policy 0, policy_version 25222 (0.0010) [2023-10-07 20:42:56,027][67838] Updated weights for policy 0, policy_version 25232 (0.0010) [2023-10-07 20:42:56,406][67838] Updated weights for policy 0, policy_version 25242 (0.0008) [2023-10-07 20:42:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 51740672. Throughput: 0: 1668.8, 1: 1658.3. Samples: 12938240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:42:57,477][66916] Avg episode reward: [(0, '36.030'), (1, '34.040')] [2023-10-07 20:42:58,443][67871] Updated weights for policy 1, policy_version 25290 (0.0010) [2023-10-07 20:42:58,811][67871] Updated weights for policy 1, policy_version 25300 (0.0007) [2023-10-07 20:42:59,179][67871] Updated weights for policy 1, policy_version 25310 (0.0007) [2023-10-07 20:43:00,577][67838] Updated weights for policy 0, policy_version 25252 (0.0007) [2023-10-07 20:43:00,949][67838] Updated weights for policy 0, policy_version 25262 (0.0008) [2023-10-07 20:43:01,319][67838] Updated weights for policy 0, policy_version 25272 (0.0009) [2023-10-07 20:43:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 51806208. Throughput: 0: 1652.4, 1: 1674.4. Samples: 12958248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:43:02,477][66916] Avg episode reward: [(0, '34.280'), (1, '33.490')] [2023-10-07 20:43:03,206][67871] Updated weights for policy 1, policy_version 25320 (0.0009) [2023-10-07 20:43:03,573][67871] Updated weights for policy 1, policy_version 25330 (0.0007) [2023-10-07 20:43:03,935][67871] Updated weights for policy 1, policy_version 25340 (0.0010) [2023-10-07 20:43:05,230][67838] Updated weights for policy 0, policy_version 25282 (0.0008) [2023-10-07 20:43:05,612][67838] Updated weights for policy 0, policy_version 25292 (0.0007) [2023-10-07 20:43:05,978][67838] Updated weights for policy 0, policy_version 25302 (0.0010) [2023-10-07 20:43:06,357][67838] Updated weights for policy 0, policy_version 25312 (0.0010) [2023-10-07 20:43:07,477][66916] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 51871744. Throughput: 0: 1658.6, 1: 1679.2. Samples: 12978238. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-10-07 20:43:07,478][66916] Avg episode reward: [(0, '35.180'), (1, '35.340')] [2023-10-07 20:43:07,492][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000025344_25952256.pth... [2023-10-07 20:43:07,492][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000025312_25919488.pth... [2023-10-07 20:43:07,530][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000023744_24313856.pth [2023-10-07 20:43:07,531][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000023808_24379392.pth [2023-10-07 20:43:08,064][67871] Updated weights for policy 1, policy_version 25350 (0.0010) [2023-10-07 20:43:08,434][67871] Updated weights for policy 1, policy_version 25360 (0.0007) [2023-10-07 20:43:08,810][67871] Updated weights for policy 1, policy_version 25370 (0.0009) [2023-10-07 20:43:10,531][67838] Updated weights for policy 0, policy_version 25322 (0.0008) [2023-10-07 20:43:10,902][67838] Updated weights for policy 0, policy_version 25332 (0.0009) [2023-10-07 20:43:11,276][67838] Updated weights for policy 0, policy_version 25342 (0.0009) [2023-10-07 20:43:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51937280. Throughput: 0: 1665.7, 1: 1668.8. Samples: 12988546. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-10-07 20:43:12,477][66916] Avg episode reward: [(0, '34.970'), (1, '32.430')] [2023-10-07 20:43:12,918][67871] Updated weights for policy 1, policy_version 25380 (0.0008) [2023-10-07 20:43:13,286][67871] Updated weights for policy 1, policy_version 25390 (0.0007) [2023-10-07 20:43:13,650][67871] Updated weights for policy 1, policy_version 25400 (0.0008) [2023-10-07 20:43:15,357][67838] Updated weights for policy 0, policy_version 25352 (0.0009) [2023-10-07 20:43:15,728][67838] Updated weights for policy 0, policy_version 25362 (0.0009) [2023-10-07 20:43:16,104][67838] Updated weights for policy 0, policy_version 25372 (0.0010) [2023-10-07 20:43:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 52002816. Throughput: 0: 1649.7, 1: 1677.6. Samples: 13008080. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-10-07 20:43:17,477][66916] Avg episode reward: [(0, '37.580'), (1, '33.300')] [2023-10-07 20:43:17,852][67871] Updated weights for policy 1, policy_version 25410 (0.0007) [2023-10-07 20:43:18,223][67871] Updated weights for policy 1, policy_version 25420 (0.0007) [2023-10-07 20:43:18,593][67871] Updated weights for policy 1, policy_version 25430 (0.0007) [2023-10-07 20:43:18,964][67871] Updated weights for policy 1, policy_version 25440 (0.0009) [2023-10-07 20:43:20,176][67838] Updated weights for policy 0, policy_version 25382 (0.0009) [2023-10-07 20:43:20,548][67838] Updated weights for policy 0, policy_version 25392 (0.0007) [2023-10-07 20:43:20,920][67838] Updated weights for policy 0, policy_version 25402 (0.0008) [2023-10-07 20:43:22,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 52068352. Throughput: 0: 1662.3, 1: 1678.1. Samples: 13028412. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) [2023-10-07 20:43:22,478][66916] Avg episode reward: [(0, '36.070'), (1, '35.760')] [2023-10-07 20:43:22,937][67871] Updated weights for policy 1, policy_version 25450 (0.0009) [2023-10-07 20:43:23,300][67871] Updated weights for policy 1, policy_version 25460 (0.0008) [2023-10-07 20:43:23,680][67871] Updated weights for policy 1, policy_version 25470 (0.0007) [2023-10-07 20:43:25,244][67838] Updated weights for policy 0, policy_version 25412 (0.0009) [2023-10-07 20:43:25,620][67838] Updated weights for policy 0, policy_version 25422 (0.0010) [2023-10-07 20:43:26,003][67838] Updated weights for policy 0, policy_version 25432 (0.0009) [2023-10-07 20:43:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 52133888. Throughput: 0: 1659.8, 1: 1678.5. Samples: 13038680. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-07 20:43:27,477][66916] Avg episode reward: [(0, '35.660'), (1, '34.510')] [2023-10-07 20:43:27,684][67871] Updated weights for policy 1, policy_version 25480 (0.0007) [2023-10-07 20:43:28,054][67871] Updated weights for policy 1, policy_version 25490 (0.0009) [2023-10-07 20:43:28,432][67871] Updated weights for policy 1, policy_version 25500 (0.0007) [2023-10-07 20:43:30,088][67838] Updated weights for policy 0, policy_version 25442 (0.0007) [2023-10-07 20:43:30,460][67838] Updated weights for policy 0, policy_version 25452 (0.0010) [2023-10-07 20:43:30,825][67838] Updated weights for policy 0, policy_version 25462 (0.0008) [2023-10-07 20:43:31,200][67838] Updated weights for policy 0, policy_version 25472 (0.0008) [2023-10-07 20:43:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 52199424. Throughput: 0: 1643.8, 1: 1683.2. Samples: 13058398. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-07 20:43:32,477][66916] Avg episode reward: [(0, '35.190'), (1, '35.100')] [2023-10-07 20:43:32,532][67871] Updated weights for policy 1, policy_version 25510 (0.0008) [2023-10-07 20:43:32,898][67871] Updated weights for policy 1, policy_version 25520 (0.0009) [2023-10-07 20:43:33,274][67871] Updated weights for policy 1, policy_version 25530 (0.0009) [2023-10-07 20:43:35,249][67838] Updated weights for policy 0, policy_version 25482 (0.0008) [2023-10-07 20:43:35,612][67838] Updated weights for policy 0, policy_version 25492 (0.0011) [2023-10-07 20:43:35,984][67838] Updated weights for policy 0, policy_version 25502 (0.0010) [2023-10-07 20:43:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 52264960. Throughput: 0: 1666.2, 1: 1685.3. Samples: 13078708. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-07 20:43:37,477][66916] Avg episode reward: [(0, '34.300'), (1, '34.880')] [2023-10-07 20:43:37,499][67871] Updated weights for policy 1, policy_version 25540 (0.0007) [2023-10-07 20:43:37,895][67871] Updated weights for policy 1, policy_version 25550 (0.0008) [2023-10-07 20:43:38,274][67871] Updated weights for policy 1, policy_version 25560 (0.0007) [2023-10-07 20:43:40,138][67838] Updated weights for policy 0, policy_version 25512 (0.0007) [2023-10-07 20:43:40,503][67838] Updated weights for policy 0, policy_version 25522 (0.0007) [2023-10-07 20:43:40,884][67838] Updated weights for policy 0, policy_version 25532 (0.0009) [2023-10-07 20:43:42,427][67871] Updated weights for policy 1, policy_version 25570 (0.0009) [2023-10-07 20:43:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 52330496. Throughput: 0: 1656.9, 1: 1686.3. Samples: 13088684. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-07 20:43:42,477][66916] Avg episode reward: [(0, '35.890'), (1, '34.390')] [2023-10-07 20:43:42,794][67871] Updated weights for policy 1, policy_version 25580 (0.0007) [2023-10-07 20:43:43,162][67871] Updated weights for policy 1, policy_version 25590 (0.0007) [2023-10-07 20:43:43,524][67871] Updated weights for policy 1, policy_version 25600 (0.0007) [2023-10-07 20:43:45,151][67838] Updated weights for policy 0, policy_version 25542 (0.0007) [2023-10-07 20:43:45,528][67838] Updated weights for policy 0, policy_version 25552 (0.0008) [2023-10-07 20:43:45,918][67838] Updated weights for policy 0, policy_version 25562 (0.0009) [2023-10-07 20:43:47,435][67871] Updated weights for policy 1, policy_version 25610 (0.0009) [2023-10-07 20:43:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 52396032. Throughput: 0: 1650.6, 1: 1682.4. Samples: 13108236. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 20:43:47,477][66916] Avg episode reward: [(0, '37.540'), (1, '34.070')] [2023-10-07 20:43:47,802][67871] Updated weights for policy 1, policy_version 25620 (0.0009) [2023-10-07 20:43:48,171][67871] Updated weights for policy 1, policy_version 25630 (0.0009) [2023-10-07 20:43:49,956][67838] Updated weights for policy 0, policy_version 25572 (0.0009) [2023-10-07 20:43:50,332][67838] Updated weights for policy 0, policy_version 25582 (0.0007) [2023-10-07 20:43:50,697][67838] Updated weights for policy 0, policy_version 25592 (0.0010) [2023-10-07 20:43:52,313][67871] Updated weights for policy 1, policy_version 25640 (0.0008) [2023-10-07 20:43:52,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 52461568. Throughput: 0: 1663.4, 1: 1678.5. Samples: 13128624. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 20:43:52,478][66916] Avg episode reward: [(0, '36.160'), (1, '36.170')] [2023-10-07 20:43:52,683][67871] Updated weights for policy 1, policy_version 25650 (0.0007) [2023-10-07 20:43:53,055][67871] Updated weights for policy 1, policy_version 25660 (0.0007) [2023-10-07 20:43:54,656][67838] Updated weights for policy 0, policy_version 25602 (0.0010) [2023-10-07 20:43:55,037][67838] Updated weights for policy 0, policy_version 25612 (0.0009) [2023-10-07 20:43:55,407][67838] Updated weights for policy 0, policy_version 25622 (0.0009) [2023-10-07 20:43:55,784][67838] Updated weights for policy 0, policy_version 25632 (0.0009) [2023-10-07 20:43:57,215][67871] Updated weights for policy 1, policy_version 25670 (0.0008) [2023-10-07 20:43:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 52527104. Throughput: 0: 1654.0, 1: 1678.8. Samples: 13138520. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 20:43:57,477][66916] Avg episode reward: [(0, '37.870'), (1, '32.870')] [2023-10-07 20:43:57,581][67871] Updated weights for policy 1, policy_version 25680 (0.0009) [2023-10-07 20:43:57,955][67871] Updated weights for policy 1, policy_version 25690 (0.0008) [2023-10-07 20:43:59,819][67838] Updated weights for policy 0, policy_version 25642 (0.0009) [2023-10-07 20:44:00,203][67838] Updated weights for policy 0, policy_version 25652 (0.0010) [2023-10-07 20:44:00,577][67838] Updated weights for policy 0, policy_version 25662 (0.0010) [2023-10-07 20:44:02,084][67871] Updated weights for policy 1, policy_version 25700 (0.0009) [2023-10-07 20:44:02,446][67871] Updated weights for policy 1, policy_version 25710 (0.0008) [2023-10-07 20:44:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 52592640. Throughput: 0: 1661.0, 1: 1675.4. Samples: 13158220. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 20:44:02,477][66916] Avg episode reward: [(0, '36.760'), (1, '34.250')] [2023-10-07 20:44:02,814][67871] Updated weights for policy 1, policy_version 25720 (0.0008) [2023-10-07 20:44:04,657][67838] Updated weights for policy 0, policy_version 25672 (0.0009) [2023-10-07 20:44:05,027][67838] Updated weights for policy 0, policy_version 25682 (0.0007) [2023-10-07 20:44:05,406][67838] Updated weights for policy 0, policy_version 25692 (0.0007) [2023-10-07 20:44:06,954][67871] Updated weights for policy 1, policy_version 25730 (0.0008) [2023-10-07 20:44:07,320][67871] Updated weights for policy 1, policy_version 25740 (0.0007) [2023-10-07 20:44:07,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 52658176. Throughput: 0: 1667.1, 1: 1669.0. Samples: 13178536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:44:07,477][66916] Avg episode reward: [(0, '36.480'), (1, '35.640')] [2023-10-07 20:44:07,683][67871] Updated weights for policy 1, policy_version 25750 (0.0008) [2023-10-07 20:44:08,043][67871] Updated weights for policy 1, policy_version 25760 (0.0009) [2023-10-07 20:44:09,586][67838] Updated weights for policy 0, policy_version 25702 (0.0008) [2023-10-07 20:44:09,967][67838] Updated weights for policy 0, policy_version 25712 (0.0008) [2023-10-07 20:44:10,335][67838] Updated weights for policy 0, policy_version 25722 (0.0008) [2023-10-07 20:44:12,142][67871] Updated weights for policy 1, policy_version 25770 (0.0009) [2023-10-07 20:44:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 52723712. Throughput: 0: 1652.0, 1: 1669.0. Samples: 13188122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:44:12,478][66916] Avg episode reward: [(0, '36.700'), (1, '33.860')] [2023-10-07 20:44:12,512][67871] Updated weights for policy 1, policy_version 25780 (0.0011) [2023-10-07 20:44:12,886][67871] Updated weights for policy 1, policy_version 25790 (0.0009) [2023-10-07 20:44:14,539][67838] Updated weights for policy 0, policy_version 25732 (0.0009) [2023-10-07 20:44:14,925][67838] Updated weights for policy 0, policy_version 25742 (0.0009) [2023-10-07 20:44:15,307][67838] Updated weights for policy 0, policy_version 25752 (0.0008) [2023-10-07 20:44:17,230][67871] Updated weights for policy 1, policy_version 25800 (0.0009) [2023-10-07 20:44:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 52789248. Throughput: 0: 1663.3, 1: 1657.4. Samples: 13207830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:44:17,478][66916] Avg episode reward: [(0, '36.250'), (1, '35.980')] [2023-10-07 20:44:17,603][67871] Updated weights for policy 1, policy_version 25810 (0.0009) [2023-10-07 20:44:17,981][67871] Updated weights for policy 1, policy_version 25820 (0.0009) [2023-10-07 20:44:19,379][67838] Updated weights for policy 0, policy_version 25762 (0.0009) [2023-10-07 20:44:19,774][67838] Updated weights for policy 0, policy_version 25772 (0.0008) [2023-10-07 20:44:20,135][67838] Updated weights for policy 0, policy_version 25782 (0.0009) [2023-10-07 20:44:20,510][67838] Updated weights for policy 0, policy_version 25792 (0.0010) [2023-10-07 20:44:21,986][67871] Updated weights for policy 1, policy_version 25830 (0.0008) [2023-10-07 20:44:22,364][67871] Updated weights for policy 1, policy_version 25840 (0.0007) [2023-10-07 20:44:22,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 52854784. Throughput: 0: 1663.0, 1: 1655.3. Samples: 13228032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:44:22,478][66916] Avg episode reward: [(0, '36.420'), (1, '36.170')] [2023-10-07 20:44:22,727][67871] Updated weights for policy 1, policy_version 25850 (0.0007) [2023-10-07 20:44:24,475][67838] Updated weights for policy 0, policy_version 25802 (0.0008) [2023-10-07 20:44:24,849][67838] Updated weights for policy 0, policy_version 25812 (0.0008) [2023-10-07 20:44:25,215][67838] Updated weights for policy 0, policy_version 25822 (0.0008) [2023-10-07 20:44:26,951][67871] Updated weights for policy 1, policy_version 25860 (0.0009) [2023-10-07 20:44:27,317][67871] Updated weights for policy 1, policy_version 25870 (0.0008) [2023-10-07 20:44:27,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 52920320. Throughput: 0: 1649.8, 1: 1655.8. Samples: 13237434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-07 20:44:27,477][66916] Avg episode reward: [(0, '37.000'), (1, '33.470')] [2023-10-07 20:44:27,693][67871] Updated weights for policy 1, policy_version 25880 (0.0007) [2023-10-07 20:44:29,447][67838] Updated weights for policy 0, policy_version 25832 (0.0009) [2023-10-07 20:44:29,813][67838] Updated weights for policy 0, policy_version 25842 (0.0010) [2023-10-07 20:44:30,178][67838] Updated weights for policy 0, policy_version 25852 (0.0009) [2023-10-07 20:44:31,809][67871] Updated weights for policy 1, policy_version 25890 (0.0007) [2023-10-07 20:44:32,180][67871] Updated weights for policy 1, policy_version 25900 (0.0008) [2023-10-07 20:44:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 52985856. Throughput: 0: 1659.7, 1: 1650.8. Samples: 13257208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-07 20:44:32,477][66916] Avg episode reward: [(0, '39.150'), (1, '36.330')] [2023-10-07 20:44:32,549][67871] Updated weights for policy 1, policy_version 25910 (0.0009) [2023-10-07 20:44:32,916][67871] Updated weights for policy 1, policy_version 25920 (0.0008) [2023-10-07 20:44:34,188][67838] Updated weights for policy 0, policy_version 25862 (0.0011) [2023-10-07 20:44:34,565][67838] Updated weights for policy 0, policy_version 25872 (0.0008) [2023-10-07 20:44:34,939][67838] Updated weights for policy 0, policy_version 25882 (0.0007) [2023-10-07 20:44:37,079][67871] Updated weights for policy 1, policy_version 25930 (0.0009) [2023-10-07 20:44:37,454][67871] Updated weights for policy 1, policy_version 25940 (0.0008) [2023-10-07 20:44:37,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 53051392. Throughput: 0: 1666.0, 1: 1643.6. Samples: 13277554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-07 20:44:37,478][66916] Avg episode reward: [(0, '36.880'), (1, '33.930')] [2023-10-07 20:44:37,824][67871] Updated weights for policy 1, policy_version 25950 (0.0010) [2023-10-07 20:44:39,014][67838] Updated weights for policy 0, policy_version 25892 (0.0008) [2023-10-07 20:44:39,398][67838] Updated weights for policy 0, policy_version 25902 (0.0007) [2023-10-07 20:44:39,777][67838] Updated weights for policy 0, policy_version 25912 (0.0009) [2023-10-07 20:44:41,878][67871] Updated weights for policy 1, policy_version 25960 (0.0008) [2023-10-07 20:44:42,251][67871] Updated weights for policy 1, policy_version 25970 (0.0008) [2023-10-07 20:44:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 53116928. Throughput: 0: 1649.6, 1: 1649.7. Samples: 13286988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-07 20:44:42,477][66916] Avg episode reward: [(0, '38.430'), (1, '35.730')] [2023-10-07 20:44:42,619][67871] Updated weights for policy 1, policy_version 25980 (0.0009) [2023-10-07 20:44:43,762][67838] Updated weights for policy 0, policy_version 25922 (0.0011) [2023-10-07 20:44:44,134][67838] Updated weights for policy 0, policy_version 25932 (0.0007) [2023-10-07 20:44:44,508][67838] Updated weights for policy 0, policy_version 25942 (0.0009) [2023-10-07 20:44:44,877][67838] Updated weights for policy 0, policy_version 25952 (0.0011) [2023-10-07 20:44:46,610][67871] Updated weights for policy 1, policy_version 25990 (0.0009) [2023-10-07 20:44:46,972][67871] Updated weights for policy 1, policy_version 26000 (0.0008) [2023-10-07 20:44:47,347][67871] Updated weights for policy 1, policy_version 26010 (0.0007) [2023-10-07 20:44:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 53182464. Throughput: 0: 1666.8, 1: 1652.8. Samples: 13307602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:44:47,477][66916] Avg episode reward: [(0, '37.450'), (1, '35.430')] [2023-10-07 20:44:49,109][67838] Updated weights for policy 0, policy_version 25962 (0.0011) [2023-10-07 20:44:49,481][67838] Updated weights for policy 0, policy_version 25972 (0.0011) [2023-10-07 20:44:49,862][67838] Updated weights for policy 0, policy_version 25982 (0.0010) [2023-10-07 20:44:51,483][67871] Updated weights for policy 1, policy_version 26020 (0.0009) [2023-10-07 20:44:51,853][67871] Updated weights for policy 1, policy_version 26030 (0.0008) [2023-10-07 20:44:52,217][67871] Updated weights for policy 1, policy_version 26040 (0.0010) [2023-10-07 20:44:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 53248000. Throughput: 0: 1663.5, 1: 1647.3. Samples: 13327524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:44:52,477][66916] Avg episode reward: [(0, '35.700'), (1, '35.530')] [2023-10-07 20:44:53,887][67838] Updated weights for policy 0, policy_version 25992 (0.0008) [2023-10-07 20:44:54,261][67838] Updated weights for policy 0, policy_version 26002 (0.0007) [2023-10-07 20:44:54,633][67838] Updated weights for policy 0, policy_version 26012 (0.0009) [2023-10-07 20:44:56,235][67871] Updated weights for policy 1, policy_version 26050 (0.0009) [2023-10-07 20:44:56,604][67871] Updated weights for policy 1, policy_version 26060 (0.0007) [2023-10-07 20:44:56,967][67871] Updated weights for policy 1, policy_version 26070 (0.0008) [2023-10-07 20:44:57,334][67871] Updated weights for policy 1, policy_version 26080 (0.0007) [2023-10-07 20:44:57,477][66916] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 53346304. Throughput: 0: 1657.2, 1: 1661.1. Samples: 13337444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:44:57,478][66916] Avg episode reward: [(0, '36.300'), (1, '35.080')] [2023-10-07 20:44:58,879][67838] Updated weights for policy 0, policy_version 26022 (0.0010) [2023-10-07 20:44:59,246][67838] Updated weights for policy 0, policy_version 26032 (0.0009) [2023-10-07 20:44:59,615][67838] Updated weights for policy 0, policy_version 26042 (0.0009) [2023-10-07 20:45:01,329][67871] Updated weights for policy 1, policy_version 26090 (0.0008) [2023-10-07 20:45:01,695][67871] Updated weights for policy 1, policy_version 26100 (0.0009) [2023-10-07 20:45:02,069][67871] Updated weights for policy 1, policy_version 26110 (0.0008) [2023-10-07 20:45:02,477][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 53411840. Throughput: 0: 1667.7, 1: 1670.5. Samples: 13358048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:45:02,478][66916] Avg episode reward: [(0, '36.320'), (1, '33.010')] [2023-10-07 20:45:03,454][67838] Updated weights for policy 0, policy_version 26052 (0.0008) [2023-10-07 20:45:03,819][67838] Updated weights for policy 0, policy_version 26062 (0.0010) [2023-10-07 20:45:04,206][67838] Updated weights for policy 0, policy_version 26072 (0.0008) [2023-10-07 20:45:06,154][67871] Updated weights for policy 1, policy_version 26120 (0.0007) [2023-10-07 20:45:06,523][67871] Updated weights for policy 1, policy_version 26130 (0.0007) [2023-10-07 20:45:06,885][67871] Updated weights for policy 1, policy_version 26140 (0.0008) [2023-10-07 20:45:07,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 53477376. Throughput: 0: 1672.2, 1: 1650.6. Samples: 13377558. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-07 20:45:07,477][66916] Avg episode reward: [(0, '38.450'), (1, '35.840')] [2023-10-07 20:45:07,487][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000026080_26705920.pth... [2023-10-07 20:45:07,487][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000026144_26771456.pth... [2023-10-07 20:45:07,520][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000024544_25133056.pth [2023-10-07 20:45:07,524][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000024576_25165824.pth [2023-10-07 20:45:08,333][67838] Updated weights for policy 0, policy_version 26082 (0.0010) [2023-10-07 20:45:08,718][67838] Updated weights for policy 0, policy_version 26092 (0.0008) [2023-10-07 20:45:09,096][67838] Updated weights for policy 0, policy_version 26102 (0.0009) [2023-10-07 20:45:09,466][67838] Updated weights for policy 0, policy_version 26112 (0.0008) [2023-10-07 20:45:11,081][67871] Updated weights for policy 1, policy_version 26150 (0.0009) [2023-10-07 20:45:11,467][67871] Updated weights for policy 1, policy_version 26160 (0.0007) [2023-10-07 20:45:11,831][67871] Updated weights for policy 1, policy_version 26170 (0.0008) [2023-10-07 20:45:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 53542912. Throughput: 0: 1660.6, 1: 1677.1. Samples: 13387630. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-07 20:45:12,477][66916] Avg episode reward: [(0, '34.830'), (1, '34.290')] [2023-10-07 20:45:13,785][67838] Updated weights for policy 0, policy_version 26122 (0.0008) [2023-10-07 20:45:14,170][67838] Updated weights for policy 0, policy_version 26132 (0.0007) [2023-10-07 20:45:14,546][67838] Updated weights for policy 0, policy_version 26142 (0.0009) [2023-10-07 20:45:15,859][67871] Updated weights for policy 1, policy_version 26180 (0.0008) [2023-10-07 20:45:16,230][67871] Updated weights for policy 1, policy_version 26190 (0.0008) [2023-10-07 20:45:16,604][67871] Updated weights for policy 1, policy_version 26200 (0.0007) [2023-10-07 20:45:17,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 53608448. Throughput: 0: 1674.9, 1: 1676.8. Samples: 13408038. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-07 20:45:17,478][66916] Avg episode reward: [(0, '37.480'), (1, '34.620')] [2023-10-07 20:45:18,761][67838] Updated weights for policy 0, policy_version 26152 (0.0009) [2023-10-07 20:45:19,131][67838] Updated weights for policy 0, policy_version 26162 (0.0009) [2023-10-07 20:45:19,503][67838] Updated weights for policy 0, policy_version 26172 (0.0009) [2023-10-07 20:45:20,818][67871] Updated weights for policy 1, policy_version 26210 (0.0008) [2023-10-07 20:45:21,189][67871] Updated weights for policy 1, policy_version 26220 (0.0008) [2023-10-07 20:45:21,570][67871] Updated weights for policy 1, policy_version 26230 (0.0009) [2023-10-07 20:45:21,932][67871] Updated weights for policy 1, policy_version 26240 (0.0009) [2023-10-07 20:45:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 53673984. Throughput: 0: 1670.5, 1: 1660.5. Samples: 13427452. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-07 20:45:22,477][66916] Avg episode reward: [(0, '37.270'), (1, '35.650')] [2023-10-07 20:45:23,650][67838] Updated weights for policy 0, policy_version 26182 (0.0009) [2023-10-07 20:45:24,017][67838] Updated weights for policy 0, policy_version 26192 (0.0009) [2023-10-07 20:45:24,386][67838] Updated weights for policy 0, policy_version 26202 (0.0008) [2023-10-07 20:45:26,027][67871] Updated weights for policy 1, policy_version 26250 (0.0011) [2023-10-07 20:45:26,398][67871] Updated weights for policy 1, policy_version 26260 (0.0009) [2023-10-07 20:45:26,759][67871] Updated weights for policy 1, policy_version 26270 (0.0009) [2023-10-07 20:45:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 53739520. Throughput: 0: 1668.2, 1: 1687.8. Samples: 13438010. Policy #0 lag: (min: 31.0, avg: 45.4, max: 63.0) [2023-10-07 20:45:27,477][66916] Avg episode reward: [(0, '36.310'), (1, '33.630')] [2023-10-07 20:45:28,430][67838] Updated weights for policy 0, policy_version 26212 (0.0008) [2023-10-07 20:45:28,797][67838] Updated weights for policy 0, policy_version 26222 (0.0007) [2023-10-07 20:45:29,175][67838] Updated weights for policy 0, policy_version 26232 (0.0007) [2023-10-07 20:45:30,935][67871] Updated weights for policy 1, policy_version 26280 (0.0008) [2023-10-07 20:45:31,308][67871] Updated weights for policy 1, policy_version 26290 (0.0007) [2023-10-07 20:45:31,684][67871] Updated weights for policy 1, policy_version 26300 (0.0007) [2023-10-07 20:45:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 53805056. Throughput: 0: 1668.0, 1: 1680.8. Samples: 13458300. Policy #0 lag: (min: 31.0, avg: 45.4, max: 63.0) [2023-10-07 20:45:32,478][66916] Avg episode reward: [(0, '39.000'), (1, '35.280')] [2023-10-07 20:45:33,169][67838] Updated weights for policy 0, policy_version 26242 (0.0009) [2023-10-07 20:45:33,533][67838] Updated weights for policy 0, policy_version 26252 (0.0007) [2023-10-07 20:45:33,922][67838] Updated weights for policy 0, policy_version 26262 (0.0009) [2023-10-07 20:45:34,285][67838] Updated weights for policy 0, policy_version 26272 (0.0009) [2023-10-07 20:45:35,884][67871] Updated weights for policy 1, policy_version 26310 (0.0007) [2023-10-07 20:45:36,248][67871] Updated weights for policy 1, policy_version 26320 (0.0007) [2023-10-07 20:45:36,621][67871] Updated weights for policy 1, policy_version 26330 (0.0007) [2023-10-07 20:45:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 53870592. Throughput: 0: 1677.1, 1: 1665.2. Samples: 13477926. Policy #0 lag: (min: 31.0, avg: 45.4, max: 63.0) [2023-10-07 20:45:37,478][66916] Avg episode reward: [(0, '36.530'), (1, '35.240')] [2023-10-07 20:45:38,358][67838] Updated weights for policy 0, policy_version 26282 (0.0009) [2023-10-07 20:45:38,729][67838] Updated weights for policy 0, policy_version 26292 (0.0008) [2023-10-07 20:45:39,091][67838] Updated weights for policy 0, policy_version 26302 (0.0007) [2023-10-07 20:45:40,604][67871] Updated weights for policy 1, policy_version 26340 (0.0007) [2023-10-07 20:45:40,966][67871] Updated weights for policy 1, policy_version 26350 (0.0007) [2023-10-07 20:45:41,323][67871] Updated weights for policy 1, policy_version 26360 (0.0007) [2023-10-07 20:45:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 53936128. Throughput: 0: 1672.6, 1: 1680.6. Samples: 13488336. Policy #0 lag: (min: 31.0, avg: 45.4, max: 63.0) [2023-10-07 20:45:42,478][66916] Avg episode reward: [(0, '38.050'), (1, '34.840')] [2023-10-07 20:45:43,182][67838] Updated weights for policy 0, policy_version 26312 (0.0009) [2023-10-07 20:45:43,554][67838] Updated weights for policy 0, policy_version 26322 (0.0010) [2023-10-07 20:45:43,931][67838] Updated weights for policy 0, policy_version 26332 (0.0009) [2023-10-07 20:45:45,373][67871] Updated weights for policy 1, policy_version 26370 (0.0009) [2023-10-07 20:45:45,748][67871] Updated weights for policy 1, policy_version 26380 (0.0008) [2023-10-07 20:45:46,120][67871] Updated weights for policy 1, policy_version 26390 (0.0009) [2023-10-07 20:45:46,480][67871] Updated weights for policy 1, policy_version 26400 (0.0009) [2023-10-07 20:45:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 54001664. Throughput: 0: 1676.5, 1: 1660.6. Samples: 13508216. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 20:45:47,477][66916] Avg episode reward: [(0, '37.090'), (1, '35.840')] [2023-10-07 20:45:47,979][67838] Updated weights for policy 0, policy_version 26342 (0.0007) [2023-10-07 20:45:48,353][67838] Updated weights for policy 0, policy_version 26352 (0.0008) [2023-10-07 20:45:48,720][67838] Updated weights for policy 0, policy_version 26362 (0.0007) [2023-10-07 20:45:50,387][67871] Updated weights for policy 1, policy_version 26410 (0.0008) [2023-10-07 20:45:50,758][67871] Updated weights for policy 1, policy_version 26420 (0.0008) [2023-10-07 20:45:51,121][67871] Updated weights for policy 1, policy_version 26430 (0.0007) [2023-10-07 20:45:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 54067200. Throughput: 0: 1675.5, 1: 1672.9. Samples: 13528238. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 20:45:52,478][66916] Avg episode reward: [(0, '38.220'), (1, '33.630')] [2023-10-07 20:45:52,815][67838] Updated weights for policy 0, policy_version 26372 (0.0009) [2023-10-07 20:45:53,206][67838] Updated weights for policy 0, policy_version 26382 (0.0008) [2023-10-07 20:45:53,579][67838] Updated weights for policy 0, policy_version 26392 (0.0008) [2023-10-07 20:45:55,298][67871] Updated weights for policy 1, policy_version 26440 (0.0008) [2023-10-07 20:45:55,663][67871] Updated weights for policy 1, policy_version 26450 (0.0008) [2023-10-07 20:45:56,035][67871] Updated weights for policy 1, policy_version 26460 (0.0010) [2023-10-07 20:45:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 54132736. Throughput: 0: 1675.5, 1: 1674.1. Samples: 13538360. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 20:45:57,478][66916] Avg episode reward: [(0, '38.180'), (1, '36.030')] [2023-10-07 20:45:57,652][67838] Updated weights for policy 0, policy_version 26402 (0.0009) [2023-10-07 20:45:58,026][67838] Updated weights for policy 0, policy_version 26412 (0.0009) [2023-10-07 20:45:58,397][67838] Updated weights for policy 0, policy_version 26422 (0.0008) [2023-10-07 20:45:58,769][67838] Updated weights for policy 0, policy_version 26432 (0.0008) [2023-10-07 20:46:00,330][67871] Updated weights for policy 1, policy_version 26470 (0.0008) [2023-10-07 20:46:00,718][67871] Updated weights for policy 1, policy_version 26480 (0.0008) [2023-10-07 20:46:01,093][67871] Updated weights for policy 1, policy_version 26490 (0.0011) [2023-10-07 20:46:02,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54198272. Throughput: 0: 1674.8, 1: 1659.3. Samples: 13558072. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 20:46:02,477][66916] Avg episode reward: [(0, '36.180'), (1, '34.630')] [2023-10-07 20:46:02,811][67838] Updated weights for policy 0, policy_version 26442 (0.0008) [2023-10-07 20:46:03,187][67838] Updated weights for policy 0, policy_version 26452 (0.0008) [2023-10-07 20:46:03,575][67838] Updated weights for policy 0, policy_version 26462 (0.0010) [2023-10-07 20:46:05,057][67871] Updated weights for policy 1, policy_version 26500 (0.0009) [2023-10-07 20:46:05,423][67871] Updated weights for policy 1, policy_version 26510 (0.0011) [2023-10-07 20:46:05,789][67871] Updated weights for policy 1, policy_version 26520 (0.0010) [2023-10-07 20:46:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 54263808. Throughput: 0: 1679.5, 1: 1670.8. Samples: 13578216. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-07 20:46:07,478][66916] Avg episode reward: [(0, '38.510'), (1, '35.240')] [2023-10-07 20:46:07,578][67838] Updated weights for policy 0, policy_version 26472 (0.0008) [2023-10-07 20:46:07,954][67838] Updated weights for policy 0, policy_version 26482 (0.0007) [2023-10-07 20:46:08,334][67838] Updated weights for policy 0, policy_version 26492 (0.0008) [2023-10-07 20:46:09,896][67871] Updated weights for policy 1, policy_version 26530 (0.0007) [2023-10-07 20:46:10,264][67871] Updated weights for policy 1, policy_version 26540 (0.0009) [2023-10-07 20:46:10,637][67871] Updated weights for policy 1, policy_version 26550 (0.0009) [2023-10-07 20:46:10,997][67871] Updated weights for policy 1, policy_version 26560 (0.0009) [2023-10-07 20:46:12,389][67838] Updated weights for policy 0, policy_version 26502 (0.0008) [2023-10-07 20:46:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54329344. Throughput: 0: 1677.1, 1: 1666.6. Samples: 13588474. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-07 20:46:12,477][66916] Avg episode reward: [(0, '36.110'), (1, '34.290')] [2023-10-07 20:46:12,753][67838] Updated weights for policy 0, policy_version 26512 (0.0007) [2023-10-07 20:46:13,122][67838] Updated weights for policy 0, policy_version 26522 (0.0007) [2023-10-07 20:46:15,270][67871] Updated weights for policy 1, policy_version 26570 (0.0008) [2023-10-07 20:46:15,651][67871] Updated weights for policy 1, policy_version 26580 (0.0010) [2023-10-07 20:46:16,014][67871] Updated weights for policy 1, policy_version 26590 (0.0011) [2023-10-07 20:46:17,085][67838] Updated weights for policy 0, policy_version 26532 (0.0007) [2023-10-07 20:46:17,466][67838] Updated weights for policy 0, policy_version 26542 (0.0007) [2023-10-07 20:46:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54394880. Throughput: 0: 1679.8, 1: 1655.8. Samples: 13608402. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-07 20:46:17,477][66916] Avg episode reward: [(0, '37.150'), (1, '35.050')] [2023-10-07 20:46:17,842][67838] Updated weights for policy 0, policy_version 26552 (0.0007) [2023-10-07 20:46:20,085][67871] Updated weights for policy 1, policy_version 26600 (0.0007) [2023-10-07 20:46:20,453][67871] Updated weights for policy 1, policy_version 26610 (0.0009) [2023-10-07 20:46:20,826][67871] Updated weights for policy 1, policy_version 26620 (0.0008) [2023-10-07 20:46:21,848][67838] Updated weights for policy 0, policy_version 26562 (0.0008) [2023-10-07 20:46:22,231][67838] Updated weights for policy 0, policy_version 26572 (0.0009) [2023-10-07 20:46:22,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 54460416. Throughput: 0: 1672.4, 1: 1671.3. Samples: 13628392. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-07 20:46:22,478][66916] Avg episode reward: [(0, '37.360'), (1, '36.200')] [2023-10-07 20:46:22,603][67838] Updated weights for policy 0, policy_version 26582 (0.0009) [2023-10-07 20:46:22,972][67838] Updated weights for policy 0, policy_version 26592 (0.0010) [2023-10-07 20:46:24,762][67871] Updated weights for policy 1, policy_version 26630 (0.0008) [2023-10-07 20:46:25,136][67871] Updated weights for policy 1, policy_version 26640 (0.0008) [2023-10-07 20:46:25,499][67871] Updated weights for policy 1, policy_version 26650 (0.0008) [2023-10-07 20:46:27,067][67838] Updated weights for policy 0, policy_version 26602 (0.0009) [2023-10-07 20:46:27,437][67838] Updated weights for policy 0, policy_version 26612 (0.0010) [2023-10-07 20:46:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54525952. Throughput: 0: 1680.2, 1: 1668.1. Samples: 13639008. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 20:46:27,478][66916] Avg episode reward: [(0, '37.080'), (1, '34.140')] [2023-10-07 20:46:27,814][67838] Updated weights for policy 0, policy_version 26622 (0.0011) [2023-10-07 20:46:29,518][67871] Updated weights for policy 1, policy_version 26660 (0.0010) [2023-10-07 20:46:29,890][67871] Updated weights for policy 1, policy_version 26670 (0.0008) [2023-10-07 20:46:30,254][67871] Updated weights for policy 1, policy_version 26680 (0.0008) [2023-10-07 20:46:31,906][67838] Updated weights for policy 0, policy_version 26632 (0.0010) [2023-10-07 20:46:32,288][67838] Updated weights for policy 0, policy_version 26642 (0.0009) [2023-10-07 20:46:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54591488. Throughput: 0: 1679.2, 1: 1662.0. Samples: 13658570. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 20:46:32,478][66916] Avg episode reward: [(0, '36.970'), (1, '33.520')] [2023-10-07 20:46:32,656][67838] Updated weights for policy 0, policy_version 26652 (0.0008) [2023-10-07 20:46:34,221][67871] Updated weights for policy 1, policy_version 26690 (0.0009) [2023-10-07 20:46:34,590][67871] Updated weights for policy 1, policy_version 26700 (0.0009) [2023-10-07 20:46:34,958][67871] Updated weights for policy 1, policy_version 26710 (0.0008) [2023-10-07 20:46:35,329][67871] Updated weights for policy 1, policy_version 26720 (0.0008) [2023-10-07 20:46:36,641][67838] Updated weights for policy 0, policy_version 26662 (0.0007) [2023-10-07 20:46:37,008][67838] Updated weights for policy 0, policy_version 26672 (0.0010) [2023-10-07 20:46:37,396][67838] Updated weights for policy 0, policy_version 26682 (0.0008) [2023-10-07 20:46:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 54657024. Throughput: 0: 1667.0, 1: 1675.1. Samples: 13678632. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 20:46:37,478][66916] Avg episode reward: [(0, '39.900'), (1, '35.210')] [2023-10-07 20:46:39,515][67871] Updated weights for policy 1, policy_version 26730 (0.0007) [2023-10-07 20:46:39,889][67871] Updated weights for policy 1, policy_version 26740 (0.0007) [2023-10-07 20:46:40,264][67871] Updated weights for policy 1, policy_version 26750 (0.0007) [2023-10-07 20:46:41,617][67838] Updated weights for policy 0, policy_version 26692 (0.0009) [2023-10-07 20:46:41,992][67838] Updated weights for policy 0, policy_version 26702 (0.0008) [2023-10-07 20:46:42,367][67838] Updated weights for policy 0, policy_version 26712 (0.0007) [2023-10-07 20:46:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54722560. Throughput: 0: 1682.2, 1: 1661.2. Samples: 13688814. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 20:46:42,477][66916] Avg episode reward: [(0, '38.900'), (1, '33.810')] [2023-10-07 20:46:44,405][67871] Updated weights for policy 1, policy_version 26760 (0.0010) [2023-10-07 20:46:44,776][67871] Updated weights for policy 1, policy_version 26770 (0.0009) [2023-10-07 20:46:45,134][67871] Updated weights for policy 1, policy_version 26780 (0.0009) [2023-10-07 20:46:46,356][67838] Updated weights for policy 0, policy_version 26722 (0.0007) [2023-10-07 20:46:46,724][67838] Updated weights for policy 0, policy_version 26732 (0.0010) [2023-10-07 20:46:47,106][67838] Updated weights for policy 0, policy_version 26742 (0.0007) [2023-10-07 20:46:47,474][67838] Updated weights for policy 0, policy_version 26752 (0.0008) [2023-10-07 20:46:47,476][66916] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 54820864. Throughput: 0: 1686.1, 1: 1662.0. Samples: 13708738. Policy #0 lag: (min: 24.0, avg: 45.7, max: 56.0) [2023-10-07 20:46:47,477][66916] Avg episode reward: [(0, '39.850'), (1, '35.990')] [2023-10-07 20:46:49,384][67871] Updated weights for policy 1, policy_version 26790 (0.0010) [2023-10-07 20:46:49,784][67871] Updated weights for policy 1, policy_version 26800 (0.0008) [2023-10-07 20:46:50,157][67871] Updated weights for policy 1, policy_version 26810 (0.0009) [2023-10-07 20:46:51,499][67838] Updated weights for policy 0, policy_version 26762 (0.0008) [2023-10-07 20:46:51,869][67838] Updated weights for policy 0, policy_version 26772 (0.0010) [2023-10-07 20:46:52,236][67838] Updated weights for policy 0, policy_version 26782 (0.0007) [2023-10-07 20:46:52,476][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 54886400. Throughput: 0: 1665.9, 1: 1672.5. Samples: 13728444. Policy #0 lag: (min: 24.0, avg: 45.7, max: 56.0) [2023-10-07 20:46:52,477][66916] Avg episode reward: [(0, '39.770'), (1, '37.560')] [2023-10-07 20:46:52,486][67676] Saving new best policy, reward=37.560! [2023-10-07 20:46:54,171][67871] Updated weights for policy 1, policy_version 26820 (0.0009) [2023-10-07 20:46:54,540][67871] Updated weights for policy 1, policy_version 26830 (0.0009) [2023-10-07 20:46:54,907][67871] Updated weights for policy 1, policy_version 26840 (0.0007) [2023-10-07 20:46:56,326][67838] Updated weights for policy 0, policy_version 26792 (0.0007) [2023-10-07 20:46:56,705][67838] Updated weights for policy 0, policy_version 26802 (0.0008) [2023-10-07 20:46:57,082][67838] Updated weights for policy 0, policy_version 26812 (0.0009) [2023-10-07 20:46:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 54951936. Throughput: 0: 1691.1, 1: 1654.8. Samples: 13739042. Policy #0 lag: (min: 24.0, avg: 45.7, max: 56.0) [2023-10-07 20:46:57,477][66916] Avg episode reward: [(0, '38.340'), (1, '34.530')] [2023-10-07 20:46:58,922][67871] Updated weights for policy 1, policy_version 26850 (0.0009) [2023-10-07 20:46:59,282][67871] Updated weights for policy 1, policy_version 26860 (0.0007) [2023-10-07 20:46:59,655][67871] Updated weights for policy 1, policy_version 26870 (0.0007) [2023-10-07 20:47:00,014][67871] Updated weights for policy 1, policy_version 26880 (0.0008) [2023-10-07 20:47:01,095][67838] Updated weights for policy 0, policy_version 26822 (0.0008) [2023-10-07 20:47:01,464][67838] Updated weights for policy 0, policy_version 26832 (0.0009) [2023-10-07 20:47:01,832][67838] Updated weights for policy 0, policy_version 26842 (0.0008) [2023-10-07 20:47:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 55017472. Throughput: 0: 1679.1, 1: 1666.3. Samples: 13758946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:47:02,478][66916] Avg episode reward: [(0, '37.310'), (1, '35.870')] [2023-10-07 20:47:04,175][67871] Updated weights for policy 1, policy_version 26890 (0.0009) [2023-10-07 20:47:04,545][67871] Updated weights for policy 1, policy_version 26900 (0.0007) [2023-10-07 20:47:04,914][67871] Updated weights for policy 1, policy_version 26910 (0.0007) [2023-10-07 20:47:05,974][67838] Updated weights for policy 0, policy_version 26852 (0.0008) [2023-10-07 20:47:06,354][67838] Updated weights for policy 0, policy_version 26862 (0.0008) [2023-10-07 20:47:06,722][67838] Updated weights for policy 0, policy_version 26872 (0.0009) [2023-10-07 20:47:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 55083008. Throughput: 0: 1660.6, 1: 1672.2. Samples: 13778370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:47:07,477][66916] Avg episode reward: [(0, '36.730'), (1, '37.300')] [2023-10-07 20:47:07,487][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000026880_27525120.pth... [2023-10-07 20:47:07,487][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000026912_27557888.pth... [2023-10-07 20:47:07,525][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000025312_25919488.pth [2023-10-07 20:47:07,526][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000025344_25952256.pth [2023-10-07 20:47:08,842][67871] Updated weights for policy 1, policy_version 26920 (0.0007) [2023-10-07 20:47:09,211][67871] Updated weights for policy 1, policy_version 26930 (0.0007) [2023-10-07 20:47:09,568][67871] Updated weights for policy 1, policy_version 26940 (0.0007) [2023-10-07 20:47:10,705][67838] Updated weights for policy 0, policy_version 26882 (0.0009) [2023-10-07 20:47:11,081][67838] Updated weights for policy 0, policy_version 26892 (0.0009) [2023-10-07 20:47:11,454][67838] Updated weights for policy 0, policy_version 26902 (0.0008) [2023-10-07 20:47:11,822][67838] Updated weights for policy 0, policy_version 26912 (0.0010) [2023-10-07 20:47:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 55148544. Throughput: 0: 1683.9, 1: 1649.8. Samples: 13789026. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:47:12,477][66916] Avg episode reward: [(0, '36.460'), (1, '35.080')] [2023-10-07 20:47:13,695][67871] Updated weights for policy 1, policy_version 26950 (0.0008) [2023-10-07 20:47:14,073][67871] Updated weights for policy 1, policy_version 26960 (0.0009) [2023-10-07 20:47:14,431][67871] Updated weights for policy 1, policy_version 26970 (0.0011) [2023-10-07 20:47:15,989][67838] Updated weights for policy 0, policy_version 26922 (0.0007) [2023-10-07 20:47:16,365][67838] Updated weights for policy 0, policy_version 26932 (0.0007) [2023-10-07 20:47:16,738][67838] Updated weights for policy 0, policy_version 26942 (0.0007) [2023-10-07 20:47:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 55214080. Throughput: 0: 1673.0, 1: 1669.2. Samples: 13808970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:47:17,478][66916] Avg episode reward: [(0, '39.790'), (1, '34.540')] [2023-10-07 20:47:18,554][67871] Updated weights for policy 1, policy_version 26980 (0.0010) [2023-10-07 20:47:18,912][67871] Updated weights for policy 1, policy_version 26990 (0.0008) [2023-10-07 20:47:19,290][67871] Updated weights for policy 1, policy_version 27000 (0.0009) [2023-10-07 20:47:20,743][67838] Updated weights for policy 0, policy_version 26952 (0.0009) [2023-10-07 20:47:21,127][67838] Updated weights for policy 0, policy_version 26962 (0.0007) [2023-10-07 20:47:21,491][67838] Updated weights for policy 0, policy_version 26972 (0.0009) [2023-10-07 20:47:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 55279616. Throughput: 0: 1668.9, 1: 1667.1. Samples: 13828754. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-07 20:47:22,478][66916] Avg episode reward: [(0, '41.770'), (1, '36.230')] [2023-10-07 20:47:22,486][67511] Saving new best policy, reward=41.770! [2023-10-07 20:47:23,703][67871] Updated weights for policy 1, policy_version 27010 (0.0008) [2023-10-07 20:47:24,074][67871] Updated weights for policy 1, policy_version 27020 (0.0010) [2023-10-07 20:47:24,445][67871] Updated weights for policy 1, policy_version 27030 (0.0008) [2023-10-07 20:47:24,823][67871] Updated weights for policy 1, policy_version 27040 (0.0008) [2023-10-07 20:47:25,515][67838] Updated weights for policy 0, policy_version 26982 (0.0010) [2023-10-07 20:47:25,884][67838] Updated weights for policy 0, policy_version 26992 (0.0010) [2023-10-07 20:47:26,257][67838] Updated weights for policy 0, policy_version 27002 (0.0008) [2023-10-07 20:47:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 55345152. Throughput: 0: 1683.9, 1: 1652.6. Samples: 13838956. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-07 20:47:27,477][66916] Avg episode reward: [(0, '43.980'), (1, '34.100')] [2023-10-07 20:47:27,478][67511] Saving new best policy, reward=43.980! [2023-10-07 20:47:28,781][67871] Updated weights for policy 1, policy_version 27050 (0.0009) [2023-10-07 20:47:29,151][67871] Updated weights for policy 1, policy_version 27060 (0.0008) [2023-10-07 20:47:29,524][67871] Updated weights for policy 1, policy_version 27070 (0.0008) [2023-10-07 20:47:30,462][67838] Updated weights for policy 0, policy_version 27012 (0.0009) [2023-10-07 20:47:30,858][67838] Updated weights for policy 0, policy_version 27022 (0.0010) [2023-10-07 20:47:31,241][67838] Updated weights for policy 0, policy_version 27032 (0.0010) [2023-10-07 20:47:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 55410688. Throughput: 0: 1659.0, 1: 1672.8. Samples: 13858668. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-07 20:47:32,477][66916] Avg episode reward: [(0, '41.330'), (1, '35.330')] [2023-10-07 20:47:33,556][67871] Updated weights for policy 1, policy_version 27080 (0.0010) [2023-10-07 20:47:33,924][67871] Updated weights for policy 1, policy_version 27090 (0.0008) [2023-10-07 20:47:34,293][67871] Updated weights for policy 1, policy_version 27100 (0.0008) [2023-10-07 20:47:35,428][67838] Updated weights for policy 0, policy_version 27042 (0.0009) [2023-10-07 20:47:35,797][67838] Updated weights for policy 0, policy_version 27052 (0.0010) [2023-10-07 20:47:36,174][67838] Updated weights for policy 0, policy_version 27062 (0.0009) [2023-10-07 20:47:36,557][67838] Updated weights for policy 0, policy_version 27072 (0.0009) [2023-10-07 20:47:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 55476224. Throughput: 0: 1661.6, 1: 1672.9. Samples: 13878498. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-07 20:47:37,478][66916] Avg episode reward: [(0, '43.160'), (1, '34.880')] [2023-10-07 20:47:38,464][67871] Updated weights for policy 1, policy_version 27110 (0.0010) [2023-10-07 20:47:38,849][67871] Updated weights for policy 1, policy_version 27120 (0.0009) [2023-10-07 20:47:39,220][67871] Updated weights for policy 1, policy_version 27130 (0.0008) [2023-10-07 20:47:40,500][67838] Updated weights for policy 0, policy_version 27082 (0.0010) [2023-10-07 20:47:40,871][67838] Updated weights for policy 0, policy_version 27092 (0.0009) [2023-10-07 20:47:41,237][67838] Updated weights for policy 0, policy_version 27102 (0.0007) [2023-10-07 20:47:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 55541760. Throughput: 0: 1670.3, 1: 1656.3. Samples: 13888738. Policy #0 lag: (min: 8.0, avg: 32.4, max: 40.0) [2023-10-07 20:47:42,477][66916] Avg episode reward: [(0, '40.880'), (1, '35.630')] [2023-10-07 20:47:43,402][67871] Updated weights for policy 1, policy_version 27140 (0.0008) [2023-10-07 20:47:43,773][67871] Updated weights for policy 1, policy_version 27150 (0.0009) [2023-10-07 20:47:44,144][67871] Updated weights for policy 1, policy_version 27160 (0.0009) [2023-10-07 20:47:45,416][67838] Updated weights for policy 0, policy_version 27112 (0.0009) [2023-10-07 20:47:45,789][67838] Updated weights for policy 0, policy_version 27122 (0.0009) [2023-10-07 20:47:46,161][67838] Updated weights for policy 0, policy_version 27132 (0.0008) [2023-10-07 20:47:47,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55607296. Throughput: 0: 1652.9, 1: 1663.3. Samples: 13908174. Policy #0 lag: (min: 8.0, avg: 32.4, max: 40.0) [2023-10-07 20:47:47,478][66916] Avg episode reward: [(0, '38.240'), (1, '35.480')] [2023-10-07 20:47:48,197][67871] Updated weights for policy 1, policy_version 27170 (0.0009) [2023-10-07 20:47:48,578][67871] Updated weights for policy 1, policy_version 27180 (0.0010) [2023-10-07 20:47:48,947][67871] Updated weights for policy 1, policy_version 27190 (0.0009) [2023-10-07 20:47:49,320][67871] Updated weights for policy 1, policy_version 27200 (0.0009) [2023-10-07 20:47:50,350][67838] Updated weights for policy 0, policy_version 27142 (0.0010) [2023-10-07 20:47:50,730][67838] Updated weights for policy 0, policy_version 27152 (0.0009) [2023-10-07 20:47:51,101][67838] Updated weights for policy 0, policy_version 27162 (0.0007) [2023-10-07 20:47:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55672832. Throughput: 0: 1664.8, 1: 1665.7. Samples: 13928244. Policy #0 lag: (min: 8.0, avg: 32.4, max: 40.0) [2023-10-07 20:47:52,477][66916] Avg episode reward: [(0, '41.660'), (1, '35.580')] [2023-10-07 20:47:53,543][67871] Updated weights for policy 1, policy_version 27210 (0.0007) [2023-10-07 20:47:53,910][67871] Updated weights for policy 1, policy_version 27220 (0.0009) [2023-10-07 20:47:54,278][67871] Updated weights for policy 1, policy_version 27230 (0.0007) [2023-10-07 20:47:55,114][67838] Updated weights for policy 0, policy_version 27172 (0.0008) [2023-10-07 20:47:55,487][67838] Updated weights for policy 0, policy_version 27182 (0.0011) [2023-10-07 20:47:55,859][67838] Updated weights for policy 0, policy_version 27192 (0.0011) [2023-10-07 20:47:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55738368. Throughput: 0: 1660.0, 1: 1663.4. Samples: 13938582. Policy #0 lag: (min: 8.0, avg: 32.4, max: 40.0) [2023-10-07 20:47:57,477][66916] Avg episode reward: [(0, '37.650'), (1, '36.080')] [2023-10-07 20:47:58,339][67871] Updated weights for policy 1, policy_version 27240 (0.0007) [2023-10-07 20:47:58,705][67871] Updated weights for policy 1, policy_version 27250 (0.0007) [2023-10-07 20:47:59,083][67871] Updated weights for policy 1, policy_version 27260 (0.0009) [2023-10-07 20:47:59,927][67838] Updated weights for policy 0, policy_version 27202 (0.0011) [2023-10-07 20:48:00,288][67838] Updated weights for policy 0, policy_version 27212 (0.0009) [2023-10-07 20:48:00,665][67838] Updated weights for policy 0, policy_version 27222 (0.0008) [2023-10-07 20:48:01,038][67838] Updated weights for policy 0, policy_version 27232 (0.0008) [2023-10-07 20:48:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55803904. Throughput: 0: 1651.2, 1: 1666.0. Samples: 13958242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:48:02,477][66916] Avg episode reward: [(0, '39.790'), (1, '36.450')] [2023-10-07 20:48:03,204][67871] Updated weights for policy 1, policy_version 27270 (0.0008) [2023-10-07 20:48:03,569][67871] Updated weights for policy 1, policy_version 27280 (0.0008) [2023-10-07 20:48:03,947][67871] Updated weights for policy 1, policy_version 27290 (0.0007) [2023-10-07 20:48:05,268][67838] Updated weights for policy 0, policy_version 27242 (0.0009) [2023-10-07 20:48:05,636][67838] Updated weights for policy 0, policy_version 27252 (0.0008) [2023-10-07 20:48:06,018][67838] Updated weights for policy 0, policy_version 27262 (0.0010) [2023-10-07 20:48:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 55869440. Throughput: 0: 1664.7, 1: 1669.1. Samples: 13978776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:48:07,478][66916] Avg episode reward: [(0, '40.060'), (1, '35.610')] [2023-10-07 20:48:07,975][67871] Updated weights for policy 1, policy_version 27300 (0.0007) [2023-10-07 20:48:08,338][67871] Updated weights for policy 1, policy_version 27310 (0.0008) [2023-10-07 20:48:08,704][67871] Updated weights for policy 1, policy_version 27320 (0.0007) [2023-10-07 20:48:10,037][67838] Updated weights for policy 0, policy_version 27272 (0.0009) [2023-10-07 20:48:10,402][67838] Updated weights for policy 0, policy_version 27282 (0.0010) [2023-10-07 20:48:10,780][67838] Updated weights for policy 0, policy_version 27292 (0.0008) [2023-10-07 20:48:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55934976. Throughput: 0: 1658.0, 1: 1673.9. Samples: 13988892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:48:12,477][66916] Avg episode reward: [(0, '42.850'), (1, '35.570')] [2023-10-07 20:48:12,740][67871] Updated weights for policy 1, policy_version 27330 (0.0008) [2023-10-07 20:48:13,110][67871] Updated weights for policy 1, policy_version 27340 (0.0007) [2023-10-07 20:48:13,470][67871] Updated weights for policy 1, policy_version 27350 (0.0007) [2023-10-07 20:48:13,849][67871] Updated weights for policy 1, policy_version 27360 (0.0007) [2023-10-07 20:48:14,826][67838] Updated weights for policy 0, policy_version 27302 (0.0008) [2023-10-07 20:48:15,200][67838] Updated weights for policy 0, policy_version 27312 (0.0008) [2023-10-07 20:48:15,579][67838] Updated weights for policy 0, policy_version 27322 (0.0009) [2023-10-07 20:48:17,477][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56000512. Throughput: 0: 1657.2, 1: 1674.1. Samples: 14008578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:48:17,478][66916] Avg episode reward: [(0, '43.180'), (1, '36.930')] [2023-10-07 20:48:17,940][67871] Updated weights for policy 1, policy_version 27370 (0.0010) [2023-10-07 20:48:18,320][67871] Updated weights for policy 1, policy_version 27380 (0.0010) [2023-10-07 20:48:18,681][67871] Updated weights for policy 1, policy_version 27390 (0.0008) [2023-10-07 20:48:19,842][67838] Updated weights for policy 0, policy_version 27332 (0.0007) [2023-10-07 20:48:20,231][67838] Updated weights for policy 0, policy_version 27342 (0.0011) [2023-10-07 20:48:20,601][67838] Updated weights for policy 0, policy_version 27352 (0.0010) [2023-10-07 20:48:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56066048. Throughput: 0: 1666.2, 1: 1673.2. Samples: 14028770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:48:22,477][66916] Avg episode reward: [(0, '39.370'), (1, '36.170')] [2023-10-07 20:48:22,749][67871] Updated weights for policy 1, policy_version 27400 (0.0010) [2023-10-07 20:48:23,123][67871] Updated weights for policy 1, policy_version 27410 (0.0010) [2023-10-07 20:48:23,489][67871] Updated weights for policy 1, policy_version 27420 (0.0007) [2023-10-07 20:48:24,826][67838] Updated weights for policy 0, policy_version 27362 (0.0011) [2023-10-07 20:48:25,201][67838] Updated weights for policy 0, policy_version 27372 (0.0010) [2023-10-07 20:48:25,576][67838] Updated weights for policy 0, policy_version 27382 (0.0009) [2023-10-07 20:48:25,947][67838] Updated weights for policy 0, policy_version 27392 (0.0009) [2023-10-07 20:48:27,445][67871] Updated weights for policy 1, policy_version 27430 (0.0008) [2023-10-07 20:48:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56131584. Throughput: 0: 1652.4, 1: 1675.8. Samples: 14038506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:48:27,477][66916] Avg episode reward: [(0, '40.660'), (1, '36.940')] [2023-10-07 20:48:27,824][67871] Updated weights for policy 1, policy_version 27440 (0.0007) [2023-10-07 20:48:28,188][67871] Updated weights for policy 1, policy_version 27450 (0.0007) [2023-10-07 20:48:29,955][67838] Updated weights for policy 0, policy_version 27402 (0.0009) [2023-10-07 20:48:30,333][67838] Updated weights for policy 0, policy_version 27412 (0.0009) [2023-10-07 20:48:30,713][67838] Updated weights for policy 0, policy_version 27422 (0.0009) [2023-10-07 20:48:32,386][67871] Updated weights for policy 1, policy_version 27460 (0.0008) [2023-10-07 20:48:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56197120. Throughput: 0: 1652.9, 1: 1680.6. Samples: 14058178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:48:32,477][66916] Avg episode reward: [(0, '41.750'), (1, '37.990')] [2023-10-07 20:48:32,743][67871] Updated weights for policy 1, policy_version 27470 (0.0008) [2023-10-07 20:48:33,107][67871] Updated weights for policy 1, policy_version 27480 (0.0008) [2023-10-07 20:48:33,406][67676] Saving new best policy, reward=37.990! [2023-10-07 20:48:34,969][67838] Updated weights for policy 0, policy_version 27432 (0.0008) [2023-10-07 20:48:35,343][67838] Updated weights for policy 0, policy_version 27442 (0.0010) [2023-10-07 20:48:35,721][67838] Updated weights for policy 0, policy_version 27452 (0.0009) [2023-10-07 20:48:37,211][67871] Updated weights for policy 1, policy_version 27490 (0.0008) [2023-10-07 20:48:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56262656. Throughput: 0: 1658.5, 1: 1680.7. Samples: 14078510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:48:37,477][66916] Avg episode reward: [(0, '38.630'), (1, '35.800')] [2023-10-07 20:48:37,574][67871] Updated weights for policy 1, policy_version 27500 (0.0009) [2023-10-07 20:48:37,952][67871] Updated weights for policy 1, policy_version 27510 (0.0009) [2023-10-07 20:48:38,318][67871] Updated weights for policy 1, policy_version 27520 (0.0010) [2023-10-07 20:48:39,704][67838] Updated weights for policy 0, policy_version 27462 (0.0009) [2023-10-07 20:48:40,073][67838] Updated weights for policy 0, policy_version 27472 (0.0009) [2023-10-07 20:48:40,455][67838] Updated weights for policy 0, policy_version 27482 (0.0010) [2023-10-07 20:48:42,464][67871] Updated weights for policy 1, policy_version 27530 (0.0007) [2023-10-07 20:48:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56328192. Throughput: 0: 1650.0, 1: 1676.8. Samples: 14088286. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-07 20:48:42,477][66916] Avg episode reward: [(0, '41.480'), (1, '37.940')] [2023-10-07 20:48:42,840][67871] Updated weights for policy 1, policy_version 27540 (0.0009) [2023-10-07 20:48:43,203][67871] Updated weights for policy 1, policy_version 27550 (0.0009) [2023-10-07 20:48:44,507][67838] Updated weights for policy 0, policy_version 27492 (0.0009) [2023-10-07 20:48:44,885][67838] Updated weights for policy 0, policy_version 27502 (0.0007) [2023-10-07 20:48:45,270][67838] Updated weights for policy 0, policy_version 27512 (0.0007) [2023-10-07 20:48:47,321][67871] Updated weights for policy 1, policy_version 27560 (0.0009) [2023-10-07 20:48:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56393728. Throughput: 0: 1660.0, 1: 1673.2. Samples: 14108234. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-07 20:48:47,477][66916] Avg episode reward: [(0, '41.640'), (1, '39.000')] [2023-10-07 20:48:47,687][67871] Updated weights for policy 1, policy_version 27570 (0.0008) [2023-10-07 20:48:48,064][67871] Updated weights for policy 1, policy_version 27580 (0.0008) [2023-10-07 20:48:48,208][67676] Saving new best policy, reward=39.000! [2023-10-07 20:48:49,269][67838] Updated weights for policy 0, policy_version 27522 (0.0008) [2023-10-07 20:48:49,647][67838] Updated weights for policy 0, policy_version 27532 (0.0008) [2023-10-07 20:48:50,013][67838] Updated weights for policy 0, policy_version 27542 (0.0009) [2023-10-07 20:48:50,380][67838] Updated weights for policy 0, policy_version 27552 (0.0009) [2023-10-07 20:48:52,064][67871] Updated weights for policy 1, policy_version 27590 (0.0008) [2023-10-07 20:48:52,425][67871] Updated weights for policy 1, policy_version 27600 (0.0010) [2023-10-07 20:48:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56459264. Throughput: 0: 1660.9, 1: 1672.5. Samples: 14128776. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-07 20:48:52,477][66916] Avg episode reward: [(0, '40.810'), (1, '36.600')] [2023-10-07 20:48:52,796][67871] Updated weights for policy 1, policy_version 27610 (0.0009) [2023-10-07 20:48:54,574][67838] Updated weights for policy 0, policy_version 27562 (0.0007) [2023-10-07 20:48:54,943][67838] Updated weights for policy 0, policy_version 27572 (0.0009) [2023-10-07 20:48:55,319][67838] Updated weights for policy 0, policy_version 27582 (0.0010) [2023-10-07 20:48:56,987][67871] Updated weights for policy 1, policy_version 27620 (0.0008) [2023-10-07 20:48:57,359][67871] Updated weights for policy 1, policy_version 27630 (0.0008) [2023-10-07 20:48:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 56524800. Throughput: 0: 1649.1, 1: 1666.7. Samples: 14138104. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-07 20:48:57,478][66916] Avg episode reward: [(0, '42.530'), (1, '40.340')] [2023-10-07 20:48:57,728][67871] Updated weights for policy 1, policy_version 27640 (0.0008) [2023-10-07 20:48:58,022][67676] Saving new best policy, reward=40.340! [2023-10-07 20:48:59,461][67838] Updated weights for policy 0, policy_version 27592 (0.0009) [2023-10-07 20:48:59,833][67838] Updated weights for policy 0, policy_version 27602 (0.0008) [2023-10-07 20:49:00,208][67838] Updated weights for policy 0, policy_version 27612 (0.0009) [2023-10-07 20:49:01,766][67871] Updated weights for policy 1, policy_version 27650 (0.0010) [2023-10-07 20:49:02,130][67871] Updated weights for policy 1, policy_version 27660 (0.0008) [2023-10-07 20:49:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56590336. Throughput: 0: 1655.9, 1: 1665.9. Samples: 14158060. Policy #0 lag: (min: 10.0, avg: 14.7, max: 42.0) [2023-10-07 20:49:02,477][66916] Avg episode reward: [(0, '41.340'), (1, '37.130')] [2023-10-07 20:49:02,501][67871] Updated weights for policy 1, policy_version 27670 (0.0010) [2023-10-07 20:49:02,870][67871] Updated weights for policy 1, policy_version 27680 (0.0008) [2023-10-07 20:49:04,488][67838] Updated weights for policy 0, policy_version 27622 (0.0008) [2023-10-07 20:49:04,870][67838] Updated weights for policy 0, policy_version 27632 (0.0007) [2023-10-07 20:49:05,253][67838] Updated weights for policy 0, policy_version 27642 (0.0010) [2023-10-07 20:49:07,092][67871] Updated weights for policy 1, policy_version 27690 (0.0008) [2023-10-07 20:49:07,448][67871] Updated weights for policy 1, policy_version 27700 (0.0009) [2023-10-07 20:49:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56655872. Throughput: 0: 1661.6, 1: 1663.3. Samples: 14178390. Policy #0 lag: (min: 10.0, avg: 14.7, max: 42.0) [2023-10-07 20:49:07,477][66916] Avg episode reward: [(0, '43.570'), (1, '36.740')] [2023-10-07 20:49:07,487][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000027648_28311552.pth... [2023-10-07 20:49:07,523][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000026080_26705920.pth [2023-10-07 20:49:07,810][67871] Updated weights for policy 1, policy_version 27710 (0.0008) [2023-10-07 20:49:07,882][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000027712_28377088.pth... [2023-10-07 20:49:07,911][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000026144_26771456.pth [2023-10-07 20:49:09,178][67838] Updated weights for policy 0, policy_version 27652 (0.0009) [2023-10-07 20:49:09,545][67838] Updated weights for policy 0, policy_version 27662 (0.0007) [2023-10-07 20:49:09,915][67838] Updated weights for policy 0, policy_version 27672 (0.0007) [2023-10-07 20:49:12,146][67871] Updated weights for policy 1, policy_version 27720 (0.0008) [2023-10-07 20:49:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56721408. Throughput: 0: 1650.0, 1: 1667.7. Samples: 14187800. Policy #0 lag: (min: 10.0, avg: 14.7, max: 42.0) [2023-10-07 20:49:12,477][66916] Avg episode reward: [(0, '40.730'), (1, '39.020')] [2023-10-07 20:49:12,530][67871] Updated weights for policy 1, policy_version 27730 (0.0008) [2023-10-07 20:49:12,898][67871] Updated weights for policy 1, policy_version 27740 (0.0007) [2023-10-07 20:49:13,918][67838] Updated weights for policy 0, policy_version 27682 (0.0008) [2023-10-07 20:49:14,290][67838] Updated weights for policy 0, policy_version 27692 (0.0007) [2023-10-07 20:49:14,661][67838] Updated weights for policy 0, policy_version 27702 (0.0007) [2023-10-07 20:49:15,034][67838] Updated weights for policy 0, policy_version 27712 (0.0008) [2023-10-07 20:49:16,993][67871] Updated weights for policy 1, policy_version 27750 (0.0008) [2023-10-07 20:49:17,373][67871] Updated weights for policy 1, policy_version 27760 (0.0009) [2023-10-07 20:49:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56786944. Throughput: 0: 1672.4, 1: 1657.7. Samples: 14208034. Policy #0 lag: (min: 10.0, avg: 14.7, max: 42.0) [2023-10-07 20:49:17,478][66916] Avg episode reward: [(0, '41.530'), (1, '37.050')] [2023-10-07 20:49:17,736][67871] Updated weights for policy 1, policy_version 27770 (0.0008) [2023-10-07 20:49:19,094][67838] Updated weights for policy 0, policy_version 27722 (0.0007) [2023-10-07 20:49:19,467][67838] Updated weights for policy 0, policy_version 27732 (0.0007) [2023-10-07 20:49:19,848][67838] Updated weights for policy 0, policy_version 27742 (0.0007) [2023-10-07 20:49:21,844][67871] Updated weights for policy 1, policy_version 27780 (0.0009) [2023-10-07 20:49:22,218][67871] Updated weights for policy 1, policy_version 27790 (0.0007) [2023-10-07 20:49:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56852480. Throughput: 0: 1673.0, 1: 1654.6. Samples: 14228254. Policy #0 lag: (min: 10.0, avg: 18.0, max: 42.0) [2023-10-07 20:49:22,478][66916] Avg episode reward: [(0, '38.080'), (1, '40.210')] [2023-10-07 20:49:22,578][67871] Updated weights for policy 1, policy_version 27800 (0.0009) [2023-10-07 20:49:24,017][67838] Updated weights for policy 0, policy_version 27752 (0.0008) [2023-10-07 20:49:24,392][67838] Updated weights for policy 0, policy_version 27762 (0.0008) [2023-10-07 20:49:24,755][67838] Updated weights for policy 0, policy_version 27772 (0.0007) [2023-10-07 20:49:26,722][67871] Updated weights for policy 1, policy_version 27810 (0.0009) [2023-10-07 20:49:27,085][67871] Updated weights for policy 1, policy_version 27820 (0.0007) [2023-10-07 20:49:27,462][67871] Updated weights for policy 1, policy_version 27830 (0.0009) [2023-10-07 20:49:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56918016. Throughput: 0: 1653.1, 1: 1659.9. Samples: 14237370. Policy #0 lag: (min: 10.0, avg: 18.0, max: 42.0) [2023-10-07 20:49:27,477][66916] Avg episode reward: [(0, '40.000'), (1, '37.880')] [2023-10-07 20:49:27,831][67871] Updated weights for policy 1, policy_version 27840 (0.0010) [2023-10-07 20:49:29,018][67838] Updated weights for policy 0, policy_version 27782 (0.0009) [2023-10-07 20:49:29,397][67838] Updated weights for policy 0, policy_version 27792 (0.0009) [2023-10-07 20:49:29,779][67838] Updated weights for policy 0, policy_version 27802 (0.0007) [2023-10-07 20:49:31,876][67871] Updated weights for policy 1, policy_version 27850 (0.0010) [2023-10-07 20:49:32,246][67871] Updated weights for policy 1, policy_version 27860 (0.0009) [2023-10-07 20:49:32,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56983552. Throughput: 0: 1659.6, 1: 1661.8. Samples: 14257698. Policy #0 lag: (min: 10.0, avg: 18.0, max: 42.0) [2023-10-07 20:49:32,477][66916] Avg episode reward: [(0, '40.600'), (1, '38.390')] [2023-10-07 20:49:32,613][67871] Updated weights for policy 1, policy_version 27870 (0.0010) [2023-10-07 20:49:33,825][67838] Updated weights for policy 0, policy_version 27812 (0.0009) [2023-10-07 20:49:34,196][67838] Updated weights for policy 0, policy_version 27822 (0.0008) [2023-10-07 20:49:34,566][67838] Updated weights for policy 0, policy_version 27832 (0.0007) [2023-10-07 20:49:36,910][67871] Updated weights for policy 1, policy_version 27880 (0.0009) [2023-10-07 20:49:37,278][67871] Updated weights for policy 1, policy_version 27890 (0.0007) [2023-10-07 20:49:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 57049088. Throughput: 0: 1661.2, 1: 1652.9. Samples: 14277912. Policy #0 lag: (min: 10.0, avg: 18.0, max: 42.0) [2023-10-07 20:49:37,477][66916] Avg episode reward: [(0, '41.240'), (1, '36.600')] [2023-10-07 20:49:37,647][67871] Updated weights for policy 1, policy_version 27900 (0.0007) [2023-10-07 20:49:38,629][67838] Updated weights for policy 0, policy_version 27842 (0.0010) [2023-10-07 20:49:39,005][67838] Updated weights for policy 0, policy_version 27852 (0.0009) [2023-10-07 20:49:39,382][67838] Updated weights for policy 0, policy_version 27862 (0.0007) [2023-10-07 20:49:39,759][67838] Updated weights for policy 0, policy_version 27872 (0.0007) [2023-10-07 20:49:41,611][67871] Updated weights for policy 1, policy_version 27910 (0.0009) [2023-10-07 20:49:41,981][67871] Updated weights for policy 1, policy_version 27920 (0.0009) [2023-10-07 20:49:42,349][67871] Updated weights for policy 1, policy_version 27930 (0.0008) [2023-10-07 20:49:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 57114624. Throughput: 0: 1653.2, 1: 1659.1. Samples: 14287154. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 20:49:42,478][66916] Avg episode reward: [(0, '39.580'), (1, '39.440')] [2023-10-07 20:49:43,796][67838] Updated weights for policy 0, policy_version 27882 (0.0009) [2023-10-07 20:49:44,162][67838] Updated weights for policy 0, policy_version 27892 (0.0009) [2023-10-07 20:49:44,542][67838] Updated weights for policy 0, policy_version 27902 (0.0009) [2023-10-07 20:49:46,231][67871] Updated weights for policy 1, policy_version 27940 (0.0009) [2023-10-07 20:49:46,603][67871] Updated weights for policy 1, policy_version 27950 (0.0008) [2023-10-07 20:49:46,969][67871] Updated weights for policy 1, policy_version 27960 (0.0009) [2023-10-07 20:49:47,476][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 57212928. Throughput: 0: 1666.9, 1: 1659.4. Samples: 14307742. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 20:49:47,477][66916] Avg episode reward: [(0, '39.090'), (1, '38.640')] [2023-10-07 20:49:48,652][67838] Updated weights for policy 0, policy_version 27912 (0.0011) [2023-10-07 20:49:49,019][67838] Updated weights for policy 0, policy_version 27922 (0.0009) [2023-10-07 20:49:49,395][67838] Updated weights for policy 0, policy_version 27932 (0.0010) [2023-10-07 20:49:51,327][67871] Updated weights for policy 1, policy_version 27970 (0.0009) [2023-10-07 20:49:51,700][67871] Updated weights for policy 1, policy_version 27980 (0.0007) [2023-10-07 20:49:52,066][67871] Updated weights for policy 1, policy_version 27990 (0.0007) [2023-10-07 20:49:52,432][67871] Updated weights for policy 1, policy_version 28000 (0.0011) [2023-10-07 20:49:52,477][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 57278464. Throughput: 0: 1663.7, 1: 1648.0. Samples: 14327416. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 20:49:52,477][66916] Avg episode reward: [(0, '40.860'), (1, '37.800')] [2023-10-07 20:49:53,666][67838] Updated weights for policy 0, policy_version 27942 (0.0009) [2023-10-07 20:49:54,042][67838] Updated weights for policy 0, policy_version 27952 (0.0007) [2023-10-07 20:49:54,410][67838] Updated weights for policy 0, policy_version 27962 (0.0009) [2023-10-07 20:49:56,548][67871] Updated weights for policy 1, policy_version 28010 (0.0008) [2023-10-07 20:49:56,924][67871] Updated weights for policy 1, policy_version 28020 (0.0008) [2023-10-07 20:49:57,281][67871] Updated weights for policy 1, policy_version 28030 (0.0008) [2023-10-07 20:49:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 57344000. Throughput: 0: 1652.6, 1: 1660.3. Samples: 14336880. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 20:49:57,477][66916] Avg episode reward: [(0, '40.320'), (1, '38.810')] [2023-10-07 20:49:58,592][67838] Updated weights for policy 0, policy_version 27972 (0.0007) [2023-10-07 20:49:58,961][67838] Updated weights for policy 0, policy_version 27982 (0.0007) [2023-10-07 20:49:59,332][67838] Updated weights for policy 0, policy_version 27992 (0.0008) [2023-10-07 20:50:01,692][67871] Updated weights for policy 1, policy_version 28040 (0.0010) [2023-10-07 20:50:02,063][67871] Updated weights for policy 1, policy_version 28050 (0.0007) [2023-10-07 20:50:02,434][67871] Updated weights for policy 1, policy_version 28060 (0.0008) [2023-10-07 20:50:02,476][66916] Fps is (10 sec: 9830.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 57376768. Throughput: 0: 1653.2, 1: 1662.2. Samples: 14357228. Policy #0 lag: (min: 2.0, avg: 5.6, max: 34.0) [2023-10-07 20:50:02,477][66916] Avg episode reward: [(0, '39.060'), (1, '37.210')] [2023-10-07 20:50:03,511][67838] Updated weights for policy 0, policy_version 28002 (0.0007) [2023-10-07 20:50:03,883][67838] Updated weights for policy 0, policy_version 28012 (0.0009) [2023-10-07 20:50:04,254][67838] Updated weights for policy 0, policy_version 28022 (0.0008) [2023-10-07 20:50:04,633][67838] Updated weights for policy 0, policy_version 28032 (0.0007) [2023-10-07 20:50:06,564][67871] Updated weights for policy 1, policy_version 28070 (0.0009) [2023-10-07 20:50:06,935][67871] Updated weights for policy 1, policy_version 28080 (0.0007) [2023-10-07 20:50:07,302][67871] Updated weights for policy 1, policy_version 28090 (0.0008) [2023-10-07 20:50:07,477][66916] Fps is (10 sec: 9830.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 57442304. Throughput: 0: 1661.7, 1: 1650.0. Samples: 14377282. Policy #0 lag: (min: 2.0, avg: 5.6, max: 34.0) [2023-10-07 20:50:07,478][66916] Avg episode reward: [(0, '42.340'), (1, '39.450')] [2023-10-07 20:50:08,559][67838] Updated weights for policy 0, policy_version 28042 (0.0008) [2023-10-07 20:50:08,938][67838] Updated weights for policy 0, policy_version 28052 (0.0009) [2023-10-07 20:50:09,310][67838] Updated weights for policy 0, policy_version 28062 (0.0008) [2023-10-07 20:50:11,349][67871] Updated weights for policy 1, policy_version 28100 (0.0012) [2023-10-07 20:50:11,716][67871] Updated weights for policy 1, policy_version 28110 (0.0010) [2023-10-07 20:50:12,087][67871] Updated weights for policy 1, policy_version 28120 (0.0007) [2023-10-07 20:50:12,476][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 57540608. Throughput: 0: 1664.5, 1: 1657.7. Samples: 14386872. Policy #0 lag: (min: 2.0, avg: 5.6, max: 34.0) [2023-10-07 20:50:12,477][66916] Avg episode reward: [(0, '37.680'), (1, '37.910')] [2023-10-07 20:50:13,391][67838] Updated weights for policy 0, policy_version 28072 (0.0007) [2023-10-07 20:50:13,763][67838] Updated weights for policy 0, policy_version 28082 (0.0008) [2023-10-07 20:50:14,133][67838] Updated weights for policy 0, policy_version 28092 (0.0007) [2023-10-07 20:50:16,081][67871] Updated weights for policy 1, policy_version 28130 (0.0010) [2023-10-07 20:50:16,443][67871] Updated weights for policy 1, policy_version 28140 (0.0007) [2023-10-07 20:50:16,813][67871] Updated weights for policy 1, policy_version 28150 (0.0008) [2023-10-07 20:50:17,178][67871] Updated weights for policy 1, policy_version 28160 (0.0007) [2023-10-07 20:50:17,476][66916] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 57606144. Throughput: 0: 1667.9, 1: 1659.8. Samples: 14407446. Policy #0 lag: (min: 2.0, avg: 5.6, max: 34.0) [2023-10-07 20:50:17,477][66916] Avg episode reward: [(0, '39.110'), (1, '40.720')] [2023-10-07 20:50:17,478][67676] Saving new best policy, reward=40.720! [2023-10-07 20:50:18,205][67838] Updated weights for policy 0, policy_version 28102 (0.0008) [2023-10-07 20:50:18,578][67838] Updated weights for policy 0, policy_version 28112 (0.0009) [2023-10-07 20:50:18,955][67838] Updated weights for policy 0, policy_version 28122 (0.0010) [2023-10-07 20:50:21,196][67871] Updated weights for policy 1, policy_version 28170 (0.0008) [2023-10-07 20:50:21,561][67871] Updated weights for policy 1, policy_version 28180 (0.0007) [2023-10-07 20:50:21,930][67871] Updated weights for policy 1, policy_version 28190 (0.0010) [2023-10-07 20:50:22,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13329.3). Total num frames: 57671680. Throughput: 0: 1670.3, 1: 1649.2. Samples: 14427292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:50:22,477][66916] Avg episode reward: [(0, '38.400'), (1, '39.760')] [2023-10-07 20:50:23,053][67838] Updated weights for policy 0, policy_version 28132 (0.0011) [2023-10-07 20:50:23,422][67838] Updated weights for policy 0, policy_version 28142 (0.0009) [2023-10-07 20:50:23,786][67838] Updated weights for policy 0, policy_version 28152 (0.0012) [2023-10-07 20:50:26,104][67871] Updated weights for policy 1, policy_version 28200 (0.0007) [2023-10-07 20:50:26,470][67871] Updated weights for policy 1, policy_version 28210 (0.0007) [2023-10-07 20:50:26,830][67871] Updated weights for policy 1, policy_version 28220 (0.0007) [2023-10-07 20:50:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 57737216. Throughput: 0: 1667.3, 1: 1667.6. Samples: 14437224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:50:27,477][66916] Avg episode reward: [(0, '38.020'), (1, '38.690')] [2023-10-07 20:50:28,102][67838] Updated weights for policy 0, policy_version 28162 (0.0010) [2023-10-07 20:50:28,471][67838] Updated weights for policy 0, policy_version 28172 (0.0010) [2023-10-07 20:50:28,848][67838] Updated weights for policy 0, policy_version 28182 (0.0008) [2023-10-07 20:50:29,230][67838] Updated weights for policy 0, policy_version 28192 (0.0008) [2023-10-07 20:50:30,994][67871] Updated weights for policy 1, policy_version 28230 (0.0009) [2023-10-07 20:50:31,369][67871] Updated weights for policy 1, policy_version 28240 (0.0010) [2023-10-07 20:50:31,732][67871] Updated weights for policy 1, policy_version 28250 (0.0008) [2023-10-07 20:50:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 57802752. Throughput: 0: 1662.0, 1: 1664.0. Samples: 14457414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:50:32,477][66916] Avg episode reward: [(0, '37.550'), (1, '38.720')] [2023-10-07 20:50:33,287][67838] Updated weights for policy 0, policy_version 28202 (0.0009) [2023-10-07 20:50:33,665][67838] Updated weights for policy 0, policy_version 28212 (0.0007) [2023-10-07 20:50:34,035][67838] Updated weights for policy 0, policy_version 28222 (0.0008) [2023-10-07 20:50:35,866][67871] Updated weights for policy 1, policy_version 28260 (0.0009) [2023-10-07 20:50:36,239][67871] Updated weights for policy 1, policy_version 28270 (0.0008) [2023-10-07 20:50:36,608][67871] Updated weights for policy 1, policy_version 28280 (0.0008) [2023-10-07 20:50:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 57868288. Throughput: 0: 1666.6, 1: 1656.5. Samples: 14476954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:50:37,477][66916] Avg episode reward: [(0, '36.270'), (1, '37.260')] [2023-10-07 20:50:38,215][67838] Updated weights for policy 0, policy_version 28232 (0.0008) [2023-10-07 20:50:38,584][67838] Updated weights for policy 0, policy_version 28242 (0.0007) [2023-10-07 20:50:38,963][67838] Updated weights for policy 0, policy_version 28252 (0.0008) [2023-10-07 20:50:40,764][67871] Updated weights for policy 1, policy_version 28290 (0.0009) [2023-10-07 20:50:41,136][67871] Updated weights for policy 1, policy_version 28300 (0.0012) [2023-10-07 20:50:41,501][67871] Updated weights for policy 1, policy_version 28310 (0.0010) [2023-10-07 20:50:41,869][67871] Updated weights for policy 1, policy_version 28320 (0.0008) [2023-10-07 20:50:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 57933824. Throughput: 0: 1666.5, 1: 1668.2. Samples: 14486942. Policy #0 lag: (min: 26.0, avg: 32.5, max: 58.0) [2023-10-07 20:50:42,478][66916] Avg episode reward: [(0, '37.060'), (1, '38.220')] [2023-10-07 20:50:43,149][67838] Updated weights for policy 0, policy_version 28262 (0.0007) [2023-10-07 20:50:43,521][67838] Updated weights for policy 0, policy_version 28272 (0.0008) [2023-10-07 20:50:43,886][67838] Updated weights for policy 0, policy_version 28282 (0.0010) [2023-10-07 20:50:45,814][67871] Updated weights for policy 1, policy_version 28330 (0.0011) [2023-10-07 20:50:46,193][67871] Updated weights for policy 1, policy_version 28340 (0.0009) [2023-10-07 20:50:46,563][67871] Updated weights for policy 1, policy_version 28350 (0.0008) [2023-10-07 20:50:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 57999360. Throughput: 0: 1668.7, 1: 1662.3. Samples: 14507124. Policy #0 lag: (min: 26.0, avg: 32.5, max: 58.0) [2023-10-07 20:50:47,477][66916] Avg episode reward: [(0, '37.440'), (1, '37.690')] [2023-10-07 20:50:47,888][67838] Updated weights for policy 0, policy_version 28292 (0.0010) [2023-10-07 20:50:48,274][67838] Updated weights for policy 0, policy_version 28302 (0.0010) [2023-10-07 20:50:48,639][67838] Updated weights for policy 0, policy_version 28312 (0.0008) [2023-10-07 20:50:50,760][67871] Updated weights for policy 1, policy_version 28360 (0.0008) [2023-10-07 20:50:51,132][67871] Updated weights for policy 1, policy_version 28370 (0.0009) [2023-10-07 20:50:51,503][67871] Updated weights for policy 1, policy_version 28380 (0.0008) [2023-10-07 20:50:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 58064896. Throughput: 0: 1659.0, 1: 1657.4. Samples: 14526522. Policy #0 lag: (min: 26.0, avg: 32.5, max: 58.0) [2023-10-07 20:50:52,477][66916] Avg episode reward: [(0, '40.800'), (1, '38.180')] [2023-10-07 20:50:52,953][67838] Updated weights for policy 0, policy_version 28322 (0.0009) [2023-10-07 20:50:53,322][67838] Updated weights for policy 0, policy_version 28332 (0.0007) [2023-10-07 20:50:53,693][67838] Updated weights for policy 0, policy_version 28342 (0.0008) [2023-10-07 20:50:54,060][67838] Updated weights for policy 0, policy_version 28352 (0.0011) [2023-10-07 20:50:55,528][67871] Updated weights for policy 1, policy_version 28390 (0.0009) [2023-10-07 20:50:55,892][67871] Updated weights for policy 1, policy_version 28400 (0.0009) [2023-10-07 20:50:56,261][67871] Updated weights for policy 1, policy_version 28410 (0.0007) [2023-10-07 20:50:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 58130432. Throughput: 0: 1653.9, 1: 1677.0. Samples: 14536762. Policy #0 lag: (min: 26.0, avg: 32.5, max: 58.0) [2023-10-07 20:50:57,477][66916] Avg episode reward: [(0, '38.460'), (1, '36.080')] [2023-10-07 20:50:58,198][67838] Updated weights for policy 0, policy_version 28362 (0.0010) [2023-10-07 20:50:58,574][67838] Updated weights for policy 0, policy_version 28372 (0.0010) [2023-10-07 20:50:58,949][67838] Updated weights for policy 0, policy_version 28382 (0.0008) [2023-10-07 20:51:00,404][67871] Updated weights for policy 1, policy_version 28420 (0.0008) [2023-10-07 20:51:00,778][67871] Updated weights for policy 1, policy_version 28430 (0.0008) [2023-10-07 20:51:01,135][67871] Updated weights for policy 1, policy_version 28440 (0.0008) [2023-10-07 20:51:02,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 58195968. Throughput: 0: 1653.5, 1: 1661.5. Samples: 14556622. Policy #0 lag: (min: 15.0, avg: 23.0, max: 47.0) [2023-10-07 20:51:02,478][66916] Avg episode reward: [(0, '36.690'), (1, '37.800')] [2023-10-07 20:51:03,138][67838] Updated weights for policy 0, policy_version 28392 (0.0011) [2023-10-07 20:51:03,510][67838] Updated weights for policy 0, policy_version 28402 (0.0008) [2023-10-07 20:51:03,883][67838] Updated weights for policy 0, policy_version 28412 (0.0008) [2023-10-07 20:51:05,202][67871] Updated weights for policy 1, policy_version 28450 (0.0008) [2023-10-07 20:51:05,575][67871] Updated weights for policy 1, policy_version 28460 (0.0009) [2023-10-07 20:51:05,948][67871] Updated weights for policy 1, policy_version 28470 (0.0010) [2023-10-07 20:51:06,307][67871] Updated weights for policy 1, policy_version 28480 (0.0009) [2023-10-07 20:51:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 58261504. Throughput: 0: 1648.1, 1: 1667.0. Samples: 14576470. Policy #0 lag: (min: 15.0, avg: 23.0, max: 47.0) [2023-10-07 20:51:07,478][66916] Avg episode reward: [(0, '37.770'), (1, '35.670')] [2023-10-07 20:51:07,491][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000028480_29163520.pth... [2023-10-07 20:51:07,491][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000028416_29097984.pth... [2023-10-07 20:51:07,526][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000026880_27525120.pth [2023-10-07 20:51:07,529][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000026912_27557888.pth [2023-10-07 20:51:08,079][67838] Updated weights for policy 0, policy_version 28422 (0.0009) [2023-10-07 20:51:08,460][67838] Updated weights for policy 0, policy_version 28432 (0.0010) [2023-10-07 20:51:08,834][67838] Updated weights for policy 0, policy_version 28442 (0.0011) [2023-10-07 20:51:10,453][67871] Updated weights for policy 1, policy_version 28490 (0.0009) [2023-10-07 20:51:10,830][67871] Updated weights for policy 1, policy_version 28500 (0.0010) [2023-10-07 20:51:11,186][67871] Updated weights for policy 1, policy_version 28510 (0.0008) [2023-10-07 20:51:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 58327040. Throughput: 0: 1648.0, 1: 1671.0. Samples: 14586578. Policy #0 lag: (min: 15.0, avg: 23.0, max: 47.0) [2023-10-07 20:51:12,477][66916] Avg episode reward: [(0, '37.360'), (1, '36.470')] [2023-10-07 20:51:12,853][67838] Updated weights for policy 0, policy_version 28452 (0.0008) [2023-10-07 20:51:13,230][67838] Updated weights for policy 0, policy_version 28462 (0.0007) [2023-10-07 20:51:13,599][67838] Updated weights for policy 0, policy_version 28472 (0.0007) [2023-10-07 20:51:15,225][67871] Updated weights for policy 1, policy_version 28520 (0.0007) [2023-10-07 20:51:15,592][67871] Updated weights for policy 1, policy_version 28530 (0.0010) [2023-10-07 20:51:15,971][67871] Updated weights for policy 1, policy_version 28540 (0.0009) [2023-10-07 20:51:17,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 58392576. Throughput: 0: 1658.2, 1: 1658.1. Samples: 14606648. Policy #0 lag: (min: 15.0, avg: 23.0, max: 47.0) [2023-10-07 20:51:17,478][66916] Avg episode reward: [(0, '34.860'), (1, '37.750')] [2023-10-07 20:51:17,633][67838] Updated weights for policy 0, policy_version 28482 (0.0009) [2023-10-07 20:51:18,005][67838] Updated weights for policy 0, policy_version 28492 (0.0007) [2023-10-07 20:51:18,374][67838] Updated weights for policy 0, policy_version 28502 (0.0008) [2023-10-07 20:51:18,742][67838] Updated weights for policy 0, policy_version 28512 (0.0009) [2023-10-07 20:51:20,104][67871] Updated weights for policy 1, policy_version 28550 (0.0009) [2023-10-07 20:51:20,470][67871] Updated weights for policy 1, policy_version 28560 (0.0008) [2023-10-07 20:51:20,841][67871] Updated weights for policy 1, policy_version 28570 (0.0007) [2023-10-07 20:51:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 58458112. Throughput: 0: 1658.7, 1: 1666.5. Samples: 14626590. Policy #0 lag: (min: 0.0, avg: 18.4, max: 32.0) [2023-10-07 20:51:22,478][66916] Avg episode reward: [(0, '36.050'), (1, '42.750')] [2023-10-07 20:51:22,485][67676] Saving new best policy, reward=42.750! [2023-10-07 20:51:22,800][67838] Updated weights for policy 0, policy_version 28522 (0.0009) [2023-10-07 20:51:23,168][67838] Updated weights for policy 0, policy_version 28532 (0.0010) [2023-10-07 20:51:23,538][67838] Updated weights for policy 0, policy_version 28542 (0.0008) [2023-10-07 20:51:24,978][67871] Updated weights for policy 1, policy_version 28580 (0.0009) [2023-10-07 20:51:25,337][67871] Updated weights for policy 1, policy_version 28590 (0.0009) [2023-10-07 20:51:25,707][67871] Updated weights for policy 1, policy_version 28600 (0.0011) [2023-10-07 20:51:27,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 58523648. Throughput: 0: 1660.5, 1: 1668.7. Samples: 14636752. Policy #0 lag: (min: 0.0, avg: 18.4, max: 32.0) [2023-10-07 20:51:27,477][66916] Avg episode reward: [(0, '37.750'), (1, '39.020')] [2023-10-07 20:51:27,776][67838] Updated weights for policy 0, policy_version 28552 (0.0007) [2023-10-07 20:51:28,150][67838] Updated weights for policy 0, policy_version 28562 (0.0008) [2023-10-07 20:51:28,527][67838] Updated weights for policy 0, policy_version 28572 (0.0009) [2023-10-07 20:51:29,901][67871] Updated weights for policy 1, policy_version 28610 (0.0007) [2023-10-07 20:51:30,266][67871] Updated weights for policy 1, policy_version 28620 (0.0008) [2023-10-07 20:51:30,645][67871] Updated weights for policy 1, policy_version 28630 (0.0008) [2023-10-07 20:51:31,014][67871] Updated weights for policy 1, policy_version 28640 (0.0010) [2023-10-07 20:51:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 58589184. Throughput: 0: 1658.8, 1: 1653.5. Samples: 14656180. Policy #0 lag: (min: 0.0, avg: 18.4, max: 32.0) [2023-10-07 20:51:32,477][66916] Avg episode reward: [(0, '35.830'), (1, '43.330')] [2023-10-07 20:51:32,478][67676] Saving new best policy, reward=43.330! [2023-10-07 20:51:32,516][67838] Updated weights for policy 0, policy_version 28582 (0.0010) [2023-10-07 20:51:32,896][67838] Updated weights for policy 0, policy_version 28592 (0.0009) [2023-10-07 20:51:33,272][67838] Updated weights for policy 0, policy_version 28602 (0.0009) [2023-10-07 20:51:35,131][67871] Updated weights for policy 1, policy_version 28650 (0.0008) [2023-10-07 20:51:35,503][67871] Updated weights for policy 1, policy_version 28660 (0.0008) [2023-10-07 20:51:35,878][67871] Updated weights for policy 1, policy_version 28670 (0.0008) [2023-10-07 20:51:37,359][67838] Updated weights for policy 0, policy_version 28612 (0.0007) [2023-10-07 20:51:37,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 58654720. Throughput: 0: 1664.0, 1: 1667.8. Samples: 14676456. Policy #0 lag: (min: 0.0, avg: 18.4, max: 32.0) [2023-10-07 20:51:37,478][66916] Avg episode reward: [(0, '37.270'), (1, '41.940')] [2023-10-07 20:51:37,735][67838] Updated weights for policy 0, policy_version 28622 (0.0007) [2023-10-07 20:51:38,114][67838] Updated weights for policy 0, policy_version 28632 (0.0008) [2023-10-07 20:51:40,066][67871] Updated weights for policy 1, policy_version 28680 (0.0010) [2023-10-07 20:51:40,435][67871] Updated weights for policy 1, policy_version 28690 (0.0008) [2023-10-07 20:51:40,809][67871] Updated weights for policy 1, policy_version 28700 (0.0009) [2023-10-07 20:51:42,164][67838] Updated weights for policy 0, policy_version 28642 (0.0007) [2023-10-07 20:51:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58720256. Throughput: 0: 1667.1, 1: 1661.0. Samples: 14686526. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 20:51:42,477][66916] Avg episode reward: [(0, '37.990'), (1, '41.480')] [2023-10-07 20:51:42,535][67838] Updated weights for policy 0, policy_version 28652 (0.0008) [2023-10-07 20:51:42,909][67838] Updated weights for policy 0, policy_version 28662 (0.0009) [2023-10-07 20:51:43,277][67838] Updated weights for policy 0, policy_version 28672 (0.0008) [2023-10-07 20:51:44,938][67871] Updated weights for policy 1, policy_version 28710 (0.0007) [2023-10-07 20:51:45,308][67871] Updated weights for policy 1, policy_version 28720 (0.0009) [2023-10-07 20:51:45,663][67871] Updated weights for policy 1, policy_version 28730 (0.0009) [2023-10-07 20:51:47,429][67838] Updated weights for policy 0, policy_version 28682 (0.0007) [2023-10-07 20:51:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58785792. Throughput: 0: 1669.3, 1: 1649.3. Samples: 14705958. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 20:51:47,477][66916] Avg episode reward: [(0, '38.080'), (1, '40.510')] [2023-10-07 20:51:47,812][67838] Updated weights for policy 0, policy_version 28692 (0.0007) [2023-10-07 20:51:48,177][67838] Updated weights for policy 0, policy_version 28702 (0.0009) [2023-10-07 20:51:49,758][67871] Updated weights for policy 1, policy_version 28740 (0.0009) [2023-10-07 20:51:50,117][67871] Updated weights for policy 1, policy_version 28750 (0.0007) [2023-10-07 20:51:50,486][67871] Updated weights for policy 1, policy_version 28760 (0.0008) [2023-10-07 20:51:52,396][67838] Updated weights for policy 0, policy_version 28712 (0.0010) [2023-10-07 20:51:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 58851328. Throughput: 0: 1663.6, 1: 1661.2. Samples: 14726084. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 20:51:52,478][66916] Avg episode reward: [(0, '41.340'), (1, '39.560')] [2023-10-07 20:51:52,766][67838] Updated weights for policy 0, policy_version 28722 (0.0010) [2023-10-07 20:51:53,140][67838] Updated weights for policy 0, policy_version 28732 (0.0008) [2023-10-07 20:51:54,604][67871] Updated weights for policy 1, policy_version 28770 (0.0009) [2023-10-07 20:51:54,979][67871] Updated weights for policy 1, policy_version 28780 (0.0009) [2023-10-07 20:51:55,347][67871] Updated weights for policy 1, policy_version 28790 (0.0010) [2023-10-07 20:51:55,703][67871] Updated weights for policy 1, policy_version 28800 (0.0008) [2023-10-07 20:51:57,401][67838] Updated weights for policy 0, policy_version 28742 (0.0008) [2023-10-07 20:51:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58916864. Throughput: 0: 1662.5, 1: 1654.3. Samples: 14735832. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 20:51:57,477][66916] Avg episode reward: [(0, '41.390'), (1, '36.290')] [2023-10-07 20:51:57,765][67838] Updated weights for policy 0, policy_version 28752 (0.0010) [2023-10-07 20:51:58,139][67838] Updated weights for policy 0, policy_version 28762 (0.0008) [2023-10-07 20:51:59,684][67871] Updated weights for policy 1, policy_version 28810 (0.0007) [2023-10-07 20:52:00,053][67871] Updated weights for policy 1, policy_version 28820 (0.0009) [2023-10-07 20:52:00,425][67871] Updated weights for policy 1, policy_version 28830 (0.0009) [2023-10-07 20:52:02,343][67838] Updated weights for policy 0, policy_version 28772 (0.0007) [2023-10-07 20:52:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58982400. Throughput: 0: 1655.7, 1: 1648.8. Samples: 14755348. Policy #0 lag: (min: 3.0, avg: 6.0, max: 35.0) [2023-10-07 20:52:02,477][66916] Avg episode reward: [(0, '41.970'), (1, '36.790')] [2023-10-07 20:52:02,722][67838] Updated weights for policy 0, policy_version 28782 (0.0008) [2023-10-07 20:52:03,090][67838] Updated weights for policy 0, policy_version 28792 (0.0010) [2023-10-07 20:52:04,412][67871] Updated weights for policy 1, policy_version 28840 (0.0010) [2023-10-07 20:52:04,776][67871] Updated weights for policy 1, policy_version 28850 (0.0007) [2023-10-07 20:52:05,149][67871] Updated weights for policy 1, policy_version 28860 (0.0008) [2023-10-07 20:52:07,143][67838] Updated weights for policy 0, policy_version 28802 (0.0010) [2023-10-07 20:52:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 59047936. Throughput: 0: 1652.5, 1: 1664.5. Samples: 14775856. Policy #0 lag: (min: 3.0, avg: 6.0, max: 35.0) [2023-10-07 20:52:07,478][66916] Avg episode reward: [(0, '40.740'), (1, '35.250')] [2023-10-07 20:52:07,522][67838] Updated weights for policy 0, policy_version 28812 (0.0007) [2023-10-07 20:52:07,900][67838] Updated weights for policy 0, policy_version 28822 (0.0008) [2023-10-07 20:52:08,264][67838] Updated weights for policy 0, policy_version 28832 (0.0009) [2023-10-07 20:52:09,240][67871] Updated weights for policy 1, policy_version 28870 (0.0008) [2023-10-07 20:52:09,610][67871] Updated weights for policy 1, policy_version 28880 (0.0009) [2023-10-07 20:52:09,974][67871] Updated weights for policy 1, policy_version 28890 (0.0007) [2023-10-07 20:52:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 59113472. Throughput: 0: 1653.2, 1: 1648.7. Samples: 14785336. Policy #0 lag: (min: 3.0, avg: 6.0, max: 35.0) [2023-10-07 20:52:12,477][66916] Avg episode reward: [(0, '38.830'), (1, '34.650')] [2023-10-07 20:52:12,582][67838] Updated weights for policy 0, policy_version 28842 (0.0007) [2023-10-07 20:52:12,956][67838] Updated weights for policy 0, policy_version 28852 (0.0008) [2023-10-07 20:52:13,327][67838] Updated weights for policy 0, policy_version 28862 (0.0009) [2023-10-07 20:52:14,060][67871] Updated weights for policy 1, policy_version 28900 (0.0009) [2023-10-07 20:52:14,433][67871] Updated weights for policy 1, policy_version 28910 (0.0011) [2023-10-07 20:52:14,800][67871] Updated weights for policy 1, policy_version 28920 (0.0009) [2023-10-07 20:52:17,395][67838] Updated weights for policy 0, policy_version 28872 (0.0009) [2023-10-07 20:52:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 59179008. Throughput: 0: 1648.9, 1: 1665.9. Samples: 14805344. Policy #0 lag: (min: 3.0, avg: 6.0, max: 35.0) [2023-10-07 20:52:17,477][66916] Avg episode reward: [(0, '36.940'), (1, '36.310')] [2023-10-07 20:52:17,767][67838] Updated weights for policy 0, policy_version 28882 (0.0009) [2023-10-07 20:52:18,153][67838] Updated weights for policy 0, policy_version 28892 (0.0008) [2023-10-07 20:52:18,915][67871] Updated weights for policy 1, policy_version 28930 (0.0008) [2023-10-07 20:52:19,274][67871] Updated weights for policy 1, policy_version 28940 (0.0008) [2023-10-07 20:52:19,641][67871] Updated weights for policy 1, policy_version 28950 (0.0008) [2023-10-07 20:52:20,011][67871] Updated weights for policy 1, policy_version 28960 (0.0009) [2023-10-07 20:52:22,179][67838] Updated weights for policy 0, policy_version 28902 (0.0007) [2023-10-07 20:52:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 59244544. Throughput: 0: 1645.1, 1: 1674.8. Samples: 14825850. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-07 20:52:22,477][66916] Avg episode reward: [(0, '39.430'), (1, '35.750')] [2023-10-07 20:52:22,546][67838] Updated weights for policy 0, policy_version 28912 (0.0007) [2023-10-07 20:52:22,923][67838] Updated weights for policy 0, policy_version 28922 (0.0007) [2023-10-07 20:52:24,217][67871] Updated weights for policy 1, policy_version 28970 (0.0009) [2023-10-07 20:52:24,590][67871] Updated weights for policy 1, policy_version 28980 (0.0008) [2023-10-07 20:52:24,958][67871] Updated weights for policy 1, policy_version 28990 (0.0008) [2023-10-07 20:52:27,131][67838] Updated weights for policy 0, policy_version 28932 (0.0007) [2023-10-07 20:52:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 59310080. Throughput: 0: 1648.5, 1: 1652.2. Samples: 14835056. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-07 20:52:27,478][66916] Avg episode reward: [(0, '36.760'), (1, '39.270')] [2023-10-07 20:52:27,520][67838] Updated weights for policy 0, policy_version 28942 (0.0009) [2023-10-07 20:52:27,898][67838] Updated weights for policy 0, policy_version 28952 (0.0009) [2023-10-07 20:52:29,094][67871] Updated weights for policy 1, policy_version 29000 (0.0009) [2023-10-07 20:52:29,465][67871] Updated weights for policy 1, policy_version 29010 (0.0007) [2023-10-07 20:52:29,835][67871] Updated weights for policy 1, policy_version 29020 (0.0010) [2023-10-07 20:52:32,001][67838] Updated weights for policy 0, policy_version 28962 (0.0010) [2023-10-07 20:52:32,378][67838] Updated weights for policy 0, policy_version 28972 (0.0007) [2023-10-07 20:52:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 59375616. Throughput: 0: 1645.6, 1: 1670.8. Samples: 14855196. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-07 20:52:32,477][66916] Avg episode reward: [(0, '38.590'), (1, '36.030')] [2023-10-07 20:52:32,754][67838] Updated weights for policy 0, policy_version 28982 (0.0008) [2023-10-07 20:52:33,129][67838] Updated weights for policy 0, policy_version 28992 (0.0007) [2023-10-07 20:52:33,883][67871] Updated weights for policy 1, policy_version 29030 (0.0007) [2023-10-07 20:52:34,252][67871] Updated weights for policy 1, policy_version 29040 (0.0007) [2023-10-07 20:52:34,616][67871] Updated weights for policy 1, policy_version 29050 (0.0009) [2023-10-07 20:52:37,156][67838] Updated weights for policy 0, policy_version 29002 (0.0008) [2023-10-07 20:52:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 59441152. Throughput: 0: 1645.9, 1: 1672.5. Samples: 14875412. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-07 20:52:37,477][66916] Avg episode reward: [(0, '38.070'), (1, '36.880')] [2023-10-07 20:52:37,529][67838] Updated weights for policy 0, policy_version 29012 (0.0010) [2023-10-07 20:52:37,898][67838] Updated weights for policy 0, policy_version 29022 (0.0009) [2023-10-07 20:52:38,757][67871] Updated weights for policy 1, policy_version 29060 (0.0008) [2023-10-07 20:52:39,122][67871] Updated weights for policy 1, policy_version 29070 (0.0009) [2023-10-07 20:52:39,494][67871] Updated weights for policy 1, policy_version 29080 (0.0009) [2023-10-07 20:52:42,010][67838] Updated weights for policy 0, policy_version 29032 (0.0009) [2023-10-07 20:52:42,388][67838] Updated weights for policy 0, policy_version 29042 (0.0008) [2023-10-07 20:52:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 59506688. Throughput: 0: 1658.7, 1: 1654.3. Samples: 14884916. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-07 20:52:42,477][66916] Avg episode reward: [(0, '38.900'), (1, '37.750')] [2023-10-07 20:52:42,752][67838] Updated weights for policy 0, policy_version 29052 (0.0008) [2023-10-07 20:52:43,649][67871] Updated weights for policy 1, policy_version 29090 (0.0007) [2023-10-07 20:52:44,026][67871] Updated weights for policy 1, policy_version 29100 (0.0009) [2023-10-07 20:52:44,403][67871] Updated weights for policy 1, policy_version 29110 (0.0009) [2023-10-07 20:52:44,769][67871] Updated weights for policy 1, policy_version 29120 (0.0010) [2023-10-07 20:52:46,883][67838] Updated weights for policy 0, policy_version 29062 (0.0007) [2023-10-07 20:52:47,264][67838] Updated weights for policy 0, policy_version 29072 (0.0007) [2023-10-07 20:52:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 59572224. Throughput: 0: 1658.8, 1: 1674.3. Samples: 14905334. Policy #0 lag: (min: 21.0, avg: 21.3, max: 29.0) [2023-10-07 20:52:47,477][66916] Avg episode reward: [(0, '37.830'), (1, '35.930')] [2023-10-07 20:52:47,629][67838] Updated weights for policy 0, policy_version 29082 (0.0009) [2023-10-07 20:52:48,869][67871] Updated weights for policy 1, policy_version 29130 (0.0008) [2023-10-07 20:52:49,239][67871] Updated weights for policy 1, policy_version 29140 (0.0007) [2023-10-07 20:52:49,610][67871] Updated weights for policy 1, policy_version 29150 (0.0009) [2023-10-07 20:52:51,880][67838] Updated weights for policy 0, policy_version 29092 (0.0008) [2023-10-07 20:52:52,257][67838] Updated weights for policy 0, policy_version 29102 (0.0009) [2023-10-07 20:52:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 59637760. Throughput: 0: 1653.2, 1: 1669.6. Samples: 14925380. Policy #0 lag: (min: 21.0, avg: 21.3, max: 29.0) [2023-10-07 20:52:52,477][66916] Avg episode reward: [(0, '37.160'), (1, '38.470')] [2023-10-07 20:52:52,636][67838] Updated weights for policy 0, policy_version 29112 (0.0008) [2023-10-07 20:52:53,737][67871] Updated weights for policy 1, policy_version 29160 (0.0007) [2023-10-07 20:52:54,108][67871] Updated weights for policy 1, policy_version 29170 (0.0010) [2023-10-07 20:52:54,475][67871] Updated weights for policy 1, policy_version 29180 (0.0010) [2023-10-07 20:52:56,501][67838] Updated weights for policy 0, policy_version 29122 (0.0007) [2023-10-07 20:52:56,871][67838] Updated weights for policy 0, policy_version 29132 (0.0007) [2023-10-07 20:52:57,240][67838] Updated weights for policy 0, policy_version 29142 (0.0008) [2023-10-07 20:52:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 59703296. Throughput: 0: 1665.4, 1: 1656.7. Samples: 14934832. Policy #0 lag: (min: 21.0, avg: 21.3, max: 29.0) [2023-10-07 20:52:57,477][66916] Avg episode reward: [(0, '36.300'), (1, '36.680')] [2023-10-07 20:52:57,614][67838] Updated weights for policy 0, policy_version 29152 (0.0008) [2023-10-07 20:52:58,637][67871] Updated weights for policy 1, policy_version 29190 (0.0009) [2023-10-07 20:52:59,001][67871] Updated weights for policy 1, policy_version 29200 (0.0007) [2023-10-07 20:52:59,375][67871] Updated weights for policy 1, policy_version 29210 (0.0009) [2023-10-07 20:53:01,631][67838] Updated weights for policy 0, policy_version 29162 (0.0009) [2023-10-07 20:53:01,998][67838] Updated weights for policy 0, policy_version 29172 (0.0008) [2023-10-07 20:53:02,377][67838] Updated weights for policy 0, policy_version 29182 (0.0008) [2023-10-07 20:53:02,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 59801600. Throughput: 0: 1672.4, 1: 1661.1. Samples: 14955352. Policy #0 lag: (min: 21.0, avg: 21.3, max: 29.0) [2023-10-07 20:53:02,478][66916] Avg episode reward: [(0, '36.620'), (1, '37.960')] [2023-10-07 20:53:03,535][67871] Updated weights for policy 1, policy_version 29220 (0.0008) [2023-10-07 20:53:03,913][67871] Updated weights for policy 1, policy_version 29230 (0.0008) [2023-10-07 20:53:04,279][67871] Updated weights for policy 1, policy_version 29240 (0.0008) [2023-10-07 20:53:06,538][67838] Updated weights for policy 0, policy_version 29192 (0.0008) [2023-10-07 20:53:06,914][67838] Updated weights for policy 0, policy_version 29202 (0.0008) [2023-10-07 20:53:07,283][67838] Updated weights for policy 0, policy_version 29212 (0.0010) [2023-10-07 20:53:07,477][66916] Fps is (10 sec: 16383.4, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 59867136. Throughput: 0: 1653.7, 1: 1655.3. Samples: 14974758. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 20:53:07,478][66916] Avg episode reward: [(0, '37.300'), (1, '36.660')] [2023-10-07 20:53:07,490][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000029216_29917184.pth... [2023-10-07 20:53:07,490][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000029248_29949952.pth... [2023-10-07 20:53:07,523][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000027648_28311552.pth [2023-10-07 20:53:07,533][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000027712_28377088.pth [2023-10-07 20:53:08,543][67871] Updated weights for policy 1, policy_version 29250 (0.0008) [2023-10-07 20:53:08,919][67871] Updated weights for policy 1, policy_version 29260 (0.0009) [2023-10-07 20:53:09,282][67871] Updated weights for policy 1, policy_version 29270 (0.0009) [2023-10-07 20:53:09,656][67871] Updated weights for policy 1, policy_version 29280 (0.0009) [2023-10-07 20:53:11,715][67838] Updated weights for policy 0, policy_version 29222 (0.0009) [2023-10-07 20:53:12,083][67838] Updated weights for policy 0, policy_version 29232 (0.0009) [2023-10-07 20:53:12,465][67838] Updated weights for policy 0, policy_version 29242 (0.0008) [2023-10-07 20:53:12,477][66916] Fps is (10 sec: 9830.3, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 59899904. Throughput: 0: 1666.1, 1: 1652.4. Samples: 14984390. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 20:53:12,478][66916] Avg episode reward: [(0, '37.810'), (1, '36.900')] [2023-10-07 20:53:13,735][67871] Updated weights for policy 1, policy_version 29290 (0.0007) [2023-10-07 20:53:14,097][67871] Updated weights for policy 1, policy_version 29300 (0.0007) [2023-10-07 20:53:14,482][67871] Updated weights for policy 1, policy_version 29310 (0.0007) [2023-10-07 20:53:16,501][67838] Updated weights for policy 0, policy_version 29252 (0.0009) [2023-10-07 20:53:16,875][67838] Updated weights for policy 0, policy_version 29262 (0.0009) [2023-10-07 20:53:17,250][67838] Updated weights for policy 0, policy_version 29272 (0.0008) [2023-10-07 20:53:17,476][66916] Fps is (10 sec: 9830.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 59965440. Throughput: 0: 1670.5, 1: 1659.5. Samples: 15005046. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 20:53:17,477][66916] Avg episode reward: [(0, '39.370'), (1, '39.440')] [2023-10-07 20:53:18,531][67871] Updated weights for policy 1, policy_version 29320 (0.0009) [2023-10-07 20:53:18,903][67871] Updated weights for policy 1, policy_version 29330 (0.0010) [2023-10-07 20:53:19,269][67871] Updated weights for policy 1, policy_version 29340 (0.0008) [2023-10-07 20:53:21,272][67838] Updated weights for policy 0, policy_version 29282 (0.0008) [2023-10-07 20:53:21,647][67838] Updated weights for policy 0, policy_version 29292 (0.0009) [2023-10-07 20:53:22,024][67838] Updated weights for policy 0, policy_version 29302 (0.0009) [2023-10-07 20:53:22,398][67838] Updated weights for policy 0, policy_version 29312 (0.0011) [2023-10-07 20:53:22,476][66916] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 60063744. Throughput: 0: 1658.0, 1: 1659.7. Samples: 15024706. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 20:53:22,477][66916] Avg episode reward: [(0, '38.740'), (1, '37.020')] [2023-10-07 20:53:23,303][67871] Updated weights for policy 1, policy_version 29350 (0.0007) [2023-10-07 20:53:23,668][67871] Updated weights for policy 1, policy_version 29360 (0.0008) [2023-10-07 20:53:24,032][67871] Updated weights for policy 1, policy_version 29370 (0.0010) [2023-10-07 20:53:26,557][67838] Updated weights for policy 0, policy_version 29322 (0.0009) [2023-10-07 20:53:26,931][67838] Updated weights for policy 0, policy_version 29332 (0.0008) [2023-10-07 20:53:27,300][67838] Updated weights for policy 0, policy_version 29342 (0.0010) [2023-10-07 20:53:27,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 60129280. Throughput: 0: 1664.9, 1: 1659.9. Samples: 15034534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:53:27,478][66916] Avg episode reward: [(0, '41.250'), (1, '38.610')] [2023-10-07 20:53:28,237][67871] Updated weights for policy 1, policy_version 29380 (0.0010) [2023-10-07 20:53:28,604][67871] Updated weights for policy 1, policy_version 29390 (0.0008) [2023-10-07 20:53:28,969][67871] Updated weights for policy 1, policy_version 29400 (0.0010) [2023-10-07 20:53:31,347][67838] Updated weights for policy 0, policy_version 29352 (0.0010) [2023-10-07 20:53:31,709][67838] Updated weights for policy 0, policy_version 29362 (0.0010) [2023-10-07 20:53:32,088][67838] Updated weights for policy 0, policy_version 29372 (0.0010) [2023-10-07 20:53:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 60194816. Throughput: 0: 1666.6, 1: 1660.6. Samples: 15055058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:53:32,477][66916] Avg episode reward: [(0, '39.410'), (1, '37.550')] [2023-10-07 20:53:33,011][67871] Updated weights for policy 1, policy_version 29410 (0.0010) [2023-10-07 20:53:33,387][67871] Updated weights for policy 1, policy_version 29420 (0.0007) [2023-10-07 20:53:33,752][67871] Updated weights for policy 1, policy_version 29430 (0.0008) [2023-10-07 20:53:34,123][67871] Updated weights for policy 1, policy_version 29440 (0.0009) [2023-10-07 20:53:36,384][67838] Updated weights for policy 0, policy_version 29382 (0.0010) [2023-10-07 20:53:36,762][67838] Updated weights for policy 0, policy_version 29392 (0.0009) [2023-10-07 20:53:37,141][67838] Updated weights for policy 0, policy_version 29402 (0.0011) [2023-10-07 20:53:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 60260352. Throughput: 0: 1648.3, 1: 1663.7. Samples: 15074420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:53:37,477][66916] Avg episode reward: [(0, '38.450'), (1, '35.620')] [2023-10-07 20:53:38,056][67871] Updated weights for policy 1, policy_version 29450 (0.0010) [2023-10-07 20:53:38,427][67871] Updated weights for policy 1, policy_version 29460 (0.0009) [2023-10-07 20:53:38,796][67871] Updated weights for policy 1, policy_version 29470 (0.0007) [2023-10-07 20:53:41,208][67838] Updated weights for policy 0, policy_version 29412 (0.0008) [2023-10-07 20:53:41,585][67838] Updated weights for policy 0, policy_version 29422 (0.0008) [2023-10-07 20:53:41,967][67838] Updated weights for policy 0, policy_version 29432 (0.0010) [2023-10-07 20:53:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 60325888. Throughput: 0: 1658.0, 1: 1666.0. Samples: 15084414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:53:42,477][66916] Avg episode reward: [(0, '37.800'), (1, '37.450')] [2023-10-07 20:53:42,974][67871] Updated weights for policy 1, policy_version 29480 (0.0007) [2023-10-07 20:53:43,348][67871] Updated weights for policy 1, policy_version 29490 (0.0008) [2023-10-07 20:53:43,709][67871] Updated weights for policy 1, policy_version 29500 (0.0008) [2023-10-07 20:53:46,083][67838] Updated weights for policy 0, policy_version 29442 (0.0008) [2023-10-07 20:53:46,488][67838] Updated weights for policy 0, policy_version 29452 (0.0011) [2023-10-07 20:53:46,868][67838] Updated weights for policy 0, policy_version 29462 (0.0011) [2023-10-07 20:53:47,246][67838] Updated weights for policy 0, policy_version 29472 (0.0009) [2023-10-07 20:53:47,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 60391424. Throughput: 0: 1652.4, 1: 1665.3. Samples: 15104648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:53:47,478][66916] Avg episode reward: [(0, '37.970'), (1, '37.570')] [2023-10-07 20:53:47,958][67871] Updated weights for policy 1, policy_version 29510 (0.0009) [2023-10-07 20:53:48,335][67871] Updated weights for policy 1, policy_version 29520 (0.0008) [2023-10-07 20:53:48,695][67871] Updated weights for policy 1, policy_version 29530 (0.0008) [2023-10-07 20:53:51,371][67838] Updated weights for policy 0, policy_version 29482 (0.0007) [2023-10-07 20:53:51,731][67838] Updated weights for policy 0, policy_version 29492 (0.0008) [2023-10-07 20:53:52,098][67838] Updated weights for policy 0, policy_version 29502 (0.0009) [2023-10-07 20:53:52,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 60456960. Throughput: 0: 1648.8, 1: 1663.9. Samples: 15123832. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-07 20:53:52,478][66916] Avg episode reward: [(0, '38.480'), (1, '39.010')] [2023-10-07 20:53:52,847][67871] Updated weights for policy 1, policy_version 29540 (0.0008) [2023-10-07 20:53:53,216][67871] Updated weights for policy 1, policy_version 29550 (0.0009) [2023-10-07 20:53:53,583][67871] Updated weights for policy 1, policy_version 29560 (0.0011) [2023-10-07 20:53:56,262][67838] Updated weights for policy 0, policy_version 29512 (0.0008) [2023-10-07 20:53:56,635][67838] Updated weights for policy 0, policy_version 29522 (0.0009) [2023-10-07 20:53:57,016][67838] Updated weights for policy 0, policy_version 29532 (0.0007) [2023-10-07 20:53:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 60522496. Throughput: 0: 1655.4, 1: 1667.3. Samples: 15133912. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-07 20:53:57,477][66916] Avg episode reward: [(0, '38.020'), (1, '37.940')] [2023-10-07 20:53:57,632][67871] Updated weights for policy 1, policy_version 29570 (0.0011) [2023-10-07 20:53:58,007][67871] Updated weights for policy 1, policy_version 29580 (0.0008) [2023-10-07 20:53:58,376][67871] Updated weights for policy 1, policy_version 29590 (0.0008) [2023-10-07 20:53:58,737][67871] Updated weights for policy 1, policy_version 29600 (0.0007) [2023-10-07 20:54:00,881][67838] Updated weights for policy 0, policy_version 29542 (0.0008) [2023-10-07 20:54:01,252][67838] Updated weights for policy 0, policy_version 29552 (0.0009) [2023-10-07 20:54:01,625][67838] Updated weights for policy 0, policy_version 29562 (0.0007) [2023-10-07 20:54:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 60588032. Throughput: 0: 1647.5, 1: 1667.3. Samples: 15154212. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-07 20:54:02,478][66916] Avg episode reward: [(0, '37.530'), (1, '37.730')] [2023-10-07 20:54:02,833][67871] Updated weights for policy 1, policy_version 29610 (0.0007) [2023-10-07 20:54:03,195][67871] Updated weights for policy 1, policy_version 29620 (0.0009) [2023-10-07 20:54:03,567][67871] Updated weights for policy 1, policy_version 29630 (0.0009) [2023-10-07 20:54:05,597][67838] Updated weights for policy 0, policy_version 29572 (0.0009) [2023-10-07 20:54:05,963][67838] Updated weights for policy 0, policy_version 29582 (0.0011) [2023-10-07 20:54:06,337][67838] Updated weights for policy 0, policy_version 29592 (0.0010) [2023-10-07 20:54:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 60653568. Throughput: 0: 1651.2, 1: 1667.2. Samples: 15174036. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-07 20:54:07,477][66916] Avg episode reward: [(0, '35.660'), (1, '38.370')] [2023-10-07 20:54:07,813][67871] Updated weights for policy 1, policy_version 29640 (0.0007) [2023-10-07 20:54:08,182][67871] Updated weights for policy 1, policy_version 29650 (0.0009) [2023-10-07 20:54:08,552][67871] Updated weights for policy 1, policy_version 29660 (0.0007) [2023-10-07 20:54:10,547][67838] Updated weights for policy 0, policy_version 29602 (0.0011) [2023-10-07 20:54:10,919][67838] Updated weights for policy 0, policy_version 29612 (0.0011) [2023-10-07 20:54:11,299][67838] Updated weights for policy 0, policy_version 29622 (0.0009) [2023-10-07 20:54:11,672][67838] Updated weights for policy 0, policy_version 29632 (0.0010) [2023-10-07 20:54:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 60719104. Throughput: 0: 1660.8, 1: 1660.1. Samples: 15183976. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-07 20:54:12,477][66916] Avg episode reward: [(0, '36.190'), (1, '37.390')] [2023-10-07 20:54:12,702][67871] Updated weights for policy 1, policy_version 29670 (0.0007) [2023-10-07 20:54:13,071][67871] Updated weights for policy 1, policy_version 29680 (0.0009) [2023-10-07 20:54:13,444][67871] Updated weights for policy 1, policy_version 29690 (0.0009) [2023-10-07 20:54:15,860][67838] Updated weights for policy 0, policy_version 29642 (0.0009) [2023-10-07 20:54:16,244][67838] Updated weights for policy 0, policy_version 29652 (0.0009) [2023-10-07 20:54:16,619][67838] Updated weights for policy 0, policy_version 29662 (0.0007) [2023-10-07 20:54:17,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 60784640. Throughput: 0: 1644.3, 1: 1662.1. Samples: 15203846. Policy #0 lag: (min: 31.0, avg: 32.7, max: 60.0) [2023-10-07 20:54:17,477][66916] Avg episode reward: [(0, '37.690'), (1, '39.110')] [2023-10-07 20:54:17,542][67871] Updated weights for policy 1, policy_version 29700 (0.0009) [2023-10-07 20:54:17,917][67871] Updated weights for policy 1, policy_version 29710 (0.0008) [2023-10-07 20:54:18,283][67871] Updated weights for policy 1, policy_version 29720 (0.0008) [2023-10-07 20:54:20,679][67838] Updated weights for policy 0, policy_version 29672 (0.0007) [2023-10-07 20:54:21,063][67838] Updated weights for policy 0, policy_version 29682 (0.0010) [2023-10-07 20:54:21,436][67838] Updated weights for policy 0, policy_version 29692 (0.0011) [2023-10-07 20:54:22,237][67871] Updated weights for policy 1, policy_version 29730 (0.0007) [2023-10-07 20:54:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 60850176. Throughput: 0: 1652.6, 1: 1666.8. Samples: 15223794. Policy #0 lag: (min: 31.0, avg: 32.7, max: 60.0) [2023-10-07 20:54:22,477][66916] Avg episode reward: [(0, '36.280'), (1, '39.240')] [2023-10-07 20:54:22,601][67871] Updated weights for policy 1, policy_version 29740 (0.0007) [2023-10-07 20:54:22,968][67871] Updated weights for policy 1, policy_version 29750 (0.0007) [2023-10-07 20:54:23,336][67871] Updated weights for policy 1, policy_version 29760 (0.0008) [2023-10-07 20:54:25,607][67838] Updated weights for policy 0, policy_version 29702 (0.0010) [2023-10-07 20:54:25,976][67838] Updated weights for policy 0, policy_version 29712 (0.0008) [2023-10-07 20:54:26,355][67838] Updated weights for policy 0, policy_version 29722 (0.0007) [2023-10-07 20:54:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 60915712. Throughput: 0: 1660.3, 1: 1664.2. Samples: 15234016. Policy #0 lag: (min: 31.0, avg: 32.7, max: 60.0) [2023-10-07 20:54:27,477][66916] Avg episode reward: [(0, '36.920'), (1, '39.540')] [2023-10-07 20:54:27,511][67871] Updated weights for policy 1, policy_version 29770 (0.0010) [2023-10-07 20:54:27,887][67871] Updated weights for policy 1, policy_version 29780 (0.0011) [2023-10-07 20:54:28,262][67871] Updated weights for policy 1, policy_version 29790 (0.0009) [2023-10-07 20:54:30,377][67838] Updated weights for policy 0, policy_version 29732 (0.0008) [2023-10-07 20:54:30,758][67838] Updated weights for policy 0, policy_version 29742 (0.0007) [2023-10-07 20:54:31,131][67838] Updated weights for policy 0, policy_version 29752 (0.0010) [2023-10-07 20:54:32,379][67871] Updated weights for policy 1, policy_version 29800 (0.0009) [2023-10-07 20:54:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 60981248. Throughput: 0: 1647.4, 1: 1665.1. Samples: 15253710. Policy #0 lag: (min: 31.0, avg: 32.7, max: 60.0) [2023-10-07 20:54:32,477][66916] Avg episode reward: [(0, '35.990'), (1, '38.580')] [2023-10-07 20:54:32,745][67871] Updated weights for policy 1, policy_version 29810 (0.0007) [2023-10-07 20:54:33,115][67871] Updated weights for policy 1, policy_version 29820 (0.0008) [2023-10-07 20:54:35,236][67838] Updated weights for policy 0, policy_version 29762 (0.0011) [2023-10-07 20:54:35,626][67838] Updated weights for policy 0, policy_version 29772 (0.0009) [2023-10-07 20:54:35,997][67838] Updated weights for policy 0, policy_version 29782 (0.0009) [2023-10-07 20:54:36,360][67838] Updated weights for policy 0, policy_version 29792 (0.0008) [2023-10-07 20:54:37,066][67871] Updated weights for policy 1, policy_version 29830 (0.0009) [2023-10-07 20:54:37,430][67871] Updated weights for policy 1, policy_version 29840 (0.0008) [2023-10-07 20:54:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 61046784. Throughput: 0: 1661.9, 1: 1672.5. Samples: 15273876. Policy #0 lag: (min: 31.0, avg: 32.7, max: 60.0) [2023-10-07 20:54:37,477][66916] Avg episode reward: [(0, '34.400'), (1, '37.800')] [2023-10-07 20:54:37,785][67871] Updated weights for policy 1, policy_version 29850 (0.0009) [2023-10-07 20:54:40,547][67838] Updated weights for policy 0, policy_version 29802 (0.0009) [2023-10-07 20:54:40,923][67838] Updated weights for policy 0, policy_version 29812 (0.0008) [2023-10-07 20:54:41,298][67838] Updated weights for policy 0, policy_version 29822 (0.0008) [2023-10-07 20:54:41,926][67871] Updated weights for policy 1, policy_version 29860 (0.0011) [2023-10-07 20:54:42,302][67871] Updated weights for policy 1, policy_version 29870 (0.0010) [2023-10-07 20:54:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 61112320. Throughput: 0: 1667.0, 1: 1669.5. Samples: 15284056. Policy #0 lag: (min: 1.0, avg: 11.5, max: 33.0) [2023-10-07 20:54:42,477][66916] Avg episode reward: [(0, '38.010'), (1, '38.540')] [2023-10-07 20:54:42,671][67871] Updated weights for policy 1, policy_version 29880 (0.0007) [2023-10-07 20:54:45,384][67838] Updated weights for policy 0, policy_version 29832 (0.0008) [2023-10-07 20:54:45,756][67838] Updated weights for policy 0, policy_version 29842 (0.0007) [2023-10-07 20:54:46,127][67838] Updated weights for policy 0, policy_version 29852 (0.0007) [2023-10-07 20:54:46,807][67871] Updated weights for policy 1, policy_version 29890 (0.0008) [2023-10-07 20:54:47,182][67871] Updated weights for policy 1, policy_version 29900 (0.0010) [2023-10-07 20:54:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 61177856. Throughput: 0: 1649.6, 1: 1668.3. Samples: 15303516. Policy #0 lag: (min: 1.0, avg: 11.5, max: 33.0) [2023-10-07 20:54:47,477][66916] Avg episode reward: [(0, '38.380'), (1, '37.410')] [2023-10-07 20:54:47,557][67871] Updated weights for policy 1, policy_version 29910 (0.0009) [2023-10-07 20:54:47,920][67871] Updated weights for policy 1, policy_version 29920 (0.0009) [2023-10-07 20:54:50,222][67838] Updated weights for policy 0, policy_version 29862 (0.0007) [2023-10-07 20:54:50,588][67838] Updated weights for policy 0, policy_version 29872 (0.0009) [2023-10-07 20:54:50,970][67838] Updated weights for policy 0, policy_version 29882 (0.0007) [2023-10-07 20:54:52,163][67871] Updated weights for policy 1, policy_version 29930 (0.0009) [2023-10-07 20:54:52,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 61243392. Throughput: 0: 1660.3, 1: 1662.9. Samples: 15323580. Policy #0 lag: (min: 1.0, avg: 11.5, max: 33.0) [2023-10-07 20:54:52,478][66916] Avg episode reward: [(0, '40.240'), (1, '38.560')] [2023-10-07 20:54:52,544][67871] Updated weights for policy 1, policy_version 29940 (0.0010) [2023-10-07 20:54:52,917][67871] Updated weights for policy 1, policy_version 29950 (0.0009) [2023-10-07 20:54:55,041][67838] Updated weights for policy 0, policy_version 29892 (0.0008) [2023-10-07 20:54:55,425][67838] Updated weights for policy 0, policy_version 29902 (0.0009) [2023-10-07 20:54:55,792][67838] Updated weights for policy 0, policy_version 29912 (0.0009) [2023-10-07 20:54:56,920][67871] Updated weights for policy 1, policy_version 29960 (0.0010) [2023-10-07 20:54:57,287][67871] Updated weights for policy 1, policy_version 29970 (0.0011) [2023-10-07 20:54:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 61308928. Throughput: 0: 1658.4, 1: 1670.0. Samples: 15333754. Policy #0 lag: (min: 1.0, avg: 11.5, max: 33.0) [2023-10-07 20:54:57,478][66916] Avg episode reward: [(0, '40.680'), (1, '39.350')] [2023-10-07 20:54:57,650][67871] Updated weights for policy 1, policy_version 29980 (0.0010) [2023-10-07 20:55:00,066][67838] Updated weights for policy 0, policy_version 29922 (0.0008) [2023-10-07 20:55:00,437][67838] Updated weights for policy 0, policy_version 29932 (0.0007) [2023-10-07 20:55:00,815][67838] Updated weights for policy 0, policy_version 29942 (0.0007) [2023-10-07 20:55:01,180][67838] Updated weights for policy 0, policy_version 29952 (0.0009) [2023-10-07 20:55:01,790][67871] Updated weights for policy 1, policy_version 29990 (0.0009) [2023-10-07 20:55:02,165][67871] Updated weights for policy 1, policy_version 30000 (0.0007) [2023-10-07 20:55:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 61374464. Throughput: 0: 1654.4, 1: 1664.5. Samples: 15353196. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-07 20:55:02,477][66916] Avg episode reward: [(0, '39.850'), (1, '40.070')] [2023-10-07 20:55:02,529][67871] Updated weights for policy 1, policy_version 30010 (0.0009) [2023-10-07 20:55:05,128][67838] Updated weights for policy 0, policy_version 29962 (0.0009) [2023-10-07 20:55:05,492][67838] Updated weights for policy 0, policy_version 29972 (0.0008) [2023-10-07 20:55:05,868][67838] Updated weights for policy 0, policy_version 29982 (0.0007) [2023-10-07 20:55:06,716][67871] Updated weights for policy 1, policy_version 30020 (0.0008) [2023-10-07 20:55:07,090][67871] Updated weights for policy 1, policy_version 30030 (0.0007) [2023-10-07 20:55:07,468][67871] Updated weights for policy 1, policy_version 30040 (0.0008) [2023-10-07 20:55:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 61440000. Throughput: 0: 1668.6, 1: 1653.2. Samples: 15373278. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-07 20:55:07,478][66916] Avg episode reward: [(0, '39.510'), (1, '39.420')] [2023-10-07 20:55:07,486][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000029984_30703616.pth... [2023-10-07 20:55:07,522][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000028416_29097984.pth [2023-10-07 20:55:07,755][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000030048_30769152.pth... [2023-10-07 20:55:07,794][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000028480_29163520.pth [2023-10-07 20:55:09,982][67838] Updated weights for policy 0, policy_version 29992 (0.0007) [2023-10-07 20:55:10,355][67838] Updated weights for policy 0, policy_version 30002 (0.0009) [2023-10-07 20:55:10,727][67838] Updated weights for policy 0, policy_version 30012 (0.0009) [2023-10-07 20:55:11,525][67871] Updated weights for policy 1, policy_version 30050 (0.0008) [2023-10-07 20:55:11,892][67871] Updated weights for policy 1, policy_version 30060 (0.0010) [2023-10-07 20:55:12,263][67871] Updated weights for policy 1, policy_version 30070 (0.0008) [2023-10-07 20:55:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 61505536. Throughput: 0: 1663.3, 1: 1658.9. Samples: 15383516. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-07 20:55:12,477][66916] Avg episode reward: [(0, '38.300'), (1, '39.060')] [2023-10-07 20:55:12,632][67871] Updated weights for policy 1, policy_version 30080 (0.0007) [2023-10-07 20:55:14,777][67838] Updated weights for policy 0, policy_version 30022 (0.0007) [2023-10-07 20:55:15,156][67838] Updated weights for policy 0, policy_version 30032 (0.0009) [2023-10-07 20:55:15,530][67838] Updated weights for policy 0, policy_version 30042 (0.0009) [2023-10-07 20:55:17,000][67871] Updated weights for policy 1, policy_version 30090 (0.0011) [2023-10-07 20:55:17,371][67871] Updated weights for policy 1, policy_version 30100 (0.0008) [2023-10-07 20:55:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 61571072. Throughput: 0: 1661.2, 1: 1662.1. Samples: 15403260. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-07 20:55:17,478][66916] Avg episode reward: [(0, '39.390'), (1, '38.480')] [2023-10-07 20:55:17,744][67871] Updated weights for policy 1, policy_version 30110 (0.0008) [2023-10-07 20:55:19,760][67838] Updated weights for policy 0, policy_version 30052 (0.0008) [2023-10-07 20:55:20,131][67838] Updated weights for policy 0, policy_version 30062 (0.0011) [2023-10-07 20:55:20,493][67838] Updated weights for policy 0, policy_version 30072 (0.0009) [2023-10-07 20:55:21,822][67871] Updated weights for policy 1, policy_version 30120 (0.0008) [2023-10-07 20:55:22,191][67871] Updated weights for policy 1, policy_version 30130 (0.0007) [2023-10-07 20:55:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 61636608. Throughput: 0: 1668.7, 1: 1647.6. Samples: 15423110. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-07 20:55:22,478][66916] Avg episode reward: [(0, '39.460'), (1, '36.840')] [2023-10-07 20:55:22,551][67871] Updated weights for policy 1, policy_version 30140 (0.0008) [2023-10-07 20:55:24,672][67838] Updated weights for policy 0, policy_version 30082 (0.0009) [2023-10-07 20:55:25,085][67838] Updated weights for policy 0, policy_version 30092 (0.0008) [2023-10-07 20:55:25,454][67838] Updated weights for policy 0, policy_version 30102 (0.0009) [2023-10-07 20:55:25,829][67838] Updated weights for policy 0, policy_version 30112 (0.0008) [2023-10-07 20:55:26,848][67871] Updated weights for policy 1, policy_version 30150 (0.0009) [2023-10-07 20:55:27,210][67871] Updated weights for policy 1, policy_version 30160 (0.0008) [2023-10-07 20:55:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 61702144. Throughput: 0: 1658.6, 1: 1657.5. Samples: 15433280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:55:27,478][66916] Avg episode reward: [(0, '34.090'), (1, '36.440')] [2023-10-07 20:55:27,572][67871] Updated weights for policy 1, policy_version 30170 (0.0009) [2023-10-07 20:55:29,763][67838] Updated weights for policy 0, policy_version 30122 (0.0009) [2023-10-07 20:55:30,128][67838] Updated weights for policy 0, policy_version 30132 (0.0010) [2023-10-07 20:55:30,495][67838] Updated weights for policy 0, policy_version 30142 (0.0010) [2023-10-07 20:55:31,513][67871] Updated weights for policy 1, policy_version 30180 (0.0010) [2023-10-07 20:55:31,877][67871] Updated weights for policy 1, policy_version 30190 (0.0010) [2023-10-07 20:55:32,249][67871] Updated weights for policy 1, policy_version 30200 (0.0010) [2023-10-07 20:55:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 61767680. Throughput: 0: 1663.7, 1: 1659.5. Samples: 15453062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:55:32,477][66916] Avg episode reward: [(0, '33.710'), (1, '37.730')] [2023-10-07 20:55:34,638][67838] Updated weights for policy 0, policy_version 30152 (0.0010) [2023-10-07 20:55:35,006][67838] Updated weights for policy 0, policy_version 30162 (0.0007) [2023-10-07 20:55:35,387][67838] Updated weights for policy 0, policy_version 30172 (0.0008) [2023-10-07 20:55:36,344][67871] Updated weights for policy 1, policy_version 30210 (0.0010) [2023-10-07 20:55:36,717][67871] Updated weights for policy 1, policy_version 30220 (0.0010) [2023-10-07 20:55:37,082][67871] Updated weights for policy 1, policy_version 30230 (0.0008) [2023-10-07 20:55:37,446][67871] Updated weights for policy 1, policy_version 30240 (0.0009) [2023-10-07 20:55:37,476][66916] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 61865984. Throughput: 0: 1673.5, 1: 1650.0. Samples: 15473134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:55:37,477][66916] Avg episode reward: [(0, '37.020'), (1, '37.350')] [2023-10-07 20:55:39,439][67838] Updated weights for policy 0, policy_version 30182 (0.0007) [2023-10-07 20:55:39,806][67838] Updated weights for policy 0, policy_version 30192 (0.0011) [2023-10-07 20:55:40,180][67838] Updated weights for policy 0, policy_version 30202 (0.0009) [2023-10-07 20:55:41,627][67871] Updated weights for policy 1, policy_version 30250 (0.0009) [2023-10-07 20:55:42,007][67871] Updated weights for policy 1, policy_version 30260 (0.0008) [2023-10-07 20:55:42,383][67871] Updated weights for policy 1, policy_version 30270 (0.0009) [2023-10-07 20:55:42,476][66916] Fps is (10 sec: 16384.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 61931520. Throughput: 0: 1658.5, 1: 1662.5. Samples: 15483196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:55:42,477][66916] Avg episode reward: [(0, '36.030'), (1, '38.820')] [2023-10-07 20:55:44,552][67838] Updated weights for policy 0, policy_version 30212 (0.0011) [2023-10-07 20:55:44,922][67838] Updated weights for policy 0, policy_version 30222 (0.0008) [2023-10-07 20:55:45,295][67838] Updated weights for policy 0, policy_version 30232 (0.0009) [2023-10-07 20:55:46,382][67871] Updated weights for policy 1, policy_version 30280 (0.0008) [2023-10-07 20:55:46,754][67871] Updated weights for policy 1, policy_version 30290 (0.0007) [2023-10-07 20:55:47,126][67871] Updated weights for policy 1, policy_version 30300 (0.0008) [2023-10-07 20:55:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 61997056. Throughput: 0: 1667.3, 1: 1666.8. Samples: 15503228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:55:47,477][66916] Avg episode reward: [(0, '39.100'), (1, '36.950')] [2023-10-07 20:55:49,375][67838] Updated weights for policy 0, policy_version 30242 (0.0011) [2023-10-07 20:55:49,743][67838] Updated weights for policy 0, policy_version 30252 (0.0008) [2023-10-07 20:55:50,115][67838] Updated weights for policy 0, policy_version 30262 (0.0009) [2023-10-07 20:55:50,486][67838] Updated weights for policy 0, policy_version 30272 (0.0008) [2023-10-07 20:55:51,144][67871] Updated weights for policy 1, policy_version 30310 (0.0011) [2023-10-07 20:55:51,508][67871] Updated weights for policy 1, policy_version 30320 (0.0008) [2023-10-07 20:55:51,887][67871] Updated weights for policy 1, policy_version 30330 (0.0010) [2023-10-07 20:55:52,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 62062592. Throughput: 0: 1665.2, 1: 1655.2. Samples: 15522692. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 20:55:52,478][66916] Avg episode reward: [(0, '38.500'), (1, '38.210')] [2023-10-07 20:55:54,613][67838] Updated weights for policy 0, policy_version 30282 (0.0009) [2023-10-07 20:55:54,992][67838] Updated weights for policy 0, policy_version 30292 (0.0007) [2023-10-07 20:55:55,370][67838] Updated weights for policy 0, policy_version 30302 (0.0010) [2023-10-07 20:55:56,129][67871] Updated weights for policy 1, policy_version 30340 (0.0007) [2023-10-07 20:55:56,505][67871] Updated weights for policy 1, policy_version 30350 (0.0008) [2023-10-07 20:55:56,868][67871] Updated weights for policy 1, policy_version 30360 (0.0007) [2023-10-07 20:55:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 62128128. Throughput: 0: 1650.8, 1: 1669.2. Samples: 15532918. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 20:55:57,477][66916] Avg episode reward: [(0, '37.640'), (1, '38.660')] [2023-10-07 20:55:59,500][67838] Updated weights for policy 0, policy_version 30312 (0.0009) [2023-10-07 20:55:59,871][67838] Updated weights for policy 0, policy_version 30322 (0.0007) [2023-10-07 20:56:00,240][67838] Updated weights for policy 0, policy_version 30332 (0.0008) [2023-10-07 20:56:00,906][67871] Updated weights for policy 1, policy_version 30370 (0.0008) [2023-10-07 20:56:01,277][67871] Updated weights for policy 1, policy_version 30380 (0.0010) [2023-10-07 20:56:01,658][67871] Updated weights for policy 1, policy_version 30390 (0.0009) [2023-10-07 20:56:02,025][67871] Updated weights for policy 1, policy_version 30400 (0.0010) [2023-10-07 20:56:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 62193664. Throughput: 0: 1658.8, 1: 1671.6. Samples: 15553128. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 20:56:02,477][66916] Avg episode reward: [(0, '35.790'), (1, '38.520')] [2023-10-07 20:56:04,089][67838] Updated weights for policy 0, policy_version 30342 (0.0007) [2023-10-07 20:56:04,459][67838] Updated weights for policy 0, policy_version 30352 (0.0008) [2023-10-07 20:56:04,835][67838] Updated weights for policy 0, policy_version 30362 (0.0008) [2023-10-07 20:56:05,943][67871] Updated weights for policy 1, policy_version 30410 (0.0009) [2023-10-07 20:56:06,306][67871] Updated weights for policy 1, policy_version 30420 (0.0009) [2023-10-07 20:56:06,671][67871] Updated weights for policy 1, policy_version 30430 (0.0011) [2023-10-07 20:56:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 62259200. Throughput: 0: 1665.5, 1: 1658.7. Samples: 15572698. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 20:56:07,477][66916] Avg episode reward: [(0, '34.740'), (1, '40.820')] [2023-10-07 20:56:09,010][67838] Updated weights for policy 0, policy_version 30372 (0.0008) [2023-10-07 20:56:09,401][67838] Updated weights for policy 0, policy_version 30382 (0.0010) [2023-10-07 20:56:09,768][67838] Updated weights for policy 0, policy_version 30392 (0.0009) [2023-10-07 20:56:10,781][67871] Updated weights for policy 1, policy_version 30440 (0.0009) [2023-10-07 20:56:11,148][67871] Updated weights for policy 1, policy_version 30450 (0.0009) [2023-10-07 20:56:11,516][67871] Updated weights for policy 1, policy_version 30460 (0.0008) [2023-10-07 20:56:12,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 62324736. Throughput: 0: 1646.1, 1: 1683.0. Samples: 15583092. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 20:56:12,478][66916] Avg episode reward: [(0, '37.170'), (1, '39.590')] [2023-10-07 20:56:13,812][67838] Updated weights for policy 0, policy_version 30402 (0.0008) [2023-10-07 20:56:14,196][67838] Updated weights for policy 0, policy_version 30412 (0.0010) [2023-10-07 20:56:14,563][67838] Updated weights for policy 0, policy_version 30422 (0.0010) [2023-10-07 20:56:14,937][67838] Updated weights for policy 0, policy_version 30432 (0.0010) [2023-10-07 20:56:15,685][67871] Updated weights for policy 1, policy_version 30470 (0.0009) [2023-10-07 20:56:16,068][67871] Updated weights for policy 1, policy_version 30480 (0.0009) [2023-10-07 20:56:16,435][67871] Updated weights for policy 1, policy_version 30490 (0.0007) [2023-10-07 20:56:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 62390272. Throughput: 0: 1664.0, 1: 1670.3. Samples: 15603104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:56:17,478][66916] Avg episode reward: [(0, '36.830'), (1, '42.430')] [2023-10-07 20:56:18,939][67838] Updated weights for policy 0, policy_version 30442 (0.0008) [2023-10-07 20:56:19,314][67838] Updated weights for policy 0, policy_version 30452 (0.0008) [2023-10-07 20:56:19,673][67838] Updated weights for policy 0, policy_version 30462 (0.0009) [2023-10-07 20:56:20,361][67871] Updated weights for policy 1, policy_version 30500 (0.0008) [2023-10-07 20:56:20,728][67871] Updated weights for policy 1, policy_version 30510 (0.0009) [2023-10-07 20:56:21,101][67871] Updated weights for policy 1, policy_version 30520 (0.0009) [2023-10-07 20:56:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 62455808. Throughput: 0: 1661.6, 1: 1667.8. Samples: 15622958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:56:22,477][66916] Avg episode reward: [(0, '37.170'), (1, '39.580')] [2023-10-07 20:56:23,977][67838] Updated weights for policy 0, policy_version 30472 (0.0009) [2023-10-07 20:56:24,360][67838] Updated weights for policy 0, policy_version 30482 (0.0010) [2023-10-07 20:56:24,727][67838] Updated weights for policy 0, policy_version 30492 (0.0011) [2023-10-07 20:56:25,322][67871] Updated weights for policy 1, policy_version 30530 (0.0010) [2023-10-07 20:56:25,684][67871] Updated weights for policy 1, policy_version 30540 (0.0010) [2023-10-07 20:56:26,054][67871] Updated weights for policy 1, policy_version 30550 (0.0007) [2023-10-07 20:56:26,427][67871] Updated weights for policy 1, policy_version 30560 (0.0008) [2023-10-07 20:56:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 62521344. Throughput: 0: 1651.1, 1: 1679.3. Samples: 15633064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:56:27,478][66916] Avg episode reward: [(0, '37.170'), (1, '41.710')] [2023-10-07 20:56:28,785][67838] Updated weights for policy 0, policy_version 30502 (0.0009) [2023-10-07 20:56:29,160][67838] Updated weights for policy 0, policy_version 30512 (0.0009) [2023-10-07 20:56:29,541][67838] Updated weights for policy 0, policy_version 30522 (0.0009) [2023-10-07 20:56:30,600][67871] Updated weights for policy 1, policy_version 30570 (0.0011) [2023-10-07 20:56:30,970][67871] Updated weights for policy 1, policy_version 30580 (0.0009) [2023-10-07 20:56:31,338][67871] Updated weights for policy 1, policy_version 30590 (0.0010) [2023-10-07 20:56:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 62586880. Throughput: 0: 1662.2, 1: 1660.7. Samples: 15652758. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:56:32,478][66916] Avg episode reward: [(0, '37.640'), (1, '41.870')] [2023-10-07 20:56:33,646][67838] Updated weights for policy 0, policy_version 30532 (0.0008) [2023-10-07 20:56:34,012][67838] Updated weights for policy 0, policy_version 30542 (0.0009) [2023-10-07 20:56:34,395][67838] Updated weights for policy 0, policy_version 30552 (0.0008) [2023-10-07 20:56:35,445][67871] Updated weights for policy 1, policy_version 30600 (0.0008) [2023-10-07 20:56:35,814][67871] Updated weights for policy 1, policy_version 30610 (0.0009) [2023-10-07 20:56:36,176][67871] Updated weights for policy 1, policy_version 30620 (0.0008) [2023-10-07 20:56:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 62652416. Throughput: 0: 1668.4, 1: 1664.4. Samples: 15672670. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:56:37,477][66916] Avg episode reward: [(0, '35.880'), (1, '42.390')] [2023-10-07 20:56:38,526][67838] Updated weights for policy 0, policy_version 30562 (0.0009) [2023-10-07 20:56:38,900][67838] Updated weights for policy 0, policy_version 30572 (0.0008) [2023-10-07 20:56:39,276][67838] Updated weights for policy 0, policy_version 30582 (0.0008) [2023-10-07 20:56:39,645][67838] Updated weights for policy 0, policy_version 30592 (0.0007) [2023-10-07 20:56:40,202][67871] Updated weights for policy 1, policy_version 30630 (0.0008) [2023-10-07 20:56:40,561][67871] Updated weights for policy 1, policy_version 30640 (0.0009) [2023-10-07 20:56:40,923][67871] Updated weights for policy 1, policy_version 30650 (0.0008) [2023-10-07 20:56:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 62717952. Throughput: 0: 1661.2, 1: 1676.0. Samples: 15683092. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 20:56:42,478][66916] Avg episode reward: [(0, '36.470'), (1, '43.680')] [2023-10-07 20:56:42,479][67676] Saving new best policy, reward=43.680! [2023-10-07 20:56:43,761][67838] Updated weights for policy 0, policy_version 30602 (0.0007) [2023-10-07 20:56:44,125][67838] Updated weights for policy 0, policy_version 30612 (0.0010) [2023-10-07 20:56:44,496][67838] Updated weights for policy 0, policy_version 30622 (0.0011) [2023-10-07 20:56:45,120][67871] Updated weights for policy 1, policy_version 30660 (0.0008) [2023-10-07 20:56:45,483][67871] Updated weights for policy 1, policy_version 30670 (0.0007) [2023-10-07 20:56:45,856][67871] Updated weights for policy 1, policy_version 30680 (0.0007) [2023-10-07 20:56:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 62783488. Throughput: 0: 1667.5, 1: 1653.5. Samples: 15702572. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 20:56:47,477][66916] Avg episode reward: [(0, '38.490'), (1, '42.440')] [2023-10-07 20:56:48,495][67838] Updated weights for policy 0, policy_version 30632 (0.0011) [2023-10-07 20:56:48,863][67838] Updated weights for policy 0, policy_version 30642 (0.0008) [2023-10-07 20:56:49,229][67838] Updated weights for policy 0, policy_version 30652 (0.0008) [2023-10-07 20:56:49,857][67871] Updated weights for policy 1, policy_version 30690 (0.0008) [2023-10-07 20:56:50,232][67871] Updated weights for policy 1, policy_version 30700 (0.0008) [2023-10-07 20:56:50,590][67871] Updated weights for policy 1, policy_version 30710 (0.0009) [2023-10-07 20:56:50,963][67871] Updated weights for policy 1, policy_version 30720 (0.0007) [2023-10-07 20:56:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 62849024. Throughput: 0: 1662.0, 1: 1671.3. Samples: 15722694. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 20:56:52,477][66916] Avg episode reward: [(0, '34.500'), (1, '41.430')] [2023-10-07 20:56:53,558][67838] Updated weights for policy 0, policy_version 30662 (0.0008) [2023-10-07 20:56:53,924][67838] Updated weights for policy 0, policy_version 30672 (0.0009) [2023-10-07 20:56:54,293][67838] Updated weights for policy 0, policy_version 30682 (0.0010) [2023-10-07 20:56:54,982][67871] Updated weights for policy 1, policy_version 30730 (0.0009) [2023-10-07 20:56:55,357][67871] Updated weights for policy 1, policy_version 30740 (0.0009) [2023-10-07 20:56:55,728][67871] Updated weights for policy 1, policy_version 30750 (0.0010) [2023-10-07 20:56:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 62914560. Throughput: 0: 1663.1, 1: 1665.2. Samples: 15732866. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 20:56:57,477][66916] Avg episode reward: [(0, '37.380'), (1, '41.960')] [2023-10-07 20:56:58,400][67838] Updated weights for policy 0, policy_version 30692 (0.0010) [2023-10-07 20:56:58,782][67838] Updated weights for policy 0, policy_version 30702 (0.0007) [2023-10-07 20:56:59,161][67838] Updated weights for policy 0, policy_version 30712 (0.0007) [2023-10-07 20:56:59,784][67871] Updated weights for policy 1, policy_version 30760 (0.0010) [2023-10-07 20:57:00,153][67871] Updated weights for policy 1, policy_version 30770 (0.0009) [2023-10-07 20:57:00,504][67871] Updated weights for policy 1, policy_version 30780 (0.0008) [2023-10-07 20:57:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 62980096. Throughput: 0: 1668.2, 1: 1656.0. Samples: 15752692. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 20:57:02,477][66916] Avg episode reward: [(0, '35.880'), (1, '39.140')] [2023-10-07 20:57:03,019][67838] Updated weights for policy 0, policy_version 30722 (0.0007) [2023-10-07 20:57:03,391][67838] Updated weights for policy 0, policy_version 30732 (0.0009) [2023-10-07 20:57:03,760][67838] Updated weights for policy 0, policy_version 30742 (0.0008) [2023-10-07 20:57:04,129][67838] Updated weights for policy 0, policy_version 30752 (0.0010) [2023-10-07 20:57:04,625][67871] Updated weights for policy 1, policy_version 30790 (0.0009) [2023-10-07 20:57:04,998][67871] Updated weights for policy 1, policy_version 30800 (0.0007) [2023-10-07 20:57:05,375][67871] Updated weights for policy 1, policy_version 30810 (0.0007) [2023-10-07 20:57:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 63045632. Throughput: 0: 1671.2, 1: 1670.9. Samples: 15773354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:57:07,478][66916] Avg episode reward: [(0, '35.540'), (1, '39.770')] [2023-10-07 20:57:07,488][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000030816_31555584.pth... [2023-10-07 20:57:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000030752_31490048.pth... [2023-10-07 20:57:07,518][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000029248_29949952.pth [2023-10-07 20:57:07,523][67676] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p1/milestones/checkpoint_000030816_31555584.pth [2023-10-07 20:57:07,527][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000029216_29917184.pth [2023-10-07 20:57:07,531][67511] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p0/milestones/checkpoint_000030752_31490048.pth [2023-10-07 20:57:08,077][67838] Updated weights for policy 0, policy_version 30762 (0.0009) [2023-10-07 20:57:08,440][67838] Updated weights for policy 0, policy_version 30772 (0.0007) [2023-10-07 20:57:08,826][67838] Updated weights for policy 0, policy_version 30782 (0.0009) [2023-10-07 20:57:09,479][67871] Updated weights for policy 1, policy_version 30820 (0.0007) [2023-10-07 20:57:09,853][67871] Updated weights for policy 1, policy_version 30830 (0.0007) [2023-10-07 20:57:10,223][67871] Updated weights for policy 1, policy_version 30840 (0.0007) [2023-10-07 20:57:12,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 63111168. Throughput: 0: 1672.5, 1: 1660.3. Samples: 15783040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:57:12,478][66916] Avg episode reward: [(0, '36.460'), (1, '38.750')] [2023-10-07 20:57:12,956][67838] Updated weights for policy 0, policy_version 30792 (0.0010) [2023-10-07 20:57:13,325][67838] Updated weights for policy 0, policy_version 30802 (0.0009) [2023-10-07 20:57:13,690][67838] Updated weights for policy 0, policy_version 30812 (0.0009) [2023-10-07 20:57:14,398][67871] Updated weights for policy 1, policy_version 30850 (0.0009) [2023-10-07 20:57:14,776][67871] Updated weights for policy 1, policy_version 30860 (0.0009) [2023-10-07 20:57:15,151][67871] Updated weights for policy 1, policy_version 30870 (0.0008) [2023-10-07 20:57:15,511][67871] Updated weights for policy 1, policy_version 30880 (0.0009) [2023-10-07 20:57:17,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63176704. Throughput: 0: 1671.2, 1: 1663.5. Samples: 15802820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:57:17,477][66916] Avg episode reward: [(0, '35.300'), (1, '36.720')] [2023-10-07 20:57:17,879][67838] Updated weights for policy 0, policy_version 30822 (0.0008) [2023-10-07 20:57:18,243][67838] Updated weights for policy 0, policy_version 30832 (0.0008) [2023-10-07 20:57:18,618][67838] Updated weights for policy 0, policy_version 30842 (0.0007) [2023-10-07 20:57:19,528][67871] Updated weights for policy 1, policy_version 30890 (0.0008) [2023-10-07 20:57:19,893][67871] Updated weights for policy 1, policy_version 30900 (0.0010) [2023-10-07 20:57:20,265][67871] Updated weights for policy 1, policy_version 30910 (0.0009) [2023-10-07 20:57:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63242240. Throughput: 0: 1669.2, 1: 1675.8. Samples: 15823196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:57:22,478][66916] Avg episode reward: [(0, '34.780'), (1, '37.480')] [2023-10-07 20:57:22,825][67838] Updated weights for policy 0, policy_version 30852 (0.0010) [2023-10-07 20:57:23,197][67838] Updated weights for policy 0, policy_version 30862 (0.0008) [2023-10-07 20:57:23,578][67838] Updated weights for policy 0, policy_version 30872 (0.0008) [2023-10-07 20:57:24,471][67871] Updated weights for policy 1, policy_version 30920 (0.0008) [2023-10-07 20:57:24,836][67871] Updated weights for policy 1, policy_version 30930 (0.0007) [2023-10-07 20:57:25,213][67871] Updated weights for policy 1, policy_version 30940 (0.0008) [2023-10-07 20:57:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63307776. Throughput: 0: 1669.5, 1: 1659.6. Samples: 15832898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:57:27,477][66916] Avg episode reward: [(0, '37.190'), (1, '38.370')] [2023-10-07 20:57:27,532][67838] Updated weights for policy 0, policy_version 30882 (0.0010) [2023-10-07 20:57:27,906][67838] Updated weights for policy 0, policy_version 30892 (0.0009) [2023-10-07 20:57:28,277][67838] Updated weights for policy 0, policy_version 30902 (0.0008) [2023-10-07 20:57:28,642][67838] Updated weights for policy 0, policy_version 30912 (0.0010) [2023-10-07 20:57:29,338][67871] Updated weights for policy 1, policy_version 30950 (0.0009) [2023-10-07 20:57:29,699][67871] Updated weights for policy 1, policy_version 30960 (0.0008) [2023-10-07 20:57:30,073][67871] Updated weights for policy 1, policy_version 30970 (0.0008) [2023-10-07 20:57:32,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63373312. Throughput: 0: 1671.4, 1: 1662.3. Samples: 15852588. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-10-07 20:57:32,478][66916] Avg episode reward: [(0, '38.000'), (1, '39.100')] [2023-10-07 20:57:32,885][67838] Updated weights for policy 0, policy_version 30922 (0.0007) [2023-10-07 20:57:33,261][67838] Updated weights for policy 0, policy_version 30932 (0.0008) [2023-10-07 20:57:33,639][67838] Updated weights for policy 0, policy_version 30942 (0.0009) [2023-10-07 20:57:34,257][67871] Updated weights for policy 1, policy_version 30980 (0.0010) [2023-10-07 20:57:34,629][67871] Updated weights for policy 1, policy_version 30990 (0.0008) [2023-10-07 20:57:34,998][67871] Updated weights for policy 1, policy_version 31000 (0.0007) [2023-10-07 20:57:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63438848. Throughput: 0: 1672.8, 1: 1666.9. Samples: 15872978. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-10-07 20:57:37,477][66916] Avg episode reward: [(0, '35.950'), (1, '39.710')] [2023-10-07 20:57:37,752][67838] Updated weights for policy 0, policy_version 30952 (0.0007) [2023-10-07 20:57:38,122][67838] Updated weights for policy 0, policy_version 30962 (0.0011) [2023-10-07 20:57:38,483][67838] Updated weights for policy 0, policy_version 30972 (0.0010) [2023-10-07 20:57:39,138][67871] Updated weights for policy 1, policy_version 31010 (0.0010) [2023-10-07 20:57:39,502][67871] Updated weights for policy 1, policy_version 31020 (0.0009) [2023-10-07 20:57:39,876][67871] Updated weights for policy 1, policy_version 31030 (0.0009) [2023-10-07 20:57:40,243][67871] Updated weights for policy 1, policy_version 31040 (0.0007) [2023-10-07 20:57:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63504384. Throughput: 0: 1673.4, 1: 1652.2. Samples: 15882518. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-10-07 20:57:42,477][66916] Avg episode reward: [(0, '36.650'), (1, '38.010')] [2023-10-07 20:57:42,630][67838] Updated weights for policy 0, policy_version 30982 (0.0008) [2023-10-07 20:57:43,005][67838] Updated weights for policy 0, policy_version 30992 (0.0010) [2023-10-07 20:57:43,379][67838] Updated weights for policy 0, policy_version 31002 (0.0007) [2023-10-07 20:57:44,306][67871] Updated weights for policy 1, policy_version 31050 (0.0009) [2023-10-07 20:57:44,679][67871] Updated weights for policy 1, policy_version 31060 (0.0008) [2023-10-07 20:57:45,052][67871] Updated weights for policy 1, policy_version 31070 (0.0008) [2023-10-07 20:57:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63569920. Throughput: 0: 1666.2, 1: 1664.6. Samples: 15902578. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-10-07 20:57:47,477][66916] Avg episode reward: [(0, '39.170'), (1, '38.740')] [2023-10-07 20:57:47,533][67838] Updated weights for policy 0, policy_version 31012 (0.0007) [2023-10-07 20:57:47,913][67838] Updated weights for policy 0, policy_version 31022 (0.0007) [2023-10-07 20:57:48,287][67838] Updated weights for policy 0, policy_version 31032 (0.0010) [2023-10-07 20:57:49,096][67871] Updated weights for policy 1, policy_version 31080 (0.0009) [2023-10-07 20:57:49,462][67871] Updated weights for policy 1, policy_version 31090 (0.0009) [2023-10-07 20:57:49,825][67871] Updated weights for policy 1, policy_version 31100 (0.0010) [2023-10-07 20:57:52,452][67838] Updated weights for policy 0, policy_version 31042 (0.0009) [2023-10-07 20:57:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63635456. Throughput: 0: 1658.1, 1: 1662.6. Samples: 15922786. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-10-07 20:57:52,477][66916] Avg episode reward: [(0, '37.750'), (1, '38.530')] [2023-10-07 20:57:52,821][67838] Updated weights for policy 0, policy_version 31052 (0.0009) [2023-10-07 20:57:53,199][67838] Updated weights for policy 0, policy_version 31062 (0.0009) [2023-10-07 20:57:53,568][67838] Updated weights for policy 0, policy_version 31072 (0.0009) [2023-10-07 20:57:54,070][67871] Updated weights for policy 1, policy_version 31110 (0.0010) [2023-10-07 20:57:54,443][67871] Updated weights for policy 1, policy_version 31120 (0.0009) [2023-10-07 20:57:54,813][67871] Updated weights for policy 1, policy_version 31130 (0.0007) [2023-10-07 20:57:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 63700992. Throughput: 0: 1657.0, 1: 1653.9. Samples: 15932032. Policy #0 lag: (min: 0.0, avg: 23.3, max: 32.0) [2023-10-07 20:57:57,477][66916] Avg episode reward: [(0, '38.180'), (1, '42.220')] [2023-10-07 20:57:57,667][67838] Updated weights for policy 0, policy_version 31082 (0.0008) [2023-10-07 20:57:58,046][67838] Updated weights for policy 0, policy_version 31092 (0.0009) [2023-10-07 20:57:58,424][67838] Updated weights for policy 0, policy_version 31102 (0.0007) [2023-10-07 20:57:58,775][67871] Updated weights for policy 1, policy_version 31140 (0.0008) [2023-10-07 20:57:59,151][67871] Updated weights for policy 1, policy_version 31150 (0.0009) [2023-10-07 20:57:59,523][67871] Updated weights for policy 1, policy_version 31160 (0.0010) [2023-10-07 20:58:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 63766528. Throughput: 0: 1657.1, 1: 1665.3. Samples: 15952328. Policy #0 lag: (min: 0.0, avg: 23.3, max: 32.0) [2023-10-07 20:58:02,478][66916] Avg episode reward: [(0, '39.620'), (1, '41.390')] [2023-10-07 20:58:02,502][67838] Updated weights for policy 0, policy_version 31112 (0.0010) [2023-10-07 20:58:02,879][67838] Updated weights for policy 0, policy_version 31122 (0.0011) [2023-10-07 20:58:03,261][67838] Updated weights for policy 0, policy_version 31132 (0.0007) [2023-10-07 20:58:03,559][67871] Updated weights for policy 1, policy_version 31170 (0.0008) [2023-10-07 20:58:03,971][67871] Updated weights for policy 1, policy_version 31180 (0.0010) [2023-10-07 20:58:04,343][67871] Updated weights for policy 1, policy_version 31190 (0.0009) [2023-10-07 20:58:04,718][67871] Updated weights for policy 1, policy_version 31200 (0.0010) [2023-10-07 20:58:07,368][67838] Updated weights for policy 0, policy_version 31142 (0.0009) [2023-10-07 20:58:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63832064. Throughput: 0: 1657.0, 1: 1670.3. Samples: 15972924. Policy #0 lag: (min: 0.0, avg: 23.3, max: 32.0) [2023-10-07 20:58:07,478][66916] Avg episode reward: [(0, '38.970'), (1, '42.010')] [2023-10-07 20:58:07,731][67838] Updated weights for policy 0, policy_version 31152 (0.0011) [2023-10-07 20:58:08,109][67838] Updated weights for policy 0, policy_version 31162 (0.0008) [2023-10-07 20:58:08,685][67871] Updated weights for policy 1, policy_version 31210 (0.0010) [2023-10-07 20:58:09,042][67871] Updated weights for policy 1, policy_version 31220 (0.0009) [2023-10-07 20:58:09,414][67871] Updated weights for policy 1, policy_version 31230 (0.0009) [2023-10-07 20:58:12,234][67838] Updated weights for policy 0, policy_version 31172 (0.0008) [2023-10-07 20:58:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 63897600. Throughput: 0: 1657.3, 1: 1656.1. Samples: 15982002. Policy #0 lag: (min: 0.0, avg: 23.3, max: 32.0) [2023-10-07 20:58:12,478][66916] Avg episode reward: [(0, '41.640'), (1, '41.670')] [2023-10-07 20:58:12,620][67838] Updated weights for policy 0, policy_version 31182 (0.0008) [2023-10-07 20:58:12,997][67838] Updated weights for policy 0, policy_version 31192 (0.0007) [2023-10-07 20:58:13,652][67871] Updated weights for policy 1, policy_version 31240 (0.0008) [2023-10-07 20:58:14,021][67871] Updated weights for policy 1, policy_version 31250 (0.0007) [2023-10-07 20:58:14,392][67871] Updated weights for policy 1, policy_version 31260 (0.0009) [2023-10-07 20:58:17,043][67838] Updated weights for policy 0, policy_version 31202 (0.0007) [2023-10-07 20:58:17,402][67838] Updated weights for policy 0, policy_version 31212 (0.0007) [2023-10-07 20:58:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 63963136. Throughput: 0: 1658.5, 1: 1672.5. Samples: 16002480. Policy #0 lag: (min: 0.0, avg: 23.3, max: 32.0) [2023-10-07 20:58:17,477][66916] Avg episode reward: [(0, '40.850'), (1, '42.260')] [2023-10-07 20:58:17,773][67838] Updated weights for policy 0, policy_version 31222 (0.0007) [2023-10-07 20:58:18,154][67838] Updated weights for policy 0, policy_version 31232 (0.0009) [2023-10-07 20:58:18,436][67871] Updated weights for policy 1, policy_version 31270 (0.0010) [2023-10-07 20:58:18,807][67871] Updated weights for policy 1, policy_version 31280 (0.0010) [2023-10-07 20:58:19,175][67871] Updated weights for policy 1, policy_version 31290 (0.0009) [2023-10-07 20:58:22,282][67838] Updated weights for policy 0, policy_version 31242 (0.0008) [2023-10-07 20:58:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 64028672. Throughput: 0: 1658.2, 1: 1671.0. Samples: 16022792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:58:22,477][66916] Avg episode reward: [(0, '42.440'), (1, '43.800')] [2023-10-07 20:58:22,485][67676] Saving new best policy, reward=43.800! [2023-10-07 20:58:22,653][67838] Updated weights for policy 0, policy_version 31252 (0.0008) [2023-10-07 20:58:23,040][67838] Updated weights for policy 0, policy_version 31262 (0.0008) [2023-10-07 20:58:23,297][67871] Updated weights for policy 1, policy_version 31300 (0.0007) [2023-10-07 20:58:23,676][67871] Updated weights for policy 1, policy_version 31310 (0.0007) [2023-10-07 20:58:24,051][67871] Updated weights for policy 1, policy_version 31320 (0.0009) [2023-10-07 20:58:27,170][67838] Updated weights for policy 0, policy_version 31272 (0.0007) [2023-10-07 20:58:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64094208. Throughput: 0: 1660.2, 1: 1661.8. Samples: 16032008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:58:27,478][66916] Avg episode reward: [(0, '38.860'), (1, '41.660')] [2023-10-07 20:58:27,541][67838] Updated weights for policy 0, policy_version 31282 (0.0008) [2023-10-07 20:58:27,923][67838] Updated weights for policy 0, policy_version 31292 (0.0007) [2023-10-07 20:58:28,136][67871] Updated weights for policy 1, policy_version 31330 (0.0008) [2023-10-07 20:58:28,499][67871] Updated weights for policy 1, policy_version 31340 (0.0009) [2023-10-07 20:58:28,867][67871] Updated weights for policy 1, policy_version 31350 (0.0011) [2023-10-07 20:58:29,226][67871] Updated weights for policy 1, policy_version 31360 (0.0010) [2023-10-07 20:58:32,090][67838] Updated weights for policy 0, policy_version 31302 (0.0009) [2023-10-07 20:58:32,469][67838] Updated weights for policy 0, policy_version 31312 (0.0007) [2023-10-07 20:58:32,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64159744. Throughput: 0: 1662.2, 1: 1669.4. Samples: 16052498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:58:32,477][66916] Avg episode reward: [(0, '38.350'), (1, '43.280')] [2023-10-07 20:58:32,842][67838] Updated weights for policy 0, policy_version 31322 (0.0007) [2023-10-07 20:58:33,420][67871] Updated weights for policy 1, policy_version 31370 (0.0009) [2023-10-07 20:58:33,786][67871] Updated weights for policy 1, policy_version 31380 (0.0009) [2023-10-07 20:58:34,149][67871] Updated weights for policy 1, policy_version 31390 (0.0011) [2023-10-07 20:58:37,033][67838] Updated weights for policy 0, policy_version 31332 (0.0008) [2023-10-07 20:58:37,410][67838] Updated weights for policy 0, policy_version 31342 (0.0010) [2023-10-07 20:58:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64225280. Throughput: 0: 1663.3, 1: 1670.8. Samples: 16072822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:58:37,477][66916] Avg episode reward: [(0, '36.380'), (1, '42.920')] [2023-10-07 20:58:37,779][67838] Updated weights for policy 0, policy_version 31352 (0.0008) [2023-10-07 20:58:38,237][67871] Updated weights for policy 1, policy_version 31400 (0.0008) [2023-10-07 20:58:38,604][67871] Updated weights for policy 1, policy_version 31410 (0.0009) [2023-10-07 20:58:38,971][67871] Updated weights for policy 1, policy_version 31420 (0.0008) [2023-10-07 20:58:41,741][67838] Updated weights for policy 0, policy_version 31362 (0.0008) [2023-10-07 20:58:42,109][67838] Updated weights for policy 0, policy_version 31372 (0.0008) [2023-10-07 20:58:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64290816. Throughput: 0: 1667.1, 1: 1670.3. Samples: 16082216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:58:42,477][66916] Avg episode reward: [(0, '35.580'), (1, '43.090')] [2023-10-07 20:58:42,489][67838] Updated weights for policy 0, policy_version 31382 (0.0007) [2023-10-07 20:58:42,852][67838] Updated weights for policy 0, policy_version 31392 (0.0010) [2023-10-07 20:58:43,086][67871] Updated weights for policy 1, policy_version 31430 (0.0009) [2023-10-07 20:58:43,459][67871] Updated weights for policy 1, policy_version 31440 (0.0008) [2023-10-07 20:58:43,826][67871] Updated weights for policy 1, policy_version 31450 (0.0009) [2023-10-07 20:58:46,951][67838] Updated weights for policy 0, policy_version 31402 (0.0007) [2023-10-07 20:58:47,328][67838] Updated weights for policy 0, policy_version 31412 (0.0008) [2023-10-07 20:58:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64356352. Throughput: 0: 1666.5, 1: 1673.9. Samples: 16102646. Policy #0 lag: (min: 18.0, avg: 18.3, max: 31.0) [2023-10-07 20:58:47,477][66916] Avg episode reward: [(0, '37.130'), (1, '41.960')] [2023-10-07 20:58:47,697][67838] Updated weights for policy 0, policy_version 31422 (0.0008) [2023-10-07 20:58:47,847][67871] Updated weights for policy 1, policy_version 31460 (0.0008) [2023-10-07 20:58:48,219][67871] Updated weights for policy 1, policy_version 31470 (0.0009) [2023-10-07 20:58:48,583][67871] Updated weights for policy 1, policy_version 31480 (0.0008) [2023-10-07 20:58:51,922][67838] Updated weights for policy 0, policy_version 31432 (0.0009) [2023-10-07 20:58:52,301][67838] Updated weights for policy 0, policy_version 31442 (0.0008) [2023-10-07 20:58:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 64421888. Throughput: 0: 1655.1, 1: 1674.9. Samples: 16122774. Policy #0 lag: (min: 18.0, avg: 18.3, max: 31.0) [2023-10-07 20:58:52,478][66916] Avg episode reward: [(0, '34.050'), (1, '45.140')] [2023-10-07 20:58:52,672][67838] Updated weights for policy 0, policy_version 31452 (0.0008) [2023-10-07 20:58:52,674][67871] Updated weights for policy 1, policy_version 31490 (0.0009) [2023-10-07 20:58:53,098][67871] Updated weights for policy 1, policy_version 31500 (0.0009) [2023-10-07 20:58:53,469][67871] Updated weights for policy 1, policy_version 31510 (0.0009) [2023-10-07 20:58:53,836][67676] Saving new best policy, reward=45.140! [2023-10-07 20:58:53,837][67871] Updated weights for policy 1, policy_version 31520 (0.0008) [2023-10-07 20:58:56,837][67838] Updated weights for policy 0, policy_version 31462 (0.0010) [2023-10-07 20:58:57,209][67838] Updated weights for policy 0, policy_version 31472 (0.0007) [2023-10-07 20:58:57,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64487424. Throughput: 0: 1663.4, 1: 1674.0. Samples: 16132186. Policy #0 lag: (min: 18.0, avg: 18.3, max: 31.0) [2023-10-07 20:58:57,478][66916] Avg episode reward: [(0, '38.270'), (1, '46.340')] [2023-10-07 20:58:57,479][67676] Saving new best policy, reward=46.340! [2023-10-07 20:58:57,584][67838] Updated weights for policy 0, policy_version 31482 (0.0011) [2023-10-07 20:58:58,028][67871] Updated weights for policy 1, policy_version 31530 (0.0009) [2023-10-07 20:58:58,397][67871] Updated weights for policy 1, policy_version 31540 (0.0009) [2023-10-07 20:58:58,769][67871] Updated weights for policy 1, policy_version 31550 (0.0008) [2023-10-07 20:59:01,803][67838] Updated weights for policy 0, policy_version 31492 (0.0008) [2023-10-07 20:59:02,177][67838] Updated weights for policy 0, policy_version 31502 (0.0010) [2023-10-07 20:59:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64552960. Throughput: 0: 1660.1, 1: 1676.7. Samples: 16152636. Policy #0 lag: (min: 18.0, avg: 18.3, max: 31.0) [2023-10-07 20:59:02,477][66916] Avg episode reward: [(0, '34.970'), (1, '43.410')] [2023-10-07 20:59:02,548][67838] Updated weights for policy 0, policy_version 31512 (0.0010) [2023-10-07 20:59:02,896][67871] Updated weights for policy 1, policy_version 31560 (0.0007) [2023-10-07 20:59:03,272][67871] Updated weights for policy 1, policy_version 31570 (0.0007) [2023-10-07 20:59:03,638][67871] Updated weights for policy 1, policy_version 31580 (0.0010) [2023-10-07 20:59:06,837][67838] Updated weights for policy 0, policy_version 31522 (0.0007) [2023-10-07 20:59:07,211][67838] Updated weights for policy 0, policy_version 31532 (0.0007) [2023-10-07 20:59:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64618496. Throughput: 0: 1653.5, 1: 1679.0. Samples: 16172754. Policy #0 lag: (min: 18.0, avg: 18.3, max: 31.0) [2023-10-07 20:59:07,477][66916] Avg episode reward: [(0, '38.910'), (1, '43.410')] [2023-10-07 20:59:07,586][67838] Updated weights for policy 0, policy_version 31542 (0.0008) [2023-10-07 20:59:07,682][67871] Updated weights for policy 1, policy_version 31590 (0.0009) [2023-10-07 20:59:07,956][67838] Updated weights for policy 0, policy_version 31552 (0.0008) [2023-10-07 20:59:07,957][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000031552_32309248.pth... [2023-10-07 20:59:07,990][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000029984_30703616.pth [2023-10-07 20:59:08,047][67871] Updated weights for policy 1, policy_version 31600 (0.0009) [2023-10-07 20:59:08,420][67871] Updated weights for policy 1, policy_version 31610 (0.0007) [2023-10-07 20:59:08,631][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000031616_32374784.pth... [2023-10-07 20:59:08,660][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000030048_30769152.pth [2023-10-07 20:59:12,023][67838] Updated weights for policy 0, policy_version 31562 (0.0011) [2023-10-07 20:59:12,390][67838] Updated weights for policy 0, policy_version 31572 (0.0009) [2023-10-07 20:59:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64684032. Throughput: 0: 1656.6, 1: 1678.1. Samples: 16182066. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 20:59:12,477][66916] Avg episode reward: [(0, '40.500'), (1, '40.560')] [2023-10-07 20:59:12,550][67871] Updated weights for policy 1, policy_version 31620 (0.0007) [2023-10-07 20:59:12,764][67838] Updated weights for policy 0, policy_version 31582 (0.0007) [2023-10-07 20:59:12,912][67871] Updated weights for policy 1, policy_version 31630 (0.0008) [2023-10-07 20:59:13,285][67871] Updated weights for policy 1, policy_version 31640 (0.0007) [2023-10-07 20:59:16,929][67838] Updated weights for policy 0, policy_version 31592 (0.0009) [2023-10-07 20:59:17,302][67838] Updated weights for policy 0, policy_version 31602 (0.0008) [2023-10-07 20:59:17,392][67871] Updated weights for policy 1, policy_version 31650 (0.0008) [2023-10-07 20:59:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64749568. Throughput: 0: 1652.4, 1: 1673.8. Samples: 16202174. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 20:59:17,477][66916] Avg episode reward: [(0, '39.070'), (1, '38.430')] [2023-10-07 20:59:17,680][67838] Updated weights for policy 0, policy_version 31612 (0.0007) [2023-10-07 20:59:17,749][67871] Updated weights for policy 1, policy_version 31660 (0.0009) [2023-10-07 20:59:18,116][67871] Updated weights for policy 1, policy_version 31670 (0.0008) [2023-10-07 20:59:18,484][67871] Updated weights for policy 1, policy_version 31680 (0.0007) [2023-10-07 20:59:21,907][67838] Updated weights for policy 0, policy_version 31622 (0.0008) [2023-10-07 20:59:22,284][67838] Updated weights for policy 0, policy_version 31632 (0.0009) [2023-10-07 20:59:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64815104. Throughput: 0: 1642.3, 1: 1677.1. Samples: 16222196. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 20:59:22,477][66916] Avg episode reward: [(0, '39.250'), (1, '39.380')] [2023-10-07 20:59:22,617][67871] Updated weights for policy 1, policy_version 31690 (0.0009) [2023-10-07 20:59:22,655][67838] Updated weights for policy 0, policy_version 31642 (0.0010) [2023-10-07 20:59:22,975][67871] Updated weights for policy 1, policy_version 31700 (0.0009) [2023-10-07 20:59:23,351][67871] Updated weights for policy 1, policy_version 31710 (0.0007) [2023-10-07 20:59:26,654][67838] Updated weights for policy 0, policy_version 31652 (0.0009) [2023-10-07 20:59:27,021][67838] Updated weights for policy 0, policy_version 31662 (0.0009) [2023-10-07 20:59:27,392][67838] Updated weights for policy 0, policy_version 31672 (0.0007) [2023-10-07 20:59:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64880640. Throughput: 0: 1649.6, 1: 1671.6. Samples: 16231670. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 20:59:27,477][66916] Avg episode reward: [(0, '36.830'), (1, '42.180')] [2023-10-07 20:59:27,513][67871] Updated weights for policy 1, policy_version 31720 (0.0010) [2023-10-07 20:59:27,879][67871] Updated weights for policy 1, policy_version 31730 (0.0009) [2023-10-07 20:59:28,238][67871] Updated weights for policy 1, policy_version 31740 (0.0011) [2023-10-07 20:59:31,477][67838] Updated weights for policy 0, policy_version 31682 (0.0008) [2023-10-07 20:59:31,847][67838] Updated weights for policy 0, policy_version 31692 (0.0010) [2023-10-07 20:59:32,208][67838] Updated weights for policy 0, policy_version 31702 (0.0011) [2023-10-07 20:59:32,444][67871] Updated weights for policy 1, policy_version 31750 (0.0009) [2023-10-07 20:59:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64946176. Throughput: 0: 1650.3, 1: 1665.8. Samples: 16251868. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 20:59:32,478][66916] Avg episode reward: [(0, '39.160'), (1, '42.330')] [2023-10-07 20:59:32,587][67838] Updated weights for policy 0, policy_version 31712 (0.0007) [2023-10-07 20:59:32,806][67871] Updated weights for policy 1, policy_version 31760 (0.0008) [2023-10-07 20:59:33,165][67871] Updated weights for policy 1, policy_version 31770 (0.0008) [2023-10-07 20:59:36,840][67838] Updated weights for policy 0, policy_version 31722 (0.0008) [2023-10-07 20:59:37,210][67838] Updated weights for policy 0, policy_version 31732 (0.0008) [2023-10-07 20:59:37,316][67871] Updated weights for policy 1, policy_version 31780 (0.0010) [2023-10-07 20:59:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65011712. Throughput: 0: 1649.6, 1: 1660.0. Samples: 16271704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:59:37,477][66916] Avg episode reward: [(0, '39.880'), (1, '43.100')] [2023-10-07 20:59:37,577][67838] Updated weights for policy 0, policy_version 31742 (0.0008) [2023-10-07 20:59:37,688][67871] Updated weights for policy 1, policy_version 31790 (0.0008) [2023-10-07 20:59:38,056][67871] Updated weights for policy 1, policy_version 31800 (0.0007) [2023-10-07 20:59:41,512][67838] Updated weights for policy 0, policy_version 31752 (0.0008) [2023-10-07 20:59:41,883][67838] Updated weights for policy 0, policy_version 31762 (0.0009) [2023-10-07 20:59:42,206][67871] Updated weights for policy 1, policy_version 31810 (0.0007) [2023-10-07 20:59:42,261][67838] Updated weights for policy 0, policy_version 31772 (0.0008) [2023-10-07 20:59:42,476][66916] Fps is (10 sec: 16384.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 65110016. Throughput: 0: 1654.9, 1: 1662.1. Samples: 16281450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:59:42,477][66916] Avg episode reward: [(0, '39.850'), (1, '42.550')] [2023-10-07 20:59:42,588][67871] Updated weights for policy 1, policy_version 31820 (0.0010) [2023-10-07 20:59:42,951][67871] Updated weights for policy 1, policy_version 31830 (0.0009) [2023-10-07 20:59:43,326][67871] Updated weights for policy 1, policy_version 31840 (0.0007) [2023-10-07 20:59:46,538][67838] Updated weights for policy 0, policy_version 31782 (0.0009) [2023-10-07 20:59:46,921][67838] Updated weights for policy 0, policy_version 31792 (0.0008) [2023-10-07 20:59:47,283][67838] Updated weights for policy 0, policy_version 31802 (0.0008) [2023-10-07 20:59:47,466][67871] Updated weights for policy 1, policy_version 31850 (0.0009) [2023-10-07 20:59:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65142784. Throughput: 0: 1654.8, 1: 1657.3. Samples: 16301682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:59:47,477][66916] Avg episode reward: [(0, '40.710'), (1, '43.740')] [2023-10-07 20:59:47,835][67871] Updated weights for policy 1, policy_version 31860 (0.0008) [2023-10-07 20:59:48,208][67871] Updated weights for policy 1, policy_version 31870 (0.0008) [2023-10-07 20:59:51,333][67838] Updated weights for policy 0, policy_version 31812 (0.0011) [2023-10-07 20:59:51,708][67838] Updated weights for policy 0, policy_version 31822 (0.0011) [2023-10-07 20:59:52,075][67838] Updated weights for policy 0, policy_version 31832 (0.0010) [2023-10-07 20:59:52,241][67871] Updated weights for policy 1, policy_version 31880 (0.0007) [2023-10-07 20:59:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 65241088. Throughput: 0: 1638.2, 1: 1658.9. Samples: 16321122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 20:59:52,477][66916] Avg episode reward: [(0, '41.620'), (1, '43.730')] [2023-10-07 20:59:52,613][67871] Updated weights for policy 1, policy_version 31890 (0.0007) [2023-10-07 20:59:52,981][67871] Updated weights for policy 1, policy_version 31900 (0.0007) [2023-10-07 20:59:56,209][67838] Updated weights for policy 0, policy_version 31842 (0.0008) [2023-10-07 20:59:56,589][67838] Updated weights for policy 0, policy_version 31852 (0.0007) [2023-10-07 20:59:56,957][67838] Updated weights for policy 0, policy_version 31862 (0.0009) [2023-10-07 20:59:57,118][67871] Updated weights for policy 1, policy_version 31910 (0.0008) [2023-10-07 20:59:57,332][67838] Updated weights for policy 0, policy_version 31872 (0.0007) [2023-10-07 20:59:57,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 65306624. Throughput: 0: 1652.4, 1: 1656.3. Samples: 16330956. Policy #0 lag: (min: 28.0, avg: 38.6, max: 60.0) [2023-10-07 20:59:57,478][66916] Avg episode reward: [(0, '39.360'), (1, '43.520')] [2023-10-07 20:59:57,484][67871] Updated weights for policy 1, policy_version 31920 (0.0010) [2023-10-07 20:59:57,858][67871] Updated weights for policy 1, policy_version 31930 (0.0010) [2023-10-07 21:00:01,333][67838] Updated weights for policy 0, policy_version 31882 (0.0010) [2023-10-07 21:00:01,705][67838] Updated weights for policy 0, policy_version 31892 (0.0010) [2023-10-07 21:00:01,954][67871] Updated weights for policy 1, policy_version 31940 (0.0010) [2023-10-07 21:00:02,069][67838] Updated weights for policy 0, policy_version 31902 (0.0008) [2023-10-07 21:00:02,321][67871] Updated weights for policy 1, policy_version 31950 (0.0008) [2023-10-07 21:00:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 65372160. Throughput: 0: 1654.1, 1: 1660.4. Samples: 16351326. Policy #0 lag: (min: 28.0, avg: 38.6, max: 60.0) [2023-10-07 21:00:02,477][66916] Avg episode reward: [(0, '41.410'), (1, '43.470')] [2023-10-07 21:00:02,684][67871] Updated weights for policy 1, policy_version 31960 (0.0007) [2023-10-07 21:00:06,341][67838] Updated weights for policy 0, policy_version 31912 (0.0008) [2023-10-07 21:00:06,640][67871] Updated weights for policy 1, policy_version 31970 (0.0008) [2023-10-07 21:00:06,725][67838] Updated weights for policy 0, policy_version 31922 (0.0009) [2023-10-07 21:00:07,008][67871] Updated weights for policy 1, policy_version 31980 (0.0009) [2023-10-07 21:00:07,082][67838] Updated weights for policy 0, policy_version 31932 (0.0009) [2023-10-07 21:00:07,372][67871] Updated weights for policy 1, policy_version 31990 (0.0008) [2023-10-07 21:00:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 65437696. Throughput: 0: 1646.1, 1: 1656.8. Samples: 16370826. Policy #0 lag: (min: 28.0, avg: 38.6, max: 60.0) [2023-10-07 21:00:07,478][66916] Avg episode reward: [(0, '39.090'), (1, '43.940')] [2023-10-07 21:00:07,745][67871] Updated weights for policy 1, policy_version 32000 (0.0010) [2023-10-07 21:00:11,161][67838] Updated weights for policy 0, policy_version 31942 (0.0009) [2023-10-07 21:00:11,535][67838] Updated weights for policy 0, policy_version 31952 (0.0009) [2023-10-07 21:00:11,901][67838] Updated weights for policy 0, policy_version 31962 (0.0008) [2023-10-07 21:00:11,904][67871] Updated weights for policy 1, policy_version 32010 (0.0009) [2023-10-07 21:00:12,271][67871] Updated weights for policy 1, policy_version 32020 (0.0009) [2023-10-07 21:00:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 65503232. Throughput: 0: 1655.6, 1: 1661.7. Samples: 16380946. Policy #0 lag: (min: 28.0, avg: 38.6, max: 60.0) [2023-10-07 21:00:12,477][66916] Avg episode reward: [(0, '38.200'), (1, '41.090')] [2023-10-07 21:00:12,642][67871] Updated weights for policy 1, policy_version 32030 (0.0009) [2023-10-07 21:00:16,178][67838] Updated weights for policy 0, policy_version 31972 (0.0009) [2023-10-07 21:00:16,543][67838] Updated weights for policy 0, policy_version 31982 (0.0007) [2023-10-07 21:00:16,668][67871] Updated weights for policy 1, policy_version 32040 (0.0008) [2023-10-07 21:00:16,915][67838] Updated weights for policy 0, policy_version 31992 (0.0007) [2023-10-07 21:00:17,040][67871] Updated weights for policy 1, policy_version 32050 (0.0008) [2023-10-07 21:00:17,399][67871] Updated weights for policy 1, policy_version 32060 (0.0007) [2023-10-07 21:00:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 65568768. Throughput: 0: 1651.9, 1: 1667.3. Samples: 16401232. Policy #0 lag: (min: 28.0, avg: 38.6, max: 60.0) [2023-10-07 21:00:17,478][66916] Avg episode reward: [(0, '37.180'), (1, '39.640')] [2023-10-07 21:00:20,971][67838] Updated weights for policy 0, policy_version 32002 (0.0007) [2023-10-07 21:00:21,340][67838] Updated weights for policy 0, policy_version 32012 (0.0010) [2023-10-07 21:00:21,538][67871] Updated weights for policy 1, policy_version 32070 (0.0010) [2023-10-07 21:00:21,709][67838] Updated weights for policy 0, policy_version 32022 (0.0009) [2023-10-07 21:00:21,905][67871] Updated weights for policy 1, policy_version 32080 (0.0008) [2023-10-07 21:00:22,067][67838] Updated weights for policy 0, policy_version 32032 (0.0009) [2023-10-07 21:00:22,276][67871] Updated weights for policy 1, policy_version 32090 (0.0009) [2023-10-07 21:00:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 65634304. Throughput: 0: 1638.0, 1: 1657.6. Samples: 16420010. Policy #0 lag: (min: 25.0, avg: 52.2, max: 56.0) [2023-10-07 21:00:22,477][66916] Avg episode reward: [(0, '36.920'), (1, '39.740')] [2023-10-07 21:00:26,134][67838] Updated weights for policy 0, policy_version 32042 (0.0008) [2023-10-07 21:00:26,350][67871] Updated weights for policy 1, policy_version 32100 (0.0008) [2023-10-07 21:00:26,511][67838] Updated weights for policy 0, policy_version 32052 (0.0008) [2023-10-07 21:00:26,713][67871] Updated weights for policy 1, policy_version 32110 (0.0007) [2023-10-07 21:00:26,873][67838] Updated weights for policy 0, policy_version 32062 (0.0010) [2023-10-07 21:00:27,077][67871] Updated weights for policy 1, policy_version 32120 (0.0008) [2023-10-07 21:00:27,476][66916] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 65732608. Throughput: 0: 1651.2, 1: 1666.3. Samples: 16430736. Policy #0 lag: (min: 25.0, avg: 52.2, max: 56.0) [2023-10-07 21:00:27,477][66916] Avg episode reward: [(0, '37.910'), (1, '43.110')] [2023-10-07 21:00:30,969][67838] Updated weights for policy 0, policy_version 32072 (0.0008) [2023-10-07 21:00:31,343][67838] Updated weights for policy 0, policy_version 32082 (0.0007) [2023-10-07 21:00:31,352][67871] Updated weights for policy 1, policy_version 32130 (0.0009) [2023-10-07 21:00:31,709][67838] Updated weights for policy 0, policy_version 32092 (0.0007) [2023-10-07 21:00:31,763][67871] Updated weights for policy 1, policy_version 32140 (0.0009) [2023-10-07 21:00:32,135][67871] Updated weights for policy 1, policy_version 32150 (0.0007) [2023-10-07 21:00:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 65765376. Throughput: 0: 1642.2, 1: 1671.8. Samples: 16450812. Policy #0 lag: (min: 25.0, avg: 52.2, max: 56.0) [2023-10-07 21:00:32,477][66916] Avg episode reward: [(0, '38.060'), (1, '44.450')] [2023-10-07 21:00:32,508][67871] Updated weights for policy 1, policy_version 32160 (0.0008) [2023-10-07 21:00:36,129][67838] Updated weights for policy 0, policy_version 32102 (0.0009) [2023-10-07 21:00:36,502][67838] Updated weights for policy 0, policy_version 32112 (0.0007) [2023-10-07 21:00:36,877][67838] Updated weights for policy 0, policy_version 32122 (0.0007) [2023-10-07 21:00:36,878][67871] Updated weights for policy 1, policy_version 32170 (0.0008) [2023-10-07 21:00:37,245][67871] Updated weights for policy 1, policy_version 32180 (0.0009) [2023-10-07 21:00:37,476][66916] Fps is (10 sec: 9830.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 65830912. Throughput: 0: 1642.4, 1: 1652.0. Samples: 16469370. Policy #0 lag: (min: 25.0, avg: 52.2, max: 56.0) [2023-10-07 21:00:37,477][66916] Avg episode reward: [(0, '41.300'), (1, '44.230')] [2023-10-07 21:00:37,611][67871] Updated weights for policy 1, policy_version 32190 (0.0009) [2023-10-07 21:00:40,980][67838] Updated weights for policy 0, policy_version 32132 (0.0008) [2023-10-07 21:00:41,357][67838] Updated weights for policy 0, policy_version 32142 (0.0008) [2023-10-07 21:00:41,620][67871] Updated weights for policy 1, policy_version 32200 (0.0007) [2023-10-07 21:00:41,723][67838] Updated weights for policy 0, policy_version 32152 (0.0009) [2023-10-07 21:00:41,994][67871] Updated weights for policy 1, policy_version 32210 (0.0008) [2023-10-07 21:00:42,366][67871] Updated weights for policy 1, policy_version 32220 (0.0009) [2023-10-07 21:00:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65896448. Throughput: 0: 1652.8, 1: 1664.1. Samples: 16480216. Policy #0 lag: (min: 25.0, avg: 52.2, max: 56.0) [2023-10-07 21:00:42,477][66916] Avg episode reward: [(0, '39.610'), (1, '42.410')] [2023-10-07 21:00:45,896][67838] Updated weights for policy 0, policy_version 32162 (0.0010) [2023-10-07 21:00:46,274][67838] Updated weights for policy 0, policy_version 32172 (0.0010) [2023-10-07 21:00:46,451][67871] Updated weights for policy 1, policy_version 32230 (0.0007) [2023-10-07 21:00:46,646][67838] Updated weights for policy 0, policy_version 32182 (0.0009) [2023-10-07 21:00:46,819][67871] Updated weights for policy 1, policy_version 32240 (0.0008) [2023-10-07 21:00:47,026][67838] Updated weights for policy 0, policy_version 32192 (0.0008) [2023-10-07 21:00:47,189][67871] Updated weights for policy 1, policy_version 32250 (0.0008) [2023-10-07 21:00:47,476][66916] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 65994752. Throughput: 0: 1645.5, 1: 1660.1. Samples: 16500078. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-07 21:00:47,477][66916] Avg episode reward: [(0, '38.590'), (1, '42.310')] [2023-10-07 21:00:51,332][67838] Updated weights for policy 0, policy_version 32202 (0.0007) [2023-10-07 21:00:51,422][67871] Updated weights for policy 1, policy_version 32260 (0.0009) [2023-10-07 21:00:51,711][67838] Updated weights for policy 0, policy_version 32212 (0.0008) [2023-10-07 21:00:51,791][67871] Updated weights for policy 1, policy_version 32270 (0.0007) [2023-10-07 21:00:52,086][67838] Updated weights for policy 0, policy_version 32222 (0.0009) [2023-10-07 21:00:52,155][67871] Updated weights for policy 1, policy_version 32280 (0.0007) [2023-10-07 21:00:52,476][66916] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 66060288. Throughput: 0: 1643.0, 1: 1646.6. Samples: 16518858. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-07 21:00:52,477][66916] Avg episode reward: [(0, '38.280'), (1, '39.090')] [2023-10-07 21:00:56,240][67838] Updated weights for policy 0, policy_version 32232 (0.0008) [2023-10-07 21:00:56,319][67871] Updated weights for policy 1, policy_version 32290 (0.0008) [2023-10-07 21:00:56,629][67838] Updated weights for policy 0, policy_version 32242 (0.0008) [2023-10-07 21:00:56,700][67871] Updated weights for policy 1, policy_version 32300 (0.0009) [2023-10-07 21:00:56,995][67838] Updated weights for policy 0, policy_version 32252 (0.0008) [2023-10-07 21:00:57,064][67871] Updated weights for policy 1, policy_version 32310 (0.0008) [2023-10-07 21:00:57,432][67871] Updated weights for policy 1, policy_version 32320 (0.0008) [2023-10-07 21:00:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 66125824. Throughput: 0: 1648.1, 1: 1656.8. Samples: 16529664. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-07 21:00:57,477][66916] Avg episode reward: [(0, '35.840'), (1, '35.780')] [2023-10-07 21:01:01,077][67838] Updated weights for policy 0, policy_version 32262 (0.0009) [2023-10-07 21:01:01,435][67838] Updated weights for policy 0, policy_version 32272 (0.0007) [2023-10-07 21:01:01,603][67871] Updated weights for policy 1, policy_version 32330 (0.0007) [2023-10-07 21:01:01,824][67838] Updated weights for policy 0, policy_version 32282 (0.0007) [2023-10-07 21:01:01,980][67871] Updated weights for policy 1, policy_version 32340 (0.0008) [2023-10-07 21:01:02,340][67871] Updated weights for policy 1, policy_version 32350 (0.0008) [2023-10-07 21:01:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 66191360. Throughput: 0: 1647.9, 1: 1652.8. Samples: 16549760. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-07 21:01:02,477][66916] Avg episode reward: [(0, '35.480'), (1, '40.750')] [2023-10-07 21:01:05,948][67838] Updated weights for policy 0, policy_version 32292 (0.0010) [2023-10-07 21:01:06,315][67838] Updated weights for policy 0, policy_version 32302 (0.0010) [2023-10-07 21:01:06,364][67871] Updated weights for policy 1, policy_version 32360 (0.0008) [2023-10-07 21:01:06,684][67838] Updated weights for policy 0, policy_version 32312 (0.0007) [2023-10-07 21:01:06,730][67871] Updated weights for policy 1, policy_version 32370 (0.0007) [2023-10-07 21:01:07,092][67871] Updated weights for policy 1, policy_version 32380 (0.0007) [2023-10-07 21:01:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 66256896. Throughput: 0: 1653.1, 1: 1645.0. Samples: 16568426. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-07 21:01:07,477][66916] Avg episode reward: [(0, '35.220'), (1, '41.900')] [2023-10-07 21:01:07,490][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000032320_33095680.pth... [2023-10-07 21:01:07,490][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000032384_33161216.pth... [2023-10-07 21:01:07,523][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000030816_31555584.pth [2023-10-07 21:01:07,527][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000030752_31490048.pth [2023-10-07 21:01:10,656][67838] Updated weights for policy 0, policy_version 32322 (0.0011) [2023-10-07 21:01:11,032][67838] Updated weights for policy 0, policy_version 32332 (0.0009) [2023-10-07 21:01:11,174][67871] Updated weights for policy 1, policy_version 32390 (0.0007) [2023-10-07 21:01:11,405][67838] Updated weights for policy 0, policy_version 32342 (0.0009) [2023-10-07 21:01:11,539][67871] Updated weights for policy 1, policy_version 32400 (0.0007) [2023-10-07 21:01:11,774][67838] Updated weights for policy 0, policy_version 32352 (0.0007) [2023-10-07 21:01:11,910][67871] Updated weights for policy 1, policy_version 32410 (0.0008) [2023-10-07 21:01:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 66322432. Throughput: 0: 1652.6, 1: 1655.5. Samples: 16579598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:01:12,477][66916] Avg episode reward: [(0, '37.790'), (1, '45.350')] [2023-10-07 21:01:15,966][67838] Updated weights for policy 0, policy_version 32362 (0.0007) [2023-10-07 21:01:16,137][67871] Updated weights for policy 1, policy_version 32420 (0.0008) [2023-10-07 21:01:16,337][67838] Updated weights for policy 0, policy_version 32372 (0.0007) [2023-10-07 21:01:16,527][67871] Updated weights for policy 1, policy_version 32430 (0.0007) [2023-10-07 21:01:16,710][67838] Updated weights for policy 0, policy_version 32382 (0.0007) [2023-10-07 21:01:16,896][67871] Updated weights for policy 1, policy_version 32440 (0.0008) [2023-10-07 21:01:17,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 66387968. Throughput: 0: 1652.8, 1: 1651.9. Samples: 16599522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:01:17,478][66916] Avg episode reward: [(0, '37.780'), (1, '47.080')] [2023-10-07 21:01:17,479][67676] Saving new best policy, reward=47.080! [2023-10-07 21:01:20,415][67838] Updated weights for policy 0, policy_version 32392 (0.0009) [2023-10-07 21:01:20,789][67838] Updated weights for policy 0, policy_version 32402 (0.0010) [2023-10-07 21:01:21,102][67871] Updated weights for policy 1, policy_version 32450 (0.0007) [2023-10-07 21:01:21,164][67838] Updated weights for policy 0, policy_version 32412 (0.0008) [2023-10-07 21:01:21,470][67871] Updated weights for policy 1, policy_version 32460 (0.0007) [2023-10-07 21:01:21,834][67871] Updated weights for policy 1, policy_version 32470 (0.0008) [2023-10-07 21:01:22,201][67871] Updated weights for policy 1, policy_version 32480 (0.0007) [2023-10-07 21:01:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 66453504. Throughput: 0: 1661.1, 1: 1649.3. Samples: 16618340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:01:22,477][66916] Avg episode reward: [(0, '32.820'), (1, '50.010')] [2023-10-07 21:01:22,487][67676] Saving new best policy, reward=50.010! [2023-10-07 21:01:25,378][67838] Updated weights for policy 0, policy_version 32422 (0.0010) [2023-10-07 21:01:25,759][67838] Updated weights for policy 0, policy_version 32432 (0.0009) [2023-10-07 21:01:26,139][67838] Updated weights for policy 0, policy_version 32442 (0.0008) [2023-10-07 21:01:26,234][67871] Updated weights for policy 1, policy_version 32490 (0.0009) [2023-10-07 21:01:26,595][67871] Updated weights for policy 1, policy_version 32500 (0.0008) [2023-10-07 21:01:26,962][67871] Updated weights for policy 1, policy_version 32510 (0.0008) [2023-10-07 21:01:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66519040. Throughput: 0: 1659.4, 1: 1656.6. Samples: 16629438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:01:27,477][66916] Avg episode reward: [(0, '36.370'), (1, '48.550')] [2023-10-07 21:01:30,221][67838] Updated weights for policy 0, policy_version 32452 (0.0009) [2023-10-07 21:01:30,587][67838] Updated weights for policy 0, policy_version 32462 (0.0009) [2023-10-07 21:01:30,959][67838] Updated weights for policy 0, policy_version 32472 (0.0009) [2023-10-07 21:01:31,131][67871] Updated weights for policy 1, policy_version 32520 (0.0009) [2023-10-07 21:01:31,500][67871] Updated weights for policy 1, policy_version 32530 (0.0010) [2023-10-07 21:01:31,868][67871] Updated weights for policy 1, policy_version 32540 (0.0010) [2023-10-07 21:01:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 66584576. Throughput: 0: 1646.8, 1: 1657.7. Samples: 16648784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:01:32,477][66916] Avg episode reward: [(0, '36.490'), (1, '48.910')] [2023-10-07 21:01:35,202][67838] Updated weights for policy 0, policy_version 32482 (0.0009) [2023-10-07 21:01:35,579][67838] Updated weights for policy 0, policy_version 32492 (0.0009) [2023-10-07 21:01:35,846][67871] Updated weights for policy 1, policy_version 32550 (0.0008) [2023-10-07 21:01:35,949][67838] Updated weights for policy 0, policy_version 32502 (0.0009) [2023-10-07 21:01:36,210][67871] Updated weights for policy 1, policy_version 32560 (0.0010) [2023-10-07 21:01:36,335][67838] Updated weights for policy 0, policy_version 32512 (0.0008) [2023-10-07 21:01:36,584][67871] Updated weights for policy 1, policy_version 32570 (0.0008) [2023-10-07 21:01:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 66650112. Throughput: 0: 1663.9, 1: 1648.8. Samples: 16667930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:01:37,477][66916] Avg episode reward: [(0, '37.760'), (1, '48.240')] [2023-10-07 21:01:40,541][67838] Updated weights for policy 0, policy_version 32522 (0.0007) [2023-10-07 21:01:40,822][67871] Updated weights for policy 1, policy_version 32580 (0.0009) [2023-10-07 21:01:40,919][67838] Updated weights for policy 0, policy_version 32532 (0.0008) [2023-10-07 21:01:41,194][67871] Updated weights for policy 1, policy_version 32590 (0.0009) [2023-10-07 21:01:41,289][67838] Updated weights for policy 0, policy_version 32542 (0.0009) [2023-10-07 21:01:41,563][67871] Updated weights for policy 1, policy_version 32600 (0.0007) [2023-10-07 21:01:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 66715648. Throughput: 0: 1666.0, 1: 1660.0. Samples: 16679336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:01:42,478][66916] Avg episode reward: [(0, '40.230'), (1, '45.880')] [2023-10-07 21:01:45,468][67838] Updated weights for policy 0, policy_version 32552 (0.0009) [2023-10-07 21:01:45,576][67871] Updated weights for policy 1, policy_version 32610 (0.0010) [2023-10-07 21:01:45,839][67838] Updated weights for policy 0, policy_version 32562 (0.0008) [2023-10-07 21:01:45,939][67871] Updated weights for policy 1, policy_version 32620 (0.0008) [2023-10-07 21:01:46,208][67838] Updated weights for policy 0, policy_version 32572 (0.0009) [2023-10-07 21:01:46,315][67871] Updated weights for policy 1, policy_version 32630 (0.0009) [2023-10-07 21:01:46,678][67871] Updated weights for policy 1, policy_version 32640 (0.0009) [2023-10-07 21:01:47,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66781184. Throughput: 0: 1648.4, 1: 1653.4. Samples: 16698344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:01:47,478][66916] Avg episode reward: [(0, '37.140'), (1, '44.000')] [2023-10-07 21:01:50,530][67838] Updated weights for policy 0, policy_version 32582 (0.0009) [2023-10-07 21:01:50,769][67871] Updated weights for policy 1, policy_version 32650 (0.0008) [2023-10-07 21:01:50,905][67838] Updated weights for policy 0, policy_version 32592 (0.0008) [2023-10-07 21:01:51,128][67871] Updated weights for policy 1, policy_version 32660 (0.0008) [2023-10-07 21:01:51,275][67838] Updated weights for policy 0, policy_version 32602 (0.0009) [2023-10-07 21:01:51,501][67871] Updated weights for policy 1, policy_version 32670 (0.0007) [2023-10-07 21:01:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66846720. Throughput: 0: 1655.1, 1: 1653.8. Samples: 16717326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:01:52,477][66916] Avg episode reward: [(0, '39.370'), (1, '44.460')] [2023-10-07 21:01:55,278][67838] Updated weights for policy 0, policy_version 32612 (0.0009) [2023-10-07 21:01:55,646][67838] Updated weights for policy 0, policy_version 32622 (0.0007) [2023-10-07 21:01:55,692][67871] Updated weights for policy 1, policy_version 32680 (0.0008) [2023-10-07 21:01:56,021][67838] Updated weights for policy 0, policy_version 32632 (0.0008) [2023-10-07 21:01:56,051][67871] Updated weights for policy 1, policy_version 32690 (0.0007) [2023-10-07 21:01:56,419][67871] Updated weights for policy 1, policy_version 32700 (0.0007) [2023-10-07 21:01:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66912256. Throughput: 0: 1659.9, 1: 1659.3. Samples: 16728962. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:01:57,477][66916] Avg episode reward: [(0, '39.410'), (1, '44.690')] [2023-10-07 21:01:59,916][67838] Updated weights for policy 0, policy_version 32642 (0.0008) [2023-10-07 21:02:00,299][67838] Updated weights for policy 0, policy_version 32652 (0.0009) [2023-10-07 21:02:00,502][67871] Updated weights for policy 1, policy_version 32710 (0.0008) [2023-10-07 21:02:00,679][67838] Updated weights for policy 0, policy_version 32662 (0.0008) [2023-10-07 21:02:00,871][67871] Updated weights for policy 1, policy_version 32720 (0.0007) [2023-10-07 21:02:01,050][67838] Updated weights for policy 0, policy_version 32672 (0.0009) [2023-10-07 21:02:01,231][67871] Updated weights for policy 1, policy_version 32730 (0.0008) [2023-10-07 21:02:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66977792. Throughput: 0: 1650.1, 1: 1651.1. Samples: 16748076. Policy #0 lag: (min: 6.0, avg: 6.8, max: 25.0) [2023-10-07 21:02:02,477][66916] Avg episode reward: [(0, '37.530'), (1, '45.700')] [2023-10-07 21:02:05,259][67838] Updated weights for policy 0, policy_version 32682 (0.0007) [2023-10-07 21:02:05,376][67871] Updated weights for policy 1, policy_version 32740 (0.0008) [2023-10-07 21:02:05,626][67838] Updated weights for policy 0, policy_version 32692 (0.0008) [2023-10-07 21:02:05,767][67871] Updated weights for policy 1, policy_version 32750 (0.0009) [2023-10-07 21:02:05,991][67838] Updated weights for policy 0, policy_version 32702 (0.0008) [2023-10-07 21:02:06,133][67871] Updated weights for policy 1, policy_version 32760 (0.0007) [2023-10-07 21:02:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67043328. Throughput: 0: 1669.1, 1: 1655.4. Samples: 16767942. Policy #0 lag: (min: 6.0, avg: 6.8, max: 25.0) [2023-10-07 21:02:07,478][66916] Avg episode reward: [(0, '39.340'), (1, '41.660')] [2023-10-07 21:02:10,075][67838] Updated weights for policy 0, policy_version 32712 (0.0009) [2023-10-07 21:02:10,367][67871] Updated weights for policy 1, policy_version 32770 (0.0009) [2023-10-07 21:02:10,442][67838] Updated weights for policy 0, policy_version 32722 (0.0010) [2023-10-07 21:02:10,725][67871] Updated weights for policy 1, policy_version 32780 (0.0008) [2023-10-07 21:02:10,818][67838] Updated weights for policy 0, policy_version 32732 (0.0008) [2023-10-07 21:02:11,094][67871] Updated weights for policy 1, policy_version 32790 (0.0008) [2023-10-07 21:02:11,468][67871] Updated weights for policy 1, policy_version 32800 (0.0008) [2023-10-07 21:02:12,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 67108864. Throughput: 0: 1666.2, 1: 1663.2. Samples: 16779260. Policy #0 lag: (min: 6.0, avg: 6.8, max: 25.0) [2023-10-07 21:02:12,478][66916] Avg episode reward: [(0, '37.160'), (1, '44.760')] [2023-10-07 21:02:15,066][67838] Updated weights for policy 0, policy_version 32742 (0.0007) [2023-10-07 21:02:15,412][67871] Updated weights for policy 1, policy_version 32810 (0.0009) [2023-10-07 21:02:15,440][67838] Updated weights for policy 0, policy_version 32752 (0.0007) [2023-10-07 21:02:15,770][67871] Updated weights for policy 1, policy_version 32820 (0.0009) [2023-10-07 21:02:15,807][67838] Updated weights for policy 0, policy_version 32762 (0.0007) [2023-10-07 21:02:16,135][67871] Updated weights for policy 1, policy_version 32830 (0.0009) [2023-10-07 21:02:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67174400. Throughput: 0: 1661.7, 1: 1651.2. Samples: 16797864. Policy #0 lag: (min: 6.0, avg: 6.8, max: 25.0) [2023-10-07 21:02:17,478][66916] Avg episode reward: [(0, '40.000'), (1, '46.810')] [2023-10-07 21:02:19,845][67838] Updated weights for policy 0, policy_version 32772 (0.0007) [2023-10-07 21:02:20,215][67838] Updated weights for policy 0, policy_version 32782 (0.0008) [2023-10-07 21:02:20,239][67871] Updated weights for policy 1, policy_version 32840 (0.0008) [2023-10-07 21:02:20,589][67838] Updated weights for policy 0, policy_version 32792 (0.0009) [2023-10-07 21:02:20,611][67871] Updated weights for policy 1, policy_version 32850 (0.0007) [2023-10-07 21:02:20,982][67871] Updated weights for policy 1, policy_version 32860 (0.0009) [2023-10-07 21:02:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 67239936. Throughput: 0: 1667.2, 1: 1667.1. Samples: 16817976. Policy #0 lag: (min: 6.0, avg: 6.8, max: 25.0) [2023-10-07 21:02:22,478][66916] Avg episode reward: [(0, '37.660'), (1, '48.890')] [2023-10-07 21:02:24,599][67838] Updated weights for policy 0, policy_version 32802 (0.0009) [2023-10-07 21:02:24,967][67838] Updated weights for policy 0, policy_version 32812 (0.0007) [2023-10-07 21:02:25,253][67871] Updated weights for policy 1, policy_version 32870 (0.0009) [2023-10-07 21:02:25,338][67838] Updated weights for policy 0, policy_version 32822 (0.0007) [2023-10-07 21:02:25,611][67871] Updated weights for policy 1, policy_version 32880 (0.0008) [2023-10-07 21:02:25,710][67838] Updated weights for policy 0, policy_version 32832 (0.0007) [2023-10-07 21:02:25,974][67871] Updated weights for policy 1, policy_version 32890 (0.0009) [2023-10-07 21:02:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 67305472. Throughput: 0: 1655.6, 1: 1667.2. Samples: 16828864. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 21:02:27,478][66916] Avg episode reward: [(0, '36.590'), (1, '47.860')] [2023-10-07 21:02:29,746][67838] Updated weights for policy 0, policy_version 32842 (0.0010) [2023-10-07 21:02:30,024][67871] Updated weights for policy 1, policy_version 32900 (0.0008) [2023-10-07 21:02:30,124][67838] Updated weights for policy 0, policy_version 32852 (0.0007) [2023-10-07 21:02:30,383][67871] Updated weights for policy 1, policy_version 32910 (0.0009) [2023-10-07 21:02:30,502][67838] Updated weights for policy 0, policy_version 32862 (0.0007) [2023-10-07 21:02:30,746][67871] Updated weights for policy 1, policy_version 32920 (0.0007) [2023-10-07 21:02:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67371008. Throughput: 0: 1658.7, 1: 1653.6. Samples: 16847394. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 21:02:32,477][66916] Avg episode reward: [(0, '41.090'), (1, '50.970')] [2023-10-07 21:02:32,478][67676] Saving new best policy, reward=50.970! [2023-10-07 21:02:34,741][67838] Updated weights for policy 0, policy_version 32872 (0.0009) [2023-10-07 21:02:34,840][67871] Updated weights for policy 1, policy_version 32930 (0.0010) [2023-10-07 21:02:35,110][67838] Updated weights for policy 0, policy_version 32882 (0.0008) [2023-10-07 21:02:35,221][67871] Updated weights for policy 1, policy_version 32940 (0.0008) [2023-10-07 21:02:35,481][67838] Updated weights for policy 0, policy_version 32892 (0.0007) [2023-10-07 21:02:35,588][67871] Updated weights for policy 1, policy_version 32950 (0.0008) [2023-10-07 21:02:35,951][67871] Updated weights for policy 1, policy_version 32960 (0.0008) [2023-10-07 21:02:37,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67436544. Throughput: 0: 1675.4, 1: 1667.7. Samples: 16867766. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 21:02:37,477][66916] Avg episode reward: [(0, '36.750'), (1, '48.930')] [2023-10-07 21:02:39,378][67838] Updated weights for policy 0, policy_version 32902 (0.0009) [2023-10-07 21:02:39,749][67838] Updated weights for policy 0, policy_version 32912 (0.0010) [2023-10-07 21:02:39,977][67871] Updated weights for policy 1, policy_version 32970 (0.0009) [2023-10-07 21:02:40,124][67838] Updated weights for policy 0, policy_version 32922 (0.0008) [2023-10-07 21:02:40,335][67871] Updated weights for policy 1, policy_version 32980 (0.0009) [2023-10-07 21:02:40,701][67871] Updated weights for policy 1, policy_version 32990 (0.0010) [2023-10-07 21:02:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67502080. Throughput: 0: 1648.6, 1: 1663.7. Samples: 16878014. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 21:02:42,477][66916] Avg episode reward: [(0, '40.860'), (1, '45.290')] [2023-10-07 21:02:44,297][67838] Updated weights for policy 0, policy_version 32932 (0.0008) [2023-10-07 21:02:44,664][67838] Updated weights for policy 0, policy_version 32942 (0.0010) [2023-10-07 21:02:44,878][67871] Updated weights for policy 1, policy_version 33000 (0.0007) [2023-10-07 21:02:45,043][67838] Updated weights for policy 0, policy_version 32952 (0.0008) [2023-10-07 21:02:45,247][67871] Updated weights for policy 1, policy_version 33010 (0.0009) [2023-10-07 21:02:45,611][67871] Updated weights for policy 1, policy_version 33020 (0.0009) [2023-10-07 21:02:47,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 67567616. Throughput: 0: 1662.9, 1: 1649.0. Samples: 16897110. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 21:02:47,478][66916] Avg episode reward: [(0, '37.830'), (1, '44.060')] [2023-10-07 21:02:49,266][67838] Updated weights for policy 0, policy_version 32962 (0.0008) [2023-10-07 21:02:49,647][67838] Updated weights for policy 0, policy_version 32972 (0.0008) [2023-10-07 21:02:49,888][67871] Updated weights for policy 1, policy_version 33030 (0.0009) [2023-10-07 21:02:50,017][67838] Updated weights for policy 0, policy_version 32982 (0.0008) [2023-10-07 21:02:50,261][67871] Updated weights for policy 1, policy_version 33040 (0.0010) [2023-10-07 21:02:50,388][67838] Updated weights for policy 0, policy_version 32992 (0.0008) [2023-10-07 21:02:50,623][67871] Updated weights for policy 1, policy_version 33050 (0.0009) [2023-10-07 21:02:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67633152. Throughput: 0: 1655.3, 1: 1661.8. Samples: 16917208. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:02:52,477][66916] Avg episode reward: [(0, '42.220'), (1, '45.020')] [2023-10-07 21:02:54,466][67838] Updated weights for policy 0, policy_version 33002 (0.0011) [2023-10-07 21:02:54,637][67871] Updated weights for policy 1, policy_version 33060 (0.0009) [2023-10-07 21:02:54,836][67838] Updated weights for policy 0, policy_version 33012 (0.0008) [2023-10-07 21:02:54,991][67871] Updated weights for policy 1, policy_version 33070 (0.0007) [2023-10-07 21:02:55,218][67838] Updated weights for policy 0, policy_version 33022 (0.0009) [2023-10-07 21:02:55,360][67871] Updated weights for policy 1, policy_version 33080 (0.0009) [2023-10-07 21:02:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67698688. Throughput: 0: 1637.3, 1: 1658.1. Samples: 16927554. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:02:57,478][66916] Avg episode reward: [(0, '39.660'), (1, '45.000')] [2023-10-07 21:02:59,386][67838] Updated weights for policy 0, policy_version 33032 (0.0009) [2023-10-07 21:02:59,640][67871] Updated weights for policy 1, policy_version 33090 (0.0010) [2023-10-07 21:02:59,748][67838] Updated weights for policy 0, policy_version 33042 (0.0007) [2023-10-07 21:03:00,003][67871] Updated weights for policy 1, policy_version 33100 (0.0007) [2023-10-07 21:03:00,120][67838] Updated weights for policy 0, policy_version 33052 (0.0009) [2023-10-07 21:03:00,377][67871] Updated weights for policy 1, policy_version 33110 (0.0007) [2023-10-07 21:03:00,746][67871] Updated weights for policy 1, policy_version 33120 (0.0007) [2023-10-07 21:03:02,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67764224. Throughput: 0: 1657.7, 1: 1654.4. Samples: 16946910. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:03:02,477][66916] Avg episode reward: [(0, '40.360'), (1, '44.790')] [2023-10-07 21:03:04,281][67838] Updated weights for policy 0, policy_version 33062 (0.0010) [2023-10-07 21:03:04,660][67838] Updated weights for policy 0, policy_version 33072 (0.0009) [2023-10-07 21:03:04,995][67871] Updated weights for policy 1, policy_version 33130 (0.0009) [2023-10-07 21:03:05,037][67838] Updated weights for policy 0, policy_version 33082 (0.0009) [2023-10-07 21:03:05,357][67871] Updated weights for policy 1, policy_version 33140 (0.0009) [2023-10-07 21:03:05,733][67871] Updated weights for policy 1, policy_version 33150 (0.0011) [2023-10-07 21:03:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67829760. Throughput: 0: 1654.5, 1: 1661.1. Samples: 16967174. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:03:07,477][66916] Avg episode reward: [(0, '40.950'), (1, '42.960')] [2023-10-07 21:03:07,485][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000033088_33882112.pth... [2023-10-07 21:03:07,485][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000033152_33947648.pth... [2023-10-07 21:03:07,516][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000031552_32309248.pth [2023-10-07 21:03:07,521][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000031616_32374784.pth [2023-10-07 21:03:09,212][67838] Updated weights for policy 0, policy_version 33092 (0.0010) [2023-10-07 21:03:09,577][67838] Updated weights for policy 0, policy_version 33102 (0.0007) [2023-10-07 21:03:09,728][67871] Updated weights for policy 1, policy_version 33160 (0.0007) [2023-10-07 21:03:09,959][67838] Updated weights for policy 0, policy_version 33112 (0.0007) [2023-10-07 21:03:10,096][67871] Updated weights for policy 1, policy_version 33170 (0.0009) [2023-10-07 21:03:10,461][67871] Updated weights for policy 1, policy_version 33180 (0.0009) [2023-10-07 21:03:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67895296. Throughput: 0: 1643.3, 1: 1651.4. Samples: 16977124. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:03:12,477][66916] Avg episode reward: [(0, '41.520'), (1, '45.970')] [2023-10-07 21:03:14,120][67838] Updated weights for policy 0, policy_version 33122 (0.0008) [2023-10-07 21:03:14,499][67838] Updated weights for policy 0, policy_version 33132 (0.0008) [2023-10-07 21:03:14,707][67871] Updated weights for policy 1, policy_version 33190 (0.0007) [2023-10-07 21:03:14,879][67838] Updated weights for policy 0, policy_version 33142 (0.0007) [2023-10-07 21:03:15,074][67871] Updated weights for policy 1, policy_version 33200 (0.0008) [2023-10-07 21:03:15,241][67838] Updated weights for policy 0, policy_version 33152 (0.0010) [2023-10-07 21:03:15,445][67871] Updated weights for policy 1, policy_version 33210 (0.0009) [2023-10-07 21:03:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67960832. Throughput: 0: 1652.9, 1: 1655.1. Samples: 16996256. Policy #0 lag: (min: 9.0, avg: 15.8, max: 41.0) [2023-10-07 21:03:17,477][66916] Avg episode reward: [(0, '43.160'), (1, '42.760')] [2023-10-07 21:03:19,459][67838] Updated weights for policy 0, policy_version 33162 (0.0007) [2023-10-07 21:03:19,522][67871] Updated weights for policy 1, policy_version 33220 (0.0007) [2023-10-07 21:03:19,838][67838] Updated weights for policy 0, policy_version 33172 (0.0008) [2023-10-07 21:03:19,883][67871] Updated weights for policy 1, policy_version 33230 (0.0009) [2023-10-07 21:03:20,204][67838] Updated weights for policy 0, policy_version 33182 (0.0008) [2023-10-07 21:03:20,240][67871] Updated weights for policy 1, policy_version 33240 (0.0009) [2023-10-07 21:03:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 68026368. Throughput: 0: 1651.7, 1: 1657.1. Samples: 17016660. Policy #0 lag: (min: 9.0, avg: 15.8, max: 41.0) [2023-10-07 21:03:22,477][66916] Avg episode reward: [(0, '40.870'), (1, '43.980')] [2023-10-07 21:03:24,167][67838] Updated weights for policy 0, policy_version 33192 (0.0007) [2023-10-07 21:03:24,447][67871] Updated weights for policy 1, policy_version 33250 (0.0010) [2023-10-07 21:03:24,532][67838] Updated weights for policy 0, policy_version 33202 (0.0009) [2023-10-07 21:03:24,816][67871] Updated weights for policy 1, policy_version 33260 (0.0009) [2023-10-07 21:03:24,914][67838] Updated weights for policy 0, policy_version 33212 (0.0008) [2023-10-07 21:03:25,180][67871] Updated weights for policy 1, policy_version 33270 (0.0008) [2023-10-07 21:03:25,547][67871] Updated weights for policy 1, policy_version 33280 (0.0007) [2023-10-07 21:03:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68091904. Throughput: 0: 1650.8, 1: 1652.3. Samples: 17026654. Policy #0 lag: (min: 9.0, avg: 15.8, max: 41.0) [2023-10-07 21:03:27,478][66916] Avg episode reward: [(0, '42.650'), (1, '46.170')] [2023-10-07 21:03:29,066][67838] Updated weights for policy 0, policy_version 33222 (0.0007) [2023-10-07 21:03:29,425][67838] Updated weights for policy 0, policy_version 33232 (0.0007) [2023-10-07 21:03:29,570][67871] Updated weights for policy 1, policy_version 33290 (0.0007) [2023-10-07 21:03:29,797][67838] Updated weights for policy 0, policy_version 33242 (0.0007) [2023-10-07 21:03:29,947][67871] Updated weights for policy 1, policy_version 33300 (0.0008) [2023-10-07 21:03:30,310][67871] Updated weights for policy 1, policy_version 33310 (0.0008) [2023-10-07 21:03:32,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68157440. Throughput: 0: 1659.5, 1: 1661.8. Samples: 17046568. Policy #0 lag: (min: 9.0, avg: 15.8, max: 41.0) [2023-10-07 21:03:32,477][66916] Avg episode reward: [(0, '44.480'), (1, '43.590')] [2023-10-07 21:03:32,478][67511] Saving new best policy, reward=44.480! [2023-10-07 21:03:33,751][67838] Updated weights for policy 0, policy_version 33252 (0.0009) [2023-10-07 21:03:34,133][67838] Updated weights for policy 0, policy_version 33262 (0.0010) [2023-10-07 21:03:34,499][67838] Updated weights for policy 0, policy_version 33272 (0.0009) [2023-10-07 21:03:34,514][67871] Updated weights for policy 1, policy_version 33320 (0.0010) [2023-10-07 21:03:34,880][67871] Updated weights for policy 1, policy_version 33330 (0.0010) [2023-10-07 21:03:35,257][67871] Updated weights for policy 1, policy_version 33340 (0.0008) [2023-10-07 21:03:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68222976. Throughput: 0: 1663.4, 1: 1664.3. Samples: 17066954. Policy #0 lag: (min: 9.0, avg: 15.8, max: 41.0) [2023-10-07 21:03:37,477][66916] Avg episode reward: [(0, '44.720'), (1, '45.650')] [2023-10-07 21:03:37,485][67511] Saving new best policy, reward=44.720! [2023-10-07 21:03:38,748][67838] Updated weights for policy 0, policy_version 33282 (0.0008) [2023-10-07 21:03:39,122][67838] Updated weights for policy 0, policy_version 33292 (0.0009) [2023-10-07 21:03:39,482][67838] Updated weights for policy 0, policy_version 33302 (0.0009) [2023-10-07 21:03:39,581][67871] Updated weights for policy 1, policy_version 33350 (0.0008) [2023-10-07 21:03:39,861][67838] Updated weights for policy 0, policy_version 33312 (0.0009) [2023-10-07 21:03:39,962][67871] Updated weights for policy 1, policy_version 33360 (0.0007) [2023-10-07 21:03:40,331][67871] Updated weights for policy 1, policy_version 33370 (0.0007) [2023-10-07 21:03:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68288512. Throughput: 0: 1652.1, 1: 1652.5. Samples: 17076264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:03:42,477][66916] Avg episode reward: [(0, '42.930'), (1, '44.370')] [2023-10-07 21:03:43,944][67838] Updated weights for policy 0, policy_version 33322 (0.0008) [2023-10-07 21:03:44,314][67838] Updated weights for policy 0, policy_version 33332 (0.0008) [2023-10-07 21:03:44,396][67871] Updated weights for policy 1, policy_version 33380 (0.0007) [2023-10-07 21:03:44,686][67838] Updated weights for policy 0, policy_version 33342 (0.0007) [2023-10-07 21:03:44,755][67871] Updated weights for policy 1, policy_version 33390 (0.0008) [2023-10-07 21:03:45,118][67871] Updated weights for policy 1, policy_version 33400 (0.0009) [2023-10-07 21:03:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68354048. Throughput: 0: 1658.6, 1: 1656.4. Samples: 17096086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:03:47,477][66916] Avg episode reward: [(0, '42.100'), (1, '45.770')] [2023-10-07 21:03:48,941][67838] Updated weights for policy 0, policy_version 33352 (0.0010) [2023-10-07 21:03:49,267][67871] Updated weights for policy 1, policy_version 33410 (0.0008) [2023-10-07 21:03:49,315][67838] Updated weights for policy 0, policy_version 33362 (0.0008) [2023-10-07 21:03:49,631][67871] Updated weights for policy 1, policy_version 33420 (0.0007) [2023-10-07 21:03:49,683][67838] Updated weights for policy 0, policy_version 33372 (0.0009) [2023-10-07 21:03:50,007][67871] Updated weights for policy 1, policy_version 33430 (0.0007) [2023-10-07 21:03:50,366][67871] Updated weights for policy 1, policy_version 33440 (0.0009) [2023-10-07 21:03:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68419584. Throughput: 0: 1658.0, 1: 1654.9. Samples: 17116258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:03:52,477][66916] Avg episode reward: [(0, '41.870'), (1, '43.840')] [2023-10-07 21:03:53,753][67838] Updated weights for policy 0, policy_version 33382 (0.0009) [2023-10-07 21:03:54,117][67838] Updated weights for policy 0, policy_version 33392 (0.0008) [2023-10-07 21:03:54,321][67871] Updated weights for policy 1, policy_version 33450 (0.0010) [2023-10-07 21:03:54,494][67838] Updated weights for policy 0, policy_version 33402 (0.0009) [2023-10-07 21:03:54,677][67871] Updated weights for policy 1, policy_version 33460 (0.0009) [2023-10-07 21:03:55,042][67871] Updated weights for policy 1, policy_version 33470 (0.0008) [2023-10-07 21:03:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68485120. Throughput: 0: 1656.2, 1: 1645.4. Samples: 17125698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:03:57,477][66916] Avg episode reward: [(0, '38.330'), (1, '46.700')] [2023-10-07 21:03:58,668][67838] Updated weights for policy 0, policy_version 33412 (0.0007) [2023-10-07 21:03:59,040][67838] Updated weights for policy 0, policy_version 33422 (0.0009) [2023-10-07 21:03:59,207][67871] Updated weights for policy 1, policy_version 33480 (0.0008) [2023-10-07 21:03:59,398][67838] Updated weights for policy 0, policy_version 33432 (0.0008) [2023-10-07 21:03:59,581][67871] Updated weights for policy 1, policy_version 33490 (0.0008) [2023-10-07 21:03:59,947][67871] Updated weights for policy 1, policy_version 33500 (0.0009) [2023-10-07 21:04:02,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 68550656. Throughput: 0: 1666.5, 1: 1659.7. Samples: 17145938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:04:02,478][66916] Avg episode reward: [(0, '41.340'), (1, '46.200')] [2023-10-07 21:04:03,478][67838] Updated weights for policy 0, policy_version 33442 (0.0009) [2023-10-07 21:04:03,847][67838] Updated weights for policy 0, policy_version 33452 (0.0007) [2023-10-07 21:04:03,923][67871] Updated weights for policy 1, policy_version 33510 (0.0008) [2023-10-07 21:04:04,217][67838] Updated weights for policy 0, policy_version 33462 (0.0008) [2023-10-07 21:04:04,294][67871] Updated weights for policy 1, policy_version 33520 (0.0008) [2023-10-07 21:04:04,583][67838] Updated weights for policy 0, policy_version 33472 (0.0010) [2023-10-07 21:04:04,656][67871] Updated weights for policy 1, policy_version 33530 (0.0009) [2023-10-07 21:04:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68616192. Throughput: 0: 1665.0, 1: 1662.9. Samples: 17166418. Policy #0 lag: (min: 14.0, avg: 16.8, max: 46.0) [2023-10-07 21:04:07,477][66916] Avg episode reward: [(0, '41.990'), (1, '49.080')] [2023-10-07 21:04:08,684][67871] Updated weights for policy 1, policy_version 33540 (0.0008) [2023-10-07 21:04:08,724][67838] Updated weights for policy 0, policy_version 33482 (0.0010) [2023-10-07 21:04:09,045][67871] Updated weights for policy 1, policy_version 33550 (0.0008) [2023-10-07 21:04:09,094][67838] Updated weights for policy 0, policy_version 33492 (0.0007) [2023-10-07 21:04:09,408][67871] Updated weights for policy 1, policy_version 33560 (0.0008) [2023-10-07 21:04:09,459][67838] Updated weights for policy 0, policy_version 33502 (0.0007) [2023-10-07 21:04:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68681728. Throughput: 0: 1656.3, 1: 1647.2. Samples: 17175312. Policy #0 lag: (min: 14.0, avg: 16.8, max: 46.0) [2023-10-07 21:04:12,477][66916] Avg episode reward: [(0, '39.750'), (1, '49.210')] [2023-10-07 21:04:13,461][67838] Updated weights for policy 0, policy_version 33512 (0.0008) [2023-10-07 21:04:13,599][67871] Updated weights for policy 1, policy_version 33570 (0.0007) [2023-10-07 21:04:13,833][67838] Updated weights for policy 0, policy_version 33522 (0.0008) [2023-10-07 21:04:13,970][67871] Updated weights for policy 1, policy_version 33580 (0.0009) [2023-10-07 21:04:14,207][67838] Updated weights for policy 0, policy_version 33532 (0.0009) [2023-10-07 21:04:14,341][67871] Updated weights for policy 1, policy_version 33590 (0.0009) [2023-10-07 21:04:14,702][67871] Updated weights for policy 1, policy_version 33600 (0.0009) [2023-10-07 21:04:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68747264. Throughput: 0: 1655.4, 1: 1660.8. Samples: 17195798. Policy #0 lag: (min: 14.0, avg: 16.8, max: 46.0) [2023-10-07 21:04:17,477][66916] Avg episode reward: [(0, '46.120'), (1, '52.230')] [2023-10-07 21:04:17,478][67511] Saving new best policy, reward=46.120! [2023-10-07 21:04:17,478][67676] Saving new best policy, reward=52.230! [2023-10-07 21:04:18,491][67838] Updated weights for policy 0, policy_version 33542 (0.0009) [2023-10-07 21:04:18,793][67871] Updated weights for policy 1, policy_version 33610 (0.0007) [2023-10-07 21:04:18,871][67838] Updated weights for policy 0, policy_version 33552 (0.0009) [2023-10-07 21:04:19,159][67871] Updated weights for policy 1, policy_version 33620 (0.0007) [2023-10-07 21:04:19,234][67838] Updated weights for policy 0, policy_version 33562 (0.0010) [2023-10-07 21:04:19,531][67871] Updated weights for policy 1, policy_version 33630 (0.0007) [2023-10-07 21:04:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68812800. Throughput: 0: 1654.9, 1: 1666.5. Samples: 17216416. Policy #0 lag: (min: 14.0, avg: 16.8, max: 46.0) [2023-10-07 21:04:22,477][66916] Avg episode reward: [(0, '40.770'), (1, '50.890')] [2023-10-07 21:04:23,215][67838] Updated weights for policy 0, policy_version 33572 (0.0009) [2023-10-07 21:04:23,600][67838] Updated weights for policy 0, policy_version 33582 (0.0009) [2023-10-07 21:04:23,665][67871] Updated weights for policy 1, policy_version 33640 (0.0007) [2023-10-07 21:04:23,965][67838] Updated weights for policy 0, policy_version 33592 (0.0008) [2023-10-07 21:04:24,030][67871] Updated weights for policy 1, policy_version 33650 (0.0009) [2023-10-07 21:04:24,403][67871] Updated weights for policy 1, policy_version 33660 (0.0008) [2023-10-07 21:04:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68878336. Throughput: 0: 1658.8, 1: 1653.8. Samples: 17225328. Policy #0 lag: (min: 14.0, avg: 16.8, max: 46.0) [2023-10-07 21:04:27,477][66916] Avg episode reward: [(0, '41.600'), (1, '52.270')] [2023-10-07 21:04:27,478][67676] Saving new best policy, reward=52.270! [2023-10-07 21:04:28,279][67838] Updated weights for policy 0, policy_version 33602 (0.0008) [2023-10-07 21:04:28,663][67838] Updated weights for policy 0, policy_version 33612 (0.0007) [2023-10-07 21:04:28,704][67871] Updated weights for policy 1, policy_version 33670 (0.0009) [2023-10-07 21:04:29,031][67838] Updated weights for policy 0, policy_version 33622 (0.0007) [2023-10-07 21:04:29,106][67871] Updated weights for policy 1, policy_version 33680 (0.0010) [2023-10-07 21:04:29,403][67838] Updated weights for policy 0, policy_version 33632 (0.0008) [2023-10-07 21:04:29,477][67871] Updated weights for policy 1, policy_version 33690 (0.0008) [2023-10-07 21:04:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68943872. Throughput: 0: 1655.5, 1: 1660.8. Samples: 17245318. Policy #0 lag: (min: 17.0, avg: 24.9, max: 49.0) [2023-10-07 21:04:32,478][66916] Avg episode reward: [(0, '40.630'), (1, '51.650')] [2023-10-07 21:04:33,593][67871] Updated weights for policy 1, policy_version 33700 (0.0008) [2023-10-07 21:04:33,756][67838] Updated weights for policy 0, policy_version 33642 (0.0007) [2023-10-07 21:04:33,947][67871] Updated weights for policy 1, policy_version 33710 (0.0007) [2023-10-07 21:04:34,131][67838] Updated weights for policy 0, policy_version 33652 (0.0007) [2023-10-07 21:04:34,315][67871] Updated weights for policy 1, policy_version 33720 (0.0007) [2023-10-07 21:04:34,505][67838] Updated weights for policy 0, policy_version 33662 (0.0007) [2023-10-07 21:04:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 69009408. Throughput: 0: 1661.1, 1: 1661.6. Samples: 17265780. Policy #0 lag: (min: 17.0, avg: 24.9, max: 49.0) [2023-10-07 21:04:37,478][66916] Avg episode reward: [(0, '41.060'), (1, '49.570')] [2023-10-07 21:04:38,451][67871] Updated weights for policy 1, policy_version 33730 (0.0009) [2023-10-07 21:04:38,539][67838] Updated weights for policy 0, policy_version 33672 (0.0008) [2023-10-07 21:04:38,823][67871] Updated weights for policy 1, policy_version 33740 (0.0009) [2023-10-07 21:04:38,907][67838] Updated weights for policy 0, policy_version 33682 (0.0009) [2023-10-07 21:04:39,188][67871] Updated weights for policy 1, policy_version 33750 (0.0009) [2023-10-07 21:04:39,271][67838] Updated weights for policy 0, policy_version 33692 (0.0009) [2023-10-07 21:04:39,563][67871] Updated weights for policy 1, policy_version 33760 (0.0009) [2023-10-07 21:04:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 69074944. Throughput: 0: 1657.8, 1: 1653.6. Samples: 17274712. Policy #0 lag: (min: 17.0, avg: 24.9, max: 49.0) [2023-10-07 21:04:42,478][66916] Avg episode reward: [(0, '41.130'), (1, '50.540')] [2023-10-07 21:04:43,317][67838] Updated weights for policy 0, policy_version 33702 (0.0009) [2023-10-07 21:04:43,688][67838] Updated weights for policy 0, policy_version 33712 (0.0007) [2023-10-07 21:04:43,839][67871] Updated weights for policy 1, policy_version 33770 (0.0009) [2023-10-07 21:04:44,059][67838] Updated weights for policy 0, policy_version 33722 (0.0008) [2023-10-07 21:04:44,198][67871] Updated weights for policy 1, policy_version 33780 (0.0009) [2023-10-07 21:04:44,569][67871] Updated weights for policy 1, policy_version 33790 (0.0010) [2023-10-07 21:04:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 69140480. Throughput: 0: 1656.0, 1: 1658.6. Samples: 17295096. Policy #0 lag: (min: 17.0, avg: 24.9, max: 49.0) [2023-10-07 21:04:47,477][66916] Avg episode reward: [(0, '40.740'), (1, '46.480')] [2023-10-07 21:04:48,053][67838] Updated weights for policy 0, policy_version 33732 (0.0010) [2023-10-07 21:04:48,415][67838] Updated weights for policy 0, policy_version 33742 (0.0009) [2023-10-07 21:04:48,758][67871] Updated weights for policy 1, policy_version 33800 (0.0008) [2023-10-07 21:04:48,797][67838] Updated weights for policy 0, policy_version 33752 (0.0007) [2023-10-07 21:04:49,122][67871] Updated weights for policy 1, policy_version 33810 (0.0008) [2023-10-07 21:04:49,484][67871] Updated weights for policy 1, policy_version 33820 (0.0010) [2023-10-07 21:04:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 69206016. Throughput: 0: 1656.3, 1: 1656.2. Samples: 17315482. Policy #0 lag: (min: 17.0, avg: 24.9, max: 49.0) [2023-10-07 21:04:52,477][66916] Avg episode reward: [(0, '41.680'), (1, '46.900')] [2023-10-07 21:04:52,977][67838] Updated weights for policy 0, policy_version 33762 (0.0008) [2023-10-07 21:04:53,351][67838] Updated weights for policy 0, policy_version 33772 (0.0007) [2023-10-07 21:04:53,670][67871] Updated weights for policy 1, policy_version 33830 (0.0007) [2023-10-07 21:04:53,720][67838] Updated weights for policy 0, policy_version 33782 (0.0007) [2023-10-07 21:04:54,029][67871] Updated weights for policy 1, policy_version 33840 (0.0008) [2023-10-07 21:04:54,091][67838] Updated weights for policy 0, policy_version 33792 (0.0009) [2023-10-07 21:04:54,406][67871] Updated weights for policy 1, policy_version 33850 (0.0009) [2023-10-07 21:04:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 69271552. Throughput: 0: 1663.5, 1: 1655.2. Samples: 17324652. Policy #0 lag: (min: 21.0, avg: 24.0, max: 53.0) [2023-10-07 21:04:57,478][66916] Avg episode reward: [(0, '39.570'), (1, '47.650')] [2023-10-07 21:04:58,118][67838] Updated weights for policy 0, policy_version 33802 (0.0007) [2023-10-07 21:04:58,267][67871] Updated weights for policy 1, policy_version 33860 (0.0008) [2023-10-07 21:04:58,506][67838] Updated weights for policy 0, policy_version 33812 (0.0008) [2023-10-07 21:04:58,639][67871] Updated weights for policy 1, policy_version 33870 (0.0007) [2023-10-07 21:04:58,874][67838] Updated weights for policy 0, policy_version 33822 (0.0007) [2023-10-07 21:04:59,005][67871] Updated weights for policy 1, policy_version 33880 (0.0008) [2023-10-07 21:05:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 69337088. Throughput: 0: 1660.4, 1: 1658.0. Samples: 17345126. Policy #0 lag: (min: 21.0, avg: 24.0, max: 53.0) [2023-10-07 21:05:02,477][66916] Avg episode reward: [(0, '41.630'), (1, '46.180')] [2023-10-07 21:05:03,127][67871] Updated weights for policy 1, policy_version 33890 (0.0008) [2023-10-07 21:05:03,131][67838] Updated weights for policy 0, policy_version 33832 (0.0009) [2023-10-07 21:05:03,497][67871] Updated weights for policy 1, policy_version 33900 (0.0007) [2023-10-07 21:05:03,502][67838] Updated weights for policy 0, policy_version 33842 (0.0007) [2023-10-07 21:05:03,862][67871] Updated weights for policy 1, policy_version 33910 (0.0007) [2023-10-07 21:05:03,879][67838] Updated weights for policy 0, policy_version 33852 (0.0008) [2023-10-07 21:05:04,225][67871] Updated weights for policy 1, policy_version 33920 (0.0009) [2023-10-07 21:05:07,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 69402624. Throughput: 0: 1664.6, 1: 1652.3. Samples: 17365678. Policy #0 lag: (min: 21.0, avg: 24.0, max: 53.0) [2023-10-07 21:05:07,478][66916] Avg episode reward: [(0, '39.090'), (1, '46.050')] [2023-10-07 21:05:07,487][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000033856_34668544.pth... [2023-10-07 21:05:07,488][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000033920_34734080.pth... [2023-10-07 21:05:07,519][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000032320_33095680.pth [2023-10-07 21:05:07,529][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000032384_33161216.pth [2023-10-07 21:05:07,834][67838] Updated weights for policy 0, policy_version 33862 (0.0009) [2023-10-07 21:05:08,212][67838] Updated weights for policy 0, policy_version 33872 (0.0007) [2023-10-07 21:05:08,328][67871] Updated weights for policy 1, policy_version 33930 (0.0008) [2023-10-07 21:05:08,577][67838] Updated weights for policy 0, policy_version 33882 (0.0008) [2023-10-07 21:05:08,692][67871] Updated weights for policy 1, policy_version 33940 (0.0007) [2023-10-07 21:05:09,058][67871] Updated weights for policy 1, policy_version 33950 (0.0008) [2023-10-07 21:05:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 69468160. Throughput: 0: 1665.2, 1: 1656.0. Samples: 17374784. Policy #0 lag: (min: 21.0, avg: 24.0, max: 53.0) [2023-10-07 21:05:12,477][66916] Avg episode reward: [(0, '39.910'), (1, '47.270')] [2023-10-07 21:05:12,698][67838] Updated weights for policy 0, policy_version 33892 (0.0008) [2023-10-07 21:05:13,066][67838] Updated weights for policy 0, policy_version 33902 (0.0010) [2023-10-07 21:05:13,088][67871] Updated weights for policy 1, policy_version 33960 (0.0008) [2023-10-07 21:05:13,435][67838] Updated weights for policy 0, policy_version 33912 (0.0009) [2023-10-07 21:05:13,458][67871] Updated weights for policy 1, policy_version 33970 (0.0007) [2023-10-07 21:05:13,829][67871] Updated weights for policy 1, policy_version 33980 (0.0007) [2023-10-07 21:05:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 69533696. Throughput: 0: 1663.6, 1: 1665.3. Samples: 17395122. Policy #0 lag: (min: 21.0, avg: 24.0, max: 53.0) [2023-10-07 21:05:17,477][66916] Avg episode reward: [(0, '40.660'), (1, '49.200')] [2023-10-07 21:05:17,661][67838] Updated weights for policy 0, policy_version 33922 (0.0008) [2023-10-07 21:05:17,859][67871] Updated weights for policy 1, policy_version 33990 (0.0008) [2023-10-07 21:05:18,026][67838] Updated weights for policy 0, policy_version 33932 (0.0008) [2023-10-07 21:05:18,226][67871] Updated weights for policy 1, policy_version 34000 (0.0007) [2023-10-07 21:05:18,395][67838] Updated weights for policy 0, policy_version 33942 (0.0008) [2023-10-07 21:05:18,589][67871] Updated weights for policy 1, policy_version 34010 (0.0008) [2023-10-07 21:05:18,767][67838] Updated weights for policy 0, policy_version 33952 (0.0009) [2023-10-07 21:05:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69599232. Throughput: 0: 1659.6, 1: 1671.4. Samples: 17415674. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-10-07 21:05:22,478][66916] Avg episode reward: [(0, '40.500'), (1, '48.610')] [2023-10-07 21:05:22,775][67871] Updated weights for policy 1, policy_version 34020 (0.0008) [2023-10-07 21:05:22,964][67838] Updated weights for policy 0, policy_version 33962 (0.0008) [2023-10-07 21:05:23,136][67871] Updated weights for policy 1, policy_version 34030 (0.0008) [2023-10-07 21:05:23,336][67838] Updated weights for policy 0, policy_version 33972 (0.0007) [2023-10-07 21:05:23,506][67871] Updated weights for policy 1, policy_version 34040 (0.0007) [2023-10-07 21:05:23,708][67838] Updated weights for policy 0, policy_version 33982 (0.0008) [2023-10-07 21:05:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 69664768. Throughput: 0: 1660.8, 1: 1673.3. Samples: 17424748. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-10-07 21:05:27,477][66916] Avg episode reward: [(0, '38.960'), (1, '46.930')] [2023-10-07 21:05:27,609][67871] Updated weights for policy 1, policy_version 34050 (0.0009) [2023-10-07 21:05:27,736][67838] Updated weights for policy 0, policy_version 33992 (0.0008) [2023-10-07 21:05:27,973][67871] Updated weights for policy 1, policy_version 34060 (0.0008) [2023-10-07 21:05:28,107][67838] Updated weights for policy 0, policy_version 34002 (0.0008) [2023-10-07 21:05:28,337][67871] Updated weights for policy 1, policy_version 34070 (0.0007) [2023-10-07 21:05:28,482][67838] Updated weights for policy 0, policy_version 34012 (0.0009) [2023-10-07 21:05:28,696][67871] Updated weights for policy 1, policy_version 34080 (0.0009) [2023-10-07 21:05:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 69730304. Throughput: 0: 1664.8, 1: 1674.6. Samples: 17445368. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-10-07 21:05:32,477][66916] Avg episode reward: [(0, '40.730'), (1, '48.930')] [2023-10-07 21:05:32,575][67838] Updated weights for policy 0, policy_version 34022 (0.0007) [2023-10-07 21:05:32,818][67871] Updated weights for policy 1, policy_version 34090 (0.0007) [2023-10-07 21:05:32,940][67838] Updated weights for policy 0, policy_version 34032 (0.0008) [2023-10-07 21:05:33,184][67871] Updated weights for policy 1, policy_version 34100 (0.0009) [2023-10-07 21:05:33,322][67838] Updated weights for policy 0, policy_version 34042 (0.0007) [2023-10-07 21:05:33,541][67871] Updated weights for policy 1, policy_version 34110 (0.0009) [2023-10-07 21:05:37,361][67838] Updated weights for policy 0, policy_version 34052 (0.0008) [2023-10-07 21:05:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 69795840. Throughput: 0: 1664.7, 1: 1674.5. Samples: 17465746. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-10-07 21:05:37,477][66916] Avg episode reward: [(0, '38.950'), (1, '48.290')] [2023-10-07 21:05:37,659][67871] Updated weights for policy 1, policy_version 34120 (0.0008) [2023-10-07 21:05:37,730][67838] Updated weights for policy 0, policy_version 34062 (0.0008) [2023-10-07 21:05:38,016][67871] Updated weights for policy 1, policy_version 34130 (0.0008) [2023-10-07 21:05:38,102][67838] Updated weights for policy 0, policy_version 34072 (0.0007) [2023-10-07 21:05:38,381][67871] Updated weights for policy 1, policy_version 34140 (0.0008) [2023-10-07 21:05:42,287][67838] Updated weights for policy 0, policy_version 34082 (0.0008) [2023-10-07 21:05:42,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69861376. Throughput: 0: 1664.4, 1: 1672.7. Samples: 17474824. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-10-07 21:05:42,477][66916] Avg episode reward: [(0, '37.290'), (1, '47.760')] [2023-10-07 21:05:42,570][67871] Updated weights for policy 1, policy_version 34150 (0.0009) [2023-10-07 21:05:42,657][67838] Updated weights for policy 0, policy_version 34092 (0.0008) [2023-10-07 21:05:42,937][67871] Updated weights for policy 1, policy_version 34160 (0.0008) [2023-10-07 21:05:43,019][67838] Updated weights for policy 0, policy_version 34102 (0.0007) [2023-10-07 21:05:43,305][67871] Updated weights for policy 1, policy_version 34170 (0.0007) [2023-10-07 21:05:43,392][67838] Updated weights for policy 0, policy_version 34112 (0.0008) [2023-10-07 21:05:47,350][67871] Updated weights for policy 1, policy_version 34180 (0.0008) [2023-10-07 21:05:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69926912. Throughput: 0: 1666.0, 1: 1666.5. Samples: 17495086. Policy #0 lag: (min: 10.0, avg: 10.1, max: 17.0) [2023-10-07 21:05:47,477][66916] Avg episode reward: [(0, '40.590'), (1, '45.950')] [2023-10-07 21:05:47,619][67838] Updated weights for policy 0, policy_version 34122 (0.0008) [2023-10-07 21:05:47,710][67871] Updated weights for policy 1, policy_version 34190 (0.0007) [2023-10-07 21:05:48,003][67838] Updated weights for policy 0, policy_version 34132 (0.0009) [2023-10-07 21:05:48,070][67871] Updated weights for policy 1, policy_version 34200 (0.0009) [2023-10-07 21:05:48,374][67838] Updated weights for policy 0, policy_version 34142 (0.0009) [2023-10-07 21:05:52,302][67871] Updated weights for policy 1, policy_version 34210 (0.0010) [2023-10-07 21:05:52,419][67838] Updated weights for policy 0, policy_version 34152 (0.0009) [2023-10-07 21:05:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69992448. Throughput: 0: 1658.4, 1: 1668.5. Samples: 17515386. Policy #0 lag: (min: 10.0, avg: 10.1, max: 17.0) [2023-10-07 21:05:52,478][66916] Avg episode reward: [(0, '39.770'), (1, '47.290')] [2023-10-07 21:05:52,659][67871] Updated weights for policy 1, policy_version 34220 (0.0007) [2023-10-07 21:05:52,793][67838] Updated weights for policy 0, policy_version 34162 (0.0008) [2023-10-07 21:05:53,023][67871] Updated weights for policy 1, policy_version 34230 (0.0008) [2023-10-07 21:05:53,166][67838] Updated weights for policy 0, policy_version 34172 (0.0009) [2023-10-07 21:05:53,391][67871] Updated weights for policy 1, policy_version 34240 (0.0008) [2023-10-07 21:05:57,250][67838] Updated weights for policy 0, policy_version 34182 (0.0009) [2023-10-07 21:05:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70057984. Throughput: 0: 1657.9, 1: 1666.8. Samples: 17524398. Policy #0 lag: (min: 10.0, avg: 10.1, max: 17.0) [2023-10-07 21:05:57,478][66916] Avg episode reward: [(0, '40.910'), (1, '46.650')] [2023-10-07 21:05:57,568][67871] Updated weights for policy 1, policy_version 34250 (0.0008) [2023-10-07 21:05:57,619][67838] Updated weights for policy 0, policy_version 34192 (0.0007) [2023-10-07 21:05:57,933][67871] Updated weights for policy 1, policy_version 34260 (0.0009) [2023-10-07 21:05:57,983][67838] Updated weights for policy 0, policy_version 34202 (0.0008) [2023-10-07 21:05:58,300][67871] Updated weights for policy 1, policy_version 34270 (0.0007) [2023-10-07 21:06:02,055][67838] Updated weights for policy 0, policy_version 34212 (0.0009) [2023-10-07 21:06:02,423][67838] Updated weights for policy 0, policy_version 34222 (0.0010) [2023-10-07 21:06:02,444][67871] Updated weights for policy 1, policy_version 34280 (0.0007) [2023-10-07 21:06:02,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70123520. Throughput: 0: 1660.9, 1: 1668.5. Samples: 17544944. Policy #0 lag: (min: 10.0, avg: 10.1, max: 17.0) [2023-10-07 21:06:02,478][66916] Avg episode reward: [(0, '42.550'), (1, '47.560')] [2023-10-07 21:06:02,802][67838] Updated weights for policy 0, policy_version 34232 (0.0009) [2023-10-07 21:06:02,821][67871] Updated weights for policy 1, policy_version 34290 (0.0008) [2023-10-07 21:06:03,177][67871] Updated weights for policy 1, policy_version 34300 (0.0009) [2023-10-07 21:06:07,009][67838] Updated weights for policy 0, policy_version 34242 (0.0007) [2023-10-07 21:06:07,170][67871] Updated weights for policy 1, policy_version 34310 (0.0009) [2023-10-07 21:06:07,388][67838] Updated weights for policy 0, policy_version 34252 (0.0007) [2023-10-07 21:06:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70189056. Throughput: 0: 1662.2, 1: 1663.0. Samples: 17565310. Policy #0 lag: (min: 10.0, avg: 10.1, max: 17.0) [2023-10-07 21:06:07,477][66916] Avg episode reward: [(0, '41.680'), (1, '45.790')] [2023-10-07 21:06:07,554][67871] Updated weights for policy 1, policy_version 34320 (0.0008) [2023-10-07 21:06:07,757][67838] Updated weights for policy 0, policy_version 34262 (0.0008) [2023-10-07 21:06:07,906][67871] Updated weights for policy 1, policy_version 34330 (0.0008) [2023-10-07 21:06:08,130][67838] Updated weights for policy 0, policy_version 34272 (0.0008) [2023-10-07 21:06:12,232][67871] Updated weights for policy 1, policy_version 34340 (0.0009) [2023-10-07 21:06:12,425][67838] Updated weights for policy 0, policy_version 34282 (0.0007) [2023-10-07 21:06:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70254592. Throughput: 0: 1662.9, 1: 1659.1. Samples: 17574236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:06:12,477][66916] Avg episode reward: [(0, '44.530'), (1, '47.580')] [2023-10-07 21:06:12,605][67871] Updated weights for policy 1, policy_version 34350 (0.0008) [2023-10-07 21:06:12,801][67838] Updated weights for policy 0, policy_version 34292 (0.0008) [2023-10-07 21:06:12,974][67871] Updated weights for policy 1, policy_version 34360 (0.0009) [2023-10-07 21:06:13,177][67838] Updated weights for policy 0, policy_version 34302 (0.0007) [2023-10-07 21:06:17,020][67871] Updated weights for policy 1, policy_version 34370 (0.0008) [2023-10-07 21:06:17,258][67838] Updated weights for policy 0, policy_version 34312 (0.0007) [2023-10-07 21:06:17,395][67871] Updated weights for policy 1, policy_version 34380 (0.0008) [2023-10-07 21:06:17,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70320128. Throughput: 0: 1660.9, 1: 1658.0. Samples: 17594720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:06:17,477][66916] Avg episode reward: [(0, '42.080'), (1, '47.940')] [2023-10-07 21:06:17,637][67838] Updated weights for policy 0, policy_version 34322 (0.0007) [2023-10-07 21:06:17,758][67871] Updated weights for policy 1, policy_version 34390 (0.0008) [2023-10-07 21:06:18,011][67838] Updated weights for policy 0, policy_version 34332 (0.0009) [2023-10-07 21:06:18,127][67871] Updated weights for policy 1, policy_version 34400 (0.0008) [2023-10-07 21:06:22,118][67838] Updated weights for policy 0, policy_version 34342 (0.0009) [2023-10-07 21:06:22,322][67871] Updated weights for policy 1, policy_version 34410 (0.0007) [2023-10-07 21:06:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 70385664. Throughput: 0: 1655.3, 1: 1660.9. Samples: 17614974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:06:22,477][66916] Avg episode reward: [(0, '46.040'), (1, '48.220')] [2023-10-07 21:06:22,486][67838] Updated weights for policy 0, policy_version 34352 (0.0009) [2023-10-07 21:06:22,694][67871] Updated weights for policy 1, policy_version 34420 (0.0008) [2023-10-07 21:06:22,851][67838] Updated weights for policy 0, policy_version 34362 (0.0008) [2023-10-07 21:06:23,053][67871] Updated weights for policy 1, policy_version 34430 (0.0007) [2023-10-07 21:06:27,090][67838] Updated weights for policy 0, policy_version 34372 (0.0009) [2023-10-07 21:06:27,095][67871] Updated weights for policy 1, policy_version 34440 (0.0007) [2023-10-07 21:06:27,462][67838] Updated weights for policy 0, policy_version 34382 (0.0009) [2023-10-07 21:06:27,465][67871] Updated weights for policy 1, policy_version 34450 (0.0007) [2023-10-07 21:06:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70451200. Throughput: 0: 1656.4, 1: 1658.7. Samples: 17624004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:06:27,477][66916] Avg episode reward: [(0, '43.980'), (1, '50.310')] [2023-10-07 21:06:27,836][67871] Updated weights for policy 1, policy_version 34460 (0.0008) [2023-10-07 21:06:27,846][67838] Updated weights for policy 0, policy_version 34392 (0.0009) [2023-10-07 21:06:31,938][67838] Updated weights for policy 0, policy_version 34402 (0.0009) [2023-10-07 21:06:31,988][67871] Updated weights for policy 1, policy_version 34470 (0.0007) [2023-10-07 21:06:32,346][67838] Updated weights for policy 0, policy_version 34412 (0.0009) [2023-10-07 21:06:32,359][67871] Updated weights for policy 1, policy_version 34480 (0.0008) [2023-10-07 21:06:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70516736. Throughput: 0: 1653.5, 1: 1665.1. Samples: 17644420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:06:32,477][66916] Avg episode reward: [(0, '48.500'), (1, '50.490')] [2023-10-07 21:06:32,711][67838] Updated weights for policy 0, policy_version 34422 (0.0007) [2023-10-07 21:06:32,714][67871] Updated weights for policy 1, policy_version 34490 (0.0008) [2023-10-07 21:06:33,081][67511] Saving new best policy, reward=48.500! [2023-10-07 21:06:33,082][67838] Updated weights for policy 0, policy_version 34432 (0.0008) [2023-10-07 21:06:36,864][67871] Updated weights for policy 1, policy_version 34500 (0.0008) [2023-10-07 21:06:37,117][67838] Updated weights for policy 0, policy_version 34442 (0.0009) [2023-10-07 21:06:37,232][67871] Updated weights for policy 1, policy_version 34510 (0.0009) [2023-10-07 21:06:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70582272. Throughput: 0: 1649.7, 1: 1663.4. Samples: 17664474. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-10-07 21:06:37,478][66916] Avg episode reward: [(0, '44.070'), (1, '48.810')] [2023-10-07 21:06:37,480][67838] Updated weights for policy 0, policy_version 34452 (0.0009) [2023-10-07 21:06:37,593][67871] Updated weights for policy 1, policy_version 34520 (0.0009) [2023-10-07 21:06:37,854][67838] Updated weights for policy 0, policy_version 34462 (0.0009) [2023-10-07 21:06:41,662][67871] Updated weights for policy 1, policy_version 34530 (0.0009) [2023-10-07 21:06:41,822][67838] Updated weights for policy 0, policy_version 34472 (0.0008) [2023-10-07 21:06:42,033][67871] Updated weights for policy 1, policy_version 34540 (0.0008) [2023-10-07 21:06:42,196][67838] Updated weights for policy 0, policy_version 34482 (0.0010) [2023-10-07 21:06:42,405][67871] Updated weights for policy 1, policy_version 34550 (0.0007) [2023-10-07 21:06:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70647808. Throughput: 0: 1658.1, 1: 1665.4. Samples: 17673954. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-10-07 21:06:42,477][66916] Avg episode reward: [(0, '41.760'), (1, '49.100')] [2023-10-07 21:06:42,565][67838] Updated weights for policy 0, policy_version 34492 (0.0008) [2023-10-07 21:06:42,766][67871] Updated weights for policy 1, policy_version 34560 (0.0007) [2023-10-07 21:06:46,863][67838] Updated weights for policy 0, policy_version 34502 (0.0008) [2023-10-07 21:06:47,042][67871] Updated weights for policy 1, policy_version 34570 (0.0008) [2023-10-07 21:06:47,232][67838] Updated weights for policy 0, policy_version 34512 (0.0009) [2023-10-07 21:06:47,403][67871] Updated weights for policy 1, policy_version 34580 (0.0008) [2023-10-07 21:06:47,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70713344. Throughput: 0: 1657.0, 1: 1660.9. Samples: 17694250. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-10-07 21:06:47,478][66916] Avg episode reward: [(0, '41.150'), (1, '46.850')] [2023-10-07 21:06:47,601][67838] Updated weights for policy 0, policy_version 34522 (0.0008) [2023-10-07 21:06:47,768][67871] Updated weights for policy 1, policy_version 34590 (0.0007) [2023-10-07 21:06:51,632][67838] Updated weights for policy 0, policy_version 34532 (0.0008) [2023-10-07 21:06:51,873][67871] Updated weights for policy 1, policy_version 34600 (0.0009) [2023-10-07 21:06:52,000][67838] Updated weights for policy 0, policy_version 34542 (0.0009) [2023-10-07 21:06:52,236][67871] Updated weights for policy 1, policy_version 34610 (0.0008) [2023-10-07 21:06:52,383][67838] Updated weights for policy 0, policy_version 34552 (0.0008) [2023-10-07 21:06:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70778880. Throughput: 0: 1646.0, 1: 1655.6. Samples: 17713884. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-10-07 21:06:52,477][66916] Avg episode reward: [(0, '37.650'), (1, '46.760')] [2023-10-07 21:06:52,596][67871] Updated weights for policy 1, policy_version 34620 (0.0007) [2023-10-07 21:06:56,569][67838] Updated weights for policy 0, policy_version 34562 (0.0010) [2023-10-07 21:06:56,772][67871] Updated weights for policy 1, policy_version 34630 (0.0007) [2023-10-07 21:06:56,947][67838] Updated weights for policy 0, policy_version 34572 (0.0007) [2023-10-07 21:06:57,141][67871] Updated weights for policy 1, policy_version 34640 (0.0007) [2023-10-07 21:06:57,317][67838] Updated weights for policy 0, policy_version 34582 (0.0008) [2023-10-07 21:06:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70844416. Throughput: 0: 1658.2, 1: 1662.3. Samples: 17723660. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-10-07 21:06:57,478][66916] Avg episode reward: [(0, '42.990'), (1, '46.430')] [2023-10-07 21:06:57,509][67871] Updated weights for policy 1, policy_version 34650 (0.0007) [2023-10-07 21:06:57,678][67838] Updated weights for policy 0, policy_version 34592 (0.0007) [2023-10-07 21:07:01,601][67838] Updated weights for policy 0, policy_version 34602 (0.0009) [2023-10-07 21:07:01,647][67871] Updated weights for policy 1, policy_version 34660 (0.0008) [2023-10-07 21:07:01,968][67838] Updated weights for policy 0, policy_version 34612 (0.0009) [2023-10-07 21:07:02,025][67871] Updated weights for policy 1, policy_version 34670 (0.0008) [2023-10-07 21:07:02,339][67838] Updated weights for policy 0, policy_version 34622 (0.0009) [2023-10-07 21:07:02,391][67871] Updated weights for policy 1, policy_version 34680 (0.0007) [2023-10-07 21:07:02,477][66916] Fps is (10 sec: 16383.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 70942720. Throughput: 0: 1656.6, 1: 1658.5. Samples: 17743902. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-07 21:07:02,478][66916] Avg episode reward: [(0, '38.150'), (1, '50.350')] [2023-10-07 21:07:06,434][67871] Updated weights for policy 1, policy_version 34690 (0.0008) [2023-10-07 21:07:06,519][67838] Updated weights for policy 0, policy_version 34632 (0.0008) [2023-10-07 21:07:06,793][67871] Updated weights for policy 1, policy_version 34700 (0.0009) [2023-10-07 21:07:06,894][67838] Updated weights for policy 0, policy_version 34642 (0.0008) [2023-10-07 21:07:07,155][67871] Updated weights for policy 1, policy_version 34710 (0.0010) [2023-10-07 21:07:07,266][67838] Updated weights for policy 0, policy_version 34652 (0.0008) [2023-10-07 21:07:07,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 71008256. Throughput: 0: 1642.6, 1: 1649.9. Samples: 17763138. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-07 21:07:07,478][66916] Avg episode reward: [(0, '40.960'), (1, '49.440')] [2023-10-07 21:07:07,488][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000034656_35487744.pth... [2023-10-07 21:07:07,517][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000034720_35553280.pth... [2023-10-07 21:07:07,520][67871] Updated weights for policy 1, policy_version 34720 (0.0009) [2023-10-07 21:07:07,521][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000033088_33882112.pth [2023-10-07 21:07:07,554][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000033152_33947648.pth [2023-10-07 21:07:11,404][67838] Updated weights for policy 0, policy_version 34662 (0.0011) [2023-10-07 21:07:11,590][67871] Updated weights for policy 1, policy_version 34730 (0.0008) [2023-10-07 21:07:11,783][67838] Updated weights for policy 0, policy_version 34672 (0.0009) [2023-10-07 21:07:11,956][67871] Updated weights for policy 1, policy_version 34740 (0.0008) [2023-10-07 21:07:12,154][67838] Updated weights for policy 0, policy_version 34682 (0.0009) [2023-10-07 21:07:12,314][67871] Updated weights for policy 1, policy_version 34750 (0.0009) [2023-10-07 21:07:12,476][66916] Fps is (10 sec: 16384.6, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 71106560. Throughput: 0: 1656.5, 1: 1666.3. Samples: 17773530. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-07 21:07:12,477][66916] Avg episode reward: [(0, '41.760'), (1, '49.800')] [2023-10-07 21:07:16,416][67838] Updated weights for policy 0, policy_version 34692 (0.0008) [2023-10-07 21:07:16,580][67871] Updated weights for policy 1, policy_version 34760 (0.0007) [2023-10-07 21:07:16,791][67838] Updated weights for policy 0, policy_version 34702 (0.0007) [2023-10-07 21:07:16,945][67871] Updated weights for policy 1, policy_version 34770 (0.0009) [2023-10-07 21:07:17,160][67838] Updated weights for policy 0, policy_version 34712 (0.0007) [2023-10-07 21:07:17,306][67871] Updated weights for policy 1, policy_version 34780 (0.0007) [2023-10-07 21:07:17,476][66916] Fps is (10 sec: 16384.5, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 71172096. Throughput: 0: 1659.9, 1: 1658.8. Samples: 17793760. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-07 21:07:17,477][66916] Avg episode reward: [(0, '39.460'), (1, '48.560')] [2023-10-07 21:07:21,293][67838] Updated weights for policy 0, policy_version 34722 (0.0008) [2023-10-07 21:07:21,423][67871] Updated weights for policy 1, policy_version 34790 (0.0009) [2023-10-07 21:07:21,673][67838] Updated weights for policy 0, policy_version 34732 (0.0007) [2023-10-07 21:07:21,772][67871] Updated weights for policy 1, policy_version 34800 (0.0009) [2023-10-07 21:07:22,044][67838] Updated weights for policy 0, policy_version 34742 (0.0008) [2023-10-07 21:07:22,130][67871] Updated weights for policy 1, policy_version 34810 (0.0007) [2023-10-07 21:07:22,409][67838] Updated weights for policy 0, policy_version 34752 (0.0008) [2023-10-07 21:07:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 71237632. Throughput: 0: 1656.1, 1: 1640.8. Samples: 17812834. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-07 21:07:22,477][66916] Avg episode reward: [(0, '39.320'), (1, '46.640')] [2023-10-07 21:07:26,257][67871] Updated weights for policy 1, policy_version 34820 (0.0008) [2023-10-07 21:07:26,509][67838] Updated weights for policy 0, policy_version 34762 (0.0010) [2023-10-07 21:07:26,622][67871] Updated weights for policy 1, policy_version 34830 (0.0008) [2023-10-07 21:07:26,876][67838] Updated weights for policy 0, policy_version 34772 (0.0007) [2023-10-07 21:07:26,979][67871] Updated weights for policy 1, policy_version 34840 (0.0008) [2023-10-07 21:07:27,245][67838] Updated weights for policy 0, policy_version 34782 (0.0007) [2023-10-07 21:07:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 71303168. Throughput: 0: 1667.1, 1: 1651.9. Samples: 17823308. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-07 21:07:27,477][66916] Avg episode reward: [(0, '39.460'), (1, '46.930')] [2023-10-07 21:07:31,293][67871] Updated weights for policy 1, policy_version 34850 (0.0009) [2023-10-07 21:07:31,337][67838] Updated weights for policy 0, policy_version 34792 (0.0009) [2023-10-07 21:07:31,651][67871] Updated weights for policy 1, policy_version 34860 (0.0008) [2023-10-07 21:07:31,702][67838] Updated weights for policy 0, policy_version 34802 (0.0009) [2023-10-07 21:07:32,017][67871] Updated weights for policy 1, policy_version 34870 (0.0008) [2023-10-07 21:07:32,067][67838] Updated weights for policy 0, policy_version 34812 (0.0008) [2023-10-07 21:07:32,383][67871] Updated weights for policy 1, policy_version 34880 (0.0009) [2023-10-07 21:07:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 71368704. Throughput: 0: 1669.3, 1: 1653.8. Samples: 17843790. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-07 21:07:32,478][66916] Avg episode reward: [(0, '39.290'), (1, '44.150')] [2023-10-07 21:07:36,257][67838] Updated weights for policy 0, policy_version 34822 (0.0009) [2023-10-07 21:07:36,606][67871] Updated weights for policy 1, policy_version 34890 (0.0008) [2023-10-07 21:07:36,619][67838] Updated weights for policy 0, policy_version 34832 (0.0009) [2023-10-07 21:07:36,976][67871] Updated weights for policy 1, policy_version 34900 (0.0009) [2023-10-07 21:07:36,999][67838] Updated weights for policy 0, policy_version 34842 (0.0008) [2023-10-07 21:07:37,339][67871] Updated weights for policy 1, policy_version 34910 (0.0008) [2023-10-07 21:07:37,477][66916] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13329.3). Total num frames: 71434240. Throughput: 0: 1660.0, 1: 1643.2. Samples: 17862528. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-07 21:07:37,477][66916] Avg episode reward: [(0, '38.770'), (1, '44.000')] [2023-10-07 21:07:40,996][67838] Updated weights for policy 0, policy_version 34852 (0.0007) [2023-10-07 21:07:41,363][67838] Updated weights for policy 0, policy_version 34862 (0.0007) [2023-10-07 21:07:41,392][67871] Updated weights for policy 1, policy_version 34920 (0.0008) [2023-10-07 21:07:41,740][67838] Updated weights for policy 0, policy_version 34872 (0.0009) [2023-10-07 21:07:41,749][67871] Updated weights for policy 1, policy_version 34930 (0.0008) [2023-10-07 21:07:42,118][67871] Updated weights for policy 1, policy_version 34940 (0.0007) [2023-10-07 21:07:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 71499776. Throughput: 0: 1671.0, 1: 1651.1. Samples: 17873154. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-07 21:07:42,478][66916] Avg episode reward: [(0, '38.250'), (1, '46.210')] [2023-10-07 21:07:45,802][67838] Updated weights for policy 0, policy_version 34882 (0.0007) [2023-10-07 21:07:46,180][67838] Updated weights for policy 0, policy_version 34892 (0.0007) [2023-10-07 21:07:46,397][67871] Updated weights for policy 1, policy_version 34950 (0.0009) [2023-10-07 21:07:46,544][67838] Updated weights for policy 0, policy_version 34902 (0.0008) [2023-10-07 21:07:46,761][67871] Updated weights for policy 1, policy_version 34960 (0.0008) [2023-10-07 21:07:46,914][67838] Updated weights for policy 0, policy_version 34912 (0.0009) [2023-10-07 21:07:47,126][67871] Updated weights for policy 1, policy_version 34970 (0.0010) [2023-10-07 21:07:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 71565312. Throughput: 0: 1666.9, 1: 1651.5. Samples: 17893230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:07:47,477][66916] Avg episode reward: [(0, '36.860'), (1, '42.250')] [2023-10-07 21:07:50,877][67838] Updated weights for policy 0, policy_version 34922 (0.0008) [2023-10-07 21:07:51,204][67871] Updated weights for policy 1, policy_version 34980 (0.0009) [2023-10-07 21:07:51,251][67838] Updated weights for policy 0, policy_version 34932 (0.0009) [2023-10-07 21:07:51,570][67871] Updated weights for policy 1, policy_version 34990 (0.0009) [2023-10-07 21:07:51,624][67838] Updated weights for policy 0, policy_version 34942 (0.0010) [2023-10-07 21:07:51,934][67871] Updated weights for policy 1, policy_version 35000 (0.0010) [2023-10-07 21:07:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 71630848. Throughput: 0: 1665.7, 1: 1641.2. Samples: 17911946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:07:52,477][66916] Avg episode reward: [(0, '37.720'), (1, '47.360')] [2023-10-07 21:07:55,832][67838] Updated weights for policy 0, policy_version 34952 (0.0009) [2023-10-07 21:07:56,160][67871] Updated weights for policy 1, policy_version 35010 (0.0010) [2023-10-07 21:07:56,204][67838] Updated weights for policy 0, policy_version 34962 (0.0007) [2023-10-07 21:07:56,527][67871] Updated weights for policy 1, policy_version 35020 (0.0007) [2023-10-07 21:07:56,573][67838] Updated weights for policy 0, policy_version 34972 (0.0007) [2023-10-07 21:07:56,888][67871] Updated weights for policy 1, policy_version 35030 (0.0008) [2023-10-07 21:07:57,255][67871] Updated weights for policy 1, policy_version 35040 (0.0007) [2023-10-07 21:07:57,477][66916] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13329.3). Total num frames: 71696384. Throughput: 0: 1673.7, 1: 1646.4. Samples: 17922938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:07:57,478][66916] Avg episode reward: [(0, '37.790'), (1, '45.620')] [2023-10-07 21:08:00,775][67838] Updated weights for policy 0, policy_version 34982 (0.0008) [2023-10-07 21:08:01,153][67838] Updated weights for policy 0, policy_version 34992 (0.0010) [2023-10-07 21:08:01,260][67871] Updated weights for policy 1, policy_version 35050 (0.0008) [2023-10-07 21:08:01,513][67838] Updated weights for policy 0, policy_version 35002 (0.0008) [2023-10-07 21:08:01,622][67871] Updated weights for policy 1, policy_version 35060 (0.0007) [2023-10-07 21:08:01,993][67871] Updated weights for policy 1, policy_version 35070 (0.0008) [2023-10-07 21:08:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 71761920. Throughput: 0: 1666.0, 1: 1651.2. Samples: 17943032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:08:02,477][66916] Avg episode reward: [(0, '39.380'), (1, '46.280')] [2023-10-07 21:08:05,562][67838] Updated weights for policy 0, policy_version 35012 (0.0008) [2023-10-07 21:08:05,930][67838] Updated weights for policy 0, policy_version 35022 (0.0008) [2023-10-07 21:08:06,236][67871] Updated weights for policy 1, policy_version 35080 (0.0009) [2023-10-07 21:08:06,305][67838] Updated weights for policy 0, policy_version 35032 (0.0008) [2023-10-07 21:08:06,595][67871] Updated weights for policy 1, policy_version 35090 (0.0008) [2023-10-07 21:08:06,966][67871] Updated weights for policy 1, policy_version 35100 (0.0009) [2023-10-07 21:08:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.3). Total num frames: 71827456. Throughput: 0: 1661.4, 1: 1651.2. Samples: 17961904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:08:07,477][66916] Avg episode reward: [(0, '38.520'), (1, '46.710')] [2023-10-07 21:08:10,380][67838] Updated weights for policy 0, policy_version 35042 (0.0008) [2023-10-07 21:08:10,796][67838] Updated weights for policy 0, policy_version 35052 (0.0009) [2023-10-07 21:08:11,036][67871] Updated weights for policy 1, policy_version 35110 (0.0008) [2023-10-07 21:08:11,164][67838] Updated weights for policy 0, policy_version 35062 (0.0008) [2023-10-07 21:08:11,414][67871] Updated weights for policy 1, policy_version 35120 (0.0007) [2023-10-07 21:08:11,542][67838] Updated weights for policy 0, policy_version 35072 (0.0008) [2023-10-07 21:08:11,775][67871] Updated weights for policy 1, policy_version 35130 (0.0009) [2023-10-07 21:08:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71892992. Throughput: 0: 1671.5, 1: 1657.9. Samples: 17973130. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-07 21:08:12,477][66916] Avg episode reward: [(0, '39.900'), (1, '45.190')] [2023-10-07 21:08:15,629][67838] Updated weights for policy 0, policy_version 35082 (0.0011) [2023-10-07 21:08:15,948][67871] Updated weights for policy 1, policy_version 35140 (0.0008) [2023-10-07 21:08:15,997][67838] Updated weights for policy 0, policy_version 35092 (0.0010) [2023-10-07 21:08:16,319][67871] Updated weights for policy 1, policy_version 35150 (0.0008) [2023-10-07 21:08:16,374][67838] Updated weights for policy 0, policy_version 35102 (0.0008) [2023-10-07 21:08:16,676][67871] Updated weights for policy 1, policy_version 35160 (0.0010) [2023-10-07 21:08:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 71958528. Throughput: 0: 1649.7, 1: 1658.5. Samples: 17992660. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-07 21:08:17,478][66916] Avg episode reward: [(0, '38.820'), (1, '44.470')] [2023-10-07 21:08:20,498][67838] Updated weights for policy 0, policy_version 35112 (0.0010) [2023-10-07 21:08:20,826][67871] Updated weights for policy 1, policy_version 35170 (0.0009) [2023-10-07 21:08:20,877][67838] Updated weights for policy 0, policy_version 35122 (0.0010) [2023-10-07 21:08:21,205][67871] Updated weights for policy 1, policy_version 35180 (0.0007) [2023-10-07 21:08:21,252][67838] Updated weights for policy 0, policy_version 35132 (0.0007) [2023-10-07 21:08:21,575][67871] Updated weights for policy 1, policy_version 35190 (0.0008) [2023-10-07 21:08:21,936][67871] Updated weights for policy 1, policy_version 35200 (0.0007) [2023-10-07 21:08:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72024064. Throughput: 0: 1663.6, 1: 1654.0. Samples: 18011820. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-07 21:08:22,478][66916] Avg episode reward: [(0, '38.380'), (1, '48.750')] [2023-10-07 21:08:25,198][67838] Updated weights for policy 0, policy_version 35142 (0.0008) [2023-10-07 21:08:25,569][67838] Updated weights for policy 0, policy_version 35152 (0.0010) [2023-10-07 21:08:25,943][67838] Updated weights for policy 0, policy_version 35162 (0.0008) [2023-10-07 21:08:25,950][67871] Updated weights for policy 1, policy_version 35210 (0.0007) [2023-10-07 21:08:26,321][67871] Updated weights for policy 1, policy_version 35220 (0.0007) [2023-10-07 21:08:26,689][67871] Updated weights for policy 1, policy_version 35230 (0.0008) [2023-10-07 21:08:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72089600. Throughput: 0: 1668.8, 1: 1664.7. Samples: 18023158. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-07 21:08:27,477][66916] Avg episode reward: [(0, '38.830'), (1, '47.340')] [2023-10-07 21:08:30,307][67838] Updated weights for policy 0, policy_version 35172 (0.0009) [2023-10-07 21:08:30,678][67838] Updated weights for policy 0, policy_version 35182 (0.0009) [2023-10-07 21:08:30,997][67871] Updated weights for policy 1, policy_version 35240 (0.0008) [2023-10-07 21:08:31,045][67838] Updated weights for policy 0, policy_version 35192 (0.0009) [2023-10-07 21:08:31,361][67871] Updated weights for policy 1, policy_version 35250 (0.0008) [2023-10-07 21:08:31,730][67871] Updated weights for policy 1, policy_version 35260 (0.0009) [2023-10-07 21:08:32,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 72155136. Throughput: 0: 1650.7, 1: 1660.6. Samples: 18042238. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-07 21:08:32,478][66916] Avg episode reward: [(0, '38.160'), (1, '48.350')] [2023-10-07 21:08:35,162][67838] Updated weights for policy 0, policy_version 35202 (0.0009) [2023-10-07 21:08:35,541][67838] Updated weights for policy 0, policy_version 35212 (0.0009) [2023-10-07 21:08:35,754][67871] Updated weights for policy 1, policy_version 35270 (0.0008) [2023-10-07 21:08:35,916][67838] Updated weights for policy 0, policy_version 35222 (0.0010) [2023-10-07 21:08:36,119][67871] Updated weights for policy 1, policy_version 35280 (0.0007) [2023-10-07 21:08:36,288][67838] Updated weights for policy 0, policy_version 35232 (0.0009) [2023-10-07 21:08:36,486][67871] Updated weights for policy 1, policy_version 35290 (0.0009) [2023-10-07 21:08:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72220672. Throughput: 0: 1661.7, 1: 1653.3. Samples: 18061120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:08:37,477][66916] Avg episode reward: [(0, '38.610'), (1, '50.050')] [2023-10-07 21:08:40,350][67838] Updated weights for policy 0, policy_version 35242 (0.0008) [2023-10-07 21:08:40,538][67871] Updated weights for policy 1, policy_version 35300 (0.0009) [2023-10-07 21:08:40,715][67838] Updated weights for policy 0, policy_version 35252 (0.0007) [2023-10-07 21:08:40,903][67871] Updated weights for policy 1, policy_version 35310 (0.0008) [2023-10-07 21:08:41,085][67838] Updated weights for policy 0, policy_version 35262 (0.0007) [2023-10-07 21:08:41,281][67871] Updated weights for policy 1, policy_version 35320 (0.0008) [2023-10-07 21:08:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72286208. Throughput: 0: 1661.1, 1: 1665.1. Samples: 18072616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:08:42,478][66916] Avg episode reward: [(0, '36.980'), (1, '49.420')] [2023-10-07 21:08:45,241][67838] Updated weights for policy 0, policy_version 35272 (0.0007) [2023-10-07 21:08:45,404][67871] Updated weights for policy 1, policy_version 35330 (0.0009) [2023-10-07 21:08:45,616][67838] Updated weights for policy 0, policy_version 35282 (0.0009) [2023-10-07 21:08:45,765][67871] Updated weights for policy 1, policy_version 35340 (0.0009) [2023-10-07 21:08:45,995][67838] Updated weights for policy 0, policy_version 35292 (0.0009) [2023-10-07 21:08:46,138][67871] Updated weights for policy 1, policy_version 35350 (0.0010) [2023-10-07 21:08:46,506][67871] Updated weights for policy 1, policy_version 35360 (0.0008) [2023-10-07 21:08:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72351744. Throughput: 0: 1647.5, 1: 1654.2. Samples: 18091608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:08:47,477][66916] Avg episode reward: [(0, '37.190'), (1, '48.920')] [2023-10-07 21:08:50,143][67838] Updated weights for policy 0, policy_version 35302 (0.0007) [2023-10-07 21:08:50,516][67838] Updated weights for policy 0, policy_version 35312 (0.0007) [2023-10-07 21:08:50,636][67871] Updated weights for policy 1, policy_version 35370 (0.0008) [2023-10-07 21:08:50,884][67838] Updated weights for policy 0, policy_version 35322 (0.0008) [2023-10-07 21:08:51,002][67871] Updated weights for policy 1, policy_version 35380 (0.0010) [2023-10-07 21:08:51,369][67871] Updated weights for policy 1, policy_version 35390 (0.0009) [2023-10-07 21:08:52,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 72417280. Throughput: 0: 1660.0, 1: 1653.1. Samples: 18110992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:08:52,478][66916] Avg episode reward: [(0, '35.120'), (1, '52.200')] [2023-10-07 21:08:55,189][67838] Updated weights for policy 0, policy_version 35332 (0.0008) [2023-10-07 21:08:55,572][67838] Updated weights for policy 0, policy_version 35342 (0.0008) [2023-10-07 21:08:55,587][67871] Updated weights for policy 1, policy_version 35400 (0.0009) [2023-10-07 21:08:55,948][67871] Updated weights for policy 1, policy_version 35410 (0.0009) [2023-10-07 21:08:55,948][67838] Updated weights for policy 0, policy_version 35352 (0.0009) [2023-10-07 21:08:56,319][67871] Updated weights for policy 1, policy_version 35420 (0.0007) [2023-10-07 21:08:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 72482816. Throughput: 0: 1653.0, 1: 1660.6. Samples: 18122242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:08:57,477][66916] Avg episode reward: [(0, '37.340'), (1, '52.420')] [2023-10-07 21:08:57,478][67676] Saving new best policy, reward=52.420! [2023-10-07 21:09:00,034][67838] Updated weights for policy 0, policy_version 35362 (0.0009) [2023-10-07 21:09:00,408][67838] Updated weights for policy 0, policy_version 35372 (0.0009) [2023-10-07 21:09:00,487][67871] Updated weights for policy 1, policy_version 35430 (0.0009) [2023-10-07 21:09:00,783][67838] Updated weights for policy 0, policy_version 35382 (0.0009) [2023-10-07 21:09:00,864][67871] Updated weights for policy 1, policy_version 35440 (0.0009) [2023-10-07 21:09:01,151][67838] Updated weights for policy 0, policy_version 35392 (0.0009) [2023-10-07 21:09:01,228][67871] Updated weights for policy 1, policy_version 35450 (0.0010) [2023-10-07 21:09:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72548352. Throughput: 0: 1647.5, 1: 1642.1. Samples: 18140690. Policy #0 lag: (min: 31.0, avg: 32.6, max: 59.0) [2023-10-07 21:09:02,477][66916] Avg episode reward: [(0, '36.510'), (1, '51.010')] [2023-10-07 21:09:05,223][67838] Updated weights for policy 0, policy_version 35402 (0.0007) [2023-10-07 21:09:05,342][67871] Updated weights for policy 1, policy_version 35460 (0.0009) [2023-10-07 21:09:05,591][67838] Updated weights for policy 0, policy_version 35412 (0.0009) [2023-10-07 21:09:05,702][67871] Updated weights for policy 1, policy_version 35470 (0.0008) [2023-10-07 21:09:05,965][67838] Updated weights for policy 0, policy_version 35422 (0.0009) [2023-10-07 21:09:06,072][67871] Updated weights for policy 1, policy_version 35480 (0.0009) [2023-10-07 21:09:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72613888. Throughput: 0: 1648.6, 1: 1645.5. Samples: 18160052. Policy #0 lag: (min: 31.0, avg: 32.6, max: 59.0) [2023-10-07 21:09:07,477][66916] Avg episode reward: [(0, '36.940'), (1, '50.720')] [2023-10-07 21:09:07,486][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000035424_36274176.pth... [2023-10-07 21:09:07,486][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000035488_36339712.pth... [2023-10-07 21:09:07,523][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000033856_34668544.pth [2023-10-07 21:09:07,524][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000033920_34734080.pth [2023-10-07 21:09:10,105][67838] Updated weights for policy 0, policy_version 35432 (0.0010) [2023-10-07 21:09:10,420][67871] Updated weights for policy 1, policy_version 35490 (0.0011) [2023-10-07 21:09:10,468][67838] Updated weights for policy 0, policy_version 35442 (0.0009) [2023-10-07 21:09:10,810][67871] Updated weights for policy 1, policy_version 35500 (0.0008) [2023-10-07 21:09:10,844][67838] Updated weights for policy 0, policy_version 35452 (0.0008) [2023-10-07 21:09:11,181][67871] Updated weights for policy 1, policy_version 35510 (0.0008) [2023-10-07 21:09:11,550][67871] Updated weights for policy 1, policy_version 35520 (0.0009) [2023-10-07 21:09:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72679424. Throughput: 0: 1642.7, 1: 1650.8. Samples: 18171368. Policy #0 lag: (min: 31.0, avg: 32.6, max: 59.0) [2023-10-07 21:09:12,477][66916] Avg episode reward: [(0, '36.880'), (1, '50.920')] [2023-10-07 21:09:14,971][67838] Updated weights for policy 0, policy_version 35462 (0.0008) [2023-10-07 21:09:15,331][67838] Updated weights for policy 0, policy_version 35472 (0.0009) [2023-10-07 21:09:15,684][67871] Updated weights for policy 1, policy_version 35530 (0.0009) [2023-10-07 21:09:15,704][67838] Updated weights for policy 0, policy_version 35482 (0.0009) [2023-10-07 21:09:16,046][67871] Updated weights for policy 1, policy_version 35540 (0.0009) [2023-10-07 21:09:16,429][67871] Updated weights for policy 1, policy_version 35550 (0.0010) [2023-10-07 21:09:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72744960. Throughput: 0: 1643.3, 1: 1646.4. Samples: 18190276. Policy #0 lag: (min: 31.0, avg: 32.6, max: 59.0) [2023-10-07 21:09:17,478][66916] Avg episode reward: [(0, '38.110'), (1, '48.670')] [2023-10-07 21:09:19,873][67838] Updated weights for policy 0, policy_version 35492 (0.0008) [2023-10-07 21:09:20,236][67838] Updated weights for policy 0, policy_version 35502 (0.0008) [2023-10-07 21:09:20,392][67871] Updated weights for policy 1, policy_version 35560 (0.0008) [2023-10-07 21:09:20,613][67838] Updated weights for policy 0, policy_version 35512 (0.0010) [2023-10-07 21:09:20,753][67871] Updated weights for policy 1, policy_version 35570 (0.0010) [2023-10-07 21:09:21,125][67871] Updated weights for policy 1, policy_version 35580 (0.0011) [2023-10-07 21:09:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 72810496. Throughput: 0: 1652.8, 1: 1656.8. Samples: 18210054. Policy #0 lag: (min: 31.0, avg: 32.6, max: 59.0) [2023-10-07 21:09:22,477][66916] Avg episode reward: [(0, '40.090'), (1, '46.210')] [2023-10-07 21:09:24,770][67838] Updated weights for policy 0, policy_version 35522 (0.0009) [2023-10-07 21:09:25,150][67838] Updated weights for policy 0, policy_version 35532 (0.0010) [2023-10-07 21:09:25,501][67871] Updated weights for policy 1, policy_version 35590 (0.0009) [2023-10-07 21:09:25,524][67838] Updated weights for policy 0, policy_version 35542 (0.0007) [2023-10-07 21:09:25,866][67871] Updated weights for policy 1, policy_version 35600 (0.0008) [2023-10-07 21:09:25,904][67838] Updated weights for policy 0, policy_version 35552 (0.0009) [2023-10-07 21:09:26,242][67871] Updated weights for policy 1, policy_version 35610 (0.0010) [2023-10-07 21:09:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72876032. Throughput: 0: 1648.4, 1: 1653.9. Samples: 18221218. Policy #0 lag: (min: 6.0, avg: 10.2, max: 38.0) [2023-10-07 21:09:27,478][66916] Avg episode reward: [(0, '38.890'), (1, '48.010')] [2023-10-07 21:09:30,131][67838] Updated weights for policy 0, policy_version 35562 (0.0007) [2023-10-07 21:09:30,319][67871] Updated weights for policy 1, policy_version 35620 (0.0009) [2023-10-07 21:09:30,499][67838] Updated weights for policy 0, policy_version 35572 (0.0007) [2023-10-07 21:09:30,690][67871] Updated weights for policy 1, policy_version 35630 (0.0009) [2023-10-07 21:09:30,874][67838] Updated weights for policy 0, policy_version 35582 (0.0008) [2023-10-07 21:09:31,059][67871] Updated weights for policy 1, policy_version 35640 (0.0008) [2023-10-07 21:09:32,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72941568. Throughput: 0: 1644.0, 1: 1648.7. Samples: 18239780. Policy #0 lag: (min: 6.0, avg: 10.2, max: 38.0) [2023-10-07 21:09:32,477][66916] Avg episode reward: [(0, '38.980'), (1, '46.210')] [2023-10-07 21:09:34,895][67838] Updated weights for policy 0, policy_version 35592 (0.0009) [2023-10-07 21:09:35,082][67871] Updated weights for policy 1, policy_version 35650 (0.0008) [2023-10-07 21:09:35,262][67838] Updated weights for policy 0, policy_version 35602 (0.0009) [2023-10-07 21:09:35,454][67871] Updated weights for policy 1, policy_version 35660 (0.0009) [2023-10-07 21:09:35,639][67838] Updated weights for policy 0, policy_version 35612 (0.0008) [2023-10-07 21:09:35,809][67871] Updated weights for policy 1, policy_version 35670 (0.0007) [2023-10-07 21:09:36,180][67871] Updated weights for policy 1, policy_version 35680 (0.0007) [2023-10-07 21:09:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 73007104. Throughput: 0: 1653.1, 1: 1655.6. Samples: 18259880. Policy #0 lag: (min: 6.0, avg: 10.2, max: 38.0) [2023-10-07 21:09:37,478][66916] Avg episode reward: [(0, '38.490'), (1, '47.080')] [2023-10-07 21:09:39,728][67838] Updated weights for policy 0, policy_version 35622 (0.0010) [2023-10-07 21:09:40,072][67871] Updated weights for policy 1, policy_version 35690 (0.0007) [2023-10-07 21:09:40,103][67838] Updated weights for policy 0, policy_version 35632 (0.0009) [2023-10-07 21:09:40,446][67871] Updated weights for policy 1, policy_version 35700 (0.0008) [2023-10-07 21:09:40,478][67838] Updated weights for policy 0, policy_version 35642 (0.0008) [2023-10-07 21:09:40,811][67871] Updated weights for policy 1, policy_version 35710 (0.0007) [2023-10-07 21:09:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 73072640. Throughput: 0: 1644.3, 1: 1653.1. Samples: 18270630. Policy #0 lag: (min: 6.0, avg: 10.2, max: 38.0) [2023-10-07 21:09:42,478][66916] Avg episode reward: [(0, '37.690'), (1, '47.760')] [2023-10-07 21:09:44,524][67838] Updated weights for policy 0, policy_version 35652 (0.0010) [2023-10-07 21:09:44,893][67838] Updated weights for policy 0, policy_version 35662 (0.0009) [2023-10-07 21:09:45,164][67871] Updated weights for policy 1, policy_version 35720 (0.0009) [2023-10-07 21:09:45,256][67838] Updated weights for policy 0, policy_version 35672 (0.0009) [2023-10-07 21:09:45,531][67871] Updated weights for policy 1, policy_version 35730 (0.0008) [2023-10-07 21:09:45,906][67871] Updated weights for policy 1, policy_version 35740 (0.0008) [2023-10-07 21:09:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 73138176. Throughput: 0: 1660.6, 1: 1646.3. Samples: 18289498. Policy #0 lag: (min: 6.0, avg: 10.2, max: 38.0) [2023-10-07 21:09:47,478][66916] Avg episode reward: [(0, '34.670'), (1, '48.560')] [2023-10-07 21:09:49,265][67838] Updated weights for policy 0, policy_version 35682 (0.0007) [2023-10-07 21:09:49,663][67838] Updated weights for policy 0, policy_version 35692 (0.0007) [2023-10-07 21:09:50,035][67838] Updated weights for policy 0, policy_version 35702 (0.0007) [2023-10-07 21:09:50,223][67871] Updated weights for policy 1, policy_version 35750 (0.0009) [2023-10-07 21:09:50,409][67838] Updated weights for policy 0, policy_version 35712 (0.0008) [2023-10-07 21:09:50,590][67871] Updated weights for policy 1, policy_version 35760 (0.0008) [2023-10-07 21:09:50,965][67871] Updated weights for policy 1, policy_version 35770 (0.0009) [2023-10-07 21:09:52,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 73203712. Throughput: 0: 1663.2, 1: 1654.6. Samples: 18309354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:09:52,478][66916] Avg episode reward: [(0, '35.560'), (1, '46.290')] [2023-10-07 21:09:54,525][67838] Updated weights for policy 0, policy_version 35722 (0.0008) [2023-10-07 21:09:54,903][67838] Updated weights for policy 0, policy_version 35732 (0.0007) [2023-10-07 21:09:55,282][67838] Updated weights for policy 0, policy_version 35742 (0.0007) [2023-10-07 21:09:55,290][67871] Updated weights for policy 1, policy_version 35780 (0.0009) [2023-10-07 21:09:55,691][67871] Updated weights for policy 1, policy_version 35790 (0.0012) [2023-10-07 21:09:56,051][67871] Updated weights for policy 1, policy_version 35800 (0.0009) [2023-10-07 21:09:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 73269248. Throughput: 0: 1652.3, 1: 1653.9. Samples: 18320144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:09:57,477][66916] Avg episode reward: [(0, '38.700'), (1, '45.740')] [2023-10-07 21:09:59,086][67838] Updated weights for policy 0, policy_version 35752 (0.0009) [2023-10-07 21:09:59,457][67838] Updated weights for policy 0, policy_version 35762 (0.0010) [2023-10-07 21:09:59,827][67838] Updated weights for policy 0, policy_version 35772 (0.0011) [2023-10-07 21:10:00,131][67871] Updated weights for policy 1, policy_version 35810 (0.0010) [2023-10-07 21:10:00,507][67871] Updated weights for policy 1, policy_version 35820 (0.0008) [2023-10-07 21:10:00,878][67871] Updated weights for policy 1, policy_version 35830 (0.0009) [2023-10-07 21:10:01,235][67871] Updated weights for policy 1, policy_version 35840 (0.0008) [2023-10-07 21:10:02,477][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 73334784. Throughput: 0: 1671.2, 1: 1644.3. Samples: 18339476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:10:02,478][66916] Avg episode reward: [(0, '35.160'), (1, '46.500')] [2023-10-07 21:10:04,088][67838] Updated weights for policy 0, policy_version 35782 (0.0008) [2023-10-07 21:10:04,461][67838] Updated weights for policy 0, policy_version 35792 (0.0009) [2023-10-07 21:10:04,838][67838] Updated weights for policy 0, policy_version 35802 (0.0008) [2023-10-07 21:10:05,254][67871] Updated weights for policy 1, policy_version 35850 (0.0007) [2023-10-07 21:10:05,615][67871] Updated weights for policy 1, policy_version 35860 (0.0008) [2023-10-07 21:10:05,986][67871] Updated weights for policy 1, policy_version 35870 (0.0010) [2023-10-07 21:10:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 73400320. Throughput: 0: 1673.7, 1: 1647.5. Samples: 18359508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:10:07,478][66916] Avg episode reward: [(0, '38.430'), (1, '47.220')] [2023-10-07 21:10:08,801][67838] Updated weights for policy 0, policy_version 35812 (0.0008) [2023-10-07 21:10:09,173][67838] Updated weights for policy 0, policy_version 35822 (0.0008) [2023-10-07 21:10:09,544][67838] Updated weights for policy 0, policy_version 35832 (0.0008) [2023-10-07 21:10:10,176][67871] Updated weights for policy 1, policy_version 35880 (0.0009) [2023-10-07 21:10:10,546][67871] Updated weights for policy 1, policy_version 35890 (0.0007) [2023-10-07 21:10:10,914][67871] Updated weights for policy 1, policy_version 35900 (0.0007) [2023-10-07 21:10:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 73465856. Throughput: 0: 1653.6, 1: 1650.9. Samples: 18369918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:10:12,478][66916] Avg episode reward: [(0, '34.820'), (1, '48.430')] [2023-10-07 21:10:13,691][67838] Updated weights for policy 0, policy_version 35842 (0.0007) [2023-10-07 21:10:14,060][67838] Updated weights for policy 0, policy_version 35852 (0.0008) [2023-10-07 21:10:14,434][67838] Updated weights for policy 0, policy_version 35862 (0.0008) [2023-10-07 21:10:14,814][67838] Updated weights for policy 0, policy_version 35872 (0.0007) [2023-10-07 21:10:14,862][67871] Updated weights for policy 1, policy_version 35910 (0.0007) [2023-10-07 21:10:15,233][67871] Updated weights for policy 1, policy_version 35920 (0.0008) [2023-10-07 21:10:15,609][67871] Updated weights for policy 1, policy_version 35930 (0.0009) [2023-10-07 21:10:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 73531392. Throughput: 0: 1680.9, 1: 1641.8. Samples: 18389300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:10:17,477][66916] Avg episode reward: [(0, '36.400'), (1, '48.880')] [2023-10-07 21:10:18,980][67838] Updated weights for policy 0, policy_version 35882 (0.0007) [2023-10-07 21:10:19,343][67838] Updated weights for policy 0, policy_version 35892 (0.0009) [2023-10-07 21:10:19,716][67838] Updated weights for policy 0, policy_version 35902 (0.0007) [2023-10-07 21:10:19,835][67871] Updated weights for policy 1, policy_version 35940 (0.0009) [2023-10-07 21:10:20,202][67871] Updated weights for policy 1, policy_version 35950 (0.0010) [2023-10-07 21:10:20,579][67871] Updated weights for policy 1, policy_version 35960 (0.0011) [2023-10-07 21:10:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 73596928. Throughput: 0: 1674.2, 1: 1652.9. Samples: 18409598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:10:22,478][66916] Avg episode reward: [(0, '36.940'), (1, '48.130')] [2023-10-07 21:10:23,870][67838] Updated weights for policy 0, policy_version 35912 (0.0008) [2023-10-07 21:10:24,237][67838] Updated weights for policy 0, policy_version 35922 (0.0009) [2023-10-07 21:10:24,615][67838] Updated weights for policy 0, policy_version 35932 (0.0007) [2023-10-07 21:10:24,742][67871] Updated weights for policy 1, policy_version 35970 (0.0009) [2023-10-07 21:10:25,110][67871] Updated weights for policy 1, policy_version 35980 (0.0009) [2023-10-07 21:10:25,473][67871] Updated weights for policy 1, policy_version 35990 (0.0008) [2023-10-07 21:10:25,835][67871] Updated weights for policy 1, policy_version 36000 (0.0009) [2023-10-07 21:10:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 73662464. Throughput: 0: 1659.2, 1: 1652.2. Samples: 18419642. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:10:27,477][66916] Avg episode reward: [(0, '34.920'), (1, '45.700')] [2023-10-07 21:10:28,809][67838] Updated weights for policy 0, policy_version 35942 (0.0007) [2023-10-07 21:10:29,193][67838] Updated weights for policy 0, policy_version 35952 (0.0008) [2023-10-07 21:10:29,558][67838] Updated weights for policy 0, policy_version 35962 (0.0007) [2023-10-07 21:10:30,000][67871] Updated weights for policy 1, policy_version 36010 (0.0010) [2023-10-07 21:10:30,363][67871] Updated weights for policy 1, policy_version 36020 (0.0009) [2023-10-07 21:10:30,727][67871] Updated weights for policy 1, policy_version 36030 (0.0009) [2023-10-07 21:10:32,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 73728000. Throughput: 0: 1670.2, 1: 1649.2. Samples: 18438872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:10:32,477][66916] Avg episode reward: [(0, '41.220'), (1, '48.550')] [2023-10-07 21:10:33,485][67838] Updated weights for policy 0, policy_version 35972 (0.0007) [2023-10-07 21:10:33,862][67838] Updated weights for policy 0, policy_version 35982 (0.0007) [2023-10-07 21:10:34,229][67838] Updated weights for policy 0, policy_version 35992 (0.0007) [2023-10-07 21:10:34,963][67871] Updated weights for policy 1, policy_version 36040 (0.0008) [2023-10-07 21:10:35,331][67871] Updated weights for policy 1, policy_version 36050 (0.0009) [2023-10-07 21:10:35,707][67871] Updated weights for policy 1, policy_version 36060 (0.0009) [2023-10-07 21:10:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 73793536. Throughput: 0: 1677.6, 1: 1655.6. Samples: 18459348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:10:37,478][66916] Avg episode reward: [(0, '36.330'), (1, '47.870')] [2023-10-07 21:10:38,319][67838] Updated weights for policy 0, policy_version 36002 (0.0009) [2023-10-07 21:10:38,709][67838] Updated weights for policy 0, policy_version 36012 (0.0010) [2023-10-07 21:10:39,079][67838] Updated weights for policy 0, policy_version 36022 (0.0008) [2023-10-07 21:10:39,450][67838] Updated weights for policy 0, policy_version 36032 (0.0008) [2023-10-07 21:10:39,852][67871] Updated weights for policy 1, policy_version 36070 (0.0009) [2023-10-07 21:10:40,219][67871] Updated weights for policy 1, policy_version 36080 (0.0010) [2023-10-07 21:10:40,579][67871] Updated weights for policy 1, policy_version 36090 (0.0010) [2023-10-07 21:10:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 73859072. Throughput: 0: 1663.5, 1: 1651.7. Samples: 18469330. Policy #0 lag: (min: 24.0, avg: 51.5, max: 56.0) [2023-10-07 21:10:42,477][66916] Avg episode reward: [(0, '40.750'), (1, '50.190')] [2023-10-07 21:10:43,362][67838] Updated weights for policy 0, policy_version 36042 (0.0007) [2023-10-07 21:10:43,744][67838] Updated weights for policy 0, policy_version 36052 (0.0009) [2023-10-07 21:10:44,116][67838] Updated weights for policy 0, policy_version 36062 (0.0009) [2023-10-07 21:10:44,722][67871] Updated weights for policy 1, policy_version 36100 (0.0010) [2023-10-07 21:10:45,095][67871] Updated weights for policy 1, policy_version 36110 (0.0007) [2023-10-07 21:10:45,452][67871] Updated weights for policy 1, policy_version 36120 (0.0010) [2023-10-07 21:10:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 73924608. Throughput: 0: 1670.2, 1: 1650.9. Samples: 18488926. Policy #0 lag: (min: 24.0, avg: 51.5, max: 56.0) [2023-10-07 21:10:47,477][66916] Avg episode reward: [(0, '39.300'), (1, '47.750')] [2023-10-07 21:10:48,291][67838] Updated weights for policy 0, policy_version 36072 (0.0010) [2023-10-07 21:10:48,663][67838] Updated weights for policy 0, policy_version 36082 (0.0008) [2023-10-07 21:10:49,032][67838] Updated weights for policy 0, policy_version 36092 (0.0008) [2023-10-07 21:10:49,614][67871] Updated weights for policy 1, policy_version 36130 (0.0009) [2023-10-07 21:10:50,035][67871] Updated weights for policy 1, policy_version 36140 (0.0009) [2023-10-07 21:10:50,414][67871] Updated weights for policy 1, policy_version 36150 (0.0009) [2023-10-07 21:10:50,775][67871] Updated weights for policy 1, policy_version 36160 (0.0009) [2023-10-07 21:10:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 73990144. Throughput: 0: 1670.7, 1: 1657.1. Samples: 18509258. Policy #0 lag: (min: 24.0, avg: 51.5, max: 56.0) [2023-10-07 21:10:52,477][66916] Avg episode reward: [(0, '37.660'), (1, '48.750')] [2023-10-07 21:10:53,091][67838] Updated weights for policy 0, policy_version 36102 (0.0009) [2023-10-07 21:10:53,472][67838] Updated weights for policy 0, policy_version 36112 (0.0008) [2023-10-07 21:10:53,840][67838] Updated weights for policy 0, policy_version 36122 (0.0009) [2023-10-07 21:10:54,747][67871] Updated weights for policy 1, policy_version 36170 (0.0007) [2023-10-07 21:10:55,118][67871] Updated weights for policy 1, policy_version 36180 (0.0007) [2023-10-07 21:10:55,481][67871] Updated weights for policy 1, policy_version 36190 (0.0009) [2023-10-07 21:10:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74055680. Throughput: 0: 1670.5, 1: 1646.5. Samples: 18519186. Policy #0 lag: (min: 24.0, avg: 51.5, max: 56.0) [2023-10-07 21:10:57,477][66916] Avg episode reward: [(0, '40.790'), (1, '49.040')] [2023-10-07 21:10:57,971][67838] Updated weights for policy 0, policy_version 36132 (0.0010) [2023-10-07 21:10:58,341][67838] Updated weights for policy 0, policy_version 36142 (0.0009) [2023-10-07 21:10:58,722][67838] Updated weights for policy 0, policy_version 36152 (0.0008) [2023-10-07 21:10:59,568][67871] Updated weights for policy 1, policy_version 36200 (0.0007) [2023-10-07 21:10:59,941][67871] Updated weights for policy 1, policy_version 36210 (0.0007) [2023-10-07 21:11:00,312][67871] Updated weights for policy 1, policy_version 36220 (0.0009) [2023-10-07 21:11:02,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 74121216. Throughput: 0: 1671.1, 1: 1659.1. Samples: 18539164. Policy #0 lag: (min: 24.0, avg: 51.5, max: 56.0) [2023-10-07 21:11:02,478][66916] Avg episode reward: [(0, '40.590'), (1, '48.850')] [2023-10-07 21:11:02,747][67838] Updated weights for policy 0, policy_version 36162 (0.0009) [2023-10-07 21:11:03,122][67838] Updated weights for policy 0, policy_version 36172 (0.0008) [2023-10-07 21:11:03,497][67838] Updated weights for policy 0, policy_version 36182 (0.0009) [2023-10-07 21:11:03,869][67838] Updated weights for policy 0, policy_version 36192 (0.0010) [2023-10-07 21:11:04,392][67871] Updated weights for policy 1, policy_version 36230 (0.0010) [2023-10-07 21:11:04,751][67871] Updated weights for policy 1, policy_version 36240 (0.0009) [2023-10-07 21:11:05,115][67871] Updated weights for policy 1, policy_version 36250 (0.0010) [2023-10-07 21:11:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 74186752. Throughput: 0: 1672.3, 1: 1666.1. Samples: 18559822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:11:07,477][66916] Avg episode reward: [(0, '41.120'), (1, '49.290')] [2023-10-07 21:11:07,488][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000036192_37060608.pth... [2023-10-07 21:11:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000036256_37126144.pth... [2023-10-07 21:11:07,527][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000034656_35487744.pth [2023-10-07 21:11:07,531][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000034720_35553280.pth [2023-10-07 21:11:08,142][67838] Updated weights for policy 0, policy_version 36202 (0.0008) [2023-10-07 21:11:08,505][67838] Updated weights for policy 0, policy_version 36212 (0.0008) [2023-10-07 21:11:08,886][67838] Updated weights for policy 0, policy_version 36222 (0.0007) [2023-10-07 21:11:08,995][67871] Updated weights for policy 1, policy_version 36260 (0.0008) [2023-10-07 21:11:09,356][67871] Updated weights for policy 1, policy_version 36270 (0.0008) [2023-10-07 21:11:09,730][67871] Updated weights for policy 1, policy_version 36280 (0.0007) [2023-10-07 21:11:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74252288. Throughput: 0: 1672.3, 1: 1647.8. Samples: 18569048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:11:12,478][66916] Avg episode reward: [(0, '40.640'), (1, '46.490')] [2023-10-07 21:11:12,978][67838] Updated weights for policy 0, policy_version 36232 (0.0007) [2023-10-07 21:11:13,356][67838] Updated weights for policy 0, policy_version 36242 (0.0008) [2023-10-07 21:11:13,733][67838] Updated weights for policy 0, policy_version 36252 (0.0008) [2023-10-07 21:11:13,959][67871] Updated weights for policy 1, policy_version 36290 (0.0007) [2023-10-07 21:11:14,319][67871] Updated weights for policy 1, policy_version 36300 (0.0009) [2023-10-07 21:11:14,682][67871] Updated weights for policy 1, policy_version 36310 (0.0007) [2023-10-07 21:11:15,056][67871] Updated weights for policy 1, policy_version 36320 (0.0008) [2023-10-07 21:11:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74317824. Throughput: 0: 1672.3, 1: 1668.5. Samples: 18589208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:11:17,477][66916] Avg episode reward: [(0, '41.990'), (1, '47.450')] [2023-10-07 21:11:17,825][67838] Updated weights for policy 0, policy_version 36262 (0.0009) [2023-10-07 21:11:18,207][67838] Updated weights for policy 0, policy_version 36272 (0.0010) [2023-10-07 21:11:18,580][67838] Updated weights for policy 0, policy_version 36282 (0.0007) [2023-10-07 21:11:19,181][67871] Updated weights for policy 1, policy_version 36330 (0.0008) [2023-10-07 21:11:19,544][67871] Updated weights for policy 1, policy_version 36340 (0.0007) [2023-10-07 21:11:19,912][67871] Updated weights for policy 1, policy_version 36350 (0.0008) [2023-10-07 21:11:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 74383360. Throughput: 0: 1670.0, 1: 1671.7. Samples: 18609722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:11:22,477][66916] Avg episode reward: [(0, '39.610'), (1, '45.830')] [2023-10-07 21:11:22,695][67838] Updated weights for policy 0, policy_version 36292 (0.0011) [2023-10-07 21:11:23,074][67838] Updated weights for policy 0, policy_version 36302 (0.0009) [2023-10-07 21:11:23,442][67838] Updated weights for policy 0, policy_version 36312 (0.0008) [2023-10-07 21:11:23,937][67871] Updated weights for policy 1, policy_version 36360 (0.0007) [2023-10-07 21:11:24,306][67871] Updated weights for policy 1, policy_version 36370 (0.0007) [2023-10-07 21:11:24,669][67871] Updated weights for policy 1, policy_version 36380 (0.0007) [2023-10-07 21:11:27,474][67838] Updated weights for policy 0, policy_version 36322 (0.0009) [2023-10-07 21:11:27,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 74448896. Throughput: 0: 1671.9, 1: 1654.7. Samples: 18619032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:11:27,477][66916] Avg episode reward: [(0, '42.870'), (1, '46.620')] [2023-10-07 21:11:27,856][67838] Updated weights for policy 0, policy_version 36332 (0.0007) [2023-10-07 21:11:28,239][67838] Updated weights for policy 0, policy_version 36342 (0.0007) [2023-10-07 21:11:28,608][67838] Updated weights for policy 0, policy_version 36352 (0.0007) [2023-10-07 21:11:28,834][67871] Updated weights for policy 1, policy_version 36390 (0.0008) [2023-10-07 21:11:29,208][67871] Updated weights for policy 1, policy_version 36400 (0.0008) [2023-10-07 21:11:29,575][67871] Updated weights for policy 1, policy_version 36410 (0.0007) [2023-10-07 21:11:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74514432. Throughput: 0: 1674.5, 1: 1671.2. Samples: 18639482. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 21:11:32,477][66916] Avg episode reward: [(0, '38.980'), (1, '47.120')] [2023-10-07 21:11:32,545][67838] Updated weights for policy 0, policy_version 36362 (0.0008) [2023-10-07 21:11:32,913][67838] Updated weights for policy 0, policy_version 36372 (0.0008) [2023-10-07 21:11:33,281][67838] Updated weights for policy 0, policy_version 36382 (0.0009) [2023-10-07 21:11:33,732][67871] Updated weights for policy 1, policy_version 36420 (0.0008) [2023-10-07 21:11:34,095][67871] Updated weights for policy 1, policy_version 36430 (0.0007) [2023-10-07 21:11:34,457][67871] Updated weights for policy 1, policy_version 36440 (0.0007) [2023-10-07 21:11:37,408][67838] Updated weights for policy 0, policy_version 36392 (0.0011) [2023-10-07 21:11:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74579968. Throughput: 0: 1674.1, 1: 1680.3. Samples: 18660204. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 21:11:37,477][66916] Avg episode reward: [(0, '39.500'), (1, '43.470')] [2023-10-07 21:11:37,775][67838] Updated weights for policy 0, policy_version 36402 (0.0008) [2023-10-07 21:11:38,149][67838] Updated weights for policy 0, policy_version 36412 (0.0007) [2023-10-07 21:11:38,566][67871] Updated weights for policy 1, policy_version 36450 (0.0008) [2023-10-07 21:11:38,978][67871] Updated weights for policy 1, policy_version 36460 (0.0007) [2023-10-07 21:11:39,334][67871] Updated weights for policy 1, policy_version 36470 (0.0009) [2023-10-07 21:11:39,704][67871] Updated weights for policy 1, policy_version 36480 (0.0009) [2023-10-07 21:11:42,278][67838] Updated weights for policy 0, policy_version 36422 (0.0011) [2023-10-07 21:11:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74645504. Throughput: 0: 1674.7, 1: 1658.6. Samples: 18669186. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 21:11:42,477][66916] Avg episode reward: [(0, '37.840'), (1, '46.630')] [2023-10-07 21:11:42,651][67838] Updated weights for policy 0, policy_version 36432 (0.0011) [2023-10-07 21:11:43,014][67838] Updated weights for policy 0, policy_version 36442 (0.0009) [2023-10-07 21:11:43,635][67871] Updated weights for policy 1, policy_version 36490 (0.0008) [2023-10-07 21:11:43,998][67871] Updated weights for policy 1, policy_version 36500 (0.0010) [2023-10-07 21:11:44,374][67871] Updated weights for policy 1, policy_version 36510 (0.0008) [2023-10-07 21:11:46,929][67838] Updated weights for policy 0, policy_version 36452 (0.0008) [2023-10-07 21:11:47,304][67838] Updated weights for policy 0, policy_version 36462 (0.0009) [2023-10-07 21:11:47,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 74711040. Throughput: 0: 1676.3, 1: 1671.6. Samples: 18689820. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 21:11:47,478][66916] Avg episode reward: [(0, '36.350'), (1, '46.220')] [2023-10-07 21:11:47,689][67838] Updated weights for policy 0, policy_version 36472 (0.0010) [2023-10-07 21:11:48,428][67871] Updated weights for policy 1, policy_version 36520 (0.0010) [2023-10-07 21:11:48,800][67871] Updated weights for policy 1, policy_version 36530 (0.0010) [2023-10-07 21:11:49,159][67871] Updated weights for policy 1, policy_version 36540 (0.0008) [2023-10-07 21:11:51,612][67838] Updated weights for policy 0, policy_version 36482 (0.0009) [2023-10-07 21:11:51,989][67838] Updated weights for policy 0, policy_version 36492 (0.0009) [2023-10-07 21:11:52,359][67838] Updated weights for policy 0, policy_version 36502 (0.0010) [2023-10-07 21:11:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74776576. Throughput: 0: 1666.1, 1: 1666.0. Samples: 18709770. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 21:11:52,477][66916] Avg episode reward: [(0, '38.550'), (1, '48.360')] [2023-10-07 21:11:52,743][67838] Updated weights for policy 0, policy_version 36512 (0.0009) [2023-10-07 21:11:53,269][67871] Updated weights for policy 1, policy_version 36550 (0.0009) [2023-10-07 21:11:53,643][67871] Updated weights for policy 1, policy_version 36560 (0.0010) [2023-10-07 21:11:54,021][67871] Updated weights for policy 1, policy_version 36570 (0.0011) [2023-10-07 21:11:56,811][67838] Updated weights for policy 0, policy_version 36522 (0.0007) [2023-10-07 21:11:57,172][67838] Updated weights for policy 0, policy_version 36532 (0.0008) [2023-10-07 21:11:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 74842112. Throughput: 0: 1682.4, 1: 1660.3. Samples: 18719470. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 21:11:57,478][66916] Avg episode reward: [(0, '36.610'), (1, '48.650')] [2023-10-07 21:11:57,549][67838] Updated weights for policy 0, policy_version 36542 (0.0007) [2023-10-07 21:11:58,265][67871] Updated weights for policy 1, policy_version 36580 (0.0010) [2023-10-07 21:11:58,634][67871] Updated weights for policy 1, policy_version 36590 (0.0010) [2023-10-07 21:11:59,006][67871] Updated weights for policy 1, policy_version 36600 (0.0008) [2023-10-07 21:12:01,690][67838] Updated weights for policy 0, policy_version 36552 (0.0009) [2023-10-07 21:12:02,062][67838] Updated weights for policy 0, policy_version 36562 (0.0010) [2023-10-07 21:12:02,427][67838] Updated weights for policy 0, policy_version 36572 (0.0009) [2023-10-07 21:12:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 74907648. Throughput: 0: 1682.2, 1: 1666.3. Samples: 18739892. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 21:12:02,477][66916] Avg episode reward: [(0, '40.280'), (1, '52.380')] [2023-10-07 21:12:03,111][67871] Updated weights for policy 1, policy_version 36610 (0.0007) [2023-10-07 21:12:03,475][67871] Updated weights for policy 1, policy_version 36620 (0.0007) [2023-10-07 21:12:03,850][67871] Updated weights for policy 1, policy_version 36630 (0.0008) [2023-10-07 21:12:04,208][67871] Updated weights for policy 1, policy_version 36640 (0.0010) [2023-10-07 21:12:06,570][67838] Updated weights for policy 0, policy_version 36582 (0.0007) [2023-10-07 21:12:06,939][67838] Updated weights for policy 0, policy_version 36592 (0.0008) [2023-10-07 21:12:07,324][67838] Updated weights for policy 0, policy_version 36602 (0.0010) [2023-10-07 21:12:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 74973184. Throughput: 0: 1667.0, 1: 1670.5. Samples: 18759910. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 21:12:07,478][66916] Avg episode reward: [(0, '39.520'), (1, '51.500')] [2023-10-07 21:12:08,305][67871] Updated weights for policy 1, policy_version 36650 (0.0009) [2023-10-07 21:12:08,679][67871] Updated weights for policy 1, policy_version 36660 (0.0008) [2023-10-07 21:12:09,047][67871] Updated weights for policy 1, policy_version 36670 (0.0008) [2023-10-07 21:12:11,428][67838] Updated weights for policy 0, policy_version 36612 (0.0007) [2023-10-07 21:12:11,798][67838] Updated weights for policy 0, policy_version 36622 (0.0007) [2023-10-07 21:12:12,179][67838] Updated weights for policy 0, policy_version 36632 (0.0009) [2023-10-07 21:12:12,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 75038720. Throughput: 0: 1683.2, 1: 1662.6. Samples: 18769596. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 21:12:12,478][66916] Avg episode reward: [(0, '40.060'), (1, '51.280')] [2023-10-07 21:12:13,308][67871] Updated weights for policy 1, policy_version 36680 (0.0007) [2023-10-07 21:12:13,671][67871] Updated weights for policy 1, policy_version 36690 (0.0009) [2023-10-07 21:12:14,036][67871] Updated weights for policy 1, policy_version 36700 (0.0010) [2023-10-07 21:12:16,370][67838] Updated weights for policy 0, policy_version 36642 (0.0009) [2023-10-07 21:12:16,767][67838] Updated weights for policy 0, policy_version 36652 (0.0007) [2023-10-07 21:12:17,135][67838] Updated weights for policy 0, policy_version 36662 (0.0011) [2023-10-07 21:12:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 75104256. Throughput: 0: 1675.0, 1: 1664.6. Samples: 18789764. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 21:12:17,477][66916] Avg episode reward: [(0, '41.870'), (1, '50.970')] [2023-10-07 21:12:17,509][67838] Updated weights for policy 0, policy_version 36672 (0.0007) [2023-10-07 21:12:18,042][67871] Updated weights for policy 1, policy_version 36710 (0.0009) [2023-10-07 21:12:18,401][67871] Updated weights for policy 1, policy_version 36720 (0.0010) [2023-10-07 21:12:18,765][67871] Updated weights for policy 1, policy_version 36730 (0.0011) [2023-10-07 21:12:21,693][67838] Updated weights for policy 0, policy_version 36682 (0.0010) [2023-10-07 21:12:22,061][67838] Updated weights for policy 0, policy_version 36692 (0.0009) [2023-10-07 21:12:22,437][67838] Updated weights for policy 0, policy_version 36702 (0.0007) [2023-10-07 21:12:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 75169792. Throughput: 0: 1653.3, 1: 1662.9. Samples: 18809434. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 21:12:22,477][66916] Avg episode reward: [(0, '38.770'), (1, '52.890')] [2023-10-07 21:12:22,484][67676] Saving new best policy, reward=52.890! [2023-10-07 21:12:22,911][67871] Updated weights for policy 1, policy_version 36740 (0.0008) [2023-10-07 21:12:23,304][67871] Updated weights for policy 1, policy_version 36750 (0.0008) [2023-10-07 21:12:23,670][67871] Updated weights for policy 1, policy_version 36760 (0.0007) [2023-10-07 21:12:26,587][67838] Updated weights for policy 0, policy_version 36712 (0.0010) [2023-10-07 21:12:26,965][67838] Updated weights for policy 0, policy_version 36722 (0.0008) [2023-10-07 21:12:27,342][67838] Updated weights for policy 0, policy_version 36732 (0.0010) [2023-10-07 21:12:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 75235328. Throughput: 0: 1669.5, 1: 1663.5. Samples: 18819170. Policy #0 lag: (min: 19.0, avg: 23.2, max: 51.0) [2023-10-07 21:12:27,477][66916] Avg episode reward: [(0, '39.440'), (1, '51.910')] [2023-10-07 21:12:27,719][67871] Updated weights for policy 1, policy_version 36770 (0.0009) [2023-10-07 21:12:28,093][67871] Updated weights for policy 1, policy_version 36780 (0.0007) [2023-10-07 21:12:28,462][67871] Updated weights for policy 1, policy_version 36790 (0.0007) [2023-10-07 21:12:28,845][67871] Updated weights for policy 1, policy_version 36800 (0.0008) [2023-10-07 21:12:31,597][67838] Updated weights for policy 0, policy_version 36742 (0.0009) [2023-10-07 21:12:31,969][67838] Updated weights for policy 0, policy_version 36752 (0.0009) [2023-10-07 21:12:32,341][67838] Updated weights for policy 0, policy_version 36762 (0.0009) [2023-10-07 21:12:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 75300864. Throughput: 0: 1663.4, 1: 1663.1. Samples: 18839512. Policy #0 lag: (min: 19.0, avg: 23.2, max: 51.0) [2023-10-07 21:12:32,477][66916] Avg episode reward: [(0, '40.100'), (1, '50.680')] [2023-10-07 21:12:33,005][67871] Updated weights for policy 1, policy_version 36810 (0.0009) [2023-10-07 21:12:33,382][67871] Updated weights for policy 1, policy_version 36820 (0.0010) [2023-10-07 21:12:33,743][67871] Updated weights for policy 1, policy_version 36830 (0.0010) [2023-10-07 21:12:36,453][67838] Updated weights for policy 0, policy_version 36772 (0.0010) [2023-10-07 21:12:36,830][67838] Updated weights for policy 0, policy_version 36782 (0.0008) [2023-10-07 21:12:37,214][67838] Updated weights for policy 0, policy_version 36792 (0.0007) [2023-10-07 21:12:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 75366400. Throughput: 0: 1655.4, 1: 1660.8. Samples: 18859000. Policy #0 lag: (min: 19.0, avg: 23.2, max: 51.0) [2023-10-07 21:12:37,477][66916] Avg episode reward: [(0, '38.190'), (1, '50.450')] [2023-10-07 21:12:37,982][67871] Updated weights for policy 1, policy_version 36840 (0.0007) [2023-10-07 21:12:38,341][67871] Updated weights for policy 1, policy_version 36850 (0.0007) [2023-10-07 21:12:38,721][67871] Updated weights for policy 1, policy_version 36860 (0.0009) [2023-10-07 21:12:41,522][67838] Updated weights for policy 0, policy_version 36802 (0.0008) [2023-10-07 21:12:41,891][67838] Updated weights for policy 0, policy_version 36812 (0.0008) [2023-10-07 21:12:42,262][67838] Updated weights for policy 0, policy_version 36822 (0.0007) [2023-10-07 21:12:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 75431936. Throughput: 0: 1653.7, 1: 1662.9. Samples: 18868720. Policy #0 lag: (min: 19.0, avg: 23.2, max: 51.0) [2023-10-07 21:12:42,477][66916] Avg episode reward: [(0, '39.760'), (1, '47.510')] [2023-10-07 21:12:42,640][67838] Updated weights for policy 0, policy_version 36832 (0.0008) [2023-10-07 21:12:42,956][67871] Updated weights for policy 1, policy_version 36870 (0.0007) [2023-10-07 21:12:43,322][67871] Updated weights for policy 1, policy_version 36880 (0.0009) [2023-10-07 21:12:43,693][67871] Updated weights for policy 1, policy_version 36890 (0.0009) [2023-10-07 21:12:46,829][67838] Updated weights for policy 0, policy_version 36842 (0.0009) [2023-10-07 21:12:47,195][67838] Updated weights for policy 0, policy_version 36852 (0.0012) [2023-10-07 21:12:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 75497472. Throughput: 0: 1651.9, 1: 1663.6. Samples: 18889086. Policy #0 lag: (min: 19.0, avg: 23.2, max: 51.0) [2023-10-07 21:12:47,478][66916] Avg episode reward: [(0, '38.770'), (1, '46.720')] [2023-10-07 21:12:47,577][67838] Updated weights for policy 0, policy_version 36862 (0.0011) [2023-10-07 21:12:47,751][67871] Updated weights for policy 1, policy_version 36900 (0.0008) [2023-10-07 21:12:48,119][67871] Updated weights for policy 1, policy_version 36910 (0.0011) [2023-10-07 21:12:48,494][67871] Updated weights for policy 1, policy_version 36920 (0.0009) [2023-10-07 21:12:51,507][67838] Updated weights for policy 0, policy_version 36872 (0.0007) [2023-10-07 21:12:51,877][67838] Updated weights for policy 0, policy_version 36882 (0.0007) [2023-10-07 21:12:52,253][67838] Updated weights for policy 0, policy_version 36892 (0.0008) [2023-10-07 21:12:52,477][66916] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 75595776. Throughput: 0: 1649.1, 1: 1655.3. Samples: 18908608. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-07 21:12:52,478][66916] Avg episode reward: [(0, '40.680'), (1, '47.310')] [2023-10-07 21:12:52,698][67871] Updated weights for policy 1, policy_version 36930 (0.0007) [2023-10-07 21:12:53,071][67871] Updated weights for policy 1, policy_version 36940 (0.0007) [2023-10-07 21:12:53,429][67871] Updated weights for policy 1, policy_version 36950 (0.0009) [2023-10-07 21:12:53,805][67871] Updated weights for policy 1, policy_version 36960 (0.0008) [2023-10-07 21:12:56,200][67838] Updated weights for policy 0, policy_version 36902 (0.0010) [2023-10-07 21:12:56,575][67838] Updated weights for policy 0, policy_version 36912 (0.0007) [2023-10-07 21:12:56,945][67838] Updated weights for policy 0, policy_version 36922 (0.0007) [2023-10-07 21:12:57,476][66916] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 75661312. Throughput: 0: 1654.0, 1: 1655.0. Samples: 18918500. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-07 21:12:57,477][66916] Avg episode reward: [(0, '41.250'), (1, '49.000')] [2023-10-07 21:12:57,962][67871] Updated weights for policy 1, policy_version 36970 (0.0007) [2023-10-07 21:12:58,334][67871] Updated weights for policy 1, policy_version 36980 (0.0007) [2023-10-07 21:12:58,704][67871] Updated weights for policy 1, policy_version 36990 (0.0008) [2023-10-07 21:13:01,140][67838] Updated weights for policy 0, policy_version 36932 (0.0007) [2023-10-07 21:13:01,542][67838] Updated weights for policy 0, policy_version 36942 (0.0007) [2023-10-07 21:13:01,917][67838] Updated weights for policy 0, policy_version 36952 (0.0007) [2023-10-07 21:13:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 75726848. Throughput: 0: 1654.6, 1: 1657.6. Samples: 18938816. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-07 21:13:02,477][66916] Avg episode reward: [(0, '41.480'), (1, '49.420')] [2023-10-07 21:13:03,018][67871] Updated weights for policy 1, policy_version 37000 (0.0010) [2023-10-07 21:13:03,380][67871] Updated weights for policy 1, policy_version 37010 (0.0011) [2023-10-07 21:13:03,743][67871] Updated weights for policy 1, policy_version 37020 (0.0010) [2023-10-07 21:13:05,970][67838] Updated weights for policy 0, policy_version 36962 (0.0007) [2023-10-07 21:13:06,340][67838] Updated weights for policy 0, policy_version 36972 (0.0009) [2023-10-07 21:13:06,711][67838] Updated weights for policy 0, policy_version 36982 (0.0010) [2023-10-07 21:13:07,090][67838] Updated weights for policy 0, policy_version 36992 (0.0010) [2023-10-07 21:13:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 75792384. Throughput: 0: 1648.4, 1: 1660.8. Samples: 18958350. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-07 21:13:07,477][66916] Avg episode reward: [(0, '40.980'), (1, '50.910')] [2023-10-07 21:13:07,485][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000036992_37879808.pth... [2023-10-07 21:13:07,522][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000035424_36274176.pth [2023-10-07 21:13:07,799][67871] Updated weights for policy 1, policy_version 37030 (0.0011) [2023-10-07 21:13:08,183][67871] Updated weights for policy 1, policy_version 37040 (0.0008) [2023-10-07 21:13:08,547][67871] Updated weights for policy 1, policy_version 37050 (0.0010) [2023-10-07 21:13:08,762][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000037056_37945344.pth... [2023-10-07 21:13:08,799][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000035488_36339712.pth [2023-10-07 21:13:11,349][67838] Updated weights for policy 0, policy_version 37002 (0.0008) [2023-10-07 21:13:11,732][67838] Updated weights for policy 0, policy_version 37012 (0.0010) [2023-10-07 21:13:12,097][67838] Updated weights for policy 0, policy_version 37022 (0.0008) [2023-10-07 21:13:12,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 75857920. Throughput: 0: 1657.8, 1: 1659.6. Samples: 18968454. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-07 21:13:12,478][66916] Avg episode reward: [(0, '38.320'), (1, '50.170')] [2023-10-07 21:13:12,606][67871] Updated weights for policy 1, policy_version 37060 (0.0010) [2023-10-07 21:13:12,974][67871] Updated weights for policy 1, policy_version 37070 (0.0008) [2023-10-07 21:13:13,345][67871] Updated weights for policy 1, policy_version 37080 (0.0009) [2023-10-07 21:13:16,077][67838] Updated weights for policy 0, policy_version 37032 (0.0008) [2023-10-07 21:13:16,458][67838] Updated weights for policy 0, policy_version 37042 (0.0010) [2023-10-07 21:13:16,829][67838] Updated weights for policy 0, policy_version 37052 (0.0010) [2023-10-07 21:13:17,371][67871] Updated weights for policy 1, policy_version 37090 (0.0008) [2023-10-07 21:13:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 75923456. Throughput: 0: 1655.8, 1: 1659.5. Samples: 18988702. Policy #0 lag: (min: 32.0, avg: 53.3, max: 56.0) [2023-10-07 21:13:17,478][66916] Avg episode reward: [(0, '37.600'), (1, '48.770')] [2023-10-07 21:13:17,741][67871] Updated weights for policy 1, policy_version 37100 (0.0007) [2023-10-07 21:13:18,103][67871] Updated weights for policy 1, policy_version 37110 (0.0008) [2023-10-07 21:13:18,474][67871] Updated weights for policy 1, policy_version 37120 (0.0009) [2023-10-07 21:13:20,842][67838] Updated weights for policy 0, policy_version 37062 (0.0009) [2023-10-07 21:13:21,216][67838] Updated weights for policy 0, policy_version 37072 (0.0008) [2023-10-07 21:13:21,594][67838] Updated weights for policy 0, policy_version 37082 (0.0008) [2023-10-07 21:13:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 75988992. Throughput: 0: 1655.1, 1: 1664.1. Samples: 19008364. Policy #0 lag: (min: 32.0, avg: 53.3, max: 56.0) [2023-10-07 21:13:22,477][66916] Avg episode reward: [(0, '37.550'), (1, '49.940')] [2023-10-07 21:13:22,579][67871] Updated weights for policy 1, policy_version 37130 (0.0008) [2023-10-07 21:13:22,947][67871] Updated weights for policy 1, policy_version 37140 (0.0008) [2023-10-07 21:13:23,320][67871] Updated weights for policy 1, policy_version 37150 (0.0009) [2023-10-07 21:13:25,540][67838] Updated weights for policy 0, policy_version 37092 (0.0008) [2023-10-07 21:13:25,909][67838] Updated weights for policy 0, policy_version 37102 (0.0008) [2023-10-07 21:13:26,281][67838] Updated weights for policy 0, policy_version 37112 (0.0008) [2023-10-07 21:13:27,410][67871] Updated weights for policy 1, policy_version 37160 (0.0009) [2023-10-07 21:13:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 76054528. Throughput: 0: 1675.4, 1: 1663.2. Samples: 19018956. Policy #0 lag: (min: 32.0, avg: 53.3, max: 56.0) [2023-10-07 21:13:27,477][66916] Avg episode reward: [(0, '36.940'), (1, '50.220')] [2023-10-07 21:13:27,775][67871] Updated weights for policy 1, policy_version 37170 (0.0011) [2023-10-07 21:13:28,148][67871] Updated weights for policy 1, policy_version 37180 (0.0009) [2023-10-07 21:13:30,258][67838] Updated weights for policy 0, policy_version 37122 (0.0008) [2023-10-07 21:13:30,632][67838] Updated weights for policy 0, policy_version 37132 (0.0010) [2023-10-07 21:13:31,006][67838] Updated weights for policy 0, policy_version 37142 (0.0009) [2023-10-07 21:13:31,376][67838] Updated weights for policy 0, policy_version 37152 (0.0007) [2023-10-07 21:13:32,325][67871] Updated weights for policy 1, policy_version 37190 (0.0011) [2023-10-07 21:13:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 76120064. Throughput: 0: 1662.8, 1: 1664.2. Samples: 19038800. Policy #0 lag: (min: 32.0, avg: 53.3, max: 56.0) [2023-10-07 21:13:32,477][66916] Avg episode reward: [(0, '36.270'), (1, '52.850')] [2023-10-07 21:13:32,680][67871] Updated weights for policy 1, policy_version 37200 (0.0009) [2023-10-07 21:13:33,050][67871] Updated weights for policy 1, policy_version 37210 (0.0009) [2023-10-07 21:13:35,497][67838] Updated weights for policy 0, policy_version 37162 (0.0008) [2023-10-07 21:13:35,865][67838] Updated weights for policy 0, policy_version 37172 (0.0008) [2023-10-07 21:13:36,232][67838] Updated weights for policy 0, policy_version 37182 (0.0008) [2023-10-07 21:13:36,966][67871] Updated weights for policy 1, policy_version 37220 (0.0008) [2023-10-07 21:13:37,329][67871] Updated weights for policy 1, policy_version 37230 (0.0011) [2023-10-07 21:13:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 76185600. Throughput: 0: 1669.7, 1: 1673.6. Samples: 19059054. Policy #0 lag: (min: 32.0, avg: 53.3, max: 56.0) [2023-10-07 21:13:37,478][66916] Avg episode reward: [(0, '35.140'), (1, '53.640')] [2023-10-07 21:13:37,700][67871] Updated weights for policy 1, policy_version 37240 (0.0007) [2023-10-07 21:13:37,982][67676] Saving new best policy, reward=53.640! [2023-10-07 21:13:40,263][67838] Updated weights for policy 0, policy_version 37192 (0.0009) [2023-10-07 21:13:40,640][67838] Updated weights for policy 0, policy_version 37202 (0.0009) [2023-10-07 21:13:41,012][67838] Updated weights for policy 0, policy_version 37212 (0.0009) [2023-10-07 21:13:41,700][67871] Updated weights for policy 1, policy_version 37250 (0.0007) [2023-10-07 21:13:42,061][67871] Updated weights for policy 1, policy_version 37260 (0.0007) [2023-10-07 21:13:42,433][67871] Updated weights for policy 1, policy_version 37270 (0.0007) [2023-10-07 21:13:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 76251136. Throughput: 0: 1673.5, 1: 1677.0. Samples: 19069272. Policy #0 lag: (min: 32.0, avg: 53.3, max: 56.0) [2023-10-07 21:13:42,478][66916] Avg episode reward: [(0, '37.920'), (1, '50.970')] [2023-10-07 21:13:42,795][67871] Updated weights for policy 1, policy_version 37280 (0.0007) [2023-10-07 21:13:44,954][67838] Updated weights for policy 0, policy_version 37222 (0.0007) [2023-10-07 21:13:45,327][67838] Updated weights for policy 0, policy_version 37232 (0.0008) [2023-10-07 21:13:45,706][67838] Updated weights for policy 0, policy_version 37242 (0.0011) [2023-10-07 21:13:46,932][67871] Updated weights for policy 1, policy_version 37290 (0.0007) [2023-10-07 21:13:47,296][67871] Updated weights for policy 1, policy_version 37300 (0.0008) [2023-10-07 21:13:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 76316672. Throughput: 0: 1652.9, 1: 1681.4. Samples: 19088860. Policy #0 lag: (min: 1.0, avg: 11.0, max: 33.0) [2023-10-07 21:13:47,477][66916] Avg episode reward: [(0, '37.490'), (1, '51.520')] [2023-10-07 21:13:47,659][67871] Updated weights for policy 1, policy_version 37310 (0.0008) [2023-10-07 21:13:50,134][67838] Updated weights for policy 0, policy_version 37252 (0.0009) [2023-10-07 21:13:50,527][67838] Updated weights for policy 0, policy_version 37262 (0.0011) [2023-10-07 21:13:50,897][67838] Updated weights for policy 0, policy_version 37272 (0.0008) [2023-10-07 21:13:51,833][67871] Updated weights for policy 1, policy_version 37320 (0.0008) [2023-10-07 21:13:52,206][67871] Updated weights for policy 1, policy_version 37330 (0.0007) [2023-10-07 21:13:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76382208. Throughput: 0: 1675.1, 1: 1666.1. Samples: 19108706. Policy #0 lag: (min: 1.0, avg: 11.0, max: 33.0) [2023-10-07 21:13:52,477][66916] Avg episode reward: [(0, '37.450'), (1, '52.390')] [2023-10-07 21:13:52,581][67871] Updated weights for policy 1, policy_version 37340 (0.0007) [2023-10-07 21:13:54,810][67838] Updated weights for policy 0, policy_version 37282 (0.0008) [2023-10-07 21:13:55,181][67838] Updated weights for policy 0, policy_version 37292 (0.0010) [2023-10-07 21:13:55,554][67838] Updated weights for policy 0, policy_version 37302 (0.0008) [2023-10-07 21:13:55,933][67838] Updated weights for policy 0, policy_version 37312 (0.0010) [2023-10-07 21:13:56,701][67871] Updated weights for policy 1, policy_version 37350 (0.0010) [2023-10-07 21:13:57,087][67871] Updated weights for policy 1, policy_version 37360 (0.0007) [2023-10-07 21:13:57,459][67871] Updated weights for policy 1, policy_version 37370 (0.0007) [2023-10-07 21:13:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76447744. Throughput: 0: 1670.0, 1: 1673.2. Samples: 19118898. Policy #0 lag: (min: 1.0, avg: 11.0, max: 33.0) [2023-10-07 21:13:57,477][66916] Avg episode reward: [(0, '39.230'), (1, '51.850')] [2023-10-07 21:14:00,005][67838] Updated weights for policy 0, policy_version 37322 (0.0007) [2023-10-07 21:14:00,380][67838] Updated weights for policy 0, policy_version 37332 (0.0008) [2023-10-07 21:14:00,752][67838] Updated weights for policy 0, policy_version 37342 (0.0010) [2023-10-07 21:14:01,536][67871] Updated weights for policy 1, policy_version 37380 (0.0009) [2023-10-07 21:14:01,908][67871] Updated weights for policy 1, policy_version 37390 (0.0008) [2023-10-07 21:14:02,274][67871] Updated weights for policy 1, policy_version 37400 (0.0009) [2023-10-07 21:14:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76513280. Throughput: 0: 1653.5, 1: 1670.9. Samples: 19138300. Policy #0 lag: (min: 1.0, avg: 11.0, max: 33.0) [2023-10-07 21:14:02,477][66916] Avg episode reward: [(0, '39.270'), (1, '51.380')] [2023-10-07 21:14:04,881][67838] Updated weights for policy 0, policy_version 37352 (0.0008) [2023-10-07 21:14:05,257][67838] Updated weights for policy 0, policy_version 37362 (0.0007) [2023-10-07 21:14:05,625][67838] Updated weights for policy 0, policy_version 37372 (0.0010) [2023-10-07 21:14:06,403][67871] Updated weights for policy 1, policy_version 37410 (0.0007) [2023-10-07 21:14:06,774][67871] Updated weights for policy 1, policy_version 37420 (0.0007) [2023-10-07 21:14:07,146][67871] Updated weights for policy 1, policy_version 37430 (0.0008) [2023-10-07 21:14:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76578816. Throughput: 0: 1672.6, 1: 1657.7. Samples: 19158228. Policy #0 lag: (min: 1.0, avg: 11.0, max: 33.0) [2023-10-07 21:14:07,477][66916] Avg episode reward: [(0, '40.680'), (1, '54.370')] [2023-10-07 21:14:07,503][67676] Saving new best policy, reward=54.370! [2023-10-07 21:14:07,506][67871] Updated weights for policy 1, policy_version 37440 (0.0009) [2023-10-07 21:14:09,922][67838] Updated weights for policy 0, policy_version 37382 (0.0011) [2023-10-07 21:14:10,299][67838] Updated weights for policy 0, policy_version 37392 (0.0010) [2023-10-07 21:14:10,670][67838] Updated weights for policy 0, policy_version 37402 (0.0010) [2023-10-07 21:14:11,743][67871] Updated weights for policy 1, policy_version 37450 (0.0009) [2023-10-07 21:14:12,114][67871] Updated weights for policy 1, policy_version 37460 (0.0008) [2023-10-07 21:14:12,469][67871] Updated weights for policy 1, policy_version 37470 (0.0011) [2023-10-07 21:14:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 76644352. Throughput: 0: 1660.0, 1: 1664.6. Samples: 19168566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:14:12,477][66916] Avg episode reward: [(0, '37.820'), (1, '53.470')] [2023-10-07 21:14:14,731][67838] Updated weights for policy 0, policy_version 37412 (0.0009) [2023-10-07 21:14:15,106][67838] Updated weights for policy 0, policy_version 37422 (0.0008) [2023-10-07 21:14:15,473][67838] Updated weights for policy 0, policy_version 37432 (0.0008) [2023-10-07 21:14:16,563][67871] Updated weights for policy 1, policy_version 37480 (0.0009) [2023-10-07 21:14:16,938][67871] Updated weights for policy 1, policy_version 37490 (0.0007) [2023-10-07 21:14:17,311][67871] Updated weights for policy 1, policy_version 37500 (0.0007) [2023-10-07 21:14:17,476][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 76742656. Throughput: 0: 1658.0, 1: 1669.3. Samples: 19188530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:14:17,477][66916] Avg episode reward: [(0, '44.120'), (1, '50.090')] [2023-10-07 21:14:19,722][67838] Updated weights for policy 0, policy_version 37442 (0.0010) [2023-10-07 21:14:20,085][67838] Updated weights for policy 0, policy_version 37452 (0.0011) [2023-10-07 21:14:20,463][67838] Updated weights for policy 0, policy_version 37462 (0.0008) [2023-10-07 21:14:20,828][67838] Updated weights for policy 0, policy_version 37472 (0.0009) [2023-10-07 21:14:21,352][67871] Updated weights for policy 1, policy_version 37510 (0.0009) [2023-10-07 21:14:21,724][67871] Updated weights for policy 1, policy_version 37520 (0.0009) [2023-10-07 21:14:22,097][67871] Updated weights for policy 1, policy_version 37530 (0.0007) [2023-10-07 21:14:22,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 76808192. Throughput: 0: 1665.1, 1: 1649.7. Samples: 19208218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:14:22,478][66916] Avg episode reward: [(0, '41.250'), (1, '49.360')] [2023-10-07 21:14:24,916][67838] Updated weights for policy 0, policy_version 37482 (0.0007) [2023-10-07 21:14:25,291][67838] Updated weights for policy 0, policy_version 37492 (0.0008) [2023-10-07 21:14:25,669][67838] Updated weights for policy 0, policy_version 37502 (0.0008) [2023-10-07 21:14:26,257][67871] Updated weights for policy 1, policy_version 37540 (0.0008) [2023-10-07 21:14:26,627][67871] Updated weights for policy 1, policy_version 37550 (0.0007) [2023-10-07 21:14:27,005][67871] Updated weights for policy 1, policy_version 37560 (0.0008) [2023-10-07 21:14:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 76873728. Throughput: 0: 1658.9, 1: 1659.6. Samples: 19218600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:14:27,477][66916] Avg episode reward: [(0, '42.320'), (1, '48.490')] [2023-10-07 21:14:29,579][67838] Updated weights for policy 0, policy_version 37512 (0.0007) [2023-10-07 21:14:29,949][67838] Updated weights for policy 0, policy_version 37522 (0.0008) [2023-10-07 21:14:30,330][67838] Updated weights for policy 0, policy_version 37532 (0.0008) [2023-10-07 21:14:31,206][67871] Updated weights for policy 1, policy_version 37570 (0.0009) [2023-10-07 21:14:31,571][67871] Updated weights for policy 1, policy_version 37580 (0.0008) [2023-10-07 21:14:31,926][67871] Updated weights for policy 1, policy_version 37590 (0.0011) [2023-10-07 21:14:32,288][67871] Updated weights for policy 1, policy_version 37600 (0.0009) [2023-10-07 21:14:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 76939264. Throughput: 0: 1665.2, 1: 1653.8. Samples: 19238216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:14:32,477][66916] Avg episode reward: [(0, '41.540'), (1, '48.400')] [2023-10-07 21:14:34,768][67838] Updated weights for policy 0, policy_version 37542 (0.0008) [2023-10-07 21:14:35,152][67838] Updated weights for policy 0, policy_version 37552 (0.0009) [2023-10-07 21:14:35,517][67838] Updated weights for policy 0, policy_version 37562 (0.0009) [2023-10-07 21:14:36,504][67871] Updated weights for policy 1, policy_version 37610 (0.0008) [2023-10-07 21:14:36,877][67871] Updated weights for policy 1, policy_version 37620 (0.0009) [2023-10-07 21:14:37,253][67871] Updated weights for policy 1, policy_version 37630 (0.0009) [2023-10-07 21:14:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 77004800. Throughput: 0: 1667.1, 1: 1646.4. Samples: 19257812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:14:37,478][66916] Avg episode reward: [(0, '41.400'), (1, '48.190')] [2023-10-07 21:14:39,414][67838] Updated weights for policy 0, policy_version 37572 (0.0008) [2023-10-07 21:14:39,794][67838] Updated weights for policy 0, policy_version 37582 (0.0008) [2023-10-07 21:14:40,159][67838] Updated weights for policy 0, policy_version 37592 (0.0007) [2023-10-07 21:14:41,414][67871] Updated weights for policy 1, policy_version 37640 (0.0010) [2023-10-07 21:14:41,793][67871] Updated weights for policy 1, policy_version 37650 (0.0009) [2023-10-07 21:14:42,162][67871] Updated weights for policy 1, policy_version 37660 (0.0009) [2023-10-07 21:14:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 77070336. Throughput: 0: 1653.3, 1: 1657.1. Samples: 19267868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:14:42,478][66916] Avg episode reward: [(0, '41.600'), (1, '47.950')] [2023-10-07 21:14:44,188][67838] Updated weights for policy 0, policy_version 37602 (0.0008) [2023-10-07 21:14:44,549][67838] Updated weights for policy 0, policy_version 37612 (0.0009) [2023-10-07 21:14:44,919][67838] Updated weights for policy 0, policy_version 37622 (0.0009) [2023-10-07 21:14:45,291][67838] Updated weights for policy 0, policy_version 37632 (0.0008) [2023-10-07 21:14:46,300][67871] Updated weights for policy 1, policy_version 37670 (0.0008) [2023-10-07 21:14:46,659][67871] Updated weights for policy 1, policy_version 37680 (0.0007) [2023-10-07 21:14:47,034][67871] Updated weights for policy 1, policy_version 37690 (0.0007) [2023-10-07 21:14:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 77135872. Throughput: 0: 1664.8, 1: 1654.4. Samples: 19287668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:14:47,477][66916] Avg episode reward: [(0, '40.710'), (1, '49.080')] [2023-10-07 21:14:49,510][67838] Updated weights for policy 0, policy_version 37642 (0.0008) [2023-10-07 21:14:49,884][67838] Updated weights for policy 0, policy_version 37652 (0.0008) [2023-10-07 21:14:50,250][67838] Updated weights for policy 0, policy_version 37662 (0.0009) [2023-10-07 21:14:51,354][67871] Updated weights for policy 1, policy_version 37700 (0.0009) [2023-10-07 21:14:51,720][67871] Updated weights for policy 1, policy_version 37710 (0.0008) [2023-10-07 21:14:52,091][67871] Updated weights for policy 1, policy_version 37720 (0.0007) [2023-10-07 21:14:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 77201408. Throughput: 0: 1662.3, 1: 1646.7. Samples: 19307136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:14:52,478][66916] Avg episode reward: [(0, '41.030'), (1, '49.350')] [2023-10-07 21:14:54,468][67838] Updated weights for policy 0, policy_version 37672 (0.0009) [2023-10-07 21:14:54,844][67838] Updated weights for policy 0, policy_version 37682 (0.0008) [2023-10-07 21:14:55,224][67838] Updated weights for policy 0, policy_version 37692 (0.0010) [2023-10-07 21:14:56,109][67871] Updated weights for policy 1, policy_version 37730 (0.0008) [2023-10-07 21:14:56,481][67871] Updated weights for policy 1, policy_version 37740 (0.0008) [2023-10-07 21:14:56,848][67871] Updated weights for policy 1, policy_version 37750 (0.0007) [2023-10-07 21:14:57,213][67871] Updated weights for policy 1, policy_version 37760 (0.0008) [2023-10-07 21:14:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 77266944. Throughput: 0: 1645.6, 1: 1659.6. Samples: 19317296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:14:57,477][66916] Avg episode reward: [(0, '41.880'), (1, '50.730')] [2023-10-07 21:14:59,263][67838] Updated weights for policy 0, policy_version 37702 (0.0008) [2023-10-07 21:14:59,623][67838] Updated weights for policy 0, policy_version 37712 (0.0010) [2023-10-07 21:15:00,008][67838] Updated weights for policy 0, policy_version 37722 (0.0009) [2023-10-07 21:15:01,281][67871] Updated weights for policy 1, policy_version 37770 (0.0008) [2023-10-07 21:15:01,638][67871] Updated weights for policy 1, policy_version 37780 (0.0009) [2023-10-07 21:15:02,008][67871] Updated weights for policy 1, policy_version 37790 (0.0009) [2023-10-07 21:15:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 77332480. Throughput: 0: 1652.5, 1: 1652.0. Samples: 19337230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:15:02,478][66916] Avg episode reward: [(0, '41.460'), (1, '51.160')] [2023-10-07 21:15:04,154][67838] Updated weights for policy 0, policy_version 37732 (0.0007) [2023-10-07 21:15:04,528][67838] Updated weights for policy 0, policy_version 37742 (0.0008) [2023-10-07 21:15:04,897][67838] Updated weights for policy 0, policy_version 37752 (0.0007) [2023-10-07 21:15:06,226][67871] Updated weights for policy 1, policy_version 37800 (0.0007) [2023-10-07 21:15:06,591][67871] Updated weights for policy 1, policy_version 37810 (0.0008) [2023-10-07 21:15:06,959][67871] Updated weights for policy 1, policy_version 37820 (0.0007) [2023-10-07 21:15:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 77398016. Throughput: 0: 1661.3, 1: 1643.7. Samples: 19356942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:15:07,477][66916] Avg episode reward: [(0, '40.490'), (1, '47.080')] [2023-10-07 21:15:07,485][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000037824_38731776.pth... [2023-10-07 21:15:07,485][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000037760_38666240.pth... [2023-10-07 21:15:07,515][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000036256_37126144.pth [2023-10-07 21:15:07,524][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000036192_37060608.pth [2023-10-07 21:15:08,918][67838] Updated weights for policy 0, policy_version 37762 (0.0007) [2023-10-07 21:15:09,291][67838] Updated weights for policy 0, policy_version 37772 (0.0007) [2023-10-07 21:15:09,675][67838] Updated weights for policy 0, policy_version 37782 (0.0007) [2023-10-07 21:15:10,042][67838] Updated weights for policy 0, policy_version 37792 (0.0009) [2023-10-07 21:15:11,004][67871] Updated weights for policy 1, policy_version 37830 (0.0009) [2023-10-07 21:15:11,372][67871] Updated weights for policy 1, policy_version 37840 (0.0010) [2023-10-07 21:15:11,733][67871] Updated weights for policy 1, policy_version 37850 (0.0010) [2023-10-07 21:15:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 77463552. Throughput: 0: 1644.7, 1: 1653.0. Samples: 19366996. Policy #0 lag: (min: 26.0, avg: 28.9, max: 58.0) [2023-10-07 21:15:12,478][66916] Avg episode reward: [(0, '38.080'), (1, '45.890')] [2023-10-07 21:15:14,239][67838] Updated weights for policy 0, policy_version 37802 (0.0009) [2023-10-07 21:15:14,620][67838] Updated weights for policy 0, policy_version 37812 (0.0008) [2023-10-07 21:15:14,999][67838] Updated weights for policy 0, policy_version 37822 (0.0009) [2023-10-07 21:15:15,997][67871] Updated weights for policy 1, policy_version 37860 (0.0008) [2023-10-07 21:15:16,366][67871] Updated weights for policy 1, policy_version 37870 (0.0007) [2023-10-07 21:15:16,726][67871] Updated weights for policy 1, policy_version 37880 (0.0008) [2023-10-07 21:15:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 77529088. Throughput: 0: 1661.8, 1: 1654.1. Samples: 19387432. Policy #0 lag: (min: 26.0, avg: 28.9, max: 58.0) [2023-10-07 21:15:17,478][66916] Avg episode reward: [(0, '35.080'), (1, '44.910')] [2023-10-07 21:15:19,024][67838] Updated weights for policy 0, policy_version 37832 (0.0009) [2023-10-07 21:15:19,393][67838] Updated weights for policy 0, policy_version 37842 (0.0008) [2023-10-07 21:15:19,757][67838] Updated weights for policy 0, policy_version 37852 (0.0007) [2023-10-07 21:15:20,831][67871] Updated weights for policy 1, policy_version 37890 (0.0009) [2023-10-07 21:15:21,192][67871] Updated weights for policy 1, policy_version 37900 (0.0009) [2023-10-07 21:15:21,555][67871] Updated weights for policy 1, policy_version 37910 (0.0007) [2023-10-07 21:15:21,929][67871] Updated weights for policy 1, policy_version 37920 (0.0008) [2023-10-07 21:15:22,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 77594624. Throughput: 0: 1666.1, 1: 1648.2. Samples: 19406956. Policy #0 lag: (min: 26.0, avg: 28.9, max: 58.0) [2023-10-07 21:15:22,478][66916] Avg episode reward: [(0, '37.370'), (1, '44.060')] [2023-10-07 21:15:23,860][67838] Updated weights for policy 0, policy_version 37862 (0.0009) [2023-10-07 21:15:24,230][67838] Updated weights for policy 0, policy_version 37872 (0.0010) [2023-10-07 21:15:24,607][67838] Updated weights for policy 0, policy_version 37882 (0.0009) [2023-10-07 21:15:26,030][67871] Updated weights for policy 1, policy_version 37930 (0.0010) [2023-10-07 21:15:26,398][67871] Updated weights for policy 1, policy_version 37940 (0.0010) [2023-10-07 21:15:26,771][67871] Updated weights for policy 1, policy_version 37950 (0.0007) [2023-10-07 21:15:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 77660160. Throughput: 0: 1658.6, 1: 1659.0. Samples: 19417158. Policy #0 lag: (min: 26.0, avg: 28.9, max: 58.0) [2023-10-07 21:15:27,477][66916] Avg episode reward: [(0, '37.990'), (1, '46.210')] [2023-10-07 21:15:28,520][67838] Updated weights for policy 0, policy_version 37892 (0.0007) [2023-10-07 21:15:28,890][67838] Updated weights for policy 0, policy_version 37902 (0.0010) [2023-10-07 21:15:29,274][67838] Updated weights for policy 0, policy_version 37912 (0.0011) [2023-10-07 21:15:30,926][67871] Updated weights for policy 1, policy_version 37960 (0.0010) [2023-10-07 21:15:31,283][67871] Updated weights for policy 1, policy_version 37970 (0.0009) [2023-10-07 21:15:31,653][67871] Updated weights for policy 1, policy_version 37980 (0.0011) [2023-10-07 21:15:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 77725696. Throughput: 0: 1671.6, 1: 1653.6. Samples: 19437300. Policy #0 lag: (min: 26.0, avg: 28.9, max: 58.0) [2023-10-07 21:15:32,477][66916] Avg episode reward: [(0, '37.810'), (1, '47.990')] [2023-10-07 21:15:33,375][67838] Updated weights for policy 0, policy_version 37922 (0.0010) [2023-10-07 21:15:33,748][67838] Updated weights for policy 0, policy_version 37932 (0.0009) [2023-10-07 21:15:34,124][67838] Updated weights for policy 0, policy_version 37942 (0.0007) [2023-10-07 21:15:34,492][67838] Updated weights for policy 0, policy_version 37952 (0.0008) [2023-10-07 21:15:35,726][67871] Updated weights for policy 1, policy_version 37990 (0.0010) [2023-10-07 21:15:36,094][67871] Updated weights for policy 1, policy_version 38000 (0.0010) [2023-10-07 21:15:36,469][67871] Updated weights for policy 1, policy_version 38010 (0.0008) [2023-10-07 21:15:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 77791232. Throughput: 0: 1674.7, 1: 1656.0. Samples: 19457016. Policy #0 lag: (min: 26.0, avg: 28.9, max: 58.0) [2023-10-07 21:15:37,478][66916] Avg episode reward: [(0, '38.960'), (1, '45.390')] [2023-10-07 21:15:38,436][67838] Updated weights for policy 0, policy_version 37962 (0.0009) [2023-10-07 21:15:38,825][67838] Updated weights for policy 0, policy_version 37972 (0.0008) [2023-10-07 21:15:39,196][67838] Updated weights for policy 0, policy_version 37982 (0.0008) [2023-10-07 21:15:40,635][67871] Updated weights for policy 1, policy_version 38020 (0.0008) [2023-10-07 21:15:40,999][67871] Updated weights for policy 1, policy_version 38030 (0.0009) [2023-10-07 21:15:41,355][67871] Updated weights for policy 1, policy_version 38040 (0.0009) [2023-10-07 21:15:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 77856768. Throughput: 0: 1674.0, 1: 1661.6. Samples: 19467400. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-07 21:15:42,477][66916] Avg episode reward: [(0, '38.830'), (1, '47.500')] [2023-10-07 21:15:43,433][67838] Updated weights for policy 0, policy_version 37992 (0.0008) [2023-10-07 21:15:43,805][67838] Updated weights for policy 0, policy_version 38002 (0.0008) [2023-10-07 21:15:44,182][67838] Updated weights for policy 0, policy_version 38012 (0.0007) [2023-10-07 21:15:45,510][67871] Updated weights for policy 1, policy_version 38050 (0.0009) [2023-10-07 21:15:45,869][67871] Updated weights for policy 1, policy_version 38060 (0.0009) [2023-10-07 21:15:46,231][67871] Updated weights for policy 1, policy_version 38070 (0.0007) [2023-10-07 21:15:46,608][67871] Updated weights for policy 1, policy_version 38080 (0.0010) [2023-10-07 21:15:47,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 77922304. Throughput: 0: 1687.1, 1: 1654.8. Samples: 19487614. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-07 21:15:47,478][66916] Avg episode reward: [(0, '38.390'), (1, '45.780')] [2023-10-07 21:15:48,323][67838] Updated weights for policy 0, policy_version 38022 (0.0008) [2023-10-07 21:15:48,695][67838] Updated weights for policy 0, policy_version 38032 (0.0008) [2023-10-07 21:15:49,068][67838] Updated weights for policy 0, policy_version 38042 (0.0010) [2023-10-07 21:15:50,664][67871] Updated weights for policy 1, policy_version 38090 (0.0008) [2023-10-07 21:15:51,027][67871] Updated weights for policy 1, policy_version 38100 (0.0010) [2023-10-07 21:15:51,402][67871] Updated weights for policy 1, policy_version 38110 (0.0009) [2023-10-07 21:15:52,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 77987840. Throughput: 0: 1681.0, 1: 1658.3. Samples: 19507208. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-07 21:15:52,478][66916] Avg episode reward: [(0, '39.100'), (1, '45.890')] [2023-10-07 21:15:53,173][67838] Updated weights for policy 0, policy_version 38052 (0.0008) [2023-10-07 21:15:53,555][67838] Updated weights for policy 0, policy_version 38062 (0.0007) [2023-10-07 21:15:53,934][67838] Updated weights for policy 0, policy_version 38072 (0.0009) [2023-10-07 21:15:55,562][67871] Updated weights for policy 1, policy_version 38120 (0.0008) [2023-10-07 21:15:55,926][67871] Updated weights for policy 1, policy_version 38130 (0.0007) [2023-10-07 21:15:56,298][67871] Updated weights for policy 1, policy_version 38140 (0.0008) [2023-10-07 21:15:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78053376. Throughput: 0: 1678.5, 1: 1662.4. Samples: 19517336. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-07 21:15:57,477][66916] Avg episode reward: [(0, '36.090'), (1, '47.660')] [2023-10-07 21:15:58,049][67838] Updated weights for policy 0, policy_version 38082 (0.0009) [2023-10-07 21:15:58,424][67838] Updated weights for policy 0, policy_version 38092 (0.0010) [2023-10-07 21:15:58,796][67838] Updated weights for policy 0, policy_version 38102 (0.0008) [2023-10-07 21:15:59,176][67838] Updated weights for policy 0, policy_version 38112 (0.0007) [2023-10-07 21:16:00,283][67871] Updated weights for policy 1, policy_version 38150 (0.0010) [2023-10-07 21:16:00,649][67871] Updated weights for policy 1, policy_version 38160 (0.0009) [2023-10-07 21:16:01,027][67871] Updated weights for policy 1, policy_version 38170 (0.0007) [2023-10-07 21:16:02,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78118912. Throughput: 0: 1676.3, 1: 1647.4. Samples: 19536998. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-07 21:16:02,478][66916] Avg episode reward: [(0, '39.020'), (1, '50.000')] [2023-10-07 21:16:03,161][67838] Updated weights for policy 0, policy_version 38122 (0.0008) [2023-10-07 21:16:03,535][67838] Updated weights for policy 0, policy_version 38132 (0.0009) [2023-10-07 21:16:03,910][67838] Updated weights for policy 0, policy_version 38142 (0.0009) [2023-10-07 21:16:05,047][67871] Updated weights for policy 1, policy_version 38180 (0.0007) [2023-10-07 21:16:05,425][67871] Updated weights for policy 1, policy_version 38190 (0.0010) [2023-10-07 21:16:05,780][67871] Updated weights for policy 1, policy_version 38200 (0.0009) [2023-10-07 21:16:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 78184448. Throughput: 0: 1674.4, 1: 1662.4. Samples: 19557110. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-07 21:16:07,478][66916] Avg episode reward: [(0, '38.650'), (1, '50.020')] [2023-10-07 21:16:08,015][67838] Updated weights for policy 0, policy_version 38152 (0.0008) [2023-10-07 21:16:08,387][67838] Updated weights for policy 0, policy_version 38162 (0.0010) [2023-10-07 21:16:08,750][67838] Updated weights for policy 0, policy_version 38172 (0.0007) [2023-10-07 21:16:09,943][67871] Updated weights for policy 1, policy_version 38210 (0.0010) [2023-10-07 21:16:10,313][67871] Updated weights for policy 1, policy_version 38220 (0.0010) [2023-10-07 21:16:10,678][67871] Updated weights for policy 1, policy_version 38230 (0.0007) [2023-10-07 21:16:11,050][67871] Updated weights for policy 1, policy_version 38240 (0.0007) [2023-10-07 21:16:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78249984. Throughput: 0: 1675.7, 1: 1665.7. Samples: 19567522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:16:12,478][66916] Avg episode reward: [(0, '38.580'), (1, '48.990')] [2023-10-07 21:16:12,856][67838] Updated weights for policy 0, policy_version 38182 (0.0009) [2023-10-07 21:16:13,224][67838] Updated weights for policy 0, policy_version 38192 (0.0009) [2023-10-07 21:16:13,603][67838] Updated weights for policy 0, policy_version 38202 (0.0009) [2023-10-07 21:16:15,298][67871] Updated weights for policy 1, policy_version 38250 (0.0008) [2023-10-07 21:16:15,666][67871] Updated weights for policy 1, policy_version 38260 (0.0009) [2023-10-07 21:16:16,036][67871] Updated weights for policy 1, policy_version 38270 (0.0008) [2023-10-07 21:16:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78315520. Throughput: 0: 1672.0, 1: 1650.4. Samples: 19586812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:16:17,477][66916] Avg episode reward: [(0, '38.520'), (1, '47.880')] [2023-10-07 21:16:17,668][67838] Updated weights for policy 0, policy_version 38212 (0.0008) [2023-10-07 21:16:18,029][67838] Updated weights for policy 0, policy_version 38222 (0.0010) [2023-10-07 21:16:18,404][67838] Updated weights for policy 0, policy_version 38232 (0.0012) [2023-10-07 21:16:20,168][67871] Updated weights for policy 1, policy_version 38280 (0.0008) [2023-10-07 21:16:20,526][67871] Updated weights for policy 1, policy_version 38290 (0.0009) [2023-10-07 21:16:20,898][67871] Updated weights for policy 1, policy_version 38300 (0.0008) [2023-10-07 21:16:22,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78381056. Throughput: 0: 1671.8, 1: 1661.9. Samples: 19607034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:16:22,478][66916] Avg episode reward: [(0, '39.260'), (1, '45.590')] [2023-10-07 21:16:22,478][67838] Updated weights for policy 0, policy_version 38242 (0.0008) [2023-10-07 21:16:22,846][67838] Updated weights for policy 0, policy_version 38252 (0.0009) [2023-10-07 21:16:23,212][67838] Updated weights for policy 0, policy_version 38262 (0.0009) [2023-10-07 21:16:23,587][67838] Updated weights for policy 0, policy_version 38272 (0.0007) [2023-10-07 21:16:24,800][67871] Updated weights for policy 1, policy_version 38310 (0.0009) [2023-10-07 21:16:25,166][67871] Updated weights for policy 1, policy_version 38320 (0.0008) [2023-10-07 21:16:25,531][67871] Updated weights for policy 1, policy_version 38330 (0.0009) [2023-10-07 21:16:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78446592. Throughput: 0: 1668.7, 1: 1660.4. Samples: 19617212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:16:27,478][66916] Avg episode reward: [(0, '41.650'), (1, '46.900')] [2023-10-07 21:16:27,799][67838] Updated weights for policy 0, policy_version 38282 (0.0008) [2023-10-07 21:16:28,176][67838] Updated weights for policy 0, policy_version 38292 (0.0009) [2023-10-07 21:16:28,550][67838] Updated weights for policy 0, policy_version 38302 (0.0010) [2023-10-07 21:16:29,656][67871] Updated weights for policy 1, policy_version 38340 (0.0009) [2023-10-07 21:16:30,023][67871] Updated weights for policy 1, policy_version 38350 (0.0008) [2023-10-07 21:16:30,397][67871] Updated weights for policy 1, policy_version 38360 (0.0008) [2023-10-07 21:16:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 78512128. Throughput: 0: 1666.4, 1: 1649.9. Samples: 19636850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:16:32,478][66916] Avg episode reward: [(0, '43.370'), (1, '46.160')] [2023-10-07 21:16:32,708][67838] Updated weights for policy 0, policy_version 38312 (0.0007) [2023-10-07 21:16:33,087][67838] Updated weights for policy 0, policy_version 38322 (0.0007) [2023-10-07 21:16:33,461][67838] Updated weights for policy 0, policy_version 38332 (0.0009) [2023-10-07 21:16:34,524][67871] Updated weights for policy 1, policy_version 38370 (0.0008) [2023-10-07 21:16:34,891][67871] Updated weights for policy 1, policy_version 38380 (0.0007) [2023-10-07 21:16:35,255][67871] Updated weights for policy 1, policy_version 38390 (0.0009) [2023-10-07 21:16:35,618][67871] Updated weights for policy 1, policy_version 38400 (0.0010) [2023-10-07 21:16:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 78577664. Throughput: 0: 1663.0, 1: 1671.5. Samples: 19657258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:16:37,478][66916] Avg episode reward: [(0, '41.180'), (1, '49.260')] [2023-10-07 21:16:37,645][67838] Updated weights for policy 0, policy_version 38342 (0.0008) [2023-10-07 21:16:38,007][67838] Updated weights for policy 0, policy_version 38352 (0.0007) [2023-10-07 21:16:38,378][67838] Updated weights for policy 0, policy_version 38362 (0.0009) [2023-10-07 21:16:39,666][67871] Updated weights for policy 1, policy_version 38410 (0.0007) [2023-10-07 21:16:40,032][67871] Updated weights for policy 1, policy_version 38420 (0.0008) [2023-10-07 21:16:40,402][67871] Updated weights for policy 1, policy_version 38430 (0.0009) [2023-10-07 21:16:42,363][67838] Updated weights for policy 0, policy_version 38372 (0.0010) [2023-10-07 21:16:42,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78643200. Throughput: 0: 1663.2, 1: 1661.0. Samples: 19666928. Policy #0 lag: (min: 16.0, avg: 40.0, max: 48.0) [2023-10-07 21:16:42,477][66916] Avg episode reward: [(0, '40.550'), (1, '48.500')] [2023-10-07 21:16:42,735][67838] Updated weights for policy 0, policy_version 38382 (0.0010) [2023-10-07 21:16:43,109][67838] Updated weights for policy 0, policy_version 38392 (0.0009) [2023-10-07 21:16:44,336][67871] Updated weights for policy 1, policy_version 38440 (0.0008) [2023-10-07 21:16:44,705][67871] Updated weights for policy 1, policy_version 38450 (0.0008) [2023-10-07 21:16:45,070][67871] Updated weights for policy 1, policy_version 38460 (0.0007) [2023-10-07 21:16:47,337][67838] Updated weights for policy 0, policy_version 38402 (0.0011) [2023-10-07 21:16:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78708736. Throughput: 0: 1664.8, 1: 1665.7. Samples: 19686866. Policy #0 lag: (min: 16.0, avg: 40.0, max: 48.0) [2023-10-07 21:16:47,477][66916] Avg episode reward: [(0, '39.760'), (1, '50.770')] [2023-10-07 21:16:47,695][67838] Updated weights for policy 0, policy_version 38412 (0.0009) [2023-10-07 21:16:48,065][67838] Updated weights for policy 0, policy_version 38422 (0.0010) [2023-10-07 21:16:48,442][67838] Updated weights for policy 0, policy_version 38432 (0.0007) [2023-10-07 21:16:49,206][67871] Updated weights for policy 1, policy_version 38470 (0.0009) [2023-10-07 21:16:49,574][67871] Updated weights for policy 1, policy_version 38480 (0.0009) [2023-10-07 21:16:49,931][67871] Updated weights for policy 1, policy_version 38490 (0.0008) [2023-10-07 21:16:52,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78774272. Throughput: 0: 1663.8, 1: 1675.5. Samples: 19707376. Policy #0 lag: (min: 16.0, avg: 40.0, max: 48.0) [2023-10-07 21:16:52,478][66916] Avg episode reward: [(0, '40.640'), (1, '50.010')] [2023-10-07 21:16:52,586][67838] Updated weights for policy 0, policy_version 38442 (0.0010) [2023-10-07 21:16:52,957][67838] Updated weights for policy 0, policy_version 38452 (0.0008) [2023-10-07 21:16:53,328][67838] Updated weights for policy 0, policy_version 38462 (0.0009) [2023-10-07 21:16:53,963][67871] Updated weights for policy 1, policy_version 38500 (0.0007) [2023-10-07 21:16:54,331][67871] Updated weights for policy 1, policy_version 38510 (0.0008) [2023-10-07 21:16:54,711][67871] Updated weights for policy 1, policy_version 38520 (0.0009) [2023-10-07 21:16:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78839808. Throughput: 0: 1661.3, 1: 1653.2. Samples: 19716670. Policy #0 lag: (min: 16.0, avg: 40.0, max: 48.0) [2023-10-07 21:16:57,477][66916] Avg episode reward: [(0, '42.640'), (1, '51.150')] [2023-10-07 21:16:57,537][67838] Updated weights for policy 0, policy_version 38472 (0.0008) [2023-10-07 21:16:57,912][67838] Updated weights for policy 0, policy_version 38482 (0.0008) [2023-10-07 21:16:58,281][67838] Updated weights for policy 0, policy_version 38492 (0.0009) [2023-10-07 21:16:58,864][67871] Updated weights for policy 1, policy_version 38530 (0.0009) [2023-10-07 21:16:59,223][67871] Updated weights for policy 1, policy_version 38540 (0.0011) [2023-10-07 21:16:59,585][67871] Updated weights for policy 1, policy_version 38550 (0.0009) [2023-10-07 21:16:59,960][67871] Updated weights for policy 1, policy_version 38560 (0.0008) [2023-10-07 21:17:02,398][67838] Updated weights for policy 0, policy_version 38502 (0.0010) [2023-10-07 21:17:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78905344. Throughput: 0: 1660.0, 1: 1674.3. Samples: 19736856. Policy #0 lag: (min: 16.0, avg: 40.0, max: 48.0) [2023-10-07 21:17:02,478][66916] Avg episode reward: [(0, '41.930'), (1, '49.040')] [2023-10-07 21:17:02,777][67838] Updated weights for policy 0, policy_version 38512 (0.0009) [2023-10-07 21:17:03,150][67838] Updated weights for policy 0, policy_version 38522 (0.0009) [2023-10-07 21:17:04,194][67871] Updated weights for policy 1, policy_version 38570 (0.0008) [2023-10-07 21:17:04,567][67871] Updated weights for policy 1, policy_version 38580 (0.0009) [2023-10-07 21:17:04,934][67871] Updated weights for policy 1, policy_version 38590 (0.0007) [2023-10-07 21:17:07,068][67838] Updated weights for policy 0, policy_version 38532 (0.0008) [2023-10-07 21:17:07,448][67838] Updated weights for policy 0, policy_version 38542 (0.0009) [2023-10-07 21:17:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78970880. Throughput: 0: 1658.9, 1: 1676.7. Samples: 19757132. Policy #0 lag: (min: 16.0, avg: 40.0, max: 48.0) [2023-10-07 21:17:07,477][66916] Avg episode reward: [(0, '39.170'), (1, '53.510')] [2023-10-07 21:17:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000038592_39518208.pth... [2023-10-07 21:17:07,522][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000037056_37945344.pth [2023-10-07 21:17:07,527][67676] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p1/milestones/checkpoint_000038592_39518208.pth [2023-10-07 21:17:07,814][67838] Updated weights for policy 0, policy_version 38552 (0.0010) [2023-10-07 21:17:08,115][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000038560_39485440.pth... [2023-10-07 21:17:08,153][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000036992_37879808.pth [2023-10-07 21:17:08,159][67511] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p0/milestones/checkpoint_000038560_39485440.pth [2023-10-07 21:17:09,124][67871] Updated weights for policy 1, policy_version 38600 (0.0010) [2023-10-07 21:17:09,494][67871] Updated weights for policy 1, policy_version 38610 (0.0010) [2023-10-07 21:17:09,863][67871] Updated weights for policy 1, policy_version 38620 (0.0010) [2023-10-07 21:17:11,827][67838] Updated weights for policy 0, policy_version 38562 (0.0009) [2023-10-07 21:17:12,210][67838] Updated weights for policy 0, policy_version 38572 (0.0009) [2023-10-07 21:17:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79036416. Throughput: 0: 1662.6, 1: 1654.8. Samples: 19766496. Policy #0 lag: (min: 3.0, avg: 3.3, max: 15.0) [2023-10-07 21:17:12,477][66916] Avg episode reward: [(0, '41.400'), (1, '53.220')] [2023-10-07 21:17:12,578][67838] Updated weights for policy 0, policy_version 38582 (0.0010) [2023-10-07 21:17:12,943][67838] Updated weights for policy 0, policy_version 38592 (0.0009) [2023-10-07 21:17:14,070][67871] Updated weights for policy 1, policy_version 38630 (0.0010) [2023-10-07 21:17:14,438][67871] Updated weights for policy 1, policy_version 38640 (0.0009) [2023-10-07 21:17:14,810][67871] Updated weights for policy 1, policy_version 38650 (0.0007) [2023-10-07 21:17:17,096][67838] Updated weights for policy 0, policy_version 38602 (0.0009) [2023-10-07 21:17:17,459][67838] Updated weights for policy 0, policy_version 38612 (0.0010) [2023-10-07 21:17:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79101952. Throughput: 0: 1662.4, 1: 1668.9. Samples: 19786756. Policy #0 lag: (min: 3.0, avg: 3.3, max: 15.0) [2023-10-07 21:17:17,477][66916] Avg episode reward: [(0, '40.240'), (1, '53.380')] [2023-10-07 21:17:17,833][67838] Updated weights for policy 0, policy_version 38622 (0.0008) [2023-10-07 21:17:18,717][67871] Updated weights for policy 1, policy_version 38660 (0.0008) [2023-10-07 21:17:19,079][67871] Updated weights for policy 1, policy_version 38670 (0.0008) [2023-10-07 21:17:19,435][67871] Updated weights for policy 1, policy_version 38680 (0.0007) [2023-10-07 21:17:21,993][67838] Updated weights for policy 0, policy_version 38632 (0.0008) [2023-10-07 21:17:22,369][67838] Updated weights for policy 0, policy_version 38642 (0.0009) [2023-10-07 21:17:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 79167488. Throughput: 0: 1658.2, 1: 1672.1. Samples: 19807120. Policy #0 lag: (min: 3.0, avg: 3.3, max: 15.0) [2023-10-07 21:17:22,477][66916] Avg episode reward: [(0, '40.630'), (1, '52.310')] [2023-10-07 21:17:22,742][67838] Updated weights for policy 0, policy_version 38652 (0.0008) [2023-10-07 21:17:23,499][67871] Updated weights for policy 1, policy_version 38690 (0.0009) [2023-10-07 21:17:23,876][67871] Updated weights for policy 1, policy_version 38700 (0.0010) [2023-10-07 21:17:24,237][67871] Updated weights for policy 1, policy_version 38710 (0.0009) [2023-10-07 21:17:24,605][67871] Updated weights for policy 1, policy_version 38720 (0.0010) [2023-10-07 21:17:26,908][67838] Updated weights for policy 0, policy_version 38662 (0.0008) [2023-10-07 21:17:27,287][67838] Updated weights for policy 0, policy_version 38672 (0.0007) [2023-10-07 21:17:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 79233024. Throughput: 0: 1668.0, 1: 1658.0. Samples: 19816596. Policy #0 lag: (min: 3.0, avg: 3.3, max: 15.0) [2023-10-07 21:17:27,477][66916] Avg episode reward: [(0, '40.710'), (1, '46.290')] [2023-10-07 21:17:27,660][67838] Updated weights for policy 0, policy_version 38682 (0.0008) [2023-10-07 21:17:28,686][67871] Updated weights for policy 1, policy_version 38730 (0.0007) [2023-10-07 21:17:29,058][67871] Updated weights for policy 1, policy_version 38740 (0.0007) [2023-10-07 21:17:29,434][67871] Updated weights for policy 1, policy_version 38750 (0.0010) [2023-10-07 21:17:31,741][67838] Updated weights for policy 0, policy_version 38692 (0.0011) [2023-10-07 21:17:32,124][67838] Updated weights for policy 0, policy_version 38702 (0.0011) [2023-10-07 21:17:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 79298560. Throughput: 0: 1667.0, 1: 1669.6. Samples: 19837014. Policy #0 lag: (min: 3.0, avg: 3.3, max: 15.0) [2023-10-07 21:17:32,478][66916] Avg episode reward: [(0, '38.800'), (1, '44.520')] [2023-10-07 21:17:32,488][67838] Updated weights for policy 0, policy_version 38712 (0.0011) [2023-10-07 21:17:33,746][67871] Updated weights for policy 1, policy_version 38760 (0.0010) [2023-10-07 21:17:34,119][67871] Updated weights for policy 1, policy_version 38770 (0.0009) [2023-10-07 21:17:34,484][67871] Updated weights for policy 1, policy_version 38780 (0.0009) [2023-10-07 21:17:36,535][67838] Updated weights for policy 0, policy_version 38722 (0.0008) [2023-10-07 21:17:36,917][67838] Updated weights for policy 0, policy_version 38732 (0.0009) [2023-10-07 21:17:37,286][67838] Updated weights for policy 0, policy_version 38742 (0.0007) [2023-10-07 21:17:37,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 79364096. Throughput: 0: 1657.1, 1: 1666.2. Samples: 19856926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:17:37,478][66916] Avg episode reward: [(0, '41.700'), (1, '46.430')] [2023-10-07 21:17:37,657][67838] Updated weights for policy 0, policy_version 38752 (0.0007) [2023-10-07 21:17:38,532][67871] Updated weights for policy 1, policy_version 38790 (0.0009) [2023-10-07 21:17:38,900][67871] Updated weights for policy 1, policy_version 38800 (0.0007) [2023-10-07 21:17:39,265][67871] Updated weights for policy 1, policy_version 38810 (0.0009) [2023-10-07 21:17:41,845][67838] Updated weights for policy 0, policy_version 38762 (0.0010) [2023-10-07 21:17:42,232][67838] Updated weights for policy 0, policy_version 38772 (0.0011) [2023-10-07 21:17:42,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79429632. Throughput: 0: 1671.3, 1: 1661.2. Samples: 19866636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:17:42,477][66916] Avg episode reward: [(0, '41.180'), (1, '45.740')] [2023-10-07 21:17:42,595][67838] Updated weights for policy 0, policy_version 38782 (0.0007) [2023-10-07 21:17:43,402][67871] Updated weights for policy 1, policy_version 38820 (0.0009) [2023-10-07 21:17:43,774][67871] Updated weights for policy 1, policy_version 38830 (0.0007) [2023-10-07 21:17:44,144][67871] Updated weights for policy 1, policy_version 38840 (0.0007) [2023-10-07 21:17:46,852][67838] Updated weights for policy 0, policy_version 38792 (0.0007) [2023-10-07 21:17:47,223][67838] Updated weights for policy 0, policy_version 38802 (0.0009) [2023-10-07 21:17:47,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 79495168. Throughput: 0: 1666.3, 1: 1670.2. Samples: 19886998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:17:47,478][66916] Avg episode reward: [(0, '44.260'), (1, '47.220')] [2023-10-07 21:17:47,595][67838] Updated weights for policy 0, policy_version 38812 (0.0009) [2023-10-07 21:17:48,281][67871] Updated weights for policy 1, policy_version 38850 (0.0007) [2023-10-07 21:17:48,643][67871] Updated weights for policy 1, policy_version 38860 (0.0007) [2023-10-07 21:17:49,011][67871] Updated weights for policy 1, policy_version 38870 (0.0008) [2023-10-07 21:17:49,379][67871] Updated weights for policy 1, policy_version 38880 (0.0007) [2023-10-07 21:17:51,782][67838] Updated weights for policy 0, policy_version 38822 (0.0009) [2023-10-07 21:17:52,157][67838] Updated weights for policy 0, policy_version 38832 (0.0008) [2023-10-07 21:17:52,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 79560704. Throughput: 0: 1651.3, 1: 1671.8. Samples: 19906674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:17:52,478][66916] Avg episode reward: [(0, '43.400'), (1, '46.960')] [2023-10-07 21:17:52,521][67838] Updated weights for policy 0, policy_version 38842 (0.0008) [2023-10-07 21:17:53,505][67871] Updated weights for policy 1, policy_version 38890 (0.0007) [2023-10-07 21:17:53,880][67871] Updated weights for policy 1, policy_version 38900 (0.0008) [2023-10-07 21:17:54,242][67871] Updated weights for policy 1, policy_version 38910 (0.0010) [2023-10-07 21:17:56,590][67838] Updated weights for policy 0, policy_version 38852 (0.0007) [2023-10-07 21:17:56,977][67838] Updated weights for policy 0, policy_version 38862 (0.0010) [2023-10-07 21:17:57,362][67838] Updated weights for policy 0, policy_version 38872 (0.0008) [2023-10-07 21:17:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 79626240. Throughput: 0: 1661.4, 1: 1668.9. Samples: 19916358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:17:57,477][66916] Avg episode reward: [(0, '43.390'), (1, '50.620')] [2023-10-07 21:17:58,295][67871] Updated weights for policy 1, policy_version 38920 (0.0009) [2023-10-07 21:17:58,663][67871] Updated weights for policy 1, policy_version 38930 (0.0008) [2023-10-07 21:17:59,023][67871] Updated weights for policy 1, policy_version 38940 (0.0007) [2023-10-07 21:18:01,232][67838] Updated weights for policy 0, policy_version 38882 (0.0010) [2023-10-07 21:18:01,602][67838] Updated weights for policy 0, policy_version 38892 (0.0008) [2023-10-07 21:18:01,991][67838] Updated weights for policy 0, policy_version 38902 (0.0008) [2023-10-07 21:18:02,363][67838] Updated weights for policy 0, policy_version 38912 (0.0009) [2023-10-07 21:18:02,476][66916] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 79724544. Throughput: 0: 1668.1, 1: 1677.4. Samples: 19937302. Policy #0 lag: (min: 31.0, avg: 45.7, max: 63.0) [2023-10-07 21:18:02,477][66916] Avg episode reward: [(0, '38.890'), (1, '50.410')] [2023-10-07 21:18:03,181][67871] Updated weights for policy 1, policy_version 38950 (0.0008) [2023-10-07 21:18:03,542][67871] Updated weights for policy 1, policy_version 38960 (0.0009) [2023-10-07 21:18:03,913][67871] Updated weights for policy 1, policy_version 38970 (0.0008) [2023-10-07 21:18:06,509][67838] Updated weights for policy 0, policy_version 38922 (0.0008) [2023-10-07 21:18:06,889][67838] Updated weights for policy 0, policy_version 38932 (0.0010) [2023-10-07 21:18:07,251][67838] Updated weights for policy 0, policy_version 38942 (0.0009) [2023-10-07 21:18:07,477][66916] Fps is (10 sec: 16383.2, 60 sec: 13653.2, 300 sec: 13329.3). Total num frames: 79790080. Throughput: 0: 1656.2, 1: 1674.8. Samples: 19957016. Policy #0 lag: (min: 31.0, avg: 45.7, max: 63.0) [2023-10-07 21:18:07,478][66916] Avg episode reward: [(0, '39.630'), (1, '50.690')] [2023-10-07 21:18:07,958][67871] Updated weights for policy 1, policy_version 38980 (0.0007) [2023-10-07 21:18:08,329][67871] Updated weights for policy 1, policy_version 38990 (0.0007) [2023-10-07 21:18:08,701][67871] Updated weights for policy 1, policy_version 39000 (0.0008) [2023-10-07 21:18:11,364][67838] Updated weights for policy 0, policy_version 38952 (0.0009) [2023-10-07 21:18:11,734][67838] Updated weights for policy 0, policy_version 38962 (0.0009) [2023-10-07 21:18:12,106][67838] Updated weights for policy 0, policy_version 38972 (0.0007) [2023-10-07 21:18:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 79855616. Throughput: 0: 1665.1, 1: 1673.1. Samples: 19966816. Policy #0 lag: (min: 31.0, avg: 45.7, max: 63.0) [2023-10-07 21:18:12,477][66916] Avg episode reward: [(0, '38.420'), (1, '46.120')] [2023-10-07 21:18:12,774][67871] Updated weights for policy 1, policy_version 39010 (0.0008) [2023-10-07 21:18:13,130][67871] Updated weights for policy 1, policy_version 39020 (0.0009) [2023-10-07 21:18:13,499][67871] Updated weights for policy 1, policy_version 39030 (0.0010) [2023-10-07 21:18:13,865][67871] Updated weights for policy 1, policy_version 39040 (0.0009) [2023-10-07 21:18:16,240][67838] Updated weights for policy 0, policy_version 38982 (0.0008) [2023-10-07 21:18:16,614][67838] Updated weights for policy 0, policy_version 38992 (0.0010) [2023-10-07 21:18:16,999][67838] Updated weights for policy 0, policy_version 39002 (0.0007) [2023-10-07 21:18:17,477][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 79921152. Throughput: 0: 1661.7, 1: 1672.4. Samples: 19987050. Policy #0 lag: (min: 31.0, avg: 45.7, max: 63.0) [2023-10-07 21:18:17,478][66916] Avg episode reward: [(0, '38.650'), (1, '44.690')] [2023-10-07 21:18:17,848][67871] Updated weights for policy 1, policy_version 39050 (0.0009) [2023-10-07 21:18:18,211][67871] Updated weights for policy 1, policy_version 39060 (0.0010) [2023-10-07 21:18:18,575][67871] Updated weights for policy 1, policy_version 39070 (0.0011) [2023-10-07 21:18:21,030][67838] Updated weights for policy 0, policy_version 39012 (0.0010) [2023-10-07 21:18:21,412][67838] Updated weights for policy 0, policy_version 39022 (0.0010) [2023-10-07 21:18:21,781][67838] Updated weights for policy 0, policy_version 39032 (0.0009) [2023-10-07 21:18:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 79986688. Throughput: 0: 1647.7, 1: 1674.9. Samples: 20006440. Policy #0 lag: (min: 31.0, avg: 45.7, max: 63.0) [2023-10-07 21:18:22,477][66916] Avg episode reward: [(0, '39.060'), (1, '41.520')] [2023-10-07 21:18:22,846][67871] Updated weights for policy 1, policy_version 39080 (0.0009) [2023-10-07 21:18:23,210][67871] Updated weights for policy 1, policy_version 39090 (0.0007) [2023-10-07 21:18:23,581][67871] Updated weights for policy 1, policy_version 39100 (0.0008) [2023-10-07 21:18:25,812][67838] Updated weights for policy 0, policy_version 39042 (0.0007) [2023-10-07 21:18:26,185][67838] Updated weights for policy 0, policy_version 39052 (0.0009) [2023-10-07 21:18:26,552][67838] Updated weights for policy 0, policy_version 39062 (0.0009) [2023-10-07 21:18:26,916][67838] Updated weights for policy 0, policy_version 39072 (0.0009) [2023-10-07 21:18:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 80052224. Throughput: 0: 1665.1, 1: 1671.3. Samples: 20016776. Policy #0 lag: (min: 31.0, avg: 45.7, max: 63.0) [2023-10-07 21:18:27,477][66916] Avg episode reward: [(0, '40.210'), (1, '44.310')] [2023-10-07 21:18:27,625][67871] Updated weights for policy 1, policy_version 39110 (0.0010) [2023-10-07 21:18:27,983][67871] Updated weights for policy 1, policy_version 39120 (0.0010) [2023-10-07 21:18:28,359][67871] Updated weights for policy 1, policy_version 39130 (0.0008) [2023-10-07 21:18:31,005][67838] Updated weights for policy 0, policy_version 39082 (0.0011) [2023-10-07 21:18:31,390][67838] Updated weights for policy 0, policy_version 39092 (0.0009) [2023-10-07 21:18:31,766][67838] Updated weights for policy 0, policy_version 39102 (0.0009) [2023-10-07 21:18:32,425][67871] Updated weights for policy 1, policy_version 39140 (0.0008) [2023-10-07 21:18:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 80117760. Throughput: 0: 1660.8, 1: 1666.8. Samples: 20036738. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-07 21:18:32,478][66916] Avg episode reward: [(0, '43.820'), (1, '46.200')] [2023-10-07 21:18:32,802][67871] Updated weights for policy 1, policy_version 39150 (0.0008) [2023-10-07 21:18:33,165][67871] Updated weights for policy 1, policy_version 39160 (0.0008) [2023-10-07 21:18:36,082][67838] Updated weights for policy 0, policy_version 39112 (0.0009) [2023-10-07 21:18:36,456][67838] Updated weights for policy 0, policy_version 39122 (0.0007) [2023-10-07 21:18:36,825][67838] Updated weights for policy 0, policy_version 39132 (0.0010) [2023-10-07 21:18:37,418][67871] Updated weights for policy 1, policy_version 39170 (0.0007) [2023-10-07 21:18:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 80183296. Throughput: 0: 1656.3, 1: 1666.2. Samples: 20056186. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-07 21:18:37,478][66916] Avg episode reward: [(0, '41.690'), (1, '49.420')] [2023-10-07 21:18:37,786][67871] Updated weights for policy 1, policy_version 39180 (0.0008) [2023-10-07 21:18:38,150][67871] Updated weights for policy 1, policy_version 39190 (0.0007) [2023-10-07 21:18:38,513][67871] Updated weights for policy 1, policy_version 39200 (0.0010) [2023-10-07 21:18:41,113][67838] Updated weights for policy 0, policy_version 39142 (0.0009) [2023-10-07 21:18:41,489][67838] Updated weights for policy 0, policy_version 39152 (0.0007) [2023-10-07 21:18:41,867][67838] Updated weights for policy 0, policy_version 39162 (0.0009) [2023-10-07 21:18:42,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 80248832. Throughput: 0: 1670.4, 1: 1664.9. Samples: 20066448. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-07 21:18:42,478][66916] Avg episode reward: [(0, '43.510'), (1, '50.060')] [2023-10-07 21:18:42,656][67871] Updated weights for policy 1, policy_version 39210 (0.0010) [2023-10-07 21:18:43,019][67871] Updated weights for policy 1, policy_version 39220 (0.0008) [2023-10-07 21:18:43,384][67871] Updated weights for policy 1, policy_version 39230 (0.0007) [2023-10-07 21:18:45,965][67838] Updated weights for policy 0, policy_version 39172 (0.0008) [2023-10-07 21:18:46,333][67838] Updated weights for policy 0, policy_version 39182 (0.0007) [2023-10-07 21:18:46,701][67838] Updated weights for policy 0, policy_version 39192 (0.0010) [2023-10-07 21:18:47,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 80314368. Throughput: 0: 1658.8, 1: 1661.8. Samples: 20086730. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-07 21:18:47,478][66916] Avg episode reward: [(0, '42.450'), (1, '49.260')] [2023-10-07 21:18:47,514][67871] Updated weights for policy 1, policy_version 39240 (0.0007) [2023-10-07 21:18:47,885][67871] Updated weights for policy 1, policy_version 39250 (0.0007) [2023-10-07 21:18:48,246][67871] Updated weights for policy 1, policy_version 39260 (0.0010) [2023-10-07 21:18:50,698][67838] Updated weights for policy 0, policy_version 39202 (0.0007) [2023-10-07 21:18:51,072][67838] Updated weights for policy 0, policy_version 39212 (0.0007) [2023-10-07 21:18:51,451][67838] Updated weights for policy 0, policy_version 39222 (0.0009) [2023-10-07 21:18:51,821][67838] Updated weights for policy 0, policy_version 39232 (0.0011) [2023-10-07 21:18:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 80379904. Throughput: 0: 1655.3, 1: 1659.3. Samples: 20106170. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-07 21:18:52,477][66916] Avg episode reward: [(0, '40.960'), (1, '49.820')] [2023-10-07 21:18:52,479][67871] Updated weights for policy 1, policy_version 39270 (0.0009) [2023-10-07 21:18:52,853][67871] Updated weights for policy 1, policy_version 39280 (0.0008) [2023-10-07 21:18:53,216][67871] Updated weights for policy 1, policy_version 39290 (0.0008) [2023-10-07 21:18:56,051][67838] Updated weights for policy 0, policy_version 39242 (0.0007) [2023-10-07 21:18:56,416][67838] Updated weights for policy 0, policy_version 39252 (0.0008) [2023-10-07 21:18:56,792][67838] Updated weights for policy 0, policy_version 39262 (0.0009) [2023-10-07 21:18:57,242][67871] Updated weights for policy 1, policy_version 39300 (0.0009) [2023-10-07 21:18:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 80445440. Throughput: 0: 1663.7, 1: 1660.0. Samples: 20116380. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-07 21:18:57,477][66916] Avg episode reward: [(0, '41.700'), (1, '45.140')] [2023-10-07 21:18:57,605][67871] Updated weights for policy 1, policy_version 39310 (0.0011) [2023-10-07 21:18:57,976][67871] Updated weights for policy 1, policy_version 39320 (0.0009) [2023-10-07 21:19:00,748][67838] Updated weights for policy 0, policy_version 39272 (0.0008) [2023-10-07 21:19:01,122][67838] Updated weights for policy 0, policy_version 39282 (0.0009) [2023-10-07 21:19:01,495][67838] Updated weights for policy 0, policy_version 39292 (0.0008) [2023-10-07 21:19:02,049][67871] Updated weights for policy 1, policy_version 39330 (0.0008) [2023-10-07 21:19:02,419][67871] Updated weights for policy 1, policy_version 39340 (0.0007) [2023-10-07 21:19:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 80510976. Throughput: 0: 1653.6, 1: 1664.3. Samples: 20136356. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-07 21:19:02,478][66916] Avg episode reward: [(0, '39.490'), (1, '43.830')] [2023-10-07 21:19:02,780][67871] Updated weights for policy 1, policy_version 39350 (0.0009) [2023-10-07 21:19:03,152][67871] Updated weights for policy 1, policy_version 39360 (0.0008) [2023-10-07 21:19:05,289][67838] Updated weights for policy 0, policy_version 39302 (0.0009) [2023-10-07 21:19:05,654][67838] Updated weights for policy 0, policy_version 39312 (0.0010) [2023-10-07 21:19:06,034][67838] Updated weights for policy 0, policy_version 39322 (0.0010) [2023-10-07 21:19:07,196][67871] Updated weights for policy 1, policy_version 39370 (0.0009) [2023-10-07 21:19:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 80576512. Throughput: 0: 1669.3, 1: 1664.7. Samples: 20156470. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-07 21:19:07,478][66916] Avg episode reward: [(0, '38.840'), (1, '44.290')] [2023-10-07 21:19:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000039328_40271872.pth... [2023-10-07 21:19:07,522][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000037760_38666240.pth [2023-10-07 21:19:07,560][67871] Updated weights for policy 1, policy_version 39380 (0.0008) [2023-10-07 21:19:07,924][67871] Updated weights for policy 1, policy_version 39390 (0.0008) [2023-10-07 21:19:07,996][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000039392_40337408.pth... [2023-10-07 21:19:08,025][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000037824_38731776.pth [2023-10-07 21:19:10,254][67838] Updated weights for policy 0, policy_version 39332 (0.0008) [2023-10-07 21:19:10,617][67838] Updated weights for policy 0, policy_version 39342 (0.0008) [2023-10-07 21:19:10,992][67838] Updated weights for policy 0, policy_version 39352 (0.0008) [2023-10-07 21:19:12,194][67871] Updated weights for policy 1, policy_version 39400 (0.0007) [2023-10-07 21:19:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 80642048. Throughput: 0: 1670.3, 1: 1665.0. Samples: 20166864. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-07 21:19:12,477][66916] Avg episode reward: [(0, '41.240'), (1, '43.280')] [2023-10-07 21:19:12,565][67871] Updated weights for policy 1, policy_version 39410 (0.0007) [2023-10-07 21:19:12,936][67871] Updated weights for policy 1, policy_version 39420 (0.0008) [2023-10-07 21:19:15,075][67838] Updated weights for policy 0, policy_version 39362 (0.0007) [2023-10-07 21:19:15,451][67838] Updated weights for policy 0, policy_version 39372 (0.0008) [2023-10-07 21:19:15,817][67838] Updated weights for policy 0, policy_version 39382 (0.0007) [2023-10-07 21:19:16,197][67838] Updated weights for policy 0, policy_version 39392 (0.0007) [2023-10-07 21:19:17,192][67871] Updated weights for policy 1, policy_version 39430 (0.0007) [2023-10-07 21:19:17,477][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 80707584. Throughput: 0: 1658.0, 1: 1659.9. Samples: 20186044. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-07 21:19:17,478][66916] Avg episode reward: [(0, '40.490'), (1, '42.580')] [2023-10-07 21:19:17,562][67871] Updated weights for policy 1, policy_version 39440 (0.0007) [2023-10-07 21:19:17,929][67871] Updated weights for policy 1, policy_version 39450 (0.0007) [2023-10-07 21:19:20,186][67838] Updated weights for policy 0, policy_version 39402 (0.0009) [2023-10-07 21:19:20,571][67838] Updated weights for policy 0, policy_version 39412 (0.0008) [2023-10-07 21:19:20,941][67838] Updated weights for policy 0, policy_version 39422 (0.0008) [2023-10-07 21:19:21,936][67871] Updated weights for policy 1, policy_version 39460 (0.0009) [2023-10-07 21:19:22,310][67871] Updated weights for policy 1, policy_version 39470 (0.0007) [2023-10-07 21:19:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 80773120. Throughput: 0: 1673.8, 1: 1663.5. Samples: 20206364. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-07 21:19:22,478][66916] Avg episode reward: [(0, '38.920'), (1, '44.270')] [2023-10-07 21:19:22,673][67871] Updated weights for policy 1, policy_version 39480 (0.0009) [2023-10-07 21:19:25,167][67838] Updated weights for policy 0, policy_version 39432 (0.0007) [2023-10-07 21:19:25,554][67838] Updated weights for policy 0, policy_version 39442 (0.0009) [2023-10-07 21:19:25,924][67838] Updated weights for policy 0, policy_version 39452 (0.0008) [2023-10-07 21:19:26,863][67871] Updated weights for policy 1, policy_version 39490 (0.0008) [2023-10-07 21:19:27,248][67871] Updated weights for policy 1, policy_version 39500 (0.0009) [2023-10-07 21:19:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 80838656. Throughput: 0: 1672.0, 1: 1665.9. Samples: 20216654. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:19:27,477][66916] Avg episode reward: [(0, '40.280'), (1, '42.870')] [2023-10-07 21:19:27,615][67871] Updated weights for policy 1, policy_version 39510 (0.0008) [2023-10-07 21:19:27,976][67871] Updated weights for policy 1, policy_version 39520 (0.0010) [2023-10-07 21:19:29,900][67838] Updated weights for policy 0, policy_version 39462 (0.0009) [2023-10-07 21:19:30,270][67838] Updated weights for policy 0, policy_version 39472 (0.0009) [2023-10-07 21:19:30,643][67838] Updated weights for policy 0, policy_version 39482 (0.0009) [2023-10-07 21:19:32,037][67871] Updated weights for policy 1, policy_version 39530 (0.0011) [2023-10-07 21:19:32,406][67871] Updated weights for policy 1, policy_version 39540 (0.0007) [2023-10-07 21:19:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 80904192. Throughput: 0: 1655.7, 1: 1664.0. Samples: 20236118. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:19:32,478][66916] Avg episode reward: [(0, '38.230'), (1, '41.510')] [2023-10-07 21:19:32,783][67871] Updated weights for policy 1, policy_version 39550 (0.0008) [2023-10-07 21:19:34,771][67838] Updated weights for policy 0, policy_version 39492 (0.0008) [2023-10-07 21:19:35,140][67838] Updated weights for policy 0, policy_version 39502 (0.0008) [2023-10-07 21:19:35,512][67838] Updated weights for policy 0, policy_version 39512 (0.0008) [2023-10-07 21:19:36,683][67871] Updated weights for policy 1, policy_version 39560 (0.0009) [2023-10-07 21:19:37,048][67871] Updated weights for policy 1, policy_version 39570 (0.0007) [2023-10-07 21:19:37,414][67871] Updated weights for policy 1, policy_version 39580 (0.0007) [2023-10-07 21:19:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 80969728. Throughput: 0: 1679.6, 1: 1655.9. Samples: 20256270. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:19:37,478][66916] Avg episode reward: [(0, '40.470'), (1, '42.590')] [2023-10-07 21:19:39,555][67838] Updated weights for policy 0, policy_version 39522 (0.0008) [2023-10-07 21:19:39,930][67838] Updated weights for policy 0, policy_version 39532 (0.0008) [2023-10-07 21:19:40,298][67838] Updated weights for policy 0, policy_version 39542 (0.0007) [2023-10-07 21:19:40,669][67838] Updated weights for policy 0, policy_version 39552 (0.0007) [2023-10-07 21:19:41,733][67871] Updated weights for policy 1, policy_version 39590 (0.0008) [2023-10-07 21:19:42,096][67871] Updated weights for policy 1, policy_version 39600 (0.0010) [2023-10-07 21:19:42,466][67871] Updated weights for policy 1, policy_version 39610 (0.0010) [2023-10-07 21:19:42,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 81035264. Throughput: 0: 1665.7, 1: 1663.4. Samples: 20266190. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:19:42,477][66916] Avg episode reward: [(0, '38.130'), (1, '43.450')] [2023-10-07 21:19:44,687][67838] Updated weights for policy 0, policy_version 39562 (0.0007) [2023-10-07 21:19:45,053][67838] Updated weights for policy 0, policy_version 39572 (0.0007) [2023-10-07 21:19:45,423][67838] Updated weights for policy 0, policy_version 39582 (0.0007) [2023-10-07 21:19:46,554][67871] Updated weights for policy 1, policy_version 39620 (0.0010) [2023-10-07 21:19:46,921][67871] Updated weights for policy 1, policy_version 39630 (0.0011) [2023-10-07 21:19:47,288][67871] Updated weights for policy 1, policy_version 39640 (0.0010) [2023-10-07 21:19:47,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 81100800. Throughput: 0: 1668.9, 1: 1658.0. Samples: 20286068. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:19:47,477][66916] Avg episode reward: [(0, '36.920'), (1, '42.750')] [2023-10-07 21:19:49,578][67838] Updated weights for policy 0, policy_version 39592 (0.0008) [2023-10-07 21:19:49,949][67838] Updated weights for policy 0, policy_version 39602 (0.0009) [2023-10-07 21:19:50,333][67838] Updated weights for policy 0, policy_version 39612 (0.0007) [2023-10-07 21:19:51,537][67871] Updated weights for policy 1, policy_version 39650 (0.0008) [2023-10-07 21:19:51,903][67871] Updated weights for policy 1, policy_version 39660 (0.0011) [2023-10-07 21:19:52,278][67871] Updated weights for policy 1, policy_version 39670 (0.0011) [2023-10-07 21:19:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 81166336. Throughput: 0: 1680.5, 1: 1647.7. Samples: 20306236. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:19:52,477][66916] Avg episode reward: [(0, '36.900'), (1, '47.610')] [2023-10-07 21:19:52,641][67871] Updated weights for policy 1, policy_version 39680 (0.0009) [2023-10-07 21:19:54,426][67838] Updated weights for policy 0, policy_version 39622 (0.0010) [2023-10-07 21:19:54,799][67838] Updated weights for policy 0, policy_version 39632 (0.0011) [2023-10-07 21:19:55,172][67838] Updated weights for policy 0, policy_version 39642 (0.0011) [2023-10-07 21:19:56,586][67871] Updated weights for policy 1, policy_version 39690 (0.0011) [2023-10-07 21:19:56,958][67871] Updated weights for policy 1, policy_version 39700 (0.0009) [2023-10-07 21:19:57,319][67871] Updated weights for policy 1, policy_version 39710 (0.0011) [2023-10-07 21:19:57,477][66916] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 81264640. Throughput: 0: 1656.6, 1: 1662.9. Samples: 20316240. Policy #0 lag: (min: 13.0, avg: 15.8, max: 45.0) [2023-10-07 21:19:57,477][66916] Avg episode reward: [(0, '36.260'), (1, '47.710')] [2023-10-07 21:19:59,354][67838] Updated weights for policy 0, policy_version 39652 (0.0009) [2023-10-07 21:19:59,731][67838] Updated weights for policy 0, policy_version 39662 (0.0007) [2023-10-07 21:20:00,102][67838] Updated weights for policy 0, policy_version 39672 (0.0009) [2023-10-07 21:20:01,403][67871] Updated weights for policy 1, policy_version 39720 (0.0010) [2023-10-07 21:20:01,761][67871] Updated weights for policy 1, policy_version 39730 (0.0009) [2023-10-07 21:20:02,135][67871] Updated weights for policy 1, policy_version 39740 (0.0007) [2023-10-07 21:20:02,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 81330176. Throughput: 0: 1669.0, 1: 1670.0. Samples: 20336300. Policy #0 lag: (min: 13.0, avg: 15.8, max: 45.0) [2023-10-07 21:20:02,478][66916] Avg episode reward: [(0, '38.720'), (1, '49.830')] [2023-10-07 21:20:04,231][67838] Updated weights for policy 0, policy_version 39682 (0.0008) [2023-10-07 21:20:04,603][67838] Updated weights for policy 0, policy_version 39692 (0.0008) [2023-10-07 21:20:04,989][67838] Updated weights for policy 0, policy_version 39702 (0.0008) [2023-10-07 21:20:05,356][67838] Updated weights for policy 0, policy_version 39712 (0.0010) [2023-10-07 21:20:06,249][67871] Updated weights for policy 1, policy_version 39750 (0.0009) [2023-10-07 21:20:06,622][67871] Updated weights for policy 1, policy_version 39760 (0.0011) [2023-10-07 21:20:07,004][67871] Updated weights for policy 1, policy_version 39770 (0.0009) [2023-10-07 21:20:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 81395712. Throughput: 0: 1671.9, 1: 1652.2. Samples: 20355948. Policy #0 lag: (min: 13.0, avg: 15.8, max: 45.0) [2023-10-07 21:20:07,478][66916] Avg episode reward: [(0, '36.870'), (1, '49.040')] [2023-10-07 21:20:09,587][67838] Updated weights for policy 0, policy_version 39722 (0.0009) [2023-10-07 21:20:09,963][67838] Updated weights for policy 0, policy_version 39732 (0.0010) [2023-10-07 21:20:10,344][67838] Updated weights for policy 0, policy_version 39742 (0.0010) [2023-10-07 21:20:11,062][67871] Updated weights for policy 1, policy_version 39780 (0.0008) [2023-10-07 21:20:11,426][67871] Updated weights for policy 1, policy_version 39790 (0.0008) [2023-10-07 21:20:11,795][67871] Updated weights for policy 1, policy_version 39800 (0.0010) [2023-10-07 21:20:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 81461248. Throughput: 0: 1652.0, 1: 1670.8. Samples: 20366182. Policy #0 lag: (min: 13.0, avg: 15.8, max: 45.0) [2023-10-07 21:20:12,477][66916] Avg episode reward: [(0, '37.030'), (1, '48.710')] [2023-10-07 21:20:14,372][67838] Updated weights for policy 0, policy_version 39752 (0.0008) [2023-10-07 21:20:14,738][67838] Updated weights for policy 0, policy_version 39762 (0.0007) [2023-10-07 21:20:15,113][67838] Updated weights for policy 0, policy_version 39772 (0.0008) [2023-10-07 21:20:16,149][67871] Updated weights for policy 1, policy_version 39810 (0.0010) [2023-10-07 21:20:16,573][67871] Updated weights for policy 1, policy_version 39820 (0.0010) [2023-10-07 21:20:16,951][67871] Updated weights for policy 1, policy_version 39830 (0.0008) [2023-10-07 21:20:17,328][67871] Updated weights for policy 1, policy_version 39840 (0.0007) [2023-10-07 21:20:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 81526784. Throughput: 0: 1662.4, 1: 1670.3. Samples: 20386090. Policy #0 lag: (min: 13.0, avg: 15.8, max: 45.0) [2023-10-07 21:20:17,477][66916] Avg episode reward: [(0, '41.560'), (1, '49.950')] [2023-10-07 21:20:19,142][67838] Updated weights for policy 0, policy_version 39782 (0.0010) [2023-10-07 21:20:19,524][67838] Updated weights for policy 0, policy_version 39792 (0.0009) [2023-10-07 21:20:19,894][67838] Updated weights for policy 0, policy_version 39802 (0.0007) [2023-10-07 21:20:21,480][67871] Updated weights for policy 1, policy_version 39850 (0.0007) [2023-10-07 21:20:21,842][67871] Updated weights for policy 1, policy_version 39860 (0.0008) [2023-10-07 21:20:22,211][67871] Updated weights for policy 1, policy_version 39870 (0.0009) [2023-10-07 21:20:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 81592320. Throughput: 0: 1663.0, 1: 1656.1. Samples: 20405626. Policy #0 lag: (min: 13.0, avg: 15.8, max: 45.0) [2023-10-07 21:20:22,477][66916] Avg episode reward: [(0, '36.840'), (1, '47.300')] [2023-10-07 21:20:24,079][67838] Updated weights for policy 0, policy_version 39812 (0.0008) [2023-10-07 21:20:24,443][67838] Updated weights for policy 0, policy_version 39822 (0.0009) [2023-10-07 21:20:24,817][67838] Updated weights for policy 0, policy_version 39832 (0.0009) [2023-10-07 21:20:26,158][67871] Updated weights for policy 1, policy_version 39880 (0.0010) [2023-10-07 21:20:26,521][67871] Updated weights for policy 1, policy_version 39890 (0.0009) [2023-10-07 21:20:26,890][67871] Updated weights for policy 1, policy_version 39900 (0.0008) [2023-10-07 21:20:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 81657856. Throughput: 0: 1649.7, 1: 1666.4. Samples: 20415414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:20:27,477][66916] Avg episode reward: [(0, '39.000'), (1, '46.690')] [2023-10-07 21:20:28,829][67838] Updated weights for policy 0, policy_version 39842 (0.0009) [2023-10-07 21:20:29,201][67838] Updated weights for policy 0, policy_version 39852 (0.0007) [2023-10-07 21:20:29,574][67838] Updated weights for policy 0, policy_version 39862 (0.0008) [2023-10-07 21:20:29,948][67838] Updated weights for policy 0, policy_version 39872 (0.0008) [2023-10-07 21:20:31,257][67871] Updated weights for policy 1, policy_version 39910 (0.0009) [2023-10-07 21:20:31,623][67871] Updated weights for policy 1, policy_version 39920 (0.0009) [2023-10-07 21:20:31,983][67871] Updated weights for policy 1, policy_version 39930 (0.0009) [2023-10-07 21:20:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 81723392. Throughput: 0: 1658.8, 1: 1669.2. Samples: 20435824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:20:32,477][66916] Avg episode reward: [(0, '38.070'), (1, '49.580')] [2023-10-07 21:20:33,878][67838] Updated weights for policy 0, policy_version 39882 (0.0010) [2023-10-07 21:20:34,246][67838] Updated weights for policy 0, policy_version 39892 (0.0009) [2023-10-07 21:20:34,616][67838] Updated weights for policy 0, policy_version 39902 (0.0010) [2023-10-07 21:20:35,970][67871] Updated weights for policy 1, policy_version 39940 (0.0010) [2023-10-07 21:20:36,345][67871] Updated weights for policy 1, policy_version 39950 (0.0009) [2023-10-07 21:20:36,725][67871] Updated weights for policy 1, policy_version 39960 (0.0008) [2023-10-07 21:20:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 81788928. Throughput: 0: 1661.9, 1: 1659.6. Samples: 20455702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:20:37,477][66916] Avg episode reward: [(0, '39.330'), (1, '48.380')] [2023-10-07 21:20:38,656][67838] Updated weights for policy 0, policy_version 39912 (0.0010) [2023-10-07 21:20:39,020][67838] Updated weights for policy 0, policy_version 39922 (0.0008) [2023-10-07 21:20:39,403][67838] Updated weights for policy 0, policy_version 39932 (0.0007) [2023-10-07 21:20:40,721][67871] Updated weights for policy 1, policy_version 39970 (0.0008) [2023-10-07 21:20:41,087][67871] Updated weights for policy 1, policy_version 39980 (0.0008) [2023-10-07 21:20:41,452][67871] Updated weights for policy 1, policy_version 39990 (0.0009) [2023-10-07 21:20:41,816][67871] Updated weights for policy 1, policy_version 40000 (0.0007) [2023-10-07 21:20:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 81854464. Throughput: 0: 1655.3, 1: 1671.9. Samples: 20465964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:20:42,477][66916] Avg episode reward: [(0, '40.220'), (1, '49.230')] [2023-10-07 21:20:43,527][67838] Updated weights for policy 0, policy_version 39942 (0.0007) [2023-10-07 21:20:43,900][67838] Updated weights for policy 0, policy_version 39952 (0.0011) [2023-10-07 21:20:44,275][67838] Updated weights for policy 0, policy_version 39962 (0.0009) [2023-10-07 21:20:45,969][67871] Updated weights for policy 1, policy_version 40010 (0.0010) [2023-10-07 21:20:46,331][67871] Updated weights for policy 1, policy_version 40020 (0.0009) [2023-10-07 21:20:46,703][67871] Updated weights for policy 1, policy_version 40030 (0.0007) [2023-10-07 21:20:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 81920000. Throughput: 0: 1665.6, 1: 1664.1. Samples: 20486136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:20:47,477][66916] Avg episode reward: [(0, '39.380'), (1, '47.950')] [2023-10-07 21:20:48,440][67838] Updated weights for policy 0, policy_version 39972 (0.0008) [2023-10-07 21:20:48,813][67838] Updated weights for policy 0, policy_version 39982 (0.0009) [2023-10-07 21:20:49,178][67838] Updated weights for policy 0, policy_version 39992 (0.0009) [2023-10-07 21:20:50,658][67871] Updated weights for policy 1, policy_version 40040 (0.0009) [2023-10-07 21:20:51,017][67871] Updated weights for policy 1, policy_version 40050 (0.0010) [2023-10-07 21:20:51,396][67871] Updated weights for policy 1, policy_version 40060 (0.0008) [2023-10-07 21:20:52,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 81985536. Throughput: 0: 1666.1, 1: 1661.2. Samples: 20505676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:20:52,478][66916] Avg episode reward: [(0, '41.450'), (1, '48.050')] [2023-10-07 21:20:53,360][67838] Updated weights for policy 0, policy_version 40002 (0.0008) [2023-10-07 21:20:53,730][67838] Updated weights for policy 0, policy_version 40012 (0.0010) [2023-10-07 21:20:54,118][67838] Updated weights for policy 0, policy_version 40022 (0.0008) [2023-10-07 21:20:54,484][67838] Updated weights for policy 0, policy_version 40032 (0.0007) [2023-10-07 21:20:55,580][67871] Updated weights for policy 1, policy_version 40070 (0.0008) [2023-10-07 21:20:55,940][67871] Updated weights for policy 1, policy_version 40080 (0.0010) [2023-10-07 21:20:56,319][67871] Updated weights for policy 1, policy_version 40090 (0.0009) [2023-10-07 21:20:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82051072. Throughput: 0: 1655.8, 1: 1667.8. Samples: 20515744. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-07 21:20:57,478][66916] Avg episode reward: [(0, '38.800'), (1, '48.540')] [2023-10-07 21:20:58,867][67838] Updated weights for policy 0, policy_version 40042 (0.0007) [2023-10-07 21:20:59,236][67838] Updated weights for policy 0, policy_version 40052 (0.0007) [2023-10-07 21:20:59,616][67838] Updated weights for policy 0, policy_version 40062 (0.0007) [2023-10-07 21:21:00,463][67871] Updated weights for policy 1, policy_version 40100 (0.0007) [2023-10-07 21:21:00,825][67871] Updated weights for policy 1, policy_version 40110 (0.0008) [2023-10-07 21:21:01,197][67871] Updated weights for policy 1, policy_version 40120 (0.0008) [2023-10-07 21:21:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82116608. Throughput: 0: 1660.4, 1: 1657.8. Samples: 20535410. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-07 21:21:02,478][66916] Avg episode reward: [(0, '42.300'), (1, '49.750')] [2023-10-07 21:21:03,854][67838] Updated weights for policy 0, policy_version 40072 (0.0008) [2023-10-07 21:21:04,231][67838] Updated weights for policy 0, policy_version 40082 (0.0008) [2023-10-07 21:21:04,599][67838] Updated weights for policy 0, policy_version 40092 (0.0008) [2023-10-07 21:21:05,198][67871] Updated weights for policy 1, policy_version 40130 (0.0008) [2023-10-07 21:21:05,560][67871] Updated weights for policy 1, policy_version 40140 (0.0010) [2023-10-07 21:21:05,936][67871] Updated weights for policy 1, policy_version 40150 (0.0008) [2023-10-07 21:21:06,307][67871] Updated weights for policy 1, policy_version 40160 (0.0007) [2023-10-07 21:21:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82182144. Throughput: 0: 1662.1, 1: 1671.8. Samples: 20555654. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-07 21:21:07,477][66916] Avg episode reward: [(0, '37.810'), (1, '49.210')] [2023-10-07 21:21:07,486][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000040160_41123840.pth... [2023-10-07 21:21:07,486][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000040096_41058304.pth... [2023-10-07 21:21:07,525][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000038560_39485440.pth [2023-10-07 21:21:07,527][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000038592_39518208.pth [2023-10-07 21:21:08,745][67838] Updated weights for policy 0, policy_version 40102 (0.0008) [2023-10-07 21:21:09,121][67838] Updated weights for policy 0, policy_version 40112 (0.0007) [2023-10-07 21:21:09,483][67838] Updated weights for policy 0, policy_version 40122 (0.0007) [2023-10-07 21:21:10,172][67871] Updated weights for policy 1, policy_version 40170 (0.0009) [2023-10-07 21:21:10,539][67871] Updated weights for policy 1, policy_version 40180 (0.0008) [2023-10-07 21:21:10,905][67871] Updated weights for policy 1, policy_version 40190 (0.0009) [2023-10-07 21:21:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 82247680. Throughput: 0: 1661.5, 1: 1684.1. Samples: 20565970. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-07 21:21:12,478][66916] Avg episode reward: [(0, '40.120'), (1, '51.080')] [2023-10-07 21:21:13,341][67838] Updated weights for policy 0, policy_version 40132 (0.0007) [2023-10-07 21:21:13,706][67838] Updated weights for policy 0, policy_version 40142 (0.0008) [2023-10-07 21:21:14,083][67838] Updated weights for policy 0, policy_version 40152 (0.0009) [2023-10-07 21:21:15,132][67871] Updated weights for policy 1, policy_version 40200 (0.0011) [2023-10-07 21:21:15,504][67871] Updated weights for policy 1, policy_version 40210 (0.0009) [2023-10-07 21:21:15,871][67871] Updated weights for policy 1, policy_version 40220 (0.0008) [2023-10-07 21:21:17,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82313216. Throughput: 0: 1664.8, 1: 1659.6. Samples: 20585422. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-07 21:21:17,477][66916] Avg episode reward: [(0, '41.360'), (1, '46.710')] [2023-10-07 21:21:18,123][67838] Updated weights for policy 0, policy_version 40162 (0.0008) [2023-10-07 21:21:18,505][67838] Updated weights for policy 0, policy_version 40172 (0.0009) [2023-10-07 21:21:18,875][67838] Updated weights for policy 0, policy_version 40182 (0.0008) [2023-10-07 21:21:19,251][67838] Updated weights for policy 0, policy_version 40192 (0.0008) [2023-10-07 21:21:20,026][67871] Updated weights for policy 1, policy_version 40230 (0.0007) [2023-10-07 21:21:20,397][67871] Updated weights for policy 1, policy_version 40240 (0.0009) [2023-10-07 21:21:20,762][67871] Updated weights for policy 1, policy_version 40250 (0.0008) [2023-10-07 21:21:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 82378752. Throughput: 0: 1661.5, 1: 1678.3. Samples: 20605994. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-07 21:21:22,478][66916] Avg episode reward: [(0, '38.890'), (1, '45.230')] [2023-10-07 21:21:23,261][67838] Updated weights for policy 0, policy_version 40202 (0.0010) [2023-10-07 21:21:23,647][67838] Updated weights for policy 0, policy_version 40212 (0.0012) [2023-10-07 21:21:24,014][67838] Updated weights for policy 0, policy_version 40222 (0.0009) [2023-10-07 21:21:24,877][67871] Updated weights for policy 1, policy_version 40260 (0.0007) [2023-10-07 21:21:25,243][67871] Updated weights for policy 1, policy_version 40270 (0.0008) [2023-10-07 21:21:25,600][67871] Updated weights for policy 1, policy_version 40280 (0.0010) [2023-10-07 21:21:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82444288. Throughput: 0: 1659.6, 1: 1675.7. Samples: 20616054. Policy #0 lag: (min: 9.0, avg: 14.2, max: 41.0) [2023-10-07 21:21:27,478][66916] Avg episode reward: [(0, '42.120'), (1, '46.140')] [2023-10-07 21:21:28,365][67838] Updated weights for policy 0, policy_version 40232 (0.0008) [2023-10-07 21:21:28,737][67838] Updated weights for policy 0, policy_version 40242 (0.0007) [2023-10-07 21:21:29,103][67838] Updated weights for policy 0, policy_version 40252 (0.0010) [2023-10-07 21:21:29,763][67871] Updated weights for policy 1, policy_version 40290 (0.0010) [2023-10-07 21:21:30,129][67871] Updated weights for policy 1, policy_version 40300 (0.0007) [2023-10-07 21:21:30,487][67871] Updated weights for policy 1, policy_version 40310 (0.0009) [2023-10-07 21:21:30,861][67871] Updated weights for policy 1, policy_version 40320 (0.0007) [2023-10-07 21:21:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82509824. Throughput: 0: 1662.2, 1: 1655.1. Samples: 20635416. Policy #0 lag: (min: 9.0, avg: 14.2, max: 41.0) [2023-10-07 21:21:32,477][66916] Avg episode reward: [(0, '41.660'), (1, '45.140')] [2023-10-07 21:21:33,033][67838] Updated weights for policy 0, policy_version 40262 (0.0007) [2023-10-07 21:21:33,407][67838] Updated weights for policy 0, policy_version 40272 (0.0007) [2023-10-07 21:21:33,789][67838] Updated weights for policy 0, policy_version 40282 (0.0007) [2023-10-07 21:21:34,992][67871] Updated weights for policy 1, policy_version 40330 (0.0007) [2023-10-07 21:21:35,369][67871] Updated weights for policy 1, policy_version 40340 (0.0008) [2023-10-07 21:21:35,742][67871] Updated weights for policy 1, policy_version 40350 (0.0010) [2023-10-07 21:21:37,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 82575360. Throughput: 0: 1665.1, 1: 1673.8. Samples: 20655926. Policy #0 lag: (min: 9.0, avg: 14.2, max: 41.0) [2023-10-07 21:21:37,478][66916] Avg episode reward: [(0, '40.750'), (1, '49.150')] [2023-10-07 21:21:37,867][67838] Updated weights for policy 0, policy_version 40292 (0.0007) [2023-10-07 21:21:38,250][67838] Updated weights for policy 0, policy_version 40302 (0.0012) [2023-10-07 21:21:38,620][67838] Updated weights for policy 0, policy_version 40312 (0.0009) [2023-10-07 21:21:39,673][67871] Updated weights for policy 1, policy_version 40360 (0.0007) [2023-10-07 21:21:40,044][67871] Updated weights for policy 1, policy_version 40370 (0.0007) [2023-10-07 21:21:40,415][67871] Updated weights for policy 1, policy_version 40380 (0.0010) [2023-10-07 21:21:42,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 82640896. Throughput: 0: 1666.7, 1: 1664.4. Samples: 20665642. Policy #0 lag: (min: 9.0, avg: 14.2, max: 41.0) [2023-10-07 21:21:42,478][66916] Avg episode reward: [(0, '39.750'), (1, '51.740')] [2023-10-07 21:21:42,815][67838] Updated weights for policy 0, policy_version 40322 (0.0009) [2023-10-07 21:21:43,186][67838] Updated weights for policy 0, policy_version 40332 (0.0010) [2023-10-07 21:21:43,552][67838] Updated weights for policy 0, policy_version 40342 (0.0007) [2023-10-07 21:21:43,922][67838] Updated weights for policy 0, policy_version 40352 (0.0009) [2023-10-07 21:21:44,466][67871] Updated weights for policy 1, policy_version 40390 (0.0010) [2023-10-07 21:21:44,838][67871] Updated weights for policy 1, policy_version 40400 (0.0009) [2023-10-07 21:21:45,205][67871] Updated weights for policy 1, policy_version 40410 (0.0010) [2023-10-07 21:21:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82706432. Throughput: 0: 1670.5, 1: 1658.9. Samples: 20685232. Policy #0 lag: (min: 9.0, avg: 14.2, max: 41.0) [2023-10-07 21:21:47,477][66916] Avg episode reward: [(0, '39.060'), (1, '50.850')] [2023-10-07 21:21:48,221][67838] Updated weights for policy 0, policy_version 40362 (0.0008) [2023-10-07 21:21:48,587][67838] Updated weights for policy 0, policy_version 40372 (0.0008) [2023-10-07 21:21:48,956][67838] Updated weights for policy 0, policy_version 40382 (0.0008) [2023-10-07 21:21:49,245][67871] Updated weights for policy 1, policy_version 40420 (0.0007) [2023-10-07 21:21:49,618][67871] Updated weights for policy 1, policy_version 40430 (0.0008) [2023-10-07 21:21:49,988][67871] Updated weights for policy 1, policy_version 40440 (0.0008) [2023-10-07 21:21:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 82771968. Throughput: 0: 1662.3, 1: 1669.6. Samples: 20705588. Policy #0 lag: (min: 9.0, avg: 14.2, max: 41.0) [2023-10-07 21:21:52,478][66916] Avg episode reward: [(0, '38.550'), (1, '49.080')] [2023-10-07 21:21:53,201][67838] Updated weights for policy 0, policy_version 40392 (0.0008) [2023-10-07 21:21:53,575][67838] Updated weights for policy 0, policy_version 40402 (0.0008) [2023-10-07 21:21:53,936][67838] Updated weights for policy 0, policy_version 40412 (0.0007) [2023-10-07 21:21:54,253][67871] Updated weights for policy 1, policy_version 40450 (0.0011) [2023-10-07 21:21:54,681][67871] Updated weights for policy 1, policy_version 40460 (0.0011) [2023-10-07 21:21:55,065][67871] Updated weights for policy 1, policy_version 40470 (0.0009) [2023-10-07 21:21:55,425][67871] Updated weights for policy 1, policy_version 40480 (0.0009) [2023-10-07 21:21:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 82837504. Throughput: 0: 1663.6, 1: 1649.2. Samples: 20715044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:21:57,477][66916] Avg episode reward: [(0, '38.510'), (1, '48.000')] [2023-10-07 21:21:57,971][67838] Updated weights for policy 0, policy_version 40422 (0.0009) [2023-10-07 21:21:58,351][67838] Updated weights for policy 0, policy_version 40432 (0.0009) [2023-10-07 21:21:58,724][67838] Updated weights for policy 0, policy_version 40442 (0.0010) [2023-10-07 21:21:59,577][67871] Updated weights for policy 1, policy_version 40490 (0.0007) [2023-10-07 21:21:59,949][67871] Updated weights for policy 1, policy_version 40500 (0.0009) [2023-10-07 21:22:00,316][67871] Updated weights for policy 1, policy_version 40510 (0.0009) [2023-10-07 21:22:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82903040. Throughput: 0: 1663.4, 1: 1658.8. Samples: 20734918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:22:02,477][66916] Avg episode reward: [(0, '41.110'), (1, '43.190')] [2023-10-07 21:22:02,851][67838] Updated weights for policy 0, policy_version 40452 (0.0009) [2023-10-07 21:22:03,229][67838] Updated weights for policy 0, policy_version 40462 (0.0009) [2023-10-07 21:22:03,612][67838] Updated weights for policy 0, policy_version 40472 (0.0012) [2023-10-07 21:22:04,416][67871] Updated weights for policy 1, policy_version 40520 (0.0008) [2023-10-07 21:22:04,786][67871] Updated weights for policy 1, policy_version 40530 (0.0007) [2023-10-07 21:22:05,145][67871] Updated weights for policy 1, policy_version 40540 (0.0009) [2023-10-07 21:22:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82968576. Throughput: 0: 1660.6, 1: 1660.1. Samples: 20755426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:22:07,477][66916] Avg episode reward: [(0, '39.820'), (1, '47.710')] [2023-10-07 21:22:07,634][67838] Updated weights for policy 0, policy_version 40482 (0.0009) [2023-10-07 21:22:08,016][67838] Updated weights for policy 0, policy_version 40492 (0.0010) [2023-10-07 21:22:08,384][67838] Updated weights for policy 0, policy_version 40502 (0.0010) [2023-10-07 21:22:08,762][67838] Updated weights for policy 0, policy_version 40512 (0.0008) [2023-10-07 21:22:09,306][67871] Updated weights for policy 1, policy_version 40550 (0.0008) [2023-10-07 21:22:09,672][67871] Updated weights for policy 1, policy_version 40560 (0.0009) [2023-10-07 21:22:10,047][67871] Updated weights for policy 1, policy_version 40570 (0.0007) [2023-10-07 21:22:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83034112. Throughput: 0: 1665.1, 1: 1646.9. Samples: 20765094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:22:12,477][66916] Avg episode reward: [(0, '41.150'), (1, '46.740')] [2023-10-07 21:22:12,866][67838] Updated weights for policy 0, policy_version 40522 (0.0010) [2023-10-07 21:22:13,236][67838] Updated weights for policy 0, policy_version 40532 (0.0011) [2023-10-07 21:22:13,616][67838] Updated weights for policy 0, policy_version 40542 (0.0008) [2023-10-07 21:22:14,135][67871] Updated weights for policy 1, policy_version 40580 (0.0009) [2023-10-07 21:22:14,504][67871] Updated weights for policy 1, policy_version 40590 (0.0010) [2023-10-07 21:22:14,868][67871] Updated weights for policy 1, policy_version 40600 (0.0009) [2023-10-07 21:22:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83099648. Throughput: 0: 1664.3, 1: 1659.2. Samples: 20784974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:22:17,477][66916] Avg episode reward: [(0, '40.550'), (1, '45.480')] [2023-10-07 21:22:17,521][67838] Updated weights for policy 0, policy_version 40552 (0.0009) [2023-10-07 21:22:17,896][67838] Updated weights for policy 0, policy_version 40562 (0.0009) [2023-10-07 21:22:18,274][67838] Updated weights for policy 0, policy_version 40572 (0.0008) [2023-10-07 21:22:19,065][67871] Updated weights for policy 1, policy_version 40610 (0.0009) [2023-10-07 21:22:19,433][67871] Updated weights for policy 1, policy_version 40620 (0.0007) [2023-10-07 21:22:19,801][67871] Updated weights for policy 1, policy_version 40630 (0.0009) [2023-10-07 21:22:20,170][67871] Updated weights for policy 1, policy_version 40640 (0.0008) [2023-10-07 21:22:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83165184. Throughput: 0: 1661.1, 1: 1660.5. Samples: 20805398. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:22:22,477][66916] Avg episode reward: [(0, '46.850'), (1, '45.190')] [2023-10-07 21:22:22,488][67838] Updated weights for policy 0, policy_version 40582 (0.0007) [2023-10-07 21:22:22,863][67838] Updated weights for policy 0, policy_version 40592 (0.0010) [2023-10-07 21:22:23,242][67838] Updated weights for policy 0, policy_version 40602 (0.0009) [2023-10-07 21:22:24,308][67871] Updated weights for policy 1, policy_version 40650 (0.0007) [2023-10-07 21:22:24,691][67871] Updated weights for policy 1, policy_version 40660 (0.0007) [2023-10-07 21:22:25,058][67871] Updated weights for policy 1, policy_version 40670 (0.0007) [2023-10-07 21:22:27,450][67838] Updated weights for policy 0, policy_version 40612 (0.0009) [2023-10-07 21:22:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83230720. Throughput: 0: 1659.8, 1: 1648.5. Samples: 20814518. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-10-07 21:22:27,477][66916] Avg episode reward: [(0, '43.070'), (1, '44.940')] [2023-10-07 21:22:27,814][67838] Updated weights for policy 0, policy_version 40622 (0.0009) [2023-10-07 21:22:28,194][67838] Updated weights for policy 0, policy_version 40632 (0.0009) [2023-10-07 21:22:29,214][67871] Updated weights for policy 1, policy_version 40680 (0.0008) [2023-10-07 21:22:29,596][67871] Updated weights for policy 1, policy_version 40690 (0.0007) [2023-10-07 21:22:29,962][67871] Updated weights for policy 1, policy_version 40700 (0.0008) [2023-10-07 21:22:32,277][67838] Updated weights for policy 0, policy_version 40642 (0.0008) [2023-10-07 21:22:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83296256. Throughput: 0: 1661.8, 1: 1655.4. Samples: 20834508. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-10-07 21:22:32,478][66916] Avg episode reward: [(0, '45.370'), (1, '47.270')] [2023-10-07 21:22:32,653][67838] Updated weights for policy 0, policy_version 40652 (0.0007) [2023-10-07 21:22:33,021][67838] Updated weights for policy 0, policy_version 40662 (0.0008) [2023-10-07 21:22:33,396][67838] Updated weights for policy 0, policy_version 40672 (0.0008) [2023-10-07 21:22:34,059][67871] Updated weights for policy 1, policy_version 40710 (0.0008) [2023-10-07 21:22:34,427][67871] Updated weights for policy 1, policy_version 40720 (0.0007) [2023-10-07 21:22:34,799][67871] Updated weights for policy 1, policy_version 40730 (0.0009) [2023-10-07 21:22:37,294][67838] Updated weights for policy 0, policy_version 40682 (0.0009) [2023-10-07 21:22:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 83361792. Throughput: 0: 1664.1, 1: 1653.0. Samples: 20854858. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-10-07 21:22:37,478][66916] Avg episode reward: [(0, '43.070'), (1, '47.040')] [2023-10-07 21:22:37,671][67838] Updated weights for policy 0, policy_version 40692 (0.0009) [2023-10-07 21:22:38,044][67838] Updated weights for policy 0, policy_version 40702 (0.0008) [2023-10-07 21:22:39,047][67871] Updated weights for policy 1, policy_version 40740 (0.0010) [2023-10-07 21:22:39,446][67871] Updated weights for policy 1, policy_version 40750 (0.0007) [2023-10-07 21:22:39,817][67871] Updated weights for policy 1, policy_version 40760 (0.0007) [2023-10-07 21:22:42,062][67838] Updated weights for policy 0, policy_version 40712 (0.0008) [2023-10-07 21:22:42,421][67838] Updated weights for policy 0, policy_version 40722 (0.0007) [2023-10-07 21:22:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83427328. Throughput: 0: 1671.0, 1: 1647.9. Samples: 20864394. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-10-07 21:22:42,477][66916] Avg episode reward: [(0, '40.350'), (1, '47.920')] [2023-10-07 21:22:42,797][67838] Updated weights for policy 0, policy_version 40732 (0.0009) [2023-10-07 21:22:44,046][67871] Updated weights for policy 1, policy_version 40770 (0.0008) [2023-10-07 21:22:44,423][67871] Updated weights for policy 1, policy_version 40780 (0.0007) [2023-10-07 21:22:44,783][67871] Updated weights for policy 1, policy_version 40790 (0.0008) [2023-10-07 21:22:45,156][67871] Updated weights for policy 1, policy_version 40800 (0.0007) [2023-10-07 21:22:47,073][67838] Updated weights for policy 0, policy_version 40742 (0.0010) [2023-10-07 21:22:47,441][67838] Updated weights for policy 0, policy_version 40752 (0.0008) [2023-10-07 21:22:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83492864. Throughput: 0: 1668.3, 1: 1651.1. Samples: 20884292. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-10-07 21:22:47,477][66916] Avg episode reward: [(0, '39.220'), (1, '48.860')] [2023-10-07 21:22:47,810][67838] Updated weights for policy 0, policy_version 40762 (0.0010) [2023-10-07 21:22:49,368][67871] Updated weights for policy 1, policy_version 40810 (0.0008) [2023-10-07 21:22:49,730][67871] Updated weights for policy 1, policy_version 40820 (0.0007) [2023-10-07 21:22:50,096][67871] Updated weights for policy 1, policy_version 40830 (0.0007) [2023-10-07 21:22:51,954][67838] Updated weights for policy 0, policy_version 40772 (0.0008) [2023-10-07 21:22:52,327][67838] Updated weights for policy 0, policy_version 40782 (0.0008) [2023-10-07 21:22:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 83558400. Throughput: 0: 1664.3, 1: 1646.3. Samples: 20904408. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-10-07 21:22:52,478][66916] Avg episode reward: [(0, '38.400'), (1, '47.700')] [2023-10-07 21:22:52,714][67838] Updated weights for policy 0, policy_version 40792 (0.0008) [2023-10-07 21:22:54,127][67871] Updated weights for policy 1, policy_version 40840 (0.0009) [2023-10-07 21:22:54,496][67871] Updated weights for policy 1, policy_version 40850 (0.0010) [2023-10-07 21:22:54,869][67871] Updated weights for policy 1, policy_version 40860 (0.0007) [2023-10-07 21:22:56,700][67838] Updated weights for policy 0, policy_version 40802 (0.0010) [2023-10-07 21:22:57,062][67838] Updated weights for policy 0, policy_version 40812 (0.0008) [2023-10-07 21:22:57,434][67838] Updated weights for policy 0, policy_version 40822 (0.0010) [2023-10-07 21:22:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83623936. Throughput: 0: 1668.9, 1: 1641.7. Samples: 20914072. Policy #0 lag: (min: 31.0, avg: 32.8, max: 62.0) [2023-10-07 21:22:57,477][66916] Avg episode reward: [(0, '37.470'), (1, '47.200')] [2023-10-07 21:22:57,810][67838] Updated weights for policy 0, policy_version 40832 (0.0010) [2023-10-07 21:22:59,062][67871] Updated weights for policy 1, policy_version 40870 (0.0011) [2023-10-07 21:22:59,436][67871] Updated weights for policy 1, policy_version 40880 (0.0011) [2023-10-07 21:22:59,808][67871] Updated weights for policy 1, policy_version 40890 (0.0008) [2023-10-07 21:23:01,790][67838] Updated weights for policy 0, policy_version 40842 (0.0009) [2023-10-07 21:23:02,154][67838] Updated weights for policy 0, policy_version 40852 (0.0011) [2023-10-07 21:23:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83689472. Throughput: 0: 1668.4, 1: 1647.8. Samples: 20934204. Policy #0 lag: (min: 31.0, avg: 32.8, max: 62.0) [2023-10-07 21:23:02,477][66916] Avg episode reward: [(0, '39.050'), (1, '46.050')] [2023-10-07 21:23:02,547][67838] Updated weights for policy 0, policy_version 40862 (0.0011) [2023-10-07 21:23:03,896][67871] Updated weights for policy 1, policy_version 40900 (0.0010) [2023-10-07 21:23:04,268][67871] Updated weights for policy 1, policy_version 40910 (0.0010) [2023-10-07 21:23:04,641][67871] Updated weights for policy 1, policy_version 40920 (0.0011) [2023-10-07 21:23:06,711][67838] Updated weights for policy 0, policy_version 40872 (0.0011) [2023-10-07 21:23:07,068][67838] Updated weights for policy 0, policy_version 40882 (0.0011) [2023-10-07 21:23:07,435][67838] Updated weights for policy 0, policy_version 40892 (0.0009) [2023-10-07 21:23:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83755008. Throughput: 0: 1655.0, 1: 1651.8. Samples: 20954204. Policy #0 lag: (min: 31.0, avg: 32.8, max: 62.0) [2023-10-07 21:23:07,477][66916] Avg episode reward: [(0, '39.290'), (1, '48.240')] [2023-10-07 21:23:07,485][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000040928_41910272.pth... [2023-10-07 21:23:07,517][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000039392_40337408.pth [2023-10-07 21:23:07,581][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000040896_41877504.pth... [2023-10-07 21:23:07,610][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000039328_40271872.pth [2023-10-07 21:23:08,619][67871] Updated weights for policy 1, policy_version 40930 (0.0010) [2023-10-07 21:23:08,994][67871] Updated weights for policy 1, policy_version 40940 (0.0009) [2023-10-07 21:23:09,356][67871] Updated weights for policy 1, policy_version 40950 (0.0008) [2023-10-07 21:23:09,720][67871] Updated weights for policy 1, policy_version 40960 (0.0007) [2023-10-07 21:23:11,685][67838] Updated weights for policy 0, policy_version 40902 (0.0009) [2023-10-07 21:23:12,062][67838] Updated weights for policy 0, policy_version 40912 (0.0011) [2023-10-07 21:23:12,425][67838] Updated weights for policy 0, policy_version 40922 (0.0009) [2023-10-07 21:23:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83820544. Throughput: 0: 1672.4, 1: 1646.8. Samples: 20963880. Policy #0 lag: (min: 31.0, avg: 32.8, max: 62.0) [2023-10-07 21:23:12,477][66916] Avg episode reward: [(0, '42.500'), (1, '48.870')] [2023-10-07 21:23:13,859][67871] Updated weights for policy 1, policy_version 40970 (0.0009) [2023-10-07 21:23:14,226][67871] Updated weights for policy 1, policy_version 40980 (0.0009) [2023-10-07 21:23:14,594][67871] Updated weights for policy 1, policy_version 40990 (0.0010) [2023-10-07 21:23:16,375][67838] Updated weights for policy 0, policy_version 40932 (0.0009) [2023-10-07 21:23:16,749][67838] Updated weights for policy 0, policy_version 40942 (0.0009) [2023-10-07 21:23:17,108][67838] Updated weights for policy 0, policy_version 40952 (0.0008) [2023-10-07 21:23:17,476][66916] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 83918848. Throughput: 0: 1672.9, 1: 1657.5. Samples: 20984378. Policy #0 lag: (min: 31.0, avg: 32.8, max: 62.0) [2023-10-07 21:23:17,477][66916] Avg episode reward: [(0, '45.000'), (1, '50.140')] [2023-10-07 21:23:18,491][67871] Updated weights for policy 1, policy_version 41000 (0.0007) [2023-10-07 21:23:18,869][67871] Updated weights for policy 1, policy_version 41010 (0.0008) [2023-10-07 21:23:19,236][67871] Updated weights for policy 1, policy_version 41020 (0.0007) [2023-10-07 21:23:21,452][67838] Updated weights for policy 0, policy_version 40962 (0.0008) [2023-10-07 21:23:21,857][67838] Updated weights for policy 0, policy_version 40972 (0.0008) [2023-10-07 21:23:22,236][67838] Updated weights for policy 0, policy_version 40982 (0.0009) [2023-10-07 21:23:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83951616. Throughput: 0: 1651.4, 1: 1664.1. Samples: 21004058. Policy #0 lag: (min: 31.0, avg: 32.8, max: 62.0) [2023-10-07 21:23:22,478][66916] Avg episode reward: [(0, '41.960'), (1, '50.710')] [2023-10-07 21:23:22,603][67838] Updated weights for policy 0, policy_version 40992 (0.0008) [2023-10-07 21:23:23,389][67871] Updated weights for policy 1, policy_version 41030 (0.0009) [2023-10-07 21:23:23,756][67871] Updated weights for policy 1, policy_version 41040 (0.0009) [2023-10-07 21:23:24,114][67871] Updated weights for policy 1, policy_version 41050 (0.0008) [2023-10-07 21:23:26,648][67838] Updated weights for policy 0, policy_version 41002 (0.0008) [2023-10-07 21:23:27,024][67838] Updated weights for policy 0, policy_version 41012 (0.0007) [2023-10-07 21:23:27,407][67838] Updated weights for policy 0, policy_version 41022 (0.0011) [2023-10-07 21:23:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 84049920. Throughput: 0: 1657.3, 1: 1659.2. Samples: 21013634. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:23:27,477][66916] Avg episode reward: [(0, '40.410'), (1, '50.380')] [2023-10-07 21:23:28,138][67871] Updated weights for policy 1, policy_version 41060 (0.0008) [2023-10-07 21:23:28,543][67871] Updated weights for policy 1, policy_version 41070 (0.0007) [2023-10-07 21:23:28,907][67871] Updated weights for policy 1, policy_version 41080 (0.0009) [2023-10-07 21:23:31,566][67838] Updated weights for policy 0, policy_version 41032 (0.0008) [2023-10-07 21:23:31,946][67838] Updated weights for policy 0, policy_version 41042 (0.0007) [2023-10-07 21:23:32,322][67838] Updated weights for policy 0, policy_version 41052 (0.0008) [2023-10-07 21:23:32,476][66916] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 84115456. Throughput: 0: 1660.9, 1: 1669.3. Samples: 21034154. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:23:32,477][66916] Avg episode reward: [(0, '40.640'), (1, '48.240')] [2023-10-07 21:23:33,138][67871] Updated weights for policy 1, policy_version 41090 (0.0008) [2023-10-07 21:23:33,508][67871] Updated weights for policy 1, policy_version 41100 (0.0009) [2023-10-07 21:23:33,870][67871] Updated weights for policy 1, policy_version 41110 (0.0009) [2023-10-07 21:23:34,236][67871] Updated weights for policy 1, policy_version 41120 (0.0008) [2023-10-07 21:23:36,486][67838] Updated weights for policy 0, policy_version 41062 (0.0008) [2023-10-07 21:23:36,864][67838] Updated weights for policy 0, policy_version 41072 (0.0009) [2023-10-07 21:23:37,231][67838] Updated weights for policy 0, policy_version 41082 (0.0007) [2023-10-07 21:23:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 84180992. Throughput: 0: 1646.0, 1: 1672.5. Samples: 21053740. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:23:37,477][66916] Avg episode reward: [(0, '40.170'), (1, '46.640')] [2023-10-07 21:23:38,266][67871] Updated weights for policy 1, policy_version 41130 (0.0007) [2023-10-07 21:23:38,626][67871] Updated weights for policy 1, policy_version 41140 (0.0007) [2023-10-07 21:23:38,997][67871] Updated weights for policy 1, policy_version 41150 (0.0010) [2023-10-07 21:23:41,173][67838] Updated weights for policy 0, policy_version 41092 (0.0008) [2023-10-07 21:23:41,552][67838] Updated weights for policy 0, policy_version 41102 (0.0008) [2023-10-07 21:23:41,939][67838] Updated weights for policy 0, policy_version 41112 (0.0010) [2023-10-07 21:23:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 84246528. Throughput: 0: 1658.5, 1: 1663.9. Samples: 21063580. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:23:42,477][66916] Avg episode reward: [(0, '43.140'), (1, '46.970')] [2023-10-07 21:23:43,189][67871] Updated weights for policy 1, policy_version 41160 (0.0010) [2023-10-07 21:23:43,554][67871] Updated weights for policy 1, policy_version 41170 (0.0010) [2023-10-07 21:23:43,927][67871] Updated weights for policy 1, policy_version 41180 (0.0007) [2023-10-07 21:23:46,095][67838] Updated weights for policy 0, policy_version 41122 (0.0009) [2023-10-07 21:23:46,475][67838] Updated weights for policy 0, policy_version 41132 (0.0008) [2023-10-07 21:23:46,846][67838] Updated weights for policy 0, policy_version 41142 (0.0008) [2023-10-07 21:23:47,228][67838] Updated weights for policy 0, policy_version 41152 (0.0008) [2023-10-07 21:23:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 84312064. Throughput: 0: 1655.6, 1: 1674.0. Samples: 21084034. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:23:47,477][66916] Avg episode reward: [(0, '42.820'), (1, '45.990')] [2023-10-07 21:23:48,056][67871] Updated weights for policy 1, policy_version 41190 (0.0009) [2023-10-07 21:23:48,421][67871] Updated weights for policy 1, policy_version 41200 (0.0009) [2023-10-07 21:23:48,792][67871] Updated weights for policy 1, policy_version 41210 (0.0009) [2023-10-07 21:23:51,497][67838] Updated weights for policy 0, policy_version 41162 (0.0007) [2023-10-07 21:23:51,867][67838] Updated weights for policy 0, policy_version 41172 (0.0009) [2023-10-07 21:23:52,237][67838] Updated weights for policy 0, policy_version 41182 (0.0011) [2023-10-07 21:23:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 84377600. Throughput: 0: 1650.9, 1: 1670.7. Samples: 21103674. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 21:23:52,477][66916] Avg episode reward: [(0, '41.890'), (1, '48.890')] [2023-10-07 21:23:52,888][67871] Updated weights for policy 1, policy_version 41220 (0.0009) [2023-10-07 21:23:53,254][67871] Updated weights for policy 1, policy_version 41230 (0.0007) [2023-10-07 21:23:53,622][67871] Updated weights for policy 1, policy_version 41240 (0.0007) [2023-10-07 21:23:56,162][67838] Updated weights for policy 0, policy_version 41192 (0.0008) [2023-10-07 21:23:56,540][67838] Updated weights for policy 0, policy_version 41202 (0.0008) [2023-10-07 21:23:56,910][67838] Updated weights for policy 0, policy_version 41212 (0.0007) [2023-10-07 21:23:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 84443136. Throughput: 0: 1657.6, 1: 1672.2. Samples: 21113718. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 21:23:57,477][66916] Avg episode reward: [(0, '41.020'), (1, '46.570')] [2023-10-07 21:23:57,708][67871] Updated weights for policy 1, policy_version 41250 (0.0008) [2023-10-07 21:23:58,080][67871] Updated weights for policy 1, policy_version 41260 (0.0008) [2023-10-07 21:23:58,447][67871] Updated weights for policy 1, policy_version 41270 (0.0008) [2023-10-07 21:23:58,818][67871] Updated weights for policy 1, policy_version 41280 (0.0008) [2023-10-07 21:24:01,033][67838] Updated weights for policy 0, policy_version 41222 (0.0009) [2023-10-07 21:24:01,401][67838] Updated weights for policy 0, policy_version 41232 (0.0010) [2023-10-07 21:24:01,772][67838] Updated weights for policy 0, policy_version 41242 (0.0009) [2023-10-07 21:24:02,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 84508672. Throughput: 0: 1651.5, 1: 1671.4. Samples: 21133912. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 21:24:02,478][66916] Avg episode reward: [(0, '41.950'), (1, '46.350')] [2023-10-07 21:24:02,940][67871] Updated weights for policy 1, policy_version 41290 (0.0008) [2023-10-07 21:24:03,304][67871] Updated weights for policy 1, policy_version 41300 (0.0008) [2023-10-07 21:24:03,677][67871] Updated weights for policy 1, policy_version 41310 (0.0008) [2023-10-07 21:24:05,886][67838] Updated weights for policy 0, policy_version 41252 (0.0008) [2023-10-07 21:24:06,265][67838] Updated weights for policy 0, policy_version 41262 (0.0007) [2023-10-07 21:24:06,632][67838] Updated weights for policy 0, policy_version 41272 (0.0010) [2023-10-07 21:24:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 84574208. Throughput: 0: 1653.9, 1: 1664.9. Samples: 21153402. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 21:24:07,477][66916] Avg episode reward: [(0, '41.300'), (1, '48.960')] [2023-10-07 21:24:07,794][67871] Updated weights for policy 1, policy_version 41320 (0.0007) [2023-10-07 21:24:08,164][67871] Updated weights for policy 1, policy_version 41330 (0.0008) [2023-10-07 21:24:08,535][67871] Updated weights for policy 1, policy_version 41340 (0.0007) [2023-10-07 21:24:10,807][67838] Updated weights for policy 0, policy_version 41282 (0.0009) [2023-10-07 21:24:11,181][67838] Updated weights for policy 0, policy_version 41292 (0.0009) [2023-10-07 21:24:11,562][67838] Updated weights for policy 0, policy_version 41302 (0.0009) [2023-10-07 21:24:11,938][67838] Updated weights for policy 0, policy_version 41312 (0.0009) [2023-10-07 21:24:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 84639744. Throughput: 0: 1668.0, 1: 1663.6. Samples: 21163558. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 21:24:12,478][66916] Avg episode reward: [(0, '41.560'), (1, '48.330')] [2023-10-07 21:24:12,843][67871] Updated weights for policy 1, policy_version 41350 (0.0008) [2023-10-07 21:24:13,209][67871] Updated weights for policy 1, policy_version 41360 (0.0009) [2023-10-07 21:24:13,577][67871] Updated weights for policy 1, policy_version 41370 (0.0009) [2023-10-07 21:24:15,849][67838] Updated weights for policy 0, policy_version 41322 (0.0009) [2023-10-07 21:24:16,220][67838] Updated weights for policy 0, policy_version 41332 (0.0009) [2023-10-07 21:24:16,602][67838] Updated weights for policy 0, policy_version 41342 (0.0007) [2023-10-07 21:24:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84705280. Throughput: 0: 1657.3, 1: 1666.5. Samples: 21183724. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-07 21:24:17,477][66916] Avg episode reward: [(0, '40.020'), (1, '49.800')] [2023-10-07 21:24:17,698][67871] Updated weights for policy 1, policy_version 41380 (0.0007) [2023-10-07 21:24:18,098][67871] Updated weights for policy 1, policy_version 41390 (0.0007) [2023-10-07 21:24:18,462][67871] Updated weights for policy 1, policy_version 41400 (0.0007) [2023-10-07 21:24:20,710][67838] Updated weights for policy 0, policy_version 41352 (0.0008) [2023-10-07 21:24:21,072][67838] Updated weights for policy 0, policy_version 41362 (0.0008) [2023-10-07 21:24:21,455][67838] Updated weights for policy 0, policy_version 41372 (0.0009) [2023-10-07 21:24:22,446][67871] Updated weights for policy 1, policy_version 41410 (0.0007) [2023-10-07 21:24:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 84770816. Throughput: 0: 1662.5, 1: 1664.0. Samples: 21203434. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 21:24:22,478][66916] Avg episode reward: [(0, '40.010'), (1, '51.340')] [2023-10-07 21:24:22,814][67871] Updated weights for policy 1, policy_version 41420 (0.0007) [2023-10-07 21:24:23,173][67871] Updated weights for policy 1, policy_version 41430 (0.0007) [2023-10-07 21:24:23,544][67871] Updated weights for policy 1, policy_version 41440 (0.0007) [2023-10-07 21:24:25,690][67838] Updated weights for policy 0, policy_version 41382 (0.0008) [2023-10-07 21:24:26,060][67838] Updated weights for policy 0, policy_version 41392 (0.0010) [2023-10-07 21:24:26,439][67838] Updated weights for policy 0, policy_version 41402 (0.0009) [2023-10-07 21:24:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84836352. Throughput: 0: 1667.6, 1: 1666.2. Samples: 21213604. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 21:24:27,478][66916] Avg episode reward: [(0, '40.090'), (1, '49.960')] [2023-10-07 21:24:27,592][67871] Updated weights for policy 1, policy_version 41450 (0.0008) [2023-10-07 21:24:27,964][67871] Updated weights for policy 1, policy_version 41460 (0.0010) [2023-10-07 21:24:28,328][67871] Updated weights for policy 1, policy_version 41470 (0.0007) [2023-10-07 21:24:30,558][67838] Updated weights for policy 0, policy_version 41412 (0.0009) [2023-10-07 21:24:30,930][67838] Updated weights for policy 0, policy_version 41422 (0.0008) [2023-10-07 21:24:31,301][67838] Updated weights for policy 0, policy_version 41432 (0.0007) [2023-10-07 21:24:32,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84901888. Throughput: 0: 1655.4, 1: 1668.1. Samples: 21233592. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 21:24:32,477][66916] Avg episode reward: [(0, '42.100'), (1, '46.080')] [2023-10-07 21:24:32,495][67871] Updated weights for policy 1, policy_version 41480 (0.0009) [2023-10-07 21:24:32,871][67871] Updated weights for policy 1, policy_version 41490 (0.0008) [2023-10-07 21:24:33,242][67871] Updated weights for policy 1, policy_version 41500 (0.0008) [2023-10-07 21:24:35,153][67838] Updated weights for policy 0, policy_version 41442 (0.0009) [2023-10-07 21:24:35,536][67838] Updated weights for policy 0, policy_version 41452 (0.0008) [2023-10-07 21:24:35,906][67838] Updated weights for policy 0, policy_version 41462 (0.0007) [2023-10-07 21:24:36,275][67838] Updated weights for policy 0, policy_version 41472 (0.0007) [2023-10-07 21:24:37,456][67871] Updated weights for policy 1, policy_version 41510 (0.0008) [2023-10-07 21:24:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 84967424. Throughput: 0: 1671.5, 1: 1667.5. Samples: 21253932. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 21:24:37,477][66916] Avg episode reward: [(0, '40.730'), (1, '46.690')] [2023-10-07 21:24:37,823][67871] Updated weights for policy 1, policy_version 41520 (0.0008) [2023-10-07 21:24:38,200][67871] Updated weights for policy 1, policy_version 41530 (0.0009) [2023-10-07 21:24:40,346][67838] Updated weights for policy 0, policy_version 41482 (0.0009) [2023-10-07 21:24:40,727][67838] Updated weights for policy 0, policy_version 41492 (0.0008) [2023-10-07 21:24:41,108][67838] Updated weights for policy 0, policy_version 41502 (0.0010) [2023-10-07 21:24:42,248][67871] Updated weights for policy 1, policy_version 41540 (0.0009) [2023-10-07 21:24:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 85032960. Throughput: 0: 1677.4, 1: 1666.5. Samples: 21264194. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 21:24:42,477][66916] Avg episode reward: [(0, '39.740'), (1, '46.640')] [2023-10-07 21:24:42,610][67871] Updated weights for policy 1, policy_version 41550 (0.0008) [2023-10-07 21:24:42,977][67871] Updated weights for policy 1, policy_version 41560 (0.0008) [2023-10-07 21:24:45,036][67838] Updated weights for policy 0, policy_version 41512 (0.0007) [2023-10-07 21:24:45,404][67838] Updated weights for policy 0, policy_version 41522 (0.0007) [2023-10-07 21:24:45,783][67838] Updated weights for policy 0, policy_version 41532 (0.0007) [2023-10-07 21:24:47,112][67871] Updated weights for policy 1, policy_version 41570 (0.0008) [2023-10-07 21:24:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 85098496. Throughput: 0: 1658.5, 1: 1667.0. Samples: 21283560. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 21:24:47,477][66916] Avg episode reward: [(0, '41.930'), (1, '46.390')] [2023-10-07 21:24:47,477][67871] Updated weights for policy 1, policy_version 41580 (0.0008) [2023-10-07 21:24:47,843][67871] Updated weights for policy 1, policy_version 41590 (0.0007) [2023-10-07 21:24:48,211][67871] Updated weights for policy 1, policy_version 41600 (0.0008) [2023-10-07 21:24:49,952][67838] Updated weights for policy 0, policy_version 41542 (0.0009) [2023-10-07 21:24:50,330][67838] Updated weights for policy 0, policy_version 41552 (0.0008) [2023-10-07 21:24:50,706][67838] Updated weights for policy 0, policy_version 41562 (0.0009) [2023-10-07 21:24:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 85164032. Throughput: 0: 1678.1, 1: 1668.0. Samples: 21303978. Policy #0 lag: (min: 13.0, avg: 35.9, max: 40.0) [2023-10-07 21:24:52,477][66916] Avg episode reward: [(0, '39.830'), (1, '45.780')] [2023-10-07 21:24:52,497][67871] Updated weights for policy 1, policy_version 41610 (0.0008) [2023-10-07 21:24:52,857][67871] Updated weights for policy 1, policy_version 41620 (0.0011) [2023-10-07 21:24:53,232][67871] Updated weights for policy 1, policy_version 41630 (0.0010) [2023-10-07 21:24:54,881][67838] Updated weights for policy 0, policy_version 41572 (0.0007) [2023-10-07 21:24:55,255][67838] Updated weights for policy 0, policy_version 41582 (0.0008) [2023-10-07 21:24:55,621][67838] Updated weights for policy 0, policy_version 41592 (0.0010) [2023-10-07 21:24:57,333][67871] Updated weights for policy 1, policy_version 41640 (0.0007) [2023-10-07 21:24:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 85229568. Throughput: 0: 1673.5, 1: 1668.5. Samples: 21313950. Policy #0 lag: (min: 13.0, avg: 35.9, max: 40.0) [2023-10-07 21:24:57,477][66916] Avg episode reward: [(0, '41.640'), (1, '45.570')] [2023-10-07 21:24:57,700][67871] Updated weights for policy 1, policy_version 41650 (0.0007) [2023-10-07 21:24:58,072][67871] Updated weights for policy 1, policy_version 41660 (0.0008) [2023-10-07 21:24:59,718][67838] Updated weights for policy 0, policy_version 41602 (0.0009) [2023-10-07 21:25:00,125][67838] Updated weights for policy 0, policy_version 41612 (0.0007) [2023-10-07 21:25:00,490][67838] Updated weights for policy 0, policy_version 41622 (0.0007) [2023-10-07 21:25:00,864][67838] Updated weights for policy 0, policy_version 41632 (0.0007) [2023-10-07 21:25:02,279][67871] Updated weights for policy 1, policy_version 41670 (0.0009) [2023-10-07 21:25:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 85295104. Throughput: 0: 1660.2, 1: 1669.5. Samples: 21333560. Policy #0 lag: (min: 13.0, avg: 35.9, max: 40.0) [2023-10-07 21:25:02,477][66916] Avg episode reward: [(0, '42.660'), (1, '45.430')] [2023-10-07 21:25:02,671][67871] Updated weights for policy 1, policy_version 41680 (0.0007) [2023-10-07 21:25:03,038][67871] Updated weights for policy 1, policy_version 41690 (0.0007) [2023-10-07 21:25:04,962][67838] Updated weights for policy 0, policy_version 41642 (0.0011) [2023-10-07 21:25:05,346][67838] Updated weights for policy 0, policy_version 41652 (0.0010) [2023-10-07 21:25:05,710][67838] Updated weights for policy 0, policy_version 41662 (0.0009) [2023-10-07 21:25:07,086][67871] Updated weights for policy 1, policy_version 41700 (0.0007) [2023-10-07 21:25:07,448][67871] Updated weights for policy 1, policy_version 41710 (0.0009) [2023-10-07 21:25:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 85360640. Throughput: 0: 1676.1, 1: 1669.3. Samples: 21353976. Policy #0 lag: (min: 13.0, avg: 35.9, max: 40.0) [2023-10-07 21:25:07,477][66916] Avg episode reward: [(0, '42.770'), (1, '47.330')] [2023-10-07 21:25:07,486][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000041664_42663936.pth... [2023-10-07 21:25:07,515][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000040096_41058304.pth [2023-10-07 21:25:07,818][67871] Updated weights for policy 1, policy_version 41720 (0.0009) [2023-10-07 21:25:08,105][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000041728_42729472.pth... [2023-10-07 21:25:08,134][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000040160_41123840.pth [2023-10-07 21:25:09,786][67838] Updated weights for policy 0, policy_version 41672 (0.0007) [2023-10-07 21:25:10,154][67838] Updated weights for policy 0, policy_version 41682 (0.0009) [2023-10-07 21:25:10,522][67838] Updated weights for policy 0, policy_version 41692 (0.0008) [2023-10-07 21:25:11,787][67871] Updated weights for policy 1, policy_version 41730 (0.0008) [2023-10-07 21:25:12,153][67871] Updated weights for policy 1, policy_version 41740 (0.0011) [2023-10-07 21:25:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 85426176. Throughput: 0: 1665.2, 1: 1669.2. Samples: 21363652. Policy #0 lag: (min: 13.0, avg: 35.9, max: 40.0) [2023-10-07 21:25:12,478][66916] Avg episode reward: [(0, '41.450'), (1, '48.410')] [2023-10-07 21:25:12,520][67871] Updated weights for policy 1, policy_version 41750 (0.0009) [2023-10-07 21:25:12,893][67871] Updated weights for policy 1, policy_version 41760 (0.0009) [2023-10-07 21:25:14,567][67838] Updated weights for policy 0, policy_version 41702 (0.0007) [2023-10-07 21:25:14,944][67838] Updated weights for policy 0, policy_version 41712 (0.0007) [2023-10-07 21:25:15,315][67838] Updated weights for policy 0, policy_version 41722 (0.0008) [2023-10-07 21:25:17,145][67871] Updated weights for policy 1, policy_version 41770 (0.0008) [2023-10-07 21:25:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 85491712. Throughput: 0: 1664.8, 1: 1663.2. Samples: 21383350. Policy #0 lag: (min: 13.0, avg: 35.9, max: 40.0) [2023-10-07 21:25:17,477][66916] Avg episode reward: [(0, '40.490'), (1, '51.920')] [2023-10-07 21:25:17,506][67871] Updated weights for policy 1, policy_version 41780 (0.0009) [2023-10-07 21:25:17,877][67871] Updated weights for policy 1, policy_version 41790 (0.0008) [2023-10-07 21:25:19,489][67838] Updated weights for policy 0, policy_version 41732 (0.0009) [2023-10-07 21:25:19,864][67838] Updated weights for policy 0, policy_version 41742 (0.0009) [2023-10-07 21:25:20,238][67838] Updated weights for policy 0, policy_version 41752 (0.0009) [2023-10-07 21:25:21,960][67871] Updated weights for policy 1, policy_version 41800 (0.0007) [2023-10-07 21:25:22,329][67871] Updated weights for policy 1, policy_version 41810 (0.0010) [2023-10-07 21:25:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 85557248. Throughput: 0: 1665.4, 1: 1656.3. Samples: 21403408. Policy #0 lag: (min: 25.0, avg: 33.2, max: 57.0) [2023-10-07 21:25:22,477][66916] Avg episode reward: [(0, '42.840'), (1, '50.900')] [2023-10-07 21:25:22,693][67871] Updated weights for policy 1, policy_version 41820 (0.0008) [2023-10-07 21:25:24,400][67838] Updated weights for policy 0, policy_version 41762 (0.0009) [2023-10-07 21:25:24,770][67838] Updated weights for policy 0, policy_version 41772 (0.0007) [2023-10-07 21:25:25,141][67838] Updated weights for policy 0, policy_version 41782 (0.0007) [2023-10-07 21:25:25,522][67838] Updated weights for policy 0, policy_version 41792 (0.0010) [2023-10-07 21:25:26,779][67871] Updated weights for policy 1, policy_version 41830 (0.0009) [2023-10-07 21:25:27,153][67871] Updated weights for policy 1, policy_version 41840 (0.0010) [2023-10-07 21:25:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 85622784. Throughput: 0: 1648.6, 1: 1661.0. Samples: 21413126. Policy #0 lag: (min: 25.0, avg: 33.2, max: 57.0) [2023-10-07 21:25:27,477][66916] Avg episode reward: [(0, '40.930'), (1, '50.260')] [2023-10-07 21:25:27,523][67871] Updated weights for policy 1, policy_version 41850 (0.0010) [2023-10-07 21:25:29,587][67838] Updated weights for policy 0, policy_version 41802 (0.0007) [2023-10-07 21:25:29,962][67838] Updated weights for policy 0, policy_version 41812 (0.0007) [2023-10-07 21:25:30,338][67838] Updated weights for policy 0, policy_version 41822 (0.0009) [2023-10-07 21:25:31,812][67871] Updated weights for policy 1, policy_version 41860 (0.0008) [2023-10-07 21:25:32,175][67871] Updated weights for policy 1, policy_version 41870 (0.0007) [2023-10-07 21:25:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 85688320. Throughput: 0: 1666.2, 1: 1657.9. Samples: 21433144. Policy #0 lag: (min: 25.0, avg: 33.2, max: 57.0) [2023-10-07 21:25:32,478][66916] Avg episode reward: [(0, '43.950'), (1, '49.970')] [2023-10-07 21:25:32,538][67871] Updated weights for policy 1, policy_version 41880 (0.0008) [2023-10-07 21:25:34,358][67838] Updated weights for policy 0, policy_version 41832 (0.0008) [2023-10-07 21:25:34,736][67838] Updated weights for policy 0, policy_version 41842 (0.0009) [2023-10-07 21:25:35,109][67838] Updated weights for policy 0, policy_version 41852 (0.0008) [2023-10-07 21:25:36,662][67871] Updated weights for policy 1, policy_version 41890 (0.0008) [2023-10-07 21:25:37,045][67871] Updated weights for policy 1, policy_version 41900 (0.0010) [2023-10-07 21:25:37,402][67871] Updated weights for policy 1, policy_version 41910 (0.0010) [2023-10-07 21:25:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 85753856. Throughput: 0: 1667.1, 1: 1649.0. Samples: 21453202. Policy #0 lag: (min: 25.0, avg: 33.2, max: 57.0) [2023-10-07 21:25:37,478][66916] Avg episode reward: [(0, '44.300'), (1, '47.380')] [2023-10-07 21:25:37,773][67871] Updated weights for policy 1, policy_version 41920 (0.0010) [2023-10-07 21:25:39,402][67838] Updated weights for policy 0, policy_version 41862 (0.0009) [2023-10-07 21:25:39,782][67838] Updated weights for policy 0, policy_version 41872 (0.0010) [2023-10-07 21:25:40,142][67838] Updated weights for policy 0, policy_version 41882 (0.0011) [2023-10-07 21:25:41,860][67871] Updated weights for policy 1, policy_version 41930 (0.0010) [2023-10-07 21:25:42,223][67871] Updated weights for policy 1, policy_version 41940 (0.0008) [2023-10-07 21:25:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 85819392. Throughput: 0: 1652.4, 1: 1655.0. Samples: 21462782. Policy #0 lag: (min: 25.0, avg: 33.2, max: 57.0) [2023-10-07 21:25:42,477][66916] Avg episode reward: [(0, '47.960'), (1, '48.320')] [2023-10-07 21:25:42,592][67871] Updated weights for policy 1, policy_version 41950 (0.0009) [2023-10-07 21:25:44,316][67838] Updated weights for policy 0, policy_version 41892 (0.0010) [2023-10-07 21:25:44,694][67838] Updated weights for policy 0, policy_version 41902 (0.0007) [2023-10-07 21:25:45,069][67838] Updated weights for policy 0, policy_version 41912 (0.0007) [2023-10-07 21:25:46,811][67871] Updated weights for policy 1, policy_version 41960 (0.0009) [2023-10-07 21:25:47,179][67871] Updated weights for policy 1, policy_version 41970 (0.0010) [2023-10-07 21:25:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 85884928. Throughput: 0: 1664.1, 1: 1651.1. Samples: 21482742. Policy #0 lag: (min: 25.0, avg: 33.2, max: 57.0) [2023-10-07 21:25:47,477][66916] Avg episode reward: [(0, '51.260'), (1, '47.100')] [2023-10-07 21:25:47,478][67511] Saving new best policy, reward=51.260! [2023-10-07 21:25:47,544][67871] Updated weights for policy 1, policy_version 41980 (0.0007) [2023-10-07 21:25:49,210][67838] Updated weights for policy 0, policy_version 41922 (0.0010) [2023-10-07 21:25:49,593][67838] Updated weights for policy 0, policy_version 41932 (0.0008) [2023-10-07 21:25:49,962][67838] Updated weights for policy 0, policy_version 41942 (0.0010) [2023-10-07 21:25:50,333][67838] Updated weights for policy 0, policy_version 41952 (0.0009) [2023-10-07 21:25:51,590][67871] Updated weights for policy 1, policy_version 41990 (0.0008) [2023-10-07 21:25:51,960][67871] Updated weights for policy 1, policy_version 42000 (0.0009) [2023-10-07 21:25:52,321][67871] Updated weights for policy 1, policy_version 42010 (0.0007) [2023-10-07 21:25:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 85950464. Throughput: 0: 1658.4, 1: 1642.4. Samples: 21502508. Policy #0 lag: (min: 9.0, avg: 9.3, max: 22.0) [2023-10-07 21:25:52,477][66916] Avg episode reward: [(0, '50.550'), (1, '48.590')] [2023-10-07 21:25:54,542][67838] Updated weights for policy 0, policy_version 41962 (0.0010) [2023-10-07 21:25:54,924][67838] Updated weights for policy 0, policy_version 41972 (0.0010) [2023-10-07 21:25:55,287][67838] Updated weights for policy 0, policy_version 41982 (0.0008) [2023-10-07 21:25:56,558][67871] Updated weights for policy 1, policy_version 42020 (0.0009) [2023-10-07 21:25:56,926][67871] Updated weights for policy 1, policy_version 42030 (0.0008) [2023-10-07 21:25:57,285][67871] Updated weights for policy 1, policy_version 42040 (0.0007) [2023-10-07 21:25:57,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 86016000. Throughput: 0: 1650.8, 1: 1653.0. Samples: 21512320. Policy #0 lag: (min: 9.0, avg: 9.3, max: 22.0) [2023-10-07 21:25:57,478][66916] Avg episode reward: [(0, '48.160'), (1, '51.840')] [2023-10-07 21:25:59,384][67838] Updated weights for policy 0, policy_version 41992 (0.0009) [2023-10-07 21:25:59,762][67838] Updated weights for policy 0, policy_version 42002 (0.0010) [2023-10-07 21:26:00,138][67838] Updated weights for policy 0, policy_version 42012 (0.0009) [2023-10-07 21:26:01,549][67871] Updated weights for policy 1, policy_version 42050 (0.0008) [2023-10-07 21:26:01,905][67871] Updated weights for policy 1, policy_version 42060 (0.0009) [2023-10-07 21:26:02,281][67871] Updated weights for policy 1, policy_version 42070 (0.0009) [2023-10-07 21:26:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 86081536. Throughput: 0: 1653.7, 1: 1655.0. Samples: 21532242. Policy #0 lag: (min: 9.0, avg: 9.3, max: 22.0) [2023-10-07 21:26:02,478][66916] Avg episode reward: [(0, '44.460'), (1, '50.440')] [2023-10-07 21:26:02,648][67871] Updated weights for policy 1, policy_version 42080 (0.0009) [2023-10-07 21:26:04,046][67838] Updated weights for policy 0, policy_version 42022 (0.0010) [2023-10-07 21:26:04,416][67838] Updated weights for policy 0, policy_version 42032 (0.0007) [2023-10-07 21:26:04,795][67838] Updated weights for policy 0, policy_version 42042 (0.0009) [2023-10-07 21:26:06,723][67871] Updated weights for policy 1, policy_version 42090 (0.0009) [2023-10-07 21:26:07,098][67871] Updated weights for policy 1, policy_version 42100 (0.0008) [2023-10-07 21:26:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 86147072. Throughput: 0: 1657.7, 1: 1651.9. Samples: 21552342. Policy #0 lag: (min: 9.0, avg: 9.3, max: 22.0) [2023-10-07 21:26:07,477][66916] Avg episode reward: [(0, '42.720'), (1, '52.590')] [2023-10-07 21:26:07,478][67871] Updated weights for policy 1, policy_version 42110 (0.0009) [2023-10-07 21:26:09,044][67838] Updated weights for policy 0, policy_version 42052 (0.0010) [2023-10-07 21:26:09,417][67838] Updated weights for policy 0, policy_version 42062 (0.0008) [2023-10-07 21:26:09,799][67838] Updated weights for policy 0, policy_version 42072 (0.0009) [2023-10-07 21:26:11,589][67871] Updated weights for policy 1, policy_version 42120 (0.0011) [2023-10-07 21:26:11,954][67871] Updated weights for policy 1, policy_version 42130 (0.0009) [2023-10-07 21:26:12,322][67871] Updated weights for policy 1, policy_version 42140 (0.0007) [2023-10-07 21:26:12,477][66916] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 86245376. Throughput: 0: 1647.5, 1: 1658.3. Samples: 21561886. Policy #0 lag: (min: 9.0, avg: 9.3, max: 22.0) [2023-10-07 21:26:12,477][66916] Avg episode reward: [(0, '41.140'), (1, '52.020')] [2023-10-07 21:26:13,893][67838] Updated weights for policy 0, policy_version 42082 (0.0011) [2023-10-07 21:26:14,254][67838] Updated weights for policy 0, policy_version 42092 (0.0012) [2023-10-07 21:26:14,634][67838] Updated weights for policy 0, policy_version 42102 (0.0009) [2023-10-07 21:26:15,001][67838] Updated weights for policy 0, policy_version 42112 (0.0007) [2023-10-07 21:26:16,583][67871] Updated weights for policy 1, policy_version 42150 (0.0009) [2023-10-07 21:26:16,950][67871] Updated weights for policy 1, policy_version 42160 (0.0007) [2023-10-07 21:26:17,324][67871] Updated weights for policy 1, policy_version 42170 (0.0008) [2023-10-07 21:26:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 86278144. Throughput: 0: 1653.4, 1: 1655.2. Samples: 21582032. Policy #0 lag: (min: 9.0, avg: 9.3, max: 22.0) [2023-10-07 21:26:17,477][66916] Avg episode reward: [(0, '43.190'), (1, '55.200')] [2023-10-07 21:26:17,534][67676] Saving new best policy, reward=55.200! [2023-10-07 21:26:19,200][67838] Updated weights for policy 0, policy_version 42122 (0.0007) [2023-10-07 21:26:19,570][67838] Updated weights for policy 0, policy_version 42132 (0.0008) [2023-10-07 21:26:19,945][67838] Updated weights for policy 0, policy_version 42142 (0.0008) [2023-10-07 21:26:21,383][67871] Updated weights for policy 1, policy_version 42180 (0.0008) [2023-10-07 21:26:21,742][67871] Updated weights for policy 1, policy_version 42190 (0.0008) [2023-10-07 21:26:22,112][67871] Updated weights for policy 1, policy_version 42200 (0.0009) [2023-10-07 21:26:22,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 86376448. Throughput: 0: 1658.7, 1: 1648.1. Samples: 21602008. Policy #0 lag: (min: 16.0, avg: 31.8, max: 48.0) [2023-10-07 21:26:22,477][66916] Avg episode reward: [(0, '43.260'), (1, '51.870')] [2023-10-07 21:26:24,062][67838] Updated weights for policy 0, policy_version 42152 (0.0009) [2023-10-07 21:26:24,442][67838] Updated weights for policy 0, policy_version 42162 (0.0009) [2023-10-07 21:26:24,827][67838] Updated weights for policy 0, policy_version 42172 (0.0009) [2023-10-07 21:26:25,840][67871] Updated weights for policy 1, policy_version 42210 (0.0008) [2023-10-07 21:26:26,216][67871] Updated weights for policy 1, policy_version 42220 (0.0008) [2023-10-07 21:26:26,570][67871] Updated weights for policy 1, policy_version 42230 (0.0009) [2023-10-07 21:26:26,933][67871] Updated weights for policy 1, policy_version 42240 (0.0008) [2023-10-07 21:26:27,476][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 86441984. Throughput: 0: 1648.1, 1: 1661.3. Samples: 21611706. Policy #0 lag: (min: 16.0, avg: 31.8, max: 48.0) [2023-10-07 21:26:27,477][66916] Avg episode reward: [(0, '44.530'), (1, '51.260')] [2023-10-07 21:26:28,980][67838] Updated weights for policy 0, policy_version 42182 (0.0010) [2023-10-07 21:26:29,345][67838] Updated weights for policy 0, policy_version 42192 (0.0008) [2023-10-07 21:26:29,725][67838] Updated weights for policy 0, policy_version 42202 (0.0010) [2023-10-07 21:26:31,384][67871] Updated weights for policy 1, policy_version 42250 (0.0011) [2023-10-07 21:26:31,750][67871] Updated weights for policy 1, policy_version 42260 (0.0009) [2023-10-07 21:26:32,116][67871] Updated weights for policy 1, policy_version 42270 (0.0007) [2023-10-07 21:26:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 86507520. Throughput: 0: 1657.8, 1: 1665.2. Samples: 21632278. Policy #0 lag: (min: 16.0, avg: 31.8, max: 48.0) [2023-10-07 21:26:32,477][66916] Avg episode reward: [(0, '44.550'), (1, '48.530')] [2023-10-07 21:26:33,688][67838] Updated weights for policy 0, policy_version 42212 (0.0007) [2023-10-07 21:26:34,073][67838] Updated weights for policy 0, policy_version 42222 (0.0010) [2023-10-07 21:26:34,447][67838] Updated weights for policy 0, policy_version 42232 (0.0007) [2023-10-07 21:26:36,297][67871] Updated weights for policy 1, policy_version 42280 (0.0008) [2023-10-07 21:26:36,681][67871] Updated weights for policy 1, policy_version 42290 (0.0010) [2023-10-07 21:26:37,044][67871] Updated weights for policy 1, policy_version 42300 (0.0011) [2023-10-07 21:26:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 86573056. Throughput: 0: 1667.1, 1: 1651.1. Samples: 21651824. Policy #0 lag: (min: 16.0, avg: 31.8, max: 48.0) [2023-10-07 21:26:37,478][66916] Avg episode reward: [(0, '40.900'), (1, '47.680')] [2023-10-07 21:26:38,414][67838] Updated weights for policy 0, policy_version 42242 (0.0009) [2023-10-07 21:26:38,805][67838] Updated weights for policy 0, policy_version 42252 (0.0010) [2023-10-07 21:26:39,175][67838] Updated weights for policy 0, policy_version 42262 (0.0010) [2023-10-07 21:26:39,548][67838] Updated weights for policy 0, policy_version 42272 (0.0009) [2023-10-07 21:26:41,182][67871] Updated weights for policy 1, policy_version 42310 (0.0010) [2023-10-07 21:26:41,557][67871] Updated weights for policy 1, policy_version 42320 (0.0008) [2023-10-07 21:26:41,927][67871] Updated weights for policy 1, policy_version 42330 (0.0008) [2023-10-07 21:26:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 86638592. Throughput: 0: 1654.8, 1: 1660.3. Samples: 21661502. Policy #0 lag: (min: 16.0, avg: 31.8, max: 48.0) [2023-10-07 21:26:42,477][66916] Avg episode reward: [(0, '42.360'), (1, '45.860')] [2023-10-07 21:26:43,794][67838] Updated weights for policy 0, policy_version 42282 (0.0009) [2023-10-07 21:26:44,165][67838] Updated weights for policy 0, policy_version 42292 (0.0010) [2023-10-07 21:26:44,548][67838] Updated weights for policy 0, policy_version 42302 (0.0009) [2023-10-07 21:26:45,982][67871] Updated weights for policy 1, policy_version 42340 (0.0008) [2023-10-07 21:26:46,348][67871] Updated weights for policy 1, policy_version 42350 (0.0008) [2023-10-07 21:26:46,725][67871] Updated weights for policy 1, policy_version 42360 (0.0010) [2023-10-07 21:26:47,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 86704128. Throughput: 0: 1663.9, 1: 1661.7. Samples: 21681890. Policy #0 lag: (min: 16.0, avg: 31.8, max: 48.0) [2023-10-07 21:26:47,477][66916] Avg episode reward: [(0, '43.610'), (1, '46.180')] [2023-10-07 21:26:48,694][67838] Updated weights for policy 0, policy_version 42312 (0.0009) [2023-10-07 21:26:49,067][67838] Updated weights for policy 0, policy_version 42322 (0.0009) [2023-10-07 21:26:49,439][67838] Updated weights for policy 0, policy_version 42332 (0.0009) [2023-10-07 21:26:50,902][67871] Updated weights for policy 1, policy_version 42370 (0.0009) [2023-10-07 21:26:51,268][67871] Updated weights for policy 1, policy_version 42380 (0.0007) [2023-10-07 21:26:51,631][67871] Updated weights for policy 1, policy_version 42390 (0.0008) [2023-10-07 21:26:51,997][67871] Updated weights for policy 1, policy_version 42400 (0.0007) [2023-10-07 21:26:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 86769664. Throughput: 0: 1659.5, 1: 1649.6. Samples: 21701254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:26:52,477][66916] Avg episode reward: [(0, '49.530'), (1, '49.460')] [2023-10-07 21:26:53,531][67838] Updated weights for policy 0, policy_version 42342 (0.0008) [2023-10-07 21:26:53,897][67838] Updated weights for policy 0, policy_version 42352 (0.0008) [2023-10-07 21:26:54,277][67838] Updated weights for policy 0, policy_version 42362 (0.0008) [2023-10-07 21:26:56,102][67871] Updated weights for policy 1, policy_version 42410 (0.0011) [2023-10-07 21:26:56,473][67871] Updated weights for policy 1, policy_version 42420 (0.0008) [2023-10-07 21:26:56,846][67871] Updated weights for policy 1, policy_version 42430 (0.0007) [2023-10-07 21:26:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 86835200. Throughput: 0: 1656.5, 1: 1662.9. Samples: 21711254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:26:57,477][66916] Avg episode reward: [(0, '50.260'), (1, '47.970')] [2023-10-07 21:26:58,516][67838] Updated weights for policy 0, policy_version 42372 (0.0008) [2023-10-07 21:26:58,887][67838] Updated weights for policy 0, policy_version 42382 (0.0008) [2023-10-07 21:26:59,255][67838] Updated weights for policy 0, policy_version 42392 (0.0009) [2023-10-07 21:27:00,693][67871] Updated weights for policy 1, policy_version 42440 (0.0009) [2023-10-07 21:27:01,058][67871] Updated weights for policy 1, policy_version 42450 (0.0007) [2023-10-07 21:27:01,416][67871] Updated weights for policy 1, policy_version 42460 (0.0010) [2023-10-07 21:27:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 86900736. Throughput: 0: 1658.7, 1: 1661.5. Samples: 21731440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:27:02,477][66916] Avg episode reward: [(0, '49.470'), (1, '48.540')] [2023-10-07 21:27:03,497][67838] Updated weights for policy 0, policy_version 42402 (0.0008) [2023-10-07 21:27:03,867][67838] Updated weights for policy 0, policy_version 42412 (0.0007) [2023-10-07 21:27:04,243][67838] Updated weights for policy 0, policy_version 42422 (0.0009) [2023-10-07 21:27:04,614][67838] Updated weights for policy 0, policy_version 42432 (0.0007) [2023-10-07 21:27:05,414][67871] Updated weights for policy 1, policy_version 42470 (0.0008) [2023-10-07 21:27:05,778][67871] Updated weights for policy 1, policy_version 42480 (0.0007) [2023-10-07 21:27:06,141][67871] Updated weights for policy 1, policy_version 42490 (0.0007) [2023-10-07 21:27:07,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 86966272. Throughput: 0: 1661.3, 1: 1662.1. Samples: 21751560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:27:07,477][66916] Avg episode reward: [(0, '51.120'), (1, '46.170')] [2023-10-07 21:27:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000042496_43515904.pth... [2023-10-07 21:27:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000042432_43450368.pth... [2023-10-07 21:27:07,525][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000040896_41877504.pth [2023-10-07 21:27:07,529][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000040928_41910272.pth [2023-10-07 21:27:08,469][67838] Updated weights for policy 0, policy_version 42442 (0.0010) [2023-10-07 21:27:08,839][67838] Updated weights for policy 0, policy_version 42452 (0.0007) [2023-10-07 21:27:09,217][67838] Updated weights for policy 0, policy_version 42462 (0.0009) [2023-10-07 21:27:10,395][67871] Updated weights for policy 1, policy_version 42500 (0.0010) [2023-10-07 21:27:10,763][67871] Updated weights for policy 1, policy_version 42510 (0.0008) [2023-10-07 21:27:11,135][67871] Updated weights for policy 1, policy_version 42520 (0.0010) [2023-10-07 21:27:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 87031808. Throughput: 0: 1667.3, 1: 1672.9. Samples: 21762016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:27:12,478][66916] Avg episode reward: [(0, '48.290'), (1, '46.450')] [2023-10-07 21:27:13,395][67838] Updated weights for policy 0, policy_version 42472 (0.0008) [2023-10-07 21:27:13,762][67838] Updated weights for policy 0, policy_version 42482 (0.0010) [2023-10-07 21:27:14,135][67838] Updated weights for policy 0, policy_version 42492 (0.0009) [2023-10-07 21:27:15,014][67871] Updated weights for policy 1, policy_version 42530 (0.0007) [2023-10-07 21:27:15,380][67871] Updated weights for policy 1, policy_version 42540 (0.0009) [2023-10-07 21:27:15,744][67871] Updated weights for policy 1, policy_version 42550 (0.0010) [2023-10-07 21:27:16,101][67871] Updated weights for policy 1, policy_version 42560 (0.0009) [2023-10-07 21:27:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 87097344. Throughput: 0: 1668.2, 1: 1652.5. Samples: 21781710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:27:17,477][66916] Avg episode reward: [(0, '43.100'), (1, '47.100')] [2023-10-07 21:27:18,201][67838] Updated weights for policy 0, policy_version 42502 (0.0007) [2023-10-07 21:27:18,573][67838] Updated weights for policy 0, policy_version 42512 (0.0009) [2023-10-07 21:27:18,938][67838] Updated weights for policy 0, policy_version 42522 (0.0008) [2023-10-07 21:27:20,113][67871] Updated weights for policy 1, policy_version 42570 (0.0007) [2023-10-07 21:27:20,485][67871] Updated weights for policy 1, policy_version 42580 (0.0007) [2023-10-07 21:27:20,846][67871] Updated weights for policy 1, policy_version 42590 (0.0008) [2023-10-07 21:27:22,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 87162880. Throughput: 0: 1666.6, 1: 1676.2. Samples: 21802252. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-07 21:27:22,478][66916] Avg episode reward: [(0, '47.000'), (1, '48.630')] [2023-10-07 21:27:23,138][67838] Updated weights for policy 0, policy_version 42532 (0.0010) [2023-10-07 21:27:23,526][67838] Updated weights for policy 0, policy_version 42542 (0.0010) [2023-10-07 21:27:23,886][67838] Updated weights for policy 0, policy_version 42552 (0.0008) [2023-10-07 21:27:25,085][67871] Updated weights for policy 1, policy_version 42600 (0.0008) [2023-10-07 21:27:25,460][67871] Updated weights for policy 1, policy_version 42610 (0.0010) [2023-10-07 21:27:25,833][67871] Updated weights for policy 1, policy_version 42620 (0.0007) [2023-10-07 21:27:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 87228416. Throughput: 0: 1667.6, 1: 1679.7. Samples: 21812132. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-07 21:27:27,477][66916] Avg episode reward: [(0, '44.820'), (1, '51.550')] [2023-10-07 21:27:27,931][67838] Updated weights for policy 0, policy_version 42562 (0.0008) [2023-10-07 21:27:28,299][67838] Updated weights for policy 0, policy_version 42572 (0.0008) [2023-10-07 21:27:28,672][67838] Updated weights for policy 0, policy_version 42582 (0.0008) [2023-10-07 21:27:29,038][67838] Updated weights for policy 0, policy_version 42592 (0.0008) [2023-10-07 21:27:29,891][67871] Updated weights for policy 1, policy_version 42630 (0.0009) [2023-10-07 21:27:30,264][67871] Updated weights for policy 1, policy_version 42640 (0.0010) [2023-10-07 21:27:30,624][67871] Updated weights for policy 1, policy_version 42650 (0.0010) [2023-10-07 21:27:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 87293952. Throughput: 0: 1675.3, 1: 1653.9. Samples: 21831706. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-07 21:27:32,478][66916] Avg episode reward: [(0, '42.940'), (1, '50.240')] [2023-10-07 21:27:32,915][67838] Updated weights for policy 0, policy_version 42602 (0.0008) [2023-10-07 21:27:33,293][67838] Updated weights for policy 0, policy_version 42612 (0.0008) [2023-10-07 21:27:33,659][67838] Updated weights for policy 0, policy_version 42622 (0.0008) [2023-10-07 21:27:34,799][67871] Updated weights for policy 1, policy_version 42660 (0.0008) [2023-10-07 21:27:35,161][67871] Updated weights for policy 1, policy_version 42670 (0.0008) [2023-10-07 21:27:35,531][67871] Updated weights for policy 1, policy_version 42680 (0.0008) [2023-10-07 21:27:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 87359488. Throughput: 0: 1678.7, 1: 1671.2. Samples: 21852000. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-07 21:27:37,477][66916] Avg episode reward: [(0, '46.150'), (1, '49.580')] [2023-10-07 21:27:37,673][67838] Updated weights for policy 0, policy_version 42632 (0.0007) [2023-10-07 21:27:38,038][67838] Updated weights for policy 0, policy_version 42642 (0.0008) [2023-10-07 21:27:38,409][67838] Updated weights for policy 0, policy_version 42652 (0.0008) [2023-10-07 21:27:39,743][67871] Updated weights for policy 1, policy_version 42690 (0.0008) [2023-10-07 21:27:40,110][67871] Updated weights for policy 1, policy_version 42700 (0.0008) [2023-10-07 21:27:40,482][67871] Updated weights for policy 1, policy_version 42710 (0.0008) [2023-10-07 21:27:40,838][67871] Updated weights for policy 1, policy_version 42720 (0.0010) [2023-10-07 21:27:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 87425024. Throughput: 0: 1684.3, 1: 1672.8. Samples: 21862324. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-07 21:27:42,478][66916] Avg episode reward: [(0, '43.570'), (1, '46.810')] [2023-10-07 21:27:42,619][67838] Updated weights for policy 0, policy_version 42662 (0.0008) [2023-10-07 21:27:42,979][67838] Updated weights for policy 0, policy_version 42672 (0.0007) [2023-10-07 21:27:43,354][67838] Updated weights for policy 0, policy_version 42682 (0.0007) [2023-10-07 21:27:45,006][67871] Updated weights for policy 1, policy_version 42730 (0.0008) [2023-10-07 21:27:45,364][67871] Updated weights for policy 1, policy_version 42740 (0.0010) [2023-10-07 21:27:45,735][67871] Updated weights for policy 1, policy_version 42750 (0.0009) [2023-10-07 21:27:47,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 87490560. Throughput: 0: 1684.4, 1: 1656.2. Samples: 21881768. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-07 21:27:47,478][66916] Avg episode reward: [(0, '49.110'), (1, '46.160')] [2023-10-07 21:27:47,491][67838] Updated weights for policy 0, policy_version 42692 (0.0008) [2023-10-07 21:27:47,865][67838] Updated weights for policy 0, policy_version 42702 (0.0009) [2023-10-07 21:27:48,244][67838] Updated weights for policy 0, policy_version 42712 (0.0009) [2023-10-07 21:27:49,886][67871] Updated weights for policy 1, policy_version 42760 (0.0008) [2023-10-07 21:27:50,256][67871] Updated weights for policy 1, policy_version 42770 (0.0007) [2023-10-07 21:27:50,619][67871] Updated weights for policy 1, policy_version 42780 (0.0010) [2023-10-07 21:27:52,287][67838] Updated weights for policy 0, policy_version 42722 (0.0009) [2023-10-07 21:27:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 87556096. Throughput: 0: 1682.1, 1: 1668.8. Samples: 21902350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:27:52,477][66916] Avg episode reward: [(0, '46.570'), (1, '47.360')] [2023-10-07 21:27:52,663][67838] Updated weights for policy 0, policy_version 42732 (0.0008) [2023-10-07 21:27:53,026][67838] Updated weights for policy 0, policy_version 42742 (0.0008) [2023-10-07 21:27:53,397][67838] Updated weights for policy 0, policy_version 42752 (0.0009) [2023-10-07 21:27:54,815][67871] Updated weights for policy 1, policy_version 42790 (0.0008) [2023-10-07 21:27:55,172][67871] Updated weights for policy 1, policy_version 42800 (0.0009) [2023-10-07 21:27:55,533][67871] Updated weights for policy 1, policy_version 42810 (0.0011) [2023-10-07 21:27:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 87621632. Throughput: 0: 1677.9, 1: 1663.4. Samples: 21912374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:27:57,477][66916] Avg episode reward: [(0, '43.790'), (1, '47.310')] [2023-10-07 21:27:57,629][67838] Updated weights for policy 0, policy_version 42762 (0.0007) [2023-10-07 21:27:57,994][67838] Updated weights for policy 0, policy_version 42772 (0.0007) [2023-10-07 21:27:58,376][67838] Updated weights for policy 0, policy_version 42782 (0.0008) [2023-10-07 21:27:59,730][67871] Updated weights for policy 1, policy_version 42820 (0.0011) [2023-10-07 21:28:00,098][67871] Updated weights for policy 1, policy_version 42830 (0.0007) [2023-10-07 21:28:00,466][67871] Updated weights for policy 1, policy_version 42840 (0.0009) [2023-10-07 21:28:02,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 87687168. Throughput: 0: 1674.5, 1: 1656.8. Samples: 21931618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:28:02,478][66916] Avg episode reward: [(0, '43.400'), (1, '50.520')] [2023-10-07 21:28:02,573][67838] Updated weights for policy 0, policy_version 42792 (0.0007) [2023-10-07 21:28:02,962][67838] Updated weights for policy 0, policy_version 42802 (0.0007) [2023-10-07 21:28:03,337][67838] Updated weights for policy 0, policy_version 42812 (0.0007) [2023-10-07 21:28:04,357][67871] Updated weights for policy 1, policy_version 42850 (0.0009) [2023-10-07 21:28:04,727][67871] Updated weights for policy 1, policy_version 42860 (0.0007) [2023-10-07 21:28:05,087][67871] Updated weights for policy 1, policy_version 42870 (0.0010) [2023-10-07 21:28:05,450][67871] Updated weights for policy 1, policy_version 42880 (0.0008) [2023-10-07 21:28:07,455][67838] Updated weights for policy 0, policy_version 42822 (0.0007) [2023-10-07 21:28:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 87752704. Throughput: 0: 1671.0, 1: 1661.2. Samples: 21952202. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:28:07,477][66916] Avg episode reward: [(0, '41.710'), (1, '50.730')] [2023-10-07 21:28:07,826][67838] Updated weights for policy 0, policy_version 42832 (0.0008) [2023-10-07 21:28:08,197][67838] Updated weights for policy 0, policy_version 42842 (0.0008) [2023-10-07 21:28:09,721][67871] Updated weights for policy 1, policy_version 42890 (0.0008) [2023-10-07 21:28:10,080][67871] Updated weights for policy 1, policy_version 42900 (0.0007) [2023-10-07 21:28:10,443][67871] Updated weights for policy 1, policy_version 42910 (0.0007) [2023-10-07 21:28:12,301][67838] Updated weights for policy 0, policy_version 42852 (0.0010) [2023-10-07 21:28:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 87818240. Throughput: 0: 1674.8, 1: 1654.1. Samples: 21961934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:28:12,477][66916] Avg episode reward: [(0, '41.800'), (1, '53.930')] [2023-10-07 21:28:12,688][67838] Updated weights for policy 0, policy_version 42862 (0.0007) [2023-10-07 21:28:13,057][67838] Updated weights for policy 0, policy_version 42872 (0.0007) [2023-10-07 21:28:14,464][67871] Updated weights for policy 1, policy_version 42920 (0.0008) [2023-10-07 21:28:14,829][67871] Updated weights for policy 1, policy_version 42930 (0.0009) [2023-10-07 21:28:15,204][67871] Updated weights for policy 1, policy_version 42940 (0.0009) [2023-10-07 21:28:16,972][67838] Updated weights for policy 0, policy_version 42882 (0.0008) [2023-10-07 21:28:17,346][67838] Updated weights for policy 0, policy_version 42892 (0.0007) [2023-10-07 21:28:17,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 87883776. Throughput: 0: 1673.3, 1: 1665.4. Samples: 21981948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:28:17,477][66916] Avg episode reward: [(0, '41.600'), (1, '51.290')] [2023-10-07 21:28:17,720][67838] Updated weights for policy 0, policy_version 42902 (0.0007) [2023-10-07 21:28:18,093][67838] Updated weights for policy 0, policy_version 42912 (0.0008) [2023-10-07 21:28:19,428][67871] Updated weights for policy 1, policy_version 42950 (0.0007) [2023-10-07 21:28:19,811][67871] Updated weights for policy 1, policy_version 42960 (0.0009) [2023-10-07 21:28:20,181][67871] Updated weights for policy 1, policy_version 42970 (0.0010) [2023-10-07 21:28:21,999][67838] Updated weights for policy 0, policy_version 42922 (0.0007) [2023-10-07 21:28:22,368][67838] Updated weights for policy 0, policy_version 42932 (0.0008) [2023-10-07 21:28:22,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 87949312. Throughput: 0: 1666.7, 1: 1667.7. Samples: 22002050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:28:22,478][66916] Avg episode reward: [(0, '44.780'), (1, '54.860')] [2023-10-07 21:28:22,736][67838] Updated weights for policy 0, policy_version 42942 (0.0010) [2023-10-07 21:28:24,264][67871] Updated weights for policy 1, policy_version 42980 (0.0009) [2023-10-07 21:28:24,630][67871] Updated weights for policy 1, policy_version 42990 (0.0008) [2023-10-07 21:28:24,996][67871] Updated weights for policy 1, policy_version 43000 (0.0008) [2023-10-07 21:28:26,733][67838] Updated weights for policy 0, policy_version 42952 (0.0009) [2023-10-07 21:28:27,106][67838] Updated weights for policy 0, policy_version 42962 (0.0007) [2023-10-07 21:28:27,471][67838] Updated weights for policy 0, policy_version 42972 (0.0009) [2023-10-07 21:28:27,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 88014848. Throughput: 0: 1677.7, 1: 1653.1. Samples: 22012210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:28:27,478][66916] Avg episode reward: [(0, '42.800'), (1, '53.010')] [2023-10-07 21:28:29,094][67871] Updated weights for policy 1, policy_version 43010 (0.0008) [2023-10-07 21:28:29,469][67871] Updated weights for policy 1, policy_version 43020 (0.0007) [2023-10-07 21:28:29,843][67871] Updated weights for policy 1, policy_version 43030 (0.0008) [2023-10-07 21:28:30,202][67871] Updated weights for policy 1, policy_version 43040 (0.0007) [2023-10-07 21:28:31,738][67838] Updated weights for policy 0, policy_version 42982 (0.0009) [2023-10-07 21:28:32,116][67838] Updated weights for policy 0, policy_version 42992 (0.0010) [2023-10-07 21:28:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 88080384. Throughput: 0: 1674.4, 1: 1664.2. Samples: 22032008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:28:32,478][66916] Avg episode reward: [(0, '38.850'), (1, '54.160')] [2023-10-07 21:28:32,488][67838] Updated weights for policy 0, policy_version 43002 (0.0008) [2023-10-07 21:28:34,179][67871] Updated weights for policy 1, policy_version 43050 (0.0007) [2023-10-07 21:28:34,545][67871] Updated weights for policy 1, policy_version 43060 (0.0008) [2023-10-07 21:28:34,917][67871] Updated weights for policy 1, policy_version 43070 (0.0009) [2023-10-07 21:28:36,538][67838] Updated weights for policy 0, policy_version 43012 (0.0008) [2023-10-07 21:28:36,911][67838] Updated weights for policy 0, policy_version 43022 (0.0009) [2023-10-07 21:28:37,286][67838] Updated weights for policy 0, policy_version 43032 (0.0011) [2023-10-07 21:28:37,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 88145920. Throughput: 0: 1655.7, 1: 1665.7. Samples: 22051812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:28:37,477][66916] Avg episode reward: [(0, '39.150'), (1, '51.930')] [2023-10-07 21:28:39,216][67871] Updated weights for policy 1, policy_version 43080 (0.0011) [2023-10-07 21:28:39,575][67871] Updated weights for policy 1, policy_version 43090 (0.0008) [2023-10-07 21:28:39,949][67871] Updated weights for policy 1, policy_version 43100 (0.0008) [2023-10-07 21:28:41,302][67838] Updated weights for policy 0, policy_version 43042 (0.0007) [2023-10-07 21:28:41,673][67838] Updated weights for policy 0, policy_version 43052 (0.0008) [2023-10-07 21:28:42,044][67838] Updated weights for policy 0, policy_version 43062 (0.0007) [2023-10-07 21:28:42,424][67838] Updated weights for policy 0, policy_version 43072 (0.0007) [2023-10-07 21:28:42,477][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 88244224. Throughput: 0: 1671.2, 1: 1650.4. Samples: 22061846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:28:42,478][66916] Avg episode reward: [(0, '41.820'), (1, '51.800')] [2023-10-07 21:28:44,083][67871] Updated weights for policy 1, policy_version 43110 (0.0008) [2023-10-07 21:28:44,440][67871] Updated weights for policy 1, policy_version 43120 (0.0009) [2023-10-07 21:28:44,809][67871] Updated weights for policy 1, policy_version 43130 (0.0010) [2023-10-07 21:28:46,712][67838] Updated weights for policy 0, policy_version 43082 (0.0007) [2023-10-07 21:28:47,092][67838] Updated weights for policy 0, policy_version 43092 (0.0008) [2023-10-07 21:28:47,462][67838] Updated weights for policy 0, policy_version 43102 (0.0007) [2023-10-07 21:28:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 88276992. Throughput: 0: 1674.9, 1: 1662.8. Samples: 22081814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:28:47,477][66916] Avg episode reward: [(0, '42.050'), (1, '49.850')] [2023-10-07 21:28:49,193][67871] Updated weights for policy 1, policy_version 43140 (0.0010) [2023-10-07 21:28:49,562][67871] Updated weights for policy 1, policy_version 43150 (0.0010) [2023-10-07 21:28:49,931][67871] Updated weights for policy 1, policy_version 43160 (0.0007) [2023-10-07 21:28:51,478][67838] Updated weights for policy 0, policy_version 43112 (0.0010) [2023-10-07 21:28:51,864][67838] Updated weights for policy 0, policy_version 43122 (0.0010) [2023-10-07 21:28:52,230][67838] Updated weights for policy 0, policy_version 43132 (0.0007) [2023-10-07 21:28:52,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 88375296. Throughput: 0: 1657.4, 1: 1655.4. Samples: 22101280. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 21:28:52,478][66916] Avg episode reward: [(0, '43.160'), (1, '54.350')] [2023-10-07 21:28:54,011][67871] Updated weights for policy 1, policy_version 43170 (0.0007) [2023-10-07 21:28:54,378][67871] Updated weights for policy 1, policy_version 43180 (0.0007) [2023-10-07 21:28:54,736][67871] Updated weights for policy 1, policy_version 43190 (0.0007) [2023-10-07 21:28:55,109][67871] Updated weights for policy 1, policy_version 43200 (0.0009) [2023-10-07 21:28:56,517][67838] Updated weights for policy 0, policy_version 43142 (0.0009) [2023-10-07 21:28:56,894][67838] Updated weights for policy 0, policy_version 43152 (0.0007) [2023-10-07 21:28:57,251][67838] Updated weights for policy 0, policy_version 43162 (0.0007) [2023-10-07 21:28:57,477][66916] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 88440832. Throughput: 0: 1674.7, 1: 1648.2. Samples: 22111464. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 21:28:57,478][66916] Avg episode reward: [(0, '43.060'), (1, '53.940')] [2023-10-07 21:28:59,186][67871] Updated weights for policy 1, policy_version 43210 (0.0008) [2023-10-07 21:28:59,550][67871] Updated weights for policy 1, policy_version 43220 (0.0007) [2023-10-07 21:28:59,921][67871] Updated weights for policy 1, policy_version 43230 (0.0007) [2023-10-07 21:29:01,395][67838] Updated weights for policy 0, policy_version 43172 (0.0008) [2023-10-07 21:29:01,786][67838] Updated weights for policy 0, policy_version 43182 (0.0007) [2023-10-07 21:29:02,147][67838] Updated weights for policy 0, policy_version 43192 (0.0008) [2023-10-07 21:29:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 88506368. Throughput: 0: 1671.0, 1: 1651.1. Samples: 22131444. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 21:29:02,478][66916] Avg episode reward: [(0, '42.380'), (1, '58.280')] [2023-10-07 21:29:02,479][67676] Saving new best policy, reward=58.280! [2023-10-07 21:29:04,027][67871] Updated weights for policy 1, policy_version 43240 (0.0010) [2023-10-07 21:29:04,398][67871] Updated weights for policy 1, policy_version 43250 (0.0008) [2023-10-07 21:29:04,757][67871] Updated weights for policy 1, policy_version 43260 (0.0009) [2023-10-07 21:29:06,288][67838] Updated weights for policy 0, policy_version 43202 (0.0008) [2023-10-07 21:29:06,665][67838] Updated weights for policy 0, policy_version 43212 (0.0008) [2023-10-07 21:29:07,029][67838] Updated weights for policy 0, policy_version 43222 (0.0009) [2023-10-07 21:29:07,406][67838] Updated weights for policy 0, policy_version 43232 (0.0009) [2023-10-07 21:29:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 88571904. Throughput: 0: 1653.0, 1: 1658.9. Samples: 22151088. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 21:29:07,477][66916] Avg episode reward: [(0, '46.680'), (1, '58.470')] [2023-10-07 21:29:07,486][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000043264_44302336.pth... [2023-10-07 21:29:07,487][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000043232_44269568.pth... [2023-10-07 21:29:07,524][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000041664_42663936.pth [2023-10-07 21:29:07,526][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000041728_42729472.pth [2023-10-07 21:29:07,530][67676] Saving new best policy, reward=58.470! [2023-10-07 21:29:08,748][67871] Updated weights for policy 1, policy_version 43270 (0.0008) [2023-10-07 21:29:09,113][67871] Updated weights for policy 1, policy_version 43280 (0.0009) [2023-10-07 21:29:09,484][67871] Updated weights for policy 1, policy_version 43290 (0.0008) [2023-10-07 21:29:11,418][67838] Updated weights for policy 0, policy_version 43242 (0.0011) [2023-10-07 21:29:11,793][67838] Updated weights for policy 0, policy_version 43252 (0.0007) [2023-10-07 21:29:12,167][67838] Updated weights for policy 0, policy_version 43262 (0.0008) [2023-10-07 21:29:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 88637440. Throughput: 0: 1659.5, 1: 1648.9. Samples: 22161088. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 21:29:12,477][66916] Avg episode reward: [(0, '45.520'), (1, '57.410')] [2023-10-07 21:29:13,590][67871] Updated weights for policy 1, policy_version 43300 (0.0008) [2023-10-07 21:29:13,957][67871] Updated weights for policy 1, policy_version 43310 (0.0010) [2023-10-07 21:29:14,329][67871] Updated weights for policy 1, policy_version 43320 (0.0011) [2023-10-07 21:29:16,350][67838] Updated weights for policy 0, policy_version 43272 (0.0007) [2023-10-07 21:29:16,722][67838] Updated weights for policy 0, policy_version 43282 (0.0009) [2023-10-07 21:29:17,090][67838] Updated weights for policy 0, policy_version 43292 (0.0008) [2023-10-07 21:29:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 88702976. Throughput: 0: 1662.5, 1: 1660.0. Samples: 22181520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:29:17,478][66916] Avg episode reward: [(0, '50.090'), (1, '56.810')] [2023-10-07 21:29:18,485][67871] Updated weights for policy 1, policy_version 43330 (0.0008) [2023-10-07 21:29:18,852][67871] Updated weights for policy 1, policy_version 43340 (0.0008) [2023-10-07 21:29:19,221][67871] Updated weights for policy 1, policy_version 43350 (0.0009) [2023-10-07 21:29:19,589][67871] Updated weights for policy 1, policy_version 43360 (0.0009) [2023-10-07 21:29:21,149][67838] Updated weights for policy 0, policy_version 43302 (0.0008) [2023-10-07 21:29:21,524][67838] Updated weights for policy 0, policy_version 43312 (0.0009) [2023-10-07 21:29:21,900][67838] Updated weights for policy 0, policy_version 43322 (0.0009) [2023-10-07 21:29:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 88768512. Throughput: 0: 1653.4, 1: 1655.9. Samples: 22200732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:29:22,478][66916] Avg episode reward: [(0, '47.770'), (1, '56.660')] [2023-10-07 21:29:23,816][67871] Updated weights for policy 1, policy_version 43370 (0.0010) [2023-10-07 21:29:24,180][67871] Updated weights for policy 1, policy_version 43380 (0.0009) [2023-10-07 21:29:24,556][67871] Updated weights for policy 1, policy_version 43390 (0.0009) [2023-10-07 21:29:25,918][67838] Updated weights for policy 0, policy_version 43332 (0.0008) [2023-10-07 21:29:26,290][67838] Updated weights for policy 0, policy_version 43342 (0.0007) [2023-10-07 21:29:26,664][67838] Updated weights for policy 0, policy_version 43352 (0.0009) [2023-10-07 21:29:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 88834048. Throughput: 0: 1664.9, 1: 1647.3. Samples: 22210894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:29:27,477][66916] Avg episode reward: [(0, '50.540'), (1, '53.280')] [2023-10-07 21:29:28,823][67871] Updated weights for policy 1, policy_version 43400 (0.0007) [2023-10-07 21:29:29,194][67871] Updated weights for policy 1, policy_version 43410 (0.0007) [2023-10-07 21:29:29,562][67871] Updated weights for policy 1, policy_version 43420 (0.0007) [2023-10-07 21:29:30,524][67838] Updated weights for policy 0, policy_version 43362 (0.0008) [2023-10-07 21:29:30,905][67838] Updated weights for policy 0, policy_version 43372 (0.0008) [2023-10-07 21:29:31,270][67838] Updated weights for policy 0, policy_version 43382 (0.0008) [2023-10-07 21:29:31,652][67838] Updated weights for policy 0, policy_version 43392 (0.0011) [2023-10-07 21:29:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 88899584. Throughput: 0: 1651.5, 1: 1655.2. Samples: 22230612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:29:32,478][66916] Avg episode reward: [(0, '45.290'), (1, '50.880')] [2023-10-07 21:29:33,625][67871] Updated weights for policy 1, policy_version 43430 (0.0007) [2023-10-07 21:29:33,995][67871] Updated weights for policy 1, policy_version 43440 (0.0009) [2023-10-07 21:29:34,364][67871] Updated weights for policy 1, policy_version 43450 (0.0007) [2023-10-07 21:29:35,831][67838] Updated weights for policy 0, policy_version 43402 (0.0010) [2023-10-07 21:29:36,210][67838] Updated weights for policy 0, policy_version 43412 (0.0007) [2023-10-07 21:29:36,582][67838] Updated weights for policy 0, policy_version 43422 (0.0007) [2023-10-07 21:29:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 88965120. Throughput: 0: 1654.5, 1: 1658.9. Samples: 22250382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:29:37,477][66916] Avg episode reward: [(0, '47.740'), (1, '51.920')] [2023-10-07 21:29:38,400][67871] Updated weights for policy 1, policy_version 43460 (0.0007) [2023-10-07 21:29:38,769][67871] Updated weights for policy 1, policy_version 43470 (0.0009) [2023-10-07 21:29:39,141][67871] Updated weights for policy 1, policy_version 43480 (0.0009) [2023-10-07 21:29:40,699][67838] Updated weights for policy 0, policy_version 43432 (0.0008) [2023-10-07 21:29:41,070][67838] Updated weights for policy 0, policy_version 43442 (0.0008) [2023-10-07 21:29:41,439][67838] Updated weights for policy 0, policy_version 43452 (0.0007) [2023-10-07 21:29:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89030656. Throughput: 0: 1664.3, 1: 1651.5. Samples: 22260672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:29:42,477][66916] Avg episode reward: [(0, '44.840'), (1, '50.410')] [2023-10-07 21:29:43,348][67871] Updated weights for policy 1, policy_version 43490 (0.0008) [2023-10-07 21:29:43,723][67871] Updated weights for policy 1, policy_version 43500 (0.0007) [2023-10-07 21:29:44,074][67871] Updated weights for policy 1, policy_version 43510 (0.0010) [2023-10-07 21:29:44,450][67871] Updated weights for policy 1, policy_version 43520 (0.0011) [2023-10-07 21:29:45,656][67838] Updated weights for policy 0, policy_version 43462 (0.0008) [2023-10-07 21:29:46,036][67838] Updated weights for policy 0, policy_version 43472 (0.0008) [2023-10-07 21:29:46,399][67838] Updated weights for policy 0, policy_version 43482 (0.0008) [2023-10-07 21:29:47,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 89096192. Throughput: 0: 1655.6, 1: 1656.8. Samples: 22280506. Policy #0 lag: (min: 4.0, avg: 14.7, max: 36.0) [2023-10-07 21:29:47,477][66916] Avg episode reward: [(0, '45.400'), (1, '49.190')] [2023-10-07 21:29:48,814][67871] Updated weights for policy 1, policy_version 43530 (0.0009) [2023-10-07 21:29:49,175][67871] Updated weights for policy 1, policy_version 43540 (0.0007) [2023-10-07 21:29:49,540][67871] Updated weights for policy 1, policy_version 43550 (0.0011) [2023-10-07 21:29:50,556][67838] Updated weights for policy 0, policy_version 43492 (0.0010) [2023-10-07 21:29:50,937][67838] Updated weights for policy 0, policy_version 43502 (0.0010) [2023-10-07 21:29:51,319][67838] Updated weights for policy 0, policy_version 43512 (0.0008) [2023-10-07 21:29:52,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 89161728. Throughput: 0: 1665.5, 1: 1653.5. Samples: 22300440. Policy #0 lag: (min: 4.0, avg: 14.7, max: 36.0) [2023-10-07 21:29:52,477][66916] Avg episode reward: [(0, '48.160'), (1, '49.090')] [2023-10-07 21:29:53,839][67871] Updated weights for policy 1, policy_version 43560 (0.0009) [2023-10-07 21:29:54,223][67871] Updated weights for policy 1, policy_version 43570 (0.0010) [2023-10-07 21:29:54,593][67871] Updated weights for policy 1, policy_version 43580 (0.0008) [2023-10-07 21:29:55,425][67838] Updated weights for policy 0, policy_version 43522 (0.0008) [2023-10-07 21:29:55,800][67838] Updated weights for policy 0, policy_version 43532 (0.0008) [2023-10-07 21:29:56,183][67838] Updated weights for policy 0, policy_version 43542 (0.0008) [2023-10-07 21:29:56,553][67838] Updated weights for policy 0, policy_version 43552 (0.0009) [2023-10-07 21:29:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89227264. Throughput: 0: 1677.7, 1: 1649.0. Samples: 22310792. Policy #0 lag: (min: 4.0, avg: 14.7, max: 36.0) [2023-10-07 21:29:57,477][66916] Avg episode reward: [(0, '48.970'), (1, '49.410')] [2023-10-07 21:29:58,723][67871] Updated weights for policy 1, policy_version 43590 (0.0010) [2023-10-07 21:29:59,088][67871] Updated weights for policy 1, policy_version 43600 (0.0007) [2023-10-07 21:29:59,455][67871] Updated weights for policy 1, policy_version 43610 (0.0007) [2023-10-07 21:30:00,622][67838] Updated weights for policy 0, policy_version 43562 (0.0010) [2023-10-07 21:30:00,991][67838] Updated weights for policy 0, policy_version 43572 (0.0008) [2023-10-07 21:30:01,370][67838] Updated weights for policy 0, policy_version 43582 (0.0007) [2023-10-07 21:30:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89292800. Throughput: 0: 1658.1, 1: 1652.3. Samples: 22330488. Policy #0 lag: (min: 4.0, avg: 14.7, max: 36.0) [2023-10-07 21:30:02,477][66916] Avg episode reward: [(0, '46.020'), (1, '49.040')] [2023-10-07 21:30:03,461][67871] Updated weights for policy 1, policy_version 43620 (0.0007) [2023-10-07 21:30:03,817][67871] Updated weights for policy 1, policy_version 43630 (0.0008) [2023-10-07 21:30:04,180][67871] Updated weights for policy 1, policy_version 43640 (0.0008) [2023-10-07 21:30:05,394][67838] Updated weights for policy 0, policy_version 43592 (0.0008) [2023-10-07 21:30:05,771][67838] Updated weights for policy 0, policy_version 43602 (0.0009) [2023-10-07 21:30:06,144][67838] Updated weights for policy 0, policy_version 43612 (0.0008) [2023-10-07 21:30:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89358336. Throughput: 0: 1673.8, 1: 1661.8. Samples: 22350836. Policy #0 lag: (min: 4.0, avg: 14.7, max: 36.0) [2023-10-07 21:30:07,478][66916] Avg episode reward: [(0, '46.400'), (1, '48.090')] [2023-10-07 21:30:08,159][67871] Updated weights for policy 1, policy_version 43650 (0.0007) [2023-10-07 21:30:08,523][67871] Updated weights for policy 1, policy_version 43660 (0.0008) [2023-10-07 21:30:08,897][67871] Updated weights for policy 1, policy_version 43670 (0.0009) [2023-10-07 21:30:09,257][67871] Updated weights for policy 1, policy_version 43680 (0.0009) [2023-10-07 21:30:10,262][67838] Updated weights for policy 0, policy_version 43622 (0.0008) [2023-10-07 21:30:10,634][67838] Updated weights for policy 0, policy_version 43632 (0.0010) [2023-10-07 21:30:11,005][67838] Updated weights for policy 0, policy_version 43642 (0.0009) [2023-10-07 21:30:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89423872. Throughput: 0: 1676.4, 1: 1663.3. Samples: 22361182. Policy #0 lag: (min: 4.0, avg: 14.7, max: 36.0) [2023-10-07 21:30:12,477][66916] Avg episode reward: [(0, '42.560'), (1, '48.890')] [2023-10-07 21:30:13,275][67871] Updated weights for policy 1, policy_version 43690 (0.0007) [2023-10-07 21:30:13,640][67871] Updated weights for policy 1, policy_version 43700 (0.0009) [2023-10-07 21:30:14,010][67871] Updated weights for policy 1, policy_version 43710 (0.0010) [2023-10-07 21:30:14,811][67838] Updated weights for policy 0, policy_version 43652 (0.0008) [2023-10-07 21:30:15,188][67838] Updated weights for policy 0, policy_version 43662 (0.0008) [2023-10-07 21:30:15,561][67838] Updated weights for policy 0, policy_version 43672 (0.0009) [2023-10-07 21:30:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 89489408. Throughput: 0: 1666.2, 1: 1664.2. Samples: 22380482. Policy #0 lag: (min: 25.0, avg: 42.8, max: 57.0) [2023-10-07 21:30:17,478][66916] Avg episode reward: [(0, '39.670'), (1, '50.520')] [2023-10-07 21:30:18,227][67871] Updated weights for policy 1, policy_version 43720 (0.0008) [2023-10-07 21:30:18,597][67871] Updated weights for policy 1, policy_version 43730 (0.0008) [2023-10-07 21:30:18,963][67871] Updated weights for policy 1, policy_version 43740 (0.0007) [2023-10-07 21:30:19,716][67838] Updated weights for policy 0, policy_version 43682 (0.0009) [2023-10-07 21:30:20,093][67838] Updated weights for policy 0, policy_version 43692 (0.0008) [2023-10-07 21:30:20,473][67838] Updated weights for policy 0, policy_version 43702 (0.0010) [2023-10-07 21:30:20,851][67838] Updated weights for policy 0, policy_version 43712 (0.0009) [2023-10-07 21:30:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 89554944. Throughput: 0: 1681.5, 1: 1664.3. Samples: 22400940. Policy #0 lag: (min: 25.0, avg: 42.8, max: 57.0) [2023-10-07 21:30:22,478][66916] Avg episode reward: [(0, '43.140'), (1, '51.790')] [2023-10-07 21:30:23,166][67871] Updated weights for policy 1, policy_version 43750 (0.0009) [2023-10-07 21:30:23,537][67871] Updated weights for policy 1, policy_version 43760 (0.0007) [2023-10-07 21:30:23,912][67871] Updated weights for policy 1, policy_version 43770 (0.0009) [2023-10-07 21:30:25,044][67838] Updated weights for policy 0, policy_version 43722 (0.0011) [2023-10-07 21:30:25,420][67838] Updated weights for policy 0, policy_version 43732 (0.0008) [2023-10-07 21:30:25,810][67838] Updated weights for policy 0, policy_version 43742 (0.0010) [2023-10-07 21:30:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89620480. Throughput: 0: 1672.9, 1: 1662.0. Samples: 22410746. Policy #0 lag: (min: 25.0, avg: 42.8, max: 57.0) [2023-10-07 21:30:27,477][66916] Avg episode reward: [(0, '39.020'), (1, '52.710')] [2023-10-07 21:30:27,924][67871] Updated weights for policy 1, policy_version 43780 (0.0008) [2023-10-07 21:30:28,306][67871] Updated weights for policy 1, policy_version 43790 (0.0009) [2023-10-07 21:30:28,662][67871] Updated weights for policy 1, policy_version 43800 (0.0009) [2023-10-07 21:30:29,671][67838] Updated weights for policy 0, policy_version 43752 (0.0009) [2023-10-07 21:30:30,045][67838] Updated weights for policy 0, policy_version 43762 (0.0007) [2023-10-07 21:30:30,412][67838] Updated weights for policy 0, policy_version 43772 (0.0007) [2023-10-07 21:30:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89686016. Throughput: 0: 1661.9, 1: 1663.0. Samples: 22430126. Policy #0 lag: (min: 25.0, avg: 42.8, max: 57.0) [2023-10-07 21:30:32,477][66916] Avg episode reward: [(0, '40.950'), (1, '52.780')] [2023-10-07 21:30:32,910][67871] Updated weights for policy 1, policy_version 43810 (0.0009) [2023-10-07 21:30:33,280][67871] Updated weights for policy 1, policy_version 43820 (0.0007) [2023-10-07 21:30:33,643][67871] Updated weights for policy 1, policy_version 43830 (0.0008) [2023-10-07 21:30:34,016][67871] Updated weights for policy 1, policy_version 43840 (0.0011) [2023-10-07 21:30:34,550][67838] Updated weights for policy 0, policy_version 43782 (0.0008) [2023-10-07 21:30:34,913][67838] Updated weights for policy 0, policy_version 43792 (0.0007) [2023-10-07 21:30:35,288][67838] Updated weights for policy 0, policy_version 43802 (0.0007) [2023-10-07 21:30:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89751552. Throughput: 0: 1675.9, 1: 1666.6. Samples: 22450850. Policy #0 lag: (min: 25.0, avg: 42.8, max: 57.0) [2023-10-07 21:30:37,477][66916] Avg episode reward: [(0, '41.960'), (1, '54.490')] [2023-10-07 21:30:38,021][67871] Updated weights for policy 1, policy_version 43850 (0.0008) [2023-10-07 21:30:38,389][67871] Updated weights for policy 1, policy_version 43860 (0.0008) [2023-10-07 21:30:38,750][67871] Updated weights for policy 1, policy_version 43870 (0.0008) [2023-10-07 21:30:39,594][67838] Updated weights for policy 0, policy_version 43812 (0.0009) [2023-10-07 21:30:39,989][67838] Updated weights for policy 0, policy_version 43822 (0.0007) [2023-10-07 21:30:40,361][67838] Updated weights for policy 0, policy_version 43832 (0.0009) [2023-10-07 21:30:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89817088. Throughput: 0: 1653.6, 1: 1663.9. Samples: 22460080. Policy #0 lag: (min: 25.0, avg: 42.8, max: 57.0) [2023-10-07 21:30:42,477][66916] Avg episode reward: [(0, '43.500'), (1, '52.150')] [2023-10-07 21:30:42,810][67871] Updated weights for policy 1, policy_version 43880 (0.0011) [2023-10-07 21:30:43,174][67871] Updated weights for policy 1, policy_version 43890 (0.0009) [2023-10-07 21:30:43,542][67871] Updated weights for policy 1, policy_version 43900 (0.0010) [2023-10-07 21:30:44,503][67838] Updated weights for policy 0, policy_version 43842 (0.0008) [2023-10-07 21:30:44,884][67838] Updated weights for policy 0, policy_version 43852 (0.0007) [2023-10-07 21:30:45,259][67838] Updated weights for policy 0, policy_version 43862 (0.0007) [2023-10-07 21:30:45,627][67838] Updated weights for policy 0, policy_version 43872 (0.0009) [2023-10-07 21:30:47,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 89882624. Throughput: 0: 1656.5, 1: 1661.1. Samples: 22479782. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 21:30:47,478][66916] Avg episode reward: [(0, '45.800'), (1, '52.260')] [2023-10-07 21:30:47,783][67871] Updated weights for policy 1, policy_version 43910 (0.0010) [2023-10-07 21:30:48,154][67871] Updated weights for policy 1, policy_version 43920 (0.0010) [2023-10-07 21:30:48,516][67871] Updated weights for policy 1, policy_version 43930 (0.0009) [2023-10-07 21:30:49,709][67838] Updated weights for policy 0, policy_version 43882 (0.0009) [2023-10-07 21:30:50,081][67838] Updated weights for policy 0, policy_version 43892 (0.0009) [2023-10-07 21:30:50,456][67838] Updated weights for policy 0, policy_version 43902 (0.0009) [2023-10-07 21:30:52,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89948160. Throughput: 0: 1664.9, 1: 1656.8. Samples: 22500310. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 21:30:52,478][66916] Avg episode reward: [(0, '43.530'), (1, '48.420')] [2023-10-07 21:30:52,616][67871] Updated weights for policy 1, policy_version 43940 (0.0008) [2023-10-07 21:30:52,976][67871] Updated weights for policy 1, policy_version 43950 (0.0008) [2023-10-07 21:30:53,341][67871] Updated weights for policy 1, policy_version 43960 (0.0009) [2023-10-07 21:30:54,494][67838] Updated weights for policy 0, policy_version 43912 (0.0007) [2023-10-07 21:30:54,859][67838] Updated weights for policy 0, policy_version 43922 (0.0007) [2023-10-07 21:30:55,232][67838] Updated weights for policy 0, policy_version 43932 (0.0007) [2023-10-07 21:30:57,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 90013696. Throughput: 0: 1645.8, 1: 1655.1. Samples: 22509722. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 21:30:57,478][66916] Avg episode reward: [(0, '47.120'), (1, '49.390')] [2023-10-07 21:30:57,493][67871] Updated weights for policy 1, policy_version 43970 (0.0008) [2023-10-07 21:30:57,867][67871] Updated weights for policy 1, policy_version 43980 (0.0007) [2023-10-07 21:30:58,235][67871] Updated weights for policy 1, policy_version 43990 (0.0009) [2023-10-07 21:30:58,600][67871] Updated weights for policy 1, policy_version 44000 (0.0007) [2023-10-07 21:30:59,172][67838] Updated weights for policy 0, policy_version 43942 (0.0009) [2023-10-07 21:30:59,536][67838] Updated weights for policy 0, policy_version 43952 (0.0008) [2023-10-07 21:30:59,905][67838] Updated weights for policy 0, policy_version 43962 (0.0008) [2023-10-07 21:31:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 90079232. Throughput: 0: 1660.1, 1: 1659.1. Samples: 22529846. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 21:31:02,477][66916] Avg episode reward: [(0, '43.440'), (1, '47.920')] [2023-10-07 21:31:02,676][67871] Updated weights for policy 1, policy_version 44010 (0.0011) [2023-10-07 21:31:03,053][67871] Updated weights for policy 1, policy_version 44020 (0.0008) [2023-10-07 21:31:03,414][67871] Updated weights for policy 1, policy_version 44030 (0.0007) [2023-10-07 21:31:04,081][67838] Updated weights for policy 0, policy_version 43972 (0.0009) [2023-10-07 21:31:04,452][67838] Updated weights for policy 0, policy_version 43982 (0.0007) [2023-10-07 21:31:04,813][67838] Updated weights for policy 0, policy_version 43992 (0.0008) [2023-10-07 21:31:07,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 90144768. Throughput: 0: 1660.6, 1: 1660.2. Samples: 22550376. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 21:31:07,478][66916] Avg episode reward: [(0, '41.500'), (1, '48.800')] [2023-10-07 21:31:07,491][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000044000_45056000.pth... [2023-10-07 21:31:07,531][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000042432_43450368.pth [2023-10-07 21:31:07,557][67871] Updated weights for policy 1, policy_version 44040 (0.0010) [2023-10-07 21:31:07,923][67871] Updated weights for policy 1, policy_version 44050 (0.0010) [2023-10-07 21:31:08,303][67871] Updated weights for policy 1, policy_version 44060 (0.0010) [2023-10-07 21:31:08,440][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000044064_45121536.pth... [2023-10-07 21:31:08,469][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000042496_43515904.pth [2023-10-07 21:31:08,907][67838] Updated weights for policy 0, policy_version 44002 (0.0008) [2023-10-07 21:31:09,288][67838] Updated weights for policy 0, policy_version 44012 (0.0010) [2023-10-07 21:31:09,650][67838] Updated weights for policy 0, policy_version 44022 (0.0008) [2023-10-07 21:31:10,035][67838] Updated weights for policy 0, policy_version 44032 (0.0010) [2023-10-07 21:31:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 90210304. Throughput: 0: 1641.4, 1: 1661.3. Samples: 22559366. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 21:31:12,477][66916] Avg episode reward: [(0, '44.550'), (1, '48.350')] [2023-10-07 21:31:12,621][67871] Updated weights for policy 1, policy_version 44070 (0.0007) [2023-10-07 21:31:12,996][67871] Updated weights for policy 1, policy_version 44080 (0.0008) [2023-10-07 21:31:13,369][67871] Updated weights for policy 1, policy_version 44090 (0.0007) [2023-10-07 21:31:14,169][67838] Updated weights for policy 0, policy_version 44042 (0.0007) [2023-10-07 21:31:14,541][67838] Updated weights for policy 0, policy_version 44052 (0.0009) [2023-10-07 21:31:14,917][67838] Updated weights for policy 0, policy_version 44062 (0.0008) [2023-10-07 21:31:17,350][67871] Updated weights for policy 1, policy_version 44100 (0.0008) [2023-10-07 21:31:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 90275840. Throughput: 0: 1659.9, 1: 1660.9. Samples: 22579560. Policy #0 lag: (min: 4.0, avg: 10.7, max: 36.0) [2023-10-07 21:31:17,477][66916] Avg episode reward: [(0, '38.490'), (1, '47.160')] [2023-10-07 21:31:17,713][67871] Updated weights for policy 1, policy_version 44110 (0.0008) [2023-10-07 21:31:18,078][67871] Updated weights for policy 1, policy_version 44120 (0.0009) [2023-10-07 21:31:19,093][67838] Updated weights for policy 0, policy_version 44072 (0.0008) [2023-10-07 21:31:19,468][67838] Updated weights for policy 0, policy_version 44082 (0.0009) [2023-10-07 21:31:19,843][67838] Updated weights for policy 0, policy_version 44092 (0.0008) [2023-10-07 21:31:22,338][67871] Updated weights for policy 1, policy_version 44130 (0.0007) [2023-10-07 21:31:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 90341376. Throughput: 0: 1660.9, 1: 1653.1. Samples: 22599980. Policy #0 lag: (min: 4.0, avg: 10.7, max: 36.0) [2023-10-07 21:31:22,478][66916] Avg episode reward: [(0, '39.430'), (1, '44.710')] [2023-10-07 21:31:22,712][67871] Updated weights for policy 1, policy_version 44140 (0.0009) [2023-10-07 21:31:23,089][67871] Updated weights for policy 1, policy_version 44150 (0.0010) [2023-10-07 21:31:23,454][67871] Updated weights for policy 1, policy_version 44160 (0.0011) [2023-10-07 21:31:23,917][67838] Updated weights for policy 0, policy_version 44102 (0.0010) [2023-10-07 21:31:24,289][67838] Updated weights for policy 0, policy_version 44112 (0.0010) [2023-10-07 21:31:24,656][67838] Updated weights for policy 0, policy_version 44122 (0.0007) [2023-10-07 21:31:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 90406912. Throughput: 0: 1651.0, 1: 1658.6. Samples: 22609014. Policy #0 lag: (min: 4.0, avg: 10.7, max: 36.0) [2023-10-07 21:31:27,477][66916] Avg episode reward: [(0, '39.310'), (1, '43.110')] [2023-10-07 21:31:27,826][67871] Updated weights for policy 1, policy_version 44170 (0.0007) [2023-10-07 21:31:28,196][67871] Updated weights for policy 1, policy_version 44180 (0.0009) [2023-10-07 21:31:28,560][67871] Updated weights for policy 1, policy_version 44190 (0.0009) [2023-10-07 21:31:28,912][67838] Updated weights for policy 0, policy_version 44132 (0.0008) [2023-10-07 21:31:29,305][67838] Updated weights for policy 0, policy_version 44142 (0.0007) [2023-10-07 21:31:29,674][67838] Updated weights for policy 0, policy_version 44152 (0.0008) [2023-10-07 21:31:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 90472448. Throughput: 0: 1665.4, 1: 1654.4. Samples: 22629172. Policy #0 lag: (min: 4.0, avg: 10.7, max: 36.0) [2023-10-07 21:31:32,477][66916] Avg episode reward: [(0, '38.430'), (1, '46.980')] [2023-10-07 21:31:32,672][67871] Updated weights for policy 1, policy_version 44200 (0.0011) [2023-10-07 21:31:33,038][67871] Updated weights for policy 1, policy_version 44210 (0.0010) [2023-10-07 21:31:33,406][67871] Updated weights for policy 1, policy_version 44220 (0.0008) [2023-10-07 21:31:33,824][67838] Updated weights for policy 0, policy_version 44162 (0.0009) [2023-10-07 21:31:34,193][67838] Updated weights for policy 0, policy_version 44172 (0.0008) [2023-10-07 21:31:34,571][67838] Updated weights for policy 0, policy_version 44182 (0.0007) [2023-10-07 21:31:34,936][67838] Updated weights for policy 0, policy_version 44192 (0.0007) [2023-10-07 21:31:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 90537984. Throughput: 0: 1658.4, 1: 1648.3. Samples: 22649108. Policy #0 lag: (min: 4.0, avg: 10.7, max: 36.0) [2023-10-07 21:31:37,477][66916] Avg episode reward: [(0, '39.730'), (1, '46.420')] [2023-10-07 21:31:37,526][67871] Updated weights for policy 1, policy_version 44230 (0.0008) [2023-10-07 21:31:37,886][67871] Updated weights for policy 1, policy_version 44240 (0.0007) [2023-10-07 21:31:38,257][67871] Updated weights for policy 1, policy_version 44250 (0.0007) [2023-10-07 21:31:39,023][67838] Updated weights for policy 0, policy_version 44202 (0.0010) [2023-10-07 21:31:39,387][67838] Updated weights for policy 0, policy_version 44212 (0.0007) [2023-10-07 21:31:39,756][67838] Updated weights for policy 0, policy_version 44222 (0.0009) [2023-10-07 21:31:42,264][67871] Updated weights for policy 1, policy_version 44260 (0.0008) [2023-10-07 21:31:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 90603520. Throughput: 0: 1648.9, 1: 1649.7. Samples: 22658162. Policy #0 lag: (min: 4.0, avg: 10.7, max: 36.0) [2023-10-07 21:31:42,477][66916] Avg episode reward: [(0, '41.090'), (1, '51.960')] [2023-10-07 21:31:42,630][67871] Updated weights for policy 1, policy_version 44270 (0.0007) [2023-10-07 21:31:43,004][67871] Updated weights for policy 1, policy_version 44280 (0.0008) [2023-10-07 21:31:44,047][67838] Updated weights for policy 0, policy_version 44232 (0.0008) [2023-10-07 21:31:44,421][67838] Updated weights for policy 0, policy_version 44242 (0.0007) [2023-10-07 21:31:44,791][67838] Updated weights for policy 0, policy_version 44252 (0.0009) [2023-10-07 21:31:47,213][67871] Updated weights for policy 1, policy_version 44290 (0.0008) [2023-10-07 21:31:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 90669056. Throughput: 0: 1653.0, 1: 1650.4. Samples: 22678496. Policy #0 lag: (min: 8.0, avg: 24.8, max: 40.0) [2023-10-07 21:31:47,477][66916] Avg episode reward: [(0, '41.400'), (1, '52.370')] [2023-10-07 21:31:47,579][67871] Updated weights for policy 1, policy_version 44300 (0.0009) [2023-10-07 21:31:47,941][67871] Updated weights for policy 1, policy_version 44310 (0.0009) [2023-10-07 21:31:48,302][67871] Updated weights for policy 1, policy_version 44320 (0.0007) [2023-10-07 21:31:48,832][67838] Updated weights for policy 0, policy_version 44262 (0.0009) [2023-10-07 21:31:49,207][67838] Updated weights for policy 0, policy_version 44272 (0.0008) [2023-10-07 21:31:49,587][67838] Updated weights for policy 0, policy_version 44282 (0.0010) [2023-10-07 21:31:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 90734592. Throughput: 0: 1653.9, 1: 1647.0. Samples: 22698916. Policy #0 lag: (min: 8.0, avg: 24.8, max: 40.0) [2023-10-07 21:31:52,477][66916] Avg episode reward: [(0, '41.850'), (1, '56.340')] [2023-10-07 21:31:52,518][67871] Updated weights for policy 1, policy_version 44330 (0.0009) [2023-10-07 21:31:52,879][67871] Updated weights for policy 1, policy_version 44340 (0.0008) [2023-10-07 21:31:53,249][67871] Updated weights for policy 1, policy_version 44350 (0.0009) [2023-10-07 21:31:53,723][67838] Updated weights for policy 0, policy_version 44292 (0.0007) [2023-10-07 21:31:54,093][67838] Updated weights for policy 0, policy_version 44302 (0.0011) [2023-10-07 21:31:54,469][67838] Updated weights for policy 0, policy_version 44312 (0.0008) [2023-10-07 21:31:57,226][67871] Updated weights for policy 1, policy_version 44360 (0.0007) [2023-10-07 21:31:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 90800128. Throughput: 0: 1654.6, 1: 1647.2. Samples: 22707950. Policy #0 lag: (min: 8.0, avg: 24.8, max: 40.0) [2023-10-07 21:31:57,477][66916] Avg episode reward: [(0, '40.960'), (1, '56.500')] [2023-10-07 21:31:57,594][67871] Updated weights for policy 1, policy_version 44370 (0.0008) [2023-10-07 21:31:57,950][67871] Updated weights for policy 1, policy_version 44380 (0.0009) [2023-10-07 21:31:58,443][67838] Updated weights for policy 0, policy_version 44322 (0.0008) [2023-10-07 21:31:58,818][67838] Updated weights for policy 0, policy_version 44332 (0.0009) [2023-10-07 21:31:59,187][67838] Updated weights for policy 0, policy_version 44342 (0.0007) [2023-10-07 21:31:59,568][67838] Updated weights for policy 0, policy_version 44352 (0.0008) [2023-10-07 21:32:02,063][67871] Updated weights for policy 1, policy_version 44390 (0.0009) [2023-10-07 21:32:02,432][67871] Updated weights for policy 1, policy_version 44400 (0.0008) [2023-10-07 21:32:02,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 90865664. Throughput: 0: 1656.6, 1: 1650.8. Samples: 22728392. Policy #0 lag: (min: 8.0, avg: 24.8, max: 40.0) [2023-10-07 21:32:02,478][66916] Avg episode reward: [(0, '41.800'), (1, '55.220')] [2023-10-07 21:32:02,803][67871] Updated weights for policy 1, policy_version 44410 (0.0007) [2023-10-07 21:32:03,706][67838] Updated weights for policy 0, policy_version 44362 (0.0010) [2023-10-07 21:32:04,071][67838] Updated weights for policy 0, policy_version 44372 (0.0008) [2023-10-07 21:32:04,442][67838] Updated weights for policy 0, policy_version 44382 (0.0007) [2023-10-07 21:32:07,055][67871] Updated weights for policy 1, policy_version 44420 (0.0009) [2023-10-07 21:32:07,430][67871] Updated weights for policy 1, policy_version 44430 (0.0010) [2023-10-07 21:32:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 90931200. Throughput: 0: 1659.0, 1: 1648.8. Samples: 22748834. Policy #0 lag: (min: 8.0, avg: 24.8, max: 40.0) [2023-10-07 21:32:07,477][66916] Avg episode reward: [(0, '39.560'), (1, '54.930')] [2023-10-07 21:32:07,804][67871] Updated weights for policy 1, policy_version 44440 (0.0009) [2023-10-07 21:32:08,534][67838] Updated weights for policy 0, policy_version 44392 (0.0008) [2023-10-07 21:32:08,893][67838] Updated weights for policy 0, policy_version 44402 (0.0008) [2023-10-07 21:32:09,271][67838] Updated weights for policy 0, policy_version 44412 (0.0008) [2023-10-07 21:32:12,044][67871] Updated weights for policy 1, policy_version 44450 (0.0009) [2023-10-07 21:32:12,473][67871] Updated weights for policy 1, policy_version 44460 (0.0008) [2023-10-07 21:32:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 90996736. Throughput: 0: 1655.7, 1: 1646.9. Samples: 22757634. Policy #0 lag: (min: 8.0, avg: 24.8, max: 40.0) [2023-10-07 21:32:12,477][66916] Avg episode reward: [(0, '41.000'), (1, '52.680')] [2023-10-07 21:32:12,847][67871] Updated weights for policy 1, policy_version 44470 (0.0008) [2023-10-07 21:32:13,211][67871] Updated weights for policy 1, policy_version 44480 (0.0007) [2023-10-07 21:32:13,433][67838] Updated weights for policy 0, policy_version 44422 (0.0010) [2023-10-07 21:32:13,812][67838] Updated weights for policy 0, policy_version 44432 (0.0010) [2023-10-07 21:32:14,189][67838] Updated weights for policy 0, policy_version 44442 (0.0011) [2023-10-07 21:32:17,430][67871] Updated weights for policy 1, policy_version 44490 (0.0010) [2023-10-07 21:32:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 91062272. Throughput: 0: 1659.8, 1: 1646.1. Samples: 22777936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:32:17,478][66916] Avg episode reward: [(0, '38.600'), (1, '49.030')] [2023-10-07 21:32:17,805][67871] Updated weights for policy 1, policy_version 44500 (0.0008) [2023-10-07 21:32:18,175][67871] Updated weights for policy 1, policy_version 44510 (0.0007) [2023-10-07 21:32:18,428][67838] Updated weights for policy 0, policy_version 44452 (0.0008) [2023-10-07 21:32:18,822][67838] Updated weights for policy 0, policy_version 44462 (0.0009) [2023-10-07 21:32:19,197][67838] Updated weights for policy 0, policy_version 44472 (0.0007) [2023-10-07 21:32:22,178][67871] Updated weights for policy 1, policy_version 44520 (0.0010) [2023-10-07 21:32:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 91127808. Throughput: 0: 1660.8, 1: 1652.3. Samples: 22798194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:32:22,477][66916] Avg episode reward: [(0, '39.060'), (1, '49.280')] [2023-10-07 21:32:22,545][67871] Updated weights for policy 1, policy_version 44530 (0.0009) [2023-10-07 21:32:22,910][67871] Updated weights for policy 1, policy_version 44540 (0.0010) [2023-10-07 21:32:23,088][67838] Updated weights for policy 0, policy_version 44482 (0.0008) [2023-10-07 21:32:23,465][67838] Updated weights for policy 0, policy_version 44492 (0.0011) [2023-10-07 21:32:23,834][67838] Updated weights for policy 0, policy_version 44502 (0.0010) [2023-10-07 21:32:24,213][67838] Updated weights for policy 0, policy_version 44512 (0.0008) [2023-10-07 21:32:27,053][67871] Updated weights for policy 1, policy_version 44550 (0.0007) [2023-10-07 21:32:27,425][67871] Updated weights for policy 1, policy_version 44560 (0.0008) [2023-10-07 21:32:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 91193344. Throughput: 0: 1660.5, 1: 1652.5. Samples: 22807248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:32:27,477][66916] Avg episode reward: [(0, '41.730'), (1, '49.630')] [2023-10-07 21:32:27,783][67871] Updated weights for policy 1, policy_version 44570 (0.0008) [2023-10-07 21:32:28,305][67838] Updated weights for policy 0, policy_version 44522 (0.0009) [2023-10-07 21:32:28,680][67838] Updated weights for policy 0, policy_version 44532 (0.0009) [2023-10-07 21:32:29,054][67838] Updated weights for policy 0, policy_version 44542 (0.0008) [2023-10-07 21:32:32,025][67871] Updated weights for policy 1, policy_version 44580 (0.0008) [2023-10-07 21:32:32,388][67871] Updated weights for policy 1, policy_version 44590 (0.0009) [2023-10-07 21:32:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 91258880. Throughput: 0: 1668.9, 1: 1650.9. Samples: 22827888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:32:32,477][66916] Avg episode reward: [(0, '43.620'), (1, '49.340')] [2023-10-07 21:32:32,748][67871] Updated weights for policy 1, policy_version 44600 (0.0008) [2023-10-07 21:32:33,052][67838] Updated weights for policy 0, policy_version 44552 (0.0009) [2023-10-07 21:32:33,434][67838] Updated weights for policy 0, policy_version 44562 (0.0010) [2023-10-07 21:32:33,800][67838] Updated weights for policy 0, policy_version 44572 (0.0008) [2023-10-07 21:32:36,851][67871] Updated weights for policy 1, policy_version 44610 (0.0009) [2023-10-07 21:32:37,226][67871] Updated weights for policy 1, policy_version 44620 (0.0010) [2023-10-07 21:32:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 91324416. Throughput: 0: 1668.9, 1: 1652.4. Samples: 22848372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:32:37,477][66916] Avg episode reward: [(0, '42.720'), (1, '48.630')] [2023-10-07 21:32:37,602][67871] Updated weights for policy 1, policy_version 44630 (0.0008) [2023-10-07 21:32:37,969][67871] Updated weights for policy 1, policy_version 44640 (0.0007) [2023-10-07 21:32:38,098][67838] Updated weights for policy 0, policy_version 44582 (0.0009) [2023-10-07 21:32:38,467][67838] Updated weights for policy 0, policy_version 44592 (0.0010) [2023-10-07 21:32:38,843][67838] Updated weights for policy 0, policy_version 44602 (0.0010) [2023-10-07 21:32:41,882][67871] Updated weights for policy 1, policy_version 44650 (0.0009) [2023-10-07 21:32:42,255][67871] Updated weights for policy 1, policy_version 44660 (0.0007) [2023-10-07 21:32:42,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 91389952. Throughput: 0: 1665.6, 1: 1655.6. Samples: 22857402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:32:42,477][66916] Avg episode reward: [(0, '39.770'), (1, '50.590')] [2023-10-07 21:32:42,624][67871] Updated weights for policy 1, policy_version 44670 (0.0007) [2023-10-07 21:32:43,031][67838] Updated weights for policy 0, policy_version 44612 (0.0007) [2023-10-07 21:32:43,405][67838] Updated weights for policy 0, policy_version 44622 (0.0008) [2023-10-07 21:32:43,764][67838] Updated weights for policy 0, policy_version 44632 (0.0011) [2023-10-07 21:32:46,686][67871] Updated weights for policy 1, policy_version 44680 (0.0009) [2023-10-07 21:32:47,057][67871] Updated weights for policy 1, policy_version 44690 (0.0009) [2023-10-07 21:32:47,421][67871] Updated weights for policy 1, policy_version 44700 (0.0007) [2023-10-07 21:32:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 91455488. Throughput: 0: 1659.9, 1: 1658.1. Samples: 22877698. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 21:32:47,477][66916] Avg episode reward: [(0, '41.970'), (1, '48.500')] [2023-10-07 21:32:47,977][67838] Updated weights for policy 0, policy_version 44642 (0.0010) [2023-10-07 21:32:48,346][67838] Updated weights for policy 0, policy_version 44652 (0.0011) [2023-10-07 21:32:48,718][67838] Updated weights for policy 0, policy_version 44662 (0.0008) [2023-10-07 21:32:49,095][67838] Updated weights for policy 0, policy_version 44672 (0.0008) [2023-10-07 21:32:51,593][67871] Updated weights for policy 1, policy_version 44710 (0.0008) [2023-10-07 21:32:51,960][67871] Updated weights for policy 1, policy_version 44720 (0.0010) [2023-10-07 21:32:52,331][67871] Updated weights for policy 1, policy_version 44730 (0.0007) [2023-10-07 21:32:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 91521024. Throughput: 0: 1655.4, 1: 1656.0. Samples: 22897844. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 21:32:52,477][66916] Avg episode reward: [(0, '36.650'), (1, '48.750')] [2023-10-07 21:32:53,280][67838] Updated weights for policy 0, policy_version 44682 (0.0008) [2023-10-07 21:32:53,657][67838] Updated weights for policy 0, policy_version 44692 (0.0010) [2023-10-07 21:32:54,030][67838] Updated weights for policy 0, policy_version 44702 (0.0008) [2023-10-07 21:32:56,376][67871] Updated weights for policy 1, policy_version 44740 (0.0007) [2023-10-07 21:32:56,741][67871] Updated weights for policy 1, policy_version 44750 (0.0008) [2023-10-07 21:32:57,105][67871] Updated weights for policy 1, policy_version 44760 (0.0010) [2023-10-07 21:32:57,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 91619328. Throughput: 0: 1658.7, 1: 1670.2. Samples: 22907434. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 21:32:57,478][66916] Avg episode reward: [(0, '39.500'), (1, '49.680')] [2023-10-07 21:32:58,151][67838] Updated weights for policy 0, policy_version 44712 (0.0008) [2023-10-07 21:32:58,520][67838] Updated weights for policy 0, policy_version 44722 (0.0009) [2023-10-07 21:32:58,892][67838] Updated weights for policy 0, policy_version 44732 (0.0010) [2023-10-07 21:33:01,073][67871] Updated weights for policy 1, policy_version 44770 (0.0008) [2023-10-07 21:33:01,451][67871] Updated weights for policy 1, policy_version 44780 (0.0007) [2023-10-07 21:33:01,813][67871] Updated weights for policy 1, policy_version 44790 (0.0009) [2023-10-07 21:33:02,183][67871] Updated weights for policy 1, policy_version 44800 (0.0009) [2023-10-07 21:33:02,476][66916] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 91684864. Throughput: 0: 1653.0, 1: 1682.3. Samples: 22928024. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 21:33:02,477][66916] Avg episode reward: [(0, '39.620'), (1, '46.740')] [2023-10-07 21:33:03,149][67838] Updated weights for policy 0, policy_version 44742 (0.0010) [2023-10-07 21:33:03,521][67838] Updated weights for policy 0, policy_version 44752 (0.0007) [2023-10-07 21:33:03,895][67838] Updated weights for policy 0, policy_version 44762 (0.0010) [2023-10-07 21:33:06,229][67871] Updated weights for policy 1, policy_version 44810 (0.0007) [2023-10-07 21:33:06,592][67871] Updated weights for policy 1, policy_version 44820 (0.0007) [2023-10-07 21:33:06,956][67871] Updated weights for policy 1, policy_version 44830 (0.0008) [2023-10-07 21:33:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 91750400. Throughput: 0: 1660.2, 1: 1664.2. Samples: 22947790. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 21:33:07,477][66916] Avg episode reward: [(0, '40.340'), (1, '50.600')] [2023-10-07 21:33:07,486][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000044832_45907968.pth... [2023-10-07 21:33:07,487][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000044768_45842432.pth... [2023-10-07 21:33:07,516][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000043264_44302336.pth [2023-10-07 21:33:07,520][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000043232_44269568.pth [2023-10-07 21:33:07,986][67838] Updated weights for policy 0, policy_version 44772 (0.0010) [2023-10-07 21:33:08,390][67838] Updated weights for policy 0, policy_version 44782 (0.0011) [2023-10-07 21:33:08,755][67838] Updated weights for policy 0, policy_version 44792 (0.0009) [2023-10-07 21:33:11,154][67871] Updated weights for policy 1, policy_version 44840 (0.0010) [2023-10-07 21:33:11,532][67871] Updated weights for policy 1, policy_version 44850 (0.0010) [2023-10-07 21:33:11,901][67871] Updated weights for policy 1, policy_version 44860 (0.0010) [2023-10-07 21:33:12,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 91815936. Throughput: 0: 1656.8, 1: 1684.3. Samples: 22957596. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 21:33:12,478][66916] Avg episode reward: [(0, '41.860'), (1, '53.460')] [2023-10-07 21:33:12,860][67838] Updated weights for policy 0, policy_version 44802 (0.0009) [2023-10-07 21:33:13,239][67838] Updated weights for policy 0, policy_version 44812 (0.0009) [2023-10-07 21:33:13,604][67838] Updated weights for policy 0, policy_version 44822 (0.0007) [2023-10-07 21:33:13,974][67838] Updated weights for policy 0, policy_version 44832 (0.0009) [2023-10-07 21:33:16,076][67871] Updated weights for policy 1, policy_version 44870 (0.0009) [2023-10-07 21:33:16,445][67871] Updated weights for policy 1, policy_version 44880 (0.0011) [2023-10-07 21:33:16,811][67871] Updated weights for policy 1, policy_version 44890 (0.0009) [2023-10-07 21:33:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 91881472. Throughput: 0: 1657.0, 1: 1681.7. Samples: 22978132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:33:17,477][66916] Avg episode reward: [(0, '40.270'), (1, '54.680')] [2023-10-07 21:33:17,816][67838] Updated weights for policy 0, policy_version 44842 (0.0009) [2023-10-07 21:33:18,195][67838] Updated weights for policy 0, policy_version 44852 (0.0011) [2023-10-07 21:33:18,575][67838] Updated weights for policy 0, policy_version 44862 (0.0011) [2023-10-07 21:33:20,701][67871] Updated weights for policy 1, policy_version 44900 (0.0008) [2023-10-07 21:33:21,075][67871] Updated weights for policy 1, policy_version 44910 (0.0008) [2023-10-07 21:33:21,435][67871] Updated weights for policy 1, policy_version 44920 (0.0008) [2023-10-07 21:33:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 91947008. Throughput: 0: 1656.3, 1: 1660.8. Samples: 22997644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:33:22,477][66916] Avg episode reward: [(0, '41.710'), (1, '51.790')] [2023-10-07 21:33:22,780][67838] Updated weights for policy 0, policy_version 44872 (0.0008) [2023-10-07 21:33:23,155][67838] Updated weights for policy 0, policy_version 44882 (0.0009) [2023-10-07 21:33:23,519][67838] Updated weights for policy 0, policy_version 44892 (0.0007) [2023-10-07 21:33:25,467][67871] Updated weights for policy 1, policy_version 44930 (0.0010) [2023-10-07 21:33:25,845][67871] Updated weights for policy 1, policy_version 44940 (0.0008) [2023-10-07 21:33:26,215][67871] Updated weights for policy 1, policy_version 44950 (0.0009) [2023-10-07 21:33:26,591][67871] Updated weights for policy 1, policy_version 44960 (0.0009) [2023-10-07 21:33:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 92012544. Throughput: 0: 1661.6, 1: 1690.1. Samples: 23008232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:33:27,478][66916] Avg episode reward: [(0, '42.170'), (1, '55.320')] [2023-10-07 21:33:27,576][67838] Updated weights for policy 0, policy_version 44902 (0.0009) [2023-10-07 21:33:27,952][67838] Updated weights for policy 0, policy_version 44912 (0.0007) [2023-10-07 21:33:28,329][67838] Updated weights for policy 0, policy_version 44922 (0.0007) [2023-10-07 21:33:30,727][67871] Updated weights for policy 1, policy_version 44970 (0.0007) [2023-10-07 21:33:31,092][67871] Updated weights for policy 1, policy_version 44980 (0.0012) [2023-10-07 21:33:31,453][67871] Updated weights for policy 1, policy_version 44990 (0.0010) [2023-10-07 21:33:32,240][67838] Updated weights for policy 0, policy_version 44932 (0.0007) [2023-10-07 21:33:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 92078080. Throughput: 0: 1670.4, 1: 1675.8. Samples: 23028276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:33:32,477][66916] Avg episode reward: [(0, '44.030'), (1, '56.620')] [2023-10-07 21:33:32,618][67838] Updated weights for policy 0, policy_version 44942 (0.0007) [2023-10-07 21:33:32,993][67838] Updated weights for policy 0, policy_version 44952 (0.0009) [2023-10-07 21:33:35,441][67871] Updated weights for policy 1, policy_version 45000 (0.0009) [2023-10-07 21:33:35,807][67871] Updated weights for policy 1, policy_version 45010 (0.0009) [2023-10-07 21:33:36,176][67871] Updated weights for policy 1, policy_version 45020 (0.0009) [2023-10-07 21:33:37,010][67838] Updated weights for policy 0, policy_version 44962 (0.0007) [2023-10-07 21:33:37,383][67838] Updated weights for policy 0, policy_version 44972 (0.0007) [2023-10-07 21:33:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 92143616. Throughput: 0: 1670.4, 1: 1666.7. Samples: 23048016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:33:37,477][66916] Avg episode reward: [(0, '46.580'), (1, '54.160')] [2023-10-07 21:33:37,764][67838] Updated weights for policy 0, policy_version 44982 (0.0008) [2023-10-07 21:33:38,137][67838] Updated weights for policy 0, policy_version 44992 (0.0008) [2023-10-07 21:33:40,377][67871] Updated weights for policy 1, policy_version 45030 (0.0007) [2023-10-07 21:33:40,753][67871] Updated weights for policy 1, policy_version 45040 (0.0007) [2023-10-07 21:33:41,115][67871] Updated weights for policy 1, policy_version 45050 (0.0008) [2023-10-07 21:33:42,244][67838] Updated weights for policy 0, policy_version 45002 (0.0009) [2023-10-07 21:33:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 92209152. Throughput: 0: 1670.7, 1: 1683.5. Samples: 23058370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:33:42,478][66916] Avg episode reward: [(0, '46.160'), (1, '52.890')] [2023-10-07 21:33:42,627][67838] Updated weights for policy 0, policy_version 45012 (0.0009) [2023-10-07 21:33:43,001][67838] Updated weights for policy 0, policy_version 45022 (0.0008) [2023-10-07 21:33:45,417][67871] Updated weights for policy 1, policy_version 45060 (0.0009) [2023-10-07 21:33:45,823][67871] Updated weights for policy 1, policy_version 45070 (0.0009) [2023-10-07 21:33:46,191][67871] Updated weights for policy 1, policy_version 45080 (0.0007) [2023-10-07 21:33:47,261][67838] Updated weights for policy 0, policy_version 45032 (0.0009) [2023-10-07 21:33:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 92274688. Throughput: 0: 1672.1, 1: 1660.8. Samples: 23078002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:33:47,477][66916] Avg episode reward: [(0, '46.320'), (1, '52.070')] [2023-10-07 21:33:47,639][67838] Updated weights for policy 0, policy_version 45042 (0.0008) [2023-10-07 21:33:48,011][67838] Updated weights for policy 0, policy_version 45052 (0.0010) [2023-10-07 21:33:50,253][67871] Updated weights for policy 1, policy_version 45090 (0.0009) [2023-10-07 21:33:50,618][67871] Updated weights for policy 1, policy_version 45100 (0.0011) [2023-10-07 21:33:50,988][67871] Updated weights for policy 1, policy_version 45110 (0.0008) [2023-10-07 21:33:51,359][67871] Updated weights for policy 1, policy_version 45120 (0.0008) [2023-10-07 21:33:52,162][67838] Updated weights for policy 0, policy_version 45062 (0.0008) [2023-10-07 21:33:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 92340224. Throughput: 0: 1666.8, 1: 1663.3. Samples: 23097648. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-07 21:33:52,477][66916] Avg episode reward: [(0, '49.300'), (1, '52.530')] [2023-10-07 21:33:52,534][67838] Updated weights for policy 0, policy_version 45072 (0.0008) [2023-10-07 21:33:52,908][67838] Updated weights for policy 0, policy_version 45082 (0.0008) [2023-10-07 21:33:55,385][67871] Updated weights for policy 1, policy_version 45130 (0.0010) [2023-10-07 21:33:55,758][67871] Updated weights for policy 1, policy_version 45140 (0.0010) [2023-10-07 21:33:56,125][67871] Updated weights for policy 1, policy_version 45150 (0.0010) [2023-10-07 21:33:57,206][67838] Updated weights for policy 0, policy_version 45092 (0.0008) [2023-10-07 21:33:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 92405760. Throughput: 0: 1670.9, 1: 1671.7. Samples: 23108014. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-07 21:33:57,478][66916] Avg episode reward: [(0, '48.540'), (1, '51.790')] [2023-10-07 21:33:57,597][67838] Updated weights for policy 0, policy_version 45102 (0.0007) [2023-10-07 21:33:57,968][67838] Updated weights for policy 0, policy_version 45112 (0.0007) [2023-10-07 21:34:00,228][67871] Updated weights for policy 1, policy_version 45160 (0.0008) [2023-10-07 21:34:00,597][67871] Updated weights for policy 1, policy_version 45170 (0.0010) [2023-10-07 21:34:00,975][67871] Updated weights for policy 1, policy_version 45180 (0.0010) [2023-10-07 21:34:02,225][67838] Updated weights for policy 0, policy_version 45122 (0.0007) [2023-10-07 21:34:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 92471296. Throughput: 0: 1660.9, 1: 1653.9. Samples: 23127296. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-07 21:34:02,477][66916] Avg episode reward: [(0, '48.350'), (1, '54.550')] [2023-10-07 21:34:02,612][67838] Updated weights for policy 0, policy_version 45132 (0.0008) [2023-10-07 21:34:02,988][67838] Updated weights for policy 0, policy_version 45142 (0.0010) [2023-10-07 21:34:03,359][67838] Updated weights for policy 0, policy_version 45152 (0.0009) [2023-10-07 21:34:04,959][67871] Updated weights for policy 1, policy_version 45190 (0.0010) [2023-10-07 21:34:05,325][67871] Updated weights for policy 1, policy_version 45200 (0.0009) [2023-10-07 21:34:05,684][67871] Updated weights for policy 1, policy_version 45210 (0.0009) [2023-10-07 21:34:07,268][67838] Updated weights for policy 0, policy_version 45162 (0.0008) [2023-10-07 21:34:07,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 92536832. Throughput: 0: 1654.3, 1: 1673.5. Samples: 23147398. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-07 21:34:07,478][66916] Avg episode reward: [(0, '45.650'), (1, '55.810')] [2023-10-07 21:34:07,633][67838] Updated weights for policy 0, policy_version 45172 (0.0008) [2023-10-07 21:34:08,006][67838] Updated weights for policy 0, policy_version 45182 (0.0007) [2023-10-07 21:34:09,862][67871] Updated weights for policy 1, policy_version 45220 (0.0008) [2023-10-07 21:34:10,234][67871] Updated weights for policy 1, policy_version 45230 (0.0008) [2023-10-07 21:34:10,589][67871] Updated weights for policy 1, policy_version 45240 (0.0010) [2023-10-07 21:34:12,203][67838] Updated weights for policy 0, policy_version 45192 (0.0008) [2023-10-07 21:34:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 92602368. Throughput: 0: 1653.2, 1: 1666.1. Samples: 23157600. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-07 21:34:12,477][66916] Avg episode reward: [(0, '43.880'), (1, '55.810')] [2023-10-07 21:34:12,568][67838] Updated weights for policy 0, policy_version 45202 (0.0009) [2023-10-07 21:34:12,939][67838] Updated weights for policy 0, policy_version 45212 (0.0011) [2023-10-07 21:34:14,700][67871] Updated weights for policy 1, policy_version 45250 (0.0009) [2023-10-07 21:34:15,069][67871] Updated weights for policy 1, policy_version 45260 (0.0007) [2023-10-07 21:34:15,434][67871] Updated weights for policy 1, policy_version 45270 (0.0008) [2023-10-07 21:34:15,801][67871] Updated weights for policy 1, policy_version 45280 (0.0010) [2023-10-07 21:34:17,049][67838] Updated weights for policy 0, policy_version 45222 (0.0009) [2023-10-07 21:34:17,424][67838] Updated weights for policy 0, policy_version 45232 (0.0007) [2023-10-07 21:34:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 92667904. Throughput: 0: 1650.3, 1: 1653.9. Samples: 23176962. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-07 21:34:17,477][66916] Avg episode reward: [(0, '44.750'), (1, '55.740')] [2023-10-07 21:34:17,810][67838] Updated weights for policy 0, policy_version 45242 (0.0007) [2023-10-07 21:34:19,912][67871] Updated weights for policy 1, policy_version 45290 (0.0008) [2023-10-07 21:34:20,284][67871] Updated weights for policy 1, policy_version 45300 (0.0007) [2023-10-07 21:34:20,651][67871] Updated weights for policy 1, policy_version 45310 (0.0008) [2023-10-07 21:34:21,870][67838] Updated weights for policy 0, policy_version 45252 (0.0008) [2023-10-07 21:34:22,233][67838] Updated weights for policy 0, policy_version 45262 (0.0008) [2023-10-07 21:34:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 92733440. Throughput: 0: 1643.7, 1: 1664.2. Samples: 23196872. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-07 21:34:22,477][66916] Avg episode reward: [(0, '44.300'), (1, '51.530')] [2023-10-07 21:34:22,607][67838] Updated weights for policy 0, policy_version 45272 (0.0009) [2023-10-07 21:34:24,771][67871] Updated weights for policy 1, policy_version 45320 (0.0008) [2023-10-07 21:34:25,141][67871] Updated weights for policy 1, policy_version 45330 (0.0008) [2023-10-07 21:34:25,500][67871] Updated weights for policy 1, policy_version 45340 (0.0009) [2023-10-07 21:34:26,871][67838] Updated weights for policy 0, policy_version 45282 (0.0007) [2023-10-07 21:34:27,241][67838] Updated weights for policy 0, policy_version 45292 (0.0007) [2023-10-07 21:34:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 92798976. Throughput: 0: 1648.4, 1: 1659.6. Samples: 23207226. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-07 21:34:27,478][66916] Avg episode reward: [(0, '41.540'), (1, '47.290')] [2023-10-07 21:34:27,616][67838] Updated weights for policy 0, policy_version 45302 (0.0007) [2023-10-07 21:34:27,988][67838] Updated weights for policy 0, policy_version 45312 (0.0009) [2023-10-07 21:34:29,758][67871] Updated weights for policy 1, policy_version 45350 (0.0009) [2023-10-07 21:34:30,123][67871] Updated weights for policy 1, policy_version 45360 (0.0008) [2023-10-07 21:34:30,482][67871] Updated weights for policy 1, policy_version 45370 (0.0008) [2023-10-07 21:34:32,259][67838] Updated weights for policy 0, policy_version 45322 (0.0008) [2023-10-07 21:34:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 92864512. Throughput: 0: 1648.8, 1: 1653.1. Samples: 23226586. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-07 21:34:32,478][66916] Avg episode reward: [(0, '44.350'), (1, '49.890')] [2023-10-07 21:34:32,638][67838] Updated weights for policy 0, policy_version 45332 (0.0009) [2023-10-07 21:34:33,008][67838] Updated weights for policy 0, policy_version 45342 (0.0008) [2023-10-07 21:34:34,654][67871] Updated weights for policy 1, policy_version 45380 (0.0008) [2023-10-07 21:34:35,065][67871] Updated weights for policy 1, policy_version 45390 (0.0008) [2023-10-07 21:34:35,424][67871] Updated weights for policy 1, policy_version 45400 (0.0010) [2023-10-07 21:34:37,219][67838] Updated weights for policy 0, policy_version 45352 (0.0010) [2023-10-07 21:34:37,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 92930048. Throughput: 0: 1643.5, 1: 1667.1. Samples: 23246622. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-07 21:34:37,478][66916] Avg episode reward: [(0, '40.510'), (1, '49.640')] [2023-10-07 21:34:37,585][67838] Updated weights for policy 0, policy_version 45362 (0.0009) [2023-10-07 21:34:37,962][67838] Updated weights for policy 0, policy_version 45372 (0.0009) [2023-10-07 21:34:39,598][67871] Updated weights for policy 1, policy_version 45410 (0.0011) [2023-10-07 21:34:39,965][67871] Updated weights for policy 1, policy_version 45420 (0.0010) [2023-10-07 21:34:40,332][67871] Updated weights for policy 1, policy_version 45430 (0.0008) [2023-10-07 21:34:40,698][67871] Updated weights for policy 1, policy_version 45440 (0.0007) [2023-10-07 21:34:41,811][67838] Updated weights for policy 0, policy_version 45382 (0.0008) [2023-10-07 21:34:42,179][67838] Updated weights for policy 0, policy_version 45392 (0.0008) [2023-10-07 21:34:42,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 92995584. Throughput: 0: 1647.8, 1: 1655.3. Samples: 23256652. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-07 21:34:42,477][66916] Avg episode reward: [(0, '43.250'), (1, '51.640')] [2023-10-07 21:34:42,547][67838] Updated weights for policy 0, policy_version 45402 (0.0008) [2023-10-07 21:34:44,583][67871] Updated weights for policy 1, policy_version 45450 (0.0009) [2023-10-07 21:34:44,943][67871] Updated weights for policy 1, policy_version 45460 (0.0010) [2023-10-07 21:34:45,303][67871] Updated weights for policy 1, policy_version 45470 (0.0008) [2023-10-07 21:34:46,787][67838] Updated weights for policy 0, policy_version 45412 (0.0008) [2023-10-07 21:34:47,158][67838] Updated weights for policy 0, policy_version 45422 (0.0008) [2023-10-07 21:34:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 93061120. Throughput: 0: 1655.5, 1: 1656.7. Samples: 23276344. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-07 21:34:47,477][66916] Avg episode reward: [(0, '42.780'), (1, '51.500')] [2023-10-07 21:34:47,535][67838] Updated weights for policy 0, policy_version 45432 (0.0008) [2023-10-07 21:34:49,574][67871] Updated weights for policy 1, policy_version 45480 (0.0008) [2023-10-07 21:34:49,930][67871] Updated weights for policy 1, policy_version 45490 (0.0008) [2023-10-07 21:34:50,304][67871] Updated weights for policy 1, policy_version 45500 (0.0007) [2023-10-07 21:34:51,654][67838] Updated weights for policy 0, policy_version 45442 (0.0009) [2023-10-07 21:34:52,016][67838] Updated weights for policy 0, policy_version 45452 (0.0007) [2023-10-07 21:34:52,394][67838] Updated weights for policy 0, policy_version 45462 (0.0007) [2023-10-07 21:34:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 93126656. Throughput: 0: 1649.7, 1: 1661.0. Samples: 23296382. Policy #0 lag: (min: 9.0, avg: 27.6, max: 41.0) [2023-10-07 21:34:52,477][66916] Avg episode reward: [(0, '42.210'), (1, '51.900')] [2023-10-07 21:34:52,768][67838] Updated weights for policy 0, policy_version 45472 (0.0007) [2023-10-07 21:34:54,375][67871] Updated weights for policy 1, policy_version 45510 (0.0009) [2023-10-07 21:34:54,744][67871] Updated weights for policy 1, policy_version 45520 (0.0008) [2023-10-07 21:34:55,114][67871] Updated weights for policy 1, policy_version 45530 (0.0007) [2023-10-07 21:34:56,876][67838] Updated weights for policy 0, policy_version 45482 (0.0010) [2023-10-07 21:34:57,253][67838] Updated weights for policy 0, policy_version 45492 (0.0008) [2023-10-07 21:34:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 93192192. Throughput: 0: 1658.4, 1: 1649.7. Samples: 23306466. Policy #0 lag: (min: 9.0, avg: 27.6, max: 41.0) [2023-10-07 21:34:57,477][66916] Avg episode reward: [(0, '46.860'), (1, '52.330')] [2023-10-07 21:34:57,620][67838] Updated weights for policy 0, policy_version 45502 (0.0008) [2023-10-07 21:34:59,047][67871] Updated weights for policy 1, policy_version 45540 (0.0008) [2023-10-07 21:34:59,410][67871] Updated weights for policy 1, policy_version 45550 (0.0009) [2023-10-07 21:34:59,778][67871] Updated weights for policy 1, policy_version 45560 (0.0008) [2023-10-07 21:35:01,710][67838] Updated weights for policy 0, policy_version 45512 (0.0009) [2023-10-07 21:35:02,079][67838] Updated weights for policy 0, policy_version 45522 (0.0008) [2023-10-07 21:35:02,446][67838] Updated weights for policy 0, policy_version 45532 (0.0007) [2023-10-07 21:35:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 93257728. Throughput: 0: 1659.7, 1: 1663.7. Samples: 23326516. Policy #0 lag: (min: 9.0, avg: 27.6, max: 41.0) [2023-10-07 21:35:02,477][66916] Avg episode reward: [(0, '43.510'), (1, '52.690')] [2023-10-07 21:35:03,983][67871] Updated weights for policy 1, policy_version 45570 (0.0009) [2023-10-07 21:35:04,346][67871] Updated weights for policy 1, policy_version 45580 (0.0009) [2023-10-07 21:35:04,716][67871] Updated weights for policy 1, policy_version 45590 (0.0009) [2023-10-07 21:35:05,074][67871] Updated weights for policy 1, policy_version 45600 (0.0010) [2023-10-07 21:35:06,475][67838] Updated weights for policy 0, policy_version 45542 (0.0008) [2023-10-07 21:35:06,848][67838] Updated weights for policy 0, policy_version 45552 (0.0011) [2023-10-07 21:35:07,221][67838] Updated weights for policy 0, policy_version 45562 (0.0009) [2023-10-07 21:35:07,477][66916] Fps is (10 sec: 16383.4, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 93356032. Throughput: 0: 1654.4, 1: 1670.2. Samples: 23346482. Policy #0 lag: (min: 9.0, avg: 27.6, max: 41.0) [2023-10-07 21:35:07,478][66916] Avg episode reward: [(0, '46.760'), (1, '49.750')] [2023-10-07 21:35:07,487][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000045568_46661632.pth... [2023-10-07 21:35:07,488][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000045600_46694400.pth... [2023-10-07 21:35:07,518][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000044000_45056000.pth [2023-10-07 21:35:07,526][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000044064_45121536.pth [2023-10-07 21:35:09,142][67871] Updated weights for policy 1, policy_version 45610 (0.0008) [2023-10-07 21:35:09,513][67871] Updated weights for policy 1, policy_version 45620 (0.0007) [2023-10-07 21:35:09,872][67871] Updated weights for policy 1, policy_version 45630 (0.0010) [2023-10-07 21:35:11,217][67838] Updated weights for policy 0, policy_version 45572 (0.0009) [2023-10-07 21:35:11,593][67838] Updated weights for policy 0, policy_version 45582 (0.0008) [2023-10-07 21:35:11,962][67838] Updated weights for policy 0, policy_version 45592 (0.0008) [2023-10-07 21:35:12,476][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 93421568. Throughput: 0: 1668.8, 1: 1649.7. Samples: 23356556. Policy #0 lag: (min: 9.0, avg: 27.6, max: 41.0) [2023-10-07 21:35:12,477][66916] Avg episode reward: [(0, '46.290'), (1, '51.450')] [2023-10-07 21:35:14,013][67871] Updated weights for policy 1, policy_version 45640 (0.0010) [2023-10-07 21:35:14,381][67871] Updated weights for policy 1, policy_version 45650 (0.0009) [2023-10-07 21:35:14,758][67871] Updated weights for policy 1, policy_version 45660 (0.0010) [2023-10-07 21:35:15,989][67838] Updated weights for policy 0, policy_version 45602 (0.0007) [2023-10-07 21:35:16,363][67838] Updated weights for policy 0, policy_version 45612 (0.0007) [2023-10-07 21:35:16,736][67838] Updated weights for policy 0, policy_version 45622 (0.0010) [2023-10-07 21:35:17,111][67838] Updated weights for policy 0, policy_version 45632 (0.0008) [2023-10-07 21:35:17,477][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 93487104. Throughput: 0: 1670.2, 1: 1669.7. Samples: 23376882. Policy #0 lag: (min: 6.0, avg: 16.7, max: 38.0) [2023-10-07 21:35:17,477][66916] Avg episode reward: [(0, '46.210'), (1, '50.590')] [2023-10-07 21:35:18,891][67871] Updated weights for policy 1, policy_version 45670 (0.0010) [2023-10-07 21:35:19,287][67871] Updated weights for policy 1, policy_version 45680 (0.0009) [2023-10-07 21:35:19,657][67871] Updated weights for policy 1, policy_version 45690 (0.0010) [2023-10-07 21:35:21,055][67838] Updated weights for policy 0, policy_version 45642 (0.0010) [2023-10-07 21:35:21,443][67838] Updated weights for policy 0, policy_version 45652 (0.0008) [2023-10-07 21:35:21,811][67838] Updated weights for policy 0, policy_version 45662 (0.0009) [2023-10-07 21:35:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 93552640. Throughput: 0: 1658.6, 1: 1666.3. Samples: 23396242. Policy #0 lag: (min: 6.0, avg: 16.7, max: 38.0) [2023-10-07 21:35:22,477][66916] Avg episode reward: [(0, '44.220'), (1, '51.000')] [2023-10-07 21:35:23,785][67871] Updated weights for policy 1, policy_version 45700 (0.0009) [2023-10-07 21:35:24,151][67871] Updated weights for policy 1, policy_version 45710 (0.0010) [2023-10-07 21:35:24,517][67871] Updated weights for policy 1, policy_version 45720 (0.0007) [2023-10-07 21:35:25,982][67838] Updated weights for policy 0, policy_version 45672 (0.0008) [2023-10-07 21:35:26,364][67838] Updated weights for policy 0, policy_version 45682 (0.0011) [2023-10-07 21:35:26,735][67838] Updated weights for policy 0, policy_version 45692 (0.0010) [2023-10-07 21:35:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 93618176. Throughput: 0: 1684.3, 1: 1650.3. Samples: 23406708. Policy #0 lag: (min: 6.0, avg: 16.7, max: 38.0) [2023-10-07 21:35:27,477][66916] Avg episode reward: [(0, '43.340'), (1, '50.260')] [2023-10-07 21:35:28,591][67871] Updated weights for policy 1, policy_version 45730 (0.0007) [2023-10-07 21:35:28,962][67871] Updated weights for policy 1, policy_version 45740 (0.0010) [2023-10-07 21:35:29,329][67871] Updated weights for policy 1, policy_version 45750 (0.0009) [2023-10-07 21:35:29,699][67871] Updated weights for policy 1, policy_version 45760 (0.0010) [2023-10-07 21:35:30,952][67838] Updated weights for policy 0, policy_version 45702 (0.0009) [2023-10-07 21:35:31,332][67838] Updated weights for policy 0, policy_version 45712 (0.0008) [2023-10-07 21:35:31,690][67838] Updated weights for policy 0, policy_version 45722 (0.0008) [2023-10-07 21:35:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 93683712. Throughput: 0: 1671.1, 1: 1666.7. Samples: 23426544. Policy #0 lag: (min: 6.0, avg: 16.7, max: 38.0) [2023-10-07 21:35:32,478][66916] Avg episode reward: [(0, '44.650'), (1, '51.770')] [2023-10-07 21:35:33,842][67871] Updated weights for policy 1, policy_version 45770 (0.0007) [2023-10-07 21:35:34,200][67871] Updated weights for policy 1, policy_version 45780 (0.0007) [2023-10-07 21:35:34,582][67871] Updated weights for policy 1, policy_version 45790 (0.0009) [2023-10-07 21:35:35,765][67838] Updated weights for policy 0, policy_version 45732 (0.0008) [2023-10-07 21:35:36,129][67838] Updated weights for policy 0, policy_version 45742 (0.0009) [2023-10-07 21:35:36,499][67838] Updated weights for policy 0, policy_version 45752 (0.0008) [2023-10-07 21:35:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 93749248. Throughput: 0: 1664.1, 1: 1670.0. Samples: 23446416. Policy #0 lag: (min: 6.0, avg: 16.7, max: 38.0) [2023-10-07 21:35:37,477][66916] Avg episode reward: [(0, '43.490'), (1, '47.960')] [2023-10-07 21:35:38,546][67871] Updated weights for policy 1, policy_version 45800 (0.0009) [2023-10-07 21:35:38,915][67871] Updated weights for policy 1, policy_version 45810 (0.0007) [2023-10-07 21:35:39,282][67871] Updated weights for policy 1, policy_version 45820 (0.0008) [2023-10-07 21:35:40,571][67838] Updated weights for policy 0, policy_version 45762 (0.0008) [2023-10-07 21:35:40,948][67838] Updated weights for policy 0, policy_version 45772 (0.0009) [2023-10-07 21:35:41,321][67838] Updated weights for policy 0, policy_version 45782 (0.0008) [2023-10-07 21:35:41,691][67838] Updated weights for policy 0, policy_version 45792 (0.0008) [2023-10-07 21:35:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 93814784. Throughput: 0: 1685.0, 1: 1657.8. Samples: 23456890. Policy #0 lag: (min: 6.0, avg: 16.7, max: 38.0) [2023-10-07 21:35:42,477][66916] Avg episode reward: [(0, '44.120'), (1, '50.400')] [2023-10-07 21:35:43,392][67871] Updated weights for policy 1, policy_version 45830 (0.0010) [2023-10-07 21:35:43,749][67871] Updated weights for policy 1, policy_version 45840 (0.0008) [2023-10-07 21:35:44,119][67871] Updated weights for policy 1, policy_version 45850 (0.0009) [2023-10-07 21:35:45,689][67838] Updated weights for policy 0, policy_version 45802 (0.0007) [2023-10-07 21:35:46,062][67838] Updated weights for policy 0, policy_version 45812 (0.0008) [2023-10-07 21:35:46,444][67838] Updated weights for policy 0, policy_version 45822 (0.0008) [2023-10-07 21:35:47,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 93880320. Throughput: 0: 1667.5, 1: 1667.1. Samples: 23476574. Policy #0 lag: (min: 23.0, avg: 23.8, max: 42.0) [2023-10-07 21:35:47,478][66916] Avg episode reward: [(0, '42.080'), (1, '47.570')] [2023-10-07 21:35:48,407][67871] Updated weights for policy 1, policy_version 45860 (0.0009) [2023-10-07 21:35:48,781][67871] Updated weights for policy 1, policy_version 45870 (0.0007) [2023-10-07 21:35:49,147][67871] Updated weights for policy 1, policy_version 45880 (0.0009) [2023-10-07 21:35:50,574][67838] Updated weights for policy 0, policy_version 45832 (0.0009) [2023-10-07 21:35:50,940][67838] Updated weights for policy 0, policy_version 45842 (0.0008) [2023-10-07 21:35:51,307][67838] Updated weights for policy 0, policy_version 45852 (0.0007) [2023-10-07 21:35:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 93945856. Throughput: 0: 1670.8, 1: 1669.4. Samples: 23496792. Policy #0 lag: (min: 23.0, avg: 23.8, max: 42.0) [2023-10-07 21:35:52,477][66916] Avg episode reward: [(0, '44.020'), (1, '47.940')] [2023-10-07 21:35:53,389][67871] Updated weights for policy 1, policy_version 45890 (0.0007) [2023-10-07 21:35:53,755][67871] Updated weights for policy 1, policy_version 45900 (0.0008) [2023-10-07 21:35:54,123][67871] Updated weights for policy 1, policy_version 45910 (0.0008) [2023-10-07 21:35:54,490][67871] Updated weights for policy 1, policy_version 45920 (0.0007) [2023-10-07 21:35:55,340][67838] Updated weights for policy 0, policy_version 45862 (0.0009) [2023-10-07 21:35:55,714][67838] Updated weights for policy 0, policy_version 45872 (0.0008) [2023-10-07 21:35:56,078][67838] Updated weights for policy 0, policy_version 45882 (0.0007) [2023-10-07 21:35:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 94011392. Throughput: 0: 1683.6, 1: 1663.8. Samples: 23507190. Policy #0 lag: (min: 23.0, avg: 23.8, max: 42.0) [2023-10-07 21:35:57,477][66916] Avg episode reward: [(0, '39.620'), (1, '50.070')] [2023-10-07 21:35:58,496][67871] Updated weights for policy 1, policy_version 45930 (0.0012) [2023-10-07 21:35:58,866][67871] Updated weights for policy 1, policy_version 45940 (0.0010) [2023-10-07 21:35:59,229][67871] Updated weights for policy 1, policy_version 45950 (0.0007) [2023-10-07 21:36:00,159][67838] Updated weights for policy 0, policy_version 45892 (0.0008) [2023-10-07 21:36:00,526][67838] Updated weights for policy 0, policy_version 45902 (0.0008) [2023-10-07 21:36:00,898][67838] Updated weights for policy 0, policy_version 45912 (0.0007) [2023-10-07 21:36:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 94076928. Throughput: 0: 1661.6, 1: 1675.5. Samples: 23527052. Policy #0 lag: (min: 23.0, avg: 23.8, max: 42.0) [2023-10-07 21:36:02,478][66916] Avg episode reward: [(0, '38.880'), (1, '54.030')] [2023-10-07 21:36:03,121][67871] Updated weights for policy 1, policy_version 45960 (0.0011) [2023-10-07 21:36:03,491][67871] Updated weights for policy 1, policy_version 45970 (0.0009) [2023-10-07 21:36:03,858][67871] Updated weights for policy 1, policy_version 45980 (0.0008) [2023-10-07 21:36:04,989][67838] Updated weights for policy 0, policy_version 45922 (0.0008) [2023-10-07 21:36:05,359][67838] Updated weights for policy 0, policy_version 45932 (0.0009) [2023-10-07 21:36:05,735][67838] Updated weights for policy 0, policy_version 45942 (0.0009) [2023-10-07 21:36:06,103][67838] Updated weights for policy 0, policy_version 45952 (0.0008) [2023-10-07 21:36:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 94142464. Throughput: 0: 1675.5, 1: 1674.2. Samples: 23546976. Policy #0 lag: (min: 23.0, avg: 23.8, max: 42.0) [2023-10-07 21:36:07,478][66916] Avg episode reward: [(0, '41.370'), (1, '53.640')] [2023-10-07 21:36:08,268][67871] Updated weights for policy 1, policy_version 45990 (0.0009) [2023-10-07 21:36:08,660][67871] Updated weights for policy 1, policy_version 46000 (0.0007) [2023-10-07 21:36:09,022][67871] Updated weights for policy 1, policy_version 46010 (0.0009) [2023-10-07 21:36:10,178][67838] Updated weights for policy 0, policy_version 45962 (0.0011) [2023-10-07 21:36:10,551][67838] Updated weights for policy 0, policy_version 45972 (0.0009) [2023-10-07 21:36:10,916][67838] Updated weights for policy 0, policy_version 45982 (0.0008) [2023-10-07 21:36:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 94208000. Throughput: 0: 1667.4, 1: 1668.6. Samples: 23556828. Policy #0 lag: (min: 23.0, avg: 23.8, max: 42.0) [2023-10-07 21:36:12,478][66916] Avg episode reward: [(0, '41.640'), (1, '56.620')] [2023-10-07 21:36:12,771][67871] Updated weights for policy 1, policy_version 46020 (0.0010) [2023-10-07 21:36:13,133][67871] Updated weights for policy 1, policy_version 46030 (0.0007) [2023-10-07 21:36:13,501][67871] Updated weights for policy 1, policy_version 46040 (0.0007) [2023-10-07 21:36:14,965][67838] Updated weights for policy 0, policy_version 45992 (0.0010) [2023-10-07 21:36:15,328][67838] Updated weights for policy 0, policy_version 46002 (0.0008) [2023-10-07 21:36:15,708][67838] Updated weights for policy 0, policy_version 46012 (0.0007) [2023-10-07 21:36:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 94273536. Throughput: 0: 1658.4, 1: 1676.0. Samples: 23576588. Policy #0 lag: (min: 23.0, avg: 23.8, max: 42.0) [2023-10-07 21:36:17,478][66916] Avg episode reward: [(0, '44.800'), (1, '52.850')] [2023-10-07 21:36:17,578][67871] Updated weights for policy 1, policy_version 46050 (0.0008) [2023-10-07 21:36:17,941][67871] Updated weights for policy 1, policy_version 46060 (0.0007) [2023-10-07 21:36:18,305][67871] Updated weights for policy 1, policy_version 46070 (0.0008) [2023-10-07 21:36:18,673][67871] Updated weights for policy 1, policy_version 46080 (0.0008) [2023-10-07 21:36:20,090][67838] Updated weights for policy 0, policy_version 46022 (0.0007) [2023-10-07 21:36:20,465][67838] Updated weights for policy 0, policy_version 46032 (0.0008) [2023-10-07 21:36:20,832][67838] Updated weights for policy 0, policy_version 46042 (0.0009) [2023-10-07 21:36:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 94339072. Throughput: 0: 1667.2, 1: 1673.8. Samples: 23596764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-07 21:36:22,477][66916] Avg episode reward: [(0, '42.060'), (1, '50.720')] [2023-10-07 21:36:22,758][67871] Updated weights for policy 1, policy_version 46090 (0.0009) [2023-10-07 21:36:23,114][67871] Updated weights for policy 1, policy_version 46100 (0.0008) [2023-10-07 21:36:23,489][67871] Updated weights for policy 1, policy_version 46110 (0.0009) [2023-10-07 21:36:24,731][67838] Updated weights for policy 0, policy_version 46052 (0.0010) [2023-10-07 21:36:25,110][67838] Updated weights for policy 0, policy_version 46062 (0.0009) [2023-10-07 21:36:25,466][67838] Updated weights for policy 0, policy_version 46072 (0.0008) [2023-10-07 21:36:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 94404608. Throughput: 0: 1657.8, 1: 1671.6. Samples: 23606716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-07 21:36:27,477][66916] Avg episode reward: [(0, '43.990'), (1, '49.090')] [2023-10-07 21:36:27,569][67871] Updated weights for policy 1, policy_version 46120 (0.0009) [2023-10-07 21:36:27,937][67871] Updated weights for policy 1, policy_version 46130 (0.0007) [2023-10-07 21:36:28,308][67871] Updated weights for policy 1, policy_version 46140 (0.0007) [2023-10-07 21:36:29,684][67838] Updated weights for policy 0, policy_version 46082 (0.0010) [2023-10-07 21:36:30,059][67838] Updated weights for policy 0, policy_version 46092 (0.0011) [2023-10-07 21:36:30,435][67838] Updated weights for policy 0, policy_version 46102 (0.0011) [2023-10-07 21:36:30,800][67838] Updated weights for policy 0, policy_version 46112 (0.0008) [2023-10-07 21:36:32,452][67871] Updated weights for policy 1, policy_version 46150 (0.0008) [2023-10-07 21:36:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 94470144. Throughput: 0: 1655.9, 1: 1679.5. Samples: 23626664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-07 21:36:32,477][66916] Avg episode reward: [(0, '43.760'), (1, '48.120')] [2023-10-07 21:36:32,829][67871] Updated weights for policy 1, policy_version 46160 (0.0011) [2023-10-07 21:36:33,194][67871] Updated weights for policy 1, policy_version 46170 (0.0008) [2023-10-07 21:36:34,795][67838] Updated weights for policy 0, policy_version 46122 (0.0010) [2023-10-07 21:36:35,164][67838] Updated weights for policy 0, policy_version 46132 (0.0009) [2023-10-07 21:36:35,540][67838] Updated weights for policy 0, policy_version 46142 (0.0010) [2023-10-07 21:36:37,324][67871] Updated weights for policy 1, policy_version 46180 (0.0009) [2023-10-07 21:36:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 94535680. Throughput: 0: 1670.2, 1: 1678.1. Samples: 23647464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-07 21:36:37,477][66916] Avg episode reward: [(0, '44.340'), (1, '49.340')] [2023-10-07 21:36:37,690][67871] Updated weights for policy 1, policy_version 46190 (0.0007) [2023-10-07 21:36:38,065][67871] Updated weights for policy 1, policy_version 46200 (0.0008) [2023-10-07 21:36:39,458][67838] Updated weights for policy 0, policy_version 46152 (0.0008) [2023-10-07 21:36:39,835][67838] Updated weights for policy 0, policy_version 46162 (0.0007) [2023-10-07 21:36:40,202][67838] Updated weights for policy 0, policy_version 46172 (0.0007) [2023-10-07 21:36:42,227][67871] Updated weights for policy 1, policy_version 46210 (0.0008) [2023-10-07 21:36:42,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 94601216. Throughput: 0: 1648.3, 1: 1677.5. Samples: 23656850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-07 21:36:42,478][66916] Avg episode reward: [(0, '44.370'), (1, '52.070')] [2023-10-07 21:36:42,597][67871] Updated weights for policy 1, policy_version 46220 (0.0007) [2023-10-07 21:36:42,958][67871] Updated weights for policy 1, policy_version 46230 (0.0007) [2023-10-07 21:36:43,323][67871] Updated weights for policy 1, policy_version 46240 (0.0007) [2023-10-07 21:36:44,474][67838] Updated weights for policy 0, policy_version 46182 (0.0010) [2023-10-07 21:36:44,852][67838] Updated weights for policy 0, policy_version 46192 (0.0008) [2023-10-07 21:36:45,216][67838] Updated weights for policy 0, policy_version 46202 (0.0008) [2023-10-07 21:36:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 94666752. Throughput: 0: 1659.8, 1: 1669.5. Samples: 23676872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-07 21:36:47,478][66916] Avg episode reward: [(0, '46.710'), (1, '53.690')] [2023-10-07 21:36:47,516][67871] Updated weights for policy 1, policy_version 46250 (0.0008) [2023-10-07 21:36:47,882][67871] Updated weights for policy 1, policy_version 46260 (0.0007) [2023-10-07 21:36:48,250][67871] Updated weights for policy 1, policy_version 46270 (0.0007) [2023-10-07 21:36:49,345][67838] Updated weights for policy 0, policy_version 46212 (0.0011) [2023-10-07 21:36:49,723][67838] Updated weights for policy 0, policy_version 46222 (0.0008) [2023-10-07 21:36:50,094][67838] Updated weights for policy 0, policy_version 46232 (0.0007) [2023-10-07 21:36:52,121][67871] Updated weights for policy 1, policy_version 46280 (0.0007) [2023-10-07 21:36:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 94732288. Throughput: 0: 1665.6, 1: 1675.5. Samples: 23697324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-07 21:36:52,477][66916] Avg episode reward: [(0, '44.340'), (1, '51.370')] [2023-10-07 21:36:52,491][67871] Updated weights for policy 1, policy_version 46290 (0.0008) [2023-10-07 21:36:52,853][67871] Updated weights for policy 1, policy_version 46300 (0.0007) [2023-10-07 21:36:54,314][67838] Updated weights for policy 0, policy_version 46242 (0.0010) [2023-10-07 21:36:54,691][67838] Updated weights for policy 0, policy_version 46252 (0.0010) [2023-10-07 21:36:55,069][67838] Updated weights for policy 0, policy_version 46262 (0.0009) [2023-10-07 21:36:55,445][67838] Updated weights for policy 0, policy_version 46272 (0.0008) [2023-10-07 21:36:56,915][67871] Updated weights for policy 1, policy_version 46310 (0.0008) [2023-10-07 21:36:57,292][67871] Updated weights for policy 1, policy_version 46320 (0.0009) [2023-10-07 21:36:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 94797824. Throughput: 0: 1650.4, 1: 1685.5. Samples: 23706944. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:36:57,478][66916] Avg episode reward: [(0, '46.410'), (1, '51.370')] [2023-10-07 21:36:57,659][67871] Updated weights for policy 1, policy_version 46330 (0.0009) [2023-10-07 21:36:59,672][67838] Updated weights for policy 0, policy_version 46282 (0.0009) [2023-10-07 21:37:00,051][67838] Updated weights for policy 0, policy_version 46292 (0.0009) [2023-10-07 21:37:00,425][67838] Updated weights for policy 0, policy_version 46302 (0.0007) [2023-10-07 21:37:01,751][67871] Updated weights for policy 1, policy_version 46340 (0.0008) [2023-10-07 21:37:02,122][67871] Updated weights for policy 1, policy_version 46350 (0.0009) [2023-10-07 21:37:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 94863360. Throughput: 0: 1657.7, 1: 1675.7. Samples: 23726590. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:37:02,477][66916] Avg episode reward: [(0, '42.840'), (1, '53.190')] [2023-10-07 21:37:02,489][67871] Updated weights for policy 1, policy_version 46360 (0.0008) [2023-10-07 21:37:04,688][67838] Updated weights for policy 0, policy_version 46312 (0.0007) [2023-10-07 21:37:05,070][67838] Updated weights for policy 0, policy_version 46322 (0.0009) [2023-10-07 21:37:05,454][67838] Updated weights for policy 0, policy_version 46332 (0.0009) [2023-10-07 21:37:06,572][67871] Updated weights for policy 1, policy_version 46370 (0.0009) [2023-10-07 21:37:06,944][67871] Updated weights for policy 1, policy_version 46380 (0.0009) [2023-10-07 21:37:07,313][67871] Updated weights for policy 1, policy_version 46390 (0.0009) [2023-10-07 21:37:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 94928896. Throughput: 0: 1663.0, 1: 1664.1. Samples: 23746484. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:37:07,477][66916] Avg episode reward: [(0, '37.780'), (1, '52.490')] [2023-10-07 21:37:07,484][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000046336_47448064.pth... [2023-10-07 21:37:07,517][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000044768_45842432.pth [2023-10-07 21:37:07,521][67511] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p0/milestones/checkpoint_000046336_47448064.pth [2023-10-07 21:37:07,678][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000046400_47513600.pth... [2023-10-07 21:37:07,678][67871] Updated weights for policy 1, policy_version 46400 (0.0009) [2023-10-07 21:37:07,706][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000044832_45907968.pth [2023-10-07 21:37:07,709][67676] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p1/milestones/checkpoint_000046400_47513600.pth [2023-10-07 21:37:09,548][67838] Updated weights for policy 0, policy_version 46342 (0.0008) [2023-10-07 21:37:09,922][67838] Updated weights for policy 0, policy_version 46352 (0.0007) [2023-10-07 21:37:10,296][67838] Updated weights for policy 0, policy_version 46362 (0.0008) [2023-10-07 21:37:12,064][67871] Updated weights for policy 1, policy_version 46410 (0.0009) [2023-10-07 21:37:12,425][67871] Updated weights for policy 1, policy_version 46420 (0.0009) [2023-10-07 21:37:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 94994432. Throughput: 0: 1651.1, 1: 1670.7. Samples: 23756198. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:37:12,478][66916] Avg episode reward: [(0, '39.010'), (1, '53.240')] [2023-10-07 21:37:12,796][67871] Updated weights for policy 1, policy_version 46430 (0.0012) [2023-10-07 21:37:14,152][67838] Updated weights for policy 0, policy_version 46372 (0.0009) [2023-10-07 21:37:14,521][67838] Updated weights for policy 0, policy_version 46382 (0.0007) [2023-10-07 21:37:14,901][67838] Updated weights for policy 0, policy_version 46392 (0.0008) [2023-10-07 21:37:16,894][67871] Updated weights for policy 1, policy_version 46440 (0.0010) [2023-10-07 21:37:17,261][67871] Updated weights for policy 1, policy_version 46450 (0.0009) [2023-10-07 21:37:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 95059968. Throughput: 0: 1660.5, 1: 1666.1. Samples: 23776364. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:37:17,477][66916] Avg episode reward: [(0, '36.790'), (1, '53.330')] [2023-10-07 21:37:17,624][67871] Updated weights for policy 1, policy_version 46460 (0.0010) [2023-10-07 21:37:19,092][67838] Updated weights for policy 0, policy_version 46402 (0.0009) [2023-10-07 21:37:19,461][67838] Updated weights for policy 0, policy_version 46412 (0.0009) [2023-10-07 21:37:19,839][67838] Updated weights for policy 0, policy_version 46422 (0.0008) [2023-10-07 21:37:20,212][67838] Updated weights for policy 0, policy_version 46432 (0.0007) [2023-10-07 21:37:21,796][67871] Updated weights for policy 1, policy_version 46470 (0.0008) [2023-10-07 21:37:22,169][67871] Updated weights for policy 1, policy_version 46480 (0.0008) [2023-10-07 21:37:22,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 95125504. Throughput: 0: 1653.7, 1: 1656.1. Samples: 23796408. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:37:22,478][66916] Avg episode reward: [(0, '38.990'), (1, '53.020')] [2023-10-07 21:37:22,537][67871] Updated weights for policy 1, policy_version 46490 (0.0010) [2023-10-07 21:37:24,481][67838] Updated weights for policy 0, policy_version 46442 (0.0008) [2023-10-07 21:37:24,847][67838] Updated weights for policy 0, policy_version 46452 (0.0008) [2023-10-07 21:37:25,222][67838] Updated weights for policy 0, policy_version 46462 (0.0010) [2023-10-07 21:37:26,469][67871] Updated weights for policy 1, policy_version 46500 (0.0007) [2023-10-07 21:37:26,834][67871] Updated weights for policy 1, policy_version 46510 (0.0008) [2023-10-07 21:37:27,201][67871] Updated weights for policy 1, policy_version 46520 (0.0010) [2023-10-07 21:37:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 95191040. Throughput: 0: 1650.7, 1: 1665.9. Samples: 23806098. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-07 21:37:27,477][66916] Avg episode reward: [(0, '41.130'), (1, '50.500')] [2023-10-07 21:37:29,241][67838] Updated weights for policy 0, policy_version 46472 (0.0010) [2023-10-07 21:37:29,613][67838] Updated weights for policy 0, policy_version 46482 (0.0007) [2023-10-07 21:37:29,987][67838] Updated weights for policy 0, policy_version 46492 (0.0007) [2023-10-07 21:37:31,518][67871] Updated weights for policy 1, policy_version 46530 (0.0009) [2023-10-07 21:37:31,883][67871] Updated weights for policy 1, policy_version 46540 (0.0009) [2023-10-07 21:37:32,249][67871] Updated weights for policy 1, policy_version 46550 (0.0010) [2023-10-07 21:37:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 95256576. Throughput: 0: 1654.2, 1: 1666.1. Samples: 23826286. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-07 21:37:32,477][66916] Avg episode reward: [(0, '42.680'), (1, '51.690')] [2023-10-07 21:37:32,620][67871] Updated weights for policy 1, policy_version 46560 (0.0010) [2023-10-07 21:37:33,910][67838] Updated weights for policy 0, policy_version 46502 (0.0008) [2023-10-07 21:37:34,280][67838] Updated weights for policy 0, policy_version 46512 (0.0010) [2023-10-07 21:37:34,657][67838] Updated weights for policy 0, policy_version 46522 (0.0007) [2023-10-07 21:37:36,737][67871] Updated weights for policy 1, policy_version 46570 (0.0007) [2023-10-07 21:37:37,106][67871] Updated weights for policy 1, policy_version 46580 (0.0007) [2023-10-07 21:37:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 95322112. Throughput: 0: 1660.0, 1: 1654.0. Samples: 23846452. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-07 21:37:37,477][66916] Avg episode reward: [(0, '42.440'), (1, '51.720')] [2023-10-07 21:37:37,479][67871] Updated weights for policy 1, policy_version 46590 (0.0007) [2023-10-07 21:37:38,775][67838] Updated weights for policy 0, policy_version 46532 (0.0008) [2023-10-07 21:37:39,152][67838] Updated weights for policy 0, policy_version 46542 (0.0007) [2023-10-07 21:37:39,524][67838] Updated weights for policy 0, policy_version 46552 (0.0009) [2023-10-07 21:37:41,467][67871] Updated weights for policy 1, policy_version 46600 (0.0009) [2023-10-07 21:37:41,819][67871] Updated weights for policy 1, policy_version 46610 (0.0011) [2023-10-07 21:37:42,201][67871] Updated weights for policy 1, policy_version 46620 (0.0011) [2023-10-07 21:37:42,476][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 95420416. Throughput: 0: 1654.4, 1: 1660.7. Samples: 23856122. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-07 21:37:42,477][66916] Avg episode reward: [(0, '40.610'), (1, '52.670')] [2023-10-07 21:37:43,664][67838] Updated weights for policy 0, policy_version 46562 (0.0007) [2023-10-07 21:37:44,036][67838] Updated weights for policy 0, policy_version 46572 (0.0007) [2023-10-07 21:37:44,414][67838] Updated weights for policy 0, policy_version 46582 (0.0007) [2023-10-07 21:37:44,789][67838] Updated weights for policy 0, policy_version 46592 (0.0007) [2023-10-07 21:37:46,458][67871] Updated weights for policy 1, policy_version 46630 (0.0010) [2023-10-07 21:37:46,842][67871] Updated weights for policy 1, policy_version 46640 (0.0009) [2023-10-07 21:37:47,204][67871] Updated weights for policy 1, policy_version 46650 (0.0008) [2023-10-07 21:37:47,477][66916] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 95485952. Throughput: 0: 1668.3, 1: 1663.8. Samples: 23876532. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-07 21:37:47,478][66916] Avg episode reward: [(0, '40.170'), (1, '52.540')] [2023-10-07 21:37:48,841][67838] Updated weights for policy 0, policy_version 46602 (0.0007) [2023-10-07 21:37:49,217][67838] Updated weights for policy 0, policy_version 46612 (0.0007) [2023-10-07 21:37:49,595][67838] Updated weights for policy 0, policy_version 46622 (0.0008) [2023-10-07 21:37:51,230][67871] Updated weights for policy 1, policy_version 46660 (0.0008) [2023-10-07 21:37:51,588][67871] Updated weights for policy 1, policy_version 46670 (0.0008) [2023-10-07 21:37:51,959][67871] Updated weights for policy 1, policy_version 46680 (0.0007) [2023-10-07 21:37:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 95551488. Throughput: 0: 1674.4, 1: 1654.7. Samples: 23896296. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-07 21:37:52,477][66916] Avg episode reward: [(0, '37.370'), (1, '54.900')] [2023-10-07 21:37:53,577][67838] Updated weights for policy 0, policy_version 46632 (0.0009) [2023-10-07 21:37:53,952][67838] Updated weights for policy 0, policy_version 46642 (0.0007) [2023-10-07 21:37:54,328][67838] Updated weights for policy 0, policy_version 46652 (0.0008) [2023-10-07 21:37:56,021][67871] Updated weights for policy 1, policy_version 46690 (0.0007) [2023-10-07 21:37:56,392][67871] Updated weights for policy 1, policy_version 46700 (0.0008) [2023-10-07 21:37:56,752][67871] Updated weights for policy 1, policy_version 46710 (0.0007) [2023-10-07 21:37:57,123][67871] Updated weights for policy 1, policy_version 46720 (0.0009) [2023-10-07 21:37:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 95617024. Throughput: 0: 1664.3, 1: 1668.5. Samples: 23906172. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-07 21:37:57,477][66916] Avg episode reward: [(0, '38.140'), (1, '56.410')] [2023-10-07 21:37:58,420][67838] Updated weights for policy 0, policy_version 46662 (0.0008) [2023-10-07 21:37:58,782][67838] Updated weights for policy 0, policy_version 46672 (0.0008) [2023-10-07 21:37:59,154][67838] Updated weights for policy 0, policy_version 46682 (0.0007) [2023-10-07 21:38:01,390][67871] Updated weights for policy 1, policy_version 46730 (0.0007) [2023-10-07 21:38:01,745][67871] Updated weights for policy 1, policy_version 46740 (0.0008) [2023-10-07 21:38:02,106][67871] Updated weights for policy 1, policy_version 46750 (0.0007) [2023-10-07 21:38:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 95682560. Throughput: 0: 1680.7, 1: 1661.9. Samples: 23926780. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-07 21:38:02,477][66916] Avg episode reward: [(0, '42.430'), (1, '55.440')] [2023-10-07 21:38:03,141][67838] Updated weights for policy 0, policy_version 46692 (0.0009) [2023-10-07 21:38:03,513][67838] Updated weights for policy 0, policy_version 46702 (0.0008) [2023-10-07 21:38:03,883][67838] Updated weights for policy 0, policy_version 46712 (0.0008) [2023-10-07 21:38:06,210][67871] Updated weights for policy 1, policy_version 46760 (0.0009) [2023-10-07 21:38:06,579][67871] Updated weights for policy 1, policy_version 46770 (0.0007) [2023-10-07 21:38:06,943][67871] Updated weights for policy 1, policy_version 46780 (0.0008) [2023-10-07 21:38:07,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 95748096. Throughput: 0: 1689.6, 1: 1647.3. Samples: 23946570. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-07 21:38:07,477][66916] Avg episode reward: [(0, '41.990'), (1, '56.650')] [2023-10-07 21:38:07,724][67838] Updated weights for policy 0, policy_version 46722 (0.0008) [2023-10-07 21:38:08,101][67838] Updated weights for policy 0, policy_version 46732 (0.0009) [2023-10-07 21:38:08,479][67838] Updated weights for policy 0, policy_version 46742 (0.0009) [2023-10-07 21:38:08,840][67838] Updated weights for policy 0, policy_version 46752 (0.0008) [2023-10-07 21:38:11,099][67871] Updated weights for policy 1, policy_version 46790 (0.0007) [2023-10-07 21:38:11,459][67871] Updated weights for policy 1, policy_version 46800 (0.0007) [2023-10-07 21:38:11,828][67871] Updated weights for policy 1, policy_version 46810 (0.0008) [2023-10-07 21:38:12,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 95813632. Throughput: 0: 1678.3, 1: 1660.9. Samples: 23956360. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-07 21:38:12,478][66916] Avg episode reward: [(0, '44.740'), (1, '55.620')] [2023-10-07 21:38:13,104][67838] Updated weights for policy 0, policy_version 46762 (0.0009) [2023-10-07 21:38:13,484][67838] Updated weights for policy 0, policy_version 46772 (0.0007) [2023-10-07 21:38:13,848][67838] Updated weights for policy 0, policy_version 46782 (0.0009) [2023-10-07 21:38:16,080][67871] Updated weights for policy 1, policy_version 46820 (0.0008) [2023-10-07 21:38:16,445][67871] Updated weights for policy 1, policy_version 46830 (0.0007) [2023-10-07 21:38:16,819][67871] Updated weights for policy 1, policy_version 46840 (0.0009) [2023-10-07 21:38:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 95879168. Throughput: 0: 1683.7, 1: 1657.0. Samples: 23976618. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-07 21:38:17,477][66916] Avg episode reward: [(0, '43.960'), (1, '52.080')] [2023-10-07 21:38:17,919][67838] Updated weights for policy 0, policy_version 46792 (0.0009) [2023-10-07 21:38:18,289][67838] Updated weights for policy 0, policy_version 46802 (0.0009) [2023-10-07 21:38:18,663][67838] Updated weights for policy 0, policy_version 46812 (0.0008) [2023-10-07 21:38:20,808][67871] Updated weights for policy 1, policy_version 46850 (0.0007) [2023-10-07 21:38:21,183][67871] Updated weights for policy 1, policy_version 46860 (0.0007) [2023-10-07 21:38:21,562][67871] Updated weights for policy 1, policy_version 46870 (0.0009) [2023-10-07 21:38:21,924][67871] Updated weights for policy 1, policy_version 46880 (0.0008) [2023-10-07 21:38:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 95944704. Throughput: 0: 1681.1, 1: 1645.5. Samples: 23996152. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-07 21:38:22,478][66916] Avg episode reward: [(0, '43.790'), (1, '51.340')] [2023-10-07 21:38:22,643][67838] Updated weights for policy 0, policy_version 46822 (0.0008) [2023-10-07 21:38:23,014][67838] Updated weights for policy 0, policy_version 46832 (0.0007) [2023-10-07 21:38:23,376][67838] Updated weights for policy 0, policy_version 46842 (0.0007) [2023-10-07 21:38:25,884][67871] Updated weights for policy 1, policy_version 46890 (0.0009) [2023-10-07 21:38:26,250][67871] Updated weights for policy 1, policy_version 46900 (0.0008) [2023-10-07 21:38:26,613][67871] Updated weights for policy 1, policy_version 46910 (0.0007) [2023-10-07 21:38:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 96010240. Throughput: 0: 1680.8, 1: 1658.9. Samples: 24006408. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-07 21:38:27,477][66916] Avg episode reward: [(0, '41.320'), (1, '46.310')] [2023-10-07 21:38:27,562][67838] Updated weights for policy 0, policy_version 46852 (0.0007) [2023-10-07 21:38:27,936][67838] Updated weights for policy 0, policy_version 46862 (0.0007) [2023-10-07 21:38:28,307][67838] Updated weights for policy 0, policy_version 46872 (0.0007) [2023-10-07 21:38:30,831][67871] Updated weights for policy 1, policy_version 46920 (0.0009) [2023-10-07 21:38:31,195][67871] Updated weights for policy 1, policy_version 46930 (0.0007) [2023-10-07 21:38:31,563][67871] Updated weights for policy 1, policy_version 46940 (0.0007) [2023-10-07 21:38:32,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 96075776. Throughput: 0: 1680.1, 1: 1652.4. Samples: 24026496. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-07 21:38:32,478][66916] Avg episode reward: [(0, '42.190'), (1, '48.760')] [2023-10-07 21:38:32,592][67838] Updated weights for policy 0, policy_version 46882 (0.0007) [2023-10-07 21:38:32,969][67838] Updated weights for policy 0, policy_version 46892 (0.0008) [2023-10-07 21:38:33,344][67838] Updated weights for policy 0, policy_version 46902 (0.0008) [2023-10-07 21:38:33,720][67838] Updated weights for policy 0, policy_version 46912 (0.0007) [2023-10-07 21:38:35,763][67871] Updated weights for policy 1, policy_version 46950 (0.0010) [2023-10-07 21:38:36,136][67871] Updated weights for policy 1, policy_version 46960 (0.0010) [2023-10-07 21:38:36,499][67871] Updated weights for policy 1, policy_version 46970 (0.0009) [2023-10-07 21:38:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 96141312. Throughput: 0: 1675.1, 1: 1650.7. Samples: 24045954. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 21:38:37,477][66916] Avg episode reward: [(0, '43.190'), (1, '48.810')] [2023-10-07 21:38:37,878][67838] Updated weights for policy 0, policy_version 46922 (0.0008) [2023-10-07 21:38:38,250][67838] Updated weights for policy 0, policy_version 46932 (0.0009) [2023-10-07 21:38:38,621][67838] Updated weights for policy 0, policy_version 46942 (0.0007) [2023-10-07 21:38:40,671][67871] Updated weights for policy 1, policy_version 46980 (0.0008) [2023-10-07 21:38:41,040][67871] Updated weights for policy 1, policy_version 46990 (0.0008) [2023-10-07 21:38:41,402][67871] Updated weights for policy 1, policy_version 47000 (0.0007) [2023-10-07 21:38:42,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 96206848. Throughput: 0: 1673.8, 1: 1660.5. Samples: 24056214. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 21:38:42,477][66916] Avg episode reward: [(0, '42.360'), (1, '47.490')] [2023-10-07 21:38:42,769][67838] Updated weights for policy 0, policy_version 46952 (0.0009) [2023-10-07 21:38:43,143][67838] Updated weights for policy 0, policy_version 46962 (0.0008) [2023-10-07 21:38:43,500][67838] Updated weights for policy 0, policy_version 46972 (0.0009) [2023-10-07 21:38:45,369][67871] Updated weights for policy 1, policy_version 47010 (0.0007) [2023-10-07 21:38:45,730][67871] Updated weights for policy 1, policy_version 47020 (0.0010) [2023-10-07 21:38:46,103][67871] Updated weights for policy 1, policy_version 47030 (0.0008) [2023-10-07 21:38:46,471][67871] Updated weights for policy 1, policy_version 47040 (0.0010) [2023-10-07 21:38:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 96272384. Throughput: 0: 1662.4, 1: 1664.0. Samples: 24076468. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 21:38:47,477][66916] Avg episode reward: [(0, '42.990'), (1, '48.270')] [2023-10-07 21:38:47,602][67838] Updated weights for policy 0, policy_version 46982 (0.0011) [2023-10-07 21:38:47,969][67838] Updated weights for policy 0, policy_version 46992 (0.0010) [2023-10-07 21:38:48,336][67838] Updated weights for policy 0, policy_version 47002 (0.0008) [2023-10-07 21:38:50,632][67871] Updated weights for policy 1, policy_version 47050 (0.0008) [2023-10-07 21:38:51,001][67871] Updated weights for policy 1, policy_version 47060 (0.0009) [2023-10-07 21:38:51,369][67871] Updated weights for policy 1, policy_version 47070 (0.0008) [2023-10-07 21:38:52,385][67838] Updated weights for policy 0, policy_version 47012 (0.0009) [2023-10-07 21:38:52,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 96337920. Throughput: 0: 1655.3, 1: 1665.1. Samples: 24095986. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 21:38:52,477][66916] Avg episode reward: [(0, '41.020'), (1, '51.990')] [2023-10-07 21:38:52,758][67838] Updated weights for policy 0, policy_version 47022 (0.0007) [2023-10-07 21:38:53,136][67838] Updated weights for policy 0, policy_version 47032 (0.0007) [2023-10-07 21:38:55,580][67871] Updated weights for policy 1, policy_version 47080 (0.0008) [2023-10-07 21:38:55,940][67871] Updated weights for policy 1, policy_version 47090 (0.0008) [2023-10-07 21:38:56,298][67871] Updated weights for policy 1, policy_version 47100 (0.0007) [2023-10-07 21:38:57,291][67838] Updated weights for policy 0, policy_version 47042 (0.0009) [2023-10-07 21:38:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 96403456. Throughput: 0: 1660.4, 1: 1673.4. Samples: 24106382. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 21:38:57,477][66916] Avg episode reward: [(0, '42.070'), (1, '53.170')] [2023-10-07 21:38:57,670][67838] Updated weights for policy 0, policy_version 47052 (0.0008) [2023-10-07 21:38:58,041][67838] Updated weights for policy 0, policy_version 47062 (0.0010) [2023-10-07 21:38:58,418][67838] Updated weights for policy 0, policy_version 47072 (0.0010) [2023-10-07 21:39:00,427][67871] Updated weights for policy 1, policy_version 47110 (0.0009) [2023-10-07 21:39:00,794][67871] Updated weights for policy 1, policy_version 47120 (0.0008) [2023-10-07 21:39:01,158][67871] Updated weights for policy 1, policy_version 47130 (0.0007) [2023-10-07 21:39:02,455][67838] Updated weights for policy 0, policy_version 47082 (0.0009) [2023-10-07 21:39:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 96468992. Throughput: 0: 1666.2, 1: 1661.6. Samples: 24126368. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 21:39:02,477][66916] Avg episode reward: [(0, '46.440'), (1, '51.820')] [2023-10-07 21:39:02,827][67838] Updated weights for policy 0, policy_version 47092 (0.0008) [2023-10-07 21:39:03,203][67838] Updated weights for policy 0, policy_version 47102 (0.0009) [2023-10-07 21:39:05,268][67871] Updated weights for policy 1, policy_version 47140 (0.0009) [2023-10-07 21:39:05,637][67871] Updated weights for policy 1, policy_version 47150 (0.0010) [2023-10-07 21:39:06,006][67871] Updated weights for policy 1, policy_version 47160 (0.0007) [2023-10-07 21:39:07,315][67838] Updated weights for policy 0, policy_version 47112 (0.0008) [2023-10-07 21:39:07,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 96534528. Throughput: 0: 1664.3, 1: 1670.8. Samples: 24146230. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 21:39:07,477][66916] Avg episode reward: [(0, '44.780'), (1, '51.340')] [2023-10-07 21:39:07,486][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000047168_48300032.pth... [2023-10-07 21:39:07,520][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000045600_46694400.pth [2023-10-07 21:39:07,690][67838] Updated weights for policy 0, policy_version 47122 (0.0008) [2023-10-07 21:39:08,066][67838] Updated weights for policy 0, policy_version 47132 (0.0007) [2023-10-07 21:39:08,211][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000047136_48267264.pth... [2023-10-07 21:39:08,239][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000045568_46661632.pth [2023-10-07 21:39:10,056][67871] Updated weights for policy 1, policy_version 47170 (0.0007) [2023-10-07 21:39:10,430][67871] Updated weights for policy 1, policy_version 47180 (0.0010) [2023-10-07 21:39:10,799][67871] Updated weights for policy 1, policy_version 47190 (0.0008) [2023-10-07 21:39:11,169][67871] Updated weights for policy 1, policy_version 47200 (0.0010) [2023-10-07 21:39:12,303][67838] Updated weights for policy 0, policy_version 47142 (0.0010) [2023-10-07 21:39:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 96600064. Throughput: 0: 1663.2, 1: 1667.0. Samples: 24156268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:39:12,477][66916] Avg episode reward: [(0, '48.200'), (1, '46.210')] [2023-10-07 21:39:12,675][67838] Updated weights for policy 0, policy_version 47152 (0.0007) [2023-10-07 21:39:13,048][67838] Updated weights for policy 0, policy_version 47162 (0.0007) [2023-10-07 21:39:15,394][67871] Updated weights for policy 1, policy_version 47210 (0.0008) [2023-10-07 21:39:15,762][67871] Updated weights for policy 1, policy_version 47220 (0.0007) [2023-10-07 21:39:16,138][67871] Updated weights for policy 1, policy_version 47230 (0.0007) [2023-10-07 21:39:16,930][67838] Updated weights for policy 0, policy_version 47172 (0.0007) [2023-10-07 21:39:17,299][67838] Updated weights for policy 0, policy_version 47182 (0.0008) [2023-10-07 21:39:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 96665600. Throughput: 0: 1665.9, 1: 1655.1. Samples: 24175940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:39:17,478][66916] Avg episode reward: [(0, '47.870'), (1, '45.700')] [2023-10-07 21:39:17,674][67838] Updated weights for policy 0, policy_version 47192 (0.0009) [2023-10-07 21:39:20,392][67871] Updated weights for policy 1, policy_version 47240 (0.0009) [2023-10-07 21:39:20,774][67871] Updated weights for policy 1, policy_version 47250 (0.0008) [2023-10-07 21:39:21,144][67871] Updated weights for policy 1, policy_version 47260 (0.0008) [2023-10-07 21:39:21,748][67838] Updated weights for policy 0, policy_version 47202 (0.0008) [2023-10-07 21:39:22,121][67838] Updated weights for policy 0, policy_version 47212 (0.0008) [2023-10-07 21:39:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 96731136. Throughput: 0: 1662.4, 1: 1660.7. Samples: 24195496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:39:22,477][66916] Avg episode reward: [(0, '49.220'), (1, '45.940')] [2023-10-07 21:39:22,495][67838] Updated weights for policy 0, policy_version 47222 (0.0009) [2023-10-07 21:39:22,868][67838] Updated weights for policy 0, policy_version 47232 (0.0009) [2023-10-07 21:39:25,076][67871] Updated weights for policy 1, policy_version 47270 (0.0009) [2023-10-07 21:39:25,444][67871] Updated weights for policy 1, policy_version 47280 (0.0008) [2023-10-07 21:39:25,804][67871] Updated weights for policy 1, policy_version 47290 (0.0008) [2023-10-07 21:39:27,084][67838] Updated weights for policy 0, policy_version 47242 (0.0007) [2023-10-07 21:39:27,471][67838] Updated weights for policy 0, policy_version 47252 (0.0007) [2023-10-07 21:39:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 96796672. Throughput: 0: 1670.1, 1: 1662.5. Samples: 24206180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:39:27,477][66916] Avg episode reward: [(0, '45.880'), (1, '46.840')] [2023-10-07 21:39:27,844][67838] Updated weights for policy 0, policy_version 47262 (0.0007) [2023-10-07 21:39:29,907][67871] Updated weights for policy 1, policy_version 47300 (0.0007) [2023-10-07 21:39:30,270][67871] Updated weights for policy 1, policy_version 47310 (0.0008) [2023-10-07 21:39:30,634][67871] Updated weights for policy 1, policy_version 47320 (0.0008) [2023-10-07 21:39:31,759][67838] Updated weights for policy 0, policy_version 47272 (0.0007) [2023-10-07 21:39:32,124][67838] Updated weights for policy 0, policy_version 47282 (0.0007) [2023-10-07 21:39:32,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 96862208. Throughput: 0: 1679.8, 1: 1640.4. Samples: 24225878. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:39:32,478][66916] Avg episode reward: [(0, '43.830'), (1, '50.450')] [2023-10-07 21:39:32,503][67838] Updated weights for policy 0, policy_version 47292 (0.0007) [2023-10-07 21:39:34,636][67871] Updated weights for policy 1, policy_version 47330 (0.0009) [2023-10-07 21:39:35,005][67871] Updated weights for policy 1, policy_version 47340 (0.0007) [2023-10-07 21:39:35,367][67871] Updated weights for policy 1, policy_version 47350 (0.0008) [2023-10-07 21:39:35,730][67871] Updated weights for policy 1, policy_version 47360 (0.0008) [2023-10-07 21:39:36,655][67838] Updated weights for policy 0, policy_version 47302 (0.0008) [2023-10-07 21:39:37,028][67838] Updated weights for policy 0, policy_version 47312 (0.0008) [2023-10-07 21:39:37,406][67838] Updated weights for policy 0, policy_version 47322 (0.0007) [2023-10-07 21:39:37,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 96927744. Throughput: 0: 1661.5, 1: 1664.4. Samples: 24245652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:39:37,477][66916] Avg episode reward: [(0, '47.130'), (1, '51.580')] [2023-10-07 21:39:39,951][67871] Updated weights for policy 1, policy_version 47370 (0.0009) [2023-10-07 21:39:40,312][67871] Updated weights for policy 1, policy_version 47380 (0.0009) [2023-10-07 21:39:40,678][67871] Updated weights for policy 1, policy_version 47390 (0.0009) [2023-10-07 21:39:41,356][67838] Updated weights for policy 0, policy_version 47332 (0.0009) [2023-10-07 21:39:41,729][67838] Updated weights for policy 0, policy_version 47342 (0.0007) [2023-10-07 21:39:42,098][67838] Updated weights for policy 0, policy_version 47352 (0.0010) [2023-10-07 21:39:42,476][66916] Fps is (10 sec: 16384.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 97026048. Throughput: 0: 1676.2, 1: 1653.6. Samples: 24256222. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:39:42,477][66916] Avg episode reward: [(0, '41.690'), (1, '54.760')] [2023-10-07 21:39:44,795][67871] Updated weights for policy 1, policy_version 47400 (0.0008) [2023-10-07 21:39:45,167][67871] Updated weights for policy 1, policy_version 47410 (0.0009) [2023-10-07 21:39:45,531][67871] Updated weights for policy 1, policy_version 47420 (0.0007) [2023-10-07 21:39:46,257][67838] Updated weights for policy 0, policy_version 47362 (0.0009) [2023-10-07 21:39:46,635][67838] Updated weights for policy 0, policy_version 47372 (0.0008) [2023-10-07 21:39:47,008][67838] Updated weights for policy 0, policy_version 47382 (0.0012) [2023-10-07 21:39:47,377][67838] Updated weights for policy 0, policy_version 47392 (0.0010) [2023-10-07 21:39:47,476][66916] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 97091584. Throughput: 0: 1673.8, 1: 1648.1. Samples: 24275856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:39:47,477][66916] Avg episode reward: [(0, '43.500'), (1, '50.180')] [2023-10-07 21:39:49,608][67871] Updated weights for policy 1, policy_version 47430 (0.0010) [2023-10-07 21:39:49,983][67871] Updated weights for policy 1, policy_version 47440 (0.0008) [2023-10-07 21:39:50,349][67871] Updated weights for policy 1, policy_version 47450 (0.0010) [2023-10-07 21:39:51,381][67838] Updated weights for policy 0, policy_version 47402 (0.0008) [2023-10-07 21:39:51,755][67838] Updated weights for policy 0, policy_version 47412 (0.0007) [2023-10-07 21:39:52,132][67838] Updated weights for policy 0, policy_version 47422 (0.0008) [2023-10-07 21:39:52,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 97157120. Throughput: 0: 1650.7, 1: 1663.6. Samples: 24295374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:39:52,477][66916] Avg episode reward: [(0, '40.240'), (1, '51.590')] [2023-10-07 21:39:54,549][67871] Updated weights for policy 1, policy_version 47460 (0.0010) [2023-10-07 21:39:54,920][67871] Updated weights for policy 1, policy_version 47470 (0.0007) [2023-10-07 21:39:55,277][67871] Updated weights for policy 1, policy_version 47480 (0.0008) [2023-10-07 21:39:56,224][67838] Updated weights for policy 0, policy_version 47432 (0.0007) [2023-10-07 21:39:56,597][67838] Updated weights for policy 0, policy_version 47442 (0.0008) [2023-10-07 21:39:56,981][67838] Updated weights for policy 0, policy_version 47452 (0.0009) [2023-10-07 21:39:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 97222656. Throughput: 0: 1676.4, 1: 1658.7. Samples: 24306350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:39:57,478][66916] Avg episode reward: [(0, '42.700'), (1, '48.110')] [2023-10-07 21:39:59,333][67871] Updated weights for policy 1, policy_version 47490 (0.0010) [2023-10-07 21:39:59,713][67871] Updated weights for policy 1, policy_version 47500 (0.0010) [2023-10-07 21:40:00,086][67871] Updated weights for policy 1, policy_version 47510 (0.0009) [2023-10-07 21:40:00,451][67871] Updated weights for policy 1, policy_version 47520 (0.0007) [2023-10-07 21:40:01,243][67838] Updated weights for policy 0, policy_version 47462 (0.0008) [2023-10-07 21:40:01,627][67838] Updated weights for policy 0, policy_version 47472 (0.0008) [2023-10-07 21:40:01,998][67838] Updated weights for policy 0, policy_version 47482 (0.0008) [2023-10-07 21:40:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 97288192. Throughput: 0: 1670.7, 1: 1664.4. Samples: 24326018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:40:02,477][66916] Avg episode reward: [(0, '43.020'), (1, '49.550')] [2023-10-07 21:40:04,341][67871] Updated weights for policy 1, policy_version 47530 (0.0009) [2023-10-07 21:40:04,721][67871] Updated weights for policy 1, policy_version 47540 (0.0007) [2023-10-07 21:40:05,091][67871] Updated weights for policy 1, policy_version 47550 (0.0008) [2023-10-07 21:40:06,033][67838] Updated weights for policy 0, policy_version 47492 (0.0009) [2023-10-07 21:40:06,404][67838] Updated weights for policy 0, policy_version 47502 (0.0008) [2023-10-07 21:40:06,771][67838] Updated weights for policy 0, policy_version 47512 (0.0011) [2023-10-07 21:40:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 97353728. Throughput: 0: 1649.7, 1: 1684.2. Samples: 24345522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:40:07,478][66916] Avg episode reward: [(0, '44.520'), (1, '49.360')] [2023-10-07 21:40:09,522][67871] Updated weights for policy 1, policy_version 47560 (0.0007) [2023-10-07 21:40:09,904][67871] Updated weights for policy 1, policy_version 47570 (0.0009) [2023-10-07 21:40:10,276][67871] Updated weights for policy 1, policy_version 47580 (0.0009) [2023-10-07 21:40:11,092][67838] Updated weights for policy 0, policy_version 47522 (0.0011) [2023-10-07 21:40:11,462][67838] Updated weights for policy 0, policy_version 47532 (0.0008) [2023-10-07 21:40:11,838][67838] Updated weights for policy 0, policy_version 47542 (0.0008) [2023-10-07 21:40:12,206][67838] Updated weights for policy 0, policy_version 47552 (0.0009) [2023-10-07 21:40:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 97419264. Throughput: 0: 1671.0, 1: 1665.8. Samples: 24356336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:40:12,477][66916] Avg episode reward: [(0, '47.650'), (1, '52.590')] [2023-10-07 21:40:14,430][67871] Updated weights for policy 1, policy_version 47590 (0.0009) [2023-10-07 21:40:14,805][67871] Updated weights for policy 1, policy_version 47600 (0.0008) [2023-10-07 21:40:15,161][67871] Updated weights for policy 1, policy_version 47610 (0.0010) [2023-10-07 21:40:16,165][67838] Updated weights for policy 0, policy_version 47562 (0.0008) [2023-10-07 21:40:16,544][67838] Updated weights for policy 0, policy_version 47572 (0.0007) [2023-10-07 21:40:16,916][67838] Updated weights for policy 0, policy_version 47582 (0.0008) [2023-10-07 21:40:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 97484800. Throughput: 0: 1660.5, 1: 1669.4. Samples: 24375720. Policy #0 lag: (min: 25.0, avg: 45.7, max: 57.0) [2023-10-07 21:40:17,477][66916] Avg episode reward: [(0, '46.450'), (1, '50.160')] [2023-10-07 21:40:19,014][67871] Updated weights for policy 1, policy_version 47620 (0.0009) [2023-10-07 21:40:19,373][67871] Updated weights for policy 1, policy_version 47630 (0.0008) [2023-10-07 21:40:19,749][67871] Updated weights for policy 1, policy_version 47640 (0.0008) [2023-10-07 21:40:20,944][67838] Updated weights for policy 0, policy_version 47592 (0.0009) [2023-10-07 21:40:21,323][67838] Updated weights for policy 0, policy_version 47602 (0.0008) [2023-10-07 21:40:21,705][67838] Updated weights for policy 0, policy_version 47612 (0.0009) [2023-10-07 21:40:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 97550336. Throughput: 0: 1658.3, 1: 1665.9. Samples: 24395240. Policy #0 lag: (min: 25.0, avg: 45.7, max: 57.0) [2023-10-07 21:40:22,478][66916] Avg episode reward: [(0, '47.470'), (1, '50.190')] [2023-10-07 21:40:23,910][67871] Updated weights for policy 1, policy_version 47650 (0.0007) [2023-10-07 21:40:24,283][67871] Updated weights for policy 1, policy_version 47660 (0.0009) [2023-10-07 21:40:24,655][67871] Updated weights for policy 1, policy_version 47670 (0.0008) [2023-10-07 21:40:25,021][67871] Updated weights for policy 1, policy_version 47680 (0.0008) [2023-10-07 21:40:25,906][67838] Updated weights for policy 0, policy_version 47622 (0.0010) [2023-10-07 21:40:26,290][67838] Updated weights for policy 0, policy_version 47632 (0.0011) [2023-10-07 21:40:26,650][67838] Updated weights for policy 0, policy_version 47642 (0.0011) [2023-10-07 21:40:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 97615872. Throughput: 0: 1671.6, 1: 1654.4. Samples: 24405888. Policy #0 lag: (min: 25.0, avg: 45.7, max: 57.0) [2023-10-07 21:40:27,477][66916] Avg episode reward: [(0, '46.150'), (1, '48.720')] [2023-10-07 21:40:29,226][67871] Updated weights for policy 1, policy_version 47690 (0.0008) [2023-10-07 21:40:29,594][67871] Updated weights for policy 1, policy_version 47700 (0.0008) [2023-10-07 21:40:29,955][67871] Updated weights for policy 1, policy_version 47710 (0.0010) [2023-10-07 21:40:30,683][67838] Updated weights for policy 0, policy_version 47652 (0.0009) [2023-10-07 21:40:31,045][67838] Updated weights for policy 0, policy_version 47662 (0.0010) [2023-10-07 21:40:31,416][67838] Updated weights for policy 0, policy_version 47672 (0.0010) [2023-10-07 21:40:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.3). Total num frames: 97681408. Throughput: 0: 1659.5, 1: 1669.1. Samples: 24425642. Policy #0 lag: (min: 25.0, avg: 45.7, max: 57.0) [2023-10-07 21:40:32,477][66916] Avg episode reward: [(0, '50.690'), (1, '46.950')] [2023-10-07 21:40:33,908][67871] Updated weights for policy 1, policy_version 47720 (0.0008) [2023-10-07 21:40:34,274][67871] Updated weights for policy 1, policy_version 47730 (0.0007) [2023-10-07 21:40:34,646][67871] Updated weights for policy 1, policy_version 47740 (0.0009) [2023-10-07 21:40:35,466][67838] Updated weights for policy 0, policy_version 47682 (0.0010) [2023-10-07 21:40:35,833][67838] Updated weights for policy 0, policy_version 47692 (0.0009) [2023-10-07 21:40:36,209][67838] Updated weights for policy 0, policy_version 47702 (0.0007) [2023-10-07 21:40:36,585][67838] Updated weights for policy 0, policy_version 47712 (0.0007) [2023-10-07 21:40:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 97746944. Throughput: 0: 1667.0, 1: 1667.6. Samples: 24445428. Policy #0 lag: (min: 25.0, avg: 45.7, max: 57.0) [2023-10-07 21:40:37,477][66916] Avg episode reward: [(0, '51.400'), (1, '47.370')] [2023-10-07 21:40:37,487][67511] Saving new best policy, reward=51.400! [2023-10-07 21:40:38,864][67871] Updated weights for policy 1, policy_version 47750 (0.0008) [2023-10-07 21:40:39,229][67871] Updated weights for policy 1, policy_version 47760 (0.0007) [2023-10-07 21:40:39,593][67871] Updated weights for policy 1, policy_version 47770 (0.0009) [2023-10-07 21:40:40,639][67838] Updated weights for policy 0, policy_version 47722 (0.0009) [2023-10-07 21:40:41,002][67838] Updated weights for policy 0, policy_version 47732 (0.0008) [2023-10-07 21:40:41,382][67838] Updated weights for policy 0, policy_version 47742 (0.0011) [2023-10-07 21:40:42,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 97812480. Throughput: 0: 1668.1, 1: 1648.5. Samples: 24455596. Policy #0 lag: (min: 25.0, avg: 45.7, max: 57.0) [2023-10-07 21:40:42,477][66916] Avg episode reward: [(0, '53.920'), (1, '46.810')] [2023-10-07 21:40:42,478][67511] Saving new best policy, reward=53.920! [2023-10-07 21:40:43,682][67871] Updated weights for policy 1, policy_version 47780 (0.0008) [2023-10-07 21:40:44,054][67871] Updated weights for policy 1, policy_version 47790 (0.0007) [2023-10-07 21:40:44,416][67871] Updated weights for policy 1, policy_version 47800 (0.0008) [2023-10-07 21:40:45,421][67838] Updated weights for policy 0, policy_version 47752 (0.0007) [2023-10-07 21:40:45,793][67838] Updated weights for policy 0, policy_version 47762 (0.0007) [2023-10-07 21:40:46,180][67838] Updated weights for policy 0, policy_version 47772 (0.0008) [2023-10-07 21:40:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 97878016. Throughput: 0: 1653.2, 1: 1664.4. Samples: 24475308. Policy #0 lag: (min: 25.0, avg: 45.7, max: 57.0) [2023-10-07 21:40:47,477][66916] Avg episode reward: [(0, '54.440'), (1, '48.430')] [2023-10-07 21:40:47,477][67511] Saving new best policy, reward=54.440! [2023-10-07 21:40:48,396][67871] Updated weights for policy 1, policy_version 47810 (0.0007) [2023-10-07 21:40:48,770][67871] Updated weights for policy 1, policy_version 47820 (0.0010) [2023-10-07 21:40:49,131][67871] Updated weights for policy 1, policy_version 47830 (0.0009) [2023-10-07 21:40:49,503][67871] Updated weights for policy 1, policy_version 47840 (0.0009) [2023-10-07 21:40:50,270][67838] Updated weights for policy 0, policy_version 47782 (0.0011) [2023-10-07 21:40:50,639][67838] Updated weights for policy 0, policy_version 47792 (0.0011) [2023-10-07 21:40:51,011][67838] Updated weights for policy 0, policy_version 47802 (0.0011) [2023-10-07 21:40:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 97943552. Throughput: 0: 1669.0, 1: 1664.5. Samples: 24495530. Policy #0 lag: (min: 24.0, avg: 43.9, max: 56.0) [2023-10-07 21:40:52,477][66916] Avg episode reward: [(0, '56.330'), (1, '46.610')] [2023-10-07 21:40:52,487][67511] Saving new best policy, reward=56.330! [2023-10-07 21:40:53,557][67871] Updated weights for policy 1, policy_version 47850 (0.0008) [2023-10-07 21:40:53,921][67871] Updated weights for policy 1, policy_version 47860 (0.0007) [2023-10-07 21:40:54,291][67871] Updated weights for policy 1, policy_version 47870 (0.0008) [2023-10-07 21:40:55,185][67838] Updated weights for policy 0, policy_version 47812 (0.0010) [2023-10-07 21:40:55,569][67838] Updated weights for policy 0, policy_version 47822 (0.0010) [2023-10-07 21:40:55,933][67838] Updated weights for policy 0, policy_version 47832 (0.0010) [2023-10-07 21:40:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 98009088. Throughput: 0: 1671.5, 1: 1654.1. Samples: 24505986. Policy #0 lag: (min: 24.0, avg: 43.9, max: 56.0) [2023-10-07 21:40:57,478][66916] Avg episode reward: [(0, '54.130'), (1, '48.630')] [2023-10-07 21:40:58,361][67871] Updated weights for policy 1, policy_version 47880 (0.0007) [2023-10-07 21:40:58,737][67871] Updated weights for policy 1, policy_version 47890 (0.0007) [2023-10-07 21:40:59,103][67871] Updated weights for policy 1, policy_version 47900 (0.0008) [2023-10-07 21:41:00,130][67838] Updated weights for policy 0, policy_version 47842 (0.0009) [2023-10-07 21:41:00,494][67838] Updated weights for policy 0, policy_version 47852 (0.0007) [2023-10-07 21:41:00,867][67838] Updated weights for policy 0, policy_version 47862 (0.0007) [2023-10-07 21:41:01,240][67838] Updated weights for policy 0, policy_version 47872 (0.0009) [2023-10-07 21:41:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 98074624. Throughput: 0: 1653.1, 1: 1678.3. Samples: 24525630. Policy #0 lag: (min: 24.0, avg: 43.9, max: 56.0) [2023-10-07 21:41:02,477][66916] Avg episode reward: [(0, '52.490'), (1, '47.670')] [2023-10-07 21:41:03,044][67871] Updated weights for policy 1, policy_version 47910 (0.0008) [2023-10-07 21:41:03,409][67871] Updated weights for policy 1, policy_version 47920 (0.0010) [2023-10-07 21:41:03,781][67871] Updated weights for policy 1, policy_version 47930 (0.0011) [2023-10-07 21:41:05,315][67838] Updated weights for policy 0, policy_version 47882 (0.0009) [2023-10-07 21:41:05,690][67838] Updated weights for policy 0, policy_version 47892 (0.0010) [2023-10-07 21:41:06,066][67838] Updated weights for policy 0, policy_version 47902 (0.0008) [2023-10-07 21:41:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 98140160. Throughput: 0: 1669.4, 1: 1677.0. Samples: 24545828. Policy #0 lag: (min: 24.0, avg: 43.9, max: 56.0) [2023-10-07 21:41:07,477][66916] Avg episode reward: [(0, '49.210'), (1, '49.020')] [2023-10-07 21:41:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000047936_49086464.pth... [2023-10-07 21:41:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000047904_49053696.pth... [2023-10-07 21:41:07,524][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000046400_47513600.pth [2023-10-07 21:41:07,525][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000046336_47448064.pth [2023-10-07 21:41:07,908][67871] Updated weights for policy 1, policy_version 47940 (0.0010) [2023-10-07 21:41:08,268][67871] Updated weights for policy 1, policy_version 47950 (0.0009) [2023-10-07 21:41:08,637][67871] Updated weights for policy 1, policy_version 47960 (0.0009) [2023-10-07 21:41:10,320][67838] Updated weights for policy 0, policy_version 47912 (0.0010) [2023-10-07 21:41:10,699][67838] Updated weights for policy 0, policy_version 47922 (0.0007) [2023-10-07 21:41:11,066][67838] Updated weights for policy 0, policy_version 47932 (0.0007) [2023-10-07 21:41:12,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 98205696. Throughput: 0: 1664.0, 1: 1670.4. Samples: 24555940. Policy #0 lag: (min: 24.0, avg: 43.9, max: 56.0) [2023-10-07 21:41:12,478][66916] Avg episode reward: [(0, '48.930'), (1, '51.110')] [2023-10-07 21:41:12,845][67871] Updated weights for policy 1, policy_version 47970 (0.0008) [2023-10-07 21:41:13,221][67871] Updated weights for policy 1, policy_version 47980 (0.0008) [2023-10-07 21:41:13,597][67871] Updated weights for policy 1, policy_version 47990 (0.0007) [2023-10-07 21:41:13,954][67871] Updated weights for policy 1, policy_version 48000 (0.0009) [2023-10-07 21:41:15,117][67838] Updated weights for policy 0, policy_version 47942 (0.0007) [2023-10-07 21:41:15,494][67838] Updated weights for policy 0, policy_version 47952 (0.0007) [2023-10-07 21:41:15,862][67838] Updated weights for policy 0, policy_version 47962 (0.0007) [2023-10-07 21:41:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 98271232. Throughput: 0: 1647.1, 1: 1674.2. Samples: 24575102. Policy #0 lag: (min: 24.0, avg: 43.9, max: 56.0) [2023-10-07 21:41:17,477][66916] Avg episode reward: [(0, '50.690'), (1, '52.110')] [2023-10-07 21:41:18,172][67871] Updated weights for policy 1, policy_version 48010 (0.0010) [2023-10-07 21:41:18,533][67871] Updated weights for policy 1, policy_version 48020 (0.0009) [2023-10-07 21:41:18,905][67871] Updated weights for policy 1, policy_version 48030 (0.0010) [2023-10-07 21:41:19,657][67838] Updated weights for policy 0, policy_version 47972 (0.0009) [2023-10-07 21:41:20,024][67838] Updated weights for policy 0, policy_version 47982 (0.0010) [2023-10-07 21:41:20,400][67838] Updated weights for policy 0, policy_version 47992 (0.0009) [2023-10-07 21:41:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 98336768. Throughput: 0: 1666.0, 1: 1673.8. Samples: 24595722. Policy #0 lag: (min: 24.0, avg: 43.9, max: 56.0) [2023-10-07 21:41:22,478][66916] Avg episode reward: [(0, '48.780'), (1, '52.760')] [2023-10-07 21:41:23,026][67871] Updated weights for policy 1, policy_version 48040 (0.0009) [2023-10-07 21:41:23,401][67871] Updated weights for policy 1, policy_version 48050 (0.0009) [2023-10-07 21:41:23,768][67871] Updated weights for policy 1, policy_version 48060 (0.0008) [2023-10-07 21:41:24,672][67838] Updated weights for policy 0, policy_version 48002 (0.0008) [2023-10-07 21:41:25,054][67838] Updated weights for policy 0, policy_version 48012 (0.0008) [2023-10-07 21:41:25,424][67838] Updated weights for policy 0, policy_version 48022 (0.0008) [2023-10-07 21:41:25,797][67838] Updated weights for policy 0, policy_version 48032 (0.0010) [2023-10-07 21:41:27,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 98402304. Throughput: 0: 1664.3, 1: 1673.9. Samples: 24605812. Policy #0 lag: (min: 11.0, avg: 11.4, max: 26.0) [2023-10-07 21:41:27,478][66916] Avg episode reward: [(0, '49.470'), (1, '51.880')] [2023-10-07 21:41:27,834][67871] Updated weights for policy 1, policy_version 48070 (0.0007) [2023-10-07 21:41:28,189][67871] Updated weights for policy 1, policy_version 48080 (0.0007) [2023-10-07 21:41:28,562][67871] Updated weights for policy 1, policy_version 48090 (0.0009) [2023-10-07 21:41:29,791][67838] Updated weights for policy 0, policy_version 48042 (0.0011) [2023-10-07 21:41:30,159][67838] Updated weights for policy 0, policy_version 48052 (0.0007) [2023-10-07 21:41:30,530][67838] Updated weights for policy 0, policy_version 48062 (0.0009) [2023-10-07 21:41:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 98467840. Throughput: 0: 1664.4, 1: 1673.3. Samples: 24625502. Policy #0 lag: (min: 11.0, avg: 11.4, max: 26.0) [2023-10-07 21:41:32,477][66916] Avg episode reward: [(0, '53.230'), (1, '49.090')] [2023-10-07 21:41:32,738][67871] Updated weights for policy 1, policy_version 48100 (0.0008) [2023-10-07 21:41:33,111][67871] Updated weights for policy 1, policy_version 48110 (0.0008) [2023-10-07 21:41:33,476][67871] Updated weights for policy 1, policy_version 48120 (0.0008) [2023-10-07 21:41:34,653][67838] Updated weights for policy 0, policy_version 48072 (0.0007) [2023-10-07 21:41:35,026][67838] Updated weights for policy 0, policy_version 48082 (0.0008) [2023-10-07 21:41:35,399][67838] Updated weights for policy 0, policy_version 48092 (0.0009) [2023-10-07 21:41:37,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 98533376. Throughput: 0: 1676.3, 1: 1670.2. Samples: 24646124. Policy #0 lag: (min: 11.0, avg: 11.4, max: 26.0) [2023-10-07 21:41:37,477][66916] Avg episode reward: [(0, '49.910'), (1, '50.180')] [2023-10-07 21:41:37,576][67871] Updated weights for policy 1, policy_version 48130 (0.0009) [2023-10-07 21:41:37,952][67871] Updated weights for policy 1, policy_version 48140 (0.0007) [2023-10-07 21:41:38,312][67871] Updated weights for policy 1, policy_version 48150 (0.0008) [2023-10-07 21:41:38,679][67871] Updated weights for policy 1, policy_version 48160 (0.0007) [2023-10-07 21:41:39,564][67838] Updated weights for policy 0, policy_version 48102 (0.0011) [2023-10-07 21:41:39,929][67838] Updated weights for policy 0, policy_version 48112 (0.0008) [2023-10-07 21:41:40,302][67838] Updated weights for policy 0, policy_version 48122 (0.0010) [2023-10-07 21:41:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 98598912. Throughput: 0: 1657.8, 1: 1666.8. Samples: 24655592. Policy #0 lag: (min: 11.0, avg: 11.4, max: 26.0) [2023-10-07 21:41:42,477][66916] Avg episode reward: [(0, '52.200'), (1, '51.290')] [2023-10-07 21:41:42,781][67871] Updated weights for policy 1, policy_version 48170 (0.0008) [2023-10-07 21:41:43,143][67871] Updated weights for policy 1, policy_version 48180 (0.0008) [2023-10-07 21:41:43,506][67871] Updated weights for policy 1, policy_version 48190 (0.0007) [2023-10-07 21:41:44,437][67838] Updated weights for policy 0, policy_version 48132 (0.0009) [2023-10-07 21:41:44,815][67838] Updated weights for policy 0, policy_version 48142 (0.0009) [2023-10-07 21:41:45,182][67838] Updated weights for policy 0, policy_version 48152 (0.0011) [2023-10-07 21:41:47,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 98664448. Throughput: 0: 1663.6, 1: 1662.8. Samples: 24675316. Policy #0 lag: (min: 11.0, avg: 11.4, max: 26.0) [2023-10-07 21:41:47,477][66916] Avg episode reward: [(0, '48.420'), (1, '50.800')] [2023-10-07 21:41:47,582][67871] Updated weights for policy 1, policy_version 48200 (0.0007) [2023-10-07 21:41:47,960][67871] Updated weights for policy 1, policy_version 48210 (0.0009) [2023-10-07 21:41:48,324][67871] Updated weights for policy 1, policy_version 48220 (0.0010) [2023-10-07 21:41:49,376][67838] Updated weights for policy 0, policy_version 48162 (0.0009) [2023-10-07 21:41:49,740][67838] Updated weights for policy 0, policy_version 48172 (0.0007) [2023-10-07 21:41:50,120][67838] Updated weights for policy 0, policy_version 48182 (0.0007) [2023-10-07 21:41:50,497][67838] Updated weights for policy 0, policy_version 48192 (0.0008) [2023-10-07 21:41:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 98729984. Throughput: 0: 1668.8, 1: 1663.3. Samples: 24695776. Policy #0 lag: (min: 11.0, avg: 11.4, max: 26.0) [2023-10-07 21:41:52,478][66916] Avg episode reward: [(0, '50.130'), (1, '52.310')] [2023-10-07 21:41:52,495][67871] Updated weights for policy 1, policy_version 48230 (0.0009) [2023-10-07 21:41:52,863][67871] Updated weights for policy 1, policy_version 48240 (0.0009) [2023-10-07 21:41:53,242][67871] Updated weights for policy 1, policy_version 48250 (0.0008) [2023-10-07 21:41:54,771][67838] Updated weights for policy 0, policy_version 48202 (0.0007) [2023-10-07 21:41:55,140][67838] Updated weights for policy 0, policy_version 48212 (0.0007) [2023-10-07 21:41:55,516][67838] Updated weights for policy 0, policy_version 48222 (0.0008) [2023-10-07 21:41:57,299][67871] Updated weights for policy 1, policy_version 48260 (0.0009) [2023-10-07 21:41:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 98795520. Throughput: 0: 1654.0, 1: 1661.6. Samples: 24705142. Policy #0 lag: (min: 11.0, avg: 11.4, max: 26.0) [2023-10-07 21:41:57,477][66916] Avg episode reward: [(0, '49.440'), (1, '52.440')] [2023-10-07 21:41:57,667][67871] Updated weights for policy 1, policy_version 48270 (0.0007) [2023-10-07 21:41:58,044][67871] Updated weights for policy 1, policy_version 48280 (0.0008) [2023-10-07 21:41:59,679][67838] Updated weights for policy 0, policy_version 48232 (0.0009) [2023-10-07 21:42:00,049][67838] Updated weights for policy 0, policy_version 48242 (0.0007) [2023-10-07 21:42:00,425][67838] Updated weights for policy 0, policy_version 48252 (0.0009) [2023-10-07 21:42:02,025][67871] Updated weights for policy 1, policy_version 48290 (0.0007) [2023-10-07 21:42:02,396][67871] Updated weights for policy 1, policy_version 48300 (0.0008) [2023-10-07 21:42:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 98861056. Throughput: 0: 1664.4, 1: 1666.9. Samples: 24725010. Policy #0 lag: (min: 18.0, avg: 20.7, max: 50.0) [2023-10-07 21:42:02,477][66916] Avg episode reward: [(0, '49.850'), (1, '51.650')] [2023-10-07 21:42:02,761][67871] Updated weights for policy 1, policy_version 48310 (0.0009) [2023-10-07 21:42:03,122][67871] Updated weights for policy 1, policy_version 48320 (0.0010) [2023-10-07 21:42:04,462][67838] Updated weights for policy 0, policy_version 48262 (0.0007) [2023-10-07 21:42:04,828][67838] Updated weights for policy 0, policy_version 48272 (0.0007) [2023-10-07 21:42:05,193][67838] Updated weights for policy 0, policy_version 48282 (0.0008) [2023-10-07 21:42:07,250][67871] Updated weights for policy 1, policy_version 48330 (0.0007) [2023-10-07 21:42:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 98926592. Throughput: 0: 1668.8, 1: 1667.3. Samples: 24745850. Policy #0 lag: (min: 18.0, avg: 20.7, max: 50.0) [2023-10-07 21:42:07,478][66916] Avg episode reward: [(0, '48.870'), (1, '52.040')] [2023-10-07 21:42:07,621][67871] Updated weights for policy 1, policy_version 48340 (0.0009) [2023-10-07 21:42:07,983][67871] Updated weights for policy 1, policy_version 48350 (0.0008) [2023-10-07 21:42:09,169][67838] Updated weights for policy 0, policy_version 48292 (0.0009) [2023-10-07 21:42:09,534][67838] Updated weights for policy 0, policy_version 48302 (0.0010) [2023-10-07 21:42:09,915][67838] Updated weights for policy 0, policy_version 48312 (0.0009) [2023-10-07 21:42:12,410][67871] Updated weights for policy 1, policy_version 48360 (0.0008) [2023-10-07 21:42:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 98992128. Throughput: 0: 1646.6, 1: 1669.4. Samples: 24755030. Policy #0 lag: (min: 18.0, avg: 20.7, max: 50.0) [2023-10-07 21:42:12,477][66916] Avg episode reward: [(0, '50.180'), (1, '52.670')] [2023-10-07 21:42:12,776][67871] Updated weights for policy 1, policy_version 48370 (0.0008) [2023-10-07 21:42:13,159][67871] Updated weights for policy 1, policy_version 48380 (0.0009) [2023-10-07 21:42:13,959][67838] Updated weights for policy 0, policy_version 48322 (0.0011) [2023-10-07 21:42:14,337][67838] Updated weights for policy 0, policy_version 48332 (0.0010) [2023-10-07 21:42:14,720][67838] Updated weights for policy 0, policy_version 48342 (0.0009) [2023-10-07 21:42:15,091][67838] Updated weights for policy 0, policy_version 48352 (0.0009) [2023-10-07 21:42:17,418][67871] Updated weights for policy 1, policy_version 48390 (0.0010) [2023-10-07 21:42:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 99057664. Throughput: 0: 1659.3, 1: 1666.4. Samples: 24775160. Policy #0 lag: (min: 18.0, avg: 20.7, max: 50.0) [2023-10-07 21:42:17,477][66916] Avg episode reward: [(0, '53.880'), (1, '52.760')] [2023-10-07 21:42:17,776][67871] Updated weights for policy 1, policy_version 48400 (0.0009) [2023-10-07 21:42:18,140][67871] Updated weights for policy 1, policy_version 48410 (0.0010) [2023-10-07 21:42:19,125][67838] Updated weights for policy 0, policy_version 48362 (0.0010) [2023-10-07 21:42:19,503][67838] Updated weights for policy 0, policy_version 48372 (0.0009) [2023-10-07 21:42:19,882][67838] Updated weights for policy 0, policy_version 48382 (0.0007) [2023-10-07 21:42:22,117][67871] Updated weights for policy 1, policy_version 48420 (0.0007) [2023-10-07 21:42:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 99123200. Throughput: 0: 1655.6, 1: 1662.8. Samples: 24795456. Policy #0 lag: (min: 18.0, avg: 20.7, max: 50.0) [2023-10-07 21:42:22,477][66916] Avg episode reward: [(0, '53.130'), (1, '53.210')] [2023-10-07 21:42:22,479][67871] Updated weights for policy 1, policy_version 48430 (0.0007) [2023-10-07 21:42:22,845][67871] Updated weights for policy 1, policy_version 48440 (0.0007) [2023-10-07 21:42:24,248][67838] Updated weights for policy 0, policy_version 48392 (0.0008) [2023-10-07 21:42:24,613][67838] Updated weights for policy 0, policy_version 48402 (0.0008) [2023-10-07 21:42:24,993][67838] Updated weights for policy 0, policy_version 48412 (0.0009) [2023-10-07 21:42:26,931][67871] Updated weights for policy 1, policy_version 48450 (0.0007) [2023-10-07 21:42:27,301][67871] Updated weights for policy 1, policy_version 48460 (0.0010) [2023-10-07 21:42:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 99188736. Throughput: 0: 1647.6, 1: 1665.4. Samples: 24804676. Policy #0 lag: (min: 18.0, avg: 20.7, max: 50.0) [2023-10-07 21:42:27,477][66916] Avg episode reward: [(0, '53.900'), (1, '50.570')] [2023-10-07 21:42:27,671][67871] Updated weights for policy 1, policy_version 48470 (0.0009) [2023-10-07 21:42:28,028][67871] Updated weights for policy 1, policy_version 48480 (0.0007) [2023-10-07 21:42:29,048][67838] Updated weights for policy 0, policy_version 48422 (0.0008) [2023-10-07 21:42:29,416][67838] Updated weights for policy 0, policy_version 48432 (0.0007) [2023-10-07 21:42:29,785][67838] Updated weights for policy 0, policy_version 48442 (0.0007) [2023-10-07 21:42:32,174][67871] Updated weights for policy 1, policy_version 48490 (0.0008) [2023-10-07 21:42:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 99254272. Throughput: 0: 1660.6, 1: 1666.8. Samples: 24825050. Policy #0 lag: (min: 18.0, avg: 20.7, max: 50.0) [2023-10-07 21:42:32,477][66916] Avg episode reward: [(0, '54.030'), (1, '49.860')] [2023-10-07 21:42:32,555][67871] Updated weights for policy 1, policy_version 48500 (0.0008) [2023-10-07 21:42:32,917][67871] Updated weights for policy 1, policy_version 48510 (0.0008) [2023-10-07 21:42:33,816][67838] Updated weights for policy 0, policy_version 48452 (0.0008) [2023-10-07 21:42:34,184][67838] Updated weights for policy 0, policy_version 48462 (0.0008) [2023-10-07 21:42:34,562][67838] Updated weights for policy 0, policy_version 48472 (0.0009) [2023-10-07 21:42:36,853][67871] Updated weights for policy 1, policy_version 48520 (0.0009) [2023-10-07 21:42:37,212][67871] Updated weights for policy 1, policy_version 48530 (0.0009) [2023-10-07 21:42:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 99319808. Throughput: 0: 1671.0, 1: 1663.4. Samples: 24845824. Policy #0 lag: (min: 25.0, avg: 29.0, max: 57.0) [2023-10-07 21:42:37,477][66916] Avg episode reward: [(0, '51.340'), (1, '50.160')] [2023-10-07 21:42:37,578][67871] Updated weights for policy 1, policy_version 48540 (0.0009) [2023-10-07 21:42:38,447][67838] Updated weights for policy 0, policy_version 48482 (0.0008) [2023-10-07 21:42:38,829][67838] Updated weights for policy 0, policy_version 48492 (0.0007) [2023-10-07 21:42:39,202][67838] Updated weights for policy 0, policy_version 48502 (0.0007) [2023-10-07 21:42:39,588][67838] Updated weights for policy 0, policy_version 48512 (0.0012) [2023-10-07 21:42:41,783][67871] Updated weights for policy 1, policy_version 48550 (0.0008) [2023-10-07 21:42:42,144][67871] Updated weights for policy 1, policy_version 48560 (0.0008) [2023-10-07 21:42:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 99385344. Throughput: 0: 1662.0, 1: 1671.0. Samples: 24855128. Policy #0 lag: (min: 25.0, avg: 29.0, max: 57.0) [2023-10-07 21:42:42,477][66916] Avg episode reward: [(0, '52.430'), (1, '53.910')] [2023-10-07 21:42:42,509][67871] Updated weights for policy 1, policy_version 48570 (0.0007) [2023-10-07 21:42:43,715][67838] Updated weights for policy 0, policy_version 48522 (0.0007) [2023-10-07 21:42:44,089][67838] Updated weights for policy 0, policy_version 48532 (0.0008) [2023-10-07 21:42:44,467][67838] Updated weights for policy 0, policy_version 48542 (0.0007) [2023-10-07 21:42:46,639][67871] Updated weights for policy 1, policy_version 48580 (0.0009) [2023-10-07 21:42:47,005][67871] Updated weights for policy 1, policy_version 48590 (0.0009) [2023-10-07 21:42:47,373][67871] Updated weights for policy 1, policy_version 48600 (0.0008) [2023-10-07 21:42:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 99450880. Throughput: 0: 1681.8, 1: 1665.2. Samples: 24875622. Policy #0 lag: (min: 25.0, avg: 29.0, max: 57.0) [2023-10-07 21:42:47,477][66916] Avg episode reward: [(0, '49.290'), (1, '56.240')] [2023-10-07 21:42:48,522][67838] Updated weights for policy 0, policy_version 48552 (0.0009) [2023-10-07 21:42:48,894][67838] Updated weights for policy 0, policy_version 48562 (0.0009) [2023-10-07 21:42:49,267][67838] Updated weights for policy 0, policy_version 48572 (0.0009) [2023-10-07 21:42:51,465][67871] Updated weights for policy 1, policy_version 48610 (0.0011) [2023-10-07 21:42:51,833][67871] Updated weights for policy 1, policy_version 48620 (0.0011) [2023-10-07 21:42:52,209][67871] Updated weights for policy 1, policy_version 48630 (0.0009) [2023-10-07 21:42:52,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 99516416. Throughput: 0: 1674.6, 1: 1659.5. Samples: 24895886. Policy #0 lag: (min: 25.0, avg: 29.0, max: 57.0) [2023-10-07 21:42:52,478][66916] Avg episode reward: [(0, '49.000'), (1, '58.270')] [2023-10-07 21:42:52,574][67871] Updated weights for policy 1, policy_version 48640 (0.0007) [2023-10-07 21:42:53,109][67838] Updated weights for policy 0, policy_version 48582 (0.0008) [2023-10-07 21:42:53,482][67838] Updated weights for policy 0, policy_version 48592 (0.0009) [2023-10-07 21:42:53,850][67838] Updated weights for policy 0, policy_version 48602 (0.0009) [2023-10-07 21:42:56,579][67871] Updated weights for policy 1, policy_version 48650 (0.0007) [2023-10-07 21:42:56,935][67871] Updated weights for policy 1, policy_version 48660 (0.0009) [2023-10-07 21:42:57,309][67871] Updated weights for policy 1, policy_version 48670 (0.0010) [2023-10-07 21:42:57,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 99614720. Throughput: 0: 1671.7, 1: 1670.0. Samples: 24905408. Policy #0 lag: (min: 25.0, avg: 29.0, max: 57.0) [2023-10-07 21:42:57,478][66916] Avg episode reward: [(0, '47.790'), (1, '54.920')] [2023-10-07 21:42:58,168][67838] Updated weights for policy 0, policy_version 48612 (0.0009) [2023-10-07 21:42:58,533][67838] Updated weights for policy 0, policy_version 48622 (0.0011) [2023-10-07 21:42:58,908][67838] Updated weights for policy 0, policy_version 48632 (0.0007) [2023-10-07 21:43:01,478][67871] Updated weights for policy 1, policy_version 48680 (0.0008) [2023-10-07 21:43:01,856][67871] Updated weights for policy 1, policy_version 48690 (0.0008) [2023-10-07 21:43:02,224][67871] Updated weights for policy 1, policy_version 48700 (0.0008) [2023-10-07 21:43:02,477][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 99680256. Throughput: 0: 1669.6, 1: 1670.9. Samples: 24925484. Policy #0 lag: (min: 25.0, avg: 29.0, max: 57.0) [2023-10-07 21:43:02,478][66916] Avg episode reward: [(0, '51.330'), (1, '57.070')] [2023-10-07 21:43:03,224][67838] Updated weights for policy 0, policy_version 48642 (0.0010) [2023-10-07 21:43:03,596][67838] Updated weights for policy 0, policy_version 48652 (0.0012) [2023-10-07 21:43:03,976][67838] Updated weights for policy 0, policy_version 48662 (0.0010) [2023-10-07 21:43:04,346][67838] Updated weights for policy 0, policy_version 48672 (0.0009) [2023-10-07 21:43:06,202][67871] Updated weights for policy 1, policy_version 48710 (0.0007) [2023-10-07 21:43:06,565][67871] Updated weights for policy 1, policy_version 48720 (0.0008) [2023-10-07 21:43:06,939][67871] Updated weights for policy 1, policy_version 48730 (0.0010) [2023-10-07 21:43:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 99745792. Throughput: 0: 1670.1, 1: 1659.9. Samples: 24945304. Policy #0 lag: (min: 25.0, avg: 29.0, max: 57.0) [2023-10-07 21:43:07,477][66916] Avg episode reward: [(0, '51.840'), (1, '55.140')] [2023-10-07 21:43:07,486][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000048672_49840128.pth... [2023-10-07 21:43:07,486][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000048736_49905664.pth... [2023-10-07 21:43:07,521][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000047168_48300032.pth [2023-10-07 21:43:07,526][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000047136_48267264.pth [2023-10-07 21:43:08,600][67838] Updated weights for policy 0, policy_version 48682 (0.0008) [2023-10-07 21:43:08,977][67838] Updated weights for policy 0, policy_version 48692 (0.0007) [2023-10-07 21:43:09,347][67838] Updated weights for policy 0, policy_version 48702 (0.0007) [2023-10-07 21:43:11,238][67871] Updated weights for policy 1, policy_version 48740 (0.0008) [2023-10-07 21:43:11,603][67871] Updated weights for policy 1, policy_version 48750 (0.0007) [2023-10-07 21:43:11,959][67871] Updated weights for policy 1, policy_version 48760 (0.0010) [2023-10-07 21:43:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 99811328. Throughput: 0: 1665.0, 1: 1676.6. Samples: 24955048. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-07 21:43:12,478][66916] Avg episode reward: [(0, '52.310'), (1, '52.130')] [2023-10-07 21:43:13,608][67838] Updated weights for policy 0, policy_version 48712 (0.0008) [2023-10-07 21:43:13,980][67838] Updated weights for policy 0, policy_version 48722 (0.0009) [2023-10-07 21:43:14,346][67838] Updated weights for policy 0, policy_version 48732 (0.0009) [2023-10-07 21:43:15,993][67871] Updated weights for policy 1, policy_version 48770 (0.0008) [2023-10-07 21:43:16,368][67871] Updated weights for policy 1, policy_version 48780 (0.0007) [2023-10-07 21:43:16,743][67871] Updated weights for policy 1, policy_version 48790 (0.0007) [2023-10-07 21:43:17,106][67871] Updated weights for policy 1, policy_version 48800 (0.0009) [2023-10-07 21:43:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 99876864. Throughput: 0: 1667.1, 1: 1674.3. Samples: 24975412. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-07 21:43:17,478][66916] Avg episode reward: [(0, '51.820'), (1, '54.390')] [2023-10-07 21:43:18,216][67838] Updated weights for policy 0, policy_version 48742 (0.0007) [2023-10-07 21:43:18,590][67838] Updated weights for policy 0, policy_version 48752 (0.0008) [2023-10-07 21:43:18,971][67838] Updated weights for policy 0, policy_version 48762 (0.0009) [2023-10-07 21:43:21,386][67871] Updated weights for policy 1, policy_version 48810 (0.0008) [2023-10-07 21:43:21,753][67871] Updated weights for policy 1, policy_version 48820 (0.0009) [2023-10-07 21:43:22,122][67871] Updated weights for policy 1, policy_version 48830 (0.0011) [2023-10-07 21:43:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 99942400. Throughput: 0: 1657.9, 1: 1660.4. Samples: 24995146. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-07 21:43:22,477][66916] Avg episode reward: [(0, '52.210'), (1, '54.680')] [2023-10-07 21:43:23,147][67838] Updated weights for policy 0, policy_version 48772 (0.0012) [2023-10-07 21:43:23,516][67838] Updated weights for policy 0, policy_version 48782 (0.0010) [2023-10-07 21:43:23,889][67838] Updated weights for policy 0, policy_version 48792 (0.0009) [2023-10-07 21:43:25,998][67871] Updated weights for policy 1, policy_version 48840 (0.0008) [2023-10-07 21:43:26,369][67871] Updated weights for policy 1, policy_version 48850 (0.0008) [2023-10-07 21:43:26,733][67871] Updated weights for policy 1, policy_version 48860 (0.0008) [2023-10-07 21:43:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 100007936. Throughput: 0: 1654.7, 1: 1673.1. Samples: 25004878. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-07 21:43:27,478][66916] Avg episode reward: [(0, '52.820'), (1, '51.670')] [2023-10-07 21:43:28,136][67838] Updated weights for policy 0, policy_version 48802 (0.0008) [2023-10-07 21:43:28,502][67838] Updated weights for policy 0, policy_version 48812 (0.0010) [2023-10-07 21:43:28,875][67838] Updated weights for policy 0, policy_version 48822 (0.0009) [2023-10-07 21:43:29,250][67838] Updated weights for policy 0, policy_version 48832 (0.0008) [2023-10-07 21:43:30,853][67871] Updated weights for policy 1, policy_version 48870 (0.0009) [2023-10-07 21:43:31,219][67871] Updated weights for policy 1, policy_version 48880 (0.0007) [2023-10-07 21:43:31,578][67871] Updated weights for policy 1, policy_version 48890 (0.0009) [2023-10-07 21:43:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 100073472. Throughput: 0: 1650.8, 1: 1673.7. Samples: 25025224. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-07 21:43:32,477][66916] Avg episode reward: [(0, '51.010'), (1, '48.980')] [2023-10-07 21:43:33,348][67838] Updated weights for policy 0, policy_version 48842 (0.0009) [2023-10-07 21:43:33,715][67838] Updated weights for policy 0, policy_version 48852 (0.0007) [2023-10-07 21:43:34,080][67838] Updated weights for policy 0, policy_version 48862 (0.0007) [2023-10-07 21:43:35,658][67871] Updated weights for policy 1, policy_version 48900 (0.0009) [2023-10-07 21:43:36,013][67871] Updated weights for policy 1, policy_version 48910 (0.0009) [2023-10-07 21:43:36,390][67871] Updated weights for policy 1, policy_version 48920 (0.0007) [2023-10-07 21:43:37,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 100139008. Throughput: 0: 1652.8, 1: 1656.9. Samples: 25044822. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-07 21:43:37,478][66916] Avg episode reward: [(0, '51.020'), (1, '46.190')] [2023-10-07 21:43:38,085][67838] Updated weights for policy 0, policy_version 48872 (0.0008) [2023-10-07 21:43:38,465][67838] Updated weights for policy 0, policy_version 48882 (0.0009) [2023-10-07 21:43:38,831][67838] Updated weights for policy 0, policy_version 48892 (0.0009) [2023-10-07 21:43:40,466][67871] Updated weights for policy 1, policy_version 48930 (0.0008) [2023-10-07 21:43:40,827][67871] Updated weights for policy 1, policy_version 48940 (0.0008) [2023-10-07 21:43:41,193][67871] Updated weights for policy 1, policy_version 48950 (0.0009) [2023-10-07 21:43:41,565][67871] Updated weights for policy 1, policy_version 48960 (0.0009) [2023-10-07 21:43:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 100204544. Throughput: 0: 1650.9, 1: 1673.1. Samples: 25054990. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-07 21:43:42,478][66916] Avg episode reward: [(0, '48.780'), (1, '47.220')] [2023-10-07 21:43:42,787][67838] Updated weights for policy 0, policy_version 48902 (0.0010) [2023-10-07 21:43:43,164][67838] Updated weights for policy 0, policy_version 48912 (0.0007) [2023-10-07 21:43:43,543][67838] Updated weights for policy 0, policy_version 48922 (0.0008) [2023-10-07 21:43:45,797][67871] Updated weights for policy 1, policy_version 48970 (0.0010) [2023-10-07 21:43:46,174][67871] Updated weights for policy 1, policy_version 48980 (0.0010) [2023-10-07 21:43:46,551][67871] Updated weights for policy 1, policy_version 48990 (0.0011) [2023-10-07 21:43:47,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 100270080. Throughput: 0: 1659.3, 1: 1667.3. Samples: 25075180. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:43:47,477][66916] Avg episode reward: [(0, '48.060'), (1, '46.100')] [2023-10-07 21:43:47,664][67838] Updated weights for policy 0, policy_version 48932 (0.0008) [2023-10-07 21:43:48,039][67838] Updated weights for policy 0, policy_version 48942 (0.0010) [2023-10-07 21:43:48,418][67838] Updated weights for policy 0, policy_version 48952 (0.0011) [2023-10-07 21:43:50,681][67871] Updated weights for policy 1, policy_version 49000 (0.0007) [2023-10-07 21:43:51,050][67871] Updated weights for policy 1, policy_version 49010 (0.0009) [2023-10-07 21:43:51,415][67871] Updated weights for policy 1, policy_version 49020 (0.0008) [2023-10-07 21:43:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 100335616. Throughput: 0: 1659.3, 1: 1658.2. Samples: 25094592. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:43:52,477][66916] Avg episode reward: [(0, '49.300'), (1, '50.110')] [2023-10-07 21:43:52,603][67838] Updated weights for policy 0, policy_version 48962 (0.0008) [2023-10-07 21:43:52,982][67838] Updated weights for policy 0, policy_version 48972 (0.0007) [2023-10-07 21:43:53,357][67838] Updated weights for policy 0, policy_version 48982 (0.0008) [2023-10-07 21:43:53,730][67838] Updated weights for policy 0, policy_version 48992 (0.0010) [2023-10-07 21:43:55,489][67871] Updated weights for policy 1, policy_version 49030 (0.0008) [2023-10-07 21:43:55,863][67871] Updated weights for policy 1, policy_version 49040 (0.0010) [2023-10-07 21:43:56,236][67871] Updated weights for policy 1, policy_version 49050 (0.0008) [2023-10-07 21:43:57,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 100401152. Throughput: 0: 1661.9, 1: 1667.7. Samples: 25104880. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:43:57,477][66916] Avg episode reward: [(0, '52.290'), (1, '53.000')] [2023-10-07 21:43:57,742][67838] Updated weights for policy 0, policy_version 49002 (0.0008) [2023-10-07 21:43:58,108][67838] Updated weights for policy 0, policy_version 49012 (0.0009) [2023-10-07 21:43:58,478][67838] Updated weights for policy 0, policy_version 49022 (0.0010) [2023-10-07 21:43:59,983][67871] Updated weights for policy 1, policy_version 49060 (0.0008) [2023-10-07 21:44:00,356][67871] Updated weights for policy 1, policy_version 49070 (0.0010) [2023-10-07 21:44:00,718][67871] Updated weights for policy 1, policy_version 49080 (0.0009) [2023-10-07 21:44:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 100466688. Throughput: 0: 1665.7, 1: 1653.2. Samples: 25124760. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:44:02,477][66916] Avg episode reward: [(0, '49.880'), (1, '54.320')] [2023-10-07 21:44:02,522][67838] Updated weights for policy 0, policy_version 49032 (0.0007) [2023-10-07 21:44:02,889][67838] Updated weights for policy 0, policy_version 49042 (0.0008) [2023-10-07 21:44:03,268][67838] Updated weights for policy 0, policy_version 49052 (0.0007) [2023-10-07 21:44:04,842][67871] Updated weights for policy 1, policy_version 49090 (0.0007) [2023-10-07 21:44:05,208][67871] Updated weights for policy 1, policy_version 49100 (0.0009) [2023-10-07 21:44:05,571][67871] Updated weights for policy 1, policy_version 49110 (0.0008) [2023-10-07 21:44:05,938][67871] Updated weights for policy 1, policy_version 49120 (0.0008) [2023-10-07 21:44:07,442][67838] Updated weights for policy 0, policy_version 49062 (0.0008) [2023-10-07 21:44:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 100532224. Throughput: 0: 1664.8, 1: 1666.1. Samples: 25145038. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:44:07,477][66916] Avg episode reward: [(0, '54.400'), (1, '50.960')] [2023-10-07 21:44:07,819][67838] Updated weights for policy 0, policy_version 49072 (0.0008) [2023-10-07 21:44:08,197][67838] Updated weights for policy 0, policy_version 49082 (0.0010) [2023-10-07 21:44:09,982][67871] Updated weights for policy 1, policy_version 49130 (0.0007) [2023-10-07 21:44:10,353][67871] Updated weights for policy 1, policy_version 49140 (0.0008) [2023-10-07 21:44:10,714][67871] Updated weights for policy 1, policy_version 49150 (0.0008) [2023-10-07 21:44:12,438][67838] Updated weights for policy 0, policy_version 49092 (0.0011) [2023-10-07 21:44:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 100597760. Throughput: 0: 1667.4, 1: 1669.4. Samples: 25155036. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:44:12,477][66916] Avg episode reward: [(0, '56.370'), (1, '49.960')] [2023-10-07 21:44:12,810][67838] Updated weights for policy 0, policy_version 49102 (0.0010) [2023-10-07 21:44:13,199][67838] Updated weights for policy 0, policy_version 49112 (0.0008) [2023-10-07 21:44:13,495][67511] Saving new best policy, reward=56.370! [2023-10-07 21:44:14,882][67871] Updated weights for policy 1, policy_version 49160 (0.0008) [2023-10-07 21:44:15,253][67871] Updated weights for policy 1, policy_version 49170 (0.0008) [2023-10-07 21:44:15,618][67871] Updated weights for policy 1, policy_version 49180 (0.0009) [2023-10-07 21:44:17,264][67838] Updated weights for policy 0, policy_version 49122 (0.0008) [2023-10-07 21:44:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 100663296. Throughput: 0: 1671.8, 1: 1649.7. Samples: 25174692. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 21:44:17,478][66916] Avg episode reward: [(0, '53.010'), (1, '50.180')] [2023-10-07 21:44:17,631][67838] Updated weights for policy 0, policy_version 49132 (0.0010) [2023-10-07 21:44:18,008][67838] Updated weights for policy 0, policy_version 49142 (0.0010) [2023-10-07 21:44:18,379][67838] Updated weights for policy 0, policy_version 49152 (0.0009) [2023-10-07 21:44:19,763][67871] Updated weights for policy 1, policy_version 49190 (0.0009) [2023-10-07 21:44:20,129][67871] Updated weights for policy 1, policy_version 49200 (0.0008) [2023-10-07 21:44:20,507][67871] Updated weights for policy 1, policy_version 49210 (0.0009) [2023-10-07 21:44:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 100728832. Throughput: 0: 1665.6, 1: 1670.5. Samples: 25194950. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 21:44:22,478][66916] Avg episode reward: [(0, '49.470'), (1, '50.110')] [2023-10-07 21:44:22,532][67838] Updated weights for policy 0, policy_version 49162 (0.0010) [2023-10-07 21:44:22,908][67838] Updated weights for policy 0, policy_version 49172 (0.0008) [2023-10-07 21:44:23,295][67838] Updated weights for policy 0, policy_version 49182 (0.0009) [2023-10-07 21:44:24,852][67871] Updated weights for policy 1, policy_version 49220 (0.0008) [2023-10-07 21:44:25,217][67871] Updated weights for policy 1, policy_version 49230 (0.0008) [2023-10-07 21:44:25,580][67871] Updated weights for policy 1, policy_version 49240 (0.0008) [2023-10-07 21:44:27,372][67838] Updated weights for policy 0, policy_version 49192 (0.0008) [2023-10-07 21:44:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 100794368. Throughput: 0: 1664.0, 1: 1667.8. Samples: 25204924. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 21:44:27,477][66916] Avg episode reward: [(0, '52.270'), (1, '51.910')] [2023-10-07 21:44:27,738][67838] Updated weights for policy 0, policy_version 49202 (0.0007) [2023-10-07 21:44:28,117][67838] Updated weights for policy 0, policy_version 49212 (0.0011) [2023-10-07 21:44:29,810][67871] Updated weights for policy 1, policy_version 49250 (0.0008) [2023-10-07 21:44:30,179][67871] Updated weights for policy 1, policy_version 49260 (0.0007) [2023-10-07 21:44:30,546][67871] Updated weights for policy 1, policy_version 49270 (0.0009) [2023-10-07 21:44:30,909][67871] Updated weights for policy 1, policy_version 49280 (0.0010) [2023-10-07 21:44:32,345][67838] Updated weights for policy 0, policy_version 49222 (0.0008) [2023-10-07 21:44:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 100859904. Throughput: 0: 1661.2, 1: 1652.0. Samples: 25224274. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 21:44:32,477][66916] Avg episode reward: [(0, '49.320'), (1, '55.030')] [2023-10-07 21:44:32,725][67838] Updated weights for policy 0, policy_version 49232 (0.0007) [2023-10-07 21:44:33,096][67838] Updated weights for policy 0, policy_version 49242 (0.0007) [2023-10-07 21:44:34,934][67871] Updated weights for policy 1, policy_version 49290 (0.0010) [2023-10-07 21:44:35,306][67871] Updated weights for policy 1, policy_version 49300 (0.0011) [2023-10-07 21:44:35,673][67871] Updated weights for policy 1, policy_version 49310 (0.0009) [2023-10-07 21:44:37,236][67838] Updated weights for policy 0, policy_version 49252 (0.0008) [2023-10-07 21:44:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 100925440. Throughput: 0: 1661.9, 1: 1674.1. Samples: 25244712. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 21:44:37,477][66916] Avg episode reward: [(0, '50.160'), (1, '52.330')] [2023-10-07 21:44:37,603][67838] Updated weights for policy 0, policy_version 49262 (0.0007) [2023-10-07 21:44:37,981][67838] Updated weights for policy 0, policy_version 49272 (0.0008) [2023-10-07 21:44:39,694][67871] Updated weights for policy 1, policy_version 49320 (0.0009) [2023-10-07 21:44:40,064][67871] Updated weights for policy 1, policy_version 49330 (0.0009) [2023-10-07 21:44:40,434][67871] Updated weights for policy 1, policy_version 49340 (0.0007) [2023-10-07 21:44:42,182][67838] Updated weights for policy 0, policy_version 49282 (0.0010) [2023-10-07 21:44:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 100990976. Throughput: 0: 1661.2, 1: 1664.4. Samples: 25254530. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 21:44:42,477][66916] Avg episode reward: [(0, '49.590'), (1, '52.030')] [2023-10-07 21:44:42,559][67838] Updated weights for policy 0, policy_version 49292 (0.0008) [2023-10-07 21:44:42,929][67838] Updated weights for policy 0, policy_version 49302 (0.0011) [2023-10-07 21:44:43,305][67838] Updated weights for policy 0, policy_version 49312 (0.0008) [2023-10-07 21:44:44,625][67871] Updated weights for policy 1, policy_version 49350 (0.0010) [2023-10-07 21:44:44,987][67871] Updated weights for policy 1, policy_version 49360 (0.0008) [2023-10-07 21:44:45,357][67871] Updated weights for policy 1, policy_version 49370 (0.0010) [2023-10-07 21:44:47,343][67838] Updated weights for policy 0, policy_version 49322 (0.0007) [2023-10-07 21:44:47,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101056512. Throughput: 0: 1663.4, 1: 1657.8. Samples: 25274214. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 21:44:47,478][66916] Avg episode reward: [(0, '47.510'), (1, '49.720')] [2023-10-07 21:44:47,707][67838] Updated weights for policy 0, policy_version 49332 (0.0010) [2023-10-07 21:44:48,081][67838] Updated weights for policy 0, policy_version 49342 (0.0011) [2023-10-07 21:44:49,595][67871] Updated weights for policy 1, policy_version 49380 (0.0008) [2023-10-07 21:44:49,955][67871] Updated weights for policy 1, policy_version 49390 (0.0009) [2023-10-07 21:44:50,317][67871] Updated weights for policy 1, policy_version 49400 (0.0011) [2023-10-07 21:44:52,164][67838] Updated weights for policy 0, policy_version 49352 (0.0009) [2023-10-07 21:44:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101122048. Throughput: 0: 1656.2, 1: 1665.5. Samples: 25294514. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 21:44:52,477][66916] Avg episode reward: [(0, '48.700'), (1, '49.370')] [2023-10-07 21:44:52,530][67838] Updated weights for policy 0, policy_version 49362 (0.0010) [2023-10-07 21:44:52,899][67838] Updated weights for policy 0, policy_version 49372 (0.0008) [2023-10-07 21:44:54,515][67871] Updated weights for policy 1, policy_version 49410 (0.0009) [2023-10-07 21:44:54,922][67871] Updated weights for policy 1, policy_version 49420 (0.0009) [2023-10-07 21:44:55,290][67871] Updated weights for policy 1, policy_version 49430 (0.0009) [2023-10-07 21:44:55,658][67871] Updated weights for policy 1, policy_version 49440 (0.0009) [2023-10-07 21:44:57,080][67838] Updated weights for policy 0, policy_version 49382 (0.0009) [2023-10-07 21:44:57,457][67838] Updated weights for policy 0, policy_version 49392 (0.0007) [2023-10-07 21:44:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101187584. Throughput: 0: 1660.7, 1: 1659.1. Samples: 25304424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:44:57,477][66916] Avg episode reward: [(0, '51.050'), (1, '51.600')] [2023-10-07 21:44:57,823][67838] Updated weights for policy 0, policy_version 49402 (0.0007) [2023-10-07 21:44:59,696][67871] Updated weights for policy 1, policy_version 49450 (0.0010) [2023-10-07 21:45:00,066][67871] Updated weights for policy 1, policy_version 49460 (0.0009) [2023-10-07 21:45:00,425][67871] Updated weights for policy 1, policy_version 49470 (0.0007) [2023-10-07 21:45:02,050][67838] Updated weights for policy 0, policy_version 49412 (0.0008) [2023-10-07 21:45:02,430][67838] Updated weights for policy 0, policy_version 49422 (0.0008) [2023-10-07 21:45:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101253120. Throughput: 0: 1651.1, 1: 1662.0. Samples: 25323778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:45:02,478][66916] Avg episode reward: [(0, '55.550'), (1, '59.190')] [2023-10-07 21:45:02,479][67676] Saving new best policy, reward=59.190! [2023-10-07 21:45:02,798][67838] Updated weights for policy 0, policy_version 49432 (0.0008) [2023-10-07 21:45:04,536][67871] Updated weights for policy 1, policy_version 49480 (0.0009) [2023-10-07 21:45:04,911][67871] Updated weights for policy 1, policy_version 49490 (0.0011) [2023-10-07 21:45:05,283][67871] Updated weights for policy 1, policy_version 49500 (0.0010) [2023-10-07 21:45:06,925][67838] Updated weights for policy 0, policy_version 49442 (0.0007) [2023-10-07 21:45:07,291][67838] Updated weights for policy 0, policy_version 49452 (0.0008) [2023-10-07 21:45:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101318656. Throughput: 0: 1647.4, 1: 1664.5. Samples: 25343988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:45:07,478][66916] Avg episode reward: [(0, '54.440'), (1, '57.070')] [2023-10-07 21:45:07,488][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000049504_50692096.pth... [2023-10-07 21:45:07,522][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000047936_49086464.pth [2023-10-07 21:45:07,668][67838] Updated weights for policy 0, policy_version 49462 (0.0009) [2023-10-07 21:45:08,029][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000049472_50659328.pth... [2023-10-07 21:45:08,031][67838] Updated weights for policy 0, policy_version 49472 (0.0007) [2023-10-07 21:45:08,068][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000047904_49053696.pth [2023-10-07 21:45:09,434][67871] Updated weights for policy 1, policy_version 49510 (0.0010) [2023-10-07 21:45:09,809][67871] Updated weights for policy 1, policy_version 49520 (0.0008) [2023-10-07 21:45:10,178][67871] Updated weights for policy 1, policy_version 49530 (0.0008) [2023-10-07 21:45:12,215][67838] Updated weights for policy 0, policy_version 49482 (0.0010) [2023-10-07 21:45:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101384192. Throughput: 0: 1652.7, 1: 1651.1. Samples: 25353594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:45:12,478][66916] Avg episode reward: [(0, '56.620'), (1, '58.620')] [2023-10-07 21:45:12,596][67838] Updated weights for policy 0, policy_version 49492 (0.0009) [2023-10-07 21:45:12,966][67838] Updated weights for policy 0, policy_version 49502 (0.0007) [2023-10-07 21:45:13,040][67511] Saving new best policy, reward=56.620! [2023-10-07 21:45:14,156][67871] Updated weights for policy 1, policy_version 49540 (0.0009) [2023-10-07 21:45:14,523][67871] Updated weights for policy 1, policy_version 49550 (0.0009) [2023-10-07 21:45:14,898][67871] Updated weights for policy 1, policy_version 49560 (0.0010) [2023-10-07 21:45:17,165][67838] Updated weights for policy 0, policy_version 49512 (0.0010) [2023-10-07 21:45:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101449728. Throughput: 0: 1653.7, 1: 1660.2. Samples: 25373402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:45:17,477][66916] Avg episode reward: [(0, '60.260'), (1, '61.660')] [2023-10-07 21:45:17,478][67676] Saving new best policy, reward=61.660! [2023-10-07 21:45:17,539][67838] Updated weights for policy 0, policy_version 49522 (0.0010) [2023-10-07 21:45:17,903][67838] Updated weights for policy 0, policy_version 49532 (0.0009) [2023-10-07 21:45:18,050][67511] Saving new best policy, reward=60.260! [2023-10-07 21:45:18,991][67871] Updated weights for policy 1, policy_version 49570 (0.0008) [2023-10-07 21:45:19,345][67871] Updated weights for policy 1, policy_version 49580 (0.0010) [2023-10-07 21:45:19,716][67871] Updated weights for policy 1, policy_version 49590 (0.0010) [2023-10-07 21:45:20,083][67871] Updated weights for policy 1, policy_version 49600 (0.0009) [2023-10-07 21:45:22,066][67838] Updated weights for policy 0, policy_version 49542 (0.0009) [2023-10-07 21:45:22,438][67838] Updated weights for policy 0, policy_version 49552 (0.0007) [2023-10-07 21:45:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101515264. Throughput: 0: 1646.2, 1: 1662.2. Samples: 25393588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:45:22,478][66916] Avg episode reward: [(0, '58.910'), (1, '57.530')] [2023-10-07 21:45:22,810][67838] Updated weights for policy 0, policy_version 49562 (0.0007) [2023-10-07 21:45:24,122][67871] Updated weights for policy 1, policy_version 49610 (0.0007) [2023-10-07 21:45:24,488][67871] Updated weights for policy 1, policy_version 49620 (0.0011) [2023-10-07 21:45:24,852][67871] Updated weights for policy 1, policy_version 49630 (0.0010) [2023-10-07 21:45:26,845][67838] Updated weights for policy 0, policy_version 49572 (0.0008) [2023-10-07 21:45:27,220][67838] Updated weights for policy 0, policy_version 49582 (0.0009) [2023-10-07 21:45:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101580800. Throughput: 0: 1650.6, 1: 1649.2. Samples: 25403020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:45:27,477][66916] Avg episode reward: [(0, '57.000'), (1, '55.410')] [2023-10-07 21:45:27,600][67838] Updated weights for policy 0, policy_version 49592 (0.0009) [2023-10-07 21:45:28,836][67871] Updated weights for policy 1, policy_version 49640 (0.0010) [2023-10-07 21:45:29,203][67871] Updated weights for policy 1, policy_version 49650 (0.0011) [2023-10-07 21:45:29,569][67871] Updated weights for policy 1, policy_version 49660 (0.0007) [2023-10-07 21:45:31,677][67838] Updated weights for policy 0, policy_version 49602 (0.0010) [2023-10-07 21:45:32,060][67838] Updated weights for policy 0, policy_version 49612 (0.0009) [2023-10-07 21:45:32,433][67838] Updated weights for policy 0, policy_version 49622 (0.0008) [2023-10-07 21:45:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101646336. Throughput: 0: 1646.4, 1: 1668.8. Samples: 25423402. Policy #0 lag: (min: 16.0, avg: 42.5, max: 48.0) [2023-10-07 21:45:32,477][66916] Avg episode reward: [(0, '59.640'), (1, '56.700')] [2023-10-07 21:45:32,809][67838] Updated weights for policy 0, policy_version 49632 (0.0008) [2023-10-07 21:45:33,761][67871] Updated weights for policy 1, policy_version 49670 (0.0010) [2023-10-07 21:45:34,125][67871] Updated weights for policy 1, policy_version 49680 (0.0009) [2023-10-07 21:45:34,489][67871] Updated weights for policy 1, policy_version 49690 (0.0009) [2023-10-07 21:45:36,933][67838] Updated weights for policy 0, policy_version 49642 (0.0008) [2023-10-07 21:45:37,298][67838] Updated weights for policy 0, policy_version 49652 (0.0008) [2023-10-07 21:45:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 101711872. Throughput: 0: 1642.4, 1: 1668.0. Samples: 25443484. Policy #0 lag: (min: 16.0, avg: 42.5, max: 48.0) [2023-10-07 21:45:37,478][66916] Avg episode reward: [(0, '57.900'), (1, '55.140')] [2023-10-07 21:45:37,677][67838] Updated weights for policy 0, policy_version 49662 (0.0007) [2023-10-07 21:45:38,618][67871] Updated weights for policy 1, policy_version 49700 (0.0008) [2023-10-07 21:45:38,980][67871] Updated weights for policy 1, policy_version 49710 (0.0010) [2023-10-07 21:45:39,349][67871] Updated weights for policy 1, policy_version 49720 (0.0011) [2023-10-07 21:45:41,770][67838] Updated weights for policy 0, policy_version 49672 (0.0010) [2023-10-07 21:45:42,150][67838] Updated weights for policy 0, policy_version 49682 (0.0011) [2023-10-07 21:45:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101777408. Throughput: 0: 1653.2, 1: 1653.9. Samples: 25453244. Policy #0 lag: (min: 16.0, avg: 42.5, max: 48.0) [2023-10-07 21:45:42,478][66916] Avg episode reward: [(0, '56.940'), (1, '53.880')] [2023-10-07 21:45:42,511][67838] Updated weights for policy 0, policy_version 49692 (0.0010) [2023-10-07 21:45:43,633][67871] Updated weights for policy 1, policy_version 49730 (0.0009) [2023-10-07 21:45:44,003][67871] Updated weights for policy 1, policy_version 49740 (0.0007) [2023-10-07 21:45:44,365][67871] Updated weights for policy 1, policy_version 49750 (0.0007) [2023-10-07 21:45:44,727][67871] Updated weights for policy 1, policy_version 49760 (0.0007) [2023-10-07 21:45:46,816][67838] Updated weights for policy 0, policy_version 49702 (0.0009) [2023-10-07 21:45:47,185][67838] Updated weights for policy 0, policy_version 49712 (0.0009) [2023-10-07 21:45:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101842944. Throughput: 0: 1655.2, 1: 1671.8. Samples: 25473492. Policy #0 lag: (min: 16.0, avg: 42.5, max: 48.0) [2023-10-07 21:45:47,478][66916] Avg episode reward: [(0, '59.640'), (1, '52.210')] [2023-10-07 21:45:47,569][67838] Updated weights for policy 0, policy_version 49722 (0.0007) [2023-10-07 21:45:48,788][67871] Updated weights for policy 1, policy_version 49770 (0.0007) [2023-10-07 21:45:49,154][67871] Updated weights for policy 1, policy_version 49780 (0.0008) [2023-10-07 21:45:49,514][67871] Updated weights for policy 1, policy_version 49790 (0.0009) [2023-10-07 21:45:51,770][67838] Updated weights for policy 0, policy_version 49732 (0.0009) [2023-10-07 21:45:52,142][67838] Updated weights for policy 0, policy_version 49742 (0.0008) [2023-10-07 21:45:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 101908480. Throughput: 0: 1651.4, 1: 1669.9. Samples: 25493448. Policy #0 lag: (min: 16.0, avg: 42.5, max: 48.0) [2023-10-07 21:45:52,478][66916] Avg episode reward: [(0, '58.090'), (1, '52.550')] [2023-10-07 21:45:52,526][67838] Updated weights for policy 0, policy_version 49752 (0.0009) [2023-10-07 21:45:53,641][67871] Updated weights for policy 1, policy_version 49800 (0.0008) [2023-10-07 21:45:54,011][67871] Updated weights for policy 1, policy_version 49810 (0.0010) [2023-10-07 21:45:54,384][67871] Updated weights for policy 1, policy_version 49820 (0.0007) [2023-10-07 21:45:56,708][67838] Updated weights for policy 0, policy_version 49762 (0.0009) [2023-10-07 21:45:57,106][67838] Updated weights for policy 0, policy_version 49772 (0.0008) [2023-10-07 21:45:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101974016. Throughput: 0: 1657.5, 1: 1662.9. Samples: 25503012. Policy #0 lag: (min: 16.0, avg: 42.5, max: 48.0) [2023-10-07 21:45:57,477][66916] Avg episode reward: [(0, '62.730'), (1, '52.910')] [2023-10-07 21:45:57,487][67838] Updated weights for policy 0, policy_version 49782 (0.0007) [2023-10-07 21:45:57,851][67511] Saving new best policy, reward=62.730! [2023-10-07 21:45:57,856][67838] Updated weights for policy 0, policy_version 49792 (0.0007) [2023-10-07 21:45:58,530][67871] Updated weights for policy 1, policy_version 49830 (0.0009) [2023-10-07 21:45:58,889][67871] Updated weights for policy 1, policy_version 49840 (0.0009) [2023-10-07 21:45:59,258][67871] Updated weights for policy 1, policy_version 49850 (0.0007) [2023-10-07 21:46:01,948][67838] Updated weights for policy 0, policy_version 49802 (0.0009) [2023-10-07 21:46:02,329][67838] Updated weights for policy 0, policy_version 49812 (0.0007) [2023-10-07 21:46:02,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 102039552. Throughput: 0: 1653.6, 1: 1672.8. Samples: 25523090. Policy #0 lag: (min: 16.0, avg: 42.5, max: 48.0) [2023-10-07 21:46:02,477][66916] Avg episode reward: [(0, '62.590'), (1, '55.440')] [2023-10-07 21:46:02,709][67838] Updated weights for policy 0, policy_version 49822 (0.0007) [2023-10-07 21:46:03,344][67871] Updated weights for policy 1, policy_version 49860 (0.0008) [2023-10-07 21:46:03,706][67871] Updated weights for policy 1, policy_version 49870 (0.0010) [2023-10-07 21:46:04,080][67871] Updated weights for policy 1, policy_version 49880 (0.0008) [2023-10-07 21:46:06,805][67838] Updated weights for policy 0, policy_version 49832 (0.0007) [2023-10-07 21:46:07,176][67838] Updated weights for policy 0, policy_version 49842 (0.0008) [2023-10-07 21:46:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 102105088. Throughput: 0: 1649.7, 1: 1668.8. Samples: 25542924. Policy #0 lag: (min: 38.0, avg: 47.4, max: 48.0) [2023-10-07 21:46:07,478][66916] Avg episode reward: [(0, '61.640'), (1, '52.850')] [2023-10-07 21:46:07,558][67838] Updated weights for policy 0, policy_version 49852 (0.0008) [2023-10-07 21:46:08,257][67871] Updated weights for policy 1, policy_version 49890 (0.0010) [2023-10-07 21:46:08,623][67871] Updated weights for policy 1, policy_version 49900 (0.0007) [2023-10-07 21:46:08,999][67871] Updated weights for policy 1, policy_version 49910 (0.0007) [2023-10-07 21:46:09,373][67871] Updated weights for policy 1, policy_version 49920 (0.0009) [2023-10-07 21:46:11,366][67838] Updated weights for policy 0, policy_version 49862 (0.0008) [2023-10-07 21:46:11,743][67838] Updated weights for policy 0, policy_version 49872 (0.0007) [2023-10-07 21:46:12,113][67838] Updated weights for policy 0, policy_version 49882 (0.0008) [2023-10-07 21:46:12,476][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 102203392. Throughput: 0: 1660.5, 1: 1662.3. Samples: 25552544. Policy #0 lag: (min: 38.0, avg: 47.4, max: 48.0) [2023-10-07 21:46:12,477][66916] Avg episode reward: [(0, '59.350'), (1, '53.660')] [2023-10-07 21:46:13,607][67871] Updated weights for policy 1, policy_version 49930 (0.0007) [2023-10-07 21:46:13,973][67871] Updated weights for policy 1, policy_version 49940 (0.0007) [2023-10-07 21:46:14,349][67871] Updated weights for policy 1, policy_version 49950 (0.0008) [2023-10-07 21:46:16,216][67838] Updated weights for policy 0, policy_version 49892 (0.0010) [2023-10-07 21:46:16,585][67838] Updated weights for policy 0, policy_version 49902 (0.0009) [2023-10-07 21:46:16,960][67838] Updated weights for policy 0, policy_version 49912 (0.0007) [2023-10-07 21:46:17,477][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 102268928. Throughput: 0: 1659.1, 1: 1663.7. Samples: 25572930. Policy #0 lag: (min: 38.0, avg: 47.4, max: 48.0) [2023-10-07 21:46:17,478][66916] Avg episode reward: [(0, '55.300'), (1, '54.180')] [2023-10-07 21:46:18,518][67871] Updated weights for policy 1, policy_version 49960 (0.0009) [2023-10-07 21:46:18,883][67871] Updated weights for policy 1, policy_version 49970 (0.0007) [2023-10-07 21:46:19,253][67871] Updated weights for policy 1, policy_version 49980 (0.0007) [2023-10-07 21:46:21,061][67838] Updated weights for policy 0, policy_version 49922 (0.0007) [2023-10-07 21:46:21,426][67838] Updated weights for policy 0, policy_version 49932 (0.0007) [2023-10-07 21:46:21,806][67838] Updated weights for policy 0, policy_version 49942 (0.0009) [2023-10-07 21:46:22,174][67838] Updated weights for policy 0, policy_version 49952 (0.0009) [2023-10-07 21:46:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 102334464. Throughput: 0: 1645.6, 1: 1662.8. Samples: 25592362. Policy #0 lag: (min: 38.0, avg: 47.4, max: 48.0) [2023-10-07 21:46:22,477][66916] Avg episode reward: [(0, '49.440'), (1, '51.660')] [2023-10-07 21:46:23,205][67871] Updated weights for policy 1, policy_version 49990 (0.0009) [2023-10-07 21:46:23,584][67871] Updated weights for policy 1, policy_version 50000 (0.0008) [2023-10-07 21:46:23,950][67871] Updated weights for policy 1, policy_version 50010 (0.0007) [2023-10-07 21:46:26,298][67838] Updated weights for policy 0, policy_version 49962 (0.0007) [2023-10-07 21:46:26,675][67838] Updated weights for policy 0, policy_version 49972 (0.0007) [2023-10-07 21:46:27,030][67838] Updated weights for policy 0, policy_version 49982 (0.0007) [2023-10-07 21:46:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 102400000. Throughput: 0: 1656.8, 1: 1665.2. Samples: 25602738. Policy #0 lag: (min: 38.0, avg: 47.4, max: 48.0) [2023-10-07 21:46:27,478][66916] Avg episode reward: [(0, '49.640'), (1, '51.680')] [2023-10-07 21:46:28,148][67871] Updated weights for policy 1, policy_version 50020 (0.0008) [2023-10-07 21:46:28,551][67871] Updated weights for policy 1, policy_version 50030 (0.0009) [2023-10-07 21:46:28,929][67871] Updated weights for policy 1, policy_version 50040 (0.0009) [2023-10-07 21:46:31,274][67838] Updated weights for policy 0, policy_version 49992 (0.0010) [2023-10-07 21:46:31,637][67838] Updated weights for policy 0, policy_version 50002 (0.0007) [2023-10-07 21:46:32,012][67838] Updated weights for policy 0, policy_version 50012 (0.0007) [2023-10-07 21:46:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 102465536. Throughput: 0: 1656.7, 1: 1659.7. Samples: 25622730. Policy #0 lag: (min: 38.0, avg: 47.4, max: 48.0) [2023-10-07 21:46:32,478][66916] Avg episode reward: [(0, '47.740'), (1, '55.430')] [2023-10-07 21:46:32,897][67871] Updated weights for policy 1, policy_version 50050 (0.0009) [2023-10-07 21:46:33,269][67871] Updated weights for policy 1, policy_version 50060 (0.0008) [2023-10-07 21:46:33,638][67871] Updated weights for policy 1, policy_version 50070 (0.0008) [2023-10-07 21:46:33,995][67871] Updated weights for policy 1, policy_version 50080 (0.0008) [2023-10-07 21:46:36,099][67838] Updated weights for policy 0, policy_version 50022 (0.0008) [2023-10-07 21:46:36,479][67838] Updated weights for policy 0, policy_version 50032 (0.0007) [2023-10-07 21:46:36,844][67838] Updated weights for policy 0, policy_version 50042 (0.0009) [2023-10-07 21:46:37,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 102531072. Throughput: 0: 1643.3, 1: 1665.8. Samples: 25642354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:46:37,477][66916] Avg episode reward: [(0, '47.380'), (1, '53.740')] [2023-10-07 21:46:38,137][67871] Updated weights for policy 1, policy_version 50090 (0.0008) [2023-10-07 21:46:38,505][67871] Updated weights for policy 1, policy_version 50100 (0.0007) [2023-10-07 21:46:38,872][67871] Updated weights for policy 1, policy_version 50110 (0.0007) [2023-10-07 21:46:41,077][67838] Updated weights for policy 0, policy_version 50052 (0.0007) [2023-10-07 21:46:41,445][67838] Updated weights for policy 0, policy_version 50062 (0.0007) [2023-10-07 21:46:41,825][67838] Updated weights for policy 0, policy_version 50072 (0.0009) [2023-10-07 21:46:42,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 102596608. Throughput: 0: 1658.1, 1: 1662.2. Samples: 25652426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:46:42,478][66916] Avg episode reward: [(0, '49.490'), (1, '52.890')] [2023-10-07 21:46:42,886][67871] Updated weights for policy 1, policy_version 50120 (0.0008) [2023-10-07 21:46:43,252][67871] Updated weights for policy 1, policy_version 50130 (0.0011) [2023-10-07 21:46:43,625][67871] Updated weights for policy 1, policy_version 50140 (0.0009) [2023-10-07 21:46:45,954][67838] Updated weights for policy 0, policy_version 50082 (0.0007) [2023-10-07 21:46:46,363][67838] Updated weights for policy 0, policy_version 50092 (0.0009) [2023-10-07 21:46:46,739][67838] Updated weights for policy 0, policy_version 50102 (0.0007) [2023-10-07 21:46:47,116][67838] Updated weights for policy 0, policy_version 50112 (0.0007) [2023-10-07 21:46:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 102662144. Throughput: 0: 1658.0, 1: 1671.1. Samples: 25672900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:46:47,477][66916] Avg episode reward: [(0, '50.090'), (1, '53.030')] [2023-10-07 21:46:47,680][67871] Updated weights for policy 1, policy_version 50150 (0.0008) [2023-10-07 21:46:48,045][67871] Updated weights for policy 1, policy_version 50160 (0.0008) [2023-10-07 21:46:48,413][67871] Updated weights for policy 1, policy_version 50170 (0.0008) [2023-10-07 21:46:51,162][67838] Updated weights for policy 0, policy_version 50122 (0.0007) [2023-10-07 21:46:51,527][67838] Updated weights for policy 0, policy_version 50132 (0.0007) [2023-10-07 21:46:51,902][67838] Updated weights for policy 0, policy_version 50142 (0.0007) [2023-10-07 21:46:52,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 102727680. Throughput: 0: 1644.5, 1: 1673.6. Samples: 25692240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:46:52,477][66916] Avg episode reward: [(0, '51.600'), (1, '52.710')] [2023-10-07 21:46:52,558][67871] Updated weights for policy 1, policy_version 50180 (0.0009) [2023-10-07 21:46:52,936][67871] Updated weights for policy 1, policy_version 50190 (0.0008) [2023-10-07 21:46:53,296][67871] Updated weights for policy 1, policy_version 50200 (0.0007) [2023-10-07 21:46:55,880][67838] Updated weights for policy 0, policy_version 50152 (0.0009) [2023-10-07 21:46:56,251][67838] Updated weights for policy 0, policy_version 50162 (0.0011) [2023-10-07 21:46:56,621][67838] Updated weights for policy 0, policy_version 50172 (0.0012) [2023-10-07 21:46:57,315][67871] Updated weights for policy 1, policy_version 50210 (0.0007) [2023-10-07 21:46:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 102793216. Throughput: 0: 1659.6, 1: 1674.9. Samples: 25702600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:46:57,478][66916] Avg episode reward: [(0, '49.850'), (1, '49.340')] [2023-10-07 21:46:57,675][67871] Updated weights for policy 1, policy_version 50220 (0.0008) [2023-10-07 21:46:58,048][67871] Updated weights for policy 1, policy_version 50230 (0.0008) [2023-10-07 21:46:58,419][67871] Updated weights for policy 1, policy_version 50240 (0.0008) [2023-10-07 21:47:00,708][67838] Updated weights for policy 0, policy_version 50182 (0.0009) [2023-10-07 21:47:01,086][67838] Updated weights for policy 0, policy_version 50192 (0.0008) [2023-10-07 21:47:01,458][67838] Updated weights for policy 0, policy_version 50202 (0.0007) [2023-10-07 21:47:02,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 102858752. Throughput: 0: 1648.9, 1: 1679.3. Samples: 25722698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:47:02,477][66916] Avg episode reward: [(0, '47.870'), (1, '52.380')] [2023-10-07 21:47:02,573][67871] Updated weights for policy 1, policy_version 50250 (0.0007) [2023-10-07 21:47:02,937][67871] Updated weights for policy 1, policy_version 50260 (0.0009) [2023-10-07 21:47:03,312][67871] Updated weights for policy 1, policy_version 50270 (0.0011) [2023-10-07 21:47:05,787][67838] Updated weights for policy 0, policy_version 50212 (0.0008) [2023-10-07 21:47:06,152][67838] Updated weights for policy 0, policy_version 50222 (0.0007) [2023-10-07 21:47:06,525][67838] Updated weights for policy 0, policy_version 50232 (0.0008) [2023-10-07 21:47:07,421][67871] Updated weights for policy 1, policy_version 50280 (0.0010) [2023-10-07 21:47:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 102924288. Throughput: 0: 1653.3, 1: 1680.4. Samples: 25742382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:47:07,477][66916] Avg episode reward: [(0, '49.380'), (1, '52.350')] [2023-10-07 21:47:07,485][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000050240_51445760.pth... [2023-10-07 21:47:07,523][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000048672_49840128.pth [2023-10-07 21:47:07,785][67871] Updated weights for policy 1, policy_version 50290 (0.0010) [2023-10-07 21:47:08,154][67871] Updated weights for policy 1, policy_version 50300 (0.0010) [2023-10-07 21:47:08,299][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000050304_51511296.pth... [2023-10-07 21:47:08,339][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000048736_49905664.pth [2023-10-07 21:47:10,495][67838] Updated weights for policy 0, policy_version 50242 (0.0009) [2023-10-07 21:47:10,859][67838] Updated weights for policy 0, policy_version 50252 (0.0007) [2023-10-07 21:47:11,233][67838] Updated weights for policy 0, policy_version 50262 (0.0007) [2023-10-07 21:47:11,606][67838] Updated weights for policy 0, policy_version 50272 (0.0008) [2023-10-07 21:47:12,192][67871] Updated weights for policy 1, policy_version 50310 (0.0008) [2023-10-07 21:47:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102989824. Throughput: 0: 1657.1, 1: 1674.0. Samples: 25752638. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-07 21:47:12,477][66916] Avg episode reward: [(0, '47.740'), (1, '53.910')] [2023-10-07 21:47:12,555][67871] Updated weights for policy 1, policy_version 50320 (0.0009) [2023-10-07 21:47:12,908][67871] Updated weights for policy 1, policy_version 50330 (0.0008) [2023-10-07 21:47:16,047][67838] Updated weights for policy 0, policy_version 50282 (0.0009) [2023-10-07 21:47:16,418][67838] Updated weights for policy 0, policy_version 50292 (0.0008) [2023-10-07 21:47:16,780][67838] Updated weights for policy 0, policy_version 50302 (0.0007) [2023-10-07 21:47:17,107][67871] Updated weights for policy 1, policy_version 50340 (0.0007) [2023-10-07 21:47:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 103055360. Throughput: 0: 1648.5, 1: 1680.6. Samples: 25772540. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-07 21:47:17,477][66916] Avg episode reward: [(0, '48.270'), (1, '53.890')] [2023-10-07 21:47:17,488][67871] Updated weights for policy 1, policy_version 50350 (0.0009) [2023-10-07 21:47:17,845][67871] Updated weights for policy 1, policy_version 50360 (0.0009) [2023-10-07 21:47:20,544][67838] Updated weights for policy 0, policy_version 50312 (0.0008) [2023-10-07 21:47:20,914][67838] Updated weights for policy 0, policy_version 50322 (0.0009) [2023-10-07 21:47:21,275][67838] Updated weights for policy 0, policy_version 50332 (0.0009) [2023-10-07 21:47:21,933][67871] Updated weights for policy 1, policy_version 50370 (0.0011) [2023-10-07 21:47:22,299][67871] Updated weights for policy 1, policy_version 50380 (0.0010) [2023-10-07 21:47:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 103120896. Throughput: 0: 1659.0, 1: 1673.2. Samples: 25792302. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-07 21:47:22,477][66916] Avg episode reward: [(0, '48.100'), (1, '55.570')] [2023-10-07 21:47:22,670][67871] Updated weights for policy 1, policy_version 50390 (0.0008) [2023-10-07 21:47:23,041][67871] Updated weights for policy 1, policy_version 50400 (0.0008) [2023-10-07 21:47:25,562][67838] Updated weights for policy 0, policy_version 50342 (0.0007) [2023-10-07 21:47:25,929][67838] Updated weights for policy 0, policy_version 50352 (0.0007) [2023-10-07 21:47:26,311][67838] Updated weights for policy 0, policy_version 50362 (0.0007) [2023-10-07 21:47:27,108][67871] Updated weights for policy 1, policy_version 50410 (0.0007) [2023-10-07 21:47:27,468][67871] Updated weights for policy 1, policy_version 50420 (0.0007) [2023-10-07 21:47:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 103186432. Throughput: 0: 1660.3, 1: 1673.1. Samples: 25802428. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-07 21:47:27,478][66916] Avg episode reward: [(0, '49.100'), (1, '54.700')] [2023-10-07 21:47:27,842][67871] Updated weights for policy 1, policy_version 50430 (0.0009) [2023-10-07 21:47:30,317][67838] Updated weights for policy 0, policy_version 50372 (0.0010) [2023-10-07 21:47:30,710][67838] Updated weights for policy 0, policy_version 50382 (0.0009) [2023-10-07 21:47:31,083][67838] Updated weights for policy 0, policy_version 50392 (0.0009) [2023-10-07 21:47:31,978][67871] Updated weights for policy 1, policy_version 50440 (0.0008) [2023-10-07 21:47:32,356][67871] Updated weights for policy 1, policy_version 50450 (0.0010) [2023-10-07 21:47:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 103251968. Throughput: 0: 1644.2, 1: 1669.3. Samples: 25822010. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-07 21:47:32,478][66916] Avg episode reward: [(0, '46.310'), (1, '54.480')] [2023-10-07 21:47:32,718][67871] Updated weights for policy 1, policy_version 50460 (0.0009) [2023-10-07 21:47:35,272][67838] Updated weights for policy 0, policy_version 50402 (0.0010) [2023-10-07 21:47:35,655][67838] Updated weights for policy 0, policy_version 50412 (0.0011) [2023-10-07 21:47:36,026][67838] Updated weights for policy 0, policy_version 50422 (0.0010) [2023-10-07 21:47:36,395][67838] Updated weights for policy 0, policy_version 50432 (0.0008) [2023-10-07 21:47:36,750][67871] Updated weights for policy 1, policy_version 50470 (0.0007) [2023-10-07 21:47:37,113][67871] Updated weights for policy 1, policy_version 50480 (0.0007) [2023-10-07 21:47:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 103317504. Throughput: 0: 1658.5, 1: 1665.5. Samples: 25841820. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-07 21:47:37,478][66916] Avg episode reward: [(0, '46.570'), (1, '54.750')] [2023-10-07 21:47:37,490][67871] Updated weights for policy 1, policy_version 50490 (0.0007) [2023-10-07 21:47:40,522][67838] Updated weights for policy 0, policy_version 50442 (0.0010) [2023-10-07 21:47:40,897][67838] Updated weights for policy 0, policy_version 50452 (0.0008) [2023-10-07 21:47:41,274][67838] Updated weights for policy 0, policy_version 50462 (0.0008) [2023-10-07 21:47:41,598][67871] Updated weights for policy 1, policy_version 50500 (0.0010) [2023-10-07 21:47:41,973][67871] Updated weights for policy 1, policy_version 50510 (0.0008) [2023-10-07 21:47:42,330][67871] Updated weights for policy 1, policy_version 50520 (0.0007) [2023-10-07 21:47:42,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 103383040. Throughput: 0: 1657.2, 1: 1673.7. Samples: 25852488. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-07 21:47:42,477][66916] Avg episode reward: [(0, '48.690'), (1, '54.770')] [2023-10-07 21:47:45,519][67838] Updated weights for policy 0, policy_version 50472 (0.0008) [2023-10-07 21:47:45,896][67838] Updated weights for policy 0, policy_version 50482 (0.0008) [2023-10-07 21:47:46,264][67838] Updated weights for policy 0, policy_version 50492 (0.0007) [2023-10-07 21:47:46,379][67871] Updated weights for policy 1, policy_version 50530 (0.0009) [2023-10-07 21:47:46,755][67871] Updated weights for policy 1, policy_version 50540 (0.0009) [2023-10-07 21:47:47,132][67871] Updated weights for policy 1, policy_version 50550 (0.0009) [2023-10-07 21:47:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 103448576. Throughput: 0: 1647.9, 1: 1671.1. Samples: 25872056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:47:47,478][66916] Avg episode reward: [(0, '49.390'), (1, '54.770')] [2023-10-07 21:47:47,504][67871] Updated weights for policy 1, policy_version 50560 (0.0008) [2023-10-07 21:47:50,301][67838] Updated weights for policy 0, policy_version 50502 (0.0007) [2023-10-07 21:47:50,675][67838] Updated weights for policy 0, policy_version 50512 (0.0007) [2023-10-07 21:47:51,044][67838] Updated weights for policy 0, policy_version 50522 (0.0011) [2023-10-07 21:47:51,760][67871] Updated weights for policy 1, policy_version 50570 (0.0010) [2023-10-07 21:47:52,137][67871] Updated weights for policy 1, policy_version 50580 (0.0010) [2023-10-07 21:47:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 103514112. Throughput: 0: 1660.4, 1: 1657.9. Samples: 25891708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:47:52,478][66916] Avg episode reward: [(0, '46.490'), (1, '54.260')] [2023-10-07 21:47:52,513][67871] Updated weights for policy 1, policy_version 50590 (0.0011) [2023-10-07 21:47:55,214][67838] Updated weights for policy 0, policy_version 50532 (0.0008) [2023-10-07 21:47:55,586][67838] Updated weights for policy 0, policy_version 50542 (0.0007) [2023-10-07 21:47:55,967][67838] Updated weights for policy 0, policy_version 50552 (0.0009) [2023-10-07 21:47:56,576][67871] Updated weights for policy 1, policy_version 50600 (0.0007) [2023-10-07 21:47:56,944][67871] Updated weights for policy 1, policy_version 50610 (0.0008) [2023-10-07 21:47:57,300][67871] Updated weights for policy 1, policy_version 50620 (0.0007) [2023-10-07 21:47:57,476][66916] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 103612416. Throughput: 0: 1657.7, 1: 1667.5. Samples: 25902270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:47:57,477][66916] Avg episode reward: [(0, '49.480'), (1, '55.810')] [2023-10-07 21:48:00,189][67838] Updated weights for policy 0, policy_version 50562 (0.0009) [2023-10-07 21:48:00,561][67838] Updated weights for policy 0, policy_version 50572 (0.0008) [2023-10-07 21:48:00,929][67838] Updated weights for policy 0, policy_version 50582 (0.0009) [2023-10-07 21:48:01,306][67838] Updated weights for policy 0, policy_version 50592 (0.0011) [2023-10-07 21:48:01,429][67871] Updated weights for policy 1, policy_version 50630 (0.0007) [2023-10-07 21:48:01,804][67871] Updated weights for policy 1, policy_version 50640 (0.0008) [2023-10-07 21:48:02,178][67871] Updated weights for policy 1, policy_version 50650 (0.0008) [2023-10-07 21:48:02,477][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 103677952. Throughput: 0: 1650.3, 1: 1668.4. Samples: 25921882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:48:02,478][66916] Avg episode reward: [(0, '45.630'), (1, '57.200')] [2023-10-07 21:48:05,493][67838] Updated weights for policy 0, policy_version 50602 (0.0007) [2023-10-07 21:48:05,861][67838] Updated weights for policy 0, policy_version 50612 (0.0007) [2023-10-07 21:48:06,225][67838] Updated weights for policy 0, policy_version 50622 (0.0007) [2023-10-07 21:48:06,376][67871] Updated weights for policy 1, policy_version 50660 (0.0008) [2023-10-07 21:48:06,771][67871] Updated weights for policy 1, policy_version 50670 (0.0008) [2023-10-07 21:48:07,147][67871] Updated weights for policy 1, policy_version 50680 (0.0009) [2023-10-07 21:48:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 103743488. Throughput: 0: 1657.6, 1: 1655.6. Samples: 25941394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:48:07,477][66916] Avg episode reward: [(0, '47.970'), (1, '53.800')] [2023-10-07 21:48:10,238][67838] Updated weights for policy 0, policy_version 50632 (0.0008) [2023-10-07 21:48:10,607][67838] Updated weights for policy 0, policy_version 50642 (0.0007) [2023-10-07 21:48:10,972][67838] Updated weights for policy 0, policy_version 50652 (0.0007) [2023-10-07 21:48:11,206][67871] Updated weights for policy 1, policy_version 50690 (0.0009) [2023-10-07 21:48:11,576][67871] Updated weights for policy 1, policy_version 50700 (0.0008) [2023-10-07 21:48:11,954][67871] Updated weights for policy 1, policy_version 50710 (0.0008) [2023-10-07 21:48:12,325][67871] Updated weights for policy 1, policy_version 50720 (0.0009) [2023-10-07 21:48:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 103809024. Throughput: 0: 1657.5, 1: 1668.4. Samples: 25952096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:48:12,478][66916] Avg episode reward: [(0, '49.160'), (1, '51.540')] [2023-10-07 21:48:15,100][67838] Updated weights for policy 0, policy_version 50662 (0.0009) [2023-10-07 21:48:15,467][67838] Updated weights for policy 0, policy_version 50672 (0.0010) [2023-10-07 21:48:15,839][67838] Updated weights for policy 0, policy_version 50682 (0.0011) [2023-10-07 21:48:16,616][67871] Updated weights for policy 1, policy_version 50730 (0.0008) [2023-10-07 21:48:16,978][67871] Updated weights for policy 1, policy_version 50740 (0.0010) [2023-10-07 21:48:17,346][67871] Updated weights for policy 1, policy_version 50750 (0.0009) [2023-10-07 21:48:17,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 103874560. Throughput: 0: 1651.0, 1: 1662.0. Samples: 25971092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:48:17,478][66916] Avg episode reward: [(0, '45.010'), (1, '51.900')] [2023-10-07 21:48:19,972][67838] Updated weights for policy 0, policy_version 50692 (0.0008) [2023-10-07 21:48:20,344][67838] Updated weights for policy 0, policy_version 50702 (0.0009) [2023-10-07 21:48:20,719][67838] Updated weights for policy 0, policy_version 50712 (0.0009) [2023-10-07 21:48:21,601][67871] Updated weights for policy 1, policy_version 50760 (0.0009) [2023-10-07 21:48:21,967][67871] Updated weights for policy 1, policy_version 50770 (0.0009) [2023-10-07 21:48:22,336][67871] Updated weights for policy 1, policy_version 50780 (0.0009) [2023-10-07 21:48:22,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 103940096. Throughput: 0: 1664.7, 1: 1649.0. Samples: 25990936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:48:22,477][66916] Avg episode reward: [(0, '40.440'), (1, '48.830')] [2023-10-07 21:48:24,746][67838] Updated weights for policy 0, policy_version 50722 (0.0009) [2023-10-07 21:48:25,116][67838] Updated weights for policy 0, policy_version 50732 (0.0011) [2023-10-07 21:48:25,490][67838] Updated weights for policy 0, policy_version 50742 (0.0008) [2023-10-07 21:48:25,861][67838] Updated weights for policy 0, policy_version 50752 (0.0008) [2023-10-07 21:48:26,548][67871] Updated weights for policy 1, policy_version 50790 (0.0009) [2023-10-07 21:48:26,905][67871] Updated weights for policy 1, policy_version 50800 (0.0009) [2023-10-07 21:48:27,279][67871] Updated weights for policy 1, policy_version 50810 (0.0008) [2023-10-07 21:48:27,477][66916] Fps is (10 sec: 9830.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 103972864. Throughput: 0: 1655.1, 1: 1651.9. Samples: 26001308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:48:27,478][66916] Avg episode reward: [(0, '38.680'), (1, '49.100')] [2023-10-07 21:48:29,910][67838] Updated weights for policy 0, policy_version 50762 (0.0007) [2023-10-07 21:48:30,283][67838] Updated weights for policy 0, policy_version 50772 (0.0009) [2023-10-07 21:48:30,656][67838] Updated weights for policy 0, policy_version 50782 (0.0007) [2023-10-07 21:48:31,373][67871] Updated weights for policy 1, policy_version 50820 (0.0009) [2023-10-07 21:48:31,748][67871] Updated weights for policy 1, policy_version 50830 (0.0009) [2023-10-07 21:48:32,105][67871] Updated weights for policy 1, policy_version 50840 (0.0010) [2023-10-07 21:48:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 104071168. Throughput: 0: 1658.5, 1: 1648.4. Samples: 26020866. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:48:32,477][66916] Avg episode reward: [(0, '36.820'), (1, '48.070')] [2023-10-07 21:48:34,725][67838] Updated weights for policy 0, policy_version 50792 (0.0007) [2023-10-07 21:48:35,098][67838] Updated weights for policy 0, policy_version 50802 (0.0007) [2023-10-07 21:48:35,456][67838] Updated weights for policy 0, policy_version 50812 (0.0008) [2023-10-07 21:48:36,165][67871] Updated weights for policy 1, policy_version 50850 (0.0011) [2023-10-07 21:48:36,531][67871] Updated weights for policy 1, policy_version 50860 (0.0010) [2023-10-07 21:48:36,895][67871] Updated weights for policy 1, policy_version 50870 (0.0010) [2023-10-07 21:48:37,263][67871] Updated weights for policy 1, policy_version 50880 (0.0009) [2023-10-07 21:48:37,477][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 104136704. Throughput: 0: 1666.0, 1: 1640.4. Samples: 26040492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:48:37,478][66916] Avg episode reward: [(0, '39.250'), (1, '51.310')] [2023-10-07 21:48:39,445][67838] Updated weights for policy 0, policy_version 50822 (0.0009) [2023-10-07 21:48:39,808][67838] Updated weights for policy 0, policy_version 50832 (0.0007) [2023-10-07 21:48:40,180][67838] Updated weights for policy 0, policy_version 50842 (0.0009) [2023-10-07 21:48:41,344][67871] Updated weights for policy 1, policy_version 50890 (0.0012) [2023-10-07 21:48:41,708][67871] Updated weights for policy 1, policy_version 50900 (0.0007) [2023-10-07 21:48:42,083][67871] Updated weights for policy 1, policy_version 50910 (0.0007) [2023-10-07 21:48:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 104202240. Throughput: 0: 1647.7, 1: 1654.0. Samples: 26050846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:48:42,477][66916] Avg episode reward: [(0, '40.950'), (1, '51.320')] [2023-10-07 21:48:44,497][67838] Updated weights for policy 0, policy_version 50852 (0.0008) [2023-10-07 21:48:44,864][67838] Updated weights for policy 0, policy_version 50862 (0.0007) [2023-10-07 21:48:45,237][67838] Updated weights for policy 0, policy_version 50872 (0.0008) [2023-10-07 21:48:46,124][67871] Updated weights for policy 1, policy_version 50920 (0.0009) [2023-10-07 21:48:46,485][67871] Updated weights for policy 1, policy_version 50930 (0.0008) [2023-10-07 21:48:46,852][67871] Updated weights for policy 1, policy_version 50940 (0.0007) [2023-10-07 21:48:47,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 104267776. Throughput: 0: 1655.5, 1: 1653.7. Samples: 26070796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:48:47,478][66916] Avg episode reward: [(0, '42.120'), (1, '53.070')] [2023-10-07 21:48:49,301][67838] Updated weights for policy 0, policy_version 50882 (0.0010) [2023-10-07 21:48:49,675][67838] Updated weights for policy 0, policy_version 50892 (0.0010) [2023-10-07 21:48:50,046][67838] Updated weights for policy 0, policy_version 50902 (0.0008) [2023-10-07 21:48:50,413][67838] Updated weights for policy 0, policy_version 50912 (0.0009) [2023-10-07 21:48:50,892][67871] Updated weights for policy 1, policy_version 50950 (0.0007) [2023-10-07 21:48:51,289][67871] Updated weights for policy 1, policy_version 50960 (0.0007) [2023-10-07 21:48:51,650][67871] Updated weights for policy 1, policy_version 50970 (0.0009) [2023-10-07 21:48:52,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 104333312. Throughput: 0: 1662.0, 1: 1644.8. Samples: 26090202. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:48:52,478][66916] Avg episode reward: [(0, '44.640'), (1, '53.470')] [2023-10-07 21:48:54,671][67838] Updated weights for policy 0, policy_version 50922 (0.0007) [2023-10-07 21:48:55,039][67838] Updated weights for policy 0, policy_version 50932 (0.0009) [2023-10-07 21:48:55,418][67838] Updated weights for policy 0, policy_version 50942 (0.0008) [2023-10-07 21:48:55,694][67871] Updated weights for policy 1, policy_version 50980 (0.0008) [2023-10-07 21:48:56,052][67871] Updated weights for policy 1, policy_version 50990 (0.0008) [2023-10-07 21:48:56,412][67871] Updated weights for policy 1, policy_version 51000 (0.0007) [2023-10-07 21:48:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 104398848. Throughput: 0: 1644.7, 1: 1659.9. Samples: 26100800. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:48:57,477][66916] Avg episode reward: [(0, '42.090'), (1, '54.470')] [2023-10-07 21:48:59,737][67838] Updated weights for policy 0, policy_version 50952 (0.0008) [2023-10-07 21:49:00,115][67838] Updated weights for policy 0, policy_version 50962 (0.0009) [2023-10-07 21:49:00,482][67838] Updated weights for policy 0, policy_version 50972 (0.0008) [2023-10-07 21:49:00,541][67871] Updated weights for policy 1, policy_version 51010 (0.0008) [2023-10-07 21:49:00,902][67871] Updated weights for policy 1, policy_version 51020 (0.0007) [2023-10-07 21:49:01,272][67871] Updated weights for policy 1, policy_version 51030 (0.0009) [2023-10-07 21:49:01,633][67871] Updated weights for policy 1, policy_version 51040 (0.0009) [2023-10-07 21:49:02,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 104464384. Throughput: 0: 1657.6, 1: 1655.1. Samples: 26120164. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:49:02,477][66916] Avg episode reward: [(0, '38.280'), (1, '56.070')] [2023-10-07 21:49:04,665][67838] Updated weights for policy 0, policy_version 50982 (0.0009) [2023-10-07 21:49:05,040][67838] Updated weights for policy 0, policy_version 50992 (0.0009) [2023-10-07 21:49:05,415][67838] Updated weights for policy 0, policy_version 51002 (0.0008) [2023-10-07 21:49:05,699][67871] Updated weights for policy 1, policy_version 51050 (0.0008) [2023-10-07 21:49:06,067][67871] Updated weights for policy 1, policy_version 51060 (0.0009) [2023-10-07 21:49:06,427][67871] Updated weights for policy 1, policy_version 51070 (0.0009) [2023-10-07 21:49:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 104529920. Throughput: 0: 1649.0, 1: 1656.3. Samples: 26139676. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:49:07,478][66916] Avg episode reward: [(0, '37.630'), (1, '56.030')] [2023-10-07 21:49:07,488][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000051072_52297728.pth... [2023-10-07 21:49:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000051008_52232192.pth... [2023-10-07 21:49:07,525][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000049472_50659328.pth [2023-10-07 21:49:07,527][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000049504_50692096.pth [2023-10-07 21:49:09,605][67838] Updated weights for policy 0, policy_version 51012 (0.0009) [2023-10-07 21:49:09,973][67838] Updated weights for policy 0, policy_version 51022 (0.0007) [2023-10-07 21:49:10,349][67838] Updated weights for policy 0, policy_version 51032 (0.0007) [2023-10-07 21:49:10,445][67871] Updated weights for policy 1, policy_version 51080 (0.0007) [2023-10-07 21:49:10,803][67871] Updated weights for policy 1, policy_version 51090 (0.0009) [2023-10-07 21:49:11,180][67871] Updated weights for policy 1, policy_version 51100 (0.0010) [2023-10-07 21:49:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 104595456. Throughput: 0: 1644.2, 1: 1676.5. Samples: 26150740. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:49:12,478][66916] Avg episode reward: [(0, '37.090'), (1, '55.620')] [2023-10-07 21:49:14,447][67838] Updated weights for policy 0, policy_version 51042 (0.0008) [2023-10-07 21:49:14,822][67838] Updated weights for policy 0, policy_version 51052 (0.0009) [2023-10-07 21:49:15,177][67871] Updated weights for policy 1, policy_version 51110 (0.0009) [2023-10-07 21:49:15,201][67838] Updated weights for policy 0, policy_version 51062 (0.0009) [2023-10-07 21:49:15,553][67871] Updated weights for policy 1, policy_version 51120 (0.0009) [2023-10-07 21:49:15,572][67838] Updated weights for policy 0, policy_version 51072 (0.0009) [2023-10-07 21:49:15,912][67871] Updated weights for policy 1, policy_version 51130 (0.0009) [2023-10-07 21:49:17,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 104660992. Throughput: 0: 1648.0, 1: 1662.0. Samples: 26169816. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:49:17,478][66916] Avg episode reward: [(0, '37.550'), (1, '53.820')] [2023-10-07 21:49:19,553][67838] Updated weights for policy 0, policy_version 51082 (0.0007) [2023-10-07 21:49:19,917][67838] Updated weights for policy 0, policy_version 51092 (0.0007) [2023-10-07 21:49:20,141][67871] Updated weights for policy 1, policy_version 51140 (0.0008) [2023-10-07 21:49:20,291][67838] Updated weights for policy 0, policy_version 51102 (0.0007) [2023-10-07 21:49:20,504][67871] Updated weights for policy 1, policy_version 51150 (0.0010) [2023-10-07 21:49:20,877][67871] Updated weights for policy 1, policy_version 51160 (0.0008) [2023-10-07 21:49:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 104726528. Throughput: 0: 1649.5, 1: 1673.8. Samples: 26190040. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:49:22,477][66916] Avg episode reward: [(0, '40.860'), (1, '54.160')] [2023-10-07 21:49:24,438][67838] Updated weights for policy 0, policy_version 51112 (0.0009) [2023-10-07 21:49:24,811][67838] Updated weights for policy 0, policy_version 51122 (0.0009) [2023-10-07 21:49:24,984][67871] Updated weights for policy 1, policy_version 51170 (0.0008) [2023-10-07 21:49:25,191][67838] Updated weights for policy 0, policy_version 51132 (0.0009) [2023-10-07 21:49:25,350][67871] Updated weights for policy 1, policy_version 51180 (0.0010) [2023-10-07 21:49:25,719][67871] Updated weights for policy 1, policy_version 51190 (0.0009) [2023-10-07 21:49:26,086][67871] Updated weights for policy 1, policy_version 51200 (0.0008) [2023-10-07 21:49:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 104792064. Throughput: 0: 1649.2, 1: 1680.8. Samples: 26200696. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 21:49:27,478][66916] Avg episode reward: [(0, '39.070'), (1, '52.590')] [2023-10-07 21:49:29,232][67838] Updated weights for policy 0, policy_version 51142 (0.0009) [2023-10-07 21:49:29,604][67838] Updated weights for policy 0, policy_version 51152 (0.0007) [2023-10-07 21:49:29,982][67838] Updated weights for policy 0, policy_version 51162 (0.0009) [2023-10-07 21:49:30,299][67871] Updated weights for policy 1, policy_version 51210 (0.0009) [2023-10-07 21:49:30,661][67871] Updated weights for policy 1, policy_version 51220 (0.0009) [2023-10-07 21:49:31,030][67871] Updated weights for policy 1, policy_version 51230 (0.0011) [2023-10-07 21:49:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 104857600. Throughput: 0: 1654.4, 1: 1658.9. Samples: 26219898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:49:32,478][66916] Avg episode reward: [(0, '42.490'), (1, '55.000')] [2023-10-07 21:49:34,063][67838] Updated weights for policy 0, policy_version 51172 (0.0010) [2023-10-07 21:49:34,436][67838] Updated weights for policy 0, policy_version 51182 (0.0011) [2023-10-07 21:49:34,810][67838] Updated weights for policy 0, policy_version 51192 (0.0009) [2023-10-07 21:49:35,052][67871] Updated weights for policy 1, policy_version 51240 (0.0009) [2023-10-07 21:49:35,414][67871] Updated weights for policy 1, policy_version 51250 (0.0008) [2023-10-07 21:49:35,784][67871] Updated weights for policy 1, policy_version 51260 (0.0007) [2023-10-07 21:49:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 104923136. Throughput: 0: 1658.0, 1: 1678.5. Samples: 26240346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:49:37,478][66916] Avg episode reward: [(0, '41.900'), (1, '50.830')] [2023-10-07 21:49:38,880][67838] Updated weights for policy 0, policy_version 51202 (0.0007) [2023-10-07 21:49:39,243][67838] Updated weights for policy 0, policy_version 51212 (0.0010) [2023-10-07 21:49:39,612][67838] Updated weights for policy 0, policy_version 51222 (0.0007) [2023-10-07 21:49:39,949][67871] Updated weights for policy 1, policy_version 51270 (0.0008) [2023-10-07 21:49:39,989][67838] Updated weights for policy 0, policy_version 51232 (0.0010) [2023-10-07 21:49:40,340][67871] Updated weights for policy 1, policy_version 51280 (0.0009) [2023-10-07 21:49:40,710][67871] Updated weights for policy 1, policy_version 51290 (0.0007) [2023-10-07 21:49:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 104988672. Throughput: 0: 1649.9, 1: 1674.8. Samples: 26250412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:49:42,478][66916] Avg episode reward: [(0, '42.290'), (1, '53.760')] [2023-10-07 21:49:44,081][67838] Updated weights for policy 0, policy_version 51242 (0.0009) [2023-10-07 21:49:44,438][67838] Updated weights for policy 0, policy_version 51252 (0.0010) [2023-10-07 21:49:44,647][67871] Updated weights for policy 1, policy_version 51300 (0.0007) [2023-10-07 21:49:44,812][67838] Updated weights for policy 0, policy_version 51262 (0.0008) [2023-10-07 21:49:45,015][67871] Updated weights for policy 1, policy_version 51310 (0.0007) [2023-10-07 21:49:45,372][67871] Updated weights for policy 1, policy_version 51320 (0.0008) [2023-10-07 21:49:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105054208. Throughput: 0: 1664.8, 1: 1660.4. Samples: 26269798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:49:47,477][66916] Avg episode reward: [(0, '42.700'), (1, '55.470')] [2023-10-07 21:49:48,935][67838] Updated weights for policy 0, policy_version 51272 (0.0009) [2023-10-07 21:49:49,307][67838] Updated weights for policy 0, policy_version 51282 (0.0009) [2023-10-07 21:49:49,620][67871] Updated weights for policy 1, policy_version 51330 (0.0009) [2023-10-07 21:49:49,691][67838] Updated weights for policy 0, policy_version 51292 (0.0008) [2023-10-07 21:49:49,993][67871] Updated weights for policy 1, policy_version 51340 (0.0009) [2023-10-07 21:49:50,354][67871] Updated weights for policy 1, policy_version 51350 (0.0009) [2023-10-07 21:49:50,725][67871] Updated weights for policy 1, policy_version 51360 (0.0008) [2023-10-07 21:49:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105119744. Throughput: 0: 1675.9, 1: 1674.1. Samples: 26290422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:49:52,477][66916] Avg episode reward: [(0, '41.170'), (1, '53.650')] [2023-10-07 21:49:53,860][67838] Updated weights for policy 0, policy_version 51302 (0.0007) [2023-10-07 21:49:54,244][67838] Updated weights for policy 0, policy_version 51312 (0.0008) [2023-10-07 21:49:54,614][67838] Updated weights for policy 0, policy_version 51322 (0.0007) [2023-10-07 21:49:54,872][67871] Updated weights for policy 1, policy_version 51370 (0.0008) [2023-10-07 21:49:55,239][67871] Updated weights for policy 1, policy_version 51380 (0.0008) [2023-10-07 21:49:55,598][67871] Updated weights for policy 1, policy_version 51390 (0.0009) [2023-10-07 21:49:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105185280. Throughput: 0: 1658.5, 1: 1664.4. Samples: 26300270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:49:57,477][66916] Avg episode reward: [(0, '41.960'), (1, '54.670')] [2023-10-07 21:49:58,721][67838] Updated weights for policy 0, policy_version 51332 (0.0009) [2023-10-07 21:49:59,092][67838] Updated weights for policy 0, policy_version 51342 (0.0010) [2023-10-07 21:49:59,475][67838] Updated weights for policy 0, policy_version 51352 (0.0009) [2023-10-07 21:49:59,612][67871] Updated weights for policy 1, policy_version 51400 (0.0007) [2023-10-07 21:49:59,984][67871] Updated weights for policy 1, policy_version 51410 (0.0008) [2023-10-07 21:50:00,343][67871] Updated weights for policy 1, policy_version 51420 (0.0010) [2023-10-07 21:50:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105250816. Throughput: 0: 1671.9, 1: 1660.0. Samples: 26319750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:50:02,478][66916] Avg episode reward: [(0, '40.750'), (1, '53.430')] [2023-10-07 21:50:03,443][67838] Updated weights for policy 0, policy_version 51362 (0.0011) [2023-10-07 21:50:03,819][67838] Updated weights for policy 0, policy_version 51372 (0.0010) [2023-10-07 21:50:04,187][67838] Updated weights for policy 0, policy_version 51382 (0.0007) [2023-10-07 21:50:04,431][67871] Updated weights for policy 1, policy_version 51430 (0.0008) [2023-10-07 21:50:04,557][67838] Updated weights for policy 0, policy_version 51392 (0.0007) [2023-10-07 21:50:04,795][67871] Updated weights for policy 1, policy_version 51440 (0.0009) [2023-10-07 21:50:05,163][67871] Updated weights for policy 1, policy_version 51450 (0.0010) [2023-10-07 21:50:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105316352. Throughput: 0: 1668.0, 1: 1672.3. Samples: 26340354. Policy #0 lag: (min: 9.0, avg: 21.0, max: 41.0) [2023-10-07 21:50:07,478][66916] Avg episode reward: [(0, '40.000'), (1, '52.020')] [2023-10-07 21:50:08,774][67838] Updated weights for policy 0, policy_version 51402 (0.0009) [2023-10-07 21:50:09,151][67838] Updated weights for policy 0, policy_version 51412 (0.0008) [2023-10-07 21:50:09,335][67871] Updated weights for policy 1, policy_version 51460 (0.0009) [2023-10-07 21:50:09,519][67838] Updated weights for policy 0, policy_version 51422 (0.0008) [2023-10-07 21:50:09,709][67871] Updated weights for policy 1, policy_version 51470 (0.0007) [2023-10-07 21:50:10,067][67871] Updated weights for policy 1, policy_version 51480 (0.0010) [2023-10-07 21:50:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 105381888. Throughput: 0: 1659.0, 1: 1657.8. Samples: 26349950. Policy #0 lag: (min: 9.0, avg: 21.0, max: 41.0) [2023-10-07 21:50:12,478][66916] Avg episode reward: [(0, '41.640'), (1, '47.770')] [2023-10-07 21:50:13,536][67838] Updated weights for policy 0, policy_version 51432 (0.0011) [2023-10-07 21:50:13,905][67838] Updated weights for policy 0, policy_version 51442 (0.0011) [2023-10-07 21:50:14,105][67871] Updated weights for policy 1, policy_version 51490 (0.0010) [2023-10-07 21:50:14,285][67838] Updated weights for policy 0, policy_version 51452 (0.0009) [2023-10-07 21:50:14,467][67871] Updated weights for policy 1, policy_version 51500 (0.0007) [2023-10-07 21:50:14,839][67871] Updated weights for policy 1, policy_version 51510 (0.0009) [2023-10-07 21:50:15,213][67871] Updated weights for policy 1, policy_version 51520 (0.0009) [2023-10-07 21:50:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105447424. Throughput: 0: 1668.8, 1: 1664.8. Samples: 26369914. Policy #0 lag: (min: 9.0, avg: 21.0, max: 41.0) [2023-10-07 21:50:17,478][66916] Avg episode reward: [(0, '43.430'), (1, '48.410')] [2023-10-07 21:50:18,362][67838] Updated weights for policy 0, policy_version 51462 (0.0008) [2023-10-07 21:50:18,725][67838] Updated weights for policy 0, policy_version 51472 (0.0009) [2023-10-07 21:50:19,098][67838] Updated weights for policy 0, policy_version 51482 (0.0008) [2023-10-07 21:50:19,434][67871] Updated weights for policy 1, policy_version 51530 (0.0008) [2023-10-07 21:50:19,798][67871] Updated weights for policy 1, policy_version 51540 (0.0009) [2023-10-07 21:50:20,169][67871] Updated weights for policy 1, policy_version 51550 (0.0008) [2023-10-07 21:50:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105512960. Throughput: 0: 1662.2, 1: 1671.1. Samples: 26390344. Policy #0 lag: (min: 9.0, avg: 21.0, max: 41.0) [2023-10-07 21:50:22,477][66916] Avg episode reward: [(0, '40.410'), (1, '48.080')] [2023-10-07 21:50:23,225][67838] Updated weights for policy 0, policy_version 51492 (0.0008) [2023-10-07 21:50:23,596][67838] Updated weights for policy 0, policy_version 51502 (0.0007) [2023-10-07 21:50:23,966][67838] Updated weights for policy 0, policy_version 51512 (0.0007) [2023-10-07 21:50:24,390][67871] Updated weights for policy 1, policy_version 51560 (0.0007) [2023-10-07 21:50:24,767][67871] Updated weights for policy 1, policy_version 51570 (0.0010) [2023-10-07 21:50:25,134][67871] Updated weights for policy 1, policy_version 51580 (0.0011) [2023-10-07 21:50:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 105578496. Throughput: 0: 1665.5, 1: 1658.0. Samples: 26399966. Policy #0 lag: (min: 9.0, avg: 21.0, max: 41.0) [2023-10-07 21:50:27,478][66916] Avg episode reward: [(0, '40.930'), (1, '46.990')] [2023-10-07 21:50:28,145][67838] Updated weights for policy 0, policy_version 51522 (0.0009) [2023-10-07 21:50:28,510][67838] Updated weights for policy 0, policy_version 51532 (0.0011) [2023-10-07 21:50:28,894][67838] Updated weights for policy 0, policy_version 51542 (0.0009) [2023-10-07 21:50:29,258][67838] Updated weights for policy 0, policy_version 51552 (0.0008) [2023-10-07 21:50:29,527][67871] Updated weights for policy 1, policy_version 51590 (0.0011) [2023-10-07 21:50:29,892][67871] Updated weights for policy 1, policy_version 51600 (0.0008) [2023-10-07 21:50:30,267][67871] Updated weights for policy 1, policy_version 51610 (0.0008) [2023-10-07 21:50:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105644032. Throughput: 0: 1661.9, 1: 1663.0. Samples: 26419418. Policy #0 lag: (min: 9.0, avg: 21.0, max: 41.0) [2023-10-07 21:50:32,477][66916] Avg episode reward: [(0, '39.690'), (1, '49.360')] [2023-10-07 21:50:33,436][67838] Updated weights for policy 0, policy_version 51562 (0.0008) [2023-10-07 21:50:33,804][67838] Updated weights for policy 0, policy_version 51572 (0.0007) [2023-10-07 21:50:34,177][67838] Updated weights for policy 0, policy_version 51582 (0.0009) [2023-10-07 21:50:34,261][67871] Updated weights for policy 1, policy_version 51620 (0.0007) [2023-10-07 21:50:34,628][67871] Updated weights for policy 1, policy_version 51630 (0.0010) [2023-10-07 21:50:34,999][67871] Updated weights for policy 1, policy_version 51640 (0.0009) [2023-10-07 21:50:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105709568. Throughput: 0: 1654.5, 1: 1666.5. Samples: 26439868. Policy #0 lag: (min: 9.0, avg: 21.0, max: 41.0) [2023-10-07 21:50:37,477][66916] Avg episode reward: [(0, '39.210'), (1, '46.830')] [2023-10-07 21:50:38,382][67838] Updated weights for policy 0, policy_version 51592 (0.0008) [2023-10-07 21:50:38,763][67838] Updated weights for policy 0, policy_version 51602 (0.0007) [2023-10-07 21:50:38,955][67871] Updated weights for policy 1, policy_version 51650 (0.0010) [2023-10-07 21:50:39,129][67838] Updated weights for policy 0, policy_version 51612 (0.0009) [2023-10-07 21:50:39,322][67871] Updated weights for policy 1, policy_version 51660 (0.0007) [2023-10-07 21:50:39,695][67871] Updated weights for policy 1, policy_version 51670 (0.0009) [2023-10-07 21:50:40,058][67871] Updated weights for policy 1, policy_version 51680 (0.0010) [2023-10-07 21:50:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105775104. Throughput: 0: 1655.2, 1: 1651.2. Samples: 26449058. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-07 21:50:42,477][66916] Avg episode reward: [(0, '42.290'), (1, '49.080')] [2023-10-07 21:50:43,343][67838] Updated weights for policy 0, policy_version 51622 (0.0009) [2023-10-07 21:50:43,724][67838] Updated weights for policy 0, policy_version 51632 (0.0011) [2023-10-07 21:50:44,094][67838] Updated weights for policy 0, policy_version 51642 (0.0009) [2023-10-07 21:50:44,144][67871] Updated weights for policy 1, policy_version 51690 (0.0008) [2023-10-07 21:50:44,508][67871] Updated weights for policy 1, policy_version 51700 (0.0008) [2023-10-07 21:50:44,867][67871] Updated weights for policy 1, policy_version 51710 (0.0007) [2023-10-07 21:50:47,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105840640. Throughput: 0: 1649.2, 1: 1669.7. Samples: 26469100. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-07 21:50:47,477][66916] Avg episode reward: [(0, '40.520'), (1, '45.800')] [2023-10-07 21:50:48,497][67838] Updated weights for policy 0, policy_version 51652 (0.0008) [2023-10-07 21:50:48,889][67838] Updated weights for policy 0, policy_version 51662 (0.0009) [2023-10-07 21:50:48,927][67871] Updated weights for policy 1, policy_version 51720 (0.0008) [2023-10-07 21:50:49,259][67838] Updated weights for policy 0, policy_version 51672 (0.0009) [2023-10-07 21:50:49,303][67871] Updated weights for policy 1, policy_version 51730 (0.0009) [2023-10-07 21:50:49,666][67871] Updated weights for policy 1, policy_version 51740 (0.0007) [2023-10-07 21:50:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 105906176. Throughput: 0: 1642.8, 1: 1662.1. Samples: 26489076. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-07 21:50:52,478][66916] Avg episode reward: [(0, '41.540'), (1, '52.450')] [2023-10-07 21:50:53,438][67838] Updated weights for policy 0, policy_version 51682 (0.0008) [2023-10-07 21:50:53,802][67838] Updated weights for policy 0, policy_version 51692 (0.0009) [2023-10-07 21:50:53,827][67871] Updated weights for policy 1, policy_version 51750 (0.0007) [2023-10-07 21:50:54,174][67838] Updated weights for policy 0, policy_version 51702 (0.0009) [2023-10-07 21:50:54,199][67871] Updated weights for policy 1, policy_version 51760 (0.0008) [2023-10-07 21:50:54,543][67838] Updated weights for policy 0, policy_version 51712 (0.0011) [2023-10-07 21:50:54,574][67871] Updated weights for policy 1, policy_version 51770 (0.0008) [2023-10-07 21:50:57,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 105971712. Throughput: 0: 1642.5, 1: 1652.0. Samples: 26498202. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-07 21:50:57,478][66916] Avg episode reward: [(0, '39.040'), (1, '50.020')] [2023-10-07 21:50:58,484][67871] Updated weights for policy 1, policy_version 51780 (0.0009) [2023-10-07 21:50:58,743][67838] Updated weights for policy 0, policy_version 51722 (0.0009) [2023-10-07 21:50:58,848][67871] Updated weights for policy 1, policy_version 51790 (0.0008) [2023-10-07 21:50:59,119][67838] Updated weights for policy 0, policy_version 51732 (0.0009) [2023-10-07 21:50:59,218][67871] Updated weights for policy 1, policy_version 51800 (0.0008) [2023-10-07 21:50:59,476][67838] Updated weights for policy 0, policy_version 51742 (0.0009) [2023-10-07 21:51:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 106037248. Throughput: 0: 1635.2, 1: 1669.2. Samples: 26518610. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-07 21:51:02,477][66916] Avg episode reward: [(0, '39.140'), (1, '55.230')] [2023-10-07 21:51:03,344][67871] Updated weights for policy 1, policy_version 51810 (0.0007) [2023-10-07 21:51:03,711][67871] Updated weights for policy 1, policy_version 51820 (0.0008) [2023-10-07 21:51:03,743][67838] Updated weights for policy 0, policy_version 51752 (0.0009) [2023-10-07 21:51:04,079][67871] Updated weights for policy 1, policy_version 51830 (0.0009) [2023-10-07 21:51:04,108][67838] Updated weights for policy 0, policy_version 51762 (0.0009) [2023-10-07 21:51:04,444][67871] Updated weights for policy 1, policy_version 51840 (0.0009) [2023-10-07 21:51:04,477][67838] Updated weights for policy 0, policy_version 51772 (0.0007) [2023-10-07 21:51:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 106102784. Throughput: 0: 1636.3, 1: 1662.4. Samples: 26538784. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-07 21:51:07,478][66916] Avg episode reward: [(0, '41.960'), (1, '54.470')] [2023-10-07 21:51:07,488][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000051840_53084160.pth... [2023-10-07 21:51:07,488][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000051776_53018624.pth... [2023-10-07 21:51:07,525][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000050304_51511296.pth [2023-10-07 21:51:07,527][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000050240_51445760.pth [2023-10-07 21:51:08,583][67871] Updated weights for policy 1, policy_version 51850 (0.0007) [2023-10-07 21:51:08,643][67838] Updated weights for policy 0, policy_version 51782 (0.0008) [2023-10-07 21:51:08,946][67871] Updated weights for policy 1, policy_version 51860 (0.0007) [2023-10-07 21:51:09,011][67838] Updated weights for policy 0, policy_version 51792 (0.0010) [2023-10-07 21:51:09,315][67871] Updated weights for policy 1, policy_version 51870 (0.0007) [2023-10-07 21:51:09,387][67838] Updated weights for policy 0, policy_version 51802 (0.0010) [2023-10-07 21:51:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 106168320. Throughput: 0: 1632.8, 1: 1652.0. Samples: 26547784. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-07 21:51:12,477][66916] Avg episode reward: [(0, '41.040'), (1, '54.930')] [2023-10-07 21:51:13,498][67838] Updated weights for policy 0, policy_version 51812 (0.0007) [2023-10-07 21:51:13,556][67871] Updated weights for policy 1, policy_version 51880 (0.0008) [2023-10-07 21:51:13,862][67838] Updated weights for policy 0, policy_version 51822 (0.0008) [2023-10-07 21:51:13,924][67871] Updated weights for policy 1, policy_version 51890 (0.0008) [2023-10-07 21:51:14,235][67838] Updated weights for policy 0, policy_version 51832 (0.0009) [2023-10-07 21:51:14,297][67871] Updated weights for policy 1, policy_version 51900 (0.0008) [2023-10-07 21:51:17,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 106233856. Throughput: 0: 1635.4, 1: 1664.9. Samples: 26567932. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-07 21:51:17,477][66916] Avg episode reward: [(0, '41.380'), (1, '52.420')] [2023-10-07 21:51:18,284][67838] Updated weights for policy 0, policy_version 51842 (0.0008) [2023-10-07 21:51:18,588][67871] Updated weights for policy 1, policy_version 51910 (0.0007) [2023-10-07 21:51:18,652][67838] Updated weights for policy 0, policy_version 51852 (0.0007) [2023-10-07 21:51:18,984][67871] Updated weights for policy 1, policy_version 51920 (0.0009) [2023-10-07 21:51:19,026][67838] Updated weights for policy 0, policy_version 51862 (0.0009) [2023-10-07 21:51:19,344][67871] Updated weights for policy 1, policy_version 51930 (0.0008) [2023-10-07 21:51:19,396][67838] Updated weights for policy 0, policy_version 51872 (0.0009) [2023-10-07 21:51:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 106299392. Throughput: 0: 1639.2, 1: 1657.3. Samples: 26588212. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-07 21:51:22,477][66916] Avg episode reward: [(0, '45.000'), (1, '55.640')] [2023-10-07 21:51:23,459][67871] Updated weights for policy 1, policy_version 51940 (0.0010) [2023-10-07 21:51:23,604][67838] Updated weights for policy 0, policy_version 51882 (0.0009) [2023-10-07 21:51:23,832][67871] Updated weights for policy 1, policy_version 51950 (0.0008) [2023-10-07 21:51:23,977][67838] Updated weights for policy 0, policy_version 51892 (0.0010) [2023-10-07 21:51:24,198][67871] Updated weights for policy 1, policy_version 51960 (0.0008) [2023-10-07 21:51:24,354][67838] Updated weights for policy 0, policy_version 51902 (0.0009) [2023-10-07 21:51:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 106364928. Throughput: 0: 1644.0, 1: 1651.6. Samples: 26597358. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-07 21:51:27,477][66916] Avg episode reward: [(0, '38.800'), (1, '52.520')] [2023-10-07 21:51:28,230][67871] Updated weights for policy 1, policy_version 51970 (0.0008) [2023-10-07 21:51:28,251][67838] Updated weights for policy 0, policy_version 51912 (0.0009) [2023-10-07 21:51:28,599][67871] Updated weights for policy 1, policy_version 51980 (0.0007) [2023-10-07 21:51:28,624][67838] Updated weights for policy 0, policy_version 51922 (0.0008) [2023-10-07 21:51:28,964][67871] Updated weights for policy 1, policy_version 51990 (0.0008) [2023-10-07 21:51:29,002][67838] Updated weights for policy 0, policy_version 51932 (0.0008) [2023-10-07 21:51:29,333][67871] Updated weights for policy 1, policy_version 52000 (0.0009) [2023-10-07 21:51:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 106430464. Throughput: 0: 1650.7, 1: 1656.4. Samples: 26617916. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-07 21:51:32,477][66916] Avg episode reward: [(0, '41.700'), (1, '54.750')] [2023-10-07 21:51:33,285][67838] Updated weights for policy 0, policy_version 51942 (0.0009) [2023-10-07 21:51:33,657][67871] Updated weights for policy 1, policy_version 52010 (0.0008) [2023-10-07 21:51:33,669][67838] Updated weights for policy 0, policy_version 51952 (0.0010) [2023-10-07 21:51:34,025][67871] Updated weights for policy 1, policy_version 52020 (0.0009) [2023-10-07 21:51:34,036][67838] Updated weights for policy 0, policy_version 51962 (0.0008) [2023-10-07 21:51:34,400][67871] Updated weights for policy 1, policy_version 52030 (0.0008) [2023-10-07 21:51:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 106496000. Throughput: 0: 1654.4, 1: 1657.3. Samples: 26638100. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-07 21:51:37,477][66916] Avg episode reward: [(0, '40.210'), (1, '54.400')] [2023-10-07 21:51:38,141][67838] Updated weights for policy 0, policy_version 51972 (0.0008) [2023-10-07 21:51:38,503][67871] Updated weights for policy 1, policy_version 52040 (0.0008) [2023-10-07 21:51:38,506][67838] Updated weights for policy 0, policy_version 51982 (0.0009) [2023-10-07 21:51:38,877][67871] Updated weights for policy 1, policy_version 52050 (0.0007) [2023-10-07 21:51:38,887][67838] Updated weights for policy 0, policy_version 51992 (0.0009) [2023-10-07 21:51:39,241][67871] Updated weights for policy 1, policy_version 52060 (0.0008) [2023-10-07 21:51:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 106561536. Throughput: 0: 1657.1, 1: 1653.0. Samples: 26647158. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-07 21:51:42,478][66916] Avg episode reward: [(0, '41.180'), (1, '55.510')] [2023-10-07 21:51:42,972][67838] Updated weights for policy 0, policy_version 52002 (0.0007) [2023-10-07 21:51:43,340][67838] Updated weights for policy 0, policy_version 52012 (0.0009) [2023-10-07 21:51:43,415][67871] Updated weights for policy 1, policy_version 52070 (0.0007) [2023-10-07 21:51:43,720][67838] Updated weights for policy 0, policy_version 52022 (0.0009) [2023-10-07 21:51:43,787][67871] Updated weights for policy 1, policy_version 52080 (0.0008) [2023-10-07 21:51:44,085][67838] Updated weights for policy 0, policy_version 52032 (0.0008) [2023-10-07 21:51:44,143][67871] Updated weights for policy 1, policy_version 52090 (0.0008) [2023-10-07 21:51:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 106627072. Throughput: 0: 1660.6, 1: 1649.1. Samples: 26667546. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-07 21:51:47,477][66916] Avg episode reward: [(0, '45.400'), (1, '56.790')] [2023-10-07 21:51:48,040][67871] Updated weights for policy 1, policy_version 52100 (0.0007) [2023-10-07 21:51:48,262][67838] Updated weights for policy 0, policy_version 52042 (0.0009) [2023-10-07 21:51:48,412][67871] Updated weights for policy 1, policy_version 52110 (0.0008) [2023-10-07 21:51:48,634][67838] Updated weights for policy 0, policy_version 52052 (0.0007) [2023-10-07 21:51:48,780][67871] Updated weights for policy 1, policy_version 52120 (0.0009) [2023-10-07 21:51:49,011][67838] Updated weights for policy 0, policy_version 52062 (0.0007) [2023-10-07 21:51:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 106692608. Throughput: 0: 1657.7, 1: 1658.2. Samples: 26687996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:51:52,477][66916] Avg episode reward: [(0, '42.260'), (1, '57.870')] [2023-10-07 21:51:52,921][67871] Updated weights for policy 1, policy_version 52130 (0.0008) [2023-10-07 21:51:53,074][67838] Updated weights for policy 0, policy_version 52072 (0.0009) [2023-10-07 21:51:53,287][67871] Updated weights for policy 1, policy_version 52140 (0.0008) [2023-10-07 21:51:53,452][67838] Updated weights for policy 0, policy_version 52082 (0.0007) [2023-10-07 21:51:53,648][67871] Updated weights for policy 1, policy_version 52150 (0.0009) [2023-10-07 21:51:53,820][67838] Updated weights for policy 0, policy_version 52092 (0.0010) [2023-10-07 21:51:54,027][67871] Updated weights for policy 1, policy_version 52160 (0.0009) [2023-10-07 21:51:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 106758144. Throughput: 0: 1657.7, 1: 1656.8. Samples: 26696936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:51:57,477][66916] Avg episode reward: [(0, '46.390'), (1, '53.530')] [2023-10-07 21:51:57,932][67838] Updated weights for policy 0, policy_version 52102 (0.0007) [2023-10-07 21:51:57,999][67871] Updated weights for policy 1, policy_version 52170 (0.0007) [2023-10-07 21:51:58,313][67838] Updated weights for policy 0, policy_version 52112 (0.0007) [2023-10-07 21:51:58,369][67871] Updated weights for policy 1, policy_version 52180 (0.0007) [2023-10-07 21:51:58,680][67838] Updated weights for policy 0, policy_version 52122 (0.0007) [2023-10-07 21:51:58,725][67871] Updated weights for policy 1, policy_version 52190 (0.0009) [2023-10-07 21:52:02,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 106823680. Throughput: 0: 1656.7, 1: 1665.1. Samples: 26717412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:52:02,477][66916] Avg episode reward: [(0, '47.150'), (1, '58.930')] [2023-10-07 21:52:02,841][67838] Updated weights for policy 0, policy_version 52132 (0.0010) [2023-10-07 21:52:02,982][67871] Updated weights for policy 1, policy_version 52200 (0.0008) [2023-10-07 21:52:03,208][67838] Updated weights for policy 0, policy_version 52142 (0.0009) [2023-10-07 21:52:03,335][67871] Updated weights for policy 1, policy_version 52210 (0.0007) [2023-10-07 21:52:03,585][67838] Updated weights for policy 0, policy_version 52152 (0.0008) [2023-10-07 21:52:03,703][67871] Updated weights for policy 1, policy_version 52220 (0.0008) [2023-10-07 21:52:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 106889216. Throughput: 0: 1657.3, 1: 1672.1. Samples: 26738036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:52:07,478][66916] Avg episode reward: [(0, '46.720'), (1, '57.230')] [2023-10-07 21:52:07,830][67838] Updated weights for policy 0, policy_version 52162 (0.0010) [2023-10-07 21:52:07,968][67871] Updated weights for policy 1, policy_version 52230 (0.0008) [2023-10-07 21:52:08,201][67838] Updated weights for policy 0, policy_version 52172 (0.0009) [2023-10-07 21:52:08,348][67871] Updated weights for policy 1, policy_version 52240 (0.0008) [2023-10-07 21:52:08,564][67838] Updated weights for policy 0, policy_version 52182 (0.0010) [2023-10-07 21:52:08,718][67871] Updated weights for policy 1, policy_version 52250 (0.0009) [2023-10-07 21:52:08,932][67838] Updated weights for policy 0, policy_version 52192 (0.0008) [2023-10-07 21:52:12,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 106954752. Throughput: 0: 1655.6, 1: 1666.9. Samples: 26746870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:52:12,478][66916] Avg episode reward: [(0, '47.990'), (1, '55.360')] [2023-10-07 21:52:12,910][67871] Updated weights for policy 1, policy_version 52260 (0.0010) [2023-10-07 21:52:13,136][67838] Updated weights for policy 0, policy_version 52202 (0.0007) [2023-10-07 21:52:13,276][67871] Updated weights for policy 1, policy_version 52270 (0.0009) [2023-10-07 21:52:13,511][67838] Updated weights for policy 0, policy_version 52212 (0.0008) [2023-10-07 21:52:13,639][67871] Updated weights for policy 1, policy_version 52280 (0.0009) [2023-10-07 21:52:13,879][67838] Updated weights for policy 0, policy_version 52222 (0.0008) [2023-10-07 21:52:17,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 107020288. Throughput: 0: 1657.4, 1: 1668.7. Samples: 26767592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:52:17,477][66916] Avg episode reward: [(0, '49.270'), (1, '55.010')] [2023-10-07 21:52:17,712][67871] Updated weights for policy 1, policy_version 52290 (0.0009) [2023-10-07 21:52:17,883][67838] Updated weights for policy 0, policy_version 52232 (0.0009) [2023-10-07 21:52:18,066][67871] Updated weights for policy 1, policy_version 52300 (0.0008) [2023-10-07 21:52:18,261][67838] Updated weights for policy 0, policy_version 52242 (0.0007) [2023-10-07 21:52:18,433][67871] Updated weights for policy 1, policy_version 52310 (0.0010) [2023-10-07 21:52:18,626][67838] Updated weights for policy 0, policy_version 52252 (0.0007) [2023-10-07 21:52:18,792][67871] Updated weights for policy 1, policy_version 52320 (0.0008) [2023-10-07 21:52:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 107085824. Throughput: 0: 1664.8, 1: 1669.5. Samples: 26788140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:52:22,477][66916] Avg episode reward: [(0, '47.770'), (1, '57.640')] [2023-10-07 21:52:22,791][67871] Updated weights for policy 1, policy_version 52330 (0.0007) [2023-10-07 21:52:22,815][67838] Updated weights for policy 0, policy_version 52262 (0.0010) [2023-10-07 21:52:23,159][67871] Updated weights for policy 1, policy_version 52340 (0.0007) [2023-10-07 21:52:23,195][67838] Updated weights for policy 0, policy_version 52272 (0.0007) [2023-10-07 21:52:23,524][67871] Updated weights for policy 1, policy_version 52350 (0.0009) [2023-10-07 21:52:23,562][67838] Updated weights for policy 0, policy_version 52282 (0.0007) [2023-10-07 21:52:27,470][67871] Updated weights for policy 1, policy_version 52360 (0.0007) [2023-10-07 21:52:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 107151360. Throughput: 0: 1660.4, 1: 1667.2. Samples: 26796898. Policy #0 lag: (min: 20.0, avg: 27.9, max: 52.0) [2023-10-07 21:52:27,477][66916] Avg episode reward: [(0, '48.870'), (1, '53.130')] [2023-10-07 21:52:27,642][67838] Updated weights for policy 0, policy_version 52292 (0.0010) [2023-10-07 21:52:27,840][67871] Updated weights for policy 1, policy_version 52370 (0.0007) [2023-10-07 21:52:28,018][67838] Updated weights for policy 0, policy_version 52302 (0.0008) [2023-10-07 21:52:28,209][67871] Updated weights for policy 1, policy_version 52380 (0.0007) [2023-10-07 21:52:28,380][67838] Updated weights for policy 0, policy_version 52312 (0.0008) [2023-10-07 21:52:32,336][67838] Updated weights for policy 0, policy_version 52322 (0.0009) [2023-10-07 21:52:32,390][67871] Updated weights for policy 1, policy_version 52390 (0.0008) [2023-10-07 21:52:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 107216896. Throughput: 0: 1660.1, 1: 1671.7. Samples: 26817478. Policy #0 lag: (min: 20.0, avg: 27.9, max: 52.0) [2023-10-07 21:52:32,477][66916] Avg episode reward: [(0, '48.590'), (1, '55.290')] [2023-10-07 21:52:32,710][67838] Updated weights for policy 0, policy_version 52332 (0.0008) [2023-10-07 21:52:32,753][67871] Updated weights for policy 1, policy_version 52400 (0.0008) [2023-10-07 21:52:33,082][67838] Updated weights for policy 0, policy_version 52342 (0.0007) [2023-10-07 21:52:33,119][67871] Updated weights for policy 1, policy_version 52410 (0.0007) [2023-10-07 21:52:33,451][67838] Updated weights for policy 0, policy_version 52352 (0.0007) [2023-10-07 21:52:37,287][67871] Updated weights for policy 1, policy_version 52420 (0.0008) [2023-10-07 21:52:37,457][67838] Updated weights for policy 0, policy_version 52362 (0.0007) [2023-10-07 21:52:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 107282432. Throughput: 0: 1671.9, 1: 1662.7. Samples: 26838052. Policy #0 lag: (min: 20.0, avg: 27.9, max: 52.0) [2023-10-07 21:52:37,477][66916] Avg episode reward: [(0, '46.660'), (1, '51.820')] [2023-10-07 21:52:37,651][67871] Updated weights for policy 1, policy_version 52430 (0.0007) [2023-10-07 21:52:37,827][67838] Updated weights for policy 0, policy_version 52372 (0.0009) [2023-10-07 21:52:38,013][67871] Updated weights for policy 1, policy_version 52440 (0.0008) [2023-10-07 21:52:38,205][67838] Updated weights for policy 0, policy_version 52382 (0.0010) [2023-10-07 21:52:42,161][67871] Updated weights for policy 1, policy_version 52450 (0.0007) [2023-10-07 21:52:42,376][67838] Updated weights for policy 0, policy_version 52392 (0.0008) [2023-10-07 21:52:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 107347968. Throughput: 0: 1670.9, 1: 1663.0. Samples: 26846962. Policy #0 lag: (min: 20.0, avg: 27.9, max: 52.0) [2023-10-07 21:52:42,478][66916] Avg episode reward: [(0, '46.200'), (1, '50.220')] [2023-10-07 21:52:42,543][67871] Updated weights for policy 1, policy_version 52460 (0.0007) [2023-10-07 21:52:42,752][67838] Updated weights for policy 0, policy_version 52402 (0.0007) [2023-10-07 21:52:42,913][67871] Updated weights for policy 1, policy_version 52470 (0.0007) [2023-10-07 21:52:43,124][67838] Updated weights for policy 0, policy_version 52412 (0.0008) [2023-10-07 21:52:43,277][67871] Updated weights for policy 1, policy_version 52480 (0.0007) [2023-10-07 21:52:47,372][67838] Updated weights for policy 0, policy_version 52422 (0.0010) [2023-10-07 21:52:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 107413504. Throughput: 0: 1666.6, 1: 1659.2. Samples: 26867070. Policy #0 lag: (min: 20.0, avg: 27.9, max: 52.0) [2023-10-07 21:52:47,478][66916] Avg episode reward: [(0, '47.150'), (1, '50.940')] [2023-10-07 21:52:47,544][67871] Updated weights for policy 1, policy_version 52490 (0.0009) [2023-10-07 21:52:47,752][67838] Updated weights for policy 0, policy_version 52432 (0.0009) [2023-10-07 21:52:47,916][67871] Updated weights for policy 1, policy_version 52500 (0.0008) [2023-10-07 21:52:48,134][67838] Updated weights for policy 0, policy_version 52442 (0.0008) [2023-10-07 21:52:48,278][67871] Updated weights for policy 1, policy_version 52510 (0.0008) [2023-10-07 21:52:52,198][67871] Updated weights for policy 1, policy_version 52520 (0.0009) [2023-10-07 21:52:52,338][67838] Updated weights for policy 0, policy_version 52452 (0.0008) [2023-10-07 21:52:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107479040. Throughput: 0: 1663.6, 1: 1660.4. Samples: 26887612. Policy #0 lag: (min: 20.0, avg: 27.9, max: 52.0) [2023-10-07 21:52:52,477][66916] Avg episode reward: [(0, '47.990'), (1, '50.130')] [2023-10-07 21:52:52,574][67871] Updated weights for policy 1, policy_version 52530 (0.0007) [2023-10-07 21:52:52,703][67838] Updated weights for policy 0, policy_version 52462 (0.0007) [2023-10-07 21:52:52,932][67871] Updated weights for policy 1, policy_version 52540 (0.0007) [2023-10-07 21:52:53,080][67838] Updated weights for policy 0, policy_version 52472 (0.0007) [2023-10-07 21:52:57,179][67838] Updated weights for policy 0, policy_version 52482 (0.0007) [2023-10-07 21:52:57,190][67871] Updated weights for policy 1, policy_version 52550 (0.0008) [2023-10-07 21:52:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107544576. Throughput: 0: 1661.7, 1: 1662.5. Samples: 26896458. Policy #0 lag: (min: 20.0, avg: 27.9, max: 52.0) [2023-10-07 21:52:57,477][66916] Avg episode reward: [(0, '45.540'), (1, '49.670')] [2023-10-07 21:52:57,542][67838] Updated weights for policy 0, policy_version 52492 (0.0008) [2023-10-07 21:52:57,558][67871] Updated weights for policy 1, policy_version 52560 (0.0007) [2023-10-07 21:52:57,912][67838] Updated weights for policy 0, policy_version 52502 (0.0008) [2023-10-07 21:52:57,926][67871] Updated weights for policy 1, policy_version 52570 (0.0008) [2023-10-07 21:52:58,288][67838] Updated weights for policy 0, policy_version 52512 (0.0010) [2023-10-07 21:53:02,133][67871] Updated weights for policy 1, policy_version 52580 (0.0009) [2023-10-07 21:53:02,454][67838] Updated weights for policy 0, policy_version 52522 (0.0008) [2023-10-07 21:53:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107610112. Throughput: 0: 1660.0, 1: 1660.6. Samples: 26917022. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:53:02,477][66916] Avg episode reward: [(0, '53.150'), (1, '50.000')] [2023-10-07 21:53:02,501][67871] Updated weights for policy 1, policy_version 52590 (0.0009) [2023-10-07 21:53:02,821][67838] Updated weights for policy 0, policy_version 52532 (0.0007) [2023-10-07 21:53:02,868][67871] Updated weights for policy 1, policy_version 52600 (0.0009) [2023-10-07 21:53:03,195][67838] Updated weights for policy 0, policy_version 52542 (0.0008) [2023-10-07 21:53:07,046][67871] Updated weights for policy 1, policy_version 52610 (0.0007) [2023-10-07 21:53:07,413][67871] Updated weights for policy 1, policy_version 52620 (0.0008) [2023-10-07 21:53:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107675648. Throughput: 0: 1651.7, 1: 1658.2. Samples: 26937084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:53:07,477][66916] Avg episode reward: [(0, '50.090'), (1, '49.960')] [2023-10-07 21:53:07,527][67838] Updated weights for policy 0, policy_version 52552 (0.0009) [2023-10-07 21:53:07,777][67871] Updated weights for policy 1, policy_version 52630 (0.0008) [2023-10-07 21:53:07,897][67838] Updated weights for policy 0, policy_version 52562 (0.0007) [2023-10-07 21:53:08,152][67871] Updated weights for policy 1, policy_version 52640 (0.0008) [2023-10-07 21:53:08,152][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000052640_53903360.pth... [2023-10-07 21:53:08,181][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000051072_52297728.pth [2023-10-07 21:53:08,272][67838] Updated weights for policy 0, policy_version 52572 (0.0008) [2023-10-07 21:53:08,418][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000052576_53837824.pth... [2023-10-07 21:53:08,446][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000051008_52232192.pth [2023-10-07 21:53:12,421][67838] Updated weights for policy 0, policy_version 52582 (0.0009) [2023-10-07 21:53:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 107741184. Throughput: 0: 1653.1, 1: 1658.3. Samples: 26945910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:53:12,477][66916] Avg episode reward: [(0, '50.260'), (1, '49.850')] [2023-10-07 21:53:12,495][67871] Updated weights for policy 1, policy_version 52650 (0.0009) [2023-10-07 21:53:12,789][67838] Updated weights for policy 0, policy_version 52592 (0.0008) [2023-10-07 21:53:12,867][67871] Updated weights for policy 1, policy_version 52660 (0.0008) [2023-10-07 21:53:13,166][67838] Updated weights for policy 0, policy_version 52602 (0.0007) [2023-10-07 21:53:13,247][67871] Updated weights for policy 1, policy_version 52670 (0.0009) [2023-10-07 21:53:17,093][67871] Updated weights for policy 1, policy_version 52680 (0.0010) [2023-10-07 21:53:17,396][67838] Updated weights for policy 0, policy_version 52612 (0.0007) [2023-10-07 21:53:17,460][67871] Updated weights for policy 1, policy_version 52690 (0.0008) [2023-10-07 21:53:17,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107806720. Throughput: 0: 1650.5, 1: 1656.1. Samples: 26966276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:53:17,477][66916] Avg episode reward: [(0, '53.440'), (1, '50.200')] [2023-10-07 21:53:17,772][67838] Updated weights for policy 0, policy_version 52622 (0.0007) [2023-10-07 21:53:17,839][67871] Updated weights for policy 1, policy_version 52700 (0.0007) [2023-10-07 21:53:18,146][67838] Updated weights for policy 0, policy_version 52632 (0.0009) [2023-10-07 21:53:22,016][67871] Updated weights for policy 1, policy_version 52710 (0.0009) [2023-10-07 21:53:22,201][67838] Updated weights for policy 0, policy_version 52642 (0.0009) [2023-10-07 21:53:22,381][67871] Updated weights for policy 1, policy_version 52720 (0.0008) [2023-10-07 21:53:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 107872256. Throughput: 0: 1644.0, 1: 1659.0. Samples: 26986684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:53:22,477][66916] Avg episode reward: [(0, '50.530'), (1, '54.620')] [2023-10-07 21:53:22,577][67838] Updated weights for policy 0, policy_version 52652 (0.0008) [2023-10-07 21:53:22,751][67871] Updated weights for policy 1, policy_version 52730 (0.0007) [2023-10-07 21:53:22,949][67838] Updated weights for policy 0, policy_version 52662 (0.0009) [2023-10-07 21:53:23,318][67838] Updated weights for policy 0, policy_version 52672 (0.0010) [2023-10-07 21:53:26,923][67871] Updated weights for policy 1, policy_version 52740 (0.0009) [2023-10-07 21:53:27,294][67871] Updated weights for policy 1, policy_version 52750 (0.0007) [2023-10-07 21:53:27,389][67838] Updated weights for policy 0, policy_version 52682 (0.0008) [2023-10-07 21:53:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107937792. Throughput: 0: 1643.1, 1: 1662.1. Samples: 26995696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:53:27,477][66916] Avg episode reward: [(0, '51.440'), (1, '53.530')] [2023-10-07 21:53:27,664][67871] Updated weights for policy 1, policy_version 52760 (0.0008) [2023-10-07 21:53:27,761][67838] Updated weights for policy 0, policy_version 52692 (0.0007) [2023-10-07 21:53:28,135][67838] Updated weights for policy 0, policy_version 52702 (0.0009) [2023-10-07 21:53:31,811][67871] Updated weights for policy 1, policy_version 52770 (0.0008) [2023-10-07 21:53:32,178][67871] Updated weights for policy 1, policy_version 52780 (0.0008) [2023-10-07 21:53:32,206][67838] Updated weights for policy 0, policy_version 52712 (0.0010) [2023-10-07 21:53:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108003328. Throughput: 0: 1649.1, 1: 1662.6. Samples: 27016098. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:53:32,477][66916] Avg episode reward: [(0, '50.100'), (1, '54.110')] [2023-10-07 21:53:32,544][67871] Updated weights for policy 1, policy_version 52790 (0.0008) [2023-10-07 21:53:32,571][67838] Updated weights for policy 0, policy_version 52722 (0.0010) [2023-10-07 21:53:32,913][67871] Updated weights for policy 1, policy_version 52800 (0.0008) [2023-10-07 21:53:32,942][67838] Updated weights for policy 0, policy_version 52732 (0.0009) [2023-10-07 21:53:37,028][67838] Updated weights for policy 0, policy_version 52742 (0.0009) [2023-10-07 21:53:37,061][67871] Updated weights for policy 1, policy_version 52810 (0.0012) [2023-10-07 21:53:37,390][67838] Updated weights for policy 0, policy_version 52752 (0.0008) [2023-10-07 21:53:37,440][67871] Updated weights for policy 1, policy_version 52820 (0.0007) [2023-10-07 21:53:37,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108068864. Throughput: 0: 1644.4, 1: 1654.2. Samples: 27036048. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-07 21:53:37,477][66916] Avg episode reward: [(0, '47.340'), (1, '55.060')] [2023-10-07 21:53:37,760][67838] Updated weights for policy 0, policy_version 52762 (0.0007) [2023-10-07 21:53:37,805][67871] Updated weights for policy 1, policy_version 52830 (0.0008) [2023-10-07 21:53:41,899][67838] Updated weights for policy 0, policy_version 52772 (0.0008) [2023-10-07 21:53:41,997][67871] Updated weights for policy 1, policy_version 52840 (0.0008) [2023-10-07 21:53:42,278][67838] Updated weights for policy 0, policy_version 52782 (0.0008) [2023-10-07 21:53:42,362][67871] Updated weights for policy 1, policy_version 52850 (0.0008) [2023-10-07 21:53:42,477][66916] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108134400. Throughput: 0: 1649.9, 1: 1658.8. Samples: 27045352. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-07 21:53:42,478][66916] Avg episode reward: [(0, '47.630'), (1, '53.560')] [2023-10-07 21:53:42,646][67838] Updated weights for policy 0, policy_version 52792 (0.0008) [2023-10-07 21:53:42,725][67871] Updated weights for policy 1, policy_version 52860 (0.0009) [2023-10-07 21:53:46,540][67871] Updated weights for policy 1, policy_version 52870 (0.0008) [2023-10-07 21:53:46,796][67838] Updated weights for policy 0, policy_version 52802 (0.0007) [2023-10-07 21:53:46,913][67871] Updated weights for policy 1, policy_version 52880 (0.0007) [2023-10-07 21:53:47,175][67838] Updated weights for policy 0, policy_version 52812 (0.0008) [2023-10-07 21:53:47,278][67871] Updated weights for policy 1, policy_version 52890 (0.0008) [2023-10-07 21:53:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 108199936. Throughput: 0: 1647.2, 1: 1658.8. Samples: 27065790. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-07 21:53:47,477][66916] Avg episode reward: [(0, '46.260'), (1, '50.280')] [2023-10-07 21:53:47,545][67838] Updated weights for policy 0, policy_version 52822 (0.0007) [2023-10-07 21:53:47,917][67838] Updated weights for policy 0, policy_version 52832 (0.0009) [2023-10-07 21:53:51,559][67871] Updated weights for policy 1, policy_version 52900 (0.0007) [2023-10-07 21:53:51,926][67871] Updated weights for policy 1, policy_version 52910 (0.0007) [2023-10-07 21:53:52,195][67838] Updated weights for policy 0, policy_version 52842 (0.0010) [2023-10-07 21:53:52,283][67871] Updated weights for policy 1, policy_version 52920 (0.0007) [2023-10-07 21:53:52,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108265472. Throughput: 0: 1643.4, 1: 1654.8. Samples: 27085504. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-07 21:53:52,477][66916] Avg episode reward: [(0, '48.160'), (1, '52.430')] [2023-10-07 21:53:52,559][67838] Updated weights for policy 0, policy_version 52852 (0.0008) [2023-10-07 21:53:52,942][67838] Updated weights for policy 0, policy_version 52862 (0.0007) [2023-10-07 21:53:56,248][67871] Updated weights for policy 1, policy_version 52930 (0.0007) [2023-10-07 21:53:56,613][67871] Updated weights for policy 1, policy_version 52940 (0.0009) [2023-10-07 21:53:56,979][67871] Updated weights for policy 1, policy_version 52950 (0.0007) [2023-10-07 21:53:57,042][67838] Updated weights for policy 0, policy_version 52872 (0.0007) [2023-10-07 21:53:57,353][67871] Updated weights for policy 1, policy_version 52960 (0.0007) [2023-10-07 21:53:57,407][67838] Updated weights for policy 0, policy_version 52882 (0.0007) [2023-10-07 21:53:57,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 108363776. Throughput: 0: 1648.8, 1: 1669.2. Samples: 27095222. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-07 21:53:57,478][66916] Avg episode reward: [(0, '48.140'), (1, '55.440')] [2023-10-07 21:53:57,780][67838] Updated weights for policy 0, policy_version 52892 (0.0009) [2023-10-07 21:54:01,363][67871] Updated weights for policy 1, policy_version 52970 (0.0008) [2023-10-07 21:54:01,726][67871] Updated weights for policy 1, policy_version 52980 (0.0008) [2023-10-07 21:54:01,752][67838] Updated weights for policy 0, policy_version 52902 (0.0009) [2023-10-07 21:54:02,099][67871] Updated weights for policy 1, policy_version 52990 (0.0008) [2023-10-07 21:54:02,126][67838] Updated weights for policy 0, policy_version 52912 (0.0008) [2023-10-07 21:54:02,476][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 108429312. Throughput: 0: 1655.9, 1: 1672.3. Samples: 27116044. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-07 21:54:02,477][66916] Avg episode reward: [(0, '48.320'), (1, '55.680')] [2023-10-07 21:54:02,499][67838] Updated weights for policy 0, policy_version 52922 (0.0008) [2023-10-07 21:54:06,310][67871] Updated weights for policy 1, policy_version 53000 (0.0008) [2023-10-07 21:54:06,674][67871] Updated weights for policy 1, policy_version 53010 (0.0009) [2023-10-07 21:54:06,699][67838] Updated weights for policy 0, policy_version 52932 (0.0007) [2023-10-07 21:54:07,041][67871] Updated weights for policy 1, policy_version 53020 (0.0008) [2023-10-07 21:54:07,075][67838] Updated weights for policy 0, policy_version 52942 (0.0007) [2023-10-07 21:54:07,446][67838] Updated weights for policy 0, policy_version 52952 (0.0008) [2023-10-07 21:54:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 108494848. Throughput: 0: 1644.3, 1: 1651.7. Samples: 27135002. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-07 21:54:07,477][66916] Avg episode reward: [(0, '47.660'), (1, '54.110')] [2023-10-07 21:54:11,406][67871] Updated weights for policy 1, policy_version 53030 (0.0008) [2023-10-07 21:54:11,456][67838] Updated weights for policy 0, policy_version 52962 (0.0008) [2023-10-07 21:54:11,769][67871] Updated weights for policy 1, policy_version 53040 (0.0007) [2023-10-07 21:54:11,827][67838] Updated weights for policy 0, policy_version 52972 (0.0009) [2023-10-07 21:54:12,127][67871] Updated weights for policy 1, policy_version 53050 (0.0007) [2023-10-07 21:54:12,196][67838] Updated weights for policy 0, policy_version 52982 (0.0007) [2023-10-07 21:54:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 108560384. Throughput: 0: 1663.0, 1: 1664.5. Samples: 27145432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:54:12,477][66916] Avg episode reward: [(0, '49.710'), (1, '53.830')] [2023-10-07 21:54:12,566][67838] Updated weights for policy 0, policy_version 52992 (0.0007) [2023-10-07 21:54:16,145][67871] Updated weights for policy 1, policy_version 53060 (0.0010) [2023-10-07 21:54:16,512][67871] Updated weights for policy 1, policy_version 53070 (0.0009) [2023-10-07 21:54:16,706][67838] Updated weights for policy 0, policy_version 53002 (0.0007) [2023-10-07 21:54:16,879][67871] Updated weights for policy 1, policy_version 53080 (0.0009) [2023-10-07 21:54:17,073][67838] Updated weights for policy 0, policy_version 53012 (0.0007) [2023-10-07 21:54:17,440][67838] Updated weights for policy 0, policy_version 53022 (0.0010) [2023-10-07 21:54:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 108625920. Throughput: 0: 1664.0, 1: 1665.5. Samples: 27165926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:54:17,477][66916] Avg episode reward: [(0, '47.800'), (1, '52.130')] [2023-10-07 21:54:21,174][67871] Updated weights for policy 1, policy_version 53090 (0.0008) [2023-10-07 21:54:21,508][67838] Updated weights for policy 0, policy_version 53032 (0.0007) [2023-10-07 21:54:21,548][67871] Updated weights for policy 1, policy_version 53100 (0.0007) [2023-10-07 21:54:21,873][67838] Updated weights for policy 0, policy_version 53042 (0.0009) [2023-10-07 21:54:21,910][67871] Updated weights for policy 1, policy_version 53110 (0.0009) [2023-10-07 21:54:22,240][67838] Updated weights for policy 0, policy_version 53052 (0.0010) [2023-10-07 21:54:22,280][67871] Updated weights for policy 1, policy_version 53120 (0.0007) [2023-10-07 21:54:22,477][66916] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 108724224. Throughput: 0: 1653.1, 1: 1651.3. Samples: 27184746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:54:22,477][66916] Avg episode reward: [(0, '47.840'), (1, '54.130')] [2023-10-07 21:54:26,459][67838] Updated weights for policy 0, policy_version 53062 (0.0007) [2023-10-07 21:54:26,538][67871] Updated weights for policy 1, policy_version 53130 (0.0007) [2023-10-07 21:54:26,825][67838] Updated weights for policy 0, policy_version 53072 (0.0007) [2023-10-07 21:54:26,901][67871] Updated weights for policy 1, policy_version 53140 (0.0009) [2023-10-07 21:54:27,191][67838] Updated weights for policy 0, policy_version 53082 (0.0009) [2023-10-07 21:54:27,267][67871] Updated weights for policy 1, policy_version 53150 (0.0009) [2023-10-07 21:54:27,477][66916] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 108789760. Throughput: 0: 1668.1, 1: 1664.4. Samples: 27195312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:54:27,478][66916] Avg episode reward: [(0, '52.010'), (1, '52.430')] [2023-10-07 21:54:31,303][67871] Updated weights for policy 1, policy_version 53160 (0.0009) [2023-10-07 21:54:31,370][67838] Updated weights for policy 0, policy_version 53092 (0.0009) [2023-10-07 21:54:31,661][67871] Updated weights for policy 1, policy_version 53170 (0.0009) [2023-10-07 21:54:31,743][67838] Updated weights for policy 0, policy_version 53102 (0.0009) [2023-10-07 21:54:32,030][67871] Updated weights for policy 1, policy_version 53180 (0.0007) [2023-10-07 21:54:32,126][67838] Updated weights for policy 0, policy_version 53112 (0.0009) [2023-10-07 21:54:32,477][66916] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 108855296. Throughput: 0: 1668.1, 1: 1657.6. Samples: 27215448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:54:32,478][66916] Avg episode reward: [(0, '50.650'), (1, '53.670')] [2023-10-07 21:54:36,093][67838] Updated weights for policy 0, policy_version 53122 (0.0009) [2023-10-07 21:54:36,295][67871] Updated weights for policy 1, policy_version 53190 (0.0009) [2023-10-07 21:54:36,493][67838] Updated weights for policy 0, policy_version 53132 (0.0008) [2023-10-07 21:54:36,662][67871] Updated weights for policy 1, policy_version 53200 (0.0009) [2023-10-07 21:54:36,860][67838] Updated weights for policy 0, policy_version 53142 (0.0010) [2023-10-07 21:54:37,034][67871] Updated weights for policy 1, policy_version 53210 (0.0008) [2023-10-07 21:54:37,230][67838] Updated weights for policy 0, policy_version 53152 (0.0007) [2023-10-07 21:54:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 108920832. Throughput: 0: 1653.5, 1: 1642.4. Samples: 27233816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:54:37,477][66916] Avg episode reward: [(0, '53.720'), (1, '54.770')] [2023-10-07 21:54:41,244][67871] Updated weights for policy 1, policy_version 53220 (0.0008) [2023-10-07 21:54:41,365][67838] Updated weights for policy 0, policy_version 53162 (0.0007) [2023-10-07 21:54:41,610][67871] Updated weights for policy 1, policy_version 53230 (0.0007) [2023-10-07 21:54:41,734][67838] Updated weights for policy 0, policy_version 53172 (0.0009) [2023-10-07 21:54:41,976][67871] Updated weights for policy 1, policy_version 53240 (0.0007) [2023-10-07 21:54:42,107][67838] Updated weights for policy 0, policy_version 53182 (0.0009) [2023-10-07 21:54:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13329.3). Total num frames: 108986368. Throughput: 0: 1667.0, 1: 1650.6. Samples: 27244516. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-07 21:54:42,478][66916] Avg episode reward: [(0, '57.070'), (1, '58.230')] [2023-10-07 21:54:46,256][67871] Updated weights for policy 1, policy_version 53250 (0.0007) [2023-10-07 21:54:46,331][67838] Updated weights for policy 0, policy_version 53192 (0.0008) [2023-10-07 21:54:46,619][67871] Updated weights for policy 1, policy_version 53260 (0.0007) [2023-10-07 21:54:46,710][67838] Updated weights for policy 0, policy_version 53202 (0.0007) [2023-10-07 21:54:46,981][67871] Updated weights for policy 1, policy_version 53270 (0.0009) [2023-10-07 21:54:47,074][67838] Updated weights for policy 0, policy_version 53212 (0.0007) [2023-10-07 21:54:47,350][67871] Updated weights for policy 1, policy_version 53280 (0.0008) [2023-10-07 21:54:47,477][66916] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 109051904. Throughput: 0: 1660.5, 1: 1637.8. Samples: 27264470. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-07 21:54:47,478][66916] Avg episode reward: [(0, '59.790'), (1, '56.670')] [2023-10-07 21:54:51,103][67838] Updated weights for policy 0, policy_version 53222 (0.0007) [2023-10-07 21:54:51,456][67871] Updated weights for policy 1, policy_version 53290 (0.0009) [2023-10-07 21:54:51,476][67838] Updated weights for policy 0, policy_version 53232 (0.0007) [2023-10-07 21:54:51,830][67871] Updated weights for policy 1, policy_version 53300 (0.0009) [2023-10-07 21:54:51,859][67838] Updated weights for policy 0, policy_version 53242 (0.0009) [2023-10-07 21:54:52,193][67871] Updated weights for policy 1, policy_version 53310 (0.0010) [2023-10-07 21:54:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 109117440. Throughput: 0: 1647.5, 1: 1639.1. Samples: 27282898. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-07 21:54:52,477][66916] Avg episode reward: [(0, '58.930'), (1, '53.940')] [2023-10-07 21:54:56,144][67838] Updated weights for policy 0, policy_version 53252 (0.0008) [2023-10-07 21:54:56,254][67871] Updated weights for policy 1, policy_version 53320 (0.0007) [2023-10-07 21:54:56,523][67838] Updated weights for policy 0, policy_version 53262 (0.0008) [2023-10-07 21:54:56,618][67871] Updated weights for policy 1, policy_version 53330 (0.0007) [2023-10-07 21:54:56,883][67838] Updated weights for policy 0, policy_version 53272 (0.0008) [2023-10-07 21:54:56,988][67871] Updated weights for policy 1, policy_version 53340 (0.0009) [2023-10-07 21:54:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 109182976. Throughput: 0: 1654.4, 1: 1638.5. Samples: 27293610. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-07 21:54:57,478][66916] Avg episode reward: [(0, '57.540'), (1, '55.410')] [2023-10-07 21:55:01,012][67838] Updated weights for policy 0, policy_version 53282 (0.0007) [2023-10-07 21:55:01,216][67871] Updated weights for policy 1, policy_version 53350 (0.0007) [2023-10-07 21:55:01,379][67838] Updated weights for policy 0, policy_version 53292 (0.0008) [2023-10-07 21:55:01,584][67871] Updated weights for policy 1, policy_version 53360 (0.0007) [2023-10-07 21:55:01,749][67838] Updated weights for policy 0, policy_version 53302 (0.0010) [2023-10-07 21:55:01,943][67871] Updated weights for policy 1, policy_version 53370 (0.0009) [2023-10-07 21:55:02,110][67838] Updated weights for policy 0, policy_version 53312 (0.0009) [2023-10-07 21:55:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 109248512. Throughput: 0: 1652.0, 1: 1639.6. Samples: 27314050. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-07 21:55:02,477][66916] Avg episode reward: [(0, '56.080'), (1, '55.750')] [2023-10-07 21:55:06,096][67871] Updated weights for policy 1, policy_version 53380 (0.0008) [2023-10-07 21:55:06,116][67838] Updated weights for policy 0, policy_version 53322 (0.0009) [2023-10-07 21:55:06,464][67871] Updated weights for policy 1, policy_version 53390 (0.0008) [2023-10-07 21:55:06,491][67838] Updated weights for policy 0, policy_version 53332 (0.0009) [2023-10-07 21:55:06,832][67871] Updated weights for policy 1, policy_version 53400 (0.0008) [2023-10-07 21:55:06,865][67838] Updated weights for policy 0, policy_version 53342 (0.0008) [2023-10-07 21:55:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 109314048. Throughput: 0: 1642.3, 1: 1637.8. Samples: 27332348. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-07 21:55:07,478][66916] Avg episode reward: [(0, '50.620'), (1, '56.640')] [2023-10-07 21:55:07,491][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000053344_54624256.pth... [2023-10-07 21:55:07,492][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000053408_54689792.pth... [2023-10-07 21:55:07,528][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000051840_53084160.pth [2023-10-07 21:55:07,534][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000051776_53018624.pth [2023-10-07 21:55:10,993][67838] Updated weights for policy 0, policy_version 53352 (0.0007) [2023-10-07 21:55:11,137][67871] Updated weights for policy 1, policy_version 53410 (0.0008) [2023-10-07 21:55:11,368][67838] Updated weights for policy 0, policy_version 53362 (0.0007) [2023-10-07 21:55:11,530][67871] Updated weights for policy 1, policy_version 53420 (0.0007) [2023-10-07 21:55:11,738][67838] Updated weights for policy 0, policy_version 53372 (0.0010) [2023-10-07 21:55:11,887][67871] Updated weights for policy 1, policy_version 53430 (0.0009) [2023-10-07 21:55:12,249][67871] Updated weights for policy 1, policy_version 53440 (0.0007) [2023-10-07 21:55:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 109379584. Throughput: 0: 1651.5, 1: 1642.2. Samples: 27343528. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-07 21:55:12,477][66916] Avg episode reward: [(0, '51.150'), (1, '55.670')] [2023-10-07 21:55:15,810][67838] Updated weights for policy 0, policy_version 53382 (0.0009) [2023-10-07 21:55:16,189][67838] Updated weights for policy 0, policy_version 53392 (0.0008) [2023-10-07 21:55:16,343][67871] Updated weights for policy 1, policy_version 53450 (0.0009) [2023-10-07 21:55:16,563][67838] Updated weights for policy 0, policy_version 53402 (0.0007) [2023-10-07 21:55:16,715][67871] Updated weights for policy 1, policy_version 53460 (0.0008) [2023-10-07 21:55:17,079][67871] Updated weights for policy 1, policy_version 53470 (0.0007) [2023-10-07 21:55:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 109445120. Throughput: 0: 1646.0, 1: 1646.8. Samples: 27363628. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 21:55:17,477][66916] Avg episode reward: [(0, '52.140'), (1, '55.130')] [2023-10-07 21:55:20,715][67838] Updated weights for policy 0, policy_version 53412 (0.0010) [2023-10-07 21:55:21,025][67871] Updated weights for policy 1, policy_version 53480 (0.0007) [2023-10-07 21:55:21,096][67838] Updated weights for policy 0, policy_version 53422 (0.0009) [2023-10-07 21:55:21,407][67871] Updated weights for policy 1, policy_version 53490 (0.0008) [2023-10-07 21:55:21,461][67838] Updated weights for policy 0, policy_version 53432 (0.0008) [2023-10-07 21:55:21,775][67871] Updated weights for policy 1, policy_version 53500 (0.0008) [2023-10-07 21:55:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 109510656. Throughput: 0: 1650.4, 1: 1647.2. Samples: 27382210. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 21:55:22,477][66916] Avg episode reward: [(0, '51.460'), (1, '57.450')] [2023-10-07 21:55:25,665][67838] Updated weights for policy 0, policy_version 53442 (0.0007) [2023-10-07 21:55:25,820][67871] Updated weights for policy 1, policy_version 53510 (0.0009) [2023-10-07 21:55:26,044][67838] Updated weights for policy 0, policy_version 53452 (0.0007) [2023-10-07 21:55:26,185][67871] Updated weights for policy 1, policy_version 53520 (0.0008) [2023-10-07 21:55:26,408][67838] Updated weights for policy 0, policy_version 53462 (0.0008) [2023-10-07 21:55:26,537][67871] Updated weights for policy 1, policy_version 53530 (0.0008) [2023-10-07 21:55:26,780][67838] Updated weights for policy 0, policy_version 53472 (0.0008) [2023-10-07 21:55:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 109576192. Throughput: 0: 1662.2, 1: 1654.7. Samples: 27393774. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 21:55:27,478][66916] Avg episode reward: [(0, '49.750'), (1, '57.830')] [2023-10-07 21:55:30,500][67871] Updated weights for policy 1, policy_version 53540 (0.0009) [2023-10-07 21:55:30,872][67871] Updated weights for policy 1, policy_version 53550 (0.0007) [2023-10-07 21:55:31,071][67838] Updated weights for policy 0, policy_version 53482 (0.0007) [2023-10-07 21:55:31,237][67871] Updated weights for policy 1, policy_version 53560 (0.0008) [2023-10-07 21:55:31,433][67838] Updated weights for policy 0, policy_version 53492 (0.0008) [2023-10-07 21:55:31,809][67838] Updated weights for policy 0, policy_version 53502 (0.0008) [2023-10-07 21:55:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 109641728. Throughput: 0: 1654.6, 1: 1654.0. Samples: 27413358. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 21:55:32,478][66916] Avg episode reward: [(0, '49.630'), (1, '55.930')] [2023-10-07 21:55:35,350][67871] Updated weights for policy 1, policy_version 53570 (0.0009) [2023-10-07 21:55:35,716][67871] Updated weights for policy 1, policy_version 53580 (0.0010) [2023-10-07 21:55:35,797][67838] Updated weights for policy 0, policy_version 53512 (0.0011) [2023-10-07 21:55:36,085][67871] Updated weights for policy 1, policy_version 53590 (0.0010) [2023-10-07 21:55:36,167][67838] Updated weights for policy 0, policy_version 53522 (0.0009) [2023-10-07 21:55:36,454][67871] Updated weights for policy 1, policy_version 53600 (0.0009) [2023-10-07 21:55:36,541][67838] Updated weights for policy 0, policy_version 53532 (0.0010) [2023-10-07 21:55:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 109707264. Throughput: 0: 1657.2, 1: 1655.6. Samples: 27431972. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 21:55:37,478][66916] Avg episode reward: [(0, '49.470'), (1, '56.780')] [2023-10-07 21:55:40,663][67871] Updated weights for policy 1, policy_version 53610 (0.0009) [2023-10-07 21:55:40,838][67838] Updated weights for policy 0, policy_version 53542 (0.0009) [2023-10-07 21:55:41,028][67871] Updated weights for policy 1, policy_version 53620 (0.0008) [2023-10-07 21:55:41,211][67838] Updated weights for policy 0, policy_version 53552 (0.0008) [2023-10-07 21:55:41,393][67871] Updated weights for policy 1, policy_version 53630 (0.0008) [2023-10-07 21:55:41,578][67838] Updated weights for policy 0, policy_version 53562 (0.0009) [2023-10-07 21:55:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 109772800. Throughput: 0: 1657.4, 1: 1666.1. Samples: 27443170. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 21:55:42,477][66916] Avg episode reward: [(0, '51.740'), (1, '53.790')] [2023-10-07 21:55:45,653][67871] Updated weights for policy 1, policy_version 53640 (0.0009) [2023-10-07 21:55:45,749][67838] Updated weights for policy 0, policy_version 53572 (0.0008) [2023-10-07 21:55:46,018][67871] Updated weights for policy 1, policy_version 53650 (0.0008) [2023-10-07 21:55:46,117][67838] Updated weights for policy 0, policy_version 53582 (0.0007) [2023-10-07 21:55:46,385][67871] Updated weights for policy 1, policy_version 53660 (0.0008) [2023-10-07 21:55:46,484][67838] Updated weights for policy 0, policy_version 53592 (0.0008) [2023-10-07 21:55:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 109838336. Throughput: 0: 1645.1, 1: 1654.1. Samples: 27462516. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 21:55:47,478][66916] Avg episode reward: [(0, '50.980'), (1, '52.850')] [2023-10-07 21:55:50,530][67871] Updated weights for policy 1, policy_version 53670 (0.0009) [2023-10-07 21:55:50,621][67838] Updated weights for policy 0, policy_version 53602 (0.0008) [2023-10-07 21:55:50,889][67871] Updated weights for policy 1, policy_version 53680 (0.0008) [2023-10-07 21:55:51,001][67838] Updated weights for policy 0, policy_version 53612 (0.0009) [2023-10-07 21:55:51,253][67871] Updated weights for policy 1, policy_version 53690 (0.0007) [2023-10-07 21:55:51,368][67838] Updated weights for policy 0, policy_version 53622 (0.0009) [2023-10-07 21:55:51,735][67838] Updated weights for policy 0, policy_version 53632 (0.0008) [2023-10-07 21:55:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 109903872. Throughput: 0: 1656.1, 1: 1660.1. Samples: 27481576. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-07 21:55:52,477][66916] Avg episode reward: [(0, '52.780'), (1, '48.200')] [2023-10-07 21:55:55,381][67871] Updated weights for policy 1, policy_version 53700 (0.0007) [2023-10-07 21:55:55,725][67838] Updated weights for policy 0, policy_version 53642 (0.0008) [2023-10-07 21:55:55,751][67871] Updated weights for policy 1, policy_version 53710 (0.0008) [2023-10-07 21:55:56,093][67838] Updated weights for policy 0, policy_version 53652 (0.0008) [2023-10-07 21:55:56,114][67871] Updated weights for policy 1, policy_version 53720 (0.0009) [2023-10-07 21:55:56,462][67838] Updated weights for policy 0, policy_version 53662 (0.0009) [2023-10-07 21:55:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 109969408. Throughput: 0: 1659.2, 1: 1667.3. Samples: 27493220. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-07 21:55:57,477][66916] Avg episode reward: [(0, '54.320'), (1, '50.940')] [2023-10-07 21:56:00,288][67871] Updated weights for policy 1, policy_version 53730 (0.0008) [2023-10-07 21:56:00,633][67838] Updated weights for policy 0, policy_version 53672 (0.0009) [2023-10-07 21:56:00,712][67871] Updated weights for policy 1, policy_version 53740 (0.0007) [2023-10-07 21:56:01,004][67838] Updated weights for policy 0, policy_version 53682 (0.0009) [2023-10-07 21:56:01,068][67871] Updated weights for policy 1, policy_version 53750 (0.0008) [2023-10-07 21:56:01,380][67838] Updated weights for policy 0, policy_version 53692 (0.0008) [2023-10-07 21:56:01,433][67871] Updated weights for policy 1, policy_version 53760 (0.0007) [2023-10-07 21:56:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110034944. Throughput: 0: 1654.1, 1: 1652.9. Samples: 27512446. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-07 21:56:02,478][66916] Avg episode reward: [(0, '53.180'), (1, '50.390')] [2023-10-07 21:56:05,414][67838] Updated weights for policy 0, policy_version 53702 (0.0008) [2023-10-07 21:56:05,564][67871] Updated weights for policy 1, policy_version 53770 (0.0008) [2023-10-07 21:56:05,790][67838] Updated weights for policy 0, policy_version 53712 (0.0010) [2023-10-07 21:56:05,919][67871] Updated weights for policy 1, policy_version 53780 (0.0007) [2023-10-07 21:56:06,160][67838] Updated weights for policy 0, policy_version 53722 (0.0009) [2023-10-07 21:56:06,295][67871] Updated weights for policy 1, policy_version 53790 (0.0007) [2023-10-07 21:56:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110100480. Throughput: 0: 1663.2, 1: 1655.1. Samples: 27531532. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-07 21:56:07,477][66916] Avg episode reward: [(0, '51.840'), (1, '51.090')] [2023-10-07 21:56:10,288][67838] Updated weights for policy 0, policy_version 53732 (0.0009) [2023-10-07 21:56:10,352][67871] Updated weights for policy 1, policy_version 53800 (0.0008) [2023-10-07 21:56:10,677][67838] Updated weights for policy 0, policy_version 53742 (0.0009) [2023-10-07 21:56:10,722][67871] Updated weights for policy 1, policy_version 53810 (0.0008) [2023-10-07 21:56:11,047][67838] Updated weights for policy 0, policy_version 53752 (0.0009) [2023-10-07 21:56:11,079][67871] Updated weights for policy 1, policy_version 53820 (0.0007) [2023-10-07 21:56:12,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 110166016. Throughput: 0: 1664.4, 1: 1656.4. Samples: 27543212. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-07 21:56:12,478][66916] Avg episode reward: [(0, '54.140'), (1, '51.180')] [2023-10-07 21:56:15,247][67838] Updated weights for policy 0, policy_version 53762 (0.0010) [2023-10-07 21:56:15,309][67871] Updated weights for policy 1, policy_version 53830 (0.0007) [2023-10-07 21:56:15,618][67838] Updated weights for policy 0, policy_version 53772 (0.0007) [2023-10-07 21:56:15,681][67871] Updated weights for policy 1, policy_version 53840 (0.0008) [2023-10-07 21:56:15,989][67838] Updated weights for policy 0, policy_version 53782 (0.0008) [2023-10-07 21:56:16,046][67871] Updated weights for policy 1, policy_version 53850 (0.0007) [2023-10-07 21:56:16,354][67838] Updated weights for policy 0, policy_version 53792 (0.0007) [2023-10-07 21:56:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110231552. Throughput: 0: 1654.1, 1: 1646.7. Samples: 27561894. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-07 21:56:17,478][66916] Avg episode reward: [(0, '53.510'), (1, '54.100')] [2023-10-07 21:56:20,182][67838] Updated weights for policy 0, policy_version 53802 (0.0008) [2023-10-07 21:56:20,219][67871] Updated weights for policy 1, policy_version 53860 (0.0007) [2023-10-07 21:56:20,552][67838] Updated weights for policy 0, policy_version 53812 (0.0009) [2023-10-07 21:56:20,587][67871] Updated weights for policy 1, policy_version 53870 (0.0008) [2023-10-07 21:56:20,927][67838] Updated weights for policy 0, policy_version 53822 (0.0008) [2023-10-07 21:56:20,955][67871] Updated weights for policy 1, policy_version 53880 (0.0007) [2023-10-07 21:56:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110297088. Throughput: 0: 1674.8, 1: 1652.7. Samples: 27581706. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-07 21:56:22,477][66916] Avg episode reward: [(0, '54.660'), (1, '54.160')] [2023-10-07 21:56:25,021][67871] Updated weights for policy 1, policy_version 53890 (0.0008) [2023-10-07 21:56:25,184][67838] Updated weights for policy 0, policy_version 53832 (0.0008) [2023-10-07 21:56:25,386][67871] Updated weights for policy 1, policy_version 53900 (0.0009) [2023-10-07 21:56:25,553][67838] Updated weights for policy 0, policy_version 53842 (0.0010) [2023-10-07 21:56:25,761][67871] Updated weights for policy 1, policy_version 53910 (0.0007) [2023-10-07 21:56:25,916][67838] Updated weights for policy 0, policy_version 53852 (0.0009) [2023-10-07 21:56:26,125][67871] Updated weights for policy 1, policy_version 53920 (0.0007) [2023-10-07 21:56:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 110362624. Throughput: 0: 1674.6, 1: 1657.2. Samples: 27593100. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-07 21:56:27,478][66916] Avg episode reward: [(0, '54.400'), (1, '52.880')] [2023-10-07 21:56:29,892][67838] Updated weights for policy 0, policy_version 53862 (0.0007) [2023-10-07 21:56:30,267][67838] Updated weights for policy 0, policy_version 53872 (0.0007) [2023-10-07 21:56:30,277][67871] Updated weights for policy 1, policy_version 53930 (0.0008) [2023-10-07 21:56:30,636][67838] Updated weights for policy 0, policy_version 53882 (0.0007) [2023-10-07 21:56:30,645][67871] Updated weights for policy 1, policy_version 53940 (0.0009) [2023-10-07 21:56:31,013][67871] Updated weights for policy 1, policy_version 53950 (0.0008) [2023-10-07 21:56:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 110428160. Throughput: 0: 1663.3, 1: 1648.3. Samples: 27611540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:56:32,478][66916] Avg episode reward: [(0, '53.030'), (1, '54.890')] [2023-10-07 21:56:34,819][67838] Updated weights for policy 0, policy_version 53892 (0.0008) [2023-10-07 21:56:35,104][67871] Updated weights for policy 1, policy_version 53960 (0.0009) [2023-10-07 21:56:35,196][67838] Updated weights for policy 0, policy_version 53902 (0.0009) [2023-10-07 21:56:35,475][67871] Updated weights for policy 1, policy_version 53970 (0.0008) [2023-10-07 21:56:35,569][67838] Updated weights for policy 0, policy_version 53912 (0.0009) [2023-10-07 21:56:35,839][67871] Updated weights for policy 1, policy_version 53980 (0.0010) [2023-10-07 21:56:37,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110493696. Throughput: 0: 1677.8, 1: 1655.5. Samples: 27631576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:56:37,478][66916] Avg episode reward: [(0, '54.400'), (1, '57.100')] [2023-10-07 21:56:39,577][67838] Updated weights for policy 0, policy_version 53922 (0.0008) [2023-10-07 21:56:39,945][67838] Updated weights for policy 0, policy_version 53932 (0.0009) [2023-10-07 21:56:40,052][67871] Updated weights for policy 1, policy_version 53990 (0.0009) [2023-10-07 21:56:40,307][67838] Updated weights for policy 0, policy_version 53942 (0.0009) [2023-10-07 21:56:40,414][67871] Updated weights for policy 1, policy_version 54000 (0.0010) [2023-10-07 21:56:40,682][67838] Updated weights for policy 0, policy_version 53952 (0.0007) [2023-10-07 21:56:40,781][67871] Updated weights for policy 1, policy_version 54010 (0.0008) [2023-10-07 21:56:42,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 110559232. Throughput: 0: 1661.9, 1: 1652.1. Samples: 27642350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:56:42,478][66916] Avg episode reward: [(0, '51.050'), (1, '57.190')] [2023-10-07 21:56:44,911][67871] Updated weights for policy 1, policy_version 54020 (0.0009) [2023-10-07 21:56:44,991][67838] Updated weights for policy 0, policy_version 53962 (0.0007) [2023-10-07 21:56:45,285][67871] Updated weights for policy 1, policy_version 54030 (0.0007) [2023-10-07 21:56:45,358][67838] Updated weights for policy 0, policy_version 53972 (0.0007) [2023-10-07 21:56:45,648][67871] Updated weights for policy 1, policy_version 54040 (0.0008) [2023-10-07 21:56:45,723][67838] Updated weights for policy 0, policy_version 53982 (0.0009) [2023-10-07 21:56:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110624768. Throughput: 0: 1652.7, 1: 1639.1. Samples: 27660576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:56:47,477][66916] Avg episode reward: [(0, '51.230'), (1, '56.020')] [2023-10-07 21:56:49,771][67871] Updated weights for policy 1, policy_version 54050 (0.0008) [2023-10-07 21:56:49,937][67838] Updated weights for policy 0, policy_version 53992 (0.0008) [2023-10-07 21:56:50,147][67871] Updated weights for policy 1, policy_version 54060 (0.0009) [2023-10-07 21:56:50,312][67838] Updated weights for policy 0, policy_version 54002 (0.0007) [2023-10-07 21:56:50,512][67871] Updated weights for policy 1, policy_version 54070 (0.0008) [2023-10-07 21:56:50,675][67838] Updated weights for policy 0, policy_version 54012 (0.0008) [2023-10-07 21:56:50,872][67871] Updated weights for policy 1, policy_version 54080 (0.0008) [2023-10-07 21:56:52,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 110690304. Throughput: 0: 1662.4, 1: 1659.0. Samples: 27680998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:56:52,478][66916] Avg episode reward: [(0, '53.860'), (1, '58.950')] [2023-10-07 21:56:54,734][67838] Updated weights for policy 0, policy_version 54022 (0.0010) [2023-10-07 21:56:54,975][67871] Updated weights for policy 1, policy_version 54090 (0.0008) [2023-10-07 21:56:55,105][67838] Updated weights for policy 0, policy_version 54032 (0.0010) [2023-10-07 21:56:55,348][67871] Updated weights for policy 1, policy_version 54100 (0.0009) [2023-10-07 21:56:55,476][67838] Updated weights for policy 0, policy_version 54042 (0.0009) [2023-10-07 21:56:55,715][67871] Updated weights for policy 1, policy_version 54110 (0.0008) [2023-10-07 21:56:57,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110755840. Throughput: 0: 1644.7, 1: 1652.1. Samples: 27691568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:56:57,478][66916] Avg episode reward: [(0, '56.690'), (1, '57.160')] [2023-10-07 21:56:59,661][67838] Updated weights for policy 0, policy_version 54052 (0.0008) [2023-10-07 21:56:59,986][67871] Updated weights for policy 1, policy_version 54120 (0.0009) [2023-10-07 21:57:00,060][67838] Updated weights for policy 0, policy_version 54062 (0.0007) [2023-10-07 21:57:00,350][67871] Updated weights for policy 1, policy_version 54130 (0.0008) [2023-10-07 21:57:00,436][67838] Updated weights for policy 0, policy_version 54072 (0.0009) [2023-10-07 21:57:00,720][67871] Updated weights for policy 1, policy_version 54140 (0.0009) [2023-10-07 21:57:02,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110821376. Throughput: 0: 1647.3, 1: 1649.3. Samples: 27710244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:57:02,478][66916] Avg episode reward: [(0, '56.690'), (1, '58.400')] [2023-10-07 21:57:04,418][67838] Updated weights for policy 0, policy_version 54082 (0.0009) [2023-10-07 21:57:04,788][67838] Updated weights for policy 0, policy_version 54092 (0.0010) [2023-10-07 21:57:04,808][67871] Updated weights for policy 1, policy_version 54150 (0.0008) [2023-10-07 21:57:05,153][67838] Updated weights for policy 0, policy_version 54102 (0.0009) [2023-10-07 21:57:05,176][67871] Updated weights for policy 1, policy_version 54160 (0.0009) [2023-10-07 21:57:05,532][67838] Updated weights for policy 0, policy_version 54112 (0.0008) [2023-10-07 21:57:05,541][67871] Updated weights for policy 1, policy_version 54170 (0.0009) [2023-10-07 21:57:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 110886912. Throughput: 0: 1648.3, 1: 1661.1. Samples: 27730632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:57:07,478][66916] Avg episode reward: [(0, '56.260'), (1, '55.290')] [2023-10-07 21:57:07,490][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000054176_55476224.pth... [2023-10-07 21:57:07,490][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000054112_55410688.pth... [2023-10-07 21:57:07,523][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000052576_53837824.pth [2023-10-07 21:57:07,528][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000052640_53903360.pth [2023-10-07 21:57:07,529][67511] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p0/milestones/checkpoint_000054112_55410688.pth [2023-10-07 21:57:07,533][67676] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p1/milestones/checkpoint_000054176_55476224.pth [2023-10-07 21:57:09,572][67838] Updated weights for policy 0, policy_version 54122 (0.0009) [2023-10-07 21:57:09,714][67871] Updated weights for policy 1, policy_version 54180 (0.0008) [2023-10-07 21:57:09,945][67838] Updated weights for policy 0, policy_version 54132 (0.0009) [2023-10-07 21:57:10,079][67871] Updated weights for policy 1, policy_version 54190 (0.0007) [2023-10-07 21:57:10,321][67838] Updated weights for policy 0, policy_version 54142 (0.0008) [2023-10-07 21:57:10,447][67871] Updated weights for policy 1, policy_version 54200 (0.0009) [2023-10-07 21:57:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110952448. Throughput: 0: 1634.0, 1: 1649.5. Samples: 27740854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:57:12,478][66916] Avg episode reward: [(0, '57.780'), (1, '57.610')] [2023-10-07 21:57:14,533][67871] Updated weights for policy 1, policy_version 54210 (0.0008) [2023-10-07 21:57:14,575][67838] Updated weights for policy 0, policy_version 54152 (0.0008) [2023-10-07 21:57:14,903][67871] Updated weights for policy 1, policy_version 54220 (0.0007) [2023-10-07 21:57:14,958][67838] Updated weights for policy 0, policy_version 54162 (0.0009) [2023-10-07 21:57:15,274][67871] Updated weights for policy 1, policy_version 54230 (0.0007) [2023-10-07 21:57:15,322][67838] Updated weights for policy 0, policy_version 54172 (0.0008) [2023-10-07 21:57:15,638][67871] Updated weights for policy 1, policy_version 54240 (0.0008) [2023-10-07 21:57:17,477][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111017984. Throughput: 0: 1649.5, 1: 1647.9. Samples: 27759922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:57:17,478][66916] Avg episode reward: [(0, '54.320'), (1, '53.710')] [2023-10-07 21:57:19,147][67838] Updated weights for policy 0, policy_version 54182 (0.0008) [2023-10-07 21:57:19,518][67838] Updated weights for policy 0, policy_version 54192 (0.0007) [2023-10-07 21:57:19,856][67871] Updated weights for policy 1, policy_version 54250 (0.0008) [2023-10-07 21:57:19,892][67838] Updated weights for policy 0, policy_version 54202 (0.0007) [2023-10-07 21:57:20,226][67871] Updated weights for policy 1, policy_version 54260 (0.0009) [2023-10-07 21:57:20,587][67871] Updated weights for policy 1, policy_version 54270 (0.0008) [2023-10-07 21:57:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111083520. Throughput: 0: 1655.3, 1: 1652.5. Samples: 27780428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:57:22,477][66916] Avg episode reward: [(0, '57.160'), (1, '55.620')] [2023-10-07 21:57:24,083][67838] Updated weights for policy 0, policy_version 54212 (0.0008) [2023-10-07 21:57:24,449][67838] Updated weights for policy 0, policy_version 54222 (0.0009) [2023-10-07 21:57:24,649][67871] Updated weights for policy 1, policy_version 54280 (0.0008) [2023-10-07 21:57:24,824][67838] Updated weights for policy 0, policy_version 54232 (0.0007) [2023-10-07 21:57:25,012][67871] Updated weights for policy 1, policy_version 54290 (0.0007) [2023-10-07 21:57:25,378][67871] Updated weights for policy 1, policy_version 54300 (0.0008) [2023-10-07 21:57:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111149056. Throughput: 0: 1646.9, 1: 1646.4. Samples: 27790550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:57:27,477][66916] Avg episode reward: [(0, '54.640'), (1, '55.930')] [2023-10-07 21:57:29,045][67838] Updated weights for policy 0, policy_version 54242 (0.0008) [2023-10-07 21:57:29,422][67838] Updated weights for policy 0, policy_version 54252 (0.0009) [2023-10-07 21:57:29,539][67871] Updated weights for policy 1, policy_version 54310 (0.0008) [2023-10-07 21:57:29,791][67838] Updated weights for policy 0, policy_version 54262 (0.0008) [2023-10-07 21:57:29,902][67871] Updated weights for policy 1, policy_version 54320 (0.0007) [2023-10-07 21:57:30,164][67838] Updated weights for policy 0, policy_version 54272 (0.0008) [2023-10-07 21:57:30,269][67871] Updated weights for policy 1, policy_version 54330 (0.0009) [2023-10-07 21:57:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111214592. Throughput: 0: 1663.1, 1: 1657.0. Samples: 27809978. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:57:32,477][66916] Avg episode reward: [(0, '58.650'), (1, '57.700')] [2023-10-07 21:57:34,248][67838] Updated weights for policy 0, policy_version 54282 (0.0009) [2023-10-07 21:57:34,376][67871] Updated weights for policy 1, policy_version 54340 (0.0008) [2023-10-07 21:57:34,620][67838] Updated weights for policy 0, policy_version 54292 (0.0007) [2023-10-07 21:57:34,753][67871] Updated weights for policy 1, policy_version 54350 (0.0007) [2023-10-07 21:57:34,996][67838] Updated weights for policy 0, policy_version 54302 (0.0007) [2023-10-07 21:57:35,124][67871] Updated weights for policy 1, policy_version 54360 (0.0009) [2023-10-07 21:57:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 111280128. Throughput: 0: 1663.6, 1: 1657.6. Samples: 27830450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:57:37,477][66916] Avg episode reward: [(0, '57.010'), (1, '53.870')] [2023-10-07 21:57:39,113][67838] Updated weights for policy 0, policy_version 54312 (0.0007) [2023-10-07 21:57:39,375][67871] Updated weights for policy 1, policy_version 54370 (0.0008) [2023-10-07 21:57:39,479][67838] Updated weights for policy 0, policy_version 54322 (0.0009) [2023-10-07 21:57:39,745][67871] Updated weights for policy 1, policy_version 54380 (0.0009) [2023-10-07 21:57:39,849][67838] Updated weights for policy 0, policy_version 54332 (0.0008) [2023-10-07 21:57:40,117][67871] Updated weights for policy 1, policy_version 54390 (0.0009) [2023-10-07 21:57:40,489][67871] Updated weights for policy 1, policy_version 54400 (0.0009) [2023-10-07 21:57:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111345664. Throughput: 0: 1651.4, 1: 1649.9. Samples: 27840128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:57:42,478][66916] Avg episode reward: [(0, '57.720'), (1, '55.840')] [2023-10-07 21:57:44,099][67838] Updated weights for policy 0, policy_version 54342 (0.0008) [2023-10-07 21:57:44,475][67838] Updated weights for policy 0, policy_version 54352 (0.0009) [2023-10-07 21:57:44,521][67871] Updated weights for policy 1, policy_version 54410 (0.0007) [2023-10-07 21:57:44,831][67838] Updated weights for policy 0, policy_version 54362 (0.0008) [2023-10-07 21:57:44,880][67871] Updated weights for policy 1, policy_version 54420 (0.0008) [2023-10-07 21:57:45,250][67871] Updated weights for policy 1, policy_version 54430 (0.0009) [2023-10-07 21:57:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111411200. Throughput: 0: 1663.0, 1: 1657.7. Samples: 27859672. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 21:57:47,477][66916] Avg episode reward: [(0, '55.340'), (1, '53.190')] [2023-10-07 21:57:49,107][67838] Updated weights for policy 0, policy_version 54372 (0.0007) [2023-10-07 21:57:49,487][67838] Updated weights for policy 0, policy_version 54382 (0.0009) [2023-10-07 21:57:49,585][67871] Updated weights for policy 1, policy_version 54440 (0.0008) [2023-10-07 21:57:49,871][67838] Updated weights for policy 0, policy_version 54392 (0.0008) [2023-10-07 21:57:49,964][67871] Updated weights for policy 1, policy_version 54450 (0.0009) [2023-10-07 21:57:50,324][67871] Updated weights for policy 1, policy_version 54460 (0.0010) [2023-10-07 21:57:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111476736. Throughput: 0: 1657.9, 1: 1654.5. Samples: 27879686. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 21:57:52,477][66916] Avg episode reward: [(0, '55.950'), (1, '50.320')] [2023-10-07 21:57:53,950][67838] Updated weights for policy 0, policy_version 54402 (0.0010) [2023-10-07 21:57:54,333][67838] Updated weights for policy 0, policy_version 54412 (0.0009) [2023-10-07 21:57:54,387][67871] Updated weights for policy 1, policy_version 54470 (0.0007) [2023-10-07 21:57:54,711][67838] Updated weights for policy 0, policy_version 54422 (0.0007) [2023-10-07 21:57:54,761][67871] Updated weights for policy 1, policy_version 54480 (0.0007) [2023-10-07 21:57:55,077][67838] Updated weights for policy 0, policy_version 54432 (0.0008) [2023-10-07 21:57:55,132][67871] Updated weights for policy 1, policy_version 54490 (0.0007) [2023-10-07 21:57:57,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 111542272. Throughput: 0: 1652.3, 1: 1652.4. Samples: 27889562. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 21:57:57,477][66916] Avg episode reward: [(0, '56.870'), (1, '50.830')] [2023-10-07 21:57:59,055][67838] Updated weights for policy 0, policy_version 54442 (0.0008) [2023-10-07 21:57:59,310][67871] Updated weights for policy 1, policy_version 54500 (0.0007) [2023-10-07 21:57:59,435][67838] Updated weights for policy 0, policy_version 54452 (0.0009) [2023-10-07 21:57:59,685][67871] Updated weights for policy 1, policy_version 54510 (0.0008) [2023-10-07 21:57:59,803][67838] Updated weights for policy 0, policy_version 54462 (0.0010) [2023-10-07 21:58:00,054][67871] Updated weights for policy 1, policy_version 54520 (0.0008) [2023-10-07 21:58:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 111607808. Throughput: 0: 1660.4, 1: 1661.6. Samples: 27909414. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 21:58:02,478][66916] Avg episode reward: [(0, '56.100'), (1, '49.480')] [2023-10-07 21:58:04,048][67838] Updated weights for policy 0, policy_version 54472 (0.0010) [2023-10-07 21:58:04,180][67871] Updated weights for policy 1, policy_version 54530 (0.0008) [2023-10-07 21:58:04,419][67838] Updated weights for policy 0, policy_version 54482 (0.0007) [2023-10-07 21:58:04,553][67871] Updated weights for policy 1, policy_version 54540 (0.0007) [2023-10-07 21:58:04,786][67838] Updated weights for policy 0, policy_version 54492 (0.0007) [2023-10-07 21:58:04,921][67871] Updated weights for policy 1, policy_version 54550 (0.0008) [2023-10-07 21:58:05,288][67871] Updated weights for policy 1, policy_version 54560 (0.0008) [2023-10-07 21:58:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.3). Total num frames: 111673344. Throughput: 0: 1657.8, 1: 1658.8. Samples: 27929678. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 21:58:07,477][66916] Avg episode reward: [(0, '55.510'), (1, '52.400')] [2023-10-07 21:58:08,902][67838] Updated weights for policy 0, policy_version 54502 (0.0010) [2023-10-07 21:58:09,271][67838] Updated weights for policy 0, policy_version 54512 (0.0008) [2023-10-07 21:58:09,531][67871] Updated weights for policy 1, policy_version 54570 (0.0007) [2023-10-07 21:58:09,644][67838] Updated weights for policy 0, policy_version 54522 (0.0007) [2023-10-07 21:58:09,897][67871] Updated weights for policy 1, policy_version 54580 (0.0009) [2023-10-07 21:58:10,261][67871] Updated weights for policy 1, policy_version 54590 (0.0010) [2023-10-07 21:58:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111738880. Throughput: 0: 1647.7, 1: 1650.2. Samples: 27938956. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 21:58:12,477][66916] Avg episode reward: [(0, '55.670'), (1, '51.920')] [2023-10-07 21:58:13,828][67838] Updated weights for policy 0, policy_version 54532 (0.0009) [2023-10-07 21:58:14,203][67838] Updated weights for policy 0, policy_version 54542 (0.0010) [2023-10-07 21:58:14,461][67871] Updated weights for policy 1, policy_version 54600 (0.0008) [2023-10-07 21:58:14,572][67838] Updated weights for policy 0, policy_version 54552 (0.0007) [2023-10-07 21:58:14,831][67871] Updated weights for policy 1, policy_version 54610 (0.0007) [2023-10-07 21:58:15,193][67871] Updated weights for policy 1, policy_version 54620 (0.0010) [2023-10-07 21:58:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111804416. Throughput: 0: 1652.7, 1: 1653.2. Samples: 27958748. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 21:58:17,478][66916] Avg episode reward: [(0, '55.460'), (1, '51.550')] [2023-10-07 21:58:18,568][67838] Updated weights for policy 0, policy_version 54562 (0.0008) [2023-10-07 21:58:18,941][67838] Updated weights for policy 0, policy_version 54572 (0.0009) [2023-10-07 21:58:19,312][67838] Updated weights for policy 0, policy_version 54582 (0.0008) [2023-10-07 21:58:19,323][67871] Updated weights for policy 1, policy_version 54630 (0.0008) [2023-10-07 21:58:19,682][67838] Updated weights for policy 0, policy_version 54592 (0.0007) [2023-10-07 21:58:19,693][67871] Updated weights for policy 1, policy_version 54640 (0.0008) [2023-10-07 21:58:20,062][67871] Updated weights for policy 1, policy_version 54650 (0.0010) [2023-10-07 21:58:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111869952. Throughput: 0: 1660.2, 1: 1655.0. Samples: 27979634. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 21:58:22,477][66916] Avg episode reward: [(0, '53.490'), (1, '54.210')] [2023-10-07 21:58:23,625][67838] Updated weights for policy 0, policy_version 54602 (0.0010) [2023-10-07 21:58:23,974][67871] Updated weights for policy 1, policy_version 54660 (0.0009) [2023-10-07 21:58:23,998][67838] Updated weights for policy 0, policy_version 54612 (0.0009) [2023-10-07 21:58:24,339][67871] Updated weights for policy 1, policy_version 54670 (0.0008) [2023-10-07 21:58:24,378][67838] Updated weights for policy 0, policy_version 54622 (0.0009) [2023-10-07 21:58:24,702][67871] Updated weights for policy 1, policy_version 54680 (0.0008) [2023-10-07 21:58:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111935488. Throughput: 0: 1659.4, 1: 1649.4. Samples: 27989024. Policy #0 lag: (min: 12.0, avg: 15.0, max: 44.0) [2023-10-07 21:58:27,477][66916] Avg episode reward: [(0, '57.860'), (1, '53.400')] [2023-10-07 21:58:28,626][67838] Updated weights for policy 0, policy_version 54632 (0.0007) [2023-10-07 21:58:28,920][67871] Updated weights for policy 1, policy_version 54690 (0.0010) [2023-10-07 21:58:28,994][67838] Updated weights for policy 0, policy_version 54642 (0.0009) [2023-10-07 21:58:29,288][67871] Updated weights for policy 1, policy_version 54700 (0.0008) [2023-10-07 21:58:29,365][67838] Updated weights for policy 0, policy_version 54652 (0.0008) [2023-10-07 21:58:29,645][67871] Updated weights for policy 1, policy_version 54710 (0.0007) [2023-10-07 21:58:30,021][67871] Updated weights for policy 1, policy_version 54720 (0.0008) [2023-10-07 21:58:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 112001024. Throughput: 0: 1666.9, 1: 1653.2. Samples: 28009076. Policy #0 lag: (min: 12.0, avg: 15.0, max: 44.0) [2023-10-07 21:58:32,478][66916] Avg episode reward: [(0, '56.950'), (1, '53.220')] [2023-10-07 21:58:33,529][67838] Updated weights for policy 0, policy_version 54662 (0.0008) [2023-10-07 21:58:33,916][67838] Updated weights for policy 0, policy_version 54672 (0.0007) [2023-10-07 21:58:34,050][67871] Updated weights for policy 1, policy_version 54730 (0.0010) [2023-10-07 21:58:34,283][67838] Updated weights for policy 0, policy_version 54682 (0.0007) [2023-10-07 21:58:34,414][67871] Updated weights for policy 1, policy_version 54740 (0.0008) [2023-10-07 21:58:34,786][67871] Updated weights for policy 1, policy_version 54750 (0.0009) [2023-10-07 21:58:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 112066560. Throughput: 0: 1670.9, 1: 1659.2. Samples: 28029542. Policy #0 lag: (min: 12.0, avg: 15.0, max: 44.0) [2023-10-07 21:58:37,477][66916] Avg episode reward: [(0, '55.690'), (1, '53.890')] [2023-10-07 21:58:38,291][67838] Updated weights for policy 0, policy_version 54692 (0.0008) [2023-10-07 21:58:38,665][67838] Updated weights for policy 0, policy_version 54702 (0.0010) [2023-10-07 21:58:38,932][67871] Updated weights for policy 1, policy_version 54760 (0.0008) [2023-10-07 21:58:39,030][67838] Updated weights for policy 0, policy_version 54712 (0.0009) [2023-10-07 21:58:39,298][67871] Updated weights for policy 1, policy_version 54770 (0.0008) [2023-10-07 21:58:39,662][67871] Updated weights for policy 1, policy_version 54780 (0.0009) [2023-10-07 21:58:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 112132096. Throughput: 0: 1666.5, 1: 1642.1. Samples: 28038448. Policy #0 lag: (min: 12.0, avg: 15.0, max: 44.0) [2023-10-07 21:58:42,478][66916] Avg episode reward: [(0, '54.440'), (1, '55.290')] [2023-10-07 21:58:43,406][67838] Updated weights for policy 0, policy_version 54722 (0.0008) [2023-10-07 21:58:43,768][67871] Updated weights for policy 1, policy_version 54790 (0.0009) [2023-10-07 21:58:43,771][67838] Updated weights for policy 0, policy_version 54732 (0.0009) [2023-10-07 21:58:44,141][67871] Updated weights for policy 1, policy_version 54800 (0.0008) [2023-10-07 21:58:44,156][67838] Updated weights for policy 0, policy_version 54742 (0.0010) [2023-10-07 21:58:44,497][67871] Updated weights for policy 1, policy_version 54810 (0.0008) [2023-10-07 21:58:44,516][67838] Updated weights for policy 0, policy_version 54752 (0.0007) [2023-10-07 21:58:47,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 112197632. Throughput: 0: 1659.5, 1: 1655.2. Samples: 28058576. Policy #0 lag: (min: 12.0, avg: 15.0, max: 44.0) [2023-10-07 21:58:47,478][66916] Avg episode reward: [(0, '55.010'), (1, '54.250')] [2023-10-07 21:58:48,652][67838] Updated weights for policy 0, policy_version 54762 (0.0010) [2023-10-07 21:58:48,664][67871] Updated weights for policy 1, policy_version 54820 (0.0008) [2023-10-07 21:58:49,024][67838] Updated weights for policy 0, policy_version 54772 (0.0008) [2023-10-07 21:58:49,031][67871] Updated weights for policy 1, policy_version 54830 (0.0007) [2023-10-07 21:58:49,392][67838] Updated weights for policy 0, policy_version 54782 (0.0010) [2023-10-07 21:58:49,394][67871] Updated weights for policy 1, policy_version 54840 (0.0008) [2023-10-07 21:58:52,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112263168. Throughput: 0: 1655.7, 1: 1661.3. Samples: 28078944. Policy #0 lag: (min: 12.0, avg: 15.0, max: 44.0) [2023-10-07 21:58:52,478][66916] Avg episode reward: [(0, '53.480'), (1, '55.890')] [2023-10-07 21:58:53,452][67871] Updated weights for policy 1, policy_version 54850 (0.0008) [2023-10-07 21:58:53,578][67838] Updated weights for policy 0, policy_version 54792 (0.0008) [2023-10-07 21:58:53,808][67871] Updated weights for policy 1, policy_version 54860 (0.0009) [2023-10-07 21:58:53,950][67838] Updated weights for policy 0, policy_version 54802 (0.0009) [2023-10-07 21:58:54,176][67871] Updated weights for policy 1, policy_version 54870 (0.0008) [2023-10-07 21:58:54,324][67838] Updated weights for policy 0, policy_version 54812 (0.0010) [2023-10-07 21:58:54,544][67871] Updated weights for policy 1, policy_version 54880 (0.0010) [2023-10-07 21:58:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112328704. Throughput: 0: 1657.4, 1: 1650.7. Samples: 28087822. Policy #0 lag: (min: 12.0, avg: 15.0, max: 44.0) [2023-10-07 21:58:57,477][66916] Avg episode reward: [(0, '53.100'), (1, '56.040')] [2023-10-07 21:58:58,326][67838] Updated weights for policy 0, policy_version 54822 (0.0007) [2023-10-07 21:58:58,699][67838] Updated weights for policy 0, policy_version 54832 (0.0007) [2023-10-07 21:58:58,926][67871] Updated weights for policy 1, policy_version 54890 (0.0007) [2023-10-07 21:58:59,063][67838] Updated weights for policy 0, policy_version 54842 (0.0007) [2023-10-07 21:58:59,298][67871] Updated weights for policy 1, policy_version 54900 (0.0009) [2023-10-07 21:58:59,663][67871] Updated weights for policy 1, policy_version 54910 (0.0009) [2023-10-07 21:59:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112394240. Throughput: 0: 1660.3, 1: 1660.2. Samples: 28108170. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:59:02,477][66916] Avg episode reward: [(0, '52.530'), (1, '55.840')] [2023-10-07 21:59:03,278][67838] Updated weights for policy 0, policy_version 54852 (0.0011) [2023-10-07 21:59:03,641][67838] Updated weights for policy 0, policy_version 54862 (0.0009) [2023-10-07 21:59:03,701][67871] Updated weights for policy 1, policy_version 54920 (0.0007) [2023-10-07 21:59:04,006][67838] Updated weights for policy 0, policy_version 54872 (0.0009) [2023-10-07 21:59:04,062][67871] Updated weights for policy 1, policy_version 54930 (0.0007) [2023-10-07 21:59:04,425][67871] Updated weights for policy 1, policy_version 54940 (0.0009) [2023-10-07 21:59:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112459776. Throughput: 0: 1655.2, 1: 1657.1. Samples: 28128686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:59:07,478][66916] Avg episode reward: [(0, '52.460'), (1, '53.950')] [2023-10-07 21:59:07,488][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000054944_56262656.pth... [2023-10-07 21:59:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000054880_56197120.pth... [2023-10-07 21:59:07,519][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000053408_54689792.pth [2023-10-07 21:59:07,531][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000053344_54624256.pth [2023-10-07 21:59:08,034][67838] Updated weights for policy 0, policy_version 54882 (0.0009) [2023-10-07 21:59:08,406][67838] Updated weights for policy 0, policy_version 54892 (0.0009) [2023-10-07 21:59:08,781][67838] Updated weights for policy 0, policy_version 54902 (0.0008) [2023-10-07 21:59:08,797][67871] Updated weights for policy 1, policy_version 54950 (0.0007) [2023-10-07 21:59:09,145][67838] Updated weights for policy 0, policy_version 54912 (0.0008) [2023-10-07 21:59:09,177][67871] Updated weights for policy 1, policy_version 54960 (0.0009) [2023-10-07 21:59:09,546][67871] Updated weights for policy 1, policy_version 54970 (0.0008) [2023-10-07 21:59:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112525312. Throughput: 0: 1654.4, 1: 1644.1. Samples: 28137456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:59:12,477][66916] Avg episode reward: [(0, '49.570'), (1, '51.980')] [2023-10-07 21:59:13,384][67838] Updated weights for policy 0, policy_version 54922 (0.0009) [2023-10-07 21:59:13,483][67871] Updated weights for policy 1, policy_version 54980 (0.0007) [2023-10-07 21:59:13,758][67838] Updated weights for policy 0, policy_version 54932 (0.0008) [2023-10-07 21:59:13,846][67871] Updated weights for policy 1, policy_version 54990 (0.0007) [2023-10-07 21:59:14,129][67838] Updated weights for policy 0, policy_version 54942 (0.0009) [2023-10-07 21:59:14,213][67871] Updated weights for policy 1, policy_version 55000 (0.0009) [2023-10-07 21:59:17,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 112590848. Throughput: 0: 1648.9, 1: 1653.9. Samples: 28157698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:59:17,477][66916] Avg episode reward: [(0, '50.930'), (1, '50.920')] [2023-10-07 21:59:18,333][67838] Updated weights for policy 0, policy_version 54952 (0.0009) [2023-10-07 21:59:18,394][67871] Updated weights for policy 1, policy_version 55010 (0.0008) [2023-10-07 21:59:18,694][67838] Updated weights for policy 0, policy_version 54962 (0.0009) [2023-10-07 21:59:18,755][67871] Updated weights for policy 1, policy_version 55020 (0.0007) [2023-10-07 21:59:19,062][67838] Updated weights for policy 0, policy_version 54972 (0.0009) [2023-10-07 21:59:19,120][67871] Updated weights for policy 1, policy_version 55030 (0.0008) [2023-10-07 21:59:19,488][67871] Updated weights for policy 1, policy_version 55040 (0.0008) [2023-10-07 21:59:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 112656384. Throughput: 0: 1646.8, 1: 1654.1. Samples: 28178082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:59:22,477][66916] Avg episode reward: [(0, '50.570'), (1, '49.130')] [2023-10-07 21:59:23,391][67838] Updated weights for policy 0, policy_version 54982 (0.0009) [2023-10-07 21:59:23,575][67871] Updated weights for policy 1, policy_version 55050 (0.0008) [2023-10-07 21:59:23,774][67838] Updated weights for policy 0, policy_version 54992 (0.0008) [2023-10-07 21:59:23,943][67871] Updated weights for policy 1, policy_version 55060 (0.0010) [2023-10-07 21:59:24,147][67838] Updated weights for policy 0, policy_version 55002 (0.0008) [2023-10-07 21:59:24,314][67871] Updated weights for policy 1, policy_version 55070 (0.0008) [2023-10-07 21:59:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 112721920. Throughput: 0: 1644.0, 1: 1653.5. Samples: 28186838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:59:27,478][66916] Avg episode reward: [(0, '47.040'), (1, '47.450')] [2023-10-07 21:59:28,439][67838] Updated weights for policy 0, policy_version 55012 (0.0008) [2023-10-07 21:59:28,584][67871] Updated weights for policy 1, policy_version 55080 (0.0009) [2023-10-07 21:59:28,807][67838] Updated weights for policy 0, policy_version 55022 (0.0011) [2023-10-07 21:59:28,951][67871] Updated weights for policy 1, policy_version 55090 (0.0009) [2023-10-07 21:59:29,170][67838] Updated weights for policy 0, policy_version 55032 (0.0010) [2023-10-07 21:59:29,317][67871] Updated weights for policy 1, policy_version 55100 (0.0009) [2023-10-07 21:59:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 112787456. Throughput: 0: 1643.7, 1: 1652.2. Samples: 28206892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 21:59:32,478][66916] Avg episode reward: [(0, '49.810'), (1, '47.950')] [2023-10-07 21:59:33,415][67838] Updated weights for policy 0, policy_version 55042 (0.0009) [2023-10-07 21:59:33,427][67871] Updated weights for policy 1, policy_version 55110 (0.0009) [2023-10-07 21:59:33,781][67838] Updated weights for policy 0, policy_version 55052 (0.0007) [2023-10-07 21:59:33,787][67871] Updated weights for policy 1, policy_version 55120 (0.0009) [2023-10-07 21:59:34,153][67871] Updated weights for policy 1, policy_version 55130 (0.0007) [2023-10-07 21:59:34,154][67838] Updated weights for policy 0, policy_version 55062 (0.0007) [2023-10-07 21:59:34,522][67838] Updated weights for policy 0, policy_version 55072 (0.0008) [2023-10-07 21:59:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 112852992. Throughput: 0: 1648.8, 1: 1648.3. Samples: 28227312. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-07 21:59:37,477][66916] Avg episode reward: [(0, '48.370'), (1, '49.840')] [2023-10-07 21:59:38,433][67871] Updated weights for policy 1, policy_version 55140 (0.0008) [2023-10-07 21:59:38,480][67838] Updated weights for policy 0, policy_version 55082 (0.0008) [2023-10-07 21:59:38,790][67871] Updated weights for policy 1, policy_version 55150 (0.0007) [2023-10-07 21:59:38,843][67838] Updated weights for policy 0, policy_version 55092 (0.0008) [2023-10-07 21:59:39,153][67871] Updated weights for policy 1, policy_version 55160 (0.0008) [2023-10-07 21:59:39,216][67838] Updated weights for policy 0, policy_version 55102 (0.0007) [2023-10-07 21:59:42,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 112918528. Throughput: 0: 1648.0, 1: 1648.7. Samples: 28236174. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-07 21:59:42,477][66916] Avg episode reward: [(0, '49.400'), (1, '51.650')] [2023-10-07 21:59:43,260][67871] Updated weights for policy 1, policy_version 55170 (0.0008) [2023-10-07 21:59:43,322][67838] Updated weights for policy 0, policy_version 55112 (0.0009) [2023-10-07 21:59:43,625][67871] Updated weights for policy 1, policy_version 55180 (0.0009) [2023-10-07 21:59:43,688][67838] Updated weights for policy 0, policy_version 55122 (0.0008) [2023-10-07 21:59:43,988][67871] Updated weights for policy 1, policy_version 55190 (0.0008) [2023-10-07 21:59:44,055][67838] Updated weights for policy 0, policy_version 55132 (0.0008) [2023-10-07 21:59:44,351][67871] Updated weights for policy 1, policy_version 55200 (0.0009) [2023-10-07 21:59:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 112984064. Throughput: 0: 1649.6, 1: 1648.4. Samples: 28256582. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-07 21:59:47,477][66916] Avg episode reward: [(0, '43.900'), (1, '53.500')] [2023-10-07 21:59:48,026][67838] Updated weights for policy 0, policy_version 55142 (0.0007) [2023-10-07 21:59:48,335][67871] Updated weights for policy 1, policy_version 55210 (0.0009) [2023-10-07 21:59:48,396][67838] Updated weights for policy 0, policy_version 55152 (0.0008) [2023-10-07 21:59:48,701][67871] Updated weights for policy 1, policy_version 55220 (0.0008) [2023-10-07 21:59:48,762][67838] Updated weights for policy 0, policy_version 55162 (0.0007) [2023-10-07 21:59:49,072][67871] Updated weights for policy 1, policy_version 55230 (0.0009) [2023-10-07 21:59:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113049600. Throughput: 0: 1649.8, 1: 1650.9. Samples: 28277216. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-07 21:59:52,477][66916] Avg episode reward: [(0, '44.070'), (1, '54.410')] [2023-10-07 21:59:52,914][67838] Updated weights for policy 0, policy_version 55172 (0.0007) [2023-10-07 21:59:53,284][67838] Updated weights for policy 0, policy_version 55182 (0.0007) [2023-10-07 21:59:53,444][67871] Updated weights for policy 1, policy_version 55240 (0.0008) [2023-10-07 21:59:53,651][67838] Updated weights for policy 0, policy_version 55192 (0.0007) [2023-10-07 21:59:53,817][67871] Updated weights for policy 1, policy_version 55250 (0.0007) [2023-10-07 21:59:54,172][67871] Updated weights for policy 1, policy_version 55260 (0.0008) [2023-10-07 21:59:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113115136. Throughput: 0: 1649.7, 1: 1654.4. Samples: 28286140. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-07 21:59:57,477][66916] Avg episode reward: [(0, '47.740'), (1, '58.120')] [2023-10-07 21:59:57,666][67838] Updated weights for policy 0, policy_version 55202 (0.0008) [2023-10-07 21:59:58,036][67838] Updated weights for policy 0, policy_version 55212 (0.0008) [2023-10-07 21:59:58,180][67871] Updated weights for policy 1, policy_version 55270 (0.0010) [2023-10-07 21:59:58,400][67838] Updated weights for policy 0, policy_version 55222 (0.0009) [2023-10-07 21:59:58,548][67871] Updated weights for policy 1, policy_version 55280 (0.0010) [2023-10-07 21:59:58,780][67838] Updated weights for policy 0, policy_version 55232 (0.0008) [2023-10-07 21:59:58,914][67871] Updated weights for policy 1, policy_version 55290 (0.0007) [2023-10-07 22:00:02,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 113180672. Throughput: 0: 1656.9, 1: 1658.5. Samples: 28306894. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-07 22:00:02,478][66916] Avg episode reward: [(0, '50.840'), (1, '55.740')] [2023-10-07 22:00:03,029][67871] Updated weights for policy 1, policy_version 55300 (0.0007) [2023-10-07 22:00:03,034][67838] Updated weights for policy 0, policy_version 55242 (0.0009) [2023-10-07 22:00:03,402][67871] Updated weights for policy 1, policy_version 55310 (0.0007) [2023-10-07 22:00:03,404][67838] Updated weights for policy 0, policy_version 55252 (0.0008) [2023-10-07 22:00:03,770][67871] Updated weights for policy 1, policy_version 55320 (0.0009) [2023-10-07 22:00:03,786][67838] Updated weights for policy 0, policy_version 55262 (0.0010) [2023-10-07 22:00:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 113246208. Throughput: 0: 1660.0, 1: 1653.8. Samples: 28327206. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-07 22:00:07,477][66916] Avg episode reward: [(0, '52.420'), (1, '56.400')] [2023-10-07 22:00:07,913][67838] Updated weights for policy 0, policy_version 55272 (0.0009) [2023-10-07 22:00:08,064][67871] Updated weights for policy 1, policy_version 55330 (0.0007) [2023-10-07 22:00:08,282][67838] Updated weights for policy 0, policy_version 55282 (0.0009) [2023-10-07 22:00:08,435][67871] Updated weights for policy 1, policy_version 55340 (0.0008) [2023-10-07 22:00:08,661][67838] Updated weights for policy 0, policy_version 55292 (0.0009) [2023-10-07 22:00:08,792][67871] Updated weights for policy 1, policy_version 55350 (0.0009) [2023-10-07 22:00:09,159][67871] Updated weights for policy 1, policy_version 55360 (0.0008) [2023-10-07 22:00:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113311744. Throughput: 0: 1662.5, 1: 1655.6. Samples: 28336156. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-07 22:00:12,478][66916] Avg episode reward: [(0, '50.370'), (1, '57.790')] [2023-10-07 22:00:12,729][67838] Updated weights for policy 0, policy_version 55302 (0.0009) [2023-10-07 22:00:13,069][67871] Updated weights for policy 1, policy_version 55370 (0.0008) [2023-10-07 22:00:13,099][67838] Updated weights for policy 0, policy_version 55312 (0.0010) [2023-10-07 22:00:13,437][67871] Updated weights for policy 1, policy_version 55380 (0.0008) [2023-10-07 22:00:13,479][67838] Updated weights for policy 0, policy_version 55322 (0.0007) [2023-10-07 22:00:13,804][67871] Updated weights for policy 1, policy_version 55390 (0.0009) [2023-10-07 22:00:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113377280. Throughput: 0: 1670.9, 1: 1661.1. Samples: 28356834. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-07 22:00:17,477][66916] Avg episode reward: [(0, '50.830'), (1, '56.210')] [2023-10-07 22:00:17,598][67838] Updated weights for policy 0, policy_version 55332 (0.0008) [2023-10-07 22:00:17,970][67838] Updated weights for policy 0, policy_version 55342 (0.0008) [2023-10-07 22:00:18,006][67871] Updated weights for policy 1, policy_version 55400 (0.0009) [2023-10-07 22:00:18,342][67838] Updated weights for policy 0, policy_version 55352 (0.0008) [2023-10-07 22:00:18,372][67871] Updated weights for policy 1, policy_version 55410 (0.0009) [2023-10-07 22:00:18,736][67871] Updated weights for policy 1, policy_version 55420 (0.0008) [2023-10-07 22:00:22,320][67838] Updated weights for policy 0, policy_version 55362 (0.0008) [2023-10-07 22:00:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113442816. Throughput: 0: 1673.1, 1: 1666.0. Samples: 28377574. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-07 22:00:22,477][66916] Avg episode reward: [(0, '48.770'), (1, '53.060')] [2023-10-07 22:00:22,687][67871] Updated weights for policy 1, policy_version 55430 (0.0009) [2023-10-07 22:00:22,691][67838] Updated weights for policy 0, policy_version 55372 (0.0007) [2023-10-07 22:00:23,046][67871] Updated weights for policy 1, policy_version 55440 (0.0008) [2023-10-07 22:00:23,061][67838] Updated weights for policy 0, policy_version 55382 (0.0008) [2023-10-07 22:00:23,414][67871] Updated weights for policy 1, policy_version 55450 (0.0007) [2023-10-07 22:00:23,424][67838] Updated weights for policy 0, policy_version 55392 (0.0007) [2023-10-07 22:00:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113508352. Throughput: 0: 1676.4, 1: 1667.8. Samples: 28386664. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-07 22:00:27,477][66916] Avg episode reward: [(0, '49.950'), (1, '53.200')] [2023-10-07 22:00:27,513][67838] Updated weights for policy 0, policy_version 55402 (0.0008) [2023-10-07 22:00:27,652][67871] Updated weights for policy 1, policy_version 55460 (0.0007) [2023-10-07 22:00:27,881][67838] Updated weights for policy 0, policy_version 55412 (0.0008) [2023-10-07 22:00:28,016][67871] Updated weights for policy 1, policy_version 55470 (0.0007) [2023-10-07 22:00:28,251][67838] Updated weights for policy 0, policy_version 55422 (0.0010) [2023-10-07 22:00:28,385][67871] Updated weights for policy 1, policy_version 55480 (0.0007) [2023-10-07 22:00:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113573888. Throughput: 0: 1668.6, 1: 1667.4. Samples: 28406704. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-07 22:00:32,477][66916] Avg episode reward: [(0, '49.440'), (1, '52.710')] [2023-10-07 22:00:32,563][67838] Updated weights for policy 0, policy_version 55432 (0.0008) [2023-10-07 22:00:32,609][67871] Updated weights for policy 1, policy_version 55490 (0.0007) [2023-10-07 22:00:32,944][67838] Updated weights for policy 0, policy_version 55442 (0.0008) [2023-10-07 22:00:32,981][67871] Updated weights for policy 1, policy_version 55500 (0.0008) [2023-10-07 22:00:33,317][67838] Updated weights for policy 0, policy_version 55452 (0.0007) [2023-10-07 22:00:33,350][67871] Updated weights for policy 1, policy_version 55510 (0.0007) [2023-10-07 22:00:33,720][67871] Updated weights for policy 1, policy_version 55520 (0.0008) [2023-10-07 22:00:37,423][67838] Updated weights for policy 0, policy_version 55462 (0.0009) [2023-10-07 22:00:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113639424. Throughput: 0: 1665.2, 1: 1665.0. Samples: 28427076. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-07 22:00:37,478][66916] Avg episode reward: [(0, '52.870'), (1, '52.090')] [2023-10-07 22:00:37,696][67871] Updated weights for policy 1, policy_version 55530 (0.0009) [2023-10-07 22:00:37,793][67838] Updated weights for policy 0, policy_version 55472 (0.0008) [2023-10-07 22:00:38,070][67871] Updated weights for policy 1, policy_version 55540 (0.0009) [2023-10-07 22:00:38,167][67838] Updated weights for policy 0, policy_version 55482 (0.0008) [2023-10-07 22:00:38,423][67871] Updated weights for policy 1, policy_version 55550 (0.0009) [2023-10-07 22:00:42,374][67838] Updated weights for policy 0, policy_version 55492 (0.0008) [2023-10-07 22:00:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113704960. Throughput: 0: 1665.3, 1: 1666.0. Samples: 28436052. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-07 22:00:42,477][66916] Avg episode reward: [(0, '55.070'), (1, '53.410')] [2023-10-07 22:00:42,742][67838] Updated weights for policy 0, policy_version 55502 (0.0009) [2023-10-07 22:00:42,781][67871] Updated weights for policy 1, policy_version 55560 (0.0008) [2023-10-07 22:00:43,113][67838] Updated weights for policy 0, policy_version 55512 (0.0009) [2023-10-07 22:00:43,155][67871] Updated weights for policy 1, policy_version 55570 (0.0007) [2023-10-07 22:00:43,519][67871] Updated weights for policy 1, policy_version 55580 (0.0008) [2023-10-07 22:00:47,191][67838] Updated weights for policy 0, policy_version 55522 (0.0007) [2023-10-07 22:00:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113770496. Throughput: 0: 1662.0, 1: 1659.6. Samples: 28456364. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-07 22:00:47,478][66916] Avg episode reward: [(0, '53.590'), (1, '52.210')] [2023-10-07 22:00:47,500][67871] Updated weights for policy 1, policy_version 55590 (0.0010) [2023-10-07 22:00:47,554][67838] Updated weights for policy 0, policy_version 55532 (0.0010) [2023-10-07 22:00:47,868][67871] Updated weights for policy 1, policy_version 55600 (0.0008) [2023-10-07 22:00:47,916][67838] Updated weights for policy 0, policy_version 55542 (0.0007) [2023-10-07 22:00:48,233][67871] Updated weights for policy 1, policy_version 55610 (0.0008) [2023-10-07 22:00:48,289][67838] Updated weights for policy 0, policy_version 55552 (0.0008) [2023-10-07 22:00:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113836032. Throughput: 0: 1664.0, 1: 1657.6. Samples: 28476680. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-07 22:00:52,477][66916] Avg episode reward: [(0, '58.260'), (1, '53.300')] [2023-10-07 22:00:52,499][67838] Updated weights for policy 0, policy_version 55562 (0.0007) [2023-10-07 22:00:52,590][67871] Updated weights for policy 1, policy_version 55620 (0.0008) [2023-10-07 22:00:52,866][67838] Updated weights for policy 0, policy_version 55572 (0.0009) [2023-10-07 22:00:52,953][67871] Updated weights for policy 1, policy_version 55630 (0.0009) [2023-10-07 22:00:53,244][67838] Updated weights for policy 0, policy_version 55582 (0.0008) [2023-10-07 22:00:53,324][67871] Updated weights for policy 1, policy_version 55640 (0.0008) [2023-10-07 22:00:57,044][67838] Updated weights for policy 0, policy_version 55592 (0.0007) [2023-10-07 22:00:57,128][67871] Updated weights for policy 1, policy_version 55650 (0.0009) [2023-10-07 22:00:57,426][67838] Updated weights for policy 0, policy_version 55602 (0.0007) [2023-10-07 22:00:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113901568. Throughput: 0: 1667.8, 1: 1662.1. Samples: 28486002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:00:57,477][66916] Avg episode reward: [(0, '58.600'), (1, '53.910')] [2023-10-07 22:00:57,493][67871] Updated weights for policy 1, policy_version 55660 (0.0008) [2023-10-07 22:00:57,791][67838] Updated weights for policy 0, policy_version 55612 (0.0009) [2023-10-07 22:00:57,852][67871] Updated weights for policy 1, policy_version 55670 (0.0009) [2023-10-07 22:00:58,220][67871] Updated weights for policy 1, policy_version 55680 (0.0010) [2023-10-07 22:01:01,911][67838] Updated weights for policy 0, policy_version 55622 (0.0008) [2023-10-07 22:01:02,273][67838] Updated weights for policy 0, policy_version 55632 (0.0007) [2023-10-07 22:01:02,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113967104. Throughput: 0: 1669.9, 1: 1661.0. Samples: 28506724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:01:02,477][66916] Avg episode reward: [(0, '59.310'), (1, '52.810')] [2023-10-07 22:01:02,521][67871] Updated weights for policy 1, policy_version 55690 (0.0008) [2023-10-07 22:01:02,641][67838] Updated weights for policy 0, policy_version 55642 (0.0009) [2023-10-07 22:01:02,880][67871] Updated weights for policy 1, policy_version 55700 (0.0009) [2023-10-07 22:01:03,249][67871] Updated weights for policy 1, policy_version 55710 (0.0009) [2023-10-07 22:01:06,792][67838] Updated weights for policy 0, policy_version 55652 (0.0008) [2023-10-07 22:01:07,160][67838] Updated weights for policy 0, policy_version 55662 (0.0008) [2023-10-07 22:01:07,309][67871] Updated weights for policy 1, policy_version 55720 (0.0007) [2023-10-07 22:01:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114032640. Throughput: 0: 1654.2, 1: 1663.9. Samples: 28526890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:01:07,477][66916] Avg episode reward: [(0, '54.790'), (1, '49.680')] [2023-10-07 22:01:07,531][67838] Updated weights for policy 0, policy_version 55672 (0.0009) [2023-10-07 22:01:07,681][67871] Updated weights for policy 1, policy_version 55730 (0.0007) [2023-10-07 22:01:07,824][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000055680_57016320.pth... [2023-10-07 22:01:07,857][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000054112_55410688.pth [2023-10-07 22:01:08,045][67871] Updated weights for policy 1, policy_version 55740 (0.0010) [2023-10-07 22:01:08,188][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000055744_57081856.pth... [2023-10-07 22:01:08,217][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000054176_55476224.pth [2023-10-07 22:01:11,604][67838] Updated weights for policy 0, policy_version 55682 (0.0009) [2023-10-07 22:01:11,965][67838] Updated weights for policy 0, policy_version 55692 (0.0008) [2023-10-07 22:01:12,139][67871] Updated weights for policy 1, policy_version 55750 (0.0010) [2023-10-07 22:01:12,335][67838] Updated weights for policy 0, policy_version 55702 (0.0007) [2023-10-07 22:01:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114098176. Throughput: 0: 1661.6, 1: 1661.8. Samples: 28536218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:01:12,477][66916] Avg episode reward: [(0, '57.280'), (1, '50.820')] [2023-10-07 22:01:12,514][67871] Updated weights for policy 1, policy_version 55760 (0.0008) [2023-10-07 22:01:12,701][67838] Updated weights for policy 0, policy_version 55712 (0.0007) [2023-10-07 22:01:12,887][67871] Updated weights for policy 1, policy_version 55770 (0.0007) [2023-10-07 22:01:16,975][67838] Updated weights for policy 0, policy_version 55722 (0.0008) [2023-10-07 22:01:17,124][67871] Updated weights for policy 1, policy_version 55780 (0.0008) [2023-10-07 22:01:17,345][67838] Updated weights for policy 0, policy_version 55732 (0.0010) [2023-10-07 22:01:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114163712. Throughput: 0: 1660.9, 1: 1663.3. Samples: 28556294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:01:17,477][66916] Avg episode reward: [(0, '54.710'), (1, '49.890')] [2023-10-07 22:01:17,494][67871] Updated weights for policy 1, policy_version 55790 (0.0007) [2023-10-07 22:01:17,720][67838] Updated weights for policy 0, policy_version 55742 (0.0008) [2023-10-07 22:01:17,872][67871] Updated weights for policy 1, policy_version 55800 (0.0008) [2023-10-07 22:01:21,892][67838] Updated weights for policy 0, policy_version 55752 (0.0009) [2023-10-07 22:01:21,971][67871] Updated weights for policy 1, policy_version 55810 (0.0010) [2023-10-07 22:01:22,271][67838] Updated weights for policy 0, policy_version 55762 (0.0010) [2023-10-07 22:01:22,331][67871] Updated weights for policy 1, policy_version 55820 (0.0009) [2023-10-07 22:01:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114229248. Throughput: 0: 1650.5, 1: 1664.6. Samples: 28576256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:01:22,477][66916] Avg episode reward: [(0, '54.920'), (1, '52.630')] [2023-10-07 22:01:22,642][67838] Updated weights for policy 0, policy_version 55772 (0.0008) [2023-10-07 22:01:22,696][67871] Updated weights for policy 1, policy_version 55830 (0.0008) [2023-10-07 22:01:23,072][67871] Updated weights for policy 1, policy_version 55840 (0.0010) [2023-10-07 22:01:26,861][67838] Updated weights for policy 0, policy_version 55782 (0.0010) [2023-10-07 22:01:27,229][67838] Updated weights for policy 0, policy_version 55792 (0.0010) [2023-10-07 22:01:27,262][67871] Updated weights for policy 1, policy_version 55850 (0.0007) [2023-10-07 22:01:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114294784. Throughput: 0: 1658.5, 1: 1663.7. Samples: 28585550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:01:27,477][66916] Avg episode reward: [(0, '54.110'), (1, '51.830')] [2023-10-07 22:01:27,591][67838] Updated weights for policy 0, policy_version 55802 (0.0009) [2023-10-07 22:01:27,625][67871] Updated weights for policy 1, policy_version 55860 (0.0007) [2023-10-07 22:01:28,000][67871] Updated weights for policy 1, policy_version 55870 (0.0009) [2023-10-07 22:01:31,648][67838] Updated weights for policy 0, policy_version 55812 (0.0008) [2023-10-07 22:01:32,014][67838] Updated weights for policy 0, policy_version 55822 (0.0009) [2023-10-07 22:01:32,168][67871] Updated weights for policy 1, policy_version 55880 (0.0008) [2023-10-07 22:01:32,382][67838] Updated weights for policy 0, policy_version 55832 (0.0010) [2023-10-07 22:01:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114360320. Throughput: 0: 1658.7, 1: 1657.5. Samples: 28605590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:01:32,478][66916] Avg episode reward: [(0, '54.150'), (1, '53.980')] [2023-10-07 22:01:32,529][67871] Updated weights for policy 1, policy_version 55890 (0.0007) [2023-10-07 22:01:32,898][67871] Updated weights for policy 1, policy_version 55900 (0.0007) [2023-10-07 22:01:36,500][67838] Updated weights for policy 0, policy_version 55842 (0.0008) [2023-10-07 22:01:36,877][67838] Updated weights for policy 0, policy_version 55852 (0.0007) [2023-10-07 22:01:36,898][67871] Updated weights for policy 1, policy_version 55910 (0.0009) [2023-10-07 22:01:37,255][67838] Updated weights for policy 0, policy_version 55862 (0.0007) [2023-10-07 22:01:37,267][67871] Updated weights for policy 1, policy_version 55920 (0.0007) [2023-10-07 22:01:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114425856. Throughput: 0: 1643.8, 1: 1665.3. Samples: 28625592. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 22:01:37,477][66916] Avg episode reward: [(0, '53.270'), (1, '55.300')] [2023-10-07 22:01:37,622][67838] Updated weights for policy 0, policy_version 55872 (0.0007) [2023-10-07 22:01:37,631][67871] Updated weights for policy 1, policy_version 55930 (0.0007) [2023-10-07 22:01:41,663][67838] Updated weights for policy 0, policy_version 55882 (0.0008) [2023-10-07 22:01:41,733][67871] Updated weights for policy 1, policy_version 55940 (0.0007) [2023-10-07 22:01:42,039][67838] Updated weights for policy 0, policy_version 55892 (0.0007) [2023-10-07 22:01:42,105][67871] Updated weights for policy 1, policy_version 55950 (0.0010) [2023-10-07 22:01:42,401][67838] Updated weights for policy 0, policy_version 55902 (0.0008) [2023-10-07 22:01:42,467][67871] Updated weights for policy 1, policy_version 55960 (0.0009) [2023-10-07 22:01:42,476][66916] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 114524160. Throughput: 0: 1656.6, 1: 1663.1. Samples: 28635390. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 22:01:42,477][66916] Avg episode reward: [(0, '54.150'), (1, '55.810')] [2023-10-07 22:01:46,606][67871] Updated weights for policy 1, policy_version 55970 (0.0007) [2023-10-07 22:01:46,626][67838] Updated weights for policy 0, policy_version 55912 (0.0007) [2023-10-07 22:01:46,969][67871] Updated weights for policy 1, policy_version 55980 (0.0007) [2023-10-07 22:01:47,001][67838] Updated weights for policy 0, policy_version 55922 (0.0008) [2023-10-07 22:01:47,344][67871] Updated weights for policy 1, policy_version 55990 (0.0008) [2023-10-07 22:01:47,374][67838] Updated weights for policy 0, policy_version 55932 (0.0007) [2023-10-07 22:01:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114556928. Throughput: 0: 1651.5, 1: 1658.9. Samples: 28655692. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 22:01:47,477][66916] Avg episode reward: [(0, '51.770'), (1, '54.950')] [2023-10-07 22:01:47,700][67871] Updated weights for policy 1, policy_version 56000 (0.0008) [2023-10-07 22:01:51,341][67838] Updated weights for policy 0, policy_version 55942 (0.0009) [2023-10-07 22:01:51,712][67838] Updated weights for policy 0, policy_version 55952 (0.0008) [2023-10-07 22:01:51,871][67871] Updated weights for policy 1, policy_version 56010 (0.0009) [2023-10-07 22:01:52,083][67838] Updated weights for policy 0, policy_version 55962 (0.0008) [2023-10-07 22:01:52,238][67871] Updated weights for policy 1, policy_version 56020 (0.0008) [2023-10-07 22:01:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 114655232. Throughput: 0: 1639.8, 1: 1644.8. Samples: 28674700. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 22:01:52,477][66916] Avg episode reward: [(0, '53.370'), (1, '51.570')] [2023-10-07 22:01:52,597][67871] Updated weights for policy 1, policy_version 56030 (0.0007) [2023-10-07 22:01:56,209][67838] Updated weights for policy 0, policy_version 55972 (0.0007) [2023-10-07 22:01:56,576][67838] Updated weights for policy 0, policy_version 55982 (0.0009) [2023-10-07 22:01:56,847][67871] Updated weights for policy 1, policy_version 56040 (0.0009) [2023-10-07 22:01:56,943][67838] Updated weights for policy 0, policy_version 55992 (0.0009) [2023-10-07 22:01:57,211][67871] Updated weights for policy 1, policy_version 56050 (0.0009) [2023-10-07 22:01:57,476][66916] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 114720768. Throughput: 0: 1652.5, 1: 1652.2. Samples: 28684930. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 22:01:57,477][66916] Avg episode reward: [(0, '55.360'), (1, '52.370')] [2023-10-07 22:01:57,582][67871] Updated weights for policy 1, policy_version 56060 (0.0009) [2023-10-07 22:02:01,319][67838] Updated weights for policy 0, policy_version 56002 (0.0008) [2023-10-07 22:02:01,697][67838] Updated weights for policy 0, policy_version 56012 (0.0010) [2023-10-07 22:02:01,885][67871] Updated weights for policy 1, policy_version 56070 (0.0007) [2023-10-07 22:02:02,065][67838] Updated weights for policy 0, policy_version 56022 (0.0008) [2023-10-07 22:02:02,234][67871] Updated weights for policy 1, policy_version 56080 (0.0007) [2023-10-07 22:02:02,440][67838] Updated weights for policy 0, policy_version 56032 (0.0007) [2023-10-07 22:02:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 114786304. Throughput: 0: 1651.5, 1: 1655.7. Samples: 28705118. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 22:02:02,477][66916] Avg episode reward: [(0, '57.150'), (1, '52.600')] [2023-10-07 22:02:02,607][67871] Updated weights for policy 1, policy_version 56090 (0.0008) [2023-10-07 22:02:06,577][67838] Updated weights for policy 0, policy_version 56042 (0.0008) [2023-10-07 22:02:06,603][67871] Updated weights for policy 1, policy_version 56100 (0.0008) [2023-10-07 22:02:06,946][67838] Updated weights for policy 0, policy_version 56052 (0.0009) [2023-10-07 22:02:06,973][67871] Updated weights for policy 1, policy_version 56110 (0.0008) [2023-10-07 22:02:07,313][67838] Updated weights for policy 0, policy_version 56062 (0.0007) [2023-10-07 22:02:07,342][67871] Updated weights for policy 1, policy_version 56120 (0.0008) [2023-10-07 22:02:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 114851840. Throughput: 0: 1645.0, 1: 1644.8. Samples: 28724296. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-07 22:02:07,477][66916] Avg episode reward: [(0, '54.670'), (1, '53.370')] [2023-10-07 22:02:11,280][67838] Updated weights for policy 0, policy_version 56072 (0.0008) [2023-10-07 22:02:11,579][67871] Updated weights for policy 1, policy_version 56130 (0.0008) [2023-10-07 22:02:11,643][67838] Updated weights for policy 0, policy_version 56082 (0.0009) [2023-10-07 22:02:11,947][67871] Updated weights for policy 1, policy_version 56140 (0.0008) [2023-10-07 22:02:12,024][67838] Updated weights for policy 0, policy_version 56092 (0.0007) [2023-10-07 22:02:12,311][67871] Updated weights for policy 1, policy_version 56150 (0.0010) [2023-10-07 22:02:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 114917376. Throughput: 0: 1661.0, 1: 1651.9. Samples: 28734628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:02:12,477][66916] Avg episode reward: [(0, '55.980'), (1, '52.130')] [2023-10-07 22:02:12,676][67871] Updated weights for policy 1, policy_version 56160 (0.0009) [2023-10-07 22:02:16,176][67838] Updated weights for policy 0, policy_version 56102 (0.0007) [2023-10-07 22:02:16,542][67838] Updated weights for policy 0, policy_version 56112 (0.0008) [2023-10-07 22:02:16,917][67838] Updated weights for policy 0, policy_version 56122 (0.0010) [2023-10-07 22:02:16,947][67871] Updated weights for policy 1, policy_version 56170 (0.0009) [2023-10-07 22:02:17,334][67871] Updated weights for policy 1, policy_version 56180 (0.0009) [2023-10-07 22:02:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 114982912. Throughput: 0: 1656.3, 1: 1662.1. Samples: 28754918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:02:17,477][66916] Avg episode reward: [(0, '57.490'), (1, '52.300')] [2023-10-07 22:02:17,706][67871] Updated weights for policy 1, policy_version 56190 (0.0009) [2023-10-07 22:02:21,136][67838] Updated weights for policy 0, policy_version 56132 (0.0008) [2023-10-07 22:02:21,500][67838] Updated weights for policy 0, policy_version 56142 (0.0008) [2023-10-07 22:02:21,745][67871] Updated weights for policy 1, policy_version 56200 (0.0008) [2023-10-07 22:02:21,873][67838] Updated weights for policy 0, policy_version 56152 (0.0011) [2023-10-07 22:02:22,116][67871] Updated weights for policy 1, policy_version 56210 (0.0010) [2023-10-07 22:02:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 115048448. Throughput: 0: 1645.9, 1: 1648.0. Samples: 28773818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:02:22,477][66916] Avg episode reward: [(0, '55.910'), (1, '51.350')] [2023-10-07 22:02:22,482][67871] Updated weights for policy 1, policy_version 56220 (0.0007) [2023-10-07 22:02:26,073][67838] Updated weights for policy 0, policy_version 56162 (0.0008) [2023-10-07 22:02:26,452][67838] Updated weights for policy 0, policy_version 56172 (0.0008) [2023-10-07 22:02:26,614][67871] Updated weights for policy 1, policy_version 56230 (0.0007) [2023-10-07 22:02:26,825][67838] Updated weights for policy 0, policy_version 56182 (0.0008) [2023-10-07 22:02:26,985][67871] Updated weights for policy 1, policy_version 56240 (0.0008) [2023-10-07 22:02:27,199][67838] Updated weights for policy 0, policy_version 56192 (0.0007) [2023-10-07 22:02:27,357][67871] Updated weights for policy 1, policy_version 56250 (0.0008) [2023-10-07 22:02:27,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 115113984. Throughput: 0: 1655.1, 1: 1653.2. Samples: 28784264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:02:27,477][66916] Avg episode reward: [(0, '53.930'), (1, '52.440')] [2023-10-07 22:02:31,313][67838] Updated weights for policy 0, policy_version 56202 (0.0007) [2023-10-07 22:02:31,420][67871] Updated weights for policy 1, policy_version 56260 (0.0009) [2023-10-07 22:02:31,690][67838] Updated weights for policy 0, policy_version 56212 (0.0008) [2023-10-07 22:02:31,781][67871] Updated weights for policy 1, policy_version 56270 (0.0008) [2023-10-07 22:02:32,056][67838] Updated weights for policy 0, policy_version 56222 (0.0010) [2023-10-07 22:02:32,154][67871] Updated weights for policy 1, policy_version 56280 (0.0008) [2023-10-07 22:02:32,477][66916] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13329.3). Total num frames: 115212288. Throughput: 0: 1653.6, 1: 1651.7. Samples: 28804434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:02:32,478][66916] Avg episode reward: [(0, '54.990'), (1, '50.610')] [2023-10-07 22:02:36,049][67838] Updated weights for policy 0, policy_version 56232 (0.0011) [2023-10-07 22:02:36,283][67871] Updated weights for policy 1, policy_version 56290 (0.0007) [2023-10-07 22:02:36,423][67838] Updated weights for policy 0, policy_version 56242 (0.0008) [2023-10-07 22:02:36,651][67871] Updated weights for policy 1, policy_version 56300 (0.0008) [2023-10-07 22:02:36,794][67838] Updated weights for policy 0, policy_version 56252 (0.0007) [2023-10-07 22:02:37,017][67871] Updated weights for policy 1, policy_version 56310 (0.0007) [2023-10-07 22:02:37,382][67871] Updated weights for policy 1, policy_version 56320 (0.0007) [2023-10-07 22:02:37,477][66916] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 115277824. Throughput: 0: 1653.0, 1: 1650.3. Samples: 28823346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:02:37,478][66916] Avg episode reward: [(0, '55.560'), (1, '47.130')] [2023-10-07 22:02:40,907][67838] Updated weights for policy 0, policy_version 56262 (0.0008) [2023-10-07 22:02:41,284][67838] Updated weights for policy 0, policy_version 56272 (0.0008) [2023-10-07 22:02:41,556][67871] Updated weights for policy 1, policy_version 56330 (0.0009) [2023-10-07 22:02:41,661][67838] Updated weights for policy 0, policy_version 56282 (0.0008) [2023-10-07 22:02:41,932][67871] Updated weights for policy 1, policy_version 56340 (0.0010) [2023-10-07 22:02:42,293][67871] Updated weights for policy 1, policy_version 56350 (0.0009) [2023-10-07 22:02:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 115343360. Throughput: 0: 1662.0, 1: 1658.6. Samples: 28834358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:02:42,477][66916] Avg episode reward: [(0, '56.090'), (1, '49.720')] [2023-10-07 22:02:45,695][67838] Updated weights for policy 0, policy_version 56292 (0.0009) [2023-10-07 22:02:46,069][67838] Updated weights for policy 0, policy_version 56302 (0.0007) [2023-10-07 22:02:46,429][67838] Updated weights for policy 0, policy_version 56312 (0.0007) [2023-10-07 22:02:46,437][67871] Updated weights for policy 1, policy_version 56360 (0.0007) [2023-10-07 22:02:46,803][67871] Updated weights for policy 1, policy_version 56370 (0.0009) [2023-10-07 22:02:47,170][67871] Updated weights for policy 1, policy_version 56380 (0.0009) [2023-10-07 22:02:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 115408896. Throughput: 0: 1652.1, 1: 1654.4. Samples: 28853914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:02:47,478][66916] Avg episode reward: [(0, '56.880'), (1, '50.250')] [2023-10-07 22:02:50,764][67838] Updated weights for policy 0, policy_version 56322 (0.0009) [2023-10-07 22:02:51,133][67838] Updated weights for policy 0, policy_version 56332 (0.0008) [2023-10-07 22:02:51,239][67871] Updated weights for policy 1, policy_version 56390 (0.0009) [2023-10-07 22:02:51,509][67838] Updated weights for policy 0, policy_version 56342 (0.0008) [2023-10-07 22:02:51,608][67871] Updated weights for policy 1, policy_version 56400 (0.0008) [2023-10-07 22:02:51,883][67838] Updated weights for policy 0, policy_version 56352 (0.0007) [2023-10-07 22:02:51,962][67871] Updated weights for policy 1, policy_version 56410 (0.0008) [2023-10-07 22:02:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 115474432. Throughput: 0: 1649.7, 1: 1648.0. Samples: 28872692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:02:52,477][66916] Avg episode reward: [(0, '57.920'), (1, '50.010')] [2023-10-07 22:02:55,901][67838] Updated weights for policy 0, policy_version 56362 (0.0008) [2023-10-07 22:02:56,108][67871] Updated weights for policy 1, policy_version 56420 (0.0008) [2023-10-07 22:02:56,275][67838] Updated weights for policy 0, policy_version 56372 (0.0007) [2023-10-07 22:02:56,471][67871] Updated weights for policy 1, policy_version 56430 (0.0009) [2023-10-07 22:02:56,638][67838] Updated weights for policy 0, policy_version 56382 (0.0008) [2023-10-07 22:02:56,829][67871] Updated weights for policy 1, policy_version 56440 (0.0009) [2023-10-07 22:02:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 115539968. Throughput: 0: 1658.6, 1: 1658.3. Samples: 28883886. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:02:57,477][66916] Avg episode reward: [(0, '59.750'), (1, '52.770')] [2023-10-07 22:03:00,639][67838] Updated weights for policy 0, policy_version 56392 (0.0010) [2023-10-07 22:03:01,010][67838] Updated weights for policy 0, policy_version 56402 (0.0009) [2023-10-07 22:03:01,111][67871] Updated weights for policy 1, policy_version 56450 (0.0008) [2023-10-07 22:03:01,372][67838] Updated weights for policy 0, policy_version 56412 (0.0009) [2023-10-07 22:03:01,536][67871] Updated weights for policy 1, policy_version 56460 (0.0010) [2023-10-07 22:03:01,894][67871] Updated weights for policy 1, policy_version 56470 (0.0008) [2023-10-07 22:03:02,272][67871] Updated weights for policy 1, policy_version 56480 (0.0010) [2023-10-07 22:03:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 115605504. Throughput: 0: 1650.4, 1: 1658.3. Samples: 28903812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:03:02,478][66916] Avg episode reward: [(0, '54.940'), (1, '52.140')] [2023-10-07 22:03:05,531][67838] Updated weights for policy 0, policy_version 56422 (0.0009) [2023-10-07 22:03:05,909][67838] Updated weights for policy 0, policy_version 56432 (0.0007) [2023-10-07 22:03:06,272][67838] Updated weights for policy 0, policy_version 56442 (0.0007) [2023-10-07 22:03:06,400][67871] Updated weights for policy 1, policy_version 56490 (0.0008) [2023-10-07 22:03:06,770][67871] Updated weights for policy 1, policy_version 56500 (0.0008) [2023-10-07 22:03:07,146][67871] Updated weights for policy 1, policy_version 56510 (0.0008) [2023-10-07 22:03:07,477][66916] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 115671040. Throughput: 0: 1663.7, 1: 1646.7. Samples: 28922784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:03:07,478][66916] Avg episode reward: [(0, '54.510'), (1, '52.470')] [2023-10-07 22:03:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000056512_57868288.pth... [2023-10-07 22:03:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000056448_57802752.pth... [2023-10-07 22:03:07,535][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000054944_56262656.pth [2023-10-07 22:03:07,535][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000054880_56197120.pth [2023-10-07 22:03:10,264][67838] Updated weights for policy 0, policy_version 56452 (0.0008) [2023-10-07 22:03:10,635][67838] Updated weights for policy 0, policy_version 56462 (0.0008) [2023-10-07 22:03:11,012][67838] Updated weights for policy 0, policy_version 56472 (0.0010) [2023-10-07 22:03:11,277][67871] Updated weights for policy 1, policy_version 56520 (0.0008) [2023-10-07 22:03:11,629][67871] Updated weights for policy 1, policy_version 56530 (0.0010) [2023-10-07 22:03:12,008][67871] Updated weights for policy 1, policy_version 56540 (0.0011) [2023-10-07 22:03:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 115736576. Throughput: 0: 1669.6, 1: 1659.8. Samples: 28934084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:03:12,477][66916] Avg episode reward: [(0, '53.590'), (1, '51.120')] [2023-10-07 22:03:15,109][67838] Updated weights for policy 0, policy_version 56482 (0.0009) [2023-10-07 22:03:15,503][67838] Updated weights for policy 0, policy_version 56492 (0.0007) [2023-10-07 22:03:15,874][67838] Updated weights for policy 0, policy_version 56502 (0.0007) [2023-10-07 22:03:16,100][67871] Updated weights for policy 1, policy_version 56550 (0.0009) [2023-10-07 22:03:16,237][67838] Updated weights for policy 0, policy_version 56512 (0.0008) [2023-10-07 22:03:16,466][67871] Updated weights for policy 1, policy_version 56560 (0.0007) [2023-10-07 22:03:16,825][67871] Updated weights for policy 1, policy_version 56570 (0.0007) [2023-10-07 22:03:17,477][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 115802112. Throughput: 0: 1652.7, 1: 1656.2. Samples: 28953334. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:03:17,478][66916] Avg episode reward: [(0, '55.970'), (1, '50.760')] [2023-10-07 22:03:20,302][67838] Updated weights for policy 0, policy_version 56522 (0.0010) [2023-10-07 22:03:20,678][67838] Updated weights for policy 0, policy_version 56532 (0.0008) [2023-10-07 22:03:20,909][67871] Updated weights for policy 1, policy_version 56580 (0.0008) [2023-10-07 22:03:21,039][67838] Updated weights for policy 0, policy_version 56542 (0.0007) [2023-10-07 22:03:21,265][67871] Updated weights for policy 1, policy_version 56590 (0.0009) [2023-10-07 22:03:21,644][67871] Updated weights for policy 1, policy_version 56600 (0.0010) [2023-10-07 22:03:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 115867648. Throughput: 0: 1669.6, 1: 1644.8. Samples: 28972490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:03:22,478][66916] Avg episode reward: [(0, '53.220'), (1, '51.540')] [2023-10-07 22:03:25,241][67838] Updated weights for policy 0, policy_version 56552 (0.0008) [2023-10-07 22:03:25,608][67838] Updated weights for policy 0, policy_version 56562 (0.0010) [2023-10-07 22:03:25,800][67871] Updated weights for policy 1, policy_version 56610 (0.0009) [2023-10-07 22:03:25,983][67838] Updated weights for policy 0, policy_version 56572 (0.0009) [2023-10-07 22:03:26,165][67871] Updated weights for policy 1, policy_version 56620 (0.0008) [2023-10-07 22:03:26,537][67871] Updated weights for policy 1, policy_version 56630 (0.0010) [2023-10-07 22:03:26,908][67871] Updated weights for policy 1, policy_version 56640 (0.0010) [2023-10-07 22:03:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 115933184. Throughput: 0: 1665.7, 1: 1655.1. Samples: 28983794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:03:27,478][66916] Avg episode reward: [(0, '60.850'), (1, '54.230')] [2023-10-07 22:03:30,055][67838] Updated weights for policy 0, policy_version 56582 (0.0008) [2023-10-07 22:03:30,435][67838] Updated weights for policy 0, policy_version 56592 (0.0008) [2023-10-07 22:03:30,801][67838] Updated weights for policy 0, policy_version 56602 (0.0008) [2023-10-07 22:03:31,065][67871] Updated weights for policy 1, policy_version 56650 (0.0008) [2023-10-07 22:03:31,430][67871] Updated weights for policy 1, policy_version 56660 (0.0008) [2023-10-07 22:03:31,801][67871] Updated weights for policy 1, policy_version 56670 (0.0008) [2023-10-07 22:03:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 115998720. Throughput: 0: 1656.0, 1: 1655.5. Samples: 29002930. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 22:03:32,477][66916] Avg episode reward: [(0, '58.450'), (1, '49.840')] [2023-10-07 22:03:34,952][67838] Updated weights for policy 0, policy_version 56612 (0.0007) [2023-10-07 22:03:35,320][67838] Updated weights for policy 0, policy_version 56622 (0.0007) [2023-10-07 22:03:35,693][67838] Updated weights for policy 0, policy_version 56632 (0.0008) [2023-10-07 22:03:35,867][67871] Updated weights for policy 1, policy_version 56680 (0.0008) [2023-10-07 22:03:36,242][67871] Updated weights for policy 1, policy_version 56690 (0.0008) [2023-10-07 22:03:36,602][67871] Updated weights for policy 1, policy_version 56700 (0.0010) [2023-10-07 22:03:37,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116064256. Throughput: 0: 1670.8, 1: 1650.9. Samples: 29022170. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 22:03:37,478][66916] Avg episode reward: [(0, '54.650'), (1, '52.220')] [2023-10-07 22:03:40,037][67838] Updated weights for policy 0, policy_version 56642 (0.0007) [2023-10-07 22:03:40,403][67838] Updated weights for policy 0, policy_version 56652 (0.0008) [2023-10-07 22:03:40,560][67871] Updated weights for policy 1, policy_version 56710 (0.0009) [2023-10-07 22:03:40,781][67838] Updated weights for policy 0, policy_version 56662 (0.0007) [2023-10-07 22:03:40,925][67871] Updated weights for policy 1, policy_version 56720 (0.0010) [2023-10-07 22:03:41,157][67838] Updated weights for policy 0, policy_version 56672 (0.0007) [2023-10-07 22:03:41,285][67871] Updated weights for policy 1, policy_version 56730 (0.0009) [2023-10-07 22:03:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116129792. Throughput: 0: 1662.8, 1: 1662.8. Samples: 29033538. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 22:03:42,477][66916] Avg episode reward: [(0, '56.910'), (1, '51.830')] [2023-10-07 22:03:45,300][67838] Updated weights for policy 0, policy_version 56682 (0.0008) [2023-10-07 22:03:45,605][67871] Updated weights for policy 1, policy_version 56740 (0.0011) [2023-10-07 22:03:45,668][67838] Updated weights for policy 0, policy_version 56692 (0.0008) [2023-10-07 22:03:45,962][67871] Updated weights for policy 1, policy_version 56750 (0.0007) [2023-10-07 22:03:46,043][67838] Updated weights for policy 0, policy_version 56702 (0.0008) [2023-10-07 22:03:46,332][67871] Updated weights for policy 1, policy_version 56760 (0.0010) [2023-10-07 22:03:47,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116195328. Throughput: 0: 1647.6, 1: 1652.0. Samples: 29052296. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 22:03:47,478][66916] Avg episode reward: [(0, '57.370'), (1, '54.790')] [2023-10-07 22:03:50,189][67838] Updated weights for policy 0, policy_version 56712 (0.0008) [2023-10-07 22:03:50,363][67871] Updated weights for policy 1, policy_version 56770 (0.0007) [2023-10-07 22:03:50,566][67838] Updated weights for policy 0, policy_version 56722 (0.0008) [2023-10-07 22:03:50,776][67871] Updated weights for policy 1, policy_version 56780 (0.0009) [2023-10-07 22:03:50,937][67838] Updated weights for policy 0, policy_version 56732 (0.0007) [2023-10-07 22:03:51,148][67871] Updated weights for policy 1, policy_version 56790 (0.0007) [2023-10-07 22:03:51,506][67871] Updated weights for policy 1, policy_version 56800 (0.0007) [2023-10-07 22:03:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116260864. Throughput: 0: 1650.9, 1: 1654.0. Samples: 29071504. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 22:03:52,477][66916] Avg episode reward: [(0, '53.450'), (1, '51.820')] [2023-10-07 22:03:55,058][67838] Updated weights for policy 0, policy_version 56742 (0.0007) [2023-10-07 22:03:55,427][67838] Updated weights for policy 0, policy_version 56752 (0.0009) [2023-10-07 22:03:55,580][67871] Updated weights for policy 1, policy_version 56810 (0.0008) [2023-10-07 22:03:55,802][67838] Updated weights for policy 0, policy_version 56762 (0.0008) [2023-10-07 22:03:55,943][67871] Updated weights for policy 1, policy_version 56820 (0.0009) [2023-10-07 22:03:56,322][67871] Updated weights for policy 1, policy_version 56830 (0.0007) [2023-10-07 22:03:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 116326400. Throughput: 0: 1645.1, 1: 1660.9. Samples: 29082852. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 22:03:57,478][66916] Avg episode reward: [(0, '55.500'), (1, '50.800')] [2023-10-07 22:03:59,853][67838] Updated weights for policy 0, policy_version 56772 (0.0010) [2023-10-07 22:04:00,215][67838] Updated weights for policy 0, policy_version 56782 (0.0007) [2023-10-07 22:04:00,461][67871] Updated weights for policy 1, policy_version 56840 (0.0008) [2023-10-07 22:04:00,591][67838] Updated weights for policy 0, policy_version 56792 (0.0009) [2023-10-07 22:04:00,830][67871] Updated weights for policy 1, policy_version 56850 (0.0009) [2023-10-07 22:04:01,193][67871] Updated weights for policy 1, policy_version 56860 (0.0008) [2023-10-07 22:04:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116391936. Throughput: 0: 1644.1, 1: 1651.5. Samples: 29101634. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 22:04:02,477][66916] Avg episode reward: [(0, '57.150'), (1, '46.430')] [2023-10-07 22:04:04,868][67838] Updated weights for policy 0, policy_version 56802 (0.0009) [2023-10-07 22:04:05,238][67871] Updated weights for policy 1, policy_version 56870 (0.0008) [2023-10-07 22:04:05,254][67838] Updated weights for policy 0, policy_version 56812 (0.0009) [2023-10-07 22:04:05,600][67871] Updated weights for policy 1, policy_version 56880 (0.0008) [2023-10-07 22:04:05,631][67838] Updated weights for policy 0, policy_version 56822 (0.0008) [2023-10-07 22:04:05,966][67871] Updated weights for policy 1, policy_version 56890 (0.0009) [2023-10-07 22:04:05,993][67838] Updated weights for policy 0, policy_version 56832 (0.0009) [2023-10-07 22:04:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 116457472. Throughput: 0: 1647.7, 1: 1663.2. Samples: 29121480. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 22:04:07,477][66916] Avg episode reward: [(0, '54.300'), (1, '44.680')] [2023-10-07 22:04:10,054][67838] Updated weights for policy 0, policy_version 56842 (0.0007) [2023-10-07 22:04:10,091][67871] Updated weights for policy 1, policy_version 56900 (0.0009) [2023-10-07 22:04:10,426][67838] Updated weights for policy 0, policy_version 56852 (0.0009) [2023-10-07 22:04:10,456][67871] Updated weights for policy 1, policy_version 56910 (0.0009) [2023-10-07 22:04:10,799][67838] Updated weights for policy 0, policy_version 56862 (0.0008) [2023-10-07 22:04:10,820][67871] Updated weights for policy 1, policy_version 56920 (0.0009) [2023-10-07 22:04:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116523008. Throughput: 0: 1638.3, 1: 1670.8. Samples: 29132704. Policy #0 lag: (min: 15.0, avg: 20.1, max: 47.0) [2023-10-07 22:04:12,477][66916] Avg episode reward: [(0, '52.520'), (1, '47.810')] [2023-10-07 22:04:14,793][67871] Updated weights for policy 1, policy_version 56930 (0.0008) [2023-10-07 22:04:14,921][67838] Updated weights for policy 0, policy_version 56872 (0.0008) [2023-10-07 22:04:15,155][67871] Updated weights for policy 1, policy_version 56940 (0.0009) [2023-10-07 22:04:15,282][67838] Updated weights for policy 0, policy_version 56882 (0.0009) [2023-10-07 22:04:15,522][67871] Updated weights for policy 1, policy_version 56950 (0.0008) [2023-10-07 22:04:15,654][67838] Updated weights for policy 0, policy_version 56892 (0.0008) [2023-10-07 22:04:15,886][67871] Updated weights for policy 1, policy_version 56960 (0.0007) [2023-10-07 22:04:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 116588544. Throughput: 0: 1642.4, 1: 1652.1. Samples: 29151184. Policy #0 lag: (min: 15.0, avg: 20.1, max: 47.0) [2023-10-07 22:04:17,478][66916] Avg episode reward: [(0, '51.080'), (1, '49.400')] [2023-10-07 22:04:19,744][67838] Updated weights for policy 0, policy_version 56902 (0.0009) [2023-10-07 22:04:20,073][67871] Updated weights for policy 1, policy_version 56970 (0.0009) [2023-10-07 22:04:20,116][67838] Updated weights for policy 0, policy_version 56912 (0.0008) [2023-10-07 22:04:20,437][67871] Updated weights for policy 1, policy_version 56980 (0.0008) [2023-10-07 22:04:20,479][67838] Updated weights for policy 0, policy_version 56922 (0.0009) [2023-10-07 22:04:20,795][67871] Updated weights for policy 1, policy_version 56990 (0.0009) [2023-10-07 22:04:22,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116654080. Throughput: 0: 1652.9, 1: 1670.4. Samples: 29171716. Policy #0 lag: (min: 15.0, avg: 20.1, max: 47.0) [2023-10-07 22:04:22,477][66916] Avg episode reward: [(0, '53.070'), (1, '51.930')] [2023-10-07 22:04:24,620][67838] Updated weights for policy 0, policy_version 56932 (0.0009) [2023-10-07 22:04:24,997][67838] Updated weights for policy 0, policy_version 56942 (0.0009) [2023-10-07 22:04:25,018][67871] Updated weights for policy 1, policy_version 57000 (0.0008) [2023-10-07 22:04:25,361][67838] Updated weights for policy 0, policy_version 56952 (0.0008) [2023-10-07 22:04:25,391][67871] Updated weights for policy 1, policy_version 57010 (0.0008) [2023-10-07 22:04:25,752][67871] Updated weights for policy 1, policy_version 57020 (0.0008) [2023-10-07 22:04:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116719616. Throughput: 0: 1644.7, 1: 1664.4. Samples: 29182448. Policy #0 lag: (min: 15.0, avg: 20.1, max: 47.0) [2023-10-07 22:04:27,477][66916] Avg episode reward: [(0, '53.460'), (1, '52.730')] [2023-10-07 22:04:29,245][67838] Updated weights for policy 0, policy_version 56962 (0.0008) [2023-10-07 22:04:29,627][67838] Updated weights for policy 0, policy_version 56972 (0.0008) [2023-10-07 22:04:29,809][67871] Updated weights for policy 1, policy_version 57030 (0.0007) [2023-10-07 22:04:29,988][67838] Updated weights for policy 0, policy_version 56982 (0.0008) [2023-10-07 22:04:30,164][67871] Updated weights for policy 1, policy_version 57040 (0.0008) [2023-10-07 22:04:30,359][67838] Updated weights for policy 0, policy_version 56992 (0.0009) [2023-10-07 22:04:30,535][67871] Updated weights for policy 1, policy_version 57050 (0.0007) [2023-10-07 22:04:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116785152. Throughput: 0: 1663.1, 1: 1651.4. Samples: 29201448. Policy #0 lag: (min: 15.0, avg: 20.1, max: 47.0) [2023-10-07 22:04:32,478][66916] Avg episode reward: [(0, '51.850'), (1, '52.600')] [2023-10-07 22:04:34,440][67838] Updated weights for policy 0, policy_version 57002 (0.0010) [2023-10-07 22:04:34,811][67838] Updated weights for policy 0, policy_version 57012 (0.0007) [2023-10-07 22:04:34,870][67871] Updated weights for policy 1, policy_version 57060 (0.0011) [2023-10-07 22:04:35,182][67838] Updated weights for policy 0, policy_version 57022 (0.0008) [2023-10-07 22:04:35,233][67871] Updated weights for policy 1, policy_version 57070 (0.0009) [2023-10-07 22:04:35,611][67871] Updated weights for policy 1, policy_version 57080 (0.0008) [2023-10-07 22:04:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 116850688. Throughput: 0: 1671.5, 1: 1671.2. Samples: 29221926. Policy #0 lag: (min: 15.0, avg: 20.1, max: 47.0) [2023-10-07 22:04:37,478][66916] Avg episode reward: [(0, '50.270'), (1, '52.470')] [2023-10-07 22:04:39,418][67838] Updated weights for policy 0, policy_version 57032 (0.0009) [2023-10-07 22:04:39,580][67871] Updated weights for policy 1, policy_version 57090 (0.0007) [2023-10-07 22:04:39,800][67838] Updated weights for policy 0, policy_version 57042 (0.0010) [2023-10-07 22:04:39,988][67871] Updated weights for policy 1, policy_version 57100 (0.0009) [2023-10-07 22:04:40,173][67838] Updated weights for policy 0, policy_version 57052 (0.0008) [2023-10-07 22:04:40,353][67871] Updated weights for policy 1, policy_version 57110 (0.0009) [2023-10-07 22:04:40,724][67871] Updated weights for policy 1, policy_version 57120 (0.0010) [2023-10-07 22:04:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 116916224. Throughput: 0: 1652.9, 1: 1662.7. Samples: 29232052. Policy #0 lag: (min: 15.0, avg: 20.1, max: 47.0) [2023-10-07 22:04:42,478][66916] Avg episode reward: [(0, '42.390'), (1, '52.980')] [2023-10-07 22:04:44,228][67838] Updated weights for policy 0, policy_version 57062 (0.0007) [2023-10-07 22:04:44,595][67838] Updated weights for policy 0, policy_version 57072 (0.0008) [2023-10-07 22:04:44,959][67838] Updated weights for policy 0, policy_version 57082 (0.0007) [2023-10-07 22:04:44,966][67871] Updated weights for policy 1, policy_version 57130 (0.0007) [2023-10-07 22:04:45,323][67871] Updated weights for policy 1, policy_version 57140 (0.0009) [2023-10-07 22:04:45,694][67871] Updated weights for policy 1, policy_version 57150 (0.0007) [2023-10-07 22:04:47,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 116981760. Throughput: 0: 1668.2, 1: 1653.6. Samples: 29251114. Policy #0 lag: (min: 15.0, avg: 20.1, max: 47.0) [2023-10-07 22:04:47,478][66916] Avg episode reward: [(0, '35.100'), (1, '52.680')] [2023-10-07 22:04:49,205][67838] Updated weights for policy 0, policy_version 57092 (0.0010) [2023-10-07 22:04:49,576][67838] Updated weights for policy 0, policy_version 57102 (0.0009) [2023-10-07 22:04:49,769][67871] Updated weights for policy 1, policy_version 57160 (0.0009) [2023-10-07 22:04:49,947][67838] Updated weights for policy 0, policy_version 57112 (0.0009) [2023-10-07 22:04:50,139][67871] Updated weights for policy 1, policy_version 57170 (0.0010) [2023-10-07 22:04:50,502][67871] Updated weights for policy 1, policy_version 57180 (0.0010) [2023-10-07 22:04:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 117047296. Throughput: 0: 1670.5, 1: 1662.4. Samples: 29271464. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 22:04:52,477][66916] Avg episode reward: [(0, '36.640'), (1, '47.740')] [2023-10-07 22:04:54,189][67838] Updated weights for policy 0, policy_version 57122 (0.0009) [2023-10-07 22:04:54,596][67838] Updated weights for policy 0, policy_version 57132 (0.0008) [2023-10-07 22:04:54,675][67871] Updated weights for policy 1, policy_version 57190 (0.0009) [2023-10-07 22:04:54,980][67838] Updated weights for policy 0, policy_version 57142 (0.0007) [2023-10-07 22:04:55,043][67871] Updated weights for policy 1, policy_version 57200 (0.0008) [2023-10-07 22:04:55,353][67838] Updated weights for policy 0, policy_version 57152 (0.0009) [2023-10-07 22:04:55,413][67871] Updated weights for policy 1, policy_version 57210 (0.0008) [2023-10-07 22:04:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 117112832. Throughput: 0: 1663.1, 1: 1648.5. Samples: 29281726. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 22:04:57,477][66916] Avg episode reward: [(0, '34.950'), (1, '47.080')] [2023-10-07 22:04:59,320][67838] Updated weights for policy 0, policy_version 57162 (0.0008) [2023-10-07 22:04:59,691][67871] Updated weights for policy 1, policy_version 57220 (0.0007) [2023-10-07 22:04:59,697][67838] Updated weights for policy 0, policy_version 57172 (0.0007) [2023-10-07 22:05:00,053][67871] Updated weights for policy 1, policy_version 57230 (0.0008) [2023-10-07 22:05:00,072][67838] Updated weights for policy 0, policy_version 57182 (0.0007) [2023-10-07 22:05:00,415][67871] Updated weights for policy 1, policy_version 57240 (0.0010) [2023-10-07 22:05:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 117178368. Throughput: 0: 1673.2, 1: 1649.9. Samples: 29300720. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 22:05:02,478][66916] Avg episode reward: [(0, '35.330'), (1, '46.360')] [2023-10-07 22:05:04,274][67838] Updated weights for policy 0, policy_version 57192 (0.0009) [2023-10-07 22:05:04,510][67871] Updated weights for policy 1, policy_version 57250 (0.0009) [2023-10-07 22:05:04,655][67838] Updated weights for policy 0, policy_version 57202 (0.0008) [2023-10-07 22:05:04,881][67871] Updated weights for policy 1, policy_version 57260 (0.0009) [2023-10-07 22:05:05,037][67838] Updated weights for policy 0, policy_version 57212 (0.0007) [2023-10-07 22:05:05,248][67871] Updated weights for policy 1, policy_version 57270 (0.0009) [2023-10-07 22:05:05,615][67871] Updated weights for policy 1, policy_version 57280 (0.0008) [2023-10-07 22:05:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 117243904. Throughput: 0: 1666.5, 1: 1650.3. Samples: 29320970. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 22:05:07,477][66916] Avg episode reward: [(0, '36.040'), (1, '50.270')] [2023-10-07 22:05:07,486][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000057280_58654720.pth... [2023-10-07 22:05:07,486][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000057216_58589184.pth... [2023-10-07 22:05:07,518][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000055680_57016320.pth [2023-10-07 22:05:07,523][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000055744_57081856.pth [2023-10-07 22:05:09,169][67838] Updated weights for policy 0, policy_version 57222 (0.0010) [2023-10-07 22:05:09,538][67838] Updated weights for policy 0, policy_version 57232 (0.0007) [2023-10-07 22:05:09,660][67871] Updated weights for policy 1, policy_version 57290 (0.0007) [2023-10-07 22:05:09,911][67838] Updated weights for policy 0, policy_version 57242 (0.0007) [2023-10-07 22:05:10,032][67871] Updated weights for policy 1, policy_version 57300 (0.0009) [2023-10-07 22:05:10,400][67871] Updated weights for policy 1, policy_version 57310 (0.0010) [2023-10-07 22:05:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 117309440. Throughput: 0: 1651.9, 1: 1643.3. Samples: 29330734. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 22:05:12,477][66916] Avg episode reward: [(0, '36.800'), (1, '47.310')] [2023-10-07 22:05:13,912][67838] Updated weights for policy 0, policy_version 57252 (0.0009) [2023-10-07 22:05:14,283][67838] Updated weights for policy 0, policy_version 57262 (0.0009) [2023-10-07 22:05:14,391][67871] Updated weights for policy 1, policy_version 57320 (0.0008) [2023-10-07 22:05:14,653][67838] Updated weights for policy 0, policy_version 57272 (0.0010) [2023-10-07 22:05:14,748][67871] Updated weights for policy 1, policy_version 57330 (0.0007) [2023-10-07 22:05:15,124][67871] Updated weights for policy 1, policy_version 57340 (0.0009) [2023-10-07 22:05:17,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 117374976. Throughput: 0: 1659.7, 1: 1651.0. Samples: 29350430. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 22:05:17,477][66916] Avg episode reward: [(0, '34.990'), (1, '52.100')] [2023-10-07 22:05:18,782][67838] Updated weights for policy 0, policy_version 57282 (0.0009) [2023-10-07 22:05:19,156][67838] Updated weights for policy 0, policy_version 57292 (0.0010) [2023-10-07 22:05:19,302][67871] Updated weights for policy 1, policy_version 57350 (0.0010) [2023-10-07 22:05:19,533][67838] Updated weights for policy 0, policy_version 57302 (0.0008) [2023-10-07 22:05:19,676][67871] Updated weights for policy 1, policy_version 57360 (0.0009) [2023-10-07 22:05:19,908][67838] Updated weights for policy 0, policy_version 57312 (0.0009) [2023-10-07 22:05:20,050][67871] Updated weights for policy 1, policy_version 57370 (0.0008) [2023-10-07 22:05:22,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 117440512. Throughput: 0: 1654.8, 1: 1654.2. Samples: 29370832. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 22:05:22,477][66916] Avg episode reward: [(0, '37.020'), (1, '52.750')] [2023-10-07 22:05:24,009][67838] Updated weights for policy 0, policy_version 57322 (0.0007) [2023-10-07 22:05:24,228][67871] Updated weights for policy 1, policy_version 57380 (0.0008) [2023-10-07 22:05:24,381][67838] Updated weights for policy 0, policy_version 57332 (0.0008) [2023-10-07 22:05:24,618][67871] Updated weights for policy 1, policy_version 57390 (0.0008) [2023-10-07 22:05:24,750][67838] Updated weights for policy 0, policy_version 57342 (0.0008) [2023-10-07 22:05:24,981][67871] Updated weights for policy 1, policy_version 57400 (0.0007) [2023-10-07 22:05:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 117506048. Throughput: 0: 1648.6, 1: 1642.8. Samples: 29380162. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 22:05:27,477][66916] Avg episode reward: [(0, '35.850'), (1, '53.120')] [2023-10-07 22:05:28,952][67838] Updated weights for policy 0, policy_version 57352 (0.0010) [2023-10-07 22:05:29,248][67871] Updated weights for policy 1, policy_version 57410 (0.0007) [2023-10-07 22:05:29,321][67838] Updated weights for policy 0, policy_version 57362 (0.0010) [2023-10-07 22:05:29,613][67871] Updated weights for policy 1, policy_version 57420 (0.0007) [2023-10-07 22:05:29,698][67838] Updated weights for policy 0, policy_version 57372 (0.0007) [2023-10-07 22:05:29,978][67871] Updated weights for policy 1, policy_version 57430 (0.0008) [2023-10-07 22:05:30,340][67871] Updated weights for policy 1, policy_version 57440 (0.0009) [2023-10-07 22:05:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 117571584. Throughput: 0: 1656.2, 1: 1655.4. Samples: 29400134. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 22:05:32,477][66916] Avg episode reward: [(0, '33.590'), (1, '52.760')] [2023-10-07 22:05:33,703][67838] Updated weights for policy 0, policy_version 57382 (0.0007) [2023-10-07 22:05:34,071][67838] Updated weights for policy 0, policy_version 57392 (0.0008) [2023-10-07 22:05:34,447][67838] Updated weights for policy 0, policy_version 57402 (0.0008) [2023-10-07 22:05:34,604][67871] Updated weights for policy 1, policy_version 57450 (0.0009) [2023-10-07 22:05:34,970][67871] Updated weights for policy 1, policy_version 57460 (0.0009) [2023-10-07 22:05:35,343][67871] Updated weights for policy 1, policy_version 57470 (0.0009) [2023-10-07 22:05:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 117637120. Throughput: 0: 1661.5, 1: 1657.0. Samples: 29420796. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 22:05:37,477][66916] Avg episode reward: [(0, '34.160'), (1, '51.080')] [2023-10-07 22:05:38,506][67838] Updated weights for policy 0, policy_version 57412 (0.0007) [2023-10-07 22:05:38,903][67838] Updated weights for policy 0, policy_version 57422 (0.0007) [2023-10-07 22:05:39,266][67838] Updated weights for policy 0, policy_version 57432 (0.0010) [2023-10-07 22:05:39,460][67871] Updated weights for policy 1, policy_version 57480 (0.0008) [2023-10-07 22:05:39,817][67871] Updated weights for policy 1, policy_version 57490 (0.0009) [2023-10-07 22:05:40,181][67871] Updated weights for policy 1, policy_version 57500 (0.0008) [2023-10-07 22:05:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 117702656. Throughput: 0: 1650.1, 1: 1653.7. Samples: 29430400. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 22:05:42,478][66916] Avg episode reward: [(0, '34.320'), (1, '47.620')] [2023-10-07 22:05:43,453][67838] Updated weights for policy 0, policy_version 57442 (0.0007) [2023-10-07 22:05:43,830][67838] Updated weights for policy 0, policy_version 57452 (0.0008) [2023-10-07 22:05:44,206][67838] Updated weights for policy 0, policy_version 57462 (0.0009) [2023-10-07 22:05:44,321][67871] Updated weights for policy 1, policy_version 57510 (0.0008) [2023-10-07 22:05:44,575][67838] Updated weights for policy 0, policy_version 57472 (0.0010) [2023-10-07 22:05:44,686][67871] Updated weights for policy 1, policy_version 57520 (0.0010) [2023-10-07 22:05:45,048][67871] Updated weights for policy 1, policy_version 57530 (0.0011) [2023-10-07 22:05:47,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 117768192. Throughput: 0: 1658.8, 1: 1661.2. Samples: 29450116. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 22:05:47,478][66916] Avg episode reward: [(0, '34.940'), (1, '48.630')] [2023-10-07 22:05:48,529][67838] Updated weights for policy 0, policy_version 57482 (0.0007) [2023-10-07 22:05:48,889][67838] Updated weights for policy 0, policy_version 57492 (0.0007) [2023-10-07 22:05:48,963][67871] Updated weights for policy 1, policy_version 57540 (0.0009) [2023-10-07 22:05:49,267][67838] Updated weights for policy 0, policy_version 57502 (0.0008) [2023-10-07 22:05:49,325][67871] Updated weights for policy 1, policy_version 57550 (0.0007) [2023-10-07 22:05:49,684][67871] Updated weights for policy 1, policy_version 57560 (0.0008) [2023-10-07 22:05:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 117833728. Throughput: 0: 1660.8, 1: 1667.9. Samples: 29470764. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 22:05:52,477][66916] Avg episode reward: [(0, '35.850'), (1, '47.240')] [2023-10-07 22:05:53,437][67838] Updated weights for policy 0, policy_version 57512 (0.0007) [2023-10-07 22:05:53,762][67871] Updated weights for policy 1, policy_version 57570 (0.0007) [2023-10-07 22:05:53,807][67838] Updated weights for policy 0, policy_version 57522 (0.0009) [2023-10-07 22:05:54,128][67871] Updated weights for policy 1, policy_version 57580 (0.0010) [2023-10-07 22:05:54,177][67838] Updated weights for policy 0, policy_version 57532 (0.0008) [2023-10-07 22:05:54,482][67871] Updated weights for policy 1, policy_version 57590 (0.0008) [2023-10-07 22:05:54,854][67871] Updated weights for policy 1, policy_version 57600 (0.0009) [2023-10-07 22:05:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 117899264. Throughput: 0: 1659.8, 1: 1654.3. Samples: 29479868. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 22:05:57,477][66916] Avg episode reward: [(0, '37.690'), (1, '44.450')] [2023-10-07 22:05:58,453][67838] Updated weights for policy 0, policy_version 57542 (0.0007) [2023-10-07 22:05:58,824][67838] Updated weights for policy 0, policy_version 57552 (0.0009) [2023-10-07 22:05:58,899][67871] Updated weights for policy 1, policy_version 57610 (0.0009) [2023-10-07 22:05:59,195][67838] Updated weights for policy 0, policy_version 57562 (0.0008) [2023-10-07 22:05:59,267][67871] Updated weights for policy 1, policy_version 57620 (0.0009) [2023-10-07 22:05:59,628][67871] Updated weights for policy 1, policy_version 57630 (0.0009) [2023-10-07 22:06:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 117964800. Throughput: 0: 1654.5, 1: 1664.8. Samples: 29499800. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 22:06:02,477][66916] Avg episode reward: [(0, '35.440'), (1, '46.210')] [2023-10-07 22:06:03,317][67838] Updated weights for policy 0, policy_version 57572 (0.0008) [2023-10-07 22:06:03,689][67838] Updated weights for policy 0, policy_version 57582 (0.0008) [2023-10-07 22:06:03,761][67871] Updated weights for policy 1, policy_version 57640 (0.0008) [2023-10-07 22:06:04,058][67838] Updated weights for policy 0, policy_version 57592 (0.0009) [2023-10-07 22:06:04,125][67871] Updated weights for policy 1, policy_version 57650 (0.0007) [2023-10-07 22:06:04,503][67871] Updated weights for policy 1, policy_version 57660 (0.0008) [2023-10-07 22:06:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 118030336. Throughput: 0: 1658.3, 1: 1667.2. Samples: 29520478. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 22:06:07,478][66916] Avg episode reward: [(0, '35.270'), (1, '46.710')] [2023-10-07 22:06:08,186][67838] Updated weights for policy 0, policy_version 57602 (0.0010) [2023-10-07 22:06:08,491][67871] Updated weights for policy 1, policy_version 57670 (0.0008) [2023-10-07 22:06:08,561][67838] Updated weights for policy 0, policy_version 57612 (0.0009) [2023-10-07 22:06:08,856][67871] Updated weights for policy 1, policy_version 57680 (0.0007) [2023-10-07 22:06:08,933][67838] Updated weights for policy 0, policy_version 57622 (0.0009) [2023-10-07 22:06:09,215][67871] Updated weights for policy 1, policy_version 57690 (0.0008) [2023-10-07 22:06:09,305][67838] Updated weights for policy 0, policy_version 57632 (0.0009) [2023-10-07 22:06:12,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 118095872. Throughput: 0: 1657.6, 1: 1658.4. Samples: 29529382. Policy #0 lag: (min: 15.0, avg: 19.7, max: 47.0) [2023-10-07 22:06:12,478][66916] Avg episode reward: [(0, '35.790'), (1, '47.780')] [2023-10-07 22:06:13,459][67871] Updated weights for policy 1, policy_version 57700 (0.0007) [2023-10-07 22:06:13,699][67838] Updated weights for policy 0, policy_version 57642 (0.0008) [2023-10-07 22:06:13,821][67871] Updated weights for policy 1, policy_version 57710 (0.0007) [2023-10-07 22:06:14,064][67838] Updated weights for policy 0, policy_version 57652 (0.0010) [2023-10-07 22:06:14,185][67871] Updated weights for policy 1, policy_version 57720 (0.0007) [2023-10-07 22:06:14,437][67838] Updated weights for policy 0, policy_version 57662 (0.0011) [2023-10-07 22:06:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 118161408. Throughput: 0: 1654.5, 1: 1672.7. Samples: 29549856. Policy #0 lag: (min: 15.0, avg: 19.7, max: 47.0) [2023-10-07 22:06:17,477][66916] Avg episode reward: [(0, '34.660'), (1, '47.290')] [2023-10-07 22:06:18,309][67871] Updated weights for policy 1, policy_version 57730 (0.0009) [2023-10-07 22:06:18,470][67838] Updated weights for policy 0, policy_version 57672 (0.0007) [2023-10-07 22:06:18,726][67871] Updated weights for policy 1, policy_version 57740 (0.0007) [2023-10-07 22:06:18,840][67838] Updated weights for policy 0, policy_version 57682 (0.0008) [2023-10-07 22:06:19,090][67871] Updated weights for policy 1, policy_version 57750 (0.0009) [2023-10-07 22:06:19,211][67838] Updated weights for policy 0, policy_version 57692 (0.0009) [2023-10-07 22:06:19,455][67871] Updated weights for policy 1, policy_version 57760 (0.0008) [2023-10-07 22:06:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 118226944. Throughput: 0: 1649.6, 1: 1673.0. Samples: 29570314. Policy #0 lag: (min: 15.0, avg: 19.7, max: 47.0) [2023-10-07 22:06:22,478][66916] Avg episode reward: [(0, '36.060'), (1, '50.820')] [2023-10-07 22:06:23,396][67838] Updated weights for policy 0, policy_version 57702 (0.0008) [2023-10-07 22:06:23,606][67871] Updated weights for policy 1, policy_version 57770 (0.0008) [2023-10-07 22:06:23,771][67838] Updated weights for policy 0, policy_version 57712 (0.0008) [2023-10-07 22:06:23,975][67871] Updated weights for policy 1, policy_version 57780 (0.0007) [2023-10-07 22:06:24,150][67838] Updated weights for policy 0, policy_version 57722 (0.0008) [2023-10-07 22:06:24,339][67871] Updated weights for policy 1, policy_version 57790 (0.0007) [2023-10-07 22:06:27,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 118292480. Throughput: 0: 1648.7, 1: 1655.7. Samples: 29579096. Policy #0 lag: (min: 15.0, avg: 19.7, max: 47.0) [2023-10-07 22:06:27,478][66916] Avg episode reward: [(0, '36.040'), (1, '52.830')] [2023-10-07 22:06:28,257][67838] Updated weights for policy 0, policy_version 57732 (0.0009) [2023-10-07 22:06:28,655][67838] Updated weights for policy 0, policy_version 57742 (0.0007) [2023-10-07 22:06:28,727][67871] Updated weights for policy 1, policy_version 57800 (0.0009) [2023-10-07 22:06:29,018][67838] Updated weights for policy 0, policy_version 57752 (0.0009) [2023-10-07 22:06:29,090][67871] Updated weights for policy 1, policy_version 57810 (0.0009) [2023-10-07 22:06:29,446][67871] Updated weights for policy 1, policy_version 57820 (0.0009) [2023-10-07 22:06:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 118358016. Throughput: 0: 1649.6, 1: 1666.1. Samples: 29599322. Policy #0 lag: (min: 15.0, avg: 19.7, max: 47.0) [2023-10-07 22:06:32,478][66916] Avg episode reward: [(0, '38.330'), (1, '53.340')] [2023-10-07 22:06:33,175][67838] Updated weights for policy 0, policy_version 57762 (0.0008) [2023-10-07 22:06:33,544][67838] Updated weights for policy 0, policy_version 57772 (0.0007) [2023-10-07 22:06:33,561][67871] Updated weights for policy 1, policy_version 57830 (0.0007) [2023-10-07 22:06:33,921][67838] Updated weights for policy 0, policy_version 57782 (0.0008) [2023-10-07 22:06:33,924][67871] Updated weights for policy 1, policy_version 57840 (0.0008) [2023-10-07 22:06:34,288][67871] Updated weights for policy 1, policy_version 57850 (0.0007) [2023-10-07 22:06:34,294][67838] Updated weights for policy 0, policy_version 57792 (0.0007) [2023-10-07 22:06:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 118423552. Throughput: 0: 1651.9, 1: 1663.4. Samples: 29619954. Policy #0 lag: (min: 15.0, avg: 19.7, max: 47.0) [2023-10-07 22:06:37,477][66916] Avg episode reward: [(0, '39.360'), (1, '52.410')] [2023-10-07 22:06:38,250][67838] Updated weights for policy 0, policy_version 57802 (0.0010) [2023-10-07 22:06:38,270][67871] Updated weights for policy 1, policy_version 57860 (0.0008) [2023-10-07 22:06:38,615][67838] Updated weights for policy 0, policy_version 57812 (0.0008) [2023-10-07 22:06:38,628][67871] Updated weights for policy 1, policy_version 57870 (0.0008) [2023-10-07 22:06:38,983][67838] Updated weights for policy 0, policy_version 57822 (0.0007) [2023-10-07 22:06:38,997][67871] Updated weights for policy 1, policy_version 57880 (0.0008) [2023-10-07 22:06:42,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 118489088. Throughput: 0: 1657.2, 1: 1659.8. Samples: 29629132. Policy #0 lag: (min: 15.0, avg: 19.7, max: 47.0) [2023-10-07 22:06:42,477][66916] Avg episode reward: [(0, '37.600'), (1, '51.400')] [2023-10-07 22:06:42,883][67838] Updated weights for policy 0, policy_version 57832 (0.0008) [2023-10-07 22:06:43,141][67871] Updated weights for policy 1, policy_version 57890 (0.0009) [2023-10-07 22:06:43,251][67838] Updated weights for policy 0, policy_version 57842 (0.0008) [2023-10-07 22:06:43,517][67871] Updated weights for policy 1, policy_version 57900 (0.0008) [2023-10-07 22:06:43,623][67838] Updated weights for policy 0, policy_version 57852 (0.0010) [2023-10-07 22:06:43,880][67871] Updated weights for policy 1, policy_version 57910 (0.0009) [2023-10-07 22:06:44,255][67871] Updated weights for policy 1, policy_version 57920 (0.0009) [2023-10-07 22:06:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 118554624. Throughput: 0: 1665.2, 1: 1664.7. Samples: 29649648. Policy #0 lag: (min: 15.0, avg: 19.7, max: 47.0) [2023-10-07 22:06:47,478][66916] Avg episode reward: [(0, '39.820'), (1, '45.300')] [2023-10-07 22:06:47,850][67838] Updated weights for policy 0, policy_version 57862 (0.0010) [2023-10-07 22:06:48,229][67838] Updated weights for policy 0, policy_version 57872 (0.0008) [2023-10-07 22:06:48,432][67871] Updated weights for policy 1, policy_version 57930 (0.0010) [2023-10-07 22:06:48,598][67838] Updated weights for policy 0, policy_version 57882 (0.0010) [2023-10-07 22:06:48,799][67871] Updated weights for policy 1, policy_version 57940 (0.0007) [2023-10-07 22:06:49,173][67871] Updated weights for policy 1, policy_version 57950 (0.0010) [2023-10-07 22:06:52,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 118620160. Throughput: 0: 1660.9, 1: 1662.6. Samples: 29670036. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:06:52,478][66916] Avg episode reward: [(0, '36.820'), (1, '45.200')] [2023-10-07 22:06:52,709][67838] Updated weights for policy 0, policy_version 57892 (0.0008) [2023-10-07 22:06:53,086][67838] Updated weights for policy 0, policy_version 57902 (0.0007) [2023-10-07 22:06:53,308][67871] Updated weights for policy 1, policy_version 57960 (0.0009) [2023-10-07 22:06:53,455][67838] Updated weights for policy 0, policy_version 57912 (0.0007) [2023-10-07 22:06:53,670][67871] Updated weights for policy 1, policy_version 57970 (0.0008) [2023-10-07 22:06:54,040][67871] Updated weights for policy 1, policy_version 57980 (0.0009) [2023-10-07 22:06:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 118685696. Throughput: 0: 1664.5, 1: 1662.4. Samples: 29679094. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:06:57,478][66916] Avg episode reward: [(0, '36.380'), (1, '47.430')] [2023-10-07 22:06:57,712][67838] Updated weights for policy 0, policy_version 57922 (0.0007) [2023-10-07 22:06:58,034][67871] Updated weights for policy 1, policy_version 57990 (0.0008) [2023-10-07 22:06:58,090][67838] Updated weights for policy 0, policy_version 57932 (0.0009) [2023-10-07 22:06:58,401][67871] Updated weights for policy 1, policy_version 58000 (0.0008) [2023-10-07 22:06:58,459][67838] Updated weights for policy 0, policy_version 57942 (0.0008) [2023-10-07 22:06:58,772][67871] Updated weights for policy 1, policy_version 58010 (0.0008) [2023-10-07 22:06:58,822][67838] Updated weights for policy 0, policy_version 57952 (0.0008) [2023-10-07 22:07:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 118751232. Throughput: 0: 1663.2, 1: 1660.2. Samples: 29699412. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:07:02,477][66916] Avg episode reward: [(0, '37.220'), (1, '46.310')] [2023-10-07 22:07:03,043][67838] Updated weights for policy 0, policy_version 57962 (0.0008) [2023-10-07 22:07:03,094][67871] Updated weights for policy 1, policy_version 58020 (0.0008) [2023-10-07 22:07:03,418][67838] Updated weights for policy 0, policy_version 57972 (0.0009) [2023-10-07 22:07:03,460][67871] Updated weights for policy 1, policy_version 58030 (0.0008) [2023-10-07 22:07:03,789][67838] Updated weights for policy 0, policy_version 57982 (0.0008) [2023-10-07 22:07:03,817][67871] Updated weights for policy 1, policy_version 58040 (0.0009) [2023-10-07 22:07:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 118816768. Throughput: 0: 1661.1, 1: 1657.3. Samples: 29719642. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:07:07,477][66916] Avg episode reward: [(0, '36.260'), (1, '45.480')] [2023-10-07 22:07:07,485][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000057984_59375616.pth... [2023-10-07 22:07:07,485][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000058048_59441152.pth... [2023-10-07 22:07:07,515][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000056448_57802752.pth [2023-10-07 22:07:07,526][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000056512_57868288.pth [2023-10-07 22:07:08,082][67838] Updated weights for policy 0, policy_version 57992 (0.0008) [2023-10-07 22:07:08,148][67871] Updated weights for policy 1, policy_version 58050 (0.0009) [2023-10-07 22:07:08,458][67838] Updated weights for policy 0, policy_version 58002 (0.0008) [2023-10-07 22:07:08,570][67871] Updated weights for policy 1, policy_version 58060 (0.0010) [2023-10-07 22:07:08,825][67838] Updated weights for policy 0, policy_version 58012 (0.0008) [2023-10-07 22:07:08,939][67871] Updated weights for policy 1, policy_version 58070 (0.0008) [2023-10-07 22:07:09,312][67871] Updated weights for policy 1, policy_version 58080 (0.0007) [2023-10-07 22:07:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 118882304. Throughput: 0: 1663.2, 1: 1655.4. Samples: 29728436. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:07:12,478][66916] Avg episode reward: [(0, '35.930'), (1, '45.830')] [2023-10-07 22:07:12,952][67838] Updated weights for policy 0, policy_version 58022 (0.0008) [2023-10-07 22:07:13,324][67838] Updated weights for policy 0, policy_version 58032 (0.0010) [2023-10-07 22:07:13,391][67871] Updated weights for policy 1, policy_version 58090 (0.0007) [2023-10-07 22:07:13,692][67838] Updated weights for policy 0, policy_version 58042 (0.0007) [2023-10-07 22:07:13,749][67871] Updated weights for policy 1, policy_version 58100 (0.0008) [2023-10-07 22:07:14,113][67871] Updated weights for policy 1, policy_version 58110 (0.0007) [2023-10-07 22:07:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 118947840. Throughput: 0: 1661.0, 1: 1657.4. Samples: 29748648. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:07:17,477][66916] Avg episode reward: [(0, '35.990'), (1, '45.910')] [2023-10-07 22:07:18,012][67838] Updated weights for policy 0, policy_version 58052 (0.0009) [2023-10-07 22:07:18,150][67871] Updated weights for policy 1, policy_version 58120 (0.0007) [2023-10-07 22:07:18,387][67838] Updated weights for policy 0, policy_version 58062 (0.0008) [2023-10-07 22:07:18,524][67871] Updated weights for policy 1, policy_version 58130 (0.0007) [2023-10-07 22:07:18,757][67838] Updated weights for policy 0, policy_version 58072 (0.0010) [2023-10-07 22:07:18,879][67871] Updated weights for policy 1, policy_version 58140 (0.0008) [2023-10-07 22:07:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 119013376. Throughput: 0: 1656.4, 1: 1656.9. Samples: 29769054. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:07:22,477][66916] Avg episode reward: [(0, '33.180'), (1, '44.600')] [2023-10-07 22:07:22,922][67838] Updated weights for policy 0, policy_version 58082 (0.0007) [2023-10-07 22:07:22,962][67871] Updated weights for policy 1, policy_version 58150 (0.0008) [2023-10-07 22:07:23,302][67838] Updated weights for policy 0, policy_version 58092 (0.0008) [2023-10-07 22:07:23,325][67871] Updated weights for policy 1, policy_version 58160 (0.0007) [2023-10-07 22:07:23,669][67838] Updated weights for policy 0, policy_version 58102 (0.0010) [2023-10-07 22:07:23,695][67871] Updated weights for policy 1, policy_version 58170 (0.0007) [2023-10-07 22:07:24,029][67838] Updated weights for policy 0, policy_version 58112 (0.0009) [2023-10-07 22:07:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119078912. Throughput: 0: 1649.7, 1: 1660.5. Samples: 29778090. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:07:27,477][66916] Avg episode reward: [(0, '38.820'), (1, '42.220')] [2023-10-07 22:07:27,797][67871] Updated weights for policy 1, policy_version 58180 (0.0007) [2023-10-07 22:07:28,151][67838] Updated weights for policy 0, policy_version 58122 (0.0007) [2023-10-07 22:07:28,167][67871] Updated weights for policy 1, policy_version 58190 (0.0007) [2023-10-07 22:07:28,520][67838] Updated weights for policy 0, policy_version 58132 (0.0008) [2023-10-07 22:07:28,530][67871] Updated weights for policy 1, policy_version 58200 (0.0008) [2023-10-07 22:07:28,885][67838] Updated weights for policy 0, policy_version 58142 (0.0008) [2023-10-07 22:07:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119144448. Throughput: 0: 1648.2, 1: 1665.2. Samples: 29798748. Policy #0 lag: (min: 25.0, avg: 46.9, max: 48.0) [2023-10-07 22:07:32,478][66916] Avg episode reward: [(0, '38.470'), (1, '42.280')] [2023-10-07 22:07:32,672][67871] Updated weights for policy 1, policy_version 58210 (0.0008) [2023-10-07 22:07:33,031][67871] Updated weights for policy 1, policy_version 58220 (0.0008) [2023-10-07 22:07:33,123][67838] Updated weights for policy 0, policy_version 58152 (0.0008) [2023-10-07 22:07:33,398][67871] Updated weights for policy 1, policy_version 58230 (0.0008) [2023-10-07 22:07:33,501][67838] Updated weights for policy 0, policy_version 58162 (0.0007) [2023-10-07 22:07:33,764][67871] Updated weights for policy 1, policy_version 58240 (0.0008) [2023-10-07 22:07:33,880][67838] Updated weights for policy 0, policy_version 58172 (0.0008) [2023-10-07 22:07:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119209984. Throughput: 0: 1654.1, 1: 1660.4. Samples: 29819188. Policy #0 lag: (min: 25.0, avg: 46.9, max: 48.0) [2023-10-07 22:07:37,477][66916] Avg episode reward: [(0, '39.310'), (1, '42.030')] [2023-10-07 22:07:37,808][67871] Updated weights for policy 1, policy_version 58250 (0.0008) [2023-10-07 22:07:37,895][67838] Updated weights for policy 0, policy_version 58182 (0.0007) [2023-10-07 22:07:38,175][67871] Updated weights for policy 1, policy_version 58260 (0.0008) [2023-10-07 22:07:38,267][67838] Updated weights for policy 0, policy_version 58192 (0.0009) [2023-10-07 22:07:38,538][67871] Updated weights for policy 1, policy_version 58270 (0.0008) [2023-10-07 22:07:38,632][67838] Updated weights for policy 0, policy_version 58202 (0.0009) [2023-10-07 22:07:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119275520. Throughput: 0: 1651.9, 1: 1660.6. Samples: 29828156. Policy #0 lag: (min: 25.0, avg: 46.9, max: 48.0) [2023-10-07 22:07:42,478][66916] Avg episode reward: [(0, '41.180'), (1, '42.930')] [2023-10-07 22:07:42,738][67871] Updated weights for policy 1, policy_version 58280 (0.0008) [2023-10-07 22:07:42,833][67838] Updated weights for policy 0, policy_version 58212 (0.0008) [2023-10-07 22:07:43,103][67871] Updated weights for policy 1, policy_version 58290 (0.0009) [2023-10-07 22:07:43,207][67838] Updated weights for policy 0, policy_version 58222 (0.0008) [2023-10-07 22:07:43,457][67871] Updated weights for policy 1, policy_version 58300 (0.0009) [2023-10-07 22:07:43,578][67838] Updated weights for policy 0, policy_version 58232 (0.0009) [2023-10-07 22:07:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119341056. Throughput: 0: 1650.4, 1: 1664.3. Samples: 29848570. Policy #0 lag: (min: 25.0, avg: 46.9, max: 48.0) [2023-10-07 22:07:47,477][66916] Avg episode reward: [(0, '41.120'), (1, '44.010')] [2023-10-07 22:07:47,496][67871] Updated weights for policy 1, policy_version 58310 (0.0010) [2023-10-07 22:07:47,528][67838] Updated weights for policy 0, policy_version 58242 (0.0009) [2023-10-07 22:07:47,858][67871] Updated weights for policy 1, policy_version 58320 (0.0009) [2023-10-07 22:07:47,898][67838] Updated weights for policy 0, policy_version 58252 (0.0010) [2023-10-07 22:07:48,228][67871] Updated weights for policy 1, policy_version 58330 (0.0008) [2023-10-07 22:07:48,268][67838] Updated weights for policy 0, policy_version 58262 (0.0008) [2023-10-07 22:07:48,631][67838] Updated weights for policy 0, policy_version 58272 (0.0009) [2023-10-07 22:07:52,328][67871] Updated weights for policy 1, policy_version 58340 (0.0009) [2023-10-07 22:07:52,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119406592. Throughput: 0: 1652.7, 1: 1672.7. Samples: 29869284. Policy #0 lag: (min: 25.0, avg: 46.9, max: 48.0) [2023-10-07 22:07:52,477][66916] Avg episode reward: [(0, '44.400'), (1, '40.980')] [2023-10-07 22:07:52,641][67838] Updated weights for policy 0, policy_version 58282 (0.0007) [2023-10-07 22:07:52,732][67871] Updated weights for policy 1, policy_version 58350 (0.0007) [2023-10-07 22:07:53,008][67838] Updated weights for policy 0, policy_version 58292 (0.0008) [2023-10-07 22:07:53,091][67871] Updated weights for policy 1, policy_version 58360 (0.0008) [2023-10-07 22:07:53,390][67838] Updated weights for policy 0, policy_version 58302 (0.0008) [2023-10-07 22:07:57,139][67871] Updated weights for policy 1, policy_version 58370 (0.0007) [2023-10-07 22:07:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119472128. Throughput: 0: 1652.7, 1: 1672.9. Samples: 29878084. Policy #0 lag: (min: 25.0, avg: 46.9, max: 48.0) [2023-10-07 22:07:57,477][66916] Avg episode reward: [(0, '43.870'), (1, '42.230')] [2023-10-07 22:07:57,512][67871] Updated weights for policy 1, policy_version 58380 (0.0009) [2023-10-07 22:07:57,575][67838] Updated weights for policy 0, policy_version 58312 (0.0008) [2023-10-07 22:07:57,878][67871] Updated weights for policy 1, policy_version 58390 (0.0009) [2023-10-07 22:07:57,935][67838] Updated weights for policy 0, policy_version 58322 (0.0008) [2023-10-07 22:07:58,241][67871] Updated weights for policy 1, policy_version 58400 (0.0008) [2023-10-07 22:07:58,308][67838] Updated weights for policy 0, policy_version 58332 (0.0010) [2023-10-07 22:08:02,361][67871] Updated weights for policy 1, policy_version 58410 (0.0008) [2023-10-07 22:08:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119537664. Throughput: 0: 1652.8, 1: 1673.9. Samples: 29898350. Policy #0 lag: (min: 25.0, avg: 46.9, max: 48.0) [2023-10-07 22:08:02,477][66916] Avg episode reward: [(0, '45.760'), (1, '39.930')] [2023-10-07 22:08:02,567][67838] Updated weights for policy 0, policy_version 58342 (0.0008) [2023-10-07 22:08:02,731][67871] Updated weights for policy 1, policy_version 58420 (0.0009) [2023-10-07 22:08:02,955][67838] Updated weights for policy 0, policy_version 58352 (0.0009) [2023-10-07 22:08:03,093][67871] Updated weights for policy 1, policy_version 58430 (0.0008) [2023-10-07 22:08:03,321][67838] Updated weights for policy 0, policy_version 58362 (0.0008) [2023-10-07 22:08:07,201][67871] Updated weights for policy 1, policy_version 58440 (0.0007) [2023-10-07 22:08:07,413][67838] Updated weights for policy 0, policy_version 58372 (0.0010) [2023-10-07 22:08:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119603200. Throughput: 0: 1650.8, 1: 1671.9. Samples: 29918572. Policy #0 lag: (min: 25.0, avg: 46.9, max: 48.0) [2023-10-07 22:08:07,477][66916] Avg episode reward: [(0, '50.410'), (1, '43.340')] [2023-10-07 22:08:07,571][67871] Updated weights for policy 1, policy_version 58450 (0.0007) [2023-10-07 22:08:07,778][67838] Updated weights for policy 0, policy_version 58382 (0.0008) [2023-10-07 22:08:07,935][67871] Updated weights for policy 1, policy_version 58460 (0.0008) [2023-10-07 22:08:08,143][67838] Updated weights for policy 0, policy_version 58392 (0.0009) [2023-10-07 22:08:12,071][67871] Updated weights for policy 1, policy_version 58470 (0.0009) [2023-10-07 22:08:12,180][67838] Updated weights for policy 0, policy_version 58402 (0.0009) [2023-10-07 22:08:12,432][67871] Updated weights for policy 1, policy_version 58480 (0.0008) [2023-10-07 22:08:12,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119668736. Throughput: 0: 1654.3, 1: 1666.7. Samples: 29927538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:08:12,478][66916] Avg episode reward: [(0, '51.240'), (1, '44.020')] [2023-10-07 22:08:12,545][67838] Updated weights for policy 0, policy_version 58412 (0.0007) [2023-10-07 22:08:12,798][67871] Updated weights for policy 1, policy_version 58490 (0.0008) [2023-10-07 22:08:12,924][67838] Updated weights for policy 0, policy_version 58422 (0.0008) [2023-10-07 22:08:13,295][67838] Updated weights for policy 0, policy_version 58432 (0.0009) [2023-10-07 22:08:17,004][67871] Updated weights for policy 1, policy_version 58500 (0.0007) [2023-10-07 22:08:17,365][67871] Updated weights for policy 1, policy_version 58510 (0.0007) [2023-10-07 22:08:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119734272. Throughput: 0: 1656.1, 1: 1660.7. Samples: 29948002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:08:17,477][66916] Avg episode reward: [(0, '53.210'), (1, '43.440')] [2023-10-07 22:08:17,585][67838] Updated weights for policy 0, policy_version 58442 (0.0008) [2023-10-07 22:08:17,726][67871] Updated weights for policy 1, policy_version 58520 (0.0008) [2023-10-07 22:08:17,942][67838] Updated weights for policy 0, policy_version 58452 (0.0009) [2023-10-07 22:08:18,317][67838] Updated weights for policy 0, policy_version 58462 (0.0009) [2023-10-07 22:08:21,870][67871] Updated weights for policy 1, policy_version 58530 (0.0008) [2023-10-07 22:08:22,234][67871] Updated weights for policy 1, policy_version 58540 (0.0009) [2023-10-07 22:08:22,393][67838] Updated weights for policy 0, policy_version 58472 (0.0009) [2023-10-07 22:08:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119799808. Throughput: 0: 1652.8, 1: 1659.9. Samples: 29968256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:08:22,477][66916] Avg episode reward: [(0, '50.610'), (1, '44.930')] [2023-10-07 22:08:22,609][67871] Updated weights for policy 1, policy_version 58550 (0.0007) [2023-10-07 22:08:22,769][67838] Updated weights for policy 0, policy_version 58482 (0.0007) [2023-10-07 22:08:22,971][67871] Updated weights for policy 1, policy_version 58560 (0.0007) [2023-10-07 22:08:23,129][67838] Updated weights for policy 0, policy_version 58492 (0.0007) [2023-10-07 22:08:27,149][67871] Updated weights for policy 1, policy_version 58570 (0.0009) [2023-10-07 22:08:27,153][67838] Updated weights for policy 0, policy_version 58502 (0.0008) [2023-10-07 22:08:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119865344. Throughput: 0: 1654.2, 1: 1661.3. Samples: 29977354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:08:27,477][66916] Avg episode reward: [(0, '47.490'), (1, '42.790')] [2023-10-07 22:08:27,525][67838] Updated weights for policy 0, policy_version 58512 (0.0010) [2023-10-07 22:08:27,525][67871] Updated weights for policy 1, policy_version 58580 (0.0007) [2023-10-07 22:08:27,894][67838] Updated weights for policy 0, policy_version 58522 (0.0009) [2023-10-07 22:08:27,900][67871] Updated weights for policy 1, policy_version 58590 (0.0008) [2023-10-07 22:08:31,846][67838] Updated weights for policy 0, policy_version 58532 (0.0007) [2023-10-07 22:08:32,077][67871] Updated weights for policy 1, policy_version 58600 (0.0008) [2023-10-07 22:08:32,214][67838] Updated weights for policy 0, policy_version 58542 (0.0007) [2023-10-07 22:08:32,436][67871] Updated weights for policy 1, policy_version 58610 (0.0007) [2023-10-07 22:08:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119930880. Throughput: 0: 1658.7, 1: 1651.1. Samples: 29997512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:08:32,478][66916] Avg episode reward: [(0, '40.400'), (1, '42.460')] [2023-10-07 22:08:32,579][67838] Updated weights for policy 0, policy_version 58552 (0.0008) [2023-10-07 22:08:32,811][67871] Updated weights for policy 1, policy_version 58620 (0.0007) [2023-10-07 22:08:36,877][67838] Updated weights for policy 0, policy_version 58562 (0.0010) [2023-10-07 22:08:36,933][67871] Updated weights for policy 1, policy_version 58630 (0.0008) [2023-10-07 22:08:37,257][67838] Updated weights for policy 0, policy_version 58572 (0.0007) [2023-10-07 22:08:37,295][67871] Updated weights for policy 1, policy_version 58640 (0.0009) [2023-10-07 22:08:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119996416. Throughput: 0: 1650.6, 1: 1642.5. Samples: 30017474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:08:37,477][66916] Avg episode reward: [(0, '43.260'), (1, '40.900')] [2023-10-07 22:08:37,629][67838] Updated weights for policy 0, policy_version 58582 (0.0008) [2023-10-07 22:08:37,668][67871] Updated weights for policy 1, policy_version 58650 (0.0008) [2023-10-07 22:08:38,005][67838] Updated weights for policy 0, policy_version 58592 (0.0008) [2023-10-07 22:08:41,901][67871] Updated weights for policy 1, policy_version 58660 (0.0008) [2023-10-07 22:08:42,131][67838] Updated weights for policy 0, policy_version 58602 (0.0007) [2023-10-07 22:08:42,297][67871] Updated weights for policy 1, policy_version 58670 (0.0009) [2023-10-07 22:08:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120061952. Throughput: 0: 1653.8, 1: 1650.2. Samples: 30026762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:08:42,477][66916] Avg episode reward: [(0, '40.890'), (1, '42.800')] [2023-10-07 22:08:42,507][67838] Updated weights for policy 0, policy_version 58612 (0.0008) [2023-10-07 22:08:42,652][67871] Updated weights for policy 1, policy_version 58680 (0.0008) [2023-10-07 22:08:42,875][67838] Updated weights for policy 0, policy_version 58622 (0.0008) [2023-10-07 22:08:46,843][67871] Updated weights for policy 1, policy_version 58690 (0.0007) [2023-10-07 22:08:47,194][67838] Updated weights for policy 0, policy_version 58632 (0.0007) [2023-10-07 22:08:47,206][67871] Updated weights for policy 1, policy_version 58700 (0.0009) [2023-10-07 22:08:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120127488. Throughput: 0: 1656.0, 1: 1648.5. Samples: 30047052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:08:47,477][66916] Avg episode reward: [(0, '42.230'), (1, '40.640')] [2023-10-07 22:08:47,557][67838] Updated weights for policy 0, policy_version 58642 (0.0008) [2023-10-07 22:08:47,575][67871] Updated weights for policy 1, policy_version 58710 (0.0009) [2023-10-07 22:08:47,939][67838] Updated weights for policy 0, policy_version 58652 (0.0007) [2023-10-07 22:08:47,939][67871] Updated weights for policy 1, policy_version 58720 (0.0010) [2023-10-07 22:08:52,010][67838] Updated weights for policy 0, policy_version 58662 (0.0008) [2023-10-07 22:08:52,040][67871] Updated weights for policy 1, policy_version 58730 (0.0007) [2023-10-07 22:08:52,388][67838] Updated weights for policy 0, policy_version 58672 (0.0007) [2023-10-07 22:08:52,403][67871] Updated weights for policy 1, policy_version 58740 (0.0007) [2023-10-07 22:08:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120193024. Throughput: 0: 1656.2, 1: 1642.6. Samples: 30067016. Policy #0 lag: (min: 26.0, avg: 32.3, max: 58.0) [2023-10-07 22:08:52,477][66916] Avg episode reward: [(0, '45.370'), (1, '45.150')] [2023-10-07 22:08:52,767][67838] Updated weights for policy 0, policy_version 58682 (0.0008) [2023-10-07 22:08:52,775][67871] Updated weights for policy 1, policy_version 58750 (0.0007) [2023-10-07 22:08:56,868][67838] Updated weights for policy 0, policy_version 58692 (0.0008) [2023-10-07 22:08:57,051][67871] Updated weights for policy 1, policy_version 58760 (0.0009) [2023-10-07 22:08:57,248][67838] Updated weights for policy 0, policy_version 58702 (0.0008) [2023-10-07 22:08:57,417][67871] Updated weights for policy 1, policy_version 58770 (0.0009) [2023-10-07 22:08:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120258560. Throughput: 0: 1659.5, 1: 1652.8. Samples: 30076592. Policy #0 lag: (min: 26.0, avg: 32.3, max: 58.0) [2023-10-07 22:08:57,477][66916] Avg episode reward: [(0, '48.730'), (1, '40.590')] [2023-10-07 22:08:57,617][67838] Updated weights for policy 0, policy_version 58712 (0.0008) [2023-10-07 22:08:57,784][67871] Updated weights for policy 1, policy_version 58780 (0.0008) [2023-10-07 22:09:01,769][67838] Updated weights for policy 0, policy_version 58722 (0.0008) [2023-10-07 22:09:01,823][67871] Updated weights for policy 1, policy_version 58790 (0.0008) [2023-10-07 22:09:02,145][67838] Updated weights for policy 0, policy_version 58732 (0.0007) [2023-10-07 22:09:02,191][67871] Updated weights for policy 1, policy_version 58800 (0.0008) [2023-10-07 22:09:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120324096. Throughput: 0: 1655.1, 1: 1651.9. Samples: 30096816. Policy #0 lag: (min: 26.0, avg: 32.3, max: 58.0) [2023-10-07 22:09:02,477][66916] Avg episode reward: [(0, '48.450'), (1, '39.340')] [2023-10-07 22:09:02,517][67838] Updated weights for policy 0, policy_version 58742 (0.0009) [2023-10-07 22:09:02,557][67871] Updated weights for policy 1, policy_version 58810 (0.0007) [2023-10-07 22:09:02,891][67838] Updated weights for policy 0, policy_version 58752 (0.0008) [2023-10-07 22:09:06,552][67871] Updated weights for policy 1, policy_version 58820 (0.0008) [2023-10-07 22:09:06,913][67871] Updated weights for policy 1, policy_version 58830 (0.0007) [2023-10-07 22:09:06,991][67838] Updated weights for policy 0, policy_version 58762 (0.0010) [2023-10-07 22:09:07,279][67871] Updated weights for policy 1, policy_version 58840 (0.0008) [2023-10-07 22:09:07,360][67838] Updated weights for policy 0, policy_version 58772 (0.0008) [2023-10-07 22:09:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120389632. Throughput: 0: 1645.6, 1: 1647.2. Samples: 30116432. Policy #0 lag: (min: 26.0, avg: 32.3, max: 58.0) [2023-10-07 22:09:07,477][66916] Avg episode reward: [(0, '53.220'), (1, '40.710')] [2023-10-07 22:09:07,567][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000058848_60260352.pth... [2023-10-07 22:09:07,595][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000057280_58654720.pth [2023-10-07 22:09:07,727][67838] Updated weights for policy 0, policy_version 58782 (0.0008) [2023-10-07 22:09:07,798][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000058784_60194816.pth... [2023-10-07 22:09:07,826][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000057216_58589184.pth [2023-10-07 22:09:11,223][67871] Updated weights for policy 1, policy_version 58850 (0.0008) [2023-10-07 22:09:11,589][67871] Updated weights for policy 1, policy_version 58860 (0.0010) [2023-10-07 22:09:11,954][67871] Updated weights for policy 1, policy_version 58870 (0.0009) [2023-10-07 22:09:12,090][67838] Updated weights for policy 0, policy_version 58792 (0.0008) [2023-10-07 22:09:12,317][67871] Updated weights for policy 1, policy_version 58880 (0.0008) [2023-10-07 22:09:12,465][67838] Updated weights for policy 0, policy_version 58802 (0.0008) [2023-10-07 22:09:12,476][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 120487936. Throughput: 0: 1650.6, 1: 1659.4. Samples: 30126302. Policy #0 lag: (min: 26.0, avg: 32.3, max: 58.0) [2023-10-07 22:09:12,477][66916] Avg episode reward: [(0, '51.780'), (1, '38.410')] [2023-10-07 22:09:12,828][67838] Updated weights for policy 0, policy_version 58812 (0.0007) [2023-10-07 22:09:16,420][67871] Updated weights for policy 1, policy_version 58890 (0.0010) [2023-10-07 22:09:16,793][67871] Updated weights for policy 1, policy_version 58900 (0.0009) [2023-10-07 22:09:16,891][67838] Updated weights for policy 0, policy_version 58822 (0.0009) [2023-10-07 22:09:17,151][67871] Updated weights for policy 1, policy_version 58910 (0.0007) [2023-10-07 22:09:17,267][67838] Updated weights for policy 0, policy_version 58832 (0.0010) [2023-10-07 22:09:17,476][66916] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 120553472. Throughput: 0: 1649.7, 1: 1670.9. Samples: 30146940. Policy #0 lag: (min: 26.0, avg: 32.3, max: 58.0) [2023-10-07 22:09:17,477][66916] Avg episode reward: [(0, '50.580'), (1, '41.370')] [2023-10-07 22:09:17,646][67838] Updated weights for policy 0, policy_version 58842 (0.0008) [2023-10-07 22:09:21,133][67871] Updated weights for policy 1, policy_version 58920 (0.0007) [2023-10-07 22:09:21,501][67871] Updated weights for policy 1, policy_version 58930 (0.0007) [2023-10-07 22:09:21,688][67838] Updated weights for policy 0, policy_version 58852 (0.0007) [2023-10-07 22:09:21,872][67871] Updated weights for policy 1, policy_version 58940 (0.0008) [2023-10-07 22:09:22,063][67838] Updated weights for policy 0, policy_version 58862 (0.0009) [2023-10-07 22:09:22,429][67838] Updated weights for policy 0, policy_version 58872 (0.0007) [2023-10-07 22:09:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 120619008. Throughput: 0: 1644.2, 1: 1654.5. Samples: 30165916. Policy #0 lag: (min: 26.0, avg: 32.3, max: 58.0) [2023-10-07 22:09:22,477][66916] Avg episode reward: [(0, '49.960'), (1, '41.140')] [2023-10-07 22:09:26,058][67871] Updated weights for policy 1, policy_version 58950 (0.0008) [2023-10-07 22:09:26,418][67871] Updated weights for policy 1, policy_version 58960 (0.0007) [2023-10-07 22:09:26,676][67838] Updated weights for policy 0, policy_version 58882 (0.0009) [2023-10-07 22:09:26,794][67871] Updated weights for policy 1, policy_version 58970 (0.0008) [2023-10-07 22:09:27,038][67838] Updated weights for policy 0, policy_version 58892 (0.0008) [2023-10-07 22:09:27,410][67838] Updated weights for policy 0, policy_version 58902 (0.0008) [2023-10-07 22:09:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 120684544. Throughput: 0: 1649.5, 1: 1678.2. Samples: 30176510. Policy #0 lag: (min: 26.0, avg: 32.3, max: 58.0) [2023-10-07 22:09:27,477][66916] Avg episode reward: [(0, '48.420'), (1, '42.620')] [2023-10-07 22:09:27,784][67838] Updated weights for policy 0, policy_version 58912 (0.0008) [2023-10-07 22:09:30,943][67871] Updated weights for policy 1, policy_version 58980 (0.0009) [2023-10-07 22:09:31,344][67871] Updated weights for policy 1, policy_version 58990 (0.0008) [2023-10-07 22:09:31,699][67871] Updated weights for policy 1, policy_version 59000 (0.0008) [2023-10-07 22:09:31,872][67838] Updated weights for policy 0, policy_version 58922 (0.0008) [2023-10-07 22:09:32,244][67838] Updated weights for policy 0, policy_version 58932 (0.0008) [2023-10-07 22:09:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 120750080. Throughput: 0: 1652.4, 1: 1680.1. Samples: 30197016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:09:32,478][66916] Avg episode reward: [(0, '50.000'), (1, '41.140')] [2023-10-07 22:09:32,627][67838] Updated weights for policy 0, policy_version 58942 (0.0009) [2023-10-07 22:09:35,902][67871] Updated weights for policy 1, policy_version 59010 (0.0008) [2023-10-07 22:09:36,261][67871] Updated weights for policy 1, policy_version 59020 (0.0008) [2023-10-07 22:09:36,635][67871] Updated weights for policy 1, policy_version 59030 (0.0009) [2023-10-07 22:09:36,788][67838] Updated weights for policy 0, policy_version 58952 (0.0007) [2023-10-07 22:09:37,002][67871] Updated weights for policy 1, policy_version 59040 (0.0008) [2023-10-07 22:09:37,169][67838] Updated weights for policy 0, policy_version 58962 (0.0009) [2023-10-07 22:09:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 120815616. Throughput: 0: 1641.3, 1: 1661.2. Samples: 30215632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:09:37,477][66916] Avg episode reward: [(0, '49.340'), (1, '41.910')] [2023-10-07 22:09:37,548][67838] Updated weights for policy 0, policy_version 58972 (0.0010) [2023-10-07 22:09:41,276][67871] Updated weights for policy 1, policy_version 59050 (0.0008) [2023-10-07 22:09:41,587][67838] Updated weights for policy 0, policy_version 58982 (0.0009) [2023-10-07 22:09:41,643][67871] Updated weights for policy 1, policy_version 59060 (0.0009) [2023-10-07 22:09:41,956][67838] Updated weights for policy 0, policy_version 58992 (0.0007) [2023-10-07 22:09:42,007][67871] Updated weights for policy 1, policy_version 59070 (0.0009) [2023-10-07 22:09:42,336][67838] Updated weights for policy 0, policy_version 59002 (0.0009) [2023-10-07 22:09:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 120881152. Throughput: 0: 1647.6, 1: 1675.0. Samples: 30226112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:09:42,477][66916] Avg episode reward: [(0, '49.350'), (1, '40.450')] [2023-10-07 22:09:45,981][67871] Updated weights for policy 1, policy_version 59080 (0.0008) [2023-10-07 22:09:46,344][67871] Updated weights for policy 1, policy_version 59090 (0.0010) [2023-10-07 22:09:46,553][67838] Updated weights for policy 0, policy_version 59012 (0.0011) [2023-10-07 22:09:46,699][67871] Updated weights for policy 1, policy_version 59100 (0.0009) [2023-10-07 22:09:46,928][67838] Updated weights for policy 0, policy_version 59022 (0.0007) [2023-10-07 22:09:47,296][67838] Updated weights for policy 0, policy_version 59032 (0.0008) [2023-10-07 22:09:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 120946688. Throughput: 0: 1650.0, 1: 1673.9. Samples: 30246394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:09:47,477][66916] Avg episode reward: [(0, '50.450'), (1, '44.500')] [2023-10-07 22:09:50,820][67871] Updated weights for policy 1, policy_version 59110 (0.0009) [2023-10-07 22:09:51,190][67871] Updated weights for policy 1, policy_version 59120 (0.0007) [2023-10-07 22:09:51,538][67838] Updated weights for policy 0, policy_version 59042 (0.0011) [2023-10-07 22:09:51,565][67871] Updated weights for policy 1, policy_version 59130 (0.0007) [2023-10-07 22:09:51,904][67838] Updated weights for policy 0, policy_version 59052 (0.0009) [2023-10-07 22:09:52,287][67838] Updated weights for policy 0, policy_version 59062 (0.0008) [2023-10-07 22:09:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 121012224. Throughput: 0: 1644.7, 1: 1658.0. Samples: 30265052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:09:52,478][66916] Avg episode reward: [(0, '53.070'), (1, '43.080')] [2023-10-07 22:09:52,658][67838] Updated weights for policy 0, policy_version 59072 (0.0007) [2023-10-07 22:09:55,624][67871] Updated weights for policy 1, policy_version 59140 (0.0008) [2023-10-07 22:09:56,000][67871] Updated weights for policy 1, policy_version 59150 (0.0007) [2023-10-07 22:09:56,374][67871] Updated weights for policy 1, policy_version 59160 (0.0008) [2023-10-07 22:09:56,654][67838] Updated weights for policy 0, policy_version 59082 (0.0007) [2023-10-07 22:09:57,022][67838] Updated weights for policy 0, policy_version 59092 (0.0008) [2023-10-07 22:09:57,403][67838] Updated weights for policy 0, policy_version 59102 (0.0008) [2023-10-07 22:09:57,477][66916] Fps is (10 sec: 16383.4, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 121110528. Throughput: 0: 1648.8, 1: 1672.8. Samples: 30275778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:09:57,478][66916] Avg episode reward: [(0, '53.190'), (1, '46.760')] [2023-10-07 22:10:00,450][67871] Updated weights for policy 1, policy_version 59170 (0.0009) [2023-10-07 22:10:00,808][67871] Updated weights for policy 1, policy_version 59180 (0.0010) [2023-10-07 22:10:01,175][67871] Updated weights for policy 1, policy_version 59190 (0.0010) [2023-10-07 22:10:01,545][67871] Updated weights for policy 1, policy_version 59200 (0.0010) [2023-10-07 22:10:01,607][67838] Updated weights for policy 0, policy_version 59112 (0.0009) [2023-10-07 22:10:01,978][67838] Updated weights for policy 0, policy_version 59122 (0.0009) [2023-10-07 22:10:02,348][67838] Updated weights for policy 0, policy_version 59132 (0.0008) [2023-10-07 22:10:02,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 121143296. Throughput: 0: 1648.2, 1: 1653.1. Samples: 30295500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:10:02,477][66916] Avg episode reward: [(0, '56.950'), (1, '47.120')] [2023-10-07 22:10:05,576][67871] Updated weights for policy 1, policy_version 59210 (0.0009) [2023-10-07 22:10:05,946][67871] Updated weights for policy 1, policy_version 59220 (0.0009) [2023-10-07 22:10:06,317][67871] Updated weights for policy 1, policy_version 59230 (0.0009) [2023-10-07 22:10:06,405][67838] Updated weights for policy 0, policy_version 59142 (0.0008) [2023-10-07 22:10:06,763][67838] Updated weights for policy 0, policy_version 59152 (0.0009) [2023-10-07 22:10:07,132][67838] Updated weights for policy 0, policy_version 59162 (0.0008) [2023-10-07 22:10:07,477][66916] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 121241600. Throughput: 0: 1643.0, 1: 1660.9. Samples: 30314590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:10:07,478][66916] Avg episode reward: [(0, '56.240'), (1, '48.160')] [2023-10-07 22:10:10,372][67871] Updated weights for policy 1, policy_version 59240 (0.0010) [2023-10-07 22:10:10,725][67871] Updated weights for policy 1, policy_version 59250 (0.0008) [2023-10-07 22:10:11,094][67871] Updated weights for policy 1, policy_version 59260 (0.0008) [2023-10-07 22:10:11,164][67838] Updated weights for policy 0, policy_version 59172 (0.0010) [2023-10-07 22:10:11,547][67838] Updated weights for policy 0, policy_version 59182 (0.0009) [2023-10-07 22:10:11,912][67838] Updated weights for policy 0, policy_version 59192 (0.0010) [2023-10-07 22:10:12,476][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 121307136. Throughput: 0: 1653.1, 1: 1662.7. Samples: 30325720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:10:12,477][66916] Avg episode reward: [(0, '59.710'), (1, '45.970')] [2023-10-07 22:10:15,210][67871] Updated weights for policy 1, policy_version 59270 (0.0010) [2023-10-07 22:10:15,568][67871] Updated weights for policy 1, policy_version 59280 (0.0008) [2023-10-07 22:10:15,941][67871] Updated weights for policy 1, policy_version 59290 (0.0009) [2023-10-07 22:10:16,106][67838] Updated weights for policy 0, policy_version 59202 (0.0009) [2023-10-07 22:10:16,469][67838] Updated weights for policy 0, policy_version 59212 (0.0009) [2023-10-07 22:10:16,843][67838] Updated weights for policy 0, policy_version 59222 (0.0009) [2023-10-07 22:10:17,216][67838] Updated weights for policy 0, policy_version 59232 (0.0009) [2023-10-07 22:10:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 121372672. Throughput: 0: 1650.5, 1: 1644.0. Samples: 30345270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:10:17,477][66916] Avg episode reward: [(0, '55.990'), (1, '46.870')] [2023-10-07 22:10:19,980][67871] Updated weights for policy 1, policy_version 59300 (0.0007) [2023-10-07 22:10:20,350][67871] Updated weights for policy 1, policy_version 59310 (0.0009) [2023-10-07 22:10:20,715][67871] Updated weights for policy 1, policy_version 59320 (0.0007) [2023-10-07 22:10:21,389][67838] Updated weights for policy 0, policy_version 59242 (0.0008) [2023-10-07 22:10:21,760][67838] Updated weights for policy 0, policy_version 59252 (0.0009) [2023-10-07 22:10:22,129][67838] Updated weights for policy 0, policy_version 59262 (0.0009) [2023-10-07 22:10:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 121438208. Throughput: 0: 1644.0, 1: 1665.2. Samples: 30364550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:10:22,477][66916] Avg episode reward: [(0, '57.350'), (1, '49.860')] [2023-10-07 22:10:24,753][67871] Updated weights for policy 1, policy_version 59330 (0.0008) [2023-10-07 22:10:25,121][67871] Updated weights for policy 1, policy_version 59340 (0.0008) [2023-10-07 22:10:25,484][67871] Updated weights for policy 1, policy_version 59350 (0.0007) [2023-10-07 22:10:25,851][67871] Updated weights for policy 1, policy_version 59360 (0.0010) [2023-10-07 22:10:26,520][67838] Updated weights for policy 0, policy_version 59272 (0.0007) [2023-10-07 22:10:26,894][67838] Updated weights for policy 0, policy_version 59282 (0.0010) [2023-10-07 22:10:27,267][67838] Updated weights for policy 0, policy_version 59292 (0.0007) [2023-10-07 22:10:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 121503744. Throughput: 0: 1659.0, 1: 1674.3. Samples: 30376112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:10:27,478][66916] Avg episode reward: [(0, '51.480'), (1, '50.040')] [2023-10-07 22:10:30,154][67871] Updated weights for policy 1, policy_version 59370 (0.0008) [2023-10-07 22:10:30,527][67871] Updated weights for policy 1, policy_version 59380 (0.0009) [2023-10-07 22:10:30,892][67871] Updated weights for policy 1, policy_version 59390 (0.0009) [2023-10-07 22:10:31,363][67838] Updated weights for policy 0, policy_version 59302 (0.0008) [2023-10-07 22:10:31,737][67838] Updated weights for policy 0, policy_version 59312 (0.0011) [2023-10-07 22:10:32,113][67838] Updated weights for policy 0, policy_version 59322 (0.0008) [2023-10-07 22:10:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 121569280. Throughput: 0: 1655.7, 1: 1656.1. Samples: 30395426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:10:32,477][66916] Avg episode reward: [(0, '50.540'), (1, '49.360')] [2023-10-07 22:10:34,906][67871] Updated weights for policy 1, policy_version 59400 (0.0007) [2023-10-07 22:10:35,277][67871] Updated weights for policy 1, policy_version 59410 (0.0007) [2023-10-07 22:10:35,647][67871] Updated weights for policy 1, policy_version 59420 (0.0007) [2023-10-07 22:10:36,308][67838] Updated weights for policy 0, policy_version 59332 (0.0007) [2023-10-07 22:10:36,689][67838] Updated weights for policy 0, policy_version 59342 (0.0010) [2023-10-07 22:10:37,068][67838] Updated weights for policy 0, policy_version 59352 (0.0010) [2023-10-07 22:10:37,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 121634816. Throughput: 0: 1649.7, 1: 1679.1. Samples: 30414848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:10:37,478][66916] Avg episode reward: [(0, '52.030'), (1, '49.850')] [2023-10-07 22:10:39,834][67871] Updated weights for policy 1, policy_version 59430 (0.0008) [2023-10-07 22:10:40,195][67871] Updated weights for policy 1, policy_version 59440 (0.0007) [2023-10-07 22:10:40,563][67871] Updated weights for policy 1, policy_version 59450 (0.0009) [2023-10-07 22:10:40,933][67838] Updated weights for policy 0, policy_version 59362 (0.0007) [2023-10-07 22:10:41,303][67838] Updated weights for policy 0, policy_version 59372 (0.0009) [2023-10-07 22:10:41,689][67838] Updated weights for policy 0, policy_version 59382 (0.0011) [2023-10-07 22:10:42,058][67838] Updated weights for policy 0, policy_version 59392 (0.0010) [2023-10-07 22:10:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 121700352. Throughput: 0: 1661.6, 1: 1673.6. Samples: 30425862. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:10:42,477][66916] Avg episode reward: [(0, '50.580'), (1, '50.550')] [2023-10-07 22:10:44,651][67871] Updated weights for policy 1, policy_version 59460 (0.0010) [2023-10-07 22:10:45,017][67871] Updated weights for policy 1, policy_version 59470 (0.0009) [2023-10-07 22:10:45,379][67871] Updated weights for policy 1, policy_version 59480 (0.0010) [2023-10-07 22:10:46,114][67838] Updated weights for policy 0, policy_version 59402 (0.0007) [2023-10-07 22:10:46,490][67838] Updated weights for policy 0, policy_version 59412 (0.0008) [2023-10-07 22:10:46,871][67838] Updated weights for policy 0, policy_version 59422 (0.0009) [2023-10-07 22:10:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 121765888. Throughput: 0: 1661.8, 1: 1667.7. Samples: 30445326. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-07 22:10:47,477][66916] Avg episode reward: [(0, '48.050'), (1, '48.070')] [2023-10-07 22:10:49,308][67871] Updated weights for policy 1, policy_version 59490 (0.0007) [2023-10-07 22:10:49,668][67871] Updated weights for policy 1, policy_version 59500 (0.0007) [2023-10-07 22:10:50,034][67871] Updated weights for policy 1, policy_version 59510 (0.0007) [2023-10-07 22:10:50,407][67871] Updated weights for policy 1, policy_version 59520 (0.0009) [2023-10-07 22:10:51,007][67838] Updated weights for policy 0, policy_version 59432 (0.0007) [2023-10-07 22:10:51,378][67838] Updated weights for policy 0, policy_version 59442 (0.0008) [2023-10-07 22:10:51,747][67838] Updated weights for policy 0, policy_version 59452 (0.0009) [2023-10-07 22:10:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 121831424. Throughput: 0: 1654.8, 1: 1687.6. Samples: 30464994. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-07 22:10:52,477][66916] Avg episode reward: [(0, '50.680'), (1, '48.090')] [2023-10-07 22:10:54,443][67871] Updated weights for policy 1, policy_version 59530 (0.0008) [2023-10-07 22:10:54,819][67871] Updated weights for policy 1, policy_version 59540 (0.0009) [2023-10-07 22:10:55,183][67871] Updated weights for policy 1, policy_version 59550 (0.0009) [2023-10-07 22:10:55,835][67838] Updated weights for policy 0, policy_version 59462 (0.0009) [2023-10-07 22:10:56,209][67838] Updated weights for policy 0, policy_version 59472 (0.0011) [2023-10-07 22:10:56,564][67838] Updated weights for policy 0, policy_version 59482 (0.0010) [2023-10-07 22:10:57,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 121896960. Throughput: 0: 1665.8, 1: 1668.3. Samples: 30475754. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-07 22:10:57,478][66916] Avg episode reward: [(0, '48.240'), (1, '49.320')] [2023-10-07 22:10:59,272][67871] Updated weights for policy 1, policy_version 59560 (0.0009) [2023-10-07 22:10:59,637][67871] Updated weights for policy 1, policy_version 59570 (0.0009) [2023-10-07 22:11:00,003][67871] Updated weights for policy 1, policy_version 59580 (0.0009) [2023-10-07 22:11:00,677][67838] Updated weights for policy 0, policy_version 59492 (0.0011) [2023-10-07 22:11:01,041][67838] Updated weights for policy 0, policy_version 59502 (0.0011) [2023-10-07 22:11:01,416][67838] Updated weights for policy 0, policy_version 59512 (0.0008) [2023-10-07 22:11:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 121962496. Throughput: 0: 1652.0, 1: 1679.3. Samples: 30495178. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-07 22:11:02,477][66916] Avg episode reward: [(0, '47.250'), (1, '46.450')] [2023-10-07 22:11:04,213][67871] Updated weights for policy 1, policy_version 59590 (0.0007) [2023-10-07 22:11:04,606][67871] Updated weights for policy 1, policy_version 59600 (0.0008) [2023-10-07 22:11:04,972][67871] Updated weights for policy 1, policy_version 59610 (0.0010) [2023-10-07 22:11:05,591][67838] Updated weights for policy 0, policy_version 59522 (0.0007) [2023-10-07 22:11:05,970][67838] Updated weights for policy 0, policy_version 59532 (0.0007) [2023-10-07 22:11:06,339][67838] Updated weights for policy 0, policy_version 59542 (0.0009) [2023-10-07 22:11:06,715][67838] Updated weights for policy 0, policy_version 59552 (0.0009) [2023-10-07 22:11:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 122028032. Throughput: 0: 1657.3, 1: 1691.9. Samples: 30515262. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-07 22:11:07,477][66916] Avg episode reward: [(0, '46.370'), (1, '45.310')] [2023-10-07 22:11:07,484][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000059552_60981248.pth... [2023-10-07 22:11:07,484][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000059616_61046784.pth... [2023-10-07 22:11:07,521][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000058048_59441152.pth [2023-10-07 22:11:07,524][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000057984_59375616.pth [2023-10-07 22:11:08,877][67871] Updated weights for policy 1, policy_version 59620 (0.0011) [2023-10-07 22:11:09,247][67871] Updated weights for policy 1, policy_version 59630 (0.0007) [2023-10-07 22:11:09,609][67871] Updated weights for policy 1, policy_version 59640 (0.0007) [2023-10-07 22:11:10,785][67838] Updated weights for policy 0, policy_version 59562 (0.0009) [2023-10-07 22:11:11,158][67838] Updated weights for policy 0, policy_version 59572 (0.0008) [2023-10-07 22:11:11,538][67838] Updated weights for policy 0, policy_version 59582 (0.0007) [2023-10-07 22:11:12,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122093568. Throughput: 0: 1661.1, 1: 1664.0. Samples: 30525738. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-07 22:11:12,478][66916] Avg episode reward: [(0, '49.230'), (1, '46.700')] [2023-10-07 22:11:13,726][67871] Updated weights for policy 1, policy_version 59650 (0.0009) [2023-10-07 22:11:14,093][67871] Updated weights for policy 1, policy_version 59660 (0.0009) [2023-10-07 22:11:14,469][67871] Updated weights for policy 1, policy_version 59670 (0.0008) [2023-10-07 22:11:14,839][67871] Updated weights for policy 1, policy_version 59680 (0.0007) [2023-10-07 22:11:15,565][67838] Updated weights for policy 0, policy_version 59592 (0.0009) [2023-10-07 22:11:15,933][67838] Updated weights for policy 0, policy_version 59602 (0.0010) [2023-10-07 22:11:16,312][67838] Updated weights for policy 0, policy_version 59612 (0.0012) [2023-10-07 22:11:17,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122159104. Throughput: 0: 1645.2, 1: 1686.6. Samples: 30545358. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-07 22:11:17,478][66916] Avg episode reward: [(0, '47.560'), (1, '47.120')] [2023-10-07 22:11:18,836][67871] Updated weights for policy 1, policy_version 59690 (0.0010) [2023-10-07 22:11:19,212][67871] Updated weights for policy 1, policy_version 59700 (0.0010) [2023-10-07 22:11:19,575][67871] Updated weights for policy 1, policy_version 59710 (0.0009) [2023-10-07 22:11:20,522][67838] Updated weights for policy 0, policy_version 59622 (0.0009) [2023-10-07 22:11:20,897][67838] Updated weights for policy 0, policy_version 59632 (0.0008) [2023-10-07 22:11:21,257][67838] Updated weights for policy 0, policy_version 59642 (0.0008) [2023-10-07 22:11:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 122224640. Throughput: 0: 1657.4, 1: 1689.3. Samples: 30565450. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-07 22:11:22,478][66916] Avg episode reward: [(0, '49.520'), (1, '51.130')] [2023-10-07 22:11:23,858][67871] Updated weights for policy 1, policy_version 59720 (0.0008) [2023-10-07 22:11:24,231][67871] Updated weights for policy 1, policy_version 59730 (0.0011) [2023-10-07 22:11:24,595][67871] Updated weights for policy 1, policy_version 59740 (0.0010) [2023-10-07 22:11:25,285][67838] Updated weights for policy 0, policy_version 59652 (0.0007) [2023-10-07 22:11:25,648][67838] Updated weights for policy 0, policy_version 59662 (0.0007) [2023-10-07 22:11:26,022][67838] Updated weights for policy 0, policy_version 59672 (0.0007) [2023-10-07 22:11:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122290176. Throughput: 0: 1667.1, 1: 1666.1. Samples: 30575854. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-07 22:11:27,478][66916] Avg episode reward: [(0, '45.080'), (1, '50.180')] [2023-10-07 22:11:28,869][67871] Updated weights for policy 1, policy_version 59750 (0.0009) [2023-10-07 22:11:29,242][67871] Updated weights for policy 1, policy_version 59760 (0.0008) [2023-10-07 22:11:29,607][67871] Updated weights for policy 1, policy_version 59770 (0.0007) [2023-10-07 22:11:30,289][67838] Updated weights for policy 0, policy_version 59682 (0.0009) [2023-10-07 22:11:30,663][67838] Updated weights for policy 0, policy_version 59692 (0.0008) [2023-10-07 22:11:31,024][67838] Updated weights for policy 0, policy_version 59702 (0.0008) [2023-10-07 22:11:31,388][67838] Updated weights for policy 0, policy_version 59712 (0.0010) [2023-10-07 22:11:32,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122355712. Throughput: 0: 1649.8, 1: 1685.1. Samples: 30595396. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-07 22:11:32,477][66916] Avg episode reward: [(0, '46.520'), (1, '50.070')] [2023-10-07 22:11:33,435][67871] Updated weights for policy 1, policy_version 59780 (0.0008) [2023-10-07 22:11:33,803][67871] Updated weights for policy 1, policy_version 59790 (0.0009) [2023-10-07 22:11:34,171][67871] Updated weights for policy 1, policy_version 59800 (0.0007) [2023-10-07 22:11:35,540][67838] Updated weights for policy 0, policy_version 59722 (0.0011) [2023-10-07 22:11:35,907][67838] Updated weights for policy 0, policy_version 59732 (0.0009) [2023-10-07 22:11:36,283][67838] Updated weights for policy 0, policy_version 59742 (0.0007) [2023-10-07 22:11:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 122421248. Throughput: 0: 1666.1, 1: 1683.2. Samples: 30615712. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-07 22:11:37,478][66916] Avg episode reward: [(0, '46.400'), (1, '50.830')] [2023-10-07 22:11:38,308][67871] Updated weights for policy 1, policy_version 59810 (0.0007) [2023-10-07 22:11:38,679][67871] Updated weights for policy 1, policy_version 59820 (0.0009) [2023-10-07 22:11:39,048][67871] Updated weights for policy 1, policy_version 59830 (0.0009) [2023-10-07 22:11:39,409][67871] Updated weights for policy 1, policy_version 59840 (0.0007) [2023-10-07 22:11:40,425][67838] Updated weights for policy 0, policy_version 59752 (0.0007) [2023-10-07 22:11:40,790][67838] Updated weights for policy 0, policy_version 59762 (0.0009) [2023-10-07 22:11:41,153][67838] Updated weights for policy 0, policy_version 59772 (0.0011) [2023-10-07 22:11:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122486784. Throughput: 0: 1668.1, 1: 1674.0. Samples: 30626150. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-07 22:11:42,478][66916] Avg episode reward: [(0, '49.130'), (1, '49.520')] [2023-10-07 22:11:43,555][67871] Updated weights for policy 1, policy_version 59850 (0.0007) [2023-10-07 22:11:43,918][67871] Updated weights for policy 1, policy_version 59860 (0.0008) [2023-10-07 22:11:44,294][67871] Updated weights for policy 1, policy_version 59870 (0.0008) [2023-10-07 22:11:45,229][67838] Updated weights for policy 0, policy_version 59782 (0.0008) [2023-10-07 22:11:45,601][67838] Updated weights for policy 0, policy_version 59792 (0.0007) [2023-10-07 22:11:45,969][67838] Updated weights for policy 0, policy_version 59802 (0.0009) [2023-10-07 22:11:47,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122552320. Throughput: 0: 1661.2, 1: 1680.5. Samples: 30645554. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-07 22:11:47,478][66916] Avg episode reward: [(0, '51.550'), (1, '47.970')] [2023-10-07 22:11:48,099][67871] Updated weights for policy 1, policy_version 59880 (0.0010) [2023-10-07 22:11:48,463][67871] Updated weights for policy 1, policy_version 59890 (0.0008) [2023-10-07 22:11:48,840][67871] Updated weights for policy 1, policy_version 59900 (0.0008) [2023-10-07 22:11:49,899][67838] Updated weights for policy 0, policy_version 59812 (0.0009) [2023-10-07 22:11:50,271][67838] Updated weights for policy 0, policy_version 59822 (0.0009) [2023-10-07 22:11:50,645][67838] Updated weights for policy 0, policy_version 59832 (0.0010) [2023-10-07 22:11:52,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122617856. Throughput: 0: 1674.6, 1: 1675.2. Samples: 30666002. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-07 22:11:52,478][66916] Avg episode reward: [(0, '48.590'), (1, '49.240')] [2023-10-07 22:11:52,996][67871] Updated weights for policy 1, policy_version 59910 (0.0008) [2023-10-07 22:11:53,355][67871] Updated weights for policy 1, policy_version 59920 (0.0011) [2023-10-07 22:11:53,734][67871] Updated weights for policy 1, policy_version 59930 (0.0012) [2023-10-07 22:11:54,825][67838] Updated weights for policy 0, policy_version 59842 (0.0011) [2023-10-07 22:11:55,194][67838] Updated weights for policy 0, policy_version 59852 (0.0008) [2023-10-07 22:11:55,581][67838] Updated weights for policy 0, policy_version 59862 (0.0010) [2023-10-07 22:11:55,941][67838] Updated weights for policy 0, policy_version 59872 (0.0008) [2023-10-07 22:11:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 122683392. Throughput: 0: 1662.4, 1: 1676.8. Samples: 30676002. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-07 22:11:57,478][66916] Avg episode reward: [(0, '50.000'), (1, '50.460')] [2023-10-07 22:11:57,758][67871] Updated weights for policy 1, policy_version 59940 (0.0009) [2023-10-07 22:11:58,115][67871] Updated weights for policy 1, policy_version 59950 (0.0007) [2023-10-07 22:11:58,475][67871] Updated weights for policy 1, policy_version 59960 (0.0007) [2023-10-07 22:12:00,157][67838] Updated weights for policy 0, policy_version 59882 (0.0009) [2023-10-07 22:12:00,539][67838] Updated weights for policy 0, policy_version 59892 (0.0010) [2023-10-07 22:12:00,909][67838] Updated weights for policy 0, policy_version 59902 (0.0011) [2023-10-07 22:12:02,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 122748928. Throughput: 0: 1660.6, 1: 1675.2. Samples: 30695468. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-07 22:12:02,478][66916] Avg episode reward: [(0, '45.400'), (1, '50.700')] [2023-10-07 22:12:02,717][67871] Updated weights for policy 1, policy_version 59970 (0.0008) [2023-10-07 22:12:03,088][67871] Updated weights for policy 1, policy_version 59980 (0.0009) [2023-10-07 22:12:03,450][67871] Updated weights for policy 1, policy_version 59990 (0.0009) [2023-10-07 22:12:03,808][67871] Updated weights for policy 1, policy_version 60000 (0.0007) [2023-10-07 22:12:04,797][67838] Updated weights for policy 0, policy_version 59912 (0.0008) [2023-10-07 22:12:05,167][67838] Updated weights for policy 0, policy_version 59922 (0.0009) [2023-10-07 22:12:05,548][67838] Updated weights for policy 0, policy_version 59932 (0.0008) [2023-10-07 22:12:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122814464. Throughput: 0: 1674.7, 1: 1677.3. Samples: 30716290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:12:07,477][66916] Avg episode reward: [(0, '45.280'), (1, '49.150')] [2023-10-07 22:12:07,829][67871] Updated weights for policy 1, policy_version 60010 (0.0007) [2023-10-07 22:12:08,199][67871] Updated weights for policy 1, policy_version 60020 (0.0009) [2023-10-07 22:12:08,564][67871] Updated weights for policy 1, policy_version 60030 (0.0008) [2023-10-07 22:12:09,709][67838] Updated weights for policy 0, policy_version 59942 (0.0007) [2023-10-07 22:12:10,071][67838] Updated weights for policy 0, policy_version 59952 (0.0007) [2023-10-07 22:12:10,439][67838] Updated weights for policy 0, policy_version 59962 (0.0010) [2023-10-07 22:12:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122880000. Throughput: 0: 1659.8, 1: 1678.7. Samples: 30726088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:12:12,477][66916] Avg episode reward: [(0, '41.840'), (1, '51.160')] [2023-10-07 22:12:12,664][67871] Updated weights for policy 1, policy_version 60040 (0.0009) [2023-10-07 22:12:13,031][67871] Updated weights for policy 1, policy_version 60050 (0.0008) [2023-10-07 22:12:13,393][67871] Updated weights for policy 1, policy_version 60060 (0.0007) [2023-10-07 22:12:14,536][67838] Updated weights for policy 0, policy_version 59972 (0.0008) [2023-10-07 22:12:14,905][67838] Updated weights for policy 0, policy_version 59982 (0.0007) [2023-10-07 22:12:15,266][67838] Updated weights for policy 0, policy_version 59992 (0.0008) [2023-10-07 22:12:17,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122945536. Throughput: 0: 1662.6, 1: 1679.3. Samples: 30745784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:12:17,477][66916] Avg episode reward: [(0, '46.240'), (1, '50.110')] [2023-10-07 22:12:17,554][67871] Updated weights for policy 1, policy_version 60070 (0.0008) [2023-10-07 22:12:17,918][67871] Updated weights for policy 1, policy_version 60080 (0.0007) [2023-10-07 22:12:18,285][67871] Updated weights for policy 1, policy_version 60090 (0.0009) [2023-10-07 22:12:19,331][67838] Updated weights for policy 0, policy_version 60002 (0.0008) [2023-10-07 22:12:19,694][67838] Updated weights for policy 0, policy_version 60012 (0.0011) [2023-10-07 22:12:20,067][67838] Updated weights for policy 0, policy_version 60022 (0.0009) [2023-10-07 22:12:20,434][67838] Updated weights for policy 0, policy_version 60032 (0.0011) [2023-10-07 22:12:22,301][67871] Updated weights for policy 1, policy_version 60100 (0.0009) [2023-10-07 22:12:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 123011072. Throughput: 0: 1669.0, 1: 1674.8. Samples: 30766184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:12:22,478][66916] Avg episode reward: [(0, '46.150'), (1, '48.460')] [2023-10-07 22:12:22,678][67871] Updated weights for policy 1, policy_version 60110 (0.0007) [2023-10-07 22:12:23,040][67871] Updated weights for policy 1, policy_version 60120 (0.0007) [2023-10-07 22:12:24,458][67838] Updated weights for policy 0, policy_version 60042 (0.0009) [2023-10-07 22:12:24,822][67838] Updated weights for policy 0, policy_version 60052 (0.0008) [2023-10-07 22:12:25,207][67838] Updated weights for policy 0, policy_version 60062 (0.0008) [2023-10-07 22:12:27,182][67871] Updated weights for policy 1, policy_version 60130 (0.0007) [2023-10-07 22:12:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 123076608. Throughput: 0: 1646.6, 1: 1677.6. Samples: 30775742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:12:27,477][66916] Avg episode reward: [(0, '48.910'), (1, '47.640')] [2023-10-07 22:12:27,563][67871] Updated weights for policy 1, policy_version 60140 (0.0008) [2023-10-07 22:12:27,934][67871] Updated weights for policy 1, policy_version 60150 (0.0007) [2023-10-07 22:12:28,304][67871] Updated weights for policy 1, policy_version 60160 (0.0009) [2023-10-07 22:12:29,144][67838] Updated weights for policy 0, policy_version 60072 (0.0007) [2023-10-07 22:12:29,523][67838] Updated weights for policy 0, policy_version 60082 (0.0007) [2023-10-07 22:12:29,888][67838] Updated weights for policy 0, policy_version 60092 (0.0008) [2023-10-07 22:12:32,423][67871] Updated weights for policy 1, policy_version 60170 (0.0010) [2023-10-07 22:12:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 123142144. Throughput: 0: 1666.6, 1: 1679.9. Samples: 30796144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:12:32,477][66916] Avg episode reward: [(0, '52.300'), (1, '47.850')] [2023-10-07 22:12:32,794][67871] Updated weights for policy 1, policy_version 60180 (0.0009) [2023-10-07 22:12:33,166][67871] Updated weights for policy 1, policy_version 60190 (0.0010) [2023-10-07 22:12:34,054][67838] Updated weights for policy 0, policy_version 60102 (0.0009) [2023-10-07 22:12:34,422][67838] Updated weights for policy 0, policy_version 60112 (0.0010) [2023-10-07 22:12:34,797][67838] Updated weights for policy 0, policy_version 60122 (0.0008) [2023-10-07 22:12:37,304][67871] Updated weights for policy 1, policy_version 60200 (0.0009) [2023-10-07 22:12:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 123207680. Throughput: 0: 1671.5, 1: 1676.2. Samples: 30816650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:12:37,478][66916] Avg episode reward: [(0, '52.320'), (1, '51.120')] [2023-10-07 22:12:37,667][67871] Updated weights for policy 1, policy_version 60210 (0.0007) [2023-10-07 22:12:38,034][67871] Updated weights for policy 1, policy_version 60220 (0.0008) [2023-10-07 22:12:38,907][67838] Updated weights for policy 0, policy_version 60132 (0.0010) [2023-10-07 22:12:39,268][67838] Updated weights for policy 0, policy_version 60142 (0.0008) [2023-10-07 22:12:39,641][67838] Updated weights for policy 0, policy_version 60152 (0.0008) [2023-10-07 22:12:42,264][67871] Updated weights for policy 1, policy_version 60230 (0.0007) [2023-10-07 22:12:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 123273216. Throughput: 0: 1653.4, 1: 1671.5. Samples: 30825624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:12:42,477][66916] Avg episode reward: [(0, '51.310'), (1, '45.280')] [2023-10-07 22:12:42,627][67871] Updated weights for policy 1, policy_version 60240 (0.0007) [2023-10-07 22:12:42,998][67871] Updated weights for policy 1, policy_version 60250 (0.0008) [2023-10-07 22:12:43,685][67838] Updated weights for policy 0, policy_version 60162 (0.0008) [2023-10-07 22:12:44,061][67838] Updated weights for policy 0, policy_version 60172 (0.0011) [2023-10-07 22:12:44,436][67838] Updated weights for policy 0, policy_version 60182 (0.0012) [2023-10-07 22:12:44,811][67838] Updated weights for policy 0, policy_version 60192 (0.0008) [2023-10-07 22:12:46,904][67871] Updated weights for policy 1, policy_version 60260 (0.0007) [2023-10-07 22:12:47,284][67871] Updated weights for policy 1, policy_version 60270 (0.0008) [2023-10-07 22:12:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 123338752. Throughput: 0: 1675.6, 1: 1669.8. Samples: 30846010. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-07 22:12:47,477][66916] Avg episode reward: [(0, '48.610'), (1, '46.080')] [2023-10-07 22:12:47,647][67871] Updated weights for policy 1, policy_version 60280 (0.0007) [2023-10-07 22:12:49,182][67838] Updated weights for policy 0, policy_version 60202 (0.0009) [2023-10-07 22:12:49,545][67838] Updated weights for policy 0, policy_version 60212 (0.0008) [2023-10-07 22:12:49,920][67838] Updated weights for policy 0, policy_version 60222 (0.0007) [2023-10-07 22:12:51,586][67871] Updated weights for policy 1, policy_version 60290 (0.0008) [2023-10-07 22:12:51,945][67871] Updated weights for policy 1, policy_version 60300 (0.0010) [2023-10-07 22:12:52,320][67871] Updated weights for policy 1, policy_version 60310 (0.0010) [2023-10-07 22:12:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 123404288. Throughput: 0: 1668.2, 1: 1663.1. Samples: 30866198. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-07 22:12:52,477][66916] Avg episode reward: [(0, '48.940'), (1, '44.290')] [2023-10-07 22:12:52,685][67871] Updated weights for policy 1, policy_version 60320 (0.0009) [2023-10-07 22:12:54,073][67838] Updated weights for policy 0, policy_version 60232 (0.0011) [2023-10-07 22:12:54,442][67838] Updated weights for policy 0, policy_version 60242 (0.0010) [2023-10-07 22:12:54,823][67838] Updated weights for policy 0, policy_version 60252 (0.0010) [2023-10-07 22:12:56,732][67871] Updated weights for policy 1, policy_version 60330 (0.0007) [2023-10-07 22:12:57,099][67871] Updated weights for policy 1, policy_version 60340 (0.0008) [2023-10-07 22:12:57,472][67871] Updated weights for policy 1, policy_version 60350 (0.0007) [2023-10-07 22:12:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 123469824. Throughput: 0: 1648.6, 1: 1670.6. Samples: 30875452. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-07 22:12:57,478][66916] Avg episode reward: [(0, '49.680'), (1, '43.260')] [2023-10-07 22:12:58,953][67838] Updated weights for policy 0, policy_version 60262 (0.0009) [2023-10-07 22:12:59,327][67838] Updated weights for policy 0, policy_version 60272 (0.0008) [2023-10-07 22:12:59,689][67838] Updated weights for policy 0, policy_version 60282 (0.0008) [2023-10-07 22:13:01,519][67871] Updated weights for policy 1, policy_version 60360 (0.0009) [2023-10-07 22:13:01,883][67871] Updated weights for policy 1, policy_version 60370 (0.0012) [2023-10-07 22:13:02,247][67871] Updated weights for policy 1, policy_version 60380 (0.0010) [2023-10-07 22:13:02,477][66916] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123568128. Throughput: 0: 1662.0, 1: 1675.6. Samples: 30895976. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-07 22:13:02,478][66916] Avg episode reward: [(0, '47.650'), (1, '44.770')] [2023-10-07 22:13:03,727][67838] Updated weights for policy 0, policy_version 60292 (0.0009) [2023-10-07 22:13:04,106][67838] Updated weights for policy 0, policy_version 60302 (0.0010) [2023-10-07 22:13:04,482][67838] Updated weights for policy 0, policy_version 60312 (0.0007) [2023-10-07 22:13:06,667][67871] Updated weights for policy 1, policy_version 60390 (0.0008) [2023-10-07 22:13:07,034][67871] Updated weights for policy 1, policy_version 60400 (0.0007) [2023-10-07 22:13:07,408][67871] Updated weights for policy 1, policy_version 60410 (0.0007) [2023-10-07 22:13:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 123600896. Throughput: 0: 1668.2, 1: 1665.1. Samples: 30916180. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-07 22:13:07,477][66916] Avg episode reward: [(0, '49.910'), (1, '43.280')] [2023-10-07 22:13:07,491][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000060320_61767680.pth... [2023-10-07 22:13:07,522][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000058784_60194816.pth [2023-10-07 22:13:07,623][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000060416_61865984.pth... [2023-10-07 22:13:07,652][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000058848_60260352.pth [2023-10-07 22:13:08,766][67838] Updated weights for policy 0, policy_version 60322 (0.0007) [2023-10-07 22:13:09,136][67838] Updated weights for policy 0, policy_version 60332 (0.0008) [2023-10-07 22:13:09,511][67838] Updated weights for policy 0, policy_version 60342 (0.0008) [2023-10-07 22:13:09,883][67838] Updated weights for policy 0, policy_version 60352 (0.0008) [2023-10-07 22:13:11,406][67871] Updated weights for policy 1, policy_version 60420 (0.0008) [2023-10-07 22:13:11,772][67871] Updated weights for policy 1, policy_version 60430 (0.0009) [2023-10-07 22:13:12,145][67871] Updated weights for policy 1, policy_version 60440 (0.0007) [2023-10-07 22:13:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123699200. Throughput: 0: 1659.3, 1: 1669.5. Samples: 30925536. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-07 22:13:12,477][66916] Avg episode reward: [(0, '47.670'), (1, '43.190')] [2023-10-07 22:13:13,838][67838] Updated weights for policy 0, policy_version 60362 (0.0009) [2023-10-07 22:13:14,218][67838] Updated weights for policy 0, policy_version 60372 (0.0009) [2023-10-07 22:13:14,604][67838] Updated weights for policy 0, policy_version 60382 (0.0009) [2023-10-07 22:13:16,149][67871] Updated weights for policy 1, policy_version 60450 (0.0007) [2023-10-07 22:13:16,523][67871] Updated weights for policy 1, policy_version 60460 (0.0007) [2023-10-07 22:13:16,896][67871] Updated weights for policy 1, policy_version 60470 (0.0007) [2023-10-07 22:13:17,254][67871] Updated weights for policy 1, policy_version 60480 (0.0008) [2023-10-07 22:13:17,476][66916] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123764736. Throughput: 0: 1663.9, 1: 1675.7. Samples: 30946426. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-07 22:13:17,477][66916] Avg episode reward: [(0, '47.670'), (1, '45.910')] [2023-10-07 22:13:18,597][67838] Updated weights for policy 0, policy_version 60392 (0.0009) [2023-10-07 22:13:18,969][67838] Updated weights for policy 0, policy_version 60402 (0.0009) [2023-10-07 22:13:19,338][67838] Updated weights for policy 0, policy_version 60412 (0.0010) [2023-10-07 22:13:21,477][67871] Updated weights for policy 1, policy_version 60490 (0.0008) [2023-10-07 22:13:21,839][67871] Updated weights for policy 1, policy_version 60500 (0.0007) [2023-10-07 22:13:22,207][67871] Updated weights for policy 1, policy_version 60510 (0.0009) [2023-10-07 22:13:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123830272. Throughput: 0: 1665.6, 1: 1660.6. Samples: 30966330. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-07 22:13:22,478][66916] Avg episode reward: [(0, '48.820'), (1, '45.950')] [2023-10-07 22:13:23,388][67838] Updated weights for policy 0, policy_version 60422 (0.0007) [2023-10-07 22:13:23,757][67838] Updated weights for policy 0, policy_version 60432 (0.0007) [2023-10-07 22:13:24,128][67838] Updated weights for policy 0, policy_version 60442 (0.0009) [2023-10-07 22:13:26,473][67871] Updated weights for policy 1, policy_version 60520 (0.0008) [2023-10-07 22:13:26,842][67871] Updated weights for policy 1, policy_version 60530 (0.0008) [2023-10-07 22:13:27,214][67871] Updated weights for policy 1, policy_version 60540 (0.0009) [2023-10-07 22:13:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123895808. Throughput: 0: 1666.8, 1: 1680.8. Samples: 30976270. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-07 22:13:27,478][66916] Avg episode reward: [(0, '48.120'), (1, '44.120')] [2023-10-07 22:13:28,064][67838] Updated weights for policy 0, policy_version 60452 (0.0008) [2023-10-07 22:13:28,433][67838] Updated weights for policy 0, policy_version 60462 (0.0008) [2023-10-07 22:13:28,798][67838] Updated weights for policy 0, policy_version 60472 (0.0008) [2023-10-07 22:13:31,234][67871] Updated weights for policy 1, policy_version 60550 (0.0008) [2023-10-07 22:13:31,599][67871] Updated weights for policy 1, policy_version 60560 (0.0008) [2023-10-07 22:13:31,952][67871] Updated weights for policy 1, policy_version 60570 (0.0008) [2023-10-07 22:13:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123961344. Throughput: 0: 1668.5, 1: 1678.5. Samples: 30996624. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-07 22:13:32,477][66916] Avg episode reward: [(0, '47.110'), (1, '47.260')] [2023-10-07 22:13:32,994][67838] Updated weights for policy 0, policy_version 60482 (0.0008) [2023-10-07 22:13:33,375][67838] Updated weights for policy 0, policy_version 60492 (0.0008) [2023-10-07 22:13:33,744][67838] Updated weights for policy 0, policy_version 60502 (0.0008) [2023-10-07 22:13:34,118][67838] Updated weights for policy 0, policy_version 60512 (0.0008) [2023-10-07 22:13:35,945][67871] Updated weights for policy 1, policy_version 60580 (0.0009) [2023-10-07 22:13:36,314][67871] Updated weights for policy 1, policy_version 60590 (0.0009) [2023-10-07 22:13:36,675][67871] Updated weights for policy 1, policy_version 60600 (0.0009) [2023-10-07 22:13:37,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 124026880. Throughput: 0: 1673.3, 1: 1660.3. Samples: 31016208. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-07 22:13:37,477][66916] Avg episode reward: [(0, '49.200'), (1, '45.630')] [2023-10-07 22:13:38,313][67838] Updated weights for policy 0, policy_version 60522 (0.0009) [2023-10-07 22:13:38,686][67838] Updated weights for policy 0, policy_version 60532 (0.0008) [2023-10-07 22:13:39,054][67838] Updated weights for policy 0, policy_version 60542 (0.0008) [2023-10-07 22:13:40,956][67871] Updated weights for policy 1, policy_version 60610 (0.0008) [2023-10-07 22:13:41,318][67871] Updated weights for policy 1, policy_version 60620 (0.0011) [2023-10-07 22:13:41,692][67871] Updated weights for policy 1, policy_version 60630 (0.0010) [2023-10-07 22:13:42,056][67871] Updated weights for policy 1, policy_version 60640 (0.0008) [2023-10-07 22:13:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 124092416. Throughput: 0: 1673.5, 1: 1675.1. Samples: 31026140. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-07 22:13:42,478][66916] Avg episode reward: [(0, '47.750'), (1, '44.660')] [2023-10-07 22:13:43,209][67838] Updated weights for policy 0, policy_version 60552 (0.0009) [2023-10-07 22:13:43,582][67838] Updated weights for policy 0, policy_version 60562 (0.0008) [2023-10-07 22:13:43,957][67838] Updated weights for policy 0, policy_version 60572 (0.0008) [2023-10-07 22:13:46,253][67871] Updated weights for policy 1, policy_version 60650 (0.0007) [2023-10-07 22:13:46,614][67871] Updated weights for policy 1, policy_version 60660 (0.0008) [2023-10-07 22:13:46,977][67871] Updated weights for policy 1, policy_version 60670 (0.0008) [2023-10-07 22:13:47,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 124157952. Throughput: 0: 1675.4, 1: 1668.1. Samples: 31046434. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-07 22:13:47,478][66916] Avg episode reward: [(0, '51.760'), (1, '47.070')] [2023-10-07 22:13:48,121][67838] Updated weights for policy 0, policy_version 60582 (0.0008) [2023-10-07 22:13:48,500][67838] Updated weights for policy 0, policy_version 60592 (0.0009) [2023-10-07 22:13:48,871][67838] Updated weights for policy 0, policy_version 60602 (0.0007) [2023-10-07 22:13:51,155][67871] Updated weights for policy 1, policy_version 60680 (0.0007) [2023-10-07 22:13:51,525][67871] Updated weights for policy 1, policy_version 60690 (0.0007) [2023-10-07 22:13:51,893][67871] Updated weights for policy 1, policy_version 60700 (0.0008) [2023-10-07 22:13:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 124223488. Throughput: 0: 1677.1, 1: 1651.2. Samples: 31065952. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-07 22:13:52,477][66916] Avg episode reward: [(0, '48.940'), (1, '48.410')] [2023-10-07 22:13:52,880][67838] Updated weights for policy 0, policy_version 60612 (0.0008) [2023-10-07 22:13:53,249][67838] Updated weights for policy 0, policy_version 60622 (0.0007) [2023-10-07 22:13:53,634][67838] Updated weights for policy 0, policy_version 60632 (0.0007) [2023-10-07 22:13:55,998][67871] Updated weights for policy 1, policy_version 60710 (0.0009) [2023-10-07 22:13:56,379][67871] Updated weights for policy 1, policy_version 60720 (0.0007) [2023-10-07 22:13:56,740][67871] Updated weights for policy 1, policy_version 60730 (0.0009) [2023-10-07 22:13:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 124289024. Throughput: 0: 1676.4, 1: 1666.1. Samples: 31075950. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-07 22:13:57,477][66916] Avg episode reward: [(0, '50.220'), (1, '49.810')] [2023-10-07 22:13:57,643][67838] Updated weights for policy 0, policy_version 60642 (0.0007) [2023-10-07 22:13:58,020][67838] Updated weights for policy 0, policy_version 60652 (0.0007) [2023-10-07 22:13:58,390][67838] Updated weights for policy 0, policy_version 60662 (0.0008) [2023-10-07 22:13:58,771][67838] Updated weights for policy 0, policy_version 60672 (0.0010) [2023-10-07 22:14:00,828][67871] Updated weights for policy 1, policy_version 60740 (0.0008) [2023-10-07 22:14:01,202][67871] Updated weights for policy 1, policy_version 60750 (0.0009) [2023-10-07 22:14:01,574][67871] Updated weights for policy 1, policy_version 60760 (0.0007) [2023-10-07 22:14:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 124354560. Throughput: 0: 1680.4, 1: 1654.5. Samples: 31096496. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-07 22:14:02,477][66916] Avg episode reward: [(0, '47.940'), (1, '49.090')] [2023-10-07 22:14:02,730][67838] Updated weights for policy 0, policy_version 60682 (0.0007) [2023-10-07 22:14:03,113][67838] Updated weights for policy 0, policy_version 60692 (0.0007) [2023-10-07 22:14:03,486][67838] Updated weights for policy 0, policy_version 60702 (0.0009) [2023-10-07 22:14:05,909][67871] Updated weights for policy 1, policy_version 60770 (0.0007) [2023-10-07 22:14:06,272][67871] Updated weights for policy 1, policy_version 60780 (0.0009) [2023-10-07 22:14:06,649][67871] Updated weights for policy 1, policy_version 60790 (0.0009) [2023-10-07 22:14:07,001][67871] Updated weights for policy 1, policy_version 60800 (0.0007) [2023-10-07 22:14:07,379][67838] Updated weights for policy 0, policy_version 60712 (0.0009) [2023-10-07 22:14:07,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 124420096. Throughput: 0: 1682.7, 1: 1649.9. Samples: 31116298. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 22:14:07,477][66916] Avg episode reward: [(0, '48.690'), (1, '50.190')] [2023-10-07 22:14:07,743][67838] Updated weights for policy 0, policy_version 60722 (0.0007) [2023-10-07 22:14:08,119][67838] Updated weights for policy 0, policy_version 60732 (0.0009) [2023-10-07 22:14:10,911][67871] Updated weights for policy 1, policy_version 60810 (0.0007) [2023-10-07 22:14:11,271][67871] Updated weights for policy 1, policy_version 60820 (0.0008) [2023-10-07 22:14:11,647][67871] Updated weights for policy 1, policy_version 60830 (0.0008) [2023-10-07 22:14:12,331][67838] Updated weights for policy 0, policy_version 60742 (0.0009) [2023-10-07 22:14:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 124485632. Throughput: 0: 1682.0, 1: 1655.9. Samples: 31126476. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 22:14:12,477][66916] Avg episode reward: [(0, '48.700'), (1, '47.120')] [2023-10-07 22:14:12,706][67838] Updated weights for policy 0, policy_version 60752 (0.0007) [2023-10-07 22:14:13,075][67838] Updated weights for policy 0, policy_version 60762 (0.0009) [2023-10-07 22:14:15,855][67871] Updated weights for policy 1, policy_version 60840 (0.0010) [2023-10-07 22:14:16,218][67871] Updated weights for policy 1, policy_version 60850 (0.0008) [2023-10-07 22:14:16,580][67871] Updated weights for policy 1, policy_version 60860 (0.0009) [2023-10-07 22:14:17,193][67838] Updated weights for policy 0, policy_version 60772 (0.0009) [2023-10-07 22:14:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 124551168. Throughput: 0: 1679.3, 1: 1649.4. Samples: 31146418. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 22:14:17,478][66916] Avg episode reward: [(0, '52.360'), (1, '44.990')] [2023-10-07 22:14:17,558][67838] Updated weights for policy 0, policy_version 60782 (0.0010) [2023-10-07 22:14:17,930][67838] Updated weights for policy 0, policy_version 60792 (0.0010) [2023-10-07 22:14:20,673][67871] Updated weights for policy 1, policy_version 60870 (0.0008) [2023-10-07 22:14:21,037][67871] Updated weights for policy 1, policy_version 60880 (0.0008) [2023-10-07 22:14:21,411][67871] Updated weights for policy 1, policy_version 60890 (0.0009) [2023-10-07 22:14:22,227][67838] Updated weights for policy 0, policy_version 60802 (0.0010) [2023-10-07 22:14:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 124616704. Throughput: 0: 1673.9, 1: 1654.4. Samples: 31165986. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 22:14:22,478][66916] Avg episode reward: [(0, '48.960'), (1, '46.470')] [2023-10-07 22:14:22,602][67838] Updated weights for policy 0, policy_version 60812 (0.0010) [2023-10-07 22:14:22,982][67838] Updated weights for policy 0, policy_version 60822 (0.0007) [2023-10-07 22:14:23,352][67838] Updated weights for policy 0, policy_version 60832 (0.0007) [2023-10-07 22:14:25,526][67871] Updated weights for policy 1, policy_version 60900 (0.0009) [2023-10-07 22:14:25,909][67871] Updated weights for policy 1, policy_version 60910 (0.0009) [2023-10-07 22:14:26,275][67871] Updated weights for policy 1, policy_version 60920 (0.0008) [2023-10-07 22:14:27,370][67838] Updated weights for policy 0, policy_version 60842 (0.0008) [2023-10-07 22:14:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 124682240. Throughput: 0: 1675.8, 1: 1657.7. Samples: 31176148. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 22:14:27,477][66916] Avg episode reward: [(0, '49.860'), (1, '46.470')] [2023-10-07 22:14:27,733][67838] Updated weights for policy 0, policy_version 60852 (0.0007) [2023-10-07 22:14:28,105][67838] Updated weights for policy 0, policy_version 60862 (0.0007) [2023-10-07 22:14:30,259][67871] Updated weights for policy 1, policy_version 60930 (0.0010) [2023-10-07 22:14:30,621][67871] Updated weights for policy 1, policy_version 60940 (0.0009) [2023-10-07 22:14:30,993][67871] Updated weights for policy 1, policy_version 60950 (0.0009) [2023-10-07 22:14:31,364][67871] Updated weights for policy 1, policy_version 60960 (0.0007) [2023-10-07 22:14:32,187][67838] Updated weights for policy 0, policy_version 60872 (0.0009) [2023-10-07 22:14:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 124747776. Throughput: 0: 1676.9, 1: 1646.5. Samples: 31195988. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 22:14:32,477][66916] Avg episode reward: [(0, '46.560'), (1, '47.610')] [2023-10-07 22:14:32,570][67838] Updated weights for policy 0, policy_version 60882 (0.0010) [2023-10-07 22:14:32,949][67838] Updated weights for policy 0, policy_version 60892 (0.0007) [2023-10-07 22:14:35,507][67871] Updated weights for policy 1, policy_version 60970 (0.0008) [2023-10-07 22:14:35,871][67871] Updated weights for policy 1, policy_version 60980 (0.0009) [2023-10-07 22:14:36,245][67871] Updated weights for policy 1, policy_version 60990 (0.0008) [2023-10-07 22:14:37,081][67838] Updated weights for policy 0, policy_version 60902 (0.0008) [2023-10-07 22:14:37,459][67838] Updated weights for policy 0, policy_version 60912 (0.0007) [2023-10-07 22:14:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 124813312. Throughput: 0: 1667.2, 1: 1659.7. Samples: 31215662. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 22:14:37,478][66916] Avg episode reward: [(0, '47.380'), (1, '47.770')] [2023-10-07 22:14:37,828][67838] Updated weights for policy 0, policy_version 60922 (0.0007) [2023-10-07 22:14:40,284][67871] Updated weights for policy 1, policy_version 61000 (0.0007) [2023-10-07 22:14:40,655][67871] Updated weights for policy 1, policy_version 61010 (0.0009) [2023-10-07 22:14:41,034][67871] Updated weights for policy 1, policy_version 61020 (0.0010) [2023-10-07 22:14:41,966][67838] Updated weights for policy 0, policy_version 60932 (0.0008) [2023-10-07 22:14:42,344][67838] Updated weights for policy 0, policy_version 60942 (0.0008) [2023-10-07 22:14:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 124878848. Throughput: 0: 1670.7, 1: 1666.3. Samples: 31226116. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-07 22:14:42,477][66916] Avg episode reward: [(0, '47.650'), (1, '46.580')] [2023-10-07 22:14:42,723][67838] Updated weights for policy 0, policy_version 60952 (0.0010) [2023-10-07 22:14:45,168][67871] Updated weights for policy 1, policy_version 61030 (0.0008) [2023-10-07 22:14:45,540][67871] Updated weights for policy 1, policy_version 61040 (0.0008) [2023-10-07 22:14:45,901][67871] Updated weights for policy 1, policy_version 61050 (0.0007) [2023-10-07 22:14:46,798][67838] Updated weights for policy 0, policy_version 60962 (0.0010) [2023-10-07 22:14:47,167][67838] Updated weights for policy 0, policy_version 60972 (0.0009) [2023-10-07 22:14:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 124944384. Throughput: 0: 1666.7, 1: 1654.0. Samples: 31245930. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:14:47,477][66916] Avg episode reward: [(0, '49.470'), (1, '44.360')] [2023-10-07 22:14:47,535][67838] Updated weights for policy 0, policy_version 60982 (0.0010) [2023-10-07 22:14:47,902][67838] Updated weights for policy 0, policy_version 60992 (0.0009) [2023-10-07 22:14:50,111][67871] Updated weights for policy 1, policy_version 61060 (0.0009) [2023-10-07 22:14:50,469][67871] Updated weights for policy 1, policy_version 61070 (0.0008) [2023-10-07 22:14:50,836][67871] Updated weights for policy 1, policy_version 61080 (0.0009) [2023-10-07 22:14:51,904][67838] Updated weights for policy 0, policy_version 61002 (0.0010) [2023-10-07 22:14:52,274][67838] Updated weights for policy 0, policy_version 61012 (0.0010) [2023-10-07 22:14:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 125009920. Throughput: 0: 1651.4, 1: 1663.6. Samples: 31265474. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:14:52,477][66916] Avg episode reward: [(0, '49.410'), (1, '45.410')] [2023-10-07 22:14:52,641][67838] Updated weights for policy 0, policy_version 61022 (0.0009) [2023-10-07 22:14:54,860][67871] Updated weights for policy 1, policy_version 61090 (0.0010) [2023-10-07 22:14:55,221][67871] Updated weights for policy 1, policy_version 61100 (0.0007) [2023-10-07 22:14:55,581][67871] Updated weights for policy 1, policy_version 61110 (0.0010) [2023-10-07 22:14:55,943][67871] Updated weights for policy 1, policy_version 61120 (0.0009) [2023-10-07 22:14:56,661][67838] Updated weights for policy 0, policy_version 61032 (0.0009) [2023-10-07 22:14:57,028][67838] Updated weights for policy 0, policy_version 61042 (0.0009) [2023-10-07 22:14:57,411][67838] Updated weights for policy 0, policy_version 61052 (0.0011) [2023-10-07 22:14:57,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 125075456. Throughput: 0: 1663.5, 1: 1669.8. Samples: 31276478. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:14:57,477][66916] Avg episode reward: [(0, '54.710'), (1, '45.040')] [2023-10-07 22:15:00,027][67871] Updated weights for policy 1, policy_version 61130 (0.0007) [2023-10-07 22:15:00,389][67871] Updated weights for policy 1, policy_version 61140 (0.0007) [2023-10-07 22:15:00,747][67871] Updated weights for policy 1, policy_version 61150 (0.0008) [2023-10-07 22:15:01,534][67838] Updated weights for policy 0, policy_version 61062 (0.0009) [2023-10-07 22:15:01,903][67838] Updated weights for policy 0, policy_version 61072 (0.0010) [2023-10-07 22:15:02,275][67838] Updated weights for policy 0, policy_version 61082 (0.0010) [2023-10-07 22:15:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 125140992. Throughput: 0: 1663.0, 1: 1658.6. Samples: 31295890. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:15:02,478][66916] Avg episode reward: [(0, '57.940'), (1, '45.500')] [2023-10-07 22:15:04,571][67871] Updated weights for policy 1, policy_version 61160 (0.0010) [2023-10-07 22:15:04,942][67871] Updated weights for policy 1, policy_version 61170 (0.0008) [2023-10-07 22:15:05,315][67871] Updated weights for policy 1, policy_version 61180 (0.0009) [2023-10-07 22:15:06,350][67838] Updated weights for policy 0, policy_version 61092 (0.0011) [2023-10-07 22:15:06,726][67838] Updated weights for policy 0, policy_version 61102 (0.0008) [2023-10-07 22:15:07,106][67838] Updated weights for policy 0, policy_version 61112 (0.0007) [2023-10-07 22:15:07,476][66916] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 125239296. Throughput: 0: 1649.5, 1: 1675.0. Samples: 31315586. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:15:07,477][66916] Avg episode reward: [(0, '59.290'), (1, '47.710')] [2023-10-07 22:15:07,485][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000061120_62586880.pth... [2023-10-07 22:15:07,485][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000061184_62652416.pth... [2023-10-07 22:15:07,514][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000059552_60981248.pth [2023-10-07 22:15:07,533][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000059616_61046784.pth [2023-10-07 22:15:09,357][67871] Updated weights for policy 1, policy_version 61190 (0.0008) [2023-10-07 22:15:09,723][67871] Updated weights for policy 1, policy_version 61200 (0.0007) [2023-10-07 22:15:10,097][67871] Updated weights for policy 1, policy_version 61210 (0.0008) [2023-10-07 22:15:11,202][67838] Updated weights for policy 0, policy_version 61122 (0.0009) [2023-10-07 22:15:11,568][67838] Updated weights for policy 0, policy_version 61132 (0.0009) [2023-10-07 22:15:11,936][67838] Updated weights for policy 0, policy_version 61142 (0.0008) [2023-10-07 22:15:12,313][67838] Updated weights for policy 0, policy_version 61152 (0.0007) [2023-10-07 22:15:12,476][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 125304832. Throughput: 0: 1668.5, 1: 1661.6. Samples: 31326006. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:15:12,477][66916] Avg episode reward: [(0, '56.080'), (1, '46.060')] [2023-10-07 22:15:14,073][67871] Updated weights for policy 1, policy_version 61220 (0.0008) [2023-10-07 22:15:14,443][67871] Updated weights for policy 1, policy_version 61230 (0.0010) [2023-10-07 22:15:14,799][67871] Updated weights for policy 1, policy_version 61240 (0.0011) [2023-10-07 22:15:16,639][67838] Updated weights for policy 0, policy_version 61162 (0.0007) [2023-10-07 22:15:17,004][67838] Updated weights for policy 0, policy_version 61172 (0.0009) [2023-10-07 22:15:17,370][67838] Updated weights for policy 0, policy_version 61182 (0.0009) [2023-10-07 22:15:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 125370368. Throughput: 0: 1664.2, 1: 1664.1. Samples: 31345764. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:15:17,477][66916] Avg episode reward: [(0, '58.170'), (1, '48.090')] [2023-10-07 22:15:19,152][67871] Updated weights for policy 1, policy_version 61250 (0.0008) [2023-10-07 22:15:19,514][67871] Updated weights for policy 1, policy_version 61260 (0.0007) [2023-10-07 22:15:19,883][67871] Updated weights for policy 1, policy_version 61270 (0.0008) [2023-10-07 22:15:20,250][67871] Updated weights for policy 1, policy_version 61280 (0.0010) [2023-10-07 22:15:21,463][67838] Updated weights for policy 0, policy_version 61192 (0.0011) [2023-10-07 22:15:21,829][67838] Updated weights for policy 0, policy_version 61202 (0.0011) [2023-10-07 22:15:22,194][67838] Updated weights for policy 0, policy_version 61212 (0.0010) [2023-10-07 22:15:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 125435904. Throughput: 0: 1649.8, 1: 1676.0. Samples: 31365324. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-07 22:15:22,478][66916] Avg episode reward: [(0, '51.600'), (1, '51.220')] [2023-10-07 22:15:24,278][67871] Updated weights for policy 1, policy_version 61290 (0.0007) [2023-10-07 22:15:24,647][67871] Updated weights for policy 1, policy_version 61300 (0.0007) [2023-10-07 22:15:25,005][67871] Updated weights for policy 1, policy_version 61310 (0.0009) [2023-10-07 22:15:26,340][67838] Updated weights for policy 0, policy_version 61222 (0.0007) [2023-10-07 22:15:26,719][67838] Updated weights for policy 0, policy_version 61232 (0.0007) [2023-10-07 22:15:27,084][67838] Updated weights for policy 0, policy_version 61242 (0.0008) [2023-10-07 22:15:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 125501440. Throughput: 0: 1665.1, 1: 1656.9. Samples: 31375608. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-07 22:15:27,477][66916] Avg episode reward: [(0, '52.500'), (1, '50.360')] [2023-10-07 22:15:29,034][67871] Updated weights for policy 1, policy_version 61320 (0.0007) [2023-10-07 22:15:29,400][67871] Updated weights for policy 1, policy_version 61330 (0.0009) [2023-10-07 22:15:29,771][67871] Updated weights for policy 1, policy_version 61340 (0.0010) [2023-10-07 22:15:31,425][67838] Updated weights for policy 0, policy_version 61252 (0.0007) [2023-10-07 22:15:31,795][67838] Updated weights for policy 0, policy_version 61262 (0.0008) [2023-10-07 22:15:32,167][67838] Updated weights for policy 0, policy_version 61272 (0.0011) [2023-10-07 22:15:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 125566976. Throughput: 0: 1662.9, 1: 1669.6. Samples: 31395894. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-07 22:15:32,477][66916] Avg episode reward: [(0, '51.620'), (1, '49.780')] [2023-10-07 22:15:34,064][67871] Updated weights for policy 1, policy_version 61350 (0.0011) [2023-10-07 22:15:34,440][67871] Updated weights for policy 1, policy_version 61360 (0.0008) [2023-10-07 22:15:34,815][67871] Updated weights for policy 1, policy_version 61370 (0.0011) [2023-10-07 22:15:36,375][67838] Updated weights for policy 0, policy_version 61282 (0.0010) [2023-10-07 22:15:36,747][67838] Updated weights for policy 0, policy_version 61292 (0.0011) [2023-10-07 22:15:37,112][67838] Updated weights for policy 0, policy_version 61302 (0.0010) [2023-10-07 22:15:37,477][66916] Fps is (10 sec: 9830.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 125599744. Throughput: 0: 1651.6, 1: 1685.5. Samples: 31415644. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-07 22:15:37,478][66916] Avg episode reward: [(0, '56.530'), (1, '53.430')] [2023-10-07 22:15:37,483][67838] Updated weights for policy 0, policy_version 61312 (0.0010) [2023-10-07 22:15:38,935][67871] Updated weights for policy 1, policy_version 61380 (0.0009) [2023-10-07 22:15:39,310][67871] Updated weights for policy 1, policy_version 61390 (0.0009) [2023-10-07 22:15:39,671][67871] Updated weights for policy 1, policy_version 61400 (0.0007) [2023-10-07 22:15:41,675][67838] Updated weights for policy 0, policy_version 61322 (0.0010) [2023-10-07 22:15:42,038][67838] Updated weights for policy 0, policy_version 61332 (0.0009) [2023-10-07 22:15:42,415][67838] Updated weights for policy 0, policy_version 61342 (0.0007) [2023-10-07 22:15:42,477][66916] Fps is (10 sec: 9830.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 125665280. Throughput: 0: 1652.4, 1: 1660.3. Samples: 31425550. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-07 22:15:42,478][66916] Avg episode reward: [(0, '53.860'), (1, '52.060')] [2023-10-07 22:15:43,635][67871] Updated weights for policy 1, policy_version 61410 (0.0008) [2023-10-07 22:15:44,004][67871] Updated weights for policy 1, policy_version 61420 (0.0008) [2023-10-07 22:15:44,360][67871] Updated weights for policy 1, policy_version 61430 (0.0007) [2023-10-07 22:15:44,730][67871] Updated weights for policy 1, policy_version 61440 (0.0008) [2023-10-07 22:15:46,539][67838] Updated weights for policy 0, policy_version 61352 (0.0009) [2023-10-07 22:15:46,913][67838] Updated weights for policy 0, policy_version 61362 (0.0011) [2023-10-07 22:15:47,283][67838] Updated weights for policy 0, policy_version 61372 (0.0011) [2023-10-07 22:15:47,476][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 125763584. Throughput: 0: 1652.8, 1: 1675.2. Samples: 31445652. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-07 22:15:47,477][66916] Avg episode reward: [(0, '56.220'), (1, '53.350')] [2023-10-07 22:15:48,897][67871] Updated weights for policy 1, policy_version 61450 (0.0009) [2023-10-07 22:15:49,260][67871] Updated weights for policy 1, policy_version 61460 (0.0010) [2023-10-07 22:15:49,638][67871] Updated weights for policy 1, policy_version 61470 (0.0008) [2023-10-07 22:15:51,289][67838] Updated weights for policy 0, policy_version 61382 (0.0010) [2023-10-07 22:15:51,667][67838] Updated weights for policy 0, policy_version 61392 (0.0009) [2023-10-07 22:15:52,044][67838] Updated weights for policy 0, policy_version 61402 (0.0008) [2023-10-07 22:15:52,477][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 125829120. Throughput: 0: 1649.8, 1: 1673.2. Samples: 31465122. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-07 22:15:52,478][66916] Avg episode reward: [(0, '55.490'), (1, '55.250')] [2023-10-07 22:15:54,000][67871] Updated weights for policy 1, policy_version 61480 (0.0010) [2023-10-07 22:15:54,374][67871] Updated weights for policy 1, policy_version 61490 (0.0010) [2023-10-07 22:15:54,744][67871] Updated weights for policy 1, policy_version 61500 (0.0011) [2023-10-07 22:15:56,180][67838] Updated weights for policy 0, policy_version 61412 (0.0009) [2023-10-07 22:15:56,548][67838] Updated weights for policy 0, policy_version 61422 (0.0008) [2023-10-07 22:15:56,918][67838] Updated weights for policy 0, policy_version 61432 (0.0008) [2023-10-07 22:15:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 125894656. Throughput: 0: 1653.5, 1: 1657.6. Samples: 31475006. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-07 22:15:57,478][66916] Avg episode reward: [(0, '51.730'), (1, '53.170')] [2023-10-07 22:15:58,988][67871] Updated weights for policy 1, policy_version 61510 (0.0010) [2023-10-07 22:15:59,349][67871] Updated weights for policy 1, policy_version 61520 (0.0007) [2023-10-07 22:15:59,709][67871] Updated weights for policy 1, policy_version 61530 (0.0009) [2023-10-07 22:16:01,027][67838] Updated weights for policy 0, policy_version 61442 (0.0010) [2023-10-07 22:16:01,434][67838] Updated weights for policy 0, policy_version 61452 (0.0010) [2023-10-07 22:16:01,810][67838] Updated weights for policy 0, policy_version 61462 (0.0009) [2023-10-07 22:16:02,180][67838] Updated weights for policy 0, policy_version 61472 (0.0010) [2023-10-07 22:16:02,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 125960192. Throughput: 0: 1655.9, 1: 1661.1. Samples: 31495028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:16:02,478][66916] Avg episode reward: [(0, '50.870'), (1, '52.290')] [2023-10-07 22:16:03,871][67871] Updated weights for policy 1, policy_version 61540 (0.0008) [2023-10-07 22:16:04,250][67871] Updated weights for policy 1, policy_version 61550 (0.0007) [2023-10-07 22:16:04,626][67871] Updated weights for policy 1, policy_version 61560 (0.0008) [2023-10-07 22:16:06,318][67838] Updated weights for policy 0, policy_version 61482 (0.0010) [2023-10-07 22:16:06,691][67838] Updated weights for policy 0, policy_version 61492 (0.0008) [2023-10-07 22:16:07,058][67838] Updated weights for policy 0, policy_version 61502 (0.0008) [2023-10-07 22:16:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126025728. Throughput: 0: 1651.6, 1: 1663.3. Samples: 31514492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:16:07,477][66916] Avg episode reward: [(0, '49.120'), (1, '47.600')] [2023-10-07 22:16:08,681][67871] Updated weights for policy 1, policy_version 61570 (0.0008) [2023-10-07 22:16:09,051][67871] Updated weights for policy 1, policy_version 61580 (0.0009) [2023-10-07 22:16:09,414][67871] Updated weights for policy 1, policy_version 61590 (0.0008) [2023-10-07 22:16:09,788][67871] Updated weights for policy 1, policy_version 61600 (0.0009) [2023-10-07 22:16:11,045][67838] Updated weights for policy 0, policy_version 61512 (0.0010) [2023-10-07 22:16:11,409][67838] Updated weights for policy 0, policy_version 61522 (0.0009) [2023-10-07 22:16:11,778][67838] Updated weights for policy 0, policy_version 61532 (0.0008) [2023-10-07 22:16:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126091264. Throughput: 0: 1657.6, 1: 1654.7. Samples: 31524664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:16:12,477][66916] Avg episode reward: [(0, '47.060'), (1, '45.660')] [2023-10-07 22:16:13,982][67871] Updated weights for policy 1, policy_version 61610 (0.0008) [2023-10-07 22:16:14,343][67871] Updated weights for policy 1, policy_version 61620 (0.0009) [2023-10-07 22:16:14,715][67871] Updated weights for policy 1, policy_version 61630 (0.0009) [2023-10-07 22:16:15,855][67838] Updated weights for policy 0, policy_version 61542 (0.0009) [2023-10-07 22:16:16,230][67838] Updated weights for policy 0, policy_version 61552 (0.0009) [2023-10-07 22:16:16,605][67838] Updated weights for policy 0, policy_version 61562 (0.0008) [2023-10-07 22:16:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126156800. Throughput: 0: 1648.0, 1: 1658.4. Samples: 31544680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:16:17,477][66916] Avg episode reward: [(0, '47.380'), (1, '45.550')] [2023-10-07 22:16:18,809][67871] Updated weights for policy 1, policy_version 61640 (0.0010) [2023-10-07 22:16:19,173][67871] Updated weights for policy 1, policy_version 61650 (0.0011) [2023-10-07 22:16:19,540][67871] Updated weights for policy 1, policy_version 61660 (0.0007) [2023-10-07 22:16:20,676][67838] Updated weights for policy 0, policy_version 61572 (0.0008) [2023-10-07 22:16:21,042][67838] Updated weights for policy 0, policy_version 61582 (0.0011) [2023-10-07 22:16:21,412][67838] Updated weights for policy 0, policy_version 61592 (0.0009) [2023-10-07 22:16:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126222336. Throughput: 0: 1648.7, 1: 1655.3. Samples: 31564324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:16:22,477][66916] Avg episode reward: [(0, '53.090'), (1, '45.120')] [2023-10-07 22:16:23,711][67871] Updated weights for policy 1, policy_version 61670 (0.0007) [2023-10-07 22:16:24,079][67871] Updated weights for policy 1, policy_version 61680 (0.0009) [2023-10-07 22:16:24,448][67871] Updated weights for policy 1, policy_version 61690 (0.0007) [2023-10-07 22:16:25,627][67838] Updated weights for policy 0, policy_version 61602 (0.0009) [2023-10-07 22:16:26,004][67838] Updated weights for policy 0, policy_version 61612 (0.0010) [2023-10-07 22:16:26,384][67838] Updated weights for policy 0, policy_version 61622 (0.0009) [2023-10-07 22:16:26,756][67838] Updated weights for policy 0, policy_version 61632 (0.0011) [2023-10-07 22:16:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126287872. Throughput: 0: 1658.9, 1: 1647.2. Samples: 31574322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:16:27,477][66916] Avg episode reward: [(0, '54.510'), (1, '46.920')] [2023-10-07 22:16:28,513][67871] Updated weights for policy 1, policy_version 61700 (0.0007) [2023-10-07 22:16:28,880][67871] Updated weights for policy 1, policy_version 61710 (0.0007) [2023-10-07 22:16:29,252][67871] Updated weights for policy 1, policy_version 61720 (0.0007) [2023-10-07 22:16:31,139][67838] Updated weights for policy 0, policy_version 61642 (0.0007) [2023-10-07 22:16:31,505][67838] Updated weights for policy 0, policy_version 61652 (0.0007) [2023-10-07 22:16:31,877][67838] Updated weights for policy 0, policy_version 61662 (0.0009) [2023-10-07 22:16:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126353408. Throughput: 0: 1647.4, 1: 1655.9. Samples: 31594302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:16:32,478][66916] Avg episode reward: [(0, '58.320'), (1, '48.040')] [2023-10-07 22:16:33,444][67871] Updated weights for policy 1, policy_version 61730 (0.0008) [2023-10-07 22:16:33,801][67871] Updated weights for policy 1, policy_version 61740 (0.0008) [2023-10-07 22:16:34,169][67871] Updated weights for policy 1, policy_version 61750 (0.0010) [2023-10-07 22:16:34,537][67871] Updated weights for policy 1, policy_version 61760 (0.0007) [2023-10-07 22:16:35,935][67838] Updated weights for policy 0, policy_version 61672 (0.0010) [2023-10-07 22:16:36,305][67838] Updated weights for policy 0, policy_version 61682 (0.0008) [2023-10-07 22:16:36,674][67838] Updated weights for policy 0, policy_version 61692 (0.0009) [2023-10-07 22:16:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 126418944. Throughput: 0: 1645.6, 1: 1659.2. Samples: 31613834. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:16:37,477][66916] Avg episode reward: [(0, '55.030'), (1, '48.010')] [2023-10-07 22:16:38,769][67871] Updated weights for policy 1, policy_version 61770 (0.0009) [2023-10-07 22:16:39,132][67871] Updated weights for policy 1, policy_version 61780 (0.0008) [2023-10-07 22:16:39,497][67871] Updated weights for policy 1, policy_version 61790 (0.0007) [2023-10-07 22:16:40,642][67838] Updated weights for policy 0, policy_version 61702 (0.0009) [2023-10-07 22:16:41,010][67838] Updated weights for policy 0, policy_version 61712 (0.0009) [2023-10-07 22:16:41,383][67838] Updated weights for policy 0, policy_version 61722 (0.0012) [2023-10-07 22:16:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 126484480. Throughput: 0: 1654.3, 1: 1658.1. Samples: 31624064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:16:42,477][66916] Avg episode reward: [(0, '55.830'), (1, '46.300')] [2023-10-07 22:16:43,571][67871] Updated weights for policy 1, policy_version 61800 (0.0009) [2023-10-07 22:16:43,933][67871] Updated weights for policy 1, policy_version 61810 (0.0007) [2023-10-07 22:16:44,304][67871] Updated weights for policy 1, policy_version 61820 (0.0007) [2023-10-07 22:16:45,818][67838] Updated weights for policy 0, policy_version 61732 (0.0009) [2023-10-07 22:16:46,209][67838] Updated weights for policy 0, policy_version 61742 (0.0008) [2023-10-07 22:16:46,582][67838] Updated weights for policy 0, policy_version 61752 (0.0007) [2023-10-07 22:16:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126550016. Throughput: 0: 1642.5, 1: 1664.3. Samples: 31643836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:16:47,478][66916] Avg episode reward: [(0, '54.960'), (1, '48.280')] [2023-10-07 22:16:48,512][67871] Updated weights for policy 1, policy_version 61830 (0.0010) [2023-10-07 22:16:48,884][67871] Updated weights for policy 1, policy_version 61840 (0.0007) [2023-10-07 22:16:49,246][67871] Updated weights for policy 1, policy_version 61850 (0.0010) [2023-10-07 22:16:50,527][67838] Updated weights for policy 0, policy_version 61762 (0.0007) [2023-10-07 22:16:50,894][67838] Updated weights for policy 0, policy_version 61772 (0.0011) [2023-10-07 22:16:51,262][67838] Updated weights for policy 0, policy_version 61782 (0.0011) [2023-10-07 22:16:51,636][67838] Updated weights for policy 0, policy_version 61792 (0.0009) [2023-10-07 22:16:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126615552. Throughput: 0: 1646.5, 1: 1667.2. Samples: 31663606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:16:52,477][66916] Avg episode reward: [(0, '49.230'), (1, '47.150')] [2023-10-07 22:16:53,224][67871] Updated weights for policy 1, policy_version 61860 (0.0008) [2023-10-07 22:16:53,587][67871] Updated weights for policy 1, policy_version 61870 (0.0007) [2023-10-07 22:16:53,950][67871] Updated weights for policy 1, policy_version 61880 (0.0009) [2023-10-07 22:16:55,593][67838] Updated weights for policy 0, policy_version 61802 (0.0011) [2023-10-07 22:16:55,962][67838] Updated weights for policy 0, policy_version 61812 (0.0010) [2023-10-07 22:16:56,329][67838] Updated weights for policy 0, policy_version 61822 (0.0009) [2023-10-07 22:16:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126681088. Throughput: 0: 1652.2, 1: 1666.0. Samples: 31673982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:16:57,478][66916] Avg episode reward: [(0, '52.890'), (1, '46.420')] [2023-10-07 22:16:57,819][67871] Updated weights for policy 1, policy_version 61890 (0.0009) [2023-10-07 22:16:58,186][67871] Updated weights for policy 1, policy_version 61900 (0.0008) [2023-10-07 22:16:58,550][67871] Updated weights for policy 1, policy_version 61910 (0.0009) [2023-10-07 22:16:58,917][67871] Updated weights for policy 1, policy_version 61920 (0.0011) [2023-10-07 22:17:00,527][67838] Updated weights for policy 0, policy_version 61832 (0.0011) [2023-10-07 22:17:00,897][67838] Updated weights for policy 0, policy_version 61842 (0.0008) [2023-10-07 22:17:01,272][67838] Updated weights for policy 0, policy_version 61852 (0.0008) [2023-10-07 22:17:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126746624. Throughput: 0: 1645.1, 1: 1673.6. Samples: 31694024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:17:02,477][66916] Avg episode reward: [(0, '56.230'), (1, '50.400')] [2023-10-07 22:17:02,971][67871] Updated weights for policy 1, policy_version 61930 (0.0009) [2023-10-07 22:17:03,343][67871] Updated weights for policy 1, policy_version 61940 (0.0008) [2023-10-07 22:17:03,710][67871] Updated weights for policy 1, policy_version 61950 (0.0007) [2023-10-07 22:17:05,451][67838] Updated weights for policy 0, policy_version 61862 (0.0008) [2023-10-07 22:17:05,811][67838] Updated weights for policy 0, policy_version 61872 (0.0008) [2023-10-07 22:17:06,191][67838] Updated weights for policy 0, policy_version 61882 (0.0009) [2023-10-07 22:17:07,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 126812160. Throughput: 0: 1652.8, 1: 1671.1. Samples: 31713900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:17:07,477][66916] Avg episode reward: [(0, '57.050'), (1, '43.300')] [2023-10-07 22:17:07,488][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000061952_63438848.pth... [2023-10-07 22:17:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000061888_63373312.pth... [2023-10-07 22:17:07,526][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000060320_61767680.pth [2023-10-07 22:17:07,527][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000060416_61865984.pth [2023-10-07 22:17:07,530][67511] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p0/milestones/checkpoint_000061888_63373312.pth [2023-10-07 22:17:07,533][67676] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p1/milestones/checkpoint_000061952_63438848.pth [2023-10-07 22:17:07,966][67871] Updated weights for policy 1, policy_version 61960 (0.0009) [2023-10-07 22:17:08,330][67871] Updated weights for policy 1, policy_version 61970 (0.0008) [2023-10-07 22:17:08,701][67871] Updated weights for policy 1, policy_version 61980 (0.0009) [2023-10-07 22:17:10,329][67838] Updated weights for policy 0, policy_version 61892 (0.0009) [2023-10-07 22:17:10,705][67838] Updated weights for policy 0, policy_version 61902 (0.0011) [2023-10-07 22:17:11,082][67838] Updated weights for policy 0, policy_version 61912 (0.0010) [2023-10-07 22:17:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126877696. Throughput: 0: 1661.0, 1: 1668.1. Samples: 31724132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:17:12,477][66916] Avg episode reward: [(0, '56.610'), (1, '44.540')] [2023-10-07 22:17:12,791][67871] Updated weights for policy 1, policy_version 61990 (0.0009) [2023-10-07 22:17:13,169][67871] Updated weights for policy 1, policy_version 62000 (0.0008) [2023-10-07 22:17:13,533][67871] Updated weights for policy 1, policy_version 62010 (0.0007) [2023-10-07 22:17:15,115][67838] Updated weights for policy 0, policy_version 61922 (0.0007) [2023-10-07 22:17:15,486][67838] Updated weights for policy 0, policy_version 61932 (0.0009) [2023-10-07 22:17:15,864][67838] Updated weights for policy 0, policy_version 61942 (0.0009) [2023-10-07 22:17:16,234][67838] Updated weights for policy 0, policy_version 61952 (0.0007) [2023-10-07 22:17:17,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126943232. Throughput: 0: 1653.9, 1: 1664.1. Samples: 31743610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:17:17,478][66916] Avg episode reward: [(0, '61.250'), (1, '45.570')] [2023-10-07 22:17:17,533][67871] Updated weights for policy 1, policy_version 62020 (0.0008) [2023-10-07 22:17:17,895][67871] Updated weights for policy 1, policy_version 62030 (0.0008) [2023-10-07 22:17:18,262][67871] Updated weights for policy 1, policy_version 62040 (0.0009) [2023-10-07 22:17:20,310][67838] Updated weights for policy 0, policy_version 61962 (0.0009) [2023-10-07 22:17:20,678][67838] Updated weights for policy 0, policy_version 61972 (0.0008) [2023-10-07 22:17:21,044][67838] Updated weights for policy 0, policy_version 61982 (0.0007) [2023-10-07 22:17:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 127008768. Throughput: 0: 1677.8, 1: 1660.2. Samples: 31764046. Policy #0 lag: (min: 7.0, avg: 7.8, max: 26.0) [2023-10-07 22:17:22,477][66916] Avg episode reward: [(0, '54.910'), (1, '45.490')] [2023-10-07 22:17:22,617][67871] Updated weights for policy 1, policy_version 62050 (0.0008) [2023-10-07 22:17:22,985][67871] Updated weights for policy 1, policy_version 62060 (0.0007) [2023-10-07 22:17:23,351][67871] Updated weights for policy 1, policy_version 62070 (0.0007) [2023-10-07 22:17:23,711][67871] Updated weights for policy 1, policy_version 62080 (0.0008) [2023-10-07 22:17:25,154][67838] Updated weights for policy 0, policy_version 61992 (0.0009) [2023-10-07 22:17:25,531][67838] Updated weights for policy 0, policy_version 62002 (0.0008) [2023-10-07 22:17:25,895][67838] Updated weights for policy 0, policy_version 62012 (0.0008) [2023-10-07 22:17:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 127074304. Throughput: 0: 1673.8, 1: 1662.3. Samples: 31774186. Policy #0 lag: (min: 7.0, avg: 7.8, max: 26.0) [2023-10-07 22:17:27,477][66916] Avg episode reward: [(0, '56.710'), (1, '45.090')] [2023-10-07 22:17:27,906][67871] Updated weights for policy 1, policy_version 62090 (0.0009) [2023-10-07 22:17:28,289][67871] Updated weights for policy 1, policy_version 62100 (0.0009) [2023-10-07 22:17:28,651][67871] Updated weights for policy 1, policy_version 62110 (0.0007) [2023-10-07 22:17:29,887][67838] Updated weights for policy 0, policy_version 62022 (0.0009) [2023-10-07 22:17:30,259][67838] Updated weights for policy 0, policy_version 62032 (0.0007) [2023-10-07 22:17:30,623][67838] Updated weights for policy 0, policy_version 62042 (0.0007) [2023-10-07 22:17:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 127139840. Throughput: 0: 1663.0, 1: 1665.3. Samples: 31793610. Policy #0 lag: (min: 7.0, avg: 7.8, max: 26.0) [2023-10-07 22:17:32,478][66916] Avg episode reward: [(0, '54.680'), (1, '44.850')] [2023-10-07 22:17:32,720][67871] Updated weights for policy 1, policy_version 62120 (0.0008) [2023-10-07 22:17:33,084][67871] Updated weights for policy 1, policy_version 62130 (0.0008) [2023-10-07 22:17:33,451][67871] Updated weights for policy 1, policy_version 62140 (0.0007) [2023-10-07 22:17:34,876][67838] Updated weights for policy 0, policy_version 62052 (0.0010) [2023-10-07 22:17:35,264][67838] Updated weights for policy 0, policy_version 62062 (0.0008) [2023-10-07 22:17:35,635][67838] Updated weights for policy 0, policy_version 62072 (0.0009) [2023-10-07 22:17:37,467][67871] Updated weights for policy 1, policy_version 62150 (0.0009) [2023-10-07 22:17:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 127205376. Throughput: 0: 1679.5, 1: 1660.5. Samples: 31813904. Policy #0 lag: (min: 7.0, avg: 7.8, max: 26.0) [2023-10-07 22:17:37,477][66916] Avg episode reward: [(0, '54.850'), (1, '43.420')] [2023-10-07 22:17:37,829][67871] Updated weights for policy 1, policy_version 62160 (0.0008) [2023-10-07 22:17:38,188][67871] Updated weights for policy 1, policy_version 62170 (0.0010) [2023-10-07 22:17:39,681][67838] Updated weights for policy 0, policy_version 62082 (0.0010) [2023-10-07 22:17:40,044][67838] Updated weights for policy 0, policy_version 62092 (0.0007) [2023-10-07 22:17:40,421][67838] Updated weights for policy 0, policy_version 62102 (0.0007) [2023-10-07 22:17:40,791][67838] Updated weights for policy 0, policy_version 62112 (0.0009) [2023-10-07 22:17:42,433][67871] Updated weights for policy 1, policy_version 62180 (0.0009) [2023-10-07 22:17:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 127270912. Throughput: 0: 1665.8, 1: 1658.1. Samples: 31823556. Policy #0 lag: (min: 7.0, avg: 7.8, max: 26.0) [2023-10-07 22:17:42,477][66916] Avg episode reward: [(0, '52.330'), (1, '44.750')] [2023-10-07 22:17:42,803][67871] Updated weights for policy 1, policy_version 62190 (0.0009) [2023-10-07 22:17:43,169][67871] Updated weights for policy 1, policy_version 62200 (0.0008) [2023-10-07 22:17:44,942][67838] Updated weights for policy 0, policy_version 62122 (0.0010) [2023-10-07 22:17:45,323][67838] Updated weights for policy 0, policy_version 62132 (0.0011) [2023-10-07 22:17:45,681][67838] Updated weights for policy 0, policy_version 62142 (0.0008) [2023-10-07 22:17:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 127336448. Throughput: 0: 1662.3, 1: 1647.0. Samples: 31842942. Policy #0 lag: (min: 7.0, avg: 7.8, max: 26.0) [2023-10-07 22:17:47,477][66916] Avg episode reward: [(0, '52.420'), (1, '43.670')] [2023-10-07 22:17:47,532][67871] Updated weights for policy 1, policy_version 62210 (0.0008) [2023-10-07 22:17:47,892][67871] Updated weights for policy 1, policy_version 62220 (0.0008) [2023-10-07 22:17:48,266][67871] Updated weights for policy 1, policy_version 62230 (0.0010) [2023-10-07 22:17:48,630][67871] Updated weights for policy 1, policy_version 62240 (0.0008) [2023-10-07 22:17:49,855][67838] Updated weights for policy 0, policy_version 62152 (0.0008) [2023-10-07 22:17:50,225][67838] Updated weights for policy 0, policy_version 62162 (0.0007) [2023-10-07 22:17:50,594][67838] Updated weights for policy 0, policy_version 62172 (0.0009) [2023-10-07 22:17:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 127401984. Throughput: 0: 1671.7, 1: 1647.4. Samples: 31863258. Policy #0 lag: (min: 7.0, avg: 7.8, max: 26.0) [2023-10-07 22:17:52,477][66916] Avg episode reward: [(0, '51.220'), (1, '45.240')] [2023-10-07 22:17:52,758][67871] Updated weights for policy 1, policy_version 62250 (0.0011) [2023-10-07 22:17:53,115][67871] Updated weights for policy 1, policy_version 62260 (0.0010) [2023-10-07 22:17:53,484][67871] Updated weights for policy 1, policy_version 62270 (0.0008) [2023-10-07 22:17:54,767][67838] Updated weights for policy 0, policy_version 62182 (0.0010) [2023-10-07 22:17:55,139][67838] Updated weights for policy 0, policy_version 62192 (0.0009) [2023-10-07 22:17:55,522][67838] Updated weights for policy 0, policy_version 62202 (0.0009) [2023-10-07 22:17:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 127467520. Throughput: 0: 1656.9, 1: 1650.6. Samples: 31872972. Policy #0 lag: (min: 7.0, avg: 7.8, max: 26.0) [2023-10-07 22:17:57,477][66916] Avg episode reward: [(0, '54.210'), (1, '44.250')] [2023-10-07 22:17:57,552][67871] Updated weights for policy 1, policy_version 62280 (0.0009) [2023-10-07 22:17:57,927][67871] Updated weights for policy 1, policy_version 62290 (0.0009) [2023-10-07 22:17:58,285][67871] Updated weights for policy 1, policy_version 62300 (0.0011) [2023-10-07 22:17:59,557][67838] Updated weights for policy 0, policy_version 62212 (0.0008) [2023-10-07 22:17:59,919][67838] Updated weights for policy 0, policy_version 62222 (0.0010) [2023-10-07 22:18:00,302][67838] Updated weights for policy 0, policy_version 62232 (0.0011) [2023-10-07 22:18:02,368][67871] Updated weights for policy 1, policy_version 62310 (0.0009) [2023-10-07 22:18:02,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 127533056. Throughput: 0: 1661.3, 1: 1657.1. Samples: 31892938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:18:02,478][66916] Avg episode reward: [(0, '55.310'), (1, '44.620')] [2023-10-07 22:18:02,731][67871] Updated weights for policy 1, policy_version 62320 (0.0008) [2023-10-07 22:18:03,099][67871] Updated weights for policy 1, policy_version 62330 (0.0008) [2023-10-07 22:18:04,564][67838] Updated weights for policy 0, policy_version 62242 (0.0009) [2023-10-07 22:18:04,940][67838] Updated weights for policy 0, policy_version 62252 (0.0008) [2023-10-07 22:18:05,308][67838] Updated weights for policy 0, policy_version 62262 (0.0008) [2023-10-07 22:18:05,686][67838] Updated weights for policy 0, policy_version 62272 (0.0009) [2023-10-07 22:18:07,186][67871] Updated weights for policy 1, policy_version 62340 (0.0009) [2023-10-07 22:18:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 127598592. Throughput: 0: 1658.7, 1: 1659.4. Samples: 31913360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:18:07,478][66916] Avg episode reward: [(0, '51.730'), (1, '46.680')] [2023-10-07 22:18:07,543][67871] Updated weights for policy 1, policy_version 62350 (0.0008) [2023-10-07 22:18:07,907][67871] Updated weights for policy 1, policy_version 62360 (0.0008) [2023-10-07 22:18:09,796][67838] Updated weights for policy 0, policy_version 62282 (0.0007) [2023-10-07 22:18:10,164][67838] Updated weights for policy 0, policy_version 62292 (0.0008) [2023-10-07 22:18:10,530][67838] Updated weights for policy 0, policy_version 62302 (0.0007) [2023-10-07 22:18:12,053][67871] Updated weights for policy 1, policy_version 62370 (0.0008) [2023-10-07 22:18:12,443][67871] Updated weights for policy 1, policy_version 62380 (0.0007) [2023-10-07 22:18:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 127664128. Throughput: 0: 1644.7, 1: 1663.8. Samples: 31923068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:18:12,477][66916] Avg episode reward: [(0, '54.210'), (1, '45.510')] [2023-10-07 22:18:12,812][67871] Updated weights for policy 1, policy_version 62390 (0.0007) [2023-10-07 22:18:13,188][67871] Updated weights for policy 1, policy_version 62400 (0.0008) [2023-10-07 22:18:14,531][67838] Updated weights for policy 0, policy_version 62312 (0.0008) [2023-10-07 22:18:14,903][67838] Updated weights for policy 0, policy_version 62322 (0.0007) [2023-10-07 22:18:15,276][67838] Updated weights for policy 0, policy_version 62332 (0.0009) [2023-10-07 22:18:17,198][67871] Updated weights for policy 1, policy_version 62410 (0.0008) [2023-10-07 22:18:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 127729664. Throughput: 0: 1656.6, 1: 1661.1. Samples: 31942906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:18:17,478][66916] Avg episode reward: [(0, '52.090'), (1, '46.890')] [2023-10-07 22:18:17,573][67871] Updated weights for policy 1, policy_version 62420 (0.0009) [2023-10-07 22:18:17,935][67871] Updated weights for policy 1, policy_version 62430 (0.0008) [2023-10-07 22:18:19,356][67838] Updated weights for policy 0, policy_version 62342 (0.0010) [2023-10-07 22:18:19,730][67838] Updated weights for policy 0, policy_version 62352 (0.0009) [2023-10-07 22:18:20,112][67838] Updated weights for policy 0, policy_version 62362 (0.0009) [2023-10-07 22:18:21,991][67871] Updated weights for policy 1, policy_version 62440 (0.0010) [2023-10-07 22:18:22,360][67871] Updated weights for policy 1, policy_version 62450 (0.0010) [2023-10-07 22:18:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 127795200. Throughput: 0: 1661.7, 1: 1662.3. Samples: 31963484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:18:22,478][66916] Avg episode reward: [(0, '48.890'), (1, '45.390')] [2023-10-07 22:18:22,728][67871] Updated weights for policy 1, policy_version 62460 (0.0008) [2023-10-07 22:18:24,252][67838] Updated weights for policy 0, policy_version 62372 (0.0008) [2023-10-07 22:18:24,644][67838] Updated weights for policy 0, policy_version 62382 (0.0009) [2023-10-07 22:18:25,020][67838] Updated weights for policy 0, policy_version 62392 (0.0010) [2023-10-07 22:18:26,700][67871] Updated weights for policy 1, policy_version 62470 (0.0007) [2023-10-07 22:18:27,064][67871] Updated weights for policy 1, policy_version 62480 (0.0007) [2023-10-07 22:18:27,429][67871] Updated weights for policy 1, policy_version 62490 (0.0007) [2023-10-07 22:18:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 127860736. Throughput: 0: 1650.5, 1: 1670.6. Samples: 31973006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:18:27,478][66916] Avg episode reward: [(0, '52.640'), (1, '47.970')] [2023-10-07 22:18:28,935][67838] Updated weights for policy 0, policy_version 62402 (0.0010) [2023-10-07 22:18:29,302][67838] Updated weights for policy 0, policy_version 62412 (0.0008) [2023-10-07 22:18:29,688][67838] Updated weights for policy 0, policy_version 62422 (0.0010) [2023-10-07 22:18:30,056][67838] Updated weights for policy 0, policy_version 62432 (0.0007) [2023-10-07 22:18:31,785][67871] Updated weights for policy 1, policy_version 62500 (0.0008) [2023-10-07 22:18:32,153][67871] Updated weights for policy 1, policy_version 62510 (0.0008) [2023-10-07 22:18:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 127926272. Throughput: 0: 1665.5, 1: 1677.4. Samples: 31993372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:18:32,477][66916] Avg episode reward: [(0, '50.670'), (1, '45.520')] [2023-10-07 22:18:32,517][67871] Updated weights for policy 1, policy_version 62520 (0.0007) [2023-10-07 22:18:34,158][67838] Updated weights for policy 0, policy_version 62442 (0.0009) [2023-10-07 22:18:34,538][67838] Updated weights for policy 0, policy_version 62452 (0.0008) [2023-10-07 22:18:34,907][67838] Updated weights for policy 0, policy_version 62462 (0.0007) [2023-10-07 22:18:36,642][67871] Updated weights for policy 1, policy_version 62530 (0.0007) [2023-10-07 22:18:37,005][67871] Updated weights for policy 1, policy_version 62540 (0.0007) [2023-10-07 22:18:37,380][67871] Updated weights for policy 1, policy_version 62550 (0.0009) [2023-10-07 22:18:37,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 127991808. Throughput: 0: 1670.6, 1: 1671.4. Samples: 32013648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:18:37,477][66916] Avg episode reward: [(0, '52.620'), (1, '45.880')] [2023-10-07 22:18:37,736][67871] Updated weights for policy 1, policy_version 62560 (0.0009) [2023-10-07 22:18:38,965][67838] Updated weights for policy 0, policy_version 62472 (0.0008) [2023-10-07 22:18:39,345][67838] Updated weights for policy 0, policy_version 62482 (0.0008) [2023-10-07 22:18:39,724][67838] Updated weights for policy 0, policy_version 62492 (0.0009) [2023-10-07 22:18:41,707][67871] Updated weights for policy 1, policy_version 62570 (0.0009) [2023-10-07 22:18:42,064][67871] Updated weights for policy 1, policy_version 62580 (0.0009) [2023-10-07 22:18:42,439][67871] Updated weights for policy 1, policy_version 62590 (0.0007) [2023-10-07 22:18:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 128057344. Throughput: 0: 1653.2, 1: 1682.8. Samples: 32023090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:18:42,477][66916] Avg episode reward: [(0, '50.090'), (1, '47.720')] [2023-10-07 22:18:44,017][67838] Updated weights for policy 0, policy_version 62502 (0.0008) [2023-10-07 22:18:44,395][67838] Updated weights for policy 0, policy_version 62512 (0.0009) [2023-10-07 22:18:44,764][67838] Updated weights for policy 0, policy_version 62522 (0.0009) [2023-10-07 22:18:46,420][67871] Updated weights for policy 1, policy_version 62600 (0.0008) [2023-10-07 22:18:46,785][67871] Updated weights for policy 1, policy_version 62610 (0.0007) [2023-10-07 22:18:47,147][67871] Updated weights for policy 1, policy_version 62620 (0.0010) [2023-10-07 22:18:47,477][66916] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 128155648. Throughput: 0: 1666.3, 1: 1676.4. Samples: 32043362. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-07 22:18:47,478][66916] Avg episode reward: [(0, '45.720'), (1, '48.970')] [2023-10-07 22:18:48,943][67838] Updated weights for policy 0, policy_version 62532 (0.0008) [2023-10-07 22:18:49,319][67838] Updated weights for policy 0, policy_version 62542 (0.0009) [2023-10-07 22:18:49,692][67838] Updated weights for policy 0, policy_version 62552 (0.0008) [2023-10-07 22:18:51,412][67871] Updated weights for policy 1, policy_version 62630 (0.0009) [2023-10-07 22:18:51,778][67871] Updated weights for policy 1, policy_version 62640 (0.0009) [2023-10-07 22:18:52,144][67871] Updated weights for policy 1, policy_version 62650 (0.0007) [2023-10-07 22:18:52,476][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 128221184. Throughput: 0: 1663.5, 1: 1658.1. Samples: 32062828. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-07 22:18:52,477][66916] Avg episode reward: [(0, '46.770'), (1, '49.070')] [2023-10-07 22:18:53,838][67838] Updated weights for policy 0, policy_version 62562 (0.0008) [2023-10-07 22:18:54,215][67838] Updated weights for policy 0, policy_version 62572 (0.0008) [2023-10-07 22:18:54,591][67838] Updated weights for policy 0, policy_version 62582 (0.0010) [2023-10-07 22:18:54,956][67838] Updated weights for policy 0, policy_version 62592 (0.0009) [2023-10-07 22:18:56,342][67871] Updated weights for policy 1, policy_version 62660 (0.0008) [2023-10-07 22:18:56,708][67871] Updated weights for policy 1, policy_version 62670 (0.0009) [2023-10-07 22:18:57,082][67871] Updated weights for policy 1, policy_version 62680 (0.0008) [2023-10-07 22:18:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 128286720. Throughput: 0: 1649.6, 1: 1670.8. Samples: 32072486. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-07 22:18:57,477][66916] Avg episode reward: [(0, '46.140'), (1, '47.980')] [2023-10-07 22:18:59,023][67838] Updated weights for policy 0, policy_version 62602 (0.0010) [2023-10-07 22:18:59,395][67838] Updated weights for policy 0, policy_version 62612 (0.0008) [2023-10-07 22:18:59,780][67838] Updated weights for policy 0, policy_version 62622 (0.0008) [2023-10-07 22:19:01,394][67871] Updated weights for policy 1, policy_version 62690 (0.0009) [2023-10-07 22:19:01,808][67871] Updated weights for policy 1, policy_version 62700 (0.0008) [2023-10-07 22:19:02,169][67871] Updated weights for policy 1, policy_version 62710 (0.0007) [2023-10-07 22:19:02,477][66916] Fps is (10 sec: 9830.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 128319488. Throughput: 0: 1664.3, 1: 1672.5. Samples: 32093064. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-07 22:19:02,478][66916] Avg episode reward: [(0, '44.570'), (1, '46.100')] [2023-10-07 22:19:02,531][67871] Updated weights for policy 1, policy_version 62720 (0.0010) [2023-10-07 22:19:03,668][67838] Updated weights for policy 0, policy_version 62632 (0.0009) [2023-10-07 22:19:04,051][67838] Updated weights for policy 0, policy_version 62642 (0.0010) [2023-10-07 22:19:04,411][67838] Updated weights for policy 0, policy_version 62652 (0.0011) [2023-10-07 22:19:06,427][67871] Updated weights for policy 1, policy_version 62730 (0.0009) [2023-10-07 22:19:06,805][67871] Updated weights for policy 1, policy_version 62740 (0.0008) [2023-10-07 22:19:07,170][67871] Updated weights for policy 1, policy_version 62750 (0.0011) [2023-10-07 22:19:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 128417792. Throughput: 0: 1665.8, 1: 1651.8. Samples: 32112776. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-07 22:19:07,477][66916] Avg episode reward: [(0, '46.300'), (1, '48.540')] [2023-10-07 22:19:07,487][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000062752_64258048.pth... [2023-10-07 22:19:07,487][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000062656_64159744.pth... [2023-10-07 22:19:07,522][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000061184_62652416.pth [2023-10-07 22:19:07,524][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000061120_62586880.pth [2023-10-07 22:19:08,596][67838] Updated weights for policy 0, policy_version 62662 (0.0010) [2023-10-07 22:19:08,974][67838] Updated weights for policy 0, policy_version 62672 (0.0008) [2023-10-07 22:19:09,337][67838] Updated weights for policy 0, policy_version 62682 (0.0009) [2023-10-07 22:19:11,330][67871] Updated weights for policy 1, policy_version 62760 (0.0008) [2023-10-07 22:19:11,693][67871] Updated weights for policy 1, policy_version 62770 (0.0009) [2023-10-07 22:19:12,057][67871] Updated weights for policy 1, policy_version 62780 (0.0008) [2023-10-07 22:19:12,476][66916] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 128483328. Throughput: 0: 1658.8, 1: 1663.9. Samples: 32122528. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-07 22:19:12,477][66916] Avg episode reward: [(0, '44.270'), (1, '46.880')] [2023-10-07 22:19:13,517][67838] Updated weights for policy 0, policy_version 62692 (0.0008) [2023-10-07 22:19:13,892][67838] Updated weights for policy 0, policy_version 62702 (0.0009) [2023-10-07 22:19:14,268][67838] Updated weights for policy 0, policy_version 62712 (0.0008) [2023-10-07 22:19:16,216][67871] Updated weights for policy 1, policy_version 62790 (0.0008) [2023-10-07 22:19:16,585][67871] Updated weights for policy 1, policy_version 62800 (0.0009) [2023-10-07 22:19:16,945][67871] Updated weights for policy 1, policy_version 62810 (0.0008) [2023-10-07 22:19:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 128548864. Throughput: 0: 1664.7, 1: 1662.9. Samples: 32143116. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-07 22:19:17,478][66916] Avg episode reward: [(0, '45.450'), (1, '47.730')] [2023-10-07 22:19:18,400][67838] Updated weights for policy 0, policy_version 62722 (0.0007) [2023-10-07 22:19:18,771][67838] Updated weights for policy 0, policy_version 62732 (0.0007) [2023-10-07 22:19:19,157][67838] Updated weights for policy 0, policy_version 62742 (0.0008) [2023-10-07 22:19:19,530][67838] Updated weights for policy 0, policy_version 62752 (0.0007) [2023-10-07 22:19:20,980][67871] Updated weights for policy 1, policy_version 62820 (0.0008) [2023-10-07 22:19:21,355][67871] Updated weights for policy 1, policy_version 62830 (0.0011) [2023-10-07 22:19:21,719][67871] Updated weights for policy 1, policy_version 62840 (0.0011) [2023-10-07 22:19:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 128614400. Throughput: 0: 1662.0, 1: 1649.6. Samples: 32162668. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-07 22:19:22,477][66916] Avg episode reward: [(0, '46.620'), (1, '48.990')] [2023-10-07 22:19:23,649][67838] Updated weights for policy 0, policy_version 62762 (0.0010) [2023-10-07 22:19:24,023][67838] Updated weights for policy 0, policy_version 62772 (0.0008) [2023-10-07 22:19:24,395][67838] Updated weights for policy 0, policy_version 62782 (0.0007) [2023-10-07 22:19:25,761][67871] Updated weights for policy 1, policy_version 62850 (0.0007) [2023-10-07 22:19:26,124][67871] Updated weights for policy 1, policy_version 62860 (0.0008) [2023-10-07 22:19:26,491][67871] Updated weights for policy 1, policy_version 62870 (0.0007) [2023-10-07 22:19:26,855][67871] Updated weights for policy 1, policy_version 62880 (0.0007) [2023-10-07 22:19:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 128679936. Throughput: 0: 1658.8, 1: 1662.9. Samples: 32172564. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-07 22:19:27,477][66916] Avg episode reward: [(0, '47.180'), (1, '49.290')] [2023-10-07 22:19:28,681][67838] Updated weights for policy 0, policy_version 62792 (0.0008) [2023-10-07 22:19:29,050][67838] Updated weights for policy 0, policy_version 62802 (0.0009) [2023-10-07 22:19:29,419][67838] Updated weights for policy 0, policy_version 62812 (0.0009) [2023-10-07 22:19:30,958][67871] Updated weights for policy 1, policy_version 62890 (0.0008) [2023-10-07 22:19:31,329][67871] Updated weights for policy 1, policy_version 62900 (0.0009) [2023-10-07 22:19:31,700][67871] Updated weights for policy 1, policy_version 62910 (0.0008) [2023-10-07 22:19:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 128745472. Throughput: 0: 1656.4, 1: 1656.9. Samples: 32192458. Policy #0 lag: (min: 1.0, avg: 11.9, max: 33.0) [2023-10-07 22:19:32,478][66916] Avg episode reward: [(0, '49.400'), (1, '48.820')] [2023-10-07 22:19:33,525][67838] Updated weights for policy 0, policy_version 62822 (0.0008) [2023-10-07 22:19:33,905][67838] Updated weights for policy 0, policy_version 62832 (0.0007) [2023-10-07 22:19:34,270][67838] Updated weights for policy 0, policy_version 62842 (0.0008) [2023-10-07 22:19:35,877][67871] Updated weights for policy 1, policy_version 62920 (0.0008) [2023-10-07 22:19:36,242][67871] Updated weights for policy 1, policy_version 62930 (0.0007) [2023-10-07 22:19:36,602][67871] Updated weights for policy 1, policy_version 62940 (0.0007) [2023-10-07 22:19:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 128811008. Throughput: 0: 1663.5, 1: 1658.6. Samples: 32212322. Policy #0 lag: (min: 1.0, avg: 11.9, max: 33.0) [2023-10-07 22:19:37,477][66916] Avg episode reward: [(0, '47.370'), (1, '44.730')] [2023-10-07 22:19:38,307][67838] Updated weights for policy 0, policy_version 62852 (0.0008) [2023-10-07 22:19:38,674][67838] Updated weights for policy 0, policy_version 62862 (0.0010) [2023-10-07 22:19:39,054][67838] Updated weights for policy 0, policy_version 62872 (0.0008) [2023-10-07 22:19:40,701][67871] Updated weights for policy 1, policy_version 62950 (0.0009) [2023-10-07 22:19:41,068][67871] Updated weights for policy 1, policy_version 62960 (0.0007) [2023-10-07 22:19:41,436][67871] Updated weights for policy 1, policy_version 62970 (0.0007) [2023-10-07 22:19:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 128876544. Throughput: 0: 1663.6, 1: 1671.0. Samples: 32222542. Policy #0 lag: (min: 1.0, avg: 11.9, max: 33.0) [2023-10-07 22:19:42,478][66916] Avg episode reward: [(0, '51.600'), (1, '45.460')] [2023-10-07 22:19:43,205][67838] Updated weights for policy 0, policy_version 62882 (0.0008) [2023-10-07 22:19:43,580][67838] Updated weights for policy 0, policy_version 62892 (0.0010) [2023-10-07 22:19:43,954][67838] Updated weights for policy 0, policy_version 62902 (0.0008) [2023-10-07 22:19:44,325][67838] Updated weights for policy 0, policy_version 62912 (0.0007) [2023-10-07 22:19:45,617][67871] Updated weights for policy 1, policy_version 62980 (0.0010) [2023-10-07 22:19:45,991][67871] Updated weights for policy 1, policy_version 62990 (0.0009) [2023-10-07 22:19:46,364][67871] Updated weights for policy 1, policy_version 63000 (0.0011) [2023-10-07 22:19:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 128942080. Throughput: 0: 1656.6, 1: 1658.3. Samples: 32242234. Policy #0 lag: (min: 1.0, avg: 11.9, max: 33.0) [2023-10-07 22:19:47,477][66916] Avg episode reward: [(0, '52.740'), (1, '44.320')] [2023-10-07 22:19:48,487][67838] Updated weights for policy 0, policy_version 62922 (0.0009) [2023-10-07 22:19:48,857][67838] Updated weights for policy 0, policy_version 62932 (0.0009) [2023-10-07 22:19:49,228][67838] Updated weights for policy 0, policy_version 62942 (0.0007) [2023-10-07 22:19:50,383][67871] Updated weights for policy 1, policy_version 63010 (0.0008) [2023-10-07 22:19:50,768][67871] Updated weights for policy 1, policy_version 63020 (0.0007) [2023-10-07 22:19:51,132][67871] Updated weights for policy 1, policy_version 63030 (0.0010) [2023-10-07 22:19:51,494][67871] Updated weights for policy 1, policy_version 63040 (0.0011) [2023-10-07 22:19:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129007616. Throughput: 0: 1652.5, 1: 1659.1. Samples: 32261802. Policy #0 lag: (min: 1.0, avg: 11.9, max: 33.0) [2023-10-07 22:19:52,478][66916] Avg episode reward: [(0, '52.670'), (1, '44.630')] [2023-10-07 22:19:53,365][67838] Updated weights for policy 0, policy_version 62952 (0.0009) [2023-10-07 22:19:53,741][67838] Updated weights for policy 0, policy_version 62962 (0.0008) [2023-10-07 22:19:54,110][67838] Updated weights for policy 0, policy_version 62972 (0.0009) [2023-10-07 22:19:55,509][67871] Updated weights for policy 1, policy_version 63050 (0.0008) [2023-10-07 22:19:55,882][67871] Updated weights for policy 1, policy_version 63060 (0.0008) [2023-10-07 22:19:56,251][67871] Updated weights for policy 1, policy_version 63070 (0.0009) [2023-10-07 22:19:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129073152. Throughput: 0: 1652.2, 1: 1676.2. Samples: 32272306. Policy #0 lag: (min: 1.0, avg: 11.9, max: 33.0) [2023-10-07 22:19:57,477][66916] Avg episode reward: [(0, '54.020'), (1, '45.690')] [2023-10-07 22:19:58,270][67838] Updated weights for policy 0, policy_version 62982 (0.0009) [2023-10-07 22:19:58,651][67838] Updated weights for policy 0, policy_version 62992 (0.0009) [2023-10-07 22:19:59,014][67838] Updated weights for policy 0, policy_version 63002 (0.0009) [2023-10-07 22:20:00,316][67871] Updated weights for policy 1, policy_version 63080 (0.0007) [2023-10-07 22:20:00,681][67871] Updated weights for policy 1, policy_version 63090 (0.0009) [2023-10-07 22:20:01,060][67871] Updated weights for policy 1, policy_version 63100 (0.0009) [2023-10-07 22:20:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 129138688. Throughput: 0: 1649.5, 1: 1659.4. Samples: 32292016. Policy #0 lag: (min: 1.0, avg: 11.9, max: 33.0) [2023-10-07 22:20:02,477][66916] Avg episode reward: [(0, '51.740'), (1, '48.690')] [2023-10-07 22:20:03,212][67838] Updated weights for policy 0, policy_version 63012 (0.0010) [2023-10-07 22:20:03,591][67838] Updated weights for policy 0, policy_version 63022 (0.0009) [2023-10-07 22:20:03,961][67838] Updated weights for policy 0, policy_version 63032 (0.0009) [2023-10-07 22:20:05,234][67871] Updated weights for policy 1, policy_version 63110 (0.0007) [2023-10-07 22:20:05,587][67871] Updated weights for policy 1, policy_version 63120 (0.0008) [2023-10-07 22:20:05,958][67871] Updated weights for policy 1, policy_version 63130 (0.0010) [2023-10-07 22:20:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 129204224. Throughput: 0: 1651.8, 1: 1666.7. Samples: 32311998. Policy #0 lag: (min: 1.0, avg: 11.9, max: 33.0) [2023-10-07 22:20:07,478][66916] Avg episode reward: [(0, '49.330'), (1, '50.540')] [2023-10-07 22:20:08,021][67838] Updated weights for policy 0, policy_version 63042 (0.0009) [2023-10-07 22:20:08,396][67838] Updated weights for policy 0, policy_version 63052 (0.0009) [2023-10-07 22:20:08,770][67838] Updated weights for policy 0, policy_version 63062 (0.0008) [2023-10-07 22:20:09,140][67838] Updated weights for policy 0, policy_version 63072 (0.0007) [2023-10-07 22:20:10,088][67871] Updated weights for policy 1, policy_version 63140 (0.0009) [2023-10-07 22:20:10,452][67871] Updated weights for policy 1, policy_version 63150 (0.0008) [2023-10-07 22:20:10,825][67871] Updated weights for policy 1, policy_version 63160 (0.0010) [2023-10-07 22:20:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 129269760. Throughput: 0: 1655.0, 1: 1677.8. Samples: 32322542. Policy #0 lag: (min: 28.0, avg: 43.7, max: 60.0) [2023-10-07 22:20:12,477][66916] Avg episode reward: [(0, '47.800'), (1, '49.030')] [2023-10-07 22:20:13,323][67838] Updated weights for policy 0, policy_version 63082 (0.0010) [2023-10-07 22:20:13,693][67838] Updated weights for policy 0, policy_version 63092 (0.0011) [2023-10-07 22:20:14,070][67838] Updated weights for policy 0, policy_version 63102 (0.0010) [2023-10-07 22:20:14,833][67871] Updated weights for policy 1, policy_version 63170 (0.0008) [2023-10-07 22:20:15,204][67871] Updated weights for policy 1, policy_version 63180 (0.0011) [2023-10-07 22:20:15,571][67871] Updated weights for policy 1, policy_version 63190 (0.0008) [2023-10-07 22:20:15,937][67871] Updated weights for policy 1, policy_version 63200 (0.0008) [2023-10-07 22:20:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 129335296. Throughput: 0: 1655.0, 1: 1662.8. Samples: 32341760. Policy #0 lag: (min: 28.0, avg: 43.7, max: 60.0) [2023-10-07 22:20:17,477][66916] Avg episode reward: [(0, '48.470'), (1, '52.040')] [2023-10-07 22:20:18,192][67838] Updated weights for policy 0, policy_version 63112 (0.0009) [2023-10-07 22:20:18,562][67838] Updated weights for policy 0, policy_version 63122 (0.0008) [2023-10-07 22:20:18,937][67838] Updated weights for policy 0, policy_version 63132 (0.0008) [2023-10-07 22:20:20,096][67871] Updated weights for policy 1, policy_version 63210 (0.0009) [2023-10-07 22:20:20,466][67871] Updated weights for policy 1, policy_version 63220 (0.0007) [2023-10-07 22:20:20,825][67871] Updated weights for policy 1, policy_version 63230 (0.0010) [2023-10-07 22:20:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 129400832. Throughput: 0: 1654.3, 1: 1678.7. Samples: 32362308. Policy #0 lag: (min: 28.0, avg: 43.7, max: 60.0) [2023-10-07 22:20:22,477][66916] Avg episode reward: [(0, '47.130'), (1, '48.130')] [2023-10-07 22:20:23,075][67838] Updated weights for policy 0, policy_version 63142 (0.0008) [2023-10-07 22:20:23,445][67838] Updated weights for policy 0, policy_version 63152 (0.0011) [2023-10-07 22:20:23,805][67838] Updated weights for policy 0, policy_version 63162 (0.0008) [2023-10-07 22:20:24,757][67871] Updated weights for policy 1, policy_version 63240 (0.0007) [2023-10-07 22:20:25,122][67871] Updated weights for policy 1, policy_version 63250 (0.0007) [2023-10-07 22:20:25,484][67871] Updated weights for policy 1, policy_version 63260 (0.0009) [2023-10-07 22:20:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 129466368. Throughput: 0: 1654.8, 1: 1675.4. Samples: 32372398. Policy #0 lag: (min: 28.0, avg: 43.7, max: 60.0) [2023-10-07 22:20:27,478][66916] Avg episode reward: [(0, '47.590'), (1, '48.680')] [2023-10-07 22:20:27,852][67838] Updated weights for policy 0, policy_version 63172 (0.0008) [2023-10-07 22:20:28,222][67838] Updated weights for policy 0, policy_version 63182 (0.0010) [2023-10-07 22:20:28,590][67838] Updated weights for policy 0, policy_version 63192 (0.0010) [2023-10-07 22:20:29,589][67871] Updated weights for policy 1, policy_version 63270 (0.0007) [2023-10-07 22:20:29,949][67871] Updated weights for policy 1, policy_version 63280 (0.0007) [2023-10-07 22:20:30,329][67871] Updated weights for policy 1, policy_version 63290 (0.0008) [2023-10-07 22:20:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129531904. Throughput: 0: 1661.0, 1: 1669.1. Samples: 32392088. Policy #0 lag: (min: 28.0, avg: 43.7, max: 60.0) [2023-10-07 22:20:32,477][66916] Avg episode reward: [(0, '49.060'), (1, '49.060')] [2023-10-07 22:20:32,775][67838] Updated weights for policy 0, policy_version 63202 (0.0009) [2023-10-07 22:20:33,152][67838] Updated weights for policy 0, policy_version 63212 (0.0009) [2023-10-07 22:20:33,528][67838] Updated weights for policy 0, policy_version 63222 (0.0007) [2023-10-07 22:20:33,900][67838] Updated weights for policy 0, policy_version 63232 (0.0009) [2023-10-07 22:20:34,458][67871] Updated weights for policy 1, policy_version 63300 (0.0008) [2023-10-07 22:20:34,834][67871] Updated weights for policy 1, policy_version 63310 (0.0007) [2023-10-07 22:20:35,197][67871] Updated weights for policy 1, policy_version 63320 (0.0007) [2023-10-07 22:20:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129597440. Throughput: 0: 1656.2, 1: 1691.0. Samples: 32412426. Policy #0 lag: (min: 28.0, avg: 43.7, max: 60.0) [2023-10-07 22:20:37,477][66916] Avg episode reward: [(0, '48.800'), (1, '49.930')] [2023-10-07 22:20:38,086][67838] Updated weights for policy 0, policy_version 63242 (0.0007) [2023-10-07 22:20:38,452][67838] Updated weights for policy 0, policy_version 63252 (0.0009) [2023-10-07 22:20:38,831][67838] Updated weights for policy 0, policy_version 63262 (0.0008) [2023-10-07 22:20:39,169][67871] Updated weights for policy 1, policy_version 63330 (0.0008) [2023-10-07 22:20:39,578][67871] Updated weights for policy 1, policy_version 63340 (0.0008) [2023-10-07 22:20:39,944][67871] Updated weights for policy 1, policy_version 63350 (0.0010) [2023-10-07 22:20:40,316][67871] Updated weights for policy 1, policy_version 63360 (0.0008) [2023-10-07 22:20:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 129662976. Throughput: 0: 1656.8, 1: 1667.4. Samples: 32421894. Policy #0 lag: (min: 28.0, avg: 43.7, max: 60.0) [2023-10-07 22:20:42,478][66916] Avg episode reward: [(0, '52.390'), (1, '48.040')] [2023-10-07 22:20:43,111][67838] Updated weights for policy 0, policy_version 63272 (0.0009) [2023-10-07 22:20:43,472][67838] Updated weights for policy 0, policy_version 63282 (0.0009) [2023-10-07 22:20:43,844][67838] Updated weights for policy 0, policy_version 63292 (0.0009) [2023-10-07 22:20:44,602][67871] Updated weights for policy 1, policy_version 63370 (0.0007) [2023-10-07 22:20:44,960][67871] Updated weights for policy 1, policy_version 63380 (0.0007) [2023-10-07 22:20:45,321][67871] Updated weights for policy 1, policy_version 63390 (0.0009) [2023-10-07 22:20:47,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 129728512. Throughput: 0: 1657.3, 1: 1665.8. Samples: 32441554. Policy #0 lag: (min: 28.0, avg: 43.7, max: 60.0) [2023-10-07 22:20:47,478][66916] Avg episode reward: [(0, '48.170'), (1, '47.400')] [2023-10-07 22:20:47,850][67838] Updated weights for policy 0, policy_version 63302 (0.0007) [2023-10-07 22:20:48,226][67838] Updated weights for policy 0, policy_version 63312 (0.0008) [2023-10-07 22:20:48,607][67838] Updated weights for policy 0, policy_version 63322 (0.0008) [2023-10-07 22:20:49,261][67871] Updated weights for policy 1, policy_version 63400 (0.0008) [2023-10-07 22:20:49,622][67871] Updated weights for policy 1, policy_version 63410 (0.0007) [2023-10-07 22:20:49,990][67871] Updated weights for policy 1, policy_version 63420 (0.0008) [2023-10-07 22:20:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 129794048. Throughput: 0: 1656.2, 1: 1683.5. Samples: 32462286. Policy #0 lag: (min: 28.0, avg: 43.7, max: 60.0) [2023-10-07 22:20:52,478][66916] Avg episode reward: [(0, '51.400'), (1, '42.800')] [2023-10-07 22:20:52,727][67838] Updated weights for policy 0, policy_version 63332 (0.0009) [2023-10-07 22:20:53,098][67838] Updated weights for policy 0, policy_version 63342 (0.0009) [2023-10-07 22:20:53,473][67838] Updated weights for policy 0, policy_version 63352 (0.0008) [2023-10-07 22:20:53,916][67871] Updated weights for policy 1, policy_version 63430 (0.0009) [2023-10-07 22:20:54,275][67871] Updated weights for policy 1, policy_version 63440 (0.0010) [2023-10-07 22:20:54,643][67871] Updated weights for policy 1, policy_version 63450 (0.0011) [2023-10-07 22:20:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 129859584. Throughput: 0: 1655.6, 1: 1655.9. Samples: 32471560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:20:57,477][66916] Avg episode reward: [(0, '49.460'), (1, '44.260')] [2023-10-07 22:20:57,546][67838] Updated weights for policy 0, policy_version 63362 (0.0008) [2023-10-07 22:20:57,926][67838] Updated weights for policy 0, policy_version 63372 (0.0009) [2023-10-07 22:20:58,307][67838] Updated weights for policy 0, policy_version 63382 (0.0010) [2023-10-07 22:20:58,680][67838] Updated weights for policy 0, policy_version 63392 (0.0009) [2023-10-07 22:20:58,785][67871] Updated weights for policy 1, policy_version 63460 (0.0009) [2023-10-07 22:20:59,155][67871] Updated weights for policy 1, policy_version 63470 (0.0007) [2023-10-07 22:20:59,527][67871] Updated weights for policy 1, policy_version 63480 (0.0007) [2023-10-07 22:21:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 129925120. Throughput: 0: 1658.7, 1: 1675.8. Samples: 32491814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:21:02,478][66916] Avg episode reward: [(0, '49.360'), (1, '46.170')] [2023-10-07 22:21:02,767][67838] Updated weights for policy 0, policy_version 63402 (0.0008) [2023-10-07 22:21:03,133][67838] Updated weights for policy 0, policy_version 63412 (0.0009) [2023-10-07 22:21:03,516][67838] Updated weights for policy 0, policy_version 63422 (0.0007) [2023-10-07 22:21:03,693][67871] Updated weights for policy 1, policy_version 63490 (0.0008) [2023-10-07 22:21:04,063][67871] Updated weights for policy 1, policy_version 63500 (0.0010) [2023-10-07 22:21:04,431][67871] Updated weights for policy 1, policy_version 63510 (0.0007) [2023-10-07 22:21:04,801][67871] Updated weights for policy 1, policy_version 63520 (0.0007) [2023-10-07 22:21:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 129990656. Throughput: 0: 1656.6, 1: 1676.9. Samples: 32512314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:21:07,477][66916] Avg episode reward: [(0, '48.870'), (1, '48.220')] [2023-10-07 22:21:07,487][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000063520_65044480.pth... [2023-10-07 22:21:07,524][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000061952_63438848.pth [2023-10-07 22:21:07,736][67838] Updated weights for policy 0, policy_version 63432 (0.0008) [2023-10-07 22:21:08,114][67838] Updated weights for policy 0, policy_version 63442 (0.0007) [2023-10-07 22:21:08,482][67838] Updated weights for policy 0, policy_version 63452 (0.0009) [2023-10-07 22:21:08,624][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000063456_64978944.pth... [2023-10-07 22:21:08,652][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000061888_63373312.pth [2023-10-07 22:21:08,851][67871] Updated weights for policy 1, policy_version 63530 (0.0010) [2023-10-07 22:21:09,218][67871] Updated weights for policy 1, policy_version 63540 (0.0012) [2023-10-07 22:21:09,584][67871] Updated weights for policy 1, policy_version 63550 (0.0010) [2023-10-07 22:21:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 130056192. Throughput: 0: 1657.4, 1: 1651.2. Samples: 32521284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:21:12,478][66916] Avg episode reward: [(0, '49.780'), (1, '50.190')] [2023-10-07 22:21:12,707][67838] Updated weights for policy 0, policy_version 63462 (0.0010) [2023-10-07 22:21:13,086][67838] Updated weights for policy 0, policy_version 63472 (0.0010) [2023-10-07 22:21:13,460][67838] Updated weights for policy 0, policy_version 63482 (0.0010) [2023-10-07 22:21:13,729][67871] Updated weights for policy 1, policy_version 63560 (0.0010) [2023-10-07 22:21:14,100][67871] Updated weights for policy 1, policy_version 63570 (0.0009) [2023-10-07 22:21:14,461][67871] Updated weights for policy 1, policy_version 63580 (0.0007) [2023-10-07 22:21:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 130121728. Throughput: 0: 1652.5, 1: 1671.1. Samples: 32541650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:21:17,477][66916] Avg episode reward: [(0, '52.230'), (1, '54.650')] [2023-10-07 22:21:17,511][67838] Updated weights for policy 0, policy_version 63492 (0.0009) [2023-10-07 22:21:17,881][67838] Updated weights for policy 0, policy_version 63502 (0.0007) [2023-10-07 22:21:18,255][67838] Updated weights for policy 0, policy_version 63512 (0.0010) [2023-10-07 22:21:18,499][67871] Updated weights for policy 1, policy_version 63590 (0.0007) [2023-10-07 22:21:18,864][67871] Updated weights for policy 1, policy_version 63600 (0.0010) [2023-10-07 22:21:19,233][67871] Updated weights for policy 1, policy_version 63610 (0.0010) [2023-10-07 22:21:22,368][67838] Updated weights for policy 0, policy_version 63522 (0.0009) [2023-10-07 22:21:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 130187264. Throughput: 0: 1658.6, 1: 1670.3. Samples: 32562224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:21:22,478][66916] Avg episode reward: [(0, '52.820'), (1, '58.010')] [2023-10-07 22:21:22,737][67838] Updated weights for policy 0, policy_version 63532 (0.0007) [2023-10-07 22:21:23,109][67838] Updated weights for policy 0, policy_version 63542 (0.0007) [2023-10-07 22:21:23,340][67871] Updated weights for policy 1, policy_version 63620 (0.0008) [2023-10-07 22:21:23,476][67838] Updated weights for policy 0, policy_version 63552 (0.0008) [2023-10-07 22:21:23,707][67871] Updated weights for policy 1, policy_version 63630 (0.0007) [2023-10-07 22:21:24,071][67871] Updated weights for policy 1, policy_version 63640 (0.0007) [2023-10-07 22:21:27,449][67838] Updated weights for policy 0, policy_version 63562 (0.0008) [2023-10-07 22:21:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 130252800. Throughput: 0: 1661.8, 1: 1658.4. Samples: 32571302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:21:27,477][66916] Avg episode reward: [(0, '52.790'), (1, '57.150')] [2023-10-07 22:21:27,828][67838] Updated weights for policy 0, policy_version 63572 (0.0009) [2023-10-07 22:21:28,204][67838] Updated weights for policy 0, policy_version 63582 (0.0010) [2023-10-07 22:21:28,227][67871] Updated weights for policy 1, policy_version 63650 (0.0008) [2023-10-07 22:21:28,638][67871] Updated weights for policy 1, policy_version 63660 (0.0009) [2023-10-07 22:21:29,007][67871] Updated weights for policy 1, policy_version 63670 (0.0009) [2023-10-07 22:21:29,384][67871] Updated weights for policy 1, policy_version 63680 (0.0009) [2023-10-07 22:21:32,453][67838] Updated weights for policy 0, policy_version 63592 (0.0007) [2023-10-07 22:21:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 130318336. Throughput: 0: 1663.4, 1: 1674.9. Samples: 32591778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:21:32,478][66916] Avg episode reward: [(0, '51.940'), (1, '55.430')] [2023-10-07 22:21:32,825][67838] Updated weights for policy 0, policy_version 63602 (0.0008) [2023-10-07 22:21:33,208][67838] Updated weights for policy 0, policy_version 63612 (0.0007) [2023-10-07 22:21:33,429][67871] Updated weights for policy 1, policy_version 63690 (0.0008) [2023-10-07 22:21:33,803][67871] Updated weights for policy 1, policy_version 63700 (0.0008) [2023-10-07 22:21:34,170][67871] Updated weights for policy 1, policy_version 63710 (0.0007) [2023-10-07 22:21:37,173][67838] Updated weights for policy 0, policy_version 63622 (0.0009) [2023-10-07 22:21:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 130383872. Throughput: 0: 1660.2, 1: 1672.6. Samples: 32612262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:21:37,478][66916] Avg episode reward: [(0, '48.640'), (1, '54.590')] [2023-10-07 22:21:37,541][67838] Updated weights for policy 0, policy_version 63632 (0.0008) [2023-10-07 22:21:37,918][67838] Updated weights for policy 0, policy_version 63642 (0.0012) [2023-10-07 22:21:38,310][67871] Updated weights for policy 1, policy_version 63720 (0.0010) [2023-10-07 22:21:38,679][67871] Updated weights for policy 1, policy_version 63730 (0.0009) [2023-10-07 22:21:39,037][67871] Updated weights for policy 1, policy_version 63740 (0.0007) [2023-10-07 22:21:42,022][67838] Updated weights for policy 0, policy_version 63652 (0.0009) [2023-10-07 22:21:42,395][67838] Updated weights for policy 0, policy_version 63662 (0.0010) [2023-10-07 22:21:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 130449408. Throughput: 0: 1662.7, 1: 1671.8. Samples: 32621612. Policy #0 lag: (min: 2.0, avg: 2.7, max: 20.0) [2023-10-07 22:21:42,477][66916] Avg episode reward: [(0, '45.680'), (1, '51.720')] [2023-10-07 22:21:42,771][67838] Updated weights for policy 0, policy_version 63672 (0.0008) [2023-10-07 22:21:43,057][67871] Updated weights for policy 1, policy_version 63750 (0.0007) [2023-10-07 22:21:43,417][67871] Updated weights for policy 1, policy_version 63760 (0.0008) [2023-10-07 22:21:43,787][67871] Updated weights for policy 1, policy_version 63770 (0.0010) [2023-10-07 22:21:46,907][67838] Updated weights for policy 0, policy_version 63682 (0.0007) [2023-10-07 22:21:47,282][67838] Updated weights for policy 0, policy_version 63692 (0.0008) [2023-10-07 22:21:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 130514944. Throughput: 0: 1663.0, 1: 1678.6. Samples: 32642188. Policy #0 lag: (min: 2.0, avg: 2.7, max: 20.0) [2023-10-07 22:21:47,478][66916] Avg episode reward: [(0, '49.180'), (1, '49.690')] [2023-10-07 22:21:47,647][67838] Updated weights for policy 0, policy_version 63702 (0.0010) [2023-10-07 22:21:47,999][67871] Updated weights for policy 1, policy_version 63780 (0.0010) [2023-10-07 22:21:48,016][67838] Updated weights for policy 0, policy_version 63712 (0.0010) [2023-10-07 22:21:48,366][67871] Updated weights for policy 1, policy_version 63790 (0.0009) [2023-10-07 22:21:48,738][67871] Updated weights for policy 1, policy_version 63800 (0.0009) [2023-10-07 22:21:52,156][67838] Updated weights for policy 0, policy_version 63722 (0.0008) [2023-10-07 22:21:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 130580480. Throughput: 0: 1654.5, 1: 1679.7. Samples: 32662356. Policy #0 lag: (min: 2.0, avg: 2.7, max: 20.0) [2023-10-07 22:21:52,478][66916] Avg episode reward: [(0, '47.860'), (1, '51.640')] [2023-10-07 22:21:52,529][67838] Updated weights for policy 0, policy_version 63732 (0.0008) [2023-10-07 22:21:52,783][67871] Updated weights for policy 1, policy_version 63810 (0.0007) [2023-10-07 22:21:52,907][67838] Updated weights for policy 0, policy_version 63742 (0.0009) [2023-10-07 22:21:53,145][67871] Updated weights for policy 1, policy_version 63820 (0.0008) [2023-10-07 22:21:53,518][67871] Updated weights for policy 1, policy_version 63830 (0.0007) [2023-10-07 22:21:53,870][67871] Updated weights for policy 1, policy_version 63840 (0.0009) [2023-10-07 22:21:56,902][67838] Updated weights for policy 0, policy_version 63752 (0.0008) [2023-10-07 22:21:57,279][67838] Updated weights for policy 0, policy_version 63762 (0.0011) [2023-10-07 22:21:57,477][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 130646016. Throughput: 0: 1659.2, 1: 1680.5. Samples: 32671570. Policy #0 lag: (min: 2.0, avg: 2.7, max: 20.0) [2023-10-07 22:21:57,478][66916] Avg episode reward: [(0, '52.520'), (1, '49.060')] [2023-10-07 22:21:57,653][67838] Updated weights for policy 0, policy_version 63772 (0.0009) [2023-10-07 22:21:57,985][67871] Updated weights for policy 1, policy_version 63850 (0.0009) [2023-10-07 22:21:58,357][67871] Updated weights for policy 1, policy_version 63860 (0.0009) [2023-10-07 22:21:58,726][67871] Updated weights for policy 1, policy_version 63870 (0.0009) [2023-10-07 22:22:01,784][67838] Updated weights for policy 0, policy_version 63782 (0.0008) [2023-10-07 22:22:02,157][67838] Updated weights for policy 0, policy_version 63792 (0.0007) [2023-10-07 22:22:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 130711552. Throughput: 0: 1661.2, 1: 1678.8. Samples: 32691950. Policy #0 lag: (min: 2.0, avg: 2.7, max: 20.0) [2023-10-07 22:22:02,477][66916] Avg episode reward: [(0, '52.800'), (1, '52.670')] [2023-10-07 22:22:02,534][67838] Updated weights for policy 0, policy_version 63802 (0.0007) [2023-10-07 22:22:02,625][67871] Updated weights for policy 1, policy_version 63880 (0.0008) [2023-10-07 22:22:02,998][67871] Updated weights for policy 1, policy_version 63890 (0.0009) [2023-10-07 22:22:03,369][67871] Updated weights for policy 1, policy_version 63900 (0.0009) [2023-10-07 22:22:06,657][67838] Updated weights for policy 0, policy_version 63812 (0.0007) [2023-10-07 22:22:07,021][67838] Updated weights for policy 0, policy_version 63822 (0.0007) [2023-10-07 22:22:07,393][67838] Updated weights for policy 0, policy_version 63832 (0.0007) [2023-10-07 22:22:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 130777088. Throughput: 0: 1648.4, 1: 1676.3. Samples: 32711836. Policy #0 lag: (min: 2.0, avg: 2.7, max: 20.0) [2023-10-07 22:22:07,477][66916] Avg episode reward: [(0, '52.100'), (1, '54.140')] [2023-10-07 22:22:07,506][67871] Updated weights for policy 1, policy_version 63910 (0.0009) [2023-10-07 22:22:07,872][67871] Updated weights for policy 1, policy_version 63920 (0.0008) [2023-10-07 22:22:08,240][67871] Updated weights for policy 1, policy_version 63930 (0.0008) [2023-10-07 22:22:11,468][67838] Updated weights for policy 0, policy_version 63842 (0.0008) [2023-10-07 22:22:11,850][67838] Updated weights for policy 0, policy_version 63852 (0.0007) [2023-10-07 22:22:12,218][67838] Updated weights for policy 0, policy_version 63862 (0.0007) [2023-10-07 22:22:12,343][67871] Updated weights for policy 1, policy_version 63940 (0.0007) [2023-10-07 22:22:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 130842624. Throughput: 0: 1659.3, 1: 1676.2. Samples: 32721398. Policy #0 lag: (min: 2.0, avg: 2.7, max: 20.0) [2023-10-07 22:22:12,477][66916] Avg episode reward: [(0, '51.920'), (1, '56.770')] [2023-10-07 22:22:12,587][67838] Updated weights for policy 0, policy_version 63872 (0.0007) [2023-10-07 22:22:12,712][67871] Updated weights for policy 1, policy_version 63950 (0.0009) [2023-10-07 22:22:13,069][67871] Updated weights for policy 1, policy_version 63960 (0.0010) [2023-10-07 22:22:16,890][67838] Updated weights for policy 0, policy_version 63882 (0.0009) [2023-10-07 22:22:17,261][67838] Updated weights for policy 0, policy_version 63892 (0.0008) [2023-10-07 22:22:17,456][67871] Updated weights for policy 1, policy_version 63970 (0.0007) [2023-10-07 22:22:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 130908160. Throughput: 0: 1659.3, 1: 1675.3. Samples: 32741834. Policy #0 lag: (min: 2.0, avg: 2.7, max: 20.0) [2023-10-07 22:22:17,477][66916] Avg episode reward: [(0, '50.640'), (1, '54.600')] [2023-10-07 22:22:17,629][67838] Updated weights for policy 0, policy_version 63902 (0.0009) [2023-10-07 22:22:17,856][67871] Updated weights for policy 1, policy_version 63980 (0.0010) [2023-10-07 22:22:18,223][67871] Updated weights for policy 1, policy_version 63990 (0.0011) [2023-10-07 22:22:18,594][67871] Updated weights for policy 1, policy_version 64000 (0.0008) [2023-10-07 22:22:21,819][67838] Updated weights for policy 0, policy_version 63912 (0.0008) [2023-10-07 22:22:22,199][67838] Updated weights for policy 0, policy_version 63922 (0.0007) [2023-10-07 22:22:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 130973696. Throughput: 0: 1644.7, 1: 1668.8. Samples: 32761370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:22:22,477][66916] Avg episode reward: [(0, '49.480'), (1, '54.550')] [2023-10-07 22:22:22,565][67838] Updated weights for policy 0, policy_version 63932 (0.0007) [2023-10-07 22:22:22,774][67871] Updated weights for policy 1, policy_version 64010 (0.0009) [2023-10-07 22:22:23,130][67871] Updated weights for policy 1, policy_version 64020 (0.0009) [2023-10-07 22:22:23,495][67871] Updated weights for policy 1, policy_version 64030 (0.0009) [2023-10-07 22:22:26,718][67838] Updated weights for policy 0, policy_version 63942 (0.0008) [2023-10-07 22:22:27,089][67838] Updated weights for policy 0, policy_version 63952 (0.0007) [2023-10-07 22:22:27,459][67838] Updated weights for policy 0, policy_version 63962 (0.0007) [2023-10-07 22:22:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 131039232. Throughput: 0: 1655.6, 1: 1664.9. Samples: 32771036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:22:27,477][66916] Avg episode reward: [(0, '50.020'), (1, '52.110')] [2023-10-07 22:22:27,512][67871] Updated weights for policy 1, policy_version 64040 (0.0008) [2023-10-07 22:22:27,883][67871] Updated weights for policy 1, policy_version 64050 (0.0009) [2023-10-07 22:22:28,249][67871] Updated weights for policy 1, policy_version 64060 (0.0009) [2023-10-07 22:22:31,470][67838] Updated weights for policy 0, policy_version 63972 (0.0008) [2023-10-07 22:22:31,838][67838] Updated weights for policy 0, policy_version 63982 (0.0010) [2023-10-07 22:22:32,222][67838] Updated weights for policy 0, policy_version 63992 (0.0011) [2023-10-07 22:22:32,431][67871] Updated weights for policy 1, policy_version 64070 (0.0008) [2023-10-07 22:22:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 131104768. Throughput: 0: 1655.3, 1: 1661.0. Samples: 32791422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:22:32,477][66916] Avg episode reward: [(0, '45.630'), (1, '48.560')] [2023-10-07 22:22:32,796][67871] Updated weights for policy 1, policy_version 64080 (0.0007) [2023-10-07 22:22:33,163][67871] Updated weights for policy 1, policy_version 64090 (0.0010) [2023-10-07 22:22:36,480][67838] Updated weights for policy 0, policy_version 64002 (0.0008) [2023-10-07 22:22:36,857][67838] Updated weights for policy 0, policy_version 64012 (0.0007) [2023-10-07 22:22:37,228][67838] Updated weights for policy 0, policy_version 64022 (0.0008) [2023-10-07 22:22:37,282][67871] Updated weights for policy 1, policy_version 64100 (0.0007) [2023-10-07 22:22:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 131170304. Throughput: 0: 1644.7, 1: 1660.3. Samples: 32811082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:22:37,477][66916] Avg episode reward: [(0, '48.270'), (1, '49.260')] [2023-10-07 22:22:37,598][67838] Updated weights for policy 0, policy_version 64032 (0.0007) [2023-10-07 22:22:37,655][67871] Updated weights for policy 1, policy_version 64110 (0.0007) [2023-10-07 22:22:38,021][67871] Updated weights for policy 1, policy_version 64120 (0.0009) [2023-10-07 22:22:41,883][67838] Updated weights for policy 0, policy_version 64042 (0.0009) [2023-10-07 22:22:42,033][67871] Updated weights for policy 1, policy_version 64130 (0.0010) [2023-10-07 22:22:42,261][67838] Updated weights for policy 0, policy_version 64052 (0.0008) [2023-10-07 22:22:42,408][67871] Updated weights for policy 1, policy_version 64140 (0.0008) [2023-10-07 22:22:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 131235840. Throughput: 0: 1649.3, 1: 1661.3. Samples: 32820544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:22:42,477][66916] Avg episode reward: [(0, '45.490'), (1, '47.750')] [2023-10-07 22:22:42,635][67838] Updated weights for policy 0, policy_version 64062 (0.0009) [2023-10-07 22:22:42,768][67871] Updated weights for policy 1, policy_version 64150 (0.0008) [2023-10-07 22:22:43,142][67871] Updated weights for policy 1, policy_version 64160 (0.0007) [2023-10-07 22:22:46,764][67838] Updated weights for policy 0, policy_version 64072 (0.0008) [2023-10-07 22:22:47,139][67838] Updated weights for policy 0, policy_version 64082 (0.0007) [2023-10-07 22:22:47,231][67871] Updated weights for policy 1, policy_version 64170 (0.0008) [2023-10-07 22:22:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 131301376. Throughput: 0: 1653.1, 1: 1660.7. Samples: 32841068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:22:47,477][66916] Avg episode reward: [(0, '46.570'), (1, '51.380')] [2023-10-07 22:22:47,504][67838] Updated weights for policy 0, policy_version 64092 (0.0007) [2023-10-07 22:22:47,595][67871] Updated weights for policy 1, policy_version 64180 (0.0010) [2023-10-07 22:22:47,952][67871] Updated weights for policy 1, policy_version 64190 (0.0008) [2023-10-07 22:22:51,423][67838] Updated weights for policy 0, policy_version 64102 (0.0008) [2023-10-07 22:22:51,795][67838] Updated weights for policy 0, policy_version 64112 (0.0010) [2023-10-07 22:22:52,151][67871] Updated weights for policy 1, policy_version 64200 (0.0009) [2023-10-07 22:22:52,168][67838] Updated weights for policy 0, policy_version 64122 (0.0008) [2023-10-07 22:22:52,476][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 131399680. Throughput: 0: 1652.4, 1: 1662.5. Samples: 32861010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:22:52,477][66916] Avg episode reward: [(0, '46.830'), (1, '50.650')] [2023-10-07 22:22:52,520][67871] Updated weights for policy 1, policy_version 64210 (0.0008) [2023-10-07 22:22:52,886][67871] Updated weights for policy 1, policy_version 64220 (0.0009) [2023-10-07 22:22:56,290][67838] Updated weights for policy 0, policy_version 64132 (0.0008) [2023-10-07 22:22:56,663][67838] Updated weights for policy 0, policy_version 64142 (0.0009) [2023-10-07 22:22:56,961][67871] Updated weights for policy 1, policy_version 64230 (0.0008) [2023-10-07 22:22:57,031][67838] Updated weights for policy 0, policy_version 64152 (0.0007) [2023-10-07 22:22:57,336][67871] Updated weights for policy 1, policy_version 64240 (0.0008) [2023-10-07 22:22:57,476][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 131465216. Throughput: 0: 1659.5, 1: 1660.1. Samples: 32870782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:22:57,477][66916] Avg episode reward: [(0, '45.700'), (1, '49.070')] [2023-10-07 22:22:57,696][67871] Updated weights for policy 1, policy_version 64250 (0.0008) [2023-10-07 22:23:01,060][67838] Updated weights for policy 0, policy_version 64162 (0.0007) [2023-10-07 22:23:01,434][67838] Updated weights for policy 0, policy_version 64172 (0.0007) [2023-10-07 22:23:01,798][67838] Updated weights for policy 0, policy_version 64182 (0.0009) [2023-10-07 22:23:01,933][67871] Updated weights for policy 1, policy_version 64260 (0.0009) [2023-10-07 22:23:02,169][67838] Updated weights for policy 0, policy_version 64192 (0.0009) [2023-10-07 22:23:02,303][67871] Updated weights for policy 1, policy_version 64270 (0.0007) [2023-10-07 22:23:02,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 131530752. Throughput: 0: 1657.9, 1: 1664.8. Samples: 32891360. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-07 22:23:02,478][66916] Avg episode reward: [(0, '46.030'), (1, '48.570')] [2023-10-07 22:23:02,666][67871] Updated weights for policy 1, policy_version 64280 (0.0009) [2023-10-07 22:23:06,435][67838] Updated weights for policy 0, policy_version 64202 (0.0008) [2023-10-07 22:23:06,761][67871] Updated weights for policy 1, policy_version 64290 (0.0007) [2023-10-07 22:23:06,808][67838] Updated weights for policy 0, policy_version 64212 (0.0007) [2023-10-07 22:23:07,178][67871] Updated weights for policy 1, policy_version 64300 (0.0007) [2023-10-07 22:23:07,178][67838] Updated weights for policy 0, policy_version 64222 (0.0008) [2023-10-07 22:23:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 131596288. Throughput: 0: 1654.4, 1: 1666.1. Samples: 32910792. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-07 22:23:07,477][66916] Avg episode reward: [(0, '45.390'), (1, '47.080')] [2023-10-07 22:23:07,484][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000064224_65765376.pth... [2023-10-07 22:23:07,521][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000062656_64159744.pth [2023-10-07 22:23:07,542][67871] Updated weights for policy 1, policy_version 64310 (0.0007) [2023-10-07 22:23:07,905][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000064320_65863680.pth... [2023-10-07 22:23:07,905][67871] Updated weights for policy 1, policy_version 64320 (0.0008) [2023-10-07 22:23:07,942][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000062752_64258048.pth [2023-10-07 22:23:11,265][67838] Updated weights for policy 0, policy_version 64232 (0.0010) [2023-10-07 22:23:11,641][67838] Updated weights for policy 0, policy_version 64242 (0.0007) [2023-10-07 22:23:11,904][67871] Updated weights for policy 1, policy_version 64330 (0.0008) [2023-10-07 22:23:12,009][67838] Updated weights for policy 0, policy_version 64252 (0.0007) [2023-10-07 22:23:12,261][67871] Updated weights for policy 1, policy_version 64340 (0.0010) [2023-10-07 22:23:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 131661824. Throughput: 0: 1661.8, 1: 1667.8. Samples: 32920868. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-07 22:23:12,478][66916] Avg episode reward: [(0, '47.510'), (1, '48.640')] [2023-10-07 22:23:12,623][67871] Updated weights for policy 1, policy_version 64350 (0.0010) [2023-10-07 22:23:16,234][67838] Updated weights for policy 0, policy_version 64262 (0.0007) [2023-10-07 22:23:16,604][67838] Updated weights for policy 0, policy_version 64272 (0.0008) [2023-10-07 22:23:16,918][67871] Updated weights for policy 1, policy_version 64360 (0.0008) [2023-10-07 22:23:16,966][67838] Updated weights for policy 0, policy_version 64282 (0.0007) [2023-10-07 22:23:17,291][67871] Updated weights for policy 1, policy_version 64370 (0.0008) [2023-10-07 22:23:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 131727360. Throughput: 0: 1658.7, 1: 1663.5. Samples: 32940924. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-07 22:23:17,478][66916] Avg episode reward: [(0, '47.140'), (1, '54.160')] [2023-10-07 22:23:17,662][67871] Updated weights for policy 1, policy_version 64380 (0.0008) [2023-10-07 22:23:21,020][67838] Updated weights for policy 0, policy_version 64292 (0.0008) [2023-10-07 22:23:21,391][67838] Updated weights for policy 0, policy_version 64302 (0.0008) [2023-10-07 22:23:21,691][67871] Updated weights for policy 1, policy_version 64390 (0.0007) [2023-10-07 22:23:21,763][67838] Updated weights for policy 0, policy_version 64312 (0.0008) [2023-10-07 22:23:22,049][67871] Updated weights for policy 1, policy_version 64400 (0.0008) [2023-10-07 22:23:22,424][67871] Updated weights for policy 1, policy_version 64410 (0.0009) [2023-10-07 22:23:22,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 131792896. Throughput: 0: 1651.1, 1: 1655.0. Samples: 32959856. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-07 22:23:22,478][66916] Avg episode reward: [(0, '43.700'), (1, '56.970')] [2023-10-07 22:23:25,866][67838] Updated weights for policy 0, policy_version 64322 (0.0009) [2023-10-07 22:23:26,242][67838] Updated weights for policy 0, policy_version 64332 (0.0007) [2023-10-07 22:23:26,453][67871] Updated weights for policy 1, policy_version 64420 (0.0009) [2023-10-07 22:23:26,609][67838] Updated weights for policy 0, policy_version 64342 (0.0007) [2023-10-07 22:23:26,823][67871] Updated weights for policy 1, policy_version 64430 (0.0008) [2023-10-07 22:23:26,972][67838] Updated weights for policy 0, policy_version 64352 (0.0007) [2023-10-07 22:23:27,187][67871] Updated weights for policy 1, policy_version 64440 (0.0008) [2023-10-07 22:23:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 131858432. Throughput: 0: 1671.1, 1: 1665.1. Samples: 32970672. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-07 22:23:27,477][66916] Avg episode reward: [(0, '43.670'), (1, '57.740')] [2023-10-07 22:23:31,138][67838] Updated weights for policy 0, policy_version 64362 (0.0010) [2023-10-07 22:23:31,425][67871] Updated weights for policy 1, policy_version 64450 (0.0009) [2023-10-07 22:23:31,506][67838] Updated weights for policy 0, policy_version 64372 (0.0009) [2023-10-07 22:23:31,792][67871] Updated weights for policy 1, policy_version 64460 (0.0008) [2023-10-07 22:23:31,863][67838] Updated weights for policy 0, policy_version 64382 (0.0009) [2023-10-07 22:23:32,159][67871] Updated weights for policy 1, policy_version 64470 (0.0008) [2023-10-07 22:23:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 131923968. Throughput: 0: 1658.9, 1: 1664.1. Samples: 32990602. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-07 22:23:32,477][66916] Avg episode reward: [(0, '40.970'), (1, '59.390')] [2023-10-07 22:23:32,527][67871] Updated weights for policy 1, policy_version 64480 (0.0008) [2023-10-07 22:23:36,033][67838] Updated weights for policy 0, policy_version 64392 (0.0007) [2023-10-07 22:23:36,406][67838] Updated weights for policy 0, policy_version 64402 (0.0007) [2023-10-07 22:23:36,539][67871] Updated weights for policy 1, policy_version 64490 (0.0009) [2023-10-07 22:23:36,779][67838] Updated weights for policy 0, policy_version 64412 (0.0007) [2023-10-07 22:23:36,897][67871] Updated weights for policy 1, policy_version 64500 (0.0008) [2023-10-07 22:23:37,274][67871] Updated weights for policy 1, policy_version 64510 (0.0011) [2023-10-07 22:23:37,477][66916] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 132022272. Throughput: 0: 1648.4, 1: 1651.4. Samples: 33009502. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-07 22:23:37,478][66916] Avg episode reward: [(0, '42.420'), (1, '60.390')] [2023-10-07 22:23:40,836][67838] Updated weights for policy 0, policy_version 64422 (0.0008) [2023-10-07 22:23:41,215][67838] Updated weights for policy 0, policy_version 64432 (0.0008) [2023-10-07 22:23:41,494][67871] Updated weights for policy 1, policy_version 64520 (0.0007) [2023-10-07 22:23:41,583][67838] Updated weights for policy 0, policy_version 64442 (0.0008) [2023-10-07 22:23:41,863][67871] Updated weights for policy 1, policy_version 64530 (0.0008) [2023-10-07 22:23:42,226][67871] Updated weights for policy 1, policy_version 64540 (0.0007) [2023-10-07 22:23:42,476][66916] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 132087808. Throughput: 0: 1656.3, 1: 1665.6. Samples: 33020268. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-07 22:23:42,477][66916] Avg episode reward: [(0, '41.360'), (1, '58.970')] [2023-10-07 22:23:45,640][67838] Updated weights for policy 0, policy_version 64452 (0.0008) [2023-10-07 22:23:46,008][67838] Updated weights for policy 0, policy_version 64462 (0.0008) [2023-10-07 22:23:46,372][67871] Updated weights for policy 1, policy_version 64550 (0.0007) [2023-10-07 22:23:46,384][67838] Updated weights for policy 0, policy_version 64472 (0.0009) [2023-10-07 22:23:46,739][67871] Updated weights for policy 1, policy_version 64560 (0.0007) [2023-10-07 22:23:47,099][67871] Updated weights for policy 1, policy_version 64570 (0.0008) [2023-10-07 22:23:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 132153344. Throughput: 0: 1643.7, 1: 1659.6. Samples: 33040008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:23:47,478][66916] Avg episode reward: [(0, '40.690'), (1, '59.110')] [2023-10-07 22:23:50,677][67838] Updated weights for policy 0, policy_version 64482 (0.0008) [2023-10-07 22:23:51,066][67838] Updated weights for policy 0, policy_version 64492 (0.0009) [2023-10-07 22:23:51,409][67871] Updated weights for policy 1, policy_version 64580 (0.0009) [2023-10-07 22:23:51,426][67838] Updated weights for policy 0, policy_version 64502 (0.0007) [2023-10-07 22:23:51,796][67838] Updated weights for policy 0, policy_version 64512 (0.0007) [2023-10-07 22:23:51,815][67871] Updated weights for policy 1, policy_version 64590 (0.0009) [2023-10-07 22:23:52,182][67871] Updated weights for policy 1, policy_version 64600 (0.0011) [2023-10-07 22:23:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 132218880. Throughput: 0: 1645.2, 1: 1643.0. Samples: 33058760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:23:52,477][66916] Avg episode reward: [(0, '39.470'), (1, '57.990')] [2023-10-07 22:23:55,815][67838] Updated weights for policy 0, policy_version 64522 (0.0009) [2023-10-07 22:23:56,194][67838] Updated weights for policy 0, policy_version 64532 (0.0008) [2023-10-07 22:23:56,262][67871] Updated weights for policy 1, policy_version 64610 (0.0009) [2023-10-07 22:23:56,549][67838] Updated weights for policy 0, policy_version 64542 (0.0009) [2023-10-07 22:23:56,616][67871] Updated weights for policy 1, policy_version 64620 (0.0009) [2023-10-07 22:23:56,975][67871] Updated weights for policy 1, policy_version 64630 (0.0010) [2023-10-07 22:23:57,346][67871] Updated weights for policy 1, policy_version 64640 (0.0010) [2023-10-07 22:23:57,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 132284416. Throughput: 0: 1657.8, 1: 1650.4. Samples: 33069740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:23:57,478][66916] Avg episode reward: [(0, '41.260'), (1, '55.230')] [2023-10-07 22:24:00,636][67838] Updated weights for policy 0, policy_version 64552 (0.0009) [2023-10-07 22:24:01,012][67838] Updated weights for policy 0, policy_version 64562 (0.0007) [2023-10-07 22:24:01,384][67838] Updated weights for policy 0, policy_version 64572 (0.0008) [2023-10-07 22:24:01,562][67871] Updated weights for policy 1, policy_version 64650 (0.0008) [2023-10-07 22:24:01,921][67871] Updated weights for policy 1, policy_version 64660 (0.0008) [2023-10-07 22:24:02,283][67871] Updated weights for policy 1, policy_version 64670 (0.0009) [2023-10-07 22:24:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 132349952. Throughput: 0: 1646.7, 1: 1654.0. Samples: 33089452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:24:02,477][66916] Avg episode reward: [(0, '42.120'), (1, '53.660')] [2023-10-07 22:24:05,559][67838] Updated weights for policy 0, policy_version 64582 (0.0008) [2023-10-07 22:24:05,941][67838] Updated weights for policy 0, policy_version 64592 (0.0007) [2023-10-07 22:24:06,307][67838] Updated weights for policy 0, policy_version 64602 (0.0008) [2023-10-07 22:24:06,428][67871] Updated weights for policy 1, policy_version 64680 (0.0007) [2023-10-07 22:24:06,797][67871] Updated weights for policy 1, policy_version 64690 (0.0007) [2023-10-07 22:24:07,166][67871] Updated weights for policy 1, policy_version 64700 (0.0009) [2023-10-07 22:24:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 132415488. Throughput: 0: 1661.2, 1: 1646.0. Samples: 33108682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:24:07,477][66916] Avg episode reward: [(0, '43.300'), (1, '51.970')] [2023-10-07 22:24:10,382][67838] Updated weights for policy 0, policy_version 64612 (0.0007) [2023-10-07 22:24:10,760][67838] Updated weights for policy 0, policy_version 64622 (0.0009) [2023-10-07 22:24:11,131][67838] Updated weights for policy 0, policy_version 64632 (0.0007) [2023-10-07 22:24:11,325][67871] Updated weights for policy 1, policy_version 64710 (0.0009) [2023-10-07 22:24:11,699][67871] Updated weights for policy 1, policy_version 64720 (0.0008) [2023-10-07 22:24:12,071][67871] Updated weights for policy 1, policy_version 64730 (0.0007) [2023-10-07 22:24:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 132481024. Throughput: 0: 1662.0, 1: 1648.7. Samples: 33119654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:24:12,477][66916] Avg episode reward: [(0, '43.900'), (1, '52.570')] [2023-10-07 22:24:15,228][67838] Updated weights for policy 0, policy_version 64642 (0.0008) [2023-10-07 22:24:15,595][67838] Updated weights for policy 0, policy_version 64652 (0.0009) [2023-10-07 22:24:15,964][67838] Updated weights for policy 0, policy_version 64662 (0.0009) [2023-10-07 22:24:16,203][67871] Updated weights for policy 1, policy_version 64740 (0.0008) [2023-10-07 22:24:16,345][67838] Updated weights for policy 0, policy_version 64672 (0.0010) [2023-10-07 22:24:16,564][67871] Updated weights for policy 1, policy_version 64750 (0.0009) [2023-10-07 22:24:16,927][67871] Updated weights for policy 1, policy_version 64760 (0.0007) [2023-10-07 22:24:17,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 132546560. Throughput: 0: 1650.9, 1: 1650.6. Samples: 33139172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:24:17,477][66916] Avg episode reward: [(0, '43.240'), (1, '53.070')] [2023-10-07 22:24:20,316][67838] Updated weights for policy 0, policy_version 64682 (0.0009) [2023-10-07 22:24:20,692][67838] Updated weights for policy 0, policy_version 64692 (0.0008) [2023-10-07 22:24:21,015][67871] Updated weights for policy 1, policy_version 64770 (0.0007) [2023-10-07 22:24:21,068][67838] Updated weights for policy 0, policy_version 64702 (0.0008) [2023-10-07 22:24:21,374][67871] Updated weights for policy 1, policy_version 64780 (0.0007) [2023-10-07 22:24:21,738][67871] Updated weights for policy 1, policy_version 64790 (0.0008) [2023-10-07 22:24:22,106][67871] Updated weights for policy 1, policy_version 64800 (0.0010) [2023-10-07 22:24:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 132612096. Throughput: 0: 1669.3, 1: 1642.8. Samples: 33158548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:24:22,477][66916] Avg episode reward: [(0, '41.400'), (1, '55.570')] [2023-10-07 22:24:25,332][67838] Updated weights for policy 0, policy_version 64712 (0.0010) [2023-10-07 22:24:25,702][67838] Updated weights for policy 0, policy_version 64722 (0.0008) [2023-10-07 22:24:26,068][67838] Updated weights for policy 0, policy_version 64732 (0.0007) [2023-10-07 22:24:26,213][67871] Updated weights for policy 1, policy_version 64810 (0.0007) [2023-10-07 22:24:26,582][67871] Updated weights for policy 1, policy_version 64820 (0.0008) [2023-10-07 22:24:26,956][67871] Updated weights for policy 1, policy_version 64830 (0.0007) [2023-10-07 22:24:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 132677632. Throughput: 0: 1666.9, 1: 1653.3. Samples: 33169678. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:24:27,477][66916] Avg episode reward: [(0, '44.040'), (1, '57.700')] [2023-10-07 22:24:30,063][67838] Updated weights for policy 0, policy_version 64742 (0.0009) [2023-10-07 22:24:30,428][67838] Updated weights for policy 0, policy_version 64752 (0.0008) [2023-10-07 22:24:30,805][67838] Updated weights for policy 0, policy_version 64762 (0.0011) [2023-10-07 22:24:31,037][67871] Updated weights for policy 1, policy_version 64840 (0.0007) [2023-10-07 22:24:31,401][67871] Updated weights for policy 1, policy_version 64850 (0.0008) [2023-10-07 22:24:31,772][67871] Updated weights for policy 1, policy_version 64860 (0.0008) [2023-10-07 22:24:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 132743168. Throughput: 0: 1651.7, 1: 1655.4. Samples: 33188828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:24:32,477][66916] Avg episode reward: [(0, '46.700'), (1, '58.580')] [2023-10-07 22:24:34,775][67838] Updated weights for policy 0, policy_version 64772 (0.0009) [2023-10-07 22:24:35,149][67838] Updated weights for policy 0, policy_version 64782 (0.0010) [2023-10-07 22:24:35,533][67838] Updated weights for policy 0, policy_version 64792 (0.0010) [2023-10-07 22:24:35,787][67871] Updated weights for policy 1, policy_version 64870 (0.0010) [2023-10-07 22:24:36,151][67871] Updated weights for policy 1, policy_version 64880 (0.0009) [2023-10-07 22:24:36,515][67871] Updated weights for policy 1, policy_version 64890 (0.0008) [2023-10-07 22:24:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132808704. Throughput: 0: 1679.2, 1: 1647.6. Samples: 33208464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:24:37,477][66916] Avg episode reward: [(0, '45.300'), (1, '59.930')] [2023-10-07 22:24:39,565][67838] Updated weights for policy 0, policy_version 64802 (0.0010) [2023-10-07 22:24:39,961][67838] Updated weights for policy 0, policy_version 64812 (0.0008) [2023-10-07 22:24:40,330][67838] Updated weights for policy 0, policy_version 64822 (0.0007) [2023-10-07 22:24:40,704][67838] Updated weights for policy 0, policy_version 64832 (0.0008) [2023-10-07 22:24:40,847][67871] Updated weights for policy 1, policy_version 64900 (0.0008) [2023-10-07 22:24:41,238][67871] Updated weights for policy 1, policy_version 64910 (0.0007) [2023-10-07 22:24:41,611][67871] Updated weights for policy 1, policy_version 64920 (0.0007) [2023-10-07 22:24:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132874240. Throughput: 0: 1660.4, 1: 1663.3. Samples: 33219302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:24:42,477][66916] Avg episode reward: [(0, '47.480'), (1, '62.060')] [2023-10-07 22:24:42,478][67676] Saving new best policy, reward=62.060! [2023-10-07 22:24:44,730][67838] Updated weights for policy 0, policy_version 64842 (0.0009) [2023-10-07 22:24:45,115][67838] Updated weights for policy 0, policy_version 64852 (0.0008) [2023-10-07 22:24:45,479][67838] Updated weights for policy 0, policy_version 64862 (0.0009) [2023-10-07 22:24:45,693][67871] Updated weights for policy 1, policy_version 64930 (0.0009) [2023-10-07 22:24:46,055][67871] Updated weights for policy 1, policy_version 64940 (0.0010) [2023-10-07 22:24:46,425][67871] Updated weights for policy 1, policy_version 64950 (0.0010) [2023-10-07 22:24:46,786][67871] Updated weights for policy 1, policy_version 64960 (0.0009) [2023-10-07 22:24:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132939776. Throughput: 0: 1663.4, 1: 1659.9. Samples: 33239002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:24:47,478][66916] Avg episode reward: [(0, '43.810'), (1, '59.400')] [2023-10-07 22:24:49,646][67838] Updated weights for policy 0, policy_version 64872 (0.0009) [2023-10-07 22:24:50,024][67838] Updated weights for policy 0, policy_version 64882 (0.0007) [2023-10-07 22:24:50,398][67838] Updated weights for policy 0, policy_version 64892 (0.0008) [2023-10-07 22:24:51,085][67871] Updated weights for policy 1, policy_version 64970 (0.0011) [2023-10-07 22:24:51,452][67871] Updated weights for policy 1, policy_version 64980 (0.0007) [2023-10-07 22:24:51,822][67871] Updated weights for policy 1, policy_version 64990 (0.0007) [2023-10-07 22:24:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133005312. Throughput: 0: 1676.8, 1: 1651.0. Samples: 33258430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:24:52,477][66916] Avg episode reward: [(0, '38.840'), (1, '60.600')] [2023-10-07 22:24:54,667][67838] Updated weights for policy 0, policy_version 64902 (0.0009) [2023-10-07 22:24:55,037][67838] Updated weights for policy 0, policy_version 64912 (0.0008) [2023-10-07 22:24:55,401][67838] Updated weights for policy 0, policy_version 64922 (0.0009) [2023-10-07 22:24:56,010][67871] Updated weights for policy 1, policy_version 65000 (0.0009) [2023-10-07 22:24:56,370][67871] Updated weights for policy 1, policy_version 65010 (0.0010) [2023-10-07 22:24:56,734][67871] Updated weights for policy 1, policy_version 65020 (0.0008) [2023-10-07 22:24:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133070848. Throughput: 0: 1656.4, 1: 1662.2. Samples: 33268988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:24:57,477][66916] Avg episode reward: [(0, '44.120'), (1, '59.370')] [2023-10-07 22:24:59,733][67838] Updated weights for policy 0, policy_version 64932 (0.0009) [2023-10-07 22:25:00,103][67838] Updated weights for policy 0, policy_version 64942 (0.0009) [2023-10-07 22:25:00,470][67838] Updated weights for policy 0, policy_version 64952 (0.0009) [2023-10-07 22:25:00,766][67871] Updated weights for policy 1, policy_version 65030 (0.0008) [2023-10-07 22:25:01,123][67871] Updated weights for policy 1, policy_version 65040 (0.0009) [2023-10-07 22:25:01,491][67871] Updated weights for policy 1, policy_version 65050 (0.0011) [2023-10-07 22:25:02,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 133136384. Throughput: 0: 1664.6, 1: 1659.7. Samples: 33288766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:25:02,478][66916] Avg episode reward: [(0, '41.050'), (1, '55.700')] [2023-10-07 22:25:04,363][67838] Updated weights for policy 0, policy_version 64962 (0.0008) [2023-10-07 22:25:04,727][67838] Updated weights for policy 0, policy_version 64972 (0.0007) [2023-10-07 22:25:05,108][67838] Updated weights for policy 0, policy_version 64982 (0.0011) [2023-10-07 22:25:05,471][67838] Updated weights for policy 0, policy_version 64992 (0.0008) [2023-10-07 22:25:05,721][67871] Updated weights for policy 1, policy_version 65060 (0.0007) [2023-10-07 22:25:06,088][67871] Updated weights for policy 1, policy_version 65070 (0.0007) [2023-10-07 22:25:06,457][67871] Updated weights for policy 1, policy_version 65080 (0.0007) [2023-10-07 22:25:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133201920. Throughput: 0: 1674.8, 1: 1658.4. Samples: 33308544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:25:07,477][66916] Avg episode reward: [(0, '36.420'), (1, '54.630')] [2023-10-07 22:25:07,484][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000065088_66650112.pth... [2023-10-07 22:25:07,485][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000064992_66551808.pth... [2023-10-07 22:25:07,524][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000063520_65044480.pth [2023-10-07 22:25:07,528][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000063456_64978944.pth [2023-10-07 22:25:09,480][67838] Updated weights for policy 0, policy_version 65002 (0.0008) [2023-10-07 22:25:09,853][67838] Updated weights for policy 0, policy_version 65012 (0.0007) [2023-10-07 22:25:10,227][67838] Updated weights for policy 0, policy_version 65022 (0.0007) [2023-10-07 22:25:10,281][67871] Updated weights for policy 1, policy_version 65090 (0.0007) [2023-10-07 22:25:10,639][67871] Updated weights for policy 1, policy_version 65100 (0.0009) [2023-10-07 22:25:11,013][67871] Updated weights for policy 1, policy_version 65110 (0.0009) [2023-10-07 22:25:11,376][67871] Updated weights for policy 1, policy_version 65120 (0.0009) [2023-10-07 22:25:12,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 133267456. Throughput: 0: 1656.3, 1: 1667.6. Samples: 33319256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:25:12,478][66916] Avg episode reward: [(0, '41.340'), (1, '52.920')] [2023-10-07 22:25:14,314][67838] Updated weights for policy 0, policy_version 65032 (0.0008) [2023-10-07 22:25:14,693][67838] Updated weights for policy 0, policy_version 65042 (0.0009) [2023-10-07 22:25:15,055][67838] Updated weights for policy 0, policy_version 65052 (0.0007) [2023-10-07 22:25:15,484][67871] Updated weights for policy 1, policy_version 65130 (0.0008) [2023-10-07 22:25:15,842][67871] Updated weights for policy 1, policy_version 65140 (0.0009) [2023-10-07 22:25:16,207][67871] Updated weights for policy 1, policy_version 65150 (0.0008) [2023-10-07 22:25:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 133332992. Throughput: 0: 1676.5, 1: 1654.0. Samples: 33338700. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-07 22:25:17,478][66916] Avg episode reward: [(0, '41.050'), (1, '55.270')] [2023-10-07 22:25:19,054][67838] Updated weights for policy 0, policy_version 65062 (0.0008) [2023-10-07 22:25:19,429][67838] Updated weights for policy 0, policy_version 65072 (0.0010) [2023-10-07 22:25:19,810][67838] Updated weights for policy 0, policy_version 65082 (0.0009) [2023-10-07 22:25:20,159][67871] Updated weights for policy 1, policy_version 65160 (0.0008) [2023-10-07 22:25:20,524][67871] Updated weights for policy 1, policy_version 65170 (0.0008) [2023-10-07 22:25:20,883][67871] Updated weights for policy 1, policy_version 65180 (0.0009) [2023-10-07 22:25:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133398528. Throughput: 0: 1668.5, 1: 1667.1. Samples: 33358564. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-07 22:25:22,477][66916] Avg episode reward: [(0, '44.660'), (1, '54.370')] [2023-10-07 22:25:23,826][67838] Updated weights for policy 0, policy_version 65092 (0.0008) [2023-10-07 22:25:24,203][67838] Updated weights for policy 0, policy_version 65102 (0.0009) [2023-10-07 22:25:24,566][67838] Updated weights for policy 0, policy_version 65112 (0.0009) [2023-10-07 22:25:25,168][67871] Updated weights for policy 1, policy_version 65190 (0.0008) [2023-10-07 22:25:25,535][67871] Updated weights for policy 1, policy_version 65200 (0.0009) [2023-10-07 22:25:25,912][67871] Updated weights for policy 1, policy_version 65210 (0.0009) [2023-10-07 22:25:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133464064. Throughput: 0: 1655.2, 1: 1668.3. Samples: 33368862. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-07 22:25:27,477][66916] Avg episode reward: [(0, '45.910'), (1, '55.810')] [2023-10-07 22:25:28,873][67838] Updated weights for policy 0, policy_version 65122 (0.0008) [2023-10-07 22:25:29,287][67838] Updated weights for policy 0, policy_version 65132 (0.0008) [2023-10-07 22:25:29,666][67838] Updated weights for policy 0, policy_version 65142 (0.0007) [2023-10-07 22:25:30,040][67838] Updated weights for policy 0, policy_version 65152 (0.0007) [2023-10-07 22:25:30,115][67871] Updated weights for policy 1, policy_version 65220 (0.0009) [2023-10-07 22:25:30,509][67871] Updated weights for policy 1, policy_version 65230 (0.0008) [2023-10-07 22:25:30,881][67871] Updated weights for policy 1, policy_version 65240 (0.0008) [2023-10-07 22:25:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133529600. Throughput: 0: 1665.4, 1: 1650.6. Samples: 33388220. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-07 22:25:32,478][66916] Avg episode reward: [(0, '45.500'), (1, '56.890')] [2023-10-07 22:25:33,974][67838] Updated weights for policy 0, policy_version 65162 (0.0008) [2023-10-07 22:25:34,355][67838] Updated weights for policy 0, policy_version 65172 (0.0009) [2023-10-07 22:25:34,715][67838] Updated weights for policy 0, policy_version 65182 (0.0009) [2023-10-07 22:25:34,939][67871] Updated weights for policy 1, policy_version 65250 (0.0010) [2023-10-07 22:25:35,305][67871] Updated weights for policy 1, policy_version 65260 (0.0009) [2023-10-07 22:25:35,663][67871] Updated weights for policy 1, policy_version 65270 (0.0009) [2023-10-07 22:25:36,032][67871] Updated weights for policy 1, policy_version 65280 (0.0009) [2023-10-07 22:25:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133595136. Throughput: 0: 1665.6, 1: 1667.1. Samples: 33408402. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-07 22:25:37,478][66916] Avg episode reward: [(0, '50.800'), (1, '59.340')] [2023-10-07 22:25:38,875][67838] Updated weights for policy 0, policy_version 65192 (0.0010) [2023-10-07 22:25:39,256][67838] Updated weights for policy 0, policy_version 65202 (0.0010) [2023-10-07 22:25:39,628][67838] Updated weights for policy 0, policy_version 65212 (0.0009) [2023-10-07 22:25:40,281][67871] Updated weights for policy 1, policy_version 65290 (0.0010) [2023-10-07 22:25:40,648][67871] Updated weights for policy 1, policy_version 65300 (0.0009) [2023-10-07 22:25:41,016][67871] Updated weights for policy 1, policy_version 65310 (0.0010) [2023-10-07 22:25:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133660672. Throughput: 0: 1653.6, 1: 1672.7. Samples: 33418670. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-07 22:25:42,477][66916] Avg episode reward: [(0, '47.050'), (1, '60.250')] [2023-10-07 22:25:43,713][67838] Updated weights for policy 0, policy_version 65222 (0.0009) [2023-10-07 22:25:44,083][67838] Updated weights for policy 0, policy_version 65232 (0.0010) [2023-10-07 22:25:44,458][67838] Updated weights for policy 0, policy_version 65242 (0.0010) [2023-10-07 22:25:45,000][67871] Updated weights for policy 1, policy_version 65320 (0.0009) [2023-10-07 22:25:45,360][67871] Updated weights for policy 1, policy_version 65330 (0.0009) [2023-10-07 22:25:45,728][67871] Updated weights for policy 1, policy_version 65340 (0.0010) [2023-10-07 22:25:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133726208. Throughput: 0: 1668.4, 1: 1646.3. Samples: 33437926. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-07 22:25:47,477][66916] Avg episode reward: [(0, '53.500'), (1, '58.300')] [2023-10-07 22:25:48,722][67838] Updated weights for policy 0, policy_version 65252 (0.0009) [2023-10-07 22:25:49,098][67838] Updated weights for policy 0, policy_version 65262 (0.0010) [2023-10-07 22:25:49,469][67838] Updated weights for policy 0, policy_version 65272 (0.0009) [2023-10-07 22:25:50,004][67871] Updated weights for policy 1, policy_version 65350 (0.0009) [2023-10-07 22:25:50,372][67871] Updated weights for policy 1, policy_version 65360 (0.0008) [2023-10-07 22:25:50,740][67871] Updated weights for policy 1, policy_version 65370 (0.0010) [2023-10-07 22:25:52,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 133791744. Throughput: 0: 1655.9, 1: 1660.7. Samples: 33457792. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-07 22:25:52,478][66916] Avg episode reward: [(0, '50.170'), (1, '58.790')] [2023-10-07 22:25:53,835][67838] Updated weights for policy 0, policy_version 65282 (0.0010) [2023-10-07 22:25:54,206][67838] Updated weights for policy 0, policy_version 65292 (0.0010) [2023-10-07 22:25:54,588][67838] Updated weights for policy 0, policy_version 65302 (0.0010) [2023-10-07 22:25:54,880][67871] Updated weights for policy 1, policy_version 65380 (0.0009) [2023-10-07 22:25:54,949][67838] Updated weights for policy 0, policy_version 65312 (0.0008) [2023-10-07 22:25:55,245][67871] Updated weights for policy 1, policy_version 65390 (0.0009) [2023-10-07 22:25:55,613][67871] Updated weights for policy 1, policy_version 65400 (0.0010) [2023-10-07 22:25:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133857280. Throughput: 0: 1646.9, 1: 1652.7. Samples: 33467738. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-07 22:25:57,477][66916] Avg episode reward: [(0, '47.300'), (1, '60.040')] [2023-10-07 22:25:59,220][67838] Updated weights for policy 0, policy_version 65322 (0.0008) [2023-10-07 22:25:59,588][67838] Updated weights for policy 0, policy_version 65332 (0.0008) [2023-10-07 22:25:59,653][67871] Updated weights for policy 1, policy_version 65410 (0.0008) [2023-10-07 22:25:59,955][67838] Updated weights for policy 0, policy_version 65342 (0.0007) [2023-10-07 22:26:00,015][67871] Updated weights for policy 1, policy_version 65420 (0.0008) [2023-10-07 22:26:00,390][67871] Updated weights for policy 1, policy_version 65430 (0.0010) [2023-10-07 22:26:00,752][67871] Updated weights for policy 1, policy_version 65440 (0.0007) [2023-10-07 22:26:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133922816. Throughput: 0: 1649.7, 1: 1636.7. Samples: 33486590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:26:02,478][66916] Avg episode reward: [(0, '47.520'), (1, '56.720')] [2023-10-07 22:26:04,101][67838] Updated weights for policy 0, policy_version 65352 (0.0008) [2023-10-07 22:26:04,470][67838] Updated weights for policy 0, policy_version 65362 (0.0008) [2023-10-07 22:26:04,786][67871] Updated weights for policy 1, policy_version 65450 (0.0008) [2023-10-07 22:26:04,841][67838] Updated weights for policy 0, policy_version 65372 (0.0009) [2023-10-07 22:26:05,157][67871] Updated weights for policy 1, policy_version 65460 (0.0007) [2023-10-07 22:26:05,527][67871] Updated weights for policy 1, policy_version 65470 (0.0011) [2023-10-07 22:26:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133988352. Throughput: 0: 1647.6, 1: 1654.0. Samples: 33507134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:26:07,477][66916] Avg episode reward: [(0, '43.840'), (1, '54.450')] [2023-10-07 22:26:08,990][67838] Updated weights for policy 0, policy_version 65382 (0.0007) [2023-10-07 22:26:09,357][67838] Updated weights for policy 0, policy_version 65392 (0.0010) [2023-10-07 22:26:09,724][67838] Updated weights for policy 0, policy_version 65402 (0.0009) [2023-10-07 22:26:09,772][67871] Updated weights for policy 1, policy_version 65480 (0.0008) [2023-10-07 22:26:10,143][67871] Updated weights for policy 1, policy_version 65490 (0.0007) [2023-10-07 22:26:10,506][67871] Updated weights for policy 1, policy_version 65500 (0.0009) [2023-10-07 22:26:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134053888. Throughput: 0: 1643.8, 1: 1647.8. Samples: 33516984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:26:12,477][66916] Avg episode reward: [(0, '44.280'), (1, '54.940')] [2023-10-07 22:26:13,714][67838] Updated weights for policy 0, policy_version 65412 (0.0009) [2023-10-07 22:26:14,082][67838] Updated weights for policy 0, policy_version 65422 (0.0008) [2023-10-07 22:26:14,449][67838] Updated weights for policy 0, policy_version 65432 (0.0007) [2023-10-07 22:26:14,570][67871] Updated weights for policy 1, policy_version 65510 (0.0008) [2023-10-07 22:26:14,940][67871] Updated weights for policy 1, policy_version 65520 (0.0009) [2023-10-07 22:26:15,309][67871] Updated weights for policy 1, policy_version 65530 (0.0008) [2023-10-07 22:26:17,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134119424. Throughput: 0: 1647.1, 1: 1652.5. Samples: 33536702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:26:17,477][66916] Avg episode reward: [(0, '43.210'), (1, '55.950')] [2023-10-07 22:26:18,723][67838] Updated weights for policy 0, policy_version 65442 (0.0008) [2023-10-07 22:26:19,136][67838] Updated weights for policy 0, policy_version 65452 (0.0010) [2023-10-07 22:26:19,503][67838] Updated weights for policy 0, policy_version 65462 (0.0009) [2023-10-07 22:26:19,582][67871] Updated weights for policy 1, policy_version 65540 (0.0008) [2023-10-07 22:26:19,868][67838] Updated weights for policy 0, policy_version 65472 (0.0008) [2023-10-07 22:26:19,968][67871] Updated weights for policy 1, policy_version 65550 (0.0009) [2023-10-07 22:26:20,339][67871] Updated weights for policy 1, policy_version 65560 (0.0007) [2023-10-07 22:26:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134184960. Throughput: 0: 1641.5, 1: 1662.7. Samples: 33557090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:26:22,477][66916] Avg episode reward: [(0, '48.400'), (1, '53.860')] [2023-10-07 22:26:23,931][67838] Updated weights for policy 0, policy_version 65482 (0.0009) [2023-10-07 22:26:24,300][67838] Updated weights for policy 0, policy_version 65492 (0.0008) [2023-10-07 22:26:24,354][67871] Updated weights for policy 1, policy_version 65570 (0.0009) [2023-10-07 22:26:24,680][67838] Updated weights for policy 0, policy_version 65502 (0.0009) [2023-10-07 22:26:24,717][67871] Updated weights for policy 1, policy_version 65580 (0.0010) [2023-10-07 22:26:25,090][67871] Updated weights for policy 1, policy_version 65590 (0.0007) [2023-10-07 22:26:25,460][67871] Updated weights for policy 1, policy_version 65600 (0.0009) [2023-10-07 22:26:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134250496. Throughput: 0: 1644.2, 1: 1647.7. Samples: 33566808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:26:27,478][66916] Avg episode reward: [(0, '46.880'), (1, '54.100')] [2023-10-07 22:26:28,634][67838] Updated weights for policy 0, policy_version 65512 (0.0007) [2023-10-07 22:26:29,007][67838] Updated weights for policy 0, policy_version 65522 (0.0008) [2023-10-07 22:26:29,385][67838] Updated weights for policy 0, policy_version 65532 (0.0009) [2023-10-07 22:26:29,693][67871] Updated weights for policy 1, policy_version 65610 (0.0008) [2023-10-07 22:26:30,064][67871] Updated weights for policy 1, policy_version 65620 (0.0007) [2023-10-07 22:26:30,429][67871] Updated weights for policy 1, policy_version 65630 (0.0009) [2023-10-07 22:26:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134316032. Throughput: 0: 1645.8, 1: 1660.5. Samples: 33586710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:26:32,478][66916] Avg episode reward: [(0, '49.260'), (1, '54.100')] [2023-10-07 22:26:33,548][67838] Updated weights for policy 0, policy_version 65542 (0.0008) [2023-10-07 22:26:33,925][67838] Updated weights for policy 0, policy_version 65552 (0.0009) [2023-10-07 22:26:34,297][67838] Updated weights for policy 0, policy_version 65562 (0.0007) [2023-10-07 22:26:34,680][67871] Updated weights for policy 1, policy_version 65640 (0.0007) [2023-10-07 22:26:35,046][67871] Updated weights for policy 1, policy_version 65650 (0.0009) [2023-10-07 22:26:35,412][67871] Updated weights for policy 1, policy_version 65660 (0.0007) [2023-10-07 22:26:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 134381568. Throughput: 0: 1661.0, 1: 1666.7. Samples: 33607538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:26:37,477][66916] Avg episode reward: [(0, '50.810'), (1, '53.820')] [2023-10-07 22:26:38,279][67838] Updated weights for policy 0, policy_version 65572 (0.0007) [2023-10-07 22:26:38,652][67838] Updated weights for policy 0, policy_version 65582 (0.0007) [2023-10-07 22:26:39,017][67838] Updated weights for policy 0, policy_version 65592 (0.0009) [2023-10-07 22:26:39,364][67871] Updated weights for policy 1, policy_version 65670 (0.0008) [2023-10-07 22:26:39,727][67871] Updated weights for policy 1, policy_version 65680 (0.0011) [2023-10-07 22:26:40,107][67871] Updated weights for policy 1, policy_version 65690 (0.0011) [2023-10-07 22:26:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134447104. Throughput: 0: 1664.9, 1: 1655.5. Samples: 33617156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:26:42,478][66916] Avg episode reward: [(0, '49.830'), (1, '55.440')] [2023-10-07 22:26:43,083][67838] Updated weights for policy 0, policy_version 65602 (0.0007) [2023-10-07 22:26:43,460][67838] Updated weights for policy 0, policy_version 65612 (0.0008) [2023-10-07 22:26:43,834][67838] Updated weights for policy 0, policy_version 65622 (0.0008) [2023-10-07 22:26:44,137][67871] Updated weights for policy 1, policy_version 65700 (0.0010) [2023-10-07 22:26:44,200][67838] Updated weights for policy 0, policy_version 65632 (0.0009) [2023-10-07 22:26:44,494][67871] Updated weights for policy 1, policy_version 65710 (0.0008) [2023-10-07 22:26:44,866][67871] Updated weights for policy 1, policy_version 65720 (0.0008) [2023-10-07 22:26:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134512640. Throughput: 0: 1671.2, 1: 1678.6. Samples: 33637330. Policy #0 lag: (min: 29.0, avg: 53.5, max: 56.0) [2023-10-07 22:26:47,478][66916] Avg episode reward: [(0, '52.380'), (1, '52.610')] [2023-10-07 22:26:48,200][67838] Updated weights for policy 0, policy_version 65642 (0.0008) [2023-10-07 22:26:48,574][67838] Updated weights for policy 0, policy_version 65652 (0.0009) [2023-10-07 22:26:48,823][67871] Updated weights for policy 1, policy_version 65730 (0.0010) [2023-10-07 22:26:48,940][67838] Updated weights for policy 0, policy_version 65662 (0.0008) [2023-10-07 22:26:49,192][67871] Updated weights for policy 1, policy_version 65740 (0.0009) [2023-10-07 22:26:49,551][67871] Updated weights for policy 1, policy_version 65750 (0.0007) [2023-10-07 22:26:49,921][67871] Updated weights for policy 1, policy_version 65760 (0.0008) [2023-10-07 22:26:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134578176. Throughput: 0: 1671.6, 1: 1675.0. Samples: 33657732. Policy #0 lag: (min: 29.0, avg: 53.5, max: 56.0) [2023-10-07 22:26:52,478][66916] Avg episode reward: [(0, '51.110'), (1, '56.490')] [2023-10-07 22:26:53,082][67838] Updated weights for policy 0, policy_version 65672 (0.0007) [2023-10-07 22:26:53,452][67838] Updated weights for policy 0, policy_version 65682 (0.0007) [2023-10-07 22:26:53,835][67838] Updated weights for policy 0, policy_version 65692 (0.0009) [2023-10-07 22:26:54,127][67871] Updated weights for policy 1, policy_version 65770 (0.0007) [2023-10-07 22:26:54,500][67871] Updated weights for policy 1, policy_version 65780 (0.0008) [2023-10-07 22:26:54,864][67871] Updated weights for policy 1, policy_version 65790 (0.0008) [2023-10-07 22:26:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134643712. Throughput: 0: 1673.3, 1: 1656.4. Samples: 33666824. Policy #0 lag: (min: 29.0, avg: 53.5, max: 56.0) [2023-10-07 22:26:57,477][66916] Avg episode reward: [(0, '52.610'), (1, '57.210')] [2023-10-07 22:26:57,878][67838] Updated weights for policy 0, policy_version 65702 (0.0008) [2023-10-07 22:26:58,245][67838] Updated weights for policy 0, policy_version 65712 (0.0008) [2023-10-07 22:26:58,623][67838] Updated weights for policy 0, policy_version 65722 (0.0007) [2023-10-07 22:26:58,997][67871] Updated weights for policy 1, policy_version 65800 (0.0008) [2023-10-07 22:26:59,360][67871] Updated weights for policy 1, policy_version 65810 (0.0010) [2023-10-07 22:26:59,730][67871] Updated weights for policy 1, policy_version 65820 (0.0011) [2023-10-07 22:27:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 134709248. Throughput: 0: 1676.3, 1: 1666.6. Samples: 33687132. Policy #0 lag: (min: 29.0, avg: 53.5, max: 56.0) [2023-10-07 22:27:02,478][66916] Avg episode reward: [(0, '49.180'), (1, '56.810')] [2023-10-07 22:27:02,726][67838] Updated weights for policy 0, policy_version 65732 (0.0009) [2023-10-07 22:27:03,112][67838] Updated weights for policy 0, policy_version 65742 (0.0009) [2023-10-07 22:27:03,487][67838] Updated weights for policy 0, policy_version 65752 (0.0008) [2023-10-07 22:27:03,997][67871] Updated weights for policy 1, policy_version 65830 (0.0009) [2023-10-07 22:27:04,377][67871] Updated weights for policy 1, policy_version 65840 (0.0008) [2023-10-07 22:27:04,751][67871] Updated weights for policy 1, policy_version 65850 (0.0009) [2023-10-07 22:27:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 134774784. Throughput: 0: 1681.8, 1: 1661.6. Samples: 33707542. Policy #0 lag: (min: 29.0, avg: 53.5, max: 56.0) [2023-10-07 22:27:07,478][66916] Avg episode reward: [(0, '51.740'), (1, '58.000')] [2023-10-07 22:27:07,488][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000065856_67436544.pth... [2023-10-07 22:27:07,529][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000064320_65863680.pth [2023-10-07 22:27:07,619][67838] Updated weights for policy 0, policy_version 65762 (0.0009) [2023-10-07 22:27:08,016][67838] Updated weights for policy 0, policy_version 65772 (0.0008) [2023-10-07 22:27:08,391][67838] Updated weights for policy 0, policy_version 65782 (0.0008) [2023-10-07 22:27:08,759][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000065792_67371008.pth... [2023-10-07 22:27:08,759][67838] Updated weights for policy 0, policy_version 65792 (0.0007) [2023-10-07 22:27:08,796][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000064224_65765376.pth [2023-10-07 22:27:08,831][67871] Updated weights for policy 1, policy_version 65860 (0.0008) [2023-10-07 22:27:09,219][67871] Updated weights for policy 1, policy_version 65870 (0.0010) [2023-10-07 22:27:09,589][67871] Updated weights for policy 1, policy_version 65880 (0.0011) [2023-10-07 22:27:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 134840320. Throughput: 0: 1683.9, 1: 1648.0. Samples: 33716742. Policy #0 lag: (min: 29.0, avg: 53.5, max: 56.0) [2023-10-07 22:27:12,478][66916] Avg episode reward: [(0, '49.980'), (1, '57.690')] [2023-10-07 22:27:12,670][67838] Updated weights for policy 0, policy_version 65802 (0.0008) [2023-10-07 22:27:13,043][67838] Updated weights for policy 0, policy_version 65812 (0.0009) [2023-10-07 22:27:13,410][67838] Updated weights for policy 0, policy_version 65822 (0.0008) [2023-10-07 22:27:13,669][67871] Updated weights for policy 1, policy_version 65890 (0.0010) [2023-10-07 22:27:14,035][67871] Updated weights for policy 1, policy_version 65900 (0.0010) [2023-10-07 22:27:14,411][67871] Updated weights for policy 1, policy_version 65910 (0.0008) [2023-10-07 22:27:14,783][67871] Updated weights for policy 1, policy_version 65920 (0.0008) [2023-10-07 22:27:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134905856. Throughput: 0: 1676.6, 1: 1662.0. Samples: 33736946. Policy #0 lag: (min: 29.0, avg: 53.5, max: 56.0) [2023-10-07 22:27:17,477][66916] Avg episode reward: [(0, '49.810'), (1, '53.660')] [2023-10-07 22:27:17,637][67838] Updated weights for policy 0, policy_version 65832 (0.0010) [2023-10-07 22:27:18,016][67838] Updated weights for policy 0, policy_version 65842 (0.0010) [2023-10-07 22:27:18,389][67838] Updated weights for policy 0, policy_version 65852 (0.0008) [2023-10-07 22:27:18,933][67871] Updated weights for policy 1, policy_version 65930 (0.0007) [2023-10-07 22:27:19,293][67871] Updated weights for policy 1, policy_version 65940 (0.0008) [2023-10-07 22:27:19,661][67871] Updated weights for policy 1, policy_version 65950 (0.0010) [2023-10-07 22:27:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 134971392. Throughput: 0: 1670.4, 1: 1661.1. Samples: 33757456. Policy #0 lag: (min: 29.0, avg: 53.5, max: 56.0) [2023-10-07 22:27:22,478][66916] Avg episode reward: [(0, '48.030'), (1, '51.290')] [2023-10-07 22:27:22,586][67838] Updated weights for policy 0, policy_version 65862 (0.0009) [2023-10-07 22:27:22,958][67838] Updated weights for policy 0, policy_version 65872 (0.0007) [2023-10-07 22:27:23,331][67838] Updated weights for policy 0, policy_version 65882 (0.0007) [2023-10-07 22:27:23,887][67871] Updated weights for policy 1, policy_version 65960 (0.0009) [2023-10-07 22:27:24,255][67871] Updated weights for policy 1, policy_version 65970 (0.0007) [2023-10-07 22:27:24,621][67871] Updated weights for policy 1, policy_version 65980 (0.0008) [2023-10-07 22:27:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 135036928. Throughput: 0: 1666.8, 1: 1648.5. Samples: 33766344. Policy #0 lag: (min: 29.0, avg: 53.5, max: 56.0) [2023-10-07 22:27:27,477][66916] Avg episode reward: [(0, '49.300'), (1, '50.900')] [2023-10-07 22:27:27,479][67838] Updated weights for policy 0, policy_version 65892 (0.0010) [2023-10-07 22:27:27,846][67838] Updated weights for policy 0, policy_version 65902 (0.0009) [2023-10-07 22:27:28,225][67838] Updated weights for policy 0, policy_version 65912 (0.0008) [2023-10-07 22:27:28,834][67871] Updated weights for policy 1, policy_version 65990 (0.0010) [2023-10-07 22:27:29,199][67871] Updated weights for policy 1, policy_version 66000 (0.0007) [2023-10-07 22:27:29,564][67871] Updated weights for policy 1, policy_version 66010 (0.0008) [2023-10-07 22:27:32,375][67838] Updated weights for policy 0, policy_version 65922 (0.0011) [2023-10-07 22:27:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 135102464. Throughput: 0: 1661.9, 1: 1651.4. Samples: 33786426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:27:32,478][66916] Avg episode reward: [(0, '46.830'), (1, '49.380')] [2023-10-07 22:27:32,746][67838] Updated weights for policy 0, policy_version 65932 (0.0012) [2023-10-07 22:27:33,129][67838] Updated weights for policy 0, policy_version 65942 (0.0010) [2023-10-07 22:27:33,509][67838] Updated weights for policy 0, policy_version 65952 (0.0008) [2023-10-07 22:27:33,647][67871] Updated weights for policy 1, policy_version 66020 (0.0009) [2023-10-07 22:27:34,012][67871] Updated weights for policy 1, policy_version 66030 (0.0009) [2023-10-07 22:27:34,377][67871] Updated weights for policy 1, policy_version 66040 (0.0008) [2023-10-07 22:27:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 135168000. Throughput: 0: 1663.6, 1: 1652.2. Samples: 33806942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:27:37,477][66916] Avg episode reward: [(0, '49.850'), (1, '53.250')] [2023-10-07 22:27:37,761][67838] Updated weights for policy 0, policy_version 65962 (0.0010) [2023-10-07 22:27:38,130][67838] Updated weights for policy 0, policy_version 65972 (0.0010) [2023-10-07 22:27:38,320][67871] Updated weights for policy 1, policy_version 66050 (0.0008) [2023-10-07 22:27:38,504][67838] Updated weights for policy 0, policy_version 65982 (0.0007) [2023-10-07 22:27:38,691][67871] Updated weights for policy 1, policy_version 66060 (0.0008) [2023-10-07 22:27:39,051][67871] Updated weights for policy 1, policy_version 66070 (0.0007) [2023-10-07 22:27:39,421][67871] Updated weights for policy 1, policy_version 66080 (0.0008) [2023-10-07 22:27:42,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 135233536. Throughput: 0: 1664.3, 1: 1649.6. Samples: 33815948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:27:42,477][66916] Avg episode reward: [(0, '49.430'), (1, '53.530')] [2023-10-07 22:27:42,686][67838] Updated weights for policy 0, policy_version 65992 (0.0007) [2023-10-07 22:27:43,053][67838] Updated weights for policy 0, policy_version 66002 (0.0007) [2023-10-07 22:27:43,427][67838] Updated weights for policy 0, policy_version 66012 (0.0007) [2023-10-07 22:27:43,619][67871] Updated weights for policy 1, policy_version 66090 (0.0008) [2023-10-07 22:27:43,982][67871] Updated weights for policy 1, policy_version 66100 (0.0008) [2023-10-07 22:27:44,347][67871] Updated weights for policy 1, policy_version 66110 (0.0009) [2023-10-07 22:27:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 135299072. Throughput: 0: 1656.6, 1: 1656.9. Samples: 33836238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:27:47,477][66916] Avg episode reward: [(0, '49.580'), (1, '56.110')] [2023-10-07 22:27:47,504][67838] Updated weights for policy 0, policy_version 66022 (0.0009) [2023-10-07 22:27:47,882][67838] Updated weights for policy 0, policy_version 66032 (0.0009) [2023-10-07 22:27:48,255][67838] Updated weights for policy 0, policy_version 66042 (0.0008) [2023-10-07 22:27:48,464][67871] Updated weights for policy 1, policy_version 66120 (0.0008) [2023-10-07 22:27:48,838][67871] Updated weights for policy 1, policy_version 66130 (0.0008) [2023-10-07 22:27:49,199][67871] Updated weights for policy 1, policy_version 66140 (0.0008) [2023-10-07 22:27:52,428][67838] Updated weights for policy 0, policy_version 66052 (0.0008) [2023-10-07 22:27:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 135364608. Throughput: 0: 1656.4, 1: 1662.1. Samples: 33856876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:27:52,477][66916] Avg episode reward: [(0, '48.890'), (1, '54.080')] [2023-10-07 22:27:52,798][67838] Updated weights for policy 0, policy_version 66062 (0.0007) [2023-10-07 22:27:53,174][67838] Updated weights for policy 0, policy_version 66072 (0.0008) [2023-10-07 22:27:53,458][67871] Updated weights for policy 1, policy_version 66150 (0.0009) [2023-10-07 22:27:53,823][67871] Updated weights for policy 1, policy_version 66160 (0.0009) [2023-10-07 22:27:54,198][67871] Updated weights for policy 1, policy_version 66170 (0.0009) [2023-10-07 22:27:57,376][67838] Updated weights for policy 0, policy_version 66082 (0.0010) [2023-10-07 22:27:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 135430144. Throughput: 0: 1655.4, 1: 1658.6. Samples: 33865872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:27:57,477][66916] Avg episode reward: [(0, '47.590'), (1, '53.600')] [2023-10-07 22:27:57,781][67838] Updated weights for policy 0, policy_version 66092 (0.0007) [2023-10-07 22:27:58,155][67838] Updated weights for policy 0, policy_version 66102 (0.0008) [2023-10-07 22:27:58,448][67871] Updated weights for policy 1, policy_version 66180 (0.0007) [2023-10-07 22:27:58,525][67838] Updated weights for policy 0, policy_version 66112 (0.0009) [2023-10-07 22:27:58,807][67871] Updated weights for policy 1, policy_version 66190 (0.0008) [2023-10-07 22:27:59,177][67871] Updated weights for policy 1, policy_version 66200 (0.0009) [2023-10-07 22:28:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 135495680. Throughput: 0: 1650.4, 1: 1661.8. Samples: 33885992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:28:02,477][66916] Avg episode reward: [(0, '48.360'), (1, '54.310')] [2023-10-07 22:28:02,579][67838] Updated weights for policy 0, policy_version 66122 (0.0010) [2023-10-07 22:28:02,949][67838] Updated weights for policy 0, policy_version 66132 (0.0011) [2023-10-07 22:28:03,213][67871] Updated weights for policy 1, policy_version 66210 (0.0007) [2023-10-07 22:28:03,326][67838] Updated weights for policy 0, policy_version 66142 (0.0010) [2023-10-07 22:28:03,582][67871] Updated weights for policy 1, policy_version 66220 (0.0009) [2023-10-07 22:28:03,952][67871] Updated weights for policy 1, policy_version 66230 (0.0011) [2023-10-07 22:28:04,324][67871] Updated weights for policy 1, policy_version 66240 (0.0008) [2023-10-07 22:28:07,354][67838] Updated weights for policy 0, policy_version 66152 (0.0010) [2023-10-07 22:28:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 135561216. Throughput: 0: 1646.9, 1: 1664.4. Samples: 33906466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:28:07,477][66916] Avg episode reward: [(0, '47.720'), (1, '51.710')] [2023-10-07 22:28:07,722][67838] Updated weights for policy 0, policy_version 66162 (0.0011) [2023-10-07 22:28:08,098][67838] Updated weights for policy 0, policy_version 66172 (0.0010) [2023-10-07 22:28:08,408][67871] Updated weights for policy 1, policy_version 66250 (0.0008) [2023-10-07 22:28:08,772][67871] Updated weights for policy 1, policy_version 66260 (0.0010) [2023-10-07 22:28:09,146][67871] Updated weights for policy 1, policy_version 66270 (0.0010) [2023-10-07 22:28:12,199][67838] Updated weights for policy 0, policy_version 66182 (0.0008) [2023-10-07 22:28:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 135626752. Throughput: 0: 1651.6, 1: 1664.6. Samples: 33915576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:28:12,477][66916] Avg episode reward: [(0, '52.190'), (1, '54.090')] [2023-10-07 22:28:12,568][67838] Updated weights for policy 0, policy_version 66192 (0.0008) [2023-10-07 22:28:12,940][67838] Updated weights for policy 0, policy_version 66202 (0.0009) [2023-10-07 22:28:13,127][67871] Updated weights for policy 1, policy_version 66280 (0.0008) [2023-10-07 22:28:13,491][67871] Updated weights for policy 1, policy_version 66290 (0.0009) [2023-10-07 22:28:13,864][67871] Updated weights for policy 1, policy_version 66300 (0.0008) [2023-10-07 22:28:17,084][67838] Updated weights for policy 0, policy_version 66212 (0.0009) [2023-10-07 22:28:17,452][67838] Updated weights for policy 0, policy_version 66222 (0.0010) [2023-10-07 22:28:17,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 135692288. Throughput: 0: 1653.1, 1: 1667.8. Samples: 33935868. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-10-07 22:28:17,477][66916] Avg episode reward: [(0, '50.840'), (1, '54.070')] [2023-10-07 22:28:17,826][67838] Updated weights for policy 0, policy_version 66232 (0.0007) [2023-10-07 22:28:17,888][67871] Updated weights for policy 1, policy_version 66310 (0.0007) [2023-10-07 22:28:18,261][67871] Updated weights for policy 1, policy_version 66320 (0.0007) [2023-10-07 22:28:18,630][67871] Updated weights for policy 1, policy_version 66330 (0.0007) [2023-10-07 22:28:21,896][67838] Updated weights for policy 0, policy_version 66242 (0.0010) [2023-10-07 22:28:22,274][67838] Updated weights for policy 0, policy_version 66252 (0.0008) [2023-10-07 22:28:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 135757824. Throughput: 0: 1651.2, 1: 1670.2. Samples: 33956406. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-10-07 22:28:22,477][66916] Avg episode reward: [(0, '51.010'), (1, '55.710')] [2023-10-07 22:28:22,648][67838] Updated weights for policy 0, policy_version 66262 (0.0007) [2023-10-07 22:28:22,672][67871] Updated weights for policy 1, policy_version 66340 (0.0007) [2023-10-07 22:28:23,020][67838] Updated weights for policy 0, policy_version 66272 (0.0007) [2023-10-07 22:28:23,039][67871] Updated weights for policy 1, policy_version 66350 (0.0007) [2023-10-07 22:28:23,402][67871] Updated weights for policy 1, policy_version 66360 (0.0007) [2023-10-07 22:28:27,201][67838] Updated weights for policy 0, policy_version 66282 (0.0008) [2023-10-07 22:28:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 135823360. Throughput: 0: 1654.6, 1: 1671.6. Samples: 33965630. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-10-07 22:28:27,478][66916] Avg episode reward: [(0, '51.560'), (1, '50.800')] [2023-10-07 22:28:27,563][67838] Updated weights for policy 0, policy_version 66292 (0.0008) [2023-10-07 22:28:27,585][67871] Updated weights for policy 1, policy_version 66370 (0.0009) [2023-10-07 22:28:27,943][67838] Updated weights for policy 0, policy_version 66302 (0.0008) [2023-10-07 22:28:27,946][67871] Updated weights for policy 1, policy_version 66380 (0.0007) [2023-10-07 22:28:28,300][67871] Updated weights for policy 1, policy_version 66390 (0.0009) [2023-10-07 22:28:28,663][67871] Updated weights for policy 1, policy_version 66400 (0.0008) [2023-10-07 22:28:32,181][67838] Updated weights for policy 0, policy_version 66312 (0.0011) [2023-10-07 22:28:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 135888896. Throughput: 0: 1656.8, 1: 1669.3. Samples: 33985916. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-10-07 22:28:32,477][66916] Avg episode reward: [(0, '49.260'), (1, '50.670')] [2023-10-07 22:28:32,557][67838] Updated weights for policy 0, policy_version 66322 (0.0009) [2023-10-07 22:28:32,912][67871] Updated weights for policy 1, policy_version 66410 (0.0008) [2023-10-07 22:28:32,928][67838] Updated weights for policy 0, policy_version 66332 (0.0008) [2023-10-07 22:28:33,286][67871] Updated weights for policy 1, policy_version 66420 (0.0008) [2023-10-07 22:28:33,651][67871] Updated weights for policy 1, policy_version 66430 (0.0010) [2023-10-07 22:28:36,950][67838] Updated weights for policy 0, policy_version 66342 (0.0008) [2023-10-07 22:28:37,320][67838] Updated weights for policy 0, policy_version 66352 (0.0008) [2023-10-07 22:28:37,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 135954432. Throughput: 0: 1649.5, 1: 1664.3. Samples: 34005996. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-10-07 22:28:37,478][66916] Avg episode reward: [(0, '49.600'), (1, '51.480')] [2023-10-07 22:28:37,699][67838] Updated weights for policy 0, policy_version 66362 (0.0008) [2023-10-07 22:28:37,804][67871] Updated weights for policy 1, policy_version 66440 (0.0008) [2023-10-07 22:28:38,175][67871] Updated weights for policy 1, policy_version 66450 (0.0008) [2023-10-07 22:28:38,532][67871] Updated weights for policy 1, policy_version 66460 (0.0007) [2023-10-07 22:28:42,005][67838] Updated weights for policy 0, policy_version 66372 (0.0007) [2023-10-07 22:28:42,396][67838] Updated weights for policy 0, policy_version 66382 (0.0008) [2023-10-07 22:28:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 136019968. Throughput: 0: 1654.3, 1: 1667.2. Samples: 34015336. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-10-07 22:28:42,478][66916] Avg episode reward: [(0, '48.850'), (1, '54.260')] [2023-10-07 22:28:42,754][67871] Updated weights for policy 1, policy_version 66470 (0.0008) [2023-10-07 22:28:42,769][67838] Updated weights for policy 0, policy_version 66392 (0.0009) [2023-10-07 22:28:43,138][67871] Updated weights for policy 1, policy_version 66480 (0.0007) [2023-10-07 22:28:43,496][67871] Updated weights for policy 1, policy_version 66490 (0.0007) [2023-10-07 22:28:46,739][67838] Updated weights for policy 0, policy_version 66402 (0.0009) [2023-10-07 22:28:47,115][67838] Updated weights for policy 0, policy_version 66412 (0.0010) [2023-10-07 22:28:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 136085504. Throughput: 0: 1658.1, 1: 1662.8. Samples: 34035434. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-10-07 22:28:47,478][66916] Avg episode reward: [(0, '52.200'), (1, '53.530')] [2023-10-07 22:28:47,490][67838] Updated weights for policy 0, policy_version 66422 (0.0007) [2023-10-07 22:28:47,503][67871] Updated weights for policy 1, policy_version 66500 (0.0009) [2023-10-07 22:28:47,848][67838] Updated weights for policy 0, policy_version 66432 (0.0009) [2023-10-07 22:28:47,866][67871] Updated weights for policy 1, policy_version 66510 (0.0010) [2023-10-07 22:28:48,229][67871] Updated weights for policy 1, policy_version 66520 (0.0009) [2023-10-07 22:28:52,118][67838] Updated weights for policy 0, policy_version 66442 (0.0010) [2023-10-07 22:28:52,341][67871] Updated weights for policy 1, policy_version 66530 (0.0008) [2023-10-07 22:28:52,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 136151040. Throughput: 0: 1647.6, 1: 1662.9. Samples: 34055442. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-10-07 22:28:52,478][66916] Avg episode reward: [(0, '53.790'), (1, '52.540')] [2023-10-07 22:28:52,481][67838] Updated weights for policy 0, policy_version 66452 (0.0008) [2023-10-07 22:28:52,717][67871] Updated weights for policy 1, policy_version 66540 (0.0007) [2023-10-07 22:28:52,852][67838] Updated weights for policy 0, policy_version 66462 (0.0007) [2023-10-07 22:28:53,089][67871] Updated weights for policy 1, policy_version 66550 (0.0008) [2023-10-07 22:28:53,456][67871] Updated weights for policy 1, policy_version 66560 (0.0007) [2023-10-07 22:28:56,976][67838] Updated weights for policy 0, policy_version 66472 (0.0008) [2023-10-07 22:28:57,356][67838] Updated weights for policy 0, policy_version 66482 (0.0007) [2023-10-07 22:28:57,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 136216576. Throughput: 0: 1651.0, 1: 1662.1. Samples: 34064666. Policy #0 lag: (min: 2.0, avg: 11.7, max: 34.0) [2023-10-07 22:28:57,477][66916] Avg episode reward: [(0, '58.530'), (1, '54.900')] [2023-10-07 22:28:57,590][67871] Updated weights for policy 1, policy_version 66570 (0.0008) [2023-10-07 22:28:57,724][67838] Updated weights for policy 0, policy_version 66492 (0.0008) [2023-10-07 22:28:57,957][67871] Updated weights for policy 1, policy_version 66580 (0.0007) [2023-10-07 22:28:58,327][67871] Updated weights for policy 1, policy_version 66590 (0.0008) [2023-10-07 22:29:01,900][67838] Updated weights for policy 0, policy_version 66502 (0.0010) [2023-10-07 22:29:02,273][67838] Updated weights for policy 0, policy_version 66512 (0.0009) [2023-10-07 22:29:02,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 136282112. Throughput: 0: 1655.2, 1: 1659.9. Samples: 34085048. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-07 22:29:02,477][66916] Avg episode reward: [(0, '57.520'), (1, '57.410')] [2023-10-07 22:29:02,510][67871] Updated weights for policy 1, policy_version 66600 (0.0008) [2023-10-07 22:29:02,638][67838] Updated weights for policy 0, policy_version 66522 (0.0008) [2023-10-07 22:29:02,878][67871] Updated weights for policy 1, policy_version 66610 (0.0008) [2023-10-07 22:29:03,243][67871] Updated weights for policy 1, policy_version 66620 (0.0009) [2023-10-07 22:29:06,722][67838] Updated weights for policy 0, policy_version 66532 (0.0007) [2023-10-07 22:29:07,087][67838] Updated weights for policy 0, policy_version 66542 (0.0007) [2023-10-07 22:29:07,437][67871] Updated weights for policy 1, policy_version 66630 (0.0008) [2023-10-07 22:29:07,464][67838] Updated weights for policy 0, policy_version 66552 (0.0007) [2023-10-07 22:29:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 136347648. Throughput: 0: 1643.8, 1: 1658.4. Samples: 34105006. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-07 22:29:07,477][66916] Avg episode reward: [(0, '57.050'), (1, '54.360')] [2023-10-07 22:29:07,748][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000066560_68157440.pth... [2023-10-07 22:29:07,776][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000064992_66551808.pth [2023-10-07 22:29:07,806][67871] Updated weights for policy 1, policy_version 66640 (0.0007) [2023-10-07 22:29:08,169][67871] Updated weights for policy 1, policy_version 66650 (0.0007) [2023-10-07 22:29:08,396][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000066656_68255744.pth... [2023-10-07 22:29:08,434][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000065088_66650112.pth [2023-10-07 22:29:11,634][67838] Updated weights for policy 0, policy_version 66562 (0.0007) [2023-10-07 22:29:11,993][67838] Updated weights for policy 0, policy_version 66572 (0.0007) [2023-10-07 22:29:12,330][67871] Updated weights for policy 1, policy_version 66660 (0.0009) [2023-10-07 22:29:12,375][67838] Updated weights for policy 0, policy_version 66582 (0.0009) [2023-10-07 22:29:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 136413184. Throughput: 0: 1652.5, 1: 1656.7. Samples: 34114544. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-07 22:29:12,477][66916] Avg episode reward: [(0, '56.130'), (1, '54.100')] [2023-10-07 22:29:12,700][67871] Updated weights for policy 1, policy_version 66670 (0.0010) [2023-10-07 22:29:12,741][67838] Updated weights for policy 0, policy_version 66592 (0.0008) [2023-10-07 22:29:13,065][67871] Updated weights for policy 1, policy_version 66680 (0.0009) [2023-10-07 22:29:16,958][67838] Updated weights for policy 0, policy_version 66602 (0.0007) [2023-10-07 22:29:17,160][67871] Updated weights for policy 1, policy_version 66690 (0.0007) [2023-10-07 22:29:17,321][67838] Updated weights for policy 0, policy_version 66612 (0.0007) [2023-10-07 22:29:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 136478720. Throughput: 0: 1650.5, 1: 1657.1. Samples: 34134758. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-07 22:29:17,477][66916] Avg episode reward: [(0, '55.380'), (1, '57.230')] [2023-10-07 22:29:17,534][67871] Updated weights for policy 1, policy_version 66700 (0.0008) [2023-10-07 22:29:17,694][67838] Updated weights for policy 0, policy_version 66622 (0.0007) [2023-10-07 22:29:17,893][67871] Updated weights for policy 1, policy_version 66710 (0.0008) [2023-10-07 22:29:18,247][67871] Updated weights for policy 1, policy_version 66720 (0.0008) [2023-10-07 22:29:21,721][67838] Updated weights for policy 0, policy_version 66632 (0.0007) [2023-10-07 22:29:22,098][67838] Updated weights for policy 0, policy_version 66642 (0.0007) [2023-10-07 22:29:22,399][67871] Updated weights for policy 1, policy_version 66730 (0.0007) [2023-10-07 22:29:22,473][67838] Updated weights for policy 0, policy_version 66652 (0.0008) [2023-10-07 22:29:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 136544256. Throughput: 0: 1643.4, 1: 1664.8. Samples: 34154866. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-07 22:29:22,477][66916] Avg episode reward: [(0, '51.600'), (1, '51.270')] [2023-10-07 22:29:22,775][67871] Updated weights for policy 1, policy_version 66740 (0.0010) [2023-10-07 22:29:23,141][67871] Updated weights for policy 1, policy_version 66750 (0.0010) [2023-10-07 22:29:26,643][67838] Updated weights for policy 0, policy_version 66662 (0.0009) [2023-10-07 22:29:27,017][67838] Updated weights for policy 0, policy_version 66672 (0.0008) [2023-10-07 22:29:27,186][67871] Updated weights for policy 1, policy_version 66760 (0.0011) [2023-10-07 22:29:27,384][67838] Updated weights for policy 0, policy_version 66682 (0.0008) [2023-10-07 22:29:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 136609792. Throughput: 0: 1650.8, 1: 1663.8. Samples: 34164490. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-07 22:29:27,477][66916] Avg episode reward: [(0, '53.070'), (1, '52.480')] [2023-10-07 22:29:27,562][67871] Updated weights for policy 1, policy_version 66770 (0.0009) [2023-10-07 22:29:27,919][67871] Updated weights for policy 1, policy_version 66780 (0.0010) [2023-10-07 22:29:31,480][67838] Updated weights for policy 0, policy_version 66692 (0.0008) [2023-10-07 22:29:31,842][67838] Updated weights for policy 0, policy_version 66702 (0.0007) [2023-10-07 22:29:31,960][67871] Updated weights for policy 1, policy_version 66790 (0.0008) [2023-10-07 22:29:32,218][67838] Updated weights for policy 0, policy_version 66712 (0.0007) [2023-10-07 22:29:32,349][67871] Updated weights for policy 1, policy_version 66800 (0.0008) [2023-10-07 22:29:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 136675328. Throughput: 0: 1655.6, 1: 1672.3. Samples: 34185188. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-07 22:29:32,477][66916] Avg episode reward: [(0, '54.010'), (1, '53.220')] [2023-10-07 22:29:32,720][67871] Updated weights for policy 1, policy_version 66810 (0.0008) [2023-10-07 22:29:36,175][67838] Updated weights for policy 0, policy_version 66722 (0.0007) [2023-10-07 22:29:36,556][67838] Updated weights for policy 0, policy_version 66732 (0.0007) [2023-10-07 22:29:36,737][67871] Updated weights for policy 1, policy_version 66820 (0.0009) [2023-10-07 22:29:36,923][67838] Updated weights for policy 0, policy_version 66742 (0.0007) [2023-10-07 22:29:37,092][67871] Updated weights for policy 1, policy_version 66830 (0.0007) [2023-10-07 22:29:37,298][67838] Updated weights for policy 0, policy_version 66752 (0.0009) [2023-10-07 22:29:37,463][67871] Updated weights for policy 1, policy_version 66840 (0.0009) [2023-10-07 22:29:37,476][66916] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 136773632. Throughput: 0: 1647.9, 1: 1663.3. Samples: 34204444. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-07 22:29:37,477][66916] Avg episode reward: [(0, '50.710'), (1, '52.970')] [2023-10-07 22:29:41,335][67838] Updated weights for policy 0, policy_version 66762 (0.0011) [2023-10-07 22:29:41,623][67871] Updated weights for policy 1, policy_version 66850 (0.0009) [2023-10-07 22:29:41,712][67838] Updated weights for policy 0, policy_version 66772 (0.0009) [2023-10-07 22:29:41,990][67871] Updated weights for policy 1, policy_version 66860 (0.0007) [2023-10-07 22:29:42,087][67838] Updated weights for policy 0, policy_version 66782 (0.0008) [2023-10-07 22:29:42,358][67871] Updated weights for policy 1, policy_version 66870 (0.0007) [2023-10-07 22:29:42,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 136839168. Throughput: 0: 1666.6, 1: 1670.7. Samples: 34214848. Policy #0 lag: (min: 29.0, avg: 36.1, max: 61.0) [2023-10-07 22:29:42,477][66916] Avg episode reward: [(0, '50.820'), (1, '56.070')] [2023-10-07 22:29:42,717][67871] Updated weights for policy 1, policy_version 66880 (0.0008) [2023-10-07 22:29:46,231][67838] Updated weights for policy 0, policy_version 66792 (0.0011) [2023-10-07 22:29:46,611][67838] Updated weights for policy 0, policy_version 66802 (0.0008) [2023-10-07 22:29:46,913][67871] Updated weights for policy 1, policy_version 66890 (0.0007) [2023-10-07 22:29:46,968][67838] Updated weights for policy 0, policy_version 66812 (0.0008) [2023-10-07 22:29:47,287][67871] Updated weights for policy 1, policy_version 66900 (0.0008) [2023-10-07 22:29:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 136904704. Throughput: 0: 1658.1, 1: 1672.2. Samples: 34234910. Policy #0 lag: (min: 29.0, avg: 36.1, max: 61.0) [2023-10-07 22:29:47,477][66916] Avg episode reward: [(0, '50.890'), (1, '56.440')] [2023-10-07 22:29:47,650][67871] Updated weights for policy 1, policy_version 66910 (0.0009) [2023-10-07 22:29:51,240][67838] Updated weights for policy 0, policy_version 66822 (0.0008) [2023-10-07 22:29:51,608][67838] Updated weights for policy 0, policy_version 66832 (0.0009) [2023-10-07 22:29:51,755][67871] Updated weights for policy 1, policy_version 66920 (0.0009) [2023-10-07 22:29:51,977][67838] Updated weights for policy 0, policy_version 66842 (0.0009) [2023-10-07 22:29:52,122][67871] Updated weights for policy 1, policy_version 66930 (0.0008) [2023-10-07 22:29:52,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 136970240. Throughput: 0: 1650.6, 1: 1660.3. Samples: 34253998. Policy #0 lag: (min: 29.0, avg: 36.1, max: 61.0) [2023-10-07 22:29:52,478][66916] Avg episode reward: [(0, '50.450'), (1, '54.680')] [2023-10-07 22:29:52,494][67871] Updated weights for policy 1, policy_version 66940 (0.0008) [2023-10-07 22:29:56,032][67838] Updated weights for policy 0, policy_version 66852 (0.0008) [2023-10-07 22:29:56,409][67838] Updated weights for policy 0, policy_version 66862 (0.0007) [2023-10-07 22:29:56,506][67871] Updated weights for policy 1, policy_version 66950 (0.0009) [2023-10-07 22:29:56,787][67838] Updated weights for policy 0, policy_version 66872 (0.0007) [2023-10-07 22:29:56,861][67871] Updated weights for policy 1, policy_version 66960 (0.0009) [2023-10-07 22:29:57,224][67871] Updated weights for policy 1, policy_version 66970 (0.0010) [2023-10-07 22:29:57,476][66916] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 137068544. Throughput: 0: 1665.2, 1: 1667.1. Samples: 34264498. Policy #0 lag: (min: 29.0, avg: 36.1, max: 61.0) [2023-10-07 22:29:57,477][66916] Avg episode reward: [(0, '54.210'), (1, '55.320')] [2023-10-07 22:30:00,990][67838] Updated weights for policy 0, policy_version 66882 (0.0008) [2023-10-07 22:30:01,356][67838] Updated weights for policy 0, policy_version 66892 (0.0007) [2023-10-07 22:30:01,493][67871] Updated weights for policy 1, policy_version 66980 (0.0008) [2023-10-07 22:30:01,729][67838] Updated weights for policy 0, policy_version 66902 (0.0007) [2023-10-07 22:30:01,853][67871] Updated weights for policy 1, policy_version 66990 (0.0007) [2023-10-07 22:30:02,092][67838] Updated weights for policy 0, policy_version 66912 (0.0009) [2023-10-07 22:30:02,218][67871] Updated weights for policy 1, policy_version 67000 (0.0008) [2023-10-07 22:30:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 137101312. Throughput: 0: 1668.0, 1: 1662.3. Samples: 34284622. Policy #0 lag: (min: 29.0, avg: 36.1, max: 61.0) [2023-10-07 22:30:02,477][66916] Avg episode reward: [(0, '49.030'), (1, '55.520')] [2023-10-07 22:30:06,270][67838] Updated weights for policy 0, policy_version 66922 (0.0007) [2023-10-07 22:30:06,494][67871] Updated weights for policy 1, policy_version 67010 (0.0007) [2023-10-07 22:30:06,650][67838] Updated weights for policy 0, policy_version 66932 (0.0007) [2023-10-07 22:30:06,856][67871] Updated weights for policy 1, policy_version 67020 (0.0007) [2023-10-07 22:30:07,032][67838] Updated weights for policy 0, policy_version 66942 (0.0008) [2023-10-07 22:30:07,222][67871] Updated weights for policy 1, policy_version 67030 (0.0007) [2023-10-07 22:30:07,476][66916] Fps is (10 sec: 9830.4, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 137166848. Throughput: 0: 1658.0, 1: 1647.4. Samples: 34303608. Policy #0 lag: (min: 29.0, avg: 36.1, max: 61.0) [2023-10-07 22:30:07,477][66916] Avg episode reward: [(0, '50.710'), (1, '54.760')] [2023-10-07 22:30:07,585][67871] Updated weights for policy 1, policy_version 67040 (0.0008) [2023-10-07 22:30:11,145][67838] Updated weights for policy 0, policy_version 66952 (0.0008) [2023-10-07 22:30:11,524][67838] Updated weights for policy 0, policy_version 66962 (0.0007) [2023-10-07 22:30:11,631][67871] Updated weights for policy 1, policy_version 67050 (0.0008) [2023-10-07 22:30:11,892][67838] Updated weights for policy 0, policy_version 66972 (0.0008) [2023-10-07 22:30:11,997][67871] Updated weights for policy 1, policy_version 67060 (0.0008) [2023-10-07 22:30:12,363][67871] Updated weights for policy 1, policy_version 67070 (0.0008) [2023-10-07 22:30:12,476][66916] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 137265152. Throughput: 0: 1670.1, 1: 1658.5. Samples: 34314276. Policy #0 lag: (min: 29.0, avg: 36.1, max: 61.0) [2023-10-07 22:30:12,477][66916] Avg episode reward: [(0, '49.230'), (1, '55.570')] [2023-10-07 22:30:16,008][67838] Updated weights for policy 0, policy_version 66982 (0.0008) [2023-10-07 22:30:16,376][67838] Updated weights for policy 0, policy_version 66992 (0.0008) [2023-10-07 22:30:16,654][67871] Updated weights for policy 1, policy_version 67080 (0.0007) [2023-10-07 22:30:16,754][67838] Updated weights for policy 0, policy_version 67002 (0.0007) [2023-10-07 22:30:17,016][67871] Updated weights for policy 1, policy_version 67090 (0.0007) [2023-10-07 22:30:17,385][67871] Updated weights for policy 1, policy_version 67100 (0.0007) [2023-10-07 22:30:17,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 137297920. Throughput: 0: 1660.5, 1: 1651.9. Samples: 34334248. Policy #0 lag: (min: 29.0, avg: 36.1, max: 61.0) [2023-10-07 22:30:17,477][66916] Avg episode reward: [(0, '48.830'), (1, '54.500')] [2023-10-07 22:30:20,810][67838] Updated weights for policy 0, policy_version 67012 (0.0007) [2023-10-07 22:30:21,181][67838] Updated weights for policy 0, policy_version 67022 (0.0007) [2023-10-07 22:30:21,542][67871] Updated weights for policy 1, policy_version 67110 (0.0009) [2023-10-07 22:30:21,556][67838] Updated weights for policy 0, policy_version 67032 (0.0009) [2023-10-07 22:30:21,908][67871] Updated weights for policy 1, policy_version 67120 (0.0007) [2023-10-07 22:30:22,278][67871] Updated weights for policy 1, policy_version 67130 (0.0007) [2023-10-07 22:30:22,477][66916] Fps is (10 sec: 9830.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 137363456. Throughput: 0: 1661.2, 1: 1646.3. Samples: 34353282. Policy #0 lag: (min: 29.0, avg: 36.1, max: 61.0) [2023-10-07 22:30:22,478][66916] Avg episode reward: [(0, '49.200'), (1, '55.680')] [2023-10-07 22:30:25,670][67838] Updated weights for policy 0, policy_version 67042 (0.0008) [2023-10-07 22:30:26,047][67838] Updated weights for policy 0, policy_version 67052 (0.0009) [2023-10-07 22:30:26,386][67871] Updated weights for policy 1, policy_version 67140 (0.0007) [2023-10-07 22:30:26,411][67838] Updated weights for policy 0, policy_version 67062 (0.0008) [2023-10-07 22:30:26,760][67871] Updated weights for policy 1, policy_version 67150 (0.0008) [2023-10-07 22:30:26,779][67838] Updated weights for policy 0, policy_version 67072 (0.0008) [2023-10-07 22:30:27,124][67871] Updated weights for policy 1, policy_version 67160 (0.0007) [2023-10-07 22:30:27,476][66916] Fps is (10 sec: 16384.2, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 137461760. Throughput: 0: 1665.9, 1: 1648.5. Samples: 34363996. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:30:27,477][66916] Avg episode reward: [(0, '50.330'), (1, '52.820')] [2023-10-07 22:30:30,937][67838] Updated weights for policy 0, policy_version 67082 (0.0008) [2023-10-07 22:30:31,306][67871] Updated weights for policy 1, policy_version 67170 (0.0008) [2023-10-07 22:30:31,320][67838] Updated weights for policy 0, policy_version 67092 (0.0009) [2023-10-07 22:30:31,668][67871] Updated weights for policy 1, policy_version 67180 (0.0009) [2023-10-07 22:30:31,680][67838] Updated weights for policy 0, policy_version 67102 (0.0009) [2023-10-07 22:30:32,034][67871] Updated weights for policy 1, policy_version 67190 (0.0008) [2023-10-07 22:30:32,396][67871] Updated weights for policy 1, policy_version 67200 (0.0010) [2023-10-07 22:30:32,476][66916] Fps is (10 sec: 16384.3, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 137527296. Throughput: 0: 1658.5, 1: 1649.1. Samples: 34383752. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:30:32,477][66916] Avg episode reward: [(0, '48.440'), (1, '53.180')] [2023-10-07 22:30:35,878][67838] Updated weights for policy 0, policy_version 67112 (0.0008) [2023-10-07 22:30:36,246][67838] Updated weights for policy 0, policy_version 67122 (0.0008) [2023-10-07 22:30:36,319][67871] Updated weights for policy 1, policy_version 67210 (0.0008) [2023-10-07 22:30:36,614][67838] Updated weights for policy 0, policy_version 67132 (0.0008) [2023-10-07 22:30:36,690][67871] Updated weights for policy 1, policy_version 67220 (0.0008) [2023-10-07 22:30:37,053][67871] Updated weights for policy 1, policy_version 67230 (0.0009) [2023-10-07 22:30:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 137592832. Throughput: 0: 1659.4, 1: 1641.5. Samples: 34402536. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:30:37,477][66916] Avg episode reward: [(0, '48.750'), (1, '49.110')] [2023-10-07 22:30:40,631][67838] Updated weights for policy 0, policy_version 67142 (0.0010) [2023-10-07 22:30:41,008][67838] Updated weights for policy 0, policy_version 67152 (0.0010) [2023-10-07 22:30:41,373][67838] Updated weights for policy 0, policy_version 67162 (0.0007) [2023-10-07 22:30:41,389][67871] Updated weights for policy 1, policy_version 67240 (0.0009) [2023-10-07 22:30:41,753][67871] Updated weights for policy 1, policy_version 67250 (0.0008) [2023-10-07 22:30:42,118][67871] Updated weights for policy 1, policy_version 67260 (0.0009) [2023-10-07 22:30:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 137658368. Throughput: 0: 1663.5, 1: 1652.3. Samples: 34413714. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:30:42,478][66916] Avg episode reward: [(0, '46.150'), (1, '50.440')] [2023-10-07 22:30:45,542][67838] Updated weights for policy 0, policy_version 67172 (0.0008) [2023-10-07 22:30:45,914][67838] Updated weights for policy 0, policy_version 67182 (0.0009) [2023-10-07 22:30:46,298][67838] Updated weights for policy 0, policy_version 67192 (0.0008) [2023-10-07 22:30:46,479][67871] Updated weights for policy 1, policy_version 67270 (0.0007) [2023-10-07 22:30:46,848][67871] Updated weights for policy 1, policy_version 67280 (0.0008) [2023-10-07 22:30:47,214][67871] Updated weights for policy 1, policy_version 67290 (0.0009) [2023-10-07 22:30:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 137723904. Throughput: 0: 1646.2, 1: 1655.7. Samples: 34433206. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:30:47,477][66916] Avg episode reward: [(0, '42.940'), (1, '48.050')] [2023-10-07 22:30:50,399][67838] Updated weights for policy 0, policy_version 67202 (0.0009) [2023-10-07 22:30:50,765][67838] Updated weights for policy 0, policy_version 67212 (0.0009) [2023-10-07 22:30:51,131][67838] Updated weights for policy 0, policy_version 67222 (0.0009) [2023-10-07 22:30:51,202][67871] Updated weights for policy 1, policy_version 67300 (0.0008) [2023-10-07 22:30:51,498][67838] Updated weights for policy 0, policy_version 67232 (0.0008) [2023-10-07 22:30:51,570][67871] Updated weights for policy 1, policy_version 67310 (0.0009) [2023-10-07 22:30:51,940][67871] Updated weights for policy 1, policy_version 67320 (0.0009) [2023-10-07 22:30:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 137789440. Throughput: 0: 1656.0, 1: 1644.4. Samples: 34452128. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:30:52,477][66916] Avg episode reward: [(0, '45.810'), (1, '47.660')] [2023-10-07 22:30:55,665][67838] Updated weights for policy 0, policy_version 67242 (0.0008) [2023-10-07 22:30:56,045][67838] Updated weights for policy 0, policy_version 67252 (0.0008) [2023-10-07 22:30:56,070][67871] Updated weights for policy 1, policy_version 67330 (0.0010) [2023-10-07 22:30:56,411][67838] Updated weights for policy 0, policy_version 67262 (0.0007) [2023-10-07 22:30:56,443][67871] Updated weights for policy 1, policy_version 67340 (0.0009) [2023-10-07 22:30:56,811][67871] Updated weights for policy 1, policy_version 67350 (0.0008) [2023-10-07 22:30:57,185][67871] Updated weights for policy 1, policy_version 67360 (0.0007) [2023-10-07 22:30:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 137854976. Throughput: 0: 1659.3, 1: 1651.4. Samples: 34463260. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:30:57,478][66916] Avg episode reward: [(0, '46.260'), (1, '48.190')] [2023-10-07 22:31:00,483][67838] Updated weights for policy 0, policy_version 67272 (0.0010) [2023-10-07 22:31:00,864][67838] Updated weights for policy 0, policy_version 67282 (0.0008) [2023-10-07 22:31:01,231][67838] Updated weights for policy 0, policy_version 67292 (0.0007) [2023-10-07 22:31:01,298][67871] Updated weights for policy 1, policy_version 67370 (0.0008) [2023-10-07 22:31:01,666][67871] Updated weights for policy 1, policy_version 67380 (0.0011) [2023-10-07 22:31:02,027][67871] Updated weights for policy 1, policy_version 67390 (0.0010) [2023-10-07 22:31:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 137920512. Throughput: 0: 1649.7, 1: 1653.9. Samples: 34482910. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:31:02,477][66916] Avg episode reward: [(0, '46.030'), (1, '51.510')] [2023-10-07 22:31:05,478][67838] Updated weights for policy 0, policy_version 67302 (0.0007) [2023-10-07 22:31:05,856][67838] Updated weights for policy 0, policy_version 67312 (0.0007) [2023-10-07 22:31:06,217][67871] Updated weights for policy 1, policy_version 67400 (0.0008) [2023-10-07 22:31:06,221][67838] Updated weights for policy 0, policy_version 67322 (0.0007) [2023-10-07 22:31:06,584][67871] Updated weights for policy 1, policy_version 67410 (0.0007) [2023-10-07 22:31:06,955][67871] Updated weights for policy 1, policy_version 67420 (0.0009) [2023-10-07 22:31:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 137986048. Throughput: 0: 1655.0, 1: 1644.8. Samples: 34501774. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:31:07,478][66916] Avg episode reward: [(0, '50.060'), (1, '50.830')] [2023-10-07 22:31:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000067424_69042176.pth... [2023-10-07 22:31:07,490][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000067328_68943872.pth... [2023-10-07 22:31:07,526][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000065856_67436544.pth [2023-10-07 22:31:07,527][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000065792_67371008.pth [2023-10-07 22:31:10,339][67838] Updated weights for policy 0, policy_version 67332 (0.0008) [2023-10-07 22:31:10,712][67838] Updated weights for policy 0, policy_version 67342 (0.0010) [2023-10-07 22:31:11,079][67838] Updated weights for policy 0, policy_version 67352 (0.0009) [2023-10-07 22:31:11,210][67871] Updated weights for policy 1, policy_version 67430 (0.0009) [2023-10-07 22:31:11,583][67871] Updated weights for policy 1, policy_version 67440 (0.0007) [2023-10-07 22:31:11,954][67871] Updated weights for policy 1, policy_version 67450 (0.0008) [2023-10-07 22:31:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138051584. Throughput: 0: 1652.9, 1: 1657.1. Samples: 34512946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:31:12,477][66916] Avg episode reward: [(0, '52.450'), (1, '50.220')] [2023-10-07 22:31:15,249][67838] Updated weights for policy 0, policy_version 67362 (0.0009) [2023-10-07 22:31:15,625][67838] Updated weights for policy 0, policy_version 67372 (0.0009) [2023-10-07 22:31:15,951][67871] Updated weights for policy 1, policy_version 67460 (0.0008) [2023-10-07 22:31:15,993][67838] Updated weights for policy 0, policy_version 67382 (0.0008) [2023-10-07 22:31:16,321][67871] Updated weights for policy 1, policy_version 67470 (0.0007) [2023-10-07 22:31:16,360][67838] Updated weights for policy 0, policy_version 67392 (0.0009) [2023-10-07 22:31:16,682][67871] Updated weights for policy 1, policy_version 67480 (0.0009) [2023-10-07 22:31:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 138117120. Throughput: 0: 1642.4, 1: 1659.4. Samples: 34532334. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:31:17,477][66916] Avg episode reward: [(0, '47.570'), (1, '48.760')] [2023-10-07 22:31:20,550][67838] Updated weights for policy 0, policy_version 67402 (0.0007) [2023-10-07 22:31:20,775][67871] Updated weights for policy 1, policy_version 67490 (0.0008) [2023-10-07 22:31:20,930][67838] Updated weights for policy 0, policy_version 67412 (0.0007) [2023-10-07 22:31:21,136][67871] Updated weights for policy 1, policy_version 67500 (0.0007) [2023-10-07 22:31:21,292][67838] Updated weights for policy 0, policy_version 67422 (0.0007) [2023-10-07 22:31:21,503][67871] Updated weights for policy 1, policy_version 67510 (0.0007) [2023-10-07 22:31:21,878][67871] Updated weights for policy 1, policy_version 67520 (0.0007) [2023-10-07 22:31:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 138182656. Throughput: 0: 1651.0, 1: 1655.1. Samples: 34551310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:31:22,477][66916] Avg episode reward: [(0, '48.310'), (1, '48.070')] [2023-10-07 22:31:25,442][67838] Updated weights for policy 0, policy_version 67432 (0.0008) [2023-10-07 22:31:25,814][67838] Updated weights for policy 0, policy_version 67442 (0.0009) [2023-10-07 22:31:26,050][67871] Updated weights for policy 1, policy_version 67530 (0.0008) [2023-10-07 22:31:26,188][67838] Updated weights for policy 0, policy_version 67452 (0.0009) [2023-10-07 22:31:26,418][67871] Updated weights for policy 1, policy_version 67540 (0.0009) [2023-10-07 22:31:26,776][67871] Updated weights for policy 1, policy_version 67550 (0.0008) [2023-10-07 22:31:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138248192. Throughput: 0: 1645.8, 1: 1663.3. Samples: 34562624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:31:27,478][66916] Avg episode reward: [(0, '46.530'), (1, '48.190')] [2023-10-07 22:31:30,251][67838] Updated weights for policy 0, policy_version 67462 (0.0009) [2023-10-07 22:31:30,626][67838] Updated weights for policy 0, policy_version 67472 (0.0010) [2023-10-07 22:31:30,800][67871] Updated weights for policy 1, policy_version 67560 (0.0008) [2023-10-07 22:31:30,984][67838] Updated weights for policy 0, policy_version 67482 (0.0008) [2023-10-07 22:31:31,170][67871] Updated weights for policy 1, policy_version 67570 (0.0009) [2023-10-07 22:31:31,530][67871] Updated weights for policy 1, policy_version 67580 (0.0007) [2023-10-07 22:31:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138313728. Throughput: 0: 1644.2, 1: 1658.0. Samples: 34581808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:31:32,478][66916] Avg episode reward: [(0, '45.400'), (1, '50.720')] [2023-10-07 22:31:35,150][67838] Updated weights for policy 0, policy_version 67492 (0.0009) [2023-10-07 22:31:35,516][67838] Updated weights for policy 0, policy_version 67502 (0.0009) [2023-10-07 22:31:35,584][67871] Updated weights for policy 1, policy_version 67590 (0.0011) [2023-10-07 22:31:35,882][67838] Updated weights for policy 0, policy_version 67512 (0.0009) [2023-10-07 22:31:35,948][67871] Updated weights for policy 1, policy_version 67600 (0.0008) [2023-10-07 22:31:36,307][67871] Updated weights for policy 1, policy_version 67610 (0.0007) [2023-10-07 22:31:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138379264. Throughput: 0: 1652.3, 1: 1664.6. Samples: 34601390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:31:37,478][66916] Avg episode reward: [(0, '45.850'), (1, '49.990')] [2023-10-07 22:31:39,934][67838] Updated weights for policy 0, policy_version 67522 (0.0008) [2023-10-07 22:31:40,254][67871] Updated weights for policy 1, policy_version 67620 (0.0007) [2023-10-07 22:31:40,303][67838] Updated weights for policy 0, policy_version 67532 (0.0010) [2023-10-07 22:31:40,628][67871] Updated weights for policy 1, policy_version 67630 (0.0008) [2023-10-07 22:31:40,679][67838] Updated weights for policy 0, policy_version 67542 (0.0008) [2023-10-07 22:31:40,989][67871] Updated weights for policy 1, policy_version 67640 (0.0008) [2023-10-07 22:31:41,050][67838] Updated weights for policy 0, policy_version 67552 (0.0008) [2023-10-07 22:31:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138444800. Throughput: 0: 1641.7, 1: 1677.6. Samples: 34612630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:31:42,478][66916] Avg episode reward: [(0, '46.050'), (1, '51.320')] [2023-10-07 22:31:44,928][67871] Updated weights for policy 1, policy_version 67650 (0.0008) [2023-10-07 22:31:45,239][67838] Updated weights for policy 0, policy_version 67562 (0.0007) [2023-10-07 22:31:45,301][67871] Updated weights for policy 1, policy_version 67660 (0.0008) [2023-10-07 22:31:45,615][67838] Updated weights for policy 0, policy_version 67572 (0.0009) [2023-10-07 22:31:45,666][67871] Updated weights for policy 1, policy_version 67670 (0.0009) [2023-10-07 22:31:45,974][67838] Updated weights for policy 0, policy_version 67582 (0.0008) [2023-10-07 22:31:46,027][67871] Updated weights for policy 1, policy_version 67680 (0.0008) [2023-10-07 22:31:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138510336. Throughput: 0: 1636.2, 1: 1659.8. Samples: 34631230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:31:47,478][66916] Avg episode reward: [(0, '48.200'), (1, '55.280')] [2023-10-07 22:31:50,309][67838] Updated weights for policy 0, policy_version 67592 (0.0008) [2023-10-07 22:31:50,386][67871] Updated weights for policy 1, policy_version 67690 (0.0007) [2023-10-07 22:31:50,696][67838] Updated weights for policy 0, policy_version 67602 (0.0008) [2023-10-07 22:31:50,748][67871] Updated weights for policy 1, policy_version 67700 (0.0007) [2023-10-07 22:31:51,061][67838] Updated weights for policy 0, policy_version 67612 (0.0010) [2023-10-07 22:31:51,108][67871] Updated weights for policy 1, policy_version 67710 (0.0008) [2023-10-07 22:31:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 138575872. Throughput: 0: 1642.1, 1: 1667.6. Samples: 34650712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:31:52,478][66916] Avg episode reward: [(0, '47.880'), (1, '53.520')] [2023-10-07 22:31:55,096][67838] Updated weights for policy 0, policy_version 67622 (0.0008) [2023-10-07 22:31:55,175][67871] Updated weights for policy 1, policy_version 67720 (0.0009) [2023-10-07 22:31:55,475][67838] Updated weights for policy 0, policy_version 67632 (0.0009) [2023-10-07 22:31:55,539][67871] Updated weights for policy 1, policy_version 67730 (0.0009) [2023-10-07 22:31:55,839][67838] Updated weights for policy 0, policy_version 67642 (0.0008) [2023-10-07 22:31:55,897][67871] Updated weights for policy 1, policy_version 67740 (0.0008) [2023-10-07 22:31:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138641408. Throughput: 0: 1640.2, 1: 1680.1. Samples: 34662360. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-07 22:31:57,477][66916] Avg episode reward: [(0, '51.740'), (1, '54.890')] [2023-10-07 22:32:00,058][67871] Updated weights for policy 1, policy_version 67750 (0.0009) [2023-10-07 22:32:00,071][67838] Updated weights for policy 0, policy_version 67652 (0.0008) [2023-10-07 22:32:00,419][67871] Updated weights for policy 1, policy_version 67760 (0.0009) [2023-10-07 22:32:00,438][67838] Updated weights for policy 0, policy_version 67662 (0.0009) [2023-10-07 22:32:00,785][67871] Updated weights for policy 1, policy_version 67770 (0.0009) [2023-10-07 22:32:00,811][67838] Updated weights for policy 0, policy_version 67672 (0.0008) [2023-10-07 22:32:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138706944. Throughput: 0: 1643.5, 1: 1656.6. Samples: 34680838. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-07 22:32:02,477][66916] Avg episode reward: [(0, '53.140'), (1, '54.090')] [2023-10-07 22:32:04,802][67871] Updated weights for policy 1, policy_version 67780 (0.0008) [2023-10-07 22:32:04,929][67838] Updated weights for policy 0, policy_version 67682 (0.0009) [2023-10-07 22:32:05,167][67871] Updated weights for policy 1, policy_version 67790 (0.0007) [2023-10-07 22:32:05,298][67838] Updated weights for policy 0, policy_version 67692 (0.0007) [2023-10-07 22:32:05,535][67871] Updated weights for policy 1, policy_version 67800 (0.0009) [2023-10-07 22:32:05,674][67838] Updated weights for policy 0, policy_version 67702 (0.0008) [2023-10-07 22:32:06,032][67838] Updated weights for policy 0, policy_version 67712 (0.0009) [2023-10-07 22:32:07,477][66916] Fps is (10 sec: 13106.6, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 138772480. Throughput: 0: 1652.7, 1: 1673.6. Samples: 34700996. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-07 22:32:07,478][66916] Avg episode reward: [(0, '52.670'), (1, '53.840')] [2023-10-07 22:32:09,598][67871] Updated weights for policy 1, policy_version 67810 (0.0009) [2023-10-07 22:32:09,965][67871] Updated weights for policy 1, policy_version 67820 (0.0008) [2023-10-07 22:32:10,227][67838] Updated weights for policy 0, policy_version 67722 (0.0007) [2023-10-07 22:32:10,331][67871] Updated weights for policy 1, policy_version 67830 (0.0007) [2023-10-07 22:32:10,592][67838] Updated weights for policy 0, policy_version 67732 (0.0007) [2023-10-07 22:32:10,696][67871] Updated weights for policy 1, policy_version 67840 (0.0008) [2023-10-07 22:32:10,964][67838] Updated weights for policy 0, policy_version 67742 (0.0009) [2023-10-07 22:32:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138838016. Throughput: 0: 1648.1, 1: 1670.1. Samples: 34711942. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-07 22:32:12,477][66916] Avg episode reward: [(0, '54.420'), (1, '53.310')] [2023-10-07 22:32:14,804][67871] Updated weights for policy 1, policy_version 67850 (0.0008) [2023-10-07 22:32:15,030][67838] Updated weights for policy 0, policy_version 67752 (0.0008) [2023-10-07 22:32:15,168][67871] Updated weights for policy 1, policy_version 67860 (0.0008) [2023-10-07 22:32:15,407][67838] Updated weights for policy 0, policy_version 67762 (0.0007) [2023-10-07 22:32:15,546][67871] Updated weights for policy 1, policy_version 67870 (0.0008) [2023-10-07 22:32:15,771][67838] Updated weights for policy 0, policy_version 67772 (0.0009) [2023-10-07 22:32:17,477][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138903552. Throughput: 0: 1643.6, 1: 1659.4. Samples: 34730442. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-07 22:32:17,477][66916] Avg episode reward: [(0, '55.030'), (1, '59.450')] [2023-10-07 22:32:19,718][67871] Updated weights for policy 1, policy_version 67880 (0.0010) [2023-10-07 22:32:19,930][67838] Updated weights for policy 0, policy_version 67782 (0.0008) [2023-10-07 22:32:20,090][67871] Updated weights for policy 1, policy_version 67890 (0.0010) [2023-10-07 22:32:20,303][67838] Updated weights for policy 0, policy_version 67792 (0.0008) [2023-10-07 22:32:20,445][67871] Updated weights for policy 1, policy_version 67900 (0.0009) [2023-10-07 22:32:20,666][67838] Updated weights for policy 0, policy_version 67802 (0.0007) [2023-10-07 22:32:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138969088. Throughput: 0: 1645.7, 1: 1677.0. Samples: 34750910. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-07 22:32:22,477][66916] Avg episode reward: [(0, '50.860'), (1, '59.840')] [2023-10-07 22:32:24,486][67871] Updated weights for policy 1, policy_version 67910 (0.0009) [2023-10-07 22:32:24,695][67838] Updated weights for policy 0, policy_version 67812 (0.0007) [2023-10-07 22:32:24,855][67871] Updated weights for policy 1, policy_version 67920 (0.0009) [2023-10-07 22:32:25,069][67838] Updated weights for policy 0, policy_version 67822 (0.0009) [2023-10-07 22:32:25,221][67871] Updated weights for policy 1, policy_version 67930 (0.0009) [2023-10-07 22:32:25,438][67838] Updated weights for policy 0, policy_version 67832 (0.0008) [2023-10-07 22:32:27,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 139034624. Throughput: 0: 1645.3, 1: 1662.3. Samples: 34761472. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-07 22:32:27,478][66916] Avg episode reward: [(0, '50.140'), (1, '58.530')] [2023-10-07 22:32:29,359][67871] Updated weights for policy 1, policy_version 67940 (0.0010) [2023-10-07 22:32:29,517][67838] Updated weights for policy 0, policy_version 67842 (0.0010) [2023-10-07 22:32:29,723][67871] Updated weights for policy 1, policy_version 67950 (0.0010) [2023-10-07 22:32:29,893][67838] Updated weights for policy 0, policy_version 67852 (0.0008) [2023-10-07 22:32:30,076][67871] Updated weights for policy 1, policy_version 67960 (0.0008) [2023-10-07 22:32:30,263][67838] Updated weights for policy 0, policy_version 67862 (0.0007) [2023-10-07 22:32:30,641][67838] Updated weights for policy 0, policy_version 67872 (0.0007) [2023-10-07 22:32:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 139100160. Throughput: 0: 1653.3, 1: 1665.8. Samples: 34780588. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-07 22:32:32,477][66916] Avg episode reward: [(0, '48.210'), (1, '60.560')] [2023-10-07 22:32:34,343][67871] Updated weights for policy 1, policy_version 67970 (0.0009) [2023-10-07 22:32:34,752][67838] Updated weights for policy 0, policy_version 67882 (0.0009) [2023-10-07 22:32:34,755][67871] Updated weights for policy 1, policy_version 67980 (0.0008) [2023-10-07 22:32:35,118][67871] Updated weights for policy 1, policy_version 67990 (0.0008) [2023-10-07 22:32:35,122][67838] Updated weights for policy 0, policy_version 67892 (0.0008) [2023-10-07 22:32:35,486][67871] Updated weights for policy 1, policy_version 68000 (0.0007) [2023-10-07 22:32:35,493][67838] Updated weights for policy 0, policy_version 67902 (0.0008) [2023-10-07 22:32:37,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 139165696. Throughput: 0: 1661.6, 1: 1676.9. Samples: 34800944. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-07 22:32:37,478][66916] Avg episode reward: [(0, '51.200'), (1, '57.810')] [2023-10-07 22:32:39,473][67871] Updated weights for policy 1, policy_version 68010 (0.0008) [2023-10-07 22:32:39,719][67838] Updated weights for policy 0, policy_version 67912 (0.0008) [2023-10-07 22:32:39,841][67871] Updated weights for policy 1, policy_version 68020 (0.0007) [2023-10-07 22:32:40,102][67838] Updated weights for policy 0, policy_version 67922 (0.0010) [2023-10-07 22:32:40,204][67871] Updated weights for policy 1, policy_version 68030 (0.0007) [2023-10-07 22:32:40,467][67838] Updated weights for policy 0, policy_version 67932 (0.0008) [2023-10-07 22:32:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 139231232. Throughput: 0: 1650.2, 1: 1654.7. Samples: 34811082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:32:42,478][66916] Avg episode reward: [(0, '50.650'), (1, '54.450')] [2023-10-07 22:32:44,321][67871] Updated weights for policy 1, policy_version 68040 (0.0007) [2023-10-07 22:32:44,685][67871] Updated weights for policy 1, policy_version 68050 (0.0007) [2023-10-07 22:32:44,687][67838] Updated weights for policy 0, policy_version 67942 (0.0007) [2023-10-07 22:32:45,047][67838] Updated weights for policy 0, policy_version 67952 (0.0007) [2023-10-07 22:32:45,055][67871] Updated weights for policy 1, policy_version 68060 (0.0007) [2023-10-07 22:32:45,424][67838] Updated weights for policy 0, policy_version 67962 (0.0008) [2023-10-07 22:32:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 139296768. Throughput: 0: 1655.5, 1: 1664.9. Samples: 34830252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:32:47,477][66916] Avg episode reward: [(0, '49.830'), (1, '54.610')] [2023-10-07 22:32:49,201][67871] Updated weights for policy 1, policy_version 68070 (0.0009) [2023-10-07 22:32:49,548][67838] Updated weights for policy 0, policy_version 67972 (0.0009) [2023-10-07 22:32:49,558][67871] Updated weights for policy 1, policy_version 68080 (0.0008) [2023-10-07 22:32:49,915][67838] Updated weights for policy 0, policy_version 67982 (0.0007) [2023-10-07 22:32:49,927][67871] Updated weights for policy 1, policy_version 68090 (0.0007) [2023-10-07 22:32:50,281][67838] Updated weights for policy 0, policy_version 67992 (0.0007) [2023-10-07 22:32:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 139362304. Throughput: 0: 1655.7, 1: 1670.7. Samples: 34850682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:32:52,478][66916] Avg episode reward: [(0, '52.150'), (1, '50.810')] [2023-10-07 22:32:54,127][67871] Updated weights for policy 1, policy_version 68100 (0.0009) [2023-10-07 22:32:54,365][67838] Updated weights for policy 0, policy_version 68002 (0.0010) [2023-10-07 22:32:54,502][67871] Updated weights for policy 1, policy_version 68110 (0.0009) [2023-10-07 22:32:54,740][67838] Updated weights for policy 0, policy_version 68012 (0.0008) [2023-10-07 22:32:54,863][67871] Updated weights for policy 1, policy_version 68120 (0.0009) [2023-10-07 22:32:55,109][67838] Updated weights for policy 0, policy_version 68022 (0.0007) [2023-10-07 22:32:55,488][67838] Updated weights for policy 0, policy_version 68032 (0.0008) [2023-10-07 22:32:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 139427840. Throughput: 0: 1646.7, 1: 1654.7. Samples: 34860504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:32:57,478][66916] Avg episode reward: [(0, '53.490'), (1, '52.560')] [2023-10-07 22:32:59,074][67871] Updated weights for policy 1, policy_version 68130 (0.0008) [2023-10-07 22:32:59,437][67871] Updated weights for policy 1, policy_version 68140 (0.0010) [2023-10-07 22:32:59,602][67838] Updated weights for policy 0, policy_version 68042 (0.0007) [2023-10-07 22:32:59,795][67871] Updated weights for policy 1, policy_version 68150 (0.0009) [2023-10-07 22:32:59,966][67838] Updated weights for policy 0, policy_version 68052 (0.0009) [2023-10-07 22:33:00,160][67871] Updated weights for policy 1, policy_version 68160 (0.0008) [2023-10-07 22:33:00,327][67838] Updated weights for policy 0, policy_version 68062 (0.0008) [2023-10-07 22:33:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 139493376. Throughput: 0: 1656.3, 1: 1663.5. Samples: 34879832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:33:02,477][66916] Avg episode reward: [(0, '53.050'), (1, '55.540')] [2023-10-07 22:33:04,336][67871] Updated weights for policy 1, policy_version 68170 (0.0009) [2023-10-07 22:33:04,515][67838] Updated weights for policy 0, policy_version 68072 (0.0008) [2023-10-07 22:33:04,699][67871] Updated weights for policy 1, policy_version 68180 (0.0008) [2023-10-07 22:33:04,888][67838] Updated weights for policy 0, policy_version 68082 (0.0008) [2023-10-07 22:33:05,073][67871] Updated weights for policy 1, policy_version 68190 (0.0009) [2023-10-07 22:33:05,263][67838] Updated weights for policy 0, policy_version 68092 (0.0007) [2023-10-07 22:33:07,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.3). Total num frames: 139558912. Throughput: 0: 1655.9, 1: 1660.0. Samples: 34900128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:33:07,477][66916] Avg episode reward: [(0, '54.250'), (1, '57.210')] [2023-10-07 22:33:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000068192_69828608.pth... [2023-10-07 22:33:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000068096_69730304.pth... [2023-10-07 22:33:07,520][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000066560_68157440.pth [2023-10-07 22:33:07,529][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000066656_68255744.pth [2023-10-07 22:33:09,243][67838] Updated weights for policy 0, policy_version 68102 (0.0008) [2023-10-07 22:33:09,261][67871] Updated weights for policy 1, policy_version 68200 (0.0007) [2023-10-07 22:33:09,614][67838] Updated weights for policy 0, policy_version 68112 (0.0009) [2023-10-07 22:33:09,628][67871] Updated weights for policy 1, policy_version 68210 (0.0007) [2023-10-07 22:33:09,995][67871] Updated weights for policy 1, policy_version 68220 (0.0010) [2023-10-07 22:33:09,995][67838] Updated weights for policy 0, policy_version 68122 (0.0007) [2023-10-07 22:33:12,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 139624448. Throughput: 0: 1644.9, 1: 1652.6. Samples: 34909860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:33:12,478][66916] Avg episode reward: [(0, '50.010'), (1, '57.380')] [2023-10-07 22:33:14,093][67871] Updated weights for policy 1, policy_version 68230 (0.0009) [2023-10-07 22:33:14,165][67838] Updated weights for policy 0, policy_version 68132 (0.0007) [2023-10-07 22:33:14,454][67871] Updated weights for policy 1, policy_version 68240 (0.0007) [2023-10-07 22:33:14,545][67838] Updated weights for policy 0, policy_version 68142 (0.0009) [2023-10-07 22:33:14,817][67871] Updated weights for policy 1, policy_version 68250 (0.0008) [2023-10-07 22:33:14,905][67838] Updated weights for policy 0, policy_version 68152 (0.0010) [2023-10-07 22:33:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 139689984. Throughput: 0: 1654.0, 1: 1656.2. Samples: 34929546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:33:17,478][66916] Avg episode reward: [(0, '48.460'), (1, '58.080')] [2023-10-07 22:33:18,834][67871] Updated weights for policy 1, policy_version 68260 (0.0008) [2023-10-07 22:33:19,061][67838] Updated weights for policy 0, policy_version 68162 (0.0010) [2023-10-07 22:33:19,198][67871] Updated weights for policy 1, policy_version 68270 (0.0009) [2023-10-07 22:33:19,437][67838] Updated weights for policy 0, policy_version 68172 (0.0007) [2023-10-07 22:33:19,562][67871] Updated weights for policy 1, policy_version 68280 (0.0008) [2023-10-07 22:33:19,802][67838] Updated weights for policy 0, policy_version 68182 (0.0008) [2023-10-07 22:33:20,174][67838] Updated weights for policy 0, policy_version 68192 (0.0010) [2023-10-07 22:33:22,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 139755520. Throughput: 0: 1651.9, 1: 1657.5. Samples: 34949866. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:33:22,478][66916] Avg episode reward: [(0, '47.530'), (1, '57.060')] [2023-10-07 22:33:23,650][67871] Updated weights for policy 1, policy_version 68290 (0.0008) [2023-10-07 22:33:24,077][67871] Updated weights for policy 1, policy_version 68300 (0.0008) [2023-10-07 22:33:24,376][67838] Updated weights for policy 0, policy_version 68202 (0.0007) [2023-10-07 22:33:24,450][67871] Updated weights for policy 1, policy_version 68310 (0.0007) [2023-10-07 22:33:24,755][67838] Updated weights for policy 0, policy_version 68212 (0.0008) [2023-10-07 22:33:24,816][67871] Updated weights for policy 1, policy_version 68320 (0.0007) [2023-10-07 22:33:25,136][67838] Updated weights for policy 0, policy_version 68222 (0.0011) [2023-10-07 22:33:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 139821056. Throughput: 0: 1642.2, 1: 1645.7. Samples: 34959038. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 22:33:27,478][66916] Avg episode reward: [(0, '46.980'), (1, '59.000')] [2023-10-07 22:33:29,079][67871] Updated weights for policy 1, policy_version 68330 (0.0009) [2023-10-07 22:33:29,206][67838] Updated weights for policy 0, policy_version 68232 (0.0008) [2023-10-07 22:33:29,440][67871] Updated weights for policy 1, policy_version 68340 (0.0008) [2023-10-07 22:33:29,575][67838] Updated weights for policy 0, policy_version 68242 (0.0009) [2023-10-07 22:33:29,807][67871] Updated weights for policy 1, policy_version 68350 (0.0007) [2023-10-07 22:33:29,946][67838] Updated weights for policy 0, policy_version 68252 (0.0008) [2023-10-07 22:33:32,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 139886592. Throughput: 0: 1652.6, 1: 1652.2. Samples: 34978968. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 22:33:32,478][66916] Avg episode reward: [(0, '49.220'), (1, '57.880')] [2023-10-07 22:33:33,963][67871] Updated weights for policy 1, policy_version 68360 (0.0009) [2023-10-07 22:33:34,164][67838] Updated weights for policy 0, policy_version 68262 (0.0008) [2023-10-07 22:33:34,336][67871] Updated weights for policy 1, policy_version 68370 (0.0009) [2023-10-07 22:33:34,527][67838] Updated weights for policy 0, policy_version 68272 (0.0008) [2023-10-07 22:33:34,707][67871] Updated weights for policy 1, policy_version 68380 (0.0008) [2023-10-07 22:33:34,905][67838] Updated weights for policy 0, policy_version 68282 (0.0007) [2023-10-07 22:33:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 139952128. Throughput: 0: 1651.2, 1: 1651.3. Samples: 34999292. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 22:33:37,477][66916] Avg episode reward: [(0, '50.180'), (1, '58.300')] [2023-10-07 22:33:38,872][67871] Updated weights for policy 1, policy_version 68390 (0.0007) [2023-10-07 22:33:39,189][67838] Updated weights for policy 0, policy_version 68292 (0.0007) [2023-10-07 22:33:39,237][67871] Updated weights for policy 1, policy_version 68400 (0.0007) [2023-10-07 22:33:39,571][67838] Updated weights for policy 0, policy_version 68302 (0.0007) [2023-10-07 22:33:39,604][67871] Updated weights for policy 1, policy_version 68410 (0.0008) [2023-10-07 22:33:39,939][67838] Updated weights for policy 0, policy_version 68312 (0.0007) [2023-10-07 22:33:42,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 140017664. Throughput: 0: 1644.4, 1: 1648.0. Samples: 35008658. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 22:33:42,478][66916] Avg episode reward: [(0, '48.450'), (1, '57.050')] [2023-10-07 22:33:43,517][67871] Updated weights for policy 1, policy_version 68420 (0.0009) [2023-10-07 22:33:43,892][67871] Updated weights for policy 1, policy_version 68430 (0.0007) [2023-10-07 22:33:44,028][67838] Updated weights for policy 0, policy_version 68322 (0.0007) [2023-10-07 22:33:44,252][67871] Updated weights for policy 1, policy_version 68440 (0.0008) [2023-10-07 22:33:44,396][67838] Updated weights for policy 0, policy_version 68332 (0.0008) [2023-10-07 22:33:44,777][67838] Updated weights for policy 0, policy_version 68342 (0.0010) [2023-10-07 22:33:45,147][67838] Updated weights for policy 0, policy_version 68352 (0.0009) [2023-10-07 22:33:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 140083200. Throughput: 0: 1652.8, 1: 1659.0. Samples: 35028860. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 22:33:47,477][66916] Avg episode reward: [(0, '49.710'), (1, '53.920')] [2023-10-07 22:33:48,432][67871] Updated weights for policy 1, policy_version 68450 (0.0008) [2023-10-07 22:33:48,791][67871] Updated weights for policy 1, policy_version 68460 (0.0009) [2023-10-07 22:33:49,042][67838] Updated weights for policy 0, policy_version 68362 (0.0008) [2023-10-07 22:33:49,163][67871] Updated weights for policy 1, policy_version 68470 (0.0009) [2023-10-07 22:33:49,412][67838] Updated weights for policy 0, policy_version 68372 (0.0009) [2023-10-07 22:33:49,524][67871] Updated weights for policy 1, policy_version 68480 (0.0008) [2023-10-07 22:33:49,778][67838] Updated weights for policy 0, policy_version 68382 (0.0007) [2023-10-07 22:33:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 140148736. Throughput: 0: 1656.0, 1: 1660.8. Samples: 35049380. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 22:33:52,478][66916] Avg episode reward: [(0, '47.400'), (1, '53.940')] [2023-10-07 22:33:53,688][67871] Updated weights for policy 1, policy_version 68490 (0.0007) [2023-10-07 22:33:54,021][67838] Updated weights for policy 0, policy_version 68392 (0.0007) [2023-10-07 22:33:54,057][67871] Updated weights for policy 1, policy_version 68500 (0.0008) [2023-10-07 22:33:54,391][67838] Updated weights for policy 0, policy_version 68402 (0.0007) [2023-10-07 22:33:54,423][67871] Updated weights for policy 1, policy_version 68510 (0.0007) [2023-10-07 22:33:54,765][67838] Updated weights for policy 0, policy_version 68412 (0.0008) [2023-10-07 22:33:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 140214272. Throughput: 0: 1646.1, 1: 1651.9. Samples: 35058272. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 22:33:57,477][66916] Avg episode reward: [(0, '47.880'), (1, '51.900')] [2023-10-07 22:33:58,446][67871] Updated weights for policy 1, policy_version 68520 (0.0007) [2023-10-07 22:33:58,816][67871] Updated weights for policy 1, policy_version 68530 (0.0008) [2023-10-07 22:33:59,077][67838] Updated weights for policy 0, policy_version 68422 (0.0007) [2023-10-07 22:33:59,190][67871] Updated weights for policy 1, policy_version 68540 (0.0009) [2023-10-07 22:33:59,446][67838] Updated weights for policy 0, policy_version 68432 (0.0008) [2023-10-07 22:33:59,823][67838] Updated weights for policy 0, policy_version 68442 (0.0009) [2023-10-07 22:34:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 140279808. Throughput: 0: 1644.3, 1: 1662.9. Samples: 35078368. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 22:34:02,477][66916] Avg episode reward: [(0, '49.090'), (1, '50.480')] [2023-10-07 22:34:03,300][67871] Updated weights for policy 1, policy_version 68550 (0.0008) [2023-10-07 22:34:03,665][67871] Updated weights for policy 1, policy_version 68560 (0.0008) [2023-10-07 22:34:03,932][67838] Updated weights for policy 0, policy_version 68452 (0.0008) [2023-10-07 22:34:04,034][67871] Updated weights for policy 1, policy_version 68570 (0.0009) [2023-10-07 22:34:04,312][67838] Updated weights for policy 0, policy_version 68462 (0.0008) [2023-10-07 22:34:04,678][67838] Updated weights for policy 0, policy_version 68472 (0.0007) [2023-10-07 22:34:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 140345344. Throughput: 0: 1644.6, 1: 1674.7. Samples: 35099236. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-07 22:34:07,478][66916] Avg episode reward: [(0, '45.380'), (1, '51.930')] [2023-10-07 22:34:07,818][67871] Updated weights for policy 1, policy_version 68580 (0.0007) [2023-10-07 22:34:08,214][67871] Updated weights for policy 1, policy_version 68590 (0.0007) [2023-10-07 22:34:08,587][67871] Updated weights for policy 1, policy_version 68600 (0.0007) [2023-10-07 22:34:08,862][67838] Updated weights for policy 0, policy_version 68482 (0.0007) [2023-10-07 22:34:09,253][67838] Updated weights for policy 0, policy_version 68492 (0.0010) [2023-10-07 22:34:09,630][67838] Updated weights for policy 0, policy_version 68502 (0.0011) [2023-10-07 22:34:10,001][67838] Updated weights for policy 0, policy_version 68512 (0.0007) [2023-10-07 22:34:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 140410880. Throughput: 0: 1637.8, 1: 1674.7. Samples: 35108100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:34:12,477][66916] Avg episode reward: [(0, '46.710'), (1, '48.690')] [2023-10-07 22:34:12,707][67871] Updated weights for policy 1, policy_version 68610 (0.0008) [2023-10-07 22:34:13,077][67871] Updated weights for policy 1, policy_version 68620 (0.0007) [2023-10-07 22:34:13,453][67871] Updated weights for policy 1, policy_version 68630 (0.0009) [2023-10-07 22:34:13,810][67871] Updated weights for policy 1, policy_version 68640 (0.0009) [2023-10-07 22:34:14,058][67838] Updated weights for policy 0, policy_version 68522 (0.0008) [2023-10-07 22:34:14,436][67838] Updated weights for policy 0, policy_version 68532 (0.0009) [2023-10-07 22:34:14,810][67838] Updated weights for policy 0, policy_version 68542 (0.0009) [2023-10-07 22:34:17,477][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 140476416. Throughput: 0: 1642.1, 1: 1680.8. Samples: 35128502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:34:17,478][66916] Avg episode reward: [(0, '47.140'), (1, '47.050')] [2023-10-07 22:34:17,922][67871] Updated weights for policy 1, policy_version 68650 (0.0008) [2023-10-07 22:34:18,287][67871] Updated weights for policy 1, policy_version 68660 (0.0008) [2023-10-07 22:34:18,651][67871] Updated weights for policy 1, policy_version 68670 (0.0008) [2023-10-07 22:34:18,959][67838] Updated weights for policy 0, policy_version 68552 (0.0010) [2023-10-07 22:34:19,331][67838] Updated weights for policy 0, policy_version 68562 (0.0007) [2023-10-07 22:34:19,705][67838] Updated weights for policy 0, policy_version 68572 (0.0009) [2023-10-07 22:34:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 140541952. Throughput: 0: 1641.1, 1: 1683.6. Samples: 35148904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:34:22,477][66916] Avg episode reward: [(0, '49.200'), (1, '50.680')] [2023-10-07 22:34:22,698][67871] Updated weights for policy 1, policy_version 68680 (0.0010) [2023-10-07 22:34:23,068][67871] Updated weights for policy 1, policy_version 68690 (0.0008) [2023-10-07 22:34:23,437][67871] Updated weights for policy 1, policy_version 68700 (0.0010) [2023-10-07 22:34:23,897][67838] Updated weights for policy 0, policy_version 68582 (0.0007) [2023-10-07 22:34:24,257][67838] Updated weights for policy 0, policy_version 68592 (0.0009) [2023-10-07 22:34:24,630][67838] Updated weights for policy 0, policy_version 68602 (0.0009) [2023-10-07 22:34:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 140607488. Throughput: 0: 1639.2, 1: 1679.3. Samples: 35157990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:34:27,477][66916] Avg episode reward: [(0, '47.820'), (1, '51.500')] [2023-10-07 22:34:27,522][67871] Updated weights for policy 1, policy_version 68710 (0.0010) [2023-10-07 22:34:27,890][67871] Updated weights for policy 1, policy_version 68720 (0.0008) [2023-10-07 22:34:28,264][67871] Updated weights for policy 1, policy_version 68730 (0.0010) [2023-10-07 22:34:28,712][67838] Updated weights for policy 0, policy_version 68612 (0.0008) [2023-10-07 22:34:29,081][67838] Updated weights for policy 0, policy_version 68622 (0.0010) [2023-10-07 22:34:29,455][67838] Updated weights for policy 0, policy_version 68632 (0.0010) [2023-10-07 22:34:32,239][67871] Updated weights for policy 1, policy_version 68740 (0.0008) [2023-10-07 22:34:32,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 140673024. Throughput: 0: 1653.7, 1: 1681.2. Samples: 35178932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:34:32,477][66916] Avg episode reward: [(0, '49.500'), (1, '53.640')] [2023-10-07 22:34:32,602][67871] Updated weights for policy 1, policy_version 68750 (0.0010) [2023-10-07 22:34:32,969][67871] Updated weights for policy 1, policy_version 68760 (0.0010) [2023-10-07 22:34:33,644][67838] Updated weights for policy 0, policy_version 68642 (0.0008) [2023-10-07 22:34:34,008][67838] Updated weights for policy 0, policy_version 68652 (0.0010) [2023-10-07 22:34:34,376][67838] Updated weights for policy 0, policy_version 68662 (0.0011) [2023-10-07 22:34:34,745][67838] Updated weights for policy 0, policy_version 68672 (0.0011) [2023-10-07 22:34:37,371][67871] Updated weights for policy 1, policy_version 68770 (0.0010) [2023-10-07 22:34:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 140738560. Throughput: 0: 1656.0, 1: 1677.9. Samples: 35199404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:34:37,477][66916] Avg episode reward: [(0, '48.890'), (1, '51.690')] [2023-10-07 22:34:37,736][67871] Updated weights for policy 1, policy_version 68780 (0.0007) [2023-10-07 22:34:38,107][67871] Updated weights for policy 1, policy_version 68790 (0.0007) [2023-10-07 22:34:38,484][67871] Updated weights for policy 1, policy_version 68800 (0.0008) [2023-10-07 22:34:38,862][67838] Updated weights for policy 0, policy_version 68682 (0.0010) [2023-10-07 22:34:39,237][67838] Updated weights for policy 0, policy_version 68692 (0.0008) [2023-10-07 22:34:39,605][67838] Updated weights for policy 0, policy_version 68702 (0.0008) [2023-10-07 22:34:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 140804096. Throughput: 0: 1658.5, 1: 1676.9. Samples: 35208368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:34:42,477][66916] Avg episode reward: [(0, '46.930'), (1, '51.780')] [2023-10-07 22:34:42,700][67871] Updated weights for policy 1, policy_version 68810 (0.0009) [2023-10-07 22:34:43,068][67871] Updated weights for policy 1, policy_version 68820 (0.0009) [2023-10-07 22:34:43,445][67871] Updated weights for policy 1, policy_version 68830 (0.0009) [2023-10-07 22:34:43,576][67838] Updated weights for policy 0, policy_version 68712 (0.0008) [2023-10-07 22:34:43,945][67838] Updated weights for policy 0, policy_version 68722 (0.0008) [2023-10-07 22:34:44,322][67838] Updated weights for policy 0, policy_version 68732 (0.0010) [2023-10-07 22:34:47,440][67871] Updated weights for policy 1, policy_version 68840 (0.0009) [2023-10-07 22:34:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 140869632. Throughput: 0: 1672.6, 1: 1673.5. Samples: 35228942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:34:47,477][66916] Avg episode reward: [(0, '47.650'), (1, '49.620')] [2023-10-07 22:34:47,810][67871] Updated weights for policy 1, policy_version 68850 (0.0010) [2023-10-07 22:34:48,170][67871] Updated weights for policy 1, policy_version 68860 (0.0009) [2023-10-07 22:34:48,472][67838] Updated weights for policy 0, policy_version 68742 (0.0008) [2023-10-07 22:34:48,848][67838] Updated weights for policy 0, policy_version 68752 (0.0009) [2023-10-07 22:34:49,223][67838] Updated weights for policy 0, policy_version 68762 (0.0008) [2023-10-07 22:34:52,368][67871] Updated weights for policy 1, policy_version 68870 (0.0007) [2023-10-07 22:34:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 140935168. Throughput: 0: 1673.7, 1: 1666.5. Samples: 35249546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:34:52,477][66916] Avg episode reward: [(0, '46.580'), (1, '49.570')] [2023-10-07 22:34:52,740][67871] Updated weights for policy 1, policy_version 68880 (0.0009) [2023-10-07 22:34:53,110][67871] Updated weights for policy 1, policy_version 68890 (0.0009) [2023-10-07 22:34:53,468][67838] Updated weights for policy 0, policy_version 68772 (0.0008) [2023-10-07 22:34:53,843][67838] Updated weights for policy 0, policy_version 68782 (0.0010) [2023-10-07 22:34:54,221][67838] Updated weights for policy 0, policy_version 68792 (0.0010) [2023-10-07 22:34:57,344][67871] Updated weights for policy 1, policy_version 68900 (0.0009) [2023-10-07 22:34:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141000704. Throughput: 0: 1676.3, 1: 1666.2. Samples: 35258514. Policy #0 lag: (min: 28.0, avg: 52.6, max: 56.0) [2023-10-07 22:34:57,477][66916] Avg episode reward: [(0, '48.140'), (1, '50.340')] [2023-10-07 22:34:57,742][67871] Updated weights for policy 1, policy_version 68910 (0.0008) [2023-10-07 22:34:58,114][67871] Updated weights for policy 1, policy_version 68920 (0.0010) [2023-10-07 22:34:58,259][67838] Updated weights for policy 0, policy_version 68802 (0.0008) [2023-10-07 22:34:58,646][67838] Updated weights for policy 0, policy_version 68812 (0.0010) [2023-10-07 22:34:59,020][67838] Updated weights for policy 0, policy_version 68822 (0.0010) [2023-10-07 22:34:59,395][67838] Updated weights for policy 0, policy_version 68832 (0.0010) [2023-10-07 22:35:02,422][67871] Updated weights for policy 1, policy_version 68930 (0.0007) [2023-10-07 22:35:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141066240. Throughput: 0: 1676.8, 1: 1663.7. Samples: 35278822. Policy #0 lag: (min: 28.0, avg: 52.6, max: 56.0) [2023-10-07 22:35:02,477][66916] Avg episode reward: [(0, '50.840'), (1, '49.460')] [2023-10-07 22:35:02,793][67871] Updated weights for policy 1, policy_version 68940 (0.0007) [2023-10-07 22:35:03,160][67871] Updated weights for policy 1, policy_version 68950 (0.0009) [2023-10-07 22:35:03,486][67838] Updated weights for policy 0, policy_version 68842 (0.0007) [2023-10-07 22:35:03,519][67871] Updated weights for policy 1, policy_version 68960 (0.0009) [2023-10-07 22:35:03,852][67838] Updated weights for policy 0, policy_version 68852 (0.0007) [2023-10-07 22:35:04,231][67838] Updated weights for policy 0, policy_version 68862 (0.0008) [2023-10-07 22:35:07,467][67871] Updated weights for policy 1, policy_version 68970 (0.0007) [2023-10-07 22:35:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 141131776. Throughput: 0: 1680.6, 1: 1660.9. Samples: 35299270. Policy #0 lag: (min: 28.0, avg: 52.6, max: 56.0) [2023-10-07 22:35:07,477][66916] Avg episode reward: [(0, '49.860'), (1, '54.220')] [2023-10-07 22:35:07,485][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000068864_70516736.pth... [2023-10-07 22:35:07,523][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000067328_68943872.pth [2023-10-07 22:35:07,837][67871] Updated weights for policy 1, policy_version 68980 (0.0009) [2023-10-07 22:35:08,206][67871] Updated weights for policy 1, policy_version 68990 (0.0009) [2023-10-07 22:35:08,271][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000068992_70647808.pth... [2023-10-07 22:35:08,302][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000067424_69042176.pth [2023-10-07 22:35:08,478][67838] Updated weights for policy 0, policy_version 68872 (0.0009) [2023-10-07 22:35:08,848][67838] Updated weights for policy 0, policy_version 68882 (0.0009) [2023-10-07 22:35:09,228][67838] Updated weights for policy 0, policy_version 68892 (0.0009) [2023-10-07 22:35:12,242][67871] Updated weights for policy 1, policy_version 69000 (0.0009) [2023-10-07 22:35:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141197312. Throughput: 0: 1677.7, 1: 1665.9. Samples: 35308454. Policy #0 lag: (min: 28.0, avg: 52.6, max: 56.0) [2023-10-07 22:35:12,478][66916] Avg episode reward: [(0, '49.470'), (1, '55.960')] [2023-10-07 22:35:12,611][67871] Updated weights for policy 1, policy_version 69010 (0.0009) [2023-10-07 22:35:12,971][67871] Updated weights for policy 1, policy_version 69020 (0.0008) [2023-10-07 22:35:13,387][67838] Updated weights for policy 0, policy_version 68902 (0.0009) [2023-10-07 22:35:13,760][67838] Updated weights for policy 0, policy_version 68912 (0.0008) [2023-10-07 22:35:14,133][67838] Updated weights for policy 0, policy_version 68922 (0.0008) [2023-10-07 22:35:16,954][67871] Updated weights for policy 1, policy_version 69030 (0.0009) [2023-10-07 22:35:17,330][67871] Updated weights for policy 1, policy_version 69040 (0.0009) [2023-10-07 22:35:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141262848. Throughput: 0: 1667.6, 1: 1665.3. Samples: 35328914. Policy #0 lag: (min: 28.0, avg: 52.6, max: 56.0) [2023-10-07 22:35:17,477][66916] Avg episode reward: [(0, '50.010'), (1, '56.430')] [2023-10-07 22:35:17,707][67871] Updated weights for policy 1, policy_version 69050 (0.0009) [2023-10-07 22:35:18,077][67838] Updated weights for policy 0, policy_version 68932 (0.0011) [2023-10-07 22:35:18,464][67838] Updated weights for policy 0, policy_version 68942 (0.0010) [2023-10-07 22:35:18,823][67838] Updated weights for policy 0, policy_version 68952 (0.0009) [2023-10-07 22:35:21,914][67871] Updated weights for policy 1, policy_version 69060 (0.0009) [2023-10-07 22:35:22,277][67871] Updated weights for policy 1, policy_version 69070 (0.0009) [2023-10-07 22:35:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 141328384. Throughput: 0: 1665.3, 1: 1666.6. Samples: 35349340. Policy #0 lag: (min: 28.0, avg: 52.6, max: 56.0) [2023-10-07 22:35:22,477][66916] Avg episode reward: [(0, '47.740'), (1, '57.400')] [2023-10-07 22:35:22,642][67871] Updated weights for policy 1, policy_version 69080 (0.0007) [2023-10-07 22:35:22,968][67838] Updated weights for policy 0, policy_version 68962 (0.0010) [2023-10-07 22:35:23,341][67838] Updated weights for policy 0, policy_version 68972 (0.0007) [2023-10-07 22:35:23,710][67838] Updated weights for policy 0, policy_version 68982 (0.0007) [2023-10-07 22:35:24,077][67838] Updated weights for policy 0, policy_version 68992 (0.0007) [2023-10-07 22:35:26,663][67871] Updated weights for policy 1, policy_version 69090 (0.0008) [2023-10-07 22:35:27,034][67871] Updated weights for policy 1, policy_version 69100 (0.0007) [2023-10-07 22:35:27,402][67871] Updated weights for policy 1, policy_version 69110 (0.0008) [2023-10-07 22:35:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 141393920. Throughput: 0: 1667.3, 1: 1671.6. Samples: 35358618. Policy #0 lag: (min: 28.0, avg: 52.6, max: 56.0) [2023-10-07 22:35:27,477][66916] Avg episode reward: [(0, '48.720'), (1, '57.490')] [2023-10-07 22:35:27,782][67871] Updated weights for policy 1, policy_version 69120 (0.0007) [2023-10-07 22:35:28,101][67838] Updated weights for policy 0, policy_version 69002 (0.0009) [2023-10-07 22:35:28,472][67838] Updated weights for policy 0, policy_version 69012 (0.0011) [2023-10-07 22:35:28,846][67838] Updated weights for policy 0, policy_version 69022 (0.0011) [2023-10-07 22:35:32,027][67871] Updated weights for policy 1, policy_version 69130 (0.0008) [2023-10-07 22:35:32,399][67871] Updated weights for policy 1, policy_version 69140 (0.0008) [2023-10-07 22:35:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 141459456. Throughput: 0: 1658.4, 1: 1670.8. Samples: 35378756. Policy #0 lag: (min: 28.0, avg: 52.6, max: 56.0) [2023-10-07 22:35:32,477][66916] Avg episode reward: [(0, '47.060'), (1, '56.490')] [2023-10-07 22:35:32,769][67871] Updated weights for policy 1, policy_version 69150 (0.0009) [2023-10-07 22:35:32,878][67838] Updated weights for policy 0, policy_version 69032 (0.0009) [2023-10-07 22:35:33,247][67838] Updated weights for policy 0, policy_version 69042 (0.0008) [2023-10-07 22:35:33,614][67838] Updated weights for policy 0, policy_version 69052 (0.0011) [2023-10-07 22:35:36,820][67871] Updated weights for policy 1, policy_version 69160 (0.0008) [2023-10-07 22:35:37,190][67871] Updated weights for policy 1, policy_version 69170 (0.0011) [2023-10-07 22:35:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 141524992. Throughput: 0: 1656.6, 1: 1660.5. Samples: 35398816. Policy #0 lag: (min: 28.0, avg: 52.6, max: 56.0) [2023-10-07 22:35:37,477][66916] Avg episode reward: [(0, '46.150'), (1, '51.960')] [2023-10-07 22:35:37,559][67871] Updated weights for policy 1, policy_version 69180 (0.0009) [2023-10-07 22:35:37,776][67838] Updated weights for policy 0, policy_version 69062 (0.0008) [2023-10-07 22:35:38,142][67838] Updated weights for policy 0, policy_version 69072 (0.0008) [2023-10-07 22:35:38,511][67838] Updated weights for policy 0, policy_version 69082 (0.0008) [2023-10-07 22:35:41,774][67871] Updated weights for policy 1, policy_version 69190 (0.0008) [2023-10-07 22:35:42,140][67871] Updated weights for policy 1, policy_version 69200 (0.0009) [2023-10-07 22:35:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 141590528. Throughput: 0: 1656.7, 1: 1665.6. Samples: 35408014. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 22:35:42,477][66916] Avg episode reward: [(0, '46.740'), (1, '54.280')] [2023-10-07 22:35:42,507][67871] Updated weights for policy 1, policy_version 69210 (0.0008) [2023-10-07 22:35:42,625][67838] Updated weights for policy 0, policy_version 69092 (0.0010) [2023-10-07 22:35:43,000][67838] Updated weights for policy 0, policy_version 69102 (0.0010) [2023-10-07 22:35:43,373][67838] Updated weights for policy 0, policy_version 69112 (0.0010) [2023-10-07 22:35:46,624][67871] Updated weights for policy 1, policy_version 69220 (0.0009) [2023-10-07 22:35:47,007][67871] Updated weights for policy 1, policy_version 69230 (0.0007) [2023-10-07 22:35:47,375][67871] Updated weights for policy 1, policy_version 69240 (0.0010) [2023-10-07 22:35:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 141656064. Throughput: 0: 1655.2, 1: 1669.7. Samples: 35428446. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 22:35:47,477][66916] Avg episode reward: [(0, '42.970'), (1, '55.410')] [2023-10-07 22:35:47,622][67838] Updated weights for policy 0, policy_version 69122 (0.0009) [2023-10-07 22:35:47,994][67838] Updated weights for policy 0, policy_version 69132 (0.0008) [2023-10-07 22:35:48,371][67838] Updated weights for policy 0, policy_version 69142 (0.0011) [2023-10-07 22:35:48,734][67838] Updated weights for policy 0, policy_version 69152 (0.0012) [2023-10-07 22:35:51,407][67871] Updated weights for policy 1, policy_version 69250 (0.0008) [2023-10-07 22:35:51,770][67871] Updated weights for policy 1, policy_version 69260 (0.0009) [2023-10-07 22:35:52,135][67871] Updated weights for policy 1, policy_version 69270 (0.0009) [2023-10-07 22:35:52,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 141721600. Throughput: 0: 1651.2, 1: 1656.7. Samples: 35448124. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 22:35:52,477][66916] Avg episode reward: [(0, '46.000'), (1, '58.140')] [2023-10-07 22:35:52,500][67871] Updated weights for policy 1, policy_version 69280 (0.0009) [2023-10-07 22:35:52,895][67838] Updated weights for policy 0, policy_version 69162 (0.0011) [2023-10-07 22:35:53,249][67838] Updated weights for policy 0, policy_version 69172 (0.0011) [2023-10-07 22:35:53,617][67838] Updated weights for policy 0, policy_version 69182 (0.0007) [2023-10-07 22:35:56,617][67871] Updated weights for policy 1, policy_version 69290 (0.0007) [2023-10-07 22:35:56,984][67871] Updated weights for policy 1, policy_version 69300 (0.0007) [2023-10-07 22:35:57,358][67871] Updated weights for policy 1, policy_version 69310 (0.0007) [2023-10-07 22:35:57,477][66916] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 141819904. Throughput: 0: 1651.4, 1: 1668.2. Samples: 35457834. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 22:35:57,478][66916] Avg episode reward: [(0, '47.370'), (1, '57.460')] [2023-10-07 22:35:57,719][67838] Updated weights for policy 0, policy_version 69192 (0.0011) [2023-10-07 22:35:58,093][67838] Updated weights for policy 0, policy_version 69202 (0.0011) [2023-10-07 22:35:58,467][67838] Updated weights for policy 0, policy_version 69212 (0.0010) [2023-10-07 22:36:01,432][67871] Updated weights for policy 1, policy_version 69320 (0.0010) [2023-10-07 22:36:01,800][67871] Updated weights for policy 1, policy_version 69330 (0.0009) [2023-10-07 22:36:02,155][67871] Updated weights for policy 1, policy_version 69340 (0.0010) [2023-10-07 22:36:02,477][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 141885440. Throughput: 0: 1651.1, 1: 1665.3. Samples: 35478152. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 22:36:02,478][66916] Avg episode reward: [(0, '47.120'), (1, '59.720')] [2023-10-07 22:36:02,647][67838] Updated weights for policy 0, policy_version 69222 (0.0009) [2023-10-07 22:36:03,028][67838] Updated weights for policy 0, policy_version 69232 (0.0009) [2023-10-07 22:36:03,409][67838] Updated weights for policy 0, policy_version 69242 (0.0011) [2023-10-07 22:36:06,220][67871] Updated weights for policy 1, policy_version 69350 (0.0009) [2023-10-07 22:36:06,590][67871] Updated weights for policy 1, policy_version 69360 (0.0009) [2023-10-07 22:36:06,958][67871] Updated weights for policy 1, policy_version 69370 (0.0009) [2023-10-07 22:36:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 141950976. Throughput: 0: 1647.8, 1: 1650.8. Samples: 35497774. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 22:36:07,477][66916] Avg episode reward: [(0, '47.200'), (1, '58.580')] [2023-10-07 22:36:07,673][67838] Updated weights for policy 0, policy_version 69252 (0.0008) [2023-10-07 22:36:08,042][67838] Updated weights for policy 0, policy_version 69262 (0.0007) [2023-10-07 22:36:08,407][67838] Updated weights for policy 0, policy_version 69272 (0.0009) [2023-10-07 22:36:11,128][67871] Updated weights for policy 1, policy_version 69380 (0.0010) [2023-10-07 22:36:11,507][67871] Updated weights for policy 1, policy_version 69390 (0.0010) [2023-10-07 22:36:11,868][67871] Updated weights for policy 1, policy_version 69400 (0.0012) [2023-10-07 22:36:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 142016512. Throughput: 0: 1643.6, 1: 1664.2. Samples: 35507468. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 22:36:12,478][66916] Avg episode reward: [(0, '47.260'), (1, '57.720')] [2023-10-07 22:36:12,524][67838] Updated weights for policy 0, policy_version 69282 (0.0008) [2023-10-07 22:36:12,898][67838] Updated weights for policy 0, policy_version 69292 (0.0010) [2023-10-07 22:36:13,269][67838] Updated weights for policy 0, policy_version 69302 (0.0010) [2023-10-07 22:36:13,636][67838] Updated weights for policy 0, policy_version 69312 (0.0009) [2023-10-07 22:36:15,933][67871] Updated weights for policy 1, policy_version 69410 (0.0009) [2023-10-07 22:36:16,299][67871] Updated weights for policy 1, policy_version 69420 (0.0009) [2023-10-07 22:36:16,666][67871] Updated weights for policy 1, policy_version 69430 (0.0009) [2023-10-07 22:36:17,027][67871] Updated weights for policy 1, policy_version 69440 (0.0008) [2023-10-07 22:36:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 142082048. Throughput: 0: 1645.2, 1: 1666.0. Samples: 35527760. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 22:36:17,478][66916] Avg episode reward: [(0, '47.290'), (1, '56.270')] [2023-10-07 22:36:17,854][67838] Updated weights for policy 0, policy_version 69322 (0.0007) [2023-10-07 22:36:18,228][67838] Updated weights for policy 0, policy_version 69332 (0.0007) [2023-10-07 22:36:18,597][67838] Updated weights for policy 0, policy_version 69342 (0.0009) [2023-10-07 22:36:21,129][67871] Updated weights for policy 1, policy_version 69450 (0.0009) [2023-10-07 22:36:21,490][67871] Updated weights for policy 1, policy_version 69460 (0.0007) [2023-10-07 22:36:21,867][67871] Updated weights for policy 1, policy_version 69470 (0.0008) [2023-10-07 22:36:22,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 142147584. Throughput: 0: 1648.3, 1: 1648.8. Samples: 35547186. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-07 22:36:22,478][66916] Avg episode reward: [(0, '44.000'), (1, '55.300')] [2023-10-07 22:36:22,693][67838] Updated weights for policy 0, policy_version 69352 (0.0009) [2023-10-07 22:36:23,064][67838] Updated weights for policy 0, policy_version 69362 (0.0007) [2023-10-07 22:36:23,440][67838] Updated weights for policy 0, policy_version 69372 (0.0008) [2023-10-07 22:36:25,965][67871] Updated weights for policy 1, policy_version 69480 (0.0010) [2023-10-07 22:36:26,333][67871] Updated weights for policy 1, policy_version 69490 (0.0007) [2023-10-07 22:36:26,703][67871] Updated weights for policy 1, policy_version 69500 (0.0007) [2023-10-07 22:36:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 142213120. Throughput: 0: 1649.0, 1: 1668.8. Samples: 35557316. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 22:36:27,477][66916] Avg episode reward: [(0, '43.300'), (1, '53.810')] [2023-10-07 22:36:27,641][67838] Updated weights for policy 0, policy_version 69382 (0.0009) [2023-10-07 22:36:28,011][67838] Updated weights for policy 0, policy_version 69392 (0.0009) [2023-10-07 22:36:28,393][67838] Updated weights for policy 0, policy_version 69402 (0.0008) [2023-10-07 22:36:30,652][67871] Updated weights for policy 1, policy_version 69510 (0.0009) [2023-10-07 22:36:31,012][67871] Updated weights for policy 1, policy_version 69520 (0.0011) [2023-10-07 22:36:31,371][67871] Updated weights for policy 1, policy_version 69530 (0.0010) [2023-10-07 22:36:32,384][67838] Updated weights for policy 0, policy_version 69412 (0.0008) [2023-10-07 22:36:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 142278656. Throughput: 0: 1650.8, 1: 1663.1. Samples: 35577574. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 22:36:32,477][66916] Avg episode reward: [(0, '45.450'), (1, '57.250')] [2023-10-07 22:36:32,752][67838] Updated weights for policy 0, policy_version 69422 (0.0008) [2023-10-07 22:36:33,129][67838] Updated weights for policy 0, policy_version 69432 (0.0007) [2023-10-07 22:36:35,637][67871] Updated weights for policy 1, policy_version 69540 (0.0010) [2023-10-07 22:36:36,033][67871] Updated weights for policy 1, policy_version 69550 (0.0009) [2023-10-07 22:36:36,401][67871] Updated weights for policy 1, policy_version 69560 (0.0009) [2023-10-07 22:36:37,049][67838] Updated weights for policy 0, policy_version 69442 (0.0008) [2023-10-07 22:36:37,421][67838] Updated weights for policy 0, policy_version 69452 (0.0008) [2023-10-07 22:36:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 142344192. Throughput: 0: 1656.5, 1: 1652.5. Samples: 35597030. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 22:36:37,477][66916] Avg episode reward: [(0, '45.320'), (1, '57.730')] [2023-10-07 22:36:37,785][67838] Updated weights for policy 0, policy_version 69462 (0.0010) [2023-10-07 22:36:38,162][67838] Updated weights for policy 0, policy_version 69472 (0.0011) [2023-10-07 22:36:40,522][67871] Updated weights for policy 1, policy_version 69570 (0.0008) [2023-10-07 22:36:40,890][67871] Updated weights for policy 1, policy_version 69580 (0.0009) [2023-10-07 22:36:41,261][67871] Updated weights for policy 1, policy_version 69590 (0.0011) [2023-10-07 22:36:41,626][67871] Updated weights for policy 1, policy_version 69600 (0.0008) [2023-10-07 22:36:42,325][67838] Updated weights for policy 0, policy_version 69482 (0.0007) [2023-10-07 22:36:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 142409728. Throughput: 0: 1656.2, 1: 1666.0. Samples: 35607330. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 22:36:42,477][66916] Avg episode reward: [(0, '45.940'), (1, '56.670')] [2023-10-07 22:36:42,698][67838] Updated weights for policy 0, policy_version 69492 (0.0008) [2023-10-07 22:36:43,067][67838] Updated weights for policy 0, policy_version 69502 (0.0007) [2023-10-07 22:36:45,882][67871] Updated weights for policy 1, policy_version 69610 (0.0009) [2023-10-07 22:36:46,244][67871] Updated weights for policy 1, policy_version 69620 (0.0008) [2023-10-07 22:36:46,614][67871] Updated weights for policy 1, policy_version 69630 (0.0007) [2023-10-07 22:36:47,327][67838] Updated weights for policy 0, policy_version 69512 (0.0009) [2023-10-07 22:36:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 142475264. Throughput: 0: 1658.7, 1: 1656.4. Samples: 35627328. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 22:36:47,477][66916] Avg episode reward: [(0, '45.260'), (1, '57.420')] [2023-10-07 22:36:47,690][67838] Updated weights for policy 0, policy_version 69522 (0.0009) [2023-10-07 22:36:48,069][67838] Updated weights for policy 0, policy_version 69532 (0.0008) [2023-10-07 22:36:50,573][67871] Updated weights for policy 1, policy_version 69640 (0.0008) [2023-10-07 22:36:50,943][67871] Updated weights for policy 1, policy_version 69650 (0.0008) [2023-10-07 22:36:51,305][67871] Updated weights for policy 1, policy_version 69660 (0.0007) [2023-10-07 22:36:52,135][67838] Updated weights for policy 0, policy_version 69542 (0.0010) [2023-10-07 22:36:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 142540800. Throughput: 0: 1656.7, 1: 1656.0. Samples: 35646846. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 22:36:52,477][66916] Avg episode reward: [(0, '48.120'), (1, '56.200')] [2023-10-07 22:36:52,512][67838] Updated weights for policy 0, policy_version 69552 (0.0008) [2023-10-07 22:36:52,890][67838] Updated weights for policy 0, policy_version 69562 (0.0008) [2023-10-07 22:36:55,384][67871] Updated weights for policy 1, policy_version 69670 (0.0009) [2023-10-07 22:36:55,753][67871] Updated weights for policy 1, policy_version 69680 (0.0008) [2023-10-07 22:36:56,124][67871] Updated weights for policy 1, policy_version 69690 (0.0010) [2023-10-07 22:36:57,097][67838] Updated weights for policy 0, policy_version 69572 (0.0008) [2023-10-07 22:36:57,463][67838] Updated weights for policy 0, policy_version 69582 (0.0007) [2023-10-07 22:36:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142606336. Throughput: 0: 1660.9, 1: 1664.9. Samples: 35657130. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 22:36:57,477][66916] Avg episode reward: [(0, '45.140'), (1, '53.610')] [2023-10-07 22:36:57,828][67838] Updated weights for policy 0, policy_version 69592 (0.0010) [2023-10-07 22:37:00,238][67871] Updated weights for policy 1, policy_version 69700 (0.0007) [2023-10-07 22:37:00,615][67871] Updated weights for policy 1, policy_version 69710 (0.0008) [2023-10-07 22:37:00,981][67871] Updated weights for policy 1, policy_version 69720 (0.0011) [2023-10-07 22:37:01,897][67838] Updated weights for policy 0, policy_version 69602 (0.0008) [2023-10-07 22:37:02,273][67838] Updated weights for policy 0, policy_version 69612 (0.0007) [2023-10-07 22:37:02,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142671872. Throughput: 0: 1665.7, 1: 1652.7. Samples: 35677088. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 22:37:02,478][66916] Avg episode reward: [(0, '46.310'), (1, '53.940')] [2023-10-07 22:37:02,640][67838] Updated weights for policy 0, policy_version 69622 (0.0007) [2023-10-07 22:37:03,001][67838] Updated weights for policy 0, policy_version 69632 (0.0007) [2023-10-07 22:37:05,040][67871] Updated weights for policy 1, policy_version 69730 (0.0007) [2023-10-07 22:37:05,409][67871] Updated weights for policy 1, policy_version 69740 (0.0007) [2023-10-07 22:37:05,785][67871] Updated weights for policy 1, policy_version 69750 (0.0010) [2023-10-07 22:37:06,153][67871] Updated weights for policy 1, policy_version 69760 (0.0007) [2023-10-07 22:37:07,381][67838] Updated weights for policy 0, policy_version 69642 (0.0007) [2023-10-07 22:37:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 142737408. Throughput: 0: 1660.2, 1: 1669.5. Samples: 35697022. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-07 22:37:07,478][66916] Avg episode reward: [(0, '45.940'), (1, '52.920')] [2023-10-07 22:37:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000069760_71434240.pth... [2023-10-07 22:37:07,521][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000068192_69828608.pth [2023-10-07 22:37:07,527][67676] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p1/milestones/checkpoint_000069760_71434240.pth [2023-10-07 22:37:07,749][67838] Updated weights for policy 0, policy_version 69652 (0.0008) [2023-10-07 22:37:08,132][67838] Updated weights for policy 0, policy_version 69662 (0.0010) [2023-10-07 22:37:08,198][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000069664_71335936.pth... [2023-10-07 22:37:08,227][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000068096_69730304.pth [2023-10-07 22:37:08,231][67511] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p0/milestones/checkpoint_000069664_71335936.pth [2023-10-07 22:37:10,154][67871] Updated weights for policy 1, policy_version 69770 (0.0008) [2023-10-07 22:37:10,522][67871] Updated weights for policy 1, policy_version 69780 (0.0007) [2023-10-07 22:37:10,881][67871] Updated weights for policy 1, policy_version 69790 (0.0007) [2023-10-07 22:37:12,187][67838] Updated weights for policy 0, policy_version 69672 (0.0008) [2023-10-07 22:37:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142802944. Throughput: 0: 1657.1, 1: 1674.0. Samples: 35707216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-07 22:37:12,478][66916] Avg episode reward: [(0, '48.090'), (1, '50.720')] [2023-10-07 22:37:12,573][67838] Updated weights for policy 0, policy_version 69682 (0.0007) [2023-10-07 22:37:12,945][67838] Updated weights for policy 0, policy_version 69692 (0.0009) [2023-10-07 22:37:15,045][67871] Updated weights for policy 1, policy_version 69800 (0.0009) [2023-10-07 22:37:15,413][67871] Updated weights for policy 1, policy_version 69810 (0.0008) [2023-10-07 22:37:15,778][67871] Updated weights for policy 1, policy_version 69820 (0.0009) [2023-10-07 22:37:17,247][67838] Updated weights for policy 0, policy_version 69702 (0.0009) [2023-10-07 22:37:17,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142868480. Throughput: 0: 1656.3, 1: 1653.7. Samples: 35726522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-07 22:37:17,477][66916] Avg episode reward: [(0, '51.710'), (1, '51.640')] [2023-10-07 22:37:17,635][67838] Updated weights for policy 0, policy_version 69712 (0.0009) [2023-10-07 22:37:18,010][67838] Updated weights for policy 0, policy_version 69722 (0.0007) [2023-10-07 22:37:19,918][67871] Updated weights for policy 1, policy_version 69830 (0.0007) [2023-10-07 22:37:20,282][67871] Updated weights for policy 1, policy_version 69840 (0.0007) [2023-10-07 22:37:20,652][67871] Updated weights for policy 1, policy_version 69850 (0.0008) [2023-10-07 22:37:21,999][67838] Updated weights for policy 0, policy_version 69732 (0.0009) [2023-10-07 22:37:22,365][67838] Updated weights for policy 0, policy_version 69742 (0.0008) [2023-10-07 22:37:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142934016. Throughput: 0: 1647.1, 1: 1676.5. Samples: 35746592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-07 22:37:22,477][66916] Avg episode reward: [(0, '52.340'), (1, '52.470')] [2023-10-07 22:37:22,745][67838] Updated weights for policy 0, policy_version 69752 (0.0008) [2023-10-07 22:37:24,873][67871] Updated weights for policy 1, policy_version 69860 (0.0009) [2023-10-07 22:37:25,257][67871] Updated weights for policy 1, policy_version 69870 (0.0010) [2023-10-07 22:37:25,619][67871] Updated weights for policy 1, policy_version 69880 (0.0010) [2023-10-07 22:37:26,846][67838] Updated weights for policy 0, policy_version 69762 (0.0008) [2023-10-07 22:37:27,219][67838] Updated weights for policy 0, policy_version 69772 (0.0009) [2023-10-07 22:37:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142999552. Throughput: 0: 1652.0, 1: 1671.9. Samples: 35756906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-07 22:37:27,477][66916] Avg episode reward: [(0, '54.460'), (1, '53.560')] [2023-10-07 22:37:27,599][67838] Updated weights for policy 0, policy_version 69782 (0.0009) [2023-10-07 22:37:27,973][67838] Updated weights for policy 0, policy_version 69792 (0.0009) [2023-10-07 22:37:29,670][67871] Updated weights for policy 1, policy_version 69890 (0.0009) [2023-10-07 22:37:30,029][67871] Updated weights for policy 1, policy_version 69900 (0.0009) [2023-10-07 22:37:30,395][67871] Updated weights for policy 1, policy_version 69910 (0.0007) [2023-10-07 22:37:30,764][67871] Updated weights for policy 1, policy_version 69920 (0.0008) [2023-10-07 22:37:32,106][67838] Updated weights for policy 0, policy_version 69802 (0.0009) [2023-10-07 22:37:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143065088. Throughput: 0: 1652.3, 1: 1656.0. Samples: 35776206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-07 22:37:32,477][66916] Avg episode reward: [(0, '47.770'), (1, '56.480')] [2023-10-07 22:37:32,483][67838] Updated weights for policy 0, policy_version 69812 (0.0008) [2023-10-07 22:37:32,850][67838] Updated weights for policy 0, policy_version 69822 (0.0008) [2023-10-07 22:37:34,886][67871] Updated weights for policy 1, policy_version 69930 (0.0008) [2023-10-07 22:37:35,247][67871] Updated weights for policy 1, policy_version 69940 (0.0007) [2023-10-07 22:37:35,619][67871] Updated weights for policy 1, policy_version 69950 (0.0010) [2023-10-07 22:37:37,004][67838] Updated weights for policy 0, policy_version 69832 (0.0010) [2023-10-07 22:37:37,368][67838] Updated weights for policy 0, policy_version 69842 (0.0007) [2023-10-07 22:37:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143130624. Throughput: 0: 1648.1, 1: 1673.4. Samples: 35796316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-07 22:37:37,478][66916] Avg episode reward: [(0, '48.820'), (1, '57.900')] [2023-10-07 22:37:37,738][67838] Updated weights for policy 0, policy_version 69852 (0.0007) [2023-10-07 22:37:39,812][67871] Updated weights for policy 1, policy_version 69960 (0.0007) [2023-10-07 22:37:40,187][67871] Updated weights for policy 1, policy_version 69970 (0.0008) [2023-10-07 22:37:40,560][67871] Updated weights for policy 1, policy_version 69980 (0.0009) [2023-10-07 22:37:41,923][67838] Updated weights for policy 0, policy_version 69862 (0.0008) [2023-10-07 22:37:42,301][67838] Updated weights for policy 0, policy_version 69872 (0.0010) [2023-10-07 22:37:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143196160. Throughput: 0: 1654.4, 1: 1667.6. Samples: 35806620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-07 22:37:42,477][66916] Avg episode reward: [(0, '47.910'), (1, '60.260')] [2023-10-07 22:37:42,660][67838] Updated weights for policy 0, policy_version 69882 (0.0009) [2023-10-07 22:37:44,454][67871] Updated weights for policy 1, policy_version 69990 (0.0009) [2023-10-07 22:37:44,835][67871] Updated weights for policy 1, policy_version 70000 (0.0008) [2023-10-07 22:37:45,191][67871] Updated weights for policy 1, policy_version 70010 (0.0007) [2023-10-07 22:37:46,674][67838] Updated weights for policy 0, policy_version 69892 (0.0009) [2023-10-07 22:37:47,039][67838] Updated weights for policy 0, policy_version 69902 (0.0007) [2023-10-07 22:37:47,407][67838] Updated weights for policy 0, policy_version 69912 (0.0008) [2023-10-07 22:37:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143261696. Throughput: 0: 1652.7, 1: 1665.6. Samples: 35826408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-07 22:37:47,477][66916] Avg episode reward: [(0, '46.430'), (1, '60.810')] [2023-10-07 22:37:49,352][67871] Updated weights for policy 1, policy_version 70020 (0.0009) [2023-10-07 22:37:49,715][67871] Updated weights for policy 1, policy_version 70030 (0.0010) [2023-10-07 22:37:50,077][67871] Updated weights for policy 1, policy_version 70040 (0.0010) [2023-10-07 22:37:51,469][67838] Updated weights for policy 0, policy_version 69922 (0.0008) [2023-10-07 22:37:51,848][67838] Updated weights for policy 0, policy_version 69932 (0.0009) [2023-10-07 22:37:52,216][67838] Updated weights for policy 0, policy_version 69942 (0.0008) [2023-10-07 22:37:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143327232. Throughput: 0: 1643.2, 1: 1671.5. Samples: 35846184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-07 22:37:52,477][66916] Avg episode reward: [(0, '47.740'), (1, '59.610')] [2023-10-07 22:37:52,590][67838] Updated weights for policy 0, policy_version 69952 (0.0007) [2023-10-07 22:37:54,186][67871] Updated weights for policy 1, policy_version 70050 (0.0010) [2023-10-07 22:37:54,553][67871] Updated weights for policy 1, policy_version 70060 (0.0009) [2023-10-07 22:37:54,925][67871] Updated weights for policy 1, policy_version 70070 (0.0010) [2023-10-07 22:37:55,296][67871] Updated weights for policy 1, policy_version 70080 (0.0008) [2023-10-07 22:37:56,838][67838] Updated weights for policy 0, policy_version 69962 (0.0010) [2023-10-07 22:37:57,211][67838] Updated weights for policy 0, policy_version 69972 (0.0010) [2023-10-07 22:37:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143392768. Throughput: 0: 1658.9, 1: 1652.7. Samples: 35856236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:37:57,477][66916] Avg episode reward: [(0, '46.020'), (1, '58.240')] [2023-10-07 22:37:57,584][67838] Updated weights for policy 0, policy_version 69982 (0.0009) [2023-10-07 22:37:59,453][67871] Updated weights for policy 1, policy_version 70090 (0.0007) [2023-10-07 22:37:59,818][67871] Updated weights for policy 1, policy_version 70100 (0.0007) [2023-10-07 22:38:00,189][67871] Updated weights for policy 1, policy_version 70110 (0.0009) [2023-10-07 22:38:01,769][67838] Updated weights for policy 0, policy_version 69992 (0.0009) [2023-10-07 22:38:02,146][67838] Updated weights for policy 0, policy_version 70002 (0.0009) [2023-10-07 22:38:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 143458304. Throughput: 0: 1657.9, 1: 1667.8. Samples: 35876176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:38:02,477][66916] Avg episode reward: [(0, '41.740'), (1, '55.210')] [2023-10-07 22:38:02,533][67838] Updated weights for policy 0, policy_version 70012 (0.0011) [2023-10-07 22:38:04,264][67871] Updated weights for policy 1, policy_version 70120 (0.0007) [2023-10-07 22:38:04,636][67871] Updated weights for policy 1, policy_version 70130 (0.0007) [2023-10-07 22:38:04,997][67871] Updated weights for policy 1, policy_version 70140 (0.0007) [2023-10-07 22:38:06,672][67838] Updated weights for policy 0, policy_version 70022 (0.0009) [2023-10-07 22:38:07,039][67838] Updated weights for policy 0, policy_version 70032 (0.0007) [2023-10-07 22:38:07,402][67838] Updated weights for policy 0, policy_version 70042 (0.0007) [2023-10-07 22:38:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143523840. Throughput: 0: 1649.9, 1: 1671.4. Samples: 35896050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:38:07,478][66916] Avg episode reward: [(0, '42.740'), (1, '53.290')] [2023-10-07 22:38:08,912][67871] Updated weights for policy 1, policy_version 70150 (0.0007) [2023-10-07 22:38:09,283][67871] Updated weights for policy 1, policy_version 70160 (0.0008) [2023-10-07 22:38:09,645][67871] Updated weights for policy 1, policy_version 70170 (0.0010) [2023-10-07 22:38:11,543][67838] Updated weights for policy 0, policy_version 70052 (0.0008) [2023-10-07 22:38:11,922][67838] Updated weights for policy 0, policy_version 70062 (0.0010) [2023-10-07 22:38:12,295][67838] Updated weights for policy 0, policy_version 70072 (0.0009) [2023-10-07 22:38:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143589376. Throughput: 0: 1660.2, 1: 1652.6. Samples: 35905982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:38:12,478][66916] Avg episode reward: [(0, '43.460'), (1, '55.620')] [2023-10-07 22:38:14,023][67871] Updated weights for policy 1, policy_version 70180 (0.0008) [2023-10-07 22:38:14,390][67871] Updated weights for policy 1, policy_version 70190 (0.0008) [2023-10-07 22:38:14,760][67871] Updated weights for policy 1, policy_version 70200 (0.0008) [2023-10-07 22:38:16,281][67838] Updated weights for policy 0, policy_version 70082 (0.0007) [2023-10-07 22:38:16,634][67838] Updated weights for policy 0, policy_version 70092 (0.0011) [2023-10-07 22:38:17,001][67838] Updated weights for policy 0, policy_version 70102 (0.0009) [2023-10-07 22:38:17,370][67838] Updated weights for policy 0, policy_version 70112 (0.0008) [2023-10-07 22:38:17,477][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 143687680. Throughput: 0: 1664.3, 1: 1676.0. Samples: 35926518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:38:17,478][66916] Avg episode reward: [(0, '44.570'), (1, '57.690')] [2023-10-07 22:38:18,785][67871] Updated weights for policy 1, policy_version 70210 (0.0008) [2023-10-07 22:38:19,182][67871] Updated weights for policy 1, policy_version 70220 (0.0007) [2023-10-07 22:38:19,548][67871] Updated weights for policy 1, policy_version 70230 (0.0009) [2023-10-07 22:38:19,920][67871] Updated weights for policy 1, policy_version 70240 (0.0011) [2023-10-07 22:38:21,490][67838] Updated weights for policy 0, policy_version 70122 (0.0009) [2023-10-07 22:38:21,869][67838] Updated weights for policy 0, policy_version 70132 (0.0009) [2023-10-07 22:38:22,243][67838] Updated weights for policy 0, policy_version 70142 (0.0008) [2023-10-07 22:38:22,477][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 143753216. Throughput: 0: 1650.5, 1: 1672.0. Samples: 35945832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:38:22,478][66916] Avg episode reward: [(0, '47.160'), (1, '58.670')] [2023-10-07 22:38:23,931][67871] Updated weights for policy 1, policy_version 70250 (0.0007) [2023-10-07 22:38:24,302][67871] Updated weights for policy 1, policy_version 70260 (0.0008) [2023-10-07 22:38:24,668][67871] Updated weights for policy 1, policy_version 70270 (0.0007) [2023-10-07 22:38:26,385][67838] Updated weights for policy 0, policy_version 70152 (0.0007) [2023-10-07 22:38:26,761][67838] Updated weights for policy 0, policy_version 70162 (0.0007) [2023-10-07 22:38:27,131][67838] Updated weights for policy 0, policy_version 70172 (0.0007) [2023-10-07 22:38:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 143818752. Throughput: 0: 1666.8, 1: 1651.9. Samples: 35955958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:38:27,477][66916] Avg episode reward: [(0, '49.580'), (1, '60.490')] [2023-10-07 22:38:28,787][67871] Updated weights for policy 1, policy_version 70280 (0.0007) [2023-10-07 22:38:29,152][67871] Updated weights for policy 1, policy_version 70290 (0.0011) [2023-10-07 22:38:29,514][67871] Updated weights for policy 1, policy_version 70300 (0.0007) [2023-10-07 22:38:31,239][67838] Updated weights for policy 0, policy_version 70182 (0.0008) [2023-10-07 22:38:31,608][67838] Updated weights for policy 0, policy_version 70192 (0.0007) [2023-10-07 22:38:31,978][67838] Updated weights for policy 0, policy_version 70202 (0.0009) [2023-10-07 22:38:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 143884288. Throughput: 0: 1664.3, 1: 1672.1. Samples: 35976546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:38:32,478][66916] Avg episode reward: [(0, '47.990'), (1, '59.520')] [2023-10-07 22:38:33,499][67871] Updated weights for policy 1, policy_version 70310 (0.0007) [2023-10-07 22:38:33,860][67871] Updated weights for policy 1, policy_version 70320 (0.0008) [2023-10-07 22:38:34,231][67871] Updated weights for policy 1, policy_version 70330 (0.0007) [2023-10-07 22:38:36,075][67838] Updated weights for policy 0, policy_version 70212 (0.0008) [2023-10-07 22:38:36,444][67838] Updated weights for policy 0, policy_version 70222 (0.0007) [2023-10-07 22:38:36,817][67838] Updated weights for policy 0, policy_version 70232 (0.0008) [2023-10-07 22:38:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 143949824. Throughput: 0: 1654.4, 1: 1674.4. Samples: 35995982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:38:37,478][66916] Avg episode reward: [(0, '50.140'), (1, '57.980')] [2023-10-07 22:38:38,488][67871] Updated weights for policy 1, policy_version 70340 (0.0008) [2023-10-07 22:38:38,859][67871] Updated weights for policy 1, policy_version 70350 (0.0007) [2023-10-07 22:38:39,222][67871] Updated weights for policy 1, policy_version 70360 (0.0007) [2023-10-07 22:38:40,895][67838] Updated weights for policy 0, policy_version 70242 (0.0007) [2023-10-07 22:38:41,263][67838] Updated weights for policy 0, policy_version 70252 (0.0008) [2023-10-07 22:38:41,644][67838] Updated weights for policy 0, policy_version 70262 (0.0010) [2023-10-07 22:38:42,014][67838] Updated weights for policy 0, policy_version 70272 (0.0008) [2023-10-07 22:38:42,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 144015360. Throughput: 0: 1670.7, 1: 1667.5. Samples: 36006454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:38:42,477][66916] Avg episode reward: [(0, '48.160'), (1, '57.000')] [2023-10-07 22:38:43,287][67871] Updated weights for policy 1, policy_version 70370 (0.0008) [2023-10-07 22:38:43,649][67871] Updated weights for policy 1, policy_version 70380 (0.0007) [2023-10-07 22:38:44,016][67871] Updated weights for policy 1, policy_version 70390 (0.0008) [2023-10-07 22:38:44,393][67871] Updated weights for policy 1, policy_version 70400 (0.0010) [2023-10-07 22:38:46,096][67838] Updated weights for policy 0, policy_version 70282 (0.0007) [2023-10-07 22:38:46,461][67838] Updated weights for policy 0, policy_version 70292 (0.0007) [2023-10-07 22:38:46,832][67838] Updated weights for policy 0, policy_version 70302 (0.0008) [2023-10-07 22:38:47,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 144080896. Throughput: 0: 1663.9, 1: 1678.7. Samples: 36026594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:38:47,478][66916] Avg episode reward: [(0, '50.670'), (1, '56.070')] [2023-10-07 22:38:48,396][67871] Updated weights for policy 1, policy_version 70410 (0.0009) [2023-10-07 22:38:48,761][67871] Updated weights for policy 1, policy_version 70420 (0.0010) [2023-10-07 22:38:49,119][67871] Updated weights for policy 1, policy_version 70430 (0.0010) [2023-10-07 22:38:50,927][67838] Updated weights for policy 0, policy_version 70312 (0.0010) [2023-10-07 22:38:51,299][67838] Updated weights for policy 0, policy_version 70322 (0.0008) [2023-10-07 22:38:51,668][67838] Updated weights for policy 0, policy_version 70332 (0.0011) [2023-10-07 22:38:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 144146432. Throughput: 0: 1659.3, 1: 1683.2. Samples: 36046460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:38:52,477][66916] Avg episode reward: [(0, '47.680'), (1, '55.710')] [2023-10-07 22:38:53,296][67871] Updated weights for policy 1, policy_version 70440 (0.0010) [2023-10-07 22:38:53,675][67871] Updated weights for policy 1, policy_version 70450 (0.0010) [2023-10-07 22:38:54,044][67871] Updated weights for policy 1, policy_version 70460 (0.0010) [2023-10-07 22:38:55,780][67838] Updated weights for policy 0, policy_version 70342 (0.0009) [2023-10-07 22:38:56,160][67838] Updated weights for policy 0, policy_version 70352 (0.0009) [2023-10-07 22:38:56,521][67838] Updated weights for policy 0, policy_version 70362 (0.0010) [2023-10-07 22:38:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 144211968. Throughput: 0: 1671.2, 1: 1677.4. Samples: 36056670. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:38:57,478][66916] Avg episode reward: [(0, '47.940'), (1, '54.000')] [2023-10-07 22:38:58,105][67871] Updated weights for policy 1, policy_version 70470 (0.0009) [2023-10-07 22:38:58,473][67871] Updated weights for policy 1, policy_version 70480 (0.0007) [2023-10-07 22:38:58,832][67871] Updated weights for policy 1, policy_version 70490 (0.0010) [2023-10-07 22:39:00,634][67838] Updated weights for policy 0, policy_version 70372 (0.0009) [2023-10-07 22:39:01,013][67838] Updated weights for policy 0, policy_version 70382 (0.0008) [2023-10-07 22:39:01,379][67838] Updated weights for policy 0, policy_version 70392 (0.0007) [2023-10-07 22:39:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 144277504. Throughput: 0: 1653.8, 1: 1678.6. Samples: 36076478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:39:02,478][66916] Avg episode reward: [(0, '46.500'), (1, '53.190')] [2023-10-07 22:39:02,881][67871] Updated weights for policy 1, policy_version 70500 (0.0009) [2023-10-07 22:39:03,251][67871] Updated weights for policy 1, policy_version 70510 (0.0008) [2023-10-07 22:39:03,622][67871] Updated weights for policy 1, policy_version 70520 (0.0007) [2023-10-07 22:39:05,507][67838] Updated weights for policy 0, policy_version 70402 (0.0008) [2023-10-07 22:39:05,887][67838] Updated weights for policy 0, policy_version 70412 (0.0010) [2023-10-07 22:39:06,254][67838] Updated weights for policy 0, policy_version 70422 (0.0008) [2023-10-07 22:39:06,629][67838] Updated weights for policy 0, policy_version 70432 (0.0008) [2023-10-07 22:39:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 144343040. Throughput: 0: 1662.4, 1: 1683.6. Samples: 36096400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:39:07,477][66916] Avg episode reward: [(0, '44.530'), (1, '52.210')] [2023-10-07 22:39:07,487][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000070528_72220672.pth... [2023-10-07 22:39:07,487][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000070432_72122368.pth... [2023-10-07 22:39:07,517][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000068992_70647808.pth [2023-10-07 22:39:07,526][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000068864_70516736.pth [2023-10-07 22:39:07,820][67871] Updated weights for policy 1, policy_version 70530 (0.0008) [2023-10-07 22:39:08,242][67871] Updated weights for policy 1, policy_version 70540 (0.0010) [2023-10-07 22:39:08,615][67871] Updated weights for policy 1, policy_version 70550 (0.0009) [2023-10-07 22:39:08,975][67871] Updated weights for policy 1, policy_version 70560 (0.0007) [2023-10-07 22:39:10,731][67838] Updated weights for policy 0, policy_version 70442 (0.0011) [2023-10-07 22:39:11,101][67838] Updated weights for policy 0, policy_version 70452 (0.0008) [2023-10-07 22:39:11,476][67838] Updated weights for policy 0, policy_version 70462 (0.0007) [2023-10-07 22:39:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 144408576. Throughput: 0: 1668.8, 1: 1681.8. Samples: 36106736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:39:12,478][66916] Avg episode reward: [(0, '47.520'), (1, '52.670')] [2023-10-07 22:39:13,008][67871] Updated weights for policy 1, policy_version 70570 (0.0007) [2023-10-07 22:39:13,363][67871] Updated weights for policy 1, policy_version 70580 (0.0007) [2023-10-07 22:39:13,725][67871] Updated weights for policy 1, policy_version 70590 (0.0008) [2023-10-07 22:39:15,767][67838] Updated weights for policy 0, policy_version 70472 (0.0010) [2023-10-07 22:39:16,136][67838] Updated weights for policy 0, policy_version 70482 (0.0008) [2023-10-07 22:39:16,512][67838] Updated weights for policy 0, policy_version 70492 (0.0009) [2023-10-07 22:39:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 144474112. Throughput: 0: 1653.6, 1: 1676.3. Samples: 36126392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:39:17,478][66916] Avg episode reward: [(0, '47.510'), (1, '53.590')] [2023-10-07 22:39:17,683][67871] Updated weights for policy 1, policy_version 70600 (0.0010) [2023-10-07 22:39:18,051][67871] Updated weights for policy 1, policy_version 70610 (0.0009) [2023-10-07 22:39:18,414][67871] Updated weights for policy 1, policy_version 70620 (0.0009) [2023-10-07 22:39:20,656][67838] Updated weights for policy 0, policy_version 70502 (0.0010) [2023-10-07 22:39:21,025][67838] Updated weights for policy 0, policy_version 70512 (0.0010) [2023-10-07 22:39:21,410][67838] Updated weights for policy 0, policy_version 70522 (0.0010) [2023-10-07 22:39:22,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144539648. Throughput: 0: 1660.3, 1: 1678.0. Samples: 36146208. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:39:22,477][66916] Avg episode reward: [(0, '50.780'), (1, '54.940')] [2023-10-07 22:39:22,573][67871] Updated weights for policy 1, policy_version 70630 (0.0008) [2023-10-07 22:39:22,941][67871] Updated weights for policy 1, policy_version 70640 (0.0009) [2023-10-07 22:39:23,308][67871] Updated weights for policy 1, policy_version 70650 (0.0009) [2023-10-07 22:39:25,489][67838] Updated weights for policy 0, policy_version 70532 (0.0009) [2023-10-07 22:39:25,863][67838] Updated weights for policy 0, policy_version 70542 (0.0007) [2023-10-07 22:39:26,224][67838] Updated weights for policy 0, policy_version 70552 (0.0007) [2023-10-07 22:39:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144605184. Throughput: 0: 1659.9, 1: 1671.9. Samples: 36156388. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:39:27,478][66916] Avg episode reward: [(0, '51.710'), (1, '55.350')] [2023-10-07 22:39:27,484][67871] Updated weights for policy 1, policy_version 70660 (0.0009) [2023-10-07 22:39:27,844][67871] Updated weights for policy 1, policy_version 70670 (0.0008) [2023-10-07 22:39:28,217][67871] Updated weights for policy 1, policy_version 70680 (0.0009) [2023-10-07 22:39:30,339][67838] Updated weights for policy 0, policy_version 70562 (0.0008) [2023-10-07 22:39:30,713][67838] Updated weights for policy 0, policy_version 70572 (0.0009) [2023-10-07 22:39:31,077][67838] Updated weights for policy 0, policy_version 70582 (0.0010) [2023-10-07 22:39:31,449][67838] Updated weights for policy 0, policy_version 70592 (0.0010) [2023-10-07 22:39:32,360][67871] Updated weights for policy 1, policy_version 70690 (0.0008) [2023-10-07 22:39:32,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144670720. Throughput: 0: 1651.5, 1: 1672.1. Samples: 36176154. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:39:32,478][66916] Avg episode reward: [(0, '52.310'), (1, '52.550')] [2023-10-07 22:39:32,722][67871] Updated weights for policy 1, policy_version 70700 (0.0007) [2023-10-07 22:39:33,091][67871] Updated weights for policy 1, policy_version 70710 (0.0007) [2023-10-07 22:39:33,460][67871] Updated weights for policy 1, policy_version 70720 (0.0007) [2023-10-07 22:39:35,482][67838] Updated weights for policy 0, policy_version 70602 (0.0008) [2023-10-07 22:39:35,852][67838] Updated weights for policy 0, policy_version 70612 (0.0010) [2023-10-07 22:39:36,220][67838] Updated weights for policy 0, policy_version 70622 (0.0007) [2023-10-07 22:39:37,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 144736256. Throughput: 0: 1661.2, 1: 1666.4. Samples: 36196204. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:39:37,477][66916] Avg episode reward: [(0, '50.330'), (1, '50.380')] [2023-10-07 22:39:37,553][67871] Updated weights for policy 1, policy_version 70730 (0.0009) [2023-10-07 22:39:37,930][67871] Updated weights for policy 1, policy_version 70740 (0.0010) [2023-10-07 22:39:38,295][67871] Updated weights for policy 1, policy_version 70750 (0.0009) [2023-10-07 22:39:40,245][67838] Updated weights for policy 0, policy_version 70632 (0.0008) [2023-10-07 22:39:40,622][67838] Updated weights for policy 0, policy_version 70642 (0.0008) [2023-10-07 22:39:40,995][67838] Updated weights for policy 0, policy_version 70652 (0.0009) [2023-10-07 22:39:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144801792. Throughput: 0: 1662.5, 1: 1665.2. Samples: 36206418. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:39:42,477][66916] Avg episode reward: [(0, '47.420'), (1, '50.740')] [2023-10-07 22:39:42,562][67871] Updated weights for policy 1, policy_version 70760 (0.0008) [2023-10-07 22:39:42,925][67871] Updated weights for policy 1, policy_version 70770 (0.0007) [2023-10-07 22:39:43,294][67871] Updated weights for policy 1, policy_version 70780 (0.0009) [2023-10-07 22:39:45,029][67838] Updated weights for policy 0, policy_version 70662 (0.0009) [2023-10-07 22:39:45,394][67838] Updated weights for policy 0, policy_version 70672 (0.0011) [2023-10-07 22:39:45,764][67838] Updated weights for policy 0, policy_version 70682 (0.0008) [2023-10-07 22:39:47,350][67871] Updated weights for policy 1, policy_version 70790 (0.0009) [2023-10-07 22:39:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144867328. Throughput: 0: 1651.0, 1: 1665.8. Samples: 36225734. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:39:47,477][66916] Avg episode reward: [(0, '48.520'), (1, '50.100')] [2023-10-07 22:39:47,717][67871] Updated weights for policy 1, policy_version 70800 (0.0009) [2023-10-07 22:39:48,088][67871] Updated weights for policy 1, policy_version 70810 (0.0008) [2023-10-07 22:39:49,679][67838] Updated weights for policy 0, policy_version 70692 (0.0010) [2023-10-07 22:39:50,056][67838] Updated weights for policy 0, policy_version 70702 (0.0012) [2023-10-07 22:39:50,421][67838] Updated weights for policy 0, policy_version 70712 (0.0011) [2023-10-07 22:39:52,277][67871] Updated weights for policy 1, policy_version 70820 (0.0010) [2023-10-07 22:39:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144932864. Throughput: 0: 1671.8, 1: 1661.2. Samples: 36246388. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:39:52,477][66916] Avg episode reward: [(0, '45.790'), (1, '49.600')] [2023-10-07 22:39:52,644][67871] Updated weights for policy 1, policy_version 70830 (0.0009) [2023-10-07 22:39:53,012][67871] Updated weights for policy 1, policy_version 70840 (0.0008) [2023-10-07 22:39:54,642][67838] Updated weights for policy 0, policy_version 70722 (0.0008) [2023-10-07 22:39:55,017][67838] Updated weights for policy 0, policy_version 70732 (0.0008) [2023-10-07 22:39:55,387][67838] Updated weights for policy 0, policy_version 70742 (0.0007) [2023-10-07 22:39:55,762][67838] Updated weights for policy 0, policy_version 70752 (0.0008) [2023-10-07 22:39:56,958][67871] Updated weights for policy 1, policy_version 70850 (0.0008) [2023-10-07 22:39:57,363][67871] Updated weights for policy 1, policy_version 70860 (0.0009) [2023-10-07 22:39:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144998400. Throughput: 0: 1653.7, 1: 1660.9. Samples: 36255894. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:39:57,477][66916] Avg episode reward: [(0, '48.920'), (1, '50.410')] [2023-10-07 22:39:57,724][67871] Updated weights for policy 1, policy_version 70870 (0.0008) [2023-10-07 22:39:58,091][67871] Updated weights for policy 1, policy_version 70880 (0.0009) [2023-10-07 22:39:59,871][67838] Updated weights for policy 0, policy_version 70762 (0.0009) [2023-10-07 22:40:00,241][67838] Updated weights for policy 0, policy_version 70772 (0.0010) [2023-10-07 22:40:00,619][67838] Updated weights for policy 0, policy_version 70782 (0.0007) [2023-10-07 22:40:02,258][67871] Updated weights for policy 1, policy_version 70890 (0.0008) [2023-10-07 22:40:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145063936. Throughput: 0: 1653.2, 1: 1662.9. Samples: 36275616. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:40:02,477][66916] Avg episode reward: [(0, '48.330'), (1, '52.170')] [2023-10-07 22:40:02,628][67871] Updated weights for policy 1, policy_version 70900 (0.0007) [2023-10-07 22:40:03,004][67871] Updated weights for policy 1, policy_version 70910 (0.0008) [2023-10-07 22:40:04,885][67838] Updated weights for policy 0, policy_version 70792 (0.0009) [2023-10-07 22:40:05,261][67838] Updated weights for policy 0, policy_version 70802 (0.0009) [2023-10-07 22:40:05,627][67838] Updated weights for policy 0, policy_version 70812 (0.0010) [2023-10-07 22:40:07,181][67871] Updated weights for policy 1, policy_version 70920 (0.0009) [2023-10-07 22:40:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145129472. Throughput: 0: 1668.0, 1: 1659.4. Samples: 36295940. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:40:07,478][66916] Avg episode reward: [(0, '52.020'), (1, '57.160')] [2023-10-07 22:40:07,547][67871] Updated weights for policy 1, policy_version 70930 (0.0009) [2023-10-07 22:40:07,925][67871] Updated weights for policy 1, policy_version 70940 (0.0008) [2023-10-07 22:40:09,800][67838] Updated weights for policy 0, policy_version 70822 (0.0009) [2023-10-07 22:40:10,179][67838] Updated weights for policy 0, policy_version 70832 (0.0011) [2023-10-07 22:40:10,546][67838] Updated weights for policy 0, policy_version 70842 (0.0010) [2023-10-07 22:40:12,076][67871] Updated weights for policy 1, policy_version 70950 (0.0007) [2023-10-07 22:40:12,440][67871] Updated weights for policy 1, policy_version 70960 (0.0007) [2023-10-07 22:40:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145195008. Throughput: 0: 1653.9, 1: 1658.4. Samples: 36305440. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:40:12,477][66916] Avg episode reward: [(0, '51.120'), (1, '58.040')] [2023-10-07 22:40:12,812][67871] Updated weights for policy 1, policy_version 70970 (0.0007) [2023-10-07 22:40:14,702][67838] Updated weights for policy 0, policy_version 70852 (0.0008) [2023-10-07 22:40:15,072][67838] Updated weights for policy 0, policy_version 70862 (0.0008) [2023-10-07 22:40:15,452][67838] Updated weights for policy 0, policy_version 70872 (0.0009) [2023-10-07 22:40:17,007][67871] Updated weights for policy 1, policy_version 70980 (0.0009) [2023-10-07 22:40:17,371][67871] Updated weights for policy 1, policy_version 70990 (0.0010) [2023-10-07 22:40:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145260544. Throughput: 0: 1653.3, 1: 1653.3. Samples: 36324954. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:40:17,477][66916] Avg episode reward: [(0, '52.020'), (1, '62.490')] [2023-10-07 22:40:17,746][67871] Updated weights for policy 1, policy_version 71000 (0.0010) [2023-10-07 22:40:18,030][67676] Saving new best policy, reward=62.490! [2023-10-07 22:40:19,348][67838] Updated weights for policy 0, policy_version 70882 (0.0008) [2023-10-07 22:40:19,721][67838] Updated weights for policy 0, policy_version 70892 (0.0007) [2023-10-07 22:40:20,089][67838] Updated weights for policy 0, policy_version 70902 (0.0007) [2023-10-07 22:40:20,463][67838] Updated weights for policy 0, policy_version 70912 (0.0010) [2023-10-07 22:40:22,026][67871] Updated weights for policy 1, policy_version 71010 (0.0008) [2023-10-07 22:40:22,397][67871] Updated weights for policy 1, policy_version 71020 (0.0008) [2023-10-07 22:40:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145326080. Throughput: 0: 1666.9, 1: 1648.5. Samples: 36345400. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:40:22,477][66916] Avg episode reward: [(0, '49.670'), (1, '62.200')] [2023-10-07 22:40:22,749][67871] Updated weights for policy 1, policy_version 71030 (0.0009) [2023-10-07 22:40:23,114][67871] Updated weights for policy 1, policy_version 71040 (0.0008) [2023-10-07 22:40:24,604][67838] Updated weights for policy 0, policy_version 70922 (0.0010) [2023-10-07 22:40:24,970][67838] Updated weights for policy 0, policy_version 70932 (0.0011) [2023-10-07 22:40:25,347][67838] Updated weights for policy 0, policy_version 70942 (0.0011) [2023-10-07 22:40:27,227][67871] Updated weights for policy 1, policy_version 71050 (0.0007) [2023-10-07 22:40:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145391616. Throughput: 0: 1649.3, 1: 1652.6. Samples: 36355006. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:40:27,477][66916] Avg episode reward: [(0, '48.870'), (1, '61.900')] [2023-10-07 22:40:27,604][67871] Updated weights for policy 1, policy_version 71060 (0.0009) [2023-10-07 22:40:27,976][67871] Updated weights for policy 1, policy_version 71070 (0.0008) [2023-10-07 22:40:29,478][67838] Updated weights for policy 0, policy_version 70952 (0.0009) [2023-10-07 22:40:29,860][67838] Updated weights for policy 0, policy_version 70962 (0.0007) [2023-10-07 22:40:30,227][67838] Updated weights for policy 0, policy_version 70972 (0.0007) [2023-10-07 22:40:32,041][67871] Updated weights for policy 1, policy_version 71080 (0.0010) [2023-10-07 22:40:32,399][67871] Updated weights for policy 1, policy_version 71090 (0.0009) [2023-10-07 22:40:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 145457152. Throughput: 0: 1667.3, 1: 1653.3. Samples: 36375160. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:40:32,478][66916] Avg episode reward: [(0, '45.550'), (1, '62.170')] [2023-10-07 22:40:32,767][67871] Updated weights for policy 1, policy_version 71100 (0.0009) [2023-10-07 22:40:34,052][67838] Updated weights for policy 0, policy_version 70982 (0.0010) [2023-10-07 22:40:34,425][67838] Updated weights for policy 0, policy_version 70992 (0.0008) [2023-10-07 22:40:34,802][67838] Updated weights for policy 0, policy_version 71002 (0.0009) [2023-10-07 22:40:36,877][67871] Updated weights for policy 1, policy_version 71110 (0.0007) [2023-10-07 22:40:37,248][67871] Updated weights for policy 1, policy_version 71120 (0.0008) [2023-10-07 22:40:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 145522688. Throughput: 0: 1663.4, 1: 1650.5. Samples: 36395516. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:40:37,478][66916] Avg episode reward: [(0, '51.380'), (1, '59.900')] [2023-10-07 22:40:37,610][67871] Updated weights for policy 1, policy_version 71130 (0.0007) [2023-10-07 22:40:39,010][67838] Updated weights for policy 0, policy_version 71012 (0.0008) [2023-10-07 22:40:39,383][67838] Updated weights for policy 0, policy_version 71022 (0.0008) [2023-10-07 22:40:39,746][67838] Updated weights for policy 0, policy_version 71032 (0.0009) [2023-10-07 22:40:41,729][67871] Updated weights for policy 1, policy_version 71140 (0.0008) [2023-10-07 22:40:42,103][67871] Updated weights for policy 1, policy_version 71150 (0.0008) [2023-10-07 22:40:42,457][67871] Updated weights for policy 1, policy_version 71160 (0.0007) [2023-10-07 22:40:42,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145588224. Throughput: 0: 1651.7, 1: 1658.2. Samples: 36404840. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:40:42,477][66916] Avg episode reward: [(0, '47.690'), (1, '60.350')] [2023-10-07 22:40:44,054][67838] Updated weights for policy 0, policy_version 71042 (0.0012) [2023-10-07 22:40:44,427][67838] Updated weights for policy 0, policy_version 71052 (0.0011) [2023-10-07 22:40:44,813][67838] Updated weights for policy 0, policy_version 71062 (0.0011) [2023-10-07 22:40:45,179][67838] Updated weights for policy 0, policy_version 71072 (0.0011) [2023-10-07 22:40:46,600][67871] Updated weights for policy 1, policy_version 71170 (0.0007) [2023-10-07 22:40:47,028][67871] Updated weights for policy 1, policy_version 71180 (0.0008) [2023-10-07 22:40:47,387][67871] Updated weights for policy 1, policy_version 71190 (0.0009) [2023-10-07 22:40:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145653760. Throughput: 0: 1666.0, 1: 1655.8. Samples: 36425100. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:40:47,477][66916] Avg episode reward: [(0, '52.200'), (1, '61.780')] [2023-10-07 22:40:47,754][67871] Updated weights for policy 1, policy_version 71200 (0.0009) [2023-10-07 22:40:49,173][67838] Updated weights for policy 0, policy_version 71082 (0.0008) [2023-10-07 22:40:49,538][67838] Updated weights for policy 0, policy_version 71092 (0.0010) [2023-10-07 22:40:49,908][67838] Updated weights for policy 0, policy_version 71102 (0.0010) [2023-10-07 22:40:51,865][67871] Updated weights for policy 1, policy_version 71210 (0.0007) [2023-10-07 22:40:52,234][67871] Updated weights for policy 1, policy_version 71220 (0.0008) [2023-10-07 22:40:52,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 145719296. Throughput: 0: 1668.2, 1: 1645.8. Samples: 36445070. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:40:52,478][66916] Avg episode reward: [(0, '50.040'), (1, '58.920')] [2023-10-07 22:40:52,606][67871] Updated weights for policy 1, policy_version 71230 (0.0010) [2023-10-07 22:40:54,089][67838] Updated weights for policy 0, policy_version 71112 (0.0008) [2023-10-07 22:40:54,458][67838] Updated weights for policy 0, policy_version 71122 (0.0007) [2023-10-07 22:40:54,832][67838] Updated weights for policy 0, policy_version 71132 (0.0007) [2023-10-07 22:40:56,785][67871] Updated weights for policy 1, policy_version 71240 (0.0007) [2023-10-07 22:40:57,148][67871] Updated weights for policy 1, policy_version 71250 (0.0008) [2023-10-07 22:40:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 145784832. Throughput: 0: 1653.7, 1: 1661.0. Samples: 36454602. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 22:40:57,478][66916] Avg episode reward: [(0, '51.730'), (1, '59.490')] [2023-10-07 22:40:57,520][67871] Updated weights for policy 1, policy_version 71260 (0.0007) [2023-10-07 22:40:59,013][67838] Updated weights for policy 0, policy_version 71142 (0.0008) [2023-10-07 22:40:59,382][67838] Updated weights for policy 0, policy_version 71152 (0.0011) [2023-10-07 22:40:59,757][67838] Updated weights for policy 0, policy_version 71162 (0.0011) [2023-10-07 22:41:01,659][67871] Updated weights for policy 1, policy_version 71270 (0.0008) [2023-10-07 22:41:02,021][67871] Updated weights for policy 1, policy_version 71280 (0.0010) [2023-10-07 22:41:02,382][67871] Updated weights for policy 1, policy_version 71290 (0.0010) [2023-10-07 22:41:02,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 145850368. Throughput: 0: 1671.5, 1: 1662.9. Samples: 36475002. Policy #0 lag: (min: 25.0, avg: 39.9, max: 57.0) [2023-10-07 22:41:02,477][66916] Avg episode reward: [(0, '47.290'), (1, '58.900')] [2023-10-07 22:41:03,613][67838] Updated weights for policy 0, policy_version 71172 (0.0010) [2023-10-07 22:41:03,978][67838] Updated weights for policy 0, policy_version 71182 (0.0009) [2023-10-07 22:41:04,353][67838] Updated weights for policy 0, policy_version 71192 (0.0010) [2023-10-07 22:41:06,505][67871] Updated weights for policy 1, policy_version 71300 (0.0010) [2023-10-07 22:41:06,868][67871] Updated weights for policy 1, policy_version 71310 (0.0010) [2023-10-07 22:41:07,235][67871] Updated weights for policy 1, policy_version 71320 (0.0009) [2023-10-07 22:41:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 145915904. Throughput: 0: 1674.8, 1: 1651.8. Samples: 36495096. Policy #0 lag: (min: 25.0, avg: 39.9, max: 57.0) [2023-10-07 22:41:07,477][66916] Avg episode reward: [(0, '48.340'), (1, '57.670')] [2023-10-07 22:41:07,487][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000071200_72908800.pth... [2023-10-07 22:41:07,522][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000069664_71335936.pth [2023-10-07 22:41:07,529][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000071328_73039872.pth... [2023-10-07 22:41:07,558][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000069760_71434240.pth [2023-10-07 22:41:08,425][67838] Updated weights for policy 0, policy_version 71202 (0.0008) [2023-10-07 22:41:08,799][67838] Updated weights for policy 0, policy_version 71212 (0.0007) [2023-10-07 22:41:09,177][67838] Updated weights for policy 0, policy_version 71222 (0.0008) [2023-10-07 22:41:09,552][67838] Updated weights for policy 0, policy_version 71232 (0.0007) [2023-10-07 22:41:11,304][67871] Updated weights for policy 1, policy_version 71330 (0.0011) [2023-10-07 22:41:11,677][67871] Updated weights for policy 1, policy_version 71340 (0.0011) [2023-10-07 22:41:12,051][67871] Updated weights for policy 1, policy_version 71350 (0.0009) [2023-10-07 22:41:12,425][67871] Updated weights for policy 1, policy_version 71360 (0.0007) [2023-10-07 22:41:12,476][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 146014208. Throughput: 0: 1662.9, 1: 1658.4. Samples: 36504462. Policy #0 lag: (min: 25.0, avg: 39.9, max: 57.0) [2023-10-07 22:41:12,477][66916] Avg episode reward: [(0, '48.460'), (1, '53.150')] [2023-10-07 22:41:13,661][67838] Updated weights for policy 0, policy_version 71242 (0.0010) [2023-10-07 22:41:14,030][67838] Updated weights for policy 0, policy_version 71252 (0.0009) [2023-10-07 22:41:14,398][67838] Updated weights for policy 0, policy_version 71262 (0.0011) [2023-10-07 22:41:16,666][67871] Updated weights for policy 1, policy_version 71370 (0.0008) [2023-10-07 22:41:17,037][67871] Updated weights for policy 1, policy_version 71380 (0.0008) [2023-10-07 22:41:17,396][67871] Updated weights for policy 1, policy_version 71390 (0.0008) [2023-10-07 22:41:17,476][66916] Fps is (10 sec: 16384.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 146079744. Throughput: 0: 1670.9, 1: 1658.9. Samples: 36525000. Policy #0 lag: (min: 25.0, avg: 39.9, max: 57.0) [2023-10-07 22:41:17,477][66916] Avg episode reward: [(0, '46.090'), (1, '51.720')] [2023-10-07 22:41:18,657][67838] Updated weights for policy 0, policy_version 71272 (0.0008) [2023-10-07 22:41:19,035][67838] Updated weights for policy 0, policy_version 71282 (0.0010) [2023-10-07 22:41:19,411][67838] Updated weights for policy 0, policy_version 71292 (0.0010) [2023-10-07 22:41:21,645][67871] Updated weights for policy 1, policy_version 71400 (0.0007) [2023-10-07 22:41:22,011][67871] Updated weights for policy 1, policy_version 71410 (0.0007) [2023-10-07 22:41:22,384][67871] Updated weights for policy 1, policy_version 71420 (0.0008) [2023-10-07 22:41:22,476][66916] Fps is (10 sec: 9830.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 146112512. Throughput: 0: 1664.4, 1: 1649.4. Samples: 36544636. Policy #0 lag: (min: 25.0, avg: 39.9, max: 57.0) [2023-10-07 22:41:22,477][66916] Avg episode reward: [(0, '49.200'), (1, '53.240')] [2023-10-07 22:41:23,611][67838] Updated weights for policy 0, policy_version 71302 (0.0011) [2023-10-07 22:41:23,978][67838] Updated weights for policy 0, policy_version 71312 (0.0009) [2023-10-07 22:41:24,353][67838] Updated weights for policy 0, policy_version 71322 (0.0008) [2023-10-07 22:41:26,553][67871] Updated weights for policy 1, policy_version 71430 (0.0008) [2023-10-07 22:41:26,921][67871] Updated weights for policy 1, policy_version 71440 (0.0008) [2023-10-07 22:41:27,290][67871] Updated weights for policy 1, policy_version 71450 (0.0008) [2023-10-07 22:41:27,477][66916] Fps is (10 sec: 9830.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 146178048. Throughput: 0: 1659.9, 1: 1656.9. Samples: 36554094. Policy #0 lag: (min: 25.0, avg: 39.9, max: 57.0) [2023-10-07 22:41:27,478][66916] Avg episode reward: [(0, '50.640'), (1, '50.100')] [2023-10-07 22:41:28,499][67838] Updated weights for policy 0, policy_version 71332 (0.0009) [2023-10-07 22:41:28,881][67838] Updated weights for policy 0, policy_version 71342 (0.0009) [2023-10-07 22:41:29,237][67838] Updated weights for policy 0, policy_version 71352 (0.0009) [2023-10-07 22:41:31,473][67871] Updated weights for policy 1, policy_version 71460 (0.0009) [2023-10-07 22:41:31,864][67871] Updated weights for policy 1, policy_version 71470 (0.0008) [2023-10-07 22:41:32,227][67871] Updated weights for policy 1, policy_version 71480 (0.0007) [2023-10-07 22:41:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 146243584. Throughput: 0: 1658.2, 1: 1658.4. Samples: 36574348. Policy #0 lag: (min: 25.0, avg: 39.9, max: 57.0) [2023-10-07 22:41:32,477][66916] Avg episode reward: [(0, '52.200'), (1, '53.820')] [2023-10-07 22:41:33,453][67838] Updated weights for policy 0, policy_version 71362 (0.0009) [2023-10-07 22:41:33,835][67838] Updated weights for policy 0, policy_version 71372 (0.0008) [2023-10-07 22:41:34,203][67838] Updated weights for policy 0, policy_version 71382 (0.0008) [2023-10-07 22:41:34,576][67838] Updated weights for policy 0, policy_version 71392 (0.0009) [2023-10-07 22:41:36,340][67871] Updated weights for policy 1, policy_version 71490 (0.0008) [2023-10-07 22:41:36,703][67871] Updated weights for policy 1, policy_version 71500 (0.0011) [2023-10-07 22:41:37,064][67871] Updated weights for policy 1, policy_version 71510 (0.0009) [2023-10-07 22:41:37,435][67871] Updated weights for policy 1, policy_version 71520 (0.0009) [2023-10-07 22:41:37,477][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 146341888. Throughput: 0: 1655.7, 1: 1652.0. Samples: 36593920. Policy #0 lag: (min: 25.0, avg: 39.9, max: 57.0) [2023-10-07 22:41:37,478][66916] Avg episode reward: [(0, '51.600'), (1, '54.600')] [2023-10-07 22:41:38,817][67838] Updated weights for policy 0, policy_version 71402 (0.0011) [2023-10-07 22:41:39,181][67838] Updated weights for policy 0, policy_version 71412 (0.0010) [2023-10-07 22:41:39,556][67838] Updated weights for policy 0, policy_version 71422 (0.0008) [2023-10-07 22:41:41,528][67871] Updated weights for policy 1, policy_version 71530 (0.0007) [2023-10-07 22:41:41,886][67871] Updated weights for policy 1, policy_version 71540 (0.0009) [2023-10-07 22:41:42,257][67871] Updated weights for policy 1, policy_version 71550 (0.0010) [2023-10-07 22:41:42,476][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 146407424. Throughput: 0: 1653.2, 1: 1654.1. Samples: 36603432. Policy #0 lag: (min: 25.0, avg: 39.9, max: 57.0) [2023-10-07 22:41:42,477][66916] Avg episode reward: [(0, '48.490'), (1, '57.660')] [2023-10-07 22:41:43,698][67838] Updated weights for policy 0, policy_version 71432 (0.0009) [2023-10-07 22:41:44,062][67838] Updated weights for policy 0, policy_version 71442 (0.0008) [2023-10-07 22:41:44,451][67838] Updated weights for policy 0, policy_version 71452 (0.0010) [2023-10-07 22:41:46,386][67871] Updated weights for policy 1, policy_version 71560 (0.0009) [2023-10-07 22:41:46,759][67871] Updated weights for policy 1, policy_version 71570 (0.0007) [2023-10-07 22:41:47,128][67871] Updated weights for policy 1, policy_version 71580 (0.0007) [2023-10-07 22:41:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 146472960. Throughput: 0: 1653.6, 1: 1654.2. Samples: 36623850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:41:47,477][66916] Avg episode reward: [(0, '46.020'), (1, '61.780')] [2023-10-07 22:41:48,586][67838] Updated weights for policy 0, policy_version 71462 (0.0012) [2023-10-07 22:41:48,955][67838] Updated weights for policy 0, policy_version 71472 (0.0009) [2023-10-07 22:41:49,333][67838] Updated weights for policy 0, policy_version 71482 (0.0007) [2023-10-07 22:41:51,171][67871] Updated weights for policy 1, policy_version 71590 (0.0008) [2023-10-07 22:41:51,534][67871] Updated weights for policy 1, policy_version 71600 (0.0009) [2023-10-07 22:41:51,904][67871] Updated weights for policy 1, policy_version 71610 (0.0008) [2023-10-07 22:41:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 146538496. Throughput: 0: 1648.1, 1: 1645.9. Samples: 36643324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:41:52,477][66916] Avg episode reward: [(0, '45.410'), (1, '65.120')] [2023-10-07 22:41:52,487][67676] Saving new best policy, reward=65.120! [2023-10-07 22:41:53,553][67838] Updated weights for policy 0, policy_version 71492 (0.0007) [2023-10-07 22:41:53,930][67838] Updated weights for policy 0, policy_version 71502 (0.0008) [2023-10-07 22:41:54,299][67838] Updated weights for policy 0, policy_version 71512 (0.0008) [2023-10-07 22:41:56,040][67871] Updated weights for policy 1, policy_version 71620 (0.0009) [2023-10-07 22:41:56,406][67871] Updated weights for policy 1, policy_version 71630 (0.0007) [2023-10-07 22:41:56,786][67871] Updated weights for policy 1, policy_version 71640 (0.0009) [2023-10-07 22:41:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 146604032. Throughput: 0: 1648.9, 1: 1657.2. Samples: 36653238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:41:57,478][66916] Avg episode reward: [(0, '42.510'), (1, '58.800')] [2023-10-07 22:41:58,432][67838] Updated weights for policy 0, policy_version 71522 (0.0008) [2023-10-07 22:41:58,809][67838] Updated weights for policy 0, policy_version 71532 (0.0010) [2023-10-07 22:41:59,174][67838] Updated weights for policy 0, policy_version 71542 (0.0008) [2023-10-07 22:41:59,550][67838] Updated weights for policy 0, policy_version 71552 (0.0008) [2023-10-07 22:42:01,069][67871] Updated weights for policy 1, policy_version 71650 (0.0008) [2023-10-07 22:42:01,441][67871] Updated weights for policy 1, policy_version 71660 (0.0007) [2023-10-07 22:42:01,801][67871] Updated weights for policy 1, policy_version 71670 (0.0007) [2023-10-07 22:42:02,173][67871] Updated weights for policy 1, policy_version 71680 (0.0007) [2023-10-07 22:42:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 146669568. Throughput: 0: 1646.4, 1: 1652.0. Samples: 36673430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:42:02,477][66916] Avg episode reward: [(0, '44.320'), (1, '58.690')] [2023-10-07 22:42:03,789][67838] Updated weights for policy 0, policy_version 71562 (0.0007) [2023-10-07 22:42:04,162][67838] Updated weights for policy 0, policy_version 71572 (0.0010) [2023-10-07 22:42:04,537][67838] Updated weights for policy 0, policy_version 71582 (0.0009) [2023-10-07 22:42:06,259][67871] Updated weights for policy 1, policy_version 71690 (0.0007) [2023-10-07 22:42:06,612][67871] Updated weights for policy 1, policy_version 71700 (0.0008) [2023-10-07 22:42:06,992][67871] Updated weights for policy 1, policy_version 71710 (0.0010) [2023-10-07 22:42:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 146735104. Throughput: 0: 1649.4, 1: 1645.9. Samples: 36692924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:42:07,478][66916] Avg episode reward: [(0, '41.730'), (1, '54.360')] [2023-10-07 22:42:08,445][67838] Updated weights for policy 0, policy_version 71592 (0.0007) [2023-10-07 22:42:08,823][67838] Updated weights for policy 0, policy_version 71602 (0.0007) [2023-10-07 22:42:09,187][67838] Updated weights for policy 0, policy_version 71612 (0.0008) [2023-10-07 22:42:11,240][67871] Updated weights for policy 1, policy_version 71720 (0.0008) [2023-10-07 22:42:11,603][67871] Updated weights for policy 1, policy_version 71730 (0.0007) [2023-10-07 22:42:11,965][67871] Updated weights for policy 1, policy_version 71740 (0.0008) [2023-10-07 22:42:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 146800640. Throughput: 0: 1654.3, 1: 1655.8. Samples: 36703048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:42:12,477][66916] Avg episode reward: [(0, '39.850'), (1, '52.680')] [2023-10-07 22:42:13,254][67838] Updated weights for policy 0, policy_version 71622 (0.0009) [2023-10-07 22:42:13,620][67838] Updated weights for policy 0, policy_version 71632 (0.0009) [2023-10-07 22:42:13,992][67838] Updated weights for policy 0, policy_version 71642 (0.0007) [2023-10-07 22:42:16,028][67871] Updated weights for policy 1, policy_version 71750 (0.0010) [2023-10-07 22:42:16,390][67871] Updated weights for policy 1, policy_version 71760 (0.0009) [2023-10-07 22:42:16,756][67871] Updated weights for policy 1, policy_version 71770 (0.0009) [2023-10-07 22:42:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 146866176. Throughput: 0: 1657.9, 1: 1652.6. Samples: 36723320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:42:17,478][66916] Avg episode reward: [(0, '46.030'), (1, '51.990')] [2023-10-07 22:42:18,161][67838] Updated weights for policy 0, policy_version 71652 (0.0008) [2023-10-07 22:42:18,531][67838] Updated weights for policy 0, policy_version 71662 (0.0009) [2023-10-07 22:42:18,902][67838] Updated weights for policy 0, policy_version 71672 (0.0008) [2023-10-07 22:42:20,956][67871] Updated weights for policy 1, policy_version 71780 (0.0009) [2023-10-07 22:42:21,325][67871] Updated weights for policy 1, policy_version 71790 (0.0007) [2023-10-07 22:42:21,686][67871] Updated weights for policy 1, policy_version 71800 (0.0007) [2023-10-07 22:42:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 146931712. Throughput: 0: 1660.0, 1: 1644.0. Samples: 36742604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:42:22,478][66916] Avg episode reward: [(0, '45.270'), (1, '55.140')] [2023-10-07 22:42:23,017][67838] Updated weights for policy 0, policy_version 71682 (0.0008) [2023-10-07 22:42:23,379][67838] Updated weights for policy 0, policy_version 71692 (0.0008) [2023-10-07 22:42:23,753][67838] Updated weights for policy 0, policy_version 71702 (0.0009) [2023-10-07 22:42:24,116][67838] Updated weights for policy 0, policy_version 71712 (0.0009) [2023-10-07 22:42:25,770][67871] Updated weights for policy 1, policy_version 71810 (0.0009) [2023-10-07 22:42:26,145][67871] Updated weights for policy 1, policy_version 71820 (0.0008) [2023-10-07 22:42:26,507][67871] Updated weights for policy 1, policy_version 71830 (0.0009) [2023-10-07 22:42:26,876][67871] Updated weights for policy 1, policy_version 71840 (0.0010) [2023-10-07 22:42:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 146997248. Throughput: 0: 1663.4, 1: 1659.4. Samples: 36752958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:42:27,477][66916] Avg episode reward: [(0, '46.090'), (1, '52.450')] [2023-10-07 22:42:27,991][67838] Updated weights for policy 0, policy_version 71722 (0.0009) [2023-10-07 22:42:28,359][67838] Updated weights for policy 0, policy_version 71732 (0.0009) [2023-10-07 22:42:28,733][67838] Updated weights for policy 0, policy_version 71742 (0.0007) [2023-10-07 22:42:30,948][67871] Updated weights for policy 1, policy_version 71850 (0.0010) [2023-10-07 22:42:31,325][67871] Updated weights for policy 1, policy_version 71860 (0.0008) [2023-10-07 22:42:31,689][67871] Updated weights for policy 1, policy_version 71870 (0.0010) [2023-10-07 22:42:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 147062784. Throughput: 0: 1662.7, 1: 1653.6. Samples: 36773086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:42:32,477][66916] Avg episode reward: [(0, '47.140'), (1, '53.300')] [2023-10-07 22:42:32,886][67838] Updated weights for policy 0, policy_version 71752 (0.0008) [2023-10-07 22:42:33,258][67838] Updated weights for policy 0, policy_version 71762 (0.0008) [2023-10-07 22:42:33,627][67838] Updated weights for policy 0, policy_version 71772 (0.0009) [2023-10-07 22:42:35,853][67871] Updated weights for policy 1, policy_version 71880 (0.0009) [2023-10-07 22:42:36,218][67871] Updated weights for policy 1, policy_version 71890 (0.0010) [2023-10-07 22:42:36,587][67871] Updated weights for policy 1, policy_version 71900 (0.0009) [2023-10-07 22:42:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 147128320. Throughput: 0: 1668.6, 1: 1652.0. Samples: 36792750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:42:37,477][66916] Avg episode reward: [(0, '39.800'), (1, '47.780')] [2023-10-07 22:42:37,674][67838] Updated weights for policy 0, policy_version 71782 (0.0007) [2023-10-07 22:42:38,045][67838] Updated weights for policy 0, policy_version 71792 (0.0009) [2023-10-07 22:42:38,417][67838] Updated weights for policy 0, policy_version 71802 (0.0007) [2023-10-07 22:42:40,790][67871] Updated weights for policy 1, policy_version 71910 (0.0008) [2023-10-07 22:42:41,164][67871] Updated weights for policy 1, policy_version 71920 (0.0007) [2023-10-07 22:42:41,530][67871] Updated weights for policy 1, policy_version 71930 (0.0007) [2023-10-07 22:42:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 147193856. Throughput: 0: 1668.0, 1: 1661.4. Samples: 36803064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:42:42,478][66916] Avg episode reward: [(0, '38.140'), (1, '43.570')] [2023-10-07 22:42:42,521][67838] Updated weights for policy 0, policy_version 71812 (0.0008) [2023-10-07 22:42:42,888][67838] Updated weights for policy 0, policy_version 71822 (0.0009) [2023-10-07 22:42:43,269][67838] Updated weights for policy 0, policy_version 71832 (0.0008) [2023-10-07 22:42:45,694][67871] Updated weights for policy 1, policy_version 71940 (0.0010) [2023-10-07 22:42:46,058][67871] Updated weights for policy 1, policy_version 71950 (0.0011) [2023-10-07 22:42:46,425][67871] Updated weights for policy 1, policy_version 71960 (0.0007) [2023-10-07 22:42:47,321][67838] Updated weights for policy 0, policy_version 71842 (0.0008) [2023-10-07 22:42:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 147259392. Throughput: 0: 1675.9, 1: 1656.5. Samples: 36823388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:42:47,477][66916] Avg episode reward: [(0, '40.290'), (1, '44.520')] [2023-10-07 22:42:47,689][67838] Updated weights for policy 0, policy_version 71852 (0.0009) [2023-10-07 22:42:48,061][67838] Updated weights for policy 0, policy_version 71862 (0.0011) [2023-10-07 22:42:48,425][67838] Updated weights for policy 0, policy_version 71872 (0.0012) [2023-10-07 22:42:50,599][67871] Updated weights for policy 1, policy_version 71970 (0.0009) [2023-10-07 22:42:50,964][67871] Updated weights for policy 1, policy_version 71980 (0.0008) [2023-10-07 22:42:51,329][67871] Updated weights for policy 1, policy_version 71990 (0.0008) [2023-10-07 22:42:51,694][67871] Updated weights for policy 1, policy_version 72000 (0.0008) [2023-10-07 22:42:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 147324928. Throughput: 0: 1676.4, 1: 1653.4. Samples: 36842766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:42:52,478][66916] Avg episode reward: [(0, '43.360'), (1, '43.910')] [2023-10-07 22:42:52,647][67838] Updated weights for policy 0, policy_version 71882 (0.0011) [2023-10-07 22:42:53,014][67838] Updated weights for policy 0, policy_version 71892 (0.0010) [2023-10-07 22:42:53,392][67838] Updated weights for policy 0, policy_version 71902 (0.0008) [2023-10-07 22:42:55,819][67871] Updated weights for policy 1, policy_version 72010 (0.0010) [2023-10-07 22:42:56,182][67871] Updated weights for policy 1, policy_version 72020 (0.0008) [2023-10-07 22:42:56,550][67871] Updated weights for policy 1, policy_version 72030 (0.0007) [2023-10-07 22:42:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 147390464. Throughput: 0: 1671.0, 1: 1657.2. Samples: 36852820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:42:57,478][66916] Avg episode reward: [(0, '41.500'), (1, '47.340')] [2023-10-07 22:42:57,599][67838] Updated weights for policy 0, policy_version 71912 (0.0009) [2023-10-07 22:42:57,974][67838] Updated weights for policy 0, policy_version 71922 (0.0009) [2023-10-07 22:42:58,357][67838] Updated weights for policy 0, policy_version 71932 (0.0009) [2023-10-07 22:43:00,571][67871] Updated weights for policy 1, policy_version 72040 (0.0010) [2023-10-07 22:43:00,935][67871] Updated weights for policy 1, policy_version 72050 (0.0009) [2023-10-07 22:43:01,310][67871] Updated weights for policy 1, policy_version 72060 (0.0010) [2023-10-07 22:43:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 147456000. Throughput: 0: 1672.2, 1: 1647.6. Samples: 36872710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:43:02,477][66916] Avg episode reward: [(0, '43.150'), (1, '47.460')] [2023-10-07 22:43:02,499][67838] Updated weights for policy 0, policy_version 71942 (0.0009) [2023-10-07 22:43:02,878][67838] Updated weights for policy 0, policy_version 71952 (0.0009) [2023-10-07 22:43:03,257][67838] Updated weights for policy 0, policy_version 71962 (0.0007) [2023-10-07 22:43:05,670][67871] Updated weights for policy 1, policy_version 72070 (0.0009) [2023-10-07 22:43:06,052][67871] Updated weights for policy 1, policy_version 72080 (0.0011) [2023-10-07 22:43:06,416][67871] Updated weights for policy 1, policy_version 72090 (0.0010) [2023-10-07 22:43:07,206][67838] Updated weights for policy 0, policy_version 71972 (0.0008) [2023-10-07 22:43:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 147521536. Throughput: 0: 1675.9, 1: 1650.4. Samples: 36892290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:43:07,477][66916] Avg episode reward: [(0, '46.580'), (1, '48.570')] [2023-10-07 22:43:07,485][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000072096_73826304.pth... [2023-10-07 22:43:07,516][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000070528_72220672.pth [2023-10-07 22:43:07,570][67838] Updated weights for policy 0, policy_version 71982 (0.0007) [2023-10-07 22:43:07,947][67838] Updated weights for policy 0, policy_version 71992 (0.0008) [2023-10-07 22:43:08,235][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000072000_73728000.pth... [2023-10-07 22:43:08,272][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000070432_72122368.pth [2023-10-07 22:43:10,380][67871] Updated weights for policy 1, policy_version 72100 (0.0007) [2023-10-07 22:43:10,755][67871] Updated weights for policy 1, policy_version 72110 (0.0008) [2023-10-07 22:43:11,121][67871] Updated weights for policy 1, policy_version 72120 (0.0010) [2023-10-07 22:43:11,999][67838] Updated weights for policy 0, policy_version 72002 (0.0007) [2023-10-07 22:43:12,366][67838] Updated weights for policy 0, policy_version 72012 (0.0007) [2023-10-07 22:43:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 147587072. Throughput: 0: 1672.5, 1: 1651.8. Samples: 36902550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:43:12,478][66916] Avg episode reward: [(0, '40.610'), (1, '48.680')] [2023-10-07 22:43:12,740][67838] Updated weights for policy 0, policy_version 72022 (0.0009) [2023-10-07 22:43:13,110][67838] Updated weights for policy 0, policy_version 72032 (0.0008) [2023-10-07 22:43:15,407][67871] Updated weights for policy 1, policy_version 72130 (0.0007) [2023-10-07 22:43:15,765][67871] Updated weights for policy 1, policy_version 72140 (0.0008) [2023-10-07 22:43:16,140][67871] Updated weights for policy 1, policy_version 72150 (0.0007) [2023-10-07 22:43:16,500][67871] Updated weights for policy 1, policy_version 72160 (0.0008) [2023-10-07 22:43:17,337][67838] Updated weights for policy 0, policy_version 72042 (0.0009) [2023-10-07 22:43:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 147652608. Throughput: 0: 1673.9, 1: 1643.7. Samples: 36922378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:43:17,478][66916] Avg episode reward: [(0, '42.390'), (1, '45.710')] [2023-10-07 22:43:17,700][67838] Updated weights for policy 0, policy_version 72052 (0.0009) [2023-10-07 22:43:18,073][67838] Updated weights for policy 0, policy_version 72062 (0.0009) [2023-10-07 22:43:20,405][67871] Updated weights for policy 1, policy_version 72170 (0.0007) [2023-10-07 22:43:20,779][67871] Updated weights for policy 1, policy_version 72180 (0.0010) [2023-10-07 22:43:21,139][67871] Updated weights for policy 1, policy_version 72190 (0.0008) [2023-10-07 22:43:22,455][67838] Updated weights for policy 0, policy_version 72072 (0.0008) [2023-10-07 22:43:22,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 147718144. Throughput: 0: 1661.7, 1: 1654.7. Samples: 36941986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:43:22,477][66916] Avg episode reward: [(0, '41.190'), (1, '45.930')] [2023-10-07 22:43:22,836][67838] Updated weights for policy 0, policy_version 72082 (0.0009) [2023-10-07 22:43:23,209][67838] Updated weights for policy 0, policy_version 72092 (0.0007) [2023-10-07 22:43:25,262][67871] Updated weights for policy 1, policy_version 72200 (0.0009) [2023-10-07 22:43:25,631][67871] Updated weights for policy 1, policy_version 72210 (0.0008) [2023-10-07 22:43:25,990][67871] Updated weights for policy 1, policy_version 72220 (0.0009) [2023-10-07 22:43:27,247][67838] Updated weights for policy 0, policy_version 72102 (0.0009) [2023-10-07 22:43:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 147783680. Throughput: 0: 1661.6, 1: 1655.7. Samples: 36952342. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:43:27,477][66916] Avg episode reward: [(0, '39.730'), (1, '49.240')] [2023-10-07 22:43:27,613][67838] Updated weights for policy 0, policy_version 72112 (0.0008) [2023-10-07 22:43:27,993][67838] Updated weights for policy 0, policy_version 72122 (0.0007) [2023-10-07 22:43:30,090][67871] Updated weights for policy 1, policy_version 72230 (0.0007) [2023-10-07 22:43:30,450][67871] Updated weights for policy 1, policy_version 72240 (0.0010) [2023-10-07 22:43:30,817][67871] Updated weights for policy 1, policy_version 72250 (0.0010) [2023-10-07 22:43:32,131][67838] Updated weights for policy 0, policy_version 72132 (0.0009) [2023-10-07 22:43:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 147849216. Throughput: 0: 1657.7, 1: 1644.4. Samples: 36971984. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:43:32,477][66916] Avg episode reward: [(0, '40.850'), (1, '48.070')] [2023-10-07 22:43:32,497][67838] Updated weights for policy 0, policy_version 72142 (0.0011) [2023-10-07 22:43:32,876][67838] Updated weights for policy 0, policy_version 72152 (0.0009) [2023-10-07 22:43:34,859][67871] Updated weights for policy 1, policy_version 72260 (0.0008) [2023-10-07 22:43:35,225][67871] Updated weights for policy 1, policy_version 72270 (0.0010) [2023-10-07 22:43:35,591][67871] Updated weights for policy 1, policy_version 72280 (0.0011) [2023-10-07 22:43:36,783][67838] Updated weights for policy 0, policy_version 72162 (0.0009) [2023-10-07 22:43:37,163][67838] Updated weights for policy 0, policy_version 72172 (0.0009) [2023-10-07 22:43:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 147914752. Throughput: 0: 1652.9, 1: 1667.6. Samples: 36992184. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:43:37,477][66916] Avg episode reward: [(0, '41.750'), (1, '51.060')] [2023-10-07 22:43:37,541][67838] Updated weights for policy 0, policy_version 72182 (0.0008) [2023-10-07 22:43:37,911][67838] Updated weights for policy 0, policy_version 72192 (0.0009) [2023-10-07 22:43:39,583][67871] Updated weights for policy 1, policy_version 72290 (0.0009) [2023-10-07 22:43:39,949][67871] Updated weights for policy 1, policy_version 72300 (0.0009) [2023-10-07 22:43:40,316][67871] Updated weights for policy 1, policy_version 72310 (0.0008) [2023-10-07 22:43:40,681][67871] Updated weights for policy 1, policy_version 72320 (0.0008) [2023-10-07 22:43:41,870][67838] Updated weights for policy 0, policy_version 72202 (0.0009) [2023-10-07 22:43:42,228][67838] Updated weights for policy 0, policy_version 72212 (0.0008) [2023-10-07 22:43:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 147980288. Throughput: 0: 1667.9, 1: 1659.9. Samples: 37002572. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:43:42,477][66916] Avg episode reward: [(0, '42.440'), (1, '56.180')] [2023-10-07 22:43:42,598][67838] Updated weights for policy 0, policy_version 72222 (0.0008) [2023-10-07 22:43:44,803][67871] Updated weights for policy 1, policy_version 72330 (0.0007) [2023-10-07 22:43:45,160][67871] Updated weights for policy 1, policy_version 72340 (0.0010) [2023-10-07 22:43:45,523][67871] Updated weights for policy 1, policy_version 72350 (0.0007) [2023-10-07 22:43:46,536][67838] Updated weights for policy 0, policy_version 72232 (0.0008) [2023-10-07 22:43:46,905][67838] Updated weights for policy 0, policy_version 72242 (0.0007) [2023-10-07 22:43:47,278][67838] Updated weights for policy 0, policy_version 72252 (0.0010) [2023-10-07 22:43:47,476][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 148078592. Throughput: 0: 1675.2, 1: 1652.3. Samples: 37022446. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:43:47,477][66916] Avg episode reward: [(0, '42.220'), (1, '56.720')] [2023-10-07 22:43:49,634][67871] Updated weights for policy 1, policy_version 72360 (0.0009) [2023-10-07 22:43:50,014][67871] Updated weights for policy 1, policy_version 72370 (0.0010) [2023-10-07 22:43:50,378][67871] Updated weights for policy 1, policy_version 72380 (0.0011) [2023-10-07 22:43:51,474][67838] Updated weights for policy 0, policy_version 72262 (0.0007) [2023-10-07 22:43:51,829][67838] Updated weights for policy 0, policy_version 72272 (0.0008) [2023-10-07 22:43:52,215][67838] Updated weights for policy 0, policy_version 72282 (0.0008) [2023-10-07 22:43:52,477][66916] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 148144128. Throughput: 0: 1649.3, 1: 1675.5. Samples: 37041908. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:43:52,478][66916] Avg episode reward: [(0, '43.880'), (1, '58.000')] [2023-10-07 22:43:54,606][67871] Updated weights for policy 1, policy_version 72390 (0.0009) [2023-10-07 22:43:54,981][67871] Updated weights for policy 1, policy_version 72400 (0.0007) [2023-10-07 22:43:55,348][67871] Updated weights for policy 1, policy_version 72410 (0.0008) [2023-10-07 22:43:56,524][67838] Updated weights for policy 0, policy_version 72292 (0.0010) [2023-10-07 22:43:56,900][67838] Updated weights for policy 0, policy_version 72302 (0.0009) [2023-10-07 22:43:57,272][67838] Updated weights for policy 0, policy_version 72312 (0.0009) [2023-10-07 22:43:57,476][66916] Fps is (10 sec: 9830.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 148176896. Throughput: 0: 1670.4, 1: 1662.9. Samples: 37052550. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:43:57,477][66916] Avg episode reward: [(0, '40.860'), (1, '58.860')] [2023-10-07 22:43:59,385][67871] Updated weights for policy 1, policy_version 72420 (0.0009) [2023-10-07 22:43:59,757][67871] Updated weights for policy 1, policy_version 72430 (0.0011) [2023-10-07 22:44:00,124][67871] Updated weights for policy 1, policy_version 72440 (0.0008) [2023-10-07 22:44:01,275][67838] Updated weights for policy 0, policy_version 72322 (0.0007) [2023-10-07 22:44:01,658][67838] Updated weights for policy 0, policy_version 72332 (0.0008) [2023-10-07 22:44:02,025][67838] Updated weights for policy 0, policy_version 72342 (0.0008) [2023-10-07 22:44:02,400][67838] Updated weights for policy 0, policy_version 72352 (0.0008) [2023-10-07 22:44:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 148275200. Throughput: 0: 1670.1, 1: 1658.8. Samples: 37072176. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:44:02,478][66916] Avg episode reward: [(0, '44.970'), (1, '59.920')] [2023-10-07 22:44:04,328][67871] Updated weights for policy 1, policy_version 72450 (0.0008) [2023-10-07 22:44:04,702][67871] Updated weights for policy 1, policy_version 72460 (0.0009) [2023-10-07 22:44:05,069][67871] Updated weights for policy 1, policy_version 72470 (0.0009) [2023-10-07 22:44:05,432][67871] Updated weights for policy 1, policy_version 72480 (0.0009) [2023-10-07 22:44:06,598][67838] Updated weights for policy 0, policy_version 72362 (0.0007) [2023-10-07 22:44:06,961][67838] Updated weights for policy 0, policy_version 72372 (0.0008) [2023-10-07 22:44:07,340][67838] Updated weights for policy 0, policy_version 72382 (0.0009) [2023-10-07 22:44:07,476][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 148340736. Throughput: 0: 1653.5, 1: 1674.3. Samples: 37091736. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 22:44:07,477][66916] Avg episode reward: [(0, '42.900'), (1, '55.210')] [2023-10-07 22:44:09,434][67871] Updated weights for policy 1, policy_version 72490 (0.0009) [2023-10-07 22:44:09,793][67871] Updated weights for policy 1, policy_version 72500 (0.0008) [2023-10-07 22:44:10,158][67871] Updated weights for policy 1, policy_version 72510 (0.0009) [2023-10-07 22:44:11,304][67838] Updated weights for policy 0, policy_version 72392 (0.0010) [2023-10-07 22:44:11,672][67838] Updated weights for policy 0, policy_version 72402 (0.0008) [2023-10-07 22:44:12,041][67838] Updated weights for policy 0, policy_version 72412 (0.0007) [2023-10-07 22:44:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 148406272. Throughput: 0: 1675.8, 1: 1653.2. Samples: 37102148. Policy #0 lag: (min: 1.0, avg: 10.4, max: 33.0) [2023-10-07 22:44:12,478][66916] Avg episode reward: [(0, '47.570'), (1, '54.260')] [2023-10-07 22:44:14,279][67871] Updated weights for policy 1, policy_version 72520 (0.0010) [2023-10-07 22:44:14,643][67871] Updated weights for policy 1, policy_version 72530 (0.0010) [2023-10-07 22:44:15,017][67871] Updated weights for policy 1, policy_version 72540 (0.0007) [2023-10-07 22:44:16,216][67838] Updated weights for policy 0, policy_version 72422 (0.0007) [2023-10-07 22:44:16,585][67838] Updated weights for policy 0, policy_version 72432 (0.0007) [2023-10-07 22:44:16,955][67838] Updated weights for policy 0, policy_version 72442 (0.0008) [2023-10-07 22:44:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 148471808. Throughput: 0: 1671.4, 1: 1660.6. Samples: 37121924. Policy #0 lag: (min: 1.0, avg: 10.4, max: 33.0) [2023-10-07 22:44:17,477][66916] Avg episode reward: [(0, '47.990'), (1, '50.890')] [2023-10-07 22:44:19,296][67871] Updated weights for policy 1, policy_version 72550 (0.0009) [2023-10-07 22:44:19,670][67871] Updated weights for policy 1, policy_version 72560 (0.0009) [2023-10-07 22:44:20,038][67871] Updated weights for policy 1, policy_version 72570 (0.0007) [2023-10-07 22:44:21,128][67838] Updated weights for policy 0, policy_version 72452 (0.0007) [2023-10-07 22:44:21,496][67838] Updated weights for policy 0, policy_version 72462 (0.0007) [2023-10-07 22:44:21,861][67838] Updated weights for policy 0, policy_version 72472 (0.0007) [2023-10-07 22:44:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 148537344. Throughput: 0: 1652.6, 1: 1665.5. Samples: 37141498. Policy #0 lag: (min: 1.0, avg: 10.4, max: 33.0) [2023-10-07 22:44:22,478][66916] Avg episode reward: [(0, '49.870'), (1, '52.000')] [2023-10-07 22:44:24,174][67871] Updated weights for policy 1, policy_version 72580 (0.0007) [2023-10-07 22:44:24,549][67871] Updated weights for policy 1, policy_version 72590 (0.0007) [2023-10-07 22:44:24,911][67871] Updated weights for policy 1, policy_version 72600 (0.0007) [2023-10-07 22:44:26,091][67838] Updated weights for policy 0, policy_version 72482 (0.0009) [2023-10-07 22:44:26,502][67838] Updated weights for policy 0, policy_version 72492 (0.0009) [2023-10-07 22:44:26,870][67838] Updated weights for policy 0, policy_version 72502 (0.0008) [2023-10-07 22:44:27,246][67838] Updated weights for policy 0, policy_version 72512 (0.0007) [2023-10-07 22:44:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 148602880. Throughput: 0: 1665.7, 1: 1658.5. Samples: 37152162. Policy #0 lag: (min: 1.0, avg: 10.4, max: 33.0) [2023-10-07 22:44:27,477][66916] Avg episode reward: [(0, '48.480'), (1, '51.130')] [2023-10-07 22:44:28,933][67871] Updated weights for policy 1, policy_version 72610 (0.0007) [2023-10-07 22:44:29,304][67871] Updated weights for policy 1, policy_version 72620 (0.0009) [2023-10-07 22:44:29,669][67871] Updated weights for policy 1, policy_version 72630 (0.0007) [2023-10-07 22:44:30,032][67871] Updated weights for policy 1, policy_version 72640 (0.0007) [2023-10-07 22:44:31,313][67838] Updated weights for policy 0, policy_version 72522 (0.0010) [2023-10-07 22:44:31,675][67838] Updated weights for policy 0, policy_version 72532 (0.0009) [2023-10-07 22:44:32,055][67838] Updated weights for policy 0, policy_version 72542 (0.0008) [2023-10-07 22:44:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 148668416. Throughput: 0: 1656.5, 1: 1669.4. Samples: 37172112. Policy #0 lag: (min: 1.0, avg: 10.4, max: 33.0) [2023-10-07 22:44:32,477][66916] Avg episode reward: [(0, '51.600'), (1, '51.240')] [2023-10-07 22:44:34,165][67871] Updated weights for policy 1, policy_version 72650 (0.0009) [2023-10-07 22:44:34,533][67871] Updated weights for policy 1, policy_version 72660 (0.0008) [2023-10-07 22:44:34,904][67871] Updated weights for policy 1, policy_version 72670 (0.0008) [2023-10-07 22:44:36,065][67838] Updated weights for policy 0, policy_version 72552 (0.0008) [2023-10-07 22:44:36,429][67838] Updated weights for policy 0, policy_version 72562 (0.0007) [2023-10-07 22:44:36,810][67838] Updated weights for policy 0, policy_version 72572 (0.0007) [2023-10-07 22:44:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 148733952. Throughput: 0: 1656.4, 1: 1671.2. Samples: 37191648. Policy #0 lag: (min: 1.0, avg: 10.4, max: 33.0) [2023-10-07 22:44:37,478][66916] Avg episode reward: [(0, '53.100'), (1, '54.300')] [2023-10-07 22:44:38,932][67871] Updated weights for policy 1, policy_version 72680 (0.0010) [2023-10-07 22:44:39,303][67871] Updated weights for policy 1, policy_version 72690 (0.0007) [2023-10-07 22:44:39,666][67871] Updated weights for policy 1, policy_version 72700 (0.0007) [2023-10-07 22:44:40,692][67838] Updated weights for policy 0, policy_version 72582 (0.0008) [2023-10-07 22:44:41,067][67838] Updated weights for policy 0, policy_version 72592 (0.0010) [2023-10-07 22:44:41,446][67838] Updated weights for policy 0, policy_version 72602 (0.0007) [2023-10-07 22:44:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 148799488. Throughput: 0: 1671.6, 1: 1652.0. Samples: 37202116. Policy #0 lag: (min: 1.0, avg: 10.4, max: 33.0) [2023-10-07 22:44:42,478][66916] Avg episode reward: [(0, '52.240'), (1, '51.750')] [2023-10-07 22:44:43,936][67871] Updated weights for policy 1, policy_version 72710 (0.0009) [2023-10-07 22:44:44,304][67871] Updated weights for policy 1, policy_version 72720 (0.0009) [2023-10-07 22:44:44,674][67871] Updated weights for policy 1, policy_version 72730 (0.0009) [2023-10-07 22:44:45,546][67838] Updated weights for policy 0, policy_version 72612 (0.0009) [2023-10-07 22:44:45,918][67838] Updated weights for policy 0, policy_version 72622 (0.0008) [2023-10-07 22:44:46,296][67838] Updated weights for policy 0, policy_version 72632 (0.0007) [2023-10-07 22:44:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 148865024. Throughput: 0: 1657.2, 1: 1667.3. Samples: 37221780. Policy #0 lag: (min: 1.0, avg: 10.4, max: 33.0) [2023-10-07 22:44:47,477][66916] Avg episode reward: [(0, '50.290'), (1, '52.400')] [2023-10-07 22:44:48,811][67871] Updated weights for policy 1, policy_version 72740 (0.0008) [2023-10-07 22:44:49,213][67871] Updated weights for policy 1, policy_version 72750 (0.0010) [2023-10-07 22:44:49,576][67871] Updated weights for policy 1, policy_version 72760 (0.0007) [2023-10-07 22:44:50,425][67838] Updated weights for policy 0, policy_version 72642 (0.0009) [2023-10-07 22:44:50,793][67838] Updated weights for policy 0, policy_version 72652 (0.0009) [2023-10-07 22:44:51,159][67838] Updated weights for policy 0, policy_version 72662 (0.0009) [2023-10-07 22:44:51,532][67838] Updated weights for policy 0, policy_version 72672 (0.0007) [2023-10-07 22:44:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 148930560. Throughput: 0: 1664.4, 1: 1663.1. Samples: 37241476. Policy #0 lag: (min: 1.0, avg: 10.4, max: 33.0) [2023-10-07 22:44:52,478][66916] Avg episode reward: [(0, '50.680'), (1, '51.160')] [2023-10-07 22:44:53,773][67871] Updated weights for policy 1, policy_version 72770 (0.0009) [2023-10-07 22:44:54,137][67871] Updated weights for policy 1, policy_version 72780 (0.0009) [2023-10-07 22:44:54,501][67871] Updated weights for policy 1, policy_version 72790 (0.0009) [2023-10-07 22:44:54,866][67871] Updated weights for policy 1, policy_version 72800 (0.0009) [2023-10-07 22:44:55,687][67838] Updated weights for policy 0, policy_version 72682 (0.0010) [2023-10-07 22:44:56,063][67838] Updated weights for policy 0, policy_version 72692 (0.0009) [2023-10-07 22:44:56,437][67838] Updated weights for policy 0, policy_version 72702 (0.0009) [2023-10-07 22:44:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 148996096. Throughput: 0: 1672.1, 1: 1653.1. Samples: 37251782. Policy #0 lag: (min: 27.0, avg: 37.6, max: 59.0) [2023-10-07 22:44:57,477][66916] Avg episode reward: [(0, '47.630'), (1, '53.530')] [2023-10-07 22:44:58,896][67871] Updated weights for policy 1, policy_version 72810 (0.0009) [2023-10-07 22:44:59,274][67871] Updated weights for policy 1, policy_version 72820 (0.0008) [2023-10-07 22:44:59,640][67871] Updated weights for policy 1, policy_version 72830 (0.0007) [2023-10-07 22:45:00,489][67838] Updated weights for policy 0, policy_version 72712 (0.0007) [2023-10-07 22:45:00,864][67838] Updated weights for policy 0, policy_version 72722 (0.0008) [2023-10-07 22:45:01,244][67838] Updated weights for policy 0, policy_version 72732 (0.0008) [2023-10-07 22:45:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149061632. Throughput: 0: 1652.3, 1: 1667.3. Samples: 37271302. Policy #0 lag: (min: 27.0, avg: 37.6, max: 59.0) [2023-10-07 22:45:02,477][66916] Avg episode reward: [(0, '49.890'), (1, '50.400')] [2023-10-07 22:45:03,651][67871] Updated weights for policy 1, policy_version 72840 (0.0007) [2023-10-07 22:45:04,024][67871] Updated weights for policy 1, policy_version 72850 (0.0007) [2023-10-07 22:45:04,381][67871] Updated weights for policy 1, policy_version 72860 (0.0007) [2023-10-07 22:45:05,342][67838] Updated weights for policy 0, policy_version 72742 (0.0008) [2023-10-07 22:45:05,704][67838] Updated weights for policy 0, policy_version 72752 (0.0009) [2023-10-07 22:45:06,088][67838] Updated weights for policy 0, policy_version 72762 (0.0007) [2023-10-07 22:45:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 149127168. Throughput: 0: 1674.4, 1: 1664.6. Samples: 37291752. Policy #0 lag: (min: 27.0, avg: 37.6, max: 59.0) [2023-10-07 22:45:07,478][66916] Avg episode reward: [(0, '47.640'), (1, '52.060')] [2023-10-07 22:45:07,488][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000072864_74612736.pth... [2023-10-07 22:45:07,488][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000072768_74514432.pth... [2023-10-07 22:45:07,520][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000071200_72908800.pth [2023-10-07 22:45:07,527][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000071328_73039872.pth [2023-10-07 22:45:08,572][67871] Updated weights for policy 1, policy_version 72870 (0.0009) [2023-10-07 22:45:08,942][67871] Updated weights for policy 1, policy_version 72880 (0.0010) [2023-10-07 22:45:09,314][67871] Updated weights for policy 1, policy_version 72890 (0.0010) [2023-10-07 22:45:10,081][67838] Updated weights for policy 0, policy_version 72772 (0.0008) [2023-10-07 22:45:10,456][67838] Updated weights for policy 0, policy_version 72782 (0.0008) [2023-10-07 22:45:10,831][67838] Updated weights for policy 0, policy_version 72792 (0.0011) [2023-10-07 22:45:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 149192704. Throughput: 0: 1674.2, 1: 1651.1. Samples: 37301804. Policy #0 lag: (min: 27.0, avg: 37.6, max: 59.0) [2023-10-07 22:45:12,478][66916] Avg episode reward: [(0, '47.800'), (1, '53.280')] [2023-10-07 22:45:13,289][67871] Updated weights for policy 1, policy_version 72900 (0.0010) [2023-10-07 22:45:13,653][67871] Updated weights for policy 1, policy_version 72910 (0.0011) [2023-10-07 22:45:14,027][67871] Updated weights for policy 1, policy_version 72920 (0.0010) [2023-10-07 22:45:15,027][67838] Updated weights for policy 0, policy_version 72802 (0.0011) [2023-10-07 22:45:15,415][67838] Updated weights for policy 0, policy_version 72812 (0.0011) [2023-10-07 22:45:15,789][67838] Updated weights for policy 0, policy_version 72822 (0.0011) [2023-10-07 22:45:16,158][67838] Updated weights for policy 0, policy_version 72832 (0.0008) [2023-10-07 22:45:17,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149258240. Throughput: 0: 1652.5, 1: 1660.7. Samples: 37321206. Policy #0 lag: (min: 27.0, avg: 37.6, max: 59.0) [2023-10-07 22:45:17,477][66916] Avg episode reward: [(0, '50.580'), (1, '51.890')] [2023-10-07 22:45:18,145][67871] Updated weights for policy 1, policy_version 72930 (0.0009) [2023-10-07 22:45:18,509][67871] Updated weights for policy 1, policy_version 72940 (0.0009) [2023-10-07 22:45:18,879][67871] Updated weights for policy 1, policy_version 72950 (0.0010) [2023-10-07 22:45:19,260][67871] Updated weights for policy 1, policy_version 72960 (0.0009) [2023-10-07 22:45:20,238][67838] Updated weights for policy 0, policy_version 72842 (0.0009) [2023-10-07 22:45:20,605][67838] Updated weights for policy 0, policy_version 72852 (0.0009) [2023-10-07 22:45:20,970][67838] Updated weights for policy 0, policy_version 72862 (0.0008) [2023-10-07 22:45:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149323776. Throughput: 0: 1669.6, 1: 1661.6. Samples: 37341556. Policy #0 lag: (min: 27.0, avg: 37.6, max: 59.0) [2023-10-07 22:45:22,477][66916] Avg episode reward: [(0, '46.840'), (1, '50.070')] [2023-10-07 22:45:23,429][67871] Updated weights for policy 1, policy_version 72970 (0.0007) [2023-10-07 22:45:23,797][67871] Updated weights for policy 1, policy_version 72980 (0.0010) [2023-10-07 22:45:24,175][67871] Updated weights for policy 1, policy_version 72990 (0.0012) [2023-10-07 22:45:25,216][67838] Updated weights for policy 0, policy_version 72872 (0.0009) [2023-10-07 22:45:25,593][67838] Updated weights for policy 0, policy_version 72882 (0.0008) [2023-10-07 22:45:25,967][67838] Updated weights for policy 0, policy_version 72892 (0.0007) [2023-10-07 22:45:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149389312. Throughput: 0: 1663.9, 1: 1661.5. Samples: 37351760. Policy #0 lag: (min: 27.0, avg: 37.6, max: 59.0) [2023-10-07 22:45:27,478][66916] Avg episode reward: [(0, '47.590'), (1, '51.210')] [2023-10-07 22:45:28,366][67871] Updated weights for policy 1, policy_version 73000 (0.0010) [2023-10-07 22:45:28,731][67871] Updated weights for policy 1, policy_version 73010 (0.0010) [2023-10-07 22:45:29,097][67871] Updated weights for policy 1, policy_version 73020 (0.0007) [2023-10-07 22:45:30,031][67838] Updated weights for policy 0, policy_version 72902 (0.0008) [2023-10-07 22:45:30,393][67838] Updated weights for policy 0, policy_version 72912 (0.0008) [2023-10-07 22:45:30,760][67838] Updated weights for policy 0, policy_version 72922 (0.0007) [2023-10-07 22:45:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149454848. Throughput: 0: 1656.1, 1: 1669.8. Samples: 37371446. Policy #0 lag: (min: 27.0, avg: 37.6, max: 59.0) [2023-10-07 22:45:32,477][66916] Avg episode reward: [(0, '45.900'), (1, '47.240')] [2023-10-07 22:45:33,015][67871] Updated weights for policy 1, policy_version 73030 (0.0009) [2023-10-07 22:45:33,393][67871] Updated weights for policy 1, policy_version 73040 (0.0007) [2023-10-07 22:45:33,763][67871] Updated weights for policy 1, policy_version 73050 (0.0008) [2023-10-07 22:45:34,842][67838] Updated weights for policy 0, policy_version 72932 (0.0007) [2023-10-07 22:45:35,221][67838] Updated weights for policy 0, policy_version 72942 (0.0010) [2023-10-07 22:45:35,597][67838] Updated weights for policy 0, policy_version 72952 (0.0007) [2023-10-07 22:45:37,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 149520384. Throughput: 0: 1670.5, 1: 1672.8. Samples: 37391924. Policy #0 lag: (min: 27.0, avg: 37.6, max: 59.0) [2023-10-07 22:45:37,477][66916] Avg episode reward: [(0, '48.350'), (1, '49.890')] [2023-10-07 22:45:37,990][67871] Updated weights for policy 1, policy_version 73060 (0.0007) [2023-10-07 22:45:38,374][67871] Updated weights for policy 1, policy_version 73070 (0.0007) [2023-10-07 22:45:38,734][67871] Updated weights for policy 1, policy_version 73080 (0.0007) [2023-10-07 22:45:39,714][67838] Updated weights for policy 0, policy_version 72962 (0.0007) [2023-10-07 22:45:40,078][67838] Updated weights for policy 0, policy_version 72972 (0.0007) [2023-10-07 22:45:40,445][67838] Updated weights for policy 0, policy_version 72982 (0.0007) [2023-10-07 22:45:40,820][67838] Updated weights for policy 0, policy_version 72992 (0.0008) [2023-10-07 22:45:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 149585920. Throughput: 0: 1658.0, 1: 1669.4. Samples: 37401516. Policy #0 lag: (min: 27.0, avg: 37.6, max: 59.0) [2023-10-07 22:45:42,478][66916] Avg episode reward: [(0, '47.140'), (1, '48.450')] [2023-10-07 22:45:42,769][67871] Updated weights for policy 1, policy_version 73090 (0.0008) [2023-10-07 22:45:43,135][67871] Updated weights for policy 1, policy_version 73100 (0.0009) [2023-10-07 22:45:43,516][67871] Updated weights for policy 1, policy_version 73110 (0.0009) [2023-10-07 22:45:43,875][67871] Updated weights for policy 1, policy_version 73120 (0.0009) [2023-10-07 22:45:44,857][67838] Updated weights for policy 0, policy_version 73002 (0.0008) [2023-10-07 22:45:45,237][67838] Updated weights for policy 0, policy_version 73012 (0.0010) [2023-10-07 22:45:45,609][67838] Updated weights for policy 0, policy_version 73022 (0.0007) [2023-10-07 22:45:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149651456. Throughput: 0: 1663.2, 1: 1669.0. Samples: 37421254. Policy #0 lag: (min: 25.0, avg: 36.8, max: 57.0) [2023-10-07 22:45:47,477][66916] Avg episode reward: [(0, '43.910'), (1, '49.550')] [2023-10-07 22:45:48,110][67871] Updated weights for policy 1, policy_version 73130 (0.0008) [2023-10-07 22:45:48,478][67871] Updated weights for policy 1, policy_version 73140 (0.0009) [2023-10-07 22:45:48,842][67871] Updated weights for policy 1, policy_version 73150 (0.0009) [2023-10-07 22:45:49,574][67838] Updated weights for policy 0, policy_version 73032 (0.0007) [2023-10-07 22:45:49,937][67838] Updated weights for policy 0, policy_version 73042 (0.0007) [2023-10-07 22:45:50,314][67838] Updated weights for policy 0, policy_version 73052 (0.0009) [2023-10-07 22:45:52,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149716992. Throughput: 0: 1671.6, 1: 1663.5. Samples: 37441830. Policy #0 lag: (min: 25.0, avg: 36.8, max: 57.0) [2023-10-07 22:45:52,478][66916] Avg episode reward: [(0, '47.140'), (1, '50.830')] [2023-10-07 22:45:53,172][67871] Updated weights for policy 1, policy_version 73160 (0.0009) [2023-10-07 22:45:53,541][67871] Updated weights for policy 1, policy_version 73170 (0.0008) [2023-10-07 22:45:53,902][67871] Updated weights for policy 1, policy_version 73180 (0.0008) [2023-10-07 22:45:54,267][67838] Updated weights for policy 0, policy_version 73062 (0.0008) [2023-10-07 22:45:54,640][67838] Updated weights for policy 0, policy_version 73072 (0.0007) [2023-10-07 22:45:55,021][67838] Updated weights for policy 0, policy_version 73082 (0.0009) [2023-10-07 22:45:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149782528. Throughput: 0: 1656.5, 1: 1668.4. Samples: 37451422. Policy #0 lag: (min: 25.0, avg: 36.8, max: 57.0) [2023-10-07 22:45:57,477][66916] Avg episode reward: [(0, '45.020'), (1, '49.920')] [2023-10-07 22:45:57,950][67871] Updated weights for policy 1, policy_version 73190 (0.0007) [2023-10-07 22:45:58,320][67871] Updated weights for policy 1, policy_version 73200 (0.0010) [2023-10-07 22:45:58,691][67871] Updated weights for policy 1, policy_version 73210 (0.0010) [2023-10-07 22:45:59,037][67838] Updated weights for policy 0, policy_version 73092 (0.0009) [2023-10-07 22:45:59,404][67838] Updated weights for policy 0, policy_version 73102 (0.0008) [2023-10-07 22:45:59,774][67838] Updated weights for policy 0, policy_version 73112 (0.0008) [2023-10-07 22:46:02,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149848064. Throughput: 0: 1681.5, 1: 1669.3. Samples: 37471994. Policy #0 lag: (min: 25.0, avg: 36.8, max: 57.0) [2023-10-07 22:46:02,478][66916] Avg episode reward: [(0, '48.260'), (1, '49.670')] [2023-10-07 22:46:02,790][67871] Updated weights for policy 1, policy_version 73220 (0.0008) [2023-10-07 22:46:03,165][67871] Updated weights for policy 1, policy_version 73230 (0.0007) [2023-10-07 22:46:03,527][67871] Updated weights for policy 1, policy_version 73240 (0.0007) [2023-10-07 22:46:03,933][67838] Updated weights for policy 0, policy_version 73122 (0.0007) [2023-10-07 22:46:04,324][67838] Updated weights for policy 0, policy_version 73132 (0.0007) [2023-10-07 22:46:04,694][67838] Updated weights for policy 0, policy_version 73142 (0.0007) [2023-10-07 22:46:05,080][67838] Updated weights for policy 0, policy_version 73152 (0.0007) [2023-10-07 22:46:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 149913600. Throughput: 0: 1688.0, 1: 1670.0. Samples: 37492668. Policy #0 lag: (min: 25.0, avg: 36.8, max: 57.0) [2023-10-07 22:46:07,478][66916] Avg episode reward: [(0, '48.770'), (1, '51.990')] [2023-10-07 22:46:07,512][67871] Updated weights for policy 1, policy_version 73250 (0.0010) [2023-10-07 22:46:07,879][67871] Updated weights for policy 1, policy_version 73260 (0.0007) [2023-10-07 22:46:08,251][67871] Updated weights for policy 1, policy_version 73270 (0.0010) [2023-10-07 22:46:08,615][67871] Updated weights for policy 1, policy_version 73280 (0.0008) [2023-10-07 22:46:09,115][67838] Updated weights for policy 0, policy_version 73162 (0.0009) [2023-10-07 22:46:09,481][67838] Updated weights for policy 0, policy_version 73172 (0.0008) [2023-10-07 22:46:09,856][67838] Updated weights for policy 0, policy_version 73182 (0.0007) [2023-10-07 22:46:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 149979136. Throughput: 0: 1658.1, 1: 1672.6. Samples: 37501640. Policy #0 lag: (min: 25.0, avg: 36.8, max: 57.0) [2023-10-07 22:46:12,478][66916] Avg episode reward: [(0, '47.040'), (1, '50.520')] [2023-10-07 22:46:12,714][67871] Updated weights for policy 1, policy_version 73290 (0.0008) [2023-10-07 22:46:13,080][67871] Updated weights for policy 1, policy_version 73300 (0.0009) [2023-10-07 22:46:13,454][67871] Updated weights for policy 1, policy_version 73310 (0.0009) [2023-10-07 22:46:13,833][67838] Updated weights for policy 0, policy_version 73192 (0.0009) [2023-10-07 22:46:14,198][67838] Updated weights for policy 0, policy_version 73202 (0.0008) [2023-10-07 22:46:14,565][67838] Updated weights for policy 0, policy_version 73212 (0.0008) [2023-10-07 22:46:17,434][67871] Updated weights for policy 1, policy_version 73320 (0.0009) [2023-10-07 22:46:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 150044672. Throughput: 0: 1683.0, 1: 1671.6. Samples: 37522400. Policy #0 lag: (min: 25.0, avg: 36.8, max: 57.0) [2023-10-07 22:46:17,477][66916] Avg episode reward: [(0, '48.950'), (1, '51.790')] [2023-10-07 22:46:17,806][67871] Updated weights for policy 1, policy_version 73330 (0.0007) [2023-10-07 22:46:18,179][67871] Updated weights for policy 1, policy_version 73340 (0.0010) [2023-10-07 22:46:18,703][67838] Updated weights for policy 0, policy_version 73222 (0.0008) [2023-10-07 22:46:19,076][67838] Updated weights for policy 0, policy_version 73232 (0.0008) [2023-10-07 22:46:19,447][67838] Updated weights for policy 0, policy_version 73242 (0.0008) [2023-10-07 22:46:22,316][67871] Updated weights for policy 1, policy_version 73350 (0.0008) [2023-10-07 22:46:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 150110208. Throughput: 0: 1681.1, 1: 1670.0. Samples: 37542722. Policy #0 lag: (min: 25.0, avg: 36.8, max: 57.0) [2023-10-07 22:46:22,477][66916] Avg episode reward: [(0, '48.170'), (1, '53.020')] [2023-10-07 22:46:22,694][67871] Updated weights for policy 1, policy_version 73360 (0.0011) [2023-10-07 22:46:23,058][67871] Updated weights for policy 1, policy_version 73370 (0.0009) [2023-10-07 22:46:23,589][67838] Updated weights for policy 0, policy_version 73252 (0.0007) [2023-10-07 22:46:23,950][67838] Updated weights for policy 0, policy_version 73262 (0.0008) [2023-10-07 22:46:24,332][67838] Updated weights for policy 0, policy_version 73272 (0.0009) [2023-10-07 22:46:27,239][67871] Updated weights for policy 1, policy_version 73380 (0.0009) [2023-10-07 22:46:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 150175744. Throughput: 0: 1664.4, 1: 1672.2. Samples: 37551662. Policy #0 lag: (min: 25.0, avg: 36.8, max: 57.0) [2023-10-07 22:46:27,477][66916] Avg episode reward: [(0, '51.860'), (1, '51.180')] [2023-10-07 22:46:27,609][67871] Updated weights for policy 1, policy_version 73390 (0.0010) [2023-10-07 22:46:27,972][67871] Updated weights for policy 1, policy_version 73400 (0.0009) [2023-10-07 22:46:28,231][67838] Updated weights for policy 0, policy_version 73282 (0.0008) [2023-10-07 22:46:28,610][67838] Updated weights for policy 0, policy_version 73292 (0.0007) [2023-10-07 22:46:28,978][67838] Updated weights for policy 0, policy_version 73302 (0.0008) [2023-10-07 22:46:29,355][67838] Updated weights for policy 0, policy_version 73312 (0.0010) [2023-10-07 22:46:32,145][67871] Updated weights for policy 1, policy_version 73410 (0.0009) [2023-10-07 22:46:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 150241280. Throughput: 0: 1685.6, 1: 1673.3. Samples: 37572404. Policy #0 lag: (min: 25.0, avg: 36.8, max: 57.0) [2023-10-07 22:46:32,477][66916] Avg episode reward: [(0, '53.540'), (1, '50.200')] [2023-10-07 22:46:32,514][67871] Updated weights for policy 1, policy_version 73420 (0.0010) [2023-10-07 22:46:32,885][67871] Updated weights for policy 1, policy_version 73430 (0.0010) [2023-10-07 22:46:33,248][67871] Updated weights for policy 1, policy_version 73440 (0.0009) [2023-10-07 22:46:33,569][67838] Updated weights for policy 0, policy_version 73322 (0.0010) [2023-10-07 22:46:33,943][67838] Updated weights for policy 0, policy_version 73332 (0.0010) [2023-10-07 22:46:34,325][67838] Updated weights for policy 0, policy_version 73342 (0.0009) [2023-10-07 22:46:37,412][67871] Updated weights for policy 1, policy_version 73450 (0.0007) [2023-10-07 22:46:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 150306816. Throughput: 0: 1679.8, 1: 1674.7. Samples: 37592782. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-07 22:46:37,478][66916] Avg episode reward: [(0, '51.930'), (1, '48.960')] [2023-10-07 22:46:37,785][67871] Updated weights for policy 1, policy_version 73460 (0.0008) [2023-10-07 22:46:38,148][67871] Updated weights for policy 1, policy_version 73470 (0.0008) [2023-10-07 22:46:38,327][67838] Updated weights for policy 0, policy_version 73352 (0.0008) [2023-10-07 22:46:38,704][67838] Updated weights for policy 0, policy_version 73362 (0.0009) [2023-10-07 22:46:39,064][67838] Updated weights for policy 0, policy_version 73372 (0.0011) [2023-10-07 22:46:42,278][67871] Updated weights for policy 1, policy_version 73480 (0.0007) [2023-10-07 22:46:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 150372352. Throughput: 0: 1671.1, 1: 1669.5. Samples: 37601750. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-07 22:46:42,477][66916] Avg episode reward: [(0, '53.650'), (1, '49.340')] [2023-10-07 22:46:42,642][67871] Updated weights for policy 1, policy_version 73490 (0.0010) [2023-10-07 22:46:43,011][67871] Updated weights for policy 1, policy_version 73500 (0.0008) [2023-10-07 22:46:43,237][67838] Updated weights for policy 0, policy_version 73382 (0.0008) [2023-10-07 22:46:43,601][67838] Updated weights for policy 0, policy_version 73392 (0.0008) [2023-10-07 22:46:43,975][67838] Updated weights for policy 0, policy_version 73402 (0.0009) [2023-10-07 22:46:47,090][67871] Updated weights for policy 1, policy_version 73510 (0.0009) [2023-10-07 22:46:47,453][67871] Updated weights for policy 1, policy_version 73520 (0.0012) [2023-10-07 22:46:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 150437888. Throughput: 0: 1669.1, 1: 1665.9. Samples: 37622066. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-07 22:46:47,477][66916] Avg episode reward: [(0, '52.800'), (1, '49.810')] [2023-10-07 22:46:47,823][67871] Updated weights for policy 1, policy_version 73530 (0.0010) [2023-10-07 22:46:48,347][67838] Updated weights for policy 0, policy_version 73412 (0.0010) [2023-10-07 22:46:48,711][67838] Updated weights for policy 0, policy_version 73422 (0.0009) [2023-10-07 22:46:49,096][67838] Updated weights for policy 0, policy_version 73432 (0.0010) [2023-10-07 22:46:51,889][67871] Updated weights for policy 1, policy_version 73540 (0.0010) [2023-10-07 22:46:52,259][67871] Updated weights for policy 1, policy_version 73550 (0.0007) [2023-10-07 22:46:52,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 150503424. Throughput: 0: 1665.8, 1: 1659.1. Samples: 37642286. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-07 22:46:52,477][66916] Avg episode reward: [(0, '50.540'), (1, '52.330')] [2023-10-07 22:46:52,629][67871] Updated weights for policy 1, policy_version 73560 (0.0008) [2023-10-07 22:46:53,312][67838] Updated weights for policy 0, policy_version 73442 (0.0011) [2023-10-07 22:46:53,699][67838] Updated weights for policy 0, policy_version 73452 (0.0010) [2023-10-07 22:46:54,073][67838] Updated weights for policy 0, policy_version 73462 (0.0010) [2023-10-07 22:46:54,435][67838] Updated weights for policy 0, policy_version 73472 (0.0010) [2023-10-07 22:46:56,666][67871] Updated weights for policy 1, policy_version 73570 (0.0008) [2023-10-07 22:46:57,037][67871] Updated weights for policy 1, policy_version 73580 (0.0008) [2023-10-07 22:46:57,406][67871] Updated weights for policy 1, policy_version 73590 (0.0011) [2023-10-07 22:46:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 150568960. Throughput: 0: 1663.0, 1: 1663.2. Samples: 37651320. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-07 22:46:57,478][66916] Avg episode reward: [(0, '49.400'), (1, '52.980')] [2023-10-07 22:46:57,776][67871] Updated weights for policy 1, policy_version 73600 (0.0008) [2023-10-07 22:46:58,574][67838] Updated weights for policy 0, policy_version 73482 (0.0009) [2023-10-07 22:46:58,933][67838] Updated weights for policy 0, policy_version 73492 (0.0009) [2023-10-07 22:46:59,307][67838] Updated weights for policy 0, policy_version 73502 (0.0010) [2023-10-07 22:47:01,827][67871] Updated weights for policy 1, policy_version 73610 (0.0008) [2023-10-07 22:47:02,199][67871] Updated weights for policy 1, policy_version 73620 (0.0007) [2023-10-07 22:47:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 150634496. Throughput: 0: 1657.3, 1: 1660.9. Samples: 37671720. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-07 22:47:02,478][66916] Avg episode reward: [(0, '53.200'), (1, '55.020')] [2023-10-07 22:47:02,572][67871] Updated weights for policy 1, policy_version 73630 (0.0007) [2023-10-07 22:47:03,532][67838] Updated weights for policy 0, policy_version 73512 (0.0007) [2023-10-07 22:47:03,904][67838] Updated weights for policy 0, policy_version 73522 (0.0007) [2023-10-07 22:47:04,273][67838] Updated weights for policy 0, policy_version 73532 (0.0007) [2023-10-07 22:47:06,834][67871] Updated weights for policy 1, policy_version 73640 (0.0008) [2023-10-07 22:47:07,199][67871] Updated weights for policy 1, policy_version 73650 (0.0008) [2023-10-07 22:47:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 150700032. Throughput: 0: 1665.2, 1: 1657.7. Samples: 37692252. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-07 22:47:07,477][66916] Avg episode reward: [(0, '50.500'), (1, '56.090')] [2023-10-07 22:47:07,488][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000073536_75300864.pth... [2023-10-07 22:47:07,518][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000072000_73728000.pth [2023-10-07 22:47:07,572][67871] Updated weights for policy 1, policy_version 73660 (0.0007) [2023-10-07 22:47:07,713][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000073664_75431936.pth... [2023-10-07 22:47:07,751][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000072096_73826304.pth [2023-10-07 22:47:08,463][67838] Updated weights for policy 0, policy_version 73542 (0.0008) [2023-10-07 22:47:08,834][67838] Updated weights for policy 0, policy_version 73552 (0.0010) [2023-10-07 22:47:09,210][67838] Updated weights for policy 0, policy_version 73562 (0.0010) [2023-10-07 22:47:11,683][67871] Updated weights for policy 1, policy_version 73670 (0.0008) [2023-10-07 22:47:12,068][67871] Updated weights for policy 1, policy_version 73680 (0.0008) [2023-10-07 22:47:12,436][67871] Updated weights for policy 1, policy_version 73690 (0.0008) [2023-10-07 22:47:12,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 150765568. Throughput: 0: 1664.1, 1: 1666.1. Samples: 37701522. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-07 22:47:12,477][66916] Avg episode reward: [(0, '52.400'), (1, '57.590')] [2023-10-07 22:47:13,270][67838] Updated weights for policy 0, policy_version 73572 (0.0009) [2023-10-07 22:47:13,634][67838] Updated weights for policy 0, policy_version 73582 (0.0007) [2023-10-07 22:47:14,020][67838] Updated weights for policy 0, policy_version 73592 (0.0010) [2023-10-07 22:47:16,437][67871] Updated weights for policy 1, policy_version 73700 (0.0008) [2023-10-07 22:47:16,792][67871] Updated weights for policy 1, policy_version 73710 (0.0007) [2023-10-07 22:47:17,159][67871] Updated weights for policy 1, policy_version 73720 (0.0007) [2023-10-07 22:47:17,477][66916] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 150863872. Throughput: 0: 1656.1, 1: 1661.4. Samples: 37721692. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-07 22:47:17,478][66916] Avg episode reward: [(0, '49.050'), (1, '53.250')] [2023-10-07 22:47:17,976][67838] Updated weights for policy 0, policy_version 73602 (0.0010) [2023-10-07 22:47:18,351][67838] Updated weights for policy 0, policy_version 73612 (0.0009) [2023-10-07 22:47:18,723][67838] Updated weights for policy 0, policy_version 73622 (0.0007) [2023-10-07 22:47:19,102][67838] Updated weights for policy 0, policy_version 73632 (0.0011) [2023-10-07 22:47:21,386][67871] Updated weights for policy 1, policy_version 73730 (0.0007) [2023-10-07 22:47:21,757][67871] Updated weights for policy 1, policy_version 73740 (0.0011) [2023-10-07 22:47:22,123][67871] Updated weights for policy 1, policy_version 73750 (0.0008) [2023-10-07 22:47:22,476][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 150896640. Throughput: 0: 1659.1, 1: 1648.6. Samples: 37741630. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-07 22:47:22,477][66916] Avg episode reward: [(0, '49.840'), (1, '54.950')] [2023-10-07 22:47:22,488][67871] Updated weights for policy 1, policy_version 73760 (0.0011) [2023-10-07 22:47:23,077][67838] Updated weights for policy 0, policy_version 73642 (0.0011) [2023-10-07 22:47:23,447][67838] Updated weights for policy 0, policy_version 73652 (0.0009) [2023-10-07 22:47:23,827][67838] Updated weights for policy 0, policy_version 73662 (0.0009) [2023-10-07 22:47:26,649][67871] Updated weights for policy 1, policy_version 73770 (0.0007) [2023-10-07 22:47:27,011][67871] Updated weights for policy 1, policy_version 73780 (0.0007) [2023-10-07 22:47:27,382][67871] Updated weights for policy 1, policy_version 73790 (0.0007) [2023-10-07 22:47:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 150994944. Throughput: 0: 1661.2, 1: 1659.0. Samples: 37751160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-07 22:47:27,477][66916] Avg episode reward: [(0, '50.860'), (1, '53.510')] [2023-10-07 22:47:28,069][67838] Updated weights for policy 0, policy_version 73672 (0.0010) [2023-10-07 22:47:28,448][67838] Updated weights for policy 0, policy_version 73682 (0.0010) [2023-10-07 22:47:28,818][67838] Updated weights for policy 0, policy_version 73692 (0.0009) [2023-10-07 22:47:31,373][67871] Updated weights for policy 1, policy_version 73800 (0.0008) [2023-10-07 22:47:31,732][67871] Updated weights for policy 1, policy_version 73810 (0.0011) [2023-10-07 22:47:32,112][67871] Updated weights for policy 1, policy_version 73820 (0.0009) [2023-10-07 22:47:32,477][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 151060480. Throughput: 0: 1665.1, 1: 1660.4. Samples: 37771716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-07 22:47:32,478][66916] Avg episode reward: [(0, '46.700'), (1, '50.180')] [2023-10-07 22:47:32,862][67838] Updated weights for policy 0, policy_version 73702 (0.0007) [2023-10-07 22:47:33,240][67838] Updated weights for policy 0, policy_version 73712 (0.0007) [2023-10-07 22:47:33,603][67838] Updated weights for policy 0, policy_version 73722 (0.0008) [2023-10-07 22:47:36,177][67871] Updated weights for policy 1, policy_version 73830 (0.0010) [2023-10-07 22:47:36,551][67871] Updated weights for policy 1, policy_version 73840 (0.0008) [2023-10-07 22:47:36,910][67871] Updated weights for policy 1, policy_version 73850 (0.0009) [2023-10-07 22:47:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 151126016. Throughput: 0: 1669.1, 1: 1648.6. Samples: 37791582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-07 22:47:37,477][66916] Avg episode reward: [(0, '49.920'), (1, '49.360')] [2023-10-07 22:47:37,672][67838] Updated weights for policy 0, policy_version 73732 (0.0010) [2023-10-07 22:47:38,067][67838] Updated weights for policy 0, policy_version 73742 (0.0008) [2023-10-07 22:47:38,433][67838] Updated weights for policy 0, policy_version 73752 (0.0008) [2023-10-07 22:47:41,333][67871] Updated weights for policy 1, policy_version 73860 (0.0010) [2023-10-07 22:47:41,695][67871] Updated weights for policy 1, policy_version 73870 (0.0011) [2023-10-07 22:47:42,067][67871] Updated weights for policy 1, policy_version 73880 (0.0010) [2023-10-07 22:47:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 151191552. Throughput: 0: 1668.2, 1: 1664.0. Samples: 37801266. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-07 22:47:42,477][66916] Avg episode reward: [(0, '49.130'), (1, '50.240')] [2023-10-07 22:47:42,542][67838] Updated weights for policy 0, policy_version 73762 (0.0007) [2023-10-07 22:47:42,917][67838] Updated weights for policy 0, policy_version 73772 (0.0007) [2023-10-07 22:47:43,288][67838] Updated weights for policy 0, policy_version 73782 (0.0007) [2023-10-07 22:47:43,650][67838] Updated weights for policy 0, policy_version 73792 (0.0008) [2023-10-07 22:47:46,271][67871] Updated weights for policy 1, policy_version 73890 (0.0010) [2023-10-07 22:47:46,637][67871] Updated weights for policy 1, policy_version 73900 (0.0007) [2023-10-07 22:47:47,003][67871] Updated weights for policy 1, policy_version 73910 (0.0008) [2023-10-07 22:47:47,387][67871] Updated weights for policy 1, policy_version 73920 (0.0009) [2023-10-07 22:47:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 151257088. Throughput: 0: 1673.7, 1: 1656.8. Samples: 37821594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-07 22:47:47,477][66916] Avg episode reward: [(0, '51.950'), (1, '49.000')] [2023-10-07 22:47:47,613][67838] Updated weights for policy 0, policy_version 73802 (0.0007) [2023-10-07 22:47:47,990][67838] Updated weights for policy 0, policy_version 73812 (0.0008) [2023-10-07 22:47:48,353][67838] Updated weights for policy 0, policy_version 73822 (0.0010) [2023-10-07 22:47:51,469][67871] Updated weights for policy 1, policy_version 73930 (0.0008) [2023-10-07 22:47:51,841][67871] Updated weights for policy 1, policy_version 73940 (0.0009) [2023-10-07 22:47:52,202][67871] Updated weights for policy 1, policy_version 73950 (0.0009) [2023-10-07 22:47:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 151322624. Throughput: 0: 1670.9, 1: 1646.6. Samples: 37841540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-07 22:47:52,478][66916] Avg episode reward: [(0, '50.560'), (1, '49.950')] [2023-10-07 22:47:52,556][67838] Updated weights for policy 0, policy_version 73832 (0.0008) [2023-10-07 22:47:52,933][67838] Updated weights for policy 0, policy_version 73842 (0.0009) [2023-10-07 22:47:53,308][67838] Updated weights for policy 0, policy_version 73852 (0.0008) [2023-10-07 22:47:56,398][67871] Updated weights for policy 1, policy_version 73960 (0.0008) [2023-10-07 22:47:56,753][67871] Updated weights for policy 1, policy_version 73970 (0.0009) [2023-10-07 22:47:57,120][67871] Updated weights for policy 1, policy_version 73980 (0.0009) [2023-10-07 22:47:57,377][67838] Updated weights for policy 0, policy_version 73862 (0.0007) [2023-10-07 22:47:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 151388160. Throughput: 0: 1672.4, 1: 1656.6. Samples: 37851324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-07 22:47:57,477][66916] Avg episode reward: [(0, '52.830'), (1, '49.380')] [2023-10-07 22:47:57,751][67838] Updated weights for policy 0, policy_version 73872 (0.0011) [2023-10-07 22:47:58,123][67838] Updated weights for policy 0, policy_version 73882 (0.0007) [2023-10-07 22:48:01,204][67871] Updated weights for policy 1, policy_version 73990 (0.0009) [2023-10-07 22:48:01,598][67871] Updated weights for policy 1, policy_version 74000 (0.0011) [2023-10-07 22:48:01,968][67871] Updated weights for policy 1, policy_version 74010 (0.0007) [2023-10-07 22:48:02,021][67838] Updated weights for policy 0, policy_version 73892 (0.0008) [2023-10-07 22:48:02,380][67838] Updated weights for policy 0, policy_version 73902 (0.0009) [2023-10-07 22:48:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 151453696. Throughput: 0: 1676.9, 1: 1663.9. Samples: 37872030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-07 22:48:02,477][66916] Avg episode reward: [(0, '50.780'), (1, '50.160')] [2023-10-07 22:48:02,760][67838] Updated weights for policy 0, policy_version 73912 (0.0008) [2023-10-07 22:48:05,977][67871] Updated weights for policy 1, policy_version 74020 (0.0008) [2023-10-07 22:48:06,344][67871] Updated weights for policy 1, policy_version 74030 (0.0010) [2023-10-07 22:48:06,686][67838] Updated weights for policy 0, policy_version 73922 (0.0010) [2023-10-07 22:48:06,703][67871] Updated weights for policy 1, policy_version 74040 (0.0009) [2023-10-07 22:48:07,066][67838] Updated weights for policy 0, policy_version 73932 (0.0009) [2023-10-07 22:48:07,431][67838] Updated weights for policy 0, policy_version 73942 (0.0010) [2023-10-07 22:48:07,477][66916] Fps is (10 sec: 13106.5, 60 sec: 13653.2, 300 sec: 13329.3). Total num frames: 151519232. Throughput: 0: 1675.4, 1: 1653.5. Samples: 37891432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-07 22:48:07,478][66916] Avg episode reward: [(0, '49.090'), (1, '48.170')] [2023-10-07 22:48:07,801][67838] Updated weights for policy 0, policy_version 73952 (0.0010) [2023-10-07 22:48:10,860][67871] Updated weights for policy 1, policy_version 74050 (0.0009) [2023-10-07 22:48:11,230][67871] Updated weights for policy 1, policy_version 74060 (0.0009) [2023-10-07 22:48:11,599][67871] Updated weights for policy 1, policy_version 74070 (0.0009) [2023-10-07 22:48:11,797][67838] Updated weights for policy 0, policy_version 73962 (0.0008) [2023-10-07 22:48:11,958][67871] Updated weights for policy 1, policy_version 74080 (0.0009) [2023-10-07 22:48:12,183][67838] Updated weights for policy 0, policy_version 73972 (0.0009) [2023-10-07 22:48:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 151584768. Throughput: 0: 1684.6, 1: 1665.2. Samples: 37901900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-07 22:48:12,477][66916] Avg episode reward: [(0, '49.180'), (1, '50.560')] [2023-10-07 22:48:12,560][67838] Updated weights for policy 0, policy_version 73982 (0.0011) [2023-10-07 22:48:15,900][67871] Updated weights for policy 1, policy_version 74090 (0.0008) [2023-10-07 22:48:16,268][67871] Updated weights for policy 1, policy_version 74100 (0.0010) [2023-10-07 22:48:16,633][67871] Updated weights for policy 1, policy_version 74110 (0.0008) [2023-10-07 22:48:16,720][67838] Updated weights for policy 0, policy_version 73992 (0.0008) [2023-10-07 22:48:17,079][67838] Updated weights for policy 0, policy_version 74002 (0.0008) [2023-10-07 22:48:17,447][67838] Updated weights for policy 0, policy_version 74012 (0.0008) [2023-10-07 22:48:17,477][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 151650304. Throughput: 0: 1677.0, 1: 1662.1. Samples: 37921978. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:48:17,478][66916] Avg episode reward: [(0, '48.080'), (1, '51.290')] [2023-10-07 22:48:20,838][67871] Updated weights for policy 1, policy_version 74120 (0.0009) [2023-10-07 22:48:21,198][67871] Updated weights for policy 1, policy_version 74130 (0.0007) [2023-10-07 22:48:21,562][67871] Updated weights for policy 1, policy_version 74140 (0.0008) [2023-10-07 22:48:21,623][67838] Updated weights for policy 0, policy_version 74022 (0.0008) [2023-10-07 22:48:22,003][67838] Updated weights for policy 0, policy_version 74032 (0.0010) [2023-10-07 22:48:22,381][67838] Updated weights for policy 0, policy_version 74042 (0.0011) [2023-10-07 22:48:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 151715840. Throughput: 0: 1664.0, 1: 1654.1. Samples: 37940892. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:48:22,477][66916] Avg episode reward: [(0, '50.740'), (1, '51.960')] [2023-10-07 22:48:25,552][67871] Updated weights for policy 1, policy_version 74150 (0.0009) [2023-10-07 22:48:25,918][67871] Updated weights for policy 1, policy_version 74160 (0.0008) [2023-10-07 22:48:26,282][67871] Updated weights for policy 1, policy_version 74170 (0.0007) [2023-10-07 22:48:26,422][67838] Updated weights for policy 0, policy_version 74052 (0.0010) [2023-10-07 22:48:26,809][67838] Updated weights for policy 0, policy_version 74062 (0.0007) [2023-10-07 22:48:27,182][67838] Updated weights for policy 0, policy_version 74072 (0.0009) [2023-10-07 22:48:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 151781376. Throughput: 0: 1684.0, 1: 1664.1. Samples: 37951928. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:48:27,477][66916] Avg episode reward: [(0, '51.780'), (1, '53.120')] [2023-10-07 22:48:30,458][67871] Updated weights for policy 1, policy_version 74180 (0.0008) [2023-10-07 22:48:30,830][67871] Updated weights for policy 1, policy_version 74190 (0.0010) [2023-10-07 22:48:31,137][67838] Updated weights for policy 0, policy_version 74082 (0.0008) [2023-10-07 22:48:31,191][67871] Updated weights for policy 1, policy_version 74200 (0.0009) [2023-10-07 22:48:31,515][67838] Updated weights for policy 0, policy_version 74092 (0.0007) [2023-10-07 22:48:31,879][67838] Updated weights for policy 0, policy_version 74102 (0.0010) [2023-10-07 22:48:32,248][67838] Updated weights for policy 0, policy_version 74112 (0.0008) [2023-10-07 22:48:32,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 151879680. Throughput: 0: 1681.8, 1: 1660.8. Samples: 37972010. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:48:32,478][66916] Avg episode reward: [(0, '52.130'), (1, '56.340')] [2023-10-07 22:48:35,166][67871] Updated weights for policy 1, policy_version 74210 (0.0008) [2023-10-07 22:48:35,527][67871] Updated weights for policy 1, policy_version 74220 (0.0008) [2023-10-07 22:48:35,896][67871] Updated weights for policy 1, policy_version 74230 (0.0007) [2023-10-07 22:48:36,254][67871] Updated weights for policy 1, policy_version 74240 (0.0009) [2023-10-07 22:48:36,429][67838] Updated weights for policy 0, policy_version 74122 (0.0008) [2023-10-07 22:48:36,806][67838] Updated weights for policy 0, policy_version 74132 (0.0009) [2023-10-07 22:48:37,172][67838] Updated weights for policy 0, policy_version 74142 (0.0008) [2023-10-07 22:48:37,476][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 151945216. Throughput: 0: 1657.2, 1: 1665.3. Samples: 37991052. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:48:37,477][66916] Avg episode reward: [(0, '49.230'), (1, '53.560')] [2023-10-07 22:48:40,346][67871] Updated weights for policy 1, policy_version 74250 (0.0012) [2023-10-07 22:48:40,710][67871] Updated weights for policy 1, policy_version 74260 (0.0012) [2023-10-07 22:48:41,080][67871] Updated weights for policy 1, policy_version 74270 (0.0009) [2023-10-07 22:48:41,346][67838] Updated weights for policy 0, policy_version 74152 (0.0010) [2023-10-07 22:48:41,730][67838] Updated weights for policy 0, policy_version 74162 (0.0009) [2023-10-07 22:48:42,095][67838] Updated weights for policy 0, policy_version 74172 (0.0011) [2023-10-07 22:48:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 152010752. Throughput: 0: 1678.7, 1: 1673.2. Samples: 38002156. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:48:42,477][66916] Avg episode reward: [(0, '48.810'), (1, '56.780')] [2023-10-07 22:48:45,207][67871] Updated weights for policy 1, policy_version 74280 (0.0007) [2023-10-07 22:48:45,577][67871] Updated weights for policy 1, policy_version 74290 (0.0008) [2023-10-07 22:48:45,943][67871] Updated weights for policy 1, policy_version 74300 (0.0010) [2023-10-07 22:48:46,088][67838] Updated weights for policy 0, policy_version 74182 (0.0008) [2023-10-07 22:48:46,465][67838] Updated weights for policy 0, policy_version 74192 (0.0007) [2023-10-07 22:48:46,833][67838] Updated weights for policy 0, policy_version 74202 (0.0008) [2023-10-07 22:48:47,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 152076288. Throughput: 0: 1677.0, 1: 1651.6. Samples: 38021816. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:48:47,478][66916] Avg episode reward: [(0, '46.280'), (1, '54.840')] [2023-10-07 22:48:50,103][67871] Updated weights for policy 1, policy_version 74310 (0.0008) [2023-10-07 22:48:50,463][67871] Updated weights for policy 1, policy_version 74320 (0.0007) [2023-10-07 22:48:50,839][67871] Updated weights for policy 1, policy_version 74330 (0.0010) [2023-10-07 22:48:51,104][67838] Updated weights for policy 0, policy_version 74212 (0.0010) [2023-10-07 22:48:51,480][67838] Updated weights for policy 0, policy_version 74222 (0.0008) [2023-10-07 22:48:51,845][67838] Updated weights for policy 0, policy_version 74232 (0.0008) [2023-10-07 22:48:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 152141824. Throughput: 0: 1657.8, 1: 1670.5. Samples: 38041204. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:48:52,477][66916] Avg episode reward: [(0, '48.820'), (1, '52.520')] [2023-10-07 22:48:54,879][67871] Updated weights for policy 1, policy_version 74340 (0.0008) [2023-10-07 22:48:55,237][67871] Updated weights for policy 1, policy_version 74350 (0.0009) [2023-10-07 22:48:55,597][67871] Updated weights for policy 1, policy_version 74360 (0.0010) [2023-10-07 22:48:55,986][67838] Updated weights for policy 0, policy_version 74242 (0.0010) [2023-10-07 22:48:56,364][67838] Updated weights for policy 0, policy_version 74252 (0.0009) [2023-10-07 22:48:56,736][67838] Updated weights for policy 0, policy_version 74262 (0.0009) [2023-10-07 22:48:57,109][67838] Updated weights for policy 0, policy_version 74272 (0.0008) [2023-10-07 22:48:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 152207360. Throughput: 0: 1668.9, 1: 1673.8. Samples: 38052322. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-07 22:48:57,477][66916] Avg episode reward: [(0, '52.770'), (1, '51.390')] [2023-10-07 22:48:59,742][67871] Updated weights for policy 1, policy_version 74370 (0.0008) [2023-10-07 22:49:00,113][67871] Updated weights for policy 1, policy_version 74380 (0.0009) [2023-10-07 22:49:00,473][67871] Updated weights for policy 1, policy_version 74390 (0.0009) [2023-10-07 22:49:00,836][67871] Updated weights for policy 1, policy_version 74400 (0.0010) [2023-10-07 22:49:01,230][67838] Updated weights for policy 0, policy_version 74282 (0.0010) [2023-10-07 22:49:01,602][67838] Updated weights for policy 0, policy_version 74292 (0.0009) [2023-10-07 22:49:01,971][67838] Updated weights for policy 0, policy_version 74302 (0.0009) [2023-10-07 22:49:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 152272896. Throughput: 0: 1669.6, 1: 1658.5. Samples: 38071742. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:49:02,477][66916] Avg episode reward: [(0, '52.340'), (1, '51.130')] [2023-10-07 22:49:04,992][67871] Updated weights for policy 1, policy_version 74410 (0.0007) [2023-10-07 22:49:05,368][67871] Updated weights for policy 1, policy_version 74420 (0.0009) [2023-10-07 22:49:05,731][67871] Updated weights for policy 1, policy_version 74430 (0.0008) [2023-10-07 22:49:06,168][67838] Updated weights for policy 0, policy_version 74312 (0.0009) [2023-10-07 22:49:06,538][67838] Updated weights for policy 0, policy_version 74322 (0.0008) [2023-10-07 22:49:06,902][67838] Updated weights for policy 0, policy_version 74332 (0.0009) [2023-10-07 22:49:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13653.4, 300 sec: 13329.3). Total num frames: 152338432. Throughput: 0: 1657.6, 1: 1682.1. Samples: 38091180. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:49:07,478][66916] Avg episode reward: [(0, '51.250'), (1, '48.710')] [2023-10-07 22:49:07,488][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000074432_76218368.pth... [2023-10-07 22:49:07,488][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000074336_76120064.pth... [2023-10-07 22:49:07,518][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000072864_74612736.pth [2023-10-07 22:49:07,519][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000072768_74514432.pth [2023-10-07 22:49:09,868][67871] Updated weights for policy 1, policy_version 74440 (0.0007) [2023-10-07 22:49:10,235][67871] Updated weights for policy 1, policy_version 74450 (0.0010) [2023-10-07 22:49:10,608][67871] Updated weights for policy 1, policy_version 74460 (0.0007) [2023-10-07 22:49:10,855][67838] Updated weights for policy 0, policy_version 74342 (0.0007) [2023-10-07 22:49:11,219][67838] Updated weights for policy 0, policy_version 74352 (0.0007) [2023-10-07 22:49:11,596][67838] Updated weights for policy 0, policy_version 74362 (0.0007) [2023-10-07 22:49:12,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 152403968. Throughput: 0: 1672.9, 1: 1671.9. Samples: 38102444. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:49:12,478][66916] Avg episode reward: [(0, '52.470'), (1, '47.310')] [2023-10-07 22:49:14,627][67871] Updated weights for policy 1, policy_version 74470 (0.0009) [2023-10-07 22:49:15,003][67871] Updated weights for policy 1, policy_version 74480 (0.0008) [2023-10-07 22:49:15,364][67871] Updated weights for policy 1, policy_version 74490 (0.0008) [2023-10-07 22:49:15,818][67838] Updated weights for policy 0, policy_version 74372 (0.0008) [2023-10-07 22:49:16,212][67838] Updated weights for policy 0, policy_version 74382 (0.0008) [2023-10-07 22:49:16,580][67838] Updated weights for policy 0, policy_version 74392 (0.0007) [2023-10-07 22:49:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 152469504. Throughput: 0: 1661.6, 1: 1661.0. Samples: 38121524. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:49:17,478][66916] Avg episode reward: [(0, '54.300'), (1, '49.800')] [2023-10-07 22:49:19,564][67871] Updated weights for policy 1, policy_version 74500 (0.0007) [2023-10-07 22:49:19,929][67871] Updated weights for policy 1, policy_version 74510 (0.0008) [2023-10-07 22:49:20,291][67871] Updated weights for policy 1, policy_version 74520 (0.0009) [2023-10-07 22:49:20,375][67838] Updated weights for policy 0, policy_version 74402 (0.0009) [2023-10-07 22:49:20,737][67838] Updated weights for policy 0, policy_version 74412 (0.0009) [2023-10-07 22:49:21,112][67838] Updated weights for policy 0, policy_version 74422 (0.0009) [2023-10-07 22:49:21,494][67838] Updated weights for policy 0, policy_version 74432 (0.0011) [2023-10-07 22:49:22,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 152535040. Throughput: 0: 1663.3, 1: 1676.8. Samples: 38141358. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:49:22,478][66916] Avg episode reward: [(0, '50.980'), (1, '50.870')] [2023-10-07 22:49:24,315][67871] Updated weights for policy 1, policy_version 74530 (0.0008) [2023-10-07 22:49:24,688][67871] Updated weights for policy 1, policy_version 74540 (0.0007) [2023-10-07 22:49:25,052][67871] Updated weights for policy 1, policy_version 74550 (0.0007) [2023-10-07 22:49:25,428][67871] Updated weights for policy 1, policy_version 74560 (0.0009) [2023-10-07 22:49:25,621][67838] Updated weights for policy 0, policy_version 74442 (0.0008) [2023-10-07 22:49:26,003][67838] Updated weights for policy 0, policy_version 74452 (0.0008) [2023-10-07 22:49:26,371][67838] Updated weights for policy 0, policy_version 74462 (0.0010) [2023-10-07 22:49:27,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 152600576. Throughput: 0: 1669.6, 1: 1666.6. Samples: 38152286. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:49:27,477][66916] Avg episode reward: [(0, '51.830'), (1, '54.800')] [2023-10-07 22:49:29,407][67871] Updated weights for policy 1, policy_version 74570 (0.0011) [2023-10-07 22:49:29,771][67871] Updated weights for policy 1, policy_version 74580 (0.0009) [2023-10-07 22:49:30,152][67871] Updated weights for policy 1, policy_version 74590 (0.0009) [2023-10-07 22:49:30,587][67838] Updated weights for policy 0, policy_version 74472 (0.0008) [2023-10-07 22:49:30,962][67838] Updated weights for policy 0, policy_version 74482 (0.0008) [2023-10-07 22:49:31,322][67838] Updated weights for policy 0, policy_version 74492 (0.0010) [2023-10-07 22:49:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152666112. Throughput: 0: 1653.2, 1: 1671.6. Samples: 38171430. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:49:32,478][66916] Avg episode reward: [(0, '48.020'), (1, '54.610')] [2023-10-07 22:49:34,282][67871] Updated weights for policy 1, policy_version 74600 (0.0008) [2023-10-07 22:49:34,650][67871] Updated weights for policy 1, policy_version 74610 (0.0009) [2023-10-07 22:49:35,022][67871] Updated weights for policy 1, policy_version 74620 (0.0007) [2023-10-07 22:49:35,464][67838] Updated weights for policy 0, policy_version 74502 (0.0008) [2023-10-07 22:49:35,828][67838] Updated weights for policy 0, policy_version 74512 (0.0007) [2023-10-07 22:49:36,216][67838] Updated weights for policy 0, policy_version 74522 (0.0011) [2023-10-07 22:49:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152731648. Throughput: 0: 1663.6, 1: 1676.8. Samples: 38191522. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:49:37,477][66916] Avg episode reward: [(0, '46.020'), (1, '58.870')] [2023-10-07 22:49:39,014][67871] Updated weights for policy 1, policy_version 74630 (0.0010) [2023-10-07 22:49:39,379][67871] Updated weights for policy 1, policy_version 74640 (0.0008) [2023-10-07 22:49:39,757][67871] Updated weights for policy 1, policy_version 74650 (0.0008) [2023-10-07 22:49:40,260][67838] Updated weights for policy 0, policy_version 74532 (0.0007) [2023-10-07 22:49:40,633][67838] Updated weights for policy 0, policy_version 74542 (0.0009) [2023-10-07 22:49:41,002][67838] Updated weights for policy 0, policy_version 74552 (0.0011) [2023-10-07 22:49:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152797184. Throughput: 0: 1670.9, 1: 1659.1. Samples: 38202172. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:49:42,477][66916] Avg episode reward: [(0, '45.900'), (1, '58.090')] [2023-10-07 22:49:43,857][67871] Updated weights for policy 1, policy_version 74660 (0.0010) [2023-10-07 22:49:44,211][67871] Updated weights for policy 1, policy_version 74670 (0.0008) [2023-10-07 22:49:44,587][67871] Updated weights for policy 1, policy_version 74680 (0.0008) [2023-10-07 22:49:45,116][67838] Updated weights for policy 0, policy_version 74562 (0.0009) [2023-10-07 22:49:45,480][67838] Updated weights for policy 0, policy_version 74572 (0.0007) [2023-10-07 22:49:45,855][67838] Updated weights for policy 0, policy_version 74582 (0.0009) [2023-10-07 22:49:46,229][67838] Updated weights for policy 0, policy_version 74592 (0.0010) [2023-10-07 22:49:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152862720. Throughput: 0: 1655.0, 1: 1671.8. Samples: 38221448. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:49:47,477][66916] Avg episode reward: [(0, '48.090'), (1, '59.940')] [2023-10-07 22:49:48,748][67871] Updated weights for policy 1, policy_version 74690 (0.0010) [2023-10-07 22:49:49,111][67871] Updated weights for policy 1, policy_version 74700 (0.0007) [2023-10-07 22:49:49,473][67871] Updated weights for policy 1, policy_version 74710 (0.0007) [2023-10-07 22:49:49,835][67871] Updated weights for policy 1, policy_version 74720 (0.0007) [2023-10-07 22:49:50,429][67838] Updated weights for policy 0, policy_version 74602 (0.0007) [2023-10-07 22:49:50,789][67838] Updated weights for policy 0, policy_version 74612 (0.0009) [2023-10-07 22:49:51,162][67838] Updated weights for policy 0, policy_version 74622 (0.0010) [2023-10-07 22:49:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152928256. Throughput: 0: 1671.8, 1: 1670.2. Samples: 38241572. Policy #0 lag: (min: 10.0, avg: 11.5, max: 33.0) [2023-10-07 22:49:52,477][66916] Avg episode reward: [(0, '49.760'), (1, '59.750')] [2023-10-07 22:49:54,012][67871] Updated weights for policy 1, policy_version 74730 (0.0009) [2023-10-07 22:49:54,382][67871] Updated weights for policy 1, policy_version 74740 (0.0008) [2023-10-07 22:49:54,745][67871] Updated weights for policy 1, policy_version 74750 (0.0009) [2023-10-07 22:49:55,221][67838] Updated weights for policy 0, policy_version 74632 (0.0008) [2023-10-07 22:49:55,599][67838] Updated weights for policy 0, policy_version 74642 (0.0009) [2023-10-07 22:49:55,964][67838] Updated weights for policy 0, policy_version 74652 (0.0008) [2023-10-07 22:49:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152993792. Throughput: 0: 1666.0, 1: 1653.8. Samples: 38251834. Policy #0 lag: (min: 10.0, avg: 11.5, max: 33.0) [2023-10-07 22:49:57,477][66916] Avg episode reward: [(0, '48.780'), (1, '59.390')] [2023-10-07 22:49:58,879][67871] Updated weights for policy 1, policy_version 74760 (0.0009) [2023-10-07 22:49:59,252][67871] Updated weights for policy 1, policy_version 74770 (0.0008) [2023-10-07 22:49:59,618][67871] Updated weights for policy 1, policy_version 74780 (0.0009) [2023-10-07 22:50:00,122][67838] Updated weights for policy 0, policy_version 74662 (0.0009) [2023-10-07 22:50:00,496][67838] Updated weights for policy 0, policy_version 74672 (0.0009) [2023-10-07 22:50:00,871][67838] Updated weights for policy 0, policy_version 74682 (0.0009) [2023-10-07 22:50:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153059328. Throughput: 0: 1653.7, 1: 1669.6. Samples: 38271072. Policy #0 lag: (min: 10.0, avg: 11.5, max: 33.0) [2023-10-07 22:50:02,477][66916] Avg episode reward: [(0, '47.320'), (1, '59.150')] [2023-10-07 22:50:03,686][67871] Updated weights for policy 1, policy_version 74790 (0.0008) [2023-10-07 22:50:04,048][67871] Updated weights for policy 1, policy_version 74800 (0.0007) [2023-10-07 22:50:04,417][67871] Updated weights for policy 1, policy_version 74810 (0.0009) [2023-10-07 22:50:04,849][67838] Updated weights for policy 0, policy_version 74692 (0.0007) [2023-10-07 22:50:05,238][67838] Updated weights for policy 0, policy_version 74702 (0.0008) [2023-10-07 22:50:05,611][67838] Updated weights for policy 0, policy_version 74712 (0.0008) [2023-10-07 22:50:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 153124864. Throughput: 0: 1676.6, 1: 1668.3. Samples: 38291876. Policy #0 lag: (min: 10.0, avg: 11.5, max: 33.0) [2023-10-07 22:50:07,477][66916] Avg episode reward: [(0, '49.180'), (1, '56.380')] [2023-10-07 22:50:08,494][67871] Updated weights for policy 1, policy_version 74820 (0.0009) [2023-10-07 22:50:08,863][67871] Updated weights for policy 1, policy_version 74830 (0.0011) [2023-10-07 22:50:09,230][67871] Updated weights for policy 1, policy_version 74840 (0.0008) [2023-10-07 22:50:09,464][67838] Updated weights for policy 0, policy_version 74722 (0.0010) [2023-10-07 22:50:09,847][67838] Updated weights for policy 0, policy_version 74732 (0.0007) [2023-10-07 22:50:10,213][67838] Updated weights for policy 0, policy_version 74742 (0.0009) [2023-10-07 22:50:10,585][67838] Updated weights for policy 0, policy_version 74752 (0.0010) [2023-10-07 22:50:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153190400. Throughput: 0: 1665.7, 1: 1650.8. Samples: 38301532. Policy #0 lag: (min: 10.0, avg: 11.5, max: 33.0) [2023-10-07 22:50:12,477][66916] Avg episode reward: [(0, '47.570'), (1, '57.660')] [2023-10-07 22:50:13,482][67871] Updated weights for policy 1, policy_version 74850 (0.0007) [2023-10-07 22:50:13,851][67871] Updated weights for policy 1, policy_version 74860 (0.0009) [2023-10-07 22:50:14,211][67871] Updated weights for policy 1, policy_version 74870 (0.0008) [2023-10-07 22:50:14,586][67871] Updated weights for policy 1, policy_version 74880 (0.0008) [2023-10-07 22:50:14,847][67838] Updated weights for policy 0, policy_version 74762 (0.0009) [2023-10-07 22:50:15,215][67838] Updated weights for policy 0, policy_version 74772 (0.0008) [2023-10-07 22:50:15,580][67838] Updated weights for policy 0, policy_version 74782 (0.0008) [2023-10-07 22:50:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153255936. Throughput: 0: 1669.2, 1: 1662.0. Samples: 38321334. Policy #0 lag: (min: 10.0, avg: 11.5, max: 33.0) [2023-10-07 22:50:17,477][66916] Avg episode reward: [(0, '51.250'), (1, '58.610')] [2023-10-07 22:50:18,849][67871] Updated weights for policy 1, policy_version 74890 (0.0008) [2023-10-07 22:50:19,213][67871] Updated weights for policy 1, policy_version 74900 (0.0009) [2023-10-07 22:50:19,576][67871] Updated weights for policy 1, policy_version 74910 (0.0007) [2023-10-07 22:50:19,727][67838] Updated weights for policy 0, policy_version 74792 (0.0007) [2023-10-07 22:50:20,102][67838] Updated weights for policy 0, policy_version 74802 (0.0007) [2023-10-07 22:50:20,482][67838] Updated weights for policy 0, policy_version 74812 (0.0007) [2023-10-07 22:50:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153321472. Throughput: 0: 1674.8, 1: 1658.5. Samples: 38341520. Policy #0 lag: (min: 10.0, avg: 11.5, max: 33.0) [2023-10-07 22:50:22,477][66916] Avg episode reward: [(0, '51.590'), (1, '60.250')] [2023-10-07 22:50:23,600][67871] Updated weights for policy 1, policy_version 74920 (0.0010) [2023-10-07 22:50:23,957][67871] Updated weights for policy 1, policy_version 74930 (0.0009) [2023-10-07 22:50:24,327][67871] Updated weights for policy 1, policy_version 74940 (0.0008) [2023-10-07 22:50:24,522][67838] Updated weights for policy 0, policy_version 74822 (0.0009) [2023-10-07 22:50:24,893][67838] Updated weights for policy 0, policy_version 74832 (0.0008) [2023-10-07 22:50:25,259][67838] Updated weights for policy 0, policy_version 74842 (0.0008) [2023-10-07 22:50:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153387008. Throughput: 0: 1661.0, 1: 1650.3. Samples: 38351180. Policy #0 lag: (min: 10.0, avg: 11.5, max: 33.0) [2023-10-07 22:50:27,477][66916] Avg episode reward: [(0, '49.830'), (1, '62.500')] [2023-10-07 22:50:28,491][67871] Updated weights for policy 1, policy_version 74950 (0.0009) [2023-10-07 22:50:28,850][67871] Updated weights for policy 1, policy_version 74960 (0.0008) [2023-10-07 22:50:29,224][67838] Updated weights for policy 0, policy_version 74852 (0.0008) [2023-10-07 22:50:29,225][67871] Updated weights for policy 1, policy_version 74970 (0.0008) [2023-10-07 22:50:29,597][67838] Updated weights for policy 0, policy_version 74862 (0.0007) [2023-10-07 22:50:29,968][67838] Updated weights for policy 0, policy_version 74872 (0.0007) [2023-10-07 22:50:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153452544. Throughput: 0: 1672.6, 1: 1657.1. Samples: 38371284. Policy #0 lag: (min: 10.0, avg: 11.5, max: 33.0) [2023-10-07 22:50:32,477][66916] Avg episode reward: [(0, '48.700'), (1, '63.220')] [2023-10-07 22:50:33,346][67871] Updated weights for policy 1, policy_version 74980 (0.0009) [2023-10-07 22:50:33,710][67871] Updated weights for policy 1, policy_version 74990 (0.0009) [2023-10-07 22:50:33,990][67838] Updated weights for policy 0, policy_version 74882 (0.0009) [2023-10-07 22:50:34,085][67871] Updated weights for policy 1, policy_version 75000 (0.0009) [2023-10-07 22:50:34,359][67838] Updated weights for policy 0, policy_version 74892 (0.0008) [2023-10-07 22:50:34,737][67838] Updated weights for policy 0, policy_version 74902 (0.0008) [2023-10-07 22:50:35,106][67838] Updated weights for policy 0, policy_version 74912 (0.0009) [2023-10-07 22:50:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153518080. Throughput: 0: 1676.3, 1: 1658.4. Samples: 38391636. Policy #0 lag: (min: 10.0, avg: 11.5, max: 33.0) [2023-10-07 22:50:37,477][66916] Avg episode reward: [(0, '51.540'), (1, '63.440')] [2023-10-07 22:50:38,142][67871] Updated weights for policy 1, policy_version 75010 (0.0009) [2023-10-07 22:50:38,504][67871] Updated weights for policy 1, policy_version 75020 (0.0007) [2023-10-07 22:50:38,879][67871] Updated weights for policy 1, policy_version 75030 (0.0008) [2023-10-07 22:50:39,241][67871] Updated weights for policy 1, policy_version 75040 (0.0009) [2023-10-07 22:50:39,310][67838] Updated weights for policy 0, policy_version 74922 (0.0010) [2023-10-07 22:50:39,687][67838] Updated weights for policy 0, policy_version 74932 (0.0007) [2023-10-07 22:50:40,056][67838] Updated weights for policy 0, policy_version 74942 (0.0008) [2023-10-07 22:50:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153583616. Throughput: 0: 1652.8, 1: 1656.0. Samples: 38400726. Policy #0 lag: (min: 17.0, avg: 20.5, max: 49.0) [2023-10-07 22:50:42,478][66916] Avg episode reward: [(0, '52.050'), (1, '63.470')] [2023-10-07 22:50:43,443][67871] Updated weights for policy 1, policy_version 75050 (0.0008) [2023-10-07 22:50:43,806][67871] Updated weights for policy 1, policy_version 75060 (0.0007) [2023-10-07 22:50:44,085][67838] Updated weights for policy 0, policy_version 74952 (0.0010) [2023-10-07 22:50:44,181][67871] Updated weights for policy 1, policy_version 75070 (0.0008) [2023-10-07 22:50:44,457][67838] Updated weights for policy 0, policy_version 74962 (0.0009) [2023-10-07 22:50:44,824][67838] Updated weights for policy 0, policy_version 74972 (0.0009) [2023-10-07 22:50:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153649152. Throughput: 0: 1673.2, 1: 1663.0. Samples: 38421202. Policy #0 lag: (min: 17.0, avg: 20.5, max: 49.0) [2023-10-07 22:50:47,477][66916] Avg episode reward: [(0, '53.100'), (1, '65.920')] [2023-10-07 22:50:47,478][67676] Saving new best policy, reward=65.920! [2023-10-07 22:50:48,244][67871] Updated weights for policy 1, policy_version 75080 (0.0009) [2023-10-07 22:50:48,608][67871] Updated weights for policy 1, policy_version 75090 (0.0009) [2023-10-07 22:50:48,979][67871] Updated weights for policy 1, policy_version 75100 (0.0010) [2023-10-07 22:50:49,021][67838] Updated weights for policy 0, policy_version 74982 (0.0009) [2023-10-07 22:50:49,399][67838] Updated weights for policy 0, policy_version 74992 (0.0008) [2023-10-07 22:50:49,760][67838] Updated weights for policy 0, policy_version 75002 (0.0010) [2023-10-07 22:50:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153714688. Throughput: 0: 1672.3, 1: 1658.9. Samples: 38441782. Policy #0 lag: (min: 17.0, avg: 20.5, max: 49.0) [2023-10-07 22:50:52,477][66916] Avg episode reward: [(0, '52.090'), (1, '63.860')] [2023-10-07 22:50:53,145][67871] Updated weights for policy 1, policy_version 75110 (0.0007) [2023-10-07 22:50:53,509][67871] Updated weights for policy 1, policy_version 75120 (0.0007) [2023-10-07 22:50:53,875][67871] Updated weights for policy 1, policy_version 75130 (0.0009) [2023-10-07 22:50:53,907][67838] Updated weights for policy 0, policy_version 75012 (0.0009) [2023-10-07 22:50:54,301][67838] Updated weights for policy 0, policy_version 75022 (0.0007) [2023-10-07 22:50:54,670][67838] Updated weights for policy 0, policy_version 75032 (0.0007) [2023-10-07 22:50:57,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153780224. Throughput: 0: 1654.1, 1: 1663.4. Samples: 38450820. Policy #0 lag: (min: 17.0, avg: 20.5, max: 49.0) [2023-10-07 22:50:57,478][66916] Avg episode reward: [(0, '50.140'), (1, '62.470')] [2023-10-07 22:50:57,890][67871] Updated weights for policy 1, policy_version 75140 (0.0007) [2023-10-07 22:50:58,259][67871] Updated weights for policy 1, policy_version 75150 (0.0007) [2023-10-07 22:50:58,622][67871] Updated weights for policy 1, policy_version 75160 (0.0007) [2023-10-07 22:50:58,647][67838] Updated weights for policy 0, policy_version 75042 (0.0008) [2023-10-07 22:50:59,023][67838] Updated weights for policy 0, policy_version 75052 (0.0010) [2023-10-07 22:50:59,398][67838] Updated weights for policy 0, policy_version 75062 (0.0010) [2023-10-07 22:50:59,760][67838] Updated weights for policy 0, policy_version 75072 (0.0010) [2023-10-07 22:51:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153845760. Throughput: 0: 1665.5, 1: 1665.0. Samples: 38471208. Policy #0 lag: (min: 17.0, avg: 20.5, max: 49.0) [2023-10-07 22:51:02,477][66916] Avg episode reward: [(0, '46.980'), (1, '65.790')] [2023-10-07 22:51:02,955][67871] Updated weights for policy 1, policy_version 75170 (0.0007) [2023-10-07 22:51:03,318][67871] Updated weights for policy 1, policy_version 75180 (0.0010) [2023-10-07 22:51:03,679][67871] Updated weights for policy 1, policy_version 75190 (0.0008) [2023-10-07 22:51:03,997][67838] Updated weights for policy 0, policy_version 75082 (0.0009) [2023-10-07 22:51:04,048][67871] Updated weights for policy 1, policy_version 75200 (0.0007) [2023-10-07 22:51:04,362][67838] Updated weights for policy 0, policy_version 75092 (0.0008) [2023-10-07 22:51:04,730][67838] Updated weights for policy 0, policy_version 75102 (0.0007) [2023-10-07 22:51:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 153911296. Throughput: 0: 1668.2, 1: 1669.0. Samples: 38491696. Policy #0 lag: (min: 17.0, avg: 20.5, max: 49.0) [2023-10-07 22:51:07,478][66916] Avg episode reward: [(0, '48.040'), (1, '62.820')] [2023-10-07 22:51:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000075200_77004800.pth... [2023-10-07 22:51:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000075104_76906496.pth... [2023-10-07 22:51:07,521][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000073664_75431936.pth [2023-10-07 22:51:07,526][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000073536_75300864.pth [2023-10-07 22:51:08,256][67871] Updated weights for policy 1, policy_version 75210 (0.0008) [2023-10-07 22:51:08,629][67871] Updated weights for policy 1, policy_version 75220 (0.0009) [2023-10-07 22:51:08,890][67838] Updated weights for policy 0, policy_version 75112 (0.0009) [2023-10-07 22:51:08,986][67871] Updated weights for policy 1, policy_version 75230 (0.0009) [2023-10-07 22:51:09,267][67838] Updated weights for policy 0, policy_version 75122 (0.0008) [2023-10-07 22:51:09,640][67838] Updated weights for policy 0, policy_version 75132 (0.0007) [2023-10-07 22:51:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153976832. Throughput: 0: 1652.7, 1: 1669.7. Samples: 38500686. Policy #0 lag: (min: 17.0, avg: 20.5, max: 49.0) [2023-10-07 22:51:12,477][66916] Avg episode reward: [(0, '46.550'), (1, '60.540')] [2023-10-07 22:51:13,074][67871] Updated weights for policy 1, policy_version 75240 (0.0009) [2023-10-07 22:51:13,442][67871] Updated weights for policy 1, policy_version 75250 (0.0010) [2023-10-07 22:51:13,472][67838] Updated weights for policy 0, policy_version 75142 (0.0007) [2023-10-07 22:51:13,814][67871] Updated weights for policy 1, policy_version 75260 (0.0008) [2023-10-07 22:51:13,843][67838] Updated weights for policy 0, policy_version 75152 (0.0008) [2023-10-07 22:51:14,218][67838] Updated weights for policy 0, policy_version 75162 (0.0010) [2023-10-07 22:51:17,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154042368. Throughput: 0: 1666.6, 1: 1671.0. Samples: 38521474. Policy #0 lag: (min: 17.0, avg: 20.5, max: 49.0) [2023-10-07 22:51:17,477][66916] Avg episode reward: [(0, '52.900'), (1, '65.370')] [2023-10-07 22:51:17,712][67871] Updated weights for policy 1, policy_version 75270 (0.0007) [2023-10-07 22:51:18,077][67871] Updated weights for policy 1, policy_version 75280 (0.0008) [2023-10-07 22:51:18,285][67838] Updated weights for policy 0, policy_version 75172 (0.0009) [2023-10-07 22:51:18,442][67871] Updated weights for policy 1, policy_version 75290 (0.0009) [2023-10-07 22:51:18,646][67838] Updated weights for policy 0, policy_version 75182 (0.0008) [2023-10-07 22:51:19,017][67838] Updated weights for policy 0, policy_version 75192 (0.0007) [2023-10-07 22:51:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154107904. Throughput: 0: 1668.4, 1: 1671.3. Samples: 38541920. Policy #0 lag: (min: 17.0, avg: 20.5, max: 49.0) [2023-10-07 22:51:22,477][66916] Avg episode reward: [(0, '54.070'), (1, '64.730')] [2023-10-07 22:51:22,503][67871] Updated weights for policy 1, policy_version 75300 (0.0007) [2023-10-07 22:51:22,871][67871] Updated weights for policy 1, policy_version 75310 (0.0008) [2023-10-07 22:51:23,192][67838] Updated weights for policy 0, policy_version 75202 (0.0008) [2023-10-07 22:51:23,237][67871] Updated weights for policy 1, policy_version 75320 (0.0007) [2023-10-07 22:51:23,572][67838] Updated weights for policy 0, policy_version 75212 (0.0008) [2023-10-07 22:51:23,943][67838] Updated weights for policy 0, policy_version 75222 (0.0008) [2023-10-07 22:51:24,313][67838] Updated weights for policy 0, policy_version 75232 (0.0009) [2023-10-07 22:51:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154173440. Throughput: 0: 1667.1, 1: 1672.4. Samples: 38551004. Policy #0 lag: (min: 17.0, avg: 20.5, max: 49.0) [2023-10-07 22:51:27,477][66916] Avg episode reward: [(0, '53.030'), (1, '60.610')] [2023-10-07 22:51:27,503][67871] Updated weights for policy 1, policy_version 75330 (0.0007) [2023-10-07 22:51:27,876][67871] Updated weights for policy 1, policy_version 75340 (0.0008) [2023-10-07 22:51:28,247][67871] Updated weights for policy 1, policy_version 75350 (0.0008) [2023-10-07 22:51:28,275][67838] Updated weights for policy 0, policy_version 75242 (0.0010) [2023-10-07 22:51:28,600][67871] Updated weights for policy 1, policy_version 75360 (0.0009) [2023-10-07 22:51:28,645][67838] Updated weights for policy 0, policy_version 75252 (0.0007) [2023-10-07 22:51:29,021][67838] Updated weights for policy 0, policy_version 75262 (0.0010) [2023-10-07 22:51:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154238976. Throughput: 0: 1671.8, 1: 1670.4. Samples: 38571600. Policy #0 lag: (min: 16.0, avg: 35.1, max: 48.0) [2023-10-07 22:51:32,477][66916] Avg episode reward: [(0, '54.600'), (1, '64.470')] [2023-10-07 22:51:32,688][67871] Updated weights for policy 1, policy_version 75370 (0.0009) [2023-10-07 22:51:33,047][67871] Updated weights for policy 1, policy_version 75380 (0.0008) [2023-10-07 22:51:33,133][67838] Updated weights for policy 0, policy_version 75272 (0.0009) [2023-10-07 22:51:33,407][67871] Updated weights for policy 1, policy_version 75390 (0.0007) [2023-10-07 22:51:33,505][67838] Updated weights for policy 0, policy_version 75282 (0.0009) [2023-10-07 22:51:33,875][67838] Updated weights for policy 0, policy_version 75292 (0.0007) [2023-10-07 22:51:37,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 154304512. Throughput: 0: 1673.6, 1: 1669.7. Samples: 38592228. Policy #0 lag: (min: 16.0, avg: 35.1, max: 48.0) [2023-10-07 22:51:37,477][66916] Avg episode reward: [(0, '51.350'), (1, '63.120')] [2023-10-07 22:51:37,631][67871] Updated weights for policy 1, policy_version 75400 (0.0010) [2023-10-07 22:51:37,908][67838] Updated weights for policy 0, policy_version 75302 (0.0009) [2023-10-07 22:51:38,000][67871] Updated weights for policy 1, policy_version 75410 (0.0010) [2023-10-07 22:51:38,286][67838] Updated weights for policy 0, policy_version 75312 (0.0007) [2023-10-07 22:51:38,361][67871] Updated weights for policy 1, policy_version 75420 (0.0009) [2023-10-07 22:51:38,662][67838] Updated weights for policy 0, policy_version 75322 (0.0007) [2023-10-07 22:51:42,309][67871] Updated weights for policy 1, policy_version 75430 (0.0008) [2023-10-07 22:51:42,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 154370048. Throughput: 0: 1674.3, 1: 1670.0. Samples: 38601310. Policy #0 lag: (min: 16.0, avg: 35.1, max: 48.0) [2023-10-07 22:51:42,478][66916] Avg episode reward: [(0, '49.700'), (1, '59.090')] [2023-10-07 22:51:42,675][67871] Updated weights for policy 1, policy_version 75440 (0.0007) [2023-10-07 22:51:42,817][67838] Updated weights for policy 0, policy_version 75332 (0.0009) [2023-10-07 22:51:43,036][67871] Updated weights for policy 1, policy_version 75450 (0.0008) [2023-10-07 22:51:43,183][67838] Updated weights for policy 0, policy_version 75342 (0.0008) [2023-10-07 22:51:43,562][67838] Updated weights for policy 0, policy_version 75352 (0.0007) [2023-10-07 22:51:47,109][67871] Updated weights for policy 1, policy_version 75460 (0.0008) [2023-10-07 22:51:47,476][67871] Updated weights for policy 1, policy_version 75470 (0.0009) [2023-10-07 22:51:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154435584. Throughput: 0: 1670.7, 1: 1675.1. Samples: 38621768. Policy #0 lag: (min: 16.0, avg: 35.1, max: 48.0) [2023-10-07 22:51:47,477][66916] Avg episode reward: [(0, '50.480'), (1, '57.490')] [2023-10-07 22:51:47,847][67871] Updated weights for policy 1, policy_version 75480 (0.0009) [2023-10-07 22:51:47,847][67838] Updated weights for policy 0, policy_version 75362 (0.0008) [2023-10-07 22:51:48,215][67838] Updated weights for policy 0, policy_version 75372 (0.0009) [2023-10-07 22:51:48,596][67838] Updated weights for policy 0, policy_version 75382 (0.0010) [2023-10-07 22:51:48,957][67838] Updated weights for policy 0, policy_version 75392 (0.0008) [2023-10-07 22:51:52,104][67871] Updated weights for policy 1, policy_version 75490 (0.0007) [2023-10-07 22:51:52,475][67871] Updated weights for policy 1, policy_version 75500 (0.0009) [2023-10-07 22:51:52,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154501120. Throughput: 0: 1668.6, 1: 1674.6. Samples: 38642140. Policy #0 lag: (min: 16.0, avg: 35.1, max: 48.0) [2023-10-07 22:51:52,478][66916] Avg episode reward: [(0, '48.510'), (1, '57.930')] [2023-10-07 22:51:52,840][67871] Updated weights for policy 1, policy_version 75510 (0.0009) [2023-10-07 22:51:53,036][67838] Updated weights for policy 0, policy_version 75402 (0.0009) [2023-10-07 22:51:53,206][67871] Updated weights for policy 1, policy_version 75520 (0.0008) [2023-10-07 22:51:53,406][67838] Updated weights for policy 0, policy_version 75412 (0.0011) [2023-10-07 22:51:53,767][67838] Updated weights for policy 0, policy_version 75422 (0.0010) [2023-10-07 22:51:57,327][67871] Updated weights for policy 1, policy_version 75530 (0.0009) [2023-10-07 22:51:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154566656. Throughput: 0: 1666.4, 1: 1676.4. Samples: 38651114. Policy #0 lag: (min: 16.0, avg: 35.1, max: 48.0) [2023-10-07 22:51:57,478][66916] Avg episode reward: [(0, '51.450'), (1, '57.450')] [2023-10-07 22:51:57,693][67871] Updated weights for policy 1, policy_version 75540 (0.0008) [2023-10-07 22:51:57,864][67838] Updated weights for policy 0, policy_version 75432 (0.0011) [2023-10-07 22:51:58,059][67871] Updated weights for policy 1, policy_version 75550 (0.0008) [2023-10-07 22:51:58,228][67838] Updated weights for policy 0, policy_version 75442 (0.0008) [2023-10-07 22:51:58,598][67838] Updated weights for policy 0, policy_version 75452 (0.0008) [2023-10-07 22:52:02,240][67871] Updated weights for policy 1, policy_version 75560 (0.0007) [2023-10-07 22:52:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154632192. Throughput: 0: 1659.3, 1: 1670.2. Samples: 38671302. Policy #0 lag: (min: 16.0, avg: 35.1, max: 48.0) [2023-10-07 22:52:02,477][66916] Avg episode reward: [(0, '50.570'), (1, '57.870')] [2023-10-07 22:52:02,613][67871] Updated weights for policy 1, policy_version 75570 (0.0007) [2023-10-07 22:52:02,971][67838] Updated weights for policy 0, policy_version 75462 (0.0007) [2023-10-07 22:52:02,986][67871] Updated weights for policy 1, policy_version 75580 (0.0008) [2023-10-07 22:52:03,345][67838] Updated weights for policy 0, policy_version 75472 (0.0009) [2023-10-07 22:52:03,728][67838] Updated weights for policy 0, policy_version 75482 (0.0011) [2023-10-07 22:52:07,139][67871] Updated weights for policy 1, policy_version 75590 (0.0008) [2023-10-07 22:52:07,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 154697728. Throughput: 0: 1655.4, 1: 1668.8. Samples: 38691506. Policy #0 lag: (min: 16.0, avg: 35.1, max: 48.0) [2023-10-07 22:52:07,477][66916] Avg episode reward: [(0, '52.070'), (1, '58.310')] [2023-10-07 22:52:07,505][67871] Updated weights for policy 1, policy_version 75600 (0.0007) [2023-10-07 22:52:07,875][67871] Updated weights for policy 1, policy_version 75610 (0.0008) [2023-10-07 22:52:08,004][67838] Updated weights for policy 0, policy_version 75492 (0.0009) [2023-10-07 22:52:08,368][67838] Updated weights for policy 0, policy_version 75502 (0.0007) [2023-10-07 22:52:08,743][67838] Updated weights for policy 0, policy_version 75512 (0.0007) [2023-10-07 22:52:11,995][67871] Updated weights for policy 1, policy_version 75620 (0.0008) [2023-10-07 22:52:12,366][67871] Updated weights for policy 1, policy_version 75630 (0.0008) [2023-10-07 22:52:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 154763264. Throughput: 0: 1655.9, 1: 1666.0. Samples: 38700490. Policy #0 lag: (min: 16.0, avg: 35.1, max: 48.0) [2023-10-07 22:52:12,478][66916] Avg episode reward: [(0, '50.800'), (1, '60.170')] [2023-10-07 22:52:12,726][67871] Updated weights for policy 1, policy_version 75640 (0.0008) [2023-10-07 22:52:12,900][67838] Updated weights for policy 0, policy_version 75522 (0.0009) [2023-10-07 22:52:13,268][67838] Updated weights for policy 0, policy_version 75532 (0.0008) [2023-10-07 22:52:13,640][67838] Updated weights for policy 0, policy_version 75542 (0.0009) [2023-10-07 22:52:14,014][67838] Updated weights for policy 0, policy_version 75552 (0.0008) [2023-10-07 22:52:16,836][67871] Updated weights for policy 1, policy_version 75650 (0.0009) [2023-10-07 22:52:17,211][67871] Updated weights for policy 1, policy_version 75660 (0.0008) [2023-10-07 22:52:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154828800. Throughput: 0: 1654.6, 1: 1666.8. Samples: 38721062. Policy #0 lag: (min: 16.0, avg: 35.1, max: 48.0) [2023-10-07 22:52:17,477][66916] Avg episode reward: [(0, '50.040'), (1, '59.040')] [2023-10-07 22:52:17,580][67871] Updated weights for policy 1, policy_version 75670 (0.0009) [2023-10-07 22:52:17,949][67871] Updated weights for policy 1, policy_version 75680 (0.0008) [2023-10-07 22:52:18,049][67838] Updated weights for policy 0, policy_version 75562 (0.0010) [2023-10-07 22:52:18,427][67838] Updated weights for policy 0, policy_version 75572 (0.0007) [2023-10-07 22:52:18,810][67838] Updated weights for policy 0, policy_version 75582 (0.0010) [2023-10-07 22:52:22,171][67871] Updated weights for policy 1, policy_version 75690 (0.0007) [2023-10-07 22:52:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 154894336. Throughput: 0: 1648.9, 1: 1665.7. Samples: 38741386. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:52:22,478][66916] Avg episode reward: [(0, '51.510'), (1, '60.810')] [2023-10-07 22:52:22,545][67871] Updated weights for policy 1, policy_version 75700 (0.0008) [2023-10-07 22:52:22,855][67838] Updated weights for policy 0, policy_version 75592 (0.0009) [2023-10-07 22:52:22,895][67871] Updated weights for policy 1, policy_version 75710 (0.0008) [2023-10-07 22:52:23,220][67838] Updated weights for policy 0, policy_version 75602 (0.0008) [2023-10-07 22:52:23,599][67838] Updated weights for policy 0, policy_version 75612 (0.0009) [2023-10-07 22:52:26,957][67871] Updated weights for policy 1, policy_version 75720 (0.0010) [2023-10-07 22:52:27,325][67871] Updated weights for policy 1, policy_version 75730 (0.0011) [2023-10-07 22:52:27,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 154959872. Throughput: 0: 1648.4, 1: 1665.9. Samples: 38750456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:52:27,477][66916] Avg episode reward: [(0, '51.350'), (1, '59.980')] [2023-10-07 22:52:27,687][67871] Updated weights for policy 1, policy_version 75740 (0.0010) [2023-10-07 22:52:27,786][67838] Updated weights for policy 0, policy_version 75622 (0.0009) [2023-10-07 22:52:28,171][67838] Updated weights for policy 0, policy_version 75632 (0.0010) [2023-10-07 22:52:28,544][67838] Updated weights for policy 0, policy_version 75642 (0.0010) [2023-10-07 22:52:31,875][67871] Updated weights for policy 1, policy_version 75750 (0.0009) [2023-10-07 22:52:32,251][67871] Updated weights for policy 1, policy_version 75760 (0.0009) [2023-10-07 22:52:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 155025408. Throughput: 0: 1652.4, 1: 1657.4. Samples: 38770712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:52:32,477][66916] Avg episode reward: [(0, '48.950'), (1, '60.360')] [2023-10-07 22:52:32,615][67871] Updated weights for policy 1, policy_version 75770 (0.0008) [2023-10-07 22:52:32,692][67838] Updated weights for policy 0, policy_version 75652 (0.0008) [2023-10-07 22:52:33,065][67838] Updated weights for policy 0, policy_version 75662 (0.0009) [2023-10-07 22:52:33,447][67838] Updated weights for policy 0, policy_version 75672 (0.0008) [2023-10-07 22:52:36,681][67871] Updated weights for policy 1, policy_version 75780 (0.0008) [2023-10-07 22:52:37,050][67871] Updated weights for policy 1, policy_version 75790 (0.0009) [2023-10-07 22:52:37,420][67871] Updated weights for policy 1, policy_version 75800 (0.0009) [2023-10-07 22:52:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 155090944. Throughput: 0: 1651.1, 1: 1649.7. Samples: 38790674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:52:37,477][66916] Avg episode reward: [(0, '50.190'), (1, '59.130')] [2023-10-07 22:52:37,516][67838] Updated weights for policy 0, policy_version 75682 (0.0007) [2023-10-07 22:52:37,882][67838] Updated weights for policy 0, policy_version 75692 (0.0008) [2023-10-07 22:52:38,257][67838] Updated weights for policy 0, policy_version 75702 (0.0009) [2023-10-07 22:52:38,632][67838] Updated weights for policy 0, policy_version 75712 (0.0008) [2023-10-07 22:52:41,621][67871] Updated weights for policy 1, policy_version 75810 (0.0009) [2023-10-07 22:52:42,041][67871] Updated weights for policy 1, policy_version 75820 (0.0008) [2023-10-07 22:52:42,408][67871] Updated weights for policy 1, policy_version 75830 (0.0007) [2023-10-07 22:52:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 155156480. Throughput: 0: 1651.2, 1: 1656.9. Samples: 38799980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:52:42,477][66916] Avg episode reward: [(0, '49.420'), (1, '60.240')] [2023-10-07 22:52:42,776][67871] Updated weights for policy 1, policy_version 75840 (0.0009) [2023-10-07 22:52:42,791][67838] Updated weights for policy 0, policy_version 75722 (0.0008) [2023-10-07 22:52:43,159][67838] Updated weights for policy 0, policy_version 75732 (0.0007) [2023-10-07 22:52:43,524][67838] Updated weights for policy 0, policy_version 75742 (0.0008) [2023-10-07 22:52:46,963][67871] Updated weights for policy 1, policy_version 75850 (0.0007) [2023-10-07 22:52:47,337][67871] Updated weights for policy 1, policy_version 75860 (0.0009) [2023-10-07 22:52:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 155222016. Throughput: 0: 1651.4, 1: 1655.1. Samples: 38820092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:52:47,477][66916] Avg episode reward: [(0, '47.280'), (1, '55.330')] [2023-10-07 22:52:47,523][67838] Updated weights for policy 0, policy_version 75752 (0.0008) [2023-10-07 22:52:47,697][67871] Updated weights for policy 1, policy_version 75870 (0.0008) [2023-10-07 22:52:47,890][67838] Updated weights for policy 0, policy_version 75762 (0.0008) [2023-10-07 22:52:48,267][67838] Updated weights for policy 0, policy_version 75772 (0.0007) [2023-10-07 22:52:51,672][67871] Updated weights for policy 1, policy_version 75880 (0.0009) [2023-10-07 22:52:52,033][67871] Updated weights for policy 1, policy_version 75890 (0.0007) [2023-10-07 22:52:52,391][67871] Updated weights for policy 1, policy_version 75900 (0.0008) [2023-10-07 22:52:52,406][67838] Updated weights for policy 0, policy_version 75782 (0.0009) [2023-10-07 22:52:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 155287552. Throughput: 0: 1657.1, 1: 1645.2. Samples: 38840108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:52:52,477][66916] Avg episode reward: [(0, '48.130'), (1, '54.440')] [2023-10-07 22:52:52,780][67838] Updated weights for policy 0, policy_version 75792 (0.0010) [2023-10-07 22:52:53,146][67838] Updated weights for policy 0, policy_version 75802 (0.0009) [2023-10-07 22:52:56,485][67871] Updated weights for policy 1, policy_version 75910 (0.0009) [2023-10-07 22:52:56,858][67871] Updated weights for policy 1, policy_version 75920 (0.0007) [2023-10-07 22:52:57,191][67838] Updated weights for policy 0, policy_version 75812 (0.0009) [2023-10-07 22:52:57,220][67871] Updated weights for policy 1, policy_version 75930 (0.0008) [2023-10-07 22:52:57,476][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 155385856. Throughput: 0: 1657.3, 1: 1656.2. Samples: 38849598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:52:57,477][66916] Avg episode reward: [(0, '47.840'), (1, '52.810')] [2023-10-07 22:52:57,567][67838] Updated weights for policy 0, policy_version 75822 (0.0009) [2023-10-07 22:52:57,942][67838] Updated weights for policy 0, policy_version 75832 (0.0007) [2023-10-07 22:53:01,223][67871] Updated weights for policy 1, policy_version 75940 (0.0008) [2023-10-07 22:53:01,585][67871] Updated weights for policy 1, policy_version 75950 (0.0007) [2023-10-07 22:53:01,954][67871] Updated weights for policy 1, policy_version 75960 (0.0007) [2023-10-07 22:53:02,271][67838] Updated weights for policy 0, policy_version 75842 (0.0009) [2023-10-07 22:53:02,476][66916] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 155451392. Throughput: 0: 1655.8, 1: 1652.6. Samples: 38869940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:53:02,477][66916] Avg episode reward: [(0, '51.370'), (1, '51.970')] [2023-10-07 22:53:02,642][67838] Updated weights for policy 0, policy_version 75852 (0.0009) [2023-10-07 22:53:03,015][67838] Updated weights for policy 0, policy_version 75862 (0.0008) [2023-10-07 22:53:03,391][67838] Updated weights for policy 0, policy_version 75872 (0.0009) [2023-10-07 22:53:06,259][67871] Updated weights for policy 1, policy_version 75970 (0.0008) [2023-10-07 22:53:06,619][67871] Updated weights for policy 1, policy_version 75980 (0.0008) [2023-10-07 22:53:06,982][67871] Updated weights for policy 1, policy_version 75990 (0.0007) [2023-10-07 22:53:07,347][67871] Updated weights for policy 1, policy_version 76000 (0.0008) [2023-10-07 22:53:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 155516928. Throughput: 0: 1659.2, 1: 1638.2. Samples: 38889766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:53:07,478][66916] Avg episode reward: [(0, '50.060'), (1, '54.920')] [2023-10-07 22:53:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000076000_77824000.pth... [2023-10-07 22:53:07,509][67838] Updated weights for policy 0, policy_version 75882 (0.0007) [2023-10-07 22:53:07,523][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000074432_76218368.pth [2023-10-07 22:53:07,882][67838] Updated weights for policy 0, policy_version 75892 (0.0008) [2023-10-07 22:53:08,262][67838] Updated weights for policy 0, policy_version 75902 (0.0008) [2023-10-07 22:53:08,329][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000075904_77725696.pth... [2023-10-07 22:53:08,358][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000074336_76120064.pth [2023-10-07 22:53:11,465][67871] Updated weights for policy 1, policy_version 76010 (0.0009) [2023-10-07 22:53:11,825][67871] Updated weights for policy 1, policy_version 76020 (0.0011) [2023-10-07 22:53:12,194][67871] Updated weights for policy 1, policy_version 76030 (0.0010) [2023-10-07 22:53:12,364][67838] Updated weights for policy 0, policy_version 75912 (0.0007) [2023-10-07 22:53:12,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 155582464. Throughput: 0: 1662.2, 1: 1651.4. Samples: 38899570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:53:12,477][66916] Avg episode reward: [(0, '51.300'), (1, '52.920')] [2023-10-07 22:53:12,739][67838] Updated weights for policy 0, policy_version 75922 (0.0007) [2023-10-07 22:53:13,109][67838] Updated weights for policy 0, policy_version 75932 (0.0008) [2023-10-07 22:53:16,549][67871] Updated weights for policy 1, policy_version 76040 (0.0008) [2023-10-07 22:53:16,918][67871] Updated weights for policy 1, policy_version 76050 (0.0010) [2023-10-07 22:53:17,117][67838] Updated weights for policy 0, policy_version 75942 (0.0008) [2023-10-07 22:53:17,288][67871] Updated weights for policy 1, policy_version 76060 (0.0009) [2023-10-07 22:53:17,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 155648000. Throughput: 0: 1665.4, 1: 1652.4. Samples: 38920014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:53:17,478][66916] Avg episode reward: [(0, '52.960'), (1, '54.530')] [2023-10-07 22:53:17,506][67838] Updated weights for policy 0, policy_version 75952 (0.0008) [2023-10-07 22:53:17,884][67838] Updated weights for policy 0, policy_version 75962 (0.0008) [2023-10-07 22:53:21,337][67871] Updated weights for policy 1, policy_version 76070 (0.0010) [2023-10-07 22:53:21,707][67871] Updated weights for policy 1, policy_version 76080 (0.0010) [2023-10-07 22:53:22,072][67871] Updated weights for policy 1, policy_version 76090 (0.0010) [2023-10-07 22:53:22,180][67838] Updated weights for policy 0, policy_version 75972 (0.0010) [2023-10-07 22:53:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 155713536. Throughput: 0: 1661.9, 1: 1644.4. Samples: 38939460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:53:22,478][66916] Avg episode reward: [(0, '51.140'), (1, '52.060')] [2023-10-07 22:53:22,556][67838] Updated weights for policy 0, policy_version 75982 (0.0009) [2023-10-07 22:53:22,930][67838] Updated weights for policy 0, policy_version 75992 (0.0009) [2023-10-07 22:53:26,248][67871] Updated weights for policy 1, policy_version 76100 (0.0007) [2023-10-07 22:53:26,650][67871] Updated weights for policy 1, policy_version 76110 (0.0007) [2023-10-07 22:53:27,016][67871] Updated weights for policy 1, policy_version 76120 (0.0008) [2023-10-07 22:53:27,057][67838] Updated weights for policy 0, policy_version 76002 (0.0009) [2023-10-07 22:53:27,424][67838] Updated weights for policy 0, policy_version 76012 (0.0010) [2023-10-07 22:53:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 155779072. Throughput: 0: 1660.1, 1: 1655.4. Samples: 38949178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:53:27,477][66916] Avg episode reward: [(0, '52.050'), (1, '51.450')] [2023-10-07 22:53:27,800][67838] Updated weights for policy 0, policy_version 76022 (0.0010) [2023-10-07 22:53:28,177][67838] Updated weights for policy 0, policy_version 76032 (0.0007) [2023-10-07 22:53:30,968][67871] Updated weights for policy 1, policy_version 76130 (0.0007) [2023-10-07 22:53:31,328][67871] Updated weights for policy 1, policy_version 76140 (0.0009) [2023-10-07 22:53:31,693][67871] Updated weights for policy 1, policy_version 76150 (0.0009) [2023-10-07 22:53:32,064][67871] Updated weights for policy 1, policy_version 76160 (0.0010) [2023-10-07 22:53:32,270][67838] Updated weights for policy 0, policy_version 76042 (0.0009) [2023-10-07 22:53:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 155844608. Throughput: 0: 1660.2, 1: 1662.0. Samples: 38969590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:53:32,477][66916] Avg episode reward: [(0, '55.310'), (1, '51.560')] [2023-10-07 22:53:32,633][67838] Updated weights for policy 0, policy_version 76052 (0.0009) [2023-10-07 22:53:33,011][67838] Updated weights for policy 0, policy_version 76062 (0.0008) [2023-10-07 22:53:36,366][67871] Updated weights for policy 1, policy_version 76170 (0.0008) [2023-10-07 22:53:36,744][67871] Updated weights for policy 1, policy_version 76180 (0.0009) [2023-10-07 22:53:37,114][67871] Updated weights for policy 1, policy_version 76190 (0.0008) [2023-10-07 22:53:37,431][67838] Updated weights for policy 0, policy_version 76072 (0.0007) [2023-10-07 22:53:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 155910144. Throughput: 0: 1657.4, 1: 1647.7. Samples: 38988838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:53:37,477][66916] Avg episode reward: [(0, '52.200'), (1, '53.710')] [2023-10-07 22:53:37,797][67838] Updated weights for policy 0, policy_version 76082 (0.0008) [2023-10-07 22:53:38,173][67838] Updated weights for policy 0, policy_version 76092 (0.0008) [2023-10-07 22:53:41,337][67871] Updated weights for policy 1, policy_version 76200 (0.0007) [2023-10-07 22:53:41,706][67871] Updated weights for policy 1, policy_version 76210 (0.0007) [2023-10-07 22:53:42,064][67871] Updated weights for policy 1, policy_version 76220 (0.0008) [2023-10-07 22:53:42,283][67838] Updated weights for policy 0, policy_version 76102 (0.0008) [2023-10-07 22:53:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 155975680. Throughput: 0: 1658.5, 1: 1658.8. Samples: 38998874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:53:42,478][66916] Avg episode reward: [(0, '52.080'), (1, '51.650')] [2023-10-07 22:53:42,661][67838] Updated weights for policy 0, policy_version 76112 (0.0012) [2023-10-07 22:53:43,047][67838] Updated weights for policy 0, policy_version 76122 (0.0010) [2023-10-07 22:53:46,115][67871] Updated weights for policy 1, policy_version 76230 (0.0007) [2023-10-07 22:53:46,485][67871] Updated weights for policy 1, policy_version 76240 (0.0007) [2023-10-07 22:53:46,845][67871] Updated weights for policy 1, policy_version 76250 (0.0009) [2023-10-07 22:53:47,036][67838] Updated weights for policy 0, policy_version 76132 (0.0010) [2023-10-07 22:53:47,419][67838] Updated weights for policy 0, policy_version 76142 (0.0009) [2023-10-07 22:53:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 156041216. Throughput: 0: 1656.4, 1: 1660.3. Samples: 39019192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:53:47,478][66916] Avg episode reward: [(0, '52.500'), (1, '54.450')] [2023-10-07 22:53:47,797][67838] Updated weights for policy 0, policy_version 76152 (0.0007) [2023-10-07 22:53:50,895][67871] Updated weights for policy 1, policy_version 76260 (0.0008) [2023-10-07 22:53:51,268][67871] Updated weights for policy 1, policy_version 76270 (0.0008) [2023-10-07 22:53:51,631][67871] Updated weights for policy 1, policy_version 76280 (0.0010) [2023-10-07 22:53:51,804][67838] Updated weights for policy 0, policy_version 76162 (0.0007) [2023-10-07 22:53:52,187][67838] Updated weights for policy 0, policy_version 76172 (0.0008) [2023-10-07 22:53:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 156106752. Throughput: 0: 1647.5, 1: 1647.5. Samples: 39038040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:53:52,477][66916] Avg episode reward: [(0, '50.280'), (1, '57.490')] [2023-10-07 22:53:52,561][67838] Updated weights for policy 0, policy_version 76182 (0.0007) [2023-10-07 22:53:52,926][67838] Updated weights for policy 0, policy_version 76192 (0.0009) [2023-10-07 22:53:55,783][67871] Updated weights for policy 1, policy_version 76290 (0.0008) [2023-10-07 22:53:56,155][67871] Updated weights for policy 1, policy_version 76300 (0.0007) [2023-10-07 22:53:56,526][67871] Updated weights for policy 1, policy_version 76310 (0.0007) [2023-10-07 22:53:56,892][67871] Updated weights for policy 1, policy_version 76320 (0.0008) [2023-10-07 22:53:57,201][67838] Updated weights for policy 0, policy_version 76202 (0.0009) [2023-10-07 22:53:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 156172288. Throughput: 0: 1651.5, 1: 1656.5. Samples: 39048430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:53:57,477][66916] Avg episode reward: [(0, '50.100'), (1, '57.690')] [2023-10-07 22:53:57,578][67838] Updated weights for policy 0, policy_version 76212 (0.0007) [2023-10-07 22:53:57,953][67838] Updated weights for policy 0, policy_version 76222 (0.0007) [2023-10-07 22:54:00,899][67871] Updated weights for policy 1, policy_version 76330 (0.0010) [2023-10-07 22:54:01,258][67871] Updated weights for policy 1, policy_version 76340 (0.0009) [2023-10-07 22:54:01,621][67871] Updated weights for policy 1, policy_version 76350 (0.0008) [2023-10-07 22:54:02,033][67838] Updated weights for policy 0, policy_version 76232 (0.0007) [2023-10-07 22:54:02,398][67838] Updated weights for policy 0, policy_version 76242 (0.0009) [2023-10-07 22:54:02,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 156237824. Throughput: 0: 1649.5, 1: 1649.5. Samples: 39068468. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) [2023-10-07 22:54:02,477][66916] Avg episode reward: [(0, '52.100'), (1, '56.480')] [2023-10-07 22:54:02,779][67838] Updated weights for policy 0, policy_version 76252 (0.0009) [2023-10-07 22:54:05,760][67871] Updated weights for policy 1, policy_version 76360 (0.0008) [2023-10-07 22:54:06,116][67871] Updated weights for policy 1, policy_version 76370 (0.0011) [2023-10-07 22:54:06,476][67871] Updated weights for policy 1, policy_version 76380 (0.0011) [2023-10-07 22:54:06,855][67838] Updated weights for policy 0, policy_version 76262 (0.0008) [2023-10-07 22:54:07,236][67838] Updated weights for policy 0, policy_version 76272 (0.0009) [2023-10-07 22:54:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 156303360. Throughput: 0: 1644.3, 1: 1651.0. Samples: 39087746. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) [2023-10-07 22:54:07,478][66916] Avg episode reward: [(0, '51.020'), (1, '60.270')] [2023-10-07 22:54:07,612][67838] Updated weights for policy 0, policy_version 76282 (0.0008) [2023-10-07 22:54:10,707][67871] Updated weights for policy 1, policy_version 76390 (0.0008) [2023-10-07 22:54:11,075][67871] Updated weights for policy 1, policy_version 76400 (0.0008) [2023-10-07 22:54:11,448][67871] Updated weights for policy 1, policy_version 76410 (0.0009) [2023-10-07 22:54:11,520][67838] Updated weights for policy 0, policy_version 76292 (0.0008) [2023-10-07 22:54:11,881][67838] Updated weights for policy 0, policy_version 76302 (0.0009) [2023-10-07 22:54:12,252][67838] Updated weights for policy 0, policy_version 76312 (0.0009) [2023-10-07 22:54:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 156368896. Throughput: 0: 1659.6, 1: 1661.8. Samples: 39098642. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) [2023-10-07 22:54:12,477][66916] Avg episode reward: [(0, '51.870'), (1, '59.270')] [2023-10-07 22:54:15,604][67871] Updated weights for policy 1, policy_version 76420 (0.0007) [2023-10-07 22:54:15,996][67871] Updated weights for policy 1, policy_version 76430 (0.0007) [2023-10-07 22:54:16,365][67871] Updated weights for policy 1, policy_version 76440 (0.0007) [2023-10-07 22:54:16,575][67838] Updated weights for policy 0, policy_version 76322 (0.0007) [2023-10-07 22:54:16,947][67838] Updated weights for policy 0, policy_version 76332 (0.0007) [2023-10-07 22:54:17,320][67838] Updated weights for policy 0, policy_version 76342 (0.0007) [2023-10-07 22:54:17,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 156434432. Throughput: 0: 1661.4, 1: 1647.2. Samples: 39118476. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) [2023-10-07 22:54:17,477][66916] Avg episode reward: [(0, '49.690'), (1, '54.920')] [2023-10-07 22:54:17,688][67838] Updated weights for policy 0, policy_version 76352 (0.0007) [2023-10-07 22:54:20,359][67871] Updated weights for policy 1, policy_version 76450 (0.0008) [2023-10-07 22:54:20,731][67871] Updated weights for policy 1, policy_version 76460 (0.0008) [2023-10-07 22:54:21,097][67871] Updated weights for policy 1, policy_version 76470 (0.0009) [2023-10-07 22:54:21,469][67871] Updated weights for policy 1, policy_version 76480 (0.0007) [2023-10-07 22:54:21,814][67838] Updated weights for policy 0, policy_version 76362 (0.0007) [2023-10-07 22:54:22,181][67838] Updated weights for policy 0, policy_version 76372 (0.0007) [2023-10-07 22:54:22,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 156499968. Throughput: 0: 1649.8, 1: 1654.3. Samples: 39137522. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) [2023-10-07 22:54:22,477][66916] Avg episode reward: [(0, '48.750'), (1, '56.120')] [2023-10-07 22:54:22,554][67838] Updated weights for policy 0, policy_version 76382 (0.0008) [2023-10-07 22:54:25,483][67871] Updated weights for policy 1, policy_version 76490 (0.0010) [2023-10-07 22:54:25,847][67871] Updated weights for policy 1, policy_version 76500 (0.0007) [2023-10-07 22:54:26,214][67871] Updated weights for policy 1, policy_version 76510 (0.0007) [2023-10-07 22:54:26,594][67838] Updated weights for policy 0, policy_version 76392 (0.0009) [2023-10-07 22:54:26,961][67838] Updated weights for policy 0, policy_version 76402 (0.0009) [2023-10-07 22:54:27,336][67838] Updated weights for policy 0, policy_version 76412 (0.0009) [2023-10-07 22:54:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 156565504. Throughput: 0: 1662.9, 1: 1666.6. Samples: 39148702. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) [2023-10-07 22:54:27,477][66916] Avg episode reward: [(0, '47.310'), (1, '53.440')] [2023-10-07 22:54:30,269][67871] Updated weights for policy 1, policy_version 76520 (0.0010) [2023-10-07 22:54:30,630][67871] Updated weights for policy 1, policy_version 76530 (0.0009) [2023-10-07 22:54:31,006][67871] Updated weights for policy 1, policy_version 76540 (0.0011) [2023-10-07 22:54:31,554][67838] Updated weights for policy 0, policy_version 76422 (0.0010) [2023-10-07 22:54:31,925][67838] Updated weights for policy 0, policy_version 76432 (0.0011) [2023-10-07 22:54:32,297][67838] Updated weights for policy 0, policy_version 76442 (0.0009) [2023-10-07 22:54:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 156631040. Throughput: 0: 1666.1, 1: 1648.7. Samples: 39168358. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) [2023-10-07 22:54:32,477][66916] Avg episode reward: [(0, '48.070'), (1, '52.250')] [2023-10-07 22:54:35,306][67871] Updated weights for policy 1, policy_version 76550 (0.0008) [2023-10-07 22:54:35,671][67871] Updated weights for policy 1, policy_version 76560 (0.0009) [2023-10-07 22:54:36,041][67871] Updated weights for policy 1, policy_version 76570 (0.0009) [2023-10-07 22:54:36,473][67838] Updated weights for policy 0, policy_version 76452 (0.0009) [2023-10-07 22:54:36,846][67838] Updated weights for policy 0, policy_version 76462 (0.0007) [2023-10-07 22:54:37,220][67838] Updated weights for policy 0, policy_version 76472 (0.0007) [2023-10-07 22:54:37,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 156696576. Throughput: 0: 1656.5, 1: 1667.3. Samples: 39187612. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) [2023-10-07 22:54:37,478][66916] Avg episode reward: [(0, '47.720'), (1, '50.080')] [2023-10-07 22:54:40,174][67871] Updated weights for policy 1, policy_version 76580 (0.0010) [2023-10-07 22:54:40,553][67871] Updated weights for policy 1, policy_version 76590 (0.0008) [2023-10-07 22:54:40,913][67871] Updated weights for policy 1, policy_version 76600 (0.0008) [2023-10-07 22:54:41,126][67838] Updated weights for policy 0, policy_version 76482 (0.0007) [2023-10-07 22:54:41,494][67838] Updated weights for policy 0, policy_version 76492 (0.0011) [2023-10-07 22:54:41,875][67838] Updated weights for policy 0, policy_version 76502 (0.0010) [2023-10-07 22:54:42,244][67838] Updated weights for policy 0, policy_version 76512 (0.0010) [2023-10-07 22:54:42,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 156794880. Throughput: 0: 1665.7, 1: 1670.6. Samples: 39198564. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) [2023-10-07 22:54:42,478][66916] Avg episode reward: [(0, '48.960'), (1, '53.050')] [2023-10-07 22:54:44,858][67871] Updated weights for policy 1, policy_version 76610 (0.0007) [2023-10-07 22:54:45,231][67871] Updated weights for policy 1, policy_version 76620 (0.0010) [2023-10-07 22:54:45,599][67871] Updated weights for policy 1, policy_version 76630 (0.0010) [2023-10-07 22:54:45,958][67871] Updated weights for policy 1, policy_version 76640 (0.0010) [2023-10-07 22:54:46,487][67838] Updated weights for policy 0, policy_version 76522 (0.0009) [2023-10-07 22:54:46,869][67838] Updated weights for policy 0, policy_version 76532 (0.0009) [2023-10-07 22:54:47,247][67838] Updated weights for policy 0, policy_version 76542 (0.0009) [2023-10-07 22:54:47,476][66916] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 156860416. Throughput: 0: 1663.8, 1: 1660.8. Samples: 39218076. Policy #0 lag: (min: 17.0, avg: 30.6, max: 49.0) [2023-10-07 22:54:47,477][66916] Avg episode reward: [(0, '48.800'), (1, '52.260')] [2023-10-07 22:54:50,170][67871] Updated weights for policy 1, policy_version 76650 (0.0007) [2023-10-07 22:54:50,540][67871] Updated weights for policy 1, policy_version 76660 (0.0007) [2023-10-07 22:54:50,901][67871] Updated weights for policy 1, policy_version 76670 (0.0007) [2023-10-07 22:54:51,226][67838] Updated weights for policy 0, policy_version 76552 (0.0008) [2023-10-07 22:54:51,593][67838] Updated weights for policy 0, policy_version 76562 (0.0008) [2023-10-07 22:54:51,964][67838] Updated weights for policy 0, policy_version 76572 (0.0011) [2023-10-07 22:54:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 156925952. Throughput: 0: 1649.9, 1: 1672.8. Samples: 39237266. Policy #0 lag: (min: 17.0, avg: 30.6, max: 49.0) [2023-10-07 22:54:52,477][66916] Avg episode reward: [(0, '50.700'), (1, '55.770')] [2023-10-07 22:54:55,051][67871] Updated weights for policy 1, policy_version 76680 (0.0009) [2023-10-07 22:54:55,413][67871] Updated weights for policy 1, policy_version 76690 (0.0010) [2023-10-07 22:54:55,781][67871] Updated weights for policy 1, policy_version 76700 (0.0010) [2023-10-07 22:54:56,121][67838] Updated weights for policy 0, policy_version 76582 (0.0010) [2023-10-07 22:54:56,499][67838] Updated weights for policy 0, policy_version 76592 (0.0007) [2023-10-07 22:54:56,860][67838] Updated weights for policy 0, policy_version 76602 (0.0007) [2023-10-07 22:54:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 156991488. Throughput: 0: 1661.9, 1: 1665.7. Samples: 39248386. Policy #0 lag: (min: 17.0, avg: 30.6, max: 49.0) [2023-10-07 22:54:57,477][66916] Avg episode reward: [(0, '50.350'), (1, '57.030')] [2023-10-07 22:54:59,811][67871] Updated weights for policy 1, policy_version 76710 (0.0008) [2023-10-07 22:55:00,187][67871] Updated weights for policy 1, policy_version 76720 (0.0008) [2023-10-07 22:55:00,550][67871] Updated weights for policy 1, policy_version 76730 (0.0007) [2023-10-07 22:55:01,039][67838] Updated weights for policy 0, policy_version 76612 (0.0011) [2023-10-07 22:55:01,421][67838] Updated weights for policy 0, policy_version 76622 (0.0009) [2023-10-07 22:55:01,792][67838] Updated weights for policy 0, policy_version 76632 (0.0008) [2023-10-07 22:55:02,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 157057024. Throughput: 0: 1652.9, 1: 1657.6. Samples: 39267448. Policy #0 lag: (min: 17.0, avg: 30.6, max: 49.0) [2023-10-07 22:55:02,477][66916] Avg episode reward: [(0, '52.420'), (1, '57.310')] [2023-10-07 22:55:04,489][67871] Updated weights for policy 1, policy_version 76740 (0.0009) [2023-10-07 22:55:04,891][67871] Updated weights for policy 1, policy_version 76750 (0.0008) [2023-10-07 22:55:05,256][67871] Updated weights for policy 1, policy_version 76760 (0.0008) [2023-10-07 22:55:06,180][67838] Updated weights for policy 0, policy_version 76642 (0.0007) [2023-10-07 22:55:06,556][67838] Updated weights for policy 0, policy_version 76652 (0.0008) [2023-10-07 22:55:06,935][67838] Updated weights for policy 0, policy_version 76662 (0.0007) [2023-10-07 22:55:07,304][67838] Updated weights for policy 0, policy_version 76672 (0.0009) [2023-10-07 22:55:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 157122560. Throughput: 0: 1646.0, 1: 1676.0. Samples: 39287014. Policy #0 lag: (min: 17.0, avg: 30.6, max: 49.0) [2023-10-07 22:55:07,478][66916] Avg episode reward: [(0, '53.500'), (1, '56.490')] [2023-10-07 22:55:07,490][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000076768_78610432.pth... [2023-10-07 22:55:07,490][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000076672_78512128.pth... [2023-10-07 22:55:07,538][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000075104_76906496.pth [2023-10-07 22:55:07,539][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000075200_77004800.pth [2023-10-07 22:55:09,339][67871] Updated weights for policy 1, policy_version 76770 (0.0009) [2023-10-07 22:55:09,708][67871] Updated weights for policy 1, policy_version 76780 (0.0007) [2023-10-07 22:55:10,064][67871] Updated weights for policy 1, policy_version 76790 (0.0011) [2023-10-07 22:55:10,434][67871] Updated weights for policy 1, policy_version 76800 (0.0011) [2023-10-07 22:55:11,597][67838] Updated weights for policy 0, policy_version 76682 (0.0008) [2023-10-07 22:55:11,972][67838] Updated weights for policy 0, policy_version 76692 (0.0009) [2023-10-07 22:55:12,336][67838] Updated weights for policy 0, policy_version 76702 (0.0007) [2023-10-07 22:55:12,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 157188096. Throughput: 0: 1651.1, 1: 1655.4. Samples: 39297492. Policy #0 lag: (min: 17.0, avg: 30.6, max: 49.0) [2023-10-07 22:55:12,478][66916] Avg episode reward: [(0, '53.830'), (1, '58.110')] [2023-10-07 22:55:14,789][67871] Updated weights for policy 1, policy_version 76810 (0.0007) [2023-10-07 22:55:15,156][67871] Updated weights for policy 1, policy_version 76820 (0.0007) [2023-10-07 22:55:15,516][67871] Updated weights for policy 1, policy_version 76830 (0.0008) [2023-10-07 22:55:16,370][67838] Updated weights for policy 0, policy_version 76712 (0.0010) [2023-10-07 22:55:16,751][67838] Updated weights for policy 0, policy_version 76722 (0.0009) [2023-10-07 22:55:17,125][67838] Updated weights for policy 0, policy_version 76732 (0.0007) [2023-10-07 22:55:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 157253632. Throughput: 0: 1651.6, 1: 1655.1. Samples: 39317158. Policy #0 lag: (min: 17.0, avg: 30.6, max: 49.0) [2023-10-07 22:55:17,477][66916] Avg episode reward: [(0, '53.490'), (1, '54.070')] [2023-10-07 22:55:19,510][67871] Updated weights for policy 1, policy_version 76840 (0.0008) [2023-10-07 22:55:19,867][67871] Updated weights for policy 1, policy_version 76850 (0.0007) [2023-10-07 22:55:20,233][67871] Updated weights for policy 1, policy_version 76860 (0.0009) [2023-10-07 22:55:21,024][67838] Updated weights for policy 0, policy_version 76742 (0.0007) [2023-10-07 22:55:21,389][67838] Updated weights for policy 0, policy_version 76752 (0.0007) [2023-10-07 22:55:21,762][67838] Updated weights for policy 0, policy_version 76762 (0.0009) [2023-10-07 22:55:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 157319168. Throughput: 0: 1640.3, 1: 1668.5. Samples: 39336510. Policy #0 lag: (min: 17.0, avg: 30.6, max: 49.0) [2023-10-07 22:55:22,477][66916] Avg episode reward: [(0, '51.380'), (1, '53.820')] [2023-10-07 22:55:24,480][67871] Updated weights for policy 1, policy_version 76870 (0.0008) [2023-10-07 22:55:24,844][67871] Updated weights for policy 1, policy_version 76880 (0.0009) [2023-10-07 22:55:25,208][67871] Updated weights for policy 1, policy_version 76890 (0.0010) [2023-10-07 22:55:25,864][67838] Updated weights for policy 0, policy_version 76772 (0.0010) [2023-10-07 22:55:26,231][67838] Updated weights for policy 0, policy_version 76782 (0.0008) [2023-10-07 22:55:26,606][67838] Updated weights for policy 0, policy_version 76792 (0.0008) [2023-10-07 22:55:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 157384704. Throughput: 0: 1653.6, 1: 1653.0. Samples: 39347358. Policy #0 lag: (min: 17.0, avg: 30.6, max: 49.0) [2023-10-07 22:55:27,478][66916] Avg episode reward: [(0, '53.880'), (1, '51.980')] [2023-10-07 22:55:29,345][67871] Updated weights for policy 1, policy_version 76900 (0.0008) [2023-10-07 22:55:29,721][67871] Updated weights for policy 1, policy_version 76910 (0.0008) [2023-10-07 22:55:30,085][67871] Updated weights for policy 1, policy_version 76920 (0.0007) [2023-10-07 22:55:30,738][67838] Updated weights for policy 0, policy_version 76802 (0.0007) [2023-10-07 22:55:31,105][67838] Updated weights for policy 0, policy_version 76812 (0.0007) [2023-10-07 22:55:31,475][67838] Updated weights for policy 0, policy_version 76822 (0.0010) [2023-10-07 22:55:31,852][67838] Updated weights for policy 0, policy_version 76832 (0.0009) [2023-10-07 22:55:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 157450240. Throughput: 0: 1643.3, 1: 1662.0. Samples: 39366812. Policy #0 lag: (min: 17.0, avg: 30.6, max: 49.0) [2023-10-07 22:55:32,477][66916] Avg episode reward: [(0, '51.950'), (1, '56.500')] [2023-10-07 22:55:34,091][67871] Updated weights for policy 1, policy_version 76930 (0.0009) [2023-10-07 22:55:34,456][67871] Updated weights for policy 1, policy_version 76940 (0.0008) [2023-10-07 22:55:34,820][67871] Updated weights for policy 1, policy_version 76950 (0.0007) [2023-10-07 22:55:35,188][67871] Updated weights for policy 1, policy_version 76960 (0.0007) [2023-10-07 22:55:36,123][67838] Updated weights for policy 0, policy_version 76842 (0.0010) [2023-10-07 22:55:36,487][67838] Updated weights for policy 0, policy_version 76852 (0.0011) [2023-10-07 22:55:36,873][67838] Updated weights for policy 0, policy_version 76862 (0.0007) [2023-10-07 22:55:37,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 157515776. Throughput: 0: 1644.2, 1: 1667.2. Samples: 39386280. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:55:37,477][66916] Avg episode reward: [(0, '51.020'), (1, '58.750')] [2023-10-07 22:55:39,321][67871] Updated weights for policy 1, policy_version 76970 (0.0007) [2023-10-07 22:55:39,684][67871] Updated weights for policy 1, policy_version 76980 (0.0007) [2023-10-07 22:55:40,044][67871] Updated weights for policy 1, policy_version 76990 (0.0007) [2023-10-07 22:55:40,903][67838] Updated weights for policy 0, policy_version 76872 (0.0011) [2023-10-07 22:55:41,280][67838] Updated weights for policy 0, policy_version 76882 (0.0010) [2023-10-07 22:55:41,649][67838] Updated weights for policy 0, policy_version 76892 (0.0008) [2023-10-07 22:55:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 157581312. Throughput: 0: 1651.8, 1: 1652.8. Samples: 39397092. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:55:42,477][66916] Avg episode reward: [(0, '49.250'), (1, '58.740')] [2023-10-07 22:55:44,019][67871] Updated weights for policy 1, policy_version 77000 (0.0010) [2023-10-07 22:55:44,381][67871] Updated weights for policy 1, policy_version 77010 (0.0008) [2023-10-07 22:55:44,748][67871] Updated weights for policy 1, policy_version 77020 (0.0007) [2023-10-07 22:55:45,826][67838] Updated weights for policy 0, policy_version 76902 (0.0007) [2023-10-07 22:55:46,199][67838] Updated weights for policy 0, policy_version 76912 (0.0008) [2023-10-07 22:55:46,575][67838] Updated weights for policy 0, policy_version 76922 (0.0010) [2023-10-07 22:55:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157646848. Throughput: 0: 1653.1, 1: 1668.6. Samples: 39416924. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:55:47,477][66916] Avg episode reward: [(0, '48.020'), (1, '60.440')] [2023-10-07 22:55:48,849][67871] Updated weights for policy 1, policy_version 77030 (0.0009) [2023-10-07 22:55:49,216][67871] Updated weights for policy 1, policy_version 77040 (0.0010) [2023-10-07 22:55:49,580][67871] Updated weights for policy 1, policy_version 77050 (0.0011) [2023-10-07 22:55:50,549][67838] Updated weights for policy 0, policy_version 76932 (0.0008) [2023-10-07 22:55:50,925][67838] Updated weights for policy 0, policy_version 76942 (0.0010) [2023-10-07 22:55:51,285][67838] Updated weights for policy 0, policy_version 76952 (0.0009) [2023-10-07 22:55:52,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157712384. Throughput: 0: 1657.5, 1: 1671.5. Samples: 39436816. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:55:52,477][66916] Avg episode reward: [(0, '50.010'), (1, '61.730')] [2023-10-07 22:55:53,770][67871] Updated weights for policy 1, policy_version 77060 (0.0010) [2023-10-07 22:55:54,162][67871] Updated weights for policy 1, policy_version 77070 (0.0007) [2023-10-07 22:55:54,538][67871] Updated weights for policy 1, policy_version 77080 (0.0008) [2023-10-07 22:55:55,352][67838] Updated weights for policy 0, policy_version 76962 (0.0008) [2023-10-07 22:55:55,727][67838] Updated weights for policy 0, policy_version 76972 (0.0010) [2023-10-07 22:55:56,106][67838] Updated weights for policy 0, policy_version 76982 (0.0007) [2023-10-07 22:55:56,477][67838] Updated weights for policy 0, policy_version 76992 (0.0008) [2023-10-07 22:55:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157777920. Throughput: 0: 1666.3, 1: 1656.6. Samples: 39447022. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:55:57,478][66916] Avg episode reward: [(0, '48.250'), (1, '59.480')] [2023-10-07 22:55:58,655][67871] Updated weights for policy 1, policy_version 77090 (0.0009) [2023-10-07 22:55:59,030][67871] Updated weights for policy 1, policy_version 77100 (0.0007) [2023-10-07 22:55:59,400][67871] Updated weights for policy 1, policy_version 77110 (0.0008) [2023-10-07 22:55:59,761][67871] Updated weights for policy 1, policy_version 77120 (0.0007) [2023-10-07 22:56:00,570][67838] Updated weights for policy 0, policy_version 77002 (0.0008) [2023-10-07 22:56:00,937][67838] Updated weights for policy 0, policy_version 77012 (0.0009) [2023-10-07 22:56:01,305][67838] Updated weights for policy 0, policy_version 77022 (0.0007) [2023-10-07 22:56:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157843456. Throughput: 0: 1648.5, 1: 1674.0. Samples: 39466674. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:56:02,478][66916] Avg episode reward: [(0, '47.810'), (1, '56.200')] [2023-10-07 22:56:03,926][67871] Updated weights for policy 1, policy_version 77130 (0.0007) [2023-10-07 22:56:04,293][67871] Updated weights for policy 1, policy_version 77140 (0.0007) [2023-10-07 22:56:04,661][67871] Updated weights for policy 1, policy_version 77150 (0.0009) [2023-10-07 22:56:05,417][67838] Updated weights for policy 0, policy_version 77032 (0.0007) [2023-10-07 22:56:05,795][67838] Updated weights for policy 0, policy_version 77042 (0.0009) [2023-10-07 22:56:06,166][67838] Updated weights for policy 0, policy_version 77052 (0.0007) [2023-10-07 22:56:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 157908992. Throughput: 0: 1670.3, 1: 1672.5. Samples: 39486936. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:56:07,478][66916] Avg episode reward: [(0, '47.060'), (1, '55.760')] [2023-10-07 22:56:08,730][67871] Updated weights for policy 1, policy_version 77160 (0.0010) [2023-10-07 22:56:09,091][67871] Updated weights for policy 1, policy_version 77170 (0.0008) [2023-10-07 22:56:09,468][67871] Updated weights for policy 1, policy_version 77180 (0.0009) [2023-10-07 22:56:10,232][67838] Updated weights for policy 0, policy_version 77062 (0.0010) [2023-10-07 22:56:10,595][67838] Updated weights for policy 0, policy_version 77072 (0.0007) [2023-10-07 22:56:10,971][67838] Updated weights for policy 0, policy_version 77082 (0.0009) [2023-10-07 22:56:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157974528. Throughput: 0: 1669.4, 1: 1657.8. Samples: 39497084. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:56:12,477][66916] Avg episode reward: [(0, '47.980'), (1, '55.180')] [2023-10-07 22:56:13,605][67871] Updated weights for policy 1, policy_version 77190 (0.0010) [2023-10-07 22:56:13,965][67871] Updated weights for policy 1, policy_version 77200 (0.0008) [2023-10-07 22:56:14,324][67871] Updated weights for policy 1, policy_version 77210 (0.0007) [2023-10-07 22:56:14,935][67838] Updated weights for policy 0, policy_version 77092 (0.0007) [2023-10-07 22:56:15,311][67838] Updated weights for policy 0, policy_version 77102 (0.0009) [2023-10-07 22:56:15,674][67838] Updated weights for policy 0, policy_version 77112 (0.0010) [2023-10-07 22:56:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 158040064. Throughput: 0: 1657.4, 1: 1666.4. Samples: 39516384. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:56:17,477][66916] Avg episode reward: [(0, '51.680'), (1, '55.350')] [2023-10-07 22:56:18,508][67871] Updated weights for policy 1, policy_version 77220 (0.0008) [2023-10-07 22:56:18,882][67871] Updated weights for policy 1, policy_version 77230 (0.0010) [2023-10-07 22:56:19,235][67871] Updated weights for policy 1, policy_version 77240 (0.0010) [2023-10-07 22:56:19,819][67838] Updated weights for policy 0, policy_version 77122 (0.0010) [2023-10-07 22:56:20,200][67838] Updated weights for policy 0, policy_version 77132 (0.0010) [2023-10-07 22:56:20,576][67838] Updated weights for policy 0, policy_version 77142 (0.0007) [2023-10-07 22:56:20,940][67838] Updated weights for policy 0, policy_version 77152 (0.0008) [2023-10-07 22:56:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 158105600. Throughput: 0: 1685.1, 1: 1666.1. Samples: 39537084. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 22:56:22,477][66916] Avg episode reward: [(0, '49.930'), (1, '55.400')] [2023-10-07 22:56:23,358][67871] Updated weights for policy 1, policy_version 77250 (0.0008) [2023-10-07 22:56:23,723][67871] Updated weights for policy 1, policy_version 77260 (0.0010) [2023-10-07 22:56:24,081][67871] Updated weights for policy 1, policy_version 77270 (0.0008) [2023-10-07 22:56:24,453][67871] Updated weights for policy 1, policy_version 77280 (0.0007) [2023-10-07 22:56:24,994][67838] Updated weights for policy 0, policy_version 77162 (0.0009) [2023-10-07 22:56:25,368][67838] Updated weights for policy 0, policy_version 77172 (0.0008) [2023-10-07 22:56:25,747][67838] Updated weights for policy 0, policy_version 77182 (0.0008) [2023-10-07 22:56:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 158171136. Throughput: 0: 1670.0, 1: 1659.3. Samples: 39546910. Policy #0 lag: (min: 4.0, avg: 10.0, max: 36.0) [2023-10-07 22:56:27,478][66916] Avg episode reward: [(0, '53.040'), (1, '58.970')] [2023-10-07 22:56:28,711][67871] Updated weights for policy 1, policy_version 77290 (0.0008) [2023-10-07 22:56:29,079][67871] Updated weights for policy 1, policy_version 77300 (0.0008) [2023-10-07 22:56:29,444][67871] Updated weights for policy 1, policy_version 77310 (0.0010) [2023-10-07 22:56:29,814][67838] Updated weights for policy 0, policy_version 77192 (0.0008) [2023-10-07 22:56:30,188][67838] Updated weights for policy 0, policy_version 77202 (0.0007) [2023-10-07 22:56:30,559][67838] Updated weights for policy 0, policy_version 77212 (0.0008) [2023-10-07 22:56:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 158236672. Throughput: 0: 1661.3, 1: 1664.1. Samples: 39566568. Policy #0 lag: (min: 4.0, avg: 10.0, max: 36.0) [2023-10-07 22:56:32,477][66916] Avg episode reward: [(0, '50.750'), (1, '57.670')] [2023-10-07 22:56:33,447][67871] Updated weights for policy 1, policy_version 77320 (0.0007) [2023-10-07 22:56:33,809][67871] Updated weights for policy 1, policy_version 77330 (0.0008) [2023-10-07 22:56:34,176][67871] Updated weights for policy 1, policy_version 77340 (0.0009) [2023-10-07 22:56:34,584][67838] Updated weights for policy 0, policy_version 77222 (0.0008) [2023-10-07 22:56:34,965][67838] Updated weights for policy 0, policy_version 77232 (0.0009) [2023-10-07 22:56:35,335][67838] Updated weights for policy 0, policy_version 77242 (0.0010) [2023-10-07 22:56:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 158302208. Throughput: 0: 1679.7, 1: 1663.5. Samples: 39587260. Policy #0 lag: (min: 4.0, avg: 10.0, max: 36.0) [2023-10-07 22:56:37,477][66916] Avg episode reward: [(0, '52.000'), (1, '56.900')] [2023-10-07 22:56:38,494][67871] Updated weights for policy 1, policy_version 77350 (0.0008) [2023-10-07 22:56:38,876][67871] Updated weights for policy 1, policy_version 77360 (0.0009) [2023-10-07 22:56:39,242][67871] Updated weights for policy 1, policy_version 77370 (0.0008) [2023-10-07 22:56:39,445][67838] Updated weights for policy 0, policy_version 77252 (0.0009) [2023-10-07 22:56:39,821][67838] Updated weights for policy 0, policy_version 77262 (0.0008) [2023-10-07 22:56:40,185][67838] Updated weights for policy 0, policy_version 77272 (0.0009) [2023-10-07 22:56:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 158367744. Throughput: 0: 1663.0, 1: 1660.3. Samples: 39596570. Policy #0 lag: (min: 4.0, avg: 10.0, max: 36.0) [2023-10-07 22:56:42,478][66916] Avg episode reward: [(0, '47.210'), (1, '56.960')] [2023-10-07 22:56:43,434][67871] Updated weights for policy 1, policy_version 77380 (0.0008) [2023-10-07 22:56:43,794][67871] Updated weights for policy 1, policy_version 77390 (0.0007) [2023-10-07 22:56:44,158][67871] Updated weights for policy 1, policy_version 77400 (0.0008) [2023-10-07 22:56:44,273][67838] Updated weights for policy 0, policy_version 77282 (0.0009) [2023-10-07 22:56:44,645][67838] Updated weights for policy 0, policy_version 77292 (0.0007) [2023-10-07 22:56:45,007][67838] Updated weights for policy 0, policy_version 77302 (0.0007) [2023-10-07 22:56:45,382][67838] Updated weights for policy 0, policy_version 77312 (0.0008) [2023-10-07 22:56:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 158433280. Throughput: 0: 1669.7, 1: 1658.6. Samples: 39616448. Policy #0 lag: (min: 4.0, avg: 10.0, max: 36.0) [2023-10-07 22:56:47,477][66916] Avg episode reward: [(0, '47.040'), (1, '57.200')] [2023-10-07 22:56:48,170][67871] Updated weights for policy 1, policy_version 77410 (0.0009) [2023-10-07 22:56:48,539][67871] Updated weights for policy 1, policy_version 77420 (0.0009) [2023-10-07 22:56:48,901][67871] Updated weights for policy 1, policy_version 77430 (0.0009) [2023-10-07 22:56:49,272][67871] Updated weights for policy 1, policy_version 77440 (0.0008) [2023-10-07 22:56:49,442][67838] Updated weights for policy 0, policy_version 77322 (0.0010) [2023-10-07 22:56:49,812][67838] Updated weights for policy 0, policy_version 77332 (0.0010) [2023-10-07 22:56:50,182][67838] Updated weights for policy 0, policy_version 77342 (0.0011) [2023-10-07 22:56:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 158498816. Throughput: 0: 1672.7, 1: 1663.1. Samples: 39637046. Policy #0 lag: (min: 4.0, avg: 10.0, max: 36.0) [2023-10-07 22:56:52,477][66916] Avg episode reward: [(0, '44.850'), (1, '55.460')] [2023-10-07 22:56:53,238][67871] Updated weights for policy 1, policy_version 77450 (0.0009) [2023-10-07 22:56:53,609][67871] Updated weights for policy 1, policy_version 77460 (0.0007) [2023-10-07 22:56:53,967][67871] Updated weights for policy 1, policy_version 77470 (0.0008) [2023-10-07 22:56:54,451][67838] Updated weights for policy 0, policy_version 77352 (0.0009) [2023-10-07 22:56:54,822][67838] Updated weights for policy 0, policy_version 77362 (0.0007) [2023-10-07 22:56:55,206][67838] Updated weights for policy 0, policy_version 77372 (0.0008) [2023-10-07 22:56:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 158564352. Throughput: 0: 1653.9, 1: 1667.7. Samples: 39646556. Policy #0 lag: (min: 4.0, avg: 10.0, max: 36.0) [2023-10-07 22:56:57,478][66916] Avg episode reward: [(0, '46.420'), (1, '55.030')] [2023-10-07 22:56:58,117][67871] Updated weights for policy 1, policy_version 77480 (0.0009) [2023-10-07 22:56:58,500][67871] Updated weights for policy 1, policy_version 77490 (0.0009) [2023-10-07 22:56:58,866][67871] Updated weights for policy 1, policy_version 77500 (0.0007) [2023-10-07 22:56:59,397][67838] Updated weights for policy 0, policy_version 77382 (0.0008) [2023-10-07 22:56:59,773][67838] Updated weights for policy 0, policy_version 77392 (0.0009) [2023-10-07 22:57:00,149][67838] Updated weights for policy 0, policy_version 77402 (0.0010) [2023-10-07 22:57:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 158629888. Throughput: 0: 1665.8, 1: 1670.0. Samples: 39666494. Policy #0 lag: (min: 4.0, avg: 10.0, max: 36.0) [2023-10-07 22:57:02,478][66916] Avg episode reward: [(0, '43.500'), (1, '56.520')] [2023-10-07 22:57:02,828][67871] Updated weights for policy 1, policy_version 77510 (0.0008) [2023-10-07 22:57:03,193][67871] Updated weights for policy 1, policy_version 77520 (0.0009) [2023-10-07 22:57:03,560][67871] Updated weights for policy 1, policy_version 77530 (0.0007) [2023-10-07 22:57:04,207][67838] Updated weights for policy 0, policy_version 77412 (0.0008) [2023-10-07 22:57:04,575][67838] Updated weights for policy 0, policy_version 77422 (0.0009) [2023-10-07 22:57:04,958][67838] Updated weights for policy 0, policy_version 77432 (0.0010) [2023-10-07 22:57:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 158695424. Throughput: 0: 1665.9, 1: 1667.8. Samples: 39687100. Policy #0 lag: (min: 4.0, avg: 10.0, max: 36.0) [2023-10-07 22:57:07,478][66916] Avg episode reward: [(0, '42.710'), (1, '56.430')] [2023-10-07 22:57:07,492][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000077440_79298560.pth... [2023-10-07 22:57:07,492][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000077536_79396864.pth... [2023-10-07 22:57:07,528][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000076000_77824000.pth [2023-10-07 22:57:07,532][67676] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p1/milestones/checkpoint_000077536_79396864.pth [2023-10-07 22:57:07,535][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000075904_77725696.pth [2023-10-07 22:57:07,539][67511] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p0/milestones/checkpoint_000077440_79298560.pth [2023-10-07 22:57:07,822][67871] Updated weights for policy 1, policy_version 77540 (0.0008) [2023-10-07 22:57:08,190][67871] Updated weights for policy 1, policy_version 77550 (0.0008) [2023-10-07 22:57:08,560][67871] Updated weights for policy 1, policy_version 77560 (0.0008) [2023-10-07 22:57:09,080][67838] Updated weights for policy 0, policy_version 77442 (0.0007) [2023-10-07 22:57:09,450][67838] Updated weights for policy 0, policy_version 77452 (0.0007) [2023-10-07 22:57:09,823][67838] Updated weights for policy 0, policy_version 77462 (0.0007) [2023-10-07 22:57:10,202][67838] Updated weights for policy 0, policy_version 77472 (0.0007) [2023-10-07 22:57:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 158760960. Throughput: 0: 1653.1, 1: 1664.7. Samples: 39696212. Policy #0 lag: (min: 4.0, avg: 10.0, max: 36.0) [2023-10-07 22:57:12,478][66916] Avg episode reward: [(0, '44.840'), (1, '55.260')] [2023-10-07 22:57:12,792][67871] Updated weights for policy 1, policy_version 77570 (0.0009) [2023-10-07 22:57:13,163][67871] Updated weights for policy 1, policy_version 77580 (0.0008) [2023-10-07 22:57:13,526][67871] Updated weights for policy 1, policy_version 77590 (0.0007) [2023-10-07 22:57:13,893][67871] Updated weights for policy 1, policy_version 77600 (0.0007) [2023-10-07 22:57:14,372][67838] Updated weights for policy 0, policy_version 77482 (0.0011) [2023-10-07 22:57:14,752][67838] Updated weights for policy 0, policy_version 77492 (0.0008) [2023-10-07 22:57:15,127][67838] Updated weights for policy 0, policy_version 77502 (0.0007) [2023-10-07 22:57:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 158826496. Throughput: 0: 1658.1, 1: 1666.4. Samples: 39716174. Policy #0 lag: (min: 16.0, avg: 41.9, max: 48.0) [2023-10-07 22:57:17,477][66916] Avg episode reward: [(0, '47.290'), (1, '59.730')] [2023-10-07 22:57:17,942][67871] Updated weights for policy 1, policy_version 77610 (0.0008) [2023-10-07 22:57:18,310][67871] Updated weights for policy 1, policy_version 77620 (0.0011) [2023-10-07 22:57:18,672][67871] Updated weights for policy 1, policy_version 77630 (0.0010) [2023-10-07 22:57:19,344][67838] Updated weights for policy 0, policy_version 77512 (0.0007) [2023-10-07 22:57:19,719][67838] Updated weights for policy 0, policy_version 77522 (0.0007) [2023-10-07 22:57:20,104][67838] Updated weights for policy 0, policy_version 77532 (0.0010) [2023-10-07 22:57:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 158892032. Throughput: 0: 1653.7, 1: 1672.7. Samples: 39736948. Policy #0 lag: (min: 16.0, avg: 41.9, max: 48.0) [2023-10-07 22:57:22,477][66916] Avg episode reward: [(0, '47.670'), (1, '59.950')] [2023-10-07 22:57:22,504][67871] Updated weights for policy 1, policy_version 77640 (0.0011) [2023-10-07 22:57:22,885][67871] Updated weights for policy 1, policy_version 77650 (0.0011) [2023-10-07 22:57:23,255][67871] Updated weights for policy 1, policy_version 77660 (0.0010) [2023-10-07 22:57:24,288][67838] Updated weights for policy 0, policy_version 77542 (0.0011) [2023-10-07 22:57:24,670][67838] Updated weights for policy 0, policy_version 77552 (0.0011) [2023-10-07 22:57:25,028][67838] Updated weights for policy 0, policy_version 77562 (0.0011) [2023-10-07 22:57:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 158957568. Throughput: 0: 1645.7, 1: 1678.4. Samples: 39746156. Policy #0 lag: (min: 16.0, avg: 41.9, max: 48.0) [2023-10-07 22:57:27,477][66916] Avg episode reward: [(0, '52.240'), (1, '58.740')] [2023-10-07 22:57:27,489][67871] Updated weights for policy 1, policy_version 77670 (0.0009) [2023-10-07 22:57:27,869][67871] Updated weights for policy 1, policy_version 77680 (0.0007) [2023-10-07 22:57:28,236][67871] Updated weights for policy 1, policy_version 77690 (0.0009) [2023-10-07 22:57:29,200][67838] Updated weights for policy 0, policy_version 77572 (0.0009) [2023-10-07 22:57:29,574][67838] Updated weights for policy 0, policy_version 77582 (0.0010) [2023-10-07 22:57:29,951][67838] Updated weights for policy 0, policy_version 77592 (0.0011) [2023-10-07 22:57:32,140][67871] Updated weights for policy 1, policy_version 77700 (0.0009) [2023-10-07 22:57:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 159023104. Throughput: 0: 1646.0, 1: 1677.6. Samples: 39766012. Policy #0 lag: (min: 16.0, avg: 41.9, max: 48.0) [2023-10-07 22:57:32,478][66916] Avg episode reward: [(0, '49.730'), (1, '62.420')] [2023-10-07 22:57:32,503][67871] Updated weights for policy 1, policy_version 77710 (0.0010) [2023-10-07 22:57:32,876][67871] Updated weights for policy 1, policy_version 77720 (0.0010) [2023-10-07 22:57:33,992][67838] Updated weights for policy 0, policy_version 77602 (0.0009) [2023-10-07 22:57:34,365][67838] Updated weights for policy 0, policy_version 77612 (0.0008) [2023-10-07 22:57:34,741][67838] Updated weights for policy 0, policy_version 77622 (0.0008) [2023-10-07 22:57:35,102][67838] Updated weights for policy 0, policy_version 77632 (0.0008) [2023-10-07 22:57:37,002][67871] Updated weights for policy 1, policy_version 77730 (0.0009) [2023-10-07 22:57:37,377][67871] Updated weights for policy 1, policy_version 77740 (0.0009) [2023-10-07 22:57:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 159088640. Throughput: 0: 1653.2, 1: 1674.7. Samples: 39786802. Policy #0 lag: (min: 16.0, avg: 41.9, max: 48.0) [2023-10-07 22:57:37,477][66916] Avg episode reward: [(0, '49.220'), (1, '62.400')] [2023-10-07 22:57:37,742][67871] Updated weights for policy 1, policy_version 77750 (0.0007) [2023-10-07 22:57:38,119][67871] Updated weights for policy 1, policy_version 77760 (0.0009) [2023-10-07 22:57:39,118][67838] Updated weights for policy 0, policy_version 77642 (0.0011) [2023-10-07 22:57:39,502][67838] Updated weights for policy 0, policy_version 77652 (0.0009) [2023-10-07 22:57:39,883][67838] Updated weights for policy 0, policy_version 77662 (0.0009) [2023-10-07 22:57:42,342][67871] Updated weights for policy 1, policy_version 77770 (0.0009) [2023-10-07 22:57:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 159154176. Throughput: 0: 1644.1, 1: 1670.2. Samples: 39795700. Policy #0 lag: (min: 16.0, avg: 41.9, max: 48.0) [2023-10-07 22:57:42,477][66916] Avg episode reward: [(0, '49.260'), (1, '62.090')] [2023-10-07 22:57:42,711][67871] Updated weights for policy 1, policy_version 77780 (0.0010) [2023-10-07 22:57:43,081][67871] Updated weights for policy 1, policy_version 77790 (0.0011) [2023-10-07 22:57:44,047][67838] Updated weights for policy 0, policy_version 77672 (0.0008) [2023-10-07 22:57:44,406][67838] Updated weights for policy 0, policy_version 77682 (0.0009) [2023-10-07 22:57:44,781][67838] Updated weights for policy 0, policy_version 77692 (0.0010) [2023-10-07 22:57:47,198][67871] Updated weights for policy 1, policy_version 77800 (0.0011) [2023-10-07 22:57:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 159219712. Throughput: 0: 1661.1, 1: 1666.0. Samples: 39816212. Policy #0 lag: (min: 16.0, avg: 41.9, max: 48.0) [2023-10-07 22:57:47,477][66916] Avg episode reward: [(0, '49.490'), (1, '58.550')] [2023-10-07 22:57:47,573][67871] Updated weights for policy 1, policy_version 77810 (0.0010) [2023-10-07 22:57:47,929][67871] Updated weights for policy 1, policy_version 77820 (0.0010) [2023-10-07 22:57:48,915][67838] Updated weights for policy 0, policy_version 77702 (0.0009) [2023-10-07 22:57:49,292][67838] Updated weights for policy 0, policy_version 77712 (0.0008) [2023-10-07 22:57:49,664][67838] Updated weights for policy 0, policy_version 77722 (0.0009) [2023-10-07 22:57:52,270][67871] Updated weights for policy 1, policy_version 77830 (0.0010) [2023-10-07 22:57:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 159285248. Throughput: 0: 1659.5, 1: 1666.5. Samples: 39836768. Policy #0 lag: (min: 16.0, avg: 41.9, max: 48.0) [2023-10-07 22:57:52,477][66916] Avg episode reward: [(0, '48.910'), (1, '57.720')] [2023-10-07 22:57:52,640][67871] Updated weights for policy 1, policy_version 77840 (0.0011) [2023-10-07 22:57:53,021][67871] Updated weights for policy 1, policy_version 77850 (0.0008) [2023-10-07 22:57:53,575][67838] Updated weights for policy 0, policy_version 77732 (0.0008) [2023-10-07 22:57:53,954][67838] Updated weights for policy 0, policy_version 77742 (0.0010) [2023-10-07 22:57:54,325][67838] Updated weights for policy 0, policy_version 77752 (0.0011) [2023-10-07 22:57:57,230][67871] Updated weights for policy 1, policy_version 77860 (0.0009) [2023-10-07 22:57:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 159350784. Throughput: 0: 1657.6, 1: 1666.4. Samples: 39845790. Policy #0 lag: (min: 16.0, avg: 41.9, max: 48.0) [2023-10-07 22:57:57,477][66916] Avg episode reward: [(0, '53.250'), (1, '57.130')] [2023-10-07 22:57:57,598][67871] Updated weights for policy 1, policy_version 77870 (0.0008) [2023-10-07 22:57:57,963][67871] Updated weights for policy 1, policy_version 77880 (0.0007) [2023-10-07 22:57:58,387][67838] Updated weights for policy 0, policy_version 77762 (0.0009) [2023-10-07 22:57:58,776][67838] Updated weights for policy 0, policy_version 77772 (0.0009) [2023-10-07 22:57:59,149][67838] Updated weights for policy 0, policy_version 77782 (0.0009) [2023-10-07 22:57:59,526][67838] Updated weights for policy 0, policy_version 77792 (0.0011) [2023-10-07 22:58:02,154][67871] Updated weights for policy 1, policy_version 77890 (0.0008) [2023-10-07 22:58:02,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 159416320. Throughput: 0: 1673.5, 1: 1667.7. Samples: 39866526. Policy #0 lag: (min: 16.0, avg: 41.9, max: 48.0) [2023-10-07 22:58:02,478][66916] Avg episode reward: [(0, '51.120'), (1, '56.870')] [2023-10-07 22:58:02,530][67871] Updated weights for policy 1, policy_version 77900 (0.0008) [2023-10-07 22:58:02,906][67871] Updated weights for policy 1, policy_version 77910 (0.0007) [2023-10-07 22:58:03,280][67871] Updated weights for policy 1, policy_version 77920 (0.0009) [2023-10-07 22:58:03,587][67838] Updated weights for policy 0, policy_version 77802 (0.0010) [2023-10-07 22:58:03,957][67838] Updated weights for policy 0, policy_version 77812 (0.0009) [2023-10-07 22:58:04,334][67838] Updated weights for policy 0, policy_version 77822 (0.0008) [2023-10-07 22:58:07,407][67871] Updated weights for policy 1, policy_version 77930 (0.0007) [2023-10-07 22:58:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 159481856. Throughput: 0: 1677.5, 1: 1656.4. Samples: 39886974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:58:07,477][66916] Avg episode reward: [(0, '53.200'), (1, '57.830')] [2023-10-07 22:58:07,781][67871] Updated weights for policy 1, policy_version 77940 (0.0009) [2023-10-07 22:58:08,147][67871] Updated weights for policy 1, policy_version 77950 (0.0009) [2023-10-07 22:58:08,325][67838] Updated weights for policy 0, policy_version 77832 (0.0009) [2023-10-07 22:58:08,695][67838] Updated weights for policy 0, policy_version 77842 (0.0010) [2023-10-07 22:58:09,064][67838] Updated weights for policy 0, policy_version 77852 (0.0009) [2023-10-07 22:58:12,386][67871] Updated weights for policy 1, policy_version 77960 (0.0009) [2023-10-07 22:58:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 159547392. Throughput: 0: 1671.1, 1: 1656.2. Samples: 39895886. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:58:12,477][66916] Avg episode reward: [(0, '50.180'), (1, '55.570')] [2023-10-07 22:58:12,747][67871] Updated weights for policy 1, policy_version 77970 (0.0010) [2023-10-07 22:58:13,114][67871] Updated weights for policy 1, policy_version 77980 (0.0007) [2023-10-07 22:58:13,333][67838] Updated weights for policy 0, policy_version 77862 (0.0010) [2023-10-07 22:58:13,717][67838] Updated weights for policy 0, policy_version 77872 (0.0011) [2023-10-07 22:58:14,081][67838] Updated weights for policy 0, policy_version 77882 (0.0010) [2023-10-07 22:58:17,081][67871] Updated weights for policy 1, policy_version 77990 (0.0008) [2023-10-07 22:58:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 159612928. Throughput: 0: 1676.9, 1: 1655.6. Samples: 39915976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:58:17,477][66916] Avg episode reward: [(0, '50.620'), (1, '58.420')] [2023-10-07 22:58:17,482][67871] Updated weights for policy 1, policy_version 78000 (0.0008) [2023-10-07 22:58:17,845][67871] Updated weights for policy 1, policy_version 78010 (0.0008) [2023-10-07 22:58:18,343][67838] Updated weights for policy 0, policy_version 77892 (0.0008) [2023-10-07 22:58:18,722][67838] Updated weights for policy 0, policy_version 77902 (0.0010) [2023-10-07 22:58:19,097][67838] Updated weights for policy 0, policy_version 77912 (0.0007) [2023-10-07 22:58:21,937][67871] Updated weights for policy 1, policy_version 78020 (0.0008) [2023-10-07 22:58:22,304][67871] Updated weights for policy 1, policy_version 78030 (0.0009) [2023-10-07 22:58:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 159678464. Throughput: 0: 1672.3, 1: 1649.4. Samples: 39936278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:58:22,477][66916] Avg episode reward: [(0, '48.740'), (1, '54.910')] [2023-10-07 22:58:22,666][67871] Updated weights for policy 1, policy_version 78040 (0.0008) [2023-10-07 22:58:23,038][67838] Updated weights for policy 0, policy_version 77922 (0.0009) [2023-10-07 22:58:23,415][67838] Updated weights for policy 0, policy_version 77932 (0.0009) [2023-10-07 22:58:23,776][67838] Updated weights for policy 0, policy_version 77942 (0.0010) [2023-10-07 22:58:24,148][67838] Updated weights for policy 0, policy_version 77952 (0.0008) [2023-10-07 22:58:26,865][67871] Updated weights for policy 1, policy_version 78050 (0.0007) [2023-10-07 22:58:27,229][67871] Updated weights for policy 1, policy_version 78060 (0.0008) [2023-10-07 22:58:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 159744000. Throughput: 0: 1672.7, 1: 1655.4. Samples: 39945462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:58:27,477][66916] Avg episode reward: [(0, '51.140'), (1, '52.980')] [2023-10-07 22:58:27,598][67871] Updated weights for policy 1, policy_version 78070 (0.0008) [2023-10-07 22:58:27,962][67871] Updated weights for policy 1, policy_version 78080 (0.0007) [2023-10-07 22:58:28,181][67838] Updated weights for policy 0, policy_version 77962 (0.0010) [2023-10-07 22:58:28,558][67838] Updated weights for policy 0, policy_version 77972 (0.0011) [2023-10-07 22:58:28,927][67838] Updated weights for policy 0, policy_version 77982 (0.0010) [2023-10-07 22:58:31,923][67871] Updated weights for policy 1, policy_version 78090 (0.0010) [2023-10-07 22:58:32,296][67871] Updated weights for policy 1, policy_version 78100 (0.0011) [2023-10-07 22:58:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 159809536. Throughput: 0: 1668.7, 1: 1659.3. Samples: 39965970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:58:32,478][66916] Avg episode reward: [(0, '48.690'), (1, '52.210')] [2023-10-07 22:58:32,661][67871] Updated weights for policy 1, policy_version 78110 (0.0009) [2023-10-07 22:58:33,074][67838] Updated weights for policy 0, policy_version 77992 (0.0009) [2023-10-07 22:58:33,437][67838] Updated weights for policy 0, policy_version 78002 (0.0007) [2023-10-07 22:58:33,804][67838] Updated weights for policy 0, policy_version 78012 (0.0009) [2023-10-07 22:58:36,771][67871] Updated weights for policy 1, policy_version 78120 (0.0007) [2023-10-07 22:58:37,138][67871] Updated weights for policy 1, policy_version 78130 (0.0007) [2023-10-07 22:58:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 159875072. Throughput: 0: 1662.1, 1: 1653.9. Samples: 39985990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:58:37,477][66916] Avg episode reward: [(0, '43.200'), (1, '52.020')] [2023-10-07 22:58:37,496][67871] Updated weights for policy 1, policy_version 78140 (0.0007) [2023-10-07 22:58:38,051][67838] Updated weights for policy 0, policy_version 78022 (0.0009) [2023-10-07 22:58:38,425][67838] Updated weights for policy 0, policy_version 78032 (0.0010) [2023-10-07 22:58:38,797][67838] Updated weights for policy 0, policy_version 78042 (0.0009) [2023-10-07 22:58:41,691][67871] Updated weights for policy 1, policy_version 78150 (0.0010) [2023-10-07 22:58:42,063][67871] Updated weights for policy 1, policy_version 78160 (0.0007) [2023-10-07 22:58:42,436][67871] Updated weights for policy 1, policy_version 78170 (0.0008) [2023-10-07 22:58:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 159940608. Throughput: 0: 1658.1, 1: 1664.1. Samples: 39995292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:58:42,477][66916] Avg episode reward: [(0, '48.630'), (1, '52.240')] [2023-10-07 22:58:42,887][67838] Updated weights for policy 0, policy_version 78052 (0.0010) [2023-10-07 22:58:43,265][67838] Updated weights for policy 0, policy_version 78062 (0.0008) [2023-10-07 22:58:43,629][67838] Updated weights for policy 0, policy_version 78072 (0.0008) [2023-10-07 22:58:46,446][67871] Updated weights for policy 1, policy_version 78180 (0.0008) [2023-10-07 22:58:46,817][67871] Updated weights for policy 1, policy_version 78190 (0.0007) [2023-10-07 22:58:47,176][67871] Updated weights for policy 1, policy_version 78200 (0.0008) [2023-10-07 22:58:47,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 160038912. Throughput: 0: 1652.1, 1: 1667.5. Samples: 40015906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:58:47,478][66916] Avg episode reward: [(0, '42.660'), (1, '51.890')] [2023-10-07 22:58:47,887][67838] Updated weights for policy 0, policy_version 78082 (0.0009) [2023-10-07 22:58:48,301][67838] Updated weights for policy 0, policy_version 78092 (0.0009) [2023-10-07 22:58:48,670][67838] Updated weights for policy 0, policy_version 78102 (0.0007) [2023-10-07 22:58:49,044][67838] Updated weights for policy 0, policy_version 78112 (0.0009) [2023-10-07 22:58:51,398][67871] Updated weights for policy 1, policy_version 78210 (0.0010) [2023-10-07 22:58:51,763][67871] Updated weights for policy 1, policy_version 78220 (0.0008) [2023-10-07 22:58:52,127][67871] Updated weights for policy 1, policy_version 78230 (0.0007) [2023-10-07 22:58:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 160071680. Throughput: 0: 1647.4, 1: 1650.0. Samples: 40035358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 22:58:52,477][66916] Avg episode reward: [(0, '47.600'), (1, '55.840')] [2023-10-07 22:58:52,491][67871] Updated weights for policy 1, policy_version 78240 (0.0008) [2023-10-07 22:58:53,131][67838] Updated weights for policy 0, policy_version 78122 (0.0008) [2023-10-07 22:58:53,494][67838] Updated weights for policy 0, policy_version 78132 (0.0008) [2023-10-07 22:58:53,877][67838] Updated weights for policy 0, policy_version 78142 (0.0009) [2023-10-07 22:58:56,704][67871] Updated weights for policy 1, policy_version 78250 (0.0008) [2023-10-07 22:58:57,072][67871] Updated weights for policy 1, policy_version 78260 (0.0007) [2023-10-07 22:58:57,432][67871] Updated weights for policy 1, policy_version 78270 (0.0007) [2023-10-07 22:58:57,476][66916] Fps is (10 sec: 9830.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 160137216. Throughput: 0: 1650.0, 1: 1662.3. Samples: 40044940. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-10-07 22:58:57,477][66916] Avg episode reward: [(0, '48.140'), (1, '56.200')] [2023-10-07 22:58:58,066][67838] Updated weights for policy 0, policy_version 78152 (0.0009) [2023-10-07 22:58:58,444][67838] Updated weights for policy 0, policy_version 78162 (0.0010) [2023-10-07 22:58:58,811][67838] Updated weights for policy 0, policy_version 78172 (0.0009) [2023-10-07 22:59:01,709][67871] Updated weights for policy 1, policy_version 78280 (0.0008) [2023-10-07 22:59:02,096][67871] Updated weights for policy 1, policy_version 78290 (0.0008) [2023-10-07 22:59:02,456][67871] Updated weights for policy 1, policy_version 78300 (0.0007) [2023-10-07 22:59:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 160202752. Throughput: 0: 1654.8, 1: 1666.1. Samples: 40065418. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-10-07 22:59:02,478][66916] Avg episode reward: [(0, '47.910'), (1, '56.810')] [2023-10-07 22:59:03,043][67838] Updated weights for policy 0, policy_version 78182 (0.0009) [2023-10-07 22:59:03,425][67838] Updated weights for policy 0, policy_version 78192 (0.0008) [2023-10-07 22:59:03,794][67838] Updated weights for policy 0, policy_version 78202 (0.0007) [2023-10-07 22:59:06,417][67871] Updated weights for policy 1, policy_version 78310 (0.0010) [2023-10-07 22:59:06,795][67871] Updated weights for policy 1, policy_version 78320 (0.0008) [2023-10-07 22:59:07,160][67871] Updated weights for policy 1, policy_version 78330 (0.0007) [2023-10-07 22:59:07,477][66916] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 160301056. Throughput: 0: 1654.3, 1: 1652.5. Samples: 40085088. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-10-07 22:59:07,478][66916] Avg episode reward: [(0, '49.630'), (1, '57.490')] [2023-10-07 22:59:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000078208_80084992.pth... [2023-10-07 22:59:07,490][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000078336_80216064.pth... [2023-10-07 22:59:07,527][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000076768_78610432.pth [2023-10-07 22:59:07,527][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000076672_78512128.pth [2023-10-07 22:59:07,943][67838] Updated weights for policy 0, policy_version 78212 (0.0010) [2023-10-07 22:59:08,303][67838] Updated weights for policy 0, policy_version 78222 (0.0010) [2023-10-07 22:59:08,670][67838] Updated weights for policy 0, policy_version 78232 (0.0012) [2023-10-07 22:59:11,177][67871] Updated weights for policy 1, policy_version 78340 (0.0009) [2023-10-07 22:59:11,548][67871] Updated weights for policy 1, policy_version 78350 (0.0007) [2023-10-07 22:59:11,907][67871] Updated weights for policy 1, policy_version 78360 (0.0008) [2023-10-07 22:59:12,476][66916] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 160366592. Throughput: 0: 1652.7, 1: 1664.1. Samples: 40094718. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-10-07 22:59:12,477][66916] Avg episode reward: [(0, '48.190'), (1, '56.150')] [2023-10-07 22:59:12,835][67838] Updated weights for policy 0, policy_version 78242 (0.0008) [2023-10-07 22:59:13,213][67838] Updated weights for policy 0, policy_version 78252 (0.0008) [2023-10-07 22:59:13,585][67838] Updated weights for policy 0, policy_version 78262 (0.0009) [2023-10-07 22:59:13,962][67838] Updated weights for policy 0, policy_version 78272 (0.0008) [2023-10-07 22:59:16,133][67871] Updated weights for policy 1, policy_version 78370 (0.0008) [2023-10-07 22:59:16,493][67871] Updated weights for policy 1, policy_version 78380 (0.0010) [2023-10-07 22:59:16,865][67871] Updated weights for policy 1, policy_version 78390 (0.0010) [2023-10-07 22:59:17,237][67871] Updated weights for policy 1, policy_version 78400 (0.0010) [2023-10-07 22:59:17,476][66916] Fps is (10 sec: 13107.8, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 160432128. Throughput: 0: 1653.2, 1: 1662.6. Samples: 40115180. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-10-07 22:59:17,477][66916] Avg episode reward: [(0, '52.570'), (1, '56.430')] [2023-10-07 22:59:17,821][67838] Updated weights for policy 0, policy_version 78282 (0.0008) [2023-10-07 22:59:18,197][67838] Updated weights for policy 0, policy_version 78292 (0.0007) [2023-10-07 22:59:18,562][67838] Updated weights for policy 0, policy_version 78302 (0.0007) [2023-10-07 22:59:21,429][67871] Updated weights for policy 1, policy_version 78410 (0.0007) [2023-10-07 22:59:21,793][67871] Updated weights for policy 1, policy_version 78420 (0.0008) [2023-10-07 22:59:22,150][67871] Updated weights for policy 1, policy_version 78430 (0.0007) [2023-10-07 22:59:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 160497664. Throughput: 0: 1660.1, 1: 1646.8. Samples: 40134802. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-10-07 22:59:22,477][66916] Avg episode reward: [(0, '50.100'), (1, '58.120')] [2023-10-07 22:59:22,697][67838] Updated weights for policy 0, policy_version 78312 (0.0007) [2023-10-07 22:59:23,075][67838] Updated weights for policy 0, policy_version 78322 (0.0007) [2023-10-07 22:59:23,445][67838] Updated weights for policy 0, policy_version 78332 (0.0008) [2023-10-07 22:59:26,255][67871] Updated weights for policy 1, policy_version 78440 (0.0010) [2023-10-07 22:59:26,616][67871] Updated weights for policy 1, policy_version 78450 (0.0011) [2023-10-07 22:59:26,988][67871] Updated weights for policy 1, policy_version 78460 (0.0009) [2023-10-07 22:59:27,337][67838] Updated weights for policy 0, policy_version 78342 (0.0008) [2023-10-07 22:59:27,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 160563200. Throughput: 0: 1660.3, 1: 1660.5. Samples: 40144728. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-10-07 22:59:27,477][66916] Avg episode reward: [(0, '50.550'), (1, '53.850')] [2023-10-07 22:59:27,716][67838] Updated weights for policy 0, policy_version 78352 (0.0009) [2023-10-07 22:59:28,079][67838] Updated weights for policy 0, policy_version 78362 (0.0010) [2023-10-07 22:59:30,926][67871] Updated weights for policy 1, policy_version 78470 (0.0007) [2023-10-07 22:59:31,298][67871] Updated weights for policy 1, policy_version 78480 (0.0008) [2023-10-07 22:59:31,666][67871] Updated weights for policy 1, policy_version 78490 (0.0007) [2023-10-07 22:59:32,224][67838] Updated weights for policy 0, policy_version 78372 (0.0009) [2023-10-07 22:59:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 160628736. Throughput: 0: 1664.7, 1: 1653.4. Samples: 40165220. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-10-07 22:59:32,477][66916] Avg episode reward: [(0, '50.890'), (1, '57.210')] [2023-10-07 22:59:32,604][67838] Updated weights for policy 0, policy_version 78382 (0.0009) [2023-10-07 22:59:32,974][67838] Updated weights for policy 0, policy_version 78392 (0.0009) [2023-10-07 22:59:35,723][67871] Updated weights for policy 1, policy_version 78500 (0.0008) [2023-10-07 22:59:36,088][67871] Updated weights for policy 1, policy_version 78510 (0.0007) [2023-10-07 22:59:36,453][67871] Updated weights for policy 1, policy_version 78520 (0.0007) [2023-10-07 22:59:37,112][67838] Updated weights for policy 0, policy_version 78402 (0.0008) [2023-10-07 22:59:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 160694272. Throughput: 0: 1669.9, 1: 1654.5. Samples: 40184956. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-10-07 22:59:37,477][66916] Avg episode reward: [(0, '51.250'), (1, '57.170')] [2023-10-07 22:59:37,521][67838] Updated weights for policy 0, policy_version 78412 (0.0009) [2023-10-07 22:59:37,892][67838] Updated weights for policy 0, policy_version 78422 (0.0009) [2023-10-07 22:59:38,266][67838] Updated weights for policy 0, policy_version 78432 (0.0008) [2023-10-07 22:59:40,567][67871] Updated weights for policy 1, policy_version 78530 (0.0008) [2023-10-07 22:59:40,940][67871] Updated weights for policy 1, policy_version 78540 (0.0009) [2023-10-07 22:59:41,307][67871] Updated weights for policy 1, policy_version 78550 (0.0009) [2023-10-07 22:59:41,670][67871] Updated weights for policy 1, policy_version 78560 (0.0009) [2023-10-07 22:59:42,468][67838] Updated weights for policy 0, policy_version 78442 (0.0012) [2023-10-07 22:59:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 160759808. Throughput: 0: 1666.2, 1: 1669.7. Samples: 40195058. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-10-07 22:59:42,477][66916] Avg episode reward: [(0, '52.460'), (1, '56.550')] [2023-10-07 22:59:42,829][67838] Updated weights for policy 0, policy_version 78452 (0.0010) [2023-10-07 22:59:43,202][67838] Updated weights for policy 0, policy_version 78462 (0.0009) [2023-10-07 22:59:45,761][67871] Updated weights for policy 1, policy_version 78570 (0.0010) [2023-10-07 22:59:46,127][67871] Updated weights for policy 1, policy_version 78580 (0.0008) [2023-10-07 22:59:46,482][67871] Updated weights for policy 1, policy_version 78590 (0.0008) [2023-10-07 22:59:47,358][67838] Updated weights for policy 0, policy_version 78472 (0.0010) [2023-10-07 22:59:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 160825344. Throughput: 0: 1666.1, 1: 1659.1. Samples: 40215054. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-07 22:59:47,477][66916] Avg episode reward: [(0, '49.150'), (1, '54.700')] [2023-10-07 22:59:47,718][67838] Updated weights for policy 0, policy_version 78482 (0.0009) [2023-10-07 22:59:48,091][67838] Updated weights for policy 0, policy_version 78492 (0.0009) [2023-10-07 22:59:50,731][67871] Updated weights for policy 1, policy_version 78600 (0.0009) [2023-10-07 22:59:51,122][67871] Updated weights for policy 1, policy_version 78610 (0.0011) [2023-10-07 22:59:51,491][67871] Updated weights for policy 1, policy_version 78620 (0.0011) [2023-10-07 22:59:52,173][67838] Updated weights for policy 0, policy_version 78502 (0.0009) [2023-10-07 22:59:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 160890880. Throughput: 0: 1667.5, 1: 1651.2. Samples: 40234428. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-07 22:59:52,477][66916] Avg episode reward: [(0, '47.150'), (1, '57.580')] [2023-10-07 22:59:52,545][67838] Updated weights for policy 0, policy_version 78512 (0.0007) [2023-10-07 22:59:52,926][67838] Updated weights for policy 0, policy_version 78522 (0.0007) [2023-10-07 22:59:55,575][67871] Updated weights for policy 1, policy_version 78630 (0.0008) [2023-10-07 22:59:55,951][67871] Updated weights for policy 1, policy_version 78640 (0.0008) [2023-10-07 22:59:56,316][67871] Updated weights for policy 1, policy_version 78650 (0.0009) [2023-10-07 22:59:57,037][67838] Updated weights for policy 0, policy_version 78532 (0.0008) [2023-10-07 22:59:57,422][67838] Updated weights for policy 0, policy_version 78542 (0.0009) [2023-10-07 22:59:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 160956416. Throughput: 0: 1669.6, 1: 1664.0. Samples: 40244732. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-07 22:59:57,477][66916] Avg episode reward: [(0, '46.740'), (1, '53.610')] [2023-10-07 22:59:57,797][67838] Updated weights for policy 0, policy_version 78552 (0.0010) [2023-10-07 23:00:00,304][67871] Updated weights for policy 1, policy_version 78660 (0.0010) [2023-10-07 23:00:00,663][67871] Updated weights for policy 1, policy_version 78670 (0.0011) [2023-10-07 23:00:01,023][67871] Updated weights for policy 1, policy_version 78680 (0.0009) [2023-10-07 23:00:01,775][67838] Updated weights for policy 0, policy_version 78562 (0.0009) [2023-10-07 23:00:02,146][67838] Updated weights for policy 0, policy_version 78572 (0.0009) [2023-10-07 23:00:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 161021952. Throughput: 0: 1672.1, 1: 1653.6. Samples: 40264836. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-07 23:00:02,477][66916] Avg episode reward: [(0, '45.970'), (1, '55.140')] [2023-10-07 23:00:02,517][67838] Updated weights for policy 0, policy_version 78582 (0.0007) [2023-10-07 23:00:02,880][67838] Updated weights for policy 0, policy_version 78592 (0.0007) [2023-10-07 23:00:05,174][67871] Updated weights for policy 1, policy_version 78690 (0.0009) [2023-10-07 23:00:05,548][67871] Updated weights for policy 1, policy_version 78700 (0.0010) [2023-10-07 23:00:05,919][67871] Updated weights for policy 1, policy_version 78710 (0.0010) [2023-10-07 23:00:06,286][67871] Updated weights for policy 1, policy_version 78720 (0.0010) [2023-10-07 23:00:07,032][67838] Updated weights for policy 0, policy_version 78602 (0.0008) [2023-10-07 23:00:07,413][67838] Updated weights for policy 0, policy_version 78612 (0.0007) [2023-10-07 23:00:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 161087488. Throughput: 0: 1660.0, 1: 1662.9. Samples: 40284334. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-07 23:00:07,477][66916] Avg episode reward: [(0, '48.630'), (1, '55.160')] [2023-10-07 23:00:07,781][67838] Updated weights for policy 0, policy_version 78622 (0.0007) [2023-10-07 23:00:10,337][67871] Updated weights for policy 1, policy_version 78730 (0.0008) [2023-10-07 23:00:10,701][67871] Updated weights for policy 1, policy_version 78740 (0.0010) [2023-10-07 23:00:11,067][67871] Updated weights for policy 1, policy_version 78750 (0.0008) [2023-10-07 23:00:11,936][67838] Updated weights for policy 0, policy_version 78632 (0.0009) [2023-10-07 23:00:12,311][67838] Updated weights for policy 0, policy_version 78642 (0.0009) [2023-10-07 23:00:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 161153024. Throughput: 0: 1667.9, 1: 1666.9. Samples: 40294794. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-07 23:00:12,478][66916] Avg episode reward: [(0, '52.700'), (1, '57.310')] [2023-10-07 23:00:12,688][67838] Updated weights for policy 0, policy_version 78652 (0.0010) [2023-10-07 23:00:15,203][67871] Updated weights for policy 1, policy_version 78760 (0.0007) [2023-10-07 23:00:15,570][67871] Updated weights for policy 1, policy_version 78770 (0.0010) [2023-10-07 23:00:15,943][67871] Updated weights for policy 1, policy_version 78780 (0.0011) [2023-10-07 23:00:16,664][67838] Updated weights for policy 0, policy_version 78662 (0.0008) [2023-10-07 23:00:17,030][67838] Updated weights for policy 0, policy_version 78672 (0.0007) [2023-10-07 23:00:17,400][67838] Updated weights for policy 0, policy_version 78682 (0.0008) [2023-10-07 23:00:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 161218560. Throughput: 0: 1663.2, 1: 1646.6. Samples: 40314162. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-07 23:00:17,477][66916] Avg episode reward: [(0, '50.660'), (1, '55.860')] [2023-10-07 23:00:19,962][67871] Updated weights for policy 1, policy_version 78790 (0.0010) [2023-10-07 23:00:20,334][67871] Updated weights for policy 1, policy_version 78800 (0.0010) [2023-10-07 23:00:20,697][67871] Updated weights for policy 1, policy_version 78810 (0.0009) [2023-10-07 23:00:21,598][67838] Updated weights for policy 0, policy_version 78692 (0.0008) [2023-10-07 23:00:21,973][67838] Updated weights for policy 0, policy_version 78702 (0.0008) [2023-10-07 23:00:22,337][67838] Updated weights for policy 0, policy_version 78712 (0.0009) [2023-10-07 23:00:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 161284096. Throughput: 0: 1646.0, 1: 1660.1. Samples: 40333730. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-07 23:00:22,477][66916] Avg episode reward: [(0, '49.910'), (1, '53.900')] [2023-10-07 23:00:24,923][67871] Updated weights for policy 1, policy_version 78820 (0.0008) [2023-10-07 23:00:25,294][67871] Updated weights for policy 1, policy_version 78830 (0.0009) [2023-10-07 23:00:25,670][67871] Updated weights for policy 1, policy_version 78840 (0.0010) [2023-10-07 23:00:26,390][67838] Updated weights for policy 0, policy_version 78722 (0.0009) [2023-10-07 23:00:26,773][67838] Updated weights for policy 0, policy_version 78732 (0.0007) [2023-10-07 23:00:27,144][67838] Updated weights for policy 0, policy_version 78742 (0.0009) [2023-10-07 23:00:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 161349632. Throughput: 0: 1663.2, 1: 1658.7. Samples: 40344540. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-07 23:00:27,477][66916] Avg episode reward: [(0, '50.920'), (1, '52.240')] [2023-10-07 23:00:27,528][67838] Updated weights for policy 0, policy_version 78752 (0.0008) [2023-10-07 23:00:29,970][67871] Updated weights for policy 1, policy_version 78850 (0.0009) [2023-10-07 23:00:30,337][67871] Updated weights for policy 1, policy_version 78860 (0.0009) [2023-10-07 23:00:30,713][67871] Updated weights for policy 1, policy_version 78870 (0.0009) [2023-10-07 23:00:31,080][67871] Updated weights for policy 1, policy_version 78880 (0.0008) [2023-10-07 23:00:31,650][67838] Updated weights for policy 0, policy_version 78762 (0.0009) [2023-10-07 23:00:32,014][67838] Updated weights for policy 0, policy_version 78772 (0.0008) [2023-10-07 23:00:32,397][67838] Updated weights for policy 0, policy_version 78782 (0.0010) [2023-10-07 23:00:32,477][66916] Fps is (10 sec: 16383.3, 60 sec: 13653.2, 300 sec: 13329.3). Total num frames: 161447936. Throughput: 0: 1661.0, 1: 1648.5. Samples: 40363984. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:00:32,478][66916] Avg episode reward: [(0, '46.410'), (1, '51.310')] [2023-10-07 23:00:35,133][67871] Updated weights for policy 1, policy_version 78890 (0.0008) [2023-10-07 23:00:35,504][67871] Updated weights for policy 1, policy_version 78900 (0.0010) [2023-10-07 23:00:35,865][67871] Updated weights for policy 1, policy_version 78910 (0.0010) [2023-10-07 23:00:36,638][67838] Updated weights for policy 0, policy_version 78792 (0.0010) [2023-10-07 23:00:37,005][67838] Updated weights for policy 0, policy_version 78802 (0.0010) [2023-10-07 23:00:37,386][67838] Updated weights for policy 0, policy_version 78812 (0.0008) [2023-10-07 23:00:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 161480704. Throughput: 0: 1642.0, 1: 1665.4. Samples: 40383262. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:00:37,478][66916] Avg episode reward: [(0, '43.780'), (1, '51.890')] [2023-10-07 23:00:40,227][67871] Updated weights for policy 1, policy_version 78920 (0.0008) [2023-10-07 23:00:40,604][67871] Updated weights for policy 1, policy_version 78930 (0.0008) [2023-10-07 23:00:40,968][67871] Updated weights for policy 1, policy_version 78940 (0.0011) [2023-10-07 23:00:41,591][67838] Updated weights for policy 0, policy_version 78822 (0.0009) [2023-10-07 23:00:41,955][67838] Updated weights for policy 0, policy_version 78832 (0.0009) [2023-10-07 23:00:42,327][67838] Updated weights for policy 0, policy_version 78842 (0.0010) [2023-10-07 23:00:42,476][66916] Fps is (10 sec: 9830.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 161546240. Throughput: 0: 1652.0, 1: 1660.0. Samples: 40393770. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:00:42,477][66916] Avg episode reward: [(0, '42.640'), (1, '56.240')] [2023-10-07 23:00:45,049][67871] Updated weights for policy 1, policy_version 78950 (0.0007) [2023-10-07 23:00:45,424][67871] Updated weights for policy 1, policy_version 78960 (0.0008) [2023-10-07 23:00:45,797][67871] Updated weights for policy 1, policy_version 78970 (0.0010) [2023-10-07 23:00:46,459][67838] Updated weights for policy 0, policy_version 78852 (0.0008) [2023-10-07 23:00:46,831][67838] Updated weights for policy 0, policy_version 78862 (0.0009) [2023-10-07 23:00:47,207][67838] Updated weights for policy 0, policy_version 78872 (0.0008) [2023-10-07 23:00:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 161611776. Throughput: 0: 1649.2, 1: 1645.8. Samples: 40413112. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:00:47,477][66916] Avg episode reward: [(0, '41.710'), (1, '58.170')] [2023-10-07 23:00:50,011][67871] Updated weights for policy 1, policy_version 78980 (0.0007) [2023-10-07 23:00:50,370][67871] Updated weights for policy 1, policy_version 78990 (0.0007) [2023-10-07 23:00:50,743][67871] Updated weights for policy 1, policy_version 79000 (0.0007) [2023-10-07 23:00:51,498][67838] Updated weights for policy 0, policy_version 78882 (0.0007) [2023-10-07 23:00:51,874][67838] Updated weights for policy 0, policy_version 78892 (0.0007) [2023-10-07 23:00:52,250][67838] Updated weights for policy 0, policy_version 78902 (0.0007) [2023-10-07 23:00:52,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 161677312. Throughput: 0: 1643.1, 1: 1652.6. Samples: 40432640. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:00:52,477][66916] Avg episode reward: [(0, '48.390'), (1, '59.730')] [2023-10-07 23:00:52,618][67838] Updated weights for policy 0, policy_version 78912 (0.0008) [2023-10-07 23:00:55,013][67871] Updated weights for policy 1, policy_version 79010 (0.0007) [2023-10-07 23:00:55,378][67871] Updated weights for policy 1, policy_version 79020 (0.0008) [2023-10-07 23:00:55,738][67871] Updated weights for policy 1, policy_version 79030 (0.0009) [2023-10-07 23:00:56,101][67871] Updated weights for policy 1, policy_version 79040 (0.0007) [2023-10-07 23:00:56,718][67838] Updated weights for policy 0, policy_version 78922 (0.0009) [2023-10-07 23:00:57,098][67838] Updated weights for policy 0, policy_version 78932 (0.0009) [2023-10-07 23:00:57,463][67838] Updated weights for policy 0, policy_version 78942 (0.0007) [2023-10-07 23:00:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 161742848. Throughput: 0: 1650.4, 1: 1651.0. Samples: 40443356. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:00:57,478][66916] Avg episode reward: [(0, '44.180'), (1, '59.240')] [2023-10-07 23:01:00,418][67871] Updated weights for policy 1, policy_version 79050 (0.0007) [2023-10-07 23:01:00,781][67871] Updated weights for policy 1, policy_version 79060 (0.0008) [2023-10-07 23:01:01,147][67871] Updated weights for policy 1, policy_version 79070 (0.0007) [2023-10-07 23:01:01,795][67838] Updated weights for policy 0, policy_version 78952 (0.0007) [2023-10-07 23:01:02,164][67838] Updated weights for policy 0, policy_version 78962 (0.0007) [2023-10-07 23:01:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 161808384. Throughput: 0: 1651.4, 1: 1649.6. Samples: 40462708. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:01:02,477][66916] Avg episode reward: [(0, '48.280'), (1, '57.680')] [2023-10-07 23:01:02,539][67838] Updated weights for policy 0, policy_version 78972 (0.0007) [2023-10-07 23:01:05,320][67871] Updated weights for policy 1, policy_version 79080 (0.0008) [2023-10-07 23:01:05,692][67871] Updated weights for policy 1, policy_version 79090 (0.0010) [2023-10-07 23:01:06,052][67871] Updated weights for policy 1, policy_version 79100 (0.0008) [2023-10-07 23:01:06,575][67838] Updated weights for policy 0, policy_version 78982 (0.0007) [2023-10-07 23:01:06,952][67838] Updated weights for policy 0, policy_version 78992 (0.0008) [2023-10-07 23:01:07,315][67838] Updated weights for policy 0, policy_version 79002 (0.0011) [2023-10-07 23:01:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 161873920. Throughput: 0: 1653.0, 1: 1646.3. Samples: 40482200. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:01:07,477][66916] Avg episode reward: [(0, '48.780'), (1, '52.940')] [2023-10-07 23:01:07,484][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000079104_81002496.pth... [2023-10-07 23:01:07,518][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000077536_79396864.pth [2023-10-07 23:01:07,533][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000079008_80904192.pth... [2023-10-07 23:01:07,574][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000077440_79298560.pth [2023-10-07 23:01:09,993][67871] Updated weights for policy 1, policy_version 79110 (0.0010) [2023-10-07 23:01:10,364][67871] Updated weights for policy 1, policy_version 79120 (0.0011) [2023-10-07 23:01:10,726][67871] Updated weights for policy 1, policy_version 79130 (0.0010) [2023-10-07 23:01:11,465][67838] Updated weights for policy 0, policy_version 79012 (0.0008) [2023-10-07 23:01:11,846][67838] Updated weights for policy 0, policy_version 79022 (0.0007) [2023-10-07 23:01:12,220][67838] Updated weights for policy 0, policy_version 79032 (0.0009) [2023-10-07 23:01:12,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 161939456. Throughput: 0: 1657.0, 1: 1647.4. Samples: 40493240. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:01:12,478][66916] Avg episode reward: [(0, '48.610'), (1, '53.110')] [2023-10-07 23:01:14,778][67871] Updated weights for policy 1, policy_version 79140 (0.0010) [2023-10-07 23:01:15,146][67871] Updated weights for policy 1, policy_version 79150 (0.0007) [2023-10-07 23:01:15,514][67871] Updated weights for policy 1, policy_version 79160 (0.0009) [2023-10-07 23:01:16,340][67838] Updated weights for policy 0, policy_version 79042 (0.0011) [2023-10-07 23:01:16,715][67838] Updated weights for policy 0, policy_version 79052 (0.0009) [2023-10-07 23:01:17,081][67838] Updated weights for policy 0, policy_version 79062 (0.0009) [2023-10-07 23:01:17,452][67838] Updated weights for policy 0, policy_version 79072 (0.0009) [2023-10-07 23:01:17,476][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 162037760. Throughput: 0: 1655.2, 1: 1644.8. Samples: 40512482. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:01:17,477][66916] Avg episode reward: [(0, '47.360'), (1, '54.090')] [2023-10-07 23:01:19,552][67871] Updated weights for policy 1, policy_version 79170 (0.0009) [2023-10-07 23:01:19,933][67871] Updated weights for policy 1, policy_version 79180 (0.0009) [2023-10-07 23:01:20,301][67871] Updated weights for policy 1, policy_version 79190 (0.0010) [2023-10-07 23:01:20,661][67871] Updated weights for policy 1, policy_version 79200 (0.0008) [2023-10-07 23:01:21,488][67838] Updated weights for policy 0, policy_version 79082 (0.0007) [2023-10-07 23:01:21,854][67838] Updated weights for policy 0, policy_version 79092 (0.0007) [2023-10-07 23:01:22,231][67838] Updated weights for policy 0, policy_version 79102 (0.0007) [2023-10-07 23:01:22,476][66916] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 162103296. Throughput: 0: 1656.9, 1: 1652.4. Samples: 40532178. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 23:01:22,477][66916] Avg episode reward: [(0, '50.560'), (1, '54.780')] [2023-10-07 23:01:24,953][67871] Updated weights for policy 1, policy_version 79210 (0.0009) [2023-10-07 23:01:25,328][67871] Updated weights for policy 1, policy_version 79220 (0.0007) [2023-10-07 23:01:25,697][67871] Updated weights for policy 1, policy_version 79230 (0.0008) [2023-10-07 23:01:26,305][67838] Updated weights for policy 0, policy_version 79112 (0.0008) [2023-10-07 23:01:26,670][67838] Updated weights for policy 0, policy_version 79122 (0.0008) [2023-10-07 23:01:27,051][67838] Updated weights for policy 0, policy_version 79132 (0.0010) [2023-10-07 23:01:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 162168832. Throughput: 0: 1665.8, 1: 1651.8. Samples: 40543062. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 23:01:27,477][66916] Avg episode reward: [(0, '50.210'), (1, '56.590')] [2023-10-07 23:01:29,706][67871] Updated weights for policy 1, policy_version 79240 (0.0010) [2023-10-07 23:01:30,069][67871] Updated weights for policy 1, policy_version 79250 (0.0009) [2023-10-07 23:01:30,443][67871] Updated weights for policy 1, policy_version 79260 (0.0011) [2023-10-07 23:01:31,122][67838] Updated weights for policy 0, policy_version 79142 (0.0009) [2023-10-07 23:01:31,492][67838] Updated weights for policy 0, policy_version 79152 (0.0008) [2023-10-07 23:01:31,862][67838] Updated weights for policy 0, policy_version 79162 (0.0007) [2023-10-07 23:01:32,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 162234368. Throughput: 0: 1662.0, 1: 1654.5. Samples: 40562352. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 23:01:32,477][66916] Avg episode reward: [(0, '49.870'), (1, '57.160')] [2023-10-07 23:01:34,708][67871] Updated weights for policy 1, policy_version 79270 (0.0011) [2023-10-07 23:01:35,073][67871] Updated weights for policy 1, policy_version 79280 (0.0008) [2023-10-07 23:01:35,430][67871] Updated weights for policy 1, policy_version 79290 (0.0011) [2023-10-07 23:01:35,928][67838] Updated weights for policy 0, policy_version 79172 (0.0007) [2023-10-07 23:01:36,294][67838] Updated weights for policy 0, policy_version 79182 (0.0008) [2023-10-07 23:01:36,664][67838] Updated weights for policy 0, policy_version 79192 (0.0010) [2023-10-07 23:01:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 162299904. Throughput: 0: 1653.1, 1: 1656.5. Samples: 40581572. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 23:01:37,477][66916] Avg episode reward: [(0, '51.410'), (1, '59.530')] [2023-10-07 23:01:39,569][67871] Updated weights for policy 1, policy_version 79300 (0.0010) [2023-10-07 23:01:39,944][67871] Updated weights for policy 1, policy_version 79310 (0.0010) [2023-10-07 23:01:40,309][67871] Updated weights for policy 1, policy_version 79320 (0.0010) [2023-10-07 23:01:40,760][67838] Updated weights for policy 0, policy_version 79202 (0.0010) [2023-10-07 23:01:41,130][67838] Updated weights for policy 0, policy_version 79212 (0.0008) [2023-10-07 23:01:41,494][67838] Updated weights for policy 0, policy_version 79222 (0.0007) [2023-10-07 23:01:41,861][67838] Updated weights for policy 0, policy_version 79232 (0.0007) [2023-10-07 23:01:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 162365440. Throughput: 0: 1667.4, 1: 1653.3. Samples: 40592786. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 23:01:42,478][66916] Avg episode reward: [(0, '48.410'), (1, '60.490')] [2023-10-07 23:01:44,314][67871] Updated weights for policy 1, policy_version 79330 (0.0009) [2023-10-07 23:01:44,681][67871] Updated weights for policy 1, policy_version 79340 (0.0009) [2023-10-07 23:01:45,047][67871] Updated weights for policy 1, policy_version 79350 (0.0008) [2023-10-07 23:01:45,411][67871] Updated weights for policy 1, policy_version 79360 (0.0008) [2023-10-07 23:01:46,141][67838] Updated weights for policy 0, policy_version 79242 (0.0009) [2023-10-07 23:01:46,515][67838] Updated weights for policy 0, policy_version 79252 (0.0008) [2023-10-07 23:01:46,884][67838] Updated weights for policy 0, policy_version 79262 (0.0009) [2023-10-07 23:01:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 162430976. Throughput: 0: 1659.5, 1: 1661.0. Samples: 40612132. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 23:01:47,477][66916] Avg episode reward: [(0, '52.680'), (1, '58.240')] [2023-10-07 23:01:49,428][67871] Updated weights for policy 1, policy_version 79370 (0.0009) [2023-10-07 23:01:49,800][67871] Updated weights for policy 1, policy_version 79380 (0.0009) [2023-10-07 23:01:50,159][67871] Updated weights for policy 1, policy_version 79390 (0.0009) [2023-10-07 23:01:50,998][67838] Updated weights for policy 0, policy_version 79272 (0.0010) [2023-10-07 23:01:51,364][67838] Updated weights for policy 0, policy_version 79282 (0.0011) [2023-10-07 23:01:51,740][67838] Updated weights for policy 0, policy_version 79292 (0.0008) [2023-10-07 23:01:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 162496512. Throughput: 0: 1651.1, 1: 1674.4. Samples: 40631850. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 23:01:52,477][66916] Avg episode reward: [(0, '47.700'), (1, '58.550')] [2023-10-07 23:01:54,199][67871] Updated weights for policy 1, policy_version 79400 (0.0009) [2023-10-07 23:01:54,579][67871] Updated weights for policy 1, policy_version 79410 (0.0010) [2023-10-07 23:01:54,947][67871] Updated weights for policy 1, policy_version 79420 (0.0009) [2023-10-07 23:01:55,918][67838] Updated weights for policy 0, policy_version 79302 (0.0008) [2023-10-07 23:01:56,300][67838] Updated weights for policy 0, policy_version 79312 (0.0007) [2023-10-07 23:01:56,661][67838] Updated weights for policy 0, policy_version 79322 (0.0008) [2023-10-07 23:01:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 162562048. Throughput: 0: 1661.3, 1: 1654.5. Samples: 40642452. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 23:01:57,478][66916] Avg episode reward: [(0, '48.980'), (1, '59.280')] [2023-10-07 23:01:59,095][67871] Updated weights for policy 1, policy_version 79430 (0.0009) [2023-10-07 23:01:59,461][67871] Updated weights for policy 1, policy_version 79440 (0.0009) [2023-10-07 23:01:59,824][67871] Updated weights for policy 1, policy_version 79450 (0.0009) [2023-10-07 23:02:00,732][67838] Updated weights for policy 0, policy_version 79332 (0.0007) [2023-10-07 23:02:01,100][67838] Updated weights for policy 0, policy_version 79342 (0.0007) [2023-10-07 23:02:01,466][67838] Updated weights for policy 0, policy_version 79352 (0.0008) [2023-10-07 23:02:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 162627584. Throughput: 0: 1651.1, 1: 1671.5. Samples: 40661998. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 23:02:02,478][66916] Avg episode reward: [(0, '48.730'), (1, '61.260')] [2023-10-07 23:02:03,881][67871] Updated weights for policy 1, policy_version 79460 (0.0009) [2023-10-07 23:02:04,244][67871] Updated weights for policy 1, policy_version 79470 (0.0008) [2023-10-07 23:02:04,599][67871] Updated weights for policy 1, policy_version 79480 (0.0007) [2023-10-07 23:02:05,603][67838] Updated weights for policy 0, policy_version 79362 (0.0010) [2023-10-07 23:02:05,973][67838] Updated weights for policy 0, policy_version 79372 (0.0007) [2023-10-07 23:02:06,354][67838] Updated weights for policy 0, policy_version 79382 (0.0007) [2023-10-07 23:02:06,729][67838] Updated weights for policy 0, policy_version 79392 (0.0008) [2023-10-07 23:02:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 162693120. Throughput: 0: 1649.6, 1: 1677.6. Samples: 40681904. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-07 23:02:07,477][66916] Avg episode reward: [(0, '50.840'), (1, '60.850')] [2023-10-07 23:02:08,734][67871] Updated weights for policy 1, policy_version 79490 (0.0007) [2023-10-07 23:02:09,092][67871] Updated weights for policy 1, policy_version 79500 (0.0007) [2023-10-07 23:02:09,463][67871] Updated weights for policy 1, policy_version 79510 (0.0007) [2023-10-07 23:02:09,832][67871] Updated weights for policy 1, policy_version 79520 (0.0008) [2023-10-07 23:02:10,838][67838] Updated weights for policy 0, policy_version 79402 (0.0009) [2023-10-07 23:02:11,205][67838] Updated weights for policy 0, policy_version 79412 (0.0008) [2023-10-07 23:02:11,582][67838] Updated weights for policy 0, policy_version 79422 (0.0008) [2023-10-07 23:02:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 162758656. Throughput: 0: 1657.5, 1: 1658.3. Samples: 40692272. Policy #0 lag: (min: 18.0, avg: 18.1, max: 23.0) [2023-10-07 23:02:12,478][66916] Avg episode reward: [(0, '50.870'), (1, '61.150')] [2023-10-07 23:02:13,916][67871] Updated weights for policy 1, policy_version 79530 (0.0010) [2023-10-07 23:02:14,297][67871] Updated weights for policy 1, policy_version 79540 (0.0011) [2023-10-07 23:02:14,659][67871] Updated weights for policy 1, policy_version 79550 (0.0007) [2023-10-07 23:02:15,667][67838] Updated weights for policy 0, policy_version 79432 (0.0008) [2023-10-07 23:02:16,040][67838] Updated weights for policy 0, policy_version 79442 (0.0008) [2023-10-07 23:02:16,417][67838] Updated weights for policy 0, policy_version 79452 (0.0010) [2023-10-07 23:02:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 162824192. Throughput: 0: 1645.1, 1: 1683.0. Samples: 40712114. Policy #0 lag: (min: 18.0, avg: 18.1, max: 23.0) [2023-10-07 23:02:17,478][66916] Avg episode reward: [(0, '51.640'), (1, '56.660')] [2023-10-07 23:02:18,828][67871] Updated weights for policy 1, policy_version 79560 (0.0009) [2023-10-07 23:02:19,189][67871] Updated weights for policy 1, policy_version 79570 (0.0009) [2023-10-07 23:02:19,561][67871] Updated weights for policy 1, policy_version 79580 (0.0010) [2023-10-07 23:02:20,611][67838] Updated weights for policy 0, policy_version 79462 (0.0008) [2023-10-07 23:02:20,985][67838] Updated weights for policy 0, policy_version 79472 (0.0008) [2023-10-07 23:02:21,359][67838] Updated weights for policy 0, policy_version 79482 (0.0008) [2023-10-07 23:02:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 162889728. Throughput: 0: 1651.9, 1: 1684.0. Samples: 40731686. Policy #0 lag: (min: 18.0, avg: 18.1, max: 23.0) [2023-10-07 23:02:22,477][66916] Avg episode reward: [(0, '50.010'), (1, '56.020')] [2023-10-07 23:02:23,635][67871] Updated weights for policy 1, policy_version 79590 (0.0009) [2023-10-07 23:02:23,997][67871] Updated weights for policy 1, policy_version 79600 (0.0010) [2023-10-07 23:02:24,368][67871] Updated weights for policy 1, policy_version 79610 (0.0010) [2023-10-07 23:02:25,413][67838] Updated weights for policy 0, policy_version 79492 (0.0009) [2023-10-07 23:02:25,782][67838] Updated weights for policy 0, policy_version 79502 (0.0009) [2023-10-07 23:02:26,160][67838] Updated weights for policy 0, policy_version 79512 (0.0009) [2023-10-07 23:02:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 162955264. Throughput: 0: 1652.6, 1: 1660.9. Samples: 40741892. Policy #0 lag: (min: 18.0, avg: 18.1, max: 23.0) [2023-10-07 23:02:27,477][66916] Avg episode reward: [(0, '46.630'), (1, '54.350')] [2023-10-07 23:02:28,581][67871] Updated weights for policy 1, policy_version 79620 (0.0010) [2023-10-07 23:02:28,937][67871] Updated weights for policy 1, policy_version 79630 (0.0008) [2023-10-07 23:02:29,317][67871] Updated weights for policy 1, policy_version 79640 (0.0009) [2023-10-07 23:02:30,273][67838] Updated weights for policy 0, policy_version 79522 (0.0008) [2023-10-07 23:02:30,653][67838] Updated weights for policy 0, policy_version 79532 (0.0008) [2023-10-07 23:02:31,013][67838] Updated weights for policy 0, policy_version 79542 (0.0009) [2023-10-07 23:02:31,387][67838] Updated weights for policy 0, policy_version 79552 (0.0009) [2023-10-07 23:02:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163020800. Throughput: 0: 1645.3, 1: 1675.8. Samples: 40761580. Policy #0 lag: (min: 18.0, avg: 18.1, max: 23.0) [2023-10-07 23:02:32,477][66916] Avg episode reward: [(0, '45.280'), (1, '54.480')] [2023-10-07 23:02:33,453][67871] Updated weights for policy 1, policy_version 79650 (0.0009) [2023-10-07 23:02:33,821][67871] Updated weights for policy 1, policy_version 79660 (0.0009) [2023-10-07 23:02:34,184][67871] Updated weights for policy 1, policy_version 79670 (0.0009) [2023-10-07 23:02:34,550][67871] Updated weights for policy 1, policy_version 79680 (0.0010) [2023-10-07 23:02:35,456][67838] Updated weights for policy 0, policy_version 79562 (0.0007) [2023-10-07 23:02:35,827][67838] Updated weights for policy 0, policy_version 79572 (0.0007) [2023-10-07 23:02:36,189][67838] Updated weights for policy 0, policy_version 79582 (0.0009) [2023-10-07 23:02:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163086336. Throughput: 0: 1660.5, 1: 1670.1. Samples: 40781728. Policy #0 lag: (min: 18.0, avg: 18.1, max: 23.0) [2023-10-07 23:02:37,477][66916] Avg episode reward: [(0, '45.150'), (1, '55.800')] [2023-10-07 23:02:38,686][67871] Updated weights for policy 1, policy_version 79690 (0.0008) [2023-10-07 23:02:39,047][67871] Updated weights for policy 1, policy_version 79700 (0.0010) [2023-10-07 23:02:39,429][67871] Updated weights for policy 1, policy_version 79710 (0.0009) [2023-10-07 23:02:40,241][67838] Updated weights for policy 0, policy_version 79592 (0.0009) [2023-10-07 23:02:40,607][67838] Updated weights for policy 0, policy_version 79602 (0.0011) [2023-10-07 23:02:40,982][67838] Updated weights for policy 0, policy_version 79612 (0.0011) [2023-10-07 23:02:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 163151872. Throughput: 0: 1660.2, 1: 1662.5. Samples: 40791972. Policy #0 lag: (min: 18.0, avg: 18.1, max: 23.0) [2023-10-07 23:02:42,478][66916] Avg episode reward: [(0, '41.640'), (1, '57.970')] [2023-10-07 23:02:43,382][67871] Updated weights for policy 1, policy_version 79720 (0.0009) [2023-10-07 23:02:43,752][67871] Updated weights for policy 1, policy_version 79730 (0.0009) [2023-10-07 23:02:44,123][67871] Updated weights for policy 1, policy_version 79740 (0.0009) [2023-10-07 23:02:45,034][67838] Updated weights for policy 0, policy_version 79622 (0.0011) [2023-10-07 23:02:45,413][67838] Updated weights for policy 0, policy_version 79632 (0.0007) [2023-10-07 23:02:45,794][67838] Updated weights for policy 0, policy_version 79642 (0.0008) [2023-10-07 23:02:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163217408. Throughput: 0: 1647.3, 1: 1672.0. Samples: 40811362. Policy #0 lag: (min: 18.0, avg: 18.1, max: 23.0) [2023-10-07 23:02:47,477][66916] Avg episode reward: [(0, '46.750'), (1, '54.460')] [2023-10-07 23:02:48,289][67871] Updated weights for policy 1, policy_version 79750 (0.0009) [2023-10-07 23:02:48,663][67871] Updated weights for policy 1, policy_version 79760 (0.0008) [2023-10-07 23:02:49,032][67871] Updated weights for policy 1, policy_version 79770 (0.0007) [2023-10-07 23:02:50,019][67838] Updated weights for policy 0, policy_version 79652 (0.0008) [2023-10-07 23:02:50,393][67838] Updated weights for policy 0, policy_version 79662 (0.0010) [2023-10-07 23:02:50,760][67838] Updated weights for policy 0, policy_version 79672 (0.0008) [2023-10-07 23:02:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 163282944. Throughput: 0: 1661.8, 1: 1662.0. Samples: 40831478. Policy #0 lag: (min: 18.0, avg: 18.1, max: 23.0) [2023-10-07 23:02:52,478][66916] Avg episode reward: [(0, '43.930'), (1, '49.760')] [2023-10-07 23:02:53,090][67871] Updated weights for policy 1, policy_version 79780 (0.0009) [2023-10-07 23:02:53,464][67871] Updated weights for policy 1, policy_version 79790 (0.0008) [2023-10-07 23:02:53,841][67871] Updated weights for policy 1, policy_version 79800 (0.0009) [2023-10-07 23:02:54,866][67838] Updated weights for policy 0, policy_version 79682 (0.0008) [2023-10-07 23:02:55,251][67838] Updated weights for policy 0, policy_version 79692 (0.0010) [2023-10-07 23:02:55,619][67838] Updated weights for policy 0, policy_version 79702 (0.0009) [2023-10-07 23:02:55,994][67838] Updated weights for policy 0, policy_version 79712 (0.0009) [2023-10-07 23:02:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163348480. Throughput: 0: 1654.4, 1: 1659.0. Samples: 40841378. Policy #0 lag: (min: 18.0, avg: 18.1, max: 23.0) [2023-10-07 23:02:57,477][66916] Avg episode reward: [(0, '44.810'), (1, '47.610')] [2023-10-07 23:02:58,034][67871] Updated weights for policy 1, policy_version 79810 (0.0008) [2023-10-07 23:02:58,397][67871] Updated weights for policy 1, policy_version 79820 (0.0010) [2023-10-07 23:02:58,774][67871] Updated weights for policy 1, policy_version 79830 (0.0009) [2023-10-07 23:02:59,138][67871] Updated weights for policy 1, policy_version 79840 (0.0008) [2023-10-07 23:03:00,241][67838] Updated weights for policy 0, policy_version 79722 (0.0009) [2023-10-07 23:03:00,610][67838] Updated weights for policy 0, policy_version 79732 (0.0010) [2023-10-07 23:03:00,991][67838] Updated weights for policy 0, policy_version 79742 (0.0010) [2023-10-07 23:03:02,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 163414016. Throughput: 0: 1645.6, 1: 1654.9. Samples: 40860636. Policy #0 lag: (min: 29.0, avg: 32.3, max: 61.0) [2023-10-07 23:03:02,478][66916] Avg episode reward: [(0, '45.950'), (1, '49.440')] [2023-10-07 23:03:03,335][67871] Updated weights for policy 1, policy_version 79850 (0.0008) [2023-10-07 23:03:03,691][67871] Updated weights for policy 1, policy_version 79860 (0.0009) [2023-10-07 23:03:04,067][67871] Updated weights for policy 1, policy_version 79870 (0.0009) [2023-10-07 23:03:05,174][67838] Updated weights for policy 0, policy_version 79752 (0.0008) [2023-10-07 23:03:05,544][67838] Updated weights for policy 0, policy_version 79762 (0.0010) [2023-10-07 23:03:05,914][67838] Updated weights for policy 0, policy_version 79772 (0.0009) [2023-10-07 23:03:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163479552. Throughput: 0: 1658.6, 1: 1665.2. Samples: 40881254. Policy #0 lag: (min: 29.0, avg: 32.3, max: 61.0) [2023-10-07 23:03:07,477][66916] Avg episode reward: [(0, '43.590'), (1, '53.740')] [2023-10-07 23:03:07,486][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000079872_81788928.pth... [2023-10-07 23:03:07,486][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000079776_81690624.pth... [2023-10-07 23:03:07,518][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000078336_80216064.pth [2023-10-07 23:03:07,527][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000078208_80084992.pth [2023-10-07 23:03:07,970][67871] Updated weights for policy 1, policy_version 79880 (0.0008) [2023-10-07 23:03:08,339][67871] Updated weights for policy 1, policy_version 79890 (0.0008) [2023-10-07 23:03:08,706][67871] Updated weights for policy 1, policy_version 79900 (0.0010) [2023-10-07 23:03:10,187][67838] Updated weights for policy 0, policy_version 79782 (0.0008) [2023-10-07 23:03:10,554][67838] Updated weights for policy 0, policy_version 79792 (0.0010) [2023-10-07 23:03:10,926][67838] Updated weights for policy 0, policy_version 79802 (0.0009) [2023-10-07 23:03:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163545088. Throughput: 0: 1654.8, 1: 1667.0. Samples: 40891370. Policy #0 lag: (min: 29.0, avg: 32.3, max: 61.0) [2023-10-07 23:03:12,477][66916] Avg episode reward: [(0, '46.410'), (1, '58.260')] [2023-10-07 23:03:12,798][67871] Updated weights for policy 1, policy_version 79910 (0.0008) [2023-10-07 23:03:13,158][67871] Updated weights for policy 1, policy_version 79920 (0.0009) [2023-10-07 23:03:13,517][67871] Updated weights for policy 1, policy_version 79930 (0.0007) [2023-10-07 23:03:15,103][67838] Updated weights for policy 0, policy_version 79812 (0.0009) [2023-10-07 23:03:15,473][67838] Updated weights for policy 0, policy_version 79822 (0.0010) [2023-10-07 23:03:15,849][67838] Updated weights for policy 0, policy_version 79832 (0.0007) [2023-10-07 23:03:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 163610624. Throughput: 0: 1646.9, 1: 1666.5. Samples: 40910686. Policy #0 lag: (min: 29.0, avg: 32.3, max: 61.0) [2023-10-07 23:03:17,478][66916] Avg episode reward: [(0, '44.680'), (1, '58.890')] [2023-10-07 23:03:17,698][67871] Updated weights for policy 1, policy_version 79940 (0.0008) [2023-10-07 23:03:18,054][67871] Updated weights for policy 1, policy_version 79950 (0.0009) [2023-10-07 23:03:18,430][67871] Updated weights for policy 1, policy_version 79960 (0.0010) [2023-10-07 23:03:19,993][67838] Updated weights for policy 0, policy_version 79842 (0.0007) [2023-10-07 23:03:20,357][67838] Updated weights for policy 0, policy_version 79852 (0.0007) [2023-10-07 23:03:20,731][67838] Updated weights for policy 0, policy_version 79862 (0.0007) [2023-10-07 23:03:21,106][67838] Updated weights for policy 0, policy_version 79872 (0.0010) [2023-10-07 23:03:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163676160. Throughput: 0: 1653.4, 1: 1671.3. Samples: 40931340. Policy #0 lag: (min: 29.0, avg: 32.3, max: 61.0) [2023-10-07 23:03:22,477][66916] Avg episode reward: [(0, '47.780'), (1, '62.240')] [2023-10-07 23:03:22,534][67871] Updated weights for policy 1, policy_version 79970 (0.0010) [2023-10-07 23:03:22,896][67871] Updated weights for policy 1, policy_version 79980 (0.0008) [2023-10-07 23:03:23,257][67871] Updated weights for policy 1, policy_version 79990 (0.0010) [2023-10-07 23:03:23,623][67871] Updated weights for policy 1, policy_version 80000 (0.0008) [2023-10-07 23:03:25,134][67838] Updated weights for policy 0, policy_version 79882 (0.0008) [2023-10-07 23:03:25,506][67838] Updated weights for policy 0, policy_version 79892 (0.0008) [2023-10-07 23:03:25,876][67838] Updated weights for policy 0, policy_version 79902 (0.0009) [2023-10-07 23:03:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163741696. Throughput: 0: 1651.7, 1: 1673.0. Samples: 40941580. Policy #0 lag: (min: 29.0, avg: 32.3, max: 61.0) [2023-10-07 23:03:27,477][66916] Avg episode reward: [(0, '46.880'), (1, '61.140')] [2023-10-07 23:03:27,607][67871] Updated weights for policy 1, policy_version 80010 (0.0007) [2023-10-07 23:03:27,971][67871] Updated weights for policy 1, policy_version 80020 (0.0009) [2023-10-07 23:03:28,337][67871] Updated weights for policy 1, policy_version 80030 (0.0008) [2023-10-07 23:03:30,066][67838] Updated weights for policy 0, policy_version 79912 (0.0010) [2023-10-07 23:03:30,441][67838] Updated weights for policy 0, policy_version 79922 (0.0008) [2023-10-07 23:03:30,807][67838] Updated weights for policy 0, policy_version 79932 (0.0010) [2023-10-07 23:03:32,437][67871] Updated weights for policy 1, policy_version 80040 (0.0007) [2023-10-07 23:03:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 163807232. Throughput: 0: 1654.7, 1: 1669.2. Samples: 40960938. Policy #0 lag: (min: 29.0, avg: 32.3, max: 61.0) [2023-10-07 23:03:32,478][66916] Avg episode reward: [(0, '48.830'), (1, '61.480')] [2023-10-07 23:03:32,808][67871] Updated weights for policy 1, policy_version 80050 (0.0007) [2023-10-07 23:03:33,170][67871] Updated weights for policy 1, policy_version 80060 (0.0008) [2023-10-07 23:03:34,943][67838] Updated weights for policy 0, policy_version 79942 (0.0007) [2023-10-07 23:03:35,321][67838] Updated weights for policy 0, policy_version 79952 (0.0008) [2023-10-07 23:03:35,684][67838] Updated weights for policy 0, policy_version 79962 (0.0009) [2023-10-07 23:03:37,335][67871] Updated weights for policy 1, policy_version 80070 (0.0008) [2023-10-07 23:03:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163872768. Throughput: 0: 1659.2, 1: 1668.8. Samples: 40981236. Policy #0 lag: (min: 29.0, avg: 32.3, max: 61.0) [2023-10-07 23:03:37,477][66916] Avg episode reward: [(0, '48.290'), (1, '57.440')] [2023-10-07 23:03:37,702][67871] Updated weights for policy 1, policy_version 80080 (0.0008) [2023-10-07 23:03:38,064][67871] Updated weights for policy 1, policy_version 80090 (0.0009) [2023-10-07 23:03:39,632][67838] Updated weights for policy 0, policy_version 79972 (0.0008) [2023-10-07 23:03:40,007][67838] Updated weights for policy 0, policy_version 79982 (0.0009) [2023-10-07 23:03:40,380][67838] Updated weights for policy 0, policy_version 79992 (0.0009) [2023-10-07 23:03:42,256][67871] Updated weights for policy 1, policy_version 80100 (0.0010) [2023-10-07 23:03:42,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 163938304. Throughput: 0: 1654.7, 1: 1672.6. Samples: 40991108. Policy #0 lag: (min: 29.0, avg: 32.3, max: 61.0) [2023-10-07 23:03:42,477][66916] Avg episode reward: [(0, '45.150'), (1, '55.580')] [2023-10-07 23:03:42,628][67871] Updated weights for policy 1, policy_version 80110 (0.0010) [2023-10-07 23:03:42,987][67871] Updated weights for policy 1, policy_version 80120 (0.0010) [2023-10-07 23:03:44,493][67838] Updated weights for policy 0, policy_version 80002 (0.0008) [2023-10-07 23:03:44,869][67838] Updated weights for policy 0, policy_version 80012 (0.0010) [2023-10-07 23:03:45,242][67838] Updated weights for policy 0, policy_version 80022 (0.0008) [2023-10-07 23:03:45,610][67838] Updated weights for policy 0, policy_version 80032 (0.0008) [2023-10-07 23:03:47,191][67871] Updated weights for policy 1, policy_version 80130 (0.0010) [2023-10-07 23:03:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 164003840. Throughput: 0: 1664.3, 1: 1670.3. Samples: 41010692. Policy #0 lag: (min: 29.0, avg: 32.3, max: 61.0) [2023-10-07 23:03:47,477][66916] Avg episode reward: [(0, '46.600'), (1, '58.310')] [2023-10-07 23:03:47,567][67871] Updated weights for policy 1, policy_version 80140 (0.0008) [2023-10-07 23:03:47,932][67871] Updated weights for policy 1, policy_version 80150 (0.0008) [2023-10-07 23:03:48,303][67871] Updated weights for policy 1, policy_version 80160 (0.0008) [2023-10-07 23:03:49,772][67838] Updated weights for policy 0, policy_version 80042 (0.0008) [2023-10-07 23:03:50,138][67838] Updated weights for policy 0, policy_version 80052 (0.0007) [2023-10-07 23:03:50,511][67838] Updated weights for policy 0, policy_version 80062 (0.0007) [2023-10-07 23:03:52,424][67871] Updated weights for policy 1, policy_version 80170 (0.0009) [2023-10-07 23:03:52,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 164069376. Throughput: 0: 1671.5, 1: 1661.4. Samples: 41031232. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 23:03:52,478][66916] Avg episode reward: [(0, '44.700'), (1, '53.060')] [2023-10-07 23:03:52,786][67871] Updated weights for policy 1, policy_version 80180 (0.0008) [2023-10-07 23:03:53,153][67871] Updated weights for policy 1, policy_version 80190 (0.0008) [2023-10-07 23:03:54,517][67838] Updated weights for policy 0, policy_version 80072 (0.0010) [2023-10-07 23:03:54,890][67838] Updated weights for policy 0, policy_version 80082 (0.0010) [2023-10-07 23:03:55,269][67838] Updated weights for policy 0, policy_version 80092 (0.0010) [2023-10-07 23:03:57,408][67871] Updated weights for policy 1, policy_version 80200 (0.0009) [2023-10-07 23:03:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 164134912. Throughput: 0: 1660.4, 1: 1661.0. Samples: 41040834. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 23:03:57,478][66916] Avg episode reward: [(0, '48.880'), (1, '52.960')] [2023-10-07 23:03:57,782][67871] Updated weights for policy 1, policy_version 80210 (0.0007) [2023-10-07 23:03:58,142][67871] Updated weights for policy 1, policy_version 80220 (0.0008) [2023-10-07 23:03:59,311][67838] Updated weights for policy 0, policy_version 80102 (0.0009) [2023-10-07 23:03:59,681][67838] Updated weights for policy 0, policy_version 80112 (0.0009) [2023-10-07 23:04:00,059][67838] Updated weights for policy 0, policy_version 80122 (0.0009) [2023-10-07 23:04:02,246][67871] Updated weights for policy 1, policy_version 80230 (0.0009) [2023-10-07 23:04:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164200448. Throughput: 0: 1675.9, 1: 1657.4. Samples: 41060684. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 23:04:02,478][66916] Avg episode reward: [(0, '46.180'), (1, '56.270')] [2023-10-07 23:04:02,625][67871] Updated weights for policy 1, policy_version 80240 (0.0009) [2023-10-07 23:04:02,989][67871] Updated weights for policy 1, policy_version 80250 (0.0007) [2023-10-07 23:04:04,080][67838] Updated weights for policy 0, policy_version 80132 (0.0008) [2023-10-07 23:04:04,458][67838] Updated weights for policy 0, policy_version 80142 (0.0007) [2023-10-07 23:04:04,823][67838] Updated weights for policy 0, policy_version 80152 (0.0010) [2023-10-07 23:04:07,240][67871] Updated weights for policy 1, policy_version 80260 (0.0007) [2023-10-07 23:04:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 164265984. Throughput: 0: 1679.1, 1: 1651.9. Samples: 41081238. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 23:04:07,478][66916] Avg episode reward: [(0, '45.280'), (1, '57.670')] [2023-10-07 23:04:07,605][67871] Updated weights for policy 1, policy_version 80270 (0.0008) [2023-10-07 23:04:07,974][67871] Updated weights for policy 1, policy_version 80280 (0.0009) [2023-10-07 23:04:08,943][67838] Updated weights for policy 0, policy_version 80162 (0.0009) [2023-10-07 23:04:09,308][67838] Updated weights for policy 0, policy_version 80172 (0.0007) [2023-10-07 23:04:09,676][67838] Updated weights for policy 0, policy_version 80182 (0.0007) [2023-10-07 23:04:10,045][67838] Updated weights for policy 0, policy_version 80192 (0.0007) [2023-10-07 23:04:12,174][67871] Updated weights for policy 1, policy_version 80290 (0.0010) [2023-10-07 23:04:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164331520. Throughput: 0: 1655.9, 1: 1650.3. Samples: 41090362. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 23:04:12,478][66916] Avg episode reward: [(0, '44.230'), (1, '52.610')] [2023-10-07 23:04:12,539][67871] Updated weights for policy 1, policy_version 80300 (0.0007) [2023-10-07 23:04:12,903][67871] Updated weights for policy 1, policy_version 80310 (0.0007) [2023-10-07 23:04:13,277][67871] Updated weights for policy 1, policy_version 80320 (0.0007) [2023-10-07 23:04:14,016][67838] Updated weights for policy 0, policy_version 80202 (0.0010) [2023-10-07 23:04:14,399][67838] Updated weights for policy 0, policy_version 80212 (0.0008) [2023-10-07 23:04:14,769][67838] Updated weights for policy 0, policy_version 80222 (0.0007) [2023-10-07 23:04:17,347][67871] Updated weights for policy 1, policy_version 80330 (0.0007) [2023-10-07 23:04:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164397056. Throughput: 0: 1682.2, 1: 1652.7. Samples: 41111006. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 23:04:17,477][66916] Avg episode reward: [(0, '41.810'), (1, '56.540')] [2023-10-07 23:04:17,704][67871] Updated weights for policy 1, policy_version 80340 (0.0007) [2023-10-07 23:04:18,081][67871] Updated weights for policy 1, policy_version 80350 (0.0008) [2023-10-07 23:04:18,900][67838] Updated weights for policy 0, policy_version 80232 (0.0009) [2023-10-07 23:04:19,270][67838] Updated weights for policy 0, policy_version 80242 (0.0008) [2023-10-07 23:04:19,651][67838] Updated weights for policy 0, policy_version 80252 (0.0008) [2023-10-07 23:04:22,251][67871] Updated weights for policy 1, policy_version 80360 (0.0009) [2023-10-07 23:04:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164462592. Throughput: 0: 1678.2, 1: 1656.1. Samples: 41131282. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 23:04:22,477][66916] Avg episode reward: [(0, '47.710'), (1, '57.080')] [2023-10-07 23:04:22,615][67871] Updated weights for policy 1, policy_version 80370 (0.0010) [2023-10-07 23:04:22,980][67871] Updated weights for policy 1, policy_version 80380 (0.0010) [2023-10-07 23:04:23,807][67838] Updated weights for policy 0, policy_version 80262 (0.0010) [2023-10-07 23:04:24,183][67838] Updated weights for policy 0, policy_version 80272 (0.0010) [2023-10-07 23:04:24,556][67838] Updated weights for policy 0, policy_version 80282 (0.0009) [2023-10-07 23:04:27,032][67871] Updated weights for policy 1, policy_version 80390 (0.0009) [2023-10-07 23:04:27,397][67871] Updated weights for policy 1, policy_version 80400 (0.0008) [2023-10-07 23:04:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164528128. Throughput: 0: 1662.1, 1: 1654.3. Samples: 41140346. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 23:04:27,477][66916] Avg episode reward: [(0, '45.630'), (1, '56.500')] [2023-10-07 23:04:27,758][67871] Updated weights for policy 1, policy_version 80410 (0.0008) [2023-10-07 23:04:28,490][67838] Updated weights for policy 0, policy_version 80292 (0.0008) [2023-10-07 23:04:28,867][67838] Updated weights for policy 0, policy_version 80302 (0.0007) [2023-10-07 23:04:29,234][67838] Updated weights for policy 0, policy_version 80312 (0.0008) [2023-10-07 23:04:32,032][67871] Updated weights for policy 1, policy_version 80420 (0.0008) [2023-10-07 23:04:32,398][67871] Updated weights for policy 1, policy_version 80430 (0.0008) [2023-10-07 23:04:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164593664. Throughput: 0: 1677.5, 1: 1657.1. Samples: 41160750. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 23:04:32,478][66916] Avg episode reward: [(0, '48.420'), (1, '56.420')] [2023-10-07 23:04:32,763][67871] Updated weights for policy 1, policy_version 80440 (0.0008) [2023-10-07 23:04:33,426][67838] Updated weights for policy 0, policy_version 80322 (0.0009) [2023-10-07 23:04:33,797][67838] Updated weights for policy 0, policy_version 80332 (0.0009) [2023-10-07 23:04:34,169][67838] Updated weights for policy 0, policy_version 80342 (0.0008) [2023-10-07 23:04:34,532][67838] Updated weights for policy 0, policy_version 80352 (0.0008) [2023-10-07 23:04:36,861][67871] Updated weights for policy 1, policy_version 80450 (0.0008) [2023-10-07 23:04:37,229][67871] Updated weights for policy 1, policy_version 80460 (0.0008) [2023-10-07 23:04:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164659200. Throughput: 0: 1673.7, 1: 1657.6. Samples: 41181138. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 23:04:37,477][66916] Avg episode reward: [(0, '47.180'), (1, '58.380')] [2023-10-07 23:04:37,592][67871] Updated weights for policy 1, policy_version 80470 (0.0008) [2023-10-07 23:04:37,955][67871] Updated weights for policy 1, policy_version 80480 (0.0008) [2023-10-07 23:04:38,676][67838] Updated weights for policy 0, policy_version 80362 (0.0010) [2023-10-07 23:04:39,047][67838] Updated weights for policy 0, policy_version 80372 (0.0012) [2023-10-07 23:04:39,424][67838] Updated weights for policy 0, policy_version 80382 (0.0011) [2023-10-07 23:04:41,968][67871] Updated weights for policy 1, policy_version 80490 (0.0010) [2023-10-07 23:04:42,328][67871] Updated weights for policy 1, policy_version 80500 (0.0010) [2023-10-07 23:04:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164724736. Throughput: 0: 1657.1, 1: 1662.1. Samples: 41190196. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-07 23:04:42,477][66916] Avg episode reward: [(0, '47.490'), (1, '58.260')] [2023-10-07 23:04:42,690][67871] Updated weights for policy 1, policy_version 80510 (0.0009) [2023-10-07 23:04:43,537][67838] Updated weights for policy 0, policy_version 80392 (0.0010) [2023-10-07 23:04:43,911][67838] Updated weights for policy 0, policy_version 80402 (0.0009) [2023-10-07 23:04:44,279][67838] Updated weights for policy 0, policy_version 80412 (0.0008) [2023-10-07 23:04:46,912][67871] Updated weights for policy 1, policy_version 80520 (0.0007) [2023-10-07 23:04:47,284][67871] Updated weights for policy 1, policy_version 80530 (0.0009) [2023-10-07 23:04:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164790272. Throughput: 0: 1662.8, 1: 1664.9. Samples: 41210432. Policy #0 lag: (min: 0.0, avg: 29.0, max: 32.0) [2023-10-07 23:04:47,477][66916] Avg episode reward: [(0, '48.570'), (1, '56.170')] [2023-10-07 23:04:47,653][67871] Updated weights for policy 1, policy_version 80540 (0.0010) [2023-10-07 23:04:48,299][67838] Updated weights for policy 0, policy_version 80422 (0.0007) [2023-10-07 23:04:48,671][67838] Updated weights for policy 0, policy_version 80432 (0.0008) [2023-10-07 23:04:49,053][67838] Updated weights for policy 0, policy_version 80442 (0.0011) [2023-10-07 23:04:51,818][67871] Updated weights for policy 1, policy_version 80550 (0.0010) [2023-10-07 23:04:52,181][67871] Updated weights for policy 1, policy_version 80560 (0.0010) [2023-10-07 23:04:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 164855808. Throughput: 0: 1660.6, 1: 1654.5. Samples: 41230418. Policy #0 lag: (min: 0.0, avg: 29.0, max: 32.0) [2023-10-07 23:04:52,477][66916] Avg episode reward: [(0, '48.030'), (1, '55.140')] [2023-10-07 23:04:52,555][67871] Updated weights for policy 1, policy_version 80570 (0.0010) [2023-10-07 23:04:53,092][67838] Updated weights for policy 0, policy_version 80452 (0.0009) [2023-10-07 23:04:53,466][67838] Updated weights for policy 0, policy_version 80462 (0.0007) [2023-10-07 23:04:53,847][67838] Updated weights for policy 0, policy_version 80472 (0.0008) [2023-10-07 23:04:56,742][67871] Updated weights for policy 1, policy_version 80580 (0.0010) [2023-10-07 23:04:57,115][67871] Updated weights for policy 1, policy_version 80590 (0.0007) [2023-10-07 23:04:57,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164921344. Throughput: 0: 1661.1, 1: 1661.8. Samples: 41239894. Policy #0 lag: (min: 0.0, avg: 29.0, max: 32.0) [2023-10-07 23:04:57,477][66916] Avg episode reward: [(0, '48.780'), (1, '56.170')] [2023-10-07 23:04:57,486][67871] Updated weights for policy 1, policy_version 80600 (0.0008) [2023-10-07 23:04:58,068][67838] Updated weights for policy 0, policy_version 80482 (0.0008) [2023-10-07 23:04:58,439][67838] Updated weights for policy 0, policy_version 80492 (0.0010) [2023-10-07 23:04:58,815][67838] Updated weights for policy 0, policy_version 80502 (0.0009) [2023-10-07 23:04:59,185][67838] Updated weights for policy 0, policy_version 80512 (0.0009) [2023-10-07 23:05:01,597][67871] Updated weights for policy 1, policy_version 80610 (0.0010) [2023-10-07 23:05:01,959][67871] Updated weights for policy 1, policy_version 80620 (0.0010) [2023-10-07 23:05:02,338][67871] Updated weights for policy 1, policy_version 80630 (0.0007) [2023-10-07 23:05:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164986880. Throughput: 0: 1658.1, 1: 1659.4. Samples: 41260292. Policy #0 lag: (min: 0.0, avg: 29.0, max: 32.0) [2023-10-07 23:05:02,478][66916] Avg episode reward: [(0, '47.870'), (1, '54.960')] [2023-10-07 23:05:02,702][67871] Updated weights for policy 1, policy_version 80640 (0.0007) [2023-10-07 23:05:03,212][67838] Updated weights for policy 0, policy_version 80522 (0.0009) [2023-10-07 23:05:03,591][67838] Updated weights for policy 0, policy_version 80532 (0.0009) [2023-10-07 23:05:03,958][67838] Updated weights for policy 0, policy_version 80542 (0.0008) [2023-10-07 23:05:06,660][67871] Updated weights for policy 1, policy_version 80650 (0.0009) [2023-10-07 23:05:07,026][67871] Updated weights for policy 1, policy_version 80660 (0.0011) [2023-10-07 23:05:07,385][67871] Updated weights for policy 1, policy_version 80670 (0.0008) [2023-10-07 23:05:07,477][66916] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 165085184. Throughput: 0: 1667.7, 1: 1646.5. Samples: 41280424. Policy #0 lag: (min: 0.0, avg: 29.0, max: 32.0) [2023-10-07 23:05:07,478][66916] Avg episode reward: [(0, '47.390'), (1, '55.780')] [2023-10-07 23:05:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000080672_82608128.pth... [2023-10-07 23:05:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000080544_82477056.pth... [2023-10-07 23:05:07,526][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000079008_80904192.pth [2023-10-07 23:05:07,530][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000079104_81002496.pth [2023-10-07 23:05:08,088][67838] Updated weights for policy 0, policy_version 80552 (0.0008) [2023-10-07 23:05:08,468][67838] Updated weights for policy 0, policy_version 80562 (0.0010) [2023-10-07 23:05:08,847][67838] Updated weights for policy 0, policy_version 80572 (0.0007) [2023-10-07 23:05:11,533][67871] Updated weights for policy 1, policy_version 80680 (0.0007) [2023-10-07 23:05:11,894][67871] Updated weights for policy 1, policy_version 80690 (0.0010) [2023-10-07 23:05:12,266][67871] Updated weights for policy 1, policy_version 80700 (0.0010) [2023-10-07 23:05:12,476][66916] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 165150720. Throughput: 0: 1667.9, 1: 1659.9. Samples: 41290096. Policy #0 lag: (min: 0.0, avg: 29.0, max: 32.0) [2023-10-07 23:05:12,477][66916] Avg episode reward: [(0, '45.600'), (1, '53.390')] [2023-10-07 23:05:12,883][67838] Updated weights for policy 0, policy_version 80582 (0.0007) [2023-10-07 23:05:13,255][67838] Updated weights for policy 0, policy_version 80592 (0.0007) [2023-10-07 23:05:13,620][67838] Updated weights for policy 0, policy_version 80602 (0.0009) [2023-10-07 23:05:16,345][67871] Updated weights for policy 1, policy_version 80710 (0.0008) [2023-10-07 23:05:16,719][67871] Updated weights for policy 1, policy_version 80720 (0.0009) [2023-10-07 23:05:17,083][67871] Updated weights for policy 1, policy_version 80730 (0.0009) [2023-10-07 23:05:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 165216256. Throughput: 0: 1664.6, 1: 1660.9. Samples: 41310398. Policy #0 lag: (min: 0.0, avg: 29.0, max: 32.0) [2023-10-07 23:05:17,478][66916] Avg episode reward: [(0, '46.670'), (1, '54.430')] [2023-10-07 23:05:17,792][67838] Updated weights for policy 0, policy_version 80612 (0.0010) [2023-10-07 23:05:18,168][67838] Updated weights for policy 0, policy_version 80622 (0.0010) [2023-10-07 23:05:18,555][67838] Updated weights for policy 0, policy_version 80632 (0.0010) [2023-10-07 23:05:21,228][67871] Updated weights for policy 1, policy_version 80740 (0.0008) [2023-10-07 23:05:21,604][67871] Updated weights for policy 1, policy_version 80750 (0.0007) [2023-10-07 23:05:21,971][67871] Updated weights for policy 1, policy_version 80760 (0.0007) [2023-10-07 23:05:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 165281792. Throughput: 0: 1669.4, 1: 1641.6. Samples: 41330132. Policy #0 lag: (min: 0.0, avg: 29.0, max: 32.0) [2023-10-07 23:05:22,478][66916] Avg episode reward: [(0, '47.110'), (1, '54.690')] [2023-10-07 23:05:22,625][67838] Updated weights for policy 0, policy_version 80642 (0.0010) [2023-10-07 23:05:22,993][67838] Updated weights for policy 0, policy_version 80652 (0.0007) [2023-10-07 23:05:23,369][67838] Updated weights for policy 0, policy_version 80662 (0.0007) [2023-10-07 23:05:23,741][67838] Updated weights for policy 0, policy_version 80672 (0.0007) [2023-10-07 23:05:26,168][67871] Updated weights for policy 1, policy_version 80770 (0.0009) [2023-10-07 23:05:26,533][67871] Updated weights for policy 1, policy_version 80780 (0.0010) [2023-10-07 23:05:26,904][67871] Updated weights for policy 1, policy_version 80790 (0.0008) [2023-10-07 23:05:27,266][67871] Updated weights for policy 1, policy_version 80800 (0.0009) [2023-10-07 23:05:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 165347328. Throughput: 0: 1672.4, 1: 1656.6. Samples: 41340004. Policy #0 lag: (min: 0.0, avg: 29.0, max: 32.0) [2023-10-07 23:05:27,477][66916] Avg episode reward: [(0, '49.160'), (1, '54.820')] [2023-10-07 23:05:27,773][67838] Updated weights for policy 0, policy_version 80682 (0.0008) [2023-10-07 23:05:28,148][67838] Updated weights for policy 0, policy_version 80692 (0.0007) [2023-10-07 23:05:28,526][67838] Updated weights for policy 0, policy_version 80702 (0.0007) [2023-10-07 23:05:31,391][67871] Updated weights for policy 1, policy_version 80810 (0.0008) [2023-10-07 23:05:31,759][67871] Updated weights for policy 1, policy_version 80820 (0.0007) [2023-10-07 23:05:32,121][67871] Updated weights for policy 1, policy_version 80830 (0.0007) [2023-10-07 23:05:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 165412864. Throughput: 0: 1676.5, 1: 1660.4. Samples: 41360594. Policy #0 lag: (min: 0.0, avg: 29.0, max: 32.0) [2023-10-07 23:05:32,478][66916] Avg episode reward: [(0, '44.790'), (1, '55.880')] [2023-10-07 23:05:32,643][67838] Updated weights for policy 0, policy_version 80712 (0.0008) [2023-10-07 23:05:33,022][67838] Updated weights for policy 0, policy_version 80722 (0.0007) [2023-10-07 23:05:33,396][67838] Updated weights for policy 0, policy_version 80732 (0.0007) [2023-10-07 23:05:36,078][67871] Updated weights for policy 1, policy_version 80840 (0.0009) [2023-10-07 23:05:36,452][67871] Updated weights for policy 1, policy_version 80850 (0.0010) [2023-10-07 23:05:36,822][67871] Updated weights for policy 1, policy_version 80860 (0.0007) [2023-10-07 23:05:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 165478400. Throughput: 0: 1678.4, 1: 1649.2. Samples: 41380162. Policy #0 lag: (min: 1.0, avg: 5.0, max: 33.0) [2023-10-07 23:05:37,477][66916] Avg episode reward: [(0, '46.980'), (1, '57.910')] [2023-10-07 23:05:37,496][67838] Updated weights for policy 0, policy_version 80742 (0.0010) [2023-10-07 23:05:37,857][67838] Updated weights for policy 0, policy_version 80752 (0.0011) [2023-10-07 23:05:38,237][67838] Updated weights for policy 0, policy_version 80762 (0.0008) [2023-10-07 23:05:41,002][67871] Updated weights for policy 1, policy_version 80870 (0.0008) [2023-10-07 23:05:41,361][67871] Updated weights for policy 1, policy_version 80880 (0.0009) [2023-10-07 23:05:41,731][67871] Updated weights for policy 1, policy_version 80890 (0.0008) [2023-10-07 23:05:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 165543936. Throughput: 0: 1673.3, 1: 1669.6. Samples: 41390324. Policy #0 lag: (min: 1.0, avg: 5.0, max: 33.0) [2023-10-07 23:05:42,478][66916] Avg episode reward: [(0, '43.440'), (1, '58.280')] [2023-10-07 23:05:42,478][67838] Updated weights for policy 0, policy_version 80772 (0.0008) [2023-10-07 23:05:42,860][67838] Updated weights for policy 0, policy_version 80782 (0.0009) [2023-10-07 23:05:43,233][67838] Updated weights for policy 0, policy_version 80792 (0.0009) [2023-10-07 23:05:45,801][67871] Updated weights for policy 1, policy_version 80900 (0.0009) [2023-10-07 23:05:46,162][67871] Updated weights for policy 1, policy_version 80910 (0.0008) [2023-10-07 23:05:46,526][67871] Updated weights for policy 1, policy_version 80920 (0.0008) [2023-10-07 23:05:47,332][67838] Updated weights for policy 0, policy_version 80802 (0.0009) [2023-10-07 23:05:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 165609472. Throughput: 0: 1668.3, 1: 1667.2. Samples: 41410388. Policy #0 lag: (min: 1.0, avg: 5.0, max: 33.0) [2023-10-07 23:05:47,478][66916] Avg episode reward: [(0, '40.590'), (1, '55.800')] [2023-10-07 23:05:47,713][67838] Updated weights for policy 0, policy_version 80812 (0.0008) [2023-10-07 23:05:48,092][67838] Updated weights for policy 0, policy_version 80822 (0.0009) [2023-10-07 23:05:48,457][67838] Updated weights for policy 0, policy_version 80832 (0.0009) [2023-10-07 23:05:50,622][67871] Updated weights for policy 1, policy_version 80930 (0.0009) [2023-10-07 23:05:51,005][67871] Updated weights for policy 1, policy_version 80940 (0.0010) [2023-10-07 23:05:51,363][67871] Updated weights for policy 1, policy_version 80950 (0.0010) [2023-10-07 23:05:51,729][67871] Updated weights for policy 1, policy_version 80960 (0.0010) [2023-10-07 23:05:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 165675008. Throughput: 0: 1665.6, 1: 1657.6. Samples: 41429972. Policy #0 lag: (min: 1.0, avg: 5.0, max: 33.0) [2023-10-07 23:05:52,478][66916] Avg episode reward: [(0, '45.140'), (1, '58.090')] [2023-10-07 23:05:52,604][67838] Updated weights for policy 0, policy_version 80842 (0.0010) [2023-10-07 23:05:52,980][67838] Updated weights for policy 0, policy_version 80852 (0.0010) [2023-10-07 23:05:53,369][67838] Updated weights for policy 0, policy_version 80862 (0.0010) [2023-10-07 23:05:55,979][67871] Updated weights for policy 1, policy_version 80970 (0.0008) [2023-10-07 23:05:56,337][67871] Updated weights for policy 1, policy_version 80980 (0.0008) [2023-10-07 23:05:56,704][67871] Updated weights for policy 1, policy_version 80990 (0.0007) [2023-10-07 23:05:57,446][67838] Updated weights for policy 0, policy_version 80872 (0.0010) [2023-10-07 23:05:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 165740544. Throughput: 0: 1666.7, 1: 1667.9. Samples: 41440152. Policy #0 lag: (min: 1.0, avg: 5.0, max: 33.0) [2023-10-07 23:05:57,477][66916] Avg episode reward: [(0, '40.620'), (1, '59.630')] [2023-10-07 23:05:57,810][67838] Updated weights for policy 0, policy_version 80882 (0.0009) [2023-10-07 23:05:58,186][67838] Updated weights for policy 0, policy_version 80892 (0.0007) [2023-10-07 23:06:00,898][67871] Updated weights for policy 1, policy_version 81000 (0.0010) [2023-10-07 23:06:01,267][67871] Updated weights for policy 1, policy_version 81010 (0.0010) [2023-10-07 23:06:01,634][67871] Updated weights for policy 1, policy_version 81020 (0.0007) [2023-10-07 23:06:02,257][67838] Updated weights for policy 0, policy_version 80902 (0.0009) [2023-10-07 23:06:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 165806080. Throughput: 0: 1670.1, 1: 1660.8. Samples: 41460292. Policy #0 lag: (min: 1.0, avg: 5.0, max: 33.0) [2023-10-07 23:06:02,478][66916] Avg episode reward: [(0, '46.030'), (1, '61.900')] [2023-10-07 23:06:02,643][67838] Updated weights for policy 0, policy_version 80912 (0.0007) [2023-10-07 23:06:03,009][67838] Updated weights for policy 0, policy_version 80922 (0.0008) [2023-10-07 23:06:05,537][67871] Updated weights for policy 1, policy_version 81030 (0.0009) [2023-10-07 23:06:05,899][67871] Updated weights for policy 1, policy_version 81040 (0.0009) [2023-10-07 23:06:06,260][67871] Updated weights for policy 1, policy_version 81050 (0.0007) [2023-10-07 23:06:07,060][67838] Updated weights for policy 0, policy_version 80932 (0.0009) [2023-10-07 23:06:07,435][67838] Updated weights for policy 0, policy_version 80942 (0.0008) [2023-10-07 23:06:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 165871616. Throughput: 0: 1665.2, 1: 1663.9. Samples: 41479940. Policy #0 lag: (min: 1.0, avg: 5.0, max: 33.0) [2023-10-07 23:06:07,477][66916] Avg episode reward: [(0, '46.570'), (1, '61.670')] [2023-10-07 23:06:07,816][67838] Updated weights for policy 0, policy_version 80952 (0.0009) [2023-10-07 23:06:10,375][67871] Updated weights for policy 1, policy_version 81060 (0.0009) [2023-10-07 23:06:10,733][67871] Updated weights for policy 1, policy_version 81070 (0.0009) [2023-10-07 23:06:11,110][67871] Updated weights for policy 1, policy_version 81080 (0.0009) [2023-10-07 23:06:11,960][67838] Updated weights for policy 0, policy_version 80962 (0.0009) [2023-10-07 23:06:12,333][67838] Updated weights for policy 0, policy_version 80972 (0.0008) [2023-10-07 23:06:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 165937152. Throughput: 0: 1666.8, 1: 1670.1. Samples: 41490166. Policy #0 lag: (min: 1.0, avg: 5.0, max: 33.0) [2023-10-07 23:06:12,477][66916] Avg episode reward: [(0, '46.360'), (1, '67.240')] [2023-10-07 23:06:12,478][67676] Saving new best policy, reward=67.240! [2023-10-07 23:06:12,715][67838] Updated weights for policy 0, policy_version 80982 (0.0011) [2023-10-07 23:06:13,087][67838] Updated weights for policy 0, policy_version 80992 (0.0011) [2023-10-07 23:06:15,366][67871] Updated weights for policy 1, policy_version 81090 (0.0008) [2023-10-07 23:06:15,784][67871] Updated weights for policy 1, policy_version 81100 (0.0011) [2023-10-07 23:06:16,155][67871] Updated weights for policy 1, policy_version 81110 (0.0011) [2023-10-07 23:06:16,524][67871] Updated weights for policy 1, policy_version 81120 (0.0011) [2023-10-07 23:06:17,257][67838] Updated weights for policy 0, policy_version 81002 (0.0010) [2023-10-07 23:06:17,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166002688. Throughput: 0: 1662.6, 1: 1657.2. Samples: 41509986. Policy #0 lag: (min: 1.0, avg: 5.0, max: 33.0) [2023-10-07 23:06:17,477][66916] Avg episode reward: [(0, '50.110'), (1, '67.980')] [2023-10-07 23:06:17,478][67676] Saving new best policy, reward=67.980! [2023-10-07 23:06:17,625][67838] Updated weights for policy 0, policy_version 81012 (0.0009) [2023-10-07 23:06:18,005][67838] Updated weights for policy 0, policy_version 81022 (0.0007) [2023-10-07 23:06:20,913][67871] Updated weights for policy 1, policy_version 81130 (0.0007) [2023-10-07 23:06:21,278][67871] Updated weights for policy 1, policy_version 81140 (0.0007) [2023-10-07 23:06:21,653][67871] Updated weights for policy 1, policy_version 81150 (0.0007) [2023-10-07 23:06:21,924][67838] Updated weights for policy 0, policy_version 81032 (0.0007) [2023-10-07 23:06:22,297][67838] Updated weights for policy 0, policy_version 81042 (0.0007) [2023-10-07 23:06:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 166068224. Throughput: 0: 1652.6, 1: 1658.7. Samples: 41529170. Policy #0 lag: (min: 1.0, avg: 5.0, max: 33.0) [2023-10-07 23:06:22,477][66916] Avg episode reward: [(0, '48.400'), (1, '66.120')] [2023-10-07 23:06:22,661][67838] Updated weights for policy 0, policy_version 81052 (0.0007) [2023-10-07 23:06:25,764][67871] Updated weights for policy 1, policy_version 81160 (0.0008) [2023-10-07 23:06:26,127][67871] Updated weights for policy 1, policy_version 81170 (0.0007) [2023-10-07 23:06:26,485][67871] Updated weights for policy 1, policy_version 81180 (0.0008) [2023-10-07 23:06:26,940][67838] Updated weights for policy 0, policy_version 81062 (0.0007) [2023-10-07 23:06:27,308][67838] Updated weights for policy 0, policy_version 81072 (0.0010) [2023-10-07 23:06:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166133760. Throughput: 0: 1661.7, 1: 1657.1. Samples: 41539668. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-10-07 23:06:27,477][66916] Avg episode reward: [(0, '47.740'), (1, '63.360')] [2023-10-07 23:06:27,679][67838] Updated weights for policy 0, policy_version 81082 (0.0007) [2023-10-07 23:06:30,288][67871] Updated weights for policy 1, policy_version 81190 (0.0007) [2023-10-07 23:06:30,659][67871] Updated weights for policy 1, policy_version 81200 (0.0008) [2023-10-07 23:06:31,019][67871] Updated weights for policy 1, policy_version 81210 (0.0007) [2023-10-07 23:06:31,833][67838] Updated weights for policy 0, policy_version 81092 (0.0009) [2023-10-07 23:06:32,204][67838] Updated weights for policy 0, policy_version 81102 (0.0011) [2023-10-07 23:06:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 166199296. Throughput: 0: 1665.4, 1: 1650.1. Samples: 41559586. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-10-07 23:06:32,477][66916] Avg episode reward: [(0, '46.200'), (1, '61.800')] [2023-10-07 23:06:32,584][67838] Updated weights for policy 0, policy_version 81112 (0.0010) [2023-10-07 23:06:35,105][67871] Updated weights for policy 1, policy_version 81220 (0.0008) [2023-10-07 23:06:35,470][67871] Updated weights for policy 1, policy_version 81230 (0.0009) [2023-10-07 23:06:35,833][67871] Updated weights for policy 1, policy_version 81240 (0.0010) [2023-10-07 23:06:36,681][67838] Updated weights for policy 0, policy_version 81122 (0.0010) [2023-10-07 23:06:37,059][67838] Updated weights for policy 0, policy_version 81132 (0.0007) [2023-10-07 23:06:37,424][67838] Updated weights for policy 0, policy_version 81142 (0.0007) [2023-10-07 23:06:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166264832. Throughput: 0: 1652.1, 1: 1662.9. Samples: 41579148. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-10-07 23:06:37,477][66916] Avg episode reward: [(0, '44.430'), (1, '60.840')] [2023-10-07 23:06:37,795][67838] Updated weights for policy 0, policy_version 81152 (0.0007) [2023-10-07 23:06:40,043][67871] Updated weights for policy 1, policy_version 81250 (0.0012) [2023-10-07 23:06:40,414][67871] Updated weights for policy 1, policy_version 81260 (0.0008) [2023-10-07 23:06:40,772][67871] Updated weights for policy 1, policy_version 81270 (0.0010) [2023-10-07 23:06:41,141][67871] Updated weights for policy 1, policy_version 81280 (0.0007) [2023-10-07 23:06:41,955][67838] Updated weights for policy 0, policy_version 81162 (0.0011) [2023-10-07 23:06:42,315][67838] Updated weights for policy 0, policy_version 81172 (0.0010) [2023-10-07 23:06:42,477][66916] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166330368. Throughput: 0: 1660.4, 1: 1663.9. Samples: 41589746. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-10-07 23:06:42,478][66916] Avg episode reward: [(0, '46.430'), (1, '58.050')] [2023-10-07 23:06:42,688][67838] Updated weights for policy 0, policy_version 81182 (0.0008) [2023-10-07 23:06:45,172][67871] Updated weights for policy 1, policy_version 81290 (0.0007) [2023-10-07 23:06:45,537][67871] Updated weights for policy 1, policy_version 81300 (0.0009) [2023-10-07 23:06:45,908][67871] Updated weights for policy 1, policy_version 81310 (0.0010) [2023-10-07 23:06:46,874][67838] Updated weights for policy 0, policy_version 81192 (0.0008) [2023-10-07 23:06:47,251][67838] Updated weights for policy 0, policy_version 81202 (0.0007) [2023-10-07 23:06:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166395904. Throughput: 0: 1660.5, 1: 1653.0. Samples: 41609400. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-10-07 23:06:47,478][66916] Avg episode reward: [(0, '45.500'), (1, '58.640')] [2023-10-07 23:06:47,624][67838] Updated weights for policy 0, policy_version 81212 (0.0010) [2023-10-07 23:06:50,068][67871] Updated weights for policy 1, policy_version 81320 (0.0008) [2023-10-07 23:06:50,423][67871] Updated weights for policy 1, policy_version 81330 (0.0008) [2023-10-07 23:06:50,798][67871] Updated weights for policy 1, policy_version 81340 (0.0009) [2023-10-07 23:06:51,796][67838] Updated weights for policy 0, policy_version 81222 (0.0009) [2023-10-07 23:06:52,173][67838] Updated weights for policy 0, policy_version 81232 (0.0009) [2023-10-07 23:06:52,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166461440. Throughput: 0: 1653.1, 1: 1665.9. Samples: 41629292. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-10-07 23:06:52,478][66916] Avg episode reward: [(0, '48.140'), (1, '60.160')] [2023-10-07 23:06:52,544][67838] Updated weights for policy 0, policy_version 81242 (0.0007) [2023-10-07 23:06:54,767][67871] Updated weights for policy 1, policy_version 81350 (0.0009) [2023-10-07 23:06:55,129][67871] Updated weights for policy 1, policy_version 81360 (0.0007) [2023-10-07 23:06:55,491][67871] Updated weights for policy 1, policy_version 81370 (0.0008) [2023-10-07 23:06:56,584][67838] Updated weights for policy 0, policy_version 81252 (0.0007) [2023-10-07 23:06:56,961][67838] Updated weights for policy 0, policy_version 81262 (0.0010) [2023-10-07 23:06:57,337][67838] Updated weights for policy 0, policy_version 81272 (0.0009) [2023-10-07 23:06:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166526976. Throughput: 0: 1660.3, 1: 1661.7. Samples: 41639656. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-10-07 23:06:57,477][66916] Avg episode reward: [(0, '50.930'), (1, '61.590')] [2023-10-07 23:06:59,559][67871] Updated weights for policy 1, policy_version 81380 (0.0007) [2023-10-07 23:06:59,920][67871] Updated weights for policy 1, policy_version 81390 (0.0007) [2023-10-07 23:07:00,289][67871] Updated weights for policy 1, policy_version 81400 (0.0008) [2023-10-07 23:07:01,585][67838] Updated weights for policy 0, policy_version 81282 (0.0008) [2023-10-07 23:07:01,961][67838] Updated weights for policy 0, policy_version 81292 (0.0010) [2023-10-07 23:07:02,335][67838] Updated weights for policy 0, policy_version 81302 (0.0008) [2023-10-07 23:07:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166592512. Throughput: 0: 1658.8, 1: 1654.3. Samples: 41659072. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-10-07 23:07:02,478][66916] Avg episode reward: [(0, '52.380'), (1, '63.430')] [2023-10-07 23:07:02,706][67838] Updated weights for policy 0, policy_version 81312 (0.0007) [2023-10-07 23:07:04,582][67871] Updated weights for policy 1, policy_version 81410 (0.0008) [2023-10-07 23:07:04,994][67871] Updated weights for policy 1, policy_version 81420 (0.0011) [2023-10-07 23:07:05,365][67871] Updated weights for policy 1, policy_version 81430 (0.0010) [2023-10-07 23:07:05,719][67871] Updated weights for policy 1, policy_version 81440 (0.0009) [2023-10-07 23:07:06,652][67838] Updated weights for policy 0, policy_version 81322 (0.0007) [2023-10-07 23:07:07,021][67838] Updated weights for policy 0, policy_version 81332 (0.0010) [2023-10-07 23:07:07,396][67838] Updated weights for policy 0, policy_version 81342 (0.0011) [2023-10-07 23:07:07,476][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 166690816. Throughput: 0: 1647.5, 1: 1672.3. Samples: 41678560. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-10-07 23:07:07,477][66916] Avg episode reward: [(0, '50.980'), (1, '62.860')] [2023-10-07 23:07:07,486][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000081344_83296256.pth... [2023-10-07 23:07:07,486][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000081440_83394560.pth... [2023-10-07 23:07:07,526][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000079776_81690624.pth [2023-10-07 23:07:07,530][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000079872_81788928.pth [2023-10-07 23:07:09,782][67871] Updated weights for policy 1, policy_version 81450 (0.0007) [2023-10-07 23:07:10,141][67871] Updated weights for policy 1, policy_version 81460 (0.0009) [2023-10-07 23:07:10,508][67871] Updated weights for policy 1, policy_version 81470 (0.0011) [2023-10-07 23:07:11,424][67838] Updated weights for policy 0, policy_version 81352 (0.0010) [2023-10-07 23:07:11,792][67838] Updated weights for policy 0, policy_version 81362 (0.0009) [2023-10-07 23:07:12,162][67838] Updated weights for policy 0, policy_version 81372 (0.0010) [2023-10-07 23:07:12,477][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 166756352. Throughput: 0: 1659.6, 1: 1669.2. Samples: 41689466. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-10-07 23:07:12,478][66916] Avg episode reward: [(0, '45.770'), (1, '64.090')] [2023-10-07 23:07:14,419][67871] Updated weights for policy 1, policy_version 81480 (0.0010) [2023-10-07 23:07:14,783][67871] Updated weights for policy 1, policy_version 81490 (0.0010) [2023-10-07 23:07:15,138][67871] Updated weights for policy 1, policy_version 81500 (0.0007) [2023-10-07 23:07:16,385][67838] Updated weights for policy 0, policy_version 81382 (0.0010) [2023-10-07 23:07:16,766][67838] Updated weights for policy 0, policy_version 81392 (0.0009) [2023-10-07 23:07:17,136][67838] Updated weights for policy 0, policy_version 81402 (0.0009) [2023-10-07 23:07:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 166821888. Throughput: 0: 1660.1, 1: 1665.0. Samples: 41709216. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-07 23:07:17,477][66916] Avg episode reward: [(0, '43.940'), (1, '62.450')] [2023-10-07 23:07:19,252][67871] Updated weights for policy 1, policy_version 81510 (0.0009) [2023-10-07 23:07:19,620][67871] Updated weights for policy 1, policy_version 81520 (0.0010) [2023-10-07 23:07:19,999][67871] Updated weights for policy 1, policy_version 81530 (0.0010) [2023-10-07 23:07:20,981][67838] Updated weights for policy 0, policy_version 81412 (0.0009) [2023-10-07 23:07:21,356][67838] Updated weights for policy 0, policy_version 81422 (0.0007) [2023-10-07 23:07:21,730][67838] Updated weights for policy 0, policy_version 81432 (0.0009) [2023-10-07 23:07:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 166887424. Throughput: 0: 1642.2, 1: 1676.1. Samples: 41728474. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-07 23:07:22,478][66916] Avg episode reward: [(0, '43.260'), (1, '62.930')] [2023-10-07 23:07:23,957][67871] Updated weights for policy 1, policy_version 81540 (0.0009) [2023-10-07 23:07:24,311][67871] Updated weights for policy 1, policy_version 81550 (0.0008) [2023-10-07 23:07:24,680][67871] Updated weights for policy 1, policy_version 81560 (0.0007) [2023-10-07 23:07:26,037][67838] Updated weights for policy 0, policy_version 81442 (0.0008) [2023-10-07 23:07:26,405][67838] Updated weights for policy 0, policy_version 81452 (0.0010) [2023-10-07 23:07:26,776][67838] Updated weights for policy 0, policy_version 81462 (0.0008) [2023-10-07 23:07:27,158][67838] Updated weights for policy 0, policy_version 81472 (0.0009) [2023-10-07 23:07:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 166952960. Throughput: 0: 1659.3, 1: 1656.0. Samples: 41738938. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-07 23:07:27,478][66916] Avg episode reward: [(0, '42.630'), (1, '62.770')] [2023-10-07 23:07:28,925][67871] Updated weights for policy 1, policy_version 81570 (0.0007) [2023-10-07 23:07:29,297][67871] Updated weights for policy 1, policy_version 81580 (0.0009) [2023-10-07 23:07:29,653][67871] Updated weights for policy 1, policy_version 81590 (0.0011) [2023-10-07 23:07:30,010][67871] Updated weights for policy 1, policy_version 81600 (0.0010) [2023-10-07 23:07:31,233][67838] Updated weights for policy 0, policy_version 81482 (0.0009) [2023-10-07 23:07:31,597][67838] Updated weights for policy 0, policy_version 81492 (0.0007) [2023-10-07 23:07:31,961][67838] Updated weights for policy 0, policy_version 81502 (0.0007) [2023-10-07 23:07:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 167018496. Throughput: 0: 1655.6, 1: 1670.6. Samples: 41759080. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-07 23:07:32,477][66916] Avg episode reward: [(0, '46.950'), (1, '61.630')] [2023-10-07 23:07:34,114][67871] Updated weights for policy 1, policy_version 81610 (0.0008) [2023-10-07 23:07:34,477][67871] Updated weights for policy 1, policy_version 81620 (0.0007) [2023-10-07 23:07:34,845][67871] Updated weights for policy 1, policy_version 81630 (0.0007) [2023-10-07 23:07:36,168][67838] Updated weights for policy 0, policy_version 81512 (0.0009) [2023-10-07 23:07:36,544][67838] Updated weights for policy 0, policy_version 81522 (0.0009) [2023-10-07 23:07:36,911][67838] Updated weights for policy 0, policy_version 81532 (0.0008) [2023-10-07 23:07:37,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13653.2, 300 sec: 13329.3). Total num frames: 167084032. Throughput: 0: 1637.0, 1: 1682.5. Samples: 41778670. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-07 23:07:37,478][66916] Avg episode reward: [(0, '49.340'), (1, '62.870')] [2023-10-07 23:07:38,801][67871] Updated weights for policy 1, policy_version 81640 (0.0010) [2023-10-07 23:07:39,165][67871] Updated weights for policy 1, policy_version 81650 (0.0011) [2023-10-07 23:07:39,536][67871] Updated weights for policy 1, policy_version 81660 (0.0011) [2023-10-07 23:07:40,965][67838] Updated weights for policy 0, policy_version 81542 (0.0011) [2023-10-07 23:07:41,329][67838] Updated weights for policy 0, policy_version 81552 (0.0011) [2023-10-07 23:07:41,711][67838] Updated weights for policy 0, policy_version 81562 (0.0011) [2023-10-07 23:07:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13329.3). Total num frames: 167149568. Throughput: 0: 1657.8, 1: 1658.0. Samples: 41788870. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-07 23:07:42,478][66916] Avg episode reward: [(0, '49.350'), (1, '58.930')] [2023-10-07 23:07:43,654][67871] Updated weights for policy 1, policy_version 81670 (0.0010) [2023-10-07 23:07:44,019][67871] Updated weights for policy 1, policy_version 81680 (0.0009) [2023-10-07 23:07:44,394][67871] Updated weights for policy 1, policy_version 81690 (0.0007) [2023-10-07 23:07:45,839][67838] Updated weights for policy 0, policy_version 81572 (0.0011) [2023-10-07 23:07:46,213][67838] Updated weights for policy 0, policy_version 81582 (0.0007) [2023-10-07 23:07:46,585][67838] Updated weights for policy 0, policy_version 81592 (0.0007) [2023-10-07 23:07:47,477][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 167215104. Throughput: 0: 1651.6, 1: 1679.4. Samples: 41808964. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-07 23:07:47,478][66916] Avg episode reward: [(0, '52.550'), (1, '58.420')] [2023-10-07 23:07:48,574][67871] Updated weights for policy 1, policy_version 81700 (0.0008) [2023-10-07 23:07:48,937][67871] Updated weights for policy 1, policy_version 81710 (0.0009) [2023-10-07 23:07:49,308][67871] Updated weights for policy 1, policy_version 81720 (0.0008) [2023-10-07 23:07:50,794][67838] Updated weights for policy 0, policy_version 81602 (0.0008) [2023-10-07 23:07:51,159][67838] Updated weights for policy 0, policy_version 81612 (0.0010) [2023-10-07 23:07:51,524][67838] Updated weights for policy 0, policy_version 81622 (0.0009) [2023-10-07 23:07:51,893][67838] Updated weights for policy 0, policy_version 81632 (0.0008) [2023-10-07 23:07:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 167280640. Throughput: 0: 1648.3, 1: 1682.9. Samples: 41828466. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-07 23:07:52,477][66916] Avg episode reward: [(0, '49.580'), (1, '60.560')] [2023-10-07 23:07:53,545][67871] Updated weights for policy 1, policy_version 81730 (0.0010) [2023-10-07 23:07:53,921][67871] Updated weights for policy 1, policy_version 81740 (0.0009) [2023-10-07 23:07:54,293][67871] Updated weights for policy 1, policy_version 81750 (0.0010) [2023-10-07 23:07:54,656][67871] Updated weights for policy 1, policy_version 81760 (0.0010) [2023-10-07 23:07:56,029][67838] Updated weights for policy 0, policy_version 81642 (0.0010) [2023-10-07 23:07:56,399][67838] Updated weights for policy 0, policy_version 81652 (0.0007) [2023-10-07 23:07:56,778][67838] Updated weights for policy 0, policy_version 81662 (0.0007) [2023-10-07 23:07:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 167346176. Throughput: 0: 1657.0, 1: 1657.0. Samples: 41838596. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-07 23:07:57,477][66916] Avg episode reward: [(0, '49.980'), (1, '63.280')] [2023-10-07 23:07:58,803][67871] Updated weights for policy 1, policy_version 81770 (0.0009) [2023-10-07 23:07:59,164][67871] Updated weights for policy 1, policy_version 81780 (0.0008) [2023-10-07 23:07:59,526][67871] Updated weights for policy 1, policy_version 81790 (0.0008) [2023-10-07 23:08:00,964][67838] Updated weights for policy 0, policy_version 81672 (0.0009) [2023-10-07 23:08:01,339][67838] Updated weights for policy 0, policy_version 81682 (0.0008) [2023-10-07 23:08:01,701][67838] Updated weights for policy 0, policy_version 81692 (0.0009) [2023-10-07 23:08:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 167411712. Throughput: 0: 1646.5, 1: 1671.7. Samples: 41858536. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-07 23:08:02,477][66916] Avg episode reward: [(0, '50.890'), (1, '62.090')] [2023-10-07 23:08:03,698][67871] Updated weights for policy 1, policy_version 81800 (0.0008) [2023-10-07 23:08:04,059][67871] Updated weights for policy 1, policy_version 81810 (0.0007) [2023-10-07 23:08:04,429][67871] Updated weights for policy 1, policy_version 81820 (0.0008) [2023-10-07 23:08:06,039][67838] Updated weights for policy 0, policy_version 81702 (0.0009) [2023-10-07 23:08:06,410][67838] Updated weights for policy 0, policy_version 81712 (0.0007) [2023-10-07 23:08:06,781][67838] Updated weights for policy 0, policy_version 81722 (0.0007) [2023-10-07 23:08:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 167477248. Throughput: 0: 1654.5, 1: 1669.5. Samples: 41878056. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-07 23:08:07,477][66916] Avg episode reward: [(0, '51.080'), (1, '64.130')] [2023-10-07 23:08:08,598][67871] Updated weights for policy 1, policy_version 81830 (0.0007) [2023-10-07 23:08:08,969][67871] Updated weights for policy 1, policy_version 81840 (0.0007) [2023-10-07 23:08:09,324][67871] Updated weights for policy 1, policy_version 81850 (0.0008) [2023-10-07 23:08:11,007][67838] Updated weights for policy 0, policy_version 81732 (0.0007) [2023-10-07 23:08:11,372][67838] Updated weights for policy 0, policy_version 81742 (0.0010) [2023-10-07 23:08:11,749][67838] Updated weights for policy 0, policy_version 81752 (0.0011) [2023-10-07 23:08:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 167542784. Throughput: 0: 1656.1, 1: 1661.3. Samples: 41888220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:08:12,477][66916] Avg episode reward: [(0, '51.150'), (1, '61.810')] [2023-10-07 23:08:13,662][67871] Updated weights for policy 1, policy_version 81860 (0.0008) [2023-10-07 23:08:14,031][67871] Updated weights for policy 1, policy_version 81870 (0.0009) [2023-10-07 23:08:14,396][67871] Updated weights for policy 1, policy_version 81880 (0.0007) [2023-10-07 23:08:15,686][67838] Updated weights for policy 0, policy_version 81762 (0.0009) [2023-10-07 23:08:16,057][67838] Updated weights for policy 0, policy_version 81772 (0.0008) [2023-10-07 23:08:16,424][67838] Updated weights for policy 0, policy_version 81782 (0.0008) [2023-10-07 23:08:16,786][67838] Updated weights for policy 0, policy_version 81792 (0.0008) [2023-10-07 23:08:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 167608320. Throughput: 0: 1652.2, 1: 1659.8. Samples: 41908118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:08:17,477][66916] Avg episode reward: [(0, '52.600'), (1, '59.600')] [2023-10-07 23:08:18,424][67871] Updated weights for policy 1, policy_version 81890 (0.0008) [2023-10-07 23:08:18,782][67871] Updated weights for policy 1, policy_version 81900 (0.0009) [2023-10-07 23:08:19,157][67871] Updated weights for policy 1, policy_version 81910 (0.0007) [2023-10-07 23:08:19,520][67871] Updated weights for policy 1, policy_version 81920 (0.0009) [2023-10-07 23:08:21,009][67838] Updated weights for policy 0, policy_version 81802 (0.0009) [2023-10-07 23:08:21,372][67838] Updated weights for policy 0, policy_version 81812 (0.0009) [2023-10-07 23:08:21,733][67838] Updated weights for policy 0, policy_version 81822 (0.0008) [2023-10-07 23:08:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 167673856. Throughput: 0: 1661.8, 1: 1653.4. Samples: 41927854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:08:22,477][66916] Avg episode reward: [(0, '49.720'), (1, '55.870')] [2023-10-07 23:08:23,671][67871] Updated weights for policy 1, policy_version 81930 (0.0009) [2023-10-07 23:08:24,036][67871] Updated weights for policy 1, policy_version 81940 (0.0007) [2023-10-07 23:08:24,399][67871] Updated weights for policy 1, policy_version 81950 (0.0007) [2023-10-07 23:08:25,764][67838] Updated weights for policy 0, policy_version 81832 (0.0011) [2023-10-07 23:08:26,142][67838] Updated weights for policy 0, policy_version 81842 (0.0009) [2023-10-07 23:08:26,509][67838] Updated weights for policy 0, policy_version 81852 (0.0009) [2023-10-07 23:08:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 167739392. Throughput: 0: 1659.8, 1: 1656.9. Samples: 41938120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:08:27,477][66916] Avg episode reward: [(0, '50.200'), (1, '59.590')] [2023-10-07 23:08:28,546][67871] Updated weights for policy 1, policy_version 81960 (0.0008) [2023-10-07 23:08:28,914][67871] Updated weights for policy 1, policy_version 81970 (0.0007) [2023-10-07 23:08:29,270][67871] Updated weights for policy 1, policy_version 81980 (0.0007) [2023-10-07 23:08:30,598][67838] Updated weights for policy 0, policy_version 81862 (0.0009) [2023-10-07 23:08:30,962][67838] Updated weights for policy 0, policy_version 81872 (0.0009) [2023-10-07 23:08:31,338][67838] Updated weights for policy 0, policy_version 81882 (0.0011) [2023-10-07 23:08:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 167804928. Throughput: 0: 1653.7, 1: 1654.0. Samples: 41957810. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:08:32,477][66916] Avg episode reward: [(0, '51.330'), (1, '59.470')] [2023-10-07 23:08:33,169][67871] Updated weights for policy 1, policy_version 81990 (0.0011) [2023-10-07 23:08:33,537][67871] Updated weights for policy 1, policy_version 82000 (0.0009) [2023-10-07 23:08:33,905][67871] Updated weights for policy 1, policy_version 82010 (0.0009) [2023-10-07 23:08:35,181][67838] Updated weights for policy 0, policy_version 81892 (0.0010) [2023-10-07 23:08:35,544][67838] Updated weights for policy 0, policy_version 81902 (0.0009) [2023-10-07 23:08:35,914][67838] Updated weights for policy 0, policy_version 81912 (0.0011) [2023-10-07 23:08:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 167870464. Throughput: 0: 1664.6, 1: 1656.1. Samples: 41977900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:08:37,477][66916] Avg episode reward: [(0, '51.240'), (1, '59.100')] [2023-10-07 23:08:38,140][67871] Updated weights for policy 1, policy_version 82020 (0.0009) [2023-10-07 23:08:38,505][67871] Updated weights for policy 1, policy_version 82030 (0.0007) [2023-10-07 23:08:38,863][67871] Updated weights for policy 1, policy_version 82040 (0.0008) [2023-10-07 23:08:39,917][67838] Updated weights for policy 0, policy_version 81922 (0.0007) [2023-10-07 23:08:40,283][67838] Updated weights for policy 0, policy_version 81932 (0.0008) [2023-10-07 23:08:40,664][67838] Updated weights for policy 0, policy_version 81942 (0.0008) [2023-10-07 23:08:41,035][67838] Updated weights for policy 0, policy_version 81952 (0.0009) [2023-10-07 23:08:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 167936000. Throughput: 0: 1666.8, 1: 1660.3. Samples: 41988312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:08:42,478][66916] Avg episode reward: [(0, '52.230'), (1, '60.100')] [2023-10-07 23:08:42,967][67871] Updated weights for policy 1, policy_version 82050 (0.0010) [2023-10-07 23:08:43,343][67871] Updated weights for policy 1, policy_version 82060 (0.0007) [2023-10-07 23:08:43,711][67871] Updated weights for policy 1, policy_version 82070 (0.0008) [2023-10-07 23:08:44,073][67871] Updated weights for policy 1, policy_version 82080 (0.0008) [2023-10-07 23:08:45,429][67838] Updated weights for policy 0, policy_version 81962 (0.0009) [2023-10-07 23:08:45,798][67838] Updated weights for policy 0, policy_version 81972 (0.0007) [2023-10-07 23:08:46,177][67838] Updated weights for policy 0, policy_version 81982 (0.0008) [2023-10-07 23:08:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 168001536. Throughput: 0: 1653.7, 1: 1660.8. Samples: 42007688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:08:47,477][66916] Avg episode reward: [(0, '54.960'), (1, '59.900')] [2023-10-07 23:08:48,246][67871] Updated weights for policy 1, policy_version 82090 (0.0009) [2023-10-07 23:08:48,613][67871] Updated weights for policy 1, policy_version 82100 (0.0007) [2023-10-07 23:08:48,982][67871] Updated weights for policy 1, policy_version 82110 (0.0008) [2023-10-07 23:08:50,303][67838] Updated weights for policy 0, policy_version 81992 (0.0009) [2023-10-07 23:08:50,685][67838] Updated weights for policy 0, policy_version 82002 (0.0009) [2023-10-07 23:08:51,056][67838] Updated weights for policy 0, policy_version 82012 (0.0011) [2023-10-07 23:08:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 168067072. Throughput: 0: 1665.0, 1: 1658.7. Samples: 42027622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:08:52,478][66916] Avg episode reward: [(0, '53.690'), (1, '59.480')] [2023-10-07 23:08:53,102][67871] Updated weights for policy 1, policy_version 82120 (0.0008) [2023-10-07 23:08:53,463][67871] Updated weights for policy 1, policy_version 82130 (0.0012) [2023-10-07 23:08:53,838][67871] Updated weights for policy 1, policy_version 82140 (0.0008) [2023-10-07 23:08:55,267][67838] Updated weights for policy 0, policy_version 82022 (0.0010) [2023-10-07 23:08:55,633][67838] Updated weights for policy 0, policy_version 82032 (0.0011) [2023-10-07 23:08:56,017][67838] Updated weights for policy 0, policy_version 82042 (0.0009) [2023-10-07 23:08:57,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 168132608. Throughput: 0: 1667.4, 1: 1662.8. Samples: 42038080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:08:57,478][66916] Avg episode reward: [(0, '49.920'), (1, '58.330')] [2023-10-07 23:08:57,875][67871] Updated weights for policy 1, policy_version 82150 (0.0008) [2023-10-07 23:08:58,236][67871] Updated weights for policy 1, policy_version 82160 (0.0009) [2023-10-07 23:08:58,607][67871] Updated weights for policy 1, policy_version 82170 (0.0008) [2023-10-07 23:09:00,118][67838] Updated weights for policy 0, policy_version 82052 (0.0008) [2023-10-07 23:09:00,482][67838] Updated weights for policy 0, policy_version 82062 (0.0007) [2023-10-07 23:09:00,859][67838] Updated weights for policy 0, policy_version 82072 (0.0007) [2023-10-07 23:09:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 168198144. Throughput: 0: 1647.1, 1: 1672.4. Samples: 42057496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:09:02,478][66916] Avg episode reward: [(0, '50.730'), (1, '65.050')] [2023-10-07 23:09:02,702][67871] Updated weights for policy 1, policy_version 82180 (0.0008) [2023-10-07 23:09:03,069][67871] Updated weights for policy 1, policy_version 82190 (0.0007) [2023-10-07 23:09:03,447][67871] Updated weights for policy 1, policy_version 82200 (0.0007) [2023-10-07 23:09:04,978][67838] Updated weights for policy 0, policy_version 82082 (0.0008) [2023-10-07 23:09:05,357][67838] Updated weights for policy 0, policy_version 82092 (0.0008) [2023-10-07 23:09:05,724][67838] Updated weights for policy 0, policy_version 82102 (0.0007) [2023-10-07 23:09:06,089][67838] Updated weights for policy 0, policy_version 82112 (0.0007) [2023-10-07 23:09:07,465][67871] Updated weights for policy 1, policy_version 82210 (0.0008) [2023-10-07 23:09:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 168263680. Throughput: 0: 1664.0, 1: 1676.8. Samples: 42078188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:09:07,477][66916] Avg episode reward: [(0, '50.740'), (1, '63.000')] [2023-10-07 23:09:07,487][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000082112_84082688.pth... [2023-10-07 23:09:07,523][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000080544_82477056.pth [2023-10-07 23:09:07,837][67871] Updated weights for policy 1, policy_version 82220 (0.0010) [2023-10-07 23:09:08,201][67871] Updated weights for policy 1, policy_version 82230 (0.0010) [2023-10-07 23:09:08,572][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000082240_84213760.pth... [2023-10-07 23:09:08,576][67871] Updated weights for policy 1, policy_version 82240 (0.0010) [2023-10-07 23:09:08,611][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000080672_82608128.pth [2023-10-07 23:09:10,062][67838] Updated weights for policy 0, policy_version 82122 (0.0009) [2023-10-07 23:09:10,428][67838] Updated weights for policy 0, policy_version 82132 (0.0009) [2023-10-07 23:09:10,793][67838] Updated weights for policy 0, policy_version 82142 (0.0008) [2023-10-07 23:09:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 168329216. Throughput: 0: 1659.6, 1: 1676.5. Samples: 42088244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:09:12,478][66916] Avg episode reward: [(0, '49.150'), (1, '65.420')] [2023-10-07 23:09:12,631][67871] Updated weights for policy 1, policy_version 82250 (0.0008) [2023-10-07 23:09:13,001][67871] Updated weights for policy 1, policy_version 82260 (0.0009) [2023-10-07 23:09:13,370][67871] Updated weights for policy 1, policy_version 82270 (0.0009) [2023-10-07 23:09:14,959][67838] Updated weights for policy 0, policy_version 82152 (0.0009) [2023-10-07 23:09:15,329][67838] Updated weights for policy 0, policy_version 82162 (0.0007) [2023-10-07 23:09:15,706][67838] Updated weights for policy 0, policy_version 82172 (0.0008) [2023-10-07 23:09:17,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 168394752. Throughput: 0: 1652.1, 1: 1680.2. Samples: 42107764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:09:17,478][66916] Avg episode reward: [(0, '46.750'), (1, '64.170')] [2023-10-07 23:09:17,569][67871] Updated weights for policy 1, policy_version 82280 (0.0007) [2023-10-07 23:09:17,943][67871] Updated weights for policy 1, policy_version 82290 (0.0008) [2023-10-07 23:09:18,301][67871] Updated weights for policy 1, policy_version 82300 (0.0008) [2023-10-07 23:09:19,939][67838] Updated weights for policy 0, policy_version 82182 (0.0008) [2023-10-07 23:09:20,311][67838] Updated weights for policy 0, policy_version 82192 (0.0009) [2023-10-07 23:09:20,689][67838] Updated weights for policy 0, policy_version 82202 (0.0008) [2023-10-07 23:09:22,409][67871] Updated weights for policy 1, policy_version 82310 (0.0007) [2023-10-07 23:09:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 168460288. Throughput: 0: 1660.7, 1: 1678.9. Samples: 42128184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:09:22,477][66916] Avg episode reward: [(0, '48.460'), (1, '64.420')] [2023-10-07 23:09:22,773][67871] Updated weights for policy 1, policy_version 82320 (0.0008) [2023-10-07 23:09:23,137][67871] Updated weights for policy 1, policy_version 82330 (0.0007) [2023-10-07 23:09:24,637][67838] Updated weights for policy 0, policy_version 82212 (0.0009) [2023-10-07 23:09:25,023][67838] Updated weights for policy 0, policy_version 82222 (0.0008) [2023-10-07 23:09:25,395][67838] Updated weights for policy 0, policy_version 82232 (0.0009) [2023-10-07 23:09:27,206][67871] Updated weights for policy 1, policy_version 82340 (0.0009) [2023-10-07 23:09:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 168525824. Throughput: 0: 1647.8, 1: 1677.8. Samples: 42137966. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:09:27,477][66916] Avg episode reward: [(0, '49.490'), (1, '59.510')] [2023-10-07 23:09:27,568][67871] Updated weights for policy 1, policy_version 82350 (0.0009) [2023-10-07 23:09:27,935][67871] Updated weights for policy 1, policy_version 82360 (0.0010) [2023-10-07 23:09:29,562][67838] Updated weights for policy 0, policy_version 82242 (0.0008) [2023-10-07 23:09:29,937][67838] Updated weights for policy 0, policy_version 82252 (0.0008) [2023-10-07 23:09:30,309][67838] Updated weights for policy 0, policy_version 82262 (0.0009) [2023-10-07 23:09:30,684][67838] Updated weights for policy 0, policy_version 82272 (0.0007) [2023-10-07 23:09:32,082][67871] Updated weights for policy 1, policy_version 82370 (0.0007) [2023-10-07 23:09:32,450][67871] Updated weights for policy 1, policy_version 82380 (0.0007) [2023-10-07 23:09:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 168591360. Throughput: 0: 1654.5, 1: 1677.1. Samples: 42157610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:09:32,477][66916] Avg episode reward: [(0, '48.330'), (1, '60.550')] [2023-10-07 23:09:32,813][67871] Updated weights for policy 1, policy_version 82390 (0.0007) [2023-10-07 23:09:33,174][67871] Updated weights for policy 1, policy_version 82400 (0.0009) [2023-10-07 23:09:34,753][67838] Updated weights for policy 0, policy_version 82282 (0.0007) [2023-10-07 23:09:35,128][67838] Updated weights for policy 0, policy_version 82292 (0.0007) [2023-10-07 23:09:35,496][67838] Updated weights for policy 0, policy_version 82302 (0.0009) [2023-10-07 23:09:37,325][67871] Updated weights for policy 1, policy_version 82410 (0.0007) [2023-10-07 23:09:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 168656896. Throughput: 0: 1666.7, 1: 1678.8. Samples: 42178170. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:09:37,477][66916] Avg episode reward: [(0, '48.720'), (1, '60.770')] [2023-10-07 23:09:37,700][67871] Updated weights for policy 1, policy_version 82420 (0.0007) [2023-10-07 23:09:38,074][67871] Updated weights for policy 1, policy_version 82430 (0.0008) [2023-10-07 23:09:39,544][67838] Updated weights for policy 0, policy_version 82312 (0.0010) [2023-10-07 23:09:39,922][67838] Updated weights for policy 0, policy_version 82322 (0.0011) [2023-10-07 23:09:40,303][67838] Updated weights for policy 0, policy_version 82332 (0.0010) [2023-10-07 23:09:42,181][67871] Updated weights for policy 1, policy_version 82440 (0.0008) [2023-10-07 23:09:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 168722432. Throughput: 0: 1647.2, 1: 1673.7. Samples: 42187520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:09:42,477][66916] Avg episode reward: [(0, '52.530'), (1, '57.540')] [2023-10-07 23:09:42,553][67871] Updated weights for policy 1, policy_version 82450 (0.0007) [2023-10-07 23:09:42,922][67871] Updated weights for policy 1, policy_version 82460 (0.0007) [2023-10-07 23:09:44,576][67838] Updated weights for policy 0, policy_version 82342 (0.0009) [2023-10-07 23:09:44,956][67838] Updated weights for policy 0, policy_version 82352 (0.0008) [2023-10-07 23:09:45,319][67838] Updated weights for policy 0, policy_version 82362 (0.0008) [2023-10-07 23:09:47,103][67871] Updated weights for policy 1, policy_version 82470 (0.0008) [2023-10-07 23:09:47,468][67871] Updated weights for policy 1, policy_version 82480 (0.0009) [2023-10-07 23:09:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 168787968. Throughput: 0: 1663.8, 1: 1668.1. Samples: 42207432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:09:47,477][66916] Avg episode reward: [(0, '51.580'), (1, '57.500')] [2023-10-07 23:09:47,833][67871] Updated weights for policy 1, policy_version 82490 (0.0008) [2023-10-07 23:09:49,341][67838] Updated weights for policy 0, policy_version 82372 (0.0007) [2023-10-07 23:09:49,710][67838] Updated weights for policy 0, policy_version 82382 (0.0009) [2023-10-07 23:09:50,088][67838] Updated weights for policy 0, policy_version 82392 (0.0012) [2023-10-07 23:09:51,949][67871] Updated weights for policy 1, policy_version 82500 (0.0008) [2023-10-07 23:09:52,304][67871] Updated weights for policy 1, policy_version 82510 (0.0007) [2023-10-07 23:09:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 168853504. Throughput: 0: 1663.0, 1: 1660.1. Samples: 42227726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:09:52,477][66916] Avg episode reward: [(0, '50.530'), (1, '59.850')] [2023-10-07 23:09:52,673][67871] Updated weights for policy 1, policy_version 82520 (0.0008) [2023-10-07 23:09:54,224][67838] Updated weights for policy 0, policy_version 82402 (0.0007) [2023-10-07 23:09:54,602][67838] Updated weights for policy 0, policy_version 82412 (0.0007) [2023-10-07 23:09:54,969][67838] Updated weights for policy 0, policy_version 82422 (0.0007) [2023-10-07 23:09:55,347][67838] Updated weights for policy 0, policy_version 82432 (0.0007) [2023-10-07 23:09:56,754][67871] Updated weights for policy 1, policy_version 82530 (0.0008) [2023-10-07 23:09:57,122][67871] Updated weights for policy 1, policy_version 82540 (0.0007) [2023-10-07 23:09:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 168919040. Throughput: 0: 1651.4, 1: 1666.6. Samples: 42237554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:09:57,477][66916] Avg episode reward: [(0, '50.150'), (1, '61.000')] [2023-10-07 23:09:57,481][67871] Updated weights for policy 1, policy_version 82550 (0.0007) [2023-10-07 23:09:57,847][67871] Updated weights for policy 1, policy_version 82560 (0.0009) [2023-10-07 23:09:59,403][67838] Updated weights for policy 0, policy_version 82442 (0.0010) [2023-10-07 23:09:59,779][67838] Updated weights for policy 0, policy_version 82452 (0.0009) [2023-10-07 23:10:00,151][67838] Updated weights for policy 0, policy_version 82462 (0.0010) [2023-10-07 23:10:01,882][67871] Updated weights for policy 1, policy_version 82570 (0.0009) [2023-10-07 23:10:02,243][67871] Updated weights for policy 1, policy_version 82580 (0.0009) [2023-10-07 23:10:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 168984576. Throughput: 0: 1667.4, 1: 1663.2. Samples: 42257640. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-07 23:10:02,477][66916] Avg episode reward: [(0, '50.600'), (1, '61.200')] [2023-10-07 23:10:02,615][67871] Updated weights for policy 1, policy_version 82590 (0.0009) [2023-10-07 23:10:04,369][67838] Updated weights for policy 0, policy_version 82472 (0.0009) [2023-10-07 23:10:04,738][67838] Updated weights for policy 0, policy_version 82482 (0.0009) [2023-10-07 23:10:05,116][67838] Updated weights for policy 0, policy_version 82492 (0.0010) [2023-10-07 23:10:06,730][67871] Updated weights for policy 1, policy_version 82600 (0.0010) [2023-10-07 23:10:07,094][67871] Updated weights for policy 1, policy_version 82610 (0.0009) [2023-10-07 23:10:07,453][67871] Updated weights for policy 1, policy_version 82620 (0.0009) [2023-10-07 23:10:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 169050112. Throughput: 0: 1670.3, 1: 1657.0. Samples: 42277912. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-07 23:10:07,477][66916] Avg episode reward: [(0, '49.990'), (1, '60.510')] [2023-10-07 23:10:09,092][67838] Updated weights for policy 0, policy_version 82502 (0.0008) [2023-10-07 23:10:09,472][67838] Updated weights for policy 0, policy_version 82512 (0.0007) [2023-10-07 23:10:09,848][67838] Updated weights for policy 0, policy_version 82522 (0.0010) [2023-10-07 23:10:11,655][67871] Updated weights for policy 1, policy_version 82630 (0.0008) [2023-10-07 23:10:12,026][67871] Updated weights for policy 1, policy_version 82640 (0.0008) [2023-10-07 23:10:12,401][67871] Updated weights for policy 1, policy_version 82650 (0.0008) [2023-10-07 23:10:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 169115648. Throughput: 0: 1654.2, 1: 1667.2. Samples: 42287428. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-07 23:10:12,478][66916] Avg episode reward: [(0, '50.950'), (1, '65.620')] [2023-10-07 23:10:13,841][67838] Updated weights for policy 0, policy_version 82532 (0.0008) [2023-10-07 23:10:14,214][67838] Updated weights for policy 0, policy_version 82542 (0.0008) [2023-10-07 23:10:14,587][67838] Updated weights for policy 0, policy_version 82552 (0.0009) [2023-10-07 23:10:16,410][67871] Updated weights for policy 1, policy_version 82660 (0.0010) [2023-10-07 23:10:16,785][67871] Updated weights for policy 1, policy_version 82670 (0.0008) [2023-10-07 23:10:17,144][67871] Updated weights for policy 1, policy_version 82680 (0.0008) [2023-10-07 23:10:17,477][66916] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 169213952. Throughput: 0: 1671.9, 1: 1664.6. Samples: 42307752. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-07 23:10:17,478][66916] Avg episode reward: [(0, '52.390'), (1, '65.440')] [2023-10-07 23:10:18,813][67838] Updated weights for policy 0, policy_version 82562 (0.0009) [2023-10-07 23:10:19,187][67838] Updated weights for policy 0, policy_version 82572 (0.0011) [2023-10-07 23:10:19,569][67838] Updated weights for policy 0, policy_version 82582 (0.0009) [2023-10-07 23:10:19,932][67838] Updated weights for policy 0, policy_version 82592 (0.0007) [2023-10-07 23:10:21,177][67871] Updated weights for policy 1, policy_version 82690 (0.0008) [2023-10-07 23:10:21,590][67871] Updated weights for policy 1, policy_version 82700 (0.0009) [2023-10-07 23:10:21,948][67871] Updated weights for policy 1, policy_version 82710 (0.0010) [2023-10-07 23:10:22,313][67871] Updated weights for policy 1, policy_version 82720 (0.0008) [2023-10-07 23:10:22,476][66916] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 169279488. Throughput: 0: 1670.8, 1: 1652.3. Samples: 42327706. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-07 23:10:22,477][66916] Avg episode reward: [(0, '49.510'), (1, '66.410')] [2023-10-07 23:10:23,986][67838] Updated weights for policy 0, policy_version 82602 (0.0008) [2023-10-07 23:10:24,358][67838] Updated weights for policy 0, policy_version 82612 (0.0009) [2023-10-07 23:10:24,726][67838] Updated weights for policy 0, policy_version 82622 (0.0009) [2023-10-07 23:10:26,331][67871] Updated weights for policy 1, policy_version 82730 (0.0008) [2023-10-07 23:10:26,697][67871] Updated weights for policy 1, policy_version 82740 (0.0009) [2023-10-07 23:10:27,069][67871] Updated weights for policy 1, policy_version 82750 (0.0008) [2023-10-07 23:10:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 169345024. Throughput: 0: 1658.6, 1: 1672.9. Samples: 42337436. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-07 23:10:27,477][66916] Avg episode reward: [(0, '48.250'), (1, '65.600')] [2023-10-07 23:10:28,864][67838] Updated weights for policy 0, policy_version 82632 (0.0008) [2023-10-07 23:10:29,230][67838] Updated weights for policy 0, policy_version 82642 (0.0010) [2023-10-07 23:10:29,611][67838] Updated weights for policy 0, policy_version 82652 (0.0008) [2023-10-07 23:10:31,242][67871] Updated weights for policy 1, policy_version 82760 (0.0008) [2023-10-07 23:10:31,606][67871] Updated weights for policy 1, policy_version 82770 (0.0007) [2023-10-07 23:10:31,964][67871] Updated weights for policy 1, policy_version 82780 (0.0008) [2023-10-07 23:10:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 169410560. Throughput: 0: 1662.3, 1: 1673.4. Samples: 42357538. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-07 23:10:32,477][66916] Avg episode reward: [(0, '47.280'), (1, '65.990')] [2023-10-07 23:10:33,666][67838] Updated weights for policy 0, policy_version 82662 (0.0010) [2023-10-07 23:10:34,032][67838] Updated weights for policy 0, policy_version 82672 (0.0007) [2023-10-07 23:10:34,405][67838] Updated weights for policy 0, policy_version 82682 (0.0007) [2023-10-07 23:10:36,065][67871] Updated weights for policy 1, policy_version 82790 (0.0008) [2023-10-07 23:10:36,440][67871] Updated weights for policy 1, policy_version 82800 (0.0008) [2023-10-07 23:10:36,804][67871] Updated weights for policy 1, policy_version 82810 (0.0008) [2023-10-07 23:10:37,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 169476096. Throughput: 0: 1671.5, 1: 1652.7. Samples: 42377312. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-07 23:10:37,477][66916] Avg episode reward: [(0, '45.420'), (1, '59.820')] [2023-10-07 23:10:38,391][67838] Updated weights for policy 0, policy_version 82692 (0.0007) [2023-10-07 23:10:38,777][67838] Updated weights for policy 0, policy_version 82702 (0.0010) [2023-10-07 23:10:39,154][67838] Updated weights for policy 0, policy_version 82712 (0.0009) [2023-10-07 23:10:41,002][67871] Updated weights for policy 1, policy_version 82820 (0.0009) [2023-10-07 23:10:41,363][67871] Updated weights for policy 1, policy_version 82830 (0.0008) [2023-10-07 23:10:41,726][67871] Updated weights for policy 1, policy_version 82840 (0.0009) [2023-10-07 23:10:42,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 169541632. Throughput: 0: 1655.6, 1: 1670.1. Samples: 42387210. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-07 23:10:42,478][66916] Avg episode reward: [(0, '45.240'), (1, '58.390')] [2023-10-07 23:10:43,358][67838] Updated weights for policy 0, policy_version 82722 (0.0009) [2023-10-07 23:10:43,731][67838] Updated weights for policy 0, policy_version 82732 (0.0008) [2023-10-07 23:10:44,108][67838] Updated weights for policy 0, policy_version 82742 (0.0009) [2023-10-07 23:10:44,470][67838] Updated weights for policy 0, policy_version 82752 (0.0008) [2023-10-07 23:10:45,873][67871] Updated weights for policy 1, policy_version 82850 (0.0009) [2023-10-07 23:10:46,243][67871] Updated weights for policy 1, policy_version 82860 (0.0007) [2023-10-07 23:10:46,618][67871] Updated weights for policy 1, policy_version 82870 (0.0007) [2023-10-07 23:10:46,971][67871] Updated weights for policy 1, policy_version 82880 (0.0009) [2023-10-07 23:10:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 169607168. Throughput: 0: 1661.6, 1: 1669.8. Samples: 42407552. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-07 23:10:47,477][66916] Avg episode reward: [(0, '48.820'), (1, '58.000')] [2023-10-07 23:10:48,670][67838] Updated weights for policy 0, policy_version 82762 (0.0008) [2023-10-07 23:10:49,040][67838] Updated weights for policy 0, policy_version 82772 (0.0008) [2023-10-07 23:10:49,413][67838] Updated weights for policy 0, policy_version 82782 (0.0009) [2023-10-07 23:10:51,110][67871] Updated weights for policy 1, policy_version 82890 (0.0007) [2023-10-07 23:10:51,472][67871] Updated weights for policy 1, policy_version 82900 (0.0007) [2023-10-07 23:10:51,838][67871] Updated weights for policy 1, policy_version 82910 (0.0007) [2023-10-07 23:10:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 169672704. Throughput: 0: 1658.8, 1: 1651.4. Samples: 42426872. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-07 23:10:52,477][66916] Avg episode reward: [(0, '51.800'), (1, '59.080')] [2023-10-07 23:10:53,648][67838] Updated weights for policy 0, policy_version 82792 (0.0009) [2023-10-07 23:10:54,020][67838] Updated weights for policy 0, policy_version 82802 (0.0008) [2023-10-07 23:10:54,392][67838] Updated weights for policy 0, policy_version 82812 (0.0009) [2023-10-07 23:10:55,942][67871] Updated weights for policy 1, policy_version 82920 (0.0007) [2023-10-07 23:10:56,311][67871] Updated weights for policy 1, policy_version 82930 (0.0007) [2023-10-07 23:10:56,691][67871] Updated weights for policy 1, policy_version 82940 (0.0007) [2023-10-07 23:10:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 169738240. Throughput: 0: 1657.6, 1: 1666.3. Samples: 42437006. Policy #0 lag: (min: 25.0, avg: 25.7, max: 44.0) [2023-10-07 23:10:57,478][66916] Avg episode reward: [(0, '53.030'), (1, '58.030')] [2023-10-07 23:10:58,406][67838] Updated weights for policy 0, policy_version 82822 (0.0007) [2023-10-07 23:10:58,794][67838] Updated weights for policy 0, policy_version 82832 (0.0009) [2023-10-07 23:10:59,168][67838] Updated weights for policy 0, policy_version 82842 (0.0008) [2023-10-07 23:11:00,756][67871] Updated weights for policy 1, policy_version 82950 (0.0009) [2023-10-07 23:11:01,130][67871] Updated weights for policy 1, policy_version 82960 (0.0009) [2023-10-07 23:11:01,491][67871] Updated weights for policy 1, policy_version 82970 (0.0007) [2023-10-07 23:11:02,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 169803776. Throughput: 0: 1654.3, 1: 1661.6. Samples: 42456964. Policy #0 lag: (min: 25.0, avg: 25.7, max: 44.0) [2023-10-07 23:11:02,478][66916] Avg episode reward: [(0, '51.270'), (1, '62.110')] [2023-10-07 23:11:03,349][67838] Updated weights for policy 0, policy_version 82852 (0.0007) [2023-10-07 23:11:03,727][67838] Updated weights for policy 0, policy_version 82862 (0.0007) [2023-10-07 23:11:04,093][67838] Updated weights for policy 0, policy_version 82872 (0.0010) [2023-10-07 23:11:05,481][67871] Updated weights for policy 1, policy_version 82980 (0.0009) [2023-10-07 23:11:05,857][67871] Updated weights for policy 1, policy_version 82990 (0.0008) [2023-10-07 23:11:06,212][67871] Updated weights for policy 1, policy_version 83000 (0.0011) [2023-10-07 23:11:07,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 169869312. Throughput: 0: 1651.4, 1: 1654.8. Samples: 42476486. Policy #0 lag: (min: 25.0, avg: 25.7, max: 44.0) [2023-10-07 23:11:07,477][66916] Avg episode reward: [(0, '50.460'), (1, '62.980')] [2023-10-07 23:11:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000082880_84869120.pth... [2023-10-07 23:11:07,490][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000083008_85000192.pth... [2023-10-07 23:11:07,521][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000081440_83394560.pth [2023-10-07 23:11:07,525][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000081344_83296256.pth [2023-10-07 23:11:08,462][67838] Updated weights for policy 0, policy_version 82882 (0.0010) [2023-10-07 23:11:08,832][67838] Updated weights for policy 0, policy_version 82892 (0.0011) [2023-10-07 23:11:09,199][67838] Updated weights for policy 0, policy_version 82902 (0.0010) [2023-10-07 23:11:09,571][67838] Updated weights for policy 0, policy_version 82912 (0.0008) [2023-10-07 23:11:10,394][67871] Updated weights for policy 1, policy_version 83010 (0.0009) [2023-10-07 23:11:10,796][67871] Updated weights for policy 1, policy_version 83020 (0.0009) [2023-10-07 23:11:11,156][67871] Updated weights for policy 1, policy_version 83030 (0.0008) [2023-10-07 23:11:11,526][67871] Updated weights for policy 1, policy_version 83040 (0.0008) [2023-10-07 23:11:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 169934848. Throughput: 0: 1653.2, 1: 1665.6. Samples: 42486782. Policy #0 lag: (min: 25.0, avg: 25.7, max: 44.0) [2023-10-07 23:11:12,477][66916] Avg episode reward: [(0, '49.490'), (1, '61.140')] [2023-10-07 23:11:13,726][67838] Updated weights for policy 0, policy_version 82922 (0.0008) [2023-10-07 23:11:14,091][67838] Updated weights for policy 0, policy_version 82932 (0.0010) [2023-10-07 23:11:14,467][67838] Updated weights for policy 0, policy_version 82942 (0.0009) [2023-10-07 23:11:15,688][67871] Updated weights for policy 1, policy_version 83050 (0.0009) [2023-10-07 23:11:16,053][67871] Updated weights for policy 1, policy_version 83060 (0.0007) [2023-10-07 23:11:16,425][67871] Updated weights for policy 1, policy_version 83070 (0.0008) [2023-10-07 23:11:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170000384. Throughput: 0: 1661.6, 1: 1656.0. Samples: 42506834. Policy #0 lag: (min: 25.0, avg: 25.7, max: 44.0) [2023-10-07 23:11:17,477][66916] Avg episode reward: [(0, '45.800'), (1, '62.120')] [2023-10-07 23:11:18,571][67838] Updated weights for policy 0, policy_version 82952 (0.0008) [2023-10-07 23:11:18,948][67838] Updated weights for policy 0, policy_version 82962 (0.0009) [2023-10-07 23:11:19,322][67838] Updated weights for policy 0, policy_version 82972 (0.0008) [2023-10-07 23:11:20,527][67871] Updated weights for policy 1, policy_version 83080 (0.0011) [2023-10-07 23:11:20,898][67871] Updated weights for policy 1, policy_version 83090 (0.0009) [2023-10-07 23:11:21,259][67871] Updated weights for policy 1, policy_version 83100 (0.0007) [2023-10-07 23:11:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 170065920. Throughput: 0: 1651.5, 1: 1666.4. Samples: 42526616. Policy #0 lag: (min: 25.0, avg: 25.7, max: 44.0) [2023-10-07 23:11:22,478][66916] Avg episode reward: [(0, '44.560'), (1, '61.610')] [2023-10-07 23:11:23,497][67838] Updated weights for policy 0, policy_version 82982 (0.0010) [2023-10-07 23:11:23,869][67838] Updated weights for policy 0, policy_version 82992 (0.0008) [2023-10-07 23:11:24,243][67838] Updated weights for policy 0, policy_version 83002 (0.0010) [2023-10-07 23:11:25,238][67871] Updated weights for policy 1, policy_version 83110 (0.0008) [2023-10-07 23:11:25,607][67871] Updated weights for policy 1, policy_version 83120 (0.0008) [2023-10-07 23:11:25,970][67871] Updated weights for policy 1, policy_version 83130 (0.0007) [2023-10-07 23:11:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170131456. Throughput: 0: 1653.5, 1: 1675.2. Samples: 42537002. Policy #0 lag: (min: 25.0, avg: 25.7, max: 44.0) [2023-10-07 23:11:27,477][66916] Avg episode reward: [(0, '43.880'), (1, '62.250')] [2023-10-07 23:11:28,227][67838] Updated weights for policy 0, policy_version 83012 (0.0010) [2023-10-07 23:11:28,594][67838] Updated weights for policy 0, policy_version 83022 (0.0009) [2023-10-07 23:11:28,958][67838] Updated weights for policy 0, policy_version 83032 (0.0009) [2023-10-07 23:11:29,923][67871] Updated weights for policy 1, policy_version 83140 (0.0007) [2023-10-07 23:11:30,286][67871] Updated weights for policy 1, policy_version 83150 (0.0009) [2023-10-07 23:11:30,651][67871] Updated weights for policy 1, policy_version 83160 (0.0009) [2023-10-07 23:11:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170196992. Throughput: 0: 1657.7, 1: 1654.0. Samples: 42556580. Policy #0 lag: (min: 25.0, avg: 25.7, max: 44.0) [2023-10-07 23:11:32,478][66916] Avg episode reward: [(0, '42.000'), (1, '61.020')] [2023-10-07 23:11:32,893][67838] Updated weights for policy 0, policy_version 83042 (0.0009) [2023-10-07 23:11:33,263][67838] Updated weights for policy 0, policy_version 83052 (0.0009) [2023-10-07 23:11:33,638][67838] Updated weights for policy 0, policy_version 83062 (0.0007) [2023-10-07 23:11:34,005][67838] Updated weights for policy 0, policy_version 83072 (0.0008) [2023-10-07 23:11:34,824][67871] Updated weights for policy 1, policy_version 83170 (0.0010) [2023-10-07 23:11:35,201][67871] Updated weights for policy 1, policy_version 83180 (0.0010) [2023-10-07 23:11:35,554][67871] Updated weights for policy 1, policy_version 83190 (0.0009) [2023-10-07 23:11:35,922][67871] Updated weights for policy 1, policy_version 83200 (0.0009) [2023-10-07 23:11:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170262528. Throughput: 0: 1660.5, 1: 1673.0. Samples: 42576882. Policy #0 lag: (min: 25.0, avg: 25.7, max: 44.0) [2023-10-07 23:11:37,478][66916] Avg episode reward: [(0, '41.140'), (1, '58.780')] [2023-10-07 23:11:38,041][67838] Updated weights for policy 0, policy_version 83082 (0.0011) [2023-10-07 23:11:38,408][67838] Updated weights for policy 0, policy_version 83092 (0.0010) [2023-10-07 23:11:38,798][67838] Updated weights for policy 0, policy_version 83102 (0.0011) [2023-10-07 23:11:39,955][67871] Updated weights for policy 1, policy_version 83210 (0.0007) [2023-10-07 23:11:40,322][67871] Updated weights for policy 1, policy_version 83220 (0.0008) [2023-10-07 23:11:40,696][67871] Updated weights for policy 1, policy_version 83230 (0.0008) [2023-10-07 23:11:42,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170328064. Throughput: 0: 1657.7, 1: 1669.8. Samples: 42586744. Policy #0 lag: (min: 25.0, avg: 25.7, max: 44.0) [2023-10-07 23:11:42,478][66916] Avg episode reward: [(0, '44.650'), (1, '56.380')] [2023-10-07 23:11:43,251][67838] Updated weights for policy 0, policy_version 83112 (0.0007) [2023-10-07 23:11:43,620][67838] Updated weights for policy 0, policy_version 83122 (0.0010) [2023-10-07 23:11:43,997][67838] Updated weights for policy 0, policy_version 83132 (0.0007) [2023-10-07 23:11:44,877][67871] Updated weights for policy 1, policy_version 83240 (0.0008) [2023-10-07 23:11:45,247][67871] Updated weights for policy 1, policy_version 83250 (0.0008) [2023-10-07 23:11:45,608][67871] Updated weights for policy 1, policy_version 83260 (0.0007) [2023-10-07 23:11:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170393600. Throughput: 0: 1663.2, 1: 1657.2. Samples: 42606380. Policy #0 lag: (min: 25.0, avg: 25.7, max: 44.0) [2023-10-07 23:11:47,477][66916] Avg episode reward: [(0, '42.670'), (1, '59.440')] [2023-10-07 23:11:48,030][67838] Updated weights for policy 0, policy_version 83142 (0.0009) [2023-10-07 23:11:48,404][67838] Updated weights for policy 0, policy_version 83152 (0.0011) [2023-10-07 23:11:48,782][67838] Updated weights for policy 0, policy_version 83162 (0.0010) [2023-10-07 23:11:49,665][67871] Updated weights for policy 1, policy_version 83270 (0.0009) [2023-10-07 23:11:50,036][67871] Updated weights for policy 1, policy_version 83280 (0.0007) [2023-10-07 23:11:50,402][67871] Updated weights for policy 1, policy_version 83290 (0.0010) [2023-10-07 23:11:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170459136. Throughput: 0: 1664.8, 1: 1679.5. Samples: 42626980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:11:52,477][66916] Avg episode reward: [(0, '42.480'), (1, '58.770')] [2023-10-07 23:11:52,949][67838] Updated weights for policy 0, policy_version 83172 (0.0008) [2023-10-07 23:11:53,321][67838] Updated weights for policy 0, policy_version 83182 (0.0011) [2023-10-07 23:11:53,692][67838] Updated weights for policy 0, policy_version 83192 (0.0007) [2023-10-07 23:11:54,567][67871] Updated weights for policy 1, policy_version 83300 (0.0009) [2023-10-07 23:11:54,932][67871] Updated weights for policy 1, policy_version 83310 (0.0009) [2023-10-07 23:11:55,289][67871] Updated weights for policy 1, policy_version 83320 (0.0009) [2023-10-07 23:11:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170524672. Throughput: 0: 1664.9, 1: 1669.3. Samples: 42636820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:11:57,477][66916] Avg episode reward: [(0, '47.780'), (1, '57.360')] [2023-10-07 23:11:57,941][67838] Updated weights for policy 0, policy_version 83202 (0.0009) [2023-10-07 23:11:58,321][67838] Updated weights for policy 0, policy_version 83212 (0.0007) [2023-10-07 23:11:58,687][67838] Updated weights for policy 0, policy_version 83222 (0.0008) [2023-10-07 23:11:59,055][67838] Updated weights for policy 0, policy_version 83232 (0.0009) [2023-10-07 23:11:59,377][67871] Updated weights for policy 1, policy_version 83330 (0.0009) [2023-10-07 23:11:59,798][67871] Updated weights for policy 1, policy_version 83340 (0.0007) [2023-10-07 23:12:00,157][67871] Updated weights for policy 1, policy_version 83350 (0.0008) [2023-10-07 23:12:00,528][67871] Updated weights for policy 1, policy_version 83360 (0.0008) [2023-10-07 23:12:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 170590208. Throughput: 0: 1661.1, 1: 1659.7. Samples: 42656270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:12:02,478][66916] Avg episode reward: [(0, '45.170'), (1, '60.370')] [2023-10-07 23:12:03,335][67838] Updated weights for policy 0, policy_version 83242 (0.0008) [2023-10-07 23:12:03,707][67838] Updated weights for policy 0, policy_version 83252 (0.0007) [2023-10-07 23:12:04,084][67838] Updated weights for policy 0, policy_version 83262 (0.0009) [2023-10-07 23:12:04,748][67871] Updated weights for policy 1, policy_version 83370 (0.0008) [2023-10-07 23:12:05,111][67871] Updated weights for policy 1, policy_version 83380 (0.0010) [2023-10-07 23:12:05,477][67871] Updated weights for policy 1, policy_version 83390 (0.0010) [2023-10-07 23:12:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 170655744. Throughput: 0: 1663.1, 1: 1672.9. Samples: 42676736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:12:07,478][66916] Avg episode reward: [(0, '49.880'), (1, '61.110')] [2023-10-07 23:12:08,191][67838] Updated weights for policy 0, policy_version 83272 (0.0009) [2023-10-07 23:12:08,552][67838] Updated weights for policy 0, policy_version 83282 (0.0009) [2023-10-07 23:12:08,932][67838] Updated weights for policy 0, policy_version 83292 (0.0008) [2023-10-07 23:12:09,488][67871] Updated weights for policy 1, policy_version 83400 (0.0009) [2023-10-07 23:12:09,856][67871] Updated weights for policy 1, policy_version 83410 (0.0011) [2023-10-07 23:12:10,212][67871] Updated weights for policy 1, policy_version 83420 (0.0009) [2023-10-07 23:12:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 170721280. Throughput: 0: 1664.3, 1: 1652.4. Samples: 42686256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:12:12,477][66916] Avg episode reward: [(0, '50.750'), (1, '57.900')] [2023-10-07 23:12:12,787][67838] Updated weights for policy 0, policy_version 83302 (0.0008) [2023-10-07 23:12:13,154][67838] Updated weights for policy 0, policy_version 83312 (0.0007) [2023-10-07 23:12:13,527][67838] Updated weights for policy 0, policy_version 83322 (0.0007) [2023-10-07 23:12:14,539][67871] Updated weights for policy 1, policy_version 83430 (0.0007) [2023-10-07 23:12:14,901][67871] Updated weights for policy 1, policy_version 83440 (0.0010) [2023-10-07 23:12:15,264][67871] Updated weights for policy 1, policy_version 83450 (0.0008) [2023-10-07 23:12:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 170786816. Throughput: 0: 1666.9, 1: 1661.8. Samples: 42706374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:12:17,478][66916] Avg episode reward: [(0, '50.520'), (1, '57.130')] [2023-10-07 23:12:17,576][67838] Updated weights for policy 0, policy_version 83332 (0.0008) [2023-10-07 23:12:17,944][67838] Updated weights for policy 0, policy_version 83342 (0.0007) [2023-10-07 23:12:18,313][67838] Updated weights for policy 0, policy_version 83352 (0.0009) [2023-10-07 23:12:19,310][67871] Updated weights for policy 1, policy_version 83460 (0.0009) [2023-10-07 23:12:19,677][67871] Updated weights for policy 1, policy_version 83470 (0.0007) [2023-10-07 23:12:20,041][67871] Updated weights for policy 1, policy_version 83480 (0.0008) [2023-10-07 23:12:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 170852352. Throughput: 0: 1662.9, 1: 1669.8. Samples: 42726854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:12:22,478][66916] Avg episode reward: [(0, '50.280'), (1, '59.570')] [2023-10-07 23:12:22,507][67838] Updated weights for policy 0, policy_version 83362 (0.0008) [2023-10-07 23:12:22,870][67838] Updated weights for policy 0, policy_version 83372 (0.0007) [2023-10-07 23:12:23,240][67838] Updated weights for policy 0, policy_version 83382 (0.0007) [2023-10-07 23:12:23,608][67838] Updated weights for policy 0, policy_version 83392 (0.0007) [2023-10-07 23:12:24,072][67871] Updated weights for policy 1, policy_version 83490 (0.0009) [2023-10-07 23:12:24,444][67871] Updated weights for policy 1, policy_version 83500 (0.0009) [2023-10-07 23:12:24,810][67871] Updated weights for policy 1, policy_version 83510 (0.0008) [2023-10-07 23:12:25,176][67871] Updated weights for policy 1, policy_version 83520 (0.0008) [2023-10-07 23:12:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 170917888. Throughput: 0: 1665.8, 1: 1660.1. Samples: 42736410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:12:27,477][66916] Avg episode reward: [(0, '45.950'), (1, '55.970')] [2023-10-07 23:12:27,813][67838] Updated weights for policy 0, policy_version 83402 (0.0008) [2023-10-07 23:12:28,198][67838] Updated weights for policy 0, policy_version 83412 (0.0010) [2023-10-07 23:12:28,575][67838] Updated weights for policy 0, policy_version 83422 (0.0009) [2023-10-07 23:12:29,227][67871] Updated weights for policy 1, policy_version 83530 (0.0011) [2023-10-07 23:12:29,594][67871] Updated weights for policy 1, policy_version 83540 (0.0011) [2023-10-07 23:12:29,953][67871] Updated weights for policy 1, policy_version 83550 (0.0010) [2023-10-07 23:12:32,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 170983424. Throughput: 0: 1659.2, 1: 1672.1. Samples: 42756288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:12:32,477][66916] Avg episode reward: [(0, '45.200'), (1, '53.920')] [2023-10-07 23:12:32,778][67838] Updated weights for policy 0, policy_version 83432 (0.0007) [2023-10-07 23:12:33,150][67838] Updated weights for policy 0, policy_version 83442 (0.0010) [2023-10-07 23:12:33,532][67838] Updated weights for policy 0, policy_version 83452 (0.0008) [2023-10-07 23:12:34,229][67871] Updated weights for policy 1, policy_version 83560 (0.0009) [2023-10-07 23:12:34,599][67871] Updated weights for policy 1, policy_version 83570 (0.0007) [2023-10-07 23:12:34,974][67871] Updated weights for policy 1, policy_version 83580 (0.0008) [2023-10-07 23:12:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 171048960. Throughput: 0: 1655.4, 1: 1669.5. Samples: 42776600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:12:37,477][66916] Avg episode reward: [(0, '42.540'), (1, '54.040')] [2023-10-07 23:12:37,582][67838] Updated weights for policy 0, policy_version 83462 (0.0007) [2023-10-07 23:12:37,959][67838] Updated weights for policy 0, policy_version 83472 (0.0007) [2023-10-07 23:12:38,328][67838] Updated weights for policy 0, policy_version 83482 (0.0007) [2023-10-07 23:12:38,908][67871] Updated weights for policy 1, policy_version 83590 (0.0009) [2023-10-07 23:12:39,282][67871] Updated weights for policy 1, policy_version 83600 (0.0010) [2023-10-07 23:12:39,654][67871] Updated weights for policy 1, policy_version 83610 (0.0010) [2023-10-07 23:12:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 171114496. Throughput: 0: 1657.9, 1: 1655.7. Samples: 42785932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:12:42,477][66916] Avg episode reward: [(0, '41.530'), (1, '54.460')] [2023-10-07 23:12:42,482][67838] Updated weights for policy 0, policy_version 83492 (0.0007) [2023-10-07 23:12:42,863][67838] Updated weights for policy 0, policy_version 83502 (0.0009) [2023-10-07 23:12:43,239][67838] Updated weights for policy 0, policy_version 83512 (0.0009) [2023-10-07 23:12:43,827][67871] Updated weights for policy 1, policy_version 83620 (0.0009) [2023-10-07 23:12:44,196][67871] Updated weights for policy 1, policy_version 83630 (0.0008) [2023-10-07 23:12:44,559][67871] Updated weights for policy 1, policy_version 83640 (0.0009) [2023-10-07 23:12:47,267][67838] Updated weights for policy 0, policy_version 83522 (0.0007) [2023-10-07 23:12:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 171180032. Throughput: 0: 1654.8, 1: 1672.7. Samples: 42806006. Policy #0 lag: (min: 20.0, avg: 23.5, max: 52.0) [2023-10-07 23:12:47,477][66916] Avg episode reward: [(0, '43.520'), (1, '54.000')] [2023-10-07 23:12:47,633][67838] Updated weights for policy 0, policy_version 83532 (0.0010) [2023-10-07 23:12:48,001][67838] Updated weights for policy 0, policy_version 83542 (0.0008) [2023-10-07 23:12:48,374][67838] Updated weights for policy 0, policy_version 83552 (0.0008) [2023-10-07 23:12:48,703][67871] Updated weights for policy 1, policy_version 83650 (0.0010) [2023-10-07 23:12:49,082][67871] Updated weights for policy 1, policy_version 83660 (0.0008) [2023-10-07 23:12:49,442][67871] Updated weights for policy 1, policy_version 83670 (0.0007) [2023-10-07 23:12:49,807][67871] Updated weights for policy 1, policy_version 83680 (0.0010) [2023-10-07 23:12:52,472][67838] Updated weights for policy 0, policy_version 83562 (0.0007) [2023-10-07 23:12:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 171245568. Throughput: 0: 1658.2, 1: 1671.2. Samples: 42826560. Policy #0 lag: (min: 20.0, avg: 23.5, max: 52.0) [2023-10-07 23:12:52,477][66916] Avg episode reward: [(0, '43.090'), (1, '56.510')] [2023-10-07 23:12:52,847][67838] Updated weights for policy 0, policy_version 83572 (0.0007) [2023-10-07 23:12:53,215][67838] Updated weights for policy 0, policy_version 83582 (0.0007) [2023-10-07 23:12:53,911][67871] Updated weights for policy 1, policy_version 83690 (0.0008) [2023-10-07 23:12:54,278][67871] Updated weights for policy 1, policy_version 83700 (0.0009) [2023-10-07 23:12:54,642][67871] Updated weights for policy 1, policy_version 83710 (0.0008) [2023-10-07 23:12:57,301][67838] Updated weights for policy 0, policy_version 83592 (0.0008) [2023-10-07 23:12:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 171311104. Throughput: 0: 1662.5, 1: 1660.6. Samples: 42835792. Policy #0 lag: (min: 20.0, avg: 23.5, max: 52.0) [2023-10-07 23:12:57,477][66916] Avg episode reward: [(0, '43.090'), (1, '55.760')] [2023-10-07 23:12:57,678][67838] Updated weights for policy 0, policy_version 83602 (0.0008) [2023-10-07 23:12:58,054][67838] Updated weights for policy 0, policy_version 83612 (0.0009) [2023-10-07 23:12:58,741][67871] Updated weights for policy 1, policy_version 83720 (0.0008) [2023-10-07 23:12:59,113][67871] Updated weights for policy 1, policy_version 83730 (0.0009) [2023-10-07 23:12:59,480][67871] Updated weights for policy 1, policy_version 83740 (0.0008) [2023-10-07 23:13:02,127][67838] Updated weights for policy 0, policy_version 83622 (0.0010) [2023-10-07 23:13:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 171376640. Throughput: 0: 1656.5, 1: 1677.2. Samples: 42856388. Policy #0 lag: (min: 20.0, avg: 23.5, max: 52.0) [2023-10-07 23:13:02,478][66916] Avg episode reward: [(0, '45.820'), (1, '56.550')] [2023-10-07 23:13:02,493][67838] Updated weights for policy 0, policy_version 83632 (0.0009) [2023-10-07 23:13:02,870][67838] Updated weights for policy 0, policy_version 83642 (0.0008) [2023-10-07 23:13:03,439][67871] Updated weights for policy 1, policy_version 83750 (0.0011) [2023-10-07 23:13:03,808][67871] Updated weights for policy 1, policy_version 83760 (0.0011) [2023-10-07 23:13:04,177][67871] Updated weights for policy 1, policy_version 83770 (0.0010) [2023-10-07 23:13:06,977][67838] Updated weights for policy 0, policy_version 83652 (0.0008) [2023-10-07 23:13:07,349][67838] Updated weights for policy 0, policy_version 83662 (0.0007) [2023-10-07 23:13:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 171442176. Throughput: 0: 1650.8, 1: 1674.3. Samples: 42876480. Policy #0 lag: (min: 20.0, avg: 23.5, max: 52.0) [2023-10-07 23:13:07,478][66916] Avg episode reward: [(0, '49.530'), (1, '59.660')] [2023-10-07 23:13:07,487][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000083776_85786624.pth... [2023-10-07 23:13:07,519][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000082240_84213760.pth [2023-10-07 23:13:07,720][67838] Updated weights for policy 0, policy_version 83672 (0.0008) [2023-10-07 23:13:08,006][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000083680_85688320.pth... [2023-10-07 23:13:08,036][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000082112_84082688.pth [2023-10-07 23:13:08,387][67871] Updated weights for policy 1, policy_version 83780 (0.0009) [2023-10-07 23:13:08,748][67871] Updated weights for policy 1, policy_version 83790 (0.0009) [2023-10-07 23:13:09,120][67871] Updated weights for policy 1, policy_version 83800 (0.0009) [2023-10-07 23:13:11,833][67838] Updated weights for policy 0, policy_version 83682 (0.0009) [2023-10-07 23:13:12,212][67838] Updated weights for policy 0, policy_version 83692 (0.0011) [2023-10-07 23:13:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 171507712. Throughput: 0: 1656.3, 1: 1662.4. Samples: 42885752. Policy #0 lag: (min: 20.0, avg: 23.5, max: 52.0) [2023-10-07 23:13:12,478][66916] Avg episode reward: [(0, '50.570'), (1, '59.060')] [2023-10-07 23:13:12,582][67838] Updated weights for policy 0, policy_version 83702 (0.0008) [2023-10-07 23:13:12,954][67838] Updated weights for policy 0, policy_version 83712 (0.0007) [2023-10-07 23:13:13,207][67871] Updated weights for policy 1, policy_version 83810 (0.0010) [2023-10-07 23:13:13,562][67871] Updated weights for policy 1, policy_version 83820 (0.0009) [2023-10-07 23:13:13,936][67871] Updated weights for policy 1, policy_version 83830 (0.0008) [2023-10-07 23:13:14,302][67871] Updated weights for policy 1, policy_version 83840 (0.0008) [2023-10-07 23:13:17,211][67838] Updated weights for policy 0, policy_version 83722 (0.0010) [2023-10-07 23:13:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 171573248. Throughput: 0: 1661.9, 1: 1667.4. Samples: 42906106. Policy #0 lag: (min: 20.0, avg: 23.5, max: 52.0) [2023-10-07 23:13:17,477][66916] Avg episode reward: [(0, '53.670'), (1, '58.520')] [2023-10-07 23:13:17,584][67838] Updated weights for policy 0, policy_version 83732 (0.0008) [2023-10-07 23:13:17,955][67838] Updated weights for policy 0, policy_version 83742 (0.0010) [2023-10-07 23:13:18,409][67871] Updated weights for policy 1, policy_version 83850 (0.0008) [2023-10-07 23:13:18,773][67871] Updated weights for policy 1, policy_version 83860 (0.0007) [2023-10-07 23:13:19,143][67871] Updated weights for policy 1, policy_version 83870 (0.0007) [2023-10-07 23:13:22,097][67838] Updated weights for policy 0, policy_version 83752 (0.0008) [2023-10-07 23:13:22,477][67838] Updated weights for policy 0, policy_version 83762 (0.0009) [2023-10-07 23:13:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 171638784. Throughput: 0: 1655.9, 1: 1668.3. Samples: 42926190. Policy #0 lag: (min: 20.0, avg: 23.5, max: 52.0) [2023-10-07 23:13:22,478][66916] Avg episode reward: [(0, '51.930'), (1, '61.110')] [2023-10-07 23:13:22,848][67838] Updated weights for policy 0, policy_version 83772 (0.0010) [2023-10-07 23:13:23,321][67871] Updated weights for policy 1, policy_version 83880 (0.0007) [2023-10-07 23:13:23,684][67871] Updated weights for policy 1, policy_version 83890 (0.0007) [2023-10-07 23:13:24,043][67871] Updated weights for policy 1, policy_version 83900 (0.0007) [2023-10-07 23:13:26,800][67838] Updated weights for policy 0, policy_version 83782 (0.0008) [2023-10-07 23:13:27,174][67838] Updated weights for policy 0, policy_version 83792 (0.0010) [2023-10-07 23:13:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 171704320. Throughput: 0: 1661.4, 1: 1664.7. Samples: 42935606. Policy #0 lag: (min: 20.0, avg: 23.5, max: 52.0) [2023-10-07 23:13:27,478][66916] Avg episode reward: [(0, '53.490'), (1, '58.340')] [2023-10-07 23:13:27,547][67838] Updated weights for policy 0, policy_version 83802 (0.0008) [2023-10-07 23:13:27,883][67871] Updated weights for policy 1, policy_version 83910 (0.0009) [2023-10-07 23:13:28,239][67871] Updated weights for policy 1, policy_version 83920 (0.0009) [2023-10-07 23:13:28,620][67871] Updated weights for policy 1, policy_version 83930 (0.0009) [2023-10-07 23:13:31,567][67838] Updated weights for policy 0, policy_version 83812 (0.0008) [2023-10-07 23:13:31,944][67838] Updated weights for policy 0, policy_version 83822 (0.0009) [2023-10-07 23:13:32,314][67838] Updated weights for policy 0, policy_version 83832 (0.0009) [2023-10-07 23:13:32,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 171769856. Throughput: 0: 1670.6, 1: 1670.8. Samples: 42956370. Policy #0 lag: (min: 20.0, avg: 23.5, max: 52.0) [2023-10-07 23:13:32,477][66916] Avg episode reward: [(0, '50.730'), (1, '57.210')] [2023-10-07 23:13:32,800][67871] Updated weights for policy 1, policy_version 83940 (0.0007) [2023-10-07 23:13:33,170][67871] Updated weights for policy 1, policy_version 83950 (0.0008) [2023-10-07 23:13:33,540][67871] Updated weights for policy 1, policy_version 83960 (0.0008) [2023-10-07 23:13:36,485][67838] Updated weights for policy 0, policy_version 83842 (0.0008) [2023-10-07 23:13:36,854][67838] Updated weights for policy 0, policy_version 83852 (0.0008) [2023-10-07 23:13:37,233][67838] Updated weights for policy 0, policy_version 83862 (0.0010) [2023-10-07 23:13:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 171835392. Throughput: 0: 1652.0, 1: 1676.8. Samples: 42976356. Policy #0 lag: (min: 20.0, avg: 23.5, max: 52.0) [2023-10-07 23:13:37,477][66916] Avg episode reward: [(0, '49.690'), (1, '58.500')] [2023-10-07 23:13:37,609][67838] Updated weights for policy 0, policy_version 83872 (0.0010) [2023-10-07 23:13:37,802][67871] Updated weights for policy 1, policy_version 83970 (0.0010) [2023-10-07 23:13:38,218][67871] Updated weights for policy 1, policy_version 83980 (0.0007) [2023-10-07 23:13:38,581][67871] Updated weights for policy 1, policy_version 83990 (0.0008) [2023-10-07 23:13:38,947][67871] Updated weights for policy 1, policy_version 84000 (0.0007) [2023-10-07 23:13:41,734][67838] Updated weights for policy 0, policy_version 83882 (0.0008) [2023-10-07 23:13:42,103][67838] Updated weights for policy 0, policy_version 83892 (0.0009) [2023-10-07 23:13:42,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 171900928. Throughput: 0: 1663.6, 1: 1674.4. Samples: 42986006. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 23:13:42,477][66916] Avg episode reward: [(0, '48.420'), (1, '60.230')] [2023-10-07 23:13:42,478][67838] Updated weights for policy 0, policy_version 83902 (0.0007) [2023-10-07 23:13:42,990][67871] Updated weights for policy 1, policy_version 84010 (0.0008) [2023-10-07 23:13:43,367][67871] Updated weights for policy 1, policy_version 84020 (0.0009) [2023-10-07 23:13:43,730][67871] Updated weights for policy 1, policy_version 84030 (0.0009) [2023-10-07 23:13:46,591][67838] Updated weights for policy 0, policy_version 83912 (0.0007) [2023-10-07 23:13:46,948][67838] Updated weights for policy 0, policy_version 83922 (0.0011) [2023-10-07 23:13:47,309][67838] Updated weights for policy 0, policy_version 83932 (0.0008) [2023-10-07 23:13:47,477][66916] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 171999232. Throughput: 0: 1665.1, 1: 1669.5. Samples: 43006444. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 23:13:47,478][66916] Avg episode reward: [(0, '49.940'), (1, '61.930')] [2023-10-07 23:13:47,753][67871] Updated weights for policy 1, policy_version 84040 (0.0010) [2023-10-07 23:13:48,126][67871] Updated weights for policy 1, policy_version 84050 (0.0009) [2023-10-07 23:13:48,483][67871] Updated weights for policy 1, policy_version 84060 (0.0010) [2023-10-07 23:13:51,433][67838] Updated weights for policy 0, policy_version 83942 (0.0007) [2023-10-07 23:13:51,800][67838] Updated weights for policy 0, policy_version 83952 (0.0007) [2023-10-07 23:13:52,167][67838] Updated weights for policy 0, policy_version 83962 (0.0007) [2023-10-07 23:13:52,476][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 172064768. Throughput: 0: 1655.6, 1: 1669.8. Samples: 43026120. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 23:13:52,477][66916] Avg episode reward: [(0, '48.110'), (1, '63.290')] [2023-10-07 23:13:52,610][67871] Updated weights for policy 1, policy_version 84070 (0.0008) [2023-10-07 23:13:52,970][67871] Updated weights for policy 1, policy_version 84080 (0.0007) [2023-10-07 23:13:53,334][67871] Updated weights for policy 1, policy_version 84090 (0.0009) [2023-10-07 23:13:56,401][67838] Updated weights for policy 0, policy_version 83972 (0.0009) [2023-10-07 23:13:56,762][67838] Updated weights for policy 0, policy_version 83982 (0.0008) [2023-10-07 23:13:57,144][67838] Updated weights for policy 0, policy_version 83992 (0.0008) [2023-10-07 23:13:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 172130304. Throughput: 0: 1665.7, 1: 1667.8. Samples: 43035760. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 23:13:57,477][66916] Avg episode reward: [(0, '48.840'), (1, '66.710')] [2023-10-07 23:13:57,495][67871] Updated weights for policy 1, policy_version 84100 (0.0008) [2023-10-07 23:13:57,866][67871] Updated weights for policy 1, policy_version 84110 (0.0009) [2023-10-07 23:13:58,235][67871] Updated weights for policy 1, policy_version 84120 (0.0008) [2023-10-07 23:14:01,245][67838] Updated weights for policy 0, policy_version 84002 (0.0010) [2023-10-07 23:14:01,620][67838] Updated weights for policy 0, policy_version 84012 (0.0007) [2023-10-07 23:14:01,998][67838] Updated weights for policy 0, policy_version 84022 (0.0009) [2023-10-07 23:14:02,295][67871] Updated weights for policy 1, policy_version 84130 (0.0009) [2023-10-07 23:14:02,369][67838] Updated weights for policy 0, policy_version 84032 (0.0007) [2023-10-07 23:14:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 172195840. Throughput: 0: 1663.5, 1: 1671.7. Samples: 43056188. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 23:14:02,477][66916] Avg episode reward: [(0, '45.070'), (1, '67.370')] [2023-10-07 23:14:02,662][67871] Updated weights for policy 1, policy_version 84140 (0.0009) [2023-10-07 23:14:03,019][67871] Updated weights for policy 1, policy_version 84150 (0.0008) [2023-10-07 23:14:03,389][67871] Updated weights for policy 1, policy_version 84160 (0.0009) [2023-10-07 23:14:06,583][67838] Updated weights for policy 0, policy_version 84042 (0.0010) [2023-10-07 23:14:06,951][67838] Updated weights for policy 0, policy_version 84052 (0.0007) [2023-10-07 23:14:07,314][67838] Updated weights for policy 0, policy_version 84062 (0.0007) [2023-10-07 23:14:07,409][67871] Updated weights for policy 1, policy_version 84170 (0.0008) [2023-10-07 23:14:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 172261376. Throughput: 0: 1649.6, 1: 1675.1. Samples: 43075798. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 23:14:07,478][66916] Avg episode reward: [(0, '44.700'), (1, '65.840')] [2023-10-07 23:14:07,773][67871] Updated weights for policy 1, policy_version 84180 (0.0008) [2023-10-07 23:14:08,139][67871] Updated weights for policy 1, policy_version 84190 (0.0010) [2023-10-07 23:14:11,425][67838] Updated weights for policy 0, policy_version 84072 (0.0008) [2023-10-07 23:14:11,793][67838] Updated weights for policy 0, policy_version 84082 (0.0009) [2023-10-07 23:14:12,164][67838] Updated weights for policy 0, policy_version 84092 (0.0008) [2023-10-07 23:14:12,204][67871] Updated weights for policy 1, policy_version 84200 (0.0010) [2023-10-07 23:14:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 172326912. Throughput: 0: 1661.0, 1: 1675.5. Samples: 43085748. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 23:14:12,477][66916] Avg episode reward: [(0, '42.220'), (1, '64.470')] [2023-10-07 23:14:12,569][67871] Updated weights for policy 1, policy_version 84210 (0.0010) [2023-10-07 23:14:12,938][67871] Updated weights for policy 1, policy_version 84220 (0.0008) [2023-10-07 23:14:16,283][67838] Updated weights for policy 0, policy_version 84102 (0.0008) [2023-10-07 23:14:16,655][67838] Updated weights for policy 0, policy_version 84112 (0.0008) [2023-10-07 23:14:17,022][67838] Updated weights for policy 0, policy_version 84122 (0.0007) [2023-10-07 23:14:17,104][67871] Updated weights for policy 1, policy_version 84230 (0.0007) [2023-10-07 23:14:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 172392448. Throughput: 0: 1659.3, 1: 1673.9. Samples: 43106364. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 23:14:17,477][66916] Avg episode reward: [(0, '41.660'), (1, '62.490')] [2023-10-07 23:14:17,479][67871] Updated weights for policy 1, policy_version 84240 (0.0008) [2023-10-07 23:14:17,838][67871] Updated weights for policy 1, policy_version 84250 (0.0010) [2023-10-07 23:14:21,136][67838] Updated weights for policy 0, policy_version 84132 (0.0007) [2023-10-07 23:14:21,508][67838] Updated weights for policy 0, policy_version 84142 (0.0007) [2023-10-07 23:14:21,876][67838] Updated weights for policy 0, policy_version 84152 (0.0011) [2023-10-07 23:14:21,964][67871] Updated weights for policy 1, policy_version 84260 (0.0009) [2023-10-07 23:14:22,342][67871] Updated weights for policy 1, policy_version 84270 (0.0009) [2023-10-07 23:14:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 172457984. Throughput: 0: 1653.3, 1: 1670.5. Samples: 43125926. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 23:14:22,478][66916] Avg episode reward: [(0, '43.060'), (1, '61.390')] [2023-10-07 23:14:22,706][67871] Updated weights for policy 1, policy_version 84280 (0.0010) [2023-10-07 23:14:25,801][67838] Updated weights for policy 0, policy_version 84162 (0.0007) [2023-10-07 23:14:26,174][67838] Updated weights for policy 0, policy_version 84172 (0.0007) [2023-10-07 23:14:26,555][67838] Updated weights for policy 0, policy_version 84182 (0.0009) [2023-10-07 23:14:26,901][67871] Updated weights for policy 1, policy_version 84290 (0.0008) [2023-10-07 23:14:26,917][67838] Updated weights for policy 0, policy_version 84192 (0.0009) [2023-10-07 23:14:27,319][67871] Updated weights for policy 1, policy_version 84300 (0.0007) [2023-10-07 23:14:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 172523520. Throughput: 0: 1662.6, 1: 1674.1. Samples: 43136158. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-07 23:14:27,477][66916] Avg episode reward: [(0, '41.730'), (1, '61.370')] [2023-10-07 23:14:27,689][67871] Updated weights for policy 1, policy_version 84310 (0.0008) [2023-10-07 23:14:28,055][67871] Updated weights for policy 1, policy_version 84320 (0.0008) [2023-10-07 23:14:31,128][67838] Updated weights for policy 0, policy_version 84202 (0.0008) [2023-10-07 23:14:31,500][67838] Updated weights for policy 0, policy_version 84212 (0.0008) [2023-10-07 23:14:31,865][67838] Updated weights for policy 0, policy_version 84222 (0.0010) [2023-10-07 23:14:32,043][67871] Updated weights for policy 1, policy_version 84330 (0.0008) [2023-10-07 23:14:32,406][67871] Updated weights for policy 1, policy_version 84340 (0.0007) [2023-10-07 23:14:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 172589056. Throughput: 0: 1654.0, 1: 1672.1. Samples: 43156116. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:14:32,477][66916] Avg episode reward: [(0, '38.640'), (1, '62.000')] [2023-10-07 23:14:32,779][67871] Updated weights for policy 1, policy_version 84350 (0.0007) [2023-10-07 23:14:36,096][67838] Updated weights for policy 0, policy_version 84232 (0.0008) [2023-10-07 23:14:36,463][67838] Updated weights for policy 0, policy_version 84242 (0.0008) [2023-10-07 23:14:36,824][67838] Updated weights for policy 0, policy_version 84252 (0.0008) [2023-10-07 23:14:36,896][67871] Updated weights for policy 1, policy_version 84360 (0.0008) [2023-10-07 23:14:37,260][67871] Updated weights for policy 1, policy_version 84370 (0.0009) [2023-10-07 23:14:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 172654592. Throughput: 0: 1650.9, 1: 1669.8. Samples: 43175552. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:14:37,478][66916] Avg episode reward: [(0, '42.670'), (1, '62.190')] [2023-10-07 23:14:37,630][67871] Updated weights for policy 1, policy_version 84380 (0.0008) [2023-10-07 23:14:40,837][67838] Updated weights for policy 0, policy_version 84262 (0.0007) [2023-10-07 23:14:41,200][67838] Updated weights for policy 0, policy_version 84272 (0.0009) [2023-10-07 23:14:41,580][67838] Updated weights for policy 0, policy_version 84282 (0.0009) [2023-10-07 23:14:41,813][67871] Updated weights for policy 1, policy_version 84390 (0.0009) [2023-10-07 23:14:42,177][67871] Updated weights for policy 1, policy_version 84400 (0.0009) [2023-10-07 23:14:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 172720128. Throughput: 0: 1666.7, 1: 1677.5. Samples: 43186250. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:14:42,477][66916] Avg episode reward: [(0, '39.770'), (1, '61.420')] [2023-10-07 23:14:42,538][67871] Updated weights for policy 1, policy_version 84410 (0.0009) [2023-10-07 23:14:45,875][67838] Updated weights for policy 0, policy_version 84292 (0.0009) [2023-10-07 23:14:46,235][67838] Updated weights for policy 0, policy_version 84302 (0.0009) [2023-10-07 23:14:46,591][67871] Updated weights for policy 1, policy_version 84420 (0.0008) [2023-10-07 23:14:46,619][67838] Updated weights for policy 0, policy_version 84312 (0.0009) [2023-10-07 23:14:46,953][67871] Updated weights for policy 1, policy_version 84430 (0.0008) [2023-10-07 23:14:47,321][67871] Updated weights for policy 1, policy_version 84440 (0.0008) [2023-10-07 23:14:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 172785664. Throughput: 0: 1656.6, 1: 1673.2. Samples: 43206032. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:14:47,477][66916] Avg episode reward: [(0, '41.300'), (1, '64.130')] [2023-10-07 23:14:50,807][67838] Updated weights for policy 0, policy_version 84322 (0.0009) [2023-10-07 23:14:51,203][67838] Updated weights for policy 0, policy_version 84332 (0.0007) [2023-10-07 23:14:51,481][67871] Updated weights for policy 1, policy_version 84450 (0.0008) [2023-10-07 23:14:51,576][67838] Updated weights for policy 0, policy_version 84342 (0.0007) [2023-10-07 23:14:51,852][67871] Updated weights for policy 1, policy_version 84460 (0.0010) [2023-10-07 23:14:51,946][67838] Updated weights for policy 0, policy_version 84352 (0.0007) [2023-10-07 23:14:52,215][67871] Updated weights for policy 1, policy_version 84470 (0.0007) [2023-10-07 23:14:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 172851200. Throughput: 0: 1653.8, 1: 1663.4. Samples: 43225072. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:14:52,477][66916] Avg episode reward: [(0, '42.300'), (1, '67.030')] [2023-10-07 23:14:52,580][67871] Updated weights for policy 1, policy_version 84480 (0.0007) [2023-10-07 23:14:56,051][67838] Updated weights for policy 0, policy_version 84362 (0.0009) [2023-10-07 23:14:56,426][67838] Updated weights for policy 0, policy_version 84372 (0.0007) [2023-10-07 23:14:56,664][67871] Updated weights for policy 1, policy_version 84490 (0.0009) [2023-10-07 23:14:56,805][67838] Updated weights for policy 0, policy_version 84382 (0.0008) [2023-10-07 23:14:57,043][67871] Updated weights for policy 1, policy_version 84500 (0.0008) [2023-10-07 23:14:57,407][67871] Updated weights for policy 1, policy_version 84510 (0.0008) [2023-10-07 23:14:57,476][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 172949504. Throughput: 0: 1664.0, 1: 1672.5. Samples: 43235892. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:14:57,477][66916] Avg episode reward: [(0, '43.570'), (1, '66.380')] [2023-10-07 23:15:00,842][67838] Updated weights for policy 0, policy_version 84392 (0.0009) [2023-10-07 23:15:01,208][67838] Updated weights for policy 0, policy_version 84402 (0.0008) [2023-10-07 23:15:01,498][67871] Updated weights for policy 1, policy_version 84520 (0.0007) [2023-10-07 23:15:01,575][67838] Updated weights for policy 0, policy_version 84412 (0.0008) [2023-10-07 23:15:01,861][67871] Updated weights for policy 1, policy_version 84530 (0.0009) [2023-10-07 23:15:02,235][67871] Updated weights for policy 1, policy_version 84540 (0.0009) [2023-10-07 23:15:02,476][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 173015040. Throughput: 0: 1651.2, 1: 1670.7. Samples: 43255852. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:15:02,477][66916] Avg episode reward: [(0, '43.970'), (1, '65.820')] [2023-10-07 23:15:05,507][67838] Updated weights for policy 0, policy_version 84422 (0.0010) [2023-10-07 23:15:05,882][67838] Updated weights for policy 0, policy_version 84432 (0.0008) [2023-10-07 23:15:06,249][67838] Updated weights for policy 0, policy_version 84442 (0.0009) [2023-10-07 23:15:06,463][67871] Updated weights for policy 1, policy_version 84550 (0.0008) [2023-10-07 23:15:06,820][67871] Updated weights for policy 1, policy_version 84560 (0.0007) [2023-10-07 23:15:07,189][67871] Updated weights for policy 1, policy_version 84570 (0.0009) [2023-10-07 23:15:07,477][66916] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 173080576. Throughput: 0: 1654.0, 1: 1655.1. Samples: 43274838. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:15:07,478][66916] Avg episode reward: [(0, '47.710'), (1, '67.810')] [2023-10-07 23:15:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000084576_86605824.pth... [2023-10-07 23:15:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000084448_86474752.pth... [2023-10-07 23:15:07,527][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000082880_84869120.pth [2023-10-07 23:15:07,528][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000083008_85000192.pth [2023-10-07 23:15:10,493][67838] Updated weights for policy 0, policy_version 84452 (0.0008) [2023-10-07 23:15:10,863][67838] Updated weights for policy 0, policy_version 84462 (0.0007) [2023-10-07 23:15:11,160][67871] Updated weights for policy 1, policy_version 84580 (0.0009) [2023-10-07 23:15:11,226][67838] Updated weights for policy 0, policy_version 84472 (0.0007) [2023-10-07 23:15:11,520][67871] Updated weights for policy 1, policy_version 84590 (0.0009) [2023-10-07 23:15:11,883][67871] Updated weights for policy 1, policy_version 84600 (0.0010) [2023-10-07 23:15:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 173146112. Throughput: 0: 1658.0, 1: 1663.2. Samples: 43285610. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:15:12,477][66916] Avg episode reward: [(0, '49.380'), (1, '69.480')] [2023-10-07 23:15:12,477][67676] Saving new best policy, reward=69.480! [2023-10-07 23:15:15,502][67838] Updated weights for policy 0, policy_version 84482 (0.0008) [2023-10-07 23:15:15,879][67838] Updated weights for policy 0, policy_version 84492 (0.0008) [2023-10-07 23:15:16,011][67871] Updated weights for policy 1, policy_version 84610 (0.0009) [2023-10-07 23:15:16,259][67838] Updated weights for policy 0, policy_version 84502 (0.0009) [2023-10-07 23:15:16,433][67871] Updated weights for policy 1, policy_version 84620 (0.0008) [2023-10-07 23:15:16,620][67838] Updated weights for policy 0, policy_version 84512 (0.0007) [2023-10-07 23:15:16,806][67871] Updated weights for policy 1, policy_version 84630 (0.0007) [2023-10-07 23:15:17,164][67871] Updated weights for policy 1, policy_version 84640 (0.0008) [2023-10-07 23:15:17,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 173211648. Throughput: 0: 1650.8, 1: 1669.9. Samples: 43305548. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:15:17,477][66916] Avg episode reward: [(0, '49.820'), (1, '67.760')] [2023-10-07 23:15:20,733][67838] Updated weights for policy 0, policy_version 84522 (0.0010) [2023-10-07 23:15:21,101][67838] Updated weights for policy 0, policy_version 84532 (0.0010) [2023-10-07 23:15:21,214][67871] Updated weights for policy 1, policy_version 84650 (0.0008) [2023-10-07 23:15:21,468][67838] Updated weights for policy 0, policy_version 84542 (0.0009) [2023-10-07 23:15:21,567][67871] Updated weights for policy 1, policy_version 84660 (0.0009) [2023-10-07 23:15:21,928][67871] Updated weights for policy 1, policy_version 84670 (0.0009) [2023-10-07 23:15:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 173277184. Throughput: 0: 1658.6, 1: 1645.4. Samples: 43324232. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:15:22,478][66916] Avg episode reward: [(0, '49.210'), (1, '66.910')] [2023-10-07 23:15:25,590][67838] Updated weights for policy 0, policy_version 84552 (0.0009) [2023-10-07 23:15:25,966][67838] Updated weights for policy 0, policy_version 84562 (0.0008) [2023-10-07 23:15:26,119][67871] Updated weights for policy 1, policy_version 84680 (0.0010) [2023-10-07 23:15:26,330][67838] Updated weights for policy 0, policy_version 84572 (0.0008) [2023-10-07 23:15:26,481][67871] Updated weights for policy 1, policy_version 84690 (0.0008) [2023-10-07 23:15:26,852][67871] Updated weights for policy 1, policy_version 84700 (0.0010) [2023-10-07 23:15:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 173342720. Throughput: 0: 1655.4, 1: 1662.6. Samples: 43335558. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 23:15:27,477][66916] Avg episode reward: [(0, '48.330'), (1, '66.010')] [2023-10-07 23:15:30,350][67838] Updated weights for policy 0, policy_version 84582 (0.0008) [2023-10-07 23:15:30,728][67838] Updated weights for policy 0, policy_version 84592 (0.0007) [2023-10-07 23:15:30,896][67871] Updated weights for policy 1, policy_version 84710 (0.0007) [2023-10-07 23:15:31,101][67838] Updated weights for policy 0, policy_version 84602 (0.0008) [2023-10-07 23:15:31,264][67871] Updated weights for policy 1, policy_version 84720 (0.0008) [2023-10-07 23:15:31,631][67871] Updated weights for policy 1, policy_version 84730 (0.0008) [2023-10-07 23:15:32,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 173408256. Throughput: 0: 1649.3, 1: 1659.1. Samples: 43354914. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 23:15:32,478][66916] Avg episode reward: [(0, '45.610'), (1, '65.460')] [2023-10-07 23:15:35,299][67838] Updated weights for policy 0, policy_version 84612 (0.0008) [2023-10-07 23:15:35,670][67838] Updated weights for policy 0, policy_version 84622 (0.0009) [2023-10-07 23:15:35,853][67871] Updated weights for policy 1, policy_version 84740 (0.0009) [2023-10-07 23:15:36,040][67838] Updated weights for policy 0, policy_version 84632 (0.0007) [2023-10-07 23:15:36,222][67871] Updated weights for policy 1, policy_version 84750 (0.0010) [2023-10-07 23:15:36,593][67871] Updated weights for policy 1, policy_version 84760 (0.0007) [2023-10-07 23:15:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 173473792. Throughput: 0: 1667.4, 1: 1641.2. Samples: 43373962. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 23:15:37,478][66916] Avg episode reward: [(0, '47.990'), (1, '63.740')] [2023-10-07 23:15:39,909][67838] Updated weights for policy 0, policy_version 84642 (0.0009) [2023-10-07 23:15:40,297][67838] Updated weights for policy 0, policy_version 84652 (0.0009) [2023-10-07 23:15:40,666][67838] Updated weights for policy 0, policy_version 84662 (0.0008) [2023-10-07 23:15:40,792][67871] Updated weights for policy 1, policy_version 84770 (0.0008) [2023-10-07 23:15:41,032][67838] Updated weights for policy 0, policy_version 84672 (0.0009) [2023-10-07 23:15:41,160][67871] Updated weights for policy 1, policy_version 84780 (0.0007) [2023-10-07 23:15:41,527][67871] Updated weights for policy 1, policy_version 84790 (0.0008) [2023-10-07 23:15:41,885][67871] Updated weights for policy 1, policy_version 84800 (0.0008) [2023-10-07 23:15:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 173539328. Throughput: 0: 1663.0, 1: 1653.3. Samples: 43385128. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 23:15:42,477][66916] Avg episode reward: [(0, '45.250'), (1, '62.230')] [2023-10-07 23:15:45,020][67838] Updated weights for policy 0, policy_version 84682 (0.0011) [2023-10-07 23:15:45,396][67838] Updated weights for policy 0, policy_version 84692 (0.0009) [2023-10-07 23:15:45,770][67838] Updated weights for policy 0, policy_version 84702 (0.0011) [2023-10-07 23:15:46,076][67871] Updated weights for policy 1, policy_version 84810 (0.0009) [2023-10-07 23:15:46,453][67871] Updated weights for policy 1, policy_version 84820 (0.0009) [2023-10-07 23:15:46,815][67871] Updated weights for policy 1, policy_version 84830 (0.0008) [2023-10-07 23:15:47,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 173604864. Throughput: 0: 1651.6, 1: 1649.0. Samples: 43404380. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 23:15:47,477][66916] Avg episode reward: [(0, '43.430'), (1, '61.760')] [2023-10-07 23:15:49,794][67838] Updated weights for policy 0, policy_version 84712 (0.0010) [2023-10-07 23:15:50,163][67838] Updated weights for policy 0, policy_version 84722 (0.0010) [2023-10-07 23:15:50,523][67838] Updated weights for policy 0, policy_version 84732 (0.0010) [2023-10-07 23:15:51,022][67871] Updated weights for policy 1, policy_version 84840 (0.0007) [2023-10-07 23:15:51,385][67871] Updated weights for policy 1, policy_version 84850 (0.0009) [2023-10-07 23:15:51,748][67871] Updated weights for policy 1, policy_version 84860 (0.0009) [2023-10-07 23:15:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 173670400. Throughput: 0: 1668.9, 1: 1638.5. Samples: 43423668. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 23:15:52,477][66916] Avg episode reward: [(0, '43.370'), (1, '63.280')] [2023-10-07 23:15:54,798][67838] Updated weights for policy 0, policy_version 84742 (0.0009) [2023-10-07 23:15:55,177][67838] Updated weights for policy 0, policy_version 84752 (0.0008) [2023-10-07 23:15:55,538][67838] Updated weights for policy 0, policy_version 84762 (0.0009) [2023-10-07 23:15:55,790][67871] Updated weights for policy 1, policy_version 84870 (0.0008) [2023-10-07 23:15:56,163][67871] Updated weights for policy 1, policy_version 84880 (0.0008) [2023-10-07 23:15:56,527][67871] Updated weights for policy 1, policy_version 84890 (0.0008) [2023-10-07 23:15:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173735936. Throughput: 0: 1655.9, 1: 1655.0. Samples: 43434600. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 23:15:57,478][66916] Avg episode reward: [(0, '43.140'), (1, '58.730')] [2023-10-07 23:15:59,678][67838] Updated weights for policy 0, policy_version 84772 (0.0007) [2023-10-07 23:16:00,051][67838] Updated weights for policy 0, policy_version 84782 (0.0007) [2023-10-07 23:16:00,430][67838] Updated weights for policy 0, policy_version 84792 (0.0008) [2023-10-07 23:16:00,736][67871] Updated weights for policy 1, policy_version 84900 (0.0009) [2023-10-07 23:16:01,123][67871] Updated weights for policy 1, policy_version 84910 (0.0009) [2023-10-07 23:16:01,489][67871] Updated weights for policy 1, policy_version 84920 (0.0007) [2023-10-07 23:16:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173801472. Throughput: 0: 1653.7, 1: 1646.3. Samples: 43454048. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 23:16:02,478][66916] Avg episode reward: [(0, '43.130'), (1, '61.840')] [2023-10-07 23:16:04,572][67838] Updated weights for policy 0, policy_version 84802 (0.0008) [2023-10-07 23:16:04,949][67838] Updated weights for policy 0, policy_version 84812 (0.0008) [2023-10-07 23:16:05,324][67838] Updated weights for policy 0, policy_version 84822 (0.0008) [2023-10-07 23:16:05,595][67871] Updated weights for policy 1, policy_version 84930 (0.0008) [2023-10-07 23:16:05,689][67838] Updated weights for policy 0, policy_version 84832 (0.0009) [2023-10-07 23:16:05,957][67871] Updated weights for policy 1, policy_version 84940 (0.0007) [2023-10-07 23:16:06,324][67871] Updated weights for policy 1, policy_version 84950 (0.0008) [2023-10-07 23:16:06,682][67871] Updated weights for policy 1, policy_version 84960 (0.0010) [2023-10-07 23:16:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 173867008. Throughput: 0: 1667.5, 1: 1650.3. Samples: 43473532. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 23:16:07,477][66916] Avg episode reward: [(0, '44.860'), (1, '60.290')] [2023-10-07 23:16:09,716][67838] Updated weights for policy 0, policy_version 84842 (0.0007) [2023-10-07 23:16:10,084][67838] Updated weights for policy 0, policy_version 84852 (0.0007) [2023-10-07 23:16:10,459][67838] Updated weights for policy 0, policy_version 84862 (0.0008) [2023-10-07 23:16:10,830][67871] Updated weights for policy 1, policy_version 84970 (0.0010) [2023-10-07 23:16:11,194][67871] Updated weights for policy 1, policy_version 84980 (0.0010) [2023-10-07 23:16:11,559][67871] Updated weights for policy 1, policy_version 84990 (0.0009) [2023-10-07 23:16:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173932544. Throughput: 0: 1650.2, 1: 1655.6. Samples: 43484320. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 23:16:12,477][66916] Avg episode reward: [(0, '42.930'), (1, '64.770')] [2023-10-07 23:16:14,578][67838] Updated weights for policy 0, policy_version 84872 (0.0010) [2023-10-07 23:16:14,958][67838] Updated weights for policy 0, policy_version 84882 (0.0008) [2023-10-07 23:16:15,323][67838] Updated weights for policy 0, policy_version 84892 (0.0007) [2023-10-07 23:16:15,758][67871] Updated weights for policy 1, policy_version 85000 (0.0009) [2023-10-07 23:16:16,128][67871] Updated weights for policy 1, policy_version 85010 (0.0009) [2023-10-07 23:16:16,494][67871] Updated weights for policy 1, policy_version 85020 (0.0010) [2023-10-07 23:16:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173998080. Throughput: 0: 1654.3, 1: 1651.0. Samples: 43503652. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-07 23:16:17,478][66916] Avg episode reward: [(0, '45.260'), (1, '64.200')] [2023-10-07 23:16:19,441][67838] Updated weights for policy 0, policy_version 84902 (0.0010) [2023-10-07 23:16:19,806][67838] Updated weights for policy 0, policy_version 84912 (0.0010) [2023-10-07 23:16:20,186][67838] Updated weights for policy 0, policy_version 84922 (0.0007) [2023-10-07 23:16:20,613][67871] Updated weights for policy 1, policy_version 85030 (0.0009) [2023-10-07 23:16:20,971][67871] Updated weights for policy 1, policy_version 85040 (0.0009) [2023-10-07 23:16:21,351][67871] Updated weights for policy 1, policy_version 85050 (0.0010) [2023-10-07 23:16:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 174063616. Throughput: 0: 1664.2, 1: 1654.6. Samples: 43523308. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-07 23:16:22,478][66916] Avg episode reward: [(0, '45.350'), (1, '62.700')] [2023-10-07 23:16:24,424][67838] Updated weights for policy 0, policy_version 84932 (0.0009) [2023-10-07 23:16:24,794][67838] Updated weights for policy 0, policy_version 84942 (0.0009) [2023-10-07 23:16:25,177][67838] Updated weights for policy 0, policy_version 84952 (0.0009) [2023-10-07 23:16:25,427][67871] Updated weights for policy 1, policy_version 85060 (0.0007) [2023-10-07 23:16:25,792][67871] Updated weights for policy 1, policy_version 85070 (0.0011) [2023-10-07 23:16:26,162][67871] Updated weights for policy 1, policy_version 85080 (0.0008) [2023-10-07 23:16:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 174129152. Throughput: 0: 1647.5, 1: 1662.9. Samples: 43534096. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-07 23:16:27,477][66916] Avg episode reward: [(0, '44.620'), (1, '65.400')] [2023-10-07 23:16:29,338][67838] Updated weights for policy 0, policy_version 84962 (0.0009) [2023-10-07 23:16:29,732][67838] Updated weights for policy 0, policy_version 84972 (0.0008) [2023-10-07 23:16:30,107][67838] Updated weights for policy 0, policy_version 84982 (0.0008) [2023-10-07 23:16:30,429][67871] Updated weights for policy 1, policy_version 85090 (0.0009) [2023-10-07 23:16:30,481][67838] Updated weights for policy 0, policy_version 84992 (0.0009) [2023-10-07 23:16:30,790][67871] Updated weights for policy 1, policy_version 85100 (0.0009) [2023-10-07 23:16:31,157][67871] Updated weights for policy 1, policy_version 85110 (0.0007) [2023-10-07 23:16:31,522][67871] Updated weights for policy 1, policy_version 85120 (0.0010) [2023-10-07 23:16:32,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 174194688. Throughput: 0: 1651.9, 1: 1654.3. Samples: 43553162. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-07 23:16:32,478][66916] Avg episode reward: [(0, '43.920'), (1, '67.820')] [2023-10-07 23:16:34,733][67838] Updated weights for policy 0, policy_version 85002 (0.0008) [2023-10-07 23:16:35,101][67838] Updated weights for policy 0, policy_version 85012 (0.0010) [2023-10-07 23:16:35,476][67838] Updated weights for policy 0, policy_version 85022 (0.0008) [2023-10-07 23:16:35,658][67871] Updated weights for policy 1, policy_version 85130 (0.0010) [2023-10-07 23:16:36,022][67871] Updated weights for policy 1, policy_version 85140 (0.0009) [2023-10-07 23:16:36,393][67871] Updated weights for policy 1, policy_version 85150 (0.0010) [2023-10-07 23:16:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 174260224. Throughput: 0: 1653.2, 1: 1663.2. Samples: 43572904. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-07 23:16:37,477][66916] Avg episode reward: [(0, '43.100'), (1, '63.240')] [2023-10-07 23:16:39,495][67838] Updated weights for policy 0, policy_version 85032 (0.0010) [2023-10-07 23:16:39,867][67838] Updated weights for policy 0, policy_version 85042 (0.0008) [2023-10-07 23:16:40,244][67838] Updated weights for policy 0, policy_version 85052 (0.0007) [2023-10-07 23:16:40,507][67871] Updated weights for policy 1, policy_version 85160 (0.0008) [2023-10-07 23:16:40,877][67871] Updated weights for policy 1, policy_version 85170 (0.0009) [2023-10-07 23:16:41,237][67871] Updated weights for policy 1, policy_version 85180 (0.0009) [2023-10-07 23:16:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 174325760. Throughput: 0: 1645.8, 1: 1662.4. Samples: 43583470. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-07 23:16:42,477][66916] Avg episode reward: [(0, '44.260'), (1, '64.400')] [2023-10-07 23:16:44,368][67838] Updated weights for policy 0, policy_version 85062 (0.0007) [2023-10-07 23:16:44,736][67838] Updated weights for policy 0, policy_version 85072 (0.0009) [2023-10-07 23:16:45,110][67838] Updated weights for policy 0, policy_version 85082 (0.0009) [2023-10-07 23:16:45,313][67871] Updated weights for policy 1, policy_version 85190 (0.0007) [2023-10-07 23:16:45,690][67871] Updated weights for policy 1, policy_version 85200 (0.0008) [2023-10-07 23:16:46,056][67871] Updated weights for policy 1, policy_version 85210 (0.0010) [2023-10-07 23:16:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 174391296. Throughput: 0: 1653.7, 1: 1649.9. Samples: 43602710. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-07 23:16:47,477][66916] Avg episode reward: [(0, '44.090'), (1, '62.910')] [2023-10-07 23:16:49,150][67838] Updated weights for policy 0, policy_version 85092 (0.0009) [2023-10-07 23:16:49,530][67838] Updated weights for policy 0, policy_version 85102 (0.0009) [2023-10-07 23:16:49,903][67838] Updated weights for policy 0, policy_version 85112 (0.0009) [2023-10-07 23:16:50,078][67871] Updated weights for policy 1, policy_version 85220 (0.0008) [2023-10-07 23:16:50,449][67871] Updated weights for policy 1, policy_version 85230 (0.0009) [2023-10-07 23:16:50,810][67871] Updated weights for policy 1, policy_version 85240 (0.0008) [2023-10-07 23:16:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 174456832. Throughput: 0: 1653.1, 1: 1661.7. Samples: 43622700. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-07 23:16:52,477][66916] Avg episode reward: [(0, '44.080'), (1, '61.150')] [2023-10-07 23:16:54,049][67838] Updated weights for policy 0, policy_version 85122 (0.0009) [2023-10-07 23:16:54,420][67838] Updated weights for policy 0, policy_version 85132 (0.0009) [2023-10-07 23:16:54,801][67838] Updated weights for policy 0, policy_version 85142 (0.0008) [2023-10-07 23:16:55,139][67871] Updated weights for policy 1, policy_version 85250 (0.0008) [2023-10-07 23:16:55,167][67838] Updated weights for policy 0, policy_version 85152 (0.0007) [2023-10-07 23:16:55,506][67871] Updated weights for policy 1, policy_version 85260 (0.0009) [2023-10-07 23:16:55,868][67871] Updated weights for policy 1, policy_version 85270 (0.0007) [2023-10-07 23:16:56,238][67871] Updated weights for policy 1, policy_version 85280 (0.0010) [2023-10-07 23:16:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 174522368. Throughput: 0: 1644.2, 1: 1658.2. Samples: 43632928. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-07 23:16:57,477][66916] Avg episode reward: [(0, '46.380'), (1, '56.990')] [2023-10-07 23:16:59,488][67838] Updated weights for policy 0, policy_version 85162 (0.0008) [2023-10-07 23:16:59,860][67838] Updated weights for policy 0, policy_version 85172 (0.0008) [2023-10-07 23:17:00,224][67838] Updated weights for policy 0, policy_version 85182 (0.0008) [2023-10-07 23:17:00,239][67871] Updated weights for policy 1, policy_version 85290 (0.0009) [2023-10-07 23:17:00,601][67871] Updated weights for policy 1, policy_version 85300 (0.0008) [2023-10-07 23:17:00,962][67871] Updated weights for policy 1, policy_version 85310 (0.0010) [2023-10-07 23:17:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 174587904. Throughput: 0: 1652.2, 1: 1648.4. Samples: 43652176. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-07 23:17:02,477][66916] Avg episode reward: [(0, '48.870'), (1, '58.370')] [2023-10-07 23:17:04,379][67838] Updated weights for policy 0, policy_version 85192 (0.0009) [2023-10-07 23:17:04,753][67838] Updated weights for policy 0, policy_version 85202 (0.0009) [2023-10-07 23:17:05,128][67838] Updated weights for policy 0, policy_version 85212 (0.0008) [2023-10-07 23:17:05,178][67871] Updated weights for policy 1, policy_version 85320 (0.0009) [2023-10-07 23:17:05,551][67871] Updated weights for policy 1, policy_version 85330 (0.0009) [2023-10-07 23:17:05,909][67871] Updated weights for policy 1, policy_version 85340 (0.0007) [2023-10-07 23:17:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 174653440. Throughput: 0: 1650.3, 1: 1664.8. Samples: 43672486. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-07 23:17:07,478][66916] Avg episode reward: [(0, '48.360'), (1, '56.580')] [2023-10-07 23:17:07,490][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000085344_87392256.pth... [2023-10-07 23:17:07,491][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000085216_87261184.pth... [2023-10-07 23:17:07,527][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000083776_85786624.pth [2023-10-07 23:17:07,531][67676] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p1/milestones/checkpoint_000085344_87392256.pth [2023-10-07 23:17:07,533][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000083680_85688320.pth [2023-10-07 23:17:07,537][67511] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p0/milestones/checkpoint_000085216_87261184.pth [2023-10-07 23:17:09,169][67838] Updated weights for policy 0, policy_version 85222 (0.0008) [2023-10-07 23:17:09,553][67838] Updated weights for policy 0, policy_version 85232 (0.0011) [2023-10-07 23:17:09,923][67838] Updated weights for policy 0, policy_version 85242 (0.0008) [2023-10-07 23:17:09,934][67871] Updated weights for policy 1, policy_version 85350 (0.0008) [2023-10-07 23:17:10,297][67871] Updated weights for policy 1, policy_version 85360 (0.0008) [2023-10-07 23:17:10,674][67871] Updated weights for policy 1, policy_version 85370 (0.0008) [2023-10-07 23:17:12,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 174718976. Throughput: 0: 1641.8, 1: 1657.7. Samples: 43682576. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-07 23:17:12,478][66916] Avg episode reward: [(0, '46.790'), (1, '55.570')] [2023-10-07 23:17:14,150][67838] Updated weights for policy 0, policy_version 85252 (0.0009) [2023-10-07 23:17:14,522][67838] Updated weights for policy 0, policy_version 85262 (0.0007) [2023-10-07 23:17:14,836][67871] Updated weights for policy 1, policy_version 85380 (0.0008) [2023-10-07 23:17:14,903][67838] Updated weights for policy 0, policy_version 85272 (0.0007) [2023-10-07 23:17:15,206][67871] Updated weights for policy 1, policy_version 85390 (0.0008) [2023-10-07 23:17:15,567][67871] Updated weights for policy 1, policy_version 85400 (0.0009) [2023-10-07 23:17:17,477][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 174784512. Throughput: 0: 1657.5, 1: 1647.1. Samples: 43701870. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:17:17,478][66916] Avg episode reward: [(0, '50.360'), (1, '56.400')] [2023-10-07 23:17:19,208][67838] Updated weights for policy 0, policy_version 85282 (0.0007) [2023-10-07 23:17:19,610][67838] Updated weights for policy 0, policy_version 85292 (0.0008) [2023-10-07 23:17:19,727][67871] Updated weights for policy 1, policy_version 85410 (0.0009) [2023-10-07 23:17:19,983][67838] Updated weights for policy 0, policy_version 85302 (0.0008) [2023-10-07 23:17:20,098][67871] Updated weights for policy 1, policy_version 85420 (0.0007) [2023-10-07 23:17:20,346][67838] Updated weights for policy 0, policy_version 85312 (0.0011) [2023-10-07 23:17:20,462][67871] Updated weights for policy 1, policy_version 85430 (0.0007) [2023-10-07 23:17:20,834][67871] Updated weights for policy 1, policy_version 85440 (0.0007) [2023-10-07 23:17:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 174850048. Throughput: 0: 1651.0, 1: 1657.7. Samples: 43721796. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:17:22,477][66916] Avg episode reward: [(0, '49.640'), (1, '56.420')] [2023-10-07 23:17:24,509][67838] Updated weights for policy 0, policy_version 85322 (0.0008) [2023-10-07 23:17:24,893][67838] Updated weights for policy 0, policy_version 85332 (0.0009) [2023-10-07 23:17:25,003][67871] Updated weights for policy 1, policy_version 85450 (0.0009) [2023-10-07 23:17:25,264][67838] Updated weights for policy 0, policy_version 85342 (0.0007) [2023-10-07 23:17:25,365][67871] Updated weights for policy 1, policy_version 85460 (0.0009) [2023-10-07 23:17:25,729][67871] Updated weights for policy 1, policy_version 85470 (0.0011) [2023-10-07 23:17:27,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 174915584. Throughput: 0: 1648.2, 1: 1655.4. Samples: 43732134. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:17:27,477][66916] Avg episode reward: [(0, '49.710'), (1, '57.170')] [2023-10-07 23:17:29,394][67838] Updated weights for policy 0, policy_version 85352 (0.0009) [2023-10-07 23:17:29,762][67838] Updated weights for policy 0, policy_version 85362 (0.0008) [2023-10-07 23:17:30,036][67871] Updated weights for policy 1, policy_version 85480 (0.0009) [2023-10-07 23:17:30,138][67838] Updated weights for policy 0, policy_version 85372 (0.0007) [2023-10-07 23:17:30,401][67871] Updated weights for policy 1, policy_version 85490 (0.0009) [2023-10-07 23:17:30,766][67871] Updated weights for policy 1, policy_version 85500 (0.0009) [2023-10-07 23:17:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 174981120. Throughput: 0: 1648.2, 1: 1653.1. Samples: 43751268. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:17:32,478][66916] Avg episode reward: [(0, '47.720'), (1, '57.950')] [2023-10-07 23:17:34,181][67838] Updated weights for policy 0, policy_version 85382 (0.0009) [2023-10-07 23:17:34,553][67838] Updated weights for policy 0, policy_version 85392 (0.0008) [2023-10-07 23:17:34,772][67871] Updated weights for policy 1, policy_version 85510 (0.0009) [2023-10-07 23:17:34,929][67838] Updated weights for policy 0, policy_version 85402 (0.0009) [2023-10-07 23:17:35,148][67871] Updated weights for policy 1, policy_version 85520 (0.0009) [2023-10-07 23:17:35,524][67871] Updated weights for policy 1, policy_version 85530 (0.0009) [2023-10-07 23:17:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175046656. Throughput: 0: 1646.8, 1: 1660.1. Samples: 43771512. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:17:37,477][66916] Avg episode reward: [(0, '46.070'), (1, '62.140')] [2023-10-07 23:17:39,053][67838] Updated weights for policy 0, policy_version 85412 (0.0008) [2023-10-07 23:17:39,424][67838] Updated weights for policy 0, policy_version 85422 (0.0007) [2023-10-07 23:17:39,569][67871] Updated weights for policy 1, policy_version 85540 (0.0011) [2023-10-07 23:17:39,795][67838] Updated weights for policy 0, policy_version 85432 (0.0009) [2023-10-07 23:17:39,940][67871] Updated weights for policy 1, policy_version 85550 (0.0009) [2023-10-07 23:17:40,305][67871] Updated weights for policy 1, policy_version 85560 (0.0009) [2023-10-07 23:17:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 175112192. Throughput: 0: 1646.4, 1: 1649.7. Samples: 43781256. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:17:42,478][66916] Avg episode reward: [(0, '44.000'), (1, '59.260')] [2023-10-07 23:17:43,868][67838] Updated weights for policy 0, policy_version 85442 (0.0010) [2023-10-07 23:17:44,241][67838] Updated weights for policy 0, policy_version 85452 (0.0010) [2023-10-07 23:17:44,568][67871] Updated weights for policy 1, policy_version 85570 (0.0009) [2023-10-07 23:17:44,609][67838] Updated weights for policy 0, policy_version 85462 (0.0008) [2023-10-07 23:17:44,941][67871] Updated weights for policy 1, policy_version 85580 (0.0007) [2023-10-07 23:17:44,982][67838] Updated weights for policy 0, policy_version 85472 (0.0009) [2023-10-07 23:17:45,298][67871] Updated weights for policy 1, policy_version 85590 (0.0008) [2023-10-07 23:17:45,666][67871] Updated weights for policy 1, policy_version 85600 (0.0011) [2023-10-07 23:17:47,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175177728. Throughput: 0: 1649.3, 1: 1652.0. Samples: 43800736. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:17:47,478][66916] Avg episode reward: [(0, '44.800'), (1, '60.780')] [2023-10-07 23:17:49,175][67838] Updated weights for policy 0, policy_version 85482 (0.0008) [2023-10-07 23:17:49,506][67871] Updated weights for policy 1, policy_version 85610 (0.0008) [2023-10-07 23:17:49,547][67838] Updated weights for policy 0, policy_version 85492 (0.0009) [2023-10-07 23:17:49,863][67871] Updated weights for policy 1, policy_version 85620 (0.0009) [2023-10-07 23:17:49,915][67838] Updated weights for policy 0, policy_version 85502 (0.0009) [2023-10-07 23:17:50,236][67871] Updated weights for policy 1, policy_version 85630 (0.0009) [2023-10-07 23:17:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175243264. Throughput: 0: 1646.1, 1: 1664.8. Samples: 43821474. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:17:52,477][66916] Avg episode reward: [(0, '47.480'), (1, '60.800')] [2023-10-07 23:17:54,152][67838] Updated weights for policy 0, policy_version 85512 (0.0009) [2023-10-07 23:17:54,342][67871] Updated weights for policy 1, policy_version 85640 (0.0007) [2023-10-07 23:17:54,526][67838] Updated weights for policy 0, policy_version 85522 (0.0008) [2023-10-07 23:17:54,702][67871] Updated weights for policy 1, policy_version 85650 (0.0007) [2023-10-07 23:17:54,893][67838] Updated weights for policy 0, policy_version 85532 (0.0010) [2023-10-07 23:17:55,069][67871] Updated weights for policy 1, policy_version 85660 (0.0009) [2023-10-07 23:17:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175308800. Throughput: 0: 1645.5, 1: 1653.3. Samples: 43831022. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:17:57,478][66916] Avg episode reward: [(0, '43.410'), (1, '58.290')] [2023-10-07 23:17:59,037][67838] Updated weights for policy 0, policy_version 85542 (0.0009) [2023-10-07 23:17:59,160][67871] Updated weights for policy 1, policy_version 85670 (0.0009) [2023-10-07 23:17:59,405][67838] Updated weights for policy 0, policy_version 85552 (0.0009) [2023-10-07 23:17:59,524][67871] Updated weights for policy 1, policy_version 85680 (0.0008) [2023-10-07 23:17:59,783][67838] Updated weights for policy 0, policy_version 85562 (0.0008) [2023-10-07 23:17:59,894][67871] Updated weights for policy 1, policy_version 85690 (0.0008) [2023-10-07 23:18:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175374336. Throughput: 0: 1645.2, 1: 1665.4. Samples: 43850850. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:18:02,478][66916] Avg episode reward: [(0, '44.150'), (1, '55.920')] [2023-10-07 23:18:04,002][67871] Updated weights for policy 1, policy_version 85700 (0.0009) [2023-10-07 23:18:04,090][67838] Updated weights for policy 0, policy_version 85572 (0.0008) [2023-10-07 23:18:04,370][67871] Updated weights for policy 1, policy_version 85710 (0.0009) [2023-10-07 23:18:04,476][67838] Updated weights for policy 0, policy_version 85582 (0.0008) [2023-10-07 23:18:04,731][67871] Updated weights for policy 1, policy_version 85720 (0.0009) [2023-10-07 23:18:04,842][67838] Updated weights for policy 0, policy_version 85592 (0.0008) [2023-10-07 23:18:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 175439872. Throughput: 0: 1644.6, 1: 1674.9. Samples: 43871176. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-07 23:18:07,477][66916] Avg episode reward: [(0, '45.200'), (1, '57.820')] [2023-10-07 23:18:08,888][67838] Updated weights for policy 0, policy_version 85602 (0.0009) [2023-10-07 23:18:09,006][67871] Updated weights for policy 1, policy_version 85730 (0.0008) [2023-10-07 23:18:09,255][67838] Updated weights for policy 0, policy_version 85612 (0.0010) [2023-10-07 23:18:09,374][67871] Updated weights for policy 1, policy_version 85740 (0.0007) [2023-10-07 23:18:09,627][67838] Updated weights for policy 0, policy_version 85622 (0.0009) [2023-10-07 23:18:09,733][67871] Updated weights for policy 1, policy_version 85750 (0.0009) [2023-10-07 23:18:09,995][67838] Updated weights for policy 0, policy_version 85632 (0.0009) [2023-10-07 23:18:10,107][67871] Updated weights for policy 1, policy_version 85760 (0.0008) [2023-10-07 23:18:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 175505408. Throughput: 0: 1639.1, 1: 1655.0. Samples: 43880370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:18:12,477][66916] Avg episode reward: [(0, '48.030'), (1, '61.280')] [2023-10-07 23:18:13,944][67838] Updated weights for policy 0, policy_version 85642 (0.0008) [2023-10-07 23:18:14,221][67871] Updated weights for policy 1, policy_version 85770 (0.0010) [2023-10-07 23:18:14,315][67838] Updated weights for policy 0, policy_version 85652 (0.0008) [2023-10-07 23:18:14,590][67871] Updated weights for policy 1, policy_version 85780 (0.0010) [2023-10-07 23:18:14,695][67838] Updated weights for policy 0, policy_version 85662 (0.0008) [2023-10-07 23:18:14,955][67871] Updated weights for policy 1, policy_version 85790 (0.0010) [2023-10-07 23:18:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175570944. Throughput: 0: 1650.3, 1: 1667.7. Samples: 43900576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:18:17,478][66916] Avg episode reward: [(0, '44.500'), (1, '58.960')] [2023-10-07 23:18:18,854][67838] Updated weights for policy 0, policy_version 85672 (0.0008) [2023-10-07 23:18:19,223][67871] Updated weights for policy 1, policy_version 85800 (0.0007) [2023-10-07 23:18:19,231][67838] Updated weights for policy 0, policy_version 85682 (0.0009) [2023-10-07 23:18:19,594][67871] Updated weights for policy 1, policy_version 85810 (0.0009) [2023-10-07 23:18:19,602][67838] Updated weights for policy 0, policy_version 85692 (0.0009) [2023-10-07 23:18:19,953][67871] Updated weights for policy 1, policy_version 85820 (0.0008) [2023-10-07 23:18:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175636480. Throughput: 0: 1647.5, 1: 1670.7. Samples: 43920834. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:18:22,478][66916] Avg episode reward: [(0, '43.420'), (1, '60.380')] [2023-10-07 23:18:23,836][67838] Updated weights for policy 0, policy_version 85702 (0.0008) [2023-10-07 23:18:24,087][67871] Updated weights for policy 1, policy_version 85830 (0.0009) [2023-10-07 23:18:24,207][67838] Updated weights for policy 0, policy_version 85712 (0.0007) [2023-10-07 23:18:24,454][67871] Updated weights for policy 1, policy_version 85840 (0.0008) [2023-10-07 23:18:24,585][67838] Updated weights for policy 0, policy_version 85722 (0.0009) [2023-10-07 23:18:24,822][67871] Updated weights for policy 1, policy_version 85850 (0.0008) [2023-10-07 23:18:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 175702016. Throughput: 0: 1642.6, 1: 1662.5. Samples: 43929988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:18:27,478][66916] Avg episode reward: [(0, '48.550'), (1, '56.740')] [2023-10-07 23:18:28,677][67838] Updated weights for policy 0, policy_version 85732 (0.0008) [2023-10-07 23:18:28,798][67871] Updated weights for policy 1, policy_version 85860 (0.0010) [2023-10-07 23:18:29,054][67838] Updated weights for policy 0, policy_version 85742 (0.0008) [2023-10-07 23:18:29,160][67871] Updated weights for policy 1, policy_version 85870 (0.0008) [2023-10-07 23:18:29,418][67838] Updated weights for policy 0, policy_version 85752 (0.0010) [2023-10-07 23:18:29,532][67871] Updated weights for policy 1, policy_version 85880 (0.0007) [2023-10-07 23:18:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175767552. Throughput: 0: 1643.0, 1: 1683.3. Samples: 43950420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:18:32,477][66916] Avg episode reward: [(0, '44.410'), (1, '57.170')] [2023-10-07 23:18:33,467][67871] Updated weights for policy 1, policy_version 85890 (0.0008) [2023-10-07 23:18:33,673][67838] Updated weights for policy 0, policy_version 85762 (0.0009) [2023-10-07 23:18:33,831][67871] Updated weights for policy 1, policy_version 85900 (0.0008) [2023-10-07 23:18:34,034][67838] Updated weights for policy 0, policy_version 85772 (0.0008) [2023-10-07 23:18:34,200][67871] Updated weights for policy 1, policy_version 85910 (0.0008) [2023-10-07 23:18:34,397][67838] Updated weights for policy 0, policy_version 85782 (0.0009) [2023-10-07 23:18:34,558][67871] Updated weights for policy 1, policy_version 85920 (0.0008) [2023-10-07 23:18:34,766][67838] Updated weights for policy 0, policy_version 85792 (0.0007) [2023-10-07 23:18:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175833088. Throughput: 0: 1643.1, 1: 1676.4. Samples: 43970850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:18:37,477][66916] Avg episode reward: [(0, '48.480'), (1, '57.640')] [2023-10-07 23:18:38,725][67871] Updated weights for policy 1, policy_version 85930 (0.0009) [2023-10-07 23:18:39,090][67871] Updated weights for policy 1, policy_version 85940 (0.0008) [2023-10-07 23:18:39,174][67838] Updated weights for policy 0, policy_version 85802 (0.0009) [2023-10-07 23:18:39,463][67871] Updated weights for policy 1, policy_version 85950 (0.0008) [2023-10-07 23:18:39,546][67838] Updated weights for policy 0, policy_version 85812 (0.0008) [2023-10-07 23:18:39,916][67838] Updated weights for policy 0, policy_version 85822 (0.0010) [2023-10-07 23:18:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 175898624. Throughput: 0: 1641.0, 1: 1662.6. Samples: 43979682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:18:42,478][66916] Avg episode reward: [(0, '47.500'), (1, '61.390')] [2023-10-07 23:18:43,569][67871] Updated weights for policy 1, policy_version 85960 (0.0007) [2023-10-07 23:18:43,939][67871] Updated weights for policy 1, policy_version 85970 (0.0008) [2023-10-07 23:18:44,163][67838] Updated weights for policy 0, policy_version 85832 (0.0008) [2023-10-07 23:18:44,308][67871] Updated weights for policy 1, policy_version 85980 (0.0010) [2023-10-07 23:18:44,530][67838] Updated weights for policy 0, policy_version 85842 (0.0009) [2023-10-07 23:18:44,897][67838] Updated weights for policy 0, policy_version 85852 (0.0008) [2023-10-07 23:18:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 175964160. Throughput: 0: 1640.6, 1: 1676.9. Samples: 44000138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:18:47,478][66916] Avg episode reward: [(0, '45.670'), (1, '60.700')] [2023-10-07 23:18:48,426][67871] Updated weights for policy 1, policy_version 85990 (0.0008) [2023-10-07 23:18:48,805][67871] Updated weights for policy 1, policy_version 86000 (0.0009) [2023-10-07 23:18:49,171][67871] Updated weights for policy 1, policy_version 86010 (0.0008) [2023-10-07 23:18:49,258][67838] Updated weights for policy 0, policy_version 85862 (0.0008) [2023-10-07 23:18:49,628][67838] Updated weights for policy 0, policy_version 85872 (0.0008) [2023-10-07 23:18:50,000][67838] Updated weights for policy 0, policy_version 85882 (0.0008) [2023-10-07 23:18:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 176029696. Throughput: 0: 1646.6, 1: 1670.6. Samples: 44020450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:18:52,477][66916] Avg episode reward: [(0, '45.750'), (1, '64.490')] [2023-10-07 23:18:53,300][67871] Updated weights for policy 1, policy_version 86020 (0.0009) [2023-10-07 23:18:53,672][67871] Updated weights for policy 1, policy_version 86030 (0.0007) [2023-10-07 23:18:54,038][67871] Updated weights for policy 1, policy_version 86040 (0.0008) [2023-10-07 23:18:54,081][67838] Updated weights for policy 0, policy_version 85892 (0.0008) [2023-10-07 23:18:54,458][67838] Updated weights for policy 0, policy_version 85902 (0.0008) [2023-10-07 23:18:54,826][67838] Updated weights for policy 0, policy_version 85912 (0.0009) [2023-10-07 23:18:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 176095232. Throughput: 0: 1649.7, 1: 1666.6. Samples: 44029606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:18:57,478][66916] Avg episode reward: [(0, '44.040'), (1, '66.100')] [2023-10-07 23:18:58,253][67871] Updated weights for policy 1, policy_version 86050 (0.0008) [2023-10-07 23:18:58,616][67871] Updated weights for policy 1, policy_version 86060 (0.0007) [2023-10-07 23:18:58,871][67838] Updated weights for policy 0, policy_version 85922 (0.0008) [2023-10-07 23:18:58,982][67871] Updated weights for policy 1, policy_version 86070 (0.0007) [2023-10-07 23:18:59,233][67838] Updated weights for policy 0, policy_version 85932 (0.0009) [2023-10-07 23:18:59,347][67871] Updated weights for policy 1, policy_version 86080 (0.0007) [2023-10-07 23:18:59,613][67838] Updated weights for policy 0, policy_version 85942 (0.0010) [2023-10-07 23:18:59,993][67838] Updated weights for policy 0, policy_version 85952 (0.0009) [2023-10-07 23:19:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 176160768. Throughput: 0: 1642.0, 1: 1674.2. Samples: 44049802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:19:02,477][66916] Avg episode reward: [(0, '45.760'), (1, '64.660')] [2023-10-07 23:19:03,553][67871] Updated weights for policy 1, policy_version 86090 (0.0009) [2023-10-07 23:19:03,929][67871] Updated weights for policy 1, policy_version 86100 (0.0008) [2023-10-07 23:19:04,117][67838] Updated weights for policy 0, policy_version 85962 (0.0009) [2023-10-07 23:19:04,288][67871] Updated weights for policy 1, policy_version 86110 (0.0009) [2023-10-07 23:19:04,489][67838] Updated weights for policy 0, policy_version 85972 (0.0008) [2023-10-07 23:19:04,865][67838] Updated weights for policy 0, policy_version 85982 (0.0008) [2023-10-07 23:19:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 176226304. Throughput: 0: 1643.4, 1: 1675.2. Samples: 44070168. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 23:19:07,478][66916] Avg episode reward: [(0, '45.490'), (1, '63.440')] [2023-10-07 23:19:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000086112_88178688.pth... [2023-10-07 23:19:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000085984_88047616.pth... [2023-10-07 23:19:07,521][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000084576_86605824.pth [2023-10-07 23:19:07,529][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000084448_86474752.pth [2023-10-07 23:19:08,193][67871] Updated weights for policy 1, policy_version 86120 (0.0008) [2023-10-07 23:19:08,556][67871] Updated weights for policy 1, policy_version 86130 (0.0007) [2023-10-07 23:19:08,920][67838] Updated weights for policy 0, policy_version 85992 (0.0008) [2023-10-07 23:19:08,926][67871] Updated weights for policy 1, policy_version 86140 (0.0008) [2023-10-07 23:19:09,298][67838] Updated weights for policy 0, policy_version 86002 (0.0009) [2023-10-07 23:19:09,661][67838] Updated weights for policy 0, policy_version 86012 (0.0009) [2023-10-07 23:19:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 176291840. Throughput: 0: 1645.2, 1: 1669.7. Samples: 44079158. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 23:19:12,477][66916] Avg episode reward: [(0, '46.110'), (1, '62.950')] [2023-10-07 23:19:12,949][67871] Updated weights for policy 1, policy_version 86150 (0.0008) [2023-10-07 23:19:13,329][67871] Updated weights for policy 1, policy_version 86160 (0.0007) [2023-10-07 23:19:13,700][67871] Updated weights for policy 1, policy_version 86170 (0.0007) [2023-10-07 23:19:13,709][67838] Updated weights for policy 0, policy_version 86022 (0.0009) [2023-10-07 23:19:14,082][67838] Updated weights for policy 0, policy_version 86032 (0.0008) [2023-10-07 23:19:14,454][67838] Updated weights for policy 0, policy_version 86042 (0.0009) [2023-10-07 23:19:17,477][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 176357376. Throughput: 0: 1648.1, 1: 1666.9. Samples: 44099594. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 23:19:17,478][66916] Avg episode reward: [(0, '47.320'), (1, '59.070')] [2023-10-07 23:19:17,896][67871] Updated weights for policy 1, policy_version 86180 (0.0008) [2023-10-07 23:19:18,260][67871] Updated weights for policy 1, policy_version 86190 (0.0008) [2023-10-07 23:19:18,626][67871] Updated weights for policy 1, policy_version 86200 (0.0007) [2023-10-07 23:19:18,700][67838] Updated weights for policy 0, policy_version 86052 (0.0008) [2023-10-07 23:19:19,062][67838] Updated weights for policy 0, policy_version 86062 (0.0011) [2023-10-07 23:19:19,430][67838] Updated weights for policy 0, policy_version 86072 (0.0010) [2023-10-07 23:19:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 176422912. Throughput: 0: 1647.8, 1: 1666.3. Samples: 44119986. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 23:19:22,477][66916] Avg episode reward: [(0, '45.870'), (1, '60.870')] [2023-10-07 23:19:22,632][67871] Updated weights for policy 1, policy_version 86210 (0.0007) [2023-10-07 23:19:22,998][67871] Updated weights for policy 1, policy_version 86220 (0.0007) [2023-10-07 23:19:23,371][67871] Updated weights for policy 1, policy_version 86230 (0.0007) [2023-10-07 23:19:23,415][67838] Updated weights for policy 0, policy_version 86082 (0.0008) [2023-10-07 23:19:23,734][67871] Updated weights for policy 1, policy_version 86240 (0.0008) [2023-10-07 23:19:23,773][67838] Updated weights for policy 0, policy_version 86092 (0.0008) [2023-10-07 23:19:24,151][67838] Updated weights for policy 0, policy_version 86102 (0.0008) [2023-10-07 23:19:24,520][67838] Updated weights for policy 0, policy_version 86112 (0.0007) [2023-10-07 23:19:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 176488448. Throughput: 0: 1652.5, 1: 1667.4. Samples: 44129074. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 23:19:27,477][66916] Avg episode reward: [(0, '45.530'), (1, '60.200')] [2023-10-07 23:19:27,815][67871] Updated weights for policy 1, policy_version 86250 (0.0010) [2023-10-07 23:19:28,184][67871] Updated weights for policy 1, policy_version 86260 (0.0010) [2023-10-07 23:19:28,554][67871] Updated weights for policy 1, policy_version 86270 (0.0008) [2023-10-07 23:19:28,643][67838] Updated weights for policy 0, policy_version 86122 (0.0007) [2023-10-07 23:19:29,000][67838] Updated weights for policy 0, policy_version 86132 (0.0007) [2023-10-07 23:19:29,378][67838] Updated weights for policy 0, policy_version 86142 (0.0010) [2023-10-07 23:19:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 176553984. Throughput: 0: 1659.5, 1: 1667.5. Samples: 44149850. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 23:19:32,477][66916] Avg episode reward: [(0, '44.790'), (1, '62.620')] [2023-10-07 23:19:32,601][67871] Updated weights for policy 1, policy_version 86280 (0.0009) [2023-10-07 23:19:32,970][67871] Updated weights for policy 1, policy_version 86290 (0.0008) [2023-10-07 23:19:33,337][67871] Updated weights for policy 1, policy_version 86300 (0.0007) [2023-10-07 23:19:33,417][67838] Updated weights for policy 0, policy_version 86152 (0.0008) [2023-10-07 23:19:33,787][67838] Updated weights for policy 0, policy_version 86162 (0.0008) [2023-10-07 23:19:34,166][67838] Updated weights for policy 0, policy_version 86172 (0.0008) [2023-10-07 23:19:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 176619520. Throughput: 0: 1665.4, 1: 1669.0. Samples: 44170496. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 23:19:37,478][66916] Avg episode reward: [(0, '39.970'), (1, '63.130')] [2023-10-07 23:19:37,735][67871] Updated weights for policy 1, policy_version 86310 (0.0007) [2023-10-07 23:19:38,096][67871] Updated weights for policy 1, policy_version 86320 (0.0009) [2023-10-07 23:19:38,263][67838] Updated weights for policy 0, policy_version 86182 (0.0009) [2023-10-07 23:19:38,459][67871] Updated weights for policy 1, policy_version 86330 (0.0007) [2023-10-07 23:19:38,645][67838] Updated weights for policy 0, policy_version 86192 (0.0008) [2023-10-07 23:19:39,012][67838] Updated weights for policy 0, policy_version 86202 (0.0010) [2023-10-07 23:19:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 176685056. Throughput: 0: 1659.4, 1: 1665.2. Samples: 44179212. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 23:19:42,477][66916] Avg episode reward: [(0, '42.110'), (1, '61.530')] [2023-10-07 23:19:42,774][67871] Updated weights for policy 1, policy_version 86340 (0.0008) [2023-10-07 23:19:43,131][67871] Updated weights for policy 1, policy_version 86350 (0.0010) [2023-10-07 23:19:43,195][67838] Updated weights for policy 0, policy_version 86212 (0.0010) [2023-10-07 23:19:43,498][67871] Updated weights for policy 1, policy_version 86360 (0.0008) [2023-10-07 23:19:43,565][67838] Updated weights for policy 0, policy_version 86222 (0.0009) [2023-10-07 23:19:43,925][67838] Updated weights for policy 0, policy_version 86232 (0.0009) [2023-10-07 23:19:47,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 176750592. Throughput: 0: 1664.8, 1: 1657.1. Samples: 44199286. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 23:19:47,477][66916] Avg episode reward: [(0, '44.200'), (1, '62.200')] [2023-10-07 23:19:47,656][67871] Updated weights for policy 1, policy_version 86370 (0.0007) [2023-10-07 23:19:48,007][67838] Updated weights for policy 0, policy_version 86242 (0.0010) [2023-10-07 23:19:48,019][67871] Updated weights for policy 1, policy_version 86380 (0.0007) [2023-10-07 23:19:48,377][67838] Updated weights for policy 0, policy_version 86252 (0.0008) [2023-10-07 23:19:48,389][67871] Updated weights for policy 1, policy_version 86390 (0.0010) [2023-10-07 23:19:48,756][67871] Updated weights for policy 1, policy_version 86400 (0.0009) [2023-10-07 23:19:48,761][67838] Updated weights for policy 0, policy_version 86262 (0.0008) [2023-10-07 23:19:49,133][67838] Updated weights for policy 0, policy_version 86272 (0.0011) [2023-10-07 23:19:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 176816128. Throughput: 0: 1671.3, 1: 1651.6. Samples: 44219696. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 23:19:52,478][66916] Avg episode reward: [(0, '44.810'), (1, '61.930')] [2023-10-07 23:19:52,890][67871] Updated weights for policy 1, policy_version 86410 (0.0007) [2023-10-07 23:19:53,239][67838] Updated weights for policy 0, policy_version 86282 (0.0007) [2023-10-07 23:19:53,251][67871] Updated weights for policy 1, policy_version 86420 (0.0008) [2023-10-07 23:19:53,607][67838] Updated weights for policy 0, policy_version 86292 (0.0009) [2023-10-07 23:19:53,619][67871] Updated weights for policy 1, policy_version 86430 (0.0007) [2023-10-07 23:19:53,985][67838] Updated weights for policy 0, policy_version 86302 (0.0008) [2023-10-07 23:19:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 176881664. Throughput: 0: 1669.2, 1: 1652.4. Samples: 44228630. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-07 23:19:57,477][66916] Avg episode reward: [(0, '42.340'), (1, '61.770')] [2023-10-07 23:19:57,768][67871] Updated weights for policy 1, policy_version 86440 (0.0007) [2023-10-07 23:19:58,020][67838] Updated weights for policy 0, policy_version 86312 (0.0008) [2023-10-07 23:19:58,139][67871] Updated weights for policy 1, policy_version 86450 (0.0009) [2023-10-07 23:19:58,391][67838] Updated weights for policy 0, policy_version 86322 (0.0008) [2023-10-07 23:19:58,505][67871] Updated weights for policy 1, policy_version 86460 (0.0008) [2023-10-07 23:19:58,764][67838] Updated weights for policy 0, policy_version 86332 (0.0008) [2023-10-07 23:20:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 176947200. Throughput: 0: 1669.2, 1: 1647.8. Samples: 44248858. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 23:20:02,477][66916] Avg episode reward: [(0, '46.220'), (1, '61.080')] [2023-10-07 23:20:02,607][67871] Updated weights for policy 1, policy_version 86470 (0.0008) [2023-10-07 23:20:02,803][67838] Updated weights for policy 0, policy_version 86342 (0.0007) [2023-10-07 23:20:02,974][67871] Updated weights for policy 1, policy_version 86480 (0.0007) [2023-10-07 23:20:03,173][67838] Updated weights for policy 0, policy_version 86352 (0.0008) [2023-10-07 23:20:03,336][67871] Updated weights for policy 1, policy_version 86490 (0.0007) [2023-10-07 23:20:03,540][67838] Updated weights for policy 0, policy_version 86362 (0.0008) [2023-10-07 23:20:07,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 177012736. Throughput: 0: 1673.8, 1: 1641.5. Samples: 44269174. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 23:20:07,477][66916] Avg episode reward: [(0, '44.040'), (1, '60.400')] [2023-10-07 23:20:07,709][67838] Updated weights for policy 0, policy_version 86372 (0.0008) [2023-10-07 23:20:07,720][67871] Updated weights for policy 1, policy_version 86500 (0.0010) [2023-10-07 23:20:08,075][67838] Updated weights for policy 0, policy_version 86382 (0.0008) [2023-10-07 23:20:08,094][67871] Updated weights for policy 1, policy_version 86510 (0.0010) [2023-10-07 23:20:08,451][67871] Updated weights for policy 1, policy_version 86520 (0.0008) [2023-10-07 23:20:08,452][67838] Updated weights for policy 0, policy_version 86392 (0.0008) [2023-10-07 23:20:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 177078272. Throughput: 0: 1671.8, 1: 1640.0. Samples: 44278104. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 23:20:12,477][66916] Avg episode reward: [(0, '43.110'), (1, '62.140')] [2023-10-07 23:20:12,553][67838] Updated weights for policy 0, policy_version 86402 (0.0009) [2023-10-07 23:20:12,662][67871] Updated weights for policy 1, policy_version 86530 (0.0009) [2023-10-07 23:20:12,922][67838] Updated weights for policy 0, policy_version 86412 (0.0009) [2023-10-07 23:20:13,024][67871] Updated weights for policy 1, policy_version 86540 (0.0008) [2023-10-07 23:20:13,289][67838] Updated weights for policy 0, policy_version 86422 (0.0008) [2023-10-07 23:20:13,388][67871] Updated weights for policy 1, policy_version 86550 (0.0008) [2023-10-07 23:20:13,649][67838] Updated weights for policy 0, policy_version 86432 (0.0008) [2023-10-07 23:20:13,751][67871] Updated weights for policy 1, policy_version 86560 (0.0008) [2023-10-07 23:20:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 177143808. Throughput: 0: 1665.7, 1: 1635.1. Samples: 44298390. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 23:20:17,478][66916] Avg episode reward: [(0, '45.560'), (1, '61.430')] [2023-10-07 23:20:17,818][67838] Updated weights for policy 0, policy_version 86442 (0.0009) [2023-10-07 23:20:17,903][67871] Updated weights for policy 1, policy_version 86570 (0.0008) [2023-10-07 23:20:18,175][67838] Updated weights for policy 0, policy_version 86452 (0.0009) [2023-10-07 23:20:18,269][67871] Updated weights for policy 1, policy_version 86580 (0.0009) [2023-10-07 23:20:18,543][67838] Updated weights for policy 0, policy_version 86462 (0.0007) [2023-10-07 23:20:18,637][67871] Updated weights for policy 1, policy_version 86590 (0.0007) [2023-10-07 23:20:22,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 177209344. Throughput: 0: 1661.5, 1: 1632.3. Samples: 44318716. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 23:20:22,477][66916] Avg episode reward: [(0, '42.920'), (1, '62.440')] [2023-10-07 23:20:22,698][67871] Updated weights for policy 1, policy_version 86600 (0.0007) [2023-10-07 23:20:22,774][67838] Updated weights for policy 0, policy_version 86472 (0.0008) [2023-10-07 23:20:23,067][67871] Updated weights for policy 1, policy_version 86610 (0.0007) [2023-10-07 23:20:23,144][67838] Updated weights for policy 0, policy_version 86482 (0.0010) [2023-10-07 23:20:23,435][67871] Updated weights for policy 1, policy_version 86620 (0.0010) [2023-10-07 23:20:23,509][67838] Updated weights for policy 0, policy_version 86492 (0.0007) [2023-10-07 23:20:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 177274880. Throughput: 0: 1666.4, 1: 1633.6. Samples: 44327712. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 23:20:27,477][66916] Avg episode reward: [(0, '42.050'), (1, '59.910')] [2023-10-07 23:20:27,538][67871] Updated weights for policy 1, policy_version 86630 (0.0009) [2023-10-07 23:20:27,726][67838] Updated weights for policy 0, policy_version 86502 (0.0007) [2023-10-07 23:20:27,904][67871] Updated weights for policy 1, policy_version 86640 (0.0008) [2023-10-07 23:20:28,114][67838] Updated weights for policy 0, policy_version 86512 (0.0009) [2023-10-07 23:20:28,268][67871] Updated weights for policy 1, policy_version 86650 (0.0008) [2023-10-07 23:20:28,479][67838] Updated weights for policy 0, policy_version 86522 (0.0009) [2023-10-07 23:20:32,429][67871] Updated weights for policy 1, policy_version 86660 (0.0008) [2023-10-07 23:20:32,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 177340416. Throughput: 0: 1659.8, 1: 1644.7. Samples: 44347988. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 23:20:32,478][66916] Avg episode reward: [(0, '40.780'), (1, '62.720')] [2023-10-07 23:20:32,510][67838] Updated weights for policy 0, policy_version 86532 (0.0010) [2023-10-07 23:20:32,790][67871] Updated weights for policy 1, policy_version 86670 (0.0007) [2023-10-07 23:20:32,886][67838] Updated weights for policy 0, policy_version 86542 (0.0009) [2023-10-07 23:20:33,154][67871] Updated weights for policy 1, policy_version 86680 (0.0009) [2023-10-07 23:20:33,248][67838] Updated weights for policy 0, policy_version 86552 (0.0007) [2023-10-07 23:20:37,299][67838] Updated weights for policy 0, policy_version 86562 (0.0007) [2023-10-07 23:20:37,402][67871] Updated weights for policy 1, policy_version 86690 (0.0007) [2023-10-07 23:20:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 177405952. Throughput: 0: 1657.7, 1: 1650.5. Samples: 44368566. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 23:20:37,478][66916] Avg episode reward: [(0, '42.870'), (1, '65.620')] [2023-10-07 23:20:37,675][67838] Updated weights for policy 0, policy_version 86572 (0.0008) [2023-10-07 23:20:37,774][67871] Updated weights for policy 1, policy_version 86700 (0.0007) [2023-10-07 23:20:38,043][67838] Updated weights for policy 0, policy_version 86582 (0.0009) [2023-10-07 23:20:38,140][67871] Updated weights for policy 1, policy_version 86710 (0.0009) [2023-10-07 23:20:38,405][67838] Updated weights for policy 0, policy_version 86592 (0.0009) [2023-10-07 23:20:38,504][67871] Updated weights for policy 1, policy_version 86720 (0.0008) [2023-10-07 23:20:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 177471488. Throughput: 0: 1661.3, 1: 1648.0. Samples: 44377550. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 23:20:42,477][66916] Avg episode reward: [(0, '43.380'), (1, '64.810')] [2023-10-07 23:20:42,564][67838] Updated weights for policy 0, policy_version 86602 (0.0008) [2023-10-07 23:20:42,794][67871] Updated weights for policy 1, policy_version 86730 (0.0007) [2023-10-07 23:20:42,943][67838] Updated weights for policy 0, policy_version 86612 (0.0008) [2023-10-07 23:20:43,160][67871] Updated weights for policy 1, policy_version 86740 (0.0008) [2023-10-07 23:20:43,316][67838] Updated weights for policy 0, policy_version 86622 (0.0009) [2023-10-07 23:20:43,526][67871] Updated weights for policy 1, policy_version 86750 (0.0008) [2023-10-07 23:20:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 177537024. Throughput: 0: 1656.2, 1: 1647.4. Samples: 44397520. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 23:20:47,478][66916] Avg episode reward: [(0, '46.150'), (1, '64.370')] [2023-10-07 23:20:47,526][67838] Updated weights for policy 0, policy_version 86632 (0.0008) [2023-10-07 23:20:47,589][67871] Updated weights for policy 1, policy_version 86760 (0.0008) [2023-10-07 23:20:47,903][67838] Updated weights for policy 0, policy_version 86642 (0.0009) [2023-10-07 23:20:47,959][67871] Updated weights for policy 1, policy_version 86770 (0.0008) [2023-10-07 23:20:48,266][67838] Updated weights for policy 0, policy_version 86652 (0.0009) [2023-10-07 23:20:48,321][67871] Updated weights for policy 1, policy_version 86780 (0.0007) [2023-10-07 23:20:52,382][67838] Updated weights for policy 0, policy_version 86662 (0.0008) [2023-10-07 23:20:52,395][67871] Updated weights for policy 1, policy_version 86790 (0.0008) [2023-10-07 23:20:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 177602560. Throughput: 0: 1654.0, 1: 1652.9. Samples: 44417982. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-07 23:20:52,478][66916] Avg episode reward: [(0, '47.280'), (1, '64.720')] [2023-10-07 23:20:52,754][67838] Updated weights for policy 0, policy_version 86672 (0.0009) [2023-10-07 23:20:52,759][67871] Updated weights for policy 1, policy_version 86800 (0.0009) [2023-10-07 23:20:53,119][67871] Updated weights for policy 1, policy_version 86810 (0.0008) [2023-10-07 23:20:53,122][67838] Updated weights for policy 0, policy_version 86682 (0.0007) [2023-10-07 23:20:57,195][67838] Updated weights for policy 0, policy_version 86692 (0.0008) [2023-10-07 23:20:57,306][67871] Updated weights for policy 1, policy_version 86820 (0.0008) [2023-10-07 23:20:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 177668096. Throughput: 0: 1652.8, 1: 1654.1. Samples: 44426918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:20:57,477][66916] Avg episode reward: [(0, '45.390'), (1, '64.970')] [2023-10-07 23:20:57,567][67838] Updated weights for policy 0, policy_version 86702 (0.0008) [2023-10-07 23:20:57,674][67871] Updated weights for policy 1, policy_version 86830 (0.0008) [2023-10-07 23:20:57,941][67838] Updated weights for policy 0, policy_version 86712 (0.0009) [2023-10-07 23:20:58,033][67871] Updated weights for policy 1, policy_version 86840 (0.0007) [2023-10-07 23:21:02,123][67838] Updated weights for policy 0, policy_version 86722 (0.0009) [2023-10-07 23:21:02,223][67871] Updated weights for policy 1, policy_version 86850 (0.0008) [2023-10-07 23:21:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 177733632. Throughput: 0: 1656.8, 1: 1654.1. Samples: 44447378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:21:02,477][66916] Avg episode reward: [(0, '45.640'), (1, '64.660')] [2023-10-07 23:21:02,491][67838] Updated weights for policy 0, policy_version 86732 (0.0007) [2023-10-07 23:21:02,590][67871] Updated weights for policy 1, policy_version 86860 (0.0007) [2023-10-07 23:21:02,860][67838] Updated weights for policy 0, policy_version 86742 (0.0007) [2023-10-07 23:21:02,963][67871] Updated weights for policy 1, policy_version 86870 (0.0008) [2023-10-07 23:21:03,239][67838] Updated weights for policy 0, policy_version 86752 (0.0009) [2023-10-07 23:21:03,333][67871] Updated weights for policy 1, policy_version 86880 (0.0008) [2023-10-07 23:21:07,337][67838] Updated weights for policy 0, policy_version 86762 (0.0009) [2023-10-07 23:21:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 177799168. Throughput: 0: 1654.3, 1: 1658.6. Samples: 44467796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:21:07,478][66916] Avg episode reward: [(0, '44.750'), (1, '66.110')] [2023-10-07 23:21:07,523][67871] Updated weights for policy 1, policy_version 86890 (0.0009) [2023-10-07 23:21:07,709][67838] Updated weights for policy 0, policy_version 86772 (0.0009) [2023-10-07 23:21:07,893][67871] Updated weights for policy 1, policy_version 86900 (0.0009) [2023-10-07 23:21:08,088][67838] Updated weights for policy 0, policy_version 86782 (0.0009) [2023-10-07 23:21:08,163][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000086784_88866816.pth... [2023-10-07 23:21:08,191][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000085216_87261184.pth [2023-10-07 23:21:08,268][67871] Updated weights for policy 1, policy_version 86910 (0.0010) [2023-10-07 23:21:08,340][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000086912_88997888.pth... [2023-10-07 23:21:08,379][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000085344_87392256.pth [2023-10-07 23:21:12,275][67838] Updated weights for policy 0, policy_version 86792 (0.0008) [2023-10-07 23:21:12,437][67871] Updated weights for policy 1, policy_version 86920 (0.0008) [2023-10-07 23:21:12,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 177864704. Throughput: 0: 1652.9, 1: 1659.7. Samples: 44476782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:21:12,478][66916] Avg episode reward: [(0, '44.980'), (1, '65.820')] [2023-10-07 23:21:12,648][67838] Updated weights for policy 0, policy_version 86802 (0.0007) [2023-10-07 23:21:12,805][67871] Updated weights for policy 1, policy_version 86930 (0.0007) [2023-10-07 23:21:13,008][67838] Updated weights for policy 0, policy_version 86812 (0.0009) [2023-10-07 23:21:13,168][67871] Updated weights for policy 1, policy_version 86940 (0.0008) [2023-10-07 23:21:17,132][67838] Updated weights for policy 0, policy_version 86822 (0.0008) [2023-10-07 23:21:17,293][67871] Updated weights for policy 1, policy_version 86950 (0.0008) [2023-10-07 23:21:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 177930240. Throughput: 0: 1655.3, 1: 1654.1. Samples: 44496912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:21:17,478][66916] Avg episode reward: [(0, '45.410'), (1, '64.410')] [2023-10-07 23:21:17,507][67838] Updated weights for policy 0, policy_version 86832 (0.0008) [2023-10-07 23:21:17,660][67871] Updated weights for policy 1, policy_version 86960 (0.0008) [2023-10-07 23:21:17,876][67838] Updated weights for policy 0, policy_version 86842 (0.0007) [2023-10-07 23:21:18,034][67871] Updated weights for policy 1, policy_version 86970 (0.0009) [2023-10-07 23:21:22,093][67838] Updated weights for policy 0, policy_version 86852 (0.0008) [2023-10-07 23:21:22,255][67871] Updated weights for policy 1, policy_version 86980 (0.0009) [2023-10-07 23:21:22,460][67838] Updated weights for policy 0, policy_version 86862 (0.0009) [2023-10-07 23:21:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 177995776. Throughput: 0: 1651.7, 1: 1651.6. Samples: 44517216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:21:22,478][66916] Avg episode reward: [(0, '46.690'), (1, '63.530')] [2023-10-07 23:21:22,615][67871] Updated weights for policy 1, policy_version 86990 (0.0009) [2023-10-07 23:21:22,830][67838] Updated weights for policy 0, policy_version 86872 (0.0008) [2023-10-07 23:21:22,977][67871] Updated weights for policy 1, policy_version 87000 (0.0010) [2023-10-07 23:21:26,958][67838] Updated weights for policy 0, policy_version 86882 (0.0009) [2023-10-07 23:21:27,053][67871] Updated weights for policy 1, policy_version 87010 (0.0010) [2023-10-07 23:21:27,326][67838] Updated weights for policy 0, policy_version 86892 (0.0009) [2023-10-07 23:21:27,423][67871] Updated weights for policy 1, policy_version 87020 (0.0009) [2023-10-07 23:21:27,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 178061312. Throughput: 0: 1652.8, 1: 1650.2. Samples: 44526182. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:21:27,477][66916] Avg episode reward: [(0, '47.080'), (1, '60.160')] [2023-10-07 23:21:27,690][67838] Updated weights for policy 0, policy_version 86902 (0.0008) [2023-10-07 23:21:27,780][67871] Updated weights for policy 1, policy_version 87030 (0.0008) [2023-10-07 23:21:28,065][67838] Updated weights for policy 0, policy_version 86912 (0.0009) [2023-10-07 23:21:28,144][67871] Updated weights for policy 1, policy_version 87040 (0.0007) [2023-10-07 23:21:32,352][67838] Updated weights for policy 0, policy_version 86922 (0.0008) [2023-10-07 23:21:32,427][67871] Updated weights for policy 1, policy_version 87050 (0.0009) [2023-10-07 23:21:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 178126848. Throughput: 0: 1653.0, 1: 1658.6. Samples: 44546544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:21:32,477][66916] Avg episode reward: [(0, '45.970'), (1, '63.950')] [2023-10-07 23:21:32,703][67838] Updated weights for policy 0, policy_version 86932 (0.0008) [2023-10-07 23:21:32,791][67871] Updated weights for policy 1, policy_version 87060 (0.0008) [2023-10-07 23:21:33,076][67838] Updated weights for policy 0, policy_version 86942 (0.0008) [2023-10-07 23:21:33,164][67871] Updated weights for policy 1, policy_version 87070 (0.0008) [2023-10-07 23:21:37,165][67838] Updated weights for policy 0, policy_version 86952 (0.0007) [2023-10-07 23:21:37,354][67871] Updated weights for policy 1, policy_version 87080 (0.0008) [2023-10-07 23:21:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 178192384. Throughput: 0: 1649.5, 1: 1651.9. Samples: 44566544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:21:37,477][66916] Avg episode reward: [(0, '49.780'), (1, '60.710')] [2023-10-07 23:21:37,526][67838] Updated weights for policy 0, policy_version 86962 (0.0009) [2023-10-07 23:21:37,722][67871] Updated weights for policy 1, policy_version 87090 (0.0007) [2023-10-07 23:21:37,901][67838] Updated weights for policy 0, policy_version 86972 (0.0007) [2023-10-07 23:21:38,087][67871] Updated weights for policy 1, policy_version 87100 (0.0007) [2023-10-07 23:21:42,003][67838] Updated weights for policy 0, policy_version 86982 (0.0009) [2023-10-07 23:21:42,314][67871] Updated weights for policy 1, policy_version 87110 (0.0008) [2023-10-07 23:21:42,383][67838] Updated weights for policy 0, policy_version 86992 (0.0009) [2023-10-07 23:21:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 178257920. Throughput: 0: 1655.3, 1: 1652.1. Samples: 44575752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:21:42,477][66916] Avg episode reward: [(0, '42.950'), (1, '58.800')] [2023-10-07 23:21:42,677][67871] Updated weights for policy 1, policy_version 87120 (0.0008) [2023-10-07 23:21:42,760][67838] Updated weights for policy 0, policy_version 87002 (0.0009) [2023-10-07 23:21:43,042][67871] Updated weights for policy 1, policy_version 87130 (0.0008) [2023-10-07 23:21:47,009][67838] Updated weights for policy 0, policy_version 87012 (0.0008) [2023-10-07 23:21:47,071][67871] Updated weights for policy 1, policy_version 87140 (0.0008) [2023-10-07 23:21:47,368][67838] Updated weights for policy 0, policy_version 87022 (0.0008) [2023-10-07 23:21:47,442][67871] Updated weights for policy 1, policy_version 87150 (0.0008) [2023-10-07 23:21:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 178323456. Throughput: 0: 1649.3, 1: 1652.5. Samples: 44595960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:21:47,477][66916] Avg episode reward: [(0, '43.560'), (1, '62.990')] [2023-10-07 23:21:47,747][67838] Updated weights for policy 0, policy_version 87032 (0.0010) [2023-10-07 23:21:47,805][67871] Updated weights for policy 1, policy_version 87160 (0.0009) [2023-10-07 23:21:51,870][67871] Updated weights for policy 1, policy_version 87170 (0.0008) [2023-10-07 23:21:51,940][67838] Updated weights for policy 0, policy_version 87042 (0.0009) [2023-10-07 23:21:52,242][67871] Updated weights for policy 1, policy_version 87180 (0.0007) [2023-10-07 23:21:52,311][67838] Updated weights for policy 0, policy_version 87052 (0.0009) [2023-10-07 23:21:52,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 178388992. Throughput: 0: 1643.4, 1: 1652.5. Samples: 44616110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:21:52,477][66916] Avg episode reward: [(0, '46.730'), (1, '63.660')] [2023-10-07 23:21:52,603][67871] Updated weights for policy 1, policy_version 87190 (0.0007) [2023-10-07 23:21:52,680][67838] Updated weights for policy 0, policy_version 87062 (0.0007) [2023-10-07 23:21:52,967][67871] Updated weights for policy 1, policy_version 87200 (0.0008) [2023-10-07 23:21:53,040][67838] Updated weights for policy 0, policy_version 87072 (0.0007) [2023-10-07 23:21:57,089][67838] Updated weights for policy 0, policy_version 87082 (0.0007) [2023-10-07 23:21:57,094][67871] Updated weights for policy 1, policy_version 87210 (0.0009) [2023-10-07 23:21:57,455][67838] Updated weights for policy 0, policy_version 87092 (0.0008) [2023-10-07 23:21:57,465][67871] Updated weights for policy 1, policy_version 87220 (0.0008) [2023-10-07 23:21:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 178454528. Throughput: 0: 1646.7, 1: 1655.6. Samples: 44625386. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:21:57,477][66916] Avg episode reward: [(0, '41.900'), (1, '62.590')] [2023-10-07 23:21:57,827][67838] Updated weights for policy 0, policy_version 87102 (0.0009) [2023-10-07 23:21:57,831][67871] Updated weights for policy 1, policy_version 87230 (0.0007) [2023-10-07 23:22:01,813][67871] Updated weights for policy 1, policy_version 87240 (0.0007) [2023-10-07 23:22:01,879][67838] Updated weights for policy 0, policy_version 87112 (0.0008) [2023-10-07 23:22:02,172][67871] Updated weights for policy 1, policy_version 87250 (0.0007) [2023-10-07 23:22:02,246][67838] Updated weights for policy 0, policy_version 87122 (0.0008) [2023-10-07 23:22:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 178520064. Throughput: 0: 1649.4, 1: 1662.4. Samples: 44645940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:22:02,477][66916] Avg episode reward: [(0, '46.020'), (1, '62.030')] [2023-10-07 23:22:02,538][67871] Updated weights for policy 1, policy_version 87260 (0.0008) [2023-10-07 23:22:02,626][67838] Updated weights for policy 0, policy_version 87132 (0.0008) [2023-10-07 23:22:06,716][67871] Updated weights for policy 1, policy_version 87270 (0.0009) [2023-10-07 23:22:06,760][67838] Updated weights for policy 0, policy_version 87142 (0.0008) [2023-10-07 23:22:07,086][67871] Updated weights for policy 1, policy_version 87280 (0.0009) [2023-10-07 23:22:07,127][67838] Updated weights for policy 0, policy_version 87152 (0.0009) [2023-10-07 23:22:07,456][67871] Updated weights for policy 1, policy_version 87290 (0.0009) [2023-10-07 23:22:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 178585600. Throughput: 0: 1640.3, 1: 1655.9. Samples: 44665542. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:22:07,477][66916] Avg episode reward: [(0, '44.420'), (1, '64.140')] [2023-10-07 23:22:07,498][67838] Updated weights for policy 0, policy_version 87162 (0.0007) [2023-10-07 23:22:11,558][67871] Updated weights for policy 1, policy_version 87300 (0.0008) [2023-10-07 23:22:11,575][67838] Updated weights for policy 0, policy_version 87172 (0.0008) [2023-10-07 23:22:11,916][67871] Updated weights for policy 1, policy_version 87310 (0.0009) [2023-10-07 23:22:11,955][67838] Updated weights for policy 0, policy_version 87182 (0.0007) [2023-10-07 23:22:12,279][67871] Updated weights for policy 1, policy_version 87320 (0.0009) [2023-10-07 23:22:12,316][67838] Updated weights for policy 0, policy_version 87192 (0.0007) [2023-10-07 23:22:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 178651136. Throughput: 0: 1652.6, 1: 1666.3. Samples: 44675534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:22:12,477][66916] Avg episode reward: [(0, '43.190'), (1, '61.920')] [2023-10-07 23:22:16,467][67871] Updated weights for policy 1, policy_version 87330 (0.0008) [2023-10-07 23:22:16,629][67838] Updated weights for policy 0, policy_version 87202 (0.0008) [2023-10-07 23:22:16,873][67871] Updated weights for policy 1, policy_version 87340 (0.0009) [2023-10-07 23:22:17,000][67838] Updated weights for policy 0, policy_version 87212 (0.0009) [2023-10-07 23:22:17,238][67871] Updated weights for policy 1, policy_version 87350 (0.0007) [2023-10-07 23:22:17,366][67838] Updated weights for policy 0, policy_version 87222 (0.0008) [2023-10-07 23:22:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 178716672. Throughput: 0: 1653.8, 1: 1663.8. Samples: 44695834. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:22:17,477][66916] Avg episode reward: [(0, '45.950'), (1, '60.650')] [2023-10-07 23:22:17,604][67871] Updated weights for policy 1, policy_version 87360 (0.0007) [2023-10-07 23:22:17,725][67838] Updated weights for policy 0, policy_version 87232 (0.0009) [2023-10-07 23:22:21,670][67871] Updated weights for policy 1, policy_version 87370 (0.0008) [2023-10-07 23:22:21,992][67838] Updated weights for policy 0, policy_version 87242 (0.0007) [2023-10-07 23:22:22,038][67871] Updated weights for policy 1, policy_version 87380 (0.0007) [2023-10-07 23:22:22,371][67838] Updated weights for policy 0, policy_version 87252 (0.0008) [2023-10-07 23:22:22,403][67871] Updated weights for policy 1, policy_version 87390 (0.0007) [2023-10-07 23:22:22,480][66916] Fps is (10 sec: 16377.9, 60 sec: 13652.5, 300 sec: 13218.1). Total num frames: 178814976. Throughput: 0: 1646.7, 1: 1656.9. Samples: 44715218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:22:22,482][66916] Avg episode reward: [(0, '45.260'), (1, '64.190')] [2023-10-07 23:22:22,745][67838] Updated weights for policy 0, policy_version 87262 (0.0009) [2023-10-07 23:22:26,665][67871] Updated weights for policy 1, policy_version 87400 (0.0010) [2023-10-07 23:22:27,027][67871] Updated weights for policy 1, policy_version 87410 (0.0009) [2023-10-07 23:22:27,111][67838] Updated weights for policy 0, policy_version 87272 (0.0009) [2023-10-07 23:22:27,391][67871] Updated weights for policy 1, policy_version 87420 (0.0010) [2023-10-07 23:22:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 178847744. Throughput: 0: 1647.5, 1: 1668.7. Samples: 44724980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:22:27,477][66916] Avg episode reward: [(0, '48.180'), (1, '64.840')] [2023-10-07 23:22:27,479][67838] Updated weights for policy 0, policy_version 87282 (0.0009) [2023-10-07 23:22:27,851][67838] Updated weights for policy 0, policy_version 87292 (0.0009) [2023-10-07 23:22:31,593][67871] Updated weights for policy 1, policy_version 87430 (0.0009) [2023-10-07 23:22:31,968][67871] Updated weights for policy 1, policy_version 87440 (0.0010) [2023-10-07 23:22:32,203][67838] Updated weights for policy 0, policy_version 87302 (0.0009) [2023-10-07 23:22:32,325][67871] Updated weights for policy 1, policy_version 87450 (0.0008) [2023-10-07 23:22:32,476][66916] Fps is (10 sec: 9834.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 178913280. Throughput: 0: 1634.2, 1: 1654.6. Samples: 44743956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:22:32,477][66916] Avg episode reward: [(0, '48.730'), (1, '64.890')] [2023-10-07 23:22:32,577][67838] Updated weights for policy 0, policy_version 87312 (0.0009) [2023-10-07 23:22:32,940][67838] Updated weights for policy 0, policy_version 87322 (0.0011) [2023-10-07 23:22:36,515][67871] Updated weights for policy 1, policy_version 87460 (0.0011) [2023-10-07 23:22:36,873][67871] Updated weights for policy 1, policy_version 87470 (0.0009) [2023-10-07 23:22:37,243][67871] Updated weights for policy 1, policy_version 87480 (0.0010) [2023-10-07 23:22:37,424][67838] Updated weights for policy 0, policy_version 87332 (0.0008) [2023-10-07 23:22:37,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 178978816. Throughput: 0: 1630.5, 1: 1638.0. Samples: 44763196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:22:37,477][66916] Avg episode reward: [(0, '48.920'), (1, '64.520')] [2023-10-07 23:22:37,796][67838] Updated weights for policy 0, policy_version 87342 (0.0009) [2023-10-07 23:22:38,167][67838] Updated weights for policy 0, policy_version 87352 (0.0009) [2023-10-07 23:22:41,970][67871] Updated weights for policy 1, policy_version 87490 (0.0009) [2023-10-07 23:22:42,337][67871] Updated weights for policy 1, policy_version 87500 (0.0010) [2023-10-07 23:22:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 179044352. Throughput: 0: 1618.1, 1: 1635.5. Samples: 44771798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:22:42,478][66916] Avg episode reward: [(0, '48.230'), (1, '66.560')] [2023-10-07 23:22:42,700][67871] Updated weights for policy 1, policy_version 87510 (0.0009) [2023-10-07 23:22:42,829][67838] Updated weights for policy 0, policy_version 87362 (0.0010) [2023-10-07 23:22:43,070][67871] Updated weights for policy 1, policy_version 87520 (0.0008) [2023-10-07 23:22:43,189][67838] Updated weights for policy 0, policy_version 87372 (0.0008) [2023-10-07 23:22:43,559][67838] Updated weights for policy 0, policy_version 87382 (0.0009) [2023-10-07 23:22:43,931][67838] Updated weights for policy 0, policy_version 87392 (0.0010) [2023-10-07 23:22:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179109888. Throughput: 0: 1599.0, 1: 1613.5. Samples: 44790504. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 23:22:47,477][66916] Avg episode reward: [(0, '49.670'), (1, '68.510')] [2023-10-07 23:22:47,516][67871] Updated weights for policy 1, policy_version 87530 (0.0008) [2023-10-07 23:22:47,879][67871] Updated weights for policy 1, policy_version 87540 (0.0008) [2023-10-07 23:22:48,240][67871] Updated weights for policy 1, policy_version 87550 (0.0009) [2023-10-07 23:22:48,444][67838] Updated weights for policy 0, policy_version 87402 (0.0009) [2023-10-07 23:22:48,807][67838] Updated weights for policy 0, policy_version 87412 (0.0009) [2023-10-07 23:22:49,182][67838] Updated weights for policy 0, policy_version 87422 (0.0011) [2023-10-07 23:22:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179175424. Throughput: 0: 1592.4, 1: 1609.8. Samples: 44809638. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 23:22:52,477][66916] Avg episode reward: [(0, '47.870'), (1, '67.400')] [2023-10-07 23:22:52,610][67871] Updated weights for policy 1, policy_version 87560 (0.0009) [2023-10-07 23:22:52,978][67871] Updated weights for policy 1, policy_version 87570 (0.0008) [2023-10-07 23:22:53,335][67871] Updated weights for policy 1, policy_version 87580 (0.0008) [2023-10-07 23:22:53,690][67838] Updated weights for policy 0, policy_version 87432 (0.0009) [2023-10-07 23:22:54,051][67838] Updated weights for policy 0, policy_version 87442 (0.0010) [2023-10-07 23:22:54,420][67838] Updated weights for policy 0, policy_version 87452 (0.0010) [2023-10-07 23:22:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179240960. Throughput: 0: 1573.9, 1: 1596.6. Samples: 44818206. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 23:22:57,477][66916] Avg episode reward: [(0, '50.170'), (1, '67.550')] [2023-10-07 23:22:57,702][67871] Updated weights for policy 1, policy_version 87590 (0.0010) [2023-10-07 23:22:58,073][67871] Updated weights for policy 1, policy_version 87600 (0.0010) [2023-10-07 23:22:58,438][67871] Updated weights for policy 1, policy_version 87610 (0.0009) [2023-10-07 23:22:58,792][67838] Updated weights for policy 0, policy_version 87462 (0.0009) [2023-10-07 23:22:59,155][67838] Updated weights for policy 0, policy_version 87472 (0.0010) [2023-10-07 23:22:59,531][67838] Updated weights for policy 0, policy_version 87482 (0.0008) [2023-10-07 23:23:02,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 179306496. Throughput: 0: 1558.6, 1: 1580.5. Samples: 44837094. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 23:23:02,478][66916] Avg episode reward: [(0, '47.280'), (1, '69.650')] [2023-10-07 23:23:02,479][67676] Saving new best policy, reward=69.650! [2023-10-07 23:23:03,063][67871] Updated weights for policy 1, policy_version 87620 (0.0010) [2023-10-07 23:23:03,451][67871] Updated weights for policy 1, policy_version 87630 (0.0009) [2023-10-07 23:23:03,814][67871] Updated weights for policy 1, policy_version 87640 (0.0008) [2023-10-07 23:23:03,928][67838] Updated weights for policy 0, policy_version 87492 (0.0010) [2023-10-07 23:23:04,309][67838] Updated weights for policy 0, policy_version 87502 (0.0009) [2023-10-07 23:23:04,687][67838] Updated weights for policy 0, policy_version 87512 (0.0007) [2023-10-07 23:23:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179372032. Throughput: 0: 1565.7, 1: 1580.9. Samples: 44856806. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 23:23:07,477][66916] Avg episode reward: [(0, '45.280'), (1, '68.700')] [2023-10-07 23:23:07,485][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000087648_89751552.pth... [2023-10-07 23:23:07,485][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000087520_89620480.pth... [2023-10-07 23:23:07,514][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000086112_88178688.pth [2023-10-07 23:23:07,516][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000085984_88047616.pth [2023-10-07 23:23:07,804][67871] Updated weights for policy 1, policy_version 87650 (0.0009) [2023-10-07 23:23:08,170][67871] Updated weights for policy 1, policy_version 87660 (0.0009) [2023-10-07 23:23:08,535][67871] Updated weights for policy 1, policy_version 87670 (0.0007) [2023-10-07 23:23:08,675][67838] Updated weights for policy 0, policy_version 87522 (0.0007) [2023-10-07 23:23:08,899][67871] Updated weights for policy 1, policy_version 87680 (0.0009) [2023-10-07 23:23:09,056][67838] Updated weights for policy 0, policy_version 87532 (0.0009) [2023-10-07 23:23:09,417][67838] Updated weights for policy 0, policy_version 87542 (0.0010) [2023-10-07 23:23:09,786][67838] Updated weights for policy 0, policy_version 87552 (0.0008) [2023-10-07 23:23:12,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179437568. Throughput: 0: 1562.1, 1: 1569.4. Samples: 44865896. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 23:23:12,478][66916] Avg episode reward: [(0, '43.730'), (1, '65.360')] [2023-10-07 23:23:13,149][67871] Updated weights for policy 1, policy_version 87690 (0.0008) [2023-10-07 23:23:13,519][67871] Updated weights for policy 1, policy_version 87700 (0.0007) [2023-10-07 23:23:13,880][67871] Updated weights for policy 1, policy_version 87710 (0.0008) [2023-10-07 23:23:13,918][67838] Updated weights for policy 0, policy_version 87562 (0.0009) [2023-10-07 23:23:14,288][67838] Updated weights for policy 0, policy_version 87572 (0.0009) [2023-10-07 23:23:14,644][67838] Updated weights for policy 0, policy_version 87582 (0.0011) [2023-10-07 23:23:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 179503104. Throughput: 0: 1576.2, 1: 1584.4. Samples: 44886186. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 23:23:17,478][66916] Avg episode reward: [(0, '46.290'), (1, '65.940')] [2023-10-07 23:23:17,761][67871] Updated weights for policy 1, policy_version 87720 (0.0008) [2023-10-07 23:23:18,124][67871] Updated weights for policy 1, policy_version 87730 (0.0007) [2023-10-07 23:23:18,483][67871] Updated weights for policy 1, policy_version 87740 (0.0008) [2023-10-07 23:23:18,781][67838] Updated weights for policy 0, policy_version 87592 (0.0010) [2023-10-07 23:23:19,155][67838] Updated weights for policy 0, policy_version 87602 (0.0008) [2023-10-07 23:23:19,519][67838] Updated weights for policy 0, policy_version 87612 (0.0007) [2023-10-07 23:23:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 12561.8, 300 sec: 13107.2). Total num frames: 179568640. Throughput: 0: 1589.6, 1: 1608.2. Samples: 44907096. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 23:23:22,478][66916] Avg episode reward: [(0, '44.660'), (1, '58.910')] [2023-10-07 23:23:22,535][67871] Updated weights for policy 1, policy_version 87750 (0.0008) [2023-10-07 23:23:22,900][67871] Updated weights for policy 1, policy_version 87760 (0.0007) [2023-10-07 23:23:23,280][67871] Updated weights for policy 1, policy_version 87770 (0.0009) [2023-10-07 23:23:23,730][67838] Updated weights for policy 0, policy_version 87622 (0.0008) [2023-10-07 23:23:24,112][67838] Updated weights for policy 0, policy_version 87632 (0.0009) [2023-10-07 23:23:24,491][67838] Updated weights for policy 0, policy_version 87642 (0.0009) [2023-10-07 23:23:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179634176. Throughput: 0: 1597.4, 1: 1609.2. Samples: 44916096. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 23:23:27,478][66916] Avg episode reward: [(0, '47.350'), (1, '57.950')] [2023-10-07 23:23:27,518][67871] Updated weights for policy 1, policy_version 87780 (0.0008) [2023-10-07 23:23:27,878][67871] Updated weights for policy 1, policy_version 87790 (0.0010) [2023-10-07 23:23:28,243][67871] Updated weights for policy 1, policy_version 87800 (0.0009) [2023-10-07 23:23:28,447][67838] Updated weights for policy 0, policy_version 87652 (0.0007) [2023-10-07 23:23:28,845][67838] Updated weights for policy 0, policy_version 87662 (0.0008) [2023-10-07 23:23:29,214][67838] Updated weights for policy 0, policy_version 87672 (0.0009) [2023-10-07 23:23:32,307][67871] Updated weights for policy 1, policy_version 87810 (0.0008) [2023-10-07 23:23:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179699712. Throughput: 0: 1620.0, 1: 1627.5. Samples: 44936638. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 23:23:32,477][66916] Avg episode reward: [(0, '44.430'), (1, '58.600')] [2023-10-07 23:23:32,668][67871] Updated weights for policy 1, policy_version 87820 (0.0007) [2023-10-07 23:23:33,030][67871] Updated weights for policy 1, policy_version 87830 (0.0007) [2023-10-07 23:23:33,383][67838] Updated weights for policy 0, policy_version 87682 (0.0009) [2023-10-07 23:23:33,400][67871] Updated weights for policy 1, policy_version 87840 (0.0008) [2023-10-07 23:23:33,748][67838] Updated weights for policy 0, policy_version 87692 (0.0009) [2023-10-07 23:23:34,131][67838] Updated weights for policy 0, policy_version 87702 (0.0011) [2023-10-07 23:23:34,509][67838] Updated weights for policy 0, policy_version 87712 (0.0009) [2023-10-07 23:23:37,331][67871] Updated weights for policy 1, policy_version 87850 (0.0009) [2023-10-07 23:23:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179765248. Throughput: 0: 1638.0, 1: 1644.2. Samples: 44957340. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-07 23:23:37,477][66916] Avg episode reward: [(0, '42.090'), (1, '57.480')] [2023-10-07 23:23:37,688][67871] Updated weights for policy 1, policy_version 87860 (0.0007) [2023-10-07 23:23:38,061][67871] Updated weights for policy 1, policy_version 87870 (0.0008) [2023-10-07 23:23:38,624][67838] Updated weights for policy 0, policy_version 87722 (0.0008) [2023-10-07 23:23:38,995][67838] Updated weights for policy 0, policy_version 87732 (0.0007) [2023-10-07 23:23:39,359][67838] Updated weights for policy 0, policy_version 87742 (0.0009) [2023-10-07 23:23:42,054][67871] Updated weights for policy 1, policy_version 87880 (0.0010) [2023-10-07 23:23:42,417][67871] Updated weights for policy 1, policy_version 87890 (0.0009) [2023-10-07 23:23:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179830784. Throughput: 0: 1641.8, 1: 1649.3. Samples: 44966308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:23:42,477][66916] Avg episode reward: [(0, '45.220'), (1, '59.580')] [2023-10-07 23:23:42,794][67871] Updated weights for policy 1, policy_version 87900 (0.0007) [2023-10-07 23:23:43,506][67838] Updated weights for policy 0, policy_version 87752 (0.0008) [2023-10-07 23:23:43,885][67838] Updated weights for policy 0, policy_version 87762 (0.0007) [2023-10-07 23:23:44,258][67838] Updated weights for policy 0, policy_version 87772 (0.0007) [2023-10-07 23:23:46,859][67871] Updated weights for policy 1, policy_version 87910 (0.0008) [2023-10-07 23:23:47,231][67871] Updated weights for policy 1, policy_version 87920 (0.0008) [2023-10-07 23:23:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179896320. Throughput: 0: 1658.9, 1: 1667.7. Samples: 44986788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:23:47,477][66916] Avg episode reward: [(0, '41.090'), (1, '62.860')] [2023-10-07 23:23:47,596][67871] Updated weights for policy 1, policy_version 87930 (0.0009) [2023-10-07 23:23:48,491][67838] Updated weights for policy 0, policy_version 87782 (0.0009) [2023-10-07 23:23:48,863][67838] Updated weights for policy 0, policy_version 87792 (0.0009) [2023-10-07 23:23:49,238][67838] Updated weights for policy 0, policy_version 87802 (0.0008) [2023-10-07 23:23:51,840][67871] Updated weights for policy 1, policy_version 87940 (0.0009) [2023-10-07 23:23:52,230][67871] Updated weights for policy 1, policy_version 87950 (0.0012) [2023-10-07 23:23:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179961856. Throughput: 0: 1662.9, 1: 1674.1. Samples: 45006972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:23:52,477][66916] Avg episode reward: [(0, '39.100'), (1, '65.810')] [2023-10-07 23:23:52,603][67871] Updated weights for policy 1, policy_version 87960 (0.0008) [2023-10-07 23:23:53,176][67838] Updated weights for policy 0, policy_version 87812 (0.0008) [2023-10-07 23:23:53,541][67838] Updated weights for policy 0, policy_version 87822 (0.0007) [2023-10-07 23:23:53,914][67838] Updated weights for policy 0, policy_version 87832 (0.0007) [2023-10-07 23:23:56,804][67871] Updated weights for policy 1, policy_version 87970 (0.0007) [2023-10-07 23:23:57,163][67871] Updated weights for policy 1, policy_version 87980 (0.0010) [2023-10-07 23:23:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180027392. Throughput: 0: 1661.9, 1: 1680.8. Samples: 45016314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:23:57,477][66916] Avg episode reward: [(0, '45.170'), (1, '67.250')] [2023-10-07 23:23:57,528][67871] Updated weights for policy 1, policy_version 87990 (0.0011) [2023-10-07 23:23:57,897][67871] Updated weights for policy 1, policy_version 88000 (0.0011) [2023-10-07 23:23:57,946][67838] Updated weights for policy 0, policy_version 87842 (0.0008) [2023-10-07 23:23:58,317][67838] Updated weights for policy 0, policy_version 87852 (0.0009) [2023-10-07 23:23:58,687][67838] Updated weights for policy 0, policy_version 87862 (0.0007) [2023-10-07 23:23:59,060][67838] Updated weights for policy 0, policy_version 87872 (0.0008) [2023-10-07 23:24:02,177][67871] Updated weights for policy 1, policy_version 88010 (0.0008) [2023-10-07 23:24:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180092928. Throughput: 0: 1668.5, 1: 1679.5. Samples: 45036844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:24:02,478][66916] Avg episode reward: [(0, '42.370'), (1, '68.160')] [2023-10-07 23:24:02,552][67871] Updated weights for policy 1, policy_version 88020 (0.0009) [2023-10-07 23:24:02,914][67871] Updated weights for policy 1, policy_version 88030 (0.0008) [2023-10-07 23:24:03,149][67838] Updated weights for policy 0, policy_version 87882 (0.0008) [2023-10-07 23:24:03,511][67838] Updated weights for policy 0, policy_version 87892 (0.0011) [2023-10-07 23:24:03,889][67838] Updated weights for policy 0, policy_version 87902 (0.0009) [2023-10-07 23:24:07,216][67871] Updated weights for policy 1, policy_version 88040 (0.0008) [2023-10-07 23:24:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180158464. Throughput: 0: 1672.7, 1: 1668.6. Samples: 45057454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:24:07,478][66916] Avg episode reward: [(0, '48.350'), (1, '65.510')] [2023-10-07 23:24:07,585][67871] Updated weights for policy 1, policy_version 88050 (0.0008) [2023-10-07 23:24:07,810][67838] Updated weights for policy 0, policy_version 87912 (0.0009) [2023-10-07 23:24:07,956][67871] Updated weights for policy 1, policy_version 88060 (0.0007) [2023-10-07 23:24:08,178][67838] Updated weights for policy 0, policy_version 87922 (0.0009) [2023-10-07 23:24:08,544][67838] Updated weights for policy 0, policy_version 87932 (0.0009) [2023-10-07 23:24:12,123][67871] Updated weights for policy 1, policy_version 88070 (0.0007) [2023-10-07 23:24:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180224000. Throughput: 0: 1672.0, 1: 1664.5. Samples: 45066236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:24:12,477][66916] Avg episode reward: [(0, '49.790'), (1, '66.450')] [2023-10-07 23:24:12,492][67871] Updated weights for policy 1, policy_version 88080 (0.0007) [2023-10-07 23:24:12,781][67838] Updated weights for policy 0, policy_version 87942 (0.0010) [2023-10-07 23:24:12,860][67871] Updated weights for policy 1, policy_version 88090 (0.0007) [2023-10-07 23:24:13,154][67838] Updated weights for policy 0, policy_version 87952 (0.0009) [2023-10-07 23:24:13,539][67838] Updated weights for policy 0, policy_version 87962 (0.0007) [2023-10-07 23:24:16,886][67871] Updated weights for policy 1, policy_version 88100 (0.0007) [2023-10-07 23:24:17,251][67871] Updated weights for policy 1, policy_version 88110 (0.0007) [2023-10-07 23:24:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 180289536. Throughput: 0: 1669.3, 1: 1663.9. Samples: 45086630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:24:17,477][66916] Avg episode reward: [(0, '49.310'), (1, '66.990')] [2023-10-07 23:24:17,620][67871] Updated weights for policy 1, policy_version 88120 (0.0010) [2023-10-07 23:24:17,851][67838] Updated weights for policy 0, policy_version 87972 (0.0007) [2023-10-07 23:24:18,240][67838] Updated weights for policy 0, policy_version 87982 (0.0010) [2023-10-07 23:24:18,612][67838] Updated weights for policy 0, policy_version 87992 (0.0009) [2023-10-07 23:24:21,867][67871] Updated weights for policy 1, policy_version 88130 (0.0009) [2023-10-07 23:24:22,232][67871] Updated weights for policy 1, policy_version 88140 (0.0008) [2023-10-07 23:24:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 180355072. Throughput: 0: 1668.1, 1: 1654.8. Samples: 45106868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:24:22,477][66916] Avg episode reward: [(0, '47.540'), (1, '66.940')] [2023-10-07 23:24:22,595][67871] Updated weights for policy 1, policy_version 88150 (0.0008) [2023-10-07 23:24:22,694][67838] Updated weights for policy 0, policy_version 88002 (0.0008) [2023-10-07 23:24:22,958][67871] Updated weights for policy 1, policy_version 88160 (0.0008) [2023-10-07 23:24:23,067][67838] Updated weights for policy 0, policy_version 88012 (0.0008) [2023-10-07 23:24:23,429][67838] Updated weights for policy 0, policy_version 88022 (0.0010) [2023-10-07 23:24:23,795][67838] Updated weights for policy 0, policy_version 88032 (0.0008) [2023-10-07 23:24:26,975][67871] Updated weights for policy 1, policy_version 88170 (0.0007) [2023-10-07 23:24:27,346][67871] Updated weights for policy 1, policy_version 88180 (0.0007) [2023-10-07 23:24:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180420608. Throughput: 0: 1666.5, 1: 1657.2. Samples: 45115872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:24:27,478][66916] Avg episode reward: [(0, '41.620'), (1, '67.570')] [2023-10-07 23:24:27,718][67871] Updated weights for policy 1, policy_version 88190 (0.0007) [2023-10-07 23:24:27,912][67838] Updated weights for policy 0, policy_version 88042 (0.0008) [2023-10-07 23:24:28,279][67838] Updated weights for policy 0, policy_version 88052 (0.0008) [2023-10-07 23:24:28,663][67838] Updated weights for policy 0, policy_version 88062 (0.0008) [2023-10-07 23:24:31,757][67871] Updated weights for policy 1, policy_version 88200 (0.0008) [2023-10-07 23:24:32,117][67871] Updated weights for policy 1, policy_version 88210 (0.0009) [2023-10-07 23:24:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180486144. Throughput: 0: 1669.3, 1: 1654.8. Samples: 45136374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:24:32,477][66916] Avg episode reward: [(0, '41.340'), (1, '61.030')] [2023-10-07 23:24:32,495][67871] Updated weights for policy 1, policy_version 88220 (0.0007) [2023-10-07 23:24:32,794][67838] Updated weights for policy 0, policy_version 88072 (0.0009) [2023-10-07 23:24:33,171][67838] Updated weights for policy 0, policy_version 88082 (0.0011) [2023-10-07 23:24:33,554][67838] Updated weights for policy 0, policy_version 88092 (0.0009) [2023-10-07 23:24:36,612][67871] Updated weights for policy 1, policy_version 88230 (0.0007) [2023-10-07 23:24:36,977][67871] Updated weights for policy 1, policy_version 88240 (0.0008) [2023-10-07 23:24:37,347][67871] Updated weights for policy 1, policy_version 88250 (0.0009) [2023-10-07 23:24:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180551680. Throughput: 0: 1669.4, 1: 1650.6. Samples: 45156374. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-07 23:24:37,477][66916] Avg episode reward: [(0, '41.680'), (1, '61.610')] [2023-10-07 23:24:37,716][67838] Updated weights for policy 0, policy_version 88102 (0.0008) [2023-10-07 23:24:38,092][67838] Updated weights for policy 0, policy_version 88112 (0.0009) [2023-10-07 23:24:38,454][67838] Updated weights for policy 0, policy_version 88122 (0.0008) [2023-10-07 23:24:41,257][67871] Updated weights for policy 1, policy_version 88260 (0.0007) [2023-10-07 23:24:41,624][67871] Updated weights for policy 1, policy_version 88270 (0.0008) [2023-10-07 23:24:41,984][67871] Updated weights for policy 1, policy_version 88280 (0.0008) [2023-10-07 23:24:42,477][66916] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 180649984. Throughput: 0: 1667.1, 1: 1657.9. Samples: 45165936. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-07 23:24:42,478][66916] Avg episode reward: [(0, '43.140'), (1, '64.300')] [2023-10-07 23:24:42,584][67838] Updated weights for policy 0, policy_version 88132 (0.0007) [2023-10-07 23:24:42,964][67838] Updated weights for policy 0, policy_version 88142 (0.0010) [2023-10-07 23:24:43,339][67838] Updated weights for policy 0, policy_version 88152 (0.0009) [2023-10-07 23:24:46,200][67871] Updated weights for policy 1, policy_version 88290 (0.0009) [2023-10-07 23:24:46,575][67871] Updated weights for policy 1, policy_version 88300 (0.0008) [2023-10-07 23:24:46,936][67871] Updated weights for policy 1, policy_version 88310 (0.0007) [2023-10-07 23:24:47,297][67871] Updated weights for policy 1, policy_version 88320 (0.0008) [2023-10-07 23:24:47,425][67838] Updated weights for policy 0, policy_version 88162 (0.0010) [2023-10-07 23:24:47,476][66916] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 180715520. Throughput: 0: 1663.9, 1: 1658.9. Samples: 45186372. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-07 23:24:47,477][66916] Avg episode reward: [(0, '46.940'), (1, '58.540')] [2023-10-07 23:24:47,796][67838] Updated weights for policy 0, policy_version 88172 (0.0010) [2023-10-07 23:24:48,169][67838] Updated weights for policy 0, policy_version 88182 (0.0008) [2023-10-07 23:24:48,538][67838] Updated weights for policy 0, policy_version 88192 (0.0008) [2023-10-07 23:24:51,599][67871] Updated weights for policy 1, policy_version 88330 (0.0007) [2023-10-07 23:24:51,962][67871] Updated weights for policy 1, policy_version 88340 (0.0007) [2023-10-07 23:24:52,323][67871] Updated weights for policy 1, policy_version 88350 (0.0008) [2023-10-07 23:24:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 180781056. Throughput: 0: 1654.0, 1: 1645.5. Samples: 45205934. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-07 23:24:52,477][66916] Avg episode reward: [(0, '40.090'), (1, '57.760')] [2023-10-07 23:24:52,717][67838] Updated weights for policy 0, policy_version 88202 (0.0009) [2023-10-07 23:24:53,086][67838] Updated weights for policy 0, policy_version 88212 (0.0010) [2023-10-07 23:24:53,453][67838] Updated weights for policy 0, policy_version 88222 (0.0010) [2023-10-07 23:24:56,198][67871] Updated weights for policy 1, policy_version 88360 (0.0008) [2023-10-07 23:24:56,566][67871] Updated weights for policy 1, policy_version 88370 (0.0009) [2023-10-07 23:24:56,934][67871] Updated weights for policy 1, policy_version 88380 (0.0010) [2023-10-07 23:24:57,418][67838] Updated weights for policy 0, policy_version 88232 (0.0010) [2023-10-07 23:24:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 180846592. Throughput: 0: 1653.6, 1: 1667.6. Samples: 45215692. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-07 23:24:57,477][66916] Avg episode reward: [(0, '43.290'), (1, '61.860')] [2023-10-07 23:24:57,787][67838] Updated weights for policy 0, policy_version 88242 (0.0009) [2023-10-07 23:24:58,165][67838] Updated weights for policy 0, policy_version 88252 (0.0010) [2023-10-07 23:25:01,090][67871] Updated weights for policy 1, policy_version 88390 (0.0010) [2023-10-07 23:25:01,460][67871] Updated weights for policy 1, policy_version 88400 (0.0009) [2023-10-07 23:25:01,832][67871] Updated weights for policy 1, policy_version 88410 (0.0007) [2023-10-07 23:25:02,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 180912128. Throughput: 0: 1653.5, 1: 1667.5. Samples: 45236072. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-07 23:25:02,478][66916] Avg episode reward: [(0, '41.870'), (1, '60.150')] [2023-10-07 23:25:02,480][67838] Updated weights for policy 0, policy_version 88262 (0.0009) [2023-10-07 23:25:02,870][67838] Updated weights for policy 0, policy_version 88272 (0.0009) [2023-10-07 23:25:03,233][67838] Updated weights for policy 0, policy_version 88282 (0.0008) [2023-10-07 23:25:05,938][67871] Updated weights for policy 1, policy_version 88420 (0.0008) [2023-10-07 23:25:06,310][67871] Updated weights for policy 1, policy_version 88430 (0.0008) [2023-10-07 23:25:06,665][67871] Updated weights for policy 1, policy_version 88440 (0.0007) [2023-10-07 23:25:07,149][67838] Updated weights for policy 0, policy_version 88292 (0.0007) [2023-10-07 23:25:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 180977664. Throughput: 0: 1660.7, 1: 1646.6. Samples: 45255696. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-07 23:25:07,478][66916] Avg episode reward: [(0, '40.360'), (1, '59.970')] [2023-10-07 23:25:07,491][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000088448_90570752.pth... [2023-10-07 23:25:07,522][67838] Updated weights for policy 0, policy_version 88302 (0.0007) [2023-10-07 23:25:07,526][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000086912_88997888.pth [2023-10-07 23:25:07,890][67838] Updated weights for policy 0, policy_version 88312 (0.0008) [2023-10-07 23:25:08,177][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000088320_90439680.pth... [2023-10-07 23:25:08,219][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000086784_88866816.pth [2023-10-07 23:25:10,849][67871] Updated weights for policy 1, policy_version 88450 (0.0009) [2023-10-07 23:25:11,217][67871] Updated weights for policy 1, policy_version 88460 (0.0009) [2023-10-07 23:25:11,581][67871] Updated weights for policy 1, policy_version 88470 (0.0010) [2023-10-07 23:25:11,951][67871] Updated weights for policy 1, policy_version 88480 (0.0008) [2023-10-07 23:25:11,992][67838] Updated weights for policy 0, policy_version 88322 (0.0008) [2023-10-07 23:25:12,358][67838] Updated weights for policy 0, policy_version 88332 (0.0007) [2023-10-07 23:25:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 181043200. Throughput: 0: 1663.9, 1: 1672.8. Samples: 45266022. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-07 23:25:12,477][66916] Avg episode reward: [(0, '45.200'), (1, '61.160')] [2023-10-07 23:25:12,722][67838] Updated weights for policy 0, policy_version 88342 (0.0007) [2023-10-07 23:25:13,093][67838] Updated weights for policy 0, policy_version 88352 (0.0009) [2023-10-07 23:25:15,955][67871] Updated weights for policy 1, policy_version 88490 (0.0008) [2023-10-07 23:25:16,320][67871] Updated weights for policy 1, policy_version 88500 (0.0007) [2023-10-07 23:25:16,687][67871] Updated weights for policy 1, policy_version 88510 (0.0007) [2023-10-07 23:25:17,169][67838] Updated weights for policy 0, policy_version 88362 (0.0010) [2023-10-07 23:25:17,477][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 181108736. Throughput: 0: 1662.5, 1: 1666.8. Samples: 45286192. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-07 23:25:17,478][66916] Avg episode reward: [(0, '42.500'), (1, '58.230')] [2023-10-07 23:25:17,537][67838] Updated weights for policy 0, policy_version 88372 (0.0007) [2023-10-07 23:25:17,912][67838] Updated weights for policy 0, policy_version 88382 (0.0007) [2023-10-07 23:25:20,991][67871] Updated weights for policy 1, policy_version 88520 (0.0008) [2023-10-07 23:25:21,350][67871] Updated weights for policy 1, policy_version 88530 (0.0009) [2023-10-07 23:25:21,718][67871] Updated weights for policy 1, policy_version 88540 (0.0009) [2023-10-07 23:25:21,829][67838] Updated weights for policy 0, policy_version 88392 (0.0010) [2023-10-07 23:25:22,199][67838] Updated weights for policy 0, policy_version 88402 (0.0010) [2023-10-07 23:25:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 181174272. Throughput: 0: 1654.7, 1: 1655.9. Samples: 45305350. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-07 23:25:22,477][66916] Avg episode reward: [(0, '45.960'), (1, '60.710')] [2023-10-07 23:25:22,579][67838] Updated weights for policy 0, policy_version 88412 (0.0009) [2023-10-07 23:25:25,823][67871] Updated weights for policy 1, policy_version 88550 (0.0007) [2023-10-07 23:25:26,194][67871] Updated weights for policy 1, policy_version 88560 (0.0008) [2023-10-07 23:25:26,565][67871] Updated weights for policy 1, policy_version 88570 (0.0009) [2023-10-07 23:25:26,756][67838] Updated weights for policy 0, policy_version 88422 (0.0008) [2023-10-07 23:25:27,124][67838] Updated weights for policy 0, policy_version 88432 (0.0009) [2023-10-07 23:25:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 181239808. Throughput: 0: 1667.6, 1: 1668.0. Samples: 45316038. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-07 23:25:27,478][66916] Avg episode reward: [(0, '45.790'), (1, '60.090')] [2023-10-07 23:25:27,509][67838] Updated weights for policy 0, policy_version 88442 (0.0010) [2023-10-07 23:25:30,788][67871] Updated weights for policy 1, policy_version 88580 (0.0008) [2023-10-07 23:25:31,159][67871] Updated weights for policy 1, policy_version 88590 (0.0007) [2023-10-07 23:25:31,436][67838] Updated weights for policy 0, policy_version 88452 (0.0007) [2023-10-07 23:25:31,520][67871] Updated weights for policy 1, policy_version 88600 (0.0007) [2023-10-07 23:25:31,803][67838] Updated weights for policy 0, policy_version 88462 (0.0008) [2023-10-07 23:25:32,173][67838] Updated weights for policy 0, policy_version 88472 (0.0009) [2023-10-07 23:25:32,476][66916] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 181338112. Throughput: 0: 1668.3, 1: 1660.8. Samples: 45336182. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-07 23:25:32,477][66916] Avg episode reward: [(0, '45.130'), (1, '61.820')] [2023-10-07 23:25:35,491][67871] Updated weights for policy 1, policy_version 88610 (0.0009) [2023-10-07 23:25:35,860][67871] Updated weights for policy 1, policy_version 88620 (0.0009) [2023-10-07 23:25:36,226][67871] Updated weights for policy 1, policy_version 88630 (0.0010) [2023-10-07 23:25:36,237][67838] Updated weights for policy 0, policy_version 88482 (0.0008) [2023-10-07 23:25:36,596][67871] Updated weights for policy 1, policy_version 88640 (0.0007) [2023-10-07 23:25:36,612][67838] Updated weights for policy 0, policy_version 88492 (0.0010) [2023-10-07 23:25:36,987][67838] Updated weights for policy 0, policy_version 88502 (0.0009) [2023-10-07 23:25:37,358][67838] Updated weights for policy 0, policy_version 88512 (0.0009) [2023-10-07 23:25:37,477][66916] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13329.3). Total num frames: 181403648. Throughput: 0: 1653.6, 1: 1659.2. Samples: 45355012. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-07 23:25:37,478][66916] Avg episode reward: [(0, '44.400'), (1, '60.340')] [2023-10-07 23:25:40,542][67871] Updated weights for policy 1, policy_version 88650 (0.0010) [2023-10-07 23:25:40,916][67871] Updated weights for policy 1, policy_version 88660 (0.0008) [2023-10-07 23:25:41,289][67871] Updated weights for policy 1, policy_version 88670 (0.0010) [2023-10-07 23:25:41,658][67838] Updated weights for policy 0, policy_version 88522 (0.0010) [2023-10-07 23:25:42,028][67838] Updated weights for policy 0, policy_version 88532 (0.0009) [2023-10-07 23:25:42,403][67838] Updated weights for policy 0, policy_version 88542 (0.0007) [2023-10-07 23:25:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 181469184. Throughput: 0: 1671.2, 1: 1673.2. Samples: 45366194. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-07 23:25:42,477][66916] Avg episode reward: [(0, '46.210'), (1, '61.070')] [2023-10-07 23:25:45,548][67871] Updated weights for policy 1, policy_version 88680 (0.0010) [2023-10-07 23:25:45,920][67871] Updated weights for policy 1, policy_version 88690 (0.0009) [2023-10-07 23:25:46,302][67871] Updated weights for policy 1, policy_version 88700 (0.0009) [2023-10-07 23:25:46,498][67838] Updated weights for policy 0, policy_version 88552 (0.0008) [2023-10-07 23:25:46,883][67838] Updated weights for policy 0, policy_version 88562 (0.0007) [2023-10-07 23:25:47,265][67838] Updated weights for policy 0, policy_version 88572 (0.0008) [2023-10-07 23:25:47,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 181534720. Throughput: 0: 1671.7, 1: 1659.5. Samples: 45385974. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-07 23:25:47,477][66916] Avg episode reward: [(0, '44.810'), (1, '67.550')] [2023-10-07 23:25:50,386][67871] Updated weights for policy 1, policy_version 88710 (0.0008) [2023-10-07 23:25:50,750][67871] Updated weights for policy 1, policy_version 88720 (0.0009) [2023-10-07 23:25:51,123][67871] Updated weights for policy 1, policy_version 88730 (0.0009) [2023-10-07 23:25:51,514][67838] Updated weights for policy 0, policy_version 88582 (0.0007) [2023-10-07 23:25:51,899][67838] Updated weights for policy 0, policy_version 88592 (0.0009) [2023-10-07 23:25:52,265][67838] Updated weights for policy 0, policy_version 88602 (0.0007) [2023-10-07 23:25:52,477][66916] Fps is (10 sec: 9830.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 181567488. Throughput: 0: 1645.0, 1: 1669.2. Samples: 45404834. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-07 23:25:52,478][66916] Avg episode reward: [(0, '46.540'), (1, '68.210')] [2023-10-07 23:25:55,305][67871] Updated weights for policy 1, policy_version 88740 (0.0010) [2023-10-07 23:25:55,668][67871] Updated weights for policy 1, policy_version 88750 (0.0010) [2023-10-07 23:25:56,038][67871] Updated weights for policy 1, policy_version 88760 (0.0010) [2023-10-07 23:25:56,371][67838] Updated weights for policy 0, policy_version 88612 (0.0009) [2023-10-07 23:25:56,740][67838] Updated weights for policy 0, policy_version 88622 (0.0010) [2023-10-07 23:25:57,115][67838] Updated weights for policy 0, policy_version 88632 (0.0008) [2023-10-07 23:25:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 181665792. Throughput: 0: 1661.7, 1: 1667.9. Samples: 45415852. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-07 23:25:57,478][66916] Avg episode reward: [(0, '45.630'), (1, '66.230')] [2023-10-07 23:26:00,096][67871] Updated weights for policy 1, policy_version 88770 (0.0009) [2023-10-07 23:26:00,456][67871] Updated weights for policy 1, policy_version 88780 (0.0009) [2023-10-07 23:26:00,823][67871] Updated weights for policy 1, policy_version 88790 (0.0007) [2023-10-07 23:26:01,184][67871] Updated weights for policy 1, policy_version 88800 (0.0008) [2023-10-07 23:26:01,311][67838] Updated weights for policy 0, policy_version 88642 (0.0008) [2023-10-07 23:26:01,677][67838] Updated weights for policy 0, policy_version 88652 (0.0008) [2023-10-07 23:26:02,038][67838] Updated weights for policy 0, policy_version 88662 (0.0009) [2023-10-07 23:26:02,409][67838] Updated weights for policy 0, policy_version 88672 (0.0010) [2023-10-07 23:26:02,476][66916] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 181731328. Throughput: 0: 1661.6, 1: 1653.7. Samples: 45435380. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-07 23:26:02,477][66916] Avg episode reward: [(0, '48.750'), (1, '65.020')] [2023-10-07 23:26:05,231][67871] Updated weights for policy 1, policy_version 88810 (0.0008) [2023-10-07 23:26:05,595][67871] Updated weights for policy 1, policy_version 88820 (0.0010) [2023-10-07 23:26:05,963][67871] Updated weights for policy 1, policy_version 88830 (0.0007) [2023-10-07 23:26:06,583][67838] Updated weights for policy 0, policy_version 88682 (0.0009) [2023-10-07 23:26:06,952][67838] Updated weights for policy 0, policy_version 88692 (0.0010) [2023-10-07 23:26:07,322][67838] Updated weights for policy 0, policy_version 88702 (0.0007) [2023-10-07 23:26:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 181796864. Throughput: 0: 1647.0, 1: 1665.4. Samples: 45454408. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-07 23:26:07,477][66916] Avg episode reward: [(0, '46.720'), (1, '66.290')] [2023-10-07 23:26:10,083][67871] Updated weights for policy 1, policy_version 88840 (0.0008) [2023-10-07 23:26:10,455][67871] Updated weights for policy 1, policy_version 88850 (0.0011) [2023-10-07 23:26:10,823][67871] Updated weights for policy 1, policy_version 88860 (0.0009) [2023-10-07 23:26:11,449][67838] Updated weights for policy 0, policy_version 88712 (0.0008) [2023-10-07 23:26:11,816][67838] Updated weights for policy 0, policy_version 88722 (0.0007) [2023-10-07 23:26:12,199][67838] Updated weights for policy 0, policy_version 88732 (0.0007) [2023-10-07 23:26:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 181862400. Throughput: 0: 1655.1, 1: 1664.9. Samples: 45465434. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-07 23:26:12,477][66916] Avg episode reward: [(0, '48.280'), (1, '62.230')] [2023-10-07 23:26:15,049][67871] Updated weights for policy 1, policy_version 88870 (0.0009) [2023-10-07 23:26:15,424][67871] Updated weights for policy 1, policy_version 88880 (0.0009) [2023-10-07 23:26:15,788][67871] Updated weights for policy 1, policy_version 88890 (0.0010) [2023-10-07 23:26:16,453][67838] Updated weights for policy 0, policy_version 88742 (0.0008) [2023-10-07 23:26:16,814][67838] Updated weights for policy 0, policy_version 88752 (0.0010) [2023-10-07 23:26:17,193][67838] Updated weights for policy 0, policy_version 88762 (0.0008) [2023-10-07 23:26:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 181927936. Throughput: 0: 1651.0, 1: 1648.0. Samples: 45484636. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-07 23:26:17,478][66916] Avg episode reward: [(0, '46.670'), (1, '61.050')] [2023-10-07 23:26:19,976][67871] Updated weights for policy 1, policy_version 88900 (0.0009) [2023-10-07 23:26:20,334][67871] Updated weights for policy 1, policy_version 88910 (0.0008) [2023-10-07 23:26:20,708][67871] Updated weights for policy 1, policy_version 88920 (0.0007) [2023-10-07 23:26:21,174][67838] Updated weights for policy 0, policy_version 88772 (0.0009) [2023-10-07 23:26:21,540][67838] Updated weights for policy 0, policy_version 88782 (0.0010) [2023-10-07 23:26:21,911][67838] Updated weights for policy 0, policy_version 88792 (0.0008) [2023-10-07 23:26:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 181993472. Throughput: 0: 1646.8, 1: 1663.6. Samples: 45503980. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-07 23:26:22,477][66916] Avg episode reward: [(0, '46.100'), (1, '60.720')] [2023-10-07 23:26:24,894][67871] Updated weights for policy 1, policy_version 88930 (0.0007) [2023-10-07 23:26:25,260][67871] Updated weights for policy 1, policy_version 88940 (0.0007) [2023-10-07 23:26:25,637][67871] Updated weights for policy 1, policy_version 88950 (0.0009) [2023-10-07 23:26:26,001][67871] Updated weights for policy 1, policy_version 88960 (0.0009) [2023-10-07 23:26:26,020][67838] Updated weights for policy 0, policy_version 88802 (0.0009) [2023-10-07 23:26:26,390][67838] Updated weights for policy 0, policy_version 88812 (0.0007) [2023-10-07 23:26:26,763][67838] Updated weights for policy 0, policy_version 88822 (0.0008) [2023-10-07 23:26:27,129][67838] Updated weights for policy 0, policy_version 88832 (0.0008) [2023-10-07 23:26:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 182059008. Throughput: 0: 1655.4, 1: 1655.6. Samples: 45515188. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-07 23:26:27,478][66916] Avg episode reward: [(0, '44.010'), (1, '60.610')] [2023-10-07 23:26:30,254][67871] Updated weights for policy 1, policy_version 88970 (0.0008) [2023-10-07 23:26:30,612][67871] Updated weights for policy 1, policy_version 88980 (0.0012) [2023-10-07 23:26:30,978][67871] Updated weights for policy 1, policy_version 88990 (0.0007) [2023-10-07 23:26:31,128][67838] Updated weights for policy 0, policy_version 88842 (0.0009) [2023-10-07 23:26:31,504][67838] Updated weights for policy 0, policy_version 88852 (0.0009) [2023-10-07 23:26:31,879][67838] Updated weights for policy 0, policy_version 88862 (0.0010) [2023-10-07 23:26:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182124544. Throughput: 0: 1650.0, 1: 1646.9. Samples: 45534332. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) [2023-10-07 23:26:32,477][66916] Avg episode reward: [(0, '43.380'), (1, '61.470')] [2023-10-07 23:26:35,009][67871] Updated weights for policy 1, policy_version 89000 (0.0007) [2023-10-07 23:26:35,370][67871] Updated weights for policy 1, policy_version 89010 (0.0008) [2023-10-07 23:26:35,741][67871] Updated weights for policy 1, policy_version 89020 (0.0009) [2023-10-07 23:26:36,126][67838] Updated weights for policy 0, policy_version 88872 (0.0009) [2023-10-07 23:26:36,490][67838] Updated weights for policy 0, policy_version 88882 (0.0007) [2023-10-07 23:26:36,871][67838] Updated weights for policy 0, policy_version 88892 (0.0007) [2023-10-07 23:26:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 182190080. Throughput: 0: 1646.8, 1: 1662.7. Samples: 45553762. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) [2023-10-07 23:26:37,477][66916] Avg episode reward: [(0, '39.600'), (1, '61.520')] [2023-10-07 23:26:39,825][67871] Updated weights for policy 1, policy_version 89030 (0.0009) [2023-10-07 23:26:40,202][67871] Updated weights for policy 1, policy_version 89040 (0.0009) [2023-10-07 23:26:40,573][67871] Updated weights for policy 1, policy_version 89050 (0.0011) [2023-10-07 23:26:40,819][67838] Updated weights for policy 0, policy_version 88902 (0.0008) [2023-10-07 23:26:41,203][67838] Updated weights for policy 0, policy_version 88912 (0.0009) [2023-10-07 23:26:41,577][67838] Updated weights for policy 0, policy_version 88922 (0.0010) [2023-10-07 23:26:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182255616. Throughput: 0: 1658.0, 1: 1654.4. Samples: 45564910. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) [2023-10-07 23:26:42,477][66916] Avg episode reward: [(0, '39.920'), (1, '64.980')] [2023-10-07 23:26:44,711][67871] Updated weights for policy 1, policy_version 89060 (0.0009) [2023-10-07 23:26:45,078][67871] Updated weights for policy 1, policy_version 89070 (0.0008) [2023-10-07 23:26:45,441][67871] Updated weights for policy 1, policy_version 89080 (0.0008) [2023-10-07 23:26:45,746][67838] Updated weights for policy 0, policy_version 88932 (0.0007) [2023-10-07 23:26:46,111][67838] Updated weights for policy 0, policy_version 88942 (0.0007) [2023-10-07 23:26:46,478][67838] Updated weights for policy 0, policy_version 88952 (0.0008) [2023-10-07 23:26:47,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 182321152. Throughput: 0: 1648.4, 1: 1652.1. Samples: 45583900. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) [2023-10-07 23:26:47,478][66916] Avg episode reward: [(0, '43.350'), (1, '64.890')] [2023-10-07 23:26:49,491][67871] Updated weights for policy 1, policy_version 89090 (0.0010) [2023-10-07 23:26:49,860][67871] Updated weights for policy 1, policy_version 89100 (0.0010) [2023-10-07 23:26:50,224][67871] Updated weights for policy 1, policy_version 89110 (0.0009) [2023-10-07 23:26:50,588][67871] Updated weights for policy 1, policy_version 89120 (0.0008) [2023-10-07 23:26:50,666][67838] Updated weights for policy 0, policy_version 88962 (0.0007) [2023-10-07 23:26:51,043][67838] Updated weights for policy 0, policy_version 88972 (0.0007) [2023-10-07 23:26:51,414][67838] Updated weights for policy 0, policy_version 88982 (0.0007) [2023-10-07 23:26:51,784][67838] Updated weights for policy 0, policy_version 88992 (0.0007) [2023-10-07 23:26:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 182386688. Throughput: 0: 1651.4, 1: 1667.2. Samples: 45603748. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) [2023-10-07 23:26:52,478][66916] Avg episode reward: [(0, '43.730'), (1, '65.470')] [2023-10-07 23:26:54,634][67871] Updated weights for policy 1, policy_version 89130 (0.0008) [2023-10-07 23:26:54,996][67871] Updated weights for policy 1, policy_version 89140 (0.0008) [2023-10-07 23:26:55,365][67871] Updated weights for policy 1, policy_version 89150 (0.0011) [2023-10-07 23:26:55,781][67838] Updated weights for policy 0, policy_version 89002 (0.0007) [2023-10-07 23:26:56,159][67838] Updated weights for policy 0, policy_version 89012 (0.0007) [2023-10-07 23:26:56,536][67838] Updated weights for policy 0, policy_version 89022 (0.0007) [2023-10-07 23:26:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 182452224. Throughput: 0: 1663.3, 1: 1654.7. Samples: 45614746. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) [2023-10-07 23:26:57,478][66916] Avg episode reward: [(0, '45.760'), (1, '63.760')] [2023-10-07 23:26:59,442][67871] Updated weights for policy 1, policy_version 89160 (0.0010) [2023-10-07 23:26:59,804][67871] Updated weights for policy 1, policy_version 89170 (0.0008) [2023-10-07 23:27:00,166][67871] Updated weights for policy 1, policy_version 89180 (0.0009) [2023-10-07 23:27:00,462][67838] Updated weights for policy 0, policy_version 89032 (0.0009) [2023-10-07 23:27:00,836][67838] Updated weights for policy 0, policy_version 89042 (0.0009) [2023-10-07 23:27:01,203][67838] Updated weights for policy 0, policy_version 89052 (0.0008) [2023-10-07 23:27:02,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182517760. Throughput: 0: 1649.4, 1: 1667.8. Samples: 45633910. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) [2023-10-07 23:27:02,477][66916] Avg episode reward: [(0, '44.580'), (1, '62.500')] [2023-10-07 23:27:04,388][67871] Updated weights for policy 1, policy_version 89190 (0.0008) [2023-10-07 23:27:04,767][67871] Updated weights for policy 1, policy_version 89200 (0.0010) [2023-10-07 23:27:05,135][67871] Updated weights for policy 1, policy_version 89210 (0.0010) [2023-10-07 23:27:05,293][67838] Updated weights for policy 0, policy_version 89062 (0.0008) [2023-10-07 23:27:05,667][67838] Updated weights for policy 0, policy_version 89072 (0.0009) [2023-10-07 23:27:06,040][67838] Updated weights for policy 0, policy_version 89082 (0.0008) [2023-10-07 23:27:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182583296. Throughput: 0: 1666.0, 1: 1668.8. Samples: 45654050. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) [2023-10-07 23:27:07,477][66916] Avg episode reward: [(0, '42.790'), (1, '62.990')] [2023-10-07 23:27:07,487][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000089216_91357184.pth... [2023-10-07 23:27:07,487][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000089088_91226112.pth... [2023-10-07 23:27:07,524][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000087648_89751552.pth [2023-10-07 23:27:07,526][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000087520_89620480.pth [2023-10-07 23:27:09,291][67871] Updated weights for policy 1, policy_version 89220 (0.0007) [2023-10-07 23:27:09,654][67871] Updated weights for policy 1, policy_version 89230 (0.0007) [2023-10-07 23:27:10,024][67871] Updated weights for policy 1, policy_version 89240 (0.0008) [2023-10-07 23:27:10,211][67838] Updated weights for policy 0, policy_version 89092 (0.0009) [2023-10-07 23:27:10,578][67838] Updated weights for policy 0, policy_version 89102 (0.0007) [2023-10-07 23:27:10,948][67838] Updated weights for policy 0, policy_version 89112 (0.0009) [2023-10-07 23:27:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182648832. Throughput: 0: 1675.1, 1: 1656.7. Samples: 45665118. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) [2023-10-07 23:27:12,477][66916] Avg episode reward: [(0, '45.230'), (1, '57.850')] [2023-10-07 23:27:14,045][67871] Updated weights for policy 1, policy_version 89250 (0.0008) [2023-10-07 23:27:14,410][67871] Updated weights for policy 1, policy_version 89260 (0.0007) [2023-10-07 23:27:14,784][67871] Updated weights for policy 1, policy_version 89270 (0.0007) [2023-10-07 23:27:15,103][67838] Updated weights for policy 0, policy_version 89122 (0.0009) [2023-10-07 23:27:15,145][67871] Updated weights for policy 1, policy_version 89280 (0.0007) [2023-10-07 23:27:15,475][67838] Updated weights for policy 0, policy_version 89132 (0.0009) [2023-10-07 23:27:15,846][67838] Updated weights for policy 0, policy_version 89142 (0.0008) [2023-10-07 23:27:16,216][67838] Updated weights for policy 0, policy_version 89152 (0.0010) [2023-10-07 23:27:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.4). Total num frames: 182714368. Throughput: 0: 1655.2, 1: 1669.9. Samples: 45683960. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) [2023-10-07 23:27:17,478][66916] Avg episode reward: [(0, '48.760'), (1, '59.760')] [2023-10-07 23:27:19,252][67871] Updated weights for policy 1, policy_version 89290 (0.0011) [2023-10-07 23:27:19,621][67871] Updated weights for policy 1, policy_version 89300 (0.0009) [2023-10-07 23:27:19,980][67871] Updated weights for policy 1, policy_version 89310 (0.0008) [2023-10-07 23:27:20,441][67838] Updated weights for policy 0, policy_version 89162 (0.0008) [2023-10-07 23:27:20,812][67838] Updated weights for policy 0, policy_version 89172 (0.0009) [2023-10-07 23:27:21,181][67838] Updated weights for policy 0, policy_version 89182 (0.0011) [2023-10-07 23:27:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182779904. Throughput: 0: 1670.0, 1: 1667.1. Samples: 45703930. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) [2023-10-07 23:27:22,477][66916] Avg episode reward: [(0, '47.310'), (1, '61.590')] [2023-10-07 23:27:24,069][67871] Updated weights for policy 1, policy_version 89320 (0.0009) [2023-10-07 23:27:24,433][67871] Updated weights for policy 1, policy_version 89330 (0.0008) [2023-10-07 23:27:24,808][67871] Updated weights for policy 1, policy_version 89340 (0.0007) [2023-10-07 23:27:25,336][67838] Updated weights for policy 0, policy_version 89192 (0.0009) [2023-10-07 23:27:25,707][67838] Updated weights for policy 0, policy_version 89202 (0.0007) [2023-10-07 23:27:26,079][67838] Updated weights for policy 0, policy_version 89212 (0.0009) [2023-10-07 23:27:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182845440. Throughput: 0: 1666.5, 1: 1653.3. Samples: 45714302. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) [2023-10-07 23:27:27,477][66916] Avg episode reward: [(0, '49.040'), (1, '61.620')] [2023-10-07 23:27:28,891][67871] Updated weights for policy 1, policy_version 89350 (0.0010) [2023-10-07 23:27:29,256][67871] Updated weights for policy 1, policy_version 89360 (0.0010) [2023-10-07 23:27:29,624][67871] Updated weights for policy 1, policy_version 89370 (0.0011) [2023-10-07 23:27:30,164][67838] Updated weights for policy 0, policy_version 89222 (0.0008) [2023-10-07 23:27:30,533][67838] Updated weights for policy 0, policy_version 89232 (0.0011) [2023-10-07 23:27:30,910][67838] Updated weights for policy 0, policy_version 89242 (0.0011) [2023-10-07 23:27:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182910976. Throughput: 0: 1651.5, 1: 1673.3. Samples: 45733518. Policy #0 lag: (min: 26.0, avg: 26.8, max: 42.0) [2023-10-07 23:27:32,478][66916] Avg episode reward: [(0, '48.380'), (1, '59.160')] [2023-10-07 23:27:33,806][67871] Updated weights for policy 1, policy_version 89380 (0.0010) [2023-10-07 23:27:34,170][67871] Updated weights for policy 1, policy_version 89390 (0.0010) [2023-10-07 23:27:34,541][67871] Updated weights for policy 1, policy_version 89400 (0.0009) [2023-10-07 23:27:35,154][67838] Updated weights for policy 0, policy_version 89252 (0.0009) [2023-10-07 23:27:35,521][67838] Updated weights for policy 0, policy_version 89262 (0.0009) [2023-10-07 23:27:35,890][67838] Updated weights for policy 0, policy_version 89272 (0.0010) [2023-10-07 23:27:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182976512. Throughput: 0: 1662.7, 1: 1671.8. Samples: 45753798. Policy #0 lag: (min: 26.0, avg: 26.8, max: 42.0) [2023-10-07 23:27:37,478][66916] Avg episode reward: [(0, '44.430'), (1, '59.210')] [2023-10-07 23:27:38,524][67871] Updated weights for policy 1, policy_version 89410 (0.0010) [2023-10-07 23:27:38,900][67871] Updated weights for policy 1, policy_version 89420 (0.0008) [2023-10-07 23:27:39,258][67871] Updated weights for policy 1, policy_version 89430 (0.0008) [2023-10-07 23:27:39,628][67871] Updated weights for policy 1, policy_version 89440 (0.0007) [2023-10-07 23:27:40,126][67838] Updated weights for policy 0, policy_version 89282 (0.0011) [2023-10-07 23:27:40,501][67838] Updated weights for policy 0, policy_version 89292 (0.0008) [2023-10-07 23:27:40,865][67838] Updated weights for policy 0, policy_version 89302 (0.0009) [2023-10-07 23:27:41,237][67838] Updated weights for policy 0, policy_version 89312 (0.0011) [2023-10-07 23:27:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 183042048. Throughput: 0: 1659.7, 1: 1660.1. Samples: 45764134. Policy #0 lag: (min: 26.0, avg: 26.8, max: 42.0) [2023-10-07 23:27:42,477][66916] Avg episode reward: [(0, '46.510'), (1, '60.020')] [2023-10-07 23:27:43,642][67871] Updated weights for policy 1, policy_version 89450 (0.0007) [2023-10-07 23:27:44,009][67871] Updated weights for policy 1, policy_version 89460 (0.0007) [2023-10-07 23:27:44,376][67871] Updated weights for policy 1, policy_version 89470 (0.0010) [2023-10-07 23:27:45,419][67838] Updated weights for policy 0, policy_version 89322 (0.0011) [2023-10-07 23:27:45,798][67838] Updated weights for policy 0, policy_version 89332 (0.0011) [2023-10-07 23:27:46,160][67838] Updated weights for policy 0, policy_version 89342 (0.0008) [2023-10-07 23:27:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 183107584. Throughput: 0: 1651.4, 1: 1675.6. Samples: 45783626. Policy #0 lag: (min: 26.0, avg: 26.8, max: 42.0) [2023-10-07 23:27:47,477][66916] Avg episode reward: [(0, '45.390'), (1, '59.940')] [2023-10-07 23:27:48,635][67871] Updated weights for policy 1, policy_version 89480 (0.0009) [2023-10-07 23:27:48,992][67871] Updated weights for policy 1, policy_version 89490 (0.0010) [2023-10-07 23:27:49,362][67871] Updated weights for policy 1, policy_version 89500 (0.0010) [2023-10-07 23:27:50,412][67838] Updated weights for policy 0, policy_version 89352 (0.0009) [2023-10-07 23:27:50,787][67838] Updated weights for policy 0, policy_version 89362 (0.0008) [2023-10-07 23:27:51,158][67838] Updated weights for policy 0, policy_version 89372 (0.0011) [2023-10-07 23:27:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 183173120. Throughput: 0: 1649.3, 1: 1677.3. Samples: 45803750. Policy #0 lag: (min: 26.0, avg: 26.8, max: 42.0) [2023-10-07 23:27:52,478][66916] Avg episode reward: [(0, '42.000'), (1, '61.730')] [2023-10-07 23:27:53,387][67871] Updated weights for policy 1, policy_version 89510 (0.0010) [2023-10-07 23:27:53,764][67871] Updated weights for policy 1, policy_version 89520 (0.0010) [2023-10-07 23:27:54,128][67871] Updated weights for policy 1, policy_version 89530 (0.0009) [2023-10-07 23:27:55,164][67838] Updated weights for policy 0, policy_version 89382 (0.0009) [2023-10-07 23:27:55,537][67838] Updated weights for policy 0, policy_version 89392 (0.0009) [2023-10-07 23:27:55,919][67838] Updated weights for policy 0, policy_version 89402 (0.0007) [2023-10-07 23:27:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 183238656. Throughput: 0: 1644.8, 1: 1661.7. Samples: 45813912. Policy #0 lag: (min: 26.0, avg: 26.8, max: 42.0) [2023-10-07 23:27:57,478][66916] Avg episode reward: [(0, '40.300'), (1, '59.800')] [2023-10-07 23:27:58,325][67871] Updated weights for policy 1, policy_version 89540 (0.0007) [2023-10-07 23:27:58,699][67871] Updated weights for policy 1, policy_version 89550 (0.0007) [2023-10-07 23:27:59,065][67871] Updated weights for policy 1, policy_version 89560 (0.0008) [2023-10-07 23:28:00,081][67838] Updated weights for policy 0, policy_version 89412 (0.0010) [2023-10-07 23:28:00,459][67838] Updated weights for policy 0, policy_version 89422 (0.0009) [2023-10-07 23:28:00,824][67838] Updated weights for policy 0, policy_version 89432 (0.0007) [2023-10-07 23:28:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 183304192. Throughput: 0: 1646.4, 1: 1672.9. Samples: 45833330. Policy #0 lag: (min: 26.0, avg: 26.8, max: 42.0) [2023-10-07 23:28:02,478][66916] Avg episode reward: [(0, '39.470'), (1, '65.140')] [2023-10-07 23:28:03,248][67871] Updated weights for policy 1, policy_version 89570 (0.0008) [2023-10-07 23:28:03,617][67871] Updated weights for policy 1, policy_version 89580 (0.0007) [2023-10-07 23:28:03,991][67871] Updated weights for policy 1, policy_version 89590 (0.0010) [2023-10-07 23:28:04,354][67871] Updated weights for policy 1, policy_version 89600 (0.0007) [2023-10-07 23:28:04,997][67838] Updated weights for policy 0, policy_version 89442 (0.0009) [2023-10-07 23:28:05,365][67838] Updated weights for policy 0, policy_version 89452 (0.0009) [2023-10-07 23:28:05,731][67838] Updated weights for policy 0, policy_version 89462 (0.0010) [2023-10-07 23:28:06,107][67838] Updated weights for policy 0, policy_version 89472 (0.0012) [2023-10-07 23:28:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 183369728. Throughput: 0: 1656.2, 1: 1677.2. Samples: 45853936. Policy #0 lag: (min: 26.0, avg: 26.8, max: 42.0) [2023-10-07 23:28:07,478][66916] Avg episode reward: [(0, '42.360'), (1, '66.580')] [2023-10-07 23:28:08,185][67871] Updated weights for policy 1, policy_version 89610 (0.0010) [2023-10-07 23:28:08,554][67871] Updated weights for policy 1, policy_version 89620 (0.0010) [2023-10-07 23:28:08,913][67871] Updated weights for policy 1, policy_version 89630 (0.0010) [2023-10-07 23:28:10,123][67838] Updated weights for policy 0, policy_version 89482 (0.0009) [2023-10-07 23:28:10,486][67838] Updated weights for policy 0, policy_version 89492 (0.0009) [2023-10-07 23:28:10,862][67838] Updated weights for policy 0, policy_version 89502 (0.0008) [2023-10-07 23:28:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 183435264. Throughput: 0: 1650.6, 1: 1670.1. Samples: 45863736. Policy #0 lag: (min: 26.0, avg: 26.8, max: 42.0) [2023-10-07 23:28:12,477][66916] Avg episode reward: [(0, '43.980'), (1, '66.320')] [2023-10-07 23:28:13,072][67871] Updated weights for policy 1, policy_version 89640 (0.0008) [2023-10-07 23:28:13,442][67871] Updated weights for policy 1, policy_version 89650 (0.0007) [2023-10-07 23:28:13,807][67871] Updated weights for policy 1, policy_version 89660 (0.0008) [2023-10-07 23:28:14,931][67838] Updated weights for policy 0, policy_version 89512 (0.0007) [2023-10-07 23:28:15,293][67838] Updated weights for policy 0, policy_version 89522 (0.0007) [2023-10-07 23:28:15,668][67838] Updated weights for policy 0, policy_version 89532 (0.0010) [2023-10-07 23:28:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 183500800. Throughput: 0: 1652.8, 1: 1674.7. Samples: 45883254. Policy #0 lag: (min: 26.0, avg: 26.8, max: 42.0) [2023-10-07 23:28:17,478][66916] Avg episode reward: [(0, '45.910'), (1, '66.600')] [2023-10-07 23:28:18,020][67871] Updated weights for policy 1, policy_version 89670 (0.0009) [2023-10-07 23:28:18,383][67871] Updated weights for policy 1, policy_version 89680 (0.0007) [2023-10-07 23:28:18,752][67871] Updated weights for policy 1, policy_version 89690 (0.0007) [2023-10-07 23:28:19,737][67838] Updated weights for policy 0, policy_version 89542 (0.0010) [2023-10-07 23:28:20,104][67838] Updated weights for policy 0, policy_version 89552 (0.0010) [2023-10-07 23:28:20,474][67838] Updated weights for policy 0, policy_version 89562 (0.0010) [2023-10-07 23:28:22,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 183566336. Throughput: 0: 1661.6, 1: 1674.4. Samples: 45903918. Policy #0 lag: (min: 26.0, avg: 26.8, max: 42.0) [2023-10-07 23:28:22,478][66916] Avg episode reward: [(0, '46.200'), (1, '67.790')] [2023-10-07 23:28:22,777][67871] Updated weights for policy 1, policy_version 89700 (0.0008) [2023-10-07 23:28:23,149][67871] Updated weights for policy 1, policy_version 89710 (0.0008) [2023-10-07 23:28:23,510][67871] Updated weights for policy 1, policy_version 89720 (0.0009) [2023-10-07 23:28:24,623][67838] Updated weights for policy 0, policy_version 89572 (0.0012) [2023-10-07 23:28:24,996][67838] Updated weights for policy 0, policy_version 89582 (0.0008) [2023-10-07 23:28:25,375][67838] Updated weights for policy 0, policy_version 89592 (0.0010) [2023-10-07 23:28:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 183631872. Throughput: 0: 1646.4, 1: 1672.9. Samples: 45913504. Policy #0 lag: (min: 26.0, avg: 26.8, max: 42.0) [2023-10-07 23:28:27,477][66916] Avg episode reward: [(0, '45.150'), (1, '67.130')] [2023-10-07 23:28:27,540][67871] Updated weights for policy 1, policy_version 89730 (0.0009) [2023-10-07 23:28:27,911][67871] Updated weights for policy 1, policy_version 89740 (0.0009) [2023-10-07 23:28:28,276][67871] Updated weights for policy 1, policy_version 89750 (0.0009) [2023-10-07 23:28:28,649][67871] Updated weights for policy 1, policy_version 89760 (0.0007) [2023-10-07 23:28:29,541][67838] Updated weights for policy 0, policy_version 89602 (0.0010) [2023-10-07 23:28:29,922][67838] Updated weights for policy 0, policy_version 89612 (0.0009) [2023-10-07 23:28:30,287][67838] Updated weights for policy 0, policy_version 89622 (0.0009) [2023-10-07 23:28:30,649][67838] Updated weights for policy 0, policy_version 89632 (0.0010) [2023-10-07 23:28:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 183697408. Throughput: 0: 1655.6, 1: 1672.2. Samples: 45933376. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 23:28:32,477][66916] Avg episode reward: [(0, '48.730'), (1, '61.870')] [2023-10-07 23:28:32,831][67871] Updated weights for policy 1, policy_version 89770 (0.0010) [2023-10-07 23:28:33,199][67871] Updated weights for policy 1, policy_version 89780 (0.0009) [2023-10-07 23:28:33,567][67871] Updated weights for policy 1, policy_version 89790 (0.0007) [2023-10-07 23:28:34,793][67838] Updated weights for policy 0, policy_version 89642 (0.0008) [2023-10-07 23:28:35,168][67838] Updated weights for policy 0, policy_version 89652 (0.0008) [2023-10-07 23:28:35,531][67838] Updated weights for policy 0, policy_version 89662 (0.0011) [2023-10-07 23:28:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 183762944. Throughput: 0: 1660.7, 1: 1672.7. Samples: 45953750. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 23:28:37,477][66916] Avg episode reward: [(0, '46.500'), (1, '60.100')] [2023-10-07 23:28:37,721][67871] Updated weights for policy 1, policy_version 89800 (0.0009) [2023-10-07 23:28:38,101][67871] Updated weights for policy 1, policy_version 89810 (0.0008) [2023-10-07 23:28:38,456][67871] Updated weights for policy 1, policy_version 89820 (0.0008) [2023-10-07 23:28:39,599][67838] Updated weights for policy 0, policy_version 89672 (0.0009) [2023-10-07 23:28:39,970][67838] Updated weights for policy 0, policy_version 89682 (0.0009) [2023-10-07 23:28:40,330][67838] Updated weights for policy 0, policy_version 89692 (0.0007) [2023-10-07 23:28:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 183828480. Throughput: 0: 1644.0, 1: 1677.6. Samples: 45963380. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 23:28:42,477][66916] Avg episode reward: [(0, '46.180'), (1, '59.010')] [2023-10-07 23:28:42,579][67871] Updated weights for policy 1, policy_version 89830 (0.0010) [2023-10-07 23:28:42,948][67871] Updated weights for policy 1, policy_version 89840 (0.0008) [2023-10-07 23:28:43,328][67871] Updated weights for policy 1, policy_version 89850 (0.0010) [2023-10-07 23:28:44,357][67838] Updated weights for policy 0, policy_version 89702 (0.0009) [2023-10-07 23:28:44,732][67838] Updated weights for policy 0, policy_version 89712 (0.0009) [2023-10-07 23:28:45,112][67838] Updated weights for policy 0, policy_version 89722 (0.0009) [2023-10-07 23:28:47,340][67871] Updated weights for policy 1, policy_version 89860 (0.0009) [2023-10-07 23:28:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 183894016. Throughput: 0: 1658.1, 1: 1673.1. Samples: 45983232. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 23:28:47,477][66916] Avg episode reward: [(0, '44.430'), (1, '60.480')] [2023-10-07 23:28:47,711][67871] Updated weights for policy 1, policy_version 89870 (0.0009) [2023-10-07 23:28:48,069][67871] Updated weights for policy 1, policy_version 89880 (0.0010) [2023-10-07 23:28:49,313][67838] Updated weights for policy 0, policy_version 89732 (0.0008) [2023-10-07 23:28:49,691][67838] Updated weights for policy 0, policy_version 89742 (0.0008) [2023-10-07 23:28:50,067][67838] Updated weights for policy 0, policy_version 89752 (0.0010) [2023-10-07 23:28:52,138][67871] Updated weights for policy 1, policy_version 89890 (0.0008) [2023-10-07 23:28:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 183959552. Throughput: 0: 1654.1, 1: 1674.7. Samples: 46003732. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 23:28:52,477][66916] Avg episode reward: [(0, '42.970'), (1, '57.750')] [2023-10-07 23:28:52,509][67871] Updated weights for policy 1, policy_version 89900 (0.0009) [2023-10-07 23:28:52,881][67871] Updated weights for policy 1, policy_version 89910 (0.0012) [2023-10-07 23:28:53,251][67871] Updated weights for policy 1, policy_version 89920 (0.0011) [2023-10-07 23:28:54,274][67838] Updated weights for policy 0, policy_version 89762 (0.0008) [2023-10-07 23:28:54,646][67838] Updated weights for policy 0, policy_version 89772 (0.0008) [2023-10-07 23:28:55,029][67838] Updated weights for policy 0, policy_version 89782 (0.0009) [2023-10-07 23:28:55,395][67838] Updated weights for policy 0, policy_version 89792 (0.0009) [2023-10-07 23:28:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 184025088. Throughput: 0: 1644.8, 1: 1673.4. Samples: 46013054. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 23:28:57,477][66916] Avg episode reward: [(0, '44.460'), (1, '59.010')] [2023-10-07 23:28:57,502][67871] Updated weights for policy 1, policy_version 89930 (0.0007) [2023-10-07 23:28:57,872][67871] Updated weights for policy 1, policy_version 89940 (0.0008) [2023-10-07 23:28:58,229][67871] Updated weights for policy 1, policy_version 89950 (0.0010) [2023-10-07 23:28:59,712][67838] Updated weights for policy 0, policy_version 89802 (0.0009) [2023-10-07 23:29:00,087][67838] Updated weights for policy 0, policy_version 89812 (0.0008) [2023-10-07 23:29:00,455][67838] Updated weights for policy 0, policy_version 89822 (0.0008) [2023-10-07 23:29:02,025][67871] Updated weights for policy 1, policy_version 89960 (0.0008) [2023-10-07 23:29:02,388][67871] Updated weights for policy 1, policy_version 89970 (0.0009) [2023-10-07 23:29:02,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 184090624. Throughput: 0: 1654.2, 1: 1678.7. Samples: 46033236. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 23:29:02,478][66916] Avg episode reward: [(0, '46.250'), (1, '58.090')] [2023-10-07 23:29:02,761][67871] Updated weights for policy 1, policy_version 89980 (0.0009) [2023-10-07 23:29:04,490][67838] Updated weights for policy 0, policy_version 89832 (0.0007) [2023-10-07 23:29:04,861][67838] Updated weights for policy 0, policy_version 89842 (0.0010) [2023-10-07 23:29:05,240][67838] Updated weights for policy 0, policy_version 89852 (0.0009) [2023-10-07 23:29:06,909][67871] Updated weights for policy 1, policy_version 89990 (0.0009) [2023-10-07 23:29:07,267][67871] Updated weights for policy 1, policy_version 90000 (0.0007) [2023-10-07 23:29:07,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 184156160. Throughput: 0: 1657.0, 1: 1672.5. Samples: 46053746. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 23:29:07,477][66916] Avg episode reward: [(0, '43.920'), (1, '59.220')] [2023-10-07 23:29:07,488][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000089856_92012544.pth... [2023-10-07 23:29:07,524][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000088320_90439680.pth [2023-10-07 23:29:07,632][67871] Updated weights for policy 1, policy_version 90010 (0.0008) [2023-10-07 23:29:07,858][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000090016_92176384.pth... [2023-10-07 23:29:07,897][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000088448_90570752.pth [2023-10-07 23:29:09,138][67838] Updated weights for policy 0, policy_version 89862 (0.0008) [2023-10-07 23:29:09,515][67838] Updated weights for policy 0, policy_version 89872 (0.0007) [2023-10-07 23:29:09,889][67838] Updated weights for policy 0, policy_version 89882 (0.0007) [2023-10-07 23:29:11,778][67871] Updated weights for policy 1, policy_version 90020 (0.0009) [2023-10-07 23:29:12,153][67871] Updated weights for policy 1, policy_version 90030 (0.0009) [2023-10-07 23:29:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 184221696. Throughput: 0: 1645.2, 1: 1677.7. Samples: 46063034. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 23:29:12,477][66916] Avg episode reward: [(0, '47.160'), (1, '61.870')] [2023-10-07 23:29:12,525][67871] Updated weights for policy 1, policy_version 90040 (0.0008) [2023-10-07 23:29:14,041][67838] Updated weights for policy 0, policy_version 89892 (0.0008) [2023-10-07 23:29:14,405][67838] Updated weights for policy 0, policy_version 89902 (0.0008) [2023-10-07 23:29:14,774][67838] Updated weights for policy 0, policy_version 89912 (0.0007) [2023-10-07 23:29:16,558][67871] Updated weights for policy 1, policy_version 90050 (0.0009) [2023-10-07 23:29:16,921][67871] Updated weights for policy 1, policy_version 90060 (0.0009) [2023-10-07 23:29:17,287][67871] Updated weights for policy 1, policy_version 90070 (0.0007) [2023-10-07 23:29:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 184287232. Throughput: 0: 1658.4, 1: 1678.6. Samples: 46083542. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 23:29:17,478][66916] Avg episode reward: [(0, '42.120'), (1, '61.480')] [2023-10-07 23:29:17,654][67871] Updated weights for policy 1, policy_version 90080 (0.0009) [2023-10-07 23:29:18,862][67838] Updated weights for policy 0, policy_version 89922 (0.0008) [2023-10-07 23:29:19,237][67838] Updated weights for policy 0, policy_version 89932 (0.0008) [2023-10-07 23:29:19,610][67838] Updated weights for policy 0, policy_version 89942 (0.0007) [2023-10-07 23:29:19,982][67838] Updated weights for policy 0, policy_version 89952 (0.0008) [2023-10-07 23:29:21,761][67871] Updated weights for policy 1, policy_version 90090 (0.0008) [2023-10-07 23:29:22,131][67871] Updated weights for policy 1, policy_version 90100 (0.0008) [2023-10-07 23:29:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 184352768. Throughput: 0: 1662.7, 1: 1668.9. Samples: 46103670. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 23:29:22,477][66916] Avg episode reward: [(0, '40.900'), (1, '64.030')] [2023-10-07 23:29:22,497][67871] Updated weights for policy 1, policy_version 90110 (0.0009) [2023-10-07 23:29:24,186][67838] Updated weights for policy 0, policy_version 89962 (0.0011) [2023-10-07 23:29:24,548][67838] Updated weights for policy 0, policy_version 89972 (0.0010) [2023-10-07 23:29:24,931][67838] Updated weights for policy 0, policy_version 89982 (0.0011) [2023-10-07 23:29:26,802][67871] Updated weights for policy 1, policy_version 90120 (0.0008) [2023-10-07 23:29:27,181][67871] Updated weights for policy 1, policy_version 90130 (0.0008) [2023-10-07 23:29:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 184418304. Throughput: 0: 1647.8, 1: 1676.5. Samples: 46112974. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-07 23:29:27,478][66916] Avg episode reward: [(0, '43.860'), (1, '65.410')] [2023-10-07 23:29:27,549][67871] Updated weights for policy 1, policy_version 90140 (0.0008) [2023-10-07 23:29:29,059][67838] Updated weights for policy 0, policy_version 89992 (0.0010) [2023-10-07 23:29:29,443][67838] Updated weights for policy 0, policy_version 90002 (0.0008) [2023-10-07 23:29:29,827][67838] Updated weights for policy 0, policy_version 90012 (0.0009) [2023-10-07 23:29:31,552][67871] Updated weights for policy 1, policy_version 90150 (0.0007) [2023-10-07 23:29:31,918][67871] Updated weights for policy 1, policy_version 90160 (0.0010) [2023-10-07 23:29:32,279][67871] Updated weights for policy 1, policy_version 90170 (0.0009) [2023-10-07 23:29:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 184483840. Throughput: 0: 1659.8, 1: 1680.2. Samples: 46133532. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-07 23:29:32,477][66916] Avg episode reward: [(0, '40.950'), (1, '65.360')] [2023-10-07 23:29:33,980][67838] Updated weights for policy 0, policy_version 90022 (0.0011) [2023-10-07 23:29:34,341][67838] Updated weights for policy 0, policy_version 90032 (0.0007) [2023-10-07 23:29:34,712][67838] Updated weights for policy 0, policy_version 90042 (0.0007) [2023-10-07 23:29:36,345][67871] Updated weights for policy 1, policy_version 90180 (0.0009) [2023-10-07 23:29:36,710][67871] Updated weights for policy 1, policy_version 90190 (0.0009) [2023-10-07 23:29:37,064][67871] Updated weights for policy 1, policy_version 90200 (0.0008) [2023-10-07 23:29:37,476][66916] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 184582144. Throughput: 0: 1664.5, 1: 1660.4. Samples: 46153352. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-07 23:29:37,477][66916] Avg episode reward: [(0, '46.010'), (1, '67.480')] [2023-10-07 23:29:38,824][67838] Updated weights for policy 0, policy_version 90052 (0.0010) [2023-10-07 23:29:39,199][67838] Updated weights for policy 0, policy_version 90062 (0.0008) [2023-10-07 23:29:39,566][67838] Updated weights for policy 0, policy_version 90072 (0.0007) [2023-10-07 23:29:41,099][67871] Updated weights for policy 1, policy_version 90210 (0.0009) [2023-10-07 23:29:41,465][67871] Updated weights for policy 1, policy_version 90220 (0.0007) [2023-10-07 23:29:41,839][67871] Updated weights for policy 1, policy_version 90230 (0.0008) [2023-10-07 23:29:42,195][67871] Updated weights for policy 1, policy_version 90240 (0.0009) [2023-10-07 23:29:42,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 184647680. Throughput: 0: 1653.3, 1: 1679.3. Samples: 46163022. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-07 23:29:42,478][66916] Avg episode reward: [(0, '47.470'), (1, '63.600')] [2023-10-07 23:29:43,776][67838] Updated weights for policy 0, policy_version 90082 (0.0008) [2023-10-07 23:29:44,145][67838] Updated weights for policy 0, policy_version 90092 (0.0010) [2023-10-07 23:29:44,519][67838] Updated weights for policy 0, policy_version 90102 (0.0011) [2023-10-07 23:29:44,899][67838] Updated weights for policy 0, policy_version 90112 (0.0011) [2023-10-07 23:29:46,234][67871] Updated weights for policy 1, policy_version 90250 (0.0009) [2023-10-07 23:29:46,604][67871] Updated weights for policy 1, policy_version 90260 (0.0011) [2023-10-07 23:29:46,970][67871] Updated weights for policy 1, policy_version 90270 (0.0009) [2023-10-07 23:29:47,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 184713216. Throughput: 0: 1661.9, 1: 1673.3. Samples: 46183320. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-07 23:29:47,477][66916] Avg episode reward: [(0, '50.010'), (1, '64.170')] [2023-10-07 23:29:49,195][67838] Updated weights for policy 0, policy_version 90122 (0.0009) [2023-10-07 23:29:49,573][67838] Updated weights for policy 0, policy_version 90132 (0.0008) [2023-10-07 23:29:49,943][67838] Updated weights for policy 0, policy_version 90142 (0.0007) [2023-10-07 23:29:51,226][67871] Updated weights for policy 1, policy_version 90280 (0.0008) [2023-10-07 23:29:51,591][67871] Updated weights for policy 1, policy_version 90290 (0.0007) [2023-10-07 23:29:51,957][67871] Updated weights for policy 1, policy_version 90300 (0.0009) [2023-10-07 23:29:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 184778752. Throughput: 0: 1658.0, 1: 1653.3. Samples: 46202754. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-07 23:29:52,477][66916] Avg episode reward: [(0, '51.640'), (1, '63.020')] [2023-10-07 23:29:53,983][67838] Updated weights for policy 0, policy_version 90152 (0.0009) [2023-10-07 23:29:54,360][67838] Updated weights for policy 0, policy_version 90162 (0.0007) [2023-10-07 23:29:54,723][67838] Updated weights for policy 0, policy_version 90172 (0.0009) [2023-10-07 23:29:55,994][67871] Updated weights for policy 1, policy_version 90310 (0.0007) [2023-10-07 23:29:56,361][67871] Updated weights for policy 1, policy_version 90320 (0.0008) [2023-10-07 23:29:56,733][67871] Updated weights for policy 1, policy_version 90330 (0.0009) [2023-10-07 23:29:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 184844288. Throughput: 0: 1656.3, 1: 1670.3. Samples: 46212732. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-07 23:29:57,477][66916] Avg episode reward: [(0, '49.080'), (1, '59.730')] [2023-10-07 23:29:58,760][67838] Updated weights for policy 0, policy_version 90182 (0.0011) [2023-10-07 23:29:59,137][67838] Updated weights for policy 0, policy_version 90192 (0.0007) [2023-10-07 23:29:59,503][67838] Updated weights for policy 0, policy_version 90202 (0.0007) [2023-10-07 23:30:00,745][67871] Updated weights for policy 1, policy_version 90340 (0.0010) [2023-10-07 23:30:01,104][67871] Updated weights for policy 1, policy_version 90350 (0.0009) [2023-10-07 23:30:01,470][67871] Updated weights for policy 1, policy_version 90360 (0.0010) [2023-10-07 23:30:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 184909824. Throughput: 0: 1656.9, 1: 1663.4. Samples: 46232952. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-07 23:30:02,477][66916] Avg episode reward: [(0, '50.330'), (1, '58.130')] [2023-10-07 23:30:03,465][67838] Updated weights for policy 0, policy_version 90212 (0.0008) [2023-10-07 23:30:03,849][67838] Updated weights for policy 0, policy_version 90222 (0.0009) [2023-10-07 23:30:04,223][67838] Updated weights for policy 0, policy_version 90232 (0.0008) [2023-10-07 23:30:05,510][67871] Updated weights for policy 1, policy_version 90370 (0.0010) [2023-10-07 23:30:05,881][67871] Updated weights for policy 1, policy_version 90380 (0.0009) [2023-10-07 23:30:06,245][67871] Updated weights for policy 1, policy_version 90390 (0.0008) [2023-10-07 23:30:06,606][67871] Updated weights for policy 1, policy_version 90400 (0.0010) [2023-10-07 23:30:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 184975360. Throughput: 0: 1660.3, 1: 1653.7. Samples: 46252800. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-07 23:30:07,477][66916] Avg episode reward: [(0, '50.330'), (1, '55.150')] [2023-10-07 23:30:08,415][67838] Updated weights for policy 0, policy_version 90242 (0.0009) [2023-10-07 23:30:08,800][67838] Updated weights for policy 0, policy_version 90252 (0.0008) [2023-10-07 23:30:09,161][67838] Updated weights for policy 0, policy_version 90262 (0.0010) [2023-10-07 23:30:09,535][67838] Updated weights for policy 0, policy_version 90272 (0.0009) [2023-10-07 23:30:10,779][67871] Updated weights for policy 1, policy_version 90410 (0.0008) [2023-10-07 23:30:11,141][67871] Updated weights for policy 1, policy_version 90420 (0.0008) [2023-10-07 23:30:11,511][67871] Updated weights for policy 1, policy_version 90430 (0.0007) [2023-10-07 23:30:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 185040896. Throughput: 0: 1664.3, 1: 1670.9. Samples: 46263058. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-07 23:30:12,477][66916] Avg episode reward: [(0, '53.050'), (1, '55.490')] [2023-10-07 23:30:13,730][67838] Updated weights for policy 0, policy_version 90282 (0.0007) [2023-10-07 23:30:14,095][67838] Updated weights for policy 0, policy_version 90292 (0.0009) [2023-10-07 23:30:14,464][67838] Updated weights for policy 0, policy_version 90302 (0.0008) [2023-10-07 23:30:15,905][67871] Updated weights for policy 1, policy_version 90440 (0.0008) [2023-10-07 23:30:16,278][67871] Updated weights for policy 1, policy_version 90450 (0.0007) [2023-10-07 23:30:16,643][67871] Updated weights for policy 1, policy_version 90460 (0.0008) [2023-10-07 23:30:17,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 185106432. Throughput: 0: 1662.3, 1: 1662.6. Samples: 46283154. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-07 23:30:17,478][66916] Avg episode reward: [(0, '51.510'), (1, '51.560')] [2023-10-07 23:30:18,648][67838] Updated weights for policy 0, policy_version 90312 (0.0007) [2023-10-07 23:30:19,025][67838] Updated weights for policy 0, policy_version 90322 (0.0007) [2023-10-07 23:30:19,396][67838] Updated weights for policy 0, policy_version 90332 (0.0009) [2023-10-07 23:30:20,854][67871] Updated weights for policy 1, policy_version 90470 (0.0010) [2023-10-07 23:30:21,216][67871] Updated weights for policy 1, policy_version 90480 (0.0009) [2023-10-07 23:30:21,584][67871] Updated weights for policy 1, policy_version 90490 (0.0008) [2023-10-07 23:30:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 185171968. Throughput: 0: 1662.6, 1: 1656.1. Samples: 46302696. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-07 23:30:22,477][66916] Avg episode reward: [(0, '49.890'), (1, '55.480')] [2023-10-07 23:30:23,359][67838] Updated weights for policy 0, policy_version 90342 (0.0008) [2023-10-07 23:30:23,720][67838] Updated weights for policy 0, policy_version 90352 (0.0009) [2023-10-07 23:30:24,100][67838] Updated weights for policy 0, policy_version 90362 (0.0008) [2023-10-07 23:30:25,649][67871] Updated weights for policy 1, policy_version 90500 (0.0010) [2023-10-07 23:30:26,016][67871] Updated weights for policy 1, policy_version 90510 (0.0008) [2023-10-07 23:30:26,379][67871] Updated weights for policy 1, policy_version 90520 (0.0007) [2023-10-07 23:30:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 185237504. Throughput: 0: 1664.9, 1: 1671.2. Samples: 46313150. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-07 23:30:27,478][66916] Avg episode reward: [(0, '49.810'), (1, '57.440')] [2023-10-07 23:30:28,061][67838] Updated weights for policy 0, policy_version 90372 (0.0009) [2023-10-07 23:30:28,439][67838] Updated weights for policy 0, policy_version 90382 (0.0008) [2023-10-07 23:30:28,806][67838] Updated weights for policy 0, policy_version 90392 (0.0009) [2023-10-07 23:30:30,413][67871] Updated weights for policy 1, policy_version 90530 (0.0007) [2023-10-07 23:30:30,772][67871] Updated weights for policy 1, policy_version 90540 (0.0009) [2023-10-07 23:30:31,135][67871] Updated weights for policy 1, policy_version 90550 (0.0009) [2023-10-07 23:30:31,498][67871] Updated weights for policy 1, policy_version 90560 (0.0007) [2023-10-07 23:30:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 185303040. Throughput: 0: 1671.2, 1: 1660.0. Samples: 46333222. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 23:30:32,477][66916] Avg episode reward: [(0, '48.690'), (1, '58.730')] [2023-10-07 23:30:33,026][67838] Updated weights for policy 0, policy_version 90402 (0.0010) [2023-10-07 23:30:33,400][67838] Updated weights for policy 0, policy_version 90412 (0.0007) [2023-10-07 23:30:33,769][67838] Updated weights for policy 0, policy_version 90422 (0.0009) [2023-10-07 23:30:34,133][67838] Updated weights for policy 0, policy_version 90432 (0.0007) [2023-10-07 23:30:35,512][67871] Updated weights for policy 1, policy_version 90570 (0.0009) [2023-10-07 23:30:35,877][67871] Updated weights for policy 1, policy_version 90580 (0.0007) [2023-10-07 23:30:36,237][67871] Updated weights for policy 1, policy_version 90590 (0.0007) [2023-10-07 23:30:37,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185368576. Throughput: 0: 1667.2, 1: 1671.1. Samples: 46352978. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 23:30:37,477][66916] Avg episode reward: [(0, '48.270'), (1, '61.060')] [2023-10-07 23:30:38,461][67838] Updated weights for policy 0, policy_version 90442 (0.0010) [2023-10-07 23:30:38,838][67838] Updated weights for policy 0, policy_version 90452 (0.0009) [2023-10-07 23:30:39,211][67838] Updated weights for policy 0, policy_version 90462 (0.0007) [2023-10-07 23:30:40,296][67871] Updated weights for policy 1, policy_version 90600 (0.0008) [2023-10-07 23:30:40,659][67871] Updated weights for policy 1, policy_version 90610 (0.0008) [2023-10-07 23:30:41,021][67871] Updated weights for policy 1, policy_version 90620 (0.0007) [2023-10-07 23:30:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185434112. Throughput: 0: 1663.1, 1: 1675.7. Samples: 46362978. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 23:30:42,478][66916] Avg episode reward: [(0, '48.000'), (1, '62.790')] [2023-10-07 23:30:43,378][67838] Updated weights for policy 0, policy_version 90472 (0.0008) [2023-10-07 23:30:43,744][67838] Updated weights for policy 0, policy_version 90482 (0.0011) [2023-10-07 23:30:44,105][67838] Updated weights for policy 0, policy_version 90492 (0.0011) [2023-10-07 23:30:45,210][67871] Updated weights for policy 1, policy_version 90630 (0.0009) [2023-10-07 23:30:45,575][67871] Updated weights for policy 1, policy_version 90640 (0.0010) [2023-10-07 23:30:45,940][67871] Updated weights for policy 1, policy_version 90650 (0.0010) [2023-10-07 23:30:47,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 185499648. Throughput: 0: 1661.9, 1: 1656.2. Samples: 46382266. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 23:30:47,478][66916] Avg episode reward: [(0, '52.390'), (1, '63.930')] [2023-10-07 23:30:48,255][67838] Updated weights for policy 0, policy_version 90502 (0.0009) [2023-10-07 23:30:48,616][67838] Updated weights for policy 0, policy_version 90512 (0.0009) [2023-10-07 23:30:48,993][67838] Updated weights for policy 0, policy_version 90522 (0.0009) [2023-10-07 23:30:50,068][67871] Updated weights for policy 1, policy_version 90660 (0.0011) [2023-10-07 23:30:50,439][67871] Updated weights for policy 1, policy_version 90670 (0.0012) [2023-10-07 23:30:50,809][67871] Updated weights for policy 1, policy_version 90680 (0.0009) [2023-10-07 23:30:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185565184. Throughput: 0: 1661.0, 1: 1667.2. Samples: 46402570. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 23:30:52,477][66916] Avg episode reward: [(0, '49.220'), (1, '63.510')] [2023-10-07 23:30:52,916][67838] Updated weights for policy 0, policy_version 90532 (0.0007) [2023-10-07 23:30:53,284][67838] Updated weights for policy 0, policy_version 90542 (0.0007) [2023-10-07 23:30:53,646][67838] Updated weights for policy 0, policy_version 90552 (0.0007) [2023-10-07 23:30:54,910][67871] Updated weights for policy 1, policy_version 90690 (0.0009) [2023-10-07 23:30:55,288][67871] Updated weights for policy 1, policy_version 90700 (0.0007) [2023-10-07 23:30:55,649][67871] Updated weights for policy 1, policy_version 90710 (0.0010) [2023-10-07 23:30:56,027][67871] Updated weights for policy 1, policy_version 90720 (0.0008) [2023-10-07 23:30:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185630720. Throughput: 0: 1660.9, 1: 1669.5. Samples: 46412930. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 23:30:57,478][66916] Avg episode reward: [(0, '46.920'), (1, '67.230')] [2023-10-07 23:30:57,822][67838] Updated weights for policy 0, policy_version 90562 (0.0008) [2023-10-07 23:30:58,183][67838] Updated weights for policy 0, policy_version 90572 (0.0009) [2023-10-07 23:30:58,556][67838] Updated weights for policy 0, policy_version 90582 (0.0008) [2023-10-07 23:30:58,920][67838] Updated weights for policy 0, policy_version 90592 (0.0009) [2023-10-07 23:31:00,148][67871] Updated weights for policy 1, policy_version 90730 (0.0008) [2023-10-07 23:31:00,518][67871] Updated weights for policy 1, policy_version 90740 (0.0008) [2023-10-07 23:31:00,879][67871] Updated weights for policy 1, policy_version 90750 (0.0010) [2023-10-07 23:31:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185696256. Throughput: 0: 1664.8, 1: 1656.0. Samples: 46432588. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 23:31:02,477][66916] Avg episode reward: [(0, '50.360'), (1, '65.640')] [2023-10-07 23:31:03,184][67838] Updated weights for policy 0, policy_version 90602 (0.0009) [2023-10-07 23:31:03,563][67838] Updated weights for policy 0, policy_version 90612 (0.0009) [2023-10-07 23:31:03,942][67838] Updated weights for policy 0, policy_version 90622 (0.0009) [2023-10-07 23:31:04,918][67871] Updated weights for policy 1, policy_version 90760 (0.0008) [2023-10-07 23:31:05,292][67871] Updated weights for policy 1, policy_version 90770 (0.0007) [2023-10-07 23:31:05,657][67871] Updated weights for policy 1, policy_version 90780 (0.0010) [2023-10-07 23:31:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185761792. Throughput: 0: 1658.4, 1: 1674.2. Samples: 46452664. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 23:31:07,477][66916] Avg episode reward: [(0, '50.130'), (1, '63.090')] [2023-10-07 23:31:07,486][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000090784_92962816.pth... [2023-10-07 23:31:07,486][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000090624_92798976.pth... [2023-10-07 23:31:07,524][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000089088_91226112.pth [2023-10-07 23:31:07,526][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000089216_91357184.pth [2023-10-07 23:31:08,073][67838] Updated weights for policy 0, policy_version 90632 (0.0009) [2023-10-07 23:31:08,442][67838] Updated weights for policy 0, policy_version 90642 (0.0007) [2023-10-07 23:31:08,820][67838] Updated weights for policy 0, policy_version 90652 (0.0007) [2023-10-07 23:31:09,745][67871] Updated weights for policy 1, policy_version 90790 (0.0009) [2023-10-07 23:31:10,110][67871] Updated weights for policy 1, policy_version 90800 (0.0009) [2023-10-07 23:31:10,474][67871] Updated weights for policy 1, policy_version 90810 (0.0009) [2023-10-07 23:31:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185827328. Throughput: 0: 1659.0, 1: 1663.4. Samples: 46462660. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 23:31:12,477][66916] Avg episode reward: [(0, '48.190'), (1, '61.450')] [2023-10-07 23:31:12,787][67838] Updated weights for policy 0, policy_version 90662 (0.0008) [2023-10-07 23:31:13,150][67838] Updated weights for policy 0, policy_version 90672 (0.0007) [2023-10-07 23:31:13,527][67838] Updated weights for policy 0, policy_version 90682 (0.0010) [2023-10-07 23:31:14,578][67871] Updated weights for policy 1, policy_version 90820 (0.0008) [2023-10-07 23:31:14,951][67871] Updated weights for policy 1, policy_version 90830 (0.0009) [2023-10-07 23:31:15,317][67871] Updated weights for policy 1, policy_version 90840 (0.0009) [2023-10-07 23:31:17,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185892864. Throughput: 0: 1657.3, 1: 1652.2. Samples: 46482150. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 23:31:17,478][66916] Avg episode reward: [(0, '47.800'), (1, '65.770')] [2023-10-07 23:31:17,605][67838] Updated weights for policy 0, policy_version 90692 (0.0010) [2023-10-07 23:31:17,983][67838] Updated weights for policy 0, policy_version 90702 (0.0012) [2023-10-07 23:31:18,349][67838] Updated weights for policy 0, policy_version 90712 (0.0012) [2023-10-07 23:31:19,283][67871] Updated weights for policy 1, policy_version 90850 (0.0010) [2023-10-07 23:31:19,658][67871] Updated weights for policy 1, policy_version 90860 (0.0008) [2023-10-07 23:31:20,032][67871] Updated weights for policy 1, policy_version 90870 (0.0007) [2023-10-07 23:31:20,397][67871] Updated weights for policy 1, policy_version 90880 (0.0008) [2023-10-07 23:31:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185958400. Throughput: 0: 1658.8, 1: 1667.3. Samples: 46502656. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 23:31:22,478][66916] Avg episode reward: [(0, '48.080'), (1, '60.000')] [2023-10-07 23:31:22,578][67838] Updated weights for policy 0, policy_version 90722 (0.0011) [2023-10-07 23:31:22,956][67838] Updated weights for policy 0, policy_version 90732 (0.0008) [2023-10-07 23:31:23,316][67838] Updated weights for policy 0, policy_version 90742 (0.0009) [2023-10-07 23:31:23,692][67838] Updated weights for policy 0, policy_version 90752 (0.0009) [2023-10-07 23:31:24,630][67871] Updated weights for policy 1, policy_version 90890 (0.0007) [2023-10-07 23:31:24,996][67871] Updated weights for policy 1, policy_version 90900 (0.0009) [2023-10-07 23:31:25,362][67871] Updated weights for policy 1, policy_version 90910 (0.0009) [2023-10-07 23:31:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 186023936. Throughput: 0: 1664.8, 1: 1655.3. Samples: 46512382. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-07 23:31:27,477][66916] Avg episode reward: [(0, '45.270'), (1, '60.490')] [2023-10-07 23:31:27,749][67838] Updated weights for policy 0, policy_version 90762 (0.0010) [2023-10-07 23:31:28,124][67838] Updated weights for policy 0, policy_version 90772 (0.0009) [2023-10-07 23:31:28,500][67838] Updated weights for policy 0, policy_version 90782 (0.0009) [2023-10-07 23:31:29,359][67871] Updated weights for policy 1, policy_version 90920 (0.0010) [2023-10-07 23:31:29,736][67871] Updated weights for policy 1, policy_version 90930 (0.0008) [2023-10-07 23:31:30,105][67871] Updated weights for policy 1, policy_version 90940 (0.0008) [2023-10-07 23:31:32,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 186089472. Throughput: 0: 1668.8, 1: 1664.0. Samples: 46532240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:31:32,478][66916] Avg episode reward: [(0, '45.180'), (1, '59.800')] [2023-10-07 23:31:32,573][67838] Updated weights for policy 0, policy_version 90792 (0.0011) [2023-10-07 23:31:32,938][67838] Updated weights for policy 0, policy_version 90802 (0.0011) [2023-10-07 23:31:33,307][67838] Updated weights for policy 0, policy_version 90812 (0.0010) [2023-10-07 23:31:34,217][67871] Updated weights for policy 1, policy_version 90950 (0.0009) [2023-10-07 23:31:34,591][67871] Updated weights for policy 1, policy_version 90960 (0.0008) [2023-10-07 23:31:34,951][67871] Updated weights for policy 1, policy_version 90970 (0.0007) [2023-10-07 23:31:37,364][67838] Updated weights for policy 0, policy_version 90822 (0.0010) [2023-10-07 23:31:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 186155008. Throughput: 0: 1668.0, 1: 1666.8. Samples: 46552636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:31:37,478][66916] Avg episode reward: [(0, '43.370'), (1, '58.110')] [2023-10-07 23:31:37,743][67838] Updated weights for policy 0, policy_version 90832 (0.0009) [2023-10-07 23:31:38,108][67838] Updated weights for policy 0, policy_version 90842 (0.0009) [2023-10-07 23:31:39,144][67871] Updated weights for policy 1, policy_version 90980 (0.0007) [2023-10-07 23:31:39,510][67871] Updated weights for policy 1, policy_version 90990 (0.0007) [2023-10-07 23:31:39,882][67871] Updated weights for policy 1, policy_version 91000 (0.0007) [2023-10-07 23:31:42,252][67838] Updated weights for policy 0, policy_version 90852 (0.0008) [2023-10-07 23:31:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 186220544. Throughput: 0: 1667.7, 1: 1649.5. Samples: 46562206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:31:42,478][66916] Avg episode reward: [(0, '43.150'), (1, '55.430')] [2023-10-07 23:31:42,633][67838] Updated weights for policy 0, policy_version 90862 (0.0007) [2023-10-07 23:31:43,015][67838] Updated weights for policy 0, policy_version 90872 (0.0009) [2023-10-07 23:31:43,919][67871] Updated weights for policy 1, policy_version 91010 (0.0008) [2023-10-07 23:31:44,288][67871] Updated weights for policy 1, policy_version 91020 (0.0007) [2023-10-07 23:31:44,657][67871] Updated weights for policy 1, policy_version 91030 (0.0009) [2023-10-07 23:31:45,021][67871] Updated weights for policy 1, policy_version 91040 (0.0008) [2023-10-07 23:31:47,134][67838] Updated weights for policy 0, policy_version 90882 (0.0009) [2023-10-07 23:31:47,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 186286080. Throughput: 0: 1661.6, 1: 1665.6. Samples: 46582316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:31:47,478][66916] Avg episode reward: [(0, '43.980'), (1, '59.140')] [2023-10-07 23:31:47,511][67838] Updated weights for policy 0, policy_version 90892 (0.0007) [2023-10-07 23:31:47,881][67838] Updated weights for policy 0, policy_version 90902 (0.0009) [2023-10-07 23:31:48,247][67838] Updated weights for policy 0, policy_version 90912 (0.0009) [2023-10-07 23:31:49,088][67871] Updated weights for policy 1, policy_version 91050 (0.0010) [2023-10-07 23:31:49,463][67871] Updated weights for policy 1, policy_version 91060 (0.0007) [2023-10-07 23:31:49,822][67871] Updated weights for policy 1, policy_version 91070 (0.0007) [2023-10-07 23:31:52,242][67838] Updated weights for policy 0, policy_version 90922 (0.0008) [2023-10-07 23:31:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 186351616. Throughput: 0: 1663.3, 1: 1672.4. Samples: 46602768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:31:52,477][66916] Avg episode reward: [(0, '41.070'), (1, '59.890')] [2023-10-07 23:31:52,605][67838] Updated weights for policy 0, policy_version 90932 (0.0009) [2023-10-07 23:31:52,978][67838] Updated weights for policy 0, policy_version 90942 (0.0010) [2023-10-07 23:31:54,003][67871] Updated weights for policy 1, policy_version 91080 (0.0008) [2023-10-07 23:31:54,374][67871] Updated weights for policy 1, policy_version 91090 (0.0007) [2023-10-07 23:31:54,745][67871] Updated weights for policy 1, policy_version 91100 (0.0007) [2023-10-07 23:31:57,192][67838] Updated weights for policy 0, policy_version 90952 (0.0008) [2023-10-07 23:31:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 186417152. Throughput: 0: 1668.6, 1: 1654.0. Samples: 46612178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:31:57,478][66916] Avg episode reward: [(0, '45.910'), (1, '59.650')] [2023-10-07 23:31:57,562][67838] Updated weights for policy 0, policy_version 90962 (0.0008) [2023-10-07 23:31:57,931][67838] Updated weights for policy 0, policy_version 90972 (0.0008) [2023-10-07 23:31:58,825][67871] Updated weights for policy 1, policy_version 91110 (0.0007) [2023-10-07 23:31:59,184][67871] Updated weights for policy 1, policy_version 91120 (0.0009) [2023-10-07 23:31:59,550][67871] Updated weights for policy 1, policy_version 91130 (0.0010) [2023-10-07 23:32:02,035][67838] Updated weights for policy 0, policy_version 90982 (0.0010) [2023-10-07 23:32:02,398][67838] Updated weights for policy 0, policy_version 90992 (0.0007) [2023-10-07 23:32:02,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 186482688. Throughput: 0: 1665.1, 1: 1672.2. Samples: 46632326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:32:02,478][66916] Avg episode reward: [(0, '46.680'), (1, '58.570')] [2023-10-07 23:32:02,783][67838] Updated weights for policy 0, policy_version 91002 (0.0009) [2023-10-07 23:32:03,639][67871] Updated weights for policy 1, policy_version 91140 (0.0009) [2023-10-07 23:32:04,003][67871] Updated weights for policy 1, policy_version 91150 (0.0010) [2023-10-07 23:32:04,365][67871] Updated weights for policy 1, policy_version 91160 (0.0010) [2023-10-07 23:32:06,797][67838] Updated weights for policy 0, policy_version 91012 (0.0009) [2023-10-07 23:32:07,170][67838] Updated weights for policy 0, policy_version 91022 (0.0009) [2023-10-07 23:32:07,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 186548224. Throughput: 0: 1660.1, 1: 1673.9. Samples: 46652684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:32:07,478][66916] Avg episode reward: [(0, '46.540'), (1, '62.600')] [2023-10-07 23:32:07,539][67838] Updated weights for policy 0, policy_version 91032 (0.0008) [2023-10-07 23:32:08,431][67871] Updated weights for policy 1, policy_version 91170 (0.0008) [2023-10-07 23:32:08,801][67871] Updated weights for policy 1, policy_version 91180 (0.0009) [2023-10-07 23:32:09,161][67871] Updated weights for policy 1, policy_version 91190 (0.0011) [2023-10-07 23:32:09,522][67871] Updated weights for policy 1, policy_version 91200 (0.0011) [2023-10-07 23:32:11,709][67838] Updated weights for policy 0, policy_version 91042 (0.0009) [2023-10-07 23:32:12,074][67838] Updated weights for policy 0, policy_version 91052 (0.0009) [2023-10-07 23:32:12,460][67838] Updated weights for policy 0, policy_version 91062 (0.0010) [2023-10-07 23:32:12,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 186613760. Throughput: 0: 1665.4, 1: 1659.0. Samples: 46661978. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:32:12,477][66916] Avg episode reward: [(0, '45.680'), (1, '61.590')] [2023-10-07 23:32:12,826][67838] Updated weights for policy 0, policy_version 91072 (0.0009) [2023-10-07 23:32:13,737][67871] Updated weights for policy 1, policy_version 91210 (0.0007) [2023-10-07 23:32:14,107][67871] Updated weights for policy 1, policy_version 91220 (0.0008) [2023-10-07 23:32:14,473][67871] Updated weights for policy 1, policy_version 91230 (0.0008) [2023-10-07 23:32:16,948][67838] Updated weights for policy 0, policy_version 91082 (0.0008) [2023-10-07 23:32:17,322][67838] Updated weights for policy 0, policy_version 91092 (0.0008) [2023-10-07 23:32:17,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 186679296. Throughput: 0: 1666.3, 1: 1671.8. Samples: 46682454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:32:17,478][66916] Avg episode reward: [(0, '46.530'), (1, '59.650')] [2023-10-07 23:32:17,709][67838] Updated weights for policy 0, policy_version 91102 (0.0008) [2023-10-07 23:32:18,611][67871] Updated weights for policy 1, policy_version 91240 (0.0011) [2023-10-07 23:32:18,986][67871] Updated weights for policy 1, policy_version 91250 (0.0009) [2023-10-07 23:32:19,357][67871] Updated weights for policy 1, policy_version 91260 (0.0007) [2023-10-07 23:32:21,870][67838] Updated weights for policy 0, policy_version 91112 (0.0008) [2023-10-07 23:32:22,252][67838] Updated weights for policy 0, policy_version 91122 (0.0007) [2023-10-07 23:32:22,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 186744832. Throughput: 0: 1649.6, 1: 1672.3. Samples: 46702120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:32:22,477][66916] Avg episode reward: [(0, '46.970'), (1, '59.460')] [2023-10-07 23:32:22,620][67838] Updated weights for policy 0, policy_version 91132 (0.0008) [2023-10-07 23:32:23,454][67871] Updated weights for policy 1, policy_version 91270 (0.0007) [2023-10-07 23:32:23,817][67871] Updated weights for policy 1, policy_version 91280 (0.0008) [2023-10-07 23:32:24,187][67871] Updated weights for policy 1, policy_version 91290 (0.0009) [2023-10-07 23:32:26,691][67838] Updated weights for policy 0, policy_version 91142 (0.0007) [2023-10-07 23:32:27,064][67838] Updated weights for policy 0, policy_version 91152 (0.0007) [2023-10-07 23:32:27,434][67838] Updated weights for policy 0, policy_version 91162 (0.0008) [2023-10-07 23:32:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 186810368. Throughput: 0: 1662.7, 1: 1661.8. Samples: 46711806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:32:27,478][66916] Avg episode reward: [(0, '45.870'), (1, '59.840')] [2023-10-07 23:32:28,249][67871] Updated weights for policy 1, policy_version 91300 (0.0009) [2023-10-07 23:32:28,619][67871] Updated weights for policy 1, policy_version 91310 (0.0008) [2023-10-07 23:32:28,992][67871] Updated weights for policy 1, policy_version 91320 (0.0008) [2023-10-07 23:32:31,639][67838] Updated weights for policy 0, policy_version 91172 (0.0008) [2023-10-07 23:32:32,004][67838] Updated weights for policy 0, policy_version 91182 (0.0008) [2023-10-07 23:32:32,387][67838] Updated weights for policy 0, policy_version 91192 (0.0007) [2023-10-07 23:32:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 186875904. Throughput: 0: 1665.6, 1: 1667.0. Samples: 46732282. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:32:32,477][66916] Avg episode reward: [(0, '47.740'), (1, '58.880')] [2023-10-07 23:32:33,189][67871] Updated weights for policy 1, policy_version 91330 (0.0008) [2023-10-07 23:32:33,562][67871] Updated weights for policy 1, policy_version 91340 (0.0007) [2023-10-07 23:32:33,918][67871] Updated weights for policy 1, policy_version 91350 (0.0007) [2023-10-07 23:32:34,281][67871] Updated weights for policy 1, policy_version 91360 (0.0007) [2023-10-07 23:32:36,384][67838] Updated weights for policy 0, policy_version 91202 (0.0008) [2023-10-07 23:32:36,761][67838] Updated weights for policy 0, policy_version 91212 (0.0008) [2023-10-07 23:32:37,133][67838] Updated weights for policy 0, policy_version 91222 (0.0009) [2023-10-07 23:32:37,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 186941440. Throughput: 0: 1650.4, 1: 1666.6. Samples: 46752034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:32:37,478][66916] Avg episode reward: [(0, '51.220'), (1, '57.650')] [2023-10-07 23:32:37,502][67838] Updated weights for policy 0, policy_version 91232 (0.0011) [2023-10-07 23:32:38,361][67871] Updated weights for policy 1, policy_version 91370 (0.0008) [2023-10-07 23:32:38,740][67871] Updated weights for policy 1, policy_version 91380 (0.0010) [2023-10-07 23:32:39,106][67871] Updated weights for policy 1, policy_version 91390 (0.0009) [2023-10-07 23:32:41,665][67838] Updated weights for policy 0, policy_version 91242 (0.0007) [2023-10-07 23:32:42,029][67838] Updated weights for policy 0, policy_version 91252 (0.0007) [2023-10-07 23:32:42,407][67838] Updated weights for policy 0, policy_version 91262 (0.0007) [2023-10-07 23:32:42,476][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 187039744. Throughput: 0: 1659.9, 1: 1668.0. Samples: 46761932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:32:42,477][66916] Avg episode reward: [(0, '49.800'), (1, '61.220')] [2023-10-07 23:32:43,168][67871] Updated weights for policy 1, policy_version 91400 (0.0009) [2023-10-07 23:32:43,540][67871] Updated weights for policy 1, policy_version 91410 (0.0008) [2023-10-07 23:32:43,897][67871] Updated weights for policy 1, policy_version 91420 (0.0009) [2023-10-07 23:32:46,487][67838] Updated weights for policy 0, policy_version 91272 (0.0008) [2023-10-07 23:32:46,849][67838] Updated weights for policy 0, policy_version 91282 (0.0007) [2023-10-07 23:32:47,222][67838] Updated weights for policy 0, policy_version 91292 (0.0008) [2023-10-07 23:32:47,476][66916] Fps is (10 sec: 16384.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 187105280. Throughput: 0: 1661.1, 1: 1671.9. Samples: 46782312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:32:47,477][66916] Avg episode reward: [(0, '48.930'), (1, '63.470')] [2023-10-07 23:32:48,141][67871] Updated weights for policy 1, policy_version 91430 (0.0008) [2023-10-07 23:32:48,508][67871] Updated weights for policy 1, policy_version 91440 (0.0009) [2023-10-07 23:32:48,869][67871] Updated weights for policy 1, policy_version 91450 (0.0009) [2023-10-07 23:32:51,180][67838] Updated weights for policy 0, policy_version 91302 (0.0010) [2023-10-07 23:32:51,557][67838] Updated weights for policy 0, policy_version 91312 (0.0010) [2023-10-07 23:32:51,927][67838] Updated weights for policy 0, policy_version 91322 (0.0011) [2023-10-07 23:32:52,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 187170816. Throughput: 0: 1647.4, 1: 1663.7. Samples: 46801684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:32:52,477][66916] Avg episode reward: [(0, '51.500'), (1, '59.520')] [2023-10-07 23:32:52,910][67871] Updated weights for policy 1, policy_version 91460 (0.0009) [2023-10-07 23:32:53,281][67871] Updated weights for policy 1, policy_version 91470 (0.0007) [2023-10-07 23:32:53,650][67871] Updated weights for policy 1, policy_version 91480 (0.0007) [2023-10-07 23:32:56,188][67838] Updated weights for policy 0, policy_version 91332 (0.0008) [2023-10-07 23:32:56,558][67838] Updated weights for policy 0, policy_version 91342 (0.0009) [2023-10-07 23:32:56,930][67838] Updated weights for policy 0, policy_version 91352 (0.0009) [2023-10-07 23:32:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 187236352. Throughput: 0: 1663.3, 1: 1663.9. Samples: 46811702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:32:57,478][66916] Avg episode reward: [(0, '47.320'), (1, '60.070')] [2023-10-07 23:32:57,800][67871] Updated weights for policy 1, policy_version 91490 (0.0010) [2023-10-07 23:32:58,157][67871] Updated weights for policy 1, policy_version 91500 (0.0009) [2023-10-07 23:32:58,526][67871] Updated weights for policy 1, policy_version 91510 (0.0008) [2023-10-07 23:32:58,899][67871] Updated weights for policy 1, policy_version 91520 (0.0011) [2023-10-07 23:33:01,010][67838] Updated weights for policy 0, policy_version 91362 (0.0011) [2023-10-07 23:33:01,373][67838] Updated weights for policy 0, policy_version 91372 (0.0008) [2023-10-07 23:33:01,747][67838] Updated weights for policy 0, policy_version 91382 (0.0009) [2023-10-07 23:33:02,118][67838] Updated weights for policy 0, policy_version 91392 (0.0010) [2023-10-07 23:33:02,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 187301888. Throughput: 0: 1658.1, 1: 1668.6. Samples: 46832156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:33:02,477][66916] Avg episode reward: [(0, '49.330'), (1, '60.400')] [2023-10-07 23:33:02,862][67871] Updated weights for policy 1, policy_version 91530 (0.0011) [2023-10-07 23:33:03,218][67871] Updated weights for policy 1, policy_version 91540 (0.0009) [2023-10-07 23:33:03,578][67871] Updated weights for policy 1, policy_version 91550 (0.0010) [2023-10-07 23:33:06,444][67838] Updated weights for policy 0, policy_version 91402 (0.0009) [2023-10-07 23:33:06,809][67838] Updated weights for policy 0, policy_version 91412 (0.0008) [2023-10-07 23:33:07,181][67838] Updated weights for policy 0, policy_version 91422 (0.0008) [2023-10-07 23:33:07,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 187367424. Throughput: 0: 1645.5, 1: 1673.8. Samples: 46851486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:33:07,478][66916] Avg episode reward: [(0, '47.480'), (1, '59.270')] [2023-10-07 23:33:07,488][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000091424_93618176.pth... [2023-10-07 23:33:07,529][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000089856_92012544.pth [2023-10-07 23:33:07,658][67871] Updated weights for policy 1, policy_version 91560 (0.0011) [2023-10-07 23:33:08,029][67871] Updated weights for policy 1, policy_version 91570 (0.0009) [2023-10-07 23:33:08,393][67871] Updated weights for policy 1, policy_version 91580 (0.0007) [2023-10-07 23:33:08,537][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000091584_93782016.pth... [2023-10-07 23:33:08,565][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000090016_92176384.pth [2023-10-07 23:33:11,301][67838] Updated weights for policy 0, policy_version 91432 (0.0008) [2023-10-07 23:33:11,685][67838] Updated weights for policy 0, policy_version 91442 (0.0012) [2023-10-07 23:33:12,049][67838] Updated weights for policy 0, policy_version 91452 (0.0009) [2023-10-07 23:33:12,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 187432960. Throughput: 0: 1653.8, 1: 1671.6. Samples: 46861450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:33:12,478][66916] Avg episode reward: [(0, '48.640'), (1, '60.730')] [2023-10-07 23:33:12,674][67871] Updated weights for policy 1, policy_version 91590 (0.0009) [2023-10-07 23:33:13,047][67871] Updated weights for policy 1, policy_version 91600 (0.0009) [2023-10-07 23:33:13,409][67871] Updated weights for policy 1, policy_version 91610 (0.0009) [2023-10-07 23:33:16,308][67838] Updated weights for policy 0, policy_version 91462 (0.0008) [2023-10-07 23:33:16,669][67838] Updated weights for policy 0, policy_version 91472 (0.0009) [2023-10-07 23:33:17,049][67838] Updated weights for policy 0, policy_version 91482 (0.0007) [2023-10-07 23:33:17,477][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 187498496. Throughput: 0: 1649.1, 1: 1666.9. Samples: 46881500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:33:17,477][66916] Avg episode reward: [(0, '49.720'), (1, '62.140')] [2023-10-07 23:33:17,720][67871] Updated weights for policy 1, policy_version 91620 (0.0009) [2023-10-07 23:33:18,080][67871] Updated weights for policy 1, policy_version 91630 (0.0008) [2023-10-07 23:33:18,456][67871] Updated weights for policy 1, policy_version 91640 (0.0008) [2023-10-07 23:33:20,938][67838] Updated weights for policy 0, policy_version 91492 (0.0009) [2023-10-07 23:33:21,322][67838] Updated weights for policy 0, policy_version 91502 (0.0011) [2023-10-07 23:33:21,690][67838] Updated weights for policy 0, policy_version 91512 (0.0008) [2023-10-07 23:33:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 187564032. Throughput: 0: 1639.8, 1: 1664.7. Samples: 46900736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:33:22,477][66916] Avg episode reward: [(0, '46.540'), (1, '65.130')] [2023-10-07 23:33:22,633][67871] Updated weights for policy 1, policy_version 91650 (0.0007) [2023-10-07 23:33:23,003][67871] Updated weights for policy 1, policy_version 91660 (0.0009) [2023-10-07 23:33:23,370][67871] Updated weights for policy 1, policy_version 91670 (0.0008) [2023-10-07 23:33:23,731][67871] Updated weights for policy 1, policy_version 91680 (0.0007) [2023-10-07 23:33:25,847][67838] Updated weights for policy 0, policy_version 91522 (0.0010) [2023-10-07 23:33:26,208][67838] Updated weights for policy 0, policy_version 91532 (0.0012) [2023-10-07 23:33:26,584][67838] Updated weights for policy 0, policy_version 91542 (0.0011) [2023-10-07 23:33:26,954][67838] Updated weights for policy 0, policy_version 91552 (0.0011) [2023-10-07 23:33:27,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 187629568. Throughput: 0: 1651.3, 1: 1665.4. Samples: 46911186. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-07 23:33:27,477][66916] Avg episode reward: [(0, '46.910'), (1, '64.680')] [2023-10-07 23:33:27,744][67871] Updated weights for policy 1, policy_version 91690 (0.0007) [2023-10-07 23:33:28,114][67871] Updated weights for policy 1, policy_version 91700 (0.0008) [2023-10-07 23:33:28,484][67871] Updated weights for policy 1, policy_version 91710 (0.0007) [2023-10-07 23:33:31,189][67838] Updated weights for policy 0, policy_version 91562 (0.0009) [2023-10-07 23:33:31,571][67838] Updated weights for policy 0, policy_version 91572 (0.0008) [2023-10-07 23:33:31,949][67838] Updated weights for policy 0, policy_version 91582 (0.0010) [2023-10-07 23:33:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 187695104. Throughput: 0: 1643.1, 1: 1669.8. Samples: 46931392. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-07 23:33:32,477][66916] Avg episode reward: [(0, '47.520'), (1, '65.890')] [2023-10-07 23:33:32,545][67871] Updated weights for policy 1, policy_version 91720 (0.0010) [2023-10-07 23:33:32,919][67871] Updated weights for policy 1, policy_version 91730 (0.0010) [2023-10-07 23:33:33,280][67871] Updated weights for policy 1, policy_version 91740 (0.0007) [2023-10-07 23:33:36,247][67838] Updated weights for policy 0, policy_version 91592 (0.0010) [2023-10-07 23:33:36,624][67838] Updated weights for policy 0, policy_version 91602 (0.0009) [2023-10-07 23:33:37,000][67838] Updated weights for policy 0, policy_version 91612 (0.0009) [2023-10-07 23:33:37,292][67871] Updated weights for policy 1, policy_version 91750 (0.0008) [2023-10-07 23:33:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 187760640. Throughput: 0: 1642.5, 1: 1676.8. Samples: 46951056. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-07 23:33:37,477][66916] Avg episode reward: [(0, '45.660'), (1, '67.060')] [2023-10-07 23:33:37,672][67871] Updated weights for policy 1, policy_version 91760 (0.0007) [2023-10-07 23:33:38,045][67871] Updated weights for policy 1, policy_version 91770 (0.0008) [2023-10-07 23:33:41,058][67838] Updated weights for policy 0, policy_version 91622 (0.0010) [2023-10-07 23:33:41,431][67838] Updated weights for policy 0, policy_version 91632 (0.0009) [2023-10-07 23:33:41,805][67838] Updated weights for policy 0, policy_version 91642 (0.0008) [2023-10-07 23:33:42,029][67871] Updated weights for policy 1, policy_version 91780 (0.0007) [2023-10-07 23:33:42,391][67871] Updated weights for policy 1, policy_version 91790 (0.0009) [2023-10-07 23:33:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 187826176. Throughput: 0: 1642.3, 1: 1674.3. Samples: 46960948. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-07 23:33:42,477][66916] Avg episode reward: [(0, '50.840'), (1, '66.000')] [2023-10-07 23:33:42,756][67871] Updated weights for policy 1, policy_version 91800 (0.0010) [2023-10-07 23:33:46,174][67838] Updated weights for policy 0, policy_version 91652 (0.0007) [2023-10-07 23:33:46,546][67838] Updated weights for policy 0, policy_version 91662 (0.0008) [2023-10-07 23:33:46,860][67871] Updated weights for policy 1, policy_version 91810 (0.0010) [2023-10-07 23:33:46,923][67838] Updated weights for policy 0, policy_version 91672 (0.0007) [2023-10-07 23:33:47,219][67871] Updated weights for policy 1, policy_version 91820 (0.0007) [2023-10-07 23:33:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 187891712. Throughput: 0: 1644.7, 1: 1670.3. Samples: 46981328. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-07 23:33:47,477][66916] Avg episode reward: [(0, '47.500'), (1, '60.840')] [2023-10-07 23:33:47,576][67871] Updated weights for policy 1, policy_version 91830 (0.0010) [2023-10-07 23:33:47,951][67871] Updated weights for policy 1, policy_version 91840 (0.0009) [2023-10-07 23:33:50,987][67838] Updated weights for policy 0, policy_version 91682 (0.0009) [2023-10-07 23:33:51,392][67838] Updated weights for policy 0, policy_version 91692 (0.0011) [2023-10-07 23:33:51,762][67838] Updated weights for policy 0, policy_version 91702 (0.0010) [2023-10-07 23:33:52,007][67871] Updated weights for policy 1, policy_version 91850 (0.0007) [2023-10-07 23:33:52,133][67838] Updated weights for policy 0, policy_version 91712 (0.0010) [2023-10-07 23:33:52,376][67871] Updated weights for policy 1, policy_version 91860 (0.0010) [2023-10-07 23:33:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 187957248. Throughput: 0: 1643.9, 1: 1664.4. Samples: 47000358. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-07 23:33:52,477][66916] Avg episode reward: [(0, '52.180'), (1, '59.940')] [2023-10-07 23:33:52,750][67871] Updated weights for policy 1, policy_version 91870 (0.0008) [2023-10-07 23:33:56,279][67838] Updated weights for policy 0, policy_version 91722 (0.0007) [2023-10-07 23:33:56,643][67838] Updated weights for policy 0, policy_version 91732 (0.0007) [2023-10-07 23:33:56,935][67871] Updated weights for policy 1, policy_version 91880 (0.0007) [2023-10-07 23:33:57,010][67838] Updated weights for policy 0, policy_version 91742 (0.0007) [2023-10-07 23:33:57,304][67871] Updated weights for policy 1, policy_version 91890 (0.0008) [2023-10-07 23:33:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 188022784. Throughput: 0: 1647.0, 1: 1667.0. Samples: 47010578. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-07 23:33:57,477][66916] Avg episode reward: [(0, '51.030'), (1, '55.860')] [2023-10-07 23:33:57,665][67871] Updated weights for policy 1, policy_version 91900 (0.0008) [2023-10-07 23:34:01,195][67838] Updated weights for policy 0, policy_version 91752 (0.0007) [2023-10-07 23:34:01,558][67838] Updated weights for policy 0, policy_version 91762 (0.0007) [2023-10-07 23:34:01,866][67871] Updated weights for policy 1, policy_version 91910 (0.0008) [2023-10-07 23:34:01,926][67838] Updated weights for policy 0, policy_version 91772 (0.0008) [2023-10-07 23:34:02,229][67871] Updated weights for policy 1, policy_version 91920 (0.0010) [2023-10-07 23:34:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 188088320. Throughput: 0: 1646.9, 1: 1671.1. Samples: 47030808. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-07 23:34:02,477][66916] Avg episode reward: [(0, '48.370'), (1, '52.600')] [2023-10-07 23:34:02,592][67871] Updated weights for policy 1, policy_version 91930 (0.0011) [2023-10-07 23:34:05,942][67838] Updated weights for policy 0, policy_version 91782 (0.0009) [2023-10-07 23:34:06,313][67838] Updated weights for policy 0, policy_version 91792 (0.0011) [2023-10-07 23:34:06,687][67838] Updated weights for policy 0, policy_version 91802 (0.0009) [2023-10-07 23:34:06,794][67871] Updated weights for policy 1, policy_version 91940 (0.0010) [2023-10-07 23:34:07,163][67871] Updated weights for policy 1, policy_version 91950 (0.0008) [2023-10-07 23:34:07,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 188153856. Throughput: 0: 1649.5, 1: 1667.8. Samples: 47050018. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-07 23:34:07,478][66916] Avg episode reward: [(0, '49.930'), (1, '52.860')] [2023-10-07 23:34:07,533][67871] Updated weights for policy 1, policy_version 91960 (0.0010) [2023-10-07 23:34:10,916][67838] Updated weights for policy 0, policy_version 91812 (0.0007) [2023-10-07 23:34:11,298][67838] Updated weights for policy 0, policy_version 91822 (0.0008) [2023-10-07 23:34:11,660][67838] Updated weights for policy 0, policy_version 91832 (0.0009) [2023-10-07 23:34:11,684][67871] Updated weights for policy 1, policy_version 91970 (0.0008) [2023-10-07 23:34:12,056][67871] Updated weights for policy 1, policy_version 91980 (0.0007) [2023-10-07 23:34:12,429][67871] Updated weights for policy 1, policy_version 91990 (0.0008) [2023-10-07 23:34:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 188219392. Throughput: 0: 1648.8, 1: 1667.6. Samples: 47060422. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-07 23:34:12,477][66916] Avg episode reward: [(0, '49.010'), (1, '56.110')] [2023-10-07 23:34:12,792][67871] Updated weights for policy 1, policy_version 92000 (0.0011) [2023-10-07 23:34:15,867][67838] Updated weights for policy 0, policy_version 91842 (0.0010) [2023-10-07 23:34:16,236][67838] Updated weights for policy 0, policy_version 91852 (0.0011) [2023-10-07 23:34:16,605][67838] Updated weights for policy 0, policy_version 91862 (0.0008) [2023-10-07 23:34:16,907][67871] Updated weights for policy 1, policy_version 92010 (0.0007) [2023-10-07 23:34:16,979][67838] Updated weights for policy 0, policy_version 91872 (0.0008) [2023-10-07 23:34:17,264][67871] Updated weights for policy 1, policy_version 92020 (0.0007) [2023-10-07 23:34:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 188284928. Throughput: 0: 1653.0, 1: 1665.0. Samples: 47080702. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-07 23:34:17,477][66916] Avg episode reward: [(0, '47.500'), (1, '56.220')] [2023-10-07 23:34:17,631][67871] Updated weights for policy 1, policy_version 92030 (0.0007) [2023-10-07 23:34:20,973][67838] Updated weights for policy 0, policy_version 91882 (0.0008) [2023-10-07 23:34:21,339][67838] Updated weights for policy 0, policy_version 91892 (0.0008) [2023-10-07 23:34:21,705][67838] Updated weights for policy 0, policy_version 91902 (0.0008) [2023-10-07 23:34:21,809][67871] Updated weights for policy 1, policy_version 92040 (0.0008) [2023-10-07 23:34:22,185][67871] Updated weights for policy 1, policy_version 92050 (0.0010) [2023-10-07 23:34:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 188350464. Throughput: 0: 1654.8, 1: 1655.1. Samples: 47100002. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-07 23:34:22,477][66916] Avg episode reward: [(0, '48.430'), (1, '62.200')] [2023-10-07 23:34:22,557][67871] Updated weights for policy 1, policy_version 92060 (0.0010) [2023-10-07 23:34:25,870][67838] Updated weights for policy 0, policy_version 91912 (0.0008) [2023-10-07 23:34:26,244][67838] Updated weights for policy 0, policy_version 91922 (0.0008) [2023-10-07 23:34:26,602][67838] Updated weights for policy 0, policy_version 91932 (0.0011) [2023-10-07 23:34:26,620][67871] Updated weights for policy 1, policy_version 92070 (0.0008) [2023-10-07 23:34:26,981][67871] Updated weights for policy 1, policy_version 92080 (0.0007) [2023-10-07 23:34:27,351][67871] Updated weights for policy 1, policy_version 92090 (0.0007) [2023-10-07 23:34:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 188416000. Throughput: 0: 1659.8, 1: 1667.5. Samples: 47110676. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 23:34:27,478][66916] Avg episode reward: [(0, '47.500'), (1, '65.500')] [2023-10-07 23:34:30,750][67838] Updated weights for policy 0, policy_version 91942 (0.0008) [2023-10-07 23:34:31,110][67838] Updated weights for policy 0, policy_version 91952 (0.0008) [2023-10-07 23:34:31,389][67871] Updated weights for policy 1, policy_version 92100 (0.0008) [2023-10-07 23:34:31,487][67838] Updated weights for policy 0, policy_version 91962 (0.0008) [2023-10-07 23:34:31,752][67871] Updated weights for policy 1, policy_version 92110 (0.0010) [2023-10-07 23:34:32,121][67871] Updated weights for policy 1, policy_version 92120 (0.0009) [2023-10-07 23:34:32,477][66916] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 188514304. Throughput: 0: 1645.6, 1: 1669.2. Samples: 47130498. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 23:34:32,478][66916] Avg episode reward: [(0, '48.390'), (1, '64.220')] [2023-10-07 23:34:35,668][67838] Updated weights for policy 0, policy_version 91972 (0.0010) [2023-10-07 23:34:36,031][67838] Updated weights for policy 0, policy_version 91982 (0.0011) [2023-10-07 23:34:36,255][67871] Updated weights for policy 1, policy_version 92130 (0.0009) [2023-10-07 23:34:36,406][67838] Updated weights for policy 0, policy_version 91992 (0.0007) [2023-10-07 23:34:36,621][67871] Updated weights for policy 1, policy_version 92140 (0.0007) [2023-10-07 23:34:36,983][67871] Updated weights for policy 1, policy_version 92150 (0.0009) [2023-10-07 23:34:37,356][67871] Updated weights for policy 1, policy_version 92160 (0.0010) [2023-10-07 23:34:37,476][66916] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 188579840. Throughput: 0: 1656.3, 1: 1663.5. Samples: 47149750. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 23:34:37,477][66916] Avg episode reward: [(0, '49.380'), (1, '65.990')] [2023-10-07 23:34:40,427][67838] Updated weights for policy 0, policy_version 92002 (0.0007) [2023-10-07 23:34:40,821][67838] Updated weights for policy 0, policy_version 92012 (0.0009) [2023-10-07 23:34:41,198][67838] Updated weights for policy 0, policy_version 92022 (0.0009) [2023-10-07 23:34:41,473][67871] Updated weights for policy 1, policy_version 92170 (0.0008) [2023-10-07 23:34:41,568][67838] Updated weights for policy 0, policy_version 92032 (0.0007) [2023-10-07 23:34:41,844][67871] Updated weights for policy 1, policy_version 92180 (0.0008) [2023-10-07 23:34:42,212][67871] Updated weights for policy 1, policy_version 92190 (0.0009) [2023-10-07 23:34:42,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 188645376. Throughput: 0: 1661.9, 1: 1674.4. Samples: 47160708. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 23:34:42,477][66916] Avg episode reward: [(0, '46.590'), (1, '66.780')] [2023-10-07 23:34:45,621][67838] Updated weights for policy 0, policy_version 92042 (0.0010) [2023-10-07 23:34:45,983][67838] Updated weights for policy 0, policy_version 92052 (0.0009) [2023-10-07 23:34:46,363][67838] Updated weights for policy 0, policy_version 92062 (0.0008) [2023-10-07 23:34:46,410][67871] Updated weights for policy 1, policy_version 92200 (0.0009) [2023-10-07 23:34:46,776][67871] Updated weights for policy 1, policy_version 92210 (0.0007) [2023-10-07 23:34:47,143][67871] Updated weights for policy 1, policy_version 92220 (0.0007) [2023-10-07 23:34:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 188710912. Throughput: 0: 1645.4, 1: 1669.1. Samples: 47179960. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 23:34:47,477][66916] Avg episode reward: [(0, '52.180'), (1, '60.600')] [2023-10-07 23:34:50,519][67838] Updated weights for policy 0, policy_version 92072 (0.0009) [2023-10-07 23:34:50,889][67838] Updated weights for policy 0, policy_version 92082 (0.0011) [2023-10-07 23:34:51,162][67871] Updated weights for policy 1, policy_version 92230 (0.0008) [2023-10-07 23:34:51,262][67838] Updated weights for policy 0, policy_version 92092 (0.0009) [2023-10-07 23:34:51,535][67871] Updated weights for policy 1, policy_version 92240 (0.0008) [2023-10-07 23:34:51,902][67871] Updated weights for policy 1, policy_version 92250 (0.0008) [2023-10-07 23:34:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 188776448. Throughput: 0: 1658.5, 1: 1657.9. Samples: 47199258. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 23:34:52,478][66916] Avg episode reward: [(0, '51.440'), (1, '57.170')] [2023-10-07 23:34:55,430][67838] Updated weights for policy 0, policy_version 92102 (0.0009) [2023-10-07 23:34:55,798][67838] Updated weights for policy 0, policy_version 92112 (0.0009) [2023-10-07 23:34:55,967][67871] Updated weights for policy 1, policy_version 92260 (0.0008) [2023-10-07 23:34:56,171][67838] Updated weights for policy 0, policy_version 92122 (0.0009) [2023-10-07 23:34:56,341][67871] Updated weights for policy 1, policy_version 92270 (0.0009) [2023-10-07 23:34:56,713][67871] Updated weights for policy 1, policy_version 92280 (0.0010) [2023-10-07 23:34:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 188841984. Throughput: 0: 1657.6, 1: 1677.8. Samples: 47210512. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 23:34:57,477][66916] Avg episode reward: [(0, '49.460'), (1, '56.580')] [2023-10-07 23:35:00,274][67838] Updated weights for policy 0, policy_version 92132 (0.0008) [2023-10-07 23:35:00,636][67838] Updated weights for policy 0, policy_version 92142 (0.0010) [2023-10-07 23:35:00,743][67871] Updated weights for policy 1, policy_version 92290 (0.0009) [2023-10-07 23:35:01,006][67838] Updated weights for policy 0, policy_version 92152 (0.0008) [2023-10-07 23:35:01,106][67871] Updated weights for policy 1, policy_version 92300 (0.0010) [2023-10-07 23:35:01,478][67871] Updated weights for policy 1, policy_version 92310 (0.0010) [2023-10-07 23:35:01,835][67871] Updated weights for policy 1, policy_version 92320 (0.0010) [2023-10-07 23:35:02,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 188907520. Throughput: 0: 1647.2, 1: 1673.1. Samples: 47230118. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 23:35:02,478][66916] Avg episode reward: [(0, '49.150'), (1, '58.330')] [2023-10-07 23:35:05,131][67838] Updated weights for policy 0, policy_version 92162 (0.0007) [2023-10-07 23:35:05,490][67838] Updated weights for policy 0, policy_version 92172 (0.0010) [2023-10-07 23:35:05,857][67838] Updated weights for policy 0, policy_version 92182 (0.0007) [2023-10-07 23:35:05,970][67871] Updated weights for policy 1, policy_version 92330 (0.0007) [2023-10-07 23:35:06,218][67838] Updated weights for policy 0, policy_version 92192 (0.0007) [2023-10-07 23:35:06,333][67871] Updated weights for policy 1, policy_version 92340 (0.0007) [2023-10-07 23:35:06,702][67871] Updated weights for policy 1, policy_version 92350 (0.0008) [2023-10-07 23:35:07,477][66916] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 188973056. Throughput: 0: 1662.2, 1: 1653.6. Samples: 47249212. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 23:35:07,478][66916] Avg episode reward: [(0, '51.830'), (1, '58.690')] [2023-10-07 23:35:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000092192_94404608.pth... [2023-10-07 23:35:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000092352_94568448.pth... [2023-10-07 23:35:07,518][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000090624_92798976.pth [2023-10-07 23:35:07,530][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000090784_92962816.pth [2023-10-07 23:35:10,378][67838] Updated weights for policy 0, policy_version 92202 (0.0008) [2023-10-07 23:35:10,746][67838] Updated weights for policy 0, policy_version 92212 (0.0009) [2023-10-07 23:35:11,003][67871] Updated weights for policy 1, policy_version 92360 (0.0010) [2023-10-07 23:35:11,112][67838] Updated weights for policy 0, policy_version 92222 (0.0007) [2023-10-07 23:35:11,380][67871] Updated weights for policy 1, policy_version 92370 (0.0009) [2023-10-07 23:35:11,745][67871] Updated weights for policy 1, policy_version 92380 (0.0009) [2023-10-07 23:35:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 189038592. Throughput: 0: 1658.8, 1: 1672.8. Samples: 47260598. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 23:35:12,478][66916] Avg episode reward: [(0, '49.670'), (1, '60.110')] [2023-10-07 23:35:15,354][67838] Updated weights for policy 0, policy_version 92232 (0.0008) [2023-10-07 23:35:15,725][67838] Updated weights for policy 0, policy_version 92242 (0.0007) [2023-10-07 23:35:15,915][67871] Updated weights for policy 1, policy_version 92390 (0.0008) [2023-10-07 23:35:16,099][67838] Updated weights for policy 0, policy_version 92252 (0.0008) [2023-10-07 23:35:16,278][67871] Updated weights for policy 1, policy_version 92400 (0.0007) [2023-10-07 23:35:16,647][67871] Updated weights for policy 1, policy_version 92410 (0.0007) [2023-10-07 23:35:17,476][66916] Fps is (10 sec: 13107.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 189104128. Throughput: 0: 1653.6, 1: 1657.7. Samples: 47279506. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 23:35:17,477][66916] Avg episode reward: [(0, '50.090'), (1, '62.010')] [2023-10-07 23:35:20,162][67838] Updated weights for policy 0, policy_version 92262 (0.0008) [2023-10-07 23:35:20,536][67838] Updated weights for policy 0, policy_version 92272 (0.0009) [2023-10-07 23:35:20,547][67871] Updated weights for policy 1, policy_version 92420 (0.0007) [2023-10-07 23:35:20,901][67838] Updated weights for policy 0, policy_version 92282 (0.0008) [2023-10-07 23:35:20,908][67871] Updated weights for policy 1, policy_version 92430 (0.0007) [2023-10-07 23:35:21,276][67871] Updated weights for policy 1, policy_version 92440 (0.0010) [2023-10-07 23:35:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 189169664. Throughput: 0: 1664.1, 1: 1649.9. Samples: 47298882. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-07 23:35:22,478][66916] Avg episode reward: [(0, '50.420'), (1, '64.970')] [2023-10-07 23:35:25,030][67838] Updated weights for policy 0, policy_version 92292 (0.0009) [2023-10-07 23:35:25,280][67871] Updated weights for policy 1, policy_version 92450 (0.0010) [2023-10-07 23:35:25,419][67838] Updated weights for policy 0, policy_version 92302 (0.0008) [2023-10-07 23:35:25,645][67871] Updated weights for policy 1, policy_version 92460 (0.0009) [2023-10-07 23:35:25,785][67838] Updated weights for policy 0, policy_version 92312 (0.0008) [2023-10-07 23:35:26,021][67871] Updated weights for policy 1, policy_version 92470 (0.0010) [2023-10-07 23:35:26,394][67871] Updated weights for policy 1, policy_version 92480 (0.0009) [2023-10-07 23:35:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 189235200. Throughput: 0: 1653.3, 1: 1665.5. Samples: 47310054. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 23:35:27,477][66916] Avg episode reward: [(0, '48.170'), (1, '68.880')] [2023-10-07 23:35:29,835][67838] Updated weights for policy 0, policy_version 92322 (0.0009) [2023-10-07 23:35:30,215][67838] Updated weights for policy 0, policy_version 92332 (0.0009) [2023-10-07 23:35:30,476][67871] Updated weights for policy 1, policy_version 92490 (0.0010) [2023-10-07 23:35:30,580][67838] Updated weights for policy 0, policy_version 92342 (0.0009) [2023-10-07 23:35:30,841][67871] Updated weights for policy 1, policy_version 92500 (0.0008) [2023-10-07 23:35:30,951][67838] Updated weights for policy 0, policy_version 92352 (0.0009) [2023-10-07 23:35:31,208][67871] Updated weights for policy 1, policy_version 92510 (0.0007) [2023-10-07 23:35:32,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189300736. Throughput: 0: 1647.2, 1: 1658.8. Samples: 47328730. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 23:35:32,477][66916] Avg episode reward: [(0, '44.030'), (1, '66.010')] [2023-10-07 23:35:35,212][67838] Updated weights for policy 0, policy_version 92362 (0.0009) [2023-10-07 23:35:35,354][67871] Updated weights for policy 1, policy_version 92520 (0.0008) [2023-10-07 23:35:35,589][67838] Updated weights for policy 0, policy_version 92372 (0.0008) [2023-10-07 23:35:35,714][67871] Updated weights for policy 1, policy_version 92530 (0.0009) [2023-10-07 23:35:35,950][67838] Updated weights for policy 0, policy_version 92382 (0.0009) [2023-10-07 23:35:36,073][67871] Updated weights for policy 1, policy_version 92540 (0.0008) [2023-10-07 23:35:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189366272. Throughput: 0: 1656.0, 1: 1664.2. Samples: 47348666. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 23:35:37,477][66916] Avg episode reward: [(0, '44.130'), (1, '67.020')] [2023-10-07 23:35:40,023][67838] Updated weights for policy 0, policy_version 92392 (0.0008) [2023-10-07 23:35:40,039][67871] Updated weights for policy 1, policy_version 92550 (0.0009) [2023-10-07 23:35:40,385][67838] Updated weights for policy 0, policy_version 92402 (0.0010) [2023-10-07 23:35:40,405][67871] Updated weights for policy 1, policy_version 92560 (0.0008) [2023-10-07 23:35:40,759][67838] Updated weights for policy 0, policy_version 92412 (0.0010) [2023-10-07 23:35:40,773][67871] Updated weights for policy 1, policy_version 92570 (0.0008) [2023-10-07 23:35:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189431808. Throughput: 0: 1650.6, 1: 1669.5. Samples: 47359918. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 23:35:42,478][66916] Avg episode reward: [(0, '44.480'), (1, '64.700')] [2023-10-07 23:35:44,918][67871] Updated weights for policy 1, policy_version 92580 (0.0009) [2023-10-07 23:35:45,150][67838] Updated weights for policy 0, policy_version 92422 (0.0008) [2023-10-07 23:35:45,287][67871] Updated weights for policy 1, policy_version 92590 (0.0007) [2023-10-07 23:35:45,521][67838] Updated weights for policy 0, policy_version 92432 (0.0007) [2023-10-07 23:35:45,642][67871] Updated weights for policy 1, policy_version 92600 (0.0008) [2023-10-07 23:35:45,887][67838] Updated weights for policy 0, policy_version 92442 (0.0007) [2023-10-07 23:35:47,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189497344. Throughput: 0: 1643.2, 1: 1651.1. Samples: 47378360. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 23:35:47,477][66916] Avg episode reward: [(0, '45.680'), (1, '68.520')] [2023-10-07 23:35:49,876][67871] Updated weights for policy 1, policy_version 92610 (0.0009) [2023-10-07 23:35:50,096][67838] Updated weights for policy 0, policy_version 92452 (0.0008) [2023-10-07 23:35:50,247][67871] Updated weights for policy 1, policy_version 92620 (0.0008) [2023-10-07 23:35:50,464][67838] Updated weights for policy 0, policy_version 92462 (0.0010) [2023-10-07 23:35:50,617][67871] Updated weights for policy 1, policy_version 92630 (0.0008) [2023-10-07 23:35:50,839][67838] Updated weights for policy 0, policy_version 92472 (0.0008) [2023-10-07 23:35:50,978][67871] Updated weights for policy 1, policy_version 92640 (0.0009) [2023-10-07 23:35:52,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189562880. Throughput: 0: 1641.0, 1: 1672.9. Samples: 47398336. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 23:35:52,477][66916] Avg episode reward: [(0, '48.090'), (1, '65.370')] [2023-10-07 23:35:55,017][67838] Updated weights for policy 0, policy_version 92482 (0.0008) [2023-10-07 23:35:55,106][67871] Updated weights for policy 1, policy_version 92650 (0.0008) [2023-10-07 23:35:55,387][67838] Updated weights for policy 0, policy_version 92492 (0.0008) [2023-10-07 23:35:55,475][67871] Updated weights for policy 1, policy_version 92660 (0.0010) [2023-10-07 23:35:55,751][67838] Updated weights for policy 0, policy_version 92502 (0.0008) [2023-10-07 23:35:55,838][67871] Updated weights for policy 1, policy_version 92670 (0.0008) [2023-10-07 23:35:56,127][67838] Updated weights for policy 0, policy_version 92512 (0.0008) [2023-10-07 23:35:57,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189628416. Throughput: 0: 1639.8, 1: 1667.8. Samples: 47409440. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 23:35:57,477][66916] Avg episode reward: [(0, '48.230'), (1, '62.450')] [2023-10-07 23:35:59,939][67871] Updated weights for policy 1, policy_version 92680 (0.0007) [2023-10-07 23:36:00,276][67838] Updated weights for policy 0, policy_version 92522 (0.0008) [2023-10-07 23:36:00,311][67871] Updated weights for policy 1, policy_version 92690 (0.0007) [2023-10-07 23:36:00,638][67838] Updated weights for policy 0, policy_version 92532 (0.0008) [2023-10-07 23:36:00,668][67871] Updated weights for policy 1, policy_version 92700 (0.0007) [2023-10-07 23:36:01,007][67838] Updated weights for policy 0, policy_version 92542 (0.0009) [2023-10-07 23:36:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189693952. Throughput: 0: 1639.4, 1: 1658.2. Samples: 47427898. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 23:36:02,477][66916] Avg episode reward: [(0, '43.580'), (1, '62.750')] [2023-10-07 23:36:04,766][67871] Updated weights for policy 1, policy_version 92710 (0.0009) [2023-10-07 23:36:05,041][67838] Updated weights for policy 0, policy_version 92552 (0.0009) [2023-10-07 23:36:05,135][67871] Updated weights for policy 1, policy_version 92720 (0.0007) [2023-10-07 23:36:05,409][67838] Updated weights for policy 0, policy_version 92562 (0.0008) [2023-10-07 23:36:05,499][67871] Updated weights for policy 1, policy_version 92730 (0.0007) [2023-10-07 23:36:05,766][67838] Updated weights for policy 0, policy_version 92572 (0.0007) [2023-10-07 23:36:07,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 189759488. Throughput: 0: 1645.7, 1: 1675.3. Samples: 47448322. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 23:36:07,477][66916] Avg episode reward: [(0, '43.210'), (1, '66.480')] [2023-10-07 23:36:09,592][67871] Updated weights for policy 1, policy_version 92740 (0.0008) [2023-10-07 23:36:09,938][67838] Updated weights for policy 0, policy_version 92582 (0.0008) [2023-10-07 23:36:09,956][67871] Updated weights for policy 1, policy_version 92750 (0.0008) [2023-10-07 23:36:10,317][67871] Updated weights for policy 1, policy_version 92760 (0.0008) [2023-10-07 23:36:10,319][67838] Updated weights for policy 0, policy_version 92592 (0.0009) [2023-10-07 23:36:10,687][67838] Updated weights for policy 0, policy_version 92602 (0.0008) [2023-10-07 23:36:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189825024. Throughput: 0: 1642.6, 1: 1668.8. Samples: 47459070. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 23:36:12,477][66916] Avg episode reward: [(0, '41.580'), (1, '67.630')] [2023-10-07 23:36:14,463][67871] Updated weights for policy 1, policy_version 92770 (0.0009) [2023-10-07 23:36:14,787][67838] Updated weights for policy 0, policy_version 92612 (0.0008) [2023-10-07 23:36:14,823][67871] Updated weights for policy 1, policy_version 92780 (0.0008) [2023-10-07 23:36:15,155][67838] Updated weights for policy 0, policy_version 92622 (0.0009) [2023-10-07 23:36:15,193][67871] Updated weights for policy 1, policy_version 92790 (0.0007) [2023-10-07 23:36:15,522][67838] Updated weights for policy 0, policy_version 92632 (0.0009) [2023-10-07 23:36:15,560][67871] Updated weights for policy 1, policy_version 92800 (0.0008) [2023-10-07 23:36:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189890560. Throughput: 0: 1647.7, 1: 1662.5. Samples: 47477690. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 23:36:17,477][66916] Avg episode reward: [(0, '44.450'), (1, '64.340')] [2023-10-07 23:36:19,606][67838] Updated weights for policy 0, policy_version 92642 (0.0009) [2023-10-07 23:36:19,620][67871] Updated weights for policy 1, policy_version 92810 (0.0007) [2023-10-07 23:36:19,976][67871] Updated weights for policy 1, policy_version 92820 (0.0007) [2023-10-07 23:36:19,979][67838] Updated weights for policy 0, policy_version 92652 (0.0007) [2023-10-07 23:36:20,343][67838] Updated weights for policy 0, policy_version 92662 (0.0007) [2023-10-07 23:36:20,347][67871] Updated weights for policy 1, policy_version 92830 (0.0008) [2023-10-07 23:36:20,722][67838] Updated weights for policy 0, policy_version 92672 (0.0008) [2023-10-07 23:36:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 189956096. Throughput: 0: 1649.8, 1: 1674.4. Samples: 47498254. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-07 23:36:22,478][66916] Avg episode reward: [(0, '45.410'), (1, '68.110')] [2023-10-07 23:36:24,538][67871] Updated weights for policy 1, policy_version 92840 (0.0009) [2023-10-07 23:36:24,900][67871] Updated weights for policy 1, policy_version 92850 (0.0009) [2023-10-07 23:36:24,970][67838] Updated weights for policy 0, policy_version 92682 (0.0007) [2023-10-07 23:36:25,270][67871] Updated weights for policy 1, policy_version 92860 (0.0008) [2023-10-07 23:36:25,346][67838] Updated weights for policy 0, policy_version 92692 (0.0008) [2023-10-07 23:36:25,719][67838] Updated weights for policy 0, policy_version 92702 (0.0010) [2023-10-07 23:36:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 190021632. Throughput: 0: 1646.6, 1: 1659.2. Samples: 47508676. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 23:36:27,478][66916] Avg episode reward: [(0, '46.900'), (1, '67.940')] [2023-10-07 23:36:29,292][67871] Updated weights for policy 1, policy_version 92870 (0.0009) [2023-10-07 23:36:29,669][67871] Updated weights for policy 1, policy_version 92880 (0.0008) [2023-10-07 23:36:29,809][67838] Updated weights for policy 0, policy_version 92712 (0.0011) [2023-10-07 23:36:30,032][67871] Updated weights for policy 1, policy_version 92890 (0.0009) [2023-10-07 23:36:30,182][67838] Updated weights for policy 0, policy_version 92722 (0.0008) [2023-10-07 23:36:30,557][67838] Updated weights for policy 0, policy_version 92732 (0.0009) [2023-10-07 23:36:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 190087168. Throughput: 0: 1650.1, 1: 1665.9. Samples: 47527578. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 23:36:32,477][66916] Avg episode reward: [(0, '47.910'), (1, '65.730')] [2023-10-07 23:36:34,168][67871] Updated weights for policy 1, policy_version 92900 (0.0008) [2023-10-07 23:36:34,540][67871] Updated weights for policy 1, policy_version 92910 (0.0009) [2023-10-07 23:36:34,896][67838] Updated weights for policy 0, policy_version 92742 (0.0007) [2023-10-07 23:36:34,904][67871] Updated weights for policy 1, policy_version 92920 (0.0007) [2023-10-07 23:36:35,267][67838] Updated weights for policy 0, policy_version 92752 (0.0008) [2023-10-07 23:36:35,625][67838] Updated weights for policy 0, policy_version 92762 (0.0009) [2023-10-07 23:36:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 190152704. Throughput: 0: 1656.4, 1: 1671.7. Samples: 47548104. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 23:36:37,477][66916] Avg episode reward: [(0, '47.220'), (1, '63.200')] [2023-10-07 23:36:38,896][67871] Updated weights for policy 1, policy_version 92930 (0.0008) [2023-10-07 23:36:39,260][67871] Updated weights for policy 1, policy_version 92940 (0.0008) [2023-10-07 23:36:39,625][67871] Updated weights for policy 1, policy_version 92950 (0.0008) [2023-10-07 23:36:39,814][67838] Updated weights for policy 0, policy_version 92772 (0.0008) [2023-10-07 23:36:39,993][67871] Updated weights for policy 1, policy_version 92960 (0.0009) [2023-10-07 23:36:40,176][67838] Updated weights for policy 0, policy_version 92782 (0.0009) [2023-10-07 23:36:40,544][67838] Updated weights for policy 0, policy_version 92792 (0.0010) [2023-10-07 23:36:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 190218240. Throughput: 0: 1654.4, 1: 1649.1. Samples: 47558098. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 23:36:42,478][66916] Avg episode reward: [(0, '48.730'), (1, '64.870')] [2023-10-07 23:36:44,068][67871] Updated weights for policy 1, policy_version 92970 (0.0008) [2023-10-07 23:36:44,426][67871] Updated weights for policy 1, policy_version 92980 (0.0008) [2023-10-07 23:36:44,576][67838] Updated weights for policy 0, policy_version 92802 (0.0009) [2023-10-07 23:36:44,802][67871] Updated weights for policy 1, policy_version 92990 (0.0008) [2023-10-07 23:36:44,953][67838] Updated weights for policy 0, policy_version 92812 (0.0010) [2023-10-07 23:36:45,323][67838] Updated weights for policy 0, policy_version 92822 (0.0009) [2023-10-07 23:36:45,697][67838] Updated weights for policy 0, policy_version 92832 (0.0009) [2023-10-07 23:36:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 190283776. Throughput: 0: 1654.0, 1: 1670.9. Samples: 47577518. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 23:36:47,478][66916] Avg episode reward: [(0, '46.060'), (1, '62.140')] [2023-10-07 23:36:49,272][67871] Updated weights for policy 1, policy_version 93000 (0.0007) [2023-10-07 23:36:49,647][67871] Updated weights for policy 1, policy_version 93010 (0.0007) [2023-10-07 23:36:49,897][67838] Updated weights for policy 0, policy_version 92842 (0.0007) [2023-10-07 23:36:50,018][67871] Updated weights for policy 1, policy_version 93020 (0.0007) [2023-10-07 23:36:50,261][67838] Updated weights for policy 0, policy_version 92852 (0.0010) [2023-10-07 23:36:50,634][67838] Updated weights for policy 0, policy_version 92862 (0.0011) [2023-10-07 23:36:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 190349312. Throughput: 0: 1653.2, 1: 1671.2. Samples: 47597922. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 23:36:52,477][66916] Avg episode reward: [(0, '46.880'), (1, '63.290')] [2023-10-07 23:36:54,042][67871] Updated weights for policy 1, policy_version 93030 (0.0008) [2023-10-07 23:36:54,404][67871] Updated weights for policy 1, policy_version 93040 (0.0007) [2023-10-07 23:36:54,650][67838] Updated weights for policy 0, policy_version 92872 (0.0008) [2023-10-07 23:36:54,772][67871] Updated weights for policy 1, policy_version 93050 (0.0007) [2023-10-07 23:36:55,027][67838] Updated weights for policy 0, policy_version 92882 (0.0007) [2023-10-07 23:36:55,387][67838] Updated weights for policy 0, policy_version 92892 (0.0009) [2023-10-07 23:36:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 190414848. Throughput: 0: 1649.5, 1: 1653.6. Samples: 47607714. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 23:36:57,478][66916] Avg episode reward: [(0, '46.670'), (1, '67.510')] [2023-10-07 23:36:58,921][67871] Updated weights for policy 1, policy_version 93060 (0.0007) [2023-10-07 23:36:59,293][67871] Updated weights for policy 1, policy_version 93070 (0.0010) [2023-10-07 23:36:59,405][67838] Updated weights for policy 0, policy_version 92902 (0.0009) [2023-10-07 23:36:59,656][67871] Updated weights for policy 1, policy_version 93080 (0.0007) [2023-10-07 23:36:59,781][67838] Updated weights for policy 0, policy_version 92912 (0.0008) [2023-10-07 23:37:00,159][67838] Updated weights for policy 0, policy_version 92922 (0.0009) [2023-10-07 23:37:02,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 190480384. Throughput: 0: 1661.6, 1: 1668.0. Samples: 47627520. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 23:37:02,477][66916] Avg episode reward: [(0, '48.600'), (1, '67.810')] [2023-10-07 23:37:03,660][67871] Updated weights for policy 1, policy_version 93090 (0.0008) [2023-10-07 23:37:04,029][67871] Updated weights for policy 1, policy_version 93100 (0.0008) [2023-10-07 23:37:04,229][67838] Updated weights for policy 0, policy_version 92932 (0.0009) [2023-10-07 23:37:04,387][67871] Updated weights for policy 1, policy_version 93110 (0.0009) [2023-10-07 23:37:04,599][67838] Updated weights for policy 0, policy_version 92942 (0.0008) [2023-10-07 23:37:04,749][67871] Updated weights for policy 1, policy_version 93120 (0.0010) [2023-10-07 23:37:04,969][67838] Updated weights for policy 0, policy_version 92952 (0.0008) [2023-10-07 23:37:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 190545920. Throughput: 0: 1660.1, 1: 1671.8. Samples: 47648188. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 23:37:07,478][66916] Avg episode reward: [(0, '47.080'), (1, '67.250')] [2023-10-07 23:37:07,492][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000093120_95354880.pth... [2023-10-07 23:37:07,492][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000092960_95191040.pth... [2023-10-07 23:37:07,527][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000091584_93782016.pth [2023-10-07 23:37:07,531][67676] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p1/milestones/checkpoint_000093120_95354880.pth [2023-10-07 23:37:07,533][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000091424_93618176.pth [2023-10-07 23:37:07,537][67511] Saving a milestone ./train_atari/atari_alien_APPO/checkpoint_p0/milestones/checkpoint_000092960_95191040.pth [2023-10-07 23:37:08,704][67871] Updated weights for policy 1, policy_version 93130 (0.0009) [2023-10-07 23:37:09,065][67871] Updated weights for policy 1, policy_version 93140 (0.0010) [2023-10-07 23:37:09,123][67838] Updated weights for policy 0, policy_version 92962 (0.0007) [2023-10-07 23:37:09,434][67871] Updated weights for policy 1, policy_version 93150 (0.0008) [2023-10-07 23:37:09,495][67838] Updated weights for policy 0, policy_version 92972 (0.0007) [2023-10-07 23:37:09,868][67838] Updated weights for policy 0, policy_version 92982 (0.0008) [2023-10-07 23:37:10,243][67838] Updated weights for policy 0, policy_version 92992 (0.0008) [2023-10-07 23:37:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 190611456. Throughput: 0: 1647.1, 1: 1656.5. Samples: 47657336. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 23:37:12,478][66916] Avg episode reward: [(0, '47.400'), (1, '66.230')] [2023-10-07 23:37:13,544][67871] Updated weights for policy 1, policy_version 93160 (0.0008) [2023-10-07 23:37:13,902][67871] Updated weights for policy 1, policy_version 93170 (0.0008) [2023-10-07 23:37:14,266][67871] Updated weights for policy 1, policy_version 93180 (0.0007) [2023-10-07 23:37:14,489][67838] Updated weights for policy 0, policy_version 93002 (0.0007) [2023-10-07 23:37:14,865][67838] Updated weights for policy 0, policy_version 93012 (0.0008) [2023-10-07 23:37:15,236][67838] Updated weights for policy 0, policy_version 93022 (0.0007) [2023-10-07 23:37:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 190676992. Throughput: 0: 1662.7, 1: 1671.4. Samples: 47677612. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 23:37:17,477][66916] Avg episode reward: [(0, '46.300'), (1, '67.260')] [2023-10-07 23:37:18,529][67871] Updated weights for policy 1, policy_version 93190 (0.0007) [2023-10-07 23:37:18,901][67871] Updated weights for policy 1, policy_version 93200 (0.0008) [2023-10-07 23:37:19,240][67838] Updated weights for policy 0, policy_version 93032 (0.0009) [2023-10-07 23:37:19,265][67871] Updated weights for policy 1, policy_version 93210 (0.0008) [2023-10-07 23:37:19,611][67838] Updated weights for policy 0, policy_version 93042 (0.0008) [2023-10-07 23:37:19,980][67838] Updated weights for policy 0, policy_version 93052 (0.0007) [2023-10-07 23:37:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 190742528. Throughput: 0: 1664.0, 1: 1670.0. Samples: 47698132. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-07 23:37:22,477][66916] Avg episode reward: [(0, '48.110'), (1, '67.450')] [2023-10-07 23:37:23,290][67871] Updated weights for policy 1, policy_version 93220 (0.0007) [2023-10-07 23:37:23,660][67871] Updated weights for policy 1, policy_version 93230 (0.0008) [2023-10-07 23:37:24,035][67871] Updated weights for policy 1, policy_version 93240 (0.0009) [2023-10-07 23:37:24,140][67838] Updated weights for policy 0, policy_version 93062 (0.0010) [2023-10-07 23:37:24,517][67838] Updated weights for policy 0, policy_version 93072 (0.0008) [2023-10-07 23:37:24,886][67838] Updated weights for policy 0, policy_version 93082 (0.0007) [2023-10-07 23:37:27,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 190808064. Throughput: 0: 1645.3, 1: 1667.6. Samples: 47707178. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-07 23:37:27,478][66916] Avg episode reward: [(0, '48.570'), (1, '61.650')] [2023-10-07 23:37:27,997][67871] Updated weights for policy 1, policy_version 93250 (0.0008) [2023-10-07 23:37:28,373][67871] Updated weights for policy 1, policy_version 93260 (0.0010) [2023-10-07 23:37:28,733][67871] Updated weights for policy 1, policy_version 93270 (0.0008) [2023-10-07 23:37:28,971][67838] Updated weights for policy 0, policy_version 93092 (0.0008) [2023-10-07 23:37:29,099][67871] Updated weights for policy 1, policy_version 93280 (0.0008) [2023-10-07 23:37:29,348][67838] Updated weights for policy 0, policy_version 93102 (0.0009) [2023-10-07 23:37:29,726][67838] Updated weights for policy 0, policy_version 93112 (0.0008) [2023-10-07 23:37:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 190873600. Throughput: 0: 1662.9, 1: 1670.1. Samples: 47727504. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-07 23:37:32,478][66916] Avg episode reward: [(0, '49.820'), (1, '63.410')] [2023-10-07 23:37:33,189][67871] Updated weights for policy 1, policy_version 93290 (0.0008) [2023-10-07 23:37:33,549][67871] Updated weights for policy 1, policy_version 93300 (0.0007) [2023-10-07 23:37:33,917][67871] Updated weights for policy 1, policy_version 93310 (0.0007) [2023-10-07 23:37:33,918][67838] Updated weights for policy 0, policy_version 93122 (0.0008) [2023-10-07 23:37:34,300][67838] Updated weights for policy 0, policy_version 93132 (0.0008) [2023-10-07 23:37:34,665][67838] Updated weights for policy 0, policy_version 93142 (0.0007) [2023-10-07 23:37:35,031][67838] Updated weights for policy 0, policy_version 93152 (0.0008) [2023-10-07 23:37:37,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 190939136. Throughput: 0: 1660.3, 1: 1674.7. Samples: 47747994. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-07 23:37:37,478][66916] Avg episode reward: [(0, '50.570'), (1, '64.300')] [2023-10-07 23:37:38,035][67871] Updated weights for policy 1, policy_version 93320 (0.0009) [2023-10-07 23:37:38,404][67871] Updated weights for policy 1, policy_version 93330 (0.0009) [2023-10-07 23:37:38,769][67871] Updated weights for policy 1, policy_version 93340 (0.0008) [2023-10-07 23:37:39,204][67838] Updated weights for policy 0, policy_version 93162 (0.0008) [2023-10-07 23:37:39,583][67838] Updated weights for policy 0, policy_version 93172 (0.0010) [2023-10-07 23:37:39,963][67838] Updated weights for policy 0, policy_version 93182 (0.0010) [2023-10-07 23:37:42,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191004672. Throughput: 0: 1646.9, 1: 1670.0. Samples: 47756970. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-07 23:37:42,477][66916] Avg episode reward: [(0, '51.050'), (1, '64.710')] [2023-10-07 23:37:42,789][67871] Updated weights for policy 1, policy_version 93350 (0.0007) [2023-10-07 23:37:43,151][67871] Updated weights for policy 1, policy_version 93360 (0.0007) [2023-10-07 23:37:43,523][67871] Updated weights for policy 1, policy_version 93370 (0.0010) [2023-10-07 23:37:44,118][67838] Updated weights for policy 0, policy_version 93192 (0.0010) [2023-10-07 23:37:44,500][67838] Updated weights for policy 0, policy_version 93202 (0.0009) [2023-10-07 23:37:44,876][67838] Updated weights for policy 0, policy_version 93212 (0.0007) [2023-10-07 23:37:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191070208. Throughput: 0: 1651.8, 1: 1673.4. Samples: 47777152. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-07 23:37:47,477][66916] Avg episode reward: [(0, '50.200'), (1, '61.900')] [2023-10-07 23:37:47,692][67871] Updated weights for policy 1, policy_version 93380 (0.0009) [2023-10-07 23:37:48,058][67871] Updated weights for policy 1, policy_version 93390 (0.0008) [2023-10-07 23:37:48,429][67871] Updated weights for policy 1, policy_version 93400 (0.0009) [2023-10-07 23:37:49,262][67838] Updated weights for policy 0, policy_version 93222 (0.0009) [2023-10-07 23:37:49,649][67838] Updated weights for policy 0, policy_version 93232 (0.0010) [2023-10-07 23:37:50,025][67838] Updated weights for policy 0, policy_version 93242 (0.0008) [2023-10-07 23:37:52,477][66916] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191135744. Throughput: 0: 1650.5, 1: 1669.0. Samples: 47797564. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-07 23:37:52,478][66916] Avg episode reward: [(0, '48.430'), (1, '62.860')] [2023-10-07 23:37:52,714][67871] Updated weights for policy 1, policy_version 93410 (0.0007) [2023-10-07 23:37:53,077][67871] Updated weights for policy 1, policy_version 93420 (0.0007) [2023-10-07 23:37:53,447][67871] Updated weights for policy 1, policy_version 93430 (0.0007) [2023-10-07 23:37:53,814][67871] Updated weights for policy 1, policy_version 93440 (0.0007) [2023-10-07 23:37:54,100][67838] Updated weights for policy 0, policy_version 93252 (0.0008) [2023-10-07 23:37:54,474][67838] Updated weights for policy 0, policy_version 93262 (0.0008) [2023-10-07 23:37:54,847][67838] Updated weights for policy 0, policy_version 93272 (0.0009) [2023-10-07 23:37:57,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191201280. Throughput: 0: 1649.1, 1: 1671.3. Samples: 47806750. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-07 23:37:57,477][66916] Avg episode reward: [(0, '50.290'), (1, '62.730')] [2023-10-07 23:37:57,982][67871] Updated weights for policy 1, policy_version 93450 (0.0009) [2023-10-07 23:37:58,357][67871] Updated weights for policy 1, policy_version 93460 (0.0010) [2023-10-07 23:37:58,720][67871] Updated weights for policy 1, policy_version 93470 (0.0008) [2023-10-07 23:37:58,848][67838] Updated weights for policy 0, policy_version 93282 (0.0008) [2023-10-07 23:37:59,215][67838] Updated weights for policy 0, policy_version 93292 (0.0009) [2023-10-07 23:37:59,583][67838] Updated weights for policy 0, policy_version 93302 (0.0009) [2023-10-07 23:37:59,958][67838] Updated weights for policy 0, policy_version 93312 (0.0009) [2023-10-07 23:38:02,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 191266816. Throughput: 0: 1649.3, 1: 1676.7. Samples: 47827282. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-07 23:38:02,478][66916] Avg episode reward: [(0, '49.750'), (1, '62.960')] [2023-10-07 23:38:02,752][67871] Updated weights for policy 1, policy_version 93480 (0.0008) [2023-10-07 23:38:03,117][67871] Updated weights for policy 1, policy_version 93490 (0.0008) [2023-10-07 23:38:03,478][67871] Updated weights for policy 1, policy_version 93500 (0.0009) [2023-10-07 23:38:04,139][67838] Updated weights for policy 0, policy_version 93322 (0.0011) [2023-10-07 23:38:04,517][67838] Updated weights for policy 0, policy_version 93332 (0.0010) [2023-10-07 23:38:04,890][67838] Updated weights for policy 0, policy_version 93342 (0.0009) [2023-10-07 23:38:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 191332352. Throughput: 0: 1647.8, 1: 1681.1. Samples: 47847934. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-07 23:38:07,477][66916] Avg episode reward: [(0, '50.660'), (1, '62.290')] [2023-10-07 23:38:07,607][67871] Updated weights for policy 1, policy_version 93510 (0.0007) [2023-10-07 23:38:07,978][67871] Updated weights for policy 1, policy_version 93520 (0.0009) [2023-10-07 23:38:08,348][67871] Updated weights for policy 1, policy_version 93530 (0.0009) [2023-10-07 23:38:09,003][67838] Updated weights for policy 0, policy_version 93352 (0.0007) [2023-10-07 23:38:09,373][67838] Updated weights for policy 0, policy_version 93362 (0.0007) [2023-10-07 23:38:09,734][67838] Updated weights for policy 0, policy_version 93372 (0.0010) [2023-10-07 23:38:12,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191397888. Throughput: 0: 1646.2, 1: 1682.7. Samples: 47856978. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-07 23:38:12,477][66916] Avg episode reward: [(0, '52.920'), (1, '61.680')] [2023-10-07 23:38:12,664][67871] Updated weights for policy 1, policy_version 93540 (0.0007) [2023-10-07 23:38:13,036][67871] Updated weights for policy 1, policy_version 93550 (0.0007) [2023-10-07 23:38:13,399][67871] Updated weights for policy 1, policy_version 93560 (0.0008) [2023-10-07 23:38:13,760][67838] Updated weights for policy 0, policy_version 93382 (0.0008) [2023-10-07 23:38:14,125][67838] Updated weights for policy 0, policy_version 93392 (0.0010) [2023-10-07 23:38:14,502][67838] Updated weights for policy 0, policy_version 93402 (0.0007) [2023-10-07 23:38:17,403][67871] Updated weights for policy 1, policy_version 93570 (0.0007) [2023-10-07 23:38:17,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191463424. Throughput: 0: 1652.9, 1: 1677.5. Samples: 47877372. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-07 23:38:17,478][66916] Avg episode reward: [(0, '50.810'), (1, '64.550')] [2023-10-07 23:38:17,763][67871] Updated weights for policy 1, policy_version 93580 (0.0009) [2023-10-07 23:38:18,126][67871] Updated weights for policy 1, policy_version 93590 (0.0008) [2023-10-07 23:38:18,492][67871] Updated weights for policy 1, policy_version 93600 (0.0007) [2023-10-07 23:38:18,607][67838] Updated weights for policy 0, policy_version 93412 (0.0007) [2023-10-07 23:38:18,971][67838] Updated weights for policy 0, policy_version 93422 (0.0008) [2023-10-07 23:38:19,343][67838] Updated weights for policy 0, policy_version 93432 (0.0009) [2023-10-07 23:38:22,391][67871] Updated weights for policy 1, policy_version 93610 (0.0009) [2023-10-07 23:38:22,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191528960. Throughput: 0: 1655.4, 1: 1679.6. Samples: 47898068. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-07 23:38:22,477][66916] Avg episode reward: [(0, '50.700'), (1, '62.010')] [2023-10-07 23:38:22,755][67871] Updated weights for policy 1, policy_version 93620 (0.0010) [2023-10-07 23:38:23,127][67871] Updated weights for policy 1, policy_version 93630 (0.0008) [2023-10-07 23:38:23,256][67838] Updated weights for policy 0, policy_version 93442 (0.0010) [2023-10-07 23:38:23,625][67838] Updated weights for policy 0, policy_version 93452 (0.0007) [2023-10-07 23:38:23,995][67838] Updated weights for policy 0, policy_version 93462 (0.0008) [2023-10-07 23:38:24,366][67838] Updated weights for policy 0, policy_version 93472 (0.0007) [2023-10-07 23:38:27,371][67871] Updated weights for policy 1, policy_version 93640 (0.0007) [2023-10-07 23:38:27,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191594496. Throughput: 0: 1659.8, 1: 1681.2. Samples: 47907318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:38:27,477][66916] Avg episode reward: [(0, '48.240'), (1, '63.950')] [2023-10-07 23:38:27,749][67871] Updated weights for policy 1, policy_version 93650 (0.0007) [2023-10-07 23:38:28,118][67871] Updated weights for policy 1, policy_version 93660 (0.0009) [2023-10-07 23:38:28,392][67838] Updated weights for policy 0, policy_version 93482 (0.0010) [2023-10-07 23:38:28,760][67838] Updated weights for policy 0, policy_version 93492 (0.0007) [2023-10-07 23:38:29,127][67838] Updated weights for policy 0, policy_version 93502 (0.0007) [2023-10-07 23:38:32,021][67871] Updated weights for policy 1, policy_version 93670 (0.0008) [2023-10-07 23:38:32,392][67871] Updated weights for policy 1, policy_version 93680 (0.0007) [2023-10-07 23:38:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191660032. Throughput: 0: 1666.9, 1: 1678.7. Samples: 47927704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:38:32,477][66916] Avg episode reward: [(0, '48.760'), (1, '64.610')] [2023-10-07 23:38:32,757][67871] Updated weights for policy 1, policy_version 93690 (0.0010) [2023-10-07 23:38:33,341][67838] Updated weights for policy 0, policy_version 93512 (0.0010) [2023-10-07 23:38:33,707][67838] Updated weights for policy 0, policy_version 93522 (0.0007) [2023-10-07 23:38:34,076][67838] Updated weights for policy 0, policy_version 93532 (0.0011) [2023-10-07 23:38:36,850][67871] Updated weights for policy 1, policy_version 93700 (0.0008) [2023-10-07 23:38:37,207][67871] Updated weights for policy 1, policy_version 93710 (0.0008) [2023-10-07 23:38:37,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191725568. Throughput: 0: 1666.3, 1: 1677.2. Samples: 47948022. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:38:37,478][66916] Avg episode reward: [(0, '49.190'), (1, '66.730')] [2023-10-07 23:38:37,584][67871] Updated weights for policy 1, policy_version 93720 (0.0009) [2023-10-07 23:38:38,295][67838] Updated weights for policy 0, policy_version 93542 (0.0008) [2023-10-07 23:38:38,679][67838] Updated weights for policy 0, policy_version 93552 (0.0008) [2023-10-07 23:38:39,059][67838] Updated weights for policy 0, policy_version 93562 (0.0009) [2023-10-07 23:38:41,609][67871] Updated weights for policy 1, policy_version 93730 (0.0010) [2023-10-07 23:38:41,972][67871] Updated weights for policy 1, policy_version 93740 (0.0008) [2023-10-07 23:38:42,341][67871] Updated weights for policy 1, policy_version 93750 (0.0008) [2023-10-07 23:38:42,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191791104. Throughput: 0: 1661.1, 1: 1680.0. Samples: 47957100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:38:42,477][66916] Avg episode reward: [(0, '45.960'), (1, '69.510')] [2023-10-07 23:38:42,706][67871] Updated weights for policy 1, policy_version 93760 (0.0008) [2023-10-07 23:38:43,200][67838] Updated weights for policy 0, policy_version 93572 (0.0007) [2023-10-07 23:38:43,568][67838] Updated weights for policy 0, policy_version 93582 (0.0008) [2023-10-07 23:38:43,941][67838] Updated weights for policy 0, policy_version 93592 (0.0007) [2023-10-07 23:38:46,876][67871] Updated weights for policy 1, policy_version 93770 (0.0008) [2023-10-07 23:38:47,237][67871] Updated weights for policy 1, policy_version 93780 (0.0009) [2023-10-07 23:38:47,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191856640. Throughput: 0: 1668.0, 1: 1674.9. Samples: 47977714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:38:47,478][66916] Avg episode reward: [(0, '46.850'), (1, '67.710')] [2023-10-07 23:38:47,603][67871] Updated weights for policy 1, policy_version 93790 (0.0009) [2023-10-07 23:38:47,942][67838] Updated weights for policy 0, policy_version 93602 (0.0007) [2023-10-07 23:38:48,306][67838] Updated weights for policy 0, policy_version 93612 (0.0010) [2023-10-07 23:38:48,672][67838] Updated weights for policy 0, policy_version 93622 (0.0007) [2023-10-07 23:38:49,049][67838] Updated weights for policy 0, policy_version 93632 (0.0010) [2023-10-07 23:38:51,687][67871] Updated weights for policy 1, policy_version 93800 (0.0009) [2023-10-07 23:38:52,050][67871] Updated weights for policy 1, policy_version 93810 (0.0010) [2023-10-07 23:38:52,412][67871] Updated weights for policy 1, policy_version 93820 (0.0008) [2023-10-07 23:38:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191922176. Throughput: 0: 1671.8, 1: 1663.5. Samples: 47998020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:38:52,478][66916] Avg episode reward: [(0, '45.940'), (1, '64.060')] [2023-10-07 23:38:53,128][67838] Updated weights for policy 0, policy_version 93642 (0.0009) [2023-10-07 23:38:53,511][67838] Updated weights for policy 0, policy_version 93652 (0.0011) [2023-10-07 23:38:53,878][67838] Updated weights for policy 0, policy_version 93662 (0.0008) [2023-10-07 23:38:56,702][67871] Updated weights for policy 1, policy_version 93830 (0.0008) [2023-10-07 23:38:57,070][67871] Updated weights for policy 1, policy_version 93840 (0.0009) [2023-10-07 23:38:57,443][67871] Updated weights for policy 1, policy_version 93850 (0.0008) [2023-10-07 23:38:57,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191987712. Throughput: 0: 1671.9, 1: 1672.6. Samples: 48007482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:38:57,477][66916] Avg episode reward: [(0, '44.440'), (1, '63.640')] [2023-10-07 23:38:58,082][67838] Updated weights for policy 0, policy_version 93672 (0.0010) [2023-10-07 23:38:58,451][67838] Updated weights for policy 0, policy_version 93682 (0.0009) [2023-10-07 23:38:58,818][67838] Updated weights for policy 0, policy_version 93692 (0.0009) [2023-10-07 23:39:01,453][67871] Updated weights for policy 1, policy_version 93860 (0.0008) [2023-10-07 23:39:01,822][67871] Updated weights for policy 1, policy_version 93870 (0.0008) [2023-10-07 23:39:02,187][67871] Updated weights for policy 1, policy_version 93880 (0.0007) [2023-10-07 23:39:02,476][66916] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 192086016. Throughput: 0: 1667.4, 1: 1680.1. Samples: 48028010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:39:02,477][66916] Avg episode reward: [(0, '48.260'), (1, '60.190')] [2023-10-07 23:39:02,922][67838] Updated weights for policy 0, policy_version 93702 (0.0011) [2023-10-07 23:39:03,295][67838] Updated weights for policy 0, policy_version 93712 (0.0010) [2023-10-07 23:39:03,665][67838] Updated weights for policy 0, policy_version 93722 (0.0011) [2023-10-07 23:39:06,267][67871] Updated weights for policy 1, policy_version 93890 (0.0008) [2023-10-07 23:39:06,628][67871] Updated weights for policy 1, policy_version 93900 (0.0008) [2023-10-07 23:39:07,000][67871] Updated weights for policy 1, policy_version 93910 (0.0009) [2023-10-07 23:39:07,366][67871] Updated weights for policy 1, policy_version 93920 (0.0009) [2023-10-07 23:39:07,476][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 192151552. Throughput: 0: 1671.5, 1: 1660.9. Samples: 48048026. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:39:07,477][66916] Avg episode reward: [(0, '47.130'), (1, '63.140')] [2023-10-07 23:39:07,484][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000093920_96174080.pth... [2023-10-07 23:39:07,484][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000093728_95977472.pth... [2023-10-07 23:39:07,524][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000092352_94568448.pth [2023-10-07 23:39:07,525][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000092192_94404608.pth [2023-10-07 23:39:07,854][67838] Updated weights for policy 0, policy_version 93732 (0.0010) [2023-10-07 23:39:08,220][67838] Updated weights for policy 0, policy_version 93742 (0.0009) [2023-10-07 23:39:08,608][67838] Updated weights for policy 0, policy_version 93752 (0.0008) [2023-10-07 23:39:11,289][67871] Updated weights for policy 1, policy_version 93930 (0.0010) [2023-10-07 23:39:11,660][67871] Updated weights for policy 1, policy_version 93940 (0.0008) [2023-10-07 23:39:12,012][67871] Updated weights for policy 1, policy_version 93950 (0.0008) [2023-10-07 23:39:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 192217088. Throughput: 0: 1664.4, 1: 1679.3. Samples: 48057786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:39:12,478][66916] Avg episode reward: [(0, '48.080'), (1, '66.600')] [2023-10-07 23:39:12,761][67838] Updated weights for policy 0, policy_version 93762 (0.0009) [2023-10-07 23:39:13,140][67838] Updated weights for policy 0, policy_version 93772 (0.0010) [2023-10-07 23:39:13,508][67838] Updated weights for policy 0, policy_version 93782 (0.0011) [2023-10-07 23:39:13,880][67838] Updated weights for policy 0, policy_version 93792 (0.0010) [2023-10-07 23:39:16,267][67871] Updated weights for policy 1, policy_version 93960 (0.0007) [2023-10-07 23:39:16,643][67871] Updated weights for policy 1, policy_version 93970 (0.0007) [2023-10-07 23:39:17,010][67871] Updated weights for policy 1, policy_version 93980 (0.0007) [2023-10-07 23:39:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 192282624. Throughput: 0: 1663.6, 1: 1679.6. Samples: 48078146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:39:17,477][66916] Avg episode reward: [(0, '46.470'), (1, '62.750')] [2023-10-07 23:39:17,683][67838] Updated weights for policy 0, policy_version 93802 (0.0011) [2023-10-07 23:39:18,051][67838] Updated weights for policy 0, policy_version 93812 (0.0010) [2023-10-07 23:39:18,431][67838] Updated weights for policy 0, policy_version 93822 (0.0008) [2023-10-07 23:39:21,022][67871] Updated weights for policy 1, policy_version 93990 (0.0008) [2023-10-07 23:39:21,389][67871] Updated weights for policy 1, policy_version 94000 (0.0007) [2023-10-07 23:39:21,753][67871] Updated weights for policy 1, policy_version 94010 (0.0007) [2023-10-07 23:39:22,474][67838] Updated weights for policy 0, policy_version 93832 (0.0009) [2023-10-07 23:39:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 192348160. Throughput: 0: 1676.2, 1: 1655.2. Samples: 48097932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:39:22,477][66916] Avg episode reward: [(0, '48.260'), (1, '63.030')] [2023-10-07 23:39:22,852][67838] Updated weights for policy 0, policy_version 93842 (0.0008) [2023-10-07 23:39:23,233][67838] Updated weights for policy 0, policy_version 93852 (0.0008) [2023-10-07 23:39:25,691][67871] Updated weights for policy 1, policy_version 94020 (0.0009) [2023-10-07 23:39:26,059][67871] Updated weights for policy 1, policy_version 94030 (0.0008) [2023-10-07 23:39:26,426][67871] Updated weights for policy 1, policy_version 94040 (0.0008) [2023-10-07 23:39:27,359][67838] Updated weights for policy 0, policy_version 93862 (0.0009) [2023-10-07 23:39:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 192413696. Throughput: 0: 1678.9, 1: 1676.6. Samples: 48108100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:39:27,478][66916] Avg episode reward: [(0, '44.970'), (1, '65.960')] [2023-10-07 23:39:27,742][67838] Updated weights for policy 0, policy_version 93872 (0.0009) [2023-10-07 23:39:28,109][67838] Updated weights for policy 0, policy_version 93882 (0.0008) [2023-10-07 23:39:30,629][67871] Updated weights for policy 1, policy_version 94050 (0.0010) [2023-10-07 23:39:30,994][67871] Updated weights for policy 1, policy_version 94060 (0.0010) [2023-10-07 23:39:31,360][67871] Updated weights for policy 1, policy_version 94070 (0.0009) [2023-10-07 23:39:31,721][67871] Updated weights for policy 1, policy_version 94080 (0.0009) [2023-10-07 23:39:32,255][67838] Updated weights for policy 0, policy_version 93892 (0.0007) [2023-10-07 23:39:32,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 192479232. Throughput: 0: 1675.6, 1: 1669.0. Samples: 48128222. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:39:32,478][66916] Avg episode reward: [(0, '44.390'), (1, '66.570')] [2023-10-07 23:39:32,630][67838] Updated weights for policy 0, policy_version 93902 (0.0008) [2023-10-07 23:39:32,998][67838] Updated weights for policy 0, policy_version 93912 (0.0008) [2023-10-07 23:39:35,943][67871] Updated weights for policy 1, policy_version 94090 (0.0009) [2023-10-07 23:39:36,303][67871] Updated weights for policy 1, policy_version 94100 (0.0009) [2023-10-07 23:39:36,671][67871] Updated weights for policy 1, policy_version 94110 (0.0009) [2023-10-07 23:39:37,214][67838] Updated weights for policy 0, policy_version 93922 (0.0010) [2023-10-07 23:39:37,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 192544768. Throughput: 0: 1673.4, 1: 1652.8. Samples: 48147696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:39:37,477][66916] Avg episode reward: [(0, '42.340'), (1, '59.760')] [2023-10-07 23:39:37,580][67838] Updated weights for policy 0, policy_version 93932 (0.0008) [2023-10-07 23:39:37,956][67838] Updated weights for policy 0, policy_version 93942 (0.0010) [2023-10-07 23:39:38,329][67838] Updated weights for policy 0, policy_version 93952 (0.0010) [2023-10-07 23:39:40,866][67871] Updated weights for policy 1, policy_version 94120 (0.0008) [2023-10-07 23:39:41,226][67871] Updated weights for policy 1, policy_version 94130 (0.0007) [2023-10-07 23:39:41,599][67871] Updated weights for policy 1, policy_version 94140 (0.0008) [2023-10-07 23:39:42,422][67838] Updated weights for policy 0, policy_version 93962 (0.0007) [2023-10-07 23:39:42,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 192610304. Throughput: 0: 1671.4, 1: 1669.2. Samples: 48157812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:39:42,477][66916] Avg episode reward: [(0, '44.840'), (1, '62.270')] [2023-10-07 23:39:42,794][67838] Updated weights for policy 0, policy_version 93972 (0.0010) [2023-10-07 23:39:43,166][67838] Updated weights for policy 0, policy_version 93982 (0.0008) [2023-10-07 23:39:45,630][67871] Updated weights for policy 1, policy_version 94150 (0.0008) [2023-10-07 23:39:45,997][67871] Updated weights for policy 1, policy_version 94160 (0.0008) [2023-10-07 23:39:46,363][67871] Updated weights for policy 1, policy_version 94170 (0.0009) [2023-10-07 23:39:47,437][67838] Updated weights for policy 0, policy_version 93992 (0.0009) [2023-10-07 23:39:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 192675840. Throughput: 0: 1670.0, 1: 1654.8. Samples: 48177622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:39:47,477][66916] Avg episode reward: [(0, '44.280'), (1, '63.020')] [2023-10-07 23:39:47,801][67838] Updated weights for policy 0, policy_version 94002 (0.0009) [2023-10-07 23:39:48,173][67838] Updated weights for policy 0, policy_version 94012 (0.0010) [2023-10-07 23:39:50,559][67871] Updated weights for policy 1, policy_version 94180 (0.0008) [2023-10-07 23:39:50,930][67871] Updated weights for policy 1, policy_version 94190 (0.0009) [2023-10-07 23:39:51,291][67871] Updated weights for policy 1, policy_version 94200 (0.0009) [2023-10-07 23:39:52,202][67838] Updated weights for policy 0, policy_version 94022 (0.0009) [2023-10-07 23:39:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 192741376. Throughput: 0: 1665.0, 1: 1654.4. Samples: 48197400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:39:52,477][66916] Avg episode reward: [(0, '48.640'), (1, '62.380')] [2023-10-07 23:39:52,581][67838] Updated weights for policy 0, policy_version 94032 (0.0007) [2023-10-07 23:39:52,955][67838] Updated weights for policy 0, policy_version 94042 (0.0008) [2023-10-07 23:39:55,213][67871] Updated weights for policy 1, policy_version 94210 (0.0011) [2023-10-07 23:39:55,579][67871] Updated weights for policy 1, policy_version 94220 (0.0009) [2023-10-07 23:39:55,946][67871] Updated weights for policy 1, policy_version 94230 (0.0010) [2023-10-07 23:39:56,302][67871] Updated weights for policy 1, policy_version 94240 (0.0009) [2023-10-07 23:39:57,036][67838] Updated weights for policy 0, policy_version 94052 (0.0008) [2023-10-07 23:39:57,402][67838] Updated weights for policy 0, policy_version 94062 (0.0009) [2023-10-07 23:39:57,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 192806912. Throughput: 0: 1667.8, 1: 1663.8. Samples: 48207706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:39:57,477][66916] Avg episode reward: [(0, '48.720'), (1, '59.310')] [2023-10-07 23:39:57,774][67838] Updated weights for policy 0, policy_version 94072 (0.0010) [2023-10-07 23:40:00,686][67871] Updated weights for policy 1, policy_version 94250 (0.0008) [2023-10-07 23:40:01,045][67871] Updated weights for policy 1, policy_version 94260 (0.0009) [2023-10-07 23:40:01,410][67871] Updated weights for policy 1, policy_version 94270 (0.0009) [2023-10-07 23:40:01,936][67838] Updated weights for policy 0, policy_version 94082 (0.0008) [2023-10-07 23:40:02,301][67838] Updated weights for policy 0, policy_version 94092 (0.0007) [2023-10-07 23:40:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 192872448. Throughput: 0: 1669.0, 1: 1657.2. Samples: 48227824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:40:02,477][66916] Avg episode reward: [(0, '48.000'), (1, '60.960')] [2023-10-07 23:40:02,676][67838] Updated weights for policy 0, policy_version 94102 (0.0008) [2023-10-07 23:40:03,037][67838] Updated weights for policy 0, policy_version 94112 (0.0008) [2023-10-07 23:40:05,374][67871] Updated weights for policy 1, policy_version 94280 (0.0008) [2023-10-07 23:40:05,751][67871] Updated weights for policy 1, policy_version 94290 (0.0009) [2023-10-07 23:40:06,112][67871] Updated weights for policy 1, policy_version 94300 (0.0009) [2023-10-07 23:40:07,090][67838] Updated weights for policy 0, policy_version 94122 (0.0008) [2023-10-07 23:40:07,461][67838] Updated weights for policy 0, policy_version 94132 (0.0007) [2023-10-07 23:40:07,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 192937984. Throughput: 0: 1656.5, 1: 1664.5. Samples: 48247376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:40:07,478][66916] Avg episode reward: [(0, '49.300'), (1, '61.470')] [2023-10-07 23:40:07,843][67838] Updated weights for policy 0, policy_version 94142 (0.0011) [2023-10-07 23:40:10,157][67871] Updated weights for policy 1, policy_version 94310 (0.0009) [2023-10-07 23:40:10,514][67871] Updated weights for policy 1, policy_version 94320 (0.0010) [2023-10-07 23:40:10,887][67871] Updated weights for policy 1, policy_version 94330 (0.0008) [2023-10-07 23:40:11,843][67838] Updated weights for policy 0, policy_version 94152 (0.0007) [2023-10-07 23:40:12,216][67838] Updated weights for policy 0, policy_version 94162 (0.0007) [2023-10-07 23:40:12,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 193003520. Throughput: 0: 1663.8, 1: 1670.5. Samples: 48258142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:40:12,477][66916] Avg episode reward: [(0, '49.960'), (1, '65.200')] [2023-10-07 23:40:12,586][67838] Updated weights for policy 0, policy_version 94172 (0.0008) [2023-10-07 23:40:15,163][67871] Updated weights for policy 1, policy_version 94340 (0.0008) [2023-10-07 23:40:15,531][67871] Updated weights for policy 1, policy_version 94350 (0.0008) [2023-10-07 23:40:15,886][67871] Updated weights for policy 1, policy_version 94360 (0.0010) [2023-10-07 23:40:16,755][67838] Updated weights for policy 0, policy_version 94182 (0.0011) [2023-10-07 23:40:17,124][67838] Updated weights for policy 0, policy_version 94192 (0.0008) [2023-10-07 23:40:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 193069056. Throughput: 0: 1662.2, 1: 1658.6. Samples: 48277658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:40:17,477][66916] Avg episode reward: [(0, '50.820'), (1, '64.880')] [2023-10-07 23:40:17,501][67838] Updated weights for policy 0, policy_version 94202 (0.0008) [2023-10-07 23:40:19,790][67871] Updated weights for policy 1, policy_version 94370 (0.0008) [2023-10-07 23:40:20,161][67871] Updated weights for policy 1, policy_version 94380 (0.0010) [2023-10-07 23:40:20,531][67871] Updated weights for policy 1, policy_version 94390 (0.0009) [2023-10-07 23:40:20,890][67871] Updated weights for policy 1, policy_version 94400 (0.0011) [2023-10-07 23:40:21,693][67838] Updated weights for policy 0, policy_version 94212 (0.0009) [2023-10-07 23:40:22,064][67838] Updated weights for policy 0, policy_version 94222 (0.0007) [2023-10-07 23:40:22,437][67838] Updated weights for policy 0, policy_version 94232 (0.0007) [2023-10-07 23:40:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 193134592. Throughput: 0: 1647.1, 1: 1673.0. Samples: 48297102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-07 23:40:22,477][66916] Avg episode reward: [(0, '51.220'), (1, '66.530')] [2023-10-07 23:40:24,943][67871] Updated weights for policy 1, policy_version 94410 (0.0009) [2023-10-07 23:40:25,312][67871] Updated weights for policy 1, policy_version 94420 (0.0007) [2023-10-07 23:40:25,685][67871] Updated weights for policy 1, policy_version 94430 (0.0009) [2023-10-07 23:40:26,470][67838] Updated weights for policy 0, policy_version 94242 (0.0009) [2023-10-07 23:40:26,841][67838] Updated weights for policy 0, policy_version 94252 (0.0007) [2023-10-07 23:40:27,208][67838] Updated weights for policy 0, policy_version 94262 (0.0007) [2023-10-07 23:40:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 193200128. Throughput: 0: 1659.6, 1: 1672.4. Samples: 48307756. Policy #0 lag: (min: 21.0, avg: 25.0, max: 53.0) [2023-10-07 23:40:27,477][66916] Avg episode reward: [(0, '53.810'), (1, '67.840')] [2023-10-07 23:40:27,578][67838] Updated weights for policy 0, policy_version 94272 (0.0007) [2023-10-07 23:40:29,682][67871] Updated weights for policy 1, policy_version 94440 (0.0008) [2023-10-07 23:40:30,054][67871] Updated weights for policy 1, policy_version 94450 (0.0008) [2023-10-07 23:40:30,418][67871] Updated weights for policy 1, policy_version 94460 (0.0009) [2023-10-07 23:40:31,712][67838] Updated weights for policy 0, policy_version 94282 (0.0007) [2023-10-07 23:40:32,082][67838] Updated weights for policy 0, policy_version 94292 (0.0010) [2023-10-07 23:40:32,451][67838] Updated weights for policy 0, policy_version 94302 (0.0007) [2023-10-07 23:40:32,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 193265664. Throughput: 0: 1662.8, 1: 1662.6. Samples: 48327264. Policy #0 lag: (min: 21.0, avg: 25.0, max: 53.0) [2023-10-07 23:40:32,477][66916] Avg episode reward: [(0, '53.760'), (1, '67.790')] [2023-10-07 23:40:34,503][67871] Updated weights for policy 1, policy_version 94470 (0.0008) [2023-10-07 23:40:34,863][67871] Updated weights for policy 1, policy_version 94480 (0.0010) [2023-10-07 23:40:35,237][67871] Updated weights for policy 1, policy_version 94490 (0.0010) [2023-10-07 23:40:36,529][67838] Updated weights for policy 0, policy_version 94312 (0.0007) [2023-10-07 23:40:36,891][67838] Updated weights for policy 0, policy_version 94322 (0.0008) [2023-10-07 23:40:37,270][67838] Updated weights for policy 0, policy_version 94332 (0.0009) [2023-10-07 23:40:37,476][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 193363968. Throughput: 0: 1647.9, 1: 1676.5. Samples: 48347000. Policy #0 lag: (min: 21.0, avg: 25.0, max: 53.0) [2023-10-07 23:40:37,477][66916] Avg episode reward: [(0, '54.690'), (1, '65.570')] [2023-10-07 23:40:39,344][67871] Updated weights for policy 1, policy_version 94500 (0.0010) [2023-10-07 23:40:39,723][67871] Updated weights for policy 1, policy_version 94510 (0.0009) [2023-10-07 23:40:40,079][67871] Updated weights for policy 1, policy_version 94520 (0.0010) [2023-10-07 23:40:41,541][67838] Updated weights for policy 0, policy_version 94342 (0.0008) [2023-10-07 23:40:41,904][67838] Updated weights for policy 0, policy_version 94352 (0.0008) [2023-10-07 23:40:42,277][67838] Updated weights for policy 0, policy_version 94362 (0.0007) [2023-10-07 23:40:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 193396736. Throughput: 0: 1664.5, 1: 1661.6. Samples: 48357380. Policy #0 lag: (min: 21.0, avg: 25.0, max: 53.0) [2023-10-07 23:40:42,477][66916] Avg episode reward: [(0, '48.870'), (1, '62.660')] [2023-10-07 23:40:44,358][67871] Updated weights for policy 1, policy_version 94530 (0.0010) [2023-10-07 23:40:44,729][67871] Updated weights for policy 1, policy_version 94540 (0.0008) [2023-10-07 23:40:45,102][67871] Updated weights for policy 1, policy_version 94550 (0.0009) [2023-10-07 23:40:45,463][67871] Updated weights for policy 1, policy_version 94560 (0.0010) [2023-10-07 23:40:46,479][67838] Updated weights for policy 0, policy_version 94372 (0.0010) [2023-10-07 23:40:46,860][67838] Updated weights for policy 0, policy_version 94382 (0.0008) [2023-10-07 23:40:47,235][67838] Updated weights for policy 0, policy_version 94392 (0.0007) [2023-10-07 23:40:47,476][66916] Fps is (10 sec: 9830.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 193462272. Throughput: 0: 1661.2, 1: 1655.6. Samples: 48377078. Policy #0 lag: (min: 21.0, avg: 25.0, max: 53.0) [2023-10-07 23:40:47,477][66916] Avg episode reward: [(0, '48.510'), (1, '61.360')] [2023-10-07 23:40:49,740][67871] Updated weights for policy 1, policy_version 94570 (0.0010) [2023-10-07 23:40:50,116][67871] Updated weights for policy 1, policy_version 94580 (0.0008) [2023-10-07 23:40:50,481][67871] Updated weights for policy 1, policy_version 94590 (0.0009) [2023-10-07 23:40:51,293][67838] Updated weights for policy 0, policy_version 94402 (0.0009) [2023-10-07 23:40:51,665][67838] Updated weights for policy 0, policy_version 94412 (0.0010) [2023-10-07 23:40:52,040][67838] Updated weights for policy 0, policy_version 94422 (0.0008) [2023-10-07 23:40:52,407][67838] Updated weights for policy 0, policy_version 94432 (0.0008) [2023-10-07 23:40:52,476][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 193560576. Throughput: 0: 1645.3, 1: 1669.1. Samples: 48396524. Policy #0 lag: (min: 21.0, avg: 25.0, max: 53.0) [2023-10-07 23:40:52,477][66916] Avg episode reward: [(0, '44.860'), (1, '61.090')] [2023-10-07 23:40:54,721][67871] Updated weights for policy 1, policy_version 94600 (0.0009) [2023-10-07 23:40:55,089][67871] Updated weights for policy 1, policy_version 94610 (0.0007) [2023-10-07 23:40:55,459][67871] Updated weights for policy 1, policy_version 94620 (0.0009) [2023-10-07 23:40:56,484][67838] Updated weights for policy 0, policy_version 94442 (0.0008) [2023-10-07 23:40:56,858][67838] Updated weights for policy 0, policy_version 94452 (0.0008) [2023-10-07 23:40:57,235][67838] Updated weights for policy 0, policy_version 94462 (0.0009) [2023-10-07 23:40:57,477][66916] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 193626112. Throughput: 0: 1655.4, 1: 1653.6. Samples: 48407048. Policy #0 lag: (min: 21.0, avg: 25.0, max: 53.0) [2023-10-07 23:40:57,478][66916] Avg episode reward: [(0, '42.880'), (1, '61.940')] [2023-10-07 23:40:59,597][67871] Updated weights for policy 1, policy_version 94630 (0.0009) [2023-10-07 23:40:59,960][67871] Updated weights for policy 1, policy_version 94640 (0.0008) [2023-10-07 23:41:00,319][67871] Updated weights for policy 1, policy_version 94650 (0.0007) [2023-10-07 23:41:01,464][67838] Updated weights for policy 0, policy_version 94472 (0.0011) [2023-10-07 23:41:01,836][67838] Updated weights for policy 0, policy_version 94482 (0.0011) [2023-10-07 23:41:02,208][67838] Updated weights for policy 0, policy_version 94492 (0.0008) [2023-10-07 23:41:02,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 193691648. Throughput: 0: 1655.8, 1: 1654.3. Samples: 48426616. Policy #0 lag: (min: 21.0, avg: 25.0, max: 53.0) [2023-10-07 23:41:02,478][66916] Avg episode reward: [(0, '46.080'), (1, '60.290')] [2023-10-07 23:41:04,429][67871] Updated weights for policy 1, policy_version 94660 (0.0009) [2023-10-07 23:41:04,794][67871] Updated weights for policy 1, policy_version 94670 (0.0007) [2023-10-07 23:41:05,163][67871] Updated weights for policy 1, policy_version 94680 (0.0007) [2023-10-07 23:41:06,400][67838] Updated weights for policy 0, policy_version 94502 (0.0008) [2023-10-07 23:41:06,779][67838] Updated weights for policy 0, policy_version 94512 (0.0007) [2023-10-07 23:41:07,147][67838] Updated weights for policy 0, policy_version 94522 (0.0007) [2023-10-07 23:41:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.3). Total num frames: 193757184. Throughput: 0: 1649.2, 1: 1664.4. Samples: 48446218. Policy #0 lag: (min: 21.0, avg: 25.0, max: 53.0) [2023-10-07 23:41:07,477][66916] Avg episode reward: [(0, '42.810'), (1, '61.820')] [2023-10-07 23:41:07,487][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000094688_96960512.pth... [2023-10-07 23:41:07,488][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000094528_96796672.pth... [2023-10-07 23:41:07,521][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000092960_95191040.pth [2023-10-07 23:41:07,525][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000093120_95354880.pth [2023-10-07 23:41:09,290][67871] Updated weights for policy 1, policy_version 94690 (0.0009) [2023-10-07 23:41:09,669][67871] Updated weights for policy 1, policy_version 94700 (0.0009) [2023-10-07 23:41:10,038][67871] Updated weights for policy 1, policy_version 94710 (0.0007) [2023-10-07 23:41:10,407][67871] Updated weights for policy 1, policy_version 94720 (0.0008) [2023-10-07 23:41:11,263][67838] Updated weights for policy 0, policy_version 94532 (0.0009) [2023-10-07 23:41:11,642][67838] Updated weights for policy 0, policy_version 94542 (0.0007) [2023-10-07 23:41:12,011][67838] Updated weights for policy 0, policy_version 94552 (0.0008) [2023-10-07 23:41:12,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 193822720. Throughput: 0: 1657.1, 1: 1653.1. Samples: 48456714. Policy #0 lag: (min: 21.0, avg: 25.0, max: 53.0) [2023-10-07 23:41:12,477][66916] Avg episode reward: [(0, '45.860'), (1, '62.420')] [2023-10-07 23:41:14,356][67871] Updated weights for policy 1, policy_version 94730 (0.0007) [2023-10-07 23:41:14,722][67871] Updated weights for policy 1, policy_version 94740 (0.0007) [2023-10-07 23:41:15,092][67871] Updated weights for policy 1, policy_version 94750 (0.0008) [2023-10-07 23:41:16,075][67838] Updated weights for policy 0, policy_version 94562 (0.0008) [2023-10-07 23:41:16,445][67838] Updated weights for policy 0, policy_version 94572 (0.0009) [2023-10-07 23:41:16,809][67838] Updated weights for policy 0, policy_version 94582 (0.0009) [2023-10-07 23:41:17,178][67838] Updated weights for policy 0, policy_version 94592 (0.0007) [2023-10-07 23:41:17,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 193888256. Throughput: 0: 1656.2, 1: 1661.1. Samples: 48476542. Policy #0 lag: (min: 21.0, avg: 25.0, max: 53.0) [2023-10-07 23:41:17,477][66916] Avg episode reward: [(0, '47.860'), (1, '59.870')] [2023-10-07 23:41:19,194][67871] Updated weights for policy 1, policy_version 94760 (0.0009) [2023-10-07 23:41:19,557][67871] Updated weights for policy 1, policy_version 94770 (0.0010) [2023-10-07 23:41:19,926][67871] Updated weights for policy 1, policy_version 94780 (0.0008) [2023-10-07 23:41:21,414][67838] Updated weights for policy 0, policy_version 94602 (0.0007) [2023-10-07 23:41:21,790][67838] Updated weights for policy 0, policy_version 94612 (0.0008) [2023-10-07 23:41:22,153][67838] Updated weights for policy 0, policy_version 94622 (0.0009) [2023-10-07 23:41:22,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 193953792. Throughput: 0: 1647.7, 1: 1662.7. Samples: 48495970. Policy #0 lag: (min: 30.0, avg: 34.1, max: 62.0) [2023-10-07 23:41:22,477][66916] Avg episode reward: [(0, '43.790'), (1, '63.090')] [2023-10-07 23:41:23,797][67871] Updated weights for policy 1, policy_version 94790 (0.0009) [2023-10-07 23:41:24,163][67871] Updated weights for policy 1, policy_version 94800 (0.0008) [2023-10-07 23:41:24,538][67871] Updated weights for policy 1, policy_version 94810 (0.0010) [2023-10-07 23:41:26,231][67838] Updated weights for policy 0, policy_version 94632 (0.0010) [2023-10-07 23:41:26,600][67838] Updated weights for policy 0, policy_version 94642 (0.0009) [2023-10-07 23:41:26,970][67838] Updated weights for policy 0, policy_version 94652 (0.0008) [2023-10-07 23:41:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 194019328. Throughput: 0: 1653.7, 1: 1653.2. Samples: 48506192. Policy #0 lag: (min: 30.0, avg: 34.1, max: 62.0) [2023-10-07 23:41:27,477][66916] Avg episode reward: [(0, '44.580'), (1, '64.970')] [2023-10-07 23:41:28,466][67871] Updated weights for policy 1, policy_version 94820 (0.0007) [2023-10-07 23:41:28,833][67871] Updated weights for policy 1, policy_version 94830 (0.0009) [2023-10-07 23:41:29,198][67871] Updated weights for policy 1, policy_version 94840 (0.0009) [2023-10-07 23:41:31,121][67838] Updated weights for policy 0, policy_version 94662 (0.0009) [2023-10-07 23:41:31,485][67838] Updated weights for policy 0, policy_version 94672 (0.0007) [2023-10-07 23:41:31,857][67838] Updated weights for policy 0, policy_version 94682 (0.0010) [2023-10-07 23:41:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 194084864. Throughput: 0: 1648.5, 1: 1674.4. Samples: 48526608. Policy #0 lag: (min: 30.0, avg: 34.1, max: 62.0) [2023-10-07 23:41:32,477][66916] Avg episode reward: [(0, '46.060'), (1, '65.010')] [2023-10-07 23:41:33,170][67871] Updated weights for policy 1, policy_version 94850 (0.0008) [2023-10-07 23:41:33,528][67871] Updated weights for policy 1, policy_version 94860 (0.0010) [2023-10-07 23:41:33,895][67871] Updated weights for policy 1, policy_version 94870 (0.0010) [2023-10-07 23:41:34,263][67871] Updated weights for policy 1, policy_version 94880 (0.0010) [2023-10-07 23:41:36,116][67838] Updated weights for policy 0, policy_version 94692 (0.0010) [2023-10-07 23:41:36,486][67838] Updated weights for policy 0, policy_version 94702 (0.0010) [2023-10-07 23:41:36,867][67838] Updated weights for policy 0, policy_version 94712 (0.0008) [2023-10-07 23:41:37,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 194150400. Throughput: 0: 1646.8, 1: 1677.7. Samples: 48546128. Policy #0 lag: (min: 30.0, avg: 34.1, max: 62.0) [2023-10-07 23:41:37,477][66916] Avg episode reward: [(0, '43.490'), (1, '65.780')] [2023-10-07 23:41:38,441][67871] Updated weights for policy 1, policy_version 94890 (0.0010) [2023-10-07 23:41:38,803][67871] Updated weights for policy 1, policy_version 94900 (0.0009) [2023-10-07 23:41:39,176][67871] Updated weights for policy 1, policy_version 94910 (0.0011) [2023-10-07 23:41:40,961][67838] Updated weights for policy 0, policy_version 94722 (0.0008) [2023-10-07 23:41:41,333][67838] Updated weights for policy 0, policy_version 94732 (0.0008) [2023-10-07 23:41:41,707][67838] Updated weights for policy 0, policy_version 94742 (0.0011) [2023-10-07 23:41:42,089][67838] Updated weights for policy 0, policy_version 94752 (0.0009) [2023-10-07 23:41:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 194215936. Throughput: 0: 1653.6, 1: 1661.6. Samples: 48556230. Policy #0 lag: (min: 30.0, avg: 34.1, max: 62.0) [2023-10-07 23:41:42,478][66916] Avg episode reward: [(0, '48.710'), (1, '65.440')] [2023-10-07 23:41:43,390][67871] Updated weights for policy 1, policy_version 94920 (0.0008) [2023-10-07 23:41:43,752][67871] Updated weights for policy 1, policy_version 94930 (0.0008) [2023-10-07 23:41:44,113][67871] Updated weights for policy 1, policy_version 94940 (0.0007) [2023-10-07 23:41:46,239][67838] Updated weights for policy 0, policy_version 94762 (0.0009) [2023-10-07 23:41:46,613][67838] Updated weights for policy 0, policy_version 94772 (0.0007) [2023-10-07 23:41:46,980][67838] Updated weights for policy 0, policy_version 94782 (0.0007) [2023-10-07 23:41:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 194281472. Throughput: 0: 1650.4, 1: 1673.6. Samples: 48576194. Policy #0 lag: (min: 30.0, avg: 34.1, max: 62.0) [2023-10-07 23:41:47,478][66916] Avg episode reward: [(0, '47.380'), (1, '64.710')] [2023-10-07 23:41:48,387][67871] Updated weights for policy 1, policy_version 94950 (0.0007) [2023-10-07 23:41:48,761][67871] Updated weights for policy 1, policy_version 94960 (0.0008) [2023-10-07 23:41:49,123][67871] Updated weights for policy 1, policy_version 94970 (0.0009) [2023-10-07 23:41:51,081][67838] Updated weights for policy 0, policy_version 94792 (0.0008) [2023-10-07 23:41:51,454][67838] Updated weights for policy 0, policy_version 94802 (0.0010) [2023-10-07 23:41:51,838][67838] Updated weights for policy 0, policy_version 94812 (0.0010) [2023-10-07 23:41:52,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 194347008. Throughput: 0: 1644.8, 1: 1671.9. Samples: 48595472. Policy #0 lag: (min: 30.0, avg: 34.1, max: 62.0) [2023-10-07 23:41:52,478][66916] Avg episode reward: [(0, '44.600'), (1, '65.350')] [2023-10-07 23:41:53,239][67871] Updated weights for policy 1, policy_version 94980 (0.0009) [2023-10-07 23:41:53,600][67871] Updated weights for policy 1, policy_version 94990 (0.0009) [2023-10-07 23:41:53,970][67871] Updated weights for policy 1, policy_version 95000 (0.0011) [2023-10-07 23:41:56,019][67838] Updated weights for policy 0, policy_version 94822 (0.0010) [2023-10-07 23:41:56,394][67838] Updated weights for policy 0, policy_version 94832 (0.0008) [2023-10-07 23:41:56,767][67838] Updated weights for policy 0, policy_version 94842 (0.0008) [2023-10-07 23:41:57,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 194412544. Throughput: 0: 1654.3, 1: 1661.6. Samples: 48605930. Policy #0 lag: (min: 30.0, avg: 34.1, max: 62.0) [2023-10-07 23:41:57,478][66916] Avg episode reward: [(0, '45.670'), (1, '61.860')] [2023-10-07 23:41:58,072][67871] Updated weights for policy 1, policy_version 95010 (0.0007) [2023-10-07 23:41:58,444][67871] Updated weights for policy 1, policy_version 95020 (0.0012) [2023-10-07 23:41:58,811][67871] Updated weights for policy 1, policy_version 95030 (0.0007) [2023-10-07 23:41:59,173][67871] Updated weights for policy 1, policy_version 95040 (0.0008) [2023-10-07 23:42:00,753][67838] Updated weights for policy 0, policy_version 94852 (0.0008) [2023-10-07 23:42:01,124][67838] Updated weights for policy 0, policy_version 94862 (0.0010) [2023-10-07 23:42:01,497][67838] Updated weights for policy 0, policy_version 94872 (0.0009) [2023-10-07 23:42:02,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 194478080. Throughput: 0: 1642.5, 1: 1676.8. Samples: 48625912. Policy #0 lag: (min: 30.0, avg: 34.1, max: 62.0) [2023-10-07 23:42:02,478][66916] Avg episode reward: [(0, '43.390'), (1, '64.900')] [2023-10-07 23:42:03,385][67871] Updated weights for policy 1, policy_version 95050 (0.0009) [2023-10-07 23:42:03,750][67871] Updated weights for policy 1, policy_version 95060 (0.0008) [2023-10-07 23:42:04,120][67871] Updated weights for policy 1, policy_version 95070 (0.0009) [2023-10-07 23:42:05,837][67838] Updated weights for policy 0, policy_version 94882 (0.0011) [2023-10-07 23:42:06,209][67838] Updated weights for policy 0, policy_version 94892 (0.0010) [2023-10-07 23:42:06,581][67838] Updated weights for policy 0, policy_version 94902 (0.0007) [2023-10-07 23:42:06,948][67838] Updated weights for policy 0, policy_version 94912 (0.0009) [2023-10-07 23:42:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 194543616. Throughput: 0: 1646.4, 1: 1677.3. Samples: 48645534. Policy #0 lag: (min: 30.0, avg: 34.1, max: 62.0) [2023-10-07 23:42:07,477][66916] Avg episode reward: [(0, '44.340'), (1, '61.040')] [2023-10-07 23:42:08,253][67871] Updated weights for policy 1, policy_version 95080 (0.0007) [2023-10-07 23:42:08,614][67871] Updated weights for policy 1, policy_version 95090 (0.0009) [2023-10-07 23:42:08,984][67871] Updated weights for policy 1, policy_version 95100 (0.0007) [2023-10-07 23:42:11,021][67838] Updated weights for policy 0, policy_version 94922 (0.0008) [2023-10-07 23:42:11,386][67838] Updated weights for policy 0, policy_version 94932 (0.0011) [2023-10-07 23:42:11,758][67838] Updated weights for policy 0, policy_version 94942 (0.0011) [2023-10-07 23:42:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 194609152. Throughput: 0: 1655.3, 1: 1673.3. Samples: 48655978. Policy #0 lag: (min: 30.0, avg: 34.1, max: 62.0) [2023-10-07 23:42:12,478][66916] Avg episode reward: [(0, '45.250'), (1, '62.410')] [2023-10-07 23:42:13,007][67871] Updated weights for policy 1, policy_version 95110 (0.0009) [2023-10-07 23:42:13,366][67871] Updated weights for policy 1, policy_version 95120 (0.0008) [2023-10-07 23:42:13,738][67871] Updated weights for policy 1, policy_version 95130 (0.0007) [2023-10-07 23:42:15,912][67838] Updated weights for policy 0, policy_version 94952 (0.0011) [2023-10-07 23:42:16,293][67838] Updated weights for policy 0, policy_version 94962 (0.0009) [2023-10-07 23:42:16,674][67838] Updated weights for policy 0, policy_version 94972 (0.0009) [2023-10-07 23:42:17,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 194674688. Throughput: 0: 1647.2, 1: 1666.0. Samples: 48675704. Policy #0 lag: (min: 30.0, avg: 34.1, max: 62.0) [2023-10-07 23:42:17,477][66916] Avg episode reward: [(0, '41.620'), (1, '61.260')] [2023-10-07 23:42:17,682][67871] Updated weights for policy 1, policy_version 95140 (0.0007) [2023-10-07 23:42:18,054][67871] Updated weights for policy 1, policy_version 95150 (0.0009) [2023-10-07 23:42:18,423][67871] Updated weights for policy 1, policy_version 95160 (0.0010) [2023-10-07 23:42:20,675][67838] Updated weights for policy 0, policy_version 94982 (0.0008) [2023-10-07 23:42:21,047][67838] Updated weights for policy 0, policy_version 94992 (0.0009) [2023-10-07 23:42:21,418][67838] Updated weights for policy 0, policy_version 95002 (0.0007) [2023-10-07 23:42:22,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 194740224. Throughput: 0: 1653.3, 1: 1669.8. Samples: 48695668. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-07 23:42:22,477][66916] Avg episode reward: [(0, '44.170'), (1, '62.190')] [2023-10-07 23:42:22,490][67871] Updated weights for policy 1, policy_version 95170 (0.0009) [2023-10-07 23:42:22,859][67871] Updated weights for policy 1, policy_version 95180 (0.0010) [2023-10-07 23:42:23,228][67871] Updated weights for policy 1, policy_version 95190 (0.0010) [2023-10-07 23:42:23,598][67871] Updated weights for policy 1, policy_version 95200 (0.0008) [2023-10-07 23:42:25,492][67838] Updated weights for policy 0, policy_version 95012 (0.0008) [2023-10-07 23:42:25,854][67838] Updated weights for policy 0, policy_version 95022 (0.0007) [2023-10-07 23:42:26,223][67838] Updated weights for policy 0, policy_version 95032 (0.0008) [2023-10-07 23:42:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 194805760. Throughput: 0: 1658.0, 1: 1665.8. Samples: 48705802. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-07 23:42:27,478][66916] Avg episode reward: [(0, '41.610'), (1, '61.230')] [2023-10-07 23:42:27,634][67871] Updated weights for policy 1, policy_version 95210 (0.0011) [2023-10-07 23:42:28,001][67871] Updated weights for policy 1, policy_version 95220 (0.0011) [2023-10-07 23:42:28,366][67871] Updated weights for policy 1, policy_version 95230 (0.0009) [2023-10-07 23:42:30,314][67838] Updated weights for policy 0, policy_version 95042 (0.0009) [2023-10-07 23:42:30,679][67838] Updated weights for policy 0, policy_version 95052 (0.0007) [2023-10-07 23:42:31,052][67838] Updated weights for policy 0, policy_version 95062 (0.0007) [2023-10-07 23:42:31,417][67838] Updated weights for policy 0, policy_version 95072 (0.0009) [2023-10-07 23:42:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 194871296. Throughput: 0: 1645.4, 1: 1672.1. Samples: 48725482. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-07 23:42:32,477][66916] Avg episode reward: [(0, '43.100'), (1, '63.560')] [2023-10-07 23:42:32,623][67871] Updated weights for policy 1, policy_version 95240 (0.0008) [2023-10-07 23:42:32,995][67871] Updated weights for policy 1, policy_version 95250 (0.0009) [2023-10-07 23:42:33,361][67871] Updated weights for policy 1, policy_version 95260 (0.0010) [2023-10-07 23:42:35,695][67838] Updated weights for policy 0, policy_version 95082 (0.0009) [2023-10-07 23:42:36,058][67838] Updated weights for policy 0, policy_version 95092 (0.0008) [2023-10-07 23:42:36,417][67838] Updated weights for policy 0, policy_version 95102 (0.0008) [2023-10-07 23:42:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 194936832. Throughput: 0: 1664.0, 1: 1668.9. Samples: 48745452. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-07 23:42:37,477][66916] Avg episode reward: [(0, '40.050'), (1, '64.870')] [2023-10-07 23:42:37,524][67871] Updated weights for policy 1, policy_version 95270 (0.0008) [2023-10-07 23:42:37,895][67871] Updated weights for policy 1, policy_version 95280 (0.0009) [2023-10-07 23:42:38,255][67871] Updated weights for policy 1, policy_version 95290 (0.0009) [2023-10-07 23:42:40,404][67838] Updated weights for policy 0, policy_version 95112 (0.0009) [2023-10-07 23:42:40,776][67838] Updated weights for policy 0, policy_version 95122 (0.0009) [2023-10-07 23:42:41,147][67838] Updated weights for policy 0, policy_version 95132 (0.0010) [2023-10-07 23:42:42,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 195002368. Throughput: 0: 1664.0, 1: 1666.4. Samples: 48755796. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-07 23:42:42,477][66916] Avg episode reward: [(0, '45.190'), (1, '66.970')] [2023-10-07 23:42:42,504][67871] Updated weights for policy 1, policy_version 95300 (0.0008) [2023-10-07 23:42:42,877][67871] Updated weights for policy 1, policy_version 95310 (0.0009) [2023-10-07 23:42:43,241][67871] Updated weights for policy 1, policy_version 95320 (0.0011) [2023-10-07 23:42:45,087][67838] Updated weights for policy 0, policy_version 95142 (0.0009) [2023-10-07 23:42:45,458][67838] Updated weights for policy 0, policy_version 95152 (0.0008) [2023-10-07 23:42:45,829][67838] Updated weights for policy 0, policy_version 95162 (0.0009) [2023-10-07 23:42:47,265][67871] Updated weights for policy 1, policy_version 95330 (0.0010) [2023-10-07 23:42:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195067904. Throughput: 0: 1649.7, 1: 1666.8. Samples: 48775152. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-07 23:42:47,478][66916] Avg episode reward: [(0, '46.890'), (1, '66.090')] [2023-10-07 23:42:47,631][67871] Updated weights for policy 1, policy_version 95340 (0.0010) [2023-10-07 23:42:47,995][67871] Updated weights for policy 1, policy_version 95350 (0.0010) [2023-10-07 23:42:48,362][67871] Updated weights for policy 1, policy_version 95360 (0.0008) [2023-10-07 23:42:49,949][67838] Updated weights for policy 0, policy_version 95172 (0.0009) [2023-10-07 23:42:50,316][67838] Updated weights for policy 0, policy_version 95182 (0.0009) [2023-10-07 23:42:50,698][67838] Updated weights for policy 0, policy_version 95192 (0.0010) [2023-10-07 23:42:52,432][67871] Updated weights for policy 1, policy_version 95370 (0.0009) [2023-10-07 23:42:52,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 195133440. Throughput: 0: 1669.4, 1: 1665.5. Samples: 48795606. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-07 23:42:52,478][66916] Avg episode reward: [(0, '48.280'), (1, '66.930')] [2023-10-07 23:42:52,800][67871] Updated weights for policy 1, policy_version 95380 (0.0010) [2023-10-07 23:42:53,176][67871] Updated weights for policy 1, policy_version 95390 (0.0010) [2023-10-07 23:42:54,858][67838] Updated weights for policy 0, policy_version 95202 (0.0010) [2023-10-07 23:42:55,237][67838] Updated weights for policy 0, policy_version 95212 (0.0010) [2023-10-07 23:42:55,602][67838] Updated weights for policy 0, policy_version 95222 (0.0008) [2023-10-07 23:42:55,969][67838] Updated weights for policy 0, policy_version 95232 (0.0009) [2023-10-07 23:42:57,423][67871] Updated weights for policy 1, policy_version 95400 (0.0009) [2023-10-07 23:42:57,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195198976. Throughput: 0: 1657.0, 1: 1662.2. Samples: 48805340. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-07 23:42:57,478][66916] Avg episode reward: [(0, '47.800'), (1, '69.250')] [2023-10-07 23:42:57,784][67871] Updated weights for policy 1, policy_version 95410 (0.0007) [2023-10-07 23:42:58,157][67871] Updated weights for policy 1, policy_version 95420 (0.0009) [2023-10-07 23:43:00,040][67838] Updated weights for policy 0, policy_version 95242 (0.0009) [2023-10-07 23:43:00,400][67838] Updated weights for policy 0, policy_version 95252 (0.0010) [2023-10-07 23:43:00,783][67838] Updated weights for policy 0, policy_version 95262 (0.0010) [2023-10-07 23:43:02,389][67871] Updated weights for policy 1, policy_version 95430 (0.0010) [2023-10-07 23:43:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195264512. Throughput: 0: 1651.4, 1: 1662.5. Samples: 48824828. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-07 23:43:02,477][66916] Avg episode reward: [(0, '42.850'), (1, '65.840')] [2023-10-07 23:43:02,755][67871] Updated weights for policy 1, policy_version 95440 (0.0011) [2023-10-07 23:43:03,121][67871] Updated weights for policy 1, policy_version 95450 (0.0010) [2023-10-07 23:43:04,995][67838] Updated weights for policy 0, policy_version 95272 (0.0009) [2023-10-07 23:43:05,370][67838] Updated weights for policy 0, policy_version 95282 (0.0009) [2023-10-07 23:43:05,740][67838] Updated weights for policy 0, policy_version 95292 (0.0010) [2023-10-07 23:43:07,235][67871] Updated weights for policy 1, policy_version 95460 (0.0008) [2023-10-07 23:43:07,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195330048. Throughput: 0: 1667.8, 1: 1656.9. Samples: 48845280. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-07 23:43:07,477][66916] Avg episode reward: [(0, '43.200'), (1, '64.700')] [2023-10-07 23:43:07,485][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000095296_97583104.pth... [2023-10-07 23:43:07,517][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000093728_95977472.pth [2023-10-07 23:43:07,596][67871] Updated weights for policy 1, policy_version 95470 (0.0009) [2023-10-07 23:43:07,960][67871] Updated weights for policy 1, policy_version 95480 (0.0009) [2023-10-07 23:43:08,252][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000095488_97779712.pth... [2023-10-07 23:43:08,280][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000093920_96174080.pth [2023-10-07 23:43:09,633][67838] Updated weights for policy 0, policy_version 95302 (0.0008) [2023-10-07 23:43:09,999][67838] Updated weights for policy 0, policy_version 95312 (0.0008) [2023-10-07 23:43:10,364][67838] Updated weights for policy 0, policy_version 95322 (0.0008) [2023-10-07 23:43:12,236][67871] Updated weights for policy 1, policy_version 95490 (0.0008) [2023-10-07 23:43:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195395584. Throughput: 0: 1654.5, 1: 1658.5. Samples: 48854888. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-07 23:43:12,477][66916] Avg episode reward: [(0, '41.100'), (1, '67.450')] [2023-10-07 23:43:12,602][67871] Updated weights for policy 1, policy_version 95500 (0.0007) [2023-10-07 23:43:12,970][67871] Updated weights for policy 1, policy_version 95510 (0.0007) [2023-10-07 23:43:13,328][67871] Updated weights for policy 1, policy_version 95520 (0.0008) [2023-10-07 23:43:14,704][67838] Updated weights for policy 0, policy_version 95332 (0.0008) [2023-10-07 23:43:15,069][67838] Updated weights for policy 0, policy_version 95342 (0.0008) [2023-10-07 23:43:15,437][67838] Updated weights for policy 0, policy_version 95352 (0.0009) [2023-10-07 23:43:17,349][67871] Updated weights for policy 1, policy_version 95530 (0.0010) [2023-10-07 23:43:17,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195461120. Throughput: 0: 1657.7, 1: 1660.8. Samples: 48874818. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-07 23:43:17,477][66916] Avg episode reward: [(0, '44.360'), (1, '63.920')] [2023-10-07 23:43:17,721][67871] Updated weights for policy 1, policy_version 95540 (0.0011) [2023-10-07 23:43:18,083][67871] Updated weights for policy 1, policy_version 95550 (0.0009) [2023-10-07 23:43:19,727][67838] Updated weights for policy 0, policy_version 95362 (0.0008) [2023-10-07 23:43:20,107][67838] Updated weights for policy 0, policy_version 95372 (0.0009) [2023-10-07 23:43:20,471][67838] Updated weights for policy 0, policy_version 95382 (0.0009) [2023-10-07 23:43:20,846][67838] Updated weights for policy 0, policy_version 95392 (0.0009) [2023-10-07 23:43:22,309][67871] Updated weights for policy 1, policy_version 95560 (0.0008) [2023-10-07 23:43:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195526656. Throughput: 0: 1658.8, 1: 1663.9. Samples: 48894974. Policy #0 lag: (min: 13.0, avg: 19.5, max: 45.0) [2023-10-07 23:43:22,477][66916] Avg episode reward: [(0, '47.100'), (1, '61.290')] [2023-10-07 23:43:22,675][67871] Updated weights for policy 1, policy_version 95570 (0.0010) [2023-10-07 23:43:23,041][67871] Updated weights for policy 1, policy_version 95580 (0.0010) [2023-10-07 23:43:24,970][67838] Updated weights for policy 0, policy_version 95402 (0.0008) [2023-10-07 23:43:25,335][67838] Updated weights for policy 0, policy_version 95412 (0.0010) [2023-10-07 23:43:25,709][67838] Updated weights for policy 0, policy_version 95422 (0.0010) [2023-10-07 23:43:27,158][67871] Updated weights for policy 1, policy_version 95590 (0.0009) [2023-10-07 23:43:27,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195592192. Throughput: 0: 1647.8, 1: 1659.7. Samples: 48904636. Policy #0 lag: (min: 13.0, avg: 19.5, max: 45.0) [2023-10-07 23:43:27,478][66916] Avg episode reward: [(0, '41.480'), (1, '60.350')] [2023-10-07 23:43:27,523][67871] Updated weights for policy 1, policy_version 95600 (0.0009) [2023-10-07 23:43:27,892][67871] Updated weights for policy 1, policy_version 95610 (0.0009) [2023-10-07 23:43:29,804][67838] Updated weights for policy 0, policy_version 95432 (0.0007) [2023-10-07 23:43:30,168][67838] Updated weights for policy 0, policy_version 95442 (0.0008) [2023-10-07 23:43:30,536][67838] Updated weights for policy 0, policy_version 95452 (0.0009) [2023-10-07 23:43:31,976][67871] Updated weights for policy 1, policy_version 95620 (0.0009) [2023-10-07 23:43:32,341][67871] Updated weights for policy 1, policy_version 95630 (0.0011) [2023-10-07 23:43:32,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195657728. Throughput: 0: 1655.6, 1: 1658.2. Samples: 48924270. Policy #0 lag: (min: 13.0, avg: 19.5, max: 45.0) [2023-10-07 23:43:32,478][66916] Avg episode reward: [(0, '40.090'), (1, '61.620')] [2023-10-07 23:43:32,715][67871] Updated weights for policy 1, policy_version 95640 (0.0008) [2023-10-07 23:43:34,525][67838] Updated weights for policy 0, policy_version 95462 (0.0009) [2023-10-07 23:43:34,901][67838] Updated weights for policy 0, policy_version 95472 (0.0009) [2023-10-07 23:43:35,268][67838] Updated weights for policy 0, policy_version 95482 (0.0007) [2023-10-07 23:43:36,813][67871] Updated weights for policy 1, policy_version 95650 (0.0008) [2023-10-07 23:43:37,167][67871] Updated weights for policy 1, policy_version 95660 (0.0009) [2023-10-07 23:43:37,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195723264. Throughput: 0: 1660.1, 1: 1655.0. Samples: 48944786. Policy #0 lag: (min: 13.0, avg: 19.5, max: 45.0) [2023-10-07 23:43:37,477][66916] Avg episode reward: [(0, '42.930'), (1, '60.140')] [2023-10-07 23:43:37,537][67871] Updated weights for policy 1, policy_version 95670 (0.0009) [2023-10-07 23:43:37,901][67871] Updated weights for policy 1, policy_version 95680 (0.0008) [2023-10-07 23:43:39,411][67838] Updated weights for policy 0, policy_version 95492 (0.0008) [2023-10-07 23:43:39,787][67838] Updated weights for policy 0, policy_version 95502 (0.0008) [2023-10-07 23:43:40,147][67838] Updated weights for policy 0, policy_version 95512 (0.0007) [2023-10-07 23:43:42,087][67871] Updated weights for policy 1, policy_version 95690 (0.0009) [2023-10-07 23:43:42,448][67871] Updated weights for policy 1, policy_version 95700 (0.0008) [2023-10-07 23:43:42,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195788800. Throughput: 0: 1650.0, 1: 1661.7. Samples: 48954368. Policy #0 lag: (min: 13.0, avg: 19.5, max: 45.0) [2023-10-07 23:43:42,477][66916] Avg episode reward: [(0, '40.200'), (1, '62.400')] [2023-10-07 23:43:42,815][67871] Updated weights for policy 1, policy_version 95710 (0.0009) [2023-10-07 23:43:44,456][67838] Updated weights for policy 0, policy_version 95522 (0.0010) [2023-10-07 23:43:44,830][67838] Updated weights for policy 0, policy_version 95532 (0.0010) [2023-10-07 23:43:45,214][67838] Updated weights for policy 0, policy_version 95542 (0.0011) [2023-10-07 23:43:45,576][67838] Updated weights for policy 0, policy_version 95552 (0.0010) [2023-10-07 23:43:46,864][67871] Updated weights for policy 1, policy_version 95720 (0.0009) [2023-10-07 23:43:47,236][67871] Updated weights for policy 1, policy_version 95730 (0.0009) [2023-10-07 23:43:47,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195854336. Throughput: 0: 1651.5, 1: 1665.0. Samples: 48974072. Policy #0 lag: (min: 13.0, avg: 19.5, max: 45.0) [2023-10-07 23:43:47,478][66916] Avg episode reward: [(0, '44.550'), (1, '64.590')] [2023-10-07 23:43:47,610][67871] Updated weights for policy 1, policy_version 95740 (0.0009) [2023-10-07 23:43:49,636][67838] Updated weights for policy 0, policy_version 95562 (0.0010) [2023-10-07 23:43:50,009][67838] Updated weights for policy 0, policy_version 95572 (0.0009) [2023-10-07 23:43:50,374][67838] Updated weights for policy 0, policy_version 95582 (0.0009) [2023-10-07 23:43:51,572][67871] Updated weights for policy 1, policy_version 95750 (0.0009) [2023-10-07 23:43:51,933][67871] Updated weights for policy 1, policy_version 95760 (0.0008) [2023-10-07 23:43:52,289][67871] Updated weights for policy 1, policy_version 95770 (0.0007) [2023-10-07 23:43:52,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 195919872. Throughput: 0: 1650.9, 1: 1660.1. Samples: 48994278. Policy #0 lag: (min: 13.0, avg: 19.5, max: 45.0) [2023-10-07 23:43:52,478][66916] Avg episode reward: [(0, '47.210'), (1, '67.630')] [2023-10-07 23:43:54,483][67838] Updated weights for policy 0, policy_version 95592 (0.0009) [2023-10-07 23:43:54,854][67838] Updated weights for policy 0, policy_version 95602 (0.0009) [2023-10-07 23:43:55,222][67838] Updated weights for policy 0, policy_version 95612 (0.0008) [2023-10-07 23:43:56,475][67871] Updated weights for policy 1, policy_version 95780 (0.0008) [2023-10-07 23:43:56,842][67871] Updated weights for policy 1, policy_version 95790 (0.0009) [2023-10-07 23:43:57,203][67871] Updated weights for policy 1, policy_version 95800 (0.0010) [2023-10-07 23:43:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 195985408. Throughput: 0: 1644.4, 1: 1675.2. Samples: 49004270. Policy #0 lag: (min: 13.0, avg: 19.5, max: 45.0) [2023-10-07 23:43:57,477][66916] Avg episode reward: [(0, '49.580'), (1, '66.290')] [2023-10-07 23:43:59,386][67838] Updated weights for policy 0, policy_version 95622 (0.0008) [2023-10-07 23:43:59,761][67838] Updated weights for policy 0, policy_version 95632 (0.0008) [2023-10-07 23:44:00,123][67838] Updated weights for policy 0, policy_version 95642 (0.0008) [2023-10-07 23:44:01,471][67871] Updated weights for policy 1, policy_version 95810 (0.0010) [2023-10-07 23:44:01,836][67871] Updated weights for policy 1, policy_version 95820 (0.0009) [2023-10-07 23:44:02,200][67871] Updated weights for policy 1, policy_version 95830 (0.0009) [2023-10-07 23:44:02,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 196050944. Throughput: 0: 1651.3, 1: 1673.5. Samples: 49024434. Policy #0 lag: (min: 13.0, avg: 19.5, max: 45.0) [2023-10-07 23:44:02,477][66916] Avg episode reward: [(0, '52.570'), (1, '66.280')] [2023-10-07 23:44:02,568][67871] Updated weights for policy 1, policy_version 95840 (0.0007) [2023-10-07 23:44:04,274][67838] Updated weights for policy 0, policy_version 95652 (0.0009) [2023-10-07 23:44:04,645][67838] Updated weights for policy 0, policy_version 95662 (0.0008) [2023-10-07 23:44:05,020][67838] Updated weights for policy 0, policy_version 95672 (0.0007) [2023-10-07 23:44:06,662][67871] Updated weights for policy 1, policy_version 95850 (0.0011) [2023-10-07 23:44:07,019][67871] Updated weights for policy 1, policy_version 95860 (0.0009) [2023-10-07 23:44:07,395][67871] Updated weights for policy 1, policy_version 95870 (0.0009) [2023-10-07 23:44:07,477][66916] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 196149248. Throughput: 0: 1658.1, 1: 1655.8. Samples: 49044102. Policy #0 lag: (min: 13.0, avg: 19.5, max: 45.0) [2023-10-07 23:44:07,478][66916] Avg episode reward: [(0, '45.010'), (1, '62.020')] [2023-10-07 23:44:09,325][67838] Updated weights for policy 0, policy_version 95682 (0.0008) [2023-10-07 23:44:09,737][67838] Updated weights for policy 0, policy_version 95692 (0.0010) [2023-10-07 23:44:10,111][67838] Updated weights for policy 0, policy_version 95702 (0.0009) [2023-10-07 23:44:10,473][67838] Updated weights for policy 0, policy_version 95712 (0.0010) [2023-10-07 23:44:11,534][67871] Updated weights for policy 1, policy_version 95880 (0.0007) [2023-10-07 23:44:11,905][67871] Updated weights for policy 1, policy_version 95890 (0.0008) [2023-10-07 23:44:12,272][67871] Updated weights for policy 1, policy_version 95900 (0.0009) [2023-10-07 23:44:12,476][66916] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 196214784. Throughput: 0: 1649.5, 1: 1670.6. Samples: 49054038. Policy #0 lag: (min: 13.0, avg: 19.5, max: 45.0) [2023-10-07 23:44:12,477][66916] Avg episode reward: [(0, '46.060'), (1, '59.080')] [2023-10-07 23:44:14,495][67838] Updated weights for policy 0, policy_version 95722 (0.0008) [2023-10-07 23:44:14,870][67838] Updated weights for policy 0, policy_version 95732 (0.0009) [2023-10-07 23:44:15,236][67838] Updated weights for policy 0, policy_version 95742 (0.0007) [2023-10-07 23:44:16,483][67871] Updated weights for policy 1, policy_version 95910 (0.0010) [2023-10-07 23:44:16,850][67871] Updated weights for policy 1, policy_version 95920 (0.0007) [2023-10-07 23:44:17,221][67871] Updated weights for policy 1, policy_version 95930 (0.0007) [2023-10-07 23:44:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 196280320. Throughput: 0: 1659.4, 1: 1666.5. Samples: 49073936. Policy #0 lag: (min: 13.0, avg: 19.5, max: 45.0) [2023-10-07 23:44:17,477][66916] Avg episode reward: [(0, '47.610'), (1, '58.470')] [2023-10-07 23:44:19,156][67838] Updated weights for policy 0, policy_version 95752 (0.0009) [2023-10-07 23:44:19,541][67838] Updated weights for policy 0, policy_version 95762 (0.0011) [2023-10-07 23:44:19,911][67838] Updated weights for policy 0, policy_version 95772 (0.0010) [2023-10-07 23:44:21,445][67871] Updated weights for policy 1, policy_version 95940 (0.0008) [2023-10-07 23:44:21,801][67871] Updated weights for policy 1, policy_version 95950 (0.0009) [2023-10-07 23:44:22,170][67871] Updated weights for policy 1, policy_version 95960 (0.0009) [2023-10-07 23:44:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 196345856. Throughput: 0: 1656.2, 1: 1650.4. Samples: 49093584. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-07 23:44:22,478][66916] Avg episode reward: [(0, '46.820'), (1, '57.510')] [2023-10-07 23:44:24,072][67838] Updated weights for policy 0, policy_version 95782 (0.0010) [2023-10-07 23:44:24,457][67838] Updated weights for policy 0, policy_version 95792 (0.0008) [2023-10-07 23:44:24,831][67838] Updated weights for policy 0, policy_version 95802 (0.0008) [2023-10-07 23:44:26,327][67871] Updated weights for policy 1, policy_version 95970 (0.0007) [2023-10-07 23:44:26,689][67871] Updated weights for policy 1, policy_version 95980 (0.0007) [2023-10-07 23:44:27,057][67871] Updated weights for policy 1, policy_version 95990 (0.0007) [2023-10-07 23:44:27,431][67871] Updated weights for policy 1, policy_version 96000 (0.0007) [2023-10-07 23:44:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 196411392. Throughput: 0: 1648.4, 1: 1659.2. Samples: 49103210. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-07 23:44:27,477][66916] Avg episode reward: [(0, '46.780'), (1, '59.170')] [2023-10-07 23:44:28,976][67838] Updated weights for policy 0, policy_version 95812 (0.0009) [2023-10-07 23:44:29,348][67838] Updated weights for policy 0, policy_version 95822 (0.0008) [2023-10-07 23:44:29,729][67838] Updated weights for policy 0, policy_version 95832 (0.0008) [2023-10-07 23:44:31,387][67871] Updated weights for policy 1, policy_version 96010 (0.0009) [2023-10-07 23:44:31,745][67871] Updated weights for policy 1, policy_version 96020 (0.0009) [2023-10-07 23:44:32,118][67871] Updated weights for policy 1, policy_version 96030 (0.0009) [2023-10-07 23:44:32,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 196476928. Throughput: 0: 1666.1, 1: 1656.0. Samples: 49123562. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-07 23:44:32,477][66916] Avg episode reward: [(0, '46.950'), (1, '60.450')] [2023-10-07 23:44:33,562][67838] Updated weights for policy 0, policy_version 95842 (0.0009) [2023-10-07 23:44:33,933][67838] Updated weights for policy 0, policy_version 95852 (0.0009) [2023-10-07 23:44:34,307][67838] Updated weights for policy 0, policy_version 95862 (0.0008) [2023-10-07 23:44:34,679][67838] Updated weights for policy 0, policy_version 95872 (0.0008) [2023-10-07 23:44:36,384][67871] Updated weights for policy 1, policy_version 96040 (0.0007) [2023-10-07 23:44:36,744][67871] Updated weights for policy 1, policy_version 96050 (0.0008) [2023-10-07 23:44:37,115][67871] Updated weights for policy 1, policy_version 96060 (0.0011) [2023-10-07 23:44:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 196542464. Throughput: 0: 1672.1, 1: 1647.0. Samples: 49143636. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-07 23:44:37,478][66916] Avg episode reward: [(0, '43.950'), (1, '63.390')] [2023-10-07 23:44:38,692][67838] Updated weights for policy 0, policy_version 95882 (0.0009) [2023-10-07 23:44:39,067][67838] Updated weights for policy 0, policy_version 95892 (0.0008) [2023-10-07 23:44:39,442][67838] Updated weights for policy 0, policy_version 95902 (0.0007) [2023-10-07 23:44:41,267][67871] Updated weights for policy 1, policy_version 96070 (0.0009) [2023-10-07 23:44:41,632][67871] Updated weights for policy 1, policy_version 96080 (0.0009) [2023-10-07 23:44:42,003][67871] Updated weights for policy 1, policy_version 96090 (0.0008) [2023-10-07 23:44:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 196608000. Throughput: 0: 1665.2, 1: 1653.7. Samples: 49153620. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-07 23:44:42,478][66916] Avg episode reward: [(0, '45.740'), (1, '63.920')] [2023-10-07 23:44:43,666][67838] Updated weights for policy 0, policy_version 95912 (0.0007) [2023-10-07 23:44:44,031][67838] Updated weights for policy 0, policy_version 95922 (0.0011) [2023-10-07 23:44:44,396][67838] Updated weights for policy 0, policy_version 95932 (0.0009) [2023-10-07 23:44:45,993][67871] Updated weights for policy 1, policy_version 96100 (0.0008) [2023-10-07 23:44:46,367][67871] Updated weights for policy 1, policy_version 96110 (0.0007) [2023-10-07 23:44:46,741][67871] Updated weights for policy 1, policy_version 96120 (0.0011) [2023-10-07 23:44:47,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 196673536. Throughput: 0: 1671.3, 1: 1651.4. Samples: 49173954. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-07 23:44:47,477][66916] Avg episode reward: [(0, '46.480'), (1, '69.000')] [2023-10-07 23:44:48,519][67838] Updated weights for policy 0, policy_version 95942 (0.0009) [2023-10-07 23:44:48,893][67838] Updated weights for policy 0, policy_version 95952 (0.0008) [2023-10-07 23:44:49,271][67838] Updated weights for policy 0, policy_version 95962 (0.0007) [2023-10-07 23:44:50,902][67871] Updated weights for policy 1, policy_version 96130 (0.0010) [2023-10-07 23:44:51,273][67871] Updated weights for policy 1, policy_version 96140 (0.0011) [2023-10-07 23:44:51,632][67871] Updated weights for policy 1, policy_version 96150 (0.0009) [2023-10-07 23:44:52,005][67871] Updated weights for policy 1, policy_version 96160 (0.0009) [2023-10-07 23:44:52,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 196739072. Throughput: 0: 1672.0, 1: 1643.5. Samples: 49193298. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-07 23:44:52,477][66916] Avg episode reward: [(0, '49.090'), (1, '65.840')] [2023-10-07 23:44:53,706][67838] Updated weights for policy 0, policy_version 95972 (0.0007) [2023-10-07 23:44:54,079][67838] Updated weights for policy 0, policy_version 95982 (0.0009) [2023-10-07 23:44:54,458][67838] Updated weights for policy 0, policy_version 95992 (0.0010) [2023-10-07 23:44:56,161][67871] Updated weights for policy 1, policy_version 96170 (0.0009) [2023-10-07 23:44:56,535][67871] Updated weights for policy 1, policy_version 96180 (0.0009) [2023-10-07 23:44:56,899][67871] Updated weights for policy 1, policy_version 96190 (0.0011) [2023-10-07 23:44:57,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 196804608. Throughput: 0: 1662.9, 1: 1655.9. Samples: 49203386. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-07 23:44:57,478][66916] Avg episode reward: [(0, '51.180'), (1, '69.010')] [2023-10-07 23:44:58,402][67838] Updated weights for policy 0, policy_version 96002 (0.0009) [2023-10-07 23:44:58,809][67838] Updated weights for policy 0, policy_version 96012 (0.0009) [2023-10-07 23:44:59,173][67838] Updated weights for policy 0, policy_version 96022 (0.0008) [2023-10-07 23:44:59,542][67838] Updated weights for policy 0, policy_version 96032 (0.0009) [2023-10-07 23:45:00,808][67871] Updated weights for policy 1, policy_version 96200 (0.0007) [2023-10-07 23:45:01,171][67871] Updated weights for policy 1, policy_version 96210 (0.0008) [2023-10-07 23:45:01,539][67871] Updated weights for policy 1, policy_version 96220 (0.0010) [2023-10-07 23:45:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 196870144. Throughput: 0: 1667.6, 1: 1656.8. Samples: 49223536. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-07 23:45:02,477][66916] Avg episode reward: [(0, '49.080'), (1, '68.120')] [2023-10-07 23:45:03,486][67838] Updated weights for policy 0, policy_version 96042 (0.0009) [2023-10-07 23:45:03,857][67838] Updated weights for policy 0, policy_version 96052 (0.0010) [2023-10-07 23:45:04,231][67838] Updated weights for policy 0, policy_version 96062 (0.0007) [2023-10-07 23:45:05,730][67871] Updated weights for policy 1, policy_version 96230 (0.0010) [2023-10-07 23:45:06,094][67871] Updated weights for policy 1, policy_version 96240 (0.0008) [2023-10-07 23:45:06,464][67871] Updated weights for policy 1, policy_version 96250 (0.0009) [2023-10-07 23:45:07,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 196935680. Throughput: 0: 1665.6, 1: 1649.9. Samples: 49242780. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-07 23:45:07,477][66916] Avg episode reward: [(0, '49.940'), (1, '63.650')] [2023-10-07 23:45:07,489][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000096256_98566144.pth... [2023-10-07 23:45:07,489][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000096064_98369536.pth... [2023-10-07 23:45:07,526][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000094528_96796672.pth [2023-10-07 23:45:07,528][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000094688_96960512.pth [2023-10-07 23:45:08,553][67838] Updated weights for policy 0, policy_version 96072 (0.0008) [2023-10-07 23:45:08,928][67838] Updated weights for policy 0, policy_version 96082 (0.0008) [2023-10-07 23:45:09,298][67838] Updated weights for policy 0, policy_version 96092 (0.0007) [2023-10-07 23:45:10,601][67871] Updated weights for policy 1, policy_version 96260 (0.0008) [2023-10-07 23:45:10,967][67871] Updated weights for policy 1, policy_version 96270 (0.0007) [2023-10-07 23:45:11,326][67871] Updated weights for policy 1, policy_version 96280 (0.0009) [2023-10-07 23:45:12,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 197001216. Throughput: 0: 1663.5, 1: 1667.2. Samples: 49253090. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-07 23:45:12,477][66916] Avg episode reward: [(0, '49.880'), (1, '62.120')] [2023-10-07 23:45:13,516][67838] Updated weights for policy 0, policy_version 96102 (0.0008) [2023-10-07 23:45:13,896][67838] Updated weights for policy 0, policy_version 96112 (0.0009) [2023-10-07 23:45:14,273][67838] Updated weights for policy 0, policy_version 96122 (0.0010) [2023-10-07 23:45:15,719][67871] Updated weights for policy 1, policy_version 96290 (0.0010) [2023-10-07 23:45:16,091][67871] Updated weights for policy 1, policy_version 96300 (0.0008) [2023-10-07 23:45:16,453][67871] Updated weights for policy 1, policy_version 96310 (0.0010) [2023-10-07 23:45:16,822][67871] Updated weights for policy 1, policy_version 96320 (0.0008) [2023-10-07 23:45:17,477][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 197066752. Throughput: 0: 1661.1, 1: 1662.0. Samples: 49273100. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-07 23:45:17,478][66916] Avg episode reward: [(0, '48.090'), (1, '64.880')] [2023-10-07 23:45:18,315][67838] Updated weights for policy 0, policy_version 96132 (0.0008) [2023-10-07 23:45:18,683][67838] Updated weights for policy 0, policy_version 96142 (0.0008) [2023-10-07 23:45:19,046][67838] Updated weights for policy 0, policy_version 96152 (0.0007) [2023-10-07 23:45:21,078][67871] Updated weights for policy 1, policy_version 96330 (0.0009) [2023-10-07 23:45:21,440][67871] Updated weights for policy 1, policy_version 96340 (0.0008) [2023-10-07 23:45:21,807][67871] Updated weights for policy 1, policy_version 96350 (0.0010) [2023-10-07 23:45:22,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 197132288. Throughput: 0: 1652.5, 1: 1652.3. Samples: 49292352. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:45:22,477][66916] Avg episode reward: [(0, '51.170'), (1, '63.970')] [2023-10-07 23:45:23,139][67838] Updated weights for policy 0, policy_version 96162 (0.0007) [2023-10-07 23:45:23,512][67838] Updated weights for policy 0, policy_version 96172 (0.0007) [2023-10-07 23:45:23,886][67838] Updated weights for policy 0, policy_version 96182 (0.0008) [2023-10-07 23:45:24,267][67838] Updated weights for policy 0, policy_version 96192 (0.0009) [2023-10-07 23:45:25,819][67871] Updated weights for policy 1, policy_version 96360 (0.0009) [2023-10-07 23:45:26,186][67871] Updated weights for policy 1, policy_version 96370 (0.0008) [2023-10-07 23:45:26,544][67871] Updated weights for policy 1, policy_version 96380 (0.0008) [2023-10-07 23:45:27,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 197197824. Throughput: 0: 1645.2, 1: 1661.3. Samples: 49302410. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:45:27,478][66916] Avg episode reward: [(0, '50.380'), (1, '60.550')] [2023-10-07 23:45:28,516][67838] Updated weights for policy 0, policy_version 96202 (0.0009) [2023-10-07 23:45:28,885][67838] Updated weights for policy 0, policy_version 96212 (0.0009) [2023-10-07 23:45:29,264][67838] Updated weights for policy 0, policy_version 96222 (0.0011) [2023-10-07 23:45:30,582][67871] Updated weights for policy 1, policy_version 96390 (0.0007) [2023-10-07 23:45:30,940][67871] Updated weights for policy 1, policy_version 96400 (0.0009) [2023-10-07 23:45:31,313][67871] Updated weights for policy 1, policy_version 96410 (0.0008) [2023-10-07 23:45:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 197263360. Throughput: 0: 1647.8, 1: 1654.4. Samples: 49322550. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:45:32,477][66916] Avg episode reward: [(0, '47.040'), (1, '60.850')] [2023-10-07 23:45:33,379][67838] Updated weights for policy 0, policy_version 96232 (0.0010) [2023-10-07 23:45:33,753][67838] Updated weights for policy 0, policy_version 96242 (0.0009) [2023-10-07 23:45:34,115][67838] Updated weights for policy 0, policy_version 96252 (0.0009) [2023-10-07 23:45:35,153][67871] Updated weights for policy 1, policy_version 96420 (0.0008) [2023-10-07 23:45:35,514][67871] Updated weights for policy 1, policy_version 96430 (0.0009) [2023-10-07 23:45:35,883][67871] Updated weights for policy 1, policy_version 96440 (0.0011) [2023-10-07 23:45:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 197328896. Throughput: 0: 1651.4, 1: 1666.5. Samples: 49342604. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:45:37,477][66916] Avg episode reward: [(0, '46.640'), (1, '63.420')] [2023-10-07 23:45:38,327][67838] Updated weights for policy 0, policy_version 96262 (0.0008) [2023-10-07 23:45:38,688][67838] Updated weights for policy 0, policy_version 96272 (0.0007) [2023-10-07 23:45:39,059][67838] Updated weights for policy 0, policy_version 96282 (0.0007) [2023-10-07 23:45:40,073][67871] Updated weights for policy 1, policy_version 96450 (0.0007) [2023-10-07 23:45:40,433][67871] Updated weights for policy 1, policy_version 96460 (0.0008) [2023-10-07 23:45:40,798][67871] Updated weights for policy 1, policy_version 96470 (0.0009) [2023-10-07 23:45:41,162][67871] Updated weights for policy 1, policy_version 96480 (0.0007) [2023-10-07 23:45:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 197394432. Throughput: 0: 1649.2, 1: 1670.9. Samples: 49352790. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:45:42,477][66916] Avg episode reward: [(0, '46.020'), (1, '62.800')] [2023-10-07 23:45:43,345][67838] Updated weights for policy 0, policy_version 96292 (0.0007) [2023-10-07 23:45:43,713][67838] Updated weights for policy 0, policy_version 96302 (0.0007) [2023-10-07 23:45:44,091][67838] Updated weights for policy 0, policy_version 96312 (0.0008) [2023-10-07 23:45:45,418][67871] Updated weights for policy 1, policy_version 96490 (0.0009) [2023-10-07 23:45:45,789][67871] Updated weights for policy 1, policy_version 96500 (0.0007) [2023-10-07 23:45:46,158][67871] Updated weights for policy 1, policy_version 96510 (0.0008) [2023-10-07 23:45:47,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 197459968. Throughput: 0: 1651.7, 1: 1655.8. Samples: 49372374. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:45:47,477][66916] Avg episode reward: [(0, '43.920'), (1, '67.800')] [2023-10-07 23:45:48,176][67838] Updated weights for policy 0, policy_version 96322 (0.0009) [2023-10-07 23:45:48,549][67838] Updated weights for policy 0, policy_version 96332 (0.0010) [2023-10-07 23:45:48,927][67838] Updated weights for policy 0, policy_version 96342 (0.0009) [2023-10-07 23:45:49,308][67838] Updated weights for policy 0, policy_version 96352 (0.0009) [2023-10-07 23:45:50,253][67871] Updated weights for policy 1, policy_version 96520 (0.0008) [2023-10-07 23:45:50,617][67871] Updated weights for policy 1, policy_version 96530 (0.0011) [2023-10-07 23:45:50,986][67871] Updated weights for policy 1, policy_version 96540 (0.0007) [2023-10-07 23:45:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 197525504. Throughput: 0: 1655.3, 1: 1668.4. Samples: 49392346. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:45:52,477][66916] Avg episode reward: [(0, '47.830'), (1, '68.800')] [2023-10-07 23:45:53,393][67838] Updated weights for policy 0, policy_version 96362 (0.0010) [2023-10-07 23:45:53,770][67838] Updated weights for policy 0, policy_version 96372 (0.0009) [2023-10-07 23:45:54,140][67838] Updated weights for policy 0, policy_version 96382 (0.0011) [2023-10-07 23:45:55,012][67871] Updated weights for policy 1, policy_version 96550 (0.0009) [2023-10-07 23:45:55,373][67871] Updated weights for policy 1, policy_version 96560 (0.0009) [2023-10-07 23:45:55,739][67871] Updated weights for policy 1, policy_version 96570 (0.0008) [2023-10-07 23:45:57,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 197591040. Throughput: 0: 1655.1, 1: 1673.1. Samples: 49402860. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:45:57,478][66916] Avg episode reward: [(0, '50.770'), (1, '65.760')] [2023-10-07 23:45:58,242][67838] Updated weights for policy 0, policy_version 96392 (0.0008) [2023-10-07 23:45:58,613][67838] Updated weights for policy 0, policy_version 96402 (0.0009) [2023-10-07 23:45:58,976][67838] Updated weights for policy 0, policy_version 96412 (0.0008) [2023-10-07 23:45:59,862][67871] Updated weights for policy 1, policy_version 96580 (0.0009) [2023-10-07 23:46:00,222][67871] Updated weights for policy 1, policy_version 96590 (0.0008) [2023-10-07 23:46:00,587][67871] Updated weights for policy 1, policy_version 96600 (0.0008) [2023-10-07 23:46:02,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 197656576. Throughput: 0: 1658.0, 1: 1653.8. Samples: 49422130. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:46:02,478][66916] Avg episode reward: [(0, '53.460'), (1, '69.080')] [2023-10-07 23:46:02,940][67838] Updated weights for policy 0, policy_version 96422 (0.0008) [2023-10-07 23:46:03,306][67838] Updated weights for policy 0, policy_version 96432 (0.0008) [2023-10-07 23:46:03,672][67838] Updated weights for policy 0, policy_version 96442 (0.0008) [2023-10-07 23:46:04,525][67871] Updated weights for policy 1, policy_version 96610 (0.0007) [2023-10-07 23:46:04,883][67871] Updated weights for policy 1, policy_version 96620 (0.0008) [2023-10-07 23:46:05,252][67871] Updated weights for policy 1, policy_version 96630 (0.0007) [2023-10-07 23:46:05,613][67871] Updated weights for policy 1, policy_version 96640 (0.0008) [2023-10-07 23:46:07,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 197722112. Throughput: 0: 1661.2, 1: 1683.4. Samples: 49442860. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:46:07,478][66916] Avg episode reward: [(0, '53.110'), (1, '65.900')] [2023-10-07 23:46:07,831][67838] Updated weights for policy 0, policy_version 96452 (0.0009) [2023-10-07 23:46:08,195][67838] Updated weights for policy 0, policy_version 96462 (0.0010) [2023-10-07 23:46:08,566][67838] Updated weights for policy 0, policy_version 96472 (0.0008) [2023-10-07 23:46:09,626][67871] Updated weights for policy 1, policy_version 96650 (0.0008) [2023-10-07 23:46:10,007][67871] Updated weights for policy 1, policy_version 96660 (0.0008) [2023-10-07 23:46:10,380][67871] Updated weights for policy 1, policy_version 96670 (0.0008) [2023-10-07 23:46:12,477][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 197787648. Throughput: 0: 1662.8, 1: 1670.4. Samples: 49452404. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:46:12,478][66916] Avg episode reward: [(0, '55.790'), (1, '67.050')] [2023-10-07 23:46:12,703][67838] Updated weights for policy 0, policy_version 96482 (0.0008) [2023-10-07 23:46:13,080][67838] Updated weights for policy 0, policy_version 96492 (0.0009) [2023-10-07 23:46:13,462][67838] Updated weights for policy 0, policy_version 96502 (0.0007) [2023-10-07 23:46:13,832][67838] Updated weights for policy 0, policy_version 96512 (0.0008) [2023-10-07 23:46:14,424][67871] Updated weights for policy 1, policy_version 96680 (0.0009) [2023-10-07 23:46:14,789][67871] Updated weights for policy 1, policy_version 96690 (0.0007) [2023-10-07 23:46:15,161][67871] Updated weights for policy 1, policy_version 96700 (0.0007) [2023-10-07 23:46:17,476][66916] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 197853184. Throughput: 0: 1662.7, 1: 1663.4. Samples: 49472224. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-07 23:46:17,477][66916] Avg episode reward: [(0, '52.360'), (1, '63.870')] [2023-10-07 23:46:17,999][67838] Updated weights for policy 0, policy_version 96522 (0.0010) [2023-10-07 23:46:18,382][67838] Updated weights for policy 0, policy_version 96532 (0.0010) [2023-10-07 23:46:18,748][67838] Updated weights for policy 0, policy_version 96542 (0.0008) [2023-10-07 23:46:19,271][67871] Updated weights for policy 1, policy_version 96710 (0.0008) [2023-10-07 23:46:19,632][67871] Updated weights for policy 1, policy_version 96720 (0.0007) [2023-10-07 23:46:19,997][67871] Updated weights for policy 1, policy_version 96730 (0.0007) [2023-10-07 23:46:22,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 197918720. Throughput: 0: 1655.3, 1: 1672.0. Samples: 49492334. Policy #0 lag: (min: 21.0, avg: 28.3, max: 53.0) [2023-10-07 23:46:22,477][66916] Avg episode reward: [(0, '48.170'), (1, '66.100')] [2023-10-07 23:46:22,945][67838] Updated weights for policy 0, policy_version 96552 (0.0007) [2023-10-07 23:46:23,317][67838] Updated weights for policy 0, policy_version 96562 (0.0007) [2023-10-07 23:46:23,700][67838] Updated weights for policy 0, policy_version 96572 (0.0011) [2023-10-07 23:46:24,191][67871] Updated weights for policy 1, policy_version 96740 (0.0008) [2023-10-07 23:46:24,552][67871] Updated weights for policy 1, policy_version 96750 (0.0008) [2023-10-07 23:46:24,921][67871] Updated weights for policy 1, policy_version 96760 (0.0009) [2023-10-07 23:46:27,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 197984256. Throughput: 0: 1658.8, 1: 1653.1. Samples: 49501828. Policy #0 lag: (min: 21.0, avg: 28.3, max: 53.0) [2023-10-07 23:46:27,478][66916] Avg episode reward: [(0, '48.710'), (1, '65.670')] [2023-10-07 23:46:27,745][67838] Updated weights for policy 0, policy_version 96582 (0.0009) [2023-10-07 23:46:28,121][67838] Updated weights for policy 0, policy_version 96592 (0.0009) [2023-10-07 23:46:28,485][67838] Updated weights for policy 0, policy_version 96602 (0.0007) [2023-10-07 23:46:29,036][67871] Updated weights for policy 1, policy_version 96770 (0.0007) [2023-10-07 23:46:29,408][67871] Updated weights for policy 1, policy_version 96780 (0.0009) [2023-10-07 23:46:29,769][67871] Updated weights for policy 1, policy_version 96790 (0.0010) [2023-10-07 23:46:30,133][67871] Updated weights for policy 1, policy_version 96800 (0.0011) [2023-10-07 23:46:32,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 198049792. Throughput: 0: 1661.2, 1: 1661.2. Samples: 49521886. Policy #0 lag: (min: 21.0, avg: 28.3, max: 53.0) [2023-10-07 23:46:32,477][66916] Avg episode reward: [(0, '48.460'), (1, '69.610')] [2023-10-07 23:46:32,747][67838] Updated weights for policy 0, policy_version 96612 (0.0008) [2023-10-07 23:46:33,113][67838] Updated weights for policy 0, policy_version 96622 (0.0007) [2023-10-07 23:46:33,485][67838] Updated weights for policy 0, policy_version 96632 (0.0008) [2023-10-07 23:46:34,335][67871] Updated weights for policy 1, policy_version 96810 (0.0010) [2023-10-07 23:46:34,701][67871] Updated weights for policy 1, policy_version 96820 (0.0009) [2023-10-07 23:46:35,078][67871] Updated weights for policy 1, policy_version 96830 (0.0008) [2023-10-07 23:46:37,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 198115328. Throughput: 0: 1657.6, 1: 1677.4. Samples: 49542418. Policy #0 lag: (min: 21.0, avg: 28.3, max: 53.0) [2023-10-07 23:46:37,477][66916] Avg episode reward: [(0, '49.620'), (1, '70.000')] [2023-10-07 23:46:37,486][67676] Saving new best policy, reward=70.000! [2023-10-07 23:46:37,697][67838] Updated weights for policy 0, policy_version 96642 (0.0011) [2023-10-07 23:46:38,090][67838] Updated weights for policy 0, policy_version 96652 (0.0008) [2023-10-07 23:46:38,461][67838] Updated weights for policy 0, policy_version 96662 (0.0008) [2023-10-07 23:46:38,838][67838] Updated weights for policy 0, policy_version 96672 (0.0010) [2023-10-07 23:46:39,168][67871] Updated weights for policy 1, policy_version 96840 (0.0008) [2023-10-07 23:46:39,523][67871] Updated weights for policy 1, policy_version 96850 (0.0008) [2023-10-07 23:46:39,893][67871] Updated weights for policy 1, policy_version 96860 (0.0007) [2023-10-07 23:46:42,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 198180864. Throughput: 0: 1658.1, 1: 1649.2. Samples: 49551684. Policy #0 lag: (min: 21.0, avg: 28.3, max: 53.0) [2023-10-07 23:46:42,477][66916] Avg episode reward: [(0, '49.140'), (1, '66.840')] [2023-10-07 23:46:42,792][67838] Updated weights for policy 0, policy_version 96682 (0.0009) [2023-10-07 23:46:43,155][67838] Updated weights for policy 0, policy_version 96692 (0.0010) [2023-10-07 23:46:43,529][67838] Updated weights for policy 0, policy_version 96702 (0.0007) [2023-10-07 23:46:44,022][67871] Updated weights for policy 1, policy_version 96870 (0.0008) [2023-10-07 23:46:44,397][67871] Updated weights for policy 1, policy_version 96880 (0.0008) [2023-10-07 23:46:44,765][67871] Updated weights for policy 1, policy_version 96890 (0.0009) [2023-10-07 23:46:47,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 198246400. Throughput: 0: 1659.3, 1: 1671.2. Samples: 49572002. Policy #0 lag: (min: 21.0, avg: 28.3, max: 53.0) [2023-10-07 23:46:47,477][66916] Avg episode reward: [(0, '50.780'), (1, '67.110')] [2023-10-07 23:46:47,675][67838] Updated weights for policy 0, policy_version 96712 (0.0008) [2023-10-07 23:46:48,040][67838] Updated weights for policy 0, policy_version 96722 (0.0008) [2023-10-07 23:46:48,416][67838] Updated weights for policy 0, policy_version 96732 (0.0008) [2023-10-07 23:46:48,782][67871] Updated weights for policy 1, policy_version 96900 (0.0008) [2023-10-07 23:46:49,148][67871] Updated weights for policy 1, policy_version 96910 (0.0008) [2023-10-07 23:46:49,517][67871] Updated weights for policy 1, policy_version 96920 (0.0008) [2023-10-07 23:46:52,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 198311936. Throughput: 0: 1656.8, 1: 1672.4. Samples: 49592672. Policy #0 lag: (min: 21.0, avg: 28.3, max: 53.0) [2023-10-07 23:46:52,477][66916] Avg episode reward: [(0, '49.590'), (1, '66.420')] [2023-10-07 23:46:52,587][67838] Updated weights for policy 0, policy_version 96742 (0.0008) [2023-10-07 23:46:52,956][67838] Updated weights for policy 0, policy_version 96752 (0.0008) [2023-10-07 23:46:53,323][67838] Updated weights for policy 0, policy_version 96762 (0.0008) [2023-10-07 23:46:53,650][67871] Updated weights for policy 1, policy_version 96930 (0.0007) [2023-10-07 23:46:54,014][67871] Updated weights for policy 1, policy_version 96940 (0.0010) [2023-10-07 23:46:54,383][67871] Updated weights for policy 1, policy_version 96950 (0.0010) [2023-10-07 23:46:54,755][67871] Updated weights for policy 1, policy_version 96960 (0.0009) [2023-10-07 23:46:57,450][67838] Updated weights for policy 0, policy_version 96772 (0.0007) [2023-10-07 23:46:57,476][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 198377472. Throughput: 0: 1658.2, 1: 1654.8. Samples: 49601490. Policy #0 lag: (min: 21.0, avg: 28.3, max: 53.0) [2023-10-07 23:46:57,477][66916] Avg episode reward: [(0, '47.560'), (1, '69.840')] [2023-10-07 23:46:57,825][67838] Updated weights for policy 0, policy_version 96782 (0.0007) [2023-10-07 23:46:58,202][67838] Updated weights for policy 0, policy_version 96792 (0.0008) [2023-10-07 23:46:58,900][67871] Updated weights for policy 1, policy_version 96970 (0.0008) [2023-10-07 23:46:59,271][67871] Updated weights for policy 1, policy_version 96980 (0.0011) [2023-10-07 23:46:59,649][67871] Updated weights for policy 1, policy_version 96990 (0.0010) [2023-10-07 23:47:02,187][67838] Updated weights for policy 0, policy_version 96802 (0.0009) [2023-10-07 23:47:02,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 198443008. Throughput: 0: 1657.4, 1: 1669.8. Samples: 49621948. Policy #0 lag: (min: 21.0, avg: 28.3, max: 53.0) [2023-10-07 23:47:02,477][66916] Avg episode reward: [(0, '49.380'), (1, '68.930')] [2023-10-07 23:47:02,561][67838] Updated weights for policy 0, policy_version 96812 (0.0007) [2023-10-07 23:47:02,925][67838] Updated weights for policy 0, policy_version 96822 (0.0007) [2023-10-07 23:47:03,297][67838] Updated weights for policy 0, policy_version 96832 (0.0007) [2023-10-07 23:47:03,698][67871] Updated weights for policy 1, policy_version 97000 (0.0009) [2023-10-07 23:47:04,061][67871] Updated weights for policy 1, policy_version 97010 (0.0008) [2023-10-07 23:47:04,423][67871] Updated weights for policy 1, policy_version 97020 (0.0009) [2023-10-07 23:47:07,418][67838] Updated weights for policy 0, policy_version 96842 (0.0009) [2023-10-07 23:47:07,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 198508544. Throughput: 0: 1663.2, 1: 1674.1. Samples: 49642512. Policy #0 lag: (min: 21.0, avg: 28.3, max: 53.0) [2023-10-07 23:47:07,477][66916] Avg episode reward: [(0, '48.760'), (1, '69.300')] [2023-10-07 23:47:07,484][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000097024_99352576.pth... [2023-10-07 23:47:07,517][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000095488_97779712.pth [2023-10-07 23:47:07,795][67838] Updated weights for policy 0, policy_version 96852 (0.0008) [2023-10-07 23:47:08,177][67838] Updated weights for policy 0, policy_version 96862 (0.0010) [2023-10-07 23:47:08,239][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000096864_99188736.pth... [2023-10-07 23:47:08,269][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000095296_97583104.pth [2023-10-07 23:47:08,666][67871] Updated weights for policy 1, policy_version 97030 (0.0009) [2023-10-07 23:47:09,031][67871] Updated weights for policy 1, policy_version 97040 (0.0007) [2023-10-07 23:47:09,399][67871] Updated weights for policy 1, policy_version 97050 (0.0007) [2023-10-07 23:47:12,460][67838] Updated weights for policy 0, policy_version 96872 (0.0009) [2023-10-07 23:47:12,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 198574080. Throughput: 0: 1662.4, 1: 1663.6. Samples: 49651500. Policy #0 lag: (min: 21.0, avg: 28.3, max: 53.0) [2023-10-07 23:47:12,478][66916] Avg episode reward: [(0, '50.660'), (1, '67.970')] [2023-10-07 23:47:12,839][67838] Updated weights for policy 0, policy_version 96882 (0.0010) [2023-10-07 23:47:13,208][67838] Updated weights for policy 0, policy_version 96892 (0.0011) [2023-10-07 23:47:13,504][67871] Updated weights for policy 1, policy_version 97060 (0.0007) [2023-10-07 23:47:13,862][67871] Updated weights for policy 1, policy_version 97070 (0.0008) [2023-10-07 23:47:14,224][67871] Updated weights for policy 1, policy_version 97080 (0.0010) [2023-10-07 23:47:17,421][67838] Updated weights for policy 0, policy_version 96902 (0.0009) [2023-10-07 23:47:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 198639616. Throughput: 0: 1655.9, 1: 1677.4. Samples: 49671882. Policy #0 lag: (min: 21.0, avg: 28.3, max: 53.0) [2023-10-07 23:47:17,477][66916] Avg episode reward: [(0, '49.160'), (1, '69.130')] [2023-10-07 23:47:17,792][67838] Updated weights for policy 0, policy_version 96912 (0.0009) [2023-10-07 23:47:18,168][67838] Updated weights for policy 0, policy_version 96922 (0.0008) [2023-10-07 23:47:18,267][67871] Updated weights for policy 1, policy_version 97090 (0.0008) [2023-10-07 23:47:18,629][67871] Updated weights for policy 1, policy_version 97100 (0.0009) [2023-10-07 23:47:19,005][67871] Updated weights for policy 1, policy_version 97110 (0.0007) [2023-10-07 23:47:19,361][67871] Updated weights for policy 1, policy_version 97120 (0.0008) [2023-10-07 23:47:22,289][67838] Updated weights for policy 0, policy_version 96932 (0.0008) [2023-10-07 23:47:22,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 198705152. Throughput: 0: 1656.4, 1: 1679.4. Samples: 49692532. Policy #0 lag: (min: 14.0, avg: 14.9, max: 34.0) [2023-10-07 23:47:22,477][66916] Avg episode reward: [(0, '49.420'), (1, '69.060')] [2023-10-07 23:47:22,675][67838] Updated weights for policy 0, policy_version 96942 (0.0009) [2023-10-07 23:47:23,036][67838] Updated weights for policy 0, policy_version 96952 (0.0007) [2023-10-07 23:47:23,487][67871] Updated weights for policy 1, policy_version 97130 (0.0009) [2023-10-07 23:47:23,852][67871] Updated weights for policy 1, policy_version 97140 (0.0009) [2023-10-07 23:47:24,226][67871] Updated weights for policy 1, policy_version 97150 (0.0007) [2023-10-07 23:47:27,010][67838] Updated weights for policy 0, policy_version 96962 (0.0007) [2023-10-07 23:47:27,387][67838] Updated weights for policy 0, policy_version 96972 (0.0009) [2023-10-07 23:47:27,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 198770688. Throughput: 0: 1655.2, 1: 1671.6. Samples: 49701388. Policy #0 lag: (min: 14.0, avg: 14.9, max: 34.0) [2023-10-07 23:47:27,477][66916] Avg episode reward: [(0, '52.800'), (1, '69.720')] [2023-10-07 23:47:27,765][67838] Updated weights for policy 0, policy_version 96982 (0.0009) [2023-10-07 23:47:28,126][67838] Updated weights for policy 0, policy_version 96992 (0.0008) [2023-10-07 23:47:28,295][67871] Updated weights for policy 1, policy_version 97160 (0.0009) [2023-10-07 23:47:28,649][67871] Updated weights for policy 1, policy_version 97170 (0.0011) [2023-10-07 23:47:29,032][67871] Updated weights for policy 1, policy_version 97180 (0.0010) [2023-10-07 23:47:32,130][67838] Updated weights for policy 0, policy_version 97002 (0.0008) [2023-10-07 23:47:32,476][66916] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 198836224. Throughput: 0: 1659.8, 1: 1679.0. Samples: 49722248. Policy #0 lag: (min: 14.0, avg: 14.9, max: 34.0) [2023-10-07 23:47:32,477][66916] Avg episode reward: [(0, '51.980'), (1, '69.510')] [2023-10-07 23:47:32,512][67838] Updated weights for policy 0, policy_version 97012 (0.0008) [2023-10-07 23:47:32,877][67838] Updated weights for policy 0, policy_version 97022 (0.0007) [2023-10-07 23:47:33,275][67871] Updated weights for policy 1, policy_version 97190 (0.0010) [2023-10-07 23:47:33,632][67871] Updated weights for policy 1, policy_version 97200 (0.0008) [2023-10-07 23:47:34,002][67871] Updated weights for policy 1, policy_version 97210 (0.0009) [2023-10-07 23:47:36,824][67838] Updated weights for policy 0, policy_version 97032 (0.0007) [2023-10-07 23:47:37,200][67838] Updated weights for policy 0, policy_version 97042 (0.0010) [2023-10-07 23:47:37,477][66916] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 198901760. Throughput: 0: 1653.1, 1: 1673.0. Samples: 49742348. Policy #0 lag: (min: 14.0, avg: 14.9, max: 34.0) [2023-10-07 23:47:37,478][66916] Avg episode reward: [(0, '51.580'), (1, '72.700')] [2023-10-07 23:47:37,488][67676] Saving new best policy, reward=72.700! [2023-10-07 23:47:37,574][67838] Updated weights for policy 0, policy_version 97052 (0.0009) [2023-10-07 23:47:38,192][67871] Updated weights for policy 1, policy_version 97220 (0.0009) [2023-10-07 23:47:38,563][67871] Updated weights for policy 1, policy_version 97230 (0.0009) [2023-10-07 23:47:38,932][67871] Updated weights for policy 1, policy_version 97240 (0.0007) [2023-10-07 23:47:41,838][67838] Updated weights for policy 0, policy_version 97062 (0.0009) [2023-10-07 23:47:42,211][67838] Updated weights for policy 0, policy_version 97072 (0.0007) [2023-10-07 23:47:42,476][66916] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 198967296. Throughput: 0: 1664.3, 1: 1675.9. Samples: 49751798. Policy #0 lag: (min: 14.0, avg: 14.9, max: 34.0) [2023-10-07 23:47:42,477][66916] Avg episode reward: [(0, '47.970'), (1, '71.640')] [2023-10-07 23:47:42,573][67838] Updated weights for policy 0, policy_version 97082 (0.0008) [2023-10-07 23:47:42,745][67871] Updated weights for policy 1, policy_version 97250 (0.0009) [2023-10-07 23:47:43,110][67871] Updated weights for policy 1, policy_version 97260 (0.0009) [2023-10-07 23:47:43,486][67871] Updated weights for policy 1, policy_version 97270 (0.0009) [2023-10-07 23:47:43,847][67871] Updated weights for policy 1, policy_version 97280 (0.0009) [2023-10-07 23:47:46,673][67838] Updated weights for policy 0, policy_version 97092 (0.0011) [2023-10-07 23:47:47,048][67838] Updated weights for policy 0, policy_version 97102 (0.0010) [2023-10-07 23:47:47,419][67838] Updated weights for policy 0, policy_version 97112 (0.0011) [2023-10-07 23:47:47,477][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 199032832. Throughput: 0: 1664.1, 1: 1678.0. Samples: 49772344. Policy #0 lag: (min: 14.0, avg: 14.9, max: 34.0) [2023-10-07 23:47:47,478][66916] Avg episode reward: [(0, '51.020'), (1, '75.310')] [2023-10-07 23:47:47,479][67676] Saving new best policy, reward=75.310! [2023-10-07 23:47:48,048][67871] Updated weights for policy 1, policy_version 97290 (0.0009) [2023-10-07 23:47:48,415][67871] Updated weights for policy 1, policy_version 97300 (0.0011) [2023-10-07 23:47:48,790][67871] Updated weights for policy 1, policy_version 97310 (0.0010) [2023-10-07 23:47:51,520][67838] Updated weights for policy 0, policy_version 97122 (0.0009) [2023-10-07 23:47:51,884][67838] Updated weights for policy 0, policy_version 97132 (0.0009) [2023-10-07 23:47:52,258][67838] Updated weights for policy 0, policy_version 97142 (0.0008) [2023-10-07 23:47:52,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 199098368. Throughput: 0: 1650.8, 1: 1680.8. Samples: 49792434. Policy #0 lag: (min: 14.0, avg: 14.9, max: 34.0) [2023-10-07 23:47:52,477][66916] Avg episode reward: [(0, '49.640'), (1, '72.290')] [2023-10-07 23:47:52,629][67838] Updated weights for policy 0, policy_version 97152 (0.0009) [2023-10-07 23:47:52,876][67871] Updated weights for policy 1, policy_version 97320 (0.0008) [2023-10-07 23:47:53,247][67871] Updated weights for policy 1, policy_version 97330 (0.0008) [2023-10-07 23:47:53,607][67871] Updated weights for policy 1, policy_version 97340 (0.0007) [2023-10-07 23:47:56,802][67838] Updated weights for policy 0, policy_version 97162 (0.0007) [2023-10-07 23:47:57,175][67838] Updated weights for policy 0, policy_version 97172 (0.0007) [2023-10-07 23:47:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 199163904. Throughput: 0: 1664.1, 1: 1682.1. Samples: 49802076. Policy #0 lag: (min: 14.0, avg: 14.9, max: 34.0) [2023-10-07 23:47:57,477][66916] Avg episode reward: [(0, '53.700'), (1, '72.410')] [2023-10-07 23:47:57,547][67838] Updated weights for policy 0, policy_version 97182 (0.0007) [2023-10-07 23:47:57,616][67871] Updated weights for policy 1, policy_version 97350 (0.0008) [2023-10-07 23:47:57,983][67871] Updated weights for policy 1, policy_version 97360 (0.0009) [2023-10-07 23:47:58,354][67871] Updated weights for policy 1, policy_version 97370 (0.0010) [2023-10-07 23:48:01,633][67838] Updated weights for policy 0, policy_version 97192 (0.0007) [2023-10-07 23:48:01,994][67838] Updated weights for policy 0, policy_version 97202 (0.0007) [2023-10-07 23:48:02,228][67871] Updated weights for policy 1, policy_version 97380 (0.0007) [2023-10-07 23:48:02,367][67838] Updated weights for policy 0, policy_version 97212 (0.0008) [2023-10-07 23:48:02,476][66916] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 199229440. Throughput: 0: 1671.3, 1: 1680.8. Samples: 49822724. Policy #0 lag: (min: 14.0, avg: 14.9, max: 34.0) [2023-10-07 23:48:02,477][66916] Avg episode reward: [(0, '52.820'), (1, '73.980')] [2023-10-07 23:48:02,593][67871] Updated weights for policy 1, policy_version 97390 (0.0008) [2023-10-07 23:48:02,952][67871] Updated weights for policy 1, policy_version 97400 (0.0010) [2023-10-07 23:48:06,349][67838] Updated weights for policy 0, policy_version 97222 (0.0009) [2023-10-07 23:48:06,715][67838] Updated weights for policy 0, policy_version 97232 (0.0008) [2023-10-07 23:48:07,021][67871] Updated weights for policy 1, policy_version 97410 (0.0011) [2023-10-07 23:48:07,083][67838] Updated weights for policy 0, policy_version 97242 (0.0007) [2023-10-07 23:48:07,386][67871] Updated weights for policy 1, policy_version 97420 (0.0009) [2023-10-07 23:48:07,476][66916] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 199327744. Throughput: 0: 1652.6, 1: 1679.7. Samples: 49842488. Policy #0 lag: (min: 14.0, avg: 14.9, max: 34.0) [2023-10-07 23:48:07,477][66916] Avg episode reward: [(0, '51.350'), (1, '72.980')] [2023-10-07 23:48:07,759][67871] Updated weights for policy 1, policy_version 97430 (0.0009) [2023-10-07 23:48:08,124][67871] Updated weights for policy 1, policy_version 97440 (0.0009) [2023-10-07 23:48:11,198][67838] Updated weights for policy 0, policy_version 97252 (0.0007) [2023-10-07 23:48:11,590][67838] Updated weights for policy 0, policy_version 97262 (0.0007) [2023-10-07 23:48:11,967][67838] Updated weights for policy 0, policy_version 97272 (0.0007) [2023-10-07 23:48:12,476][66916] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 199393280. Throughput: 0: 1677.9, 1: 1683.6. Samples: 49852656. Policy #0 lag: (min: 14.0, avg: 14.9, max: 34.0) [2023-10-07 23:48:12,481][66916] Avg episode reward: [(0, '53.400'), (1, '72.030')] [2023-10-07 23:48:12,492][67871] Updated weights for policy 1, policy_version 97450 (0.0007) [2023-10-07 23:48:12,856][67871] Updated weights for policy 1, policy_version 97460 (0.0007) [2023-10-07 23:48:13,216][67871] Updated weights for policy 1, policy_version 97470 (0.0010) [2023-10-07 23:48:16,179][67838] Updated weights for policy 0, policy_version 97282 (0.0008) [2023-10-07 23:48:16,553][67838] Updated weights for policy 0, policy_version 97292 (0.0007) [2023-10-07 23:48:16,927][67838] Updated weights for policy 0, policy_version 97302 (0.0009) [2023-10-07 23:48:17,217][67871] Updated weights for policy 1, policy_version 97480 (0.0007) [2023-10-07 23:48:17,297][67838] Updated weights for policy 0, policy_version 97312 (0.0009) [2023-10-07 23:48:17,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 199458816. Throughput: 0: 1670.1, 1: 1678.7. Samples: 49872942. Policy #0 lag: (min: 14.0, avg: 14.9, max: 34.0) [2023-10-07 23:48:17,477][66916] Avg episode reward: [(0, '54.710'), (1, '66.650')] [2023-10-07 23:48:17,582][67871] Updated weights for policy 1, policy_version 97490 (0.0010) [2023-10-07 23:48:17,949][67871] Updated weights for policy 1, policy_version 97500 (0.0010) [2023-10-07 23:48:21,557][67838] Updated weights for policy 0, policy_version 97322 (0.0008) [2023-10-07 23:48:21,930][67838] Updated weights for policy 0, policy_version 97332 (0.0008) [2023-10-07 23:48:22,148][67871] Updated weights for policy 1, policy_version 97510 (0.0009) [2023-10-07 23:48:22,305][67838] Updated weights for policy 0, policy_version 97342 (0.0007) [2023-10-07 23:48:22,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 199524352. Throughput: 0: 1656.4, 1: 1683.3. Samples: 49892634. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-07 23:48:22,478][66916] Avg episode reward: [(0, '52.690'), (1, '68.640')] [2023-10-07 23:48:22,508][67871] Updated weights for policy 1, policy_version 97520 (0.0007) [2023-10-07 23:48:22,877][67871] Updated weights for policy 1, policy_version 97530 (0.0008) [2023-10-07 23:48:26,336][67838] Updated weights for policy 0, policy_version 97352 (0.0009) [2023-10-07 23:48:26,709][67838] Updated weights for policy 0, policy_version 97362 (0.0010) [2023-10-07 23:48:26,951][67871] Updated weights for policy 1, policy_version 97540 (0.0009) [2023-10-07 23:48:27,078][67838] Updated weights for policy 0, policy_version 97372 (0.0007) [2023-10-07 23:48:27,316][67871] Updated weights for policy 1, policy_version 97550 (0.0007) [2023-10-07 23:48:27,476][66916] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 199589888. Throughput: 0: 1666.8, 1: 1682.7. Samples: 49902524. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-07 23:48:27,477][66916] Avg episode reward: [(0, '54.340'), (1, '69.600')] [2023-10-07 23:48:27,690][67871] Updated weights for policy 1, policy_version 97560 (0.0010) [2023-10-07 23:48:31,120][67838] Updated weights for policy 0, policy_version 97382 (0.0010) [2023-10-07 23:48:31,496][67838] Updated weights for policy 0, policy_version 97392 (0.0010) [2023-10-07 23:48:31,603][67871] Updated weights for policy 1, policy_version 97570 (0.0007) [2023-10-07 23:48:31,859][67838] Updated weights for policy 0, policy_version 97402 (0.0007) [2023-10-07 23:48:31,960][67871] Updated weights for policy 1, policy_version 97580 (0.0008) [2023-10-07 23:48:32,324][67871] Updated weights for policy 1, policy_version 97590 (0.0007) [2023-10-07 23:48:32,476][66916] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 199655424. Throughput: 0: 1661.7, 1: 1681.3. Samples: 49922778. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-07 23:48:32,477][66916] Avg episode reward: [(0, '60.560'), (1, '70.520')] [2023-10-07 23:48:32,693][67871] Updated weights for policy 1, policy_version 97600 (0.0008) [2023-10-07 23:48:36,128][67838] Updated weights for policy 0, policy_version 97412 (0.0008) [2023-10-07 23:48:36,501][67838] Updated weights for policy 0, policy_version 97422 (0.0007) [2023-10-07 23:48:36,687][67871] Updated weights for policy 1, policy_version 97610 (0.0008) [2023-10-07 23:48:36,875][67838] Updated weights for policy 0, policy_version 97432 (0.0007) [2023-10-07 23:48:37,046][67871] Updated weights for policy 1, policy_version 97620 (0.0007) [2023-10-07 23:48:37,413][67871] Updated weights for policy 1, policy_version 97630 (0.0008) [2023-10-07 23:48:37,476][66916] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 199753728. Throughput: 0: 1651.5, 1: 1671.7. Samples: 49941976. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-07 23:48:37,477][66916] Avg episode reward: [(0, '58.660'), (1, '67.970')] [2023-10-07 23:48:40,876][67838] Updated weights for policy 0, policy_version 97442 (0.0008) [2023-10-07 23:48:41,253][67838] Updated weights for policy 0, policy_version 97452 (0.0008) [2023-10-07 23:48:41,626][67838] Updated weights for policy 0, policy_version 97462 (0.0007) [2023-10-07 23:48:41,632][67871] Updated weights for policy 1, policy_version 97640 (0.0008) [2023-10-07 23:48:41,999][67838] Updated weights for policy 0, policy_version 97472 (0.0009) [2023-10-07 23:48:42,007][67871] Updated weights for policy 1, policy_version 97650 (0.0009) [2023-10-07 23:48:42,366][67871] Updated weights for policy 1, policy_version 97660 (0.0009) [2023-10-07 23:48:42,477][66916] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 199786496. Throughput: 0: 1664.5, 1: 1681.1. Samples: 49952630. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-07 23:48:42,478][66916] Avg episode reward: [(0, '58.200'), (1, '71.870')] [2023-10-07 23:48:46,373][67838] Updated weights for policy 0, policy_version 97482 (0.0010) [2023-10-07 23:48:46,416][67871] Updated weights for policy 1, policy_version 97670 (0.0008) [2023-10-07 23:48:46,745][67838] Updated weights for policy 0, policy_version 97492 (0.0008) [2023-10-07 23:48:46,777][67871] Updated weights for policy 1, policy_version 97680 (0.0008) [2023-10-07 23:48:47,115][67838] Updated weights for policy 0, policy_version 97502 (0.0008) [2023-10-07 23:48:47,137][67871] Updated weights for policy 1, policy_version 97690 (0.0008) [2023-10-07 23:48:47,477][66916] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 199884800. Throughput: 0: 1655.4, 1: 1678.2. Samples: 49972736. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-07 23:48:47,478][66916] Avg episode reward: [(0, '60.770'), (1, '70.890')] [2023-10-07 23:48:51,258][67838] Updated weights for policy 0, policy_version 97512 (0.0009) [2023-10-07 23:48:51,413][67871] Updated weights for policy 1, policy_version 97700 (0.0008) [2023-10-07 23:48:51,629][67838] Updated weights for policy 0, policy_version 97522 (0.0010) [2023-10-07 23:48:51,776][67871] Updated weights for policy 1, policy_version 97710 (0.0009) [2023-10-07 23:48:52,003][67838] Updated weights for policy 0, policy_version 97532 (0.0008) [2023-10-07 23:48:52,148][67871] Updated weights for policy 1, policy_version 97720 (0.0009) [2023-10-07 23:48:52,477][66916] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 199950336. Throughput: 0: 1647.5, 1: 1661.3. Samples: 49991384. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-07 23:48:52,478][66916] Avg episode reward: [(0, '58.060'), (1, '71.830')] [2023-10-07 23:48:56,293][67838] Updated weights for policy 0, policy_version 97542 (0.0007) [2023-10-07 23:48:56,325][67871] Updated weights for policy 1, policy_version 97730 (0.0010) [2023-10-07 23:48:56,672][67838] Updated weights for policy 0, policy_version 97552 (0.0008) [2023-10-07 23:48:56,691][67871] Updated weights for policy 1, policy_version 97740 (0.0011) [2023-10-07 23:48:57,044][67838] Updated weights for policy 0, policy_version 97562 (0.0008) [2023-10-07 23:48:57,053][67871] Updated weights for policy 1, policy_version 97750 (0.0008) [2023-10-07 23:48:57,425][67871] Updated weights for policy 1, policy_version 97760 (0.0010) [2023-10-07 23:48:57,476][66916] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 200015872. Throughput: 0: 1648.5, 1: 1670.3. Samples: 50002000. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-07 23:48:57,477][66916] Avg episode reward: [(0, '56.140'), (1, '72.170')] [2023-10-07 23:49:01,023][67838] Updated weights for policy 0, policy_version 97572 (0.0009) [2023-10-07 23:49:01,405][67838] Updated weights for policy 0, policy_version 97582 (0.0010) [2023-10-07 23:49:01,615][67871] Updated weights for policy 1, policy_version 97770 (0.0008) [2023-10-07 23:49:01,771][67838] Updated weights for policy 0, policy_version 97592 (0.0009) [2023-10-07 23:49:01,976][67871] Updated weights for policy 1, policy_version 97780 (0.0009) [2023-10-07 23:49:02,341][67871] Updated weights for policy 1, policy_version 97790 (0.0007) [2023-10-07 23:49:02,476][66916] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 200081408. Throughput: 0: 1645.1, 1: 1665.3. Samples: 50021910. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-07 23:49:02,477][66916] Avg episode reward: [(0, '54.980'), (1, '67.720')] [2023-10-07 23:49:05,931][67838] Updated weights for policy 0, policy_version 97602 (0.0009) [2023-10-07 23:49:06,301][67838] Updated weights for policy 0, policy_version 97612 (0.0008) [2023-10-07 23:49:06,648][67871] Updated weights for policy 1, policy_version 97800 (0.0007) [2023-10-07 23:49:06,677][67838] Updated weights for policy 0, policy_version 97622 (0.0008) [2023-10-07 23:49:07,010][67871] Updated weights for policy 1, policy_version 97810 (0.0008) [2023-10-07 23:49:07,041][67838] Updated weights for policy 0, policy_version 97632 (0.0007) [2023-10-07 23:49:07,369][67871] Updated weights for policy 1, policy_version 97820 (0.0011) [2023-10-07 23:49:07,477][66916] Fps is (10 sec: 9830.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 200114176. Throughput: 0: 1644.6, 1: 1648.8. Samples: 50040838. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-07 23:49:07,477][66916] Avg episode reward: [(0, '56.850'), (1, '67.490')] [2023-10-07 23:49:07,486][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000097632_99975168.pth... [2023-10-07 23:49:07,519][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000097824_100171776.pth... [2023-10-07 23:49:07,522][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000096064_98369536.pth [2023-10-07 23:49:07,557][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000096256_98566144.pth [2023-10-07 23:49:11,201][67838] Updated weights for policy 0, policy_version 97642 (0.0008) [2023-10-07 23:49:11,285][67871] Updated weights for policy 1, policy_version 97830 (0.0009) [2023-10-07 23:49:11,574][67838] Updated weights for policy 0, policy_version 97652 (0.0009) [2023-10-07 23:49:11,641][67871] Updated weights for policy 1, policy_version 97840 (0.0008) [2023-10-07 23:49:11,944][67838] Updated weights for policy 0, policy_version 97662 (0.0009) [2023-10-07 23:49:12,007][67871] Updated weights for policy 1, policy_version 97850 (0.0008) [2023-10-07 23:49:12,220][67875] Stopping RolloutWorker_w8... [2023-10-07 23:49:12,221][67875] Loop rollout_proc8_evt_loop terminating... [2023-10-07 23:49:12,220][67874] Stopping RolloutWorker_w2... [2023-10-07 23:49:12,220][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000097664_100007936.pth... [2023-10-07 23:49:12,220][67876] Stopping RolloutWorker_w4... [2023-10-07 23:49:12,221][67873] Stopping RolloutWorker_w0... [2023-10-07 23:49:12,221][67915] Stopping RolloutWorker_w10... [2023-10-07 23:49:12,221][67874] Loop rollout_proc2_evt_loop terminating... [2023-10-07 23:49:12,221][67876] Loop rollout_proc4_evt_loop terminating... [2023-10-07 23:49:12,221][67873] Loop rollout_proc0_evt_loop terminating... [2023-10-07 23:49:12,221][67915] Loop rollout_proc10_evt_loop terminating... [2023-10-07 23:49:12,221][66916] Component RolloutWorker_w8 stopped! [2023-10-07 23:49:12,222][66916] Component RolloutWorker_w2 stopped! [2023-10-07 23:49:12,222][68572] Stopping RolloutWorker_w14... [2023-10-07 23:49:12,223][68572] Loop rollout_proc14_evt_loop terminating... [2023-10-07 23:49:12,223][66916] Component RolloutWorker_w4 stopped! [2023-10-07 23:49:12,223][66916] Component Batcher_0 stopped! [2023-10-07 23:49:12,224][66916] Component RolloutWorker_w0 stopped! [2023-10-07 23:49:12,224][67919] Stopping RolloutWorker_w13... [2023-10-07 23:49:12,224][67877] Stopping RolloutWorker_w5... [2023-10-07 23:49:12,224][67919] Loop rollout_proc13_evt_loop terminating... [2023-10-07 23:49:12,225][67877] Loop rollout_proc5_evt_loop terminating... [2023-10-07 23:49:12,224][66916] Component RolloutWorker_w10 stopped! [2023-10-07 23:49:12,225][67918] Stopping RolloutWorker_w12... [2023-10-07 23:49:12,225][67916] Stopping RolloutWorker_w9... [2023-10-07 23:49:12,225][67918] Loop rollout_proc12_evt_loop terminating... [2023-10-07 23:49:12,225][66916] Component RolloutWorker_w14 stopped! [2023-10-07 23:49:12,225][67887] Stopping RolloutWorker_w3... [2023-10-07 23:49:12,225][67916] Loop rollout_proc9_evt_loop terminating... [2023-10-07 23:49:12,226][67887] Loop rollout_proc3_evt_loop terminating... [2023-10-07 23:49:12,226][67885] Stopping RolloutWorker_w7... [2023-10-07 23:49:12,226][67917] Stopping RolloutWorker_w11... [2023-10-07 23:49:12,226][67884] Stopping RolloutWorker_w6... [2023-10-07 23:49:12,226][68573] Stopping RolloutWorker_w15... [2023-10-07 23:49:12,221][67511] Stopping Batcher_0... [2023-10-07 23:49:12,226][66916] Component RolloutWorker_w13 stopped! [2023-10-07 23:49:12,226][67885] Loop rollout_proc7_evt_loop terminating... [2023-10-07 23:49:12,226][68573] Loop rollout_proc15_evt_loop terminating... [2023-10-07 23:49:12,226][67917] Loop rollout_proc11_evt_loop terminating... [2023-10-07 23:49:12,226][67884] Loop rollout_proc6_evt_loop terminating... [2023-10-07 23:49:12,226][66916] Component RolloutWorker_w5 stopped! [2023-10-07 23:49:12,226][67870] Stopping RolloutWorker_w1... [2023-10-07 23:49:12,227][66916] Component RolloutWorker_w12 stopped! [2023-10-07 23:49:12,227][67870] Loop rollout_proc1_evt_loop terminating... [2023-10-07 23:49:12,227][66916] Component RolloutWorker_w9 stopped! [2023-10-07 23:49:12,227][66916] Component RolloutWorker_w3 stopped! [2023-10-07 23:49:12,228][66916] Component RolloutWorker_w7 stopped! [2023-10-07 23:49:12,228][66916] Component RolloutWorker_w6 stopped! [2023-10-07 23:49:12,228][66916] Component RolloutWorker_w11 stopped! [2023-10-07 23:49:12,228][66916] Component RolloutWorker_w15 stopped! [2023-10-07 23:49:12,229][66916] Component RolloutWorker_w1 stopped! [2023-10-07 23:49:12,230][66916] Component Batcher_1 stopped! [2023-10-07 23:49:12,230][67676] Stopping Batcher_1... [2023-10-07 23:49:12,230][67676] Loop batcher_evt_loop terminating... [2023-10-07 23:49:12,231][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000097856_100204544.pth... [2023-10-07 23:49:12,246][67838] Weights refcount: 2 0 [2023-10-07 23:49:12,248][67838] Stopping InferenceWorker_p0-w0... [2023-10-07 23:49:12,248][67871] Weights refcount: 2 0 [2023-10-07 23:49:12,249][67838] Loop inference_proc0-0_evt_loop terminating... [2023-10-07 23:49:12,248][66916] Component InferenceWorker_p0-w0 stopped! [2023-10-07 23:49:12,255][67871] Stopping InferenceWorker_p1-w0... [2023-10-07 23:49:12,255][67871] Loop inference_proc1-0_evt_loop terminating... [2023-10-07 23:49:12,255][66916] Component InferenceWorker_p1-w0 stopped! [2023-10-07 23:49:12,242][67511] Loop batcher_evt_loop terminating... [2023-10-07 23:49:12,262][67676] Removing ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000097024_99352576.pth [2023-10-07 23:49:12,267][67676] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000097856_100204544.pth... [2023-10-07 23:49:12,269][67511] Removing ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000096864_99188736.pth [2023-10-07 23:49:12,275][67511] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000097664_100007936.pth... [2023-10-07 23:49:12,307][67676] Stopping LearnerWorker_p1... [2023-10-07 23:49:12,307][67676] Loop learner_proc1_evt_loop terminating... [2023-10-07 23:49:12,307][66916] Component LearnerWorker_p1 stopped! [2023-10-07 23:49:12,331][67511] Stopping LearnerWorker_p0... [2023-10-07 23:49:12,331][67511] Loop learner_proc0_evt_loop terminating... [2023-10-07 23:49:12,331][66916] Component LearnerWorker_p0 stopped! [2023-10-07 23:49:12,332][66916] Waiting for process learner_proc0 to stop... [2023-10-07 23:49:13,164][66916] Waiting for process learner_proc1 to stop... [2023-10-07 23:49:13,165][66916] Waiting for process inference_proc0-0 to join... [2023-10-07 23:49:13,166][66916] Waiting for process inference_proc1-0 to join... [2023-10-07 23:49:13,167][66916] Waiting for process rollout_proc0 to join... [2023-10-07 23:49:13,168][66916] Waiting for process rollout_proc1 to join... [2023-10-07 23:49:13,168][66916] Waiting for process rollout_proc2 to join... [2023-10-07 23:49:13,169][66916] Waiting for process rollout_proc3 to join... [2023-10-07 23:49:13,170][66916] Waiting for process rollout_proc4 to join... [2023-10-07 23:49:13,170][66916] Waiting for process rollout_proc5 to join... [2023-10-07 23:49:13,171][66916] Waiting for process rollout_proc6 to join... [2023-10-07 23:49:13,172][66916] Waiting for process rollout_proc7 to join... [2023-10-07 23:49:13,173][66916] Waiting for process rollout_proc8 to join... [2023-10-07 23:49:13,173][66916] Waiting for process rollout_proc9 to join... [2023-10-07 23:49:13,174][66916] Waiting for process rollout_proc10 to join... [2023-10-07 23:49:13,175][66916] Waiting for process rollout_proc11 to join... [2023-10-07 23:49:13,176][66916] Waiting for process rollout_proc12 to join... [2023-10-07 23:49:13,176][66916] Waiting for process rollout_proc13 to join... [2023-10-07 23:49:13,177][66916] Waiting for process rollout_proc14 to join... [2023-10-07 23:49:13,177][66916] Waiting for process rollout_proc15 to join... [2023-10-07 23:49:13,178][66916] Batcher 0 profile tree view: batching: 167.6860, releasing_batches: 0.0909 [2023-10-07 23:49:13,178][66916] Batcher 1 profile tree view: batching: 167.6353, releasing_batches: 0.0911 [2023-10-07 23:49:13,178][66916] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0001 wait_policy_total: 2785.4962 update_model: 209.1077 weight_update: 0.0009 one_step: 0.0024 handle_policy_step: 11412.1403 deserialize: 64.2841, stack: 192.6336, obs_to_device_normalize: 2547.9968, forward: 5137.4494, prepare_outputs: 2492.4677, send_messages: 469.5180 [2023-10-07 23:49:13,178][66916] InferenceWorker_p1-w0 profile tree view: wait_policy: 0.0001 wait_policy_total: 2809.7113 update_model: 204.3974 weight_update: 0.0008 one_step: 0.0030 handle_policy_step: 11379.1606 deserialize: 65.7015, stack: 191.6660, obs_to_device_normalize: 2554.6521, forward: 5150.4857, prepare_outputs: 2439.3559, send_messages: 470.8436 [2023-10-07 23:49:13,179][66916] Learner 0 profile tree view: misc: 0.0188, prepare_batch: 270.3944 train: 3652.7800 epoch_init: 0.1845, minibatch_init: 13.0836, losses_postprocess: 899.4954, kl_divergence: 32.6001, update: 387.9252, after_optimizer: 2136.3307 calculate_losses: 166.5893 losses_init: 0.3928, forward_head: 55.2338, bptt_initial: 1.4278, bptt: 1.9835, tail: 38.6063, advantages_returns: 11.1988, losses: 44.1473 [2023-10-07 23:49:13,179][66916] Learner 1 profile tree view: misc: 0.0185, prepare_batch: 270.3057 train: 3614.4447 epoch_init: 0.1928, minibatch_init: 13.2166, losses_postprocess: 892.4884, kl_divergence: 31.9287, update: 382.3972, after_optimizer: 2111.3996 calculate_losses: 166.3274 losses_init: 0.3822, forward_head: 55.7357, bptt_initial: 1.4305, bptt: 1.9494, tail: 38.2724, advantages_returns: 11.2079, losses: 43.7157 [2023-10-07 23:49:13,179][66916] RolloutWorker_w0 profile tree view: wait_for_trajectories: 1.2173, enqueue_policy_requests: 401.3823, process_policy_outputs: 190.1966, env_step: 7888.6930, finalize_trajectories: 3.5065, complete_rollouts: 2.9513 post_env_step: 371.7437 process_env_step: 83.3935 [2023-10-07 23:49:13,180][66916] RolloutWorker_w15 profile tree view: wait_for_trajectories: 1.2279, enqueue_policy_requests: 401.9207, process_policy_outputs: 189.5683, env_step: 7868.6738, finalize_trajectories: 3.4062, complete_rollouts: 2.9182 post_env_step: 376.8363 process_env_step: 84.9913 [2023-10-07 23:49:13,180][66916] Loop Runner_EvtLoop terminating... [2023-10-07 23:49:13,180][66916] Runner profile tree view: main_loop: 15121.2298 [2023-10-07 23:49:13,181][66916] Collected {0: 100007936, 1: 100204544}, FPS: 13240.5