[2023-10-08 03:56:22,625][130385] Saving configuration to ./train_atari/atari_assault_APPO/config.json... [2023-10-08 03:56:22,943][130385] Rollout worker 0 uses device cpu [2023-10-08 03:56:22,943][130385] Rollout worker 1 uses device cpu [2023-10-08 03:56:22,944][130385] Rollout worker 2 uses device cpu [2023-10-08 03:56:22,945][130385] Rollout worker 3 uses device cpu [2023-10-08 03:56:22,945][130385] Rollout worker 4 uses device cpu [2023-10-08 03:56:22,946][130385] Rollout worker 5 uses device cpu [2023-10-08 03:56:22,946][130385] Rollout worker 6 uses device cpu [2023-10-08 03:56:22,947][130385] Rollout worker 7 uses device cpu [2023-10-08 03:56:22,947][130385] Rollout worker 8 uses device cpu [2023-10-08 03:56:22,948][130385] Rollout worker 9 uses device cpu [2023-10-08 03:56:22,948][130385] Rollout worker 10 uses device cpu [2023-10-08 03:56:22,949][130385] Rollout worker 11 uses device cpu [2023-10-08 03:56:22,949][130385] Rollout worker 12 uses device cpu [2023-10-08 03:56:22,950][130385] Rollout worker 13 uses device cpu [2023-10-08 03:56:22,950][130385] Rollout worker 14 uses device cpu [2023-10-08 03:56:22,950][130385] Rollout worker 15 uses device cpu [2023-10-08 03:56:23,230][130385] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-08 03:56:23,230][130385] InferenceWorker_p0-w0: min num requests: 2 [2023-10-08 03:56:23,234][130385] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-08 03:56:23,234][130385] InferenceWorker_p1-w0: min num requests: 2 [2023-10-08 03:56:23,279][130385] Starting all processes... [2023-10-08 03:56:23,280][130385] Starting process learner_proc0 [2023-10-08 03:56:24,973][130385] Starting process learner_proc1 [2023-10-08 03:56:24,976][00365] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-08 03:56:24,976][00365] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-10-08 03:56:24,994][00365] Num visible devices: 1 [2023-10-08 03:56:25,010][00365] Setting fixed seed 1234 [2023-10-08 03:56:25,011][00365] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-08 03:56:25,011][00365] Initializing actor-critic model on device cuda:0 [2023-10-08 03:56:25,012][00365] RunningMeanStd input shape: (4, 84, 84) [2023-10-08 03:56:25,012][00365] RunningMeanStd input shape: (1,) [2023-10-08 03:56:25,024][00365] ConvEncoder: input_channels=4 [2023-10-08 03:56:25,208][00365] Conv encoder output size: 512 [2023-10-08 03:56:25,210][00365] Created Actor Critic model with architecture: [2023-10-08 03:56:25,211][00365] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=7, bias=True) ) ) [2023-10-08 03:56:25,789][00365] Using optimizer [2023-10-08 03:56:25,790][00365] No checkpoints found [2023-10-08 03:56:25,790][00365] Did not load from checkpoint, starting from scratch! [2023-10-08 03:56:25,790][00365] Initialized policy 0 weights for model version 0 [2023-10-08 03:56:25,792][00365] LearnerWorker_p0 finished initialization! [2023-10-08 03:56:25,792][00365] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-08 03:56:26,723][130385] Starting all processes... [2023-10-08 03:56:26,726][00425] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-08 03:56:26,726][00425] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1 [2023-10-08 03:56:26,731][130385] Starting process inference_proc0-0 [2023-10-08 03:56:26,731][130385] Starting process inference_proc1-0 [2023-10-08 03:56:26,732][130385] Starting process rollout_proc0 [2023-10-08 03:56:26,732][130385] Starting process rollout_proc1 [2023-10-08 03:56:26,745][00425] Num visible devices: 1 [2023-10-08 03:56:26,732][130385] Starting process rollout_proc2 [2023-10-08 03:56:26,733][130385] Starting process rollout_proc3 [2023-10-08 03:56:26,733][130385] Starting process rollout_proc4 [2023-10-08 03:56:26,736][130385] Starting process rollout_proc5 [2023-10-08 03:56:26,775][00425] Setting fixed seed 1234 [2023-10-08 03:56:26,776][00425] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-08 03:56:26,777][00425] Initializing actor-critic model on device cuda:0 [2023-10-08 03:56:26,737][130385] Starting process rollout_proc6 [2023-10-08 03:56:26,777][00425] RunningMeanStd input shape: (4, 84, 84) [2023-10-08 03:56:26,778][00425] RunningMeanStd input shape: (1,) [2023-10-08 03:56:26,738][130385] Starting process rollout_proc7 [2023-10-08 03:56:26,740][130385] Starting process rollout_proc8 [2023-10-08 03:56:26,741][130385] Starting process rollout_proc9 [2023-10-08 03:56:26,742][130385] Starting process rollout_proc10 [2023-10-08 03:56:26,745][130385] Starting process rollout_proc11 [2023-10-08 03:56:26,790][00425] ConvEncoder: input_channels=4 [2023-10-08 03:56:26,747][130385] Starting process rollout_proc12 [2023-10-08 03:56:26,748][130385] Starting process rollout_proc13 [2023-10-08 03:56:27,199][00425] Conv encoder output size: 512 [2023-10-08 03:56:27,201][00425] Created Actor Critic model with architecture: [2023-10-08 03:56:27,202][00425] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=7, bias=True) ) ) [2023-10-08 03:56:27,831][00425] Using optimizer [2023-10-08 03:56:27,831][00425] No checkpoints found [2023-10-08 03:56:27,832][00425] Did not load from checkpoint, starting from scratch! [2023-10-08 03:56:27,832][00425] Initialized policy 1 weights for model version 0 [2023-10-08 03:56:27,833][00425] LearnerWorker_p1 finished initialization! [2023-10-08 03:56:27,833][00425] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-08 03:56:28,969][130385] Starting process rollout_proc14 [2023-10-08 03:56:28,973][00646] Worker 2 uses CPU cores [4, 5] [2023-10-08 03:56:29,018][130385] Starting process rollout_proc15 [2023-10-08 03:56:29,023][00647] Worker 3 uses CPU cores [6, 7] [2023-10-08 03:56:29,095][00652] Worker 4 uses CPU cores [8, 9] [2023-10-08 03:56:29,108][00644] Worker 1 uses CPU cores [2, 3] [2023-10-08 03:56:29,136][00656] Worker 10 uses CPU cores [20, 21] [2023-10-08 03:56:29,308][00611] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-08 03:56:29,308][00611] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-10-08 03:56:29,326][00611] Num visible devices: 1 [2023-10-08 03:56:29,348][00655] Worker 9 uses CPU cores [18, 19] [2023-10-08 03:56:29,349][00651] Worker 6 uses CPU cores [12, 13] [2023-10-08 03:56:29,356][00650] Worker 5 uses CPU cores [10, 11] [2023-10-08 03:56:29,360][00612] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-08 03:56:29,360][00612] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1 [2023-10-08 03:56:29,385][00659] Worker 13 uses CPU cores [26, 27] [2023-10-08 03:56:29,396][00612] Num visible devices: 1 [2023-10-08 03:56:29,421][00657] Worker 12 uses CPU cores [24, 25] [2023-10-08 03:56:29,449][00658] Worker 11 uses CPU cores [22, 23] [2023-10-08 03:56:29,455][00654] Worker 8 uses CPU cores [16, 17] [2023-10-08 03:56:29,462][00645] Worker 0 uses CPU cores [0, 1] [2023-10-08 03:56:29,566][00653] Worker 7 uses CPU cores [14, 15] [2023-10-08 03:56:29,932][00611] RunningMeanStd input shape: (4, 84, 84) [2023-10-08 03:56:29,933][00611] RunningMeanStd input shape: (1,) [2023-10-08 03:56:29,945][00611] ConvEncoder: input_channels=4 [2023-10-08 03:56:30,000][00612] RunningMeanStd input shape: (4, 84, 84) [2023-10-08 03:56:30,001][00612] RunningMeanStd input shape: (1,) [2023-10-08 03:56:30,012][00612] ConvEncoder: input_channels=4 [2023-10-08 03:56:30,067][00611] Conv encoder output size: 512 [2023-10-08 03:56:30,110][00612] Conv encoder output size: 512 [2023-10-08 03:56:30,875][01361] Worker 14 uses CPU cores [28, 29] [2023-10-08 03:56:30,913][130385] Inference worker 0-0 is ready! [2023-10-08 03:56:30,914][130385] Inference worker 1-0 is ready! [2023-10-08 03:56:30,914][01411] Worker 15 uses CPU cores [30, 31] [2023-10-08 03:56:30,914][130385] All inference workers are ready! Signal rollout workers to start! [2023-10-08 03:56:30,916][00651] EnvRunner 6-0 uses policy 0 [2023-10-08 03:56:30,916][00654] EnvRunner 8-0 uses policy 0 [2023-10-08 03:56:30,916][00656] EnvRunner 10-0 uses policy 0 [2023-10-08 03:56:30,916][00659] EnvRunner 13-0 uses policy 1 [2023-10-08 03:56:30,916][00650] EnvRunner 5-0 uses policy 1 [2023-10-08 03:56:30,916][00657] EnvRunner 12-0 uses policy 0 [2023-10-08 03:56:30,916][00653] EnvRunner 7-0 uses policy 1 [2023-10-08 03:56:30,916][00652] EnvRunner 4-0 uses policy 0 [2023-10-08 03:56:30,916][00646] EnvRunner 2-0 uses policy 0 [2023-10-08 03:56:30,916][00647] EnvRunner 3-0 uses policy 1 [2023-10-08 03:56:30,916][00655] EnvRunner 9-0 uses policy 1 [2023-10-08 03:56:30,916][00645] EnvRunner 0-0 uses policy 0 [2023-10-08 03:56:30,916][130385] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-08 03:56:30,916][00644] EnvRunner 1-0 uses policy 1 [2023-10-08 03:56:30,916][00658] EnvRunner 11-0 uses policy 1 [2023-10-08 03:56:31,084][01361] EnvRunner 14-0 uses policy 0 [2023-10-08 03:56:31,128][01411] EnvRunner 15-0 uses policy 1 [2023-10-08 03:56:33,218][130385] Heartbeat connected on Batcher_0 [2023-10-08 03:56:33,220][130385] Heartbeat connected on LearnerWorker_p0 [2023-10-08 03:56:33,223][130385] Heartbeat connected on Batcher_1 [2023-10-08 03:56:33,226][130385] Heartbeat connected on LearnerWorker_p1 [2023-10-08 03:56:33,233][130385] Heartbeat connected on InferenceWorker_p0-w0 [2023-10-08 03:56:33,236][130385] Heartbeat connected on InferenceWorker_p1-w0 [2023-10-08 03:56:33,237][130385] Heartbeat connected on RolloutWorker_w0 [2023-10-08 03:56:33,241][130385] Heartbeat connected on RolloutWorker_w1 [2023-10-08 03:56:33,243][130385] Heartbeat connected on RolloutWorker_w2 [2023-10-08 03:56:33,246][130385] Heartbeat connected on RolloutWorker_w3 [2023-10-08 03:56:33,248][130385] Heartbeat connected on RolloutWorker_w4 [2023-10-08 03:56:33,254][130385] Heartbeat connected on RolloutWorker_w6 [2023-10-08 03:56:33,255][130385] Heartbeat connected on RolloutWorker_w5 [2023-10-08 03:56:33,257][130385] Heartbeat connected on RolloutWorker_w7 [2023-10-08 03:56:33,259][130385] Heartbeat connected on RolloutWorker_w8 [2023-10-08 03:56:33,266][130385] Heartbeat connected on RolloutWorker_w10 [2023-10-08 03:56:33,268][130385] Heartbeat connected on RolloutWorker_w9 [2023-10-08 03:56:33,268][130385] Heartbeat connected on RolloutWorker_w11 [2023-10-08 03:56:33,270][130385] Heartbeat connected on RolloutWorker_w12 [2023-10-08 03:56:33,273][130385] Heartbeat connected on RolloutWorker_w13 [2023-10-08 03:56:33,277][130385] Heartbeat connected on RolloutWorker_w14 [2023-10-08 03:56:33,279][130385] Heartbeat connected on RolloutWorker_w15 [2023-10-08 03:56:33,754][130385] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 586.4, 1: 539.9. Samples: 3196. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-08 03:56:33,754][130385] Avg episode reward: [(0, '2.765'), (1, '3.000')] [2023-10-08 03:56:38,754][130385] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 1041.4, 1: 1015.4. Samples: 16120. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-08 03:56:38,754][130385] Avg episode reward: [(0, '3.049'), (1, '3.091')] [2023-10-08 03:56:40,614][00612] Updated weights for policy 1, policy_version 10 (0.0010) [2023-10-08 03:56:40,825][00611] Updated weights for policy 0, policy_version 10 (0.0010) [2023-10-08 03:56:40,990][00612] Updated weights for policy 1, policy_version 20 (0.0008) [2023-10-08 03:56:41,194][00611] Updated weights for policy 0, policy_version 20 (0.0008) [2023-10-08 03:56:41,354][00612] Updated weights for policy 1, policy_version 30 (0.0008) [2023-10-08 03:56:41,572][00611] Updated weights for policy 0, policy_version 30 (0.0010) [2023-10-08 03:56:43,612][00612] Updated weights for policy 1, policy_version 40 (0.0008) [2023-10-08 03:56:43,625][00611] Updated weights for policy 0, policy_version 40 (0.0008) [2023-10-08 03:56:43,754][130385] Fps is (10 sec: 6553.5, 60 sec: 5105.0, 300 sec: 5105.0). Total num frames: 65536. Throughput: 0: 1310.5, 1: 1315.2. Samples: 33708. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 03:56:43,755][130385] Avg episode reward: [(0, '3.140'), (1, '3.040')] [2023-10-08 03:56:43,972][00612] Updated weights for policy 1, policy_version 50 (0.0007) [2023-10-08 03:56:43,990][00611] Updated weights for policy 0, policy_version 50 (0.0009) [2023-10-08 03:56:44,335][00612] Updated weights for policy 1, policy_version 60 (0.0008) [2023-10-08 03:56:44,360][00611] Updated weights for policy 0, policy_version 60 (0.0008) [2023-10-08 03:56:47,559][00612] Updated weights for policy 1, policy_version 70 (0.0009) [2023-10-08 03:56:47,699][00611] Updated weights for policy 0, policy_version 70 (0.0009) [2023-10-08 03:56:47,922][00612] Updated weights for policy 1, policy_version 80 (0.0008) [2023-10-08 03:56:48,065][00611] Updated weights for policy 0, policy_version 80 (0.0009) [2023-10-08 03:56:48,287][00612] Updated weights for policy 1, policy_version 90 (0.0007) [2023-10-08 03:56:48,435][00611] Updated weights for policy 0, policy_version 90 (0.0009) [2023-10-08 03:56:48,754][130385] Fps is (10 sec: 19660.8, 60 sec: 11022.2, 300 sec: 11022.2). Total num frames: 196608. Throughput: 0: 1530.6, 1: 1520.8. Samples: 54430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 03:56:48,754][130385] Avg episode reward: [(0, '3.150'), (1, '3.150')] [2023-10-08 03:56:51,686][00612] Updated weights for policy 1, policy_version 100 (0.0007) [2023-10-08 03:56:51,910][00611] Updated weights for policy 0, policy_version 100 (0.0009) [2023-10-08 03:56:52,047][00612] Updated weights for policy 1, policy_version 110 (0.0008) [2023-10-08 03:56:52,280][00611] Updated weights for policy 0, policy_version 110 (0.0007) [2023-10-08 03:56:52,414][00612] Updated weights for policy 1, policy_version 120 (0.0007) [2023-10-08 03:56:52,653][00611] Updated weights for policy 0, policy_version 120 (0.0007) [2023-10-08 03:56:53,754][130385] Fps is (10 sec: 19660.8, 60 sec: 11478.6, 300 sec: 11478.6). Total num frames: 262144. Throughput: 0: 1438.0, 1: 1445.2. Samples: 65846. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-08 03:56:53,755][130385] Avg episode reward: [(0, '3.040'), (1, '3.240')] [2023-10-08 03:56:53,756][00365] Saving new best policy, reward=3.040! [2023-10-08 03:56:53,756][00425] Saving new best policy, reward=3.240! [2023-10-08 03:56:56,142][00612] Updated weights for policy 1, policy_version 130 (0.0008) [2023-10-08 03:56:56,435][00611] Updated weights for policy 0, policy_version 130 (0.0007) [2023-10-08 03:56:56,504][00612] Updated weights for policy 1, policy_version 140 (0.0008) [2023-10-08 03:56:56,802][00611] Updated weights for policy 0, policy_version 140 (0.0008) [2023-10-08 03:56:56,876][00612] Updated weights for policy 1, policy_version 150 (0.0007) [2023-10-08 03:56:57,178][00611] Updated weights for policy 0, policy_version 150 (0.0008) [2023-10-08 03:56:57,246][00612] Updated weights for policy 1, policy_version 160 (0.0007) [2023-10-08 03:56:57,546][00611] Updated weights for policy 0, policy_version 160 (0.0009) [2023-10-08 03:56:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 11771.1, 300 sec: 11771.1). Total num frames: 327680. Throughput: 0: 1547.1, 1: 1551.5. Samples: 86258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 03:56:58,754][130385] Avg episode reward: [(0, '3.300'), (1, '3.390')] [2023-10-08 03:56:58,755][00425] Saving new best policy, reward=3.390! [2023-10-08 03:56:58,755][00365] Saving new best policy, reward=3.300! [2023-10-08 03:57:00,763][00612] Updated weights for policy 1, policy_version 170 (0.0010) [2023-10-08 03:57:01,134][00612] Updated weights for policy 1, policy_version 180 (0.0007) [2023-10-08 03:57:01,346][00611] Updated weights for policy 0, policy_version 170 (0.0007) [2023-10-08 03:57:01,507][00612] Updated weights for policy 1, policy_version 190 (0.0007) [2023-10-08 03:57:01,714][00611] Updated weights for policy 0, policy_version 180 (0.0009) [2023-10-08 03:57:02,081][00611] Updated weights for policy 0, policy_version 190 (0.0008) [2023-10-08 03:57:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 11974.6, 300 sec: 11974.6). Total num frames: 393216. Throughput: 0: 1628.7, 1: 1669.9. Samples: 108320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 03:57:03,754][130385] Avg episode reward: [(0, '3.190'), (1, '3.020')] [2023-10-08 03:57:05,068][00612] Updated weights for policy 1, policy_version 200 (0.0008) [2023-10-08 03:57:05,443][00612] Updated weights for policy 1, policy_version 210 (0.0008) [2023-10-08 03:57:05,805][00612] Updated weights for policy 1, policy_version 220 (0.0008) [2023-10-08 03:57:05,817][00611] Updated weights for policy 0, policy_version 200 (0.0008) [2023-10-08 03:57:06,183][00611] Updated weights for policy 0, policy_version 210 (0.0008) [2023-10-08 03:57:06,558][00611] Updated weights for policy 0, policy_version 220 (0.0009) [2023-10-08 03:57:08,754][130385] Fps is (10 sec: 13107.0, 60 sec: 12124.2, 300 sec: 12124.2). Total num frames: 458752. Throughput: 0: 1561.4, 1: 1579.5. Samples: 118846. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-08 03:57:08,755][130385] Avg episode reward: [(0, '3.520'), (1, '3.110')] [2023-10-08 03:57:08,756][00365] Saving new best policy, reward=3.520! [2023-10-08 03:57:09,622][00612] Updated weights for policy 1, policy_version 230 (0.0007) [2023-10-08 03:57:09,977][00612] Updated weights for policy 1, policy_version 240 (0.0008) [2023-10-08 03:57:10,340][00612] Updated weights for policy 1, policy_version 250 (0.0008) [2023-10-08 03:57:10,391][00611] Updated weights for policy 0, policy_version 230 (0.0009) [2023-10-08 03:57:10,750][00611] Updated weights for policy 0, policy_version 240 (0.0007) [2023-10-08 03:57:11,125][00611] Updated weights for policy 0, policy_version 250 (0.0009) [2023-10-08 03:57:13,754][130385] Fps is (10 sec: 13107.4, 60 sec: 12239.0, 300 sec: 12239.0). Total num frames: 524288. Throughput: 0: 1621.9, 1: 1666.0. Samples: 140844. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 03:57:13,754][130385] Avg episode reward: [(0, '3.380'), (1, '3.470')] [2023-10-08 03:57:13,755][00425] Saving new best policy, reward=3.470! [2023-10-08 03:57:14,025][00612] Updated weights for policy 1, policy_version 260 (0.0008) [2023-10-08 03:57:14,393][00612] Updated weights for policy 1, policy_version 270 (0.0009) [2023-10-08 03:57:14,755][00612] Updated weights for policy 1, policy_version 280 (0.0010) [2023-10-08 03:57:14,758][00611] Updated weights for policy 0, policy_version 260 (0.0008) [2023-10-08 03:57:15,125][00611] Updated weights for policy 0, policy_version 270 (0.0009) [2023-10-08 03:57:15,504][00611] Updated weights for policy 0, policy_version 280 (0.0011) [2023-10-08 03:57:18,379][00612] Updated weights for policy 1, policy_version 290 (0.0008) [2023-10-08 03:57:18,754][00612] Updated weights for policy 1, policy_version 300 (0.0007) [2023-10-08 03:57:18,754][130385] Fps is (10 sec: 13107.3, 60 sec: 12329.7, 300 sec: 12329.7). Total num frames: 589824. Throughput: 0: 1766.7, 1: 1805.3. Samples: 163936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 03:57:18,755][130385] Avg episode reward: [(0, '3.270'), (1, '3.840')] [2023-10-08 03:57:19,110][00612] Updated weights for policy 1, policy_version 310 (0.0010) [2023-10-08 03:57:19,196][00611] Updated weights for policy 0, policy_version 290 (0.0009) [2023-10-08 03:57:19,474][00612] Updated weights for policy 1, policy_version 320 (0.0008) [2023-10-08 03:57:19,474][00425] Saving new best policy, reward=3.840! [2023-10-08 03:57:19,559][00611] Updated weights for policy 0, policy_version 300 (0.0010) [2023-10-08 03:57:19,929][00611] Updated weights for policy 0, policy_version 310 (0.0009) [2023-10-08 03:57:20,294][00611] Updated weights for policy 0, policy_version 320 (0.0009) [2023-10-08 03:57:23,236][00612] Updated weights for policy 1, policy_version 330 (0.0009) [2023-10-08 03:57:23,605][00612] Updated weights for policy 1, policy_version 340 (0.0008) [2023-10-08 03:57:23,754][130385] Fps is (10 sec: 13106.9, 60 sec: 12403.3, 300 sec: 12403.3). Total num frames: 655360. Throughput: 0: 1733.0, 1: 1775.7. Samples: 174012. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-08 03:57:23,755][130385] Avg episode reward: [(0, '3.810'), (1, '3.710')] [2023-10-08 03:57:23,962][00612] Updated weights for policy 1, policy_version 350 (0.0008) [2023-10-08 03:57:24,051][00611] Updated weights for policy 0, policy_version 330 (0.0008) [2023-10-08 03:57:24,415][00611] Updated weights for policy 0, policy_version 340 (0.0008) [2023-10-08 03:57:24,790][00611] Updated weights for policy 0, policy_version 350 (0.0010) [2023-10-08 03:57:24,863][00365] Saving new best policy, reward=3.810! [2023-10-08 03:57:27,551][00612] Updated weights for policy 1, policy_version 360 (0.0008) [2023-10-08 03:57:27,919][00612] Updated weights for policy 1, policy_version 370 (0.0008) [2023-10-08 03:57:28,288][00612] Updated weights for policy 1, policy_version 380 (0.0007) [2023-10-08 03:57:28,397][00611] Updated weights for policy 0, policy_version 360 (0.0010) [2023-10-08 03:57:28,754][130385] Fps is (10 sec: 16384.0, 60 sec: 13030.7, 300 sec: 13030.7). Total num frames: 753664. Throughput: 0: 1790.4, 1: 1827.6. Samples: 196514. Policy #0 lag: (min: 26.0, avg: 26.3, max: 37.0) [2023-10-08 03:57:28,755][130385] Avg episode reward: [(0, '3.460'), (1, '3.450')] [2023-10-08 03:57:28,762][00611] Updated weights for policy 0, policy_version 370 (0.0010) [2023-10-08 03:57:29,141][00611] Updated weights for policy 0, policy_version 380 (0.0010) [2023-10-08 03:57:32,046][00612] Updated weights for policy 1, policy_version 390 (0.0007) [2023-10-08 03:57:32,406][00612] Updated weights for policy 1, policy_version 400 (0.0008) [2023-10-08 03:57:32,790][00612] Updated weights for policy 1, policy_version 410 (0.0008) [2023-10-08 03:57:32,983][00611] Updated weights for policy 0, policy_version 390 (0.0009) [2023-10-08 03:57:33,349][00611] Updated weights for policy 0, policy_version 400 (0.0007) [2023-10-08 03:57:33,718][00611] Updated weights for policy 0, policy_version 410 (0.0007) [2023-10-08 03:57:33,754][130385] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13036.8). Total num frames: 819200. Throughput: 0: 1796.0, 1: 1817.6. Samples: 217042. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 03:57:33,754][130385] Avg episode reward: [(0, '3.550'), (1, '3.590')] [2023-10-08 03:57:36,553][00612] Updated weights for policy 1, policy_version 420 (0.0009) [2023-10-08 03:57:36,919][00612] Updated weights for policy 1, policy_version 430 (0.0009) [2023-10-08 03:57:37,287][00612] Updated weights for policy 1, policy_version 440 (0.0008) [2023-10-08 03:57:37,484][00611] Updated weights for policy 0, policy_version 420 (0.0008) [2023-10-08 03:57:37,849][00611] Updated weights for policy 0, policy_version 430 (0.0007) [2023-10-08 03:57:38,218][00611] Updated weights for policy 0, policy_version 440 (0.0009) [2023-10-08 03:57:38,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 13525.0). Total num frames: 917504. Throughput: 0: 1790.6, 1: 1829.0. Samples: 228730. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-08 03:57:38,754][130385] Avg episode reward: [(0, '3.490'), (1, '3.690')] [2023-10-08 03:57:41,012][00612] Updated weights for policy 1, policy_version 450 (0.0007) [2023-10-08 03:57:41,385][00612] Updated weights for policy 1, policy_version 460 (0.0008) [2023-10-08 03:57:41,688][00611] Updated weights for policy 0, policy_version 450 (0.0008) [2023-10-08 03:57:41,751][00612] Updated weights for policy 1, policy_version 470 (0.0009) [2023-10-08 03:57:42,060][00611] Updated weights for policy 0, policy_version 460 (0.0010) [2023-10-08 03:57:42,109][00612] Updated weights for policy 1, policy_version 480 (0.0007) [2023-10-08 03:57:42,426][00611] Updated weights for policy 0, policy_version 470 (0.0008) [2023-10-08 03:57:42,788][00611] Updated weights for policy 0, policy_version 480 (0.0008) [2023-10-08 03:57:43,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 13496.3). Total num frames: 983040. Throughput: 0: 1804.8, 1: 1820.5. Samples: 249396. Policy #0 lag: (min: 1.0, avg: 6.4, max: 33.0) [2023-10-08 03:57:43,754][130385] Avg episode reward: [(0, '3.790'), (1, '3.780')] [2023-10-08 03:57:45,878][00612] Updated weights for policy 1, policy_version 490 (0.0010) [2023-10-08 03:57:46,249][00612] Updated weights for policy 1, policy_version 500 (0.0008) [2023-10-08 03:57:46,557][00611] Updated weights for policy 0, policy_version 490 (0.0008) [2023-10-08 03:57:46,606][00612] Updated weights for policy 1, policy_version 510 (0.0007) [2023-10-08 03:57:46,923][00611] Updated weights for policy 0, policy_version 500 (0.0007) [2023-10-08 03:57:47,295][00611] Updated weights for policy 0, policy_version 510 (0.0007) [2023-10-08 03:57:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13471.3). Total num frames: 1048576. Throughput: 0: 1804.7, 1: 1814.3. Samples: 271174. Policy #0 lag: (min: 26.0, avg: 33.3, max: 58.0) [2023-10-08 03:57:48,754][130385] Avg episode reward: [(0, '3.660'), (1, '3.860')] [2023-10-08 03:57:48,763][00425] Saving new best policy, reward=3.860! [2023-10-08 03:57:50,337][00612] Updated weights for policy 1, policy_version 520 (0.0008) [2023-10-08 03:57:50,702][00612] Updated weights for policy 1, policy_version 530 (0.0010) [2023-10-08 03:57:51,034][00611] Updated weights for policy 0, policy_version 520 (0.0007) [2023-10-08 03:57:51,071][00612] Updated weights for policy 1, policy_version 540 (0.0008) [2023-10-08 03:57:51,394][00611] Updated weights for policy 0, policy_version 530 (0.0008) [2023-10-08 03:57:51,765][00611] Updated weights for policy 0, policy_version 540 (0.0009) [2023-10-08 03:57:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13449.4). Total num frames: 1114112. Throughput: 0: 1814.1, 1: 1814.0. Samples: 282108. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 03:57:53,754][130385] Avg episode reward: [(0, '3.390'), (1, '3.860')] [2023-10-08 03:57:54,815][00612] Updated weights for policy 1, policy_version 550 (0.0008) [2023-10-08 03:57:55,178][00612] Updated weights for policy 1, policy_version 560 (0.0008) [2023-10-08 03:57:55,538][00611] Updated weights for policy 0, policy_version 550 (0.0010) [2023-10-08 03:57:55,543][00612] Updated weights for policy 1, policy_version 570 (0.0008) [2023-10-08 03:57:55,907][00611] Updated weights for policy 0, policy_version 560 (0.0010) [2023-10-08 03:57:56,278][00611] Updated weights for policy 0, policy_version 570 (0.0007) [2023-10-08 03:57:58,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 13429.8). Total num frames: 1179648. Throughput: 0: 1815.4, 1: 1806.5. Samples: 303832. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-08 03:57:58,755][130385] Avg episode reward: [(0, '3.680'), (1, '3.860')] [2023-10-08 03:57:59,200][00612] Updated weights for policy 1, policy_version 580 (0.0007) [2023-10-08 03:57:59,564][00612] Updated weights for policy 1, policy_version 590 (0.0008) [2023-10-08 03:57:59,849][00611] Updated weights for policy 0, policy_version 580 (0.0008) [2023-10-08 03:57:59,929][00612] Updated weights for policy 1, policy_version 600 (0.0009) [2023-10-08 03:58:00,213][00611] Updated weights for policy 0, policy_version 590 (0.0007) [2023-10-08 03:58:00,579][00611] Updated weights for policy 0, policy_version 600 (0.0007) [2023-10-08 03:58:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13412.5). Total num frames: 1245184. Throughput: 0: 1806.4, 1: 1802.0. Samples: 326314. Policy #0 lag: (min: 17.0, avg: 20.2, max: 49.0) [2023-10-08 03:58:03,754][130385] Avg episode reward: [(0, '3.910'), (1, '3.640')] [2023-10-08 03:58:03,761][00365] Saving new best policy, reward=3.910! [2023-10-08 03:58:03,768][00612] Updated weights for policy 1, policy_version 610 (0.0008) [2023-10-08 03:58:04,133][00612] Updated weights for policy 1, policy_version 620 (0.0009) [2023-10-08 03:58:04,156][00611] Updated weights for policy 0, policy_version 610 (0.0009) [2023-10-08 03:58:04,503][00612] Updated weights for policy 1, policy_version 630 (0.0010) [2023-10-08 03:58:04,522][00611] Updated weights for policy 0, policy_version 620 (0.0008) [2023-10-08 03:58:04,866][00612] Updated weights for policy 1, policy_version 640 (0.0010) [2023-10-08 03:58:04,900][00611] Updated weights for policy 0, policy_version 630 (0.0009) [2023-10-08 03:58:05,263][00611] Updated weights for policy 0, policy_version 640 (0.0009) [2023-10-08 03:58:08,643][00612] Updated weights for policy 1, policy_version 650 (0.0008) [2023-10-08 03:58:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13396.9). Total num frames: 1310720. Throughput: 0: 1806.0, 1: 1798.0. Samples: 336194. Policy #0 lag: (min: 28.0, avg: 39.2, max: 60.0) [2023-10-08 03:58:08,755][130385] Avg episode reward: [(0, '3.600'), (1, '3.560')] [2023-10-08 03:58:08,946][00611] Updated weights for policy 0, policy_version 650 (0.0007) [2023-10-08 03:58:09,012][00612] Updated weights for policy 1, policy_version 660 (0.0007) [2023-10-08 03:58:09,318][00611] Updated weights for policy 0, policy_version 660 (0.0009) [2023-10-08 03:58:09,373][00612] Updated weights for policy 1, policy_version 670 (0.0007) [2023-10-08 03:58:09,689][00611] Updated weights for policy 0, policy_version 670 (0.0009) [2023-10-08 03:58:13,092][00612] Updated weights for policy 1, policy_version 680 (0.0008) [2023-10-08 03:58:13,465][00612] Updated weights for policy 1, policy_version 690 (0.0008) [2023-10-08 03:58:13,522][00611] Updated weights for policy 0, policy_version 680 (0.0007) [2023-10-08 03:58:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13382.8). Total num frames: 1376256. Throughput: 0: 1810.4, 1: 1796.8. Samples: 358840. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-08 03:58:13,754][130385] Avg episode reward: [(0, '3.790'), (1, '3.820')] [2023-10-08 03:58:13,821][00612] Updated weights for policy 1, policy_version 700 (0.0008) [2023-10-08 03:58:13,898][00611] Updated weights for policy 0, policy_version 690 (0.0008) [2023-10-08 03:58:14,268][00611] Updated weights for policy 0, policy_version 700 (0.0009) [2023-10-08 03:58:17,649][00612] Updated weights for policy 1, policy_version 710 (0.0011) [2023-10-08 03:58:18,022][00612] Updated weights for policy 1, policy_version 720 (0.0007) [2023-10-08 03:58:18,096][00611] Updated weights for policy 0, policy_version 710 (0.0008) [2023-10-08 03:58:18,385][00612] Updated weights for policy 1, policy_version 730 (0.0010) [2023-10-08 03:58:18,462][00611] Updated weights for policy 0, policy_version 720 (0.0008) [2023-10-08 03:58:18,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 13673.9). Total num frames: 1474560. Throughput: 0: 1808.8, 1: 1808.8. Samples: 379836. Policy #0 lag: (min: 15.0, avg: 21.3, max: 47.0) [2023-10-08 03:58:18,755][130385] Avg episode reward: [(0, '4.160'), (1, '3.800')] [2023-10-08 03:58:18,765][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000000736_753664.pth... [2023-10-08 03:58:18,832][00611] Updated weights for policy 0, policy_version 730 (0.0008) [2023-10-08 03:58:19,057][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000000736_753664.pth... [2023-10-08 03:58:19,085][00365] Saving new best policy, reward=4.160! [2023-10-08 03:58:22,072][00612] Updated weights for policy 1, policy_version 740 (0.0008) [2023-10-08 03:58:22,437][00612] Updated weights for policy 1, policy_version 750 (0.0007) [2023-10-08 03:58:22,689][00611] Updated weights for policy 0, policy_version 740 (0.0009) [2023-10-08 03:58:22,814][00612] Updated weights for policy 1, policy_version 760 (0.0008) [2023-10-08 03:58:23,052][00611] Updated weights for policy 0, policy_version 750 (0.0007) [2023-10-08 03:58:23,421][00611] Updated weights for policy 0, policy_version 760 (0.0007) [2023-10-08 03:58:23,754][130385] Fps is (10 sec: 19660.2, 60 sec: 15291.7, 300 sec: 13939.1). Total num frames: 1572864. Throughput: 0: 1802.8, 1: 1797.4. Samples: 390738. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-08 03:58:23,755][130385] Avg episode reward: [(0, '3.990'), (1, '3.860')] [2023-10-08 03:58:26,497][00612] Updated weights for policy 1, policy_version 770 (0.0007) [2023-10-08 03:58:26,858][00612] Updated weights for policy 1, policy_version 780 (0.0008) [2023-10-08 03:58:27,036][00611] Updated weights for policy 0, policy_version 770 (0.0008) [2023-10-08 03:58:27,234][00612] Updated weights for policy 1, policy_version 790 (0.0009) [2023-10-08 03:58:27,405][00611] Updated weights for policy 0, policy_version 780 (0.0008) [2023-10-08 03:58:27,591][00612] Updated weights for policy 1, policy_version 800 (0.0009) [2023-10-08 03:58:27,782][00611] Updated weights for policy 0, policy_version 790 (0.0008) [2023-10-08 03:58:28,153][00611] Updated weights for policy 0, policy_version 800 (0.0011) [2023-10-08 03:58:28,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 13903.9). Total num frames: 1638400. Throughput: 0: 1811.6, 1: 1812.8. Samples: 412496. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 03:58:28,754][130385] Avg episode reward: [(0, '4.010'), (1, '4.040')] [2023-10-08 03:58:28,756][00425] Saving new best policy, reward=4.040! [2023-10-08 03:58:31,224][00612] Updated weights for policy 1, policy_version 810 (0.0007) [2023-10-08 03:58:31,589][00612] Updated weights for policy 1, policy_version 820 (0.0009) [2023-10-08 03:58:31,822][00611] Updated weights for policy 0, policy_version 810 (0.0007) [2023-10-08 03:58:31,961][00612] Updated weights for policy 1, policy_version 830 (0.0009) [2023-10-08 03:58:32,192][00611] Updated weights for policy 0, policy_version 820 (0.0007) [2023-10-08 03:58:32,569][00611] Updated weights for policy 0, policy_version 830 (0.0007) [2023-10-08 03:58:33,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 13871.5). Total num frames: 1703936. Throughput: 0: 1801.6, 1: 1808.7. Samples: 433638. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-08 03:58:33,754][130385] Avg episode reward: [(0, '4.200'), (1, '4.100')] [2023-10-08 03:58:33,763][00425] Saving new best policy, reward=4.100! [2023-10-08 03:58:33,763][00365] Saving new best policy, reward=4.200! [2023-10-08 03:58:35,600][00612] Updated weights for policy 1, policy_version 840 (0.0010) [2023-10-08 03:58:35,970][00612] Updated weights for policy 1, policy_version 850 (0.0010) [2023-10-08 03:58:36,284][00611] Updated weights for policy 0, policy_version 840 (0.0008) [2023-10-08 03:58:36,336][00612] Updated weights for policy 1, policy_version 860 (0.0009) [2023-10-08 03:58:36,663][00611] Updated weights for policy 0, policy_version 850 (0.0007) [2023-10-08 03:58:37,043][00611] Updated weights for policy 0, policy_version 860 (0.0009) [2023-10-08 03:58:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13841.6). Total num frames: 1769472. Throughput: 0: 1805.4, 1: 1817.4. Samples: 445134. Policy #0 lag: (min: 8.0, avg: 28.2, max: 40.0) [2023-10-08 03:58:38,754][130385] Avg episode reward: [(0, '4.140'), (1, '3.890')] [2023-10-08 03:58:39,896][00612] Updated weights for policy 1, policy_version 870 (0.0007) [2023-10-08 03:58:40,272][00612] Updated weights for policy 1, policy_version 880 (0.0007) [2023-10-08 03:58:40,633][00612] Updated weights for policy 1, policy_version 890 (0.0008) [2023-10-08 03:58:40,686][00611] Updated weights for policy 0, policy_version 870 (0.0009) [2023-10-08 03:58:41,057][00611] Updated weights for policy 0, policy_version 880 (0.0010) [2023-10-08 03:58:41,423][00611] Updated weights for policy 0, policy_version 890 (0.0010) [2023-10-08 03:58:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13813.9). Total num frames: 1835008. Throughput: 0: 1794.8, 1: 1814.2. Samples: 466234. Policy #0 lag: (min: 16.0, avg: 38.8, max: 48.0) [2023-10-08 03:58:43,754][130385] Avg episode reward: [(0, '4.350'), (1, '3.810')] [2023-10-08 03:58:43,755][00365] Saving new best policy, reward=4.350! [2023-10-08 03:58:44,465][00612] Updated weights for policy 1, policy_version 900 (0.0008) [2023-10-08 03:58:44,828][00612] Updated weights for policy 1, policy_version 910 (0.0009) [2023-10-08 03:58:45,193][00612] Updated weights for policy 1, policy_version 920 (0.0010) [2023-10-08 03:58:45,295][00611] Updated weights for policy 0, policy_version 900 (0.0009) [2023-10-08 03:58:45,669][00611] Updated weights for policy 0, policy_version 910 (0.0010) [2023-10-08 03:58:46,032][00611] Updated weights for policy 0, policy_version 920 (0.0008) [2023-10-08 03:58:48,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 13788.3). Total num frames: 1900544. Throughput: 0: 1794.3, 1: 1815.6. Samples: 488760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 03:58:48,755][130385] Avg episode reward: [(0, '3.990'), (1, '4.140')] [2023-10-08 03:58:48,922][00612] Updated weights for policy 1, policy_version 930 (0.0009) [2023-10-08 03:58:49,285][00612] Updated weights for policy 1, policy_version 940 (0.0009) [2023-10-08 03:58:49,656][00612] Updated weights for policy 1, policy_version 950 (0.0010) [2023-10-08 03:58:49,802][00611] Updated weights for policy 0, policy_version 930 (0.0008) [2023-10-08 03:58:50,024][00425] Saving new best policy, reward=4.140! [2023-10-08 03:58:50,024][00612] Updated weights for policy 1, policy_version 960 (0.0007) [2023-10-08 03:58:50,171][00611] Updated weights for policy 0, policy_version 940 (0.0010) [2023-10-08 03:58:50,549][00611] Updated weights for policy 0, policy_version 950 (0.0009) [2023-10-08 03:58:50,914][00611] Updated weights for policy 0, policy_version 960 (0.0011) [2023-10-08 03:58:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13764.4). Total num frames: 1966080. Throughput: 0: 1794.4, 1: 1816.3. Samples: 498676. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-08 03:58:53,754][130385] Avg episode reward: [(0, '4.340'), (1, '4.500')] [2023-10-08 03:58:53,871][00612] Updated weights for policy 1, policy_version 970 (0.0007) [2023-10-08 03:58:54,236][00612] Updated weights for policy 1, policy_version 980 (0.0007) [2023-10-08 03:58:54,598][00612] Updated weights for policy 1, policy_version 990 (0.0007) [2023-10-08 03:58:54,672][00425] Saving new best policy, reward=4.500! [2023-10-08 03:58:54,710][00611] Updated weights for policy 0, policy_version 970 (0.0008) [2023-10-08 03:58:55,084][00611] Updated weights for policy 0, policy_version 980 (0.0008) [2023-10-08 03:58:55,466][00611] Updated weights for policy 0, policy_version 990 (0.0010) [2023-10-08 03:58:57,996][00612] Updated weights for policy 1, policy_version 1000 (0.0009) [2023-10-08 03:58:58,364][00612] Updated weights for policy 1, policy_version 1010 (0.0009) [2023-10-08 03:58:58,734][00612] Updated weights for policy 1, policy_version 1020 (0.0009) [2023-10-08 03:58:58,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13742.2). Total num frames: 2031616. Throughput: 0: 1789.7, 1: 1818.6. Samples: 521216. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 03:58:58,755][130385] Avg episode reward: [(0, '4.460'), (1, '5.000')] [2023-10-08 03:58:58,755][00365] Saving new best policy, reward=4.460! [2023-10-08 03:58:58,883][00425] Saving new best policy, reward=5.000! [2023-10-08 03:58:59,334][00611] Updated weights for policy 0, policy_version 1000 (0.0010) [2023-10-08 03:58:59,714][00611] Updated weights for policy 0, policy_version 1010 (0.0009) [2023-10-08 03:59:00,086][00611] Updated weights for policy 0, policy_version 1020 (0.0010) [2023-10-08 03:59:02,538][00612] Updated weights for policy 1, policy_version 1030 (0.0007) [2023-10-08 03:59:02,910][00612] Updated weights for policy 1, policy_version 1040 (0.0008) [2023-10-08 03:59:03,287][00612] Updated weights for policy 1, policy_version 1050 (0.0008) [2023-10-08 03:59:03,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 13935.8). Total num frames: 2129920. Throughput: 0: 1798.0, 1: 1816.7. Samples: 542496. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 03:59:03,755][130385] Avg episode reward: [(0, '4.840'), (1, '4.450')] [2023-10-08 03:59:03,931][00611] Updated weights for policy 0, policy_version 1030 (0.0011) [2023-10-08 03:59:04,301][00611] Updated weights for policy 0, policy_version 1040 (0.0007) [2023-10-08 03:59:04,664][00611] Updated weights for policy 0, policy_version 1050 (0.0009) [2023-10-08 03:59:04,885][00365] Saving new best policy, reward=4.840! [2023-10-08 03:59:07,098][00612] Updated weights for policy 1, policy_version 1060 (0.0007) [2023-10-08 03:59:07,468][00612] Updated weights for policy 1, policy_version 1070 (0.0008) [2023-10-08 03:59:07,836][00612] Updated weights for policy 1, policy_version 1080 (0.0010) [2023-10-08 03:59:08,364][00611] Updated weights for policy 0, policy_version 1060 (0.0009) [2023-10-08 03:59:08,741][00611] Updated weights for policy 0, policy_version 1070 (0.0011) [2023-10-08 03:59:08,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 13909.6). Total num frames: 2195456. Throughput: 0: 1795.8, 1: 1819.3. Samples: 553416. Policy #0 lag: (min: 3.0, avg: 7.1, max: 35.0) [2023-10-08 03:59:08,754][130385] Avg episode reward: [(0, '4.780'), (1, '4.520')] [2023-10-08 03:59:09,113][00611] Updated weights for policy 0, policy_version 1080 (0.0009) [2023-10-08 03:59:11,539][00612] Updated weights for policy 1, policy_version 1090 (0.0009) [2023-10-08 03:59:11,909][00612] Updated weights for policy 1, policy_version 1100 (0.0007) [2023-10-08 03:59:12,280][00612] Updated weights for policy 1, policy_version 1110 (0.0007) [2023-10-08 03:59:12,621][00611] Updated weights for policy 0, policy_version 1090 (0.0009) [2023-10-08 03:59:12,647][00612] Updated weights for policy 1, policy_version 1120 (0.0008) [2023-10-08 03:59:12,987][00611] Updated weights for policy 0, policy_version 1100 (0.0008) [2023-10-08 03:59:13,355][00611] Updated weights for policy 0, policy_version 1110 (0.0007) [2023-10-08 03:59:13,727][00611] Updated weights for policy 0, policy_version 1120 (0.0008) [2023-10-08 03:59:13,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14086.2). Total num frames: 2293760. Throughput: 0: 1799.6, 1: 1818.4. Samples: 575308. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 03:59:13,755][130385] Avg episode reward: [(0, '4.130'), (1, '4.490')] [2023-10-08 03:59:16,343][00612] Updated weights for policy 1, policy_version 1130 (0.0008) [2023-10-08 03:59:16,701][00612] Updated weights for policy 1, policy_version 1140 (0.0009) [2023-10-08 03:59:17,065][00612] Updated weights for policy 1, policy_version 1150 (0.0009) [2023-10-08 03:59:17,491][00611] Updated weights for policy 0, policy_version 1130 (0.0010) [2023-10-08 03:59:17,857][00611] Updated weights for policy 0, policy_version 1140 (0.0007) [2023-10-08 03:59:18,219][00611] Updated weights for policy 0, policy_version 1150 (0.0007) [2023-10-08 03:59:18,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14057.0). Total num frames: 2359296. Throughput: 0: 1799.3, 1: 1812.5. Samples: 596170. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-08 03:59:18,758][130385] Avg episode reward: [(0, '4.230'), (1, '4.660')] [2023-10-08 03:59:20,644][00612] Updated weights for policy 1, policy_version 1160 (0.0009) [2023-10-08 03:59:21,016][00612] Updated weights for policy 1, policy_version 1170 (0.0009) [2023-10-08 03:59:21,382][00612] Updated weights for policy 1, policy_version 1180 (0.0012) [2023-10-08 03:59:21,878][00611] Updated weights for policy 0, policy_version 1160 (0.0007) [2023-10-08 03:59:22,251][00611] Updated weights for policy 0, policy_version 1170 (0.0008) [2023-10-08 03:59:22,621][00611] Updated weights for policy 0, policy_version 1180 (0.0009) [2023-10-08 03:59:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14029.5). Total num frames: 2424832. Throughput: 0: 1801.0, 1: 1817.0. Samples: 607946. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 03:59:23,755][130385] Avg episode reward: [(0, '4.730'), (1, '4.830')] [2023-10-08 03:59:24,964][00612] Updated weights for policy 1, policy_version 1190 (0.0008) [2023-10-08 03:59:25,329][00612] Updated weights for policy 1, policy_version 1200 (0.0008) [2023-10-08 03:59:25,703][00612] Updated weights for policy 1, policy_version 1210 (0.0009) [2023-10-08 03:59:26,461][00611] Updated weights for policy 0, policy_version 1190 (0.0007) [2023-10-08 03:59:26,827][00611] Updated weights for policy 0, policy_version 1200 (0.0011) [2023-10-08 03:59:27,191][00611] Updated weights for policy 0, policy_version 1210 (0.0009) [2023-10-08 03:59:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14003.6). Total num frames: 2490368. Throughput: 0: 1802.7, 1: 1815.7. Samples: 629062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 03:59:28,755][130385] Avg episode reward: [(0, '4.620'), (1, '4.820')] [2023-10-08 03:59:29,393][00612] Updated weights for policy 1, policy_version 1220 (0.0009) [2023-10-08 03:59:29,767][00612] Updated weights for policy 1, policy_version 1230 (0.0009) [2023-10-08 03:59:30,136][00612] Updated weights for policy 1, policy_version 1240 (0.0011) [2023-10-08 03:59:30,867][00611] Updated weights for policy 0, policy_version 1220 (0.0008) [2023-10-08 03:59:31,243][00611] Updated weights for policy 0, policy_version 1230 (0.0010) [2023-10-08 03:59:31,607][00611] Updated weights for policy 0, policy_version 1240 (0.0010) [2023-10-08 03:59:33,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13979.1). Total num frames: 2555904. Throughput: 0: 1798.4, 1: 1820.7. Samples: 651618. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-08 03:59:33,754][130385] Avg episode reward: [(0, '4.840'), (1, '4.630')] [2023-10-08 03:59:33,836][00612] Updated weights for policy 1, policy_version 1250 (0.0009) [2023-10-08 03:59:34,216][00612] Updated weights for policy 1, policy_version 1260 (0.0010) [2023-10-08 03:59:34,582][00612] Updated weights for policy 1, policy_version 1270 (0.0009) [2023-10-08 03:59:34,940][00612] Updated weights for policy 1, policy_version 1280 (0.0009) [2023-10-08 03:59:35,309][00611] Updated weights for policy 0, policy_version 1250 (0.0009) [2023-10-08 03:59:35,678][00611] Updated weights for policy 0, policy_version 1260 (0.0008) [2023-10-08 03:59:36,053][00611] Updated weights for policy 0, policy_version 1270 (0.0009) [2023-10-08 03:59:36,426][00611] Updated weights for policy 0, policy_version 1280 (0.0007) [2023-10-08 03:59:38,630][00612] Updated weights for policy 1, policy_version 1290 (0.0009) [2023-10-08 03:59:38,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13955.9). Total num frames: 2621440. Throughput: 0: 1807.4, 1: 1820.9. Samples: 661950. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 03:59:38,755][130385] Avg episode reward: [(0, '4.870'), (1, '4.450')] [2023-10-08 03:59:38,756][00365] Saving new best policy, reward=4.870! [2023-10-08 03:59:38,995][00612] Updated weights for policy 1, policy_version 1300 (0.0012) [2023-10-08 03:59:39,370][00612] Updated weights for policy 1, policy_version 1310 (0.0009) [2023-10-08 03:59:40,046][00611] Updated weights for policy 0, policy_version 1290 (0.0010) [2023-10-08 03:59:40,417][00611] Updated weights for policy 0, policy_version 1300 (0.0011) [2023-10-08 03:59:40,786][00611] Updated weights for policy 0, policy_version 1310 (0.0008) [2023-10-08 03:59:43,161][00612] Updated weights for policy 1, policy_version 1320 (0.0010) [2023-10-08 03:59:43,536][00612] Updated weights for policy 1, policy_version 1330 (0.0008) [2023-10-08 03:59:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13933.9). Total num frames: 2686976. Throughput: 0: 1803.6, 1: 1823.1. Samples: 684418. Policy #0 lag: (min: 26.0, avg: 29.2, max: 58.0) [2023-10-08 03:59:43,754][130385] Avg episode reward: [(0, '4.250'), (1, '4.280')] [2023-10-08 03:59:43,905][00612] Updated weights for policy 1, policy_version 1340 (0.0008) [2023-10-08 03:59:44,568][00611] Updated weights for policy 0, policy_version 1320 (0.0008) [2023-10-08 03:59:44,946][00611] Updated weights for policy 0, policy_version 1330 (0.0010) [2023-10-08 03:59:45,305][00611] Updated weights for policy 0, policy_version 1340 (0.0009) [2023-10-08 03:59:47,577][00612] Updated weights for policy 1, policy_version 1350 (0.0011) [2023-10-08 03:59:47,942][00612] Updated weights for policy 1, policy_version 1360 (0.0007) [2023-10-08 03:59:48,314][00612] Updated weights for policy 1, policy_version 1370 (0.0010) [2023-10-08 03:59:48,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14078.6). Total num frames: 2785280. Throughput: 0: 1809.3, 1: 1823.1. Samples: 705956. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 03:59:48,755][130385] Avg episode reward: [(0, '4.720'), (1, '4.370')] [2023-10-08 03:59:49,070][00611] Updated weights for policy 0, policy_version 1350 (0.0010) [2023-10-08 03:59:49,443][00611] Updated weights for policy 0, policy_version 1360 (0.0008) [2023-10-08 03:59:49,812][00611] Updated weights for policy 0, policy_version 1370 (0.0008) [2023-10-08 03:59:52,002][00612] Updated weights for policy 1, policy_version 1380 (0.0007) [2023-10-08 03:59:52,362][00612] Updated weights for policy 1, policy_version 1390 (0.0007) [2023-10-08 03:59:52,745][00612] Updated weights for policy 1, policy_version 1400 (0.0011) [2023-10-08 03:59:53,524][00611] Updated weights for policy 0, policy_version 1380 (0.0009) [2023-10-08 03:59:53,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14054.7). Total num frames: 2850816. Throughput: 0: 1805.7, 1: 1819.6. Samples: 716556. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) [2023-10-08 03:59:53,754][130385] Avg episode reward: [(0, '5.380'), (1, '4.890')] [2023-10-08 03:59:53,900][00611] Updated weights for policy 0, policy_version 1390 (0.0009) [2023-10-08 03:59:54,268][00611] Updated weights for policy 0, policy_version 1400 (0.0008) [2023-10-08 03:59:54,550][00365] Saving new best policy, reward=5.380! [2023-10-08 03:59:56,387][00612] Updated weights for policy 1, policy_version 1410 (0.0008) [2023-10-08 03:59:56,756][00612] Updated weights for policy 1, policy_version 1420 (0.0007) [2023-10-08 03:59:57,125][00612] Updated weights for policy 1, policy_version 1430 (0.0009) [2023-10-08 03:59:57,491][00612] Updated weights for policy 1, policy_version 1440 (0.0008) [2023-10-08 03:59:57,737][00611] Updated weights for policy 0, policy_version 1410 (0.0008) [2023-10-08 03:59:58,104][00611] Updated weights for policy 0, policy_version 1420 (0.0008) [2023-10-08 03:59:58,477][00611] Updated weights for policy 0, policy_version 1430 (0.0008) [2023-10-08 03:59:58,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14031.9). Total num frames: 2916352. Throughput: 0: 1813.0, 1: 1820.0. Samples: 738790. Policy #0 lag: (min: 10.0, avg: 13.7, max: 42.0) [2023-10-08 03:59:58,754][130385] Avg episode reward: [(0, '5.810'), (1, '5.000')] [2023-10-08 03:59:58,848][00365] Saving new best policy, reward=5.810! [2023-10-08 03:59:58,848][00611] Updated weights for policy 0, policy_version 1440 (0.0008) [2023-10-08 04:00:01,008][00612] Updated weights for policy 1, policy_version 1450 (0.0008) [2023-10-08 04:00:01,374][00612] Updated weights for policy 1, policy_version 1460 (0.0007) [2023-10-08 04:00:01,748][00612] Updated weights for policy 1, policy_version 1470 (0.0007) [2023-10-08 04:00:02,609][00611] Updated weights for policy 0, policy_version 1450 (0.0008) [2023-10-08 04:00:02,978][00611] Updated weights for policy 0, policy_version 1460 (0.0009) [2023-10-08 04:00:03,344][00611] Updated weights for policy 0, policy_version 1470 (0.0007) [2023-10-08 04:00:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14164.1). Total num frames: 3014656. Throughput: 0: 1820.0, 1: 1829.2. Samples: 760384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:00:03,754][130385] Avg episode reward: [(0, '5.410'), (1, '4.910')] [2023-10-08 04:00:05,345][00612] Updated weights for policy 1, policy_version 1480 (0.0007) [2023-10-08 04:00:05,715][00612] Updated weights for policy 1, policy_version 1490 (0.0007) [2023-10-08 04:00:06,085][00612] Updated weights for policy 1, policy_version 1500 (0.0009) [2023-10-08 04:00:07,090][00611] Updated weights for policy 0, policy_version 1480 (0.0008) [2023-10-08 04:00:07,456][00611] Updated weights for policy 0, policy_version 1490 (0.0008) [2023-10-08 04:00:07,824][00611] Updated weights for policy 0, policy_version 1500 (0.0010) [2023-10-08 04:00:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14139.9). Total num frames: 3080192. Throughput: 0: 1818.2, 1: 1819.9. Samples: 771660. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-08 04:00:08,754][130385] Avg episode reward: [(0, '5.130'), (1, '5.070')] [2023-10-08 04:00:08,755][00425] Saving new best policy, reward=5.070! [2023-10-08 04:00:09,742][00612] Updated weights for policy 1, policy_version 1510 (0.0008) [2023-10-08 04:00:10,120][00612] Updated weights for policy 1, policy_version 1520 (0.0008) [2023-10-08 04:00:10,482][00612] Updated weights for policy 1, policy_version 1530 (0.0010) [2023-10-08 04:00:11,465][00611] Updated weights for policy 0, policy_version 1510 (0.0009) [2023-10-08 04:00:11,832][00611] Updated weights for policy 0, policy_version 1520 (0.0007) [2023-10-08 04:00:12,203][00611] Updated weights for policy 0, policy_version 1530 (0.0009) [2023-10-08 04:00:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14116.7). Total num frames: 3145728. Throughput: 0: 1820.5, 1: 1828.4. Samples: 793262. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 04:00:13,754][130385] Avg episode reward: [(0, '5.180'), (1, '5.280')] [2023-10-08 04:00:13,755][00425] Saving new best policy, reward=5.280! [2023-10-08 04:00:14,291][00612] Updated weights for policy 1, policy_version 1540 (0.0010) [2023-10-08 04:00:14,662][00612] Updated weights for policy 1, policy_version 1550 (0.0009) [2023-10-08 04:00:15,039][00612] Updated weights for policy 1, policy_version 1560 (0.0007) [2023-10-08 04:00:15,781][00611] Updated weights for policy 0, policy_version 1540 (0.0010) [2023-10-08 04:00:16,148][00611] Updated weights for policy 0, policy_version 1550 (0.0007) [2023-10-08 04:00:16,524][00611] Updated weights for policy 0, policy_version 1560 (0.0009) [2023-10-08 04:00:18,692][00612] Updated weights for policy 1, policy_version 1570 (0.0009) [2023-10-08 04:00:18,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14094.5). Total num frames: 3211264. Throughput: 0: 1824.6, 1: 1827.3. Samples: 815956. Policy #0 lag: (min: 17.0, avg: 25.2, max: 49.0) [2023-10-08 04:00:18,754][130385] Avg episode reward: [(0, '5.200'), (1, '5.150')] [2023-10-08 04:00:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000001568_1605632.pth... [2023-10-08 04:00:19,058][00612] Updated weights for policy 1, policy_version 1580 (0.0010) [2023-10-08 04:00:19,434][00612] Updated weights for policy 1, policy_version 1590 (0.0010) [2023-10-08 04:00:19,799][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000001600_1638400.pth... [2023-10-08 04:00:19,799][00612] Updated weights for policy 1, policy_version 1600 (0.0010) [2023-10-08 04:00:20,135][00611] Updated weights for policy 0, policy_version 1570 (0.0009) [2023-10-08 04:00:20,496][00611] Updated weights for policy 0, policy_version 1580 (0.0007) [2023-10-08 04:00:20,877][00611] Updated weights for policy 0, policy_version 1590 (0.0008) [2023-10-08 04:00:21,245][00611] Updated weights for policy 0, policy_version 1600 (0.0009) [2023-10-08 04:00:23,413][00612] Updated weights for policy 1, policy_version 1610 (0.0009) [2023-10-08 04:00:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14073.3). Total num frames: 3276800. Throughput: 0: 1823.2, 1: 1826.1. Samples: 826170. Policy #0 lag: (min: 30.0, avg: 33.1, max: 62.0) [2023-10-08 04:00:23,754][130385] Avg episode reward: [(0, '5.310'), (1, '5.440')] [2023-10-08 04:00:23,795][00612] Updated weights for policy 1, policy_version 1620 (0.0007) [2023-10-08 04:00:24,160][00612] Updated weights for policy 1, policy_version 1630 (0.0007) [2023-10-08 04:00:24,231][00425] Saving new best policy, reward=5.440! [2023-10-08 04:00:24,959][00611] Updated weights for policy 0, policy_version 1610 (0.0007) [2023-10-08 04:00:25,323][00611] Updated weights for policy 0, policy_version 1620 (0.0008) [2023-10-08 04:00:25,697][00611] Updated weights for policy 0, policy_version 1630 (0.0008) [2023-10-08 04:00:27,972][00612] Updated weights for policy 1, policy_version 1640 (0.0007) [2023-10-08 04:00:28,359][00612] Updated weights for policy 1, policy_version 1650 (0.0008) [2023-10-08 04:00:28,728][00612] Updated weights for policy 1, policy_version 1660 (0.0010) [2023-10-08 04:00:28,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14053.0). Total num frames: 3342336. Throughput: 0: 1823.9, 1: 1823.9. Samples: 848570. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 04:00:28,755][130385] Avg episode reward: [(0, '5.520'), (1, '5.220')] [2023-10-08 04:00:29,549][00611] Updated weights for policy 0, policy_version 1640 (0.0008) [2023-10-08 04:00:29,917][00611] Updated weights for policy 0, policy_version 1650 (0.0009) [2023-10-08 04:00:30,293][00611] Updated weights for policy 0, policy_version 1660 (0.0008) [2023-10-08 04:00:32,279][00612] Updated weights for policy 1, policy_version 1670 (0.0011) [2023-10-08 04:00:32,654][00612] Updated weights for policy 1, policy_version 1680 (0.0007) [2023-10-08 04:00:33,031][00612] Updated weights for policy 1, policy_version 1690 (0.0008) [2023-10-08 04:00:33,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14168.5). Total num frames: 3440640. Throughput: 0: 1824.8, 1: 1818.9. Samples: 869922. Policy #0 lag: (min: 4.0, avg: 6.9, max: 36.0) [2023-10-08 04:00:33,755][130385] Avg episode reward: [(0, '5.520'), (1, '5.350')] [2023-10-08 04:00:33,940][00611] Updated weights for policy 0, policy_version 1670 (0.0009) [2023-10-08 04:00:34,319][00611] Updated weights for policy 0, policy_version 1680 (0.0007) [2023-10-08 04:00:34,692][00611] Updated weights for policy 0, policy_version 1690 (0.0008) [2023-10-08 04:00:36,667][00612] Updated weights for policy 1, policy_version 1700 (0.0009) [2023-10-08 04:00:37,025][00612] Updated weights for policy 1, policy_version 1710 (0.0007) [2023-10-08 04:00:37,401][00612] Updated weights for policy 1, policy_version 1720 (0.0007) [2023-10-08 04:00:38,255][00611] Updated weights for policy 0, policy_version 1700 (0.0008) [2023-10-08 04:00:38,634][00611] Updated weights for policy 0, policy_version 1710 (0.0010) [2023-10-08 04:00:38,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14147.1). Total num frames: 3506176. Throughput: 0: 1826.9, 1: 1827.3. Samples: 880996. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:00:38,754][130385] Avg episode reward: [(0, '5.480'), (1, '5.560')] [2023-10-08 04:00:38,755][00425] Saving new best policy, reward=5.560! [2023-10-08 04:00:38,994][00611] Updated weights for policy 0, policy_version 1720 (0.0009) [2023-10-08 04:00:41,064][00612] Updated weights for policy 1, policy_version 1730 (0.0008) [2023-10-08 04:00:41,435][00612] Updated weights for policy 1, policy_version 1740 (0.0008) [2023-10-08 04:00:41,795][00612] Updated weights for policy 1, policy_version 1750 (0.0008) [2023-10-08 04:00:42,164][00612] Updated weights for policy 1, policy_version 1760 (0.0008) [2023-10-08 04:00:42,629][00611] Updated weights for policy 0, policy_version 1730 (0.0007) [2023-10-08 04:00:42,990][00611] Updated weights for policy 0, policy_version 1740 (0.0007) [2023-10-08 04:00:43,366][00611] Updated weights for policy 0, policy_version 1750 (0.0008) [2023-10-08 04:00:43,738][00611] Updated weights for policy 0, policy_version 1760 (0.0007) [2023-10-08 04:00:43,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14256.1). Total num frames: 3604480. Throughput: 0: 1826.0, 1: 1817.0. Samples: 902722. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-10-08 04:00:43,755][130385] Avg episode reward: [(0, '5.900'), (1, '5.720')] [2023-10-08 04:00:43,755][00425] Saving new best policy, reward=5.720! [2023-10-08 04:00:43,755][00365] Saving new best policy, reward=5.900! [2023-10-08 04:00:45,758][00612] Updated weights for policy 1, policy_version 1770 (0.0008) [2023-10-08 04:00:46,127][00612] Updated weights for policy 1, policy_version 1780 (0.0007) [2023-10-08 04:00:46,486][00612] Updated weights for policy 1, policy_version 1790 (0.0009) [2023-10-08 04:00:47,399][00611] Updated weights for policy 0, policy_version 1770 (0.0008) [2023-10-08 04:00:47,771][00611] Updated weights for policy 0, policy_version 1780 (0.0008) [2023-10-08 04:00:48,151][00611] Updated weights for policy 0, policy_version 1790 (0.0009) [2023-10-08 04:00:48,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14233.8). Total num frames: 3670016. Throughput: 0: 1821.1, 1: 1823.1. Samples: 924370. Policy #0 lag: (min: 8.0, avg: 20.3, max: 40.0) [2023-10-08 04:00:48,755][130385] Avg episode reward: [(0, '5.940'), (1, '5.280')] [2023-10-08 04:00:48,764][00365] Saving new best policy, reward=5.940! [2023-10-08 04:00:50,290][00612] Updated weights for policy 1, policy_version 1800 (0.0007) [2023-10-08 04:00:50,656][00612] Updated weights for policy 1, policy_version 1810 (0.0008) [2023-10-08 04:00:51,025][00612] Updated weights for policy 1, policy_version 1820 (0.0009) [2023-10-08 04:00:51,828][00611] Updated weights for policy 0, policy_version 1800 (0.0009) [2023-10-08 04:00:52,201][00611] Updated weights for policy 0, policy_version 1810 (0.0010) [2023-10-08 04:00:52,569][00611] Updated weights for policy 0, policy_version 1820 (0.0010) [2023-10-08 04:00:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14212.4). Total num frames: 3735552. Throughput: 0: 1823.3, 1: 1821.2. Samples: 935660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:00:53,754][130385] Avg episode reward: [(0, '6.040'), (1, '5.480')] [2023-10-08 04:00:53,755][00365] Saving new best policy, reward=6.040! [2023-10-08 04:00:55,239][00612] Updated weights for policy 1, policy_version 1830 (0.0010) [2023-10-08 04:00:55,610][00612] Updated weights for policy 1, policy_version 1840 (0.0010) [2023-10-08 04:00:55,976][00612] Updated weights for policy 1, policy_version 1850 (0.0010) [2023-10-08 04:00:56,811][00611] Updated weights for policy 0, policy_version 1830 (0.0008) [2023-10-08 04:00:57,189][00611] Updated weights for policy 0, policy_version 1840 (0.0008) [2023-10-08 04:00:57,557][00611] Updated weights for policy 0, policy_version 1850 (0.0008) [2023-10-08 04:00:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14191.8). Total num frames: 3801088. Throughput: 0: 1807.2, 1: 1789.5. Samples: 955114. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-08 04:00:58,754][130385] Avg episode reward: [(0, '5.600'), (1, '5.580')] [2023-10-08 04:00:59,959][00612] Updated weights for policy 1, policy_version 1860 (0.0009) [2023-10-08 04:01:00,327][00612] Updated weights for policy 1, policy_version 1870 (0.0009) [2023-10-08 04:01:00,693][00612] Updated weights for policy 1, policy_version 1880 (0.0008) [2023-10-08 04:01:01,414][00611] Updated weights for policy 0, policy_version 1860 (0.0009) [2023-10-08 04:01:01,794][00611] Updated weights for policy 0, policy_version 1870 (0.0007) [2023-10-08 04:01:02,163][00611] Updated weights for policy 0, policy_version 1880 (0.0008) [2023-10-08 04:01:03,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14171.9). Total num frames: 3866624. Throughput: 0: 1783.6, 1: 1784.1. Samples: 976502. Policy #0 lag: (min: 26.0, avg: 28.9, max: 58.0) [2023-10-08 04:01:03,755][130385] Avg episode reward: [(0, '5.950'), (1, '5.010')] [2023-10-08 04:01:04,519][00612] Updated weights for policy 1, policy_version 1890 (0.0007) [2023-10-08 04:01:04,881][00612] Updated weights for policy 1, policy_version 1900 (0.0007) [2023-10-08 04:01:05,253][00612] Updated weights for policy 1, policy_version 1910 (0.0007) [2023-10-08 04:01:05,621][00612] Updated weights for policy 1, policy_version 1920 (0.0009) [2023-10-08 04:01:05,914][00611] Updated weights for policy 0, policy_version 1890 (0.0008) [2023-10-08 04:01:06,291][00611] Updated weights for policy 0, policy_version 1900 (0.0008) [2023-10-08 04:01:06,652][00611] Updated weights for policy 0, policy_version 1910 (0.0007) [2023-10-08 04:01:07,022][00611] Updated weights for policy 0, policy_version 1920 (0.0007) [2023-10-08 04:01:08,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14152.7). Total num frames: 3932160. Throughput: 0: 1798.6, 1: 1784.6. Samples: 987414. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 04:01:08,754][130385] Avg episode reward: [(0, '6.110'), (1, '5.420')] [2023-10-08 04:01:08,755][00365] Saving new best policy, reward=6.110! [2023-10-08 04:01:09,238][00612] Updated weights for policy 1, policy_version 1930 (0.0007) [2023-10-08 04:01:09,608][00612] Updated weights for policy 1, policy_version 1940 (0.0007) [2023-10-08 04:01:09,983][00612] Updated weights for policy 1, policy_version 1950 (0.0007) [2023-10-08 04:01:10,719][00611] Updated weights for policy 0, policy_version 1930 (0.0010) [2023-10-08 04:01:11,086][00611] Updated weights for policy 0, policy_version 1940 (0.0011) [2023-10-08 04:01:11,454][00611] Updated weights for policy 0, policy_version 1950 (0.0010) [2023-10-08 04:01:13,634][00612] Updated weights for policy 1, policy_version 1960 (0.0010) [2023-10-08 04:01:13,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14134.2). Total num frames: 3997696. Throughput: 0: 1778.1, 1: 1790.8. Samples: 1009174. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:01:13,756][130385] Avg episode reward: [(0, '5.850'), (1, '5.090')] [2023-10-08 04:01:14,000][00612] Updated weights for policy 1, policy_version 1970 (0.0010) [2023-10-08 04:01:14,372][00612] Updated weights for policy 1, policy_version 1980 (0.0009) [2023-10-08 04:01:15,296][00611] Updated weights for policy 0, policy_version 1960 (0.0010) [2023-10-08 04:01:15,673][00611] Updated weights for policy 0, policy_version 1970 (0.0010) [2023-10-08 04:01:16,049][00611] Updated weights for policy 0, policy_version 1980 (0.0008) [2023-10-08 04:01:17,989][00612] Updated weights for policy 1, policy_version 1990 (0.0009) [2023-10-08 04:01:18,355][00612] Updated weights for policy 1, policy_version 2000 (0.0007) [2023-10-08 04:01:18,717][00612] Updated weights for policy 1, policy_version 2010 (0.0008) [2023-10-08 04:01:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14116.4). Total num frames: 4063232. Throughput: 0: 1781.0, 1: 1806.7. Samples: 1031366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:01:18,755][130385] Avg episode reward: [(0, '6.050'), (1, '5.650')] [2023-10-08 04:01:19,650][00611] Updated weights for policy 0, policy_version 1990 (0.0007) [2023-10-08 04:01:20,016][00611] Updated weights for policy 0, policy_version 2000 (0.0007) [2023-10-08 04:01:20,394][00611] Updated weights for policy 0, policy_version 2010 (0.0008) [2023-10-08 04:01:22,318][00612] Updated weights for policy 1, policy_version 2020 (0.0008) [2023-10-08 04:01:22,687][00612] Updated weights for policy 1, policy_version 2030 (0.0008) [2023-10-08 04:01:23,057][00612] Updated weights for policy 1, policy_version 2040 (0.0007) [2023-10-08 04:01:23,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14211.1). Total num frames: 4161536. Throughput: 0: 1783.0, 1: 1791.6. Samples: 1041854. Policy #0 lag: (min: 1.0, avg: 4.1, max: 33.0) [2023-10-08 04:01:23,755][130385] Avg episode reward: [(0, '6.310'), (1, '5.480')] [2023-10-08 04:01:23,756][00365] Saving new best policy, reward=6.310! [2023-10-08 04:01:24,122][00611] Updated weights for policy 0, policy_version 2020 (0.0008) [2023-10-08 04:01:24,494][00611] Updated weights for policy 0, policy_version 2030 (0.0009) [2023-10-08 04:01:24,875][00611] Updated weights for policy 0, policy_version 2040 (0.0009) [2023-10-08 04:01:26,703][00612] Updated weights for policy 1, policy_version 2050 (0.0007) [2023-10-08 04:01:27,071][00612] Updated weights for policy 1, policy_version 2060 (0.0007) [2023-10-08 04:01:27,444][00612] Updated weights for policy 1, policy_version 2070 (0.0008) [2023-10-08 04:01:27,811][00612] Updated weights for policy 1, policy_version 2080 (0.0007) [2023-10-08 04:01:28,458][00611] Updated weights for policy 0, policy_version 2050 (0.0009) [2023-10-08 04:01:28,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 4227072. Throughput: 0: 1777.1, 1: 1811.5. Samples: 1064208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:01:28,755][130385] Avg episode reward: [(0, '6.100'), (1, '5.820')] [2023-10-08 04:01:28,755][00425] Saving new best policy, reward=5.820! [2023-10-08 04:01:28,830][00611] Updated weights for policy 0, policy_version 2060 (0.0007) [2023-10-08 04:01:29,201][00611] Updated weights for policy 0, policy_version 2070 (0.0007) [2023-10-08 04:01:29,582][00611] Updated weights for policy 0, policy_version 2080 (0.0008) [2023-10-08 04:01:31,513][00612] Updated weights for policy 1, policy_version 2090 (0.0009) [2023-10-08 04:01:31,874][00612] Updated weights for policy 1, policy_version 2100 (0.0007) [2023-10-08 04:01:32,241][00612] Updated weights for policy 1, policy_version 2110 (0.0008) [2023-10-08 04:01:33,276][00611] Updated weights for policy 0, policy_version 2090 (0.0009) [2023-10-08 04:01:33,638][00611] Updated weights for policy 0, policy_version 2100 (0.0007) [2023-10-08 04:01:33,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 4292608. Throughput: 0: 1800.5, 1: 1787.4. Samples: 1085826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:01:33,754][130385] Avg episode reward: [(0, '6.460'), (1, '6.150')] [2023-10-08 04:01:33,761][00425] Saving new best policy, reward=6.150! [2023-10-08 04:01:34,007][00611] Updated weights for policy 0, policy_version 2110 (0.0008) [2023-10-08 04:01:34,081][00365] Saving new best policy, reward=6.460! [2023-10-08 04:01:35,936][00612] Updated weights for policy 1, policy_version 2120 (0.0011) [2023-10-08 04:01:36,307][00612] Updated weights for policy 1, policy_version 2130 (0.0010) [2023-10-08 04:01:36,677][00612] Updated weights for policy 1, policy_version 2140 (0.0009) [2023-10-08 04:01:37,736][00611] Updated weights for policy 0, policy_version 2120 (0.0009) [2023-10-08 04:01:38,115][00611] Updated weights for policy 0, policy_version 2130 (0.0008) [2023-10-08 04:01:38,477][00611] Updated weights for policy 0, policy_version 2140 (0.0007) [2023-10-08 04:01:38,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 4390912. Throughput: 0: 1775.0, 1: 1807.0. Samples: 1096848. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 04:01:38,755][130385] Avg episode reward: [(0, '6.460'), (1, '6.040')] [2023-10-08 04:01:40,306][00612] Updated weights for policy 1, policy_version 2150 (0.0009) [2023-10-08 04:01:40,675][00612] Updated weights for policy 1, policy_version 2160 (0.0009) [2023-10-08 04:01:41,038][00612] Updated weights for policy 1, policy_version 2170 (0.0009) [2023-10-08 04:01:42,463][00611] Updated weights for policy 0, policy_version 2150 (0.0010) [2023-10-08 04:01:42,840][00611] Updated weights for policy 0, policy_version 2160 (0.0009) [2023-10-08 04:01:43,211][00611] Updated weights for policy 0, policy_version 2170 (0.0009) [2023-10-08 04:01:43,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4456448. Throughput: 0: 1821.7, 1: 1817.5. Samples: 1118876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:01:43,754][130385] Avg episode reward: [(0, '6.210'), (1, '6.230')] [2023-10-08 04:01:43,755][00425] Saving new best policy, reward=6.230! [2023-10-08 04:01:44,745][00612] Updated weights for policy 1, policy_version 2180 (0.0008) [2023-10-08 04:01:45,115][00612] Updated weights for policy 1, policy_version 2190 (0.0007) [2023-10-08 04:01:45,480][00612] Updated weights for policy 1, policy_version 2200 (0.0009) [2023-10-08 04:01:46,911][00611] Updated weights for policy 0, policy_version 2180 (0.0009) [2023-10-08 04:01:47,277][00611] Updated weights for policy 0, policy_version 2190 (0.0008) [2023-10-08 04:01:47,644][00611] Updated weights for policy 0, policy_version 2200 (0.0008) [2023-10-08 04:01:48,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4521984. Throughput: 0: 1812.5, 1: 1832.1. Samples: 1140508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:01:48,754][130385] Avg episode reward: [(0, '6.790'), (1, '6.230')] [2023-10-08 04:01:48,761][00365] Saving new best policy, reward=6.790! [2023-10-08 04:01:49,078][00612] Updated weights for policy 1, policy_version 2210 (0.0007) [2023-10-08 04:01:49,445][00612] Updated weights for policy 1, policy_version 2220 (0.0007) [2023-10-08 04:01:49,815][00612] Updated weights for policy 1, policy_version 2230 (0.0010) [2023-10-08 04:01:50,191][00612] Updated weights for policy 1, policy_version 2240 (0.0007) [2023-10-08 04:01:51,338][00611] Updated weights for policy 0, policy_version 2210 (0.0007) [2023-10-08 04:01:51,718][00611] Updated weights for policy 0, policy_version 2220 (0.0008) [2023-10-08 04:01:52,084][00611] Updated weights for policy 0, policy_version 2230 (0.0008) [2023-10-08 04:01:52,462][00611] Updated weights for policy 0, policy_version 2240 (0.0010) [2023-10-08 04:01:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4587520. Throughput: 0: 1826.5, 1: 1833.0. Samples: 1152094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:01:53,754][130385] Avg episode reward: [(0, '7.400'), (1, '5.920')] [2023-10-08 04:01:53,755][00365] Saving new best policy, reward=7.400! [2023-10-08 04:01:53,842][00612] Updated weights for policy 1, policy_version 2250 (0.0007) [2023-10-08 04:01:54,213][00612] Updated weights for policy 1, policy_version 2260 (0.0010) [2023-10-08 04:01:54,587][00612] Updated weights for policy 1, policy_version 2270 (0.0008) [2023-10-08 04:01:56,025][00611] Updated weights for policy 0, policy_version 2250 (0.0010) [2023-10-08 04:01:56,394][00611] Updated weights for policy 0, policy_version 2260 (0.0009) [2023-10-08 04:01:56,760][00611] Updated weights for policy 0, policy_version 2270 (0.0007) [2023-10-08 04:01:58,364][00612] Updated weights for policy 1, policy_version 2280 (0.0007) [2023-10-08 04:01:58,733][00612] Updated weights for policy 1, policy_version 2290 (0.0009) [2023-10-08 04:01:58,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 4653056. Throughput: 0: 1818.2, 1: 1830.2. Samples: 1173350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:01:58,755][130385] Avg episode reward: [(0, '6.720'), (1, '5.900')] [2023-10-08 04:01:59,110][00612] Updated weights for policy 1, policy_version 2300 (0.0008) [2023-10-08 04:02:00,431][00611] Updated weights for policy 0, policy_version 2280 (0.0009) [2023-10-08 04:02:00,813][00611] Updated weights for policy 0, policy_version 2290 (0.0008) [2023-10-08 04:02:01,178][00611] Updated weights for policy 0, policy_version 2300 (0.0009) [2023-10-08 04:02:02,896][00612] Updated weights for policy 1, policy_version 2310 (0.0007) [2023-10-08 04:02:03,275][00612] Updated weights for policy 1, policy_version 2320 (0.0008) [2023-10-08 04:02:03,642][00612] Updated weights for policy 1, policy_version 2330 (0.0007) [2023-10-08 04:02:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4718592. Throughput: 0: 1817.8, 1: 1827.3. Samples: 1195394. Policy #0 lag: (min: 9.0, avg: 16.2, max: 41.0) [2023-10-08 04:02:03,754][130385] Avg episode reward: [(0, '6.910'), (1, '6.210')] [2023-10-08 04:02:04,793][00611] Updated weights for policy 0, policy_version 2310 (0.0007) [2023-10-08 04:02:05,164][00611] Updated weights for policy 0, policy_version 2320 (0.0008) [2023-10-08 04:02:05,536][00611] Updated weights for policy 0, policy_version 2330 (0.0007) [2023-10-08 04:02:07,285][00612] Updated weights for policy 1, policy_version 2340 (0.0008) [2023-10-08 04:02:07,662][00612] Updated weights for policy 1, policy_version 2350 (0.0008) [2023-10-08 04:02:08,029][00612] Updated weights for policy 1, policy_version 2360 (0.0008) [2023-10-08 04:02:08,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 4816896. Throughput: 0: 1819.1, 1: 1834.1. Samples: 1206248. Policy #0 lag: (min: 31.0, avg: 44.2, max: 63.0) [2023-10-08 04:02:08,754][130385] Avg episode reward: [(0, '7.220'), (1, '6.470')] [2023-10-08 04:02:08,755][00425] Saving new best policy, reward=6.470! [2023-10-08 04:02:09,341][00611] Updated weights for policy 0, policy_version 2340 (0.0009) [2023-10-08 04:02:09,716][00611] Updated weights for policy 0, policy_version 2350 (0.0009) [2023-10-08 04:02:10,083][00611] Updated weights for policy 0, policy_version 2360 (0.0007) [2023-10-08 04:02:11,870][00612] Updated weights for policy 1, policy_version 2370 (0.0011) [2023-10-08 04:02:12,242][00612] Updated weights for policy 1, policy_version 2380 (0.0007) [2023-10-08 04:02:12,606][00612] Updated weights for policy 1, policy_version 2390 (0.0008) [2023-10-08 04:02:12,973][00612] Updated weights for policy 1, policy_version 2400 (0.0008) [2023-10-08 04:02:13,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 4882432. Throughput: 0: 1808.5, 1: 1823.5. Samples: 1227650. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 04:02:13,754][130385] Avg episode reward: [(0, '7.120'), (1, '6.670')] [2023-10-08 04:02:13,755][00425] Saving new best policy, reward=6.670! [2023-10-08 04:02:13,850][00611] Updated weights for policy 0, policy_version 2370 (0.0009) [2023-10-08 04:02:14,223][00611] Updated weights for policy 0, policy_version 2380 (0.0010) [2023-10-08 04:02:14,597][00611] Updated weights for policy 0, policy_version 2390 (0.0009) [2023-10-08 04:02:14,971][00611] Updated weights for policy 0, policy_version 2400 (0.0011) [2023-10-08 04:02:16,832][00612] Updated weights for policy 1, policy_version 2410 (0.0008) [2023-10-08 04:02:17,189][00612] Updated weights for policy 1, policy_version 2420 (0.0008) [2023-10-08 04:02:17,554][00612] Updated weights for policy 1, policy_version 2430 (0.0008) [2023-10-08 04:02:18,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 4947968. Throughput: 0: 1809.3, 1: 1808.3. Samples: 1248620. Policy #0 lag: (min: 31.0, avg: 42.1, max: 63.0) [2023-10-08 04:02:18,754][130385] Avg episode reward: [(0, '7.280'), (1, '7.210')] [2023-10-08 04:02:18,764][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000002432_2490368.pth... [2023-10-08 04:02:18,801][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000000736_753664.pth [2023-10-08 04:02:18,805][00425] Saving new best policy, reward=7.210! [2023-10-08 04:02:18,842][00611] Updated weights for policy 0, policy_version 2410 (0.0008) [2023-10-08 04:02:19,202][00611] Updated weights for policy 0, policy_version 2420 (0.0009) [2023-10-08 04:02:19,582][00611] Updated weights for policy 0, policy_version 2430 (0.0010) [2023-10-08 04:02:19,649][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000002432_2490368.pth... [2023-10-08 04:02:19,685][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000000736_753664.pth [2023-10-08 04:02:21,432][00612] Updated weights for policy 1, policy_version 2440 (0.0009) [2023-10-08 04:02:21,800][00612] Updated weights for policy 1, policy_version 2450 (0.0010) [2023-10-08 04:02:22,168][00612] Updated weights for policy 1, policy_version 2460 (0.0010) [2023-10-08 04:02:23,409][00611] Updated weights for policy 0, policy_version 2440 (0.0010) [2023-10-08 04:02:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5013504. Throughput: 0: 1797.6, 1: 1812.9. Samples: 1259320. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 04:02:23,754][130385] Avg episode reward: [(0, '7.420'), (1, '6.990')] [2023-10-08 04:02:23,777][00611] Updated weights for policy 0, policy_version 2450 (0.0010) [2023-10-08 04:02:24,151][00611] Updated weights for policy 0, policy_version 2460 (0.0011) [2023-10-08 04:02:24,300][00365] Saving new best policy, reward=7.420! [2023-10-08 04:02:25,959][00612] Updated weights for policy 1, policy_version 2470 (0.0008) [2023-10-08 04:02:26,325][00612] Updated weights for policy 1, policy_version 2480 (0.0007) [2023-10-08 04:02:26,693][00612] Updated weights for policy 1, policy_version 2490 (0.0007) [2023-10-08 04:02:27,988][00611] Updated weights for policy 0, policy_version 2470 (0.0009) [2023-10-08 04:02:28,356][00611] Updated weights for policy 0, policy_version 2480 (0.0008) [2023-10-08 04:02:28,734][00611] Updated weights for policy 0, policy_version 2490 (0.0007) [2023-10-08 04:02:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5079040. Throughput: 0: 1787.9, 1: 1796.4. Samples: 1280166. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-10-08 04:02:28,754][130385] Avg episode reward: [(0, '7.560'), (1, '6.480')] [2023-10-08 04:02:28,955][00365] Saving new best policy, reward=7.560! [2023-10-08 04:02:30,501][00612] Updated weights for policy 1, policy_version 2500 (0.0009) [2023-10-08 04:02:30,870][00612] Updated weights for policy 1, policy_version 2510 (0.0010) [2023-10-08 04:02:31,236][00612] Updated weights for policy 1, policy_version 2520 (0.0009) [2023-10-08 04:02:32,548][00611] Updated weights for policy 0, policy_version 2500 (0.0008) [2023-10-08 04:02:32,921][00611] Updated weights for policy 0, policy_version 2510 (0.0009) [2023-10-08 04:02:33,284][00611] Updated weights for policy 0, policy_version 2520 (0.0008) [2023-10-08 04:02:33,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 5177344. Throughput: 0: 1795.1, 1: 1780.7. Samples: 1301418. Policy #0 lag: (min: 18.0, avg: 21.4, max: 50.0) [2023-10-08 04:02:33,754][130385] Avg episode reward: [(0, '7.370'), (1, '6.640')] [2023-10-08 04:02:34,980][00612] Updated weights for policy 1, policy_version 2530 (0.0009) [2023-10-08 04:02:35,347][00612] Updated weights for policy 1, policy_version 2540 (0.0009) [2023-10-08 04:02:35,718][00612] Updated weights for policy 1, policy_version 2550 (0.0011) [2023-10-08 04:02:36,089][00612] Updated weights for policy 1, policy_version 2560 (0.0010) [2023-10-08 04:02:37,072][00611] Updated weights for policy 0, policy_version 2530 (0.0011) [2023-10-08 04:02:37,433][00611] Updated weights for policy 0, policy_version 2540 (0.0007) [2023-10-08 04:02:37,815][00611] Updated weights for policy 0, policy_version 2550 (0.0008) [2023-10-08 04:02:38,185][00611] Updated weights for policy 0, policy_version 2560 (0.0009) [2023-10-08 04:02:38,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5242880. Throughput: 0: 1775.0, 1: 1775.6. Samples: 1311874. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 04:02:38,754][130385] Avg episode reward: [(0, '7.670'), (1, '6.290')] [2023-10-08 04:02:38,755][00365] Saving new best policy, reward=7.670! [2023-10-08 04:02:39,998][00612] Updated weights for policy 1, policy_version 2570 (0.0010) [2023-10-08 04:02:40,368][00612] Updated weights for policy 1, policy_version 2580 (0.0010) [2023-10-08 04:02:40,730][00612] Updated weights for policy 1, policy_version 2590 (0.0009) [2023-10-08 04:02:42,000][00611] Updated weights for policy 0, policy_version 2570 (0.0011) [2023-10-08 04:02:42,367][00611] Updated weights for policy 0, policy_version 2580 (0.0010) [2023-10-08 04:02:42,736][00611] Updated weights for policy 0, policy_version 2590 (0.0010) [2023-10-08 04:02:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5308416. Throughput: 0: 1790.0, 1: 1766.6. Samples: 1333396. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 04:02:43,754][130385] Avg episode reward: [(0, '8.190'), (1, '6.500')] [2023-10-08 04:02:43,755][00365] Saving new best policy, reward=8.190! [2023-10-08 04:02:44,703][00612] Updated weights for policy 1, policy_version 2600 (0.0009) [2023-10-08 04:02:45,069][00612] Updated weights for policy 1, policy_version 2610 (0.0009) [2023-10-08 04:02:45,424][00612] Updated weights for policy 1, policy_version 2620 (0.0009) [2023-10-08 04:02:46,760][00611] Updated weights for policy 0, policy_version 2600 (0.0008) [2023-10-08 04:02:47,129][00611] Updated weights for policy 0, policy_version 2610 (0.0009) [2023-10-08 04:02:47,506][00611] Updated weights for policy 0, policy_version 2620 (0.0010) [2023-10-08 04:02:48,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5373952. Throughput: 0: 1750.5, 1: 1763.7. Samples: 1353536. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-08 04:02:48,754][130385] Avg episode reward: [(0, '8.310'), (1, '6.740')] [2023-10-08 04:02:48,762][00365] Saving new best policy, reward=8.310! [2023-10-08 04:02:49,570][00612] Updated weights for policy 1, policy_version 2630 (0.0009) [2023-10-08 04:02:49,947][00612] Updated weights for policy 1, policy_version 2640 (0.0008) [2023-10-08 04:02:50,324][00612] Updated weights for policy 1, policy_version 2650 (0.0009) [2023-10-08 04:02:51,453][00611] Updated weights for policy 0, policy_version 2630 (0.0011) [2023-10-08 04:02:51,823][00611] Updated weights for policy 0, policy_version 2640 (0.0009) [2023-10-08 04:02:52,190][00611] Updated weights for policy 0, policy_version 2650 (0.0008) [2023-10-08 04:02:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 5439488. Throughput: 0: 1777.3, 1: 1735.5. Samples: 1364324. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:02:53,754][130385] Avg episode reward: [(0, '8.000'), (1, '6.750')] [2023-10-08 04:02:53,947][00612] Updated weights for policy 1, policy_version 2660 (0.0009) [2023-10-08 04:02:54,326][00612] Updated weights for policy 1, policy_version 2670 (0.0009) [2023-10-08 04:02:54,687][00612] Updated weights for policy 1, policy_version 2680 (0.0011) [2023-10-08 04:02:55,833][00611] Updated weights for policy 0, policy_version 2660 (0.0008) [2023-10-08 04:02:56,208][00611] Updated weights for policy 0, policy_version 2670 (0.0007) [2023-10-08 04:02:56,575][00611] Updated weights for policy 0, policy_version 2680 (0.0007) [2023-10-08 04:02:58,197][00612] Updated weights for policy 1, policy_version 2690 (0.0011) [2023-10-08 04:02:58,572][00612] Updated weights for policy 1, policy_version 2700 (0.0009) [2023-10-08 04:02:58,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5505024. Throughput: 0: 1749.2, 1: 1763.4. Samples: 1385716. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:02:58,755][130385] Avg episode reward: [(0, '8.030'), (1, '6.600')] [2023-10-08 04:02:58,942][00612] Updated weights for policy 1, policy_version 2710 (0.0009) [2023-10-08 04:02:59,311][00612] Updated weights for policy 1, policy_version 2720 (0.0010) [2023-10-08 04:03:00,273][00611] Updated weights for policy 0, policy_version 2690 (0.0010) [2023-10-08 04:03:00,645][00611] Updated weights for policy 0, policy_version 2700 (0.0008) [2023-10-08 04:03:01,019][00611] Updated weights for policy 0, policy_version 2710 (0.0008) [2023-10-08 04:03:01,387][00611] Updated weights for policy 0, policy_version 2720 (0.0008) [2023-10-08 04:03:02,836][00612] Updated weights for policy 1, policy_version 2730 (0.0007) [2023-10-08 04:03:03,198][00612] Updated weights for policy 1, policy_version 2740 (0.0009) [2023-10-08 04:03:03,562][00612] Updated weights for policy 1, policy_version 2750 (0.0010) [2023-10-08 04:03:03,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 5603328. Throughput: 0: 1760.1, 1: 1786.0. Samples: 1408196. Policy #0 lag: (min: 14.0, avg: 24.1, max: 46.0) [2023-10-08 04:03:03,755][130385] Avg episode reward: [(0, '8.920'), (1, '6.520')] [2023-10-08 04:03:03,765][00365] Saving new best policy, reward=8.920! [2023-10-08 04:03:05,058][00611] Updated weights for policy 0, policy_version 2730 (0.0008) [2023-10-08 04:03:05,423][00611] Updated weights for policy 0, policy_version 2740 (0.0010) [2023-10-08 04:03:05,798][00611] Updated weights for policy 0, policy_version 2750 (0.0010) [2023-10-08 04:03:07,286][00612] Updated weights for policy 1, policy_version 2760 (0.0008) [2023-10-08 04:03:07,660][00612] Updated weights for policy 1, policy_version 2770 (0.0007) [2023-10-08 04:03:08,026][00612] Updated weights for policy 1, policy_version 2780 (0.0008) [2023-10-08 04:03:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 5668864. Throughput: 0: 1765.5, 1: 1780.7. Samples: 1418900. Policy #0 lag: (min: 15.0, avg: 20.9, max: 47.0) [2023-10-08 04:03:08,755][130385] Avg episode reward: [(0, '9.360'), (1, '6.370')] [2023-10-08 04:03:08,757][00365] Saving new best policy, reward=9.360! [2023-10-08 04:03:09,450][00611] Updated weights for policy 0, policy_version 2760 (0.0007) [2023-10-08 04:03:09,827][00611] Updated weights for policy 0, policy_version 2770 (0.0007) [2023-10-08 04:03:10,194][00611] Updated weights for policy 0, policy_version 2780 (0.0008) [2023-10-08 04:03:11,714][00612] Updated weights for policy 1, policy_version 2790 (0.0011) [2023-10-08 04:03:12,082][00612] Updated weights for policy 1, policy_version 2800 (0.0010) [2023-10-08 04:03:12,457][00612] Updated weights for policy 1, policy_version 2810 (0.0009) [2023-10-08 04:03:13,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5734400. Throughput: 0: 1775.7, 1: 1795.1. Samples: 1440852. Policy #0 lag: (min: 25.0, avg: 33.9, max: 57.0) [2023-10-08 04:03:13,754][130385] Avg episode reward: [(0, '9.090'), (1, '7.140')] [2023-10-08 04:03:13,941][00611] Updated weights for policy 0, policy_version 2790 (0.0010) [2023-10-08 04:03:14,306][00611] Updated weights for policy 0, policy_version 2800 (0.0007) [2023-10-08 04:03:14,680][00611] Updated weights for policy 0, policy_version 2810 (0.0008) [2023-10-08 04:03:16,336][00612] Updated weights for policy 1, policy_version 2820 (0.0009) [2023-10-08 04:03:16,712][00612] Updated weights for policy 1, policy_version 2830 (0.0009) [2023-10-08 04:03:17,071][00612] Updated weights for policy 1, policy_version 2840 (0.0007) [2023-10-08 04:03:18,568][00611] Updated weights for policy 0, policy_version 2820 (0.0010) [2023-10-08 04:03:18,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 5799936. Throughput: 0: 1791.9, 1: 1774.6. Samples: 1461912. Policy #0 lag: (min: 25.0, avg: 33.9, max: 57.0) [2023-10-08 04:03:18,754][130385] Avg episode reward: [(0, '8.610'), (1, '7.540')] [2023-10-08 04:03:18,763][00425] Saving new best policy, reward=7.540! [2023-10-08 04:03:18,943][00611] Updated weights for policy 0, policy_version 2830 (0.0008) [2023-10-08 04:03:19,310][00611] Updated weights for policy 0, policy_version 2840 (0.0009) [2023-10-08 04:03:20,864][00612] Updated weights for policy 1, policy_version 2850 (0.0008) [2023-10-08 04:03:21,231][00612] Updated weights for policy 1, policy_version 2860 (0.0008) [2023-10-08 04:03:21,601][00612] Updated weights for policy 1, policy_version 2870 (0.0008) [2023-10-08 04:03:21,966][00612] Updated weights for policy 1, policy_version 2880 (0.0008) [2023-10-08 04:03:23,040][00611] Updated weights for policy 0, policy_version 2850 (0.0008) [2023-10-08 04:03:23,394][00611] Updated weights for policy 0, policy_version 2860 (0.0008) [2023-10-08 04:03:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 5865472. Throughput: 0: 1771.7, 1: 1800.6. Samples: 1472628. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-08 04:03:23,755][130385] Avg episode reward: [(0, '8.740'), (1, '7.460')] [2023-10-08 04:03:23,764][00611] Updated weights for policy 0, policy_version 2870 (0.0009) [2023-10-08 04:03:24,133][00611] Updated weights for policy 0, policy_version 2880 (0.0010) [2023-10-08 04:03:25,893][00612] Updated weights for policy 1, policy_version 2890 (0.0007) [2023-10-08 04:03:26,259][00612] Updated weights for policy 1, policy_version 2900 (0.0010) [2023-10-08 04:03:26,619][00612] Updated weights for policy 1, policy_version 2910 (0.0009) [2023-10-08 04:03:27,990][00611] Updated weights for policy 0, policy_version 2890 (0.0009) [2023-10-08 04:03:28,354][00611] Updated weights for policy 0, policy_version 2900 (0.0009) [2023-10-08 04:03:28,739][00611] Updated weights for policy 0, policy_version 2910 (0.0009) [2023-10-08 04:03:28,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 5931008. Throughput: 0: 1786.7, 1: 1779.7. Samples: 1493882. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-08 04:03:28,754][130385] Avg episode reward: [(0, '8.930'), (1, '7.530')] [2023-10-08 04:03:30,444][00612] Updated weights for policy 1, policy_version 2920 (0.0008) [2023-10-08 04:03:30,814][00612] Updated weights for policy 1, policy_version 2930 (0.0009) [2023-10-08 04:03:31,177][00612] Updated weights for policy 1, policy_version 2940 (0.0010) [2023-10-08 04:03:32,673][00611] Updated weights for policy 0, policy_version 2920 (0.0009) [2023-10-08 04:03:33,042][00611] Updated weights for policy 0, policy_version 2930 (0.0008) [2023-10-08 04:03:33,409][00611] Updated weights for policy 0, policy_version 2940 (0.0008) [2023-10-08 04:03:33,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 6029312. Throughput: 0: 1792.9, 1: 1794.3. Samples: 1514960. Policy #0 lag: (min: 3.0, avg: 13.3, max: 35.0) [2023-10-08 04:03:33,754][130385] Avg episode reward: [(0, '8.710'), (1, '7.110')] [2023-10-08 04:03:34,977][00612] Updated weights for policy 1, policy_version 2950 (0.0008) [2023-10-08 04:03:35,356][00612] Updated weights for policy 1, policy_version 2960 (0.0008) [2023-10-08 04:03:35,719][00612] Updated weights for policy 1, policy_version 2970 (0.0009) [2023-10-08 04:03:37,203][00611] Updated weights for policy 0, policy_version 2950 (0.0010) [2023-10-08 04:03:37,572][00611] Updated weights for policy 0, policy_version 2960 (0.0008) [2023-10-08 04:03:37,939][00611] Updated weights for policy 0, policy_version 2970 (0.0010) [2023-10-08 04:03:38,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 6094848. Throughput: 0: 1780.3, 1: 1797.3. Samples: 1525314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:03:38,755][130385] Avg episode reward: [(0, '9.030'), (1, '6.530')] [2023-10-08 04:03:39,505][00612] Updated weights for policy 1, policy_version 2980 (0.0009) [2023-10-08 04:03:39,876][00612] Updated weights for policy 1, policy_version 2990 (0.0007) [2023-10-08 04:03:40,251][00612] Updated weights for policy 1, policy_version 3000 (0.0008) [2023-10-08 04:03:41,634][00611] Updated weights for policy 0, policy_version 2980 (0.0010) [2023-10-08 04:03:42,008][00611] Updated weights for policy 0, policy_version 2990 (0.0010) [2023-10-08 04:03:42,383][00611] Updated weights for policy 0, policy_version 3000 (0.0008) [2023-10-08 04:03:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 6160384. Throughput: 0: 1801.0, 1: 1785.4. Samples: 1547104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:03:43,754][130385] Avg episode reward: [(0, '9.370'), (1, '6.970')] [2023-10-08 04:03:43,755][00365] Saving new best policy, reward=9.370! [2023-10-08 04:03:43,853][00612] Updated weights for policy 1, policy_version 3010 (0.0008) [2023-10-08 04:03:44,214][00612] Updated weights for policy 1, policy_version 3020 (0.0008) [2023-10-08 04:03:44,585][00612] Updated weights for policy 1, policy_version 3030 (0.0009) [2023-10-08 04:03:44,961][00612] Updated weights for policy 1, policy_version 3040 (0.0010) [2023-10-08 04:03:46,030][00611] Updated weights for policy 0, policy_version 3010 (0.0007) [2023-10-08 04:03:46,397][00611] Updated weights for policy 0, policy_version 3020 (0.0008) [2023-10-08 04:03:46,762][00611] Updated weights for policy 0, policy_version 3030 (0.0008) [2023-10-08 04:03:47,137][00611] Updated weights for policy 0, policy_version 3040 (0.0007) [2023-10-08 04:03:48,723][00612] Updated weights for policy 1, policy_version 3050 (0.0008) [2023-10-08 04:03:48,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 6225920. Throughput: 0: 1779.4, 1: 1794.0. Samples: 1568996. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 04:03:48,754][130385] Avg episode reward: [(0, '9.420'), (1, '7.570')] [2023-10-08 04:03:48,762][00365] Saving new best policy, reward=9.420! [2023-10-08 04:03:49,084][00612] Updated weights for policy 1, policy_version 3060 (0.0008) [2023-10-08 04:03:49,445][00612] Updated weights for policy 1, policy_version 3070 (0.0007) [2023-10-08 04:03:49,517][00425] Saving new best policy, reward=7.570! [2023-10-08 04:03:50,724][00611] Updated weights for policy 0, policy_version 3050 (0.0008) [2023-10-08 04:03:51,092][00611] Updated weights for policy 0, policy_version 3060 (0.0009) [2023-10-08 04:03:51,461][00611] Updated weights for policy 0, policy_version 3070 (0.0008) [2023-10-08 04:03:52,988][00612] Updated weights for policy 1, policy_version 3080 (0.0010) [2023-10-08 04:03:53,353][00612] Updated weights for policy 1, policy_version 3090 (0.0009) [2023-10-08 04:03:53,725][00612] Updated weights for policy 1, policy_version 3100 (0.0007) [2023-10-08 04:03:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 6291456. Throughput: 0: 1796.9, 1: 1777.3. Samples: 1579740. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 04:03:53,754][130385] Avg episode reward: [(0, '9.080'), (1, '7.760')] [2023-10-08 04:03:53,865][00425] Saving new best policy, reward=7.760! [2023-10-08 04:03:55,146][00611] Updated weights for policy 0, policy_version 3080 (0.0009) [2023-10-08 04:03:55,518][00611] Updated weights for policy 0, policy_version 3090 (0.0011) [2023-10-08 04:03:55,890][00611] Updated weights for policy 0, policy_version 3100 (0.0009) [2023-10-08 04:03:57,456][00612] Updated weights for policy 1, policy_version 3110 (0.0008) [2023-10-08 04:03:57,823][00612] Updated weights for policy 1, policy_version 3120 (0.0008) [2023-10-08 04:03:58,189][00612] Updated weights for policy 1, policy_version 3130 (0.0007) [2023-10-08 04:03:58,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 6389760. Throughput: 0: 1789.5, 1: 1804.0. Samples: 1602556. Policy #0 lag: (min: 3.0, avg: 5.8, max: 35.0) [2023-10-08 04:03:58,754][130385] Avg episode reward: [(0, '9.300'), (1, '7.520')] [2023-10-08 04:03:59,433][00611] Updated weights for policy 0, policy_version 3110 (0.0008) [2023-10-08 04:03:59,793][00611] Updated weights for policy 0, policy_version 3120 (0.0007) [2023-10-08 04:04:00,161][00611] Updated weights for policy 0, policy_version 3130 (0.0008) [2023-10-08 04:04:01,750][00612] Updated weights for policy 1, policy_version 3140 (0.0009) [2023-10-08 04:04:02,114][00612] Updated weights for policy 1, policy_version 3150 (0.0008) [2023-10-08 04:04:02,488][00612] Updated weights for policy 1, policy_version 3160 (0.0008) [2023-10-08 04:04:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 6455296. Throughput: 0: 1806.4, 1: 1802.0. Samples: 1624288. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-08 04:04:03,754][130385] Avg episode reward: [(0, '9.320'), (1, '6.890')] [2023-10-08 04:04:03,846][00611] Updated weights for policy 0, policy_version 3140 (0.0007) [2023-10-08 04:04:04,215][00611] Updated weights for policy 0, policy_version 3150 (0.0010) [2023-10-08 04:04:04,593][00611] Updated weights for policy 0, policy_version 3160 (0.0008) [2023-10-08 04:04:06,239][00612] Updated weights for policy 1, policy_version 3170 (0.0007) [2023-10-08 04:04:06,610][00612] Updated weights for policy 1, policy_version 3180 (0.0007) [2023-10-08 04:04:06,977][00612] Updated weights for policy 1, policy_version 3190 (0.0007) [2023-10-08 04:04:07,361][00612] Updated weights for policy 1, policy_version 3200 (0.0007) [2023-10-08 04:04:08,152][00611] Updated weights for policy 0, policy_version 3170 (0.0009) [2023-10-08 04:04:08,531][00611] Updated weights for policy 0, policy_version 3180 (0.0008) [2023-10-08 04:04:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 6520832. Throughput: 0: 1810.4, 1: 1815.6. Samples: 1635796. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-08 04:04:08,754][130385] Avg episode reward: [(0, '9.540'), (1, '6.890')] [2023-10-08 04:04:08,919][00611] Updated weights for policy 0, policy_version 3190 (0.0009) [2023-10-08 04:04:09,282][00365] Saving new best policy, reward=9.540! [2023-10-08 04:04:09,284][00611] Updated weights for policy 0, policy_version 3200 (0.0010) [2023-10-08 04:04:10,948][00612] Updated weights for policy 1, policy_version 3210 (0.0011) [2023-10-08 04:04:11,325][00612] Updated weights for policy 1, policy_version 3220 (0.0007) [2023-10-08 04:04:11,691][00612] Updated weights for policy 1, policy_version 3230 (0.0008) [2023-10-08 04:04:13,035][00611] Updated weights for policy 0, policy_version 3210 (0.0009) [2023-10-08 04:04:13,406][00611] Updated weights for policy 0, policy_version 3220 (0.0007) [2023-10-08 04:04:13,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 6586368. Throughput: 0: 1819.6, 1: 1815.3. Samples: 1657450. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 04:04:13,755][130385] Avg episode reward: [(0, '9.360'), (1, '7.450')] [2023-10-08 04:04:13,784][00611] Updated weights for policy 0, policy_version 3230 (0.0008) [2023-10-08 04:04:15,272][00612] Updated weights for policy 1, policy_version 3240 (0.0008) [2023-10-08 04:04:15,649][00612] Updated weights for policy 1, policy_version 3250 (0.0009) [2023-10-08 04:04:16,009][00612] Updated weights for policy 1, policy_version 3260 (0.0010) [2023-10-08 04:04:17,523][00611] Updated weights for policy 0, policy_version 3240 (0.0008) [2023-10-08 04:04:17,893][00611] Updated weights for policy 0, policy_version 3250 (0.0008) [2023-10-08 04:04:18,270][00611] Updated weights for policy 0, policy_version 3260 (0.0009) [2023-10-08 04:04:18,754][130385] Fps is (10 sec: 16383.0, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 6684672. Throughput: 0: 1822.4, 1: 1826.4. Samples: 1679156. Policy #0 lag: (min: 19.0, avg: 25.0, max: 51.0) [2023-10-08 04:04:18,756][130385] Avg episode reward: [(0, '9.420'), (1, '7.720')] [2023-10-08 04:04:18,764][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000003264_3342336.pth... [2023-10-08 04:04:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000003264_3342336.pth... [2023-10-08 04:04:18,795][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000001600_1638400.pth [2023-10-08 04:04:18,803][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000001568_1605632.pth [2023-10-08 04:04:19,862][00612] Updated weights for policy 1, policy_version 3270 (0.0010) [2023-10-08 04:04:20,242][00612] Updated weights for policy 1, policy_version 3280 (0.0009) [2023-10-08 04:04:20,615][00612] Updated weights for policy 1, policy_version 3290 (0.0009) [2023-10-08 04:04:22,040][00611] Updated weights for policy 0, policy_version 3270 (0.0010) [2023-10-08 04:04:22,408][00611] Updated weights for policy 0, policy_version 3280 (0.0008) [2023-10-08 04:04:22,781][00611] Updated weights for policy 0, policy_version 3290 (0.0009) [2023-10-08 04:04:23,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 6750208. Throughput: 0: 1831.1, 1: 1827.3. Samples: 1689942. Policy #0 lag: (min: 19.0, avg: 25.0, max: 51.0) [2023-10-08 04:04:23,754][130385] Avg episode reward: [(0, '9.740'), (1, '7.500')] [2023-10-08 04:04:23,755][00365] Saving new best policy, reward=9.740! [2023-10-08 04:04:24,451][00612] Updated weights for policy 1, policy_version 3300 (0.0009) [2023-10-08 04:04:24,820][00612] Updated weights for policy 1, policy_version 3310 (0.0009) [2023-10-08 04:04:25,179][00612] Updated weights for policy 1, policy_version 3320 (0.0008) [2023-10-08 04:04:26,546][00611] Updated weights for policy 0, policy_version 3300 (0.0007) [2023-10-08 04:04:26,914][00611] Updated weights for policy 0, policy_version 3310 (0.0008) [2023-10-08 04:04:27,290][00611] Updated weights for policy 0, policy_version 3320 (0.0007) [2023-10-08 04:04:28,715][00612] Updated weights for policy 1, policy_version 3330 (0.0009) [2023-10-08 04:04:28,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 6815744. Throughput: 0: 1823.1, 1: 1834.5. Samples: 1711700. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 04:04:28,755][130385] Avg episode reward: [(0, '9.850'), (1, '8.020')] [2023-10-08 04:04:28,756][00365] Saving new best policy, reward=9.850! [2023-10-08 04:04:29,079][00612] Updated weights for policy 1, policy_version 3340 (0.0009) [2023-10-08 04:04:29,443][00612] Updated weights for policy 1, policy_version 3350 (0.0008) [2023-10-08 04:04:29,815][00425] Saving new best policy, reward=8.020! [2023-10-08 04:04:29,817][00612] Updated weights for policy 1, policy_version 3360 (0.0009) [2023-10-08 04:04:30,979][00611] Updated weights for policy 0, policy_version 3330 (0.0008) [2023-10-08 04:04:31,356][00611] Updated weights for policy 0, policy_version 3340 (0.0007) [2023-10-08 04:04:31,729][00611] Updated weights for policy 0, policy_version 3350 (0.0008) [2023-10-08 04:04:32,103][00611] Updated weights for policy 0, policy_version 3360 (0.0008) [2023-10-08 04:04:33,364][00612] Updated weights for policy 1, policy_version 3370 (0.0008) [2023-10-08 04:04:33,735][00612] Updated weights for policy 1, policy_version 3380 (0.0011) [2023-10-08 04:04:33,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 6881280. Throughput: 0: 1824.7, 1: 1838.7. Samples: 1733850. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 04:04:33,755][130385] Avg episode reward: [(0, '10.030'), (1, '7.990')] [2023-10-08 04:04:33,764][00365] Saving new best policy, reward=10.030! [2023-10-08 04:04:34,102][00612] Updated weights for policy 1, policy_version 3390 (0.0009) [2023-10-08 04:04:35,705][00611] Updated weights for policy 0, policy_version 3370 (0.0010) [2023-10-08 04:04:36,084][00611] Updated weights for policy 0, policy_version 3380 (0.0011) [2023-10-08 04:04:36,441][00611] Updated weights for policy 0, policy_version 3390 (0.0010) [2023-10-08 04:04:37,913][00612] Updated weights for policy 1, policy_version 3400 (0.0008) [2023-10-08 04:04:38,274][00612] Updated weights for policy 1, policy_version 3410 (0.0008) [2023-10-08 04:04:38,644][00612] Updated weights for policy 1, policy_version 3420 (0.0009) [2023-10-08 04:04:38,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 6946816. Throughput: 0: 1821.0, 1: 1839.5. Samples: 1744462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:04:38,754][130385] Avg episode reward: [(0, '10.260'), (1, '7.590')] [2023-10-08 04:04:38,755][00365] Saving new best policy, reward=10.260! [2023-10-08 04:04:40,196][00611] Updated weights for policy 0, policy_version 3400 (0.0008) [2023-10-08 04:04:40,568][00611] Updated weights for policy 0, policy_version 3410 (0.0008) [2023-10-08 04:04:40,935][00611] Updated weights for policy 0, policy_version 3420 (0.0008) [2023-10-08 04:04:42,358][00612] Updated weights for policy 1, policy_version 3430 (0.0009) [2023-10-08 04:04:42,731][00612] Updated weights for policy 1, policy_version 3440 (0.0007) [2023-10-08 04:04:43,086][00612] Updated weights for policy 1, policy_version 3450 (0.0007) [2023-10-08 04:04:43,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 7045120. Throughput: 0: 1814.5, 1: 1835.3. Samples: 1766798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:04:43,755][130385] Avg episode reward: [(0, '9.970'), (1, '7.900')] [2023-10-08 04:04:44,622][00611] Updated weights for policy 0, policy_version 3430 (0.0007) [2023-10-08 04:04:44,990][00611] Updated weights for policy 0, policy_version 3440 (0.0007) [2023-10-08 04:04:45,362][00611] Updated weights for policy 0, policy_version 3450 (0.0007) [2023-10-08 04:04:46,839][00612] Updated weights for policy 1, policy_version 3460 (0.0010) [2023-10-08 04:04:47,215][00612] Updated weights for policy 1, policy_version 3470 (0.0009) [2023-10-08 04:04:47,580][00612] Updated weights for policy 1, policy_version 3480 (0.0009) [2023-10-08 04:04:48,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 7110656. Throughput: 0: 1813.4, 1: 1827.8. Samples: 1788142. Policy #0 lag: (min: 31.0, avg: 48.4, max: 63.0) [2023-10-08 04:04:48,754][130385] Avg episode reward: [(0, '9.600'), (1, '8.190')] [2023-10-08 04:04:48,763][00425] Saving new best policy, reward=8.190! [2023-10-08 04:04:49,027][00611] Updated weights for policy 0, policy_version 3460 (0.0007) [2023-10-08 04:04:49,400][00611] Updated weights for policy 0, policy_version 3470 (0.0010) [2023-10-08 04:04:49,777][00611] Updated weights for policy 0, policy_version 3480 (0.0010) [2023-10-08 04:04:51,308][00612] Updated weights for policy 1, policy_version 3490 (0.0007) [2023-10-08 04:04:51,666][00612] Updated weights for policy 1, policy_version 3500 (0.0007) [2023-10-08 04:04:52,035][00612] Updated weights for policy 1, policy_version 3510 (0.0008) [2023-10-08 04:04:52,400][00612] Updated weights for policy 1, policy_version 3520 (0.0009) [2023-10-08 04:04:53,344][00611] Updated weights for policy 0, policy_version 3490 (0.0010) [2023-10-08 04:04:53,702][00611] Updated weights for policy 0, policy_version 3500 (0.0010) [2023-10-08 04:04:53,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 7176192. Throughput: 0: 1811.2, 1: 1825.8. Samples: 1799462. Policy #0 lag: (min: 31.0, avg: 48.4, max: 63.0) [2023-10-08 04:04:53,754][130385] Avg episode reward: [(0, '10.800'), (1, '8.200')] [2023-10-08 04:04:53,755][00425] Saving new best policy, reward=8.200! [2023-10-08 04:04:54,065][00611] Updated weights for policy 0, policy_version 3510 (0.0007) [2023-10-08 04:04:54,433][00611] Updated weights for policy 0, policy_version 3520 (0.0007) [2023-10-08 04:04:54,433][00365] Saving new best policy, reward=10.800! [2023-10-08 04:04:56,014][00612] Updated weights for policy 1, policy_version 3530 (0.0009) [2023-10-08 04:04:56,391][00612] Updated weights for policy 1, policy_version 3540 (0.0008) [2023-10-08 04:04:56,755][00612] Updated weights for policy 1, policy_version 3550 (0.0009) [2023-10-08 04:04:58,088][00611] Updated weights for policy 0, policy_version 3530 (0.0008) [2023-10-08 04:04:58,465][00611] Updated weights for policy 0, policy_version 3540 (0.0010) [2023-10-08 04:04:58,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 7241728. Throughput: 0: 1811.5, 1: 1823.8. Samples: 1821040. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-08 04:04:58,755][130385] Avg episode reward: [(0, '10.220'), (1, '8.320')] [2023-10-08 04:04:58,756][00425] Saving new best policy, reward=8.320! [2023-10-08 04:04:58,828][00611] Updated weights for policy 0, policy_version 3550 (0.0009) [2023-10-08 04:05:00,384][00612] Updated weights for policy 1, policy_version 3560 (0.0008) [2023-10-08 04:05:00,758][00612] Updated weights for policy 1, policy_version 3570 (0.0009) [2023-10-08 04:05:01,126][00612] Updated weights for policy 1, policy_version 3580 (0.0010) [2023-10-08 04:05:02,739][00611] Updated weights for policy 0, policy_version 3560 (0.0009) [2023-10-08 04:05:03,114][00611] Updated weights for policy 0, policy_version 3570 (0.0010) [2023-10-08 04:05:03,498][00611] Updated weights for policy 0, policy_version 3580 (0.0008) [2023-10-08 04:05:03,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 7340032. Throughput: 0: 1818.2, 1: 1826.6. Samples: 1843172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:05:03,755][130385] Avg episode reward: [(0, '10.520'), (1, '8.270')] [2023-10-08 04:05:04,692][00612] Updated weights for policy 1, policy_version 3590 (0.0010) [2023-10-08 04:05:05,061][00612] Updated weights for policy 1, policy_version 3600 (0.0008) [2023-10-08 04:05:05,426][00612] Updated weights for policy 1, policy_version 3610 (0.0009) [2023-10-08 04:05:07,081][00611] Updated weights for policy 0, policy_version 3590 (0.0008) [2023-10-08 04:05:07,453][00611] Updated weights for policy 0, policy_version 3600 (0.0009) [2023-10-08 04:05:07,836][00611] Updated weights for policy 0, policy_version 3610 (0.0008) [2023-10-08 04:05:08,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 7405568. Throughput: 0: 1814.9, 1: 1833.5. Samples: 1854118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:05:08,754][130385] Avg episode reward: [(0, '10.690'), (1, '9.240')] [2023-10-08 04:05:08,981][00612] Updated weights for policy 1, policy_version 3620 (0.0011) [2023-10-08 04:05:09,338][00612] Updated weights for policy 1, policy_version 3630 (0.0007) [2023-10-08 04:05:09,709][00612] Updated weights for policy 1, policy_version 3640 (0.0007) [2023-10-08 04:05:10,005][00425] Saving new best policy, reward=9.240! [2023-10-08 04:05:11,365][00611] Updated weights for policy 0, policy_version 3620 (0.0009) [2023-10-08 04:05:11,726][00611] Updated weights for policy 0, policy_version 3630 (0.0009) [2023-10-08 04:05:12,092][00611] Updated weights for policy 0, policy_version 3640 (0.0007) [2023-10-08 04:05:13,242][00612] Updated weights for policy 1, policy_version 3650 (0.0009) [2023-10-08 04:05:13,619][00612] Updated weights for policy 1, policy_version 3660 (0.0010) [2023-10-08 04:05:13,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 7471104. Throughput: 0: 1821.5, 1: 1840.7. Samples: 1876498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:05:13,754][130385] Avg episode reward: [(0, '10.980'), (1, '9.000')] [2023-10-08 04:05:13,755][00365] Saving new best policy, reward=10.980! [2023-10-08 04:05:13,988][00612] Updated weights for policy 1, policy_version 3670 (0.0009) [2023-10-08 04:05:14,363][00612] Updated weights for policy 1, policy_version 3680 (0.0009) [2023-10-08 04:05:15,967][00611] Updated weights for policy 0, policy_version 3650 (0.0008) [2023-10-08 04:05:16,335][00611] Updated weights for policy 0, policy_version 3660 (0.0008) [2023-10-08 04:05:16,712][00611] Updated weights for policy 0, policy_version 3670 (0.0007) [2023-10-08 04:05:17,092][00611] Updated weights for policy 0, policy_version 3680 (0.0008) [2023-10-08 04:05:18,195][00612] Updated weights for policy 1, policy_version 3690 (0.0009) [2023-10-08 04:05:18,563][00612] Updated weights for policy 1, policy_version 3700 (0.0007) [2023-10-08 04:05:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 7536640. Throughput: 0: 1823.7, 1: 1830.1. Samples: 1898272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:05:18,754][130385] Avg episode reward: [(0, '11.410'), (1, '8.230')] [2023-10-08 04:05:18,762][00365] Saving new best policy, reward=11.410! [2023-10-08 04:05:18,937][00612] Updated weights for policy 1, policy_version 3710 (0.0010) [2023-10-08 04:05:20,783][00611] Updated weights for policy 0, policy_version 3690 (0.0010) [2023-10-08 04:05:21,166][00611] Updated weights for policy 0, policy_version 3700 (0.0011) [2023-10-08 04:05:21,534][00611] Updated weights for policy 0, policy_version 3710 (0.0009) [2023-10-08 04:05:22,618][00612] Updated weights for policy 1, policy_version 3720 (0.0008) [2023-10-08 04:05:22,997][00612] Updated weights for policy 1, policy_version 3730 (0.0009) [2023-10-08 04:05:23,365][00612] Updated weights for policy 1, policy_version 3740 (0.0007) [2023-10-08 04:05:23,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 7634944. Throughput: 0: 1825.0, 1: 1837.3. Samples: 1909264. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 04:05:23,755][130385] Avg episode reward: [(0, '11.780'), (1, '8.640')] [2023-10-08 04:05:23,760][00365] Saving new best policy, reward=11.780! [2023-10-08 04:05:25,134][00611] Updated weights for policy 0, policy_version 3720 (0.0008) [2023-10-08 04:05:25,503][00611] Updated weights for policy 0, policy_version 3730 (0.0008) [2023-10-08 04:05:25,876][00611] Updated weights for policy 0, policy_version 3740 (0.0007) [2023-10-08 04:05:26,988][00612] Updated weights for policy 1, policy_version 3750 (0.0007) [2023-10-08 04:05:27,359][00612] Updated weights for policy 1, policy_version 3760 (0.0009) [2023-10-08 04:05:27,730][00612] Updated weights for policy 1, policy_version 3770 (0.0010) [2023-10-08 04:05:28,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 7700480. Throughput: 0: 1825.1, 1: 1833.6. Samples: 1931436. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 04:05:28,755][130385] Avg episode reward: [(0, '11.460'), (1, '8.550')] [2023-10-08 04:05:29,508][00611] Updated weights for policy 0, policy_version 3750 (0.0009) [2023-10-08 04:05:29,870][00611] Updated weights for policy 0, policy_version 3760 (0.0010) [2023-10-08 04:05:30,250][00611] Updated weights for policy 0, policy_version 3770 (0.0008) [2023-10-08 04:05:31,296][00612] Updated weights for policy 1, policy_version 3780 (0.0008) [2023-10-08 04:05:31,669][00612] Updated weights for policy 1, policy_version 3790 (0.0010) [2023-10-08 04:05:32,046][00612] Updated weights for policy 1, policy_version 3800 (0.0012) [2023-10-08 04:05:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 7766016. Throughput: 0: 1827.7, 1: 1849.4. Samples: 1953612. Policy #0 lag: (min: 16.0, avg: 36.7, max: 48.0) [2023-10-08 04:05:33,755][130385] Avg episode reward: [(0, '11.070'), (1, '8.790')] [2023-10-08 04:05:33,902][00611] Updated weights for policy 0, policy_version 3780 (0.0007) [2023-10-08 04:05:34,260][00611] Updated weights for policy 0, policy_version 3790 (0.0007) [2023-10-08 04:05:34,624][00611] Updated weights for policy 0, policy_version 3800 (0.0007) [2023-10-08 04:05:35,652][00612] Updated weights for policy 1, policy_version 3810 (0.0009) [2023-10-08 04:05:36,015][00612] Updated weights for policy 1, policy_version 3820 (0.0007) [2023-10-08 04:05:36,386][00612] Updated weights for policy 1, policy_version 3830 (0.0008) [2023-10-08 04:05:36,763][00612] Updated weights for policy 1, policy_version 3840 (0.0008) [2023-10-08 04:05:38,214][00611] Updated weights for policy 0, policy_version 3810 (0.0008) [2023-10-08 04:05:38,577][00611] Updated weights for policy 0, policy_version 3820 (0.0009) [2023-10-08 04:05:38,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 7831552. Throughput: 0: 1830.7, 1: 1836.6. Samples: 1964488. Policy #0 lag: (min: 16.0, avg: 36.7, max: 48.0) [2023-10-08 04:05:38,754][130385] Avg episode reward: [(0, '11.280'), (1, '9.270')] [2023-10-08 04:05:38,755][00425] Saving new best policy, reward=9.270! [2023-10-08 04:05:38,947][00611] Updated weights for policy 0, policy_version 3830 (0.0010) [2023-10-08 04:05:39,314][00611] Updated weights for policy 0, policy_version 3840 (0.0009) [2023-10-08 04:05:40,433][00612] Updated weights for policy 1, policy_version 3850 (0.0011) [2023-10-08 04:05:40,797][00612] Updated weights for policy 1, policy_version 3860 (0.0011) [2023-10-08 04:05:41,171][00612] Updated weights for policy 1, policy_version 3870 (0.0010) [2023-10-08 04:05:43,139][00611] Updated weights for policy 0, policy_version 3850 (0.0008) [2023-10-08 04:05:43,519][00611] Updated weights for policy 0, policy_version 3860 (0.0008) [2023-10-08 04:05:43,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 7897088. Throughput: 0: 1827.3, 1: 1849.1. Samples: 1986476. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 04:05:43,754][130385] Avg episode reward: [(0, '11.220'), (1, '9.350')] [2023-10-08 04:05:43,755][00425] Saving new best policy, reward=9.350! [2023-10-08 04:05:43,880][00611] Updated weights for policy 0, policy_version 3870 (0.0009) [2023-10-08 04:05:44,803][00612] Updated weights for policy 1, policy_version 3880 (0.0008) [2023-10-08 04:05:45,179][00612] Updated weights for policy 1, policy_version 3890 (0.0010) [2023-10-08 04:05:45,540][00612] Updated weights for policy 1, policy_version 3900 (0.0009) [2023-10-08 04:05:47,557][00611] Updated weights for policy 0, policy_version 3880 (0.0009) [2023-10-08 04:05:47,934][00611] Updated weights for policy 0, policy_version 3890 (0.0008) [2023-10-08 04:05:48,307][00611] Updated weights for policy 0, policy_version 3900 (0.0007) [2023-10-08 04:05:48,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 7995392. Throughput: 0: 1827.4, 1: 1847.1. Samples: 2008526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-08 04:05:48,754][130385] Avg episode reward: [(0, '11.020'), (1, '9.300')] [2023-10-08 04:05:49,334][00612] Updated weights for policy 1, policy_version 3910 (0.0011) [2023-10-08 04:05:49,705][00612] Updated weights for policy 1, policy_version 3920 (0.0011) [2023-10-08 04:05:50,077][00612] Updated weights for policy 1, policy_version 3930 (0.0011) [2023-10-08 04:05:51,863][00611] Updated weights for policy 0, policy_version 3910 (0.0008) [2023-10-08 04:05:52,238][00611] Updated weights for policy 0, policy_version 3920 (0.0007) [2023-10-08 04:05:52,611][00611] Updated weights for policy 0, policy_version 3930 (0.0007) [2023-10-08 04:05:53,628][00612] Updated weights for policy 1, policy_version 3940 (0.0008) [2023-10-08 04:05:53,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 8060928. Throughput: 0: 1832.1, 1: 1841.1. Samples: 2019414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-08 04:05:53,755][130385] Avg episode reward: [(0, '12.210'), (1, '9.280')] [2023-10-08 04:05:53,755][00365] Saving new best policy, reward=12.210! [2023-10-08 04:05:54,012][00612] Updated weights for policy 1, policy_version 3950 (0.0008) [2023-10-08 04:05:54,374][00612] Updated weights for policy 1, policy_version 3960 (0.0011) [2023-10-08 04:05:56,327][00611] Updated weights for policy 0, policy_version 3940 (0.0008) [2023-10-08 04:05:56,700][00611] Updated weights for policy 0, policy_version 3950 (0.0008) [2023-10-08 04:05:57,074][00611] Updated weights for policy 0, policy_version 3960 (0.0010) [2023-10-08 04:05:57,820][00612] Updated weights for policy 1, policy_version 3970 (0.0010) [2023-10-08 04:05:58,193][00612] Updated weights for policy 1, policy_version 3980 (0.0009) [2023-10-08 04:05:58,560][00612] Updated weights for policy 1, policy_version 3990 (0.0009) [2023-10-08 04:05:58,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 8126464. Throughput: 0: 1826.8, 1: 1840.9. Samples: 2041546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:05:58,755][130385] Avg episode reward: [(0, '11.460'), (1, '9.510')] [2023-10-08 04:05:58,925][00425] Saving new best policy, reward=9.510! [2023-10-08 04:05:58,931][00612] Updated weights for policy 1, policy_version 4000 (0.0008) [2023-10-08 04:06:00,652][00611] Updated weights for policy 0, policy_version 3970 (0.0008) [2023-10-08 04:06:01,015][00611] Updated weights for policy 0, policy_version 3980 (0.0008) [2023-10-08 04:06:01,394][00611] Updated weights for policy 0, policy_version 3990 (0.0007) [2023-10-08 04:06:01,762][00611] Updated weights for policy 0, policy_version 4000 (0.0009) [2023-10-08 04:06:02,610][00612] Updated weights for policy 1, policy_version 4010 (0.0008) [2023-10-08 04:06:02,975][00612] Updated weights for policy 1, policy_version 4020 (0.0007) [2023-10-08 04:06:03,354][00612] Updated weights for policy 1, policy_version 4030 (0.0008) [2023-10-08 04:06:03,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 8224768. Throughput: 0: 1832.8, 1: 1826.7. Samples: 2062950. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:06:03,754][130385] Avg episode reward: [(0, '12.670'), (1, '10.390')] [2023-10-08 04:06:03,762][00365] Saving new best policy, reward=12.670! [2023-10-08 04:06:03,762][00425] Saving new best policy, reward=10.390! [2023-10-08 04:06:05,414][00611] Updated weights for policy 0, policy_version 4010 (0.0008) [2023-10-08 04:06:05,781][00611] Updated weights for policy 0, policy_version 4020 (0.0008) [2023-10-08 04:06:06,158][00611] Updated weights for policy 0, policy_version 4030 (0.0007) [2023-10-08 04:06:07,032][00612] Updated weights for policy 1, policy_version 4040 (0.0009) [2023-10-08 04:06:07,396][00612] Updated weights for policy 1, policy_version 4050 (0.0008) [2023-10-08 04:06:07,774][00612] Updated weights for policy 1, policy_version 4060 (0.0009) [2023-10-08 04:06:08,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 8290304. Throughput: 0: 1825.6, 1: 1839.7. Samples: 2074204. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-08 04:06:08,755][130385] Avg episode reward: [(0, '11.640'), (1, '9.950')] [2023-10-08 04:06:09,906][00611] Updated weights for policy 0, policy_version 4040 (0.0009) [2023-10-08 04:06:10,283][00611] Updated weights for policy 0, policy_version 4050 (0.0008) [2023-10-08 04:06:10,662][00611] Updated weights for policy 0, policy_version 4060 (0.0007) [2023-10-08 04:06:11,399][00612] Updated weights for policy 1, policy_version 4070 (0.0008) [2023-10-08 04:06:11,768][00612] Updated weights for policy 1, policy_version 4080 (0.0010) [2023-10-08 04:06:12,134][00612] Updated weights for policy 1, policy_version 4090 (0.0008) [2023-10-08 04:06:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 8355840. Throughput: 0: 1832.9, 1: 1816.4. Samples: 2095652. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-08 04:06:13,754][130385] Avg episode reward: [(0, '14.050'), (1, '9.540')] [2023-10-08 04:06:13,755][00365] Saving new best policy, reward=14.050! [2023-10-08 04:06:14,250][00611] Updated weights for policy 0, policy_version 4070 (0.0009) [2023-10-08 04:06:14,625][00611] Updated weights for policy 0, policy_version 4080 (0.0008) [2023-10-08 04:06:15,007][00611] Updated weights for policy 0, policy_version 4090 (0.0007) [2023-10-08 04:06:15,965][00612] Updated weights for policy 1, policy_version 4100 (0.0007) [2023-10-08 04:06:16,339][00612] Updated weights for policy 1, policy_version 4110 (0.0007) [2023-10-08 04:06:16,706][00612] Updated weights for policy 1, policy_version 4120 (0.0008) [2023-10-08 04:06:18,504][00611] Updated weights for policy 0, policy_version 4100 (0.0007) [2023-10-08 04:06:18,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 8421376. Throughput: 0: 1835.7, 1: 1829.9. Samples: 2118566. Policy #0 lag: (min: 2.0, avg: 2.3, max: 13.0) [2023-10-08 04:06:18,754][130385] Avg episode reward: [(0, '13.680'), (1, '9.830')] [2023-10-08 04:06:18,760][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000004128_4227072.pth... [2023-10-08 04:06:18,793][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000002432_2490368.pth [2023-10-08 04:06:18,883][00611] Updated weights for policy 0, policy_version 4110 (0.0007) [2023-10-08 04:06:19,250][00611] Updated weights for policy 0, policy_version 4120 (0.0008) [2023-10-08 04:06:19,546][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000004128_4227072.pth... [2023-10-08 04:06:19,575][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000002432_2490368.pth [2023-10-08 04:06:20,338][00612] Updated weights for policy 1, policy_version 4130 (0.0007) [2023-10-08 04:06:20,711][00612] Updated weights for policy 1, policy_version 4140 (0.0008) [2023-10-08 04:06:21,074][00612] Updated weights for policy 1, policy_version 4150 (0.0009) [2023-10-08 04:06:21,445][00612] Updated weights for policy 1, policy_version 4160 (0.0009) [2023-10-08 04:06:23,020][00611] Updated weights for policy 0, policy_version 4130 (0.0009) [2023-10-08 04:06:23,401][00611] Updated weights for policy 0, policy_version 4140 (0.0007) [2023-10-08 04:06:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 8486912. Throughput: 0: 1831.4, 1: 1822.0. Samples: 2128892. Policy #0 lag: (min: 2.0, avg: 2.3, max: 13.0) [2023-10-08 04:06:23,755][130385] Avg episode reward: [(0, '13.580'), (1, '9.410')] [2023-10-08 04:06:23,771][00611] Updated weights for policy 0, policy_version 4150 (0.0008) [2023-10-08 04:06:24,142][00611] Updated weights for policy 0, policy_version 4160 (0.0011) [2023-10-08 04:06:25,099][00612] Updated weights for policy 1, policy_version 4170 (0.0008) [2023-10-08 04:06:25,461][00612] Updated weights for policy 1, policy_version 4180 (0.0008) [2023-10-08 04:06:25,824][00612] Updated weights for policy 1, policy_version 4190 (0.0007) [2023-10-08 04:06:27,877][00611] Updated weights for policy 0, policy_version 4170 (0.0010) [2023-10-08 04:06:28,248][00611] Updated weights for policy 0, policy_version 4180 (0.0008) [2023-10-08 04:06:28,624][00611] Updated weights for policy 0, policy_version 4190 (0.0009) [2023-10-08 04:06:28,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 8585216. Throughput: 0: 1829.8, 1: 1834.2. Samples: 2151356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:06:28,755][130385] Avg episode reward: [(0, '13.120'), (1, '9.350')] [2023-10-08 04:06:29,455][00612] Updated weights for policy 1, policy_version 4200 (0.0008) [2023-10-08 04:06:29,830][00612] Updated weights for policy 1, policy_version 4210 (0.0008) [2023-10-08 04:06:30,203][00612] Updated weights for policy 1, policy_version 4220 (0.0009) [2023-10-08 04:06:32,252][00611] Updated weights for policy 0, policy_version 4200 (0.0011) [2023-10-08 04:06:32,624][00611] Updated weights for policy 0, policy_version 4210 (0.0008) [2023-10-08 04:06:32,992][00611] Updated weights for policy 0, policy_version 4220 (0.0010) [2023-10-08 04:06:33,754][130385] Fps is (10 sec: 16383.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 8650752. Throughput: 0: 1822.3, 1: 1831.6. Samples: 2172956. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-08 04:06:33,755][130385] Avg episode reward: [(0, '14.020'), (1, '10.400')] [2023-10-08 04:06:33,851][00612] Updated weights for policy 1, policy_version 4230 (0.0008) [2023-10-08 04:06:34,218][00612] Updated weights for policy 1, policy_version 4240 (0.0008) [2023-10-08 04:06:34,586][00612] Updated weights for policy 1, policy_version 4250 (0.0009) [2023-10-08 04:06:34,813][00425] Saving new best policy, reward=10.400! [2023-10-08 04:06:36,781][00611] Updated weights for policy 0, policy_version 4230 (0.0008) [2023-10-08 04:06:37,148][00611] Updated weights for policy 0, policy_version 4240 (0.0008) [2023-10-08 04:06:37,527][00611] Updated weights for policy 0, policy_version 4250 (0.0008) [2023-10-08 04:06:38,286][00612] Updated weights for policy 1, policy_version 4260 (0.0011) [2023-10-08 04:06:38,649][00612] Updated weights for policy 1, policy_version 4270 (0.0012) [2023-10-08 04:06:38,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 8716288. Throughput: 0: 1829.1, 1: 1836.5. Samples: 2184364. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-08 04:06:38,754][130385] Avg episode reward: [(0, '13.770'), (1, '10.860')] [2023-10-08 04:06:39,021][00612] Updated weights for policy 1, policy_version 4280 (0.0008) [2023-10-08 04:06:39,312][00425] Saving new best policy, reward=10.860! [2023-10-08 04:06:41,440][00611] Updated weights for policy 0, policy_version 4260 (0.0009) [2023-10-08 04:06:41,821][00611] Updated weights for policy 0, policy_version 4270 (0.0008) [2023-10-08 04:06:42,199][00611] Updated weights for policy 0, policy_version 4280 (0.0007) [2023-10-08 04:06:42,608][00612] Updated weights for policy 1, policy_version 4290 (0.0007) [2023-10-08 04:06:43,033][00612] Updated weights for policy 1, policy_version 4300 (0.0009) [2023-10-08 04:06:43,409][00612] Updated weights for policy 1, policy_version 4310 (0.0009) [2023-10-08 04:06:43,754][130385] Fps is (10 sec: 13107.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 8781824. Throughput: 0: 1825.7, 1: 1840.8. Samples: 2206538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:06:43,754][130385] Avg episode reward: [(0, '14.670'), (1, '9.190')] [2023-10-08 04:06:43,755][00365] Saving new best policy, reward=14.670! [2023-10-08 04:06:43,765][00612] Updated weights for policy 1, policy_version 4320 (0.0008) [2023-10-08 04:06:45,803][00611] Updated weights for policy 0, policy_version 4290 (0.0007) [2023-10-08 04:06:46,170][00611] Updated weights for policy 0, policy_version 4300 (0.0007) [2023-10-08 04:06:46,537][00611] Updated weights for policy 0, policy_version 4310 (0.0007) [2023-10-08 04:06:46,909][00611] Updated weights for policy 0, policy_version 4320 (0.0007) [2023-10-08 04:06:47,455][00612] Updated weights for policy 1, policy_version 4330 (0.0007) [2023-10-08 04:06:47,833][00612] Updated weights for policy 1, policy_version 4340 (0.0010) [2023-10-08 04:06:48,205][00612] Updated weights for policy 1, policy_version 4350 (0.0008) [2023-10-08 04:06:48,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 8880128. Throughput: 0: 1825.8, 1: 1829.9. Samples: 2227454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:06:48,754][130385] Avg episode reward: [(0, '16.130'), (1, '9.070')] [2023-10-08 04:06:48,764][00365] Saving new best policy, reward=16.130! [2023-10-08 04:06:50,676][00611] Updated weights for policy 0, policy_version 4330 (0.0010) [2023-10-08 04:06:51,038][00611] Updated weights for policy 0, policy_version 4340 (0.0007) [2023-10-08 04:06:51,420][00611] Updated weights for policy 0, policy_version 4350 (0.0011) [2023-10-08 04:06:51,814][00612] Updated weights for policy 1, policy_version 4360 (0.0008) [2023-10-08 04:06:52,187][00612] Updated weights for policy 1, policy_version 4370 (0.0008) [2023-10-08 04:06:52,548][00612] Updated weights for policy 1, policy_version 4380 (0.0008) [2023-10-08 04:06:53,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 8945664. Throughput: 0: 1827.1, 1: 1840.4. Samples: 2239244. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-08 04:06:53,755][130385] Avg episode reward: [(0, '16.690'), (1, '9.430')] [2023-10-08 04:06:53,756][00365] Saving new best policy, reward=16.690! [2023-10-08 04:06:55,143][00611] Updated weights for policy 0, policy_version 4360 (0.0010) [2023-10-08 04:06:55,516][00611] Updated weights for policy 0, policy_version 4370 (0.0011) [2023-10-08 04:06:55,900][00611] Updated weights for policy 0, policy_version 4380 (0.0011) [2023-10-08 04:06:56,246][00612] Updated weights for policy 1, policy_version 4390 (0.0008) [2023-10-08 04:06:56,609][00612] Updated weights for policy 1, policy_version 4400 (0.0007) [2023-10-08 04:06:56,974][00612] Updated weights for policy 1, policy_version 4410 (0.0007) [2023-10-08 04:06:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 9011200. Throughput: 0: 1819.2, 1: 1837.3. Samples: 2260194. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-08 04:06:58,754][130385] Avg episode reward: [(0, '15.380'), (1, '9.780')] [2023-10-08 04:06:59,501][00611] Updated weights for policy 0, policy_version 4390 (0.0009) [2023-10-08 04:06:59,876][00611] Updated weights for policy 0, policy_version 4400 (0.0012) [2023-10-08 04:07:00,251][00611] Updated weights for policy 0, policy_version 4410 (0.0010) [2023-10-08 04:07:00,498][00612] Updated weights for policy 1, policy_version 4420 (0.0007) [2023-10-08 04:07:00,862][00612] Updated weights for policy 1, policy_version 4430 (0.0010) [2023-10-08 04:07:01,235][00612] Updated weights for policy 1, policy_version 4440 (0.0008) [2023-10-08 04:07:03,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 9076736. Throughput: 0: 1813.5, 1: 1844.2. Samples: 2283162. Policy #0 lag: (min: 31.0, avg: 32.6, max: 58.0) [2023-10-08 04:07:03,754][130385] Avg episode reward: [(0, '15.660'), (1, '10.150')] [2023-10-08 04:07:03,928][00611] Updated weights for policy 0, policy_version 4420 (0.0008) [2023-10-08 04:07:04,305][00611] Updated weights for policy 0, policy_version 4430 (0.0009) [2023-10-08 04:07:04,668][00611] Updated weights for policy 0, policy_version 4440 (0.0007) [2023-10-08 04:07:04,803][00612] Updated weights for policy 1, policy_version 4450 (0.0010) [2023-10-08 04:07:05,169][00612] Updated weights for policy 1, policy_version 4460 (0.0007) [2023-10-08 04:07:05,539][00612] Updated weights for policy 1, policy_version 4470 (0.0008) [2023-10-08 04:07:05,906][00612] Updated weights for policy 1, policy_version 4480 (0.0009) [2023-10-08 04:07:08,248][00611] Updated weights for policy 0, policy_version 4450 (0.0008) [2023-10-08 04:07:08,620][00611] Updated weights for policy 0, policy_version 4460 (0.0010) [2023-10-08 04:07:08,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 9142272. Throughput: 0: 1816.8, 1: 1834.8. Samples: 2293216. Policy #0 lag: (min: 31.0, avg: 32.6, max: 58.0) [2023-10-08 04:07:08,754][130385] Avg episode reward: [(0, '16.070'), (1, '10.280')] [2023-10-08 04:07:08,991][00611] Updated weights for policy 0, policy_version 4470 (0.0007) [2023-10-08 04:07:09,359][00611] Updated weights for policy 0, policy_version 4480 (0.0007) [2023-10-08 04:07:09,506][00612] Updated weights for policy 1, policy_version 4490 (0.0008) [2023-10-08 04:07:09,863][00612] Updated weights for policy 1, policy_version 4500 (0.0008) [2023-10-08 04:07:10,234][00612] Updated weights for policy 1, policy_version 4510 (0.0007) [2023-10-08 04:07:13,000][00611] Updated weights for policy 0, policy_version 4490 (0.0009) [2023-10-08 04:07:13,375][00611] Updated weights for policy 0, policy_version 4500 (0.0007) [2023-10-08 04:07:13,748][00611] Updated weights for policy 0, policy_version 4510 (0.0009) [2023-10-08 04:07:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 9207808. Throughput: 0: 1820.4, 1: 1847.7. Samples: 2316418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:07:13,755][130385] Avg episode reward: [(0, '15.310'), (1, '11.120')] [2023-10-08 04:07:14,054][00612] Updated weights for policy 1, policy_version 4520 (0.0009) [2023-10-08 04:07:14,421][00612] Updated weights for policy 1, policy_version 4530 (0.0008) [2023-10-08 04:07:14,793][00612] Updated weights for policy 1, policy_version 4540 (0.0009) [2023-10-08 04:07:14,947][00425] Saving new best policy, reward=11.120! [2023-10-08 04:07:17,389][00611] Updated weights for policy 0, policy_version 4520 (0.0007) [2023-10-08 04:07:17,755][00611] Updated weights for policy 0, policy_version 4530 (0.0007) [2023-10-08 04:07:18,124][00611] Updated weights for policy 0, policy_version 4540 (0.0008) [2023-10-08 04:07:18,547][00612] Updated weights for policy 1, policy_version 4550 (0.0009) [2023-10-08 04:07:18,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9306112. Throughput: 0: 1826.9, 1: 1839.9. Samples: 2337960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:07:18,754][130385] Avg episode reward: [(0, '15.870'), (1, '11.780')] [2023-10-08 04:07:18,919][00612] Updated weights for policy 1, policy_version 4560 (0.0008) [2023-10-08 04:07:19,285][00612] Updated weights for policy 1, policy_version 4570 (0.0008) [2023-10-08 04:07:19,509][00425] Saving new best policy, reward=11.780! [2023-10-08 04:07:21,699][00611] Updated weights for policy 0, policy_version 4550 (0.0010) [2023-10-08 04:07:22,070][00611] Updated weights for policy 0, policy_version 4560 (0.0010) [2023-10-08 04:07:22,441][00611] Updated weights for policy 0, policy_version 4570 (0.0008) [2023-10-08 04:07:22,810][00612] Updated weights for policy 1, policy_version 4580 (0.0010) [2023-10-08 04:07:23,174][00612] Updated weights for policy 1, policy_version 4590 (0.0008) [2023-10-08 04:07:23,541][00612] Updated weights for policy 1, policy_version 4600 (0.0007) [2023-10-08 04:07:23,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 9371648. Throughput: 0: 1825.5, 1: 1839.5. Samples: 2349288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:07:23,755][130385] Avg episode reward: [(0, '15.110'), (1, '11.080')] [2023-10-08 04:07:26,048][00611] Updated weights for policy 0, policy_version 4580 (0.0008) [2023-10-08 04:07:26,419][00611] Updated weights for policy 0, policy_version 4590 (0.0009) [2023-10-08 04:07:26,799][00611] Updated weights for policy 0, policy_version 4600 (0.0012) [2023-10-08 04:07:27,247][00612] Updated weights for policy 1, policy_version 4610 (0.0008) [2023-10-08 04:07:27,616][00612] Updated weights for policy 1, policy_version 4620 (0.0008) [2023-10-08 04:07:27,992][00612] Updated weights for policy 1, policy_version 4630 (0.0011) [2023-10-08 04:07:28,350][00612] Updated weights for policy 1, policy_version 4640 (0.0009) [2023-10-08 04:07:28,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9469952. Throughput: 0: 1818.8, 1: 1826.5. Samples: 2370578. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 04:07:28,755][130385] Avg episode reward: [(0, '16.570'), (1, '10.440')] [2023-10-08 04:07:30,551][00611] Updated weights for policy 0, policy_version 4610 (0.0010) [2023-10-08 04:07:30,925][00611] Updated weights for policy 0, policy_version 4620 (0.0008) [2023-10-08 04:07:31,297][00611] Updated weights for policy 0, policy_version 4630 (0.0007) [2023-10-08 04:07:31,660][00611] Updated weights for policy 0, policy_version 4640 (0.0008) [2023-10-08 04:07:31,990][00612] Updated weights for policy 1, policy_version 4650 (0.0008) [2023-10-08 04:07:32,361][00612] Updated weights for policy 1, policy_version 4660 (0.0008) [2023-10-08 04:07:32,729][00612] Updated weights for policy 1, policy_version 4670 (0.0007) [2023-10-08 04:07:33,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 9535488. Throughput: 0: 1831.1, 1: 1832.0. Samples: 2392294. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 04:07:33,755][130385] Avg episode reward: [(0, '15.140'), (1, '11.630')] [2023-10-08 04:07:35,095][00611] Updated weights for policy 0, policy_version 4650 (0.0011) [2023-10-08 04:07:35,463][00611] Updated weights for policy 0, policy_version 4660 (0.0010) [2023-10-08 04:07:35,835][00611] Updated weights for policy 0, policy_version 4670 (0.0010) [2023-10-08 04:07:36,479][00612] Updated weights for policy 1, policy_version 4680 (0.0007) [2023-10-08 04:07:36,845][00612] Updated weights for policy 1, policy_version 4690 (0.0009) [2023-10-08 04:07:37,209][00612] Updated weights for policy 1, policy_version 4700 (0.0009) [2023-10-08 04:07:38,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9601024. Throughput: 0: 1823.2, 1: 1829.8. Samples: 2403628. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:07:38,754][130385] Avg episode reward: [(0, '15.460'), (1, '11.070')] [2023-10-08 04:07:39,540][00611] Updated weights for policy 0, policy_version 4680 (0.0007) [2023-10-08 04:07:39,922][00611] Updated weights for policy 0, policy_version 4690 (0.0007) [2023-10-08 04:07:40,285][00611] Updated weights for policy 0, policy_version 4700 (0.0010) [2023-10-08 04:07:40,928][00612] Updated weights for policy 1, policy_version 4710 (0.0009) [2023-10-08 04:07:41,306][00612] Updated weights for policy 1, policy_version 4720 (0.0007) [2023-10-08 04:07:41,673][00612] Updated weights for policy 1, policy_version 4730 (0.0007) [2023-10-08 04:07:43,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9666560. Throughput: 0: 1831.6, 1: 1826.8. Samples: 2424822. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:07:43,754][130385] Avg episode reward: [(0, '14.920'), (1, '11.360')] [2023-10-08 04:07:43,968][00611] Updated weights for policy 0, policy_version 4710 (0.0008) [2023-10-08 04:07:44,324][00611] Updated weights for policy 0, policy_version 4720 (0.0008) [2023-10-08 04:07:44,686][00611] Updated weights for policy 0, policy_version 4730 (0.0008) [2023-10-08 04:07:45,357][00612] Updated weights for policy 1, policy_version 4740 (0.0008) [2023-10-08 04:07:45,730][00612] Updated weights for policy 1, policy_version 4750 (0.0008) [2023-10-08 04:07:46,096][00612] Updated weights for policy 1, policy_version 4760 (0.0009) [2023-10-08 04:07:48,313][00611] Updated weights for policy 0, policy_version 4740 (0.0009) [2023-10-08 04:07:48,685][00611] Updated weights for policy 0, policy_version 4750 (0.0008) [2023-10-08 04:07:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 9732096. Throughput: 0: 1834.8, 1: 1821.6. Samples: 2447698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:07:48,754][130385] Avg episode reward: [(0, '13.890'), (1, '11.900')] [2023-10-08 04:07:48,764][00425] Saving new best policy, reward=11.900! [2023-10-08 04:07:49,065][00611] Updated weights for policy 0, policy_version 4760 (0.0009) [2023-10-08 04:07:49,834][00612] Updated weights for policy 1, policy_version 4770 (0.0007) [2023-10-08 04:07:50,205][00612] Updated weights for policy 1, policy_version 4780 (0.0008) [2023-10-08 04:07:50,571][00612] Updated weights for policy 1, policy_version 4790 (0.0008) [2023-10-08 04:07:50,938][00612] Updated weights for policy 1, policy_version 4800 (0.0008) [2023-10-08 04:07:52,664][00611] Updated weights for policy 0, policy_version 4770 (0.0009) [2023-10-08 04:07:53,031][00611] Updated weights for policy 0, policy_version 4780 (0.0007) [2023-10-08 04:07:53,407][00611] Updated weights for policy 0, policy_version 4790 (0.0007) [2023-10-08 04:07:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 9797632. Throughput: 0: 1834.9, 1: 1824.0. Samples: 2457868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:07:53,754][130385] Avg episode reward: [(0, '14.450'), (1, '13.090')] [2023-10-08 04:07:53,755][00425] Saving new best policy, reward=13.090! [2023-10-08 04:07:53,774][00611] Updated weights for policy 0, policy_version 4800 (0.0008) [2023-10-08 04:07:54,621][00612] Updated weights for policy 1, policy_version 4810 (0.0007) [2023-10-08 04:07:54,980][00612] Updated weights for policy 1, policy_version 4820 (0.0008) [2023-10-08 04:07:55,359][00612] Updated weights for policy 1, policy_version 4830 (0.0008) [2023-10-08 04:07:57,400][00611] Updated weights for policy 0, policy_version 4810 (0.0009) [2023-10-08 04:07:57,768][00611] Updated weights for policy 0, policy_version 4820 (0.0010) [2023-10-08 04:07:58,133][00611] Updated weights for policy 0, policy_version 4830 (0.0009) [2023-10-08 04:07:58,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 9895936. Throughput: 0: 1832.7, 1: 1821.6. Samples: 2480864. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 04:07:58,755][130385] Avg episode reward: [(0, '15.390'), (1, '13.520')] [2023-10-08 04:07:58,966][00612] Updated weights for policy 1, policy_version 4840 (0.0008) [2023-10-08 04:07:59,340][00612] Updated weights for policy 1, policy_version 4850 (0.0010) [2023-10-08 04:07:59,711][00612] Updated weights for policy 1, policy_version 4860 (0.0008) [2023-10-08 04:07:59,851][00425] Saving new best policy, reward=13.520! [2023-10-08 04:08:01,854][00611] Updated weights for policy 0, policy_version 4840 (0.0009) [2023-10-08 04:08:02,225][00611] Updated weights for policy 0, policy_version 4850 (0.0009) [2023-10-08 04:08:02,580][00611] Updated weights for policy 0, policy_version 4860 (0.0007) [2023-10-08 04:08:03,280][00612] Updated weights for policy 1, policy_version 4870 (0.0008) [2023-10-08 04:08:03,657][00612] Updated weights for policy 1, policy_version 4880 (0.0008) [2023-10-08 04:08:03,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 9961472. Throughput: 0: 1828.9, 1: 1828.9. Samples: 2502562. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-08 04:08:03,755][130385] Avg episode reward: [(0, '18.190'), (1, '12.450')] [2023-10-08 04:08:03,765][00365] Saving new best policy, reward=18.190! [2023-10-08 04:08:04,019][00612] Updated weights for policy 1, policy_version 4890 (0.0008) [2023-10-08 04:08:06,274][00611] Updated weights for policy 0, policy_version 4870 (0.0008) [2023-10-08 04:08:06,653][00611] Updated weights for policy 0, policy_version 4880 (0.0008) [2023-10-08 04:08:07,025][00611] Updated weights for policy 0, policy_version 4890 (0.0010) [2023-10-08 04:08:07,602][00612] Updated weights for policy 1, policy_version 4900 (0.0009) [2023-10-08 04:08:07,971][00612] Updated weights for policy 1, policy_version 4910 (0.0009) [2023-10-08 04:08:08,339][00612] Updated weights for policy 1, policy_version 4920 (0.0007) [2023-10-08 04:08:08,754][130385] Fps is (10 sec: 16384.6, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 10059776. Throughput: 0: 1831.1, 1: 1834.4. Samples: 2514234. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-08 04:08:08,754][130385] Avg episode reward: [(0, '19.490'), (1, '13.540')] [2023-10-08 04:08:08,755][00365] Saving new best policy, reward=19.490! [2023-10-08 04:08:08,755][00425] Saving new best policy, reward=13.540! [2023-10-08 04:08:10,805][00611] Updated weights for policy 0, policy_version 4900 (0.0009) [2023-10-08 04:08:11,179][00611] Updated weights for policy 0, policy_version 4910 (0.0007) [2023-10-08 04:08:11,552][00611] Updated weights for policy 0, policy_version 4920 (0.0007) [2023-10-08 04:08:12,032][00612] Updated weights for policy 1, policy_version 4930 (0.0007) [2023-10-08 04:08:12,402][00612] Updated weights for policy 1, policy_version 4940 (0.0007) [2023-10-08 04:08:12,778][00612] Updated weights for policy 1, policy_version 4950 (0.0007) [2023-10-08 04:08:13,150][00612] Updated weights for policy 1, policy_version 4960 (0.0008) [2023-10-08 04:08:13,754][130385] Fps is (10 sec: 16384.6, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 10125312. Throughput: 0: 1836.2, 1: 1834.1. Samples: 2535742. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-10-08 04:08:13,754][130385] Avg episode reward: [(0, '19.890'), (1, '12.070')] [2023-10-08 04:08:13,755][00365] Saving new best policy, reward=19.890! [2023-10-08 04:08:15,193][00611] Updated weights for policy 0, policy_version 4930 (0.0008) [2023-10-08 04:08:15,563][00611] Updated weights for policy 0, policy_version 4940 (0.0009) [2023-10-08 04:08:15,943][00611] Updated weights for policy 0, policy_version 4950 (0.0007) [2023-10-08 04:08:16,306][00611] Updated weights for policy 0, policy_version 4960 (0.0010) [2023-10-08 04:08:16,781][00612] Updated weights for policy 1, policy_version 4970 (0.0007) [2023-10-08 04:08:17,148][00612] Updated weights for policy 1, policy_version 4980 (0.0007) [2023-10-08 04:08:17,515][00612] Updated weights for policy 1, policy_version 4990 (0.0007) [2023-10-08 04:08:18,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 10190848. Throughput: 0: 1840.6, 1: 1835.3. Samples: 2557712. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-10-08 04:08:18,755][130385] Avg episode reward: [(0, '19.660'), (1, '13.170')] [2023-10-08 04:08:18,766][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000004960_5079040.pth... [2023-10-08 04:08:18,766][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000004992_5111808.pth... [2023-10-08 04:08:18,807][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000003264_3342336.pth [2023-10-08 04:08:18,809][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000003264_3342336.pth [2023-10-08 04:08:19,858][00611] Updated weights for policy 0, policy_version 4970 (0.0007) [2023-10-08 04:08:20,225][00611] Updated weights for policy 0, policy_version 4980 (0.0008) [2023-10-08 04:08:20,598][00611] Updated weights for policy 0, policy_version 4990 (0.0009) [2023-10-08 04:08:21,220][00612] Updated weights for policy 1, policy_version 5000 (0.0009) [2023-10-08 04:08:21,587][00612] Updated weights for policy 1, policy_version 5010 (0.0009) [2023-10-08 04:08:21,955][00612] Updated weights for policy 1, policy_version 5020 (0.0009) [2023-10-08 04:08:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 10256384. Throughput: 0: 1841.0, 1: 1830.7. Samples: 2568856. Policy #0 lag: (min: 18.0, avg: 38.6, max: 40.0) [2023-10-08 04:08:23,754][130385] Avg episode reward: [(0, '20.850'), (1, '12.100')] [2023-10-08 04:08:23,755][00365] Saving new best policy, reward=20.850! [2023-10-08 04:08:24,307][00611] Updated weights for policy 0, policy_version 5000 (0.0009) [2023-10-08 04:08:24,677][00611] Updated weights for policy 0, policy_version 5010 (0.0008) [2023-10-08 04:08:25,049][00611] Updated weights for policy 0, policy_version 5020 (0.0009) [2023-10-08 04:08:25,671][00612] Updated weights for policy 1, policy_version 5030 (0.0009) [2023-10-08 04:08:26,053][00612] Updated weights for policy 1, policy_version 5040 (0.0009) [2023-10-08 04:08:26,420][00612] Updated weights for policy 1, policy_version 5050 (0.0007) [2023-10-08 04:08:28,628][00611] Updated weights for policy 0, policy_version 5030 (0.0009) [2023-10-08 04:08:28,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 10321920. Throughput: 0: 1844.6, 1: 1839.2. Samples: 2590592. Policy #0 lag: (min: 18.0, avg: 38.6, max: 40.0) [2023-10-08 04:08:28,755][130385] Avg episode reward: [(0, '19.250'), (1, '12.310')] [2023-10-08 04:08:29,002][00611] Updated weights for policy 0, policy_version 5040 (0.0008) [2023-10-08 04:08:29,378][00611] Updated weights for policy 0, policy_version 5050 (0.0008) [2023-10-08 04:08:30,078][00612] Updated weights for policy 1, policy_version 5060 (0.0007) [2023-10-08 04:08:30,446][00612] Updated weights for policy 1, policy_version 5070 (0.0007) [2023-10-08 04:08:30,814][00612] Updated weights for policy 1, policy_version 5080 (0.0009) [2023-10-08 04:08:33,148][00611] Updated weights for policy 0, policy_version 5060 (0.0008) [2023-10-08 04:08:33,514][00611] Updated weights for policy 0, policy_version 5070 (0.0008) [2023-10-08 04:08:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 10387456. Throughput: 0: 1832.5, 1: 1850.2. Samples: 2613420. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 04:08:33,754][130385] Avg episode reward: [(0, '18.270'), (1, '12.210')] [2023-10-08 04:08:33,890][00611] Updated weights for policy 0, policy_version 5080 (0.0008) [2023-10-08 04:08:34,313][00612] Updated weights for policy 1, policy_version 5090 (0.0008) [2023-10-08 04:08:34,682][00612] Updated weights for policy 1, policy_version 5100 (0.0007) [2023-10-08 04:08:35,053][00612] Updated weights for policy 1, policy_version 5110 (0.0008) [2023-10-08 04:08:35,421][00612] Updated weights for policy 1, policy_version 5120 (0.0008) [2023-10-08 04:08:37,486][00611] Updated weights for policy 0, policy_version 5090 (0.0008) [2023-10-08 04:08:37,851][00611] Updated weights for policy 0, policy_version 5100 (0.0007) [2023-10-08 04:08:38,222][00611] Updated weights for policy 0, policy_version 5110 (0.0007) [2023-10-08 04:08:38,596][00611] Updated weights for policy 0, policy_version 5120 (0.0008) [2023-10-08 04:08:38,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 10485760. Throughput: 0: 1836.6, 1: 1846.3. Samples: 2623598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:08:38,755][130385] Avg episode reward: [(0, '18.620'), (1, '11.330')] [2023-10-08 04:08:39,052][00612] Updated weights for policy 1, policy_version 5130 (0.0010) [2023-10-08 04:08:39,428][00612] Updated weights for policy 1, policy_version 5140 (0.0009) [2023-10-08 04:08:39,796][00612] Updated weights for policy 1, policy_version 5150 (0.0007) [2023-10-08 04:08:42,238][00611] Updated weights for policy 0, policy_version 5130 (0.0010) [2023-10-08 04:08:42,593][00611] Updated weights for policy 0, policy_version 5140 (0.0008) [2023-10-08 04:08:42,966][00611] Updated weights for policy 0, policy_version 5150 (0.0007) [2023-10-08 04:08:43,478][00612] Updated weights for policy 1, policy_version 5160 (0.0007) [2023-10-08 04:08:43,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 10551296. Throughput: 0: 1833.7, 1: 1846.2. Samples: 2646456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:08:43,755][130385] Avg episode reward: [(0, '20.360'), (1, '11.590')] [2023-10-08 04:08:43,846][00612] Updated weights for policy 1, policy_version 5170 (0.0007) [2023-10-08 04:08:44,217][00612] Updated weights for policy 1, policy_version 5180 (0.0010) [2023-10-08 04:08:46,581][00611] Updated weights for policy 0, policy_version 5160 (0.0009) [2023-10-08 04:08:46,952][00611] Updated weights for policy 0, policy_version 5170 (0.0008) [2023-10-08 04:08:47,334][00611] Updated weights for policy 0, policy_version 5180 (0.0009) [2023-10-08 04:08:47,829][00612] Updated weights for policy 1, policy_version 5190 (0.0008) [2023-10-08 04:08:48,196][00612] Updated weights for policy 1, policy_version 5200 (0.0008) [2023-10-08 04:08:48,562][00612] Updated weights for policy 1, policy_version 5210 (0.0007) [2023-10-08 04:08:48,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 10616832. Throughput: 0: 1836.8, 1: 1831.5. Samples: 2667634. Policy #0 lag: (min: 26.0, avg: 33.7, max: 58.0) [2023-10-08 04:08:48,754][130385] Avg episode reward: [(0, '19.080'), (1, '12.730')] [2023-10-08 04:08:51,056][00611] Updated weights for policy 0, policy_version 5190 (0.0009) [2023-10-08 04:08:51,438][00611] Updated weights for policy 0, policy_version 5200 (0.0010) [2023-10-08 04:08:51,814][00611] Updated weights for policy 0, policy_version 5210 (0.0010) [2023-10-08 04:08:52,363][00612] Updated weights for policy 1, policy_version 5220 (0.0008) [2023-10-08 04:08:52,727][00612] Updated weights for policy 1, policy_version 5230 (0.0008) [2023-10-08 04:08:53,094][00612] Updated weights for policy 1, policy_version 5240 (0.0011) [2023-10-08 04:08:53,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 10715136. Throughput: 0: 1823.8, 1: 1841.0. Samples: 2679152. Policy #0 lag: (min: 26.0, avg: 33.7, max: 58.0) [2023-10-08 04:08:53,755][130385] Avg episode reward: [(0, '18.940'), (1, '13.690')] [2023-10-08 04:08:53,756][00425] Saving new best policy, reward=13.690! [2023-10-08 04:08:55,410][00611] Updated weights for policy 0, policy_version 5220 (0.0008) [2023-10-08 04:08:55,789][00611] Updated weights for policy 0, policy_version 5230 (0.0008) [2023-10-08 04:08:56,156][00611] Updated weights for policy 0, policy_version 5240 (0.0007) [2023-10-08 04:08:56,822][00612] Updated weights for policy 1, policy_version 5250 (0.0009) [2023-10-08 04:08:57,187][00612] Updated weights for policy 1, policy_version 5260 (0.0007) [2023-10-08 04:08:57,547][00612] Updated weights for policy 1, policy_version 5270 (0.0007) [2023-10-08 04:08:57,916][00612] Updated weights for policy 1, policy_version 5280 (0.0007) [2023-10-08 04:08:58,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 10780672. Throughput: 0: 1833.8, 1: 1827.8. Samples: 2700516. Policy #0 lag: (min: 1.0, avg: 9.7, max: 33.0) [2023-10-08 04:08:58,755][130385] Avg episode reward: [(0, '20.870'), (1, '14.420')] [2023-10-08 04:08:58,755][00365] Saving new best policy, reward=20.870! [2023-10-08 04:08:58,756][00425] Saving new best policy, reward=14.420! [2023-10-08 04:08:59,782][00611] Updated weights for policy 0, policy_version 5250 (0.0007) [2023-10-08 04:09:00,153][00611] Updated weights for policy 0, policy_version 5260 (0.0007) [2023-10-08 04:09:00,518][00611] Updated weights for policy 0, policy_version 5270 (0.0007) [2023-10-08 04:09:00,892][00611] Updated weights for policy 0, policy_version 5280 (0.0009) [2023-10-08 04:09:01,520][00612] Updated weights for policy 1, policy_version 5290 (0.0007) [2023-10-08 04:09:01,887][00612] Updated weights for policy 1, policy_version 5300 (0.0008) [2023-10-08 04:09:02,266][00612] Updated weights for policy 1, policy_version 5310 (0.0007) [2023-10-08 04:09:03,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 10846208. Throughput: 0: 1835.6, 1: 1835.9. Samples: 2722930. Policy #0 lag: (min: 1.0, avg: 9.7, max: 33.0) [2023-10-08 04:09:03,754][130385] Avg episode reward: [(0, '20.650'), (1, '14.280')] [2023-10-08 04:09:04,502][00611] Updated weights for policy 0, policy_version 5290 (0.0008) [2023-10-08 04:09:04,876][00611] Updated weights for policy 0, policy_version 5300 (0.0007) [2023-10-08 04:09:05,248][00611] Updated weights for policy 0, policy_version 5310 (0.0009) [2023-10-08 04:09:05,858][00612] Updated weights for policy 1, policy_version 5320 (0.0007) [2023-10-08 04:09:06,234][00612] Updated weights for policy 1, policy_version 5330 (0.0007) [2023-10-08 04:09:06,605][00612] Updated weights for policy 1, policy_version 5340 (0.0007) [2023-10-08 04:09:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 10911744. Throughput: 0: 1834.5, 1: 1828.0. Samples: 2733668. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-08 04:09:08,754][130385] Avg episode reward: [(0, '21.370'), (1, '14.530')] [2023-10-08 04:09:08,755][00425] Saving new best policy, reward=14.530! [2023-10-08 04:09:08,809][00611] Updated weights for policy 0, policy_version 5320 (0.0007) [2023-10-08 04:09:09,193][00611] Updated weights for policy 0, policy_version 5330 (0.0010) [2023-10-08 04:09:09,556][00611] Updated weights for policy 0, policy_version 5340 (0.0008) [2023-10-08 04:09:09,702][00365] Saving new best policy, reward=21.370! [2023-10-08 04:09:10,213][00612] Updated weights for policy 1, policy_version 5350 (0.0009) [2023-10-08 04:09:10,577][00612] Updated weights for policy 1, policy_version 5360 (0.0010) [2023-10-08 04:09:10,942][00612] Updated weights for policy 1, policy_version 5370 (0.0011) [2023-10-08 04:09:13,182][00611] Updated weights for policy 0, policy_version 5350 (0.0007) [2023-10-08 04:09:13,545][00611] Updated weights for policy 0, policy_version 5360 (0.0007) [2023-10-08 04:09:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 10977280. Throughput: 0: 1844.5, 1: 1838.4. Samples: 2756322. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-08 04:09:13,755][130385] Avg episode reward: [(0, '21.760'), (1, '14.800')] [2023-10-08 04:09:13,755][00425] Saving new best policy, reward=14.800! [2023-10-08 04:09:13,912][00611] Updated weights for policy 0, policy_version 5370 (0.0009) [2023-10-08 04:09:14,131][00365] Saving new best policy, reward=21.760! [2023-10-08 04:09:14,524][00612] Updated weights for policy 1, policy_version 5380 (0.0009) [2023-10-08 04:09:14,897][00612] Updated weights for policy 1, policy_version 5390 (0.0009) [2023-10-08 04:09:15,253][00612] Updated weights for policy 1, policy_version 5400 (0.0007) [2023-10-08 04:09:17,633][00611] Updated weights for policy 0, policy_version 5380 (0.0007) [2023-10-08 04:09:18,008][00611] Updated weights for policy 0, policy_version 5390 (0.0008) [2023-10-08 04:09:18,377][00611] Updated weights for policy 0, policy_version 5400 (0.0009) [2023-10-08 04:09:18,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 11075584. Throughput: 0: 1833.7, 1: 1836.0. Samples: 2778556. Policy #0 lag: (min: 4.0, avg: 5.1, max: 26.0) [2023-10-08 04:09:18,755][130385] Avg episode reward: [(0, '23.590'), (1, '13.830')] [2023-10-08 04:09:18,768][00365] Saving new best policy, reward=23.590! [2023-10-08 04:09:18,996][00612] Updated weights for policy 1, policy_version 5410 (0.0010) [2023-10-08 04:09:19,363][00612] Updated weights for policy 1, policy_version 5420 (0.0008) [2023-10-08 04:09:19,743][00612] Updated weights for policy 1, policy_version 5430 (0.0009) [2023-10-08 04:09:20,111][00612] Updated weights for policy 1, policy_version 5440 (0.0007) [2023-10-08 04:09:22,029][00611] Updated weights for policy 0, policy_version 5410 (0.0010) [2023-10-08 04:09:22,400][00611] Updated weights for policy 0, policy_version 5420 (0.0008) [2023-10-08 04:09:22,760][00611] Updated weights for policy 0, policy_version 5430 (0.0009) [2023-10-08 04:09:23,137][00611] Updated weights for policy 0, policy_version 5440 (0.0009) [2023-10-08 04:09:23,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 11141120. Throughput: 0: 1847.1, 1: 1834.4. Samples: 2789268. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) [2023-10-08 04:09:23,754][130385] Avg episode reward: [(0, '22.410'), (1, '14.430')] [2023-10-08 04:09:23,789][00612] Updated weights for policy 1, policy_version 5450 (0.0007) [2023-10-08 04:09:24,159][00612] Updated weights for policy 1, policy_version 5460 (0.0008) [2023-10-08 04:09:24,521][00612] Updated weights for policy 1, policy_version 5470 (0.0009) [2023-10-08 04:09:26,903][00611] Updated weights for policy 0, policy_version 5450 (0.0008) [2023-10-08 04:09:27,281][00611] Updated weights for policy 0, policy_version 5460 (0.0008) [2023-10-08 04:09:27,659][00611] Updated weights for policy 0, policy_version 5470 (0.0008) [2023-10-08 04:09:28,013][00612] Updated weights for policy 1, policy_version 5480 (0.0010) [2023-10-08 04:09:28,384][00612] Updated weights for policy 1, policy_version 5490 (0.0009) [2023-10-08 04:09:28,754][00612] Updated weights for policy 1, policy_version 5500 (0.0011) [2023-10-08 04:09:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 11206656. Throughput: 0: 1830.7, 1: 1835.2. Samples: 2811422. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) [2023-10-08 04:09:28,755][130385] Avg episode reward: [(0, '23.800'), (1, '15.160')] [2023-10-08 04:09:28,757][00365] Saving new best policy, reward=23.800! [2023-10-08 04:09:28,899][00425] Saving new best policy, reward=15.160! [2023-10-08 04:09:31,185][00611] Updated weights for policy 0, policy_version 5480 (0.0007) [2023-10-08 04:09:31,568][00611] Updated weights for policy 0, policy_version 5490 (0.0007) [2023-10-08 04:09:31,948][00611] Updated weights for policy 0, policy_version 5500 (0.0011) [2023-10-08 04:09:32,533][00612] Updated weights for policy 1, policy_version 5510 (0.0009) [2023-10-08 04:09:32,899][00612] Updated weights for policy 1, policy_version 5520 (0.0007) [2023-10-08 04:09:33,274][00612] Updated weights for policy 1, policy_version 5530 (0.0010) [2023-10-08 04:09:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 11304960. Throughput: 0: 1844.1, 1: 1824.6. Samples: 2832726. Policy #0 lag: (min: 17.0, avg: 28.7, max: 49.0) [2023-10-08 04:09:33,754][130385] Avg episode reward: [(0, '26.740'), (1, '15.880')] [2023-10-08 04:09:33,762][00365] Saving new best policy, reward=26.740! [2023-10-08 04:09:33,762][00425] Saving new best policy, reward=15.880! [2023-10-08 04:09:35,488][00611] Updated weights for policy 0, policy_version 5510 (0.0010) [2023-10-08 04:09:35,849][00611] Updated weights for policy 0, policy_version 5520 (0.0009) [2023-10-08 04:09:36,224][00611] Updated weights for policy 0, policy_version 5530 (0.0008) [2023-10-08 04:09:36,851][00612] Updated weights for policy 1, policy_version 5540 (0.0009) [2023-10-08 04:09:37,224][00612] Updated weights for policy 1, policy_version 5550 (0.0009) [2023-10-08 04:09:37,594][00612] Updated weights for policy 1, policy_version 5560 (0.0008) [2023-10-08 04:09:38,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 11370496. Throughput: 0: 1836.6, 1: 1833.5. Samples: 2844308. Policy #0 lag: (min: 17.0, avg: 28.7, max: 49.0) [2023-10-08 04:09:38,755][130385] Avg episode reward: [(0, '24.980'), (1, '16.250')] [2023-10-08 04:09:38,757][00425] Saving new best policy, reward=16.250! [2023-10-08 04:09:39,983][00611] Updated weights for policy 0, policy_version 5540 (0.0008) [2023-10-08 04:09:40,370][00611] Updated weights for policy 0, policy_version 5550 (0.0009) [2023-10-08 04:09:40,750][00611] Updated weights for policy 0, policy_version 5560 (0.0009) [2023-10-08 04:09:41,302][00612] Updated weights for policy 1, policy_version 5570 (0.0008) [2023-10-08 04:09:41,675][00612] Updated weights for policy 1, policy_version 5580 (0.0009) [2023-10-08 04:09:42,040][00612] Updated weights for policy 1, policy_version 5590 (0.0011) [2023-10-08 04:09:42,403][00612] Updated weights for policy 1, policy_version 5600 (0.0010) [2023-10-08 04:09:43,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 11436032. Throughput: 0: 1844.3, 1: 1822.7. Samples: 2865534. Policy #0 lag: (min: 17.0, avg: 25.6, max: 49.0) [2023-10-08 04:09:43,755][130385] Avg episode reward: [(0, '27.620'), (1, '16.920')] [2023-10-08 04:09:43,756][00365] Saving new best policy, reward=27.620! [2023-10-08 04:09:43,756][00425] Saving new best policy, reward=16.920! [2023-10-08 04:09:44,289][00611] Updated weights for policy 0, policy_version 5570 (0.0008) [2023-10-08 04:09:44,676][00611] Updated weights for policy 0, policy_version 5580 (0.0009) [2023-10-08 04:09:45,055][00611] Updated weights for policy 0, policy_version 5590 (0.0009) [2023-10-08 04:09:45,419][00611] Updated weights for policy 0, policy_version 5600 (0.0010) [2023-10-08 04:09:46,047][00612] Updated weights for policy 1, policy_version 5610 (0.0011) [2023-10-08 04:09:46,424][00612] Updated weights for policy 1, policy_version 5620 (0.0009) [2023-10-08 04:09:46,785][00612] Updated weights for policy 1, policy_version 5630 (0.0009) [2023-10-08 04:09:48,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 11501568. Throughput: 0: 1835.8, 1: 1838.1. Samples: 2888256. Policy #0 lag: (min: 17.0, avg: 25.6, max: 49.0) [2023-10-08 04:09:48,754][130385] Avg episode reward: [(0, '29.020'), (1, '15.390')] [2023-10-08 04:09:48,995][00611] Updated weights for policy 0, policy_version 5610 (0.0011) [2023-10-08 04:09:49,365][00611] Updated weights for policy 0, policy_version 5620 (0.0009) [2023-10-08 04:09:49,734][00611] Updated weights for policy 0, policy_version 5630 (0.0007) [2023-10-08 04:09:49,812][00365] Saving new best policy, reward=29.020! [2023-10-08 04:09:50,463][00612] Updated weights for policy 1, policy_version 5640 (0.0010) [2023-10-08 04:09:50,830][00612] Updated weights for policy 1, policy_version 5650 (0.0009) [2023-10-08 04:09:51,206][00612] Updated weights for policy 1, policy_version 5660 (0.0010) [2023-10-08 04:09:53,543][00611] Updated weights for policy 0, policy_version 5640 (0.0008) [2023-10-08 04:09:53,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 11567104. Throughput: 0: 1838.8, 1: 1827.5. Samples: 2898652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:09:53,754][130385] Avg episode reward: [(0, '27.490'), (1, '15.140')] [2023-10-08 04:09:53,917][00611] Updated weights for policy 0, policy_version 5650 (0.0008) [2023-10-08 04:09:54,290][00611] Updated weights for policy 0, policy_version 5660 (0.0009) [2023-10-08 04:09:54,901][00612] Updated weights for policy 1, policy_version 5670 (0.0009) [2023-10-08 04:09:55,271][00612] Updated weights for policy 1, policy_version 5680 (0.0008) [2023-10-08 04:09:55,642][00612] Updated weights for policy 1, policy_version 5690 (0.0007) [2023-10-08 04:09:57,803][00611] Updated weights for policy 0, policy_version 5670 (0.0010) [2023-10-08 04:09:58,173][00611] Updated weights for policy 0, policy_version 5680 (0.0009) [2023-10-08 04:09:58,545][00611] Updated weights for policy 0, policy_version 5690 (0.0008) [2023-10-08 04:09:58,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 11632640. Throughput: 0: 1832.1, 1: 1840.2. Samples: 2921576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:09:58,755][130385] Avg episode reward: [(0, '26.860'), (1, '14.490')] [2023-10-08 04:09:59,321][00612] Updated weights for policy 1, policy_version 5700 (0.0007) [2023-10-08 04:09:59,712][00612] Updated weights for policy 1, policy_version 5710 (0.0008) [2023-10-08 04:10:00,085][00612] Updated weights for policy 1, policy_version 5720 (0.0010) [2023-10-08 04:10:02,165][00611] Updated weights for policy 0, policy_version 5700 (0.0009) [2023-10-08 04:10:02,538][00611] Updated weights for policy 0, policy_version 5710 (0.0011) [2023-10-08 04:10:02,912][00611] Updated weights for policy 0, policy_version 5720 (0.0009) [2023-10-08 04:10:03,472][00612] Updated weights for policy 1, policy_version 5730 (0.0008) [2023-10-08 04:10:03,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 11730944. Throughput: 0: 1820.0, 1: 1841.8. Samples: 2943336. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:10:03,754][130385] Avg episode reward: [(0, '26.230'), (1, '14.690')] [2023-10-08 04:10:03,842][00612] Updated weights for policy 1, policy_version 5740 (0.0007) [2023-10-08 04:10:04,203][00612] Updated weights for policy 1, policy_version 5750 (0.0008) [2023-10-08 04:10:04,577][00612] Updated weights for policy 1, policy_version 5760 (0.0008) [2023-10-08 04:10:06,663][00611] Updated weights for policy 0, policy_version 5730 (0.0007) [2023-10-08 04:10:07,024][00611] Updated weights for policy 0, policy_version 5740 (0.0010) [2023-10-08 04:10:07,391][00611] Updated weights for policy 0, policy_version 5750 (0.0010) [2023-10-08 04:10:07,755][00611] Updated weights for policy 0, policy_version 5760 (0.0008) [2023-10-08 04:10:08,248][00612] Updated weights for policy 1, policy_version 5770 (0.0009) [2023-10-08 04:10:08,624][00612] Updated weights for policy 1, policy_version 5780 (0.0008) [2023-10-08 04:10:08,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 11796480. Throughput: 0: 1833.3, 1: 1843.4. Samples: 2954718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-08 04:10:08,755][130385] Avg episode reward: [(0, '24.730'), (1, '14.470')] [2023-10-08 04:10:08,989][00612] Updated weights for policy 1, policy_version 5790 (0.0010) [2023-10-08 04:10:11,520][00611] Updated weights for policy 0, policy_version 5770 (0.0008) [2023-10-08 04:10:11,904][00611] Updated weights for policy 0, policy_version 5780 (0.0008) [2023-10-08 04:10:12,272][00611] Updated weights for policy 0, policy_version 5790 (0.0009) [2023-10-08 04:10:12,507][00612] Updated weights for policy 1, policy_version 5800 (0.0007) [2023-10-08 04:10:12,869][00612] Updated weights for policy 1, policy_version 5810 (0.0007) [2023-10-08 04:10:13,243][00612] Updated weights for policy 1, policy_version 5820 (0.0009) [2023-10-08 04:10:13,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 11894784. Throughput: 0: 1822.9, 1: 1846.3. Samples: 2976538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-08 04:10:13,754][130385] Avg episode reward: [(0, '24.230'), (1, '15.130')] [2023-10-08 04:10:15,961][00611] Updated weights for policy 0, policy_version 5800 (0.0009) [2023-10-08 04:10:16,338][00611] Updated weights for policy 0, policy_version 5810 (0.0009) [2023-10-08 04:10:16,701][00611] Updated weights for policy 0, policy_version 5820 (0.0008) [2023-10-08 04:10:16,900][00612] Updated weights for policy 1, policy_version 5830 (0.0008) [2023-10-08 04:10:17,273][00612] Updated weights for policy 1, policy_version 5840 (0.0007) [2023-10-08 04:10:17,643][00612] Updated weights for policy 1, policy_version 5850 (0.0009) [2023-10-08 04:10:18,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 11960320. Throughput: 0: 1831.8, 1: 1838.0. Samples: 2997870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:10:18,755][130385] Avg episode reward: [(0, '24.810'), (1, '14.550')] [2023-10-08 04:10:18,764][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000005856_5996544.pth... [2023-10-08 04:10:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000005824_5963776.pth... [2023-10-08 04:10:18,803][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000004128_4227072.pth [2023-10-08 04:10:18,805][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000004128_4227072.pth [2023-10-08 04:10:20,395][00611] Updated weights for policy 0, policy_version 5830 (0.0007) [2023-10-08 04:10:20,768][00611] Updated weights for policy 0, policy_version 5840 (0.0008) [2023-10-08 04:10:21,139][00611] Updated weights for policy 0, policy_version 5850 (0.0008) [2023-10-08 04:10:21,181][00612] Updated weights for policy 1, policy_version 5860 (0.0009) [2023-10-08 04:10:21,540][00612] Updated weights for policy 1, policy_version 5870 (0.0007) [2023-10-08 04:10:21,910][00612] Updated weights for policy 1, policy_version 5880 (0.0008) [2023-10-08 04:10:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 12025856. Throughput: 0: 1821.7, 1: 1846.1. Samples: 3009358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:10:23,754][130385] Avg episode reward: [(0, '26.410'), (1, '16.120')] [2023-10-08 04:10:24,689][00611] Updated weights for policy 0, policy_version 5860 (0.0008) [2023-10-08 04:10:25,067][00611] Updated weights for policy 0, policy_version 5870 (0.0008) [2023-10-08 04:10:25,446][00611] Updated weights for policy 0, policy_version 5880 (0.0009) [2023-10-08 04:10:25,544][00612] Updated weights for policy 1, policy_version 5890 (0.0009) [2023-10-08 04:10:25,914][00612] Updated weights for policy 1, policy_version 5900 (0.0009) [2023-10-08 04:10:26,278][00612] Updated weights for policy 1, policy_version 5910 (0.0007) [2023-10-08 04:10:26,644][00612] Updated weights for policy 1, policy_version 5920 (0.0007) [2023-10-08 04:10:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 12091392. Throughput: 0: 1832.3, 1: 1845.7. Samples: 3031046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:10:28,754][130385] Avg episode reward: [(0, '26.700'), (1, '16.390')] [2023-10-08 04:10:29,073][00611] Updated weights for policy 0, policy_version 5890 (0.0008) [2023-10-08 04:10:29,484][00611] Updated weights for policy 0, policy_version 5900 (0.0007) [2023-10-08 04:10:29,864][00611] Updated weights for policy 0, policy_version 5910 (0.0009) [2023-10-08 04:10:30,226][00611] Updated weights for policy 0, policy_version 5920 (0.0009) [2023-10-08 04:10:30,395][00612] Updated weights for policy 1, policy_version 5930 (0.0007) [2023-10-08 04:10:30,768][00612] Updated weights for policy 1, policy_version 5940 (0.0007) [2023-10-08 04:10:31,136][00612] Updated weights for policy 1, policy_version 5950 (0.0009) [2023-10-08 04:10:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 12156928. Throughput: 0: 1832.7, 1: 1852.8. Samples: 3054102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:10:33,754][130385] Avg episode reward: [(0, '27.580'), (1, '16.770')] [2023-10-08 04:10:33,842][00611] Updated weights for policy 0, policy_version 5930 (0.0007) [2023-10-08 04:10:34,209][00611] Updated weights for policy 0, policy_version 5940 (0.0008) [2023-10-08 04:10:34,585][00611] Updated weights for policy 0, policy_version 5950 (0.0009) [2023-10-08 04:10:34,927][00612] Updated weights for policy 1, policy_version 5960 (0.0007) [2023-10-08 04:10:35,301][00612] Updated weights for policy 1, policy_version 5970 (0.0008) [2023-10-08 04:10:35,670][00612] Updated weights for policy 1, policy_version 5980 (0.0009) [2023-10-08 04:10:38,232][00611] Updated weights for policy 0, policy_version 5960 (0.0007) [2023-10-08 04:10:38,611][00611] Updated weights for policy 0, policy_version 5970 (0.0008) [2023-10-08 04:10:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 12222464. Throughput: 0: 1832.6, 1: 1844.0. Samples: 3064098. Policy #0 lag: (min: 17.0, avg: 32.9, max: 49.0) [2023-10-08 04:10:38,754][130385] Avg episode reward: [(0, '28.880'), (1, '17.650')] [2023-10-08 04:10:38,755][00425] Saving new best policy, reward=17.650! [2023-10-08 04:10:38,992][00611] Updated weights for policy 0, policy_version 5980 (0.0008) [2023-10-08 04:10:39,360][00612] Updated weights for policy 1, policy_version 5990 (0.0010) [2023-10-08 04:10:39,734][00612] Updated weights for policy 1, policy_version 6000 (0.0008) [2023-10-08 04:10:40,106][00612] Updated weights for policy 1, policy_version 6010 (0.0007) [2023-10-08 04:10:42,606][00611] Updated weights for policy 0, policy_version 5990 (0.0008) [2023-10-08 04:10:42,981][00611] Updated weights for policy 0, policy_version 6000 (0.0007) [2023-10-08 04:10:43,349][00611] Updated weights for policy 0, policy_version 6010 (0.0007) [2023-10-08 04:10:43,596][00612] Updated weights for policy 1, policy_version 6020 (0.0008) [2023-10-08 04:10:43,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 12320768. Throughput: 0: 1831.6, 1: 1848.0. Samples: 3087158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:10:43,755][130385] Avg episode reward: [(0, '30.360'), (1, '17.150')] [2023-10-08 04:10:43,756][00365] Saving new best policy, reward=30.360! [2023-10-08 04:10:43,963][00612] Updated weights for policy 1, policy_version 6030 (0.0008) [2023-10-08 04:10:44,343][00612] Updated weights for policy 1, policy_version 6040 (0.0007) [2023-10-08 04:10:47,077][00611] Updated weights for policy 0, policy_version 6020 (0.0007) [2023-10-08 04:10:47,448][00611] Updated weights for policy 0, policy_version 6030 (0.0008) [2023-10-08 04:10:47,829][00611] Updated weights for policy 0, policy_version 6040 (0.0009) [2023-10-08 04:10:48,021][00612] Updated weights for policy 1, policy_version 6050 (0.0007) [2023-10-08 04:10:48,391][00612] Updated weights for policy 1, policy_version 6060 (0.0009) [2023-10-08 04:10:48,754][130385] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 12386304. Throughput: 0: 1828.8, 1: 1841.1. Samples: 3108484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:10:48,755][130385] Avg episode reward: [(0, '29.400'), (1, '19.210')] [2023-10-08 04:10:48,773][00612] Updated weights for policy 1, policy_version 6070 (0.0008) [2023-10-08 04:10:49,140][00425] Saving new best policy, reward=19.210! [2023-10-08 04:10:49,141][00612] Updated weights for policy 1, policy_version 6080 (0.0011) [2023-10-08 04:10:51,763][00611] Updated weights for policy 0, policy_version 6050 (0.0009) [2023-10-08 04:10:52,133][00611] Updated weights for policy 0, policy_version 6060 (0.0008) [2023-10-08 04:10:52,502][00611] Updated weights for policy 0, policy_version 6070 (0.0007) [2023-10-08 04:10:52,812][00612] Updated weights for policy 1, policy_version 6090 (0.0007) [2023-10-08 04:10:52,863][00611] Updated weights for policy 0, policy_version 6080 (0.0007) [2023-10-08 04:10:53,178][00612] Updated weights for policy 1, policy_version 6100 (0.0009) [2023-10-08 04:10:53,553][00612] Updated weights for policy 1, policy_version 6110 (0.0011) [2023-10-08 04:10:53,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 12484608. Throughput: 0: 1828.4, 1: 1844.8. Samples: 3120016. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-08 04:10:53,754][130385] Avg episode reward: [(0, '28.480'), (1, '20.290')] [2023-10-08 04:10:53,755][00425] Saving new best policy, reward=20.290! [2023-10-08 04:10:56,483][00611] Updated weights for policy 0, policy_version 6090 (0.0008) [2023-10-08 04:10:56,861][00611] Updated weights for policy 0, policy_version 6100 (0.0011) [2023-10-08 04:10:57,224][00612] Updated weights for policy 1, policy_version 6120 (0.0007) [2023-10-08 04:10:57,241][00611] Updated weights for policy 0, policy_version 6110 (0.0007) [2023-10-08 04:10:57,598][00612] Updated weights for policy 1, policy_version 6130 (0.0008) [2023-10-08 04:10:57,964][00612] Updated weights for policy 1, policy_version 6140 (0.0008) [2023-10-08 04:10:58,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 12550144. Throughput: 0: 1832.0, 1: 1835.6. Samples: 3141584. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-08 04:10:58,755][130385] Avg episode reward: [(0, '26.220'), (1, '19.900')] [2023-10-08 04:11:00,733][00611] Updated weights for policy 0, policy_version 6120 (0.0010) [2023-10-08 04:11:01,108][00611] Updated weights for policy 0, policy_version 6130 (0.0009) [2023-10-08 04:11:01,483][00611] Updated weights for policy 0, policy_version 6140 (0.0009) [2023-10-08 04:11:01,559][00612] Updated weights for policy 1, policy_version 6150 (0.0008) [2023-10-08 04:11:01,919][00612] Updated weights for policy 1, policy_version 6160 (0.0010) [2023-10-08 04:11:02,286][00612] Updated weights for policy 1, policy_version 6170 (0.0010) [2023-10-08 04:11:03,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 12615680. Throughput: 0: 1834.9, 1: 1843.9. Samples: 3163416. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 04:11:03,755][130385] Avg episode reward: [(0, '24.000'), (1, '20.150')] [2023-10-08 04:11:05,064][00611] Updated weights for policy 0, policy_version 6150 (0.0008) [2023-10-08 04:11:05,433][00611] Updated weights for policy 0, policy_version 6160 (0.0008) [2023-10-08 04:11:05,807][00611] Updated weights for policy 0, policy_version 6170 (0.0008) [2023-10-08 04:11:05,908][00612] Updated weights for policy 1, policy_version 6180 (0.0009) [2023-10-08 04:11:06,277][00612] Updated weights for policy 1, policy_version 6190 (0.0007) [2023-10-08 04:11:06,635][00612] Updated weights for policy 1, policy_version 6200 (0.0008) [2023-10-08 04:11:08,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 12681216. Throughput: 0: 1831.5, 1: 1834.6. Samples: 3174330. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 04:11:08,754][130385] Avg episode reward: [(0, '24.980'), (1, '21.110')] [2023-10-08 04:11:08,755][00425] Saving new best policy, reward=21.110! [2023-10-08 04:11:09,272][00611] Updated weights for policy 0, policy_version 6180 (0.0008) [2023-10-08 04:11:09,652][00611] Updated weights for policy 0, policy_version 6190 (0.0007) [2023-10-08 04:11:10,023][00611] Updated weights for policy 0, policy_version 6200 (0.0007) [2023-10-08 04:11:10,208][00612] Updated weights for policy 1, policy_version 6210 (0.0010) [2023-10-08 04:11:10,575][00612] Updated weights for policy 1, policy_version 6220 (0.0007) [2023-10-08 04:11:10,945][00612] Updated weights for policy 1, policy_version 6230 (0.0008) [2023-10-08 04:11:11,305][00612] Updated weights for policy 1, policy_version 6240 (0.0007) [2023-10-08 04:11:13,707][00611] Updated weights for policy 0, policy_version 6210 (0.0008) [2023-10-08 04:11:13,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 12746752. Throughput: 0: 1837.6, 1: 1844.8. Samples: 3196754. Policy #0 lag: (min: 30.0, avg: 30.8, max: 49.0) [2023-10-08 04:11:13,755][130385] Avg episode reward: [(0, '26.350'), (1, '21.020')] [2023-10-08 04:11:14,080][00611] Updated weights for policy 0, policy_version 6220 (0.0007) [2023-10-08 04:11:14,450][00611] Updated weights for policy 0, policy_version 6230 (0.0007) [2023-10-08 04:11:14,829][00611] Updated weights for policy 0, policy_version 6240 (0.0007) [2023-10-08 04:11:14,866][00612] Updated weights for policy 1, policy_version 6250 (0.0008) [2023-10-08 04:11:15,235][00612] Updated weights for policy 1, policy_version 6260 (0.0007) [2023-10-08 04:11:15,606][00612] Updated weights for policy 1, policy_version 6270 (0.0010) [2023-10-08 04:11:18,707][00611] Updated weights for policy 0, policy_version 6250 (0.0010) [2023-10-08 04:11:18,754][130385] Fps is (10 sec: 13106.5, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 12812288. Throughput: 0: 1838.9, 1: 1846.1. Samples: 3219928. Policy #0 lag: (min: 30.0, avg: 30.8, max: 49.0) [2023-10-08 04:11:18,756][130385] Avg episode reward: [(0, '25.900'), (1, '21.520')] [2023-10-08 04:11:18,767][00425] Saving new best policy, reward=21.520! [2023-10-08 04:11:19,084][00611] Updated weights for policy 0, policy_version 6260 (0.0010) [2023-10-08 04:11:19,258][00612] Updated weights for policy 1, policy_version 6280 (0.0009) [2023-10-08 04:11:19,458][00611] Updated weights for policy 0, policy_version 6270 (0.0008) [2023-10-08 04:11:19,630][00612] Updated weights for policy 1, policy_version 6290 (0.0009) [2023-10-08 04:11:19,991][00612] Updated weights for policy 1, policy_version 6300 (0.0008) [2023-10-08 04:11:23,100][00611] Updated weights for policy 0, policy_version 6280 (0.0007) [2023-10-08 04:11:23,471][00611] Updated weights for policy 0, policy_version 6290 (0.0008) [2023-10-08 04:11:23,641][00612] Updated weights for policy 1, policy_version 6310 (0.0007) [2023-10-08 04:11:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 12877824. Throughput: 0: 1831.7, 1: 1850.3. Samples: 3229788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:11:23,754][130385] Avg episode reward: [(0, '26.330'), (1, '22.850')] [2023-10-08 04:11:23,848][00611] Updated weights for policy 0, policy_version 6300 (0.0009) [2023-10-08 04:11:24,007][00612] Updated weights for policy 1, policy_version 6320 (0.0010) [2023-10-08 04:11:24,381][00612] Updated weights for policy 1, policy_version 6330 (0.0007) [2023-10-08 04:11:24,609][00425] Saving new best policy, reward=22.850! [2023-10-08 04:11:27,455][00611] Updated weights for policy 0, policy_version 6310 (0.0008) [2023-10-08 04:11:27,837][00611] Updated weights for policy 0, policy_version 6320 (0.0009) [2023-10-08 04:11:27,925][00612] Updated weights for policy 1, policy_version 6340 (0.0008) [2023-10-08 04:11:28,202][00611] Updated weights for policy 0, policy_version 6330 (0.0008) [2023-10-08 04:11:28,287][00612] Updated weights for policy 1, policy_version 6350 (0.0007) [2023-10-08 04:11:28,656][00612] Updated weights for policy 1, policy_version 6360 (0.0007) [2023-10-08 04:11:28,754][130385] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 12976128. Throughput: 0: 1832.7, 1: 1848.4. Samples: 3252804. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 04:11:28,754][130385] Avg episode reward: [(0, '25.970'), (1, '22.010')] [2023-10-08 04:11:31,982][00611] Updated weights for policy 0, policy_version 6340 (0.0008) [2023-10-08 04:11:32,355][00611] Updated weights for policy 0, policy_version 6350 (0.0008) [2023-10-08 04:11:32,468][00612] Updated weights for policy 1, policy_version 6370 (0.0008) [2023-10-08 04:11:32,730][00611] Updated weights for policy 0, policy_version 6360 (0.0007) [2023-10-08 04:11:32,862][00612] Updated weights for policy 1, policy_version 6380 (0.0009) [2023-10-08 04:11:33,230][00612] Updated weights for policy 1, policy_version 6390 (0.0008) [2023-10-08 04:11:33,605][00612] Updated weights for policy 1, policy_version 6400 (0.0007) [2023-10-08 04:11:33,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 13074432. Throughput: 0: 1828.2, 1: 1828.8. Samples: 3273048. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 04:11:33,755][130385] Avg episode reward: [(0, '24.500'), (1, '23.460')] [2023-10-08 04:11:33,764][00425] Saving new best policy, reward=23.460! [2023-10-08 04:11:36,431][00611] Updated weights for policy 0, policy_version 6370 (0.0009) [2023-10-08 04:11:36,806][00611] Updated weights for policy 0, policy_version 6380 (0.0007) [2023-10-08 04:11:37,171][00611] Updated weights for policy 0, policy_version 6390 (0.0008) [2023-10-08 04:11:37,333][00612] Updated weights for policy 1, policy_version 6410 (0.0008) [2023-10-08 04:11:37,541][00611] Updated weights for policy 0, policy_version 6400 (0.0008) [2023-10-08 04:11:37,708][00612] Updated weights for policy 1, policy_version 6420 (0.0008) [2023-10-08 04:11:38,075][00612] Updated weights for policy 1, policy_version 6430 (0.0007) [2023-10-08 04:11:38,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 13139968. Throughput: 0: 1833.9, 1: 1841.9. Samples: 3285424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:11:38,755][130385] Avg episode reward: [(0, '22.710'), (1, '24.170')] [2023-10-08 04:11:38,756][00425] Saving new best policy, reward=24.170! [2023-10-08 04:11:41,297][00611] Updated weights for policy 0, policy_version 6410 (0.0007) [2023-10-08 04:11:41,670][00611] Updated weights for policy 0, policy_version 6420 (0.0008) [2023-10-08 04:11:41,774][00612] Updated weights for policy 1, policy_version 6440 (0.0007) [2023-10-08 04:11:42,046][00611] Updated weights for policy 0, policy_version 6430 (0.0010) [2023-10-08 04:11:42,142][00612] Updated weights for policy 1, policy_version 6450 (0.0007) [2023-10-08 04:11:42,512][00612] Updated weights for policy 1, policy_version 6460 (0.0007) [2023-10-08 04:11:43,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 13205504. Throughput: 0: 1824.4, 1: 1826.3. Samples: 3305864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:11:43,754][130385] Avg episode reward: [(0, '22.710'), (1, '22.780')] [2023-10-08 04:11:45,559][00611] Updated weights for policy 0, policy_version 6440 (0.0009) [2023-10-08 04:11:45,896][00612] Updated weights for policy 1, policy_version 6470 (0.0007) [2023-10-08 04:11:45,921][00611] Updated weights for policy 0, policy_version 6450 (0.0008) [2023-10-08 04:11:46,272][00612] Updated weights for policy 1, policy_version 6480 (0.0007) [2023-10-08 04:11:46,303][00611] Updated weights for policy 0, policy_version 6460 (0.0008) [2023-10-08 04:11:46,636][00612] Updated weights for policy 1, policy_version 6490 (0.0007) [2023-10-08 04:11:48,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 13271040. Throughput: 0: 1822.9, 1: 1840.0. Samples: 3328248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:11:48,755][130385] Avg episode reward: [(0, '20.910'), (1, '20.930')] [2023-10-08 04:11:50,134][00611] Updated weights for policy 0, policy_version 6470 (0.0008) [2023-10-08 04:11:50,294][00612] Updated weights for policy 1, policy_version 6500 (0.0008) [2023-10-08 04:11:50,514][00611] Updated weights for policy 0, policy_version 6480 (0.0008) [2023-10-08 04:11:50,662][00612] Updated weights for policy 1, policy_version 6510 (0.0007) [2023-10-08 04:11:50,879][00611] Updated weights for policy 0, policy_version 6490 (0.0009) [2023-10-08 04:11:51,027][00612] Updated weights for policy 1, policy_version 6520 (0.0010) [2023-10-08 04:11:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 13336576. Throughput: 0: 1821.2, 1: 1824.0. Samples: 3338364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:11:53,754][130385] Avg episode reward: [(0, '20.910'), (1, '20.560')] [2023-10-08 04:11:54,523][00611] Updated weights for policy 0, policy_version 6500 (0.0009) [2023-10-08 04:11:54,773][00612] Updated weights for policy 1, policy_version 6530 (0.0010) [2023-10-08 04:11:54,891][00611] Updated weights for policy 0, policy_version 6510 (0.0008) [2023-10-08 04:11:55,144][00612] Updated weights for policy 1, policy_version 6540 (0.0008) [2023-10-08 04:11:55,263][00611] Updated weights for policy 0, policy_version 6520 (0.0007) [2023-10-08 04:11:55,517][00612] Updated weights for policy 1, policy_version 6550 (0.0008) [2023-10-08 04:11:55,879][00612] Updated weights for policy 1, policy_version 6560 (0.0009) [2023-10-08 04:11:58,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 13402112. Throughput: 0: 1821.1, 1: 1834.9. Samples: 3361272. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-10-08 04:11:58,754][130385] Avg episode reward: [(0, '24.050'), (1, '21.260')] [2023-10-08 04:11:58,804][00611] Updated weights for policy 0, policy_version 6530 (0.0009) [2023-10-08 04:11:59,164][00611] Updated weights for policy 0, policy_version 6540 (0.0011) [2023-10-08 04:11:59,533][00611] Updated weights for policy 0, policy_version 6550 (0.0007) [2023-10-08 04:11:59,649][00612] Updated weights for policy 1, policy_version 6570 (0.0009) [2023-10-08 04:11:59,909][00611] Updated weights for policy 0, policy_version 6560 (0.0008) [2023-10-08 04:12:00,008][00612] Updated weights for policy 1, policy_version 6580 (0.0009) [2023-10-08 04:12:00,381][00612] Updated weights for policy 1, policy_version 6590 (0.0008) [2023-10-08 04:12:03,674][00611] Updated weights for policy 0, policy_version 6570 (0.0010) [2023-10-08 04:12:03,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 13467648. Throughput: 0: 1813.7, 1: 1828.1. Samples: 3383806. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-10-08 04:12:03,755][130385] Avg episode reward: [(0, '24.490'), (1, '23.120')] [2023-10-08 04:12:04,051][00611] Updated weights for policy 0, policy_version 6580 (0.0008) [2023-10-08 04:12:04,090][00612] Updated weights for policy 1, policy_version 6600 (0.0009) [2023-10-08 04:12:04,416][00611] Updated weights for policy 0, policy_version 6590 (0.0009) [2023-10-08 04:12:04,455][00612] Updated weights for policy 1, policy_version 6610 (0.0008) [2023-10-08 04:12:04,820][00612] Updated weights for policy 1, policy_version 6620 (0.0009) [2023-10-08 04:12:08,117][00611] Updated weights for policy 0, policy_version 6600 (0.0009) [2023-10-08 04:12:08,431][00612] Updated weights for policy 1, policy_version 6630 (0.0009) [2023-10-08 04:12:08,494][00611] Updated weights for policy 0, policy_version 6610 (0.0009) [2023-10-08 04:12:08,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 13533184. Throughput: 0: 1814.8, 1: 1826.8. Samples: 3393664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:12:08,754][130385] Avg episode reward: [(0, '24.020'), (1, '22.300')] [2023-10-08 04:12:08,795][00612] Updated weights for policy 1, policy_version 6640 (0.0008) [2023-10-08 04:12:08,860][00611] Updated weights for policy 0, policy_version 6620 (0.0009) [2023-10-08 04:12:09,173][00612] Updated weights for policy 1, policy_version 6650 (0.0011) [2023-10-08 04:12:12,443][00611] Updated weights for policy 0, policy_version 6630 (0.0008) [2023-10-08 04:12:12,809][00611] Updated weights for policy 0, policy_version 6640 (0.0008) [2023-10-08 04:12:12,940][00612] Updated weights for policy 1, policy_version 6660 (0.0011) [2023-10-08 04:12:13,187][00611] Updated weights for policy 0, policy_version 6650 (0.0008) [2023-10-08 04:12:13,311][00612] Updated weights for policy 1, policy_version 6670 (0.0009) [2023-10-08 04:12:13,685][00612] Updated weights for policy 1, policy_version 6680 (0.0008) [2023-10-08 04:12:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 13631488. Throughput: 0: 1817.3, 1: 1821.8. Samples: 3416562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:12:13,755][130385] Avg episode reward: [(0, '26.660'), (1, '22.750')] [2023-10-08 04:12:16,941][00611] Updated weights for policy 0, policy_version 6660 (0.0008) [2023-10-08 04:12:17,304][00611] Updated weights for policy 0, policy_version 6670 (0.0007) [2023-10-08 04:12:17,446][00612] Updated weights for policy 1, policy_version 6690 (0.0007) [2023-10-08 04:12:17,673][00611] Updated weights for policy 0, policy_version 6680 (0.0008) [2023-10-08 04:12:17,854][00612] Updated weights for policy 1, policy_version 6700 (0.0008) [2023-10-08 04:12:18,217][00612] Updated weights for policy 1, policy_version 6710 (0.0008) [2023-10-08 04:12:18,586][00612] Updated weights for policy 1, policy_version 6720 (0.0008) [2023-10-08 04:12:18,754][130385] Fps is (10 sec: 19660.2, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 13729792. Throughput: 0: 1820.2, 1: 1823.2. Samples: 3437004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:12:18,755][130385] Avg episode reward: [(0, '29.270'), (1, '24.060')] [2023-10-08 04:12:18,767][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000006720_6881280.pth... [2023-10-08 04:12:18,767][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000006688_6848512.pth... [2023-10-08 04:12:18,809][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000004960_5079040.pth [2023-10-08 04:12:18,809][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000004992_5111808.pth [2023-10-08 04:12:21,320][00611] Updated weights for policy 0, policy_version 6690 (0.0007) [2023-10-08 04:12:21,698][00611] Updated weights for policy 0, policy_version 6700 (0.0009) [2023-10-08 04:12:22,067][00611] Updated weights for policy 0, policy_version 6710 (0.0007) [2023-10-08 04:12:22,227][00612] Updated weights for policy 1, policy_version 6730 (0.0009) [2023-10-08 04:12:22,433][00611] Updated weights for policy 0, policy_version 6720 (0.0008) [2023-10-08 04:12:22,592][00612] Updated weights for policy 1, policy_version 6740 (0.0008) [2023-10-08 04:12:22,956][00612] Updated weights for policy 1, policy_version 6750 (0.0007) [2023-10-08 04:12:23,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 13795328. Throughput: 0: 1820.1, 1: 1825.8. Samples: 3449492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:12:23,754][130385] Avg episode reward: [(0, '30.060'), (1, '22.470')] [2023-10-08 04:12:26,190][00611] Updated weights for policy 0, policy_version 6730 (0.0012) [2023-10-08 04:12:26,560][00611] Updated weights for policy 0, policy_version 6740 (0.0008) [2023-10-08 04:12:26,637][00612] Updated weights for policy 1, policy_version 6760 (0.0009) [2023-10-08 04:12:26,938][00611] Updated weights for policy 0, policy_version 6750 (0.0007) [2023-10-08 04:12:27,012][00612] Updated weights for policy 1, policy_version 6770 (0.0008) [2023-10-08 04:12:27,382][00612] Updated weights for policy 1, policy_version 6780 (0.0007) [2023-10-08 04:12:28,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 13860864. Throughput: 0: 1818.1, 1: 1821.0. Samples: 3469626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:12:28,754][130385] Avg episode reward: [(0, '28.050'), (1, '22.970')] [2023-10-08 04:12:30,614][00611] Updated weights for policy 0, policy_version 6760 (0.0007) [2023-10-08 04:12:30,840][00612] Updated weights for policy 1, policy_version 6790 (0.0008) [2023-10-08 04:12:30,988][00611] Updated weights for policy 0, policy_version 6770 (0.0009) [2023-10-08 04:12:31,199][00612] Updated weights for policy 1, policy_version 6800 (0.0007) [2023-10-08 04:12:31,357][00611] Updated weights for policy 0, policy_version 6780 (0.0010) [2023-10-08 04:12:31,572][00612] Updated weights for policy 1, policy_version 6810 (0.0010) [2023-10-08 04:12:33,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 13926400. Throughput: 0: 1821.6, 1: 1821.0. Samples: 3492164. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 04:12:33,756][130385] Avg episode reward: [(0, '26.300'), (1, '22.080')] [2023-10-08 04:12:35,049][00611] Updated weights for policy 0, policy_version 6790 (0.0009) [2023-10-08 04:12:35,221][00612] Updated weights for policy 1, policy_version 6820 (0.0010) [2023-10-08 04:12:35,415][00611] Updated weights for policy 0, policy_version 6800 (0.0009) [2023-10-08 04:12:35,583][00612] Updated weights for policy 1, policy_version 6830 (0.0010) [2023-10-08 04:12:35,789][00611] Updated weights for policy 0, policy_version 6810 (0.0007) [2023-10-08 04:12:35,942][00612] Updated weights for policy 1, policy_version 6840 (0.0008) [2023-10-08 04:12:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 13991936. Throughput: 0: 1822.4, 1: 1820.6. Samples: 3502300. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 04:12:38,755][130385] Avg episode reward: [(0, '26.170'), (1, '23.210')] [2023-10-08 04:12:39,410][00611] Updated weights for policy 0, policy_version 6820 (0.0009) [2023-10-08 04:12:39,625][00612] Updated weights for policy 1, policy_version 6850 (0.0008) [2023-10-08 04:12:39,776][00611] Updated weights for policy 0, policy_version 6830 (0.0007) [2023-10-08 04:12:39,998][00612] Updated weights for policy 1, policy_version 6860 (0.0009) [2023-10-08 04:12:40,157][00611] Updated weights for policy 0, policy_version 6840 (0.0008) [2023-10-08 04:12:40,359][00612] Updated weights for policy 1, policy_version 6870 (0.0007) [2023-10-08 04:12:40,728][00612] Updated weights for policy 1, policy_version 6880 (0.0010) [2023-10-08 04:12:43,720][00611] Updated weights for policy 0, policy_version 6850 (0.0009) [2023-10-08 04:12:43,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 14057472. Throughput: 0: 1819.2, 1: 1824.0. Samples: 3525218. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 04:12:43,754][130385] Avg episode reward: [(0, '24.990'), (1, '22.960')] [2023-10-08 04:12:44,093][00611] Updated weights for policy 0, policy_version 6860 (0.0009) [2023-10-08 04:12:44,461][00612] Updated weights for policy 1, policy_version 6890 (0.0007) [2023-10-08 04:12:44,466][00611] Updated weights for policy 0, policy_version 6870 (0.0009) [2023-10-08 04:12:44,831][00612] Updated weights for policy 1, policy_version 6900 (0.0007) [2023-10-08 04:12:44,838][00611] Updated weights for policy 0, policy_version 6880 (0.0009) [2023-10-08 04:12:45,209][00612] Updated weights for policy 1, policy_version 6910 (0.0008) [2023-10-08 04:12:48,560][00611] Updated weights for policy 0, policy_version 6890 (0.0007) [2023-10-08 04:12:48,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.6, 300 sec: 14662.3). Total num frames: 14123008. Throughput: 0: 1821.6, 1: 1828.4. Samples: 3548056. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 04:12:48,754][130385] Avg episode reward: [(0, '26.570'), (1, '23.050')] [2023-10-08 04:12:48,884][00612] Updated weights for policy 1, policy_version 6920 (0.0007) [2023-10-08 04:12:48,926][00611] Updated weights for policy 0, policy_version 6900 (0.0008) [2023-10-08 04:12:49,254][00612] Updated weights for policy 1, policy_version 6930 (0.0007) [2023-10-08 04:12:49,312][00611] Updated weights for policy 0, policy_version 6910 (0.0008) [2023-10-08 04:12:49,616][00612] Updated weights for policy 1, policy_version 6940 (0.0009) [2023-10-08 04:12:53,131][00611] Updated weights for policy 0, policy_version 6920 (0.0007) [2023-10-08 04:12:53,480][00612] Updated weights for policy 1, policy_version 6950 (0.0009) [2023-10-08 04:12:53,514][00611] Updated weights for policy 0, policy_version 6930 (0.0007) [2023-10-08 04:12:53,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 14188544. Throughput: 0: 1824.0, 1: 1820.5. Samples: 3557668. Policy #0 lag: (min: 30.0, avg: 50.8, max: 56.0) [2023-10-08 04:12:53,755][130385] Avg episode reward: [(0, '27.990'), (1, '21.750')] [2023-10-08 04:12:53,850][00612] Updated weights for policy 1, policy_version 6960 (0.0007) [2023-10-08 04:12:53,880][00611] Updated weights for policy 0, policy_version 6940 (0.0008) [2023-10-08 04:12:54,223][00612] Updated weights for policy 1, policy_version 6970 (0.0008) [2023-10-08 04:12:57,562][00611] Updated weights for policy 0, policy_version 6950 (0.0008) [2023-10-08 04:12:57,866][00612] Updated weights for policy 1, policy_version 6980 (0.0010) [2023-10-08 04:12:57,927][00611] Updated weights for policy 0, policy_version 6960 (0.0008) [2023-10-08 04:12:58,246][00612] Updated weights for policy 1, policy_version 6990 (0.0007) [2023-10-08 04:12:58,302][00611] Updated weights for policy 0, policy_version 6970 (0.0007) [2023-10-08 04:12:58,610][00612] Updated weights for policy 1, policy_version 7000 (0.0008) [2023-10-08 04:12:58,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 14286848. Throughput: 0: 1818.3, 1: 1834.6. Samples: 3580942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:12:58,754][130385] Avg episode reward: [(0, '28.280'), (1, '22.140')] [2023-10-08 04:13:02,041][00611] Updated weights for policy 0, policy_version 6980 (0.0008) [2023-10-08 04:13:02,172][00612] Updated weights for policy 1, policy_version 7010 (0.0008) [2023-10-08 04:13:02,409][00611] Updated weights for policy 0, policy_version 6990 (0.0009) [2023-10-08 04:13:02,536][00612] Updated weights for policy 1, policy_version 7020 (0.0008) [2023-10-08 04:13:02,773][00611] Updated weights for policy 0, policy_version 7000 (0.0010) [2023-10-08 04:13:02,911][00612] Updated weights for policy 1, policy_version 7030 (0.0007) [2023-10-08 04:13:03,276][00612] Updated weights for policy 1, policy_version 7040 (0.0007) [2023-10-08 04:13:03,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 14385152. Throughput: 0: 1813.7, 1: 1830.8. Samples: 3601008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:13:03,755][130385] Avg episode reward: [(0, '27.240'), (1, '23.060')] [2023-10-08 04:13:06,516][00611] Updated weights for policy 0, policy_version 7010 (0.0009) [2023-10-08 04:13:06,832][00612] Updated weights for policy 1, policy_version 7050 (0.0007) [2023-10-08 04:13:06,884][00611] Updated weights for policy 0, policy_version 7020 (0.0008) [2023-10-08 04:13:07,196][00612] Updated weights for policy 1, policy_version 7060 (0.0007) [2023-10-08 04:13:07,257][00611] Updated weights for policy 0, policy_version 7030 (0.0008) [2023-10-08 04:13:07,564][00612] Updated weights for policy 1, policy_version 7070 (0.0007) [2023-10-08 04:13:07,615][00611] Updated weights for policy 0, policy_version 7040 (0.0008) [2023-10-08 04:13:08,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 14450688. Throughput: 0: 1811.4, 1: 1843.3. Samples: 3613952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:13:08,754][130385] Avg episode reward: [(0, '28.620'), (1, '24.050')] [2023-10-08 04:13:11,213][00611] Updated weights for policy 0, policy_version 7050 (0.0007) [2023-10-08 04:13:11,331][00612] Updated weights for policy 1, policy_version 7080 (0.0007) [2023-10-08 04:13:11,602][00611] Updated weights for policy 0, policy_version 7060 (0.0008) [2023-10-08 04:13:11,697][00612] Updated weights for policy 1, policy_version 7090 (0.0007) [2023-10-08 04:13:11,964][00611] Updated weights for policy 0, policy_version 7070 (0.0009) [2023-10-08 04:13:12,065][00612] Updated weights for policy 1, policy_version 7100 (0.0007) [2023-10-08 04:13:13,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 14516224. Throughput: 0: 1818.0, 1: 1830.8. Samples: 3633826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:13:13,754][130385] Avg episode reward: [(0, '29.930'), (1, '26.250')] [2023-10-08 04:13:13,755][00425] Saving new best policy, reward=26.250! [2023-10-08 04:13:15,600][00611] Updated weights for policy 0, policy_version 7080 (0.0009) [2023-10-08 04:13:15,744][00612] Updated weights for policy 1, policy_version 7110 (0.0008) [2023-10-08 04:13:15,970][00611] Updated weights for policy 0, policy_version 7090 (0.0008) [2023-10-08 04:13:16,117][00612] Updated weights for policy 1, policy_version 7120 (0.0008) [2023-10-08 04:13:16,339][00611] Updated weights for policy 0, policy_version 7100 (0.0008) [2023-10-08 04:13:16,491][00612] Updated weights for policy 1, policy_version 7130 (0.0009) [2023-10-08 04:13:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 14581760. Throughput: 0: 1818.1, 1: 1838.7. Samples: 3656718. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 04:13:18,754][130385] Avg episode reward: [(0, '30.010'), (1, '25.380')] [2023-10-08 04:13:19,935][00611] Updated weights for policy 0, policy_version 7110 (0.0007) [2023-10-08 04:13:20,194][00612] Updated weights for policy 1, policy_version 7140 (0.0007) [2023-10-08 04:13:20,310][00611] Updated weights for policy 0, policy_version 7120 (0.0008) [2023-10-08 04:13:20,564][00612] Updated weights for policy 1, policy_version 7150 (0.0009) [2023-10-08 04:13:20,682][00611] Updated weights for policy 0, policy_version 7130 (0.0007) [2023-10-08 04:13:20,928][00612] Updated weights for policy 1, policy_version 7160 (0.0009) [2023-10-08 04:13:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 14647296. Throughput: 0: 1821.1, 1: 1833.8. Samples: 3666772. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 04:13:23,755][130385] Avg episode reward: [(0, '28.430'), (1, '26.680')] [2023-10-08 04:13:23,756][00425] Saving new best policy, reward=26.680! [2023-10-08 04:13:24,471][00611] Updated weights for policy 0, policy_version 7140 (0.0007) [2023-10-08 04:13:24,493][00612] Updated weights for policy 1, policy_version 7170 (0.0008) [2023-10-08 04:13:24,838][00611] Updated weights for policy 0, policy_version 7150 (0.0007) [2023-10-08 04:13:24,866][00612] Updated weights for policy 1, policy_version 7180 (0.0007) [2023-10-08 04:13:25,206][00611] Updated weights for policy 0, policy_version 7160 (0.0007) [2023-10-08 04:13:25,226][00612] Updated weights for policy 1, policy_version 7190 (0.0007) [2023-10-08 04:13:25,596][00612] Updated weights for policy 1, policy_version 7200 (0.0008) [2023-10-08 04:13:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 14712832. Throughput: 0: 1816.2, 1: 1836.4. Samples: 3689582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:13:28,754][130385] Avg episode reward: [(0, '29.820'), (1, '26.070')] [2023-10-08 04:13:29,074][00611] Updated weights for policy 0, policy_version 7170 (0.0008) [2023-10-08 04:13:29,301][00612] Updated weights for policy 1, policy_version 7210 (0.0007) [2023-10-08 04:13:29,442][00611] Updated weights for policy 0, policy_version 7180 (0.0009) [2023-10-08 04:13:29,664][00612] Updated weights for policy 1, policy_version 7220 (0.0009) [2023-10-08 04:13:29,814][00611] Updated weights for policy 0, policy_version 7190 (0.0009) [2023-10-08 04:13:30,035][00612] Updated weights for policy 1, policy_version 7230 (0.0007) [2023-10-08 04:13:30,181][00611] Updated weights for policy 0, policy_version 7200 (0.0009) [2023-10-08 04:13:33,675][00612] Updated weights for policy 1, policy_version 7240 (0.0009) [2023-10-08 04:13:33,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 14778368. Throughput: 0: 1815.4, 1: 1837.1. Samples: 3712418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:13:33,754][130385] Avg episode reward: [(0, '30.880'), (1, '25.300')] [2023-10-08 04:13:33,932][00611] Updated weights for policy 0, policy_version 7210 (0.0009) [2023-10-08 04:13:34,043][00612] Updated weights for policy 1, policy_version 7250 (0.0008) [2023-10-08 04:13:34,302][00611] Updated weights for policy 0, policy_version 7220 (0.0008) [2023-10-08 04:13:34,408][00612] Updated weights for policy 1, policy_version 7260 (0.0010) [2023-10-08 04:13:34,670][00611] Updated weights for policy 0, policy_version 7230 (0.0007) [2023-10-08 04:13:34,742][00365] Saving new best policy, reward=30.880! [2023-10-08 04:13:37,922][00612] Updated weights for policy 1, policy_version 7270 (0.0008) [2023-10-08 04:13:38,290][00612] Updated weights for policy 1, policy_version 7280 (0.0007) [2023-10-08 04:13:38,392][00611] Updated weights for policy 0, policy_version 7240 (0.0007) [2023-10-08 04:13:38,653][00612] Updated weights for policy 1, policy_version 7290 (0.0007) [2023-10-08 04:13:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 14843904. Throughput: 0: 1811.1, 1: 1842.9. Samples: 3722096. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 04:13:38,754][130385] Avg episode reward: [(0, '30.270'), (1, '23.870')] [2023-10-08 04:13:38,766][00611] Updated weights for policy 0, policy_version 7250 (0.0008) [2023-10-08 04:13:39,139][00611] Updated weights for policy 0, policy_version 7260 (0.0010) [2023-10-08 04:13:42,311][00612] Updated weights for policy 1, policy_version 7300 (0.0008) [2023-10-08 04:13:42,682][00612] Updated weights for policy 1, policy_version 7310 (0.0008) [2023-10-08 04:13:42,824][00611] Updated weights for policy 0, policy_version 7270 (0.0009) [2023-10-08 04:13:43,053][00612] Updated weights for policy 1, policy_version 7320 (0.0008) [2023-10-08 04:13:43,185][00611] Updated weights for policy 0, policy_version 7280 (0.0009) [2023-10-08 04:13:43,569][00611] Updated weights for policy 0, policy_version 7290 (0.0007) [2023-10-08 04:13:43,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 14942208. Throughput: 0: 1808.8, 1: 1840.2. Samples: 3745146. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 04:13:43,755][130385] Avg episode reward: [(0, '28.050'), (1, '25.750')] [2023-10-08 04:13:46,630][00612] Updated weights for policy 1, policy_version 7330 (0.0008) [2023-10-08 04:13:47,002][00612] Updated weights for policy 1, policy_version 7340 (0.0008) [2023-10-08 04:13:47,289][00611] Updated weights for policy 0, policy_version 7300 (0.0009) [2023-10-08 04:13:47,371][00612] Updated weights for policy 1, policy_version 7350 (0.0007) [2023-10-08 04:13:47,652][00611] Updated weights for policy 0, policy_version 7310 (0.0009) [2023-10-08 04:13:47,738][00612] Updated weights for policy 1, policy_version 7360 (0.0009) [2023-10-08 04:13:48,027][00611] Updated weights for policy 0, policy_version 7320 (0.0008) [2023-10-08 04:13:48,754][130385] Fps is (10 sec: 19660.2, 60 sec: 15291.6, 300 sec: 14662.3). Total num frames: 15040512. Throughput: 0: 1815.6, 1: 1832.9. Samples: 3765190. Policy #0 lag: (min: 1.0, avg: 7.1, max: 31.0) [2023-10-08 04:13:48,755][130385] Avg episode reward: [(0, '29.720'), (1, '25.870')] [2023-10-08 04:13:51,368][00612] Updated weights for policy 1, policy_version 7370 (0.0007) [2023-10-08 04:13:51,736][00612] Updated weights for policy 1, policy_version 7380 (0.0007) [2023-10-08 04:13:51,813][00611] Updated weights for policy 0, policy_version 7330 (0.0009) [2023-10-08 04:13:52,104][00612] Updated weights for policy 1, policy_version 7390 (0.0009) [2023-10-08 04:13:52,181][00611] Updated weights for policy 0, policy_version 7340 (0.0007) [2023-10-08 04:13:52,551][00611] Updated weights for policy 0, policy_version 7350 (0.0007) [2023-10-08 04:13:52,913][00611] Updated weights for policy 0, policy_version 7360 (0.0010) [2023-10-08 04:13:53,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 15106048. Throughput: 0: 1803.5, 1: 1830.5. Samples: 3777482. Policy #0 lag: (min: 13.0, avg: 16.9, max: 45.0) [2023-10-08 04:13:53,754][130385] Avg episode reward: [(0, '30.570'), (1, '27.060')] [2023-10-08 04:13:53,755][00425] Saving new best policy, reward=27.060! [2023-10-08 04:13:55,801][00612] Updated weights for policy 1, policy_version 7400 (0.0007) [2023-10-08 04:13:56,178][00612] Updated weights for policy 1, policy_version 7410 (0.0008) [2023-10-08 04:13:56,550][00612] Updated weights for policy 1, policy_version 7420 (0.0008) [2023-10-08 04:13:56,737][00611] Updated weights for policy 0, policy_version 7370 (0.0008) [2023-10-08 04:13:57,110][00611] Updated weights for policy 0, policy_version 7380 (0.0007) [2023-10-08 04:13:57,488][00611] Updated weights for policy 0, policy_version 7390 (0.0007) [2023-10-08 04:13:58,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 15171584. Throughput: 0: 1811.9, 1: 1837.4. Samples: 3798044. Policy #0 lag: (min: 13.0, avg: 16.9, max: 45.0) [2023-10-08 04:13:58,754][130385] Avg episode reward: [(0, '30.460'), (1, '28.690')] [2023-10-08 04:13:58,755][00425] Saving new best policy, reward=28.690! [2023-10-08 04:14:00,251][00612] Updated weights for policy 1, policy_version 7430 (0.0008) [2023-10-08 04:14:00,633][00612] Updated weights for policy 1, policy_version 7440 (0.0007) [2023-10-08 04:14:01,004][00612] Updated weights for policy 1, policy_version 7450 (0.0009) [2023-10-08 04:14:01,131][00611] Updated weights for policy 0, policy_version 7400 (0.0008) [2023-10-08 04:14:01,502][00611] Updated weights for policy 0, policy_version 7410 (0.0009) [2023-10-08 04:14:01,879][00611] Updated weights for policy 0, policy_version 7420 (0.0008) [2023-10-08 04:14:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14662.3). Total num frames: 15237120. Throughput: 0: 1800.8, 1: 1838.9. Samples: 3820504. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-10-08 04:14:03,754][130385] Avg episode reward: [(0, '32.150'), (1, '29.320')] [2023-10-08 04:14:03,763][00365] Saving new best policy, reward=32.150! [2023-10-08 04:14:03,763][00425] Saving new best policy, reward=29.320! [2023-10-08 04:14:04,551][00612] Updated weights for policy 1, policy_version 7460 (0.0009) [2023-10-08 04:14:04,922][00612] Updated weights for policy 1, policy_version 7470 (0.0011) [2023-10-08 04:14:05,288][00612] Updated weights for policy 1, policy_version 7480 (0.0009) [2023-10-08 04:14:05,406][00611] Updated weights for policy 0, policy_version 7430 (0.0008) [2023-10-08 04:14:05,783][00611] Updated weights for policy 0, policy_version 7440 (0.0007) [2023-10-08 04:14:06,157][00611] Updated weights for policy 0, policy_version 7450 (0.0007) [2023-10-08 04:14:08,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 15302656. Throughput: 0: 1812.0, 1: 1835.7. Samples: 3830918. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) [2023-10-08 04:14:08,755][130385] Avg episode reward: [(0, '30.850'), (1, '28.370')] [2023-10-08 04:14:09,174][00612] Updated weights for policy 1, policy_version 7490 (0.0008) [2023-10-08 04:14:09,543][00612] Updated weights for policy 1, policy_version 7500 (0.0011) [2023-10-08 04:14:09,734][00611] Updated weights for policy 0, policy_version 7460 (0.0009) [2023-10-08 04:14:09,912][00612] Updated weights for policy 1, policy_version 7510 (0.0009) [2023-10-08 04:14:10,108][00611] Updated weights for policy 0, policy_version 7470 (0.0007) [2023-10-08 04:14:10,282][00612] Updated weights for policy 1, policy_version 7520 (0.0009) [2023-10-08 04:14:10,472][00611] Updated weights for policy 0, policy_version 7480 (0.0010) [2023-10-08 04:14:13,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 15368192. Throughput: 0: 1811.5, 1: 1831.2. Samples: 3853502. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-08 04:14:13,754][130385] Avg episode reward: [(0, '28.870'), (1, '25.930')] [2023-10-08 04:14:13,928][00612] Updated weights for policy 1, policy_version 7530 (0.0008) [2023-10-08 04:14:14,065][00611] Updated weights for policy 0, policy_version 7490 (0.0011) [2023-10-08 04:14:14,293][00612] Updated weights for policy 1, policy_version 7540 (0.0007) [2023-10-08 04:14:14,434][00611] Updated weights for policy 0, policy_version 7500 (0.0009) [2023-10-08 04:14:14,662][00612] Updated weights for policy 1, policy_version 7550 (0.0008) [2023-10-08 04:14:14,806][00611] Updated weights for policy 0, policy_version 7510 (0.0008) [2023-10-08 04:14:15,186][00611] Updated weights for policy 0, policy_version 7520 (0.0008) [2023-10-08 04:14:18,302][00612] Updated weights for policy 1, policy_version 7560 (0.0007) [2023-10-08 04:14:18,668][00612] Updated weights for policy 1, policy_version 7570 (0.0007) [2023-10-08 04:14:18,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 15433728. Throughput: 0: 1818.1, 1: 1830.3. Samples: 3876596. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-08 04:14:18,754][130385] Avg episode reward: [(0, '29.340'), (1, '26.680')] [2023-10-08 04:14:18,960][00611] Updated weights for policy 0, policy_version 7530 (0.0007) [2023-10-08 04:14:19,040][00612] Updated weights for policy 1, policy_version 7580 (0.0007) [2023-10-08 04:14:19,179][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000007584_7766016.pth... [2023-10-08 04:14:19,208][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000005856_5996544.pth [2023-10-08 04:14:19,346][00611] Updated weights for policy 0, policy_version 7540 (0.0008) [2023-10-08 04:14:19,720][00611] Updated weights for policy 0, policy_version 7550 (0.0009) [2023-10-08 04:14:19,796][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000007552_7733248.pth... [2023-10-08 04:14:19,831][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000005824_5963776.pth [2023-10-08 04:14:22,636][00612] Updated weights for policy 1, policy_version 7590 (0.0010) [2023-10-08 04:14:23,013][00612] Updated weights for policy 1, policy_version 7600 (0.0009) [2023-10-08 04:14:23,373][00611] Updated weights for policy 0, policy_version 7560 (0.0008) [2023-10-08 04:14:23,384][00612] Updated weights for policy 1, policy_version 7610 (0.0007) [2023-10-08 04:14:23,752][00611] Updated weights for policy 0, policy_version 7570 (0.0010) [2023-10-08 04:14:23,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 15532032. Throughput: 0: 1818.8, 1: 1836.0. Samples: 3886562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:14:23,755][130385] Avg episode reward: [(0, '26.980'), (1, '28.470')] [2023-10-08 04:14:24,132][00611] Updated weights for policy 0, policy_version 7580 (0.0009) [2023-10-08 04:14:27,033][00612] Updated weights for policy 1, policy_version 7620 (0.0007) [2023-10-08 04:14:27,395][00612] Updated weights for policy 1, policy_version 7630 (0.0007) [2023-10-08 04:14:27,598][00611] Updated weights for policy 0, policy_version 7590 (0.0009) [2023-10-08 04:14:27,769][00612] Updated weights for policy 1, policy_version 7640 (0.0009) [2023-10-08 04:14:28,040][00611] Updated weights for policy 0, policy_version 7602 (0.0008) [2023-10-08 04:14:28,401][00611] Updated weights for policy 0, policy_version 7612 (0.0008) [2023-10-08 04:14:28,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 15630336. Throughput: 0: 1828.7, 1: 1823.4. Samples: 3909492. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 04:14:28,755][130385] Avg episode reward: [(0, '27.920'), (1, '28.210')] [2023-10-08 04:14:31,403][00612] Updated weights for policy 1, policy_version 7650 (0.0008) [2023-10-08 04:14:31,773][00612] Updated weights for policy 1, policy_version 7660 (0.0007) [2023-10-08 04:14:32,130][00611] Updated weights for policy 0, policy_version 7622 (0.0009) [2023-10-08 04:14:32,136][00612] Updated weights for policy 1, policy_version 7670 (0.0007) [2023-10-08 04:14:32,506][00612] Updated weights for policy 1, policy_version 7680 (0.0007) [2023-10-08 04:14:32,508][00611] Updated weights for policy 0, policy_version 7632 (0.0007) [2023-10-08 04:14:32,884][00611] Updated weights for policy 0, policy_version 7642 (0.0007) [2023-10-08 04:14:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 15695872. Throughput: 0: 1827.1, 1: 1830.9. Samples: 3929798. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 04:14:33,754][130385] Avg episode reward: [(0, '26.540'), (1, '30.570')] [2023-10-08 04:14:33,764][00425] Saving new best policy, reward=30.570! [2023-10-08 04:14:36,149][00612] Updated weights for policy 1, policy_version 7690 (0.0009) [2023-10-08 04:14:36,514][00612] Updated weights for policy 1, policy_version 7700 (0.0009) [2023-10-08 04:14:36,559][00611] Updated weights for policy 0, policy_version 7652 (0.0007) [2023-10-08 04:14:36,891][00612] Updated weights for policy 1, policy_version 7710 (0.0008) [2023-10-08 04:14:36,930][00611] Updated weights for policy 0, policy_version 7662 (0.0007) [2023-10-08 04:14:37,304][00611] Updated weights for policy 0, policy_version 7672 (0.0007) [2023-10-08 04:14:38,754][130385] Fps is (10 sec: 13107.5, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 15761408. Throughput: 0: 1838.4, 1: 1821.3. Samples: 3942168. Policy #0 lag: (min: 25.0, avg: 33.2, max: 57.0) [2023-10-08 04:14:38,754][130385] Avg episode reward: [(0, '26.540'), (1, '30.040')] [2023-10-08 04:14:40,608][00612] Updated weights for policy 1, policy_version 7720 (0.0008) [2023-10-08 04:14:40,969][00612] Updated weights for policy 1, policy_version 7730 (0.0009) [2023-10-08 04:14:40,988][00611] Updated weights for policy 0, policy_version 7682 (0.0010) [2023-10-08 04:14:41,339][00612] Updated weights for policy 1, policy_version 7740 (0.0008) [2023-10-08 04:14:41,356][00611] Updated weights for policy 0, policy_version 7692 (0.0008) [2023-10-08 04:14:41,731][00611] Updated weights for policy 0, policy_version 7702 (0.0007) [2023-10-08 04:14:42,102][00611] Updated weights for policy 0, policy_version 7712 (0.0007) [2023-10-08 04:14:43,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 15826944. Throughput: 0: 1829.9, 1: 1826.8. Samples: 3962600. Policy #0 lag: (min: 25.0, avg: 33.2, max: 57.0) [2023-10-08 04:14:43,755][130385] Avg episode reward: [(0, '26.800'), (1, '30.510')] [2023-10-08 04:14:44,946][00612] Updated weights for policy 1, policy_version 7750 (0.0009) [2023-10-08 04:14:45,329][00612] Updated weights for policy 1, policy_version 7760 (0.0008) [2023-10-08 04:14:45,694][00612] Updated weights for policy 1, policy_version 7770 (0.0009) [2023-10-08 04:14:45,725][00611] Updated weights for policy 0, policy_version 7722 (0.0008) [2023-10-08 04:14:46,093][00611] Updated weights for policy 0, policy_version 7732 (0.0009) [2023-10-08 04:14:46,461][00611] Updated weights for policy 0, policy_version 7742 (0.0009) [2023-10-08 04:14:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.6, 300 sec: 14662.3). Total num frames: 15892480. Throughput: 0: 1841.3, 1: 1835.1. Samples: 3985944. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 04:14:48,754][130385] Avg episode reward: [(0, '25.630'), (1, '28.690')] [2023-10-08 04:14:49,267][00612] Updated weights for policy 1, policy_version 7780 (0.0007) [2023-10-08 04:14:49,624][00612] Updated weights for policy 1, policy_version 7790 (0.0007) [2023-10-08 04:14:49,990][00612] Updated weights for policy 1, policy_version 7800 (0.0007) [2023-10-08 04:14:50,101][00611] Updated weights for policy 0, policy_version 7752 (0.0007) [2023-10-08 04:14:50,474][00611] Updated weights for policy 0, policy_version 7762 (0.0007) [2023-10-08 04:14:50,851][00611] Updated weights for policy 0, policy_version 7772 (0.0008) [2023-10-08 04:14:53,623][00612] Updated weights for policy 1, policy_version 7810 (0.0009) [2023-10-08 04:14:53,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 15958016. Throughput: 0: 1828.1, 1: 1838.5. Samples: 3995914. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 04:14:53,754][130385] Avg episode reward: [(0, '24.920'), (1, '27.780')] [2023-10-08 04:14:53,996][00612] Updated weights for policy 1, policy_version 7820 (0.0007) [2023-10-08 04:14:54,360][00612] Updated weights for policy 1, policy_version 7830 (0.0007) [2023-10-08 04:14:54,506][00611] Updated weights for policy 0, policy_version 7782 (0.0009) [2023-10-08 04:14:54,728][00612] Updated weights for policy 1, policy_version 7840 (0.0009) [2023-10-08 04:14:54,871][00611] Updated weights for policy 0, policy_version 7792 (0.0009) [2023-10-08 04:14:55,252][00611] Updated weights for policy 0, policy_version 7802 (0.0010) [2023-10-08 04:14:58,363][00612] Updated weights for policy 1, policy_version 7850 (0.0008) [2023-10-08 04:14:58,736][00612] Updated weights for policy 1, policy_version 7860 (0.0007) [2023-10-08 04:14:58,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 16023552. Throughput: 0: 1828.6, 1: 1846.4. Samples: 4018880. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-08 04:14:58,755][130385] Avg episode reward: [(0, '26.530'), (1, '27.490')] [2023-10-08 04:14:58,856][00611] Updated weights for policy 0, policy_version 7812 (0.0011) [2023-10-08 04:14:59,113][00612] Updated weights for policy 1, policy_version 7870 (0.0009) [2023-10-08 04:14:59,217][00611] Updated weights for policy 0, policy_version 7822 (0.0008) [2023-10-08 04:14:59,592][00611] Updated weights for policy 0, policy_version 7832 (0.0007) [2023-10-08 04:15:02,927][00612] Updated weights for policy 1, policy_version 7880 (0.0007) [2023-10-08 04:15:03,269][00611] Updated weights for policy 0, policy_version 7842 (0.0007) [2023-10-08 04:15:03,288][00612] Updated weights for policy 1, policy_version 7890 (0.0007) [2023-10-08 04:15:03,661][00611] Updated weights for policy 0, policy_version 7852 (0.0008) [2023-10-08 04:15:03,661][00612] Updated weights for policy 1, policy_version 7900 (0.0009) [2023-10-08 04:15:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 16089088. Throughput: 0: 1831.5, 1: 1828.4. Samples: 4041290. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-08 04:15:03,754][130385] Avg episode reward: [(0, '26.550'), (1, '24.550')] [2023-10-08 04:15:04,033][00611] Updated weights for policy 0, policy_version 7862 (0.0009) [2023-10-08 04:15:04,410][00611] Updated weights for policy 0, policy_version 7872 (0.0008) [2023-10-08 04:15:07,358][00612] Updated weights for policy 1, policy_version 7910 (0.0009) [2023-10-08 04:15:07,723][00612] Updated weights for policy 1, policy_version 7920 (0.0010) [2023-10-08 04:15:08,100][00612] Updated weights for policy 1, policy_version 7930 (0.0007) [2023-10-08 04:15:08,117][00611] Updated weights for policy 0, policy_version 7882 (0.0009) [2023-10-08 04:15:08,497][00611] Updated weights for policy 0, policy_version 7892 (0.0009) [2023-10-08 04:15:08,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 16187392. Throughput: 0: 1833.9, 1: 1837.8. Samples: 4051788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:15:08,754][130385] Avg episode reward: [(0, '25.990'), (1, '26.510')] [2023-10-08 04:15:08,870][00611] Updated weights for policy 0, policy_version 7902 (0.0011) [2023-10-08 04:15:11,654][00612] Updated weights for policy 1, policy_version 7940 (0.0008) [2023-10-08 04:15:12,017][00612] Updated weights for policy 1, policy_version 7950 (0.0010) [2023-10-08 04:15:12,385][00612] Updated weights for policy 1, policy_version 7960 (0.0010) [2023-10-08 04:15:12,646][00611] Updated weights for policy 0, policy_version 7912 (0.0007) [2023-10-08 04:15:13,015][00611] Updated weights for policy 0, policy_version 7922 (0.0007) [2023-10-08 04:15:13,384][00611] Updated weights for policy 0, policy_version 7932 (0.0008) [2023-10-08 04:15:13,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 16285696. Throughput: 0: 1823.6, 1: 1829.5. Samples: 4073882. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-08 04:15:13,754][130385] Avg episode reward: [(0, '25.850'), (1, '27.370')] [2023-10-08 04:15:15,882][00612] Updated weights for policy 1, policy_version 7970 (0.0009) [2023-10-08 04:15:16,245][00612] Updated weights for policy 1, policy_version 7980 (0.0010) [2023-10-08 04:15:16,622][00612] Updated weights for policy 1, policy_version 7990 (0.0009) [2023-10-08 04:15:16,987][00612] Updated weights for policy 1, policy_version 8000 (0.0007) [2023-10-08 04:15:16,987][00611] Updated weights for policy 0, policy_version 7942 (0.0008) [2023-10-08 04:15:17,361][00611] Updated weights for policy 0, policy_version 7952 (0.0008) [2023-10-08 04:15:17,726][00611] Updated weights for policy 0, policy_version 7962 (0.0007) [2023-10-08 04:15:18,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 16351232. Throughput: 0: 1821.6, 1: 1844.5. Samples: 4094772. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-08 04:15:18,755][130385] Avg episode reward: [(0, '24.580'), (1, '29.120')] [2023-10-08 04:15:20,549][00612] Updated weights for policy 1, policy_version 8010 (0.0009) [2023-10-08 04:15:20,912][00612] Updated weights for policy 1, policy_version 8020 (0.0009) [2023-10-08 04:15:21,283][00612] Updated weights for policy 1, policy_version 8030 (0.0008) [2023-10-08 04:15:21,523][00611] Updated weights for policy 0, policy_version 7972 (0.0008) [2023-10-08 04:15:21,894][00611] Updated weights for policy 0, policy_version 7982 (0.0007) [2023-10-08 04:15:22,265][00611] Updated weights for policy 0, policy_version 7992 (0.0008) [2023-10-08 04:15:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 16416768. Throughput: 0: 1819.6, 1: 1831.7. Samples: 4106476. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-08 04:15:23,754][130385] Avg episode reward: [(0, '22.520'), (1, '28.980')] [2023-10-08 04:15:24,784][00612] Updated weights for policy 1, policy_version 8040 (0.0008) [2023-10-08 04:15:25,161][00612] Updated weights for policy 1, policy_version 8050 (0.0007) [2023-10-08 04:15:25,530][00612] Updated weights for policy 1, policy_version 8060 (0.0008) [2023-10-08 04:15:25,934][00611] Updated weights for policy 0, policy_version 8002 (0.0008) [2023-10-08 04:15:26,311][00611] Updated weights for policy 0, policy_version 8012 (0.0007) [2023-10-08 04:15:26,687][00611] Updated weights for policy 0, policy_version 8022 (0.0007) [2023-10-08 04:15:27,054][00611] Updated weights for policy 0, policy_version 8032 (0.0008) [2023-10-08 04:15:28,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 16482304. Throughput: 0: 1816.3, 1: 1856.5. Samples: 4127872. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-08 04:15:28,754][130385] Avg episode reward: [(0, '22.200'), (1, '27.430')] [2023-10-08 04:15:29,339][00612] Updated weights for policy 1, policy_version 8070 (0.0010) [2023-10-08 04:15:29,712][00612] Updated weights for policy 1, policy_version 8080 (0.0008) [2023-10-08 04:15:30,089][00612] Updated weights for policy 1, policy_version 8090 (0.0007) [2023-10-08 04:15:30,600][00611] Updated weights for policy 0, policy_version 8042 (0.0009) [2023-10-08 04:15:30,975][00611] Updated weights for policy 0, policy_version 8052 (0.0009) [2023-10-08 04:15:31,358][00611] Updated weights for policy 0, policy_version 8062 (0.0008) [2023-10-08 04:15:33,747][00612] Updated weights for policy 1, policy_version 8100 (0.0007) [2023-10-08 04:15:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 16547840. Throughput: 0: 1814.2, 1: 1853.9. Samples: 4151008. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 04:15:33,754][130385] Avg episode reward: [(0, '23.120'), (1, '28.770')] [2023-10-08 04:15:34,132][00612] Updated weights for policy 1, policy_version 8110 (0.0008) [2023-10-08 04:15:34,506][00612] Updated weights for policy 1, policy_version 8120 (0.0008) [2023-10-08 04:15:35,099][00611] Updated weights for policy 0, policy_version 8072 (0.0010) [2023-10-08 04:15:35,466][00611] Updated weights for policy 0, policy_version 8082 (0.0008) [2023-10-08 04:15:35,843][00611] Updated weights for policy 0, policy_version 8092 (0.0010) [2023-10-08 04:15:38,094][00612] Updated weights for policy 1, policy_version 8130 (0.0008) [2023-10-08 04:15:38,471][00612] Updated weights for policy 1, policy_version 8140 (0.0008) [2023-10-08 04:15:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 16613376. Throughput: 0: 1814.8, 1: 1848.2. Samples: 4160748. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 04:15:38,754][130385] Avg episode reward: [(0, '24.400'), (1, '28.020')] [2023-10-08 04:15:38,839][00612] Updated weights for policy 1, policy_version 8150 (0.0009) [2023-10-08 04:15:39,197][00612] Updated weights for policy 1, policy_version 8160 (0.0009) [2023-10-08 04:15:39,453][00611] Updated weights for policy 0, policy_version 8102 (0.0008) [2023-10-08 04:15:39,833][00611] Updated weights for policy 0, policy_version 8112 (0.0011) [2023-10-08 04:15:40,206][00611] Updated weights for policy 0, policy_version 8122 (0.0009) [2023-10-08 04:15:42,790][00612] Updated weights for policy 1, policy_version 8170 (0.0009) [2023-10-08 04:15:43,152][00612] Updated weights for policy 1, policy_version 8180 (0.0007) [2023-10-08 04:15:43,519][00612] Updated weights for policy 1, policy_version 8190 (0.0007) [2023-10-08 04:15:43,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 16711680. Throughput: 0: 1813.6, 1: 1844.9. Samples: 4183512. Policy #0 lag: (min: 2.0, avg: 9.8, max: 34.0) [2023-10-08 04:15:43,755][130385] Avg episode reward: [(0, '25.580'), (1, '28.240')] [2023-10-08 04:15:43,931][00611] Updated weights for policy 0, policy_version 8132 (0.0011) [2023-10-08 04:15:44,304][00611] Updated weights for policy 0, policy_version 8142 (0.0008) [2023-10-08 04:15:44,686][00611] Updated weights for policy 0, policy_version 8152 (0.0009) [2023-10-08 04:15:47,216][00612] Updated weights for policy 1, policy_version 8200 (0.0007) [2023-10-08 04:15:47,583][00612] Updated weights for policy 1, policy_version 8210 (0.0010) [2023-10-08 04:15:47,956][00612] Updated weights for policy 1, policy_version 8220 (0.0012) [2023-10-08 04:15:48,604][00611] Updated weights for policy 0, policy_version 8162 (0.0008) [2023-10-08 04:15:48,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 16777216. Throughput: 0: 1804.9, 1: 1829.8. Samples: 4204850. Policy #0 lag: (min: 2.0, avg: 9.8, max: 34.0) [2023-10-08 04:15:48,754][130385] Avg episode reward: [(0, '24.540'), (1, '29.010')] [2023-10-08 04:15:48,998][00611] Updated weights for policy 0, policy_version 8172 (0.0010) [2023-10-08 04:15:49,367][00611] Updated weights for policy 0, policy_version 8182 (0.0007) [2023-10-08 04:15:49,739][00611] Updated weights for policy 0, policy_version 8192 (0.0007) [2023-10-08 04:15:51,580][00612] Updated weights for policy 1, policy_version 8230 (0.0010) [2023-10-08 04:15:51,950][00612] Updated weights for policy 1, policy_version 8240 (0.0009) [2023-10-08 04:15:52,309][00612] Updated weights for policy 1, policy_version 8250 (0.0008) [2023-10-08 04:15:53,426][00611] Updated weights for policy 0, policy_version 8202 (0.0009) [2023-10-08 04:15:53,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 16842752. Throughput: 0: 1803.6, 1: 1852.2. Samples: 4216300. Policy #0 lag: (min: 15.0, avg: 19.8, max: 47.0) [2023-10-08 04:15:53,754][130385] Avg episode reward: [(0, '25.730'), (1, '29.280')] [2023-10-08 04:15:53,798][00611] Updated weights for policy 0, policy_version 8212 (0.0009) [2023-10-08 04:15:54,175][00611] Updated weights for policy 0, policy_version 8222 (0.0007) [2023-10-08 04:15:55,785][00612] Updated weights for policy 1, policy_version 8260 (0.0009) [2023-10-08 04:15:56,159][00612] Updated weights for policy 1, policy_version 8270 (0.0008) [2023-10-08 04:15:56,527][00612] Updated weights for policy 1, policy_version 8280 (0.0009) [2023-10-08 04:15:57,899][00611] Updated weights for policy 0, policy_version 8232 (0.0007) [2023-10-08 04:15:58,268][00611] Updated weights for policy 0, policy_version 8242 (0.0007) [2023-10-08 04:15:58,634][00611] Updated weights for policy 0, policy_version 8252 (0.0008) [2023-10-08 04:15:58,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 16908288. Throughput: 0: 1807.4, 1: 1837.2. Samples: 4237890. Policy #0 lag: (min: 15.0, avg: 19.8, max: 47.0) [2023-10-08 04:15:58,755][130385] Avg episode reward: [(0, '25.020'), (1, '31.180')] [2023-10-08 04:15:58,756][00425] Saving new best policy, reward=31.180! [2023-10-08 04:16:00,242][00612] Updated weights for policy 1, policy_version 8290 (0.0010) [2023-10-08 04:16:00,614][00612] Updated weights for policy 1, policy_version 8300 (0.0011) [2023-10-08 04:16:00,993][00612] Updated weights for policy 1, policy_version 8310 (0.0009) [2023-10-08 04:16:01,357][00612] Updated weights for policy 1, policy_version 8320 (0.0008) [2023-10-08 04:16:02,208][00611] Updated weights for policy 0, policy_version 8262 (0.0009) [2023-10-08 04:16:02,576][00611] Updated weights for policy 0, policy_version 8272 (0.0007) [2023-10-08 04:16:02,948][00611] Updated weights for policy 0, policy_version 8282 (0.0008) [2023-10-08 04:16:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 17006592. Throughput: 0: 1818.0, 1: 1848.2. Samples: 4259748. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 04:16:03,754][130385] Avg episode reward: [(0, '25.360'), (1, '29.160')] [2023-10-08 04:16:04,978][00612] Updated weights for policy 1, policy_version 8330 (0.0010) [2023-10-08 04:16:05,348][00612] Updated weights for policy 1, policy_version 8340 (0.0009) [2023-10-08 04:16:05,718][00612] Updated weights for policy 1, policy_version 8350 (0.0009) [2023-10-08 04:16:06,501][00611] Updated weights for policy 0, policy_version 8292 (0.0007) [2023-10-08 04:16:06,869][00611] Updated weights for policy 0, policy_version 8302 (0.0008) [2023-10-08 04:16:07,240][00611] Updated weights for policy 0, policy_version 8312 (0.0009) [2023-10-08 04:16:08,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17072128. Throughput: 0: 1822.0, 1: 1839.0. Samples: 4271220. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-08 04:16:08,754][130385] Avg episode reward: [(0, '24.860'), (1, '28.630')] [2023-10-08 04:16:09,397][00612] Updated weights for policy 1, policy_version 8360 (0.0007) [2023-10-08 04:16:09,770][00612] Updated weights for policy 1, policy_version 8370 (0.0007) [2023-10-08 04:16:10,134][00612] Updated weights for policy 1, policy_version 8380 (0.0007) [2023-10-08 04:16:10,808][00611] Updated weights for policy 0, policy_version 8322 (0.0010) [2023-10-08 04:16:11,172][00611] Updated weights for policy 0, policy_version 8332 (0.0010) [2023-10-08 04:16:11,546][00611] Updated weights for policy 0, policy_version 8342 (0.0010) [2023-10-08 04:16:11,920][00611] Updated weights for policy 0, policy_version 8352 (0.0007) [2023-10-08 04:16:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 17137664. Throughput: 0: 1825.3, 1: 1841.0. Samples: 4292856. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-08 04:16:13,754][130385] Avg episode reward: [(0, '28.200'), (1, '28.070')] [2023-10-08 04:16:13,805][00612] Updated weights for policy 1, policy_version 8390 (0.0010) [2023-10-08 04:16:14,165][00612] Updated weights for policy 1, policy_version 8400 (0.0007) [2023-10-08 04:16:14,539][00612] Updated weights for policy 1, policy_version 8410 (0.0007) [2023-10-08 04:16:15,584][00611] Updated weights for policy 0, policy_version 8362 (0.0010) [2023-10-08 04:16:15,967][00611] Updated weights for policy 0, policy_version 8372 (0.0007) [2023-10-08 04:16:16,354][00611] Updated weights for policy 0, policy_version 8382 (0.0010) [2023-10-08 04:16:18,237][00612] Updated weights for policy 1, policy_version 8420 (0.0009) [2023-10-08 04:16:18,619][00612] Updated weights for policy 1, policy_version 8430 (0.0009) [2023-10-08 04:16:18,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 17203200. Throughput: 0: 1824.7, 1: 1834.6. Samples: 4315676. Policy #0 lag: (min: 31.0, avg: 39.6, max: 63.0) [2023-10-08 04:16:18,754][130385] Avg episode reward: [(0, '28.580'), (1, '28.900')] [2023-10-08 04:16:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000008384_8585216.pth... [2023-10-08 04:16:18,798][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000006688_6848512.pth [2023-10-08 04:16:18,802][00365] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p0/milestones/checkpoint_000008384_8585216.pth [2023-10-08 04:16:18,988][00612] Updated weights for policy 1, policy_version 8440 (0.0010) [2023-10-08 04:16:19,282][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000008448_8650752.pth... [2023-10-08 04:16:19,310][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000006720_6881280.pth [2023-10-08 04:16:19,314][00425] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p1/milestones/checkpoint_000008448_8650752.pth [2023-10-08 04:16:19,855][00611] Updated weights for policy 0, policy_version 8392 (0.0011) [2023-10-08 04:16:20,222][00611] Updated weights for policy 0, policy_version 8402 (0.0009) [2023-10-08 04:16:20,594][00611] Updated weights for policy 0, policy_version 8412 (0.0009) [2023-10-08 04:16:22,702][00612] Updated weights for policy 1, policy_version 8450 (0.0009) [2023-10-08 04:16:23,112][00612] Updated weights for policy 1, policy_version 8460 (0.0009) [2023-10-08 04:16:23,494][00612] Updated weights for policy 1, policy_version 8470 (0.0009) [2023-10-08 04:16:23,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 17268736. Throughput: 0: 1827.7, 1: 1844.1. Samples: 4325980. Policy #0 lag: (min: 31.0, avg: 39.6, max: 63.0) [2023-10-08 04:16:23,755][130385] Avg episode reward: [(0, '29.670'), (1, '27.060')] [2023-10-08 04:16:23,854][00612] Updated weights for policy 1, policy_version 8480 (0.0007) [2023-10-08 04:16:24,239][00611] Updated weights for policy 0, policy_version 8422 (0.0008) [2023-10-08 04:16:24,621][00611] Updated weights for policy 0, policy_version 8432 (0.0010) [2023-10-08 04:16:24,996][00611] Updated weights for policy 0, policy_version 8442 (0.0008) [2023-10-08 04:16:27,529][00612] Updated weights for policy 1, policy_version 8490 (0.0007) [2023-10-08 04:16:27,895][00612] Updated weights for policy 1, policy_version 8500 (0.0008) [2023-10-08 04:16:28,272][00612] Updated weights for policy 1, policy_version 8510 (0.0009) [2023-10-08 04:16:28,593][00611] Updated weights for policy 0, policy_version 8452 (0.0008) [2023-10-08 04:16:28,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 17367040. Throughput: 0: 1831.2, 1: 1835.6. Samples: 4348514. Policy #0 lag: (min: 21.0, avg: 24.1, max: 53.0) [2023-10-08 04:16:28,755][130385] Avg episode reward: [(0, '26.310'), (1, '27.700')] [2023-10-08 04:16:28,968][00611] Updated weights for policy 0, policy_version 8462 (0.0010) [2023-10-08 04:16:29,345][00611] Updated weights for policy 0, policy_version 8472 (0.0008) [2023-10-08 04:16:32,056][00612] Updated weights for policy 1, policy_version 8520 (0.0009) [2023-10-08 04:16:32,422][00612] Updated weights for policy 1, policy_version 8530 (0.0008) [2023-10-08 04:16:32,790][00612] Updated weights for policy 1, policy_version 8540 (0.0009) [2023-10-08 04:16:33,025][00611] Updated weights for policy 0, policy_version 8482 (0.0010) [2023-10-08 04:16:33,417][00611] Updated weights for policy 0, policy_version 8492 (0.0009) [2023-10-08 04:16:33,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 17432576. Throughput: 0: 1828.7, 1: 1831.0. Samples: 4369536. Policy #0 lag: (min: 21.0, avg: 24.1, max: 53.0) [2023-10-08 04:16:33,754][130385] Avg episode reward: [(0, '25.540'), (1, '25.980')] [2023-10-08 04:16:33,791][00611] Updated weights for policy 0, policy_version 8502 (0.0008) [2023-10-08 04:16:34,171][00611] Updated weights for policy 0, policy_version 8512 (0.0008) [2023-10-08 04:16:36,327][00612] Updated weights for policy 1, policy_version 8550 (0.0008) [2023-10-08 04:16:36,694][00612] Updated weights for policy 1, policy_version 8560 (0.0010) [2023-10-08 04:16:37,064][00612] Updated weights for policy 1, policy_version 8570 (0.0011) [2023-10-08 04:16:37,907][00611] Updated weights for policy 0, policy_version 8522 (0.0008) [2023-10-08 04:16:38,275][00611] Updated weights for policy 0, policy_version 8532 (0.0010) [2023-10-08 04:16:38,658][00611] Updated weights for policy 0, policy_version 8542 (0.0010) [2023-10-08 04:16:38,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.6, 300 sec: 14662.3). Total num frames: 17530880. Throughput: 0: 1835.2, 1: 1828.1. Samples: 4381150. Policy #0 lag: (min: 26.0, avg: 29.6, max: 58.0) [2023-10-08 04:16:38,755][130385] Avg episode reward: [(0, '25.660'), (1, '26.230')] [2023-10-08 04:16:40,724][00612] Updated weights for policy 1, policy_version 8580 (0.0009) [2023-10-08 04:16:41,085][00612] Updated weights for policy 1, policy_version 8590 (0.0007) [2023-10-08 04:16:41,458][00612] Updated weights for policy 1, policy_version 8600 (0.0008) [2023-10-08 04:16:42,318][00611] Updated weights for policy 0, policy_version 8552 (0.0008) [2023-10-08 04:16:42,697][00611] Updated weights for policy 0, policy_version 8562 (0.0009) [2023-10-08 04:16:43,071][00611] Updated weights for policy 0, policy_version 8572 (0.0007) [2023-10-08 04:16:43,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17596416. Throughput: 0: 1833.6, 1: 1828.1. Samples: 4402664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:16:43,757][130385] Avg episode reward: [(0, '24.050'), (1, '24.830')] [2023-10-08 04:16:45,031][00612] Updated weights for policy 1, policy_version 8610 (0.0008) [2023-10-08 04:16:45,397][00612] Updated weights for policy 1, policy_version 8620 (0.0009) [2023-10-08 04:16:45,768][00612] Updated weights for policy 1, policy_version 8630 (0.0010) [2023-10-08 04:16:46,141][00612] Updated weights for policy 1, policy_version 8640 (0.0007) [2023-10-08 04:16:46,766][00611] Updated weights for policy 0, policy_version 8582 (0.0008) [2023-10-08 04:16:47,145][00611] Updated weights for policy 0, policy_version 8592 (0.0009) [2023-10-08 04:16:47,515][00611] Updated weights for policy 0, policy_version 8602 (0.0008) [2023-10-08 04:16:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 17661952. Throughput: 0: 1824.4, 1: 1829.4. Samples: 4424172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:16:48,756][130385] Avg episode reward: [(0, '26.590'), (1, '23.560')] [2023-10-08 04:16:49,886][00612] Updated weights for policy 1, policy_version 8650 (0.0007) [2023-10-08 04:16:50,252][00612] Updated weights for policy 1, policy_version 8660 (0.0007) [2023-10-08 04:16:50,626][00612] Updated weights for policy 1, policy_version 8670 (0.0007) [2023-10-08 04:16:51,165][00611] Updated weights for policy 0, policy_version 8612 (0.0008) [2023-10-08 04:16:51,537][00611] Updated weights for policy 0, policy_version 8622 (0.0007) [2023-10-08 04:16:51,906][00611] Updated weights for policy 0, policy_version 8632 (0.0009) [2023-10-08 04:16:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17727488. Throughput: 0: 1825.6, 1: 1829.8. Samples: 4435710. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-08 04:16:53,755][130385] Avg episode reward: [(0, '29.690'), (1, '23.340')] [2023-10-08 04:16:54,397][00612] Updated weights for policy 1, policy_version 8680 (0.0008) [2023-10-08 04:16:54,770][00612] Updated weights for policy 1, policy_version 8690 (0.0008) [2023-10-08 04:16:55,143][00612] Updated weights for policy 1, policy_version 8700 (0.0007) [2023-10-08 04:16:55,833][00611] Updated weights for policy 0, policy_version 8642 (0.0008) [2023-10-08 04:16:56,196][00611] Updated weights for policy 0, policy_version 8652 (0.0009) [2023-10-08 04:16:56,571][00611] Updated weights for policy 0, policy_version 8662 (0.0009) [2023-10-08 04:16:56,937][00611] Updated weights for policy 0, policy_version 8672 (0.0007) [2023-10-08 04:16:58,754][130385] Fps is (10 sec: 13107.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17793024. Throughput: 0: 1820.4, 1: 1826.0. Samples: 4456948. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-08 04:16:58,754][130385] Avg episode reward: [(0, '29.520'), (1, '23.860')] [2023-10-08 04:16:58,825][00612] Updated weights for policy 1, policy_version 8710 (0.0011) [2023-10-08 04:16:59,188][00612] Updated weights for policy 1, policy_version 8720 (0.0008) [2023-10-08 04:16:59,564][00612] Updated weights for policy 1, policy_version 8730 (0.0008) [2023-10-08 04:17:00,571][00611] Updated weights for policy 0, policy_version 8682 (0.0008) [2023-10-08 04:17:00,955][00611] Updated weights for policy 0, policy_version 8692 (0.0008) [2023-10-08 04:17:01,326][00611] Updated weights for policy 0, policy_version 8702 (0.0007) [2023-10-08 04:17:03,134][00612] Updated weights for policy 1, policy_version 8740 (0.0008) [2023-10-08 04:17:03,508][00612] Updated weights for policy 1, policy_version 8750 (0.0008) [2023-10-08 04:17:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 17858560. Throughput: 0: 1822.5, 1: 1828.8. Samples: 4479984. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 04:17:03,754][130385] Avg episode reward: [(0, '28.450'), (1, '24.110')] [2023-10-08 04:17:03,880][00612] Updated weights for policy 1, policy_version 8760 (0.0008) [2023-10-08 04:17:05,088][00611] Updated weights for policy 0, policy_version 8712 (0.0008) [2023-10-08 04:17:05,473][00611] Updated weights for policy 0, policy_version 8722 (0.0009) [2023-10-08 04:17:05,832][00611] Updated weights for policy 0, policy_version 8732 (0.0012) [2023-10-08 04:17:07,578][00612] Updated weights for policy 1, policy_version 8770 (0.0008) [2023-10-08 04:17:07,992][00612] Updated weights for policy 1, policy_version 8780 (0.0009) [2023-10-08 04:17:08,361][00612] Updated weights for policy 1, policy_version 8790 (0.0010) [2023-10-08 04:17:08,731][00612] Updated weights for policy 1, policy_version 8800 (0.0009) [2023-10-08 04:17:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17956864. Throughput: 0: 1823.2, 1: 1828.2. Samples: 4490292. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 04:17:08,754][130385] Avg episode reward: [(0, '29.080'), (1, '23.710')] [2023-10-08 04:17:09,575][00611] Updated weights for policy 0, policy_version 8742 (0.0009) [2023-10-08 04:17:09,949][00611] Updated weights for policy 0, policy_version 8752 (0.0008) [2023-10-08 04:17:10,315][00611] Updated weights for policy 0, policy_version 8762 (0.0009) [2023-10-08 04:17:12,385][00612] Updated weights for policy 1, policy_version 8810 (0.0008) [2023-10-08 04:17:12,750][00612] Updated weights for policy 1, policy_version 8820 (0.0008) [2023-10-08 04:17:13,120][00612] Updated weights for policy 1, policy_version 8830 (0.0007) [2023-10-08 04:17:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 18022400. Throughput: 0: 1820.0, 1: 1830.8. Samples: 4512800. Policy #0 lag: (min: 9.0, avg: 21.7, max: 41.0) [2023-10-08 04:17:13,755][130385] Avg episode reward: [(0, '30.180'), (1, '25.820')] [2023-10-08 04:17:13,970][00611] Updated weights for policy 0, policy_version 8772 (0.0007) [2023-10-08 04:17:14,335][00611] Updated weights for policy 0, policy_version 8782 (0.0007) [2023-10-08 04:17:14,711][00611] Updated weights for policy 0, policy_version 8792 (0.0008) [2023-10-08 04:17:16,741][00612] Updated weights for policy 1, policy_version 8840 (0.0009) [2023-10-08 04:17:17,122][00612] Updated weights for policy 1, policy_version 8850 (0.0008) [2023-10-08 04:17:17,489][00612] Updated weights for policy 1, policy_version 8860 (0.0008) [2023-10-08 04:17:18,349][00611] Updated weights for policy 0, policy_version 8802 (0.0008) [2023-10-08 04:17:18,744][00611] Updated weights for policy 0, policy_version 8812 (0.0007) [2023-10-08 04:17:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 18087936. Throughput: 0: 1828.7, 1: 1841.9. Samples: 4534716. Policy #0 lag: (min: 9.0, avg: 21.7, max: 41.0) [2023-10-08 04:17:18,755][130385] Avg episode reward: [(0, '26.500'), (1, '25.530')] [2023-10-08 04:17:19,113][00611] Updated weights for policy 0, policy_version 8822 (0.0008) [2023-10-08 04:17:19,490][00611] Updated weights for policy 0, policy_version 8832 (0.0010) [2023-10-08 04:17:21,215][00612] Updated weights for policy 1, policy_version 8870 (0.0009) [2023-10-08 04:17:21,587][00612] Updated weights for policy 1, policy_version 8880 (0.0009) [2023-10-08 04:17:21,963][00612] Updated weights for policy 1, policy_version 8890 (0.0009) [2023-10-08 04:17:23,135][00611] Updated weights for policy 0, policy_version 8842 (0.0010) [2023-10-08 04:17:23,515][00611] Updated weights for policy 0, policy_version 8852 (0.0009) [2023-10-08 04:17:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 18153472. Throughput: 0: 1822.2, 1: 1836.8. Samples: 4545804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:17:23,755][130385] Avg episode reward: [(0, '27.710'), (1, '26.580')] [2023-10-08 04:17:23,888][00611] Updated weights for policy 0, policy_version 8862 (0.0010) [2023-10-08 04:17:25,587][00612] Updated weights for policy 1, policy_version 8900 (0.0008) [2023-10-08 04:17:25,963][00612] Updated weights for policy 1, policy_version 8910 (0.0007) [2023-10-08 04:17:26,326][00612] Updated weights for policy 1, policy_version 8920 (0.0007) [2023-10-08 04:17:27,464][00611] Updated weights for policy 0, policy_version 8872 (0.0009) [2023-10-08 04:17:27,835][00611] Updated weights for policy 0, policy_version 8882 (0.0007) [2023-10-08 04:17:28,215][00611] Updated weights for policy 0, policy_version 8892 (0.0008) [2023-10-08 04:17:28,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 18251776. Throughput: 0: 1830.1, 1: 1838.7. Samples: 4567760. Policy #0 lag: (min: 17.0, avg: 28.2, max: 49.0) [2023-10-08 04:17:28,755][130385] Avg episode reward: [(0, '27.340'), (1, '27.290')] [2023-10-08 04:17:29,794][00612] Updated weights for policy 1, policy_version 8930 (0.0009) [2023-10-08 04:17:30,152][00612] Updated weights for policy 1, policy_version 8940 (0.0007) [2023-10-08 04:17:30,520][00612] Updated weights for policy 1, policy_version 8950 (0.0008) [2023-10-08 04:17:30,883][00612] Updated weights for policy 1, policy_version 8960 (0.0009) [2023-10-08 04:17:31,884][00611] Updated weights for policy 0, policy_version 8902 (0.0008) [2023-10-08 04:17:32,252][00611] Updated weights for policy 0, policy_version 8912 (0.0008) [2023-10-08 04:17:32,630][00611] Updated weights for policy 0, policy_version 8922 (0.0009) [2023-10-08 04:17:33,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 18317312. Throughput: 0: 1832.4, 1: 1846.1. Samples: 4589702. Policy #0 lag: (min: 17.0, avg: 28.2, max: 49.0) [2023-10-08 04:17:33,755][130385] Avg episode reward: [(0, '28.170'), (1, '27.270')] [2023-10-08 04:17:34,545][00612] Updated weights for policy 1, policy_version 8970 (0.0007) [2023-10-08 04:17:34,908][00612] Updated weights for policy 1, policy_version 8980 (0.0009) [2023-10-08 04:17:35,283][00612] Updated weights for policy 1, policy_version 8990 (0.0007) [2023-10-08 04:17:36,216][00611] Updated weights for policy 0, policy_version 8932 (0.0009) [2023-10-08 04:17:36,579][00611] Updated weights for policy 0, policy_version 8942 (0.0008) [2023-10-08 04:17:36,961][00611] Updated weights for policy 0, policy_version 8952 (0.0007) [2023-10-08 04:17:38,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 18382848. Throughput: 0: 1828.1, 1: 1843.9. Samples: 4600950. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:17:38,754][130385] Avg episode reward: [(0, '28.710'), (1, '27.680')] [2023-10-08 04:17:38,962][00612] Updated weights for policy 1, policy_version 9000 (0.0008) [2023-10-08 04:17:39,328][00612] Updated weights for policy 1, policy_version 9010 (0.0007) [2023-10-08 04:17:39,685][00612] Updated weights for policy 1, policy_version 9020 (0.0007) [2023-10-08 04:17:40,700][00611] Updated weights for policy 0, policy_version 8962 (0.0008) [2023-10-08 04:17:41,077][00611] Updated weights for policy 0, policy_version 8972 (0.0010) [2023-10-08 04:17:41,443][00611] Updated weights for policy 0, policy_version 8982 (0.0008) [2023-10-08 04:17:41,812][00611] Updated weights for policy 0, policy_version 8992 (0.0009) [2023-10-08 04:17:43,327][00612] Updated weights for policy 1, policy_version 9030 (0.0007) [2023-10-08 04:17:43,695][00612] Updated weights for policy 1, policy_version 9040 (0.0007) [2023-10-08 04:17:43,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 18448384. Throughput: 0: 1828.8, 1: 1842.7. Samples: 4622166. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:17:43,754][130385] Avg episode reward: [(0, '27.240'), (1, '28.490')] [2023-10-08 04:17:44,061][00612] Updated weights for policy 1, policy_version 9050 (0.0008) [2023-10-08 04:17:45,424][00611] Updated weights for policy 0, policy_version 9002 (0.0008) [2023-10-08 04:17:45,789][00611] Updated weights for policy 0, policy_version 9012 (0.0008) [2023-10-08 04:17:46,164][00611] Updated weights for policy 0, policy_version 9022 (0.0009) [2023-10-08 04:17:47,677][00612] Updated weights for policy 1, policy_version 9060 (0.0009) [2023-10-08 04:17:48,045][00612] Updated weights for policy 1, policy_version 9070 (0.0009) [2023-10-08 04:17:48,410][00612] Updated weights for policy 1, policy_version 9080 (0.0010) [2023-10-08 04:17:48,754][130385] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 18546688. Throughput: 0: 1830.8, 1: 1828.7. Samples: 4644662. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 04:17:48,755][130385] Avg episode reward: [(0, '28.800'), (1, '29.090')] [2023-10-08 04:17:49,908][00611] Updated weights for policy 0, policy_version 9032 (0.0011) [2023-10-08 04:17:50,283][00611] Updated weights for policy 0, policy_version 9042 (0.0009) [2023-10-08 04:17:50,652][00611] Updated weights for policy 0, policy_version 9052 (0.0011) [2023-10-08 04:17:52,081][00612] Updated weights for policy 1, policy_version 9090 (0.0009) [2023-10-08 04:17:52,464][00612] Updated weights for policy 1, policy_version 9100 (0.0007) [2023-10-08 04:17:52,835][00612] Updated weights for policy 1, policy_version 9110 (0.0007) [2023-10-08 04:17:53,212][00612] Updated weights for policy 1, policy_version 9120 (0.0007) [2023-10-08 04:17:53,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 18612224. Throughput: 0: 1826.8, 1: 1841.8. Samples: 4655380. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 04:17:53,755][130385] Avg episode reward: [(0, '29.830'), (1, '28.820')] [2023-10-08 04:17:54,251][00611] Updated weights for policy 0, policy_version 9062 (0.0008) [2023-10-08 04:17:54,620][00611] Updated weights for policy 0, policy_version 9072 (0.0008) [2023-10-08 04:17:54,996][00611] Updated weights for policy 0, policy_version 9082 (0.0008) [2023-10-08 04:17:56,879][00612] Updated weights for policy 1, policy_version 9130 (0.0010) [2023-10-08 04:17:57,251][00612] Updated weights for policy 1, policy_version 9140 (0.0008) [2023-10-08 04:17:57,613][00612] Updated weights for policy 1, policy_version 9150 (0.0009) [2023-10-08 04:17:58,573][00611] Updated weights for policy 0, policy_version 9092 (0.0008) [2023-10-08 04:17:58,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 18677760. Throughput: 0: 1838.5, 1: 1827.7. Samples: 4677780. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-08 04:17:58,754][130385] Avg episode reward: [(0, '30.160'), (1, '29.070')] [2023-10-08 04:17:58,935][00611] Updated weights for policy 0, policy_version 9102 (0.0008) [2023-10-08 04:17:59,305][00611] Updated weights for policy 0, policy_version 9112 (0.0008) [2023-10-08 04:18:01,139][00612] Updated weights for policy 1, policy_version 9160 (0.0011) [2023-10-08 04:18:01,506][00612] Updated weights for policy 1, policy_version 9170 (0.0011) [2023-10-08 04:18:01,878][00612] Updated weights for policy 1, policy_version 9180 (0.0009) [2023-10-08 04:18:02,943][00611] Updated weights for policy 0, policy_version 9122 (0.0007) [2023-10-08 04:18:03,330][00611] Updated weights for policy 0, policy_version 9132 (0.0009) [2023-10-08 04:18:03,692][00611] Updated weights for policy 0, policy_version 9142 (0.0009) [2023-10-08 04:18:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 18743296. Throughput: 0: 1832.1, 1: 1842.1. Samples: 4700054. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-08 04:18:03,754][130385] Avg episode reward: [(0, '30.210'), (1, '27.820')] [2023-10-08 04:18:04,070][00611] Updated weights for policy 0, policy_version 9152 (0.0009) [2023-10-08 04:18:05,543][00612] Updated weights for policy 1, policy_version 9190 (0.0010) [2023-10-08 04:18:05,906][00612] Updated weights for policy 1, policy_version 9200 (0.0008) [2023-10-08 04:18:06,286][00612] Updated weights for policy 1, policy_version 9210 (0.0008) [2023-10-08 04:18:07,740][00611] Updated weights for policy 0, policy_version 9162 (0.0008) [2023-10-08 04:18:08,110][00611] Updated weights for policy 0, policy_version 9172 (0.0009) [2023-10-08 04:18:08,482][00611] Updated weights for policy 0, policy_version 9182 (0.0008) [2023-10-08 04:18:08,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 18841600. Throughput: 0: 1842.8, 1: 1824.5. Samples: 4710832. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-08 04:18:08,755][130385] Avg episode reward: [(0, '29.120'), (1, '26.690')] [2023-10-08 04:18:10,010][00612] Updated weights for policy 1, policy_version 9220 (0.0010) [2023-10-08 04:18:10,383][00612] Updated weights for policy 1, policy_version 9230 (0.0011) [2023-10-08 04:18:10,748][00612] Updated weights for policy 1, policy_version 9240 (0.0011) [2023-10-08 04:18:12,136][00611] Updated weights for policy 0, policy_version 9192 (0.0007) [2023-10-08 04:18:12,508][00611] Updated weights for policy 0, policy_version 9202 (0.0009) [2023-10-08 04:18:12,874][00611] Updated weights for policy 0, policy_version 9212 (0.0008) [2023-10-08 04:18:13,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 18907136. Throughput: 0: 1830.0, 1: 1843.6. Samples: 4733072. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:18:13,755][130385] Avg episode reward: [(0, '29.410'), (1, '27.900')] [2023-10-08 04:18:14,365][00612] Updated weights for policy 1, policy_version 9250 (0.0011) [2023-10-08 04:18:14,741][00612] Updated weights for policy 1, policy_version 9260 (0.0008) [2023-10-08 04:18:15,109][00612] Updated weights for policy 1, policy_version 9270 (0.0007) [2023-10-08 04:18:15,481][00612] Updated weights for policy 1, policy_version 9280 (0.0010) [2023-10-08 04:18:16,413][00611] Updated weights for policy 0, policy_version 9222 (0.0011) [2023-10-08 04:18:16,790][00611] Updated weights for policy 0, policy_version 9232 (0.0007) [2023-10-08 04:18:17,150][00611] Updated weights for policy 0, policy_version 9242 (0.0009) [2023-10-08 04:18:18,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 18972672. Throughput: 0: 1841.8, 1: 1834.4. Samples: 4755130. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:18:18,754][130385] Avg episode reward: [(0, '29.370'), (1, '28.060')] [2023-10-08 04:18:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000009248_9469952.pth... [2023-10-08 04:18:18,797][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000007552_7733248.pth [2023-10-08 04:18:19,037][00612] Updated weights for policy 1, policy_version 9290 (0.0008) [2023-10-08 04:18:19,401][00612] Updated weights for policy 1, policy_version 9300 (0.0007) [2023-10-08 04:18:19,767][00612] Updated weights for policy 1, policy_version 9310 (0.0010) [2023-10-08 04:18:19,841][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000009312_9535488.pth... [2023-10-08 04:18:19,879][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000007584_7766016.pth [2023-10-08 04:18:20,723][00611] Updated weights for policy 0, policy_version 9252 (0.0008) [2023-10-08 04:18:21,096][00611] Updated weights for policy 0, policy_version 9262 (0.0008) [2023-10-08 04:18:21,460][00611] Updated weights for policy 0, policy_version 9272 (0.0007) [2023-10-08 04:18:23,514][00612] Updated weights for policy 1, policy_version 9320 (0.0008) [2023-10-08 04:18:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19038208. Throughput: 0: 1831.6, 1: 1837.8. Samples: 4766072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:18:23,755][130385] Avg episode reward: [(0, '28.850'), (1, '27.720')] [2023-10-08 04:18:23,882][00612] Updated weights for policy 1, policy_version 9330 (0.0009) [2023-10-08 04:18:24,265][00612] Updated weights for policy 1, policy_version 9340 (0.0011) [2023-10-08 04:18:25,112][00611] Updated weights for policy 0, policy_version 9282 (0.0007) [2023-10-08 04:18:25,481][00611] Updated weights for policy 0, policy_version 9292 (0.0009) [2023-10-08 04:18:25,869][00611] Updated weights for policy 0, policy_version 9302 (0.0009) [2023-10-08 04:18:26,237][00611] Updated weights for policy 0, policy_version 9312 (0.0008) [2023-10-08 04:18:27,846][00612] Updated weights for policy 1, policy_version 9350 (0.0009) [2023-10-08 04:18:28,202][00612] Updated weights for policy 1, policy_version 9360 (0.0010) [2023-10-08 04:18:28,579][00612] Updated weights for policy 1, policy_version 9370 (0.0008) [2023-10-08 04:18:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 19103744. Throughput: 0: 1849.7, 1: 1845.3. Samples: 4788440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:18:28,754][130385] Avg episode reward: [(0, '29.430'), (1, '27.820')] [2023-10-08 04:18:29,859][00611] Updated weights for policy 0, policy_version 9322 (0.0008) [2023-10-08 04:18:30,228][00611] Updated weights for policy 0, policy_version 9332 (0.0009) [2023-10-08 04:18:30,598][00611] Updated weights for policy 0, policy_version 9342 (0.0009) [2023-10-08 04:18:32,193][00612] Updated weights for policy 1, policy_version 9380 (0.0009) [2023-10-08 04:18:32,555][00612] Updated weights for policy 1, policy_version 9390 (0.0010) [2023-10-08 04:18:32,933][00612] Updated weights for policy 1, policy_version 9400 (0.0009) [2023-10-08 04:18:33,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 19202048. Throughput: 0: 1855.6, 1: 1823.6. Samples: 4810230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:18:33,755][130385] Avg episode reward: [(0, '27.730'), (1, '30.270')] [2023-10-08 04:18:34,184][00611] Updated weights for policy 0, policy_version 9352 (0.0008) [2023-10-08 04:18:34,556][00611] Updated weights for policy 0, policy_version 9362 (0.0007) [2023-10-08 04:18:34,923][00611] Updated weights for policy 0, policy_version 9372 (0.0007) [2023-10-08 04:18:36,560][00612] Updated weights for policy 1, policy_version 9410 (0.0009) [2023-10-08 04:18:36,928][00612] Updated weights for policy 1, policy_version 9420 (0.0007) [2023-10-08 04:18:37,291][00612] Updated weights for policy 1, policy_version 9430 (0.0008) [2023-10-08 04:18:37,662][00612] Updated weights for policy 1, policy_version 9440 (0.0007) [2023-10-08 04:18:38,540][00611] Updated weights for policy 0, policy_version 9382 (0.0008) [2023-10-08 04:18:38,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19267584. Throughput: 0: 1855.7, 1: 1838.8. Samples: 4821634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:18:38,754][130385] Avg episode reward: [(0, '27.320'), (1, '31.620')] [2023-10-08 04:18:38,755][00425] Saving new best policy, reward=31.620! [2023-10-08 04:18:38,914][00611] Updated weights for policy 0, policy_version 9392 (0.0010) [2023-10-08 04:18:39,287][00611] Updated weights for policy 0, policy_version 9402 (0.0009) [2023-10-08 04:18:41,221][00612] Updated weights for policy 1, policy_version 9450 (0.0009) [2023-10-08 04:18:41,589][00612] Updated weights for policy 1, policy_version 9460 (0.0007) [2023-10-08 04:18:41,965][00612] Updated weights for policy 1, policy_version 9470 (0.0008) [2023-10-08 04:18:42,866][00611] Updated weights for policy 0, policy_version 9412 (0.0008) [2023-10-08 04:18:43,229][00611] Updated weights for policy 0, policy_version 9422 (0.0007) [2023-10-08 04:18:43,606][00611] Updated weights for policy 0, policy_version 9432 (0.0009) [2023-10-08 04:18:43,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19333120. Throughput: 0: 1851.4, 1: 1828.8. Samples: 4843388. Policy #0 lag: (min: 31.0, avg: 45.2, max: 63.0) [2023-10-08 04:18:43,754][130385] Avg episode reward: [(0, '25.460'), (1, '31.420')] [2023-10-08 04:18:45,623][00612] Updated weights for policy 1, policy_version 9480 (0.0010) [2023-10-08 04:18:46,005][00612] Updated weights for policy 1, policy_version 9490 (0.0010) [2023-10-08 04:18:46,375][00612] Updated weights for policy 1, policy_version 9500 (0.0008) [2023-10-08 04:18:47,104][00611] Updated weights for policy 0, policy_version 9442 (0.0009) [2023-10-08 04:18:47,476][00611] Updated weights for policy 0, policy_version 9452 (0.0009) [2023-10-08 04:18:47,839][00611] Updated weights for policy 0, policy_version 9462 (0.0010) [2023-10-08 04:18:48,207][00611] Updated weights for policy 0, policy_version 9472 (0.0010) [2023-10-08 04:18:48,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 19431424. Throughput: 0: 1826.2, 1: 1846.8. Samples: 4865340. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 04:18:48,755][130385] Avg episode reward: [(0, '26.070'), (1, '31.500')] [2023-10-08 04:18:49,928][00612] Updated weights for policy 1, policy_version 9510 (0.0007) [2023-10-08 04:18:50,311][00612] Updated weights for policy 1, policy_version 9520 (0.0008) [2023-10-08 04:18:50,674][00612] Updated weights for policy 1, policy_version 9530 (0.0009) [2023-10-08 04:18:51,944][00611] Updated weights for policy 0, policy_version 9482 (0.0009) [2023-10-08 04:18:52,316][00611] Updated weights for policy 0, policy_version 9492 (0.0008) [2023-10-08 04:18:52,684][00611] Updated weights for policy 0, policy_version 9502 (0.0008) [2023-10-08 04:18:53,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19496960. Throughput: 0: 1855.1, 1: 1832.1. Samples: 4876752. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 04:18:53,754][130385] Avg episode reward: [(0, '25.020'), (1, '32.230')] [2023-10-08 04:18:53,755][00425] Saving new best policy, reward=32.230! [2023-10-08 04:18:54,235][00612] Updated weights for policy 1, policy_version 9540 (0.0009) [2023-10-08 04:18:54,598][00612] Updated weights for policy 1, policy_version 9550 (0.0007) [2023-10-08 04:18:54,969][00612] Updated weights for policy 1, policy_version 9560 (0.0007) [2023-10-08 04:18:56,351][00611] Updated weights for policy 0, policy_version 9512 (0.0008) [2023-10-08 04:18:56,734][00611] Updated weights for policy 0, policy_version 9522 (0.0008) [2023-10-08 04:18:57,100][00611] Updated weights for policy 0, policy_version 9532 (0.0007) [2023-10-08 04:18:58,501][00612] Updated weights for policy 1, policy_version 9570 (0.0008) [2023-10-08 04:18:58,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19562496. Throughput: 0: 1826.3, 1: 1853.7. Samples: 4898672. Policy #0 lag: (min: 9.0, avg: 20.6, max: 41.0) [2023-10-08 04:18:58,754][130385] Avg episode reward: [(0, '24.280'), (1, '33.190')] [2023-10-08 04:18:58,880][00612] Updated weights for policy 1, policy_version 9580 (0.0009) [2023-10-08 04:18:59,237][00612] Updated weights for policy 1, policy_version 9590 (0.0008) [2023-10-08 04:18:59,610][00612] Updated weights for policy 1, policy_version 9600 (0.0007) [2023-10-08 04:18:59,611][00425] Saving new best policy, reward=33.190! [2023-10-08 04:19:00,744][00611] Updated weights for policy 0, policy_version 9542 (0.0009) [2023-10-08 04:19:01,108][00611] Updated weights for policy 0, policy_version 9552 (0.0010) [2023-10-08 04:19:01,487][00611] Updated weights for policy 0, policy_version 9562 (0.0007) [2023-10-08 04:19:03,374][00612] Updated weights for policy 1, policy_version 9610 (0.0007) [2023-10-08 04:19:03,751][00612] Updated weights for policy 1, policy_version 9620 (0.0009) [2023-10-08 04:19:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19628032. Throughput: 0: 1845.4, 1: 1851.9. Samples: 4921506. Policy #0 lag: (min: 9.0, avg: 20.6, max: 41.0) [2023-10-08 04:19:03,754][130385] Avg episode reward: [(0, '24.780'), (1, '33.660')] [2023-10-08 04:19:04,120][00612] Updated weights for policy 1, policy_version 9630 (0.0008) [2023-10-08 04:19:04,186][00425] Saving new best policy, reward=33.660! [2023-10-08 04:19:05,027][00611] Updated weights for policy 0, policy_version 9572 (0.0007) [2023-10-08 04:19:05,398][00611] Updated weights for policy 0, policy_version 9582 (0.0007) [2023-10-08 04:19:05,770][00611] Updated weights for policy 0, policy_version 9592 (0.0008) [2023-10-08 04:19:07,716][00612] Updated weights for policy 1, policy_version 9640 (0.0010) [2023-10-08 04:19:08,093][00612] Updated weights for policy 1, policy_version 9650 (0.0009) [2023-10-08 04:19:08,462][00612] Updated weights for policy 1, policy_version 9660 (0.0009) [2023-10-08 04:19:08,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 19726336. Throughput: 0: 1824.9, 1: 1859.1. Samples: 4931852. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-08 04:19:08,755][130385] Avg episode reward: [(0, '26.450'), (1, '33.910')] [2023-10-08 04:19:08,757][00425] Saving new best policy, reward=33.910! [2023-10-08 04:19:09,513][00611] Updated weights for policy 0, policy_version 9602 (0.0007) [2023-10-08 04:19:09,885][00611] Updated weights for policy 0, policy_version 9612 (0.0009) [2023-10-08 04:19:10,258][00611] Updated weights for policy 0, policy_version 9622 (0.0009) [2023-10-08 04:19:10,625][00611] Updated weights for policy 0, policy_version 9632 (0.0011) [2023-10-08 04:19:12,180][00612] Updated weights for policy 1, policy_version 9670 (0.0011) [2023-10-08 04:19:12,545][00612] Updated weights for policy 1, policy_version 9680 (0.0010) [2023-10-08 04:19:12,912][00612] Updated weights for policy 1, policy_version 9690 (0.0010) [2023-10-08 04:19:13,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 19791872. Throughput: 0: 1840.6, 1: 1843.7. Samples: 4954236. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-08 04:19:13,754][130385] Avg episode reward: [(0, '27.340'), (1, '33.250')] [2023-10-08 04:19:14,148][00611] Updated weights for policy 0, policy_version 9642 (0.0007) [2023-10-08 04:19:14,518][00611] Updated weights for policy 0, policy_version 9652 (0.0008) [2023-10-08 04:19:14,903][00611] Updated weights for policy 0, policy_version 9662 (0.0007) [2023-10-08 04:19:16,444][00612] Updated weights for policy 1, policy_version 9700 (0.0010) [2023-10-08 04:19:16,821][00612] Updated weights for policy 1, policy_version 9710 (0.0009) [2023-10-08 04:19:17,194][00612] Updated weights for policy 1, policy_version 9720 (0.0010) [2023-10-08 04:19:18,644][00611] Updated weights for policy 0, policy_version 9672 (0.0008) [2023-10-08 04:19:18,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19857408. Throughput: 0: 1835.4, 1: 1852.8. Samples: 4976198. Policy #0 lag: (min: 25.0, avg: 37.8, max: 57.0) [2023-10-08 04:19:18,755][130385] Avg episode reward: [(0, '27.480'), (1, '32.790')] [2023-10-08 04:19:19,011][00611] Updated weights for policy 0, policy_version 9682 (0.0008) [2023-10-08 04:19:19,376][00611] Updated weights for policy 0, policy_version 9692 (0.0007) [2023-10-08 04:19:20,857][00612] Updated weights for policy 1, policy_version 9730 (0.0009) [2023-10-08 04:19:21,224][00612] Updated weights for policy 1, policy_version 9740 (0.0011) [2023-10-08 04:19:21,602][00612] Updated weights for policy 1, policy_version 9750 (0.0010) [2023-10-08 04:19:21,975][00612] Updated weights for policy 1, policy_version 9760 (0.0009) [2023-10-08 04:19:23,005][00611] Updated weights for policy 0, policy_version 9702 (0.0008) [2023-10-08 04:19:23,378][00611] Updated weights for policy 0, policy_version 9712 (0.0008) [2023-10-08 04:19:23,753][00611] Updated weights for policy 0, policy_version 9722 (0.0008) [2023-10-08 04:19:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19922944. Throughput: 0: 1837.7, 1: 1843.0. Samples: 4987264. Policy #0 lag: (min: 25.0, avg: 37.8, max: 57.0) [2023-10-08 04:19:23,755][130385] Avg episode reward: [(0, '27.970'), (1, '30.000')] [2023-10-08 04:19:25,800][00612] Updated weights for policy 1, policy_version 9770 (0.0008) [2023-10-08 04:19:26,176][00612] Updated weights for policy 1, policy_version 9780 (0.0009) [2023-10-08 04:19:26,537][00612] Updated weights for policy 1, policy_version 9790 (0.0009) [2023-10-08 04:19:27,329][00611] Updated weights for policy 0, policy_version 9732 (0.0011) [2023-10-08 04:19:27,697][00611] Updated weights for policy 0, policy_version 9742 (0.0010) [2023-10-08 04:19:28,075][00611] Updated weights for policy 0, policy_version 9752 (0.0008) [2023-10-08 04:19:28,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 20021248. Throughput: 0: 1842.4, 1: 1846.0. Samples: 5009364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:19:28,755][130385] Avg episode reward: [(0, '29.480'), (1, '29.500')] [2023-10-08 04:19:30,074][00612] Updated weights for policy 1, policy_version 9800 (0.0010) [2023-10-08 04:19:30,445][00612] Updated weights for policy 1, policy_version 9810 (0.0007) [2023-10-08 04:19:30,823][00612] Updated weights for policy 1, policy_version 9820 (0.0009) [2023-10-08 04:19:31,686][00611] Updated weights for policy 0, policy_version 9762 (0.0008) [2023-10-08 04:19:32,062][00611] Updated weights for policy 0, policy_version 9772 (0.0007) [2023-10-08 04:19:32,426][00611] Updated weights for policy 0, policy_version 9782 (0.0007) [2023-10-08 04:19:32,791][00611] Updated weights for policy 0, policy_version 9792 (0.0007) [2023-10-08 04:19:33,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20086784. Throughput: 0: 1834.5, 1: 1846.4. Samples: 5030980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:19:33,756][130385] Avg episode reward: [(0, '30.850'), (1, '27.430')] [2023-10-08 04:19:34,714][00612] Updated weights for policy 1, policy_version 9830 (0.0008) [2023-10-08 04:19:35,096][00612] Updated weights for policy 1, policy_version 9840 (0.0010) [2023-10-08 04:19:35,462][00612] Updated weights for policy 1, policy_version 9850 (0.0008) [2023-10-08 04:19:36,512][00611] Updated weights for policy 0, policy_version 9802 (0.0008) [2023-10-08 04:19:36,884][00611] Updated weights for policy 0, policy_version 9812 (0.0007) [2023-10-08 04:19:37,265][00611] Updated weights for policy 0, policy_version 9822 (0.0008) [2023-10-08 04:19:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 20152320. Throughput: 0: 1833.1, 1: 1844.1. Samples: 5042228. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 04:19:38,755][130385] Avg episode reward: [(0, '31.470'), (1, '28.370')] [2023-10-08 04:19:39,110][00612] Updated weights for policy 1, policy_version 9860 (0.0007) [2023-10-08 04:19:39,476][00612] Updated weights for policy 1, policy_version 9870 (0.0007) [2023-10-08 04:19:39,844][00612] Updated weights for policy 1, policy_version 9880 (0.0008) [2023-10-08 04:19:41,173][00611] Updated weights for policy 0, policy_version 9832 (0.0008) [2023-10-08 04:19:41,553][00611] Updated weights for policy 0, policy_version 9842 (0.0010) [2023-10-08 04:19:41,929][00611] Updated weights for policy 0, policy_version 9852 (0.0009) [2023-10-08 04:19:43,452][00612] Updated weights for policy 1, policy_version 9890 (0.0008) [2023-10-08 04:19:43,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20217856. Throughput: 0: 1832.2, 1: 1837.5. Samples: 5063806. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 04:19:43,755][130385] Avg episode reward: [(0, '31.080'), (1, '28.830')] [2023-10-08 04:19:43,810][00612] Updated weights for policy 1, policy_version 9900 (0.0009) [2023-10-08 04:19:44,180][00612] Updated weights for policy 1, policy_version 9910 (0.0008) [2023-10-08 04:19:44,559][00612] Updated weights for policy 1, policy_version 9920 (0.0008) [2023-10-08 04:19:45,658][00611] Updated weights for policy 0, policy_version 9862 (0.0010) [2023-10-08 04:19:46,024][00611] Updated weights for policy 0, policy_version 9872 (0.0009) [2023-10-08 04:19:46,393][00611] Updated weights for policy 0, policy_version 9882 (0.0008) [2023-10-08 04:19:48,117][00612] Updated weights for policy 1, policy_version 9930 (0.0009) [2023-10-08 04:19:48,494][00612] Updated weights for policy 1, policy_version 9940 (0.0011) [2023-10-08 04:19:48,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 20283392. Throughput: 0: 1836.1, 1: 1829.7. Samples: 5086468. Policy #0 lag: (min: 26.0, avg: 27.6, max: 47.0) [2023-10-08 04:19:48,755][130385] Avg episode reward: [(0, '30.450'), (1, '30.600')] [2023-10-08 04:19:48,863][00612] Updated weights for policy 1, policy_version 9950 (0.0008) [2023-10-08 04:19:49,975][00611] Updated weights for policy 0, policy_version 9892 (0.0009) [2023-10-08 04:19:50,350][00611] Updated weights for policy 0, policy_version 9902 (0.0008) [2023-10-08 04:19:50,728][00611] Updated weights for policy 0, policy_version 9912 (0.0008) [2023-10-08 04:19:52,493][00612] Updated weights for policy 1, policy_version 9960 (0.0009) [2023-10-08 04:19:52,861][00612] Updated weights for policy 1, policy_version 9970 (0.0007) [2023-10-08 04:19:53,229][00612] Updated weights for policy 1, policy_version 9980 (0.0007) [2023-10-08 04:19:53,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 20381696. Throughput: 0: 1836.8, 1: 1835.4. Samples: 5097102. Policy #0 lag: (min: 26.0, avg: 27.6, max: 47.0) [2023-10-08 04:19:53,755][130385] Avg episode reward: [(0, '30.970'), (1, '30.670')] [2023-10-08 04:19:54,240][00611] Updated weights for policy 0, policy_version 9922 (0.0008) [2023-10-08 04:19:54,619][00611] Updated weights for policy 0, policy_version 9932 (0.0009) [2023-10-08 04:19:54,989][00611] Updated weights for policy 0, policy_version 9942 (0.0007) [2023-10-08 04:19:55,366][00611] Updated weights for policy 0, policy_version 9952 (0.0009) [2023-10-08 04:19:56,858][00612] Updated weights for policy 1, policy_version 9990 (0.0007) [2023-10-08 04:19:57,241][00612] Updated weights for policy 1, policy_version 10000 (0.0008) [2023-10-08 04:19:57,605][00612] Updated weights for policy 1, policy_version 10010 (0.0009) [2023-10-08 04:19:58,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 20447232. Throughput: 0: 1840.9, 1: 1834.0. Samples: 5119608. Policy #0 lag: (min: 26.0, avg: 27.6, max: 47.0) [2023-10-08 04:19:58,755][130385] Avg episode reward: [(0, '34.250'), (1, '31.530')] [2023-10-08 04:19:59,037][00611] Updated weights for policy 0, policy_version 9962 (0.0009) [2023-10-08 04:19:59,409][00611] Updated weights for policy 0, policy_version 9972 (0.0008) [2023-10-08 04:19:59,781][00611] Updated weights for policy 0, policy_version 9982 (0.0008) [2023-10-08 04:19:59,856][00365] Saving new best policy, reward=34.250! [2023-10-08 04:20:01,051][00612] Updated weights for policy 1, policy_version 10020 (0.0010) [2023-10-08 04:20:01,427][00612] Updated weights for policy 1, policy_version 10030 (0.0007) [2023-10-08 04:20:01,789][00612] Updated weights for policy 1, policy_version 10040 (0.0010) [2023-10-08 04:20:03,345][00611] Updated weights for policy 0, policy_version 9992 (0.0008) [2023-10-08 04:20:03,723][00611] Updated weights for policy 0, policy_version 10002 (0.0008) [2023-10-08 04:20:03,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20512768. Throughput: 0: 1838.7, 1: 1844.5. Samples: 5141940. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 04:20:03,754][130385] Avg episode reward: [(0, '35.480'), (1, '31.630')] [2023-10-08 04:20:04,096][00611] Updated weights for policy 0, policy_version 10012 (0.0008) [2023-10-08 04:20:04,234][00365] Saving new best policy, reward=35.480! [2023-10-08 04:20:05,454][00612] Updated weights for policy 1, policy_version 10050 (0.0008) [2023-10-08 04:20:05,827][00612] Updated weights for policy 1, policy_version 10060 (0.0008) [2023-10-08 04:20:06,191][00612] Updated weights for policy 1, policy_version 10070 (0.0007) [2023-10-08 04:20:06,559][00612] Updated weights for policy 1, policy_version 10080 (0.0007) [2023-10-08 04:20:07,632][00611] Updated weights for policy 0, policy_version 10022 (0.0008) [2023-10-08 04:20:08,006][00611] Updated weights for policy 0, policy_version 10032 (0.0010) [2023-10-08 04:20:08,376][00611] Updated weights for policy 0, policy_version 10042 (0.0007) [2023-10-08 04:20:08,754][130385] Fps is (10 sec: 16384.6, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 20611072. Throughput: 0: 1843.7, 1: 1833.6. Samples: 5152740. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 04:20:08,754][130385] Avg episode reward: [(0, '33.080'), (1, '33.730')] [2023-10-08 04:20:10,219][00612] Updated weights for policy 1, policy_version 10090 (0.0008) [2023-10-08 04:20:10,590][00612] Updated weights for policy 1, policy_version 10100 (0.0007) [2023-10-08 04:20:10,948][00612] Updated weights for policy 1, policy_version 10110 (0.0009) [2023-10-08 04:20:11,964][00611] Updated weights for policy 0, policy_version 10052 (0.0008) [2023-10-08 04:20:12,339][00611] Updated weights for policy 0, policy_version 10062 (0.0007) [2023-10-08 04:20:12,721][00611] Updated weights for policy 0, policy_version 10072 (0.0008) [2023-10-08 04:20:13,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20676608. Throughput: 0: 1831.5, 1: 1844.9. Samples: 5174802. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 04:20:13,754][130385] Avg episode reward: [(0, '33.200'), (1, '34.270')] [2023-10-08 04:20:13,755][00425] Saving new best policy, reward=34.270! [2023-10-08 04:20:14,753][00612] Updated weights for policy 1, policy_version 10120 (0.0010) [2023-10-08 04:20:15,127][00612] Updated weights for policy 1, policy_version 10130 (0.0008) [2023-10-08 04:20:15,494][00612] Updated weights for policy 1, policy_version 10140 (0.0007) [2023-10-08 04:20:16,477][00611] Updated weights for policy 0, policy_version 10082 (0.0008) [2023-10-08 04:20:16,838][00611] Updated weights for policy 0, policy_version 10092 (0.0008) [2023-10-08 04:20:17,212][00611] Updated weights for policy 0, policy_version 10102 (0.0008) [2023-10-08 04:20:17,593][00611] Updated weights for policy 0, policy_version 10112 (0.0011) [2023-10-08 04:20:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20742144. Throughput: 0: 1836.9, 1: 1846.4. Samples: 5196726. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 04:20:18,755][130385] Avg episode reward: [(0, '33.560'), (1, '32.740')] [2023-10-08 04:20:18,765][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000010112_10354688.pth... [2023-10-08 04:20:18,800][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000008384_8585216.pth [2023-10-08 04:20:19,111][00612] Updated weights for policy 1, policy_version 10150 (0.0007) [2023-10-08 04:20:19,506][00612] Updated weights for policy 1, policy_version 10160 (0.0009) [2023-10-08 04:20:19,874][00612] Updated weights for policy 1, policy_version 10170 (0.0009) [2023-10-08 04:20:20,089][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000010176_10420224.pth... [2023-10-08 04:20:20,127][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000008448_8650752.pth [2023-10-08 04:20:21,363][00611] Updated weights for policy 0, policy_version 10122 (0.0008) [2023-10-08 04:20:21,730][00611] Updated weights for policy 0, policy_version 10132 (0.0009) [2023-10-08 04:20:22,110][00611] Updated weights for policy 0, policy_version 10142 (0.0007) [2023-10-08 04:20:23,439][00612] Updated weights for policy 1, policy_version 10180 (0.0007) [2023-10-08 04:20:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20807680. Throughput: 0: 1831.1, 1: 1850.5. Samples: 5207898. Policy #0 lag: (min: 9.0, avg: 19.9, max: 41.0) [2023-10-08 04:20:23,754][130385] Avg episode reward: [(0, '35.260'), (1, '31.410')] [2023-10-08 04:20:23,810][00612] Updated weights for policy 1, policy_version 10190 (0.0010) [2023-10-08 04:20:24,186][00612] Updated weights for policy 1, policy_version 10200 (0.0008) [2023-10-08 04:20:25,821][00611] Updated weights for policy 0, policy_version 10152 (0.0008) [2023-10-08 04:20:26,190][00611] Updated weights for policy 0, policy_version 10162 (0.0008) [2023-10-08 04:20:26,566][00611] Updated weights for policy 0, policy_version 10172 (0.0011) [2023-10-08 04:20:27,750][00612] Updated weights for policy 1, policy_version 10210 (0.0007) [2023-10-08 04:20:28,121][00612] Updated weights for policy 1, policy_version 10220 (0.0009) [2023-10-08 04:20:28,494][00612] Updated weights for policy 1, policy_version 10230 (0.0007) [2023-10-08 04:20:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 20873216. Throughput: 0: 1836.6, 1: 1851.7. Samples: 5229778. Policy #0 lag: (min: 9.0, avg: 19.9, max: 41.0) [2023-10-08 04:20:28,754][130385] Avg episode reward: [(0, '33.870'), (1, '29.390')] [2023-10-08 04:20:28,865][00612] Updated weights for policy 1, policy_version 10240 (0.0009) [2023-10-08 04:20:30,279][00611] Updated weights for policy 0, policy_version 10182 (0.0008) [2023-10-08 04:20:30,653][00611] Updated weights for policy 0, policy_version 10192 (0.0008) [2023-10-08 04:20:31,019][00611] Updated weights for policy 0, policy_version 10202 (0.0008) [2023-10-08 04:20:32,401][00612] Updated weights for policy 1, policy_version 10250 (0.0009) [2023-10-08 04:20:32,761][00612] Updated weights for policy 1, policy_version 10260 (0.0008) [2023-10-08 04:20:33,138][00612] Updated weights for policy 1, policy_version 10270 (0.0008) [2023-10-08 04:20:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 20971520. Throughput: 0: 1834.8, 1: 1833.1. Samples: 5251522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:20:33,754][130385] Avg episode reward: [(0, '31.360'), (1, '29.280')] [2023-10-08 04:20:34,620][00611] Updated weights for policy 0, policy_version 10212 (0.0008) [2023-10-08 04:20:34,995][00611] Updated weights for policy 0, policy_version 10222 (0.0007) [2023-10-08 04:20:35,371][00611] Updated weights for policy 0, policy_version 10232 (0.0008) [2023-10-08 04:20:36,698][00612] Updated weights for policy 1, policy_version 10280 (0.0007) [2023-10-08 04:20:37,067][00612] Updated weights for policy 1, policy_version 10290 (0.0010) [2023-10-08 04:20:37,440][00612] Updated weights for policy 1, policy_version 10300 (0.0009) [2023-10-08 04:20:38,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 21037056. Throughput: 0: 1834.3, 1: 1854.7. Samples: 5263104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:20:38,755][130385] Avg episode reward: [(0, '26.920'), (1, '26.710')] [2023-10-08 04:20:38,963][00611] Updated weights for policy 0, policy_version 10242 (0.0007) [2023-10-08 04:20:39,340][00611] Updated weights for policy 0, policy_version 10252 (0.0011) [2023-10-08 04:20:39,719][00611] Updated weights for policy 0, policy_version 10262 (0.0008) [2023-10-08 04:20:40,078][00611] Updated weights for policy 0, policy_version 10272 (0.0011) [2023-10-08 04:20:41,154][00612] Updated weights for policy 1, policy_version 10310 (0.0009) [2023-10-08 04:20:41,517][00612] Updated weights for policy 1, policy_version 10320 (0.0009) [2023-10-08 04:20:41,886][00612] Updated weights for policy 1, policy_version 10330 (0.0007) [2023-10-08 04:20:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 21102592. Throughput: 0: 1836.1, 1: 1828.9. Samples: 5284530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:20:43,754][130385] Avg episode reward: [(0, '26.890'), (1, '25.820')] [2023-10-08 04:20:43,804][00611] Updated weights for policy 0, policy_version 10282 (0.0010) [2023-10-08 04:20:44,186][00611] Updated weights for policy 0, policy_version 10292 (0.0010) [2023-10-08 04:20:44,561][00611] Updated weights for policy 0, policy_version 10302 (0.0007) [2023-10-08 04:20:45,499][00612] Updated weights for policy 1, policy_version 10340 (0.0009) [2023-10-08 04:20:45,868][00612] Updated weights for policy 1, policy_version 10350 (0.0011) [2023-10-08 04:20:46,241][00612] Updated weights for policy 1, policy_version 10360 (0.0009) [2023-10-08 04:20:48,135][00611] Updated weights for policy 0, policy_version 10312 (0.0009) [2023-10-08 04:20:48,510][00611] Updated weights for policy 0, policy_version 10322 (0.0009) [2023-10-08 04:20:48,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 21168128. Throughput: 0: 1827.8, 1: 1844.4. Samples: 5307192. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 04:20:48,755][130385] Avg episode reward: [(0, '24.690'), (1, '26.810')] [2023-10-08 04:20:48,877][00611] Updated weights for policy 0, policy_version 10332 (0.0009) [2023-10-08 04:20:49,880][00612] Updated weights for policy 1, policy_version 10370 (0.0008) [2023-10-08 04:20:50,256][00612] Updated weights for policy 1, policy_version 10380 (0.0009) [2023-10-08 04:20:50,620][00612] Updated weights for policy 1, policy_version 10390 (0.0010) [2023-10-08 04:20:50,990][00612] Updated weights for policy 1, policy_version 10400 (0.0009) [2023-10-08 04:20:52,528][00611] Updated weights for policy 0, policy_version 10342 (0.0008) [2023-10-08 04:20:52,899][00611] Updated weights for policy 0, policy_version 10352 (0.0007) [2023-10-08 04:20:53,277][00611] Updated weights for policy 0, policy_version 10362 (0.0009) [2023-10-08 04:20:53,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 21266432. Throughput: 0: 1830.3, 1: 1832.7. Samples: 5317576. Policy #0 lag: (min: 22.0, avg: 24.9, max: 54.0) [2023-10-08 04:20:53,755][130385] Avg episode reward: [(0, '24.160'), (1, '26.490')] [2023-10-08 04:20:54,583][00612] Updated weights for policy 1, policy_version 10410 (0.0009) [2023-10-08 04:20:54,958][00612] Updated weights for policy 1, policy_version 10420 (0.0007) [2023-10-08 04:20:55,326][00612] Updated weights for policy 1, policy_version 10430 (0.0007) [2023-10-08 04:20:56,841][00611] Updated weights for policy 0, policy_version 10372 (0.0008) [2023-10-08 04:20:57,213][00611] Updated weights for policy 0, policy_version 10382 (0.0009) [2023-10-08 04:20:57,595][00611] Updated weights for policy 0, policy_version 10392 (0.0008) [2023-10-08 04:20:58,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 21331968. Throughput: 0: 1822.3, 1: 1855.0. Samples: 5340282. Policy #0 lag: (min: 22.0, avg: 24.9, max: 54.0) [2023-10-08 04:20:58,754][130385] Avg episode reward: [(0, '25.430'), (1, '29.120')] [2023-10-08 04:20:59,024][00612] Updated weights for policy 1, policy_version 10440 (0.0009) [2023-10-08 04:20:59,393][00612] Updated weights for policy 1, policy_version 10450 (0.0009) [2023-10-08 04:20:59,764][00612] Updated weights for policy 1, policy_version 10460 (0.0008) [2023-10-08 04:21:01,277][00611] Updated weights for policy 0, policy_version 10402 (0.0007) [2023-10-08 04:21:01,644][00611] Updated weights for policy 0, policy_version 10412 (0.0008) [2023-10-08 04:21:02,015][00611] Updated weights for policy 0, policy_version 10422 (0.0007) [2023-10-08 04:21:02,390][00611] Updated weights for policy 0, policy_version 10432 (0.0007) [2023-10-08 04:21:03,312][00612] Updated weights for policy 1, policy_version 10470 (0.0007) [2023-10-08 04:21:03,682][00612] Updated weights for policy 1, policy_version 10480 (0.0007) [2023-10-08 04:21:03,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 21397504. Throughput: 0: 1830.4, 1: 1850.1. Samples: 5362348. Policy #0 lag: (min: 31.0, avg: 42.1, max: 63.0) [2023-10-08 04:21:03,755][130385] Avg episode reward: [(0, '28.580'), (1, '28.420')] [2023-10-08 04:21:04,047][00612] Updated weights for policy 1, policy_version 10490 (0.0009) [2023-10-08 04:21:05,842][00611] Updated weights for policy 0, policy_version 10442 (0.0007) [2023-10-08 04:21:06,215][00611] Updated weights for policy 0, policy_version 10452 (0.0007) [2023-10-08 04:21:06,587][00611] Updated weights for policy 0, policy_version 10462 (0.0009) [2023-10-08 04:21:07,781][00612] Updated weights for policy 1, policy_version 10500 (0.0010) [2023-10-08 04:21:08,148][00612] Updated weights for policy 1, policy_version 10510 (0.0010) [2023-10-08 04:21:08,526][00612] Updated weights for policy 1, policy_version 10520 (0.0012) [2023-10-08 04:21:08,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 21463040. Throughput: 0: 1826.6, 1: 1853.3. Samples: 5373496. Policy #0 lag: (min: 31.0, avg: 42.1, max: 63.0) [2023-10-08 04:21:08,754][130385] Avg episode reward: [(0, '31.760'), (1, '28.540')] [2023-10-08 04:21:10,311][00611] Updated weights for policy 0, policy_version 10472 (0.0008) [2023-10-08 04:21:10,695][00611] Updated weights for policy 0, policy_version 10482 (0.0011) [2023-10-08 04:21:11,062][00611] Updated weights for policy 0, policy_version 10492 (0.0007) [2023-10-08 04:21:12,155][00612] Updated weights for policy 1, policy_version 10530 (0.0008) [2023-10-08 04:21:12,531][00612] Updated weights for policy 1, policy_version 10540 (0.0010) [2023-10-08 04:21:12,907][00612] Updated weights for policy 1, policy_version 10550 (0.0009) [2023-10-08 04:21:13,276][00612] Updated weights for policy 1, policy_version 10560 (0.0008) [2023-10-08 04:21:13,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 21561344. Throughput: 0: 1837.2, 1: 1842.8. Samples: 5395376. Policy #0 lag: (min: 31.0, avg: 42.1, max: 63.0) [2023-10-08 04:21:13,754][130385] Avg episode reward: [(0, '31.080'), (1, '28.880')] [2023-10-08 04:21:14,805][00611] Updated weights for policy 0, policy_version 10502 (0.0009) [2023-10-08 04:21:15,177][00611] Updated weights for policy 0, policy_version 10512 (0.0011) [2023-10-08 04:21:15,547][00611] Updated weights for policy 0, policy_version 10522 (0.0010) [2023-10-08 04:21:16,900][00612] Updated weights for policy 1, policy_version 10570 (0.0008) [2023-10-08 04:21:17,264][00612] Updated weights for policy 1, policy_version 10580 (0.0007) [2023-10-08 04:21:17,625][00612] Updated weights for policy 1, policy_version 10590 (0.0008) [2023-10-08 04:21:18,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 21626880. Throughput: 0: 1839.1, 1: 1840.9. Samples: 5417124. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 04:21:18,754][130385] Avg episode reward: [(0, '33.100'), (1, '30.810')] [2023-10-08 04:21:19,154][00611] Updated weights for policy 0, policy_version 10532 (0.0010) [2023-10-08 04:21:19,532][00611] Updated weights for policy 0, policy_version 10542 (0.0008) [2023-10-08 04:21:19,902][00611] Updated weights for policy 0, policy_version 10552 (0.0008) [2023-10-08 04:21:21,220][00612] Updated weights for policy 1, policy_version 10600 (0.0008) [2023-10-08 04:21:21,593][00612] Updated weights for policy 1, policy_version 10610 (0.0007) [2023-10-08 04:21:21,960][00612] Updated weights for policy 1, policy_version 10620 (0.0008) [2023-10-08 04:21:23,475][00611] Updated weights for policy 0, policy_version 10562 (0.0009) [2023-10-08 04:21:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 21692416. Throughput: 0: 1836.5, 1: 1833.7. Samples: 5428260. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 04:21:23,754][130385] Avg episode reward: [(0, '32.920'), (1, '31.130')] [2023-10-08 04:21:23,856][00611] Updated weights for policy 0, policy_version 10572 (0.0009) [2023-10-08 04:21:24,219][00611] Updated weights for policy 0, policy_version 10582 (0.0009) [2023-10-08 04:21:24,588][00611] Updated weights for policy 0, policy_version 10592 (0.0008) [2023-10-08 04:21:25,706][00612] Updated weights for policy 1, policy_version 10630 (0.0007) [2023-10-08 04:21:26,080][00612] Updated weights for policy 1, policy_version 10640 (0.0008) [2023-10-08 04:21:26,451][00612] Updated weights for policy 1, policy_version 10650 (0.0007) [2023-10-08 04:21:28,259][00611] Updated weights for policy 0, policy_version 10602 (0.0009) [2023-10-08 04:21:28,625][00611] Updated weights for policy 0, policy_version 10612 (0.0009) [2023-10-08 04:21:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 21757952. Throughput: 0: 1834.9, 1: 1846.5. Samples: 5450194. Policy #0 lag: (min: 25.0, avg: 45.2, max: 57.0) [2023-10-08 04:21:28,754][130385] Avg episode reward: [(0, '30.460'), (1, '29.280')] [2023-10-08 04:21:29,008][00611] Updated weights for policy 0, policy_version 10622 (0.0010) [2023-10-08 04:21:30,104][00612] Updated weights for policy 1, policy_version 10660 (0.0008) [2023-10-08 04:21:30,470][00612] Updated weights for policy 1, policy_version 10670 (0.0010) [2023-10-08 04:21:30,835][00612] Updated weights for policy 1, policy_version 10680 (0.0009) [2023-10-08 04:21:32,577][00611] Updated weights for policy 0, policy_version 10632 (0.0008) [2023-10-08 04:21:32,950][00611] Updated weights for policy 0, policy_version 10642 (0.0010) [2023-10-08 04:21:33,332][00611] Updated weights for policy 0, policy_version 10652 (0.0010) [2023-10-08 04:21:33,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 21856256. Throughput: 0: 1824.0, 1: 1845.4. Samples: 5472312. Policy #0 lag: (min: 25.0, avg: 45.2, max: 57.0) [2023-10-08 04:21:33,754][130385] Avg episode reward: [(0, '31.030'), (1, '30.760')] [2023-10-08 04:21:34,518][00612] Updated weights for policy 1, policy_version 10690 (0.0009) [2023-10-08 04:21:34,881][00612] Updated weights for policy 1, policy_version 10700 (0.0008) [2023-10-08 04:21:35,249][00612] Updated weights for policy 1, policy_version 10710 (0.0008) [2023-10-08 04:21:35,621][00612] Updated weights for policy 1, policy_version 10720 (0.0008) [2023-10-08 04:21:36,887][00611] Updated weights for policy 0, policy_version 10662 (0.0008) [2023-10-08 04:21:37,259][00611] Updated weights for policy 0, policy_version 10672 (0.0008) [2023-10-08 04:21:37,636][00611] Updated weights for policy 0, policy_version 10682 (0.0009) [2023-10-08 04:21:38,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 21921792. Throughput: 0: 1841.5, 1: 1842.8. Samples: 5483372. Policy #0 lag: (min: 27.0, avg: 27.1, max: 34.0) [2023-10-08 04:21:38,755][130385] Avg episode reward: [(0, '32.440'), (1, '32.020')] [2023-10-08 04:21:39,294][00612] Updated weights for policy 1, policy_version 10730 (0.0008) [2023-10-08 04:21:39,657][00612] Updated weights for policy 1, policy_version 10740 (0.0007) [2023-10-08 04:21:40,021][00612] Updated weights for policy 1, policy_version 10750 (0.0010) [2023-10-08 04:21:41,327][00611] Updated weights for policy 0, policy_version 10692 (0.0009) [2023-10-08 04:21:41,700][00611] Updated weights for policy 0, policy_version 10702 (0.0007) [2023-10-08 04:21:42,070][00611] Updated weights for policy 0, policy_version 10712 (0.0008) [2023-10-08 04:21:43,592][00612] Updated weights for policy 1, policy_version 10760 (0.0012) [2023-10-08 04:21:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 21987328. Throughput: 0: 1827.6, 1: 1841.8. Samples: 5505404. Policy #0 lag: (min: 27.0, avg: 27.1, max: 34.0) [2023-10-08 04:21:43,754][130385] Avg episode reward: [(0, '32.300'), (1, '33.890')] [2023-10-08 04:21:43,950][00612] Updated weights for policy 1, policy_version 10770 (0.0008) [2023-10-08 04:21:44,308][00612] Updated weights for policy 1, policy_version 10780 (0.0007) [2023-10-08 04:21:45,505][00611] Updated weights for policy 0, policy_version 10722 (0.0008) [2023-10-08 04:21:45,870][00611] Updated weights for policy 0, policy_version 10732 (0.0009) [2023-10-08 04:21:46,246][00611] Updated weights for policy 0, policy_version 10742 (0.0007) [2023-10-08 04:21:46,610][00611] Updated weights for policy 0, policy_version 10752 (0.0008) [2023-10-08 04:21:47,965][00612] Updated weights for policy 1, policy_version 10790 (0.0010) [2023-10-08 04:21:48,329][00612] Updated weights for policy 1, policy_version 10800 (0.0012) [2023-10-08 04:21:48,697][00612] Updated weights for policy 1, policy_version 10810 (0.0010) [2023-10-08 04:21:48,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 22052864. Throughput: 0: 1849.9, 1: 1825.3. Samples: 5527734. Policy #0 lag: (min: 27.0, avg: 27.1, max: 34.0) [2023-10-08 04:21:48,754][130385] Avg episode reward: [(0, '33.950'), (1, '33.650')] [2023-10-08 04:21:50,291][00611] Updated weights for policy 0, policy_version 10762 (0.0008) [2023-10-08 04:21:50,658][00611] Updated weights for policy 0, policy_version 10772 (0.0008) [2023-10-08 04:21:51,033][00611] Updated weights for policy 0, policy_version 10782 (0.0010) [2023-10-08 04:21:52,348][00612] Updated weights for policy 1, policy_version 10820 (0.0009) [2023-10-08 04:21:52,719][00612] Updated weights for policy 1, policy_version 10830 (0.0010) [2023-10-08 04:21:53,096][00612] Updated weights for policy 1, policy_version 10840 (0.0008) [2023-10-08 04:21:53,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 22151168. Throughput: 0: 1826.3, 1: 1837.1. Samples: 5538346. Policy #0 lag: (min: 5.0, avg: 11.1, max: 37.0) [2023-10-08 04:21:53,754][130385] Avg episode reward: [(0, '34.430'), (1, '33.190')] [2023-10-08 04:21:54,743][00611] Updated weights for policy 0, policy_version 10792 (0.0009) [2023-10-08 04:21:55,115][00611] Updated weights for policy 0, policy_version 10802 (0.0007) [2023-10-08 04:21:55,476][00611] Updated weights for policy 0, policy_version 10812 (0.0009) [2023-10-08 04:21:56,832][00612] Updated weights for policy 1, policy_version 10850 (0.0009) [2023-10-08 04:21:57,245][00612] Updated weights for policy 1, policy_version 10860 (0.0008) [2023-10-08 04:21:57,615][00612] Updated weights for policy 1, policy_version 10870 (0.0009) [2023-10-08 04:21:57,982][00612] Updated weights for policy 1, policy_version 10880 (0.0008) [2023-10-08 04:21:58,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 22216704. Throughput: 0: 1852.0, 1: 1824.5. Samples: 5560818. Policy #0 lag: (min: 5.0, avg: 11.1, max: 37.0) [2023-10-08 04:21:58,755][130385] Avg episode reward: [(0, '34.550'), (1, '31.600')] [2023-10-08 04:21:59,187][00611] Updated weights for policy 0, policy_version 10822 (0.0010) [2023-10-08 04:21:59,573][00611] Updated weights for policy 0, policy_version 10832 (0.0007) [2023-10-08 04:21:59,947][00611] Updated weights for policy 0, policy_version 10842 (0.0009) [2023-10-08 04:22:01,511][00612] Updated weights for policy 1, policy_version 10890 (0.0007) [2023-10-08 04:22:01,877][00612] Updated weights for policy 1, policy_version 10900 (0.0008) [2023-10-08 04:22:02,243][00612] Updated weights for policy 1, policy_version 10910 (0.0009) [2023-10-08 04:22:03,627][00611] Updated weights for policy 0, policy_version 10852 (0.0010) [2023-10-08 04:22:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 22282240. Throughput: 0: 1847.2, 1: 1832.2. Samples: 5582696. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-08 04:22:03,754][130385] Avg episode reward: [(0, '35.120'), (1, '31.380')] [2023-10-08 04:22:03,988][00611] Updated weights for policy 0, policy_version 10862 (0.0010) [2023-10-08 04:22:04,365][00611] Updated weights for policy 0, policy_version 10872 (0.0011) [2023-10-08 04:22:05,827][00612] Updated weights for policy 1, policy_version 10920 (0.0007) [2023-10-08 04:22:06,196][00612] Updated weights for policy 1, policy_version 10930 (0.0007) [2023-10-08 04:22:06,570][00612] Updated weights for policy 1, policy_version 10940 (0.0007) [2023-10-08 04:22:08,065][00611] Updated weights for policy 0, policy_version 10882 (0.0010) [2023-10-08 04:22:08,443][00611] Updated weights for policy 0, policy_version 10892 (0.0008) [2023-10-08 04:22:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 22347776. Throughput: 0: 1843.5, 1: 1829.6. Samples: 5593550. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-08 04:22:08,755][130385] Avg episode reward: [(0, '34.790'), (1, '31.650')] [2023-10-08 04:22:08,812][00611] Updated weights for policy 0, policy_version 10902 (0.0009) [2023-10-08 04:22:09,184][00611] Updated weights for policy 0, policy_version 10912 (0.0008) [2023-10-08 04:22:10,323][00612] Updated weights for policy 1, policy_version 10950 (0.0009) [2023-10-08 04:22:10,691][00612] Updated weights for policy 1, policy_version 10960 (0.0009) [2023-10-08 04:22:11,058][00612] Updated weights for policy 1, policy_version 10970 (0.0009) [2023-10-08 04:22:12,905][00611] Updated weights for policy 0, policy_version 10922 (0.0010) [2023-10-08 04:22:13,280][00611] Updated weights for policy 0, policy_version 10932 (0.0011) [2023-10-08 04:22:13,652][00611] Updated weights for policy 0, policy_version 10942 (0.0010) [2023-10-08 04:22:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 22446080. Throughput: 0: 1836.8, 1: 1839.5. Samples: 5615628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:22:13,755][130385] Avg episode reward: [(0, '34.410'), (1, '29.140')] [2023-10-08 04:22:14,683][00612] Updated weights for policy 1, policy_version 10980 (0.0009) [2023-10-08 04:22:15,057][00612] Updated weights for policy 1, policy_version 10990 (0.0008) [2023-10-08 04:22:15,428][00612] Updated weights for policy 1, policy_version 11000 (0.0009) [2023-10-08 04:22:17,284][00611] Updated weights for policy 0, policy_version 10952 (0.0007) [2023-10-08 04:22:17,662][00611] Updated weights for policy 0, policy_version 10962 (0.0009) [2023-10-08 04:22:18,026][00611] Updated weights for policy 0, policy_version 10972 (0.0007) [2023-10-08 04:22:18,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 22511616. Throughput: 0: 1827.5, 1: 1843.4. Samples: 5637502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:22:18,755][130385] Avg episode reward: [(0, '32.480'), (1, '27.170')] [2023-10-08 04:22:18,768][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000011008_11272192.pth... [2023-10-08 04:22:18,768][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000010976_11239424.pth... [2023-10-08 04:22:18,802][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000009312_9535488.pth [2023-10-08 04:22:18,803][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000009248_9469952.pth [2023-10-08 04:22:19,027][00612] Updated weights for policy 1, policy_version 11010 (0.0008) [2023-10-08 04:22:19,385][00612] Updated weights for policy 1, policy_version 11020 (0.0008) [2023-10-08 04:22:19,759][00612] Updated weights for policy 1, policy_version 11030 (0.0009) [2023-10-08 04:22:20,133][00612] Updated weights for policy 1, policy_version 11040 (0.0009) [2023-10-08 04:22:21,816][00611] Updated weights for policy 0, policy_version 10982 (0.0008) [2023-10-08 04:22:22,186][00611] Updated weights for policy 0, policy_version 10992 (0.0009) [2023-10-08 04:22:22,547][00611] Updated weights for policy 0, policy_version 11002 (0.0010) [2023-10-08 04:22:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 22577152. Throughput: 0: 1829.7, 1: 1845.5. Samples: 5648754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:22:23,754][130385] Avg episode reward: [(0, '32.090'), (1, '28.540')] [2023-10-08 04:22:23,773][00612] Updated weights for policy 1, policy_version 11050 (0.0008) [2023-10-08 04:22:24,131][00612] Updated weights for policy 1, policy_version 11060 (0.0008) [2023-10-08 04:22:24,499][00612] Updated weights for policy 1, policy_version 11070 (0.0009) [2023-10-08 04:22:26,128][00611] Updated weights for policy 0, policy_version 11012 (0.0009) [2023-10-08 04:22:26,491][00611] Updated weights for policy 0, policy_version 11022 (0.0010) [2023-10-08 04:22:26,866][00611] Updated weights for policy 0, policy_version 11032 (0.0007) [2023-10-08 04:22:28,026][00612] Updated weights for policy 1, policy_version 11080 (0.0010) [2023-10-08 04:22:28,393][00612] Updated weights for policy 1, policy_version 11090 (0.0007) [2023-10-08 04:22:28,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 22642688. Throughput: 0: 1828.4, 1: 1846.2. Samples: 5670758. Policy #0 lag: (min: 26.0, avg: 29.0, max: 58.0) [2023-10-08 04:22:28,754][130385] Avg episode reward: [(0, '33.000'), (1, '27.090')] [2023-10-08 04:22:28,766][00612] Updated weights for policy 1, policy_version 11100 (0.0009) [2023-10-08 04:22:30,433][00611] Updated weights for policy 0, policy_version 11042 (0.0008) [2023-10-08 04:22:30,801][00611] Updated weights for policy 0, policy_version 11052 (0.0009) [2023-10-08 04:22:31,171][00611] Updated weights for policy 0, policy_version 11062 (0.0008) [2023-10-08 04:22:31,535][00611] Updated weights for policy 0, policy_version 11072 (0.0008) [2023-10-08 04:22:32,289][00612] Updated weights for policy 1, policy_version 11110 (0.0009) [2023-10-08 04:22:32,667][00612] Updated weights for policy 1, policy_version 11120 (0.0009) [2023-10-08 04:22:33,036][00612] Updated weights for policy 1, policy_version 11130 (0.0007) [2023-10-08 04:22:33,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 22740992. Throughput: 0: 1827.5, 1: 1833.8. Samples: 5692492. Policy #0 lag: (min: 26.0, avg: 29.0, max: 58.0) [2023-10-08 04:22:33,755][130385] Avg episode reward: [(0, '33.340'), (1, '25.490')] [2023-10-08 04:22:35,212][00611] Updated weights for policy 0, policy_version 11082 (0.0010) [2023-10-08 04:22:35,592][00611] Updated weights for policy 0, policy_version 11092 (0.0011) [2023-10-08 04:22:35,957][00611] Updated weights for policy 0, policy_version 11102 (0.0010) [2023-10-08 04:22:36,886][00612] Updated weights for policy 1, policy_version 11140 (0.0010) [2023-10-08 04:22:37,256][00612] Updated weights for policy 1, policy_version 11150 (0.0008) [2023-10-08 04:22:37,629][00612] Updated weights for policy 1, policy_version 11160 (0.0007) [2023-10-08 04:22:38,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 22806528. Throughput: 0: 1826.3, 1: 1851.0. Samples: 5703824. Policy #0 lag: (min: 26.0, avg: 29.0, max: 58.0) [2023-10-08 04:22:38,755][130385] Avg episode reward: [(0, '32.870'), (1, '25.800')] [2023-10-08 04:22:39,717][00611] Updated weights for policy 0, policy_version 11112 (0.0008) [2023-10-08 04:22:40,088][00611] Updated weights for policy 0, policy_version 11122 (0.0009) [2023-10-08 04:22:40,463][00611] Updated weights for policy 0, policy_version 11132 (0.0009) [2023-10-08 04:22:41,104][00612] Updated weights for policy 1, policy_version 11170 (0.0008) [2023-10-08 04:22:41,486][00612] Updated weights for policy 1, policy_version 11180 (0.0009) [2023-10-08 04:22:41,850][00612] Updated weights for policy 1, policy_version 11190 (0.0007) [2023-10-08 04:22:42,221][00612] Updated weights for policy 1, policy_version 11200 (0.0008) [2023-10-08 04:22:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 22872064. Throughput: 0: 1825.8, 1: 1842.9. Samples: 5725908. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 04:22:43,754][130385] Avg episode reward: [(0, '34.000'), (1, '26.000')] [2023-10-08 04:22:44,206][00611] Updated weights for policy 0, policy_version 11142 (0.0009) [2023-10-08 04:22:44,591][00611] Updated weights for policy 0, policy_version 11152 (0.0009) [2023-10-08 04:22:44,962][00611] Updated weights for policy 0, policy_version 11162 (0.0008) [2023-10-08 04:22:45,865][00612] Updated weights for policy 1, policy_version 11210 (0.0007) [2023-10-08 04:22:46,237][00612] Updated weights for policy 1, policy_version 11220 (0.0009) [2023-10-08 04:22:46,611][00612] Updated weights for policy 1, policy_version 11230 (0.0010) [2023-10-08 04:22:48,601][00611] Updated weights for policy 0, policy_version 11172 (0.0008) [2023-10-08 04:22:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 22937600. Throughput: 0: 1825.8, 1: 1859.5. Samples: 5748536. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 04:22:48,755][130385] Avg episode reward: [(0, '32.270'), (1, '26.530')] [2023-10-08 04:22:48,977][00611] Updated weights for policy 0, policy_version 11182 (0.0008) [2023-10-08 04:22:49,352][00611] Updated weights for policy 0, policy_version 11192 (0.0009) [2023-10-08 04:22:50,304][00612] Updated weights for policy 1, policy_version 11240 (0.0009) [2023-10-08 04:22:50,677][00612] Updated weights for policy 1, policy_version 11250 (0.0009) [2023-10-08 04:22:51,047][00612] Updated weights for policy 1, policy_version 11260 (0.0010) [2023-10-08 04:22:52,900][00611] Updated weights for policy 0, policy_version 11202 (0.0008) [2023-10-08 04:22:53,269][00611] Updated weights for policy 0, policy_version 11212 (0.0009) [2023-10-08 04:22:53,649][00611] Updated weights for policy 0, policy_version 11222 (0.0007) [2023-10-08 04:22:53,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 23003136. Throughput: 0: 1832.3, 1: 1836.0. Samples: 5758626. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 04:22:53,755][130385] Avg episode reward: [(0, '35.360'), (1, '27.250')] [2023-10-08 04:22:54,020][00611] Updated weights for policy 0, policy_version 11232 (0.0008) [2023-10-08 04:22:54,519][00612] Updated weights for policy 1, policy_version 11270 (0.0012) [2023-10-08 04:22:54,895][00612] Updated weights for policy 1, policy_version 11280 (0.0009) [2023-10-08 04:22:55,256][00612] Updated weights for policy 1, policy_version 11290 (0.0009) [2023-10-08 04:22:57,753][00611] Updated weights for policy 0, policy_version 11242 (0.0007) [2023-10-08 04:22:58,123][00611] Updated weights for policy 0, policy_version 11252 (0.0009) [2023-10-08 04:22:58,502][00611] Updated weights for policy 0, policy_version 11262 (0.0008) [2023-10-08 04:22:58,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 23101440. Throughput: 0: 1835.2, 1: 1852.3. Samples: 5781564. Policy #0 lag: (min: 17.0, avg: 28.3, max: 49.0) [2023-10-08 04:22:58,754][130385] Avg episode reward: [(0, '32.070'), (1, '27.150')] [2023-10-08 04:22:58,940][00612] Updated weights for policy 1, policy_version 11300 (0.0009) [2023-10-08 04:22:59,316][00612] Updated weights for policy 1, policy_version 11310 (0.0008) [2023-10-08 04:22:59,685][00612] Updated weights for policy 1, policy_version 11320 (0.0009) [2023-10-08 04:23:02,093][00611] Updated weights for policy 0, policy_version 11272 (0.0008) [2023-10-08 04:23:02,472][00611] Updated weights for policy 0, policy_version 11282 (0.0008) [2023-10-08 04:23:02,845][00611] Updated weights for policy 0, policy_version 11292 (0.0010) [2023-10-08 04:23:03,342][00612] Updated weights for policy 1, policy_version 11330 (0.0008) [2023-10-08 04:23:03,717][00612] Updated weights for policy 1, policy_version 11340 (0.0007) [2023-10-08 04:23:03,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 23166976. Throughput: 0: 1827.1, 1: 1851.5. Samples: 5803040. Policy #0 lag: (min: 17.0, avg: 28.3, max: 49.0) [2023-10-08 04:23:03,755][130385] Avg episode reward: [(0, '31.010'), (1, '27.420')] [2023-10-08 04:23:04,084][00612] Updated weights for policy 1, policy_version 11350 (0.0009) [2023-10-08 04:23:04,455][00612] Updated weights for policy 1, policy_version 11360 (0.0010) [2023-10-08 04:23:06,474][00611] Updated weights for policy 0, policy_version 11302 (0.0007) [2023-10-08 04:23:06,846][00611] Updated weights for policy 0, policy_version 11312 (0.0008) [2023-10-08 04:23:07,214][00611] Updated weights for policy 0, policy_version 11322 (0.0007) [2023-10-08 04:23:08,009][00612] Updated weights for policy 1, policy_version 11370 (0.0008) [2023-10-08 04:23:08,370][00612] Updated weights for policy 1, policy_version 11380 (0.0007) [2023-10-08 04:23:08,739][00612] Updated weights for policy 1, policy_version 11390 (0.0007) [2023-10-08 04:23:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 23232512. Throughput: 0: 1837.5, 1: 1854.0. Samples: 5814872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:23:08,754][130385] Avg episode reward: [(0, '31.040'), (1, '26.190')] [2023-10-08 04:23:10,853][00611] Updated weights for policy 0, policy_version 11332 (0.0009) [2023-10-08 04:23:11,211][00611] Updated weights for policy 0, policy_version 11342 (0.0009) [2023-10-08 04:23:11,580][00611] Updated weights for policy 0, policy_version 11352 (0.0008) [2023-10-08 04:23:12,355][00612] Updated weights for policy 1, policy_version 11400 (0.0010) [2023-10-08 04:23:12,735][00612] Updated weights for policy 1, policy_version 11410 (0.0008) [2023-10-08 04:23:13,106][00612] Updated weights for policy 1, policy_version 11420 (0.0008) [2023-10-08 04:23:13,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 23330816. Throughput: 0: 1833.2, 1: 1843.2. Samples: 5836198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:23:13,754][130385] Avg episode reward: [(0, '31.100'), (1, '26.330')] [2023-10-08 04:23:15,092][00611] Updated weights for policy 0, policy_version 11362 (0.0009) [2023-10-08 04:23:15,462][00611] Updated weights for policy 0, policy_version 11372 (0.0008) [2023-10-08 04:23:15,843][00611] Updated weights for policy 0, policy_version 11382 (0.0008) [2023-10-08 04:23:16,214][00611] Updated weights for policy 0, policy_version 11392 (0.0009) [2023-10-08 04:23:16,611][00612] Updated weights for policy 1, policy_version 11430 (0.0008) [2023-10-08 04:23:16,984][00612] Updated weights for policy 1, policy_version 11440 (0.0007) [2023-10-08 04:23:17,361][00612] Updated weights for policy 1, policy_version 11450 (0.0007) [2023-10-08 04:23:18,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 23396352. Throughput: 0: 1839.8, 1: 1845.1. Samples: 5858316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:23:18,755][130385] Avg episode reward: [(0, '32.970'), (1, '25.170')] [2023-10-08 04:23:19,949][00611] Updated weights for policy 0, policy_version 11402 (0.0008) [2023-10-08 04:23:20,321][00611] Updated weights for policy 0, policy_version 11412 (0.0008) [2023-10-08 04:23:20,689][00611] Updated weights for policy 0, policy_version 11422 (0.0009) [2023-10-08 04:23:21,078][00612] Updated weights for policy 1, policy_version 11460 (0.0009) [2023-10-08 04:23:21,447][00612] Updated weights for policy 1, policy_version 11470 (0.0008) [2023-10-08 04:23:21,811][00612] Updated weights for policy 1, policy_version 11480 (0.0011) [2023-10-08 04:23:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 23461888. Throughput: 0: 1838.1, 1: 1840.9. Samples: 5869380. Policy #0 lag: (min: 24.0, avg: 49.9, max: 56.0) [2023-10-08 04:23:23,754][130385] Avg episode reward: [(0, '33.290'), (1, '26.480')] [2023-10-08 04:23:24,292][00611] Updated weights for policy 0, policy_version 11432 (0.0009) [2023-10-08 04:23:24,660][00611] Updated weights for policy 0, policy_version 11442 (0.0007) [2023-10-08 04:23:25,035][00611] Updated weights for policy 0, policy_version 11452 (0.0008) [2023-10-08 04:23:25,497][00612] Updated weights for policy 1, policy_version 11490 (0.0011) [2023-10-08 04:23:25,861][00612] Updated weights for policy 1, policy_version 11500 (0.0009) [2023-10-08 04:23:26,232][00612] Updated weights for policy 1, policy_version 11510 (0.0008) [2023-10-08 04:23:26,596][00612] Updated weights for policy 1, policy_version 11520 (0.0009) [2023-10-08 04:23:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 23527424. Throughput: 0: 1836.9, 1: 1838.7. Samples: 5891312. Policy #0 lag: (min: 24.0, avg: 49.9, max: 56.0) [2023-10-08 04:23:28,755][130385] Avg episode reward: [(0, '32.570'), (1, '25.840')] [2023-10-08 04:23:28,797][00611] Updated weights for policy 0, policy_version 11462 (0.0009) [2023-10-08 04:23:29,173][00611] Updated weights for policy 0, policy_version 11472 (0.0008) [2023-10-08 04:23:29,543][00611] Updated weights for policy 0, policy_version 11482 (0.0007) [2023-10-08 04:23:30,253][00612] Updated weights for policy 1, policy_version 11530 (0.0007) [2023-10-08 04:23:30,627][00612] Updated weights for policy 1, policy_version 11540 (0.0008) [2023-10-08 04:23:30,986][00612] Updated weights for policy 1, policy_version 11550 (0.0010) [2023-10-08 04:23:33,373][00611] Updated weights for policy 0, policy_version 11492 (0.0009) [2023-10-08 04:23:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 23592960. Throughput: 0: 1834.7, 1: 1843.6. Samples: 5914056. Policy #0 lag: (min: 24.0, avg: 49.9, max: 56.0) [2023-10-08 04:23:33,755][130385] Avg episode reward: [(0, '34.820'), (1, '28.250')] [2023-10-08 04:23:33,760][00611] Updated weights for policy 0, policy_version 11502 (0.0009) [2023-10-08 04:23:34,137][00611] Updated weights for policy 0, policy_version 11512 (0.0008) [2023-10-08 04:23:34,772][00612] Updated weights for policy 1, policy_version 11560 (0.0007) [2023-10-08 04:23:35,152][00612] Updated weights for policy 1, policy_version 11570 (0.0008) [2023-10-08 04:23:35,515][00612] Updated weights for policy 1, policy_version 11580 (0.0008) [2023-10-08 04:23:37,709][00611] Updated weights for policy 0, policy_version 11522 (0.0007) [2023-10-08 04:23:38,072][00611] Updated weights for policy 0, policy_version 11532 (0.0008) [2023-10-08 04:23:38,440][00611] Updated weights for policy 0, policy_version 11542 (0.0009) [2023-10-08 04:23:38,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 23658496. Throughput: 0: 1831.3, 1: 1842.8. Samples: 5923958. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 04:23:38,754][130385] Avg episode reward: [(0, '34.490'), (1, '29.960')] [2023-10-08 04:23:38,808][00611] Updated weights for policy 0, policy_version 11552 (0.0011) [2023-10-08 04:23:39,129][00612] Updated weights for policy 1, policy_version 11590 (0.0008) [2023-10-08 04:23:39,491][00612] Updated weights for policy 1, policy_version 11600 (0.0007) [2023-10-08 04:23:39,862][00612] Updated weights for policy 1, policy_version 11610 (0.0008) [2023-10-08 04:23:42,469][00611] Updated weights for policy 0, policy_version 11562 (0.0010) [2023-10-08 04:23:42,832][00611] Updated weights for policy 0, policy_version 11572 (0.0009) [2023-10-08 04:23:43,202][00611] Updated weights for policy 0, policy_version 11582 (0.0007) [2023-10-08 04:23:43,401][00612] Updated weights for policy 1, policy_version 11620 (0.0010) [2023-10-08 04:23:43,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 23756800. Throughput: 0: 1832.3, 1: 1843.1. Samples: 5946958. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:23:43,754][130385] Avg episode reward: [(0, '32.890'), (1, '29.460')] [2023-10-08 04:23:43,781][00612] Updated weights for policy 1, policy_version 11630 (0.0008) [2023-10-08 04:23:44,142][00612] Updated weights for policy 1, policy_version 11640 (0.0008) [2023-10-08 04:23:46,885][00611] Updated weights for policy 0, policy_version 11592 (0.0007) [2023-10-08 04:23:47,257][00611] Updated weights for policy 0, policy_version 11602 (0.0009) [2023-10-08 04:23:47,623][00611] Updated weights for policy 0, policy_version 11612 (0.0008) [2023-10-08 04:23:47,724][00612] Updated weights for policy 1, policy_version 11650 (0.0008) [2023-10-08 04:23:48,085][00612] Updated weights for policy 1, policy_version 11660 (0.0009) [2023-10-08 04:23:48,461][00612] Updated weights for policy 1, policy_version 11670 (0.0008) [2023-10-08 04:23:48,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 23822336. Throughput: 0: 1832.8, 1: 1836.6. Samples: 5968164. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:23:48,754][130385] Avg episode reward: [(0, '31.940'), (1, '29.610')] [2023-10-08 04:23:48,826][00612] Updated weights for policy 1, policy_version 11680 (0.0007) [2023-10-08 04:23:51,252][00611] Updated weights for policy 0, policy_version 11622 (0.0009) [2023-10-08 04:23:51,620][00611] Updated weights for policy 0, policy_version 11632 (0.0009) [2023-10-08 04:23:51,992][00611] Updated weights for policy 0, policy_version 11642 (0.0008) [2023-10-08 04:23:52,564][00612] Updated weights for policy 1, policy_version 11690 (0.0007) [2023-10-08 04:23:52,930][00612] Updated weights for policy 1, policy_version 11700 (0.0009) [2023-10-08 04:23:53,297][00612] Updated weights for policy 1, policy_version 11710 (0.0008) [2023-10-08 04:23:53,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 23920640. Throughput: 0: 1829.5, 1: 1845.5. Samples: 5980246. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:23:53,754][130385] Avg episode reward: [(0, '32.210'), (1, '31.580')] [2023-10-08 04:23:55,578][00611] Updated weights for policy 0, policy_version 11652 (0.0008) [2023-10-08 04:23:55,947][00611] Updated weights for policy 0, policy_version 11662 (0.0008) [2023-10-08 04:23:56,320][00611] Updated weights for policy 0, policy_version 11672 (0.0009) [2023-10-08 04:23:56,946][00612] Updated weights for policy 1, policy_version 11720 (0.0007) [2023-10-08 04:23:57,305][00612] Updated weights for policy 1, policy_version 11730 (0.0009) [2023-10-08 04:23:57,682][00612] Updated weights for policy 1, policy_version 11740 (0.0008) [2023-10-08 04:23:58,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 23986176. Throughput: 0: 1834.0, 1: 1838.1. Samples: 6001444. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-08 04:23:58,754][130385] Avg episode reward: [(0, '31.410'), (1, '31.630')] [2023-10-08 04:23:59,927][00611] Updated weights for policy 0, policy_version 11682 (0.0009) [2023-10-08 04:24:00,304][00611] Updated weights for policy 0, policy_version 11692 (0.0007) [2023-10-08 04:24:00,668][00611] Updated weights for policy 0, policy_version 11702 (0.0007) [2023-10-08 04:24:01,041][00611] Updated weights for policy 0, policy_version 11712 (0.0008) [2023-10-08 04:24:01,293][00612] Updated weights for policy 1, policy_version 11750 (0.0008) [2023-10-08 04:24:01,664][00612] Updated weights for policy 1, policy_version 11760 (0.0009) [2023-10-08 04:24:02,045][00612] Updated weights for policy 1, policy_version 11770 (0.0010) [2023-10-08 04:24:03,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 24051712. Throughput: 0: 1834.1, 1: 1852.1. Samples: 6024192. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-08 04:24:03,754][130385] Avg episode reward: [(0, '33.390'), (1, '32.340')] [2023-10-08 04:24:04,602][00611] Updated weights for policy 0, policy_version 11722 (0.0011) [2023-10-08 04:24:04,973][00611] Updated weights for policy 0, policy_version 11732 (0.0010) [2023-10-08 04:24:05,352][00611] Updated weights for policy 0, policy_version 11742 (0.0010) [2023-10-08 04:24:05,457][00612] Updated weights for policy 1, policy_version 11780 (0.0008) [2023-10-08 04:24:05,834][00612] Updated weights for policy 1, policy_version 11790 (0.0010) [2023-10-08 04:24:06,203][00612] Updated weights for policy 1, policy_version 11800 (0.0007) [2023-10-08 04:24:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 24117248. Throughput: 0: 1836.8, 1: 1837.6. Samples: 6034732. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-08 04:24:08,755][130385] Avg episode reward: [(0, '34.020'), (1, '32.810')] [2023-10-08 04:24:09,092][00611] Updated weights for policy 0, policy_version 11752 (0.0008) [2023-10-08 04:24:09,466][00611] Updated weights for policy 0, policy_version 11762 (0.0008) [2023-10-08 04:24:09,777][00612] Updated weights for policy 1, policy_version 11810 (0.0008) [2023-10-08 04:24:09,849][00611] Updated weights for policy 0, policy_version 11772 (0.0008) [2023-10-08 04:24:10,140][00612] Updated weights for policy 1, policy_version 11820 (0.0010) [2023-10-08 04:24:10,515][00612] Updated weights for policy 1, policy_version 11830 (0.0008) [2023-10-08 04:24:10,886][00612] Updated weights for policy 1, policy_version 11840 (0.0011) [2023-10-08 04:24:13,420][00611] Updated weights for policy 0, policy_version 11782 (0.0007) [2023-10-08 04:24:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 24182784. Throughput: 0: 1839.0, 1: 1858.3. Samples: 6057692. Policy #0 lag: (min: 9.0, avg: 15.8, max: 41.0) [2023-10-08 04:24:13,754][130385] Avg episode reward: [(0, '33.950'), (1, '33.960')] [2023-10-08 04:24:13,796][00611] Updated weights for policy 0, policy_version 11792 (0.0007) [2023-10-08 04:24:14,167][00611] Updated weights for policy 0, policy_version 11802 (0.0008) [2023-10-08 04:24:14,415][00612] Updated weights for policy 1, policy_version 11850 (0.0009) [2023-10-08 04:24:14,780][00612] Updated weights for policy 1, policy_version 11860 (0.0009) [2023-10-08 04:24:15,147][00612] Updated weights for policy 1, policy_version 11870 (0.0012) [2023-10-08 04:24:17,904][00611] Updated weights for policy 0, policy_version 11812 (0.0008) [2023-10-08 04:24:18,276][00611] Updated weights for policy 0, policy_version 11822 (0.0009) [2023-10-08 04:24:18,646][00611] Updated weights for policy 0, policy_version 11832 (0.0010) [2023-10-08 04:24:18,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 24248320. Throughput: 0: 1832.6, 1: 1860.8. Samples: 6080260. Policy #0 lag: (min: 9.0, avg: 15.8, max: 41.0) [2023-10-08 04:24:18,754][130385] Avg episode reward: [(0, '33.650'), (1, '34.850')] [2023-10-08 04:24:18,941][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000011840_12124160.pth... [2023-10-08 04:24:18,971][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000010112_10354688.pth [2023-10-08 04:24:19,007][00612] Updated weights for policy 1, policy_version 11880 (0.0009) [2023-10-08 04:24:19,375][00612] Updated weights for policy 1, policy_version 11890 (0.0009) [2023-10-08 04:24:19,759][00612] Updated weights for policy 1, policy_version 11900 (0.0010) [2023-10-08 04:24:19,901][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000011904_12189696.pth... [2023-10-08 04:24:19,930][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000010176_10420224.pth [2023-10-08 04:24:19,933][00425] Saving new best policy, reward=34.850! [2023-10-08 04:24:22,303][00611] Updated weights for policy 0, policy_version 11842 (0.0008) [2023-10-08 04:24:22,671][00611] Updated weights for policy 0, policy_version 11852 (0.0007) [2023-10-08 04:24:23,044][00611] Updated weights for policy 0, policy_version 11862 (0.0007) [2023-10-08 04:24:23,315][00612] Updated weights for policy 1, policy_version 11910 (0.0009) [2023-10-08 04:24:23,418][00611] Updated weights for policy 0, policy_version 11872 (0.0008) [2023-10-08 04:24:23,693][00612] Updated weights for policy 1, policy_version 11920 (0.0009) [2023-10-08 04:24:23,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 24346624. Throughput: 0: 1846.0, 1: 1856.0. Samples: 6090544. Policy #0 lag: (min: 26.0, avg: 39.5, max: 58.0) [2023-10-08 04:24:23,754][130385] Avg episode reward: [(0, '34.520'), (1, '36.000')] [2023-10-08 04:24:24,052][00612] Updated weights for policy 1, policy_version 11930 (0.0007) [2023-10-08 04:24:24,273][00425] Saving new best policy, reward=36.000! [2023-10-08 04:24:27,011][00611] Updated weights for policy 0, policy_version 11882 (0.0007) [2023-10-08 04:24:27,388][00611] Updated weights for policy 0, policy_version 11892 (0.0010) [2023-10-08 04:24:27,724][00612] Updated weights for policy 1, policy_version 11940 (0.0007) [2023-10-08 04:24:27,749][00611] Updated weights for policy 0, policy_version 11902 (0.0007) [2023-10-08 04:24:28,099][00612] Updated weights for policy 1, policy_version 11950 (0.0009) [2023-10-08 04:24:28,467][00612] Updated weights for policy 1, policy_version 11960 (0.0007) [2023-10-08 04:24:28,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 24412160. Throughput: 0: 1833.8, 1: 1855.4. Samples: 6112974. Policy #0 lag: (min: 26.0, avg: 39.5, max: 58.0) [2023-10-08 04:24:28,754][130385] Avg episode reward: [(0, '32.640'), (1, '35.450')] [2023-10-08 04:24:31,349][00611] Updated weights for policy 0, policy_version 11912 (0.0010) [2023-10-08 04:24:31,719][00611] Updated weights for policy 0, policy_version 11922 (0.0008) [2023-10-08 04:24:32,087][00611] Updated weights for policy 0, policy_version 11932 (0.0009) [2023-10-08 04:24:32,173][00612] Updated weights for policy 1, policy_version 11970 (0.0007) [2023-10-08 04:24:32,532][00612] Updated weights for policy 1, policy_version 11980 (0.0009) [2023-10-08 04:24:32,908][00612] Updated weights for policy 1, policy_version 11990 (0.0008) [2023-10-08 04:24:33,275][00612] Updated weights for policy 1, policy_version 12000 (0.0008) [2023-10-08 04:24:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 24510464. Throughput: 0: 1846.7, 1: 1828.9. Samples: 6133566. Policy #0 lag: (min: 26.0, avg: 39.5, max: 58.0) [2023-10-08 04:24:33,754][130385] Avg episode reward: [(0, '35.520'), (1, '36.450')] [2023-10-08 04:24:33,765][00365] Saving new best policy, reward=35.520! [2023-10-08 04:24:33,765][00425] Saving new best policy, reward=36.450! [2023-10-08 04:24:35,745][00611] Updated weights for policy 0, policy_version 11942 (0.0010) [2023-10-08 04:24:36,115][00611] Updated weights for policy 0, policy_version 11952 (0.0008) [2023-10-08 04:24:36,496][00611] Updated weights for policy 0, policy_version 11962 (0.0009) [2023-10-08 04:24:36,967][00612] Updated weights for policy 1, policy_version 12010 (0.0007) [2023-10-08 04:24:37,329][00612] Updated weights for policy 1, policy_version 12020 (0.0009) [2023-10-08 04:24:37,695][00612] Updated weights for policy 1, policy_version 12030 (0.0012) [2023-10-08 04:24:38,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 24576000. Throughput: 0: 1829.0, 1: 1847.1. Samples: 6145670. Policy #0 lag: (min: 27.0, avg: 28.1, max: 50.0) [2023-10-08 04:24:38,754][130385] Avg episode reward: [(0, '33.850'), (1, '36.830')] [2023-10-08 04:24:38,755][00425] Saving new best policy, reward=36.830! [2023-10-08 04:24:40,011][00611] Updated weights for policy 0, policy_version 11972 (0.0009) [2023-10-08 04:24:40,372][00611] Updated weights for policy 0, policy_version 11982 (0.0008) [2023-10-08 04:24:40,744][00611] Updated weights for policy 0, policy_version 11992 (0.0009) [2023-10-08 04:24:41,382][00612] Updated weights for policy 1, policy_version 12040 (0.0008) [2023-10-08 04:24:41,749][00612] Updated weights for policy 1, policy_version 12050 (0.0008) [2023-10-08 04:24:42,114][00612] Updated weights for policy 1, policy_version 12060 (0.0008) [2023-10-08 04:24:43,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 24641536. Throughput: 0: 1850.1, 1: 1828.7. Samples: 6166990. Policy #0 lag: (min: 27.0, avg: 28.1, max: 50.0) [2023-10-08 04:24:43,754][130385] Avg episode reward: [(0, '34.400'), (1, '36.560')] [2023-10-08 04:24:44,372][00611] Updated weights for policy 0, policy_version 12002 (0.0008) [2023-10-08 04:24:44,753][00611] Updated weights for policy 0, policy_version 12012 (0.0009) [2023-10-08 04:24:45,124][00611] Updated weights for policy 0, policy_version 12022 (0.0008) [2023-10-08 04:24:45,494][00611] Updated weights for policy 0, policy_version 12032 (0.0009) [2023-10-08 04:24:45,772][00612] Updated weights for policy 1, policy_version 12070 (0.0008) [2023-10-08 04:24:46,136][00612] Updated weights for policy 1, policy_version 12080 (0.0008) [2023-10-08 04:24:46,512][00612] Updated weights for policy 1, policy_version 12090 (0.0008) [2023-10-08 04:24:48,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 24707072. Throughput: 0: 1843.3, 1: 1839.2. Samples: 6189908. Policy #0 lag: (min: 27.0, avg: 28.1, max: 50.0) [2023-10-08 04:24:48,755][130385] Avg episode reward: [(0, '33.250'), (1, '35.480')] [2023-10-08 04:24:49,294][00611] Updated weights for policy 0, policy_version 12042 (0.0008) [2023-10-08 04:24:49,672][00611] Updated weights for policy 0, policy_version 12052 (0.0007) [2023-10-08 04:24:50,049][00611] Updated weights for policy 0, policy_version 12062 (0.0009) [2023-10-08 04:24:50,127][00612] Updated weights for policy 1, policy_version 12100 (0.0010) [2023-10-08 04:24:50,499][00612] Updated weights for policy 1, policy_version 12110 (0.0008) [2023-10-08 04:24:50,867][00612] Updated weights for policy 1, policy_version 12120 (0.0008) [2023-10-08 04:24:53,698][00611] Updated weights for policy 0, policy_version 12072 (0.0009) [2023-10-08 04:24:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 24772608. Throughput: 0: 1841.0, 1: 1830.2. Samples: 6199938. Policy #0 lag: (min: 13.0, avg: 14.6, max: 41.0) [2023-10-08 04:24:53,754][130385] Avg episode reward: [(0, '30.790'), (1, '35.120')] [2023-10-08 04:24:54,066][00611] Updated weights for policy 0, policy_version 12082 (0.0010) [2023-10-08 04:24:54,436][00611] Updated weights for policy 0, policy_version 12092 (0.0007) [2023-10-08 04:24:54,524][00612] Updated weights for policy 1, policy_version 12130 (0.0007) [2023-10-08 04:24:54,893][00612] Updated weights for policy 1, policy_version 12140 (0.0008) [2023-10-08 04:24:55,271][00612] Updated weights for policy 1, policy_version 12150 (0.0011) [2023-10-08 04:24:55,633][00612] Updated weights for policy 1, policy_version 12160 (0.0008) [2023-10-08 04:24:58,071][00611] Updated weights for policy 0, policy_version 12102 (0.0008) [2023-10-08 04:24:58,441][00611] Updated weights for policy 0, policy_version 12112 (0.0007) [2023-10-08 04:24:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 24838144. Throughput: 0: 1834.8, 1: 1845.0. Samples: 6223282. Policy #0 lag: (min: 13.0, avg: 14.6, max: 41.0) [2023-10-08 04:24:58,755][130385] Avg episode reward: [(0, '30.730'), (1, '35.370')] [2023-10-08 04:24:58,818][00611] Updated weights for policy 0, policy_version 12122 (0.0008) [2023-10-08 04:24:59,025][00612] Updated weights for policy 1, policy_version 12170 (0.0009) [2023-10-08 04:24:59,394][00612] Updated weights for policy 1, policy_version 12180 (0.0007) [2023-10-08 04:24:59,769][00612] Updated weights for policy 1, policy_version 12190 (0.0007) [2023-10-08 04:25:02,515][00611] Updated weights for policy 0, policy_version 12132 (0.0010) [2023-10-08 04:25:02,883][00611] Updated weights for policy 0, policy_version 12142 (0.0009) [2023-10-08 04:25:03,254][00611] Updated weights for policy 0, policy_version 12152 (0.0008) [2023-10-08 04:25:03,430][00612] Updated weights for policy 1, policy_version 12200 (0.0009) [2023-10-08 04:25:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 24936448. Throughput: 0: 1819.5, 1: 1844.5. Samples: 6245138. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-08 04:25:03,754][130385] Avg episode reward: [(0, '30.110'), (1, '35.030')] [2023-10-08 04:25:03,796][00612] Updated weights for policy 1, policy_version 12210 (0.0011) [2023-10-08 04:25:04,158][00612] Updated weights for policy 1, policy_version 12220 (0.0009) [2023-10-08 04:25:07,060][00611] Updated weights for policy 0, policy_version 12162 (0.0009) [2023-10-08 04:25:07,463][00611] Updated weights for policy 0, policy_version 12172 (0.0010) [2023-10-08 04:25:07,829][00611] Updated weights for policy 0, policy_version 12182 (0.0009) [2023-10-08 04:25:07,940][00612] Updated weights for policy 1, policy_version 12230 (0.0009) [2023-10-08 04:25:08,204][00611] Updated weights for policy 0, policy_version 12192 (0.0009) [2023-10-08 04:25:08,330][00612] Updated weights for policy 1, policy_version 12240 (0.0009) [2023-10-08 04:25:08,699][00612] Updated weights for policy 1, policy_version 12250 (0.0008) [2023-10-08 04:25:08,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 25001984. Throughput: 0: 1830.1, 1: 1849.2. Samples: 6256114. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-08 04:25:08,754][130385] Avg episode reward: [(0, '29.940'), (1, '35.050')] [2023-10-08 04:25:11,857][00611] Updated weights for policy 0, policy_version 12202 (0.0007) [2023-10-08 04:25:12,218][00611] Updated weights for policy 0, policy_version 12212 (0.0009) [2023-10-08 04:25:12,285][00612] Updated weights for policy 1, policy_version 12260 (0.0007) [2023-10-08 04:25:12,589][00611] Updated weights for policy 0, policy_version 12222 (0.0008) [2023-10-08 04:25:12,657][00612] Updated weights for policy 1, policy_version 12270 (0.0008) [2023-10-08 04:25:13,016][00612] Updated weights for policy 1, policy_version 12280 (0.0007) [2023-10-08 04:25:13,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 25100288. Throughput: 0: 1824.8, 1: 1849.2. Samples: 6278308. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-08 04:25:13,754][130385] Avg episode reward: [(0, '30.180'), (1, '35.010')] [2023-10-08 04:25:16,198][00611] Updated weights for policy 0, policy_version 12232 (0.0009) [2023-10-08 04:25:16,574][00611] Updated weights for policy 0, policy_version 12242 (0.0009) [2023-10-08 04:25:16,714][00612] Updated weights for policy 1, policy_version 12290 (0.0007) [2023-10-08 04:25:16,944][00611] Updated weights for policy 0, policy_version 12252 (0.0008) [2023-10-08 04:25:17,076][00612] Updated weights for policy 1, policy_version 12300 (0.0008) [2023-10-08 04:25:17,444][00612] Updated weights for policy 1, policy_version 12310 (0.0007) [2023-10-08 04:25:17,812][00612] Updated weights for policy 1, policy_version 12320 (0.0008) [2023-10-08 04:25:18,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 25165824. Throughput: 0: 1832.3, 1: 1845.9. Samples: 6299090. Policy #0 lag: (min: 24.0, avg: 39.1, max: 40.0) [2023-10-08 04:25:18,755][130385] Avg episode reward: [(0, '30.240'), (1, '35.020')] [2023-10-08 04:25:20,486][00611] Updated weights for policy 0, policy_version 12262 (0.0008) [2023-10-08 04:25:20,854][00611] Updated weights for policy 0, policy_version 12272 (0.0008) [2023-10-08 04:25:21,227][00611] Updated weights for policy 0, policy_version 12282 (0.0008) [2023-10-08 04:25:21,435][00612] Updated weights for policy 1, policy_version 12330 (0.0008) [2023-10-08 04:25:21,799][00612] Updated weights for policy 1, policy_version 12340 (0.0007) [2023-10-08 04:25:22,170][00612] Updated weights for policy 1, policy_version 12350 (0.0010) [2023-10-08 04:25:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 25231360. Throughput: 0: 1827.7, 1: 1848.0. Samples: 6311078. Policy #0 lag: (min: 24.0, avg: 39.1, max: 40.0) [2023-10-08 04:25:23,754][130385] Avg episode reward: [(0, '31.790'), (1, '34.370')] [2023-10-08 04:25:24,963][00611] Updated weights for policy 0, policy_version 12292 (0.0007) [2023-10-08 04:25:25,351][00611] Updated weights for policy 0, policy_version 12302 (0.0011) [2023-10-08 04:25:25,720][00611] Updated weights for policy 0, policy_version 12312 (0.0008) [2023-10-08 04:25:25,768][00612] Updated weights for policy 1, policy_version 12360 (0.0007) [2023-10-08 04:25:26,133][00612] Updated weights for policy 1, policy_version 12370 (0.0008) [2023-10-08 04:25:26,503][00612] Updated weights for policy 1, policy_version 12380 (0.0007) [2023-10-08 04:25:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 25296896. Throughput: 0: 1826.1, 1: 1851.5. Samples: 6332480. Policy #0 lag: (min: 24.0, avg: 39.1, max: 40.0) [2023-10-08 04:25:28,755][130385] Avg episode reward: [(0, '32.880'), (1, '35.490')] [2023-10-08 04:25:29,405][00611] Updated weights for policy 0, policy_version 12322 (0.0008) [2023-10-08 04:25:29,777][00611] Updated weights for policy 0, policy_version 12332 (0.0007) [2023-10-08 04:25:29,941][00612] Updated weights for policy 1, policy_version 12390 (0.0007) [2023-10-08 04:25:30,152][00611] Updated weights for policy 0, policy_version 12342 (0.0008) [2023-10-08 04:25:30,306][00612] Updated weights for policy 1, policy_version 12400 (0.0009) [2023-10-08 04:25:30,525][00611] Updated weights for policy 0, policy_version 12352 (0.0008) [2023-10-08 04:25:30,673][00612] Updated weights for policy 1, policy_version 12410 (0.0007) [2023-10-08 04:25:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 25362432. Throughput: 0: 1826.7, 1: 1859.1. Samples: 6355766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:25:33,754][130385] Avg episode reward: [(0, '34.140'), (1, '37.490')] [2023-10-08 04:25:33,762][00425] Saving new best policy, reward=37.490! [2023-10-08 04:25:34,191][00611] Updated weights for policy 0, policy_version 12362 (0.0010) [2023-10-08 04:25:34,374][00612] Updated weights for policy 1, policy_version 12420 (0.0008) [2023-10-08 04:25:34,559][00611] Updated weights for policy 0, policy_version 12372 (0.0008) [2023-10-08 04:25:34,747][00612] Updated weights for policy 1, policy_version 12430 (0.0007) [2023-10-08 04:25:34,928][00611] Updated weights for policy 0, policy_version 12382 (0.0008) [2023-10-08 04:25:35,102][00612] Updated weights for policy 1, policy_version 12440 (0.0009) [2023-10-08 04:25:38,585][00611] Updated weights for policy 0, policy_version 12392 (0.0008) [2023-10-08 04:25:38,647][00612] Updated weights for policy 1, policy_version 12450 (0.0010) [2023-10-08 04:25:38,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 25427968. Throughput: 0: 1829.4, 1: 1853.8. Samples: 6365680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:25:38,754][130385] Avg episode reward: [(0, '34.800'), (1, '37.520')] [2023-10-08 04:25:38,955][00611] Updated weights for policy 0, policy_version 12402 (0.0008) [2023-10-08 04:25:39,013][00612] Updated weights for policy 1, policy_version 12460 (0.0007) [2023-10-08 04:25:39,331][00611] Updated weights for policy 0, policy_version 12412 (0.0008) [2023-10-08 04:25:39,383][00612] Updated weights for policy 1, policy_version 12470 (0.0007) [2023-10-08 04:25:39,743][00425] Saving new best policy, reward=37.520! [2023-10-08 04:25:39,744][00612] Updated weights for policy 1, policy_version 12480 (0.0010) [2023-10-08 04:25:42,867][00611] Updated weights for policy 0, policy_version 12422 (0.0009) [2023-10-08 04:25:43,246][00611] Updated weights for policy 0, policy_version 12432 (0.0007) [2023-10-08 04:25:43,490][00612] Updated weights for policy 1, policy_version 12490 (0.0007) [2023-10-08 04:25:43,612][00611] Updated weights for policy 0, policy_version 12442 (0.0007) [2023-10-08 04:25:43,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 25493504. Throughput: 0: 1825.5, 1: 1848.3. Samples: 6388602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:25:43,754][130385] Avg episode reward: [(0, '34.900'), (1, '35.210')] [2023-10-08 04:25:43,862][00612] Updated weights for policy 1, policy_version 12500 (0.0008) [2023-10-08 04:25:44,222][00612] Updated weights for policy 1, policy_version 12510 (0.0009) [2023-10-08 04:25:47,282][00611] Updated weights for policy 0, policy_version 12452 (0.0007) [2023-10-08 04:25:47,645][00611] Updated weights for policy 0, policy_version 12462 (0.0008) [2023-10-08 04:25:47,876][00612] Updated weights for policy 1, policy_version 12520 (0.0008) [2023-10-08 04:25:48,021][00611] Updated weights for policy 0, policy_version 12472 (0.0007) [2023-10-08 04:25:48,248][00612] Updated weights for policy 1, policy_version 12530 (0.0008) [2023-10-08 04:25:48,624][00612] Updated weights for policy 1, policy_version 12540 (0.0009) [2023-10-08 04:25:48,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 25591808. Throughput: 0: 1825.6, 1: 1830.0. Samples: 6409640. Policy #0 lag: (min: 23.0, avg: 39.2, max: 40.0) [2023-10-08 04:25:48,754][130385] Avg episode reward: [(0, '35.490'), (1, '34.400')] [2023-10-08 04:25:51,673][00611] Updated weights for policy 0, policy_version 12482 (0.0008) [2023-10-08 04:25:52,050][00611] Updated weights for policy 0, policy_version 12492 (0.0010) [2023-10-08 04:25:52,316][00612] Updated weights for policy 1, policy_version 12550 (0.0008) [2023-10-08 04:25:52,418][00611] Updated weights for policy 0, policy_version 12502 (0.0008) [2023-10-08 04:25:52,688][00612] Updated weights for policy 1, policy_version 12560 (0.0007) [2023-10-08 04:25:52,783][00611] Updated weights for policy 0, policy_version 12512 (0.0007) [2023-10-08 04:25:53,067][00612] Updated weights for policy 1, policy_version 12570 (0.0008) [2023-10-08 04:25:53,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 25690112. Throughput: 0: 1829.0, 1: 1846.0. Samples: 6421488. Policy #0 lag: (min: 23.0, avg: 39.2, max: 40.0) [2023-10-08 04:25:53,754][130385] Avg episode reward: [(0, '37.480'), (1, '34.760')] [2023-10-08 04:25:53,755][00365] Saving new best policy, reward=37.480! [2023-10-08 04:25:56,391][00611] Updated weights for policy 0, policy_version 12522 (0.0008) [2023-10-08 04:25:56,762][00611] Updated weights for policy 0, policy_version 12532 (0.0009) [2023-10-08 04:25:56,825][00612] Updated weights for policy 1, policy_version 12580 (0.0007) [2023-10-08 04:25:57,141][00611] Updated weights for policy 0, policy_version 12542 (0.0008) [2023-10-08 04:25:57,201][00612] Updated weights for policy 1, policy_version 12590 (0.0008) [2023-10-08 04:25:57,573][00612] Updated weights for policy 1, policy_version 12600 (0.0010) [2023-10-08 04:25:58,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 25755648. Throughput: 0: 1819.2, 1: 1825.3. Samples: 6442310. Policy #0 lag: (min: 17.0, avg: 22.6, max: 49.0) [2023-10-08 04:25:58,755][130385] Avg episode reward: [(0, '36.590'), (1, '33.590')] [2023-10-08 04:26:01,032][00611] Updated weights for policy 0, policy_version 12552 (0.0009) [2023-10-08 04:26:01,128][00612] Updated weights for policy 1, policy_version 12610 (0.0008) [2023-10-08 04:26:01,394][00611] Updated weights for policy 0, policy_version 12562 (0.0008) [2023-10-08 04:26:01,497][00612] Updated weights for policy 1, policy_version 12620 (0.0009) [2023-10-08 04:26:01,765][00611] Updated weights for policy 0, policy_version 12572 (0.0008) [2023-10-08 04:26:01,858][00612] Updated weights for policy 1, policy_version 12630 (0.0008) [2023-10-08 04:26:02,224][00612] Updated weights for policy 1, policy_version 12640 (0.0008) [2023-10-08 04:26:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 25821184. Throughput: 0: 1824.9, 1: 1849.7. Samples: 6464448. Policy #0 lag: (min: 17.0, avg: 22.6, max: 49.0) [2023-10-08 04:26:03,754][130385] Avg episode reward: [(0, '35.560'), (1, '34.100')] [2023-10-08 04:26:05,480][00611] Updated weights for policy 0, policy_version 12582 (0.0007) [2023-10-08 04:26:05,559][00612] Updated weights for policy 1, policy_version 12650 (0.0008) [2023-10-08 04:26:05,858][00611] Updated weights for policy 0, policy_version 12592 (0.0009) [2023-10-08 04:26:05,924][00612] Updated weights for policy 1, policy_version 12660 (0.0009) [2023-10-08 04:26:06,223][00611] Updated weights for policy 0, policy_version 12602 (0.0007) [2023-10-08 04:26:06,293][00612] Updated weights for policy 1, policy_version 12670 (0.0009) [2023-10-08 04:26:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 25886720. Throughput: 0: 1821.2, 1: 1829.8. Samples: 6475372. Policy #0 lag: (min: 17.0, avg: 22.6, max: 49.0) [2023-10-08 04:26:08,754][130385] Avg episode reward: [(0, '31.310'), (1, '34.750')] [2023-10-08 04:26:09,783][00611] Updated weights for policy 0, policy_version 12612 (0.0009) [2023-10-08 04:26:10,128][00612] Updated weights for policy 1, policy_version 12680 (0.0007) [2023-10-08 04:26:10,167][00611] Updated weights for policy 0, policy_version 12622 (0.0008) [2023-10-08 04:26:10,492][00612] Updated weights for policy 1, policy_version 12690 (0.0007) [2023-10-08 04:26:10,533][00611] Updated weights for policy 0, policy_version 12632 (0.0007) [2023-10-08 04:26:10,855][00612] Updated weights for policy 1, policy_version 12700 (0.0011) [2023-10-08 04:26:13,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 25952256. Throughput: 0: 1829.7, 1: 1852.6. Samples: 6498184. Policy #0 lag: (min: 19.0, avg: 19.9, max: 38.0) [2023-10-08 04:26:13,755][130385] Avg episode reward: [(0, '30.980'), (1, '33.890')] [2023-10-08 04:26:14,068][00611] Updated weights for policy 0, policy_version 12642 (0.0008) [2023-10-08 04:26:14,445][00611] Updated weights for policy 0, policy_version 12652 (0.0007) [2023-10-08 04:26:14,523][00612] Updated weights for policy 1, policy_version 12710 (0.0008) [2023-10-08 04:26:14,815][00611] Updated weights for policy 0, policy_version 12662 (0.0008) [2023-10-08 04:26:14,904][00612] Updated weights for policy 1, policy_version 12720 (0.0007) [2023-10-08 04:26:15,184][00611] Updated weights for policy 0, policy_version 12672 (0.0007) [2023-10-08 04:26:15,270][00612] Updated weights for policy 1, policy_version 12730 (0.0008) [2023-10-08 04:26:18,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 26017792. Throughput: 0: 1825.2, 1: 1846.5. Samples: 6520994. Policy #0 lag: (min: 19.0, avg: 19.9, max: 38.0) [2023-10-08 04:26:18,755][130385] Avg episode reward: [(0, '32.700'), (1, '33.440')] [2023-10-08 04:26:18,765][00612] Updated weights for policy 1, policy_version 12740 (0.0009) [2023-10-08 04:26:18,961][00611] Updated weights for policy 0, policy_version 12682 (0.0007) [2023-10-08 04:26:19,125][00612] Updated weights for policy 1, policy_version 12750 (0.0008) [2023-10-08 04:26:19,336][00611] Updated weights for policy 0, policy_version 12692 (0.0007) [2023-10-08 04:26:19,490][00612] Updated weights for policy 1, policy_version 12760 (0.0008) [2023-10-08 04:26:19,711][00611] Updated weights for policy 0, policy_version 12702 (0.0007) [2023-10-08 04:26:19,780][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000012704_13008896.pth... [2023-10-08 04:26:19,787][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000012768_13074432.pth... [2023-10-08 04:26:19,809][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000010976_11239424.pth [2023-10-08 04:26:19,827][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000011008_11272192.pth [2023-10-08 04:26:23,234][00612] Updated weights for policy 1, policy_version 12770 (0.0008) [2023-10-08 04:26:23,428][00611] Updated weights for policy 0, policy_version 12712 (0.0007) [2023-10-08 04:26:23,608][00612] Updated weights for policy 1, policy_version 12780 (0.0008) [2023-10-08 04:26:23,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 26083328. Throughput: 0: 1824.1, 1: 1843.3. Samples: 6530716. Policy #0 lag: (min: 19.0, avg: 19.9, max: 38.0) [2023-10-08 04:26:23,754][130385] Avg episode reward: [(0, '32.720'), (1, '35.450')] [2023-10-08 04:26:23,798][00611] Updated weights for policy 0, policy_version 12722 (0.0008) [2023-10-08 04:26:23,972][00612] Updated weights for policy 1, policy_version 12790 (0.0008) [2023-10-08 04:26:24,172][00611] Updated weights for policy 0, policy_version 12732 (0.0008) [2023-10-08 04:26:24,338][00612] Updated weights for policy 1, policy_version 12800 (0.0008) [2023-10-08 04:26:27,795][00611] Updated weights for policy 0, policy_version 12742 (0.0008) [2023-10-08 04:26:28,002][00612] Updated weights for policy 1, policy_version 12810 (0.0007) [2023-10-08 04:26:28,167][00611] Updated weights for policy 0, policy_version 12752 (0.0007) [2023-10-08 04:26:28,380][00612] Updated weights for policy 1, policy_version 12820 (0.0008) [2023-10-08 04:26:28,529][00611] Updated weights for policy 0, policy_version 12762 (0.0007) [2023-10-08 04:26:28,750][00612] Updated weights for policy 1, policy_version 12830 (0.0007) [2023-10-08 04:26:28,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 26181632. Throughput: 0: 1828.6, 1: 1845.1. Samples: 6553918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:26:28,755][130385] Avg episode reward: [(0, '34.780'), (1, '33.470')] [2023-10-08 04:26:32,347][00611] Updated weights for policy 0, policy_version 12772 (0.0009) [2023-10-08 04:26:32,351][00612] Updated weights for policy 1, policy_version 12840 (0.0008) [2023-10-08 04:26:32,709][00611] Updated weights for policy 0, policy_version 12782 (0.0007) [2023-10-08 04:26:32,724][00612] Updated weights for policy 1, policy_version 12850 (0.0009) [2023-10-08 04:26:33,075][00611] Updated weights for policy 0, policy_version 12792 (0.0007) [2023-10-08 04:26:33,093][00612] Updated weights for policy 1, policy_version 12860 (0.0009) [2023-10-08 04:26:33,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 26279936. Throughput: 0: 1824.3, 1: 1833.6. Samples: 6574248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:26:33,754][130385] Avg episode reward: [(0, '35.390'), (1, '32.560')] [2023-10-08 04:26:36,750][00611] Updated weights for policy 0, policy_version 12802 (0.0007) [2023-10-08 04:26:36,780][00612] Updated weights for policy 1, policy_version 12870 (0.0008) [2023-10-08 04:26:37,128][00611] Updated weights for policy 0, policy_version 12812 (0.0008) [2023-10-08 04:26:37,152][00612] Updated weights for policy 1, policy_version 12880 (0.0008) [2023-10-08 04:26:37,502][00611] Updated weights for policy 0, policy_version 12822 (0.0007) [2023-10-08 04:26:37,522][00612] Updated weights for policy 1, policy_version 12890 (0.0007) [2023-10-08 04:26:37,880][00611] Updated weights for policy 0, policy_version 12832 (0.0007) [2023-10-08 04:26:38,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 26345472. Throughput: 0: 1825.0, 1: 1849.2. Samples: 6586824. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-08 04:26:38,754][130385] Avg episode reward: [(0, '33.640'), (1, '29.990')] [2023-10-08 04:26:40,985][00612] Updated weights for policy 1, policy_version 12900 (0.0007) [2023-10-08 04:26:41,354][00612] Updated weights for policy 1, policy_version 12910 (0.0008) [2023-10-08 04:26:41,525][00611] Updated weights for policy 0, policy_version 12842 (0.0008) [2023-10-08 04:26:41,708][00612] Updated weights for policy 1, policy_version 12920 (0.0008) [2023-10-08 04:26:41,898][00611] Updated weights for policy 0, policy_version 12852 (0.0008) [2023-10-08 04:26:42,264][00611] Updated weights for policy 0, policy_version 12862 (0.0008) [2023-10-08 04:26:43,754][130385] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 26411008. Throughput: 0: 1826.0, 1: 1835.4. Samples: 6607072. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-08 04:26:43,755][130385] Avg episode reward: [(0, '36.490'), (1, '30.870')] [2023-10-08 04:26:45,298][00612] Updated weights for policy 1, policy_version 12930 (0.0009) [2023-10-08 04:26:45,684][00612] Updated weights for policy 1, policy_version 12940 (0.0010) [2023-10-08 04:26:45,939][00611] Updated weights for policy 0, policy_version 12872 (0.0008) [2023-10-08 04:26:46,063][00612] Updated weights for policy 1, policy_version 12950 (0.0010) [2023-10-08 04:26:46,307][00611] Updated weights for policy 0, policy_version 12882 (0.0007) [2023-10-08 04:26:46,430][00612] Updated weights for policy 1, policy_version 12960 (0.0008) [2023-10-08 04:26:46,686][00611] Updated weights for policy 0, policy_version 12892 (0.0008) [2023-10-08 04:26:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 26476544. Throughput: 0: 1828.2, 1: 1847.2. Samples: 6629842. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-08 04:26:48,754][130385] Avg episode reward: [(0, '37.820'), (1, '32.670')] [2023-10-08 04:26:48,764][00365] Saving new best policy, reward=37.820! [2023-10-08 04:26:50,003][00612] Updated weights for policy 1, policy_version 12970 (0.0007) [2023-10-08 04:26:50,291][00611] Updated weights for policy 0, policy_version 12902 (0.0008) [2023-10-08 04:26:50,369][00612] Updated weights for policy 1, policy_version 12980 (0.0007) [2023-10-08 04:26:50,658][00611] Updated weights for policy 0, policy_version 12912 (0.0008) [2023-10-08 04:26:50,739][00612] Updated weights for policy 1, policy_version 12990 (0.0008) [2023-10-08 04:26:51,030][00611] Updated weights for policy 0, policy_version 12922 (0.0007) [2023-10-08 04:26:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 26542080. Throughput: 0: 1828.3, 1: 1838.4. Samples: 6640376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:26:53,754][130385] Avg episode reward: [(0, '36.760'), (1, '34.420')] [2023-10-08 04:26:54,593][00611] Updated weights for policy 0, policy_version 12932 (0.0007) [2023-10-08 04:26:54,616][00612] Updated weights for policy 1, policy_version 13000 (0.0007) [2023-10-08 04:26:54,957][00611] Updated weights for policy 0, policy_version 12942 (0.0007) [2023-10-08 04:26:54,975][00612] Updated weights for policy 1, policy_version 13010 (0.0008) [2023-10-08 04:26:55,329][00611] Updated weights for policy 0, policy_version 12952 (0.0007) [2023-10-08 04:26:55,338][00612] Updated weights for policy 1, policy_version 13020 (0.0007) [2023-10-08 04:26:58,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 26607616. Throughput: 0: 1825.1, 1: 1842.9. Samples: 6663244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:26:58,755][130385] Avg episode reward: [(0, '33.590'), (1, '33.590')] [2023-10-08 04:26:58,925][00611] Updated weights for policy 0, policy_version 12962 (0.0008) [2023-10-08 04:26:58,928][00612] Updated weights for policy 1, policy_version 13030 (0.0010) [2023-10-08 04:26:59,295][00611] Updated weights for policy 0, policy_version 12972 (0.0010) [2023-10-08 04:26:59,295][00612] Updated weights for policy 1, policy_version 13040 (0.0007) [2023-10-08 04:26:59,667][00612] Updated weights for policy 1, policy_version 13050 (0.0009) [2023-10-08 04:26:59,667][00611] Updated weights for policy 0, policy_version 12982 (0.0009) [2023-10-08 04:27:00,032][00611] Updated weights for policy 0, policy_version 12992 (0.0008) [2023-10-08 04:27:03,347][00612] Updated weights for policy 1, policy_version 13060 (0.0008) [2023-10-08 04:27:03,699][00611] Updated weights for policy 0, policy_version 13002 (0.0008) [2023-10-08 04:27:03,711][00612] Updated weights for policy 1, policy_version 13070 (0.0010) [2023-10-08 04:27:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 26673152. Throughput: 0: 1824.5, 1: 1846.9. Samples: 6686206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:27:03,754][130385] Avg episode reward: [(0, '33.320'), (1, '34.470')] [2023-10-08 04:27:04,064][00611] Updated weights for policy 0, policy_version 13012 (0.0007) [2023-10-08 04:27:04,085][00612] Updated weights for policy 1, policy_version 13080 (0.0008) [2023-10-08 04:27:04,432][00611] Updated weights for policy 0, policy_version 13022 (0.0007) [2023-10-08 04:27:07,758][00612] Updated weights for policy 1, policy_version 13090 (0.0008) [2023-10-08 04:27:08,122][00612] Updated weights for policy 1, policy_version 13100 (0.0009) [2023-10-08 04:27:08,247][00611] Updated weights for policy 0, policy_version 13032 (0.0009) [2023-10-08 04:27:08,482][00612] Updated weights for policy 1, policy_version 13110 (0.0007) [2023-10-08 04:27:08,619][00611] Updated weights for policy 0, policy_version 13042 (0.0010) [2023-10-08 04:27:08,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 26738688. Throughput: 0: 1822.7, 1: 1852.4. Samples: 6696096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-08 04:27:08,754][130385] Avg episode reward: [(0, '34.950'), (1, '33.020')] [2023-10-08 04:27:08,853][00612] Updated weights for policy 1, policy_version 13120 (0.0007) [2023-10-08 04:27:08,982][00611] Updated weights for policy 0, policy_version 13052 (0.0008) [2023-10-08 04:27:12,411][00612] Updated weights for policy 1, policy_version 13130 (0.0008) [2023-10-08 04:27:12,718][00611] Updated weights for policy 0, policy_version 13062 (0.0010) [2023-10-08 04:27:12,778][00612] Updated weights for policy 1, policy_version 13140 (0.0007) [2023-10-08 04:27:13,094][00611] Updated weights for policy 0, policy_version 13072 (0.0008) [2023-10-08 04:27:13,137][00612] Updated weights for policy 1, policy_version 13150 (0.0010) [2023-10-08 04:27:13,468][00611] Updated weights for policy 0, policy_version 13082 (0.0008) [2023-10-08 04:27:13,754][130385] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 26869760. Throughput: 0: 1816.3, 1: 1848.6. Samples: 6718838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-08 04:27:13,754][130385] Avg episode reward: [(0, '34.810'), (1, '36.070')] [2023-10-08 04:27:16,808][00612] Updated weights for policy 1, policy_version 13160 (0.0009) [2023-10-08 04:27:17,171][00612] Updated weights for policy 1, policy_version 13170 (0.0008) [2023-10-08 04:27:17,331][00611] Updated weights for policy 0, policy_version 13092 (0.0009) [2023-10-08 04:27:17,540][00612] Updated weights for policy 1, policy_version 13180 (0.0010) [2023-10-08 04:27:17,709][00611] Updated weights for policy 0, policy_version 13102 (0.0007) [2023-10-08 04:27:18,076][00611] Updated weights for policy 0, policy_version 13112 (0.0008) [2023-10-08 04:27:18,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 26935296. Throughput: 0: 1816.5, 1: 1848.1. Samples: 6739158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:27:18,754][130385] Avg episode reward: [(0, '36.890'), (1, '35.030')] [2023-10-08 04:27:21,254][00612] Updated weights for policy 1, policy_version 13190 (0.0008) [2023-10-08 04:27:21,626][00612] Updated weights for policy 1, policy_version 13200 (0.0007) [2023-10-08 04:27:21,797][00611] Updated weights for policy 0, policy_version 13122 (0.0008) [2023-10-08 04:27:22,005][00612] Updated weights for policy 1, policy_version 13210 (0.0009) [2023-10-08 04:27:22,175][00611] Updated weights for policy 0, policy_version 13132 (0.0007) [2023-10-08 04:27:22,547][00611] Updated weights for policy 0, policy_version 13142 (0.0011) [2023-10-08 04:27:22,909][00611] Updated weights for policy 0, policy_version 13152 (0.0010) [2023-10-08 04:27:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 27000832. Throughput: 0: 1816.6, 1: 1844.9. Samples: 6751594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:27:23,755][130385] Avg episode reward: [(0, '36.020'), (1, '32.750')] [2023-10-08 04:27:25,758][00612] Updated weights for policy 1, policy_version 13220 (0.0009) [2023-10-08 04:27:26,134][00612] Updated weights for policy 1, policy_version 13230 (0.0011) [2023-10-08 04:27:26,504][00612] Updated weights for policy 1, policy_version 13240 (0.0008) [2023-10-08 04:27:26,668][00611] Updated weights for policy 0, policy_version 13162 (0.0009) [2023-10-08 04:27:27,044][00611] Updated weights for policy 0, policy_version 13172 (0.0010) [2023-10-08 04:27:27,404][00611] Updated weights for policy 0, policy_version 13182 (0.0008) [2023-10-08 04:27:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 27066368. Throughput: 0: 1821.8, 1: 1846.4. Samples: 6772142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:27:28,754][130385] Avg episode reward: [(0, '35.730'), (1, '31.130')] [2023-10-08 04:27:29,966][00612] Updated weights for policy 1, policy_version 13250 (0.0007) [2023-10-08 04:27:30,340][00612] Updated weights for policy 1, policy_version 13260 (0.0007) [2023-10-08 04:27:30,713][00612] Updated weights for policy 1, policy_version 13270 (0.0008) [2023-10-08 04:27:31,078][00612] Updated weights for policy 1, policy_version 13280 (0.0007) [2023-10-08 04:27:31,103][00611] Updated weights for policy 0, policy_version 13192 (0.0008) [2023-10-08 04:27:31,481][00611] Updated weights for policy 0, policy_version 13202 (0.0011) [2023-10-08 04:27:31,860][00611] Updated weights for policy 0, policy_version 13212 (0.0010) [2023-10-08 04:27:33,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 27131904. Throughput: 0: 1816.1, 1: 1851.4. Samples: 6794878. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 04:27:33,754][130385] Avg episode reward: [(0, '36.470'), (1, '30.470')] [2023-10-08 04:27:34,826][00612] Updated weights for policy 1, policy_version 13290 (0.0010) [2023-10-08 04:27:35,195][00612] Updated weights for policy 1, policy_version 13300 (0.0009) [2023-10-08 04:27:35,424][00611] Updated weights for policy 0, policy_version 13222 (0.0008) [2023-10-08 04:27:35,566][00612] Updated weights for policy 1, policy_version 13310 (0.0007) [2023-10-08 04:27:35,800][00611] Updated weights for policy 0, policy_version 13232 (0.0008) [2023-10-08 04:27:36,176][00611] Updated weights for policy 0, policy_version 13242 (0.0011) [2023-10-08 04:27:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 27197440. Throughput: 0: 1818.3, 1: 1842.9. Samples: 6805128. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 04:27:38,754][130385] Avg episode reward: [(0, '37.090'), (1, '31.590')] [2023-10-08 04:27:39,071][00612] Updated weights for policy 1, policy_version 13320 (0.0009) [2023-10-08 04:27:39,446][00612] Updated weights for policy 1, policy_version 13330 (0.0008) [2023-10-08 04:27:39,814][00612] Updated weights for policy 1, policy_version 13340 (0.0008) [2023-10-08 04:27:39,934][00611] Updated weights for policy 0, policy_version 13252 (0.0009) [2023-10-08 04:27:40,305][00611] Updated weights for policy 0, policy_version 13262 (0.0010) [2023-10-08 04:27:40,675][00611] Updated weights for policy 0, policy_version 13272 (0.0009) [2023-10-08 04:27:43,558][00612] Updated weights for policy 1, policy_version 13350 (0.0009) [2023-10-08 04:27:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 27262976. Throughput: 0: 1809.2, 1: 1845.3. Samples: 6827694. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 04:27:43,754][130385] Avg episode reward: [(0, '37.550'), (1, '32.070')] [2023-10-08 04:27:43,918][00612] Updated weights for policy 1, policy_version 13360 (0.0008) [2023-10-08 04:27:44,281][00611] Updated weights for policy 0, policy_version 13282 (0.0009) [2023-10-08 04:27:44,286][00612] Updated weights for policy 1, policy_version 13370 (0.0008) [2023-10-08 04:27:44,652][00611] Updated weights for policy 0, policy_version 13292 (0.0008) [2023-10-08 04:27:45,020][00611] Updated weights for policy 0, policy_version 13302 (0.0009) [2023-10-08 04:27:45,405][00611] Updated weights for policy 0, policy_version 13312 (0.0009) [2023-10-08 04:27:47,959][00612] Updated weights for policy 1, policy_version 13380 (0.0009) [2023-10-08 04:27:48,331][00612] Updated weights for policy 1, policy_version 13390 (0.0008) [2023-10-08 04:27:48,699][00612] Updated weights for policy 1, policy_version 13400 (0.0007) [2023-10-08 04:27:48,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 27328512. Throughput: 0: 1819.7, 1: 1827.0. Samples: 6850310. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-08 04:27:48,755][130385] Avg episode reward: [(0, '37.440'), (1, '32.250')] [2023-10-08 04:27:48,861][00611] Updated weights for policy 0, policy_version 13322 (0.0008) [2023-10-08 04:27:49,233][00611] Updated weights for policy 0, policy_version 13332 (0.0010) [2023-10-08 04:27:49,603][00611] Updated weights for policy 0, policy_version 13342 (0.0007) [2023-10-08 04:27:52,460][00612] Updated weights for policy 1, policy_version 13410 (0.0007) [2023-10-08 04:27:52,831][00612] Updated weights for policy 1, policy_version 13420 (0.0008) [2023-10-08 04:27:53,205][00612] Updated weights for policy 1, policy_version 13430 (0.0008) [2023-10-08 04:27:53,269][00611] Updated weights for policy 0, policy_version 13352 (0.0008) [2023-10-08 04:27:53,572][00612] Updated weights for policy 1, policy_version 13440 (0.0008) [2023-10-08 04:27:53,646][00611] Updated weights for policy 0, policy_version 13362 (0.0008) [2023-10-08 04:27:53,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 27426816. Throughput: 0: 1820.5, 1: 1837.9. Samples: 6860724. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-08 04:27:53,754][130385] Avg episode reward: [(0, '41.040'), (1, '33.900')] [2023-10-08 04:27:54,019][00611] Updated weights for policy 0, policy_version 13372 (0.0009) [2023-10-08 04:27:54,168][00365] Saving new best policy, reward=41.040! [2023-10-08 04:27:57,309][00612] Updated weights for policy 1, policy_version 13450 (0.0009) [2023-10-08 04:27:57,677][00612] Updated weights for policy 1, policy_version 13460 (0.0010) [2023-10-08 04:27:57,818][00611] Updated weights for policy 0, policy_version 13382 (0.0007) [2023-10-08 04:27:58,038][00612] Updated weights for policy 1, policy_version 13470 (0.0008) [2023-10-08 04:27:58,178][00611] Updated weights for policy 0, policy_version 13392 (0.0009) [2023-10-08 04:27:58,546][00611] Updated weights for policy 0, policy_version 13402 (0.0007) [2023-10-08 04:27:58,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 27492352. Throughput: 0: 1821.8, 1: 1821.7. Samples: 6882794. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-08 04:27:58,754][130385] Avg episode reward: [(0, '40.420'), (1, '33.340')] [2023-10-08 04:28:01,598][00612] Updated weights for policy 1, policy_version 13480 (0.0008) [2023-10-08 04:28:01,971][00612] Updated weights for policy 1, policy_version 13490 (0.0008) [2023-10-08 04:28:02,109][00611] Updated weights for policy 0, policy_version 13412 (0.0009) [2023-10-08 04:28:02,329][00612] Updated weights for policy 1, policy_version 13500 (0.0008) [2023-10-08 04:28:02,491][00611] Updated weights for policy 0, policy_version 13422 (0.0008) [2023-10-08 04:28:02,858][00611] Updated weights for policy 0, policy_version 13432 (0.0010) [2023-10-08 04:28:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 27590656. Throughput: 0: 1825.5, 1: 1830.9. Samples: 6903696. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-08 04:28:03,754][130385] Avg episode reward: [(0, '40.570'), (1, '34.930')] [2023-10-08 04:28:05,851][00612] Updated weights for policy 1, policy_version 13510 (0.0008) [2023-10-08 04:28:06,218][00612] Updated weights for policy 1, policy_version 13520 (0.0007) [2023-10-08 04:28:06,453][00611] Updated weights for policy 0, policy_version 13442 (0.0008) [2023-10-08 04:28:06,581][00612] Updated weights for policy 1, policy_version 13530 (0.0008) [2023-10-08 04:28:06,826][00611] Updated weights for policy 0, policy_version 13452 (0.0007) [2023-10-08 04:28:07,194][00611] Updated weights for policy 0, policy_version 13462 (0.0007) [2023-10-08 04:28:07,561][00611] Updated weights for policy 0, policy_version 13472 (0.0008) [2023-10-08 04:28:08,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 27656192. Throughput: 0: 1836.4, 1: 1820.9. Samples: 6916174. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-08 04:28:08,755][130385] Avg episode reward: [(0, '39.850'), (1, '37.370')] [2023-10-08 04:28:10,154][00612] Updated weights for policy 1, policy_version 13540 (0.0008) [2023-10-08 04:28:10,534][00612] Updated weights for policy 1, policy_version 13550 (0.0011) [2023-10-08 04:28:10,895][00612] Updated weights for policy 1, policy_version 13560 (0.0009) [2023-10-08 04:28:11,322][00611] Updated weights for policy 0, policy_version 13482 (0.0009) [2023-10-08 04:28:11,703][00611] Updated weights for policy 0, policy_version 13492 (0.0010) [2023-10-08 04:28:12,083][00611] Updated weights for policy 0, policy_version 13502 (0.0009) [2023-10-08 04:28:13,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 27721728. Throughput: 0: 1827.1, 1: 1838.7. Samples: 6937102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:28:13,755][130385] Avg episode reward: [(0, '42.480'), (1, '36.990')] [2023-10-08 04:28:13,756][00365] Saving new best policy, reward=42.480! [2023-10-08 04:28:14,557][00612] Updated weights for policy 1, policy_version 13570 (0.0009) [2023-10-08 04:28:14,923][00612] Updated weights for policy 1, policy_version 13580 (0.0007) [2023-10-08 04:28:15,288][00612] Updated weights for policy 1, policy_version 13590 (0.0007) [2023-10-08 04:28:15,560][00611] Updated weights for policy 0, policy_version 13512 (0.0007) [2023-10-08 04:28:15,654][00612] Updated weights for policy 1, policy_version 13600 (0.0009) [2023-10-08 04:28:15,935][00611] Updated weights for policy 0, policy_version 13522 (0.0009) [2023-10-08 04:28:16,303][00611] Updated weights for policy 0, policy_version 13532 (0.0009) [2023-10-08 04:28:18,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 27787264. Throughput: 0: 1848.5, 1: 1832.0. Samples: 6960502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:28:18,755][130385] Avg episode reward: [(0, '42.960'), (1, '36.270')] [2023-10-08 04:28:18,763][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000013536_13860864.pth... [2023-10-08 04:28:18,794][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000011840_12124160.pth [2023-10-08 04:28:18,797][00365] Saving new best policy, reward=42.960! [2023-10-08 04:28:19,212][00612] Updated weights for policy 1, policy_version 13610 (0.0007) [2023-10-08 04:28:19,576][00612] Updated weights for policy 1, policy_version 13620 (0.0007) [2023-10-08 04:28:19,944][00612] Updated weights for policy 1, policy_version 13630 (0.0008) [2023-10-08 04:28:20,019][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000013632_13959168.pth... [2023-10-08 04:28:20,055][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000011904_12189696.pth [2023-10-08 04:28:20,081][00611] Updated weights for policy 0, policy_version 13542 (0.0007) [2023-10-08 04:28:20,467][00611] Updated weights for policy 0, policy_version 13552 (0.0007) [2023-10-08 04:28:20,849][00611] Updated weights for policy 0, policy_version 13562 (0.0007) [2023-10-08 04:28:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 27852800. Throughput: 0: 1831.4, 1: 1839.3. Samples: 6970310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:28:23,754][130385] Avg episode reward: [(0, '40.330'), (1, '36.590')] [2023-10-08 04:28:23,798][00612] Updated weights for policy 1, policy_version 13640 (0.0008) [2023-10-08 04:28:24,180][00612] Updated weights for policy 1, policy_version 13650 (0.0009) [2023-10-08 04:28:24,441][00611] Updated weights for policy 0, policy_version 13572 (0.0008) [2023-10-08 04:28:24,536][00612] Updated weights for policy 1, policy_version 13660 (0.0008) [2023-10-08 04:28:24,801][00611] Updated weights for policy 0, policy_version 13582 (0.0007) [2023-10-08 04:28:25,169][00611] Updated weights for policy 0, policy_version 13592 (0.0010) [2023-10-08 04:28:28,126][00612] Updated weights for policy 1, policy_version 13670 (0.0010) [2023-10-08 04:28:28,503][00612] Updated weights for policy 1, policy_version 13680 (0.0008) [2023-10-08 04:28:28,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 27918336. Throughput: 0: 1846.8, 1: 1830.8. Samples: 6993188. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-08 04:28:28,755][130385] Avg episode reward: [(0, '41.810'), (1, '35.980')] [2023-10-08 04:28:28,850][00611] Updated weights for policy 0, policy_version 13602 (0.0009) [2023-10-08 04:28:28,864][00612] Updated weights for policy 1, policy_version 13690 (0.0008) [2023-10-08 04:28:29,225][00611] Updated weights for policy 0, policy_version 13612 (0.0009) [2023-10-08 04:28:29,583][00611] Updated weights for policy 0, policy_version 13622 (0.0008) [2023-10-08 04:28:29,953][00611] Updated weights for policy 0, policy_version 13632 (0.0008) [2023-10-08 04:28:32,446][00612] Updated weights for policy 1, policy_version 13700 (0.0009) [2023-10-08 04:28:32,814][00612] Updated weights for policy 1, policy_version 13710 (0.0010) [2023-10-08 04:28:33,181][00612] Updated weights for policy 1, policy_version 13720 (0.0009) [2023-10-08 04:28:33,581][00611] Updated weights for policy 0, policy_version 13642 (0.0007) [2023-10-08 04:28:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 28016640. Throughput: 0: 1839.3, 1: 1823.3. Samples: 7015124. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-08 04:28:33,754][130385] Avg episode reward: [(0, '40.840'), (1, '34.610')] [2023-10-08 04:28:33,953][00611] Updated weights for policy 0, policy_version 13652 (0.0008) [2023-10-08 04:28:34,327][00611] Updated weights for policy 0, policy_version 13662 (0.0008) [2023-10-08 04:28:36,746][00612] Updated weights for policy 1, policy_version 13730 (0.0009) [2023-10-08 04:28:37,117][00612] Updated weights for policy 1, policy_version 13740 (0.0008) [2023-10-08 04:28:37,486][00612] Updated weights for policy 1, policy_version 13750 (0.0009) [2023-10-08 04:28:37,855][00612] Updated weights for policy 1, policy_version 13760 (0.0009) [2023-10-08 04:28:37,890][00611] Updated weights for policy 0, policy_version 13672 (0.0007) [2023-10-08 04:28:38,264][00611] Updated weights for policy 0, policy_version 13682 (0.0008) [2023-10-08 04:28:38,647][00611] Updated weights for policy 0, policy_version 13692 (0.0008) [2023-10-08 04:28:38,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 28082176. Throughput: 0: 1839.9, 1: 1839.9. Samples: 7026316. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-08 04:28:38,754][130385] Avg episode reward: [(0, '40.190'), (1, '34.300')] [2023-10-08 04:28:41,455][00612] Updated weights for policy 1, policy_version 13770 (0.0007) [2023-10-08 04:28:41,812][00612] Updated weights for policy 1, policy_version 13780 (0.0011) [2023-10-08 04:28:42,177][00612] Updated weights for policy 1, policy_version 13790 (0.0010) [2023-10-08 04:28:42,352][00611] Updated weights for policy 0, policy_version 13702 (0.0008) [2023-10-08 04:28:42,724][00611] Updated weights for policy 0, policy_version 13712 (0.0010) [2023-10-08 04:28:43,099][00611] Updated weights for policy 0, policy_version 13722 (0.0011) [2023-10-08 04:28:43,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 28180480. Throughput: 0: 1845.8, 1: 1826.7. Samples: 7048054. Policy #0 lag: (min: 21.0, avg: 24.6, max: 53.0) [2023-10-08 04:28:43,754][130385] Avg episode reward: [(0, '42.310'), (1, '33.510')] [2023-10-08 04:28:45,978][00612] Updated weights for policy 1, policy_version 13800 (0.0007) [2023-10-08 04:28:46,346][00612] Updated weights for policy 1, policy_version 13810 (0.0008) [2023-10-08 04:28:46,623][00611] Updated weights for policy 0, policy_version 13732 (0.0008) [2023-10-08 04:28:46,715][00612] Updated weights for policy 1, policy_version 13820 (0.0007) [2023-10-08 04:28:46,998][00611] Updated weights for policy 0, policy_version 13742 (0.0008) [2023-10-08 04:28:47,366][00611] Updated weights for policy 0, policy_version 13752 (0.0007) [2023-10-08 04:28:48,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 28246016. Throughput: 0: 1842.4, 1: 1839.1. Samples: 7069362. Policy #0 lag: (min: 21.0, avg: 24.6, max: 53.0) [2023-10-08 04:28:48,754][130385] Avg episode reward: [(0, '42.000'), (1, '34.780')] [2023-10-08 04:28:50,355][00612] Updated weights for policy 1, policy_version 13830 (0.0008) [2023-10-08 04:28:50,721][00612] Updated weights for policy 1, policy_version 13840 (0.0007) [2023-10-08 04:28:51,058][00611] Updated weights for policy 0, policy_version 13762 (0.0009) [2023-10-08 04:28:51,090][00612] Updated weights for policy 1, policy_version 13850 (0.0007) [2023-10-08 04:28:51,426][00611] Updated weights for policy 0, policy_version 13772 (0.0008) [2023-10-08 04:28:51,794][00611] Updated weights for policy 0, policy_version 13782 (0.0008) [2023-10-08 04:28:52,169][00611] Updated weights for policy 0, policy_version 13792 (0.0008) [2023-10-08 04:28:53,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 28311552. Throughput: 0: 1835.7, 1: 1825.7. Samples: 7080938. Policy #0 lag: (min: 31.0, avg: 32.5, max: 56.0) [2023-10-08 04:28:53,755][130385] Avg episode reward: [(0, '39.540'), (1, '36.170')] [2023-10-08 04:28:54,779][00612] Updated weights for policy 1, policy_version 13860 (0.0008) [2023-10-08 04:28:55,151][00612] Updated weights for policy 1, policy_version 13870 (0.0009) [2023-10-08 04:28:55,512][00612] Updated weights for policy 1, policy_version 13880 (0.0009) [2023-10-08 04:28:55,820][00611] Updated weights for policy 0, policy_version 13802 (0.0008) [2023-10-08 04:28:56,202][00611] Updated weights for policy 0, policy_version 13812 (0.0008) [2023-10-08 04:28:56,584][00611] Updated weights for policy 0, policy_version 13822 (0.0008) [2023-10-08 04:28:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 28377088. Throughput: 0: 1839.0, 1: 1837.2. Samples: 7102532. Policy #0 lag: (min: 31.0, avg: 32.5, max: 56.0) [2023-10-08 04:28:58,754][130385] Avg episode reward: [(0, '42.760'), (1, '37.990')] [2023-10-08 04:28:58,755][00425] Saving new best policy, reward=37.990! [2023-10-08 04:28:59,080][00612] Updated weights for policy 1, policy_version 13890 (0.0010) [2023-10-08 04:28:59,451][00612] Updated weights for policy 1, policy_version 13900 (0.0011) [2023-10-08 04:28:59,829][00612] Updated weights for policy 1, policy_version 13910 (0.0009) [2023-10-08 04:29:00,083][00611] Updated weights for policy 0, policy_version 13832 (0.0008) [2023-10-08 04:29:00,198][00612] Updated weights for policy 1, policy_version 13920 (0.0008) [2023-10-08 04:29:00,462][00611] Updated weights for policy 0, policy_version 13842 (0.0008) [2023-10-08 04:29:00,823][00611] Updated weights for policy 0, policy_version 13852 (0.0008) [2023-10-08 04:29:03,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 28442624. Throughput: 0: 1839.6, 1: 1836.5. Samples: 7125928. Policy #0 lag: (min: 31.0, avg: 32.5, max: 56.0) [2023-10-08 04:29:03,754][130385] Avg episode reward: [(0, '41.850'), (1, '35.560')] [2023-10-08 04:29:03,881][00612] Updated weights for policy 1, policy_version 13930 (0.0010) [2023-10-08 04:29:04,254][00612] Updated weights for policy 1, policy_version 13940 (0.0008) [2023-10-08 04:29:04,488][00611] Updated weights for policy 0, policy_version 13862 (0.0008) [2023-10-08 04:29:04,623][00612] Updated weights for policy 1, policy_version 13950 (0.0007) [2023-10-08 04:29:04,865][00611] Updated weights for policy 0, policy_version 13872 (0.0009) [2023-10-08 04:29:05,234][00611] Updated weights for policy 0, policy_version 13882 (0.0009) [2023-10-08 04:29:08,392][00612] Updated weights for policy 1, policy_version 13960 (0.0010) [2023-10-08 04:29:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 28508160. Throughput: 0: 1844.1, 1: 1837.3. Samples: 7135972. Policy #0 lag: (min: 18.0, avg: 27.3, max: 50.0) [2023-10-08 04:29:08,754][130385] Avg episode reward: [(0, '37.750'), (1, '37.350')] [2023-10-08 04:29:08,764][00611] Updated weights for policy 0, policy_version 13892 (0.0007) [2023-10-08 04:29:08,772][00612] Updated weights for policy 1, policy_version 13970 (0.0008) [2023-10-08 04:29:09,142][00611] Updated weights for policy 0, policy_version 13902 (0.0007) [2023-10-08 04:29:09,143][00612] Updated weights for policy 1, policy_version 13980 (0.0007) [2023-10-08 04:29:09,514][00611] Updated weights for policy 0, policy_version 13912 (0.0009) [2023-10-08 04:29:12,764][00612] Updated weights for policy 1, policy_version 13990 (0.0009) [2023-10-08 04:29:13,133][00612] Updated weights for policy 1, policy_version 14000 (0.0007) [2023-10-08 04:29:13,173][00611] Updated weights for policy 0, policy_version 13922 (0.0010) [2023-10-08 04:29:13,491][00612] Updated weights for policy 1, policy_version 14010 (0.0007) [2023-10-08 04:29:13,532][00611] Updated weights for policy 0, policy_version 13932 (0.0007) [2023-10-08 04:29:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 28606464. Throughput: 0: 1842.8, 1: 1841.4. Samples: 7158978. Policy #0 lag: (min: 18.0, avg: 27.3, max: 50.0) [2023-10-08 04:29:13,754][130385] Avg episode reward: [(0, '40.810'), (1, '39.530')] [2023-10-08 04:29:13,756][00425] Saving new best policy, reward=39.530! [2023-10-08 04:29:13,909][00611] Updated weights for policy 0, policy_version 13942 (0.0010) [2023-10-08 04:29:14,274][00611] Updated weights for policy 0, policy_version 13952 (0.0011) [2023-10-08 04:29:17,043][00612] Updated weights for policy 1, policy_version 14020 (0.0008) [2023-10-08 04:29:17,410][00612] Updated weights for policy 1, policy_version 14030 (0.0008) [2023-10-08 04:29:17,784][00612] Updated weights for policy 1, policy_version 14040 (0.0007) [2023-10-08 04:29:17,977][00611] Updated weights for policy 0, policy_version 13962 (0.0009) [2023-10-08 04:29:18,337][00611] Updated weights for policy 0, policy_version 13972 (0.0009) [2023-10-08 04:29:18,709][00611] Updated weights for policy 0, policy_version 13982 (0.0009) [2023-10-08 04:29:18,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 28672000. Throughput: 0: 1827.8, 1: 1827.5. Samples: 7179614. Policy #0 lag: (min: 18.0, avg: 27.3, max: 50.0) [2023-10-08 04:29:18,754][130385] Avg episode reward: [(0, '39.710'), (1, '38.330')] [2023-10-08 04:29:21,510][00612] Updated weights for policy 1, policy_version 14050 (0.0009) [2023-10-08 04:29:21,883][00612] Updated weights for policy 1, policy_version 14060 (0.0010) [2023-10-08 04:29:22,252][00612] Updated weights for policy 1, policy_version 14070 (0.0009) [2023-10-08 04:29:22,542][00611] Updated weights for policy 0, policy_version 13992 (0.0009) [2023-10-08 04:29:22,616][00612] Updated weights for policy 1, policy_version 14080 (0.0007) [2023-10-08 04:29:22,913][00611] Updated weights for policy 0, policy_version 14002 (0.0008) [2023-10-08 04:29:23,287][00611] Updated weights for policy 0, policy_version 14012 (0.0008) [2023-10-08 04:29:23,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 28770304. Throughput: 0: 1839.9, 1: 1831.7. Samples: 7191538. Policy #0 lag: (min: 24.0, avg: 49.8, max: 56.0) [2023-10-08 04:29:23,755][130385] Avg episode reward: [(0, '40.010'), (1, '35.290')] [2023-10-08 04:29:26,213][00612] Updated weights for policy 1, policy_version 14090 (0.0011) [2023-10-08 04:29:26,576][00612] Updated weights for policy 1, policy_version 14100 (0.0010) [2023-10-08 04:29:26,953][00612] Updated weights for policy 1, policy_version 14110 (0.0009) [2023-10-08 04:29:27,013][00611] Updated weights for policy 0, policy_version 14022 (0.0008) [2023-10-08 04:29:27,375][00611] Updated weights for policy 0, policy_version 14032 (0.0008) [2023-10-08 04:29:27,757][00611] Updated weights for policy 0, policy_version 14042 (0.0008) [2023-10-08 04:29:28,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 28835840. Throughput: 0: 1827.5, 1: 1827.7. Samples: 7212536. Policy #0 lag: (min: 24.0, avg: 49.8, max: 56.0) [2023-10-08 04:29:28,754][130385] Avg episode reward: [(0, '40.350'), (1, '34.200')] [2023-10-08 04:29:30,640][00612] Updated weights for policy 1, policy_version 14120 (0.0008) [2023-10-08 04:29:31,012][00612] Updated weights for policy 1, policy_version 14130 (0.0007) [2023-10-08 04:29:31,374][00612] Updated weights for policy 1, policy_version 14140 (0.0007) [2023-10-08 04:29:31,380][00611] Updated weights for policy 0, policy_version 14052 (0.0008) [2023-10-08 04:29:31,755][00611] Updated weights for policy 0, policy_version 14062 (0.0009) [2023-10-08 04:29:32,129][00611] Updated weights for policy 0, policy_version 14072 (0.0008) [2023-10-08 04:29:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 28901376. Throughput: 0: 1829.8, 1: 1839.3. Samples: 7234472. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-10-08 04:29:33,755][130385] Avg episode reward: [(0, '41.460'), (1, '33.100')] [2023-10-08 04:29:35,016][00612] Updated weights for policy 1, policy_version 14150 (0.0007) [2023-10-08 04:29:35,380][00612] Updated weights for policy 1, policy_version 14160 (0.0008) [2023-10-08 04:29:35,745][00612] Updated weights for policy 1, policy_version 14170 (0.0008) [2023-10-08 04:29:35,748][00611] Updated weights for policy 0, policy_version 14082 (0.0007) [2023-10-08 04:29:36,120][00611] Updated weights for policy 0, policy_version 14092 (0.0009) [2023-10-08 04:29:36,487][00611] Updated weights for policy 0, policy_version 14102 (0.0007) [2023-10-08 04:29:36,859][00611] Updated weights for policy 0, policy_version 14112 (0.0008) [2023-10-08 04:29:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 28966912. Throughput: 0: 1821.7, 1: 1830.6. Samples: 7245292. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-10-08 04:29:38,755][130385] Avg episode reward: [(0, '38.590'), (1, '36.050')] [2023-10-08 04:29:39,467][00612] Updated weights for policy 1, policy_version 14180 (0.0009) [2023-10-08 04:29:39,833][00612] Updated weights for policy 1, policy_version 14190 (0.0010) [2023-10-08 04:29:40,202][00612] Updated weights for policy 1, policy_version 14200 (0.0009) [2023-10-08 04:29:40,601][00611] Updated weights for policy 0, policy_version 14122 (0.0008) [2023-10-08 04:29:40,975][00611] Updated weights for policy 0, policy_version 14132 (0.0007) [2023-10-08 04:29:41,352][00611] Updated weights for policy 0, policy_version 14142 (0.0009) [2023-10-08 04:29:43,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 29032448. Throughput: 0: 1831.0, 1: 1832.6. Samples: 7267396. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-10-08 04:29:43,754][130385] Avg episode reward: [(0, '40.640'), (1, '37.070')] [2023-10-08 04:29:43,965][00612] Updated weights for policy 1, policy_version 14210 (0.0008) [2023-10-08 04:29:44,339][00612] Updated weights for policy 1, policy_version 14220 (0.0009) [2023-10-08 04:29:44,704][00612] Updated weights for policy 1, policy_version 14230 (0.0008) [2023-10-08 04:29:44,986][00611] Updated weights for policy 0, policy_version 14152 (0.0007) [2023-10-08 04:29:45,069][00612] Updated weights for policy 1, policy_version 14240 (0.0008) [2023-10-08 04:29:45,360][00611] Updated weights for policy 0, policy_version 14162 (0.0009) [2023-10-08 04:29:45,727][00611] Updated weights for policy 0, policy_version 14172 (0.0010) [2023-10-08 04:29:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 29097984. Throughput: 0: 1819.4, 1: 1825.5. Samples: 7289952. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-08 04:29:48,755][130385] Avg episode reward: [(0, '42.630'), (1, '37.430')] [2023-10-08 04:29:48,844][00612] Updated weights for policy 1, policy_version 14250 (0.0009) [2023-10-08 04:29:49,214][00612] Updated weights for policy 1, policy_version 14260 (0.0007) [2023-10-08 04:29:49,583][00612] Updated weights for policy 1, policy_version 14270 (0.0007) [2023-10-08 04:29:49,591][00611] Updated weights for policy 0, policy_version 14182 (0.0007) [2023-10-08 04:29:49,979][00611] Updated weights for policy 0, policy_version 14192 (0.0007) [2023-10-08 04:29:50,359][00611] Updated weights for policy 0, policy_version 14202 (0.0008) [2023-10-08 04:29:53,425][00612] Updated weights for policy 1, policy_version 14280 (0.0010) [2023-10-08 04:29:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 29163520. Throughput: 0: 1812.3, 1: 1823.3. Samples: 7299574. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-08 04:29:53,754][130385] Avg episode reward: [(0, '41.230'), (1, '36.450')] [2023-10-08 04:29:53,798][00612] Updated weights for policy 1, policy_version 14290 (0.0009) [2023-10-08 04:29:54,057][00611] Updated weights for policy 0, policy_version 14212 (0.0009) [2023-10-08 04:29:54,174][00612] Updated weights for policy 1, policy_version 14300 (0.0008) [2023-10-08 04:29:54,434][00611] Updated weights for policy 0, policy_version 14222 (0.0008) [2023-10-08 04:29:54,808][00611] Updated weights for policy 0, policy_version 14232 (0.0009) [2023-10-08 04:29:57,885][00612] Updated weights for policy 1, policy_version 14310 (0.0008) [2023-10-08 04:29:58,256][00612] Updated weights for policy 1, policy_version 14320 (0.0009) [2023-10-08 04:29:58,487][00611] Updated weights for policy 0, policy_version 14242 (0.0007) [2023-10-08 04:29:58,629][00612] Updated weights for policy 1, policy_version 14330 (0.0009) [2023-10-08 04:29:58,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 29229056. Throughput: 0: 1806.6, 1: 1815.8. Samples: 7321988. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-08 04:29:58,754][130385] Avg episode reward: [(0, '41.890'), (1, '36.920')] [2023-10-08 04:29:58,860][00611] Updated weights for policy 0, policy_version 14252 (0.0010) [2023-10-08 04:29:59,232][00611] Updated weights for policy 0, policy_version 14262 (0.0010) [2023-10-08 04:29:59,597][00611] Updated weights for policy 0, policy_version 14272 (0.0007) [2023-10-08 04:30:02,388][00612] Updated weights for policy 1, policy_version 14340 (0.0008) [2023-10-08 04:30:02,763][00612] Updated weights for policy 1, policy_version 14350 (0.0010) [2023-10-08 04:30:03,125][00612] Updated weights for policy 1, policy_version 14360 (0.0008) [2023-10-08 04:30:03,257][00611] Updated weights for policy 0, policy_version 14282 (0.0007) [2023-10-08 04:30:03,639][00611] Updated weights for policy 0, policy_version 14292 (0.0009) [2023-10-08 04:30:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 29327360. Throughput: 0: 1813.7, 1: 1820.7. Samples: 7343160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:30:03,754][130385] Avg episode reward: [(0, '40.760'), (1, '39.230')] [2023-10-08 04:30:03,999][00611] Updated weights for policy 0, policy_version 14302 (0.0007) [2023-10-08 04:30:06,831][00612] Updated weights for policy 1, policy_version 14370 (0.0008) [2023-10-08 04:30:07,198][00612] Updated weights for policy 1, policy_version 14380 (0.0007) [2023-10-08 04:30:07,575][00612] Updated weights for policy 1, policy_version 14390 (0.0007) [2023-10-08 04:30:07,717][00611] Updated weights for policy 0, policy_version 14312 (0.0007) [2023-10-08 04:30:07,952][00612] Updated weights for policy 1, policy_version 14400 (0.0008) [2023-10-08 04:30:08,093][00611] Updated weights for policy 0, policy_version 14322 (0.0007) [2023-10-08 04:30:08,462][00611] Updated weights for policy 0, policy_version 14332 (0.0008) [2023-10-08 04:30:08,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 29425664. Throughput: 0: 1808.1, 1: 1814.1. Samples: 7354536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:30:08,755][130385] Avg episode reward: [(0, '40.760'), (1, '42.610')] [2023-10-08 04:30:08,756][00425] Saving new best policy, reward=42.610! [2023-10-08 04:30:11,512][00612] Updated weights for policy 1, policy_version 14410 (0.0009) [2023-10-08 04:30:11,880][00612] Updated weights for policy 1, policy_version 14420 (0.0010) [2023-10-08 04:30:12,198][00611] Updated weights for policy 0, policy_version 14342 (0.0007) [2023-10-08 04:30:12,245][00612] Updated weights for policy 1, policy_version 14430 (0.0008) [2023-10-08 04:30:12,582][00611] Updated weights for policy 0, policy_version 14352 (0.0009) [2023-10-08 04:30:12,943][00611] Updated weights for policy 0, policy_version 14362 (0.0010) [2023-10-08 04:30:13,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 29491200. Throughput: 0: 1813.8, 1: 1820.5. Samples: 7376080. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-08 04:30:13,754][130385] Avg episode reward: [(0, '41.090'), (1, '40.750')] [2023-10-08 04:30:15,826][00612] Updated weights for policy 1, policy_version 14440 (0.0009) [2023-10-08 04:30:16,202][00612] Updated weights for policy 1, policy_version 14450 (0.0008) [2023-10-08 04:30:16,457][00611] Updated weights for policy 0, policy_version 14372 (0.0008) [2023-10-08 04:30:16,563][00612] Updated weights for policy 1, policy_version 14460 (0.0007) [2023-10-08 04:30:16,829][00611] Updated weights for policy 0, policy_version 14382 (0.0009) [2023-10-08 04:30:17,194][00611] Updated weights for policy 0, policy_version 14392 (0.0007) [2023-10-08 04:30:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 29556736. Throughput: 0: 1815.8, 1: 1819.9. Samples: 7398076. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-08 04:30:18,755][130385] Avg episode reward: [(0, '41.560'), (1, '37.120')] [2023-10-08 04:30:18,768][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000014464_14811136.pth... [2023-10-08 04:30:18,768][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000014400_14745600.pth... [2023-10-08 04:30:18,798][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000012768_13074432.pth [2023-10-08 04:30:18,809][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000012704_13008896.pth [2023-10-08 04:30:20,103][00612] Updated weights for policy 1, policy_version 14470 (0.0008) [2023-10-08 04:30:20,475][00612] Updated weights for policy 1, policy_version 14480 (0.0008) [2023-10-08 04:30:20,842][00612] Updated weights for policy 1, policy_version 14490 (0.0009) [2023-10-08 04:30:20,969][00611] Updated weights for policy 0, policy_version 14402 (0.0007) [2023-10-08 04:30:21,348][00611] Updated weights for policy 0, policy_version 14412 (0.0008) [2023-10-08 04:30:21,707][00611] Updated weights for policy 0, policy_version 14422 (0.0010) [2023-10-08 04:30:22,078][00611] Updated weights for policy 0, policy_version 14432 (0.0011) [2023-10-08 04:30:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 29622272. Throughput: 0: 1819.2, 1: 1823.9. Samples: 7409232. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-08 04:30:23,754][130385] Avg episode reward: [(0, '43.390'), (1, '38.550')] [2023-10-08 04:30:23,755][00365] Saving new best policy, reward=43.390! [2023-10-08 04:30:24,442][00612] Updated weights for policy 1, policy_version 14500 (0.0007) [2023-10-08 04:30:24,808][00612] Updated weights for policy 1, policy_version 14510 (0.0007) [2023-10-08 04:30:25,171][00612] Updated weights for policy 1, policy_version 14520 (0.0007) [2023-10-08 04:30:25,857][00611] Updated weights for policy 0, policy_version 14442 (0.0007) [2023-10-08 04:30:26,236][00611] Updated weights for policy 0, policy_version 14452 (0.0007) [2023-10-08 04:30:26,606][00611] Updated weights for policy 0, policy_version 14462 (0.0007) [2023-10-08 04:30:28,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 29687808. Throughput: 0: 1806.1, 1: 1831.7. Samples: 7431100. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 04:30:28,754][130385] Avg episode reward: [(0, '42.600'), (1, '39.680')] [2023-10-08 04:30:28,815][00612] Updated weights for policy 1, policy_version 14530 (0.0008) [2023-10-08 04:30:29,181][00612] Updated weights for policy 1, policy_version 14540 (0.0007) [2023-10-08 04:30:29,558][00612] Updated weights for policy 1, policy_version 14550 (0.0008) [2023-10-08 04:30:29,930][00612] Updated weights for policy 1, policy_version 14560 (0.0007) [2023-10-08 04:30:30,099][00611] Updated weights for policy 0, policy_version 14472 (0.0008) [2023-10-08 04:30:30,459][00611] Updated weights for policy 0, policy_version 14482 (0.0007) [2023-10-08 04:30:30,830][00611] Updated weights for policy 0, policy_version 14492 (0.0008) [2023-10-08 04:30:33,541][00612] Updated weights for policy 1, policy_version 14570 (0.0009) [2023-10-08 04:30:33,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 29753344. Throughput: 0: 1813.0, 1: 1836.6. Samples: 7454184. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 04:30:33,755][130385] Avg episode reward: [(0, '42.560'), (1, '38.420')] [2023-10-08 04:30:33,912][00612] Updated weights for policy 1, policy_version 14580 (0.0010) [2023-10-08 04:30:34,287][00612] Updated weights for policy 1, policy_version 14590 (0.0008) [2023-10-08 04:30:34,510][00611] Updated weights for policy 0, policy_version 14502 (0.0008) [2023-10-08 04:30:34,900][00611] Updated weights for policy 0, policy_version 14512 (0.0008) [2023-10-08 04:30:35,267][00611] Updated weights for policy 0, policy_version 14522 (0.0009) [2023-10-08 04:30:37,979][00612] Updated weights for policy 1, policy_version 14600 (0.0008) [2023-10-08 04:30:38,354][00612] Updated weights for policy 1, policy_version 14610 (0.0008) [2023-10-08 04:30:38,737][00612] Updated weights for policy 1, policy_version 14620 (0.0010) [2023-10-08 04:30:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 29818880. Throughput: 0: 1819.5, 1: 1837.1. Samples: 7464120. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 04:30:38,755][130385] Avg episode reward: [(0, '43.500'), (1, '36.310')] [2023-10-08 04:30:38,969][00611] Updated weights for policy 0, policy_version 14532 (0.0007) [2023-10-08 04:30:39,352][00611] Updated weights for policy 0, policy_version 14542 (0.0008) [2023-10-08 04:30:39,723][00611] Updated weights for policy 0, policy_version 14552 (0.0007) [2023-10-08 04:30:40,023][00365] Saving new best policy, reward=43.500! [2023-10-08 04:30:42,425][00612] Updated weights for policy 1, policy_version 14630 (0.0007) [2023-10-08 04:30:42,795][00612] Updated weights for policy 1, policy_version 14640 (0.0007) [2023-10-08 04:30:43,169][00612] Updated weights for policy 1, policy_version 14650 (0.0007) [2023-10-08 04:30:43,383][00611] Updated weights for policy 0, policy_version 14562 (0.0009) [2023-10-08 04:30:43,748][00611] Updated weights for policy 0, policy_version 14572 (0.0008) [2023-10-08 04:30:43,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 29917184. Throughput: 0: 1818.2, 1: 1839.8. Samples: 7486598. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-08 04:30:43,754][130385] Avg episode reward: [(0, '42.430'), (1, '36.760')] [2023-10-08 04:30:44,117][00611] Updated weights for policy 0, policy_version 14582 (0.0009) [2023-10-08 04:30:44,487][00611] Updated weights for policy 0, policy_version 14592 (0.0010) [2023-10-08 04:30:46,807][00612] Updated weights for policy 1, policy_version 14660 (0.0007) [2023-10-08 04:30:47,177][00612] Updated weights for policy 1, policy_version 14670 (0.0008) [2023-10-08 04:30:47,536][00612] Updated weights for policy 1, policy_version 14680 (0.0010) [2023-10-08 04:30:48,146][00611] Updated weights for policy 0, policy_version 14602 (0.0008) [2023-10-08 04:30:48,516][00611] Updated weights for policy 0, policy_version 14612 (0.0009) [2023-10-08 04:30:48,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 29982720. Throughput: 0: 1825.3, 1: 1838.1. Samples: 7508012. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-08 04:30:48,755][130385] Avg episode reward: [(0, '41.710'), (1, '35.030')] [2023-10-08 04:30:48,894][00611] Updated weights for policy 0, policy_version 14622 (0.0008) [2023-10-08 04:30:51,249][00612] Updated weights for policy 1, policy_version 14690 (0.0009) [2023-10-08 04:30:51,620][00612] Updated weights for policy 1, policy_version 14700 (0.0010) [2023-10-08 04:30:51,984][00612] Updated weights for policy 1, policy_version 14710 (0.0009) [2023-10-08 04:30:52,352][00612] Updated weights for policy 1, policy_version 14720 (0.0008) [2023-10-08 04:30:52,627][00611] Updated weights for policy 0, policy_version 14632 (0.0008) [2023-10-08 04:30:53,001][00611] Updated weights for policy 0, policy_version 14642 (0.0009) [2023-10-08 04:30:53,365][00611] Updated weights for policy 0, policy_version 14652 (0.0007) [2023-10-08 04:30:53,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 30081024. Throughput: 0: 1823.1, 1: 1845.4. Samples: 7519618. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-08 04:30:53,754][130385] Avg episode reward: [(0, '40.440'), (1, '38.640')] [2023-10-08 04:30:55,785][00612] Updated weights for policy 1, policy_version 14730 (0.0008) [2023-10-08 04:30:56,155][00612] Updated weights for policy 1, policy_version 14740 (0.0008) [2023-10-08 04:30:56,524][00612] Updated weights for policy 1, policy_version 14750 (0.0009) [2023-10-08 04:30:57,065][00611] Updated weights for policy 0, policy_version 14662 (0.0009) [2023-10-08 04:30:57,433][00611] Updated weights for policy 0, policy_version 14672 (0.0008) [2023-10-08 04:30:57,811][00611] Updated weights for policy 0, policy_version 14682 (0.0008) [2023-10-08 04:30:58,754][130385] Fps is (10 sec: 16384.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 30146560. Throughput: 0: 1819.7, 1: 1847.4. Samples: 7541100. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-08 04:30:58,754][130385] Avg episode reward: [(0, '40.700'), (1, '38.430')] [2023-10-08 04:31:00,177][00612] Updated weights for policy 1, policy_version 14760 (0.0010) [2023-10-08 04:31:00,541][00612] Updated weights for policy 1, policy_version 14770 (0.0011) [2023-10-08 04:31:00,919][00612] Updated weights for policy 1, policy_version 14780 (0.0012) [2023-10-08 04:31:01,536][00611] Updated weights for policy 0, policy_version 14692 (0.0009) [2023-10-08 04:31:01,912][00611] Updated weights for policy 0, policy_version 14702 (0.0008) [2023-10-08 04:31:02,284][00611] Updated weights for policy 0, policy_version 14712 (0.0007) [2023-10-08 04:31:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 30212096. Throughput: 0: 1813.9, 1: 1849.0. Samples: 7562906. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-08 04:31:03,755][130385] Avg episode reward: [(0, '39.670'), (1, '39.400')] [2023-10-08 04:31:04,504][00612] Updated weights for policy 1, policy_version 14790 (0.0010) [2023-10-08 04:31:04,884][00612] Updated weights for policy 1, policy_version 14800 (0.0007) [2023-10-08 04:31:05,252][00612] Updated weights for policy 1, policy_version 14810 (0.0008) [2023-10-08 04:31:05,931][00611] Updated weights for policy 0, policy_version 14722 (0.0008) [2023-10-08 04:31:06,302][00611] Updated weights for policy 0, policy_version 14732 (0.0007) [2023-10-08 04:31:06,674][00611] Updated weights for policy 0, policy_version 14742 (0.0007) [2023-10-08 04:31:07,049][00611] Updated weights for policy 0, policy_version 14752 (0.0009) [2023-10-08 04:31:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 30277632. Throughput: 0: 1813.0, 1: 1849.7. Samples: 7574054. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 04:31:08,754][130385] Avg episode reward: [(0, '39.080'), (1, '39.480')] [2023-10-08 04:31:08,799][00612] Updated weights for policy 1, policy_version 14820 (0.0008) [2023-10-08 04:31:09,172][00612] Updated weights for policy 1, policy_version 14830 (0.0007) [2023-10-08 04:31:09,538][00612] Updated weights for policy 1, policy_version 14840 (0.0007) [2023-10-08 04:31:10,751][00611] Updated weights for policy 0, policy_version 14762 (0.0008) [2023-10-08 04:31:11,129][00611] Updated weights for policy 0, policy_version 14772 (0.0007) [2023-10-08 04:31:11,501][00611] Updated weights for policy 0, policy_version 14782 (0.0008) [2023-10-08 04:31:13,177][00612] Updated weights for policy 1, policy_version 14850 (0.0008) [2023-10-08 04:31:13,543][00612] Updated weights for policy 1, policy_version 14860 (0.0007) [2023-10-08 04:31:13,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 30343168. Throughput: 0: 1821.3, 1: 1850.1. Samples: 7596312. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 04:31:13,754][130385] Avg episode reward: [(0, '40.370'), (1, '38.890')] [2023-10-08 04:31:13,911][00612] Updated weights for policy 1, policy_version 14870 (0.0009) [2023-10-08 04:31:14,282][00612] Updated weights for policy 1, policy_version 14880 (0.0009) [2023-10-08 04:31:15,114][00611] Updated weights for policy 0, policy_version 14792 (0.0009) [2023-10-08 04:31:15,487][00611] Updated weights for policy 0, policy_version 14802 (0.0009) [2023-10-08 04:31:15,858][00611] Updated weights for policy 0, policy_version 14812 (0.0008) [2023-10-08 04:31:17,828][00612] Updated weights for policy 1, policy_version 14890 (0.0008) [2023-10-08 04:31:18,207][00612] Updated weights for policy 1, policy_version 14900 (0.0009) [2023-10-08 04:31:18,565][00612] Updated weights for policy 1, policy_version 14910 (0.0007) [2023-10-08 04:31:18,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 30441472. Throughput: 0: 1819.7, 1: 1834.8. Samples: 7618638. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 04:31:18,755][130385] Avg episode reward: [(0, '37.290'), (1, '38.110')] [2023-10-08 04:31:19,613][00611] Updated weights for policy 0, policy_version 14822 (0.0011) [2023-10-08 04:31:19,992][00611] Updated weights for policy 0, policy_version 14832 (0.0008) [2023-10-08 04:31:20,373][00611] Updated weights for policy 0, policy_version 14842 (0.0009) [2023-10-08 04:31:22,064][00612] Updated weights for policy 1, policy_version 14920 (0.0007) [2023-10-08 04:31:22,425][00612] Updated weights for policy 1, policy_version 14930 (0.0007) [2023-10-08 04:31:22,793][00612] Updated weights for policy 1, policy_version 14940 (0.0008) [2023-10-08 04:31:23,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 30507008. Throughput: 0: 1815.0, 1: 1859.7. Samples: 7629480. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 04:31:23,754][130385] Avg episode reward: [(0, '37.480'), (1, '38.370')] [2023-10-08 04:31:23,998][00611] Updated weights for policy 0, policy_version 14852 (0.0008) [2023-10-08 04:31:24,365][00611] Updated weights for policy 0, policy_version 14862 (0.0007) [2023-10-08 04:31:24,744][00611] Updated weights for policy 0, policy_version 14872 (0.0007) [2023-10-08 04:31:26,544][00612] Updated weights for policy 1, policy_version 14950 (0.0007) [2023-10-08 04:31:26,927][00612] Updated weights for policy 1, policy_version 14960 (0.0010) [2023-10-08 04:31:27,300][00612] Updated weights for policy 1, policy_version 14970 (0.0007) [2023-10-08 04:31:28,426][00611] Updated weights for policy 0, policy_version 14882 (0.0008) [2023-10-08 04:31:28,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 30572544. Throughput: 0: 1822.9, 1: 1840.0. Samples: 7651430. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 04:31:28,754][130385] Avg episode reward: [(0, '36.700'), (1, '36.500')] [2023-10-08 04:31:28,794][00611] Updated weights for policy 0, policy_version 14892 (0.0008) [2023-10-08 04:31:29,164][00611] Updated weights for policy 0, policy_version 14902 (0.0007) [2023-10-08 04:31:29,536][00611] Updated weights for policy 0, policy_version 14912 (0.0007) [2023-10-08 04:31:30,889][00612] Updated weights for policy 1, policy_version 14980 (0.0008) [2023-10-08 04:31:31,256][00612] Updated weights for policy 1, policy_version 14990 (0.0008) [2023-10-08 04:31:31,627][00612] Updated weights for policy 1, policy_version 15000 (0.0008) [2023-10-08 04:31:33,280][00611] Updated weights for policy 0, policy_version 14922 (0.0009) [2023-10-08 04:31:33,654][00611] Updated weights for policy 0, policy_version 14932 (0.0010) [2023-10-08 04:31:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 30638080. Throughput: 0: 1818.1, 1: 1864.0. Samples: 7673702. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 04:31:33,754][130385] Avg episode reward: [(0, '35.520'), (1, '34.570')] [2023-10-08 04:31:34,032][00611] Updated weights for policy 0, policy_version 14942 (0.0011) [2023-10-08 04:31:35,351][00612] Updated weights for policy 1, policy_version 15010 (0.0010) [2023-10-08 04:31:35,723][00612] Updated weights for policy 1, policy_version 15020 (0.0009) [2023-10-08 04:31:36,085][00612] Updated weights for policy 1, policy_version 15030 (0.0008) [2023-10-08 04:31:36,456][00612] Updated weights for policy 1, policy_version 15040 (0.0008) [2023-10-08 04:31:37,812][00611] Updated weights for policy 0, policy_version 14952 (0.0008) [2023-10-08 04:31:38,187][00611] Updated weights for policy 0, policy_version 14962 (0.0010) [2023-10-08 04:31:38,572][00611] Updated weights for policy 0, policy_version 14972 (0.0010) [2023-10-08 04:31:38,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 30736384. Throughput: 0: 1817.7, 1: 1838.0. Samples: 7684128. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 04:31:38,754][130385] Avg episode reward: [(0, '35.420'), (1, '34.840')] [2023-10-08 04:31:40,199][00612] Updated weights for policy 1, policy_version 15050 (0.0010) [2023-10-08 04:31:40,565][00612] Updated weights for policy 1, policy_version 15060 (0.0009) [2023-10-08 04:31:40,945][00612] Updated weights for policy 1, policy_version 15070 (0.0007) [2023-10-08 04:31:42,254][00611] Updated weights for policy 0, policy_version 14982 (0.0009) [2023-10-08 04:31:42,613][00611] Updated weights for policy 0, policy_version 14992 (0.0008) [2023-10-08 04:31:42,987][00611] Updated weights for policy 0, policy_version 15002 (0.0007) [2023-10-08 04:31:43,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 30801920. Throughput: 0: 1823.5, 1: 1855.5. Samples: 7706656. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 04:31:43,754][130385] Avg episode reward: [(0, '35.460'), (1, '35.340')] [2023-10-08 04:31:44,582][00612] Updated weights for policy 1, policy_version 15080 (0.0007) [2023-10-08 04:31:44,949][00612] Updated weights for policy 1, policy_version 15090 (0.0007) [2023-10-08 04:31:45,331][00612] Updated weights for policy 1, policy_version 15100 (0.0009) [2023-10-08 04:31:46,550][00611] Updated weights for policy 0, policy_version 15012 (0.0007) [2023-10-08 04:31:46,922][00611] Updated weights for policy 0, policy_version 15022 (0.0009) [2023-10-08 04:31:47,296][00611] Updated weights for policy 0, policy_version 15032 (0.0008) [2023-10-08 04:31:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 30867456. Throughput: 0: 1830.0, 1: 1857.2. Samples: 7728828. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-08 04:31:48,755][130385] Avg episode reward: [(0, '32.780'), (1, '35.730')] [2023-10-08 04:31:48,919][00612] Updated weights for policy 1, policy_version 15110 (0.0009) [2023-10-08 04:31:49,291][00612] Updated weights for policy 1, policy_version 15120 (0.0009) [2023-10-08 04:31:49,676][00612] Updated weights for policy 1, policy_version 15130 (0.0008) [2023-10-08 04:31:51,026][00611] Updated weights for policy 0, policy_version 15042 (0.0007) [2023-10-08 04:31:51,402][00611] Updated weights for policy 0, policy_version 15052 (0.0007) [2023-10-08 04:31:51,770][00611] Updated weights for policy 0, policy_version 15062 (0.0008) [2023-10-08 04:31:52,139][00611] Updated weights for policy 0, policy_version 15072 (0.0008) [2023-10-08 04:31:53,317][00612] Updated weights for policy 1, policy_version 15140 (0.0010) [2023-10-08 04:31:53,694][00612] Updated weights for policy 1, policy_version 15150 (0.0008) [2023-10-08 04:31:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 30932992. Throughput: 0: 1832.0, 1: 1853.0. Samples: 7739880. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-08 04:31:53,754][130385] Avg episode reward: [(0, '33.080'), (1, '35.120')] [2023-10-08 04:31:54,059][00612] Updated weights for policy 1, policy_version 15160 (0.0007) [2023-10-08 04:31:55,710][00611] Updated weights for policy 0, policy_version 15082 (0.0007) [2023-10-08 04:31:56,081][00611] Updated weights for policy 0, policy_version 15092 (0.0007) [2023-10-08 04:31:56,451][00611] Updated weights for policy 0, policy_version 15102 (0.0007) [2023-10-08 04:31:57,748][00612] Updated weights for policy 1, policy_version 15170 (0.0008) [2023-10-08 04:31:58,116][00612] Updated weights for policy 1, policy_version 15180 (0.0009) [2023-10-08 04:31:58,484][00612] Updated weights for policy 1, policy_version 15190 (0.0009) [2023-10-08 04:31:58,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 30998528. Throughput: 0: 1833.4, 1: 1843.1. Samples: 7761758. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-08 04:31:58,755][130385] Avg episode reward: [(0, '34.620'), (1, '32.950')] [2023-10-08 04:31:58,850][00612] Updated weights for policy 1, policy_version 15200 (0.0009) [2023-10-08 04:31:59,887][00611] Updated weights for policy 0, policy_version 15112 (0.0008) [2023-10-08 04:32:00,263][00611] Updated weights for policy 0, policy_version 15122 (0.0008) [2023-10-08 04:32:00,638][00611] Updated weights for policy 0, policy_version 15132 (0.0007) [2023-10-08 04:32:02,407][00612] Updated weights for policy 1, policy_version 15210 (0.0008) [2023-10-08 04:32:02,784][00612] Updated weights for policy 1, policy_version 15220 (0.0010) [2023-10-08 04:32:03,153][00612] Updated weights for policy 1, policy_version 15230 (0.0008) [2023-10-08 04:32:03,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 31096832. Throughput: 0: 1832.2, 1: 1830.4. Samples: 7783454. Policy #0 lag: (min: 24.0, avg: 50.0, max: 56.0) [2023-10-08 04:32:03,755][130385] Avg episode reward: [(0, '34.970'), (1, '34.070')] [2023-10-08 04:32:04,306][00611] Updated weights for policy 0, policy_version 15142 (0.0007) [2023-10-08 04:32:04,690][00611] Updated weights for policy 0, policy_version 15152 (0.0007) [2023-10-08 04:32:05,059][00611] Updated weights for policy 0, policy_version 15162 (0.0007) [2023-10-08 04:32:06,850][00612] Updated weights for policy 1, policy_version 15240 (0.0010) [2023-10-08 04:32:07,221][00612] Updated weights for policy 1, policy_version 15250 (0.0009) [2023-10-08 04:32:07,596][00612] Updated weights for policy 1, policy_version 15260 (0.0007) [2023-10-08 04:32:08,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31162368. Throughput: 0: 1836.7, 1: 1833.8. Samples: 7794652. Policy #0 lag: (min: 24.0, avg: 50.0, max: 56.0) [2023-10-08 04:32:08,754][130385] Avg episode reward: [(0, '38.460'), (1, '33.600')] [2023-10-08 04:32:08,782][00611] Updated weights for policy 0, policy_version 15172 (0.0008) [2023-10-08 04:32:09,154][00611] Updated weights for policy 0, policy_version 15182 (0.0009) [2023-10-08 04:32:09,521][00611] Updated weights for policy 0, policy_version 15192 (0.0009) [2023-10-08 04:32:11,301][00612] Updated weights for policy 1, policy_version 15270 (0.0009) [2023-10-08 04:32:11,666][00612] Updated weights for policy 1, policy_version 15280 (0.0009) [2023-10-08 04:32:12,037][00612] Updated weights for policy 1, policy_version 15290 (0.0009) [2023-10-08 04:32:13,274][00611] Updated weights for policy 0, policy_version 15202 (0.0008) [2023-10-08 04:32:13,643][00611] Updated weights for policy 0, policy_version 15212 (0.0010) [2023-10-08 04:32:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31227904. Throughput: 0: 1832.7, 1: 1833.9. Samples: 7816426. Policy #0 lag: (min: 24.0, avg: 50.0, max: 56.0) [2023-10-08 04:32:13,755][130385] Avg episode reward: [(0, '34.510'), (1, '34.450')] [2023-10-08 04:32:14,011][00611] Updated weights for policy 0, policy_version 15222 (0.0009) [2023-10-08 04:32:14,380][00611] Updated weights for policy 0, policy_version 15232 (0.0008) [2023-10-08 04:32:15,649][00612] Updated weights for policy 1, policy_version 15300 (0.0010) [2023-10-08 04:32:16,030][00612] Updated weights for policy 1, policy_version 15310 (0.0008) [2023-10-08 04:32:16,409][00612] Updated weights for policy 1, policy_version 15320 (0.0008) [2023-10-08 04:32:17,860][00611] Updated weights for policy 0, policy_version 15242 (0.0007) [2023-10-08 04:32:18,236][00611] Updated weights for policy 0, policy_version 15252 (0.0009) [2023-10-08 04:32:18,611][00611] Updated weights for policy 0, policy_version 15262 (0.0009) [2023-10-08 04:32:18,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 31326208. Throughput: 0: 1826.4, 1: 1839.4. Samples: 7838664. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-08 04:32:18,755][130385] Avg episode reward: [(0, '34.150'), (1, '35.540')] [2023-10-08 04:32:18,763][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000015264_15630336.pth... [2023-10-08 04:32:18,764][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000015328_15695872.pth... [2023-10-08 04:32:18,804][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000013536_13860864.pth [2023-10-08 04:32:18,810][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000013632_13959168.pth [2023-10-08 04:32:20,014][00612] Updated weights for policy 1, policy_version 15330 (0.0010) [2023-10-08 04:32:20,384][00612] Updated weights for policy 1, policy_version 15340 (0.0009) [2023-10-08 04:32:20,752][00612] Updated weights for policy 1, policy_version 15350 (0.0011) [2023-10-08 04:32:21,125][00612] Updated weights for policy 1, policy_version 15360 (0.0010) [2023-10-08 04:32:22,494][00611] Updated weights for policy 0, policy_version 15272 (0.0008) [2023-10-08 04:32:22,878][00611] Updated weights for policy 0, policy_version 15282 (0.0010) [2023-10-08 04:32:23,253][00611] Updated weights for policy 0, policy_version 15292 (0.0009) [2023-10-08 04:32:23,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 31391744. Throughput: 0: 1839.1, 1: 1830.3. Samples: 7849252. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-08 04:32:23,754][130385] Avg episode reward: [(0, '34.940'), (1, '35.300')] [2023-10-08 04:32:24,821][00612] Updated weights for policy 1, policy_version 15370 (0.0008) [2023-10-08 04:32:25,194][00612] Updated weights for policy 1, policy_version 15380 (0.0008) [2023-10-08 04:32:25,563][00612] Updated weights for policy 1, policy_version 15390 (0.0010) [2023-10-08 04:32:27,023][00611] Updated weights for policy 0, policy_version 15302 (0.0009) [2023-10-08 04:32:27,388][00611] Updated weights for policy 0, policy_version 15312 (0.0007) [2023-10-08 04:32:27,770][00611] Updated weights for policy 0, policy_version 15322 (0.0007) [2023-10-08 04:32:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 31457280. Throughput: 0: 1828.4, 1: 1842.9. Samples: 7871862. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:32:28,754][130385] Avg episode reward: [(0, '35.250'), (1, '34.440')] [2023-10-08 04:32:29,224][00612] Updated weights for policy 1, policy_version 15400 (0.0011) [2023-10-08 04:32:29,590][00612] Updated weights for policy 1, policy_version 15410 (0.0008) [2023-10-08 04:32:29,966][00612] Updated weights for policy 1, policy_version 15420 (0.0007) [2023-10-08 04:32:31,283][00611] Updated weights for policy 0, policy_version 15332 (0.0009) [2023-10-08 04:32:31,660][00611] Updated weights for policy 0, policy_version 15342 (0.0008) [2023-10-08 04:32:32,024][00611] Updated weights for policy 0, policy_version 15352 (0.0007) [2023-10-08 04:32:33,496][00612] Updated weights for policy 1, policy_version 15430 (0.0009) [2023-10-08 04:32:33,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 31522816. Throughput: 0: 1831.1, 1: 1835.5. Samples: 7893822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:32:33,755][130385] Avg episode reward: [(0, '37.390'), (1, '37.090')] [2023-10-08 04:32:33,864][00612] Updated weights for policy 1, policy_version 15440 (0.0009) [2023-10-08 04:32:34,229][00612] Updated weights for policy 1, policy_version 15450 (0.0008) [2023-10-08 04:32:35,710][00611] Updated weights for policy 0, policy_version 15362 (0.0007) [2023-10-08 04:32:36,077][00611] Updated weights for policy 0, policy_version 15372 (0.0008) [2023-10-08 04:32:36,457][00611] Updated weights for policy 0, policy_version 15382 (0.0007) [2023-10-08 04:32:36,825][00611] Updated weights for policy 0, policy_version 15392 (0.0007) [2023-10-08 04:32:37,931][00612] Updated weights for policy 1, policy_version 15460 (0.0008) [2023-10-08 04:32:38,293][00612] Updated weights for policy 1, policy_version 15470 (0.0011) [2023-10-08 04:32:38,658][00612] Updated weights for policy 1, policy_version 15480 (0.0010) [2023-10-08 04:32:38,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 31588352. Throughput: 0: 1824.2, 1: 1838.3. Samples: 7904692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:32:38,755][130385] Avg episode reward: [(0, '39.010'), (1, '37.410')] [2023-10-08 04:32:40,393][00611] Updated weights for policy 0, policy_version 15402 (0.0011) [2023-10-08 04:32:40,765][00611] Updated weights for policy 0, policy_version 15412 (0.0010) [2023-10-08 04:32:41,152][00611] Updated weights for policy 0, policy_version 15422 (0.0009) [2023-10-08 04:32:42,395][00612] Updated weights for policy 1, policy_version 15490 (0.0010) [2023-10-08 04:32:42,756][00612] Updated weights for policy 1, policy_version 15500 (0.0007) [2023-10-08 04:32:43,119][00612] Updated weights for policy 1, policy_version 15510 (0.0007) [2023-10-08 04:32:43,490][00612] Updated weights for policy 1, policy_version 15520 (0.0008) [2023-10-08 04:32:43,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 31686656. Throughput: 0: 1823.3, 1: 1842.3. Samples: 7926710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:32:43,754][130385] Avg episode reward: [(0, '38.470'), (1, '37.700')] [2023-10-08 04:32:44,777][00611] Updated weights for policy 0, policy_version 15432 (0.0009) [2023-10-08 04:32:45,155][00611] Updated weights for policy 0, policy_version 15442 (0.0010) [2023-10-08 04:32:45,526][00611] Updated weights for policy 0, policy_version 15452 (0.0011) [2023-10-08 04:32:47,102][00612] Updated weights for policy 1, policy_version 15530 (0.0011) [2023-10-08 04:32:47,464][00612] Updated weights for policy 1, policy_version 15540 (0.0010) [2023-10-08 04:32:47,839][00612] Updated weights for policy 1, policy_version 15550 (0.0009) [2023-10-08 04:32:48,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 31752192. Throughput: 0: 1827.9, 1: 1839.4. Samples: 7948484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:32:48,755][130385] Avg episode reward: [(0, '36.880'), (1, '35.880')] [2023-10-08 04:32:49,132][00611] Updated weights for policy 0, policy_version 15462 (0.0009) [2023-10-08 04:32:49,517][00611] Updated weights for policy 0, policy_version 15472 (0.0011) [2023-10-08 04:32:49,888][00611] Updated weights for policy 0, policy_version 15482 (0.0008) [2023-10-08 04:32:51,535][00612] Updated weights for policy 1, policy_version 15560 (0.0008) [2023-10-08 04:32:51,903][00612] Updated weights for policy 1, policy_version 15570 (0.0010) [2023-10-08 04:32:52,281][00612] Updated weights for policy 1, policy_version 15580 (0.0010) [2023-10-08 04:32:53,638][00611] Updated weights for policy 0, policy_version 15492 (0.0007) [2023-10-08 04:32:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 31817728. Throughput: 0: 1828.0, 1: 1842.5. Samples: 7959824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:32:53,754][130385] Avg episode reward: [(0, '36.330'), (1, '35.230')] [2023-10-08 04:32:54,011][00611] Updated weights for policy 0, policy_version 15502 (0.0009) [2023-10-08 04:32:54,378][00611] Updated weights for policy 0, policy_version 15512 (0.0010) [2023-10-08 04:32:55,886][00612] Updated weights for policy 1, policy_version 15590 (0.0009) [2023-10-08 04:32:56,256][00612] Updated weights for policy 1, policy_version 15600 (0.0010) [2023-10-08 04:32:56,634][00612] Updated weights for policy 1, policy_version 15610 (0.0008) [2023-10-08 04:32:58,083][00611] Updated weights for policy 0, policy_version 15522 (0.0009) [2023-10-08 04:32:58,454][00611] Updated weights for policy 0, policy_version 15532 (0.0008) [2023-10-08 04:32:58,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 31883264. Throughput: 0: 1823.2, 1: 1838.1. Samples: 7981186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:32:58,754][130385] Avg episode reward: [(0, '37.080'), (1, '35.700')] [2023-10-08 04:32:58,823][00611] Updated weights for policy 0, policy_version 15542 (0.0008) [2023-10-08 04:32:59,191][00611] Updated weights for policy 0, policy_version 15552 (0.0007) [2023-10-08 04:33:00,118][00612] Updated weights for policy 1, policy_version 15620 (0.0009) [2023-10-08 04:33:00,488][00612] Updated weights for policy 1, policy_version 15630 (0.0009) [2023-10-08 04:33:00,867][00612] Updated weights for policy 1, policy_version 15640 (0.0007) [2023-10-08 04:33:02,819][00611] Updated weights for policy 0, policy_version 15562 (0.0007) [2023-10-08 04:33:03,195][00611] Updated weights for policy 0, policy_version 15572 (0.0010) [2023-10-08 04:33:03,568][00611] Updated weights for policy 0, policy_version 15582 (0.0008) [2023-10-08 04:33:03,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 31981568. Throughput: 0: 1820.6, 1: 1844.4. Samples: 8003592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:33:03,754][130385] Avg episode reward: [(0, '38.260'), (1, '37.070')] [2023-10-08 04:33:04,610][00612] Updated weights for policy 1, policy_version 15650 (0.0008) [2023-10-08 04:33:05,015][00612] Updated weights for policy 1, policy_version 15660 (0.0011) [2023-10-08 04:33:05,375][00612] Updated weights for policy 1, policy_version 15670 (0.0010) [2023-10-08 04:33:05,744][00612] Updated weights for policy 1, policy_version 15680 (0.0008) [2023-10-08 04:33:07,303][00611] Updated weights for policy 0, policy_version 15592 (0.0010) [2023-10-08 04:33:07,679][00611] Updated weights for policy 0, policy_version 15602 (0.0010) [2023-10-08 04:33:08,042][00611] Updated weights for policy 0, policy_version 15612 (0.0011) [2023-10-08 04:33:08,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 32047104. Throughput: 0: 1823.8, 1: 1839.8. Samples: 8014114. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 04:33:08,754][130385] Avg episode reward: [(0, '38.230'), (1, '36.310')] [2023-10-08 04:33:09,327][00612] Updated weights for policy 1, policy_version 15690 (0.0007) [2023-10-08 04:33:09,709][00612] Updated weights for policy 1, policy_version 15700 (0.0008) [2023-10-08 04:33:10,070][00612] Updated weights for policy 1, policy_version 15710 (0.0010) [2023-10-08 04:33:11,780][00611] Updated weights for policy 0, policy_version 15622 (0.0008) [2023-10-08 04:33:12,161][00611] Updated weights for policy 0, policy_version 15632 (0.0007) [2023-10-08 04:33:12,536][00611] Updated weights for policy 0, policy_version 15642 (0.0008) [2023-10-08 04:33:13,743][00612] Updated weights for policy 1, policy_version 15720 (0.0008) [2023-10-08 04:33:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 32112640. Throughput: 0: 1814.8, 1: 1838.4. Samples: 8036260. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 04:33:13,755][130385] Avg episode reward: [(0, '37.200'), (1, '33.860')] [2023-10-08 04:33:14,114][00612] Updated weights for policy 1, policy_version 15730 (0.0008) [2023-10-08 04:33:14,491][00612] Updated weights for policy 1, policy_version 15740 (0.0008) [2023-10-08 04:33:16,202][00611] Updated weights for policy 0, policy_version 15652 (0.0007) [2023-10-08 04:33:16,577][00611] Updated weights for policy 0, policy_version 15662 (0.0008) [2023-10-08 04:33:16,956][00611] Updated weights for policy 0, policy_version 15672 (0.0008) [2023-10-08 04:33:18,162][00612] Updated weights for policy 1, policy_version 15750 (0.0010) [2023-10-08 04:33:18,527][00612] Updated weights for policy 1, policy_version 15760 (0.0008) [2023-10-08 04:33:18,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 32178176. Throughput: 0: 1820.5, 1: 1829.2. Samples: 8058060. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 04:33:18,754][130385] Avg episode reward: [(0, '36.720'), (1, '34.210')] [2023-10-08 04:33:18,891][00612] Updated weights for policy 1, policy_version 15770 (0.0007) [2023-10-08 04:33:20,752][00611] Updated weights for policy 0, policy_version 15682 (0.0007) [2023-10-08 04:33:21,126][00611] Updated weights for policy 0, policy_version 15692 (0.0007) [2023-10-08 04:33:21,500][00611] Updated weights for policy 0, policy_version 15702 (0.0008) [2023-10-08 04:33:21,874][00611] Updated weights for policy 0, policy_version 15712 (0.0009) [2023-10-08 04:33:22,559][00612] Updated weights for policy 1, policy_version 15780 (0.0009) [2023-10-08 04:33:22,931][00612] Updated weights for policy 1, policy_version 15790 (0.0007) [2023-10-08 04:33:23,294][00612] Updated weights for policy 1, policy_version 15800 (0.0008) [2023-10-08 04:33:23,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 32276480. Throughput: 0: 1823.3, 1: 1837.0. Samples: 8069404. Policy #0 lag: (min: 1.0, avg: 3.0, max: 29.0) [2023-10-08 04:33:23,754][130385] Avg episode reward: [(0, '37.400'), (1, '34.700')] [2023-10-08 04:33:25,343][00611] Updated weights for policy 0, policy_version 15722 (0.0008) [2023-10-08 04:33:25,719][00611] Updated weights for policy 0, policy_version 15732 (0.0007) [2023-10-08 04:33:26,088][00611] Updated weights for policy 0, policy_version 15742 (0.0008) [2023-10-08 04:33:26,948][00612] Updated weights for policy 1, policy_version 15810 (0.0009) [2023-10-08 04:33:27,317][00612] Updated weights for policy 1, policy_version 15820 (0.0009) [2023-10-08 04:33:27,680][00612] Updated weights for policy 1, policy_version 15830 (0.0010) [2023-10-08 04:33:28,050][00612] Updated weights for policy 1, policy_version 15840 (0.0009) [2023-10-08 04:33:28,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 32342016. Throughput: 0: 1826.4, 1: 1827.3. Samples: 8091128. Policy #0 lag: (min: 1.0, avg: 3.0, max: 29.0) [2023-10-08 04:33:28,755][130385] Avg episode reward: [(0, '39.920'), (1, '34.880')] [2023-10-08 04:33:29,815][00611] Updated weights for policy 0, policy_version 15752 (0.0008) [2023-10-08 04:33:30,181][00611] Updated weights for policy 0, policy_version 15762 (0.0007) [2023-10-08 04:33:30,562][00611] Updated weights for policy 0, policy_version 15772 (0.0008) [2023-10-08 04:33:31,601][00612] Updated weights for policy 1, policy_version 15850 (0.0007) [2023-10-08 04:33:31,966][00612] Updated weights for policy 1, policy_version 15860 (0.0009) [2023-10-08 04:33:32,340][00612] Updated weights for policy 1, policy_version 15870 (0.0007) [2023-10-08 04:33:33,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 32407552. Throughput: 0: 1822.7, 1: 1843.6. Samples: 8113464. Policy #0 lag: (min: 1.0, avg: 3.0, max: 29.0) [2023-10-08 04:33:33,755][130385] Avg episode reward: [(0, '40.450'), (1, '37.950')] [2023-10-08 04:33:34,205][00611] Updated weights for policy 0, policy_version 15782 (0.0010) [2023-10-08 04:33:34,572][00611] Updated weights for policy 0, policy_version 15792 (0.0011) [2023-10-08 04:33:34,945][00611] Updated weights for policy 0, policy_version 15802 (0.0008) [2023-10-08 04:33:35,975][00612] Updated weights for policy 1, policy_version 15880 (0.0007) [2023-10-08 04:33:36,342][00612] Updated weights for policy 1, policy_version 15890 (0.0009) [2023-10-08 04:33:36,712][00612] Updated weights for policy 1, policy_version 15900 (0.0008) [2023-10-08 04:33:38,554][00611] Updated weights for policy 0, policy_version 15812 (0.0009) [2023-10-08 04:33:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 32473088. Throughput: 0: 1820.5, 1: 1830.5. Samples: 8124118. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-08 04:33:38,755][130385] Avg episode reward: [(0, '40.750'), (1, '35.090')] [2023-10-08 04:33:38,923][00611] Updated weights for policy 0, policy_version 15822 (0.0010) [2023-10-08 04:33:39,297][00611] Updated weights for policy 0, policy_version 15832 (0.0009) [2023-10-08 04:33:40,136][00612] Updated weights for policy 1, policy_version 15910 (0.0008) [2023-10-08 04:33:40,507][00612] Updated weights for policy 1, policy_version 15920 (0.0007) [2023-10-08 04:33:40,872][00612] Updated weights for policy 1, policy_version 15930 (0.0010) [2023-10-08 04:33:43,029][00611] Updated weights for policy 0, policy_version 15842 (0.0009) [2023-10-08 04:33:43,407][00611] Updated weights for policy 0, policy_version 15852 (0.0008) [2023-10-08 04:33:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 32538624. Throughput: 0: 1828.5, 1: 1842.6. Samples: 8146388. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-08 04:33:43,754][130385] Avg episode reward: [(0, '39.410'), (1, '35.230')] [2023-10-08 04:33:43,773][00611] Updated weights for policy 0, policy_version 15862 (0.0008) [2023-10-08 04:33:44,141][00611] Updated weights for policy 0, policy_version 15872 (0.0009) [2023-10-08 04:33:44,666][00612] Updated weights for policy 1, policy_version 15940 (0.0010) [2023-10-08 04:33:45,027][00612] Updated weights for policy 1, policy_version 15950 (0.0008) [2023-10-08 04:33:45,402][00612] Updated weights for policy 1, policy_version 15960 (0.0008) [2023-10-08 04:33:47,679][00611] Updated weights for policy 0, policy_version 15882 (0.0007) [2023-10-08 04:33:48,061][00611] Updated weights for policy 0, policy_version 15892 (0.0007) [2023-10-08 04:33:48,432][00611] Updated weights for policy 0, policy_version 15902 (0.0008) [2023-10-08 04:33:48,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 32636928. Throughput: 0: 1825.2, 1: 1845.3. Samples: 8168768. Policy #0 lag: (min: 6.0, avg: 28.4, max: 32.0) [2023-10-08 04:33:48,754][130385] Avg episode reward: [(0, '37.590'), (1, '33.930')] [2023-10-08 04:33:48,939][00612] Updated weights for policy 1, policy_version 15970 (0.0009) [2023-10-08 04:33:49,314][00612] Updated weights for policy 1, policy_version 15980 (0.0011) [2023-10-08 04:33:49,685][00612] Updated weights for policy 1, policy_version 15990 (0.0009) [2023-10-08 04:33:50,057][00612] Updated weights for policy 1, policy_version 16000 (0.0008) [2023-10-08 04:33:52,045][00611] Updated weights for policy 0, policy_version 15912 (0.0009) [2023-10-08 04:33:52,423][00611] Updated weights for policy 0, policy_version 15922 (0.0007) [2023-10-08 04:33:52,785][00611] Updated weights for policy 0, policy_version 15932 (0.0008) [2023-10-08 04:33:53,685][00612] Updated weights for policy 1, policy_version 16010 (0.0007) [2023-10-08 04:33:53,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 32702464. Throughput: 0: 1831.7, 1: 1852.2. Samples: 8179892. Policy #0 lag: (min: 6.0, avg: 28.4, max: 32.0) [2023-10-08 04:33:53,754][130385] Avg episode reward: [(0, '39.640'), (1, '35.810')] [2023-10-08 04:33:54,052][00612] Updated weights for policy 1, policy_version 16020 (0.0009) [2023-10-08 04:33:54,416][00612] Updated weights for policy 1, policy_version 16030 (0.0007) [2023-10-08 04:33:56,510][00611] Updated weights for policy 0, policy_version 15942 (0.0010) [2023-10-08 04:33:56,884][00611] Updated weights for policy 0, policy_version 15952 (0.0007) [2023-10-08 04:33:57,253][00611] Updated weights for policy 0, policy_version 15962 (0.0008) [2023-10-08 04:33:58,064][00612] Updated weights for policy 1, policy_version 16040 (0.0008) [2023-10-08 04:33:58,424][00612] Updated weights for policy 1, policy_version 16050 (0.0009) [2023-10-08 04:33:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 32768000. Throughput: 0: 1825.5, 1: 1851.4. Samples: 8201720. Policy #0 lag: (min: 6.0, avg: 28.4, max: 32.0) [2023-10-08 04:33:58,754][130385] Avg episode reward: [(0, '38.140'), (1, '39.010')] [2023-10-08 04:33:58,802][00612] Updated weights for policy 1, policy_version 16060 (0.0007) [2023-10-08 04:34:00,974][00611] Updated weights for policy 0, policy_version 15972 (0.0008) [2023-10-08 04:34:01,343][00611] Updated weights for policy 0, policy_version 15982 (0.0008) [2023-10-08 04:34:01,722][00611] Updated weights for policy 0, policy_version 15992 (0.0008) [2023-10-08 04:34:02,414][00612] Updated weights for policy 1, policy_version 16070 (0.0008) [2023-10-08 04:34:02,775][00612] Updated weights for policy 1, policy_version 16080 (0.0008) [2023-10-08 04:34:03,139][00612] Updated weights for policy 1, policy_version 16090 (0.0008) [2023-10-08 04:34:03,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 32866304. Throughput: 0: 1834.5, 1: 1835.7. Samples: 8223222. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:34:03,755][130385] Avg episode reward: [(0, '38.210'), (1, '40.110')] [2023-10-08 04:34:05,440][00611] Updated weights for policy 0, policy_version 16002 (0.0009) [2023-10-08 04:34:05,809][00611] Updated weights for policy 0, policy_version 16012 (0.0010) [2023-10-08 04:34:06,189][00611] Updated weights for policy 0, policy_version 16022 (0.0010) [2023-10-08 04:34:06,553][00611] Updated weights for policy 0, policy_version 16032 (0.0010) [2023-10-08 04:34:06,673][00612] Updated weights for policy 1, policy_version 16100 (0.0007) [2023-10-08 04:34:07,039][00612] Updated weights for policy 1, policy_version 16110 (0.0007) [2023-10-08 04:34:07,414][00612] Updated weights for policy 1, policy_version 16120 (0.0008) [2023-10-08 04:34:08,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 32931840. Throughput: 0: 1819.1, 1: 1854.2. Samples: 8234704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:34:08,755][130385] Avg episode reward: [(0, '37.520'), (1, '39.600')] [2023-10-08 04:34:10,284][00611] Updated weights for policy 0, policy_version 16042 (0.0007) [2023-10-08 04:34:10,658][00611] Updated weights for policy 0, policy_version 16052 (0.0007) [2023-10-08 04:34:11,035][00611] Updated weights for policy 0, policy_version 16062 (0.0007) [2023-10-08 04:34:11,048][00612] Updated weights for policy 1, policy_version 16130 (0.0008) [2023-10-08 04:34:11,425][00612] Updated weights for policy 1, policy_version 16140 (0.0008) [2023-10-08 04:34:11,788][00612] Updated weights for policy 1, policy_version 16150 (0.0008) [2023-10-08 04:34:12,160][00612] Updated weights for policy 1, policy_version 16160 (0.0008) [2023-10-08 04:34:13,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 32997376. Throughput: 0: 1826.9, 1: 1828.7. Samples: 8255628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:34:13,754][130385] Avg episode reward: [(0, '37.280'), (1, '39.230')] [2023-10-08 04:34:14,617][00611] Updated weights for policy 0, policy_version 16072 (0.0008) [2023-10-08 04:34:14,994][00611] Updated weights for policy 0, policy_version 16082 (0.0008) [2023-10-08 04:34:15,372][00611] Updated weights for policy 0, policy_version 16092 (0.0007) [2023-10-08 04:34:15,886][00612] Updated weights for policy 1, policy_version 16170 (0.0007) [2023-10-08 04:34:16,254][00612] Updated weights for policy 1, policy_version 16180 (0.0007) [2023-10-08 04:34:16,631][00612] Updated weights for policy 1, policy_version 16190 (0.0007) [2023-10-08 04:34:18,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33062912. Throughput: 0: 1822.5, 1: 1850.4. Samples: 8278746. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-08 04:34:18,754][130385] Avg episode reward: [(0, '39.830'), (1, '40.690')] [2023-10-08 04:34:18,762][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000016192_16580608.pth... [2023-10-08 04:34:18,811][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000014464_14811136.pth [2023-10-08 04:34:18,998][00611] Updated weights for policy 0, policy_version 16102 (0.0009) [2023-10-08 04:34:19,387][00611] Updated weights for policy 0, policy_version 16112 (0.0008) [2023-10-08 04:34:19,762][00611] Updated weights for policy 0, policy_version 16122 (0.0008) [2023-10-08 04:34:19,974][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000016128_16515072.pth... [2023-10-08 04:34:20,009][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000014400_14745600.pth [2023-10-08 04:34:20,171][00612] Updated weights for policy 1, policy_version 16200 (0.0009) [2023-10-08 04:34:20,546][00612] Updated weights for policy 1, policy_version 16210 (0.0007) [2023-10-08 04:34:20,911][00612] Updated weights for policy 1, policy_version 16220 (0.0007) [2023-10-08 04:34:23,350][00611] Updated weights for policy 0, policy_version 16132 (0.0008) [2023-10-08 04:34:23,724][00611] Updated weights for policy 0, policy_version 16142 (0.0009) [2023-10-08 04:34:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 33128448. Throughput: 0: 1825.6, 1: 1832.6. Samples: 8288736. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-08 04:34:23,755][130385] Avg episode reward: [(0, '40.380'), (1, '40.900')] [2023-10-08 04:34:24,097][00611] Updated weights for policy 0, policy_version 16152 (0.0009) [2023-10-08 04:34:24,526][00612] Updated weights for policy 1, policy_version 16230 (0.0007) [2023-10-08 04:34:24,892][00612] Updated weights for policy 1, policy_version 16240 (0.0010) [2023-10-08 04:34:25,268][00612] Updated weights for policy 1, policy_version 16250 (0.0009) [2023-10-08 04:34:27,634][00611] Updated weights for policy 0, policy_version 16162 (0.0009) [2023-10-08 04:34:28,010][00611] Updated weights for policy 0, policy_version 16172 (0.0009) [2023-10-08 04:34:28,380][00611] Updated weights for policy 0, policy_version 16182 (0.0009) [2023-10-08 04:34:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 33193984. Throughput: 0: 1829.6, 1: 1851.6. Samples: 8312042. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-08 04:34:28,754][00611] Updated weights for policy 0, policy_version 16192 (0.0007) [2023-10-08 04:34:28,755][130385] Avg episode reward: [(0, '41.140'), (1, '43.530')] [2023-10-08 04:34:28,902][00612] Updated weights for policy 1, policy_version 16260 (0.0010) [2023-10-08 04:34:29,269][00612] Updated weights for policy 1, policy_version 16270 (0.0009) [2023-10-08 04:34:29,638][00612] Updated weights for policy 1, policy_version 16280 (0.0008) [2023-10-08 04:34:29,934][00425] Saving new best policy, reward=43.530! [2023-10-08 04:34:32,535][00611] Updated weights for policy 0, policy_version 16202 (0.0010) [2023-10-08 04:34:32,918][00611] Updated weights for policy 0, policy_version 16212 (0.0008) [2023-10-08 04:34:33,288][00611] Updated weights for policy 0, policy_version 16222 (0.0008) [2023-10-08 04:34:33,289][00612] Updated weights for policy 1, policy_version 16290 (0.0009) [2023-10-08 04:34:33,652][00612] Updated weights for policy 1, policy_version 16300 (0.0008) [2023-10-08 04:34:33,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 33292288. Throughput: 0: 1823.6, 1: 1850.6. Samples: 8334106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:34:33,754][130385] Avg episode reward: [(0, '39.450'), (1, '40.430')] [2023-10-08 04:34:34,027][00612] Updated weights for policy 1, policy_version 16310 (0.0009) [2023-10-08 04:34:34,406][00612] Updated weights for policy 1, policy_version 16320 (0.0009) [2023-10-08 04:34:36,788][00611] Updated weights for policy 0, policy_version 16232 (0.0009) [2023-10-08 04:34:37,165][00611] Updated weights for policy 0, policy_version 16242 (0.0008) [2023-10-08 04:34:37,547][00611] Updated weights for policy 0, policy_version 16252 (0.0008) [2023-10-08 04:34:38,044][00612] Updated weights for policy 1, policy_version 16330 (0.0010) [2023-10-08 04:34:38,420][00612] Updated weights for policy 1, policy_version 16340 (0.0010) [2023-10-08 04:34:38,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 33357824. Throughput: 0: 1832.3, 1: 1850.9. Samples: 8345640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:34:38,755][130385] Avg episode reward: [(0, '36.540'), (1, '38.440')] [2023-10-08 04:34:38,795][00612] Updated weights for policy 1, policy_version 16350 (0.0009) [2023-10-08 04:34:41,158][00611] Updated weights for policy 0, policy_version 16262 (0.0008) [2023-10-08 04:34:41,528][00611] Updated weights for policy 0, policy_version 16272 (0.0010) [2023-10-08 04:34:41,900][00611] Updated weights for policy 0, policy_version 16282 (0.0008) [2023-10-08 04:34:42,448][00612] Updated weights for policy 1, policy_version 16360 (0.0009) [2023-10-08 04:34:42,823][00612] Updated weights for policy 1, policy_version 16370 (0.0009) [2023-10-08 04:34:43,193][00612] Updated weights for policy 1, policy_version 16380 (0.0008) [2023-10-08 04:34:43,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 33456128. Throughput: 0: 1826.0, 1: 1848.9. Samples: 8367092. Policy #0 lag: (min: 27.0, avg: 27.0, max: 31.0) [2023-10-08 04:34:43,754][130385] Avg episode reward: [(0, '37.760'), (1, '37.250')] [2023-10-08 04:34:45,648][00611] Updated weights for policy 0, policy_version 16292 (0.0008) [2023-10-08 04:34:46,022][00611] Updated weights for policy 0, policy_version 16302 (0.0009) [2023-10-08 04:34:46,391][00611] Updated weights for policy 0, policy_version 16312 (0.0008) [2023-10-08 04:34:46,826][00612] Updated weights for policy 1, policy_version 16390 (0.0009) [2023-10-08 04:34:47,195][00612] Updated weights for policy 1, policy_version 16400 (0.0009) [2023-10-08 04:34:47,563][00612] Updated weights for policy 1, policy_version 16410 (0.0009) [2023-10-08 04:34:48,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 33521664. Throughput: 0: 1832.7, 1: 1843.6. Samples: 8388654. Policy #0 lag: (min: 27.0, avg: 27.0, max: 31.0) [2023-10-08 04:34:48,755][130385] Avg episode reward: [(0, '39.350'), (1, '38.400')] [2023-10-08 04:34:50,078][00611] Updated weights for policy 0, policy_version 16322 (0.0008) [2023-10-08 04:34:50,451][00611] Updated weights for policy 0, policy_version 16332 (0.0011) [2023-10-08 04:34:50,826][00611] Updated weights for policy 0, policy_version 16342 (0.0010) [2023-10-08 04:34:51,192][00611] Updated weights for policy 0, policy_version 16352 (0.0010) [2023-10-08 04:34:51,246][00612] Updated weights for policy 1, policy_version 16420 (0.0009) [2023-10-08 04:34:51,618][00612] Updated weights for policy 1, policy_version 16430 (0.0007) [2023-10-08 04:34:51,979][00612] Updated weights for policy 1, policy_version 16440 (0.0008) [2023-10-08 04:34:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 33587200. Throughput: 0: 1829.3, 1: 1847.0. Samples: 8400138. Policy #0 lag: (min: 27.0, avg: 27.0, max: 31.0) [2023-10-08 04:34:53,754][130385] Avg episode reward: [(0, '38.740'), (1, '38.970')] [2023-10-08 04:34:54,734][00611] Updated weights for policy 0, policy_version 16362 (0.0008) [2023-10-08 04:34:55,115][00611] Updated weights for policy 0, policy_version 16372 (0.0008) [2023-10-08 04:34:55,475][00611] Updated weights for policy 0, policy_version 16382 (0.0008) [2023-10-08 04:34:55,688][00612] Updated weights for policy 1, policy_version 16450 (0.0007) [2023-10-08 04:34:56,058][00612] Updated weights for policy 1, policy_version 16460 (0.0010) [2023-10-08 04:34:56,431][00612] Updated weights for policy 1, policy_version 16470 (0.0009) [2023-10-08 04:34:56,798][00612] Updated weights for policy 1, policy_version 16480 (0.0007) [2023-10-08 04:34:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 33652736. Throughput: 0: 1844.6, 1: 1849.8. Samples: 8421878. Policy #0 lag: (min: 24.0, avg: 39.4, max: 40.0) [2023-10-08 04:34:58,755][130385] Avg episode reward: [(0, '37.720'), (1, '38.760')] [2023-10-08 04:34:59,104][00611] Updated weights for policy 0, policy_version 16392 (0.0008) [2023-10-08 04:34:59,468][00611] Updated weights for policy 0, policy_version 16402 (0.0007) [2023-10-08 04:34:59,839][00611] Updated weights for policy 0, policy_version 16412 (0.0007) [2023-10-08 04:35:00,351][00612] Updated weights for policy 1, policy_version 16490 (0.0008) [2023-10-08 04:35:00,711][00612] Updated weights for policy 1, policy_version 16500 (0.0010) [2023-10-08 04:35:01,077][00612] Updated weights for policy 1, policy_version 16510 (0.0010) [2023-10-08 04:35:03,677][00611] Updated weights for policy 0, policy_version 16422 (0.0008) [2023-10-08 04:35:03,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 33718272. Throughput: 0: 1838.5, 1: 1851.6. Samples: 8444802. Policy #0 lag: (min: 24.0, avg: 39.4, max: 40.0) [2023-10-08 04:35:03,755][130385] Avg episode reward: [(0, '36.820'), (1, '37.900')] [2023-10-08 04:35:04,059][00611] Updated weights for policy 0, policy_version 16432 (0.0011) [2023-10-08 04:35:04,426][00611] Updated weights for policy 0, policy_version 16442 (0.0009) [2023-10-08 04:35:04,764][00612] Updated weights for policy 1, policy_version 16520 (0.0010) [2023-10-08 04:35:05,129][00612] Updated weights for policy 1, policy_version 16530 (0.0010) [2023-10-08 04:35:05,501][00612] Updated weights for policy 1, policy_version 16540 (0.0010) [2023-10-08 04:35:08,087][00611] Updated weights for policy 0, policy_version 16452 (0.0008) [2023-10-08 04:35:08,466][00611] Updated weights for policy 0, policy_version 16462 (0.0009) [2023-10-08 04:35:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 33783808. Throughput: 0: 1837.3, 1: 1850.4. Samples: 8454684. Policy #0 lag: (min: 24.0, avg: 39.4, max: 40.0) [2023-10-08 04:35:08,755][130385] Avg episode reward: [(0, '37.750'), (1, '37.530')] [2023-10-08 04:35:08,841][00611] Updated weights for policy 0, policy_version 16472 (0.0008) [2023-10-08 04:35:09,242][00612] Updated weights for policy 1, policy_version 16550 (0.0007) [2023-10-08 04:35:09,605][00612] Updated weights for policy 1, policy_version 16560 (0.0008) [2023-10-08 04:35:09,969][00612] Updated weights for policy 1, policy_version 16570 (0.0007) [2023-10-08 04:35:12,478][00611] Updated weights for policy 0, policy_version 16482 (0.0008) [2023-10-08 04:35:12,854][00611] Updated weights for policy 0, policy_version 16492 (0.0008) [2023-10-08 04:35:13,232][00611] Updated weights for policy 0, policy_version 16502 (0.0007) [2023-10-08 04:35:13,521][00612] Updated weights for policy 1, policy_version 16580 (0.0007) [2023-10-08 04:35:13,608][00611] Updated weights for policy 0, policy_version 16512 (0.0007) [2023-10-08 04:35:13,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 33882112. Throughput: 0: 1833.2, 1: 1846.2. Samples: 8477616. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-10-08 04:35:13,754][130385] Avg episode reward: [(0, '37.070'), (1, '35.490')] [2023-10-08 04:35:13,882][00612] Updated weights for policy 1, policy_version 16590 (0.0008) [2023-10-08 04:35:14,244][00612] Updated weights for policy 1, policy_version 16600 (0.0010) [2023-10-08 04:35:17,283][00611] Updated weights for policy 0, policy_version 16522 (0.0008) [2023-10-08 04:35:17,652][00611] Updated weights for policy 0, policy_version 16532 (0.0007) [2023-10-08 04:35:17,948][00612] Updated weights for policy 1, policy_version 16610 (0.0007) [2023-10-08 04:35:18,022][00611] Updated weights for policy 0, policy_version 16542 (0.0007) [2023-10-08 04:35:18,324][00612] Updated weights for policy 1, policy_version 16620 (0.0009) [2023-10-08 04:35:18,691][00612] Updated weights for policy 1, policy_version 16630 (0.0007) [2023-10-08 04:35:18,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 33947648. Throughput: 0: 1825.6, 1: 1834.7. Samples: 8498818. Policy #0 lag: (min: 6.0, avg: 10.0, max: 38.0) [2023-10-08 04:35:18,755][130385] Avg episode reward: [(0, '37.660'), (1, '34.690')] [2023-10-08 04:35:19,058][00612] Updated weights for policy 1, policy_version 16640 (0.0009) [2023-10-08 04:35:21,727][00611] Updated weights for policy 0, policy_version 16552 (0.0009) [2023-10-08 04:35:22,100][00611] Updated weights for policy 0, policy_version 16562 (0.0010) [2023-10-08 04:35:22,469][00611] Updated weights for policy 0, policy_version 16572 (0.0008) [2023-10-08 04:35:22,665][00612] Updated weights for policy 1, policy_version 16650 (0.0008) [2023-10-08 04:35:23,035][00612] Updated weights for policy 1, policy_version 16660 (0.0009) [2023-10-08 04:35:23,405][00612] Updated weights for policy 1, policy_version 16670 (0.0010) [2023-10-08 04:35:23,754][130385] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 34045952. Throughput: 0: 1823.6, 1: 1842.5. Samples: 8510616. Policy #0 lag: (min: 9.0, avg: 20.0, max: 41.0) [2023-10-08 04:35:23,755][130385] Avg episode reward: [(0, '34.590'), (1, '31.410')] [2023-10-08 04:35:26,113][00611] Updated weights for policy 0, policy_version 16582 (0.0009) [2023-10-08 04:35:26,488][00611] Updated weights for policy 0, policy_version 16592 (0.0009) [2023-10-08 04:35:26,862][00611] Updated weights for policy 0, policy_version 16602 (0.0008) [2023-10-08 04:35:27,155][00612] Updated weights for policy 1, policy_version 16680 (0.0009) [2023-10-08 04:35:27,538][00612] Updated weights for policy 1, policy_version 16690 (0.0007) [2023-10-08 04:35:27,902][00612] Updated weights for policy 1, policy_version 16700 (0.0008) [2023-10-08 04:35:28,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 34111488. Throughput: 0: 1822.6, 1: 1829.9. Samples: 8531456. Policy #0 lag: (min: 9.0, avg: 20.0, max: 41.0) [2023-10-08 04:35:28,755][130385] Avg episode reward: [(0, '34.950'), (1, '30.600')] [2023-10-08 04:35:30,509][00611] Updated weights for policy 0, policy_version 16612 (0.0007) [2023-10-08 04:35:30,878][00611] Updated weights for policy 0, policy_version 16622 (0.0009) [2023-10-08 04:35:31,258][00611] Updated weights for policy 0, policy_version 16632 (0.0009) [2023-10-08 04:35:31,640][00612] Updated weights for policy 1, policy_version 16710 (0.0009) [2023-10-08 04:35:32,007][00612] Updated weights for policy 1, policy_version 16720 (0.0008) [2023-10-08 04:35:32,382][00612] Updated weights for policy 1, policy_version 16730 (0.0008) [2023-10-08 04:35:33,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 34177024. Throughput: 0: 1832.2, 1: 1829.1. Samples: 8553410. Policy #0 lag: (min: 9.0, avg: 20.0, max: 41.0) [2023-10-08 04:35:33,754][130385] Avg episode reward: [(0, '36.510'), (1, '31.000')] [2023-10-08 04:35:34,874][00611] Updated weights for policy 0, policy_version 16642 (0.0008) [2023-10-08 04:35:35,254][00611] Updated weights for policy 0, policy_version 16652 (0.0007) [2023-10-08 04:35:35,624][00611] Updated weights for policy 0, policy_version 16662 (0.0009) [2023-10-08 04:35:35,834][00612] Updated weights for policy 1, policy_version 16740 (0.0009) [2023-10-08 04:35:35,992][00611] Updated weights for policy 0, policy_version 16672 (0.0009) [2023-10-08 04:35:36,214][00612] Updated weights for policy 1, policy_version 16750 (0.0010) [2023-10-08 04:35:36,582][00612] Updated weights for policy 1, policy_version 16760 (0.0007) [2023-10-08 04:35:38,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 34242560. Throughput: 0: 1826.0, 1: 1825.2. Samples: 8564442. Policy #0 lag: (min: 23.0, avg: 25.4, max: 55.0) [2023-10-08 04:35:38,755][130385] Avg episode reward: [(0, '32.750'), (1, '31.740')] [2023-10-08 04:35:39,706][00611] Updated weights for policy 0, policy_version 16682 (0.0010) [2023-10-08 04:35:40,076][00611] Updated weights for policy 0, policy_version 16692 (0.0010) [2023-10-08 04:35:40,297][00612] Updated weights for policy 1, policy_version 16770 (0.0009) [2023-10-08 04:35:40,444][00611] Updated weights for policy 0, policy_version 16702 (0.0007) [2023-10-08 04:35:40,655][00612] Updated weights for policy 1, policy_version 16780 (0.0011) [2023-10-08 04:35:41,024][00612] Updated weights for policy 1, policy_version 16790 (0.0008) [2023-10-08 04:35:41,390][00612] Updated weights for policy 1, policy_version 16800 (0.0007) [2023-10-08 04:35:43,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 34308096. Throughput: 0: 1823.6, 1: 1835.1. Samples: 8586520. Policy #0 lag: (min: 23.0, avg: 25.4, max: 55.0) [2023-10-08 04:35:43,754][130385] Avg episode reward: [(0, '35.280'), (1, '33.380')] [2023-10-08 04:35:44,210][00611] Updated weights for policy 0, policy_version 16712 (0.0008) [2023-10-08 04:35:44,589][00611] Updated weights for policy 0, policy_version 16722 (0.0009) [2023-10-08 04:35:44,946][00612] Updated weights for policy 1, policy_version 16810 (0.0008) [2023-10-08 04:35:44,959][00611] Updated weights for policy 0, policy_version 16732 (0.0007) [2023-10-08 04:35:45,315][00612] Updated weights for policy 1, policy_version 16820 (0.0009) [2023-10-08 04:35:45,691][00612] Updated weights for policy 1, policy_version 16830 (0.0010) [2023-10-08 04:35:48,569][00611] Updated weights for policy 0, policy_version 16742 (0.0008) [2023-10-08 04:35:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 34373632. Throughput: 0: 1826.5, 1: 1835.8. Samples: 8609606. Policy #0 lag: (min: 23.0, avg: 25.4, max: 55.0) [2023-10-08 04:35:48,755][130385] Avg episode reward: [(0, '34.940'), (1, '34.590')] [2023-10-08 04:35:48,934][00611] Updated weights for policy 0, policy_version 16752 (0.0008) [2023-10-08 04:35:49,302][00611] Updated weights for policy 0, policy_version 16762 (0.0008) [2023-10-08 04:35:49,345][00612] Updated weights for policy 1, policy_version 16840 (0.0007) [2023-10-08 04:35:49,707][00612] Updated weights for policy 1, policy_version 16850 (0.0009) [2023-10-08 04:35:50,079][00612] Updated weights for policy 1, policy_version 16860 (0.0009) [2023-10-08 04:35:52,997][00611] Updated weights for policy 0, policy_version 16772 (0.0009) [2023-10-08 04:35:53,366][00611] Updated weights for policy 0, policy_version 16782 (0.0009) [2023-10-08 04:35:53,632][00612] Updated weights for policy 1, policy_version 16870 (0.0008) [2023-10-08 04:35:53,738][00611] Updated weights for policy 0, policy_version 16792 (0.0008) [2023-10-08 04:35:53,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 34439168. Throughput: 0: 1829.0, 1: 1835.7. Samples: 8619596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:35:53,755][130385] Avg episode reward: [(0, '37.160'), (1, '34.870')] [2023-10-08 04:35:54,001][00612] Updated weights for policy 1, policy_version 16880 (0.0008) [2023-10-08 04:35:54,366][00612] Updated weights for policy 1, policy_version 16890 (0.0009) [2023-10-08 04:35:57,534][00611] Updated weights for policy 0, policy_version 16802 (0.0009) [2023-10-08 04:35:57,907][00612] Updated weights for policy 1, policy_version 16900 (0.0010) [2023-10-08 04:35:57,918][00611] Updated weights for policy 0, policy_version 16812 (0.0008) [2023-10-08 04:35:58,276][00612] Updated weights for policy 1, policy_version 16910 (0.0009) [2023-10-08 04:35:58,292][00611] Updated weights for policy 0, policy_version 16822 (0.0008) [2023-10-08 04:35:58,651][00612] Updated weights for policy 1, policy_version 16920 (0.0007) [2023-10-08 04:35:58,659][00611] Updated weights for policy 0, policy_version 16832 (0.0008) [2023-10-08 04:35:58,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 34537472. Throughput: 0: 1824.0, 1: 1847.0. Samples: 8642810. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:35:58,754][130385] Avg episode reward: [(0, '37.000'), (1, '36.210')] [2023-10-08 04:36:02,453][00611] Updated weights for policy 0, policy_version 16842 (0.0009) [2023-10-08 04:36:02,538][00612] Updated weights for policy 1, policy_version 16930 (0.0010) [2023-10-08 04:36:02,823][00611] Updated weights for policy 0, policy_version 16852 (0.0007) [2023-10-08 04:36:02,910][00612] Updated weights for policy 1, policy_version 16940 (0.0008) [2023-10-08 04:36:03,198][00611] Updated weights for policy 0, policy_version 16862 (0.0007) [2023-10-08 04:36:03,291][00612] Updated weights for policy 1, policy_version 16950 (0.0010) [2023-10-08 04:36:03,655][00612] Updated weights for policy 1, policy_version 16960 (0.0012) [2023-10-08 04:36:03,754][130385] Fps is (10 sec: 19661.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 34635776. Throughput: 0: 1821.7, 1: 1829.9. Samples: 8663142. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-08 04:36:03,754][130385] Avg episode reward: [(0, '39.890'), (1, '38.490')] [2023-10-08 04:36:06,868][00611] Updated weights for policy 0, policy_version 16872 (0.0008) [2023-10-08 04:36:07,163][00612] Updated weights for policy 1, policy_version 16970 (0.0009) [2023-10-08 04:36:07,243][00611] Updated weights for policy 0, policy_version 16882 (0.0008) [2023-10-08 04:36:07,537][00612] Updated weights for policy 1, policy_version 16980 (0.0009) [2023-10-08 04:36:07,609][00611] Updated weights for policy 0, policy_version 16892 (0.0009) [2023-10-08 04:36:07,894][00612] Updated weights for policy 1, policy_version 16990 (0.0009) [2023-10-08 04:36:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 34701312. Throughput: 0: 1818.0, 1: 1840.9. Samples: 8675264. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-08 04:36:08,754][130385] Avg episode reward: [(0, '36.990'), (1, '36.030')] [2023-10-08 04:36:11,155][00611] Updated weights for policy 0, policy_version 16902 (0.0007) [2023-10-08 04:36:11,522][00611] Updated weights for policy 0, policy_version 16912 (0.0009) [2023-10-08 04:36:11,660][00612] Updated weights for policy 1, policy_version 17000 (0.0008) [2023-10-08 04:36:11,899][00611] Updated weights for policy 0, policy_version 16922 (0.0008) [2023-10-08 04:36:12,032][00612] Updated weights for policy 1, policy_version 17010 (0.0010) [2023-10-08 04:36:12,393][00612] Updated weights for policy 1, policy_version 17020 (0.0010) [2023-10-08 04:36:13,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 34766848. Throughput: 0: 1822.1, 1: 1825.2. Samples: 8695582. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-08 04:36:13,755][130385] Avg episode reward: [(0, '38.070'), (1, '36.650')] [2023-10-08 04:36:15,466][00611] Updated weights for policy 0, policy_version 16932 (0.0009) [2023-10-08 04:36:15,836][00611] Updated weights for policy 0, policy_version 16942 (0.0009) [2023-10-08 04:36:16,209][00611] Updated weights for policy 0, policy_version 16952 (0.0008) [2023-10-08 04:36:16,212][00612] Updated weights for policy 1, policy_version 17030 (0.0007) [2023-10-08 04:36:16,582][00612] Updated weights for policy 1, policy_version 17040 (0.0007) [2023-10-08 04:36:16,949][00612] Updated weights for policy 1, policy_version 17050 (0.0008) [2023-10-08 04:36:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 34832384. Throughput: 0: 1817.0, 1: 1840.6. Samples: 8718002. Policy #0 lag: (min: 12.0, avg: 15.3, max: 44.0) [2023-10-08 04:36:18,755][130385] Avg episode reward: [(0, '36.710'), (1, '36.140')] [2023-10-08 04:36:18,768][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000017056_17465344.pth... [2023-10-08 04:36:18,768][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000016960_17367040.pth... [2023-10-08 04:36:18,806][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000015328_15695872.pth [2023-10-08 04:36:18,807][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000015264_15630336.pth [2023-10-08 04:36:18,810][00425] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p1/milestones/checkpoint_000017056_17465344.pth [2023-10-08 04:36:18,812][00365] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p0/milestones/checkpoint_000016960_17367040.pth [2023-10-08 04:36:19,932][00611] Updated weights for policy 0, policy_version 16962 (0.0009) [2023-10-08 04:36:20,309][00611] Updated weights for policy 0, policy_version 16972 (0.0009) [2023-10-08 04:36:20,639][00612] Updated weights for policy 1, policy_version 17060 (0.0009) [2023-10-08 04:36:20,678][00611] Updated weights for policy 0, policy_version 16982 (0.0008) [2023-10-08 04:36:21,004][00612] Updated weights for policy 1, policy_version 17070 (0.0008) [2023-10-08 04:36:21,040][00611] Updated weights for policy 0, policy_version 16992 (0.0009) [2023-10-08 04:36:21,384][00612] Updated weights for policy 1, policy_version 17080 (0.0010) [2023-10-08 04:36:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 34897920. Throughput: 0: 1820.7, 1: 1831.0. Samples: 8728766. Policy #0 lag: (min: 12.0, avg: 15.3, max: 44.0) [2023-10-08 04:36:23,754][130385] Avg episode reward: [(0, '35.050'), (1, '35.260')] [2023-10-08 04:36:24,640][00611] Updated weights for policy 0, policy_version 17002 (0.0008) [2023-10-08 04:36:25,008][00611] Updated weights for policy 0, policy_version 17012 (0.0010) [2023-10-08 04:36:25,040][00612] Updated weights for policy 1, policy_version 17090 (0.0010) [2023-10-08 04:36:25,390][00611] Updated weights for policy 0, policy_version 17022 (0.0007) [2023-10-08 04:36:25,409][00612] Updated weights for policy 1, policy_version 17100 (0.0008) [2023-10-08 04:36:25,784][00612] Updated weights for policy 1, policy_version 17110 (0.0010) [2023-10-08 04:36:26,148][00612] Updated weights for policy 1, policy_version 17120 (0.0007) [2023-10-08 04:36:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 34963456. Throughput: 0: 1824.3, 1: 1836.8. Samples: 8751268. Policy #0 lag: (min: 12.0, avg: 15.3, max: 44.0) [2023-10-08 04:36:28,755][130385] Avg episode reward: [(0, '36.970'), (1, '32.950')] [2023-10-08 04:36:29,129][00611] Updated weights for policy 0, policy_version 17032 (0.0010) [2023-10-08 04:36:29,505][00611] Updated weights for policy 0, policy_version 17042 (0.0009) [2023-10-08 04:36:29,783][00612] Updated weights for policy 1, policy_version 17130 (0.0007) [2023-10-08 04:36:29,881][00611] Updated weights for policy 0, policy_version 17052 (0.0007) [2023-10-08 04:36:30,144][00612] Updated weights for policy 1, policy_version 17140 (0.0007) [2023-10-08 04:36:30,510][00612] Updated weights for policy 1, policy_version 17150 (0.0008) [2023-10-08 04:36:33,669][00611] Updated weights for policy 0, policy_version 17062 (0.0009) [2023-10-08 04:36:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 35028992. Throughput: 0: 1822.2, 1: 1832.4. Samples: 8774066. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 04:36:33,755][130385] Avg episode reward: [(0, '33.540'), (1, '33.240')] [2023-10-08 04:36:34,030][00611] Updated weights for policy 0, policy_version 17072 (0.0008) [2023-10-08 04:36:34,146][00612] Updated weights for policy 1, policy_version 17160 (0.0008) [2023-10-08 04:36:34,403][00611] Updated weights for policy 0, policy_version 17082 (0.0007) [2023-10-08 04:36:34,515][00612] Updated weights for policy 1, policy_version 17170 (0.0008) [2023-10-08 04:36:34,879][00612] Updated weights for policy 1, policy_version 17180 (0.0007) [2023-10-08 04:36:37,950][00611] Updated weights for policy 0, policy_version 17092 (0.0008) [2023-10-08 04:36:38,333][00611] Updated weights for policy 0, policy_version 17102 (0.0011) [2023-10-08 04:36:38,555][00612] Updated weights for policy 1, policy_version 17190 (0.0008) [2023-10-08 04:36:38,704][00611] Updated weights for policy 0, policy_version 17112 (0.0009) [2023-10-08 04:36:38,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 35094528. Throughput: 0: 1824.1, 1: 1832.1. Samples: 8784124. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 04:36:38,754][130385] Avg episode reward: [(0, '33.170'), (1, '32.460')] [2023-10-08 04:36:38,930][00612] Updated weights for policy 1, policy_version 17200 (0.0008) [2023-10-08 04:36:39,296][00612] Updated weights for policy 1, policy_version 17210 (0.0008) [2023-10-08 04:36:42,410][00611] Updated weights for policy 0, policy_version 17122 (0.0008) [2023-10-08 04:36:42,811][00611] Updated weights for policy 0, policy_version 17132 (0.0009) [2023-10-08 04:36:43,040][00612] Updated weights for policy 1, policy_version 17220 (0.0010) [2023-10-08 04:36:43,175][00611] Updated weights for policy 0, policy_version 17142 (0.0009) [2023-10-08 04:36:43,416][00612] Updated weights for policy 1, policy_version 17230 (0.0008) [2023-10-08 04:36:43,553][00611] Updated weights for policy 0, policy_version 17152 (0.0009) [2023-10-08 04:36:43,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 35192832. Throughput: 0: 1828.7, 1: 1820.3. Samples: 8807020. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:36:43,755][130385] Avg episode reward: [(0, '35.040'), (1, '36.100')] [2023-10-08 04:36:43,778][00612] Updated weights for policy 1, policy_version 17240 (0.0011) [2023-10-08 04:36:47,216][00611] Updated weights for policy 0, policy_version 17162 (0.0007) [2023-10-08 04:36:47,266][00612] Updated weights for policy 1, policy_version 17250 (0.0009) [2023-10-08 04:36:47,577][00611] Updated weights for policy 0, policy_version 17172 (0.0008) [2023-10-08 04:36:47,639][00612] Updated weights for policy 1, policy_version 17260 (0.0007) [2023-10-08 04:36:47,944][00611] Updated weights for policy 0, policy_version 17182 (0.0008) [2023-10-08 04:36:48,003][00612] Updated weights for policy 1, policy_version 17270 (0.0007) [2023-10-08 04:36:48,369][00612] Updated weights for policy 1, policy_version 17280 (0.0010) [2023-10-08 04:36:48,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 35291136. Throughput: 0: 1828.4, 1: 1823.0. Samples: 8827454. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:36:48,754][130385] Avg episode reward: [(0, '37.650'), (1, '36.930')] [2023-10-08 04:36:51,662][00611] Updated weights for policy 0, policy_version 17192 (0.0007) [2023-10-08 04:36:52,005][00612] Updated weights for policy 1, policy_version 17290 (0.0009) [2023-10-08 04:36:52,044][00611] Updated weights for policy 0, policy_version 17202 (0.0007) [2023-10-08 04:36:52,377][00612] Updated weights for policy 1, policy_version 17300 (0.0009) [2023-10-08 04:36:52,420][00611] Updated weights for policy 0, policy_version 17212 (0.0010) [2023-10-08 04:36:52,732][00612] Updated weights for policy 1, policy_version 17310 (0.0011) [2023-10-08 04:36:53,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 35356672. Throughput: 0: 1834.2, 1: 1827.1. Samples: 8840024. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:36:53,754][130385] Avg episode reward: [(0, '39.520'), (1, '36.960')] [2023-10-08 04:36:56,059][00611] Updated weights for policy 0, policy_version 17222 (0.0009) [2023-10-08 04:36:56,430][00611] Updated weights for policy 0, policy_version 17232 (0.0008) [2023-10-08 04:36:56,529][00612] Updated weights for policy 1, policy_version 17320 (0.0009) [2023-10-08 04:36:56,798][00611] Updated weights for policy 0, policy_version 17242 (0.0009) [2023-10-08 04:36:56,893][00612] Updated weights for policy 1, policy_version 17330 (0.0007) [2023-10-08 04:36:57,265][00612] Updated weights for policy 1, policy_version 17340 (0.0008) [2023-10-08 04:36:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 35422208. Throughput: 0: 1830.6, 1: 1823.2. Samples: 8860002. Policy #0 lag: (min: 26.0, avg: 31.8, max: 58.0) [2023-10-08 04:36:58,755][130385] Avg episode reward: [(0, '39.930'), (1, '37.640')] [2023-10-08 04:37:00,536][00611] Updated weights for policy 0, policy_version 17252 (0.0008) [2023-10-08 04:37:00,900][00611] Updated weights for policy 0, policy_version 17262 (0.0008) [2023-10-08 04:37:01,065][00612] Updated weights for policy 1, policy_version 17350 (0.0008) [2023-10-08 04:37:01,274][00611] Updated weights for policy 0, policy_version 17272 (0.0008) [2023-10-08 04:37:01,447][00612] Updated weights for policy 1, policy_version 17360 (0.0007) [2023-10-08 04:37:01,812][00612] Updated weights for policy 1, policy_version 17370 (0.0011) [2023-10-08 04:37:03,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 35487744. Throughput: 0: 1828.1, 1: 1827.4. Samples: 8882498. Policy #0 lag: (min: 26.0, avg: 31.8, max: 58.0) [2023-10-08 04:37:03,755][130385] Avg episode reward: [(0, '37.630'), (1, '39.720')] [2023-10-08 04:37:04,790][00611] Updated weights for policy 0, policy_version 17282 (0.0008) [2023-10-08 04:37:05,168][00611] Updated weights for policy 0, policy_version 17292 (0.0009) [2023-10-08 04:37:05,325][00612] Updated weights for policy 1, policy_version 17380 (0.0009) [2023-10-08 04:37:05,542][00611] Updated weights for policy 0, policy_version 17302 (0.0008) [2023-10-08 04:37:05,687][00612] Updated weights for policy 1, policy_version 17390 (0.0008) [2023-10-08 04:37:05,912][00611] Updated weights for policy 0, policy_version 17312 (0.0009) [2023-10-08 04:37:06,058][00612] Updated weights for policy 1, policy_version 17400 (0.0007) [2023-10-08 04:37:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 35553280. Throughput: 0: 1824.7, 1: 1817.2. Samples: 8892648. Policy #0 lag: (min: 26.0, avg: 31.8, max: 58.0) [2023-10-08 04:37:08,754][130385] Avg episode reward: [(0, '37.550'), (1, '39.190')] [2023-10-08 04:37:09,620][00611] Updated weights for policy 0, policy_version 17322 (0.0010) [2023-10-08 04:37:09,895][00612] Updated weights for policy 1, policy_version 17410 (0.0008) [2023-10-08 04:37:09,991][00611] Updated weights for policy 0, policy_version 17332 (0.0009) [2023-10-08 04:37:10,264][00612] Updated weights for policy 1, policy_version 17420 (0.0009) [2023-10-08 04:37:10,356][00611] Updated weights for policy 0, policy_version 17342 (0.0007) [2023-10-08 04:37:10,643][00612] Updated weights for policy 1, policy_version 17430 (0.0008) [2023-10-08 04:37:11,013][00612] Updated weights for policy 1, policy_version 17440 (0.0008) [2023-10-08 04:37:13,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 35618816. Throughput: 0: 1821.6, 1: 1819.9. Samples: 8915136. Policy #0 lag: (min: 15.0, avg: 15.7, max: 33.0) [2023-10-08 04:37:13,754][130385] Avg episode reward: [(0, '38.520'), (1, '40.560')] [2023-10-08 04:37:14,028][00611] Updated weights for policy 0, policy_version 17352 (0.0010) [2023-10-08 04:37:14,406][00611] Updated weights for policy 0, policy_version 17362 (0.0007) [2023-10-08 04:37:14,634][00612] Updated weights for policy 1, policy_version 17450 (0.0008) [2023-10-08 04:37:14,778][00611] Updated weights for policy 0, policy_version 17372 (0.0008) [2023-10-08 04:37:14,999][00612] Updated weights for policy 1, policy_version 17460 (0.0007) [2023-10-08 04:37:15,367][00612] Updated weights for policy 1, policy_version 17470 (0.0010) [2023-10-08 04:37:18,582][00611] Updated weights for policy 0, policy_version 17382 (0.0008) [2023-10-08 04:37:18,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 35684352. Throughput: 0: 1823.3, 1: 1819.2. Samples: 8937980. Policy #0 lag: (min: 15.0, avg: 15.7, max: 33.0) [2023-10-08 04:37:18,755][130385] Avg episode reward: [(0, '38.480'), (1, '40.100')] [2023-10-08 04:37:18,955][00611] Updated weights for policy 0, policy_version 17392 (0.0008) [2023-10-08 04:37:19,036][00612] Updated weights for policy 1, policy_version 17480 (0.0008) [2023-10-08 04:37:19,326][00611] Updated weights for policy 0, policy_version 17402 (0.0009) [2023-10-08 04:37:19,403][00612] Updated weights for policy 1, policy_version 17490 (0.0008) [2023-10-08 04:37:19,775][00612] Updated weights for policy 1, policy_version 17500 (0.0008) [2023-10-08 04:37:22,968][00611] Updated weights for policy 0, policy_version 17412 (0.0008) [2023-10-08 04:37:23,278][00612] Updated weights for policy 1, policy_version 17510 (0.0007) [2023-10-08 04:37:23,343][00611] Updated weights for policy 0, policy_version 17422 (0.0007) [2023-10-08 04:37:23,645][00612] Updated weights for policy 1, policy_version 17520 (0.0007) [2023-10-08 04:37:23,711][00611] Updated weights for policy 0, policy_version 17432 (0.0007) [2023-10-08 04:37:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 35749888. Throughput: 0: 1818.8, 1: 1825.4. Samples: 8948114. Policy #0 lag: (min: 15.0, avg: 15.7, max: 33.0) [2023-10-08 04:37:23,755][130385] Avg episode reward: [(0, '37.060'), (1, '37.860')] [2023-10-08 04:37:24,015][00612] Updated weights for policy 1, policy_version 17530 (0.0008) [2023-10-08 04:37:27,420][00611] Updated weights for policy 0, policy_version 17442 (0.0007) [2023-10-08 04:37:27,742][00612] Updated weights for policy 1, policy_version 17540 (0.0007) [2023-10-08 04:37:27,820][00611] Updated weights for policy 0, policy_version 17452 (0.0008) [2023-10-08 04:37:28,108][00612] Updated weights for policy 1, policy_version 17550 (0.0008) [2023-10-08 04:37:28,191][00611] Updated weights for policy 0, policy_version 17462 (0.0008) [2023-10-08 04:37:28,475][00612] Updated weights for policy 1, policy_version 17560 (0.0008) [2023-10-08 04:37:28,551][00611] Updated weights for policy 0, policy_version 17472 (0.0008) [2023-10-08 04:37:28,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 35848192. Throughput: 0: 1818.0, 1: 1825.2. Samples: 8970966. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-08 04:37:28,755][130385] Avg episode reward: [(0, '36.780'), (1, '36.020')] [2023-10-08 04:37:32,044][00612] Updated weights for policy 1, policy_version 17570 (0.0008) [2023-10-08 04:37:32,174][00611] Updated weights for policy 0, policy_version 17482 (0.0009) [2023-10-08 04:37:32,414][00612] Updated weights for policy 1, policy_version 17580 (0.0008) [2023-10-08 04:37:32,543][00611] Updated weights for policy 0, policy_version 17492 (0.0008) [2023-10-08 04:37:32,784][00612] Updated weights for policy 1, policy_version 17590 (0.0007) [2023-10-08 04:37:32,910][00611] Updated weights for policy 0, policy_version 17502 (0.0008) [2023-10-08 04:37:33,150][00612] Updated weights for policy 1, policy_version 17600 (0.0007) [2023-10-08 04:37:33,754][130385] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 35946496. Throughput: 0: 1814.4, 1: 1815.6. Samples: 8990804. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-08 04:37:33,754][130385] Avg episode reward: [(0, '37.030'), (1, '36.830')] [2023-10-08 04:37:36,796][00611] Updated weights for policy 0, policy_version 17512 (0.0010) [2023-10-08 04:37:37,000][00612] Updated weights for policy 1, policy_version 17610 (0.0008) [2023-10-08 04:37:37,167][00611] Updated weights for policy 0, policy_version 17522 (0.0008) [2023-10-08 04:37:37,363][00612] Updated weights for policy 1, policy_version 17620 (0.0008) [2023-10-08 04:37:37,537][00611] Updated weights for policy 0, policy_version 17532 (0.0009) [2023-10-08 04:37:37,727][00612] Updated weights for policy 1, policy_version 17630 (0.0007) [2023-10-08 04:37:38,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 36012032. Throughput: 0: 1810.4, 1: 1824.9. Samples: 9003616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:37:38,754][130385] Avg episode reward: [(0, '37.420'), (1, '35.320')] [2023-10-08 04:37:41,274][00611] Updated weights for policy 0, policy_version 17542 (0.0008) [2023-10-08 04:37:41,652][00611] Updated weights for policy 0, policy_version 17552 (0.0008) [2023-10-08 04:37:41,667][00612] Updated weights for policy 1, policy_version 17640 (0.0008) [2023-10-08 04:37:42,030][00611] Updated weights for policy 0, policy_version 17562 (0.0008) [2023-10-08 04:37:42,031][00612] Updated weights for policy 1, policy_version 17650 (0.0007) [2023-10-08 04:37:42,404][00612] Updated weights for policy 1, policy_version 17660 (0.0008) [2023-10-08 04:37:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 36077568. Throughput: 0: 1814.2, 1: 1828.6. Samples: 9023928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:37:43,754][130385] Avg episode reward: [(0, '38.660'), (1, '36.830')] [2023-10-08 04:37:45,687][00611] Updated weights for policy 0, policy_version 17572 (0.0008) [2023-10-08 04:37:46,045][00611] Updated weights for policy 0, policy_version 17582 (0.0007) [2023-10-08 04:37:46,063][00612] Updated weights for policy 1, policy_version 17670 (0.0010) [2023-10-08 04:37:46,418][00611] Updated weights for policy 0, policy_version 17592 (0.0007) [2023-10-08 04:37:46,445][00612] Updated weights for policy 1, policy_version 17680 (0.0009) [2023-10-08 04:37:46,819][00612] Updated weights for policy 1, policy_version 17690 (0.0007) [2023-10-08 04:37:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 36143104. Throughput: 0: 1811.6, 1: 1828.4. Samples: 9046298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:37:48,755][130385] Avg episode reward: [(0, '40.180'), (1, '36.360')] [2023-10-08 04:37:50,149][00611] Updated weights for policy 0, policy_version 17602 (0.0008) [2023-10-08 04:37:50,361][00612] Updated weights for policy 1, policy_version 17700 (0.0008) [2023-10-08 04:37:50,509][00611] Updated weights for policy 0, policy_version 17612 (0.0008) [2023-10-08 04:37:50,722][00612] Updated weights for policy 1, policy_version 17710 (0.0008) [2023-10-08 04:37:50,878][00611] Updated weights for policy 0, policy_version 17622 (0.0008) [2023-10-08 04:37:51,088][00612] Updated weights for policy 1, policy_version 17720 (0.0008) [2023-10-08 04:37:51,251][00611] Updated weights for policy 0, policy_version 17632 (0.0010) [2023-10-08 04:37:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 36208640. Throughput: 0: 1813.2, 1: 1830.6. Samples: 9056622. Policy #0 lag: (min: 47.0, avg: 55.7, max: 56.0) [2023-10-08 04:37:53,755][130385] Avg episode reward: [(0, '41.440'), (1, '35.800')] [2023-10-08 04:37:54,747][00612] Updated weights for policy 1, policy_version 17730 (0.0007) [2023-10-08 04:37:54,891][00611] Updated weights for policy 0, policy_version 17642 (0.0007) [2023-10-08 04:37:55,121][00612] Updated weights for policy 1, policy_version 17740 (0.0008) [2023-10-08 04:37:55,266][00611] Updated weights for policy 0, policy_version 17652 (0.0007) [2023-10-08 04:37:55,490][00612] Updated weights for policy 1, policy_version 17750 (0.0008) [2023-10-08 04:37:55,645][00611] Updated weights for policy 0, policy_version 17662 (0.0009) [2023-10-08 04:37:55,846][00612] Updated weights for policy 1, policy_version 17760 (0.0008) [2023-10-08 04:37:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 36274176. Throughput: 0: 1812.6, 1: 1833.1. Samples: 9079192. Policy #0 lag: (min: 47.0, avg: 55.7, max: 56.0) [2023-10-08 04:37:58,755][130385] Avg episode reward: [(0, '36.900'), (1, '36.540')] [2023-10-08 04:37:59,159][00611] Updated weights for policy 0, policy_version 17672 (0.0009) [2023-10-08 04:37:59,534][00611] Updated weights for policy 0, policy_version 17682 (0.0007) [2023-10-08 04:37:59,588][00612] Updated weights for policy 1, policy_version 17770 (0.0007) [2023-10-08 04:37:59,912][00611] Updated weights for policy 0, policy_version 17692 (0.0007) [2023-10-08 04:37:59,966][00612] Updated weights for policy 1, policy_version 17780 (0.0007) [2023-10-08 04:38:00,325][00612] Updated weights for policy 1, policy_version 17790 (0.0008) [2023-10-08 04:38:03,522][00611] Updated weights for policy 0, policy_version 17702 (0.0008) [2023-10-08 04:38:03,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 36339712. Throughput: 0: 1817.4, 1: 1831.0. Samples: 9102158. Policy #0 lag: (min: 47.0, avg: 55.7, max: 56.0) [2023-10-08 04:38:03,754][130385] Avg episode reward: [(0, '38.830'), (1, '37.830')] [2023-10-08 04:38:03,882][00611] Updated weights for policy 0, policy_version 17712 (0.0008) [2023-10-08 04:38:04,072][00612] Updated weights for policy 1, policy_version 17800 (0.0008) [2023-10-08 04:38:04,250][00611] Updated weights for policy 0, policy_version 17722 (0.0007) [2023-10-08 04:38:04,437][00612] Updated weights for policy 1, policy_version 17810 (0.0007) [2023-10-08 04:38:04,811][00612] Updated weights for policy 1, policy_version 17820 (0.0008) [2023-10-08 04:38:08,016][00611] Updated weights for policy 0, policy_version 17732 (0.0008) [2023-10-08 04:38:08,364][00612] Updated weights for policy 1, policy_version 17830 (0.0007) [2023-10-08 04:38:08,393][00611] Updated weights for policy 0, policy_version 17742 (0.0007) [2023-10-08 04:38:08,740][00612] Updated weights for policy 1, policy_version 17840 (0.0007) [2023-10-08 04:38:08,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 36405248. Throughput: 0: 1816.3, 1: 1827.7. Samples: 9112094. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-08 04:38:08,754][130385] Avg episode reward: [(0, '38.810'), (1, '36.960')] [2023-10-08 04:38:08,761][00611] Updated weights for policy 0, policy_version 17752 (0.0009) [2023-10-08 04:38:09,117][00612] Updated weights for policy 1, policy_version 17850 (0.0009) [2023-10-08 04:38:12,403][00611] Updated weights for policy 0, policy_version 17762 (0.0009) [2023-10-08 04:38:12,808][00612] Updated weights for policy 1, policy_version 17860 (0.0009) [2023-10-08 04:38:12,823][00611] Updated weights for policy 0, policy_version 17772 (0.0009) [2023-10-08 04:38:13,166][00612] Updated weights for policy 1, policy_version 17870 (0.0008) [2023-10-08 04:38:13,196][00611] Updated weights for policy 0, policy_version 17782 (0.0009) [2023-10-08 04:38:13,536][00612] Updated weights for policy 1, policy_version 17880 (0.0008) [2023-10-08 04:38:13,576][00611] Updated weights for policy 0, policy_version 17792 (0.0007) [2023-10-08 04:38:13,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 36503552. Throughput: 0: 1814.4, 1: 1831.1. Samples: 9135014. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-08 04:38:13,755][130385] Avg episode reward: [(0, '40.800'), (1, '37.700')] [2023-10-08 04:38:17,165][00612] Updated weights for policy 1, policy_version 17890 (0.0010) [2023-10-08 04:38:17,323][00611] Updated weights for policy 0, policy_version 17802 (0.0007) [2023-10-08 04:38:17,533][00612] Updated weights for policy 1, policy_version 17900 (0.0009) [2023-10-08 04:38:17,684][00611] Updated weights for policy 0, policy_version 17812 (0.0007) [2023-10-08 04:38:17,895][00612] Updated weights for policy 1, policy_version 17910 (0.0009) [2023-10-08 04:38:18,055][00611] Updated weights for policy 0, policy_version 17822 (0.0008) [2023-10-08 04:38:18,263][00612] Updated weights for policy 1, policy_version 17920 (0.0008) [2023-10-08 04:38:18,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 36601856. Throughput: 0: 1818.9, 1: 1832.0. Samples: 9155094. Policy #0 lag: (min: 18.0, avg: 18.0, max: 20.0) [2023-10-08 04:38:18,754][130385] Avg episode reward: [(0, '39.100'), (1, '37.150')] [2023-10-08 04:38:18,764][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000017920_18350080.pth... [2023-10-08 04:38:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000017824_18251776.pth... [2023-10-08 04:38:18,803][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000016128_16515072.pth [2023-10-08 04:38:18,805][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000016192_16580608.pth [2023-10-08 04:38:21,846][00611] Updated weights for policy 0, policy_version 17832 (0.0009) [2023-10-08 04:38:21,919][00612] Updated weights for policy 1, policy_version 17930 (0.0010) [2023-10-08 04:38:22,213][00611] Updated weights for policy 0, policy_version 17842 (0.0008) [2023-10-08 04:38:22,278][00612] Updated weights for policy 1, policy_version 17940 (0.0008) [2023-10-08 04:38:22,580][00611] Updated weights for policy 0, policy_version 17852 (0.0008) [2023-10-08 04:38:22,643][00612] Updated weights for policy 1, policy_version 17950 (0.0008) [2023-10-08 04:38:23,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 36667392. Throughput: 0: 1818.8, 1: 1829.3. Samples: 9167778. Policy #0 lag: (min: 18.0, avg: 18.0, max: 20.0) [2023-10-08 04:38:23,755][130385] Avg episode reward: [(0, '37.860'), (1, '35.470')] [2023-10-08 04:38:26,283][00612] Updated weights for policy 1, policy_version 17960 (0.0007) [2023-10-08 04:38:26,334][00611] Updated weights for policy 0, policy_version 17862 (0.0008) [2023-10-08 04:38:26,662][00612] Updated weights for policy 1, policy_version 17970 (0.0007) [2023-10-08 04:38:26,710][00611] Updated weights for policy 0, policy_version 17872 (0.0009) [2023-10-08 04:38:27,031][00612] Updated weights for policy 1, policy_version 17980 (0.0007) [2023-10-08 04:38:27,084][00611] Updated weights for policy 0, policy_version 17882 (0.0008) [2023-10-08 04:38:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 36732928. Throughput: 0: 1821.0, 1: 1825.9. Samples: 9188040. Policy #0 lag: (min: 18.0, avg: 18.0, max: 20.0) [2023-10-08 04:38:28,754][130385] Avg episode reward: [(0, '36.610'), (1, '35.530')] [2023-10-08 04:38:30,622][00611] Updated weights for policy 0, policy_version 17892 (0.0008) [2023-10-08 04:38:30,642][00612] Updated weights for policy 1, policy_version 17990 (0.0009) [2023-10-08 04:38:30,988][00611] Updated weights for policy 0, policy_version 17902 (0.0007) [2023-10-08 04:38:31,016][00612] Updated weights for policy 1, policy_version 18000 (0.0007) [2023-10-08 04:38:31,349][00611] Updated weights for policy 0, policy_version 17912 (0.0008) [2023-10-08 04:38:31,376][00612] Updated weights for policy 1, policy_version 18010 (0.0007) [2023-10-08 04:38:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 36798464. Throughput: 0: 1818.7, 1: 1834.8. Samples: 9210708. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-10-08 04:38:33,754][130385] Avg episode reward: [(0, '39.060'), (1, '35.330')] [2023-10-08 04:38:35,081][00611] Updated weights for policy 0, policy_version 17922 (0.0008) [2023-10-08 04:38:35,095][00612] Updated weights for policy 1, policy_version 18020 (0.0008) [2023-10-08 04:38:35,447][00611] Updated weights for policy 0, policy_version 17932 (0.0007) [2023-10-08 04:38:35,454][00612] Updated weights for policy 1, policy_version 18030 (0.0008) [2023-10-08 04:38:35,826][00611] Updated weights for policy 0, policy_version 17942 (0.0008) [2023-10-08 04:38:35,826][00612] Updated weights for policy 1, policy_version 18040 (0.0008) [2023-10-08 04:38:36,187][00611] Updated weights for policy 0, policy_version 17952 (0.0009) [2023-10-08 04:38:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 36864000. Throughput: 0: 1820.1, 1: 1823.3. Samples: 9220572. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-10-08 04:38:38,754][130385] Avg episode reward: [(0, '41.160'), (1, '37.080')] [2023-10-08 04:38:39,582][00612] Updated weights for policy 1, policy_version 18050 (0.0007) [2023-10-08 04:38:39,825][00611] Updated weights for policy 0, policy_version 17962 (0.0008) [2023-10-08 04:38:39,949][00612] Updated weights for policy 1, policy_version 18060 (0.0007) [2023-10-08 04:38:40,201][00611] Updated weights for policy 0, policy_version 17972 (0.0010) [2023-10-08 04:38:40,305][00612] Updated weights for policy 1, policy_version 18070 (0.0007) [2023-10-08 04:38:40,572][00611] Updated weights for policy 0, policy_version 17982 (0.0008) [2023-10-08 04:38:40,673][00612] Updated weights for policy 1, policy_version 18080 (0.0009) [2023-10-08 04:38:43,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 36929536. Throughput: 0: 1814.1, 1: 1831.1. Samples: 9243228. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-10-08 04:38:43,755][130385] Avg episode reward: [(0, '41.390'), (1, '37.850')] [2023-10-08 04:38:44,179][00611] Updated weights for policy 0, policy_version 17992 (0.0008) [2023-10-08 04:38:44,441][00612] Updated weights for policy 1, policy_version 18090 (0.0009) [2023-10-08 04:38:44,549][00611] Updated weights for policy 0, policy_version 18002 (0.0007) [2023-10-08 04:38:44,805][00612] Updated weights for policy 1, policy_version 18100 (0.0007) [2023-10-08 04:38:44,929][00611] Updated weights for policy 0, policy_version 18012 (0.0007) [2023-10-08 04:38:45,168][00612] Updated weights for policy 1, policy_version 18110 (0.0008) [2023-10-08 04:38:48,597][00611] Updated weights for policy 0, policy_version 18022 (0.0009) [2023-10-08 04:38:48,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 36995072. Throughput: 0: 1822.1, 1: 1834.4. Samples: 9266704. Policy #0 lag: (min: 24.0, avg: 41.2, max: 56.0) [2023-10-08 04:38:48,755][130385] Avg episode reward: [(0, '42.800'), (1, '36.930')] [2023-10-08 04:38:48,840][00612] Updated weights for policy 1, policy_version 18120 (0.0007) [2023-10-08 04:38:48,971][00611] Updated weights for policy 0, policy_version 18032 (0.0007) [2023-10-08 04:38:49,213][00612] Updated weights for policy 1, policy_version 18130 (0.0008) [2023-10-08 04:38:49,347][00611] Updated weights for policy 0, policy_version 18042 (0.0008) [2023-10-08 04:38:49,586][00612] Updated weights for policy 1, policy_version 18140 (0.0008) [2023-10-08 04:38:52,969][00611] Updated weights for policy 0, policy_version 18052 (0.0007) [2023-10-08 04:38:52,973][00612] Updated weights for policy 1, policy_version 18150 (0.0008) [2023-10-08 04:38:53,340][00612] Updated weights for policy 1, policy_version 18160 (0.0008) [2023-10-08 04:38:53,344][00611] Updated weights for policy 0, policy_version 18062 (0.0007) [2023-10-08 04:38:53,703][00612] Updated weights for policy 1, policy_version 18170 (0.0007) [2023-10-08 04:38:53,719][00611] Updated weights for policy 0, policy_version 18072 (0.0009) [2023-10-08 04:38:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 37060608. Throughput: 0: 1823.0, 1: 1832.6. Samples: 9276598. Policy #0 lag: (min: 24.0, avg: 41.2, max: 56.0) [2023-10-08 04:38:53,755][130385] Avg episode reward: [(0, '41.250'), (1, '39.050')] [2023-10-08 04:38:57,415][00611] Updated weights for policy 0, policy_version 18082 (0.0007) [2023-10-08 04:38:57,416][00612] Updated weights for policy 1, policy_version 18180 (0.0010) [2023-10-08 04:38:57,774][00612] Updated weights for policy 1, policy_version 18190 (0.0008) [2023-10-08 04:38:57,775][00611] Updated weights for policy 0, policy_version 18092 (0.0007) [2023-10-08 04:38:58,148][00611] Updated weights for policy 0, policy_version 18102 (0.0008) [2023-10-08 04:38:58,151][00612] Updated weights for policy 1, policy_version 18200 (0.0008) [2023-10-08 04:38:58,530][00611] Updated weights for policy 0, policy_version 18112 (0.0008) [2023-10-08 04:38:58,754][130385] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 37191680. Throughput: 0: 1820.0, 1: 1837.2. Samples: 9299588. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 04:38:58,754][130385] Avg episode reward: [(0, '39.540'), (1, '39.680')] [2023-10-08 04:39:01,788][00612] Updated weights for policy 1, policy_version 18210 (0.0008) [2023-10-08 04:39:02,145][00611] Updated weights for policy 0, policy_version 18122 (0.0008) [2023-10-08 04:39:02,161][00612] Updated weights for policy 1, policy_version 18220 (0.0009) [2023-10-08 04:39:02,523][00611] Updated weights for policy 0, policy_version 18132 (0.0009) [2023-10-08 04:39:02,527][00612] Updated weights for policy 1, policy_version 18230 (0.0009) [2023-10-08 04:39:02,891][00612] Updated weights for policy 1, policy_version 18240 (0.0008) [2023-10-08 04:39:02,897][00611] Updated weights for policy 0, policy_version 18142 (0.0008) [2023-10-08 04:39:03,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 37257216. Throughput: 0: 1819.8, 1: 1831.9. Samples: 9319422. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 04:39:03,755][130385] Avg episode reward: [(0, '40.680'), (1, '41.030')] [2023-10-08 04:39:06,528][00611] Updated weights for policy 0, policy_version 18152 (0.0007) [2023-10-08 04:39:06,533][00612] Updated weights for policy 1, policy_version 18250 (0.0008) [2023-10-08 04:39:06,892][00612] Updated weights for policy 1, policy_version 18260 (0.0007) [2023-10-08 04:39:06,901][00611] Updated weights for policy 0, policy_version 18162 (0.0008) [2023-10-08 04:39:07,255][00612] Updated weights for policy 1, policy_version 18270 (0.0007) [2023-10-08 04:39:07,267][00611] Updated weights for policy 0, policy_version 18172 (0.0008) [2023-10-08 04:39:08,754][130385] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 37322752. Throughput: 0: 1827.6, 1: 1833.6. Samples: 9332534. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 04:39:08,755][130385] Avg episode reward: [(0, '42.410'), (1, '39.060')] [2023-10-08 04:39:10,925][00612] Updated weights for policy 1, policy_version 18280 (0.0007) [2023-10-08 04:39:10,930][00611] Updated weights for policy 0, policy_version 18182 (0.0008) [2023-10-08 04:39:11,298][00612] Updated weights for policy 1, policy_version 18290 (0.0008) [2023-10-08 04:39:11,301][00611] Updated weights for policy 0, policy_version 18192 (0.0007) [2023-10-08 04:39:11,661][00612] Updated weights for policy 1, policy_version 18300 (0.0007) [2023-10-08 04:39:11,662][00611] Updated weights for policy 0, policy_version 18202 (0.0008) [2023-10-08 04:39:13,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 37388288. Throughput: 0: 1820.5, 1: 1827.4. Samples: 9352198. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 04:39:13,754][130385] Avg episode reward: [(0, '36.510'), (1, '35.900')] [2023-10-08 04:39:15,331][00612] Updated weights for policy 1, policy_version 18310 (0.0008) [2023-10-08 04:39:15,374][00611] Updated weights for policy 0, policy_version 18212 (0.0009) [2023-10-08 04:39:15,705][00612] Updated weights for policy 1, policy_version 18320 (0.0008) [2023-10-08 04:39:15,751][00611] Updated weights for policy 0, policy_version 18222 (0.0009) [2023-10-08 04:39:16,068][00612] Updated weights for policy 1, policy_version 18330 (0.0008) [2023-10-08 04:39:16,125][00611] Updated weights for policy 0, policy_version 18232 (0.0009) [2023-10-08 04:39:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 37453824. Throughput: 0: 1826.5, 1: 1832.2. Samples: 9375350. Policy #0 lag: (min: 1.0, avg: 28.4, max: 32.0) [2023-10-08 04:39:18,755][130385] Avg episode reward: [(0, '36.800'), (1, '34.960')] [2023-10-08 04:39:19,795][00612] Updated weights for policy 1, policy_version 18340 (0.0008) [2023-10-08 04:39:19,810][00611] Updated weights for policy 0, policy_version 18242 (0.0008) [2023-10-08 04:39:20,185][00612] Updated weights for policy 1, policy_version 18350 (0.0007) [2023-10-08 04:39:20,190][00611] Updated weights for policy 0, policy_version 18252 (0.0008) [2023-10-08 04:39:20,551][00612] Updated weights for policy 1, policy_version 18360 (0.0007) [2023-10-08 04:39:20,556][00611] Updated weights for policy 0, policy_version 18262 (0.0008) [2023-10-08 04:39:20,921][00611] Updated weights for policy 0, policy_version 18272 (0.0009) [2023-10-08 04:39:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 37519360. Throughput: 0: 1824.5, 1: 1834.1. Samples: 9385208. Policy #0 lag: (min: 1.0, avg: 28.4, max: 32.0) [2023-10-08 04:39:23,754][130385] Avg episode reward: [(0, '35.180'), (1, '32.100')] [2023-10-08 04:39:24,230][00612] Updated weights for policy 1, policy_version 18370 (0.0009) [2023-10-08 04:39:24,588][00612] Updated weights for policy 1, policy_version 18380 (0.0008) [2023-10-08 04:39:24,666][00611] Updated weights for policy 0, policy_version 18282 (0.0009) [2023-10-08 04:39:24,964][00612] Updated weights for policy 1, policy_version 18390 (0.0008) [2023-10-08 04:39:25,029][00611] Updated weights for policy 0, policy_version 18292 (0.0009) [2023-10-08 04:39:25,322][00612] Updated weights for policy 1, policy_version 18400 (0.0008) [2023-10-08 04:39:25,391][00611] Updated weights for policy 0, policy_version 18302 (0.0009) [2023-10-08 04:39:28,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 37584896. Throughput: 0: 1833.1, 1: 1834.6. Samples: 9408274. Policy #0 lag: (min: 1.0, avg: 28.4, max: 32.0) [2023-10-08 04:39:28,754][130385] Avg episode reward: [(0, '37.080'), (1, '33.590')] [2023-10-08 04:39:28,942][00612] Updated weights for policy 1, policy_version 18410 (0.0008) [2023-10-08 04:39:28,989][00611] Updated weights for policy 0, policy_version 18312 (0.0008) [2023-10-08 04:39:29,306][00612] Updated weights for policy 1, policy_version 18420 (0.0007) [2023-10-08 04:39:29,359][00611] Updated weights for policy 0, policy_version 18322 (0.0007) [2023-10-08 04:39:29,676][00612] Updated weights for policy 1, policy_version 18430 (0.0007) [2023-10-08 04:39:29,729][00611] Updated weights for policy 0, policy_version 18332 (0.0009) [2023-10-08 04:39:33,363][00612] Updated weights for policy 1, policy_version 18440 (0.0007) [2023-10-08 04:39:33,427][00611] Updated weights for policy 0, policy_version 18342 (0.0007) [2023-10-08 04:39:33,737][00612] Updated weights for policy 1, policy_version 18450 (0.0007) [2023-10-08 04:39:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 37650432. Throughput: 0: 1823.8, 1: 1826.2. Samples: 9430954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:39:33,754][130385] Avg episode reward: [(0, '35.780'), (1, '32.710')] [2023-10-08 04:39:33,795][00611] Updated weights for policy 0, policy_version 18352 (0.0007) [2023-10-08 04:39:34,095][00612] Updated weights for policy 1, policy_version 18460 (0.0007) [2023-10-08 04:39:34,171][00611] Updated weights for policy 0, policy_version 18362 (0.0007) [2023-10-08 04:39:37,770][00612] Updated weights for policy 1, policy_version 18470 (0.0008) [2023-10-08 04:39:37,784][00611] Updated weights for policy 0, policy_version 18372 (0.0008) [2023-10-08 04:39:38,132][00612] Updated weights for policy 1, policy_version 18480 (0.0010) [2023-10-08 04:39:38,161][00611] Updated weights for policy 0, policy_version 18382 (0.0008) [2023-10-08 04:39:38,506][00612] Updated weights for policy 1, policy_version 18490 (0.0009) [2023-10-08 04:39:38,527][00611] Updated weights for policy 0, policy_version 18392 (0.0008) [2023-10-08 04:39:38,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37748736. Throughput: 0: 1826.1, 1: 1828.8. Samples: 9441066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:39:38,754][130385] Avg episode reward: [(0, '34.680'), (1, '33.240')] [2023-10-08 04:39:42,253][00612] Updated weights for policy 1, policy_version 18500 (0.0008) [2023-10-08 04:39:42,282][00611] Updated weights for policy 0, policy_version 18402 (0.0009) [2023-10-08 04:39:42,614][00612] Updated weights for policy 1, policy_version 18510 (0.0007) [2023-10-08 04:39:42,681][00611] Updated weights for policy 0, policy_version 18412 (0.0008) [2023-10-08 04:39:42,985][00612] Updated weights for policy 1, policy_version 18520 (0.0008) [2023-10-08 04:39:43,051][00611] Updated weights for policy 0, policy_version 18422 (0.0008) [2023-10-08 04:39:43,437][00611] Updated weights for policy 0, policy_version 18432 (0.0010) [2023-10-08 04:39:43,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 37847040. Throughput: 0: 1824.8, 1: 1822.0. Samples: 9463694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:39:43,754][130385] Avg episode reward: [(0, '35.120'), (1, '34.700')] [2023-10-08 04:39:46,755][00612] Updated weights for policy 1, policy_version 18530 (0.0009) [2023-10-08 04:39:47,025][00611] Updated weights for policy 0, policy_version 18442 (0.0008) [2023-10-08 04:39:47,123][00612] Updated weights for policy 1, policy_version 18540 (0.0009) [2023-10-08 04:39:47,393][00611] Updated weights for policy 0, policy_version 18452 (0.0008) [2023-10-08 04:39:47,484][00612] Updated weights for policy 1, policy_version 18550 (0.0007) [2023-10-08 04:39:47,763][00611] Updated weights for policy 0, policy_version 18462 (0.0008) [2023-10-08 04:39:47,860][00612] Updated weights for policy 1, policy_version 18560 (0.0007) [2023-10-08 04:39:48,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 37912576. Throughput: 0: 1825.9, 1: 1819.3. Samples: 9483458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:39:48,754][130385] Avg episode reward: [(0, '37.190'), (1, '38.160')] [2023-10-08 04:39:51,496][00611] Updated weights for policy 0, policy_version 18472 (0.0007) [2023-10-08 04:39:51,522][00612] Updated weights for policy 1, policy_version 18570 (0.0007) [2023-10-08 04:39:51,872][00611] Updated weights for policy 0, policy_version 18482 (0.0009) [2023-10-08 04:39:51,883][00612] Updated weights for policy 1, policy_version 18580 (0.0008) [2023-10-08 04:39:52,240][00611] Updated weights for policy 0, policy_version 18492 (0.0008) [2023-10-08 04:39:52,258][00612] Updated weights for policy 1, policy_version 18590 (0.0009) [2023-10-08 04:39:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 37978112. Throughput: 0: 1822.3, 1: 1820.2. Samples: 9496448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:39:53,754][130385] Avg episode reward: [(0, '39.880'), (1, '40.340')] [2023-10-08 04:39:55,978][00611] Updated weights for policy 0, policy_version 18502 (0.0008) [2023-10-08 04:39:56,021][00612] Updated weights for policy 1, policy_version 18600 (0.0009) [2023-10-08 04:39:56,345][00611] Updated weights for policy 0, policy_version 18512 (0.0007) [2023-10-08 04:39:56,382][00612] Updated weights for policy 1, policy_version 18610 (0.0009) [2023-10-08 04:39:56,715][00611] Updated weights for policy 0, policy_version 18522 (0.0007) [2023-10-08 04:39:56,751][00612] Updated weights for policy 1, policy_version 18620 (0.0007) [2023-10-08 04:39:58,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 38043648. Throughput: 0: 1820.8, 1: 1820.6. Samples: 9516062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:39:58,755][130385] Avg episode reward: [(0, '40.910'), (1, '41.750')] [2023-10-08 04:40:00,370][00611] Updated weights for policy 0, policy_version 18532 (0.0010) [2023-10-08 04:40:00,418][00612] Updated weights for policy 1, policy_version 18630 (0.0009) [2023-10-08 04:40:00,739][00611] Updated weights for policy 0, policy_version 18542 (0.0009) [2023-10-08 04:40:00,784][00612] Updated weights for policy 1, policy_version 18640 (0.0008) [2023-10-08 04:40:01,114][00611] Updated weights for policy 0, policy_version 18552 (0.0008) [2023-10-08 04:40:01,154][00612] Updated weights for policy 1, policy_version 18650 (0.0009) [2023-10-08 04:40:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 38109184. Throughput: 0: 1822.0, 1: 1812.0. Samples: 9538880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:40:03,754][130385] Avg episode reward: [(0, '41.160'), (1, '43.090')] [2023-10-08 04:40:04,901][00611] Updated weights for policy 0, policy_version 18562 (0.0009) [2023-10-08 04:40:04,958][00612] Updated weights for policy 1, policy_version 18660 (0.0008) [2023-10-08 04:40:05,280][00611] Updated weights for policy 0, policy_version 18572 (0.0008) [2023-10-08 04:40:05,352][00612] Updated weights for policy 1, policy_version 18670 (0.0009) [2023-10-08 04:40:05,642][00611] Updated weights for policy 0, policy_version 18582 (0.0010) [2023-10-08 04:40:05,725][00612] Updated weights for policy 1, policy_version 18680 (0.0009) [2023-10-08 04:40:06,017][00611] Updated weights for policy 0, policy_version 18592 (0.0008) [2023-10-08 04:40:08,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 38174720. Throughput: 0: 1820.1, 1: 1807.7. Samples: 9548460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:40:08,754][130385] Avg episode reward: [(0, '42.610'), (1, '43.020')] [2023-10-08 04:40:09,332][00612] Updated weights for policy 1, policy_version 18690 (0.0008) [2023-10-08 04:40:09,707][00612] Updated weights for policy 1, policy_version 18700 (0.0008) [2023-10-08 04:40:09,767][00611] Updated weights for policy 0, policy_version 18602 (0.0008) [2023-10-08 04:40:10,079][00612] Updated weights for policy 1, policy_version 18710 (0.0010) [2023-10-08 04:40:10,148][00611] Updated weights for policy 0, policy_version 18612 (0.0007) [2023-10-08 04:40:10,444][00612] Updated weights for policy 1, policy_version 18720 (0.0008) [2023-10-08 04:40:10,528][00611] Updated weights for policy 0, policy_version 18622 (0.0009) [2023-10-08 04:40:13,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 38240256. Throughput: 0: 1813.3, 1: 1811.1. Samples: 9571372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:40:13,755][130385] Avg episode reward: [(0, '43.580'), (1, '38.650')] [2023-10-08 04:40:13,756][00365] Saving new best policy, reward=43.580! [2023-10-08 04:40:14,242][00612] Updated weights for policy 1, policy_version 18730 (0.0009) [2023-10-08 04:40:14,329][00611] Updated weights for policy 0, policy_version 18632 (0.0009) [2023-10-08 04:40:14,607][00612] Updated weights for policy 1, policy_version 18740 (0.0008) [2023-10-08 04:40:14,689][00611] Updated weights for policy 0, policy_version 18642 (0.0008) [2023-10-08 04:40:14,972][00612] Updated weights for policy 1, policy_version 18750 (0.0007) [2023-10-08 04:40:15,057][00611] Updated weights for policy 0, policy_version 18652 (0.0009) [2023-10-08 04:40:18,468][00612] Updated weights for policy 1, policy_version 18760 (0.0007) [2023-10-08 04:40:18,677][00611] Updated weights for policy 0, policy_version 18662 (0.0008) [2023-10-08 04:40:18,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 38305792. Throughput: 0: 1811.9, 1: 1818.2. Samples: 9594310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:40:18,754][130385] Avg episode reward: [(0, '43.410'), (1, '37.820')] [2023-10-08 04:40:18,830][00612] Updated weights for policy 1, policy_version 18770 (0.0007) [2023-10-08 04:40:19,048][00611] Updated weights for policy 0, policy_version 18672 (0.0008) [2023-10-08 04:40:19,192][00612] Updated weights for policy 1, policy_version 18780 (0.0007) [2023-10-08 04:40:19,338][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000018784_19234816.pth... [2023-10-08 04:40:19,370][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000017056_17465344.pth [2023-10-08 04:40:19,411][00611] Updated weights for policy 0, policy_version 18682 (0.0008) [2023-10-08 04:40:19,632][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000018688_19136512.pth... [2023-10-08 04:40:19,670][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000016960_17367040.pth [2023-10-08 04:40:23,028][00612] Updated weights for policy 1, policy_version 18790 (0.0007) [2023-10-08 04:40:23,031][00611] Updated weights for policy 0, policy_version 18692 (0.0008) [2023-10-08 04:40:23,399][00611] Updated weights for policy 0, policy_version 18702 (0.0008) [2023-10-08 04:40:23,401][00612] Updated weights for policy 1, policy_version 18800 (0.0007) [2023-10-08 04:40:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 38371328. Throughput: 0: 1812.7, 1: 1813.1. Samples: 9604228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:40:23,754][130385] Avg episode reward: [(0, '44.390'), (1, '37.540')] [2023-10-08 04:40:23,767][00611] Updated weights for policy 0, policy_version 18712 (0.0008) [2023-10-08 04:40:23,770][00612] Updated weights for policy 1, policy_version 18810 (0.0009) [2023-10-08 04:40:24,060][00365] Saving new best policy, reward=44.390! [2023-10-08 04:40:27,448][00611] Updated weights for policy 0, policy_version 18722 (0.0007) [2023-10-08 04:40:27,568][00612] Updated weights for policy 1, policy_version 18820 (0.0008) [2023-10-08 04:40:27,825][00611] Updated weights for policy 0, policy_version 18732 (0.0007) [2023-10-08 04:40:27,927][00612] Updated weights for policy 1, policy_version 18830 (0.0007) [2023-10-08 04:40:28,203][00611] Updated weights for policy 0, policy_version 18742 (0.0009) [2023-10-08 04:40:28,301][00612] Updated weights for policy 1, policy_version 18840 (0.0008) [2023-10-08 04:40:28,566][00611] Updated weights for policy 0, policy_version 18752 (0.0008) [2023-10-08 04:40:28,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 38502400. Throughput: 0: 1817.4, 1: 1809.6. Samples: 9626912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:40:28,754][130385] Avg episode reward: [(0, '44.350'), (1, '37.260')] [2023-10-08 04:40:31,926][00612] Updated weights for policy 1, policy_version 18850 (0.0008) [2023-10-08 04:40:32,147][00611] Updated weights for policy 0, policy_version 18762 (0.0008) [2023-10-08 04:40:32,283][00612] Updated weights for policy 1, policy_version 18860 (0.0008) [2023-10-08 04:40:32,515][00611] Updated weights for policy 0, policy_version 18772 (0.0009) [2023-10-08 04:40:32,650][00612] Updated weights for policy 1, policy_version 18870 (0.0008) [2023-10-08 04:40:32,887][00611] Updated weights for policy 0, policy_version 18782 (0.0008) [2023-10-08 04:40:33,019][00612] Updated weights for policy 1, policy_version 18880 (0.0009) [2023-10-08 04:40:33,754][130385] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 38567936. Throughput: 0: 1818.5, 1: 1809.6. Samples: 9646726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:40:33,755][130385] Avg episode reward: [(0, '39.390'), (1, '37.390')] [2023-10-08 04:40:36,609][00611] Updated weights for policy 0, policy_version 18792 (0.0007) [2023-10-08 04:40:36,806][00612] Updated weights for policy 1, policy_version 18890 (0.0007) [2023-10-08 04:40:36,981][00611] Updated weights for policy 0, policy_version 18802 (0.0007) [2023-10-08 04:40:37,167][00612] Updated weights for policy 1, policy_version 18900 (0.0009) [2023-10-08 04:40:37,345][00611] Updated weights for policy 0, policy_version 18812 (0.0010) [2023-10-08 04:40:37,541][00612] Updated weights for policy 1, policy_version 18910 (0.0009) [2023-10-08 04:40:38,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 38633472. Throughput: 0: 1817.0, 1: 1807.6. Samples: 9659556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:40:38,754][130385] Avg episode reward: [(0, '37.020'), (1, '36.660')] [2023-10-08 04:40:41,069][00612] Updated weights for policy 1, policy_version 18920 (0.0007) [2023-10-08 04:40:41,133][00611] Updated weights for policy 0, policy_version 18822 (0.0007) [2023-10-08 04:40:41,438][00612] Updated weights for policy 1, policy_version 18930 (0.0007) [2023-10-08 04:40:41,497][00611] Updated weights for policy 0, policy_version 18832 (0.0008) [2023-10-08 04:40:41,813][00612] Updated weights for policy 1, policy_version 18940 (0.0010) [2023-10-08 04:40:41,871][00611] Updated weights for policy 0, policy_version 18842 (0.0008) [2023-10-08 04:40:43,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 38699008. Throughput: 0: 1819.0, 1: 1814.9. Samples: 9679590. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-08 04:40:43,754][130385] Avg episode reward: [(0, '35.020'), (1, '36.440')] [2023-10-08 04:40:45,416][00612] Updated weights for policy 1, policy_version 18950 (0.0007) [2023-10-08 04:40:45,467][00611] Updated weights for policy 0, policy_version 18852 (0.0008) [2023-10-08 04:40:45,771][00612] Updated weights for policy 1, policy_version 18960 (0.0009) [2023-10-08 04:40:45,827][00611] Updated weights for policy 0, policy_version 18862 (0.0007) [2023-10-08 04:40:46,138][00612] Updated weights for policy 1, policy_version 18970 (0.0009) [2023-10-08 04:40:46,200][00611] Updated weights for policy 0, policy_version 18872 (0.0007) [2023-10-08 04:40:48,754][130385] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 38764544. Throughput: 0: 1819.0, 1: 1822.6. Samples: 9702752. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-08 04:40:48,756][130385] Avg episode reward: [(0, '33.650'), (1, '37.770')] [2023-10-08 04:40:49,798][00611] Updated weights for policy 0, policy_version 18882 (0.0007) [2023-10-08 04:40:49,981][00612] Updated weights for policy 1, policy_version 18980 (0.0008) [2023-10-08 04:40:50,158][00611] Updated weights for policy 0, policy_version 18892 (0.0010) [2023-10-08 04:40:50,365][00612] Updated weights for policy 1, policy_version 18990 (0.0008) [2023-10-08 04:40:50,528][00611] Updated weights for policy 0, policy_version 18902 (0.0009) [2023-10-08 04:40:50,728][00612] Updated weights for policy 1, policy_version 19000 (0.0007) [2023-10-08 04:40:50,895][00611] Updated weights for policy 0, policy_version 18912 (0.0009) [2023-10-08 04:40:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 38830080. Throughput: 0: 1818.5, 1: 1823.9. Samples: 9712370. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-08 04:40:53,754][130385] Avg episode reward: [(0, '36.950'), (1, '38.510')] [2023-10-08 04:40:54,268][00612] Updated weights for policy 1, policy_version 19010 (0.0007) [2023-10-08 04:40:54,633][00612] Updated weights for policy 1, policy_version 19020 (0.0007) [2023-10-08 04:40:54,644][00611] Updated weights for policy 0, policy_version 18922 (0.0007) [2023-10-08 04:40:55,002][00612] Updated weights for policy 1, policy_version 19030 (0.0007) [2023-10-08 04:40:55,017][00611] Updated weights for policy 0, policy_version 18932 (0.0007) [2023-10-08 04:40:55,364][00612] Updated weights for policy 1, policy_version 19040 (0.0007) [2023-10-08 04:40:55,380][00611] Updated weights for policy 0, policy_version 18942 (0.0007) [2023-10-08 04:40:58,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 38895616. Throughput: 0: 1821.6, 1: 1824.9. Samples: 9735464. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-08 04:40:58,754][130385] Avg episode reward: [(0, '36.620'), (1, '39.250')] [2023-10-08 04:40:59,016][00612] Updated weights for policy 1, policy_version 19050 (0.0007) [2023-10-08 04:40:59,044][00611] Updated weights for policy 0, policy_version 18952 (0.0010) [2023-10-08 04:40:59,387][00612] Updated weights for policy 1, policy_version 19060 (0.0008) [2023-10-08 04:40:59,415][00611] Updated weights for policy 0, policy_version 18962 (0.0007) [2023-10-08 04:40:59,766][00612] Updated weights for policy 1, policy_version 19070 (0.0008) [2023-10-08 04:40:59,782][00611] Updated weights for policy 0, policy_version 18972 (0.0007) [2023-10-08 04:41:03,443][00611] Updated weights for policy 0, policy_version 18982 (0.0007) [2023-10-08 04:41:03,510][00612] Updated weights for policy 1, policy_version 19080 (0.0007) [2023-10-08 04:41:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 38961152. Throughput: 0: 1825.1, 1: 1822.1. Samples: 9758434. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 04:41:03,754][130385] Avg episode reward: [(0, '36.020'), (1, '38.670')] [2023-10-08 04:41:03,807][00611] Updated weights for policy 0, policy_version 18992 (0.0007) [2023-10-08 04:41:03,872][00612] Updated weights for policy 1, policy_version 19090 (0.0009) [2023-10-08 04:41:04,184][00611] Updated weights for policy 0, policy_version 19002 (0.0007) [2023-10-08 04:41:04,241][00612] Updated weights for policy 1, policy_version 19100 (0.0010) [2023-10-08 04:41:07,889][00611] Updated weights for policy 0, policy_version 19012 (0.0008) [2023-10-08 04:41:07,915][00612] Updated weights for policy 1, policy_version 19110 (0.0007) [2023-10-08 04:41:08,247][00611] Updated weights for policy 0, policy_version 19022 (0.0008) [2023-10-08 04:41:08,274][00612] Updated weights for policy 1, policy_version 19120 (0.0008) [2023-10-08 04:41:08,630][00611] Updated weights for policy 0, policy_version 19032 (0.0008) [2023-10-08 04:41:08,645][00612] Updated weights for policy 1, policy_version 19130 (0.0008) [2023-10-08 04:41:08,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 39026688. Throughput: 0: 1823.9, 1: 1822.1. Samples: 9768300. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 04:41:08,755][130385] Avg episode reward: [(0, '34.970'), (1, '39.370')] [2023-10-08 04:41:12,224][00612] Updated weights for policy 1, policy_version 19140 (0.0007) [2023-10-08 04:41:12,302][00611] Updated weights for policy 0, policy_version 19042 (0.0007) [2023-10-08 04:41:12,585][00612] Updated weights for policy 1, policy_version 19150 (0.0007) [2023-10-08 04:41:12,699][00611] Updated weights for policy 0, policy_version 19052 (0.0009) [2023-10-08 04:41:12,957][00612] Updated weights for policy 1, policy_version 19160 (0.0008) [2023-10-08 04:41:13,070][00611] Updated weights for policy 0, policy_version 19062 (0.0008) [2023-10-08 04:41:13,438][00611] Updated weights for policy 0, policy_version 19072 (0.0009) [2023-10-08 04:41:13,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 39157760. Throughput: 0: 1829.2, 1: 1830.5. Samples: 9791598. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 04:41:13,755][130385] Avg episode reward: [(0, '40.050'), (1, '38.380')] [2023-10-08 04:41:16,467][00612] Updated weights for policy 1, policy_version 19170 (0.0008) [2023-10-08 04:41:16,845][00612] Updated weights for policy 1, policy_version 19180 (0.0007) [2023-10-08 04:41:17,062][00611] Updated weights for policy 0, policy_version 19082 (0.0009) [2023-10-08 04:41:17,218][00612] Updated weights for policy 1, policy_version 19190 (0.0007) [2023-10-08 04:41:17,428][00611] Updated weights for policy 0, policy_version 19092 (0.0008) [2023-10-08 04:41:17,585][00612] Updated weights for policy 1, policy_version 19200 (0.0007) [2023-10-08 04:41:17,802][00611] Updated weights for policy 0, policy_version 19102 (0.0007) [2023-10-08 04:41:18,754][130385] Fps is (10 sec: 19661.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 39223296. Throughput: 0: 1825.6, 1: 1840.1. Samples: 9811684. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) [2023-10-08 04:41:18,754][130385] Avg episode reward: [(0, '41.010'), (1, '36.970')] [2023-10-08 04:41:21,120][00612] Updated weights for policy 1, policy_version 19210 (0.0008) [2023-10-08 04:41:21,448][00611] Updated weights for policy 0, policy_version 19112 (0.0008) [2023-10-08 04:41:21,485][00612] Updated weights for policy 1, policy_version 19220 (0.0008) [2023-10-08 04:41:21,820][00611] Updated weights for policy 0, policy_version 19122 (0.0009) [2023-10-08 04:41:21,859][00612] Updated weights for policy 1, policy_version 19230 (0.0009) [2023-10-08 04:41:22,187][00611] Updated weights for policy 0, policy_version 19132 (0.0008) [2023-10-08 04:41:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 39288832. Throughput: 0: 1826.4, 1: 1833.6. Samples: 9824254. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) [2023-10-08 04:41:23,754][130385] Avg episode reward: [(0, '41.260'), (1, '36.380')] [2023-10-08 04:41:25,504][00612] Updated weights for policy 1, policy_version 19240 (0.0008) [2023-10-08 04:41:25,861][00612] Updated weights for policy 1, policy_version 19250 (0.0007) [2023-10-08 04:41:25,883][00611] Updated weights for policy 0, policy_version 19142 (0.0007) [2023-10-08 04:41:26,225][00612] Updated weights for policy 1, policy_version 19260 (0.0007) [2023-10-08 04:41:26,247][00611] Updated weights for policy 0, policy_version 19152 (0.0007) [2023-10-08 04:41:26,617][00611] Updated weights for policy 0, policy_version 19162 (0.0008) [2023-10-08 04:41:28,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 39354368. Throughput: 0: 1823.6, 1: 1844.9. Samples: 9844672. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) [2023-10-08 04:41:28,754][130385] Avg episode reward: [(0, '42.540'), (1, '36.590')] [2023-10-08 04:41:30,057][00612] Updated weights for policy 1, policy_version 19270 (0.0009) [2023-10-08 04:41:30,214][00611] Updated weights for policy 0, policy_version 19172 (0.0007) [2023-10-08 04:41:30,425][00612] Updated weights for policy 1, policy_version 19280 (0.0008) [2023-10-08 04:41:30,590][00611] Updated weights for policy 0, policy_version 19182 (0.0008) [2023-10-08 04:41:30,793][00612] Updated weights for policy 1, policy_version 19290 (0.0009) [2023-10-08 04:41:30,963][00611] Updated weights for policy 0, policy_version 19192 (0.0009) [2023-10-08 04:41:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 39419904. Throughput: 0: 1822.0, 1: 1845.4. Samples: 9867782. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) [2023-10-08 04:41:33,754][130385] Avg episode reward: [(0, '38.220'), (1, '38.480')] [2023-10-08 04:41:34,381][00612] Updated weights for policy 1, policy_version 19300 (0.0008) [2023-10-08 04:41:34,640][00611] Updated weights for policy 0, policy_version 19202 (0.0008) [2023-10-08 04:41:34,763][00612] Updated weights for policy 1, policy_version 19310 (0.0008) [2023-10-08 04:41:35,012][00611] Updated weights for policy 0, policy_version 19212 (0.0009) [2023-10-08 04:41:35,136][00612] Updated weights for policy 1, policy_version 19320 (0.0009) [2023-10-08 04:41:35,378][00611] Updated weights for policy 0, policy_version 19222 (0.0008) [2023-10-08 04:41:35,749][00611] Updated weights for policy 0, policy_version 19232 (0.0011) [2023-10-08 04:41:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 39485440. Throughput: 0: 1824.4, 1: 1845.8. Samples: 9877532. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-08 04:41:38,755][130385] Avg episode reward: [(0, '40.740'), (1, '39.440')] [2023-10-08 04:41:38,909][00612] Updated weights for policy 1, policy_version 19330 (0.0010) [2023-10-08 04:41:39,329][00612] Updated weights for policy 1, policy_version 19340 (0.0009) [2023-10-08 04:41:39,411][00611] Updated weights for policy 0, policy_version 19242 (0.0009) [2023-10-08 04:41:39,695][00612] Updated weights for policy 1, policy_version 19350 (0.0008) [2023-10-08 04:41:39,782][00611] Updated weights for policy 0, policy_version 19252 (0.0008) [2023-10-08 04:41:40,063][00612] Updated weights for policy 1, policy_version 19360 (0.0007) [2023-10-08 04:41:40,146][00611] Updated weights for policy 0, policy_version 19262 (0.0007) [2023-10-08 04:41:43,629][00612] Updated weights for policy 1, policy_version 19370 (0.0009) [2023-10-08 04:41:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 39550976. Throughput: 0: 1825.3, 1: 1838.4. Samples: 9900334. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-08 04:41:43,754][130385] Avg episode reward: [(0, '42.010'), (1, '41.020')] [2023-10-08 04:41:43,896][00611] Updated weights for policy 0, policy_version 19272 (0.0010) [2023-10-08 04:41:43,986][00612] Updated weights for policy 1, policy_version 19380 (0.0007) [2023-10-08 04:41:44,268][00611] Updated weights for policy 0, policy_version 19282 (0.0007) [2023-10-08 04:41:44,349][00612] Updated weights for policy 1, policy_version 19390 (0.0008) [2023-10-08 04:41:44,635][00611] Updated weights for policy 0, policy_version 19292 (0.0009) [2023-10-08 04:41:48,062][00612] Updated weights for policy 1, policy_version 19400 (0.0007) [2023-10-08 04:41:48,441][00612] Updated weights for policy 1, policy_version 19410 (0.0008) [2023-10-08 04:41:48,444][00611] Updated weights for policy 0, policy_version 19302 (0.0008) [2023-10-08 04:41:48,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 39616512. Throughput: 0: 1826.1, 1: 1829.9. Samples: 9922952. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-08 04:41:48,754][130385] Avg episode reward: [(0, '39.200'), (1, '38.820')] [2023-10-08 04:41:48,803][00612] Updated weights for policy 1, policy_version 19420 (0.0007) [2023-10-08 04:41:48,824][00611] Updated weights for policy 0, policy_version 19312 (0.0008) [2023-10-08 04:41:49,188][00611] Updated weights for policy 0, policy_version 19322 (0.0009) [2023-10-08 04:41:52,300][00612] Updated weights for policy 1, policy_version 19430 (0.0007) [2023-10-08 04:41:52,666][00612] Updated weights for policy 1, policy_version 19440 (0.0007) [2023-10-08 04:41:52,886][00611] Updated weights for policy 0, policy_version 19332 (0.0008) [2023-10-08 04:41:53,028][00612] Updated weights for policy 1, policy_version 19450 (0.0007) [2023-10-08 04:41:53,262][00611] Updated weights for policy 0, policy_version 19342 (0.0007) [2023-10-08 04:41:53,636][00611] Updated weights for policy 0, policy_version 19352 (0.0009) [2023-10-08 04:41:53,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39714816. Throughput: 0: 1824.4, 1: 1847.4. Samples: 9933528. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-08 04:41:53,754][130385] Avg episode reward: [(0, '38.690'), (1, '39.640')] [2023-10-08 04:41:56,717][00612] Updated weights for policy 1, policy_version 19460 (0.0009) [2023-10-08 04:41:57,086][00612] Updated weights for policy 1, policy_version 19470 (0.0007) [2023-10-08 04:41:57,455][00612] Updated weights for policy 1, policy_version 19480 (0.0007) [2023-10-08 04:41:57,482][00611] Updated weights for policy 0, policy_version 19362 (0.0008) [2023-10-08 04:41:57,892][00611] Updated weights for policy 0, policy_version 19372 (0.0007) [2023-10-08 04:41:58,261][00611] Updated weights for policy 0, policy_version 19382 (0.0008) [2023-10-08 04:41:58,629][00611] Updated weights for policy 0, policy_version 19392 (0.0008) [2023-10-08 04:41:58,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 39813120. Throughput: 0: 1818.9, 1: 1829.8. Samples: 9955790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:41:58,755][130385] Avg episode reward: [(0, '37.890'), (1, '40.870')] [2023-10-08 04:42:01,082][00612] Updated weights for policy 1, policy_version 19490 (0.0007) [2023-10-08 04:42:01,448][00612] Updated weights for policy 1, policy_version 19500 (0.0008) [2023-10-08 04:42:01,814][00612] Updated weights for policy 1, policy_version 19510 (0.0009) [2023-10-08 04:42:02,181][00612] Updated weights for policy 1, policy_version 19520 (0.0008) [2023-10-08 04:42:02,232][00611] Updated weights for policy 0, policy_version 19402 (0.0007) [2023-10-08 04:42:02,603][00611] Updated weights for policy 0, policy_version 19412 (0.0008) [2023-10-08 04:42:02,975][00611] Updated weights for policy 0, policy_version 19422 (0.0008) [2023-10-08 04:42:03,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 39878656. Throughput: 0: 1816.4, 1: 1847.3. Samples: 9976550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:42:03,754][130385] Avg episode reward: [(0, '42.400'), (1, '40.090')] [2023-10-08 04:42:05,776][00612] Updated weights for policy 1, policy_version 19530 (0.0007) [2023-10-08 04:42:06,145][00612] Updated weights for policy 1, policy_version 19540 (0.0009) [2023-10-08 04:42:06,483][00611] Updated weights for policy 0, policy_version 19432 (0.0009) [2023-10-08 04:42:06,514][00612] Updated weights for policy 1, policy_version 19550 (0.0008) [2023-10-08 04:42:06,852][00611] Updated weights for policy 0, policy_version 19442 (0.0007) [2023-10-08 04:42:07,225][00611] Updated weights for policy 0, policy_version 19452 (0.0007) [2023-10-08 04:42:08,754][130385] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 39944192. Throughput: 0: 1823.1, 1: 1837.3. Samples: 9988976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:42:08,755][130385] Avg episode reward: [(0, '41.240'), (1, '38.430')] [2023-10-08 04:42:10,017][00612] Updated weights for policy 1, policy_version 19560 (0.0007) [2023-10-08 04:42:10,383][00612] Updated weights for policy 1, policy_version 19570 (0.0010) [2023-10-08 04:42:10,763][00612] Updated weights for policy 1, policy_version 19580 (0.0008) [2023-10-08 04:42:10,952][00611] Updated weights for policy 0, policy_version 19462 (0.0010) [2023-10-08 04:42:11,329][00611] Updated weights for policy 0, policy_version 19472 (0.0010) [2023-10-08 04:42:11,709][00611] Updated weights for policy 0, policy_version 19482 (0.0010) [2023-10-08 04:42:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 40009728. Throughput: 0: 1825.7, 1: 1847.8. Samples: 10009980. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 04:42:13,754][130385] Avg episode reward: [(0, '43.560'), (1, '38.650')] [2023-10-08 04:42:14,403][00612] Updated weights for policy 1, policy_version 19590 (0.0009) [2023-10-08 04:42:14,779][00612] Updated weights for policy 1, policy_version 19600 (0.0009) [2023-10-08 04:42:15,151][00612] Updated weights for policy 1, policy_version 19610 (0.0010) [2023-10-08 04:42:15,414][00611] Updated weights for policy 0, policy_version 19492 (0.0007) [2023-10-08 04:42:15,776][00611] Updated weights for policy 0, policy_version 19502 (0.0008) [2023-10-08 04:42:16,161][00611] Updated weights for policy 0, policy_version 19512 (0.0009) [2023-10-08 04:42:18,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 40075264. Throughput: 0: 1825.9, 1: 1848.8. Samples: 10033144. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 04:42:18,755][130385] Avg episode reward: [(0, '42.570'), (1, '37.370')] [2023-10-08 04:42:18,757][00612] Updated weights for policy 1, policy_version 19620 (0.0008) [2023-10-08 04:42:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000019520_19988480.pth... [2023-10-08 04:42:18,798][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000017824_18251776.pth [2023-10-08 04:42:19,120][00612] Updated weights for policy 1, policy_version 19630 (0.0011) [2023-10-08 04:42:19,489][00612] Updated weights for policy 1, policy_version 19640 (0.0008) [2023-10-08 04:42:19,786][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000019648_20119552.pth... [2023-10-08 04:42:19,817][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000017920_18350080.pth [2023-10-08 04:42:19,828][00611] Updated weights for policy 0, policy_version 19522 (0.0008) [2023-10-08 04:42:20,200][00611] Updated weights for policy 0, policy_version 19532 (0.0009) [2023-10-08 04:42:20,568][00611] Updated weights for policy 0, policy_version 19542 (0.0010) [2023-10-08 04:42:20,945][00611] Updated weights for policy 0, policy_version 19552 (0.0008) [2023-10-08 04:42:23,209][00612] Updated weights for policy 1, policy_version 19650 (0.0008) [2023-10-08 04:42:23,576][00612] Updated weights for policy 1, policy_version 19660 (0.0007) [2023-10-08 04:42:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 40140800. Throughput: 0: 1825.6, 1: 1850.4. Samples: 10042952. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 04:42:23,754][130385] Avg episode reward: [(0, '40.890'), (1, '37.030')] [2023-10-08 04:42:23,938][00612] Updated weights for policy 1, policy_version 19670 (0.0008) [2023-10-08 04:42:24,308][00612] Updated weights for policy 1, policy_version 19680 (0.0007) [2023-10-08 04:42:24,513][00611] Updated weights for policy 0, policy_version 19562 (0.0009) [2023-10-08 04:42:24,887][00611] Updated weights for policy 0, policy_version 19572 (0.0010) [2023-10-08 04:42:25,251][00611] Updated weights for policy 0, policy_version 19582 (0.0010) [2023-10-08 04:42:28,026][00612] Updated weights for policy 1, policy_version 19690 (0.0007) [2023-10-08 04:42:28,391][00612] Updated weights for policy 1, policy_version 19700 (0.0008) [2023-10-08 04:42:28,754][00612] Updated weights for policy 1, policy_version 19710 (0.0009) [2023-10-08 04:42:28,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 40206336. Throughput: 0: 1829.2, 1: 1855.7. Samples: 10066156. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 04:42:28,755][130385] Avg episode reward: [(0, '40.220'), (1, '37.150')] [2023-10-08 04:42:28,919][00611] Updated weights for policy 0, policy_version 19592 (0.0008) [2023-10-08 04:42:29,294][00611] Updated weights for policy 0, policy_version 19602 (0.0009) [2023-10-08 04:42:29,666][00611] Updated weights for policy 0, policy_version 19612 (0.0007) [2023-10-08 04:42:32,441][00612] Updated weights for policy 1, policy_version 19720 (0.0008) [2023-10-08 04:42:32,814][00612] Updated weights for policy 1, policy_version 19730 (0.0010) [2023-10-08 04:42:33,164][00611] Updated weights for policy 0, policy_version 19622 (0.0007) [2023-10-08 04:42:33,175][00612] Updated weights for policy 1, policy_version 19740 (0.0007) [2023-10-08 04:42:33,548][00611] Updated weights for policy 0, policy_version 19632 (0.0007) [2023-10-08 04:42:33,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 40304640. Throughput: 0: 1821.9, 1: 1831.2. Samples: 10087340. Policy #0 lag: (min: 34.0, avg: 54.2, max: 56.0) [2023-10-08 04:42:33,754][130385] Avg episode reward: [(0, '43.200'), (1, '40.080')] [2023-10-08 04:42:33,921][00611] Updated weights for policy 0, policy_version 19642 (0.0008) [2023-10-08 04:42:36,793][00612] Updated weights for policy 1, policy_version 19750 (0.0007) [2023-10-08 04:42:37,166][00612] Updated weights for policy 1, policy_version 19760 (0.0008) [2023-10-08 04:42:37,513][00611] Updated weights for policy 0, policy_version 19652 (0.0008) [2023-10-08 04:42:37,530][00612] Updated weights for policy 1, policy_version 19770 (0.0008) [2023-10-08 04:42:37,877][00611] Updated weights for policy 0, policy_version 19662 (0.0007) [2023-10-08 04:42:38,246][00611] Updated weights for policy 0, policy_version 19672 (0.0007) [2023-10-08 04:42:38,754][130385] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 40402944. Throughput: 0: 1831.0, 1: 1846.0. Samples: 10098992. Policy #0 lag: (min: 34.0, avg: 54.2, max: 56.0) [2023-10-08 04:42:38,754][130385] Avg episode reward: [(0, '43.740'), (1, '38.750')] [2023-10-08 04:42:41,247][00612] Updated weights for policy 1, policy_version 19780 (0.0008) [2023-10-08 04:42:41,619][00612] Updated weights for policy 1, policy_version 19790 (0.0011) [2023-10-08 04:42:41,967][00611] Updated weights for policy 0, policy_version 19682 (0.0009) [2023-10-08 04:42:41,980][00612] Updated weights for policy 1, policy_version 19800 (0.0010) [2023-10-08 04:42:42,336][00611] Updated weights for policy 0, policy_version 19692 (0.0008) [2023-10-08 04:42:42,717][00611] Updated weights for policy 0, policy_version 19702 (0.0009) [2023-10-08 04:42:43,089][00611] Updated weights for policy 0, policy_version 19712 (0.0011) [2023-10-08 04:42:43,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 40468480. Throughput: 0: 1824.8, 1: 1833.7. Samples: 10120422. Policy #0 lag: (min: 34.0, avg: 54.2, max: 56.0) [2023-10-08 04:42:43,754][130385] Avg episode reward: [(0, '41.080'), (1, '37.550')] [2023-10-08 04:42:45,703][00612] Updated weights for policy 1, policy_version 19810 (0.0008) [2023-10-08 04:42:46,070][00612] Updated weights for policy 1, policy_version 19820 (0.0007) [2023-10-08 04:42:46,445][00612] Updated weights for policy 1, policy_version 19830 (0.0009) [2023-10-08 04:42:46,810][00612] Updated weights for policy 1, policy_version 19840 (0.0009) [2023-10-08 04:42:46,849][00611] Updated weights for policy 0, policy_version 19722 (0.0009) [2023-10-08 04:42:47,218][00611] Updated weights for policy 0, policy_version 19732 (0.0009) [2023-10-08 04:42:47,584][00611] Updated weights for policy 0, policy_version 19742 (0.0012) [2023-10-08 04:42:48,754][130385] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 40534016. Throughput: 0: 1834.0, 1: 1837.7. Samples: 10141780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:42:48,755][130385] Avg episode reward: [(0, '40.270'), (1, '38.740')] [2023-10-08 04:42:50,405][00612] Updated weights for policy 1, policy_version 19850 (0.0008) [2023-10-08 04:42:50,779][00612] Updated weights for policy 1, policy_version 19860 (0.0007) [2023-10-08 04:42:51,149][00612] Updated weights for policy 1, policy_version 19870 (0.0007) [2023-10-08 04:42:51,283][00611] Updated weights for policy 0, policy_version 19752 (0.0009) [2023-10-08 04:42:51,655][00611] Updated weights for policy 0, policy_version 19762 (0.0010) [2023-10-08 04:42:52,035][00611] Updated weights for policy 0, policy_version 19772 (0.0011) [2023-10-08 04:42:53,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 40599552. Throughput: 0: 1822.4, 1: 1825.0. Samples: 10153108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:42:53,754][130385] Avg episode reward: [(0, '40.820'), (1, '39.160')] [2023-10-08 04:42:54,805][00612] Updated weights for policy 1, policy_version 19880 (0.0008) [2023-10-08 04:42:55,171][00612] Updated weights for policy 1, policy_version 19890 (0.0008) [2023-10-08 04:42:55,541][00612] Updated weights for policy 1, policy_version 19900 (0.0013) [2023-10-08 04:42:55,725][00611] Updated weights for policy 0, policy_version 19782 (0.0009) [2023-10-08 04:42:56,098][00611] Updated weights for policy 0, policy_version 19792 (0.0010) [2023-10-08 04:42:56,476][00611] Updated weights for policy 0, policy_version 19802 (0.0007) [2023-10-08 04:42:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 40665088. Throughput: 0: 1829.5, 1: 1831.2. Samples: 10174714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:42:58,755][130385] Avg episode reward: [(0, '40.550'), (1, '40.670')] [2023-10-08 04:42:59,191][00612] Updated weights for policy 1, policy_version 19910 (0.0008) [2023-10-08 04:42:59,578][00612] Updated weights for policy 1, policy_version 19920 (0.0009) [2023-10-08 04:42:59,944][00612] Updated weights for policy 1, policy_version 19930 (0.0007) [2023-10-08 04:43:00,115][00611] Updated weights for policy 0, policy_version 19812 (0.0008) [2023-10-08 04:43:00,474][00611] Updated weights for policy 0, policy_version 19822 (0.0011) [2023-10-08 04:43:00,841][00611] Updated weights for policy 0, policy_version 19832 (0.0010) [2023-10-08 04:43:03,652][00612] Updated weights for policy 1, policy_version 19940 (0.0008) [2023-10-08 04:43:03,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 40730624. Throughput: 0: 1830.9, 1: 1825.7. Samples: 10197690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:43:03,755][130385] Avg episode reward: [(0, '42.280'), (1, '44.220')] [2023-10-08 04:43:04,027][00612] Updated weights for policy 1, policy_version 19950 (0.0008) [2023-10-08 04:43:04,342][00611] Updated weights for policy 0, policy_version 19842 (0.0009) [2023-10-08 04:43:04,387][00612] Updated weights for policy 1, policy_version 19960 (0.0007) [2023-10-08 04:43:04,673][00425] Saving new best policy, reward=44.220! [2023-10-08 04:43:04,716][00611] Updated weights for policy 0, policy_version 19852 (0.0010) [2023-10-08 04:43:05,094][00611] Updated weights for policy 0, policy_version 19862 (0.0007) [2023-10-08 04:43:05,460][00611] Updated weights for policy 0, policy_version 19872 (0.0008) [2023-10-08 04:43:08,131][00612] Updated weights for policy 1, policy_version 19970 (0.0008) [2023-10-08 04:43:08,511][00612] Updated weights for policy 1, policy_version 19980 (0.0007) [2023-10-08 04:43:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 40796160. Throughput: 0: 1832.3, 1: 1829.2. Samples: 10207720. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-08 04:43:08,754][130385] Avg episode reward: [(0, '39.770'), (1, '43.630')] [2023-10-08 04:43:08,878][00612] Updated weights for policy 1, policy_version 19990 (0.0008) [2023-10-08 04:43:09,060][00611] Updated weights for policy 0, policy_version 19882 (0.0009) [2023-10-08 04:43:09,263][00612] Updated weights for policy 1, policy_version 20000 (0.0008) [2023-10-08 04:43:09,428][00611] Updated weights for policy 0, policy_version 19892 (0.0008) [2023-10-08 04:43:09,813][00611] Updated weights for policy 0, policy_version 19902 (0.0010) [2023-10-08 04:43:12,840][00612] Updated weights for policy 1, policy_version 20010 (0.0008) [2023-10-08 04:43:13,211][00612] Updated weights for policy 1, policy_version 20020 (0.0007) [2023-10-08 04:43:13,293][00611] Updated weights for policy 0, policy_version 19912 (0.0007) [2023-10-08 04:43:13,581][00612] Updated weights for policy 1, policy_version 20030 (0.0007) [2023-10-08 04:43:13,672][00611] Updated weights for policy 0, policy_version 19922 (0.0007) [2023-10-08 04:43:13,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 40894464. Throughput: 0: 1837.7, 1: 1831.5. Samples: 10231266. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-08 04:43:13,754][130385] Avg episode reward: [(0, '39.380'), (1, '41.260')] [2023-10-08 04:43:14,042][00611] Updated weights for policy 0, policy_version 19932 (0.0007) [2023-10-08 04:43:17,206][00612] Updated weights for policy 1, policy_version 20040 (0.0010) [2023-10-08 04:43:17,566][00612] Updated weights for policy 1, policy_version 20050 (0.0010) [2023-10-08 04:43:17,731][00611] Updated weights for policy 0, policy_version 19942 (0.0007) [2023-10-08 04:43:17,945][00612] Updated weights for policy 1, policy_version 20060 (0.0008) [2023-10-08 04:43:18,099][00611] Updated weights for policy 0, policy_version 19952 (0.0008) [2023-10-08 04:43:18,470][00611] Updated weights for policy 0, policy_version 19962 (0.0007) [2023-10-08 04:43:18,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 40992768. Throughput: 0: 1830.9, 1: 1824.4. Samples: 10251830. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-08 04:43:18,754][130385] Avg episode reward: [(0, '39.290'), (1, '46.140')] [2023-10-08 04:43:18,763][00425] Saving new best policy, reward=46.140! [2023-10-08 04:43:21,599][00612] Updated weights for policy 1, policy_version 20070 (0.0009) [2023-10-08 04:43:21,975][00612] Updated weights for policy 1, policy_version 20080 (0.0010) [2023-10-08 04:43:22,120][00611] Updated weights for policy 0, policy_version 19972 (0.0010) [2023-10-08 04:43:22,343][00612] Updated weights for policy 1, policy_version 20090 (0.0008) [2023-10-08 04:43:22,490][00611] Updated weights for policy 0, policy_version 19982 (0.0007) [2023-10-08 04:43:22,860][00611] Updated weights for policy 0, policy_version 19992 (0.0011) [2023-10-08 04:43:23,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 41058304. Throughput: 0: 1837.6, 1: 1825.6. Samples: 10263836. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) [2023-10-08 04:43:23,755][130385] Avg episode reward: [(0, '40.460'), (1, '43.990')] [2023-10-08 04:43:26,017][00612] Updated weights for policy 1, policy_version 20100 (0.0008) [2023-10-08 04:43:26,390][00612] Updated weights for policy 1, policy_version 20110 (0.0008) [2023-10-08 04:43:26,635][00611] Updated weights for policy 0, policy_version 20002 (0.0010) [2023-10-08 04:43:26,759][00612] Updated weights for policy 1, policy_version 20120 (0.0007) [2023-10-08 04:43:26,990][00611] Updated weights for policy 0, policy_version 20012 (0.0007) [2023-10-08 04:43:27,364][00611] Updated weights for policy 0, policy_version 20022 (0.0010) [2023-10-08 04:43:27,736][00611] Updated weights for policy 0, policy_version 20032 (0.0009) [2023-10-08 04:43:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 41123840. Throughput: 0: 1823.4, 1: 1820.8. Samples: 10284408. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) [2023-10-08 04:43:28,754][130385] Avg episode reward: [(0, '40.560'), (1, '45.100')] [2023-10-08 04:43:30,394][00612] Updated weights for policy 1, policy_version 20130 (0.0007) [2023-10-08 04:43:30,764][00612] Updated weights for policy 1, policy_version 20140 (0.0011) [2023-10-08 04:43:31,125][00612] Updated weights for policy 1, policy_version 20150 (0.0008) [2023-10-08 04:43:31,416][00611] Updated weights for policy 0, policy_version 20042 (0.0009) [2023-10-08 04:43:31,494][00612] Updated weights for policy 1, policy_version 20160 (0.0008) [2023-10-08 04:43:31,789][00611] Updated weights for policy 0, policy_version 20052 (0.0009) [2023-10-08 04:43:32,161][00611] Updated weights for policy 0, policy_version 20062 (0.0008) [2023-10-08 04:43:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 41189376. Throughput: 0: 1834.7, 1: 1831.9. Samples: 10306778. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) [2023-10-08 04:43:33,755][130385] Avg episode reward: [(0, '39.890'), (1, '47.160')] [2023-10-08 04:43:33,764][00425] Saving new best policy, reward=47.160! [2023-10-08 04:43:35,109][00612] Updated weights for policy 1, policy_version 20170 (0.0008) [2023-10-08 04:43:35,485][00612] Updated weights for policy 1, policy_version 20180 (0.0007) [2023-10-08 04:43:35,852][00612] Updated weights for policy 1, policy_version 20190 (0.0008) [2023-10-08 04:43:35,986][00611] Updated weights for policy 0, policy_version 20072 (0.0009) [2023-10-08 04:43:36,367][00611] Updated weights for policy 0, policy_version 20082 (0.0010) [2023-10-08 04:43:36,746][00611] Updated weights for policy 0, policy_version 20092 (0.0008) [2023-10-08 04:43:38,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 41254912. Throughput: 0: 1825.1, 1: 1829.9. Samples: 10317582. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) [2023-10-08 04:43:38,755][130385] Avg episode reward: [(0, '43.450'), (1, '39.690')] [2023-10-08 04:43:39,400][00612] Updated weights for policy 1, policy_version 20200 (0.0009) [2023-10-08 04:43:39,762][00612] Updated weights for policy 1, policy_version 20210 (0.0008) [2023-10-08 04:43:40,131][00612] Updated weights for policy 1, policy_version 20220 (0.0008) [2023-10-08 04:43:40,404][00611] Updated weights for policy 0, policy_version 20102 (0.0008) [2023-10-08 04:43:40,781][00611] Updated weights for policy 0, policy_version 20112 (0.0009) [2023-10-08 04:43:41,163][00611] Updated weights for policy 0, policy_version 20122 (0.0011) [2023-10-08 04:43:43,744][00612] Updated weights for policy 1, policy_version 20230 (0.0009) [2023-10-08 04:43:43,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 41320448. Throughput: 0: 1837.3, 1: 1837.8. Samples: 10340090. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-08 04:43:43,754][130385] Avg episode reward: [(0, '43.130'), (1, '42.020')] [2023-10-08 04:43:44,108][00612] Updated weights for policy 1, policy_version 20240 (0.0009) [2023-10-08 04:43:44,473][00612] Updated weights for policy 1, policy_version 20250 (0.0011) [2023-10-08 04:43:44,845][00611] Updated weights for policy 0, policy_version 20132 (0.0011) [2023-10-08 04:43:45,221][00611] Updated weights for policy 0, policy_version 20142 (0.0009) [2023-10-08 04:43:45,587][00611] Updated weights for policy 0, policy_version 20152 (0.0008) [2023-10-08 04:43:48,137][00612] Updated weights for policy 1, policy_version 20260 (0.0009) [2023-10-08 04:43:48,494][00612] Updated weights for policy 1, policy_version 20270 (0.0010) [2023-10-08 04:43:48,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 41385984. Throughput: 0: 1834.8, 1: 1835.4. Samples: 10362850. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-08 04:43:48,755][130385] Avg episode reward: [(0, '45.250'), (1, '42.240')] [2023-10-08 04:43:48,763][00365] Saving new best policy, reward=45.250! [2023-10-08 04:43:48,863][00612] Updated weights for policy 1, policy_version 20280 (0.0011) [2023-10-08 04:43:49,362][00611] Updated weights for policy 0, policy_version 20162 (0.0007) [2023-10-08 04:43:49,730][00611] Updated weights for policy 0, policy_version 20172 (0.0008) [2023-10-08 04:43:50,098][00611] Updated weights for policy 0, policy_version 20182 (0.0007) [2023-10-08 04:43:50,460][00611] Updated weights for policy 0, policy_version 20192 (0.0007) [2023-10-08 04:43:52,620][00612] Updated weights for policy 1, policy_version 20290 (0.0009) [2023-10-08 04:43:52,980][00612] Updated weights for policy 1, policy_version 20300 (0.0007) [2023-10-08 04:43:53,355][00612] Updated weights for policy 1, policy_version 20310 (0.0007) [2023-10-08 04:43:53,713][00612] Updated weights for policy 1, policy_version 20320 (0.0010) [2023-10-08 04:43:53,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 41484288. Throughput: 0: 1834.9, 1: 1841.7. Samples: 10373170. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-08 04:43:53,755][130385] Avg episode reward: [(0, '45.230'), (1, '39.600')] [2023-10-08 04:43:54,052][00611] Updated weights for policy 0, policy_version 20202 (0.0009) [2023-10-08 04:43:54,431][00611] Updated weights for policy 0, policy_version 20212 (0.0007) [2023-10-08 04:43:54,798][00611] Updated weights for policy 0, policy_version 20222 (0.0010) [2023-10-08 04:43:57,404][00612] Updated weights for policy 1, policy_version 20330 (0.0008) [2023-10-08 04:43:57,778][00612] Updated weights for policy 1, policy_version 20340 (0.0008) [2023-10-08 04:43:58,152][00612] Updated weights for policy 1, policy_version 20350 (0.0007) [2023-10-08 04:43:58,383][00611] Updated weights for policy 0, policy_version 20232 (0.0008) [2023-10-08 04:43:58,749][00611] Updated weights for policy 0, policy_version 20242 (0.0009) [2023-10-08 04:43:58,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 41549824. Throughput: 0: 1830.4, 1: 1827.7. Samples: 10395880. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-08 04:43:58,754][130385] Avg episode reward: [(0, '43.160'), (1, '35.870')] [2023-10-08 04:43:59,122][00611] Updated weights for policy 0, policy_version 20252 (0.0009) [2023-10-08 04:44:01,865][00612] Updated weights for policy 1, policy_version 20360 (0.0009) [2023-10-08 04:44:02,238][00612] Updated weights for policy 1, policy_version 20370 (0.0010) [2023-10-08 04:44:02,604][00612] Updated weights for policy 1, policy_version 20380 (0.0010) [2023-10-08 04:44:02,901][00611] Updated weights for policy 0, policy_version 20262 (0.0009) [2023-10-08 04:44:03,273][00611] Updated weights for policy 0, policy_version 20272 (0.0007) [2023-10-08 04:44:03,647][00611] Updated weights for policy 0, policy_version 20282 (0.0007) [2023-10-08 04:44:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 41615360. Throughput: 0: 1829.7, 1: 1840.9. Samples: 10417006. Policy #0 lag: (min: 31.0, avg: 46.6, max: 63.0) [2023-10-08 04:44:03,754][130385] Avg episode reward: [(0, '43.970'), (1, '39.130')] [2023-10-08 04:44:06,121][00612] Updated weights for policy 1, policy_version 20390 (0.0007) [2023-10-08 04:44:06,493][00612] Updated weights for policy 1, policy_version 20400 (0.0009) [2023-10-08 04:44:06,852][00612] Updated weights for policy 1, policy_version 20410 (0.0008) [2023-10-08 04:44:07,138][00611] Updated weights for policy 0, policy_version 20292 (0.0008) [2023-10-08 04:44:07,505][00611] Updated weights for policy 0, policy_version 20302 (0.0007) [2023-10-08 04:44:07,877][00611] Updated weights for policy 0, policy_version 20312 (0.0010) [2023-10-08 04:44:08,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 41713664. Throughput: 0: 1830.1, 1: 1841.1. Samples: 10429040. Policy #0 lag: (min: 31.0, avg: 46.6, max: 63.0) [2023-10-08 04:44:08,754][130385] Avg episode reward: [(0, '42.760'), (1, '38.640')] [2023-10-08 04:44:10,591][00612] Updated weights for policy 1, policy_version 20420 (0.0007) [2023-10-08 04:44:10,968][00612] Updated weights for policy 1, policy_version 20430 (0.0008) [2023-10-08 04:44:11,329][00612] Updated weights for policy 1, policy_version 20440 (0.0007) [2023-10-08 04:44:11,532][00611] Updated weights for policy 0, policy_version 20322 (0.0008) [2023-10-08 04:44:11,906][00611] Updated weights for policy 0, policy_version 20332 (0.0009) [2023-10-08 04:44:12,281][00611] Updated weights for policy 0, policy_version 20342 (0.0008) [2023-10-08 04:44:12,651][00611] Updated weights for policy 0, policy_version 20352 (0.0009) [2023-10-08 04:44:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 41779200. Throughput: 0: 1836.6, 1: 1847.6. Samples: 10450196. Policy #0 lag: (min: 31.0, avg: 46.6, max: 63.0) [2023-10-08 04:44:13,754][130385] Avg episode reward: [(0, '42.980'), (1, '40.570')] [2023-10-08 04:44:14,795][00612] Updated weights for policy 1, policy_version 20450 (0.0007) [2023-10-08 04:44:15,162][00612] Updated weights for policy 1, policy_version 20460 (0.0008) [2023-10-08 04:44:15,538][00612] Updated weights for policy 1, policy_version 20470 (0.0009) [2023-10-08 04:44:15,903][00612] Updated weights for policy 1, policy_version 20480 (0.0007) [2023-10-08 04:44:16,311][00611] Updated weights for policy 0, policy_version 20362 (0.0008) [2023-10-08 04:44:16,681][00611] Updated weights for policy 0, policy_version 20372 (0.0007) [2023-10-08 04:44:17,052][00611] Updated weights for policy 0, policy_version 20382 (0.0007) [2023-10-08 04:44:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 41844736. Throughput: 0: 1840.6, 1: 1849.6. Samples: 10472838. Policy #0 lag: (min: 18.0, avg: 19.0, max: 40.0) [2023-10-08 04:44:18,755][130385] Avg episode reward: [(0, '40.540'), (1, '40.300')] [2023-10-08 04:44:18,764][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000020480_20971520.pth... [2023-10-08 04:44:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000020384_20873216.pth... [2023-10-08 04:44:18,794][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000018688_19136512.pth [2023-10-08 04:44:18,802][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000018784_19234816.pth [2023-10-08 04:44:19,564][00612] Updated weights for policy 1, policy_version 20490 (0.0008) [2023-10-08 04:44:19,927][00612] Updated weights for policy 1, policy_version 20500 (0.0008) [2023-10-08 04:44:20,296][00612] Updated weights for policy 1, policy_version 20510 (0.0009) [2023-10-08 04:44:20,616][00611] Updated weights for policy 0, policy_version 20392 (0.0010) [2023-10-08 04:44:20,993][00611] Updated weights for policy 0, policy_version 20402 (0.0008) [2023-10-08 04:44:21,367][00611] Updated weights for policy 0, policy_version 20412 (0.0008) [2023-10-08 04:44:23,741][00612] Updated weights for policy 1, policy_version 20520 (0.0008) [2023-10-08 04:44:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 41910272. Throughput: 0: 1831.3, 1: 1854.0. Samples: 10483420. Policy #0 lag: (min: 18.0, avg: 19.0, max: 40.0) [2023-10-08 04:44:23,755][130385] Avg episode reward: [(0, '39.110'), (1, '40.270')] [2023-10-08 04:44:24,115][00612] Updated weights for policy 1, policy_version 20530 (0.0009) [2023-10-08 04:44:24,488][00612] Updated weights for policy 1, policy_version 20540 (0.0008) [2023-10-08 04:44:25,041][00611] Updated weights for policy 0, policy_version 20422 (0.0008) [2023-10-08 04:44:25,406][00611] Updated weights for policy 0, policy_version 20432 (0.0008) [2023-10-08 04:44:25,780][00611] Updated weights for policy 0, policy_version 20442 (0.0008) [2023-10-08 04:44:28,114][00612] Updated weights for policy 1, policy_version 20550 (0.0008) [2023-10-08 04:44:28,485][00612] Updated weights for policy 1, policy_version 20560 (0.0007) [2023-10-08 04:44:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 41975808. Throughput: 0: 1836.3, 1: 1854.1. Samples: 10506160. Policy #0 lag: (min: 18.0, avg: 19.0, max: 40.0) [2023-10-08 04:44:28,755][130385] Avg episode reward: [(0, '40.160'), (1, '43.090')] [2023-10-08 04:44:28,849][00612] Updated weights for policy 1, policy_version 20570 (0.0009) [2023-10-08 04:44:29,282][00611] Updated weights for policy 0, policy_version 20452 (0.0008) [2023-10-08 04:44:29,659][00611] Updated weights for policy 0, policy_version 20462 (0.0011) [2023-10-08 04:44:30,025][00611] Updated weights for policy 0, policy_version 20472 (0.0011) [2023-10-08 04:44:32,493][00612] Updated weights for policy 1, policy_version 20580 (0.0008) [2023-10-08 04:44:32,864][00612] Updated weights for policy 1, policy_version 20590 (0.0009) [2023-10-08 04:44:33,222][00612] Updated weights for policy 1, policy_version 20600 (0.0008) [2023-10-08 04:44:33,754][00611] Updated weights for policy 0, policy_version 20482 (0.0008) [2023-10-08 04:44:33,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 42074112. Throughput: 0: 1836.7, 1: 1833.2. Samples: 10527994. Policy #0 lag: (min: 18.0, avg: 19.0, max: 40.0) [2023-10-08 04:44:33,754][130385] Avg episode reward: [(0, '42.690'), (1, '42.040')] [2023-10-08 04:44:34,138][00611] Updated weights for policy 0, policy_version 20492 (0.0010) [2023-10-08 04:44:34,505][00611] Updated weights for policy 0, policy_version 20502 (0.0010) [2023-10-08 04:44:34,882][00611] Updated weights for policy 0, policy_version 20512 (0.0010) [2023-10-08 04:44:36,928][00612] Updated weights for policy 1, policy_version 20610 (0.0007) [2023-10-08 04:44:37,302][00612] Updated weights for policy 1, policy_version 20620 (0.0009) [2023-10-08 04:44:37,671][00612] Updated weights for policy 1, policy_version 20630 (0.0011) [2023-10-08 04:44:38,038][00612] Updated weights for policy 1, policy_version 20640 (0.0009) [2023-10-08 04:44:38,478][00611] Updated weights for policy 0, policy_version 20522 (0.0008) [2023-10-08 04:44:38,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 42139648. Throughput: 0: 1836.9, 1: 1853.2. Samples: 10539226. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 04:44:38,754][130385] Avg episode reward: [(0, '40.400'), (1, '44.500')] [2023-10-08 04:44:38,846][00611] Updated weights for policy 0, policy_version 20532 (0.0010) [2023-10-08 04:44:39,208][00611] Updated weights for policy 0, policy_version 20542 (0.0010) [2023-10-08 04:44:41,644][00612] Updated weights for policy 1, policy_version 20650 (0.0007) [2023-10-08 04:44:42,008][00612] Updated weights for policy 1, policy_version 20660 (0.0009) [2023-10-08 04:44:42,378][00612] Updated weights for policy 1, policy_version 20670 (0.0009) [2023-10-08 04:44:42,924][00611] Updated weights for policy 0, policy_version 20552 (0.0009) [2023-10-08 04:44:43,297][00611] Updated weights for policy 0, policy_version 20562 (0.0007) [2023-10-08 04:44:43,671][00611] Updated weights for policy 0, policy_version 20572 (0.0007) [2023-10-08 04:44:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 42205184. Throughput: 0: 1838.9, 1: 1839.2. Samples: 10561394. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 04:44:43,754][130385] Avg episode reward: [(0, '42.380'), (1, '43.550')] [2023-10-08 04:44:45,988][00612] Updated weights for policy 1, policy_version 20680 (0.0011) [2023-10-08 04:44:46,364][00612] Updated weights for policy 1, policy_version 20690 (0.0008) [2023-10-08 04:44:46,729][00612] Updated weights for policy 1, policy_version 20700 (0.0010) [2023-10-08 04:44:47,224][00611] Updated weights for policy 0, policy_version 20582 (0.0009) [2023-10-08 04:44:47,603][00611] Updated weights for policy 0, policy_version 20592 (0.0009) [2023-10-08 04:44:47,974][00611] Updated weights for policy 0, policy_version 20602 (0.0007) [2023-10-08 04:44:48,754][130385] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 42303488. Throughput: 0: 1820.7, 1: 1864.3. Samples: 10582832. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 04:44:48,755][130385] Avg episode reward: [(0, '42.360'), (1, '40.040')] [2023-10-08 04:44:50,382][00612] Updated weights for policy 1, policy_version 20710 (0.0009) [2023-10-08 04:44:50,768][00612] Updated weights for policy 1, policy_version 20720 (0.0011) [2023-10-08 04:44:51,141][00612] Updated weights for policy 1, policy_version 20730 (0.0011) [2023-10-08 04:44:51,657][00611] Updated weights for policy 0, policy_version 20612 (0.0009) [2023-10-08 04:44:52,019][00611] Updated weights for policy 0, policy_version 20622 (0.0011) [2023-10-08 04:44:52,399][00611] Updated weights for policy 0, policy_version 20632 (0.0010) [2023-10-08 04:44:53,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 42369024. Throughput: 0: 1835.3, 1: 1835.4. Samples: 10594224. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 04:44:53,755][130385] Avg episode reward: [(0, '40.420'), (1, '40.110')] [2023-10-08 04:44:54,818][00612] Updated weights for policy 1, policy_version 20740 (0.0009) [2023-10-08 04:44:55,194][00612] Updated weights for policy 1, policy_version 20750 (0.0009) [2023-10-08 04:44:55,556][00612] Updated weights for policy 1, policy_version 20760 (0.0008) [2023-10-08 04:44:56,099][00611] Updated weights for policy 0, policy_version 20642 (0.0009) [2023-10-08 04:44:56,470][00611] Updated weights for policy 0, policy_version 20652 (0.0009) [2023-10-08 04:44:56,845][00611] Updated weights for policy 0, policy_version 20662 (0.0007) [2023-10-08 04:44:57,218][00611] Updated weights for policy 0, policy_version 20672 (0.0008) [2023-10-08 04:44:58,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 42434560. Throughput: 0: 1817.9, 1: 1858.6. Samples: 10615638. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 04:44:58,755][130385] Avg episode reward: [(0, '41.820'), (1, '40.050')] [2023-10-08 04:44:59,182][00612] Updated weights for policy 1, policy_version 20770 (0.0009) [2023-10-08 04:44:59,554][00612] Updated weights for policy 1, policy_version 20780 (0.0007) [2023-10-08 04:44:59,931][00612] Updated weights for policy 1, policy_version 20790 (0.0008) [2023-10-08 04:45:00,296][00612] Updated weights for policy 1, policy_version 20800 (0.0009) [2023-10-08 04:45:00,968][00611] Updated weights for policy 0, policy_version 20682 (0.0012) [2023-10-08 04:45:01,339][00611] Updated weights for policy 0, policy_version 20692 (0.0008) [2023-10-08 04:45:01,712][00611] Updated weights for policy 0, policy_version 20702 (0.0009) [2023-10-08 04:45:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 42500096. Throughput: 0: 1831.7, 1: 1855.6. Samples: 10638770. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 04:45:03,755][130385] Avg episode reward: [(0, '38.150'), (1, '38.160')] [2023-10-08 04:45:03,849][00612] Updated weights for policy 1, policy_version 20810 (0.0010) [2023-10-08 04:45:04,217][00612] Updated weights for policy 1, policy_version 20820 (0.0009) [2023-10-08 04:45:04,585][00612] Updated weights for policy 1, policy_version 20830 (0.0007) [2023-10-08 04:45:05,413][00611] Updated weights for policy 0, policy_version 20712 (0.0008) [2023-10-08 04:45:05,785][00611] Updated weights for policy 0, policy_version 20722 (0.0010) [2023-10-08 04:45:06,154][00611] Updated weights for policy 0, policy_version 20732 (0.0009) [2023-10-08 04:45:08,160][00612] Updated weights for policy 1, policy_version 20840 (0.0009) [2023-10-08 04:45:08,532][00612] Updated weights for policy 1, policy_version 20850 (0.0008) [2023-10-08 04:45:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 42565632. Throughput: 0: 1829.5, 1: 1848.1. Samples: 10648912. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 04:45:08,755][130385] Avg episode reward: [(0, '35.850'), (1, '41.230')] [2023-10-08 04:45:08,892][00612] Updated weights for policy 1, policy_version 20860 (0.0008) [2023-10-08 04:45:09,746][00611] Updated weights for policy 0, policy_version 20742 (0.0009) [2023-10-08 04:45:10,118][00611] Updated weights for policy 0, policy_version 20752 (0.0008) [2023-10-08 04:45:10,498][00611] Updated weights for policy 0, policy_version 20762 (0.0010) [2023-10-08 04:45:12,572][00612] Updated weights for policy 1, policy_version 20870 (0.0009) [2023-10-08 04:45:12,951][00612] Updated weights for policy 1, policy_version 20880 (0.0008) [2023-10-08 04:45:13,318][00612] Updated weights for policy 1, policy_version 20890 (0.0008) [2023-10-08 04:45:13,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 42663936. Throughput: 0: 1833.7, 1: 1844.7. Samples: 10671684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:45:13,754][130385] Avg episode reward: [(0, '39.590'), (1, '42.920')] [2023-10-08 04:45:14,068][00611] Updated weights for policy 0, policy_version 20772 (0.0009) [2023-10-08 04:45:14,438][00611] Updated weights for policy 0, policy_version 20782 (0.0008) [2023-10-08 04:45:14,804][00611] Updated weights for policy 0, policy_version 20792 (0.0009) [2023-10-08 04:45:16,903][00612] Updated weights for policy 1, policy_version 20900 (0.0009) [2023-10-08 04:45:17,280][00612] Updated weights for policy 1, policy_version 20910 (0.0008) [2023-10-08 04:45:17,642][00612] Updated weights for policy 1, policy_version 20920 (0.0007) [2023-10-08 04:45:18,572][00611] Updated weights for policy 0, policy_version 20802 (0.0008) [2023-10-08 04:45:18,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 42729472. Throughput: 0: 1838.7, 1: 1839.8. Samples: 10693528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:45:18,755][130385] Avg episode reward: [(0, '39.660'), (1, '40.700')] [2023-10-08 04:45:18,951][00611] Updated weights for policy 0, policy_version 20812 (0.0008) [2023-10-08 04:45:19,327][00611] Updated weights for policy 0, policy_version 20822 (0.0007) [2023-10-08 04:45:19,693][00611] Updated weights for policy 0, policy_version 20832 (0.0007) [2023-10-08 04:45:21,199][00612] Updated weights for policy 1, policy_version 20930 (0.0009) [2023-10-08 04:45:21,568][00612] Updated weights for policy 1, policy_version 20940 (0.0009) [2023-10-08 04:45:21,935][00612] Updated weights for policy 1, policy_version 20950 (0.0008) [2023-10-08 04:45:22,304][00612] Updated weights for policy 1, policy_version 20960 (0.0008) [2023-10-08 04:45:23,248][00611] Updated weights for policy 0, policy_version 20842 (0.0008) [2023-10-08 04:45:23,616][00611] Updated weights for policy 0, policy_version 20852 (0.0007) [2023-10-08 04:45:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 42795008. Throughput: 0: 1837.0, 1: 1852.1. Samples: 10705236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:45:23,755][130385] Avg episode reward: [(0, '40.780'), (1, '40.160')] [2023-10-08 04:45:23,993][00611] Updated weights for policy 0, policy_version 20862 (0.0009) [2023-10-08 04:45:26,061][00612] Updated weights for policy 1, policy_version 20970 (0.0008) [2023-10-08 04:45:26,428][00612] Updated weights for policy 1, policy_version 20980 (0.0010) [2023-10-08 04:45:26,797][00612] Updated weights for policy 1, policy_version 20990 (0.0010) [2023-10-08 04:45:27,482][00611] Updated weights for policy 0, policy_version 20872 (0.0009) [2023-10-08 04:45:27,850][00611] Updated weights for policy 0, policy_version 20882 (0.0007) [2023-10-08 04:45:28,226][00611] Updated weights for policy 0, policy_version 20892 (0.0007) [2023-10-08 04:45:28,754][130385] Fps is (10 sec: 16384.5, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 42893312. Throughput: 0: 1834.8, 1: 1844.0. Samples: 10726940. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-10-08 04:45:28,754][130385] Avg episode reward: [(0, '40.870'), (1, '40.750')] [2023-10-08 04:45:30,494][00612] Updated weights for policy 1, policy_version 21000 (0.0010) [2023-10-08 04:45:30,871][00612] Updated weights for policy 1, policy_version 21010 (0.0008) [2023-10-08 04:45:31,248][00612] Updated weights for policy 1, policy_version 21020 (0.0008) [2023-10-08 04:45:31,926][00611] Updated weights for policy 0, policy_version 20902 (0.0008) [2023-10-08 04:45:32,290][00611] Updated weights for policy 0, policy_version 20912 (0.0008) [2023-10-08 04:45:32,654][00611] Updated weights for policy 0, policy_version 20922 (0.0007) [2023-10-08 04:45:33,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 42958848. Throughput: 0: 1829.8, 1: 1848.7. Samples: 10748362. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-10-08 04:45:33,754][130385] Avg episode reward: [(0, '40.480'), (1, '41.450')] [2023-10-08 04:45:34,895][00612] Updated weights for policy 1, policy_version 21030 (0.0009) [2023-10-08 04:45:35,282][00612] Updated weights for policy 1, policy_version 21040 (0.0007) [2023-10-08 04:45:35,643][00612] Updated weights for policy 1, policy_version 21050 (0.0008) [2023-10-08 04:45:36,487][00611] Updated weights for policy 0, policy_version 20932 (0.0008) [2023-10-08 04:45:36,853][00611] Updated weights for policy 0, policy_version 20942 (0.0007) [2023-10-08 04:45:37,222][00611] Updated weights for policy 0, policy_version 20952 (0.0009) [2023-10-08 04:45:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 43024384. Throughput: 0: 1835.8, 1: 1841.6. Samples: 10759710. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-10-08 04:45:38,754][130385] Avg episode reward: [(0, '42.020'), (1, '41.450')] [2023-10-08 04:45:39,286][00612] Updated weights for policy 1, policy_version 21060 (0.0009) [2023-10-08 04:45:39,647][00612] Updated weights for policy 1, policy_version 21070 (0.0008) [2023-10-08 04:45:40,009][00612] Updated weights for policy 1, policy_version 21080 (0.0007) [2023-10-08 04:45:40,977][00611] Updated weights for policy 0, policy_version 20962 (0.0008) [2023-10-08 04:45:41,342][00611] Updated weights for policy 0, policy_version 20972 (0.0007) [2023-10-08 04:45:41,718][00611] Updated weights for policy 0, policy_version 20982 (0.0009) [2023-10-08 04:45:42,097][00611] Updated weights for policy 0, policy_version 20992 (0.0008) [2023-10-08 04:45:43,634][00612] Updated weights for policy 1, policy_version 21090 (0.0009) [2023-10-08 04:45:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 43089920. Throughput: 0: 1831.2, 1: 1848.8. Samples: 10781236. Policy #0 lag: (min: 3.0, avg: 7.3, max: 35.0) [2023-10-08 04:45:43,754][130385] Avg episode reward: [(0, '43.080'), (1, '41.780')] [2023-10-08 04:45:44,002][00612] Updated weights for policy 1, policy_version 21100 (0.0008) [2023-10-08 04:45:44,361][00612] Updated weights for policy 1, policy_version 21110 (0.0009) [2023-10-08 04:45:44,729][00612] Updated weights for policy 1, policy_version 21120 (0.0007) [2023-10-08 04:45:45,637][00611] Updated weights for policy 0, policy_version 21002 (0.0008) [2023-10-08 04:45:46,008][00611] Updated weights for policy 0, policy_version 21012 (0.0009) [2023-10-08 04:45:46,383][00611] Updated weights for policy 0, policy_version 21022 (0.0010) [2023-10-08 04:45:48,191][00612] Updated weights for policy 1, policy_version 21130 (0.0007) [2023-10-08 04:45:48,560][00612] Updated weights for policy 1, policy_version 21140 (0.0007) [2023-10-08 04:45:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 43155456. Throughput: 0: 1837.2, 1: 1840.8. Samples: 10804280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:45:48,754][130385] Avg episode reward: [(0, '44.920'), (1, '38.290')] [2023-10-08 04:45:48,931][00612] Updated weights for policy 1, policy_version 21150 (0.0009) [2023-10-08 04:45:50,019][00611] Updated weights for policy 0, policy_version 21032 (0.0009) [2023-10-08 04:45:50,392][00611] Updated weights for policy 0, policy_version 21042 (0.0010) [2023-10-08 04:45:50,770][00611] Updated weights for policy 0, policy_version 21052 (0.0008) [2023-10-08 04:45:52,491][00612] Updated weights for policy 1, policy_version 21160 (0.0008) [2023-10-08 04:45:52,856][00612] Updated weights for policy 1, policy_version 21170 (0.0009) [2023-10-08 04:45:53,233][00612] Updated weights for policy 1, policy_version 21180 (0.0008) [2023-10-08 04:45:53,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 43253760. Throughput: 0: 1828.2, 1: 1858.0. Samples: 10814790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:45:53,754][130385] Avg episode reward: [(0, '47.780'), (1, '38.990')] [2023-10-08 04:45:53,755][00365] Saving new best policy, reward=47.780! [2023-10-08 04:45:54,463][00611] Updated weights for policy 0, policy_version 21062 (0.0007) [2023-10-08 04:45:54,835][00611] Updated weights for policy 0, policy_version 21072 (0.0010) [2023-10-08 04:45:55,209][00611] Updated weights for policy 0, policy_version 21082 (0.0009) [2023-10-08 04:45:56,918][00612] Updated weights for policy 1, policy_version 21190 (0.0007) [2023-10-08 04:45:57,283][00612] Updated weights for policy 1, policy_version 21200 (0.0009) [2023-10-08 04:45:57,652][00612] Updated weights for policy 1, policy_version 21210 (0.0007) [2023-10-08 04:45:58,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 43319296. Throughput: 0: 1843.9, 1: 1843.0. Samples: 10837596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:45:58,754][130385] Avg episode reward: [(0, '45.430'), (1, '37.500')] [2023-10-08 04:45:58,823][00611] Updated weights for policy 0, policy_version 21092 (0.0008) [2023-10-08 04:45:59,209][00611] Updated weights for policy 0, policy_version 21102 (0.0009) [2023-10-08 04:45:59,574][00611] Updated weights for policy 0, policy_version 21112 (0.0008) [2023-10-08 04:46:01,382][00612] Updated weights for policy 1, policy_version 21220 (0.0007) [2023-10-08 04:46:01,745][00612] Updated weights for policy 1, policy_version 21230 (0.0010) [2023-10-08 04:46:02,124][00612] Updated weights for policy 1, policy_version 21240 (0.0010) [2023-10-08 04:46:03,043][00611] Updated weights for policy 0, policy_version 21122 (0.0008) [2023-10-08 04:46:03,411][00611] Updated weights for policy 0, policy_version 21132 (0.0009) [2023-10-08 04:46:03,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 43384832. Throughput: 0: 1841.6, 1: 1852.0. Samples: 10859738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:46:03,755][130385] Avg episode reward: [(0, '44.120'), (1, '35.170')] [2023-10-08 04:46:03,788][00611] Updated weights for policy 0, policy_version 21142 (0.0008) [2023-10-08 04:46:04,151][00611] Updated weights for policy 0, policy_version 21152 (0.0008) [2023-10-08 04:46:05,612][00612] Updated weights for policy 1, policy_version 21250 (0.0008) [2023-10-08 04:46:05,980][00612] Updated weights for policy 1, policy_version 21260 (0.0007) [2023-10-08 04:46:06,350][00612] Updated weights for policy 1, policy_version 21270 (0.0010) [2023-10-08 04:46:06,725][00612] Updated weights for policy 1, policy_version 21280 (0.0011) [2023-10-08 04:46:07,763][00611] Updated weights for policy 0, policy_version 21162 (0.0008) [2023-10-08 04:46:08,131][00611] Updated weights for policy 0, policy_version 21172 (0.0007) [2023-10-08 04:46:08,504][00611] Updated weights for policy 0, policy_version 21182 (0.0008) [2023-10-08 04:46:08,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 43483136. Throughput: 0: 1849.8, 1: 1829.8. Samples: 10870818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:46:08,754][130385] Avg episode reward: [(0, '42.400'), (1, '37.520')] [2023-10-08 04:46:10,445][00612] Updated weights for policy 1, policy_version 21290 (0.0009) [2023-10-08 04:46:10,813][00612] Updated weights for policy 1, policy_version 21300 (0.0008) [2023-10-08 04:46:11,179][00612] Updated weights for policy 1, policy_version 21310 (0.0007) [2023-10-08 04:46:12,117][00611] Updated weights for policy 0, policy_version 21192 (0.0008) [2023-10-08 04:46:12,493][00611] Updated weights for policy 0, policy_version 21202 (0.0008) [2023-10-08 04:46:12,875][00611] Updated weights for policy 0, policy_version 21212 (0.0008) [2023-10-08 04:46:13,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 43548672. Throughput: 0: 1842.6, 1: 1846.6. Samples: 10892954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:46:13,754][130385] Avg episode reward: [(0, '42.640'), (1, '40.330')] [2023-10-08 04:46:14,757][00612] Updated weights for policy 1, policy_version 21320 (0.0009) [2023-10-08 04:46:15,130][00612] Updated weights for policy 1, policy_version 21330 (0.0007) [2023-10-08 04:46:15,501][00612] Updated weights for policy 1, policy_version 21340 (0.0009) [2023-10-08 04:46:16,381][00611] Updated weights for policy 0, policy_version 21222 (0.0007) [2023-10-08 04:46:16,755][00611] Updated weights for policy 0, policy_version 21232 (0.0008) [2023-10-08 04:46:17,131][00611] Updated weights for policy 0, policy_version 21242 (0.0008) [2023-10-08 04:46:18,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 43614208. Throughput: 0: 1861.0, 1: 1852.3. Samples: 10915460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:46:18,755][130385] Avg episode reward: [(0, '44.120'), (1, '42.260')] [2023-10-08 04:46:18,764][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000021344_21856256.pth... [2023-10-08 04:46:18,765][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000021248_21757952.pth... [2023-10-08 04:46:18,799][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000019520_19988480.pth [2023-10-08 04:46:18,803][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000019648_20119552.pth [2023-10-08 04:46:19,146][00612] Updated weights for policy 1, policy_version 21350 (0.0008) [2023-10-08 04:46:19,521][00612] Updated weights for policy 1, policy_version 21360 (0.0008) [2023-10-08 04:46:19,896][00612] Updated weights for policy 1, policy_version 21370 (0.0008) [2023-10-08 04:46:20,702][00611] Updated weights for policy 0, policy_version 21252 (0.0008) [2023-10-08 04:46:21,059][00611] Updated weights for policy 0, policy_version 21262 (0.0011) [2023-10-08 04:46:21,428][00611] Updated weights for policy 0, policy_version 21272 (0.0008) [2023-10-08 04:46:23,647][00612] Updated weights for policy 1, policy_version 21380 (0.0009) [2023-10-08 04:46:23,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 43679744. Throughput: 0: 1842.7, 1: 1853.8. Samples: 10926052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:46:23,755][130385] Avg episode reward: [(0, '40.370'), (1, '40.230')] [2023-10-08 04:46:24,020][00612] Updated weights for policy 1, policy_version 21390 (0.0009) [2023-10-08 04:46:24,397][00612] Updated weights for policy 1, policy_version 21400 (0.0009) [2023-10-08 04:46:25,105][00611] Updated weights for policy 0, policy_version 21282 (0.0010) [2023-10-08 04:46:25,481][00611] Updated weights for policy 0, policy_version 21292 (0.0009) [2023-10-08 04:46:25,848][00611] Updated weights for policy 0, policy_version 21302 (0.0009) [2023-10-08 04:46:26,217][00611] Updated weights for policy 0, policy_version 21312 (0.0009) [2023-10-08 04:46:27,993][00612] Updated weights for policy 1, policy_version 21410 (0.0009) [2023-10-08 04:46:28,360][00612] Updated weights for policy 1, policy_version 21420 (0.0008) [2023-10-08 04:46:28,738][00612] Updated weights for policy 1, policy_version 21430 (0.0007) [2023-10-08 04:46:28,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 43745280. Throughput: 0: 1862.4, 1: 1845.4. Samples: 10948086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:46:28,754][130385] Avg episode reward: [(0, '40.750'), (1, '42.120')] [2023-10-08 04:46:29,105][00612] Updated weights for policy 1, policy_version 21440 (0.0008) [2023-10-08 04:46:29,829][00611] Updated weights for policy 0, policy_version 21322 (0.0009) [2023-10-08 04:46:30,209][00611] Updated weights for policy 0, policy_version 21332 (0.0008) [2023-10-08 04:46:30,587][00611] Updated weights for policy 0, policy_version 21342 (0.0010) [2023-10-08 04:46:32,467][00612] Updated weights for policy 1, policy_version 21450 (0.0010) [2023-10-08 04:46:32,842][00612] Updated weights for policy 1, policy_version 21460 (0.0008) [2023-10-08 04:46:33,218][00612] Updated weights for policy 1, policy_version 21470 (0.0007) [2023-10-08 04:46:33,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 43843584. Throughput: 0: 1860.0, 1: 1829.4. Samples: 10970304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:46:33,754][130385] Avg episode reward: [(0, '37.370'), (1, '40.310')] [2023-10-08 04:46:34,210][00611] Updated weights for policy 0, policy_version 21352 (0.0009) [2023-10-08 04:46:34,586][00611] Updated weights for policy 0, policy_version 21362 (0.0008) [2023-10-08 04:46:34,953][00611] Updated weights for policy 0, policy_version 21372 (0.0007) [2023-10-08 04:46:36,971][00612] Updated weights for policy 1, policy_version 21480 (0.0007) [2023-10-08 04:46:37,336][00612] Updated weights for policy 1, policy_version 21490 (0.0007) [2023-10-08 04:46:37,707][00612] Updated weights for policy 1, policy_version 21500 (0.0007) [2023-10-08 04:46:38,574][00611] Updated weights for policy 0, policy_version 21382 (0.0009) [2023-10-08 04:46:38,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 43909120. Throughput: 0: 1864.8, 1: 1846.9. Samples: 10981818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:46:38,755][130385] Avg episode reward: [(0, '40.420'), (1, '43.130')] [2023-10-08 04:46:38,951][00611] Updated weights for policy 0, policy_version 21392 (0.0009) [2023-10-08 04:46:39,317][00611] Updated weights for policy 0, policy_version 21402 (0.0011) [2023-10-08 04:46:41,382][00612] Updated weights for policy 1, policy_version 21510 (0.0008) [2023-10-08 04:46:41,756][00612] Updated weights for policy 1, policy_version 21520 (0.0009) [2023-10-08 04:46:42,128][00612] Updated weights for policy 1, policy_version 21530 (0.0009) [2023-10-08 04:46:42,919][00611] Updated weights for policy 0, policy_version 21412 (0.0010) [2023-10-08 04:46:43,309][00611] Updated weights for policy 0, policy_version 21422 (0.0008) [2023-10-08 04:46:43,684][00611] Updated weights for policy 0, policy_version 21432 (0.0009) [2023-10-08 04:46:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 43974656. Throughput: 0: 1859.2, 1: 1827.4. Samples: 11003490. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-08 04:46:43,754][130385] Avg episode reward: [(0, '40.240'), (1, '45.660')] [2023-10-08 04:46:45,748][00612] Updated weights for policy 1, policy_version 21540 (0.0008) [2023-10-08 04:46:46,121][00612] Updated weights for policy 1, policy_version 21550 (0.0008) [2023-10-08 04:46:46,487][00612] Updated weights for policy 1, policy_version 21560 (0.0007) [2023-10-08 04:46:47,310][00611] Updated weights for policy 0, policy_version 21442 (0.0009) [2023-10-08 04:46:47,686][00611] Updated weights for policy 0, policy_version 21452 (0.0009) [2023-10-08 04:46:48,058][00611] Updated weights for policy 0, policy_version 21462 (0.0009) [2023-10-08 04:46:48,428][00611] Updated weights for policy 0, policy_version 21472 (0.0008) [2023-10-08 04:46:48,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 44072960. Throughput: 0: 1834.0, 1: 1844.9. Samples: 11025288. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-08 04:46:48,755][130385] Avg episode reward: [(0, '40.320'), (1, '43.790')] [2023-10-08 04:46:50,017][00612] Updated weights for policy 1, policy_version 21570 (0.0008) [2023-10-08 04:46:50,382][00612] Updated weights for policy 1, policy_version 21580 (0.0008) [2023-10-08 04:46:50,755][00612] Updated weights for policy 1, policy_version 21590 (0.0008) [2023-10-08 04:46:51,120][00612] Updated weights for policy 1, policy_version 21600 (0.0008) [2023-10-08 04:46:51,993][00611] Updated weights for policy 0, policy_version 21482 (0.0008) [2023-10-08 04:46:52,365][00611] Updated weights for policy 0, policy_version 21492 (0.0009) [2023-10-08 04:46:52,731][00611] Updated weights for policy 0, policy_version 21502 (0.0008) [2023-10-08 04:46:53,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 44138496. Throughput: 0: 1854.9, 1: 1830.6. Samples: 11036666. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-08 04:46:53,755][130385] Avg episode reward: [(0, '37.260'), (1, '42.180')] [2023-10-08 04:46:54,668][00612] Updated weights for policy 1, policy_version 21610 (0.0008) [2023-10-08 04:46:55,034][00612] Updated weights for policy 1, policy_version 21620 (0.0008) [2023-10-08 04:46:55,397][00612] Updated weights for policy 1, policy_version 21630 (0.0010) [2023-10-08 04:46:56,438][00611] Updated weights for policy 0, policy_version 21512 (0.0007) [2023-10-08 04:46:56,817][00611] Updated weights for policy 0, policy_version 21522 (0.0007) [2023-10-08 04:46:57,192][00611] Updated weights for policy 0, policy_version 21532 (0.0008) [2023-10-08 04:46:58,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 44204032. Throughput: 0: 1829.4, 1: 1850.0. Samples: 11058526. Policy #0 lag: (min: 10.0, avg: 12.9, max: 42.0) [2023-10-08 04:46:58,755][130385] Avg episode reward: [(0, '37.720'), (1, '40.700')] [2023-10-08 04:46:58,981][00612] Updated weights for policy 1, policy_version 21640 (0.0008) [2023-10-08 04:46:59,361][00612] Updated weights for policy 1, policy_version 21650 (0.0010) [2023-10-08 04:46:59,721][00612] Updated weights for policy 1, policy_version 21660 (0.0009) [2023-10-08 04:47:00,771][00611] Updated weights for policy 0, policy_version 21542 (0.0010) [2023-10-08 04:47:01,140][00611] Updated weights for policy 0, policy_version 21552 (0.0009) [2023-10-08 04:47:01,514][00611] Updated weights for policy 0, policy_version 21562 (0.0007) [2023-10-08 04:47:03,570][00612] Updated weights for policy 1, policy_version 21670 (0.0007) [2023-10-08 04:47:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 44269568. Throughput: 0: 1841.1, 1: 1848.3. Samples: 11081482. Policy #0 lag: (min: 10.0, avg: 12.9, max: 42.0) [2023-10-08 04:47:03,754][130385] Avg episode reward: [(0, '38.590'), (1, '42.300')] [2023-10-08 04:47:03,935][00612] Updated weights for policy 1, policy_version 21680 (0.0008) [2023-10-08 04:47:04,313][00612] Updated weights for policy 1, policy_version 21690 (0.0007) [2023-10-08 04:47:05,117][00611] Updated weights for policy 0, policy_version 21572 (0.0009) [2023-10-08 04:47:05,478][00611] Updated weights for policy 0, policy_version 21582 (0.0010) [2023-10-08 04:47:05,862][00611] Updated weights for policy 0, policy_version 21592 (0.0010) [2023-10-08 04:47:08,027][00612] Updated weights for policy 1, policy_version 21700 (0.0008) [2023-10-08 04:47:08,423][00612] Updated weights for policy 1, policy_version 21710 (0.0009) [2023-10-08 04:47:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 44335104. Throughput: 0: 1830.4, 1: 1850.0. Samples: 11091670. Policy #0 lag: (min: 10.0, avg: 12.9, max: 42.0) [2023-10-08 04:47:08,754][130385] Avg episode reward: [(0, '38.920'), (1, '45.130')] [2023-10-08 04:47:08,793][00612] Updated weights for policy 1, policy_version 21720 (0.0008) [2023-10-08 04:47:09,586][00611] Updated weights for policy 0, policy_version 21602 (0.0009) [2023-10-08 04:47:09,955][00611] Updated weights for policy 0, policy_version 21612 (0.0008) [2023-10-08 04:47:10,332][00611] Updated weights for policy 0, policy_version 21622 (0.0009) [2023-10-08 04:47:10,712][00611] Updated weights for policy 0, policy_version 21632 (0.0008) [2023-10-08 04:47:12,384][00612] Updated weights for policy 1, policy_version 21730 (0.0010) [2023-10-08 04:47:12,760][00612] Updated weights for policy 1, policy_version 21740 (0.0008) [2023-10-08 04:47:13,124][00612] Updated weights for policy 1, policy_version 21750 (0.0008) [2023-10-08 04:47:13,486][00612] Updated weights for policy 1, policy_version 21760 (0.0008) [2023-10-08 04:47:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 44433408. Throughput: 0: 1843.2, 1: 1852.2. Samples: 11114376. Policy #0 lag: (min: 10.0, avg: 12.9, max: 42.0) [2023-10-08 04:47:13,755][130385] Avg episode reward: [(0, '44.320'), (1, '43.780')] [2023-10-08 04:47:14,366][00611] Updated weights for policy 0, policy_version 21642 (0.0010) [2023-10-08 04:47:14,740][00611] Updated weights for policy 0, policy_version 21652 (0.0009) [2023-10-08 04:47:15,104][00611] Updated weights for policy 0, policy_version 21662 (0.0009) [2023-10-08 04:47:17,018][00612] Updated weights for policy 1, policy_version 21770 (0.0007) [2023-10-08 04:47:17,389][00612] Updated weights for policy 1, policy_version 21780 (0.0007) [2023-10-08 04:47:17,761][00612] Updated weights for policy 1, policy_version 21790 (0.0007) [2023-10-08 04:47:18,698][00611] Updated weights for policy 0, policy_version 21672 (0.0008) [2023-10-08 04:47:18,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 44498944. Throughput: 0: 1842.5, 1: 1845.6. Samples: 11136268. Policy #0 lag: (min: 28.0, avg: 31.9, max: 60.0) [2023-10-08 04:47:18,755][130385] Avg episode reward: [(0, '43.570'), (1, '47.920')] [2023-10-08 04:47:18,768][00425] Saving new best policy, reward=47.920! [2023-10-08 04:47:19,063][00611] Updated weights for policy 0, policy_version 21682 (0.0007) [2023-10-08 04:47:19,440][00611] Updated weights for policy 0, policy_version 21692 (0.0007) [2023-10-08 04:47:21,365][00612] Updated weights for policy 1, policy_version 21800 (0.0009) [2023-10-08 04:47:21,728][00612] Updated weights for policy 1, policy_version 21810 (0.0007) [2023-10-08 04:47:22,095][00612] Updated weights for policy 1, policy_version 21820 (0.0007) [2023-10-08 04:47:23,146][00611] Updated weights for policy 0, policy_version 21702 (0.0007) [2023-10-08 04:47:23,522][00611] Updated weights for policy 0, policy_version 21712 (0.0007) [2023-10-08 04:47:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 44564480. Throughput: 0: 1835.8, 1: 1849.0. Samples: 11147634. Policy #0 lag: (min: 28.0, avg: 31.9, max: 60.0) [2023-10-08 04:47:23,758][130385] Avg episode reward: [(0, '44.270'), (1, '47.350')] [2023-10-08 04:47:23,886][00611] Updated weights for policy 0, policy_version 21722 (0.0008) [2023-10-08 04:47:25,728][00612] Updated weights for policy 1, policy_version 21830 (0.0007) [2023-10-08 04:47:26,098][00612] Updated weights for policy 1, policy_version 21840 (0.0009) [2023-10-08 04:47:26,478][00612] Updated weights for policy 1, policy_version 21850 (0.0009) [2023-10-08 04:47:27,573][00611] Updated weights for policy 0, policy_version 21732 (0.0008) [2023-10-08 04:47:27,953][00611] Updated weights for policy 0, policy_version 21742 (0.0007) [2023-10-08 04:47:28,325][00611] Updated weights for policy 0, policy_version 21752 (0.0008) [2023-10-08 04:47:28,754][130385] Fps is (10 sec: 16384.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 44662784. Throughput: 0: 1837.6, 1: 1851.3. Samples: 11169488. Policy #0 lag: (min: 28.0, avg: 31.9, max: 60.0) [2023-10-08 04:47:28,754][130385] Avg episode reward: [(0, '44.180'), (1, '48.300')] [2023-10-08 04:47:28,755][00425] Saving new best policy, reward=48.300! [2023-10-08 04:47:30,079][00612] Updated weights for policy 1, policy_version 21860 (0.0009) [2023-10-08 04:47:30,446][00612] Updated weights for policy 1, policy_version 21870 (0.0008) [2023-10-08 04:47:30,815][00612] Updated weights for policy 1, policy_version 21880 (0.0008) [2023-10-08 04:47:32,014][00611] Updated weights for policy 0, policy_version 21762 (0.0008) [2023-10-08 04:47:32,413][00611] Updated weights for policy 0, policy_version 21772 (0.0007) [2023-10-08 04:47:32,787][00611] Updated weights for policy 0, policy_version 21782 (0.0007) [2023-10-08 04:47:33,175][00611] Updated weights for policy 0, policy_version 21792 (0.0008) [2023-10-08 04:47:33,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 44728320. Throughput: 0: 1822.4, 1: 1867.4. Samples: 11191330. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-08 04:47:33,755][130385] Avg episode reward: [(0, '45.830'), (1, '47.250')] [2023-10-08 04:47:34,345][00612] Updated weights for policy 1, policy_version 21890 (0.0009) [2023-10-08 04:47:34,713][00612] Updated weights for policy 1, policy_version 21900 (0.0010) [2023-10-08 04:47:35,085][00612] Updated weights for policy 1, policy_version 21910 (0.0010) [2023-10-08 04:47:35,447][00612] Updated weights for policy 1, policy_version 21920 (0.0009) [2023-10-08 04:47:36,788][00611] Updated weights for policy 0, policy_version 21802 (0.0007) [2023-10-08 04:47:37,165][00611] Updated weights for policy 0, policy_version 21812 (0.0008) [2023-10-08 04:47:37,547][00611] Updated weights for policy 0, policy_version 21822 (0.0008) [2023-10-08 04:47:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 44793856. Throughput: 0: 1828.4, 1: 1866.8. Samples: 11202950. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-08 04:47:38,754][130385] Avg episode reward: [(0, '45.330'), (1, '46.680')] [2023-10-08 04:47:38,905][00612] Updated weights for policy 1, policy_version 21930 (0.0008) [2023-10-08 04:47:39,270][00612] Updated weights for policy 1, policy_version 21940 (0.0008) [2023-10-08 04:47:39,640][00612] Updated weights for policy 1, policy_version 21950 (0.0007) [2023-10-08 04:47:40,997][00611] Updated weights for policy 0, policy_version 21832 (0.0009) [2023-10-08 04:47:41,369][00611] Updated weights for policy 0, policy_version 21842 (0.0009) [2023-10-08 04:47:41,741][00611] Updated weights for policy 0, policy_version 21852 (0.0009) [2023-10-08 04:47:43,253][00612] Updated weights for policy 1, policy_version 21960 (0.0007) [2023-10-08 04:47:43,634][00612] Updated weights for policy 1, policy_version 21970 (0.0007) [2023-10-08 04:47:43,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 44859392. Throughput: 0: 1825.3, 1: 1866.5. Samples: 11224658. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-08 04:47:43,754][130385] Avg episode reward: [(0, '45.330'), (1, '49.240')] [2023-10-08 04:47:43,999][00612] Updated weights for policy 1, policy_version 21980 (0.0008) [2023-10-08 04:47:44,142][00425] Saving new best policy, reward=49.240! [2023-10-08 04:47:45,414][00611] Updated weights for policy 0, policy_version 21862 (0.0010) [2023-10-08 04:47:45,786][00611] Updated weights for policy 0, policy_version 21872 (0.0010) [2023-10-08 04:47:46,160][00611] Updated weights for policy 0, policy_version 21882 (0.0008) [2023-10-08 04:47:47,662][00612] Updated weights for policy 1, policy_version 21990 (0.0008) [2023-10-08 04:47:48,032][00612] Updated weights for policy 1, policy_version 22000 (0.0010) [2023-10-08 04:47:48,402][00612] Updated weights for policy 1, policy_version 22010 (0.0010) [2023-10-08 04:47:48,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 44957696. Throughput: 0: 1836.3, 1: 1843.4. Samples: 11247070. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-08 04:47:48,755][130385] Avg episode reward: [(0, '47.550'), (1, '47.180')] [2023-10-08 04:47:49,853][00611] Updated weights for policy 0, policy_version 21892 (0.0009) [2023-10-08 04:47:50,220][00611] Updated weights for policy 0, policy_version 21902 (0.0010) [2023-10-08 04:47:50,595][00611] Updated weights for policy 0, policy_version 21912 (0.0007) [2023-10-08 04:47:52,026][00612] Updated weights for policy 1, policy_version 22020 (0.0010) [2023-10-08 04:47:52,397][00612] Updated weights for policy 1, policy_version 22030 (0.0009) [2023-10-08 04:47:52,772][00612] Updated weights for policy 1, policy_version 22040 (0.0008) [2023-10-08 04:47:53,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 45023232. Throughput: 0: 1829.9, 1: 1865.8. Samples: 11257976. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 04:47:53,755][130385] Avg episode reward: [(0, '45.330'), (1, '46.880')] [2023-10-08 04:47:54,164][00611] Updated weights for policy 0, policy_version 21922 (0.0008) [2023-10-08 04:47:54,543][00611] Updated weights for policy 0, policy_version 21932 (0.0009) [2023-10-08 04:47:54,907][00611] Updated weights for policy 0, policy_version 21942 (0.0008) [2023-10-08 04:47:55,276][00611] Updated weights for policy 0, policy_version 21952 (0.0010) [2023-10-08 04:47:56,555][00612] Updated weights for policy 1, policy_version 22050 (0.0009) [2023-10-08 04:47:56,948][00612] Updated weights for policy 1, policy_version 22060 (0.0008) [2023-10-08 04:47:57,310][00612] Updated weights for policy 1, policy_version 22070 (0.0009) [2023-10-08 04:47:57,678][00612] Updated weights for policy 1, policy_version 22080 (0.0010) [2023-10-08 04:47:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 45088768. Throughput: 0: 1832.9, 1: 1844.2. Samples: 11279848. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 04:47:58,755][130385] Avg episode reward: [(0, '44.240'), (1, '47.550')] [2023-10-08 04:47:59,100][00611] Updated weights for policy 0, policy_version 21962 (0.0007) [2023-10-08 04:47:59,480][00611] Updated weights for policy 0, policy_version 21972 (0.0007) [2023-10-08 04:47:59,846][00611] Updated weights for policy 0, policy_version 21982 (0.0007) [2023-10-08 04:48:01,356][00612] Updated weights for policy 1, policy_version 22090 (0.0010) [2023-10-08 04:48:01,730][00612] Updated weights for policy 1, policy_version 22100 (0.0010) [2023-10-08 04:48:02,092][00612] Updated weights for policy 1, policy_version 22110 (0.0009) [2023-10-08 04:48:03,424][00611] Updated weights for policy 0, policy_version 21992 (0.0007) [2023-10-08 04:48:03,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 45154304. Throughput: 0: 1831.3, 1: 1857.3. Samples: 11302250. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 04:48:03,754][130385] Avg episode reward: [(0, '42.850'), (1, '43.860')] [2023-10-08 04:48:03,794][00611] Updated weights for policy 0, policy_version 22002 (0.0008) [2023-10-08 04:48:04,167][00611] Updated weights for policy 0, policy_version 22012 (0.0011) [2023-10-08 04:48:05,630][00612] Updated weights for policy 1, policy_version 22120 (0.0011) [2023-10-08 04:48:05,997][00612] Updated weights for policy 1, policy_version 22130 (0.0007) [2023-10-08 04:48:06,359][00612] Updated weights for policy 1, policy_version 22140 (0.0009) [2023-10-08 04:48:07,940][00611] Updated weights for policy 0, policy_version 22022 (0.0010) [2023-10-08 04:48:08,306][00611] Updated weights for policy 0, policy_version 22032 (0.0007) [2023-10-08 04:48:08,675][00611] Updated weights for policy 0, policy_version 22042 (0.0009) [2023-10-08 04:48:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 45219840. Throughput: 0: 1838.2, 1: 1835.8. Samples: 11312964. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 04:48:08,754][130385] Avg episode reward: [(0, '42.760'), (1, '43.090')] [2023-10-08 04:48:09,993][00612] Updated weights for policy 1, policy_version 22150 (0.0008) [2023-10-08 04:48:10,364][00612] Updated weights for policy 1, policy_version 22160 (0.0009) [2023-10-08 04:48:10,736][00612] Updated weights for policy 1, policy_version 22170 (0.0007) [2023-10-08 04:48:12,427][00611] Updated weights for policy 0, policy_version 22052 (0.0010) [2023-10-08 04:48:12,810][00611] Updated weights for policy 0, policy_version 22062 (0.0007) [2023-10-08 04:48:13,182][00611] Updated weights for policy 0, policy_version 22072 (0.0010) [2023-10-08 04:48:13,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 45318144. Throughput: 0: 1831.4, 1: 1855.4. Samples: 11335396. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 04:48:13,754][130385] Avg episode reward: [(0, '42.500'), (1, '44.920')] [2023-10-08 04:48:14,363][00612] Updated weights for policy 1, policy_version 22180 (0.0007) [2023-10-08 04:48:14,727][00612] Updated weights for policy 1, policy_version 22190 (0.0007) [2023-10-08 04:48:15,090][00612] Updated weights for policy 1, policy_version 22200 (0.0007) [2023-10-08 04:48:16,687][00611] Updated weights for policy 0, policy_version 22082 (0.0009) [2023-10-08 04:48:17,062][00611] Updated weights for policy 0, policy_version 22092 (0.0007) [2023-10-08 04:48:17,432][00611] Updated weights for policy 0, policy_version 22102 (0.0007) [2023-10-08 04:48:17,810][00611] Updated weights for policy 0, policy_version 22112 (0.0007) [2023-10-08 04:48:18,752][00612] Updated weights for policy 1, policy_version 22210 (0.0009) [2023-10-08 04:48:18,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 45383680. Throughput: 0: 1837.9, 1: 1851.2. Samples: 11357338. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 04:48:18,755][130385] Avg episode reward: [(0, '43.580'), (1, '42.170')] [2023-10-08 04:48:18,765][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000022112_22642688.pth... [2023-10-08 04:48:18,798][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000020384_20873216.pth [2023-10-08 04:48:19,127][00612] Updated weights for policy 1, policy_version 22220 (0.0008) [2023-10-08 04:48:19,482][00612] Updated weights for policy 1, policy_version 22230 (0.0007) [2023-10-08 04:48:19,858][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000022240_22773760.pth... [2023-10-08 04:48:19,859][00612] Updated weights for policy 1, policy_version 22240 (0.0008) [2023-10-08 04:48:19,887][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000020480_20971520.pth [2023-10-08 04:48:21,508][00611] Updated weights for policy 0, policy_version 22122 (0.0007) [2023-10-08 04:48:21,882][00611] Updated weights for policy 0, policy_version 22132 (0.0009) [2023-10-08 04:48:22,252][00611] Updated weights for policy 0, policy_version 22142 (0.0008) [2023-10-08 04:48:23,533][00612] Updated weights for policy 1, policy_version 22250 (0.0007) [2023-10-08 04:48:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 45449216. Throughput: 0: 1840.8, 1: 1845.7. Samples: 11368842. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 04:48:23,754][130385] Avg episode reward: [(0, '43.270'), (1, '39.740')] [2023-10-08 04:48:23,901][00612] Updated weights for policy 1, policy_version 22260 (0.0008) [2023-10-08 04:48:24,271][00612] Updated weights for policy 1, policy_version 22270 (0.0007) [2023-10-08 04:48:25,887][00611] Updated weights for policy 0, policy_version 22152 (0.0007) [2023-10-08 04:48:26,256][00611] Updated weights for policy 0, policy_version 22162 (0.0008) [2023-10-08 04:48:26,640][00611] Updated weights for policy 0, policy_version 22172 (0.0008) [2023-10-08 04:48:27,832][00612] Updated weights for policy 1, policy_version 22280 (0.0010) [2023-10-08 04:48:28,199][00612] Updated weights for policy 1, policy_version 22290 (0.0011) [2023-10-08 04:48:28,574][00612] Updated weights for policy 1, policy_version 22300 (0.0011) [2023-10-08 04:48:28,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 45547520. Throughput: 0: 1838.4, 1: 1845.2. Samples: 11390422. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:48:28,754][130385] Avg episode reward: [(0, '39.410'), (1, '39.590')] [2023-10-08 04:48:30,236][00611] Updated weights for policy 0, policy_version 22182 (0.0009) [2023-10-08 04:48:30,610][00611] Updated weights for policy 0, policy_version 22192 (0.0009) [2023-10-08 04:48:30,993][00611] Updated weights for policy 0, policy_version 22202 (0.0010) [2023-10-08 04:48:32,287][00612] Updated weights for policy 1, policy_version 22310 (0.0009) [2023-10-08 04:48:32,649][00612] Updated weights for policy 1, policy_version 22320 (0.0010) [2023-10-08 04:48:33,010][00612] Updated weights for policy 1, policy_version 22330 (0.0011) [2023-10-08 04:48:33,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 45613056. Throughput: 0: 1831.1, 1: 1830.9. Samples: 11411858. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:48:33,754][130385] Avg episode reward: [(0, '40.240'), (1, '40.460')] [2023-10-08 04:48:34,675][00611] Updated weights for policy 0, policy_version 22212 (0.0009) [2023-10-08 04:48:35,055][00611] Updated weights for policy 0, policy_version 22222 (0.0010) [2023-10-08 04:48:35,428][00611] Updated weights for policy 0, policy_version 22232 (0.0008) [2023-10-08 04:48:36,767][00612] Updated weights for policy 1, policy_version 22340 (0.0010) [2023-10-08 04:48:37,145][00612] Updated weights for policy 1, policy_version 22350 (0.0011) [2023-10-08 04:48:37,524][00612] Updated weights for policy 1, policy_version 22360 (0.0010) [2023-10-08 04:48:38,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 45678592. Throughput: 0: 1836.1, 1: 1842.3. Samples: 11423504. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:48:38,755][130385] Avg episode reward: [(0, '41.970'), (1, '41.720')] [2023-10-08 04:48:39,046][00611] Updated weights for policy 0, policy_version 22242 (0.0008) [2023-10-08 04:48:39,418][00611] Updated weights for policy 0, policy_version 22252 (0.0010) [2023-10-08 04:48:39,799][00611] Updated weights for policy 0, policy_version 22262 (0.0007) [2023-10-08 04:48:40,175][00611] Updated weights for policy 0, policy_version 22272 (0.0008) [2023-10-08 04:48:41,066][00612] Updated weights for policy 1, policy_version 22370 (0.0010) [2023-10-08 04:48:41,440][00612] Updated weights for policy 1, policy_version 22380 (0.0008) [2023-10-08 04:48:41,808][00612] Updated weights for policy 1, policy_version 22390 (0.0009) [2023-10-08 04:48:42,174][00612] Updated weights for policy 1, policy_version 22400 (0.0008) [2023-10-08 04:48:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 45744128. Throughput: 0: 1837.6, 1: 1829.9. Samples: 11444882. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:48:43,754][130385] Avg episode reward: [(0, '43.370'), (1, '37.380')] [2023-10-08 04:48:43,857][00611] Updated weights for policy 0, policy_version 22282 (0.0009) [2023-10-08 04:48:44,222][00611] Updated weights for policy 0, policy_version 22292 (0.0011) [2023-10-08 04:48:44,602][00611] Updated weights for policy 0, policy_version 22302 (0.0009) [2023-10-08 04:48:45,777][00612] Updated weights for policy 1, policy_version 22410 (0.0009) [2023-10-08 04:48:46,147][00612] Updated weights for policy 1, policy_version 22420 (0.0010) [2023-10-08 04:48:46,516][00612] Updated weights for policy 1, policy_version 22430 (0.0010) [2023-10-08 04:48:48,081][00611] Updated weights for policy 0, policy_version 22312 (0.0008) [2023-10-08 04:48:48,448][00611] Updated weights for policy 0, policy_version 22322 (0.0007) [2023-10-08 04:48:48,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 45809664. Throughput: 0: 1828.0, 1: 1845.7. Samples: 11467568. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 04:48:48,754][130385] Avg episode reward: [(0, '45.360'), (1, '39.380')] [2023-10-08 04:48:48,829][00611] Updated weights for policy 0, policy_version 22332 (0.0009) [2023-10-08 04:48:50,167][00612] Updated weights for policy 1, policy_version 22440 (0.0008) [2023-10-08 04:48:50,543][00612] Updated weights for policy 1, policy_version 22450 (0.0008) [2023-10-08 04:48:50,916][00612] Updated weights for policy 1, policy_version 22460 (0.0009) [2023-10-08 04:48:52,582][00611] Updated weights for policy 0, policy_version 22342 (0.0009) [2023-10-08 04:48:52,967][00611] Updated weights for policy 0, policy_version 22352 (0.0009) [2023-10-08 04:48:53,334][00611] Updated weights for policy 0, policy_version 22362 (0.0009) [2023-10-08 04:48:53,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 45907968. Throughput: 0: 1831.8, 1: 1833.8. Samples: 11477916. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 04:48:53,755][130385] Avg episode reward: [(0, '45.810'), (1, '39.650')] [2023-10-08 04:48:54,542][00612] Updated weights for policy 1, policy_version 22470 (0.0008) [2023-10-08 04:48:54,902][00612] Updated weights for policy 1, policy_version 22480 (0.0007) [2023-10-08 04:48:55,270][00612] Updated weights for policy 1, policy_version 22490 (0.0008) [2023-10-08 04:48:57,026][00611] Updated weights for policy 0, policy_version 22372 (0.0010) [2023-10-08 04:48:57,392][00611] Updated weights for policy 0, policy_version 22382 (0.0010) [2023-10-08 04:48:57,767][00611] Updated weights for policy 0, policy_version 22392 (0.0007) [2023-10-08 04:48:58,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 45973504. Throughput: 0: 1827.9, 1: 1838.7. Samples: 11500398. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 04:48:58,755][130385] Avg episode reward: [(0, '46.210'), (1, '40.820')] [2023-10-08 04:48:58,893][00612] Updated weights for policy 1, policy_version 22500 (0.0009) [2023-10-08 04:48:59,258][00612] Updated weights for policy 1, policy_version 22510 (0.0010) [2023-10-08 04:48:59,632][00612] Updated weights for policy 1, policy_version 22520 (0.0010) [2023-10-08 04:49:01,538][00611] Updated weights for policy 0, policy_version 22402 (0.0009) [2023-10-08 04:49:01,903][00611] Updated weights for policy 0, policy_version 22412 (0.0008) [2023-10-08 04:49:02,273][00611] Updated weights for policy 0, policy_version 22422 (0.0010) [2023-10-08 04:49:02,639][00611] Updated weights for policy 0, policy_version 22432 (0.0009) [2023-10-08 04:49:03,233][00612] Updated weights for policy 1, policy_version 22530 (0.0008) [2023-10-08 04:49:03,609][00612] Updated weights for policy 1, policy_version 22540 (0.0007) [2023-10-08 04:49:03,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 46039040. Throughput: 0: 1827.2, 1: 1838.9. Samples: 11522310. Policy #0 lag: (min: 23.0, avg: 30.7, max: 55.0) [2023-10-08 04:49:03,754][130385] Avg episode reward: [(0, '44.720'), (1, '43.010')] [2023-10-08 04:49:03,979][00612] Updated weights for policy 1, policy_version 22550 (0.0007) [2023-10-08 04:49:04,341][00612] Updated weights for policy 1, policy_version 22560 (0.0008) [2023-10-08 04:49:06,405][00611] Updated weights for policy 0, policy_version 22442 (0.0007) [2023-10-08 04:49:06,766][00611] Updated weights for policy 0, policy_version 22452 (0.0008) [2023-10-08 04:49:07,149][00611] Updated weights for policy 0, policy_version 22462 (0.0007) [2023-10-08 04:49:07,950][00612] Updated weights for policy 1, policy_version 22570 (0.0008) [2023-10-08 04:49:08,323][00612] Updated weights for policy 1, policy_version 22580 (0.0009) [2023-10-08 04:49:08,694][00612] Updated weights for policy 1, policy_version 22590 (0.0009) [2023-10-08 04:49:08,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 46104576. Throughput: 0: 1823.1, 1: 1842.9. Samples: 11533812. Policy #0 lag: (min: 23.0, avg: 30.7, max: 55.0) [2023-10-08 04:49:08,754][130385] Avg episode reward: [(0, '44.300'), (1, '42.750')] [2023-10-08 04:49:10,815][00611] Updated weights for policy 0, policy_version 22472 (0.0008) [2023-10-08 04:49:11,194][00611] Updated weights for policy 0, policy_version 22482 (0.0008) [2023-10-08 04:49:11,569][00611] Updated weights for policy 0, policy_version 22492 (0.0007) [2023-10-08 04:49:12,451][00612] Updated weights for policy 1, policy_version 22600 (0.0009) [2023-10-08 04:49:12,828][00612] Updated weights for policy 1, policy_version 22610 (0.0011) [2023-10-08 04:49:13,190][00612] Updated weights for policy 1, policy_version 22620 (0.0009) [2023-10-08 04:49:13,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 46202880. Throughput: 0: 1826.1, 1: 1837.5. Samples: 11555284. Policy #0 lag: (min: 23.0, avg: 30.7, max: 55.0) [2023-10-08 04:49:13,755][130385] Avg episode reward: [(0, '47.940'), (1, '43.640')] [2023-10-08 04:49:13,756][00365] Saving new best policy, reward=47.940! [2023-10-08 04:49:15,120][00611] Updated weights for policy 0, policy_version 22502 (0.0007) [2023-10-08 04:49:15,496][00611] Updated weights for policy 0, policy_version 22512 (0.0010) [2023-10-08 04:49:15,865][00611] Updated weights for policy 0, policy_version 22522 (0.0009) [2023-10-08 04:49:16,946][00612] Updated weights for policy 1, policy_version 22630 (0.0010) [2023-10-08 04:49:17,307][00612] Updated weights for policy 1, policy_version 22640 (0.0007) [2023-10-08 04:49:17,680][00612] Updated weights for policy 1, policy_version 22650 (0.0009) [2023-10-08 04:49:18,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 46268416. Throughput: 0: 1841.2, 1: 1833.6. Samples: 11577226. Policy #0 lag: (min: 23.0, avg: 30.7, max: 55.0) [2023-10-08 04:49:18,755][130385] Avg episode reward: [(0, '47.480'), (1, '41.110')] [2023-10-08 04:49:19,412][00611] Updated weights for policy 0, policy_version 22532 (0.0008) [2023-10-08 04:49:19,782][00611] Updated weights for policy 0, policy_version 22542 (0.0007) [2023-10-08 04:49:20,161][00611] Updated weights for policy 0, policy_version 22552 (0.0008) [2023-10-08 04:49:21,450][00612] Updated weights for policy 1, policy_version 22660 (0.0009) [2023-10-08 04:49:21,812][00612] Updated weights for policy 1, policy_version 22670 (0.0009) [2023-10-08 04:49:22,178][00612] Updated weights for policy 1, policy_version 22680 (0.0009) [2023-10-08 04:49:23,714][00611] Updated weights for policy 0, policy_version 22562 (0.0009) [2023-10-08 04:49:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 46333952. Throughput: 0: 1837.4, 1: 1835.2. Samples: 11588772. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:49:23,754][130385] Avg episode reward: [(0, '44.490'), (1, '42.070')] [2023-10-08 04:49:24,079][00611] Updated weights for policy 0, policy_version 22572 (0.0009) [2023-10-08 04:49:24,451][00611] Updated weights for policy 0, policy_version 22582 (0.0009) [2023-10-08 04:49:24,820][00611] Updated weights for policy 0, policy_version 22592 (0.0007) [2023-10-08 04:49:25,886][00612] Updated weights for policy 1, policy_version 22690 (0.0008) [2023-10-08 04:49:26,251][00612] Updated weights for policy 1, policy_version 22700 (0.0008) [2023-10-08 04:49:26,615][00612] Updated weights for policy 1, policy_version 22710 (0.0009) [2023-10-08 04:49:26,978][00612] Updated weights for policy 1, policy_version 22720 (0.0010) [2023-10-08 04:49:28,532][00611] Updated weights for policy 0, policy_version 22602 (0.0010) [2023-10-08 04:49:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 46399488. Throughput: 0: 1840.8, 1: 1836.1. Samples: 11610344. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:49:28,755][130385] Avg episode reward: [(0, '43.190'), (1, '43.930')] [2023-10-08 04:49:28,901][00611] Updated weights for policy 0, policy_version 22612 (0.0010) [2023-10-08 04:49:29,271][00611] Updated weights for policy 0, policy_version 22622 (0.0011) [2023-10-08 04:49:30,699][00612] Updated weights for policy 1, policy_version 22730 (0.0009) [2023-10-08 04:49:31,061][00612] Updated weights for policy 1, policy_version 22740 (0.0007) [2023-10-08 04:49:31,432][00612] Updated weights for policy 1, policy_version 22750 (0.0007) [2023-10-08 04:49:32,822][00611] Updated weights for policy 0, policy_version 22632 (0.0008) [2023-10-08 04:49:33,197][00611] Updated weights for policy 0, policy_version 22642 (0.0008) [2023-10-08 04:49:33,578][00611] Updated weights for policy 0, policy_version 22652 (0.0007) [2023-10-08 04:49:33,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 46497792. Throughput: 0: 1829.0, 1: 1834.2. Samples: 11632412. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 04:49:33,755][130385] Avg episode reward: [(0, '43.890'), (1, '44.590')] [2023-10-08 04:49:34,904][00612] Updated weights for policy 1, policy_version 22760 (0.0007) [2023-10-08 04:49:35,275][00612] Updated weights for policy 1, policy_version 22770 (0.0008) [2023-10-08 04:49:35,646][00612] Updated weights for policy 1, policy_version 22780 (0.0008) [2023-10-08 04:49:37,271][00611] Updated weights for policy 0, policy_version 22662 (0.0008) [2023-10-08 04:49:37,639][00611] Updated weights for policy 0, policy_version 22672 (0.0008) [2023-10-08 04:49:38,011][00611] Updated weights for policy 0, policy_version 22682 (0.0009) [2023-10-08 04:49:38,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 46563328. Throughput: 0: 1839.8, 1: 1833.5. Samples: 11643214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:49:38,755][130385] Avg episode reward: [(0, '42.490'), (1, '42.750')] [2023-10-08 04:49:39,310][00612] Updated weights for policy 1, policy_version 22790 (0.0007) [2023-10-08 04:49:39,683][00612] Updated weights for policy 1, policy_version 22800 (0.0007) [2023-10-08 04:49:40,046][00612] Updated weights for policy 1, policy_version 22810 (0.0008) [2023-10-08 04:49:41,642][00611] Updated weights for policy 0, policy_version 22692 (0.0010) [2023-10-08 04:49:42,012][00611] Updated weights for policy 0, policy_version 22702 (0.0010) [2023-10-08 04:49:42,387][00611] Updated weights for policy 0, policy_version 22712 (0.0008) [2023-10-08 04:49:43,641][00612] Updated weights for policy 1, policy_version 22820 (0.0010) [2023-10-08 04:49:43,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 46628864. Throughput: 0: 1828.8, 1: 1846.5. Samples: 11665786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:49:43,754][130385] Avg episode reward: [(0, '47.850'), (1, '41.410')] [2023-10-08 04:49:44,007][00612] Updated weights for policy 1, policy_version 22830 (0.0010) [2023-10-08 04:49:44,377][00612] Updated weights for policy 1, policy_version 22840 (0.0009) [2023-10-08 04:49:45,960][00611] Updated weights for policy 0, policy_version 22722 (0.0010) [2023-10-08 04:49:46,337][00611] Updated weights for policy 0, policy_version 22732 (0.0008) [2023-10-08 04:49:46,710][00611] Updated weights for policy 0, policy_version 22742 (0.0007) [2023-10-08 04:49:47,083][00611] Updated weights for policy 0, policy_version 22752 (0.0009) [2023-10-08 04:49:47,966][00612] Updated weights for policy 1, policy_version 22850 (0.0009) [2023-10-08 04:49:48,327][00612] Updated weights for policy 1, policy_version 22860 (0.0009) [2023-10-08 04:49:48,696][00612] Updated weights for policy 1, policy_version 22870 (0.0008) [2023-10-08 04:49:48,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 46694400. Throughput: 0: 1844.8, 1: 1830.7. Samples: 11687708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:49:48,754][130385] Avg episode reward: [(0, '47.880'), (1, '41.000')] [2023-10-08 04:49:49,064][00612] Updated weights for policy 1, policy_version 22880 (0.0009) [2023-10-08 04:49:50,872][00611] Updated weights for policy 0, policy_version 22762 (0.0010) [2023-10-08 04:49:51,247][00611] Updated weights for policy 0, policy_version 22772 (0.0008) [2023-10-08 04:49:51,612][00611] Updated weights for policy 0, policy_version 22782 (0.0008) [2023-10-08 04:49:52,558][00612] Updated weights for policy 1, policy_version 22890 (0.0009) [2023-10-08 04:49:52,929][00612] Updated weights for policy 1, policy_version 22900 (0.0008) [2023-10-08 04:49:53,303][00612] Updated weights for policy 1, policy_version 22910 (0.0008) [2023-10-08 04:49:53,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 46792704. Throughput: 0: 1828.5, 1: 1840.9. Samples: 11698936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:49:53,755][130385] Avg episode reward: [(0, '46.470'), (1, '40.980')] [2023-10-08 04:49:55,210][00611] Updated weights for policy 0, policy_version 22792 (0.0008) [2023-10-08 04:49:55,579][00611] Updated weights for policy 0, policy_version 22802 (0.0009) [2023-10-08 04:49:55,958][00611] Updated weights for policy 0, policy_version 22812 (0.0008) [2023-10-08 04:49:56,814][00612] Updated weights for policy 1, policy_version 22920 (0.0007) [2023-10-08 04:49:57,185][00612] Updated weights for policy 1, policy_version 22930 (0.0008) [2023-10-08 04:49:57,563][00612] Updated weights for policy 1, policy_version 22940 (0.0008) [2023-10-08 04:49:58,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 46858240. Throughput: 0: 1845.0, 1: 1832.1. Samples: 11720756. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 04:49:58,754][130385] Avg episode reward: [(0, '42.920'), (1, '41.970')] [2023-10-08 04:49:59,673][00611] Updated weights for policy 0, policy_version 22822 (0.0007) [2023-10-08 04:50:00,051][00611] Updated weights for policy 0, policy_version 22832 (0.0008) [2023-10-08 04:50:00,420][00611] Updated weights for policy 0, policy_version 22842 (0.0007) [2023-10-08 04:50:01,129][00612] Updated weights for policy 1, policy_version 22950 (0.0008) [2023-10-08 04:50:01,497][00612] Updated weights for policy 1, policy_version 22960 (0.0008) [2023-10-08 04:50:01,859][00612] Updated weights for policy 1, policy_version 22970 (0.0007) [2023-10-08 04:50:03,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 46923776. Throughput: 0: 1826.7, 1: 1858.3. Samples: 11743050. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 04:50:03,754][130385] Avg episode reward: [(0, '47.190'), (1, '42.450')] [2023-10-08 04:50:04,058][00611] Updated weights for policy 0, policy_version 22852 (0.0008) [2023-10-08 04:50:04,439][00611] Updated weights for policy 0, policy_version 22862 (0.0007) [2023-10-08 04:50:04,796][00611] Updated weights for policy 0, policy_version 22872 (0.0008) [2023-10-08 04:50:05,428][00612] Updated weights for policy 1, policy_version 22980 (0.0008) [2023-10-08 04:50:05,802][00612] Updated weights for policy 1, policy_version 22990 (0.0008) [2023-10-08 04:50:06,167][00612] Updated weights for policy 1, policy_version 23000 (0.0009) [2023-10-08 04:50:08,538][00611] Updated weights for policy 0, policy_version 22882 (0.0008) [2023-10-08 04:50:08,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 46989312. Throughput: 0: 1829.2, 1: 1836.7. Samples: 11753740. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 04:50:08,755][130385] Avg episode reward: [(0, '47.180'), (1, '43.080')] [2023-10-08 04:50:08,907][00611] Updated weights for policy 0, policy_version 22892 (0.0009) [2023-10-08 04:50:09,274][00611] Updated weights for policy 0, policy_version 22902 (0.0011) [2023-10-08 04:50:09,638][00611] Updated weights for policy 0, policy_version 22912 (0.0009) [2023-10-08 04:50:09,925][00612] Updated weights for policy 1, policy_version 23010 (0.0008) [2023-10-08 04:50:10,293][00612] Updated weights for policy 1, policy_version 23020 (0.0009) [2023-10-08 04:50:10,666][00612] Updated weights for policy 1, policy_version 23030 (0.0009) [2023-10-08 04:50:11,027][00612] Updated weights for policy 1, policy_version 23040 (0.0007) [2023-10-08 04:50:13,213][00611] Updated weights for policy 0, policy_version 22922 (0.0010) [2023-10-08 04:50:13,581][00611] Updated weights for policy 0, policy_version 22932 (0.0007) [2023-10-08 04:50:13,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 47054848. Throughput: 0: 1828.2, 1: 1861.7. Samples: 11776388. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 04:50:13,755][130385] Avg episode reward: [(0, '47.370'), (1, '38.890')] [2023-10-08 04:50:13,949][00611] Updated weights for policy 0, policy_version 22942 (0.0010) [2023-10-08 04:50:14,606][00612] Updated weights for policy 1, policy_version 23050 (0.0009) [2023-10-08 04:50:14,987][00612] Updated weights for policy 1, policy_version 23060 (0.0007) [2023-10-08 04:50:15,346][00612] Updated weights for policy 1, policy_version 23070 (0.0008) [2023-10-08 04:50:17,483][00611] Updated weights for policy 0, policy_version 22952 (0.0010) [2023-10-08 04:50:17,857][00611] Updated weights for policy 0, policy_version 22962 (0.0008) [2023-10-08 04:50:18,220][00611] Updated weights for policy 0, policy_version 22972 (0.0008) [2023-10-08 04:50:18,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 47153152. Throughput: 0: 1827.7, 1: 1865.9. Samples: 11798624. Policy #0 lag: (min: 13.0, avg: 19.9, max: 45.0) [2023-10-08 04:50:18,754][130385] Avg episode reward: [(0, '47.330'), (1, '42.990')] [2023-10-08 04:50:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000022976_23527424.pth... [2023-10-08 04:50:18,798][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000021248_21757952.pth [2023-10-08 04:50:18,927][00612] Updated weights for policy 1, policy_version 23080 (0.0009) [2023-10-08 04:50:19,293][00612] Updated weights for policy 1, policy_version 23090 (0.0010) [2023-10-08 04:50:19,664][00612] Updated weights for policy 1, policy_version 23100 (0.0009) [2023-10-08 04:50:19,815][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000023104_23658496.pth... [2023-10-08 04:50:19,843][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000021344_21856256.pth [2023-10-08 04:50:21,874][00611] Updated weights for policy 0, policy_version 22982 (0.0008) [2023-10-08 04:50:22,244][00611] Updated weights for policy 0, policy_version 22992 (0.0009) [2023-10-08 04:50:22,617][00611] Updated weights for policy 0, policy_version 23002 (0.0009) [2023-10-08 04:50:23,398][00612] Updated weights for policy 1, policy_version 23110 (0.0009) [2023-10-08 04:50:23,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 47218688. Throughput: 0: 1834.0, 1: 1862.5. Samples: 11809552. Policy #0 lag: (min: 13.0, avg: 19.9, max: 45.0) [2023-10-08 04:50:23,754][130385] Avg episode reward: [(0, '44.170'), (1, '40.680')] [2023-10-08 04:50:23,778][00612] Updated weights for policy 1, policy_version 23120 (0.0007) [2023-10-08 04:50:24,150][00612] Updated weights for policy 1, policy_version 23130 (0.0008) [2023-10-08 04:50:26,233][00611] Updated weights for policy 0, policy_version 23012 (0.0007) [2023-10-08 04:50:26,605][00611] Updated weights for policy 0, policy_version 23022 (0.0007) [2023-10-08 04:50:26,985][00611] Updated weights for policy 0, policy_version 23032 (0.0007) [2023-10-08 04:50:27,783][00612] Updated weights for policy 1, policy_version 23140 (0.0008) [2023-10-08 04:50:28,154][00612] Updated weights for policy 1, policy_version 23150 (0.0009) [2023-10-08 04:50:28,511][00612] Updated weights for policy 1, policy_version 23160 (0.0010) [2023-10-08 04:50:28,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 47284224. Throughput: 0: 1827.4, 1: 1855.5. Samples: 11831518. Policy #0 lag: (min: 13.0, avg: 19.9, max: 45.0) [2023-10-08 04:50:28,755][130385] Avg episode reward: [(0, '43.430'), (1, '40.000')] [2023-10-08 04:50:30,617][00611] Updated weights for policy 0, policy_version 23042 (0.0007) [2023-10-08 04:50:30,996][00611] Updated weights for policy 0, policy_version 23052 (0.0010) [2023-10-08 04:50:31,366][00611] Updated weights for policy 0, policy_version 23062 (0.0007) [2023-10-08 04:50:31,743][00611] Updated weights for policy 0, policy_version 23072 (0.0011) [2023-10-08 04:50:32,128][00612] Updated weights for policy 1, policy_version 23170 (0.0009) [2023-10-08 04:50:32,496][00612] Updated weights for policy 1, policy_version 23180 (0.0008) [2023-10-08 04:50:32,877][00612] Updated weights for policy 1, policy_version 23190 (0.0010) [2023-10-08 04:50:33,250][00612] Updated weights for policy 1, policy_version 23200 (0.0010) [2023-10-08 04:50:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 47382528. Throughput: 0: 1843.5, 1: 1828.7. Samples: 11852956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:50:33,754][130385] Avg episode reward: [(0, '44.520'), (1, '39.510')] [2023-10-08 04:50:35,259][00611] Updated weights for policy 0, policy_version 23082 (0.0008) [2023-10-08 04:50:35,630][00611] Updated weights for policy 0, policy_version 23092 (0.0008) [2023-10-08 04:50:36,010][00611] Updated weights for policy 0, policy_version 23102 (0.0008) [2023-10-08 04:50:36,892][00612] Updated weights for policy 1, policy_version 23210 (0.0010) [2023-10-08 04:50:37,264][00612] Updated weights for policy 1, policy_version 23220 (0.0011) [2023-10-08 04:50:37,631][00612] Updated weights for policy 1, policy_version 23230 (0.0010) [2023-10-08 04:50:38,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 47448064. Throughput: 0: 1824.7, 1: 1850.0. Samples: 11864298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:50:38,755][130385] Avg episode reward: [(0, '48.830'), (1, '40.970')] [2023-10-08 04:50:38,757][00365] Saving new best policy, reward=48.830! [2023-10-08 04:50:39,658][00611] Updated weights for policy 0, policy_version 23112 (0.0008) [2023-10-08 04:50:40,032][00611] Updated weights for policy 0, policy_version 23122 (0.0008) [2023-10-08 04:50:40,396][00611] Updated weights for policy 0, policy_version 23132 (0.0009) [2023-10-08 04:50:41,339][00612] Updated weights for policy 1, policy_version 23240 (0.0009) [2023-10-08 04:50:41,705][00612] Updated weights for policy 1, policy_version 23250 (0.0008) [2023-10-08 04:50:42,078][00612] Updated weights for policy 1, policy_version 23260 (0.0010) [2023-10-08 04:50:43,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 47513600. Throughput: 0: 1845.6, 1: 1827.2. Samples: 11886032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:50:43,755][130385] Avg episode reward: [(0, '45.470'), (1, '40.930')] [2023-10-08 04:50:43,899][00611] Updated weights for policy 0, policy_version 23142 (0.0010) [2023-10-08 04:50:44,280][00611] Updated weights for policy 0, policy_version 23152 (0.0009) [2023-10-08 04:50:44,661][00611] Updated weights for policy 0, policy_version 23162 (0.0007) [2023-10-08 04:50:45,708][00612] Updated weights for policy 1, policy_version 23270 (0.0008) [2023-10-08 04:50:46,075][00612] Updated weights for policy 1, policy_version 23280 (0.0008) [2023-10-08 04:50:46,443][00612] Updated weights for policy 1, policy_version 23290 (0.0008) [2023-10-08 04:50:48,411][00611] Updated weights for policy 0, policy_version 23172 (0.0008) [2023-10-08 04:50:48,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 47579136. Throughput: 0: 1853.9, 1: 1835.2. Samples: 11909060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:50:48,754][130385] Avg episode reward: [(0, '47.310'), (1, '38.390')] [2023-10-08 04:50:48,807][00611] Updated weights for policy 0, policy_version 23182 (0.0010) [2023-10-08 04:50:49,175][00611] Updated weights for policy 0, policy_version 23192 (0.0007) [2023-10-08 04:50:50,113][00612] Updated weights for policy 1, policy_version 23300 (0.0008) [2023-10-08 04:50:50,493][00612] Updated weights for policy 1, policy_version 23310 (0.0008) [2023-10-08 04:50:50,852][00612] Updated weights for policy 1, policy_version 23320 (0.0008) [2023-10-08 04:50:52,742][00611] Updated weights for policy 0, policy_version 23202 (0.0009) [2023-10-08 04:50:53,111][00611] Updated weights for policy 0, policy_version 23212 (0.0009) [2023-10-08 04:50:53,475][00611] Updated weights for policy 0, policy_version 23222 (0.0009) [2023-10-08 04:50:53,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 47644672. Throughput: 0: 1848.5, 1: 1823.0. Samples: 11918960. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 04:50:53,754][130385] Avg episode reward: [(0, '47.110'), (1, '39.100')] [2023-10-08 04:50:53,851][00611] Updated weights for policy 0, policy_version 23232 (0.0007) [2023-10-08 04:50:54,570][00612] Updated weights for policy 1, policy_version 23330 (0.0008) [2023-10-08 04:50:54,934][00612] Updated weights for policy 1, policy_version 23340 (0.0010) [2023-10-08 04:50:55,310][00612] Updated weights for policy 1, policy_version 23350 (0.0008) [2023-10-08 04:50:55,670][00612] Updated weights for policy 1, policy_version 23360 (0.0010) [2023-10-08 04:50:57,326][00611] Updated weights for policy 0, policy_version 23242 (0.0011) [2023-10-08 04:50:57,696][00611] Updated weights for policy 0, policy_version 23252 (0.0010) [2023-10-08 04:50:58,062][00611] Updated weights for policy 0, policy_version 23262 (0.0009) [2023-10-08 04:50:58,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 47742976. Throughput: 0: 1849.6, 1: 1833.1. Samples: 11942110. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 04:50:58,755][130385] Avg episode reward: [(0, '45.250'), (1, '41.250')] [2023-10-08 04:50:59,311][00612] Updated weights for policy 1, policy_version 23370 (0.0009) [2023-10-08 04:50:59,676][00612] Updated weights for policy 1, policy_version 23380 (0.0011) [2023-10-08 04:51:00,050][00612] Updated weights for policy 1, policy_version 23390 (0.0011) [2023-10-08 04:51:01,978][00611] Updated weights for policy 0, policy_version 23272 (0.0008) [2023-10-08 04:51:02,349][00611] Updated weights for policy 0, policy_version 23282 (0.0007) [2023-10-08 04:51:02,721][00611] Updated weights for policy 0, policy_version 23292 (0.0007) [2023-10-08 04:51:03,721][00612] Updated weights for policy 1, policy_version 23400 (0.0008) [2023-10-08 04:51:03,754][130385] Fps is (10 sec: 16383.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 47808512. Throughput: 0: 1835.9, 1: 1833.0. Samples: 11963724. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 04:51:03,756][130385] Avg episode reward: [(0, '46.710'), (1, '41.570')] [2023-10-08 04:51:04,086][00612] Updated weights for policy 1, policy_version 23410 (0.0009) [2023-10-08 04:51:04,453][00612] Updated weights for policy 1, policy_version 23420 (0.0009) [2023-10-08 04:51:06,221][00611] Updated weights for policy 0, policy_version 23302 (0.0008) [2023-10-08 04:51:06,604][00611] Updated weights for policy 0, policy_version 23312 (0.0009) [2023-10-08 04:51:06,982][00611] Updated weights for policy 0, policy_version 23322 (0.0007) [2023-10-08 04:51:08,224][00612] Updated weights for policy 1, policy_version 23430 (0.0008) [2023-10-08 04:51:08,604][00612] Updated weights for policy 1, policy_version 23440 (0.0008) [2023-10-08 04:51:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 47874048. Throughput: 0: 1845.7, 1: 1832.0. Samples: 11975046. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 04:51:08,754][130385] Avg episode reward: [(0, '48.590'), (1, '39.900')] [2023-10-08 04:51:08,975][00612] Updated weights for policy 1, policy_version 23450 (0.0009) [2023-10-08 04:51:10,574][00611] Updated weights for policy 0, policy_version 23332 (0.0009) [2023-10-08 04:51:10,953][00611] Updated weights for policy 0, policy_version 23342 (0.0007) [2023-10-08 04:51:11,327][00611] Updated weights for policy 0, policy_version 23352 (0.0008) [2023-10-08 04:51:12,405][00612] Updated weights for policy 1, policy_version 23460 (0.0007) [2023-10-08 04:51:12,781][00612] Updated weights for policy 1, policy_version 23470 (0.0008) [2023-10-08 04:51:13,144][00612] Updated weights for policy 1, policy_version 23480 (0.0009) [2023-10-08 04:51:13,754][130385] Fps is (10 sec: 16384.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 47972352. Throughput: 0: 1843.8, 1: 1832.8. Samples: 11996962. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 04:51:13,754][130385] Avg episode reward: [(0, '47.950'), (1, '41.770')] [2023-10-08 04:51:14,926][00611] Updated weights for policy 0, policy_version 23362 (0.0010) [2023-10-08 04:51:15,307][00611] Updated weights for policy 0, policy_version 23372 (0.0009) [2023-10-08 04:51:15,674][00611] Updated weights for policy 0, policy_version 23382 (0.0011) [2023-10-08 04:51:16,047][00611] Updated weights for policy 0, policy_version 23392 (0.0009) [2023-10-08 04:51:16,697][00612] Updated weights for policy 1, policy_version 23490 (0.0008) [2023-10-08 04:51:17,065][00612] Updated weights for policy 1, policy_version 23500 (0.0007) [2023-10-08 04:51:17,431][00612] Updated weights for policy 1, policy_version 23510 (0.0007) [2023-10-08 04:51:17,802][00612] Updated weights for policy 1, policy_version 23520 (0.0008) [2023-10-08 04:51:18,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 48037888. Throughput: 0: 1852.9, 1: 1836.7. Samples: 12018990. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 04:51:18,755][130385] Avg episode reward: [(0, '46.340'), (1, '42.160')] [2023-10-08 04:51:19,709][00611] Updated weights for policy 0, policy_version 23402 (0.0009) [2023-10-08 04:51:20,087][00611] Updated weights for policy 0, policy_version 23412 (0.0010) [2023-10-08 04:51:20,451][00611] Updated weights for policy 0, policy_version 23422 (0.0010) [2023-10-08 04:51:21,670][00612] Updated weights for policy 1, policy_version 23530 (0.0008) [2023-10-08 04:51:22,045][00612] Updated weights for policy 1, policy_version 23540 (0.0007) [2023-10-08 04:51:22,415][00612] Updated weights for policy 1, policy_version 23550 (0.0007) [2023-10-08 04:51:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 48103424. Throughput: 0: 1853.0, 1: 1835.5. Samples: 12030278. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 04:51:23,754][130385] Avg episode reward: [(0, '46.150'), (1, '40.990')] [2023-10-08 04:51:23,967][00611] Updated weights for policy 0, policy_version 23432 (0.0009) [2023-10-08 04:51:24,336][00611] Updated weights for policy 0, policy_version 23442 (0.0010) [2023-10-08 04:51:24,707][00611] Updated weights for policy 0, policy_version 23452 (0.0010) [2023-10-08 04:51:26,099][00612] Updated weights for policy 1, policy_version 23560 (0.0008) [2023-10-08 04:51:26,461][00612] Updated weights for policy 1, policy_version 23570 (0.0007) [2023-10-08 04:51:26,835][00612] Updated weights for policy 1, policy_version 23580 (0.0008) [2023-10-08 04:51:28,292][00611] Updated weights for policy 0, policy_version 23462 (0.0009) [2023-10-08 04:51:28,658][00611] Updated weights for policy 0, policy_version 23472 (0.0008) [2023-10-08 04:51:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 48168960. Throughput: 0: 1856.3, 1: 1837.5. Samples: 12052252. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-08 04:51:28,755][130385] Avg episode reward: [(0, '46.330'), (1, '42.380')] [2023-10-08 04:51:29,027][00611] Updated weights for policy 0, policy_version 23482 (0.0011) [2023-10-08 04:51:30,352][00612] Updated weights for policy 1, policy_version 23590 (0.0009) [2023-10-08 04:51:30,716][00612] Updated weights for policy 1, policy_version 23600 (0.0010) [2023-10-08 04:51:31,086][00612] Updated weights for policy 1, policy_version 23610 (0.0009) [2023-10-08 04:51:32,744][00611] Updated weights for policy 0, policy_version 23492 (0.0008) [2023-10-08 04:51:33,119][00611] Updated weights for policy 0, policy_version 23502 (0.0008) [2023-10-08 04:51:33,482][00611] Updated weights for policy 0, policy_version 23512 (0.0010) [2023-10-08 04:51:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 48234496. Throughput: 0: 1838.2, 1: 1846.7. Samples: 12074880. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-08 04:51:33,754][130385] Avg episode reward: [(0, '46.010'), (1, '43.610')] [2023-10-08 04:51:34,691][00612] Updated weights for policy 1, policy_version 23620 (0.0009) [2023-10-08 04:51:35,051][00612] Updated weights for policy 1, policy_version 23630 (0.0007) [2023-10-08 04:51:35,418][00612] Updated weights for policy 1, policy_version 23640 (0.0009) [2023-10-08 04:51:37,005][00611] Updated weights for policy 0, policy_version 23522 (0.0008) [2023-10-08 04:51:37,403][00611] Updated weights for policy 0, policy_version 23532 (0.0007) [2023-10-08 04:51:37,779][00611] Updated weights for policy 0, policy_version 23542 (0.0008) [2023-10-08 04:51:38,155][00611] Updated weights for policy 0, policy_version 23552 (0.0007) [2023-10-08 04:51:38,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 48332800. Throughput: 0: 1862.1, 1: 1843.0. Samples: 12085690. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-08 04:51:38,754][130385] Avg episode reward: [(0, '49.310'), (1, '41.710')] [2023-10-08 04:51:38,755][00365] Saving new best policy, reward=49.310! [2023-10-08 04:51:38,969][00612] Updated weights for policy 1, policy_version 23650 (0.0009) [2023-10-08 04:51:39,345][00612] Updated weights for policy 1, policy_version 23660 (0.0009) [2023-10-08 04:51:39,718][00612] Updated weights for policy 1, policy_version 23670 (0.0008) [2023-10-08 04:51:40,088][00612] Updated weights for policy 1, policy_version 23680 (0.0008) [2023-10-08 04:51:41,804][00611] Updated weights for policy 0, policy_version 23562 (0.0011) [2023-10-08 04:51:42,175][00611] Updated weights for policy 0, policy_version 23572 (0.0010) [2023-10-08 04:51:42,548][00611] Updated weights for policy 0, policy_version 23582 (0.0010) [2023-10-08 04:51:43,480][00612] Updated weights for policy 1, policy_version 23690 (0.0008) [2023-10-08 04:51:43,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 48398336. Throughput: 0: 1836.7, 1: 1852.9. Samples: 12108140. Policy #0 lag: (min: 26.0, avg: 27.3, max: 43.0) [2023-10-08 04:51:43,754][130385] Avg episode reward: [(0, '51.170'), (1, '42.620')] [2023-10-08 04:51:43,755][00365] Saving new best policy, reward=51.170! [2023-10-08 04:51:43,850][00612] Updated weights for policy 1, policy_version 23700 (0.0010) [2023-10-08 04:51:44,216][00612] Updated weights for policy 1, policy_version 23710 (0.0011) [2023-10-08 04:51:46,220][00611] Updated weights for policy 0, policy_version 23592 (0.0008) [2023-10-08 04:51:46,584][00611] Updated weights for policy 0, policy_version 23602 (0.0008) [2023-10-08 04:51:46,957][00611] Updated weights for policy 0, policy_version 23612 (0.0009) [2023-10-08 04:51:47,822][00612] Updated weights for policy 1, policy_version 23720 (0.0008) [2023-10-08 04:51:48,197][00612] Updated weights for policy 1, policy_version 23730 (0.0009) [2023-10-08 04:51:48,571][00612] Updated weights for policy 1, policy_version 23740 (0.0009) [2023-10-08 04:51:48,754][130385] Fps is (10 sec: 16383.3, 60 sec: 15291.6, 300 sec: 14773.4). Total num frames: 48496640. Throughput: 0: 1857.2, 1: 1834.8. Samples: 12129866. Policy #0 lag: (min: 26.0, avg: 27.3, max: 43.0) [2023-10-08 04:51:48,756][130385] Avg episode reward: [(0, '49.070'), (1, '42.160')] [2023-10-08 04:51:50,521][00611] Updated weights for policy 0, policy_version 23622 (0.0010) [2023-10-08 04:51:50,898][00611] Updated weights for policy 0, policy_version 23632 (0.0011) [2023-10-08 04:51:51,266][00611] Updated weights for policy 0, policy_version 23642 (0.0007) [2023-10-08 04:51:52,358][00612] Updated weights for policy 1, policy_version 23750 (0.0010) [2023-10-08 04:51:52,736][00612] Updated weights for policy 1, policy_version 23760 (0.0011) [2023-10-08 04:51:53,104][00612] Updated weights for policy 1, policy_version 23770 (0.0008) [2023-10-08 04:51:53,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 48562176. Throughput: 0: 1837.6, 1: 1856.2. Samples: 12141268. Policy #0 lag: (min: 26.0, avg: 27.3, max: 43.0) [2023-10-08 04:51:53,754][130385] Avg episode reward: [(0, '46.020'), (1, '43.020')] [2023-10-08 04:51:54,990][00611] Updated weights for policy 0, policy_version 23652 (0.0007) [2023-10-08 04:51:55,358][00611] Updated weights for policy 0, policy_version 23662 (0.0008) [2023-10-08 04:51:55,734][00611] Updated weights for policy 0, policy_version 23672 (0.0009) [2023-10-08 04:51:57,027][00612] Updated weights for policy 1, policy_version 23780 (0.0009) [2023-10-08 04:51:57,417][00612] Updated weights for policy 1, policy_version 23790 (0.0007) [2023-10-08 04:51:57,789][00612] Updated weights for policy 1, policy_version 23800 (0.0009) [2023-10-08 04:51:58,754][130385] Fps is (10 sec: 13107.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 48627712. Throughput: 0: 1856.1, 1: 1839.6. Samples: 12163270. Policy #0 lag: (min: 26.0, avg: 27.3, max: 43.0) [2023-10-08 04:51:58,754][130385] Avg episode reward: [(0, '43.600'), (1, '42.790')] [2023-10-08 04:51:59,346][00611] Updated weights for policy 0, policy_version 23682 (0.0008) [2023-10-08 04:51:59,711][00611] Updated weights for policy 0, policy_version 23692 (0.0011) [2023-10-08 04:52:00,083][00611] Updated weights for policy 0, policy_version 23702 (0.0008) [2023-10-08 04:52:00,460][00611] Updated weights for policy 0, policy_version 23712 (0.0009) [2023-10-08 04:52:01,450][00612] Updated weights for policy 1, policy_version 23810 (0.0008) [2023-10-08 04:52:01,821][00612] Updated weights for policy 1, policy_version 23820 (0.0008) [2023-10-08 04:52:02,190][00612] Updated weights for policy 1, policy_version 23830 (0.0008) [2023-10-08 04:52:02,556][00612] Updated weights for policy 1, policy_version 23840 (0.0008) [2023-10-08 04:52:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 48693248. Throughput: 0: 1848.0, 1: 1845.2. Samples: 12185182. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-08 04:52:03,754][130385] Avg episode reward: [(0, '43.980'), (1, '42.500')] [2023-10-08 04:52:04,169][00611] Updated weights for policy 0, policy_version 23722 (0.0009) [2023-10-08 04:52:04,546][00611] Updated weights for policy 0, policy_version 23732 (0.0008) [2023-10-08 04:52:04,917][00611] Updated weights for policy 0, policy_version 23742 (0.0010) [2023-10-08 04:52:06,026][00612] Updated weights for policy 1, policy_version 23850 (0.0009) [2023-10-08 04:52:06,395][00612] Updated weights for policy 1, policy_version 23860 (0.0009) [2023-10-08 04:52:06,773][00612] Updated weights for policy 1, policy_version 23870 (0.0009) [2023-10-08 04:52:08,625][00611] Updated weights for policy 0, policy_version 23752 (0.0009) [2023-10-08 04:52:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 48758784. Throughput: 0: 1846.7, 1: 1838.1. Samples: 12196094. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-08 04:52:08,754][130385] Avg episode reward: [(0, '44.710'), (1, '41.360')] [2023-10-08 04:52:08,996][00611] Updated weights for policy 0, policy_version 23762 (0.0011) [2023-10-08 04:52:09,359][00611] Updated weights for policy 0, policy_version 23772 (0.0009) [2023-10-08 04:52:10,259][00612] Updated weights for policy 1, policy_version 23880 (0.0007) [2023-10-08 04:52:10,634][00612] Updated weights for policy 1, policy_version 23890 (0.0009) [2023-10-08 04:52:11,003][00612] Updated weights for policy 1, policy_version 23900 (0.0007) [2023-10-08 04:52:12,868][00611] Updated weights for policy 0, policy_version 23782 (0.0008) [2023-10-08 04:52:13,249][00611] Updated weights for policy 0, policy_version 23792 (0.0008) [2023-10-08 04:52:13,611][00611] Updated weights for policy 0, policy_version 23802 (0.0008) [2023-10-08 04:52:13,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 48824320. Throughput: 0: 1840.4, 1: 1848.6. Samples: 12218258. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-08 04:52:13,755][130385] Avg episode reward: [(0, '46.930'), (1, '40.130')] [2023-10-08 04:52:14,632][00612] Updated weights for policy 1, policy_version 23910 (0.0007) [2023-10-08 04:52:15,003][00612] Updated weights for policy 1, policy_version 23920 (0.0008) [2023-10-08 04:52:15,372][00612] Updated weights for policy 1, policy_version 23930 (0.0009) [2023-10-08 04:52:17,247][00611] Updated weights for policy 0, policy_version 23812 (0.0008) [2023-10-08 04:52:17,618][00611] Updated weights for policy 0, policy_version 23822 (0.0009) [2023-10-08 04:52:17,979][00611] Updated weights for policy 0, policy_version 23832 (0.0008) [2023-10-08 04:52:18,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 48922624. Throughput: 0: 1827.9, 1: 1849.2. Samples: 12240348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:52:18,755][130385] Avg episode reward: [(0, '47.470'), (1, '41.810')] [2023-10-08 04:52:18,769][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000023840_24412160.pth... [2023-10-08 04:52:18,810][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000022112_22642688.pth [2023-10-08 04:52:18,956][00612] Updated weights for policy 1, policy_version 23940 (0.0008) [2023-10-08 04:52:19,321][00612] Updated weights for policy 1, policy_version 23950 (0.0008) [2023-10-08 04:52:19,685][00612] Updated weights for policy 1, policy_version 23960 (0.0008) [2023-10-08 04:52:19,974][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000023968_24543232.pth... [2023-10-08 04:52:20,003][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000022240_22773760.pth [2023-10-08 04:52:21,550][00611] Updated weights for policy 0, policy_version 23842 (0.0008) [2023-10-08 04:52:21,917][00611] Updated weights for policy 0, policy_version 23852 (0.0007) [2023-10-08 04:52:22,299][00611] Updated weights for policy 0, policy_version 23862 (0.0011) [2023-10-08 04:52:22,678][00611] Updated weights for policy 0, policy_version 23872 (0.0010) [2023-10-08 04:52:23,234][00612] Updated weights for policy 1, policy_version 23970 (0.0007) [2023-10-08 04:52:23,611][00612] Updated weights for policy 1, policy_version 23980 (0.0007) [2023-10-08 04:52:23,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 48988160. Throughput: 0: 1836.8, 1: 1850.0. Samples: 12251600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:52:23,754][130385] Avg episode reward: [(0, '44.260'), (1, '42.230')] [2023-10-08 04:52:23,973][00612] Updated weights for policy 1, policy_version 23990 (0.0008) [2023-10-08 04:52:24,345][00612] Updated weights for policy 1, policy_version 24000 (0.0008) [2023-10-08 04:52:26,414][00611] Updated weights for policy 0, policy_version 23882 (0.0010) [2023-10-08 04:52:26,795][00611] Updated weights for policy 0, policy_version 23892 (0.0010) [2023-10-08 04:52:27,153][00611] Updated weights for policy 0, policy_version 23902 (0.0010) [2023-10-08 04:52:27,947][00612] Updated weights for policy 1, policy_version 24010 (0.0010) [2023-10-08 04:52:28,319][00612] Updated weights for policy 1, policy_version 24020 (0.0007) [2023-10-08 04:52:28,681][00612] Updated weights for policy 1, policy_version 24030 (0.0007) [2023-10-08 04:52:28,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 49086464. Throughput: 0: 1827.7, 1: 1848.8. Samples: 12273580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:52:28,754][130385] Avg episode reward: [(0, '43.600'), (1, '42.280')] [2023-10-08 04:52:30,942][00611] Updated weights for policy 0, policy_version 23912 (0.0009) [2023-10-08 04:52:31,311][00611] Updated weights for policy 0, policy_version 23922 (0.0008) [2023-10-08 04:52:31,685][00611] Updated weights for policy 0, policy_version 23932 (0.0007) [2023-10-08 04:52:32,522][00612] Updated weights for policy 1, policy_version 24040 (0.0010) [2023-10-08 04:52:32,890][00612] Updated weights for policy 1, policy_version 24050 (0.0007) [2023-10-08 04:52:33,256][00612] Updated weights for policy 1, policy_version 24060 (0.0008) [2023-10-08 04:52:33,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 49152000. Throughput: 0: 1836.6, 1: 1830.7. Samples: 12294894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:52:33,755][130385] Avg episode reward: [(0, '43.920'), (1, '40.900')] [2023-10-08 04:52:35,292][00611] Updated weights for policy 0, policy_version 23942 (0.0008) [2023-10-08 04:52:35,662][00611] Updated weights for policy 0, policy_version 23952 (0.0009) [2023-10-08 04:52:36,034][00611] Updated weights for policy 0, policy_version 23962 (0.0009) [2023-10-08 04:52:36,938][00612] Updated weights for policy 1, policy_version 24070 (0.0008) [2023-10-08 04:52:37,302][00612] Updated weights for policy 1, policy_version 24080 (0.0008) [2023-10-08 04:52:37,671][00612] Updated weights for policy 1, policy_version 24090 (0.0009) [2023-10-08 04:52:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 49217536. Throughput: 0: 1829.3, 1: 1840.9. Samples: 12306428. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 04:52:38,754][130385] Avg episode reward: [(0, '44.170'), (1, '38.960')] [2023-10-08 04:52:39,747][00611] Updated weights for policy 0, policy_version 23972 (0.0010) [2023-10-08 04:52:40,118][00611] Updated weights for policy 0, policy_version 23982 (0.0009) [2023-10-08 04:52:40,492][00611] Updated weights for policy 0, policy_version 23992 (0.0007) [2023-10-08 04:52:41,235][00612] Updated weights for policy 1, policy_version 24100 (0.0009) [2023-10-08 04:52:41,602][00612] Updated weights for policy 1, policy_version 24110 (0.0010) [2023-10-08 04:52:41,975][00612] Updated weights for policy 1, policy_version 24120 (0.0007) [2023-10-08 04:52:43,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 49283072. Throughput: 0: 1834.4, 1: 1829.2. Samples: 12328130. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 04:52:43,755][130385] Avg episode reward: [(0, '46.220'), (1, '40.200')] [2023-10-08 04:52:44,090][00611] Updated weights for policy 0, policy_version 24002 (0.0008) [2023-10-08 04:52:44,457][00611] Updated weights for policy 0, policy_version 24012 (0.0009) [2023-10-08 04:52:44,839][00611] Updated weights for policy 0, policy_version 24022 (0.0010) [2023-10-08 04:52:45,221][00611] Updated weights for policy 0, policy_version 24032 (0.0009) [2023-10-08 04:52:45,440][00612] Updated weights for policy 1, policy_version 24130 (0.0008) [2023-10-08 04:52:45,856][00612] Updated weights for policy 1, policy_version 24140 (0.0009) [2023-10-08 04:52:46,222][00612] Updated weights for policy 1, policy_version 24150 (0.0011) [2023-10-08 04:52:46,590][00612] Updated weights for policy 1, policy_version 24160 (0.0010) [2023-10-08 04:52:48,737][00611] Updated weights for policy 0, policy_version 24042 (0.0010) [2023-10-08 04:52:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.6, 300 sec: 14662.3). Total num frames: 49348608. Throughput: 0: 1834.5, 1: 1854.6. Samples: 12351190. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 04:52:48,754][130385] Avg episode reward: [(0, '52.890'), (1, '38.790')] [2023-10-08 04:52:49,105][00611] Updated weights for policy 0, policy_version 24052 (0.0011) [2023-10-08 04:52:49,474][00611] Updated weights for policy 0, policy_version 24062 (0.0009) [2023-10-08 04:52:49,545][00365] Saving new best policy, reward=52.890! [2023-10-08 04:52:50,208][00612] Updated weights for policy 1, policy_version 24170 (0.0010) [2023-10-08 04:52:50,576][00612] Updated weights for policy 1, policy_version 24180 (0.0009) [2023-10-08 04:52:50,945][00612] Updated weights for policy 1, policy_version 24190 (0.0010) [2023-10-08 04:52:53,044][00611] Updated weights for policy 0, policy_version 24072 (0.0009) [2023-10-08 04:52:53,424][00611] Updated weights for policy 0, policy_version 24082 (0.0007) [2023-10-08 04:52:53,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 49414144. Throughput: 0: 1838.2, 1: 1832.4. Samples: 12361274. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 04:52:53,755][130385] Avg episode reward: [(0, '50.560'), (1, '37.610')] [2023-10-08 04:52:53,810][00611] Updated weights for policy 0, policy_version 24092 (0.0008) [2023-10-08 04:52:54,569][00612] Updated weights for policy 1, policy_version 24200 (0.0010) [2023-10-08 04:52:54,931][00612] Updated weights for policy 1, policy_version 24210 (0.0007) [2023-10-08 04:52:55,302][00612] Updated weights for policy 1, policy_version 24220 (0.0009) [2023-10-08 04:52:57,598][00611] Updated weights for policy 0, policy_version 24102 (0.0008) [2023-10-08 04:52:57,982][00611] Updated weights for policy 0, policy_version 24112 (0.0007) [2023-10-08 04:52:58,349][00611] Updated weights for policy 0, policy_version 24122 (0.0009) [2023-10-08 04:52:58,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 49512448. Throughput: 0: 1834.4, 1: 1854.7. Samples: 12384266. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:52:58,754][130385] Avg episode reward: [(0, '48.770'), (1, '38.640')] [2023-10-08 04:52:59,058][00612] Updated weights for policy 1, policy_version 24230 (0.0009) [2023-10-08 04:52:59,431][00612] Updated weights for policy 1, policy_version 24240 (0.0010) [2023-10-08 04:52:59,798][00612] Updated weights for policy 1, policy_version 24250 (0.0009) [2023-10-08 04:53:01,905][00611] Updated weights for policy 0, policy_version 24132 (0.0008) [2023-10-08 04:53:02,272][00611] Updated weights for policy 0, policy_version 24142 (0.0010) [2023-10-08 04:53:02,647][00611] Updated weights for policy 0, policy_version 24152 (0.0009) [2023-10-08 04:53:03,526][00612] Updated weights for policy 1, policy_version 24260 (0.0008) [2023-10-08 04:53:03,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 49577984. Throughput: 0: 1831.3, 1: 1846.4. Samples: 12405848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:53:03,755][130385] Avg episode reward: [(0, '47.460'), (1, '39.810')] [2023-10-08 04:53:03,891][00612] Updated weights for policy 1, policy_version 24270 (0.0007) [2023-10-08 04:53:04,257][00612] Updated weights for policy 1, policy_version 24280 (0.0007) [2023-10-08 04:53:06,216][00611] Updated weights for policy 0, policy_version 24162 (0.0007) [2023-10-08 04:53:06,592][00611] Updated weights for policy 0, policy_version 24172 (0.0007) [2023-10-08 04:53:06,968][00611] Updated weights for policy 0, policy_version 24182 (0.0008) [2023-10-08 04:53:07,340][00611] Updated weights for policy 0, policy_version 24192 (0.0009) [2023-10-08 04:53:07,787][00612] Updated weights for policy 1, policy_version 24290 (0.0008) [2023-10-08 04:53:08,146][00612] Updated weights for policy 1, policy_version 24300 (0.0010) [2023-10-08 04:53:08,523][00612] Updated weights for policy 1, policy_version 24310 (0.0009) [2023-10-08 04:53:08,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 49643520. Throughput: 0: 1838.2, 1: 1847.9. Samples: 12417474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:53:08,755][130385] Avg episode reward: [(0, '48.310'), (1, '39.840')] [2023-10-08 04:53:08,887][00612] Updated weights for policy 1, policy_version 24320 (0.0007) [2023-10-08 04:53:10,991][00611] Updated weights for policy 0, policy_version 24202 (0.0008) [2023-10-08 04:53:11,363][00611] Updated weights for policy 0, policy_version 24212 (0.0007) [2023-10-08 04:53:11,735][00611] Updated weights for policy 0, policy_version 24222 (0.0007) [2023-10-08 04:53:12,374][00612] Updated weights for policy 1, policy_version 24330 (0.0010) [2023-10-08 04:53:12,744][00612] Updated weights for policy 1, policy_version 24340 (0.0009) [2023-10-08 04:53:13,124][00612] Updated weights for policy 1, policy_version 24350 (0.0008) [2023-10-08 04:53:13,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 49741824. Throughput: 0: 1837.1, 1: 1845.1. Samples: 12439280. Policy #0 lag: (min: 2.0, avg: 26.7, max: 32.0) [2023-10-08 04:53:13,755][130385] Avg episode reward: [(0, '50.630'), (1, '40.070')] [2023-10-08 04:53:15,554][00611] Updated weights for policy 0, policy_version 24232 (0.0008) [2023-10-08 04:53:15,932][00611] Updated weights for policy 0, policy_version 24242 (0.0009) [2023-10-08 04:53:16,287][00611] Updated weights for policy 0, policy_version 24252 (0.0007) [2023-10-08 04:53:16,713][00612] Updated weights for policy 1, policy_version 24360 (0.0009) [2023-10-08 04:53:17,080][00612] Updated weights for policy 1, policy_version 24370 (0.0007) [2023-10-08 04:53:17,448][00612] Updated weights for policy 1, policy_version 24380 (0.0007) [2023-10-08 04:53:18,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 49807360. Throughput: 0: 1845.8, 1: 1849.2. Samples: 12461170. Policy #0 lag: (min: 2.0, avg: 26.7, max: 32.0) [2023-10-08 04:53:18,755][130385] Avg episode reward: [(0, '49.500'), (1, '41.840')] [2023-10-08 04:53:19,837][00611] Updated weights for policy 0, policy_version 24262 (0.0009) [2023-10-08 04:53:20,211][00611] Updated weights for policy 0, policy_version 24272 (0.0008) [2023-10-08 04:53:20,593][00611] Updated weights for policy 0, policy_version 24282 (0.0009) [2023-10-08 04:53:21,142][00612] Updated weights for policy 1, policy_version 24390 (0.0010) [2023-10-08 04:53:21,511][00612] Updated weights for policy 1, policy_version 24400 (0.0008) [2023-10-08 04:53:21,888][00612] Updated weights for policy 1, policy_version 24410 (0.0008) [2023-10-08 04:53:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 49872896. Throughput: 0: 1838.2, 1: 1850.3. Samples: 12472410. Policy #0 lag: (min: 2.0, avg: 26.7, max: 32.0) [2023-10-08 04:53:23,755][130385] Avg episode reward: [(0, '46.640'), (1, '41.130')] [2023-10-08 04:53:24,159][00611] Updated weights for policy 0, policy_version 24292 (0.0008) [2023-10-08 04:53:24,534][00611] Updated weights for policy 0, policy_version 24302 (0.0007) [2023-10-08 04:53:24,897][00611] Updated weights for policy 0, policy_version 24312 (0.0008) [2023-10-08 04:53:25,616][00612] Updated weights for policy 1, policy_version 24420 (0.0009) [2023-10-08 04:53:25,982][00612] Updated weights for policy 1, policy_version 24430 (0.0009) [2023-10-08 04:53:26,339][00612] Updated weights for policy 1, policy_version 24440 (0.0008) [2023-10-08 04:53:28,533][00611] Updated weights for policy 0, policy_version 24322 (0.0010) [2023-10-08 04:53:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 49938432. Throughput: 0: 1845.0, 1: 1845.7. Samples: 12494212. Policy #0 lag: (min: 2.0, avg: 26.7, max: 32.0) [2023-10-08 04:53:28,755][130385] Avg episode reward: [(0, '41.510'), (1, '42.730')] [2023-10-08 04:53:28,906][00611] Updated weights for policy 0, policy_version 24332 (0.0007) [2023-10-08 04:53:29,285][00611] Updated weights for policy 0, policy_version 24342 (0.0008) [2023-10-08 04:53:29,662][00611] Updated weights for policy 0, policy_version 24352 (0.0009) [2023-10-08 04:53:30,152][00612] Updated weights for policy 1, policy_version 24450 (0.0008) [2023-10-08 04:53:30,523][00612] Updated weights for policy 1, policy_version 24460 (0.0008) [2023-10-08 04:53:30,892][00612] Updated weights for policy 1, policy_version 24470 (0.0009) [2023-10-08 04:53:31,262][00612] Updated weights for policy 1, policy_version 24480 (0.0009) [2023-10-08 04:53:33,325][00611] Updated weights for policy 0, policy_version 24362 (0.0007) [2023-10-08 04:53:33,701][00611] Updated weights for policy 0, policy_version 24372 (0.0008) [2023-10-08 04:53:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 50003968. Throughput: 0: 1842.7, 1: 1848.2. Samples: 12517278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:53:33,755][130385] Avg episode reward: [(0, '42.790'), (1, '45.930')] [2023-10-08 04:53:34,070][00611] Updated weights for policy 0, policy_version 24382 (0.0007) [2023-10-08 04:53:34,934][00612] Updated weights for policy 1, policy_version 24490 (0.0007) [2023-10-08 04:53:35,306][00612] Updated weights for policy 1, policy_version 24500 (0.0008) [2023-10-08 04:53:35,670][00612] Updated weights for policy 1, policy_version 24510 (0.0010) [2023-10-08 04:53:37,665][00611] Updated weights for policy 0, policy_version 24392 (0.0010) [2023-10-08 04:53:38,030][00611] Updated weights for policy 0, policy_version 24402 (0.0010) [2023-10-08 04:53:38,404][00611] Updated weights for policy 0, policy_version 24412 (0.0010) [2023-10-08 04:53:38,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 50102272. Throughput: 0: 1851.4, 1: 1842.4. Samples: 12527496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:53:38,755][130385] Avg episode reward: [(0, '44.570'), (1, '45.760')] [2023-10-08 04:53:39,356][00612] Updated weights for policy 1, policy_version 24520 (0.0009) [2023-10-08 04:53:39,717][00612] Updated weights for policy 1, policy_version 24530 (0.0007) [2023-10-08 04:53:40,082][00612] Updated weights for policy 1, policy_version 24540 (0.0009) [2023-10-08 04:53:41,990][00611] Updated weights for policy 0, policy_version 24422 (0.0008) [2023-10-08 04:53:42,359][00611] Updated weights for policy 0, policy_version 24432 (0.0007) [2023-10-08 04:53:42,734][00611] Updated weights for policy 0, policy_version 24442 (0.0007) [2023-10-08 04:53:43,689][00612] Updated weights for policy 1, policy_version 24550 (0.0008) [2023-10-08 04:53:43,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 50167808. Throughput: 0: 1850.6, 1: 1841.5. Samples: 12550412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:53:43,754][130385] Avg episode reward: [(0, '41.740'), (1, '44.530')] [2023-10-08 04:53:44,053][00612] Updated weights for policy 1, policy_version 24560 (0.0007) [2023-10-08 04:53:44,428][00612] Updated weights for policy 1, policy_version 24570 (0.0008) [2023-10-08 04:53:46,409][00611] Updated weights for policy 0, policy_version 24452 (0.0008) [2023-10-08 04:53:46,780][00611] Updated weights for policy 0, policy_version 24462 (0.0007) [2023-10-08 04:53:47,150][00611] Updated weights for policy 0, policy_version 24472 (0.0007) [2023-10-08 04:53:47,987][00612] Updated weights for policy 1, policy_version 24580 (0.0009) [2023-10-08 04:53:48,354][00612] Updated weights for policy 1, policy_version 24590 (0.0011) [2023-10-08 04:53:48,722][00612] Updated weights for policy 1, policy_version 24600 (0.0007) [2023-10-08 04:53:48,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 50233344. Throughput: 0: 1855.0, 1: 1836.4. Samples: 12571964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:53:48,755][130385] Avg episode reward: [(0, '42.290'), (1, '44.510')] [2023-10-08 04:53:50,685][00611] Updated weights for policy 0, policy_version 24482 (0.0008) [2023-10-08 04:53:51,056][00611] Updated weights for policy 0, policy_version 24492 (0.0008) [2023-10-08 04:53:51,424][00611] Updated weights for policy 0, policy_version 24502 (0.0007) [2023-10-08 04:53:51,799][00611] Updated weights for policy 0, policy_version 24512 (0.0009) [2023-10-08 04:53:52,480][00612] Updated weights for policy 1, policy_version 24610 (0.0008) [2023-10-08 04:53:52,850][00612] Updated weights for policy 1, policy_version 24620 (0.0008) [2023-10-08 04:53:53,215][00612] Updated weights for policy 1, policy_version 24630 (0.0008) [2023-10-08 04:53:53,576][00612] Updated weights for policy 1, policy_version 24640 (0.0010) [2023-10-08 04:53:53,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 50331648. Throughput: 0: 1842.7, 1: 1846.2. Samples: 12583472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:53:53,754][130385] Avg episode reward: [(0, '41.150'), (1, '45.180')] [2023-10-08 04:53:55,360][00611] Updated weights for policy 0, policy_version 24522 (0.0008) [2023-10-08 04:53:55,736][00611] Updated weights for policy 0, policy_version 24532 (0.0008) [2023-10-08 04:53:56,107][00611] Updated weights for policy 0, policy_version 24542 (0.0007) [2023-10-08 04:53:57,167][00612] Updated weights for policy 1, policy_version 24650 (0.0010) [2023-10-08 04:53:57,544][00612] Updated weights for policy 1, policy_version 24660 (0.0010) [2023-10-08 04:53:57,906][00612] Updated weights for policy 1, policy_version 24670 (0.0010) [2023-10-08 04:53:58,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 50397184. Throughput: 0: 1854.7, 1: 1829.2. Samples: 12605056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:53:58,755][130385] Avg episode reward: [(0, '41.590'), (1, '43.960')] [2023-10-08 04:53:59,473][00611] Updated weights for policy 0, policy_version 24552 (0.0009) [2023-10-08 04:53:59,849][00611] Updated weights for policy 0, policy_version 24562 (0.0011) [2023-10-08 04:54:00,213][00611] Updated weights for policy 0, policy_version 24572 (0.0009) [2023-10-08 04:54:01,568][00612] Updated weights for policy 1, policy_version 24680 (0.0011) [2023-10-08 04:54:01,940][00612] Updated weights for policy 1, policy_version 24690 (0.0009) [2023-10-08 04:54:02,308][00612] Updated weights for policy 1, policy_version 24700 (0.0007) [2023-10-08 04:54:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 50462720. Throughput: 0: 1858.4, 1: 1833.9. Samples: 12627320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:54:03,754][130385] Avg episode reward: [(0, '42.170'), (1, '45.680')] [2023-10-08 04:54:03,941][00611] Updated weights for policy 0, policy_version 24582 (0.0010) [2023-10-08 04:54:04,324][00611] Updated weights for policy 0, policy_version 24592 (0.0008) [2023-10-08 04:54:04,683][00611] Updated weights for policy 0, policy_version 24602 (0.0007) [2023-10-08 04:54:05,742][00612] Updated weights for policy 1, policy_version 24710 (0.0008) [2023-10-08 04:54:06,112][00612] Updated weights for policy 1, policy_version 24720 (0.0009) [2023-10-08 04:54:06,477][00612] Updated weights for policy 1, policy_version 24730 (0.0009) [2023-10-08 04:54:08,469][00611] Updated weights for policy 0, policy_version 24612 (0.0007) [2023-10-08 04:54:08,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 50528256. Throughput: 0: 1858.5, 1: 1824.1. Samples: 12638130. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 04:54:08,754][130385] Avg episode reward: [(0, '43.890'), (1, '44.410')] [2023-10-08 04:54:08,841][00611] Updated weights for policy 0, policy_version 24622 (0.0007) [2023-10-08 04:54:09,214][00611] Updated weights for policy 0, policy_version 24632 (0.0007) [2023-10-08 04:54:10,031][00612] Updated weights for policy 1, policy_version 24740 (0.0010) [2023-10-08 04:54:10,412][00612] Updated weights for policy 1, policy_version 24750 (0.0011) [2023-10-08 04:54:10,771][00612] Updated weights for policy 1, policy_version 24760 (0.0008) [2023-10-08 04:54:12,831][00611] Updated weights for policy 0, policy_version 24642 (0.0008) [2023-10-08 04:54:13,194][00611] Updated weights for policy 0, policy_version 24652 (0.0011) [2023-10-08 04:54:13,568][00611] Updated weights for policy 0, policy_version 24662 (0.0010) [2023-10-08 04:54:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 50593792. Throughput: 0: 1860.6, 1: 1839.4. Samples: 12660710. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 04:54:13,754][130385] Avg episode reward: [(0, '45.210'), (1, '42.070')] [2023-10-08 04:54:13,938][00611] Updated weights for policy 0, policy_version 24672 (0.0011) [2023-10-08 04:54:14,387][00612] Updated weights for policy 1, policy_version 24770 (0.0010) [2023-10-08 04:54:14,752][00612] Updated weights for policy 1, policy_version 24780 (0.0009) [2023-10-08 04:54:15,124][00612] Updated weights for policy 1, policy_version 24790 (0.0008) [2023-10-08 04:54:15,486][00612] Updated weights for policy 1, policy_version 24800 (0.0008) [2023-10-08 04:54:17,472][00611] Updated weights for policy 0, policy_version 24682 (0.0008) [2023-10-08 04:54:17,845][00611] Updated weights for policy 0, policy_version 24692 (0.0007) [2023-10-08 04:54:18,207][00611] Updated weights for policy 0, policy_version 24702 (0.0007) [2023-10-08 04:54:18,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 50692096. Throughput: 0: 1836.8, 1: 1841.9. Samples: 12682820. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 04:54:18,755][130385] Avg episode reward: [(0, '41.220'), (1, '44.620')] [2023-10-08 04:54:18,768][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000024800_25395200.pth... [2023-10-08 04:54:18,768][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000024704_25296896.pth... [2023-10-08 04:54:18,799][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000022976_23527424.pth [2023-10-08 04:54:18,808][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000023104_23658496.pth [2023-10-08 04:54:19,330][00612] Updated weights for policy 1, policy_version 24810 (0.0008) [2023-10-08 04:54:19,710][00612] Updated weights for policy 1, policy_version 24820 (0.0008) [2023-10-08 04:54:20,082][00612] Updated weights for policy 1, policy_version 24830 (0.0008) [2023-10-08 04:54:21,763][00611] Updated weights for policy 0, policy_version 24712 (0.0007) [2023-10-08 04:54:22,144][00611] Updated weights for policy 0, policy_version 24722 (0.0007) [2023-10-08 04:54:22,503][00611] Updated weights for policy 0, policy_version 24732 (0.0007) [2023-10-08 04:54:23,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 50757632. Throughput: 0: 1858.4, 1: 1841.2. Samples: 12693978. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 04:54:23,754][130385] Avg episode reward: [(0, '44.440'), (1, '44.740')] [2023-10-08 04:54:23,781][00612] Updated weights for policy 1, policy_version 24840 (0.0008) [2023-10-08 04:54:24,153][00612] Updated weights for policy 1, policy_version 24850 (0.0007) [2023-10-08 04:54:24,509][00612] Updated weights for policy 1, policy_version 24860 (0.0009) [2023-10-08 04:54:25,915][00611] Updated weights for policy 0, policy_version 24742 (0.0007) [2023-10-08 04:54:26,289][00611] Updated weights for policy 0, policy_version 24752 (0.0007) [2023-10-08 04:54:26,664][00611] Updated weights for policy 0, policy_version 24762 (0.0008) [2023-10-08 04:54:27,977][00612] Updated weights for policy 1, policy_version 24870 (0.0009) [2023-10-08 04:54:28,352][00612] Updated weights for policy 1, policy_version 24880 (0.0009) [2023-10-08 04:54:28,715][00612] Updated weights for policy 1, policy_version 24890 (0.0008) [2023-10-08 04:54:28,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 50823168. Throughput: 0: 1834.1, 1: 1842.6. Samples: 12715862. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 04:54:28,754][130385] Avg episode reward: [(0, '45.830'), (1, '43.990')] [2023-10-08 04:54:30,289][00611] Updated weights for policy 0, policy_version 24772 (0.0008) [2023-10-08 04:54:30,654][00611] Updated weights for policy 0, policy_version 24782 (0.0007) [2023-10-08 04:54:31,035][00611] Updated weights for policy 0, policy_version 24792 (0.0008) [2023-10-08 04:54:32,492][00612] Updated weights for policy 1, policy_version 24900 (0.0007) [2023-10-08 04:54:32,860][00612] Updated weights for policy 1, policy_version 24910 (0.0007) [2023-10-08 04:54:33,224][00612] Updated weights for policy 1, policy_version 24920 (0.0008) [2023-10-08 04:54:33,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 50921472. Throughput: 0: 1857.9, 1: 1825.6. Samples: 12737720. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 04:54:33,754][130385] Avg episode reward: [(0, '45.000'), (1, '43.050')] [2023-10-08 04:54:34,720][00611] Updated weights for policy 0, policy_version 24802 (0.0009) [2023-10-08 04:54:35,091][00611] Updated weights for policy 0, policy_version 24812 (0.0008) [2023-10-08 04:54:35,462][00611] Updated weights for policy 0, policy_version 24822 (0.0008) [2023-10-08 04:54:35,832][00611] Updated weights for policy 0, policy_version 24832 (0.0007) [2023-10-08 04:54:36,826][00612] Updated weights for policy 1, policy_version 24930 (0.0007) [2023-10-08 04:54:37,199][00612] Updated weights for policy 1, policy_version 24940 (0.0011) [2023-10-08 04:54:37,565][00612] Updated weights for policy 1, policy_version 24950 (0.0011) [2023-10-08 04:54:37,933][00612] Updated weights for policy 1, policy_version 24960 (0.0009) [2023-10-08 04:54:38,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 50987008. Throughput: 0: 1830.3, 1: 1842.0. Samples: 12748728. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 04:54:38,754][130385] Avg episode reward: [(0, '46.300'), (1, '42.890')] [2023-10-08 04:54:39,496][00611] Updated weights for policy 0, policy_version 24842 (0.0007) [2023-10-08 04:54:39,872][00611] Updated weights for policy 0, policy_version 24852 (0.0008) [2023-10-08 04:54:40,238][00611] Updated weights for policy 0, policy_version 24862 (0.0008) [2023-10-08 04:54:41,722][00612] Updated weights for policy 1, policy_version 24970 (0.0007) [2023-10-08 04:54:42,097][00612] Updated weights for policy 1, policy_version 24980 (0.0009) [2023-10-08 04:54:42,462][00612] Updated weights for policy 1, policy_version 24990 (0.0007) [2023-10-08 04:54:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 51052544. Throughput: 0: 1853.8, 1: 1826.9. Samples: 12770686. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-08 04:54:43,754][130385] Avg episode reward: [(0, '46.560'), (1, '43.460')] [2023-10-08 04:54:43,872][00611] Updated weights for policy 0, policy_version 24872 (0.0008) [2023-10-08 04:54:44,253][00611] Updated weights for policy 0, policy_version 24882 (0.0009) [2023-10-08 04:54:44,627][00611] Updated weights for policy 0, policy_version 24892 (0.0009) [2023-10-08 04:54:46,123][00612] Updated weights for policy 1, policy_version 25000 (0.0009) [2023-10-08 04:54:46,490][00612] Updated weights for policy 1, policy_version 25010 (0.0007) [2023-10-08 04:54:46,856][00612] Updated weights for policy 1, policy_version 25020 (0.0008) [2023-10-08 04:54:48,436][00611] Updated weights for policy 0, policy_version 24902 (0.0008) [2023-10-08 04:54:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 51118080. Throughput: 0: 1844.5, 1: 1837.7. Samples: 12793020. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-08 04:54:48,754][130385] Avg episode reward: [(0, '47.080'), (1, '41.450')] [2023-10-08 04:54:48,810][00611] Updated weights for policy 0, policy_version 24912 (0.0009) [2023-10-08 04:54:49,181][00611] Updated weights for policy 0, policy_version 24922 (0.0008) [2023-10-08 04:54:50,603][00612] Updated weights for policy 1, policy_version 25030 (0.0009) [2023-10-08 04:54:50,967][00612] Updated weights for policy 1, policy_version 25040 (0.0010) [2023-10-08 04:54:51,332][00612] Updated weights for policy 1, policy_version 25050 (0.0011) [2023-10-08 04:54:52,719][00611] Updated weights for policy 0, policy_version 24932 (0.0008) [2023-10-08 04:54:53,092][00611] Updated weights for policy 0, policy_version 24942 (0.0009) [2023-10-08 04:54:53,460][00611] Updated weights for policy 0, policy_version 24952 (0.0009) [2023-10-08 04:54:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 51183616. Throughput: 0: 1844.0, 1: 1830.5. Samples: 12803486. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-08 04:54:53,754][130385] Avg episode reward: [(0, '46.570'), (1, '40.960')] [2023-10-08 04:54:55,032][00612] Updated weights for policy 1, policy_version 25060 (0.0010) [2023-10-08 04:54:55,400][00612] Updated weights for policy 1, policy_version 25070 (0.0007) [2023-10-08 04:54:55,766][00612] Updated weights for policy 1, policy_version 25080 (0.0008) [2023-10-08 04:54:57,184][00611] Updated weights for policy 0, policy_version 24962 (0.0009) [2023-10-08 04:54:57,555][00611] Updated weights for policy 0, policy_version 24972 (0.0009) [2023-10-08 04:54:57,932][00611] Updated weights for policy 0, policy_version 24982 (0.0008) [2023-10-08 04:54:58,302][00611] Updated weights for policy 0, policy_version 24992 (0.0008) [2023-10-08 04:54:58,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 51281920. Throughput: 0: 1842.6, 1: 1831.7. Samples: 12826052. Policy #0 lag: (min: 14.0, avg: 21.7, max: 46.0) [2023-10-08 04:54:58,754][130385] Avg episode reward: [(0, '47.850'), (1, '40.360')] [2023-10-08 04:54:59,322][00612] Updated weights for policy 1, policy_version 25090 (0.0008) [2023-10-08 04:54:59,703][00612] Updated weights for policy 1, policy_version 25100 (0.0009) [2023-10-08 04:55:00,067][00612] Updated weights for policy 1, policy_version 25110 (0.0008) [2023-10-08 04:55:00,436][00612] Updated weights for policy 1, policy_version 25120 (0.0009) [2023-10-08 04:55:01,941][00611] Updated weights for policy 0, policy_version 25002 (0.0007) [2023-10-08 04:55:02,318][00611] Updated weights for policy 0, policy_version 25012 (0.0009) [2023-10-08 04:55:02,676][00611] Updated weights for policy 0, policy_version 25022 (0.0007) [2023-10-08 04:55:03,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 51347456. Throughput: 0: 1836.4, 1: 1832.0. Samples: 12847896. Policy #0 lag: (min: 14.0, avg: 21.7, max: 46.0) [2023-10-08 04:55:03,755][130385] Avg episode reward: [(0, '47.430'), (1, '42.920')] [2023-10-08 04:55:04,109][00612] Updated weights for policy 1, policy_version 25130 (0.0010) [2023-10-08 04:55:04,475][00612] Updated weights for policy 1, policy_version 25140 (0.0010) [2023-10-08 04:55:04,842][00612] Updated weights for policy 1, policy_version 25150 (0.0011) [2023-10-08 04:55:06,329][00611] Updated weights for policy 0, policy_version 25032 (0.0010) [2023-10-08 04:55:06,699][00611] Updated weights for policy 0, policy_version 25042 (0.0010) [2023-10-08 04:55:07,066][00611] Updated weights for policy 0, policy_version 25052 (0.0010) [2023-10-08 04:55:08,591][00612] Updated weights for policy 1, policy_version 25160 (0.0009) [2023-10-08 04:55:08,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 51412992. Throughput: 0: 1839.2, 1: 1833.5. Samples: 12859252. Policy #0 lag: (min: 14.0, avg: 21.7, max: 46.0) [2023-10-08 04:55:08,755][130385] Avg episode reward: [(0, '46.640'), (1, '42.380')] [2023-10-08 04:55:08,957][00612] Updated weights for policy 1, policy_version 25170 (0.0009) [2023-10-08 04:55:09,328][00612] Updated weights for policy 1, policy_version 25180 (0.0011) [2023-10-08 04:55:10,510][00611] Updated weights for policy 0, policy_version 25062 (0.0010) [2023-10-08 04:55:10,880][00611] Updated weights for policy 0, policy_version 25072 (0.0007) [2023-10-08 04:55:11,248][00611] Updated weights for policy 0, policy_version 25082 (0.0007) [2023-10-08 04:55:12,960][00612] Updated weights for policy 1, policy_version 25190 (0.0008) [2023-10-08 04:55:13,324][00612] Updated weights for policy 1, policy_version 25200 (0.0007) [2023-10-08 04:55:13,699][00612] Updated weights for policy 1, policy_version 25210 (0.0008) [2023-10-08 04:55:13,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 51478528. Throughput: 0: 1834.2, 1: 1826.2. Samples: 12880578. Policy #0 lag: (min: 14.0, avg: 21.7, max: 46.0) [2023-10-08 04:55:13,754][130385] Avg episode reward: [(0, '45.350'), (1, '45.950')] [2023-10-08 04:55:14,893][00611] Updated weights for policy 0, policy_version 25092 (0.0009) [2023-10-08 04:55:15,264][00611] Updated weights for policy 0, policy_version 25102 (0.0010) [2023-10-08 04:55:15,631][00611] Updated weights for policy 0, policy_version 25112 (0.0010) [2023-10-08 04:55:17,217][00612] Updated weights for policy 1, policy_version 25220 (0.0009) [2023-10-08 04:55:17,587][00612] Updated weights for policy 1, policy_version 25230 (0.0008) [2023-10-08 04:55:17,952][00612] Updated weights for policy 1, policy_version 25240 (0.0010) [2023-10-08 04:55:18,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 51576832. Throughput: 0: 1844.2, 1: 1825.0. Samples: 12902836. Policy #0 lag: (min: 10.0, avg: 16.6, max: 42.0) [2023-10-08 04:55:18,755][130385] Avg episode reward: [(0, '46.490'), (1, '44.720')] [2023-10-08 04:55:19,261][00611] Updated weights for policy 0, policy_version 25122 (0.0009) [2023-10-08 04:55:19,632][00611] Updated weights for policy 0, policy_version 25132 (0.0007) [2023-10-08 04:55:20,007][00611] Updated weights for policy 0, policy_version 25142 (0.0008) [2023-10-08 04:55:20,368][00611] Updated weights for policy 0, policy_version 25152 (0.0008) [2023-10-08 04:55:21,647][00612] Updated weights for policy 1, policy_version 25250 (0.0010) [2023-10-08 04:55:22,006][00612] Updated weights for policy 1, policy_version 25260 (0.0008) [2023-10-08 04:55:22,380][00612] Updated weights for policy 1, policy_version 25270 (0.0009) [2023-10-08 04:55:22,760][00612] Updated weights for policy 1, policy_version 25280 (0.0008) [2023-10-08 04:55:23,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 51642368. Throughput: 0: 1844.4, 1: 1832.7. Samples: 12914194. Policy #0 lag: (min: 10.0, avg: 16.6, max: 42.0) [2023-10-08 04:55:23,754][130385] Avg episode reward: [(0, '50.530'), (1, '43.800')] [2023-10-08 04:55:23,922][00611] Updated weights for policy 0, policy_version 25162 (0.0007) [2023-10-08 04:55:24,295][00611] Updated weights for policy 0, policy_version 25172 (0.0009) [2023-10-08 04:55:24,662][00611] Updated weights for policy 0, policy_version 25182 (0.0010) [2023-10-08 04:55:26,341][00612] Updated weights for policy 1, policy_version 25290 (0.0008) [2023-10-08 04:55:26,715][00612] Updated weights for policy 1, policy_version 25300 (0.0009) [2023-10-08 04:55:27,084][00612] Updated weights for policy 1, policy_version 25310 (0.0009) [2023-10-08 04:55:28,223][00611] Updated weights for policy 0, policy_version 25192 (0.0008) [2023-10-08 04:55:28,596][00611] Updated weights for policy 0, policy_version 25202 (0.0007) [2023-10-08 04:55:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 51707904. Throughput: 0: 1845.2, 1: 1829.9. Samples: 12936066. Policy #0 lag: (min: 10.0, avg: 16.6, max: 42.0) [2023-10-08 04:55:28,755][130385] Avg episode reward: [(0, '49.600'), (1, '48.880')] [2023-10-08 04:55:28,963][00611] Updated weights for policy 0, policy_version 25212 (0.0009) [2023-10-08 04:55:30,759][00612] Updated weights for policy 1, policy_version 25320 (0.0009) [2023-10-08 04:55:31,121][00612] Updated weights for policy 1, policy_version 25330 (0.0008) [2023-10-08 04:55:31,487][00612] Updated weights for policy 1, policy_version 25340 (0.0009) [2023-10-08 04:55:32,696][00611] Updated weights for policy 0, policy_version 25222 (0.0009) [2023-10-08 04:55:33,076][00611] Updated weights for policy 0, policy_version 25232 (0.0008) [2023-10-08 04:55:33,446][00611] Updated weights for policy 0, policy_version 25242 (0.0008) [2023-10-08 04:55:33,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 51806208. Throughput: 0: 1827.5, 1: 1845.0. Samples: 12958280. Policy #0 lag: (min: 25.0, avg: 28.2, max: 57.0) [2023-10-08 04:55:33,755][130385] Avg episode reward: [(0, '49.600'), (1, '46.840')] [2023-10-08 04:55:35,133][00612] Updated weights for policy 1, policy_version 25350 (0.0009) [2023-10-08 04:55:35,512][00612] Updated weights for policy 1, policy_version 25360 (0.0010) [2023-10-08 04:55:35,867][00612] Updated weights for policy 1, policy_version 25370 (0.0010) [2023-10-08 04:55:37,289][00611] Updated weights for policy 0, policy_version 25252 (0.0009) [2023-10-08 04:55:37,680][00611] Updated weights for policy 0, policy_version 25262 (0.0007) [2023-10-08 04:55:38,054][00611] Updated weights for policy 0, policy_version 25272 (0.0008) [2023-10-08 04:55:38,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 51871744. Throughput: 0: 1849.2, 1: 1830.6. Samples: 12969074. Policy #0 lag: (min: 25.0, avg: 28.2, max: 57.0) [2023-10-08 04:55:38,755][130385] Avg episode reward: [(0, '50.180'), (1, '48.060')] [2023-10-08 04:55:39,591][00612] Updated weights for policy 1, policy_version 25380 (0.0010) [2023-10-08 04:55:39,965][00612] Updated weights for policy 1, policy_version 25390 (0.0010) [2023-10-08 04:55:40,332][00612] Updated weights for policy 1, policy_version 25400 (0.0009) [2023-10-08 04:55:41,612][00611] Updated weights for policy 0, policy_version 25282 (0.0008) [2023-10-08 04:55:41,986][00611] Updated weights for policy 0, policy_version 25292 (0.0008) [2023-10-08 04:55:42,359][00611] Updated weights for policy 0, policy_version 25302 (0.0009) [2023-10-08 04:55:42,732][00611] Updated weights for policy 0, policy_version 25312 (0.0008) [2023-10-08 04:55:43,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 51937280. Throughput: 0: 1825.2, 1: 1842.0. Samples: 12991080. Policy #0 lag: (min: 25.0, avg: 28.2, max: 57.0) [2023-10-08 04:55:43,754][130385] Avg episode reward: [(0, '53.260'), (1, '48.220')] [2023-10-08 04:55:43,755][00365] Saving new best policy, reward=53.260! [2023-10-08 04:55:43,880][00612] Updated weights for policy 1, policy_version 25410 (0.0009) [2023-10-08 04:55:44,249][00612] Updated weights for policy 1, policy_version 25420 (0.0008) [2023-10-08 04:55:44,626][00612] Updated weights for policy 1, policy_version 25430 (0.0008) [2023-10-08 04:55:44,986][00612] Updated weights for policy 1, policy_version 25440 (0.0009) [2023-10-08 04:55:46,348][00611] Updated weights for policy 0, policy_version 25322 (0.0010) [2023-10-08 04:55:46,724][00611] Updated weights for policy 0, policy_version 25332 (0.0008) [2023-10-08 04:55:47,096][00611] Updated weights for policy 0, policy_version 25342 (0.0007) [2023-10-08 04:55:48,704][00612] Updated weights for policy 1, policy_version 25450 (0.0007) [2023-10-08 04:55:48,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 52002816. Throughput: 0: 1840.9, 1: 1841.2. Samples: 13013592. Policy #0 lag: (min: 25.0, avg: 28.2, max: 57.0) [2023-10-08 04:55:48,755][130385] Avg episode reward: [(0, '52.500'), (1, '47.300')] [2023-10-08 04:55:49,079][00612] Updated weights for policy 1, policy_version 25460 (0.0009) [2023-10-08 04:55:49,447][00612] Updated weights for policy 1, policy_version 25470 (0.0007) [2023-10-08 04:55:50,799][00611] Updated weights for policy 0, policy_version 25352 (0.0009) [2023-10-08 04:55:51,177][00611] Updated weights for policy 0, policy_version 25362 (0.0008) [2023-10-08 04:55:51,554][00611] Updated weights for policy 0, policy_version 25372 (0.0010) [2023-10-08 04:55:53,105][00612] Updated weights for policy 1, policy_version 25480 (0.0008) [2023-10-08 04:55:53,477][00612] Updated weights for policy 1, policy_version 25490 (0.0010) [2023-10-08 04:55:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 52068352. Throughput: 0: 1825.5, 1: 1842.4. Samples: 13024306. Policy #0 lag: (min: 18.0, avg: 18.2, max: 26.0) [2023-10-08 04:55:53,754][130385] Avg episode reward: [(0, '51.620'), (1, '43.630')] [2023-10-08 04:55:53,847][00612] Updated weights for policy 1, policy_version 25500 (0.0010) [2023-10-08 04:55:55,186][00611] Updated weights for policy 0, policy_version 25382 (0.0008) [2023-10-08 04:55:55,549][00611] Updated weights for policy 0, policy_version 25392 (0.0009) [2023-10-08 04:55:55,917][00611] Updated weights for policy 0, policy_version 25402 (0.0007) [2023-10-08 04:55:57,503][00612] Updated weights for policy 1, policy_version 25510 (0.0009) [2023-10-08 04:55:57,870][00612] Updated weights for policy 1, policy_version 25520 (0.0007) [2023-10-08 04:55:58,241][00612] Updated weights for policy 1, policy_version 25530 (0.0010) [2023-10-08 04:55:58,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 52166656. Throughput: 0: 1844.5, 1: 1850.0. Samples: 13046830. Policy #0 lag: (min: 18.0, avg: 18.2, max: 26.0) [2023-10-08 04:55:58,754][130385] Avg episode reward: [(0, '48.930'), (1, '45.370')] [2023-10-08 04:55:59,408][00611] Updated weights for policy 0, policy_version 25412 (0.0009) [2023-10-08 04:55:59,786][00611] Updated weights for policy 0, policy_version 25422 (0.0007) [2023-10-08 04:56:00,153][00611] Updated weights for policy 0, policy_version 25432 (0.0009) [2023-10-08 04:56:01,939][00612] Updated weights for policy 1, policy_version 25540 (0.0009) [2023-10-08 04:56:02,306][00612] Updated weights for policy 1, policy_version 25550 (0.0007) [2023-10-08 04:56:02,681][00612] Updated weights for policy 1, policy_version 25560 (0.0008) [2023-10-08 04:56:03,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 52232192. Throughput: 0: 1842.2, 1: 1840.4. Samples: 13068550. Policy #0 lag: (min: 18.0, avg: 18.2, max: 26.0) [2023-10-08 04:56:03,755][130385] Avg episode reward: [(0, '47.440'), (1, '46.770')] [2023-10-08 04:56:03,844][00611] Updated weights for policy 0, policy_version 25442 (0.0008) [2023-10-08 04:56:04,227][00611] Updated weights for policy 0, policy_version 25452 (0.0007) [2023-10-08 04:56:04,598][00611] Updated weights for policy 0, policy_version 25462 (0.0007) [2023-10-08 04:56:04,977][00611] Updated weights for policy 0, policy_version 25472 (0.0007) [2023-10-08 04:56:06,354][00612] Updated weights for policy 1, policy_version 25570 (0.0009) [2023-10-08 04:56:06,719][00612] Updated weights for policy 1, policy_version 25580 (0.0008) [2023-10-08 04:56:07,093][00612] Updated weights for policy 1, policy_version 25590 (0.0007) [2023-10-08 04:56:07,469][00612] Updated weights for policy 1, policy_version 25600 (0.0007) [2023-10-08 04:56:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 52297728. Throughput: 0: 1843.1, 1: 1838.8. Samples: 13079880. Policy #0 lag: (min: 18.0, avg: 18.2, max: 26.0) [2023-10-08 04:56:08,754][130385] Avg episode reward: [(0, '47.660'), (1, '44.510')] [2023-10-08 04:56:08,833][00611] Updated weights for policy 0, policy_version 25482 (0.0009) [2023-10-08 04:56:09,212][00611] Updated weights for policy 0, policy_version 25492 (0.0009) [2023-10-08 04:56:09,573][00611] Updated weights for policy 0, policy_version 25502 (0.0008) [2023-10-08 04:56:11,110][00612] Updated weights for policy 1, policy_version 25610 (0.0007) [2023-10-08 04:56:11,475][00612] Updated weights for policy 1, policy_version 25620 (0.0007) [2023-10-08 04:56:11,838][00612] Updated weights for policy 1, policy_version 25630 (0.0007) [2023-10-08 04:56:13,282][00611] Updated weights for policy 0, policy_version 25512 (0.0008) [2023-10-08 04:56:13,664][00611] Updated weights for policy 0, policy_version 25522 (0.0008) [2023-10-08 04:56:13,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 52363264. Throughput: 0: 1839.4, 1: 1837.7. Samples: 13101536. Policy #0 lag: (min: 29.0, avg: 32.0, max: 61.0) [2023-10-08 04:56:13,754][130385] Avg episode reward: [(0, '48.490'), (1, '45.120')] [2023-10-08 04:56:14,026][00611] Updated weights for policy 0, policy_version 25532 (0.0010) [2023-10-08 04:56:15,465][00612] Updated weights for policy 1, policy_version 25640 (0.0008) [2023-10-08 04:56:15,838][00612] Updated weights for policy 1, policy_version 25650 (0.0008) [2023-10-08 04:56:16,211][00612] Updated weights for policy 1, policy_version 25660 (0.0009) [2023-10-08 04:56:17,675][00611] Updated weights for policy 0, policy_version 25542 (0.0010) [2023-10-08 04:56:18,046][00611] Updated weights for policy 0, policy_version 25552 (0.0008) [2023-10-08 04:56:18,428][00611] Updated weights for policy 0, policy_version 25562 (0.0008) [2023-10-08 04:56:18,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 52461568. Throughput: 0: 1838.8, 1: 1837.7. Samples: 13123726. Policy #0 lag: (min: 29.0, avg: 32.0, max: 61.0) [2023-10-08 04:56:18,755][130385] Avg episode reward: [(0, '49.020'), (1, '43.180')] [2023-10-08 04:56:18,764][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000025664_26279936.pth... [2023-10-08 04:56:18,765][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000025568_26181632.pth... [2023-10-08 04:56:18,802][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000023968_24543232.pth [2023-10-08 04:56:18,805][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000023840_24412160.pth [2023-10-08 04:56:18,807][00425] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p1/milestones/checkpoint_000025664_26279936.pth [2023-10-08 04:56:18,809][00365] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p0/milestones/checkpoint_000025568_26181632.pth [2023-10-08 04:56:19,839][00612] Updated weights for policy 1, policy_version 25670 (0.0008) [2023-10-08 04:56:20,212][00612] Updated weights for policy 1, policy_version 25680 (0.0011) [2023-10-08 04:56:20,577][00612] Updated weights for policy 1, policy_version 25690 (0.0009) [2023-10-08 04:56:22,012][00611] Updated weights for policy 0, policy_version 25572 (0.0009) [2023-10-08 04:56:22,374][00611] Updated weights for policy 0, policy_version 25582 (0.0009) [2023-10-08 04:56:22,738][00611] Updated weights for policy 0, policy_version 25592 (0.0008) [2023-10-08 04:56:23,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 52527104. Throughput: 0: 1840.9, 1: 1841.2. Samples: 13134768. Policy #0 lag: (min: 29.0, avg: 32.0, max: 61.0) [2023-10-08 04:56:23,755][130385] Avg episode reward: [(0, '49.830'), (1, '43.710')] [2023-10-08 04:56:24,154][00612] Updated weights for policy 1, policy_version 25700 (0.0010) [2023-10-08 04:56:24,523][00612] Updated weights for policy 1, policy_version 25710 (0.0007) [2023-10-08 04:56:24,889][00612] Updated weights for policy 1, policy_version 25720 (0.0008) [2023-10-08 04:56:26,471][00611] Updated weights for policy 0, policy_version 25602 (0.0011) [2023-10-08 04:56:26,861][00611] Updated weights for policy 0, policy_version 25612 (0.0011) [2023-10-08 04:56:27,224][00611] Updated weights for policy 0, policy_version 25622 (0.0010) [2023-10-08 04:56:27,594][00611] Updated weights for policy 0, policy_version 25632 (0.0011) [2023-10-08 04:56:28,316][00612] Updated weights for policy 1, policy_version 25730 (0.0010) [2023-10-08 04:56:28,686][00612] Updated weights for policy 1, policy_version 25740 (0.0007) [2023-10-08 04:56:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 52592640. Throughput: 0: 1836.8, 1: 1850.8. Samples: 13157022. Policy #0 lag: (min: 29.0, avg: 32.0, max: 61.0) [2023-10-08 04:56:28,754][130385] Avg episode reward: [(0, '47.190'), (1, '45.340')] [2023-10-08 04:56:29,061][00612] Updated weights for policy 1, policy_version 25750 (0.0007) [2023-10-08 04:56:29,420][00612] Updated weights for policy 1, policy_version 25760 (0.0007) [2023-10-08 04:56:31,227][00611] Updated weights for policy 0, policy_version 25642 (0.0007) [2023-10-08 04:56:31,587][00611] Updated weights for policy 0, policy_version 25652 (0.0007) [2023-10-08 04:56:31,956][00611] Updated weights for policy 0, policy_version 25662 (0.0010) [2023-10-08 04:56:32,994][00612] Updated weights for policy 1, policy_version 25770 (0.0009) [2023-10-08 04:56:33,369][00612] Updated weights for policy 1, policy_version 25780 (0.0009) [2023-10-08 04:56:33,737][00612] Updated weights for policy 1, policy_version 25790 (0.0009) [2023-10-08 04:56:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 52658176. Throughput: 0: 1834.0, 1: 1837.2. Samples: 13178796. Policy #0 lag: (min: 9.0, avg: 31.0, max: 41.0) [2023-10-08 04:56:33,755][130385] Avg episode reward: [(0, '46.640'), (1, '47.600')] [2023-10-08 04:56:35,763][00611] Updated weights for policy 0, policy_version 25672 (0.0010) [2023-10-08 04:56:36,131][00611] Updated weights for policy 0, policy_version 25682 (0.0008) [2023-10-08 04:56:36,511][00611] Updated weights for policy 0, policy_version 25692 (0.0009) [2023-10-08 04:56:37,306][00612] Updated weights for policy 1, policy_version 25800 (0.0008) [2023-10-08 04:56:37,668][00612] Updated weights for policy 1, policy_version 25810 (0.0009) [2023-10-08 04:56:38,045][00612] Updated weights for policy 1, policy_version 25820 (0.0009) [2023-10-08 04:56:38,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 52756480. Throughput: 0: 1826.8, 1: 1853.0. Samples: 13189896. Policy #0 lag: (min: 9.0, avg: 31.0, max: 41.0) [2023-10-08 04:56:38,755][130385] Avg episode reward: [(0, '49.300'), (1, '49.350')] [2023-10-08 04:56:38,756][00425] Saving new best policy, reward=49.350! [2023-10-08 04:56:40,174][00611] Updated weights for policy 0, policy_version 25702 (0.0009) [2023-10-08 04:56:40,543][00611] Updated weights for policy 0, policy_version 25712 (0.0010) [2023-10-08 04:56:40,920][00611] Updated weights for policy 0, policy_version 25722 (0.0010) [2023-10-08 04:56:41,707][00612] Updated weights for policy 1, policy_version 25830 (0.0010) [2023-10-08 04:56:42,093][00612] Updated weights for policy 1, policy_version 25840 (0.0009) [2023-10-08 04:56:42,468][00612] Updated weights for policy 1, policy_version 25850 (0.0008) [2023-10-08 04:56:43,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 52822016. Throughput: 0: 1825.9, 1: 1833.3. Samples: 13211496. Policy #0 lag: (min: 9.0, avg: 31.0, max: 41.0) [2023-10-08 04:56:43,755][130385] Avg episode reward: [(0, '50.280'), (1, '48.780')] [2023-10-08 04:56:44,562][00611] Updated weights for policy 0, policy_version 25732 (0.0009) [2023-10-08 04:56:44,931][00611] Updated weights for policy 0, policy_version 25742 (0.0007) [2023-10-08 04:56:45,310][00611] Updated weights for policy 0, policy_version 25752 (0.0008) [2023-10-08 04:56:45,893][00612] Updated weights for policy 1, policy_version 25860 (0.0010) [2023-10-08 04:56:46,257][00612] Updated weights for policy 1, policy_version 25870 (0.0008) [2023-10-08 04:56:46,628][00612] Updated weights for policy 1, policy_version 25880 (0.0008) [2023-10-08 04:56:48,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 52887552. Throughput: 0: 1823.5, 1: 1859.9. Samples: 13234304. Policy #0 lag: (min: 9.0, avg: 31.0, max: 41.0) [2023-10-08 04:56:48,754][130385] Avg episode reward: [(0, '48.720'), (1, '48.230')] [2023-10-08 04:56:48,920][00611] Updated weights for policy 0, policy_version 25762 (0.0008) [2023-10-08 04:56:49,295][00611] Updated weights for policy 0, policy_version 25772 (0.0007) [2023-10-08 04:56:49,660][00611] Updated weights for policy 0, policy_version 25782 (0.0007) [2023-10-08 04:56:50,036][00611] Updated weights for policy 0, policy_version 25792 (0.0008) [2023-10-08 04:56:50,362][00612] Updated weights for policy 1, policy_version 25890 (0.0007) [2023-10-08 04:56:50,726][00612] Updated weights for policy 1, policy_version 25900 (0.0007) [2023-10-08 04:56:51,099][00612] Updated weights for policy 1, policy_version 25910 (0.0008) [2023-10-08 04:56:51,464][00612] Updated weights for policy 1, policy_version 25920 (0.0009) [2023-10-08 04:56:53,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 52953088. Throughput: 0: 1822.2, 1: 1837.7. Samples: 13244576. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 04:56:53,754][130385] Avg episode reward: [(0, '44.980'), (1, '48.130')] [2023-10-08 04:56:53,782][00611] Updated weights for policy 0, policy_version 25802 (0.0010) [2023-10-08 04:56:54,148][00611] Updated weights for policy 0, policy_version 25812 (0.0012) [2023-10-08 04:56:54,522][00611] Updated weights for policy 0, policy_version 25822 (0.0008) [2023-10-08 04:56:55,134][00612] Updated weights for policy 1, policy_version 25930 (0.0008) [2023-10-08 04:56:55,509][00612] Updated weights for policy 1, policy_version 25940 (0.0009) [2023-10-08 04:56:55,881][00612] Updated weights for policy 1, policy_version 25950 (0.0008) [2023-10-08 04:56:58,201][00611] Updated weights for policy 0, policy_version 25832 (0.0008) [2023-10-08 04:56:58,578][00611] Updated weights for policy 0, policy_version 25842 (0.0007) [2023-10-08 04:56:58,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 53018624. Throughput: 0: 1818.6, 1: 1862.0. Samples: 13267166. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 04:56:58,755][130385] Avg episode reward: [(0, '42.790'), (1, '48.470')] [2023-10-08 04:56:58,947][00611] Updated weights for policy 0, policy_version 25852 (0.0007) [2023-10-08 04:56:59,459][00612] Updated weights for policy 1, policy_version 25960 (0.0009) [2023-10-08 04:56:59,830][00612] Updated weights for policy 1, policy_version 25970 (0.0009) [2023-10-08 04:57:00,207][00612] Updated weights for policy 1, policy_version 25980 (0.0010) [2023-10-08 04:57:02,392][00611] Updated weights for policy 0, policy_version 25862 (0.0008) [2023-10-08 04:57:02,764][00611] Updated weights for policy 0, policy_version 25872 (0.0008) [2023-10-08 04:57:03,134][00611] Updated weights for policy 0, policy_version 25882 (0.0007) [2023-10-08 04:57:03,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 53116928. Throughput: 0: 1818.2, 1: 1862.5. Samples: 13289356. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 04:57:03,754][130385] Avg episode reward: [(0, '42.970'), (1, '52.310')] [2023-10-08 04:57:03,847][00612] Updated weights for policy 1, policy_version 25990 (0.0009) [2023-10-08 04:57:04,222][00612] Updated weights for policy 1, policy_version 26000 (0.0009) [2023-10-08 04:57:04,580][00612] Updated weights for policy 1, policy_version 26010 (0.0008) [2023-10-08 04:57:04,797][00425] Saving new best policy, reward=52.310! [2023-10-08 04:57:06,790][00611] Updated weights for policy 0, policy_version 25892 (0.0009) [2023-10-08 04:57:07,160][00611] Updated weights for policy 0, policy_version 25902 (0.0009) [2023-10-08 04:57:07,542][00611] Updated weights for policy 0, policy_version 25912 (0.0010) [2023-10-08 04:57:08,238][00612] Updated weights for policy 1, policy_version 26020 (0.0010) [2023-10-08 04:57:08,605][00612] Updated weights for policy 1, policy_version 26030 (0.0010) [2023-10-08 04:57:08,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 53182464. Throughput: 0: 1822.7, 1: 1862.5. Samples: 13300602. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 04:57:08,754][130385] Avg episode reward: [(0, '45.930'), (1, '51.290')] [2023-10-08 04:57:08,961][00612] Updated weights for policy 1, policy_version 26040 (0.0009) [2023-10-08 04:57:11,319][00611] Updated weights for policy 0, policy_version 25922 (0.0011) [2023-10-08 04:57:11,721][00611] Updated weights for policy 0, policy_version 25932 (0.0008) [2023-10-08 04:57:12,090][00611] Updated weights for policy 0, policy_version 25942 (0.0008) [2023-10-08 04:57:12,457][00611] Updated weights for policy 0, policy_version 25952 (0.0008) [2023-10-08 04:57:12,663][00612] Updated weights for policy 1, policy_version 26050 (0.0009) [2023-10-08 04:57:13,035][00612] Updated weights for policy 1, policy_version 26060 (0.0009) [2023-10-08 04:57:13,403][00612] Updated weights for policy 1, policy_version 26070 (0.0008) [2023-10-08 04:57:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 53248000. Throughput: 0: 1815.0, 1: 1857.9. Samples: 13322302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:57:13,754][130385] Avg episode reward: [(0, '47.660'), (1, '50.030')] [2023-10-08 04:57:13,769][00612] Updated weights for policy 1, policy_version 26080 (0.0007) [2023-10-08 04:57:16,158][00611] Updated weights for policy 0, policy_version 25962 (0.0007) [2023-10-08 04:57:16,531][00611] Updated weights for policy 0, policy_version 25972 (0.0007) [2023-10-08 04:57:16,905][00611] Updated weights for policy 0, policy_version 25982 (0.0008) [2023-10-08 04:57:17,352][00612] Updated weights for policy 1, policy_version 26090 (0.0008) [2023-10-08 04:57:17,722][00612] Updated weights for policy 1, policy_version 26100 (0.0007) [2023-10-08 04:57:18,098][00612] Updated weights for policy 1, policy_version 26110 (0.0008) [2023-10-08 04:57:18,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 53346304. Throughput: 0: 1824.8, 1: 1838.3. Samples: 13343632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:57:18,755][130385] Avg episode reward: [(0, '44.490'), (1, '49.380')] [2023-10-08 04:57:20,557][00611] Updated weights for policy 0, policy_version 25992 (0.0008) [2023-10-08 04:57:20,932][00611] Updated weights for policy 0, policy_version 26002 (0.0008) [2023-10-08 04:57:21,301][00611] Updated weights for policy 0, policy_version 26012 (0.0008) [2023-10-08 04:57:21,774][00612] Updated weights for policy 1, policy_version 26120 (0.0008) [2023-10-08 04:57:22,151][00612] Updated weights for policy 1, policy_version 26130 (0.0009) [2023-10-08 04:57:22,518][00612] Updated weights for policy 1, policy_version 26140 (0.0009) [2023-10-08 04:57:23,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 53411840. Throughput: 0: 1819.3, 1: 1853.7. Samples: 13355180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:57:23,754][130385] Avg episode reward: [(0, '44.470'), (1, '50.870')] [2023-10-08 04:57:25,090][00611] Updated weights for policy 0, policy_version 26022 (0.0010) [2023-10-08 04:57:25,459][00611] Updated weights for policy 0, policy_version 26032 (0.0009) [2023-10-08 04:57:25,823][00611] Updated weights for policy 0, policy_version 26042 (0.0008) [2023-10-08 04:57:25,879][00612] Updated weights for policy 1, policy_version 26150 (0.0007) [2023-10-08 04:57:26,253][00612] Updated weights for policy 1, policy_version 26160 (0.0007) [2023-10-08 04:57:26,622][00612] Updated weights for policy 1, policy_version 26170 (0.0007) [2023-10-08 04:57:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 53477376. Throughput: 0: 1818.8, 1: 1846.1. Samples: 13376416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:57:28,754][130385] Avg episode reward: [(0, '44.250'), (1, '49.110')] [2023-10-08 04:57:29,598][00611] Updated weights for policy 0, policy_version 26052 (0.0008) [2023-10-08 04:57:29,964][00611] Updated weights for policy 0, policy_version 26062 (0.0008) [2023-10-08 04:57:30,339][00611] Updated weights for policy 0, policy_version 26072 (0.0008) [2023-10-08 04:57:30,383][00612] Updated weights for policy 1, policy_version 26180 (0.0009) [2023-10-08 04:57:30,753][00612] Updated weights for policy 1, policy_version 26190 (0.0008) [2023-10-08 04:57:31,114][00612] Updated weights for policy 1, policy_version 26200 (0.0008) [2023-10-08 04:57:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 53542912. Throughput: 0: 1807.6, 1: 1856.8. Samples: 13399202. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:57:33,754][130385] Avg episode reward: [(0, '43.700'), (1, '50.730')] [2023-10-08 04:57:34,132][00611] Updated weights for policy 0, policy_version 26082 (0.0007) [2023-10-08 04:57:34,511][00611] Updated weights for policy 0, policy_version 26092 (0.0009) [2023-10-08 04:57:34,787][00612] Updated weights for policy 1, policy_version 26210 (0.0007) [2023-10-08 04:57:34,883][00611] Updated weights for policy 0, policy_version 26102 (0.0007) [2023-10-08 04:57:35,154][00612] Updated weights for policy 1, policy_version 26220 (0.0007) [2023-10-08 04:57:35,252][00611] Updated weights for policy 0, policy_version 26112 (0.0007) [2023-10-08 04:57:35,517][00612] Updated weights for policy 1, policy_version 26230 (0.0009) [2023-10-08 04:57:35,892][00612] Updated weights for policy 1, policy_version 26240 (0.0007) [2023-10-08 04:57:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 53608448. Throughput: 0: 1809.7, 1: 1845.4. Samples: 13409058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:57:38,755][130385] Avg episode reward: [(0, '44.980'), (1, '49.530')] [2023-10-08 04:57:38,922][00611] Updated weights for policy 0, policy_version 26122 (0.0008) [2023-10-08 04:57:39,307][00611] Updated weights for policy 0, policy_version 26132 (0.0009) [2023-10-08 04:57:39,315][00612] Updated weights for policy 1, policy_version 26250 (0.0007) [2023-10-08 04:57:39,678][00611] Updated weights for policy 0, policy_version 26142 (0.0008) [2023-10-08 04:57:39,689][00612] Updated weights for policy 1, policy_version 26260 (0.0008) [2023-10-08 04:57:40,059][00612] Updated weights for policy 1, policy_version 26270 (0.0007) [2023-10-08 04:57:43,512][00611] Updated weights for policy 0, policy_version 26152 (0.0009) [2023-10-08 04:57:43,724][00612] Updated weights for policy 1, policy_version 26280 (0.0007) [2023-10-08 04:57:43,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 53673984. Throughput: 0: 1806.7, 1: 1858.1. Samples: 13432082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:57:43,755][130385] Avg episode reward: [(0, '49.250'), (1, '46.630')] [2023-10-08 04:57:43,879][00611] Updated weights for policy 0, policy_version 26162 (0.0007) [2023-10-08 04:57:44,094][00612] Updated weights for policy 1, policy_version 26290 (0.0007) [2023-10-08 04:57:44,243][00611] Updated weights for policy 0, policy_version 26172 (0.0008) [2023-10-08 04:57:44,453][00612] Updated weights for policy 1, policy_version 26300 (0.0008) [2023-10-08 04:57:47,941][00611] Updated weights for policy 0, policy_version 26182 (0.0008) [2023-10-08 04:57:48,219][00612] Updated weights for policy 1, policy_version 26310 (0.0008) [2023-10-08 04:57:48,312][00611] Updated weights for policy 0, policy_version 26192 (0.0009) [2023-10-08 04:57:48,591][00612] Updated weights for policy 1, policy_version 26320 (0.0007) [2023-10-08 04:57:48,677][00611] Updated weights for policy 0, policy_version 26202 (0.0009) [2023-10-08 04:57:48,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 53739520. Throughput: 0: 1810.7, 1: 1845.7. Samples: 13453894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:57:48,754][130385] Avg episode reward: [(0, '48.540'), (1, '48.160')] [2023-10-08 04:57:48,951][00612] Updated weights for policy 1, policy_version 26330 (0.0008) [2023-10-08 04:57:52,233][00611] Updated weights for policy 0, policy_version 26212 (0.0010) [2023-10-08 04:57:52,602][00611] Updated weights for policy 0, policy_version 26222 (0.0011) [2023-10-08 04:57:52,699][00612] Updated weights for policy 1, policy_version 26340 (0.0010) [2023-10-08 04:57:52,978][00611] Updated weights for policy 0, policy_version 26232 (0.0008) [2023-10-08 04:57:53,068][00612] Updated weights for policy 1, policy_version 26350 (0.0007) [2023-10-08 04:57:53,440][00612] Updated weights for policy 1, policy_version 26360 (0.0008) [2023-10-08 04:57:53,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 53870592. Throughput: 0: 1795.1, 1: 1848.6. Samples: 13464568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:57:53,755][130385] Avg episode reward: [(0, '46.640'), (1, '47.100')] [2023-10-08 04:57:56,689][00611] Updated weights for policy 0, policy_version 26242 (0.0007) [2023-10-08 04:57:57,083][00611] Updated weights for policy 0, policy_version 26252 (0.0008) [2023-10-08 04:57:57,141][00612] Updated weights for policy 1, policy_version 26370 (0.0008) [2023-10-08 04:57:57,465][00611] Updated weights for policy 0, policy_version 26262 (0.0009) [2023-10-08 04:57:57,514][00612] Updated weights for policy 1, policy_version 26380 (0.0008) [2023-10-08 04:57:57,832][00611] Updated weights for policy 0, policy_version 26272 (0.0008) [2023-10-08 04:57:57,871][00612] Updated weights for policy 1, policy_version 26390 (0.0008) [2023-10-08 04:57:58,239][00612] Updated weights for policy 1, policy_version 26400 (0.0008) [2023-10-08 04:57:58,754][130385] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 53936128. Throughput: 0: 1811.8, 1: 1842.2. Samples: 13486730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:57:58,755][130385] Avg episode reward: [(0, '45.350'), (1, '47.650')] [2023-10-08 04:58:01,520][00611] Updated weights for policy 0, policy_version 26282 (0.0010) [2023-10-08 04:58:01,888][00611] Updated weights for policy 0, policy_version 26292 (0.0009) [2023-10-08 04:58:02,019][00612] Updated weights for policy 1, policy_version 26410 (0.0008) [2023-10-08 04:58:02,260][00611] Updated weights for policy 0, policy_version 26302 (0.0008) [2023-10-08 04:58:02,388][00612] Updated weights for policy 1, policy_version 26420 (0.0009) [2023-10-08 04:58:02,755][00612] Updated weights for policy 1, policy_version 26430 (0.0008) [2023-10-08 04:58:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 54001664. Throughput: 0: 1798.4, 1: 1836.3. Samples: 13507192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:58:03,755][130385] Avg episode reward: [(0, '44.580'), (1, '45.270')] [2023-10-08 04:58:05,873][00611] Updated weights for policy 0, policy_version 26312 (0.0008) [2023-10-08 04:58:06,239][00611] Updated weights for policy 0, policy_version 26322 (0.0007) [2023-10-08 04:58:06,456][00612] Updated weights for policy 1, policy_version 26440 (0.0008) [2023-10-08 04:58:06,609][00611] Updated weights for policy 0, policy_version 26332 (0.0008) [2023-10-08 04:58:06,819][00612] Updated weights for policy 1, policy_version 26450 (0.0009) [2023-10-08 04:58:07,197][00612] Updated weights for policy 1, policy_version 26460 (0.0008) [2023-10-08 04:58:08,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 54067200. Throughput: 0: 1810.9, 1: 1837.8. Samples: 13519370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:58:08,755][130385] Avg episode reward: [(0, '45.030'), (1, '46.620')] [2023-10-08 04:58:10,194][00611] Updated weights for policy 0, policy_version 26342 (0.0008) [2023-10-08 04:58:10,556][00611] Updated weights for policy 0, policy_version 26352 (0.0010) [2023-10-08 04:58:10,878][00612] Updated weights for policy 1, policy_version 26470 (0.0007) [2023-10-08 04:58:10,938][00611] Updated weights for policy 0, policy_version 26362 (0.0009) [2023-10-08 04:58:11,253][00612] Updated weights for policy 1, policy_version 26480 (0.0008) [2023-10-08 04:58:11,612][00612] Updated weights for policy 1, policy_version 26490 (0.0009) [2023-10-08 04:58:13,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 54132736. Throughput: 0: 1807.2, 1: 1828.5. Samples: 13540022. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:58:13,755][130385] Avg episode reward: [(0, '46.410'), (1, '44.860')] [2023-10-08 04:58:14,639][00611] Updated weights for policy 0, policy_version 26372 (0.0007) [2023-10-08 04:58:15,016][00611] Updated weights for policy 0, policy_version 26382 (0.0010) [2023-10-08 04:58:15,350][00612] Updated weights for policy 1, policy_version 26500 (0.0009) [2023-10-08 04:58:15,385][00611] Updated weights for policy 0, policy_version 26392 (0.0008) [2023-10-08 04:58:15,744][00612] Updated weights for policy 1, policy_version 26510 (0.0009) [2023-10-08 04:58:16,121][00612] Updated weights for policy 1, policy_version 26520 (0.0008) [2023-10-08 04:58:18,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 54198272. Throughput: 0: 1815.9, 1: 1822.0. Samples: 13562910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:58:18,755][130385] Avg episode reward: [(0, '48.770'), (1, '45.980')] [2023-10-08 04:58:18,769][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000026528_27164672.pth... [2023-10-08 04:58:18,769][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000026400_27033600.pth... [2023-10-08 04:58:18,809][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000024800_25395200.pth [2023-10-08 04:58:18,809][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000024704_25296896.pth [2023-10-08 04:58:19,106][00611] Updated weights for policy 0, policy_version 26402 (0.0008) [2023-10-08 04:58:19,482][00611] Updated weights for policy 0, policy_version 26412 (0.0007) [2023-10-08 04:58:19,819][00612] Updated weights for policy 1, policy_version 26530 (0.0010) [2023-10-08 04:58:19,846][00611] Updated weights for policy 0, policy_version 26422 (0.0007) [2023-10-08 04:58:20,180][00612] Updated weights for policy 1, policy_version 26540 (0.0008) [2023-10-08 04:58:20,224][00611] Updated weights for policy 0, policy_version 26432 (0.0008) [2023-10-08 04:58:20,549][00612] Updated weights for policy 1, policy_version 26550 (0.0009) [2023-10-08 04:58:20,916][00612] Updated weights for policy 1, policy_version 26560 (0.0009) [2023-10-08 04:58:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 54263808. Throughput: 0: 1814.9, 1: 1823.4. Samples: 13572778. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-08 04:58:23,754][130385] Avg episode reward: [(0, '45.990'), (1, '47.740')] [2023-10-08 04:58:23,918][00611] Updated weights for policy 0, policy_version 26442 (0.0008) [2023-10-08 04:58:24,284][00611] Updated weights for policy 0, policy_version 26452 (0.0007) [2023-10-08 04:58:24,374][00612] Updated weights for policy 1, policy_version 26570 (0.0008) [2023-10-08 04:58:24,669][00611] Updated weights for policy 0, policy_version 26462 (0.0008) [2023-10-08 04:58:24,742][00612] Updated weights for policy 1, policy_version 26580 (0.0008) [2023-10-08 04:58:25,119][00612] Updated weights for policy 1, policy_version 26590 (0.0007) [2023-10-08 04:58:28,505][00611] Updated weights for policy 0, policy_version 26472 (0.0008) [2023-10-08 04:58:28,639][00612] Updated weights for policy 1, policy_version 26600 (0.0007) [2023-10-08 04:58:28,754][130385] Fps is (10 sec: 13108.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 54329344. Throughput: 0: 1819.4, 1: 1823.8. Samples: 13596024. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-08 04:58:28,754][130385] Avg episode reward: [(0, '45.690'), (1, '46.520')] [2023-10-08 04:58:28,874][00611] Updated weights for policy 0, policy_version 26482 (0.0008) [2023-10-08 04:58:29,009][00612] Updated weights for policy 1, policy_version 26610 (0.0008) [2023-10-08 04:58:29,249][00611] Updated weights for policy 0, policy_version 26492 (0.0008) [2023-10-08 04:58:29,375][00612] Updated weights for policy 1, policy_version 26620 (0.0007) [2023-10-08 04:58:33,037][00612] Updated weights for policy 1, policy_version 26630 (0.0008) [2023-10-08 04:58:33,056][00611] Updated weights for policy 0, policy_version 26502 (0.0009) [2023-10-08 04:58:33,409][00612] Updated weights for policy 1, policy_version 26640 (0.0008) [2023-10-08 04:58:33,426][00611] Updated weights for policy 0, policy_version 26512 (0.0008) [2023-10-08 04:58:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 54394880. Throughput: 0: 1820.7, 1: 1825.1. Samples: 13617952. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-08 04:58:33,754][130385] Avg episode reward: [(0, '44.070'), (1, '46.070')] [2023-10-08 04:58:33,772][00612] Updated weights for policy 1, policy_version 26650 (0.0008) [2023-10-08 04:58:33,795][00611] Updated weights for policy 0, policy_version 26522 (0.0007) [2023-10-08 04:58:37,314][00612] Updated weights for policy 1, policy_version 26660 (0.0008) [2023-10-08 04:58:37,381][00611] Updated weights for policy 0, policy_version 26532 (0.0007) [2023-10-08 04:58:37,684][00612] Updated weights for policy 1, policy_version 26670 (0.0008) [2023-10-08 04:58:37,750][00611] Updated weights for policy 0, policy_version 26542 (0.0008) [2023-10-08 04:58:38,043][00612] Updated weights for policy 1, policy_version 26680 (0.0008) [2023-10-08 04:58:38,115][00611] Updated weights for policy 0, policy_version 26552 (0.0007) [2023-10-08 04:58:38,754][130385] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 54525952. Throughput: 0: 1817.4, 1: 1837.6. Samples: 13629044. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-08 04:58:38,755][130385] Avg episode reward: [(0, '43.440'), (1, '47.700')] [2023-10-08 04:58:41,631][00612] Updated weights for policy 1, policy_version 26690 (0.0009) [2023-10-08 04:58:41,849][00611] Updated weights for policy 0, policy_version 26562 (0.0008) [2023-10-08 04:58:41,993][00612] Updated weights for policy 1, policy_version 26700 (0.0009) [2023-10-08 04:58:42,258][00611] Updated weights for policy 0, policy_version 26572 (0.0008) [2023-10-08 04:58:42,362][00612] Updated weights for policy 1, policy_version 26710 (0.0008) [2023-10-08 04:58:42,625][00611] Updated weights for policy 0, policy_version 26582 (0.0008) [2023-10-08 04:58:42,723][00612] Updated weights for policy 1, policy_version 26720 (0.0008) [2023-10-08 04:58:42,996][00611] Updated weights for policy 0, policy_version 26592 (0.0007) [2023-10-08 04:58:43,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 54591488. Throughput: 0: 1819.2, 1: 1825.8. Samples: 13650754. Policy #0 lag: (min: 7.0, avg: 11.9, max: 39.0) [2023-10-08 04:58:43,755][130385] Avg episode reward: [(0, '41.670'), (1, '48.130')] [2023-10-08 04:58:46,464][00612] Updated weights for policy 1, policy_version 26730 (0.0007) [2023-10-08 04:58:46,527][00611] Updated weights for policy 0, policy_version 26602 (0.0008) [2023-10-08 04:58:46,824][00612] Updated weights for policy 1, policy_version 26740 (0.0007) [2023-10-08 04:58:46,907][00611] Updated weights for policy 0, policy_version 26612 (0.0008) [2023-10-08 04:58:47,193][00612] Updated weights for policy 1, policy_version 26750 (0.0009) [2023-10-08 04:58:47,272][00611] Updated weights for policy 0, policy_version 26622 (0.0009) [2023-10-08 04:58:48,754][130385] Fps is (10 sec: 13107.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 54657024. Throughput: 0: 1818.7, 1: 1844.3. Samples: 13672028. Policy #0 lag: (min: 7.0, avg: 11.9, max: 39.0) [2023-10-08 04:58:48,755][130385] Avg episode reward: [(0, '44.650'), (1, '47.600')] [2023-10-08 04:58:50,772][00612] Updated weights for policy 1, policy_version 26760 (0.0008) [2023-10-08 04:58:50,916][00611] Updated weights for policy 0, policy_version 26632 (0.0008) [2023-10-08 04:58:51,139][00612] Updated weights for policy 1, policy_version 26770 (0.0008) [2023-10-08 04:58:51,288][00611] Updated weights for policy 0, policy_version 26642 (0.0007) [2023-10-08 04:58:51,506][00612] Updated weights for policy 1, policy_version 26780 (0.0009) [2023-10-08 04:58:51,656][00611] Updated weights for policy 0, policy_version 26652 (0.0007) [2023-10-08 04:58:53,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 54722560. Throughput: 0: 1822.8, 1: 1825.3. Samples: 13683534. Policy #0 lag: (min: 7.0, avg: 11.9, max: 39.0) [2023-10-08 04:58:53,754][130385] Avg episode reward: [(0, '41.160'), (1, '46.980')] [2023-10-08 04:58:55,194][00612] Updated weights for policy 1, policy_version 26790 (0.0008) [2023-10-08 04:58:55,341][00611] Updated weights for policy 0, policy_version 26662 (0.0008) [2023-10-08 04:58:55,562][00612] Updated weights for policy 1, policy_version 26800 (0.0009) [2023-10-08 04:58:55,718][00611] Updated weights for policy 0, policy_version 26672 (0.0008) [2023-10-08 04:58:55,927][00612] Updated weights for policy 1, policy_version 26810 (0.0010) [2023-10-08 04:58:56,085][00611] Updated weights for policy 0, policy_version 26682 (0.0009) [2023-10-08 04:58:58,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 54788096. Throughput: 0: 1815.2, 1: 1848.1. Samples: 13704870. Policy #0 lag: (min: 7.0, avg: 11.9, max: 39.0) [2023-10-08 04:58:58,755][130385] Avg episode reward: [(0, '37.550'), (1, '46.030')] [2023-10-08 04:58:59,579][00612] Updated weights for policy 1, policy_version 26820 (0.0009) [2023-10-08 04:58:59,739][00611] Updated weights for policy 0, policy_version 26692 (0.0008) [2023-10-08 04:58:59,950][00612] Updated weights for policy 1, policy_version 26830 (0.0009) [2023-10-08 04:59:00,107][00611] Updated weights for policy 0, policy_version 26702 (0.0007) [2023-10-08 04:59:00,309][00612] Updated weights for policy 1, policy_version 26840 (0.0008) [2023-10-08 04:59:00,482][00611] Updated weights for policy 0, policy_version 26712 (0.0007) [2023-10-08 04:59:03,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 54853632. Throughput: 0: 1818.1, 1: 1854.5. Samples: 13728174. Policy #0 lag: (min: 24.0, avg: 41.5, max: 56.0) [2023-10-08 04:59:03,755][130385] Avg episode reward: [(0, '38.360'), (1, '41.720')] [2023-10-08 04:59:03,933][00612] Updated weights for policy 1, policy_version 26850 (0.0007) [2023-10-08 04:59:04,225][00611] Updated weights for policy 0, policy_version 26722 (0.0008) [2023-10-08 04:59:04,301][00612] Updated weights for policy 1, policy_version 26860 (0.0008) [2023-10-08 04:59:04,579][00611] Updated weights for policy 0, policy_version 26732 (0.0007) [2023-10-08 04:59:04,679][00612] Updated weights for policy 1, policy_version 26870 (0.0008) [2023-10-08 04:59:04,951][00611] Updated weights for policy 0, policy_version 26742 (0.0008) [2023-10-08 04:59:05,047][00612] Updated weights for policy 1, policy_version 26880 (0.0008) [2023-10-08 04:59:05,308][00611] Updated weights for policy 0, policy_version 26752 (0.0008) [2023-10-08 04:59:08,748][00612] Updated weights for policy 1, policy_version 26890 (0.0007) [2023-10-08 04:59:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 54919168. Throughput: 0: 1819.2, 1: 1853.1. Samples: 13738034. Policy #0 lag: (min: 24.0, avg: 41.5, max: 56.0) [2023-10-08 04:59:08,755][130385] Avg episode reward: [(0, '37.940'), (1, '43.860')] [2023-10-08 04:59:08,856][00611] Updated weights for policy 0, policy_version 26762 (0.0007) [2023-10-08 04:59:09,116][00612] Updated weights for policy 1, policy_version 26900 (0.0010) [2023-10-08 04:59:09,224][00611] Updated weights for policy 0, policy_version 26772 (0.0009) [2023-10-08 04:59:09,485][00612] Updated weights for policy 1, policy_version 26910 (0.0009) [2023-10-08 04:59:09,603][00611] Updated weights for policy 0, policy_version 26782 (0.0008) [2023-10-08 04:59:13,119][00612] Updated weights for policy 1, policy_version 26920 (0.0010) [2023-10-08 04:59:13,292][00611] Updated weights for policy 0, policy_version 26792 (0.0008) [2023-10-08 04:59:13,485][00612] Updated weights for policy 1, policy_version 26930 (0.0009) [2023-10-08 04:59:13,656][00611] Updated weights for policy 0, policy_version 26802 (0.0008) [2023-10-08 04:59:13,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 54984704. Throughput: 0: 1821.5, 1: 1844.3. Samples: 13760986. Policy #0 lag: (min: 24.0, avg: 41.5, max: 56.0) [2023-10-08 04:59:13,754][130385] Avg episode reward: [(0, '39.310'), (1, '45.550')] [2023-10-08 04:59:13,853][00612] Updated weights for policy 1, policy_version 26940 (0.0009) [2023-10-08 04:59:14,043][00611] Updated weights for policy 0, policy_version 26812 (0.0009) [2023-10-08 04:59:17,576][00612] Updated weights for policy 1, policy_version 26950 (0.0008) [2023-10-08 04:59:17,841][00611] Updated weights for policy 0, policy_version 26822 (0.0008) [2023-10-08 04:59:17,949][00612] Updated weights for policy 1, policy_version 26960 (0.0008) [2023-10-08 04:59:18,206][00611] Updated weights for policy 0, policy_version 26832 (0.0008) [2023-10-08 04:59:18,313][00612] Updated weights for policy 1, policy_version 26970 (0.0007) [2023-10-08 04:59:18,581][00611] Updated weights for policy 0, policy_version 26842 (0.0008) [2023-10-08 04:59:18,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 55083008. Throughput: 0: 1820.9, 1: 1828.9. Samples: 13782196. Policy #0 lag: (min: 24.0, avg: 41.5, max: 56.0) [2023-10-08 04:59:18,754][130385] Avg episode reward: [(0, '38.570'), (1, '43.710')] [2023-10-08 04:59:21,954][00612] Updated weights for policy 1, policy_version 26980 (0.0008) [2023-10-08 04:59:22,235][00611] Updated weights for policy 0, policy_version 26852 (0.0008) [2023-10-08 04:59:22,327][00612] Updated weights for policy 1, policy_version 26990 (0.0007) [2023-10-08 04:59:22,618][00611] Updated weights for policy 0, policy_version 26862 (0.0007) [2023-10-08 04:59:22,701][00612] Updated weights for policy 1, policy_version 27000 (0.0007) [2023-10-08 04:59:22,996][00611] Updated weights for policy 0, policy_version 26872 (0.0009) [2023-10-08 04:59:23,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 55181312. Throughput: 0: 1827.8, 1: 1832.0. Samples: 13793736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:59:23,754][130385] Avg episode reward: [(0, '41.750'), (1, '44.320')] [2023-10-08 04:59:26,309][00612] Updated weights for policy 1, policy_version 27010 (0.0008) [2023-10-08 04:59:26,670][00612] Updated weights for policy 1, policy_version 27020 (0.0008) [2023-10-08 04:59:26,770][00611] Updated weights for policy 0, policy_version 26882 (0.0010) [2023-10-08 04:59:27,034][00612] Updated weights for policy 1, policy_version 27030 (0.0008) [2023-10-08 04:59:27,177][00611] Updated weights for policy 0, policy_version 26892 (0.0009) [2023-10-08 04:59:27,400][00612] Updated weights for policy 1, policy_version 27040 (0.0009) [2023-10-08 04:59:27,541][00611] Updated weights for policy 0, policy_version 26902 (0.0007) [2023-10-08 04:59:27,903][00611] Updated weights for policy 0, policy_version 26912 (0.0011) [2023-10-08 04:59:28,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 55246848. Throughput: 0: 1828.8, 1: 1823.7. Samples: 13815114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:59:28,755][130385] Avg episode reward: [(0, '40.020'), (1, '42.430')] [2023-10-08 04:59:31,040][00612] Updated weights for policy 1, policy_version 27050 (0.0007) [2023-10-08 04:59:31,416][00612] Updated weights for policy 1, policy_version 27060 (0.0008) [2023-10-08 04:59:31,445][00611] Updated weights for policy 0, policy_version 26922 (0.0008) [2023-10-08 04:59:31,784][00612] Updated weights for policy 1, policy_version 27070 (0.0009) [2023-10-08 04:59:31,804][00611] Updated weights for policy 0, policy_version 26932 (0.0008) [2023-10-08 04:59:32,183][00611] Updated weights for policy 0, policy_version 26942 (0.0010) [2023-10-08 04:59:33,754][130385] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 55312384. Throughput: 0: 1826.7, 1: 1837.7. Samples: 13836926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:59:33,754][130385] Avg episode reward: [(0, '42.950'), (1, '42.710')] [2023-10-08 04:59:35,498][00612] Updated weights for policy 1, policy_version 27080 (0.0008) [2023-10-08 04:59:35,861][00612] Updated weights for policy 1, policy_version 27090 (0.0009) [2023-10-08 04:59:36,053][00611] Updated weights for policy 0, policy_version 26952 (0.0008) [2023-10-08 04:59:36,228][00612] Updated weights for policy 1, policy_version 27100 (0.0007) [2023-10-08 04:59:36,419][00611] Updated weights for policy 0, policy_version 26962 (0.0009) [2023-10-08 04:59:36,793][00611] Updated weights for policy 0, policy_version 26972 (0.0008) [2023-10-08 04:59:38,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 55377920. Throughput: 0: 1822.8, 1: 1834.0. Samples: 13848088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:59:38,754][130385] Avg episode reward: [(0, '43.270'), (1, '46.860')] [2023-10-08 04:59:39,788][00612] Updated weights for policy 1, policy_version 27110 (0.0009) [2023-10-08 04:59:40,161][00612] Updated weights for policy 1, policy_version 27120 (0.0010) [2023-10-08 04:59:40,529][00612] Updated weights for policy 1, policy_version 27130 (0.0007) [2023-10-08 04:59:40,543][00611] Updated weights for policy 0, policy_version 26982 (0.0007) [2023-10-08 04:59:40,906][00611] Updated weights for policy 0, policy_version 26992 (0.0008) [2023-10-08 04:59:41,273][00611] Updated weights for policy 0, policy_version 27002 (0.0010) [2023-10-08 04:59:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 55443456. Throughput: 0: 1821.3, 1: 1845.3. Samples: 13869870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 04:59:43,754][130385] Avg episode reward: [(0, '41.280'), (1, '46.510')] [2023-10-08 04:59:44,223][00612] Updated weights for policy 1, policy_version 27140 (0.0007) [2023-10-08 04:59:44,591][00612] Updated weights for policy 1, policy_version 27150 (0.0008) [2023-10-08 04:59:44,906][00611] Updated weights for policy 0, policy_version 27012 (0.0007) [2023-10-08 04:59:44,962][00612] Updated weights for policy 1, policy_version 27160 (0.0007) [2023-10-08 04:59:45,280][00611] Updated weights for policy 0, policy_version 27022 (0.0009) [2023-10-08 04:59:45,644][00611] Updated weights for policy 0, policy_version 27032 (0.0010) [2023-10-08 04:59:48,593][00612] Updated weights for policy 1, policy_version 27170 (0.0007) [2023-10-08 04:59:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 55508992. Throughput: 0: 1813.3, 1: 1841.8. Samples: 13892652. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-08 04:59:48,754][130385] Avg episode reward: [(0, '41.920'), (1, '48.870')] [2023-10-08 04:59:48,958][00612] Updated weights for policy 1, policy_version 27180 (0.0009) [2023-10-08 04:59:49,327][00612] Updated weights for policy 1, policy_version 27190 (0.0010) [2023-10-08 04:59:49,444][00611] Updated weights for policy 0, policy_version 27042 (0.0011) [2023-10-08 04:59:49,698][00612] Updated weights for policy 1, policy_version 27200 (0.0007) [2023-10-08 04:59:49,819][00611] Updated weights for policy 0, policy_version 27052 (0.0008) [2023-10-08 04:59:50,204][00611] Updated weights for policy 0, policy_version 27062 (0.0009) [2023-10-08 04:59:50,576][00611] Updated weights for policy 0, policy_version 27072 (0.0007) [2023-10-08 04:59:53,363][00612] Updated weights for policy 1, policy_version 27210 (0.0007) [2023-10-08 04:59:53,741][00612] Updated weights for policy 1, policy_version 27220 (0.0008) [2023-10-08 04:59:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 55574528. Throughput: 0: 1813.6, 1: 1845.1. Samples: 13902674. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-08 04:59:53,754][130385] Avg episode reward: [(0, '40.960'), (1, '47.530')] [2023-10-08 04:59:54,109][00612] Updated weights for policy 1, policy_version 27230 (0.0007) [2023-10-08 04:59:54,263][00611] Updated weights for policy 0, policy_version 27082 (0.0008) [2023-10-08 04:59:54,636][00611] Updated weights for policy 0, policy_version 27092 (0.0007) [2023-10-08 04:59:55,010][00611] Updated weights for policy 0, policy_version 27102 (0.0009) [2023-10-08 04:59:57,681][00612] Updated weights for policy 1, policy_version 27240 (0.0008) [2023-10-08 04:59:58,051][00612] Updated weights for policy 1, policy_version 27250 (0.0007) [2023-10-08 04:59:58,373][00611] Updated weights for policy 0, policy_version 27112 (0.0009) [2023-10-08 04:59:58,413][00612] Updated weights for policy 1, policy_version 27260 (0.0007) [2023-10-08 04:59:58,751][00611] Updated weights for policy 0, policy_version 27122 (0.0009) [2023-10-08 04:59:58,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 55672832. Throughput: 0: 1819.5, 1: 1846.5. Samples: 13925956. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-08 04:59:58,754][130385] Avg episode reward: [(0, '39.390'), (1, '47.860')] [2023-10-08 04:59:59,117][00611] Updated weights for policy 0, policy_version 27132 (0.0008) [2023-10-08 05:00:01,960][00612] Updated weights for policy 1, policy_version 27270 (0.0009) [2023-10-08 05:00:02,334][00612] Updated weights for policy 1, policy_version 27280 (0.0009) [2023-10-08 05:00:02,706][00612] Updated weights for policy 1, policy_version 27290 (0.0008) [2023-10-08 05:00:02,780][00611] Updated weights for policy 0, policy_version 27142 (0.0008) [2023-10-08 05:00:03,151][00611] Updated weights for policy 0, policy_version 27152 (0.0009) [2023-10-08 05:00:03,524][00611] Updated weights for policy 0, policy_version 27162 (0.0009) [2023-10-08 05:00:03,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 55771136. Throughput: 0: 1819.3, 1: 1835.4. Samples: 13946656. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-08 05:00:03,754][130385] Avg episode reward: [(0, '40.910'), (1, '47.360')] [2023-10-08 05:00:06,429][00612] Updated weights for policy 1, policy_version 27300 (0.0008) [2023-10-08 05:00:06,793][00612] Updated weights for policy 1, policy_version 27310 (0.0008) [2023-10-08 05:00:07,166][00612] Updated weights for policy 1, policy_version 27320 (0.0008) [2023-10-08 05:00:07,284][00611] Updated weights for policy 0, policy_version 27172 (0.0008) [2023-10-08 05:00:07,655][00611] Updated weights for policy 0, policy_version 27182 (0.0009) [2023-10-08 05:00:08,032][00611] Updated weights for policy 0, policy_version 27192 (0.0010) [2023-10-08 05:00:08,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 55836672. Throughput: 0: 1818.3, 1: 1853.3. Samples: 13958960. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 05:00:08,754][130385] Avg episode reward: [(0, '39.960'), (1, '49.040')] [2023-10-08 05:00:10,775][00612] Updated weights for policy 1, policy_version 27330 (0.0009) [2023-10-08 05:00:11,149][00612] Updated weights for policy 1, policy_version 27340 (0.0008) [2023-10-08 05:00:11,515][00612] Updated weights for policy 1, policy_version 27350 (0.0007) [2023-10-08 05:00:11,729][00611] Updated weights for policy 0, policy_version 27202 (0.0010) [2023-10-08 05:00:11,886][00612] Updated weights for policy 1, policy_version 27360 (0.0008) [2023-10-08 05:00:12,151][00611] Updated weights for policy 0, policy_version 27212 (0.0010) [2023-10-08 05:00:12,516][00611] Updated weights for policy 0, policy_version 27222 (0.0010) [2023-10-08 05:00:12,885][00611] Updated weights for policy 0, policy_version 27232 (0.0009) [2023-10-08 05:00:13,754][130385] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 55902208. Throughput: 0: 1812.4, 1: 1842.3. Samples: 13979578. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 05:00:13,755][130385] Avg episode reward: [(0, '40.920'), (1, '49.610')] [2023-10-08 05:00:15,463][00612] Updated weights for policy 1, policy_version 27370 (0.0010) [2023-10-08 05:00:15,832][00612] Updated weights for policy 1, policy_version 27380 (0.0009) [2023-10-08 05:00:16,207][00612] Updated weights for policy 1, policy_version 27390 (0.0007) [2023-10-08 05:00:16,471][00611] Updated weights for policy 0, policy_version 27242 (0.0007) [2023-10-08 05:00:16,845][00611] Updated weights for policy 0, policy_version 27252 (0.0011) [2023-10-08 05:00:17,212][00611] Updated weights for policy 0, policy_version 27262 (0.0009) [2023-10-08 05:00:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 55967744. Throughput: 0: 1815.0, 1: 1848.8. Samples: 14001798. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 05:00:18,755][130385] Avg episode reward: [(0, '39.870'), (1, '47.860')] [2023-10-08 05:00:18,764][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000027392_28049408.pth... [2023-10-08 05:00:18,765][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000027264_27918336.pth... [2023-10-08 05:00:18,799][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000025664_26279936.pth [2023-10-08 05:00:18,804][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000025568_26181632.pth [2023-10-08 05:00:19,748][00612] Updated weights for policy 1, policy_version 27400 (0.0007) [2023-10-08 05:00:20,108][00612] Updated weights for policy 1, policy_version 27410 (0.0007) [2023-10-08 05:00:20,476][00612] Updated weights for policy 1, policy_version 27420 (0.0007) [2023-10-08 05:00:21,005][00611] Updated weights for policy 0, policy_version 27272 (0.0008) [2023-10-08 05:00:21,381][00611] Updated weights for policy 0, policy_version 27282 (0.0009) [2023-10-08 05:00:21,763][00611] Updated weights for policy 0, policy_version 27292 (0.0009) [2023-10-08 05:00:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 56033280. Throughput: 0: 1820.5, 1: 1841.1. Samples: 14012860. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 05:00:23,755][130385] Avg episode reward: [(0, '43.690'), (1, '45.330')] [2023-10-08 05:00:24,126][00612] Updated weights for policy 1, policy_version 27430 (0.0008) [2023-10-08 05:00:24,503][00612] Updated weights for policy 1, policy_version 27440 (0.0009) [2023-10-08 05:00:24,869][00612] Updated weights for policy 1, policy_version 27450 (0.0008) [2023-10-08 05:00:25,304][00611] Updated weights for policy 0, policy_version 27302 (0.0009) [2023-10-08 05:00:25,671][00611] Updated weights for policy 0, policy_version 27312 (0.0008) [2023-10-08 05:00:26,041][00611] Updated weights for policy 0, policy_version 27322 (0.0008) [2023-10-08 05:00:28,307][00612] Updated weights for policy 1, policy_version 27460 (0.0008) [2023-10-08 05:00:28,674][00612] Updated weights for policy 1, policy_version 27470 (0.0010) [2023-10-08 05:00:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 56098816. Throughput: 0: 1823.3, 1: 1849.6. Samples: 14035152. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 05:00:28,755][130385] Avg episode reward: [(0, '44.290'), (1, '43.610')] [2023-10-08 05:00:29,050][00612] Updated weights for policy 1, policy_version 27480 (0.0008) [2023-10-08 05:00:29,819][00611] Updated weights for policy 0, policy_version 27332 (0.0007) [2023-10-08 05:00:30,206][00611] Updated weights for policy 0, policy_version 27342 (0.0010) [2023-10-08 05:00:30,565][00611] Updated weights for policy 0, policy_version 27352 (0.0009) [2023-10-08 05:00:32,523][00612] Updated weights for policy 1, policy_version 27490 (0.0009) [2023-10-08 05:00:32,893][00612] Updated weights for policy 1, policy_version 27500 (0.0007) [2023-10-08 05:00:33,255][00612] Updated weights for policy 1, policy_version 27510 (0.0009) [2023-10-08 05:00:33,628][00612] Updated weights for policy 1, policy_version 27520 (0.0008) [2023-10-08 05:00:33,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56197120. Throughput: 0: 1827.3, 1: 1840.2. Samples: 14057688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:00:33,755][130385] Avg episode reward: [(0, '44.330'), (1, '43.890')] [2023-10-08 05:00:34,294][00611] Updated weights for policy 0, policy_version 27362 (0.0011) [2023-10-08 05:00:34,663][00611] Updated weights for policy 0, policy_version 27372 (0.0008) [2023-10-08 05:00:35,037][00611] Updated weights for policy 0, policy_version 27382 (0.0009) [2023-10-08 05:00:35,415][00611] Updated weights for policy 0, policy_version 27392 (0.0010) [2023-10-08 05:00:37,322][00612] Updated weights for policy 1, policy_version 27530 (0.0008) [2023-10-08 05:00:37,681][00612] Updated weights for policy 1, policy_version 27540 (0.0008) [2023-10-08 05:00:38,051][00612] Updated weights for policy 1, policy_version 27550 (0.0008) [2023-10-08 05:00:38,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 56262656. Throughput: 0: 1825.1, 1: 1859.9. Samples: 14068496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:00:38,755][130385] Avg episode reward: [(0, '43.880'), (1, '44.630')] [2023-10-08 05:00:38,939][00611] Updated weights for policy 0, policy_version 27402 (0.0009) [2023-10-08 05:00:39,312][00611] Updated weights for policy 0, policy_version 27412 (0.0008) [2023-10-08 05:00:39,677][00611] Updated weights for policy 0, policy_version 27422 (0.0008) [2023-10-08 05:00:41,882][00612] Updated weights for policy 1, policy_version 27560 (0.0007) [2023-10-08 05:00:42,240][00612] Updated weights for policy 1, policy_version 27570 (0.0007) [2023-10-08 05:00:42,611][00612] Updated weights for policy 1, policy_version 27580 (0.0007) [2023-10-08 05:00:43,510][00611] Updated weights for policy 0, policy_version 27432 (0.0010) [2023-10-08 05:00:43,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56328192. Throughput: 0: 1820.1, 1: 1837.0. Samples: 14090526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:00:43,755][130385] Avg episode reward: [(0, '50.400'), (1, '45.710')] [2023-10-08 05:00:43,885][00611] Updated weights for policy 0, policy_version 27442 (0.0009) [2023-10-08 05:00:44,254][00611] Updated weights for policy 0, policy_version 27452 (0.0008) [2023-10-08 05:00:46,307][00612] Updated weights for policy 1, policy_version 27590 (0.0008) [2023-10-08 05:00:46,676][00612] Updated weights for policy 1, policy_version 27600 (0.0008) [2023-10-08 05:00:47,040][00612] Updated weights for policy 1, policy_version 27610 (0.0007) [2023-10-08 05:00:47,914][00611] Updated weights for policy 0, policy_version 27462 (0.0011) [2023-10-08 05:00:48,281][00611] Updated weights for policy 0, policy_version 27472 (0.0010) [2023-10-08 05:00:48,656][00611] Updated weights for policy 0, policy_version 27482 (0.0009) [2023-10-08 05:00:48,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56393728. Throughput: 0: 1825.3, 1: 1856.0. Samples: 14112318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:00:48,754][130385] Avg episode reward: [(0, '50.800'), (1, '46.820')] [2023-10-08 05:00:50,561][00612] Updated weights for policy 1, policy_version 27620 (0.0008) [2023-10-08 05:00:50,929][00612] Updated weights for policy 1, policy_version 27630 (0.0010) [2023-10-08 05:00:51,298][00612] Updated weights for policy 1, policy_version 27640 (0.0008) [2023-10-08 05:00:52,419][00611] Updated weights for policy 0, policy_version 27492 (0.0008) [2023-10-08 05:00:52,806][00611] Updated weights for policy 0, policy_version 27502 (0.0008) [2023-10-08 05:00:53,168][00611] Updated weights for policy 0, policy_version 27512 (0.0007) [2023-10-08 05:00:53,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 56492032. Throughput: 0: 1820.9, 1: 1834.0. Samples: 14123432. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 05:00:53,754][130385] Avg episode reward: [(0, '51.150'), (1, '46.360')] [2023-10-08 05:00:55,132][00612] Updated weights for policy 1, policy_version 27650 (0.0011) [2023-10-08 05:00:55,502][00612] Updated weights for policy 1, policy_version 27660 (0.0009) [2023-10-08 05:00:55,866][00612] Updated weights for policy 1, policy_version 27670 (0.0009) [2023-10-08 05:00:56,232][00612] Updated weights for policy 1, policy_version 27680 (0.0010) [2023-10-08 05:00:56,936][00611] Updated weights for policy 0, policy_version 27522 (0.0008) [2023-10-08 05:00:57,351][00611] Updated weights for policy 0, policy_version 27532 (0.0007) [2023-10-08 05:00:57,732][00611] Updated weights for policy 0, policy_version 27542 (0.0007) [2023-10-08 05:00:58,102][00611] Updated weights for policy 0, policy_version 27552 (0.0009) [2023-10-08 05:00:58,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56557568. Throughput: 0: 1822.5, 1: 1849.5. Samples: 14144818. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 05:00:58,755][130385] Avg episode reward: [(0, '48.650'), (1, '45.860')] [2023-10-08 05:00:59,792][00612] Updated weights for policy 1, policy_version 27690 (0.0007) [2023-10-08 05:01:00,161][00612] Updated weights for policy 1, policy_version 27700 (0.0008) [2023-10-08 05:01:00,533][00612] Updated weights for policy 1, policy_version 27710 (0.0008) [2023-10-08 05:01:01,779][00611] Updated weights for policy 0, policy_version 27562 (0.0007) [2023-10-08 05:01:02,159][00611] Updated weights for policy 0, policy_version 27572 (0.0009) [2023-10-08 05:01:02,521][00611] Updated weights for policy 0, policy_version 27582 (0.0011) [2023-10-08 05:01:03,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 56623104. Throughput: 0: 1815.2, 1: 1854.8. Samples: 14166948. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 05:01:03,755][130385] Avg episode reward: [(0, '50.270'), (1, '48.520')] [2023-10-08 05:01:04,142][00612] Updated weights for policy 1, policy_version 27720 (0.0009) [2023-10-08 05:01:04,517][00612] Updated weights for policy 1, policy_version 27730 (0.0008) [2023-10-08 05:01:04,897][00612] Updated weights for policy 1, policy_version 27740 (0.0008) [2023-10-08 05:01:06,024][00611] Updated weights for policy 0, policy_version 27592 (0.0008) [2023-10-08 05:01:06,403][00611] Updated weights for policy 0, policy_version 27602 (0.0007) [2023-10-08 05:01:06,780][00611] Updated weights for policy 0, policy_version 27612 (0.0009) [2023-10-08 05:01:08,387][00612] Updated weights for policy 1, policy_version 27750 (0.0009) [2023-10-08 05:01:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 56688640. Throughput: 0: 1820.4, 1: 1852.4. Samples: 14178136. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 05:01:08,755][130385] Avg episode reward: [(0, '48.500'), (1, '49.540')] [2023-10-08 05:01:08,761][00612] Updated weights for policy 1, policy_version 27760 (0.0009) [2023-10-08 05:01:09,136][00612] Updated weights for policy 1, policy_version 27770 (0.0008) [2023-10-08 05:01:10,301][00611] Updated weights for policy 0, policy_version 27622 (0.0011) [2023-10-08 05:01:10,673][00611] Updated weights for policy 0, policy_version 27632 (0.0010) [2023-10-08 05:01:11,047][00611] Updated weights for policy 0, policy_version 27642 (0.0009) [2023-10-08 05:01:12,875][00612] Updated weights for policy 1, policy_version 27780 (0.0008) [2023-10-08 05:01:13,235][00612] Updated weights for policy 1, policy_version 27790 (0.0008) [2023-10-08 05:01:13,608][00612] Updated weights for policy 1, policy_version 27800 (0.0009) [2023-10-08 05:01:13,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 56754176. Throughput: 0: 1823.5, 1: 1846.3. Samples: 14200294. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 05:01:13,754][130385] Avg episode reward: [(0, '48.240'), (1, '51.430')] [2023-10-08 05:01:14,688][00611] Updated weights for policy 0, policy_version 27652 (0.0010) [2023-10-08 05:01:15,067][00611] Updated weights for policy 0, policy_version 27662 (0.0008) [2023-10-08 05:01:15,433][00611] Updated weights for policy 0, policy_version 27672 (0.0009) [2023-10-08 05:01:17,199][00612] Updated weights for policy 1, policy_version 27810 (0.0008) [2023-10-08 05:01:17,566][00612] Updated weights for policy 1, policy_version 27820 (0.0009) [2023-10-08 05:01:17,943][00612] Updated weights for policy 1, policy_version 27830 (0.0011) [2023-10-08 05:01:18,302][00612] Updated weights for policy 1, policy_version 27840 (0.0009) [2023-10-08 05:01:18,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 56852480. Throughput: 0: 1832.0, 1: 1829.6. Samples: 14222458. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-08 05:01:18,754][130385] Avg episode reward: [(0, '44.900'), (1, '51.920')] [2023-10-08 05:01:18,863][00611] Updated weights for policy 0, policy_version 27682 (0.0009) [2023-10-08 05:01:19,227][00611] Updated weights for policy 0, policy_version 27692 (0.0008) [2023-10-08 05:01:19,608][00611] Updated weights for policy 0, policy_version 27702 (0.0008) [2023-10-08 05:01:19,973][00611] Updated weights for policy 0, policy_version 27712 (0.0011) [2023-10-08 05:01:21,941][00612] Updated weights for policy 1, policy_version 27850 (0.0008) [2023-10-08 05:01:22,314][00612] Updated weights for policy 1, policy_version 27860 (0.0008) [2023-10-08 05:01:22,682][00612] Updated weights for policy 1, policy_version 27870 (0.0009) [2023-10-08 05:01:23,626][00611] Updated weights for policy 0, policy_version 27722 (0.0007) [2023-10-08 05:01:23,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56918016. Throughput: 0: 1832.5, 1: 1838.6. Samples: 14233696. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-08 05:01:23,755][130385] Avg episode reward: [(0, '41.180'), (1, '52.100')] [2023-10-08 05:01:24,005][00611] Updated weights for policy 0, policy_version 27732 (0.0008) [2023-10-08 05:01:24,370][00611] Updated weights for policy 0, policy_version 27742 (0.0008) [2023-10-08 05:01:26,395][00612] Updated weights for policy 1, policy_version 27880 (0.0008) [2023-10-08 05:01:26,773][00612] Updated weights for policy 1, policy_version 27890 (0.0007) [2023-10-08 05:01:27,135][00612] Updated weights for policy 1, policy_version 27900 (0.0007) [2023-10-08 05:01:28,067][00611] Updated weights for policy 0, policy_version 27752 (0.0008) [2023-10-08 05:01:28,424][00611] Updated weights for policy 0, policy_version 27762 (0.0007) [2023-10-08 05:01:28,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56983552. Throughput: 0: 1836.1, 1: 1838.8. Samples: 14255900. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-08 05:01:28,755][130385] Avg episode reward: [(0, '43.260'), (1, '49.630')] [2023-10-08 05:01:28,806][00611] Updated weights for policy 0, policy_version 27772 (0.0008) [2023-10-08 05:01:30,734][00612] Updated weights for policy 1, policy_version 27910 (0.0009) [2023-10-08 05:01:31,114][00612] Updated weights for policy 1, policy_version 27920 (0.0009) [2023-10-08 05:01:31,468][00612] Updated weights for policy 1, policy_version 27930 (0.0009) [2023-10-08 05:01:32,516][00611] Updated weights for policy 0, policy_version 27782 (0.0008) [2023-10-08 05:01:32,893][00611] Updated weights for policy 0, policy_version 27792 (0.0008) [2023-10-08 05:01:33,267][00611] Updated weights for policy 0, policy_version 27802 (0.0008) [2023-10-08 05:01:33,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57081856. Throughput: 0: 1821.6, 1: 1851.6. Samples: 14277614. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-08 05:01:33,754][130385] Avg episode reward: [(0, '44.510'), (1, '50.750')] [2023-10-08 05:01:35,104][00612] Updated weights for policy 1, policy_version 27940 (0.0009) [2023-10-08 05:01:35,472][00612] Updated weights for policy 1, policy_version 27950 (0.0009) [2023-10-08 05:01:35,839][00612] Updated weights for policy 1, policy_version 27960 (0.0007) [2023-10-08 05:01:36,803][00611] Updated weights for policy 0, policy_version 27812 (0.0008) [2023-10-08 05:01:37,175][00611] Updated weights for policy 0, policy_version 27822 (0.0009) [2023-10-08 05:01:37,545][00611] Updated weights for policy 0, policy_version 27832 (0.0009) [2023-10-08 05:01:38,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57147392. Throughput: 0: 1841.8, 1: 1840.5. Samples: 14289134. Policy #0 lag: (min: 4.0, avg: 15.1, max: 36.0) [2023-10-08 05:01:38,755][130385] Avg episode reward: [(0, '49.330'), (1, '50.680')] [2023-10-08 05:01:39,301][00612] Updated weights for policy 1, policy_version 27970 (0.0009) [2023-10-08 05:01:39,663][00612] Updated weights for policy 1, policy_version 27980 (0.0009) [2023-10-08 05:01:40,046][00612] Updated weights for policy 1, policy_version 27990 (0.0007) [2023-10-08 05:01:40,403][00612] Updated weights for policy 1, policy_version 28000 (0.0009) [2023-10-08 05:01:41,181][00611] Updated weights for policy 0, policy_version 27842 (0.0010) [2023-10-08 05:01:41,587][00611] Updated weights for policy 0, policy_version 27852 (0.0009) [2023-10-08 05:01:41,953][00611] Updated weights for policy 0, policy_version 27862 (0.0009) [2023-10-08 05:01:42,324][00611] Updated weights for policy 0, policy_version 27872 (0.0010) [2023-10-08 05:01:43,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57212928. Throughput: 0: 1828.8, 1: 1869.9. Samples: 14311260. Policy #0 lag: (min: 4.0, avg: 15.1, max: 36.0) [2023-10-08 05:01:43,755][130385] Avg episode reward: [(0, '48.180'), (1, '51.460')] [2023-10-08 05:01:43,995][00612] Updated weights for policy 1, policy_version 28010 (0.0010) [2023-10-08 05:01:44,361][00612] Updated weights for policy 1, policy_version 28020 (0.0009) [2023-10-08 05:01:44,732][00612] Updated weights for policy 1, policy_version 28030 (0.0010) [2023-10-08 05:01:45,949][00611] Updated weights for policy 0, policy_version 27882 (0.0007) [2023-10-08 05:01:46,327][00611] Updated weights for policy 0, policy_version 27892 (0.0009) [2023-10-08 05:01:46,706][00611] Updated weights for policy 0, policy_version 27902 (0.0008) [2023-10-08 05:01:48,278][00612] Updated weights for policy 1, policy_version 28040 (0.0007) [2023-10-08 05:01:48,645][00612] Updated weights for policy 1, policy_version 28050 (0.0008) [2023-10-08 05:01:48,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57278464. Throughput: 0: 1855.5, 1: 1860.0. Samples: 14334144. Policy #0 lag: (min: 4.0, avg: 15.1, max: 36.0) [2023-10-08 05:01:48,754][130385] Avg episode reward: [(0, '47.590'), (1, '50.190')] [2023-10-08 05:01:49,020][00612] Updated weights for policy 1, policy_version 28060 (0.0008) [2023-10-08 05:01:50,275][00611] Updated weights for policy 0, policy_version 27912 (0.0009) [2023-10-08 05:01:50,641][00611] Updated weights for policy 0, policy_version 27922 (0.0007) [2023-10-08 05:01:51,009][00611] Updated weights for policy 0, policy_version 27932 (0.0009) [2023-10-08 05:01:52,534][00612] Updated weights for policy 1, policy_version 28070 (0.0007) [2023-10-08 05:01:52,906][00612] Updated weights for policy 1, policy_version 28080 (0.0007) [2023-10-08 05:01:53,268][00612] Updated weights for policy 1, policy_version 28090 (0.0008) [2023-10-08 05:01:53,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 57376768. Throughput: 0: 1830.0, 1: 1871.3. Samples: 14344698. Policy #0 lag: (min: 4.0, avg: 15.1, max: 36.0) [2023-10-08 05:01:53,755][130385] Avg episode reward: [(0, '47.350'), (1, '49.730')] [2023-10-08 05:01:54,778][00611] Updated weights for policy 0, policy_version 27942 (0.0009) [2023-10-08 05:01:55,144][00611] Updated weights for policy 0, policy_version 27952 (0.0008) [2023-10-08 05:01:55,513][00611] Updated weights for policy 0, policy_version 27962 (0.0009) [2023-10-08 05:01:56,813][00612] Updated weights for policy 1, policy_version 28100 (0.0008) [2023-10-08 05:01:57,182][00612] Updated weights for policy 1, policy_version 28110 (0.0007) [2023-10-08 05:01:57,554][00612] Updated weights for policy 1, policy_version 28120 (0.0007) [2023-10-08 05:01:58,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57442304. Throughput: 0: 1846.2, 1: 1860.9. Samples: 14367114. Policy #0 lag: (min: 4.0, avg: 15.1, max: 36.0) [2023-10-08 05:01:58,755][130385] Avg episode reward: [(0, '49.960'), (1, '49.710')] [2023-10-08 05:01:59,022][00611] Updated weights for policy 0, policy_version 27972 (0.0009) [2023-10-08 05:01:59,392][00611] Updated weights for policy 0, policy_version 27982 (0.0008) [2023-10-08 05:01:59,760][00611] Updated weights for policy 0, policy_version 27992 (0.0009) [2023-10-08 05:02:01,155][00612] Updated weights for policy 1, policy_version 28130 (0.0007) [2023-10-08 05:02:01,522][00612] Updated weights for policy 1, policy_version 28140 (0.0008) [2023-10-08 05:02:01,902][00612] Updated weights for policy 1, policy_version 28150 (0.0007) [2023-10-08 05:02:02,269][00612] Updated weights for policy 1, policy_version 28160 (0.0009) [2023-10-08 05:02:03,423][00611] Updated weights for policy 0, policy_version 28002 (0.0009) [2023-10-08 05:02:03,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 57507840. Throughput: 0: 1842.8, 1: 1877.1. Samples: 14389856. Policy #0 lag: (min: 2.0, avg: 2.4, max: 15.0) [2023-10-08 05:02:03,754][130385] Avg episode reward: [(0, '50.630'), (1, '51.910')] [2023-10-08 05:02:03,789][00611] Updated weights for policy 0, policy_version 28012 (0.0010) [2023-10-08 05:02:04,169][00611] Updated weights for policy 0, policy_version 28022 (0.0012) [2023-10-08 05:02:04,529][00611] Updated weights for policy 0, policy_version 28032 (0.0010) [2023-10-08 05:02:05,784][00612] Updated weights for policy 1, policy_version 28170 (0.0008) [2023-10-08 05:02:06,144][00612] Updated weights for policy 1, policy_version 28180 (0.0010) [2023-10-08 05:02:06,516][00612] Updated weights for policy 1, policy_version 28190 (0.0008) [2023-10-08 05:02:08,133][00611] Updated weights for policy 0, policy_version 28042 (0.0007) [2023-10-08 05:02:08,502][00611] Updated weights for policy 0, policy_version 28052 (0.0007) [2023-10-08 05:02:08,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 57573376. Throughput: 0: 1846.4, 1: 1861.8. Samples: 14400564. Policy #0 lag: (min: 2.0, avg: 2.4, max: 15.0) [2023-10-08 05:02:08,754][130385] Avg episode reward: [(0, '52.690'), (1, '49.850')] [2023-10-08 05:02:08,874][00611] Updated weights for policy 0, policy_version 28062 (0.0007) [2023-10-08 05:02:10,068][00612] Updated weights for policy 1, policy_version 28200 (0.0008) [2023-10-08 05:02:10,443][00612] Updated weights for policy 1, policy_version 28210 (0.0010) [2023-10-08 05:02:10,807][00612] Updated weights for policy 1, policy_version 28220 (0.0009) [2023-10-08 05:02:12,364][00611] Updated weights for policy 0, policy_version 28072 (0.0007) [2023-10-08 05:02:12,742][00611] Updated weights for policy 0, policy_version 28082 (0.0008) [2023-10-08 05:02:13,107][00611] Updated weights for policy 0, policy_version 28092 (0.0008) [2023-10-08 05:02:13,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 57671680. Throughput: 0: 1849.9, 1: 1876.3. Samples: 14423580. Policy #0 lag: (min: 2.0, avg: 2.4, max: 15.0) [2023-10-08 05:02:13,755][130385] Avg episode reward: [(0, '53.940'), (1, '51.900')] [2023-10-08 05:02:13,755][00365] Saving new best policy, reward=53.940! [2023-10-08 05:02:14,581][00612] Updated weights for policy 1, policy_version 28230 (0.0009) [2023-10-08 05:02:14,962][00612] Updated weights for policy 1, policy_version 28240 (0.0011) [2023-10-08 05:02:15,330][00612] Updated weights for policy 1, policy_version 28250 (0.0008) [2023-10-08 05:02:16,861][00611] Updated weights for policy 0, policy_version 28102 (0.0008) [2023-10-08 05:02:17,234][00611] Updated weights for policy 0, policy_version 28112 (0.0010) [2023-10-08 05:02:17,618][00611] Updated weights for policy 0, policy_version 28122 (0.0008) [2023-10-08 05:02:18,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57737216. Throughput: 0: 1840.2, 1: 1881.6. Samples: 14445094. Policy #0 lag: (min: 2.0, avg: 2.4, max: 15.0) [2023-10-08 05:02:18,755][130385] Avg episode reward: [(0, '53.840'), (1, '52.310')] [2023-10-08 05:02:18,762][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000028128_28803072.pth... [2023-10-08 05:02:18,763][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000028256_28934144.pth... [2023-10-08 05:02:18,802][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000026528_27164672.pth [2023-10-08 05:02:18,804][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000026400_27033600.pth [2023-10-08 05:02:19,135][00612] Updated weights for policy 1, policy_version 28260 (0.0010) [2023-10-08 05:02:19,530][00612] Updated weights for policy 1, policy_version 28270 (0.0009) [2023-10-08 05:02:19,900][00612] Updated weights for policy 1, policy_version 28280 (0.0009) [2023-10-08 05:02:21,235][00611] Updated weights for policy 0, policy_version 28132 (0.0008) [2023-10-08 05:02:21,606][00611] Updated weights for policy 0, policy_version 28142 (0.0010) [2023-10-08 05:02:21,979][00611] Updated weights for policy 0, policy_version 28152 (0.0007) [2023-10-08 05:02:23,487][00612] Updated weights for policy 1, policy_version 28290 (0.0008) [2023-10-08 05:02:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57802752. Throughput: 0: 1843.6, 1: 1871.3. Samples: 14456300. Policy #0 lag: (min: 1.0, avg: 16.2, max: 33.0) [2023-10-08 05:02:23,754][130385] Avg episode reward: [(0, '52.680'), (1, '52.960')] [2023-10-08 05:02:23,845][00612] Updated weights for policy 1, policy_version 28300 (0.0009) [2023-10-08 05:02:24,216][00612] Updated weights for policy 1, policy_version 28310 (0.0007) [2023-10-08 05:02:24,582][00425] Saving new best policy, reward=52.960! [2023-10-08 05:02:24,584][00612] Updated weights for policy 1, policy_version 28320 (0.0007) [2023-10-08 05:02:25,697][00611] Updated weights for policy 0, policy_version 28162 (0.0008) [2023-10-08 05:02:26,073][00611] Updated weights for policy 0, policy_version 28172 (0.0007) [2023-10-08 05:02:26,450][00611] Updated weights for policy 0, policy_version 28182 (0.0007) [2023-10-08 05:02:26,818][00611] Updated weights for policy 0, policy_version 28192 (0.0007) [2023-10-08 05:02:28,051][00612] Updated weights for policy 1, policy_version 28330 (0.0009) [2023-10-08 05:02:28,427][00612] Updated weights for policy 1, policy_version 28340 (0.0008) [2023-10-08 05:02:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57868288. Throughput: 0: 1840.4, 1: 1869.1. Samples: 14478186. Policy #0 lag: (min: 1.0, avg: 16.2, max: 33.0) [2023-10-08 05:02:28,754][130385] Avg episode reward: [(0, '52.370'), (1, '54.140')] [2023-10-08 05:02:28,805][00612] Updated weights for policy 1, policy_version 28350 (0.0011) [2023-10-08 05:02:28,871][00425] Saving new best policy, reward=54.140! [2023-10-08 05:02:30,634][00611] Updated weights for policy 0, policy_version 28202 (0.0007) [2023-10-08 05:02:31,006][00611] Updated weights for policy 0, policy_version 28212 (0.0010) [2023-10-08 05:02:31,381][00611] Updated weights for policy 0, policy_version 28222 (0.0011) [2023-10-08 05:02:32,303][00612] Updated weights for policy 1, policy_version 28360 (0.0009) [2023-10-08 05:02:32,680][00612] Updated weights for policy 1, policy_version 28370 (0.0011) [2023-10-08 05:02:33,061][00612] Updated weights for policy 1, policy_version 28380 (0.0010) [2023-10-08 05:02:33,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 57966592. Throughput: 0: 1843.5, 1: 1840.3. Samples: 14499916. Policy #0 lag: (min: 1.0, avg: 16.2, max: 33.0) [2023-10-08 05:02:33,755][130385] Avg episode reward: [(0, '52.780'), (1, '52.750')] [2023-10-08 05:02:34,790][00611] Updated weights for policy 0, policy_version 28232 (0.0007) [2023-10-08 05:02:35,163][00611] Updated weights for policy 0, policy_version 28242 (0.0007) [2023-10-08 05:02:35,531][00611] Updated weights for policy 0, policy_version 28252 (0.0009) [2023-10-08 05:02:36,683][00612] Updated weights for policy 1, policy_version 28390 (0.0009) [2023-10-08 05:02:37,059][00612] Updated weights for policy 1, policy_version 28400 (0.0009) [2023-10-08 05:02:37,421][00612] Updated weights for policy 1, policy_version 28410 (0.0010) [2023-10-08 05:02:38,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 58032128. Throughput: 0: 1839.8, 1: 1866.2. Samples: 14511466. Policy #0 lag: (min: 1.0, avg: 16.2, max: 33.0) [2023-10-08 05:02:38,754][130385] Avg episode reward: [(0, '55.640'), (1, '55.160')] [2023-10-08 05:02:38,755][00365] Saving new best policy, reward=55.640! [2023-10-08 05:02:38,755][00425] Saving new best policy, reward=55.160! [2023-10-08 05:02:39,285][00611] Updated weights for policy 0, policy_version 28262 (0.0009) [2023-10-08 05:02:39,647][00611] Updated weights for policy 0, policy_version 28272 (0.0008) [2023-10-08 05:02:40,027][00611] Updated weights for policy 0, policy_version 28282 (0.0010) [2023-10-08 05:02:40,958][00612] Updated weights for policy 1, policy_version 28420 (0.0010) [2023-10-08 05:02:41,335][00612] Updated weights for policy 1, policy_version 28430 (0.0008) [2023-10-08 05:02:41,715][00612] Updated weights for policy 1, policy_version 28440 (0.0011) [2023-10-08 05:02:43,707][00611] Updated weights for policy 0, policy_version 28292 (0.0010) [2023-10-08 05:02:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 58097664. Throughput: 0: 1841.7, 1: 1839.6. Samples: 14532776. Policy #0 lag: (min: 1.0, avg: 16.2, max: 33.0) [2023-10-08 05:02:43,754][130385] Avg episode reward: [(0, '55.670'), (1, '56.870')] [2023-10-08 05:02:43,755][00425] Saving new best policy, reward=56.870! [2023-10-08 05:02:44,069][00611] Updated weights for policy 0, policy_version 28302 (0.0010) [2023-10-08 05:02:44,443][00611] Updated weights for policy 0, policy_version 28312 (0.0010) [2023-10-08 05:02:44,741][00365] Saving new best policy, reward=55.670! [2023-10-08 05:02:45,269][00612] Updated weights for policy 1, policy_version 28450 (0.0009) [2023-10-08 05:02:45,642][00612] Updated weights for policy 1, policy_version 28460 (0.0010) [2023-10-08 05:02:46,012][00612] Updated weights for policy 1, policy_version 28470 (0.0010) [2023-10-08 05:02:46,383][00612] Updated weights for policy 1, policy_version 28480 (0.0010) [2023-10-08 05:02:48,225][00611] Updated weights for policy 0, policy_version 28322 (0.0007) [2023-10-08 05:02:48,600][00611] Updated weights for policy 0, policy_version 28332 (0.0007) [2023-10-08 05:02:48,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 58163200. Throughput: 0: 1832.9, 1: 1849.5. Samples: 14555566. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 05:02:48,755][130385] Avg episode reward: [(0, '51.970'), (1, '54.550')] [2023-10-08 05:02:48,966][00611] Updated weights for policy 0, policy_version 28342 (0.0007) [2023-10-08 05:02:49,337][00611] Updated weights for policy 0, policy_version 28352 (0.0007) [2023-10-08 05:02:50,204][00612] Updated weights for policy 1, policy_version 28490 (0.0010) [2023-10-08 05:02:50,574][00612] Updated weights for policy 1, policy_version 28500 (0.0010) [2023-10-08 05:02:50,948][00612] Updated weights for policy 1, policy_version 28510 (0.0008) [2023-10-08 05:02:53,019][00611] Updated weights for policy 0, policy_version 28362 (0.0009) [2023-10-08 05:02:53,391][00611] Updated weights for policy 0, policy_version 28372 (0.0007) [2023-10-08 05:02:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 58228736. Throughput: 0: 1831.1, 1: 1831.0. Samples: 14565358. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 05:02:53,754][130385] Avg episode reward: [(0, '49.420'), (1, '53.070')] [2023-10-08 05:02:53,765][00611] Updated weights for policy 0, policy_version 28382 (0.0007) [2023-10-08 05:02:54,492][00612] Updated weights for policy 1, policy_version 28520 (0.0011) [2023-10-08 05:02:54,872][00612] Updated weights for policy 1, policy_version 28530 (0.0009) [2023-10-08 05:02:55,239][00612] Updated weights for policy 1, policy_version 28540 (0.0009) [2023-10-08 05:02:57,517][00611] Updated weights for policy 0, policy_version 28392 (0.0009) [2023-10-08 05:02:57,891][00611] Updated weights for policy 0, policy_version 28402 (0.0007) [2023-10-08 05:02:58,265][00611] Updated weights for policy 0, policy_version 28412 (0.0010) [2023-10-08 05:02:58,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 58327040. Throughput: 0: 1819.7, 1: 1841.7. Samples: 14588344. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 05:02:58,754][130385] Avg episode reward: [(0, '50.460'), (1, '55.010')] [2023-10-08 05:02:58,921][00612] Updated weights for policy 1, policy_version 28550 (0.0010) [2023-10-08 05:02:59,288][00612] Updated weights for policy 1, policy_version 28560 (0.0009) [2023-10-08 05:02:59,654][00612] Updated weights for policy 1, policy_version 28570 (0.0008) [2023-10-08 05:03:01,909][00611] Updated weights for policy 0, policy_version 28422 (0.0007) [2023-10-08 05:03:02,282][00611] Updated weights for policy 0, policy_version 28432 (0.0008) [2023-10-08 05:03:02,654][00611] Updated weights for policy 0, policy_version 28442 (0.0008) [2023-10-08 05:03:03,239][00612] Updated weights for policy 1, policy_version 28580 (0.0010) [2023-10-08 05:03:03,605][00612] Updated weights for policy 1, policy_version 28590 (0.0010) [2023-10-08 05:03:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 58392576. Throughput: 0: 1821.6, 1: 1843.2. Samples: 14610014. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 05:03:03,754][130385] Avg episode reward: [(0, '49.810'), (1, '56.160')] [2023-10-08 05:03:03,981][00612] Updated weights for policy 1, policy_version 28600 (0.0008) [2023-10-08 05:03:06,246][00611] Updated weights for policy 0, policy_version 28452 (0.0009) [2023-10-08 05:03:06,625][00611] Updated weights for policy 0, policy_version 28462 (0.0010) [2023-10-08 05:03:06,996][00611] Updated weights for policy 0, policy_version 28472 (0.0010) [2023-10-08 05:03:07,696][00612] Updated weights for policy 1, policy_version 28610 (0.0008) [2023-10-08 05:03:08,086][00612] Updated weights for policy 1, policy_version 28620 (0.0011) [2023-10-08 05:03:08,450][00612] Updated weights for policy 1, policy_version 28630 (0.0010) [2023-10-08 05:03:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 58458112. Throughput: 0: 1823.5, 1: 1852.4. Samples: 14621714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:03:08,754][130385] Avg episode reward: [(0, '48.590'), (1, '54.300')] [2023-10-08 05:03:08,818][00612] Updated weights for policy 1, policy_version 28640 (0.0011) [2023-10-08 05:03:10,709][00611] Updated weights for policy 0, policy_version 28482 (0.0009) [2023-10-08 05:03:11,077][00611] Updated weights for policy 0, policy_version 28492 (0.0008) [2023-10-08 05:03:11,457][00611] Updated weights for policy 0, policy_version 28502 (0.0010) [2023-10-08 05:03:11,816][00611] Updated weights for policy 0, policy_version 28512 (0.0009) [2023-10-08 05:03:12,393][00612] Updated weights for policy 1, policy_version 28650 (0.0008) [2023-10-08 05:03:12,767][00612] Updated weights for policy 1, policy_version 28660 (0.0007) [2023-10-08 05:03:13,137][00612] Updated weights for policy 1, policy_version 28670 (0.0008) [2023-10-08 05:03:13,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 58556416. Throughput: 0: 1823.0, 1: 1844.3. Samples: 14643212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:03:13,755][130385] Avg episode reward: [(0, '48.600'), (1, '53.100')] [2023-10-08 05:03:15,489][00611] Updated weights for policy 0, policy_version 28522 (0.0008) [2023-10-08 05:03:15,858][00611] Updated weights for policy 0, policy_version 28532 (0.0008) [2023-10-08 05:03:16,240][00611] Updated weights for policy 0, policy_version 28542 (0.0008) [2023-10-08 05:03:16,787][00612] Updated weights for policy 1, policy_version 28680 (0.0008) [2023-10-08 05:03:17,154][00612] Updated weights for policy 1, policy_version 28690 (0.0008) [2023-10-08 05:03:17,526][00612] Updated weights for policy 1, policy_version 28700 (0.0007) [2023-10-08 05:03:18,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 58621952. Throughput: 0: 1828.1, 1: 1848.1. Samples: 14665346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:03:18,755][130385] Avg episode reward: [(0, '46.460'), (1, '51.940')] [2023-10-08 05:03:19,880][00611] Updated weights for policy 0, policy_version 28552 (0.0008) [2023-10-08 05:03:20,252][00611] Updated weights for policy 0, policy_version 28562 (0.0007) [2023-10-08 05:03:20,622][00611] Updated weights for policy 0, policy_version 28572 (0.0007) [2023-10-08 05:03:21,159][00612] Updated weights for policy 1, policy_version 28710 (0.0009) [2023-10-08 05:03:21,522][00612] Updated weights for policy 1, policy_version 28720 (0.0007) [2023-10-08 05:03:21,899][00612] Updated weights for policy 1, policy_version 28730 (0.0009) [2023-10-08 05:03:23,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 58687488. Throughput: 0: 1825.4, 1: 1843.6. Samples: 14676568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:03:23,755][130385] Avg episode reward: [(0, '47.990'), (1, '51.270')] [2023-10-08 05:03:24,295][00611] Updated weights for policy 0, policy_version 28582 (0.0008) [2023-10-08 05:03:24,669][00611] Updated weights for policy 0, policy_version 28592 (0.0009) [2023-10-08 05:03:25,038][00611] Updated weights for policy 0, policy_version 28602 (0.0008) [2023-10-08 05:03:25,300][00612] Updated weights for policy 1, policy_version 28740 (0.0008) [2023-10-08 05:03:25,663][00612] Updated weights for policy 1, policy_version 28750 (0.0009) [2023-10-08 05:03:26,031][00612] Updated weights for policy 1, policy_version 28760 (0.0009) [2023-10-08 05:03:28,649][00611] Updated weights for policy 0, policy_version 28612 (0.0007) [2023-10-08 05:03:28,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 58753024. Throughput: 0: 1830.0, 1: 1860.2. Samples: 14698832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:03:28,754][130385] Avg episode reward: [(0, '45.560'), (1, '48.190')] [2023-10-08 05:03:29,017][00611] Updated weights for policy 0, policy_version 28622 (0.0009) [2023-10-08 05:03:29,389][00611] Updated weights for policy 0, policy_version 28632 (0.0011) [2023-10-08 05:03:29,553][00612] Updated weights for policy 1, policy_version 28770 (0.0008) [2023-10-08 05:03:29,924][00612] Updated weights for policy 1, policy_version 28780 (0.0008) [2023-10-08 05:03:30,292][00612] Updated weights for policy 1, policy_version 28790 (0.0009) [2023-10-08 05:03:30,652][00612] Updated weights for policy 1, policy_version 28800 (0.0008) [2023-10-08 05:03:32,933][00611] Updated weights for policy 0, policy_version 28642 (0.0007) [2023-10-08 05:03:33,314][00611] Updated weights for policy 0, policy_version 28652 (0.0007) [2023-10-08 05:03:33,691][00611] Updated weights for policy 0, policy_version 28662 (0.0008) [2023-10-08 05:03:33,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 58818560. Throughput: 0: 1830.9, 1: 1869.8. Samples: 14722098. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 05:03:33,755][130385] Avg episode reward: [(0, '47.160'), (1, '47.240')] [2023-10-08 05:03:34,061][00611] Updated weights for policy 0, policy_version 28672 (0.0008) [2023-10-08 05:03:34,233][00612] Updated weights for policy 1, policy_version 28810 (0.0009) [2023-10-08 05:03:34,604][00612] Updated weights for policy 1, policy_version 28820 (0.0009) [2023-10-08 05:03:34,977][00612] Updated weights for policy 1, policy_version 28830 (0.0009) [2023-10-08 05:03:37,679][00611] Updated weights for policy 0, policy_version 28682 (0.0008) [2023-10-08 05:03:38,047][00611] Updated weights for policy 0, policy_version 28692 (0.0008) [2023-10-08 05:03:38,417][00611] Updated weights for policy 0, policy_version 28702 (0.0008) [2023-10-08 05:03:38,571][00612] Updated weights for policy 1, policy_version 28840 (0.0010) [2023-10-08 05:03:38,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 58916864. Throughput: 0: 1840.3, 1: 1874.5. Samples: 14732526. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 05:03:38,754][130385] Avg episode reward: [(0, '45.380'), (1, '47.320')] [2023-10-08 05:03:38,941][00612] Updated weights for policy 1, policy_version 28850 (0.0008) [2023-10-08 05:03:39,306][00612] Updated weights for policy 1, policy_version 28860 (0.0008) [2023-10-08 05:03:42,072][00611] Updated weights for policy 0, policy_version 28712 (0.0010) [2023-10-08 05:03:42,440][00611] Updated weights for policy 0, policy_version 28722 (0.0011) [2023-10-08 05:03:42,808][00611] Updated weights for policy 0, policy_version 28732 (0.0010) [2023-10-08 05:03:43,027][00612] Updated weights for policy 1, policy_version 28870 (0.0008) [2023-10-08 05:03:43,393][00612] Updated weights for policy 1, policy_version 28880 (0.0008) [2023-10-08 05:03:43,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 58982400. Throughput: 0: 1833.7, 1: 1874.9. Samples: 14755230. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 05:03:43,754][130385] Avg episode reward: [(0, '46.530'), (1, '45.310')] [2023-10-08 05:03:43,773][00612] Updated weights for policy 1, policy_version 28890 (0.0008) [2023-10-08 05:03:46,479][00611] Updated weights for policy 0, policy_version 28742 (0.0009) [2023-10-08 05:03:46,849][00611] Updated weights for policy 0, policy_version 28752 (0.0008) [2023-10-08 05:03:47,220][00611] Updated weights for policy 0, policy_version 28762 (0.0008) [2023-10-08 05:03:47,382][00612] Updated weights for policy 1, policy_version 28900 (0.0007) [2023-10-08 05:03:47,745][00612] Updated weights for policy 1, policy_version 28910 (0.0008) [2023-10-08 05:03:48,115][00612] Updated weights for policy 1, policy_version 28920 (0.0009) [2023-10-08 05:03:48,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 59080704. Throughput: 0: 1842.6, 1: 1846.1. Samples: 14776006. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 05:03:48,754][130385] Avg episode reward: [(0, '42.600'), (1, '43.060')] [2023-10-08 05:03:50,934][00611] Updated weights for policy 0, policy_version 28772 (0.0009) [2023-10-08 05:03:51,306][00611] Updated weights for policy 0, policy_version 28782 (0.0008) [2023-10-08 05:03:51,691][00611] Updated weights for policy 0, policy_version 28792 (0.0008) [2023-10-08 05:03:51,784][00612] Updated weights for policy 1, policy_version 28930 (0.0009) [2023-10-08 05:03:52,150][00612] Updated weights for policy 1, policy_version 28940 (0.0010) [2023-10-08 05:03:52,520][00612] Updated weights for policy 1, policy_version 28950 (0.0008) [2023-10-08 05:03:52,884][00612] Updated weights for policy 1, policy_version 28960 (0.0008) [2023-10-08 05:03:53,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 59146240. Throughput: 0: 1829.1, 1: 1867.5. Samples: 14788062. Policy #0 lag: (min: 24.0, avg: 50.1, max: 56.0) [2023-10-08 05:03:53,755][130385] Avg episode reward: [(0, '45.190'), (1, '45.670')] [2023-10-08 05:03:55,144][00611] Updated weights for policy 0, policy_version 28802 (0.0009) [2023-10-08 05:03:55,522][00611] Updated weights for policy 0, policy_version 28812 (0.0008) [2023-10-08 05:03:55,886][00611] Updated weights for policy 0, policy_version 28822 (0.0007) [2023-10-08 05:03:56,255][00611] Updated weights for policy 0, policy_version 28832 (0.0009) [2023-10-08 05:03:56,612][00612] Updated weights for policy 1, policy_version 28970 (0.0008) [2023-10-08 05:03:56,989][00612] Updated weights for policy 1, policy_version 28980 (0.0008) [2023-10-08 05:03:57,358][00612] Updated weights for policy 1, policy_version 28990 (0.0009) [2023-10-08 05:03:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 59211776. Throughput: 0: 1839.8, 1: 1839.5. Samples: 14808782. Policy #0 lag: (min: 24.0, avg: 50.1, max: 56.0) [2023-10-08 05:03:58,754][130385] Avg episode reward: [(0, '45.940'), (1, '45.430')] [2023-10-08 05:04:00,014][00611] Updated weights for policy 0, policy_version 28842 (0.0009) [2023-10-08 05:04:00,378][00611] Updated weights for policy 0, policy_version 28852 (0.0008) [2023-10-08 05:04:00,750][00611] Updated weights for policy 0, policy_version 28862 (0.0009) [2023-10-08 05:04:00,987][00612] Updated weights for policy 1, policy_version 29000 (0.0007) [2023-10-08 05:04:01,364][00612] Updated weights for policy 1, policy_version 29010 (0.0007) [2023-10-08 05:04:01,731][00612] Updated weights for policy 1, policy_version 29020 (0.0009) [2023-10-08 05:04:03,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 59277312. Throughput: 0: 1830.6, 1: 1857.2. Samples: 14831298. Policy #0 lag: (min: 24.0, avg: 50.1, max: 56.0) [2023-10-08 05:04:03,754][130385] Avg episode reward: [(0, '45.560'), (1, '44.050')] [2023-10-08 05:04:04,604][00611] Updated weights for policy 0, policy_version 28872 (0.0008) [2023-10-08 05:04:04,974][00611] Updated weights for policy 0, policy_version 28882 (0.0010) [2023-10-08 05:04:05,210][00612] Updated weights for policy 1, policy_version 29030 (0.0009) [2023-10-08 05:04:05,345][00611] Updated weights for policy 0, policy_version 28892 (0.0008) [2023-10-08 05:04:05,570][00612] Updated weights for policy 1, policy_version 29040 (0.0007) [2023-10-08 05:04:05,944][00612] Updated weights for policy 1, policy_version 29050 (0.0010) [2023-10-08 05:04:08,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 59342848. Throughput: 0: 1832.3, 1: 1829.6. Samples: 14841354. Policy #0 lag: (min: 24.0, avg: 50.1, max: 56.0) [2023-10-08 05:04:08,755][130385] Avg episode reward: [(0, '43.550'), (1, '47.550')] [2023-10-08 05:04:08,978][00611] Updated weights for policy 0, policy_version 28902 (0.0008) [2023-10-08 05:04:09,353][00611] Updated weights for policy 0, policy_version 28912 (0.0008) [2023-10-08 05:04:09,566][00612] Updated weights for policy 1, policy_version 29060 (0.0010) [2023-10-08 05:04:09,729][00611] Updated weights for policy 0, policy_version 28922 (0.0008) [2023-10-08 05:04:09,937][00612] Updated weights for policy 1, policy_version 29070 (0.0009) [2023-10-08 05:04:10,306][00612] Updated weights for policy 1, policy_version 29080 (0.0010) [2023-10-08 05:04:13,358][00611] Updated weights for policy 0, policy_version 28932 (0.0008) [2023-10-08 05:04:13,744][00611] Updated weights for policy 0, policy_version 28942 (0.0009) [2023-10-08 05:04:13,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 59408384. Throughput: 0: 1830.5, 1: 1846.0. Samples: 14864278. Policy #0 lag: (min: 24.0, avg: 50.1, max: 56.0) [2023-10-08 05:04:13,754][130385] Avg episode reward: [(0, '47.060'), (1, '47.430')] [2023-10-08 05:04:13,957][00612] Updated weights for policy 1, policy_version 29090 (0.0010) [2023-10-08 05:04:14,115][00611] Updated weights for policy 0, policy_version 28952 (0.0008) [2023-10-08 05:04:14,335][00612] Updated weights for policy 1, policy_version 29100 (0.0008) [2023-10-08 05:04:14,699][00612] Updated weights for policy 1, policy_version 29110 (0.0008) [2023-10-08 05:04:15,070][00612] Updated weights for policy 1, policy_version 29120 (0.0007) [2023-10-08 05:04:17,804][00611] Updated weights for policy 0, policy_version 28962 (0.0007) [2023-10-08 05:04:18,175][00611] Updated weights for policy 0, policy_version 28972 (0.0009) [2023-10-08 05:04:18,547][00611] Updated weights for policy 0, policy_version 28982 (0.0009) [2023-10-08 05:04:18,590][00612] Updated weights for policy 1, policy_version 29130 (0.0008) [2023-10-08 05:04:18,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 59473920. Throughput: 0: 1819.9, 1: 1847.5. Samples: 14887132. Policy #0 lag: (min: 19.0, avg: 21.8, max: 51.0) [2023-10-08 05:04:18,755][130385] Avg episode reward: [(0, '44.890'), (1, '46.100')] [2023-10-08 05:04:18,916][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000028992_29687808.pth... [2023-10-08 05:04:18,918][00611] Updated weights for policy 0, policy_version 28992 (0.0010) [2023-10-08 05:04:18,950][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000027264_27918336.pth [2023-10-08 05:04:18,953][00612] Updated weights for policy 1, policy_version 29140 (0.0008) [2023-10-08 05:04:19,320][00612] Updated weights for policy 1, policy_version 29150 (0.0008) [2023-10-08 05:04:19,399][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000029152_29851648.pth... [2023-10-08 05:04:19,435][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000027392_28049408.pth [2023-10-08 05:04:22,737][00611] Updated weights for policy 0, policy_version 29002 (0.0008) [2023-10-08 05:04:23,054][00612] Updated weights for policy 1, policy_version 29160 (0.0007) [2023-10-08 05:04:23,107][00611] Updated weights for policy 0, policy_version 29012 (0.0007) [2023-10-08 05:04:23,421][00612] Updated weights for policy 1, policy_version 29170 (0.0008) [2023-10-08 05:04:23,484][00611] Updated weights for policy 0, policy_version 29022 (0.0008) [2023-10-08 05:04:23,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 59572224. Throughput: 0: 1821.2, 1: 1846.1. Samples: 14897558. Policy #0 lag: (min: 19.0, avg: 21.8, max: 51.0) [2023-10-08 05:04:23,755][130385] Avg episode reward: [(0, '45.830'), (1, '47.290')] [2023-10-08 05:04:23,775][00612] Updated weights for policy 1, policy_version 29180 (0.0008) [2023-10-08 05:04:27,299][00611] Updated weights for policy 0, policy_version 29032 (0.0010) [2023-10-08 05:04:27,535][00612] Updated weights for policy 1, policy_version 29190 (0.0008) [2023-10-08 05:04:27,666][00611] Updated weights for policy 0, policy_version 29042 (0.0007) [2023-10-08 05:04:27,903][00612] Updated weights for policy 1, policy_version 29200 (0.0008) [2023-10-08 05:04:28,040][00611] Updated weights for policy 0, policy_version 29052 (0.0007) [2023-10-08 05:04:28,270][00612] Updated weights for policy 1, policy_version 29210 (0.0009) [2023-10-08 05:04:28,754][130385] Fps is (10 sec: 19661.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 59670528. Throughput: 0: 1821.2, 1: 1842.3. Samples: 14920088. Policy #0 lag: (min: 19.0, avg: 21.8, max: 51.0) [2023-10-08 05:04:28,754][130385] Avg episode reward: [(0, '44.850'), (1, '46.440')] [2023-10-08 05:04:31,592][00611] Updated weights for policy 0, policy_version 29062 (0.0008) [2023-10-08 05:04:31,920][00612] Updated weights for policy 1, policy_version 29220 (0.0010) [2023-10-08 05:04:31,972][00611] Updated weights for policy 0, policy_version 29072 (0.0008) [2023-10-08 05:04:32,289][00612] Updated weights for policy 1, policy_version 29230 (0.0007) [2023-10-08 05:04:32,348][00611] Updated weights for policy 0, policy_version 29082 (0.0007) [2023-10-08 05:04:32,669][00612] Updated weights for policy 1, policy_version 29240 (0.0008) [2023-10-08 05:04:33,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 59736064. Throughput: 0: 1818.6, 1: 1831.3. Samples: 14940254. Policy #0 lag: (min: 19.0, avg: 21.8, max: 51.0) [2023-10-08 05:04:33,755][130385] Avg episode reward: [(0, '44.000'), (1, '46.900')] [2023-10-08 05:04:35,900][00612] Updated weights for policy 1, policy_version 29250 (0.0009) [2023-10-08 05:04:36,086][00611] Updated weights for policy 0, policy_version 29092 (0.0008) [2023-10-08 05:04:36,267][00612] Updated weights for policy 1, policy_version 29260 (0.0010) [2023-10-08 05:04:36,463][00611] Updated weights for policy 0, policy_version 29102 (0.0008) [2023-10-08 05:04:36,646][00612] Updated weights for policy 1, policy_version 29270 (0.0008) [2023-10-08 05:04:36,823][00611] Updated weights for policy 0, policy_version 29112 (0.0009) [2023-10-08 05:04:37,008][00612] Updated weights for policy 1, policy_version 29280 (0.0008) [2023-10-08 05:04:38,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 59801600. Throughput: 0: 1820.8, 1: 1841.7. Samples: 14952874. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 05:04:38,755][130385] Avg episode reward: [(0, '44.430'), (1, '46.790')] [2023-10-08 05:04:40,405][00611] Updated weights for policy 0, policy_version 29122 (0.0009) [2023-10-08 05:04:40,543][00612] Updated weights for policy 1, policy_version 29290 (0.0008) [2023-10-08 05:04:40,762][00611] Updated weights for policy 0, policy_version 29132 (0.0009) [2023-10-08 05:04:40,919][00612] Updated weights for policy 1, policy_version 29300 (0.0010) [2023-10-08 05:04:41,131][00611] Updated weights for policy 0, policy_version 29142 (0.0009) [2023-10-08 05:04:41,286][00612] Updated weights for policy 1, policy_version 29310 (0.0007) [2023-10-08 05:04:41,497][00611] Updated weights for policy 0, policy_version 29152 (0.0008) [2023-10-08 05:04:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 59867136. Throughput: 0: 1816.5, 1: 1849.9. Samples: 14973772. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 05:04:43,755][130385] Avg episode reward: [(0, '46.260'), (1, '49.620')] [2023-10-08 05:04:44,999][00612] Updated weights for policy 1, policy_version 29320 (0.0009) [2023-10-08 05:04:45,015][00611] Updated weights for policy 0, policy_version 29162 (0.0007) [2023-10-08 05:04:45,370][00612] Updated weights for policy 1, policy_version 29330 (0.0009) [2023-10-08 05:04:45,390][00611] Updated weights for policy 0, policy_version 29172 (0.0007) [2023-10-08 05:04:45,751][00612] Updated weights for policy 1, policy_version 29340 (0.0007) [2023-10-08 05:04:45,757][00611] Updated weights for policy 0, policy_version 29182 (0.0010) [2023-10-08 05:04:48,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14773.4). Total num frames: 59932672. Throughput: 0: 1820.4, 1: 1854.5. Samples: 14996670. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 05:04:48,754][130385] Avg episode reward: [(0, '46.600'), (1, '49.700')] [2023-10-08 05:04:49,464][00611] Updated weights for policy 0, policy_version 29192 (0.0007) [2023-10-08 05:04:49,514][00612] Updated weights for policy 1, policy_version 29350 (0.0008) [2023-10-08 05:04:49,834][00611] Updated weights for policy 0, policy_version 29202 (0.0007) [2023-10-08 05:04:49,880][00612] Updated weights for policy 1, policy_version 29360 (0.0007) [2023-10-08 05:04:50,208][00611] Updated weights for policy 0, policy_version 29212 (0.0007) [2023-10-08 05:04:50,246][00612] Updated weights for policy 1, policy_version 29370 (0.0008) [2023-10-08 05:04:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 59998208. Throughput: 0: 1818.5, 1: 1847.5. Samples: 15006324. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 05:04:53,755][130385] Avg episode reward: [(0, '44.490'), (1, '47.550')] [2023-10-08 05:04:53,870][00611] Updated weights for policy 0, policy_version 29222 (0.0008) [2023-10-08 05:04:53,949][00612] Updated weights for policy 1, policy_version 29380 (0.0007) [2023-10-08 05:04:54,239][00611] Updated weights for policy 0, policy_version 29232 (0.0007) [2023-10-08 05:04:54,316][00612] Updated weights for policy 1, policy_version 29390 (0.0009) [2023-10-08 05:04:54,612][00611] Updated weights for policy 0, policy_version 29242 (0.0007) [2023-10-08 05:04:54,691][00612] Updated weights for policy 1, policy_version 29400 (0.0008) [2023-10-08 05:04:58,354][00611] Updated weights for policy 0, policy_version 29252 (0.0008) [2023-10-08 05:04:58,466][00612] Updated weights for policy 1, policy_version 29410 (0.0010) [2023-10-08 05:04:58,724][00611] Updated weights for policy 0, policy_version 29262 (0.0009) [2023-10-08 05:04:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 60063744. Throughput: 0: 1818.9, 1: 1845.1. Samples: 15029158. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 05:04:58,754][130385] Avg episode reward: [(0, '44.900'), (1, '48.450')] [2023-10-08 05:04:58,823][00612] Updated weights for policy 1, policy_version 29420 (0.0008) [2023-10-08 05:04:59,091][00611] Updated weights for policy 0, policy_version 29272 (0.0009) [2023-10-08 05:04:59,194][00612] Updated weights for policy 1, policy_version 29430 (0.0008) [2023-10-08 05:04:59,560][00612] Updated weights for policy 1, policy_version 29440 (0.0009) [2023-10-08 05:05:02,810][00611] Updated weights for policy 0, policy_version 29282 (0.0008) [2023-10-08 05:05:03,186][00612] Updated weights for policy 1, policy_version 29450 (0.0007) [2023-10-08 05:05:03,191][00611] Updated weights for policy 0, policy_version 29292 (0.0007) [2023-10-08 05:05:03,565][00612] Updated weights for policy 1, policy_version 29460 (0.0009) [2023-10-08 05:05:03,566][00611] Updated weights for policy 0, policy_version 29302 (0.0009) [2023-10-08 05:05:03,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 60129280. Throughput: 0: 1822.0, 1: 1825.9. Samples: 15051290. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-08 05:05:03,755][130385] Avg episode reward: [(0, '46.800'), (1, '49.630')] [2023-10-08 05:05:03,929][00612] Updated weights for policy 1, policy_version 29470 (0.0008) [2023-10-08 05:05:03,938][00611] Updated weights for policy 0, policy_version 29312 (0.0008) [2023-10-08 05:05:07,580][00611] Updated weights for policy 0, policy_version 29322 (0.0009) [2023-10-08 05:05:07,732][00612] Updated weights for policy 1, policy_version 29480 (0.0009) [2023-10-08 05:05:07,952][00611] Updated weights for policy 0, policy_version 29332 (0.0007) [2023-10-08 05:05:08,102][00612] Updated weights for policy 1, policy_version 29490 (0.0009) [2023-10-08 05:05:08,326][00611] Updated weights for policy 0, policy_version 29342 (0.0009) [2023-10-08 05:05:08,461][00612] Updated weights for policy 1, policy_version 29500 (0.0009) [2023-10-08 05:05:08,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 60260352. Throughput: 0: 1822.8, 1: 1837.2. Samples: 15062256. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-08 05:05:08,755][130385] Avg episode reward: [(0, '49.930'), (1, '49.530')] [2023-10-08 05:05:12,056][00611] Updated weights for policy 0, policy_version 29352 (0.0008) [2023-10-08 05:05:12,062][00612] Updated weights for policy 1, policy_version 29510 (0.0007) [2023-10-08 05:05:12,420][00611] Updated weights for policy 0, policy_version 29362 (0.0008) [2023-10-08 05:05:12,427][00612] Updated weights for policy 1, policy_version 29520 (0.0007) [2023-10-08 05:05:12,787][00611] Updated weights for policy 0, policy_version 29372 (0.0008) [2023-10-08 05:05:12,792][00612] Updated weights for policy 1, policy_version 29530 (0.0007) [2023-10-08 05:05:13,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 60325888. Throughput: 0: 1817.0, 1: 1831.2. Samples: 15084260. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-08 05:05:13,755][130385] Avg episode reward: [(0, '51.470'), (1, '47.210')] [2023-10-08 05:05:16,397][00612] Updated weights for policy 1, policy_version 29540 (0.0008) [2023-10-08 05:05:16,470][00611] Updated weights for policy 0, policy_version 29382 (0.0009) [2023-10-08 05:05:16,765][00612] Updated weights for policy 1, policy_version 29550 (0.0007) [2023-10-08 05:05:16,837][00611] Updated weights for policy 0, policy_version 29392 (0.0007) [2023-10-08 05:05:17,128][00612] Updated weights for policy 1, policy_version 29560 (0.0008) [2023-10-08 05:05:17,205][00611] Updated weights for policy 0, policy_version 29402 (0.0009) [2023-10-08 05:05:18,754][130385] Fps is (10 sec: 13107.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 60391424. Throughput: 0: 1821.2, 1: 1841.4. Samples: 15105070. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-08 05:05:18,755][130385] Avg episode reward: [(0, '48.420'), (1, '47.450')] [2023-10-08 05:05:20,791][00611] Updated weights for policy 0, policy_version 29412 (0.0008) [2023-10-08 05:05:20,810][00612] Updated weights for policy 1, policy_version 29570 (0.0008) [2023-10-08 05:05:21,167][00611] Updated weights for policy 0, policy_version 29422 (0.0008) [2023-10-08 05:05:21,168][00612] Updated weights for policy 1, policy_version 29580 (0.0007) [2023-10-08 05:05:21,530][00611] Updated weights for policy 0, policy_version 29432 (0.0009) [2023-10-08 05:05:21,535][00612] Updated weights for policy 1, policy_version 29590 (0.0008) [2023-10-08 05:05:21,908][00612] Updated weights for policy 1, policy_version 29600 (0.0009) [2023-10-08 05:05:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 60456960. Throughput: 0: 1817.9, 1: 1826.0. Samples: 15116848. Policy #0 lag: (min: 19.0, avg: 26.5, max: 51.0) [2023-10-08 05:05:23,754][130385] Avg episode reward: [(0, '51.780'), (1, '46.190')] [2023-10-08 05:05:25,312][00611] Updated weights for policy 0, policy_version 29442 (0.0008) [2023-10-08 05:05:25,556][00612] Updated weights for policy 1, policy_version 29610 (0.0008) [2023-10-08 05:05:25,689][00611] Updated weights for policy 0, policy_version 29452 (0.0007) [2023-10-08 05:05:25,922][00612] Updated weights for policy 1, policy_version 29620 (0.0009) [2023-10-08 05:05:26,063][00611] Updated weights for policy 0, policy_version 29462 (0.0007) [2023-10-08 05:05:26,291][00612] Updated weights for policy 1, policy_version 29630 (0.0010) [2023-10-08 05:05:26,436][00611] Updated weights for policy 0, policy_version 29472 (0.0007) [2023-10-08 05:05:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 60522496. Throughput: 0: 1822.1, 1: 1826.4. Samples: 15137952. Policy #0 lag: (min: 19.0, avg: 26.5, max: 51.0) [2023-10-08 05:05:28,754][130385] Avg episode reward: [(0, '48.320'), (1, '45.790')] [2023-10-08 05:05:29,964][00612] Updated weights for policy 1, policy_version 29640 (0.0008) [2023-10-08 05:05:30,022][00611] Updated weights for policy 0, policy_version 29482 (0.0008) [2023-10-08 05:05:30,335][00612] Updated weights for policy 1, policy_version 29650 (0.0008) [2023-10-08 05:05:30,383][00611] Updated weights for policy 0, policy_version 29492 (0.0009) [2023-10-08 05:05:30,700][00612] Updated weights for policy 1, policy_version 29660 (0.0008) [2023-10-08 05:05:30,755][00611] Updated weights for policy 0, policy_version 29502 (0.0009) [2023-10-08 05:05:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 60588032. Throughput: 0: 1823.1, 1: 1830.0. Samples: 15161056. Policy #0 lag: (min: 19.0, avg: 26.5, max: 51.0) [2023-10-08 05:05:33,754][130385] Avg episode reward: [(0, '49.760'), (1, '46.610')] [2023-10-08 05:05:34,272][00612] Updated weights for policy 1, policy_version 29670 (0.0009) [2023-10-08 05:05:34,501][00611] Updated weights for policy 0, policy_version 29512 (0.0009) [2023-10-08 05:05:34,632][00612] Updated weights for policy 1, policy_version 29680 (0.0008) [2023-10-08 05:05:34,872][00611] Updated weights for policy 0, policy_version 29522 (0.0008) [2023-10-08 05:05:34,993][00612] Updated weights for policy 1, policy_version 29690 (0.0009) [2023-10-08 05:05:35,242][00611] Updated weights for policy 0, policy_version 29532 (0.0007) [2023-10-08 05:05:38,653][00612] Updated weights for policy 1, policy_version 29700 (0.0008) [2023-10-08 05:05:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 60653568. Throughput: 0: 1821.2, 1: 1833.8. Samples: 15170800. Policy #0 lag: (min: 19.0, avg: 26.5, max: 51.0) [2023-10-08 05:05:38,754][130385] Avg episode reward: [(0, '50.630'), (1, '43.740')] [2023-10-08 05:05:38,869][00611] Updated weights for policy 0, policy_version 29542 (0.0008) [2023-10-08 05:05:39,020][00612] Updated weights for policy 1, policy_version 29710 (0.0009) [2023-10-08 05:05:39,237][00611] Updated weights for policy 0, policy_version 29552 (0.0008) [2023-10-08 05:05:39,381][00612] Updated weights for policy 1, policy_version 29720 (0.0007) [2023-10-08 05:05:39,601][00611] Updated weights for policy 0, policy_version 29562 (0.0008) [2023-10-08 05:05:43,034][00612] Updated weights for policy 1, policy_version 29730 (0.0007) [2023-10-08 05:05:43,243][00611] Updated weights for policy 0, policy_version 29572 (0.0009) [2023-10-08 05:05:43,410][00612] Updated weights for policy 1, policy_version 29740 (0.0008) [2023-10-08 05:05:43,626][00611] Updated weights for policy 0, policy_version 29582 (0.0007) [2023-10-08 05:05:43,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 60719104. Throughput: 0: 1825.6, 1: 1838.1. Samples: 15194028. Policy #0 lag: (min: 19.0, avg: 26.5, max: 51.0) [2023-10-08 05:05:43,754][130385] Avg episode reward: [(0, '53.620'), (1, '43.090')] [2023-10-08 05:05:43,773][00612] Updated weights for policy 1, policy_version 29750 (0.0007) [2023-10-08 05:05:43,989][00611] Updated weights for policy 0, policy_version 29592 (0.0008) [2023-10-08 05:05:44,143][00612] Updated weights for policy 1, policy_version 29760 (0.0008) [2023-10-08 05:05:47,531][00611] Updated weights for policy 0, policy_version 29602 (0.0007) [2023-10-08 05:05:47,721][00612] Updated weights for policy 1, policy_version 29770 (0.0009) [2023-10-08 05:05:47,899][00611] Updated weights for policy 0, policy_version 29612 (0.0008) [2023-10-08 05:05:48,082][00612] Updated weights for policy 1, policy_version 29780 (0.0007) [2023-10-08 05:05:48,260][00611] Updated weights for policy 0, policy_version 29622 (0.0008) [2023-10-08 05:05:48,457][00612] Updated weights for policy 1, policy_version 29790 (0.0007) [2023-10-08 05:05:48,637][00611] Updated weights for policy 0, policy_version 29632 (0.0008) [2023-10-08 05:05:48,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 60850176. Throughput: 0: 1820.3, 1: 1823.4. Samples: 15215256. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 05:05:48,754][130385] Avg episode reward: [(0, '57.280'), (1, '43.800')] [2023-10-08 05:05:48,765][00365] Saving new best policy, reward=57.280! [2023-10-08 05:05:52,232][00612] Updated weights for policy 1, policy_version 29800 (0.0007) [2023-10-08 05:05:52,373][00611] Updated weights for policy 0, policy_version 29642 (0.0008) [2023-10-08 05:05:52,599][00612] Updated weights for policy 1, policy_version 29810 (0.0007) [2023-10-08 05:05:52,740][00611] Updated weights for policy 0, policy_version 29652 (0.0009) [2023-10-08 05:05:52,972][00612] Updated weights for policy 1, policy_version 29820 (0.0008) [2023-10-08 05:05:53,110][00611] Updated weights for policy 0, policy_version 29662 (0.0007) [2023-10-08 05:05:53,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 60915712. Throughput: 0: 1827.3, 1: 1833.4. Samples: 15226988. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 05:05:53,755][130385] Avg episode reward: [(0, '58.170'), (1, '43.370')] [2023-10-08 05:05:53,756][00365] Saving new best policy, reward=58.170! [2023-10-08 05:05:56,777][00612] Updated weights for policy 1, policy_version 29830 (0.0008) [2023-10-08 05:05:56,826][00611] Updated weights for policy 0, policy_version 29672 (0.0007) [2023-10-08 05:05:57,148][00612] Updated weights for policy 1, policy_version 29840 (0.0008) [2023-10-08 05:05:57,191][00611] Updated weights for policy 0, policy_version 29682 (0.0007) [2023-10-08 05:05:57,515][00612] Updated weights for policy 1, policy_version 29850 (0.0007) [2023-10-08 05:05:57,554][00611] Updated weights for policy 0, policy_version 29692 (0.0008) [2023-10-08 05:05:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 60981248. Throughput: 0: 1828.6, 1: 1820.7. Samples: 15248480. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 05:05:58,755][130385] Avg episode reward: [(0, '56.400'), (1, '46.830')] [2023-10-08 05:06:01,165][00612] Updated weights for policy 1, policy_version 29860 (0.0008) [2023-10-08 05:06:01,345][00611] Updated weights for policy 0, policy_version 29702 (0.0009) [2023-10-08 05:06:01,536][00612] Updated weights for policy 1, policy_version 29870 (0.0008) [2023-10-08 05:06:01,708][00611] Updated weights for policy 0, policy_version 29712 (0.0010) [2023-10-08 05:06:01,899][00612] Updated weights for policy 1, policy_version 29880 (0.0007) [2023-10-08 05:06:02,077][00611] Updated weights for policy 0, policy_version 29722 (0.0009) [2023-10-08 05:06:03,754][130385] Fps is (10 sec: 13107.4, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 61046784. Throughput: 0: 1830.8, 1: 1833.5. Samples: 15269962. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 05:06:03,754][130385] Avg episode reward: [(0, '55.590'), (1, '47.420')] [2023-10-08 05:06:05,675][00611] Updated weights for policy 0, policy_version 29732 (0.0009) [2023-10-08 05:06:05,704][00612] Updated weights for policy 1, policy_version 29890 (0.0008) [2023-10-08 05:06:06,041][00611] Updated weights for policy 0, policy_version 29742 (0.0009) [2023-10-08 05:06:06,071][00612] Updated weights for policy 1, policy_version 29900 (0.0008) [2023-10-08 05:06:06,417][00611] Updated weights for policy 0, policy_version 29752 (0.0007) [2023-10-08 05:06:06,439][00612] Updated weights for policy 1, policy_version 29910 (0.0008) [2023-10-08 05:06:06,805][00612] Updated weights for policy 1, policy_version 29920 (0.0009) [2023-10-08 05:06:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14773.4). Total num frames: 61112320. Throughput: 0: 1826.0, 1: 1830.7. Samples: 15281398. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:06:08,754][130385] Avg episode reward: [(0, '55.370'), (1, '48.680')] [2023-10-08 05:06:10,152][00611] Updated weights for policy 0, policy_version 29762 (0.0009) [2023-10-08 05:06:10,457][00612] Updated weights for policy 1, policy_version 29930 (0.0008) [2023-10-08 05:06:10,516][00611] Updated weights for policy 0, policy_version 29772 (0.0008) [2023-10-08 05:06:10,830][00612] Updated weights for policy 1, policy_version 29940 (0.0008) [2023-10-08 05:06:10,892][00611] Updated weights for policy 0, policy_version 29782 (0.0007) [2023-10-08 05:06:11,198][00612] Updated weights for policy 1, policy_version 29950 (0.0008) [2023-10-08 05:06:11,250][00611] Updated weights for policy 0, policy_version 29792 (0.0007) [2023-10-08 05:06:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 61177856. Throughput: 0: 1830.3, 1: 1827.8. Samples: 15302566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:06:13,754][130385] Avg episode reward: [(0, '57.860'), (1, '47.850')] [2023-10-08 05:06:14,939][00612] Updated weights for policy 1, policy_version 29960 (0.0008) [2023-10-08 05:06:14,999][00611] Updated weights for policy 0, policy_version 29802 (0.0007) [2023-10-08 05:06:15,313][00612] Updated weights for policy 1, policy_version 29970 (0.0009) [2023-10-08 05:06:15,380][00611] Updated weights for policy 0, policy_version 29812 (0.0007) [2023-10-08 05:06:15,684][00612] Updated weights for policy 1, policy_version 29980 (0.0007) [2023-10-08 05:06:15,741][00611] Updated weights for policy 0, policy_version 29822 (0.0008) [2023-10-08 05:06:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 61243392. Throughput: 0: 1825.7, 1: 1827.0. Samples: 15325426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:06:18,755][130385] Avg episode reward: [(0, '56.610'), (1, '46.570')] [2023-10-08 05:06:18,765][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000029984_30703616.pth... [2023-10-08 05:06:18,765][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000029824_30539776.pth... [2023-10-08 05:06:18,801][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000028128_28803072.pth [2023-10-08 05:06:18,806][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000028256_28934144.pth [2023-10-08 05:06:19,327][00612] Updated weights for policy 1, policy_version 29990 (0.0009) [2023-10-08 05:06:19,507][00611] Updated weights for policy 0, policy_version 29832 (0.0008) [2023-10-08 05:06:19,707][00612] Updated weights for policy 1, policy_version 30000 (0.0008) [2023-10-08 05:06:19,878][00611] Updated weights for policy 0, policy_version 29842 (0.0007) [2023-10-08 05:06:20,076][00612] Updated weights for policy 1, policy_version 30010 (0.0009) [2023-10-08 05:06:20,251][00611] Updated weights for policy 0, policy_version 29852 (0.0008) [2023-10-08 05:06:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 61308928. Throughput: 0: 1825.7, 1: 1823.7. Samples: 15335024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:06:23,754][130385] Avg episode reward: [(0, '56.790'), (1, '47.310')] [2023-10-08 05:06:23,836][00612] Updated weights for policy 1, policy_version 30020 (0.0008) [2023-10-08 05:06:23,918][00611] Updated weights for policy 0, policy_version 29862 (0.0009) [2023-10-08 05:06:24,204][00612] Updated weights for policy 1, policy_version 30030 (0.0009) [2023-10-08 05:06:24,287][00611] Updated weights for policy 0, policy_version 29872 (0.0007) [2023-10-08 05:06:24,577][00612] Updated weights for policy 1, policy_version 30040 (0.0008) [2023-10-08 05:06:24,657][00611] Updated weights for policy 0, policy_version 29882 (0.0007) [2023-10-08 05:06:28,139][00612] Updated weights for policy 1, policy_version 30050 (0.0007) [2023-10-08 05:06:28,383][00611] Updated weights for policy 0, policy_version 29892 (0.0009) [2023-10-08 05:06:28,510][00612] Updated weights for policy 1, policy_version 30060 (0.0007) [2023-10-08 05:06:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 61374464. Throughput: 0: 1819.6, 1: 1824.9. Samples: 15358030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:06:28,755][130385] Avg episode reward: [(0, '57.480'), (1, '47.480')] [2023-10-08 05:06:28,763][00611] Updated weights for policy 0, policy_version 29902 (0.0009) [2023-10-08 05:06:28,878][00612] Updated weights for policy 1, policy_version 30070 (0.0007) [2023-10-08 05:06:29,133][00611] Updated weights for policy 0, policy_version 29912 (0.0008) [2023-10-08 05:06:29,246][00612] Updated weights for policy 1, policy_version 30080 (0.0007) [2023-10-08 05:06:32,724][00611] Updated weights for policy 0, policy_version 29922 (0.0008) [2023-10-08 05:06:32,828][00612] Updated weights for policy 1, policy_version 30090 (0.0010) [2023-10-08 05:06:33,105][00611] Updated weights for policy 0, policy_version 29932 (0.0009) [2023-10-08 05:06:33,194][00612] Updated weights for policy 1, policy_version 30100 (0.0008) [2023-10-08 05:06:33,472][00611] Updated weights for policy 0, policy_version 29942 (0.0009) [2023-10-08 05:06:33,554][00612] Updated weights for policy 1, policy_version 30110 (0.0008) [2023-10-08 05:06:33,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 61472768. Throughput: 0: 1823.3, 1: 1830.3. Samples: 15379668. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-08 05:06:33,754][130385] Avg episode reward: [(0, '53.390'), (1, '47.790')] [2023-10-08 05:06:33,840][00611] Updated weights for policy 0, policy_version 29952 (0.0010) [2023-10-08 05:06:37,188][00612] Updated weights for policy 1, policy_version 30120 (0.0008) [2023-10-08 05:06:37,474][00611] Updated weights for policy 0, policy_version 29962 (0.0007) [2023-10-08 05:06:37,553][00612] Updated weights for policy 1, policy_version 30130 (0.0007) [2023-10-08 05:06:37,842][00611] Updated weights for policy 0, policy_version 29972 (0.0009) [2023-10-08 05:06:37,923][00612] Updated weights for policy 1, policy_version 30140 (0.0007) [2023-10-08 05:06:38,217][00611] Updated weights for policy 0, policy_version 29982 (0.0007) [2023-10-08 05:06:38,754][130385] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 61571072. Throughput: 0: 1817.1, 1: 1828.3. Samples: 15391030. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-08 05:06:38,754][130385] Avg episode reward: [(0, '49.470'), (1, '48.090')] [2023-10-08 05:06:41,638][00612] Updated weights for policy 1, policy_version 30150 (0.0009) [2023-10-08 05:06:41,970][00611] Updated weights for policy 0, policy_version 29992 (0.0009) [2023-10-08 05:06:42,010][00612] Updated weights for policy 1, policy_version 30160 (0.0007) [2023-10-08 05:06:42,335][00611] Updated weights for policy 0, policy_version 30002 (0.0007) [2023-10-08 05:06:42,368][00612] Updated weights for policy 1, policy_version 30170 (0.0007) [2023-10-08 05:06:42,707][00611] Updated weights for policy 0, policy_version 30012 (0.0009) [2023-10-08 05:06:43,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 61636608. Throughput: 0: 1821.7, 1: 1827.1. Samples: 15412676. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-08 05:06:43,754][130385] Avg episode reward: [(0, '49.450'), (1, '49.170')] [2023-10-08 05:06:46,061][00612] Updated weights for policy 1, policy_version 30180 (0.0008) [2023-10-08 05:06:46,399][00611] Updated weights for policy 0, policy_version 30022 (0.0009) [2023-10-08 05:06:46,429][00612] Updated weights for policy 1, policy_version 30190 (0.0010) [2023-10-08 05:06:46,768][00611] Updated weights for policy 0, policy_version 30032 (0.0008) [2023-10-08 05:06:46,807][00612] Updated weights for policy 1, policy_version 30200 (0.0008) [2023-10-08 05:06:47,136][00611] Updated weights for policy 0, policy_version 30042 (0.0008) [2023-10-08 05:06:48,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 61702144. Throughput: 0: 1816.3, 1: 1827.3. Samples: 15433924. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-08 05:06:48,755][130385] Avg episode reward: [(0, '50.690'), (1, '48.970')] [2023-10-08 05:06:50,380][00612] Updated weights for policy 1, policy_version 30210 (0.0008) [2023-10-08 05:06:50,757][00612] Updated weights for policy 1, policy_version 30220 (0.0007) [2023-10-08 05:06:50,912][00611] Updated weights for policy 0, policy_version 30052 (0.0010) [2023-10-08 05:06:51,124][00612] Updated weights for policy 1, policy_version 30230 (0.0007) [2023-10-08 05:06:51,280][00611] Updated weights for policy 0, policy_version 30062 (0.0008) [2023-10-08 05:06:51,497][00612] Updated weights for policy 1, policy_version 30240 (0.0008) [2023-10-08 05:06:51,656][00611] Updated weights for policy 0, policy_version 30072 (0.0009) [2023-10-08 05:06:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 61767680. Throughput: 0: 1826.1, 1: 1821.4. Samples: 15445536. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 05:06:53,754][130385] Avg episode reward: [(0, '50.830'), (1, '50.620')] [2023-10-08 05:06:55,178][00611] Updated weights for policy 0, policy_version 30082 (0.0008) [2023-10-08 05:06:55,268][00612] Updated weights for policy 1, policy_version 30250 (0.0010) [2023-10-08 05:06:55,543][00611] Updated weights for policy 0, policy_version 30092 (0.0009) [2023-10-08 05:06:55,645][00612] Updated weights for policy 1, policy_version 30260 (0.0008) [2023-10-08 05:06:55,910][00611] Updated weights for policy 0, policy_version 30102 (0.0009) [2023-10-08 05:06:56,020][00612] Updated weights for policy 1, policy_version 30270 (0.0008) [2023-10-08 05:06:56,292][00611] Updated weights for policy 0, policy_version 30112 (0.0011) [2023-10-08 05:06:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 61833216. Throughput: 0: 1818.8, 1: 1834.5. Samples: 15466964. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 05:06:58,755][130385] Avg episode reward: [(0, '47.940'), (1, '53.090')] [2023-10-08 05:06:59,556][00612] Updated weights for policy 1, policy_version 30280 (0.0008) [2023-10-08 05:06:59,927][00612] Updated weights for policy 1, policy_version 30290 (0.0007) [2023-10-08 05:07:00,008][00611] Updated weights for policy 0, policy_version 30122 (0.0007) [2023-10-08 05:07:00,298][00612] Updated weights for policy 1, policy_version 30300 (0.0009) [2023-10-08 05:07:00,388][00611] Updated weights for policy 0, policy_version 30132 (0.0008) [2023-10-08 05:07:00,755][00611] Updated weights for policy 0, policy_version 30142 (0.0009) [2023-10-08 05:07:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 61898752. Throughput: 0: 1819.2, 1: 1832.9. Samples: 15489766. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 05:07:03,754][130385] Avg episode reward: [(0, '50.220'), (1, '53.800')] [2023-10-08 05:07:04,049][00612] Updated weights for policy 1, policy_version 30310 (0.0009) [2023-10-08 05:07:04,430][00612] Updated weights for policy 1, policy_version 30320 (0.0008) [2023-10-08 05:07:04,475][00611] Updated weights for policy 0, policy_version 30152 (0.0008) [2023-10-08 05:07:04,807][00612] Updated weights for policy 1, policy_version 30330 (0.0007) [2023-10-08 05:07:04,855][00611] Updated weights for policy 0, policy_version 30162 (0.0009) [2023-10-08 05:07:05,216][00611] Updated weights for policy 0, policy_version 30172 (0.0010) [2023-10-08 05:07:08,547][00612] Updated weights for policy 1, policy_version 30340 (0.0009) [2023-10-08 05:07:08,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 61964288. Throughput: 0: 1820.6, 1: 1832.6. Samples: 15499416. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 05:07:08,754][130385] Avg episode reward: [(0, '49.750'), (1, '53.860')] [2023-10-08 05:07:08,843][00611] Updated weights for policy 0, policy_version 30182 (0.0010) [2023-10-08 05:07:08,922][00612] Updated weights for policy 1, policy_version 30350 (0.0007) [2023-10-08 05:07:09,223][00611] Updated weights for policy 0, policy_version 30192 (0.0010) [2023-10-08 05:07:09,284][00612] Updated weights for policy 1, policy_version 30360 (0.0007) [2023-10-08 05:07:09,591][00611] Updated weights for policy 0, policy_version 30202 (0.0008) [2023-10-08 05:07:12,947][00612] Updated weights for policy 1, policy_version 30370 (0.0008) [2023-10-08 05:07:13,312][00612] Updated weights for policy 1, policy_version 30380 (0.0007) [2023-10-08 05:07:13,376][00611] Updated weights for policy 0, policy_version 30212 (0.0008) [2023-10-08 05:07:13,671][00612] Updated weights for policy 1, policy_version 30390 (0.0007) [2023-10-08 05:07:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 62029824. Throughput: 0: 1819.2, 1: 1829.2. Samples: 15522210. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 05:07:13,754][00611] Updated weights for policy 0, policy_version 30222 (0.0008) [2023-10-08 05:07:13,755][130385] Avg episode reward: [(0, '47.600'), (1, '52.940')] [2023-10-08 05:07:14,050][00612] Updated weights for policy 1, policy_version 30400 (0.0007) [2023-10-08 05:07:14,126][00611] Updated weights for policy 0, policy_version 30232 (0.0007) [2023-10-08 05:07:17,707][00612] Updated weights for policy 1, policy_version 30410 (0.0010) [2023-10-08 05:07:17,911][00611] Updated weights for policy 0, policy_version 30242 (0.0008) [2023-10-08 05:07:18,068][00612] Updated weights for policy 1, policy_version 30420 (0.0008) [2023-10-08 05:07:18,268][00611] Updated weights for policy 0, policy_version 30252 (0.0008) [2023-10-08 05:07:18,433][00612] Updated weights for policy 1, policy_version 30430 (0.0007) [2023-10-08 05:07:18,650][00611] Updated weights for policy 0, policy_version 30262 (0.0009) [2023-10-08 05:07:18,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 62128128. Throughput: 0: 1817.8, 1: 1819.5. Samples: 15543346. Policy #0 lag: (min: 10.0, avg: 14.6, max: 42.0) [2023-10-08 05:07:18,755][130385] Avg episode reward: [(0, '47.110'), (1, '53.650')] [2023-10-08 05:07:19,017][00611] Updated weights for policy 0, policy_version 30272 (0.0009) [2023-10-08 05:07:22,250][00612] Updated weights for policy 1, policy_version 30440 (0.0007) [2023-10-08 05:07:22,612][00612] Updated weights for policy 1, policy_version 30450 (0.0008) [2023-10-08 05:07:22,758][00611] Updated weights for policy 0, policy_version 30282 (0.0008) [2023-10-08 05:07:22,976][00612] Updated weights for policy 1, policy_version 30460 (0.0007) [2023-10-08 05:07:23,116][00611] Updated weights for policy 0, policy_version 30292 (0.0007) [2023-10-08 05:07:23,498][00611] Updated weights for policy 0, policy_version 30302 (0.0009) [2023-10-08 05:07:23,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 62226432. Throughput: 0: 1810.9, 1: 1821.3. Samples: 15554482. Policy #0 lag: (min: 10.0, avg: 14.6, max: 42.0) [2023-10-08 05:07:23,754][130385] Avg episode reward: [(0, '49.880'), (1, '52.870')] [2023-10-08 05:07:26,630][00612] Updated weights for policy 1, policy_version 30470 (0.0007) [2023-10-08 05:07:26,993][00612] Updated weights for policy 1, policy_version 30480 (0.0007) [2023-10-08 05:07:27,143][00611] Updated weights for policy 0, policy_version 30312 (0.0007) [2023-10-08 05:07:27,363][00612] Updated weights for policy 1, policy_version 30490 (0.0007) [2023-10-08 05:07:27,515][00611] Updated weights for policy 0, policy_version 30322 (0.0008) [2023-10-08 05:07:27,885][00611] Updated weights for policy 0, policy_version 30332 (0.0007) [2023-10-08 05:07:28,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 62291968. Throughput: 0: 1813.2, 1: 1820.7. Samples: 15576204. Policy #0 lag: (min: 10.0, avg: 14.6, max: 42.0) [2023-10-08 05:07:28,755][130385] Avg episode reward: [(0, '50.120'), (1, '50.130')] [2023-10-08 05:07:31,027][00612] Updated weights for policy 1, policy_version 30500 (0.0008) [2023-10-08 05:07:31,396][00612] Updated weights for policy 1, policy_version 30510 (0.0008) [2023-10-08 05:07:31,506][00611] Updated weights for policy 0, policy_version 30342 (0.0009) [2023-10-08 05:07:31,766][00612] Updated weights for policy 1, policy_version 30520 (0.0009) [2023-10-08 05:07:31,881][00611] Updated weights for policy 0, policy_version 30352 (0.0007) [2023-10-08 05:07:32,249][00611] Updated weights for policy 0, policy_version 30362 (0.0010) [2023-10-08 05:07:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 62357504. Throughput: 0: 1813.4, 1: 1824.7. Samples: 15597640. Policy #0 lag: (min: 10.0, avg: 14.6, max: 42.0) [2023-10-08 05:07:33,754][130385] Avg episode reward: [(0, '52.310'), (1, '49.640')] [2023-10-08 05:07:35,284][00612] Updated weights for policy 1, policy_version 30530 (0.0009) [2023-10-08 05:07:35,647][00612] Updated weights for policy 1, policy_version 30540 (0.0010) [2023-10-08 05:07:35,953][00611] Updated weights for policy 0, policy_version 30372 (0.0008) [2023-10-08 05:07:36,015][00612] Updated weights for policy 1, policy_version 30550 (0.0007) [2023-10-08 05:07:36,323][00611] Updated weights for policy 0, policy_version 30382 (0.0010) [2023-10-08 05:07:36,384][00612] Updated weights for policy 1, policy_version 30560 (0.0008) [2023-10-08 05:07:36,699][00611] Updated weights for policy 0, policy_version 30392 (0.0008) [2023-10-08 05:07:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 62423040. Throughput: 0: 1814.5, 1: 1821.8. Samples: 15609170. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 05:07:38,755][130385] Avg episode reward: [(0, '53.040'), (1, '50.810')] [2023-10-08 05:07:40,130][00612] Updated weights for policy 1, policy_version 30570 (0.0007) [2023-10-08 05:07:40,375][00611] Updated weights for policy 0, policy_version 30402 (0.0007) [2023-10-08 05:07:40,498][00612] Updated weights for policy 1, policy_version 30580 (0.0007) [2023-10-08 05:07:40,751][00611] Updated weights for policy 0, policy_version 30412 (0.0007) [2023-10-08 05:07:40,861][00612] Updated weights for policy 1, policy_version 30590 (0.0009) [2023-10-08 05:07:41,115][00611] Updated weights for policy 0, policy_version 30422 (0.0010) [2023-10-08 05:07:41,481][00611] Updated weights for policy 0, policy_version 30432 (0.0011) [2023-10-08 05:07:43,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 62488576. Throughput: 0: 1813.1, 1: 1824.9. Samples: 15630670. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 05:07:43,755][130385] Avg episode reward: [(0, '52.010'), (1, '48.720')] [2023-10-08 05:07:44,431][00612] Updated weights for policy 1, policy_version 30600 (0.0008) [2023-10-08 05:07:44,793][00612] Updated weights for policy 1, policy_version 30610 (0.0009) [2023-10-08 05:07:45,096][00611] Updated weights for policy 0, policy_version 30442 (0.0007) [2023-10-08 05:07:45,161][00612] Updated weights for policy 1, policy_version 30620 (0.0008) [2023-10-08 05:07:45,475][00611] Updated weights for policy 0, policy_version 30452 (0.0010) [2023-10-08 05:07:45,845][00611] Updated weights for policy 0, policy_version 30462 (0.0011) [2023-10-08 05:07:48,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 62554112. Throughput: 0: 1812.0, 1: 1826.8. Samples: 15653510. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 05:07:48,754][130385] Avg episode reward: [(0, '55.160'), (1, '49.290')] [2023-10-08 05:07:48,945][00612] Updated weights for policy 1, policy_version 30630 (0.0009) [2023-10-08 05:07:49,317][00612] Updated weights for policy 1, policy_version 30640 (0.0010) [2023-10-08 05:07:49,685][00612] Updated weights for policy 1, policy_version 30650 (0.0008) [2023-10-08 05:07:49,766][00611] Updated weights for policy 0, policy_version 30472 (0.0008) [2023-10-08 05:07:50,145][00611] Updated weights for policy 0, policy_version 30482 (0.0009) [2023-10-08 05:07:50,522][00611] Updated weights for policy 0, policy_version 30492 (0.0009) [2023-10-08 05:07:53,327][00612] Updated weights for policy 1, policy_version 30660 (0.0007) [2023-10-08 05:07:53,707][00612] Updated weights for policy 1, policy_version 30670 (0.0008) [2023-10-08 05:07:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 62619648. Throughput: 0: 1811.6, 1: 1826.0. Samples: 15663104. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 05:07:53,755][130385] Avg episode reward: [(0, '57.240'), (1, '48.960')] [2023-10-08 05:07:54,069][00611] Updated weights for policy 0, policy_version 30502 (0.0009) [2023-10-08 05:07:54,070][00612] Updated weights for policy 1, policy_version 30680 (0.0007) [2023-10-08 05:07:54,447][00611] Updated weights for policy 0, policy_version 30512 (0.0007) [2023-10-08 05:07:54,823][00611] Updated weights for policy 0, policy_version 30522 (0.0007) [2023-10-08 05:07:57,634][00612] Updated weights for policy 1, policy_version 30690 (0.0009) [2023-10-08 05:07:58,006][00612] Updated weights for policy 1, policy_version 30700 (0.0008) [2023-10-08 05:07:58,376][00612] Updated weights for policy 1, policy_version 30710 (0.0009) [2023-10-08 05:07:58,475][00611] Updated weights for policy 0, policy_version 30532 (0.0007) [2023-10-08 05:07:58,739][00612] Updated weights for policy 1, policy_version 30720 (0.0008) [2023-10-08 05:07:58,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 62717952. Throughput: 0: 1816.7, 1: 1827.9. Samples: 15686216. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 05:07:58,754][130385] Avg episode reward: [(0, '58.230'), (1, '51.670')] [2023-10-08 05:07:58,848][00611] Updated weights for policy 0, policy_version 30542 (0.0008) [2023-10-08 05:07:59,208][00611] Updated weights for policy 0, policy_version 30552 (0.0010) [2023-10-08 05:07:59,505][00365] Saving new best policy, reward=58.230! [2023-10-08 05:08:02,491][00612] Updated weights for policy 1, policy_version 30730 (0.0009) [2023-10-08 05:08:02,752][00611] Updated weights for policy 0, policy_version 30562 (0.0009) [2023-10-08 05:08:02,856][00612] Updated weights for policy 1, policy_version 30740 (0.0008) [2023-10-08 05:08:03,117][00611] Updated weights for policy 0, policy_version 30572 (0.0009) [2023-10-08 05:08:03,222][00612] Updated weights for policy 1, policy_version 30750 (0.0009) [2023-10-08 05:08:03,500][00611] Updated weights for policy 0, policy_version 30582 (0.0009) [2023-10-08 05:08:03,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 62783488. Throughput: 0: 1823.3, 1: 1825.7. Samples: 15707550. Policy #0 lag: (min: 28.0, avg: 28.9, max: 49.0) [2023-10-08 05:08:03,754][130385] Avg episode reward: [(0, '57.730'), (1, '55.030')] [2023-10-08 05:08:03,867][00611] Updated weights for policy 0, policy_version 30592 (0.0010) [2023-10-08 05:08:06,865][00612] Updated weights for policy 1, policy_version 30760 (0.0007) [2023-10-08 05:08:07,233][00612] Updated weights for policy 1, policy_version 30770 (0.0009) [2023-10-08 05:08:07,564][00611] Updated weights for policy 0, policy_version 30602 (0.0008) [2023-10-08 05:08:07,610][00612] Updated weights for policy 1, policy_version 30780 (0.0009) [2023-10-08 05:08:07,937][00611] Updated weights for policy 0, policy_version 30612 (0.0008) [2023-10-08 05:08:08,302][00611] Updated weights for policy 0, policy_version 30622 (0.0010) [2023-10-08 05:08:08,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 62881792. Throughput: 0: 1826.9, 1: 1835.6. Samples: 15719296. Policy #0 lag: (min: 28.0, avg: 28.9, max: 49.0) [2023-10-08 05:08:08,754][130385] Avg episode reward: [(0, '56.700'), (1, '53.020')] [2023-10-08 05:08:11,398][00612] Updated weights for policy 1, policy_version 30790 (0.0007) [2023-10-08 05:08:11,763][00612] Updated weights for policy 1, policy_version 30800 (0.0007) [2023-10-08 05:08:11,940][00611] Updated weights for policy 0, policy_version 30632 (0.0009) [2023-10-08 05:08:12,133][00612] Updated weights for policy 1, policy_version 30810 (0.0007) [2023-10-08 05:08:12,313][00611] Updated weights for policy 0, policy_version 30642 (0.0008) [2023-10-08 05:08:12,682][00611] Updated weights for policy 0, policy_version 30652 (0.0010) [2023-10-08 05:08:13,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 62947328. Throughput: 0: 1825.5, 1: 1828.8. Samples: 15740646. Policy #0 lag: (min: 28.0, avg: 28.9, max: 49.0) [2023-10-08 05:08:13,755][130385] Avg episode reward: [(0, '54.980'), (1, '53.240')] [2023-10-08 05:08:15,647][00612] Updated weights for policy 1, policy_version 30820 (0.0008) [2023-10-08 05:08:16,026][00612] Updated weights for policy 1, policy_version 30830 (0.0012) [2023-10-08 05:08:16,392][00612] Updated weights for policy 1, policy_version 30840 (0.0009) [2023-10-08 05:08:16,505][00611] Updated weights for policy 0, policy_version 30662 (0.0009) [2023-10-08 05:08:16,871][00611] Updated weights for policy 0, policy_version 30672 (0.0010) [2023-10-08 05:08:17,251][00611] Updated weights for policy 0, policy_version 30682 (0.0011) [2023-10-08 05:08:18,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 63012864. Throughput: 0: 1821.1, 1: 1829.6. Samples: 15761924. Policy #0 lag: (min: 28.0, avg: 28.9, max: 49.0) [2023-10-08 05:08:18,754][130385] Avg episode reward: [(0, '55.930'), (1, '50.510')] [2023-10-08 05:08:18,767][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000030848_31588352.pth... [2023-10-08 05:08:18,767][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000030688_31424512.pth... [2023-10-08 05:08:18,805][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000028992_29687808.pth [2023-10-08 05:08:18,807][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000029152_29851648.pth [2023-10-08 05:08:19,950][00612] Updated weights for policy 1, policy_version 30850 (0.0009) [2023-10-08 05:08:20,329][00612] Updated weights for policy 1, policy_version 30860 (0.0007) [2023-10-08 05:08:20,694][00612] Updated weights for policy 1, policy_version 30870 (0.0007) [2023-10-08 05:08:20,991][00611] Updated weights for policy 0, policy_version 30692 (0.0010) [2023-10-08 05:08:21,065][00612] Updated weights for policy 1, policy_version 30880 (0.0010) [2023-10-08 05:08:21,367][00611] Updated weights for policy 0, policy_version 30702 (0.0008) [2023-10-08 05:08:21,749][00611] Updated weights for policy 0, policy_version 30712 (0.0008) [2023-10-08 05:08:23,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 63078400. Throughput: 0: 1817.5, 1: 1825.6. Samples: 15773106. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-08 05:08:23,754][130385] Avg episode reward: [(0, '52.570'), (1, '48.370')] [2023-10-08 05:08:24,740][00612] Updated weights for policy 1, policy_version 30890 (0.0008) [2023-10-08 05:08:25,118][00612] Updated weights for policy 1, policy_version 30900 (0.0007) [2023-10-08 05:08:25,320][00611] Updated weights for policy 0, policy_version 30722 (0.0008) [2023-10-08 05:08:25,495][00612] Updated weights for policy 1, policy_version 30910 (0.0008) [2023-10-08 05:08:25,693][00611] Updated weights for policy 0, policy_version 30732 (0.0009) [2023-10-08 05:08:26,064][00611] Updated weights for policy 0, policy_version 30742 (0.0009) [2023-10-08 05:08:26,430][00611] Updated weights for policy 0, policy_version 30752 (0.0008) [2023-10-08 05:08:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 63143936. Throughput: 0: 1820.1, 1: 1829.5. Samples: 15794902. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-08 05:08:28,754][130385] Avg episode reward: [(0, '50.100'), (1, '47.460')] [2023-10-08 05:08:29,238][00612] Updated weights for policy 1, policy_version 30920 (0.0007) [2023-10-08 05:08:29,605][00612] Updated weights for policy 1, policy_version 30930 (0.0007) [2023-10-08 05:08:29,976][00612] Updated weights for policy 1, policy_version 30940 (0.0008) [2023-10-08 05:08:30,157][00611] Updated weights for policy 0, policy_version 30762 (0.0007) [2023-10-08 05:08:30,527][00611] Updated weights for policy 0, policy_version 30772 (0.0009) [2023-10-08 05:08:30,897][00611] Updated weights for policy 0, policy_version 30782 (0.0007) [2023-10-08 05:08:33,729][00612] Updated weights for policy 1, policy_version 30950 (0.0009) [2023-10-08 05:08:33,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 63209472. Throughput: 0: 1823.2, 1: 1831.1. Samples: 15817954. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-08 05:08:33,755][130385] Avg episode reward: [(0, '47.480'), (1, '45.070')] [2023-10-08 05:08:34,105][00612] Updated weights for policy 1, policy_version 30960 (0.0010) [2023-10-08 05:08:34,473][00612] Updated weights for policy 1, policy_version 30970 (0.0007) [2023-10-08 05:08:34,588][00611] Updated weights for policy 0, policy_version 30792 (0.0007) [2023-10-08 05:08:34,968][00611] Updated weights for policy 0, policy_version 30802 (0.0007) [2023-10-08 05:08:35,337][00611] Updated weights for policy 0, policy_version 30812 (0.0008) [2023-10-08 05:08:38,231][00612] Updated weights for policy 1, policy_version 30980 (0.0008) [2023-10-08 05:08:38,616][00612] Updated weights for policy 1, policy_version 30990 (0.0009) [2023-10-08 05:08:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 63275008. Throughput: 0: 1827.0, 1: 1836.2. Samples: 15827946. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-08 05:08:38,755][130385] Avg episode reward: [(0, '47.230'), (1, '43.730')] [2023-10-08 05:08:38,940][00611] Updated weights for policy 0, policy_version 30822 (0.0007) [2023-10-08 05:08:38,984][00612] Updated weights for policy 1, policy_version 31000 (0.0009) [2023-10-08 05:08:39,308][00611] Updated weights for policy 0, policy_version 30832 (0.0007) [2023-10-08 05:08:39,682][00611] Updated weights for policy 0, policy_version 30842 (0.0007) [2023-10-08 05:08:42,479][00612] Updated weights for policy 1, policy_version 31010 (0.0009) [2023-10-08 05:08:42,847][00612] Updated weights for policy 1, policy_version 31020 (0.0007) [2023-10-08 05:08:43,184][00611] Updated weights for policy 0, policy_version 30852 (0.0008) [2023-10-08 05:08:43,222][00612] Updated weights for policy 1, policy_version 31030 (0.0007) [2023-10-08 05:08:43,559][00611] Updated weights for policy 0, policy_version 30862 (0.0007) [2023-10-08 05:08:43,594][00612] Updated weights for policy 1, policy_version 31040 (0.0008) [2023-10-08 05:08:43,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63373312. Throughput: 0: 1825.7, 1: 1837.6. Samples: 15851064. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-08 05:08:43,755][130385] Avg episode reward: [(0, '45.680'), (1, '41.540')] [2023-10-08 05:08:43,925][00611] Updated weights for policy 0, policy_version 30872 (0.0009) [2023-10-08 05:08:47,257][00612] Updated weights for policy 1, policy_version 31050 (0.0008) [2023-10-08 05:08:47,557][00611] Updated weights for policy 0, policy_version 30882 (0.0009) [2023-10-08 05:08:47,625][00612] Updated weights for policy 1, policy_version 31060 (0.0010) [2023-10-08 05:08:47,931][00611] Updated weights for policy 0, policy_version 30892 (0.0007) [2023-10-08 05:08:47,993][00612] Updated weights for policy 1, policy_version 31070 (0.0008) [2023-10-08 05:08:48,299][00611] Updated weights for policy 0, policy_version 30902 (0.0007) [2023-10-08 05:08:48,667][00611] Updated weights for policy 0, policy_version 30912 (0.0007) [2023-10-08 05:08:48,754][130385] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 63471616. Throughput: 0: 1818.1, 1: 1827.5. Samples: 15871600. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 05:08:48,754][130385] Avg episode reward: [(0, '43.260'), (1, '43.660')] [2023-10-08 05:08:51,510][00612] Updated weights for policy 1, policy_version 31080 (0.0008) [2023-10-08 05:08:51,880][00612] Updated weights for policy 1, policy_version 31090 (0.0010) [2023-10-08 05:08:52,247][00612] Updated weights for policy 1, policy_version 31100 (0.0008) [2023-10-08 05:08:52,294][00611] Updated weights for policy 0, policy_version 30922 (0.0010) [2023-10-08 05:08:52,661][00611] Updated weights for policy 0, policy_version 30932 (0.0008) [2023-10-08 05:08:53,038][00611] Updated weights for policy 0, policy_version 30942 (0.0009) [2023-10-08 05:08:53,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 63537152. Throughput: 0: 1824.6, 1: 1834.8. Samples: 15883972. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 05:08:53,754][130385] Avg episode reward: [(0, '43.480'), (1, '47.570')] [2023-10-08 05:08:55,921][00612] Updated weights for policy 1, policy_version 31110 (0.0010) [2023-10-08 05:08:56,289][00612] Updated weights for policy 1, policy_version 31120 (0.0007) [2023-10-08 05:08:56,657][00612] Updated weights for policy 1, policy_version 31130 (0.0009) [2023-10-08 05:08:56,837][00611] Updated weights for policy 0, policy_version 30952 (0.0008) [2023-10-08 05:08:57,209][00611] Updated weights for policy 0, policy_version 30962 (0.0008) [2023-10-08 05:08:57,578][00611] Updated weights for policy 0, policy_version 30972 (0.0009) [2023-10-08 05:08:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 63602688. Throughput: 0: 1814.1, 1: 1826.8. Samples: 15904486. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 05:08:58,755][130385] Avg episode reward: [(0, '44.290'), (1, '47.290')] [2023-10-08 05:09:00,111][00612] Updated weights for policy 1, policy_version 31140 (0.0009) [2023-10-08 05:09:00,482][00612] Updated weights for policy 1, policy_version 31150 (0.0008) [2023-10-08 05:09:00,849][00612] Updated weights for policy 1, policy_version 31160 (0.0008) [2023-10-08 05:09:01,307][00611] Updated weights for policy 0, policy_version 30982 (0.0007) [2023-10-08 05:09:01,676][00611] Updated weights for policy 0, policy_version 30992 (0.0010) [2023-10-08 05:09:02,051][00611] Updated weights for policy 0, policy_version 31002 (0.0008) [2023-10-08 05:09:03,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 63668224. Throughput: 0: 1828.6, 1: 1847.8. Samples: 15927360. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 05:09:03,755][130385] Avg episode reward: [(0, '46.060'), (1, '48.840')] [2023-10-08 05:09:04,428][00612] Updated weights for policy 1, policy_version 31170 (0.0007) [2023-10-08 05:09:04,804][00612] Updated weights for policy 1, policy_version 31180 (0.0011) [2023-10-08 05:09:05,167][00612] Updated weights for policy 1, policy_version 31190 (0.0011) [2023-10-08 05:09:05,532][00612] Updated weights for policy 1, policy_version 31200 (0.0007) [2023-10-08 05:09:05,725][00611] Updated weights for policy 0, policy_version 31012 (0.0008) [2023-10-08 05:09:06,095][00611] Updated weights for policy 0, policy_version 31022 (0.0007) [2023-10-08 05:09:06,472][00611] Updated weights for policy 0, policy_version 31032 (0.0007) [2023-10-08 05:09:08,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 63733760. Throughput: 0: 1827.4, 1: 1846.8. Samples: 15938446. Policy #0 lag: (min: 30.0, avg: 37.5, max: 62.0) [2023-10-08 05:09:08,755][130385] Avg episode reward: [(0, '44.870'), (1, '50.810')] [2023-10-08 05:09:09,115][00612] Updated weights for policy 1, policy_version 31210 (0.0007) [2023-10-08 05:09:09,480][00612] Updated weights for policy 1, policy_version 31220 (0.0009) [2023-10-08 05:09:09,843][00612] Updated weights for policy 1, policy_version 31230 (0.0007) [2023-10-08 05:09:09,968][00611] Updated weights for policy 0, policy_version 31042 (0.0008) [2023-10-08 05:09:10,338][00611] Updated weights for policy 0, policy_version 31052 (0.0008) [2023-10-08 05:09:10,705][00611] Updated weights for policy 0, policy_version 31062 (0.0008) [2023-10-08 05:09:11,075][00611] Updated weights for policy 0, policy_version 31072 (0.0007) [2023-10-08 05:09:13,627][00612] Updated weights for policy 1, policy_version 31240 (0.0007) [2023-10-08 05:09:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 63799296. Throughput: 0: 1835.7, 1: 1852.9. Samples: 15960888. Policy #0 lag: (min: 30.0, avg: 37.5, max: 62.0) [2023-10-08 05:09:13,755][130385] Avg episode reward: [(0, '46.450'), (1, '51.980')] [2023-10-08 05:09:14,006][00612] Updated weights for policy 1, policy_version 31250 (0.0008) [2023-10-08 05:09:14,372][00612] Updated weights for policy 1, policy_version 31260 (0.0009) [2023-10-08 05:09:14,605][00611] Updated weights for policy 0, policy_version 31082 (0.0009) [2023-10-08 05:09:14,980][00611] Updated weights for policy 0, policy_version 31092 (0.0010) [2023-10-08 05:09:15,359][00611] Updated weights for policy 0, policy_version 31102 (0.0009) [2023-10-08 05:09:18,164][00612] Updated weights for policy 1, policy_version 31270 (0.0008) [2023-10-08 05:09:18,525][00612] Updated weights for policy 1, policy_version 31280 (0.0008) [2023-10-08 05:09:18,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 63864832. Throughput: 0: 1840.0, 1: 1841.8. Samples: 15983636. Policy #0 lag: (min: 30.0, avg: 37.5, max: 62.0) [2023-10-08 05:09:18,754][130385] Avg episode reward: [(0, '45.390'), (1, '51.200')] [2023-10-08 05:09:18,891][00612] Updated weights for policy 1, policy_version 31290 (0.0008) [2023-10-08 05:09:18,925][00611] Updated weights for policy 0, policy_version 31112 (0.0008) [2023-10-08 05:09:19,304][00611] Updated weights for policy 0, policy_version 31122 (0.0009) [2023-10-08 05:09:19,675][00611] Updated weights for policy 0, policy_version 31132 (0.0010) [2023-10-08 05:09:22,532][00612] Updated weights for policy 1, policy_version 31300 (0.0009) [2023-10-08 05:09:22,903][00612] Updated weights for policy 1, policy_version 31310 (0.0009) [2023-10-08 05:09:23,276][00612] Updated weights for policy 1, policy_version 31320 (0.0007) [2023-10-08 05:09:23,293][00611] Updated weights for policy 0, policy_version 31142 (0.0010) [2023-10-08 05:09:23,668][00611] Updated weights for policy 0, policy_version 31152 (0.0009) [2023-10-08 05:09:23,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63963136. Throughput: 0: 1840.3, 1: 1847.0. Samples: 15993874. Policy #0 lag: (min: 30.0, avg: 37.5, max: 62.0) [2023-10-08 05:09:23,754][130385] Avg episode reward: [(0, '48.180'), (1, '51.090')] [2023-10-08 05:09:24,040][00611] Updated weights for policy 0, policy_version 31162 (0.0008) [2023-10-08 05:09:26,865][00612] Updated weights for policy 1, policy_version 31330 (0.0007) [2023-10-08 05:09:27,246][00612] Updated weights for policy 1, policy_version 31340 (0.0008) [2023-10-08 05:09:27,616][00612] Updated weights for policy 1, policy_version 31350 (0.0008) [2023-10-08 05:09:27,685][00611] Updated weights for policy 0, policy_version 31172 (0.0007) [2023-10-08 05:09:27,969][00612] Updated weights for policy 1, policy_version 31360 (0.0008) [2023-10-08 05:09:28,060][00611] Updated weights for policy 0, policy_version 31182 (0.0008) [2023-10-08 05:09:28,437][00611] Updated weights for policy 0, policy_version 31192 (0.0011) [2023-10-08 05:09:28,754][130385] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 64061440. Throughput: 0: 1834.7, 1: 1838.4. Samples: 16016352. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 05:09:28,755][130385] Avg episode reward: [(0, '47.970'), (1, '50.850')] [2023-10-08 05:09:31,710][00612] Updated weights for policy 1, policy_version 31370 (0.0007) [2023-10-08 05:09:32,076][00612] Updated weights for policy 1, policy_version 31380 (0.0011) [2023-10-08 05:09:32,229][00611] Updated weights for policy 0, policy_version 31202 (0.0008) [2023-10-08 05:09:32,457][00612] Updated weights for policy 1, policy_version 31390 (0.0010) [2023-10-08 05:09:32,612][00611] Updated weights for policy 0, policy_version 31212 (0.0008) [2023-10-08 05:09:32,983][00611] Updated weights for policy 0, policy_version 31222 (0.0008) [2023-10-08 05:09:33,354][00611] Updated weights for policy 0, policy_version 31232 (0.0008) [2023-10-08 05:09:33,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 64126976. Throughput: 0: 1821.8, 1: 1854.4. Samples: 16037030. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 05:09:33,754][130385] Avg episode reward: [(0, '51.390'), (1, '50.350')] [2023-10-08 05:09:36,139][00612] Updated weights for policy 1, policy_version 31400 (0.0008) [2023-10-08 05:09:36,517][00612] Updated weights for policy 1, policy_version 31410 (0.0008) [2023-10-08 05:09:36,879][00612] Updated weights for policy 1, policy_version 31420 (0.0010) [2023-10-08 05:09:36,947][00611] Updated weights for policy 0, policy_version 31242 (0.0007) [2023-10-08 05:09:37,318][00611] Updated weights for policy 0, policy_version 31252 (0.0008) [2023-10-08 05:09:37,685][00611] Updated weights for policy 0, policy_version 31262 (0.0010) [2023-10-08 05:09:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 64192512. Throughput: 0: 1833.2, 1: 1837.9. Samples: 16049174. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 05:09:38,755][130385] Avg episode reward: [(0, '53.270'), (1, '50.070')] [2023-10-08 05:09:40,485][00612] Updated weights for policy 1, policy_version 31430 (0.0008) [2023-10-08 05:09:40,853][00612] Updated weights for policy 1, policy_version 31440 (0.0009) [2023-10-08 05:09:41,223][00612] Updated weights for policy 1, policy_version 31450 (0.0010) [2023-10-08 05:09:41,419][00611] Updated weights for policy 0, policy_version 31272 (0.0009) [2023-10-08 05:09:41,789][00611] Updated weights for policy 0, policy_version 31282 (0.0010) [2023-10-08 05:09:42,165][00611] Updated weights for policy 0, policy_version 31292 (0.0007) [2023-10-08 05:09:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 64258048. Throughput: 0: 1827.4, 1: 1852.8. Samples: 16070096. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 05:09:43,755][130385] Avg episode reward: [(0, '54.720'), (1, '52.540')] [2023-10-08 05:09:44,663][00612] Updated weights for policy 1, policy_version 31460 (0.0009) [2023-10-08 05:09:45,036][00612] Updated weights for policy 1, policy_version 31470 (0.0008) [2023-10-08 05:09:45,402][00612] Updated weights for policy 1, policy_version 31480 (0.0009) [2023-10-08 05:09:45,896][00611] Updated weights for policy 0, policy_version 31302 (0.0008) [2023-10-08 05:09:46,273][00611] Updated weights for policy 0, policy_version 31312 (0.0008) [2023-10-08 05:09:46,643][00611] Updated weights for policy 0, policy_version 31322 (0.0010) [2023-10-08 05:09:48,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 64323584. Throughput: 0: 1842.7, 1: 1843.9. Samples: 16093254. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 05:09:48,754][130385] Avg episode reward: [(0, '51.930'), (1, '52.840')] [2023-10-08 05:09:49,104][00612] Updated weights for policy 1, policy_version 31490 (0.0011) [2023-10-08 05:09:49,474][00612] Updated weights for policy 1, policy_version 31500 (0.0007) [2023-10-08 05:09:49,843][00612] Updated weights for policy 1, policy_version 31510 (0.0008) [2023-10-08 05:09:50,210][00612] Updated weights for policy 1, policy_version 31520 (0.0007) [2023-10-08 05:09:50,342][00611] Updated weights for policy 0, policy_version 31332 (0.0008) [2023-10-08 05:09:50,708][00611] Updated weights for policy 0, policy_version 31342 (0.0009) [2023-10-08 05:09:51,082][00611] Updated weights for policy 0, policy_version 31352 (0.0008) [2023-10-08 05:09:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 64389120. Throughput: 0: 1831.7, 1: 1839.2. Samples: 16103638. Policy #0 lag: (min: 5.0, avg: 11.5, max: 37.0) [2023-10-08 05:09:53,755][130385] Avg episode reward: [(0, '53.910'), (1, '51.300')] [2023-10-08 05:09:54,102][00612] Updated weights for policy 1, policy_version 31530 (0.0008) [2023-10-08 05:09:54,484][00612] Updated weights for policy 1, policy_version 31540 (0.0008) [2023-10-08 05:09:54,848][00612] Updated weights for policy 1, policy_version 31550 (0.0008) [2023-10-08 05:09:54,885][00611] Updated weights for policy 0, policy_version 31362 (0.0008) [2023-10-08 05:09:55,258][00611] Updated weights for policy 0, policy_version 31372 (0.0009) [2023-10-08 05:09:55,635][00611] Updated weights for policy 0, policy_version 31382 (0.0011) [2023-10-08 05:09:56,007][00611] Updated weights for policy 0, policy_version 31392 (0.0010) [2023-10-08 05:09:58,489][00612] Updated weights for policy 1, policy_version 31560 (0.0010) [2023-10-08 05:09:58,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 64454656. Throughput: 0: 1833.3, 1: 1832.9. Samples: 16125868. Policy #0 lag: (min: 5.0, avg: 11.5, max: 37.0) [2023-10-08 05:09:58,755][130385] Avg episode reward: [(0, '53.460'), (1, '50.680')] [2023-10-08 05:09:58,857][00612] Updated weights for policy 1, policy_version 31570 (0.0011) [2023-10-08 05:09:59,222][00612] Updated weights for policy 1, policy_version 31580 (0.0010) [2023-10-08 05:09:59,660][00611] Updated weights for policy 0, policy_version 31402 (0.0007) [2023-10-08 05:10:00,036][00611] Updated weights for policy 0, policy_version 31412 (0.0007) [2023-10-08 05:10:00,403][00611] Updated weights for policy 0, policy_version 31422 (0.0010) [2023-10-08 05:10:02,874][00612] Updated weights for policy 1, policy_version 31590 (0.0008) [2023-10-08 05:10:03,239][00612] Updated weights for policy 1, policy_version 31600 (0.0007) [2023-10-08 05:10:03,613][00612] Updated weights for policy 1, policy_version 31610 (0.0008) [2023-10-08 05:10:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 64520192. Throughput: 0: 1835.5, 1: 1824.0. Samples: 16148312. Policy #0 lag: (min: 5.0, avg: 11.5, max: 37.0) [2023-10-08 05:10:03,755][130385] Avg episode reward: [(0, '54.160'), (1, '49.520')] [2023-10-08 05:10:03,926][00611] Updated weights for policy 0, policy_version 31432 (0.0009) [2023-10-08 05:10:04,306][00611] Updated weights for policy 0, policy_version 31442 (0.0009) [2023-10-08 05:10:04,679][00611] Updated weights for policy 0, policy_version 31452 (0.0009) [2023-10-08 05:10:07,221][00612] Updated weights for policy 1, policy_version 31620 (0.0008) [2023-10-08 05:10:07,594][00612] Updated weights for policy 1, policy_version 31630 (0.0008) [2023-10-08 05:10:07,959][00612] Updated weights for policy 1, policy_version 31640 (0.0011) [2023-10-08 05:10:08,293][00611] Updated weights for policy 0, policy_version 31462 (0.0008) [2023-10-08 05:10:08,683][00611] Updated weights for policy 0, policy_version 31472 (0.0009) [2023-10-08 05:10:08,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 64618496. Throughput: 0: 1835.6, 1: 1833.0. Samples: 16158960. Policy #0 lag: (min: 5.0, avg: 11.5, max: 37.0) [2023-10-08 05:10:08,754][130385] Avg episode reward: [(0, '53.580'), (1, '48.980')] [2023-10-08 05:10:09,053][00611] Updated weights for policy 0, policy_version 31482 (0.0008) [2023-10-08 05:10:11,617][00612] Updated weights for policy 1, policy_version 31650 (0.0007) [2023-10-08 05:10:11,981][00612] Updated weights for policy 1, policy_version 31660 (0.0008) [2023-10-08 05:10:12,347][00612] Updated weights for policy 1, policy_version 31670 (0.0009) [2023-10-08 05:10:12,701][00611] Updated weights for policy 0, policy_version 31492 (0.0010) [2023-10-08 05:10:12,723][00612] Updated weights for policy 1, policy_version 31680 (0.0009) [2023-10-08 05:10:13,072][00611] Updated weights for policy 0, policy_version 31502 (0.0009) [2023-10-08 05:10:13,445][00611] Updated weights for policy 0, policy_version 31512 (0.0007) [2023-10-08 05:10:13,757][130385] Fps is (10 sec: 19654.1, 60 sec: 15290.9, 300 sec: 14662.1). Total num frames: 64716800. Throughput: 0: 1835.0, 1: 1825.3. Samples: 16181078. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) [2023-10-08 05:10:13,758][130385] Avg episode reward: [(0, '52.340'), (1, '49.270')] [2023-10-08 05:10:16,430][00612] Updated weights for policy 1, policy_version 31690 (0.0008) [2023-10-08 05:10:16,799][00612] Updated weights for policy 1, policy_version 31700 (0.0007) [2023-10-08 05:10:17,047][00611] Updated weights for policy 0, policy_version 31522 (0.0009) [2023-10-08 05:10:17,169][00612] Updated weights for policy 1, policy_version 31710 (0.0009) [2023-10-08 05:10:17,417][00611] Updated weights for policy 0, policy_version 31532 (0.0008) [2023-10-08 05:10:17,791][00611] Updated weights for policy 0, policy_version 31542 (0.0007) [2023-10-08 05:10:18,170][00611] Updated weights for policy 0, policy_version 31552 (0.0007) [2023-10-08 05:10:18,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 64782336. Throughput: 0: 1828.3, 1: 1831.0. Samples: 16201698. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) [2023-10-08 05:10:18,755][130385] Avg episode reward: [(0, '53.720'), (1, '49.760')] [2023-10-08 05:10:18,764][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000031712_32473088.pth... [2023-10-08 05:10:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000031552_32309248.pth... [2023-10-08 05:10:18,798][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000029824_30539776.pth [2023-10-08 05:10:18,800][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000029984_30703616.pth [2023-10-08 05:10:20,931][00612] Updated weights for policy 1, policy_version 31720 (0.0007) [2023-10-08 05:10:21,294][00612] Updated weights for policy 1, policy_version 31730 (0.0007) [2023-10-08 05:10:21,674][00612] Updated weights for policy 1, policy_version 31740 (0.0008) [2023-10-08 05:10:21,923][00611] Updated weights for policy 0, policy_version 31562 (0.0008) [2023-10-08 05:10:22,308][00611] Updated weights for policy 0, policy_version 31572 (0.0007) [2023-10-08 05:10:22,684][00611] Updated weights for policy 0, policy_version 31582 (0.0010) [2023-10-08 05:10:23,754][130385] Fps is (10 sec: 13111.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 64847872. Throughput: 0: 1835.7, 1: 1825.9. Samples: 16213946. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) [2023-10-08 05:10:23,754][130385] Avg episode reward: [(0, '52.190'), (1, '48.520')] [2023-10-08 05:10:25,263][00612] Updated weights for policy 1, policy_version 31750 (0.0008) [2023-10-08 05:10:25,622][00612] Updated weights for policy 1, policy_version 31760 (0.0008) [2023-10-08 05:10:25,994][00612] Updated weights for policy 1, policy_version 31770 (0.0008) [2023-10-08 05:10:26,335][00611] Updated weights for policy 0, policy_version 31592 (0.0008) [2023-10-08 05:10:26,712][00611] Updated weights for policy 0, policy_version 31602 (0.0008) [2023-10-08 05:10:27,078][00611] Updated weights for policy 0, policy_version 31612 (0.0010) [2023-10-08 05:10:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 64913408. Throughput: 0: 1832.5, 1: 1830.4. Samples: 16234926. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) [2023-10-08 05:10:28,754][130385] Avg episode reward: [(0, '56.340'), (1, '48.020')] [2023-10-08 05:10:29,659][00612] Updated weights for policy 1, policy_version 31780 (0.0009) [2023-10-08 05:10:30,025][00612] Updated weights for policy 1, policy_version 31790 (0.0008) [2023-10-08 05:10:30,393][00612] Updated weights for policy 1, policy_version 31800 (0.0009) [2023-10-08 05:10:30,717][00611] Updated weights for policy 0, policy_version 31622 (0.0007) [2023-10-08 05:10:31,092][00611] Updated weights for policy 0, policy_version 31632 (0.0011) [2023-10-08 05:10:31,466][00611] Updated weights for policy 0, policy_version 31642 (0.0010) [2023-10-08 05:10:33,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 64978944. Throughput: 0: 1832.6, 1: 1831.9. Samples: 16258156. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) [2023-10-08 05:10:33,755][130385] Avg episode reward: [(0, '55.520'), (1, '48.480')] [2023-10-08 05:10:33,766][00612] Updated weights for policy 1, policy_version 31810 (0.0008) [2023-10-08 05:10:34,132][00612] Updated weights for policy 1, policy_version 31820 (0.0007) [2023-10-08 05:10:34,493][00612] Updated weights for policy 1, policy_version 31830 (0.0008) [2023-10-08 05:10:34,871][00612] Updated weights for policy 1, policy_version 31840 (0.0007) [2023-10-08 05:10:35,155][00611] Updated weights for policy 0, policy_version 31652 (0.0008) [2023-10-08 05:10:35,523][00611] Updated weights for policy 0, policy_version 31662 (0.0007) [2023-10-08 05:10:35,893][00611] Updated weights for policy 0, policy_version 31672 (0.0007) [2023-10-08 05:10:38,414][00612] Updated weights for policy 1, policy_version 31850 (0.0008) [2023-10-08 05:10:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 65044480. Throughput: 0: 1824.7, 1: 1836.1. Samples: 16268374. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-10-08 05:10:38,754][130385] Avg episode reward: [(0, '54.990'), (1, '50.230')] [2023-10-08 05:10:38,777][00612] Updated weights for policy 1, policy_version 31860 (0.0008) [2023-10-08 05:10:39,154][00612] Updated weights for policy 1, policy_version 31870 (0.0009) [2023-10-08 05:10:39,378][00611] Updated weights for policy 0, policy_version 31682 (0.0009) [2023-10-08 05:10:39,742][00611] Updated weights for policy 0, policy_version 31692 (0.0011) [2023-10-08 05:10:40,111][00611] Updated weights for policy 0, policy_version 31702 (0.0008) [2023-10-08 05:10:40,489][00611] Updated weights for policy 0, policy_version 31712 (0.0010) [2023-10-08 05:10:42,826][00612] Updated weights for policy 1, policy_version 31880 (0.0009) [2023-10-08 05:10:43,196][00612] Updated weights for policy 1, policy_version 31890 (0.0007) [2023-10-08 05:10:43,570][00612] Updated weights for policy 1, policy_version 31900 (0.0009) [2023-10-08 05:10:43,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65142784. Throughput: 0: 1841.7, 1: 1840.6. Samples: 16291570. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-10-08 05:10:43,754][130385] Avg episode reward: [(0, '57.110'), (1, '50.640')] [2023-10-08 05:10:43,961][00611] Updated weights for policy 0, policy_version 31722 (0.0009) [2023-10-08 05:10:44,332][00611] Updated weights for policy 0, policy_version 31732 (0.0009) [2023-10-08 05:10:44,712][00611] Updated weights for policy 0, policy_version 31742 (0.0010) [2023-10-08 05:10:47,370][00612] Updated weights for policy 1, policy_version 31910 (0.0008) [2023-10-08 05:10:47,732][00612] Updated weights for policy 1, policy_version 31920 (0.0008) [2023-10-08 05:10:48,100][00612] Updated weights for policy 1, policy_version 31930 (0.0008) [2023-10-08 05:10:48,391][00611] Updated weights for policy 0, policy_version 31752 (0.0007) [2023-10-08 05:10:48,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65208320. Throughput: 0: 1840.1, 1: 1831.9. Samples: 16313554. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-10-08 05:10:48,754][130385] Avg episode reward: [(0, '56.800'), (1, '52.800')] [2023-10-08 05:10:48,768][00611] Updated weights for policy 0, policy_version 31762 (0.0008) [2023-10-08 05:10:49,136][00611] Updated weights for policy 0, policy_version 31772 (0.0009) [2023-10-08 05:10:51,637][00612] Updated weights for policy 1, policy_version 31940 (0.0008) [2023-10-08 05:10:52,013][00612] Updated weights for policy 1, policy_version 31950 (0.0010) [2023-10-08 05:10:52,382][00612] Updated weights for policy 1, policy_version 31960 (0.0009) [2023-10-08 05:10:52,805][00611] Updated weights for policy 0, policy_version 31782 (0.0009) [2023-10-08 05:10:53,176][00611] Updated weights for policy 0, policy_version 31792 (0.0007) [2023-10-08 05:10:53,549][00611] Updated weights for policy 0, policy_version 31802 (0.0007) [2023-10-08 05:10:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65273856. Throughput: 0: 1842.4, 1: 1849.6. Samples: 16325104. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-10-08 05:10:53,754][130385] Avg episode reward: [(0, '57.050'), (1, '53.280')] [2023-10-08 05:10:56,146][00612] Updated weights for policy 1, policy_version 31970 (0.0009) [2023-10-08 05:10:56,524][00612] Updated weights for policy 1, policy_version 31980 (0.0011) [2023-10-08 05:10:56,887][00612] Updated weights for policy 1, policy_version 31990 (0.0012) [2023-10-08 05:10:57,195][00611] Updated weights for policy 0, policy_version 31812 (0.0010) [2023-10-08 05:10:57,257][00612] Updated weights for policy 1, policy_version 32000 (0.0009) [2023-10-08 05:10:57,570][00611] Updated weights for policy 0, policy_version 31822 (0.0011) [2023-10-08 05:10:57,951][00611] Updated weights for policy 0, policy_version 31832 (0.0010) [2023-10-08 05:10:58,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 65372160. Throughput: 0: 1842.9, 1: 1830.5. Samples: 16346366. Policy #0 lag: (min: 24.0, avg: 44.2, max: 56.0) [2023-10-08 05:10:58,754][130385] Avg episode reward: [(0, '58.470'), (1, '53.490')] [2023-10-08 05:10:58,755][00365] Saving new best policy, reward=58.470! [2023-10-08 05:11:00,800][00612] Updated weights for policy 1, policy_version 32010 (0.0008) [2023-10-08 05:11:01,159][00612] Updated weights for policy 1, policy_version 32020 (0.0007) [2023-10-08 05:11:01,525][00612] Updated weights for policy 1, policy_version 32030 (0.0007) [2023-10-08 05:11:01,656][00611] Updated weights for policy 0, policy_version 31842 (0.0010) [2023-10-08 05:11:02,032][00611] Updated weights for policy 0, policy_version 31852 (0.0010) [2023-10-08 05:11:02,394][00611] Updated weights for policy 0, policy_version 31862 (0.0009) [2023-10-08 05:11:02,767][00611] Updated weights for policy 0, policy_version 31872 (0.0008) [2023-10-08 05:11:03,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 65437696. Throughput: 0: 1844.1, 1: 1851.6. Samples: 16368002. Policy #0 lag: (min: 24.0, avg: 44.2, max: 56.0) [2023-10-08 05:11:03,754][130385] Avg episode reward: [(0, '59.590'), (1, '56.530')] [2023-10-08 05:11:03,763][00365] Saving new best policy, reward=59.590! [2023-10-08 05:11:05,077][00612] Updated weights for policy 1, policy_version 32040 (0.0008) [2023-10-08 05:11:05,444][00612] Updated weights for policy 1, policy_version 32050 (0.0009) [2023-10-08 05:11:05,815][00612] Updated weights for policy 1, policy_version 32060 (0.0008) [2023-10-08 05:11:06,357][00611] Updated weights for policy 0, policy_version 31882 (0.0008) [2023-10-08 05:11:06,727][00611] Updated weights for policy 0, policy_version 31892 (0.0009) [2023-10-08 05:11:07,105][00611] Updated weights for policy 0, policy_version 31902 (0.0008) [2023-10-08 05:11:08,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 65503232. Throughput: 0: 1841.8, 1: 1838.2. Samples: 16379546. Policy #0 lag: (min: 24.0, avg: 44.2, max: 56.0) [2023-10-08 05:11:08,755][130385] Avg episode reward: [(0, '59.810'), (1, '55.130')] [2023-10-08 05:11:08,757][00365] Saving new best policy, reward=59.810! [2023-10-08 05:11:09,303][00612] Updated weights for policy 1, policy_version 32070 (0.0008) [2023-10-08 05:11:09,664][00612] Updated weights for policy 1, policy_version 32080 (0.0007) [2023-10-08 05:11:10,040][00612] Updated weights for policy 1, policy_version 32090 (0.0008) [2023-10-08 05:11:10,816][00611] Updated weights for policy 0, policy_version 31912 (0.0009) [2023-10-08 05:11:11,194][00611] Updated weights for policy 0, policy_version 31922 (0.0009) [2023-10-08 05:11:11,560][00611] Updated weights for policy 0, policy_version 31932 (0.0008) [2023-10-08 05:11:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14200.3, 300 sec: 14662.3). Total num frames: 65568768. Throughput: 0: 1837.2, 1: 1860.2. Samples: 16401308. Policy #0 lag: (min: 24.0, avg: 44.2, max: 56.0) [2023-10-08 05:11:13,754][130385] Avg episode reward: [(0, '58.420'), (1, '53.730')] [2023-10-08 05:11:13,763][00612] Updated weights for policy 1, policy_version 32100 (0.0008) [2023-10-08 05:11:14,134][00612] Updated weights for policy 1, policy_version 32110 (0.0007) [2023-10-08 05:11:14,512][00612] Updated weights for policy 1, policy_version 32120 (0.0009) [2023-10-08 05:11:15,101][00611] Updated weights for policy 0, policy_version 31942 (0.0009) [2023-10-08 05:11:15,474][00611] Updated weights for policy 0, policy_version 31952 (0.0009) [2023-10-08 05:11:15,842][00611] Updated weights for policy 0, policy_version 31962 (0.0008) [2023-10-08 05:11:18,060][00612] Updated weights for policy 1, policy_version 32130 (0.0008) [2023-10-08 05:11:18,421][00612] Updated weights for policy 1, policy_version 32140 (0.0008) [2023-10-08 05:11:18,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 65634304. Throughput: 0: 1849.0, 1: 1854.4. Samples: 16424810. Policy #0 lag: (min: 24.0, avg: 44.2, max: 56.0) [2023-10-08 05:11:18,755][130385] Avg episode reward: [(0, '56.090'), (1, '58.170')] [2023-10-08 05:11:18,792][00612] Updated weights for policy 1, policy_version 32150 (0.0010) [2023-10-08 05:11:19,154][00425] Saving new best policy, reward=58.170! [2023-10-08 05:11:19,155][00612] Updated weights for policy 1, policy_version 32160 (0.0010) [2023-10-08 05:11:19,387][00611] Updated weights for policy 0, policy_version 31972 (0.0008) [2023-10-08 05:11:19,760][00611] Updated weights for policy 0, policy_version 31982 (0.0010) [2023-10-08 05:11:20,133][00611] Updated weights for policy 0, policy_version 31992 (0.0009) [2023-10-08 05:11:22,831][00612] Updated weights for policy 1, policy_version 32170 (0.0010) [2023-10-08 05:11:23,202][00612] Updated weights for policy 1, policy_version 32180 (0.0008) [2023-10-08 05:11:23,568][00612] Updated weights for policy 1, policy_version 32190 (0.0007) [2023-10-08 05:11:23,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 65732608. Throughput: 0: 1846.0, 1: 1859.7. Samples: 16435132. Policy #0 lag: (min: 30.0, avg: 46.4, max: 48.0) [2023-10-08 05:11:23,755][130385] Avg episode reward: [(0, '55.960'), (1, '57.140')] [2023-10-08 05:11:23,885][00611] Updated weights for policy 0, policy_version 32002 (0.0008) [2023-10-08 05:11:24,249][00611] Updated weights for policy 0, policy_version 32012 (0.0009) [2023-10-08 05:11:24,626][00611] Updated weights for policy 0, policy_version 32022 (0.0009) [2023-10-08 05:11:25,013][00611] Updated weights for policy 0, policy_version 32032 (0.0010) [2023-10-08 05:11:27,117][00612] Updated weights for policy 1, policy_version 32200 (0.0009) [2023-10-08 05:11:27,494][00612] Updated weights for policy 1, policy_version 32210 (0.0010) [2023-10-08 05:11:27,867][00612] Updated weights for policy 1, policy_version 32220 (0.0011) [2023-10-08 05:11:28,602][00611] Updated weights for policy 0, policy_version 32042 (0.0009) [2023-10-08 05:11:28,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 65798144. Throughput: 0: 1837.6, 1: 1849.1. Samples: 16457470. Policy #0 lag: (min: 30.0, avg: 46.4, max: 48.0) [2023-10-08 05:11:28,755][130385] Avg episode reward: [(0, '55.050'), (1, '56.580')] [2023-10-08 05:11:28,983][00611] Updated weights for policy 0, policy_version 32052 (0.0007) [2023-10-08 05:11:29,349][00611] Updated weights for policy 0, policy_version 32062 (0.0007) [2023-10-08 05:11:31,548][00612] Updated weights for policy 1, policy_version 32230 (0.0009) [2023-10-08 05:11:31,908][00612] Updated weights for policy 1, policy_version 32240 (0.0010) [2023-10-08 05:11:32,285][00612] Updated weights for policy 1, policy_version 32250 (0.0008) [2023-10-08 05:11:32,955][00611] Updated weights for policy 0, policy_version 32072 (0.0008) [2023-10-08 05:11:33,326][00611] Updated weights for policy 0, policy_version 32082 (0.0008) [2023-10-08 05:11:33,695][00611] Updated weights for policy 0, policy_version 32092 (0.0007) [2023-10-08 05:11:33,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65863680. Throughput: 0: 1825.9, 1: 1851.9. Samples: 16479058. Policy #0 lag: (min: 30.0, avg: 46.4, max: 48.0) [2023-10-08 05:11:33,755][130385] Avg episode reward: [(0, '52.150'), (1, '58.220')] [2023-10-08 05:11:33,765][00425] Saving new best policy, reward=58.220! [2023-10-08 05:11:35,879][00612] Updated weights for policy 1, policy_version 32260 (0.0010) [2023-10-08 05:11:36,256][00612] Updated weights for policy 1, policy_version 32270 (0.0009) [2023-10-08 05:11:36,622][00612] Updated weights for policy 1, policy_version 32280 (0.0009) [2023-10-08 05:11:37,393][00611] Updated weights for policy 0, policy_version 32102 (0.0008) [2023-10-08 05:11:37,759][00611] Updated weights for policy 0, policy_version 32112 (0.0008) [2023-10-08 05:11:38,132][00611] Updated weights for policy 0, policy_version 32122 (0.0008) [2023-10-08 05:11:38,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 65961984. Throughput: 0: 1836.7, 1: 1844.0. Samples: 16490732. Policy #0 lag: (min: 30.0, avg: 46.4, max: 48.0) [2023-10-08 05:11:38,754][130385] Avg episode reward: [(0, '49.210'), (1, '56.900')] [2023-10-08 05:11:40,272][00612] Updated weights for policy 1, policy_version 32290 (0.0008) [2023-10-08 05:11:40,648][00612] Updated weights for policy 1, policy_version 32300 (0.0008) [2023-10-08 05:11:41,021][00612] Updated weights for policy 1, policy_version 32310 (0.0010) [2023-10-08 05:11:41,373][00612] Updated weights for policy 1, policy_version 32320 (0.0009) [2023-10-08 05:11:41,681][00611] Updated weights for policy 0, policy_version 32132 (0.0008) [2023-10-08 05:11:42,062][00611] Updated weights for policy 0, policy_version 32142 (0.0008) [2023-10-08 05:11:42,438][00611] Updated weights for policy 0, policy_version 32152 (0.0008) [2023-10-08 05:11:43,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 66027520. Throughput: 0: 1828.5, 1: 1862.5. Samples: 16512462. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 05:11:43,755][130385] Avg episode reward: [(0, '49.640'), (1, '57.590')] [2023-10-08 05:11:45,035][00612] Updated weights for policy 1, policy_version 32330 (0.0010) [2023-10-08 05:11:45,410][00612] Updated weights for policy 1, policy_version 32340 (0.0011) [2023-10-08 05:11:45,777][00612] Updated weights for policy 1, policy_version 32350 (0.0011) [2023-10-08 05:11:45,993][00611] Updated weights for policy 0, policy_version 32162 (0.0008) [2023-10-08 05:11:46,378][00611] Updated weights for policy 0, policy_version 32172 (0.0010) [2023-10-08 05:11:46,746][00611] Updated weights for policy 0, policy_version 32182 (0.0009) [2023-10-08 05:11:47,117][00611] Updated weights for policy 0, policy_version 32192 (0.0007) [2023-10-08 05:11:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 66093056. Throughput: 0: 1844.8, 1: 1858.4. Samples: 16534644. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 05:11:48,754][130385] Avg episode reward: [(0, '49.110'), (1, '59.660')] [2023-10-08 05:11:48,763][00425] Saving new best policy, reward=59.660! [2023-10-08 05:11:49,433][00612] Updated weights for policy 1, policy_version 32360 (0.0007) [2023-10-08 05:11:49,803][00612] Updated weights for policy 1, policy_version 32370 (0.0007) [2023-10-08 05:11:50,172][00612] Updated weights for policy 1, policy_version 32380 (0.0007) [2023-10-08 05:11:50,694][00611] Updated weights for policy 0, policy_version 32202 (0.0008) [2023-10-08 05:11:51,068][00611] Updated weights for policy 0, policy_version 32212 (0.0008) [2023-10-08 05:11:51,455][00611] Updated weights for policy 0, policy_version 32222 (0.0008) [2023-10-08 05:11:53,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 66158592. Throughput: 0: 1827.4, 1: 1851.4. Samples: 16545092. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 05:11:53,754][130385] Avg episode reward: [(0, '50.050'), (1, '58.060')] [2023-10-08 05:11:53,760][00612] Updated weights for policy 1, policy_version 32390 (0.0008) [2023-10-08 05:11:54,122][00612] Updated weights for policy 1, policy_version 32400 (0.0008) [2023-10-08 05:11:54,495][00612] Updated weights for policy 1, policy_version 32410 (0.0007) [2023-10-08 05:11:55,098][00611] Updated weights for policy 0, policy_version 32232 (0.0009) [2023-10-08 05:11:55,460][00611] Updated weights for policy 0, policy_version 32242 (0.0009) [2023-10-08 05:11:55,827][00611] Updated weights for policy 0, policy_version 32252 (0.0008) [2023-10-08 05:11:58,040][00612] Updated weights for policy 1, policy_version 32420 (0.0010) [2023-10-08 05:11:58,403][00612] Updated weights for policy 1, policy_version 32430 (0.0009) [2023-10-08 05:11:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 66224128. Throughput: 0: 1852.0, 1: 1846.6. Samples: 16567744. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 05:11:58,754][130385] Avg episode reward: [(0, '50.920'), (1, '57.660')] [2023-10-08 05:11:58,762][00612] Updated weights for policy 1, policy_version 32440 (0.0007) [2023-10-08 05:11:59,572][00611] Updated weights for policy 0, policy_version 32262 (0.0007) [2023-10-08 05:11:59,946][00611] Updated weights for policy 0, policy_version 32272 (0.0008) [2023-10-08 05:12:00,312][00611] Updated weights for policy 0, policy_version 32282 (0.0008) [2023-10-08 05:12:02,369][00612] Updated weights for policy 1, policy_version 32450 (0.0008) [2023-10-08 05:12:02,734][00612] Updated weights for policy 1, policy_version 32460 (0.0007) [2023-10-08 05:12:03,100][00612] Updated weights for policy 1, policy_version 32470 (0.0008) [2023-10-08 05:12:03,475][00612] Updated weights for policy 1, policy_version 32480 (0.0009) [2023-10-08 05:12:03,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 66322432. Throughput: 0: 1842.8, 1: 1827.4. Samples: 16589966. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 05:12:03,755][130385] Avg episode reward: [(0, '49.690'), (1, '57.010')] [2023-10-08 05:12:03,957][00611] Updated weights for policy 0, policy_version 32292 (0.0009) [2023-10-08 05:12:04,331][00611] Updated weights for policy 0, policy_version 32302 (0.0011) [2023-10-08 05:12:04,708][00611] Updated weights for policy 0, policy_version 32312 (0.0010) [2023-10-08 05:12:07,075][00612] Updated weights for policy 1, policy_version 32490 (0.0010) [2023-10-08 05:12:07,452][00612] Updated weights for policy 1, policy_version 32500 (0.0011) [2023-10-08 05:12:07,809][00612] Updated weights for policy 1, policy_version 32510 (0.0010) [2023-10-08 05:12:08,398][00611] Updated weights for policy 0, policy_version 32322 (0.0010) [2023-10-08 05:12:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 66387968. Throughput: 0: 1839.9, 1: 1846.4. Samples: 16601016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:12:08,754][130385] Avg episode reward: [(0, '49.740'), (1, '52.850')] [2023-10-08 05:12:08,774][00611] Updated weights for policy 0, policy_version 32332 (0.0008) [2023-10-08 05:12:09,144][00611] Updated weights for policy 0, policy_version 32342 (0.0008) [2023-10-08 05:12:09,517][00611] Updated weights for policy 0, policy_version 32352 (0.0008) [2023-10-08 05:12:11,449][00612] Updated weights for policy 1, policy_version 32520 (0.0008) [2023-10-08 05:12:11,819][00612] Updated weights for policy 1, policy_version 32530 (0.0008) [2023-10-08 05:12:12,184][00612] Updated weights for policy 1, policy_version 32540 (0.0007) [2023-10-08 05:12:13,155][00611] Updated weights for policy 0, policy_version 32362 (0.0008) [2023-10-08 05:12:13,527][00611] Updated weights for policy 0, policy_version 32372 (0.0009) [2023-10-08 05:12:13,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 66453504. Throughput: 0: 1850.4, 1: 1827.7. Samples: 16622980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:12:13,754][130385] Avg episode reward: [(0, '48.930'), (1, '54.590')] [2023-10-08 05:12:13,896][00611] Updated weights for policy 0, policy_version 32382 (0.0009) [2023-10-08 05:12:15,794][00612] Updated weights for policy 1, policy_version 32550 (0.0008) [2023-10-08 05:12:16,167][00612] Updated weights for policy 1, policy_version 32560 (0.0008) [2023-10-08 05:12:16,545][00612] Updated weights for policy 1, policy_version 32570 (0.0007) [2023-10-08 05:12:17,553][00611] Updated weights for policy 0, policy_version 32392 (0.0009) [2023-10-08 05:12:17,932][00611] Updated weights for policy 0, policy_version 32402 (0.0009) [2023-10-08 05:12:18,306][00611] Updated weights for policy 0, policy_version 32412 (0.0009) [2023-10-08 05:12:18,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 66551808. Throughput: 0: 1833.9, 1: 1851.4. Samples: 16644896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:12:18,754][130385] Avg episode reward: [(0, '51.120'), (1, '53.080')] [2023-10-08 05:12:18,763][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000032576_33357824.pth... [2023-10-08 05:12:18,763][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000032416_33193984.pth... [2023-10-08 05:12:18,804][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000030848_31588352.pth [2023-10-08 05:12:18,804][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000030688_31424512.pth [2023-10-08 05:12:20,289][00612] Updated weights for policy 1, policy_version 32580 (0.0008) [2023-10-08 05:12:20,670][00612] Updated weights for policy 1, policy_version 32590 (0.0007) [2023-10-08 05:12:21,050][00612] Updated weights for policy 1, policy_version 32600 (0.0008) [2023-10-08 05:12:21,811][00611] Updated weights for policy 0, policy_version 32422 (0.0008) [2023-10-08 05:12:22,187][00611] Updated weights for policy 0, policy_version 32432 (0.0007) [2023-10-08 05:12:22,561][00611] Updated weights for policy 0, policy_version 32442 (0.0007) [2023-10-08 05:12:23,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 66617344. Throughput: 0: 1852.1, 1: 1830.7. Samples: 16656458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:12:23,754][130385] Avg episode reward: [(0, '49.990'), (1, '52.130')] [2023-10-08 05:12:24,620][00612] Updated weights for policy 1, policy_version 32610 (0.0007) [2023-10-08 05:12:24,986][00612] Updated weights for policy 1, policy_version 32620 (0.0007) [2023-10-08 05:12:25,359][00612] Updated weights for policy 1, policy_version 32630 (0.0008) [2023-10-08 05:12:25,726][00612] Updated weights for policy 1, policy_version 32640 (0.0008) [2023-10-08 05:12:26,255][00611] Updated weights for policy 0, policy_version 32452 (0.0008) [2023-10-08 05:12:26,638][00611] Updated weights for policy 0, policy_version 32462 (0.0008) [2023-10-08 05:12:27,012][00611] Updated weights for policy 0, policy_version 32472 (0.0007) [2023-10-08 05:12:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 66682880. Throughput: 0: 1837.8, 1: 1849.1. Samples: 16678372. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-08 05:12:28,754][130385] Avg episode reward: [(0, '48.930'), (1, '52.300')] [2023-10-08 05:12:29,310][00612] Updated weights for policy 1, policy_version 32650 (0.0007) [2023-10-08 05:12:29,677][00612] Updated weights for policy 1, policy_version 32660 (0.0007) [2023-10-08 05:12:30,055][00612] Updated weights for policy 1, policy_version 32670 (0.0007) [2023-10-08 05:12:30,541][00611] Updated weights for policy 0, policy_version 32482 (0.0008) [2023-10-08 05:12:30,911][00611] Updated weights for policy 0, policy_version 32492 (0.0007) [2023-10-08 05:12:31,290][00611] Updated weights for policy 0, policy_version 32502 (0.0010) [2023-10-08 05:12:31,655][00611] Updated weights for policy 0, policy_version 32512 (0.0011) [2023-10-08 05:12:33,583][00612] Updated weights for policy 1, policy_version 32680 (0.0007) [2023-10-08 05:12:33,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 66748416. Throughput: 0: 1849.8, 1: 1856.1. Samples: 16701412. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-08 05:12:33,755][130385] Avg episode reward: [(0, '47.690'), (1, '51.760')] [2023-10-08 05:12:33,947][00612] Updated weights for policy 1, policy_version 32690 (0.0008) [2023-10-08 05:12:34,317][00612] Updated weights for policy 1, policy_version 32700 (0.0009) [2023-10-08 05:12:35,373][00611] Updated weights for policy 0, policy_version 32522 (0.0009) [2023-10-08 05:12:35,754][00611] Updated weights for policy 0, policy_version 32532 (0.0007) [2023-10-08 05:12:36,122][00611] Updated weights for policy 0, policy_version 32542 (0.0007) [2023-10-08 05:12:37,934][00612] Updated weights for policy 1, policy_version 32710 (0.0007) [2023-10-08 05:12:38,311][00612] Updated weights for policy 1, policy_version 32720 (0.0010) [2023-10-08 05:12:38,684][00612] Updated weights for policy 1, policy_version 32730 (0.0008) [2023-10-08 05:12:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 66813952. Throughput: 0: 1838.8, 1: 1862.6. Samples: 16711656. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-08 05:12:38,755][130385] Avg episode reward: [(0, '47.540'), (1, '51.190')] [2023-10-08 05:12:39,770][00611] Updated weights for policy 0, policy_version 32552 (0.0007) [2023-10-08 05:12:40,151][00611] Updated weights for policy 0, policy_version 32562 (0.0008) [2023-10-08 05:12:40,526][00611] Updated weights for policy 0, policy_version 32572 (0.0009) [2023-10-08 05:12:42,322][00612] Updated weights for policy 1, policy_version 32740 (0.0007) [2023-10-08 05:12:42,686][00612] Updated weights for policy 1, policy_version 32750 (0.0007) [2023-10-08 05:12:43,059][00612] Updated weights for policy 1, policy_version 32760 (0.0009) [2023-10-08 05:12:43,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 66912256. Throughput: 0: 1841.6, 1: 1861.0. Samples: 16734364. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-08 05:12:43,754][130385] Avg episode reward: [(0, '46.410'), (1, '53.070')] [2023-10-08 05:12:44,103][00611] Updated weights for policy 0, policy_version 32582 (0.0011) [2023-10-08 05:12:44,473][00611] Updated weights for policy 0, policy_version 32592 (0.0010) [2023-10-08 05:12:44,851][00611] Updated weights for policy 0, policy_version 32602 (0.0008) [2023-10-08 05:12:46,709][00612] Updated weights for policy 1, policy_version 32770 (0.0008) [2023-10-08 05:12:47,073][00612] Updated weights for policy 1, policy_version 32780 (0.0007) [2023-10-08 05:12:47,443][00612] Updated weights for policy 1, policy_version 32790 (0.0007) [2023-10-08 05:12:47,814][00612] Updated weights for policy 1, policy_version 32800 (0.0010) [2023-10-08 05:12:48,463][00611] Updated weights for policy 0, policy_version 32612 (0.0008) [2023-10-08 05:12:48,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 66977792. Throughput: 0: 1846.2, 1: 1845.5. Samples: 16756092. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-08 05:12:48,754][130385] Avg episode reward: [(0, '45.490'), (1, '52.450')] [2023-10-08 05:12:48,835][00611] Updated weights for policy 0, policy_version 32622 (0.0008) [2023-10-08 05:12:49,197][00611] Updated weights for policy 0, policy_version 32632 (0.0008) [2023-10-08 05:12:51,484][00612] Updated weights for policy 1, policy_version 32810 (0.0009) [2023-10-08 05:12:51,853][00612] Updated weights for policy 1, policy_version 32820 (0.0007) [2023-10-08 05:12:52,224][00612] Updated weights for policy 1, policy_version 32830 (0.0008) [2023-10-08 05:12:52,937][00611] Updated weights for policy 0, policy_version 32642 (0.0009) [2023-10-08 05:12:53,309][00611] Updated weights for policy 0, policy_version 32652 (0.0008) [2023-10-08 05:12:53,672][00611] Updated weights for policy 0, policy_version 32662 (0.0010) [2023-10-08 05:12:53,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 67043328. Throughput: 0: 1848.0, 1: 1851.5. Samples: 16767494. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 05:12:53,756][130385] Avg episode reward: [(0, '46.020'), (1, '57.360')] [2023-10-08 05:12:54,041][00611] Updated weights for policy 0, policy_version 32672 (0.0008) [2023-10-08 05:12:55,831][00612] Updated weights for policy 1, policy_version 32840 (0.0007) [2023-10-08 05:12:56,215][00612] Updated weights for policy 1, policy_version 32850 (0.0009) [2023-10-08 05:12:56,581][00612] Updated weights for policy 1, policy_version 32860 (0.0010) [2023-10-08 05:12:57,620][00611] Updated weights for policy 0, policy_version 32682 (0.0007) [2023-10-08 05:12:57,998][00611] Updated weights for policy 0, policy_version 32692 (0.0008) [2023-10-08 05:12:58,372][00611] Updated weights for policy 0, policy_version 32702 (0.0010) [2023-10-08 05:12:58,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 67141632. Throughput: 0: 1850.4, 1: 1849.2. Samples: 16789462. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 05:12:58,754][130385] Avg episode reward: [(0, '47.580'), (1, '57.260')] [2023-10-08 05:13:00,222][00612] Updated weights for policy 1, policy_version 32870 (0.0009) [2023-10-08 05:13:00,582][00612] Updated weights for policy 1, policy_version 32880 (0.0011) [2023-10-08 05:13:00,960][00612] Updated weights for policy 1, policy_version 32890 (0.0008) [2023-10-08 05:13:01,884][00611] Updated weights for policy 0, policy_version 32712 (0.0008) [2023-10-08 05:13:02,262][00611] Updated weights for policy 0, policy_version 32722 (0.0008) [2023-10-08 05:13:02,629][00611] Updated weights for policy 0, policy_version 32732 (0.0008) [2023-10-08 05:13:03,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 67207168. Throughput: 0: 1848.2, 1: 1852.1. Samples: 16811410. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 05:13:03,754][130385] Avg episode reward: [(0, '49.760'), (1, '57.690')] [2023-10-08 05:13:04,552][00612] Updated weights for policy 1, policy_version 32900 (0.0009) [2023-10-08 05:13:04,918][00612] Updated weights for policy 1, policy_version 32910 (0.0007) [2023-10-08 05:13:05,297][00612] Updated weights for policy 1, policy_version 32920 (0.0007) [2023-10-08 05:13:06,248][00611] Updated weights for policy 0, policy_version 32742 (0.0008) [2023-10-08 05:13:06,625][00611] Updated weights for policy 0, policy_version 32752 (0.0007) [2023-10-08 05:13:07,009][00611] Updated weights for policy 0, policy_version 32762 (0.0007) [2023-10-08 05:13:08,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 67272704. Throughput: 0: 1849.5, 1: 1847.3. Samples: 16822816. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 05:13:08,755][130385] Avg episode reward: [(0, '53.740'), (1, '61.670')] [2023-10-08 05:13:08,756][00425] Saving new best policy, reward=61.670! [2023-10-08 05:13:08,960][00612] Updated weights for policy 1, policy_version 32930 (0.0007) [2023-10-08 05:13:09,330][00612] Updated weights for policy 1, policy_version 32940 (0.0008) [2023-10-08 05:13:09,704][00612] Updated weights for policy 1, policy_version 32950 (0.0010) [2023-10-08 05:13:10,070][00612] Updated weights for policy 1, policy_version 32960 (0.0009) [2023-10-08 05:13:10,744][00611] Updated weights for policy 0, policy_version 32772 (0.0008) [2023-10-08 05:13:11,113][00611] Updated weights for policy 0, policy_version 32782 (0.0008) [2023-10-08 05:13:11,485][00611] Updated weights for policy 0, policy_version 32792 (0.0009) [2023-10-08 05:13:13,621][00612] Updated weights for policy 1, policy_version 32970 (0.0007) [2023-10-08 05:13:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 67338240. Throughput: 0: 1838.8, 1: 1852.1. Samples: 16844464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:13:13,754][130385] Avg episode reward: [(0, '52.480'), (1, '62.120')] [2023-10-08 05:13:13,989][00612] Updated weights for policy 1, policy_version 32980 (0.0007) [2023-10-08 05:13:14,362][00612] Updated weights for policy 1, policy_version 32990 (0.0009) [2023-10-08 05:13:14,428][00425] Saving new best policy, reward=62.120! [2023-10-08 05:13:15,078][00611] Updated weights for policy 0, policy_version 32802 (0.0009) [2023-10-08 05:13:15,441][00611] Updated weights for policy 0, policy_version 32812 (0.0010) [2023-10-08 05:13:15,814][00611] Updated weights for policy 0, policy_version 32822 (0.0009) [2023-10-08 05:13:16,186][00611] Updated weights for policy 0, policy_version 32832 (0.0007) [2023-10-08 05:13:17,903][00612] Updated weights for policy 1, policy_version 33000 (0.0009) [2023-10-08 05:13:18,277][00612] Updated weights for policy 1, policy_version 33010 (0.0009) [2023-10-08 05:13:18,651][00612] Updated weights for policy 1, policy_version 33020 (0.0007) [2023-10-08 05:13:18,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 67403776. Throughput: 0: 1851.6, 1: 1836.9. Samples: 16867394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:13:18,754][130385] Avg episode reward: [(0, '52.230'), (1, '64.550')] [2023-10-08 05:13:18,799][00425] Saving new best policy, reward=64.550! [2023-10-08 05:13:19,829][00611] Updated weights for policy 0, policy_version 32842 (0.0007) [2023-10-08 05:13:20,204][00611] Updated weights for policy 0, policy_version 32852 (0.0007) [2023-10-08 05:13:20,588][00611] Updated weights for policy 0, policy_version 32862 (0.0009) [2023-10-08 05:13:22,230][00612] Updated weights for policy 1, policy_version 33030 (0.0008) [2023-10-08 05:13:22,599][00612] Updated weights for policy 1, policy_version 33040 (0.0007) [2023-10-08 05:13:22,968][00612] Updated weights for policy 1, policy_version 33050 (0.0007) [2023-10-08 05:13:23,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 67502080. Throughput: 0: 1847.2, 1: 1851.0. Samples: 16878072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:13:23,755][130385] Avg episode reward: [(0, '53.460'), (1, '64.270')] [2023-10-08 05:13:24,143][00611] Updated weights for policy 0, policy_version 32872 (0.0011) [2023-10-08 05:13:24,515][00611] Updated weights for policy 0, policy_version 32882 (0.0010) [2023-10-08 05:13:24,877][00611] Updated weights for policy 0, policy_version 32892 (0.0007) [2023-10-08 05:13:26,728][00612] Updated weights for policy 1, policy_version 33060 (0.0008) [2023-10-08 05:13:27,129][00612] Updated weights for policy 1, policy_version 33070 (0.0012) [2023-10-08 05:13:27,495][00612] Updated weights for policy 1, policy_version 33080 (0.0010) [2023-10-08 05:13:28,588][00611] Updated weights for policy 0, policy_version 32902 (0.0007) [2023-10-08 05:13:28,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 67567616. Throughput: 0: 1854.3, 1: 1832.2. Samples: 16900256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:13:28,754][130385] Avg episode reward: [(0, '53.600'), (1, '62.760')] [2023-10-08 05:13:28,956][00611] Updated weights for policy 0, policy_version 32912 (0.0007) [2023-10-08 05:13:29,336][00611] Updated weights for policy 0, policy_version 32922 (0.0008) [2023-10-08 05:13:31,068][00612] Updated weights for policy 1, policy_version 33090 (0.0008) [2023-10-08 05:13:31,438][00612] Updated weights for policy 1, policy_version 33100 (0.0010) [2023-10-08 05:13:31,804][00612] Updated weights for policy 1, policy_version 33110 (0.0009) [2023-10-08 05:13:32,173][00612] Updated weights for policy 1, policy_version 33120 (0.0010) [2023-10-08 05:13:32,844][00611] Updated weights for policy 0, policy_version 32932 (0.0010) [2023-10-08 05:13:33,219][00611] Updated weights for policy 0, policy_version 32942 (0.0008) [2023-10-08 05:13:33,585][00611] Updated weights for policy 0, policy_version 32952 (0.0007) [2023-10-08 05:13:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 67633152. Throughput: 0: 1834.2, 1: 1853.1. Samples: 16922020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:13:33,754][130385] Avg episode reward: [(0, '55.130'), (1, '63.740')] [2023-10-08 05:13:35,933][00612] Updated weights for policy 1, policy_version 33130 (0.0009) [2023-10-08 05:13:36,298][00612] Updated weights for policy 1, policy_version 33140 (0.0009) [2023-10-08 05:13:36,671][00612] Updated weights for policy 1, policy_version 33150 (0.0007) [2023-10-08 05:13:37,264][00611] Updated weights for policy 0, policy_version 32962 (0.0009) [2023-10-08 05:13:37,637][00611] Updated weights for policy 0, policy_version 32972 (0.0008) [2023-10-08 05:13:38,009][00611] Updated weights for policy 0, policy_version 32982 (0.0008) [2023-10-08 05:13:38,383][00611] Updated weights for policy 0, policy_version 32992 (0.0011) [2023-10-08 05:13:38,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 67731456. Throughput: 0: 1848.0, 1: 1835.5. Samples: 16933252. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 05:13:38,755][130385] Avg episode reward: [(0, '55.860'), (1, '59.920')] [2023-10-08 05:13:40,115][00612] Updated weights for policy 1, policy_version 33160 (0.0007) [2023-10-08 05:13:40,483][00612] Updated weights for policy 1, policy_version 33170 (0.0008) [2023-10-08 05:13:40,856][00612] Updated weights for policy 1, policy_version 33180 (0.0010) [2023-10-08 05:13:42,004][00611] Updated weights for policy 0, policy_version 33002 (0.0010) [2023-10-08 05:13:42,368][00611] Updated weights for policy 0, policy_version 33012 (0.0009) [2023-10-08 05:13:42,740][00611] Updated weights for policy 0, policy_version 33022 (0.0009) [2023-10-08 05:13:43,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 67796992. Throughput: 0: 1827.7, 1: 1851.8. Samples: 16955040. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 05:13:43,755][130385] Avg episode reward: [(0, '56.680'), (1, '58.310')] [2023-10-08 05:13:44,592][00612] Updated weights for policy 1, policy_version 33190 (0.0008) [2023-10-08 05:13:44,961][00612] Updated weights for policy 1, policy_version 33200 (0.0008) [2023-10-08 05:13:45,325][00612] Updated weights for policy 1, policy_version 33210 (0.0010) [2023-10-08 05:13:46,419][00611] Updated weights for policy 0, policy_version 33032 (0.0009) [2023-10-08 05:13:46,788][00611] Updated weights for policy 0, policy_version 33042 (0.0008) [2023-10-08 05:13:47,156][00611] Updated weights for policy 0, policy_version 33052 (0.0008) [2023-10-08 05:13:48,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 67862528. Throughput: 0: 1834.8, 1: 1856.0. Samples: 16977496. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 05:13:48,755][130385] Avg episode reward: [(0, '55.510'), (1, '53.560')] [2023-10-08 05:13:48,841][00612] Updated weights for policy 1, policy_version 33220 (0.0010) [2023-10-08 05:13:49,209][00612] Updated weights for policy 1, policy_version 33230 (0.0009) [2023-10-08 05:13:49,583][00612] Updated weights for policy 1, policy_version 33240 (0.0008) [2023-10-08 05:13:50,822][00611] Updated weights for policy 0, policy_version 33062 (0.0009) [2023-10-08 05:13:51,191][00611] Updated weights for policy 0, policy_version 33072 (0.0007) [2023-10-08 05:13:51,563][00611] Updated weights for policy 0, policy_version 33082 (0.0007) [2023-10-08 05:13:53,268][00612] Updated weights for policy 1, policy_version 33250 (0.0009) [2023-10-08 05:13:53,641][00612] Updated weights for policy 1, policy_version 33260 (0.0010) [2023-10-08 05:13:53,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 67928064. Throughput: 0: 1822.9, 1: 1855.2. Samples: 16988330. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 05:13:53,754][130385] Avg episode reward: [(0, '58.450'), (1, '49.740')] [2023-10-08 05:13:54,010][00612] Updated weights for policy 1, policy_version 33270 (0.0010) [2023-10-08 05:13:54,378][00612] Updated weights for policy 1, policy_version 33280 (0.0008) [2023-10-08 05:13:55,141][00611] Updated weights for policy 0, policy_version 33092 (0.0008) [2023-10-08 05:13:55,510][00611] Updated weights for policy 0, policy_version 33102 (0.0007) [2023-10-08 05:13:55,885][00611] Updated weights for policy 0, policy_version 33112 (0.0009) [2023-10-08 05:13:58,087][00612] Updated weights for policy 1, policy_version 33290 (0.0007) [2023-10-08 05:13:58,448][00612] Updated weights for policy 1, policy_version 33300 (0.0008) [2023-10-08 05:13:58,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 67993600. Throughput: 0: 1840.0, 1: 1844.0. Samples: 17010246. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 05:13:58,754][130385] Avg episode reward: [(0, '57.080'), (1, '47.900')] [2023-10-08 05:13:58,820][00612] Updated weights for policy 1, policy_version 33310 (0.0008) [2023-10-08 05:13:59,492][00611] Updated weights for policy 0, policy_version 33122 (0.0008) [2023-10-08 05:13:59,865][00611] Updated weights for policy 0, policy_version 33132 (0.0008) [2023-10-08 05:14:00,241][00611] Updated weights for policy 0, policy_version 33142 (0.0011) [2023-10-08 05:14:00,624][00611] Updated weights for policy 0, policy_version 33152 (0.0010) [2023-10-08 05:14:02,466][00612] Updated weights for policy 1, policy_version 33320 (0.0007) [2023-10-08 05:14:02,834][00612] Updated weights for policy 1, policy_version 33330 (0.0009) [2023-10-08 05:14:03,204][00612] Updated weights for policy 1, policy_version 33340 (0.0009) [2023-10-08 05:14:03,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 68091904. Throughput: 0: 1837.9, 1: 1827.4. Samples: 17032334. Policy #0 lag: (min: 9.0, avg: 15.9, max: 41.0) [2023-10-08 05:14:03,754][130385] Avg episode reward: [(0, '57.070'), (1, '45.340')] [2023-10-08 05:14:04,285][00611] Updated weights for policy 0, policy_version 33162 (0.0009) [2023-10-08 05:14:04,656][00611] Updated weights for policy 0, policy_version 33172 (0.0011) [2023-10-08 05:14:05,029][00611] Updated weights for policy 0, policy_version 33182 (0.0010) [2023-10-08 05:14:06,870][00612] Updated weights for policy 1, policy_version 33350 (0.0009) [2023-10-08 05:14:07,244][00612] Updated weights for policy 1, policy_version 33360 (0.0009) [2023-10-08 05:14:07,615][00612] Updated weights for policy 1, policy_version 33370 (0.0007) [2023-10-08 05:14:08,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 68157440. Throughput: 0: 1836.3, 1: 1838.0. Samples: 17043412. Policy #0 lag: (min: 9.0, avg: 15.9, max: 41.0) [2023-10-08 05:14:08,754][130385] Avg episode reward: [(0, '58.440'), (1, '46.900')] [2023-10-08 05:14:08,820][00611] Updated weights for policy 0, policy_version 33192 (0.0009) [2023-10-08 05:14:09,198][00611] Updated weights for policy 0, policy_version 33202 (0.0007) [2023-10-08 05:14:09,568][00611] Updated weights for policy 0, policy_version 33212 (0.0007) [2023-10-08 05:14:11,230][00612] Updated weights for policy 1, policy_version 33380 (0.0009) [2023-10-08 05:14:11,601][00612] Updated weights for policy 1, policy_version 33390 (0.0007) [2023-10-08 05:14:11,961][00612] Updated weights for policy 1, policy_version 33400 (0.0010) [2023-10-08 05:14:13,206][00611] Updated weights for policy 0, policy_version 33222 (0.0007) [2023-10-08 05:14:13,578][00611] Updated weights for policy 0, policy_version 33232 (0.0010) [2023-10-08 05:14:13,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 68222976. Throughput: 0: 1826.7, 1: 1833.0. Samples: 17064942. Policy #0 lag: (min: 9.0, avg: 15.9, max: 41.0) [2023-10-08 05:14:13,754][130385] Avg episode reward: [(0, '56.970'), (1, '48.930')] [2023-10-08 05:14:13,957][00611] Updated weights for policy 0, policy_version 33242 (0.0010) [2023-10-08 05:14:15,513][00612] Updated weights for policy 1, policy_version 33410 (0.0010) [2023-10-08 05:14:15,937][00612] Updated weights for policy 1, policy_version 33420 (0.0008) [2023-10-08 05:14:16,297][00612] Updated weights for policy 1, policy_version 33430 (0.0009) [2023-10-08 05:14:16,668][00612] Updated weights for policy 1, policy_version 33440 (0.0008) [2023-10-08 05:14:17,695][00611] Updated weights for policy 0, policy_version 33252 (0.0008) [2023-10-08 05:14:18,063][00611] Updated weights for policy 0, policy_version 33262 (0.0008) [2023-10-08 05:14:18,443][00611] Updated weights for policy 0, policy_version 33272 (0.0009) [2023-10-08 05:14:18,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 68321280. Throughput: 0: 1824.3, 1: 1846.3. Samples: 17087196. Policy #0 lag: (min: 9.0, avg: 15.9, max: 41.0) [2023-10-08 05:14:18,754][130385] Avg episode reward: [(0, '56.020'), (1, '49.110')] [2023-10-08 05:14:18,765][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000033280_34078720.pth... [2023-10-08 05:14:18,765][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000033440_34242560.pth... [2023-10-08 05:14:18,801][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000031712_32473088.pth [2023-10-08 05:14:18,804][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000031552_32309248.pth [2023-10-08 05:14:20,201][00612] Updated weights for policy 1, policy_version 33450 (0.0009) [2023-10-08 05:14:20,578][00612] Updated weights for policy 1, policy_version 33460 (0.0009) [2023-10-08 05:14:20,950][00612] Updated weights for policy 1, policy_version 33470 (0.0010) [2023-10-08 05:14:22,111][00611] Updated weights for policy 0, policy_version 33282 (0.0010) [2023-10-08 05:14:22,475][00611] Updated weights for policy 0, policy_version 33292 (0.0009) [2023-10-08 05:14:22,857][00611] Updated weights for policy 0, policy_version 33302 (0.0009) [2023-10-08 05:14:23,229][00611] Updated weights for policy 0, policy_version 33312 (0.0009) [2023-10-08 05:14:23,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 68386816. Throughput: 0: 1828.4, 1: 1831.0. Samples: 17097922. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-08 05:14:23,754][130385] Avg episode reward: [(0, '54.760'), (1, '48.900')] [2023-10-08 05:14:24,536][00612] Updated weights for policy 1, policy_version 33480 (0.0008) [2023-10-08 05:14:24,914][00612] Updated weights for policy 1, policy_version 33490 (0.0007) [2023-10-08 05:14:25,290][00612] Updated weights for policy 1, policy_version 33500 (0.0008) [2023-10-08 05:14:26,894][00611] Updated weights for policy 0, policy_version 33322 (0.0007) [2023-10-08 05:14:27,278][00611] Updated weights for policy 0, policy_version 33332 (0.0007) [2023-10-08 05:14:27,637][00611] Updated weights for policy 0, policy_version 33342 (0.0010) [2023-10-08 05:14:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 68452352. Throughput: 0: 1823.8, 1: 1852.1. Samples: 17120456. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-08 05:14:28,754][130385] Avg episode reward: [(0, '55.860'), (1, '53.210')] [2023-10-08 05:14:29,010][00612] Updated weights for policy 1, policy_version 33510 (0.0010) [2023-10-08 05:14:29,377][00612] Updated weights for policy 1, policy_version 33520 (0.0009) [2023-10-08 05:14:29,751][00612] Updated weights for policy 1, policy_version 33530 (0.0008) [2023-10-08 05:14:31,386][00611] Updated weights for policy 0, policy_version 33352 (0.0008) [2023-10-08 05:14:31,759][00611] Updated weights for policy 0, policy_version 33362 (0.0008) [2023-10-08 05:14:32,124][00611] Updated weights for policy 0, policy_version 33372 (0.0007) [2023-10-08 05:14:33,379][00612] Updated weights for policy 1, policy_version 33540 (0.0008) [2023-10-08 05:14:33,748][00612] Updated weights for policy 1, policy_version 33550 (0.0007) [2023-10-08 05:14:33,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 68517888. Throughput: 0: 1821.2, 1: 1846.8. Samples: 17142554. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-08 05:14:33,755][130385] Avg episode reward: [(0, '55.110'), (1, '54.540')] [2023-10-08 05:14:34,127][00612] Updated weights for policy 1, policy_version 33560 (0.0010) [2023-10-08 05:14:35,742][00611] Updated weights for policy 0, policy_version 33382 (0.0010) [2023-10-08 05:14:36,114][00611] Updated weights for policy 0, policy_version 33392 (0.0008) [2023-10-08 05:14:36,487][00611] Updated weights for policy 0, policy_version 33402 (0.0008) [2023-10-08 05:14:37,790][00612] Updated weights for policy 1, policy_version 33570 (0.0009) [2023-10-08 05:14:38,161][00612] Updated weights for policy 1, policy_version 33580 (0.0011) [2023-10-08 05:14:38,531][00612] Updated weights for policy 1, policy_version 33590 (0.0010) [2023-10-08 05:14:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 68583424. Throughput: 0: 1816.1, 1: 1849.0. Samples: 17153260. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-08 05:14:38,754][130385] Avg episode reward: [(0, '56.880'), (1, '55.130')] [2023-10-08 05:14:38,901][00612] Updated weights for policy 1, policy_version 33600 (0.0009) [2023-10-08 05:14:40,246][00611] Updated weights for policy 0, policy_version 33412 (0.0010) [2023-10-08 05:14:40,610][00611] Updated weights for policy 0, policy_version 33422 (0.0007) [2023-10-08 05:14:40,989][00611] Updated weights for policy 0, policy_version 33432 (0.0008) [2023-10-08 05:14:42,467][00612] Updated weights for policy 1, policy_version 33610 (0.0011) [2023-10-08 05:14:42,838][00612] Updated weights for policy 1, policy_version 33620 (0.0010) [2023-10-08 05:14:43,211][00612] Updated weights for policy 1, policy_version 33630 (0.0007) [2023-10-08 05:14:43,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 68681728. Throughput: 0: 1819.1, 1: 1851.9. Samples: 17175440. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-08 05:14:43,754][130385] Avg episode reward: [(0, '52.720'), (1, '58.290')] [2023-10-08 05:14:44,635][00611] Updated weights for policy 0, policy_version 33442 (0.0007) [2023-10-08 05:14:44,997][00611] Updated weights for policy 0, policy_version 33452 (0.0007) [2023-10-08 05:14:45,373][00611] Updated weights for policy 0, policy_version 33462 (0.0008) [2023-10-08 05:14:45,741][00611] Updated weights for policy 0, policy_version 33472 (0.0008) [2023-10-08 05:14:46,756][00612] Updated weights for policy 1, policy_version 33640 (0.0008) [2023-10-08 05:14:47,132][00612] Updated weights for policy 1, policy_version 33650 (0.0010) [2023-10-08 05:14:47,490][00612] Updated weights for policy 1, policy_version 33660 (0.0010) [2023-10-08 05:14:48,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 68747264. Throughput: 0: 1822.2, 1: 1847.9. Samples: 17197490. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-08 05:14:48,754][130385] Avg episode reward: [(0, '53.350'), (1, '57.770')] [2023-10-08 05:14:49,357][00611] Updated weights for policy 0, policy_version 33482 (0.0008) [2023-10-08 05:14:49,732][00611] Updated weights for policy 0, policy_version 33492 (0.0008) [2023-10-08 05:14:50,104][00611] Updated weights for policy 0, policy_version 33502 (0.0011) [2023-10-08 05:14:51,070][00612] Updated weights for policy 1, policy_version 33670 (0.0008) [2023-10-08 05:14:51,445][00612] Updated weights for policy 1, policy_version 33680 (0.0008) [2023-10-08 05:14:51,804][00612] Updated weights for policy 1, policy_version 33690 (0.0008) [2023-10-08 05:14:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 68812800. Throughput: 0: 1821.9, 1: 1851.4. Samples: 17208712. Policy #0 lag: (min: 17.0, avg: 26.0, max: 49.0) [2023-10-08 05:14:53,754][130385] Avg episode reward: [(0, '52.200'), (1, '60.070')] [2023-10-08 05:14:53,847][00611] Updated weights for policy 0, policy_version 33512 (0.0009) [2023-10-08 05:14:54,226][00611] Updated weights for policy 0, policy_version 33522 (0.0007) [2023-10-08 05:14:54,600][00611] Updated weights for policy 0, policy_version 33532 (0.0007) [2023-10-08 05:14:55,488][00612] Updated weights for policy 1, policy_version 33700 (0.0007) [2023-10-08 05:14:55,861][00612] Updated weights for policy 1, policy_version 33710 (0.0008) [2023-10-08 05:14:56,223][00612] Updated weights for policy 1, policy_version 33720 (0.0008) [2023-10-08 05:14:58,098][00611] Updated weights for policy 0, policy_version 33542 (0.0008) [2023-10-08 05:14:58,471][00611] Updated weights for policy 0, policy_version 33552 (0.0008) [2023-10-08 05:14:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 68878336. Throughput: 0: 1834.3, 1: 1852.5. Samples: 17230848. Policy #0 lag: (min: 17.0, avg: 26.0, max: 49.0) [2023-10-08 05:14:58,754][130385] Avg episode reward: [(0, '53.610'), (1, '61.430')] [2023-10-08 05:14:58,838][00611] Updated weights for policy 0, policy_version 33562 (0.0008) [2023-10-08 05:14:59,895][00612] Updated weights for policy 1, policy_version 33730 (0.0007) [2023-10-08 05:15:00,300][00612] Updated weights for policy 1, policy_version 33740 (0.0011) [2023-10-08 05:15:00,672][00612] Updated weights for policy 1, policy_version 33750 (0.0010) [2023-10-08 05:15:01,038][00612] Updated weights for policy 1, policy_version 33760 (0.0009) [2023-10-08 05:15:02,412][00611] Updated weights for policy 0, policy_version 33572 (0.0008) [2023-10-08 05:15:02,784][00611] Updated weights for policy 0, policy_version 33582 (0.0009) [2023-10-08 05:15:03,160][00611] Updated weights for policy 0, policy_version 33592 (0.0009) [2023-10-08 05:15:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 68976640. Throughput: 0: 1832.0, 1: 1852.3. Samples: 17252988. Policy #0 lag: (min: 17.0, avg: 26.0, max: 49.0) [2023-10-08 05:15:03,754][130385] Avg episode reward: [(0, '53.140'), (1, '60.150')] [2023-10-08 05:15:04,562][00612] Updated weights for policy 1, policy_version 33770 (0.0009) [2023-10-08 05:15:04,927][00612] Updated weights for policy 1, policy_version 33780 (0.0011) [2023-10-08 05:15:05,299][00612] Updated weights for policy 1, policy_version 33790 (0.0011) [2023-10-08 05:15:06,896][00611] Updated weights for policy 0, policy_version 33602 (0.0007) [2023-10-08 05:15:07,263][00611] Updated weights for policy 0, policy_version 33612 (0.0008) [2023-10-08 05:15:07,638][00611] Updated weights for policy 0, policy_version 33622 (0.0008) [2023-10-08 05:15:08,003][00611] Updated weights for policy 0, policy_version 33632 (0.0007) [2023-10-08 05:15:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.5). Total num frames: 69042176. Throughput: 0: 1838.8, 1: 1849.1. Samples: 17263876. Policy #0 lag: (min: 17.0, avg: 26.0, max: 49.0) [2023-10-08 05:15:08,755][130385] Avg episode reward: [(0, '54.750'), (1, '62.040')] [2023-10-08 05:15:08,960][00612] Updated weights for policy 1, policy_version 33800 (0.0009) [2023-10-08 05:15:09,320][00612] Updated weights for policy 1, policy_version 33810 (0.0007) [2023-10-08 05:15:09,697][00612] Updated weights for policy 1, policy_version 33820 (0.0009) [2023-10-08 05:15:11,666][00611] Updated weights for policy 0, policy_version 33642 (0.0007) [2023-10-08 05:15:12,042][00611] Updated weights for policy 0, policy_version 33652 (0.0007) [2023-10-08 05:15:12,412][00611] Updated weights for policy 0, policy_version 33662 (0.0008) [2023-10-08 05:15:13,278][00612] Updated weights for policy 1, policy_version 33830 (0.0008) [2023-10-08 05:15:13,651][00612] Updated weights for policy 1, policy_version 33840 (0.0007) [2023-10-08 05:15:13,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69107712. Throughput: 0: 1833.0, 1: 1852.4. Samples: 17286300. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-10-08 05:15:13,755][130385] Avg episode reward: [(0, '57.880'), (1, '59.240')] [2023-10-08 05:15:14,011][00612] Updated weights for policy 1, policy_version 33850 (0.0008) [2023-10-08 05:15:15,920][00611] Updated weights for policy 0, policy_version 33672 (0.0008) [2023-10-08 05:15:16,297][00611] Updated weights for policy 0, policy_version 33682 (0.0007) [2023-10-08 05:15:16,662][00611] Updated weights for policy 0, policy_version 33692 (0.0008) [2023-10-08 05:15:17,514][00612] Updated weights for policy 1, policy_version 33860 (0.0008) [2023-10-08 05:15:17,888][00612] Updated weights for policy 1, policy_version 33870 (0.0008) [2023-10-08 05:15:18,254][00612] Updated weights for policy 1, policy_version 33880 (0.0011) [2023-10-08 05:15:18,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 69206016. Throughput: 0: 1847.7, 1: 1833.3. Samples: 17308198. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-10-08 05:15:18,755][130385] Avg episode reward: [(0, '60.890'), (1, '57.410')] [2023-10-08 05:15:18,766][00365] Saving new best policy, reward=60.890! [2023-10-08 05:15:20,407][00611] Updated weights for policy 0, policy_version 33702 (0.0007) [2023-10-08 05:15:20,783][00611] Updated weights for policy 0, policy_version 33712 (0.0007) [2023-10-08 05:15:21,161][00611] Updated weights for policy 0, policy_version 33722 (0.0010) [2023-10-08 05:15:21,797][00612] Updated weights for policy 1, policy_version 33890 (0.0008) [2023-10-08 05:15:22,171][00612] Updated weights for policy 1, policy_version 33900 (0.0007) [2023-10-08 05:15:22,541][00612] Updated weights for policy 1, policy_version 33910 (0.0008) [2023-10-08 05:15:22,903][00612] Updated weights for policy 1, policy_version 33920 (0.0010) [2023-10-08 05:15:23,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 69271552. Throughput: 0: 1834.9, 1: 1856.4. Samples: 17319370. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-10-08 05:15:23,754][130385] Avg episode reward: [(0, '61.010'), (1, '53.340')] [2023-10-08 05:15:23,755][00365] Saving new best policy, reward=61.010! [2023-10-08 05:15:24,818][00611] Updated weights for policy 0, policy_version 33732 (0.0008) [2023-10-08 05:15:25,179][00611] Updated weights for policy 0, policy_version 33742 (0.0010) [2023-10-08 05:15:25,564][00611] Updated weights for policy 0, policy_version 33752 (0.0009) [2023-10-08 05:15:26,629][00612] Updated weights for policy 1, policy_version 33930 (0.0008) [2023-10-08 05:15:26,993][00612] Updated weights for policy 1, policy_version 33940 (0.0008) [2023-10-08 05:15:27,359][00612] Updated weights for policy 1, policy_version 33950 (0.0007) [2023-10-08 05:15:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 69337088. Throughput: 0: 1852.7, 1: 1836.6. Samples: 17341462. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-10-08 05:15:28,755][130385] Avg episode reward: [(0, '61.340'), (1, '52.150')] [2023-10-08 05:15:28,756][00365] Saving new best policy, reward=61.340! [2023-10-08 05:15:29,134][00611] Updated weights for policy 0, policy_version 33762 (0.0008) [2023-10-08 05:15:29,505][00611] Updated weights for policy 0, policy_version 33772 (0.0007) [2023-10-08 05:15:29,878][00611] Updated weights for policy 0, policy_version 33782 (0.0008) [2023-10-08 05:15:30,244][00611] Updated weights for policy 0, policy_version 33792 (0.0008) [2023-10-08 05:15:30,934][00612] Updated weights for policy 1, policy_version 33960 (0.0010) [2023-10-08 05:15:31,297][00612] Updated weights for policy 1, policy_version 33970 (0.0008) [2023-10-08 05:15:31,662][00612] Updated weights for policy 1, policy_version 33980 (0.0009) [2023-10-08 05:15:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 69402624. Throughput: 0: 1847.0, 1: 1857.1. Samples: 17364176. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-10-08 05:15:33,754][130385] Avg episode reward: [(0, '58.520'), (1, '53.360')] [2023-10-08 05:15:33,911][00611] Updated weights for policy 0, policy_version 33802 (0.0009) [2023-10-08 05:15:34,284][00611] Updated weights for policy 0, policy_version 33812 (0.0009) [2023-10-08 05:15:34,660][00611] Updated weights for policy 0, policy_version 33822 (0.0009) [2023-10-08 05:15:35,278][00612] Updated weights for policy 1, policy_version 33990 (0.0008) [2023-10-08 05:15:35,637][00612] Updated weights for policy 1, policy_version 34000 (0.0007) [2023-10-08 05:15:36,007][00612] Updated weights for policy 1, policy_version 34010 (0.0009) [2023-10-08 05:15:38,392][00611] Updated weights for policy 0, policy_version 33832 (0.0009) [2023-10-08 05:15:38,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69468160. Throughput: 0: 1846.8, 1: 1831.6. Samples: 17374238. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-10-08 05:15:38,754][130385] Avg episode reward: [(0, '59.270'), (1, '52.040')] [2023-10-08 05:15:38,774][00611] Updated weights for policy 0, policy_version 33842 (0.0009) [2023-10-08 05:15:39,133][00611] Updated weights for policy 0, policy_version 33852 (0.0009) [2023-10-08 05:15:39,836][00612] Updated weights for policy 1, policy_version 34020 (0.0009) [2023-10-08 05:15:40,203][00612] Updated weights for policy 1, policy_version 34030 (0.0008) [2023-10-08 05:15:40,576][00612] Updated weights for policy 1, policy_version 34040 (0.0010) [2023-10-08 05:15:42,838][00611] Updated weights for policy 0, policy_version 33862 (0.0007) [2023-10-08 05:15:43,229][00611] Updated weights for policy 0, policy_version 33872 (0.0007) [2023-10-08 05:15:43,601][00611] Updated weights for policy 0, policy_version 33882 (0.0007) [2023-10-08 05:15:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 69533696. Throughput: 0: 1844.8, 1: 1850.4. Samples: 17397130. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 05:15:43,754][130385] Avg episode reward: [(0, '62.900'), (1, '50.320')] [2023-10-08 05:15:43,819][00365] Saving new best policy, reward=62.900! [2023-10-08 05:15:44,187][00612] Updated weights for policy 1, policy_version 34050 (0.0009) [2023-10-08 05:15:44,552][00612] Updated weights for policy 1, policy_version 34060 (0.0007) [2023-10-08 05:15:44,932][00612] Updated weights for policy 1, policy_version 34070 (0.0008) [2023-10-08 05:15:45,289][00612] Updated weights for policy 1, policy_version 34080 (0.0011) [2023-10-08 05:15:46,909][00611] Updated weights for policy 0, policy_version 33892 (0.0008) [2023-10-08 05:15:47,290][00611] Updated weights for policy 0, policy_version 33902 (0.0007) [2023-10-08 05:15:47,661][00611] Updated weights for policy 0, policy_version 33912 (0.0008) [2023-10-08 05:15:48,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 69632000. Throughput: 0: 1833.4, 1: 1854.4. Samples: 17418940. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 05:15:48,755][130385] Avg episode reward: [(0, '61.370'), (1, '49.660')] [2023-10-08 05:15:49,081][00612] Updated weights for policy 1, policy_version 34090 (0.0007) [2023-10-08 05:15:49,448][00612] Updated weights for policy 1, policy_version 34100 (0.0010) [2023-10-08 05:15:49,814][00612] Updated weights for policy 1, policy_version 34110 (0.0010) [2023-10-08 05:15:51,371][00611] Updated weights for policy 0, policy_version 33922 (0.0009) [2023-10-08 05:15:51,737][00611] Updated weights for policy 0, policy_version 33932 (0.0008) [2023-10-08 05:15:52,111][00611] Updated weights for policy 0, policy_version 33942 (0.0007) [2023-10-08 05:15:52,479][00611] Updated weights for policy 0, policy_version 33952 (0.0007) [2023-10-08 05:15:53,482][00612] Updated weights for policy 1, policy_version 34120 (0.0009) [2023-10-08 05:15:53,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69697536. Throughput: 0: 1846.1, 1: 1854.4. Samples: 17430400. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 05:15:53,755][130385] Avg episode reward: [(0, '59.000'), (1, '51.130')] [2023-10-08 05:15:53,856][00612] Updated weights for policy 1, policy_version 34130 (0.0009) [2023-10-08 05:15:54,227][00612] Updated weights for policy 1, policy_version 34140 (0.0007) [2023-10-08 05:15:56,077][00611] Updated weights for policy 0, policy_version 33962 (0.0007) [2023-10-08 05:15:56,441][00611] Updated weights for policy 0, policy_version 33972 (0.0009) [2023-10-08 05:15:56,808][00611] Updated weights for policy 0, policy_version 33982 (0.0009) [2023-10-08 05:15:57,847][00612] Updated weights for policy 1, policy_version 34150 (0.0009) [2023-10-08 05:15:58,210][00612] Updated weights for policy 1, policy_version 34160 (0.0008) [2023-10-08 05:15:58,577][00612] Updated weights for policy 1, policy_version 34170 (0.0008) [2023-10-08 05:15:58,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69763072. Throughput: 0: 1839.2, 1: 1843.3. Samples: 17452010. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 05:15:58,754][130385] Avg episode reward: [(0, '60.750'), (1, '49.840')] [2023-10-08 05:16:00,271][00611] Updated weights for policy 0, policy_version 33992 (0.0008) [2023-10-08 05:16:00,646][00611] Updated weights for policy 0, policy_version 34002 (0.0008) [2023-10-08 05:16:01,019][00611] Updated weights for policy 0, policy_version 34012 (0.0009) [2023-10-08 05:16:02,150][00612] Updated weights for policy 1, policy_version 34180 (0.0008) [2023-10-08 05:16:02,519][00612] Updated weights for policy 1, policy_version 34190 (0.0008) [2023-10-08 05:16:02,893][00612] Updated weights for policy 1, policy_version 34200 (0.0008) [2023-10-08 05:16:03,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 69861376. Throughput: 0: 1858.3, 1: 1827.1. Samples: 17474038. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 05:16:03,754][130385] Avg episode reward: [(0, '59.910'), (1, '51.420')] [2023-10-08 05:16:04,687][00611] Updated weights for policy 0, policy_version 34022 (0.0009) [2023-10-08 05:16:05,057][00611] Updated weights for policy 0, policy_version 34032 (0.0007) [2023-10-08 05:16:05,437][00611] Updated weights for policy 0, policy_version 34042 (0.0010) [2023-10-08 05:16:06,749][00612] Updated weights for policy 1, policy_version 34210 (0.0010) [2023-10-08 05:16:07,118][00612] Updated weights for policy 1, policy_version 34220 (0.0007) [2023-10-08 05:16:07,484][00612] Updated weights for policy 1, policy_version 34230 (0.0009) [2023-10-08 05:16:07,856][00612] Updated weights for policy 1, policy_version 34240 (0.0007) [2023-10-08 05:16:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 69926912. Throughput: 0: 1849.8, 1: 1831.7. Samples: 17485036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:16:08,754][130385] Avg episode reward: [(0, '61.760'), (1, '51.420')] [2023-10-08 05:16:09,165][00611] Updated weights for policy 0, policy_version 34052 (0.0008) [2023-10-08 05:16:09,548][00611] Updated weights for policy 0, policy_version 34062 (0.0007) [2023-10-08 05:16:09,917][00611] Updated weights for policy 0, policy_version 34072 (0.0007) [2023-10-08 05:16:11,488][00612] Updated weights for policy 1, policy_version 34250 (0.0009) [2023-10-08 05:16:11,853][00612] Updated weights for policy 1, policy_version 34260 (0.0008) [2023-10-08 05:16:12,224][00612] Updated weights for policy 1, policy_version 34270 (0.0008) [2023-10-08 05:16:13,595][00611] Updated weights for policy 0, policy_version 34082 (0.0009) [2023-10-08 05:16:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 69992448. Throughput: 0: 1845.5, 1: 1820.2. Samples: 17506418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:16:13,754][130385] Avg episode reward: [(0, '61.400'), (1, '54.090')] [2023-10-08 05:16:13,963][00611] Updated weights for policy 0, policy_version 34092 (0.0008) [2023-10-08 05:16:14,334][00611] Updated weights for policy 0, policy_version 34102 (0.0008) [2023-10-08 05:16:14,708][00611] Updated weights for policy 0, policy_version 34112 (0.0009) [2023-10-08 05:16:15,627][00612] Updated weights for policy 1, policy_version 34280 (0.0008) [2023-10-08 05:16:15,991][00612] Updated weights for policy 1, policy_version 34290 (0.0007) [2023-10-08 05:16:16,367][00612] Updated weights for policy 1, policy_version 34300 (0.0007) [2023-10-08 05:16:18,329][00611] Updated weights for policy 0, policy_version 34122 (0.0011) [2023-10-08 05:16:18,706][00611] Updated weights for policy 0, policy_version 34132 (0.0010) [2023-10-08 05:16:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 70057984. Throughput: 0: 1838.6, 1: 1838.7. Samples: 17529656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:16:18,755][130385] Avg episode reward: [(0, '57.530'), (1, '54.890')] [2023-10-08 05:16:18,765][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000034304_35127296.pth... [2023-10-08 05:16:18,804][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000032576_33357824.pth [2023-10-08 05:16:18,809][00425] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p1/milestones/checkpoint_000034304_35127296.pth [2023-10-08 05:16:19,073][00611] Updated weights for policy 0, policy_version 34142 (0.0011) [2023-10-08 05:16:19,140][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000034144_34963456.pth... [2023-10-08 05:16:19,169][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000032416_33193984.pth [2023-10-08 05:16:19,173][00365] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p0/milestones/checkpoint_000034144_34963456.pth [2023-10-08 05:16:19,918][00612] Updated weights for policy 1, policy_version 34310 (0.0009) [2023-10-08 05:16:20,294][00612] Updated weights for policy 1, policy_version 34320 (0.0008) [2023-10-08 05:16:20,665][00612] Updated weights for policy 1, policy_version 34330 (0.0008) [2023-10-08 05:16:22,752][00611] Updated weights for policy 0, policy_version 34152 (0.0011) [2023-10-08 05:16:23,123][00611] Updated weights for policy 0, policy_version 34162 (0.0008) [2023-10-08 05:16:23,503][00611] Updated weights for policy 0, policy_version 34172 (0.0007) [2023-10-08 05:16:23,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 70156288. Throughput: 0: 1848.2, 1: 1834.6. Samples: 17539964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:16:23,755][130385] Avg episode reward: [(0, '60.260'), (1, '55.840')] [2023-10-08 05:16:24,372][00612] Updated weights for policy 1, policy_version 34340 (0.0008) [2023-10-08 05:16:24,742][00612] Updated weights for policy 1, policy_version 34350 (0.0009) [2023-10-08 05:16:25,102][00612] Updated weights for policy 1, policy_version 34360 (0.0007) [2023-10-08 05:16:27,142][00611] Updated weights for policy 0, policy_version 34182 (0.0007) [2023-10-08 05:16:27,505][00611] Updated weights for policy 0, policy_version 34192 (0.0007) [2023-10-08 05:16:27,886][00611] Updated weights for policy 0, policy_version 34202 (0.0007) [2023-10-08 05:16:28,532][00612] Updated weights for policy 1, policy_version 34370 (0.0010) [2023-10-08 05:16:28,754][130385] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 70221824. Throughput: 0: 1838.5, 1: 1840.2. Samples: 17562672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:16:28,754][130385] Avg episode reward: [(0, '57.680'), (1, '57.990')] [2023-10-08 05:16:28,900][00612] Updated weights for policy 1, policy_version 34380 (0.0010) [2023-10-08 05:16:29,267][00612] Updated weights for policy 1, policy_version 34390 (0.0010) [2023-10-08 05:16:29,641][00612] Updated weights for policy 1, policy_version 34400 (0.0012) [2023-10-08 05:16:31,622][00611] Updated weights for policy 0, policy_version 34212 (0.0009) [2023-10-08 05:16:32,017][00611] Updated weights for policy 0, policy_version 34222 (0.0009) [2023-10-08 05:16:32,388][00611] Updated weights for policy 0, policy_version 34232 (0.0009) [2023-10-08 05:16:33,449][00612] Updated weights for policy 1, policy_version 34410 (0.0008) [2023-10-08 05:16:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 70287360. Throughput: 0: 1837.8, 1: 1842.0. Samples: 17584530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:16:33,755][130385] Avg episode reward: [(0, '55.460'), (1, '55.970')] [2023-10-08 05:16:33,817][00612] Updated weights for policy 1, policy_version 34420 (0.0009) [2023-10-08 05:16:34,185][00612] Updated weights for policy 1, policy_version 34430 (0.0011) [2023-10-08 05:16:36,153][00611] Updated weights for policy 0, policy_version 34242 (0.0010) [2023-10-08 05:16:36,520][00611] Updated weights for policy 0, policy_version 34252 (0.0009) [2023-10-08 05:16:36,891][00611] Updated weights for policy 0, policy_version 34262 (0.0008) [2023-10-08 05:16:37,259][00611] Updated weights for policy 0, policy_version 34272 (0.0007) [2023-10-08 05:16:37,836][00612] Updated weights for policy 1, policy_version 34440 (0.0007) [2023-10-08 05:16:38,202][00612] Updated weights for policy 1, policy_version 34450 (0.0009) [2023-10-08 05:16:38,583][00612] Updated weights for policy 1, policy_version 34460 (0.0010) [2023-10-08 05:16:38,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 70385664. Throughput: 0: 1827.4, 1: 1848.0. Samples: 17595792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:16:38,754][130385] Avg episode reward: [(0, '57.010'), (1, '58.600')] [2023-10-08 05:16:40,935][00611] Updated weights for policy 0, policy_version 34282 (0.0010) [2023-10-08 05:16:41,311][00611] Updated weights for policy 0, policy_version 34292 (0.0010) [2023-10-08 05:16:41,679][00611] Updated weights for policy 0, policy_version 34302 (0.0007) [2023-10-08 05:16:42,368][00612] Updated weights for policy 1, policy_version 34470 (0.0009) [2023-10-08 05:16:42,745][00612] Updated weights for policy 1, policy_version 34480 (0.0009) [2023-10-08 05:16:43,117][00612] Updated weights for policy 1, policy_version 34490 (0.0007) [2023-10-08 05:16:43,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 70451200. Throughput: 0: 1821.6, 1: 1845.0. Samples: 17617008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:16:43,755][130385] Avg episode reward: [(0, '56.620'), (1, '61.540')] [2023-10-08 05:16:45,402][00611] Updated weights for policy 0, policy_version 34312 (0.0008) [2023-10-08 05:16:45,771][00611] Updated weights for policy 0, policy_version 34322 (0.0008) [2023-10-08 05:16:46,149][00611] Updated weights for policy 0, policy_version 34332 (0.0008) [2023-10-08 05:16:46,656][00612] Updated weights for policy 1, policy_version 34500 (0.0008) [2023-10-08 05:16:47,034][00612] Updated weights for policy 1, policy_version 34510 (0.0007) [2023-10-08 05:16:47,401][00612] Updated weights for policy 1, policy_version 34520 (0.0007) [2023-10-08 05:16:48,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 70516736. Throughput: 0: 1807.3, 1: 1851.2. Samples: 17638672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:16:48,755][130385] Avg episode reward: [(0, '55.320'), (1, '60.310')] [2023-10-08 05:16:49,855][00611] Updated weights for policy 0, policy_version 34342 (0.0007) [2023-10-08 05:16:50,227][00611] Updated weights for policy 0, policy_version 34352 (0.0008) [2023-10-08 05:16:50,595][00611] Updated weights for policy 0, policy_version 34362 (0.0008) [2023-10-08 05:16:51,003][00612] Updated weights for policy 1, policy_version 34530 (0.0012) [2023-10-08 05:16:51,364][00612] Updated weights for policy 1, policy_version 34540 (0.0010) [2023-10-08 05:16:51,728][00612] Updated weights for policy 1, policy_version 34550 (0.0007) [2023-10-08 05:16:52,093][00612] Updated weights for policy 1, policy_version 34560 (0.0008) [2023-10-08 05:16:53,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 70582272. Throughput: 0: 1810.3, 1: 1850.4. Samples: 17649766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:16:53,754][130385] Avg episode reward: [(0, '53.280'), (1, '58.170')] [2023-10-08 05:16:54,306][00611] Updated weights for policy 0, policy_version 34372 (0.0008) [2023-10-08 05:16:54,675][00611] Updated weights for policy 0, policy_version 34382 (0.0007) [2023-10-08 05:16:55,049][00611] Updated weights for policy 0, policy_version 34392 (0.0010) [2023-10-08 05:16:55,753][00612] Updated weights for policy 1, policy_version 34570 (0.0009) [2023-10-08 05:16:56,125][00612] Updated weights for policy 1, policy_version 34580 (0.0008) [2023-10-08 05:16:56,495][00612] Updated weights for policy 1, policy_version 34590 (0.0010) [2023-10-08 05:16:58,703][00611] Updated weights for policy 0, policy_version 34402 (0.0010) [2023-10-08 05:16:58,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 70647808. Throughput: 0: 1813.6, 1: 1856.9. Samples: 17671592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:16:58,755][130385] Avg episode reward: [(0, '52.820'), (1, '57.940')] [2023-10-08 05:16:59,078][00611] Updated weights for policy 0, policy_version 34412 (0.0008) [2023-10-08 05:16:59,456][00611] Updated weights for policy 0, policy_version 34422 (0.0008) [2023-10-08 05:16:59,820][00611] Updated weights for policy 0, policy_version 34432 (0.0009) [2023-10-08 05:17:00,178][00612] Updated weights for policy 1, policy_version 34600 (0.0010) [2023-10-08 05:17:00,547][00612] Updated weights for policy 1, policy_version 34610 (0.0009) [2023-10-08 05:17:00,916][00612] Updated weights for policy 1, policy_version 34620 (0.0009) [2023-10-08 05:17:03,381][00611] Updated weights for policy 0, policy_version 34442 (0.0009) [2023-10-08 05:17:03,752][00611] Updated weights for policy 0, policy_version 34452 (0.0007) [2023-10-08 05:17:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 70713344. Throughput: 0: 1814.4, 1: 1854.8. Samples: 17694766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:17:03,754][130385] Avg episode reward: [(0, '53.110'), (1, '56.590')] [2023-10-08 05:17:04,121][00611] Updated weights for policy 0, policy_version 34462 (0.0007) [2023-10-08 05:17:04,536][00612] Updated weights for policy 1, policy_version 34630 (0.0008) [2023-10-08 05:17:04,898][00612] Updated weights for policy 1, policy_version 34640 (0.0008) [2023-10-08 05:17:05,270][00612] Updated weights for policy 1, policy_version 34650 (0.0008) [2023-10-08 05:17:07,728][00611] Updated weights for policy 0, policy_version 34472 (0.0008) [2023-10-08 05:17:08,090][00611] Updated weights for policy 0, policy_version 34482 (0.0008) [2023-10-08 05:17:08,469][00611] Updated weights for policy 0, policy_version 34492 (0.0008) [2023-10-08 05:17:08,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 70811648. Throughput: 0: 1813.5, 1: 1855.9. Samples: 17705086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:17:08,755][130385] Avg episode reward: [(0, '53.430'), (1, '57.860')] [2023-10-08 05:17:08,776][00612] Updated weights for policy 1, policy_version 34660 (0.0010) [2023-10-08 05:17:09,142][00612] Updated weights for policy 1, policy_version 34670 (0.0008) [2023-10-08 05:17:09,520][00612] Updated weights for policy 1, policy_version 34680 (0.0010) [2023-10-08 05:17:12,129][00611] Updated weights for policy 0, policy_version 34502 (0.0008) [2023-10-08 05:17:12,507][00611] Updated weights for policy 0, policy_version 34512 (0.0010) [2023-10-08 05:17:12,872][00611] Updated weights for policy 0, policy_version 34522 (0.0007) [2023-10-08 05:17:13,017][00612] Updated weights for policy 1, policy_version 34690 (0.0011) [2023-10-08 05:17:13,390][00612] Updated weights for policy 1, policy_version 34700 (0.0008) [2023-10-08 05:17:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 70877184. Throughput: 0: 1812.0, 1: 1858.6. Samples: 17727846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:17:13,754][130385] Avg episode reward: [(0, '52.780'), (1, '56.620')] [2023-10-08 05:17:13,761][00612] Updated weights for policy 1, policy_version 34710 (0.0008) [2023-10-08 05:17:14,120][00612] Updated weights for policy 1, policy_version 34720 (0.0010) [2023-10-08 05:17:16,572][00611] Updated weights for policy 0, policy_version 34532 (0.0007) [2023-10-08 05:17:16,956][00611] Updated weights for policy 0, policy_version 34542 (0.0008) [2023-10-08 05:17:17,327][00611] Updated weights for policy 0, policy_version 34552 (0.0009) [2023-10-08 05:17:17,739][00612] Updated weights for policy 1, policy_version 34730 (0.0009) [2023-10-08 05:17:18,111][00612] Updated weights for policy 1, policy_version 34740 (0.0009) [2023-10-08 05:17:18,470][00612] Updated weights for policy 1, policy_version 34750 (0.0007) [2023-10-08 05:17:18,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 70975488. Throughput: 0: 1812.8, 1: 1830.8. Samples: 17748490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:17:18,754][130385] Avg episode reward: [(0, '54.640'), (1, '54.720')] [2023-10-08 05:17:21,106][00611] Updated weights for policy 0, policy_version 34562 (0.0009) [2023-10-08 05:17:21,484][00611] Updated weights for policy 0, policy_version 34572 (0.0007) [2023-10-08 05:17:21,859][00611] Updated weights for policy 0, policy_version 34582 (0.0008) [2023-10-08 05:17:22,221][00611] Updated weights for policy 0, policy_version 34592 (0.0008) [2023-10-08 05:17:22,257][00612] Updated weights for policy 1, policy_version 34760 (0.0008) [2023-10-08 05:17:22,630][00612] Updated weights for policy 1, policy_version 34770 (0.0008) [2023-10-08 05:17:23,001][00612] Updated weights for policy 1, policy_version 34780 (0.0007) [2023-10-08 05:17:23,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 71041024. Throughput: 0: 1818.7, 1: 1850.0. Samples: 17760882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:17:23,754][130385] Avg episode reward: [(0, '51.480'), (1, '51.610')] [2023-10-08 05:17:25,909][00611] Updated weights for policy 0, policy_version 34602 (0.0007) [2023-10-08 05:17:26,274][00611] Updated weights for policy 0, policy_version 34612 (0.0007) [2023-10-08 05:17:26,641][00612] Updated weights for policy 1, policy_version 34790 (0.0009) [2023-10-08 05:17:26,643][00611] Updated weights for policy 0, policy_version 34622 (0.0007) [2023-10-08 05:17:27,006][00612] Updated weights for policy 1, policy_version 34800 (0.0008) [2023-10-08 05:17:27,375][00612] Updated weights for policy 1, policy_version 34810 (0.0008) [2023-10-08 05:17:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 71106560. Throughput: 0: 1824.3, 1: 1830.6. Samples: 17781478. Policy #0 lag: (min: 18.0, avg: 21.2, max: 50.0) [2023-10-08 05:17:28,754][130385] Avg episode reward: [(0, '49.070'), (1, '51.680')] [2023-10-08 05:17:30,142][00611] Updated weights for policy 0, policy_version 34632 (0.0010) [2023-10-08 05:17:30,499][00611] Updated weights for policy 0, policy_version 34642 (0.0011) [2023-10-08 05:17:30,874][00611] Updated weights for policy 0, policy_version 34652 (0.0008) [2023-10-08 05:17:31,012][00612] Updated weights for policy 1, policy_version 34820 (0.0007) [2023-10-08 05:17:31,381][00612] Updated weights for policy 1, policy_version 34830 (0.0008) [2023-10-08 05:17:31,749][00612] Updated weights for policy 1, policy_version 34840 (0.0010) [2023-10-08 05:17:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 71172096. Throughput: 0: 1838.5, 1: 1849.3. Samples: 17804624. Policy #0 lag: (min: 18.0, avg: 21.2, max: 50.0) [2023-10-08 05:17:33,754][130385] Avg episode reward: [(0, '52.380'), (1, '50.990')] [2023-10-08 05:17:34,344][00611] Updated weights for policy 0, policy_version 34662 (0.0008) [2023-10-08 05:17:34,726][00611] Updated weights for policy 0, policy_version 34672 (0.0008) [2023-10-08 05:17:35,100][00611] Updated weights for policy 0, policy_version 34682 (0.0009) [2023-10-08 05:17:35,450][00612] Updated weights for policy 1, policy_version 34850 (0.0009) [2023-10-08 05:17:35,810][00612] Updated weights for policy 1, policy_version 34860 (0.0007) [2023-10-08 05:17:36,185][00612] Updated weights for policy 1, policy_version 34870 (0.0007) [2023-10-08 05:17:36,546][00612] Updated weights for policy 1, policy_version 34880 (0.0008) [2023-10-08 05:17:38,707][00611] Updated weights for policy 0, policy_version 34692 (0.0009) [2023-10-08 05:17:38,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 71237632. Throughput: 0: 1841.5, 1: 1834.1. Samples: 17815166. Policy #0 lag: (min: 18.0, avg: 21.2, max: 50.0) [2023-10-08 05:17:38,754][130385] Avg episode reward: [(0, '46.420'), (1, '54.320')] [2023-10-08 05:17:39,078][00611] Updated weights for policy 0, policy_version 34702 (0.0009) [2023-10-08 05:17:39,450][00611] Updated weights for policy 0, policy_version 34712 (0.0009) [2023-10-08 05:17:40,262][00612] Updated weights for policy 1, policy_version 34890 (0.0009) [2023-10-08 05:17:40,634][00612] Updated weights for policy 1, policy_version 34900 (0.0008) [2023-10-08 05:17:40,997][00612] Updated weights for policy 1, policy_version 34910 (0.0009) [2023-10-08 05:17:43,076][00611] Updated weights for policy 0, policy_version 34722 (0.0008) [2023-10-08 05:17:43,449][00611] Updated weights for policy 0, policy_version 34732 (0.0008) [2023-10-08 05:17:43,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 71303168. Throughput: 0: 1848.1, 1: 1853.0. Samples: 17838142. Policy #0 lag: (min: 18.0, avg: 21.2, max: 50.0) [2023-10-08 05:17:43,755][130385] Avg episode reward: [(0, '45.600'), (1, '56.050')] [2023-10-08 05:17:43,830][00611] Updated weights for policy 0, policy_version 34742 (0.0008) [2023-10-08 05:17:44,193][00611] Updated weights for policy 0, policy_version 34752 (0.0008) [2023-10-08 05:17:44,606][00612] Updated weights for policy 1, policy_version 34920 (0.0008) [2023-10-08 05:17:44,972][00612] Updated weights for policy 1, policy_version 34930 (0.0010) [2023-10-08 05:17:45,339][00612] Updated weights for policy 1, policy_version 34940 (0.0007) [2023-10-08 05:17:47,720][00611] Updated weights for policy 0, policy_version 34762 (0.0008) [2023-10-08 05:17:48,096][00611] Updated weights for policy 0, policy_version 34772 (0.0008) [2023-10-08 05:17:48,473][00611] Updated weights for policy 0, policy_version 34782 (0.0009) [2023-10-08 05:17:48,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 71401472. Throughput: 0: 1831.4, 1: 1848.5. Samples: 17860362. Policy #0 lag: (min: 18.0, avg: 21.2, max: 50.0) [2023-10-08 05:17:48,754][130385] Avg episode reward: [(0, '44.210'), (1, '58.550')] [2023-10-08 05:17:48,858][00612] Updated weights for policy 1, policy_version 34950 (0.0008) [2023-10-08 05:17:49,228][00612] Updated weights for policy 1, policy_version 34960 (0.0008) [2023-10-08 05:17:49,595][00612] Updated weights for policy 1, policy_version 34970 (0.0008) [2023-10-08 05:17:52,192][00611] Updated weights for policy 0, policy_version 34792 (0.0010) [2023-10-08 05:17:52,555][00611] Updated weights for policy 0, policy_version 34802 (0.0008) [2023-10-08 05:17:52,921][00611] Updated weights for policy 0, policy_version 34812 (0.0008) [2023-10-08 05:17:53,124][00612] Updated weights for policy 1, policy_version 34980 (0.0008) [2023-10-08 05:17:53,485][00612] Updated weights for policy 1, policy_version 34990 (0.0008) [2023-10-08 05:17:53,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 71467008. Throughput: 0: 1847.4, 1: 1848.2. Samples: 17871388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:17:53,754][130385] Avg episode reward: [(0, '44.080'), (1, '57.450')] [2023-10-08 05:17:53,860][00612] Updated weights for policy 1, policy_version 35000 (0.0008) [2023-10-08 05:17:56,558][00611] Updated weights for policy 0, policy_version 34822 (0.0007) [2023-10-08 05:17:56,925][00611] Updated weights for policy 0, policy_version 34832 (0.0008) [2023-10-08 05:17:57,295][00611] Updated weights for policy 0, policy_version 34842 (0.0009) [2023-10-08 05:17:57,532][00612] Updated weights for policy 1, policy_version 35010 (0.0011) [2023-10-08 05:17:57,895][00612] Updated weights for policy 1, policy_version 35020 (0.0010) [2023-10-08 05:17:58,258][00612] Updated weights for policy 1, policy_version 35030 (0.0010) [2023-10-08 05:17:58,625][00612] Updated weights for policy 1, policy_version 35040 (0.0009) [2023-10-08 05:17:58,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 71565312. Throughput: 0: 1832.3, 1: 1844.4. Samples: 17893300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:17:58,755][130385] Avg episode reward: [(0, '47.280'), (1, '57.310')] [2023-10-08 05:18:00,913][00611] Updated weights for policy 0, policy_version 34852 (0.0009) [2023-10-08 05:18:01,283][00611] Updated weights for policy 0, policy_version 34862 (0.0011) [2023-10-08 05:18:01,650][00611] Updated weights for policy 0, policy_version 34872 (0.0009) [2023-10-08 05:18:02,225][00612] Updated weights for policy 1, policy_version 35050 (0.0007) [2023-10-08 05:18:02,587][00612] Updated weights for policy 1, policy_version 35060 (0.0008) [2023-10-08 05:18:02,961][00612] Updated weights for policy 1, policy_version 35070 (0.0009) [2023-10-08 05:18:03,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 71630848. Throughput: 0: 1854.2, 1: 1833.4. Samples: 17914432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:18:03,754][130385] Avg episode reward: [(0, '45.350'), (1, '56.880')] [2023-10-08 05:18:05,385][00611] Updated weights for policy 0, policy_version 34882 (0.0008) [2023-10-08 05:18:05,775][00611] Updated weights for policy 0, policy_version 34892 (0.0008) [2023-10-08 05:18:06,146][00611] Updated weights for policy 0, policy_version 34902 (0.0008) [2023-10-08 05:18:06,525][00611] Updated weights for policy 0, policy_version 34912 (0.0008) [2023-10-08 05:18:06,645][00612] Updated weights for policy 1, policy_version 35080 (0.0007) [2023-10-08 05:18:07,029][00612] Updated weights for policy 1, policy_version 35090 (0.0007) [2023-10-08 05:18:07,394][00612] Updated weights for policy 1, policy_version 35100 (0.0007) [2023-10-08 05:18:08,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 71696384. Throughput: 0: 1829.8, 1: 1846.5. Samples: 17926316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:18:08,754][130385] Avg episode reward: [(0, '47.800'), (1, '55.790')] [2023-10-08 05:18:10,235][00611] Updated weights for policy 0, policy_version 34922 (0.0010) [2023-10-08 05:18:10,610][00611] Updated weights for policy 0, policy_version 34932 (0.0009) [2023-10-08 05:18:10,980][00611] Updated weights for policy 0, policy_version 34942 (0.0007) [2023-10-08 05:18:11,089][00612] Updated weights for policy 1, policy_version 35110 (0.0008) [2023-10-08 05:18:11,475][00612] Updated weights for policy 1, policy_version 35120 (0.0008) [2023-10-08 05:18:11,844][00612] Updated weights for policy 1, policy_version 35130 (0.0007) [2023-10-08 05:18:13,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 71761920. Throughput: 0: 1849.7, 1: 1831.4. Samples: 17947126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:18:13,755][130385] Avg episode reward: [(0, '48.800'), (1, '55.490')] [2023-10-08 05:18:14,539][00611] Updated weights for policy 0, policy_version 34952 (0.0008) [2023-10-08 05:18:14,913][00611] Updated weights for policy 0, policy_version 34962 (0.0008) [2023-10-08 05:18:15,272][00611] Updated weights for policy 0, policy_version 34972 (0.0008) [2023-10-08 05:18:15,353][00612] Updated weights for policy 1, policy_version 35140 (0.0007) [2023-10-08 05:18:15,715][00612] Updated weights for policy 1, policy_version 35150 (0.0009) [2023-10-08 05:18:16,095][00612] Updated weights for policy 1, policy_version 35160 (0.0010) [2023-10-08 05:18:18,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 71827456. Throughput: 0: 1836.2, 1: 1844.7. Samples: 17970264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:18:18,754][130385] Avg episode reward: [(0, '47.840'), (1, '58.540')] [2023-10-08 05:18:18,765][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000035168_36012032.pth... [2023-10-08 05:18:18,806][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000033440_34242560.pth [2023-10-08 05:18:19,030][00611] Updated weights for policy 0, policy_version 34982 (0.0010) [2023-10-08 05:18:19,397][00611] Updated weights for policy 0, policy_version 34992 (0.0009) [2023-10-08 05:18:19,743][00612] Updated weights for policy 1, policy_version 35170 (0.0010) [2023-10-08 05:18:19,766][00611] Updated weights for policy 0, policy_version 35002 (0.0010) [2023-10-08 05:18:19,995][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000035008_35848192.pth... [2023-10-08 05:18:20,026][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000033280_34078720.pth [2023-10-08 05:18:20,102][00612] Updated weights for policy 1, policy_version 35180 (0.0008) [2023-10-08 05:18:20,475][00612] Updated weights for policy 1, policy_version 35190 (0.0008) [2023-10-08 05:18:20,842][00612] Updated weights for policy 1, policy_version 35200 (0.0007) [2023-10-08 05:18:23,415][00611] Updated weights for policy 0, policy_version 35012 (0.0009) [2023-10-08 05:18:23,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 71892992. Throughput: 0: 1831.7, 1: 1832.4. Samples: 17980054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:18:23,754][130385] Avg episode reward: [(0, '50.440'), (1, '58.440')] [2023-10-08 05:18:23,789][00611] Updated weights for policy 0, policy_version 35022 (0.0009) [2023-10-08 05:18:24,171][00611] Updated weights for policy 0, policy_version 35032 (0.0008) [2023-10-08 05:18:24,537][00612] Updated weights for policy 1, policy_version 35210 (0.0008) [2023-10-08 05:18:24,912][00612] Updated weights for policy 1, policy_version 35220 (0.0008) [2023-10-08 05:18:25,276][00612] Updated weights for policy 1, policy_version 35230 (0.0010) [2023-10-08 05:18:27,844][00611] Updated weights for policy 0, policy_version 35042 (0.0007) [2023-10-08 05:18:28,216][00611] Updated weights for policy 0, policy_version 35052 (0.0009) [2023-10-08 05:18:28,592][00611] Updated weights for policy 0, policy_version 35062 (0.0008) [2023-10-08 05:18:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 71958528. Throughput: 0: 1826.4, 1: 1838.9. Samples: 18003080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:18:28,754][130385] Avg episode reward: [(0, '52.380'), (1, '52.500')] [2023-10-08 05:18:28,865][00612] Updated weights for policy 1, policy_version 35240 (0.0007) [2023-10-08 05:18:28,965][00611] Updated weights for policy 0, policy_version 35072 (0.0008) [2023-10-08 05:18:29,232][00612] Updated weights for policy 1, policy_version 35250 (0.0008) [2023-10-08 05:18:29,598][00612] Updated weights for policy 1, policy_version 35260 (0.0010) [2023-10-08 05:18:32,759][00611] Updated weights for policy 0, policy_version 35082 (0.0008) [2023-10-08 05:18:33,127][00611] Updated weights for policy 0, policy_version 35092 (0.0008) [2023-10-08 05:18:33,198][00612] Updated weights for policy 1, policy_version 35270 (0.0009) [2023-10-08 05:18:33,503][00611] Updated weights for policy 0, policy_version 35102 (0.0008) [2023-10-08 05:18:33,569][00612] Updated weights for policy 1, policy_version 35280 (0.0008) [2023-10-08 05:18:33,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 72056832. Throughput: 0: 1827.3, 1: 1833.3. Samples: 18025090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:18:33,755][130385] Avg episode reward: [(0, '53.450'), (1, '52.340')] [2023-10-08 05:18:33,932][00612] Updated weights for policy 1, policy_version 35290 (0.0008) [2023-10-08 05:18:37,134][00611] Updated weights for policy 0, policy_version 35112 (0.0011) [2023-10-08 05:18:37,503][00611] Updated weights for policy 0, policy_version 35122 (0.0008) [2023-10-08 05:18:37,612][00612] Updated weights for policy 1, policy_version 35300 (0.0007) [2023-10-08 05:18:37,874][00611] Updated weights for policy 0, policy_version 35132 (0.0007) [2023-10-08 05:18:37,984][00612] Updated weights for policy 1, policy_version 35310 (0.0007) [2023-10-08 05:18:38,347][00612] Updated weights for policy 1, policy_version 35320 (0.0007) [2023-10-08 05:18:38,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 72155136. Throughput: 0: 1828.2, 1: 1833.9. Samples: 18036182. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:18:38,755][130385] Avg episode reward: [(0, '55.240'), (1, '50.490')] [2023-10-08 05:18:41,489][00611] Updated weights for policy 0, policy_version 35142 (0.0010) [2023-10-08 05:18:41,847][00611] Updated weights for policy 0, policy_version 35152 (0.0008) [2023-10-08 05:18:42,086][00612] Updated weights for policy 1, policy_version 35330 (0.0008) [2023-10-08 05:18:42,224][00611] Updated weights for policy 0, policy_version 35162 (0.0009) [2023-10-08 05:18:42,448][00612] Updated weights for policy 1, policy_version 35340 (0.0008) [2023-10-08 05:18:42,825][00612] Updated weights for policy 1, policy_version 35350 (0.0008) [2023-10-08 05:18:43,194][00612] Updated weights for policy 1, policy_version 35360 (0.0011) [2023-10-08 05:18:43,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 72220672. Throughput: 0: 1823.0, 1: 1828.4. Samples: 18057610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:18:43,754][130385] Avg episode reward: [(0, '55.850'), (1, '49.130')] [2023-10-08 05:18:45,975][00611] Updated weights for policy 0, policy_version 35172 (0.0008) [2023-10-08 05:18:46,349][00611] Updated weights for policy 0, policy_version 35182 (0.0007) [2023-10-08 05:18:46,723][00611] Updated weights for policy 0, policy_version 35192 (0.0008) [2023-10-08 05:18:46,942][00612] Updated weights for policy 1, policy_version 35370 (0.0008) [2023-10-08 05:18:47,318][00612] Updated weights for policy 1, policy_version 35380 (0.0010) [2023-10-08 05:18:47,679][00612] Updated weights for policy 1, policy_version 35390 (0.0009) [2023-10-08 05:18:48,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14745.5, 300 sec: 14773.3). Total num frames: 72286208. Throughput: 0: 1821.6, 1: 1832.8. Samples: 18078880. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-08 05:18:48,755][130385] Avg episode reward: [(0, '56.580'), (1, '51.590')] [2023-10-08 05:18:50,459][00611] Updated weights for policy 0, policy_version 35202 (0.0008) [2023-10-08 05:18:50,872][00611] Updated weights for policy 0, policy_version 35212 (0.0007) [2023-10-08 05:18:51,185][00612] Updated weights for policy 1, policy_version 35400 (0.0008) [2023-10-08 05:18:51,242][00611] Updated weights for policy 0, policy_version 35222 (0.0009) [2023-10-08 05:18:51,546][00612] Updated weights for policy 1, policy_version 35410 (0.0007) [2023-10-08 05:18:51,605][00611] Updated weights for policy 0, policy_version 35232 (0.0007) [2023-10-08 05:18:51,919][00612] Updated weights for policy 1, policy_version 35420 (0.0009) [2023-10-08 05:18:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 72351744. Throughput: 0: 1823.0, 1: 1825.9. Samples: 18090516. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-08 05:18:53,754][130385] Avg episode reward: [(0, '57.020'), (1, '51.750')] [2023-10-08 05:18:55,284][00611] Updated weights for policy 0, policy_version 35242 (0.0007) [2023-10-08 05:18:55,531][00612] Updated weights for policy 1, policy_version 35430 (0.0008) [2023-10-08 05:18:55,654][00611] Updated weights for policy 0, policy_version 35252 (0.0008) [2023-10-08 05:18:55,898][00612] Updated weights for policy 1, policy_version 35440 (0.0009) [2023-10-08 05:18:56,021][00611] Updated weights for policy 0, policy_version 35262 (0.0008) [2023-10-08 05:18:56,272][00612] Updated weights for policy 1, policy_version 35450 (0.0007) [2023-10-08 05:18:58,754][130385] Fps is (10 sec: 13107.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 72417280. Throughput: 0: 1820.9, 1: 1843.5. Samples: 18112022. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-08 05:18:58,754][130385] Avg episode reward: [(0, '58.760'), (1, '53.690')] [2023-10-08 05:18:59,560][00611] Updated weights for policy 0, policy_version 35272 (0.0007) [2023-10-08 05:18:59,939][00611] Updated weights for policy 0, policy_version 35282 (0.0009) [2023-10-08 05:19:00,092][00612] Updated weights for policy 1, policy_version 35460 (0.0008) [2023-10-08 05:19:00,303][00611] Updated weights for policy 0, policy_version 35292 (0.0007) [2023-10-08 05:19:00,476][00612] Updated weights for policy 1, policy_version 35470 (0.0008) [2023-10-08 05:19:00,846][00612] Updated weights for policy 1, policy_version 35480 (0.0011) [2023-10-08 05:19:03,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 72482816. Throughput: 0: 1826.6, 1: 1837.7. Samples: 18135160. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-08 05:19:03,755][130385] Avg episode reward: [(0, '62.720'), (1, '55.790')] [2023-10-08 05:19:03,944][00611] Updated weights for policy 0, policy_version 35302 (0.0008) [2023-10-08 05:19:04,317][00611] Updated weights for policy 0, policy_version 35312 (0.0008) [2023-10-08 05:19:04,496][00612] Updated weights for policy 1, policy_version 35490 (0.0009) [2023-10-08 05:19:04,689][00611] Updated weights for policy 0, policy_version 35322 (0.0007) [2023-10-08 05:19:04,870][00612] Updated weights for policy 1, policy_version 35500 (0.0007) [2023-10-08 05:19:05,244][00612] Updated weights for policy 1, policy_version 35510 (0.0009) [2023-10-08 05:19:05,616][00612] Updated weights for policy 1, policy_version 35520 (0.0009) [2023-10-08 05:19:08,348][00611] Updated weights for policy 0, policy_version 35332 (0.0008) [2023-10-08 05:19:08,725][00611] Updated weights for policy 0, policy_version 35342 (0.0007) [2023-10-08 05:19:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 72548352. Throughput: 0: 1829.8, 1: 1835.2. Samples: 18144980. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-08 05:19:08,754][130385] Avg episode reward: [(0, '62.150'), (1, '53.750')] [2023-10-08 05:19:09,094][00611] Updated weights for policy 0, policy_version 35352 (0.0008) [2023-10-08 05:19:09,206][00612] Updated weights for policy 1, policy_version 35530 (0.0007) [2023-10-08 05:19:09,565][00612] Updated weights for policy 1, policy_version 35540 (0.0008) [2023-10-08 05:19:09,942][00612] Updated weights for policy 1, policy_version 35550 (0.0008) [2023-10-08 05:19:12,890][00611] Updated weights for policy 0, policy_version 35362 (0.0008) [2023-10-08 05:19:13,275][00611] Updated weights for policy 0, policy_version 35372 (0.0010) [2023-10-08 05:19:13,637][00611] Updated weights for policy 0, policy_version 35382 (0.0008) [2023-10-08 05:19:13,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 72613888. Throughput: 0: 1820.1, 1: 1835.6. Samples: 18167586. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-08 05:19:13,754][130385] Avg episode reward: [(0, '62.870'), (1, '52.920')] [2023-10-08 05:19:13,789][00612] Updated weights for policy 1, policy_version 35560 (0.0008) [2023-10-08 05:19:14,016][00611] Updated weights for policy 0, policy_version 35392 (0.0009) [2023-10-08 05:19:14,155][00612] Updated weights for policy 1, policy_version 35570 (0.0009) [2023-10-08 05:19:14,520][00612] Updated weights for policy 1, policy_version 35580 (0.0008) [2023-10-08 05:19:17,575][00611] Updated weights for policy 0, policy_version 35402 (0.0008) [2023-10-08 05:19:17,958][00611] Updated weights for policy 0, policy_version 35412 (0.0008) [2023-10-08 05:19:18,219][00612] Updated weights for policy 1, policy_version 35590 (0.0008) [2023-10-08 05:19:18,326][00611] Updated weights for policy 0, policy_version 35422 (0.0009) [2023-10-08 05:19:18,591][00612] Updated weights for policy 1, policy_version 35600 (0.0009) [2023-10-08 05:19:18,754][130385] Fps is (10 sec: 16383.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 72712192. Throughput: 0: 1814.7, 1: 1832.4. Samples: 18189210. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 05:19:18,755][130385] Avg episode reward: [(0, '62.480'), (1, '53.820')] [2023-10-08 05:19:18,958][00612] Updated weights for policy 1, policy_version 35610 (0.0008) [2023-10-08 05:19:22,192][00611] Updated weights for policy 0, policy_version 35432 (0.0010) [2023-10-08 05:19:22,549][00611] Updated weights for policy 0, policy_version 35442 (0.0010) [2023-10-08 05:19:22,641][00612] Updated weights for policy 1, policy_version 35620 (0.0009) [2023-10-08 05:19:22,924][00611] Updated weights for policy 0, policy_version 35452 (0.0008) [2023-10-08 05:19:23,016][00612] Updated weights for policy 1, policy_version 35630 (0.0007) [2023-10-08 05:19:23,386][00612] Updated weights for policy 1, policy_version 35640 (0.0008) [2023-10-08 05:19:23,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 72810496. Throughput: 0: 1815.4, 1: 1834.5. Samples: 18200430. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 05:19:23,755][130385] Avg episode reward: [(0, '65.640'), (1, '57.610')] [2023-10-08 05:19:23,756][00365] Saving new best policy, reward=65.640! [2023-10-08 05:19:26,597][00611] Updated weights for policy 0, policy_version 35462 (0.0011) [2023-10-08 05:19:26,975][00611] Updated weights for policy 0, policy_version 35472 (0.0009) [2023-10-08 05:19:27,146][00612] Updated weights for policy 1, policy_version 35650 (0.0009) [2023-10-08 05:19:27,354][00611] Updated weights for policy 0, policy_version 35482 (0.0007) [2023-10-08 05:19:27,508][00612] Updated weights for policy 1, policy_version 35660 (0.0009) [2023-10-08 05:19:27,884][00612] Updated weights for policy 1, policy_version 35670 (0.0008) [2023-10-08 05:19:28,248][00612] Updated weights for policy 1, policy_version 35680 (0.0009) [2023-10-08 05:19:28,754][130385] Fps is (10 sec: 16384.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 72876032. Throughput: 0: 1828.3, 1: 1832.5. Samples: 18222344. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 05:19:28,754][130385] Avg episode reward: [(0, '64.350'), (1, '58.300')] [2023-10-08 05:19:30,910][00611] Updated weights for policy 0, policy_version 35492 (0.0008) [2023-10-08 05:19:31,270][00611] Updated weights for policy 0, policy_version 35502 (0.0011) [2023-10-08 05:19:31,644][00611] Updated weights for policy 0, policy_version 35512 (0.0007) [2023-10-08 05:19:31,754][00612] Updated weights for policy 1, policy_version 35690 (0.0008) [2023-10-08 05:19:32,128][00612] Updated weights for policy 1, policy_version 35700 (0.0008) [2023-10-08 05:19:32,487][00612] Updated weights for policy 1, policy_version 35710 (0.0008) [2023-10-08 05:19:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 72941568. Throughput: 0: 1829.1, 1: 1838.3. Samples: 18243912. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 05:19:33,754][130385] Avg episode reward: [(0, '63.200'), (1, '58.670')] [2023-10-08 05:19:35,374][00611] Updated weights for policy 0, policy_version 35522 (0.0008) [2023-10-08 05:19:35,786][00611] Updated weights for policy 0, policy_version 35532 (0.0008) [2023-10-08 05:19:36,073][00612] Updated weights for policy 1, policy_version 35720 (0.0007) [2023-10-08 05:19:36,159][00611] Updated weights for policy 0, policy_version 35542 (0.0007) [2023-10-08 05:19:36,444][00612] Updated weights for policy 1, policy_version 35730 (0.0007) [2023-10-08 05:19:36,527][00611] Updated weights for policy 0, policy_version 35552 (0.0007) [2023-10-08 05:19:36,803][00612] Updated weights for policy 1, policy_version 35740 (0.0009) [2023-10-08 05:19:38,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 73007104. Throughput: 0: 1829.6, 1: 1832.3. Samples: 18255300. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 05:19:38,755][130385] Avg episode reward: [(0, '63.720'), (1, '61.030')] [2023-10-08 05:19:40,252][00611] Updated weights for policy 0, policy_version 35562 (0.0010) [2023-10-08 05:19:40,551][00612] Updated weights for policy 1, policy_version 35750 (0.0010) [2023-10-08 05:19:40,622][00611] Updated weights for policy 0, policy_version 35572 (0.0009) [2023-10-08 05:19:40,910][00612] Updated weights for policy 1, policy_version 35760 (0.0008) [2023-10-08 05:19:40,993][00611] Updated weights for policy 0, policy_version 35582 (0.0010) [2023-10-08 05:19:41,282][00612] Updated weights for policy 1, policy_version 35770 (0.0009) [2023-10-08 05:19:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 73072640. Throughput: 0: 1828.9, 1: 1827.1. Samples: 18276542. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:19:43,754][130385] Avg episode reward: [(0, '67.460'), (1, '64.380')] [2023-10-08 05:19:43,755][00365] Saving new best policy, reward=67.460! [2023-10-08 05:19:44,641][00611] Updated weights for policy 0, policy_version 35592 (0.0008) [2023-10-08 05:19:44,994][00612] Updated weights for policy 1, policy_version 35780 (0.0007) [2023-10-08 05:19:45,019][00611] Updated weights for policy 0, policy_version 35602 (0.0008) [2023-10-08 05:19:45,371][00612] Updated weights for policy 1, policy_version 35790 (0.0008) [2023-10-08 05:19:45,383][00611] Updated weights for policy 0, policy_version 35612 (0.0008) [2023-10-08 05:19:45,741][00612] Updated weights for policy 1, policy_version 35800 (0.0008) [2023-10-08 05:19:48,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.6, 300 sec: 14662.3). Total num frames: 73138176. Throughput: 0: 1823.6, 1: 1833.2. Samples: 18299712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:19:48,754][130385] Avg episode reward: [(0, '62.900'), (1, '61.110')] [2023-10-08 05:19:49,042][00611] Updated weights for policy 0, policy_version 35622 (0.0009) [2023-10-08 05:19:49,390][00612] Updated weights for policy 1, policy_version 35810 (0.0008) [2023-10-08 05:19:49,423][00611] Updated weights for policy 0, policy_version 35632 (0.0009) [2023-10-08 05:19:49,759][00612] Updated weights for policy 1, policy_version 35820 (0.0008) [2023-10-08 05:19:49,789][00611] Updated weights for policy 0, policy_version 35642 (0.0007) [2023-10-08 05:19:50,123][00612] Updated weights for policy 1, policy_version 35830 (0.0009) [2023-10-08 05:19:50,490][00612] Updated weights for policy 1, policy_version 35840 (0.0012) [2023-10-08 05:19:53,450][00611] Updated weights for policy 0, policy_version 35652 (0.0009) [2023-10-08 05:19:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 73203712. Throughput: 0: 1822.7, 1: 1833.1. Samples: 18309492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:19:53,754][130385] Avg episode reward: [(0, '63.370'), (1, '57.350')] [2023-10-08 05:19:53,811][00611] Updated weights for policy 0, policy_version 35662 (0.0008) [2023-10-08 05:19:54,120][00612] Updated weights for policy 1, policy_version 35850 (0.0008) [2023-10-08 05:19:54,180][00611] Updated weights for policy 0, policy_version 35672 (0.0008) [2023-10-08 05:19:54,486][00612] Updated weights for policy 1, policy_version 35860 (0.0008) [2023-10-08 05:19:54,851][00612] Updated weights for policy 1, policy_version 35870 (0.0008) [2023-10-08 05:19:57,752][00611] Updated weights for policy 0, policy_version 35682 (0.0008) [2023-10-08 05:19:58,116][00611] Updated weights for policy 0, policy_version 35692 (0.0008) [2023-10-08 05:19:58,439][00612] Updated weights for policy 1, policy_version 35880 (0.0009) [2023-10-08 05:19:58,487][00611] Updated weights for policy 0, policy_version 35702 (0.0007) [2023-10-08 05:19:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 73269248. Throughput: 0: 1832.9, 1: 1842.1. Samples: 18332962. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:19:58,754][130385] Avg episode reward: [(0, '58.410'), (1, '57.270')] [2023-10-08 05:19:58,806][00612] Updated weights for policy 1, policy_version 35890 (0.0007) [2023-10-08 05:19:58,858][00611] Updated weights for policy 0, policy_version 35712 (0.0008) [2023-10-08 05:19:59,182][00612] Updated weights for policy 1, policy_version 35900 (0.0010) [2023-10-08 05:20:02,444][00611] Updated weights for policy 0, policy_version 35722 (0.0008) [2023-10-08 05:20:02,676][00612] Updated weights for policy 1, policy_version 35910 (0.0009) [2023-10-08 05:20:02,808][00611] Updated weights for policy 0, policy_version 35732 (0.0008) [2023-10-08 05:20:03,031][00612] Updated weights for policy 1, policy_version 35920 (0.0007) [2023-10-08 05:20:03,187][00611] Updated weights for policy 0, policy_version 35742 (0.0010) [2023-10-08 05:20:03,399][00612] Updated weights for policy 1, policy_version 35930 (0.0008) [2023-10-08 05:20:03,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 73400320. Throughput: 0: 1830.9, 1: 1833.8. Samples: 18354124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:20:03,755][130385] Avg episode reward: [(0, '58.170'), (1, '55.890')] [2023-10-08 05:20:06,864][00611] Updated weights for policy 0, policy_version 35752 (0.0010) [2023-10-08 05:20:07,035][00612] Updated weights for policy 1, policy_version 35940 (0.0008) [2023-10-08 05:20:07,236][00611] Updated weights for policy 0, policy_version 35762 (0.0008) [2023-10-08 05:20:07,408][00612] Updated weights for policy 1, policy_version 35950 (0.0009) [2023-10-08 05:20:07,621][00611] Updated weights for policy 0, policy_version 35772 (0.0009) [2023-10-08 05:20:07,769][00612] Updated weights for policy 1, policy_version 35960 (0.0007) [2023-10-08 05:20:08,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 73465856. Throughput: 0: 1839.4, 1: 1850.7. Samples: 18366486. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) [2023-10-08 05:20:08,754][130385] Avg episode reward: [(0, '59.960'), (1, '52.640')] [2023-10-08 05:20:11,253][00611] Updated weights for policy 0, policy_version 35782 (0.0008) [2023-10-08 05:20:11,449][00612] Updated weights for policy 1, policy_version 35970 (0.0007) [2023-10-08 05:20:11,630][00611] Updated weights for policy 0, policy_version 35792 (0.0008) [2023-10-08 05:20:11,812][00612] Updated weights for policy 1, policy_version 35980 (0.0007) [2023-10-08 05:20:11,995][00611] Updated weights for policy 0, policy_version 35802 (0.0007) [2023-10-08 05:20:12,188][00612] Updated weights for policy 1, policy_version 35990 (0.0007) [2023-10-08 05:20:12,548][00612] Updated weights for policy 1, policy_version 36000 (0.0008) [2023-10-08 05:20:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 73531392. Throughput: 0: 1822.6, 1: 1835.5. Samples: 18386958. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) [2023-10-08 05:20:13,754][130385] Avg episode reward: [(0, '59.290'), (1, '55.630')] [2023-10-08 05:20:15,622][00611] Updated weights for policy 0, policy_version 35812 (0.0009) [2023-10-08 05:20:15,987][00611] Updated weights for policy 0, policy_version 35822 (0.0009) [2023-10-08 05:20:16,162][00612] Updated weights for policy 1, policy_version 36010 (0.0008) [2023-10-08 05:20:16,354][00611] Updated weights for policy 0, policy_version 35832 (0.0008) [2023-10-08 05:20:16,534][00612] Updated weights for policy 1, policy_version 36020 (0.0009) [2023-10-08 05:20:16,911][00612] Updated weights for policy 1, policy_version 36030 (0.0008) [2023-10-08 05:20:18,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 73596928. Throughput: 0: 1830.6, 1: 1847.2. Samples: 18409412. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) [2023-10-08 05:20:18,755][130385] Avg episode reward: [(0, '57.970'), (1, '53.900')] [2023-10-08 05:20:18,766][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000035840_36700160.pth... [2023-10-08 05:20:18,766][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000036032_36896768.pth... [2023-10-08 05:20:18,810][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000034144_34963456.pth [2023-10-08 05:20:18,813][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000034304_35127296.pth [2023-10-08 05:20:20,063][00611] Updated weights for policy 0, policy_version 35842 (0.0008) [2023-10-08 05:20:20,434][00611] Updated weights for policy 0, policy_version 35852 (0.0009) [2023-10-08 05:20:20,566][00612] Updated weights for policy 1, policy_version 36040 (0.0007) [2023-10-08 05:20:20,800][00611] Updated weights for policy 0, policy_version 35862 (0.0008) [2023-10-08 05:20:20,931][00612] Updated weights for policy 1, policy_version 36050 (0.0007) [2023-10-08 05:20:21,172][00611] Updated weights for policy 0, policy_version 35872 (0.0008) [2023-10-08 05:20:21,299][00612] Updated weights for policy 1, policy_version 36060 (0.0007) [2023-10-08 05:20:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 73662464. Throughput: 0: 1822.1, 1: 1834.1. Samples: 18419828. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) [2023-10-08 05:20:23,755][130385] Avg episode reward: [(0, '58.370'), (1, '54.380')] [2023-10-08 05:20:24,891][00612] Updated weights for policy 1, policy_version 36070 (0.0007) [2023-10-08 05:20:24,991][00611] Updated weights for policy 0, policy_version 35882 (0.0007) [2023-10-08 05:20:25,247][00612] Updated weights for policy 1, policy_version 36080 (0.0008) [2023-10-08 05:20:25,363][00611] Updated weights for policy 0, policy_version 35892 (0.0008) [2023-10-08 05:20:25,614][00612] Updated weights for policy 1, policy_version 36090 (0.0009) [2023-10-08 05:20:25,733][00611] Updated weights for policy 0, policy_version 35902 (0.0007) [2023-10-08 05:20:28,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 73728000. Throughput: 0: 1831.8, 1: 1852.7. Samples: 18442348. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) [2023-10-08 05:20:28,755][130385] Avg episode reward: [(0, '59.300'), (1, '51.350')] [2023-10-08 05:20:29,331][00612] Updated weights for policy 1, policy_version 36100 (0.0009) [2023-10-08 05:20:29,344][00611] Updated weights for policy 0, policy_version 35912 (0.0008) [2023-10-08 05:20:29,691][00612] Updated weights for policy 1, policy_version 36110 (0.0007) [2023-10-08 05:20:29,719][00611] Updated weights for policy 0, policy_version 35922 (0.0007) [2023-10-08 05:20:30,051][00612] Updated weights for policy 1, policy_version 36120 (0.0007) [2023-10-08 05:20:30,090][00611] Updated weights for policy 0, policy_version 35932 (0.0007) [2023-10-08 05:20:33,712][00612] Updated weights for policy 1, policy_version 36130 (0.0007) [2023-10-08 05:20:33,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 73793536. Throughput: 0: 1825.3, 1: 1851.2. Samples: 18465152. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) [2023-10-08 05:20:33,754][130385] Avg episode reward: [(0, '58.900'), (1, '51.220')] [2023-10-08 05:20:33,847][00611] Updated weights for policy 0, policy_version 35942 (0.0008) [2023-10-08 05:20:34,119][00612] Updated weights for policy 1, policy_version 36140 (0.0008) [2023-10-08 05:20:34,218][00611] Updated weights for policy 0, policy_version 35952 (0.0008) [2023-10-08 05:20:34,488][00612] Updated weights for policy 1, policy_version 36150 (0.0008) [2023-10-08 05:20:34,589][00611] Updated weights for policy 0, policy_version 35962 (0.0008) [2023-10-08 05:20:34,856][00612] Updated weights for policy 1, policy_version 36160 (0.0008) [2023-10-08 05:20:38,195][00611] Updated weights for policy 0, policy_version 35972 (0.0007) [2023-10-08 05:20:38,565][00611] Updated weights for policy 0, policy_version 35982 (0.0007) [2023-10-08 05:20:38,574][00612] Updated weights for policy 1, policy_version 36170 (0.0009) [2023-10-08 05:20:38,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14199.6, 300 sec: 14662.3). Total num frames: 73859072. Throughput: 0: 1824.7, 1: 1846.8. Samples: 18474708. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-08 05:20:38,754][130385] Avg episode reward: [(0, '57.980'), (1, '52.210')] [2023-10-08 05:20:38,944][00612] Updated weights for policy 1, policy_version 36180 (0.0008) [2023-10-08 05:20:38,945][00611] Updated weights for policy 0, policy_version 35992 (0.0010) [2023-10-08 05:20:39,310][00612] Updated weights for policy 1, policy_version 36190 (0.0008) [2023-10-08 05:20:42,600][00611] Updated weights for policy 0, policy_version 36002 (0.0008) [2023-10-08 05:20:42,979][00611] Updated weights for policy 0, policy_version 36012 (0.0007) [2023-10-08 05:20:43,042][00612] Updated weights for policy 1, policy_version 36200 (0.0009) [2023-10-08 05:20:43,352][00611] Updated weights for policy 0, policy_version 36022 (0.0010) [2023-10-08 05:20:43,409][00612] Updated weights for policy 1, policy_version 36210 (0.0009) [2023-10-08 05:20:43,717][00611] Updated weights for policy 0, policy_version 36032 (0.0009) [2023-10-08 05:20:43,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 73957376. Throughput: 0: 1825.4, 1: 1835.8. Samples: 18497714. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-08 05:20:43,754][130385] Avg episode reward: [(0, '57.470'), (1, '54.630')] [2023-10-08 05:20:43,783][00612] Updated weights for policy 1, policy_version 36220 (0.0009) [2023-10-08 05:20:47,346][00611] Updated weights for policy 0, policy_version 36042 (0.0008) [2023-10-08 05:20:47,451][00612] Updated weights for policy 1, policy_version 36230 (0.0009) [2023-10-08 05:20:47,714][00611] Updated weights for policy 0, policy_version 36052 (0.0008) [2023-10-08 05:20:47,826][00612] Updated weights for policy 1, policy_version 36240 (0.0007) [2023-10-08 05:20:48,081][00611] Updated weights for policy 0, policy_version 36062 (0.0008) [2023-10-08 05:20:48,193][00612] Updated weights for policy 1, policy_version 36250 (0.0008) [2023-10-08 05:20:48,754][130385] Fps is (10 sec: 19660.0, 60 sec: 15291.6, 300 sec: 14773.4). Total num frames: 74055680. Throughput: 0: 1817.2, 1: 1821.9. Samples: 18517886. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-08 05:20:48,755][130385] Avg episode reward: [(0, '55.840'), (1, '55.520')] [2023-10-08 05:20:51,778][00612] Updated weights for policy 1, policy_version 36260 (0.0007) [2023-10-08 05:20:51,900][00611] Updated weights for policy 0, policy_version 36072 (0.0009) [2023-10-08 05:20:52,148][00612] Updated weights for policy 1, policy_version 36270 (0.0007) [2023-10-08 05:20:52,262][00611] Updated weights for policy 0, policy_version 36082 (0.0009) [2023-10-08 05:20:52,520][00612] Updated weights for policy 1, policy_version 36280 (0.0008) [2023-10-08 05:20:52,639][00611] Updated weights for policy 0, policy_version 36092 (0.0007) [2023-10-08 05:20:53,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 74121216. Throughput: 0: 1816.4, 1: 1829.3. Samples: 18530544. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-08 05:20:53,754][130385] Avg episode reward: [(0, '54.910'), (1, '55.360')] [2023-10-08 05:20:56,212][00612] Updated weights for policy 1, policy_version 36290 (0.0008) [2023-10-08 05:20:56,278][00611] Updated weights for policy 0, policy_version 36102 (0.0008) [2023-10-08 05:20:56,578][00612] Updated weights for policy 1, policy_version 36300 (0.0007) [2023-10-08 05:20:56,646][00611] Updated weights for policy 0, policy_version 36112 (0.0009) [2023-10-08 05:20:56,941][00612] Updated weights for policy 1, policy_version 36310 (0.0007) [2023-10-08 05:20:57,016][00611] Updated weights for policy 0, policy_version 36122 (0.0008) [2023-10-08 05:20:57,311][00612] Updated weights for policy 1, policy_version 36320 (0.0007) [2023-10-08 05:20:58,754][130385] Fps is (10 sec: 13107.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 74186752. Throughput: 0: 1821.4, 1: 1823.0. Samples: 18550956. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-08 05:20:58,755][130385] Avg episode reward: [(0, '52.680'), (1, '59.080')] [2023-10-08 05:21:00,697][00611] Updated weights for policy 0, policy_version 36132 (0.0009) [2023-10-08 05:21:00,895][00612] Updated weights for policy 1, policy_version 36330 (0.0008) [2023-10-08 05:21:01,061][00611] Updated weights for policy 0, policy_version 36142 (0.0008) [2023-10-08 05:21:01,270][00612] Updated weights for policy 1, policy_version 36340 (0.0008) [2023-10-08 05:21:01,427][00611] Updated weights for policy 0, policy_version 36152 (0.0009) [2023-10-08 05:21:01,638][00612] Updated weights for policy 1, policy_version 36350 (0.0008) [2023-10-08 05:21:03,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 74252288. Throughput: 0: 1815.3, 1: 1833.4. Samples: 18573606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:21:03,755][130385] Avg episode reward: [(0, '49.850'), (1, '60.930')] [2023-10-08 05:21:05,029][00611] Updated weights for policy 0, policy_version 36162 (0.0009) [2023-10-08 05:21:05,248][00612] Updated weights for policy 1, policy_version 36360 (0.0008) [2023-10-08 05:21:05,397][00611] Updated weights for policy 0, policy_version 36172 (0.0008) [2023-10-08 05:21:05,621][00612] Updated weights for policy 1, policy_version 36370 (0.0010) [2023-10-08 05:21:05,758][00611] Updated weights for policy 0, policy_version 36182 (0.0008) [2023-10-08 05:21:05,985][00612] Updated weights for policy 1, policy_version 36380 (0.0008) [2023-10-08 05:21:06,133][00611] Updated weights for policy 0, policy_version 36192 (0.0008) [2023-10-08 05:21:08,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 74317824. Throughput: 0: 1815.3, 1: 1824.4. Samples: 18583612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:21:08,754][130385] Avg episode reward: [(0, '51.670'), (1, '57.900')] [2023-10-08 05:21:09,730][00612] Updated weights for policy 1, policy_version 36390 (0.0008) [2023-10-08 05:21:09,775][00611] Updated weights for policy 0, policy_version 36202 (0.0008) [2023-10-08 05:21:10,097][00612] Updated weights for policy 1, policy_version 36400 (0.0007) [2023-10-08 05:21:10,151][00611] Updated weights for policy 0, policy_version 36212 (0.0009) [2023-10-08 05:21:10,465][00612] Updated weights for policy 1, policy_version 36410 (0.0008) [2023-10-08 05:21:10,523][00611] Updated weights for policy 0, policy_version 36222 (0.0008) [2023-10-08 05:21:13,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 74383360. Throughput: 0: 1819.9, 1: 1831.4. Samples: 18606654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:21:13,754][130385] Avg episode reward: [(0, '51.940'), (1, '58.740')] [2023-10-08 05:21:14,064][00611] Updated weights for policy 0, policy_version 36232 (0.0007) [2023-10-08 05:21:14,068][00612] Updated weights for policy 1, policy_version 36420 (0.0008) [2023-10-08 05:21:14,431][00611] Updated weights for policy 0, policy_version 36242 (0.0008) [2023-10-08 05:21:14,436][00612] Updated weights for policy 1, policy_version 36430 (0.0009) [2023-10-08 05:21:14,798][00612] Updated weights for policy 1, policy_version 36440 (0.0008) [2023-10-08 05:21:14,803][00611] Updated weights for policy 0, policy_version 36252 (0.0009) [2023-10-08 05:21:18,510][00612] Updated weights for policy 1, policy_version 36450 (0.0007) [2023-10-08 05:21:18,699][00611] Updated weights for policy 0, policy_version 36262 (0.0007) [2023-10-08 05:21:18,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 74448896. Throughput: 0: 1824.8, 1: 1831.6. Samples: 18629690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:21:18,754][130385] Avg episode reward: [(0, '51.850'), (1, '60.050')] [2023-10-08 05:21:18,874][00612] Updated weights for policy 1, policy_version 36460 (0.0007) [2023-10-08 05:21:19,079][00611] Updated weights for policy 0, policy_version 36272 (0.0010) [2023-10-08 05:21:19,236][00612] Updated weights for policy 1, policy_version 36470 (0.0007) [2023-10-08 05:21:19,450][00611] Updated weights for policy 0, policy_version 36282 (0.0007) [2023-10-08 05:21:19,603][00612] Updated weights for policy 1, policy_version 36480 (0.0007) [2023-10-08 05:21:23,052][00611] Updated weights for policy 0, policy_version 36292 (0.0007) [2023-10-08 05:21:23,318][00612] Updated weights for policy 1, policy_version 36490 (0.0007) [2023-10-08 05:21:23,423][00611] Updated weights for policy 0, policy_version 36302 (0.0008) [2023-10-08 05:21:23,689][00612] Updated weights for policy 1, policy_version 36500 (0.0007) [2023-10-08 05:21:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 74514432. Throughput: 0: 1822.5, 1: 1839.2. Samples: 18639484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:21:23,755][130385] Avg episode reward: [(0, '54.910'), (1, '59.450')] [2023-10-08 05:21:23,802][00611] Updated weights for policy 0, policy_version 36312 (0.0007) [2023-10-08 05:21:24,052][00612] Updated weights for policy 1, policy_version 36510 (0.0008) [2023-10-08 05:21:27,483][00611] Updated weights for policy 0, policy_version 36322 (0.0009) [2023-10-08 05:21:27,697][00612] Updated weights for policy 1, policy_version 36520 (0.0007) [2023-10-08 05:21:27,838][00611] Updated weights for policy 0, policy_version 36332 (0.0008) [2023-10-08 05:21:28,074][00612] Updated weights for policy 1, policy_version 36530 (0.0009) [2023-10-08 05:21:28,214][00611] Updated weights for policy 0, policy_version 36342 (0.0009) [2023-10-08 05:21:28,440][00612] Updated weights for policy 1, policy_version 36540 (0.0007) [2023-10-08 05:21:28,579][00611] Updated weights for policy 0, policy_version 36352 (0.0008) [2023-10-08 05:21:28,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 74645504. Throughput: 0: 1823.7, 1: 1841.5. Samples: 18662650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:21:28,754][130385] Avg episode reward: [(0, '53.830'), (1, '59.360')] [2023-10-08 05:21:32,159][00612] Updated weights for policy 1, policy_version 36550 (0.0008) [2023-10-08 05:21:32,191][00611] Updated weights for policy 0, policy_version 36362 (0.0008) [2023-10-08 05:21:32,515][00612] Updated weights for policy 1, policy_version 36560 (0.0008) [2023-10-08 05:21:32,563][00611] Updated weights for policy 0, policy_version 36372 (0.0009) [2023-10-08 05:21:32,885][00612] Updated weights for policy 1, policy_version 36570 (0.0008) [2023-10-08 05:21:32,934][00611] Updated weights for policy 0, policy_version 36382 (0.0009) [2023-10-08 05:21:33,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 74711040. Throughput: 0: 1826.9, 1: 1830.2. Samples: 18682456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:21:33,755][130385] Avg episode reward: [(0, '54.950'), (1, '59.580')] [2023-10-08 05:21:36,433][00612] Updated weights for policy 1, policy_version 36580 (0.0008) [2023-10-08 05:21:36,587][00611] Updated weights for policy 0, policy_version 36392 (0.0007) [2023-10-08 05:21:36,805][00612] Updated weights for policy 1, policy_version 36590 (0.0008) [2023-10-08 05:21:36,958][00611] Updated weights for policy 0, policy_version 36402 (0.0008) [2023-10-08 05:21:37,171][00612] Updated weights for policy 1, policy_version 36600 (0.0007) [2023-10-08 05:21:37,337][00611] Updated weights for policy 0, policy_version 36412 (0.0008) [2023-10-08 05:21:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 74776576. Throughput: 0: 1833.5, 1: 1838.1. Samples: 18695768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:21:38,755][130385] Avg episode reward: [(0, '54.430'), (1, '58.800')] [2023-10-08 05:21:40,941][00612] Updated weights for policy 1, policy_version 36610 (0.0007) [2023-10-08 05:21:40,989][00611] Updated weights for policy 0, policy_version 36422 (0.0008) [2023-10-08 05:21:41,322][00612] Updated weights for policy 1, policy_version 36620 (0.0010) [2023-10-08 05:21:41,361][00611] Updated weights for policy 0, policy_version 36432 (0.0007) [2023-10-08 05:21:41,689][00612] Updated weights for policy 1, policy_version 36630 (0.0009) [2023-10-08 05:21:41,739][00611] Updated weights for policy 0, policy_version 36442 (0.0008) [2023-10-08 05:21:42,059][00612] Updated weights for policy 1, policy_version 36640 (0.0010) [2023-10-08 05:21:43,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 74842112. Throughput: 0: 1826.0, 1: 1824.3. Samples: 18715218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:21:43,754][130385] Avg episode reward: [(0, '53.280'), (1, '56.840')] [2023-10-08 05:21:45,426][00611] Updated weights for policy 0, policy_version 36452 (0.0009) [2023-10-08 05:21:45,682][00612] Updated weights for policy 1, policy_version 36650 (0.0008) [2023-10-08 05:21:45,798][00611] Updated weights for policy 0, policy_version 36462 (0.0008) [2023-10-08 05:21:46,048][00612] Updated weights for policy 1, policy_version 36660 (0.0007) [2023-10-08 05:21:46,172][00611] Updated weights for policy 0, policy_version 36472 (0.0007) [2023-10-08 05:21:46,418][00612] Updated weights for policy 1, policy_version 36670 (0.0007) [2023-10-08 05:21:48,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 74907648. Throughput: 0: 1827.1, 1: 1825.8. Samples: 18737986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:21:48,754][130385] Avg episode reward: [(0, '54.370'), (1, '59.870')] [2023-10-08 05:21:49,990][00611] Updated weights for policy 0, policy_version 36482 (0.0007) [2023-10-08 05:21:50,146][00612] Updated weights for policy 1, policy_version 36680 (0.0007) [2023-10-08 05:21:50,360][00611] Updated weights for policy 0, policy_version 36492 (0.0007) [2023-10-08 05:21:50,523][00612] Updated weights for policy 1, policy_version 36690 (0.0009) [2023-10-08 05:21:50,726][00611] Updated weights for policy 0, policy_version 36502 (0.0008) [2023-10-08 05:21:50,885][00612] Updated weights for policy 1, policy_version 36700 (0.0010) [2023-10-08 05:21:51,104][00611] Updated weights for policy 0, policy_version 36512 (0.0007) [2023-10-08 05:21:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 74973184. Throughput: 0: 1824.9, 1: 1825.3. Samples: 18747870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:21:53,754][130385] Avg episode reward: [(0, '57.550'), (1, '58.220')] [2023-10-08 05:21:54,655][00612] Updated weights for policy 1, policy_version 36710 (0.0010) [2023-10-08 05:21:54,879][00611] Updated weights for policy 0, policy_version 36522 (0.0008) [2023-10-08 05:21:55,013][00612] Updated weights for policy 1, policy_version 36720 (0.0008) [2023-10-08 05:21:55,249][00611] Updated weights for policy 0, policy_version 36532 (0.0008) [2023-10-08 05:21:55,372][00612] Updated weights for policy 1, policy_version 36730 (0.0007) [2023-10-08 05:21:55,617][00611] Updated weights for policy 0, policy_version 36542 (0.0008) [2023-10-08 05:21:58,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 75038720. Throughput: 0: 1820.8, 1: 1823.0. Samples: 18770626. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 05:21:58,755][130385] Avg episode reward: [(0, '57.490'), (1, '59.680')] [2023-10-08 05:21:58,936][00612] Updated weights for policy 1, policy_version 36740 (0.0010) [2023-10-08 05:21:59,305][00612] Updated weights for policy 1, policy_version 36750 (0.0007) [2023-10-08 05:21:59,341][00611] Updated weights for policy 0, policy_version 36552 (0.0007) [2023-10-08 05:21:59,664][00612] Updated weights for policy 1, policy_version 36760 (0.0008) [2023-10-08 05:21:59,712][00611] Updated weights for policy 0, policy_version 36562 (0.0008) [2023-10-08 05:22:00,088][00611] Updated weights for policy 0, policy_version 36572 (0.0008) [2023-10-08 05:22:03,351][00612] Updated weights for policy 1, policy_version 36770 (0.0009) [2023-10-08 05:22:03,679][00611] Updated weights for policy 0, policy_version 36582 (0.0008) [2023-10-08 05:22:03,723][00612] Updated weights for policy 1, policy_version 36780 (0.0007) [2023-10-08 05:22:03,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 75104256. Throughput: 0: 1821.0, 1: 1820.4. Samples: 18793552. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 05:22:03,754][130385] Avg episode reward: [(0, '60.120'), (1, '58.070')] [2023-10-08 05:22:04,062][00611] Updated weights for policy 0, policy_version 36592 (0.0007) [2023-10-08 05:22:04,092][00612] Updated weights for policy 1, policy_version 36790 (0.0008) [2023-10-08 05:22:04,434][00611] Updated weights for policy 0, policy_version 36602 (0.0008) [2023-10-08 05:22:04,464][00612] Updated weights for policy 1, policy_version 36800 (0.0008) [2023-10-08 05:22:08,174][00611] Updated weights for policy 0, policy_version 36612 (0.0009) [2023-10-08 05:22:08,323][00612] Updated weights for policy 1, policy_version 36810 (0.0009) [2023-10-08 05:22:08,547][00611] Updated weights for policy 0, policy_version 36622 (0.0007) [2023-10-08 05:22:08,696][00612] Updated weights for policy 1, policy_version 36820 (0.0009) [2023-10-08 05:22:08,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 75169792. Throughput: 0: 1820.4, 1: 1814.8. Samples: 18803068. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 05:22:08,755][130385] Avg episode reward: [(0, '56.290'), (1, '59.680')] [2023-10-08 05:22:08,929][00611] Updated weights for policy 0, policy_version 36632 (0.0008) [2023-10-08 05:22:09,056][00612] Updated weights for policy 1, policy_version 36830 (0.0009) [2023-10-08 05:22:12,566][00611] Updated weights for policy 0, policy_version 36642 (0.0009) [2023-10-08 05:22:12,752][00612] Updated weights for policy 1, policy_version 36840 (0.0007) [2023-10-08 05:22:12,939][00611] Updated weights for policy 0, policy_version 36652 (0.0008) [2023-10-08 05:22:13,117][00612] Updated weights for policy 1, policy_version 36850 (0.0007) [2023-10-08 05:22:13,294][00611] Updated weights for policy 0, policy_version 36662 (0.0007) [2023-10-08 05:22:13,482][00612] Updated weights for policy 1, policy_version 36860 (0.0007) [2023-10-08 05:22:13,660][00611] Updated weights for policy 0, policy_version 36672 (0.0008) [2023-10-08 05:22:13,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 75300864. Throughput: 0: 1821.0, 1: 1812.2. Samples: 18826144. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 05:22:13,754][130385] Avg episode reward: [(0, '52.820'), (1, '56.220')] [2023-10-08 05:22:17,199][00612] Updated weights for policy 1, policy_version 36870 (0.0007) [2023-10-08 05:22:17,364][00611] Updated weights for policy 0, policy_version 36682 (0.0007) [2023-10-08 05:22:17,573][00612] Updated weights for policy 1, policy_version 36880 (0.0009) [2023-10-08 05:22:17,740][00611] Updated weights for policy 0, policy_version 36692 (0.0008) [2023-10-08 05:22:17,940][00612] Updated weights for policy 1, policy_version 36890 (0.0010) [2023-10-08 05:22:18,107][00611] Updated weights for policy 0, policy_version 36702 (0.0008) [2023-10-08 05:22:18,754][130385] Fps is (10 sec: 19660.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 75366400. Throughput: 0: 1821.5, 1: 1819.2. Samples: 18846284. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 05:22:18,755][130385] Avg episode reward: [(0, '52.430'), (1, '59.860')] [2023-10-08 05:22:18,767][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000036896_37781504.pth... [2023-10-08 05:22:18,767][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000036704_37584896.pth... [2023-10-08 05:22:18,805][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000035008_35848192.pth [2023-10-08 05:22:18,808][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000035168_36012032.pth [2023-10-08 05:22:21,704][00612] Updated weights for policy 1, policy_version 36900 (0.0008) [2023-10-08 05:22:21,770][00611] Updated weights for policy 0, policy_version 36712 (0.0008) [2023-10-08 05:22:22,068][00612] Updated weights for policy 1, policy_version 36910 (0.0007) [2023-10-08 05:22:22,140][00611] Updated weights for policy 0, policy_version 36722 (0.0007) [2023-10-08 05:22:22,435][00612] Updated weights for policy 1, policy_version 36920 (0.0010) [2023-10-08 05:22:22,506][00611] Updated weights for policy 0, policy_version 36732 (0.0007) [2023-10-08 05:22:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 75431936. Throughput: 0: 1816.0, 1: 1811.2. Samples: 18858988. Policy #0 lag: (min: 27.0, avg: 30.0, max: 59.0) [2023-10-08 05:22:23,754][130385] Avg episode reward: [(0, '52.830'), (1, '59.360')] [2023-10-08 05:22:25,945][00612] Updated weights for policy 1, policy_version 36930 (0.0008) [2023-10-08 05:22:26,281][00611] Updated weights for policy 0, policy_version 36742 (0.0007) [2023-10-08 05:22:26,315][00612] Updated weights for policy 1, policy_version 36940 (0.0007) [2023-10-08 05:22:26,638][00611] Updated weights for policy 0, policy_version 36752 (0.0007) [2023-10-08 05:22:26,675][00612] Updated weights for policy 1, policy_version 36950 (0.0007) [2023-10-08 05:22:27,012][00611] Updated weights for policy 0, policy_version 36762 (0.0008) [2023-10-08 05:22:27,040][00612] Updated weights for policy 1, policy_version 36960 (0.0009) [2023-10-08 05:22:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 75497472. Throughput: 0: 1815.1, 1: 1822.6. Samples: 18878918. Policy #0 lag: (min: 27.0, avg: 30.0, max: 59.0) [2023-10-08 05:22:28,755][130385] Avg episode reward: [(0, '53.430'), (1, '58.080')] [2023-10-08 05:22:30,733][00611] Updated weights for policy 0, policy_version 36772 (0.0009) [2023-10-08 05:22:30,734][00612] Updated weights for policy 1, policy_version 36970 (0.0007) [2023-10-08 05:22:31,105][00612] Updated weights for policy 1, policy_version 36980 (0.0010) [2023-10-08 05:22:31,122][00611] Updated weights for policy 0, policy_version 36782 (0.0008) [2023-10-08 05:22:31,472][00612] Updated weights for policy 1, policy_version 36990 (0.0007) [2023-10-08 05:22:31,491][00611] Updated weights for policy 0, policy_version 36792 (0.0009) [2023-10-08 05:22:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 75563008. Throughput: 0: 1812.1, 1: 1824.7. Samples: 18901640. Policy #0 lag: (min: 27.0, avg: 30.0, max: 59.0) [2023-10-08 05:22:33,754][130385] Avg episode reward: [(0, '57.350'), (1, '61.440')] [2023-10-08 05:22:35,039][00611] Updated weights for policy 0, policy_version 36802 (0.0009) [2023-10-08 05:22:35,163][00612] Updated weights for policy 1, policy_version 37000 (0.0007) [2023-10-08 05:22:35,403][00611] Updated weights for policy 0, policy_version 36812 (0.0007) [2023-10-08 05:22:35,518][00612] Updated weights for policy 1, policy_version 37010 (0.0009) [2023-10-08 05:22:35,775][00611] Updated weights for policy 0, policy_version 36822 (0.0008) [2023-10-08 05:22:35,892][00612] Updated weights for policy 1, policy_version 37020 (0.0007) [2023-10-08 05:22:36,146][00611] Updated weights for policy 0, policy_version 36832 (0.0010) [2023-10-08 05:22:38,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 75628544. Throughput: 0: 1816.5, 1: 1824.9. Samples: 18911734. Policy #0 lag: (min: 27.0, avg: 30.0, max: 59.0) [2023-10-08 05:22:38,754][130385] Avg episode reward: [(0, '55.610'), (1, '63.280')] [2023-10-08 05:22:39,566][00612] Updated weights for policy 1, policy_version 37030 (0.0007) [2023-10-08 05:22:39,761][00611] Updated weights for policy 0, policy_version 36842 (0.0008) [2023-10-08 05:22:39,924][00612] Updated weights for policy 1, policy_version 37040 (0.0007) [2023-10-08 05:22:40,125][00611] Updated weights for policy 0, policy_version 36852 (0.0008) [2023-10-08 05:22:40,289][00612] Updated weights for policy 1, policy_version 37050 (0.0008) [2023-10-08 05:22:40,490][00611] Updated weights for policy 0, policy_version 36862 (0.0009) [2023-10-08 05:22:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 75694080. Throughput: 0: 1819.6, 1: 1826.1. Samples: 18934680. Policy #0 lag: (min: 27.0, avg: 30.0, max: 59.0) [2023-10-08 05:22:43,755][130385] Avg episode reward: [(0, '51.220'), (1, '65.870')] [2023-10-08 05:22:43,848][00612] Updated weights for policy 1, policy_version 37060 (0.0007) [2023-10-08 05:22:44,213][00612] Updated weights for policy 1, policy_version 37070 (0.0009) [2023-10-08 05:22:44,257][00611] Updated weights for policy 0, policy_version 36872 (0.0008) [2023-10-08 05:22:44,583][00612] Updated weights for policy 1, policy_version 37080 (0.0010) [2023-10-08 05:22:44,626][00611] Updated weights for policy 0, policy_version 36882 (0.0008) [2023-10-08 05:22:44,876][00425] Saving new best policy, reward=65.870! [2023-10-08 05:22:44,998][00611] Updated weights for policy 0, policy_version 36892 (0.0010) [2023-10-08 05:22:48,193][00612] Updated weights for policy 1, policy_version 37090 (0.0009) [2023-10-08 05:22:48,566][00612] Updated weights for policy 1, policy_version 37100 (0.0009) [2023-10-08 05:22:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 75759616. Throughput: 0: 1810.5, 1: 1830.3. Samples: 18957388. Policy #0 lag: (min: 27.0, avg: 30.0, max: 59.0) [2023-10-08 05:22:48,754][130385] Avg episode reward: [(0, '49.230'), (1, '61.240')] [2023-10-08 05:22:48,881][00611] Updated weights for policy 0, policy_version 36902 (0.0009) [2023-10-08 05:22:48,932][00612] Updated weights for policy 1, policy_version 37110 (0.0007) [2023-10-08 05:22:49,264][00611] Updated weights for policy 0, policy_version 36912 (0.0008) [2023-10-08 05:22:49,294][00612] Updated weights for policy 1, policy_version 37120 (0.0007) [2023-10-08 05:22:49,635][00611] Updated weights for policy 0, policy_version 36922 (0.0008) [2023-10-08 05:22:53,178][00612] Updated weights for policy 1, policy_version 37130 (0.0009) [2023-10-08 05:22:53,220][00611] Updated weights for policy 0, policy_version 36932 (0.0009) [2023-10-08 05:22:53,548][00612] Updated weights for policy 1, policy_version 37140 (0.0007) [2023-10-08 05:22:53,591][00611] Updated weights for policy 0, policy_version 36942 (0.0008) [2023-10-08 05:22:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 75825152. Throughput: 0: 1812.1, 1: 1835.7. Samples: 18967222. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-10-08 05:22:53,754][130385] Avg episode reward: [(0, '52.200'), (1, '61.970')] [2023-10-08 05:22:53,918][00612] Updated weights for policy 1, policy_version 37150 (0.0008) [2023-10-08 05:22:53,967][00611] Updated weights for policy 0, policy_version 36952 (0.0008) [2023-10-08 05:22:57,410][00612] Updated weights for policy 1, policy_version 37160 (0.0008) [2023-10-08 05:22:57,606][00611] Updated weights for policy 0, policy_version 36962 (0.0011) [2023-10-08 05:22:57,778][00612] Updated weights for policy 1, policy_version 37170 (0.0007) [2023-10-08 05:22:57,984][00611] Updated weights for policy 0, policy_version 36972 (0.0007) [2023-10-08 05:22:58,143][00612] Updated weights for policy 1, policy_version 37180 (0.0008) [2023-10-08 05:22:58,351][00611] Updated weights for policy 0, policy_version 36982 (0.0007) [2023-10-08 05:22:58,726][00611] Updated weights for policy 0, policy_version 36992 (0.0011) [2023-10-08 05:22:58,754][130385] Fps is (10 sec: 19660.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 75956224. Throughput: 0: 1807.7, 1: 1835.7. Samples: 18990098. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-10-08 05:22:58,755][130385] Avg episode reward: [(0, '52.700'), (1, '59.000')] [2023-10-08 05:23:01,764][00612] Updated weights for policy 1, policy_version 37190 (0.0008) [2023-10-08 05:23:02,134][00612] Updated weights for policy 1, policy_version 37200 (0.0007) [2023-10-08 05:23:02,490][00612] Updated weights for policy 1, policy_version 37210 (0.0008) [2023-10-08 05:23:02,538][00611] Updated weights for policy 0, policy_version 37002 (0.0007) [2023-10-08 05:23:02,911][00611] Updated weights for policy 0, policy_version 37012 (0.0008) [2023-10-08 05:23:03,293][00611] Updated weights for policy 0, policy_version 37022 (0.0009) [2023-10-08 05:23:03,754][130385] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 76021760. Throughput: 0: 1809.9, 1: 1838.1. Samples: 19010446. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-10-08 05:23:03,755][130385] Avg episode reward: [(0, '53.430'), (1, '62.860')] [2023-10-08 05:23:06,147][00612] Updated weights for policy 1, policy_version 37220 (0.0008) [2023-10-08 05:23:06,519][00612] Updated weights for policy 1, policy_version 37230 (0.0007) [2023-10-08 05:23:06,887][00612] Updated weights for policy 1, policy_version 37240 (0.0007) [2023-10-08 05:23:06,989][00611] Updated weights for policy 0, policy_version 37032 (0.0007) [2023-10-08 05:23:07,357][00611] Updated weights for policy 0, policy_version 37042 (0.0007) [2023-10-08 05:23:07,733][00611] Updated weights for policy 0, policy_version 37052 (0.0008) [2023-10-08 05:23:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 76087296. Throughput: 0: 1801.7, 1: 1838.1. Samples: 19022782. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-10-08 05:23:08,755][130385] Avg episode reward: [(0, '51.840'), (1, '62.610')] [2023-10-08 05:23:10,435][00612] Updated weights for policy 1, policy_version 37250 (0.0007) [2023-10-08 05:23:10,805][00612] Updated weights for policy 1, policy_version 37260 (0.0010) [2023-10-08 05:23:11,177][00612] Updated weights for policy 1, policy_version 37270 (0.0011) [2023-10-08 05:23:11,399][00611] Updated weights for policy 0, policy_version 37062 (0.0008) [2023-10-08 05:23:11,535][00612] Updated weights for policy 1, policy_version 37280 (0.0009) [2023-10-08 05:23:11,773][00611] Updated weights for policy 0, policy_version 37072 (0.0007) [2023-10-08 05:23:12,140][00611] Updated weights for policy 0, policy_version 37082 (0.0007) [2023-10-08 05:23:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 76152832. Throughput: 0: 1812.4, 1: 1845.2. Samples: 19043514. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-10-08 05:23:13,755][130385] Avg episode reward: [(0, '52.740'), (1, '60.360')] [2023-10-08 05:23:15,212][00612] Updated weights for policy 1, policy_version 37290 (0.0008) [2023-10-08 05:23:15,588][00612] Updated weights for policy 1, policy_version 37300 (0.0008) [2023-10-08 05:23:15,832][00611] Updated weights for policy 0, policy_version 37092 (0.0007) [2023-10-08 05:23:15,953][00612] Updated weights for policy 1, policy_version 37310 (0.0007) [2023-10-08 05:23:16,201][00611] Updated weights for policy 0, policy_version 37102 (0.0007) [2023-10-08 05:23:16,568][00611] Updated weights for policy 0, policy_version 37112 (0.0007) [2023-10-08 05:23:18,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 76218368. Throughput: 0: 1812.1, 1: 1842.8. Samples: 19066112. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-10-08 05:23:18,755][130385] Avg episode reward: [(0, '55.050'), (1, '60.380')] [2023-10-08 05:23:19,504][00612] Updated weights for policy 1, policy_version 37320 (0.0007) [2023-10-08 05:23:19,886][00612] Updated weights for policy 1, policy_version 37330 (0.0007) [2023-10-08 05:23:20,181][00611] Updated weights for policy 0, policy_version 37122 (0.0007) [2023-10-08 05:23:20,247][00612] Updated weights for policy 1, policy_version 37340 (0.0007) [2023-10-08 05:23:20,550][00611] Updated weights for policy 0, policy_version 37132 (0.0010) [2023-10-08 05:23:20,930][00611] Updated weights for policy 0, policy_version 37142 (0.0009) [2023-10-08 05:23:21,292][00611] Updated weights for policy 0, policy_version 37152 (0.0011) [2023-10-08 05:23:23,749][00612] Updated weights for policy 1, policy_version 37350 (0.0009) [2023-10-08 05:23:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 76283904. Throughput: 0: 1816.8, 1: 1848.0. Samples: 19076652. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-10-08 05:23:23,754][130385] Avg episode reward: [(0, '53.580'), (1, '59.060')] [2023-10-08 05:23:24,116][00612] Updated weights for policy 1, policy_version 37360 (0.0007) [2023-10-08 05:23:24,493][00612] Updated weights for policy 1, policy_version 37370 (0.0009) [2023-10-08 05:23:25,027][00611] Updated weights for policy 0, policy_version 37162 (0.0007) [2023-10-08 05:23:25,402][00611] Updated weights for policy 0, policy_version 37172 (0.0007) [2023-10-08 05:23:25,779][00611] Updated weights for policy 0, policy_version 37182 (0.0007) [2023-10-08 05:23:28,200][00612] Updated weights for policy 1, policy_version 37380 (0.0007) [2023-10-08 05:23:28,571][00612] Updated weights for policy 1, policy_version 37390 (0.0007) [2023-10-08 05:23:28,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 76349440. Throughput: 0: 1810.0, 1: 1857.2. Samples: 19099706. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-10-08 05:23:28,755][130385] Avg episode reward: [(0, '52.000'), (1, '60.640')] [2023-10-08 05:23:28,949][00612] Updated weights for policy 1, policy_version 37400 (0.0009) [2023-10-08 05:23:29,370][00611] Updated weights for policy 0, policy_version 37192 (0.0008) [2023-10-08 05:23:29,733][00611] Updated weights for policy 0, policy_version 37202 (0.0009) [2023-10-08 05:23:30,105][00611] Updated weights for policy 0, policy_version 37212 (0.0008) [2023-10-08 05:23:32,495][00612] Updated weights for policy 1, policy_version 37410 (0.0007) [2023-10-08 05:23:32,863][00612] Updated weights for policy 1, policy_version 37420 (0.0010) [2023-10-08 05:23:33,233][00612] Updated weights for policy 1, policy_version 37430 (0.0010) [2023-10-08 05:23:33,599][00612] Updated weights for policy 1, policy_version 37440 (0.0009) [2023-10-08 05:23:33,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76447744. Throughput: 0: 1819.3, 1: 1836.3. Samples: 19121890. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-10-08 05:23:33,754][130385] Avg episode reward: [(0, '52.490'), (1, '58.970')] [2023-10-08 05:23:33,850][00611] Updated weights for policy 0, policy_version 37222 (0.0009) [2023-10-08 05:23:34,237][00611] Updated weights for policy 0, policy_version 37232 (0.0008) [2023-10-08 05:23:34,612][00611] Updated weights for policy 0, policy_version 37242 (0.0009) [2023-10-08 05:23:37,332][00612] Updated weights for policy 1, policy_version 37450 (0.0011) [2023-10-08 05:23:37,702][00612] Updated weights for policy 1, policy_version 37460 (0.0008) [2023-10-08 05:23:38,065][00612] Updated weights for policy 1, policy_version 37470 (0.0008) [2023-10-08 05:23:38,286][00611] Updated weights for policy 0, policy_version 37252 (0.0008) [2023-10-08 05:23:38,656][00611] Updated weights for policy 0, policy_version 37262 (0.0008) [2023-10-08 05:23:38,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76513280. Throughput: 0: 1819.1, 1: 1858.8. Samples: 19132728. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-10-08 05:23:38,754][130385] Avg episode reward: [(0, '60.460'), (1, '59.190')] [2023-10-08 05:23:39,025][00611] Updated weights for policy 0, policy_version 37272 (0.0010) [2023-10-08 05:23:41,853][00612] Updated weights for policy 1, policy_version 37480 (0.0009) [2023-10-08 05:23:42,229][00612] Updated weights for policy 1, policy_version 37490 (0.0008) [2023-10-08 05:23:42,599][00612] Updated weights for policy 1, policy_version 37500 (0.0008) [2023-10-08 05:23:42,610][00611] Updated weights for policy 0, policy_version 37282 (0.0007) [2023-10-08 05:23:42,992][00611] Updated weights for policy 0, policy_version 37292 (0.0008) [2023-10-08 05:23:43,360][00611] Updated weights for policy 0, policy_version 37302 (0.0009) [2023-10-08 05:23:43,735][00611] Updated weights for policy 0, policy_version 37312 (0.0009) [2023-10-08 05:23:43,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 76611584. Throughput: 0: 1822.3, 1: 1837.8. Samples: 19154800. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 05:23:43,754][130385] Avg episode reward: [(0, '61.160'), (1, '58.770')] [2023-10-08 05:23:46,179][00612] Updated weights for policy 1, policy_version 37510 (0.0008) [2023-10-08 05:23:46,554][00612] Updated weights for policy 1, policy_version 37520 (0.0008) [2023-10-08 05:23:46,931][00612] Updated weights for policy 1, policy_version 37530 (0.0008) [2023-10-08 05:23:47,376][00611] Updated weights for policy 0, policy_version 37322 (0.0008) [2023-10-08 05:23:47,740][00611] Updated weights for policy 0, policy_version 37332 (0.0008) [2023-10-08 05:23:48,119][00611] Updated weights for policy 0, policy_version 37342 (0.0009) [2023-10-08 05:23:48,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 76677120. Throughput: 0: 1818.2, 1: 1852.4. Samples: 19175624. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 05:23:48,755][130385] Avg episode reward: [(0, '60.020'), (1, '58.590')] [2023-10-08 05:23:50,633][00612] Updated weights for policy 1, policy_version 37540 (0.0007) [2023-10-08 05:23:50,998][00612] Updated weights for policy 1, policy_version 37550 (0.0011) [2023-10-08 05:23:51,364][00612] Updated weights for policy 1, policy_version 37560 (0.0007) [2023-10-08 05:23:51,739][00611] Updated weights for policy 0, policy_version 37352 (0.0009) [2023-10-08 05:23:52,117][00611] Updated weights for policy 0, policy_version 37362 (0.0008) [2023-10-08 05:23:52,499][00611] Updated weights for policy 0, policy_version 37372 (0.0007) [2023-10-08 05:23:53,754][130385] Fps is (10 sec: 13106.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 76742656. Throughput: 0: 1832.2, 1: 1835.6. Samples: 19187832. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 05:23:53,755][130385] Avg episode reward: [(0, '58.350'), (1, '60.350')] [2023-10-08 05:23:55,041][00612] Updated weights for policy 1, policy_version 37570 (0.0007) [2023-10-08 05:23:55,413][00612] Updated weights for policy 1, policy_version 37580 (0.0007) [2023-10-08 05:23:55,786][00612] Updated weights for policy 1, policy_version 37590 (0.0007) [2023-10-08 05:23:56,089][00611] Updated weights for policy 0, policy_version 37382 (0.0008) [2023-10-08 05:23:56,147][00612] Updated weights for policy 1, policy_version 37600 (0.0009) [2023-10-08 05:23:56,464][00611] Updated weights for policy 0, policy_version 37392 (0.0008) [2023-10-08 05:23:56,829][00611] Updated weights for policy 0, policy_version 37402 (0.0009) [2023-10-08 05:23:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 76808192. Throughput: 0: 1821.7, 1: 1847.2. Samples: 19208614. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 05:23:58,755][130385] Avg episode reward: [(0, '58.260'), (1, '57.750')] [2023-10-08 05:23:59,606][00612] Updated weights for policy 1, policy_version 37610 (0.0007) [2023-10-08 05:23:59,972][00612] Updated weights for policy 1, policy_version 37620 (0.0009) [2023-10-08 05:24:00,337][00612] Updated weights for policy 1, policy_version 37630 (0.0010) [2023-10-08 05:24:00,586][00611] Updated weights for policy 0, policy_version 37412 (0.0008) [2023-10-08 05:24:00,957][00611] Updated weights for policy 0, policy_version 37422 (0.0007) [2023-10-08 05:24:01,334][00611] Updated weights for policy 0, policy_version 37432 (0.0007) [2023-10-08 05:24:03,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 76873728. Throughput: 0: 1825.8, 1: 1858.0. Samples: 19231884. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 05:24:03,754][130385] Avg episode reward: [(0, '61.460'), (1, '61.820')] [2023-10-08 05:24:03,792][00612] Updated weights for policy 1, policy_version 37640 (0.0010) [2023-10-08 05:24:04,163][00612] Updated weights for policy 1, policy_version 37650 (0.0012) [2023-10-08 05:24:04,530][00612] Updated weights for policy 1, policy_version 37660 (0.0007) [2023-10-08 05:24:05,041][00611] Updated weights for policy 0, policy_version 37442 (0.0007) [2023-10-08 05:24:05,415][00611] Updated weights for policy 0, policy_version 37452 (0.0009) [2023-10-08 05:24:05,781][00611] Updated weights for policy 0, policy_version 37462 (0.0009) [2023-10-08 05:24:06,152][00611] Updated weights for policy 0, policy_version 37472 (0.0010) [2023-10-08 05:24:07,945][00612] Updated weights for policy 1, policy_version 37670 (0.0007) [2023-10-08 05:24:08,322][00612] Updated weights for policy 1, policy_version 37680 (0.0007) [2023-10-08 05:24:08,687][00612] Updated weights for policy 1, policy_version 37690 (0.0007) [2023-10-08 05:24:08,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 76939264. Throughput: 0: 1818.9, 1: 1855.4. Samples: 19241996. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 05:24:08,755][130385] Avg episode reward: [(0, '61.080'), (1, '60.310')] [2023-10-08 05:24:09,756][00611] Updated weights for policy 0, policy_version 37482 (0.0007) [2023-10-08 05:24:10,126][00611] Updated weights for policy 0, policy_version 37492 (0.0009) [2023-10-08 05:24:10,501][00611] Updated weights for policy 0, policy_version 37502 (0.0009) [2023-10-08 05:24:12,411][00612] Updated weights for policy 1, policy_version 37700 (0.0008) [2023-10-08 05:24:12,775][00612] Updated weights for policy 1, policy_version 37710 (0.0007) [2023-10-08 05:24:13,150][00612] Updated weights for policy 1, policy_version 37720 (0.0007) [2023-10-08 05:24:13,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 77037568. Throughput: 0: 1829.6, 1: 1851.4. Samples: 19265348. Policy #0 lag: (min: 8.0, avg: 30.1, max: 40.0) [2023-10-08 05:24:13,755][130385] Avg episode reward: [(0, '61.810'), (1, '63.640')] [2023-10-08 05:24:14,120][00611] Updated weights for policy 0, policy_version 37512 (0.0007) [2023-10-08 05:24:14,495][00611] Updated weights for policy 0, policy_version 37522 (0.0007) [2023-10-08 05:24:14,874][00611] Updated weights for policy 0, policy_version 37532 (0.0008) [2023-10-08 05:24:16,803][00612] Updated weights for policy 1, policy_version 37730 (0.0008) [2023-10-08 05:24:17,167][00612] Updated weights for policy 1, policy_version 37740 (0.0009) [2023-10-08 05:24:17,535][00612] Updated weights for policy 1, policy_version 37750 (0.0011) [2023-10-08 05:24:17,902][00612] Updated weights for policy 1, policy_version 37760 (0.0010) [2023-10-08 05:24:18,397][00611] Updated weights for policy 0, policy_version 37542 (0.0009) [2023-10-08 05:24:18,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77103104. Throughput: 0: 1841.8, 1: 1831.6. Samples: 19287194. Policy #0 lag: (min: 8.0, avg: 30.1, max: 40.0) [2023-10-08 05:24:18,754][130385] Avg episode reward: [(0, '61.530'), (1, '63.770')] [2023-10-08 05:24:18,762][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000037760_38666240.pth... [2023-10-08 05:24:18,764][00611] Updated weights for policy 0, policy_version 37552 (0.0010) [2023-10-08 05:24:18,803][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000036032_36896768.pth [2023-10-08 05:24:19,145][00611] Updated weights for policy 0, policy_version 37562 (0.0009) [2023-10-08 05:24:19,362][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000037568_38469632.pth... [2023-10-08 05:24:19,391][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000035840_36700160.pth [2023-10-08 05:24:21,512][00612] Updated weights for policy 1, policy_version 37770 (0.0008) [2023-10-08 05:24:21,890][00612] Updated weights for policy 1, policy_version 37780 (0.0008) [2023-10-08 05:24:22,265][00612] Updated weights for policy 1, policy_version 37790 (0.0008) [2023-10-08 05:24:22,865][00611] Updated weights for policy 0, policy_version 37572 (0.0009) [2023-10-08 05:24:23,238][00611] Updated weights for policy 0, policy_version 37582 (0.0007) [2023-10-08 05:24:23,614][00611] Updated weights for policy 0, policy_version 37592 (0.0007) [2023-10-08 05:24:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77168640. Throughput: 0: 1842.3, 1: 1840.1. Samples: 19298436. Policy #0 lag: (min: 8.0, avg: 30.1, max: 40.0) [2023-10-08 05:24:23,754][130385] Avg episode reward: [(0, '62.450'), (1, '65.110')] [2023-10-08 05:24:25,859][00612] Updated weights for policy 1, policy_version 37800 (0.0007) [2023-10-08 05:24:26,238][00612] Updated weights for policy 1, policy_version 37810 (0.0007) [2023-10-08 05:24:26,603][00612] Updated weights for policy 1, policy_version 37820 (0.0007) [2023-10-08 05:24:27,365][00611] Updated weights for policy 0, policy_version 37602 (0.0008) [2023-10-08 05:24:27,741][00611] Updated weights for policy 0, policy_version 37612 (0.0008) [2023-10-08 05:24:28,114][00611] Updated weights for policy 0, policy_version 37622 (0.0008) [2023-10-08 05:24:28,487][00611] Updated weights for policy 0, policy_version 37632 (0.0009) [2023-10-08 05:24:28,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 77266944. Throughput: 0: 1841.4, 1: 1830.3. Samples: 19320024. Policy #0 lag: (min: 8.0, avg: 30.1, max: 40.0) [2023-10-08 05:24:28,755][130385] Avg episode reward: [(0, '57.540'), (1, '64.010')] [2023-10-08 05:24:30,435][00612] Updated weights for policy 1, policy_version 37830 (0.0011) [2023-10-08 05:24:30,819][00612] Updated weights for policy 1, policy_version 37840 (0.0009) [2023-10-08 05:24:31,185][00612] Updated weights for policy 1, policy_version 37850 (0.0009) [2023-10-08 05:24:32,019][00611] Updated weights for policy 0, policy_version 37642 (0.0007) [2023-10-08 05:24:32,390][00611] Updated weights for policy 0, policy_version 37652 (0.0007) [2023-10-08 05:24:32,757][00611] Updated weights for policy 0, policy_version 37662 (0.0009) [2023-10-08 05:24:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 77332480. Throughput: 0: 1841.5, 1: 1843.2. Samples: 19341438. Policy #0 lag: (min: 8.0, avg: 30.1, max: 40.0) [2023-10-08 05:24:33,755][130385] Avg episode reward: [(0, '59.410'), (1, '62.480')] [2023-10-08 05:24:35,006][00612] Updated weights for policy 1, policy_version 37860 (0.0007) [2023-10-08 05:24:35,368][00612] Updated weights for policy 1, policy_version 37870 (0.0008) [2023-10-08 05:24:35,731][00612] Updated weights for policy 1, policy_version 37880 (0.0008) [2023-10-08 05:24:36,288][00611] Updated weights for policy 0, policy_version 37672 (0.0009) [2023-10-08 05:24:36,659][00611] Updated weights for policy 0, policy_version 37682 (0.0007) [2023-10-08 05:24:37,029][00611] Updated weights for policy 0, policy_version 37692 (0.0009) [2023-10-08 05:24:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 77398016. Throughput: 0: 1842.1, 1: 1826.9. Samples: 19352938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:24:38,755][130385] Avg episode reward: [(0, '58.830'), (1, '62.680')] [2023-10-08 05:24:39,443][00612] Updated weights for policy 1, policy_version 37890 (0.0008) [2023-10-08 05:24:39,815][00612] Updated weights for policy 1, policy_version 37900 (0.0009) [2023-10-08 05:24:40,177][00612] Updated weights for policy 1, policy_version 37910 (0.0007) [2023-10-08 05:24:40,542][00612] Updated weights for policy 1, policy_version 37920 (0.0007) [2023-10-08 05:24:40,779][00611] Updated weights for policy 0, policy_version 37702 (0.0009) [2023-10-08 05:24:41,150][00611] Updated weights for policy 0, policy_version 37712 (0.0008) [2023-10-08 05:24:41,517][00611] Updated weights for policy 0, policy_version 37722 (0.0010) [2023-10-08 05:24:43,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 77463552. Throughput: 0: 1844.2, 1: 1846.7. Samples: 19374706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:24:43,754][130385] Avg episode reward: [(0, '58.890'), (1, '61.980')] [2023-10-08 05:24:44,114][00612] Updated weights for policy 1, policy_version 37930 (0.0007) [2023-10-08 05:24:44,484][00612] Updated weights for policy 1, policy_version 37940 (0.0008) [2023-10-08 05:24:44,850][00612] Updated weights for policy 1, policy_version 37950 (0.0007) [2023-10-08 05:24:45,118][00611] Updated weights for policy 0, policy_version 37732 (0.0010) [2023-10-08 05:24:45,489][00611] Updated weights for policy 0, policy_version 37742 (0.0008) [2023-10-08 05:24:45,861][00611] Updated weights for policy 0, policy_version 37752 (0.0009) [2023-10-08 05:24:48,466][00612] Updated weights for policy 1, policy_version 37960 (0.0008) [2023-10-08 05:24:48,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 77529088. Throughput: 0: 1855.6, 1: 1840.2. Samples: 19398196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:24:48,754][130385] Avg episode reward: [(0, '57.760'), (1, '59.930')] [2023-10-08 05:24:48,833][00612] Updated weights for policy 1, policy_version 37970 (0.0009) [2023-10-08 05:24:49,198][00612] Updated weights for policy 1, policy_version 37980 (0.0007) [2023-10-08 05:24:49,378][00611] Updated weights for policy 0, policy_version 37762 (0.0007) [2023-10-08 05:24:49,752][00611] Updated weights for policy 0, policy_version 37772 (0.0008) [2023-10-08 05:24:50,123][00611] Updated weights for policy 0, policy_version 37782 (0.0007) [2023-10-08 05:24:50,488][00611] Updated weights for policy 0, policy_version 37792 (0.0009) [2023-10-08 05:24:52,941][00612] Updated weights for policy 1, policy_version 37990 (0.0009) [2023-10-08 05:24:53,311][00612] Updated weights for policy 1, policy_version 38000 (0.0010) [2023-10-08 05:24:53,677][00612] Updated weights for policy 1, policy_version 38010 (0.0010) [2023-10-08 05:24:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 77594624. Throughput: 0: 1860.0, 1: 1836.8. Samples: 19408354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:24:53,754][130385] Avg episode reward: [(0, '59.390'), (1, '63.180')] [2023-10-08 05:24:53,982][00611] Updated weights for policy 0, policy_version 37802 (0.0009) [2023-10-08 05:24:54,357][00611] Updated weights for policy 0, policy_version 37812 (0.0007) [2023-10-08 05:24:54,728][00611] Updated weights for policy 0, policy_version 37822 (0.0009) [2023-10-08 05:24:57,236][00612] Updated weights for policy 1, policy_version 38020 (0.0008) [2023-10-08 05:24:57,599][00612] Updated weights for policy 1, policy_version 38030 (0.0008) [2023-10-08 05:24:57,970][00612] Updated weights for policy 1, policy_version 38040 (0.0009) [2023-10-08 05:24:58,263][00611] Updated weights for policy 0, policy_version 37832 (0.0008) [2023-10-08 05:24:58,634][00611] Updated weights for policy 0, policy_version 37842 (0.0009) [2023-10-08 05:24:58,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77692928. Throughput: 0: 1864.6, 1: 1826.2. Samples: 19431434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:24:58,754][130385] Avg episode reward: [(0, '57.370'), (1, '61.490')] [2023-10-08 05:24:59,006][00611] Updated weights for policy 0, policy_version 37852 (0.0009) [2023-10-08 05:25:01,588][00612] Updated weights for policy 1, policy_version 38050 (0.0010) [2023-10-08 05:25:01,947][00612] Updated weights for policy 1, policy_version 38060 (0.0010) [2023-10-08 05:25:02,314][00612] Updated weights for policy 1, policy_version 38070 (0.0008) [2023-10-08 05:25:02,620][00611] Updated weights for policy 0, policy_version 37862 (0.0008) [2023-10-08 05:25:02,681][00612] Updated weights for policy 1, policy_version 38080 (0.0007) [2023-10-08 05:25:02,992][00611] Updated weights for policy 0, policy_version 37872 (0.0007) [2023-10-08 05:25:03,375][00611] Updated weights for policy 0, policy_version 37882 (0.0008) [2023-10-08 05:25:03,754][130385] Fps is (10 sec: 19660.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 77791232. Throughput: 0: 1835.6, 1: 1835.9. Samples: 19452412. Policy #0 lag: (min: 25.0, avg: 35.0, max: 57.0) [2023-10-08 05:25:03,755][130385] Avg episode reward: [(0, '57.590'), (1, '61.530')] [2023-10-08 05:25:06,229][00612] Updated weights for policy 1, policy_version 38090 (0.0009) [2023-10-08 05:25:06,595][00612] Updated weights for policy 1, policy_version 38100 (0.0008) [2023-10-08 05:25:06,963][00612] Updated weights for policy 1, policy_version 38110 (0.0007) [2023-10-08 05:25:07,208][00611] Updated weights for policy 0, policy_version 37892 (0.0009) [2023-10-08 05:25:07,586][00611] Updated weights for policy 0, policy_version 37902 (0.0008) [2023-10-08 05:25:07,965][00611] Updated weights for policy 0, policy_version 37912 (0.0008) [2023-10-08 05:25:08,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 77856768. Throughput: 0: 1855.8, 1: 1832.6. Samples: 19464412. Policy #0 lag: (min: 25.0, avg: 35.0, max: 57.0) [2023-10-08 05:25:08,755][130385] Avg episode reward: [(0, '55.930'), (1, '55.790')] [2023-10-08 05:25:10,594][00612] Updated weights for policy 1, policy_version 38120 (0.0008) [2023-10-08 05:25:10,955][00612] Updated weights for policy 1, policy_version 38130 (0.0007) [2023-10-08 05:25:11,325][00612] Updated weights for policy 1, policy_version 38140 (0.0007) [2023-10-08 05:25:11,535][00611] Updated weights for policy 0, policy_version 37922 (0.0007) [2023-10-08 05:25:11,902][00611] Updated weights for policy 0, policy_version 37932 (0.0007) [2023-10-08 05:25:12,264][00611] Updated weights for policy 0, policy_version 37942 (0.0007) [2023-10-08 05:25:12,639][00611] Updated weights for policy 0, policy_version 37952 (0.0007) [2023-10-08 05:25:13,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 77922304. Throughput: 0: 1837.9, 1: 1842.0. Samples: 19485618. Policy #0 lag: (min: 25.0, avg: 35.0, max: 57.0) [2023-10-08 05:25:13,754][130385] Avg episode reward: [(0, '58.590'), (1, '53.700')] [2023-10-08 05:25:15,179][00612] Updated weights for policy 1, policy_version 38150 (0.0009) [2023-10-08 05:25:15,557][00612] Updated weights for policy 1, policy_version 38160 (0.0010) [2023-10-08 05:25:15,922][00612] Updated weights for policy 1, policy_version 38170 (0.0008) [2023-10-08 05:25:16,347][00611] Updated weights for policy 0, policy_version 37962 (0.0010) [2023-10-08 05:25:16,721][00611] Updated weights for policy 0, policy_version 37972 (0.0007) [2023-10-08 05:25:17,087][00611] Updated weights for policy 0, policy_version 37982 (0.0007) [2023-10-08 05:25:18,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 77987840. Throughput: 0: 1853.5, 1: 1841.6. Samples: 19507718. Policy #0 lag: (min: 25.0, avg: 35.0, max: 57.0) [2023-10-08 05:25:18,754][130385] Avg episode reward: [(0, '58.350'), (1, '55.190')] [2023-10-08 05:25:19,463][00612] Updated weights for policy 1, policy_version 38180 (0.0008) [2023-10-08 05:25:19,832][00612] Updated weights for policy 1, policy_version 38190 (0.0007) [2023-10-08 05:25:20,198][00612] Updated weights for policy 1, policy_version 38200 (0.0007) [2023-10-08 05:25:20,699][00611] Updated weights for policy 0, policy_version 37992 (0.0009) [2023-10-08 05:25:21,072][00611] Updated weights for policy 0, policy_version 38002 (0.0007) [2023-10-08 05:25:21,454][00611] Updated weights for policy 0, policy_version 38012 (0.0007) [2023-10-08 05:25:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 78053376. Throughput: 0: 1830.0, 1: 1850.5. Samples: 19518558. Policy #0 lag: (min: 25.0, avg: 35.0, max: 57.0) [2023-10-08 05:25:23,754][130385] Avg episode reward: [(0, '60.970'), (1, '53.060')] [2023-10-08 05:25:23,816][00612] Updated weights for policy 1, policy_version 38210 (0.0008) [2023-10-08 05:25:24,178][00612] Updated weights for policy 1, policy_version 38220 (0.0008) [2023-10-08 05:25:24,553][00612] Updated weights for policy 1, policy_version 38230 (0.0009) [2023-10-08 05:25:24,916][00612] Updated weights for policy 1, policy_version 38240 (0.0009) [2023-10-08 05:25:25,078][00611] Updated weights for policy 0, policy_version 38022 (0.0009) [2023-10-08 05:25:25,439][00611] Updated weights for policy 0, policy_version 38032 (0.0008) [2023-10-08 05:25:25,818][00611] Updated weights for policy 0, policy_version 38042 (0.0010) [2023-10-08 05:25:28,368][00612] Updated weights for policy 1, policy_version 38250 (0.0007) [2023-10-08 05:25:28,734][00612] Updated weights for policy 1, policy_version 38260 (0.0008) [2023-10-08 05:25:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 78118912. Throughput: 0: 1852.9, 1: 1856.1. Samples: 19541610. Policy #0 lag: (min: 25.0, avg: 35.0, max: 57.0) [2023-10-08 05:25:28,754][130385] Avg episode reward: [(0, '58.020'), (1, '55.510')] [2023-10-08 05:25:29,106][00612] Updated weights for policy 1, policy_version 38270 (0.0007) [2023-10-08 05:25:29,355][00611] Updated weights for policy 0, policy_version 38052 (0.0008) [2023-10-08 05:25:29,724][00611] Updated weights for policy 0, policy_version 38062 (0.0010) [2023-10-08 05:25:30,093][00611] Updated weights for policy 0, policy_version 38072 (0.0007) [2023-10-08 05:25:32,449][00612] Updated weights for policy 1, policy_version 38280 (0.0009) [2023-10-08 05:25:32,829][00612] Updated weights for policy 1, policy_version 38290 (0.0008) [2023-10-08 05:25:33,193][00612] Updated weights for policy 1, policy_version 38300 (0.0007) [2023-10-08 05:25:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 78217216. Throughput: 0: 1852.7, 1: 1836.0. Samples: 19564188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:25:33,754][130385] Avg episode reward: [(0, '59.540'), (1, '54.110')] [2023-10-08 05:25:33,773][00611] Updated weights for policy 0, policy_version 38082 (0.0010) [2023-10-08 05:25:34,153][00611] Updated weights for policy 0, policy_version 38092 (0.0010) [2023-10-08 05:25:34,532][00611] Updated weights for policy 0, policy_version 38102 (0.0008) [2023-10-08 05:25:34,910][00611] Updated weights for policy 0, policy_version 38112 (0.0007) [2023-10-08 05:25:36,821][00612] Updated weights for policy 1, policy_version 38310 (0.0008) [2023-10-08 05:25:37,187][00612] Updated weights for policy 1, policy_version 38320 (0.0009) [2023-10-08 05:25:37,559][00612] Updated weights for policy 1, policy_version 38330 (0.0009) [2023-10-08 05:25:38,466][00611] Updated weights for policy 0, policy_version 38122 (0.0008) [2023-10-08 05:25:38,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 78282752. Throughput: 0: 1848.3, 1: 1869.0. Samples: 19575632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:25:38,754][130385] Avg episode reward: [(0, '60.960'), (1, '53.510')] [2023-10-08 05:25:38,835][00611] Updated weights for policy 0, policy_version 38132 (0.0009) [2023-10-08 05:25:39,209][00611] Updated weights for policy 0, policy_version 38142 (0.0008) [2023-10-08 05:25:41,187][00612] Updated weights for policy 1, policy_version 38340 (0.0009) [2023-10-08 05:25:41,561][00612] Updated weights for policy 1, policy_version 38350 (0.0010) [2023-10-08 05:25:41,933][00612] Updated weights for policy 1, policy_version 38360 (0.0008) [2023-10-08 05:25:42,780][00611] Updated weights for policy 0, policy_version 38152 (0.0007) [2023-10-08 05:25:43,165][00611] Updated weights for policy 0, policy_version 38162 (0.0011) [2023-10-08 05:25:43,550][00611] Updated weights for policy 0, policy_version 38172 (0.0011) [2023-10-08 05:25:43,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 78381056. Throughput: 0: 1839.3, 1: 1841.4. Samples: 19597068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:25:43,755][130385] Avg episode reward: [(0, '62.480'), (1, '51.590')] [2023-10-08 05:25:45,522][00612] Updated weights for policy 1, policy_version 38370 (0.0009) [2023-10-08 05:25:45,885][00612] Updated weights for policy 1, policy_version 38380 (0.0010) [2023-10-08 05:25:46,246][00612] Updated weights for policy 1, policy_version 38390 (0.0007) [2023-10-08 05:25:46,614][00612] Updated weights for policy 1, policy_version 38400 (0.0010) [2023-10-08 05:25:47,045][00611] Updated weights for policy 0, policy_version 38182 (0.0008) [2023-10-08 05:25:47,414][00611] Updated weights for policy 0, policy_version 38192 (0.0009) [2023-10-08 05:25:47,786][00611] Updated weights for policy 0, policy_version 38202 (0.0008) [2023-10-08 05:25:48,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 78446592. Throughput: 0: 1831.2, 1: 1868.2. Samples: 19618884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:25:48,754][130385] Avg episode reward: [(0, '59.650'), (1, '54.240')] [2023-10-08 05:25:50,356][00612] Updated weights for policy 1, policy_version 38410 (0.0007) [2023-10-08 05:25:50,720][00612] Updated weights for policy 1, policy_version 38420 (0.0009) [2023-10-08 05:25:51,090][00612] Updated weights for policy 1, policy_version 38430 (0.0009) [2023-10-08 05:25:51,507][00611] Updated weights for policy 0, policy_version 38212 (0.0008) [2023-10-08 05:25:51,900][00611] Updated weights for policy 0, policy_version 38222 (0.0007) [2023-10-08 05:25:52,261][00611] Updated weights for policy 0, policy_version 38232 (0.0010) [2023-10-08 05:25:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 78512128. Throughput: 0: 1849.5, 1: 1839.8. Samples: 19630430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:25:53,755][130385] Avg episode reward: [(0, '64.750'), (1, '57.200')] [2023-10-08 05:25:54,828][00612] Updated weights for policy 1, policy_version 38440 (0.0009) [2023-10-08 05:25:55,192][00612] Updated weights for policy 1, policy_version 38450 (0.0007) [2023-10-08 05:25:55,561][00612] Updated weights for policy 1, policy_version 38460 (0.0009) [2023-10-08 05:25:56,013][00611] Updated weights for policy 0, policy_version 38242 (0.0010) [2023-10-08 05:25:56,381][00611] Updated weights for policy 0, policy_version 38252 (0.0010) [2023-10-08 05:25:56,749][00611] Updated weights for policy 0, policy_version 38262 (0.0009) [2023-10-08 05:25:57,115][00611] Updated weights for policy 0, policy_version 38272 (0.0008) [2023-10-08 05:25:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 78577664. Throughput: 0: 1826.3, 1: 1866.4. Samples: 19651790. Policy #0 lag: (min: 10.0, avg: 15.7, max: 42.0) [2023-10-08 05:25:58,755][130385] Avg episode reward: [(0, '59.850'), (1, '57.110')] [2023-10-08 05:25:59,144][00612] Updated weights for policy 1, policy_version 38470 (0.0008) [2023-10-08 05:25:59,503][00612] Updated weights for policy 1, policy_version 38480 (0.0009) [2023-10-08 05:25:59,880][00612] Updated weights for policy 1, policy_version 38490 (0.0007) [2023-10-08 05:26:00,705][00611] Updated weights for policy 0, policy_version 38282 (0.0009) [2023-10-08 05:26:01,074][00611] Updated weights for policy 0, policy_version 38292 (0.0009) [2023-10-08 05:26:01,453][00611] Updated weights for policy 0, policy_version 38302 (0.0007) [2023-10-08 05:26:03,726][00612] Updated weights for policy 1, policy_version 38500 (0.0007) [2023-10-08 05:26:03,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 78643200. Throughput: 0: 1847.2, 1: 1862.0. Samples: 19674632. Policy #0 lag: (min: 10.0, avg: 15.7, max: 42.0) [2023-10-08 05:26:03,754][130385] Avg episode reward: [(0, '60.660'), (1, '57.320')] [2023-10-08 05:26:04,098][00612] Updated weights for policy 1, policy_version 38510 (0.0007) [2023-10-08 05:26:04,459][00612] Updated weights for policy 1, policy_version 38520 (0.0008) [2023-10-08 05:26:05,192][00611] Updated weights for policy 0, policy_version 38312 (0.0009) [2023-10-08 05:26:05,556][00611] Updated weights for policy 0, policy_version 38322 (0.0007) [2023-10-08 05:26:05,931][00611] Updated weights for policy 0, policy_version 38332 (0.0010) [2023-10-08 05:26:08,120][00612] Updated weights for policy 1, policy_version 38530 (0.0008) [2023-10-08 05:26:08,485][00612] Updated weights for policy 1, policy_version 38540 (0.0010) [2023-10-08 05:26:08,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 78708736. Throughput: 0: 1832.5, 1: 1861.2. Samples: 19684776. Policy #0 lag: (min: 10.0, avg: 15.7, max: 42.0) [2023-10-08 05:26:08,754][130385] Avg episode reward: [(0, '58.090'), (1, '56.660')] [2023-10-08 05:26:08,860][00612] Updated weights for policy 1, policy_version 38550 (0.0010) [2023-10-08 05:26:09,235][00612] Updated weights for policy 1, policy_version 38560 (0.0009) [2023-10-08 05:26:09,478][00611] Updated weights for policy 0, policy_version 38342 (0.0010) [2023-10-08 05:26:09,867][00611] Updated weights for policy 0, policy_version 38352 (0.0010) [2023-10-08 05:26:10,237][00611] Updated weights for policy 0, policy_version 38362 (0.0009) [2023-10-08 05:26:12,860][00612] Updated weights for policy 1, policy_version 38570 (0.0007) [2023-10-08 05:26:13,229][00612] Updated weights for policy 1, policy_version 38580 (0.0007) [2023-10-08 05:26:13,607][00612] Updated weights for policy 1, policy_version 38590 (0.0007) [2023-10-08 05:26:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 78807040. Throughput: 0: 1844.4, 1: 1849.1. Samples: 19707818. Policy #0 lag: (min: 10.0, avg: 15.7, max: 42.0) [2023-10-08 05:26:13,754][130385] Avg episode reward: [(0, '58.880'), (1, '57.200')] [2023-10-08 05:26:13,902][00611] Updated weights for policy 0, policy_version 38372 (0.0009) [2023-10-08 05:26:14,275][00611] Updated weights for policy 0, policy_version 38382 (0.0009) [2023-10-08 05:26:14,645][00611] Updated weights for policy 0, policy_version 38392 (0.0009) [2023-10-08 05:26:17,283][00612] Updated weights for policy 1, policy_version 38600 (0.0008) [2023-10-08 05:26:17,653][00612] Updated weights for policy 1, policy_version 38610 (0.0008) [2023-10-08 05:26:18,033][00612] Updated weights for policy 1, policy_version 38620 (0.0008) [2023-10-08 05:26:18,219][00611] Updated weights for policy 0, policy_version 38402 (0.0010) [2023-10-08 05:26:18,596][00611] Updated weights for policy 0, policy_version 38412 (0.0009) [2023-10-08 05:26:18,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 78872576. Throughput: 0: 1841.1, 1: 1829.8. Samples: 19729380. Policy #0 lag: (min: 10.0, avg: 15.7, max: 42.0) [2023-10-08 05:26:18,754][130385] Avg episode reward: [(0, '57.090'), (1, '59.140')] [2023-10-08 05:26:18,763][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000038624_39550976.pth... [2023-10-08 05:26:18,801][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000036896_37781504.pth [2023-10-08 05:26:18,963][00611] Updated weights for policy 0, policy_version 38422 (0.0010) [2023-10-08 05:26:19,327][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000038432_39354368.pth... [2023-10-08 05:26:19,328][00611] Updated weights for policy 0, policy_version 38432 (0.0010) [2023-10-08 05:26:19,356][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000036704_37584896.pth [2023-10-08 05:26:21,742][00612] Updated weights for policy 1, policy_version 38630 (0.0010) [2023-10-08 05:26:22,101][00612] Updated weights for policy 1, policy_version 38640 (0.0009) [2023-10-08 05:26:22,479][00612] Updated weights for policy 1, policy_version 38650 (0.0008) [2023-10-08 05:26:22,878][00611] Updated weights for policy 0, policy_version 38442 (0.0007) [2023-10-08 05:26:23,253][00611] Updated weights for policy 0, policy_version 38452 (0.0007) [2023-10-08 05:26:23,627][00611] Updated weights for policy 0, policy_version 38462 (0.0008) [2023-10-08 05:26:23,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 78970880. Throughput: 0: 1838.3, 1: 1832.8. Samples: 19740836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:26:23,755][130385] Avg episode reward: [(0, '59.190'), (1, '59.190')] [2023-10-08 05:26:25,996][00612] Updated weights for policy 1, policy_version 38660 (0.0008) [2023-10-08 05:26:26,373][00612] Updated weights for policy 1, policy_version 38670 (0.0009) [2023-10-08 05:26:26,737][00612] Updated weights for policy 1, policy_version 38680 (0.0007) [2023-10-08 05:26:27,340][00611] Updated weights for policy 0, policy_version 38472 (0.0010) [2023-10-08 05:26:27,708][00611] Updated weights for policy 0, policy_version 38482 (0.0009) [2023-10-08 05:26:28,079][00611] Updated weights for policy 0, policy_version 38492 (0.0008) [2023-10-08 05:26:28,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 79036416. Throughput: 0: 1837.3, 1: 1833.5. Samples: 19762254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:26:28,754][130385] Avg episode reward: [(0, '55.300'), (1, '60.190')] [2023-10-08 05:26:30,351][00612] Updated weights for policy 1, policy_version 38690 (0.0009) [2023-10-08 05:26:30,720][00612] Updated weights for policy 1, policy_version 38700 (0.0007) [2023-10-08 05:26:31,084][00612] Updated weights for policy 1, policy_version 38710 (0.0008) [2023-10-08 05:26:31,453][00612] Updated weights for policy 1, policy_version 38720 (0.0007) [2023-10-08 05:26:31,745][00611] Updated weights for policy 0, policy_version 38502 (0.0009) [2023-10-08 05:26:32,107][00611] Updated weights for policy 0, policy_version 38512 (0.0007) [2023-10-08 05:26:32,475][00611] Updated weights for policy 0, policy_version 38522 (0.0008) [2023-10-08 05:26:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79101952. Throughput: 0: 1832.0, 1: 1844.4. Samples: 19784320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:26:33,754][130385] Avg episode reward: [(0, '54.310'), (1, '58.950')] [2023-10-08 05:26:35,115][00612] Updated weights for policy 1, policy_version 38730 (0.0008) [2023-10-08 05:26:35,490][00612] Updated weights for policy 1, policy_version 38740 (0.0009) [2023-10-08 05:26:35,867][00612] Updated weights for policy 1, policy_version 38750 (0.0009) [2023-10-08 05:26:36,169][00611] Updated weights for policy 0, policy_version 38532 (0.0008) [2023-10-08 05:26:36,545][00611] Updated weights for policy 0, policy_version 38542 (0.0009) [2023-10-08 05:26:36,920][00611] Updated weights for policy 0, policy_version 38552 (0.0008) [2023-10-08 05:26:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79167488. Throughput: 0: 1828.9, 1: 1840.8. Samples: 19795564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:26:38,754][130385] Avg episode reward: [(0, '55.470'), (1, '52.560')] [2023-10-08 05:26:39,516][00612] Updated weights for policy 1, policy_version 38760 (0.0008) [2023-10-08 05:26:39,886][00612] Updated weights for policy 1, policy_version 38770 (0.0008) [2023-10-08 05:26:40,256][00612] Updated weights for policy 1, policy_version 38780 (0.0008) [2023-10-08 05:26:40,569][00611] Updated weights for policy 0, policy_version 38562 (0.0008) [2023-10-08 05:26:40,958][00611] Updated weights for policy 0, policy_version 38572 (0.0008) [2023-10-08 05:26:41,335][00611] Updated weights for policy 0, policy_version 38582 (0.0008) [2023-10-08 05:26:41,696][00611] Updated weights for policy 0, policy_version 38592 (0.0009) [2023-10-08 05:26:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 79233024. Throughput: 0: 1836.8, 1: 1844.0. Samples: 19817424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:26:43,754][130385] Avg episode reward: [(0, '56.710'), (1, '52.060')] [2023-10-08 05:26:43,802][00612] Updated weights for policy 1, policy_version 38790 (0.0008) [2023-10-08 05:26:44,164][00612] Updated weights for policy 1, policy_version 38800 (0.0007) [2023-10-08 05:26:44,542][00612] Updated weights for policy 1, policy_version 38810 (0.0008) [2023-10-08 05:26:45,383][00611] Updated weights for policy 0, policy_version 38602 (0.0010) [2023-10-08 05:26:45,759][00611] Updated weights for policy 0, policy_version 38612 (0.0008) [2023-10-08 05:26:46,133][00611] Updated weights for policy 0, policy_version 38622 (0.0007) [2023-10-08 05:26:48,188][00612] Updated weights for policy 1, policy_version 38820 (0.0009) [2023-10-08 05:26:48,577][00612] Updated weights for policy 1, policy_version 38830 (0.0009) [2023-10-08 05:26:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 79298560. Throughput: 0: 1837.8, 1: 1848.1. Samples: 19840498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:26:48,754][130385] Avg episode reward: [(0, '54.810'), (1, '53.640')] [2023-10-08 05:26:48,946][00612] Updated weights for policy 1, policy_version 38840 (0.0009) [2023-10-08 05:26:49,769][00611] Updated weights for policy 0, policy_version 38632 (0.0009) [2023-10-08 05:26:50,142][00611] Updated weights for policy 0, policy_version 38642 (0.0011) [2023-10-08 05:26:50,500][00611] Updated weights for policy 0, policy_version 38652 (0.0010) [2023-10-08 05:26:52,559][00612] Updated weights for policy 1, policy_version 38850 (0.0008) [2023-10-08 05:26:52,936][00612] Updated weights for policy 1, policy_version 38860 (0.0009) [2023-10-08 05:26:53,305][00612] Updated weights for policy 1, policy_version 38870 (0.0008) [2023-10-08 05:26:53,674][00612] Updated weights for policy 1, policy_version 38880 (0.0008) [2023-10-08 05:26:53,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 79396864. Throughput: 0: 1835.1, 1: 1850.7. Samples: 19850638. Policy #0 lag: (min: 1.0, avg: 10.1, max: 33.0) [2023-10-08 05:26:53,755][130385] Avg episode reward: [(0, '58.840'), (1, '53.130')] [2023-10-08 05:26:54,263][00611] Updated weights for policy 0, policy_version 38662 (0.0008) [2023-10-08 05:26:54,638][00611] Updated weights for policy 0, policy_version 38672 (0.0008) [2023-10-08 05:26:55,003][00611] Updated weights for policy 0, policy_version 38682 (0.0009) [2023-10-08 05:26:57,353][00612] Updated weights for policy 1, policy_version 38890 (0.0009) [2023-10-08 05:26:57,713][00612] Updated weights for policy 1, policy_version 38900 (0.0007) [2023-10-08 05:26:58,088][00612] Updated weights for policy 1, policy_version 38910 (0.0008) [2023-10-08 05:26:58,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 79462400. Throughput: 0: 1835.3, 1: 1844.8. Samples: 19873420. Policy #0 lag: (min: 1.0, avg: 10.1, max: 33.0) [2023-10-08 05:26:58,754][130385] Avg episode reward: [(0, '57.040'), (1, '53.440')] [2023-10-08 05:26:58,802][00611] Updated weights for policy 0, policy_version 38692 (0.0007) [2023-10-08 05:26:59,177][00611] Updated weights for policy 0, policy_version 38702 (0.0008) [2023-10-08 05:26:59,548][00611] Updated weights for policy 0, policy_version 38712 (0.0008) [2023-10-08 05:27:01,679][00612] Updated weights for policy 1, policy_version 38920 (0.0008) [2023-10-08 05:27:02,058][00612] Updated weights for policy 1, policy_version 38930 (0.0009) [2023-10-08 05:27:02,425][00612] Updated weights for policy 1, policy_version 38940 (0.0007) [2023-10-08 05:27:03,217][00611] Updated weights for policy 0, policy_version 38722 (0.0007) [2023-10-08 05:27:03,590][00611] Updated weights for policy 0, policy_version 38732 (0.0007) [2023-10-08 05:27:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 79527936. Throughput: 0: 1834.0, 1: 1851.6. Samples: 19895232. Policy #0 lag: (min: 1.0, avg: 10.1, max: 33.0) [2023-10-08 05:27:03,755][130385] Avg episode reward: [(0, '57.320'), (1, '54.460')] [2023-10-08 05:27:03,963][00611] Updated weights for policy 0, policy_version 38742 (0.0010) [2023-10-08 05:27:04,333][00611] Updated weights for policy 0, policy_version 38752 (0.0009) [2023-10-08 05:27:06,010][00612] Updated weights for policy 1, policy_version 38950 (0.0008) [2023-10-08 05:27:06,375][00612] Updated weights for policy 1, policy_version 38960 (0.0007) [2023-10-08 05:27:06,746][00612] Updated weights for policy 1, policy_version 38970 (0.0009) [2023-10-08 05:27:07,864][00611] Updated weights for policy 0, policy_version 38762 (0.0010) [2023-10-08 05:27:08,232][00611] Updated weights for policy 0, policy_version 38772 (0.0008) [2023-10-08 05:27:08,617][00611] Updated weights for policy 0, policy_version 38782 (0.0008) [2023-10-08 05:27:08,755][130385] Fps is (10 sec: 16382.7, 60 sec: 15291.5, 300 sec: 14662.2). Total num frames: 79626240. Throughput: 0: 1834.3, 1: 1841.0. Samples: 19906230. Policy #0 lag: (min: 1.0, avg: 10.1, max: 33.0) [2023-10-08 05:27:08,756][130385] Avg episode reward: [(0, '54.260'), (1, '55.090')] [2023-10-08 05:27:10,437][00612] Updated weights for policy 1, policy_version 38980 (0.0008) [2023-10-08 05:27:10,812][00612] Updated weights for policy 1, policy_version 38990 (0.0008) [2023-10-08 05:27:11,175][00612] Updated weights for policy 1, policy_version 39000 (0.0007) [2023-10-08 05:27:12,116][00611] Updated weights for policy 0, policy_version 38792 (0.0009) [2023-10-08 05:27:12,492][00611] Updated weights for policy 0, policy_version 38802 (0.0009) [2023-10-08 05:27:12,860][00611] Updated weights for policy 0, policy_version 38812 (0.0007) [2023-10-08 05:27:13,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79691776. Throughput: 0: 1832.7, 1: 1855.2. Samples: 19928208. Policy #0 lag: (min: 1.0, avg: 10.1, max: 33.0) [2023-10-08 05:27:13,754][130385] Avg episode reward: [(0, '54.020'), (1, '54.400')] [2023-10-08 05:27:14,664][00612] Updated weights for policy 1, policy_version 39010 (0.0007) [2023-10-08 05:27:15,037][00612] Updated weights for policy 1, policy_version 39020 (0.0009) [2023-10-08 05:27:15,401][00612] Updated weights for policy 1, policy_version 39030 (0.0008) [2023-10-08 05:27:15,774][00612] Updated weights for policy 1, policy_version 39040 (0.0010) [2023-10-08 05:27:16,439][00611] Updated weights for policy 0, policy_version 38822 (0.0008) [2023-10-08 05:27:16,811][00611] Updated weights for policy 0, policy_version 38832 (0.0008) [2023-10-08 05:27:17,186][00611] Updated weights for policy 0, policy_version 38842 (0.0010) [2023-10-08 05:27:18,754][130385] Fps is (10 sec: 13107.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 79757312. Throughput: 0: 1843.7, 1: 1849.0. Samples: 19950492. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-08 05:27:18,755][130385] Avg episode reward: [(0, '52.450'), (1, '54.860')] [2023-10-08 05:27:19,392][00612] Updated weights for policy 1, policy_version 39050 (0.0008) [2023-10-08 05:27:19,757][00612] Updated weights for policy 1, policy_version 39060 (0.0007) [2023-10-08 05:27:20,133][00612] Updated weights for policy 1, policy_version 39070 (0.0007) [2023-10-08 05:27:20,941][00611] Updated weights for policy 0, policy_version 38852 (0.0008) [2023-10-08 05:27:21,310][00611] Updated weights for policy 0, policy_version 38862 (0.0009) [2023-10-08 05:27:21,677][00611] Updated weights for policy 0, policy_version 38872 (0.0010) [2023-10-08 05:27:23,689][00612] Updated weights for policy 1, policy_version 39080 (0.0009) [2023-10-08 05:27:23,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 79822848. Throughput: 0: 1834.0, 1: 1853.0. Samples: 19961480. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-08 05:27:23,755][130385] Avg episode reward: [(0, '52.500'), (1, '57.310')] [2023-10-08 05:27:24,057][00612] Updated weights for policy 1, policy_version 39090 (0.0009) [2023-10-08 05:27:24,433][00612] Updated weights for policy 1, policy_version 39100 (0.0009) [2023-10-08 05:27:25,375][00611] Updated weights for policy 0, policy_version 38882 (0.0009) [2023-10-08 05:27:25,753][00611] Updated weights for policy 0, policy_version 38892 (0.0009) [2023-10-08 05:27:26,118][00611] Updated weights for policy 0, policy_version 38902 (0.0010) [2023-10-08 05:27:26,496][00611] Updated weights for policy 0, policy_version 38912 (0.0010) [2023-10-08 05:27:28,211][00612] Updated weights for policy 1, policy_version 39110 (0.0007) [2023-10-08 05:27:28,583][00612] Updated weights for policy 1, policy_version 39120 (0.0007) [2023-10-08 05:27:28,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 79888384. Throughput: 0: 1835.4, 1: 1844.7. Samples: 19983026. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-08 05:27:28,754][130385] Avg episode reward: [(0, '49.490'), (1, '60.040')] [2023-10-08 05:27:28,944][00612] Updated weights for policy 1, policy_version 39130 (0.0010) [2023-10-08 05:27:30,260][00611] Updated weights for policy 0, policy_version 38922 (0.0008) [2023-10-08 05:27:30,628][00611] Updated weights for policy 0, policy_version 38932 (0.0007) [2023-10-08 05:27:30,996][00611] Updated weights for policy 0, policy_version 38942 (0.0007) [2023-10-08 05:27:32,526][00612] Updated weights for policy 1, policy_version 39140 (0.0009) [2023-10-08 05:27:32,891][00612] Updated weights for policy 1, policy_version 39150 (0.0010) [2023-10-08 05:27:33,255][00612] Updated weights for policy 1, policy_version 39160 (0.0008) [2023-10-08 05:27:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 79986688. Throughput: 0: 1832.9, 1: 1825.1. Samples: 20005112. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-08 05:27:33,755][130385] Avg episode reward: [(0, '48.680'), (1, '60.380')] [2023-10-08 05:27:34,664][00611] Updated weights for policy 0, policy_version 38952 (0.0010) [2023-10-08 05:27:35,039][00611] Updated weights for policy 0, policy_version 38962 (0.0008) [2023-10-08 05:27:35,414][00611] Updated weights for policy 0, policy_version 38972 (0.0007) [2023-10-08 05:27:37,130][00612] Updated weights for policy 1, policy_version 39170 (0.0008) [2023-10-08 05:27:37,535][00612] Updated weights for policy 1, policy_version 39180 (0.0007) [2023-10-08 05:27:37,899][00612] Updated weights for policy 1, policy_version 39190 (0.0008) [2023-10-08 05:27:38,266][00612] Updated weights for policy 1, policy_version 39200 (0.0007) [2023-10-08 05:27:38,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 80052224. Throughput: 0: 1831.9, 1: 1842.6. Samples: 20015992. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-08 05:27:38,755][130385] Avg episode reward: [(0, '50.300'), (1, '62.080')] [2023-10-08 05:27:38,999][00611] Updated weights for policy 0, policy_version 38982 (0.0010) [2023-10-08 05:27:39,366][00611] Updated weights for policy 0, policy_version 38992 (0.0008) [2023-10-08 05:27:39,751][00611] Updated weights for policy 0, policy_version 39002 (0.0009) [2023-10-08 05:27:41,937][00612] Updated weights for policy 1, policy_version 39210 (0.0009) [2023-10-08 05:27:42,307][00612] Updated weights for policy 1, policy_version 39220 (0.0010) [2023-10-08 05:27:42,666][00612] Updated weights for policy 1, policy_version 39230 (0.0008) [2023-10-08 05:27:43,445][00611] Updated weights for policy 0, policy_version 39012 (0.0009) [2023-10-08 05:27:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 80117760. Throughput: 0: 1832.4, 1: 1824.1. Samples: 20037960. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-08 05:27:43,755][130385] Avg episode reward: [(0, '51.680'), (1, '61.660')] [2023-10-08 05:27:43,806][00611] Updated weights for policy 0, policy_version 39022 (0.0008) [2023-10-08 05:27:44,193][00611] Updated weights for policy 0, policy_version 39032 (0.0008) [2023-10-08 05:27:46,223][00612] Updated weights for policy 1, policy_version 39240 (0.0007) [2023-10-08 05:27:46,598][00612] Updated weights for policy 1, policy_version 39250 (0.0007) [2023-10-08 05:27:46,964][00612] Updated weights for policy 1, policy_version 39260 (0.0007) [2023-10-08 05:27:47,750][00611] Updated weights for policy 0, policy_version 39042 (0.0009) [2023-10-08 05:27:48,117][00611] Updated weights for policy 0, policy_version 39052 (0.0007) [2023-10-08 05:27:48,493][00611] Updated weights for policy 0, policy_version 39062 (0.0008) [2023-10-08 05:27:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 80183296. Throughput: 0: 1822.1, 1: 1836.8. Samples: 20059884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:27:48,755][130385] Avg episode reward: [(0, '52.730'), (1, '62.030')] [2023-10-08 05:27:48,865][00611] Updated weights for policy 0, policy_version 39072 (0.0009) [2023-10-08 05:27:50,715][00612] Updated weights for policy 1, policy_version 39270 (0.0009) [2023-10-08 05:27:51,079][00612] Updated weights for policy 1, policy_version 39280 (0.0008) [2023-10-08 05:27:51,441][00612] Updated weights for policy 1, policy_version 39290 (0.0008) [2023-10-08 05:27:52,577][00611] Updated weights for policy 0, policy_version 39082 (0.0009) [2023-10-08 05:27:52,946][00611] Updated weights for policy 0, policy_version 39092 (0.0009) [2023-10-08 05:27:53,308][00611] Updated weights for policy 0, policy_version 39102 (0.0011) [2023-10-08 05:27:53,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80281600. Throughput: 0: 1833.9, 1: 1824.0. Samples: 20070832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:27:53,754][130385] Avg episode reward: [(0, '53.030'), (1, '58.990')] [2023-10-08 05:27:55,078][00612] Updated weights for policy 1, policy_version 39300 (0.0010) [2023-10-08 05:27:55,443][00612] Updated weights for policy 1, policy_version 39310 (0.0009) [2023-10-08 05:27:55,807][00612] Updated weights for policy 1, policy_version 39320 (0.0009) [2023-10-08 05:27:56,759][00611] Updated weights for policy 0, policy_version 39112 (0.0010) [2023-10-08 05:27:57,130][00611] Updated weights for policy 0, policy_version 39122 (0.0008) [2023-10-08 05:27:57,499][00611] Updated weights for policy 0, policy_version 39132 (0.0008) [2023-10-08 05:27:58,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80347136. Throughput: 0: 1825.7, 1: 1837.5. Samples: 20093052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:27:58,755][130385] Avg episode reward: [(0, '55.120'), (1, '58.410')] [2023-10-08 05:27:59,500][00612] Updated weights for policy 1, policy_version 39330 (0.0011) [2023-10-08 05:27:59,866][00612] Updated weights for policy 1, policy_version 39340 (0.0008) [2023-10-08 05:28:00,239][00612] Updated weights for policy 1, policy_version 39350 (0.0008) [2023-10-08 05:28:00,602][00612] Updated weights for policy 1, policy_version 39360 (0.0008) [2023-10-08 05:28:01,192][00611] Updated weights for policy 0, policy_version 39142 (0.0007) [2023-10-08 05:28:01,560][00611] Updated weights for policy 0, policy_version 39152 (0.0008) [2023-10-08 05:28:01,930][00611] Updated weights for policy 0, policy_version 39162 (0.0008) [2023-10-08 05:28:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80412672. Throughput: 0: 1827.3, 1: 1831.3. Samples: 20115126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:28:03,754][130385] Avg episode reward: [(0, '54.890'), (1, '56.410')] [2023-10-08 05:28:04,228][00612] Updated weights for policy 1, policy_version 39370 (0.0008) [2023-10-08 05:28:04,597][00612] Updated weights for policy 1, policy_version 39380 (0.0007) [2023-10-08 05:28:04,961][00612] Updated weights for policy 1, policy_version 39390 (0.0009) [2023-10-08 05:28:05,655][00611] Updated weights for policy 0, policy_version 39172 (0.0009) [2023-10-08 05:28:06,029][00611] Updated weights for policy 0, policy_version 39182 (0.0008) [2023-10-08 05:28:06,405][00611] Updated weights for policy 0, policy_version 39192 (0.0007) [2023-10-08 05:28:08,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.7, 300 sec: 14662.3). Total num frames: 80478208. Throughput: 0: 1820.3, 1: 1828.0. Samples: 20125652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:28:08,754][130385] Avg episode reward: [(0, '54.690'), (1, '55.710')] [2023-10-08 05:28:08,764][00612] Updated weights for policy 1, policy_version 39400 (0.0007) [2023-10-08 05:28:09,127][00612] Updated weights for policy 1, policy_version 39410 (0.0010) [2023-10-08 05:28:09,505][00612] Updated weights for policy 1, policy_version 39420 (0.0009) [2023-10-08 05:28:10,116][00611] Updated weights for policy 0, policy_version 39202 (0.0008) [2023-10-08 05:28:10,492][00611] Updated weights for policy 0, policy_version 39212 (0.0008) [2023-10-08 05:28:10,856][00611] Updated weights for policy 0, policy_version 39222 (0.0008) [2023-10-08 05:28:11,230][00611] Updated weights for policy 0, policy_version 39232 (0.0009) [2023-10-08 05:28:13,215][00612] Updated weights for policy 1, policy_version 39430 (0.0009) [2023-10-08 05:28:13,575][00612] Updated weights for policy 1, policy_version 39440 (0.0008) [2023-10-08 05:28:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 80543744. Throughput: 0: 1831.9, 1: 1825.0. Samples: 20147586. Policy #0 lag: (min: 23.0, avg: 23.1, max: 30.0) [2023-10-08 05:28:13,754][130385] Avg episode reward: [(0, '56.010'), (1, '57.040')] [2023-10-08 05:28:13,948][00612] Updated weights for policy 1, policy_version 39450 (0.0008) [2023-10-08 05:28:14,980][00611] Updated weights for policy 0, policy_version 39242 (0.0008) [2023-10-08 05:28:15,346][00611] Updated weights for policy 0, policy_version 39252 (0.0007) [2023-10-08 05:28:15,713][00611] Updated weights for policy 0, policy_version 39262 (0.0009) [2023-10-08 05:28:17,535][00612] Updated weights for policy 1, policy_version 39460 (0.0008) [2023-10-08 05:28:17,901][00612] Updated weights for policy 1, policy_version 39470 (0.0007) [2023-10-08 05:28:18,268][00612] Updated weights for policy 1, policy_version 39480 (0.0008) [2023-10-08 05:28:18,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 80642048. Throughput: 0: 1828.2, 1: 1823.3. Samples: 20169430. Policy #0 lag: (min: 23.0, avg: 23.1, max: 30.0) [2023-10-08 05:28:18,754][130385] Avg episode reward: [(0, '59.600'), (1, '58.200')] [2023-10-08 05:28:18,762][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000039488_40435712.pth... [2023-10-08 05:28:18,762][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000039264_40206336.pth... [2023-10-08 05:28:18,792][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000037760_38666240.pth [2023-10-08 05:28:18,800][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000037568_38469632.pth [2023-10-08 05:28:19,321][00611] Updated weights for policy 0, policy_version 39272 (0.0008) [2023-10-08 05:28:19,689][00611] Updated weights for policy 0, policy_version 39282 (0.0010) [2023-10-08 05:28:20,060][00611] Updated weights for policy 0, policy_version 39292 (0.0010) [2023-10-08 05:28:21,919][00612] Updated weights for policy 1, policy_version 39490 (0.0009) [2023-10-08 05:28:22,306][00612] Updated weights for policy 1, policy_version 39500 (0.0007) [2023-10-08 05:28:22,680][00612] Updated weights for policy 1, policy_version 39510 (0.0007) [2023-10-08 05:28:23,042][00612] Updated weights for policy 1, policy_version 39520 (0.0007) [2023-10-08 05:28:23,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 80707584. Throughput: 0: 1834.0, 1: 1826.0. Samples: 20180690. Policy #0 lag: (min: 23.0, avg: 23.1, max: 30.0) [2023-10-08 05:28:23,755][130385] Avg episode reward: [(0, '57.390'), (1, '56.400')] [2023-10-08 05:28:23,800][00611] Updated weights for policy 0, policy_version 39302 (0.0008) [2023-10-08 05:28:24,168][00611] Updated weights for policy 0, policy_version 39312 (0.0008) [2023-10-08 05:28:24,545][00611] Updated weights for policy 0, policy_version 39322 (0.0007) [2023-10-08 05:28:26,537][00612] Updated weights for policy 1, policy_version 39530 (0.0007) [2023-10-08 05:28:26,914][00612] Updated weights for policy 1, policy_version 39540 (0.0008) [2023-10-08 05:28:27,285][00612] Updated weights for policy 1, policy_version 39550 (0.0008) [2023-10-08 05:28:28,239][00611] Updated weights for policy 0, policy_version 39332 (0.0010) [2023-10-08 05:28:28,607][00611] Updated weights for policy 0, policy_version 39342 (0.0010) [2023-10-08 05:28:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80773120. Throughput: 0: 1833.7, 1: 1825.6. Samples: 20202630. Policy #0 lag: (min: 23.0, avg: 23.1, max: 30.0) [2023-10-08 05:28:28,754][130385] Avg episode reward: [(0, '53.760'), (1, '57.860')] [2023-10-08 05:28:28,979][00611] Updated weights for policy 0, policy_version 39352 (0.0010) [2023-10-08 05:28:30,910][00612] Updated weights for policy 1, policy_version 39560 (0.0009) [2023-10-08 05:28:31,279][00612] Updated weights for policy 1, policy_version 39570 (0.0009) [2023-10-08 05:28:31,651][00612] Updated weights for policy 1, policy_version 39580 (0.0008) [2023-10-08 05:28:32,705][00611] Updated weights for policy 0, policy_version 39362 (0.0009) [2023-10-08 05:28:33,084][00611] Updated weights for policy 0, policy_version 39372 (0.0008) [2023-10-08 05:28:33,457][00611] Updated weights for policy 0, policy_version 39382 (0.0008) [2023-10-08 05:28:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 80838656. Throughput: 0: 1832.6, 1: 1837.4. Samples: 20225034. Policy #0 lag: (min: 23.0, avg: 23.1, max: 30.0) [2023-10-08 05:28:33,755][130385] Avg episode reward: [(0, '50.770'), (1, '55.450')] [2023-10-08 05:28:33,830][00611] Updated weights for policy 0, policy_version 39392 (0.0008) [2023-10-08 05:28:35,315][00612] Updated weights for policy 1, policy_version 39590 (0.0009) [2023-10-08 05:28:35,681][00612] Updated weights for policy 1, policy_version 39600 (0.0011) [2023-10-08 05:28:36,049][00612] Updated weights for policy 1, policy_version 39610 (0.0009) [2023-10-08 05:28:37,427][00611] Updated weights for policy 0, policy_version 39402 (0.0008) [2023-10-08 05:28:37,793][00611] Updated weights for policy 0, policy_version 39412 (0.0009) [2023-10-08 05:28:38,168][00611] Updated weights for policy 0, policy_version 39422 (0.0009) [2023-10-08 05:28:38,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80936960. Throughput: 0: 1834.3, 1: 1833.2. Samples: 20235870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:28:38,755][130385] Avg episode reward: [(0, '46.000'), (1, '56.030')] [2023-10-08 05:28:39,639][00612] Updated weights for policy 1, policy_version 39620 (0.0007) [2023-10-08 05:28:40,024][00612] Updated weights for policy 1, policy_version 39630 (0.0008) [2023-10-08 05:28:40,388][00612] Updated weights for policy 1, policy_version 39640 (0.0009) [2023-10-08 05:28:41,642][00611] Updated weights for policy 0, policy_version 39432 (0.0010) [2023-10-08 05:28:42,014][00611] Updated weights for policy 0, policy_version 39442 (0.0010) [2023-10-08 05:28:42,387][00611] Updated weights for policy 0, policy_version 39452 (0.0009) [2023-10-08 05:28:43,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81002496. Throughput: 0: 1825.2, 1: 1839.6. Samples: 20257964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:28:43,754][130385] Avg episode reward: [(0, '47.520'), (1, '54.950')] [2023-10-08 05:28:44,007][00612] Updated weights for policy 1, policy_version 39650 (0.0010) [2023-10-08 05:28:44,375][00612] Updated weights for policy 1, policy_version 39660 (0.0008) [2023-10-08 05:28:44,740][00612] Updated weights for policy 1, policy_version 39670 (0.0008) [2023-10-08 05:28:45,110][00612] Updated weights for policy 1, policy_version 39680 (0.0008) [2023-10-08 05:28:46,003][00611] Updated weights for policy 0, policy_version 39462 (0.0007) [2023-10-08 05:28:46,370][00611] Updated weights for policy 0, policy_version 39472 (0.0007) [2023-10-08 05:28:46,749][00611] Updated weights for policy 0, policy_version 39482 (0.0010) [2023-10-08 05:28:48,634][00612] Updated weights for policy 1, policy_version 39690 (0.0010) [2023-10-08 05:28:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81068032. Throughput: 0: 1831.5, 1: 1843.2. Samples: 20280486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:28:48,755][130385] Avg episode reward: [(0, '46.830'), (1, '55.720')] [2023-10-08 05:28:49,001][00612] Updated weights for policy 1, policy_version 39700 (0.0007) [2023-10-08 05:28:49,374][00612] Updated weights for policy 1, policy_version 39710 (0.0007) [2023-10-08 05:28:50,483][00611] Updated weights for policy 0, policy_version 39492 (0.0008) [2023-10-08 05:28:50,850][00611] Updated weights for policy 0, policy_version 39502 (0.0009) [2023-10-08 05:28:51,218][00611] Updated weights for policy 0, policy_version 39512 (0.0008) [2023-10-08 05:28:52,962][00612] Updated weights for policy 1, policy_version 39720 (0.0010) [2023-10-08 05:28:53,340][00612] Updated weights for policy 1, policy_version 39730 (0.0010) [2023-10-08 05:28:53,706][00612] Updated weights for policy 1, policy_version 39740 (0.0007) [2023-10-08 05:28:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 81133568. Throughput: 0: 1829.0, 1: 1848.3. Samples: 20291130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:28:53,754][130385] Avg episode reward: [(0, '44.590'), (1, '55.220')] [2023-10-08 05:28:54,788][00611] Updated weights for policy 0, policy_version 39522 (0.0007) [2023-10-08 05:28:55,162][00611] Updated weights for policy 0, policy_version 39532 (0.0008) [2023-10-08 05:28:55,532][00611] Updated weights for policy 0, policy_version 39542 (0.0008) [2023-10-08 05:28:55,896][00611] Updated weights for policy 0, policy_version 39552 (0.0008) [2023-10-08 05:28:57,370][00612] Updated weights for policy 1, policy_version 39750 (0.0008) [2023-10-08 05:28:57,734][00612] Updated weights for policy 1, policy_version 39760 (0.0008) [2023-10-08 05:28:58,109][00612] Updated weights for policy 1, policy_version 39770 (0.0008) [2023-10-08 05:28:58,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 81231872. Throughput: 0: 1843.2, 1: 1853.0. Samples: 20313916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:28:58,755][130385] Avg episode reward: [(0, '42.000'), (1, '57.350')] [2023-10-08 05:28:59,292][00611] Updated weights for policy 0, policy_version 39562 (0.0009) [2023-10-08 05:28:59,667][00611] Updated weights for policy 0, policy_version 39572 (0.0008) [2023-10-08 05:29:00,034][00611] Updated weights for policy 0, policy_version 39582 (0.0007) [2023-10-08 05:29:01,644][00612] Updated weights for policy 1, policy_version 39780 (0.0007) [2023-10-08 05:29:02,012][00612] Updated weights for policy 1, policy_version 39790 (0.0009) [2023-10-08 05:29:02,376][00612] Updated weights for policy 1, policy_version 39800 (0.0008) [2023-10-08 05:29:03,617][00611] Updated weights for policy 0, policy_version 39592 (0.0008) [2023-10-08 05:29:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 81297408. Throughput: 0: 1853.1, 1: 1846.0. Samples: 20335888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:29:03,754][130385] Avg episode reward: [(0, '42.640'), (1, '60.230')] [2023-10-08 05:29:03,982][00611] Updated weights for policy 0, policy_version 39602 (0.0009) [2023-10-08 05:29:04,360][00611] Updated weights for policy 0, policy_version 39612 (0.0007) [2023-10-08 05:29:06,051][00612] Updated weights for policy 1, policy_version 39810 (0.0008) [2023-10-08 05:29:06,422][00612] Updated weights for policy 1, policy_version 39820 (0.0008) [2023-10-08 05:29:06,802][00612] Updated weights for policy 1, policy_version 39830 (0.0009) [2023-10-08 05:29:07,166][00612] Updated weights for policy 1, policy_version 39840 (0.0009) [2023-10-08 05:29:07,888][00611] Updated weights for policy 0, policy_version 39622 (0.0007) [2023-10-08 05:29:08,264][00611] Updated weights for policy 0, policy_version 39632 (0.0007) [2023-10-08 05:29:08,634][00611] Updated weights for policy 0, policy_version 39642 (0.0007) [2023-10-08 05:29:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 81362944. Throughput: 0: 1851.9, 1: 1849.9. Samples: 20347270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:29:08,755][130385] Avg episode reward: [(0, '43.780'), (1, '56.990')] [2023-10-08 05:29:10,766][00612] Updated weights for policy 1, policy_version 39850 (0.0011) [2023-10-08 05:29:11,134][00612] Updated weights for policy 1, policy_version 39860 (0.0010) [2023-10-08 05:29:11,492][00612] Updated weights for policy 1, policy_version 39870 (0.0010) [2023-10-08 05:29:12,390][00611] Updated weights for policy 0, policy_version 39652 (0.0009) [2023-10-08 05:29:12,755][00611] Updated weights for policy 0, policy_version 39662 (0.0007) [2023-10-08 05:29:13,129][00611] Updated weights for policy 0, policy_version 39672 (0.0008) [2023-10-08 05:29:13,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 81461248. Throughput: 0: 1853.6, 1: 1849.5. Samples: 20369270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:29:13,755][130385] Avg episode reward: [(0, '47.060'), (1, '56.510')] [2023-10-08 05:29:15,093][00612] Updated weights for policy 1, policy_version 39880 (0.0008) [2023-10-08 05:29:15,457][00612] Updated weights for policy 1, policy_version 39890 (0.0007) [2023-10-08 05:29:15,821][00612] Updated weights for policy 1, policy_version 39900 (0.0007) [2023-10-08 05:29:16,744][00611] Updated weights for policy 0, policy_version 39682 (0.0008) [2023-10-08 05:29:17,119][00611] Updated weights for policy 0, policy_version 39692 (0.0008) [2023-10-08 05:29:17,486][00611] Updated weights for policy 0, policy_version 39702 (0.0009) [2023-10-08 05:29:17,857][00611] Updated weights for policy 0, policy_version 39712 (0.0008) [2023-10-08 05:29:18,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 81526784. Throughput: 0: 1831.4, 1: 1855.3. Samples: 20390936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:29:18,755][130385] Avg episode reward: [(0, '49.620'), (1, '54.990')] [2023-10-08 05:29:19,508][00612] Updated weights for policy 1, policy_version 39910 (0.0008) [2023-10-08 05:29:19,874][00612] Updated weights for policy 1, policy_version 39920 (0.0009) [2023-10-08 05:29:20,237][00612] Updated weights for policy 1, policy_version 39930 (0.0008) [2023-10-08 05:29:21,518][00611] Updated weights for policy 0, policy_version 39722 (0.0008) [2023-10-08 05:29:21,880][00611] Updated weights for policy 0, policy_version 39732 (0.0007) [2023-10-08 05:29:22,259][00611] Updated weights for policy 0, policy_version 39742 (0.0007) [2023-10-08 05:29:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81592320. Throughput: 0: 1858.1, 1: 1844.6. Samples: 20402494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:29:23,754][130385] Avg episode reward: [(0, '51.460'), (1, '56.580')] [2023-10-08 05:29:23,837][00612] Updated weights for policy 1, policy_version 39940 (0.0008) [2023-10-08 05:29:24,199][00612] Updated weights for policy 1, policy_version 39950 (0.0011) [2023-10-08 05:29:24,571][00612] Updated weights for policy 1, policy_version 39960 (0.0007) [2023-10-08 05:29:25,916][00611] Updated weights for policy 0, policy_version 39752 (0.0008) [2023-10-08 05:29:26,293][00611] Updated weights for policy 0, policy_version 39762 (0.0007) [2023-10-08 05:29:26,656][00611] Updated weights for policy 0, policy_version 39772 (0.0008) [2023-10-08 05:29:28,300][00612] Updated weights for policy 1, policy_version 39970 (0.0010) [2023-10-08 05:29:28,676][00612] Updated weights for policy 1, policy_version 39980 (0.0010) [2023-10-08 05:29:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81657856. Throughput: 0: 1847.1, 1: 1845.0. Samples: 20424110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:29:28,755][130385] Avg episode reward: [(0, '50.160'), (1, '55.240')] [2023-10-08 05:29:29,047][00612] Updated weights for policy 1, policy_version 39990 (0.0008) [2023-10-08 05:29:29,422][00612] Updated weights for policy 1, policy_version 40000 (0.0008) [2023-10-08 05:29:30,257][00611] Updated weights for policy 0, policy_version 39782 (0.0008) [2023-10-08 05:29:30,632][00611] Updated weights for policy 0, policy_version 39792 (0.0007) [2023-10-08 05:29:31,005][00611] Updated weights for policy 0, policy_version 39802 (0.0008) [2023-10-08 05:29:32,964][00612] Updated weights for policy 1, policy_version 40010 (0.0009) [2023-10-08 05:29:33,326][00612] Updated weights for policy 1, policy_version 40020 (0.0010) [2023-10-08 05:29:33,694][00612] Updated weights for policy 1, policy_version 40030 (0.0008) [2023-10-08 05:29:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81723392. Throughput: 0: 1864.0, 1: 1833.3. Samples: 20446864. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-08 05:29:33,755][130385] Avg episode reward: [(0, '51.790'), (1, '54.270')] [2023-10-08 05:29:34,730][00611] Updated weights for policy 0, policy_version 39812 (0.0007) [2023-10-08 05:29:35,096][00611] Updated weights for policy 0, policy_version 39822 (0.0007) [2023-10-08 05:29:35,469][00611] Updated weights for policy 0, policy_version 39832 (0.0009) [2023-10-08 05:29:37,255][00612] Updated weights for policy 1, policy_version 40040 (0.0009) [2023-10-08 05:29:37,628][00612] Updated weights for policy 1, policy_version 40050 (0.0008) [2023-10-08 05:29:37,994][00612] Updated weights for policy 1, policy_version 40060 (0.0010) [2023-10-08 05:29:38,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 81821696. Throughput: 0: 1849.3, 1: 1855.4. Samples: 20457840. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-08 05:29:38,755][130385] Avg episode reward: [(0, '54.270'), (1, '54.840')] [2023-10-08 05:29:39,157][00611] Updated weights for policy 0, policy_version 39842 (0.0009) [2023-10-08 05:29:39,527][00611] Updated weights for policy 0, policy_version 39852 (0.0007) [2023-10-08 05:29:39,896][00611] Updated weights for policy 0, policy_version 39862 (0.0008) [2023-10-08 05:29:40,269][00611] Updated weights for policy 0, policy_version 39872 (0.0007) [2023-10-08 05:29:41,599][00612] Updated weights for policy 1, policy_version 40070 (0.0010) [2023-10-08 05:29:41,965][00612] Updated weights for policy 1, policy_version 40080 (0.0011) [2023-10-08 05:29:42,337][00612] Updated weights for policy 1, policy_version 40090 (0.0011) [2023-10-08 05:29:43,744][00611] Updated weights for policy 0, policy_version 39882 (0.0008) [2023-10-08 05:29:43,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 81887232. Throughput: 0: 1855.8, 1: 1839.2. Samples: 20480190. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-08 05:29:43,754][130385] Avg episode reward: [(0, '53.570'), (1, '55.270')] [2023-10-08 05:29:44,119][00611] Updated weights for policy 0, policy_version 39892 (0.0007) [2023-10-08 05:29:44,492][00611] Updated weights for policy 0, policy_version 39902 (0.0007) [2023-10-08 05:29:45,780][00612] Updated weights for policy 1, policy_version 40100 (0.0009) [2023-10-08 05:29:46,141][00612] Updated weights for policy 1, policy_version 40110 (0.0007) [2023-10-08 05:29:46,512][00612] Updated weights for policy 1, policy_version 40120 (0.0010) [2023-10-08 05:29:48,066][00611] Updated weights for policy 0, policy_version 39912 (0.0009) [2023-10-08 05:29:48,445][00611] Updated weights for policy 0, policy_version 39922 (0.0010) [2023-10-08 05:29:48,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.3). Total num frames: 81952768. Throughput: 0: 1842.5, 1: 1865.0. Samples: 20502728. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-08 05:29:48,755][130385] Avg episode reward: [(0, '53.990'), (1, '51.630')] [2023-10-08 05:29:48,822][00611] Updated weights for policy 0, policy_version 39932 (0.0010) [2023-10-08 05:29:50,245][00612] Updated weights for policy 1, policy_version 40130 (0.0008) [2023-10-08 05:29:50,626][00612] Updated weights for policy 1, policy_version 40140 (0.0011) [2023-10-08 05:29:50,987][00612] Updated weights for policy 1, policy_version 40150 (0.0010) [2023-10-08 05:29:51,355][00612] Updated weights for policy 1, policy_version 40160 (0.0010) [2023-10-08 05:29:52,488][00611] Updated weights for policy 0, policy_version 39942 (0.0009) [2023-10-08 05:29:52,855][00611] Updated weights for policy 0, policy_version 39952 (0.0008) [2023-10-08 05:29:53,231][00611] Updated weights for policy 0, policy_version 39962 (0.0007) [2023-10-08 05:29:53,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 82051072. Throughput: 0: 1850.8, 1: 1838.9. Samples: 20513308. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-08 05:29:53,754][130385] Avg episode reward: [(0, '57.710'), (1, '52.490')] [2023-10-08 05:29:55,083][00612] Updated weights for policy 1, policy_version 40170 (0.0009) [2023-10-08 05:29:55,461][00612] Updated weights for policy 1, policy_version 40180 (0.0009) [2023-10-08 05:29:55,821][00612] Updated weights for policy 1, policy_version 40190 (0.0007) [2023-10-08 05:29:57,067][00611] Updated weights for policy 0, policy_version 39972 (0.0008) [2023-10-08 05:29:57,435][00611] Updated weights for policy 0, policy_version 39982 (0.0009) [2023-10-08 05:29:57,802][00611] Updated weights for policy 0, policy_version 39992 (0.0008) [2023-10-08 05:29:58,754][130385] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82116608. Throughput: 0: 1836.2, 1: 1857.2. Samples: 20535470. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 05:29:58,754][130385] Avg episode reward: [(0, '55.940'), (1, '52.950')] [2023-10-08 05:29:59,624][00612] Updated weights for policy 1, policy_version 40200 (0.0008) [2023-10-08 05:30:00,005][00612] Updated weights for policy 1, policy_version 40210 (0.0010) [2023-10-08 05:30:00,370][00612] Updated weights for policy 1, policy_version 40220 (0.0009) [2023-10-08 05:30:01,429][00611] Updated weights for policy 0, policy_version 40002 (0.0007) [2023-10-08 05:30:01,798][00611] Updated weights for policy 0, policy_version 40012 (0.0010) [2023-10-08 05:30:02,171][00611] Updated weights for policy 0, policy_version 40022 (0.0010) [2023-10-08 05:30:02,535][00611] Updated weights for policy 0, policy_version 40032 (0.0009) [2023-10-08 05:30:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82182144. Throughput: 0: 1842.8, 1: 1850.7. Samples: 20557144. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 05:30:03,754][130385] Avg episode reward: [(0, '54.770'), (1, '54.400')] [2023-10-08 05:30:03,982][00612] Updated weights for policy 1, policy_version 40230 (0.0008) [2023-10-08 05:30:04,351][00612] Updated weights for policy 1, policy_version 40240 (0.0009) [2023-10-08 05:30:04,720][00612] Updated weights for policy 1, policy_version 40250 (0.0008) [2023-10-08 05:30:06,226][00611] Updated weights for policy 0, policy_version 40042 (0.0007) [2023-10-08 05:30:06,603][00611] Updated weights for policy 0, policy_version 40052 (0.0009) [2023-10-08 05:30:06,965][00611] Updated weights for policy 0, policy_version 40062 (0.0008) [2023-10-08 05:30:08,261][00612] Updated weights for policy 1, policy_version 40260 (0.0009) [2023-10-08 05:30:08,625][00612] Updated weights for policy 1, policy_version 40270 (0.0009) [2023-10-08 05:30:08,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82247680. Throughput: 0: 1826.5, 1: 1857.9. Samples: 20568292. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 05:30:08,754][130385] Avg episode reward: [(0, '52.790'), (1, '54.920')] [2023-10-08 05:30:08,994][00612] Updated weights for policy 1, policy_version 40280 (0.0011) [2023-10-08 05:30:10,554][00611] Updated weights for policy 0, policy_version 40072 (0.0009) [2023-10-08 05:30:10,918][00611] Updated weights for policy 0, policy_version 40082 (0.0009) [2023-10-08 05:30:11,284][00611] Updated weights for policy 0, policy_version 40092 (0.0009) [2023-10-08 05:30:12,501][00612] Updated weights for policy 1, policy_version 40290 (0.0010) [2023-10-08 05:30:12,876][00612] Updated weights for policy 1, policy_version 40300 (0.0007) [2023-10-08 05:30:13,240][00612] Updated weights for policy 1, policy_version 40310 (0.0009) [2023-10-08 05:30:13,612][00612] Updated weights for policy 1, policy_version 40320 (0.0008) [2023-10-08 05:30:13,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 82345984. Throughput: 0: 1828.7, 1: 1867.8. Samples: 20590450. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 05:30:13,755][130385] Avg episode reward: [(0, '51.730'), (1, '54.550')] [2023-10-08 05:30:15,089][00611] Updated weights for policy 0, policy_version 40102 (0.0008) [2023-10-08 05:30:15,467][00611] Updated weights for policy 0, policy_version 40112 (0.0007) [2023-10-08 05:30:15,835][00611] Updated weights for policy 0, policy_version 40122 (0.0007) [2023-10-08 05:30:17,185][00612] Updated weights for policy 1, policy_version 40330 (0.0007) [2023-10-08 05:30:17,548][00612] Updated weights for policy 1, policy_version 40340 (0.0007) [2023-10-08 05:30:17,912][00612] Updated weights for policy 1, policy_version 40350 (0.0008) [2023-10-08 05:30:18,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 82411520. Throughput: 0: 1833.0, 1: 1845.6. Samples: 20612400. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 05:30:18,755][130385] Avg episode reward: [(0, '52.120'), (1, '56.770')] [2023-10-08 05:30:18,763][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000040128_41091072.pth... [2023-10-08 05:30:18,763][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000040352_41320448.pth... [2023-10-08 05:30:18,802][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000038624_39550976.pth [2023-10-08 05:30:18,803][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000038432_39354368.pth [2023-10-08 05:30:19,310][00611] Updated weights for policy 0, policy_version 40132 (0.0008) [2023-10-08 05:30:19,678][00611] Updated weights for policy 0, policy_version 40142 (0.0010) [2023-10-08 05:30:20,065][00611] Updated weights for policy 0, policy_version 40152 (0.0007) [2023-10-08 05:30:21,505][00612] Updated weights for policy 1, policy_version 40360 (0.0008) [2023-10-08 05:30:21,867][00612] Updated weights for policy 1, policy_version 40370 (0.0008) [2023-10-08 05:30:22,237][00612] Updated weights for policy 1, policy_version 40380 (0.0009) [2023-10-08 05:30:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 82477056. Throughput: 0: 1833.8, 1: 1858.4. Samples: 20623986. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 05:30:23,754][130385] Avg episode reward: [(0, '49.390'), (1, '57.490')] [2023-10-08 05:30:23,779][00611] Updated weights for policy 0, policy_version 40162 (0.0008) [2023-10-08 05:30:24,146][00611] Updated weights for policy 0, policy_version 40172 (0.0009) [2023-10-08 05:30:24,527][00611] Updated weights for policy 0, policy_version 40182 (0.0007) [2023-10-08 05:30:24,895][00611] Updated weights for policy 0, policy_version 40192 (0.0008) [2023-10-08 05:30:25,875][00612] Updated weights for policy 1, policy_version 40390 (0.0010) [2023-10-08 05:30:26,233][00612] Updated weights for policy 1, policy_version 40400 (0.0008) [2023-10-08 05:30:26,604][00612] Updated weights for policy 1, policy_version 40410 (0.0008) [2023-10-08 05:30:28,634][00611] Updated weights for policy 0, policy_version 40202 (0.0008) [2023-10-08 05:30:28,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82542592. Throughput: 0: 1838.6, 1: 1841.5. Samples: 20645794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:30:28,755][130385] Avg episode reward: [(0, '49.390'), (1, '60.130')] [2023-10-08 05:30:29,004][00611] Updated weights for policy 0, policy_version 40212 (0.0010) [2023-10-08 05:30:29,380][00611] Updated weights for policy 0, policy_version 40222 (0.0010) [2023-10-08 05:30:30,186][00612] Updated weights for policy 1, policy_version 40420 (0.0009) [2023-10-08 05:30:30,554][00612] Updated weights for policy 1, policy_version 40430 (0.0009) [2023-10-08 05:30:30,923][00612] Updated weights for policy 1, policy_version 40440 (0.0009) [2023-10-08 05:30:33,056][00611] Updated weights for policy 0, policy_version 40232 (0.0007) [2023-10-08 05:30:33,420][00611] Updated weights for policy 0, policy_version 40242 (0.0009) [2023-10-08 05:30:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82608128. Throughput: 0: 1826.3, 1: 1851.1. Samples: 20668212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:30:33,755][130385] Avg episode reward: [(0, '52.250'), (1, '59.200')] [2023-10-08 05:30:33,789][00611] Updated weights for policy 0, policy_version 40252 (0.0009) [2023-10-08 05:30:34,527][00612] Updated weights for policy 1, policy_version 40450 (0.0008) [2023-10-08 05:30:34,903][00612] Updated weights for policy 1, policy_version 40460 (0.0009) [2023-10-08 05:30:35,266][00612] Updated weights for policy 1, policy_version 40470 (0.0007) [2023-10-08 05:30:35,633][00612] Updated weights for policy 1, policy_version 40480 (0.0010) [2023-10-08 05:30:37,429][00611] Updated weights for policy 0, policy_version 40262 (0.0008) [2023-10-08 05:30:37,803][00611] Updated weights for policy 0, policy_version 40272 (0.0008) [2023-10-08 05:30:38,165][00611] Updated weights for policy 0, policy_version 40282 (0.0008) [2023-10-08 05:30:38,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 82706432. Throughput: 0: 1826.2, 1: 1850.0. Samples: 20678740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:30:38,754][130385] Avg episode reward: [(0, '51.370'), (1, '59.280')] [2023-10-08 05:30:39,203][00612] Updated weights for policy 1, policy_version 40490 (0.0011) [2023-10-08 05:30:39,571][00612] Updated weights for policy 1, policy_version 40500 (0.0010) [2023-10-08 05:30:39,950][00612] Updated weights for policy 1, policy_version 40510 (0.0011) [2023-10-08 05:30:41,784][00611] Updated weights for policy 0, policy_version 40292 (0.0010) [2023-10-08 05:30:42,165][00611] Updated weights for policy 0, policy_version 40302 (0.0011) [2023-10-08 05:30:42,527][00611] Updated weights for policy 0, policy_version 40312 (0.0010) [2023-10-08 05:30:43,659][00612] Updated weights for policy 1, policy_version 40520 (0.0007) [2023-10-08 05:30:43,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82771968. Throughput: 0: 1823.4, 1: 1861.2. Samples: 20701278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:30:43,755][130385] Avg episode reward: [(0, '51.670'), (1, '53.600')] [2023-10-08 05:30:44,040][00612] Updated weights for policy 1, policy_version 40530 (0.0010) [2023-10-08 05:30:44,406][00612] Updated weights for policy 1, policy_version 40540 (0.0010) [2023-10-08 05:30:46,177][00611] Updated weights for policy 0, policy_version 40322 (0.0009) [2023-10-08 05:30:46,560][00611] Updated weights for policy 0, policy_version 40332 (0.0008) [2023-10-08 05:30:46,937][00611] Updated weights for policy 0, policy_version 40342 (0.0007) [2023-10-08 05:30:47,307][00611] Updated weights for policy 0, policy_version 40352 (0.0007) [2023-10-08 05:30:48,046][00612] Updated weights for policy 1, policy_version 40550 (0.0009) [2023-10-08 05:30:48,432][00612] Updated weights for policy 1, policy_version 40560 (0.0008) [2023-10-08 05:30:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 82837504. Throughput: 0: 1828.2, 1: 1855.2. Samples: 20722894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:30:48,755][130385] Avg episode reward: [(0, '51.420'), (1, '54.050')] [2023-10-08 05:30:48,797][00612] Updated weights for policy 1, policy_version 40570 (0.0009) [2023-10-08 05:30:50,888][00611] Updated weights for policy 0, policy_version 40362 (0.0008) [2023-10-08 05:30:51,274][00611] Updated weights for policy 0, policy_version 40372 (0.0007) [2023-10-08 05:30:51,642][00611] Updated weights for policy 0, policy_version 40382 (0.0009) [2023-10-08 05:30:52,364][00612] Updated weights for policy 1, policy_version 40580 (0.0008) [2023-10-08 05:30:52,728][00612] Updated weights for policy 1, policy_version 40590 (0.0008) [2023-10-08 05:30:53,101][00612] Updated weights for policy 1, policy_version 40600 (0.0008) [2023-10-08 05:30:53,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 82935808. Throughput: 0: 1825.4, 1: 1865.1. Samples: 20734362. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 05:30:53,755][130385] Avg episode reward: [(0, '51.550'), (1, '52.610')] [2023-10-08 05:30:55,365][00611] Updated weights for policy 0, policy_version 40392 (0.0008) [2023-10-08 05:30:55,735][00611] Updated weights for policy 0, policy_version 40402 (0.0008) [2023-10-08 05:30:56,106][00611] Updated weights for policy 0, policy_version 40412 (0.0008) [2023-10-08 05:30:56,877][00612] Updated weights for policy 1, policy_version 40610 (0.0009) [2023-10-08 05:30:57,246][00612] Updated weights for policy 1, policy_version 40620 (0.0007) [2023-10-08 05:30:57,606][00612] Updated weights for policy 1, policy_version 40630 (0.0007) [2023-10-08 05:30:57,973][00612] Updated weights for policy 1, policy_version 40640 (0.0008) [2023-10-08 05:30:58,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 83001344. Throughput: 0: 1833.2, 1: 1837.4. Samples: 20755628. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 05:30:58,754][130385] Avg episode reward: [(0, '53.210'), (1, '55.220')] [2023-10-08 05:30:59,875][00611] Updated weights for policy 0, policy_version 40422 (0.0009) [2023-10-08 05:31:00,241][00611] Updated weights for policy 0, policy_version 40432 (0.0008) [2023-10-08 05:31:00,627][00611] Updated weights for policy 0, policy_version 40442 (0.0009) [2023-10-08 05:31:01,720][00612] Updated weights for policy 1, policy_version 40650 (0.0009) [2023-10-08 05:31:02,092][00612] Updated weights for policy 1, policy_version 40660 (0.0010) [2023-10-08 05:31:02,461][00612] Updated weights for policy 1, policy_version 40670 (0.0009) [2023-10-08 05:31:03,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 83066880. Throughput: 0: 1824.7, 1: 1841.8. Samples: 20777394. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 05:31:03,754][130385] Avg episode reward: [(0, '57.930'), (1, '54.430')] [2023-10-08 05:31:04,134][00611] Updated weights for policy 0, policy_version 40452 (0.0008) [2023-10-08 05:31:04,499][00611] Updated weights for policy 0, policy_version 40462 (0.0007) [2023-10-08 05:31:04,872][00611] Updated weights for policy 0, policy_version 40472 (0.0007) [2023-10-08 05:31:05,972][00612] Updated weights for policy 1, policy_version 40680 (0.0009) [2023-10-08 05:31:06,335][00612] Updated weights for policy 1, policy_version 40690 (0.0008) [2023-10-08 05:31:06,709][00612] Updated weights for policy 1, policy_version 40700 (0.0008) [2023-10-08 05:31:08,461][00611] Updated weights for policy 0, policy_version 40482 (0.0010) [2023-10-08 05:31:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83132416. Throughput: 0: 1824.1, 1: 1823.9. Samples: 20788146. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 05:31:08,754][130385] Avg episode reward: [(0, '58.610'), (1, '56.340')] [2023-10-08 05:31:08,815][00611] Updated weights for policy 0, policy_version 40492 (0.0010) [2023-10-08 05:31:09,194][00611] Updated weights for policy 0, policy_version 40502 (0.0011) [2023-10-08 05:31:09,569][00611] Updated weights for policy 0, policy_version 40512 (0.0011) [2023-10-08 05:31:10,432][00612] Updated weights for policy 1, policy_version 40710 (0.0008) [2023-10-08 05:31:10,804][00612] Updated weights for policy 1, policy_version 40720 (0.0009) [2023-10-08 05:31:11,180][00612] Updated weights for policy 1, policy_version 40730 (0.0008) [2023-10-08 05:31:13,309][00611] Updated weights for policy 0, policy_version 40522 (0.0009) [2023-10-08 05:31:13,688][00611] Updated weights for policy 0, policy_version 40532 (0.0010) [2023-10-08 05:31:13,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 83197952. Throughput: 0: 1821.5, 1: 1832.9. Samples: 20810240. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 05:31:13,755][130385] Avg episode reward: [(0, '57.420'), (1, '55.070')] [2023-10-08 05:31:14,046][00611] Updated weights for policy 0, policy_version 40542 (0.0010) [2023-10-08 05:31:14,859][00612] Updated weights for policy 1, policy_version 40740 (0.0007) [2023-10-08 05:31:15,223][00612] Updated weights for policy 1, policy_version 40750 (0.0008) [2023-10-08 05:31:15,587][00612] Updated weights for policy 1, policy_version 40760 (0.0008) [2023-10-08 05:31:17,758][00611] Updated weights for policy 0, policy_version 40552 (0.0009) [2023-10-08 05:31:18,135][00611] Updated weights for policy 0, policy_version 40562 (0.0007) [2023-10-08 05:31:18,508][00611] Updated weights for policy 0, policy_version 40572 (0.0009) [2023-10-08 05:31:18,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83296256. Throughput: 0: 1820.0, 1: 1828.2. Samples: 20832378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:31:18,754][130385] Avg episode reward: [(0, '58.720'), (1, '56.240')] [2023-10-08 05:31:19,378][00612] Updated weights for policy 1, policy_version 40770 (0.0009) [2023-10-08 05:31:19,741][00612] Updated weights for policy 1, policy_version 40780 (0.0007) [2023-10-08 05:31:20,118][00612] Updated weights for policy 1, policy_version 40790 (0.0007) [2023-10-08 05:31:20,490][00612] Updated weights for policy 1, policy_version 40800 (0.0008) [2023-10-08 05:31:22,319][00611] Updated weights for policy 0, policy_version 40582 (0.0009) [2023-10-08 05:31:22,696][00611] Updated weights for policy 0, policy_version 40592 (0.0008) [2023-10-08 05:31:23,072][00611] Updated weights for policy 0, policy_version 40602 (0.0007) [2023-10-08 05:31:23,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83361792. Throughput: 0: 1827.6, 1: 1826.8. Samples: 20843186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:31:23,754][130385] Avg episode reward: [(0, '61.070'), (1, '55.740')] [2023-10-08 05:31:24,174][00612] Updated weights for policy 1, policy_version 40810 (0.0010) [2023-10-08 05:31:24,538][00612] Updated weights for policy 1, policy_version 40820 (0.0010) [2023-10-08 05:31:24,910][00612] Updated weights for policy 1, policy_version 40830 (0.0010) [2023-10-08 05:31:26,809][00611] Updated weights for policy 0, policy_version 40612 (0.0009) [2023-10-08 05:31:27,172][00611] Updated weights for policy 0, policy_version 40622 (0.0008) [2023-10-08 05:31:27,547][00611] Updated weights for policy 0, policy_version 40632 (0.0011) [2023-10-08 05:31:28,581][00612] Updated weights for policy 1, policy_version 40840 (0.0008) [2023-10-08 05:31:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83427328. Throughput: 0: 1829.6, 1: 1820.2. Samples: 20865518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:31:28,754][130385] Avg episode reward: [(0, '59.400'), (1, '53.000')] [2023-10-08 05:31:28,953][00612] Updated weights for policy 1, policy_version 40850 (0.0008) [2023-10-08 05:31:29,320][00612] Updated weights for policy 1, policy_version 40860 (0.0009) [2023-10-08 05:31:31,151][00611] Updated weights for policy 0, policy_version 40642 (0.0007) [2023-10-08 05:31:31,521][00611] Updated weights for policy 0, policy_version 40652 (0.0007) [2023-10-08 05:31:31,906][00611] Updated weights for policy 0, policy_version 40662 (0.0008) [2023-10-08 05:31:32,283][00611] Updated weights for policy 0, policy_version 40672 (0.0008) [2023-10-08 05:31:33,025][00612] Updated weights for policy 1, policy_version 40870 (0.0009) [2023-10-08 05:31:33,396][00612] Updated weights for policy 1, policy_version 40880 (0.0008) [2023-10-08 05:31:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83492864. Throughput: 0: 1831.6, 1: 1819.6. Samples: 20887196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:31:33,754][130385] Avg episode reward: [(0, '59.840'), (1, '56.750')] [2023-10-08 05:31:33,766][00612] Updated weights for policy 1, policy_version 40890 (0.0007) [2023-10-08 05:31:35,894][00611] Updated weights for policy 0, policy_version 40682 (0.0009) [2023-10-08 05:31:36,261][00611] Updated weights for policy 0, policy_version 40692 (0.0010) [2023-10-08 05:31:36,631][00611] Updated weights for policy 0, policy_version 40702 (0.0009) [2023-10-08 05:31:37,531][00612] Updated weights for policy 1, policy_version 40900 (0.0010) [2023-10-08 05:31:37,901][00612] Updated weights for policy 1, policy_version 40910 (0.0007) [2023-10-08 05:31:38,269][00612] Updated weights for policy 1, policy_version 40920 (0.0009) [2023-10-08 05:31:38,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 83591168. Throughput: 0: 1826.4, 1: 1813.2. Samples: 20898142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:31:38,754][130385] Avg episode reward: [(0, '59.490'), (1, '58.470')] [2023-10-08 05:31:40,306][00611] Updated weights for policy 0, policy_version 40712 (0.0009) [2023-10-08 05:31:40,687][00611] Updated weights for policy 0, policy_version 40722 (0.0011) [2023-10-08 05:31:41,050][00611] Updated weights for policy 0, policy_version 40732 (0.0011) [2023-10-08 05:31:41,943][00612] Updated weights for policy 1, policy_version 40930 (0.0010) [2023-10-08 05:31:42,305][00612] Updated weights for policy 1, policy_version 40940 (0.0008) [2023-10-08 05:31:42,676][00612] Updated weights for policy 1, policy_version 40950 (0.0007) [2023-10-08 05:31:43,037][00612] Updated weights for policy 1, policy_version 40960 (0.0008) [2023-10-08 05:31:43,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 83656704. Throughput: 0: 1827.3, 1: 1821.2. Samples: 20919808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:31:43,754][130385] Avg episode reward: [(0, '57.780'), (1, '60.080')] [2023-10-08 05:31:44,567][00611] Updated weights for policy 0, policy_version 40742 (0.0010) [2023-10-08 05:31:44,930][00611] Updated weights for policy 0, policy_version 40752 (0.0009) [2023-10-08 05:31:45,304][00611] Updated weights for policy 0, policy_version 40762 (0.0011) [2023-10-08 05:31:46,591][00612] Updated weights for policy 1, policy_version 40970 (0.0011) [2023-10-08 05:31:46,958][00612] Updated weights for policy 1, policy_version 40980 (0.0009) [2023-10-08 05:31:47,313][00612] Updated weights for policy 1, policy_version 40990 (0.0007) [2023-10-08 05:31:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83722240. Throughput: 0: 1833.6, 1: 1826.0. Samples: 20942076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:31:48,754][130385] Avg episode reward: [(0, '58.820'), (1, '58.720')] [2023-10-08 05:31:49,002][00611] Updated weights for policy 0, policy_version 40772 (0.0009) [2023-10-08 05:31:49,372][00611] Updated weights for policy 0, policy_version 40782 (0.0007) [2023-10-08 05:31:49,750][00611] Updated weights for policy 0, policy_version 40792 (0.0009) [2023-10-08 05:31:50,855][00612] Updated weights for policy 1, policy_version 41000 (0.0009) [2023-10-08 05:31:51,226][00612] Updated weights for policy 1, policy_version 41010 (0.0007) [2023-10-08 05:31:51,596][00612] Updated weights for policy 1, policy_version 41020 (0.0008) [2023-10-08 05:31:53,445][00611] Updated weights for policy 0, policy_version 40802 (0.0009) [2023-10-08 05:31:53,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 83787776. Throughput: 0: 1833.1, 1: 1829.4. Samples: 20952958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:31:53,755][130385] Avg episode reward: [(0, '53.400'), (1, '60.980')] [2023-10-08 05:31:53,818][00611] Updated weights for policy 0, policy_version 40812 (0.0010) [2023-10-08 05:31:54,199][00611] Updated weights for policy 0, policy_version 40822 (0.0010) [2023-10-08 05:31:54,578][00611] Updated weights for policy 0, policy_version 40832 (0.0011) [2023-10-08 05:31:55,160][00612] Updated weights for policy 1, policy_version 41030 (0.0010) [2023-10-08 05:31:55,525][00612] Updated weights for policy 1, policy_version 41040 (0.0008) [2023-10-08 05:31:55,889][00612] Updated weights for policy 1, policy_version 41050 (0.0009) [2023-10-08 05:31:58,077][00611] Updated weights for policy 0, policy_version 40842 (0.0007) [2023-10-08 05:31:58,448][00611] Updated weights for policy 0, policy_version 40852 (0.0008) [2023-10-08 05:31:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 83853312. Throughput: 0: 1831.4, 1: 1837.9. Samples: 20975358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:31:58,754][130385] Avg episode reward: [(0, '53.800'), (1, '59.200')] [2023-10-08 05:31:58,824][00611] Updated weights for policy 0, policy_version 40862 (0.0008) [2023-10-08 05:31:59,670][00612] Updated weights for policy 1, policy_version 41060 (0.0009) [2023-10-08 05:32:00,037][00612] Updated weights for policy 1, policy_version 41070 (0.0008) [2023-10-08 05:32:00,403][00612] Updated weights for policy 1, policy_version 41080 (0.0009) [2023-10-08 05:32:02,551][00611] Updated weights for policy 0, policy_version 40872 (0.0009) [2023-10-08 05:32:02,939][00611] Updated weights for policy 0, policy_version 40882 (0.0008) [2023-10-08 05:32:03,314][00611] Updated weights for policy 0, policy_version 40892 (0.0008) [2023-10-08 05:32:03,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83951616. Throughput: 0: 1826.3, 1: 1843.1. Samples: 20997500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:32:03,755][130385] Avg episode reward: [(0, '52.240'), (1, '62.350')] [2023-10-08 05:32:03,875][00612] Updated weights for policy 1, policy_version 41090 (0.0009) [2023-10-08 05:32:04,240][00612] Updated weights for policy 1, policy_version 41100 (0.0008) [2023-10-08 05:32:04,614][00612] Updated weights for policy 1, policy_version 41110 (0.0008) [2023-10-08 05:32:04,986][00612] Updated weights for policy 1, policy_version 41120 (0.0011) [2023-10-08 05:32:06,927][00611] Updated weights for policy 0, policy_version 40902 (0.0009) [2023-10-08 05:32:07,297][00611] Updated weights for policy 0, policy_version 40912 (0.0008) [2023-10-08 05:32:07,666][00611] Updated weights for policy 0, policy_version 40922 (0.0009) [2023-10-08 05:32:08,467][00612] Updated weights for policy 1, policy_version 41130 (0.0008) [2023-10-08 05:32:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84017152. Throughput: 0: 1833.9, 1: 1841.3. Samples: 21008568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:32:08,755][130385] Avg episode reward: [(0, '54.190'), (1, '60.570')] [2023-10-08 05:32:08,829][00612] Updated weights for policy 1, policy_version 41140 (0.0008) [2023-10-08 05:32:09,196][00612] Updated weights for policy 1, policy_version 41150 (0.0010) [2023-10-08 05:32:11,359][00611] Updated weights for policy 0, policy_version 40932 (0.0008) [2023-10-08 05:32:11,738][00611] Updated weights for policy 0, policy_version 40942 (0.0009) [2023-10-08 05:32:12,108][00611] Updated weights for policy 0, policy_version 40952 (0.0008) [2023-10-08 05:32:13,059][00612] Updated weights for policy 1, policy_version 41160 (0.0010) [2023-10-08 05:32:13,428][00612] Updated weights for policy 1, policy_version 41170 (0.0011) [2023-10-08 05:32:13,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84082688. Throughput: 0: 1817.5, 1: 1849.4. Samples: 21030528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:32:13,754][130385] Avg episode reward: [(0, '54.300'), (1, '58.860')] [2023-10-08 05:32:13,798][00612] Updated weights for policy 1, policy_version 41180 (0.0010) [2023-10-08 05:32:15,800][00611] Updated weights for policy 0, policy_version 40962 (0.0008) [2023-10-08 05:32:16,178][00611] Updated weights for policy 0, policy_version 40972 (0.0007) [2023-10-08 05:32:16,549][00611] Updated weights for policy 0, policy_version 40982 (0.0008) [2023-10-08 05:32:16,922][00611] Updated weights for policy 0, policy_version 40992 (0.0008) [2023-10-08 05:32:17,479][00612] Updated weights for policy 1, policy_version 41190 (0.0008) [2023-10-08 05:32:17,846][00612] Updated weights for policy 1, policy_version 41200 (0.0008) [2023-10-08 05:32:18,212][00612] Updated weights for policy 1, policy_version 41210 (0.0008) [2023-10-08 05:32:18,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 84180992. Throughput: 0: 1828.7, 1: 1833.6. Samples: 21052000. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 05:32:18,755][130385] Avg episode reward: [(0, '53.270'), (1, '58.400')] [2023-10-08 05:32:18,765][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000041216_42205184.pth... [2023-10-08 05:32:18,765][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000040992_41975808.pth... [2023-10-08 05:32:18,805][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000039264_40206336.pth [2023-10-08 05:32:18,805][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000039488_40435712.pth [2023-10-08 05:32:20,605][00611] Updated weights for policy 0, policy_version 41002 (0.0007) [2023-10-08 05:32:20,982][00611] Updated weights for policy 0, policy_version 41012 (0.0010) [2023-10-08 05:32:21,363][00611] Updated weights for policy 0, policy_version 41022 (0.0008) [2023-10-08 05:32:22,055][00612] Updated weights for policy 1, policy_version 41220 (0.0010) [2023-10-08 05:32:22,447][00612] Updated weights for policy 1, policy_version 41230 (0.0008) [2023-10-08 05:32:22,817][00612] Updated weights for policy 1, policy_version 41240 (0.0007) [2023-10-08 05:32:23,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 84246528. Throughput: 0: 1823.7, 1: 1847.4. Samples: 21063344. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 05:32:23,755][130385] Avg episode reward: [(0, '54.330'), (1, '56.960')] [2023-10-08 05:32:24,971][00611] Updated weights for policy 0, policy_version 41032 (0.0008) [2023-10-08 05:32:25,347][00611] Updated weights for policy 0, policy_version 41042 (0.0007) [2023-10-08 05:32:25,713][00611] Updated weights for policy 0, policy_version 41052 (0.0011) [2023-10-08 05:32:26,311][00612] Updated weights for policy 1, policy_version 41250 (0.0008) [2023-10-08 05:32:26,673][00612] Updated weights for policy 1, policy_version 41260 (0.0011) [2023-10-08 05:32:27,030][00612] Updated weights for policy 1, policy_version 41270 (0.0009) [2023-10-08 05:32:27,401][00612] Updated weights for policy 1, policy_version 41280 (0.0007) [2023-10-08 05:32:28,755][130385] Fps is (10 sec: 13106.4, 60 sec: 14745.4, 300 sec: 14662.3). Total num frames: 84312064. Throughput: 0: 1837.9, 1: 1829.2. Samples: 21084834. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 05:32:28,755][130385] Avg episode reward: [(0, '54.360'), (1, '56.890')] [2023-10-08 05:32:29,298][00611] Updated weights for policy 0, policy_version 41062 (0.0010) [2023-10-08 05:32:29,676][00611] Updated weights for policy 0, policy_version 41072 (0.0008) [2023-10-08 05:32:30,048][00611] Updated weights for policy 0, policy_version 41082 (0.0009) [2023-10-08 05:32:30,976][00612] Updated weights for policy 1, policy_version 41290 (0.0010) [2023-10-08 05:32:31,337][00612] Updated weights for policy 1, policy_version 41300 (0.0009) [2023-10-08 05:32:31,707][00612] Updated weights for policy 1, policy_version 41310 (0.0008) [2023-10-08 05:32:33,668][00611] Updated weights for policy 0, policy_version 41092 (0.0008) [2023-10-08 05:32:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 84377600. Throughput: 0: 1834.8, 1: 1849.9. Samples: 21107890. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 05:32:33,755][130385] Avg episode reward: [(0, '56.020'), (1, '58.610')] [2023-10-08 05:32:34,043][00611] Updated weights for policy 0, policy_version 41102 (0.0010) [2023-10-08 05:32:34,413][00611] Updated weights for policy 0, policy_version 41112 (0.0010) [2023-10-08 05:32:35,323][00612] Updated weights for policy 1, policy_version 41320 (0.0007) [2023-10-08 05:32:35,692][00612] Updated weights for policy 1, policy_version 41330 (0.0008) [2023-10-08 05:32:36,067][00612] Updated weights for policy 1, policy_version 41340 (0.0009) [2023-10-08 05:32:38,090][00611] Updated weights for policy 0, policy_version 41122 (0.0009) [2023-10-08 05:32:38,465][00611] Updated weights for policy 0, policy_version 41132 (0.0010) [2023-10-08 05:32:38,754][130385] Fps is (10 sec: 13108.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 84443136. Throughput: 0: 1836.5, 1: 1830.1. Samples: 21117954. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 05:32:38,754][130385] Avg episode reward: [(0, '55.310'), (1, '58.910')] [2023-10-08 05:32:38,838][00611] Updated weights for policy 0, policy_version 41142 (0.0008) [2023-10-08 05:32:39,215][00611] Updated weights for policy 0, policy_version 41152 (0.0008) [2023-10-08 05:32:39,743][00612] Updated weights for policy 1, policy_version 41350 (0.0008) [2023-10-08 05:32:40,113][00612] Updated weights for policy 1, policy_version 41360 (0.0009) [2023-10-08 05:32:40,487][00612] Updated weights for policy 1, policy_version 41370 (0.0007) [2023-10-08 05:32:42,839][00611] Updated weights for policy 0, policy_version 41162 (0.0011) [2023-10-08 05:32:43,217][00611] Updated weights for policy 0, policy_version 41172 (0.0010) [2023-10-08 05:32:43,589][00611] Updated weights for policy 0, policy_version 41182 (0.0009) [2023-10-08 05:32:43,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 84541440. Throughput: 0: 1836.0, 1: 1843.9. Samples: 21140954. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 05:32:43,755][130385] Avg episode reward: [(0, '55.530'), (1, '55.600')] [2023-10-08 05:32:44,046][00612] Updated weights for policy 1, policy_version 41380 (0.0008) [2023-10-08 05:32:44,417][00612] Updated weights for policy 1, policy_version 41390 (0.0008) [2023-10-08 05:32:44,779][00612] Updated weights for policy 1, policy_version 41400 (0.0007) [2023-10-08 05:32:47,286][00611] Updated weights for policy 0, policy_version 41192 (0.0008) [2023-10-08 05:32:47,658][00611] Updated weights for policy 0, policy_version 41202 (0.0010) [2023-10-08 05:32:48,039][00611] Updated weights for policy 0, policy_version 41212 (0.0009) [2023-10-08 05:32:48,300][00612] Updated weights for policy 1, policy_version 41410 (0.0007) [2023-10-08 05:32:48,671][00612] Updated weights for policy 1, policy_version 41420 (0.0010) [2023-10-08 05:32:48,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84606976. Throughput: 0: 1826.1, 1: 1849.2. Samples: 21162888. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 05:32:48,755][130385] Avg episode reward: [(0, '56.800'), (1, '56.790')] [2023-10-08 05:32:49,039][00612] Updated weights for policy 1, policy_version 41430 (0.0007) [2023-10-08 05:32:49,402][00612] Updated weights for policy 1, policy_version 41440 (0.0010) [2023-10-08 05:32:51,740][00611] Updated weights for policy 0, policy_version 41222 (0.0007) [2023-10-08 05:32:52,118][00611] Updated weights for policy 0, policy_version 41232 (0.0009) [2023-10-08 05:32:52,491][00611] Updated weights for policy 0, policy_version 41242 (0.0008) [2023-10-08 05:32:53,058][00612] Updated weights for policy 1, policy_version 41450 (0.0007) [2023-10-08 05:32:53,424][00612] Updated weights for policy 1, policy_version 41460 (0.0009) [2023-10-08 05:32:53,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84672512. Throughput: 0: 1833.6, 1: 1846.8. Samples: 21174188. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 05:32:53,754][130385] Avg episode reward: [(0, '58.070'), (1, '57.440')] [2023-10-08 05:32:53,796][00612] Updated weights for policy 1, policy_version 41470 (0.0007) [2023-10-08 05:32:56,125][00611] Updated weights for policy 0, policy_version 41252 (0.0010) [2023-10-08 05:32:56,496][00611] Updated weights for policy 0, policy_version 41262 (0.0009) [2023-10-08 05:32:56,859][00611] Updated weights for policy 0, policy_version 41272 (0.0007) [2023-10-08 05:32:57,449][00612] Updated weights for policy 1, policy_version 41480 (0.0007) [2023-10-08 05:32:57,821][00612] Updated weights for policy 1, policy_version 41490 (0.0007) [2023-10-08 05:32:58,191][00612] Updated weights for policy 1, policy_version 41500 (0.0008) [2023-10-08 05:32:58,754][130385] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 84770816. Throughput: 0: 1830.0, 1: 1842.6. Samples: 21195796. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 05:32:58,754][130385] Avg episode reward: [(0, '55.670'), (1, '57.170')] [2023-10-08 05:33:00,421][00611] Updated weights for policy 0, policy_version 41282 (0.0008) [2023-10-08 05:33:00,781][00611] Updated weights for policy 0, policy_version 41292 (0.0009) [2023-10-08 05:33:01,160][00611] Updated weights for policy 0, policy_version 41302 (0.0007) [2023-10-08 05:33:01,536][00611] Updated weights for policy 0, policy_version 41312 (0.0008) [2023-10-08 05:33:01,774][00612] Updated weights for policy 1, policy_version 41510 (0.0008) [2023-10-08 05:33:02,152][00612] Updated weights for policy 1, policy_version 41520 (0.0009) [2023-10-08 05:33:02,520][00612] Updated weights for policy 1, policy_version 41530 (0.0008) [2023-10-08 05:33:03,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 84836352. Throughput: 0: 1839.6, 1: 1839.1. Samples: 21217540. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 05:33:03,754][130385] Avg episode reward: [(0, '54.750'), (1, '59.570')] [2023-10-08 05:33:05,148][00611] Updated weights for policy 0, policy_version 41322 (0.0007) [2023-10-08 05:33:05,517][00611] Updated weights for policy 0, policy_version 41332 (0.0009) [2023-10-08 05:33:05,885][00611] Updated weights for policy 0, policy_version 41342 (0.0008) [2023-10-08 05:33:06,120][00612] Updated weights for policy 1, policy_version 41540 (0.0007) [2023-10-08 05:33:06,501][00612] Updated weights for policy 1, policy_version 41550 (0.0010) [2023-10-08 05:33:06,863][00612] Updated weights for policy 1, policy_version 41560 (0.0008) [2023-10-08 05:33:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 84901888. Throughput: 0: 1828.8, 1: 1848.4. Samples: 21228820. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 05:33:08,754][130385] Avg episode reward: [(0, '56.320'), (1, '64.670')] [2023-10-08 05:33:09,459][00611] Updated weights for policy 0, policy_version 41352 (0.0010) [2023-10-08 05:33:09,825][00611] Updated weights for policy 0, policy_version 41362 (0.0010) [2023-10-08 05:33:10,205][00611] Updated weights for policy 0, policy_version 41372 (0.0010) [2023-10-08 05:33:10,607][00612] Updated weights for policy 1, policy_version 41570 (0.0008) [2023-10-08 05:33:11,024][00612] Updated weights for policy 1, policy_version 41580 (0.0007) [2023-10-08 05:33:11,392][00612] Updated weights for policy 1, policy_version 41590 (0.0008) [2023-10-08 05:33:11,758][00612] Updated weights for policy 1, policy_version 41600 (0.0007) [2023-10-08 05:33:13,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84967424. Throughput: 0: 1840.5, 1: 1845.4. Samples: 21250696. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 05:33:13,755][130385] Avg episode reward: [(0, '55.980'), (1, '63.050')] [2023-10-08 05:33:13,853][00611] Updated weights for policy 0, policy_version 41382 (0.0009) [2023-10-08 05:33:14,230][00611] Updated weights for policy 0, policy_version 41392 (0.0008) [2023-10-08 05:33:14,601][00611] Updated weights for policy 0, policy_version 41402 (0.0007) [2023-10-08 05:33:15,266][00612] Updated weights for policy 1, policy_version 41610 (0.0009) [2023-10-08 05:33:15,624][00612] Updated weights for policy 1, policy_version 41620 (0.0007) [2023-10-08 05:33:15,999][00612] Updated weights for policy 1, policy_version 41630 (0.0009) [2023-10-08 05:33:18,323][00611] Updated weights for policy 0, policy_version 41412 (0.0009) [2023-10-08 05:33:18,707][00611] Updated weights for policy 0, policy_version 41422 (0.0010) [2023-10-08 05:33:18,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14662.3). Total num frames: 85032960. Throughput: 0: 1833.5, 1: 1849.5. Samples: 21273622. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 05:33:18,754][130385] Avg episode reward: [(0, '53.480'), (1, '60.980')] [2023-10-08 05:33:19,076][00611] Updated weights for policy 0, policy_version 41432 (0.0009) [2023-10-08 05:33:19,677][00612] Updated weights for policy 1, policy_version 41640 (0.0008) [2023-10-08 05:33:20,051][00612] Updated weights for policy 1, policy_version 41650 (0.0007) [2023-10-08 05:33:20,425][00612] Updated weights for policy 1, policy_version 41660 (0.0008) [2023-10-08 05:33:22,889][00611] Updated weights for policy 0, policy_version 41442 (0.0011) [2023-10-08 05:33:23,262][00611] Updated weights for policy 0, policy_version 41452 (0.0009) [2023-10-08 05:33:23,621][00611] Updated weights for policy 0, policy_version 41462 (0.0010) [2023-10-08 05:33:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 85098496. Throughput: 0: 1831.4, 1: 1849.9. Samples: 21283614. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 05:33:23,754][130385] Avg episode reward: [(0, '48.920'), (1, '61.570')] [2023-10-08 05:33:23,997][00611] Updated weights for policy 0, policy_version 41472 (0.0009) [2023-10-08 05:33:24,152][00612] Updated weights for policy 1, policy_version 41670 (0.0008) [2023-10-08 05:33:24,524][00612] Updated weights for policy 1, policy_version 41680 (0.0011) [2023-10-08 05:33:24,887][00612] Updated weights for policy 1, policy_version 41690 (0.0010) [2023-10-08 05:33:27,448][00611] Updated weights for policy 0, policy_version 41482 (0.0008) [2023-10-08 05:33:27,816][00611] Updated weights for policy 0, policy_version 41492 (0.0010) [2023-10-08 05:33:28,196][00611] Updated weights for policy 0, policy_version 41502 (0.0009) [2023-10-08 05:33:28,545][00612] Updated weights for policy 1, policy_version 41700 (0.0010) [2023-10-08 05:33:28,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.8, 300 sec: 14773.4). Total num frames: 85196800. Throughput: 0: 1826.6, 1: 1848.8. Samples: 21306346. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 05:33:28,755][130385] Avg episode reward: [(0, '51.180'), (1, '64.240')] [2023-10-08 05:33:28,910][00612] Updated weights for policy 1, policy_version 41710 (0.0007) [2023-10-08 05:33:29,280][00612] Updated weights for policy 1, policy_version 41720 (0.0009) [2023-10-08 05:33:32,068][00611] Updated weights for policy 0, policy_version 41512 (0.0009) [2023-10-08 05:33:32,439][00611] Updated weights for policy 0, policy_version 41522 (0.0010) [2023-10-08 05:33:32,809][00611] Updated weights for policy 0, policy_version 41532 (0.0009) [2023-10-08 05:33:33,005][00612] Updated weights for policy 1, policy_version 41730 (0.0007) [2023-10-08 05:33:33,368][00612] Updated weights for policy 1, policy_version 41740 (0.0008) [2023-10-08 05:33:33,737][00612] Updated weights for policy 1, policy_version 41750 (0.0008) [2023-10-08 05:33:33,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85262336. Throughput: 0: 1825.6, 1: 1835.8. Samples: 21327654. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 05:33:33,755][130385] Avg episode reward: [(0, '50.670'), (1, '62.880')] [2023-10-08 05:33:34,108][00612] Updated weights for policy 1, policy_version 41760 (0.0008) [2023-10-08 05:33:36,463][00611] Updated weights for policy 0, policy_version 41542 (0.0007) [2023-10-08 05:33:36,849][00611] Updated weights for policy 0, policy_version 41552 (0.0008) [2023-10-08 05:33:37,225][00611] Updated weights for policy 0, policy_version 41562 (0.0007) [2023-10-08 05:33:37,669][00612] Updated weights for policy 1, policy_version 41770 (0.0007) [2023-10-08 05:33:38,041][00612] Updated weights for policy 1, policy_version 41780 (0.0007) [2023-10-08 05:33:38,405][00612] Updated weights for policy 1, policy_version 41790 (0.0008) [2023-10-08 05:33:38,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 85360640. Throughput: 0: 1833.9, 1: 1845.9. Samples: 21339782. Policy #0 lag: (min: 11.0, avg: 36.1, max: 40.0) [2023-10-08 05:33:38,755][130385] Avg episode reward: [(0, '53.880'), (1, '61.900')] [2023-10-08 05:33:40,839][00611] Updated weights for policy 0, policy_version 41572 (0.0010) [2023-10-08 05:33:41,209][00611] Updated weights for policy 0, policy_version 41582 (0.0009) [2023-10-08 05:33:41,585][00611] Updated weights for policy 0, policy_version 41592 (0.0010) [2023-10-08 05:33:41,879][00612] Updated weights for policy 1, policy_version 41800 (0.0007) [2023-10-08 05:33:42,245][00612] Updated weights for policy 1, policy_version 41810 (0.0010) [2023-10-08 05:33:42,614][00612] Updated weights for policy 1, policy_version 41820 (0.0010) [2023-10-08 05:33:43,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 85426176. Throughput: 0: 1828.9, 1: 1838.0. Samples: 21360808. Policy #0 lag: (min: 11.0, avg: 36.1, max: 40.0) [2023-10-08 05:33:43,755][130385] Avg episode reward: [(0, '52.380'), (1, '66.810')] [2023-10-08 05:33:43,757][00425] Saving new best policy, reward=66.810! [2023-10-08 05:33:45,449][00611] Updated weights for policy 0, policy_version 41602 (0.0010) [2023-10-08 05:33:45,827][00611] Updated weights for policy 0, policy_version 41612 (0.0008) [2023-10-08 05:33:46,197][00611] Updated weights for policy 0, policy_version 41622 (0.0007) [2023-10-08 05:33:46,278][00612] Updated weights for policy 1, policy_version 41830 (0.0008) [2023-10-08 05:33:46,571][00611] Updated weights for policy 0, policy_version 41632 (0.0008) [2023-10-08 05:33:46,649][00612] Updated weights for policy 1, policy_version 41840 (0.0008) [2023-10-08 05:33:47,021][00612] Updated weights for policy 1, policy_version 41850 (0.0007) [2023-10-08 05:33:48,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.3). Total num frames: 85491712. Throughput: 0: 1827.9, 1: 1850.7. Samples: 21383082. Policy #0 lag: (min: 11.0, avg: 36.1, max: 40.0) [2023-10-08 05:33:48,755][130385] Avg episode reward: [(0, '51.280'), (1, '65.630')] [2023-10-08 05:33:50,118][00611] Updated weights for policy 0, policy_version 41642 (0.0008) [2023-10-08 05:33:50,485][00611] Updated weights for policy 0, policy_version 41652 (0.0007) [2023-10-08 05:33:50,653][00612] Updated weights for policy 1, policy_version 41860 (0.0008) [2023-10-08 05:33:50,859][00611] Updated weights for policy 0, policy_version 41662 (0.0007) [2023-10-08 05:33:51,021][00612] Updated weights for policy 1, policy_version 41870 (0.0009) [2023-10-08 05:33:51,386][00612] Updated weights for policy 1, policy_version 41880 (0.0007) [2023-10-08 05:33:53,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85557248. Throughput: 0: 1827.3, 1: 1836.6. Samples: 21393696. Policy #0 lag: (min: 11.0, avg: 36.1, max: 40.0) [2023-10-08 05:33:53,754][130385] Avg episode reward: [(0, '51.340'), (1, '66.110')] [2023-10-08 05:33:54,543][00611] Updated weights for policy 0, policy_version 41672 (0.0009) [2023-10-08 05:33:54,883][00612] Updated weights for policy 1, policy_version 41890 (0.0008) [2023-10-08 05:33:54,900][00611] Updated weights for policy 0, policy_version 41682 (0.0010) [2023-10-08 05:33:55,250][00612] Updated weights for policy 1, policy_version 41900 (0.0007) [2023-10-08 05:33:55,271][00611] Updated weights for policy 0, policy_version 41692 (0.0009) [2023-10-08 05:33:55,623][00612] Updated weights for policy 1, policy_version 41910 (0.0007) [2023-10-08 05:33:55,987][00612] Updated weights for policy 1, policy_version 41920 (0.0007) [2023-10-08 05:33:58,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 85622784. Throughput: 0: 1820.3, 1: 1860.8. Samples: 21416346. Policy #0 lag: (min: 11.0, avg: 36.1, max: 40.0) [2023-10-08 05:33:58,754][130385] Avg episode reward: [(0, '54.270'), (1, '63.300')] [2023-10-08 05:33:59,028][00611] Updated weights for policy 0, policy_version 41702 (0.0010) [2023-10-08 05:33:59,396][00611] Updated weights for policy 0, policy_version 41712 (0.0008) [2023-10-08 05:33:59,577][00612] Updated weights for policy 1, policy_version 41930 (0.0008) [2023-10-08 05:33:59,771][00611] Updated weights for policy 0, policy_version 41722 (0.0007) [2023-10-08 05:33:59,954][00612] Updated weights for policy 1, policy_version 41940 (0.0007) [2023-10-08 05:34:00,322][00612] Updated weights for policy 1, policy_version 41950 (0.0010) [2023-10-08 05:34:03,278][00611] Updated weights for policy 0, policy_version 41732 (0.0008) [2023-10-08 05:34:03,647][00611] Updated weights for policy 0, policy_version 41742 (0.0010) [2023-10-08 05:34:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 85688320. Throughput: 0: 1826.3, 1: 1861.1. Samples: 21439552. Policy #0 lag: (min: 11.0, avg: 36.1, max: 40.0) [2023-10-08 05:34:03,754][130385] Avg episode reward: [(0, '54.430'), (1, '60.200')] [2023-10-08 05:34:03,937][00612] Updated weights for policy 1, policy_version 41960 (0.0009) [2023-10-08 05:34:04,005][00611] Updated weights for policy 0, policy_version 41752 (0.0007) [2023-10-08 05:34:04,303][00612] Updated weights for policy 1, policy_version 41970 (0.0007) [2023-10-08 05:34:04,670][00612] Updated weights for policy 1, policy_version 41980 (0.0009) [2023-10-08 05:34:07,721][00611] Updated weights for policy 0, policy_version 41762 (0.0007) [2023-10-08 05:34:08,103][00611] Updated weights for policy 0, policy_version 41772 (0.0007) [2023-10-08 05:34:08,153][00612] Updated weights for policy 1, policy_version 41990 (0.0008) [2023-10-08 05:34:08,463][00611] Updated weights for policy 0, policy_version 41782 (0.0007) [2023-10-08 05:34:08,519][00612] Updated weights for policy 1, policy_version 42000 (0.0009) [2023-10-08 05:34:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 85753856. Throughput: 0: 1828.0, 1: 1856.8. Samples: 21449430. Policy #0 lag: (min: 25.0, avg: 37.6, max: 57.0) [2023-10-08 05:34:08,755][130385] Avg episode reward: [(0, '56.320'), (1, '59.000')] [2023-10-08 05:34:08,833][00611] Updated weights for policy 0, policy_version 41792 (0.0009) [2023-10-08 05:34:08,870][00612] Updated weights for policy 1, policy_version 42010 (0.0009) [2023-10-08 05:34:12,673][00611] Updated weights for policy 0, policy_version 41802 (0.0007) [2023-10-08 05:34:12,747][00612] Updated weights for policy 1, policy_version 42020 (0.0009) [2023-10-08 05:34:13,053][00611] Updated weights for policy 0, policy_version 41812 (0.0007) [2023-10-08 05:34:13,108][00612] Updated weights for policy 1, policy_version 42030 (0.0007) [2023-10-08 05:34:13,414][00611] Updated weights for policy 0, policy_version 41822 (0.0009) [2023-10-08 05:34:13,473][00612] Updated weights for policy 1, policy_version 42040 (0.0009) [2023-10-08 05:34:13,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85852160. Throughput: 0: 1825.4, 1: 1857.7. Samples: 21472086. Policy #0 lag: (min: 25.0, avg: 37.6, max: 57.0) [2023-10-08 05:34:13,755][130385] Avg episode reward: [(0, '58.440'), (1, '62.190')] [2023-10-08 05:34:17,031][00612] Updated weights for policy 1, policy_version 42050 (0.0011) [2023-10-08 05:34:17,092][00611] Updated weights for policy 0, policy_version 41832 (0.0008) [2023-10-08 05:34:17,403][00612] Updated weights for policy 1, policy_version 42060 (0.0009) [2023-10-08 05:34:17,472][00611] Updated weights for policy 0, policy_version 41842 (0.0007) [2023-10-08 05:34:17,765][00612] Updated weights for policy 1, policy_version 42070 (0.0007) [2023-10-08 05:34:17,850][00611] Updated weights for policy 0, policy_version 41852 (0.0007) [2023-10-08 05:34:18,131][00612] Updated weights for policy 1, policy_version 42080 (0.0009) [2023-10-08 05:34:18,754][130385] Fps is (10 sec: 19660.2, 60 sec: 15291.6, 300 sec: 14773.4). Total num frames: 85950464. Throughput: 0: 1820.8, 1: 1832.4. Samples: 21492050. Policy #0 lag: (min: 25.0, avg: 37.6, max: 57.0) [2023-10-08 05:34:18,755][130385] Avg episode reward: [(0, '60.530'), (1, '64.170')] [2023-10-08 05:34:18,767][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000042080_43089920.pth... [2023-10-08 05:34:18,767][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000041856_42860544.pth... [2023-10-08 05:34:18,812][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000040128_41091072.pth [2023-10-08 05:34:18,813][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000040352_41320448.pth [2023-10-08 05:34:21,526][00611] Updated weights for policy 0, policy_version 41862 (0.0009) [2023-10-08 05:34:21,692][00612] Updated weights for policy 1, policy_version 42090 (0.0007) [2023-10-08 05:34:21,899][00611] Updated weights for policy 0, policy_version 41872 (0.0007) [2023-10-08 05:34:22,071][00612] Updated weights for policy 1, policy_version 42100 (0.0009) [2023-10-08 05:34:22,267][00611] Updated weights for policy 0, policy_version 41882 (0.0007) [2023-10-08 05:34:22,438][00612] Updated weights for policy 1, policy_version 42110 (0.0011) [2023-10-08 05:34:23,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 86016000. Throughput: 0: 1818.2, 1: 1853.9. Samples: 21505028. Policy #0 lag: (min: 25.0, avg: 37.6, max: 57.0) [2023-10-08 05:34:23,754][130385] Avg episode reward: [(0, '60.100'), (1, '63.330')] [2023-10-08 05:34:26,050][00611] Updated weights for policy 0, policy_version 41892 (0.0007) [2023-10-08 05:34:26,077][00612] Updated weights for policy 1, policy_version 42120 (0.0009) [2023-10-08 05:34:26,413][00611] Updated weights for policy 0, policy_version 41902 (0.0007) [2023-10-08 05:34:26,451][00612] Updated weights for policy 1, policy_version 42130 (0.0009) [2023-10-08 05:34:26,793][00611] Updated weights for policy 0, policy_version 41912 (0.0008) [2023-10-08 05:34:26,811][00612] Updated weights for policy 1, policy_version 42140 (0.0007) [2023-10-08 05:34:28,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 86081536. Throughput: 0: 1815.7, 1: 1826.9. Samples: 21524722. Policy #0 lag: (min: 25.0, avg: 37.6, max: 57.0) [2023-10-08 05:34:28,754][130385] Avg episode reward: [(0, '56.550'), (1, '62.700')] [2023-10-08 05:34:30,274][00611] Updated weights for policy 0, policy_version 41922 (0.0010) [2023-10-08 05:34:30,454][00612] Updated weights for policy 1, policy_version 42150 (0.0007) [2023-10-08 05:34:30,643][00611] Updated weights for policy 0, policy_version 41932 (0.0010) [2023-10-08 05:34:30,827][00612] Updated weights for policy 1, policy_version 42160 (0.0009) [2023-10-08 05:34:31,000][00611] Updated weights for policy 0, policy_version 41942 (0.0007) [2023-10-08 05:34:31,192][00612] Updated weights for policy 1, policy_version 42170 (0.0008) [2023-10-08 05:34:31,370][00611] Updated weights for policy 0, policy_version 41952 (0.0007) [2023-10-08 05:34:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86147072. Throughput: 0: 1819.3, 1: 1846.1. Samples: 21548028. Policy #0 lag: (min: 25.0, avg: 37.6, max: 57.0) [2023-10-08 05:34:33,754][130385] Avg episode reward: [(0, '55.970'), (1, '65.340')] [2023-10-08 05:34:34,913][00612] Updated weights for policy 1, policy_version 42180 (0.0007) [2023-10-08 05:34:35,158][00611] Updated weights for policy 0, policy_version 41962 (0.0008) [2023-10-08 05:34:35,281][00612] Updated weights for policy 1, policy_version 42190 (0.0007) [2023-10-08 05:34:35,535][00611] Updated weights for policy 0, policy_version 41972 (0.0009) [2023-10-08 05:34:35,646][00612] Updated weights for policy 1, policy_version 42200 (0.0009) [2023-10-08 05:34:35,897][00611] Updated weights for policy 0, policy_version 41982 (0.0009) [2023-10-08 05:34:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 86212608. Throughput: 0: 1818.2, 1: 1830.5. Samples: 21557888. Policy #0 lag: (min: 23.0, avg: 23.1, max: 31.0) [2023-10-08 05:34:38,754][130385] Avg episode reward: [(0, '59.620'), (1, '64.970')] [2023-10-08 05:34:39,310][00612] Updated weights for policy 1, policy_version 42210 (0.0007) [2023-10-08 05:34:39,607][00611] Updated weights for policy 0, policy_version 41992 (0.0008) [2023-10-08 05:34:39,686][00612] Updated weights for policy 1, policy_version 42220 (0.0007) [2023-10-08 05:34:39,971][00611] Updated weights for policy 0, policy_version 42002 (0.0008) [2023-10-08 05:34:40,050][00612] Updated weights for policy 1, policy_version 42230 (0.0007) [2023-10-08 05:34:40,341][00611] Updated weights for policy 0, policy_version 42012 (0.0008) [2023-10-08 05:34:40,424][00612] Updated weights for policy 1, policy_version 42240 (0.0008) [2023-10-08 05:34:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 86278144. Throughput: 0: 1816.9, 1: 1838.9. Samples: 21580860. Policy #0 lag: (min: 23.0, avg: 23.1, max: 31.0) [2023-10-08 05:34:43,754][130385] Avg episode reward: [(0, '61.320'), (1, '60.760')] [2023-10-08 05:34:44,031][00611] Updated weights for policy 0, policy_version 42022 (0.0008) [2023-10-08 05:34:44,085][00612] Updated weights for policy 1, policy_version 42250 (0.0009) [2023-10-08 05:34:44,392][00611] Updated weights for policy 0, policy_version 42032 (0.0007) [2023-10-08 05:34:44,446][00612] Updated weights for policy 1, policy_version 42260 (0.0008) [2023-10-08 05:34:44,771][00611] Updated weights for policy 0, policy_version 42042 (0.0007) [2023-10-08 05:34:44,822][00612] Updated weights for policy 1, policy_version 42270 (0.0009) [2023-10-08 05:34:48,538][00611] Updated weights for policy 0, policy_version 42052 (0.0008) [2023-10-08 05:34:48,551][00612] Updated weights for policy 1, policy_version 42280 (0.0007) [2023-10-08 05:34:48,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 86343680. Throughput: 0: 1816.8, 1: 1836.0. Samples: 21603930. Policy #0 lag: (min: 23.0, avg: 23.1, max: 31.0) [2023-10-08 05:34:48,755][130385] Avg episode reward: [(0, '57.950'), (1, '59.890')] [2023-10-08 05:34:48,904][00611] Updated weights for policy 0, policy_version 42062 (0.0008) [2023-10-08 05:34:48,918][00612] Updated weights for policy 1, policy_version 42290 (0.0008) [2023-10-08 05:34:49,280][00611] Updated weights for policy 0, policy_version 42072 (0.0009) [2023-10-08 05:34:49,287][00612] Updated weights for policy 1, policy_version 42300 (0.0007) [2023-10-08 05:34:52,994][00611] Updated weights for policy 0, policy_version 42082 (0.0008) [2023-10-08 05:34:53,024][00612] Updated weights for policy 1, policy_version 42310 (0.0008) [2023-10-08 05:34:53,369][00611] Updated weights for policy 0, policy_version 42092 (0.0008) [2023-10-08 05:34:53,402][00612] Updated weights for policy 1, policy_version 42320 (0.0007) [2023-10-08 05:34:53,741][00611] Updated weights for policy 0, policy_version 42102 (0.0009) [2023-10-08 05:34:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 86409216. Throughput: 0: 1816.7, 1: 1833.5. Samples: 21613686. Policy #0 lag: (min: 23.0, avg: 23.1, max: 31.0) [2023-10-08 05:34:53,755][130385] Avg episode reward: [(0, '56.940'), (1, '62.170')] [2023-10-08 05:34:53,767][00612] Updated weights for policy 1, policy_version 42330 (0.0008) [2023-10-08 05:34:54,110][00611] Updated weights for policy 0, policy_version 42112 (0.0008) [2023-10-08 05:34:57,542][00612] Updated weights for policy 1, policy_version 42340 (0.0008) [2023-10-08 05:34:57,766][00611] Updated weights for policy 0, policy_version 42122 (0.0008) [2023-10-08 05:34:57,906][00612] Updated weights for policy 1, policy_version 42350 (0.0009) [2023-10-08 05:34:58,136][00611] Updated weights for policy 0, policy_version 42132 (0.0009) [2023-10-08 05:34:58,273][00612] Updated weights for policy 1, policy_version 42360 (0.0008) [2023-10-08 05:34:58,506][00611] Updated weights for policy 0, policy_version 42142 (0.0008) [2023-10-08 05:34:58,754][130385] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 86540288. Throughput: 0: 1817.9, 1: 1838.1. Samples: 21636604. Policy #0 lag: (min: 23.0, avg: 23.1, max: 31.0) [2023-10-08 05:34:58,754][130385] Avg episode reward: [(0, '57.240'), (1, '61.350')] [2023-10-08 05:35:01,889][00612] Updated weights for policy 1, policy_version 42370 (0.0009) [2023-10-08 05:35:02,252][00612] Updated weights for policy 1, policy_version 42380 (0.0009) [2023-10-08 05:35:02,270][00611] Updated weights for policy 0, policy_version 42152 (0.0008) [2023-10-08 05:35:02,620][00612] Updated weights for policy 1, policy_version 42390 (0.0008) [2023-10-08 05:35:02,647][00611] Updated weights for policy 0, policy_version 42162 (0.0008) [2023-10-08 05:35:02,979][00612] Updated weights for policy 1, policy_version 42400 (0.0007) [2023-10-08 05:35:03,009][00611] Updated weights for policy 0, policy_version 42172 (0.0009) [2023-10-08 05:35:03,754][130385] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 86605824. Throughput: 0: 1821.0, 1: 1827.9. Samples: 21656250. Policy #0 lag: (min: 23.0, avg: 23.1, max: 31.0) [2023-10-08 05:35:03,754][130385] Avg episode reward: [(0, '55.510'), (1, '58.600')] [2023-10-08 05:35:06,682][00611] Updated weights for policy 0, policy_version 42182 (0.0008) [2023-10-08 05:35:06,772][00612] Updated weights for policy 1, policy_version 42410 (0.0008) [2023-10-08 05:35:07,055][00611] Updated weights for policy 0, policy_version 42192 (0.0009) [2023-10-08 05:35:07,141][00612] Updated weights for policy 1, policy_version 42420 (0.0007) [2023-10-08 05:35:07,434][00611] Updated weights for policy 0, policy_version 42202 (0.0007) [2023-10-08 05:35:07,514][00612] Updated weights for policy 1, policy_version 42430 (0.0008) [2023-10-08 05:35:08,754][130385] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 86671360. Throughput: 0: 1820.4, 1: 1831.9. Samples: 21669380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:35:08,755][130385] Avg episode reward: [(0, '56.650'), (1, '62.590')] [2023-10-08 05:35:11,072][00611] Updated weights for policy 0, policy_version 42212 (0.0007) [2023-10-08 05:35:11,226][00612] Updated weights for policy 1, policy_version 42440 (0.0009) [2023-10-08 05:35:11,444][00611] Updated weights for policy 0, policy_version 42222 (0.0009) [2023-10-08 05:35:11,601][00612] Updated weights for policy 1, policy_version 42450 (0.0008) [2023-10-08 05:35:11,810][00611] Updated weights for policy 0, policy_version 42232 (0.0009) [2023-10-08 05:35:11,965][00612] Updated weights for policy 1, policy_version 42460 (0.0008) [2023-10-08 05:35:13,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86736896. Throughput: 0: 1820.0, 1: 1827.6. Samples: 21688864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:35:13,755][130385] Avg episode reward: [(0, '56.110'), (1, '63.080')] [2023-10-08 05:35:15,458][00611] Updated weights for policy 0, policy_version 42242 (0.0008) [2023-10-08 05:35:15,642][00612] Updated weights for policy 1, policy_version 42470 (0.0010) [2023-10-08 05:35:15,830][00611] Updated weights for policy 0, policy_version 42252 (0.0009) [2023-10-08 05:35:16,012][00612] Updated weights for policy 1, policy_version 42480 (0.0009) [2023-10-08 05:35:16,202][00611] Updated weights for policy 0, policy_version 42262 (0.0008) [2023-10-08 05:35:16,374][00612] Updated weights for policy 1, policy_version 42490 (0.0008) [2023-10-08 05:35:16,575][00611] Updated weights for policy 0, policy_version 42272 (0.0011) [2023-10-08 05:35:18,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 86802432. Throughput: 0: 1813.5, 1: 1826.1. Samples: 21711812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:35:18,754][130385] Avg episode reward: [(0, '55.500'), (1, '64.410')] [2023-10-08 05:35:20,118][00612] Updated weights for policy 1, policy_version 42500 (0.0007) [2023-10-08 05:35:20,171][00611] Updated weights for policy 0, policy_version 42282 (0.0007) [2023-10-08 05:35:20,483][00612] Updated weights for policy 1, policy_version 42510 (0.0008) [2023-10-08 05:35:20,535][00611] Updated weights for policy 0, policy_version 42292 (0.0007) [2023-10-08 05:35:20,846][00612] Updated weights for policy 1, policy_version 42520 (0.0007) [2023-10-08 05:35:20,911][00611] Updated weights for policy 0, policy_version 42302 (0.0009) [2023-10-08 05:35:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 86867968. Throughput: 0: 1814.9, 1: 1823.4. Samples: 21721612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:35:23,754][130385] Avg episode reward: [(0, '52.910'), (1, '63.210')] [2023-10-08 05:35:24,586][00612] Updated weights for policy 1, policy_version 42530 (0.0009) [2023-10-08 05:35:24,628][00611] Updated weights for policy 0, policy_version 42312 (0.0009) [2023-10-08 05:35:24,964][00612] Updated weights for policy 1, policy_version 42540 (0.0007) [2023-10-08 05:35:24,989][00611] Updated weights for policy 0, policy_version 42322 (0.0010) [2023-10-08 05:35:25,332][00612] Updated weights for policy 1, policy_version 42550 (0.0007) [2023-10-08 05:35:25,361][00611] Updated weights for policy 0, policy_version 42332 (0.0008) [2023-10-08 05:35:25,697][00612] Updated weights for policy 1, policy_version 42560 (0.0008) [2023-10-08 05:35:28,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 86933504. Throughput: 0: 1819.0, 1: 1819.7. Samples: 21744604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:35:28,755][130385] Avg episode reward: [(0, '49.960'), (1, '64.450')] [2023-10-08 05:35:29,132][00611] Updated weights for policy 0, policy_version 42342 (0.0009) [2023-10-08 05:35:29,277][00612] Updated weights for policy 1, policy_version 42570 (0.0007) [2023-10-08 05:35:29,507][00611] Updated weights for policy 0, policy_version 42352 (0.0009) [2023-10-08 05:35:29,639][00612] Updated weights for policy 1, policy_version 42580 (0.0007) [2023-10-08 05:35:29,881][00611] Updated weights for policy 0, policy_version 42362 (0.0007) [2023-10-08 05:35:30,011][00612] Updated weights for policy 1, policy_version 42590 (0.0007) [2023-10-08 05:35:33,440][00611] Updated weights for policy 0, policy_version 42372 (0.0008) [2023-10-08 05:35:33,597][00612] Updated weights for policy 1, policy_version 42600 (0.0008) [2023-10-08 05:35:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 86999040. Throughput: 0: 1824.4, 1: 1822.2. Samples: 21768024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:35:33,754][130385] Avg episode reward: [(0, '52.360'), (1, '63.290')] [2023-10-08 05:35:33,810][00611] Updated weights for policy 0, policy_version 42382 (0.0008) [2023-10-08 05:35:33,963][00612] Updated weights for policy 1, policy_version 42610 (0.0009) [2023-10-08 05:35:34,187][00611] Updated weights for policy 0, policy_version 42392 (0.0008) [2023-10-08 05:35:34,342][00612] Updated weights for policy 1, policy_version 42620 (0.0009) [2023-10-08 05:35:37,877][00612] Updated weights for policy 1, policy_version 42630 (0.0008) [2023-10-08 05:35:37,956][00611] Updated weights for policy 0, policy_version 42402 (0.0009) [2023-10-08 05:35:38,248][00612] Updated weights for policy 1, policy_version 42640 (0.0010) [2023-10-08 05:35:38,317][00611] Updated weights for policy 0, policy_version 42412 (0.0007) [2023-10-08 05:35:38,614][00612] Updated weights for policy 1, policy_version 42650 (0.0009) [2023-10-08 05:35:38,700][00611] Updated weights for policy 0, policy_version 42422 (0.0007) [2023-10-08 05:35:38,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 87064576. Throughput: 0: 1818.8, 1: 1823.3. Samples: 21777580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:35:38,754][130385] Avg episode reward: [(0, '53.310'), (1, '63.580')] [2023-10-08 05:35:39,072][00611] Updated weights for policy 0, policy_version 42432 (0.0008) [2023-10-08 05:35:42,359][00612] Updated weights for policy 1, policy_version 42660 (0.0007) [2023-10-08 05:35:42,727][00612] Updated weights for policy 1, policy_version 42670 (0.0009) [2023-10-08 05:35:42,849][00611] Updated weights for policy 0, policy_version 42442 (0.0007) [2023-10-08 05:35:43,098][00612] Updated weights for policy 1, policy_version 42680 (0.0008) [2023-10-08 05:35:43,206][00611] Updated weights for policy 0, policy_version 42452 (0.0008) [2023-10-08 05:35:43,585][00611] Updated weights for policy 0, policy_version 42462 (0.0009) [2023-10-08 05:35:43,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 87195648. Throughput: 0: 1820.8, 1: 1822.3. Samples: 21800542. Policy #0 lag: (min: 48.0, avg: 55.8, max: 56.0) [2023-10-08 05:35:43,755][130385] Avg episode reward: [(0, '51.560'), (1, '64.760')] [2023-10-08 05:35:46,772][00612] Updated weights for policy 1, policy_version 42690 (0.0009) [2023-10-08 05:35:47,135][00612] Updated weights for policy 1, policy_version 42700 (0.0007) [2023-10-08 05:35:47,177][00611] Updated weights for policy 0, policy_version 42472 (0.0009) [2023-10-08 05:35:47,510][00612] Updated weights for policy 1, policy_version 42710 (0.0007) [2023-10-08 05:35:47,550][00611] Updated weights for policy 0, policy_version 42482 (0.0008) [2023-10-08 05:35:47,874][00612] Updated weights for policy 1, policy_version 42720 (0.0007) [2023-10-08 05:35:47,920][00611] Updated weights for policy 0, policy_version 42492 (0.0008) [2023-10-08 05:35:48,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 87261184. Throughput: 0: 1824.1, 1: 1826.5. Samples: 21820526. Policy #0 lag: (min: 48.0, avg: 55.8, max: 56.0) [2023-10-08 05:35:48,755][130385] Avg episode reward: [(0, '54.720'), (1, '62.570')] [2023-10-08 05:35:51,469][00612] Updated weights for policy 1, policy_version 42730 (0.0009) [2023-10-08 05:35:51,616][00611] Updated weights for policy 0, policy_version 42502 (0.0007) [2023-10-08 05:35:51,834][00612] Updated weights for policy 1, policy_version 42740 (0.0009) [2023-10-08 05:35:51,985][00611] Updated weights for policy 0, policy_version 42512 (0.0008) [2023-10-08 05:35:52,213][00612] Updated weights for policy 1, policy_version 42750 (0.0010) [2023-10-08 05:35:52,358][00611] Updated weights for policy 0, policy_version 42522 (0.0009) [2023-10-08 05:35:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 87326720. Throughput: 0: 1821.0, 1: 1827.2. Samples: 21833552. Policy #0 lag: (min: 48.0, avg: 55.8, max: 56.0) [2023-10-08 05:35:53,755][130385] Avg episode reward: [(0, '54.550'), (1, '64.830')] [2023-10-08 05:35:55,950][00612] Updated weights for policy 1, policy_version 42760 (0.0008) [2023-10-08 05:35:55,951][00611] Updated weights for policy 0, policy_version 42532 (0.0009) [2023-10-08 05:35:56,318][00611] Updated weights for policy 0, policy_version 42542 (0.0008) [2023-10-08 05:35:56,322][00612] Updated weights for policy 1, policy_version 42770 (0.0007) [2023-10-08 05:35:56,682][00612] Updated weights for policy 1, policy_version 42780 (0.0008) [2023-10-08 05:35:56,691][00611] Updated weights for policy 0, policy_version 42552 (0.0008) [2023-10-08 05:35:58,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 87392256. Throughput: 0: 1823.5, 1: 1832.9. Samples: 21853400. Policy #0 lag: (min: 48.0, avg: 55.8, max: 56.0) [2023-10-08 05:35:58,754][130385] Avg episode reward: [(0, '52.780'), (1, '67.740')] [2023-10-08 05:35:58,755][00425] Saving new best policy, reward=67.740! [2023-10-08 05:36:00,359][00611] Updated weights for policy 0, policy_version 42562 (0.0008) [2023-10-08 05:36:00,395][00612] Updated weights for policy 1, policy_version 42790 (0.0009) [2023-10-08 05:36:00,725][00611] Updated weights for policy 0, policy_version 42572 (0.0010) [2023-10-08 05:36:00,755][00612] Updated weights for policy 1, policy_version 42800 (0.0009) [2023-10-08 05:36:01,098][00611] Updated weights for policy 0, policy_version 42582 (0.0008) [2023-10-08 05:36:01,119][00612] Updated weights for policy 1, policy_version 42810 (0.0008) [2023-10-08 05:36:01,467][00611] Updated weights for policy 0, policy_version 42592 (0.0007) [2023-10-08 05:36:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 87457792. Throughput: 0: 1827.5, 1: 1835.2. Samples: 21876636. Policy #0 lag: (min: 48.0, avg: 55.8, max: 56.0) [2023-10-08 05:36:03,755][130385] Avg episode reward: [(0, '52.380'), (1, '71.270')] [2023-10-08 05:36:03,765][00425] Saving new best policy, reward=71.270! [2023-10-08 05:36:04,767][00612] Updated weights for policy 1, policy_version 42820 (0.0007) [2023-10-08 05:36:05,110][00611] Updated weights for policy 0, policy_version 42602 (0.0009) [2023-10-08 05:36:05,132][00612] Updated weights for policy 1, policy_version 42830 (0.0007) [2023-10-08 05:36:05,479][00611] Updated weights for policy 0, policy_version 42612 (0.0009) [2023-10-08 05:36:05,508][00612] Updated weights for policy 1, policy_version 42840 (0.0008) [2023-10-08 05:36:05,856][00611] Updated weights for policy 0, policy_version 42622 (0.0008) [2023-10-08 05:36:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 87523328. Throughput: 0: 1829.8, 1: 1838.1. Samples: 21886666. Policy #0 lag: (min: 48.0, avg: 55.8, max: 56.0) [2023-10-08 05:36:08,754][130385] Avg episode reward: [(0, '51.810'), (1, '73.500')] [2023-10-08 05:36:08,755][00425] Saving new best policy, reward=73.500! [2023-10-08 05:36:09,140][00612] Updated weights for policy 1, policy_version 42850 (0.0007) [2023-10-08 05:36:09,322][00611] Updated weights for policy 0, policy_version 42632 (0.0008) [2023-10-08 05:36:09,510][00612] Updated weights for policy 1, policy_version 42860 (0.0008) [2023-10-08 05:36:09,698][00611] Updated weights for policy 0, policy_version 42642 (0.0009) [2023-10-08 05:36:09,879][00612] Updated weights for policy 1, policy_version 42870 (0.0009) [2023-10-08 05:36:10,067][00611] Updated weights for policy 0, policy_version 42652 (0.0009) [2023-10-08 05:36:10,243][00612] Updated weights for policy 1, policy_version 42880 (0.0008) [2023-10-08 05:36:13,606][00611] Updated weights for policy 0, policy_version 42662 (0.0007) [2023-10-08 05:36:13,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 87588864. Throughput: 0: 1838.1, 1: 1827.5. Samples: 21909556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:36:13,754][130385] Avg episode reward: [(0, '52.470'), (1, '68.800')] [2023-10-08 05:36:13,970][00611] Updated weights for policy 0, policy_version 42672 (0.0009) [2023-10-08 05:36:14,010][00612] Updated weights for policy 1, policy_version 42890 (0.0007) [2023-10-08 05:36:14,338][00611] Updated weights for policy 0, policy_version 42682 (0.0007) [2023-10-08 05:36:14,371][00612] Updated weights for policy 1, policy_version 42900 (0.0008) [2023-10-08 05:36:14,740][00612] Updated weights for policy 1, policy_version 42910 (0.0007) [2023-10-08 05:36:17,895][00611] Updated weights for policy 0, policy_version 42692 (0.0008) [2023-10-08 05:36:18,268][00611] Updated weights for policy 0, policy_version 42702 (0.0008) [2023-10-08 05:36:18,392][00612] Updated weights for policy 1, policy_version 42920 (0.0007) [2023-10-08 05:36:18,648][00611] Updated weights for policy 0, policy_version 42712 (0.0009) [2023-10-08 05:36:18,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 87654400. Throughput: 0: 1825.4, 1: 1824.3. Samples: 21932258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:36:18,754][130385] Avg episode reward: [(0, '53.320'), (1, '69.620')] [2023-10-08 05:36:18,775][00612] Updated weights for policy 1, policy_version 42930 (0.0007) [2023-10-08 05:36:18,943][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000042720_43745280.pth... [2023-10-08 05:36:18,976][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000040992_41975808.pth [2023-10-08 05:36:18,980][00365] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p0/milestones/checkpoint_000042720_43745280.pth [2023-10-08 05:36:19,134][00612] Updated weights for policy 1, policy_version 42940 (0.0007) [2023-10-08 05:36:19,282][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000042944_43974656.pth... [2023-10-08 05:36:19,310][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000041216_42205184.pth [2023-10-08 05:36:19,313][00425] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p1/milestones/checkpoint_000042944_43974656.pth [2023-10-08 05:36:22,373][00611] Updated weights for policy 0, policy_version 42722 (0.0009) [2023-10-08 05:36:22,736][00611] Updated weights for policy 0, policy_version 42732 (0.0007) [2023-10-08 05:36:22,830][00612] Updated weights for policy 1, policy_version 42950 (0.0008) [2023-10-08 05:36:23,100][00611] Updated weights for policy 0, policy_version 42742 (0.0007) [2023-10-08 05:36:23,197][00612] Updated weights for policy 1, policy_version 42960 (0.0008) [2023-10-08 05:36:23,462][00611] Updated weights for policy 0, policy_version 42752 (0.0007) [2023-10-08 05:36:23,567][00612] Updated weights for policy 1, policy_version 42970 (0.0007) [2023-10-08 05:36:23,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87752704. Throughput: 0: 1844.6, 1: 1826.1. Samples: 21942764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:36:23,754][130385] Avg episode reward: [(0, '54.400'), (1, '67.270')] [2023-10-08 05:36:27,203][00611] Updated weights for policy 0, policy_version 42762 (0.0007) [2023-10-08 05:36:27,213][00612] Updated weights for policy 1, policy_version 42980 (0.0008) [2023-10-08 05:36:27,566][00611] Updated weights for policy 0, policy_version 42772 (0.0008) [2023-10-08 05:36:27,578][00612] Updated weights for policy 1, policy_version 42990 (0.0007) [2023-10-08 05:36:27,935][00611] Updated weights for policy 0, policy_version 42782 (0.0009) [2023-10-08 05:36:27,945][00612] Updated weights for policy 1, policy_version 43000 (0.0009) [2023-10-08 05:36:28,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 87851008. Throughput: 0: 1833.7, 1: 1823.2. Samples: 21965102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:36:28,754][130385] Avg episode reward: [(0, '53.820'), (1, '67.910')] [2023-10-08 05:36:31,573][00612] Updated weights for policy 1, policy_version 43010 (0.0008) [2023-10-08 05:36:31,637][00611] Updated weights for policy 0, policy_version 42792 (0.0008) [2023-10-08 05:36:31,934][00612] Updated weights for policy 1, policy_version 43020 (0.0009) [2023-10-08 05:36:32,002][00611] Updated weights for policy 0, policy_version 42802 (0.0010) [2023-10-08 05:36:32,306][00612] Updated weights for policy 1, policy_version 43030 (0.0007) [2023-10-08 05:36:32,377][00611] Updated weights for policy 0, policy_version 42812 (0.0008) [2023-10-08 05:36:32,672][00612] Updated weights for policy 1, policy_version 43040 (0.0009) [2023-10-08 05:36:33,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 87916544. Throughput: 0: 1839.8, 1: 1831.1. Samples: 21985718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:36:33,755][130385] Avg episode reward: [(0, '54.690'), (1, '66.860')] [2023-10-08 05:36:36,050][00611] Updated weights for policy 0, policy_version 42822 (0.0008) [2023-10-08 05:36:36,292][00612] Updated weights for policy 1, policy_version 43050 (0.0007) [2023-10-08 05:36:36,428][00611] Updated weights for policy 0, policy_version 42832 (0.0007) [2023-10-08 05:36:36,666][00612] Updated weights for policy 1, policy_version 43060 (0.0008) [2023-10-08 05:36:36,787][00611] Updated weights for policy 0, policy_version 42842 (0.0008) [2023-10-08 05:36:37,023][00612] Updated weights for policy 1, policy_version 43070 (0.0009) [2023-10-08 05:36:38,754][130385] Fps is (10 sec: 13107.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 87982080. Throughput: 0: 1833.5, 1: 1825.9. Samples: 21998226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:36:38,754][130385] Avg episode reward: [(0, '52.840'), (1, '66.520')] [2023-10-08 05:36:40,522][00611] Updated weights for policy 0, policy_version 42852 (0.0009) [2023-10-08 05:36:40,693][00612] Updated weights for policy 1, policy_version 43080 (0.0008) [2023-10-08 05:36:40,911][00611] Updated weights for policy 0, policy_version 42862 (0.0008) [2023-10-08 05:36:41,055][00612] Updated weights for policy 1, policy_version 43090 (0.0007) [2023-10-08 05:36:41,275][00611] Updated weights for policy 0, policy_version 42872 (0.0007) [2023-10-08 05:36:41,423][00612] Updated weights for policy 1, policy_version 43100 (0.0008) [2023-10-08 05:36:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 88047616. Throughput: 0: 1842.5, 1: 1832.3. Samples: 22018770. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 05:36:43,755][130385] Avg episode reward: [(0, '53.180'), (1, '65.050')] [2023-10-08 05:36:44,968][00611] Updated weights for policy 0, policy_version 42882 (0.0009) [2023-10-08 05:36:45,094][00612] Updated weights for policy 1, policy_version 43110 (0.0009) [2023-10-08 05:36:45,334][00611] Updated weights for policy 0, policy_version 42892 (0.0009) [2023-10-08 05:36:45,456][00612] Updated weights for policy 1, policy_version 43120 (0.0009) [2023-10-08 05:36:45,699][00611] Updated weights for policy 0, policy_version 42902 (0.0008) [2023-10-08 05:36:45,827][00612] Updated weights for policy 1, policy_version 43130 (0.0007) [2023-10-08 05:36:46,070][00611] Updated weights for policy 0, policy_version 42912 (0.0007) [2023-10-08 05:36:48,754][130385] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 88113152. Throughput: 0: 1845.1, 1: 1834.3. Samples: 22042210. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 05:36:48,756][130385] Avg episode reward: [(0, '54.020'), (1, '63.100')] [2023-10-08 05:36:49,399][00612] Updated weights for policy 1, policy_version 43140 (0.0007) [2023-10-08 05:36:49,665][00611] Updated weights for policy 0, policy_version 42922 (0.0010) [2023-10-08 05:36:49,760][00612] Updated weights for policy 1, policy_version 43150 (0.0007) [2023-10-08 05:36:50,032][00611] Updated weights for policy 0, policy_version 42932 (0.0008) [2023-10-08 05:36:50,123][00612] Updated weights for policy 1, policy_version 43160 (0.0009) [2023-10-08 05:36:50,413][00611] Updated weights for policy 0, policy_version 42942 (0.0007) [2023-10-08 05:36:53,721][00612] Updated weights for policy 1, policy_version 43170 (0.0008) [2023-10-08 05:36:53,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 88178688. Throughput: 0: 1842.3, 1: 1830.9. Samples: 22051962. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 05:36:53,754][130385] Avg episode reward: [(0, '57.130'), (1, '64.840')] [2023-10-08 05:36:54,044][00611] Updated weights for policy 0, policy_version 42952 (0.0008) [2023-10-08 05:36:54,102][00612] Updated weights for policy 1, policy_version 43180 (0.0007) [2023-10-08 05:36:54,410][00611] Updated weights for policy 0, policy_version 42962 (0.0008) [2023-10-08 05:36:54,461][00612] Updated weights for policy 1, policy_version 43190 (0.0008) [2023-10-08 05:36:54,774][00611] Updated weights for policy 0, policy_version 42972 (0.0008) [2023-10-08 05:36:54,834][00612] Updated weights for policy 1, policy_version 43200 (0.0007) [2023-10-08 05:36:58,413][00612] Updated weights for policy 1, policy_version 43210 (0.0008) [2023-10-08 05:36:58,529][00611] Updated weights for policy 0, policy_version 42982 (0.0008) [2023-10-08 05:36:58,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88244224. Throughput: 0: 1830.8, 1: 1851.2. Samples: 22075242. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 05:36:58,754][130385] Avg episode reward: [(0, '56.810'), (1, '65.670')] [2023-10-08 05:36:58,781][00612] Updated weights for policy 1, policy_version 43220 (0.0007) [2023-10-08 05:36:58,899][00611] Updated weights for policy 0, policy_version 42992 (0.0007) [2023-10-08 05:36:59,156][00612] Updated weights for policy 1, policy_version 43230 (0.0008) [2023-10-08 05:36:59,265][00611] Updated weights for policy 0, policy_version 43002 (0.0007) [2023-10-08 05:37:02,723][00612] Updated weights for policy 1, policy_version 43240 (0.0010) [2023-10-08 05:37:02,853][00611] Updated weights for policy 0, policy_version 43012 (0.0009) [2023-10-08 05:37:03,088][00612] Updated weights for policy 1, policy_version 43250 (0.0007) [2023-10-08 05:37:03,222][00611] Updated weights for policy 0, policy_version 43022 (0.0007) [2023-10-08 05:37:03,469][00612] Updated weights for policy 1, policy_version 43260 (0.0009) [2023-10-08 05:37:03,599][00611] Updated weights for policy 0, policy_version 43032 (0.0008) [2023-10-08 05:37:03,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 88342528. Throughput: 0: 1822.2, 1: 1829.3. Samples: 22096576. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 05:37:03,755][130385] Avg episode reward: [(0, '54.470'), (1, '64.800')] [2023-10-08 05:37:07,285][00611] Updated weights for policy 0, policy_version 43042 (0.0008) [2023-10-08 05:37:07,366][00612] Updated weights for policy 1, policy_version 43270 (0.0007) [2023-10-08 05:37:07,644][00611] Updated weights for policy 0, policy_version 43052 (0.0009) [2023-10-08 05:37:07,745][00612] Updated weights for policy 1, policy_version 43280 (0.0007) [2023-10-08 05:37:08,014][00611] Updated weights for policy 0, policy_version 43062 (0.0008) [2023-10-08 05:37:08,108][00612] Updated weights for policy 1, policy_version 43290 (0.0007) [2023-10-08 05:37:08,380][00611] Updated weights for policy 0, policy_version 43072 (0.0010) [2023-10-08 05:37:08,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 88440832. Throughput: 0: 1819.2, 1: 1850.7. Samples: 22107912. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 05:37:08,754][130385] Avg episode reward: [(0, '52.150'), (1, '62.570')] [2023-10-08 05:37:11,776][00612] Updated weights for policy 1, policy_version 43300 (0.0009) [2023-10-08 05:37:12,071][00611] Updated weights for policy 0, policy_version 43082 (0.0007) [2023-10-08 05:37:12,144][00612] Updated weights for policy 1, policy_version 43310 (0.0008) [2023-10-08 05:37:12,444][00611] Updated weights for policy 0, policy_version 43092 (0.0008) [2023-10-08 05:37:12,511][00612] Updated weights for policy 1, policy_version 43320 (0.0007) [2023-10-08 05:37:12,818][00611] Updated weights for policy 0, policy_version 43102 (0.0008) [2023-10-08 05:37:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 88506368. Throughput: 0: 1819.3, 1: 1834.1. Samples: 22129508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:37:13,755][130385] Avg episode reward: [(0, '51.660'), (1, '63.660')] [2023-10-08 05:37:16,148][00612] Updated weights for policy 1, policy_version 43330 (0.0007) [2023-10-08 05:37:16,519][00612] Updated weights for policy 1, policy_version 43340 (0.0007) [2023-10-08 05:37:16,572][00611] Updated weights for policy 0, policy_version 43112 (0.0007) [2023-10-08 05:37:16,890][00612] Updated weights for policy 1, policy_version 43350 (0.0007) [2023-10-08 05:37:16,939][00611] Updated weights for policy 0, policy_version 43122 (0.0009) [2023-10-08 05:37:17,247][00612] Updated weights for policy 1, policy_version 43360 (0.0008) [2023-10-08 05:37:17,304][00611] Updated weights for policy 0, policy_version 43132 (0.0007) [2023-10-08 05:37:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 88571904. Throughput: 0: 1823.3, 1: 1843.6. Samples: 22150726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:37:18,755][130385] Avg episode reward: [(0, '51.170'), (1, '64.610')] [2023-10-08 05:37:20,878][00612] Updated weights for policy 1, policy_version 43370 (0.0009) [2023-10-08 05:37:20,950][00611] Updated weights for policy 0, policy_version 43142 (0.0007) [2023-10-08 05:37:21,238][00612] Updated weights for policy 1, policy_version 43380 (0.0008) [2023-10-08 05:37:21,318][00611] Updated weights for policy 0, policy_version 43152 (0.0007) [2023-10-08 05:37:21,607][00612] Updated weights for policy 1, policy_version 43390 (0.0009) [2023-10-08 05:37:21,691][00611] Updated weights for policy 0, policy_version 43162 (0.0009) [2023-10-08 05:37:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 88637440. Throughput: 0: 1820.8, 1: 1826.0. Samples: 22162336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:37:23,755][130385] Avg episode reward: [(0, '53.000'), (1, '66.600')] [2023-10-08 05:37:25,229][00612] Updated weights for policy 1, policy_version 43400 (0.0010) [2023-10-08 05:37:25,438][00611] Updated weights for policy 0, policy_version 43172 (0.0009) [2023-10-08 05:37:25,586][00612] Updated weights for policy 1, policy_version 43410 (0.0008) [2023-10-08 05:37:25,805][00611] Updated weights for policy 0, policy_version 43182 (0.0010) [2023-10-08 05:37:25,951][00612] Updated weights for policy 1, policy_version 43420 (0.0008) [2023-10-08 05:37:26,178][00611] Updated weights for policy 0, policy_version 43192 (0.0007) [2023-10-08 05:37:28,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 88702976. Throughput: 0: 1828.3, 1: 1839.6. Samples: 22183822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:37:28,754][130385] Avg episode reward: [(0, '52.730'), (1, '65.990')] [2023-10-08 05:37:29,557][00612] Updated weights for policy 1, policy_version 43430 (0.0008) [2023-10-08 05:37:29,834][00611] Updated weights for policy 0, policy_version 43202 (0.0007) [2023-10-08 05:37:29,922][00612] Updated weights for policy 1, policy_version 43440 (0.0007) [2023-10-08 05:37:30,235][00611] Updated weights for policy 0, policy_version 43212 (0.0008) [2023-10-08 05:37:30,289][00612] Updated weights for policy 1, policy_version 43450 (0.0007) [2023-10-08 05:37:30,608][00611] Updated weights for policy 0, policy_version 43222 (0.0008) [2023-10-08 05:37:30,974][00611] Updated weights for policy 0, policy_version 43232 (0.0009) [2023-10-08 05:37:33,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 88768512. Throughput: 0: 1819.2, 1: 1839.9. Samples: 22206870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:37:33,755][130385] Avg episode reward: [(0, '49.060'), (1, '65.000')] [2023-10-08 05:37:33,831][00612] Updated weights for policy 1, policy_version 43460 (0.0008) [2023-10-08 05:37:34,202][00612] Updated weights for policy 1, policy_version 43470 (0.0007) [2023-10-08 05:37:34,577][00612] Updated weights for policy 1, policy_version 43480 (0.0008) [2023-10-08 05:37:34,580][00611] Updated weights for policy 0, policy_version 43242 (0.0008) [2023-10-08 05:37:34,957][00611] Updated weights for policy 0, policy_version 43252 (0.0007) [2023-10-08 05:37:35,325][00611] Updated weights for policy 0, policy_version 43262 (0.0008) [2023-10-08 05:37:38,145][00612] Updated weights for policy 1, policy_version 43490 (0.0007) [2023-10-08 05:37:38,514][00612] Updated weights for policy 1, policy_version 43500 (0.0011) [2023-10-08 05:37:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88834048. Throughput: 0: 1820.6, 1: 1842.0. Samples: 22216776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:37:38,755][130385] Avg episode reward: [(0, '50.080'), (1, '63.230')] [2023-10-08 05:37:38,883][00612] Updated weights for policy 1, policy_version 43510 (0.0008) [2023-10-08 05:37:39,016][00611] Updated weights for policy 0, policy_version 43272 (0.0009) [2023-10-08 05:37:39,254][00612] Updated weights for policy 1, policy_version 43520 (0.0009) [2023-10-08 05:37:39,387][00611] Updated weights for policy 0, policy_version 43282 (0.0007) [2023-10-08 05:37:39,759][00611] Updated weights for policy 0, policy_version 43292 (0.0007) [2023-10-08 05:37:42,948][00612] Updated weights for policy 1, policy_version 43530 (0.0008) [2023-10-08 05:37:43,321][00612] Updated weights for policy 1, policy_version 43540 (0.0009) [2023-10-08 05:37:43,356][00611] Updated weights for policy 0, policy_version 43302 (0.0008) [2023-10-08 05:37:43,691][00612] Updated weights for policy 1, policy_version 43550 (0.0008) [2023-10-08 05:37:43,718][00611] Updated weights for policy 0, policy_version 43312 (0.0009) [2023-10-08 05:37:43,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 88932352. Throughput: 0: 1820.9, 1: 1838.8. Samples: 22239930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:37:43,754][130385] Avg episode reward: [(0, '48.660'), (1, '67.370')] [2023-10-08 05:37:44,092][00611] Updated weights for policy 0, policy_version 43322 (0.0007) [2023-10-08 05:37:47,323][00612] Updated weights for policy 1, policy_version 43560 (0.0010) [2023-10-08 05:37:47,631][00611] Updated weights for policy 0, policy_version 43332 (0.0007) [2023-10-08 05:37:47,698][00612] Updated weights for policy 1, policy_version 43570 (0.0008) [2023-10-08 05:37:47,996][00611] Updated weights for policy 0, policy_version 43342 (0.0007) [2023-10-08 05:37:48,076][00612] Updated weights for policy 1, policy_version 43580 (0.0009) [2023-10-08 05:37:48,373][00611] Updated weights for policy 0, policy_version 43352 (0.0008) [2023-10-08 05:37:48,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 89030656. Throughput: 0: 1821.4, 1: 1831.1. Samples: 22260940. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 05:37:48,755][130385] Avg episode reward: [(0, '50.430'), (1, '66.560')] [2023-10-08 05:37:51,773][00612] Updated weights for policy 1, policy_version 43590 (0.0009) [2023-10-08 05:37:52,126][00612] Updated weights for policy 1, policy_version 43600 (0.0009) [2023-10-08 05:37:52,137][00611] Updated weights for policy 0, policy_version 43362 (0.0007) [2023-10-08 05:37:52,486][00612] Updated weights for policy 1, policy_version 43610 (0.0007) [2023-10-08 05:37:52,522][00611] Updated weights for policy 0, policy_version 43372 (0.0008) [2023-10-08 05:37:52,890][00611] Updated weights for policy 0, policy_version 43382 (0.0008) [2023-10-08 05:37:53,267][00611] Updated weights for policy 0, policy_version 43392 (0.0008) [2023-10-08 05:37:53,754][130385] Fps is (10 sec: 16383.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 89096192. Throughput: 0: 1835.1, 1: 1842.3. Samples: 22273394. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 05:37:53,755][130385] Avg episode reward: [(0, '54.660'), (1, '64.510')] [2023-10-08 05:37:56,188][00612] Updated weights for policy 1, policy_version 43620 (0.0007) [2023-10-08 05:37:56,562][00612] Updated weights for policy 1, policy_version 43630 (0.0008) [2023-10-08 05:37:56,922][00612] Updated weights for policy 1, policy_version 43640 (0.0008) [2023-10-08 05:37:57,006][00611] Updated weights for policy 0, policy_version 43402 (0.0009) [2023-10-08 05:37:57,372][00611] Updated weights for policy 0, policy_version 43412 (0.0010) [2023-10-08 05:37:57,744][00611] Updated weights for policy 0, policy_version 43422 (0.0008) [2023-10-08 05:37:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 89161728. Throughput: 0: 1830.9, 1: 1829.2. Samples: 22294214. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 05:37:58,754][130385] Avg episode reward: [(0, '54.100'), (1, '60.140')] [2023-10-08 05:38:00,791][00612] Updated weights for policy 1, policy_version 43650 (0.0008) [2023-10-08 05:38:01,156][00612] Updated weights for policy 1, policy_version 43660 (0.0009) [2023-10-08 05:38:01,411][00611] Updated weights for policy 0, policy_version 43432 (0.0007) [2023-10-08 05:38:01,534][00612] Updated weights for policy 1, policy_version 43670 (0.0008) [2023-10-08 05:38:01,781][00611] Updated weights for policy 0, policy_version 43442 (0.0007) [2023-10-08 05:38:01,892][00612] Updated weights for policy 1, policy_version 43680 (0.0008) [2023-10-08 05:38:02,161][00611] Updated weights for policy 0, policy_version 43452 (0.0008) [2023-10-08 05:38:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 89227264. Throughput: 0: 1831.2, 1: 1837.7. Samples: 22315828. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 05:38:03,755][130385] Avg episode reward: [(0, '55.960'), (1, '58.690')] [2023-10-08 05:38:05,677][00612] Updated weights for policy 1, policy_version 43690 (0.0007) [2023-10-08 05:38:06,052][00612] Updated weights for policy 1, policy_version 43700 (0.0007) [2023-10-08 05:38:06,068][00611] Updated weights for policy 0, policy_version 43462 (0.0007) [2023-10-08 05:38:06,418][00612] Updated weights for policy 1, policy_version 43710 (0.0008) [2023-10-08 05:38:06,434][00611] Updated weights for policy 0, policy_version 43472 (0.0007) [2023-10-08 05:38:06,814][00611] Updated weights for policy 0, policy_version 43482 (0.0007) [2023-10-08 05:38:08,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 89292800. Throughput: 0: 1826.2, 1: 1834.0. Samples: 22327042. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 05:38:08,755][130385] Avg episode reward: [(0, '56.550'), (1, '59.180')] [2023-10-08 05:38:10,006][00612] Updated weights for policy 1, policy_version 43720 (0.0008) [2023-10-08 05:38:10,371][00612] Updated weights for policy 1, policy_version 43730 (0.0010) [2023-10-08 05:38:10,455][00611] Updated weights for policy 0, policy_version 43492 (0.0008) [2023-10-08 05:38:10,746][00612] Updated weights for policy 1, policy_version 43740 (0.0007) [2023-10-08 05:38:10,827][00611] Updated weights for policy 0, policy_version 43502 (0.0008) [2023-10-08 05:38:11,192][00611] Updated weights for policy 0, policy_version 43512 (0.0009) [2023-10-08 05:38:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 89358336. Throughput: 0: 1821.5, 1: 1836.6. Samples: 22348434. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 05:38:13,755][130385] Avg episode reward: [(0, '55.980'), (1, '60.700')] [2023-10-08 05:38:14,454][00612] Updated weights for policy 1, policy_version 43750 (0.0008) [2023-10-08 05:38:14,712][00611] Updated weights for policy 0, policy_version 43522 (0.0009) [2023-10-08 05:38:14,824][00612] Updated weights for policy 1, policy_version 43760 (0.0009) [2023-10-08 05:38:15,112][00611] Updated weights for policy 0, policy_version 43532 (0.0009) [2023-10-08 05:38:15,188][00612] Updated weights for policy 1, policy_version 43770 (0.0008) [2023-10-08 05:38:15,485][00611] Updated weights for policy 0, policy_version 43542 (0.0009) [2023-10-08 05:38:15,862][00611] Updated weights for policy 0, policy_version 43552 (0.0011) [2023-10-08 05:38:18,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 89423872. Throughput: 0: 1833.5, 1: 1832.8. Samples: 22371852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:38:18,754][130385] Avg episode reward: [(0, '55.260'), (1, '64.740')] [2023-10-08 05:38:18,762][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000043552_44597248.pth... [2023-10-08 05:38:18,790][00612] Updated weights for policy 1, policy_version 43780 (0.0009) [2023-10-08 05:38:18,799][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000041856_42860544.pth [2023-10-08 05:38:19,157][00612] Updated weights for policy 1, policy_version 43790 (0.0009) [2023-10-08 05:38:19,406][00611] Updated weights for policy 0, policy_version 43562 (0.0008) [2023-10-08 05:38:19,526][00612] Updated weights for policy 1, policy_version 43800 (0.0008) [2023-10-08 05:38:19,783][00611] Updated weights for policy 0, policy_version 43572 (0.0009) [2023-10-08 05:38:19,815][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000043808_44859392.pth... [2023-10-08 05:38:19,852][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000042080_43089920.pth [2023-10-08 05:38:20,146][00611] Updated weights for policy 0, policy_version 43582 (0.0008) [2023-10-08 05:38:23,167][00612] Updated weights for policy 1, policy_version 43810 (0.0007) [2023-10-08 05:38:23,524][00612] Updated weights for policy 1, policy_version 43820 (0.0009) [2023-10-08 05:38:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 89489408. Throughput: 0: 1831.7, 1: 1834.5. Samples: 22381754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:38:23,755][130385] Avg episode reward: [(0, '54.750'), (1, '63.870')] [2023-10-08 05:38:23,894][00612] Updated weights for policy 1, policy_version 43830 (0.0007) [2023-10-08 05:38:23,936][00611] Updated weights for policy 0, policy_version 43592 (0.0009) [2023-10-08 05:38:24,261][00612] Updated weights for policy 1, policy_version 43840 (0.0007) [2023-10-08 05:38:24,317][00611] Updated weights for policy 0, policy_version 43602 (0.0008) [2023-10-08 05:38:24,688][00611] Updated weights for policy 0, policy_version 43612 (0.0007) [2023-10-08 05:38:27,704][00612] Updated weights for policy 1, policy_version 43850 (0.0009) [2023-10-08 05:38:28,068][00612] Updated weights for policy 1, policy_version 43860 (0.0008) [2023-10-08 05:38:28,232][00611] Updated weights for policy 0, policy_version 43622 (0.0007) [2023-10-08 05:38:28,434][00612] Updated weights for policy 1, policy_version 43870 (0.0007) [2023-10-08 05:38:28,590][00611] Updated weights for policy 0, policy_version 43632 (0.0008) [2023-10-08 05:38:28,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 89587712. Throughput: 0: 1835.2, 1: 1832.9. Samples: 22404994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:38:28,755][130385] Avg episode reward: [(0, '54.040'), (1, '64.280')] [2023-10-08 05:38:28,965][00611] Updated weights for policy 0, policy_version 43642 (0.0008) [2023-10-08 05:38:31,987][00612] Updated weights for policy 1, policy_version 43880 (0.0009) [2023-10-08 05:38:32,355][00612] Updated weights for policy 1, policy_version 43890 (0.0008) [2023-10-08 05:38:32,589][00611] Updated weights for policy 0, policy_version 43652 (0.0008) [2023-10-08 05:38:32,724][00612] Updated weights for policy 1, policy_version 43900 (0.0007) [2023-10-08 05:38:32,952][00611] Updated weights for policy 0, policy_version 43662 (0.0009) [2023-10-08 05:38:33,320][00611] Updated weights for policy 0, policy_version 43672 (0.0011) [2023-10-08 05:38:33,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 89686016. Throughput: 0: 1831.2, 1: 1833.4. Samples: 22425846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:38:33,755][130385] Avg episode reward: [(0, '55.050'), (1, '65.110')] [2023-10-08 05:38:36,372][00612] Updated weights for policy 1, policy_version 43910 (0.0008) [2023-10-08 05:38:36,746][00612] Updated weights for policy 1, policy_version 43920 (0.0009) [2023-10-08 05:38:37,033][00611] Updated weights for policy 0, policy_version 43682 (0.0009) [2023-10-08 05:38:37,116][00612] Updated weights for policy 1, policy_version 43930 (0.0007) [2023-10-08 05:38:37,396][00611] Updated weights for policy 0, policy_version 43692 (0.0009) [2023-10-08 05:38:37,773][00611] Updated weights for policy 0, policy_version 43702 (0.0007) [2023-10-08 05:38:38,138][00611] Updated weights for policy 0, policy_version 43712 (0.0007) [2023-10-08 05:38:38,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 89751552. Throughput: 0: 1823.1, 1: 1835.7. Samples: 22438038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:38:38,754][130385] Avg episode reward: [(0, '53.160'), (1, '66.620')] [2023-10-08 05:38:40,731][00612] Updated weights for policy 1, policy_version 43940 (0.0008) [2023-10-08 05:38:41,091][00612] Updated weights for policy 1, policy_version 43950 (0.0010) [2023-10-08 05:38:41,464][00612] Updated weights for policy 1, policy_version 43960 (0.0007) [2023-10-08 05:38:41,842][00611] Updated weights for policy 0, policy_version 43722 (0.0008) [2023-10-08 05:38:42,219][00611] Updated weights for policy 0, policy_version 43732 (0.0008) [2023-10-08 05:38:42,581][00611] Updated weights for policy 0, policy_version 43742 (0.0008) [2023-10-08 05:38:43,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 89817088. Throughput: 0: 1825.2, 1: 1836.3. Samples: 22458980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:38:43,754][130385] Avg episode reward: [(0, '55.880'), (1, '64.060')] [2023-10-08 05:38:45,325][00612] Updated weights for policy 1, policy_version 43970 (0.0007) [2023-10-08 05:38:45,694][00612] Updated weights for policy 1, policy_version 43980 (0.0009) [2023-10-08 05:38:46,058][00612] Updated weights for policy 1, policy_version 43990 (0.0009) [2023-10-08 05:38:46,339][00611] Updated weights for policy 0, policy_version 43752 (0.0008) [2023-10-08 05:38:46,430][00612] Updated weights for policy 1, policy_version 44000 (0.0007) [2023-10-08 05:38:46,715][00611] Updated weights for policy 0, policy_version 43762 (0.0007) [2023-10-08 05:38:47,087][00611] Updated weights for policy 0, policy_version 43772 (0.0007) [2023-10-08 05:38:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 89882624. Throughput: 0: 1828.7, 1: 1844.8. Samples: 22481136. Policy #0 lag: (min: 11.0, avg: 20.2, max: 43.0) [2023-10-08 05:38:48,754][130385] Avg episode reward: [(0, '54.560'), (1, '66.890')] [2023-10-08 05:38:50,207][00612] Updated weights for policy 1, policy_version 44010 (0.0008) [2023-10-08 05:38:50,582][00612] Updated weights for policy 1, policy_version 44020 (0.0007) [2023-10-08 05:38:50,626][00611] Updated weights for policy 0, policy_version 43782 (0.0008) [2023-10-08 05:38:50,951][00612] Updated weights for policy 1, policy_version 44030 (0.0008) [2023-10-08 05:38:50,983][00611] Updated weights for policy 0, policy_version 43792 (0.0007) [2023-10-08 05:38:51,357][00611] Updated weights for policy 0, policy_version 43802 (0.0008) [2023-10-08 05:38:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 89948160. Throughput: 0: 1825.4, 1: 1836.3. Samples: 22491820. Policy #0 lag: (min: 11.0, avg: 20.2, max: 43.0) [2023-10-08 05:38:53,754][130385] Avg episode reward: [(0, '52.010'), (1, '67.280')] [2023-10-08 05:38:54,686][00612] Updated weights for policy 1, policy_version 44040 (0.0008) [2023-10-08 05:38:55,032][00611] Updated weights for policy 0, policy_version 43812 (0.0007) [2023-10-08 05:38:55,049][00612] Updated weights for policy 1, policy_version 44050 (0.0008) [2023-10-08 05:38:55,399][00611] Updated weights for policy 0, policy_version 43822 (0.0008) [2023-10-08 05:38:55,421][00612] Updated weights for policy 1, policy_version 44060 (0.0007) [2023-10-08 05:38:55,767][00611] Updated weights for policy 0, policy_version 43832 (0.0009) [2023-10-08 05:38:58,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 90013696. Throughput: 0: 1836.9, 1: 1846.6. Samples: 22514192. Policy #0 lag: (min: 11.0, avg: 20.2, max: 43.0) [2023-10-08 05:38:58,755][130385] Avg episode reward: [(0, '51.510'), (1, '65.900')] [2023-10-08 05:38:58,986][00612] Updated weights for policy 1, policy_version 44070 (0.0009) [2023-10-08 05:38:59,351][00612] Updated weights for policy 1, policy_version 44080 (0.0009) [2023-10-08 05:38:59,476][00611] Updated weights for policy 0, policy_version 43842 (0.0008) [2023-10-08 05:38:59,732][00612] Updated weights for policy 1, policy_version 44090 (0.0009) [2023-10-08 05:38:59,849][00611] Updated weights for policy 0, policy_version 43852 (0.0009) [2023-10-08 05:39:00,226][00611] Updated weights for policy 0, policy_version 43862 (0.0010) [2023-10-08 05:39:00,598][00611] Updated weights for policy 0, policy_version 43872 (0.0010) [2023-10-08 05:39:03,301][00612] Updated weights for policy 1, policy_version 44100 (0.0007) [2023-10-08 05:39:03,664][00612] Updated weights for policy 1, policy_version 44110 (0.0007) [2023-10-08 05:39:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 90079232. Throughput: 0: 1828.7, 1: 1846.8. Samples: 22537248. Policy #0 lag: (min: 11.0, avg: 20.2, max: 43.0) [2023-10-08 05:39:03,754][130385] Avg episode reward: [(0, '50.900'), (1, '66.380')] [2023-10-08 05:39:04,035][00612] Updated weights for policy 1, policy_version 44120 (0.0008) [2023-10-08 05:39:04,083][00611] Updated weights for policy 0, policy_version 43882 (0.0010) [2023-10-08 05:39:04,449][00611] Updated weights for policy 0, policy_version 43892 (0.0007) [2023-10-08 05:39:04,825][00611] Updated weights for policy 0, policy_version 43902 (0.0008) [2023-10-08 05:39:07,734][00612] Updated weights for policy 1, policy_version 44130 (0.0008) [2023-10-08 05:39:08,097][00612] Updated weights for policy 1, policy_version 44140 (0.0009) [2023-10-08 05:39:08,454][00611] Updated weights for policy 0, policy_version 43912 (0.0007) [2023-10-08 05:39:08,463][00612] Updated weights for policy 1, policy_version 44150 (0.0008) [2023-10-08 05:39:08,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90144768. Throughput: 0: 1832.7, 1: 1844.5. Samples: 22547228. Policy #0 lag: (min: 11.0, avg: 20.2, max: 43.0) [2023-10-08 05:39:08,754][130385] Avg episode reward: [(0, '49.780'), (1, '64.510')] [2023-10-08 05:39:08,815][00611] Updated weights for policy 0, policy_version 43922 (0.0007) [2023-10-08 05:39:08,834][00612] Updated weights for policy 1, policy_version 44160 (0.0008) [2023-10-08 05:39:09,185][00611] Updated weights for policy 0, policy_version 43932 (0.0007) [2023-10-08 05:39:12,479][00612] Updated weights for policy 1, policy_version 44170 (0.0008) [2023-10-08 05:39:12,851][00612] Updated weights for policy 1, policy_version 44180 (0.0007) [2023-10-08 05:39:12,903][00611] Updated weights for policy 0, policy_version 43942 (0.0008) [2023-10-08 05:39:13,220][00612] Updated weights for policy 1, policy_version 44190 (0.0009) [2023-10-08 05:39:13,274][00611] Updated weights for policy 0, policy_version 43952 (0.0008) [2023-10-08 05:39:13,646][00611] Updated weights for policy 0, policy_version 43962 (0.0008) [2023-10-08 05:39:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 90243072. Throughput: 0: 1832.3, 1: 1842.3. Samples: 22570352. Policy #0 lag: (min: 11.0, avg: 20.2, max: 43.0) [2023-10-08 05:39:13,754][130385] Avg episode reward: [(0, '50.690'), (1, '62.140')] [2023-10-08 05:39:16,825][00612] Updated weights for policy 1, policy_version 44200 (0.0007) [2023-10-08 05:39:17,203][00612] Updated weights for policy 1, policy_version 44210 (0.0007) [2023-10-08 05:39:17,342][00611] Updated weights for policy 0, policy_version 43972 (0.0008) [2023-10-08 05:39:17,557][00612] Updated weights for policy 1, policy_version 44220 (0.0008) [2023-10-08 05:39:17,714][00611] Updated weights for policy 0, policy_version 43982 (0.0007) [2023-10-08 05:39:18,092][00611] Updated weights for policy 0, policy_version 43992 (0.0009) [2023-10-08 05:39:18,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 90341376. Throughput: 0: 1823.8, 1: 1842.5. Samples: 22590832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:39:18,755][130385] Avg episode reward: [(0, '51.650'), (1, '60.510')] [2023-10-08 05:39:21,074][00612] Updated weights for policy 1, policy_version 44230 (0.0008) [2023-10-08 05:39:21,442][00612] Updated weights for policy 1, policy_version 44240 (0.0010) [2023-10-08 05:39:21,799][00611] Updated weights for policy 0, policy_version 44002 (0.0009) [2023-10-08 05:39:21,815][00612] Updated weights for policy 1, policy_version 44250 (0.0009) [2023-10-08 05:39:22,174][00611] Updated weights for policy 0, policy_version 44012 (0.0007) [2023-10-08 05:39:22,541][00611] Updated weights for policy 0, policy_version 44022 (0.0008) [2023-10-08 05:39:22,906][00611] Updated weights for policy 0, policy_version 44032 (0.0010) [2023-10-08 05:39:23,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 90406912. Throughput: 0: 1832.4, 1: 1836.7. Samples: 22603146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:39:23,755][130385] Avg episode reward: [(0, '56.180'), (1, '57.860')] [2023-10-08 05:39:25,479][00612] Updated weights for policy 1, policy_version 44260 (0.0007) [2023-10-08 05:39:25,852][00612] Updated weights for policy 1, policy_version 44270 (0.0008) [2023-10-08 05:39:26,209][00612] Updated weights for policy 1, policy_version 44280 (0.0007) [2023-10-08 05:39:26,596][00611] Updated weights for policy 0, policy_version 44042 (0.0008) [2023-10-08 05:39:26,973][00611] Updated weights for policy 0, policy_version 44052 (0.0007) [2023-10-08 05:39:27,339][00611] Updated weights for policy 0, policy_version 44062 (0.0007) [2023-10-08 05:39:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 90472448. Throughput: 0: 1819.0, 1: 1842.2. Samples: 22623734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:39:28,754][130385] Avg episode reward: [(0, '53.000'), (1, '61.000')] [2023-10-08 05:39:29,829][00612] Updated weights for policy 1, policy_version 44290 (0.0008) [2023-10-08 05:39:30,235][00612] Updated weights for policy 1, policy_version 44300 (0.0009) [2023-10-08 05:39:30,607][00612] Updated weights for policy 1, policy_version 44310 (0.0009) [2023-10-08 05:39:30,972][00612] Updated weights for policy 1, policy_version 44320 (0.0009) [2023-10-08 05:39:31,123][00611] Updated weights for policy 0, policy_version 44072 (0.0007) [2023-10-08 05:39:31,495][00611] Updated weights for policy 0, policy_version 44082 (0.0007) [2023-10-08 05:39:31,860][00611] Updated weights for policy 0, policy_version 44092 (0.0008) [2023-10-08 05:39:33,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 90537984. Throughput: 0: 1830.0, 1: 1849.9. Samples: 22646730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:39:33,755][130385] Avg episode reward: [(0, '51.130'), (1, '59.990')] [2023-10-08 05:39:34,400][00612] Updated weights for policy 1, policy_version 44330 (0.0009) [2023-10-08 05:39:34,767][00612] Updated weights for policy 1, policy_version 44340 (0.0007) [2023-10-08 05:39:35,133][00612] Updated weights for policy 1, policy_version 44350 (0.0008) [2023-10-08 05:39:35,483][00611] Updated weights for policy 0, policy_version 44102 (0.0010) [2023-10-08 05:39:35,859][00611] Updated weights for policy 0, policy_version 44112 (0.0011) [2023-10-08 05:39:36,222][00611] Updated weights for policy 0, policy_version 44122 (0.0009) [2023-10-08 05:39:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 90603520. Throughput: 0: 1822.6, 1: 1851.5. Samples: 22657156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:39:38,754][130385] Avg episode reward: [(0, '48.790'), (1, '60.680')] [2023-10-08 05:39:38,853][00612] Updated weights for policy 1, policy_version 44360 (0.0007) [2023-10-08 05:39:39,228][00612] Updated weights for policy 1, policy_version 44370 (0.0009) [2023-10-08 05:39:39,603][00612] Updated weights for policy 1, policy_version 44380 (0.0009) [2023-10-08 05:39:39,711][00611] Updated weights for policy 0, policy_version 44132 (0.0008) [2023-10-08 05:39:40,091][00611] Updated weights for policy 0, policy_version 44142 (0.0007) [2023-10-08 05:39:40,457][00611] Updated weights for policy 0, policy_version 44152 (0.0007) [2023-10-08 05:39:43,242][00612] Updated weights for policy 1, policy_version 44390 (0.0008) [2023-10-08 05:39:43,613][00612] Updated weights for policy 1, policy_version 44400 (0.0007) [2023-10-08 05:39:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 90669056. Throughput: 0: 1838.0, 1: 1855.4. Samples: 22680392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:39:43,755][130385] Avg episode reward: [(0, '49.650'), (1, '58.310')] [2023-10-08 05:39:43,842][00611] Updated weights for policy 0, policy_version 44162 (0.0009) [2023-10-08 05:39:43,973][00612] Updated weights for policy 1, policy_version 44410 (0.0007) [2023-10-08 05:39:44,225][00611] Updated weights for policy 0, policy_version 44172 (0.0009) [2023-10-08 05:39:44,599][00611] Updated weights for policy 0, policy_version 44182 (0.0010) [2023-10-08 05:39:44,974][00611] Updated weights for policy 0, policy_version 44192 (0.0010) [2023-10-08 05:39:47,512][00612] Updated weights for policy 1, policy_version 44420 (0.0008) [2023-10-08 05:39:47,878][00612] Updated weights for policy 1, policy_version 44430 (0.0010) [2023-10-08 05:39:48,252][00612] Updated weights for policy 1, policy_version 44440 (0.0007) [2023-10-08 05:39:48,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 90767360. Throughput: 0: 1842.6, 1: 1836.2. Samples: 22702796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:39:48,755][130385] Avg episode reward: [(0, '50.800'), (1, '59.000')] [2023-10-08 05:39:48,791][00611] Updated weights for policy 0, policy_version 44202 (0.0010) [2023-10-08 05:39:49,154][00611] Updated weights for policy 0, policy_version 44212 (0.0009) [2023-10-08 05:39:49,529][00611] Updated weights for policy 0, policy_version 44222 (0.0008) [2023-10-08 05:39:51,827][00612] Updated weights for policy 1, policy_version 44450 (0.0008) [2023-10-08 05:39:52,200][00612] Updated weights for policy 1, policy_version 44460 (0.0009) [2023-10-08 05:39:52,568][00612] Updated weights for policy 1, policy_version 44470 (0.0009) [2023-10-08 05:39:52,935][00612] Updated weights for policy 1, policy_version 44480 (0.0009) [2023-10-08 05:39:53,101][00611] Updated weights for policy 0, policy_version 44232 (0.0011) [2023-10-08 05:39:53,464][00611] Updated weights for policy 0, policy_version 44242 (0.0009) [2023-10-08 05:39:53,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 90832896. Throughput: 0: 1837.9, 1: 1863.6. Samples: 22713798. Policy #0 lag: (min: 1.0, avg: 1.1, max: 7.0) [2023-10-08 05:39:53,754][130385] Avg episode reward: [(0, '50.640'), (1, '62.520')] [2023-10-08 05:39:53,831][00611] Updated weights for policy 0, policy_version 44252 (0.0009) [2023-10-08 05:39:56,405][00612] Updated weights for policy 1, policy_version 44490 (0.0008) [2023-10-08 05:39:56,777][00612] Updated weights for policy 1, policy_version 44500 (0.0007) [2023-10-08 05:39:57,142][00612] Updated weights for policy 1, policy_version 44510 (0.0009) [2023-10-08 05:39:57,564][00611] Updated weights for policy 0, policy_version 44262 (0.0008) [2023-10-08 05:39:57,929][00611] Updated weights for policy 0, policy_version 44272 (0.0008) [2023-10-08 05:39:58,305][00611] Updated weights for policy 0, policy_version 44282 (0.0007) [2023-10-08 05:39:58,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 90931200. Throughput: 0: 1836.7, 1: 1828.5. Samples: 22735288. Policy #0 lag: (min: 1.0, avg: 1.1, max: 7.0) [2023-10-08 05:39:58,755][130385] Avg episode reward: [(0, '49.790'), (1, '64.140')] [2023-10-08 05:40:00,818][00612] Updated weights for policy 1, policy_version 44520 (0.0010) [2023-10-08 05:40:01,179][00612] Updated weights for policy 1, policy_version 44530 (0.0010) [2023-10-08 05:40:01,545][00612] Updated weights for policy 1, policy_version 44540 (0.0011) [2023-10-08 05:40:02,034][00611] Updated weights for policy 0, policy_version 44292 (0.0010) [2023-10-08 05:40:02,401][00611] Updated weights for policy 0, policy_version 44302 (0.0008) [2023-10-08 05:40:02,770][00611] Updated weights for policy 0, policy_version 44312 (0.0007) [2023-10-08 05:40:03,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 90996736. Throughput: 0: 1829.6, 1: 1856.3. Samples: 22756696. Policy #0 lag: (min: 1.0, avg: 1.1, max: 7.0) [2023-10-08 05:40:03,754][130385] Avg episode reward: [(0, '49.140'), (1, '65.230')] [2023-10-08 05:40:05,038][00612] Updated weights for policy 1, policy_version 44550 (0.0009) [2023-10-08 05:40:05,408][00612] Updated weights for policy 1, policy_version 44560 (0.0007) [2023-10-08 05:40:05,771][00612] Updated weights for policy 1, policy_version 44570 (0.0009) [2023-10-08 05:40:06,292][00611] Updated weights for policy 0, policy_version 44322 (0.0007) [2023-10-08 05:40:06,672][00611] Updated weights for policy 0, policy_version 44332 (0.0007) [2023-10-08 05:40:07,041][00611] Updated weights for policy 0, policy_version 44342 (0.0009) [2023-10-08 05:40:07,412][00611] Updated weights for policy 0, policy_version 44352 (0.0009) [2023-10-08 05:40:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 91062272. Throughput: 0: 1839.9, 1: 1834.0. Samples: 22768472. Policy #0 lag: (min: 1.0, avg: 1.1, max: 7.0) [2023-10-08 05:40:08,754][130385] Avg episode reward: [(0, '46.800'), (1, '64.230')] [2023-10-08 05:40:09,308][00612] Updated weights for policy 1, policy_version 44580 (0.0009) [2023-10-08 05:40:09,681][00612] Updated weights for policy 1, policy_version 44590 (0.0010) [2023-10-08 05:40:10,044][00612] Updated weights for policy 1, policy_version 44600 (0.0007) [2023-10-08 05:40:11,166][00611] Updated weights for policy 0, policy_version 44362 (0.0008) [2023-10-08 05:40:11,539][00611] Updated weights for policy 0, policy_version 44372 (0.0008) [2023-10-08 05:40:11,911][00611] Updated weights for policy 0, policy_version 44382 (0.0009) [2023-10-08 05:40:13,679][00612] Updated weights for policy 1, policy_version 44610 (0.0009) [2023-10-08 05:40:13,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 91127808. Throughput: 0: 1836.2, 1: 1867.5. Samples: 22790400. Policy #0 lag: (min: 1.0, avg: 1.1, max: 7.0) [2023-10-08 05:40:13,755][130385] Avg episode reward: [(0, '48.610'), (1, '65.250')] [2023-10-08 05:40:14,057][00612] Updated weights for policy 1, policy_version 44620 (0.0011) [2023-10-08 05:40:14,415][00612] Updated weights for policy 1, policy_version 44630 (0.0009) [2023-10-08 05:40:14,790][00612] Updated weights for policy 1, policy_version 44640 (0.0010) [2023-10-08 05:40:15,473][00611] Updated weights for policy 0, policy_version 44392 (0.0009) [2023-10-08 05:40:15,856][00611] Updated weights for policy 0, policy_version 44402 (0.0009) [2023-10-08 05:40:16,221][00611] Updated weights for policy 0, policy_version 44412 (0.0009) [2023-10-08 05:40:18,453][00612] Updated weights for policy 1, policy_version 44650 (0.0008) [2023-10-08 05:40:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 91193344. Throughput: 0: 1849.7, 1: 1854.5. Samples: 22813418. Policy #0 lag: (min: 1.0, avg: 1.1, max: 7.0) [2023-10-08 05:40:18,755][130385] Avg episode reward: [(0, '51.440'), (1, '66.270')] [2023-10-08 05:40:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000044416_45481984.pth... [2023-10-08 05:40:18,803][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000042720_43745280.pth [2023-10-08 05:40:18,827][00612] Updated weights for policy 1, policy_version 44660 (0.0008) [2023-10-08 05:40:19,184][00612] Updated weights for policy 1, policy_version 44670 (0.0008) [2023-10-08 05:40:19,259][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000044672_45744128.pth... [2023-10-08 05:40:19,296][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000042944_43974656.pth [2023-10-08 05:40:19,786][00611] Updated weights for policy 0, policy_version 44422 (0.0008) [2023-10-08 05:40:20,156][00611] Updated weights for policy 0, policy_version 44432 (0.0008) [2023-10-08 05:40:20,534][00611] Updated weights for policy 0, policy_version 44442 (0.0007) [2023-10-08 05:40:22,911][00612] Updated weights for policy 1, policy_version 44680 (0.0008) [2023-10-08 05:40:23,286][00612] Updated weights for policy 1, policy_version 44690 (0.0009) [2023-10-08 05:40:23,651][00612] Updated weights for policy 1, policy_version 44700 (0.0011) [2023-10-08 05:40:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 91258880. Throughput: 0: 1835.9, 1: 1857.2. Samples: 22823342. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:40:23,754][130385] Avg episode reward: [(0, '52.750'), (1, '66.850')] [2023-10-08 05:40:24,285][00611] Updated weights for policy 0, policy_version 44452 (0.0009) [2023-10-08 05:40:24,656][00611] Updated weights for policy 0, policy_version 44462 (0.0009) [2023-10-08 05:40:25,026][00611] Updated weights for policy 0, policy_version 44472 (0.0010) [2023-10-08 05:40:27,372][00612] Updated weights for policy 1, policy_version 44710 (0.0010) [2023-10-08 05:40:27,731][00612] Updated weights for policy 1, policy_version 44720 (0.0009) [2023-10-08 05:40:28,105][00612] Updated weights for policy 1, policy_version 44730 (0.0008) [2023-10-08 05:40:28,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 91357184. Throughput: 0: 1829.0, 1: 1851.7. Samples: 22846026. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:40:28,755][130385] Avg episode reward: [(0, '56.350'), (1, '65.620')] [2023-10-08 05:40:28,802][00611] Updated weights for policy 0, policy_version 44482 (0.0009) [2023-10-08 05:40:29,172][00611] Updated weights for policy 0, policy_version 44492 (0.0008) [2023-10-08 05:40:29,543][00611] Updated weights for policy 0, policy_version 44502 (0.0008) [2023-10-08 05:40:29,925][00611] Updated weights for policy 0, policy_version 44512 (0.0008) [2023-10-08 05:40:31,744][00612] Updated weights for policy 1, policy_version 44740 (0.0010) [2023-10-08 05:40:32,111][00612] Updated weights for policy 1, policy_version 44750 (0.0007) [2023-10-08 05:40:32,475][00612] Updated weights for policy 1, policy_version 44760 (0.0008) [2023-10-08 05:40:33,581][00611] Updated weights for policy 0, policy_version 44522 (0.0007) [2023-10-08 05:40:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 91422720. Throughput: 0: 1824.2, 1: 1842.8. Samples: 22867812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:40:33,754][130385] Avg episode reward: [(0, '56.400'), (1, '65.330')] [2023-10-08 05:40:33,962][00611] Updated weights for policy 0, policy_version 44532 (0.0008) [2023-10-08 05:40:34,331][00611] Updated weights for policy 0, policy_version 44542 (0.0007) [2023-10-08 05:40:35,971][00612] Updated weights for policy 1, policy_version 44770 (0.0007) [2023-10-08 05:40:36,336][00612] Updated weights for policy 1, policy_version 44780 (0.0010) [2023-10-08 05:40:36,707][00612] Updated weights for policy 1, policy_version 44790 (0.0011) [2023-10-08 05:40:37,072][00612] Updated weights for policy 1, policy_version 44800 (0.0008) [2023-10-08 05:40:38,139][00611] Updated weights for policy 0, policy_version 44552 (0.0007) [2023-10-08 05:40:38,509][00611] Updated weights for policy 0, policy_version 44562 (0.0007) [2023-10-08 05:40:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 91488256. Throughput: 0: 1825.3, 1: 1849.3. Samples: 22879156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:40:38,756][130385] Avg episode reward: [(0, '56.230'), (1, '64.440')] [2023-10-08 05:40:38,879][00611] Updated weights for policy 0, policy_version 44572 (0.0008) [2023-10-08 05:40:40,623][00612] Updated weights for policy 1, policy_version 44810 (0.0008) [2023-10-08 05:40:40,983][00612] Updated weights for policy 1, policy_version 44820 (0.0007) [2023-10-08 05:40:41,352][00612] Updated weights for policy 1, policy_version 44830 (0.0007) [2023-10-08 05:40:42,569][00611] Updated weights for policy 0, policy_version 44582 (0.0009) [2023-10-08 05:40:42,941][00611] Updated weights for policy 0, policy_version 44592 (0.0008) [2023-10-08 05:40:43,317][00611] Updated weights for policy 0, policy_version 44602 (0.0009) [2023-10-08 05:40:43,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 91586560. Throughput: 0: 1820.8, 1: 1860.5. Samples: 22900946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:40:43,754][130385] Avg episode reward: [(0, '57.050'), (1, '68.140')] [2023-10-08 05:40:44,959][00612] Updated weights for policy 1, policy_version 44840 (0.0007) [2023-10-08 05:40:45,336][00612] Updated weights for policy 1, policy_version 44850 (0.0009) [2023-10-08 05:40:45,705][00612] Updated weights for policy 1, policy_version 44860 (0.0008) [2023-10-08 05:40:46,960][00611] Updated weights for policy 0, policy_version 44612 (0.0008) [2023-10-08 05:40:47,333][00611] Updated weights for policy 0, policy_version 44622 (0.0008) [2023-10-08 05:40:47,693][00611] Updated weights for policy 0, policy_version 44632 (0.0010) [2023-10-08 05:40:48,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 91652096. Throughput: 0: 1818.3, 1: 1864.8. Samples: 22922434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:40:48,754][130385] Avg episode reward: [(0, '59.230'), (1, '70.030')] [2023-10-08 05:40:49,294][00612] Updated weights for policy 1, policy_version 44870 (0.0007) [2023-10-08 05:40:49,665][00612] Updated weights for policy 1, policy_version 44880 (0.0007) [2023-10-08 05:40:50,032][00612] Updated weights for policy 1, policy_version 44890 (0.0007) [2023-10-08 05:40:51,497][00611] Updated weights for policy 0, policy_version 44642 (0.0010) [2023-10-08 05:40:51,865][00611] Updated weights for policy 0, policy_version 44652 (0.0008) [2023-10-08 05:40:52,237][00611] Updated weights for policy 0, policy_version 44662 (0.0007) [2023-10-08 05:40:52,606][00611] Updated weights for policy 0, policy_version 44672 (0.0007) [2023-10-08 05:40:53,577][00612] Updated weights for policy 1, policy_version 44900 (0.0009) [2023-10-08 05:40:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 91717632. Throughput: 0: 1818.8, 1: 1857.3. Samples: 22933894. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-08 05:40:53,754][130385] Avg episode reward: [(0, '59.860'), (1, '70.300')] [2023-10-08 05:40:53,943][00612] Updated weights for policy 1, policy_version 44910 (0.0010) [2023-10-08 05:40:54,314][00612] Updated weights for policy 1, policy_version 44920 (0.0008) [2023-10-08 05:40:56,316][00611] Updated weights for policy 0, policy_version 44682 (0.0009) [2023-10-08 05:40:56,699][00611] Updated weights for policy 0, policy_version 44692 (0.0009) [2023-10-08 05:40:57,071][00611] Updated weights for policy 0, policy_version 44702 (0.0010) [2023-10-08 05:40:58,049][00612] Updated weights for policy 1, policy_version 44930 (0.0010) [2023-10-08 05:40:58,423][00612] Updated weights for policy 1, policy_version 44940 (0.0007) [2023-10-08 05:40:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 91783168. Throughput: 0: 1814.6, 1: 1859.4. Samples: 22955730. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-08 05:40:58,755][130385] Avg episode reward: [(0, '59.470'), (1, '71.840')] [2023-10-08 05:40:58,803][00612] Updated weights for policy 1, policy_version 44950 (0.0008) [2023-10-08 05:40:59,168][00612] Updated weights for policy 1, policy_version 44960 (0.0009) [2023-10-08 05:41:00,699][00611] Updated weights for policy 0, policy_version 44712 (0.0008) [2023-10-08 05:41:01,076][00611] Updated weights for policy 0, policy_version 44722 (0.0010) [2023-10-08 05:41:01,438][00611] Updated weights for policy 0, policy_version 44732 (0.0008) [2023-10-08 05:41:02,652][00612] Updated weights for policy 1, policy_version 44970 (0.0008) [2023-10-08 05:41:03,009][00612] Updated weights for policy 1, policy_version 44980 (0.0007) [2023-10-08 05:41:03,379][00612] Updated weights for policy 1, policy_version 44990 (0.0008) [2023-10-08 05:41:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 91881472. Throughput: 0: 1809.0, 1: 1843.2. Samples: 22977766. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-08 05:41:03,754][130385] Avg episode reward: [(0, '58.070'), (1, '73.020')] [2023-10-08 05:41:05,027][00611] Updated weights for policy 0, policy_version 44742 (0.0009) [2023-10-08 05:41:05,406][00611] Updated weights for policy 0, policy_version 44752 (0.0009) [2023-10-08 05:41:05,776][00611] Updated weights for policy 0, policy_version 44762 (0.0009) [2023-10-08 05:41:07,216][00612] Updated weights for policy 1, policy_version 45000 (0.0008) [2023-10-08 05:41:07,595][00612] Updated weights for policy 1, policy_version 45010 (0.0008) [2023-10-08 05:41:07,971][00612] Updated weights for policy 1, policy_version 45020 (0.0010) [2023-10-08 05:41:08,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 91947008. Throughput: 0: 1813.8, 1: 1862.6. Samples: 22988780. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-08 05:41:08,754][130385] Avg episode reward: [(0, '59.910'), (1, '69.800')] [2023-10-08 05:41:09,393][00611] Updated weights for policy 0, policy_version 44772 (0.0007) [2023-10-08 05:41:09,758][00611] Updated weights for policy 0, policy_version 44782 (0.0009) [2023-10-08 05:41:10,129][00611] Updated weights for policy 0, policy_version 44792 (0.0009) [2023-10-08 05:41:11,623][00612] Updated weights for policy 1, policy_version 45030 (0.0009) [2023-10-08 05:41:11,986][00612] Updated weights for policy 1, policy_version 45040 (0.0007) [2023-10-08 05:41:12,353][00612] Updated weights for policy 1, policy_version 45050 (0.0008) [2023-10-08 05:41:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 92012544. Throughput: 0: 1820.5, 1: 1840.6. Samples: 23010772. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-08 05:41:13,754][130385] Avg episode reward: [(0, '59.140'), (1, '70.350')] [2023-10-08 05:41:13,903][00611] Updated weights for policy 0, policy_version 44802 (0.0008) [2023-10-08 05:41:14,284][00611] Updated weights for policy 0, policy_version 44812 (0.0009) [2023-10-08 05:41:14,663][00611] Updated weights for policy 0, policy_version 44822 (0.0010) [2023-10-08 05:41:15,035][00611] Updated weights for policy 0, policy_version 44832 (0.0007) [2023-10-08 05:41:15,990][00612] Updated weights for policy 1, policy_version 45060 (0.0007) [2023-10-08 05:41:16,368][00612] Updated weights for policy 1, policy_version 45070 (0.0010) [2023-10-08 05:41:16,741][00612] Updated weights for policy 1, policy_version 45080 (0.0010) [2023-10-08 05:41:18,617][00611] Updated weights for policy 0, policy_version 44842 (0.0009) [2023-10-08 05:41:18,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92078080. Throughput: 0: 1823.2, 1: 1853.8. Samples: 23033278. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-08 05:41:18,755][130385] Avg episode reward: [(0, '59.320'), (1, '67.680')] [2023-10-08 05:41:18,981][00611] Updated weights for policy 0, policy_version 44852 (0.0010) [2023-10-08 05:41:19,357][00611] Updated weights for policy 0, policy_version 44862 (0.0010) [2023-10-08 05:41:20,493][00612] Updated weights for policy 1, policy_version 45090 (0.0008) [2023-10-08 05:41:20,868][00612] Updated weights for policy 1, policy_version 45100 (0.0010) [2023-10-08 05:41:21,227][00612] Updated weights for policy 1, policy_version 45110 (0.0007) [2023-10-08 05:41:21,594][00612] Updated weights for policy 1, policy_version 45120 (0.0007) [2023-10-08 05:41:23,140][00611] Updated weights for policy 0, policy_version 44872 (0.0009) [2023-10-08 05:41:23,523][00611] Updated weights for policy 0, policy_version 44882 (0.0008) [2023-10-08 05:41:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92143616. Throughput: 0: 1823.7, 1: 1832.9. Samples: 23043702. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-08 05:41:23,754][130385] Avg episode reward: [(0, '62.700'), (1, '67.390')] [2023-10-08 05:41:23,895][00611] Updated weights for policy 0, policy_version 44892 (0.0010) [2023-10-08 05:41:25,246][00612] Updated weights for policy 1, policy_version 45130 (0.0008) [2023-10-08 05:41:25,616][00612] Updated weights for policy 1, policy_version 45140 (0.0007) [2023-10-08 05:41:25,985][00612] Updated weights for policy 1, policy_version 45150 (0.0007) [2023-10-08 05:41:27,445][00611] Updated weights for policy 0, policy_version 44902 (0.0008) [2023-10-08 05:41:27,820][00611] Updated weights for policy 0, policy_version 44912 (0.0007) [2023-10-08 05:41:28,198][00611] Updated weights for policy 0, policy_version 44922 (0.0007) [2023-10-08 05:41:28,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92241920. Throughput: 0: 1824.7, 1: 1844.8. Samples: 23066076. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 05:41:28,755][130385] Avg episode reward: [(0, '59.460'), (1, '67.580')] [2023-10-08 05:41:29,570][00612] Updated weights for policy 1, policy_version 45160 (0.0007) [2023-10-08 05:41:29,934][00612] Updated weights for policy 1, policy_version 45170 (0.0007) [2023-10-08 05:41:30,310][00612] Updated weights for policy 1, policy_version 45180 (0.0007) [2023-10-08 05:41:31,645][00611] Updated weights for policy 0, policy_version 44932 (0.0008) [2023-10-08 05:41:32,020][00611] Updated weights for policy 0, policy_version 44942 (0.0008) [2023-10-08 05:41:32,383][00611] Updated weights for policy 0, policy_version 44952 (0.0010) [2023-10-08 05:41:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92307456. Throughput: 0: 1831.8, 1: 1846.4. Samples: 23087956. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 05:41:33,754][130385] Avg episode reward: [(0, '57.390'), (1, '63.730')] [2023-10-08 05:41:34,027][00612] Updated weights for policy 1, policy_version 45190 (0.0008) [2023-10-08 05:41:34,395][00612] Updated weights for policy 1, policy_version 45200 (0.0008) [2023-10-08 05:41:34,773][00612] Updated weights for policy 1, policy_version 45210 (0.0009) [2023-10-08 05:41:36,070][00611] Updated weights for policy 0, policy_version 44962 (0.0008) [2023-10-08 05:41:36,445][00611] Updated weights for policy 0, policy_version 44972 (0.0011) [2023-10-08 05:41:36,815][00611] Updated weights for policy 0, policy_version 44982 (0.0008) [2023-10-08 05:41:37,192][00611] Updated weights for policy 0, policy_version 44992 (0.0007) [2023-10-08 05:41:38,266][00612] Updated weights for policy 1, policy_version 45220 (0.0008) [2023-10-08 05:41:38,635][00612] Updated weights for policy 1, policy_version 45230 (0.0009) [2023-10-08 05:41:38,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92372992. Throughput: 0: 1828.0, 1: 1848.1. Samples: 23099320. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 05:41:38,754][130385] Avg episode reward: [(0, '57.110'), (1, '68.840')] [2023-10-08 05:41:39,002][00612] Updated weights for policy 1, policy_version 45240 (0.0009) [2023-10-08 05:41:40,876][00611] Updated weights for policy 0, policy_version 45002 (0.0008) [2023-10-08 05:41:41,236][00611] Updated weights for policy 0, policy_version 45012 (0.0007) [2023-10-08 05:41:41,604][00611] Updated weights for policy 0, policy_version 45022 (0.0009) [2023-10-08 05:41:42,771][00612] Updated weights for policy 1, policy_version 45250 (0.0010) [2023-10-08 05:41:43,139][00612] Updated weights for policy 1, policy_version 45260 (0.0008) [2023-10-08 05:41:43,509][00612] Updated weights for policy 1, policy_version 45270 (0.0007) [2023-10-08 05:41:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 92438528. Throughput: 0: 1835.4, 1: 1835.9. Samples: 23120938. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 05:41:43,754][130385] Avg episode reward: [(0, '56.390'), (1, '65.220')] [2023-10-08 05:41:43,878][00612] Updated weights for policy 1, policy_version 45280 (0.0007) [2023-10-08 05:41:45,338][00611] Updated weights for policy 0, policy_version 45032 (0.0009) [2023-10-08 05:41:45,714][00611] Updated weights for policy 0, policy_version 45042 (0.0010) [2023-10-08 05:41:46,081][00611] Updated weights for policy 0, policy_version 45052 (0.0009) [2023-10-08 05:41:47,392][00612] Updated weights for policy 1, policy_version 45290 (0.0009) [2023-10-08 05:41:47,765][00612] Updated weights for policy 1, policy_version 45300 (0.0010) [2023-10-08 05:41:48,127][00612] Updated weights for policy 1, policy_version 45310 (0.0007) [2023-10-08 05:41:48,754][130385] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 92536832. Throughput: 0: 1838.3, 1: 1831.4. Samples: 23142902. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 05:41:48,756][130385] Avg episode reward: [(0, '55.540'), (1, '60.910')] [2023-10-08 05:41:49,577][00611] Updated weights for policy 0, policy_version 45062 (0.0008) [2023-10-08 05:41:49,954][00611] Updated weights for policy 0, policy_version 45072 (0.0008) [2023-10-08 05:41:50,317][00611] Updated weights for policy 0, policy_version 45082 (0.0009) [2023-10-08 05:41:51,732][00612] Updated weights for policy 1, policy_version 45320 (0.0009) [2023-10-08 05:41:52,112][00612] Updated weights for policy 1, policy_version 45330 (0.0010) [2023-10-08 05:41:52,478][00612] Updated weights for policy 1, policy_version 45340 (0.0011) [2023-10-08 05:41:53,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 92602368. Throughput: 0: 1836.7, 1: 1843.9. Samples: 23154406. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 05:41:53,754][130385] Avg episode reward: [(0, '56.150'), (1, '58.690')] [2023-10-08 05:41:53,878][00611] Updated weights for policy 0, policy_version 45092 (0.0008) [2023-10-08 05:41:54,244][00611] Updated weights for policy 0, policy_version 45102 (0.0007) [2023-10-08 05:41:54,615][00611] Updated weights for policy 0, policy_version 45112 (0.0008) [2023-10-08 05:41:56,148][00612] Updated weights for policy 1, policy_version 45350 (0.0009) [2023-10-08 05:41:56,517][00612] Updated weights for policy 1, policy_version 45360 (0.0008) [2023-10-08 05:41:56,899][00612] Updated weights for policy 1, policy_version 45370 (0.0008) [2023-10-08 05:41:58,279][00611] Updated weights for policy 0, policy_version 45122 (0.0009) [2023-10-08 05:41:58,655][00611] Updated weights for policy 0, policy_version 45132 (0.0008) [2023-10-08 05:41:58,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92667904. Throughput: 0: 1841.1, 1: 1827.4. Samples: 23175856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:41:58,755][130385] Avg episode reward: [(0, '57.220'), (1, '58.840')] [2023-10-08 05:41:59,025][00611] Updated weights for policy 0, policy_version 45142 (0.0008) [2023-10-08 05:41:59,397][00611] Updated weights for policy 0, policy_version 45152 (0.0007) [2023-10-08 05:42:00,465][00612] Updated weights for policy 1, policy_version 45380 (0.0009) [2023-10-08 05:42:00,835][00612] Updated weights for policy 1, policy_version 45390 (0.0009) [2023-10-08 05:42:01,194][00612] Updated weights for policy 1, policy_version 45400 (0.0007) [2023-10-08 05:42:03,041][00611] Updated weights for policy 0, policy_version 45162 (0.0009) [2023-10-08 05:42:03,415][00611] Updated weights for policy 0, policy_version 45172 (0.0008) [2023-10-08 05:42:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 92733440. Throughput: 0: 1824.9, 1: 1844.9. Samples: 23198418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:42:03,755][130385] Avg episode reward: [(0, '57.220'), (1, '56.840')] [2023-10-08 05:42:03,781][00611] Updated weights for policy 0, policy_version 45182 (0.0008) [2023-10-08 05:42:04,713][00612] Updated weights for policy 1, policy_version 45410 (0.0007) [2023-10-08 05:42:05,079][00612] Updated weights for policy 1, policy_version 45420 (0.0009) [2023-10-08 05:42:05,440][00612] Updated weights for policy 1, policy_version 45430 (0.0008) [2023-10-08 05:42:05,810][00612] Updated weights for policy 1, policy_version 45440 (0.0009) [2023-10-08 05:42:07,454][00611] Updated weights for policy 0, policy_version 45192 (0.0010) [2023-10-08 05:42:07,830][00611] Updated weights for policy 0, policy_version 45202 (0.0007) [2023-10-08 05:42:08,192][00611] Updated weights for policy 0, policy_version 45212 (0.0008) [2023-10-08 05:42:08,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92831744. Throughput: 0: 1840.2, 1: 1833.8. Samples: 23209030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:42:08,754][130385] Avg episode reward: [(0, '60.540'), (1, '55.280')] [2023-10-08 05:42:09,531][00612] Updated weights for policy 1, policy_version 45450 (0.0008) [2023-10-08 05:42:09,895][00612] Updated weights for policy 1, policy_version 45460 (0.0010) [2023-10-08 05:42:10,257][00612] Updated weights for policy 1, policy_version 45470 (0.0011) [2023-10-08 05:42:11,958][00611] Updated weights for policy 0, policy_version 45222 (0.0009) [2023-10-08 05:42:12,342][00611] Updated weights for policy 0, policy_version 45232 (0.0008) [2023-10-08 05:42:12,712][00611] Updated weights for policy 0, policy_version 45242 (0.0008) [2023-10-08 05:42:13,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92897280. Throughput: 0: 1836.9, 1: 1844.9. Samples: 23231756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:42:13,754][130385] Avg episode reward: [(0, '57.470'), (1, '55.130')] [2023-10-08 05:42:13,851][00612] Updated weights for policy 1, policy_version 45480 (0.0009) [2023-10-08 05:42:14,228][00612] Updated weights for policy 1, policy_version 45490 (0.0009) [2023-10-08 05:42:14,594][00612] Updated weights for policy 1, policy_version 45500 (0.0008) [2023-10-08 05:42:16,198][00611] Updated weights for policy 0, policy_version 45252 (0.0009) [2023-10-08 05:42:16,574][00611] Updated weights for policy 0, policy_version 45262 (0.0009) [2023-10-08 05:42:16,947][00611] Updated weights for policy 0, policy_version 45272 (0.0009) [2023-10-08 05:42:18,329][00612] Updated weights for policy 1, policy_version 45510 (0.0008) [2023-10-08 05:42:18,703][00612] Updated weights for policy 1, policy_version 45520 (0.0009) [2023-10-08 05:42:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 92962816. Throughput: 0: 1841.9, 1: 1841.4. Samples: 23253704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:42:18,754][130385] Avg episode reward: [(0, '59.950'), (1, '58.080')] [2023-10-08 05:42:18,762][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000045280_46366720.pth... [2023-10-08 05:42:18,796][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000043552_44597248.pth [2023-10-08 05:42:19,070][00612] Updated weights for policy 1, policy_version 45530 (0.0008) [2023-10-08 05:42:19,280][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000045536_46628864.pth... [2023-10-08 05:42:19,310][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000043808_44859392.pth [2023-10-08 05:42:20,597][00611] Updated weights for policy 0, policy_version 45282 (0.0009) [2023-10-08 05:42:20,970][00611] Updated weights for policy 0, policy_version 45292 (0.0008) [2023-10-08 05:42:21,343][00611] Updated weights for policy 0, policy_version 45302 (0.0007) [2023-10-08 05:42:21,720][00611] Updated weights for policy 0, policy_version 45312 (0.0009) [2023-10-08 05:42:22,670][00612] Updated weights for policy 1, policy_version 45540 (0.0007) [2023-10-08 05:42:23,041][00612] Updated weights for policy 1, policy_version 45550 (0.0008) [2023-10-08 05:42:23,411][00612] Updated weights for policy 1, policy_version 45560 (0.0008) [2023-10-08 05:42:23,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 93061120. Throughput: 0: 1828.4, 1: 1845.0. Samples: 23264622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:42:23,755][130385] Avg episode reward: [(0, '60.450'), (1, '57.710')] [2023-10-08 05:42:25,392][00611] Updated weights for policy 0, policy_version 45322 (0.0009) [2023-10-08 05:42:25,770][00611] Updated weights for policy 0, policy_version 45332 (0.0008) [2023-10-08 05:42:26,139][00611] Updated weights for policy 0, policy_version 45342 (0.0007) [2023-10-08 05:42:27,008][00612] Updated weights for policy 1, policy_version 45570 (0.0007) [2023-10-08 05:42:27,376][00612] Updated weights for policy 1, policy_version 45580 (0.0010) [2023-10-08 05:42:27,757][00612] Updated weights for policy 1, policy_version 45590 (0.0011) [2023-10-08 05:42:28,124][00612] Updated weights for policy 1, policy_version 45600 (0.0009) [2023-10-08 05:42:28,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 93126656. Throughput: 0: 1842.3, 1: 1843.1. Samples: 23286782. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 05:42:28,755][130385] Avg episode reward: [(0, '62.780'), (1, '56.100')] [2023-10-08 05:42:29,940][00611] Updated weights for policy 0, policy_version 45352 (0.0007) [2023-10-08 05:42:30,307][00611] Updated weights for policy 0, policy_version 45362 (0.0007) [2023-10-08 05:42:30,688][00611] Updated weights for policy 0, policy_version 45372 (0.0009) [2023-10-08 05:42:31,759][00612] Updated weights for policy 1, policy_version 45610 (0.0008) [2023-10-08 05:42:32,125][00612] Updated weights for policy 1, policy_version 45620 (0.0007) [2023-10-08 05:42:32,504][00612] Updated weights for policy 1, policy_version 45630 (0.0008) [2023-10-08 05:42:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 93192192. Throughput: 0: 1839.2, 1: 1843.8. Samples: 23308636. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 05:42:33,754][130385] Avg episode reward: [(0, '61.860'), (1, '57.140')] [2023-10-08 05:42:34,175][00611] Updated weights for policy 0, policy_version 45382 (0.0010) [2023-10-08 05:42:34,537][00611] Updated weights for policy 0, policy_version 45392 (0.0009) [2023-10-08 05:42:34,923][00611] Updated weights for policy 0, policy_version 45402 (0.0011) [2023-10-08 05:42:36,244][00612] Updated weights for policy 1, policy_version 45640 (0.0009) [2023-10-08 05:42:36,610][00612] Updated weights for policy 1, policy_version 45650 (0.0009) [2023-10-08 05:42:36,989][00612] Updated weights for policy 1, policy_version 45660 (0.0009) [2023-10-08 05:42:38,688][00611] Updated weights for policy 0, policy_version 45412 (0.0010) [2023-10-08 05:42:38,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93257728. Throughput: 0: 1836.8, 1: 1834.5. Samples: 23319612. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 05:42:38,754][130385] Avg episode reward: [(0, '59.500'), (1, '55.790')] [2023-10-08 05:42:39,051][00611] Updated weights for policy 0, policy_version 45422 (0.0010) [2023-10-08 05:42:39,430][00611] Updated weights for policy 0, policy_version 45432 (0.0009) [2023-10-08 05:42:40,665][00612] Updated weights for policy 1, policy_version 45670 (0.0008) [2023-10-08 05:42:41,026][00612] Updated weights for policy 1, policy_version 45680 (0.0011) [2023-10-08 05:42:41,396][00612] Updated weights for policy 1, policy_version 45690 (0.0012) [2023-10-08 05:42:43,057][00611] Updated weights for policy 0, policy_version 45442 (0.0010) [2023-10-08 05:42:43,424][00611] Updated weights for policy 0, policy_version 45452 (0.0011) [2023-10-08 05:42:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 93323264. Throughput: 0: 1832.5, 1: 1844.9. Samples: 23341342. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 05:42:43,754][130385] Avg episode reward: [(0, '59.040'), (1, '58.960')] [2023-10-08 05:42:43,803][00611] Updated weights for policy 0, policy_version 45462 (0.0011) [2023-10-08 05:42:44,169][00611] Updated weights for policy 0, policy_version 45472 (0.0008) [2023-10-08 05:42:45,051][00612] Updated weights for policy 1, policy_version 45700 (0.0009) [2023-10-08 05:42:45,434][00612] Updated weights for policy 1, policy_version 45710 (0.0007) [2023-10-08 05:42:45,797][00612] Updated weights for policy 1, policy_version 45720 (0.0010) [2023-10-08 05:42:47,783][00611] Updated weights for policy 0, policy_version 45482 (0.0009) [2023-10-08 05:42:48,158][00611] Updated weights for policy 0, policy_version 45492 (0.0011) [2023-10-08 05:42:48,534][00611] Updated weights for policy 0, policy_version 45502 (0.0010) [2023-10-08 05:42:48,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93421568. Throughput: 0: 1824.6, 1: 1846.7. Samples: 23363628. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 05:42:48,755][130385] Avg episode reward: [(0, '55.720'), (1, '59.760')] [2023-10-08 05:42:49,362][00612] Updated weights for policy 1, policy_version 45730 (0.0007) [2023-10-08 05:42:49,737][00612] Updated weights for policy 1, policy_version 45740 (0.0008) [2023-10-08 05:42:50,108][00612] Updated weights for policy 1, policy_version 45750 (0.0008) [2023-10-08 05:42:50,479][00612] Updated weights for policy 1, policy_version 45760 (0.0008) [2023-10-08 05:42:52,203][00611] Updated weights for policy 0, policy_version 45512 (0.0008) [2023-10-08 05:42:52,571][00611] Updated weights for policy 0, policy_version 45522 (0.0011) [2023-10-08 05:42:52,939][00611] Updated weights for policy 0, policy_version 45532 (0.0009) [2023-10-08 05:42:53,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93487104. Throughput: 0: 1830.4, 1: 1842.5. Samples: 23374314. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 05:42:53,754][130385] Avg episode reward: [(0, '54.430'), (1, '62.850')] [2023-10-08 05:42:54,318][00612] Updated weights for policy 1, policy_version 45770 (0.0008) [2023-10-08 05:42:54,682][00612] Updated weights for policy 1, policy_version 45780 (0.0008) [2023-10-08 05:42:55,047][00612] Updated weights for policy 1, policy_version 45790 (0.0007) [2023-10-08 05:42:56,693][00611] Updated weights for policy 0, policy_version 45542 (0.0009) [2023-10-08 05:42:57,079][00611] Updated weights for policy 0, policy_version 45552 (0.0010) [2023-10-08 05:42:57,452][00611] Updated weights for policy 0, policy_version 45562 (0.0007) [2023-10-08 05:42:58,591][00612] Updated weights for policy 1, policy_version 45800 (0.0007) [2023-10-08 05:42:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93552640. Throughput: 0: 1816.2, 1: 1842.2. Samples: 23396386. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:42:58,755][130385] Avg episode reward: [(0, '51.590'), (1, '62.770')] [2023-10-08 05:42:58,968][00612] Updated weights for policy 1, policy_version 45810 (0.0009) [2023-10-08 05:42:59,328][00612] Updated weights for policy 1, policy_version 45820 (0.0008) [2023-10-08 05:43:01,039][00611] Updated weights for policy 0, policy_version 45572 (0.0009) [2023-10-08 05:43:01,409][00611] Updated weights for policy 0, policy_version 45582 (0.0009) [2023-10-08 05:43:01,784][00611] Updated weights for policy 0, policy_version 45592 (0.0008) [2023-10-08 05:43:03,161][00612] Updated weights for policy 1, policy_version 45830 (0.0008) [2023-10-08 05:43:03,525][00612] Updated weights for policy 1, policy_version 45840 (0.0007) [2023-10-08 05:43:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93618176. Throughput: 0: 1827.5, 1: 1834.0. Samples: 23418470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:43:03,754][130385] Avg episode reward: [(0, '52.770'), (1, '65.500')] [2023-10-08 05:43:03,892][00612] Updated weights for policy 1, policy_version 45850 (0.0010) [2023-10-08 05:43:05,406][00611] Updated weights for policy 0, policy_version 45602 (0.0008) [2023-10-08 05:43:05,764][00611] Updated weights for policy 0, policy_version 45612 (0.0010) [2023-10-08 05:43:06,142][00611] Updated weights for policy 0, policy_version 45622 (0.0010) [2023-10-08 05:43:06,508][00611] Updated weights for policy 0, policy_version 45632 (0.0011) [2023-10-08 05:43:07,415][00612] Updated weights for policy 1, policy_version 45860 (0.0007) [2023-10-08 05:43:07,785][00612] Updated weights for policy 1, policy_version 45870 (0.0008) [2023-10-08 05:43:08,149][00612] Updated weights for policy 1, policy_version 45880 (0.0008) [2023-10-08 05:43:08,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 93716480. Throughput: 0: 1820.0, 1: 1840.7. Samples: 23429358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:43:08,755][130385] Avg episode reward: [(0, '54.610'), (1, '66.900')] [2023-10-08 05:43:10,194][00611] Updated weights for policy 0, policy_version 45642 (0.0008) [2023-10-08 05:43:10,553][00611] Updated weights for policy 0, policy_version 45652 (0.0011) [2023-10-08 05:43:10,931][00611] Updated weights for policy 0, policy_version 45662 (0.0010) [2023-10-08 05:43:11,755][00612] Updated weights for policy 1, policy_version 45890 (0.0009) [2023-10-08 05:43:12,123][00612] Updated weights for policy 1, policy_version 45900 (0.0009) [2023-10-08 05:43:12,502][00612] Updated weights for policy 1, policy_version 45910 (0.0007) [2023-10-08 05:43:12,866][00612] Updated weights for policy 1, policy_version 45920 (0.0007) [2023-10-08 05:43:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 93782016. Throughput: 0: 1829.6, 1: 1834.2. Samples: 23451656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:43:13,754][130385] Avg episode reward: [(0, '56.110'), (1, '69.220')] [2023-10-08 05:43:14,571][00611] Updated weights for policy 0, policy_version 45672 (0.0010) [2023-10-08 05:43:14,949][00611] Updated weights for policy 0, policy_version 45682 (0.0008) [2023-10-08 05:43:15,327][00611] Updated weights for policy 0, policy_version 45692 (0.0010) [2023-10-08 05:43:16,421][00612] Updated weights for policy 1, policy_version 45930 (0.0008) [2023-10-08 05:43:16,791][00612] Updated weights for policy 1, policy_version 45940 (0.0009) [2023-10-08 05:43:17,155][00612] Updated weights for policy 1, policy_version 45950 (0.0009) [2023-10-08 05:43:18,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 93847552. Throughput: 0: 1833.3, 1: 1843.9. Samples: 23474110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:43:18,754][130385] Avg episode reward: [(0, '54.440'), (1, '69.200')] [2023-10-08 05:43:18,869][00611] Updated weights for policy 0, policy_version 45702 (0.0010) [2023-10-08 05:43:19,240][00611] Updated weights for policy 0, policy_version 45712 (0.0009) [2023-10-08 05:43:19,612][00611] Updated weights for policy 0, policy_version 45722 (0.0010) [2023-10-08 05:43:20,774][00612] Updated weights for policy 1, policy_version 45960 (0.0008) [2023-10-08 05:43:21,140][00612] Updated weights for policy 1, policy_version 45970 (0.0009) [2023-10-08 05:43:21,516][00612] Updated weights for policy 1, policy_version 45980 (0.0010) [2023-10-08 05:43:23,239][00611] Updated weights for policy 0, policy_version 45732 (0.0009) [2023-10-08 05:43:23,609][00611] Updated weights for policy 0, policy_version 45742 (0.0008) [2023-10-08 05:43:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 93913088. Throughput: 0: 1836.9, 1: 1834.0. Samples: 23484804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:43:23,755][130385] Avg episode reward: [(0, '57.570'), (1, '69.010')] [2023-10-08 05:43:23,983][00611] Updated weights for policy 0, policy_version 45752 (0.0007) [2023-10-08 05:43:25,080][00612] Updated weights for policy 1, policy_version 45990 (0.0008) [2023-10-08 05:43:25,453][00612] Updated weights for policy 1, policy_version 46000 (0.0007) [2023-10-08 05:43:25,836][00612] Updated weights for policy 1, policy_version 46010 (0.0009) [2023-10-08 05:43:27,687][00611] Updated weights for policy 0, policy_version 45762 (0.0009) [2023-10-08 05:43:28,063][00611] Updated weights for policy 0, policy_version 45772 (0.0009) [2023-10-08 05:43:28,441][00611] Updated weights for policy 0, policy_version 45782 (0.0007) [2023-10-08 05:43:28,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 93978624. Throughput: 0: 1838.0, 1: 1849.4. Samples: 23507276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:43:28,755][130385] Avg episode reward: [(0, '62.320'), (1, '71.640')] [2023-10-08 05:43:28,814][00611] Updated weights for policy 0, policy_version 45792 (0.0010) [2023-10-08 05:43:29,450][00612] Updated weights for policy 1, policy_version 46020 (0.0009) [2023-10-08 05:43:29,825][00612] Updated weights for policy 1, policy_version 46030 (0.0007) [2023-10-08 05:43:30,200][00612] Updated weights for policy 1, policy_version 46040 (0.0008) [2023-10-08 05:43:32,644][00611] Updated weights for policy 0, policy_version 45802 (0.0008) [2023-10-08 05:43:33,017][00611] Updated weights for policy 0, policy_version 45812 (0.0008) [2023-10-08 05:43:33,383][00611] Updated weights for policy 0, policy_version 45822 (0.0008) [2023-10-08 05:43:33,714][00612] Updated weights for policy 1, policy_version 46050 (0.0009) [2023-10-08 05:43:33,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94076928. Throughput: 0: 1834.5, 1: 1847.8. Samples: 23529330. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 05:43:33,754][130385] Avg episode reward: [(0, '61.940'), (1, '73.720')] [2023-10-08 05:43:34,082][00612] Updated weights for policy 1, policy_version 46060 (0.0009) [2023-10-08 05:43:34,460][00612] Updated weights for policy 1, policy_version 46070 (0.0007) [2023-10-08 05:43:34,820][00425] Saving new best policy, reward=73.720! [2023-10-08 05:43:34,824][00612] Updated weights for policy 1, policy_version 46080 (0.0007) [2023-10-08 05:43:37,053][00611] Updated weights for policy 0, policy_version 45832 (0.0010) [2023-10-08 05:43:37,438][00611] Updated weights for policy 0, policy_version 45842 (0.0010) [2023-10-08 05:43:37,809][00611] Updated weights for policy 0, policy_version 45852 (0.0007) [2023-10-08 05:43:38,346][00612] Updated weights for policy 1, policy_version 46090 (0.0011) [2023-10-08 05:43:38,709][00612] Updated weights for policy 1, policy_version 46100 (0.0011) [2023-10-08 05:43:38,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94142464. Throughput: 0: 1838.1, 1: 1854.0. Samples: 23540462. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 05:43:38,754][130385] Avg episode reward: [(0, '62.990'), (1, '72.790')] [2023-10-08 05:43:39,083][00612] Updated weights for policy 1, policy_version 46110 (0.0009) [2023-10-08 05:43:41,394][00611] Updated weights for policy 0, policy_version 45862 (0.0007) [2023-10-08 05:43:41,762][00611] Updated weights for policy 0, policy_version 45872 (0.0009) [2023-10-08 05:43:42,131][00611] Updated weights for policy 0, policy_version 45882 (0.0010) [2023-10-08 05:43:42,739][00612] Updated weights for policy 1, policy_version 46120 (0.0009) [2023-10-08 05:43:43,102][00612] Updated weights for policy 1, policy_version 46130 (0.0007) [2023-10-08 05:43:43,471][00612] Updated weights for policy 1, policy_version 46140 (0.0009) [2023-10-08 05:43:43,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 94240768. Throughput: 0: 1835.5, 1: 1855.6. Samples: 23562486. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 05:43:43,755][130385] Avg episode reward: [(0, '63.020'), (1, '73.480')] [2023-10-08 05:43:45,848][00611] Updated weights for policy 0, policy_version 45892 (0.0007) [2023-10-08 05:43:46,230][00611] Updated weights for policy 0, policy_version 45902 (0.0007) [2023-10-08 05:43:46,610][00611] Updated weights for policy 0, policy_version 45912 (0.0009) [2023-10-08 05:43:47,165][00612] Updated weights for policy 1, policy_version 46150 (0.0009) [2023-10-08 05:43:47,529][00612] Updated weights for policy 1, policy_version 46160 (0.0008) [2023-10-08 05:43:47,907][00612] Updated weights for policy 1, policy_version 46170 (0.0009) [2023-10-08 05:43:48,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 94306304. Throughput: 0: 1840.3, 1: 1827.4. Samples: 23583516. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 05:43:48,755][130385] Avg episode reward: [(0, '64.690'), (1, '73.070')] [2023-10-08 05:43:50,281][00611] Updated weights for policy 0, policy_version 45922 (0.0009) [2023-10-08 05:43:50,638][00611] Updated weights for policy 0, policy_version 45932 (0.0009) [2023-10-08 05:43:51,012][00611] Updated weights for policy 0, policy_version 45942 (0.0008) [2023-10-08 05:43:51,379][00611] Updated weights for policy 0, policy_version 45952 (0.0009) [2023-10-08 05:43:51,659][00612] Updated weights for policy 1, policy_version 46180 (0.0007) [2023-10-08 05:43:52,022][00612] Updated weights for policy 1, policy_version 46190 (0.0007) [2023-10-08 05:43:52,401][00612] Updated weights for policy 1, policy_version 46200 (0.0007) [2023-10-08 05:43:53,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 94371840. Throughput: 0: 1835.3, 1: 1857.2. Samples: 23595518. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 05:43:53,754][130385] Avg episode reward: [(0, '64.790'), (1, '67.940')] [2023-10-08 05:43:55,122][00611] Updated weights for policy 0, policy_version 45962 (0.0008) [2023-10-08 05:43:55,492][00611] Updated weights for policy 0, policy_version 45972 (0.0010) [2023-10-08 05:43:55,863][00611] Updated weights for policy 0, policy_version 45982 (0.0010) [2023-10-08 05:43:56,084][00612] Updated weights for policy 1, policy_version 46210 (0.0008) [2023-10-08 05:43:56,452][00612] Updated weights for policy 1, policy_version 46220 (0.0009) [2023-10-08 05:43:56,816][00612] Updated weights for policy 1, policy_version 46230 (0.0008) [2023-10-08 05:43:57,195][00612] Updated weights for policy 1, policy_version 46240 (0.0007) [2023-10-08 05:43:58,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 94437376. Throughput: 0: 1832.0, 1: 1840.3. Samples: 23616910. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 05:43:58,754][130385] Avg episode reward: [(0, '61.720'), (1, '64.880')] [2023-10-08 05:43:59,563][00611] Updated weights for policy 0, policy_version 45992 (0.0008) [2023-10-08 05:43:59,936][00611] Updated weights for policy 0, policy_version 46002 (0.0009) [2023-10-08 05:44:00,308][00611] Updated weights for policy 0, policy_version 46012 (0.0008) [2023-10-08 05:44:00,865][00612] Updated weights for policy 1, policy_version 46250 (0.0009) [2023-10-08 05:44:01,234][00612] Updated weights for policy 1, policy_version 46260 (0.0007) [2023-10-08 05:44:01,606][00612] Updated weights for policy 1, policy_version 46270 (0.0009) [2023-10-08 05:44:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 94502912. Throughput: 0: 1827.9, 1: 1850.9. Samples: 23639658. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 05:44:03,754][130385] Avg episode reward: [(0, '58.740'), (1, '63.030')] [2023-10-08 05:44:03,907][00611] Updated weights for policy 0, policy_version 46022 (0.0009) [2023-10-08 05:44:04,273][00611] Updated weights for policy 0, policy_version 46032 (0.0008) [2023-10-08 05:44:04,648][00611] Updated weights for policy 0, policy_version 46042 (0.0009) [2023-10-08 05:44:05,308][00612] Updated weights for policy 1, policy_version 46280 (0.0008) [2023-10-08 05:44:05,665][00612] Updated weights for policy 1, policy_version 46290 (0.0008) [2023-10-08 05:44:06,036][00612] Updated weights for policy 1, policy_version 46300 (0.0007) [2023-10-08 05:44:08,117][00611] Updated weights for policy 0, policy_version 46052 (0.0009) [2023-10-08 05:44:08,482][00611] Updated weights for policy 0, policy_version 46062 (0.0007) [2023-10-08 05:44:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 94568448. Throughput: 0: 1829.3, 1: 1837.7. Samples: 23649818. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 05:44:08,754][130385] Avg episode reward: [(0, '59.460'), (1, '63.060')] [2023-10-08 05:44:08,854][00611] Updated weights for policy 0, policy_version 46072 (0.0008) [2023-10-08 05:44:09,587][00612] Updated weights for policy 1, policy_version 46310 (0.0010) [2023-10-08 05:44:09,952][00612] Updated weights for policy 1, policy_version 46320 (0.0009) [2023-10-08 05:44:10,324][00612] Updated weights for policy 1, policy_version 46330 (0.0008) [2023-10-08 05:44:12,461][00611] Updated weights for policy 0, policy_version 46082 (0.0008) [2023-10-08 05:44:12,828][00611] Updated weights for policy 0, policy_version 46092 (0.0007) [2023-10-08 05:44:13,196][00611] Updated weights for policy 0, policy_version 46102 (0.0009) [2023-10-08 05:44:13,566][00611] Updated weights for policy 0, policy_version 46112 (0.0009) [2023-10-08 05:44:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94666752. Throughput: 0: 1831.7, 1: 1850.9. Samples: 23672992. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 05:44:13,754][130385] Avg episode reward: [(0, '57.970'), (1, '55.890')] [2023-10-08 05:44:13,876][00612] Updated weights for policy 1, policy_version 46340 (0.0010) [2023-10-08 05:44:14,239][00612] Updated weights for policy 1, policy_version 46350 (0.0009) [2023-10-08 05:44:14,603][00612] Updated weights for policy 1, policy_version 46360 (0.0007) [2023-10-08 05:44:17,224][00611] Updated weights for policy 0, policy_version 46122 (0.0007) [2023-10-08 05:44:17,600][00611] Updated weights for policy 0, policy_version 46132 (0.0009) [2023-10-08 05:44:17,976][00611] Updated weights for policy 0, policy_version 46142 (0.0009) [2023-10-08 05:44:18,087][00612] Updated weights for policy 1, policy_version 46370 (0.0010) [2023-10-08 05:44:18,448][00612] Updated weights for policy 1, policy_version 46380 (0.0008) [2023-10-08 05:44:18,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94732288. Throughput: 0: 1826.7, 1: 1851.4. Samples: 23694842. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 05:44:18,754][130385] Avg episode reward: [(0, '55.440'), (1, '55.510')] [2023-10-08 05:44:18,763][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000046144_47251456.pth... [2023-10-08 05:44:18,794][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000044416_45481984.pth [2023-10-08 05:44:18,813][00612] Updated weights for policy 1, policy_version 46390 (0.0009) [2023-10-08 05:44:19,182][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000046400_47513600.pth... [2023-10-08 05:44:19,185][00612] Updated weights for policy 1, policy_version 46400 (0.0010) [2023-10-08 05:44:19,210][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000044672_45744128.pth [2023-10-08 05:44:21,660][00611] Updated weights for policy 0, policy_version 46152 (0.0010) [2023-10-08 05:44:22,039][00611] Updated weights for policy 0, policy_version 46162 (0.0009) [2023-10-08 05:44:22,413][00611] Updated weights for policy 0, policy_version 46172 (0.0007) [2023-10-08 05:44:22,877][00612] Updated weights for policy 1, policy_version 46410 (0.0007) [2023-10-08 05:44:23,257][00612] Updated weights for policy 1, policy_version 46420 (0.0009) [2023-10-08 05:44:23,621][00612] Updated weights for policy 1, policy_version 46430 (0.0008) [2023-10-08 05:44:23,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 94830592. Throughput: 0: 1833.4, 1: 1853.9. Samples: 23706390. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 05:44:23,754][130385] Avg episode reward: [(0, '55.380'), (1, '53.130')] [2023-10-08 05:44:26,198][00611] Updated weights for policy 0, policy_version 46182 (0.0008) [2023-10-08 05:44:26,572][00611] Updated weights for policy 0, policy_version 46192 (0.0009) [2023-10-08 05:44:26,943][00611] Updated weights for policy 0, policy_version 46202 (0.0007) [2023-10-08 05:44:27,276][00612] Updated weights for policy 1, policy_version 46440 (0.0008) [2023-10-08 05:44:27,652][00612] Updated weights for policy 1, policy_version 46450 (0.0008) [2023-10-08 05:44:28,017][00612] Updated weights for policy 1, policy_version 46460 (0.0007) [2023-10-08 05:44:28,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 94896128. Throughput: 0: 1823.0, 1: 1838.7. Samples: 23727262. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 05:44:28,754][130385] Avg episode reward: [(0, '55.900'), (1, '55.510')] [2023-10-08 05:44:30,778][00611] Updated weights for policy 0, policy_version 46212 (0.0009) [2023-10-08 05:44:31,164][00611] Updated weights for policy 0, policy_version 46222 (0.0008) [2023-10-08 05:44:31,538][00611] Updated weights for policy 0, policy_version 46232 (0.0007) [2023-10-08 05:44:31,628][00612] Updated weights for policy 1, policy_version 46470 (0.0008) [2023-10-08 05:44:32,004][00612] Updated weights for policy 1, policy_version 46480 (0.0010) [2023-10-08 05:44:32,361][00612] Updated weights for policy 1, policy_version 46490 (0.0008) [2023-10-08 05:44:33,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 94961664. Throughput: 0: 1824.4, 1: 1849.4. Samples: 23748838. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 05:44:33,755][130385] Avg episode reward: [(0, '54.780'), (1, '54.810')] [2023-10-08 05:44:35,114][00611] Updated weights for policy 0, policy_version 46242 (0.0008) [2023-10-08 05:44:35,486][00611] Updated weights for policy 0, policy_version 46252 (0.0010) [2023-10-08 05:44:35,852][00611] Updated weights for policy 0, policy_version 46262 (0.0010) [2023-10-08 05:44:35,964][00612] Updated weights for policy 1, policy_version 46500 (0.0008) [2023-10-08 05:44:36,220][00611] Updated weights for policy 0, policy_version 46272 (0.0008) [2023-10-08 05:44:36,329][00612] Updated weights for policy 1, policy_version 46510 (0.0007) [2023-10-08 05:44:36,694][00612] Updated weights for policy 1, policy_version 46520 (0.0011) [2023-10-08 05:44:38,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 95027200. Throughput: 0: 1823.5, 1: 1833.1. Samples: 23760064. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 05:44:38,754][130385] Avg episode reward: [(0, '52.380'), (1, '55.640')] [2023-10-08 05:44:39,917][00611] Updated weights for policy 0, policy_version 46282 (0.0010) [2023-10-08 05:44:40,263][00612] Updated weights for policy 1, policy_version 46530 (0.0009) [2023-10-08 05:44:40,285][00611] Updated weights for policy 0, policy_version 46292 (0.0008) [2023-10-08 05:44:40,625][00612] Updated weights for policy 1, policy_version 46540 (0.0008) [2023-10-08 05:44:40,644][00611] Updated weights for policy 0, policy_version 46302 (0.0008) [2023-10-08 05:44:40,984][00612] Updated weights for policy 1, policy_version 46550 (0.0008) [2023-10-08 05:44:41,358][00612] Updated weights for policy 1, policy_version 46560 (0.0007) [2023-10-08 05:44:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 95092736. Throughput: 0: 1829.2, 1: 1844.0. Samples: 23782202. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 05:44:43,755][130385] Avg episode reward: [(0, '54.750'), (1, '55.590')] [2023-10-08 05:44:44,301][00611] Updated weights for policy 0, policy_version 46312 (0.0008) [2023-10-08 05:44:44,662][00611] Updated weights for policy 0, policy_version 46322 (0.0008) [2023-10-08 05:44:44,823][00612] Updated weights for policy 1, policy_version 46570 (0.0008) [2023-10-08 05:44:45,028][00611] Updated weights for policy 0, policy_version 46332 (0.0008) [2023-10-08 05:44:45,198][00612] Updated weights for policy 1, policy_version 46580 (0.0007) [2023-10-08 05:44:45,560][00612] Updated weights for policy 1, policy_version 46590 (0.0009) [2023-10-08 05:44:48,656][00611] Updated weights for policy 0, policy_version 46342 (0.0010) [2023-10-08 05:44:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 95158272. Throughput: 0: 1827.6, 1: 1858.1. Samples: 23805514. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 05:44:48,754][130385] Avg episode reward: [(0, '54.680'), (1, '55.750')] [2023-10-08 05:44:49,025][00611] Updated weights for policy 0, policy_version 46352 (0.0008) [2023-10-08 05:44:49,187][00612] Updated weights for policy 1, policy_version 46600 (0.0008) [2023-10-08 05:44:49,402][00611] Updated weights for policy 0, policy_version 46362 (0.0007) [2023-10-08 05:44:49,553][00612] Updated weights for policy 1, policy_version 46610 (0.0008) [2023-10-08 05:44:49,918][00612] Updated weights for policy 1, policy_version 46620 (0.0009) [2023-10-08 05:44:53,127][00611] Updated weights for policy 0, policy_version 46372 (0.0007) [2023-10-08 05:44:53,503][00611] Updated weights for policy 0, policy_version 46382 (0.0007) [2023-10-08 05:44:53,684][00612] Updated weights for policy 1, policy_version 46630 (0.0009) [2023-10-08 05:44:53,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 95223808. Throughput: 0: 1824.7, 1: 1856.2. Samples: 23815458. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 05:44:53,754][130385] Avg episode reward: [(0, '53.280'), (1, '56.570')] [2023-10-08 05:44:53,878][00611] Updated weights for policy 0, policy_version 46392 (0.0007) [2023-10-08 05:44:54,038][00612] Updated weights for policy 1, policy_version 46640 (0.0007) [2023-10-08 05:44:54,399][00612] Updated weights for policy 1, policy_version 46650 (0.0007) [2023-10-08 05:44:57,577][00611] Updated weights for policy 0, policy_version 46402 (0.0009) [2023-10-08 05:44:57,951][00611] Updated weights for policy 0, policy_version 46412 (0.0007) [2023-10-08 05:44:57,976][00612] Updated weights for policy 1, policy_version 46660 (0.0007) [2023-10-08 05:44:58,318][00611] Updated weights for policy 0, policy_version 46422 (0.0007) [2023-10-08 05:44:58,340][00612] Updated weights for policy 1, policy_version 46670 (0.0008) [2023-10-08 05:44:58,684][00611] Updated weights for policy 0, policy_version 46432 (0.0008) [2023-10-08 05:44:58,705][00612] Updated weights for policy 1, policy_version 46680 (0.0007) [2023-10-08 05:44:58,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 95322112. Throughput: 0: 1817.5, 1: 1856.6. Samples: 23838326. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 05:44:58,754][130385] Avg episode reward: [(0, '52.430'), (1, '54.670')] [2023-10-08 05:45:02,355][00612] Updated weights for policy 1, policy_version 46690 (0.0008) [2023-10-08 05:45:02,393][00611] Updated weights for policy 0, policy_version 46442 (0.0008) [2023-10-08 05:45:02,732][00612] Updated weights for policy 1, policy_version 46700 (0.0008) [2023-10-08 05:45:02,767][00611] Updated weights for policy 0, policy_version 46452 (0.0008) [2023-10-08 05:45:03,103][00612] Updated weights for policy 1, policy_version 46710 (0.0009) [2023-10-08 05:45:03,139][00611] Updated weights for policy 0, policy_version 46462 (0.0008) [2023-10-08 05:45:03,474][00612] Updated weights for policy 1, policy_version 46720 (0.0009) [2023-10-08 05:45:03,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 95420416. Throughput: 0: 1817.1, 1: 1828.8. Samples: 23858906. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 05:45:03,755][130385] Avg episode reward: [(0, '49.770'), (1, '57.650')] [2023-10-08 05:45:06,630][00611] Updated weights for policy 0, policy_version 46472 (0.0007) [2023-10-08 05:45:06,997][00611] Updated weights for policy 0, policy_version 46482 (0.0008) [2023-10-08 05:45:07,231][00612] Updated weights for policy 1, policy_version 46730 (0.0007) [2023-10-08 05:45:07,381][00611] Updated weights for policy 0, policy_version 46492 (0.0007) [2023-10-08 05:45:07,603][00612] Updated weights for policy 1, policy_version 46740 (0.0007) [2023-10-08 05:45:07,965][00612] Updated weights for policy 1, policy_version 46750 (0.0008) [2023-10-08 05:45:08,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 95485952. Throughput: 0: 1819.8, 1: 1850.0. Samples: 23871530. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 05:45:08,755][130385] Avg episode reward: [(0, '52.010'), (1, '58.250')] [2023-10-08 05:45:10,911][00611] Updated weights for policy 0, policy_version 46502 (0.0009) [2023-10-08 05:45:11,282][00611] Updated weights for policy 0, policy_version 46512 (0.0008) [2023-10-08 05:45:11,569][00612] Updated weights for policy 1, policy_version 46760 (0.0008) [2023-10-08 05:45:11,655][00611] Updated weights for policy 0, policy_version 46522 (0.0007) [2023-10-08 05:45:11,942][00612] Updated weights for policy 1, policy_version 46770 (0.0010) [2023-10-08 05:45:12,318][00612] Updated weights for policy 1, policy_version 46780 (0.0010) [2023-10-08 05:45:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 95551488. Throughput: 0: 1825.0, 1: 1835.5. Samples: 23891986. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 05:45:13,754][130385] Avg episode reward: [(0, '50.060'), (1, '60.130')] [2023-10-08 05:45:15,487][00611] Updated weights for policy 0, policy_version 46532 (0.0007) [2023-10-08 05:45:15,857][00611] Updated weights for policy 0, policy_version 46542 (0.0009) [2023-10-08 05:45:16,007][00612] Updated weights for policy 1, policy_version 46790 (0.0009) [2023-10-08 05:45:16,227][00611] Updated weights for policy 0, policy_version 46552 (0.0008) [2023-10-08 05:45:16,371][00612] Updated weights for policy 1, policy_version 46800 (0.0008) [2023-10-08 05:45:16,748][00612] Updated weights for policy 1, policy_version 46810 (0.0007) [2023-10-08 05:45:18,754][130385] Fps is (10 sec: 13106.6, 60 sec: 14745.5, 300 sec: 14773.3). Total num frames: 95617024. Throughput: 0: 1829.1, 1: 1850.9. Samples: 23914436. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 05:45:18,755][130385] Avg episode reward: [(0, '54.080'), (1, '60.910')] [2023-10-08 05:45:19,967][00611] Updated weights for policy 0, policy_version 46562 (0.0009) [2023-10-08 05:45:20,296][00612] Updated weights for policy 1, policy_version 46820 (0.0009) [2023-10-08 05:45:20,377][00611] Updated weights for policy 0, policy_version 46572 (0.0009) [2023-10-08 05:45:20,662][00612] Updated weights for policy 1, policy_version 46830 (0.0009) [2023-10-08 05:45:20,754][00611] Updated weights for policy 0, policy_version 46582 (0.0007) [2023-10-08 05:45:21,028][00612] Updated weights for policy 1, policy_version 46840 (0.0007) [2023-10-08 05:45:21,116][00611] Updated weights for policy 0, policy_version 46592 (0.0008) [2023-10-08 05:45:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 95682560. Throughput: 0: 1824.1, 1: 1836.1. Samples: 23924774. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 05:45:23,754][130385] Avg episode reward: [(0, '54.850'), (1, '58.080')] [2023-10-08 05:45:24,687][00612] Updated weights for policy 1, policy_version 46850 (0.0007) [2023-10-08 05:45:24,747][00611] Updated weights for policy 0, policy_version 46602 (0.0007) [2023-10-08 05:45:25,062][00612] Updated weights for policy 1, policy_version 46860 (0.0009) [2023-10-08 05:45:25,120][00611] Updated weights for policy 0, policy_version 46612 (0.0007) [2023-10-08 05:45:25,430][00612] Updated weights for policy 1, policy_version 46870 (0.0008) [2023-10-08 05:45:25,492][00611] Updated weights for policy 0, policy_version 46622 (0.0009) [2023-10-08 05:45:25,799][00612] Updated weights for policy 1, policy_version 46880 (0.0010) [2023-10-08 05:45:28,754][130385] Fps is (10 sec: 13108.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 95748096. Throughput: 0: 1827.3, 1: 1850.7. Samples: 23947708. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 05:45:28,754][130385] Avg episode reward: [(0, '50.620'), (1, '61.520')] [2023-10-08 05:45:29,169][00611] Updated weights for policy 0, policy_version 46632 (0.0009) [2023-10-08 05:45:29,420][00612] Updated weights for policy 1, policy_version 46890 (0.0007) [2023-10-08 05:45:29,544][00611] Updated weights for policy 0, policy_version 46642 (0.0007) [2023-10-08 05:45:29,794][00612] Updated weights for policy 1, policy_version 46900 (0.0008) [2023-10-08 05:45:29,912][00611] Updated weights for policy 0, policy_version 46652 (0.0007) [2023-10-08 05:45:30,162][00612] Updated weights for policy 1, policy_version 46910 (0.0007) [2023-10-08 05:45:33,741][00611] Updated weights for policy 0, policy_version 46662 (0.0008) [2023-10-08 05:45:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 95813632. Throughput: 0: 1824.1, 1: 1839.6. Samples: 23970380. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 05:45:33,754][130385] Avg episode reward: [(0, '51.360'), (1, '62.280')] [2023-10-08 05:45:33,911][00612] Updated weights for policy 1, policy_version 46920 (0.0007) [2023-10-08 05:45:34,111][00611] Updated weights for policy 0, policy_version 46672 (0.0009) [2023-10-08 05:45:34,277][00612] Updated weights for policy 1, policy_version 46930 (0.0007) [2023-10-08 05:45:34,483][00611] Updated weights for policy 0, policy_version 46682 (0.0007) [2023-10-08 05:45:34,641][00612] Updated weights for policy 1, policy_version 46940 (0.0007) [2023-10-08 05:45:38,146][00611] Updated weights for policy 0, policy_version 46692 (0.0008) [2023-10-08 05:45:38,284][00612] Updated weights for policy 1, policy_version 46950 (0.0008) [2023-10-08 05:45:38,517][00611] Updated weights for policy 0, policy_version 46702 (0.0007) [2023-10-08 05:45:38,651][00612] Updated weights for policy 1, policy_version 46960 (0.0007) [2023-10-08 05:45:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 95879168. Throughput: 0: 1822.7, 1: 1834.6. Samples: 23980038. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-08 05:45:38,754][130385] Avg episode reward: [(0, '53.270'), (1, '62.850')] [2023-10-08 05:45:38,885][00611] Updated weights for policy 0, policy_version 46712 (0.0008) [2023-10-08 05:45:39,019][00612] Updated weights for policy 1, policy_version 46970 (0.0008) [2023-10-08 05:45:42,636][00611] Updated weights for policy 0, policy_version 46722 (0.0008) [2023-10-08 05:45:42,691][00612] Updated weights for policy 1, policy_version 46980 (0.0007) [2023-10-08 05:45:42,997][00611] Updated weights for policy 0, policy_version 46732 (0.0007) [2023-10-08 05:45:43,064][00612] Updated weights for policy 1, policy_version 46990 (0.0007) [2023-10-08 05:45:43,364][00611] Updated weights for policy 0, policy_version 46742 (0.0008) [2023-10-08 05:45:43,427][00612] Updated weights for policy 1, policy_version 47000 (0.0007) [2023-10-08 05:45:43,737][00611] Updated weights for policy 0, policy_version 46752 (0.0009) [2023-10-08 05:45:43,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 96010240. Throughput: 0: 1824.0, 1: 1832.8. Samples: 24002882. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-08 05:45:43,754][130385] Avg episode reward: [(0, '56.240'), (1, '63.880')] [2023-10-08 05:45:47,217][00612] Updated weights for policy 1, policy_version 47010 (0.0007) [2023-10-08 05:45:47,276][00611] Updated weights for policy 0, policy_version 46762 (0.0007) [2023-10-08 05:45:47,582][00612] Updated weights for policy 1, policy_version 47020 (0.0007) [2023-10-08 05:45:47,649][00611] Updated weights for policy 0, policy_version 46772 (0.0009) [2023-10-08 05:45:47,941][00612] Updated weights for policy 1, policy_version 47030 (0.0007) [2023-10-08 05:45:48,028][00611] Updated weights for policy 0, policy_version 46782 (0.0008) [2023-10-08 05:45:48,313][00612] Updated weights for policy 1, policy_version 47040 (0.0009) [2023-10-08 05:45:48,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 96075776. Throughput: 0: 1824.5, 1: 1822.0. Samples: 24022996. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-08 05:45:48,754][130385] Avg episode reward: [(0, '52.650'), (1, '62.280')] [2023-10-08 05:45:51,649][00611] Updated weights for policy 0, policy_version 46792 (0.0008) [2023-10-08 05:45:51,943][00612] Updated weights for policy 1, policy_version 47050 (0.0008) [2023-10-08 05:45:52,025][00611] Updated weights for policy 0, policy_version 46802 (0.0011) [2023-10-08 05:45:52,307][00612] Updated weights for policy 1, policy_version 47060 (0.0008) [2023-10-08 05:45:52,392][00611] Updated weights for policy 0, policy_version 46812 (0.0008) [2023-10-08 05:45:52,673][00612] Updated weights for policy 1, policy_version 47070 (0.0007) [2023-10-08 05:45:53,754][130385] Fps is (10 sec: 13106.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 96141312. Throughput: 0: 1824.1, 1: 1829.7. Samples: 24035950. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-08 05:45:53,755][130385] Avg episode reward: [(0, '54.480'), (1, '63.170')] [2023-10-08 05:45:56,190][00611] Updated weights for policy 0, policy_version 46822 (0.0008) [2023-10-08 05:45:56,269][00612] Updated weights for policy 1, policy_version 47080 (0.0008) [2023-10-08 05:45:56,560][00611] Updated weights for policy 0, policy_version 46832 (0.0009) [2023-10-08 05:45:56,639][00612] Updated weights for policy 1, policy_version 47090 (0.0008) [2023-10-08 05:45:56,931][00611] Updated weights for policy 0, policy_version 46842 (0.0007) [2023-10-08 05:45:57,001][00612] Updated weights for policy 1, policy_version 47100 (0.0008) [2023-10-08 05:45:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 96206848. Throughput: 0: 1815.9, 1: 1825.2. Samples: 24055834. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-08 05:45:58,754][130385] Avg episode reward: [(0, '56.280'), (1, '65.340')] [2023-10-08 05:46:00,522][00611] Updated weights for policy 0, policy_version 46852 (0.0009) [2023-10-08 05:46:00,664][00612] Updated weights for policy 1, policy_version 47110 (0.0007) [2023-10-08 05:46:00,892][00611] Updated weights for policy 0, policy_version 46862 (0.0008) [2023-10-08 05:46:01,023][00612] Updated weights for policy 1, policy_version 47120 (0.0007) [2023-10-08 05:46:01,258][00611] Updated weights for policy 0, policy_version 46872 (0.0007) [2023-10-08 05:46:01,395][00612] Updated weights for policy 1, policy_version 47130 (0.0008) [2023-10-08 05:46:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 96272384. Throughput: 0: 1821.6, 1: 1832.3. Samples: 24078858. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-08 05:46:03,755][130385] Avg episode reward: [(0, '58.050'), (1, '61.340')] [2023-10-08 05:46:04,946][00611] Updated weights for policy 0, policy_version 46882 (0.0008) [2023-10-08 05:46:05,072][00612] Updated weights for policy 1, policy_version 47140 (0.0008) [2023-10-08 05:46:05,331][00611] Updated weights for policy 0, policy_version 46892 (0.0008) [2023-10-08 05:46:05,439][00612] Updated weights for policy 1, policy_version 47150 (0.0008) [2023-10-08 05:46:05,707][00611] Updated weights for policy 0, policy_version 46902 (0.0007) [2023-10-08 05:46:05,803][00612] Updated weights for policy 1, policy_version 47160 (0.0009) [2023-10-08 05:46:06,077][00611] Updated weights for policy 0, policy_version 46912 (0.0008) [2023-10-08 05:46:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 96337920. Throughput: 0: 1821.2, 1: 1821.7. Samples: 24088704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:46:08,754][130385] Avg episode reward: [(0, '58.260'), (1, '60.350')] [2023-10-08 05:46:09,520][00612] Updated weights for policy 1, policy_version 47170 (0.0008) [2023-10-08 05:46:09,602][00611] Updated weights for policy 0, policy_version 46922 (0.0009) [2023-10-08 05:46:09,886][00612] Updated weights for policy 1, policy_version 47180 (0.0008) [2023-10-08 05:46:09,971][00611] Updated weights for policy 0, policy_version 46932 (0.0008) [2023-10-08 05:46:10,265][00612] Updated weights for policy 1, policy_version 47190 (0.0008) [2023-10-08 05:46:10,353][00611] Updated weights for policy 0, policy_version 46942 (0.0008) [2023-10-08 05:46:10,623][00612] Updated weights for policy 1, policy_version 47200 (0.0009) [2023-10-08 05:46:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 96403456. Throughput: 0: 1822.7, 1: 1823.1. Samples: 24111772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:46:13,755][130385] Avg episode reward: [(0, '57.910'), (1, '61.530')] [2023-10-08 05:46:14,125][00611] Updated weights for policy 0, policy_version 46952 (0.0007) [2023-10-08 05:46:14,171][00612] Updated weights for policy 1, policy_version 47210 (0.0008) [2023-10-08 05:46:14,488][00611] Updated weights for policy 0, policy_version 46962 (0.0007) [2023-10-08 05:46:14,547][00612] Updated weights for policy 1, policy_version 47220 (0.0007) [2023-10-08 05:46:14,867][00611] Updated weights for policy 0, policy_version 46972 (0.0007) [2023-10-08 05:46:14,915][00612] Updated weights for policy 1, policy_version 47230 (0.0007) [2023-10-08 05:46:18,486][00611] Updated weights for policy 0, policy_version 46982 (0.0007) [2023-10-08 05:46:18,494][00612] Updated weights for policy 1, policy_version 47240 (0.0008) [2023-10-08 05:46:18,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.6, 300 sec: 14662.3). Total num frames: 96468992. Throughput: 0: 1828.7, 1: 1835.6. Samples: 24135272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:46:18,754][130385] Avg episode reward: [(0, '60.060'), (1, '61.740')] [2023-10-08 05:46:18,856][00611] Updated weights for policy 0, policy_version 46992 (0.0009) [2023-10-08 05:46:18,871][00612] Updated weights for policy 1, policy_version 47250 (0.0008) [2023-10-08 05:46:19,230][00611] Updated weights for policy 0, policy_version 47002 (0.0007) [2023-10-08 05:46:19,238][00612] Updated weights for policy 1, policy_version 47260 (0.0008) [2023-10-08 05:46:19,384][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000047264_48398336.pth... [2023-10-08 05:46:19,416][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000045536_46628864.pth [2023-10-08 05:46:19,454][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000047008_48136192.pth... [2023-10-08 05:46:19,493][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000045280_46366720.pth [2023-10-08 05:46:22,845][00611] Updated weights for policy 0, policy_version 47012 (0.0007) [2023-10-08 05:46:22,929][00612] Updated weights for policy 1, policy_version 47270 (0.0007) [2023-10-08 05:46:23,215][00611] Updated weights for policy 0, policy_version 47022 (0.0009) [2023-10-08 05:46:23,295][00612] Updated weights for policy 1, policy_version 47280 (0.0007) [2023-10-08 05:46:23,592][00611] Updated weights for policy 0, policy_version 47032 (0.0008) [2023-10-08 05:46:23,664][00612] Updated weights for policy 1, policy_version 47290 (0.0009) [2023-10-08 05:46:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 96534528. Throughput: 0: 1832.3, 1: 1841.7. Samples: 24145370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:46:23,755][130385] Avg episode reward: [(0, '61.320'), (1, '59.690')] [2023-10-08 05:46:27,258][00611] Updated weights for policy 0, policy_version 47042 (0.0007) [2023-10-08 05:46:27,406][00612] Updated weights for policy 1, policy_version 47300 (0.0008) [2023-10-08 05:46:27,623][00611] Updated weights for policy 0, policy_version 47052 (0.0009) [2023-10-08 05:46:27,777][00612] Updated weights for policy 1, policy_version 47310 (0.0008) [2023-10-08 05:46:27,996][00611] Updated weights for policy 0, policy_version 47062 (0.0008) [2023-10-08 05:46:28,144][00612] Updated weights for policy 1, policy_version 47320 (0.0008) [2023-10-08 05:46:28,363][00611] Updated weights for policy 0, policy_version 47072 (0.0007) [2023-10-08 05:46:28,754][130385] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 96665600. Throughput: 0: 1833.6, 1: 1840.1. Samples: 24168202. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:46:28,755][130385] Avg episode reward: [(0, '61.500'), (1, '58.610')] [2023-10-08 05:46:31,619][00612] Updated weights for policy 1, policy_version 47330 (0.0008) [2023-10-08 05:46:31,988][00612] Updated weights for policy 1, policy_version 47340 (0.0008) [2023-10-08 05:46:32,075][00611] Updated weights for policy 0, policy_version 47082 (0.0008) [2023-10-08 05:46:32,350][00612] Updated weights for policy 1, policy_version 47350 (0.0007) [2023-10-08 05:46:32,445][00611] Updated weights for policy 0, policy_version 47092 (0.0007) [2023-10-08 05:46:32,719][00612] Updated weights for policy 1, policy_version 47360 (0.0007) [2023-10-08 05:46:32,807][00611] Updated weights for policy 0, policy_version 47102 (0.0008) [2023-10-08 05:46:33,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 96731136. Throughput: 0: 1827.1, 1: 1846.3. Samples: 24188304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:46:33,755][130385] Avg episode reward: [(0, '61.640'), (1, '58.610')] [2023-10-08 05:46:36,356][00612] Updated weights for policy 1, policy_version 47370 (0.0009) [2023-10-08 05:46:36,542][00611] Updated weights for policy 0, policy_version 47112 (0.0008) [2023-10-08 05:46:36,720][00612] Updated weights for policy 1, policy_version 47380 (0.0008) [2023-10-08 05:46:36,908][00611] Updated weights for policy 0, policy_version 47122 (0.0007) [2023-10-08 05:46:37,096][00612] Updated weights for policy 1, policy_version 47390 (0.0009) [2023-10-08 05:46:37,280][00611] Updated weights for policy 0, policy_version 47132 (0.0007) [2023-10-08 05:46:38,754][130385] Fps is (10 sec: 13107.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 96796672. Throughput: 0: 1825.4, 1: 1838.3. Samples: 24200818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:46:38,754][130385] Avg episode reward: [(0, '63.570'), (1, '59.320')] [2023-10-08 05:46:40,805][00612] Updated weights for policy 1, policy_version 47400 (0.0009) [2023-10-08 05:46:40,907][00611] Updated weights for policy 0, policy_version 47142 (0.0008) [2023-10-08 05:46:41,169][00612] Updated weights for policy 1, policy_version 47410 (0.0009) [2023-10-08 05:46:41,287][00611] Updated weights for policy 0, policy_version 47152 (0.0009) [2023-10-08 05:46:41,540][00612] Updated weights for policy 1, policy_version 47420 (0.0007) [2023-10-08 05:46:41,658][00611] Updated weights for policy 0, policy_version 47162 (0.0009) [2023-10-08 05:46:43,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 96862208. Throughput: 0: 1833.4, 1: 1839.9. Samples: 24221130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:46:43,754][130385] Avg episode reward: [(0, '64.760'), (1, '64.040')] [2023-10-08 05:46:45,265][00612] Updated weights for policy 1, policy_version 47430 (0.0010) [2023-10-08 05:46:45,289][00611] Updated weights for policy 0, policy_version 47172 (0.0010) [2023-10-08 05:46:45,643][00612] Updated weights for policy 1, policy_version 47440 (0.0008) [2023-10-08 05:46:45,656][00611] Updated weights for policy 0, policy_version 47182 (0.0008) [2023-10-08 05:46:46,017][00612] Updated weights for policy 1, policy_version 47450 (0.0007) [2023-10-08 05:46:46,031][00611] Updated weights for policy 0, policy_version 47192 (0.0009) [2023-10-08 05:46:48,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 96927744. Throughput: 0: 1828.9, 1: 1850.2. Samples: 24244418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:46:48,755][130385] Avg episode reward: [(0, '65.750'), (1, '66.390')] [2023-10-08 05:46:49,513][00612] Updated weights for policy 1, policy_version 47460 (0.0009) [2023-10-08 05:46:49,766][00611] Updated weights for policy 0, policy_version 47202 (0.0008) [2023-10-08 05:46:49,878][00612] Updated weights for policy 1, policy_version 47470 (0.0009) [2023-10-08 05:46:50,173][00611] Updated weights for policy 0, policy_version 47212 (0.0007) [2023-10-08 05:46:50,242][00612] Updated weights for policy 1, policy_version 47480 (0.0008) [2023-10-08 05:46:50,534][00611] Updated weights for policy 0, policy_version 47222 (0.0007) [2023-10-08 05:46:50,913][00611] Updated weights for policy 0, policy_version 47232 (0.0008) [2023-10-08 05:46:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 96993280. Throughput: 0: 1828.8, 1: 1852.1. Samples: 24254344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:46:53,755][130385] Avg episode reward: [(0, '65.790'), (1, '63.530')] [2023-10-08 05:46:53,928][00612] Updated weights for policy 1, policy_version 47490 (0.0008) [2023-10-08 05:46:54,300][00612] Updated weights for policy 1, policy_version 47500 (0.0007) [2023-10-08 05:46:54,542][00611] Updated weights for policy 0, policy_version 47242 (0.0007) [2023-10-08 05:46:54,655][00612] Updated weights for policy 1, policy_version 47510 (0.0007) [2023-10-08 05:46:54,913][00611] Updated weights for policy 0, policy_version 47252 (0.0007) [2023-10-08 05:46:55,021][00612] Updated weights for policy 1, policy_version 47520 (0.0007) [2023-10-08 05:46:55,296][00611] Updated weights for policy 0, policy_version 47262 (0.0008) [2023-10-08 05:46:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 97058816. Throughput: 0: 1823.5, 1: 1852.8. Samples: 24277204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:46:58,755][130385] Avg episode reward: [(0, '66.740'), (1, '62.210')] [2023-10-08 05:46:58,815][00612] Updated weights for policy 1, policy_version 47530 (0.0009) [2023-10-08 05:46:58,988][00611] Updated weights for policy 0, policy_version 47272 (0.0008) [2023-10-08 05:46:59,184][00612] Updated weights for policy 1, policy_version 47540 (0.0008) [2023-10-08 05:46:59,372][00611] Updated weights for policy 0, policy_version 47282 (0.0008) [2023-10-08 05:46:59,553][00612] Updated weights for policy 1, policy_version 47550 (0.0007) [2023-10-08 05:46:59,744][00611] Updated weights for policy 0, policy_version 47292 (0.0009) [2023-10-08 05:47:03,109][00612] Updated weights for policy 1, policy_version 47560 (0.0008) [2023-10-08 05:47:03,476][00612] Updated weights for policy 1, policy_version 47570 (0.0007) [2023-10-08 05:47:03,506][00611] Updated weights for policy 0, policy_version 47302 (0.0008) [2023-10-08 05:47:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 97124352. Throughput: 0: 1818.4, 1: 1831.6. Samples: 24299526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:47:03,755][130385] Avg episode reward: [(0, '66.170'), (1, '62.970')] [2023-10-08 05:47:03,843][00612] Updated weights for policy 1, policy_version 47580 (0.0010) [2023-10-08 05:47:03,872][00611] Updated weights for policy 0, policy_version 47312 (0.0008) [2023-10-08 05:47:04,244][00611] Updated weights for policy 0, policy_version 47322 (0.0008) [2023-10-08 05:47:07,396][00612] Updated weights for policy 1, policy_version 47590 (0.0008) [2023-10-08 05:47:07,767][00612] Updated weights for policy 1, policy_version 47600 (0.0008) [2023-10-08 05:47:07,788][00611] Updated weights for policy 0, policy_version 47332 (0.0008) [2023-10-08 05:47:08,129][00612] Updated weights for policy 1, policy_version 47610 (0.0007) [2023-10-08 05:47:08,168][00611] Updated weights for policy 0, policy_version 47342 (0.0007) [2023-10-08 05:47:08,539][00611] Updated weights for policy 0, policy_version 47352 (0.0008) [2023-10-08 05:47:08,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 97222656. Throughput: 0: 1816.3, 1: 1840.9. Samples: 24309944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:47:08,754][130385] Avg episode reward: [(0, '66.430'), (1, '65.400')] [2023-10-08 05:47:11,781][00612] Updated weights for policy 1, policy_version 47620 (0.0008) [2023-10-08 05:47:12,157][00612] Updated weights for policy 1, policy_version 47630 (0.0009) [2023-10-08 05:47:12,231][00611] Updated weights for policy 0, policy_version 47362 (0.0009) [2023-10-08 05:47:12,518][00612] Updated weights for policy 1, policy_version 47640 (0.0008) [2023-10-08 05:47:12,595][00611] Updated weights for policy 0, policy_version 47372 (0.0009) [2023-10-08 05:47:12,961][00611] Updated weights for policy 0, policy_version 47382 (0.0008) [2023-10-08 05:47:13,339][00611] Updated weights for policy 0, policy_version 47392 (0.0007) [2023-10-08 05:47:13,754][130385] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 97320960. Throughput: 0: 1815.7, 1: 1829.2. Samples: 24332220. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 05:47:13,754][130385] Avg episode reward: [(0, '67.650'), (1, '69.390')] [2023-10-08 05:47:13,755][00365] Saving new best policy, reward=67.650! [2023-10-08 05:47:16,184][00612] Updated weights for policy 1, policy_version 47650 (0.0009) [2023-10-08 05:47:16,564][00612] Updated weights for policy 1, policy_version 47660 (0.0007) [2023-10-08 05:47:16,925][00612] Updated weights for policy 1, policy_version 47670 (0.0007) [2023-10-08 05:47:17,049][00611] Updated weights for policy 0, policy_version 47402 (0.0008) [2023-10-08 05:47:17,299][00612] Updated weights for policy 1, policy_version 47680 (0.0007) [2023-10-08 05:47:17,434][00611] Updated weights for policy 0, policy_version 47412 (0.0010) [2023-10-08 05:47:17,811][00611] Updated weights for policy 0, policy_version 47422 (0.0008) [2023-10-08 05:47:18,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 97386496. Throughput: 0: 1815.2, 1: 1841.1. Samples: 24352836. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 05:47:18,755][130385] Avg episode reward: [(0, '65.330'), (1, '68.770')] [2023-10-08 05:47:20,946][00612] Updated weights for policy 1, policy_version 47690 (0.0010) [2023-10-08 05:47:21,321][00612] Updated weights for policy 1, policy_version 47700 (0.0008) [2023-10-08 05:47:21,491][00611] Updated weights for policy 0, policy_version 47432 (0.0008) [2023-10-08 05:47:21,683][00612] Updated weights for policy 1, policy_version 47710 (0.0009) [2023-10-08 05:47:21,863][00611] Updated weights for policy 0, policy_version 47442 (0.0010) [2023-10-08 05:47:22,240][00611] Updated weights for policy 0, policy_version 47452 (0.0007) [2023-10-08 05:47:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 97452032. Throughput: 0: 1821.2, 1: 1830.2. Samples: 24365132. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 05:47:23,755][130385] Avg episode reward: [(0, '63.580'), (1, '67.420')] [2023-10-08 05:47:25,391][00612] Updated weights for policy 1, policy_version 47720 (0.0009) [2023-10-08 05:47:25,759][00612] Updated weights for policy 1, policy_version 47730 (0.0007) [2023-10-08 05:47:25,864][00611] Updated weights for policy 0, policy_version 47462 (0.0008) [2023-10-08 05:47:26,127][00612] Updated weights for policy 1, policy_version 47740 (0.0008) [2023-10-08 05:47:26,231][00611] Updated weights for policy 0, policy_version 47472 (0.0007) [2023-10-08 05:47:26,603][00611] Updated weights for policy 0, policy_version 47482 (0.0009) [2023-10-08 05:47:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 97517568. Throughput: 0: 1816.8, 1: 1845.2. Samples: 24385924. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 05:47:28,755][130385] Avg episode reward: [(0, '63.370'), (1, '69.520')] [2023-10-08 05:47:29,606][00612] Updated weights for policy 1, policy_version 47750 (0.0009) [2023-10-08 05:47:29,980][00612] Updated weights for policy 1, policy_version 47760 (0.0009) [2023-10-08 05:47:30,060][00611] Updated weights for policy 0, policy_version 47492 (0.0008) [2023-10-08 05:47:30,346][00612] Updated weights for policy 1, policy_version 47770 (0.0008) [2023-10-08 05:47:30,435][00611] Updated weights for policy 0, policy_version 47502 (0.0008) [2023-10-08 05:47:30,802][00611] Updated weights for policy 0, policy_version 47512 (0.0009) [2023-10-08 05:47:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 97583104. Throughput: 0: 1830.7, 1: 1845.1. Samples: 24409826. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 05:47:33,755][130385] Avg episode reward: [(0, '68.000'), (1, '68.720')] [2023-10-08 05:47:33,765][00365] Saving new best policy, reward=68.000! [2023-10-08 05:47:34,042][00612] Updated weights for policy 1, policy_version 47780 (0.0008) [2023-10-08 05:47:34,431][00612] Updated weights for policy 1, policy_version 47790 (0.0007) [2023-10-08 05:47:34,481][00611] Updated weights for policy 0, policy_version 47522 (0.0007) [2023-10-08 05:47:34,795][00612] Updated weights for policy 1, policy_version 47800 (0.0007) [2023-10-08 05:47:34,876][00611] Updated weights for policy 0, policy_version 47532 (0.0009) [2023-10-08 05:47:35,242][00611] Updated weights for policy 0, policy_version 47542 (0.0008) [2023-10-08 05:47:35,616][00611] Updated weights for policy 0, policy_version 47552 (0.0009) [2023-10-08 05:47:38,420][00612] Updated weights for policy 1, policy_version 47810 (0.0008) [2023-10-08 05:47:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 97648640. Throughput: 0: 1830.7, 1: 1844.0. Samples: 24419706. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 05:47:38,755][130385] Avg episode reward: [(0, '68.540'), (1, '65.660')] [2023-10-08 05:47:38,786][00612] Updated weights for policy 1, policy_version 47820 (0.0009) [2023-10-08 05:47:39,149][00612] Updated weights for policy 1, policy_version 47830 (0.0007) [2023-10-08 05:47:39,195][00611] Updated weights for policy 0, policy_version 47562 (0.0007) [2023-10-08 05:47:39,517][00612] Updated weights for policy 1, policy_version 47840 (0.0008) [2023-10-08 05:47:39,574][00611] Updated weights for policy 0, policy_version 47572 (0.0007) [2023-10-08 05:47:39,940][00611] Updated weights for policy 0, policy_version 47582 (0.0007) [2023-10-08 05:47:40,014][00365] Saving new best policy, reward=68.540! [2023-10-08 05:47:43,007][00612] Updated weights for policy 1, policy_version 47850 (0.0008) [2023-10-08 05:47:43,370][00612] Updated weights for policy 1, policy_version 47860 (0.0008) [2023-10-08 05:47:43,655][00611] Updated weights for policy 0, policy_version 47592 (0.0009) [2023-10-08 05:47:43,736][00612] Updated weights for policy 1, policy_version 47870 (0.0008) [2023-10-08 05:47:43,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 97714176. Throughput: 0: 1836.2, 1: 1847.0. Samples: 24442948. Policy #0 lag: (min: 25.0, avg: 40.3, max: 57.0) [2023-10-08 05:47:43,754][130385] Avg episode reward: [(0, '66.430'), (1, '64.160')] [2023-10-08 05:47:44,019][00611] Updated weights for policy 0, policy_version 47602 (0.0007) [2023-10-08 05:47:44,402][00611] Updated weights for policy 0, policy_version 47612 (0.0008) [2023-10-08 05:47:47,588][00612] Updated weights for policy 1, policy_version 47880 (0.0011) [2023-10-08 05:47:47,963][00612] Updated weights for policy 1, policy_version 47890 (0.0008) [2023-10-08 05:47:48,200][00611] Updated weights for policy 0, policy_version 47622 (0.0007) [2023-10-08 05:47:48,319][00612] Updated weights for policy 1, policy_version 47900 (0.0009) [2023-10-08 05:47:48,569][00611] Updated weights for policy 0, policy_version 47632 (0.0008) [2023-10-08 05:47:48,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 97812480. Throughput: 0: 1836.7, 1: 1829.6. Samples: 24464512. Policy #0 lag: (min: 25.0, avg: 40.3, max: 57.0) [2023-10-08 05:47:48,755][130385] Avg episode reward: [(0, '62.630'), (1, '65.990')] [2023-10-08 05:47:48,939][00611] Updated weights for policy 0, policy_version 47642 (0.0009) [2023-10-08 05:47:52,065][00612] Updated weights for policy 1, policy_version 47910 (0.0007) [2023-10-08 05:47:52,437][00612] Updated weights for policy 1, policy_version 47920 (0.0009) [2023-10-08 05:47:52,710][00611] Updated weights for policy 0, policy_version 47652 (0.0008) [2023-10-08 05:47:52,817][00612] Updated weights for policy 1, policy_version 47930 (0.0008) [2023-10-08 05:47:53,072][00611] Updated weights for policy 0, policy_version 47662 (0.0007) [2023-10-08 05:47:53,441][00611] Updated weights for policy 0, policy_version 47672 (0.0009) [2023-10-08 05:47:53,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 97910784. Throughput: 0: 1841.5, 1: 1847.1. Samples: 24475930. Policy #0 lag: (min: 25.0, avg: 40.3, max: 57.0) [2023-10-08 05:47:53,754][130385] Avg episode reward: [(0, '62.100'), (1, '67.890')] [2023-10-08 05:47:56,213][00612] Updated weights for policy 1, policy_version 47940 (0.0008) [2023-10-08 05:47:56,583][00612] Updated weights for policy 1, policy_version 47950 (0.0007) [2023-10-08 05:47:56,958][00612] Updated weights for policy 1, policy_version 47960 (0.0007) [2023-10-08 05:47:57,118][00611] Updated weights for policy 0, policy_version 47682 (0.0008) [2023-10-08 05:47:57,491][00611] Updated weights for policy 0, policy_version 47692 (0.0008) [2023-10-08 05:47:57,866][00611] Updated weights for policy 0, policy_version 47702 (0.0008) [2023-10-08 05:47:58,238][00611] Updated weights for policy 0, policy_version 47712 (0.0009) [2023-10-08 05:47:58,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 97976320. Throughput: 0: 1834.0, 1: 1841.5. Samples: 24497616. Policy #0 lag: (min: 25.0, avg: 40.3, max: 57.0) [2023-10-08 05:47:58,754][130385] Avg episode reward: [(0, '62.950'), (1, '68.660')] [2023-10-08 05:48:00,613][00612] Updated weights for policy 1, policy_version 47970 (0.0007) [2023-10-08 05:48:00,984][00612] Updated weights for policy 1, policy_version 47980 (0.0008) [2023-10-08 05:48:01,358][00612] Updated weights for policy 1, policy_version 47990 (0.0007) [2023-10-08 05:48:01,725][00612] Updated weights for policy 1, policy_version 48000 (0.0008) [2023-10-08 05:48:01,887][00611] Updated weights for policy 0, policy_version 47722 (0.0007) [2023-10-08 05:48:02,253][00611] Updated weights for policy 0, policy_version 47732 (0.0009) [2023-10-08 05:48:02,633][00611] Updated weights for policy 0, policy_version 47742 (0.0009) [2023-10-08 05:48:03,754][130385] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 98041856. Throughput: 0: 1836.8, 1: 1858.8. Samples: 24519138. Policy #0 lag: (min: 25.0, avg: 40.3, max: 57.0) [2023-10-08 05:48:03,755][130385] Avg episode reward: [(0, '59.650'), (1, '66.120')] [2023-10-08 05:48:05,268][00612] Updated weights for policy 1, policy_version 48010 (0.0008) [2023-10-08 05:48:05,629][00612] Updated weights for policy 1, policy_version 48020 (0.0008) [2023-10-08 05:48:05,993][00612] Updated weights for policy 1, policy_version 48030 (0.0007) [2023-10-08 05:48:06,241][00611] Updated weights for policy 0, policy_version 47752 (0.0009) [2023-10-08 05:48:06,612][00611] Updated weights for policy 0, policy_version 47762 (0.0009) [2023-10-08 05:48:06,992][00611] Updated weights for policy 0, policy_version 47772 (0.0008) [2023-10-08 05:48:08,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 98107392. Throughput: 0: 1827.6, 1: 1842.6. Samples: 24530292. Policy #0 lag: (min: 25.0, avg: 40.3, max: 57.0) [2023-10-08 05:48:08,755][130385] Avg episode reward: [(0, '59.080'), (1, '64.740')] [2023-10-08 05:48:09,423][00612] Updated weights for policy 1, policy_version 48040 (0.0008) [2023-10-08 05:48:09,797][00612] Updated weights for policy 1, policy_version 48050 (0.0009) [2023-10-08 05:48:10,173][00612] Updated weights for policy 1, policy_version 48060 (0.0007) [2023-10-08 05:48:10,515][00611] Updated weights for policy 0, policy_version 47782 (0.0010) [2023-10-08 05:48:10,890][00611] Updated weights for policy 0, policy_version 47792 (0.0008) [2023-10-08 05:48:11,263][00611] Updated weights for policy 0, policy_version 47802 (0.0009) [2023-10-08 05:48:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 98172928. Throughput: 0: 1832.0, 1: 1866.8. Samples: 24552370. Policy #0 lag: (min: 24.0, avg: 38.7, max: 56.0) [2023-10-08 05:48:13,755][130385] Avg episode reward: [(0, '56.520'), (1, '65.840')] [2023-10-08 05:48:13,888][00612] Updated weights for policy 1, policy_version 48070 (0.0007) [2023-10-08 05:48:14,254][00612] Updated weights for policy 1, policy_version 48080 (0.0007) [2023-10-08 05:48:14,617][00612] Updated weights for policy 1, policy_version 48090 (0.0009) [2023-10-08 05:48:14,721][00611] Updated weights for policy 0, policy_version 47812 (0.0008) [2023-10-08 05:48:15,098][00611] Updated weights for policy 0, policy_version 47822 (0.0007) [2023-10-08 05:48:15,460][00611] Updated weights for policy 0, policy_version 47832 (0.0009) [2023-10-08 05:48:18,081][00612] Updated weights for policy 1, policy_version 48100 (0.0009) [2023-10-08 05:48:18,452][00612] Updated weights for policy 1, policy_version 48110 (0.0010) [2023-10-08 05:48:18,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 98238464. Throughput: 0: 1826.4, 1: 1857.6. Samples: 24575608. Policy #0 lag: (min: 24.0, avg: 38.7, max: 56.0) [2023-10-08 05:48:18,755][130385] Avg episode reward: [(0, '58.680'), (1, '62.210')] [2023-10-08 05:48:18,763][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000047840_48988160.pth... [2023-10-08 05:48:18,805][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000046144_47251456.pth [2023-10-08 05:48:18,829][00612] Updated weights for policy 1, policy_version 48120 (0.0007) [2023-10-08 05:48:19,099][00611] Updated weights for policy 0, policy_version 47842 (0.0008) [2023-10-08 05:48:19,116][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000048128_49283072.pth... [2023-10-08 05:48:19,146][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000046400_47513600.pth [2023-10-08 05:48:19,461][00611] Updated weights for policy 0, policy_version 47852 (0.0008) [2023-10-08 05:48:19,840][00611] Updated weights for policy 0, policy_version 47862 (0.0007) [2023-10-08 05:48:20,204][00611] Updated weights for policy 0, policy_version 47872 (0.0008) [2023-10-08 05:48:22,572][00612] Updated weights for policy 1, policy_version 48130 (0.0009) [2023-10-08 05:48:22,988][00612] Updated weights for policy 1, policy_version 48140 (0.0008) [2023-10-08 05:48:23,348][00612] Updated weights for policy 1, policy_version 48150 (0.0009) [2023-10-08 05:48:23,710][00612] Updated weights for policy 1, policy_version 48160 (0.0007) [2023-10-08 05:48:23,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 98336768. Throughput: 0: 1826.4, 1: 1866.8. Samples: 24585898. Policy #0 lag: (min: 24.0, avg: 38.7, max: 56.0) [2023-10-08 05:48:23,754][130385] Avg episode reward: [(0, '54.600'), (1, '65.880')] [2023-10-08 05:48:23,867][00611] Updated weights for policy 0, policy_version 47882 (0.0008) [2023-10-08 05:48:24,243][00611] Updated weights for policy 0, policy_version 47892 (0.0010) [2023-10-08 05:48:24,614][00611] Updated weights for policy 0, policy_version 47902 (0.0009) [2023-10-08 05:48:27,369][00612] Updated weights for policy 1, policy_version 48170 (0.0010) [2023-10-08 05:48:27,746][00612] Updated weights for policy 1, policy_version 48180 (0.0009) [2023-10-08 05:48:28,109][00612] Updated weights for policy 1, policy_version 48190 (0.0008) [2023-10-08 05:48:28,220][00611] Updated weights for policy 0, policy_version 47912 (0.0007) [2023-10-08 05:48:28,594][00611] Updated weights for policy 0, policy_version 47922 (0.0010) [2023-10-08 05:48:28,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 98402304. Throughput: 0: 1827.4, 1: 1853.6. Samples: 24608596. Policy #0 lag: (min: 24.0, avg: 38.7, max: 56.0) [2023-10-08 05:48:28,754][130385] Avg episode reward: [(0, '52.590'), (1, '63.210')] [2023-10-08 05:48:28,970][00611] Updated weights for policy 0, policy_version 47932 (0.0009) [2023-10-08 05:48:31,849][00612] Updated weights for policy 1, policy_version 48200 (0.0010) [2023-10-08 05:48:32,224][00612] Updated weights for policy 1, policy_version 48210 (0.0011) [2023-10-08 05:48:32,590][00612] Updated weights for policy 1, policy_version 48220 (0.0009) [2023-10-08 05:48:32,753][00611] Updated weights for policy 0, policy_version 47942 (0.0009) [2023-10-08 05:48:33,127][00611] Updated weights for policy 0, policy_version 47952 (0.0007) [2023-10-08 05:48:33,502][00611] Updated weights for policy 0, policy_version 47962 (0.0009) [2023-10-08 05:48:33,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 98500608. Throughput: 0: 1813.2, 1: 1851.2. Samples: 24629408. Policy #0 lag: (min: 24.0, avg: 38.7, max: 56.0) [2023-10-08 05:48:33,755][130385] Avg episode reward: [(0, '52.720'), (1, '66.780')] [2023-10-08 05:48:36,229][00612] Updated weights for policy 1, policy_version 48230 (0.0010) [2023-10-08 05:48:36,604][00612] Updated weights for policy 1, policy_version 48240 (0.0007) [2023-10-08 05:48:36,965][00612] Updated weights for policy 1, policy_version 48250 (0.0007) [2023-10-08 05:48:37,348][00611] Updated weights for policy 0, policy_version 47972 (0.0008) [2023-10-08 05:48:37,704][00611] Updated weights for policy 0, policy_version 47982 (0.0007) [2023-10-08 05:48:38,081][00611] Updated weights for policy 0, policy_version 47992 (0.0009) [2023-10-08 05:48:38,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 98566144. Throughput: 0: 1824.2, 1: 1849.6. Samples: 24641252. Policy #0 lag: (min: 24.0, avg: 38.7, max: 56.0) [2023-10-08 05:48:38,754][130385] Avg episode reward: [(0, '54.050'), (1, '65.890')] [2023-10-08 05:48:40,691][00612] Updated weights for policy 1, policy_version 48260 (0.0009) [2023-10-08 05:48:41,055][00612] Updated weights for policy 1, policy_version 48270 (0.0009) [2023-10-08 05:48:41,428][00612] Updated weights for policy 1, policy_version 48280 (0.0008) [2023-10-08 05:48:41,716][00611] Updated weights for policy 0, policy_version 48002 (0.0008) [2023-10-08 05:48:42,083][00611] Updated weights for policy 0, policy_version 48012 (0.0010) [2023-10-08 05:48:42,462][00611] Updated weights for policy 0, policy_version 48022 (0.0009) [2023-10-08 05:48:42,836][00611] Updated weights for policy 0, policy_version 48032 (0.0007) [2023-10-08 05:48:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 98631680. Throughput: 0: 1824.0, 1: 1842.5. Samples: 24662608. Policy #0 lag: (min: 11.0, avg: 25.2, max: 43.0) [2023-10-08 05:48:43,755][130385] Avg episode reward: [(0, '54.100'), (1, '71.890')] [2023-10-08 05:48:45,041][00612] Updated weights for policy 1, policy_version 48290 (0.0009) [2023-10-08 05:48:45,414][00612] Updated weights for policy 1, policy_version 48300 (0.0008) [2023-10-08 05:48:45,778][00612] Updated weights for policy 1, policy_version 48310 (0.0007) [2023-10-08 05:48:46,140][00612] Updated weights for policy 1, policy_version 48320 (0.0007) [2023-10-08 05:48:46,650][00611] Updated weights for policy 0, policy_version 48042 (0.0008) [2023-10-08 05:48:47,021][00611] Updated weights for policy 0, policy_version 48052 (0.0009) [2023-10-08 05:48:47,404][00611] Updated weights for policy 0, policy_version 48062 (0.0008) [2023-10-08 05:48:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 98697216. Throughput: 0: 1834.1, 1: 1846.8. Samples: 24684778. Policy #0 lag: (min: 11.0, avg: 25.2, max: 43.0) [2023-10-08 05:48:48,754][130385] Avg episode reward: [(0, '58.070'), (1, '73.440')] [2023-10-08 05:48:49,845][00612] Updated weights for policy 1, policy_version 48330 (0.0008) [2023-10-08 05:48:50,218][00612] Updated weights for policy 1, policy_version 48340 (0.0007) [2023-10-08 05:48:50,585][00612] Updated weights for policy 1, policy_version 48350 (0.0007) [2023-10-08 05:48:50,989][00611] Updated weights for policy 0, policy_version 48072 (0.0011) [2023-10-08 05:48:51,365][00611] Updated weights for policy 0, policy_version 48082 (0.0008) [2023-10-08 05:48:51,736][00611] Updated weights for policy 0, policy_version 48092 (0.0009) [2023-10-08 05:48:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 98762752. Throughput: 0: 1832.1, 1: 1846.3. Samples: 24695816. Policy #0 lag: (min: 11.0, avg: 25.2, max: 43.0) [2023-10-08 05:48:53,754][130385] Avg episode reward: [(0, '57.730'), (1, '70.870')] [2023-10-08 05:48:54,198][00612] Updated weights for policy 1, policy_version 48360 (0.0008) [2023-10-08 05:48:54,566][00612] Updated weights for policy 1, policy_version 48370 (0.0007) [2023-10-08 05:48:54,939][00612] Updated weights for policy 1, policy_version 48380 (0.0007) [2023-10-08 05:48:55,275][00611] Updated weights for policy 0, policy_version 48102 (0.0010) [2023-10-08 05:48:55,651][00611] Updated weights for policy 0, policy_version 48112 (0.0010) [2023-10-08 05:48:56,026][00611] Updated weights for policy 0, policy_version 48122 (0.0010) [2023-10-08 05:48:58,692][00612] Updated weights for policy 1, policy_version 48390 (0.0008) [2023-10-08 05:48:58,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 98828288. Throughput: 0: 1844.3, 1: 1839.5. Samples: 24718140. Policy #0 lag: (min: 11.0, avg: 25.2, max: 43.0) [2023-10-08 05:48:58,755][130385] Avg episode reward: [(0, '59.630'), (1, '69.700')] [2023-10-08 05:48:59,056][00612] Updated weights for policy 1, policy_version 48400 (0.0007) [2023-10-08 05:48:59,425][00612] Updated weights for policy 1, policy_version 48410 (0.0007) [2023-10-08 05:48:59,637][00611] Updated weights for policy 0, policy_version 48132 (0.0009) [2023-10-08 05:49:00,009][00611] Updated weights for policy 0, policy_version 48142 (0.0008) [2023-10-08 05:49:00,375][00611] Updated weights for policy 0, policy_version 48152 (0.0009) [2023-10-08 05:49:02,913][00612] Updated weights for policy 1, policy_version 48420 (0.0008) [2023-10-08 05:49:03,276][00612] Updated weights for policy 1, policy_version 48430 (0.0009) [2023-10-08 05:49:03,653][00612] Updated weights for policy 1, policy_version 48440 (0.0008) [2023-10-08 05:49:03,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 98893824. Throughput: 0: 1835.7, 1: 1833.5. Samples: 24740720. Policy #0 lag: (min: 11.0, avg: 25.2, max: 43.0) [2023-10-08 05:49:03,755][130385] Avg episode reward: [(0, '61.240'), (1, '69.590')] [2023-10-08 05:49:04,078][00611] Updated weights for policy 0, policy_version 48162 (0.0011) [2023-10-08 05:49:04,455][00611] Updated weights for policy 0, policy_version 48172 (0.0010) [2023-10-08 05:49:04,829][00611] Updated weights for policy 0, policy_version 48182 (0.0008) [2023-10-08 05:49:05,208][00611] Updated weights for policy 0, policy_version 48192 (0.0008) [2023-10-08 05:49:07,248][00612] Updated weights for policy 1, policy_version 48450 (0.0009) [2023-10-08 05:49:07,639][00612] Updated weights for policy 1, policy_version 48460 (0.0008) [2023-10-08 05:49:08,005][00612] Updated weights for policy 1, policy_version 48470 (0.0009) [2023-10-08 05:49:08,374][00612] Updated weights for policy 1, policy_version 48480 (0.0008) [2023-10-08 05:49:08,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 98992128. Throughput: 0: 1837.5, 1: 1839.1. Samples: 24751346. Policy #0 lag: (min: 11.0, avg: 25.2, max: 43.0) [2023-10-08 05:49:08,755][130385] Avg episode reward: [(0, '62.610'), (1, '71.740')] [2023-10-08 05:49:08,922][00611] Updated weights for policy 0, policy_version 48202 (0.0008) [2023-10-08 05:49:09,289][00611] Updated weights for policy 0, policy_version 48212 (0.0008) [2023-10-08 05:49:09,670][00611] Updated weights for policy 0, policy_version 48222 (0.0009) [2023-10-08 05:49:11,908][00612] Updated weights for policy 1, policy_version 48490 (0.0008) [2023-10-08 05:49:12,287][00612] Updated weights for policy 1, policy_version 48500 (0.0009) [2023-10-08 05:49:12,655][00612] Updated weights for policy 1, policy_version 48510 (0.0010) [2023-10-08 05:49:13,292][00611] Updated weights for policy 0, policy_version 48232 (0.0011) [2023-10-08 05:49:13,658][00611] Updated weights for policy 0, policy_version 48242 (0.0009) [2023-10-08 05:49:13,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 99057664. Throughput: 0: 1829.7, 1: 1831.5. Samples: 24773350. Policy #0 lag: (min: 11.0, avg: 25.2, max: 43.0) [2023-10-08 05:49:13,754][130385] Avg episode reward: [(0, '66.300'), (1, '70.680')] [2023-10-08 05:49:14,035][00611] Updated weights for policy 0, policy_version 48252 (0.0009) [2023-10-08 05:49:16,445][00612] Updated weights for policy 1, policy_version 48520 (0.0008) [2023-10-08 05:49:16,811][00612] Updated weights for policy 1, policy_version 48530 (0.0008) [2023-10-08 05:49:17,179][00612] Updated weights for policy 1, policy_version 48540 (0.0008) [2023-10-08 05:49:17,826][00611] Updated weights for policy 0, policy_version 48262 (0.0009) [2023-10-08 05:49:18,192][00611] Updated weights for policy 0, policy_version 48272 (0.0008) [2023-10-08 05:49:18,575][00611] Updated weights for policy 0, policy_version 48282 (0.0009) [2023-10-08 05:49:18,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 99123200. Throughput: 0: 1828.8, 1: 1844.7. Samples: 24794714. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-08 05:49:18,754][130385] Avg episode reward: [(0, '67.110'), (1, '70.270')] [2023-10-08 05:49:20,909][00612] Updated weights for policy 1, policy_version 48550 (0.0010) [2023-10-08 05:49:21,286][00612] Updated weights for policy 1, policy_version 48560 (0.0008) [2023-10-08 05:49:21,660][00612] Updated weights for policy 1, policy_version 48570 (0.0008) [2023-10-08 05:49:22,243][00611] Updated weights for policy 0, policy_version 48292 (0.0008) [2023-10-08 05:49:22,620][00611] Updated weights for policy 0, policy_version 48302 (0.0008) [2023-10-08 05:49:23,006][00611] Updated weights for policy 0, policy_version 48312 (0.0011) [2023-10-08 05:49:23,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 99221504. Throughput: 0: 1828.8, 1: 1833.6. Samples: 24806060. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-08 05:49:23,754][130385] Avg episode reward: [(0, '67.290'), (1, '73.550')] [2023-10-08 05:49:25,225][00612] Updated weights for policy 1, policy_version 48580 (0.0008) [2023-10-08 05:49:25,603][00612] Updated weights for policy 1, policy_version 48590 (0.0011) [2023-10-08 05:49:25,966][00612] Updated weights for policy 1, policy_version 48600 (0.0009) [2023-10-08 05:49:26,690][00611] Updated weights for policy 0, policy_version 48322 (0.0009) [2023-10-08 05:49:27,061][00611] Updated weights for policy 0, policy_version 48332 (0.0008) [2023-10-08 05:49:27,429][00611] Updated weights for policy 0, policy_version 48342 (0.0008) [2023-10-08 05:49:27,794][00611] Updated weights for policy 0, policy_version 48352 (0.0008) [2023-10-08 05:49:28,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 99287040. Throughput: 0: 1819.2, 1: 1842.4. Samples: 24827380. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-08 05:49:28,755][130385] Avg episode reward: [(0, '67.510'), (1, '73.230')] [2023-10-08 05:49:29,579][00612] Updated weights for policy 1, policy_version 48610 (0.0008) [2023-10-08 05:49:29,944][00612] Updated weights for policy 1, policy_version 48620 (0.0007) [2023-10-08 05:49:30,322][00612] Updated weights for policy 1, policy_version 48630 (0.0010) [2023-10-08 05:49:30,687][00612] Updated weights for policy 1, policy_version 48640 (0.0008) [2023-10-08 05:49:31,383][00611] Updated weights for policy 0, policy_version 48362 (0.0008) [2023-10-08 05:49:31,758][00611] Updated weights for policy 0, policy_version 48372 (0.0009) [2023-10-08 05:49:32,119][00611] Updated weights for policy 0, policy_version 48382 (0.0009) [2023-10-08 05:49:33,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 99352576. Throughput: 0: 1825.5, 1: 1842.0. Samples: 24849814. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-08 05:49:33,755][130385] Avg episode reward: [(0, '66.830'), (1, '72.380')] [2023-10-08 05:49:34,280][00612] Updated weights for policy 1, policy_version 48650 (0.0010) [2023-10-08 05:49:34,645][00612] Updated weights for policy 1, policy_version 48660 (0.0010) [2023-10-08 05:49:35,024][00612] Updated weights for policy 1, policy_version 48670 (0.0007) [2023-10-08 05:49:35,846][00611] Updated weights for policy 0, policy_version 48392 (0.0007) [2023-10-08 05:49:36,228][00611] Updated weights for policy 0, policy_version 48402 (0.0010) [2023-10-08 05:49:36,605][00611] Updated weights for policy 0, policy_version 48412 (0.0010) [2023-10-08 05:49:38,634][00612] Updated weights for policy 1, policy_version 48680 (0.0009) [2023-10-08 05:49:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 99418112. Throughput: 0: 1816.7, 1: 1844.3. Samples: 24860566. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-08 05:49:38,755][130385] Avg episode reward: [(0, '67.780'), (1, '70.880')] [2023-10-08 05:49:39,014][00612] Updated weights for policy 1, policy_version 48690 (0.0008) [2023-10-08 05:49:39,383][00612] Updated weights for policy 1, policy_version 48700 (0.0009) [2023-10-08 05:49:40,266][00611] Updated weights for policy 0, policy_version 48422 (0.0008) [2023-10-08 05:49:40,646][00611] Updated weights for policy 0, policy_version 48432 (0.0008) [2023-10-08 05:49:41,026][00611] Updated weights for policy 0, policy_version 48442 (0.0011) [2023-10-08 05:49:42,968][00612] Updated weights for policy 1, policy_version 48710 (0.0007) [2023-10-08 05:49:43,334][00612] Updated weights for policy 1, policy_version 48720 (0.0008) [2023-10-08 05:49:43,707][00612] Updated weights for policy 1, policy_version 48730 (0.0010) [2023-10-08 05:49:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 99483648. Throughput: 0: 1816.2, 1: 1846.3. Samples: 24882952. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-08 05:49:43,755][130385] Avg episode reward: [(0, '67.870'), (1, '72.380')] [2023-10-08 05:49:44,640][00611] Updated weights for policy 0, policy_version 48452 (0.0009) [2023-10-08 05:49:45,011][00611] Updated weights for policy 0, policy_version 48462 (0.0010) [2023-10-08 05:49:45,383][00611] Updated weights for policy 0, policy_version 48472 (0.0011) [2023-10-08 05:49:47,317][00612] Updated weights for policy 1, policy_version 48740 (0.0008) [2023-10-08 05:49:47,683][00612] Updated weights for policy 1, policy_version 48750 (0.0008) [2023-10-08 05:49:48,061][00612] Updated weights for policy 1, policy_version 48760 (0.0009) [2023-10-08 05:49:48,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 99581952. Throughput: 0: 1816.8, 1: 1832.0. Samples: 24904912. Policy #0 lag: (min: 25.0, avg: 38.5, max: 57.0) [2023-10-08 05:49:48,754][130385] Avg episode reward: [(0, '70.460'), (1, '68.710')] [2023-10-08 05:49:48,766][00365] Saving new best policy, reward=70.460! [2023-10-08 05:49:49,183][00611] Updated weights for policy 0, policy_version 48482 (0.0009) [2023-10-08 05:49:49,552][00611] Updated weights for policy 0, policy_version 48492 (0.0010) [2023-10-08 05:49:49,914][00611] Updated weights for policy 0, policy_version 48502 (0.0007) [2023-10-08 05:49:50,289][00611] Updated weights for policy 0, policy_version 48512 (0.0008) [2023-10-08 05:49:51,582][00612] Updated weights for policy 1, policy_version 48770 (0.0009) [2023-10-08 05:49:51,944][00612] Updated weights for policy 1, policy_version 48780 (0.0009) [2023-10-08 05:49:52,313][00612] Updated weights for policy 1, policy_version 48790 (0.0008) [2023-10-08 05:49:52,676][00612] Updated weights for policy 1, policy_version 48800 (0.0008) [2023-10-08 05:49:53,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 99647488. Throughput: 0: 1813.7, 1: 1848.8. Samples: 24916154. Policy #0 lag: (min: 25.0, avg: 38.5, max: 57.0) [2023-10-08 05:49:53,754][130385] Avg episode reward: [(0, '71.530'), (1, '67.710')] [2023-10-08 05:49:54,057][00611] Updated weights for policy 0, policy_version 48522 (0.0007) [2023-10-08 05:49:54,424][00611] Updated weights for policy 0, policy_version 48532 (0.0008) [2023-10-08 05:49:54,793][00611] Updated weights for policy 0, policy_version 48542 (0.0010) [2023-10-08 05:49:54,867][00365] Saving new best policy, reward=71.530! [2023-10-08 05:49:56,273][00612] Updated weights for policy 1, policy_version 48810 (0.0011) [2023-10-08 05:49:56,640][00612] Updated weights for policy 1, policy_version 48820 (0.0011) [2023-10-08 05:49:57,017][00612] Updated weights for policy 1, policy_version 48830 (0.0009) [2023-10-08 05:49:58,478][00611] Updated weights for policy 0, policy_version 48552 (0.0007) [2023-10-08 05:49:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 99713024. Throughput: 0: 1814.8, 1: 1833.9. Samples: 24937540. Policy #0 lag: (min: 25.0, avg: 38.5, max: 57.0) [2023-10-08 05:49:58,754][130385] Avg episode reward: [(0, '67.650'), (1, '70.460')] [2023-10-08 05:49:58,845][00611] Updated weights for policy 0, policy_version 48562 (0.0009) [2023-10-08 05:49:59,214][00611] Updated weights for policy 0, policy_version 48572 (0.0009) [2023-10-08 05:50:00,722][00612] Updated weights for policy 1, policy_version 48840 (0.0009) [2023-10-08 05:50:01,091][00612] Updated weights for policy 1, policy_version 48850 (0.0009) [2023-10-08 05:50:01,458][00612] Updated weights for policy 1, policy_version 48860 (0.0008) [2023-10-08 05:50:02,885][00611] Updated weights for policy 0, policy_version 48582 (0.0010) [2023-10-08 05:50:03,266][00611] Updated weights for policy 0, policy_version 48592 (0.0010) [2023-10-08 05:50:03,628][00611] Updated weights for policy 0, policy_version 48602 (0.0011) [2023-10-08 05:50:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 99778560. Throughput: 0: 1815.3, 1: 1852.7. Samples: 24959776. Policy #0 lag: (min: 25.0, avg: 38.5, max: 57.0) [2023-10-08 05:50:03,754][130385] Avg episode reward: [(0, '65.390'), (1, '70.580')] [2023-10-08 05:50:05,102][00612] Updated weights for policy 1, policy_version 48870 (0.0010) [2023-10-08 05:50:05,467][00612] Updated weights for policy 1, policy_version 48880 (0.0011) [2023-10-08 05:50:05,836][00612] Updated weights for policy 1, policy_version 48890 (0.0009) [2023-10-08 05:50:07,212][00611] Updated weights for policy 0, policy_version 48612 (0.0008) [2023-10-08 05:50:07,583][00611] Updated weights for policy 0, policy_version 48622 (0.0007) [2023-10-08 05:50:07,957][00611] Updated weights for policy 0, policy_version 48632 (0.0009) [2023-10-08 05:50:08,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 99876864. Throughput: 0: 1813.7, 1: 1834.7. Samples: 24970238. Policy #0 lag: (min: 25.0, avg: 38.5, max: 57.0) [2023-10-08 05:50:08,754][130385] Avg episode reward: [(0, '62.210'), (1, '70.250')] [2023-10-08 05:50:09,543][00612] Updated weights for policy 1, policy_version 48900 (0.0007) [2023-10-08 05:50:09,922][00612] Updated weights for policy 1, policy_version 48910 (0.0010) [2023-10-08 05:50:10,278][00612] Updated weights for policy 1, policy_version 48920 (0.0009) [2023-10-08 05:50:11,555][00611] Updated weights for policy 0, policy_version 48642 (0.0008) [2023-10-08 05:50:11,929][00611] Updated weights for policy 0, policy_version 48652 (0.0010) [2023-10-08 05:50:12,299][00611] Updated weights for policy 0, policy_version 48662 (0.0009) [2023-10-08 05:50:12,674][00611] Updated weights for policy 0, policy_version 48672 (0.0009) [2023-10-08 05:50:13,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 99942400. Throughput: 0: 1818.0, 1: 1850.7. Samples: 24992472. Policy #0 lag: (min: 25.0, avg: 38.5, max: 57.0) [2023-10-08 05:50:13,755][130385] Avg episode reward: [(0, '63.890'), (1, '70.380')] [2023-10-08 05:50:14,014][00612] Updated weights for policy 1, policy_version 48930 (0.0008) [2023-10-08 05:50:14,377][00612] Updated weights for policy 1, policy_version 48940 (0.0009) [2023-10-08 05:50:14,744][00612] Updated weights for policy 1, policy_version 48950 (0.0008) [2023-10-08 05:50:15,112][00612] Updated weights for policy 1, policy_version 48960 (0.0011) [2023-10-08 05:50:16,368][00611] Updated weights for policy 0, policy_version 48682 (0.0009) [2023-10-08 05:50:16,737][00611] Updated weights for policy 0, policy_version 48692 (0.0010) [2023-10-08 05:50:17,107][00611] Updated weights for policy 0, policy_version 48702 (0.0009) [2023-10-08 05:50:18,639][00612] Updated weights for policy 1, policy_version 48970 (0.0009) [2023-10-08 05:50:18,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 100007936. Throughput: 0: 1814.2, 1: 1852.1. Samples: 25014798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:50:18,755][130385] Avg episode reward: [(0, '63.190'), (1, '71.090')] [2023-10-08 05:50:18,765][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000048704_49872896.pth... [2023-10-08 05:50:18,804][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000047008_48136192.pth [2023-10-08 05:50:19,015][00612] Updated weights for policy 1, policy_version 48980 (0.0010) [2023-10-08 05:50:19,385][00612] Updated weights for policy 1, policy_version 48990 (0.0010) [2023-10-08 05:50:19,450][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000048992_50167808.pth... [2023-10-08 05:50:19,478][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000047264_48398336.pth [2023-10-08 05:50:20,860][00611] Updated weights for policy 0, policy_version 48712 (0.0010) [2023-10-08 05:50:21,220][00611] Updated weights for policy 0, policy_version 48722 (0.0009) [2023-10-08 05:50:21,601][00611] Updated weights for policy 0, policy_version 48732 (0.0008) [2023-10-08 05:50:23,115][00612] Updated weights for policy 1, policy_version 49000 (0.0007) [2023-10-08 05:50:23,482][00612] Updated weights for policy 1, policy_version 49010 (0.0007) [2023-10-08 05:50:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 100073472. Throughput: 0: 1819.7, 1: 1851.1. Samples: 25025748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:50:23,755][130385] Avg episode reward: [(0, '61.050'), (1, '75.490')] [2023-10-08 05:50:23,850][00612] Updated weights for policy 1, policy_version 49020 (0.0009) [2023-10-08 05:50:23,997][00425] Saving new best policy, reward=75.490! [2023-10-08 05:50:25,385][00611] Updated weights for policy 0, policy_version 48742 (0.0009) [2023-10-08 05:50:25,755][00611] Updated weights for policy 0, policy_version 48752 (0.0010) [2023-10-08 05:50:26,140][00611] Updated weights for policy 0, policy_version 48762 (0.0010) [2023-10-08 05:50:27,581][00612] Updated weights for policy 1, policy_version 49030 (0.0010) [2023-10-08 05:50:27,955][00612] Updated weights for policy 1, policy_version 49040 (0.0010) [2023-10-08 05:50:28,316][00612] Updated weights for policy 1, policy_version 49050 (0.0009) [2023-10-08 05:50:28,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 100171776. Throughput: 0: 1820.0, 1: 1847.6. Samples: 25047996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:50:28,755][130385] Avg episode reward: [(0, '62.680'), (1, '75.670')] [2023-10-08 05:50:28,757][00425] Saving new best policy, reward=75.670! [2023-10-08 05:50:29,686][00611] Updated weights for policy 0, policy_version 48772 (0.0008) [2023-10-08 05:50:30,060][00611] Updated weights for policy 0, policy_version 48782 (0.0008) [2023-10-08 05:50:30,442][00611] Updated weights for policy 0, policy_version 48792 (0.0008) [2023-10-08 05:50:31,927][00612] Updated weights for policy 1, policy_version 49060 (0.0007) [2023-10-08 05:50:32,297][00612] Updated weights for policy 1, policy_version 49070 (0.0007) [2023-10-08 05:50:32,652][00612] Updated weights for policy 1, policy_version 49080 (0.0007) [2023-10-08 05:50:33,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 100237312. Throughput: 0: 1832.2, 1: 1840.5. Samples: 25070184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:50:33,754][130385] Avg episode reward: [(0, '62.080'), (1, '74.360')] [2023-10-08 05:50:33,995][00611] Updated weights for policy 0, policy_version 48802 (0.0009) [2023-10-08 05:50:34,361][00611] Updated weights for policy 0, policy_version 48812 (0.0009) [2023-10-08 05:50:34,732][00611] Updated weights for policy 0, policy_version 48822 (0.0010) [2023-10-08 05:50:35,099][00611] Updated weights for policy 0, policy_version 48832 (0.0010) [2023-10-08 05:50:36,031][00612] Updated weights for policy 1, policy_version 49090 (0.0007) [2023-10-08 05:50:36,398][00612] Updated weights for policy 1, policy_version 49100 (0.0007) [2023-10-08 05:50:36,768][00612] Updated weights for policy 1, policy_version 49110 (0.0009) [2023-10-08 05:50:37,142][00612] Updated weights for policy 1, policy_version 49120 (0.0007) [2023-10-08 05:50:38,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 100302848. Throughput: 0: 1827.7, 1: 1842.0. Samples: 25081290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:50:38,754][130385] Avg episode reward: [(0, '61.040'), (1, '74.380')] [2023-10-08 05:50:38,934][00611] Updated weights for policy 0, policy_version 48842 (0.0007) [2023-10-08 05:50:39,307][00611] Updated weights for policy 0, policy_version 48852 (0.0008) [2023-10-08 05:50:39,680][00611] Updated weights for policy 0, policy_version 48862 (0.0008) [2023-10-08 05:50:40,740][00612] Updated weights for policy 1, policy_version 49130 (0.0009) [2023-10-08 05:50:41,104][00612] Updated weights for policy 1, policy_version 49140 (0.0008) [2023-10-08 05:50:41,470][00612] Updated weights for policy 1, policy_version 49150 (0.0007) [2023-10-08 05:50:43,436][00611] Updated weights for policy 0, policy_version 48872 (0.0009) [2023-10-08 05:50:43,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 100368384. Throughput: 0: 1828.2, 1: 1851.1. Samples: 25103106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:50:43,755][130385] Avg episode reward: [(0, '59.890'), (1, '70.070')] [2023-10-08 05:50:43,807][00611] Updated weights for policy 0, policy_version 48882 (0.0011) [2023-10-08 05:50:44,178][00611] Updated weights for policy 0, policy_version 48892 (0.0009) [2023-10-08 05:50:45,045][00612] Updated weights for policy 1, policy_version 49160 (0.0009) [2023-10-08 05:50:45,405][00612] Updated weights for policy 1, policy_version 49170 (0.0007) [2023-10-08 05:50:45,769][00612] Updated weights for policy 1, policy_version 49180 (0.0010) [2023-10-08 05:50:47,811][00611] Updated weights for policy 0, policy_version 48902 (0.0009) [2023-10-08 05:50:48,190][00611] Updated weights for policy 0, policy_version 48912 (0.0008) [2023-10-08 05:50:48,565][00611] Updated weights for policy 0, policy_version 48922 (0.0010) [2023-10-08 05:50:48,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 100433920. Throughput: 0: 1831.1, 1: 1849.1. Samples: 25125388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:50:48,755][130385] Avg episode reward: [(0, '55.660'), (1, '70.680')] [2023-10-08 05:50:49,364][00612] Updated weights for policy 1, policy_version 49190 (0.0008) [2023-10-08 05:50:49,721][00612] Updated weights for policy 1, policy_version 49200 (0.0007) [2023-10-08 05:50:50,093][00612] Updated weights for policy 1, policy_version 49210 (0.0007) [2023-10-08 05:50:52,199][00611] Updated weights for policy 0, policy_version 48932 (0.0007) [2023-10-08 05:50:52,578][00611] Updated weights for policy 0, policy_version 48942 (0.0008) [2023-10-08 05:50:52,950][00611] Updated weights for policy 0, policy_version 48952 (0.0007) [2023-10-08 05:50:53,739][00612] Updated weights for policy 1, policy_version 49220 (0.0008) [2023-10-08 05:50:53,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 100532224. Throughput: 0: 1836.3, 1: 1853.2. Samples: 25136268. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 05:50:53,754][130385] Avg episode reward: [(0, '59.540'), (1, '69.140')] [2023-10-08 05:50:54,105][00612] Updated weights for policy 1, policy_version 49230 (0.0007) [2023-10-08 05:50:54,471][00612] Updated weights for policy 1, policy_version 49240 (0.0008) [2023-10-08 05:50:56,521][00611] Updated weights for policy 0, policy_version 48962 (0.0008) [2023-10-08 05:50:56,894][00611] Updated weights for policy 0, policy_version 48972 (0.0008) [2023-10-08 05:50:57,271][00611] Updated weights for policy 0, policy_version 48982 (0.0007) [2023-10-08 05:50:57,648][00611] Updated weights for policy 0, policy_version 48992 (0.0007) [2023-10-08 05:50:58,176][00612] Updated weights for policy 1, policy_version 49250 (0.0007) [2023-10-08 05:50:58,533][00612] Updated weights for policy 1, policy_version 49260 (0.0007) [2023-10-08 05:50:58,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 100597760. Throughput: 0: 1832.6, 1: 1859.6. Samples: 25158620. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 05:50:58,755][130385] Avg episode reward: [(0, '59.550'), (1, '72.960')] [2023-10-08 05:50:58,904][00612] Updated weights for policy 1, policy_version 49270 (0.0008) [2023-10-08 05:50:59,273][00612] Updated weights for policy 1, policy_version 49280 (0.0009) [2023-10-08 05:51:01,019][00611] Updated weights for policy 0, policy_version 49002 (0.0009) [2023-10-08 05:51:01,397][00611] Updated weights for policy 0, policy_version 49012 (0.0008) [2023-10-08 05:51:01,769][00611] Updated weights for policy 0, policy_version 49022 (0.0008) [2023-10-08 05:51:02,905][00612] Updated weights for policy 1, policy_version 49290 (0.0010) [2023-10-08 05:51:03,277][00612] Updated weights for policy 1, policy_version 49300 (0.0009) [2023-10-08 05:51:03,644][00612] Updated weights for policy 1, policy_version 49310 (0.0009) [2023-10-08 05:51:03,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 100696064. Throughput: 0: 1845.3, 1: 1832.9. Samples: 25180314. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 05:51:03,754][130385] Avg episode reward: [(0, '60.290'), (1, '76.900')] [2023-10-08 05:51:03,764][00425] Saving new best policy, reward=76.900! [2023-10-08 05:51:05,415][00611] Updated weights for policy 0, policy_version 49032 (0.0011) [2023-10-08 05:51:05,794][00611] Updated weights for policy 0, policy_version 49042 (0.0010) [2023-10-08 05:51:06,165][00611] Updated weights for policy 0, policy_version 49052 (0.0010) [2023-10-08 05:51:07,152][00612] Updated weights for policy 1, policy_version 49320 (0.0009) [2023-10-08 05:51:07,522][00612] Updated weights for policy 1, policy_version 49330 (0.0010) [2023-10-08 05:51:07,889][00612] Updated weights for policy 1, policy_version 49340 (0.0008) [2023-10-08 05:51:08,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 100761600. Throughput: 0: 1827.9, 1: 1858.6. Samples: 25191642. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 05:51:08,755][130385] Avg episode reward: [(0, '57.790'), (1, '75.670')] [2023-10-08 05:51:09,845][00611] Updated weights for policy 0, policy_version 49062 (0.0008) [2023-10-08 05:51:10,220][00611] Updated weights for policy 0, policy_version 49072 (0.0008) [2023-10-08 05:51:10,594][00611] Updated weights for policy 0, policy_version 49082 (0.0008) [2023-10-08 05:51:11,575][00612] Updated weights for policy 1, policy_version 49350 (0.0008) [2023-10-08 05:51:11,938][00612] Updated weights for policy 1, policy_version 49360 (0.0008) [2023-10-08 05:51:12,318][00612] Updated weights for policy 1, policy_version 49370 (0.0008) [2023-10-08 05:51:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 100827136. Throughput: 0: 1846.6, 1: 1832.4. Samples: 25213552. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 05:51:13,754][130385] Avg episode reward: [(0, '59.130'), (1, '74.060')] [2023-10-08 05:51:14,096][00611] Updated weights for policy 0, policy_version 49092 (0.0008) [2023-10-08 05:51:14,471][00611] Updated weights for policy 0, policy_version 49102 (0.0007) [2023-10-08 05:51:14,851][00611] Updated weights for policy 0, policy_version 49112 (0.0008) [2023-10-08 05:51:15,908][00612] Updated weights for policy 1, policy_version 49380 (0.0010) [2023-10-08 05:51:16,276][00612] Updated weights for policy 1, policy_version 49390 (0.0008) [2023-10-08 05:51:16,652][00612] Updated weights for policy 1, policy_version 49400 (0.0007) [2023-10-08 05:51:18,435][00611] Updated weights for policy 0, policy_version 49122 (0.0008) [2023-10-08 05:51:18,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 100892672. Throughput: 0: 1840.6, 1: 1851.4. Samples: 25236326. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 05:51:18,754][130385] Avg episode reward: [(0, '60.010'), (1, '72.880')] [2023-10-08 05:51:18,815][00611] Updated weights for policy 0, policy_version 49132 (0.0009) [2023-10-08 05:51:19,183][00611] Updated weights for policy 0, policy_version 49142 (0.0008) [2023-10-08 05:51:19,556][00611] Updated weights for policy 0, policy_version 49152 (0.0009) [2023-10-08 05:51:20,369][00612] Updated weights for policy 1, policy_version 49410 (0.0010) [2023-10-08 05:51:20,744][00612] Updated weights for policy 1, policy_version 49420 (0.0010) [2023-10-08 05:51:21,109][00612] Updated weights for policy 1, policy_version 49430 (0.0007) [2023-10-08 05:51:21,482][00612] Updated weights for policy 1, policy_version 49440 (0.0007) [2023-10-08 05:51:23,236][00611] Updated weights for policy 0, policy_version 49162 (0.0009) [2023-10-08 05:51:23,598][00611] Updated weights for policy 0, policy_version 49172 (0.0008) [2023-10-08 05:51:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 100958208. Throughput: 0: 1848.8, 1: 1826.7. Samples: 25246688. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 05:51:23,754][130385] Avg episode reward: [(0, '64.610'), (1, '72.800')] [2023-10-08 05:51:23,968][00611] Updated weights for policy 0, policy_version 49182 (0.0008) [2023-10-08 05:51:25,231][00612] Updated weights for policy 1, policy_version 49450 (0.0011) [2023-10-08 05:51:25,606][00612] Updated weights for policy 1, policy_version 49460 (0.0010) [2023-10-08 05:51:25,979][00612] Updated weights for policy 1, policy_version 49470 (0.0007) [2023-10-08 05:51:27,413][00611] Updated weights for policy 0, policy_version 49192 (0.0007) [2023-10-08 05:51:27,787][00611] Updated weights for policy 0, policy_version 49202 (0.0009) [2023-10-08 05:51:28,162][00611] Updated weights for policy 0, policy_version 49212 (0.0008) [2023-10-08 05:51:28,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 101056512. Throughput: 0: 1860.6, 1: 1843.9. Samples: 25269808. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 05:51:28,754][130385] Avg episode reward: [(0, '64.770'), (1, '75.780')] [2023-10-08 05:51:29,587][00612] Updated weights for policy 1, policy_version 49480 (0.0009) [2023-10-08 05:51:29,948][00612] Updated weights for policy 1, policy_version 49490 (0.0010) [2023-10-08 05:51:30,313][00612] Updated weights for policy 1, policy_version 49500 (0.0009) [2023-10-08 05:51:31,702][00611] Updated weights for policy 0, policy_version 49222 (0.0007) [2023-10-08 05:51:32,066][00611] Updated weights for policy 0, policy_version 49232 (0.0008) [2023-10-08 05:51:32,445][00611] Updated weights for policy 0, policy_version 49242 (0.0008) [2023-10-08 05:51:33,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 101122048. Throughput: 0: 1844.9, 1: 1847.0. Samples: 25291526. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 05:51:33,754][130385] Avg episode reward: [(0, '63.120'), (1, '73.520')] [2023-10-08 05:51:34,041][00612] Updated weights for policy 1, policy_version 49510 (0.0008) [2023-10-08 05:51:34,417][00612] Updated weights for policy 1, policy_version 49520 (0.0011) [2023-10-08 05:51:34,784][00612] Updated weights for policy 1, policy_version 49530 (0.0007) [2023-10-08 05:51:36,097][00611] Updated weights for policy 0, policy_version 49252 (0.0008) [2023-10-08 05:51:36,467][00611] Updated weights for policy 0, policy_version 49262 (0.0011) [2023-10-08 05:51:36,840][00611] Updated weights for policy 0, policy_version 49272 (0.0011) [2023-10-08 05:51:38,388][00612] Updated weights for policy 1, policy_version 49540 (0.0007) [2023-10-08 05:51:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 101187584. Throughput: 0: 1854.2, 1: 1841.0. Samples: 25302552. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 05:51:38,754][130385] Avg episode reward: [(0, '60.060'), (1, '69.490')] [2023-10-08 05:51:38,756][00612] Updated weights for policy 1, policy_version 49550 (0.0008) [2023-10-08 05:51:39,127][00612] Updated weights for policy 1, policy_version 49560 (0.0008) [2023-10-08 05:51:40,612][00611] Updated weights for policy 0, policy_version 49282 (0.0009) [2023-10-08 05:51:40,996][00611] Updated weights for policy 0, policy_version 49292 (0.0009) [2023-10-08 05:51:41,367][00611] Updated weights for policy 0, policy_version 49302 (0.0011) [2023-10-08 05:51:41,741][00611] Updated weights for policy 0, policy_version 49312 (0.0008) [2023-10-08 05:51:42,774][00612] Updated weights for policy 1, policy_version 49570 (0.0007) [2023-10-08 05:51:43,143][00612] Updated weights for policy 1, policy_version 49580 (0.0008) [2023-10-08 05:51:43,510][00612] Updated weights for policy 1, policy_version 49590 (0.0007) [2023-10-08 05:51:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 101253120. Throughput: 0: 1839.3, 1: 1841.5. Samples: 25324254. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 05:51:43,754][130385] Avg episode reward: [(0, '63.380'), (1, '69.010')] [2023-10-08 05:51:43,880][00612] Updated weights for policy 1, policy_version 49600 (0.0008) [2023-10-08 05:51:45,340][00611] Updated weights for policy 0, policy_version 49322 (0.0008) [2023-10-08 05:51:45,719][00611] Updated weights for policy 0, policy_version 49332 (0.0008) [2023-10-08 05:51:46,087][00611] Updated weights for policy 0, policy_version 49342 (0.0008) [2023-10-08 05:51:47,472][00612] Updated weights for policy 1, policy_version 49610 (0.0009) [2023-10-08 05:51:47,849][00612] Updated weights for policy 1, policy_version 49620 (0.0012) [2023-10-08 05:51:48,215][00612] Updated weights for policy 1, policy_version 49630 (0.0010) [2023-10-08 05:51:48,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 101351424. Throughput: 0: 1850.1, 1: 1832.2. Samples: 25346018. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 05:51:48,755][130385] Avg episode reward: [(0, '64.090'), (1, '71.860')] [2023-10-08 05:51:49,731][00611] Updated weights for policy 0, policy_version 49352 (0.0010) [2023-10-08 05:51:50,089][00611] Updated weights for policy 0, policy_version 49362 (0.0009) [2023-10-08 05:51:50,454][00611] Updated weights for policy 0, policy_version 49372 (0.0011) [2023-10-08 05:51:51,921][00612] Updated weights for policy 1, policy_version 49640 (0.0009) [2023-10-08 05:51:52,290][00612] Updated weights for policy 1, policy_version 49650 (0.0008) [2023-10-08 05:51:52,652][00612] Updated weights for policy 1, policy_version 49660 (0.0008) [2023-10-08 05:51:53,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 101416960. Throughput: 0: 1842.7, 1: 1835.9. Samples: 25357180. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 05:51:53,755][130385] Avg episode reward: [(0, '61.780'), (1, '73.090')] [2023-10-08 05:51:54,202][00611] Updated weights for policy 0, policy_version 49382 (0.0009) [2023-10-08 05:51:54,570][00611] Updated weights for policy 0, policy_version 49392 (0.0010) [2023-10-08 05:51:54,944][00611] Updated weights for policy 0, policy_version 49402 (0.0009) [2023-10-08 05:51:56,274][00612] Updated weights for policy 1, policy_version 49670 (0.0009) [2023-10-08 05:51:56,640][00612] Updated weights for policy 1, policy_version 49680 (0.0012) [2023-10-08 05:51:57,010][00612] Updated weights for policy 1, policy_version 49690 (0.0010) [2023-10-08 05:51:58,642][00611] Updated weights for policy 0, policy_version 49412 (0.0010) [2023-10-08 05:51:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 101482496. Throughput: 0: 1839.7, 1: 1828.8. Samples: 25378638. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 05:51:58,755][130385] Avg episode reward: [(0, '62.630'), (1, '71.580')] [2023-10-08 05:51:59,017][00611] Updated weights for policy 0, policy_version 49422 (0.0011) [2023-10-08 05:51:59,393][00611] Updated weights for policy 0, policy_version 49432 (0.0009) [2023-10-08 05:52:00,506][00612] Updated weights for policy 1, policy_version 49700 (0.0009) [2023-10-08 05:52:00,871][00612] Updated weights for policy 1, policy_version 49710 (0.0009) [2023-10-08 05:52:01,239][00612] Updated weights for policy 1, policy_version 49720 (0.0008) [2023-10-08 05:52:03,031][00611] Updated weights for policy 0, policy_version 49442 (0.0008) [2023-10-08 05:52:03,415][00611] Updated weights for policy 0, policy_version 49452 (0.0011) [2023-10-08 05:52:03,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 101548032. Throughput: 0: 1827.8, 1: 1843.3. Samples: 25401528. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 05:52:03,754][130385] Avg episode reward: [(0, '62.490'), (1, '64.830')] [2023-10-08 05:52:03,777][00611] Updated weights for policy 0, policy_version 49462 (0.0009) [2023-10-08 05:52:04,144][00611] Updated weights for policy 0, policy_version 49472 (0.0009) [2023-10-08 05:52:04,905][00612] Updated weights for policy 1, policy_version 49730 (0.0008) [2023-10-08 05:52:05,281][00612] Updated weights for policy 1, policy_version 49740 (0.0009) [2023-10-08 05:52:05,647][00612] Updated weights for policy 1, policy_version 49750 (0.0008) [2023-10-08 05:52:06,004][00612] Updated weights for policy 1, policy_version 49760 (0.0007) [2023-10-08 05:52:07,736][00611] Updated weights for policy 0, policy_version 49482 (0.0008) [2023-10-08 05:52:08,109][00611] Updated weights for policy 0, policy_version 49492 (0.0008) [2023-10-08 05:52:08,472][00611] Updated weights for policy 0, policy_version 49502 (0.0008) [2023-10-08 05:52:08,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 101646336. Throughput: 0: 1833.3, 1: 1834.4. Samples: 25411738. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 05:52:08,754][130385] Avg episode reward: [(0, '65.970'), (1, '62.560')] [2023-10-08 05:52:09,737][00612] Updated weights for policy 1, policy_version 49770 (0.0009) [2023-10-08 05:52:10,110][00612] Updated weights for policy 1, policy_version 49780 (0.0009) [2023-10-08 05:52:10,480][00612] Updated weights for policy 1, policy_version 49790 (0.0011) [2023-10-08 05:52:12,236][00611] Updated weights for policy 0, policy_version 49512 (0.0009) [2023-10-08 05:52:12,607][00611] Updated weights for policy 0, policy_version 49522 (0.0009) [2023-10-08 05:52:12,978][00611] Updated weights for policy 0, policy_version 49532 (0.0009) [2023-10-08 05:52:13,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 101711872. Throughput: 0: 1816.1, 1: 1841.9. Samples: 25434418. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 05:52:13,755][130385] Avg episode reward: [(0, '67.470'), (1, '64.180')] [2023-10-08 05:52:14,203][00612] Updated weights for policy 1, policy_version 49800 (0.0008) [2023-10-08 05:52:14,565][00612] Updated weights for policy 1, policy_version 49810 (0.0008) [2023-10-08 05:52:14,938][00612] Updated weights for policy 1, policy_version 49820 (0.0007) [2023-10-08 05:52:16,703][00611] Updated weights for policy 0, policy_version 49542 (0.0007) [2023-10-08 05:52:17,088][00611] Updated weights for policy 0, policy_version 49552 (0.0007) [2023-10-08 05:52:17,463][00611] Updated weights for policy 0, policy_version 49562 (0.0007) [2023-10-08 05:52:18,339][00612] Updated weights for policy 1, policy_version 49830 (0.0009) [2023-10-08 05:52:18,703][00612] Updated weights for policy 1, policy_version 49840 (0.0007) [2023-10-08 05:52:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 101777408. Throughput: 0: 1814.9, 1: 1843.1. Samples: 25456134. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 05:52:18,754][130385] Avg episode reward: [(0, '66.540'), (1, '65.920')] [2023-10-08 05:52:18,761][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000049568_50757632.pth... [2023-10-08 05:52:18,800][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000047840_48988160.pth [2023-10-08 05:52:19,077][00612] Updated weights for policy 1, policy_version 49850 (0.0010) [2023-10-08 05:52:19,296][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000049856_51052544.pth... [2023-10-08 05:52:19,326][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000048128_49283072.pth [2023-10-08 05:52:21,134][00611] Updated weights for policy 0, policy_version 49572 (0.0009) [2023-10-08 05:52:21,511][00611] Updated weights for policy 0, policy_version 49582 (0.0010) [2023-10-08 05:52:21,885][00611] Updated weights for policy 0, policy_version 49592 (0.0010) [2023-10-08 05:52:22,759][00612] Updated weights for policy 1, policy_version 49860 (0.0009) [2023-10-08 05:52:23,143][00612] Updated weights for policy 1, policy_version 49870 (0.0009) [2023-10-08 05:52:23,511][00612] Updated weights for policy 1, policy_version 49880 (0.0007) [2023-10-08 05:52:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 101842944. Throughput: 0: 1821.3, 1: 1847.2. Samples: 25467632. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 05:52:23,754][130385] Avg episode reward: [(0, '62.290'), (1, '64.220')] [2023-10-08 05:52:25,454][00611] Updated weights for policy 0, policy_version 49602 (0.0009) [2023-10-08 05:52:25,828][00611] Updated weights for policy 0, policy_version 49612 (0.0008) [2023-10-08 05:52:26,206][00611] Updated weights for policy 0, policy_version 49622 (0.0008) [2023-10-08 05:52:26,571][00611] Updated weights for policy 0, policy_version 49632 (0.0008) [2023-10-08 05:52:27,181][00612] Updated weights for policy 1, policy_version 49890 (0.0008) [2023-10-08 05:52:27,556][00612] Updated weights for policy 1, policy_version 49900 (0.0009) [2023-10-08 05:52:27,914][00612] Updated weights for policy 1, policy_version 49910 (0.0008) [2023-10-08 05:52:28,284][00612] Updated weights for policy 1, policy_version 49920 (0.0009) [2023-10-08 05:52:28,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 101941248. Throughput: 0: 1830.3, 1: 1839.6. Samples: 25489402. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 05:52:28,755][130385] Avg episode reward: [(0, '62.030'), (1, '63.920')] [2023-10-08 05:52:30,204][00611] Updated weights for policy 0, policy_version 49642 (0.0008) [2023-10-08 05:52:30,573][00611] Updated weights for policy 0, policy_version 49652 (0.0007) [2023-10-08 05:52:30,942][00611] Updated weights for policy 0, policy_version 49662 (0.0007) [2023-10-08 05:52:31,930][00612] Updated weights for policy 1, policy_version 49930 (0.0007) [2023-10-08 05:52:32,301][00612] Updated weights for policy 1, policy_version 49940 (0.0007) [2023-10-08 05:52:32,680][00612] Updated weights for policy 1, policy_version 49950 (0.0008) [2023-10-08 05:52:33,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 102006784. Throughput: 0: 1832.1, 1: 1837.3. Samples: 25511144. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 05:52:33,754][130385] Avg episode reward: [(0, '62.170'), (1, '61.420')] [2023-10-08 05:52:34,652][00611] Updated weights for policy 0, policy_version 49672 (0.0010) [2023-10-08 05:52:35,017][00611] Updated weights for policy 0, policy_version 49682 (0.0009) [2023-10-08 05:52:35,393][00611] Updated weights for policy 0, policy_version 49692 (0.0011) [2023-10-08 05:52:36,194][00612] Updated weights for policy 1, policy_version 49960 (0.0010) [2023-10-08 05:52:36,563][00612] Updated weights for policy 1, policy_version 49970 (0.0008) [2023-10-08 05:52:36,937][00612] Updated weights for policy 1, policy_version 49980 (0.0008) [2023-10-08 05:52:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 102072320. Throughput: 0: 1837.6, 1: 1835.0. Samples: 25522444. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-08 05:52:38,755][130385] Avg episode reward: [(0, '66.090'), (1, '64.310')] [2023-10-08 05:52:38,992][00611] Updated weights for policy 0, policy_version 49702 (0.0009) [2023-10-08 05:52:39,358][00611] Updated weights for policy 0, policy_version 49712 (0.0007) [2023-10-08 05:52:39,727][00611] Updated weights for policy 0, policy_version 49722 (0.0007) [2023-10-08 05:52:40,750][00612] Updated weights for policy 1, policy_version 49990 (0.0009) [2023-10-08 05:52:41,120][00612] Updated weights for policy 1, policy_version 50000 (0.0008) [2023-10-08 05:52:41,483][00612] Updated weights for policy 1, policy_version 50010 (0.0007) [2023-10-08 05:52:43,282][00611] Updated weights for policy 0, policy_version 49732 (0.0008) [2023-10-08 05:52:43,664][00611] Updated weights for policy 0, policy_version 49742 (0.0010) [2023-10-08 05:52:43,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 102137856. Throughput: 0: 1844.1, 1: 1837.5. Samples: 25544308. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-08 05:52:43,755][130385] Avg episode reward: [(0, '68.170'), (1, '63.870')] [2023-10-08 05:52:44,031][00611] Updated weights for policy 0, policy_version 49752 (0.0008) [2023-10-08 05:52:45,097][00612] Updated weights for policy 1, policy_version 50020 (0.0009) [2023-10-08 05:52:45,455][00612] Updated weights for policy 1, policy_version 50030 (0.0009) [2023-10-08 05:52:45,823][00612] Updated weights for policy 1, policy_version 50040 (0.0009) [2023-10-08 05:52:47,531][00611] Updated weights for policy 0, policy_version 49762 (0.0007) [2023-10-08 05:52:47,895][00611] Updated weights for policy 0, policy_version 49772 (0.0010) [2023-10-08 05:52:48,270][00611] Updated weights for policy 0, policy_version 49782 (0.0007) [2023-10-08 05:52:48,639][00611] Updated weights for policy 0, policy_version 49792 (0.0007) [2023-10-08 05:52:48,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 102236160. Throughput: 0: 1837.7, 1: 1838.0. Samples: 25566936. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-08 05:52:48,754][130385] Avg episode reward: [(0, '66.970'), (1, '65.670')] [2023-10-08 05:52:49,609][00612] Updated weights for policy 1, policy_version 50050 (0.0009) [2023-10-08 05:52:49,990][00612] Updated weights for policy 1, policy_version 50060 (0.0011) [2023-10-08 05:52:50,351][00612] Updated weights for policy 1, policy_version 50070 (0.0009) [2023-10-08 05:52:50,708][00612] Updated weights for policy 1, policy_version 50080 (0.0009) [2023-10-08 05:52:52,373][00611] Updated weights for policy 0, policy_version 49802 (0.0007) [2023-10-08 05:52:52,746][00611] Updated weights for policy 0, policy_version 49812 (0.0007) [2023-10-08 05:52:53,118][00611] Updated weights for policy 0, policy_version 49822 (0.0008) [2023-10-08 05:52:53,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 102301696. Throughput: 0: 1854.0, 1: 1836.8. Samples: 25577824. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-08 05:52:53,754][130385] Avg episode reward: [(0, '66.660'), (1, '63.560')] [2023-10-08 05:52:54,380][00612] Updated weights for policy 1, policy_version 50090 (0.0007) [2023-10-08 05:52:54,753][00612] Updated weights for policy 1, policy_version 50100 (0.0007) [2023-10-08 05:52:55,119][00612] Updated weights for policy 1, policy_version 50110 (0.0007) [2023-10-08 05:52:56,615][00611] Updated weights for policy 0, policy_version 49832 (0.0009) [2023-10-08 05:52:56,986][00611] Updated weights for policy 0, policy_version 49842 (0.0008) [2023-10-08 05:52:57,359][00611] Updated weights for policy 0, policy_version 49852 (0.0008) [2023-10-08 05:52:58,645][00612] Updated weights for policy 1, policy_version 50120 (0.0010) [2023-10-08 05:52:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 102367232. Throughput: 0: 1840.2, 1: 1845.0. Samples: 25600252. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-08 05:52:58,754][130385] Avg episode reward: [(0, '67.680'), (1, '63.740')] [2023-10-08 05:52:59,013][00612] Updated weights for policy 1, policy_version 50130 (0.0010) [2023-10-08 05:52:59,372][00612] Updated weights for policy 1, policy_version 50140 (0.0009) [2023-10-08 05:53:01,054][00611] Updated weights for policy 0, policy_version 49862 (0.0007) [2023-10-08 05:53:01,413][00611] Updated weights for policy 0, policy_version 49872 (0.0008) [2023-10-08 05:53:01,794][00611] Updated weights for policy 0, policy_version 49882 (0.0009) [2023-10-08 05:53:02,924][00612] Updated weights for policy 1, policy_version 50150 (0.0011) [2023-10-08 05:53:03,293][00612] Updated weights for policy 1, policy_version 50160 (0.0007) [2023-10-08 05:53:03,671][00612] Updated weights for policy 1, policy_version 50170 (0.0008) [2023-10-08 05:53:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 102432768. Throughput: 0: 1858.2, 1: 1834.9. Samples: 25622324. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-08 05:53:03,754][130385] Avg episode reward: [(0, '66.040'), (1, '67.690')] [2023-10-08 05:53:05,552][00611] Updated weights for policy 0, policy_version 49892 (0.0008) [2023-10-08 05:53:05,951][00611] Updated weights for policy 0, policy_version 49902 (0.0007) [2023-10-08 05:53:06,328][00611] Updated weights for policy 0, policy_version 49912 (0.0008) [2023-10-08 05:53:07,197][00612] Updated weights for policy 1, policy_version 50180 (0.0007) [2023-10-08 05:53:07,583][00612] Updated weights for policy 1, policy_version 50190 (0.0008) [2023-10-08 05:53:07,956][00612] Updated weights for policy 1, policy_version 50200 (0.0008) [2023-10-08 05:53:08,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 102531072. Throughput: 0: 1836.9, 1: 1850.9. Samples: 25633584. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-08 05:53:08,755][130385] Avg episode reward: [(0, '64.080'), (1, '66.440')] [2023-10-08 05:53:09,964][00611] Updated weights for policy 0, policy_version 49922 (0.0007) [2023-10-08 05:53:10,336][00611] Updated weights for policy 0, policy_version 49932 (0.0009) [2023-10-08 05:53:10,709][00611] Updated weights for policy 0, policy_version 49942 (0.0010) [2023-10-08 05:53:11,080][00611] Updated weights for policy 0, policy_version 49952 (0.0008) [2023-10-08 05:53:11,533][00612] Updated weights for policy 1, policy_version 50210 (0.0008) [2023-10-08 05:53:11,892][00612] Updated weights for policy 1, policy_version 50220 (0.0009) [2023-10-08 05:53:12,265][00612] Updated weights for policy 1, policy_version 50230 (0.0007) [2023-10-08 05:53:12,640][00612] Updated weights for policy 1, policy_version 50240 (0.0008) [2023-10-08 05:53:13,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 102596608. Throughput: 0: 1853.6, 1: 1837.1. Samples: 25655482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:53:13,754][130385] Avg episode reward: [(0, '65.440'), (1, '64.350')] [2023-10-08 05:53:14,519][00611] Updated weights for policy 0, policy_version 49962 (0.0009) [2023-10-08 05:53:14,891][00611] Updated weights for policy 0, policy_version 49972 (0.0011) [2023-10-08 05:53:15,261][00611] Updated weights for policy 0, policy_version 49982 (0.0010) [2023-10-08 05:53:16,188][00612] Updated weights for policy 1, policy_version 50250 (0.0007) [2023-10-08 05:53:16,573][00612] Updated weights for policy 1, policy_version 50260 (0.0008) [2023-10-08 05:53:16,947][00612] Updated weights for policy 1, policy_version 50270 (0.0009) [2023-10-08 05:53:18,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 102662144. Throughput: 0: 1850.3, 1: 1855.0. Samples: 25677884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:53:18,755][130385] Avg episode reward: [(0, '64.780'), (1, '64.990')] [2023-10-08 05:53:18,956][00611] Updated weights for policy 0, policy_version 49992 (0.0010) [2023-10-08 05:53:19,317][00611] Updated weights for policy 0, policy_version 50002 (0.0011) [2023-10-08 05:53:19,689][00611] Updated weights for policy 0, policy_version 50012 (0.0007) [2023-10-08 05:53:20,545][00612] Updated weights for policy 1, policy_version 50280 (0.0008) [2023-10-08 05:53:20,907][00612] Updated weights for policy 1, policy_version 50290 (0.0010) [2023-10-08 05:53:21,283][00612] Updated weights for policy 1, policy_version 50300 (0.0008) [2023-10-08 05:53:23,435][00611] Updated weights for policy 0, policy_version 50022 (0.0009) [2023-10-08 05:53:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 102727680. Throughput: 0: 1847.7, 1: 1840.8. Samples: 25688428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:53:23,755][130385] Avg episode reward: [(0, '66.220'), (1, '65.030')] [2023-10-08 05:53:23,808][00611] Updated weights for policy 0, policy_version 50032 (0.0011) [2023-10-08 05:53:24,177][00611] Updated weights for policy 0, policy_version 50042 (0.0011) [2023-10-08 05:53:25,006][00612] Updated weights for policy 1, policy_version 50310 (0.0008) [2023-10-08 05:53:25,371][00612] Updated weights for policy 1, policy_version 50320 (0.0008) [2023-10-08 05:53:25,738][00612] Updated weights for policy 1, policy_version 50330 (0.0008) [2023-10-08 05:53:27,717][00611] Updated weights for policy 0, policy_version 50052 (0.0010) [2023-10-08 05:53:28,081][00611] Updated weights for policy 0, policy_version 50062 (0.0008) [2023-10-08 05:53:28,453][00611] Updated weights for policy 0, policy_version 50072 (0.0008) [2023-10-08 05:53:28,754][130385] Fps is (10 sec: 16384.6, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 102825984. Throughput: 0: 1847.8, 1: 1860.9. Samples: 25711202. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:53:28,754][130385] Avg episode reward: [(0, '65.150'), (1, '64.170')] [2023-10-08 05:53:29,362][00612] Updated weights for policy 1, policy_version 50340 (0.0008) [2023-10-08 05:53:29,729][00612] Updated weights for policy 1, policy_version 50350 (0.0010) [2023-10-08 05:53:30,099][00612] Updated weights for policy 1, policy_version 50360 (0.0008) [2023-10-08 05:53:32,107][00611] Updated weights for policy 0, policy_version 50082 (0.0007) [2023-10-08 05:53:32,482][00611] Updated weights for policy 0, policy_version 50092 (0.0009) [2023-10-08 05:53:32,854][00611] Updated weights for policy 0, policy_version 50102 (0.0011) [2023-10-08 05:53:33,220][00611] Updated weights for policy 0, policy_version 50112 (0.0007) [2023-10-08 05:53:33,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 102891520. Throughput: 0: 1826.0, 1: 1856.8. Samples: 25732662. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:53:33,755][130385] Avg episode reward: [(0, '60.020'), (1, '63.290')] [2023-10-08 05:53:33,830][00612] Updated weights for policy 1, policy_version 50370 (0.0009) [2023-10-08 05:53:34,198][00612] Updated weights for policy 1, policy_version 50380 (0.0009) [2023-10-08 05:53:34,560][00612] Updated weights for policy 1, policy_version 50390 (0.0010) [2023-10-08 05:53:34,930][00612] Updated weights for policy 1, policy_version 50400 (0.0010) [2023-10-08 05:53:36,882][00611] Updated weights for policy 0, policy_version 50122 (0.0009) [2023-10-08 05:53:37,260][00611] Updated weights for policy 0, policy_version 50132 (0.0008) [2023-10-08 05:53:37,630][00611] Updated weights for policy 0, policy_version 50142 (0.0008) [2023-10-08 05:53:38,515][00612] Updated weights for policy 1, policy_version 50410 (0.0008) [2023-10-08 05:53:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 102957056. Throughput: 0: 1836.4, 1: 1858.8. Samples: 25744104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:53:38,754][130385] Avg episode reward: [(0, '61.340'), (1, '64.750')] [2023-10-08 05:53:38,875][00612] Updated weights for policy 1, policy_version 50420 (0.0009) [2023-10-08 05:53:39,247][00612] Updated weights for policy 1, policy_version 50430 (0.0009) [2023-10-08 05:53:41,259][00611] Updated weights for policy 0, policy_version 50152 (0.0009) [2023-10-08 05:53:41,625][00611] Updated weights for policy 0, policy_version 50162 (0.0010) [2023-10-08 05:53:42,006][00611] Updated weights for policy 0, policy_version 50172 (0.0010) [2023-10-08 05:53:42,892][00612] Updated weights for policy 1, policy_version 50440 (0.0010) [2023-10-08 05:53:43,268][00612] Updated weights for policy 1, policy_version 50450 (0.0010) [2023-10-08 05:53:43,637][00612] Updated weights for policy 1, policy_version 50460 (0.0010) [2023-10-08 05:53:43,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 103022592. Throughput: 0: 1824.3, 1: 1855.2. Samples: 25765828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:53:43,754][130385] Avg episode reward: [(0, '63.040'), (1, '64.090')] [2023-10-08 05:53:45,709][00611] Updated weights for policy 0, policy_version 50182 (0.0010) [2023-10-08 05:53:46,075][00611] Updated weights for policy 0, policy_version 50192 (0.0007) [2023-10-08 05:53:46,446][00611] Updated weights for policy 0, policy_version 50202 (0.0007) [2023-10-08 05:53:47,465][00612] Updated weights for policy 1, policy_version 50470 (0.0008) [2023-10-08 05:53:47,841][00612] Updated weights for policy 1, policy_version 50480 (0.0009) [2023-10-08 05:53:48,215][00612] Updated weights for policy 1, policy_version 50490 (0.0011) [2023-10-08 05:53:48,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 103120896. Throughput: 0: 1835.9, 1: 1830.4. Samples: 25787312. Policy #0 lag: (min: 16.0, avg: 40.5, max: 48.0) [2023-10-08 05:53:48,755][130385] Avg episode reward: [(0, '61.480'), (1, '63.810')] [2023-10-08 05:53:50,184][00611] Updated weights for policy 0, policy_version 50212 (0.0008) [2023-10-08 05:53:50,561][00611] Updated weights for policy 0, policy_version 50222 (0.0008) [2023-10-08 05:53:50,928][00611] Updated weights for policy 0, policy_version 50232 (0.0010) [2023-10-08 05:53:51,844][00612] Updated weights for policy 1, policy_version 50500 (0.0009) [2023-10-08 05:53:52,210][00612] Updated weights for policy 1, policy_version 50510 (0.0008) [2023-10-08 05:53:52,575][00612] Updated weights for policy 1, policy_version 50520 (0.0008) [2023-10-08 05:53:53,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 103186432. Throughput: 0: 1827.5, 1: 1843.6. Samples: 25798782. Policy #0 lag: (min: 16.0, avg: 40.5, max: 48.0) [2023-10-08 05:53:53,755][130385] Avg episode reward: [(0, '63.020'), (1, '66.390')] [2023-10-08 05:53:54,501][00611] Updated weights for policy 0, policy_version 50242 (0.0009) [2023-10-08 05:53:54,878][00611] Updated weights for policy 0, policy_version 50252 (0.0008) [2023-10-08 05:53:55,240][00611] Updated weights for policy 0, policy_version 50262 (0.0008) [2023-10-08 05:53:55,610][00611] Updated weights for policy 0, policy_version 50272 (0.0009) [2023-10-08 05:53:56,208][00612] Updated weights for policy 1, policy_version 50530 (0.0008) [2023-10-08 05:53:56,576][00612] Updated weights for policy 1, policy_version 50540 (0.0007) [2023-10-08 05:53:56,951][00612] Updated weights for policy 1, policy_version 50550 (0.0007) [2023-10-08 05:53:57,320][00612] Updated weights for policy 1, policy_version 50560 (0.0008) [2023-10-08 05:53:58,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 103251968. Throughput: 0: 1834.7, 1: 1833.7. Samples: 25820560. Policy #0 lag: (min: 16.0, avg: 40.5, max: 48.0) [2023-10-08 05:53:58,754][130385] Avg episode reward: [(0, '59.720'), (1, '64.720')] [2023-10-08 05:53:59,287][00611] Updated weights for policy 0, policy_version 50282 (0.0007) [2023-10-08 05:53:59,648][00611] Updated weights for policy 0, policy_version 50292 (0.0007) [2023-10-08 05:54:00,020][00611] Updated weights for policy 0, policy_version 50302 (0.0007) [2023-10-08 05:54:01,056][00612] Updated weights for policy 1, policy_version 50570 (0.0010) [2023-10-08 05:54:01,426][00612] Updated weights for policy 1, policy_version 50580 (0.0009) [2023-10-08 05:54:01,789][00612] Updated weights for policy 1, policy_version 50590 (0.0009) [2023-10-08 05:54:03,725][00611] Updated weights for policy 0, policy_version 50312 (0.0010) [2023-10-08 05:54:03,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 103317504. Throughput: 0: 1837.1, 1: 1845.9. Samples: 25843618. Policy #0 lag: (min: 16.0, avg: 40.5, max: 48.0) [2023-10-08 05:54:03,754][130385] Avg episode reward: [(0, '58.340'), (1, '67.320')] [2023-10-08 05:54:04,089][00611] Updated weights for policy 0, policy_version 50322 (0.0010) [2023-10-08 05:54:04,466][00611] Updated weights for policy 0, policy_version 50332 (0.0007) [2023-10-08 05:54:05,269][00612] Updated weights for policy 1, policy_version 50600 (0.0010) [2023-10-08 05:54:05,634][00612] Updated weights for policy 1, policy_version 50610 (0.0009) [2023-10-08 05:54:06,001][00612] Updated weights for policy 1, policy_version 50620 (0.0009) [2023-10-08 05:54:08,132][00611] Updated weights for policy 0, policy_version 50342 (0.0008) [2023-10-08 05:54:08,511][00611] Updated weights for policy 0, policy_version 50352 (0.0009) [2023-10-08 05:54:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 103383040. Throughput: 0: 1840.8, 1: 1837.2. Samples: 25853936. Policy #0 lag: (min: 16.0, avg: 40.5, max: 48.0) [2023-10-08 05:54:08,754][130385] Avg episode reward: [(0, '61.270'), (1, '67.430')] [2023-10-08 05:54:08,876][00611] Updated weights for policy 0, policy_version 50362 (0.0009) [2023-10-08 05:54:09,602][00612] Updated weights for policy 1, policy_version 50630 (0.0010) [2023-10-08 05:54:09,975][00612] Updated weights for policy 1, policy_version 50640 (0.0008) [2023-10-08 05:54:10,333][00612] Updated weights for policy 1, policy_version 50650 (0.0008) [2023-10-08 05:54:12,441][00611] Updated weights for policy 0, policy_version 50372 (0.0009) [2023-10-08 05:54:12,815][00611] Updated weights for policy 0, policy_version 50382 (0.0009) [2023-10-08 05:54:13,181][00611] Updated weights for policy 0, policy_version 50392 (0.0008) [2023-10-08 05:54:13,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 103481344. Throughput: 0: 1834.2, 1: 1850.0. Samples: 25876990. Policy #0 lag: (min: 16.0, avg: 40.5, max: 48.0) [2023-10-08 05:54:13,755][130385] Avg episode reward: [(0, '60.980'), (1, '69.920')] [2023-10-08 05:54:13,801][00612] Updated weights for policy 1, policy_version 50660 (0.0010) [2023-10-08 05:54:14,179][00612] Updated weights for policy 1, policy_version 50670 (0.0010) [2023-10-08 05:54:14,544][00612] Updated weights for policy 1, policy_version 50680 (0.0010) [2023-10-08 05:54:16,917][00611] Updated weights for policy 0, policy_version 50402 (0.0008) [2023-10-08 05:54:17,285][00611] Updated weights for policy 0, policy_version 50412 (0.0011) [2023-10-08 05:54:17,642][00611] Updated weights for policy 0, policy_version 50422 (0.0009) [2023-10-08 05:54:18,014][00611] Updated weights for policy 0, policy_version 50432 (0.0010) [2023-10-08 05:54:18,040][00612] Updated weights for policy 1, policy_version 50690 (0.0010) [2023-10-08 05:54:18,413][00612] Updated weights for policy 1, policy_version 50700 (0.0010) [2023-10-08 05:54:18,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 103546880. Throughput: 0: 1830.7, 1: 1851.7. Samples: 25898366. Policy #0 lag: (min: 16.0, avg: 40.5, max: 48.0) [2023-10-08 05:54:18,754][130385] Avg episode reward: [(0, '60.130'), (1, '71.320')] [2023-10-08 05:54:18,762][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000050432_51642368.pth... [2023-10-08 05:54:18,777][00612] Updated weights for policy 1, policy_version 50710 (0.0008) [2023-10-08 05:54:18,797][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000048704_49872896.pth [2023-10-08 05:54:19,141][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000050720_51937280.pth... [2023-10-08 05:54:19,142][00612] Updated weights for policy 1, policy_version 50720 (0.0008) [2023-10-08 05:54:19,169][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000048992_50167808.pth [2023-10-08 05:54:21,698][00611] Updated weights for policy 0, policy_version 50442 (0.0009) [2023-10-08 05:54:22,072][00611] Updated weights for policy 0, policy_version 50452 (0.0008) [2023-10-08 05:54:22,432][00611] Updated weights for policy 0, policy_version 50462 (0.0007) [2023-10-08 05:54:22,597][00612] Updated weights for policy 1, policy_version 50730 (0.0008) [2023-10-08 05:54:22,975][00612] Updated weights for policy 1, policy_version 50740 (0.0009) [2023-10-08 05:54:23,355][00612] Updated weights for policy 1, policy_version 50750 (0.0009) [2023-10-08 05:54:23,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 103645184. Throughput: 0: 1833.2, 1: 1860.4. Samples: 25910316. Policy #0 lag: (min: 3.0, avg: 3.0, max: 6.0) [2023-10-08 05:54:23,755][130385] Avg episode reward: [(0, '62.340'), (1, '69.410')] [2023-10-08 05:54:26,183][00611] Updated weights for policy 0, policy_version 50472 (0.0008) [2023-10-08 05:54:26,559][00611] Updated weights for policy 0, policy_version 50482 (0.0010) [2023-10-08 05:54:26,928][00611] Updated weights for policy 0, policy_version 50492 (0.0007) [2023-10-08 05:54:26,965][00612] Updated weights for policy 1, policy_version 50760 (0.0007) [2023-10-08 05:54:27,324][00612] Updated weights for policy 1, policy_version 50770 (0.0009) [2023-10-08 05:54:27,704][00612] Updated weights for policy 1, policy_version 50780 (0.0007) [2023-10-08 05:54:28,754][130385] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 103710720. Throughput: 0: 1838.3, 1: 1846.5. Samples: 25931648. Policy #0 lag: (min: 3.0, avg: 3.0, max: 6.0) [2023-10-08 05:54:28,755][130385] Avg episode reward: [(0, '62.930'), (1, '69.760')] [2023-10-08 05:54:30,556][00611] Updated weights for policy 0, policy_version 50502 (0.0009) [2023-10-08 05:54:30,913][00611] Updated weights for policy 0, policy_version 50512 (0.0009) [2023-10-08 05:54:31,296][00611] Updated weights for policy 0, policy_version 50522 (0.0009) [2023-10-08 05:54:31,342][00612] Updated weights for policy 1, policy_version 50790 (0.0007) [2023-10-08 05:54:31,702][00612] Updated weights for policy 1, policy_version 50800 (0.0008) [2023-10-08 05:54:32,076][00612] Updated weights for policy 1, policy_version 50810 (0.0008) [2023-10-08 05:54:33,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 103776256. Throughput: 0: 1840.0, 1: 1856.6. Samples: 25953660. Policy #0 lag: (min: 3.0, avg: 3.0, max: 6.0) [2023-10-08 05:54:33,755][130385] Avg episode reward: [(0, '61.460'), (1, '71.510')] [2023-10-08 05:54:34,836][00611] Updated weights for policy 0, policy_version 50532 (0.0009) [2023-10-08 05:54:35,205][00611] Updated weights for policy 0, policy_version 50542 (0.0009) [2023-10-08 05:54:35,577][00611] Updated weights for policy 0, policy_version 50552 (0.0009) [2023-10-08 05:54:35,879][00612] Updated weights for policy 1, policy_version 50820 (0.0009) [2023-10-08 05:54:36,254][00612] Updated weights for policy 1, policy_version 50830 (0.0007) [2023-10-08 05:54:36,617][00612] Updated weights for policy 1, policy_version 50840 (0.0007) [2023-10-08 05:54:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 103841792. Throughput: 0: 1836.4, 1: 1844.9. Samples: 25964442. Policy #0 lag: (min: 3.0, avg: 3.0, max: 6.0) [2023-10-08 05:54:38,755][130385] Avg episode reward: [(0, '59.150'), (1, '70.010')] [2023-10-08 05:54:39,248][00611] Updated weights for policy 0, policy_version 50562 (0.0009) [2023-10-08 05:54:39,616][00611] Updated weights for policy 0, policy_version 50572 (0.0007) [2023-10-08 05:54:39,988][00611] Updated weights for policy 0, policy_version 50582 (0.0008) [2023-10-08 05:54:40,076][00612] Updated weights for policy 1, policy_version 50850 (0.0007) [2023-10-08 05:54:40,359][00611] Updated weights for policy 0, policy_version 50592 (0.0010) [2023-10-08 05:54:40,451][00612] Updated weights for policy 1, policy_version 50860 (0.0007) [2023-10-08 05:54:40,815][00612] Updated weights for policy 1, policy_version 50870 (0.0007) [2023-10-08 05:54:41,178][00612] Updated weights for policy 1, policy_version 50880 (0.0007) [2023-10-08 05:54:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 103907328. Throughput: 0: 1843.7, 1: 1856.2. Samples: 25987056. Policy #0 lag: (min: 3.0, avg: 3.0, max: 6.0) [2023-10-08 05:54:43,754][130385] Avg episode reward: [(0, '58.720'), (1, '67.340')] [2023-10-08 05:54:43,914][00611] Updated weights for policy 0, policy_version 50602 (0.0008) [2023-10-08 05:54:44,289][00611] Updated weights for policy 0, policy_version 50612 (0.0008) [2023-10-08 05:54:44,657][00611] Updated weights for policy 0, policy_version 50622 (0.0008) [2023-10-08 05:54:44,753][00612] Updated weights for policy 1, policy_version 50890 (0.0008) [2023-10-08 05:54:45,121][00612] Updated weights for policy 1, policy_version 50900 (0.0007) [2023-10-08 05:54:45,494][00612] Updated weights for policy 1, policy_version 50910 (0.0009) [2023-10-08 05:54:48,183][00611] Updated weights for policy 0, policy_version 50632 (0.0008) [2023-10-08 05:54:48,557][00611] Updated weights for policy 0, policy_version 50642 (0.0007) [2023-10-08 05:54:48,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 103972864. Throughput: 0: 1837.7, 1: 1859.0. Samples: 26009970. Policy #0 lag: (min: 3.0, avg: 3.0, max: 6.0) [2023-10-08 05:54:48,755][130385] Avg episode reward: [(0, '61.930'), (1, '64.640')] [2023-10-08 05:54:48,930][00611] Updated weights for policy 0, policy_version 50652 (0.0007) [2023-10-08 05:54:49,343][00612] Updated weights for policy 1, policy_version 50920 (0.0008) [2023-10-08 05:54:49,711][00612] Updated weights for policy 1, policy_version 50930 (0.0008) [2023-10-08 05:54:50,067][00612] Updated weights for policy 1, policy_version 50940 (0.0008) [2023-10-08 05:54:52,777][00611] Updated weights for policy 0, policy_version 50662 (0.0007) [2023-10-08 05:54:53,161][00611] Updated weights for policy 0, policy_version 50672 (0.0008) [2023-10-08 05:54:53,531][00611] Updated weights for policy 0, policy_version 50682 (0.0007) [2023-10-08 05:54:53,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 104071168. Throughput: 0: 1841.2, 1: 1850.9. Samples: 26020080. Policy #0 lag: (min: 3.0, avg: 3.0, max: 6.0) [2023-10-08 05:54:53,755][00612] Updated weights for policy 1, policy_version 50950 (0.0009) [2023-10-08 05:54:53,755][130385] Avg episode reward: [(0, '63.080'), (1, '66.050')] [2023-10-08 05:54:54,128][00612] Updated weights for policy 1, policy_version 50960 (0.0008) [2023-10-08 05:54:54,505][00612] Updated weights for policy 1, policy_version 50970 (0.0009) [2023-10-08 05:54:57,343][00611] Updated weights for policy 0, policy_version 50692 (0.0009) [2023-10-08 05:54:57,713][00611] Updated weights for policy 0, policy_version 50702 (0.0010) [2023-10-08 05:54:58,090][00611] Updated weights for policy 0, policy_version 50712 (0.0009) [2023-10-08 05:54:58,233][00612] Updated weights for policy 1, policy_version 50980 (0.0007) [2023-10-08 05:54:58,597][00612] Updated weights for policy 1, policy_version 50990 (0.0007) [2023-10-08 05:54:58,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 104136704. Throughput: 0: 1837.3, 1: 1846.5. Samples: 26042758. Policy #0 lag: (min: 31.0, avg: 31.4, max: 43.0) [2023-10-08 05:54:58,754][130385] Avg episode reward: [(0, '62.920'), (1, '65.920')] [2023-10-08 05:54:58,971][00612] Updated weights for policy 1, policy_version 51000 (0.0008) [2023-10-08 05:55:01,628][00611] Updated weights for policy 0, policy_version 50722 (0.0008) [2023-10-08 05:55:01,990][00611] Updated weights for policy 0, policy_version 50732 (0.0008) [2023-10-08 05:55:02,359][00611] Updated weights for policy 0, policy_version 50742 (0.0008) [2023-10-08 05:55:02,568][00612] Updated weights for policy 1, policy_version 51010 (0.0009) [2023-10-08 05:55:02,735][00611] Updated weights for policy 0, policy_version 50752 (0.0010) [2023-10-08 05:55:02,944][00612] Updated weights for policy 1, policy_version 51020 (0.0008) [2023-10-08 05:55:03,314][00612] Updated weights for policy 1, policy_version 51030 (0.0007) [2023-10-08 05:55:03,682][00612] Updated weights for policy 1, policy_version 51040 (0.0008) [2023-10-08 05:55:03,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 104235008. Throughput: 0: 1844.3, 1: 1829.5. Samples: 26063684. Policy #0 lag: (min: 31.0, avg: 31.4, max: 43.0) [2023-10-08 05:55:03,755][130385] Avg episode reward: [(0, '61.690'), (1, '65.500')] [2023-10-08 05:55:06,458][00611] Updated weights for policy 0, policy_version 50762 (0.0010) [2023-10-08 05:55:06,837][00611] Updated weights for policy 0, policy_version 50772 (0.0010) [2023-10-08 05:55:07,214][00611] Updated weights for policy 0, policy_version 50782 (0.0008) [2023-10-08 05:55:07,371][00612] Updated weights for policy 1, policy_version 51050 (0.0007) [2023-10-08 05:55:07,735][00612] Updated weights for policy 1, policy_version 51060 (0.0008) [2023-10-08 05:55:08,109][00612] Updated weights for policy 1, policy_version 51070 (0.0008) [2023-10-08 05:55:08,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 104300544. Throughput: 0: 1837.0, 1: 1839.3. Samples: 26075750. Policy #0 lag: (min: 31.0, avg: 31.4, max: 43.0) [2023-10-08 05:55:08,755][130385] Avg episode reward: [(0, '63.060'), (1, '64.410')] [2023-10-08 05:55:10,800][00611] Updated weights for policy 0, policy_version 50792 (0.0008) [2023-10-08 05:55:11,169][00611] Updated weights for policy 0, policy_version 50802 (0.0009) [2023-10-08 05:55:11,547][00611] Updated weights for policy 0, policy_version 50812 (0.0009) [2023-10-08 05:55:11,826][00612] Updated weights for policy 1, policy_version 51080 (0.0007) [2023-10-08 05:55:12,192][00612] Updated weights for policy 1, policy_version 51090 (0.0007) [2023-10-08 05:55:12,576][00612] Updated weights for policy 1, policy_version 51100 (0.0008) [2023-10-08 05:55:13,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 104366080. Throughput: 0: 1833.3, 1: 1831.9. Samples: 26096584. Policy #0 lag: (min: 31.0, avg: 31.4, max: 43.0) [2023-10-08 05:55:13,754][130385] Avg episode reward: [(0, '65.410'), (1, '67.310')] [2023-10-08 05:55:15,114][00611] Updated weights for policy 0, policy_version 50822 (0.0008) [2023-10-08 05:55:15,491][00611] Updated weights for policy 0, policy_version 50832 (0.0009) [2023-10-08 05:55:15,856][00611] Updated weights for policy 0, policy_version 50842 (0.0008) [2023-10-08 05:55:16,042][00612] Updated weights for policy 1, policy_version 51110 (0.0010) [2023-10-08 05:55:16,413][00612] Updated weights for policy 1, policy_version 51120 (0.0010) [2023-10-08 05:55:16,787][00612] Updated weights for policy 1, policy_version 51130 (0.0007) [2023-10-08 05:55:18,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 104431616. Throughput: 0: 1833.6, 1: 1842.2. Samples: 26119070. Policy #0 lag: (min: 31.0, avg: 31.4, max: 43.0) [2023-10-08 05:55:18,754][130385] Avg episode reward: [(0, '62.750'), (1, '65.960')] [2023-10-08 05:55:19,587][00611] Updated weights for policy 0, policy_version 50852 (0.0009) [2023-10-08 05:55:19,973][00611] Updated weights for policy 0, policy_version 50862 (0.0011) [2023-10-08 05:55:20,345][00611] Updated weights for policy 0, policy_version 50872 (0.0010) [2023-10-08 05:55:20,402][00612] Updated weights for policy 1, policy_version 51140 (0.0007) [2023-10-08 05:55:20,765][00612] Updated weights for policy 1, policy_version 51150 (0.0008) [2023-10-08 05:55:21,137][00612] Updated weights for policy 1, policy_version 51160 (0.0008) [2023-10-08 05:55:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 104497152. Throughput: 0: 1827.6, 1: 1835.1. Samples: 26129264. Policy #0 lag: (min: 31.0, avg: 31.4, max: 43.0) [2023-10-08 05:55:23,754][130385] Avg episode reward: [(0, '63.150'), (1, '66.850')] [2023-10-08 05:55:23,964][00611] Updated weights for policy 0, policy_version 50882 (0.0009) [2023-10-08 05:55:24,341][00611] Updated weights for policy 0, policy_version 50892 (0.0007) [2023-10-08 05:55:24,707][00612] Updated weights for policy 1, policy_version 51170 (0.0008) [2023-10-08 05:55:24,717][00611] Updated weights for policy 0, policy_version 50902 (0.0007) [2023-10-08 05:55:25,069][00612] Updated weights for policy 1, policy_version 51180 (0.0007) [2023-10-08 05:55:25,091][00611] Updated weights for policy 0, policy_version 50912 (0.0008) [2023-10-08 05:55:25,434][00612] Updated weights for policy 1, policy_version 51190 (0.0009) [2023-10-08 05:55:25,807][00612] Updated weights for policy 1, policy_version 51200 (0.0010) [2023-10-08 05:55:28,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 104562688. Throughput: 0: 1818.4, 1: 1847.1. Samples: 26152004. Policy #0 lag: (min: 31.0, avg: 31.4, max: 43.0) [2023-10-08 05:55:28,755][130385] Avg episode reward: [(0, '62.380'), (1, '65.510')] [2023-10-08 05:55:28,966][00611] Updated weights for policy 0, policy_version 50922 (0.0008) [2023-10-08 05:55:29,337][00611] Updated weights for policy 0, policy_version 50932 (0.0008) [2023-10-08 05:55:29,471][00612] Updated weights for policy 1, policy_version 51210 (0.0007) [2023-10-08 05:55:29,714][00611] Updated weights for policy 0, policy_version 50942 (0.0008) [2023-10-08 05:55:29,838][00612] Updated weights for policy 1, policy_version 51220 (0.0007) [2023-10-08 05:55:30,212][00612] Updated weights for policy 1, policy_version 51230 (0.0008) [2023-10-08 05:55:33,372][00611] Updated weights for policy 0, policy_version 50952 (0.0008) [2023-10-08 05:55:33,741][00611] Updated weights for policy 0, policy_version 50962 (0.0008) [2023-10-08 05:55:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 104628224. Throughput: 0: 1814.0, 1: 1848.2. Samples: 26174768. Policy #0 lag: (min: 31.0, avg: 31.4, max: 43.0) [2023-10-08 05:55:33,754][130385] Avg episode reward: [(0, '62.720'), (1, '65.300')] [2023-10-08 05:55:33,832][00612] Updated weights for policy 1, policy_version 51240 (0.0008) [2023-10-08 05:55:34,112][00611] Updated weights for policy 0, policy_version 50972 (0.0008) [2023-10-08 05:55:34,185][00612] Updated weights for policy 1, policy_version 51250 (0.0008) [2023-10-08 05:55:34,558][00612] Updated weights for policy 1, policy_version 51260 (0.0010) [2023-10-08 05:55:37,802][00611] Updated weights for policy 0, policy_version 50982 (0.0008) [2023-10-08 05:55:38,178][00611] Updated weights for policy 0, policy_version 50992 (0.0007) [2023-10-08 05:55:38,310][00612] Updated weights for policy 1, policy_version 51270 (0.0008) [2023-10-08 05:55:38,554][00611] Updated weights for policy 0, policy_version 51002 (0.0008) [2023-10-08 05:55:38,685][00612] Updated weights for policy 1, policy_version 51280 (0.0008) [2023-10-08 05:55:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 104693760. Throughput: 0: 1806.3, 1: 1853.6. Samples: 26184774. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-08 05:55:38,755][130385] Avg episode reward: [(0, '63.670'), (1, '64.290')] [2023-10-08 05:55:39,052][00612] Updated weights for policy 1, policy_version 51290 (0.0007) [2023-10-08 05:55:42,455][00611] Updated weights for policy 0, policy_version 51012 (0.0008) [2023-10-08 05:55:42,612][00612] Updated weights for policy 1, policy_version 51300 (0.0008) [2023-10-08 05:55:42,831][00611] Updated weights for policy 0, policy_version 51022 (0.0007) [2023-10-08 05:55:42,981][00612] Updated weights for policy 1, policy_version 51310 (0.0009) [2023-10-08 05:55:43,206][00611] Updated weights for policy 0, policy_version 51032 (0.0009) [2023-10-08 05:55:43,347][00612] Updated weights for policy 1, policy_version 51320 (0.0007) [2023-10-08 05:55:43,754][130385] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14884.5). Total num frames: 104824832. Throughput: 0: 1806.4, 1: 1854.9. Samples: 26207518. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-08 05:55:43,755][130385] Avg episode reward: [(0, '68.410'), (1, '67.080')] [2023-10-08 05:55:46,810][00611] Updated weights for policy 0, policy_version 51042 (0.0008) [2023-10-08 05:55:47,017][00612] Updated weights for policy 1, policy_version 51330 (0.0007) [2023-10-08 05:55:47,174][00611] Updated weights for policy 0, policy_version 51052 (0.0009) [2023-10-08 05:55:47,382][00612] Updated weights for policy 1, policy_version 51340 (0.0007) [2023-10-08 05:55:47,541][00611] Updated weights for policy 0, policy_version 51062 (0.0010) [2023-10-08 05:55:47,754][00612] Updated weights for policy 1, policy_version 51350 (0.0008) [2023-10-08 05:55:47,908][00611] Updated weights for policy 0, policy_version 51072 (0.0009) [2023-10-08 05:55:48,116][00612] Updated weights for policy 1, policy_version 51360 (0.0008) [2023-10-08 05:55:48,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 104890368. Throughput: 0: 1804.6, 1: 1838.0. Samples: 26227604. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-08 05:55:48,755][130385] Avg episode reward: [(0, '66.950'), (1, '65.150')] [2023-10-08 05:55:51,603][00612] Updated weights for policy 1, policy_version 51370 (0.0007) [2023-10-08 05:55:51,634][00611] Updated weights for policy 0, policy_version 51082 (0.0007) [2023-10-08 05:55:51,966][00612] Updated weights for policy 1, policy_version 51380 (0.0009) [2023-10-08 05:55:52,015][00611] Updated weights for policy 0, policy_version 51092 (0.0007) [2023-10-08 05:55:52,331][00612] Updated weights for policy 1, policy_version 51390 (0.0009) [2023-10-08 05:55:52,376][00611] Updated weights for policy 0, policy_version 51102 (0.0007) [2023-10-08 05:55:53,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 104955904. Throughput: 0: 1804.5, 1: 1855.4. Samples: 26240446. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-08 05:55:53,754][130385] Avg episode reward: [(0, '67.330'), (1, '64.130')] [2023-10-08 05:55:56,017][00612] Updated weights for policy 1, policy_version 51400 (0.0008) [2023-10-08 05:55:56,145][00611] Updated weights for policy 0, policy_version 51112 (0.0009) [2023-10-08 05:55:56,396][00612] Updated weights for policy 1, policy_version 51410 (0.0009) [2023-10-08 05:55:56,519][00611] Updated weights for policy 0, policy_version 51122 (0.0009) [2023-10-08 05:55:56,757][00612] Updated weights for policy 1, policy_version 51420 (0.0007) [2023-10-08 05:55:56,892][00611] Updated weights for policy 0, policy_version 51132 (0.0007) [2023-10-08 05:55:58,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 105021440. Throughput: 0: 1800.9, 1: 1834.0. Samples: 26260154. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-08 05:55:58,754][130385] Avg episode reward: [(0, '70.110'), (1, '63.230')] [2023-10-08 05:56:00,332][00612] Updated weights for policy 1, policy_version 51430 (0.0007) [2023-10-08 05:56:00,508][00611] Updated weights for policy 0, policy_version 51142 (0.0009) [2023-10-08 05:56:00,703][00612] Updated weights for policy 1, policy_version 51440 (0.0008) [2023-10-08 05:56:00,875][00611] Updated weights for policy 0, policy_version 51152 (0.0007) [2023-10-08 05:56:01,069][00612] Updated weights for policy 1, policy_version 51450 (0.0007) [2023-10-08 05:56:01,257][00611] Updated weights for policy 0, policy_version 51162 (0.0008) [2023-10-08 05:56:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 105086976. Throughput: 0: 1808.2, 1: 1849.3. Samples: 26283660. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-08 05:56:03,754][130385] Avg episode reward: [(0, '64.100'), (1, '60.600')] [2023-10-08 05:56:04,803][00612] Updated weights for policy 1, policy_version 51460 (0.0008) [2023-10-08 05:56:04,912][00611] Updated weights for policy 0, policy_version 51172 (0.0008) [2023-10-08 05:56:05,167][00612] Updated weights for policy 1, policy_version 51470 (0.0008) [2023-10-08 05:56:05,272][00611] Updated weights for policy 0, policy_version 51182 (0.0010) [2023-10-08 05:56:05,533][00612] Updated weights for policy 1, policy_version 51480 (0.0010) [2023-10-08 05:56:05,643][00611] Updated weights for policy 0, policy_version 51192 (0.0010) [2023-10-08 05:56:08,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 105152512. Throughput: 0: 1810.9, 1: 1839.0. Samples: 26293512. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-08 05:56:08,755][130385] Avg episode reward: [(0, '63.160'), (1, '60.900')] [2023-10-08 05:56:09,254][00611] Updated weights for policy 0, policy_version 51202 (0.0008) [2023-10-08 05:56:09,286][00612] Updated weights for policy 1, policy_version 51490 (0.0010) [2023-10-08 05:56:09,628][00611] Updated weights for policy 0, policy_version 51212 (0.0008) [2023-10-08 05:56:09,644][00612] Updated weights for policy 1, policy_version 51500 (0.0007) [2023-10-08 05:56:10,000][00611] Updated weights for policy 0, policy_version 51222 (0.0007) [2023-10-08 05:56:10,019][00612] Updated weights for policy 1, policy_version 51510 (0.0009) [2023-10-08 05:56:10,368][00611] Updated weights for policy 0, policy_version 51232 (0.0008) [2023-10-08 05:56:10,376][00612] Updated weights for policy 1, policy_version 51520 (0.0008) [2023-10-08 05:56:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 105218048. Throughput: 0: 1819.2, 1: 1843.2. Samples: 26316812. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 05:56:13,755][130385] Avg episode reward: [(0, '67.140'), (1, '59.610')] [2023-10-08 05:56:13,976][00612] Updated weights for policy 1, policy_version 51530 (0.0007) [2023-10-08 05:56:14,233][00611] Updated weights for policy 0, policy_version 51242 (0.0008) [2023-10-08 05:56:14,352][00612] Updated weights for policy 1, policy_version 51540 (0.0008) [2023-10-08 05:56:14,604][00611] Updated weights for policy 0, policy_version 51252 (0.0009) [2023-10-08 05:56:14,714][00612] Updated weights for policy 1, policy_version 51550 (0.0007) [2023-10-08 05:56:14,974][00611] Updated weights for policy 0, policy_version 51262 (0.0008) [2023-10-08 05:56:18,487][00612] Updated weights for policy 1, policy_version 51560 (0.0008) [2023-10-08 05:56:18,586][00611] Updated weights for policy 0, policy_version 51272 (0.0007) [2023-10-08 05:56:18,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 105283584. Throughput: 0: 1819.7, 1: 1841.0. Samples: 26339500. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 05:56:18,754][130385] Avg episode reward: [(0, '66.880'), (1, '59.650')] [2023-10-08 05:56:18,856][00612] Updated weights for policy 1, policy_version 51570 (0.0007) [2023-10-08 05:56:18,961][00611] Updated weights for policy 0, policy_version 51282 (0.0008) [2023-10-08 05:56:19,228][00612] Updated weights for policy 1, policy_version 51580 (0.0009) [2023-10-08 05:56:19,342][00611] Updated weights for policy 0, policy_version 51292 (0.0007) [2023-10-08 05:56:19,369][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000051584_52822016.pth... [2023-10-08 05:56:19,397][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000049856_51052544.pth [2023-10-08 05:56:19,401][00425] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p1/milestones/checkpoint_000051584_52822016.pth [2023-10-08 05:56:19,481][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000051296_52527104.pth... [2023-10-08 05:56:19,510][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000049568_50757632.pth [2023-10-08 05:56:19,515][00365] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p0/milestones/checkpoint_000051296_52527104.pth [2023-10-08 05:56:22,868][00612] Updated weights for policy 1, policy_version 51590 (0.0008) [2023-10-08 05:56:23,042][00611] Updated weights for policy 0, policy_version 51302 (0.0008) [2023-10-08 05:56:23,240][00612] Updated weights for policy 1, policy_version 51600 (0.0009) [2023-10-08 05:56:23,408][00611] Updated weights for policy 0, policy_version 51312 (0.0008) [2023-10-08 05:56:23,605][00612] Updated weights for policy 1, policy_version 51610 (0.0008) [2023-10-08 05:56:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 105349120. Throughput: 0: 1817.0, 1: 1836.9. Samples: 26349202. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 05:56:23,754][130385] Avg episode reward: [(0, '66.510'), (1, '58.230')] [2023-10-08 05:56:23,784][00611] Updated weights for policy 0, policy_version 51322 (0.0009) [2023-10-08 05:56:27,305][00612] Updated weights for policy 1, policy_version 51620 (0.0009) [2023-10-08 05:56:27,459][00611] Updated weights for policy 0, policy_version 51332 (0.0009) [2023-10-08 05:56:27,683][00612] Updated weights for policy 1, policy_version 51630 (0.0007) [2023-10-08 05:56:27,835][00611] Updated weights for policy 0, policy_version 51342 (0.0009) [2023-10-08 05:56:28,041][00612] Updated weights for policy 1, policy_version 51640 (0.0009) [2023-10-08 05:56:28,203][00611] Updated weights for policy 0, policy_version 51352 (0.0010) [2023-10-08 05:56:28,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 105480192. Throughput: 0: 1819.4, 1: 1835.3. Samples: 26371978. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 05:56:28,754][130385] Avg episode reward: [(0, '66.270'), (1, '59.130')] [2023-10-08 05:56:31,612][00612] Updated weights for policy 1, policy_version 51650 (0.0009) [2023-10-08 05:56:31,933][00611] Updated weights for policy 0, policy_version 51362 (0.0009) [2023-10-08 05:56:31,967][00612] Updated weights for policy 1, policy_version 51660 (0.0008) [2023-10-08 05:56:32,303][00611] Updated weights for policy 0, policy_version 51372 (0.0007) [2023-10-08 05:56:32,339][00612] Updated weights for policy 1, policy_version 51670 (0.0007) [2023-10-08 05:56:32,674][00611] Updated weights for policy 0, policy_version 51382 (0.0008) [2023-10-08 05:56:32,708][00612] Updated weights for policy 1, policy_version 51680 (0.0007) [2023-10-08 05:56:33,051][00611] Updated weights for policy 0, policy_version 51392 (0.0011) [2023-10-08 05:56:33,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 105545728. Throughput: 0: 1813.6, 1: 1836.0. Samples: 26391836. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 05:56:33,754][130385] Avg episode reward: [(0, '69.980'), (1, '60.690')] [2023-10-08 05:56:36,317][00612] Updated weights for policy 1, policy_version 51690 (0.0009) [2023-10-08 05:56:36,670][00611] Updated weights for policy 0, policy_version 51402 (0.0008) [2023-10-08 05:56:36,684][00612] Updated weights for policy 1, policy_version 51700 (0.0007) [2023-10-08 05:56:37,029][00611] Updated weights for policy 0, policy_version 51412 (0.0007) [2023-10-08 05:56:37,047][00612] Updated weights for policy 1, policy_version 51710 (0.0008) [2023-10-08 05:56:37,400][00611] Updated weights for policy 0, policy_version 51422 (0.0009) [2023-10-08 05:56:38,754][130385] Fps is (10 sec: 13106.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 105611264. Throughput: 0: 1820.6, 1: 1826.0. Samples: 26404546. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 05:56:38,755][130385] Avg episode reward: [(0, '69.940'), (1, '61.610')] [2023-10-08 05:56:40,631][00612] Updated weights for policy 1, policy_version 51720 (0.0008) [2023-10-08 05:56:40,999][00612] Updated weights for policy 1, policy_version 51730 (0.0007) [2023-10-08 05:56:41,182][00611] Updated weights for policy 0, policy_version 51432 (0.0009) [2023-10-08 05:56:41,365][00612] Updated weights for policy 1, policy_version 51740 (0.0008) [2023-10-08 05:56:41,558][00611] Updated weights for policy 0, policy_version 51442 (0.0009) [2023-10-08 05:56:41,942][00611] Updated weights for policy 0, policy_version 51452 (0.0008) [2023-10-08 05:56:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 105676800. Throughput: 0: 1816.4, 1: 1841.3. Samples: 26424750. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 05:56:43,754][130385] Avg episode reward: [(0, '70.330'), (1, '60.940')] [2023-10-08 05:56:44,995][00612] Updated weights for policy 1, policy_version 51750 (0.0007) [2023-10-08 05:56:45,361][00612] Updated weights for policy 1, policy_version 51760 (0.0009) [2023-10-08 05:56:45,677][00611] Updated weights for policy 0, policy_version 51462 (0.0010) [2023-10-08 05:56:45,743][00612] Updated weights for policy 1, policy_version 51770 (0.0007) [2023-10-08 05:56:46,049][00611] Updated weights for policy 0, policy_version 51472 (0.0009) [2023-10-08 05:56:46,416][00611] Updated weights for policy 0, policy_version 51482 (0.0008) [2023-10-08 05:56:48,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 105742336. Throughput: 0: 1810.6, 1: 1831.8. Samples: 26447568. Policy #0 lag: (min: 11.0, avg: 17.3, max: 43.0) [2023-10-08 05:56:48,754][130385] Avg episode reward: [(0, '69.570'), (1, '61.100')] [2023-10-08 05:56:49,402][00612] Updated weights for policy 1, policy_version 51780 (0.0010) [2023-10-08 05:56:49,781][00612] Updated weights for policy 1, policy_version 51790 (0.0008) [2023-10-08 05:56:50,005][00611] Updated weights for policy 0, policy_version 51492 (0.0007) [2023-10-08 05:56:50,148][00612] Updated weights for policy 1, policy_version 51800 (0.0007) [2023-10-08 05:56:50,374][00611] Updated weights for policy 0, policy_version 51502 (0.0008) [2023-10-08 05:56:50,759][00611] Updated weights for policy 0, policy_version 51512 (0.0009) [2023-10-08 05:56:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 105807872. Throughput: 0: 1814.2, 1: 1834.9. Samples: 26457720. Policy #0 lag: (min: 11.0, avg: 17.3, max: 43.0) [2023-10-08 05:56:53,754][130385] Avg episode reward: [(0, '66.420'), (1, '61.620')] [2023-10-08 05:56:53,850][00612] Updated weights for policy 1, policy_version 51810 (0.0008) [2023-10-08 05:56:54,218][00612] Updated weights for policy 1, policy_version 51820 (0.0011) [2023-10-08 05:56:54,464][00611] Updated weights for policy 0, policy_version 51522 (0.0008) [2023-10-08 05:56:54,585][00612] Updated weights for policy 1, policy_version 51830 (0.0008) [2023-10-08 05:56:54,844][00611] Updated weights for policy 0, policy_version 51532 (0.0008) [2023-10-08 05:56:54,960][00612] Updated weights for policy 1, policy_version 51840 (0.0007) [2023-10-08 05:56:55,221][00611] Updated weights for policy 0, policy_version 51542 (0.0010) [2023-10-08 05:56:55,586][00611] Updated weights for policy 0, policy_version 51552 (0.0009) [2023-10-08 05:56:58,548][00612] Updated weights for policy 1, policy_version 51850 (0.0009) [2023-10-08 05:56:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 105873408. Throughput: 0: 1806.0, 1: 1839.2. Samples: 26480850. Policy #0 lag: (min: 11.0, avg: 17.3, max: 43.0) [2023-10-08 05:56:58,754][130385] Avg episode reward: [(0, '64.230'), (1, '65.050')] [2023-10-08 05:56:58,909][00612] Updated weights for policy 1, policy_version 51860 (0.0009) [2023-10-08 05:56:59,283][00612] Updated weights for policy 1, policy_version 51870 (0.0008) [2023-10-08 05:56:59,338][00611] Updated weights for policy 0, policy_version 51562 (0.0008) [2023-10-08 05:56:59,700][00611] Updated weights for policy 0, policy_version 51572 (0.0011) [2023-10-08 05:57:00,082][00611] Updated weights for policy 0, policy_version 51582 (0.0010) [2023-10-08 05:57:02,948][00612] Updated weights for policy 1, policy_version 51880 (0.0009) [2023-10-08 05:57:03,313][00612] Updated weights for policy 1, policy_version 51890 (0.0010) [2023-10-08 05:57:03,606][00611] Updated weights for policy 0, policy_version 51592 (0.0007) [2023-10-08 05:57:03,678][00612] Updated weights for policy 1, policy_version 51900 (0.0008) [2023-10-08 05:57:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 105938944. Throughput: 0: 1815.2, 1: 1819.2. Samples: 26503046. Policy #0 lag: (min: 11.0, avg: 17.3, max: 43.0) [2023-10-08 05:57:03,754][130385] Avg episode reward: [(0, '67.450'), (1, '67.330')] [2023-10-08 05:57:03,975][00611] Updated weights for policy 0, policy_version 51602 (0.0008) [2023-10-08 05:57:04,345][00611] Updated weights for policy 0, policy_version 51612 (0.0007) [2023-10-08 05:57:07,301][00612] Updated weights for policy 1, policy_version 51910 (0.0008) [2023-10-08 05:57:07,670][00612] Updated weights for policy 1, policy_version 51920 (0.0008) [2023-10-08 05:57:08,015][00611] Updated weights for policy 0, policy_version 51622 (0.0008) [2023-10-08 05:57:08,038][00612] Updated weights for policy 1, policy_version 51930 (0.0008) [2023-10-08 05:57:08,376][00611] Updated weights for policy 0, policy_version 51632 (0.0008) [2023-10-08 05:57:08,750][00611] Updated weights for policy 0, policy_version 51642 (0.0007) [2023-10-08 05:57:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106037248. Throughput: 0: 1816.0, 1: 1837.9. Samples: 26513628. Policy #0 lag: (min: 11.0, avg: 17.3, max: 43.0) [2023-10-08 05:57:08,754][130385] Avg episode reward: [(0, '68.510'), (1, '69.740')] [2023-10-08 05:57:11,697][00612] Updated weights for policy 1, policy_version 51940 (0.0007) [2023-10-08 05:57:12,067][00612] Updated weights for policy 1, policy_version 51950 (0.0008) [2023-10-08 05:57:12,348][00611] Updated weights for policy 0, policy_version 51652 (0.0009) [2023-10-08 05:57:12,426][00612] Updated weights for policy 1, policy_version 51960 (0.0007) [2023-10-08 05:57:12,724][00611] Updated weights for policy 0, policy_version 51662 (0.0008) [2023-10-08 05:57:13,089][00611] Updated weights for policy 0, policy_version 51672 (0.0008) [2023-10-08 05:57:13,754][130385] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 106135552. Throughput: 0: 1822.3, 1: 1825.6. Samples: 26536132. Policy #0 lag: (min: 11.0, avg: 17.3, max: 43.0) [2023-10-08 05:57:13,754][130385] Avg episode reward: [(0, '67.220'), (1, '69.970')] [2023-10-08 05:57:16,109][00612] Updated weights for policy 1, policy_version 51970 (0.0008) [2023-10-08 05:57:16,511][00612] Updated weights for policy 1, policy_version 51980 (0.0011) [2023-10-08 05:57:16,792][00611] Updated weights for policy 0, policy_version 51682 (0.0007) [2023-10-08 05:57:16,874][00612] Updated weights for policy 1, policy_version 51990 (0.0010) [2023-10-08 05:57:17,162][00611] Updated weights for policy 0, policy_version 51692 (0.0007) [2023-10-08 05:57:17,238][00612] Updated weights for policy 1, policy_version 52000 (0.0007) [2023-10-08 05:57:17,532][00611] Updated weights for policy 0, policy_version 51702 (0.0007) [2023-10-08 05:57:17,912][00611] Updated weights for policy 0, policy_version 51712 (0.0009) [2023-10-08 05:57:18,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 106201088. Throughput: 0: 1822.9, 1: 1840.0. Samples: 26556670. Policy #0 lag: (min: 11.0, avg: 17.3, max: 43.0) [2023-10-08 05:57:18,755][130385] Avg episode reward: [(0, '66.610'), (1, '72.380')] [2023-10-08 05:57:20,958][00612] Updated weights for policy 1, policy_version 52010 (0.0007) [2023-10-08 05:57:21,327][00612] Updated weights for policy 1, policy_version 52020 (0.0007) [2023-10-08 05:57:21,688][00612] Updated weights for policy 1, policy_version 52030 (0.0009) [2023-10-08 05:57:21,725][00611] Updated weights for policy 0, policy_version 51722 (0.0010) [2023-10-08 05:57:22,093][00611] Updated weights for policy 0, policy_version 51732 (0.0010) [2023-10-08 05:57:22,471][00611] Updated weights for policy 0, policy_version 51742 (0.0008) [2023-10-08 05:57:23,754][130385] Fps is (10 sec: 13106.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 106266624. Throughput: 0: 1821.2, 1: 1831.7. Samples: 26568924. Policy #0 lag: (min: 1.0, avg: 18.5, max: 33.0) [2023-10-08 05:57:23,755][130385] Avg episode reward: [(0, '65.320'), (1, '72.700')] [2023-10-08 05:57:25,397][00612] Updated weights for policy 1, policy_version 52040 (0.0008) [2023-10-08 05:57:25,767][00612] Updated weights for policy 1, policy_version 52050 (0.0007) [2023-10-08 05:57:26,129][00611] Updated weights for policy 0, policy_version 51752 (0.0008) [2023-10-08 05:57:26,137][00612] Updated weights for policy 1, policy_version 52060 (0.0008) [2023-10-08 05:57:26,512][00611] Updated weights for policy 0, policy_version 51762 (0.0009) [2023-10-08 05:57:26,892][00611] Updated weights for policy 0, policy_version 51772 (0.0008) [2023-10-08 05:57:28,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 106332160. Throughput: 0: 1824.4, 1: 1843.3. Samples: 26589798. Policy #0 lag: (min: 1.0, avg: 18.5, max: 33.0) [2023-10-08 05:57:28,754][130385] Avg episode reward: [(0, '66.170'), (1, '76.230')] [2023-10-08 05:57:29,697][00612] Updated weights for policy 1, policy_version 52070 (0.0008) [2023-10-08 05:57:30,067][00612] Updated weights for policy 1, policy_version 52080 (0.0008) [2023-10-08 05:57:30,413][00611] Updated weights for policy 0, policy_version 51782 (0.0007) [2023-10-08 05:57:30,422][00612] Updated weights for policy 1, policy_version 52090 (0.0007) [2023-10-08 05:57:30,793][00611] Updated weights for policy 0, policy_version 51792 (0.0009) [2023-10-08 05:57:31,169][00611] Updated weights for policy 0, policy_version 51802 (0.0008) [2023-10-08 05:57:33,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 106397696. Throughput: 0: 1824.3, 1: 1853.5. Samples: 26613074. Policy #0 lag: (min: 1.0, avg: 18.5, max: 33.0) [2023-10-08 05:57:33,756][130385] Avg episode reward: [(0, '64.980'), (1, '74.290')] [2023-10-08 05:57:33,782][00612] Updated weights for policy 1, policy_version 52100 (0.0008) [2023-10-08 05:57:34,155][00612] Updated weights for policy 1, policy_version 52110 (0.0008) [2023-10-08 05:57:34,529][00612] Updated weights for policy 1, policy_version 52120 (0.0009) [2023-10-08 05:57:34,937][00611] Updated weights for policy 0, policy_version 51812 (0.0009) [2023-10-08 05:57:35,306][00611] Updated weights for policy 0, policy_version 51822 (0.0007) [2023-10-08 05:57:35,687][00611] Updated weights for policy 0, policy_version 51832 (0.0009) [2023-10-08 05:57:38,067][00612] Updated weights for policy 1, policy_version 52130 (0.0009) [2023-10-08 05:57:38,432][00612] Updated weights for policy 1, policy_version 52140 (0.0010) [2023-10-08 05:57:38,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 106463232. Throughput: 0: 1822.4, 1: 1851.3. Samples: 26623036. Policy #0 lag: (min: 1.0, avg: 18.5, max: 33.0) [2023-10-08 05:57:38,754][130385] Avg episode reward: [(0, '65.330'), (1, '70.290')] [2023-10-08 05:57:38,802][00612] Updated weights for policy 1, policy_version 52150 (0.0011) [2023-10-08 05:57:39,171][00612] Updated weights for policy 1, policy_version 52160 (0.0010) [2023-10-08 05:57:39,323][00611] Updated weights for policy 0, policy_version 51842 (0.0007) [2023-10-08 05:57:39,703][00611] Updated weights for policy 0, policy_version 51852 (0.0009) [2023-10-08 05:57:40,086][00611] Updated weights for policy 0, policy_version 51862 (0.0008) [2023-10-08 05:57:40,455][00611] Updated weights for policy 0, policy_version 51872 (0.0009) [2023-10-08 05:57:42,789][00612] Updated weights for policy 1, policy_version 52170 (0.0008) [2023-10-08 05:57:43,160][00612] Updated weights for policy 1, policy_version 52180 (0.0007) [2023-10-08 05:57:43,527][00612] Updated weights for policy 1, policy_version 52190 (0.0007) [2023-10-08 05:57:43,754][130385] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106561536. Throughput: 0: 1819.0, 1: 1851.2. Samples: 26646010. Policy #0 lag: (min: 1.0, avg: 18.5, max: 33.0) [2023-10-08 05:57:43,754][130385] Avg episode reward: [(0, '64.680'), (1, '73.570')] [2023-10-08 05:57:44,197][00611] Updated weights for policy 0, policy_version 51882 (0.0008) [2023-10-08 05:57:44,560][00611] Updated weights for policy 0, policy_version 51892 (0.0010) [2023-10-08 05:57:44,939][00611] Updated weights for policy 0, policy_version 51902 (0.0009) [2023-10-08 05:57:47,110][00612] Updated weights for policy 1, policy_version 52200 (0.0010) [2023-10-08 05:57:47,484][00612] Updated weights for policy 1, policy_version 52210 (0.0008) [2023-10-08 05:57:47,853][00612] Updated weights for policy 1, policy_version 52220 (0.0008) [2023-10-08 05:57:48,660][00611] Updated weights for policy 0, policy_version 51912 (0.0008) [2023-10-08 05:57:48,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106627072. Throughput: 0: 1817.0, 1: 1838.1. Samples: 26667526. Policy #0 lag: (min: 1.0, avg: 18.5, max: 33.0) [2023-10-08 05:57:48,754][130385] Avg episode reward: [(0, '64.380'), (1, '71.960')] [2023-10-08 05:57:49,038][00611] Updated weights for policy 0, policy_version 51922 (0.0011) [2023-10-08 05:57:49,409][00611] Updated weights for policy 0, policy_version 51932 (0.0010) [2023-10-08 05:57:51,531][00612] Updated weights for policy 1, policy_version 52230 (0.0008) [2023-10-08 05:57:51,894][00612] Updated weights for policy 1, policy_version 52240 (0.0009) [2023-10-08 05:57:52,271][00612] Updated weights for policy 1, policy_version 52250 (0.0009) [2023-10-08 05:57:53,184][00611] Updated weights for policy 0, policy_version 51942 (0.0008) [2023-10-08 05:57:53,564][00611] Updated weights for policy 0, policy_version 51952 (0.0009) [2023-10-08 05:57:53,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106692608. Throughput: 0: 1814.8, 1: 1859.8. Samples: 26678988. Policy #0 lag: (min: 1.0, avg: 18.5, max: 33.0) [2023-10-08 05:57:53,755][130385] Avg episode reward: [(0, '63.690'), (1, '73.530')] [2023-10-08 05:57:53,925][00611] Updated weights for policy 0, policy_version 51962 (0.0009) [2023-10-08 05:57:55,918][00612] Updated weights for policy 1, policy_version 52260 (0.0010) [2023-10-08 05:57:56,283][00612] Updated weights for policy 1, policy_version 52270 (0.0007) [2023-10-08 05:57:56,654][00612] Updated weights for policy 1, policy_version 52280 (0.0008) [2023-10-08 05:57:57,784][00611] Updated weights for policy 0, policy_version 51972 (0.0010) [2023-10-08 05:57:58,153][00611] Updated weights for policy 0, policy_version 51982 (0.0011) [2023-10-08 05:57:58,527][00611] Updated weights for policy 0, policy_version 51992 (0.0010) [2023-10-08 05:57:58,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 106758144. Throughput: 0: 1808.8, 1: 1841.5. Samples: 26700398. Policy #0 lag: (min: 1.0, avg: 18.5, max: 33.0) [2023-10-08 05:57:58,755][130385] Avg episode reward: [(0, '63.760'), (1, '72.830')] [2023-10-08 05:58:00,204][00612] Updated weights for policy 1, policy_version 52290 (0.0008) [2023-10-08 05:58:00,570][00612] Updated weights for policy 1, policy_version 52300 (0.0010) [2023-10-08 05:58:00,948][00612] Updated weights for policy 1, policy_version 52310 (0.0009) [2023-10-08 05:58:01,310][00612] Updated weights for policy 1, policy_version 52320 (0.0009) [2023-10-08 05:58:02,261][00611] Updated weights for policy 0, policy_version 52002 (0.0008) [2023-10-08 05:58:02,629][00611] Updated weights for policy 0, policy_version 52012 (0.0008) [2023-10-08 05:58:02,998][00611] Updated weights for policy 0, policy_version 52022 (0.0009) [2023-10-08 05:58:03,372][00611] Updated weights for policy 0, policy_version 52032 (0.0008) [2023-10-08 05:58:03,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 106856448. Throughput: 0: 1819.1, 1: 1862.9. Samples: 26722358. Policy #0 lag: (min: 18.0, avg: 18.1, max: 25.0) [2023-10-08 05:58:03,755][130385] Avg episode reward: [(0, '65.730'), (1, '70.450')] [2023-10-08 05:58:05,084][00612] Updated weights for policy 1, policy_version 52330 (0.0009) [2023-10-08 05:58:05,455][00612] Updated weights for policy 1, policy_version 52340 (0.0011) [2023-10-08 05:58:05,833][00612] Updated weights for policy 1, policy_version 52350 (0.0011) [2023-10-08 05:58:07,131][00611] Updated weights for policy 0, policy_version 52042 (0.0007) [2023-10-08 05:58:07,510][00611] Updated weights for policy 0, policy_version 52052 (0.0007) [2023-10-08 05:58:07,875][00611] Updated weights for policy 0, policy_version 52062 (0.0007) [2023-10-08 05:58:08,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106921984. Throughput: 0: 1810.8, 1: 1839.9. Samples: 26733202. Policy #0 lag: (min: 18.0, avg: 18.1, max: 25.0) [2023-10-08 05:58:08,754][130385] Avg episode reward: [(0, '66.890'), (1, '67.540')] [2023-10-08 05:58:09,553][00612] Updated weights for policy 1, policy_version 52360 (0.0009) [2023-10-08 05:58:09,919][00612] Updated weights for policy 1, policy_version 52370 (0.0010) [2023-10-08 05:58:10,295][00612] Updated weights for policy 1, policy_version 52380 (0.0008) [2023-10-08 05:58:11,537][00611] Updated weights for policy 0, policy_version 52072 (0.0008) [2023-10-08 05:58:11,909][00611] Updated weights for policy 0, policy_version 52082 (0.0009) [2023-10-08 05:58:12,283][00611] Updated weights for policy 0, policy_version 52092 (0.0008) [2023-10-08 05:58:13,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 106987520. Throughput: 0: 1821.9, 1: 1853.7. Samples: 26755200. Policy #0 lag: (min: 18.0, avg: 18.1, max: 25.0) [2023-10-08 05:58:13,754][130385] Avg episode reward: [(0, '64.080'), (1, '65.070')] [2023-10-08 05:58:14,011][00612] Updated weights for policy 1, policy_version 52390 (0.0008) [2023-10-08 05:58:14,380][00612] Updated weights for policy 1, policy_version 52400 (0.0008) [2023-10-08 05:58:14,746][00612] Updated weights for policy 1, policy_version 52410 (0.0007) [2023-10-08 05:58:15,863][00611] Updated weights for policy 0, policy_version 52102 (0.0007) [2023-10-08 05:58:16,247][00611] Updated weights for policy 0, policy_version 52112 (0.0007) [2023-10-08 05:58:16,620][00611] Updated weights for policy 0, policy_version 52122 (0.0009) [2023-10-08 05:58:18,489][00612] Updated weights for policy 1, policy_version 52420 (0.0008) [2023-10-08 05:58:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 107053056. Throughput: 0: 1816.9, 1: 1848.0. Samples: 26777992. Policy #0 lag: (min: 18.0, avg: 18.1, max: 25.0) [2023-10-08 05:58:18,755][130385] Avg episode reward: [(0, '62.750'), (1, '65.490')] [2023-10-08 05:58:18,762][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000052128_53379072.pth... [2023-10-08 05:58:18,796][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000050432_51642368.pth [2023-10-08 05:58:18,848][00612] Updated weights for policy 1, policy_version 52430 (0.0007) [2023-10-08 05:58:19,221][00612] Updated weights for policy 1, policy_version 52440 (0.0009) [2023-10-08 05:58:19,510][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000052448_53706752.pth... [2023-10-08 05:58:19,538][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000050720_51937280.pth [2023-10-08 05:58:20,168][00611] Updated weights for policy 0, policy_version 52132 (0.0007) [2023-10-08 05:58:20,541][00611] Updated weights for policy 0, policy_version 52142 (0.0007) [2023-10-08 05:58:20,910][00611] Updated weights for policy 0, policy_version 52152 (0.0009) [2023-10-08 05:58:22,830][00612] Updated weights for policy 1, policy_version 52450 (0.0008) [2023-10-08 05:58:23,198][00612] Updated weights for policy 1, policy_version 52460 (0.0010) [2023-10-08 05:58:23,564][00612] Updated weights for policy 1, policy_version 52470 (0.0008) [2023-10-08 05:58:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 107118592. Throughput: 0: 1821.3, 1: 1847.2. Samples: 26788122. Policy #0 lag: (min: 18.0, avg: 18.1, max: 25.0) [2023-10-08 05:58:23,754][130385] Avg episode reward: [(0, '65.020'), (1, '62.580')] [2023-10-08 05:58:23,926][00612] Updated weights for policy 1, policy_version 52480 (0.0008) [2023-10-08 05:58:24,430][00611] Updated weights for policy 0, policy_version 52162 (0.0009) [2023-10-08 05:58:24,793][00611] Updated weights for policy 0, policy_version 52172 (0.0007) [2023-10-08 05:58:25,179][00611] Updated weights for policy 0, policy_version 52182 (0.0008) [2023-10-08 05:58:25,539][00611] Updated weights for policy 0, policy_version 52192 (0.0009) [2023-10-08 05:58:27,478][00612] Updated weights for policy 1, policy_version 52490 (0.0007) [2023-10-08 05:58:27,846][00612] Updated weights for policy 1, policy_version 52500 (0.0007) [2023-10-08 05:58:28,219][00612] Updated weights for policy 1, policy_version 52510 (0.0007) [2023-10-08 05:58:28,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107216896. Throughput: 0: 1825.0, 1: 1848.2. Samples: 26811304. Policy #0 lag: (min: 18.0, avg: 18.1, max: 25.0) [2023-10-08 05:58:28,754][130385] Avg episode reward: [(0, '64.220'), (1, '63.060')] [2023-10-08 05:58:29,146][00611] Updated weights for policy 0, policy_version 52202 (0.0008) [2023-10-08 05:58:29,526][00611] Updated weights for policy 0, policy_version 52212 (0.0009) [2023-10-08 05:58:29,890][00611] Updated weights for policy 0, policy_version 52222 (0.0008) [2023-10-08 05:58:31,898][00612] Updated weights for policy 1, policy_version 52520 (0.0010) [2023-10-08 05:58:32,267][00612] Updated weights for policy 1, policy_version 52530 (0.0008) [2023-10-08 05:58:32,633][00612] Updated weights for policy 1, policy_version 52540 (0.0009) [2023-10-08 05:58:33,612][00611] Updated weights for policy 0, policy_version 52232 (0.0008) [2023-10-08 05:58:33,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 107282432. Throughput: 0: 1826.3, 1: 1850.4. Samples: 26832974. Policy #0 lag: (min: 18.0, avg: 18.1, max: 25.0) [2023-10-08 05:58:33,754][130385] Avg episode reward: [(0, '60.650'), (1, '66.980')] [2023-10-08 05:58:33,979][00611] Updated weights for policy 0, policy_version 52242 (0.0010) [2023-10-08 05:58:34,345][00611] Updated weights for policy 0, policy_version 52252 (0.0010) [2023-10-08 05:58:36,185][00612] Updated weights for policy 1, policy_version 52550 (0.0008) [2023-10-08 05:58:36,558][00612] Updated weights for policy 1, policy_version 52560 (0.0008) [2023-10-08 05:58:36,936][00612] Updated weights for policy 1, policy_version 52570 (0.0009) [2023-10-08 05:58:37,955][00611] Updated weights for policy 0, policy_version 52262 (0.0010) [2023-10-08 05:58:38,328][00611] Updated weights for policy 0, policy_version 52272 (0.0009) [2023-10-08 05:58:38,692][00611] Updated weights for policy 0, policy_version 52282 (0.0007) [2023-10-08 05:58:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107347968. Throughput: 0: 1828.0, 1: 1841.8. Samples: 26844132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:58:38,755][130385] Avg episode reward: [(0, '58.400'), (1, '62.950')] [2023-10-08 05:58:40,586][00612] Updated weights for policy 1, policy_version 52580 (0.0010) [2023-10-08 05:58:40,959][00612] Updated weights for policy 1, policy_version 52590 (0.0010) [2023-10-08 05:58:41,332][00612] Updated weights for policy 1, policy_version 52600 (0.0011) [2023-10-08 05:58:42,399][00611] Updated weights for policy 0, policy_version 52292 (0.0009) [2023-10-08 05:58:42,777][00611] Updated weights for policy 0, policy_version 52302 (0.0010) [2023-10-08 05:58:43,139][00611] Updated weights for policy 0, policy_version 52312 (0.0009) [2023-10-08 05:58:43,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107446272. Throughput: 0: 1830.1, 1: 1849.7. Samples: 26865988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:58:43,754][130385] Avg episode reward: [(0, '62.440'), (1, '64.390')] [2023-10-08 05:58:44,868][00612] Updated weights for policy 1, policy_version 52610 (0.0010) [2023-10-08 05:58:45,235][00612] Updated weights for policy 1, policy_version 52620 (0.0008) [2023-10-08 05:58:45,604][00612] Updated weights for policy 1, policy_version 52630 (0.0008) [2023-10-08 05:58:45,975][00612] Updated weights for policy 1, policy_version 52640 (0.0007) [2023-10-08 05:58:46,707][00611] Updated weights for policy 0, policy_version 52322 (0.0007) [2023-10-08 05:58:47,075][00611] Updated weights for policy 0, policy_version 52332 (0.0007) [2023-10-08 05:58:47,443][00611] Updated weights for policy 0, policy_version 52342 (0.0008) [2023-10-08 05:58:47,803][00611] Updated weights for policy 0, policy_version 52352 (0.0008) [2023-10-08 05:58:48,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107511808. Throughput: 0: 1825.9, 1: 1858.5. Samples: 26888154. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:58:48,754][130385] Avg episode reward: [(0, '62.340'), (1, '61.450')] [2023-10-08 05:58:49,501][00612] Updated weights for policy 1, policy_version 52650 (0.0010) [2023-10-08 05:58:49,867][00612] Updated weights for policy 1, policy_version 52660 (0.0010) [2023-10-08 05:58:50,239][00612] Updated weights for policy 1, policy_version 52670 (0.0007) [2023-10-08 05:58:51,359][00611] Updated weights for policy 0, policy_version 52362 (0.0008) [2023-10-08 05:58:51,725][00611] Updated weights for policy 0, policy_version 52372 (0.0008) [2023-10-08 05:58:52,103][00611] Updated weights for policy 0, policy_version 52382 (0.0009) [2023-10-08 05:58:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107577344. Throughput: 0: 1833.5, 1: 1859.9. Samples: 26899404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:58:53,754][130385] Avg episode reward: [(0, '64.920'), (1, '59.850')] [2023-10-08 05:58:53,964][00612] Updated weights for policy 1, policy_version 52680 (0.0007) [2023-10-08 05:58:54,330][00612] Updated weights for policy 1, policy_version 52690 (0.0011) [2023-10-08 05:58:54,705][00612] Updated weights for policy 1, policy_version 52700 (0.0009) [2023-10-08 05:58:55,825][00611] Updated weights for policy 0, policy_version 52392 (0.0008) [2023-10-08 05:58:56,206][00611] Updated weights for policy 0, policy_version 52402 (0.0008) [2023-10-08 05:58:56,576][00611] Updated weights for policy 0, policy_version 52412 (0.0008) [2023-10-08 05:58:58,464][00612] Updated weights for policy 1, policy_version 52710 (0.0009) [2023-10-08 05:58:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 107642880. Throughput: 0: 1828.9, 1: 1854.0. Samples: 26920934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:58:58,754][130385] Avg episode reward: [(0, '63.830'), (1, '60.790')] [2023-10-08 05:58:58,834][00612] Updated weights for policy 1, policy_version 52720 (0.0011) [2023-10-08 05:58:59,202][00612] Updated weights for policy 1, policy_version 52730 (0.0009) [2023-10-08 05:59:00,251][00611] Updated weights for policy 0, policy_version 52422 (0.0008) [2023-10-08 05:59:00,623][00611] Updated weights for policy 0, policy_version 52432 (0.0008) [2023-10-08 05:59:00,988][00611] Updated weights for policy 0, policy_version 52442 (0.0009) [2023-10-08 05:59:02,776][00612] Updated weights for policy 1, policy_version 52740 (0.0008) [2023-10-08 05:59:03,141][00612] Updated weights for policy 1, policy_version 52750 (0.0008) [2023-10-08 05:59:03,508][00612] Updated weights for policy 1, policy_version 52760 (0.0008) [2023-10-08 05:59:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 107708416. Throughput: 0: 1836.5, 1: 1837.9. Samples: 26943340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:59:03,754][130385] Avg episode reward: [(0, '62.290'), (1, '64.260')] [2023-10-08 05:59:04,532][00611] Updated weights for policy 0, policy_version 52452 (0.0009) [2023-10-08 05:59:04,898][00611] Updated weights for policy 0, policy_version 52462 (0.0010) [2023-10-08 05:59:05,266][00611] Updated weights for policy 0, policy_version 52472 (0.0008) [2023-10-08 05:59:07,012][00612] Updated weights for policy 1, policy_version 52770 (0.0008) [2023-10-08 05:59:07,383][00612] Updated weights for policy 1, policy_version 52780 (0.0010) [2023-10-08 05:59:07,763][00612] Updated weights for policy 1, policy_version 52790 (0.0010) [2023-10-08 05:59:08,125][00612] Updated weights for policy 1, policy_version 52800 (0.0007) [2023-10-08 05:59:08,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107806720. Throughput: 0: 1836.2, 1: 1857.4. Samples: 26954334. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:59:08,754][130385] Avg episode reward: [(0, '65.110'), (1, '64.700')] [2023-10-08 05:59:08,948][00611] Updated weights for policy 0, policy_version 52482 (0.0009) [2023-10-08 05:59:09,316][00611] Updated weights for policy 0, policy_version 52492 (0.0009) [2023-10-08 05:59:09,686][00611] Updated weights for policy 0, policy_version 52502 (0.0011) [2023-10-08 05:59:10,065][00611] Updated weights for policy 0, policy_version 52512 (0.0009) [2023-10-08 05:59:11,748][00612] Updated weights for policy 1, policy_version 52810 (0.0008) [2023-10-08 05:59:12,113][00612] Updated weights for policy 1, policy_version 52820 (0.0008) [2023-10-08 05:59:12,474][00612] Updated weights for policy 1, policy_version 52830 (0.0007) [2023-10-08 05:59:13,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107872256. Throughput: 0: 1836.3, 1: 1831.3. Samples: 26976348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:59:13,754][130385] Avg episode reward: [(0, '62.130'), (1, '67.640')] [2023-10-08 05:59:13,762][00611] Updated weights for policy 0, policy_version 52522 (0.0010) [2023-10-08 05:59:14,138][00611] Updated weights for policy 0, policy_version 52532 (0.0010) [2023-10-08 05:59:14,504][00611] Updated weights for policy 0, policy_version 52542 (0.0007) [2023-10-08 05:59:16,074][00612] Updated weights for policy 1, policy_version 52840 (0.0009) [2023-10-08 05:59:16,444][00612] Updated weights for policy 1, policy_version 52850 (0.0007) [2023-10-08 05:59:16,822][00612] Updated weights for policy 1, policy_version 52860 (0.0007) [2023-10-08 05:59:18,147][00611] Updated weights for policy 0, policy_version 52552 (0.0007) [2023-10-08 05:59:18,524][00611] Updated weights for policy 0, policy_version 52562 (0.0007) [2023-10-08 05:59:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 107937792. Throughput: 0: 1831.6, 1: 1853.8. Samples: 26998818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:59:18,755][130385] Avg episode reward: [(0, '66.460'), (1, '72.260')] [2023-10-08 05:59:18,899][00611] Updated weights for policy 0, policy_version 52572 (0.0008) [2023-10-08 05:59:20,636][00612] Updated weights for policy 1, policy_version 52870 (0.0008) [2023-10-08 05:59:21,008][00612] Updated weights for policy 1, policy_version 52880 (0.0008) [2023-10-08 05:59:21,375][00612] Updated weights for policy 1, policy_version 52890 (0.0008) [2023-10-08 05:59:22,624][00611] Updated weights for policy 0, policy_version 52582 (0.0009) [2023-10-08 05:59:22,992][00611] Updated weights for policy 0, policy_version 52592 (0.0007) [2023-10-08 05:59:23,353][00611] Updated weights for policy 0, policy_version 52602 (0.0007) [2023-10-08 05:59:23,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 108036096. Throughput: 0: 1840.4, 1: 1834.9. Samples: 27009522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:59:23,755][130385] Avg episode reward: [(0, '64.280'), (1, '70.280')] [2023-10-08 05:59:24,975][00612] Updated weights for policy 1, policy_version 52900 (0.0007) [2023-10-08 05:59:25,347][00612] Updated weights for policy 1, policy_version 52910 (0.0009) [2023-10-08 05:59:25,717][00612] Updated weights for policy 1, policy_version 52920 (0.0011) [2023-10-08 05:59:26,989][00611] Updated weights for policy 0, policy_version 52612 (0.0008) [2023-10-08 05:59:27,354][00611] Updated weights for policy 0, policy_version 52622 (0.0009) [2023-10-08 05:59:27,724][00611] Updated weights for policy 0, policy_version 52632 (0.0009) [2023-10-08 05:59:28,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 108101632. Throughput: 0: 1828.8, 1: 1849.6. Samples: 27031516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:59:28,754][130385] Avg episode reward: [(0, '66.690'), (1, '70.540')] [2023-10-08 05:59:29,232][00612] Updated weights for policy 1, policy_version 52930 (0.0011) [2023-10-08 05:59:29,599][00612] Updated weights for policy 1, policy_version 52940 (0.0007) [2023-10-08 05:59:29,968][00612] Updated weights for policy 1, policy_version 52950 (0.0008) [2023-10-08 05:59:30,326][00612] Updated weights for policy 1, policy_version 52960 (0.0009) [2023-10-08 05:59:31,313][00611] Updated weights for policy 0, policy_version 52642 (0.0010) [2023-10-08 05:59:31,690][00611] Updated weights for policy 0, policy_version 52652 (0.0008) [2023-10-08 05:59:32,070][00611] Updated weights for policy 0, policy_version 52662 (0.0010) [2023-10-08 05:59:32,439][00611] Updated weights for policy 0, policy_version 52672 (0.0010) [2023-10-08 05:59:33,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 108167168. Throughput: 0: 1836.7, 1: 1843.6. Samples: 27053764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:59:33,754][130385] Avg episode reward: [(0, '65.270'), (1, '73.230')] [2023-10-08 05:59:33,999][00612] Updated weights for policy 1, policy_version 52970 (0.0009) [2023-10-08 05:59:34,356][00612] Updated weights for policy 1, policy_version 52980 (0.0009) [2023-10-08 05:59:34,726][00612] Updated weights for policy 1, policy_version 52990 (0.0009) [2023-10-08 05:59:36,062][00611] Updated weights for policy 0, policy_version 52682 (0.0007) [2023-10-08 05:59:36,444][00611] Updated weights for policy 0, policy_version 52692 (0.0008) [2023-10-08 05:59:36,820][00611] Updated weights for policy 0, policy_version 52702 (0.0009) [2023-10-08 05:59:38,405][00612] Updated weights for policy 1, policy_version 53000 (0.0007) [2023-10-08 05:59:38,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 108232704. Throughput: 0: 1828.1, 1: 1845.0. Samples: 27064694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:59:38,755][130385] Avg episode reward: [(0, '68.080'), (1, '72.240')] [2023-10-08 05:59:38,791][00612] Updated weights for policy 1, policy_version 53010 (0.0011) [2023-10-08 05:59:39,159][00612] Updated weights for policy 1, policy_version 53020 (0.0008) [2023-10-08 05:59:40,632][00611] Updated weights for policy 0, policy_version 52712 (0.0008) [2023-10-08 05:59:40,993][00611] Updated weights for policy 0, policy_version 52722 (0.0009) [2023-10-08 05:59:41,368][00611] Updated weights for policy 0, policy_version 52732 (0.0007) [2023-10-08 05:59:42,886][00612] Updated weights for policy 1, policy_version 53030 (0.0008) [2023-10-08 05:59:43,259][00612] Updated weights for policy 1, policy_version 53040 (0.0009) [2023-10-08 05:59:43,623][00612] Updated weights for policy 1, policy_version 53050 (0.0008) [2023-10-08 05:59:43,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 108298240. Throughput: 0: 1835.4, 1: 1845.7. Samples: 27086582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 05:59:43,755][130385] Avg episode reward: [(0, '69.110'), (1, '73.520')] [2023-10-08 05:59:44,935][00611] Updated weights for policy 0, policy_version 52742 (0.0008) [2023-10-08 05:59:45,301][00611] Updated weights for policy 0, policy_version 52752 (0.0010) [2023-10-08 05:59:45,680][00611] Updated weights for policy 0, policy_version 52762 (0.0007) [2023-10-08 05:59:47,281][00612] Updated weights for policy 1, policy_version 53060 (0.0008) [2023-10-08 05:59:47,642][00612] Updated weights for policy 1, policy_version 53070 (0.0008) [2023-10-08 05:59:48,015][00612] Updated weights for policy 1, policy_version 53080 (0.0010) [2023-10-08 05:59:48,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 108396544. Throughput: 0: 1834.7, 1: 1831.9. Samples: 27108338. Policy #0 lag: (min: 1.0, avg: 10.3, max: 33.0) [2023-10-08 05:59:48,755][130385] Avg episode reward: [(0, '66.240'), (1, '71.910')] [2023-10-08 05:59:49,484][00611] Updated weights for policy 0, policy_version 52772 (0.0010) [2023-10-08 05:59:49,858][00611] Updated weights for policy 0, policy_version 52782 (0.0010) [2023-10-08 05:59:50,235][00611] Updated weights for policy 0, policy_version 52792 (0.0010) [2023-10-08 05:59:51,700][00612] Updated weights for policy 1, policy_version 53090 (0.0009) [2023-10-08 05:59:52,066][00612] Updated weights for policy 1, policy_version 53100 (0.0008) [2023-10-08 05:59:52,431][00612] Updated weights for policy 1, policy_version 53110 (0.0009) [2023-10-08 05:59:52,799][00612] Updated weights for policy 1, policy_version 53120 (0.0007) [2023-10-08 05:59:53,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 108462080. Throughput: 0: 1825.0, 1: 1841.9. Samples: 27119342. Policy #0 lag: (min: 1.0, avg: 10.3, max: 33.0) [2023-10-08 05:59:53,754][130385] Avg episode reward: [(0, '63.530'), (1, '75.980')] [2023-10-08 05:59:53,842][00611] Updated weights for policy 0, policy_version 52802 (0.0009) [2023-10-08 05:59:54,215][00611] Updated weights for policy 0, policy_version 52812 (0.0008) [2023-10-08 05:59:54,574][00611] Updated weights for policy 0, policy_version 52822 (0.0008) [2023-10-08 05:59:54,950][00611] Updated weights for policy 0, policy_version 52832 (0.0009) [2023-10-08 05:59:56,489][00612] Updated weights for policy 1, policy_version 53130 (0.0012) [2023-10-08 05:59:56,865][00612] Updated weights for policy 1, policy_version 53140 (0.0010) [2023-10-08 05:59:57,225][00612] Updated weights for policy 1, policy_version 53150 (0.0007) [2023-10-08 05:59:58,628][00611] Updated weights for policy 0, policy_version 52842 (0.0009) [2023-10-08 05:59:58,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 108527616. Throughput: 0: 1829.2, 1: 1833.3. Samples: 27141164. Policy #0 lag: (min: 1.0, avg: 10.3, max: 33.0) [2023-10-08 05:59:58,754][130385] Avg episode reward: [(0, '66.610'), (1, '76.090')] [2023-10-08 05:59:58,999][00611] Updated weights for policy 0, policy_version 52852 (0.0008) [2023-10-08 05:59:59,376][00611] Updated weights for policy 0, policy_version 52862 (0.0008) [2023-10-08 06:00:00,811][00612] Updated weights for policy 1, policy_version 53160 (0.0008) [2023-10-08 06:00:01,186][00612] Updated weights for policy 1, policy_version 53170 (0.0008) [2023-10-08 06:00:01,547][00612] Updated weights for policy 1, policy_version 53180 (0.0010) [2023-10-08 06:00:03,099][00611] Updated weights for policy 0, policy_version 52872 (0.0009) [2023-10-08 06:00:03,472][00611] Updated weights for policy 0, policy_version 52882 (0.0008) [2023-10-08 06:00:03,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 108593152. Throughput: 0: 1824.7, 1: 1840.0. Samples: 27163726. Policy #0 lag: (min: 1.0, avg: 10.3, max: 33.0) [2023-10-08 06:00:03,755][130385] Avg episode reward: [(0, '69.690'), (1, '76.290')] [2023-10-08 06:00:03,839][00611] Updated weights for policy 0, policy_version 52892 (0.0008) [2023-10-08 06:00:05,016][00612] Updated weights for policy 1, policy_version 53190 (0.0008) [2023-10-08 06:00:05,380][00612] Updated weights for policy 1, policy_version 53200 (0.0009) [2023-10-08 06:00:05,746][00612] Updated weights for policy 1, policy_version 53210 (0.0010) [2023-10-08 06:00:07,450][00611] Updated weights for policy 0, policy_version 52902 (0.0010) [2023-10-08 06:00:07,808][00611] Updated weights for policy 0, policy_version 52912 (0.0007) [2023-10-08 06:00:08,180][00611] Updated weights for policy 0, policy_version 52922 (0.0008) [2023-10-08 06:00:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 108691456. Throughput: 0: 1826.9, 1: 1827.7. Samples: 27173982. Policy #0 lag: (min: 1.0, avg: 10.3, max: 33.0) [2023-10-08 06:00:08,755][130385] Avg episode reward: [(0, '71.040'), (1, '73.380')] [2023-10-08 06:00:09,456][00612] Updated weights for policy 1, policy_version 53220 (0.0009) [2023-10-08 06:00:09,823][00612] Updated weights for policy 1, policy_version 53230 (0.0010) [2023-10-08 06:00:10,197][00612] Updated weights for policy 1, policy_version 53240 (0.0009) [2023-10-08 06:00:11,896][00611] Updated weights for policy 0, policy_version 52932 (0.0007) [2023-10-08 06:00:12,269][00611] Updated weights for policy 0, policy_version 52942 (0.0009) [2023-10-08 06:00:12,642][00611] Updated weights for policy 0, policy_version 52952 (0.0008) [2023-10-08 06:00:13,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 108756992. Throughput: 0: 1828.5, 1: 1839.4. Samples: 27196572. Policy #0 lag: (min: 1.0, avg: 10.3, max: 33.0) [2023-10-08 06:00:13,754][130385] Avg episode reward: [(0, '70.180'), (1, '70.870')] [2023-10-08 06:00:13,910][00612] Updated weights for policy 1, policy_version 53250 (0.0008) [2023-10-08 06:00:14,279][00612] Updated weights for policy 1, policy_version 53260 (0.0010) [2023-10-08 06:00:14,650][00612] Updated weights for policy 1, policy_version 53270 (0.0007) [2023-10-08 06:00:15,014][00612] Updated weights for policy 1, policy_version 53280 (0.0008) [2023-10-08 06:00:16,155][00611] Updated weights for policy 0, policy_version 52962 (0.0010) [2023-10-08 06:00:16,527][00611] Updated weights for policy 0, policy_version 52972 (0.0011) [2023-10-08 06:00:16,894][00611] Updated weights for policy 0, policy_version 52982 (0.0008) [2023-10-08 06:00:17,271][00611] Updated weights for policy 0, policy_version 52992 (0.0009) [2023-10-08 06:00:18,519][00612] Updated weights for policy 1, policy_version 53290 (0.0007) [2023-10-08 06:00:18,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 108822528. Throughput: 0: 1832.5, 1: 1838.6. Samples: 27218962. Policy #0 lag: (min: 1.0, avg: 10.3, max: 33.0) [2023-10-08 06:00:18,755][130385] Avg episode reward: [(0, '69.270'), (1, '69.290')] [2023-10-08 06:00:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000052992_54263808.pth... [2023-10-08 06:00:18,798][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000051296_52527104.pth [2023-10-08 06:00:18,889][00612] Updated weights for policy 1, policy_version 53300 (0.0009) [2023-10-08 06:00:19,250][00612] Updated weights for policy 1, policy_version 53310 (0.0008) [2023-10-08 06:00:19,323][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000053312_54591488.pth... [2023-10-08 06:00:19,357][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000051584_52822016.pth [2023-10-08 06:00:20,911][00611] Updated weights for policy 0, policy_version 53002 (0.0008) [2023-10-08 06:00:21,276][00611] Updated weights for policy 0, policy_version 53012 (0.0007) [2023-10-08 06:00:21,652][00611] Updated weights for policy 0, policy_version 53022 (0.0009) [2023-10-08 06:00:22,988][00612] Updated weights for policy 1, policy_version 53320 (0.0008) [2023-10-08 06:00:23,357][00612] Updated weights for policy 1, policy_version 53330 (0.0008) [2023-10-08 06:00:23,713][00612] Updated weights for policy 1, policy_version 53340 (0.0007) [2023-10-08 06:00:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 108888064. Throughput: 0: 1826.5, 1: 1840.1. Samples: 27229688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:00:23,754][130385] Avg episode reward: [(0, '73.350'), (1, '69.840')] [2023-10-08 06:00:23,755][00365] Saving new best policy, reward=73.350! [2023-10-08 06:00:25,235][00611] Updated weights for policy 0, policy_version 53032 (0.0009) [2023-10-08 06:00:25,610][00611] Updated weights for policy 0, policy_version 53042 (0.0010) [2023-10-08 06:00:25,982][00611] Updated weights for policy 0, policy_version 53052 (0.0009) [2023-10-08 06:00:27,418][00612] Updated weights for policy 1, policy_version 53350 (0.0007) [2023-10-08 06:00:27,810][00612] Updated weights for policy 1, policy_version 53360 (0.0007) [2023-10-08 06:00:28,184][00612] Updated weights for policy 1, policy_version 53370 (0.0010) [2023-10-08 06:00:28,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 108986368. Throughput: 0: 1829.8, 1: 1843.4. Samples: 27251874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:00:28,754][130385] Avg episode reward: [(0, '74.050'), (1, '62.340')] [2023-10-08 06:00:28,755][00365] Saving new best policy, reward=74.050! [2023-10-08 06:00:29,625][00611] Updated weights for policy 0, policy_version 53062 (0.0009) [2023-10-08 06:00:30,009][00611] Updated weights for policy 0, policy_version 53072 (0.0007) [2023-10-08 06:00:30,380][00611] Updated weights for policy 0, policy_version 53082 (0.0009) [2023-10-08 06:00:31,751][00612] Updated weights for policy 1, policy_version 53380 (0.0008) [2023-10-08 06:00:32,127][00612] Updated weights for policy 1, policy_version 53390 (0.0008) [2023-10-08 06:00:32,496][00612] Updated weights for policy 1, policy_version 53400 (0.0008) [2023-10-08 06:00:33,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 109051904. Throughput: 0: 1830.1, 1: 1840.3. Samples: 27273504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:00:33,755][130385] Avg episode reward: [(0, '74.590'), (1, '66.850')] [2023-10-08 06:00:33,769][00365] Saving new best policy, reward=74.590! [2023-10-08 06:00:34,126][00611] Updated weights for policy 0, policy_version 53092 (0.0009) [2023-10-08 06:00:34,496][00611] Updated weights for policy 0, policy_version 53102 (0.0007) [2023-10-08 06:00:34,869][00611] Updated weights for policy 0, policy_version 53112 (0.0008) [2023-10-08 06:00:36,047][00612] Updated weights for policy 1, policy_version 53410 (0.0011) [2023-10-08 06:00:36,412][00612] Updated weights for policy 1, policy_version 53420 (0.0011) [2023-10-08 06:00:36,783][00612] Updated weights for policy 1, policy_version 53430 (0.0008) [2023-10-08 06:00:37,151][00612] Updated weights for policy 1, policy_version 53440 (0.0008) [2023-10-08 06:00:38,384][00611] Updated weights for policy 0, policy_version 53122 (0.0008) [2023-10-08 06:00:38,747][00611] Updated weights for policy 0, policy_version 53132 (0.0007) [2023-10-08 06:00:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 109117440. Throughput: 0: 1835.7, 1: 1841.0. Samples: 27284792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:00:38,754][130385] Avg episode reward: [(0, '70.810'), (1, '66.690')] [2023-10-08 06:00:39,120][00611] Updated weights for policy 0, policy_version 53142 (0.0008) [2023-10-08 06:00:39,491][00611] Updated weights for policy 0, policy_version 53152 (0.0010) [2023-10-08 06:00:40,739][00612] Updated weights for policy 1, policy_version 53450 (0.0008) [2023-10-08 06:00:41,099][00612] Updated weights for policy 1, policy_version 53460 (0.0009) [2023-10-08 06:00:41,468][00612] Updated weights for policy 1, policy_version 53470 (0.0009) [2023-10-08 06:00:43,105][00611] Updated weights for policy 0, policy_version 53162 (0.0010) [2023-10-08 06:00:43,471][00611] Updated weights for policy 0, policy_version 53172 (0.0011) [2023-10-08 06:00:43,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 109182976. Throughput: 0: 1840.6, 1: 1846.6. Samples: 27307088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:00:43,754][130385] Avg episode reward: [(0, '72.490'), (1, '67.540')] [2023-10-08 06:00:43,845][00611] Updated weights for policy 0, policy_version 53182 (0.0010) [2023-10-08 06:00:45,125][00612] Updated weights for policy 1, policy_version 53480 (0.0009) [2023-10-08 06:00:45,489][00612] Updated weights for policy 1, policy_version 53490 (0.0008) [2023-10-08 06:00:45,857][00612] Updated weights for policy 1, policy_version 53500 (0.0008) [2023-10-08 06:00:47,637][00611] Updated weights for policy 0, policy_version 53192 (0.0009) [2023-10-08 06:00:48,011][00611] Updated weights for policy 0, policy_version 53202 (0.0010) [2023-10-08 06:00:48,376][00611] Updated weights for policy 0, policy_version 53212 (0.0010) [2023-10-08 06:00:48,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 109281280. Throughput: 0: 1826.6, 1: 1850.5. Samples: 27329198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:00:48,754][130385] Avg episode reward: [(0, '72.160'), (1, '67.660')] [2023-10-08 06:00:49,540][00612] Updated weights for policy 1, policy_version 53510 (0.0009) [2023-10-08 06:00:49,901][00612] Updated weights for policy 1, policy_version 53520 (0.0009) [2023-10-08 06:00:50,272][00612] Updated weights for policy 1, policy_version 53530 (0.0007) [2023-10-08 06:00:52,005][00611] Updated weights for policy 0, policy_version 53222 (0.0009) [2023-10-08 06:00:52,385][00611] Updated weights for policy 0, policy_version 53232 (0.0007) [2023-10-08 06:00:52,754][00611] Updated weights for policy 0, policy_version 53242 (0.0007) [2023-10-08 06:00:53,740][00612] Updated weights for policy 1, policy_version 53540 (0.0007) [2023-10-08 06:00:53,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109346816. Throughput: 0: 1842.9, 1: 1851.6. Samples: 27340234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:00:53,755][130385] Avg episode reward: [(0, '67.800'), (1, '66.830')] [2023-10-08 06:00:54,117][00612] Updated weights for policy 1, policy_version 53550 (0.0007) [2023-10-08 06:00:54,479][00612] Updated weights for policy 1, policy_version 53560 (0.0008) [2023-10-08 06:00:56,524][00611] Updated weights for policy 0, policy_version 53252 (0.0007) [2023-10-08 06:00:56,890][00611] Updated weights for policy 0, policy_version 53262 (0.0008) [2023-10-08 06:00:57,268][00611] Updated weights for policy 0, policy_version 53272 (0.0009) [2023-10-08 06:00:58,050][00612] Updated weights for policy 1, policy_version 53570 (0.0008) [2023-10-08 06:00:58,423][00612] Updated weights for policy 1, policy_version 53580 (0.0007) [2023-10-08 06:00:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109412352. Throughput: 0: 1828.2, 1: 1856.8. Samples: 27362394. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 06:00:58,754][130385] Avg episode reward: [(0, '69.410'), (1, '65.450')] [2023-10-08 06:00:58,793][00612] Updated weights for policy 1, policy_version 53590 (0.0007) [2023-10-08 06:00:59,153][00612] Updated weights for policy 1, policy_version 53600 (0.0009) [2023-10-08 06:01:00,957][00611] Updated weights for policy 0, policy_version 53282 (0.0008) [2023-10-08 06:01:01,319][00611] Updated weights for policy 0, policy_version 53292 (0.0008) [2023-10-08 06:01:01,704][00611] Updated weights for policy 0, policy_version 53302 (0.0009) [2023-10-08 06:01:02,060][00611] Updated weights for policy 0, policy_version 53312 (0.0011) [2023-10-08 06:01:02,784][00612] Updated weights for policy 1, policy_version 53610 (0.0012) [2023-10-08 06:01:03,151][00612] Updated weights for policy 1, policy_version 53620 (0.0011) [2023-10-08 06:01:03,523][00612] Updated weights for policy 1, policy_version 53630 (0.0008) [2023-10-08 06:01:03,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 109510656. Throughput: 0: 1830.8, 1: 1830.3. Samples: 27383710. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 06:01:03,757][130385] Avg episode reward: [(0, '69.980'), (1, '67.940')] [2023-10-08 06:01:05,731][00611] Updated weights for policy 0, policy_version 53322 (0.0007) [2023-10-08 06:01:06,103][00611] Updated weights for policy 0, policy_version 53332 (0.0007) [2023-10-08 06:01:06,473][00611] Updated weights for policy 0, policy_version 53342 (0.0007) [2023-10-08 06:01:07,061][00612] Updated weights for policy 1, policy_version 53640 (0.0007) [2023-10-08 06:01:07,434][00612] Updated weights for policy 1, policy_version 53650 (0.0009) [2023-10-08 06:01:07,803][00612] Updated weights for policy 1, policy_version 53660 (0.0010) [2023-10-08 06:01:08,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 109576192. Throughput: 0: 1826.4, 1: 1856.3. Samples: 27395410. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 06:01:08,755][130385] Avg episode reward: [(0, '68.070'), (1, '70.070')] [2023-10-08 06:01:10,095][00611] Updated weights for policy 0, policy_version 53352 (0.0009) [2023-10-08 06:01:10,463][00611] Updated weights for policy 0, policy_version 53362 (0.0009) [2023-10-08 06:01:10,839][00611] Updated weights for policy 0, policy_version 53372 (0.0009) [2023-10-08 06:01:11,504][00612] Updated weights for policy 1, policy_version 53670 (0.0010) [2023-10-08 06:01:11,866][00612] Updated weights for policy 1, policy_version 53680 (0.0009) [2023-10-08 06:01:12,232][00612] Updated weights for policy 1, policy_version 53690 (0.0010) [2023-10-08 06:01:13,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 109641728. Throughput: 0: 1842.0, 1: 1828.7. Samples: 27417054. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 06:01:13,754][130385] Avg episode reward: [(0, '71.170'), (1, '68.570')] [2023-10-08 06:01:14,386][00611] Updated weights for policy 0, policy_version 53382 (0.0009) [2023-10-08 06:01:14,756][00611] Updated weights for policy 0, policy_version 53392 (0.0010) [2023-10-08 06:01:15,129][00611] Updated weights for policy 0, policy_version 53402 (0.0010) [2023-10-08 06:01:16,097][00612] Updated weights for policy 1, policy_version 53700 (0.0010) [2023-10-08 06:01:16,494][00612] Updated weights for policy 1, policy_version 53710 (0.0007) [2023-10-08 06:01:16,862][00612] Updated weights for policy 1, policy_version 53720 (0.0007) [2023-10-08 06:01:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 109707264. Throughput: 0: 1839.0, 1: 1847.1. Samples: 27439378. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 06:01:18,755][130385] Avg episode reward: [(0, '70.690'), (1, '68.160')] [2023-10-08 06:01:18,840][00611] Updated weights for policy 0, policy_version 53412 (0.0011) [2023-10-08 06:01:19,203][00611] Updated weights for policy 0, policy_version 53422 (0.0011) [2023-10-08 06:01:19,574][00611] Updated weights for policy 0, policy_version 53432 (0.0009) [2023-10-08 06:01:20,477][00612] Updated weights for policy 1, policy_version 53730 (0.0007) [2023-10-08 06:01:20,851][00612] Updated weights for policy 1, policy_version 53740 (0.0008) [2023-10-08 06:01:21,216][00612] Updated weights for policy 1, policy_version 53750 (0.0009) [2023-10-08 06:01:21,575][00612] Updated weights for policy 1, policy_version 53760 (0.0009) [2023-10-08 06:01:23,338][00611] Updated weights for policy 0, policy_version 53442 (0.0009) [2023-10-08 06:01:23,718][00611] Updated weights for policy 0, policy_version 53452 (0.0010) [2023-10-08 06:01:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 109772800. Throughput: 0: 1837.6, 1: 1827.5. Samples: 27449720. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 06:01:23,755][130385] Avg episode reward: [(0, '72.330'), (1, '71.530')] [2023-10-08 06:01:24,084][00611] Updated weights for policy 0, policy_version 53462 (0.0009) [2023-10-08 06:01:24,453][00611] Updated weights for policy 0, policy_version 53472 (0.0008) [2023-10-08 06:01:25,312][00612] Updated weights for policy 1, policy_version 53770 (0.0008) [2023-10-08 06:01:25,686][00612] Updated weights for policy 1, policy_version 53780 (0.0009) [2023-10-08 06:01:26,049][00612] Updated weights for policy 1, policy_version 53790 (0.0008) [2023-10-08 06:01:27,899][00611] Updated weights for policy 0, policy_version 53482 (0.0008) [2023-10-08 06:01:28,268][00611] Updated weights for policy 0, policy_version 53492 (0.0010) [2023-10-08 06:01:28,653][00611] Updated weights for policy 0, policy_version 53502 (0.0009) [2023-10-08 06:01:28,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109871104. Throughput: 0: 1832.2, 1: 1840.0. Samples: 27472338. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 06:01:28,754][130385] Avg episode reward: [(0, '66.600'), (1, '72.010')] [2023-10-08 06:01:29,564][00612] Updated weights for policy 1, policy_version 53800 (0.0007) [2023-10-08 06:01:29,934][00612] Updated weights for policy 1, policy_version 53810 (0.0009) [2023-10-08 06:01:30,305][00612] Updated weights for policy 1, policy_version 53820 (0.0008) [2023-10-08 06:01:32,220][00611] Updated weights for policy 0, policy_version 53512 (0.0010) [2023-10-08 06:01:32,587][00611] Updated weights for policy 0, policy_version 53522 (0.0010) [2023-10-08 06:01:32,971][00611] Updated weights for policy 0, policy_version 53532 (0.0008) [2023-10-08 06:01:33,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 109936640. Throughput: 0: 1822.6, 1: 1844.5. Samples: 27494218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:01:33,754][130385] Avg episode reward: [(0, '69.090'), (1, '74.970')] [2023-10-08 06:01:33,831][00612] Updated weights for policy 1, policy_version 53830 (0.0008) [2023-10-08 06:01:34,200][00612] Updated weights for policy 1, policy_version 53840 (0.0008) [2023-10-08 06:01:34,568][00612] Updated weights for policy 1, policy_version 53850 (0.0007) [2023-10-08 06:01:36,682][00611] Updated weights for policy 0, policy_version 53542 (0.0007) [2023-10-08 06:01:37,068][00611] Updated weights for policy 0, policy_version 53552 (0.0009) [2023-10-08 06:01:37,432][00611] Updated weights for policy 0, policy_version 53562 (0.0010) [2023-10-08 06:01:38,291][00612] Updated weights for policy 1, policy_version 53860 (0.0009) [2023-10-08 06:01:38,654][00612] Updated weights for policy 1, policy_version 53870 (0.0009) [2023-10-08 06:01:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 110002176. Throughput: 0: 1832.8, 1: 1841.8. Samples: 27505590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:01:38,754][130385] Avg episode reward: [(0, '68.020'), (1, '73.180')] [2023-10-08 06:01:39,021][00612] Updated weights for policy 1, policy_version 53880 (0.0007) [2023-10-08 06:01:41,182][00611] Updated weights for policy 0, policy_version 53572 (0.0008) [2023-10-08 06:01:41,558][00611] Updated weights for policy 0, policy_version 53582 (0.0010) [2023-10-08 06:01:41,935][00611] Updated weights for policy 0, policy_version 53592 (0.0007) [2023-10-08 06:01:42,601][00612] Updated weights for policy 1, policy_version 53890 (0.0008) [2023-10-08 06:01:42,969][00612] Updated weights for policy 1, policy_version 53900 (0.0007) [2023-10-08 06:01:43,339][00612] Updated weights for policy 1, policy_version 53910 (0.0007) [2023-10-08 06:01:43,709][00612] Updated weights for policy 1, policy_version 53920 (0.0009) [2023-10-08 06:01:43,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 110100480. Throughput: 0: 1822.0, 1: 1840.7. Samples: 27527216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:01:43,754][130385] Avg episode reward: [(0, '70.400'), (1, '69.930')] [2023-10-08 06:01:45,611][00611] Updated weights for policy 0, policy_version 53602 (0.0007) [2023-10-08 06:01:45,979][00611] Updated weights for policy 0, policy_version 53612 (0.0009) [2023-10-08 06:01:46,355][00611] Updated weights for policy 0, policy_version 53622 (0.0008) [2023-10-08 06:01:46,727][00611] Updated weights for policy 0, policy_version 53632 (0.0009) [2023-10-08 06:01:47,196][00612] Updated weights for policy 1, policy_version 53930 (0.0007) [2023-10-08 06:01:47,559][00612] Updated weights for policy 1, policy_version 53940 (0.0008) [2023-10-08 06:01:47,939][00612] Updated weights for policy 1, policy_version 53950 (0.0010) [2023-10-08 06:01:48,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 110166016. Throughput: 0: 1835.4, 1: 1830.0. Samples: 27548654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:01:48,755][130385] Avg episode reward: [(0, '68.470'), (1, '73.630')] [2023-10-08 06:01:50,438][00611] Updated weights for policy 0, policy_version 53642 (0.0009) [2023-10-08 06:01:50,801][00611] Updated weights for policy 0, policy_version 53652 (0.0008) [2023-10-08 06:01:51,172][00611] Updated weights for policy 0, policy_version 53662 (0.0011) [2023-10-08 06:01:51,735][00612] Updated weights for policy 1, policy_version 53960 (0.0008) [2023-10-08 06:01:52,110][00612] Updated weights for policy 1, policy_version 53970 (0.0008) [2023-10-08 06:01:52,479][00612] Updated weights for policy 1, policy_version 53980 (0.0009) [2023-10-08 06:01:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 110231552. Throughput: 0: 1822.5, 1: 1838.9. Samples: 27560170. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:01:53,754][130385] Avg episode reward: [(0, '69.200'), (1, '75.700')] [2023-10-08 06:01:54,774][00611] Updated weights for policy 0, policy_version 53672 (0.0008) [2023-10-08 06:01:55,149][00611] Updated weights for policy 0, policy_version 53682 (0.0009) [2023-10-08 06:01:55,514][00611] Updated weights for policy 0, policy_version 53692 (0.0010) [2023-10-08 06:01:56,069][00612] Updated weights for policy 1, policy_version 53990 (0.0008) [2023-10-08 06:01:56,443][00612] Updated weights for policy 1, policy_version 54000 (0.0007) [2023-10-08 06:01:56,817][00612] Updated weights for policy 1, policy_version 54010 (0.0008) [2023-10-08 06:01:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 110297088. Throughput: 0: 1830.8, 1: 1838.9. Samples: 27582194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:01:58,755][130385] Avg episode reward: [(0, '69.850'), (1, '77.830')] [2023-10-08 06:01:58,756][00425] Saving new best policy, reward=77.830! [2023-10-08 06:01:59,132][00611] Updated weights for policy 0, policy_version 53702 (0.0008) [2023-10-08 06:01:59,512][00611] Updated weights for policy 0, policy_version 53712 (0.0009) [2023-10-08 06:01:59,882][00611] Updated weights for policy 0, policy_version 53722 (0.0008) [2023-10-08 06:02:00,399][00612] Updated weights for policy 1, policy_version 54020 (0.0009) [2023-10-08 06:02:00,796][00612] Updated weights for policy 1, policy_version 54030 (0.0009) [2023-10-08 06:02:01,161][00612] Updated weights for policy 1, policy_version 54040 (0.0009) [2023-10-08 06:02:03,655][00611] Updated weights for policy 0, policy_version 53732 (0.0009) [2023-10-08 06:02:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 110362624. Throughput: 0: 1827.7, 1: 1857.7. Samples: 27605220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:02:03,754][130385] Avg episode reward: [(0, '70.560'), (1, '73.740')] [2023-10-08 06:02:04,016][00611] Updated weights for policy 0, policy_version 53742 (0.0009) [2023-10-08 06:02:04,398][00611] Updated weights for policy 0, policy_version 53752 (0.0008) [2023-10-08 06:02:04,650][00612] Updated weights for policy 1, policy_version 54050 (0.0007) [2023-10-08 06:02:05,022][00612] Updated weights for policy 1, policy_version 54060 (0.0007) [2023-10-08 06:02:05,391][00612] Updated weights for policy 1, policy_version 54070 (0.0008) [2023-10-08 06:02:05,753][00612] Updated weights for policy 1, policy_version 54080 (0.0007) [2023-10-08 06:02:08,171][00611] Updated weights for policy 0, policy_version 53762 (0.0009) [2023-10-08 06:02:08,552][00611] Updated weights for policy 0, policy_version 53772 (0.0012) [2023-10-08 06:02:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 110428160. Throughput: 0: 1829.1, 1: 1850.0. Samples: 27615278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:02:08,754][130385] Avg episode reward: [(0, '67.640'), (1, '74.450')] [2023-10-08 06:02:08,929][00611] Updated weights for policy 0, policy_version 53782 (0.0009) [2023-10-08 06:02:09,229][00612] Updated weights for policy 1, policy_version 54090 (0.0007) [2023-10-08 06:02:09,301][00611] Updated weights for policy 0, policy_version 53792 (0.0009) [2023-10-08 06:02:09,601][00612] Updated weights for policy 1, policy_version 54100 (0.0009) [2023-10-08 06:02:09,970][00612] Updated weights for policy 1, policy_version 54110 (0.0011) [2023-10-08 06:02:12,819][00611] Updated weights for policy 0, policy_version 53802 (0.0009) [2023-10-08 06:02:13,190][00611] Updated weights for policy 0, policy_version 53812 (0.0008) [2023-10-08 06:02:13,505][00612] Updated weights for policy 1, policy_version 54120 (0.0008) [2023-10-08 06:02:13,561][00611] Updated weights for policy 0, policy_version 53822 (0.0007) [2023-10-08 06:02:13,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 110526464. Throughput: 0: 1825.3, 1: 1869.6. Samples: 27638606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-08 06:02:13,754][130385] Avg episode reward: [(0, '70.280'), (1, '71.370')] [2023-10-08 06:02:13,870][00612] Updated weights for policy 1, policy_version 54130 (0.0009) [2023-10-08 06:02:14,241][00612] Updated weights for policy 1, policy_version 54140 (0.0009) [2023-10-08 06:02:17,387][00611] Updated weights for policy 0, policy_version 53832 (0.0010) [2023-10-08 06:02:17,762][00611] Updated weights for policy 0, policy_version 53842 (0.0009) [2023-10-08 06:02:17,880][00612] Updated weights for policy 1, policy_version 54150 (0.0007) [2023-10-08 06:02:18,130][00611] Updated weights for policy 0, policy_version 53852 (0.0008) [2023-10-08 06:02:18,246][00612] Updated weights for policy 1, policy_version 54160 (0.0009) [2023-10-08 06:02:18,616][00612] Updated weights for policy 1, policy_version 54170 (0.0007) [2023-10-08 06:02:18,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 110592000. Throughput: 0: 1827.1, 1: 1853.2. Samples: 27659830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-08 06:02:18,754][130385] Avg episode reward: [(0, '65.680'), (1, '71.320')] [2023-10-08 06:02:18,762][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000053856_55148544.pth... [2023-10-08 06:02:18,801][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000052128_53379072.pth [2023-10-08 06:02:18,832][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000054176_55476224.pth... [2023-10-08 06:02:18,861][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000052448_53706752.pth [2023-10-08 06:02:21,967][00611] Updated weights for policy 0, policy_version 53862 (0.0008) [2023-10-08 06:02:22,203][00612] Updated weights for policy 1, policy_version 54180 (0.0009) [2023-10-08 06:02:22,355][00611] Updated weights for policy 0, policy_version 53872 (0.0009) [2023-10-08 06:02:22,579][00612] Updated weights for policy 1, policy_version 54190 (0.0008) [2023-10-08 06:02:22,725][00611] Updated weights for policy 0, policy_version 53882 (0.0007) [2023-10-08 06:02:22,943][00612] Updated weights for policy 1, policy_version 54200 (0.0008) [2023-10-08 06:02:23,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 110690304. Throughput: 0: 1820.2, 1: 1873.7. Samples: 27671816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-08 06:02:23,755][130385] Avg episode reward: [(0, '62.470'), (1, '70.130')] [2023-10-08 06:02:26,301][00611] Updated weights for policy 0, policy_version 53892 (0.0007) [2023-10-08 06:02:26,667][00611] Updated weights for policy 0, policy_version 53902 (0.0007) [2023-10-08 06:02:26,718][00612] Updated weights for policy 1, policy_version 54210 (0.0010) [2023-10-08 06:02:27,034][00611] Updated weights for policy 0, policy_version 53912 (0.0007) [2023-10-08 06:02:27,084][00612] Updated weights for policy 1, policy_version 54220 (0.0009) [2023-10-08 06:02:27,461][00612] Updated weights for policy 1, policy_version 54230 (0.0009) [2023-10-08 06:02:27,822][00612] Updated weights for policy 1, policy_version 54240 (0.0008) [2023-10-08 06:02:28,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 110755840. Throughput: 0: 1826.8, 1: 1855.6. Samples: 27692926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-08 06:02:28,755][130385] Avg episode reward: [(0, '65.770'), (1, '65.040')] [2023-10-08 06:02:30,524][00611] Updated weights for policy 0, policy_version 53922 (0.0007) [2023-10-08 06:02:30,908][00611] Updated weights for policy 0, policy_version 53932 (0.0009) [2023-10-08 06:02:31,278][00611] Updated weights for policy 0, policy_version 53942 (0.0008) [2023-10-08 06:02:31,391][00612] Updated weights for policy 1, policy_version 54250 (0.0008) [2023-10-08 06:02:31,643][00611] Updated weights for policy 0, policy_version 53952 (0.0007) [2023-10-08 06:02:31,758][00612] Updated weights for policy 1, policy_version 54260 (0.0008) [2023-10-08 06:02:32,125][00612] Updated weights for policy 1, policy_version 54270 (0.0007) [2023-10-08 06:02:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 110821376. Throughput: 0: 1824.1, 1: 1872.9. Samples: 27715020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-08 06:02:33,754][130385] Avg episode reward: [(0, '65.740'), (1, '67.420')] [2023-10-08 06:02:35,320][00611] Updated weights for policy 0, policy_version 53962 (0.0008) [2023-10-08 06:02:35,681][00611] Updated weights for policy 0, policy_version 53972 (0.0008) [2023-10-08 06:02:35,732][00612] Updated weights for policy 1, policy_version 54280 (0.0007) [2023-10-08 06:02:36,052][00611] Updated weights for policy 0, policy_version 53982 (0.0007) [2023-10-08 06:02:36,105][00612] Updated weights for policy 1, policy_version 54290 (0.0007) [2023-10-08 06:02:36,465][00612] Updated weights for policy 1, policy_version 54300 (0.0009) [2023-10-08 06:02:38,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 110886912. Throughput: 0: 1824.8, 1: 1856.5. Samples: 27725828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-08 06:02:38,754][130385] Avg episode reward: [(0, '66.230'), (1, '66.470')] [2023-10-08 06:02:39,704][00611] Updated weights for policy 0, policy_version 53992 (0.0007) [2023-10-08 06:02:39,996][00612] Updated weights for policy 1, policy_version 54310 (0.0009) [2023-10-08 06:02:40,074][00611] Updated weights for policy 0, policy_version 54002 (0.0008) [2023-10-08 06:02:40,376][00612] Updated weights for policy 1, policy_version 54320 (0.0011) [2023-10-08 06:02:40,447][00611] Updated weights for policy 0, policy_version 54012 (0.0007) [2023-10-08 06:02:40,744][00612] Updated weights for policy 1, policy_version 54330 (0.0009) [2023-10-08 06:02:43,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 110952448. Throughput: 0: 1824.4, 1: 1870.3. Samples: 27748454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-08 06:02:43,755][130385] Avg episode reward: [(0, '65.190'), (1, '69.330')] [2023-10-08 06:02:43,934][00611] Updated weights for policy 0, policy_version 54022 (0.0009) [2023-10-08 06:02:44,313][00611] Updated weights for policy 0, policy_version 54032 (0.0009) [2023-10-08 06:02:44,404][00612] Updated weights for policy 1, policy_version 54340 (0.0010) [2023-10-08 06:02:44,684][00611] Updated weights for policy 0, policy_version 54042 (0.0008) [2023-10-08 06:02:44,782][00612] Updated weights for policy 1, policy_version 54350 (0.0007) [2023-10-08 06:02:45,152][00612] Updated weights for policy 1, policy_version 54360 (0.0008) [2023-10-08 06:02:48,460][00611] Updated weights for policy 0, policy_version 54052 (0.0010) [2023-10-08 06:02:48,713][00612] Updated weights for policy 1, policy_version 54370 (0.0008) [2023-10-08 06:02:48,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 111017984. Throughput: 0: 1821.5, 1: 1868.9. Samples: 27771290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:02:48,755][130385] Avg episode reward: [(0, '63.980'), (1, '72.510')] [2023-10-08 06:02:48,830][00611] Updated weights for policy 0, policy_version 54062 (0.0008) [2023-10-08 06:02:49,117][00612] Updated weights for policy 1, policy_version 54380 (0.0007) [2023-10-08 06:02:49,210][00611] Updated weights for policy 0, policy_version 54072 (0.0007) [2023-10-08 06:02:49,478][00612] Updated weights for policy 1, policy_version 54390 (0.0009) [2023-10-08 06:02:49,842][00612] Updated weights for policy 1, policy_version 54400 (0.0011) [2023-10-08 06:02:52,944][00611] Updated weights for policy 0, policy_version 54082 (0.0008) [2023-10-08 06:02:53,308][00611] Updated weights for policy 0, policy_version 54092 (0.0008) [2023-10-08 06:02:53,513][00612] Updated weights for policy 1, policy_version 54410 (0.0007) [2023-10-08 06:02:53,687][00611] Updated weights for policy 0, policy_version 54102 (0.0007) [2023-10-08 06:02:53,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 111083520. Throughput: 0: 1819.4, 1: 1860.2. Samples: 27780860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:02:53,754][130385] Avg episode reward: [(0, '63.780'), (1, '67.960')] [2023-10-08 06:02:53,875][00612] Updated weights for policy 1, policy_version 54420 (0.0010) [2023-10-08 06:02:54,054][00611] Updated weights for policy 0, policy_version 54112 (0.0008) [2023-10-08 06:02:54,243][00612] Updated weights for policy 1, policy_version 54430 (0.0009) [2023-10-08 06:02:57,963][00611] Updated weights for policy 0, policy_version 54122 (0.0008) [2023-10-08 06:02:58,125][00612] Updated weights for policy 1, policy_version 54440 (0.0008) [2023-10-08 06:02:58,332][00611] Updated weights for policy 0, policy_version 54132 (0.0009) [2023-10-08 06:02:58,491][00612] Updated weights for policy 1, policy_version 54450 (0.0008) [2023-10-08 06:02:58,705][00611] Updated weights for policy 0, policy_version 54142 (0.0009) [2023-10-08 06:02:58,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 111149056. Throughput: 0: 1823.0, 1: 1841.5. Samples: 27803508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:02:58,755][130385] Avg episode reward: [(0, '64.080'), (1, '68.360')] [2023-10-08 06:02:58,859][00612] Updated weights for policy 1, policy_version 54460 (0.0009) [2023-10-08 06:03:02,362][00611] Updated weights for policy 0, policy_version 54152 (0.0007) [2023-10-08 06:03:02,412][00612] Updated weights for policy 1, policy_version 54470 (0.0010) [2023-10-08 06:03:02,735][00611] Updated weights for policy 0, policy_version 54162 (0.0009) [2023-10-08 06:03:02,774][00612] Updated weights for policy 1, policy_version 54480 (0.0007) [2023-10-08 06:03:03,111][00611] Updated weights for policy 0, policy_version 54172 (0.0007) [2023-10-08 06:03:03,132][00612] Updated weights for policy 1, policy_version 54490 (0.0008) [2023-10-08 06:03:03,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 111280128. Throughput: 0: 1823.2, 1: 1828.0. Samples: 27824134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:03:03,755][130385] Avg episode reward: [(0, '63.500'), (1, '67.660')] [2023-10-08 06:03:06,880][00611] Updated weights for policy 0, policy_version 54182 (0.0007) [2023-10-08 06:03:06,889][00612] Updated weights for policy 1, policy_version 54500 (0.0008) [2023-10-08 06:03:07,256][00612] Updated weights for policy 1, policy_version 54510 (0.0008) [2023-10-08 06:03:07,257][00611] Updated weights for policy 0, policy_version 54192 (0.0007) [2023-10-08 06:03:07,631][00612] Updated weights for policy 1, policy_version 54520 (0.0008) [2023-10-08 06:03:07,637][00611] Updated weights for policy 0, policy_version 54202 (0.0007) [2023-10-08 06:03:08,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 111345664. Throughput: 0: 1821.2, 1: 1836.1. Samples: 27836396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:03:08,755][130385] Avg episode reward: [(0, '63.270'), (1, '67.100')] [2023-10-08 06:03:11,214][00611] Updated weights for policy 0, policy_version 54212 (0.0008) [2023-10-08 06:03:11,359][00612] Updated weights for policy 1, policy_version 54530 (0.0010) [2023-10-08 06:03:11,581][00611] Updated weights for policy 0, policy_version 54222 (0.0008) [2023-10-08 06:03:11,721][00612] Updated weights for policy 1, policy_version 54540 (0.0007) [2023-10-08 06:03:11,955][00611] Updated weights for policy 0, policy_version 54232 (0.0009) [2023-10-08 06:03:12,090][00612] Updated weights for policy 1, policy_version 54550 (0.0008) [2023-10-08 06:03:12,475][00612] Updated weights for policy 1, policy_version 54560 (0.0011) [2023-10-08 06:03:13,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 111411200. Throughput: 0: 1820.3, 1: 1823.7. Samples: 27856906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:03:13,754][130385] Avg episode reward: [(0, '61.420'), (1, '69.620')] [2023-10-08 06:03:15,683][00611] Updated weights for policy 0, policy_version 54242 (0.0008) [2023-10-08 06:03:16,055][00611] Updated weights for policy 0, policy_version 54252 (0.0008) [2023-10-08 06:03:16,125][00612] Updated weights for policy 1, policy_version 54570 (0.0008) [2023-10-08 06:03:16,419][00611] Updated weights for policy 0, policy_version 54262 (0.0008) [2023-10-08 06:03:16,489][00612] Updated weights for policy 1, policy_version 54580 (0.0009) [2023-10-08 06:03:16,786][00611] Updated weights for policy 0, policy_version 54272 (0.0008) [2023-10-08 06:03:16,859][00612] Updated weights for policy 1, policy_version 54590 (0.0009) [2023-10-08 06:03:18,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 111476736. Throughput: 0: 1819.9, 1: 1827.7. Samples: 27879164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:03:18,754][130385] Avg episode reward: [(0, '57.820'), (1, '69.620')] [2023-10-08 06:03:20,454][00611] Updated weights for policy 0, policy_version 54282 (0.0009) [2023-10-08 06:03:20,578][00612] Updated weights for policy 1, policy_version 54600 (0.0010) [2023-10-08 06:03:20,834][00611] Updated weights for policy 0, policy_version 54292 (0.0008) [2023-10-08 06:03:20,939][00612] Updated weights for policy 1, policy_version 54610 (0.0009) [2023-10-08 06:03:21,191][00611] Updated weights for policy 0, policy_version 54302 (0.0008) [2023-10-08 06:03:21,308][00612] Updated weights for policy 1, policy_version 54620 (0.0008) [2023-10-08 06:03:23,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 111542272. Throughput: 0: 1821.5, 1: 1818.6. Samples: 27889632. Policy #0 lag: (min: 19.0, avg: 19.8, max: 39.0) [2023-10-08 06:03:23,755][130385] Avg episode reward: [(0, '52.970'), (1, '75.060')] [2023-10-08 06:03:24,637][00611] Updated weights for policy 0, policy_version 54312 (0.0010) [2023-10-08 06:03:24,944][00612] Updated weights for policy 1, policy_version 54630 (0.0009) [2023-10-08 06:03:25,009][00611] Updated weights for policy 0, policy_version 54322 (0.0010) [2023-10-08 06:03:25,309][00612] Updated weights for policy 1, policy_version 54640 (0.0009) [2023-10-08 06:03:25,377][00611] Updated weights for policy 0, policy_version 54332 (0.0007) [2023-10-08 06:03:25,668][00612] Updated weights for policy 1, policy_version 54650 (0.0008) [2023-10-08 06:03:28,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 111607808. Throughput: 0: 1823.5, 1: 1821.1. Samples: 27912462. Policy #0 lag: (min: 19.0, avg: 19.8, max: 39.0) [2023-10-08 06:03:28,754][130385] Avg episode reward: [(0, '53.470'), (1, '76.260')] [2023-10-08 06:03:28,958][00611] Updated weights for policy 0, policy_version 54342 (0.0009) [2023-10-08 06:03:29,331][00611] Updated weights for policy 0, policy_version 54352 (0.0007) [2023-10-08 06:03:29,377][00612] Updated weights for policy 1, policy_version 54660 (0.0007) [2023-10-08 06:03:29,695][00611] Updated weights for policy 0, policy_version 54362 (0.0007) [2023-10-08 06:03:29,737][00612] Updated weights for policy 1, policy_version 54670 (0.0007) [2023-10-08 06:03:30,103][00612] Updated weights for policy 1, policy_version 54680 (0.0007) [2023-10-08 06:03:33,364][00611] Updated weights for policy 0, policy_version 54372 (0.0008) [2023-10-08 06:03:33,739][00611] Updated weights for policy 0, policy_version 54382 (0.0009) [2023-10-08 06:03:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 111673344. Throughput: 0: 1832.9, 1: 1815.5. Samples: 27935466. Policy #0 lag: (min: 19.0, avg: 19.8, max: 39.0) [2023-10-08 06:03:33,754][130385] Avg episode reward: [(0, '55.570'), (1, '73.280')] [2023-10-08 06:03:33,841][00612] Updated weights for policy 1, policy_version 54690 (0.0010) [2023-10-08 06:03:34,111][00611] Updated weights for policy 0, policy_version 54392 (0.0008) [2023-10-08 06:03:34,235][00612] Updated weights for policy 1, policy_version 54700 (0.0007) [2023-10-08 06:03:34,596][00612] Updated weights for policy 1, policy_version 54710 (0.0007) [2023-10-08 06:03:34,958][00612] Updated weights for policy 1, policy_version 54720 (0.0008) [2023-10-08 06:03:37,821][00611] Updated weights for policy 0, policy_version 54402 (0.0008) [2023-10-08 06:03:38,199][00611] Updated weights for policy 0, policy_version 54412 (0.0010) [2023-10-08 06:03:38,566][00611] Updated weights for policy 0, policy_version 54422 (0.0007) [2023-10-08 06:03:38,611][00612] Updated weights for policy 1, policy_version 54730 (0.0008) [2023-10-08 06:03:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 111738880. Throughput: 0: 1831.8, 1: 1818.9. Samples: 27945144. Policy #0 lag: (min: 19.0, avg: 19.8, max: 39.0) [2023-10-08 06:03:38,754][130385] Avg episode reward: [(0, '55.360'), (1, '79.430')] [2023-10-08 06:03:38,936][00611] Updated weights for policy 0, policy_version 54432 (0.0007) [2023-10-08 06:03:38,975][00612] Updated weights for policy 1, policy_version 54740 (0.0008) [2023-10-08 06:03:39,353][00612] Updated weights for policy 1, policy_version 54750 (0.0007) [2023-10-08 06:03:39,428][00425] Saving new best policy, reward=79.430! [2023-10-08 06:03:42,701][00611] Updated weights for policy 0, policy_version 54442 (0.0009) [2023-10-08 06:03:42,913][00612] Updated weights for policy 1, policy_version 54760 (0.0007) [2023-10-08 06:03:43,074][00611] Updated weights for policy 0, policy_version 54452 (0.0010) [2023-10-08 06:03:43,289][00612] Updated weights for policy 1, policy_version 54770 (0.0007) [2023-10-08 06:03:43,445][00611] Updated weights for policy 0, policy_version 54462 (0.0008) [2023-10-08 06:03:43,646][00612] Updated weights for policy 1, policy_version 54780 (0.0007) [2023-10-08 06:03:43,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 111837184. Throughput: 0: 1826.1, 1: 1832.5. Samples: 27968144. Policy #0 lag: (min: 19.0, avg: 19.8, max: 39.0) [2023-10-08 06:03:43,755][130385] Avg episode reward: [(0, '57.650'), (1, '79.730')] [2023-10-08 06:03:43,793][00425] Saving new best policy, reward=79.730! [2023-10-08 06:03:47,243][00612] Updated weights for policy 1, policy_version 54790 (0.0008) [2023-10-08 06:03:47,251][00611] Updated weights for policy 0, policy_version 54472 (0.0009) [2023-10-08 06:03:47,617][00612] Updated weights for policy 1, policy_version 54800 (0.0007) [2023-10-08 06:03:47,626][00611] Updated weights for policy 0, policy_version 54482 (0.0008) [2023-10-08 06:03:47,986][00611] Updated weights for policy 0, policy_version 54492 (0.0009) [2023-10-08 06:03:47,990][00612] Updated weights for policy 1, policy_version 54810 (0.0008) [2023-10-08 06:03:48,754][130385] Fps is (10 sec: 19660.0, 60 sec: 15291.7, 300 sec: 14773.3). Total num frames: 111935488. Throughput: 0: 1820.6, 1: 1828.7. Samples: 27988352. Policy #0 lag: (min: 19.0, avg: 19.8, max: 39.0) [2023-10-08 06:03:48,755][130385] Avg episode reward: [(0, '57.770'), (1, '76.040')] [2023-10-08 06:03:51,674][00611] Updated weights for policy 0, policy_version 54502 (0.0009) [2023-10-08 06:03:51,752][00612] Updated weights for policy 1, policy_version 54820 (0.0008) [2023-10-08 06:03:52,042][00611] Updated weights for policy 0, policy_version 54512 (0.0008) [2023-10-08 06:03:52,119][00612] Updated weights for policy 1, policy_version 54830 (0.0009) [2023-10-08 06:03:52,419][00611] Updated weights for policy 0, policy_version 54522 (0.0008) [2023-10-08 06:03:52,483][00612] Updated weights for policy 1, policy_version 54840 (0.0008) [2023-10-08 06:03:53,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 112001024. Throughput: 0: 1826.1, 1: 1834.9. Samples: 28001136. Policy #0 lag: (min: 19.0, avg: 19.8, max: 39.0) [2023-10-08 06:03:53,754][130385] Avg episode reward: [(0, '59.390'), (1, '77.300')] [2023-10-08 06:03:55,910][00611] Updated weights for policy 0, policy_version 54532 (0.0008) [2023-10-08 06:03:56,176][00612] Updated weights for policy 1, policy_version 54850 (0.0009) [2023-10-08 06:03:56,277][00611] Updated weights for policy 0, policy_version 54542 (0.0007) [2023-10-08 06:03:56,541][00612] Updated weights for policy 1, policy_version 54860 (0.0009) [2023-10-08 06:03:56,648][00611] Updated weights for policy 0, policy_version 54552 (0.0007) [2023-10-08 06:03:56,915][00612] Updated weights for policy 1, policy_version 54870 (0.0008) [2023-10-08 06:03:57,286][00612] Updated weights for policy 1, policy_version 54880 (0.0008) [2023-10-08 06:03:58,754][130385] Fps is (10 sec: 13107.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 112066560. Throughput: 0: 1821.9, 1: 1828.3. Samples: 28021164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:03:58,754][130385] Avg episode reward: [(0, '61.000'), (1, '75.740')] [2023-10-08 06:04:00,253][00611] Updated weights for policy 0, policy_version 54562 (0.0008) [2023-10-08 06:04:00,629][00611] Updated weights for policy 0, policy_version 54572 (0.0008) [2023-10-08 06:04:00,886][00612] Updated weights for policy 1, policy_version 54890 (0.0009) [2023-10-08 06:04:01,011][00611] Updated weights for policy 0, policy_version 54582 (0.0008) [2023-10-08 06:04:01,261][00612] Updated weights for policy 1, policy_version 54900 (0.0008) [2023-10-08 06:04:01,375][00611] Updated weights for policy 0, policy_version 54592 (0.0007) [2023-10-08 06:04:01,629][00612] Updated weights for policy 1, policy_version 54910 (0.0008) [2023-10-08 06:04:03,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 112132096. Throughput: 0: 1831.1, 1: 1836.0. Samples: 28044184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:04:03,755][130385] Avg episode reward: [(0, '62.510'), (1, '78.380')] [2023-10-08 06:04:05,188][00611] Updated weights for policy 0, policy_version 54602 (0.0010) [2023-10-08 06:04:05,290][00612] Updated weights for policy 1, policy_version 54920 (0.0008) [2023-10-08 06:04:05,558][00611] Updated weights for policy 0, policy_version 54612 (0.0009) [2023-10-08 06:04:05,657][00612] Updated weights for policy 1, policy_version 54930 (0.0009) [2023-10-08 06:04:05,920][00611] Updated weights for policy 0, policy_version 54622 (0.0007) [2023-10-08 06:04:06,014][00612] Updated weights for policy 1, policy_version 54940 (0.0007) [2023-10-08 06:04:08,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 112197632. Throughput: 0: 1824.5, 1: 1827.2. Samples: 28053960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:04:08,755][130385] Avg episode reward: [(0, '66.150'), (1, '83.130')] [2023-10-08 06:04:08,757][00425] Saving new best policy, reward=83.130! [2023-10-08 06:04:09,531][00612] Updated weights for policy 1, policy_version 54950 (0.0008) [2023-10-08 06:04:09,605][00611] Updated weights for policy 0, policy_version 54632 (0.0008) [2023-10-08 06:04:09,901][00612] Updated weights for policy 1, policy_version 54960 (0.0007) [2023-10-08 06:04:09,980][00611] Updated weights for policy 0, policy_version 54642 (0.0008) [2023-10-08 06:04:10,267][00612] Updated weights for policy 1, policy_version 54970 (0.0007) [2023-10-08 06:04:10,341][00611] Updated weights for policy 0, policy_version 54652 (0.0010) [2023-10-08 06:04:13,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 112263168. Throughput: 0: 1820.4, 1: 1843.5. Samples: 28077336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:04:13,755][130385] Avg episode reward: [(0, '63.830'), (1, '81.720')] [2023-10-08 06:04:13,790][00612] Updated weights for policy 1, policy_version 54980 (0.0009) [2023-10-08 06:04:13,853][00611] Updated weights for policy 0, policy_version 54662 (0.0007) [2023-10-08 06:04:14,158][00612] Updated weights for policy 1, policy_version 54990 (0.0009) [2023-10-08 06:04:14,229][00611] Updated weights for policy 0, policy_version 54672 (0.0008) [2023-10-08 06:04:14,526][00612] Updated weights for policy 1, policy_version 55000 (0.0008) [2023-10-08 06:04:14,601][00611] Updated weights for policy 0, policy_version 54682 (0.0008) [2023-10-08 06:04:18,069][00612] Updated weights for policy 1, policy_version 55010 (0.0008) [2023-10-08 06:04:18,237][00611] Updated weights for policy 0, policy_version 54692 (0.0008) [2023-10-08 06:04:18,467][00612] Updated weights for policy 1, policy_version 55020 (0.0008) [2023-10-08 06:04:18,600][00611] Updated weights for policy 0, policy_version 54702 (0.0010) [2023-10-08 06:04:18,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 112328704. Throughput: 0: 1822.5, 1: 1849.2. Samples: 28100692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:04:18,754][130385] Avg episode reward: [(0, '63.640'), (1, '75.090')] [2023-10-08 06:04:18,827][00612] Updated weights for policy 1, policy_version 55030 (0.0007) [2023-10-08 06:04:18,968][00611] Updated weights for policy 0, policy_version 54712 (0.0008) [2023-10-08 06:04:19,200][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000055040_56360960.pth... [2023-10-08 06:04:19,205][00612] Updated weights for policy 1, policy_version 55040 (0.0007) [2023-10-08 06:04:19,230][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000053312_54591488.pth [2023-10-08 06:04:19,267][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000054720_56033280.pth... [2023-10-08 06:04:19,296][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000052992_54263808.pth [2023-10-08 06:04:22,737][00611] Updated weights for policy 0, policy_version 54722 (0.0008) [2023-10-08 06:04:22,885][00612] Updated weights for policy 1, policy_version 55050 (0.0008) [2023-10-08 06:04:23,108][00611] Updated weights for policy 0, policy_version 54732 (0.0008) [2023-10-08 06:04:23,253][00612] Updated weights for policy 1, policy_version 55060 (0.0007) [2023-10-08 06:04:23,488][00611] Updated weights for policy 0, policy_version 54742 (0.0009) [2023-10-08 06:04:23,613][00612] Updated weights for policy 1, policy_version 55070 (0.0007) [2023-10-08 06:04:23,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 112427008. Throughput: 0: 1825.1, 1: 1854.5. Samples: 28110726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:04:23,754][130385] Avg episode reward: [(0, '64.750'), (1, '72.450')] [2023-10-08 06:04:23,850][00611] Updated weights for policy 0, policy_version 54752 (0.0009) [2023-10-08 06:04:27,236][00612] Updated weights for policy 1, policy_version 55080 (0.0010) [2023-10-08 06:04:27,486][00611] Updated weights for policy 0, policy_version 54762 (0.0009) [2023-10-08 06:04:27,600][00612] Updated weights for policy 1, policy_version 55090 (0.0008) [2023-10-08 06:04:27,865][00611] Updated weights for policy 0, policy_version 54772 (0.0009) [2023-10-08 06:04:27,966][00612] Updated weights for policy 1, policy_version 55100 (0.0009) [2023-10-08 06:04:28,234][00611] Updated weights for policy 0, policy_version 54782 (0.0009) [2023-10-08 06:04:28,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 112525312. Throughput: 0: 1830.5, 1: 1845.3. Samples: 28133556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:04:28,754][130385] Avg episode reward: [(0, '64.890'), (1, '69.870')] [2023-10-08 06:04:31,579][00612] Updated weights for policy 1, policy_version 55110 (0.0009) [2023-10-08 06:04:31,936][00611] Updated weights for policy 0, policy_version 54792 (0.0008) [2023-10-08 06:04:31,948][00612] Updated weights for policy 1, policy_version 55120 (0.0007) [2023-10-08 06:04:32,312][00611] Updated weights for policy 0, policy_version 54802 (0.0007) [2023-10-08 06:04:32,326][00612] Updated weights for policy 1, policy_version 55130 (0.0008) [2023-10-08 06:04:32,688][00611] Updated weights for policy 0, policy_version 54812 (0.0007) [2023-10-08 06:04:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 112590848. Throughput: 0: 1831.7, 1: 1847.1. Samples: 28153900. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:04:33,754][130385] Avg episode reward: [(0, '64.450'), (1, '72.520')] [2023-10-08 06:04:36,032][00612] Updated weights for policy 1, policy_version 55140 (0.0007) [2023-10-08 06:04:36,187][00611] Updated weights for policy 0, policy_version 54822 (0.0008) [2023-10-08 06:04:36,407][00612] Updated weights for policy 1, policy_version 55150 (0.0008) [2023-10-08 06:04:36,551][00611] Updated weights for policy 0, policy_version 54832 (0.0008) [2023-10-08 06:04:36,784][00612] Updated weights for policy 1, policy_version 55160 (0.0007) [2023-10-08 06:04:36,914][00611] Updated weights for policy 0, policy_version 54842 (0.0007) [2023-10-08 06:04:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 112656384. Throughput: 0: 1833.7, 1: 1838.4. Samples: 28166382. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:04:38,754][130385] Avg episode reward: [(0, '63.240'), (1, '66.790')] [2023-10-08 06:04:40,335][00612] Updated weights for policy 1, policy_version 55170 (0.0007) [2023-10-08 06:04:40,701][00612] Updated weights for policy 1, policy_version 55180 (0.0007) [2023-10-08 06:04:40,720][00611] Updated weights for policy 0, policy_version 54852 (0.0007) [2023-10-08 06:04:41,064][00612] Updated weights for policy 1, policy_version 55190 (0.0008) [2023-10-08 06:04:41,107][00611] Updated weights for policy 0, policy_version 54862 (0.0007) [2023-10-08 06:04:41,430][00612] Updated weights for policy 1, policy_version 55200 (0.0007) [2023-10-08 06:04:41,471][00611] Updated weights for policy 0, policy_version 54872 (0.0007) [2023-10-08 06:04:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 112721920. Throughput: 0: 1837.2, 1: 1848.0. Samples: 28186996. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:04:43,754][130385] Avg episode reward: [(0, '61.530'), (1, '67.550')] [2023-10-08 06:04:44,877][00611] Updated weights for policy 0, policy_version 54882 (0.0008) [2023-10-08 06:04:44,971][00612] Updated weights for policy 1, policy_version 55210 (0.0007) [2023-10-08 06:04:45,235][00611] Updated weights for policy 0, policy_version 54892 (0.0008) [2023-10-08 06:04:45,339][00612] Updated weights for policy 1, policy_version 55220 (0.0007) [2023-10-08 06:04:45,612][00611] Updated weights for policy 0, policy_version 54902 (0.0009) [2023-10-08 06:04:45,704][00612] Updated weights for policy 1, policy_version 55230 (0.0008) [2023-10-08 06:04:45,990][00611] Updated weights for policy 0, policy_version 54912 (0.0008) [2023-10-08 06:04:48,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 112787456. Throughput: 0: 1838.7, 1: 1856.8. Samples: 28210482. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:04:48,755][130385] Avg episode reward: [(0, '62.910'), (1, '68.540')] [2023-10-08 06:04:49,343][00612] Updated weights for policy 1, policy_version 55240 (0.0007) [2023-10-08 06:04:49,712][00612] Updated weights for policy 1, policy_version 55250 (0.0009) [2023-10-08 06:04:49,714][00611] Updated weights for policy 0, policy_version 54922 (0.0010) [2023-10-08 06:04:50,074][00612] Updated weights for policy 1, policy_version 55260 (0.0007) [2023-10-08 06:04:50,085][00611] Updated weights for policy 0, policy_version 54932 (0.0008) [2023-10-08 06:04:50,451][00611] Updated weights for policy 0, policy_version 54942 (0.0008) [2023-10-08 06:04:53,714][00612] Updated weights for policy 1, policy_version 55270 (0.0007) [2023-10-08 06:04:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 112852992. Throughput: 0: 1841.6, 1: 1858.4. Samples: 28220460. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:04:53,754][130385] Avg episode reward: [(0, '63.490'), (1, '70.060')] [2023-10-08 06:04:54,087][00612] Updated weights for policy 1, policy_version 55280 (0.0009) [2023-10-08 06:04:54,170][00611] Updated weights for policy 0, policy_version 54952 (0.0008) [2023-10-08 06:04:54,465][00612] Updated weights for policy 1, policy_version 55290 (0.0010) [2023-10-08 06:04:54,535][00611] Updated weights for policy 0, policy_version 54962 (0.0008) [2023-10-08 06:04:54,903][00611] Updated weights for policy 0, policy_version 54972 (0.0009) [2023-10-08 06:04:58,095][00612] Updated weights for policy 1, policy_version 55300 (0.0008) [2023-10-08 06:04:58,462][00612] Updated weights for policy 1, policy_version 55310 (0.0008) [2023-10-08 06:04:58,524][00611] Updated weights for policy 0, policy_version 54982 (0.0008) [2023-10-08 06:04:58,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 112918528. Throughput: 0: 1839.4, 1: 1855.2. Samples: 28243596. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:04:58,755][130385] Avg episode reward: [(0, '57.890'), (1, '73.200')] [2023-10-08 06:04:58,825][00612] Updated weights for policy 1, policy_version 55320 (0.0009) [2023-10-08 06:04:58,903][00611] Updated weights for policy 0, policy_version 54992 (0.0008) [2023-10-08 06:04:59,270][00611] Updated weights for policy 0, policy_version 55002 (0.0007) [2023-10-08 06:05:02,562][00612] Updated weights for policy 1, policy_version 55330 (0.0008) [2023-10-08 06:05:02,864][00611] Updated weights for policy 0, policy_version 55012 (0.0008) [2023-10-08 06:05:02,930][00612] Updated weights for policy 1, policy_version 55340 (0.0008) [2023-10-08 06:05:03,244][00611] Updated weights for policy 0, policy_version 55022 (0.0008) [2023-10-08 06:05:03,295][00612] Updated weights for policy 1, policy_version 55350 (0.0007) [2023-10-08 06:05:03,610][00611] Updated weights for policy 0, policy_version 55032 (0.0008) [2023-10-08 06:05:03,658][00612] Updated weights for policy 1, policy_version 55360 (0.0008) [2023-10-08 06:05:03,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 113016832. Throughput: 0: 1824.8, 1: 1834.3. Samples: 28265352. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:05:03,754][130385] Avg episode reward: [(0, '58.520'), (1, '72.710')] [2023-10-08 06:05:07,341][00611] Updated weights for policy 0, policy_version 55042 (0.0009) [2023-10-08 06:05:07,488][00612] Updated weights for policy 1, policy_version 55370 (0.0008) [2023-10-08 06:05:07,717][00611] Updated weights for policy 0, policy_version 55052 (0.0008) [2023-10-08 06:05:07,857][00612] Updated weights for policy 1, policy_version 55380 (0.0007) [2023-10-08 06:05:08,089][00611] Updated weights for policy 0, policy_version 55062 (0.0009) [2023-10-08 06:05:08,226][00612] Updated weights for policy 1, policy_version 55390 (0.0007) [2023-10-08 06:05:08,456][00611] Updated weights for policy 0, policy_version 55072 (0.0009) [2023-10-08 06:05:08,754][130385] Fps is (10 sec: 19661.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 113115136. Throughput: 0: 1833.6, 1: 1847.3. Samples: 28276366. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 06:05:08,754][130385] Avg episode reward: [(0, '56.720'), (1, '73.970')] [2023-10-08 06:05:11,860][00612] Updated weights for policy 1, policy_version 55400 (0.0008) [2023-10-08 06:05:12,184][00611] Updated weights for policy 0, policy_version 55082 (0.0007) [2023-10-08 06:05:12,224][00612] Updated weights for policy 1, policy_version 55410 (0.0008) [2023-10-08 06:05:12,552][00611] Updated weights for policy 0, policy_version 55092 (0.0008) [2023-10-08 06:05:12,594][00612] Updated weights for policy 1, policy_version 55420 (0.0009) [2023-10-08 06:05:12,932][00611] Updated weights for policy 0, policy_version 55102 (0.0007) [2023-10-08 06:05:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 113180672. Throughput: 0: 1822.2, 1: 1836.5. Samples: 28298198. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 06:05:13,754][130385] Avg episode reward: [(0, '59.790'), (1, '72.600')] [2023-10-08 06:05:16,047][00612] Updated weights for policy 1, policy_version 55430 (0.0008) [2023-10-08 06:05:16,419][00612] Updated weights for policy 1, policy_version 55440 (0.0007) [2023-10-08 06:05:16,573][00611] Updated weights for policy 0, policy_version 55112 (0.0007) [2023-10-08 06:05:16,785][00612] Updated weights for policy 1, policy_version 55450 (0.0009) [2023-10-08 06:05:16,950][00611] Updated weights for policy 0, policy_version 55122 (0.0009) [2023-10-08 06:05:17,320][00611] Updated weights for policy 0, policy_version 55132 (0.0007) [2023-10-08 06:05:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 113246208. Throughput: 0: 1830.7, 1: 1849.2. Samples: 28319494. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 06:05:18,754][130385] Avg episode reward: [(0, '61.210'), (1, '69.080')] [2023-10-08 06:05:20,516][00612] Updated weights for policy 1, policy_version 55460 (0.0008) [2023-10-08 06:05:20,881][00612] Updated weights for policy 1, policy_version 55470 (0.0010) [2023-10-08 06:05:20,964][00611] Updated weights for policy 0, policy_version 55142 (0.0008) [2023-10-08 06:05:21,244][00612] Updated weights for policy 1, policy_version 55480 (0.0007) [2023-10-08 06:05:21,341][00611] Updated weights for policy 0, policy_version 55152 (0.0009) [2023-10-08 06:05:21,721][00611] Updated weights for policy 0, policy_version 55162 (0.0007) [2023-10-08 06:05:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 113311744. Throughput: 0: 1822.0, 1: 1836.4. Samples: 28331008. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 06:05:23,754][130385] Avg episode reward: [(0, '58.810'), (1, '72.820')] [2023-10-08 06:05:25,037][00612] Updated weights for policy 1, policy_version 55490 (0.0008) [2023-10-08 06:05:25,327][00611] Updated weights for policy 0, policy_version 55172 (0.0008) [2023-10-08 06:05:25,405][00612] Updated weights for policy 1, policy_version 55500 (0.0010) [2023-10-08 06:05:25,688][00611] Updated weights for policy 0, policy_version 55182 (0.0009) [2023-10-08 06:05:25,779][00612] Updated weights for policy 1, policy_version 55510 (0.0009) [2023-10-08 06:05:26,062][00611] Updated weights for policy 0, policy_version 55192 (0.0007) [2023-10-08 06:05:26,148][00612] Updated weights for policy 1, policy_version 55520 (0.0009) [2023-10-08 06:05:28,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 113377280. Throughput: 0: 1834.3, 1: 1839.2. Samples: 28352300. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 06:05:28,755][130385] Avg episode reward: [(0, '57.980'), (1, '72.440')] [2023-10-08 06:05:29,697][00611] Updated weights for policy 0, policy_version 55202 (0.0009) [2023-10-08 06:05:29,921][00612] Updated weights for policy 1, policy_version 55530 (0.0007) [2023-10-08 06:05:30,098][00611] Updated weights for policy 0, policy_version 55212 (0.0007) [2023-10-08 06:05:30,293][00612] Updated weights for policy 1, policy_version 55540 (0.0008) [2023-10-08 06:05:30,464][00611] Updated weights for policy 0, policy_version 55222 (0.0008) [2023-10-08 06:05:30,658][00612] Updated weights for policy 1, policy_version 55550 (0.0009) [2023-10-08 06:05:30,837][00611] Updated weights for policy 0, policy_version 55232 (0.0009) [2023-10-08 06:05:33,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 113442816. Throughput: 0: 1832.5, 1: 1831.8. Samples: 28375374. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 06:05:33,755][130385] Avg episode reward: [(0, '59.010'), (1, '72.700')] [2023-10-08 06:05:34,267][00612] Updated weights for policy 1, policy_version 55560 (0.0010) [2023-10-08 06:05:34,536][00611] Updated weights for policy 0, policy_version 55242 (0.0007) [2023-10-08 06:05:34,633][00612] Updated weights for policy 1, policy_version 55570 (0.0008) [2023-10-08 06:05:34,900][00611] Updated weights for policy 0, policy_version 55252 (0.0008) [2023-10-08 06:05:34,999][00612] Updated weights for policy 1, policy_version 55580 (0.0007) [2023-10-08 06:05:35,270][00611] Updated weights for policy 0, policy_version 55262 (0.0009) [2023-10-08 06:05:38,601][00612] Updated weights for policy 1, policy_version 55590 (0.0009) [2023-10-08 06:05:38,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 113508352. Throughput: 0: 1833.6, 1: 1830.6. Samples: 28385348. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 06:05:38,755][130385] Avg episode reward: [(0, '59.470'), (1, '71.800')] [2023-10-08 06:05:38,842][00611] Updated weights for policy 0, policy_version 55272 (0.0009) [2023-10-08 06:05:38,966][00612] Updated weights for policy 1, policy_version 55600 (0.0007) [2023-10-08 06:05:39,219][00611] Updated weights for policy 0, policy_version 55282 (0.0007) [2023-10-08 06:05:39,327][00612] Updated weights for policy 1, policy_version 55610 (0.0007) [2023-10-08 06:05:39,579][00611] Updated weights for policy 0, policy_version 55292 (0.0008) [2023-10-08 06:05:43,072][00612] Updated weights for policy 1, policy_version 55620 (0.0008) [2023-10-08 06:05:43,235][00611] Updated weights for policy 0, policy_version 55302 (0.0009) [2023-10-08 06:05:43,435][00612] Updated weights for policy 1, policy_version 55630 (0.0009) [2023-10-08 06:05:43,605][00611] Updated weights for policy 0, policy_version 55312 (0.0007) [2023-10-08 06:05:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 113573888. Throughput: 0: 1833.4, 1: 1827.5. Samples: 28408336. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 06:05:43,754][130385] Avg episode reward: [(0, '62.530'), (1, '73.250')] [2023-10-08 06:05:43,804][00612] Updated weights for policy 1, policy_version 55640 (0.0008) [2023-10-08 06:05:43,975][00611] Updated weights for policy 0, policy_version 55322 (0.0007) [2023-10-08 06:05:47,394][00612] Updated weights for policy 1, policy_version 55650 (0.0009) [2023-10-08 06:05:47,686][00611] Updated weights for policy 0, policy_version 55332 (0.0009) [2023-10-08 06:05:47,768][00612] Updated weights for policy 1, policy_version 55660 (0.0008) [2023-10-08 06:05:48,057][00611] Updated weights for policy 0, policy_version 55342 (0.0010) [2023-10-08 06:05:48,130][00612] Updated weights for policy 1, policy_version 55670 (0.0008) [2023-10-08 06:05:48,423][00611] Updated weights for policy 0, policy_version 55352 (0.0008) [2023-10-08 06:05:48,498][00612] Updated weights for policy 1, policy_version 55680 (0.0007) [2023-10-08 06:05:48,754][130385] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 113704960. Throughput: 0: 1828.2, 1: 1825.1. Samples: 28429752. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 06:05:48,755][130385] Avg episode reward: [(0, '62.620'), (1, '74.020')] [2023-10-08 06:05:52,144][00611] Updated weights for policy 0, policy_version 55362 (0.0010) [2023-10-08 06:05:52,261][00612] Updated weights for policy 1, policy_version 55690 (0.0007) [2023-10-08 06:05:52,510][00611] Updated weights for policy 0, policy_version 55372 (0.0007) [2023-10-08 06:05:52,638][00612] Updated weights for policy 1, policy_version 55700 (0.0007) [2023-10-08 06:05:52,878][00611] Updated weights for policy 0, policy_version 55382 (0.0007) [2023-10-08 06:05:52,996][00612] Updated weights for policy 1, policy_version 55710 (0.0007) [2023-10-08 06:05:53,241][00611] Updated weights for policy 0, policy_version 55392 (0.0008) [2023-10-08 06:05:53,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 113770496. Throughput: 0: 1837.8, 1: 1829.8. Samples: 28441406. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 06:05:53,754][130385] Avg episode reward: [(0, '62.880'), (1, '76.240')] [2023-10-08 06:05:56,847][00612] Updated weights for policy 1, policy_version 55720 (0.0008) [2023-10-08 06:05:56,964][00611] Updated weights for policy 0, policy_version 55402 (0.0008) [2023-10-08 06:05:57,219][00612] Updated weights for policy 1, policy_version 55730 (0.0008) [2023-10-08 06:05:57,335][00611] Updated weights for policy 0, policy_version 55412 (0.0008) [2023-10-08 06:05:57,585][00612] Updated weights for policy 1, policy_version 55740 (0.0009) [2023-10-08 06:05:57,700][00611] Updated weights for policy 0, policy_version 55422 (0.0007) [2023-10-08 06:05:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 113836032. Throughput: 0: 1825.9, 1: 1818.0. Samples: 28462174. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 06:05:58,755][130385] Avg episode reward: [(0, '62.290'), (1, '71.010')] [2023-10-08 06:06:01,097][00612] Updated weights for policy 1, policy_version 55750 (0.0007) [2023-10-08 06:06:01,397][00611] Updated weights for policy 0, policy_version 55432 (0.0008) [2023-10-08 06:06:01,465][00612] Updated weights for policy 1, policy_version 55760 (0.0008) [2023-10-08 06:06:01,764][00611] Updated weights for policy 0, policy_version 55442 (0.0007) [2023-10-08 06:06:01,837][00612] Updated weights for policy 1, policy_version 55770 (0.0008) [2023-10-08 06:06:02,137][00611] Updated weights for policy 0, policy_version 55452 (0.0010) [2023-10-08 06:06:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 113901568. Throughput: 0: 1829.4, 1: 1820.0. Samples: 28483716. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 06:06:03,755][130385] Avg episode reward: [(0, '62.780'), (1, '69.850')] [2023-10-08 06:06:05,587][00612] Updated weights for policy 1, policy_version 55780 (0.0007) [2023-10-08 06:06:05,921][00611] Updated weights for policy 0, policy_version 55462 (0.0008) [2023-10-08 06:06:05,944][00612] Updated weights for policy 1, policy_version 55790 (0.0009) [2023-10-08 06:06:06,291][00611] Updated weights for policy 0, policy_version 55472 (0.0008) [2023-10-08 06:06:06,317][00612] Updated weights for policy 1, policy_version 55800 (0.0010) [2023-10-08 06:06:06,653][00611] Updated weights for policy 0, policy_version 55482 (0.0007) [2023-10-08 06:06:08,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 113967104. Throughput: 0: 1822.1, 1: 1821.3. Samples: 28494960. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 06:06:08,754][130385] Avg episode reward: [(0, '69.740'), (1, '72.970')] [2023-10-08 06:06:10,076][00612] Updated weights for policy 1, policy_version 55810 (0.0010) [2023-10-08 06:06:10,312][00611] Updated weights for policy 0, policy_version 55492 (0.0007) [2023-10-08 06:06:10,434][00612] Updated weights for policy 1, policy_version 55820 (0.0010) [2023-10-08 06:06:10,677][00611] Updated weights for policy 0, policy_version 55502 (0.0007) [2023-10-08 06:06:10,805][00612] Updated weights for policy 1, policy_version 55830 (0.0007) [2023-10-08 06:06:11,053][00611] Updated weights for policy 0, policy_version 55512 (0.0007) [2023-10-08 06:06:11,171][00612] Updated weights for policy 1, policy_version 55840 (0.0009) [2023-10-08 06:06:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 114032640. Throughput: 0: 1822.4, 1: 1824.3. Samples: 28516398. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 06:06:13,754][130385] Avg episode reward: [(0, '68.940'), (1, '70.460')] [2023-10-08 06:06:14,655][00612] Updated weights for policy 1, policy_version 55850 (0.0008) [2023-10-08 06:06:14,883][00611] Updated weights for policy 0, policy_version 55522 (0.0008) [2023-10-08 06:06:15,028][00612] Updated weights for policy 1, policy_version 55860 (0.0008) [2023-10-08 06:06:15,288][00611] Updated weights for policy 0, policy_version 55532 (0.0009) [2023-10-08 06:06:15,400][00612] Updated weights for policy 1, policy_version 55870 (0.0008) [2023-10-08 06:06:15,650][00611] Updated weights for policy 0, policy_version 55542 (0.0009) [2023-10-08 06:06:16,021][00611] Updated weights for policy 0, policy_version 55552 (0.0012) [2023-10-08 06:06:18,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 114098176. Throughput: 0: 1815.9, 1: 1830.6. Samples: 28539464. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 06:06:18,754][130385] Avg episode reward: [(0, '73.720'), (1, '70.720')] [2023-10-08 06:06:18,766][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000055552_56885248.pth... [2023-10-08 06:06:18,796][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000053856_55148544.pth [2023-10-08 06:06:18,947][00612] Updated weights for policy 1, policy_version 55880 (0.0008) [2023-10-08 06:06:19,308][00612] Updated weights for policy 1, policy_version 55890 (0.0008) [2023-10-08 06:06:19,605][00611] Updated weights for policy 0, policy_version 55562 (0.0009) [2023-10-08 06:06:19,674][00612] Updated weights for policy 1, policy_version 55900 (0.0007) [2023-10-08 06:06:19,816][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000055904_57245696.pth... [2023-10-08 06:06:19,850][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000054176_55476224.pth [2023-10-08 06:06:19,973][00611] Updated weights for policy 0, policy_version 55572 (0.0009) [2023-10-08 06:06:20,342][00611] Updated weights for policy 0, policy_version 55582 (0.0010) [2023-10-08 06:06:23,184][00612] Updated weights for policy 1, policy_version 55910 (0.0007) [2023-10-08 06:06:23,565][00612] Updated weights for policy 1, policy_version 55920 (0.0010) [2023-10-08 06:06:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 114163712. Throughput: 0: 1811.7, 1: 1833.2. Samples: 28549368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:06:23,754][130385] Avg episode reward: [(0, '72.370'), (1, '72.160')] [2023-10-08 06:06:23,938][00612] Updated weights for policy 1, policy_version 55930 (0.0008) [2023-10-08 06:06:24,110][00611] Updated weights for policy 0, policy_version 55592 (0.0009) [2023-10-08 06:06:24,486][00611] Updated weights for policy 0, policy_version 55602 (0.0008) [2023-10-08 06:06:24,852][00611] Updated weights for policy 0, policy_version 55612 (0.0008) [2023-10-08 06:06:27,506][00612] Updated weights for policy 1, policy_version 55940 (0.0008) [2023-10-08 06:06:27,870][00612] Updated weights for policy 1, policy_version 55950 (0.0008) [2023-10-08 06:06:28,233][00612] Updated weights for policy 1, policy_version 55960 (0.0008) [2023-10-08 06:06:28,411][00611] Updated weights for policy 0, policy_version 55622 (0.0008) [2023-10-08 06:06:28,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 114262016. Throughput: 0: 1809.3, 1: 1838.4. Samples: 28572480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:06:28,754][130385] Avg episode reward: [(0, '71.710'), (1, '70.450')] [2023-10-08 06:06:28,782][00611] Updated weights for policy 0, policy_version 55632 (0.0007) [2023-10-08 06:06:29,152][00611] Updated weights for policy 0, policy_version 55642 (0.0009) [2023-10-08 06:06:31,945][00612] Updated weights for policy 1, policy_version 55970 (0.0008) [2023-10-08 06:06:32,316][00612] Updated weights for policy 1, policy_version 55980 (0.0008) [2023-10-08 06:06:32,678][00612] Updated weights for policy 1, policy_version 55990 (0.0009) [2023-10-08 06:06:32,788][00611] Updated weights for policy 0, policy_version 55652 (0.0007) [2023-10-08 06:06:33,045][00612] Updated weights for policy 1, policy_version 56000 (0.0009) [2023-10-08 06:06:33,164][00611] Updated weights for policy 0, policy_version 55662 (0.0008) [2023-10-08 06:06:33,546][00611] Updated weights for policy 0, policy_version 55672 (0.0008) [2023-10-08 06:06:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114327552. Throughput: 0: 1812.4, 1: 1826.2. Samples: 28593490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:06:33,756][130385] Avg episode reward: [(0, '74.430'), (1, '70.080')] [2023-10-08 06:06:36,886][00612] Updated weights for policy 1, policy_version 56010 (0.0009) [2023-10-08 06:06:37,234][00611] Updated weights for policy 0, policy_version 55682 (0.0010) [2023-10-08 06:06:37,257][00612] Updated weights for policy 1, policy_version 56020 (0.0007) [2023-10-08 06:06:37,602][00611] Updated weights for policy 0, policy_version 55692 (0.0007) [2023-10-08 06:06:37,621][00612] Updated weights for policy 1, policy_version 56030 (0.0008) [2023-10-08 06:06:37,978][00611] Updated weights for policy 0, policy_version 55702 (0.0007) [2023-10-08 06:06:38,354][00611] Updated weights for policy 0, policy_version 55712 (0.0011) [2023-10-08 06:06:38,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 114425856. Throughput: 0: 1810.4, 1: 1835.3. Samples: 28605464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:06:38,754][130385] Avg episode reward: [(0, '75.650'), (1, '71.970')] [2023-10-08 06:06:38,755][00365] Saving new best policy, reward=75.650! [2023-10-08 06:06:41,534][00612] Updated weights for policy 1, policy_version 56040 (0.0009) [2023-10-08 06:06:41,916][00612] Updated weights for policy 1, policy_version 56050 (0.0008) [2023-10-08 06:06:42,002][00611] Updated weights for policy 0, policy_version 55722 (0.0008) [2023-10-08 06:06:42,277][00612] Updated weights for policy 1, policy_version 56060 (0.0007) [2023-10-08 06:06:42,365][00611] Updated weights for policy 0, policy_version 55732 (0.0007) [2023-10-08 06:06:42,745][00611] Updated weights for policy 0, policy_version 55742 (0.0007) [2023-10-08 06:06:43,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 114491392. Throughput: 0: 1817.6, 1: 1832.0. Samples: 28626404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:06:43,754][130385] Avg episode reward: [(0, '75.140'), (1, '80.740')] [2023-10-08 06:06:45,737][00612] Updated weights for policy 1, policy_version 56070 (0.0008) [2023-10-08 06:06:46,115][00612] Updated weights for policy 1, policy_version 56080 (0.0009) [2023-10-08 06:06:46,455][00611] Updated weights for policy 0, policy_version 55752 (0.0008) [2023-10-08 06:06:46,481][00612] Updated weights for policy 1, policy_version 56090 (0.0009) [2023-10-08 06:06:46,820][00611] Updated weights for policy 0, policy_version 55762 (0.0007) [2023-10-08 06:06:47,199][00611] Updated weights for policy 0, policy_version 55772 (0.0008) [2023-10-08 06:06:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 114556928. Throughput: 0: 1818.9, 1: 1835.1. Samples: 28648146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:06:48,754][130385] Avg episode reward: [(0, '70.680'), (1, '77.760')] [2023-10-08 06:06:50,100][00612] Updated weights for policy 1, policy_version 56100 (0.0007) [2023-10-08 06:06:50,470][00612] Updated weights for policy 1, policy_version 56110 (0.0007) [2023-10-08 06:06:50,819][00611] Updated weights for policy 0, policy_version 55782 (0.0008) [2023-10-08 06:06:50,833][00612] Updated weights for policy 1, policy_version 56120 (0.0008) [2023-10-08 06:06:51,187][00611] Updated weights for policy 0, policy_version 55792 (0.0008) [2023-10-08 06:06:51,551][00611] Updated weights for policy 0, policy_version 55802 (0.0007) [2023-10-08 06:06:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 114622464. Throughput: 0: 1822.2, 1: 1827.8. Samples: 28659212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:06:53,754][130385] Avg episode reward: [(0, '68.810'), (1, '69.960')] [2023-10-08 06:06:54,466][00612] Updated weights for policy 1, policy_version 56130 (0.0008) [2023-10-08 06:06:54,833][00612] Updated weights for policy 1, policy_version 56140 (0.0008) [2023-10-08 06:06:55,193][00612] Updated weights for policy 1, policy_version 56150 (0.0007) [2023-10-08 06:06:55,313][00611] Updated weights for policy 0, policy_version 55812 (0.0008) [2023-10-08 06:06:55,555][00612] Updated weights for policy 1, policy_version 56160 (0.0007) [2023-10-08 06:06:55,684][00611] Updated weights for policy 0, policy_version 55822 (0.0009) [2023-10-08 06:06:56,052][00611] Updated weights for policy 0, policy_version 55832 (0.0007) [2023-10-08 06:06:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 114688000. Throughput: 0: 1825.1, 1: 1849.0. Samples: 28681730. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-08 06:06:58,754][130385] Avg episode reward: [(0, '71.670'), (1, '69.800')] [2023-10-08 06:06:59,147][00612] Updated weights for policy 1, policy_version 56170 (0.0009) [2023-10-08 06:06:59,512][00612] Updated weights for policy 1, policy_version 56180 (0.0010) [2023-10-08 06:06:59,638][00611] Updated weights for policy 0, policy_version 55842 (0.0009) [2023-10-08 06:06:59,876][00612] Updated weights for policy 1, policy_version 56190 (0.0008) [2023-10-08 06:07:00,024][00611] Updated weights for policy 0, policy_version 55852 (0.0009) [2023-10-08 06:07:00,401][00611] Updated weights for policy 0, policy_version 55862 (0.0010) [2023-10-08 06:07:00,774][00611] Updated weights for policy 0, policy_version 55872 (0.0011) [2023-10-08 06:07:03,545][00612] Updated weights for policy 1, policy_version 56200 (0.0007) [2023-10-08 06:07:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 114753536. Throughput: 0: 1823.7, 1: 1839.2. Samples: 28704294. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-08 06:07:03,754][130385] Avg episode reward: [(0, '67.430'), (1, '70.720')] [2023-10-08 06:07:03,917][00612] Updated weights for policy 1, policy_version 56210 (0.0008) [2023-10-08 06:07:04,280][00612] Updated weights for policy 1, policy_version 56220 (0.0008) [2023-10-08 06:07:04,508][00611] Updated weights for policy 0, policy_version 55882 (0.0007) [2023-10-08 06:07:04,881][00611] Updated weights for policy 0, policy_version 55892 (0.0009) [2023-10-08 06:07:05,254][00611] Updated weights for policy 0, policy_version 55902 (0.0009) [2023-10-08 06:07:07,980][00612] Updated weights for policy 1, policy_version 56230 (0.0008) [2023-10-08 06:07:08,337][00612] Updated weights for policy 1, policy_version 56240 (0.0008) [2023-10-08 06:07:08,706][00612] Updated weights for policy 1, policy_version 56250 (0.0008) [2023-10-08 06:07:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 114819072. Throughput: 0: 1826.9, 1: 1836.3. Samples: 28714214. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-08 06:07:08,754][130385] Avg episode reward: [(0, '65.240'), (1, '68.680')] [2023-10-08 06:07:08,820][00611] Updated weights for policy 0, policy_version 55912 (0.0009) [2023-10-08 06:07:09,194][00611] Updated weights for policy 0, policy_version 55922 (0.0010) [2023-10-08 06:07:09,558][00611] Updated weights for policy 0, policy_version 55932 (0.0010) [2023-10-08 06:07:12,421][00612] Updated weights for policy 1, policy_version 56260 (0.0008) [2023-10-08 06:07:12,798][00612] Updated weights for policy 1, policy_version 56270 (0.0009) [2023-10-08 06:07:13,160][00612] Updated weights for policy 1, policy_version 56280 (0.0010) [2023-10-08 06:07:13,259][00611] Updated weights for policy 0, policy_version 55942 (0.0008) [2023-10-08 06:07:13,633][00611] Updated weights for policy 0, policy_version 55952 (0.0007) [2023-10-08 06:07:13,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114917376. Throughput: 0: 1832.0, 1: 1829.8. Samples: 28737262. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-08 06:07:13,754][130385] Avg episode reward: [(0, '65.430'), (1, '67.960')] [2023-10-08 06:07:14,007][00611] Updated weights for policy 0, policy_version 55962 (0.0010) [2023-10-08 06:07:16,748][00612] Updated weights for policy 1, policy_version 56290 (0.0008) [2023-10-08 06:07:17,120][00612] Updated weights for policy 1, policy_version 56300 (0.0007) [2023-10-08 06:07:17,490][00612] Updated weights for policy 1, policy_version 56310 (0.0008) [2023-10-08 06:07:17,723][00611] Updated weights for policy 0, policy_version 55972 (0.0009) [2023-10-08 06:07:17,859][00612] Updated weights for policy 1, policy_version 56320 (0.0008) [2023-10-08 06:07:18,104][00611] Updated weights for policy 0, policy_version 55982 (0.0009) [2023-10-08 06:07:18,471][00611] Updated weights for policy 0, policy_version 55992 (0.0007) [2023-10-08 06:07:18,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 114982912. Throughput: 0: 1826.6, 1: 1830.4. Samples: 28758054. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-08 06:07:18,755][130385] Avg episode reward: [(0, '62.550'), (1, '65.800')] [2023-10-08 06:07:21,557][00612] Updated weights for policy 1, policy_version 56330 (0.0007) [2023-10-08 06:07:21,931][00612] Updated weights for policy 1, policy_version 56340 (0.0007) [2023-10-08 06:07:22,217][00611] Updated weights for policy 0, policy_version 56002 (0.0008) [2023-10-08 06:07:22,293][00612] Updated weights for policy 1, policy_version 56350 (0.0007) [2023-10-08 06:07:22,588][00611] Updated weights for policy 0, policy_version 56012 (0.0007) [2023-10-08 06:07:22,962][00611] Updated weights for policy 0, policy_version 56022 (0.0007) [2023-10-08 06:07:23,339][00611] Updated weights for policy 0, policy_version 56032 (0.0007) [2023-10-08 06:07:23,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 115081216. Throughput: 0: 1826.3, 1: 1832.7. Samples: 28770124. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-08 06:07:23,755][130385] Avg episode reward: [(0, '63.580'), (1, '66.300')] [2023-10-08 06:07:25,944][00612] Updated weights for policy 1, policy_version 56360 (0.0008) [2023-10-08 06:07:26,311][00612] Updated weights for policy 1, policy_version 56370 (0.0011) [2023-10-08 06:07:26,677][00612] Updated weights for policy 1, policy_version 56380 (0.0009) [2023-10-08 06:07:26,971][00611] Updated weights for policy 0, policy_version 56042 (0.0008) [2023-10-08 06:07:27,342][00611] Updated weights for policy 0, policy_version 56052 (0.0009) [2023-10-08 06:07:27,713][00611] Updated weights for policy 0, policy_version 56062 (0.0008) [2023-10-08 06:07:28,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 115146752. Throughput: 0: 1822.9, 1: 1832.7. Samples: 28790910. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-08 06:07:28,755][130385] Avg episode reward: [(0, '66.060'), (1, '63.300')] [2023-10-08 06:07:30,477][00612] Updated weights for policy 1, policy_version 56390 (0.0008) [2023-10-08 06:07:30,864][00612] Updated weights for policy 1, policy_version 56400 (0.0008) [2023-10-08 06:07:31,231][00612] Updated weights for policy 1, policy_version 56410 (0.0007) [2023-10-08 06:07:31,490][00611] Updated weights for policy 0, policy_version 56072 (0.0009) [2023-10-08 06:07:31,863][00611] Updated weights for policy 0, policy_version 56082 (0.0009) [2023-10-08 06:07:32,238][00611] Updated weights for policy 0, policy_version 56092 (0.0010) [2023-10-08 06:07:33,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115212288. Throughput: 0: 1822.0, 1: 1842.7. Samples: 28813058. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-08 06:07:33,754][130385] Avg episode reward: [(0, '66.560'), (1, '64.330')] [2023-10-08 06:07:34,780][00612] Updated weights for policy 1, policy_version 56420 (0.0008) [2023-10-08 06:07:35,140][00612] Updated weights for policy 1, policy_version 56430 (0.0008) [2023-10-08 06:07:35,505][00612] Updated weights for policy 1, policy_version 56440 (0.0011) [2023-10-08 06:07:35,850][00611] Updated weights for policy 0, policy_version 56102 (0.0007) [2023-10-08 06:07:36,227][00611] Updated weights for policy 0, policy_version 56112 (0.0009) [2023-10-08 06:07:36,600][00611] Updated weights for policy 0, policy_version 56122 (0.0009) [2023-10-08 06:07:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 115277824. Throughput: 0: 1819.9, 1: 1836.6. Samples: 28823754. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-08 06:07:38,755][130385] Avg episode reward: [(0, '64.950'), (1, '66.790')] [2023-10-08 06:07:39,045][00612] Updated weights for policy 1, policy_version 56450 (0.0008) [2023-10-08 06:07:39,422][00612] Updated weights for policy 1, policy_version 56460 (0.0008) [2023-10-08 06:07:39,791][00612] Updated weights for policy 1, policy_version 56470 (0.0008) [2023-10-08 06:07:40,150][00612] Updated weights for policy 1, policy_version 56480 (0.0008) [2023-10-08 06:07:40,264][00611] Updated weights for policy 0, policy_version 56132 (0.0009) [2023-10-08 06:07:40,641][00611] Updated weights for policy 0, policy_version 56142 (0.0008) [2023-10-08 06:07:41,020][00611] Updated weights for policy 0, policy_version 56152 (0.0009) [2023-10-08 06:07:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 115343360. Throughput: 0: 1810.9, 1: 1829.6. Samples: 28845552. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-08 06:07:43,754][130385] Avg episode reward: [(0, '64.730'), (1, '64.500')] [2023-10-08 06:07:43,984][00612] Updated weights for policy 1, policy_version 56490 (0.0008) [2023-10-08 06:07:44,349][00612] Updated weights for policy 1, policy_version 56500 (0.0007) [2023-10-08 06:07:44,714][00611] Updated weights for policy 0, policy_version 56162 (0.0009) [2023-10-08 06:07:44,718][00612] Updated weights for policy 1, policy_version 56510 (0.0007) [2023-10-08 06:07:45,111][00611] Updated weights for policy 0, policy_version 56172 (0.0010) [2023-10-08 06:07:45,481][00611] Updated weights for policy 0, policy_version 56182 (0.0008) [2023-10-08 06:07:45,854][00611] Updated weights for policy 0, policy_version 56192 (0.0007) [2023-10-08 06:07:48,362][00612] Updated weights for policy 1, policy_version 56520 (0.0010) [2023-10-08 06:07:48,731][00612] Updated weights for policy 1, policy_version 56530 (0.0011) [2023-10-08 06:07:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 115408896. Throughput: 0: 1821.5, 1: 1833.0. Samples: 28868746. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-08 06:07:48,755][130385] Avg episode reward: [(0, '65.740'), (1, '61.980')] [2023-10-08 06:07:49,099][00612] Updated weights for policy 1, policy_version 56540 (0.0010) [2023-10-08 06:07:49,498][00611] Updated weights for policy 0, policy_version 56202 (0.0007) [2023-10-08 06:07:49,863][00611] Updated weights for policy 0, policy_version 56212 (0.0008) [2023-10-08 06:07:50,240][00611] Updated weights for policy 0, policy_version 56222 (0.0008) [2023-10-08 06:07:52,700][00612] Updated weights for policy 1, policy_version 56550 (0.0009) [2023-10-08 06:07:53,066][00612] Updated weights for policy 1, policy_version 56560 (0.0007) [2023-10-08 06:07:53,435][00612] Updated weights for policy 1, policy_version 56570 (0.0007) [2023-10-08 06:07:53,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 115507200. Throughput: 0: 1821.3, 1: 1842.4. Samples: 28879082. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-08 06:07:53,754][130385] Avg episode reward: [(0, '63.340'), (1, '66.380')] [2023-10-08 06:07:53,782][00611] Updated weights for policy 0, policy_version 56232 (0.0007) [2023-10-08 06:07:54,148][00611] Updated weights for policy 0, policy_version 56242 (0.0010) [2023-10-08 06:07:54,520][00611] Updated weights for policy 0, policy_version 56252 (0.0008) [2023-10-08 06:07:57,059][00612] Updated weights for policy 1, policy_version 56580 (0.0009) [2023-10-08 06:07:57,426][00612] Updated weights for policy 1, policy_version 56590 (0.0008) [2023-10-08 06:07:57,797][00612] Updated weights for policy 1, policy_version 56600 (0.0007) [2023-10-08 06:07:58,127][00611] Updated weights for policy 0, policy_version 56262 (0.0009) [2023-10-08 06:07:58,499][00611] Updated weights for policy 0, policy_version 56272 (0.0009) [2023-10-08 06:07:58,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 115572736. Throughput: 0: 1826.8, 1: 1835.0. Samples: 28902046. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-08 06:07:58,754][130385] Avg episode reward: [(0, '60.910'), (1, '67.120')] [2023-10-08 06:07:58,867][00611] Updated weights for policy 0, policy_version 56282 (0.0010) [2023-10-08 06:08:01,382][00612] Updated weights for policy 1, policy_version 56610 (0.0008) [2023-10-08 06:08:01,750][00612] Updated weights for policy 1, policy_version 56620 (0.0010) [2023-10-08 06:08:02,114][00612] Updated weights for policy 1, policy_version 56630 (0.0009) [2023-10-08 06:08:02,483][00612] Updated weights for policy 1, policy_version 56640 (0.0007) [2023-10-08 06:08:02,612][00611] Updated weights for policy 0, policy_version 56292 (0.0009) [2023-10-08 06:08:02,967][00611] Updated weights for policy 0, policy_version 56302 (0.0007) [2023-10-08 06:08:03,331][00611] Updated weights for policy 0, policy_version 56312 (0.0007) [2023-10-08 06:08:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 115671040. Throughput: 0: 1824.3, 1: 1839.9. Samples: 28922944. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-08 06:08:03,754][130385] Avg episode reward: [(0, '65.820'), (1, '69.060')] [2023-10-08 06:08:06,036][00612] Updated weights for policy 1, policy_version 56650 (0.0008) [2023-10-08 06:08:06,404][00612] Updated weights for policy 1, policy_version 56660 (0.0008) [2023-10-08 06:08:06,773][00612] Updated weights for policy 1, policy_version 56670 (0.0008) [2023-10-08 06:08:06,975][00611] Updated weights for policy 0, policy_version 56322 (0.0007) [2023-10-08 06:08:07,343][00611] Updated weights for policy 0, policy_version 56332 (0.0010) [2023-10-08 06:08:07,710][00611] Updated weights for policy 0, policy_version 56342 (0.0008) [2023-10-08 06:08:08,090][00611] Updated weights for policy 0, policy_version 56352 (0.0007) [2023-10-08 06:08:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 115736576. Throughput: 0: 1830.5, 1: 1832.5. Samples: 28934960. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-08 06:08:08,754][130385] Avg episode reward: [(0, '66.990'), (1, '70.110')] [2023-10-08 06:08:10,186][00612] Updated weights for policy 1, policy_version 56680 (0.0009) [2023-10-08 06:08:10,554][00612] Updated weights for policy 1, policy_version 56690 (0.0008) [2023-10-08 06:08:10,921][00612] Updated weights for policy 1, policy_version 56700 (0.0007) [2023-10-08 06:08:11,711][00611] Updated weights for policy 0, policy_version 56362 (0.0007) [2023-10-08 06:08:12,084][00611] Updated weights for policy 0, policy_version 56372 (0.0007) [2023-10-08 06:08:12,458][00611] Updated weights for policy 0, policy_version 56382 (0.0007) [2023-10-08 06:08:13,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115802112. Throughput: 0: 1828.0, 1: 1854.0. Samples: 28956602. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 06:08:13,755][130385] Avg episode reward: [(0, '66.560'), (1, '73.530')] [2023-10-08 06:08:14,766][00612] Updated weights for policy 1, policy_version 56710 (0.0008) [2023-10-08 06:08:15,128][00612] Updated weights for policy 1, policy_version 56720 (0.0008) [2023-10-08 06:08:15,487][00612] Updated weights for policy 1, policy_version 56730 (0.0007) [2023-10-08 06:08:15,860][00611] Updated weights for policy 0, policy_version 56392 (0.0008) [2023-10-08 06:08:16,238][00611] Updated weights for policy 0, policy_version 56402 (0.0009) [2023-10-08 06:08:16,603][00611] Updated weights for policy 0, policy_version 56412 (0.0008) [2023-10-08 06:08:18,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115867648. Throughput: 0: 1847.7, 1: 1855.8. Samples: 28979716. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 06:08:18,755][130385] Avg episode reward: [(0, '68.280'), (1, '75.110')] [2023-10-08 06:08:18,768][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000056416_57769984.pth... [2023-10-08 06:08:18,768][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000056736_58097664.pth... [2023-10-08 06:08:18,818][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000054720_56033280.pth [2023-10-08 06:08:18,819][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000055040_56360960.pth [2023-10-08 06:08:19,200][00612] Updated weights for policy 1, policy_version 56740 (0.0008) [2023-10-08 06:08:19,587][00612] Updated weights for policy 1, policy_version 56750 (0.0008) [2023-10-08 06:08:19,951][00612] Updated weights for policy 1, policy_version 56760 (0.0009) [2023-10-08 06:08:20,280][00611] Updated weights for policy 0, policy_version 56422 (0.0008) [2023-10-08 06:08:20,652][00611] Updated weights for policy 0, policy_version 56432 (0.0007) [2023-10-08 06:08:21,018][00611] Updated weights for policy 0, policy_version 56442 (0.0009) [2023-10-08 06:08:23,550][00612] Updated weights for policy 1, policy_version 56770 (0.0008) [2023-10-08 06:08:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 115933184. Throughput: 0: 1833.6, 1: 1854.5. Samples: 28989718. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 06:08:23,754][130385] Avg episode reward: [(0, '66.700'), (1, '76.670')] [2023-10-08 06:08:23,922][00612] Updated weights for policy 1, policy_version 56780 (0.0007) [2023-10-08 06:08:24,281][00612] Updated weights for policy 1, policy_version 56790 (0.0007) [2023-10-08 06:08:24,651][00612] Updated weights for policy 1, policy_version 56800 (0.0007) [2023-10-08 06:08:24,718][00611] Updated weights for policy 0, policy_version 56452 (0.0008) [2023-10-08 06:08:25,090][00611] Updated weights for policy 0, policy_version 56462 (0.0009) [2023-10-08 06:08:25,455][00611] Updated weights for policy 0, policy_version 56472 (0.0008) [2023-10-08 06:08:28,295][00612] Updated weights for policy 1, policy_version 56810 (0.0008) [2023-10-08 06:08:28,660][00612] Updated weights for policy 1, policy_version 56820 (0.0008) [2023-10-08 06:08:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 115998720. Throughput: 0: 1860.1, 1: 1857.5. Samples: 29012848. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 06:08:28,755][130385] Avg episode reward: [(0, '65.440'), (1, '70.450')] [2023-10-08 06:08:29,039][00612] Updated weights for policy 1, policy_version 56830 (0.0008) [2023-10-08 06:08:29,162][00611] Updated weights for policy 0, policy_version 56482 (0.0009) [2023-10-08 06:08:29,535][00611] Updated weights for policy 0, policy_version 56492 (0.0007) [2023-10-08 06:08:29,909][00611] Updated weights for policy 0, policy_version 56502 (0.0010) [2023-10-08 06:08:30,283][00611] Updated weights for policy 0, policy_version 56512 (0.0009) [2023-10-08 06:08:32,518][00612] Updated weights for policy 1, policy_version 56840 (0.0007) [2023-10-08 06:08:32,891][00612] Updated weights for policy 1, policy_version 56850 (0.0008) [2023-10-08 06:08:33,252][00612] Updated weights for policy 1, policy_version 56860 (0.0008) [2023-10-08 06:08:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 116097024. Throughput: 0: 1852.0, 1: 1838.0. Samples: 29034794. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 06:08:33,754][130385] Avg episode reward: [(0, '64.140'), (1, '70.070')] [2023-10-08 06:08:34,088][00611] Updated weights for policy 0, policy_version 56522 (0.0008) [2023-10-08 06:08:34,462][00611] Updated weights for policy 0, policy_version 56532 (0.0008) [2023-10-08 06:08:34,830][00611] Updated weights for policy 0, policy_version 56542 (0.0008) [2023-10-08 06:08:36,908][00612] Updated weights for policy 1, policy_version 56870 (0.0009) [2023-10-08 06:08:37,274][00612] Updated weights for policy 1, policy_version 56880 (0.0009) [2023-10-08 06:08:37,644][00612] Updated weights for policy 1, policy_version 56890 (0.0008) [2023-10-08 06:08:38,379][00611] Updated weights for policy 0, policy_version 56552 (0.0008) [2023-10-08 06:08:38,750][00611] Updated weights for policy 0, policy_version 56562 (0.0011) [2023-10-08 06:08:38,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 116162560. Throughput: 0: 1848.0, 1: 1854.8. Samples: 29045706. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 06:08:38,754][130385] Avg episode reward: [(0, '63.570'), (1, '74.550')] [2023-10-08 06:08:39,128][00611] Updated weights for policy 0, policy_version 56572 (0.0009) [2023-10-08 06:08:41,278][00612] Updated weights for policy 1, policy_version 56900 (0.0008) [2023-10-08 06:08:41,640][00612] Updated weights for policy 1, policy_version 56910 (0.0007) [2023-10-08 06:08:42,008][00612] Updated weights for policy 1, policy_version 56920 (0.0008) [2023-10-08 06:08:42,594][00611] Updated weights for policy 0, policy_version 56582 (0.0008) [2023-10-08 06:08:42,977][00611] Updated weights for policy 0, policy_version 56592 (0.0011) [2023-10-08 06:08:43,348][00611] Updated weights for policy 0, policy_version 56602 (0.0009) [2023-10-08 06:08:43,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 116260864. Throughput: 0: 1845.7, 1: 1834.4. Samples: 29067654. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 06:08:43,754][130385] Avg episode reward: [(0, '63.440'), (1, '75.630')] [2023-10-08 06:08:45,616][00612] Updated weights for policy 1, policy_version 56930 (0.0011) [2023-10-08 06:08:45,974][00612] Updated weights for policy 1, policy_version 56940 (0.0009) [2023-10-08 06:08:46,355][00612] Updated weights for policy 1, policy_version 56950 (0.0008) [2023-10-08 06:08:46,718][00612] Updated weights for policy 1, policy_version 56960 (0.0009) [2023-10-08 06:08:46,992][00611] Updated weights for policy 0, policy_version 56612 (0.0009) [2023-10-08 06:08:47,360][00611] Updated weights for policy 0, policy_version 56622 (0.0008) [2023-10-08 06:08:47,729][00611] Updated weights for policy 0, policy_version 56632 (0.0007) [2023-10-08 06:08:48,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 116326400. Throughput: 0: 1831.9, 1: 1858.9. Samples: 29089030. Policy #0 lag: (min: 25.0, avg: 26.5, max: 47.0) [2023-10-08 06:08:48,755][130385] Avg episode reward: [(0, '63.440'), (1, '77.400')] [2023-10-08 06:08:50,429][00612] Updated weights for policy 1, policy_version 56970 (0.0009) [2023-10-08 06:08:50,801][00612] Updated weights for policy 1, policy_version 56980 (0.0008) [2023-10-08 06:08:51,175][00612] Updated weights for policy 1, policy_version 56990 (0.0007) [2023-10-08 06:08:51,505][00611] Updated weights for policy 0, policy_version 56642 (0.0008) [2023-10-08 06:08:51,880][00611] Updated weights for policy 0, policy_version 56652 (0.0010) [2023-10-08 06:08:52,260][00611] Updated weights for policy 0, policy_version 56662 (0.0011) [2023-10-08 06:08:52,623][00611] Updated weights for policy 0, policy_version 56672 (0.0008) [2023-10-08 06:08:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 116391936. Throughput: 0: 1846.9, 1: 1835.2. Samples: 29100652. Policy #0 lag: (min: 25.0, avg: 26.5, max: 47.0) [2023-10-08 06:08:53,754][130385] Avg episode reward: [(0, '65.360'), (1, '78.140')] [2023-10-08 06:08:54,755][00612] Updated weights for policy 1, policy_version 57000 (0.0009) [2023-10-08 06:08:55,118][00612] Updated weights for policy 1, policy_version 57010 (0.0010) [2023-10-08 06:08:55,491][00612] Updated weights for policy 1, policy_version 57020 (0.0009) [2023-10-08 06:08:56,330][00611] Updated weights for policy 0, policy_version 56682 (0.0007) [2023-10-08 06:08:56,697][00611] Updated weights for policy 0, policy_version 56692 (0.0008) [2023-10-08 06:08:57,077][00611] Updated weights for policy 0, policy_version 56702 (0.0011) [2023-10-08 06:08:58,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 116457472. Throughput: 0: 1833.3, 1: 1847.7. Samples: 29122250. Policy #0 lag: (min: 25.0, avg: 26.5, max: 47.0) [2023-10-08 06:08:58,754][130385] Avg episode reward: [(0, '60.260'), (1, '76.150')] [2023-10-08 06:08:59,200][00612] Updated weights for policy 1, policy_version 57030 (0.0007) [2023-10-08 06:08:59,568][00612] Updated weights for policy 1, policy_version 57040 (0.0007) [2023-10-08 06:08:59,937][00612] Updated weights for policy 1, policy_version 57050 (0.0008) [2023-10-08 06:09:00,713][00611] Updated weights for policy 0, policy_version 56712 (0.0007) [2023-10-08 06:09:01,089][00611] Updated weights for policy 0, policy_version 56722 (0.0011) [2023-10-08 06:09:01,466][00611] Updated weights for policy 0, policy_version 56732 (0.0009) [2023-10-08 06:09:03,573][00612] Updated weights for policy 1, policy_version 57060 (0.0011) [2023-10-08 06:09:03,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 116523008. Throughput: 0: 1842.0, 1: 1844.1. Samples: 29145588. Policy #0 lag: (min: 25.0, avg: 26.5, max: 47.0) [2023-10-08 06:09:03,755][130385] Avg episode reward: [(0, '61.880'), (1, '75.470')] [2023-10-08 06:09:03,948][00612] Updated weights for policy 1, policy_version 57070 (0.0010) [2023-10-08 06:09:04,311][00612] Updated weights for policy 1, policy_version 57080 (0.0010) [2023-10-08 06:09:04,936][00611] Updated weights for policy 0, policy_version 56742 (0.0007) [2023-10-08 06:09:05,306][00611] Updated weights for policy 0, policy_version 56752 (0.0008) [2023-10-08 06:09:05,673][00611] Updated weights for policy 0, policy_version 56762 (0.0011) [2023-10-08 06:09:07,841][00612] Updated weights for policy 1, policy_version 57090 (0.0009) [2023-10-08 06:09:08,231][00612] Updated weights for policy 1, policy_version 57100 (0.0009) [2023-10-08 06:09:08,603][00612] Updated weights for policy 1, policy_version 57110 (0.0007) [2023-10-08 06:09:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 116588544. Throughput: 0: 1838.4, 1: 1851.8. Samples: 29155774. Policy #0 lag: (min: 25.0, avg: 26.5, max: 47.0) [2023-10-08 06:09:08,754][130385] Avg episode reward: [(0, '62.160'), (1, '75.210')] [2023-10-08 06:09:08,969][00612] Updated weights for policy 1, policy_version 57120 (0.0009) [2023-10-08 06:09:09,052][00611] Updated weights for policy 0, policy_version 56772 (0.0008) [2023-10-08 06:09:09,421][00611] Updated weights for policy 0, policy_version 56782 (0.0008) [2023-10-08 06:09:09,792][00611] Updated weights for policy 0, policy_version 56792 (0.0008) [2023-10-08 06:09:12,611][00612] Updated weights for policy 1, policy_version 57130 (0.0009) [2023-10-08 06:09:12,985][00612] Updated weights for policy 1, policy_version 57140 (0.0007) [2023-10-08 06:09:13,351][00612] Updated weights for policy 1, policy_version 57150 (0.0008) [2023-10-08 06:09:13,425][00611] Updated weights for policy 0, policy_version 56802 (0.0008) [2023-10-08 06:09:13,754][130385] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 116686848. Throughput: 0: 1842.8, 1: 1850.5. Samples: 29179046. Policy #0 lag: (min: 25.0, avg: 26.5, max: 47.0) [2023-10-08 06:09:13,754][130385] Avg episode reward: [(0, '62.810'), (1, '76.110')] [2023-10-08 06:09:13,796][00611] Updated weights for policy 0, policy_version 56812 (0.0011) [2023-10-08 06:09:14,166][00611] Updated weights for policy 0, policy_version 56822 (0.0010) [2023-10-08 06:09:14,536][00611] Updated weights for policy 0, policy_version 56832 (0.0009) [2023-10-08 06:09:16,915][00612] Updated weights for policy 1, policy_version 57160 (0.0008) [2023-10-08 06:09:17,287][00612] Updated weights for policy 1, policy_version 57170 (0.0007) [2023-10-08 06:09:17,665][00612] Updated weights for policy 1, policy_version 57180 (0.0008) [2023-10-08 06:09:18,386][00611] Updated weights for policy 0, policy_version 56842 (0.0008) [2023-10-08 06:09:18,752][00611] Updated weights for policy 0, policy_version 56852 (0.0009) [2023-10-08 06:09:18,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 116752384. Throughput: 0: 1842.9, 1: 1839.0. Samples: 29200482. Policy #0 lag: (min: 25.0, avg: 26.5, max: 47.0) [2023-10-08 06:09:18,755][130385] Avg episode reward: [(0, '60.280'), (1, '74.050')] [2023-10-08 06:09:19,139][00611] Updated weights for policy 0, policy_version 56862 (0.0009) [2023-10-08 06:09:21,162][00612] Updated weights for policy 1, policy_version 57190 (0.0008) [2023-10-08 06:09:21,527][00612] Updated weights for policy 1, policy_version 57200 (0.0007) [2023-10-08 06:09:21,886][00612] Updated weights for policy 1, policy_version 57210 (0.0008) [2023-10-08 06:09:22,884][00611] Updated weights for policy 0, policy_version 56872 (0.0008) [2023-10-08 06:09:23,249][00611] Updated weights for policy 0, policy_version 56882 (0.0009) [2023-10-08 06:09:23,622][00611] Updated weights for policy 0, policy_version 56892 (0.0008) [2023-10-08 06:09:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 116817920. Throughput: 0: 1850.7, 1: 1847.1. Samples: 29212104. Policy #0 lag: (min: 25.0, avg: 26.5, max: 47.0) [2023-10-08 06:09:23,758][130385] Avg episode reward: [(0, '60.740'), (1, '72.610')] [2023-10-08 06:09:25,406][00612] Updated weights for policy 1, policy_version 57220 (0.0010) [2023-10-08 06:09:25,774][00612] Updated weights for policy 1, policy_version 57230 (0.0011) [2023-10-08 06:09:26,153][00612] Updated weights for policy 1, policy_version 57240 (0.0010) [2023-10-08 06:09:27,241][00611] Updated weights for policy 0, policy_version 56902 (0.0008) [2023-10-08 06:09:27,619][00611] Updated weights for policy 0, policy_version 56912 (0.0007) [2023-10-08 06:09:27,994][00611] Updated weights for policy 0, policy_version 56922 (0.0008) [2023-10-08 06:09:28,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 116916224. Throughput: 0: 1840.9, 1: 1855.1. Samples: 29233978. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-08 06:09:28,755][130385] Avg episode reward: [(0, '58.860'), (1, '72.380')] [2023-10-08 06:09:29,801][00612] Updated weights for policy 1, policy_version 57250 (0.0007) [2023-10-08 06:09:30,174][00612] Updated weights for policy 1, policy_version 57260 (0.0008) [2023-10-08 06:09:30,546][00612] Updated weights for policy 1, policy_version 57270 (0.0011) [2023-10-08 06:09:30,916][00612] Updated weights for policy 1, policy_version 57280 (0.0010) [2023-10-08 06:09:31,397][00611] Updated weights for policy 0, policy_version 56932 (0.0008) [2023-10-08 06:09:31,773][00611] Updated weights for policy 0, policy_version 56942 (0.0010) [2023-10-08 06:09:32,140][00611] Updated weights for policy 0, policy_version 56952 (0.0010) [2023-10-08 06:09:33,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 116981760. Throughput: 0: 1848.7, 1: 1860.1. Samples: 29255924. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-08 06:09:33,755][130385] Avg episode reward: [(0, '60.840'), (1, '76.120')] [2023-10-08 06:09:34,573][00612] Updated weights for policy 1, policy_version 57290 (0.0010) [2023-10-08 06:09:34,944][00612] Updated weights for policy 1, policy_version 57300 (0.0010) [2023-10-08 06:09:35,315][00612] Updated weights for policy 1, policy_version 57310 (0.0010) [2023-10-08 06:09:35,952][00611] Updated weights for policy 0, policy_version 56962 (0.0009) [2023-10-08 06:09:36,332][00611] Updated weights for policy 0, policy_version 56972 (0.0010) [2023-10-08 06:09:36,700][00611] Updated weights for policy 0, policy_version 56982 (0.0008) [2023-10-08 06:09:37,067][00611] Updated weights for policy 0, policy_version 56992 (0.0009) [2023-10-08 06:09:38,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 117047296. Throughput: 0: 1839.6, 1: 1856.3. Samples: 29266968. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-08 06:09:38,755][130385] Avg episode reward: [(0, '65.570'), (1, '78.390')] [2023-10-08 06:09:38,906][00612] Updated weights for policy 1, policy_version 57320 (0.0010) [2023-10-08 06:09:39,275][00612] Updated weights for policy 1, policy_version 57330 (0.0009) [2023-10-08 06:09:39,648][00612] Updated weights for policy 1, policy_version 57340 (0.0010) [2023-10-08 06:09:40,738][00611] Updated weights for policy 0, policy_version 57002 (0.0008) [2023-10-08 06:09:41,110][00611] Updated weights for policy 0, policy_version 57012 (0.0007) [2023-10-08 06:09:41,489][00611] Updated weights for policy 0, policy_version 57022 (0.0007) [2023-10-08 06:09:43,099][00612] Updated weights for policy 1, policy_version 57350 (0.0008) [2023-10-08 06:09:43,462][00612] Updated weights for policy 1, policy_version 57360 (0.0008) [2023-10-08 06:09:43,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 117112832. Throughput: 0: 1842.8, 1: 1867.0. Samples: 29289188. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-08 06:09:43,754][130385] Avg episode reward: [(0, '64.180'), (1, '79.440')] [2023-10-08 06:09:43,830][00612] Updated weights for policy 1, policy_version 57370 (0.0008) [2023-10-08 06:09:45,119][00611] Updated weights for policy 0, policy_version 57032 (0.0009) [2023-10-08 06:09:45,476][00611] Updated weights for policy 0, policy_version 57042 (0.0009) [2023-10-08 06:09:45,857][00611] Updated weights for policy 0, policy_version 57052 (0.0008) [2023-10-08 06:09:47,566][00612] Updated weights for policy 1, policy_version 57380 (0.0007) [2023-10-08 06:09:47,933][00612] Updated weights for policy 1, policy_version 57390 (0.0009) [2023-10-08 06:09:48,293][00612] Updated weights for policy 1, policy_version 57400 (0.0007) [2023-10-08 06:09:48,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 117211136. Throughput: 0: 1841.2, 1: 1848.3. Samples: 29311612. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-08 06:09:48,754][130385] Avg episode reward: [(0, '64.090'), (1, '78.280')] [2023-10-08 06:09:49,513][00611] Updated weights for policy 0, policy_version 57062 (0.0008) [2023-10-08 06:09:49,879][00611] Updated weights for policy 0, policy_version 57072 (0.0009) [2023-10-08 06:09:50,247][00611] Updated weights for policy 0, policy_version 57082 (0.0009) [2023-10-08 06:09:51,802][00612] Updated weights for policy 1, policy_version 57410 (0.0008) [2023-10-08 06:09:52,167][00612] Updated weights for policy 1, policy_version 57420 (0.0007) [2023-10-08 06:09:52,540][00612] Updated weights for policy 1, policy_version 57430 (0.0008) [2023-10-08 06:09:52,903][00612] Updated weights for policy 1, policy_version 57440 (0.0008) [2023-10-08 06:09:53,754][130385] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 117276672. Throughput: 0: 1837.9, 1: 1866.2. Samples: 29322460. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-08 06:09:53,755][130385] Avg episode reward: [(0, '58.930'), (1, '80.060')] [2023-10-08 06:09:53,955][00611] Updated weights for policy 0, policy_version 57092 (0.0010) [2023-10-08 06:09:54,324][00611] Updated weights for policy 0, policy_version 57102 (0.0011) [2023-10-08 06:09:54,706][00611] Updated weights for policy 0, policy_version 57112 (0.0009) [2023-10-08 06:09:56,726][00612] Updated weights for policy 1, policy_version 57450 (0.0009) [2023-10-08 06:09:57,094][00612] Updated weights for policy 1, policy_version 57460 (0.0009) [2023-10-08 06:09:57,466][00612] Updated weights for policy 1, policy_version 57470 (0.0009) [2023-10-08 06:09:58,273][00611] Updated weights for policy 0, policy_version 57122 (0.0008) [2023-10-08 06:09:58,643][00611] Updated weights for policy 0, policy_version 57132 (0.0010) [2023-10-08 06:09:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 117342208. Throughput: 0: 1837.1, 1: 1844.6. Samples: 29344720. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-08 06:09:58,755][130385] Avg episode reward: [(0, '60.580'), (1, '75.450')] [2023-10-08 06:09:59,021][00611] Updated weights for policy 0, policy_version 57142 (0.0008) [2023-10-08 06:09:59,396][00611] Updated weights for policy 0, policy_version 57152 (0.0008) [2023-10-08 06:10:00,926][00612] Updated weights for policy 1, policy_version 57480 (0.0010) [2023-10-08 06:10:01,294][00612] Updated weights for policy 1, policy_version 57490 (0.0011) [2023-10-08 06:10:01,659][00612] Updated weights for policy 1, policy_version 57500 (0.0011) [2023-10-08 06:10:02,841][00611] Updated weights for policy 0, policy_version 57162 (0.0008) [2023-10-08 06:10:03,208][00611] Updated weights for policy 0, policy_version 57172 (0.0008) [2023-10-08 06:10:03,591][00611] Updated weights for policy 0, policy_version 57182 (0.0010) [2023-10-08 06:10:03,754][130385] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 117440512. Throughput: 0: 1827.2, 1: 1868.5. Samples: 29366790. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-08 06:10:03,754][130385] Avg episode reward: [(0, '62.730'), (1, '75.270')] [2023-10-08 06:10:05,256][00612] Updated weights for policy 1, policy_version 57510 (0.0009) [2023-10-08 06:10:05,613][00612] Updated weights for policy 1, policy_version 57520 (0.0011) [2023-10-08 06:10:05,974][00612] Updated weights for policy 1, policy_version 57530 (0.0008) [2023-10-08 06:10:07,265][00611] Updated weights for policy 0, policy_version 57192 (0.0009) [2023-10-08 06:10:07,633][00611] Updated weights for policy 0, policy_version 57202 (0.0007) [2023-10-08 06:10:08,005][00611] Updated weights for policy 0, policy_version 57212 (0.0008) [2023-10-08 06:10:08,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 117506048. Throughput: 0: 1843.1, 1: 1837.3. Samples: 29377724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:10:08,754][130385] Avg episode reward: [(0, '62.930'), (1, '74.030')] [2023-10-08 06:10:09,617][00612] Updated weights for policy 1, policy_version 57540 (0.0007) [2023-10-08 06:10:09,989][00612] Updated weights for policy 1, policy_version 57550 (0.0007) [2023-10-08 06:10:10,362][00612] Updated weights for policy 1, policy_version 57560 (0.0010) [2023-10-08 06:10:11,624][00611] Updated weights for policy 0, policy_version 57222 (0.0011) [2023-10-08 06:10:12,017][00611] Updated weights for policy 0, policy_version 57232 (0.0009) [2023-10-08 06:10:12,384][00611] Updated weights for policy 0, policy_version 57242 (0.0009) [2023-10-08 06:10:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 117571584. Throughput: 0: 1826.7, 1: 1858.6. Samples: 29399818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:10:13,754][130385] Avg episode reward: [(0, '62.870'), (1, '74.610')] [2023-10-08 06:10:14,034][00612] Updated weights for policy 1, policy_version 57570 (0.0011) [2023-10-08 06:10:14,406][00612] Updated weights for policy 1, policy_version 57580 (0.0007) [2023-10-08 06:10:14,772][00612] Updated weights for policy 1, policy_version 57590 (0.0008) [2023-10-08 06:10:15,136][00612] Updated weights for policy 1, policy_version 57600 (0.0007) [2023-10-08 06:10:16,059][00611] Updated weights for policy 0, policy_version 57252 (0.0009) [2023-10-08 06:10:16,429][00611] Updated weights for policy 0, policy_version 57262 (0.0008) [2023-10-08 06:10:16,808][00611] Updated weights for policy 0, policy_version 57272 (0.0010) [2023-10-08 06:10:18,717][00612] Updated weights for policy 1, policy_version 57610 (0.0009) [2023-10-08 06:10:18,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 117637120. Throughput: 0: 1834.7, 1: 1860.2. Samples: 29422194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:10:18,754][130385] Avg episode reward: [(0, '61.470'), (1, '74.390')] [2023-10-08 06:10:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000057280_58654720.pth... [2023-10-08 06:10:18,802][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000055552_56885248.pth [2023-10-08 06:10:19,075][00612] Updated weights for policy 1, policy_version 57620 (0.0008) [2023-10-08 06:10:19,442][00612] Updated weights for policy 1, policy_version 57630 (0.0008) [2023-10-08 06:10:19,515][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000057632_59015168.pth... [2023-10-08 06:10:19,554][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000055904_57245696.pth [2023-10-08 06:10:20,717][00611] Updated weights for policy 0, policy_version 57282 (0.0008) [2023-10-08 06:10:21,096][00611] Updated weights for policy 0, policy_version 57292 (0.0008) [2023-10-08 06:10:21,473][00611] Updated weights for policy 0, policy_version 57302 (0.0009) [2023-10-08 06:10:21,839][00611] Updated weights for policy 0, policy_version 57312 (0.0007) [2023-10-08 06:10:23,032][00612] Updated weights for policy 1, policy_version 57640 (0.0008) [2023-10-08 06:10:23,393][00612] Updated weights for policy 1, policy_version 57650 (0.0008) [2023-10-08 06:10:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 117702656. Throughput: 0: 1827.4, 1: 1861.9. Samples: 29432988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:10:23,754][130385] Avg episode reward: [(0, '63.470'), (1, '74.110')] [2023-10-08 06:10:23,761][00612] Updated weights for policy 1, policy_version 57660 (0.0008) [2023-10-08 06:10:25,448][00611] Updated weights for policy 0, policy_version 57322 (0.0009) [2023-10-08 06:10:25,829][00611] Updated weights for policy 0, policy_version 57332 (0.0010) [2023-10-08 06:10:26,186][00611] Updated weights for policy 0, policy_version 57342 (0.0007) [2023-10-08 06:10:27,381][00612] Updated weights for policy 1, policy_version 57670 (0.0007) [2023-10-08 06:10:27,750][00612] Updated weights for policy 1, policy_version 57680 (0.0007) [2023-10-08 06:10:28,124][00612] Updated weights for policy 1, policy_version 57690 (0.0009) [2023-10-08 06:10:28,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 117800960. Throughput: 0: 1837.6, 1: 1855.2. Samples: 29455366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:10:28,755][130385] Avg episode reward: [(0, '68.040'), (1, '72.840')] [2023-10-08 06:10:29,823][00611] Updated weights for policy 0, policy_version 57352 (0.0008) [2023-10-08 06:10:30,197][00611] Updated weights for policy 0, policy_version 57362 (0.0008) [2023-10-08 06:10:30,571][00611] Updated weights for policy 0, policy_version 57372 (0.0008) [2023-10-08 06:10:31,796][00612] Updated weights for policy 1, policy_version 57700 (0.0011) [2023-10-08 06:10:32,164][00612] Updated weights for policy 1, policy_version 57710 (0.0008) [2023-10-08 06:10:32,540][00612] Updated weights for policy 1, policy_version 57720 (0.0009) [2023-10-08 06:10:33,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 117866496. Throughput: 0: 1834.4, 1: 1839.1. Samples: 29476920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:10:33,754][130385] Avg episode reward: [(0, '62.150'), (1, '72.890')] [2023-10-08 06:10:34,239][00611] Updated weights for policy 0, policy_version 57382 (0.0008) [2023-10-08 06:10:34,605][00611] Updated weights for policy 0, policy_version 57392 (0.0008) [2023-10-08 06:10:34,968][00611] Updated weights for policy 0, policy_version 57402 (0.0009) [2023-10-08 06:10:36,192][00612] Updated weights for policy 1, policy_version 57730 (0.0008) [2023-10-08 06:10:36,551][00612] Updated weights for policy 1, policy_version 57740 (0.0008) [2023-10-08 06:10:36,937][00612] Updated weights for policy 1, policy_version 57750 (0.0010) [2023-10-08 06:10:37,306][00612] Updated weights for policy 1, policy_version 57760 (0.0008) [2023-10-08 06:10:38,559][00611] Updated weights for policy 0, policy_version 57412 (0.0010) [2023-10-08 06:10:38,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 117932032. Throughput: 0: 1831.6, 1: 1850.2. Samples: 29488142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:10:38,754][130385] Avg episode reward: [(0, '63.450'), (1, '71.570')] [2023-10-08 06:10:38,928][00611] Updated weights for policy 0, policy_version 57422 (0.0011) [2023-10-08 06:10:39,298][00611] Updated weights for policy 0, policy_version 57432 (0.0009) [2023-10-08 06:10:40,831][00612] Updated weights for policy 1, policy_version 57770 (0.0008) [2023-10-08 06:10:41,198][00612] Updated weights for policy 1, policy_version 57780 (0.0009) [2023-10-08 06:10:41,568][00612] Updated weights for policy 1, policy_version 57790 (0.0008) [2023-10-08 06:10:42,898][00611] Updated weights for policy 0, policy_version 57442 (0.0007) [2023-10-08 06:10:43,260][00611] Updated weights for policy 0, policy_version 57452 (0.0007) [2023-10-08 06:10:43,637][00611] Updated weights for policy 0, policy_version 57462 (0.0007) [2023-10-08 06:10:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 117997568. Throughput: 0: 1828.0, 1: 1841.5. Samples: 29509846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:10:43,754][130385] Avg episode reward: [(0, '63.940'), (1, '71.170')] [2023-10-08 06:10:44,004][00611] Updated weights for policy 0, policy_version 57472 (0.0008) [2023-10-08 06:10:45,347][00612] Updated weights for policy 1, policy_version 57800 (0.0010) [2023-10-08 06:10:45,715][00612] Updated weights for policy 1, policy_version 57810 (0.0010) [2023-10-08 06:10:46,081][00612] Updated weights for policy 1, policy_version 57820 (0.0011) [2023-10-08 06:10:47,623][00611] Updated weights for policy 0, policy_version 57482 (0.0009) [2023-10-08 06:10:47,982][00611] Updated weights for policy 0, policy_version 57492 (0.0009) [2023-10-08 06:10:48,368][00611] Updated weights for policy 0, policy_version 57502 (0.0007) [2023-10-08 06:10:48,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 118095872. Throughput: 0: 1818.2, 1: 1845.7. Samples: 29531666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:10:48,755][130385] Avg episode reward: [(0, '65.590'), (1, '68.790')] [2023-10-08 06:10:49,692][00612] Updated weights for policy 1, policy_version 57830 (0.0008) [2023-10-08 06:10:50,058][00612] Updated weights for policy 1, policy_version 57840 (0.0007) [2023-10-08 06:10:50,423][00612] Updated weights for policy 1, policy_version 57850 (0.0008) [2023-10-08 06:10:52,104][00611] Updated weights for policy 0, policy_version 57512 (0.0008) [2023-10-08 06:10:52,468][00611] Updated weights for policy 0, policy_version 57522 (0.0008) [2023-10-08 06:10:52,840][00611] Updated weights for policy 0, policy_version 57532 (0.0009) [2023-10-08 06:10:53,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 118161408. Throughput: 0: 1824.8, 1: 1844.8. Samples: 29542854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:10:53,754][130385] Avg episode reward: [(0, '64.600'), (1, '71.650')] [2023-10-08 06:10:54,183][00612] Updated weights for policy 1, policy_version 57860 (0.0010) [2023-10-08 06:10:54,552][00612] Updated weights for policy 1, policy_version 57870 (0.0007) [2023-10-08 06:10:54,909][00612] Updated weights for policy 1, policy_version 57880 (0.0007) [2023-10-08 06:10:56,491][00611] Updated weights for policy 0, policy_version 57542 (0.0007) [2023-10-08 06:10:56,865][00611] Updated weights for policy 0, policy_version 57552 (0.0008) [2023-10-08 06:10:57,231][00611] Updated weights for policy 0, policy_version 57562 (0.0008) [2023-10-08 06:10:58,636][00612] Updated weights for policy 1, policy_version 57890 (0.0008) [2023-10-08 06:10:58,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 118226944. Throughput: 0: 1823.0, 1: 1842.6. Samples: 29564772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:10:58,754][130385] Avg episode reward: [(0, '60.530'), (1, '71.540')] [2023-10-08 06:10:59,004][00612] Updated weights for policy 1, policy_version 57900 (0.0008) [2023-10-08 06:10:59,383][00612] Updated weights for policy 1, policy_version 57910 (0.0009) [2023-10-08 06:10:59,754][00612] Updated weights for policy 1, policy_version 57920 (0.0008) [2023-10-08 06:11:01,088][00611] Updated weights for policy 0, policy_version 57572 (0.0008) [2023-10-08 06:11:01,465][00611] Updated weights for policy 0, policy_version 57582 (0.0008) [2023-10-08 06:11:01,833][00611] Updated weights for policy 0, policy_version 57592 (0.0008) [2023-10-08 06:11:03,432][00612] Updated weights for policy 1, policy_version 57930 (0.0011) [2023-10-08 06:11:03,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 118292480. Throughput: 0: 1827.6, 1: 1838.0. Samples: 29587146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:11:03,755][130385] Avg episode reward: [(0, '58.280'), (1, '64.950')] [2023-10-08 06:11:03,789][00612] Updated weights for policy 1, policy_version 57940 (0.0011) [2023-10-08 06:11:04,166][00612] Updated weights for policy 1, policy_version 57950 (0.0011) [2023-10-08 06:11:05,561][00611] Updated weights for policy 0, policy_version 57602 (0.0008) [2023-10-08 06:11:05,931][00611] Updated weights for policy 0, policy_version 57612 (0.0010) [2023-10-08 06:11:06,290][00611] Updated weights for policy 0, policy_version 57622 (0.0007) [2023-10-08 06:11:06,661][00611] Updated weights for policy 0, policy_version 57632 (0.0008) [2023-10-08 06:11:07,848][00612] Updated weights for policy 1, policy_version 57960 (0.0008) [2023-10-08 06:11:08,219][00612] Updated weights for policy 1, policy_version 57970 (0.0009) [2023-10-08 06:11:08,594][00612] Updated weights for policy 1, policy_version 57980 (0.0008) [2023-10-08 06:11:08,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 118390784. Throughput: 0: 1823.5, 1: 1840.9. Samples: 29597884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:11:08,755][130385] Avg episode reward: [(0, '59.560'), (1, '63.820')] [2023-10-08 06:11:10,287][00611] Updated weights for policy 0, policy_version 57642 (0.0010) [2023-10-08 06:11:10,652][00611] Updated weights for policy 0, policy_version 57652 (0.0009) [2023-10-08 06:11:11,017][00611] Updated weights for policy 0, policy_version 57662 (0.0009) [2023-10-08 06:11:12,189][00612] Updated weights for policy 1, policy_version 57990 (0.0010) [2023-10-08 06:11:12,562][00612] Updated weights for policy 1, policy_version 58000 (0.0007) [2023-10-08 06:11:12,937][00612] Updated weights for policy 1, policy_version 58010 (0.0009) [2023-10-08 06:11:13,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 118456320. Throughput: 0: 1830.3, 1: 1832.9. Samples: 29620210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:11:13,754][130385] Avg episode reward: [(0, '57.390'), (1, '66.120')] [2023-10-08 06:11:14,805][00611] Updated weights for policy 0, policy_version 57672 (0.0009) [2023-10-08 06:11:15,180][00611] Updated weights for policy 0, policy_version 57682 (0.0010) [2023-10-08 06:11:15,545][00611] Updated weights for policy 0, policy_version 57692 (0.0007) [2023-10-08 06:11:16,605][00612] Updated weights for policy 1, policy_version 58020 (0.0010) [2023-10-08 06:11:16,970][00612] Updated weights for policy 1, policy_version 58030 (0.0007) [2023-10-08 06:11:17,347][00612] Updated weights for policy 1, policy_version 58040 (0.0008) [2023-10-08 06:11:18,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 118521856. Throughput: 0: 1827.8, 1: 1836.5. Samples: 29641812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:11:18,754][130385] Avg episode reward: [(0, '53.040'), (1, '64.530')] [2023-10-08 06:11:19,082][00611] Updated weights for policy 0, policy_version 57702 (0.0008) [2023-10-08 06:11:19,446][00611] Updated weights for policy 0, policy_version 57712 (0.0008) [2023-10-08 06:11:19,814][00611] Updated weights for policy 0, policy_version 57722 (0.0007) [2023-10-08 06:11:21,000][00612] Updated weights for policy 1, policy_version 58050 (0.0009) [2023-10-08 06:11:21,360][00612] Updated weights for policy 1, policy_version 58060 (0.0009) [2023-10-08 06:11:21,733][00612] Updated weights for policy 1, policy_version 58070 (0.0008) [2023-10-08 06:11:22,097][00612] Updated weights for policy 1, policy_version 58080 (0.0007) [2023-10-08 06:11:23,516][00611] Updated weights for policy 0, policy_version 57732 (0.0008) [2023-10-08 06:11:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 118587392. Throughput: 0: 1833.1, 1: 1830.0. Samples: 29652982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:11:23,754][130385] Avg episode reward: [(0, '56.120'), (1, '66.890')] [2023-10-08 06:11:23,885][00611] Updated weights for policy 0, policy_version 57742 (0.0008) [2023-10-08 06:11:24,264][00611] Updated weights for policy 0, policy_version 57752 (0.0007) [2023-10-08 06:11:25,552][00612] Updated weights for policy 1, policy_version 58090 (0.0010) [2023-10-08 06:11:25,924][00612] Updated weights for policy 1, policy_version 58100 (0.0010) [2023-10-08 06:11:26,294][00612] Updated weights for policy 1, policy_version 58110 (0.0007) [2023-10-08 06:11:27,814][00611] Updated weights for policy 0, policy_version 57762 (0.0010) [2023-10-08 06:11:28,183][00611] Updated weights for policy 0, policy_version 57772 (0.0010) [2023-10-08 06:11:28,553][00611] Updated weights for policy 0, policy_version 57782 (0.0011) [2023-10-08 06:11:28,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 118652928. Throughput: 0: 1829.9, 1: 1842.8. Samples: 29675116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:11:28,755][130385] Avg episode reward: [(0, '57.190'), (1, '68.140')] [2023-10-08 06:11:28,926][00611] Updated weights for policy 0, policy_version 57792 (0.0010) [2023-10-08 06:11:29,625][00612] Updated weights for policy 1, policy_version 58120 (0.0011) [2023-10-08 06:11:30,001][00612] Updated weights for policy 1, policy_version 58130 (0.0009) [2023-10-08 06:11:30,373][00612] Updated weights for policy 1, policy_version 58140 (0.0009) [2023-10-08 06:11:32,449][00611] Updated weights for policy 0, policy_version 57802 (0.0008) [2023-10-08 06:11:32,825][00611] Updated weights for policy 0, policy_version 57812 (0.0010) [2023-10-08 06:11:33,209][00611] Updated weights for policy 0, policy_version 57822 (0.0008) [2023-10-08 06:11:33,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 118751232. Throughput: 0: 1828.8, 1: 1852.5. Samples: 29697322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:11:33,754][130385] Avg episode reward: [(0, '56.510'), (1, '71.240')] [2023-10-08 06:11:33,995][00612] Updated weights for policy 1, policy_version 58150 (0.0010) [2023-10-08 06:11:34,363][00612] Updated weights for policy 1, policy_version 58160 (0.0011) [2023-10-08 06:11:34,731][00612] Updated weights for policy 1, policy_version 58170 (0.0010) [2023-10-08 06:11:37,030][00611] Updated weights for policy 0, policy_version 57832 (0.0011) [2023-10-08 06:11:37,403][00611] Updated weights for policy 0, policy_version 57842 (0.0009) [2023-10-08 06:11:37,771][00611] Updated weights for policy 0, policy_version 57852 (0.0010) [2023-10-08 06:11:38,354][00612] Updated weights for policy 1, policy_version 58180 (0.0008) [2023-10-08 06:11:38,726][00612] Updated weights for policy 1, policy_version 58190 (0.0011) [2023-10-08 06:11:38,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 118816768. Throughput: 0: 1830.8, 1: 1853.2. Samples: 29708630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:11:38,754][130385] Avg episode reward: [(0, '56.730'), (1, '71.450')] [2023-10-08 06:11:39,087][00612] Updated weights for policy 1, policy_version 58200 (0.0010) [2023-10-08 06:11:41,585][00611] Updated weights for policy 0, policy_version 57862 (0.0008) [2023-10-08 06:11:41,953][00611] Updated weights for policy 0, policy_version 57872 (0.0008) [2023-10-08 06:11:42,324][00611] Updated weights for policy 0, policy_version 57882 (0.0010) [2023-10-08 06:11:42,754][00612] Updated weights for policy 1, policy_version 58210 (0.0009) [2023-10-08 06:11:43,124][00612] Updated weights for policy 1, policy_version 58220 (0.0008) [2023-10-08 06:11:43,495][00612] Updated weights for policy 1, policy_version 58230 (0.0010) [2023-10-08 06:11:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 118882304. Throughput: 0: 1827.7, 1: 1859.9. Samples: 29730712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:11:43,754][130385] Avg episode reward: [(0, '57.360'), (1, '66.530')] [2023-10-08 06:11:43,860][00612] Updated weights for policy 1, policy_version 58240 (0.0009) [2023-10-08 06:11:45,857][00611] Updated weights for policy 0, policy_version 57892 (0.0008) [2023-10-08 06:11:46,250][00611] Updated weights for policy 0, policy_version 57902 (0.0008) [2023-10-08 06:11:46,616][00611] Updated weights for policy 0, policy_version 57912 (0.0008) [2023-10-08 06:11:47,365][00612] Updated weights for policy 1, policy_version 58250 (0.0009) [2023-10-08 06:11:47,725][00612] Updated weights for policy 1, policy_version 58260 (0.0009) [2023-10-08 06:11:48,092][00612] Updated weights for policy 1, policy_version 58270 (0.0009) [2023-10-08 06:11:48,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 118980608. Throughput: 0: 1833.4, 1: 1833.1. Samples: 29752140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:11:48,754][130385] Avg episode reward: [(0, '63.630'), (1, '64.680')] [2023-10-08 06:11:50,214][00611] Updated weights for policy 0, policy_version 57922 (0.0008) [2023-10-08 06:11:50,584][00611] Updated weights for policy 0, policy_version 57932 (0.0009) [2023-10-08 06:11:50,959][00611] Updated weights for policy 0, policy_version 57942 (0.0008) [2023-10-08 06:11:51,327][00611] Updated weights for policy 0, policy_version 57952 (0.0011) [2023-10-08 06:11:51,771][00612] Updated weights for policy 1, policy_version 58280 (0.0009) [2023-10-08 06:11:52,135][00612] Updated weights for policy 1, policy_version 58290 (0.0009) [2023-10-08 06:11:52,492][00612] Updated weights for policy 1, policy_version 58300 (0.0012) [2023-10-08 06:11:53,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 119046144. Throughput: 0: 1824.8, 1: 1865.6. Samples: 29763952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:11:53,754][130385] Avg episode reward: [(0, '62.980'), (1, '62.880')] [2023-10-08 06:11:54,908][00611] Updated weights for policy 0, policy_version 57962 (0.0008) [2023-10-08 06:11:55,278][00611] Updated weights for policy 0, policy_version 57972 (0.0009) [2023-10-08 06:11:55,649][00611] Updated weights for policy 0, policy_version 57982 (0.0008) [2023-10-08 06:11:56,110][00612] Updated weights for policy 1, policy_version 58310 (0.0008) [2023-10-08 06:11:56,472][00612] Updated weights for policy 1, policy_version 58320 (0.0009) [2023-10-08 06:11:56,838][00612] Updated weights for policy 1, policy_version 58330 (0.0010) [2023-10-08 06:11:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 119111680. Throughput: 0: 1836.3, 1: 1833.5. Samples: 29785352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:11:58,754][130385] Avg episode reward: [(0, '64.670'), (1, '63.400')] [2023-10-08 06:11:59,226][00611] Updated weights for policy 0, policy_version 57992 (0.0009) [2023-10-08 06:11:59,599][00611] Updated weights for policy 0, policy_version 58002 (0.0009) [2023-10-08 06:11:59,962][00611] Updated weights for policy 0, policy_version 58012 (0.0010) [2023-10-08 06:12:00,398][00612] Updated weights for policy 1, policy_version 58340 (0.0010) [2023-10-08 06:12:00,768][00612] Updated weights for policy 1, policy_version 58350 (0.0010) [2023-10-08 06:12:01,128][00612] Updated weights for policy 1, policy_version 58360 (0.0007) [2023-10-08 06:12:03,666][00611] Updated weights for policy 0, policy_version 58022 (0.0009) [2023-10-08 06:12:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 119177216. Throughput: 0: 1836.2, 1: 1877.7. Samples: 29808938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:12:03,754][130385] Avg episode reward: [(0, '65.030'), (1, '65.560')] [2023-10-08 06:12:04,038][00611] Updated weights for policy 0, policy_version 58032 (0.0010) [2023-10-08 06:12:04,411][00611] Updated weights for policy 0, policy_version 58042 (0.0010) [2023-10-08 06:12:04,664][00612] Updated weights for policy 1, policy_version 58370 (0.0008) [2023-10-08 06:12:05,034][00612] Updated weights for policy 1, policy_version 58380 (0.0007) [2023-10-08 06:12:05,406][00612] Updated weights for policy 1, policy_version 58390 (0.0009) [2023-10-08 06:12:05,775][00612] Updated weights for policy 1, policy_version 58400 (0.0012) [2023-10-08 06:12:08,109][00611] Updated weights for policy 0, policy_version 58052 (0.0008) [2023-10-08 06:12:08,466][00611] Updated weights for policy 0, policy_version 58062 (0.0008) [2023-10-08 06:12:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 119242752. Throughput: 0: 1831.9, 1: 1855.9. Samples: 29818934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:12:08,754][130385] Avg episode reward: [(0, '67.240'), (1, '65.600')] [2023-10-08 06:12:08,840][00611] Updated weights for policy 0, policy_version 58072 (0.0007) [2023-10-08 06:12:09,381][00612] Updated weights for policy 1, policy_version 58410 (0.0008) [2023-10-08 06:12:09,746][00612] Updated weights for policy 1, policy_version 58420 (0.0008) [2023-10-08 06:12:10,117][00612] Updated weights for policy 1, policy_version 58430 (0.0008) [2023-10-08 06:12:12,569][00611] Updated weights for policy 0, policy_version 58082 (0.0009) [2023-10-08 06:12:12,951][00611] Updated weights for policy 0, policy_version 58092 (0.0009) [2023-10-08 06:12:13,318][00611] Updated weights for policy 0, policy_version 58102 (0.0010) [2023-10-08 06:12:13,640][00612] Updated weights for policy 1, policy_version 58440 (0.0008) [2023-10-08 06:12:13,682][00611] Updated weights for policy 0, policy_version 58112 (0.0008) [2023-10-08 06:12:13,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 119341056. Throughput: 0: 1833.7, 1: 1872.9. Samples: 29841916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:12:13,755][130385] Avg episode reward: [(0, '68.030'), (1, '67.410')] [2023-10-08 06:12:14,017][00612] Updated weights for policy 1, policy_version 58450 (0.0008) [2023-10-08 06:12:14,383][00612] Updated weights for policy 1, policy_version 58460 (0.0007) [2023-10-08 06:12:17,325][00611] Updated weights for policy 0, policy_version 58122 (0.0008) [2023-10-08 06:12:17,691][00611] Updated weights for policy 0, policy_version 58132 (0.0008) [2023-10-08 06:12:18,063][00611] Updated weights for policy 0, policy_version 58142 (0.0007) [2023-10-08 06:12:18,134][00612] Updated weights for policy 1, policy_version 58470 (0.0009) [2023-10-08 06:12:18,522][00612] Updated weights for policy 1, policy_version 58480 (0.0009) [2023-10-08 06:12:18,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 119406592. Throughput: 0: 1823.5, 1: 1856.4. Samples: 29862916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:12:18,754][130385] Avg episode reward: [(0, '65.660'), (1, '66.370')] [2023-10-08 06:12:18,762][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000058144_59539456.pth... [2023-10-08 06:12:18,795][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000056416_57769984.pth [2023-10-08 06:12:18,890][00612] Updated weights for policy 1, policy_version 58490 (0.0008) [2023-10-08 06:12:19,111][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000058496_59899904.pth... [2023-10-08 06:12:19,139][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000056736_58097664.pth [2023-10-08 06:12:21,842][00611] Updated weights for policy 0, policy_version 58152 (0.0010) [2023-10-08 06:12:22,210][00611] Updated weights for policy 0, policy_version 58162 (0.0011) [2023-10-08 06:12:22,582][00611] Updated weights for policy 0, policy_version 58172 (0.0008) [2023-10-08 06:12:22,695][00612] Updated weights for policy 1, policy_version 58500 (0.0009) [2023-10-08 06:12:23,057][00612] Updated weights for policy 1, policy_version 58510 (0.0011) [2023-10-08 06:12:23,428][00612] Updated weights for policy 1, policy_version 58520 (0.0009) [2023-10-08 06:12:23,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 119504896. Throughput: 0: 1828.4, 1: 1857.8. Samples: 29874506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:12:23,754][130385] Avg episode reward: [(0, '63.810'), (1, '65.930')] [2023-10-08 06:12:26,159][00611] Updated weights for policy 0, policy_version 58182 (0.0007) [2023-10-08 06:12:26,528][00611] Updated weights for policy 0, policy_version 58192 (0.0008) [2023-10-08 06:12:26,896][00611] Updated weights for policy 0, policy_version 58202 (0.0008) [2023-10-08 06:12:26,935][00612] Updated weights for policy 1, policy_version 58530 (0.0008) [2023-10-08 06:12:27,298][00612] Updated weights for policy 1, policy_version 58540 (0.0008) [2023-10-08 06:12:27,665][00612] Updated weights for policy 1, policy_version 58550 (0.0008) [2023-10-08 06:12:28,034][00612] Updated weights for policy 1, policy_version 58560 (0.0007) [2023-10-08 06:12:28,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 119570432. Throughput: 0: 1818.2, 1: 1850.1. Samples: 29895788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:12:28,754][130385] Avg episode reward: [(0, '63.740'), (1, '66.870')] [2023-10-08 06:12:30,705][00611] Updated weights for policy 0, policy_version 58212 (0.0009) [2023-10-08 06:12:31,098][00611] Updated weights for policy 0, policy_version 58222 (0.0008) [2023-10-08 06:12:31,461][00611] Updated weights for policy 0, policy_version 58232 (0.0008) [2023-10-08 06:12:31,654][00612] Updated weights for policy 1, policy_version 58570 (0.0007) [2023-10-08 06:12:32,013][00612] Updated weights for policy 1, policy_version 58580 (0.0008) [2023-10-08 06:12:32,385][00612] Updated weights for policy 1, policy_version 58590 (0.0009) [2023-10-08 06:12:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 119635968. Throughput: 0: 1822.0, 1: 1857.9. Samples: 29917734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:12:33,754][130385] Avg episode reward: [(0, '66.280'), (1, '66.290')] [2023-10-08 06:12:34,951][00611] Updated weights for policy 0, policy_version 58242 (0.0008) [2023-10-08 06:12:35,320][00611] Updated weights for policy 0, policy_version 58252 (0.0008) [2023-10-08 06:12:35,691][00611] Updated weights for policy 0, policy_version 58262 (0.0009) [2023-10-08 06:12:36,064][00611] Updated weights for policy 0, policy_version 58272 (0.0008) [2023-10-08 06:12:36,085][00612] Updated weights for policy 1, policy_version 58600 (0.0007) [2023-10-08 06:12:36,456][00612] Updated weights for policy 1, policy_version 58610 (0.0008) [2023-10-08 06:12:36,828][00612] Updated weights for policy 1, policy_version 58620 (0.0008) [2023-10-08 06:12:38,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 119701504. Throughput: 0: 1816.4, 1: 1843.2. Samples: 29928636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:12:38,755][130385] Avg episode reward: [(0, '65.720'), (1, '64.670')] [2023-10-08 06:12:39,626][00611] Updated weights for policy 0, policy_version 58282 (0.0010) [2023-10-08 06:12:40,010][00611] Updated weights for policy 0, policy_version 58292 (0.0008) [2023-10-08 06:12:40,375][00611] Updated weights for policy 0, policy_version 58302 (0.0008) [2023-10-08 06:12:40,513][00612] Updated weights for policy 1, policy_version 58630 (0.0009) [2023-10-08 06:12:40,880][00612] Updated weights for policy 1, policy_version 58640 (0.0011) [2023-10-08 06:12:41,258][00612] Updated weights for policy 1, policy_version 58650 (0.0010) [2023-10-08 06:12:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 119767040. Throughput: 0: 1821.6, 1: 1852.1. Samples: 29950668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:12:43,754][130385] Avg episode reward: [(0, '64.170'), (1, '65.500')] [2023-10-08 06:12:44,036][00611] Updated weights for policy 0, policy_version 58312 (0.0010) [2023-10-08 06:12:44,405][00611] Updated weights for policy 0, policy_version 58322 (0.0010) [2023-10-08 06:12:44,775][00611] Updated weights for policy 0, policy_version 58332 (0.0010) [2023-10-08 06:12:44,916][00612] Updated weights for policy 1, policy_version 58660 (0.0010) [2023-10-08 06:12:45,294][00612] Updated weights for policy 1, policy_version 58670 (0.0009) [2023-10-08 06:12:45,661][00612] Updated weights for policy 1, policy_version 58680 (0.0009) [2023-10-08 06:12:48,528][00611] Updated weights for policy 0, policy_version 58342 (0.0008) [2023-10-08 06:12:48,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 119832576. Throughput: 0: 1822.1, 1: 1843.7. Samples: 29973900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:12:48,754][130385] Avg episode reward: [(0, '62.330'), (1, '68.550')] [2023-10-08 06:12:48,897][00611] Updated weights for policy 0, policy_version 58352 (0.0007) [2023-10-08 06:12:49,172][00612] Updated weights for policy 1, policy_version 58690 (0.0007) [2023-10-08 06:12:49,267][00611] Updated weights for policy 0, policy_version 58362 (0.0009) [2023-10-08 06:12:49,545][00612] Updated weights for policy 1, policy_version 58700 (0.0008) [2023-10-08 06:12:49,923][00612] Updated weights for policy 1, policy_version 58710 (0.0009) [2023-10-08 06:12:50,297][00612] Updated weights for policy 1, policy_version 58720 (0.0008) [2023-10-08 06:12:53,032][00611] Updated weights for policy 0, policy_version 58372 (0.0008) [2023-10-08 06:12:53,408][00611] Updated weights for policy 0, policy_version 58382 (0.0010) [2023-10-08 06:12:53,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 119898112. Throughput: 0: 1822.4, 1: 1840.6. Samples: 29983770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:12:53,755][130385] Avg episode reward: [(0, '65.260'), (1, '68.690')] [2023-10-08 06:12:53,783][00611] Updated weights for policy 0, policy_version 58392 (0.0009) [2023-10-08 06:12:53,869][00612] Updated weights for policy 1, policy_version 58730 (0.0008) [2023-10-08 06:12:54,238][00612] Updated weights for policy 1, policy_version 58740 (0.0009) [2023-10-08 06:12:54,605][00612] Updated weights for policy 1, policy_version 58750 (0.0009) [2023-10-08 06:12:57,634][00611] Updated weights for policy 0, policy_version 58402 (0.0008) [2023-10-08 06:12:58,000][00611] Updated weights for policy 0, policy_version 58412 (0.0009) [2023-10-08 06:12:58,346][00612] Updated weights for policy 1, policy_version 58760 (0.0008) [2023-10-08 06:12:58,376][00611] Updated weights for policy 0, policy_version 58422 (0.0010) [2023-10-08 06:12:58,707][00612] Updated weights for policy 1, policy_version 58770 (0.0009) [2023-10-08 06:12:58,742][00611] Updated weights for policy 0, policy_version 58432 (0.0008) [2023-10-08 06:12:58,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 119996416. Throughput: 0: 1818.8, 1: 1845.6. Samples: 30006814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:12:58,754][130385] Avg episode reward: [(0, '63.290'), (1, '67.880')] [2023-10-08 06:12:59,077][00612] Updated weights for policy 1, policy_version 58780 (0.0010) [2023-10-08 06:13:02,427][00611] Updated weights for policy 0, policy_version 58442 (0.0007) [2023-10-08 06:13:02,793][00611] Updated weights for policy 0, policy_version 58452 (0.0008) [2023-10-08 06:13:02,894][00612] Updated weights for policy 1, policy_version 58790 (0.0011) [2023-10-08 06:13:03,166][00611] Updated weights for policy 0, policy_version 58462 (0.0008) [2023-10-08 06:13:03,280][00612] Updated weights for policy 1, policy_version 58800 (0.0009) [2023-10-08 06:13:03,650][00612] Updated weights for policy 1, policy_version 58810 (0.0007) [2023-10-08 06:13:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 120061952. Throughput: 0: 1825.2, 1: 1842.0. Samples: 30027942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:13:03,754][130385] Avg episode reward: [(0, '61.250'), (1, '69.470')] [2023-10-08 06:13:06,856][00611] Updated weights for policy 0, policy_version 58472 (0.0008) [2023-10-08 06:13:07,228][00611] Updated weights for policy 0, policy_version 58482 (0.0009) [2023-10-08 06:13:07,280][00612] Updated weights for policy 1, policy_version 58820 (0.0008) [2023-10-08 06:13:07,600][00611] Updated weights for policy 0, policy_version 58492 (0.0007) [2023-10-08 06:13:07,642][00612] Updated weights for policy 1, policy_version 58830 (0.0007) [2023-10-08 06:13:08,010][00612] Updated weights for policy 1, policy_version 58840 (0.0008) [2023-10-08 06:13:08,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 120160256. Throughput: 0: 1821.6, 1: 1850.3. Samples: 30039740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:13:08,754][130385] Avg episode reward: [(0, '60.130'), (1, '74.750')] [2023-10-08 06:13:11,204][00611] Updated weights for policy 0, policy_version 58502 (0.0009) [2023-10-08 06:13:11,570][00611] Updated weights for policy 0, policy_version 58512 (0.0008) [2023-10-08 06:13:11,600][00612] Updated weights for policy 1, policy_version 58850 (0.0008) [2023-10-08 06:13:11,944][00611] Updated weights for policy 0, policy_version 58522 (0.0009) [2023-10-08 06:13:11,970][00612] Updated weights for policy 1, policy_version 58860 (0.0009) [2023-10-08 06:13:12,336][00612] Updated weights for policy 1, policy_version 58870 (0.0008) [2023-10-08 06:13:12,708][00612] Updated weights for policy 1, policy_version 58880 (0.0007) [2023-10-08 06:13:13,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 120225792. Throughput: 0: 1830.6, 1: 1832.0. Samples: 30060604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:13:13,754][130385] Avg episode reward: [(0, '63.120'), (1, '75.470')] [2023-10-08 06:13:15,547][00611] Updated weights for policy 0, policy_version 58532 (0.0008) [2023-10-08 06:13:15,949][00611] Updated weights for policy 0, policy_version 58542 (0.0008) [2023-10-08 06:13:16,318][00611] Updated weights for policy 0, policy_version 58552 (0.0008) [2023-10-08 06:13:16,338][00612] Updated weights for policy 1, policy_version 58890 (0.0007) [2023-10-08 06:13:16,712][00612] Updated weights for policy 1, policy_version 58900 (0.0007) [2023-10-08 06:13:17,082][00612] Updated weights for policy 1, policy_version 58910 (0.0008) [2023-10-08 06:13:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 120291328. Throughput: 0: 1838.7, 1: 1842.3. Samples: 30083376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:13:18,754][130385] Avg episode reward: [(0, '65.150'), (1, '80.480')] [2023-10-08 06:13:19,932][00611] Updated weights for policy 0, policy_version 58562 (0.0007) [2023-10-08 06:13:20,303][00611] Updated weights for policy 0, policy_version 58572 (0.0009) [2023-10-08 06:13:20,634][00612] Updated weights for policy 1, policy_version 58920 (0.0008) [2023-10-08 06:13:20,669][00611] Updated weights for policy 0, policy_version 58582 (0.0008) [2023-10-08 06:13:21,004][00612] Updated weights for policy 1, policy_version 58930 (0.0008) [2023-10-08 06:13:21,032][00611] Updated weights for policy 0, policy_version 58592 (0.0008) [2023-10-08 06:13:21,369][00612] Updated weights for policy 1, policy_version 58940 (0.0007) [2023-10-08 06:13:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14773.4). Total num frames: 120356864. Throughput: 0: 1834.5, 1: 1829.5. Samples: 30093514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:13:23,755][130385] Avg episode reward: [(0, '59.520'), (1, '79.910')] [2023-10-08 06:13:24,571][00611] Updated weights for policy 0, policy_version 58602 (0.0007) [2023-10-08 06:13:24,945][00611] Updated weights for policy 0, policy_version 58612 (0.0007) [2023-10-08 06:13:25,099][00612] Updated weights for policy 1, policy_version 58950 (0.0007) [2023-10-08 06:13:25,310][00611] Updated weights for policy 0, policy_version 58622 (0.0009) [2023-10-08 06:13:25,464][00612] Updated weights for policy 1, policy_version 58960 (0.0008) [2023-10-08 06:13:25,838][00612] Updated weights for policy 1, policy_version 58970 (0.0007) [2023-10-08 06:13:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 120422400. Throughput: 0: 1834.4, 1: 1848.9. Samples: 30116416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:13:28,754][130385] Avg episode reward: [(0, '59.360'), (1, '81.630')] [2023-10-08 06:13:29,101][00611] Updated weights for policy 0, policy_version 58632 (0.0008) [2023-10-08 06:13:29,347][00612] Updated weights for policy 1, policy_version 58980 (0.0007) [2023-10-08 06:13:29,474][00611] Updated weights for policy 0, policy_version 58642 (0.0007) [2023-10-08 06:13:29,709][00612] Updated weights for policy 1, policy_version 58990 (0.0007) [2023-10-08 06:13:29,846][00611] Updated weights for policy 0, policy_version 58652 (0.0007) [2023-10-08 06:13:30,073][00612] Updated weights for policy 1, policy_version 59000 (0.0008) [2023-10-08 06:13:33,413][00611] Updated weights for policy 0, policy_version 58662 (0.0007) [2023-10-08 06:13:33,752][00612] Updated weights for policy 1, policy_version 59010 (0.0008) [2023-10-08 06:13:33,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 120487936. Throughput: 0: 1837.3, 1: 1845.7. Samples: 30139636. Policy #0 lag: (min: 26.0, avg: 26.2, max: 33.0) [2023-10-08 06:13:33,755][130385] Avg episode reward: [(0, '62.110'), (1, '79.100')] [2023-10-08 06:13:33,783][00611] Updated weights for policy 0, policy_version 58672 (0.0009) [2023-10-08 06:13:34,113][00612] Updated weights for policy 1, policy_version 59020 (0.0008) [2023-10-08 06:13:34,142][00611] Updated weights for policy 0, policy_version 58682 (0.0009) [2023-10-08 06:13:34,486][00612] Updated weights for policy 1, policy_version 59030 (0.0008) [2023-10-08 06:13:34,864][00612] Updated weights for policy 1, policy_version 59040 (0.0008) [2023-10-08 06:13:37,688][00611] Updated weights for policy 0, policy_version 58692 (0.0010) [2023-10-08 06:13:38,068][00611] Updated weights for policy 0, policy_version 58702 (0.0007) [2023-10-08 06:13:38,432][00611] Updated weights for policy 0, policy_version 58712 (0.0008) [2023-10-08 06:13:38,460][00612] Updated weights for policy 1, policy_version 59050 (0.0009) [2023-10-08 06:13:38,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 120586240. Throughput: 0: 1842.0, 1: 1843.5. Samples: 30149618. Policy #0 lag: (min: 26.0, avg: 26.2, max: 33.0) [2023-10-08 06:13:38,754][130385] Avg episode reward: [(0, '63.700'), (1, '77.540')] [2023-10-08 06:13:38,828][00612] Updated weights for policy 1, policy_version 59060 (0.0008) [2023-10-08 06:13:39,199][00612] Updated weights for policy 1, policy_version 59070 (0.0007) [2023-10-08 06:13:42,253][00611] Updated weights for policy 0, policy_version 58722 (0.0007) [2023-10-08 06:13:42,621][00611] Updated weights for policy 0, policy_version 58732 (0.0007) [2023-10-08 06:13:42,818][00612] Updated weights for policy 1, policy_version 59080 (0.0007) [2023-10-08 06:13:42,982][00611] Updated weights for policy 0, policy_version 58742 (0.0007) [2023-10-08 06:13:43,188][00612] Updated weights for policy 1, policy_version 59090 (0.0007) [2023-10-08 06:13:43,357][00611] Updated weights for policy 0, policy_version 58752 (0.0007) [2023-10-08 06:13:43,562][00612] Updated weights for policy 1, policy_version 59100 (0.0007) [2023-10-08 06:13:43,754][130385] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 120684544. Throughput: 0: 1843.5, 1: 1842.9. Samples: 30172700. Policy #0 lag: (min: 26.0, avg: 26.2, max: 33.0) [2023-10-08 06:13:43,754][130385] Avg episode reward: [(0, '60.560'), (1, '74.800')] [2023-10-08 06:13:46,962][00611] Updated weights for policy 0, policy_version 58762 (0.0007) [2023-10-08 06:13:47,203][00612] Updated weights for policy 1, policy_version 59110 (0.0008) [2023-10-08 06:13:47,334][00611] Updated weights for policy 0, policy_version 58772 (0.0009) [2023-10-08 06:13:47,601][00612] Updated weights for policy 1, policy_version 59120 (0.0009) [2023-10-08 06:13:47,698][00611] Updated weights for policy 0, policy_version 58782 (0.0009) [2023-10-08 06:13:47,966][00612] Updated weights for policy 1, policy_version 59130 (0.0007) [2023-10-08 06:13:48,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 120750080. Throughput: 0: 1836.4, 1: 1825.9. Samples: 30192742. Policy #0 lag: (min: 26.0, avg: 26.2, max: 33.0) [2023-10-08 06:13:48,755][130385] Avg episode reward: [(0, '60.560'), (1, '76.230')] [2023-10-08 06:13:51,385][00611] Updated weights for policy 0, policy_version 58792 (0.0009) [2023-10-08 06:13:51,557][00612] Updated weights for policy 1, policy_version 59140 (0.0008) [2023-10-08 06:13:51,750][00611] Updated weights for policy 0, policy_version 58802 (0.0009) [2023-10-08 06:13:51,931][00612] Updated weights for policy 1, policy_version 59150 (0.0009) [2023-10-08 06:13:52,116][00611] Updated weights for policy 0, policy_version 58812 (0.0008) [2023-10-08 06:13:52,289][00612] Updated weights for policy 1, policy_version 59160 (0.0008) [2023-10-08 06:13:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 120815616. Throughput: 0: 1838.2, 1: 1846.0. Samples: 30205530. Policy #0 lag: (min: 26.0, avg: 26.2, max: 33.0) [2023-10-08 06:13:53,755][130385] Avg episode reward: [(0, '64.070'), (1, '76.960')] [2023-10-08 06:13:55,623][00611] Updated weights for policy 0, policy_version 58822 (0.0007) [2023-10-08 06:13:55,994][00611] Updated weights for policy 0, policy_version 58832 (0.0009) [2023-10-08 06:13:56,026][00612] Updated weights for policy 1, policy_version 59170 (0.0010) [2023-10-08 06:13:56,366][00611] Updated weights for policy 0, policy_version 58842 (0.0008) [2023-10-08 06:13:56,392][00612] Updated weights for policy 1, policy_version 59180 (0.0007) [2023-10-08 06:13:56,766][00612] Updated weights for policy 1, policy_version 59190 (0.0008) [2023-10-08 06:13:57,125][00612] Updated weights for policy 1, policy_version 59200 (0.0008) [2023-10-08 06:13:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 120881152. Throughput: 0: 1833.8, 1: 1830.5. Samples: 30225498. Policy #0 lag: (min: 26.0, avg: 26.2, max: 33.0) [2023-10-08 06:13:58,754][130385] Avg episode reward: [(0, '65.280'), (1, '75.700')] [2023-10-08 06:14:00,113][00611] Updated weights for policy 0, policy_version 58852 (0.0008) [2023-10-08 06:14:00,478][00611] Updated weights for policy 0, policy_version 58862 (0.0010) [2023-10-08 06:14:00,830][00612] Updated weights for policy 1, policy_version 59210 (0.0008) [2023-10-08 06:14:00,845][00611] Updated weights for policy 0, policy_version 58872 (0.0009) [2023-10-08 06:14:01,195][00612] Updated weights for policy 1, policy_version 59220 (0.0011) [2023-10-08 06:14:01,566][00612] Updated weights for policy 1, policy_version 59230 (0.0007) [2023-10-08 06:14:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 120946688. Throughput: 0: 1828.0, 1: 1839.8. Samples: 30248428. Policy #0 lag: (min: 26.0, avg: 26.2, max: 33.0) [2023-10-08 06:14:03,754][130385] Avg episode reward: [(0, '68.450'), (1, '77.560')] [2023-10-08 06:14:04,446][00611] Updated weights for policy 0, policy_version 58882 (0.0009) [2023-10-08 06:14:04,828][00611] Updated weights for policy 0, policy_version 58892 (0.0009) [2023-10-08 06:14:05,191][00611] Updated weights for policy 0, policy_version 58902 (0.0008) [2023-10-08 06:14:05,299][00612] Updated weights for policy 1, policy_version 59240 (0.0007) [2023-10-08 06:14:05,557][00611] Updated weights for policy 0, policy_version 58912 (0.0010) [2023-10-08 06:14:05,667][00612] Updated weights for policy 1, policy_version 59250 (0.0009) [2023-10-08 06:14:06,041][00612] Updated weights for policy 1, policy_version 59260 (0.0008) [2023-10-08 06:14:08,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 121012224. Throughput: 0: 1831.5, 1: 1834.9. Samples: 30258504. Policy #0 lag: (min: 26.0, avg: 26.2, max: 33.0) [2023-10-08 06:14:08,754][130385] Avg episode reward: [(0, '64.830'), (1, '81.090')] [2023-10-08 06:14:09,215][00611] Updated weights for policy 0, policy_version 58922 (0.0007) [2023-10-08 06:14:09,539][00612] Updated weights for policy 1, policy_version 59270 (0.0007) [2023-10-08 06:14:09,590][00611] Updated weights for policy 0, policy_version 58932 (0.0008) [2023-10-08 06:14:09,907][00612] Updated weights for policy 1, policy_version 59280 (0.0007) [2023-10-08 06:14:09,953][00611] Updated weights for policy 0, policy_version 58942 (0.0009) [2023-10-08 06:14:10,269][00612] Updated weights for policy 1, policy_version 59290 (0.0007) [2023-10-08 06:14:13,653][00611] Updated weights for policy 0, policy_version 58952 (0.0010) [2023-10-08 06:14:13,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 121077760. Throughput: 0: 1825.6, 1: 1843.0. Samples: 30281502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:14:13,755][130385] Avg episode reward: [(0, '65.960'), (1, '78.840')] [2023-10-08 06:14:13,943][00612] Updated weights for policy 1, policy_version 59300 (0.0008) [2023-10-08 06:14:14,026][00611] Updated weights for policy 0, policy_version 58962 (0.0009) [2023-10-08 06:14:14,306][00612] Updated weights for policy 1, policy_version 59310 (0.0007) [2023-10-08 06:14:14,396][00611] Updated weights for policy 0, policy_version 58972 (0.0009) [2023-10-08 06:14:14,673][00612] Updated weights for policy 1, policy_version 59320 (0.0007) [2023-10-08 06:14:18,033][00611] Updated weights for policy 0, policy_version 58982 (0.0008) [2023-10-08 06:14:18,394][00611] Updated weights for policy 0, policy_version 58992 (0.0008) [2023-10-08 06:14:18,531][00612] Updated weights for policy 1, policy_version 59330 (0.0007) [2023-10-08 06:14:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 121143296. Throughput: 0: 1818.0, 1: 1838.1. Samples: 30304158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:14:18,754][130385] Avg episode reward: [(0, '66.020'), (1, '75.030')] [2023-10-08 06:14:18,763][00611] Updated weights for policy 0, policy_version 59002 (0.0009) [2023-10-08 06:14:18,897][00612] Updated weights for policy 1, policy_version 59340 (0.0008) [2023-10-08 06:14:18,983][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000059008_60424192.pth... [2023-10-08 06:14:19,022][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000057280_58654720.pth [2023-10-08 06:14:19,271][00612] Updated weights for policy 1, policy_version 59350 (0.0009) [2023-10-08 06:14:19,643][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000059360_60784640.pth... [2023-10-08 06:14:19,645][00612] Updated weights for policy 1, policy_version 59360 (0.0010) [2023-10-08 06:14:19,680][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000057632_59015168.pth [2023-10-08 06:14:22,538][00611] Updated weights for policy 0, policy_version 59012 (0.0007) [2023-10-08 06:14:22,907][00611] Updated weights for policy 0, policy_version 59022 (0.0007) [2023-10-08 06:14:23,220][00612] Updated weights for policy 1, policy_version 59370 (0.0007) [2023-10-08 06:14:23,277][00611] Updated weights for policy 0, policy_version 59032 (0.0008) [2023-10-08 06:14:23,588][00612] Updated weights for policy 1, policy_version 59380 (0.0007) [2023-10-08 06:14:23,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121241600. Throughput: 0: 1828.7, 1: 1836.7. Samples: 30314560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:14:23,754][130385] Avg episode reward: [(0, '66.410'), (1, '75.180')] [2023-10-08 06:14:23,961][00612] Updated weights for policy 1, policy_version 59390 (0.0011) [2023-10-08 06:14:26,831][00611] Updated weights for policy 0, policy_version 59042 (0.0008) [2023-10-08 06:14:27,202][00611] Updated weights for policy 0, policy_version 59052 (0.0007) [2023-10-08 06:14:27,576][00611] Updated weights for policy 0, policy_version 59062 (0.0007) [2023-10-08 06:14:27,660][00612] Updated weights for policy 1, policy_version 59400 (0.0008) [2023-10-08 06:14:27,947][00611] Updated weights for policy 0, policy_version 59072 (0.0010) [2023-10-08 06:14:28,025][00612] Updated weights for policy 1, policy_version 59410 (0.0008) [2023-10-08 06:14:28,383][00612] Updated weights for policy 1, policy_version 59420 (0.0009) [2023-10-08 06:14:28,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 121339904. Throughput: 0: 1820.2, 1: 1832.3. Samples: 30337062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:14:28,754][130385] Avg episode reward: [(0, '69.380'), (1, '75.140')] [2023-10-08 06:14:31,580][00611] Updated weights for policy 0, policy_version 59082 (0.0008) [2023-10-08 06:14:31,906][00612] Updated weights for policy 1, policy_version 59430 (0.0007) [2023-10-08 06:14:31,954][00611] Updated weights for policy 0, policy_version 59092 (0.0008) [2023-10-08 06:14:32,272][00612] Updated weights for policy 1, policy_version 59440 (0.0007) [2023-10-08 06:14:32,336][00611] Updated weights for policy 0, policy_version 59102 (0.0009) [2023-10-08 06:14:32,633][00612] Updated weights for policy 1, policy_version 59450 (0.0007) [2023-10-08 06:14:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 121405440. Throughput: 0: 1835.3, 1: 1838.4. Samples: 30358060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:14:33,754][130385] Avg episode reward: [(0, '70.350'), (1, '73.170')] [2023-10-08 06:14:36,040][00611] Updated weights for policy 0, policy_version 59112 (0.0008) [2023-10-08 06:14:36,274][00612] Updated weights for policy 1, policy_version 59460 (0.0007) [2023-10-08 06:14:36,405][00611] Updated weights for policy 0, policy_version 59122 (0.0008) [2023-10-08 06:14:36,659][00612] Updated weights for policy 1, policy_version 59470 (0.0008) [2023-10-08 06:14:36,772][00611] Updated weights for policy 0, policy_version 59132 (0.0007) [2023-10-08 06:14:37,028][00612] Updated weights for policy 1, policy_version 59480 (0.0008) [2023-10-08 06:14:38,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 121470976. Throughput: 0: 1822.2, 1: 1838.4. Samples: 30370260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:14:38,755][130385] Avg episode reward: [(0, '73.450'), (1, '72.730')] [2023-10-08 06:14:40,574][00611] Updated weights for policy 0, policy_version 59142 (0.0008) [2023-10-08 06:14:40,732][00612] Updated weights for policy 1, policy_version 59490 (0.0007) [2023-10-08 06:14:40,945][00611] Updated weights for policy 0, policy_version 59152 (0.0009) [2023-10-08 06:14:41,099][00612] Updated weights for policy 1, policy_version 59500 (0.0008) [2023-10-08 06:14:41,324][00611] Updated weights for policy 0, policy_version 59162 (0.0007) [2023-10-08 06:14:41,466][00612] Updated weights for policy 1, policy_version 59510 (0.0007) [2023-10-08 06:14:41,828][00612] Updated weights for policy 1, policy_version 59520 (0.0009) [2023-10-08 06:14:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 121536512. Throughput: 0: 1832.5, 1: 1842.3. Samples: 30390862. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:14:43,754][130385] Avg episode reward: [(0, '74.170'), (1, '76.230')] [2023-10-08 06:14:45,003][00611] Updated weights for policy 0, policy_version 59172 (0.0008) [2023-10-08 06:14:45,366][00612] Updated weights for policy 1, policy_version 59530 (0.0008) [2023-10-08 06:14:45,372][00611] Updated weights for policy 0, policy_version 59182 (0.0007) [2023-10-08 06:14:45,733][00612] Updated weights for policy 1, policy_version 59540 (0.0008) [2023-10-08 06:14:45,743][00611] Updated weights for policy 0, policy_version 59192 (0.0008) [2023-10-08 06:14:46,103][00612] Updated weights for policy 1, policy_version 59550 (0.0007) [2023-10-08 06:14:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 121602048. Throughput: 0: 1833.1, 1: 1846.5. Samples: 30414010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:14:48,755][130385] Avg episode reward: [(0, '72.960'), (1, '77.060')] [2023-10-08 06:14:49,567][00611] Updated weights for policy 0, policy_version 59202 (0.0009) [2023-10-08 06:14:49,676][00612] Updated weights for policy 1, policy_version 59560 (0.0007) [2023-10-08 06:14:49,959][00611] Updated weights for policy 0, policy_version 59212 (0.0008) [2023-10-08 06:14:50,044][00612] Updated weights for policy 1, policy_version 59570 (0.0008) [2023-10-08 06:14:50,343][00611] Updated weights for policy 0, policy_version 59222 (0.0009) [2023-10-08 06:14:50,413][00612] Updated weights for policy 1, policy_version 59580 (0.0007) [2023-10-08 06:14:50,712][00611] Updated weights for policy 0, policy_version 59232 (0.0008) [2023-10-08 06:14:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 121667584. Throughput: 0: 1829.2, 1: 1845.8. Samples: 30423882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:14:53,754][130385] Avg episode reward: [(0, '74.190'), (1, '76.470')] [2023-10-08 06:14:54,134][00612] Updated weights for policy 1, policy_version 59590 (0.0008) [2023-10-08 06:14:54,347][00611] Updated weights for policy 0, policy_version 59242 (0.0009) [2023-10-08 06:14:54,498][00612] Updated weights for policy 1, policy_version 59600 (0.0008) [2023-10-08 06:14:54,719][00611] Updated weights for policy 0, policy_version 59252 (0.0008) [2023-10-08 06:14:54,874][00612] Updated weights for policy 1, policy_version 59610 (0.0008) [2023-10-08 06:14:55,093][00611] Updated weights for policy 0, policy_version 59262 (0.0007) [2023-10-08 06:14:58,593][00611] Updated weights for policy 0, policy_version 59272 (0.0010) [2023-10-08 06:14:58,636][00612] Updated weights for policy 1, policy_version 59620 (0.0008) [2023-10-08 06:14:58,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 121733120. Throughput: 0: 1830.4, 1: 1839.8. Samples: 30446664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:14:58,754][130385] Avg episode reward: [(0, '75.300'), (1, '75.460')] [2023-10-08 06:14:58,954][00611] Updated weights for policy 0, policy_version 59282 (0.0008) [2023-10-08 06:14:59,001][00612] Updated weights for policy 1, policy_version 59630 (0.0009) [2023-10-08 06:14:59,333][00611] Updated weights for policy 0, policy_version 59292 (0.0008) [2023-10-08 06:14:59,375][00612] Updated weights for policy 1, policy_version 59640 (0.0008) [2023-10-08 06:15:03,006][00611] Updated weights for policy 0, policy_version 59302 (0.0009) [2023-10-08 06:15:03,138][00612] Updated weights for policy 1, policy_version 59650 (0.0009) [2023-10-08 06:15:03,377][00611] Updated weights for policy 0, policy_version 59312 (0.0007) [2023-10-08 06:15:03,506][00612] Updated weights for policy 1, policy_version 59660 (0.0007) [2023-10-08 06:15:03,743][00611] Updated weights for policy 0, policy_version 59322 (0.0007) [2023-10-08 06:15:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 121798656. Throughput: 0: 1831.0, 1: 1832.7. Samples: 30469024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:15:03,754][130385] Avg episode reward: [(0, '72.830'), (1, '76.110')] [2023-10-08 06:15:03,870][00612] Updated weights for policy 1, policy_version 59670 (0.0008) [2023-10-08 06:15:04,245][00612] Updated weights for policy 1, policy_version 59680 (0.0007) [2023-10-08 06:15:07,462][00611] Updated weights for policy 0, policy_version 59332 (0.0009) [2023-10-08 06:15:07,833][00611] Updated weights for policy 0, policy_version 59342 (0.0007) [2023-10-08 06:15:07,892][00612] Updated weights for policy 1, policy_version 59690 (0.0007) [2023-10-08 06:15:08,210][00611] Updated weights for policy 0, policy_version 59352 (0.0007) [2023-10-08 06:15:08,263][00612] Updated weights for policy 1, policy_version 59700 (0.0009) [2023-10-08 06:15:08,636][00612] Updated weights for policy 1, policy_version 59710 (0.0009) [2023-10-08 06:15:08,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 121929728. Throughput: 0: 1828.8, 1: 1836.6. Samples: 30479502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:15:08,754][130385] Avg episode reward: [(0, '72.780'), (1, '71.720')] [2023-10-08 06:15:11,805][00611] Updated weights for policy 0, policy_version 59362 (0.0007) [2023-10-08 06:15:12,183][00611] Updated weights for policy 0, policy_version 59372 (0.0007) [2023-10-08 06:15:12,398][00612] Updated weights for policy 1, policy_version 59720 (0.0008) [2023-10-08 06:15:12,554][00611] Updated weights for policy 0, policy_version 59382 (0.0008) [2023-10-08 06:15:12,753][00612] Updated weights for policy 1, policy_version 59730 (0.0010) [2023-10-08 06:15:12,924][00611] Updated weights for policy 0, policy_version 59392 (0.0008) [2023-10-08 06:15:13,119][00612] Updated weights for policy 1, policy_version 59740 (0.0010) [2023-10-08 06:15:13,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 121995264. Throughput: 0: 1827.6, 1: 1832.9. Samples: 30501788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:15:13,755][130385] Avg episode reward: [(0, '70.790'), (1, '71.480')] [2023-10-08 06:15:16,601][00611] Updated weights for policy 0, policy_version 59402 (0.0008) [2023-10-08 06:15:16,735][00612] Updated weights for policy 1, policy_version 59750 (0.0009) [2023-10-08 06:15:16,973][00611] Updated weights for policy 0, policy_version 59412 (0.0007) [2023-10-08 06:15:17,101][00612] Updated weights for policy 1, policy_version 59760 (0.0008) [2023-10-08 06:15:17,348][00611] Updated weights for policy 0, policy_version 59422 (0.0007) [2023-10-08 06:15:17,459][00612] Updated weights for policy 1, policy_version 59770 (0.0007) [2023-10-08 06:15:18,754][130385] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 122060800. Throughput: 0: 1827.1, 1: 1826.8. Samples: 30522486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:15:18,754][130385] Avg episode reward: [(0, '69.040'), (1, '71.890')] [2023-10-08 06:15:20,889][00611] Updated weights for policy 0, policy_version 59432 (0.0009) [2023-10-08 06:15:21,140][00612] Updated weights for policy 1, policy_version 59780 (0.0008) [2023-10-08 06:15:21,262][00611] Updated weights for policy 0, policy_version 59442 (0.0010) [2023-10-08 06:15:21,527][00612] Updated weights for policy 1, policy_version 59790 (0.0007) [2023-10-08 06:15:21,628][00611] Updated weights for policy 0, policy_version 59452 (0.0007) [2023-10-08 06:15:21,897][00612] Updated weights for policy 1, policy_version 59800 (0.0008) [2023-10-08 06:15:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 122126336. Throughput: 0: 1832.8, 1: 1820.5. Samples: 30534654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:15:23,754][130385] Avg episode reward: [(0, '66.120'), (1, '73.960')] [2023-10-08 06:15:25,056][00611] Updated weights for policy 0, policy_version 59462 (0.0007) [2023-10-08 06:15:25,432][00611] Updated weights for policy 0, policy_version 59472 (0.0009) [2023-10-08 06:15:25,687][00612] Updated weights for policy 1, policy_version 59810 (0.0009) [2023-10-08 06:15:25,797][00611] Updated weights for policy 0, policy_version 59482 (0.0007) [2023-10-08 06:15:26,052][00612] Updated weights for policy 1, policy_version 59820 (0.0008) [2023-10-08 06:15:26,422][00612] Updated weights for policy 1, policy_version 59830 (0.0007) [2023-10-08 06:15:26,783][00612] Updated weights for policy 1, policy_version 59840 (0.0007) [2023-10-08 06:15:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 122191872. Throughput: 0: 1839.7, 1: 1820.9. Samples: 30555592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:15:28,754][130385] Avg episode reward: [(0, '64.420'), (1, '77.290')] [2023-10-08 06:15:29,543][00611] Updated weights for policy 0, policy_version 59492 (0.0009) [2023-10-08 06:15:29,908][00611] Updated weights for policy 0, policy_version 59502 (0.0008) [2023-10-08 06:15:30,245][00612] Updated weights for policy 1, policy_version 59850 (0.0009) [2023-10-08 06:15:30,274][00611] Updated weights for policy 0, policy_version 59512 (0.0009) [2023-10-08 06:15:30,601][00612] Updated weights for policy 1, policy_version 59860 (0.0007) [2023-10-08 06:15:30,965][00612] Updated weights for policy 1, policy_version 59870 (0.0009) [2023-10-08 06:15:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 122257408. Throughput: 0: 1840.9, 1: 1815.3. Samples: 30578540. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 06:15:33,756][130385] Avg episode reward: [(0, '60.900'), (1, '75.670')] [2023-10-08 06:15:33,818][00611] Updated weights for policy 0, policy_version 59522 (0.0007) [2023-10-08 06:15:34,196][00611] Updated weights for policy 0, policy_version 59532 (0.0007) [2023-10-08 06:15:34,565][00611] Updated weights for policy 0, policy_version 59542 (0.0007) [2023-10-08 06:15:34,724][00612] Updated weights for policy 1, policy_version 59880 (0.0007) [2023-10-08 06:15:34,932][00611] Updated weights for policy 0, policy_version 59552 (0.0007) [2023-10-08 06:15:35,096][00612] Updated weights for policy 1, policy_version 59890 (0.0007) [2023-10-08 06:15:35,452][00612] Updated weights for policy 1, policy_version 59900 (0.0008) [2023-10-08 06:15:38,691][00611] Updated weights for policy 0, policy_version 59562 (0.0009) [2023-10-08 06:15:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 122322944. Throughput: 0: 1848.4, 1: 1813.9. Samples: 30588686. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 06:15:38,754][130385] Avg episode reward: [(0, '60.320'), (1, '72.740')] [2023-10-08 06:15:39,058][00611] Updated weights for policy 0, policy_version 59572 (0.0008) [2023-10-08 06:15:39,125][00612] Updated weights for policy 1, policy_version 59910 (0.0008) [2023-10-08 06:15:39,429][00611] Updated weights for policy 0, policy_version 59582 (0.0008) [2023-10-08 06:15:39,497][00612] Updated weights for policy 1, policy_version 59920 (0.0010) [2023-10-08 06:15:39,867][00612] Updated weights for policy 1, policy_version 59930 (0.0007) [2023-10-08 06:15:43,104][00611] Updated weights for policy 0, policy_version 59592 (0.0008) [2023-10-08 06:15:43,479][00611] Updated weights for policy 0, policy_version 59602 (0.0008) [2023-10-08 06:15:43,567][00612] Updated weights for policy 1, policy_version 59940 (0.0009) [2023-10-08 06:15:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 122388480. Throughput: 0: 1846.0, 1: 1815.6. Samples: 30611434. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 06:15:43,754][130385] Avg episode reward: [(0, '57.700'), (1, '71.020')] [2023-10-08 06:15:43,850][00611] Updated weights for policy 0, policy_version 59612 (0.0009) [2023-10-08 06:15:43,927][00612] Updated weights for policy 1, policy_version 59950 (0.0008) [2023-10-08 06:15:44,296][00612] Updated weights for policy 1, policy_version 59960 (0.0007) [2023-10-08 06:15:47,406][00611] Updated weights for policy 0, policy_version 59622 (0.0008) [2023-10-08 06:15:47,776][00611] Updated weights for policy 0, policy_version 59632 (0.0009) [2023-10-08 06:15:47,953][00612] Updated weights for policy 1, policy_version 59970 (0.0007) [2023-10-08 06:15:48,149][00611] Updated weights for policy 0, policy_version 59642 (0.0008) [2023-10-08 06:15:48,310][00612] Updated weights for policy 1, policy_version 59980 (0.0009) [2023-10-08 06:15:48,677][00612] Updated weights for policy 1, policy_version 59990 (0.0010) [2023-10-08 06:15:48,754][130385] Fps is (10 sec: 16383.3, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 122486784. Throughput: 0: 1830.9, 1: 1820.0. Samples: 30633316. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 06:15:48,755][130385] Avg episode reward: [(0, '57.210'), (1, '67.690')] [2023-10-08 06:15:49,043][00612] Updated weights for policy 1, policy_version 60000 (0.0007) [2023-10-08 06:15:51,789][00611] Updated weights for policy 0, policy_version 59652 (0.0008) [2023-10-08 06:15:52,166][00611] Updated weights for policy 0, policy_version 59662 (0.0007) [2023-10-08 06:15:52,527][00611] Updated weights for policy 0, policy_version 59672 (0.0009) [2023-10-08 06:15:52,696][00612] Updated weights for policy 1, policy_version 60010 (0.0008) [2023-10-08 06:15:53,074][00612] Updated weights for policy 1, policy_version 60020 (0.0009) [2023-10-08 06:15:53,450][00612] Updated weights for policy 1, policy_version 60030 (0.0007) [2023-10-08 06:15:53,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 122585088. Throughput: 0: 1851.3, 1: 1830.0. Samples: 30645160. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 06:15:53,754][130385] Avg episode reward: [(0, '57.690'), (1, '68.470')] [2023-10-08 06:15:56,161][00611] Updated weights for policy 0, policy_version 59682 (0.0008) [2023-10-08 06:15:56,539][00611] Updated weights for policy 0, policy_version 59692 (0.0009) [2023-10-08 06:15:56,904][00611] Updated weights for policy 0, policy_version 59702 (0.0007) [2023-10-08 06:15:57,074][00612] Updated weights for policy 1, policy_version 60040 (0.0007) [2023-10-08 06:15:57,271][00611] Updated weights for policy 0, policy_version 59712 (0.0008) [2023-10-08 06:15:57,431][00612] Updated weights for policy 1, policy_version 60050 (0.0007) [2023-10-08 06:15:57,806][00612] Updated weights for policy 1, policy_version 60060 (0.0007) [2023-10-08 06:15:58,754][130385] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 122650624. Throughput: 0: 1837.4, 1: 1823.7. Samples: 30666538. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 06:15:58,755][130385] Avg episode reward: [(0, '55.480'), (1, '72.250')] [2023-10-08 06:16:00,972][00611] Updated weights for policy 0, policy_version 59722 (0.0009) [2023-10-08 06:16:01,297][00612] Updated weights for policy 1, policy_version 60070 (0.0010) [2023-10-08 06:16:01,345][00611] Updated weights for policy 0, policy_version 59732 (0.0009) [2023-10-08 06:16:01,668][00612] Updated weights for policy 1, policy_version 60080 (0.0007) [2023-10-08 06:16:01,714][00611] Updated weights for policy 0, policy_version 59742 (0.0009) [2023-10-08 06:16:02,053][00612] Updated weights for policy 1, policy_version 60090 (0.0009) [2023-10-08 06:16:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 122716160. Throughput: 0: 1849.9, 1: 1836.3. Samples: 30688364. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 06:16:03,754][130385] Avg episode reward: [(0, '54.000'), (1, '69.740')] [2023-10-08 06:16:05,373][00611] Updated weights for policy 0, policy_version 59752 (0.0009) [2023-10-08 06:16:05,692][00612] Updated weights for policy 1, policy_version 60100 (0.0010) [2023-10-08 06:16:05,739][00611] Updated weights for policy 0, policy_version 59762 (0.0008) [2023-10-08 06:16:06,063][00612] Updated weights for policy 1, policy_version 60110 (0.0009) [2023-10-08 06:16:06,115][00611] Updated weights for policy 0, policy_version 59772 (0.0007) [2023-10-08 06:16:06,437][00612] Updated weights for policy 1, policy_version 60120 (0.0007) [2023-10-08 06:16:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 122781696. Throughput: 0: 1830.7, 1: 1825.9. Samples: 30699206. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 06:16:08,755][130385] Avg episode reward: [(0, '56.730'), (1, '70.580')] [2023-10-08 06:16:09,701][00611] Updated weights for policy 0, policy_version 59782 (0.0010) [2023-10-08 06:16:10,071][00611] Updated weights for policy 0, policy_version 59792 (0.0010) [2023-10-08 06:16:10,106][00612] Updated weights for policy 1, policy_version 60130 (0.0007) [2023-10-08 06:16:10,455][00611] Updated weights for policy 0, policy_version 59802 (0.0009) [2023-10-08 06:16:10,481][00612] Updated weights for policy 1, policy_version 60140 (0.0010) [2023-10-08 06:16:10,846][00612] Updated weights for policy 1, policy_version 60150 (0.0009) [2023-10-08 06:16:11,213][00612] Updated weights for policy 1, policy_version 60160 (0.0010) [2023-10-08 06:16:13,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 122847232. Throughput: 0: 1842.5, 1: 1840.1. Samples: 30721310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:16:13,754][130385] Avg episode reward: [(0, '60.820'), (1, '71.890')] [2023-10-08 06:16:14,058][00611] Updated weights for policy 0, policy_version 59812 (0.0007) [2023-10-08 06:16:14,425][00611] Updated weights for policy 0, policy_version 59822 (0.0007) [2023-10-08 06:16:14,807][00611] Updated weights for policy 0, policy_version 59832 (0.0007) [2023-10-08 06:16:14,814][00612] Updated weights for policy 1, policy_version 60170 (0.0009) [2023-10-08 06:16:15,179][00612] Updated weights for policy 1, policy_version 60180 (0.0007) [2023-10-08 06:16:15,540][00612] Updated weights for policy 1, policy_version 60190 (0.0009) [2023-10-08 06:16:18,416][00611] Updated weights for policy 0, policy_version 59842 (0.0008) [2023-10-08 06:16:18,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 122912768. Throughput: 0: 1848.2, 1: 1843.0. Samples: 30744642. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:16:18,755][130385] Avg episode reward: [(0, '62.980'), (1, '73.330')] [2023-10-08 06:16:18,767][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000060192_61636608.pth... [2023-10-08 06:16:18,786][00611] Updated weights for policy 0, policy_version 59852 (0.0010) [2023-10-08 06:16:18,797][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000058496_59899904.pth [2023-10-08 06:16:18,801][00425] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p1/milestones/checkpoint_000060192_61636608.pth [2023-10-08 06:16:19,153][00611] Updated weights for policy 0, policy_version 59862 (0.0007) [2023-10-08 06:16:19,259][00612] Updated weights for policy 1, policy_version 60200 (0.0009) [2023-10-08 06:16:19,527][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000059872_61308928.pth... [2023-10-08 06:16:19,532][00611] Updated weights for policy 0, policy_version 59872 (0.0008) [2023-10-08 06:16:19,562][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000058144_59539456.pth [2023-10-08 06:16:19,568][00365] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p0/milestones/checkpoint_000059872_61308928.pth [2023-10-08 06:16:19,626][00612] Updated weights for policy 1, policy_version 60210 (0.0008) [2023-10-08 06:16:19,992][00612] Updated weights for policy 1, policy_version 60220 (0.0010) [2023-10-08 06:16:23,298][00611] Updated weights for policy 0, policy_version 59882 (0.0008) [2023-10-08 06:16:23,522][00612] Updated weights for policy 1, policy_version 60230 (0.0008) [2023-10-08 06:16:23,671][00611] Updated weights for policy 0, policy_version 59892 (0.0007) [2023-10-08 06:16:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 122978304. Throughput: 0: 1841.2, 1: 1842.6. Samples: 30754460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:16:23,754][130385] Avg episode reward: [(0, '62.970'), (1, '69.350')] [2023-10-08 06:16:23,886][00612] Updated weights for policy 1, policy_version 60240 (0.0008) [2023-10-08 06:16:24,050][00611] Updated weights for policy 0, policy_version 59902 (0.0008) [2023-10-08 06:16:24,252][00612] Updated weights for policy 1, policy_version 60250 (0.0009) [2023-10-08 06:16:27,790][00611] Updated weights for policy 0, policy_version 59912 (0.0007) [2023-10-08 06:16:28,003][00612] Updated weights for policy 1, policy_version 60260 (0.0008) [2023-10-08 06:16:28,158][00611] Updated weights for policy 0, policy_version 59922 (0.0008) [2023-10-08 06:16:28,370][00612] Updated weights for policy 1, policy_version 60270 (0.0008) [2023-10-08 06:16:28,535][00611] Updated weights for policy 0, policy_version 59932 (0.0007) [2023-10-08 06:16:28,729][00612] Updated weights for policy 1, policy_version 60280 (0.0008) [2023-10-08 06:16:28,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123076608. Throughput: 0: 1839.5, 1: 1845.2. Samples: 30777242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:16:28,754][130385] Avg episode reward: [(0, '67.200'), (1, '68.450')] [2023-10-08 06:16:32,029][00611] Updated weights for policy 0, policy_version 59942 (0.0008) [2023-10-08 06:16:32,330][00612] Updated weights for policy 1, policy_version 60290 (0.0008) [2023-10-08 06:16:32,399][00611] Updated weights for policy 0, policy_version 59952 (0.0008) [2023-10-08 06:16:32,701][00612] Updated weights for policy 1, policy_version 60300 (0.0009) [2023-10-08 06:16:32,767][00611] Updated weights for policy 0, policy_version 59962 (0.0007) [2023-10-08 06:16:33,067][00612] Updated weights for policy 1, policy_version 60310 (0.0007) [2023-10-08 06:16:33,433][00612] Updated weights for policy 1, policy_version 60320 (0.0008) [2023-10-08 06:16:33,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 123174912. Throughput: 0: 1829.1, 1: 1828.4. Samples: 30797900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:16:33,755][130385] Avg episode reward: [(0, '69.580'), (1, '69.170')] [2023-10-08 06:16:36,424][00611] Updated weights for policy 0, policy_version 59972 (0.0010) [2023-10-08 06:16:36,799][00611] Updated weights for policy 0, policy_version 59982 (0.0010) [2023-10-08 06:16:37,138][00612] Updated weights for policy 1, policy_version 60330 (0.0007) [2023-10-08 06:16:37,176][00611] Updated weights for policy 0, policy_version 59992 (0.0007) [2023-10-08 06:16:37,509][00612] Updated weights for policy 1, policy_version 60340 (0.0009) [2023-10-08 06:16:37,881][00612] Updated weights for policy 1, policy_version 60350 (0.0009) [2023-10-08 06:16:38,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 123240448. Throughput: 0: 1831.7, 1: 1839.6. Samples: 30810368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:16:38,755][130385] Avg episode reward: [(0, '71.510'), (1, '68.270')] [2023-10-08 06:16:40,793][00611] Updated weights for policy 0, policy_version 60002 (0.0007) [2023-10-08 06:16:41,165][00611] Updated weights for policy 0, policy_version 60012 (0.0008) [2023-10-08 06:16:41,468][00612] Updated weights for policy 1, policy_version 60360 (0.0010) [2023-10-08 06:16:41,535][00611] Updated weights for policy 0, policy_version 60022 (0.0008) [2023-10-08 06:16:41,837][00612] Updated weights for policy 1, policy_version 60370 (0.0007) [2023-10-08 06:16:41,908][00611] Updated weights for policy 0, policy_version 60032 (0.0008) [2023-10-08 06:16:42,201][00612] Updated weights for policy 1, policy_version 60380 (0.0009) [2023-10-08 06:16:43,754][130385] Fps is (10 sec: 13107.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 123305984. Throughput: 0: 1825.4, 1: 1822.8. Samples: 30830708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:16:43,754][130385] Avg episode reward: [(0, '72.630'), (1, '68.760')] [2023-10-08 06:16:45,533][00611] Updated weights for policy 0, policy_version 60042 (0.0007) [2023-10-08 06:16:45,871][00612] Updated weights for policy 1, policy_version 60390 (0.0009) [2023-10-08 06:16:45,908][00611] Updated weights for policy 0, policy_version 60052 (0.0009) [2023-10-08 06:16:46,250][00612] Updated weights for policy 1, policy_version 60400 (0.0009) [2023-10-08 06:16:46,276][00611] Updated weights for policy 0, policy_version 60062 (0.0009) [2023-10-08 06:16:46,613][00612] Updated weights for policy 1, policy_version 60410 (0.0009) [2023-10-08 06:16:48,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 123371520. Throughput: 0: 1834.4, 1: 1836.8. Samples: 30853564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:16:48,754][130385] Avg episode reward: [(0, '72.010'), (1, '69.240')] [2023-10-08 06:16:49,957][00611] Updated weights for policy 0, policy_version 60072 (0.0008) [2023-10-08 06:16:50,189][00612] Updated weights for policy 1, policy_version 60420 (0.0008) [2023-10-08 06:16:50,324][00611] Updated weights for policy 0, policy_version 60082 (0.0009) [2023-10-08 06:16:50,553][00612] Updated weights for policy 1, policy_version 60430 (0.0007) [2023-10-08 06:16:50,694][00611] Updated weights for policy 0, policy_version 60092 (0.0007) [2023-10-08 06:16:50,920][00612] Updated weights for policy 1, policy_version 60440 (0.0009) [2023-10-08 06:16:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 123437056. Throughput: 0: 1830.6, 1: 1828.8. Samples: 30863876. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 06:16:53,754][130385] Avg episode reward: [(0, '78.070'), (1, '71.810')] [2023-10-08 06:16:53,755][00365] Saving new best policy, reward=78.070! [2023-10-08 06:16:54,277][00611] Updated weights for policy 0, policy_version 60102 (0.0010) [2023-10-08 06:16:54,508][00612] Updated weights for policy 1, policy_version 60450 (0.0010) [2023-10-08 06:16:54,650][00611] Updated weights for policy 0, policy_version 60112 (0.0009) [2023-10-08 06:16:54,877][00612] Updated weights for policy 1, policy_version 60460 (0.0008) [2023-10-08 06:16:55,012][00611] Updated weights for policy 0, policy_version 60122 (0.0007) [2023-10-08 06:16:55,242][00612] Updated weights for policy 1, policy_version 60470 (0.0007) [2023-10-08 06:16:55,609][00612] Updated weights for policy 1, policy_version 60480 (0.0010) [2023-10-08 06:16:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 123502592. Throughput: 0: 1833.1, 1: 1850.0. Samples: 30887050. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 06:16:58,754][130385] Avg episode reward: [(0, '78.750'), (1, '70.050')] [2023-10-08 06:16:58,776][00611] Updated weights for policy 0, policy_version 60132 (0.0009) [2023-10-08 06:16:59,149][00611] Updated weights for policy 0, policy_version 60142 (0.0008) [2023-10-08 06:16:59,429][00612] Updated weights for policy 1, policy_version 60490 (0.0007) [2023-10-08 06:16:59,524][00611] Updated weights for policy 0, policy_version 60152 (0.0009) [2023-10-08 06:16:59,796][00612] Updated weights for policy 1, policy_version 60500 (0.0007) [2023-10-08 06:16:59,813][00365] Saving new best policy, reward=78.750! [2023-10-08 06:17:00,158][00612] Updated weights for policy 1, policy_version 60510 (0.0009) [2023-10-08 06:17:03,088][00611] Updated weights for policy 0, policy_version 60162 (0.0007) [2023-10-08 06:17:03,462][00611] Updated weights for policy 0, policy_version 60172 (0.0008) [2023-10-08 06:17:03,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 123568128. Throughput: 0: 1821.4, 1: 1847.4. Samples: 30909738. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 06:17:03,755][130385] Avg episode reward: [(0, '77.970'), (1, '68.400')] [2023-10-08 06:17:03,784][00612] Updated weights for policy 1, policy_version 60520 (0.0008) [2023-10-08 06:17:03,831][00611] Updated weights for policy 0, policy_version 60182 (0.0007) [2023-10-08 06:17:04,153][00612] Updated weights for policy 1, policy_version 60530 (0.0010) [2023-10-08 06:17:04,210][00611] Updated weights for policy 0, policy_version 60192 (0.0011) [2023-10-08 06:17:04,523][00612] Updated weights for policy 1, policy_version 60540 (0.0007) [2023-10-08 06:17:08,015][00611] Updated weights for policy 0, policy_version 60202 (0.0009) [2023-10-08 06:17:08,136][00612] Updated weights for policy 1, policy_version 60550 (0.0010) [2023-10-08 06:17:08,387][00611] Updated weights for policy 0, policy_version 60212 (0.0009) [2023-10-08 06:17:08,500][00612] Updated weights for policy 1, policy_version 60560 (0.0009) [2023-10-08 06:17:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 123633664. Throughput: 0: 1827.9, 1: 1846.5. Samples: 30919806. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 06:17:08,754][130385] Avg episode reward: [(0, '79.810'), (1, '73.830')] [2023-10-08 06:17:08,760][00611] Updated weights for policy 0, policy_version 60222 (0.0007) [2023-10-08 06:17:08,832][00365] Saving new best policy, reward=79.810! [2023-10-08 06:17:08,875][00612] Updated weights for policy 1, policy_version 60570 (0.0010) [2023-10-08 06:17:12,244][00611] Updated weights for policy 0, policy_version 60232 (0.0008) [2023-10-08 06:17:12,458][00612] Updated weights for policy 1, policy_version 60580 (0.0009) [2023-10-08 06:17:12,623][00611] Updated weights for policy 0, policy_version 60242 (0.0008) [2023-10-08 06:17:12,829][00612] Updated weights for policy 1, policy_version 60590 (0.0007) [2023-10-08 06:17:12,998][00611] Updated weights for policy 0, policy_version 60252 (0.0008) [2023-10-08 06:17:13,195][00612] Updated weights for policy 1, policy_version 60600 (0.0008) [2023-10-08 06:17:13,754][130385] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 123764736. Throughput: 0: 1829.4, 1: 1847.0. Samples: 30942678. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 06:17:13,754][130385] Avg episode reward: [(0, '80.520'), (1, '77.760')] [2023-10-08 06:17:13,755][00365] Saving new best policy, reward=80.520! [2023-10-08 06:17:16,686][00611] Updated weights for policy 0, policy_version 60262 (0.0007) [2023-10-08 06:17:16,755][00612] Updated weights for policy 1, policy_version 60610 (0.0008) [2023-10-08 06:17:17,049][00611] Updated weights for policy 0, policy_version 60272 (0.0007) [2023-10-08 06:17:17,112][00612] Updated weights for policy 1, policy_version 60620 (0.0007) [2023-10-08 06:17:17,423][00611] Updated weights for policy 0, policy_version 60282 (0.0008) [2023-10-08 06:17:17,475][00612] Updated weights for policy 1, policy_version 60630 (0.0008) [2023-10-08 06:17:17,841][00612] Updated weights for policy 1, policy_version 60640 (0.0008) [2023-10-08 06:17:18,754][130385] Fps is (10 sec: 19660.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 123830272. Throughput: 0: 1834.6, 1: 1837.9. Samples: 30963162. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 06:17:18,755][130385] Avg episode reward: [(0, '79.830'), (1, '78.180')] [2023-10-08 06:17:21,153][00611] Updated weights for policy 0, policy_version 60292 (0.0009) [2023-10-08 06:17:21,428][00612] Updated weights for policy 1, policy_version 60650 (0.0008) [2023-10-08 06:17:21,519][00611] Updated weights for policy 0, policy_version 60302 (0.0008) [2023-10-08 06:17:21,793][00612] Updated weights for policy 1, policy_version 60660 (0.0007) [2023-10-08 06:17:21,885][00611] Updated weights for policy 0, policy_version 60312 (0.0008) [2023-10-08 06:17:22,149][00612] Updated weights for policy 1, policy_version 60670 (0.0007) [2023-10-08 06:17:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 123895808. Throughput: 0: 1828.7, 1: 1849.5. Samples: 30975884. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 06:17:23,754][130385] Avg episode reward: [(0, '81.990'), (1, '79.430')] [2023-10-08 06:17:23,755][00365] Saving new best policy, reward=81.990! [2023-10-08 06:17:25,610][00611] Updated weights for policy 0, policy_version 60322 (0.0009) [2023-10-08 06:17:25,753][00612] Updated weights for policy 1, policy_version 60680 (0.0009) [2023-10-08 06:17:25,981][00611] Updated weights for policy 0, policy_version 60332 (0.0008) [2023-10-08 06:17:26,113][00612] Updated weights for policy 1, policy_version 60690 (0.0007) [2023-10-08 06:17:26,353][00611] Updated weights for policy 0, policy_version 60342 (0.0008) [2023-10-08 06:17:26,480][00612] Updated weights for policy 1, policy_version 60700 (0.0010) [2023-10-08 06:17:26,729][00611] Updated weights for policy 0, policy_version 60352 (0.0008) [2023-10-08 06:17:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123961344. Throughput: 0: 1827.9, 1: 1847.3. Samples: 30996094. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 06:17:28,755][130385] Avg episode reward: [(0, '78.400'), (1, '77.540')] [2023-10-08 06:17:30,182][00612] Updated weights for policy 1, policy_version 60710 (0.0008) [2023-10-08 06:17:30,396][00611] Updated weights for policy 0, policy_version 60362 (0.0008) [2023-10-08 06:17:30,550][00612] Updated weights for policy 1, policy_version 60720 (0.0008) [2023-10-08 06:17:30,760][00611] Updated weights for policy 0, policy_version 60372 (0.0008) [2023-10-08 06:17:30,918][00612] Updated weights for policy 1, policy_version 60730 (0.0008) [2023-10-08 06:17:31,133][00611] Updated weights for policy 0, policy_version 60382 (0.0009) [2023-10-08 06:17:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 124026880. Throughput: 0: 1828.6, 1: 1854.2. Samples: 31019292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:17:33,755][130385] Avg episode reward: [(0, '77.560'), (1, '77.160')] [2023-10-08 06:17:34,523][00612] Updated weights for policy 1, policy_version 60740 (0.0008) [2023-10-08 06:17:34,890][00612] Updated weights for policy 1, policy_version 60750 (0.0007) [2023-10-08 06:17:34,926][00611] Updated weights for policy 0, policy_version 60392 (0.0008) [2023-10-08 06:17:35,270][00612] Updated weights for policy 1, policy_version 60760 (0.0007) [2023-10-08 06:17:35,300][00611] Updated weights for policy 0, policy_version 60402 (0.0007) [2023-10-08 06:17:35,663][00611] Updated weights for policy 0, policy_version 60412 (0.0008) [2023-10-08 06:17:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 124092416. Throughput: 0: 1824.6, 1: 1848.0. Samples: 31029142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:17:38,755][130385] Avg episode reward: [(0, '76.250'), (1, '76.750')] [2023-10-08 06:17:38,992][00612] Updated weights for policy 1, policy_version 60770 (0.0009) [2023-10-08 06:17:39,294][00611] Updated weights for policy 0, policy_version 60422 (0.0008) [2023-10-08 06:17:39,352][00612] Updated weights for policy 1, policy_version 60780 (0.0007) [2023-10-08 06:17:39,665][00611] Updated weights for policy 0, policy_version 60432 (0.0007) [2023-10-08 06:17:39,716][00612] Updated weights for policy 1, policy_version 60790 (0.0008) [2023-10-08 06:17:40,041][00611] Updated weights for policy 0, policy_version 60442 (0.0007) [2023-10-08 06:17:40,076][00612] Updated weights for policy 1, policy_version 60800 (0.0007) [2023-10-08 06:17:43,697][00611] Updated weights for policy 0, policy_version 60452 (0.0008) [2023-10-08 06:17:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 124157952. Throughput: 0: 1824.5, 1: 1841.7. Samples: 31052030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:17:43,754][130385] Avg episode reward: [(0, '72.380'), (1, '79.160')] [2023-10-08 06:17:44,065][00611] Updated weights for policy 0, policy_version 60462 (0.0007) [2023-10-08 06:17:44,171][00612] Updated weights for policy 1, policy_version 60810 (0.0008) [2023-10-08 06:17:44,444][00611] Updated weights for policy 0, policy_version 60472 (0.0007) [2023-10-08 06:17:44,539][00612] Updated weights for policy 1, policy_version 60820 (0.0007) [2023-10-08 06:17:44,903][00612] Updated weights for policy 1, policy_version 60830 (0.0008) [2023-10-08 06:17:48,246][00611] Updated weights for policy 0, policy_version 60482 (0.0007) [2023-10-08 06:17:48,302][00612] Updated weights for policy 1, policy_version 60840 (0.0010) [2023-10-08 06:17:48,618][00611] Updated weights for policy 0, policy_version 60492 (0.0010) [2023-10-08 06:17:48,685][00612] Updated weights for policy 1, policy_version 60850 (0.0010) [2023-10-08 06:17:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 124223488. Throughput: 0: 1829.2, 1: 1836.2. Samples: 31074680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:17:48,755][130385] Avg episode reward: [(0, '69.740'), (1, '79.340')] [2023-10-08 06:17:48,981][00611] Updated weights for policy 0, policy_version 60502 (0.0009) [2023-10-08 06:17:49,048][00612] Updated weights for policy 1, policy_version 60860 (0.0008) [2023-10-08 06:17:49,346][00611] Updated weights for policy 0, policy_version 60512 (0.0009) [2023-10-08 06:17:52,725][00612] Updated weights for policy 1, policy_version 60870 (0.0009) [2023-10-08 06:17:53,087][00612] Updated weights for policy 1, policy_version 60880 (0.0007) [2023-10-08 06:17:53,103][00611] Updated weights for policy 0, policy_version 60522 (0.0007) [2023-10-08 06:17:53,461][00612] Updated weights for policy 1, policy_version 60890 (0.0008) [2023-10-08 06:17:53,487][00611] Updated weights for policy 0, policy_version 60532 (0.0007) [2023-10-08 06:17:53,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 124321792. Throughput: 0: 1823.2, 1: 1841.2. Samples: 31084700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:17:53,754][130385] Avg episode reward: [(0, '66.220'), (1, '77.380')] [2023-10-08 06:17:53,847][00611] Updated weights for policy 0, policy_version 60542 (0.0009) [2023-10-08 06:17:57,164][00612] Updated weights for policy 1, policy_version 60900 (0.0008) [2023-10-08 06:17:57,527][00612] Updated weights for policy 1, policy_version 60910 (0.0009) [2023-10-08 06:17:57,539][00611] Updated weights for policy 0, policy_version 60552 (0.0007) [2023-10-08 06:17:57,898][00612] Updated weights for policy 1, policy_version 60920 (0.0009) [2023-10-08 06:17:57,909][00611] Updated weights for policy 0, policy_version 60562 (0.0008) [2023-10-08 06:17:58,277][00611] Updated weights for policy 0, policy_version 60572 (0.0008) [2023-10-08 06:17:58,754][130385] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 124420096. Throughput: 0: 1824.9, 1: 1828.6. Samples: 31107086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:17:58,755][130385] Avg episode reward: [(0, '66.110'), (1, '75.680')] [2023-10-08 06:18:01,531][00612] Updated weights for policy 1, policy_version 60930 (0.0009) [2023-10-08 06:18:01,900][00612] Updated weights for policy 1, policy_version 60940 (0.0009) [2023-10-08 06:18:01,965][00611] Updated weights for policy 0, policy_version 60582 (0.0007) [2023-10-08 06:18:02,263][00612] Updated weights for policy 1, policy_version 60950 (0.0008) [2023-10-08 06:18:02,336][00611] Updated weights for policy 0, policy_version 60592 (0.0008) [2023-10-08 06:18:02,632][00612] Updated weights for policy 1, policy_version 60960 (0.0007) [2023-10-08 06:18:02,708][00611] Updated weights for policy 0, policy_version 60602 (0.0008) [2023-10-08 06:18:03,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 124485632. Throughput: 0: 1812.0, 1: 1828.5. Samples: 31126986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:18:03,754][130385] Avg episode reward: [(0, '66.570'), (1, '78.070')] [2023-10-08 06:18:06,361][00612] Updated weights for policy 1, policy_version 60970 (0.0008) [2023-10-08 06:18:06,402][00611] Updated weights for policy 0, policy_version 60612 (0.0008) [2023-10-08 06:18:06,717][00612] Updated weights for policy 1, policy_version 60980 (0.0008) [2023-10-08 06:18:06,778][00611] Updated weights for policy 0, policy_version 60622 (0.0009) [2023-10-08 06:18:07,094][00612] Updated weights for policy 1, policy_version 60990 (0.0008) [2023-10-08 06:18:07,140][00611] Updated weights for policy 0, policy_version 60632 (0.0009) [2023-10-08 06:18:08,754][130385] Fps is (10 sec: 13107.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 124551168. Throughput: 0: 1819.2, 1: 1820.0. Samples: 31139646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:18:08,754][130385] Avg episode reward: [(0, '62.500'), (1, '71.920')] [2023-10-08 06:18:10,877][00612] Updated weights for policy 1, policy_version 61000 (0.0009) [2023-10-08 06:18:11,086][00611] Updated weights for policy 0, policy_version 60642 (0.0009) [2023-10-08 06:18:11,247][00612] Updated weights for policy 1, policy_version 61010 (0.0009) [2023-10-08 06:18:11,460][00611] Updated weights for policy 0, policy_version 60652 (0.0007) [2023-10-08 06:18:11,615][00612] Updated weights for policy 1, policy_version 61020 (0.0007) [2023-10-08 06:18:11,829][00611] Updated weights for policy 0, policy_version 60662 (0.0008) [2023-10-08 06:18:12,210][00611] Updated weights for policy 0, policy_version 60672 (0.0010) [2023-10-08 06:18:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 124616704. Throughput: 0: 1813.7, 1: 1816.8. Samples: 31159468. Policy #0 lag: (min: 29.0, avg: 36.3, max: 61.0) [2023-10-08 06:18:13,754][130385] Avg episode reward: [(0, '62.810'), (1, '72.350')] [2023-10-08 06:18:15,477][00612] Updated weights for policy 1, policy_version 61030 (0.0008) [2023-10-08 06:18:15,839][00612] Updated weights for policy 1, policy_version 61040 (0.0008) [2023-10-08 06:18:15,845][00611] Updated weights for policy 0, policy_version 60682 (0.0008) [2023-10-08 06:18:16,211][00611] Updated weights for policy 0, policy_version 60692 (0.0008) [2023-10-08 06:18:16,215][00612] Updated weights for policy 1, policy_version 61050 (0.0008) [2023-10-08 06:18:16,589][00611] Updated weights for policy 0, policy_version 60702 (0.0007) [2023-10-08 06:18:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 124682240. Throughput: 0: 1812.8, 1: 1810.3. Samples: 31182334. Policy #0 lag: (min: 29.0, avg: 36.3, max: 61.0) [2023-10-08 06:18:18,755][130385] Avg episode reward: [(0, '58.810'), (1, '72.670')] [2023-10-08 06:18:18,767][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000061056_62521344.pth... [2023-10-08 06:18:18,767][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000060704_62160896.pth... [2023-10-08 06:18:18,800][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000059360_60784640.pth [2023-10-08 06:18:18,804][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000059008_60424192.pth [2023-10-08 06:18:19,972][00612] Updated weights for policy 1, policy_version 61060 (0.0009) [2023-10-08 06:18:20,167][00611] Updated weights for policy 0, policy_version 60712 (0.0008) [2023-10-08 06:18:20,345][00612] Updated weights for policy 1, policy_version 61070 (0.0007) [2023-10-08 06:18:20,534][00611] Updated weights for policy 0, policy_version 60722 (0.0007) [2023-10-08 06:18:20,711][00612] Updated weights for policy 1, policy_version 61080 (0.0007) [2023-10-08 06:18:20,899][00611] Updated weights for policy 0, policy_version 60732 (0.0008) [2023-10-08 06:18:23,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 124747776. Throughput: 0: 1817.6, 1: 1809.3. Samples: 31192352. Policy #0 lag: (min: 29.0, avg: 36.3, max: 61.0) [2023-10-08 06:18:23,755][130385] Avg episode reward: [(0, '56.450'), (1, '73.550')] [2023-10-08 06:18:24,362][00612] Updated weights for policy 1, policy_version 61090 (0.0008) [2023-10-08 06:18:24,654][00611] Updated weights for policy 0, policy_version 60742 (0.0009) [2023-10-08 06:18:24,727][00612] Updated weights for policy 1, policy_version 61100 (0.0008) [2023-10-08 06:18:25,035][00611] Updated weights for policy 0, policy_version 60752 (0.0009) [2023-10-08 06:18:25,086][00612] Updated weights for policy 1, policy_version 61110 (0.0008) [2023-10-08 06:18:25,397][00611] Updated weights for policy 0, policy_version 60762 (0.0008) [2023-10-08 06:18:25,455][00612] Updated weights for policy 1, policy_version 61120 (0.0007) [2023-10-08 06:18:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 124813312. Throughput: 0: 1813.8, 1: 1809.7. Samples: 31215086. Policy #0 lag: (min: 29.0, avg: 36.3, max: 61.0) [2023-10-08 06:18:28,755][130385] Avg episode reward: [(0, '59.770'), (1, '73.070')] [2023-10-08 06:18:28,884][00611] Updated weights for policy 0, policy_version 60772 (0.0008) [2023-10-08 06:18:29,230][00612] Updated weights for policy 1, policy_version 61130 (0.0009) [2023-10-08 06:18:29,262][00611] Updated weights for policy 0, policy_version 60782 (0.0009) [2023-10-08 06:18:29,603][00612] Updated weights for policy 1, policy_version 61140 (0.0009) [2023-10-08 06:18:29,628][00611] Updated weights for policy 0, policy_version 60792 (0.0009) [2023-10-08 06:18:29,964][00612] Updated weights for policy 1, policy_version 61150 (0.0008) [2023-10-08 06:18:33,189][00611] Updated weights for policy 0, policy_version 60802 (0.0009) [2023-10-08 06:18:33,542][00612] Updated weights for policy 1, policy_version 61160 (0.0007) [2023-10-08 06:18:33,565][00611] Updated weights for policy 0, policy_version 60812 (0.0007) [2023-10-08 06:18:33,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124878848. Throughput: 0: 1820.3, 1: 1815.0. Samples: 31238268. Policy #0 lag: (min: 29.0, avg: 36.3, max: 61.0) [2023-10-08 06:18:33,754][130385] Avg episode reward: [(0, '59.340'), (1, '72.200')] [2023-10-08 06:18:33,906][00612] Updated weights for policy 1, policy_version 61170 (0.0009) [2023-10-08 06:18:33,931][00611] Updated weights for policy 0, policy_version 60822 (0.0008) [2023-10-08 06:18:34,274][00612] Updated weights for policy 1, policy_version 61180 (0.0008) [2023-10-08 06:18:34,296][00611] Updated weights for policy 0, policy_version 60832 (0.0009) [2023-10-08 06:18:37,788][00612] Updated weights for policy 1, policy_version 61190 (0.0009) [2023-10-08 06:18:38,018][00611] Updated weights for policy 0, policy_version 60842 (0.0009) [2023-10-08 06:18:38,168][00612] Updated weights for policy 1, policy_version 61200 (0.0008) [2023-10-08 06:18:38,387][00611] Updated weights for policy 0, policy_version 60852 (0.0009) [2023-10-08 06:18:38,533][00612] Updated weights for policy 1, policy_version 61210 (0.0009) [2023-10-08 06:18:38,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 124977152. Throughput: 0: 1824.0, 1: 1812.0. Samples: 31248316. Policy #0 lag: (min: 29.0, avg: 36.3, max: 61.0) [2023-10-08 06:18:38,754][130385] Avg episode reward: [(0, '59.320'), (1, '70.650')] [2023-10-08 06:18:38,763][00611] Updated weights for policy 0, policy_version 60862 (0.0009) [2023-10-08 06:18:42,296][00612] Updated weights for policy 1, policy_version 61220 (0.0009) [2023-10-08 06:18:42,493][00611] Updated weights for policy 0, policy_version 60872 (0.0009) [2023-10-08 06:18:42,663][00612] Updated weights for policy 1, policy_version 61230 (0.0009) [2023-10-08 06:18:42,859][00611] Updated weights for policy 0, policy_version 60882 (0.0007) [2023-10-08 06:18:43,024][00612] Updated weights for policy 1, policy_version 61240 (0.0008) [2023-10-08 06:18:43,232][00611] Updated weights for policy 0, policy_version 60892 (0.0009) [2023-10-08 06:18:43,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 125075456. Throughput: 0: 1825.8, 1: 1821.2. Samples: 31271202. Policy #0 lag: (min: 29.0, avg: 36.3, max: 61.0) [2023-10-08 06:18:43,754][130385] Avg episode reward: [(0, '59.900'), (1, '69.680')] [2023-10-08 06:18:46,644][00612] Updated weights for policy 1, policy_version 61250 (0.0008) [2023-10-08 06:18:46,858][00611] Updated weights for policy 0, policy_version 60902 (0.0007) [2023-10-08 06:18:47,008][00612] Updated weights for policy 1, policy_version 61260 (0.0007) [2023-10-08 06:18:47,235][00611] Updated weights for policy 0, policy_version 60912 (0.0008) [2023-10-08 06:18:47,373][00612] Updated weights for policy 1, policy_version 61270 (0.0008) [2023-10-08 06:18:47,599][00611] Updated weights for policy 0, policy_version 60922 (0.0009) [2023-10-08 06:18:47,738][00612] Updated weights for policy 1, policy_version 61280 (0.0008) [2023-10-08 06:18:48,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 125140992. Throughput: 0: 1826.0, 1: 1819.5. Samples: 31291034. Policy #0 lag: (min: 29.0, avg: 36.3, max: 61.0) [2023-10-08 06:18:48,754][130385] Avg episode reward: [(0, '58.290'), (1, '68.720')] [2023-10-08 06:18:51,335][00611] Updated weights for policy 0, policy_version 60932 (0.0008) [2023-10-08 06:18:51,572][00612] Updated weights for policy 1, policy_version 61290 (0.0007) [2023-10-08 06:18:51,717][00611] Updated weights for policy 0, policy_version 60942 (0.0008) [2023-10-08 06:18:51,937][00612] Updated weights for policy 1, policy_version 61300 (0.0008) [2023-10-08 06:18:52,080][00611] Updated weights for policy 0, policy_version 60952 (0.0008) [2023-10-08 06:18:52,305][00612] Updated weights for policy 1, policy_version 61310 (0.0009) [2023-10-08 06:18:53,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 125206528. Throughput: 0: 1825.9, 1: 1824.0. Samples: 31303894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:18:53,755][130385] Avg episode reward: [(0, '55.610'), (1, '67.230')] [2023-10-08 06:18:55,795][00611] Updated weights for policy 0, policy_version 60962 (0.0010) [2023-10-08 06:18:56,085][00612] Updated weights for policy 1, policy_version 61320 (0.0007) [2023-10-08 06:18:56,171][00611] Updated weights for policy 0, policy_version 60972 (0.0008) [2023-10-08 06:18:56,455][00612] Updated weights for policy 1, policy_version 61330 (0.0008) [2023-10-08 06:18:56,536][00611] Updated weights for policy 0, policy_version 60982 (0.0009) [2023-10-08 06:18:56,827][00612] Updated weights for policy 1, policy_version 61340 (0.0008) [2023-10-08 06:18:56,904][00611] Updated weights for policy 0, policy_version 60992 (0.0007) [2023-10-08 06:18:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 125272064. Throughput: 0: 1823.5, 1: 1821.1. Samples: 31323472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:18:58,754][130385] Avg episode reward: [(0, '58.800'), (1, '72.560')] [2023-10-08 06:19:00,260][00612] Updated weights for policy 1, policy_version 61350 (0.0008) [2023-10-08 06:19:00,521][00611] Updated weights for policy 0, policy_version 61002 (0.0008) [2023-10-08 06:19:00,627][00612] Updated weights for policy 1, policy_version 61360 (0.0008) [2023-10-08 06:19:00,902][00611] Updated weights for policy 0, policy_version 61012 (0.0008) [2023-10-08 06:19:01,001][00612] Updated weights for policy 1, policy_version 61370 (0.0007) [2023-10-08 06:19:01,268][00611] Updated weights for policy 0, policy_version 61022 (0.0009) [2023-10-08 06:19:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 125337600. Throughput: 0: 1828.0, 1: 1824.8. Samples: 31346712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:19:03,755][130385] Avg episode reward: [(0, '60.420'), (1, '66.610')] [2023-10-08 06:19:04,804][00612] Updated weights for policy 1, policy_version 61380 (0.0008) [2023-10-08 06:19:04,948][00611] Updated weights for policy 0, policy_version 61032 (0.0009) [2023-10-08 06:19:05,172][00612] Updated weights for policy 1, policy_version 61390 (0.0009) [2023-10-08 06:19:05,312][00611] Updated weights for policy 0, policy_version 61042 (0.0007) [2023-10-08 06:19:05,542][00612] Updated weights for policy 1, policy_version 61400 (0.0009) [2023-10-08 06:19:05,684][00611] Updated weights for policy 0, policy_version 61052 (0.0009) [2023-10-08 06:19:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 125403136. Throughput: 0: 1825.2, 1: 1823.2. Samples: 31356530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:19:08,755][130385] Avg episode reward: [(0, '58.490'), (1, '70.810')] [2023-10-08 06:19:09,315][00611] Updated weights for policy 0, policy_version 61062 (0.0009) [2023-10-08 06:19:09,336][00612] Updated weights for policy 1, policy_version 61410 (0.0009) [2023-10-08 06:19:09,692][00611] Updated weights for policy 0, policy_version 61072 (0.0009) [2023-10-08 06:19:09,711][00612] Updated weights for policy 1, policy_version 61420 (0.0008) [2023-10-08 06:19:10,065][00611] Updated weights for policy 0, policy_version 61082 (0.0008) [2023-10-08 06:19:10,077][00612] Updated weights for policy 1, policy_version 61430 (0.0008) [2023-10-08 06:19:10,444][00612] Updated weights for policy 1, policy_version 61440 (0.0008) [2023-10-08 06:19:13,741][00611] Updated weights for policy 0, policy_version 61092 (0.0008) [2023-10-08 06:19:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 125468672. Throughput: 0: 1824.3, 1: 1828.3. Samples: 31379452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:19:13,755][130385] Avg episode reward: [(0, '58.500'), (1, '73.030')] [2023-10-08 06:19:14,008][00612] Updated weights for policy 1, policy_version 61450 (0.0008) [2023-10-08 06:19:14,118][00611] Updated weights for policy 0, policy_version 61102 (0.0008) [2023-10-08 06:19:14,377][00612] Updated weights for policy 1, policy_version 61460 (0.0008) [2023-10-08 06:19:14,481][00611] Updated weights for policy 0, policy_version 61112 (0.0008) [2023-10-08 06:19:14,753][00612] Updated weights for policy 1, policy_version 61470 (0.0009) [2023-10-08 06:19:18,063][00611] Updated weights for policy 0, policy_version 61122 (0.0008) [2023-10-08 06:19:18,314][00612] Updated weights for policy 1, policy_version 61480 (0.0008) [2023-10-08 06:19:18,435][00611] Updated weights for policy 0, policy_version 61132 (0.0008) [2023-10-08 06:19:18,683][00612] Updated weights for policy 1, policy_version 61490 (0.0007) [2023-10-08 06:19:18,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 125534208. Throughput: 0: 1817.0, 1: 1830.0. Samples: 31402382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:19:18,754][130385] Avg episode reward: [(0, '55.650'), (1, '70.210')] [2023-10-08 06:19:18,810][00611] Updated weights for policy 0, policy_version 61142 (0.0009) [2023-10-08 06:19:19,044][00612] Updated weights for policy 1, policy_version 61500 (0.0010) [2023-10-08 06:19:19,191][00611] Updated weights for policy 0, policy_version 61152 (0.0009) [2023-10-08 06:19:22,686][00611] Updated weights for policy 0, policy_version 61162 (0.0009) [2023-10-08 06:19:22,926][00612] Updated weights for policy 1, policy_version 61510 (0.0007) [2023-10-08 06:19:23,065][00611] Updated weights for policy 0, policy_version 61172 (0.0010) [2023-10-08 06:19:23,288][00612] Updated weights for policy 1, policy_version 61520 (0.0007) [2023-10-08 06:19:23,428][00611] Updated weights for policy 0, policy_version 61182 (0.0008) [2023-10-08 06:19:23,664][00612] Updated weights for policy 1, policy_version 61530 (0.0008) [2023-10-08 06:19:23,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 125632512. Throughput: 0: 1822.9, 1: 1830.8. Samples: 31412734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:19:23,754][130385] Avg episode reward: [(0, '57.640'), (1, '71.350')] [2023-10-08 06:19:27,234][00611] Updated weights for policy 0, policy_version 61192 (0.0008) [2023-10-08 06:19:27,260][00612] Updated weights for policy 1, policy_version 61540 (0.0010) [2023-10-08 06:19:27,608][00611] Updated weights for policy 0, policy_version 61202 (0.0008) [2023-10-08 06:19:27,625][00612] Updated weights for policy 1, policy_version 61550 (0.0008) [2023-10-08 06:19:27,981][00611] Updated weights for policy 0, policy_version 61212 (0.0008) [2023-10-08 06:19:27,994][00612] Updated weights for policy 1, policy_version 61560 (0.0008) [2023-10-08 06:19:28,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 125730816. Throughput: 0: 1809.5, 1: 1828.1. Samples: 31434896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:19:28,755][130385] Avg episode reward: [(0, '58.610'), (1, '71.800')] [2023-10-08 06:19:31,724][00612] Updated weights for policy 1, policy_version 61570 (0.0008) [2023-10-08 06:19:31,759][00611] Updated weights for policy 0, policy_version 61222 (0.0009) [2023-10-08 06:19:32,090][00612] Updated weights for policy 1, policy_version 61580 (0.0009) [2023-10-08 06:19:32,122][00611] Updated weights for policy 0, policy_version 61232 (0.0008) [2023-10-08 06:19:32,459][00612] Updated weights for policy 1, policy_version 61590 (0.0008) [2023-10-08 06:19:32,499][00611] Updated weights for policy 0, policy_version 61242 (0.0010) [2023-10-08 06:19:32,829][00612] Updated weights for policy 1, policy_version 61600 (0.0007) [2023-10-08 06:19:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 125796352. Throughput: 0: 1821.7, 1: 1824.4. Samples: 31455106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:19:33,754][130385] Avg episode reward: [(0, '58.270'), (1, '70.670')] [2023-10-08 06:19:36,241][00611] Updated weights for policy 0, policy_version 61252 (0.0007) [2023-10-08 06:19:36,289][00612] Updated weights for policy 1, policy_version 61610 (0.0008) [2023-10-08 06:19:36,601][00611] Updated weights for policy 0, policy_version 61262 (0.0008) [2023-10-08 06:19:36,656][00612] Updated weights for policy 1, policy_version 61620 (0.0008) [2023-10-08 06:19:36,971][00611] Updated weights for policy 0, policy_version 61272 (0.0007) [2023-10-08 06:19:37,027][00612] Updated weights for policy 1, policy_version 61630 (0.0007) [2023-10-08 06:19:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 125861888. Throughput: 0: 1820.8, 1: 1824.9. Samples: 31467948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:19:38,755][130385] Avg episode reward: [(0, '60.540'), (1, '72.310')] [2023-10-08 06:19:40,670][00612] Updated weights for policy 1, policy_version 61640 (0.0008) [2023-10-08 06:19:40,769][00611] Updated weights for policy 0, policy_version 61282 (0.0009) [2023-10-08 06:19:41,029][00612] Updated weights for policy 1, policy_version 61650 (0.0007) [2023-10-08 06:19:41,136][00611] Updated weights for policy 0, policy_version 61292 (0.0010) [2023-10-08 06:19:41,395][00612] Updated weights for policy 1, policy_version 61660 (0.0009) [2023-10-08 06:19:41,508][00611] Updated weights for policy 0, policy_version 61302 (0.0010) [2023-10-08 06:19:41,867][00611] Updated weights for policy 0, policy_version 61312 (0.0009) [2023-10-08 06:19:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 125927424. Throughput: 0: 1825.8, 1: 1832.8. Samples: 31488108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:19:43,754][130385] Avg episode reward: [(0, '58.910'), (1, '76.640')] [2023-10-08 06:19:45,025][00612] Updated weights for policy 1, policy_version 61670 (0.0008) [2023-10-08 06:19:45,401][00612] Updated weights for policy 1, policy_version 61680 (0.0008) [2023-10-08 06:19:45,571][00611] Updated weights for policy 0, policy_version 61322 (0.0010) [2023-10-08 06:19:45,762][00612] Updated weights for policy 1, policy_version 61690 (0.0008) [2023-10-08 06:19:45,946][00611] Updated weights for policy 0, policy_version 61332 (0.0007) [2023-10-08 06:19:46,328][00611] Updated weights for policy 0, policy_version 61342 (0.0008) [2023-10-08 06:19:48,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 125992960. Throughput: 0: 1822.0, 1: 1838.0. Samples: 31511412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:19:48,754][130385] Avg episode reward: [(0, '63.370'), (1, '76.350')] [2023-10-08 06:19:49,333][00612] Updated weights for policy 1, policy_version 61700 (0.0007) [2023-10-08 06:19:49,704][00612] Updated weights for policy 1, policy_version 61710 (0.0008) [2023-10-08 06:19:50,016][00611] Updated weights for policy 0, policy_version 61352 (0.0008) [2023-10-08 06:19:50,077][00612] Updated weights for policy 1, policy_version 61720 (0.0009) [2023-10-08 06:19:50,379][00611] Updated weights for policy 0, policy_version 61362 (0.0008) [2023-10-08 06:19:50,751][00611] Updated weights for policy 0, policy_version 61372 (0.0007) [2023-10-08 06:19:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 126058496. Throughput: 0: 1820.9, 1: 1841.0. Samples: 31521318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:19:53,754][130385] Avg episode reward: [(0, '66.120'), (1, '74.750')] [2023-10-08 06:19:53,820][00612] Updated weights for policy 1, policy_version 61730 (0.0008) [2023-10-08 06:19:54,184][00612] Updated weights for policy 1, policy_version 61740 (0.0010) [2023-10-08 06:19:54,488][00611] Updated weights for policy 0, policy_version 61382 (0.0009) [2023-10-08 06:19:54,554][00612] Updated weights for policy 1, policy_version 61750 (0.0008) [2023-10-08 06:19:54,851][00611] Updated weights for policy 0, policy_version 61392 (0.0010) [2023-10-08 06:19:54,920][00612] Updated weights for policy 1, policy_version 61760 (0.0008) [2023-10-08 06:19:55,216][00611] Updated weights for policy 0, policy_version 61402 (0.0008) [2023-10-08 06:19:58,615][00612] Updated weights for policy 1, policy_version 61770 (0.0008) [2023-10-08 06:19:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 126124032. Throughput: 0: 1822.6, 1: 1839.0. Samples: 31544224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:19:58,754][130385] Avg episode reward: [(0, '67.040'), (1, '78.620')] [2023-10-08 06:19:58,795][00611] Updated weights for policy 0, policy_version 61412 (0.0010) [2023-10-08 06:19:58,970][00612] Updated weights for policy 1, policy_version 61780 (0.0008) [2023-10-08 06:19:59,170][00611] Updated weights for policy 0, policy_version 61422 (0.0008) [2023-10-08 06:19:59,346][00612] Updated weights for policy 1, policy_version 61790 (0.0009) [2023-10-08 06:19:59,537][00611] Updated weights for policy 0, policy_version 61432 (0.0009) [2023-10-08 06:20:02,882][00612] Updated weights for policy 1, policy_version 61800 (0.0009) [2023-10-08 06:20:03,247][00612] Updated weights for policy 1, policy_version 61810 (0.0010) [2023-10-08 06:20:03,326][00611] Updated weights for policy 0, policy_version 61442 (0.0007) [2023-10-08 06:20:03,617][00612] Updated weights for policy 1, policy_version 61820 (0.0010) [2023-10-08 06:20:03,693][00611] Updated weights for policy 0, policy_version 61452 (0.0009) [2023-10-08 06:20:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 126189568. Throughput: 0: 1822.3, 1: 1822.3. Samples: 31566388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:20:03,754][130385] Avg episode reward: [(0, '65.200'), (1, '77.600')] [2023-10-08 06:20:04,068][00611] Updated weights for policy 0, policy_version 61462 (0.0008) [2023-10-08 06:20:04,436][00611] Updated weights for policy 0, policy_version 61472 (0.0007) [2023-10-08 06:20:07,213][00612] Updated weights for policy 1, policy_version 61830 (0.0009) [2023-10-08 06:20:07,582][00612] Updated weights for policy 1, policy_version 61840 (0.0010) [2023-10-08 06:20:07,947][00612] Updated weights for policy 1, policy_version 61850 (0.0009) [2023-10-08 06:20:08,181][00611] Updated weights for policy 0, policy_version 61482 (0.0008) [2023-10-08 06:20:08,554][00611] Updated weights for policy 0, policy_version 61492 (0.0007) [2023-10-08 06:20:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126287872. Throughput: 0: 1813.0, 1: 1838.7. Samples: 31577058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:20:08,754][130385] Avg episode reward: [(0, '68.410'), (1, '77.040')] [2023-10-08 06:20:08,935][00611] Updated weights for policy 0, policy_version 61502 (0.0007) [2023-10-08 06:20:11,568][00612] Updated weights for policy 1, policy_version 61860 (0.0008) [2023-10-08 06:20:11,938][00612] Updated weights for policy 1, policy_version 61870 (0.0008) [2023-10-08 06:20:12,294][00612] Updated weights for policy 1, policy_version 61880 (0.0009) [2023-10-08 06:20:12,406][00611] Updated weights for policy 0, policy_version 61512 (0.0010) [2023-10-08 06:20:12,776][00611] Updated weights for policy 0, policy_version 61522 (0.0007) [2023-10-08 06:20:13,149][00611] Updated weights for policy 0, policy_version 61532 (0.0007) [2023-10-08 06:20:13,754][130385] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 126386176. Throughput: 0: 1828.8, 1: 1824.5. Samples: 31599294. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:20:13,754][130385] Avg episode reward: [(0, '68.930'), (1, '79.780')] [2023-10-08 06:20:15,845][00612] Updated weights for policy 1, policy_version 61890 (0.0008) [2023-10-08 06:20:16,216][00612] Updated weights for policy 1, policy_version 61900 (0.0011) [2023-10-08 06:20:16,584][00612] Updated weights for policy 1, policy_version 61910 (0.0010) [2023-10-08 06:20:16,847][00611] Updated weights for policy 0, policy_version 61542 (0.0007) [2023-10-08 06:20:16,957][00612] Updated weights for policy 1, policy_version 61920 (0.0007) [2023-10-08 06:20:17,220][00611] Updated weights for policy 0, policy_version 61552 (0.0011) [2023-10-08 06:20:17,586][00611] Updated weights for policy 0, policy_version 61562 (0.0009) [2023-10-08 06:20:18,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 126451712. Throughput: 0: 1820.8, 1: 1855.5. Samples: 31620544. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:20:18,755][130385] Avg episode reward: [(0, '73.400'), (1, '76.230')] [2023-10-08 06:20:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000061568_63045632.pth... [2023-10-08 06:20:18,764][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000061920_63406080.pth... [2023-10-08 06:20:18,803][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000059872_61308928.pth [2023-10-08 06:20:18,811][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000060192_61636608.pth [2023-10-08 06:20:20,523][00612] Updated weights for policy 1, policy_version 61930 (0.0009) [2023-10-08 06:20:20,892][00612] Updated weights for policy 1, policy_version 61940 (0.0009) [2023-10-08 06:20:21,160][00611] Updated weights for policy 0, policy_version 61572 (0.0011) [2023-10-08 06:20:21,261][00612] Updated weights for policy 1, policy_version 61950 (0.0010) [2023-10-08 06:20:21,531][00611] Updated weights for policy 0, policy_version 61582 (0.0008) [2023-10-08 06:20:21,895][00611] Updated weights for policy 0, policy_version 61592 (0.0010) [2023-10-08 06:20:23,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 126517248. Throughput: 0: 1821.9, 1: 1830.4. Samples: 31632300. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:20:23,755][130385] Avg episode reward: [(0, '73.480'), (1, '74.400')] [2023-10-08 06:20:24,797][00612] Updated weights for policy 1, policy_version 61960 (0.0008) [2023-10-08 06:20:25,159][00612] Updated weights for policy 1, policy_version 61970 (0.0010) [2023-10-08 06:20:25,527][00612] Updated weights for policy 1, policy_version 61980 (0.0010) [2023-10-08 06:20:25,720][00611] Updated weights for policy 0, policy_version 61602 (0.0008) [2023-10-08 06:20:26,101][00611] Updated weights for policy 0, policy_version 61612 (0.0007) [2023-10-08 06:20:26,479][00611] Updated weights for policy 0, policy_version 61622 (0.0008) [2023-10-08 06:20:26,845][00611] Updated weights for policy 0, policy_version 61632 (0.0008) [2023-10-08 06:20:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 126582784. Throughput: 0: 1819.8, 1: 1858.2. Samples: 31653618. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:20:28,755][130385] Avg episode reward: [(0, '75.030'), (1, '74.340')] [2023-10-08 06:20:29,310][00612] Updated weights for policy 1, policy_version 61990 (0.0009) [2023-10-08 06:20:29,680][00612] Updated weights for policy 1, policy_version 62000 (0.0009) [2023-10-08 06:20:30,039][00612] Updated weights for policy 1, policy_version 62010 (0.0008) [2023-10-08 06:20:30,624][00611] Updated weights for policy 0, policy_version 61642 (0.0007) [2023-10-08 06:20:30,999][00611] Updated weights for policy 0, policy_version 61652 (0.0009) [2023-10-08 06:20:31,364][00611] Updated weights for policy 0, policy_version 61662 (0.0007) [2023-10-08 06:20:33,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 126648320. Throughput: 0: 1813.5, 1: 1857.1. Samples: 31676588. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:20:33,754][130385] Avg episode reward: [(0, '71.240'), (1, '72.170')] [2023-10-08 06:20:33,803][00612] Updated weights for policy 1, policy_version 62020 (0.0009) [2023-10-08 06:20:34,179][00612] Updated weights for policy 1, policy_version 62030 (0.0008) [2023-10-08 06:20:34,544][00612] Updated weights for policy 1, policy_version 62040 (0.0007) [2023-10-08 06:20:35,062][00611] Updated weights for policy 0, policy_version 61672 (0.0009) [2023-10-08 06:20:35,435][00611] Updated weights for policy 0, policy_version 61682 (0.0008) [2023-10-08 06:20:35,798][00611] Updated weights for policy 0, policy_version 61692 (0.0008) [2023-10-08 06:20:38,128][00612] Updated weights for policy 1, policy_version 62050 (0.0008) [2023-10-08 06:20:38,496][00612] Updated weights for policy 1, policy_version 62060 (0.0007) [2023-10-08 06:20:38,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 126713856. Throughput: 0: 1816.2, 1: 1856.3. Samples: 31686582. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:20:38,754][130385] Avg episode reward: [(0, '74.370'), (1, '72.480')] [2023-10-08 06:20:38,872][00612] Updated weights for policy 1, policy_version 62070 (0.0008) [2023-10-08 06:20:39,240][00612] Updated weights for policy 1, policy_version 62080 (0.0007) [2023-10-08 06:20:39,352][00611] Updated weights for policy 0, policy_version 61702 (0.0008) [2023-10-08 06:20:39,723][00611] Updated weights for policy 0, policy_version 61712 (0.0008) [2023-10-08 06:20:40,095][00611] Updated weights for policy 0, policy_version 61722 (0.0007) [2023-10-08 06:20:42,988][00612] Updated weights for policy 1, policy_version 62090 (0.0009) [2023-10-08 06:20:43,361][00612] Updated weights for policy 1, policy_version 62100 (0.0009) [2023-10-08 06:20:43,720][00612] Updated weights for policy 1, policy_version 62110 (0.0008) [2023-10-08 06:20:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 126779392. Throughput: 0: 1822.1, 1: 1853.7. Samples: 31709634. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:20:43,754][130385] Avg episode reward: [(0, '71.770'), (1, '68.790')] [2023-10-08 06:20:43,769][00611] Updated weights for policy 0, policy_version 61732 (0.0009) [2023-10-08 06:20:44,141][00611] Updated weights for policy 0, policy_version 61742 (0.0009) [2023-10-08 06:20:44,524][00611] Updated weights for policy 0, policy_version 61752 (0.0009) [2023-10-08 06:20:47,271][00612] Updated weights for policy 1, policy_version 62120 (0.0008) [2023-10-08 06:20:47,634][00612] Updated weights for policy 1, policy_version 62130 (0.0008) [2023-10-08 06:20:48,002][00612] Updated weights for policy 1, policy_version 62140 (0.0009) [2023-10-08 06:20:48,119][00611] Updated weights for policy 0, policy_version 61762 (0.0008) [2023-10-08 06:20:48,487][00611] Updated weights for policy 0, policy_version 61772 (0.0008) [2023-10-08 06:20:48,754][130385] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 126877696. Throughput: 0: 1823.3, 1: 1841.8. Samples: 31731320. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:20:48,755][130385] Avg episode reward: [(0, '73.310'), (1, '68.730')] [2023-10-08 06:20:48,859][00611] Updated weights for policy 0, policy_version 61782 (0.0010) [2023-10-08 06:20:49,222][00611] Updated weights for policy 0, policy_version 61792 (0.0009) [2023-10-08 06:20:51,574][00612] Updated weights for policy 1, policy_version 62150 (0.0009) [2023-10-08 06:20:51,943][00612] Updated weights for policy 1, policy_version 62160 (0.0009) [2023-10-08 06:20:52,311][00612] Updated weights for policy 1, policy_version 62170 (0.0007) [2023-10-08 06:20:52,950][00611] Updated weights for policy 0, policy_version 61802 (0.0009) [2023-10-08 06:20:53,323][00611] Updated weights for policy 0, policy_version 61812 (0.0009) [2023-10-08 06:20:53,690][00611] Updated weights for policy 0, policy_version 61822 (0.0009) [2023-10-08 06:20:53,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126943232. Throughput: 0: 1828.3, 1: 1862.8. Samples: 31743160. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:20:53,755][130385] Avg episode reward: [(0, '70.660'), (1, '71.350')] [2023-10-08 06:20:56,039][00612] Updated weights for policy 1, policy_version 62180 (0.0008) [2023-10-08 06:20:56,414][00612] Updated weights for policy 1, policy_version 62190 (0.0009) [2023-10-08 06:20:56,787][00612] Updated weights for policy 1, policy_version 62200 (0.0011) [2023-10-08 06:20:57,417][00611] Updated weights for policy 0, policy_version 61832 (0.0010) [2023-10-08 06:20:57,788][00611] Updated weights for policy 0, policy_version 61842 (0.0009) [2023-10-08 06:20:58,159][00611] Updated weights for policy 0, policy_version 61852 (0.0009) [2023-10-08 06:20:58,754][130385] Fps is (10 sec: 16384.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 127041536. Throughput: 0: 1825.6, 1: 1844.8. Samples: 31764464. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 06:20:58,754][130385] Avg episode reward: [(0, '68.910'), (1, '73.340')] [2023-10-08 06:21:00,378][00612] Updated weights for policy 1, policy_version 62210 (0.0009) [2023-10-08 06:21:00,744][00612] Updated weights for policy 1, policy_version 62220 (0.0008) [2023-10-08 06:21:01,113][00612] Updated weights for policy 1, policy_version 62230 (0.0007) [2023-10-08 06:21:01,486][00612] Updated weights for policy 1, policy_version 62240 (0.0008) [2023-10-08 06:21:01,753][00611] Updated weights for policy 0, policy_version 61862 (0.0010) [2023-10-08 06:21:02,135][00611] Updated weights for policy 0, policy_version 61872 (0.0010) [2023-10-08 06:21:02,512][00611] Updated weights for policy 0, policy_version 61882 (0.0011) [2023-10-08 06:21:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 127107072. Throughput: 0: 1822.1, 1: 1852.9. Samples: 31785918. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 06:21:03,755][130385] Avg episode reward: [(0, '67.770'), (1, '70.960')] [2023-10-08 06:21:05,068][00612] Updated weights for policy 1, policy_version 62250 (0.0008) [2023-10-08 06:21:05,445][00612] Updated weights for policy 1, policy_version 62260 (0.0009) [2023-10-08 06:21:05,808][00612] Updated weights for policy 1, policy_version 62270 (0.0008) [2023-10-08 06:21:06,226][00611] Updated weights for policy 0, policy_version 61892 (0.0009) [2023-10-08 06:21:06,597][00611] Updated weights for policy 0, policy_version 61902 (0.0007) [2023-10-08 06:21:06,970][00611] Updated weights for policy 0, policy_version 61912 (0.0008) [2023-10-08 06:21:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 127172608. Throughput: 0: 1818.6, 1: 1849.7. Samples: 31797374. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 06:21:08,755][130385] Avg episode reward: [(0, '65.850'), (1, '70.640')] [2023-10-08 06:21:09,181][00612] Updated weights for policy 1, policy_version 62280 (0.0010) [2023-10-08 06:21:09,548][00612] Updated weights for policy 1, policy_version 62290 (0.0008) [2023-10-08 06:21:09,921][00612] Updated weights for policy 1, policy_version 62300 (0.0009) [2023-10-08 06:21:10,642][00611] Updated weights for policy 0, policy_version 61922 (0.0009) [2023-10-08 06:21:11,016][00611] Updated weights for policy 0, policy_version 61932 (0.0008) [2023-10-08 06:21:11,390][00611] Updated weights for policy 0, policy_version 61942 (0.0010) [2023-10-08 06:21:11,759][00611] Updated weights for policy 0, policy_version 61952 (0.0009) [2023-10-08 06:21:13,498][00612] Updated weights for policy 1, policy_version 62310 (0.0007) [2023-10-08 06:21:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 127238144. Throughput: 0: 1821.2, 1: 1863.7. Samples: 31819440. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 06:21:13,754][130385] Avg episode reward: [(0, '60.690'), (1, '69.280')] [2023-10-08 06:21:13,871][00612] Updated weights for policy 1, policy_version 62320 (0.0009) [2023-10-08 06:21:14,234][00612] Updated weights for policy 1, policy_version 62330 (0.0009) [2023-10-08 06:21:15,401][00611] Updated weights for policy 0, policy_version 61962 (0.0009) [2023-10-08 06:21:15,779][00611] Updated weights for policy 0, policy_version 61972 (0.0010) [2023-10-08 06:21:16,147][00611] Updated weights for policy 0, policy_version 61982 (0.0009) [2023-10-08 06:21:17,754][00612] Updated weights for policy 1, policy_version 62340 (0.0008) [2023-10-08 06:21:18,110][00612] Updated weights for policy 1, policy_version 62350 (0.0011) [2023-10-08 06:21:18,484][00612] Updated weights for policy 1, policy_version 62360 (0.0008) [2023-10-08 06:21:18,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 127303680. Throughput: 0: 1828.6, 1: 1849.3. Samples: 31842092. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 06:21:18,755][130385] Avg episode reward: [(0, '63.130'), (1, '72.250')] [2023-10-08 06:21:19,855][00611] Updated weights for policy 0, policy_version 61992 (0.0008) [2023-10-08 06:21:20,223][00611] Updated weights for policy 0, policy_version 62002 (0.0008) [2023-10-08 06:21:20,599][00611] Updated weights for policy 0, policy_version 62012 (0.0009) [2023-10-08 06:21:22,028][00612] Updated weights for policy 1, policy_version 62370 (0.0009) [2023-10-08 06:21:22,398][00612] Updated weights for policy 1, policy_version 62380 (0.0008) [2023-10-08 06:21:22,765][00612] Updated weights for policy 1, policy_version 62390 (0.0008) [2023-10-08 06:21:23,131][00612] Updated weights for policy 1, policy_version 62400 (0.0007) [2023-10-08 06:21:23,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 127401984. Throughput: 0: 1829.4, 1: 1867.0. Samples: 31852920. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 06:21:23,755][130385] Avg episode reward: [(0, '62.320'), (1, '72.590')] [2023-10-08 06:21:24,158][00611] Updated weights for policy 0, policy_version 62022 (0.0010) [2023-10-08 06:21:24,529][00611] Updated weights for policy 0, policy_version 62032 (0.0008) [2023-10-08 06:21:24,896][00611] Updated weights for policy 0, policy_version 62042 (0.0007) [2023-10-08 06:21:26,716][00612] Updated weights for policy 1, policy_version 62410 (0.0007) [2023-10-08 06:21:27,079][00612] Updated weights for policy 1, policy_version 62420 (0.0009) [2023-10-08 06:21:27,443][00612] Updated weights for policy 1, policy_version 62430 (0.0008) [2023-10-08 06:21:28,449][00611] Updated weights for policy 0, policy_version 62052 (0.0009) [2023-10-08 06:21:28,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127467520. Throughput: 0: 1833.5, 1: 1850.6. Samples: 31875422. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 06:21:28,755][130385] Avg episode reward: [(0, '61.930'), (1, '70.300')] [2023-10-08 06:21:28,812][00611] Updated weights for policy 0, policy_version 62062 (0.0008) [2023-10-08 06:21:29,185][00611] Updated weights for policy 0, policy_version 62072 (0.0009) [2023-10-08 06:21:31,203][00612] Updated weights for policy 1, policy_version 62440 (0.0007) [2023-10-08 06:21:31,570][00612] Updated weights for policy 1, policy_version 62450 (0.0007) [2023-10-08 06:21:31,937][00612] Updated weights for policy 1, policy_version 62460 (0.0007) [2023-10-08 06:21:32,824][00611] Updated weights for policy 0, policy_version 62082 (0.0008) [2023-10-08 06:21:33,193][00611] Updated weights for policy 0, policy_version 62092 (0.0008) [2023-10-08 06:21:33,566][00611] Updated weights for policy 0, policy_version 62102 (0.0009) [2023-10-08 06:21:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127533056. Throughput: 0: 1830.8, 1: 1869.2. Samples: 31897820. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 06:21:33,755][130385] Avg episode reward: [(0, '61.240'), (1, '68.610')] [2023-10-08 06:21:33,927][00611] Updated weights for policy 0, policy_version 62112 (0.0008) [2023-10-08 06:21:35,520][00612] Updated weights for policy 1, policy_version 62470 (0.0008) [2023-10-08 06:21:35,902][00612] Updated weights for policy 1, policy_version 62480 (0.0007) [2023-10-08 06:21:36,262][00612] Updated weights for policy 1, policy_version 62490 (0.0007) [2023-10-08 06:21:37,528][00611] Updated weights for policy 0, policy_version 62122 (0.0011) [2023-10-08 06:21:37,905][00611] Updated weights for policy 0, policy_version 62132 (0.0007) [2023-10-08 06:21:38,277][00611] Updated weights for policy 0, policy_version 62142 (0.0011) [2023-10-08 06:21:38,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 127631360. Throughput: 0: 1838.9, 1: 1840.2. Samples: 31908722. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 06:21:38,755][130385] Avg episode reward: [(0, '63.020'), (1, '69.840')] [2023-10-08 06:21:39,827][00612] Updated weights for policy 1, policy_version 62500 (0.0007) [2023-10-08 06:21:40,202][00612] Updated weights for policy 1, policy_version 62510 (0.0008) [2023-10-08 06:21:40,576][00612] Updated weights for policy 1, policy_version 62520 (0.0009) [2023-10-08 06:21:42,067][00611] Updated weights for policy 0, policy_version 62152 (0.0009) [2023-10-08 06:21:42,440][00611] Updated weights for policy 0, policy_version 62162 (0.0007) [2023-10-08 06:21:42,816][00611] Updated weights for policy 0, policy_version 62172 (0.0008) [2023-10-08 06:21:43,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 127696896. Throughput: 0: 1823.1, 1: 1870.4. Samples: 31930670. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 06:21:43,754][130385] Avg episode reward: [(0, '64.740'), (1, '68.510')] [2023-10-08 06:21:44,161][00612] Updated weights for policy 1, policy_version 62530 (0.0009) [2023-10-08 06:21:44,531][00612] Updated weights for policy 1, policy_version 62540 (0.0008) [2023-10-08 06:21:44,896][00612] Updated weights for policy 1, policy_version 62550 (0.0009) [2023-10-08 06:21:45,266][00612] Updated weights for policy 1, policy_version 62560 (0.0008) [2023-10-08 06:21:46,446][00611] Updated weights for policy 0, policy_version 62182 (0.0008) [2023-10-08 06:21:46,820][00611] Updated weights for policy 0, policy_version 62192 (0.0008) [2023-10-08 06:21:47,200][00611] Updated weights for policy 0, policy_version 62202 (0.0008) [2023-10-08 06:21:48,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 127762432. Throughput: 0: 1831.9, 1: 1876.5. Samples: 31952796. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 06:21:48,754][130385] Avg episode reward: [(0, '65.710'), (1, '68.150')] [2023-10-08 06:21:48,835][00612] Updated weights for policy 1, policy_version 62570 (0.0008) [2023-10-08 06:21:49,208][00612] Updated weights for policy 1, policy_version 62580 (0.0008) [2023-10-08 06:21:49,568][00612] Updated weights for policy 1, policy_version 62590 (0.0008) [2023-10-08 06:21:50,765][00611] Updated weights for policy 0, policy_version 62212 (0.0009) [2023-10-08 06:21:51,130][00611] Updated weights for policy 0, policy_version 62222 (0.0009) [2023-10-08 06:21:51,510][00611] Updated weights for policy 0, policy_version 62232 (0.0008) [2023-10-08 06:21:53,146][00612] Updated weights for policy 1, policy_version 62600 (0.0010) [2023-10-08 06:21:53,518][00612] Updated weights for policy 1, policy_version 62610 (0.0008) [2023-10-08 06:21:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 127827968. Throughput: 0: 1823.6, 1: 1874.7. Samples: 31963800. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 06:21:53,754][130385] Avg episode reward: [(0, '67.210'), (1, '70.810')] [2023-10-08 06:21:53,887][00612] Updated weights for policy 1, policy_version 62620 (0.0007) [2023-10-08 06:21:55,130][00611] Updated weights for policy 0, policy_version 62242 (0.0008) [2023-10-08 06:21:55,510][00611] Updated weights for policy 0, policy_version 62252 (0.0009) [2023-10-08 06:21:55,870][00611] Updated weights for policy 0, policy_version 62262 (0.0010) [2023-10-08 06:21:56,242][00611] Updated weights for policy 0, policy_version 62272 (0.0009) [2023-10-08 06:21:57,473][00612] Updated weights for policy 1, policy_version 62630 (0.0009) [2023-10-08 06:21:57,831][00612] Updated weights for policy 1, policy_version 62640 (0.0008) [2023-10-08 06:21:58,202][00612] Updated weights for policy 1, policy_version 62650 (0.0009) [2023-10-08 06:21:58,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 127926272. Throughput: 0: 1843.1, 1: 1870.0. Samples: 31986532. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 06:21:58,755][130385] Avg episode reward: [(0, '67.680'), (1, '67.380')] [2023-10-08 06:21:59,947][00611] Updated weights for policy 0, policy_version 62282 (0.0007) [2023-10-08 06:22:00,323][00611] Updated weights for policy 0, policy_version 62292 (0.0008) [2023-10-08 06:22:00,700][00611] Updated weights for policy 0, policy_version 62302 (0.0009) [2023-10-08 06:22:01,841][00612] Updated weights for policy 1, policy_version 62660 (0.0009) [2023-10-08 06:22:02,203][00612] Updated weights for policy 1, policy_version 62670 (0.0007) [2023-10-08 06:22:02,577][00612] Updated weights for policy 1, policy_version 62680 (0.0007) [2023-10-08 06:22:03,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 127991808. Throughput: 0: 1843.4, 1: 1846.7. Samples: 32008146. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 06:22:03,754][130385] Avg episode reward: [(0, '65.590'), (1, '69.140')] [2023-10-08 06:22:04,339][00611] Updated weights for policy 0, policy_version 62312 (0.0011) [2023-10-08 06:22:04,716][00611] Updated weights for policy 0, policy_version 62322 (0.0009) [2023-10-08 06:22:05,092][00611] Updated weights for policy 0, policy_version 62332 (0.0008) [2023-10-08 06:22:06,239][00612] Updated weights for policy 1, policy_version 62690 (0.0007) [2023-10-08 06:22:06,609][00612] Updated weights for policy 1, policy_version 62700 (0.0007) [2023-10-08 06:22:06,962][00612] Updated weights for policy 1, policy_version 62710 (0.0007) [2023-10-08 06:22:07,329][00612] Updated weights for policy 1, policy_version 62720 (0.0008) [2023-10-08 06:22:08,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128057344. Throughput: 0: 1841.0, 1: 1862.4. Samples: 32019570. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 06:22:08,755][130385] Avg episode reward: [(0, '64.440'), (1, '67.790')] [2023-10-08 06:22:08,762][00611] Updated weights for policy 0, policy_version 62342 (0.0009) [2023-10-08 06:22:09,148][00611] Updated weights for policy 0, policy_version 62352 (0.0012) [2023-10-08 06:22:09,519][00611] Updated weights for policy 0, policy_version 62362 (0.0008) [2023-10-08 06:22:10,856][00612] Updated weights for policy 1, policy_version 62730 (0.0008) [2023-10-08 06:22:11,220][00612] Updated weights for policy 1, policy_version 62740 (0.0009) [2023-10-08 06:22:11,592][00612] Updated weights for policy 1, policy_version 62750 (0.0008) [2023-10-08 06:22:12,971][00611] Updated weights for policy 0, policy_version 62372 (0.0007) [2023-10-08 06:22:13,353][00611] Updated weights for policy 0, policy_version 62382 (0.0009) [2023-10-08 06:22:13,731][00611] Updated weights for policy 0, policy_version 62392 (0.0009) [2023-10-08 06:22:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128122880. Throughput: 0: 1842.4, 1: 1850.2. Samples: 32041590. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 06:22:13,754][130385] Avg episode reward: [(0, '66.290'), (1, '66.510')] [2023-10-08 06:22:15,138][00612] Updated weights for policy 1, policy_version 62760 (0.0010) [2023-10-08 06:22:15,504][00612] Updated weights for policy 1, policy_version 62770 (0.0011) [2023-10-08 06:22:15,871][00612] Updated weights for policy 1, policy_version 62780 (0.0011) [2023-10-08 06:22:17,198][00611] Updated weights for policy 0, policy_version 62402 (0.0008) [2023-10-08 06:22:17,571][00611] Updated weights for policy 0, policy_version 62412 (0.0010) [2023-10-08 06:22:17,938][00611] Updated weights for policy 0, policy_version 62422 (0.0009) [2023-10-08 06:22:18,315][00611] Updated weights for policy 0, policy_version 62432 (0.0009) [2023-10-08 06:22:18,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 128221184. Throughput: 0: 1820.4, 1: 1867.9. Samples: 32063792. Policy #0 lag: (min: 31.0, avg: 32.5, max: 56.0) [2023-10-08 06:22:18,755][130385] Avg episode reward: [(0, '69.070'), (1, '67.680')] [2023-10-08 06:22:18,762][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000062432_63930368.pth... [2023-10-08 06:22:18,762][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000062784_64290816.pth... [2023-10-08 06:22:18,803][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000060704_62160896.pth [2023-10-08 06:22:18,803][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000061056_62521344.pth [2023-10-08 06:22:19,354][00612] Updated weights for policy 1, policy_version 62790 (0.0008) [2023-10-08 06:22:19,719][00612] Updated weights for policy 1, policy_version 62800 (0.0007) [2023-10-08 06:22:20,086][00612] Updated weights for policy 1, policy_version 62810 (0.0008) [2023-10-08 06:22:21,883][00611] Updated weights for policy 0, policy_version 62442 (0.0007) [2023-10-08 06:22:22,256][00611] Updated weights for policy 0, policy_version 62452 (0.0009) [2023-10-08 06:22:22,619][00611] Updated weights for policy 0, policy_version 62462 (0.0009) [2023-10-08 06:22:23,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 128286720. Throughput: 0: 1840.8, 1: 1857.4. Samples: 32075142. Policy #0 lag: (min: 31.0, avg: 32.5, max: 56.0) [2023-10-08 06:22:23,754][130385] Avg episode reward: [(0, '68.260'), (1, '67.360')] [2023-10-08 06:22:23,837][00612] Updated weights for policy 1, policy_version 62820 (0.0009) [2023-10-08 06:22:24,212][00612] Updated weights for policy 1, policy_version 62830 (0.0008) [2023-10-08 06:22:24,585][00612] Updated weights for policy 1, policy_version 62840 (0.0009) [2023-10-08 06:22:26,309][00611] Updated weights for policy 0, policy_version 62472 (0.0008) [2023-10-08 06:22:26,673][00611] Updated weights for policy 0, policy_version 62482 (0.0007) [2023-10-08 06:22:27,048][00611] Updated weights for policy 0, policy_version 62492 (0.0007) [2023-10-08 06:22:28,112][00612] Updated weights for policy 1, policy_version 62850 (0.0008) [2023-10-08 06:22:28,476][00612] Updated weights for policy 1, policy_version 62860 (0.0008) [2023-10-08 06:22:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 128352256. Throughput: 0: 1831.9, 1: 1868.4. Samples: 32097182. Policy #0 lag: (min: 31.0, avg: 32.5, max: 56.0) [2023-10-08 06:22:28,754][130385] Avg episode reward: [(0, '68.940'), (1, '70.020')] [2023-10-08 06:22:28,846][00612] Updated weights for policy 1, policy_version 62870 (0.0009) [2023-10-08 06:22:29,212][00612] Updated weights for policy 1, policy_version 62880 (0.0008) [2023-10-08 06:22:30,707][00611] Updated weights for policy 0, policy_version 62502 (0.0008) [2023-10-08 06:22:31,090][00611] Updated weights for policy 0, policy_version 62512 (0.0009) [2023-10-08 06:22:31,472][00611] Updated weights for policy 0, policy_version 62522 (0.0009) [2023-10-08 06:22:32,989][00612] Updated weights for policy 1, policy_version 62890 (0.0008) [2023-10-08 06:22:33,353][00612] Updated weights for policy 1, policy_version 62900 (0.0007) [2023-10-08 06:22:33,723][00612] Updated weights for policy 1, policy_version 62910 (0.0008) [2023-10-08 06:22:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 128417792. Throughput: 0: 1859.3, 1: 1845.2. Samples: 32119500. Policy #0 lag: (min: 31.0, avg: 32.5, max: 56.0) [2023-10-08 06:22:33,754][130385] Avg episode reward: [(0, '69.510'), (1, '71.250')] [2023-10-08 06:22:35,069][00611] Updated weights for policy 0, policy_version 62532 (0.0008) [2023-10-08 06:22:35,443][00611] Updated weights for policy 0, policy_version 62542 (0.0008) [2023-10-08 06:22:35,816][00611] Updated weights for policy 0, policy_version 62552 (0.0008) [2023-10-08 06:22:37,226][00612] Updated weights for policy 1, policy_version 62920 (0.0008) [2023-10-08 06:22:37,592][00612] Updated weights for policy 1, policy_version 62930 (0.0008) [2023-10-08 06:22:37,962][00612] Updated weights for policy 1, policy_version 62940 (0.0008) [2023-10-08 06:22:38,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 128516096. Throughput: 0: 1840.6, 1: 1862.8. Samples: 32130450. Policy #0 lag: (min: 31.0, avg: 32.5, max: 56.0) [2023-10-08 06:22:38,754][130385] Avg episode reward: [(0, '68.460'), (1, '72.430')] [2023-10-08 06:22:39,558][00611] Updated weights for policy 0, policy_version 62562 (0.0008) [2023-10-08 06:22:39,929][00611] Updated weights for policy 0, policy_version 62572 (0.0008) [2023-10-08 06:22:40,306][00611] Updated weights for policy 0, policy_version 62582 (0.0007) [2023-10-08 06:22:40,675][00611] Updated weights for policy 0, policy_version 62592 (0.0008) [2023-10-08 06:22:41,603][00612] Updated weights for policy 1, policy_version 62950 (0.0008) [2023-10-08 06:22:41,970][00612] Updated weights for policy 1, policy_version 62960 (0.0010) [2023-10-08 06:22:42,334][00612] Updated weights for policy 1, policy_version 62970 (0.0008) [2023-10-08 06:22:43,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 128581632. Throughput: 0: 1857.8, 1: 1834.9. Samples: 32152700. Policy #0 lag: (min: 31.0, avg: 32.5, max: 56.0) [2023-10-08 06:22:43,754][130385] Avg episode reward: [(0, '67.050'), (1, '72.910')] [2023-10-08 06:22:43,992][00611] Updated weights for policy 0, policy_version 62602 (0.0007) [2023-10-08 06:22:44,352][00611] Updated weights for policy 0, policy_version 62612 (0.0009) [2023-10-08 06:22:44,723][00611] Updated weights for policy 0, policy_version 62622 (0.0008) [2023-10-08 06:22:46,047][00612] Updated weights for policy 1, policy_version 62980 (0.0008) [2023-10-08 06:22:46,419][00612] Updated weights for policy 1, policy_version 62990 (0.0008) [2023-10-08 06:22:46,785][00612] Updated weights for policy 1, policy_version 63000 (0.0007) [2023-10-08 06:22:48,407][00611] Updated weights for policy 0, policy_version 62632 (0.0008) [2023-10-08 06:22:48,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 128647168. Throughput: 0: 1861.7, 1: 1857.6. Samples: 32175516. Policy #0 lag: (min: 31.0, avg: 32.5, max: 56.0) [2023-10-08 06:22:48,754][130385] Avg episode reward: [(0, '66.160'), (1, '71.300')] [2023-10-08 06:22:48,779][00611] Updated weights for policy 0, policy_version 62642 (0.0008) [2023-10-08 06:22:49,161][00611] Updated weights for policy 0, policy_version 62652 (0.0009) [2023-10-08 06:22:50,421][00612] Updated weights for policy 1, policy_version 63010 (0.0007) [2023-10-08 06:22:50,789][00612] Updated weights for policy 1, policy_version 63020 (0.0007) [2023-10-08 06:22:51,151][00612] Updated weights for policy 1, policy_version 63030 (0.0007) [2023-10-08 06:22:51,517][00612] Updated weights for policy 1, policy_version 63040 (0.0008) [2023-10-08 06:22:52,707][00611] Updated weights for policy 0, policy_version 62662 (0.0010) [2023-10-08 06:22:53,079][00611] Updated weights for policy 0, policy_version 62672 (0.0010) [2023-10-08 06:22:53,460][00611] Updated weights for policy 0, policy_version 62682 (0.0009) [2023-10-08 06:22:53,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 128745472. Throughput: 0: 1866.3, 1: 1833.7. Samples: 32186072. Policy #0 lag: (min: 8.0, avg: 32.0, max: 40.0) [2023-10-08 06:22:53,754][130385] Avg episode reward: [(0, '64.000'), (1, '73.210')] [2023-10-08 06:22:55,215][00612] Updated weights for policy 1, policy_version 63050 (0.0008) [2023-10-08 06:22:55,593][00612] Updated weights for policy 1, policy_version 63060 (0.0009) [2023-10-08 06:22:55,955][00612] Updated weights for policy 1, policy_version 63070 (0.0010) [2023-10-08 06:22:57,125][00611] Updated weights for policy 0, policy_version 62692 (0.0008) [2023-10-08 06:22:57,515][00611] Updated weights for policy 0, policy_version 62702 (0.0007) [2023-10-08 06:22:57,886][00611] Updated weights for policy 0, policy_version 62712 (0.0007) [2023-10-08 06:22:58,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 128811008. Throughput: 0: 1856.6, 1: 1857.5. Samples: 32208724. Policy #0 lag: (min: 8.0, avg: 32.0, max: 40.0) [2023-10-08 06:22:58,755][130385] Avg episode reward: [(0, '64.830'), (1, '70.190')] [2023-10-08 06:22:59,433][00612] Updated weights for policy 1, policy_version 63080 (0.0008) [2023-10-08 06:22:59,806][00612] Updated weights for policy 1, policy_version 63090 (0.0007) [2023-10-08 06:23:00,172][00612] Updated weights for policy 1, policy_version 63100 (0.0009) [2023-10-08 06:23:01,517][00611] Updated weights for policy 0, policy_version 62722 (0.0008) [2023-10-08 06:23:01,895][00611] Updated weights for policy 0, policy_version 62732 (0.0007) [2023-10-08 06:23:02,273][00611] Updated weights for policy 0, policy_version 62742 (0.0008) [2023-10-08 06:23:02,640][00611] Updated weights for policy 0, policy_version 62752 (0.0009) [2023-10-08 06:23:03,698][00612] Updated weights for policy 1, policy_version 63110 (0.0008) [2023-10-08 06:23:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 128876544. Throughput: 0: 1853.6, 1: 1855.8. Samples: 32230718. Policy #0 lag: (min: 8.0, avg: 32.0, max: 40.0) [2023-10-08 06:23:03,754][130385] Avg episode reward: [(0, '65.460'), (1, '72.000')] [2023-10-08 06:23:04,063][00612] Updated weights for policy 1, policy_version 63120 (0.0009) [2023-10-08 06:23:04,440][00612] Updated weights for policy 1, policy_version 63130 (0.0008) [2023-10-08 06:23:06,169][00611] Updated weights for policy 0, policy_version 62762 (0.0007) [2023-10-08 06:23:06,546][00611] Updated weights for policy 0, policy_version 62772 (0.0007) [2023-10-08 06:23:06,917][00611] Updated weights for policy 0, policy_version 62782 (0.0007) [2023-10-08 06:23:08,036][00612] Updated weights for policy 1, policy_version 63140 (0.0008) [2023-10-08 06:23:08,405][00612] Updated weights for policy 1, policy_version 63150 (0.0010) [2023-10-08 06:23:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 128942080. Throughput: 0: 1849.1, 1: 1856.2. Samples: 32241882. Policy #0 lag: (min: 8.0, avg: 32.0, max: 40.0) [2023-10-08 06:23:08,754][130385] Avg episode reward: [(0, '65.740'), (1, '70.730')] [2023-10-08 06:23:08,774][00612] Updated weights for policy 1, policy_version 63160 (0.0008) [2023-10-08 06:23:10,564][00611] Updated weights for policy 0, policy_version 62792 (0.0010) [2023-10-08 06:23:10,931][00611] Updated weights for policy 0, policy_version 62802 (0.0010) [2023-10-08 06:23:11,302][00611] Updated weights for policy 0, policy_version 62812 (0.0007) [2023-10-08 06:23:12,601][00612] Updated weights for policy 1, policy_version 63170 (0.0009) [2023-10-08 06:23:13,007][00612] Updated weights for policy 1, policy_version 63180 (0.0007) [2023-10-08 06:23:13,376][00612] Updated weights for policy 1, policy_version 63190 (0.0007) [2023-10-08 06:23:13,741][00612] Updated weights for policy 1, policy_version 63200 (0.0008) [2023-10-08 06:23:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 129040384. Throughput: 0: 1851.2, 1: 1855.9. Samples: 32264004. Policy #0 lag: (min: 8.0, avg: 32.0, max: 40.0) [2023-10-08 06:23:13,754][130385] Avg episode reward: [(0, '68.780'), (1, '73.160')] [2023-10-08 06:23:14,916][00611] Updated weights for policy 0, policy_version 62822 (0.0007) [2023-10-08 06:23:15,279][00611] Updated weights for policy 0, policy_version 62832 (0.0010) [2023-10-08 06:23:15,654][00611] Updated weights for policy 0, policy_version 62842 (0.0010) [2023-10-08 06:23:17,365][00612] Updated weights for policy 1, policy_version 63210 (0.0008) [2023-10-08 06:23:17,746][00612] Updated weights for policy 1, policy_version 63220 (0.0009) [2023-10-08 06:23:18,111][00612] Updated weights for policy 1, policy_version 63230 (0.0008) [2023-10-08 06:23:18,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 129105920. Throughput: 0: 1855.7, 1: 1838.1. Samples: 32285720. Policy #0 lag: (min: 8.0, avg: 32.0, max: 40.0) [2023-10-08 06:23:18,754][130385] Avg episode reward: [(0, '65.600'), (1, '75.130')] [2023-10-08 06:23:19,307][00611] Updated weights for policy 0, policy_version 62852 (0.0009) [2023-10-08 06:23:19,701][00611] Updated weights for policy 0, policy_version 62862 (0.0009) [2023-10-08 06:23:20,070][00611] Updated weights for policy 0, policy_version 62872 (0.0009) [2023-10-08 06:23:21,608][00612] Updated weights for policy 1, policy_version 63240 (0.0009) [2023-10-08 06:23:21,975][00612] Updated weights for policy 1, policy_version 63250 (0.0009) [2023-10-08 06:23:22,343][00612] Updated weights for policy 1, policy_version 63260 (0.0009) [2023-10-08 06:23:23,703][00611] Updated weights for policy 0, policy_version 62882 (0.0008) [2023-10-08 06:23:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 129171456. Throughput: 0: 1848.4, 1: 1858.2. Samples: 32297246. Policy #0 lag: (min: 8.0, avg: 32.0, max: 40.0) [2023-10-08 06:23:23,754][130385] Avg episode reward: [(0, '66.020'), (1, '79.350')] [2023-10-08 06:23:24,073][00611] Updated weights for policy 0, policy_version 62892 (0.0011) [2023-10-08 06:23:24,439][00611] Updated weights for policy 0, policy_version 62902 (0.0010) [2023-10-08 06:23:24,818][00611] Updated weights for policy 0, policy_version 62912 (0.0010) [2023-10-08 06:23:25,861][00612] Updated weights for policy 1, policy_version 63270 (0.0009) [2023-10-08 06:23:26,234][00612] Updated weights for policy 1, policy_version 63280 (0.0008) [2023-10-08 06:23:26,604][00612] Updated weights for policy 1, policy_version 63290 (0.0008) [2023-10-08 06:23:28,416][00611] Updated weights for policy 0, policy_version 62922 (0.0009) [2023-10-08 06:23:28,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 129236992. Throughput: 0: 1850.1, 1: 1846.8. Samples: 32319060. Policy #0 lag: (min: 8.0, avg: 32.0, max: 40.0) [2023-10-08 06:23:28,755][130385] Avg episode reward: [(0, '66.360'), (1, '78.230')] [2023-10-08 06:23:28,791][00611] Updated weights for policy 0, policy_version 62932 (0.0010) [2023-10-08 06:23:29,158][00611] Updated weights for policy 0, policy_version 62942 (0.0008) [2023-10-08 06:23:30,139][00612] Updated weights for policy 1, policy_version 63300 (0.0008) [2023-10-08 06:23:30,511][00612] Updated weights for policy 1, policy_version 63310 (0.0008) [2023-10-08 06:23:30,875][00612] Updated weights for policy 1, policy_version 63320 (0.0008) [2023-10-08 06:23:32,870][00611] Updated weights for policy 0, policy_version 62952 (0.0007) [2023-10-08 06:23:33,243][00611] Updated weights for policy 0, policy_version 62962 (0.0010) [2023-10-08 06:23:33,615][00611] Updated weights for policy 0, policy_version 62972 (0.0008) [2023-10-08 06:23:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 129302528. Throughput: 0: 1826.5, 1: 1860.3. Samples: 32341420. Policy #0 lag: (min: 8.0, avg: 32.0, max: 40.0) [2023-10-08 06:23:33,754][130385] Avg episode reward: [(0, '64.600'), (1, '78.760')] [2023-10-08 06:23:34,538][00612] Updated weights for policy 1, policy_version 63330 (0.0009) [2023-10-08 06:23:34,917][00612] Updated weights for policy 1, policy_version 63340 (0.0012) [2023-10-08 06:23:35,282][00612] Updated weights for policy 1, policy_version 63350 (0.0011) [2023-10-08 06:23:35,651][00612] Updated weights for policy 1, policy_version 63360 (0.0008) [2023-10-08 06:23:37,415][00611] Updated weights for policy 0, policy_version 62982 (0.0008) [2023-10-08 06:23:37,791][00611] Updated weights for policy 0, policy_version 62992 (0.0010) [2023-10-08 06:23:38,164][00611] Updated weights for policy 0, policy_version 63002 (0.0011) [2023-10-08 06:23:38,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 129400832. Throughput: 0: 1840.7, 1: 1848.3. Samples: 32352074. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 06:23:38,755][130385] Avg episode reward: [(0, '65.640'), (1, '80.680')] [2023-10-08 06:23:39,298][00612] Updated weights for policy 1, policy_version 63370 (0.0009) [2023-10-08 06:23:39,668][00612] Updated weights for policy 1, policy_version 63380 (0.0009) [2023-10-08 06:23:40,041][00612] Updated weights for policy 1, policy_version 63390 (0.0009) [2023-10-08 06:23:41,674][00611] Updated weights for policy 0, policy_version 63012 (0.0008) [2023-10-08 06:23:42,054][00611] Updated weights for policy 0, policy_version 63022 (0.0009) [2023-10-08 06:23:42,415][00611] Updated weights for policy 0, policy_version 63032 (0.0010) [2023-10-08 06:23:43,743][00612] Updated weights for policy 1, policy_version 63400 (0.0010) [2023-10-08 06:23:43,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 129466368. Throughput: 0: 1824.9, 1: 1858.8. Samples: 32374490. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 06:23:43,754][130385] Avg episode reward: [(0, '64.200'), (1, '80.950')] [2023-10-08 06:23:44,110][00612] Updated weights for policy 1, policy_version 63410 (0.0010) [2023-10-08 06:23:44,478][00612] Updated weights for policy 1, policy_version 63420 (0.0009) [2023-10-08 06:23:46,034][00611] Updated weights for policy 0, policy_version 63042 (0.0010) [2023-10-08 06:23:46,406][00611] Updated weights for policy 0, policy_version 63052 (0.0009) [2023-10-08 06:23:46,782][00611] Updated weights for policy 0, policy_version 63062 (0.0008) [2023-10-08 06:23:47,157][00611] Updated weights for policy 0, policy_version 63072 (0.0007) [2023-10-08 06:23:48,175][00612] Updated weights for policy 1, policy_version 63430 (0.0008) [2023-10-08 06:23:48,542][00612] Updated weights for policy 1, policy_version 63440 (0.0009) [2023-10-08 06:23:48,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 129531904. Throughput: 0: 1837.9, 1: 1845.3. Samples: 32396464. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 06:23:48,754][130385] Avg episode reward: [(0, '64.860'), (1, '81.530')] [2023-10-08 06:23:48,910][00612] Updated weights for policy 1, policy_version 63450 (0.0007) [2023-10-08 06:23:50,997][00611] Updated weights for policy 0, policy_version 63082 (0.0007) [2023-10-08 06:23:51,371][00611] Updated weights for policy 0, policy_version 63092 (0.0007) [2023-10-08 06:23:51,741][00611] Updated weights for policy 0, policy_version 63102 (0.0009) [2023-10-08 06:23:52,423][00612] Updated weights for policy 1, policy_version 63460 (0.0009) [2023-10-08 06:23:52,792][00612] Updated weights for policy 1, policy_version 63470 (0.0008) [2023-10-08 06:23:53,163][00612] Updated weights for policy 1, policy_version 63480 (0.0010) [2023-10-08 06:23:53,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 129630208. Throughput: 0: 1828.2, 1: 1849.4. Samples: 32407376. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 06:23:53,754][130385] Avg episode reward: [(0, '67.560'), (1, '82.010')] [2023-10-08 06:23:55,426][00611] Updated weights for policy 0, policy_version 63112 (0.0009) [2023-10-08 06:23:55,794][00611] Updated weights for policy 0, policy_version 63122 (0.0010) [2023-10-08 06:23:56,162][00611] Updated weights for policy 0, policy_version 63132 (0.0010) [2023-10-08 06:23:56,902][00612] Updated weights for policy 1, policy_version 63490 (0.0010) [2023-10-08 06:23:57,264][00612] Updated weights for policy 1, policy_version 63500 (0.0007) [2023-10-08 06:23:57,637][00612] Updated weights for policy 1, policy_version 63510 (0.0009) [2023-10-08 06:23:57,998][00612] Updated weights for policy 1, policy_version 63520 (0.0009) [2023-10-08 06:23:58,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 129695744. Throughput: 0: 1830.8, 1: 1840.0. Samples: 32429188. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 06:23:58,754][130385] Avg episode reward: [(0, '66.460'), (1, '84.520')] [2023-10-08 06:23:58,755][00425] Saving new best policy, reward=84.520! [2023-10-08 06:23:59,782][00611] Updated weights for policy 0, policy_version 63142 (0.0008) [2023-10-08 06:24:00,142][00611] Updated weights for policy 0, policy_version 63152 (0.0007) [2023-10-08 06:24:00,517][00611] Updated weights for policy 0, policy_version 63162 (0.0007) [2023-10-08 06:24:01,579][00612] Updated weights for policy 1, policy_version 63530 (0.0007) [2023-10-08 06:24:01,951][00612] Updated weights for policy 1, policy_version 63540 (0.0009) [2023-10-08 06:24:02,322][00612] Updated weights for policy 1, policy_version 63550 (0.0008) [2023-10-08 06:24:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 129761280. Throughput: 0: 1833.6, 1: 1847.8. Samples: 32451386. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 06:24:03,754][130385] Avg episode reward: [(0, '67.100'), (1, '84.990')] [2023-10-08 06:24:03,762][00425] Saving new best policy, reward=84.990! [2023-10-08 06:24:04,172][00611] Updated weights for policy 0, policy_version 63172 (0.0009) [2023-10-08 06:24:04,577][00611] Updated weights for policy 0, policy_version 63182 (0.0009) [2023-10-08 06:24:04,950][00611] Updated weights for policy 0, policy_version 63192 (0.0010) [2023-10-08 06:24:06,038][00612] Updated weights for policy 1, policy_version 63560 (0.0010) [2023-10-08 06:24:06,405][00612] Updated weights for policy 1, policy_version 63570 (0.0009) [2023-10-08 06:24:06,771][00612] Updated weights for policy 1, policy_version 63580 (0.0008) [2023-10-08 06:24:08,611][00611] Updated weights for policy 0, policy_version 63202 (0.0010) [2023-10-08 06:24:08,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 129826816. Throughput: 0: 1832.4, 1: 1833.2. Samples: 32462200. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 06:24:08,755][130385] Avg episode reward: [(0, '68.920'), (1, '82.090')] [2023-10-08 06:24:08,976][00611] Updated weights for policy 0, policy_version 63212 (0.0009) [2023-10-08 06:24:09,361][00611] Updated weights for policy 0, policy_version 63222 (0.0009) [2023-10-08 06:24:09,729][00611] Updated weights for policy 0, policy_version 63232 (0.0007) [2023-10-08 06:24:10,314][00612] Updated weights for policy 1, policy_version 63590 (0.0011) [2023-10-08 06:24:10,687][00612] Updated weights for policy 1, policy_version 63600 (0.0007) [2023-10-08 06:24:11,059][00612] Updated weights for policy 1, policy_version 63610 (0.0009) [2023-10-08 06:24:13,359][00611] Updated weights for policy 0, policy_version 63242 (0.0008) [2023-10-08 06:24:13,730][00611] Updated weights for policy 0, policy_version 63252 (0.0008) [2023-10-08 06:24:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14773.4). Total num frames: 129892352. Throughput: 0: 1832.3, 1: 1849.8. Samples: 32484754. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 06:24:13,754][130385] Avg episode reward: [(0, '67.990'), (1, '83.000')] [2023-10-08 06:24:14,112][00611] Updated weights for policy 0, policy_version 63262 (0.0008) [2023-10-08 06:24:14,735][00612] Updated weights for policy 1, policy_version 63620 (0.0008) [2023-10-08 06:24:15,105][00612] Updated weights for policy 1, policy_version 63630 (0.0008) [2023-10-08 06:24:15,473][00612] Updated weights for policy 1, policy_version 63640 (0.0009) [2023-10-08 06:24:17,921][00611] Updated weights for policy 0, policy_version 63272 (0.0008) [2023-10-08 06:24:18,287][00611] Updated weights for policy 0, policy_version 63282 (0.0010) [2023-10-08 06:24:18,668][00611] Updated weights for policy 0, policy_version 63292 (0.0009) [2023-10-08 06:24:18,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 129957888. Throughput: 0: 1830.0, 1: 1853.0. Samples: 32507156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:24:18,754][130385] Avg episode reward: [(0, '67.860'), (1, '83.090')] [2023-10-08 06:24:18,764][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000063648_65175552.pth... [2023-10-08 06:24:18,792][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000061920_63406080.pth [2023-10-08 06:24:18,811][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000063296_64815104.pth... [2023-10-08 06:24:18,841][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000061568_63045632.pth [2023-10-08 06:24:19,118][00612] Updated weights for policy 1, policy_version 63650 (0.0009) [2023-10-08 06:24:19,482][00612] Updated weights for policy 1, policy_version 63660 (0.0007) [2023-10-08 06:24:19,853][00612] Updated weights for policy 1, policy_version 63670 (0.0008) [2023-10-08 06:24:20,225][00612] Updated weights for policy 1, policy_version 63680 (0.0009) [2023-10-08 06:24:22,193][00611] Updated weights for policy 0, policy_version 63302 (0.0009) [2023-10-08 06:24:22,573][00611] Updated weights for policy 0, policy_version 63312 (0.0007) [2023-10-08 06:24:22,948][00611] Updated weights for policy 0, policy_version 63322 (0.0008) [2023-10-08 06:24:23,668][00612] Updated weights for policy 1, policy_version 63690 (0.0008) [2023-10-08 06:24:23,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 130056192. Throughput: 0: 1833.5, 1: 1854.9. Samples: 32518050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:24:23,754][130385] Avg episode reward: [(0, '66.420'), (1, '87.890')] [2023-10-08 06:24:24,031][00612] Updated weights for policy 1, policy_version 63700 (0.0008) [2023-10-08 06:24:24,393][00612] Updated weights for policy 1, policy_version 63710 (0.0009) [2023-10-08 06:24:24,465][00425] Saving new best policy, reward=87.890! [2023-10-08 06:24:26,444][00611] Updated weights for policy 0, policy_version 63332 (0.0008) [2023-10-08 06:24:26,820][00611] Updated weights for policy 0, policy_version 63342 (0.0009) [2023-10-08 06:24:27,192][00611] Updated weights for policy 0, policy_version 63352 (0.0007) [2023-10-08 06:24:28,109][00612] Updated weights for policy 1, policy_version 63720 (0.0009) [2023-10-08 06:24:28,473][00612] Updated weights for policy 1, policy_version 63730 (0.0007) [2023-10-08 06:24:28,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 130121728. Throughput: 0: 1827.7, 1: 1851.2. Samples: 32540040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:24:28,755][130385] Avg episode reward: [(0, '66.970'), (1, '86.720')] [2023-10-08 06:24:28,841][00612] Updated weights for policy 1, policy_version 63740 (0.0008) [2023-10-08 06:24:30,850][00611] Updated weights for policy 0, policy_version 63362 (0.0008) [2023-10-08 06:24:31,218][00611] Updated weights for policy 0, policy_version 63372 (0.0010) [2023-10-08 06:24:31,589][00611] Updated weights for policy 0, policy_version 63382 (0.0009) [2023-10-08 06:24:31,946][00611] Updated weights for policy 0, policy_version 63392 (0.0008) [2023-10-08 06:24:32,593][00612] Updated weights for policy 1, policy_version 63750 (0.0009) [2023-10-08 06:24:32,959][00612] Updated weights for policy 1, policy_version 63760 (0.0007) [2023-10-08 06:24:33,335][00612] Updated weights for policy 1, policy_version 63770 (0.0007) [2023-10-08 06:24:33,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 130220032. Throughput: 0: 1834.6, 1: 1839.2. Samples: 32561786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:24:33,754][130385] Avg episode reward: [(0, '66.870'), (1, '85.820')] [2023-10-08 06:24:35,561][00611] Updated weights for policy 0, policy_version 63402 (0.0011) [2023-10-08 06:24:35,938][00611] Updated weights for policy 0, policy_version 63412 (0.0008) [2023-10-08 06:24:36,302][00611] Updated weights for policy 0, policy_version 63422 (0.0008) [2023-10-08 06:24:36,767][00612] Updated weights for policy 1, policy_version 63780 (0.0008) [2023-10-08 06:24:37,143][00612] Updated weights for policy 1, policy_version 63790 (0.0011) [2023-10-08 06:24:37,504][00612] Updated weights for policy 1, policy_version 63800 (0.0009) [2023-10-08 06:24:38,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 130285568. Throughput: 0: 1827.6, 1: 1863.8. Samples: 32573490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:24:38,754][130385] Avg episode reward: [(0, '64.760'), (1, '82.410')] [2023-10-08 06:24:39,949][00611] Updated weights for policy 0, policy_version 63432 (0.0009) [2023-10-08 06:24:40,319][00611] Updated weights for policy 0, policy_version 63442 (0.0010) [2023-10-08 06:24:40,694][00611] Updated weights for policy 0, policy_version 63452 (0.0010) [2023-10-08 06:24:41,165][00612] Updated weights for policy 1, policy_version 63810 (0.0012) [2023-10-08 06:24:41,535][00612] Updated weights for policy 1, policy_version 63820 (0.0007) [2023-10-08 06:24:41,901][00612] Updated weights for policy 1, policy_version 63830 (0.0009) [2023-10-08 06:24:42,270][00612] Updated weights for policy 1, policy_version 63840 (0.0007) [2023-10-08 06:24:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 130351104. Throughput: 0: 1846.4, 1: 1841.3. Samples: 32595136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:24:43,754][130385] Avg episode reward: [(0, '62.060'), (1, '79.560')] [2023-10-08 06:24:44,323][00611] Updated weights for policy 0, policy_version 63462 (0.0009) [2023-10-08 06:24:44,691][00611] Updated weights for policy 0, policy_version 63472 (0.0009) [2023-10-08 06:24:45,064][00611] Updated weights for policy 0, policy_version 63482 (0.0007) [2023-10-08 06:24:45,984][00612] Updated weights for policy 1, policy_version 63850 (0.0008) [2023-10-08 06:24:46,359][00612] Updated weights for policy 1, policy_version 63860 (0.0007) [2023-10-08 06:24:46,719][00612] Updated weights for policy 1, policy_version 63870 (0.0008) [2023-10-08 06:24:48,718][00611] Updated weights for policy 0, policy_version 63492 (0.0009) [2023-10-08 06:24:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 130416640. Throughput: 0: 1845.1, 1: 1862.9. Samples: 32618248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:24:48,755][130385] Avg episode reward: [(0, '65.820'), (1, '76.440')] [2023-10-08 06:24:49,104][00611] Updated weights for policy 0, policy_version 63502 (0.0009) [2023-10-08 06:24:49,477][00611] Updated weights for policy 0, policy_version 63512 (0.0009) [2023-10-08 06:24:50,495][00612] Updated weights for policy 1, policy_version 63880 (0.0010) [2023-10-08 06:24:50,872][00612] Updated weights for policy 1, policy_version 63890 (0.0010) [2023-10-08 06:24:51,243][00612] Updated weights for policy 1, policy_version 63900 (0.0009) [2023-10-08 06:24:53,214][00611] Updated weights for policy 0, policy_version 63522 (0.0008) [2023-10-08 06:24:53,590][00611] Updated weights for policy 0, policy_version 63532 (0.0009) [2023-10-08 06:24:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14773.4). Total num frames: 130482176. Throughput: 0: 1846.6, 1: 1843.2. Samples: 32628240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:24:53,754][130385] Avg episode reward: [(0, '63.350'), (1, '76.100')] [2023-10-08 06:24:53,953][00611] Updated weights for policy 0, policy_version 63542 (0.0007) [2023-10-08 06:24:54,328][00611] Updated weights for policy 0, policy_version 63552 (0.0007) [2023-10-08 06:24:54,709][00612] Updated weights for policy 1, policy_version 63910 (0.0008) [2023-10-08 06:24:55,070][00612] Updated weights for policy 1, policy_version 63920 (0.0009) [2023-10-08 06:24:55,440][00612] Updated weights for policy 1, policy_version 63930 (0.0009) [2023-10-08 06:24:57,939][00611] Updated weights for policy 0, policy_version 63562 (0.0008) [2023-10-08 06:24:58,311][00611] Updated weights for policy 0, policy_version 63572 (0.0009) [2023-10-08 06:24:58,681][00611] Updated weights for policy 0, policy_version 63582 (0.0007) [2023-10-08 06:24:58,755][130385] Fps is (10 sec: 16382.0, 60 sec: 14745.3, 300 sec: 14884.4). Total num frames: 130580480. Throughput: 0: 1839.2, 1: 1855.2. Samples: 32651010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:24:58,756][130385] Avg episode reward: [(0, '60.710'), (1, '79.320')] [2023-10-08 06:24:59,035][00612] Updated weights for policy 1, policy_version 63940 (0.0010) [2023-10-08 06:24:59,411][00612] Updated weights for policy 1, policy_version 63950 (0.0008) [2023-10-08 06:24:59,771][00612] Updated weights for policy 1, policy_version 63960 (0.0007) [2023-10-08 06:25:02,258][00611] Updated weights for policy 0, policy_version 63592 (0.0007) [2023-10-08 06:25:02,635][00611] Updated weights for policy 0, policy_version 63602 (0.0008) [2023-10-08 06:25:03,004][00611] Updated weights for policy 0, policy_version 63612 (0.0008) [2023-10-08 06:25:03,419][00612] Updated weights for policy 1, policy_version 63970 (0.0008) [2023-10-08 06:25:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 130646016. Throughput: 0: 1830.0, 1: 1852.9. Samples: 32672888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:25:03,754][130385] Avg episode reward: [(0, '59.690'), (1, '77.120')] [2023-10-08 06:25:03,789][00612] Updated weights for policy 1, policy_version 63980 (0.0007) [2023-10-08 06:25:04,157][00612] Updated weights for policy 1, policy_version 63990 (0.0010) [2023-10-08 06:25:04,515][00612] Updated weights for policy 1, policy_version 64000 (0.0009) [2023-10-08 06:25:06,583][00611] Updated weights for policy 0, policy_version 63622 (0.0009) [2023-10-08 06:25:06,942][00611] Updated weights for policy 0, policy_version 63632 (0.0011) [2023-10-08 06:25:07,309][00611] Updated weights for policy 0, policy_version 63642 (0.0011) [2023-10-08 06:25:08,025][00612] Updated weights for policy 1, policy_version 64010 (0.0007) [2023-10-08 06:25:08,391][00612] Updated weights for policy 1, policy_version 64020 (0.0009) [2023-10-08 06:25:08,754][130385] Fps is (10 sec: 13108.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 130711552. Throughput: 0: 1846.5, 1: 1854.5. Samples: 32684598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:25:08,755][130385] Avg episode reward: [(0, '59.900'), (1, '75.730')] [2023-10-08 06:25:08,761][00612] Updated weights for policy 1, policy_version 64030 (0.0007) [2023-10-08 06:25:10,858][00611] Updated weights for policy 0, policy_version 63652 (0.0009) [2023-10-08 06:25:11,224][00611] Updated weights for policy 0, policy_version 63662 (0.0008) [2023-10-08 06:25:11,596][00611] Updated weights for policy 0, policy_version 63672 (0.0007) [2023-10-08 06:25:12,396][00612] Updated weights for policy 1, policy_version 64040 (0.0009) [2023-10-08 06:25:12,769][00612] Updated weights for policy 1, policy_version 64050 (0.0009) [2023-10-08 06:25:13,141][00612] Updated weights for policy 1, policy_version 64060 (0.0009) [2023-10-08 06:25:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 130809856. Throughput: 0: 1840.4, 1: 1853.7. Samples: 32706274. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:25:13,754][130385] Avg episode reward: [(0, '57.880'), (1, '73.620')] [2023-10-08 06:25:15,038][00611] Updated weights for policy 0, policy_version 63682 (0.0009) [2023-10-08 06:25:15,425][00611] Updated weights for policy 0, policy_version 63692 (0.0008) [2023-10-08 06:25:15,794][00611] Updated weights for policy 0, policy_version 63702 (0.0008) [2023-10-08 06:25:16,167][00611] Updated weights for policy 0, policy_version 63712 (0.0007) [2023-10-08 06:25:16,842][00612] Updated weights for policy 1, policy_version 64070 (0.0008) [2023-10-08 06:25:17,200][00612] Updated weights for policy 1, policy_version 64080 (0.0007) [2023-10-08 06:25:17,572][00612] Updated weights for policy 1, policy_version 64090 (0.0007) [2023-10-08 06:25:18,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 130875392. Throughput: 0: 1858.4, 1: 1843.4. Samples: 32728368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:25:18,755][130385] Avg episode reward: [(0, '59.920'), (1, '74.730')] [2023-10-08 06:25:19,783][00611] Updated weights for policy 0, policy_version 63722 (0.0007) [2023-10-08 06:25:20,157][00611] Updated weights for policy 0, policy_version 63732 (0.0007) [2023-10-08 06:25:20,535][00611] Updated weights for policy 0, policy_version 63742 (0.0009) [2023-10-08 06:25:21,126][00612] Updated weights for policy 1, policy_version 64100 (0.0009) [2023-10-08 06:25:21,497][00612] Updated weights for policy 1, policy_version 64110 (0.0008) [2023-10-08 06:25:21,865][00612] Updated weights for policy 1, policy_version 64120 (0.0008) [2023-10-08 06:25:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 130940928. Throughput: 0: 1848.8, 1: 1845.5. Samples: 32739736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:25:23,754][130385] Avg episode reward: [(0, '57.110'), (1, '71.830')] [2023-10-08 06:25:24,019][00611] Updated weights for policy 0, policy_version 63752 (0.0007) [2023-10-08 06:25:24,386][00611] Updated weights for policy 0, policy_version 63762 (0.0007) [2023-10-08 06:25:24,762][00611] Updated weights for policy 0, policy_version 63772 (0.0007) [2023-10-08 06:25:25,405][00612] Updated weights for policy 1, policy_version 64130 (0.0008) [2023-10-08 06:25:25,777][00612] Updated weights for policy 1, policy_version 64140 (0.0010) [2023-10-08 06:25:26,145][00612] Updated weights for policy 1, policy_version 64150 (0.0007) [2023-10-08 06:25:26,518][00612] Updated weights for policy 1, policy_version 64160 (0.0008) [2023-10-08 06:25:28,318][00611] Updated weights for policy 0, policy_version 63782 (0.0007) [2023-10-08 06:25:28,698][00611] Updated weights for policy 0, policy_version 63792 (0.0007) [2023-10-08 06:25:28,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 131006464. Throughput: 0: 1861.5, 1: 1850.5. Samples: 32762174. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:25:28,754][130385] Avg episode reward: [(0, '57.390'), (1, '67.010')] [2023-10-08 06:25:29,069][00611] Updated weights for policy 0, policy_version 63802 (0.0008) [2023-10-08 06:25:30,145][00612] Updated weights for policy 1, policy_version 64170 (0.0008) [2023-10-08 06:25:30,516][00612] Updated weights for policy 1, policy_version 64180 (0.0007) [2023-10-08 06:25:30,875][00612] Updated weights for policy 1, policy_version 64190 (0.0008) [2023-10-08 06:25:32,702][00611] Updated weights for policy 0, policy_version 63812 (0.0008) [2023-10-08 06:25:33,069][00611] Updated weights for policy 0, policy_version 63822 (0.0007) [2023-10-08 06:25:33,436][00611] Updated weights for policy 0, policy_version 63832 (0.0009) [2023-10-08 06:25:33,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14884.5). Total num frames: 131104768. Throughput: 0: 1847.4, 1: 1853.2. Samples: 32784774. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:25:33,754][130385] Avg episode reward: [(0, '58.450'), (1, '65.780')] [2023-10-08 06:25:34,595][00612] Updated weights for policy 1, policy_version 64200 (0.0008) [2023-10-08 06:25:34,966][00612] Updated weights for policy 1, policy_version 64210 (0.0007) [2023-10-08 06:25:35,336][00612] Updated weights for policy 1, policy_version 64220 (0.0008) [2023-10-08 06:25:37,102][00611] Updated weights for policy 0, policy_version 63842 (0.0009) [2023-10-08 06:25:37,488][00611] Updated weights for policy 0, policy_version 63852 (0.0008) [2023-10-08 06:25:37,860][00611] Updated weights for policy 0, policy_version 63862 (0.0007) [2023-10-08 06:25:38,227][00611] Updated weights for policy 0, policy_version 63872 (0.0009) [2023-10-08 06:25:38,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14884.4). Total num frames: 131170304. Throughput: 0: 1870.2, 1: 1849.6. Samples: 32795634. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:25:38,754][130385] Avg episode reward: [(0, '57.330'), (1, '62.980')] [2023-10-08 06:25:38,860][00612] Updated weights for policy 1, policy_version 64230 (0.0010) [2023-10-08 06:25:39,233][00612] Updated weights for policy 1, policy_version 64240 (0.0008) [2023-10-08 06:25:39,600][00612] Updated weights for policy 1, policy_version 64250 (0.0007) [2023-10-08 06:25:41,869][00611] Updated weights for policy 0, policy_version 63882 (0.0009) [2023-10-08 06:25:42,245][00611] Updated weights for policy 0, policy_version 63892 (0.0009) [2023-10-08 06:25:42,623][00611] Updated weights for policy 0, policy_version 63902 (0.0008) [2023-10-08 06:25:43,262][00612] Updated weights for policy 1, policy_version 64260 (0.0008) [2023-10-08 06:25:43,624][00612] Updated weights for policy 1, policy_version 64270 (0.0009) [2023-10-08 06:25:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 131235840. Throughput: 0: 1852.1, 1: 1857.0. Samples: 32817912. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:25:43,754][130385] Avg episode reward: [(0, '59.720'), (1, '65.140')] [2023-10-08 06:25:43,997][00612] Updated weights for policy 1, policy_version 64280 (0.0010) [2023-10-08 06:25:46,292][00611] Updated weights for policy 0, policy_version 63912 (0.0009) [2023-10-08 06:25:46,661][00611] Updated weights for policy 0, policy_version 63922 (0.0009) [2023-10-08 06:25:47,038][00611] Updated weights for policy 0, policy_version 63932 (0.0008) [2023-10-08 06:25:47,647][00612] Updated weights for policy 1, policy_version 64290 (0.0010) [2023-10-08 06:25:48,010][00612] Updated weights for policy 1, policy_version 64300 (0.0009) [2023-10-08 06:25:48,380][00612] Updated weights for policy 1, policy_version 64310 (0.0010) [2023-10-08 06:25:48,743][00612] Updated weights for policy 1, policy_version 64320 (0.0008) [2023-10-08 06:25:48,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14884.5). Total num frames: 131334144. Throughput: 0: 1862.7, 1: 1836.2. Samples: 32839340. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:25:48,754][130385] Avg episode reward: [(0, '61.360'), (1, '69.000')] [2023-10-08 06:25:50,770][00611] Updated weights for policy 0, policy_version 63942 (0.0009) [2023-10-08 06:25:51,136][00611] Updated weights for policy 0, policy_version 63952 (0.0010) [2023-10-08 06:25:51,513][00611] Updated weights for policy 0, policy_version 63962 (0.0011) [2023-10-08 06:25:52,400][00612] Updated weights for policy 1, policy_version 64330 (0.0007) [2023-10-08 06:25:52,766][00612] Updated weights for policy 1, policy_version 64340 (0.0007) [2023-10-08 06:25:53,139][00612] Updated weights for policy 1, policy_version 64350 (0.0009) [2023-10-08 06:25:53,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 131399680. Throughput: 0: 1842.2, 1: 1849.7. Samples: 32850736. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:25:53,755][130385] Avg episode reward: [(0, '61.260'), (1, '73.320')] [2023-10-08 06:25:55,018][00611] Updated weights for policy 0, policy_version 63972 (0.0008) [2023-10-08 06:25:55,389][00611] Updated weights for policy 0, policy_version 63982 (0.0008) [2023-10-08 06:25:55,758][00611] Updated weights for policy 0, policy_version 63992 (0.0010) [2023-10-08 06:25:56,711][00612] Updated weights for policy 1, policy_version 64360 (0.0009) [2023-10-08 06:25:57,080][00612] Updated weights for policy 1, policy_version 64370 (0.0008) [2023-10-08 06:25:57,451][00612] Updated weights for policy 1, policy_version 64380 (0.0007) [2023-10-08 06:25:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.9, 300 sec: 14773.4). Total num frames: 131465216. Throughput: 0: 1859.5, 1: 1834.7. Samples: 32872510. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:25:58,754][130385] Avg episode reward: [(0, '63.440'), (1, '74.320')] [2023-10-08 06:25:59,481][00611] Updated weights for policy 0, policy_version 64002 (0.0007) [2023-10-08 06:25:59,855][00611] Updated weights for policy 0, policy_version 64012 (0.0008) [2023-10-08 06:26:00,227][00611] Updated weights for policy 0, policy_version 64022 (0.0010) [2023-10-08 06:26:00,595][00611] Updated weights for policy 0, policy_version 64032 (0.0011) [2023-10-08 06:26:00,979][00612] Updated weights for policy 1, policy_version 64390 (0.0007) [2023-10-08 06:26:01,346][00612] Updated weights for policy 1, policy_version 64400 (0.0008) [2023-10-08 06:26:01,711][00612] Updated weights for policy 1, policy_version 64410 (0.0008) [2023-10-08 06:26:03,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 131530752. Throughput: 0: 1851.4, 1: 1858.1. Samples: 32895296. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:26:03,754][130385] Avg episode reward: [(0, '60.140'), (1, '73.860')] [2023-10-08 06:26:04,254][00611] Updated weights for policy 0, policy_version 64042 (0.0009) [2023-10-08 06:26:04,621][00611] Updated weights for policy 0, policy_version 64052 (0.0007) [2023-10-08 06:26:04,988][00611] Updated weights for policy 0, policy_version 64062 (0.0007) [2023-10-08 06:26:05,317][00612] Updated weights for policy 1, policy_version 64420 (0.0011) [2023-10-08 06:26:05,690][00612] Updated weights for policy 1, policy_version 64430 (0.0010) [2023-10-08 06:26:06,053][00612] Updated weights for policy 1, policy_version 64440 (0.0010) [2023-10-08 06:26:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 131596288. Throughput: 0: 1849.6, 1: 1835.5. Samples: 32905566. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:26:08,754][130385] Avg episode reward: [(0, '63.600'), (1, '77.110')] [2023-10-08 06:26:08,793][00611] Updated weights for policy 0, policy_version 64072 (0.0009) [2023-10-08 06:26:09,174][00611] Updated weights for policy 0, policy_version 64082 (0.0010) [2023-10-08 06:26:09,533][00611] Updated weights for policy 0, policy_version 64092 (0.0010) [2023-10-08 06:26:09,534][00612] Updated weights for policy 1, policy_version 64450 (0.0007) [2023-10-08 06:26:09,904][00612] Updated weights for policy 1, policy_version 64460 (0.0008) [2023-10-08 06:26:10,271][00612] Updated weights for policy 1, policy_version 64470 (0.0008) [2023-10-08 06:26:10,638][00612] Updated weights for policy 1, policy_version 64480 (0.0010) [2023-10-08 06:26:13,196][00611] Updated weights for policy 0, policy_version 64102 (0.0008) [2023-10-08 06:26:13,565][00611] Updated weights for policy 0, policy_version 64112 (0.0009) [2023-10-08 06:26:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14773.4). Total num frames: 131661824. Throughput: 0: 1838.6, 1: 1855.8. Samples: 32928424. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:26:13,754][130385] Avg episode reward: [(0, '63.620'), (1, '73.280')] [2023-10-08 06:26:13,931][00611] Updated weights for policy 0, policy_version 64122 (0.0009) [2023-10-08 06:26:14,371][00612] Updated weights for policy 1, policy_version 64490 (0.0010) [2023-10-08 06:26:14,746][00612] Updated weights for policy 1, policy_version 64500 (0.0011) [2023-10-08 06:26:15,109][00612] Updated weights for policy 1, policy_version 64510 (0.0010) [2023-10-08 06:26:17,404][00611] Updated weights for policy 0, policy_version 64132 (0.0007) [2023-10-08 06:26:17,775][00611] Updated weights for policy 0, policy_version 64142 (0.0007) [2023-10-08 06:26:18,156][00611] Updated weights for policy 0, policy_version 64152 (0.0007) [2023-10-08 06:26:18,752][00612] Updated weights for policy 1, policy_version 64520 (0.0008) [2023-10-08 06:26:18,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 131760128. Throughput: 0: 1835.0, 1: 1858.7. Samples: 32950988. Policy #0 lag: (min: 26.0, avg: 36.7, max: 58.0) [2023-10-08 06:26:18,755][130385] Avg episode reward: [(0, '63.490'), (1, '77.340')] [2023-10-08 06:26:18,766][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000064160_65699840.pth... [2023-10-08 06:26:18,805][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000062432_63930368.pth [2023-10-08 06:26:19,121][00612] Updated weights for policy 1, policy_version 64530 (0.0008) [2023-10-08 06:26:19,493][00612] Updated weights for policy 1, policy_version 64540 (0.0009) [2023-10-08 06:26:19,634][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000064544_66093056.pth... [2023-10-08 06:26:19,674][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000062784_64290816.pth [2023-10-08 06:26:21,853][00611] Updated weights for policy 0, policy_version 64162 (0.0011) [2023-10-08 06:26:22,263][00611] Updated weights for policy 0, policy_version 64172 (0.0008) [2023-10-08 06:26:22,641][00611] Updated weights for policy 0, policy_version 64182 (0.0007) [2023-10-08 06:26:23,005][00611] Updated weights for policy 0, policy_version 64192 (0.0008) [2023-10-08 06:26:23,067][00612] Updated weights for policy 1, policy_version 64550 (0.0009) [2023-10-08 06:26:23,447][00612] Updated weights for policy 1, policy_version 64560 (0.0008) [2023-10-08 06:26:23,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 131825664. Throughput: 0: 1840.2, 1: 1855.6. Samples: 32961942. Policy #0 lag: (min: 26.0, avg: 36.7, max: 58.0) [2023-10-08 06:26:23,754][130385] Avg episode reward: [(0, '66.110'), (1, '78.740')] [2023-10-08 06:26:23,821][00612] Updated weights for policy 1, policy_version 64570 (0.0009) [2023-10-08 06:26:26,613][00611] Updated weights for policy 0, policy_version 64202 (0.0009) [2023-10-08 06:26:26,977][00611] Updated weights for policy 0, policy_version 64212 (0.0009) [2023-10-08 06:26:27,346][00611] Updated weights for policy 0, policy_version 64222 (0.0008) [2023-10-08 06:26:27,412][00612] Updated weights for policy 1, policy_version 64580 (0.0008) [2023-10-08 06:26:27,777][00612] Updated weights for policy 1, policy_version 64590 (0.0007) [2023-10-08 06:26:28,140][00612] Updated weights for policy 1, policy_version 64600 (0.0010) [2023-10-08 06:26:28,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14884.5). Total num frames: 131923968. Throughput: 0: 1832.5, 1: 1856.0. Samples: 32983894. Policy #0 lag: (min: 26.0, avg: 36.7, max: 58.0) [2023-10-08 06:26:28,754][130385] Avg episode reward: [(0, '67.390'), (1, '81.210')] [2023-10-08 06:26:30,880][00611] Updated weights for policy 0, policy_version 64232 (0.0009) [2023-10-08 06:26:31,256][00611] Updated weights for policy 0, policy_version 64242 (0.0009) [2023-10-08 06:26:31,618][00611] Updated weights for policy 0, policy_version 64252 (0.0008) [2023-10-08 06:26:31,785][00612] Updated weights for policy 1, policy_version 64610 (0.0009) [2023-10-08 06:26:32,144][00612] Updated weights for policy 1, policy_version 64620 (0.0009) [2023-10-08 06:26:32,516][00612] Updated weights for policy 1, policy_version 64630 (0.0007) [2023-10-08 06:26:32,879][00612] Updated weights for policy 1, policy_version 64640 (0.0008) [2023-10-08 06:26:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 131989504. Throughput: 0: 1851.0, 1: 1837.6. Samples: 33005328. Policy #0 lag: (min: 26.0, avg: 36.7, max: 58.0) [2023-10-08 06:26:33,754][130385] Avg episode reward: [(0, '70.030'), (1, '79.740')] [2023-10-08 06:26:35,072][00611] Updated weights for policy 0, policy_version 64262 (0.0009) [2023-10-08 06:26:35,437][00611] Updated weights for policy 0, policy_version 64272 (0.0008) [2023-10-08 06:26:35,816][00611] Updated weights for policy 0, policy_version 64282 (0.0008) [2023-10-08 06:26:36,556][00612] Updated weights for policy 1, policy_version 64650 (0.0008) [2023-10-08 06:26:36,928][00612] Updated weights for policy 1, policy_version 64660 (0.0007) [2023-10-08 06:26:37,292][00612] Updated weights for policy 1, policy_version 64670 (0.0007) [2023-10-08 06:26:38,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 132055040. Throughput: 0: 1835.5, 1: 1857.4. Samples: 33016918. Policy #0 lag: (min: 26.0, avg: 36.7, max: 58.0) [2023-10-08 06:26:38,755][130385] Avg episode reward: [(0, '70.680'), (1, '80.420')] [2023-10-08 06:26:39,659][00611] Updated weights for policy 0, policy_version 64292 (0.0008) [2023-10-08 06:26:40,026][00611] Updated weights for policy 0, policy_version 64302 (0.0007) [2023-10-08 06:26:40,405][00611] Updated weights for policy 0, policy_version 64312 (0.0007) [2023-10-08 06:26:40,825][00612] Updated weights for policy 1, policy_version 64680 (0.0010) [2023-10-08 06:26:41,189][00612] Updated weights for policy 1, policy_version 64690 (0.0007) [2023-10-08 06:26:41,557][00612] Updated weights for policy 1, policy_version 64700 (0.0009) [2023-10-08 06:26:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 132120576. Throughput: 0: 1843.3, 1: 1840.2. Samples: 33038268. Policy #0 lag: (min: 26.0, avg: 36.7, max: 58.0) [2023-10-08 06:26:43,754][130385] Avg episode reward: [(0, '68.790'), (1, '81.420')] [2023-10-08 06:26:44,068][00611] Updated weights for policy 0, policy_version 64322 (0.0009) [2023-10-08 06:26:44,439][00611] Updated weights for policy 0, policy_version 64332 (0.0008) [2023-10-08 06:26:44,813][00611] Updated weights for policy 0, policy_version 64342 (0.0008) [2023-10-08 06:26:45,155][00612] Updated weights for policy 1, policy_version 64710 (0.0008) [2023-10-08 06:26:45,182][00611] Updated weights for policy 0, policy_version 64352 (0.0007) [2023-10-08 06:26:45,524][00612] Updated weights for policy 1, policy_version 64720 (0.0008) [2023-10-08 06:26:45,896][00612] Updated weights for policy 1, policy_version 64730 (0.0008) [2023-10-08 06:26:48,750][00611] Updated weights for policy 0, policy_version 64362 (0.0009) [2023-10-08 06:26:48,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14773.4). Total num frames: 132186112. Throughput: 0: 1844.7, 1: 1850.7. Samples: 33061588. Policy #0 lag: (min: 26.0, avg: 36.7, max: 58.0) [2023-10-08 06:26:48,754][130385] Avg episode reward: [(0, '67.700'), (1, '81.630')] [2023-10-08 06:26:49,121][00611] Updated weights for policy 0, policy_version 64372 (0.0010) [2023-10-08 06:26:49,494][00611] Updated weights for policy 0, policy_version 64382 (0.0010) [2023-10-08 06:26:49,584][00612] Updated weights for policy 1, policy_version 64740 (0.0008) [2023-10-08 06:26:49,952][00612] Updated weights for policy 1, policy_version 64750 (0.0007) [2023-10-08 06:26:50,325][00612] Updated weights for policy 1, policy_version 64760 (0.0010) [2023-10-08 06:26:53,128][00611] Updated weights for policy 0, policy_version 64392 (0.0008) [2023-10-08 06:26:53,504][00611] Updated weights for policy 0, policy_version 64402 (0.0010) [2023-10-08 06:26:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 132251648. Throughput: 0: 1844.3, 1: 1844.2. Samples: 33071550. Policy #0 lag: (min: 26.0, avg: 36.7, max: 58.0) [2023-10-08 06:26:53,754][130385] Avg episode reward: [(0, '66.800'), (1, '85.680')] [2023-10-08 06:26:53,868][00611] Updated weights for policy 0, policy_version 64412 (0.0009) [2023-10-08 06:26:53,913][00612] Updated weights for policy 1, policy_version 64770 (0.0007) [2023-10-08 06:26:54,268][00612] Updated weights for policy 1, policy_version 64780 (0.0009) [2023-10-08 06:26:54,636][00612] Updated weights for policy 1, policy_version 64790 (0.0008) [2023-10-08 06:26:55,006][00612] Updated weights for policy 1, policy_version 64800 (0.0007) [2023-10-08 06:26:57,585][00611] Updated weights for policy 0, policy_version 64422 (0.0008) [2023-10-08 06:26:57,942][00611] Updated weights for policy 0, policy_version 64432 (0.0009) [2023-10-08 06:26:58,310][00611] Updated weights for policy 0, policy_version 64442 (0.0010) [2023-10-08 06:26:58,727][00612] Updated weights for policy 1, policy_version 64810 (0.0007) [2023-10-08 06:26:58,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 132349952. Throughput: 0: 1847.1, 1: 1843.3. Samples: 33094492. Policy #0 lag: (min: 26.0, avg: 36.7, max: 58.0) [2023-10-08 06:26:58,755][130385] Avg episode reward: [(0, '66.630'), (1, '87.180')] [2023-10-08 06:26:59,094][00612] Updated weights for policy 1, policy_version 64820 (0.0007) [2023-10-08 06:26:59,464][00612] Updated weights for policy 1, policy_version 64830 (0.0007) [2023-10-08 06:27:01,891][00611] Updated weights for policy 0, policy_version 64452 (0.0008) [2023-10-08 06:27:02,255][00611] Updated weights for policy 0, policy_version 64462 (0.0007) [2023-10-08 06:27:02,634][00611] Updated weights for policy 0, policy_version 64472 (0.0008) [2023-10-08 06:27:03,171][00612] Updated weights for policy 1, policy_version 64840 (0.0009) [2023-10-08 06:27:03,548][00612] Updated weights for policy 1, policy_version 64850 (0.0008) [2023-10-08 06:27:03,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 132415488. Throughput: 0: 1825.0, 1: 1832.4. Samples: 33115572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:27:03,755][130385] Avg episode reward: [(0, '67.820'), (1, '87.960')] [2023-10-08 06:27:03,908][00612] Updated weights for policy 1, policy_version 64860 (0.0010) [2023-10-08 06:27:04,054][00425] Saving new best policy, reward=87.960! [2023-10-08 06:27:06,139][00611] Updated weights for policy 0, policy_version 64482 (0.0007) [2023-10-08 06:27:06,506][00611] Updated weights for policy 0, policy_version 64492 (0.0009) [2023-10-08 06:27:06,874][00611] Updated weights for policy 0, policy_version 64502 (0.0009) [2023-10-08 06:27:07,248][00611] Updated weights for policy 0, policy_version 64512 (0.0010) [2023-10-08 06:27:07,561][00612] Updated weights for policy 1, policy_version 64870 (0.0009) [2023-10-08 06:27:07,933][00612] Updated weights for policy 1, policy_version 64880 (0.0007) [2023-10-08 06:27:08,295][00612] Updated weights for policy 1, policy_version 64890 (0.0010) [2023-10-08 06:27:08,754][130385] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14884.4). Total num frames: 132513792. Throughput: 0: 1836.3, 1: 1843.3. Samples: 33127524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:27:08,754][130385] Avg episode reward: [(0, '66.190'), (1, '83.590')] [2023-10-08 06:27:10,836][00611] Updated weights for policy 0, policy_version 64522 (0.0010) [2023-10-08 06:27:11,200][00611] Updated weights for policy 0, policy_version 64532 (0.0010) [2023-10-08 06:27:11,568][00611] Updated weights for policy 0, policy_version 64542 (0.0010) [2023-10-08 06:27:11,980][00612] Updated weights for policy 1, policy_version 64900 (0.0010) [2023-10-08 06:27:12,368][00612] Updated weights for policy 1, policy_version 64910 (0.0009) [2023-10-08 06:27:12,733][00612] Updated weights for policy 1, policy_version 64920 (0.0008) [2023-10-08 06:27:13,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 132579328. Throughput: 0: 1835.2, 1: 1829.2. Samples: 33148792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:27:13,754][130385] Avg episode reward: [(0, '63.270'), (1, '85.370')] [2023-10-08 06:27:15,048][00611] Updated weights for policy 0, policy_version 64552 (0.0009) [2023-10-08 06:27:15,417][00611] Updated weights for policy 0, policy_version 64562 (0.0009) [2023-10-08 06:27:15,790][00611] Updated weights for policy 0, policy_version 64572 (0.0010) [2023-10-08 06:27:16,457][00612] Updated weights for policy 1, policy_version 64930 (0.0008) [2023-10-08 06:27:16,818][00612] Updated weights for policy 1, policy_version 64940 (0.0010) [2023-10-08 06:27:17,188][00612] Updated weights for policy 1, policy_version 64950 (0.0008) [2023-10-08 06:27:17,557][00612] Updated weights for policy 1, policy_version 64960 (0.0008) [2023-10-08 06:27:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 132644864. Throughput: 0: 1847.6, 1: 1837.8. Samples: 33171174. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:27:18,754][130385] Avg episode reward: [(0, '65.360'), (1, '81.910')] [2023-10-08 06:27:19,371][00611] Updated weights for policy 0, policy_version 64582 (0.0008) [2023-10-08 06:27:19,744][00611] Updated weights for policy 0, policy_version 64592 (0.0008) [2023-10-08 06:27:20,118][00611] Updated weights for policy 0, policy_version 64602 (0.0011) [2023-10-08 06:27:21,170][00612] Updated weights for policy 1, policy_version 64970 (0.0009) [2023-10-08 06:27:21,552][00612] Updated weights for policy 1, policy_version 64980 (0.0007) [2023-10-08 06:27:21,917][00612] Updated weights for policy 1, policy_version 64990 (0.0007) [2023-10-08 06:27:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 132710400. Throughput: 0: 1844.8, 1: 1831.2. Samples: 33182336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:27:23,755][130385] Avg episode reward: [(0, '63.750'), (1, '83.510')] [2023-10-08 06:27:23,792][00611] Updated weights for policy 0, policy_version 64612 (0.0008) [2023-10-08 06:27:24,150][00611] Updated weights for policy 0, policy_version 64622 (0.0009) [2023-10-08 06:27:24,530][00611] Updated weights for policy 0, policy_version 64632 (0.0008) [2023-10-08 06:27:25,289][00612] Updated weights for policy 1, policy_version 65000 (0.0010) [2023-10-08 06:27:25,657][00612] Updated weights for policy 1, policy_version 65010 (0.0009) [2023-10-08 06:27:26,031][00612] Updated weights for policy 1, policy_version 65020 (0.0009) [2023-10-08 06:27:28,124][00611] Updated weights for policy 0, policy_version 64642 (0.0009) [2023-10-08 06:27:28,491][00611] Updated weights for policy 0, policy_version 64652 (0.0009) [2023-10-08 06:27:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14773.4). Total num frames: 132775936. Throughput: 0: 1858.7, 1: 1846.0. Samples: 33204978. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:27:28,755][130385] Avg episode reward: [(0, '61.480'), (1, '78.020')] [2023-10-08 06:27:28,863][00611] Updated weights for policy 0, policy_version 64662 (0.0010) [2023-10-08 06:27:29,236][00611] Updated weights for policy 0, policy_version 64672 (0.0009) [2023-10-08 06:27:29,717][00612] Updated weights for policy 1, policy_version 65030 (0.0009) [2023-10-08 06:27:30,087][00612] Updated weights for policy 1, policy_version 65040 (0.0008) [2023-10-08 06:27:30,458][00612] Updated weights for policy 1, policy_version 65050 (0.0008) [2023-10-08 06:27:32,940][00611] Updated weights for policy 0, policy_version 64682 (0.0007) [2023-10-08 06:27:33,307][00611] Updated weights for policy 0, policy_version 64692 (0.0007) [2023-10-08 06:27:33,668][00611] Updated weights for policy 0, policy_version 64702 (0.0008) [2023-10-08 06:27:33,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 132874240. Throughput: 0: 1835.9, 1: 1845.4. Samples: 33227248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:27:33,755][130385] Avg episode reward: [(0, '62.990'), (1, '79.740')] [2023-10-08 06:27:34,135][00612] Updated weights for policy 1, policy_version 65060 (0.0010) [2023-10-08 06:27:34,506][00612] Updated weights for policy 1, policy_version 65070 (0.0011) [2023-10-08 06:27:34,880][00612] Updated weights for policy 1, policy_version 65080 (0.0007) [2023-10-08 06:27:37,411][00611] Updated weights for policy 0, policy_version 64712 (0.0010) [2023-10-08 06:27:37,775][00611] Updated weights for policy 0, policy_version 64722 (0.0007) [2023-10-08 06:27:38,145][00611] Updated weights for policy 0, policy_version 64732 (0.0008) [2023-10-08 06:27:38,427][00612] Updated weights for policy 1, policy_version 65090 (0.0007) [2023-10-08 06:27:38,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 132939776. Throughput: 0: 1860.0, 1: 1844.0. Samples: 33238230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:27:38,754][130385] Avg episode reward: [(0, '66.970'), (1, '77.640')] [2023-10-08 06:27:38,798][00612] Updated weights for policy 1, policy_version 65100 (0.0010) [2023-10-08 06:27:39,160][00612] Updated weights for policy 1, policy_version 65110 (0.0010) [2023-10-08 06:27:39,526][00612] Updated weights for policy 1, policy_version 65120 (0.0009) [2023-10-08 06:27:41,741][00611] Updated weights for policy 0, policy_version 64742 (0.0010) [2023-10-08 06:27:42,114][00611] Updated weights for policy 0, policy_version 64752 (0.0007) [2023-10-08 06:27:42,492][00611] Updated weights for policy 0, policy_version 64762 (0.0007) [2023-10-08 06:27:43,269][00612] Updated weights for policy 1, policy_version 65130 (0.0008) [2023-10-08 06:27:43,638][00612] Updated weights for policy 1, policy_version 65140 (0.0008) [2023-10-08 06:27:43,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 133005312. Throughput: 0: 1843.4, 1: 1854.0. Samples: 33260872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:27:43,754][130385] Avg episode reward: [(0, '64.810'), (1, '76.150')] [2023-10-08 06:27:44,001][00612] Updated weights for policy 1, policy_version 65150 (0.0007) [2023-10-08 06:27:46,219][00611] Updated weights for policy 0, policy_version 64772 (0.0009) [2023-10-08 06:27:46,588][00611] Updated weights for policy 0, policy_version 64782 (0.0007) [2023-10-08 06:27:46,963][00611] Updated weights for policy 0, policy_version 64792 (0.0007) [2023-10-08 06:27:47,527][00612] Updated weights for policy 1, policy_version 65160 (0.0007) [2023-10-08 06:27:47,894][00612] Updated weights for policy 1, policy_version 65170 (0.0007) [2023-10-08 06:27:48,260][00612] Updated weights for policy 1, policy_version 65180 (0.0008) [2023-10-08 06:27:48,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 133103616. Throughput: 0: 1858.9, 1: 1845.6. Samples: 33282274. Policy #0 lag: (min: 31.0, avg: 46.2, max: 63.0) [2023-10-08 06:27:48,754][130385] Avg episode reward: [(0, '66.880'), (1, '77.070')] [2023-10-08 06:27:50,632][00611] Updated weights for policy 0, policy_version 64802 (0.0008) [2023-10-08 06:27:50,996][00611] Updated weights for policy 0, policy_version 64812 (0.0011) [2023-10-08 06:27:51,374][00611] Updated weights for policy 0, policy_version 64822 (0.0011) [2023-10-08 06:27:51,745][00611] Updated weights for policy 0, policy_version 64832 (0.0007) [2023-10-08 06:27:51,803][00612] Updated weights for policy 1, policy_version 65190 (0.0008) [2023-10-08 06:27:52,176][00612] Updated weights for policy 1, policy_version 65200 (0.0007) [2023-10-08 06:27:52,539][00612] Updated weights for policy 1, policy_version 65210 (0.0007) [2023-10-08 06:27:53,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 133169152. Throughput: 0: 1839.2, 1: 1863.0. Samples: 33294126. Policy #0 lag: (min: 31.0, avg: 46.2, max: 63.0) [2023-10-08 06:27:53,754][130385] Avg episode reward: [(0, '69.050'), (1, '78.830')] [2023-10-08 06:27:55,503][00611] Updated weights for policy 0, policy_version 64842 (0.0010) [2023-10-08 06:27:55,877][00611] Updated weights for policy 0, policy_version 64852 (0.0009) [2023-10-08 06:27:56,244][00611] Updated weights for policy 0, policy_version 64862 (0.0009) [2023-10-08 06:27:56,298][00612] Updated weights for policy 1, policy_version 65220 (0.0008) [2023-10-08 06:27:56,675][00612] Updated weights for policy 1, policy_version 65230 (0.0008) [2023-10-08 06:27:57,047][00612] Updated weights for policy 1, policy_version 65240 (0.0010) [2023-10-08 06:27:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 133234688. Throughput: 0: 1853.2, 1: 1843.2. Samples: 33315128. Policy #0 lag: (min: 31.0, avg: 46.2, max: 63.0) [2023-10-08 06:27:58,754][130385] Avg episode reward: [(0, '72.490'), (1, '75.350')] [2023-10-08 06:27:59,824][00611] Updated weights for policy 0, policy_version 64872 (0.0007) [2023-10-08 06:28:00,201][00611] Updated weights for policy 0, policy_version 64882 (0.0009) [2023-10-08 06:28:00,539][00612] Updated weights for policy 1, policy_version 65250 (0.0011) [2023-10-08 06:28:00,570][00611] Updated weights for policy 0, policy_version 64892 (0.0008) [2023-10-08 06:28:00,909][00612] Updated weights for policy 1, policy_version 65260 (0.0009) [2023-10-08 06:28:01,281][00612] Updated weights for policy 1, policy_version 65270 (0.0007) [2023-10-08 06:28:01,655][00612] Updated weights for policy 1, policy_version 65280 (0.0008) [2023-10-08 06:28:03,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 133300224. Throughput: 0: 1838.9, 1: 1863.9. Samples: 33337802. Policy #0 lag: (min: 31.0, avg: 46.2, max: 63.0) [2023-10-08 06:28:03,755][130385] Avg episode reward: [(0, '72.930'), (1, '77.480')] [2023-10-08 06:28:04,194][00611] Updated weights for policy 0, policy_version 64902 (0.0010) [2023-10-08 06:28:04,567][00611] Updated weights for policy 0, policy_version 64912 (0.0008) [2023-10-08 06:28:04,935][00611] Updated weights for policy 0, policy_version 64922 (0.0010) [2023-10-08 06:28:05,309][00612] Updated weights for policy 1, policy_version 65290 (0.0009) [2023-10-08 06:28:05,673][00612] Updated weights for policy 1, policy_version 65300 (0.0010) [2023-10-08 06:28:06,050][00612] Updated weights for policy 1, policy_version 65310 (0.0010) [2023-10-08 06:28:08,679][00611] Updated weights for policy 0, policy_version 64932 (0.0009) [2023-10-08 06:28:08,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 133365760. Throughput: 0: 1840.8, 1: 1838.4. Samples: 33347898. Policy #0 lag: (min: 31.0, avg: 46.2, max: 63.0) [2023-10-08 06:28:08,754][130385] Avg episode reward: [(0, '73.520'), (1, '74.870')] [2023-10-08 06:28:09,044][00611] Updated weights for policy 0, policy_version 64942 (0.0011) [2023-10-08 06:28:09,426][00611] Updated weights for policy 0, policy_version 64952 (0.0009) [2023-10-08 06:28:09,617][00612] Updated weights for policy 1, policy_version 65320 (0.0007) [2023-10-08 06:28:09,985][00612] Updated weights for policy 1, policy_version 65330 (0.0007) [2023-10-08 06:28:10,367][00612] Updated weights for policy 1, policy_version 65340 (0.0008) [2023-10-08 06:28:13,012][00611] Updated weights for policy 0, policy_version 64962 (0.0009) [2023-10-08 06:28:13,386][00611] Updated weights for policy 0, policy_version 64972 (0.0008) [2023-10-08 06:28:13,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 133431296. Throughput: 0: 1829.5, 1: 1860.7. Samples: 33371034. Policy #0 lag: (min: 31.0, avg: 46.2, max: 63.0) [2023-10-08 06:28:13,754][130385] Avg episode reward: [(0, '70.790'), (1, '74.270')] [2023-10-08 06:28:13,760][00611] Updated weights for policy 0, policy_version 64982 (0.0008) [2023-10-08 06:28:13,818][00612] Updated weights for policy 1, policy_version 65350 (0.0007) [2023-10-08 06:28:14,126][00611] Updated weights for policy 0, policy_version 64992 (0.0008) [2023-10-08 06:28:14,183][00612] Updated weights for policy 1, policy_version 65360 (0.0010) [2023-10-08 06:28:14,543][00612] Updated weights for policy 1, policy_version 65370 (0.0009) [2023-10-08 06:28:17,856][00611] Updated weights for policy 0, policy_version 65002 (0.0009) [2023-10-08 06:28:18,089][00612] Updated weights for policy 1, policy_version 65380 (0.0008) [2023-10-08 06:28:18,224][00611] Updated weights for policy 0, policy_version 65012 (0.0007) [2023-10-08 06:28:18,462][00612] Updated weights for policy 1, policy_version 65390 (0.0007) [2023-10-08 06:28:18,591][00611] Updated weights for policy 0, policy_version 65022 (0.0007) [2023-10-08 06:28:18,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 133529600. Throughput: 0: 1830.9, 1: 1857.5. Samples: 33393226. Policy #0 lag: (min: 31.0, avg: 46.2, max: 63.0) [2023-10-08 06:28:18,754][130385] Avg episode reward: [(0, '68.590'), (1, '70.270')] [2023-10-08 06:28:18,766][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000065024_66584576.pth... [2023-10-08 06:28:18,805][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000063296_64815104.pth [2023-10-08 06:28:18,829][00612] Updated weights for policy 1, policy_version 65400 (0.0007) [2023-10-08 06:28:19,115][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000065408_66977792.pth... [2023-10-08 06:28:19,153][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000063648_65175552.pth [2023-10-08 06:28:22,433][00611] Updated weights for policy 0, policy_version 65032 (0.0010) [2023-10-08 06:28:22,593][00612] Updated weights for policy 1, policy_version 65410 (0.0009) [2023-10-08 06:28:22,794][00611] Updated weights for policy 0, policy_version 65042 (0.0008) [2023-10-08 06:28:22,965][00612] Updated weights for policy 1, policy_version 65420 (0.0009) [2023-10-08 06:28:23,173][00611] Updated weights for policy 0, policy_version 65052 (0.0009) [2023-10-08 06:28:23,335][00612] Updated weights for policy 1, policy_version 65430 (0.0008) [2023-10-08 06:28:23,693][00612] Updated weights for policy 1, policy_version 65440 (0.0008) [2023-10-08 06:28:23,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14884.5). Total num frames: 133627904. Throughput: 0: 1823.4, 1: 1864.2. Samples: 33404174. Policy #0 lag: (min: 31.0, avg: 46.2, max: 63.0) [2023-10-08 06:28:23,754][130385] Avg episode reward: [(0, '66.860'), (1, '72.600')] [2023-10-08 06:28:26,878][00611] Updated weights for policy 0, policy_version 65062 (0.0008) [2023-10-08 06:28:27,242][00611] Updated weights for policy 0, policy_version 65072 (0.0008) [2023-10-08 06:28:27,334][00612] Updated weights for policy 1, policy_version 65450 (0.0009) [2023-10-08 06:28:27,621][00611] Updated weights for policy 0, policy_version 65082 (0.0010) [2023-10-08 06:28:27,694][00612] Updated weights for policy 1, policy_version 65460 (0.0008) [2023-10-08 06:28:28,059][00612] Updated weights for policy 1, policy_version 65470 (0.0008) [2023-10-08 06:28:28,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14884.4). Total num frames: 133693440. Throughput: 0: 1819.5, 1: 1848.6. Samples: 33425934. Policy #0 lag: (min: 31.0, avg: 46.2, max: 63.0) [2023-10-08 06:28:28,754][130385] Avg episode reward: [(0, '68.380'), (1, '67.970')] [2023-10-08 06:28:31,352][00611] Updated weights for policy 0, policy_version 65092 (0.0008) [2023-10-08 06:28:31,713][00611] Updated weights for policy 0, policy_version 65102 (0.0007) [2023-10-08 06:28:31,819][00612] Updated weights for policy 1, policy_version 65480 (0.0007) [2023-10-08 06:28:32,081][00611] Updated weights for policy 0, policy_version 65112 (0.0008) [2023-10-08 06:28:32,188][00612] Updated weights for policy 1, policy_version 65490 (0.0009) [2023-10-08 06:28:32,552][00612] Updated weights for policy 1, policy_version 65500 (0.0008) [2023-10-08 06:28:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 133758976. Throughput: 0: 1815.2, 1: 1834.5. Samples: 33446512. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-08 06:28:33,754][130385] Avg episode reward: [(0, '65.460'), (1, '71.170')] [2023-10-08 06:28:35,845][00611] Updated weights for policy 0, policy_version 65122 (0.0008) [2023-10-08 06:28:36,215][00611] Updated weights for policy 0, policy_version 65132 (0.0008) [2023-10-08 06:28:36,337][00612] Updated weights for policy 1, policy_version 65510 (0.0008) [2023-10-08 06:28:36,595][00611] Updated weights for policy 0, policy_version 65142 (0.0009) [2023-10-08 06:28:36,722][00612] Updated weights for policy 1, policy_version 65520 (0.0008) [2023-10-08 06:28:36,957][00611] Updated weights for policy 0, policy_version 65152 (0.0007) [2023-10-08 06:28:37,078][00612] Updated weights for policy 1, policy_version 65530 (0.0008) [2023-10-08 06:28:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 133824512. Throughput: 0: 1821.2, 1: 1837.9. Samples: 33458786. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-08 06:28:38,755][130385] Avg episode reward: [(0, '65.120'), (1, '71.240')] [2023-10-08 06:28:40,542][00611] Updated weights for policy 0, policy_version 65162 (0.0008) [2023-10-08 06:28:40,917][00611] Updated weights for policy 0, policy_version 65172 (0.0008) [2023-10-08 06:28:40,921][00612] Updated weights for policy 1, policy_version 65540 (0.0010) [2023-10-08 06:28:41,288][00611] Updated weights for policy 0, policy_version 65182 (0.0007) [2023-10-08 06:28:41,293][00612] Updated weights for policy 1, policy_version 65550 (0.0007) [2023-10-08 06:28:41,668][00612] Updated weights for policy 1, policy_version 65560 (0.0007) [2023-10-08 06:28:43,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 133890048. Throughput: 0: 1817.4, 1: 1831.1. Samples: 33479310. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-08 06:28:43,755][130385] Avg episode reward: [(0, '65.810'), (1, '72.530')] [2023-10-08 06:28:44,968][00611] Updated weights for policy 0, policy_version 65192 (0.0011) [2023-10-08 06:28:45,337][00612] Updated weights for policy 1, policy_version 65570 (0.0007) [2023-10-08 06:28:45,344][00611] Updated weights for policy 0, policy_version 65202 (0.0009) [2023-10-08 06:28:45,709][00611] Updated weights for policy 0, policy_version 65212 (0.0007) [2023-10-08 06:28:45,750][00612] Updated weights for policy 1, policy_version 65580 (0.0008) [2023-10-08 06:28:46,119][00612] Updated weights for policy 1, policy_version 65590 (0.0010) [2023-10-08 06:28:46,479][00612] Updated weights for policy 1, policy_version 65600 (0.0010) [2023-10-08 06:28:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 133955584. Throughput: 0: 1815.5, 1: 1841.6. Samples: 33502370. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-08 06:28:48,755][130385] Avg episode reward: [(0, '64.470'), (1, '71.870')] [2023-10-08 06:28:49,368][00611] Updated weights for policy 0, policy_version 65222 (0.0008) [2023-10-08 06:28:49,744][00611] Updated weights for policy 0, policy_version 65232 (0.0008) [2023-10-08 06:28:49,971][00612] Updated weights for policy 1, policy_version 65610 (0.0007) [2023-10-08 06:28:50,102][00611] Updated weights for policy 0, policy_version 65242 (0.0007) [2023-10-08 06:28:50,342][00612] Updated weights for policy 1, policy_version 65620 (0.0007) [2023-10-08 06:28:50,711][00612] Updated weights for policy 1, policy_version 65630 (0.0008) [2023-10-08 06:28:53,729][00611] Updated weights for policy 0, policy_version 65252 (0.0007) [2023-10-08 06:28:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 134021120. Throughput: 0: 1815.7, 1: 1840.5. Samples: 33512426. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-08 06:28:53,755][130385] Avg episode reward: [(0, '63.810'), (1, '71.380')] [2023-10-08 06:28:54,094][00611] Updated weights for policy 0, policy_version 65262 (0.0007) [2023-10-08 06:28:54,371][00612] Updated weights for policy 1, policy_version 65640 (0.0007) [2023-10-08 06:28:54,461][00611] Updated weights for policy 0, policy_version 65272 (0.0008) [2023-10-08 06:28:54,742][00612] Updated weights for policy 1, policy_version 65650 (0.0007) [2023-10-08 06:28:55,104][00612] Updated weights for policy 1, policy_version 65660 (0.0009) [2023-10-08 06:28:58,228][00611] Updated weights for policy 0, policy_version 65282 (0.0008) [2023-10-08 06:28:58,594][00611] Updated weights for policy 0, policy_version 65292 (0.0008) [2023-10-08 06:28:58,736][00612] Updated weights for policy 1, policy_version 65670 (0.0008) [2023-10-08 06:28:58,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 134086656. Throughput: 0: 1816.0, 1: 1839.3. Samples: 33535522. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-08 06:28:58,754][130385] Avg episode reward: [(0, '63.700'), (1, '73.480')] [2023-10-08 06:28:58,960][00611] Updated weights for policy 0, policy_version 65302 (0.0008) [2023-10-08 06:28:59,102][00612] Updated weights for policy 1, policy_version 65680 (0.0007) [2023-10-08 06:28:59,333][00611] Updated weights for policy 0, policy_version 65312 (0.0008) [2023-10-08 06:28:59,469][00612] Updated weights for policy 1, policy_version 65690 (0.0008) [2023-10-08 06:29:03,014][00611] Updated weights for policy 0, policy_version 65322 (0.0008) [2023-10-08 06:29:03,281][00612] Updated weights for policy 1, policy_version 65700 (0.0008) [2023-10-08 06:29:03,391][00611] Updated weights for policy 0, policy_version 65332 (0.0008) [2023-10-08 06:29:03,643][00612] Updated weights for policy 1, policy_version 65710 (0.0007) [2023-10-08 06:29:03,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 134152192. Throughput: 0: 1820.4, 1: 1838.0. Samples: 33557854. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-08 06:29:03,754][130385] Avg episode reward: [(0, '63.130'), (1, '73.420')] [2023-10-08 06:29:03,769][00611] Updated weights for policy 0, policy_version 65342 (0.0008) [2023-10-08 06:29:04,015][00612] Updated weights for policy 1, policy_version 65720 (0.0008) [2023-10-08 06:29:07,209][00611] Updated weights for policy 0, policy_version 65352 (0.0007) [2023-10-08 06:29:07,588][00611] Updated weights for policy 0, policy_version 65362 (0.0008) [2023-10-08 06:29:07,616][00612] Updated weights for policy 1, policy_version 65730 (0.0007) [2023-10-08 06:29:07,953][00611] Updated weights for policy 0, policy_version 65372 (0.0008) [2023-10-08 06:29:07,979][00612] Updated weights for policy 1, policy_version 65740 (0.0007) [2023-10-08 06:29:08,354][00612] Updated weights for policy 1, policy_version 65750 (0.0009) [2023-10-08 06:29:08,723][00612] Updated weights for policy 1, policy_version 65760 (0.0009) [2023-10-08 06:29:08,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14884.4). Total num frames: 134283264. Throughput: 0: 1819.0, 1: 1831.3. Samples: 33568438. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-08 06:29:08,755][130385] Avg episode reward: [(0, '63.840'), (1, '74.780')] [2023-10-08 06:29:11,583][00611] Updated weights for policy 0, policy_version 65382 (0.0008) [2023-10-08 06:29:11,943][00611] Updated weights for policy 0, policy_version 65392 (0.0007) [2023-10-08 06:29:12,316][00611] Updated weights for policy 0, policy_version 65402 (0.0007) [2023-10-08 06:29:12,400][00612] Updated weights for policy 1, policy_version 65770 (0.0008) [2023-10-08 06:29:12,765][00612] Updated weights for policy 1, policy_version 65780 (0.0009) [2023-10-08 06:29:13,131][00612] Updated weights for policy 1, policy_version 65790 (0.0009) [2023-10-08 06:29:13,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14884.4). Total num frames: 134348800. Throughput: 0: 1815.9, 1: 1839.3. Samples: 33590418. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-08 06:29:13,754][130385] Avg episode reward: [(0, '65.580'), (1, '74.350')] [2023-10-08 06:29:15,926][00611] Updated weights for policy 0, policy_version 65412 (0.0007) [2023-10-08 06:29:16,297][00611] Updated weights for policy 0, policy_version 65422 (0.0008) [2023-10-08 06:29:16,676][00611] Updated weights for policy 0, policy_version 65432 (0.0010) [2023-10-08 06:29:16,738][00612] Updated weights for policy 1, policy_version 65800 (0.0007) [2023-10-08 06:29:17,114][00612] Updated weights for policy 1, policy_version 65810 (0.0008) [2023-10-08 06:29:17,492][00612] Updated weights for policy 1, policy_version 65820 (0.0008) [2023-10-08 06:29:18,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 134414336. Throughput: 0: 1831.2, 1: 1840.3. Samples: 33611730. Policy #0 lag: (min: 2.0, avg: 9.0, max: 34.0) [2023-10-08 06:29:18,754][130385] Avg episode reward: [(0, '65.660'), (1, '72.320')] [2023-10-08 06:29:20,408][00611] Updated weights for policy 0, policy_version 65442 (0.0008) [2023-10-08 06:29:20,770][00611] Updated weights for policy 0, policy_version 65452 (0.0007) [2023-10-08 06:29:21,026][00612] Updated weights for policy 1, policy_version 65830 (0.0009) [2023-10-08 06:29:21,140][00611] Updated weights for policy 0, policy_version 65462 (0.0009) [2023-10-08 06:29:21,395][00612] Updated weights for policy 1, policy_version 65840 (0.0007) [2023-10-08 06:29:21,511][00611] Updated weights for policy 0, policy_version 65472 (0.0007) [2023-10-08 06:29:21,761][00612] Updated weights for policy 1, policy_version 65850 (0.0008) [2023-10-08 06:29:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14773.4). Total num frames: 134479872. Throughput: 0: 1817.3, 1: 1838.2. Samples: 33623284. Policy #0 lag: (min: 2.0, avg: 9.0, max: 34.0) [2023-10-08 06:29:23,754][130385] Avg episode reward: [(0, '69.190'), (1, '71.680')] [2023-10-08 06:29:25,189][00611] Updated weights for policy 0, policy_version 65482 (0.0009) [2023-10-08 06:29:25,469][00612] Updated weights for policy 1, policy_version 65860 (0.0007) [2023-10-08 06:29:25,558][00611] Updated weights for policy 0, policy_version 65492 (0.0008) [2023-10-08 06:29:25,835][00612] Updated weights for policy 1, policy_version 65870 (0.0008) [2023-10-08 06:29:25,931][00611] Updated weights for policy 0, policy_version 65502 (0.0007) [2023-10-08 06:29:26,207][00612] Updated weights for policy 1, policy_version 65880 (0.0009) [2023-10-08 06:29:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 134545408. Throughput: 0: 1821.5, 1: 1850.3. Samples: 33644540. Policy #0 lag: (min: 2.0, avg: 9.0, max: 34.0) [2023-10-08 06:29:28,754][130385] Avg episode reward: [(0, '74.560'), (1, '68.260')] [2023-10-08 06:29:29,655][00611] Updated weights for policy 0, policy_version 65512 (0.0007) [2023-10-08 06:29:29,735][00612] Updated weights for policy 1, policy_version 65890 (0.0010) [2023-10-08 06:29:30,027][00611] Updated weights for policy 0, policy_version 65522 (0.0008) [2023-10-08 06:29:30,112][00612] Updated weights for policy 1, policy_version 65900 (0.0007) [2023-10-08 06:29:30,395][00611] Updated weights for policy 0, policy_version 65532 (0.0009) [2023-10-08 06:29:30,480][00612] Updated weights for policy 1, policy_version 65910 (0.0009) [2023-10-08 06:29:30,838][00612] Updated weights for policy 1, policy_version 65920 (0.0008) [2023-10-08 06:29:33,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 134610944. Throughput: 0: 1823.6, 1: 1846.1. Samples: 33667506. Policy #0 lag: (min: 2.0, avg: 9.0, max: 34.0) [2023-10-08 06:29:33,755][130385] Avg episode reward: [(0, '72.040'), (1, '65.970')] [2023-10-08 06:29:34,255][00611] Updated weights for policy 0, policy_version 65542 (0.0007) [2023-10-08 06:29:34,531][00612] Updated weights for policy 1, policy_version 65930 (0.0009) [2023-10-08 06:29:34,640][00611] Updated weights for policy 0, policy_version 65552 (0.0007) [2023-10-08 06:29:34,890][00612] Updated weights for policy 1, policy_version 65940 (0.0007) [2023-10-08 06:29:35,013][00611] Updated weights for policy 0, policy_version 65562 (0.0007) [2023-10-08 06:29:35,269][00612] Updated weights for policy 1, policy_version 65950 (0.0008) [2023-10-08 06:29:38,510][00611] Updated weights for policy 0, policy_version 65572 (0.0009) [2023-10-08 06:29:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 134676480. Throughput: 0: 1817.6, 1: 1840.9. Samples: 33677058. Policy #0 lag: (min: 2.0, avg: 9.0, max: 34.0) [2023-10-08 06:29:38,755][130385] Avg episode reward: [(0, '72.420'), (1, '62.570')] [2023-10-08 06:29:38,887][00611] Updated weights for policy 0, policy_version 65582 (0.0009) [2023-10-08 06:29:38,955][00612] Updated weights for policy 1, policy_version 65960 (0.0010) [2023-10-08 06:29:39,255][00611] Updated weights for policy 0, policy_version 65592 (0.0007) [2023-10-08 06:29:39,320][00612] Updated weights for policy 1, policy_version 65970 (0.0007) [2023-10-08 06:29:39,688][00612] Updated weights for policy 1, policy_version 65980 (0.0008) [2023-10-08 06:29:42,899][00611] Updated weights for policy 0, policy_version 65602 (0.0007) [2023-10-08 06:29:43,273][00611] Updated weights for policy 0, policy_version 65612 (0.0009) [2023-10-08 06:29:43,375][00612] Updated weights for policy 1, policy_version 65990 (0.0008) [2023-10-08 06:29:43,642][00611] Updated weights for policy 0, policy_version 65622 (0.0009) [2023-10-08 06:29:43,745][00612] Updated weights for policy 1, policy_version 66000 (0.0007) [2023-10-08 06:29:43,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 134742016. Throughput: 0: 1823.2, 1: 1839.8. Samples: 33700358. Policy #0 lag: (min: 2.0, avg: 9.0, max: 34.0) [2023-10-08 06:29:43,754][130385] Avg episode reward: [(0, '73.320'), (1, '60.730')] [2023-10-08 06:29:44,016][00611] Updated weights for policy 0, policy_version 65632 (0.0009) [2023-10-08 06:29:44,106][00612] Updated weights for policy 1, policy_version 66010 (0.0009) [2023-10-08 06:29:47,574][00611] Updated weights for policy 0, policy_version 65642 (0.0008) [2023-10-08 06:29:47,615][00612] Updated weights for policy 1, policy_version 66020 (0.0008) [2023-10-08 06:29:47,935][00611] Updated weights for policy 0, policy_version 65652 (0.0008) [2023-10-08 06:29:47,984][00612] Updated weights for policy 1, policy_version 66030 (0.0007) [2023-10-08 06:29:48,305][00611] Updated weights for policy 0, policy_version 65662 (0.0008) [2023-10-08 06:29:48,356][00612] Updated weights for policy 1, policy_version 66040 (0.0008) [2023-10-08 06:29:48,754][130385] Fps is (10 sec: 19661.2, 60 sec: 15291.8, 300 sec: 14884.5). Total num frames: 134873088. Throughput: 0: 1812.0, 1: 1821.8. Samples: 33721374. Policy #0 lag: (min: 2.0, avg: 9.0, max: 34.0) [2023-10-08 06:29:48,754][130385] Avg episode reward: [(0, '67.760'), (1, '64.460')] [2023-10-08 06:29:51,948][00612] Updated weights for policy 1, policy_version 66050 (0.0009) [2023-10-08 06:29:52,013][00611] Updated weights for policy 0, policy_version 65672 (0.0007) [2023-10-08 06:29:52,310][00612] Updated weights for policy 1, policy_version 66060 (0.0007) [2023-10-08 06:29:52,383][00611] Updated weights for policy 0, policy_version 65682 (0.0007) [2023-10-08 06:29:52,681][00612] Updated weights for policy 1, policy_version 66070 (0.0008) [2023-10-08 06:29:52,762][00611] Updated weights for policy 0, policy_version 65692 (0.0008) [2023-10-08 06:29:53,044][00612] Updated weights for policy 1, policy_version 66080 (0.0009) [2023-10-08 06:29:53,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 134938624. Throughput: 0: 1827.6, 1: 1841.1. Samples: 33733530. Policy #0 lag: (min: 2.0, avg: 9.0, max: 34.0) [2023-10-08 06:29:53,754][130385] Avg episode reward: [(0, '70.510'), (1, '63.700')] [2023-10-08 06:29:56,588][00611] Updated weights for policy 0, policy_version 65702 (0.0007) [2023-10-08 06:29:56,666][00612] Updated weights for policy 1, policy_version 66090 (0.0008) [2023-10-08 06:29:56,955][00611] Updated weights for policy 0, policy_version 65712 (0.0007) [2023-10-08 06:29:57,032][00612] Updated weights for policy 1, policy_version 66100 (0.0008) [2023-10-08 06:29:57,321][00611] Updated weights for policy 0, policy_version 65722 (0.0007) [2023-10-08 06:29:57,402][00612] Updated weights for policy 1, policy_version 66110 (0.0007) [2023-10-08 06:29:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 135004160. Throughput: 0: 1823.9, 1: 1821.5. Samples: 33754458. Policy #0 lag: (min: 20.0, avg: 25.7, max: 52.0) [2023-10-08 06:29:58,754][130385] Avg episode reward: [(0, '73.220'), (1, '60.270')] [2023-10-08 06:30:01,000][00611] Updated weights for policy 0, policy_version 65732 (0.0009) [2023-10-08 06:30:01,029][00612] Updated weights for policy 1, policy_version 66120 (0.0008) [2023-10-08 06:30:01,370][00611] Updated weights for policy 0, policy_version 65742 (0.0009) [2023-10-08 06:30:01,394][00612] Updated weights for policy 1, policy_version 66130 (0.0009) [2023-10-08 06:30:01,737][00611] Updated weights for policy 0, policy_version 65752 (0.0008) [2023-10-08 06:30:01,764][00612] Updated weights for policy 1, policy_version 66140 (0.0008) [2023-10-08 06:30:03,754][130385] Fps is (10 sec: 13106.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 135069696. Throughput: 0: 1817.5, 1: 1840.6. Samples: 33776346. Policy #0 lag: (min: 20.0, avg: 25.7, max: 52.0) [2023-10-08 06:30:03,755][130385] Avg episode reward: [(0, '71.220'), (1, '64.860')] [2023-10-08 06:30:05,326][00611] Updated weights for policy 0, policy_version 65762 (0.0008) [2023-10-08 06:30:05,525][00612] Updated weights for policy 1, policy_version 66150 (0.0008) [2023-10-08 06:30:05,702][00611] Updated weights for policy 0, policy_version 65772 (0.0008) [2023-10-08 06:30:05,887][00612] Updated weights for policy 1, policy_version 66160 (0.0007) [2023-10-08 06:30:06,064][00611] Updated weights for policy 0, policy_version 65782 (0.0009) [2023-10-08 06:30:06,248][00612] Updated weights for policy 1, policy_version 66170 (0.0007) [2023-10-08 06:30:06,431][00611] Updated weights for policy 0, policy_version 65792 (0.0007) [2023-10-08 06:30:08,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 135135232. Throughput: 0: 1820.9, 1: 1822.5. Samples: 33787236. Policy #0 lag: (min: 20.0, avg: 25.7, max: 52.0) [2023-10-08 06:30:08,755][130385] Avg episode reward: [(0, '70.020'), (1, '65.000')] [2023-10-08 06:30:09,858][00612] Updated weights for policy 1, policy_version 66180 (0.0008) [2023-10-08 06:30:10,106][00611] Updated weights for policy 0, policy_version 65802 (0.0008) [2023-10-08 06:30:10,224][00612] Updated weights for policy 1, policy_version 66190 (0.0009) [2023-10-08 06:30:10,479][00611] Updated weights for policy 0, policy_version 65812 (0.0009) [2023-10-08 06:30:10,582][00612] Updated weights for policy 1, policy_version 66200 (0.0007) [2023-10-08 06:30:10,839][00611] Updated weights for policy 0, policy_version 65822 (0.0008) [2023-10-08 06:30:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 135200768. Throughput: 0: 1822.7, 1: 1837.5. Samples: 33809248. Policy #0 lag: (min: 20.0, avg: 25.7, max: 52.0) [2023-10-08 06:30:13,755][130385] Avg episode reward: [(0, '67.690'), (1, '66.500')] [2023-10-08 06:30:14,322][00612] Updated weights for policy 1, policy_version 66210 (0.0007) [2023-10-08 06:30:14,688][00612] Updated weights for policy 1, policy_version 66220 (0.0008) [2023-10-08 06:30:14,707][00611] Updated weights for policy 0, policy_version 65832 (0.0007) [2023-10-08 06:30:15,050][00612] Updated weights for policy 1, policy_version 66230 (0.0007) [2023-10-08 06:30:15,066][00611] Updated weights for policy 0, policy_version 65842 (0.0007) [2023-10-08 06:30:15,418][00612] Updated weights for policy 1, policy_version 66240 (0.0009) [2023-10-08 06:30:15,431][00611] Updated weights for policy 0, policy_version 65852 (0.0008) [2023-10-08 06:30:18,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 135266304. Throughput: 0: 1825.7, 1: 1841.0. Samples: 33832508. Policy #0 lag: (min: 20.0, avg: 25.7, max: 52.0) [2023-10-08 06:30:18,754][130385] Avg episode reward: [(0, '66.560'), (1, '65.470')] [2023-10-08 06:30:18,963][00612] Updated weights for policy 1, policy_version 66250 (0.0007) [2023-10-08 06:30:19,121][00611] Updated weights for policy 0, policy_version 65862 (0.0008) [2023-10-08 06:30:19,325][00612] Updated weights for policy 1, policy_version 66260 (0.0007) [2023-10-08 06:30:19,500][00611] Updated weights for policy 0, policy_version 65872 (0.0009) [2023-10-08 06:30:19,698][00612] Updated weights for policy 1, policy_version 66270 (0.0007) [2023-10-08 06:30:19,764][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000066272_67862528.pth... [2023-10-08 06:30:19,803][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000064544_66093056.pth [2023-10-08 06:30:19,877][00611] Updated weights for policy 0, policy_version 65882 (0.0009) [2023-10-08 06:30:20,087][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000065888_67469312.pth... [2023-10-08 06:30:20,116][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000064160_65699840.pth [2023-10-08 06:30:23,577][00612] Updated weights for policy 1, policy_version 66280 (0.0008) [2023-10-08 06:30:23,629][00611] Updated weights for policy 0, policy_version 65892 (0.0010) [2023-10-08 06:30:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 135331840. Throughput: 0: 1827.3, 1: 1844.3. Samples: 33842282. Policy #0 lag: (min: 20.0, avg: 25.7, max: 52.0) [2023-10-08 06:30:23,754][130385] Avg episode reward: [(0, '67.840'), (1, '67.730')] [2023-10-08 06:30:23,949][00612] Updated weights for policy 1, policy_version 66290 (0.0008) [2023-10-08 06:30:23,994][00611] Updated weights for policy 0, policy_version 65902 (0.0008) [2023-10-08 06:30:24,314][00612] Updated weights for policy 1, policy_version 66300 (0.0008) [2023-10-08 06:30:24,372][00611] Updated weights for policy 0, policy_version 65912 (0.0008) [2023-10-08 06:30:27,867][00612] Updated weights for policy 1, policy_version 66310 (0.0007) [2023-10-08 06:30:28,067][00611] Updated weights for policy 0, policy_version 65922 (0.0010) [2023-10-08 06:30:28,237][00612] Updated weights for policy 1, policy_version 66320 (0.0008) [2023-10-08 06:30:28,432][00611] Updated weights for policy 0, policy_version 65932 (0.0008) [2023-10-08 06:30:28,605][00612] Updated weights for policy 1, policy_version 66330 (0.0007) [2023-10-08 06:30:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 135397376. Throughput: 0: 1823.1, 1: 1840.7. Samples: 33865228. Policy #0 lag: (min: 20.0, avg: 25.7, max: 52.0) [2023-10-08 06:30:28,754][130385] Avg episode reward: [(0, '66.370'), (1, '68.650')] [2023-10-08 06:30:28,800][00611] Updated weights for policy 0, policy_version 65942 (0.0008) [2023-10-08 06:30:29,170][00611] Updated weights for policy 0, policy_version 65952 (0.0009) [2023-10-08 06:30:32,340][00612] Updated weights for policy 1, policy_version 66340 (0.0010) [2023-10-08 06:30:32,710][00612] Updated weights for policy 1, policy_version 66350 (0.0010) [2023-10-08 06:30:32,817][00611] Updated weights for policy 0, policy_version 65962 (0.0009) [2023-10-08 06:30:33,077][00612] Updated weights for policy 1, policy_version 66360 (0.0007) [2023-10-08 06:30:33,179][00611] Updated weights for policy 0, policy_version 65972 (0.0009) [2023-10-08 06:30:33,557][00611] Updated weights for policy 0, policy_version 65982 (0.0009) [2023-10-08 06:30:33,754][130385] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 135528448. Throughput: 0: 1829.6, 1: 1830.2. Samples: 33886064. Policy #0 lag: (min: 20.0, avg: 25.7, max: 52.0) [2023-10-08 06:30:33,754][130385] Avg episode reward: [(0, '66.690'), (1, '66.610')] [2023-10-08 06:30:36,764][00612] Updated weights for policy 1, policy_version 66370 (0.0007) [2023-10-08 06:30:37,135][00612] Updated weights for policy 1, policy_version 66380 (0.0007) [2023-10-08 06:30:37,222][00611] Updated weights for policy 0, policy_version 65992 (0.0008) [2023-10-08 06:30:37,504][00612] Updated weights for policy 1, policy_version 66390 (0.0007) [2023-10-08 06:30:37,598][00611] Updated weights for policy 0, policy_version 66002 (0.0008) [2023-10-08 06:30:37,863][00612] Updated weights for policy 1, policy_version 66400 (0.0008) [2023-10-08 06:30:37,963][00611] Updated weights for policy 0, policy_version 66012 (0.0008) [2023-10-08 06:30:38,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 135593984. Throughput: 0: 1818.8, 1: 1839.3. Samples: 33898148. Policy #0 lag: (min: 20.0, avg: 25.7, max: 52.0) [2023-10-08 06:30:38,754][130385] Avg episode reward: [(0, '66.000'), (1, '71.330')] [2023-10-08 06:30:41,500][00612] Updated weights for policy 1, policy_version 66410 (0.0008) [2023-10-08 06:30:41,550][00611] Updated weights for policy 0, policy_version 66022 (0.0009) [2023-10-08 06:30:41,864][00612] Updated weights for policy 1, policy_version 66420 (0.0008) [2023-10-08 06:30:41,918][00611] Updated weights for policy 0, policy_version 66032 (0.0007) [2023-10-08 06:30:42,230][00612] Updated weights for policy 1, policy_version 66430 (0.0009) [2023-10-08 06:30:42,284][00611] Updated weights for policy 0, policy_version 66042 (0.0008) [2023-10-08 06:30:43,754][130385] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 135659520. Throughput: 0: 1823.3, 1: 1830.3. Samples: 33918870. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-08 06:30:43,754][130385] Avg episode reward: [(0, '66.530'), (1, '74.730')] [2023-10-08 06:30:45,855][00612] Updated weights for policy 1, policy_version 66440 (0.0009) [2023-10-08 06:30:45,988][00611] Updated weights for policy 0, policy_version 66052 (0.0007) [2023-10-08 06:30:46,216][00612] Updated weights for policy 1, policy_version 66450 (0.0008) [2023-10-08 06:30:46,352][00611] Updated weights for policy 0, policy_version 66062 (0.0008) [2023-10-08 06:30:46,590][00612] Updated weights for policy 1, policy_version 66460 (0.0008) [2023-10-08 06:30:46,729][00611] Updated weights for policy 0, policy_version 66072 (0.0008) [2023-10-08 06:30:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 135725056. Throughput: 0: 1823.0, 1: 1836.5. Samples: 33941024. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-08 06:30:48,754][130385] Avg episode reward: [(0, '65.760'), (1, '74.150')] [2023-10-08 06:30:50,236][00612] Updated weights for policy 1, policy_version 66470 (0.0008) [2023-10-08 06:30:50,245][00611] Updated weights for policy 0, policy_version 66082 (0.0009) [2023-10-08 06:30:50,610][00612] Updated weights for policy 1, policy_version 66480 (0.0008) [2023-10-08 06:30:50,621][00611] Updated weights for policy 0, policy_version 66092 (0.0008) [2023-10-08 06:30:50,979][00611] Updated weights for policy 0, policy_version 66102 (0.0007) [2023-10-08 06:30:50,982][00612] Updated weights for policy 1, policy_version 66490 (0.0008) [2023-10-08 06:30:51,359][00611] Updated weights for policy 0, policy_version 66112 (0.0009) [2023-10-08 06:30:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 135790592. Throughput: 0: 1819.5, 1: 1830.2. Samples: 33951472. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-08 06:30:53,754][130385] Avg episode reward: [(0, '65.410'), (1, '73.640')] [2023-10-08 06:30:54,642][00612] Updated weights for policy 1, policy_version 66500 (0.0009) [2023-10-08 06:30:55,007][00612] Updated weights for policy 1, policy_version 66510 (0.0008) [2023-10-08 06:30:55,167][00611] Updated weights for policy 0, policy_version 66122 (0.0007) [2023-10-08 06:30:55,372][00612] Updated weights for policy 1, policy_version 66520 (0.0008) [2023-10-08 06:30:55,537][00611] Updated weights for policy 0, policy_version 66132 (0.0009) [2023-10-08 06:30:55,901][00611] Updated weights for policy 0, policy_version 66142 (0.0009) [2023-10-08 06:30:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 135856128. Throughput: 0: 1823.3, 1: 1835.0. Samples: 33973870. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-08 06:30:58,754][130385] Avg episode reward: [(0, '64.270'), (1, '75.580')] [2023-10-08 06:30:58,911][00612] Updated weights for policy 1, policy_version 66530 (0.0007) [2023-10-08 06:30:59,275][00612] Updated weights for policy 1, policy_version 66540 (0.0007) [2023-10-08 06:30:59,644][00612] Updated weights for policy 1, policy_version 66550 (0.0008) [2023-10-08 06:30:59,686][00611] Updated weights for policy 0, policy_version 66152 (0.0009) [2023-10-08 06:31:00,008][00612] Updated weights for policy 1, policy_version 66560 (0.0009) [2023-10-08 06:31:00,057][00611] Updated weights for policy 0, policy_version 66162 (0.0009) [2023-10-08 06:31:00,419][00611] Updated weights for policy 0, policy_version 66172 (0.0008) [2023-10-08 06:31:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 135921664. Throughput: 0: 1818.5, 1: 1829.2. Samples: 33996654. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-08 06:31:03,754][130385] Avg episode reward: [(0, '63.590'), (1, '74.830')] [2023-10-08 06:31:03,808][00612] Updated weights for policy 1, policy_version 66570 (0.0007) [2023-10-08 06:31:04,058][00611] Updated weights for policy 0, policy_version 66182 (0.0009) [2023-10-08 06:31:04,175][00612] Updated weights for policy 1, policy_version 66580 (0.0008) [2023-10-08 06:31:04,431][00611] Updated weights for policy 0, policy_version 66192 (0.0009) [2023-10-08 06:31:04,543][00612] Updated weights for policy 1, policy_version 66590 (0.0008) [2023-10-08 06:31:04,810][00611] Updated weights for policy 0, policy_version 66202 (0.0008) [2023-10-08 06:31:08,451][00612] Updated weights for policy 1, policy_version 66600 (0.0008) [2023-10-08 06:31:08,515][00611] Updated weights for policy 0, policy_version 66212 (0.0008) [2023-10-08 06:31:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 135987200. Throughput: 0: 1817.6, 1: 1824.6. Samples: 34006180. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-08 06:31:08,754][130385] Avg episode reward: [(0, '65.230'), (1, '70.250')] [2023-10-08 06:31:08,827][00612] Updated weights for policy 1, policy_version 66610 (0.0009) [2023-10-08 06:31:08,882][00611] Updated weights for policy 0, policy_version 66222 (0.0007) [2023-10-08 06:31:09,187][00612] Updated weights for policy 1, policy_version 66620 (0.0009) [2023-10-08 06:31:09,256][00611] Updated weights for policy 0, policy_version 66232 (0.0008) [2023-10-08 06:31:12,930][00612] Updated weights for policy 1, policy_version 66630 (0.0008) [2023-10-08 06:31:12,998][00611] Updated weights for policy 0, policy_version 66242 (0.0007) [2023-10-08 06:31:13,305][00612] Updated weights for policy 1, policy_version 66640 (0.0009) [2023-10-08 06:31:13,367][00611] Updated weights for policy 0, policy_version 66252 (0.0007) [2023-10-08 06:31:13,676][00612] Updated weights for policy 1, policy_version 66650 (0.0007) [2023-10-08 06:31:13,740][00611] Updated weights for policy 0, policy_version 66262 (0.0007) [2023-10-08 06:31:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 136052736. Throughput: 0: 1821.3, 1: 1821.7. Samples: 34029164. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-08 06:31:13,754][130385] Avg episode reward: [(0, '65.660'), (1, '69.480')] [2023-10-08 06:31:14,104][00611] Updated weights for policy 0, policy_version 66272 (0.0009) [2023-10-08 06:31:17,192][00612] Updated weights for policy 1, policy_version 66660 (0.0008) [2023-10-08 06:31:17,561][00612] Updated weights for policy 1, policy_version 66670 (0.0008) [2023-10-08 06:31:17,787][00611] Updated weights for policy 0, policy_version 66282 (0.0007) [2023-10-08 06:31:17,936][00612] Updated weights for policy 1, policy_version 66680 (0.0008) [2023-10-08 06:31:18,164][00611] Updated weights for policy 0, policy_version 66292 (0.0009) [2023-10-08 06:31:18,525][00611] Updated weights for policy 0, policy_version 66302 (0.0008) [2023-10-08 06:31:18,754][130385] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 136183808. Throughput: 0: 1823.7, 1: 1817.5. Samples: 34049916. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-08 06:31:18,755][130385] Avg episode reward: [(0, '61.010'), (1, '73.310')] [2023-10-08 06:31:21,528][00612] Updated weights for policy 1, policy_version 66690 (0.0007) [2023-10-08 06:31:21,893][00612] Updated weights for policy 1, policy_version 66700 (0.0009) [2023-10-08 06:31:22,124][00611] Updated weights for policy 0, policy_version 66312 (0.0008) [2023-10-08 06:31:22,268][00612] Updated weights for policy 1, policy_version 66710 (0.0009) [2023-10-08 06:31:22,492][00611] Updated weights for policy 0, policy_version 66322 (0.0007) [2023-10-08 06:31:22,630][00612] Updated weights for policy 1, policy_version 66720 (0.0007) [2023-10-08 06:31:22,861][00611] Updated weights for policy 0, policy_version 66332 (0.0007) [2023-10-08 06:31:23,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 136249344. Throughput: 0: 1824.7, 1: 1820.5. Samples: 34062184. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-08 06:31:23,754][130385] Avg episode reward: [(0, '60.750'), (1, '74.460')] [2023-10-08 06:31:26,229][00612] Updated weights for policy 1, policy_version 66730 (0.0007) [2023-10-08 06:31:26,495][00611] Updated weights for policy 0, policy_version 66342 (0.0007) [2023-10-08 06:31:26,594][00612] Updated weights for policy 1, policy_version 66740 (0.0008) [2023-10-08 06:31:26,866][00611] Updated weights for policy 0, policy_version 66352 (0.0007) [2023-10-08 06:31:26,963][00612] Updated weights for policy 1, policy_version 66750 (0.0007) [2023-10-08 06:31:27,226][00611] Updated weights for policy 0, policy_version 66362 (0.0009) [2023-10-08 06:31:28,754][130385] Fps is (10 sec: 13107.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 136314880. Throughput: 0: 1821.5, 1: 1818.7. Samples: 34082680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:31:28,754][130385] Avg episode reward: [(0, '60.700'), (1, '75.400')] [2023-10-08 06:31:30,512][00612] Updated weights for policy 1, policy_version 66760 (0.0007) [2023-10-08 06:31:30,801][00611] Updated weights for policy 0, policy_version 66372 (0.0008) [2023-10-08 06:31:30,884][00612] Updated weights for policy 1, policy_version 66770 (0.0010) [2023-10-08 06:31:31,179][00611] Updated weights for policy 0, policy_version 66382 (0.0010) [2023-10-08 06:31:31,262][00612] Updated weights for policy 1, policy_version 66780 (0.0008) [2023-10-08 06:31:31,548][00611] Updated weights for policy 0, policy_version 66392 (0.0008) [2023-10-08 06:31:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 136380416. Throughput: 0: 1833.5, 1: 1825.6. Samples: 34105682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:31:33,754][130385] Avg episode reward: [(0, '60.410'), (1, '74.480')] [2023-10-08 06:31:34,937][00612] Updated weights for policy 1, policy_version 66790 (0.0007) [2023-10-08 06:31:35,073][00611] Updated weights for policy 0, policy_version 66402 (0.0008) [2023-10-08 06:31:35,297][00612] Updated weights for policy 1, policy_version 66800 (0.0008) [2023-10-08 06:31:35,450][00611] Updated weights for policy 0, policy_version 66412 (0.0007) [2023-10-08 06:31:35,661][00612] Updated weights for policy 1, policy_version 66810 (0.0008) [2023-10-08 06:31:35,816][00611] Updated weights for policy 0, policy_version 66422 (0.0008) [2023-10-08 06:31:36,191][00611] Updated weights for policy 0, policy_version 66432 (0.0010) [2023-10-08 06:31:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 136445952. Throughput: 0: 1831.6, 1: 1823.3. Samples: 34115944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:31:38,754][130385] Avg episode reward: [(0, '61.760'), (1, '75.830')] [2023-10-08 06:31:39,356][00612] Updated weights for policy 1, policy_version 66820 (0.0009) [2023-10-08 06:31:39,731][00612] Updated weights for policy 1, policy_version 66830 (0.0010) [2023-10-08 06:31:39,775][00611] Updated weights for policy 0, policy_version 66442 (0.0007) [2023-10-08 06:31:40,096][00612] Updated weights for policy 1, policy_version 66840 (0.0010) [2023-10-08 06:31:40,142][00611] Updated weights for policy 0, policy_version 66452 (0.0008) [2023-10-08 06:31:40,514][00611] Updated weights for policy 0, policy_version 66462 (0.0008) [2023-10-08 06:31:43,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 136511488. Throughput: 0: 1838.1, 1: 1822.0. Samples: 34138576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:31:43,755][130385] Avg episode reward: [(0, '60.700'), (1, '78.130')] [2023-10-08 06:31:43,858][00612] Updated weights for policy 1, policy_version 66850 (0.0008) [2023-10-08 06:31:44,193][00611] Updated weights for policy 0, policy_version 66472 (0.0008) [2023-10-08 06:31:44,230][00612] Updated weights for policy 1, policy_version 66860 (0.0008) [2023-10-08 06:31:44,555][00611] Updated weights for policy 0, policy_version 66482 (0.0008) [2023-10-08 06:31:44,602][00612] Updated weights for policy 1, policy_version 66870 (0.0008) [2023-10-08 06:31:44,922][00611] Updated weights for policy 0, policy_version 66492 (0.0009) [2023-10-08 06:31:44,969][00612] Updated weights for policy 1, policy_version 66880 (0.0008) [2023-10-08 06:31:48,723][00612] Updated weights for policy 1, policy_version 66890 (0.0007) [2023-10-08 06:31:48,728][00611] Updated weights for policy 0, policy_version 66502 (0.0009) [2023-10-08 06:31:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 136577024. Throughput: 0: 1838.7, 1: 1819.5. Samples: 34161272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:31:48,754][130385] Avg episode reward: [(0, '63.890'), (1, '76.880')] [2023-10-08 06:31:49,096][00612] Updated weights for policy 1, policy_version 66900 (0.0008) [2023-10-08 06:31:49,106][00611] Updated weights for policy 0, policy_version 66512 (0.0008) [2023-10-08 06:31:49,459][00612] Updated weights for policy 1, policy_version 66910 (0.0009) [2023-10-08 06:31:49,465][00611] Updated weights for policy 0, policy_version 66522 (0.0010) [2023-10-08 06:31:53,021][00611] Updated weights for policy 0, policy_version 66532 (0.0008) [2023-10-08 06:31:53,133][00612] Updated weights for policy 1, policy_version 66920 (0.0007) [2023-10-08 06:31:53,406][00611] Updated weights for policy 0, policy_version 66542 (0.0007) [2023-10-08 06:31:53,504][00612] Updated weights for policy 1, policy_version 66930 (0.0007) [2023-10-08 06:31:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 136642560. Throughput: 0: 1843.0, 1: 1822.4. Samples: 34171124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:31:53,754][130385] Avg episode reward: [(0, '67.730'), (1, '76.970')] [2023-10-08 06:31:53,773][00611] Updated weights for policy 0, policy_version 66552 (0.0007) [2023-10-08 06:31:53,869][00612] Updated weights for policy 1, policy_version 66940 (0.0008) [2023-10-08 06:31:57,595][00611] Updated weights for policy 0, policy_version 66562 (0.0009) [2023-10-08 06:31:57,620][00612] Updated weights for policy 1, policy_version 66950 (0.0008) [2023-10-08 06:31:57,969][00611] Updated weights for policy 0, policy_version 66572 (0.0007) [2023-10-08 06:31:58,000][00612] Updated weights for policy 1, policy_version 66960 (0.0008) [2023-10-08 06:31:58,335][00611] Updated weights for policy 0, policy_version 66582 (0.0007) [2023-10-08 06:31:58,363][00612] Updated weights for policy 1, policy_version 66970 (0.0009) [2023-10-08 06:31:58,702][00611] Updated weights for policy 0, policy_version 66592 (0.0008) [2023-10-08 06:31:58,754][130385] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 136773632. Throughput: 0: 1841.2, 1: 1829.6. Samples: 34194354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:31:58,754][130385] Avg episode reward: [(0, '67.490'), (1, '74.880')] [2023-10-08 06:32:02,097][00612] Updated weights for policy 1, policy_version 66980 (0.0009) [2023-10-08 06:32:02,466][00611] Updated weights for policy 0, policy_version 66602 (0.0007) [2023-10-08 06:32:02,470][00612] Updated weights for policy 1, policy_version 66990 (0.0007) [2023-10-08 06:32:02,841][00612] Updated weights for policy 1, policy_version 67000 (0.0009) [2023-10-08 06:32:02,844][00611] Updated weights for policy 0, policy_version 66612 (0.0007) [2023-10-08 06:32:03,204][00611] Updated weights for policy 0, policy_version 66622 (0.0007) [2023-10-08 06:32:03,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 136839168. Throughput: 0: 1826.5, 1: 1824.1. Samples: 34214194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:32:03,754][130385] Avg episode reward: [(0, '68.310'), (1, '71.190')] [2023-10-08 06:32:06,499][00612] Updated weights for policy 1, policy_version 67010 (0.0008) [2023-10-08 06:32:06,858][00612] Updated weights for policy 1, policy_version 67020 (0.0009) [2023-10-08 06:32:06,875][00611] Updated weights for policy 0, policy_version 66632 (0.0007) [2023-10-08 06:32:07,227][00612] Updated weights for policy 1, policy_version 67030 (0.0008) [2023-10-08 06:32:07,240][00611] Updated weights for policy 0, policy_version 66642 (0.0008) [2023-10-08 06:32:07,592][00612] Updated weights for policy 1, policy_version 67040 (0.0008) [2023-10-08 06:32:07,620][00611] Updated weights for policy 0, policy_version 66652 (0.0008) [2023-10-08 06:32:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 136904704. Throughput: 0: 1835.1, 1: 1827.2. Samples: 34226990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:32:08,754][130385] Avg episode reward: [(0, '66.920'), (1, '70.250')] [2023-10-08 06:32:11,296][00612] Updated weights for policy 1, policy_version 67050 (0.0008) [2023-10-08 06:32:11,300][00611] Updated weights for policy 0, policy_version 66662 (0.0008) [2023-10-08 06:32:11,656][00611] Updated weights for policy 0, policy_version 66672 (0.0008) [2023-10-08 06:32:11,662][00612] Updated weights for policy 1, policy_version 67060 (0.0007) [2023-10-08 06:32:12,024][00611] Updated weights for policy 0, policy_version 66682 (0.0008) [2023-10-08 06:32:12,035][00612] Updated weights for policy 1, policy_version 67070 (0.0007) [2023-10-08 06:32:13,754][130385] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 136970240. Throughput: 0: 1828.7, 1: 1823.4. Samples: 34247026. Policy #0 lag: (min: 22.0, avg: 29.5, max: 54.0) [2023-10-08 06:32:13,755][130385] Avg episode reward: [(0, '70.340'), (1, '76.330')] [2023-10-08 06:32:15,669][00611] Updated weights for policy 0, policy_version 66692 (0.0008) [2023-10-08 06:32:15,783][00612] Updated weights for policy 1, policy_version 67080 (0.0008) [2023-10-08 06:32:16,032][00611] Updated weights for policy 0, policy_version 66702 (0.0008) [2023-10-08 06:32:16,153][00612] Updated weights for policy 1, policy_version 67090 (0.0009) [2023-10-08 06:32:16,408][00611] Updated weights for policy 0, policy_version 66712 (0.0009) [2023-10-08 06:32:16,520][00612] Updated weights for policy 1, policy_version 67100 (0.0008) [2023-10-08 06:32:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 137035776. Throughput: 0: 1827.3, 1: 1815.2. Samples: 34269596. Policy #0 lag: (min: 22.0, avg: 29.5, max: 54.0) [2023-10-08 06:32:18,755][130385] Avg episode reward: [(0, '69.770'), (1, '72.790')] [2023-10-08 06:32:18,765][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000067104_68714496.pth... [2023-10-08 06:32:18,766][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000066720_68321280.pth... [2023-10-08 06:32:18,806][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000065408_66977792.pth [2023-10-08 06:32:18,806][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000065024_66584576.pth [2023-10-08 06:32:20,012][00611] Updated weights for policy 0, policy_version 66722 (0.0007) [2023-10-08 06:32:20,324][00612] Updated weights for policy 1, policy_version 67110 (0.0008) [2023-10-08 06:32:20,384][00611] Updated weights for policy 0, policy_version 66732 (0.0009) [2023-10-08 06:32:20,690][00612] Updated weights for policy 1, policy_version 67120 (0.0008) [2023-10-08 06:32:20,756][00611] Updated weights for policy 0, policy_version 66742 (0.0009) [2023-10-08 06:32:21,057][00612] Updated weights for policy 1, policy_version 67130 (0.0010) [2023-10-08 06:32:21,119][00611] Updated weights for policy 0, policy_version 66752 (0.0007) [2023-10-08 06:32:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 137101312. Throughput: 0: 1820.4, 1: 1815.7. Samples: 34279568. Policy #0 lag: (min: 22.0, avg: 29.5, max: 54.0) [2023-10-08 06:32:23,754][130385] Avg episode reward: [(0, '69.290'), (1, '72.750')] [2023-10-08 06:32:24,749][00611] Updated weights for policy 0, policy_version 66762 (0.0009) [2023-10-08 06:32:24,772][00612] Updated weights for policy 1, policy_version 67140 (0.0007) [2023-10-08 06:32:25,124][00611] Updated weights for policy 0, policy_version 66772 (0.0009) [2023-10-08 06:32:25,148][00612] Updated weights for policy 1, policy_version 67150 (0.0007) [2023-10-08 06:32:25,494][00611] Updated weights for policy 0, policy_version 66782 (0.0010) [2023-10-08 06:32:25,513][00612] Updated weights for policy 1, policy_version 67160 (0.0009) [2023-10-08 06:32:28,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 137166848. Throughput: 0: 1827.2, 1: 1816.0. Samples: 34302522. Policy #0 lag: (min: 22.0, avg: 29.5, max: 54.0) [2023-10-08 06:32:28,754][130385] Avg episode reward: [(0, '69.020'), (1, '71.480')] [2023-10-08 06:32:28,980][00611] Updated weights for policy 0, policy_version 66792 (0.0008) [2023-10-08 06:32:29,190][00612] Updated weights for policy 1, policy_version 67170 (0.0009) [2023-10-08 06:32:29,345][00611] Updated weights for policy 0, policy_version 66802 (0.0007) [2023-10-08 06:32:29,551][00612] Updated weights for policy 1, policy_version 67180 (0.0007) [2023-10-08 06:32:29,713][00611] Updated weights for policy 0, policy_version 66812 (0.0009) [2023-10-08 06:32:29,914][00612] Updated weights for policy 1, policy_version 67190 (0.0007) [2023-10-08 06:32:30,280][00612] Updated weights for policy 1, policy_version 67200 (0.0009) [2023-10-08 06:32:33,479][00611] Updated weights for policy 0, policy_version 66822 (0.0009) [2023-10-08 06:32:33,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 137232384. Throughput: 0: 1827.9, 1: 1822.8. Samples: 34325550. Policy #0 lag: (min: 22.0, avg: 29.5, max: 54.0) [2023-10-08 06:32:33,755][130385] Avg episode reward: [(0, '69.660'), (1, '69.720')] [2023-10-08 06:32:33,849][00611] Updated weights for policy 0, policy_version 66832 (0.0008) [2023-10-08 06:32:33,900][00612] Updated weights for policy 1, policy_version 67210 (0.0008) [2023-10-08 06:32:34,224][00611] Updated weights for policy 0, policy_version 66842 (0.0007) [2023-10-08 06:32:34,271][00612] Updated weights for policy 1, policy_version 67220 (0.0007) [2023-10-08 06:32:34,627][00612] Updated weights for policy 1, policy_version 67230 (0.0008) [2023-10-08 06:32:38,057][00611] Updated weights for policy 0, policy_version 66852 (0.0008) [2023-10-08 06:32:38,268][00612] Updated weights for policy 1, policy_version 67240 (0.0008) [2023-10-08 06:32:38,447][00611] Updated weights for policy 0, policy_version 66862 (0.0008) [2023-10-08 06:32:38,644][00612] Updated weights for policy 1, policy_version 67250 (0.0008) [2023-10-08 06:32:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 137297920. Throughput: 0: 1825.5, 1: 1825.0. Samples: 34335396. Policy #0 lag: (min: 22.0, avg: 29.5, max: 54.0) [2023-10-08 06:32:38,754][130385] Avg episode reward: [(0, '68.190'), (1, '71.530')] [2023-10-08 06:32:38,830][00611] Updated weights for policy 0, policy_version 66872 (0.0009) [2023-10-08 06:32:39,002][00612] Updated weights for policy 1, policy_version 67260 (0.0007) [2023-10-08 06:32:42,432][00611] Updated weights for policy 0, policy_version 66882 (0.0008) [2023-10-08 06:32:42,799][00611] Updated weights for policy 0, policy_version 66892 (0.0007) [2023-10-08 06:32:42,891][00612] Updated weights for policy 1, policy_version 67270 (0.0009) [2023-10-08 06:32:43,167][00611] Updated weights for policy 0, policy_version 66902 (0.0007) [2023-10-08 06:32:43,284][00612] Updated weights for policy 1, policy_version 67280 (0.0009) [2023-10-08 06:32:43,538][00611] Updated weights for policy 0, policy_version 66912 (0.0008) [2023-10-08 06:32:43,645][00612] Updated weights for policy 1, policy_version 67290 (0.0007) [2023-10-08 06:32:43,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 137396224. Throughput: 0: 1817.7, 1: 1818.8. Samples: 34357998. Policy #0 lag: (min: 22.0, avg: 29.5, max: 54.0) [2023-10-08 06:32:43,754][130385] Avg episode reward: [(0, '68.660'), (1, '71.800')] [2023-10-08 06:32:47,244][00611] Updated weights for policy 0, policy_version 66922 (0.0008) [2023-10-08 06:32:47,305][00612] Updated weights for policy 1, policy_version 67300 (0.0008) [2023-10-08 06:32:47,615][00611] Updated weights for policy 0, policy_version 66932 (0.0008) [2023-10-08 06:32:47,672][00612] Updated weights for policy 1, policy_version 67310 (0.0008) [2023-10-08 06:32:47,983][00611] Updated weights for policy 0, policy_version 66942 (0.0009) [2023-10-08 06:32:48,037][00612] Updated weights for policy 1, policy_version 67320 (0.0007) [2023-10-08 06:32:48,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 137494528. Throughput: 0: 1815.5, 1: 1823.2. Samples: 34377932. Policy #0 lag: (min: 22.0, avg: 29.5, max: 54.0) [2023-10-08 06:32:48,754][130385] Avg episode reward: [(0, '66.330'), (1, '71.820')] [2023-10-08 06:32:51,720][00612] Updated weights for policy 1, policy_version 67330 (0.0008) [2023-10-08 06:32:51,743][00611] Updated weights for policy 0, policy_version 66952 (0.0008) [2023-10-08 06:32:52,095][00612] Updated weights for policy 1, policy_version 67340 (0.0007) [2023-10-08 06:32:52,102][00611] Updated weights for policy 0, policy_version 66962 (0.0009) [2023-10-08 06:32:52,452][00612] Updated weights for policy 1, policy_version 67350 (0.0007) [2023-10-08 06:32:52,468][00611] Updated weights for policy 0, policy_version 66972 (0.0007) [2023-10-08 06:32:52,820][00612] Updated weights for policy 1, policy_version 67360 (0.0009) [2023-10-08 06:32:53,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 137560064. Throughput: 0: 1822.1, 1: 1810.9. Samples: 34390478. Policy #0 lag: (min: 22.0, avg: 29.5, max: 54.0) [2023-10-08 06:32:53,755][130385] Avg episode reward: [(0, '66.940'), (1, '68.090')] [2023-10-08 06:32:56,008][00611] Updated weights for policy 0, policy_version 66982 (0.0009) [2023-10-08 06:32:56,380][00611] Updated weights for policy 0, policy_version 66992 (0.0010) [2023-10-08 06:32:56,567][00612] Updated weights for policy 1, policy_version 67370 (0.0008) [2023-10-08 06:32:56,740][00611] Updated weights for policy 0, policy_version 67002 (0.0007) [2023-10-08 06:32:56,939][00612] Updated weights for policy 1, policy_version 67380 (0.0007) [2023-10-08 06:32:57,314][00612] Updated weights for policy 1, policy_version 67390 (0.0007) [2023-10-08 06:32:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 137625600. Throughput: 0: 1823.6, 1: 1815.8. Samples: 34410802. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 06:32:58,755][130385] Avg episode reward: [(0, '65.760'), (1, '71.770')] [2023-10-08 06:33:00,453][00611] Updated weights for policy 0, policy_version 67012 (0.0009) [2023-10-08 06:33:00,822][00611] Updated weights for policy 0, policy_version 67022 (0.0008) [2023-10-08 06:33:00,838][00612] Updated weights for policy 1, policy_version 67400 (0.0009) [2023-10-08 06:33:01,197][00611] Updated weights for policy 0, policy_version 67032 (0.0007) [2023-10-08 06:33:01,206][00612] Updated weights for policy 1, policy_version 67410 (0.0008) [2023-10-08 06:33:01,578][00612] Updated weights for policy 1, policy_version 67420 (0.0008) [2023-10-08 06:33:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 137691136. Throughput: 0: 1829.6, 1: 1818.1. Samples: 34433746. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 06:33:03,755][130385] Avg episode reward: [(0, '63.010'), (1, '76.560')] [2023-10-08 06:33:04,789][00611] Updated weights for policy 0, policy_version 67042 (0.0007) [2023-10-08 06:33:05,152][00611] Updated weights for policy 0, policy_version 67052 (0.0009) [2023-10-08 06:33:05,183][00612] Updated weights for policy 1, policy_version 67430 (0.0010) [2023-10-08 06:33:05,519][00611] Updated weights for policy 0, policy_version 67062 (0.0008) [2023-10-08 06:33:05,552][00612] Updated weights for policy 1, policy_version 67440 (0.0008) [2023-10-08 06:33:05,890][00611] Updated weights for policy 0, policy_version 67072 (0.0007) [2023-10-08 06:33:05,915][00612] Updated weights for policy 1, policy_version 67450 (0.0007) [2023-10-08 06:33:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 137756672. Throughput: 0: 1832.6, 1: 1817.3. Samples: 34443812. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 06:33:08,755][130385] Avg episode reward: [(0, '60.430'), (1, '77.660')] [2023-10-08 06:33:09,484][00611] Updated weights for policy 0, policy_version 67082 (0.0007) [2023-10-08 06:33:09,552][00612] Updated weights for policy 1, policy_version 67460 (0.0007) [2023-10-08 06:33:09,851][00611] Updated weights for policy 0, policy_version 67092 (0.0008) [2023-10-08 06:33:09,917][00612] Updated weights for policy 1, policy_version 67470 (0.0007) [2023-10-08 06:33:10,233][00611] Updated weights for policy 0, policy_version 67102 (0.0009) [2023-10-08 06:33:10,285][00612] Updated weights for policy 1, policy_version 67480 (0.0007) [2023-10-08 06:33:13,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 137822208. Throughput: 0: 1831.6, 1: 1823.3. Samples: 34466994. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 06:33:13,754][130385] Avg episode reward: [(0, '64.340'), (1, '76.760')] [2023-10-08 06:33:13,826][00612] Updated weights for policy 1, policy_version 67490 (0.0007) [2023-10-08 06:33:13,912][00611] Updated weights for policy 0, policy_version 67112 (0.0007) [2023-10-08 06:33:14,205][00612] Updated weights for policy 1, policy_version 67500 (0.0007) [2023-10-08 06:33:14,283][00611] Updated weights for policy 0, policy_version 67122 (0.0007) [2023-10-08 06:33:14,564][00612] Updated weights for policy 1, policy_version 67510 (0.0008) [2023-10-08 06:33:14,652][00611] Updated weights for policy 0, policy_version 67132 (0.0007) [2023-10-08 06:33:14,930][00612] Updated weights for policy 1, policy_version 67520 (0.0007) [2023-10-08 06:33:18,251][00611] Updated weights for policy 0, policy_version 67142 (0.0008) [2023-10-08 06:33:18,465][00612] Updated weights for policy 1, policy_version 67530 (0.0007) [2023-10-08 06:33:18,619][00611] Updated weights for policy 0, policy_version 67152 (0.0007) [2023-10-08 06:33:18,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137887744. Throughput: 0: 1829.4, 1: 1822.6. Samples: 34489890. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 06:33:18,754][130385] Avg episode reward: [(0, '64.300'), (1, '83.930')] [2023-10-08 06:33:18,831][00612] Updated weights for policy 1, policy_version 67540 (0.0007) [2023-10-08 06:33:18,985][00611] Updated weights for policy 0, policy_version 67162 (0.0007) [2023-10-08 06:33:19,191][00612] Updated weights for policy 1, policy_version 67550 (0.0007) [2023-10-08 06:33:22,779][00611] Updated weights for policy 0, policy_version 67172 (0.0008) [2023-10-08 06:33:22,957][00612] Updated weights for policy 1, policy_version 67560 (0.0007) [2023-10-08 06:33:23,136][00611] Updated weights for policy 0, policy_version 67182 (0.0009) [2023-10-08 06:33:23,317][00612] Updated weights for policy 1, policy_version 67570 (0.0007) [2023-10-08 06:33:23,511][00611] Updated weights for policy 0, policy_version 67192 (0.0008) [2023-10-08 06:33:23,691][00612] Updated weights for policy 1, policy_version 67580 (0.0008) [2023-10-08 06:33:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137953280. Throughput: 0: 1835.1, 1: 1822.3. Samples: 34499976. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 06:33:23,754][130385] Avg episode reward: [(0, '63.740'), (1, '82.220')] [2023-10-08 06:33:27,198][00611] Updated weights for policy 0, policy_version 67202 (0.0008) [2023-10-08 06:33:27,382][00612] Updated weights for policy 1, policy_version 67590 (0.0009) [2023-10-08 06:33:27,599][00611] Updated weights for policy 0, policy_version 67212 (0.0009) [2023-10-08 06:33:27,753][00612] Updated weights for policy 1, policy_version 67600 (0.0007) [2023-10-08 06:33:27,963][00611] Updated weights for policy 0, policy_version 67222 (0.0007) [2023-10-08 06:33:28,123][00612] Updated weights for policy 1, policy_version 67610 (0.0008) [2023-10-08 06:33:28,338][00611] Updated weights for policy 0, policy_version 67232 (0.0007) [2023-10-08 06:33:28,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 138084352. Throughput: 0: 1837.1, 1: 1830.2. Samples: 34523026. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 06:33:28,754][130385] Avg episode reward: [(0, '71.620'), (1, '76.170')] [2023-10-08 06:33:31,894][00612] Updated weights for policy 1, policy_version 67620 (0.0009) [2023-10-08 06:33:32,160][00611] Updated weights for policy 0, policy_version 67242 (0.0008) [2023-10-08 06:33:32,258][00612] Updated weights for policy 1, policy_version 67630 (0.0008) [2023-10-08 06:33:32,528][00611] Updated weights for policy 0, policy_version 67252 (0.0009) [2023-10-08 06:33:32,625][00612] Updated weights for policy 1, policy_version 67640 (0.0007) [2023-10-08 06:33:32,899][00611] Updated weights for policy 0, policy_version 67262 (0.0008) [2023-10-08 06:33:33,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 138149888. Throughput: 0: 1827.2, 1: 1832.1. Samples: 34542602. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 06:33:33,754][130385] Avg episode reward: [(0, '69.970'), (1, '77.510')] [2023-10-08 06:33:36,324][00612] Updated weights for policy 1, policy_version 67650 (0.0007) [2023-10-08 06:33:36,480][00611] Updated weights for policy 0, policy_version 67272 (0.0008) [2023-10-08 06:33:36,698][00612] Updated weights for policy 1, policy_version 67660 (0.0007) [2023-10-08 06:33:36,847][00611] Updated weights for policy 0, policy_version 67282 (0.0009) [2023-10-08 06:33:37,054][00612] Updated weights for policy 1, policy_version 67670 (0.0007) [2023-10-08 06:33:37,220][00611] Updated weights for policy 0, policy_version 67292 (0.0008) [2023-10-08 06:33:37,426][00612] Updated weights for policy 1, policy_version 67680 (0.0010) [2023-10-08 06:33:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 138215424. Throughput: 0: 1828.5, 1: 1847.3. Samples: 34555890. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 06:33:38,754][130385] Avg episode reward: [(0, '68.540'), (1, '77.900')] [2023-10-08 06:33:41,094][00611] Updated weights for policy 0, policy_version 67302 (0.0008) [2023-10-08 06:33:41,145][00612] Updated weights for policy 1, policy_version 67690 (0.0009) [2023-10-08 06:33:41,462][00611] Updated weights for policy 0, policy_version 67312 (0.0008) [2023-10-08 06:33:41,509][00612] Updated weights for policy 1, policy_version 67700 (0.0010) [2023-10-08 06:33:41,831][00611] Updated weights for policy 0, policy_version 67322 (0.0007) [2023-10-08 06:33:41,875][00612] Updated weights for policy 1, policy_version 67710 (0.0010) [2023-10-08 06:33:43,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 138280960. Throughput: 0: 1819.0, 1: 1841.7. Samples: 34575536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:33:43,755][130385] Avg episode reward: [(0, '66.410'), (1, '80.030')] [2023-10-08 06:33:45,426][00612] Updated weights for policy 1, policy_version 67720 (0.0007) [2023-10-08 06:33:45,504][00611] Updated weights for policy 0, policy_version 67332 (0.0008) [2023-10-08 06:33:45,799][00612] Updated weights for policy 1, policy_version 67730 (0.0007) [2023-10-08 06:33:45,874][00611] Updated weights for policy 0, policy_version 67342 (0.0008) [2023-10-08 06:33:46,164][00612] Updated weights for policy 1, policy_version 67740 (0.0008) [2023-10-08 06:33:46,244][00611] Updated weights for policy 0, policy_version 67352 (0.0009) [2023-10-08 06:33:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 138346496. Throughput: 0: 1815.5, 1: 1847.6. Samples: 34598586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:33:48,755][130385] Avg episode reward: [(0, '68.450'), (1, '79.070')] [2023-10-08 06:33:49,738][00612] Updated weights for policy 1, policy_version 67750 (0.0008) [2023-10-08 06:33:50,106][00612] Updated weights for policy 1, policy_version 67760 (0.0008) [2023-10-08 06:33:50,149][00611] Updated weights for policy 0, policy_version 67362 (0.0008) [2023-10-08 06:33:50,472][00612] Updated weights for policy 1, policy_version 67770 (0.0010) [2023-10-08 06:33:50,514][00611] Updated weights for policy 0, policy_version 67372 (0.0008) [2023-10-08 06:33:50,886][00611] Updated weights for policy 0, policy_version 67382 (0.0007) [2023-10-08 06:33:51,257][00611] Updated weights for policy 0, policy_version 67392 (0.0010) [2023-10-08 06:33:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 138412032. Throughput: 0: 1814.2, 1: 1851.2. Samples: 34608752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:33:53,755][130385] Avg episode reward: [(0, '69.950'), (1, '79.690')] [2023-10-08 06:33:54,247][00612] Updated weights for policy 1, policy_version 67780 (0.0008) [2023-10-08 06:33:54,610][00612] Updated weights for policy 1, policy_version 67790 (0.0008) [2023-10-08 06:33:54,855][00611] Updated weights for policy 0, policy_version 67402 (0.0007) [2023-10-08 06:33:54,979][00612] Updated weights for policy 1, policy_version 67800 (0.0007) [2023-10-08 06:33:55,235][00611] Updated weights for policy 0, policy_version 67412 (0.0007) [2023-10-08 06:33:55,595][00611] Updated weights for policy 0, policy_version 67422 (0.0008) [2023-10-08 06:33:58,676][00612] Updated weights for policy 1, policy_version 67810 (0.0007) [2023-10-08 06:33:58,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 138477568. Throughput: 0: 1815.8, 1: 1843.5. Samples: 34631666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:33:58,755][130385] Avg episode reward: [(0, '68.700'), (1, '80.610')] [2023-10-08 06:33:59,037][00612] Updated weights for policy 1, policy_version 67820 (0.0007) [2023-10-08 06:33:59,178][00611] Updated weights for policy 0, policy_version 67432 (0.0009) [2023-10-08 06:33:59,411][00612] Updated weights for policy 1, policy_version 67830 (0.0008) [2023-10-08 06:33:59,560][00611] Updated weights for policy 0, policy_version 67442 (0.0008) [2023-10-08 06:33:59,774][00612] Updated weights for policy 1, policy_version 67840 (0.0008) [2023-10-08 06:33:59,920][00611] Updated weights for policy 0, policy_version 67452 (0.0007) [2023-10-08 06:34:03,212][00612] Updated weights for policy 1, policy_version 67850 (0.0007) [2023-10-08 06:34:03,520][00611] Updated weights for policy 0, policy_version 67462 (0.0008) [2023-10-08 06:34:03,577][00612] Updated weights for policy 1, policy_version 67860 (0.0007) [2023-10-08 06:34:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 138543104. Throughput: 0: 1814.6, 1: 1841.3. Samples: 34654404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:34:03,755][130385] Avg episode reward: [(0, '69.140'), (1, '80.330')] [2023-10-08 06:34:03,890][00611] Updated weights for policy 0, policy_version 67472 (0.0009) [2023-10-08 06:34:03,941][00612] Updated weights for policy 1, policy_version 67870 (0.0008) [2023-10-08 06:34:04,261][00611] Updated weights for policy 0, policy_version 67482 (0.0008) [2023-10-08 06:34:07,557][00612] Updated weights for policy 1, policy_version 67880 (0.0009) [2023-10-08 06:34:07,863][00611] Updated weights for policy 0, policy_version 67492 (0.0007) [2023-10-08 06:34:07,924][00612] Updated weights for policy 1, policy_version 67890 (0.0008) [2023-10-08 06:34:08,224][00611] Updated weights for policy 0, policy_version 67502 (0.0008) [2023-10-08 06:34:08,281][00612] Updated weights for policy 1, policy_version 67900 (0.0008) [2023-10-08 06:34:08,586][00611] Updated weights for policy 0, policy_version 67512 (0.0008) [2023-10-08 06:34:08,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 138641408. Throughput: 0: 1814.6, 1: 1849.0. Samples: 34664840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:34:08,755][130385] Avg episode reward: [(0, '71.580'), (1, '81.590')] [2023-10-08 06:34:12,028][00612] Updated weights for policy 1, policy_version 67910 (0.0009) [2023-10-08 06:34:12,242][00611] Updated weights for policy 0, policy_version 67522 (0.0009) [2023-10-08 06:34:12,391][00612] Updated weights for policy 1, policy_version 67920 (0.0009) [2023-10-08 06:34:12,654][00611] Updated weights for policy 0, policy_version 67532 (0.0008) [2023-10-08 06:34:12,754][00612] Updated weights for policy 1, policy_version 67930 (0.0010) [2023-10-08 06:34:13,014][00611] Updated weights for policy 0, policy_version 67542 (0.0008) [2023-10-08 06:34:13,388][00611] Updated weights for policy 0, policy_version 67552 (0.0008) [2023-10-08 06:34:13,754][130385] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 138739712. Throughput: 0: 1820.4, 1: 1833.3. Samples: 34687444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:34:13,754][130385] Avg episode reward: [(0, '72.820'), (1, '74.840')] [2023-10-08 06:34:16,478][00612] Updated weights for policy 1, policy_version 67940 (0.0009) [2023-10-08 06:34:16,864][00612] Updated weights for policy 1, policy_version 67950 (0.0007) [2023-10-08 06:34:17,005][00611] Updated weights for policy 0, policy_version 67562 (0.0009) [2023-10-08 06:34:17,227][00612] Updated weights for policy 1, policy_version 67960 (0.0007) [2023-10-08 06:34:17,372][00611] Updated weights for policy 0, policy_version 67572 (0.0008) [2023-10-08 06:34:17,755][00611] Updated weights for policy 0, policy_version 67582 (0.0008) [2023-10-08 06:34:18,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 138805248. Throughput: 0: 1826.3, 1: 1846.4. Samples: 34707874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:34:18,755][130385] Avg episode reward: [(0, '71.530'), (1, '77.100')] [2023-10-08 06:34:18,763][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000067968_69599232.pth... [2023-10-08 06:34:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000067584_69206016.pth... [2023-10-08 06:34:18,810][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000065888_67469312.pth [2023-10-08 06:34:18,810][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000066272_67862528.pth [2023-10-08 06:34:20,735][00612] Updated weights for policy 1, policy_version 67970 (0.0008) [2023-10-08 06:34:21,103][00612] Updated weights for policy 1, policy_version 67980 (0.0010) [2023-10-08 06:34:21,468][00612] Updated weights for policy 1, policy_version 67990 (0.0008) [2023-10-08 06:34:21,617][00611] Updated weights for policy 0, policy_version 67592 (0.0010) [2023-10-08 06:34:21,839][00612] Updated weights for policy 1, policy_version 68000 (0.0008) [2023-10-08 06:34:21,979][00611] Updated weights for policy 0, policy_version 67602 (0.0008) [2023-10-08 06:34:22,354][00611] Updated weights for policy 0, policy_version 67612 (0.0008) [2023-10-08 06:34:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 138870784. Throughput: 0: 1822.0, 1: 1829.2. Samples: 34720192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:34:23,754][130385] Avg episode reward: [(0, '70.190'), (1, '75.320')] [2023-10-08 06:34:25,602][00612] Updated weights for policy 1, policy_version 68010 (0.0011) [2023-10-08 06:34:25,972][00612] Updated weights for policy 1, policy_version 68020 (0.0008) [2023-10-08 06:34:26,075][00611] Updated weights for policy 0, policy_version 67622 (0.0009) [2023-10-08 06:34:26,341][00612] Updated weights for policy 1, policy_version 68030 (0.0008) [2023-10-08 06:34:26,441][00611] Updated weights for policy 0, policy_version 67632 (0.0009) [2023-10-08 06:34:26,825][00611] Updated weights for policy 0, policy_version 67642 (0.0009) [2023-10-08 06:34:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 138936320. Throughput: 0: 1826.0, 1: 1842.2. Samples: 34740604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:34:28,755][130385] Avg episode reward: [(0, '72.050'), (1, '70.120')] [2023-10-08 06:34:30,026][00612] Updated weights for policy 1, policy_version 68040 (0.0008) [2023-10-08 06:34:30,381][00612] Updated weights for policy 1, policy_version 68050 (0.0007) [2023-10-08 06:34:30,531][00611] Updated weights for policy 0, policy_version 67652 (0.0009) [2023-10-08 06:34:30,743][00612] Updated weights for policy 1, policy_version 68060 (0.0008) [2023-10-08 06:34:30,899][00611] Updated weights for policy 0, policy_version 67662 (0.0008) [2023-10-08 06:34:31,274][00611] Updated weights for policy 0, policy_version 67672 (0.0009) [2023-10-08 06:34:33,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 139001856. Throughput: 0: 1824.9, 1: 1847.6. Samples: 34763852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:34:33,754][130385] Avg episode reward: [(0, '72.770'), (1, '69.740')] [2023-10-08 06:34:34,154][00612] Updated weights for policy 1, policy_version 68070 (0.0007) [2023-10-08 06:34:34,525][00612] Updated weights for policy 1, policy_version 68080 (0.0007) [2023-10-08 06:34:34,892][00612] Updated weights for policy 1, policy_version 68090 (0.0007) [2023-10-08 06:34:34,973][00611] Updated weights for policy 0, policy_version 67682 (0.0008) [2023-10-08 06:34:35,346][00611] Updated weights for policy 0, policy_version 67692 (0.0011) [2023-10-08 06:34:35,717][00611] Updated weights for policy 0, policy_version 67702 (0.0008) [2023-10-08 06:34:36,090][00611] Updated weights for policy 0, policy_version 67712 (0.0007) [2023-10-08 06:34:38,559][00612] Updated weights for policy 1, policy_version 68100 (0.0007) [2023-10-08 06:34:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 139067392. Throughput: 0: 1823.1, 1: 1844.3. Samples: 34773782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:34:38,755][130385] Avg episode reward: [(0, '71.230'), (1, '72.750')] [2023-10-08 06:34:38,923][00612] Updated weights for policy 1, policy_version 68110 (0.0008) [2023-10-08 06:34:39,304][00612] Updated weights for policy 1, policy_version 68120 (0.0007) [2023-10-08 06:34:39,705][00611] Updated weights for policy 0, policy_version 67722 (0.0007) [2023-10-08 06:34:40,083][00611] Updated weights for policy 0, policy_version 67732 (0.0010) [2023-10-08 06:34:40,443][00611] Updated weights for policy 0, policy_version 67742 (0.0010) [2023-10-08 06:34:42,867][00612] Updated weights for policy 1, policy_version 68130 (0.0007) [2023-10-08 06:34:43,241][00612] Updated weights for policy 1, policy_version 68140 (0.0007) [2023-10-08 06:34:43,615][00612] Updated weights for policy 1, policy_version 68150 (0.0009) [2023-10-08 06:34:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139132928. Throughput: 0: 1815.8, 1: 1853.9. Samples: 34796800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:34:43,754][130385] Avg episode reward: [(0, '70.960'), (1, '71.700')] [2023-10-08 06:34:43,985][00612] Updated weights for policy 1, policy_version 68160 (0.0007) [2023-10-08 06:34:44,183][00611] Updated weights for policy 0, policy_version 67752 (0.0010) [2023-10-08 06:34:44,560][00611] Updated weights for policy 0, policy_version 67762 (0.0011) [2023-10-08 06:34:44,938][00611] Updated weights for policy 0, policy_version 67772 (0.0011) [2023-10-08 06:34:47,652][00612] Updated weights for policy 1, policy_version 68170 (0.0007) [2023-10-08 06:34:48,017][00612] Updated weights for policy 1, policy_version 68180 (0.0007) [2023-10-08 06:34:48,395][00612] Updated weights for policy 1, policy_version 68190 (0.0007) [2023-10-08 06:34:48,531][00611] Updated weights for policy 0, policy_version 67782 (0.0008) [2023-10-08 06:34:48,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 139231232. Throughput: 0: 1825.2, 1: 1830.7. Samples: 34818922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:34:48,755][130385] Avg episode reward: [(0, '73.110'), (1, '74.530')] [2023-10-08 06:34:48,906][00611] Updated weights for policy 0, policy_version 67792 (0.0009) [2023-10-08 06:34:49,277][00611] Updated weights for policy 0, policy_version 67802 (0.0007) [2023-10-08 06:34:51,939][00612] Updated weights for policy 1, policy_version 68200 (0.0007) [2023-10-08 06:34:52,315][00612] Updated weights for policy 1, policy_version 68210 (0.0009) [2023-10-08 06:34:52,684][00612] Updated weights for policy 1, policy_version 68220 (0.0008) [2023-10-08 06:34:52,835][00611] Updated weights for policy 0, policy_version 67812 (0.0008) [2023-10-08 06:34:53,200][00611] Updated weights for policy 0, policy_version 67822 (0.0010) [2023-10-08 06:34:53,571][00611] Updated weights for policy 0, policy_version 67832 (0.0010) [2023-10-08 06:34:53,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 139296768. Throughput: 0: 1821.0, 1: 1849.1. Samples: 34829994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:34:53,756][130385] Avg episode reward: [(0, '74.140'), (1, '72.250')] [2023-10-08 06:34:56,407][00612] Updated weights for policy 1, policy_version 68230 (0.0007) [2023-10-08 06:34:56,776][00612] Updated weights for policy 1, policy_version 68240 (0.0011) [2023-10-08 06:34:57,141][00612] Updated weights for policy 1, policy_version 68250 (0.0010) [2023-10-08 06:34:57,403][00611] Updated weights for policy 0, policy_version 67842 (0.0009) [2023-10-08 06:34:57,814][00611] Updated weights for policy 0, policy_version 67852 (0.0008) [2023-10-08 06:34:58,193][00611] Updated weights for policy 0, policy_version 67862 (0.0008) [2023-10-08 06:34:58,578][00611] Updated weights for policy 0, policy_version 67872 (0.0008) [2023-10-08 06:34:58,754][130385] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 139395072. Throughput: 0: 1818.5, 1: 1832.8. Samples: 34851756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:34:58,754][130385] Avg episode reward: [(0, '73.480'), (1, '75.630')] [2023-10-08 06:35:00,723][00612] Updated weights for policy 1, policy_version 68260 (0.0008) [2023-10-08 06:35:01,095][00612] Updated weights for policy 1, policy_version 68270 (0.0008) [2023-10-08 06:35:01,465][00612] Updated weights for policy 1, policy_version 68280 (0.0009) [2023-10-08 06:35:02,274][00611] Updated weights for policy 0, policy_version 67882 (0.0008) [2023-10-08 06:35:02,646][00611] Updated weights for policy 0, policy_version 67892 (0.0008) [2023-10-08 06:35:03,020][00611] Updated weights for policy 0, policy_version 67902 (0.0009) [2023-10-08 06:35:03,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 139460608. Throughput: 0: 1813.9, 1: 1852.4. Samples: 34872856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:35:03,755][130385] Avg episode reward: [(0, '72.340'), (1, '73.450')] [2023-10-08 06:35:05,161][00612] Updated weights for policy 1, policy_version 68290 (0.0008) [2023-10-08 06:35:05,537][00612] Updated weights for policy 1, policy_version 68300 (0.0008) [2023-10-08 06:35:05,903][00612] Updated weights for policy 1, policy_version 68310 (0.0009) [2023-10-08 06:35:06,275][00612] Updated weights for policy 1, policy_version 68320 (0.0010) [2023-10-08 06:35:06,649][00611] Updated weights for policy 0, policy_version 67912 (0.0010) [2023-10-08 06:35:07,022][00611] Updated weights for policy 0, policy_version 67922 (0.0007) [2023-10-08 06:35:07,391][00611] Updated weights for policy 0, policy_version 67932 (0.0010) [2023-10-08 06:35:08,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 139526144. Throughput: 0: 1818.1, 1: 1835.8. Samples: 34884618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:35:08,754][130385] Avg episode reward: [(0, '72.440'), (1, '75.290')] [2023-10-08 06:35:09,799][00612] Updated weights for policy 1, policy_version 68330 (0.0007) [2023-10-08 06:35:10,172][00612] Updated weights for policy 1, policy_version 68340 (0.0008) [2023-10-08 06:35:10,539][00612] Updated weights for policy 1, policy_version 68350 (0.0009) [2023-10-08 06:35:11,075][00611] Updated weights for policy 0, policy_version 67942 (0.0011) [2023-10-08 06:35:11,440][00611] Updated weights for policy 0, policy_version 67952 (0.0008) [2023-10-08 06:35:11,809][00611] Updated weights for policy 0, policy_version 67962 (0.0007) [2023-10-08 06:35:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 139591680. Throughput: 0: 1816.6, 1: 1855.7. Samples: 34905858. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 06:35:13,755][130385] Avg episode reward: [(0, '71.810'), (1, '75.520')] [2023-10-08 06:35:14,118][00612] Updated weights for policy 1, policy_version 68360 (0.0007) [2023-10-08 06:35:14,478][00612] Updated weights for policy 1, policy_version 68370 (0.0007) [2023-10-08 06:35:14,848][00612] Updated weights for policy 1, policy_version 68380 (0.0008) [2023-10-08 06:35:15,406][00611] Updated weights for policy 0, policy_version 67972 (0.0008) [2023-10-08 06:35:15,777][00611] Updated weights for policy 0, policy_version 67982 (0.0011) [2023-10-08 06:35:16,144][00611] Updated weights for policy 0, policy_version 67992 (0.0008) [2023-10-08 06:35:18,417][00612] Updated weights for policy 1, policy_version 68390 (0.0008) [2023-10-08 06:35:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 139657216. Throughput: 0: 1825.8, 1: 1848.4. Samples: 34929192. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 06:35:18,755][130385] Avg episode reward: [(0, '75.300'), (1, '70.480')] [2023-10-08 06:35:18,783][00612] Updated weights for policy 1, policy_version 68400 (0.0008) [2023-10-08 06:35:19,156][00612] Updated weights for policy 1, policy_version 68410 (0.0008) [2023-10-08 06:35:19,714][00611] Updated weights for policy 0, policy_version 68002 (0.0008) [2023-10-08 06:35:20,092][00611] Updated weights for policy 0, policy_version 68012 (0.0008) [2023-10-08 06:35:20,461][00611] Updated weights for policy 0, policy_version 68022 (0.0010) [2023-10-08 06:35:20,837][00611] Updated weights for policy 0, policy_version 68032 (0.0008) [2023-10-08 06:35:22,768][00612] Updated weights for policy 1, policy_version 68420 (0.0009) [2023-10-08 06:35:23,139][00612] Updated weights for policy 1, policy_version 68430 (0.0007) [2023-10-08 06:35:23,500][00612] Updated weights for policy 1, policy_version 68440 (0.0007) [2023-10-08 06:35:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 139722752. Throughput: 0: 1824.0, 1: 1849.8. Samples: 34939102. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 06:35:23,754][130385] Avg episode reward: [(0, '71.900'), (1, '71.590')] [2023-10-08 06:35:24,566][00611] Updated weights for policy 0, policy_version 68042 (0.0007) [2023-10-08 06:35:24,939][00611] Updated weights for policy 0, policy_version 68052 (0.0007) [2023-10-08 06:35:25,314][00611] Updated weights for policy 0, policy_version 68062 (0.0007) [2023-10-08 06:35:27,143][00612] Updated weights for policy 1, policy_version 68450 (0.0007) [2023-10-08 06:35:27,510][00612] Updated weights for policy 1, policy_version 68460 (0.0008) [2023-10-08 06:35:27,879][00612] Updated weights for policy 1, policy_version 68470 (0.0009) [2023-10-08 06:35:28,244][00612] Updated weights for policy 1, policy_version 68480 (0.0008) [2023-10-08 06:35:28,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 139821056. Throughput: 0: 1829.5, 1: 1844.9. Samples: 34962148. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 06:35:28,754][130385] Avg episode reward: [(0, '71.630'), (1, '75.400')] [2023-10-08 06:35:28,899][00611] Updated weights for policy 0, policy_version 68072 (0.0008) [2023-10-08 06:35:29,270][00611] Updated weights for policy 0, policy_version 68082 (0.0008) [2023-10-08 06:35:29,649][00611] Updated weights for policy 0, policy_version 68092 (0.0008) [2023-10-08 06:35:31,884][00612] Updated weights for policy 1, policy_version 68490 (0.0009) [2023-10-08 06:35:32,253][00612] Updated weights for policy 1, policy_version 68500 (0.0007) [2023-10-08 06:35:32,625][00612] Updated weights for policy 1, policy_version 68510 (0.0007) [2023-10-08 06:35:33,215][00611] Updated weights for policy 0, policy_version 68102 (0.0008) [2023-10-08 06:35:33,581][00611] Updated weights for policy 0, policy_version 68112 (0.0008) [2023-10-08 06:35:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 139886592. Throughput: 0: 1825.1, 1: 1841.2. Samples: 34983904. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 06:35:33,754][130385] Avg episode reward: [(0, '68.650'), (1, '77.120')] [2023-10-08 06:35:33,952][00611] Updated weights for policy 0, policy_version 68122 (0.0008) [2023-10-08 06:35:36,299][00612] Updated weights for policy 1, policy_version 68520 (0.0007) [2023-10-08 06:35:36,668][00612] Updated weights for policy 1, policy_version 68530 (0.0007) [2023-10-08 06:35:37,029][00612] Updated weights for policy 1, policy_version 68540 (0.0007) [2023-10-08 06:35:37,708][00611] Updated weights for policy 0, policy_version 68132 (0.0007) [2023-10-08 06:35:38,085][00611] Updated weights for policy 0, policy_version 68142 (0.0007) [2023-10-08 06:35:38,448][00611] Updated weights for policy 0, policy_version 68152 (0.0007) [2023-10-08 06:35:38,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 139984896. Throughput: 0: 1831.7, 1: 1839.5. Samples: 34995200. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 06:35:38,754][130385] Avg episode reward: [(0, '67.290'), (1, '73.030')] [2023-10-08 06:35:40,984][00612] Updated weights for policy 1, policy_version 68550 (0.0008) [2023-10-08 06:35:41,352][00612] Updated weights for policy 1, policy_version 68560 (0.0010) [2023-10-08 06:35:41,722][00612] Updated weights for policy 1, policy_version 68570 (0.0009) [2023-10-08 06:35:42,021][00611] Updated weights for policy 0, policy_version 68162 (0.0008) [2023-10-08 06:35:42,407][00611] Updated weights for policy 0, policy_version 68172 (0.0010) [2023-10-08 06:35:42,784][00611] Updated weights for policy 0, policy_version 68182 (0.0008) [2023-10-08 06:35:43,142][00611] Updated weights for policy 0, policy_version 68192 (0.0011) [2023-10-08 06:35:43,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 140050432. Throughput: 0: 1829.8, 1: 1831.9. Samples: 35016534. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 06:35:43,754][130385] Avg episode reward: [(0, '65.600'), (1, '74.830')] [2023-10-08 06:35:45,235][00612] Updated weights for policy 1, policy_version 68580 (0.0009) [2023-10-08 06:35:45,599][00612] Updated weights for policy 1, policy_version 68590 (0.0010) [2023-10-08 06:35:45,967][00612] Updated weights for policy 1, policy_version 68600 (0.0011) [2023-10-08 06:35:46,758][00611] Updated weights for policy 0, policy_version 68202 (0.0009) [2023-10-08 06:35:47,126][00611] Updated weights for policy 0, policy_version 68212 (0.0008) [2023-10-08 06:35:47,493][00611] Updated weights for policy 0, policy_version 68222 (0.0007) [2023-10-08 06:35:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 140115968. Throughput: 0: 1841.4, 1: 1842.8. Samples: 35038644. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 06:35:48,754][130385] Avg episode reward: [(0, '63.070'), (1, '74.430')] [2023-10-08 06:35:49,554][00612] Updated weights for policy 1, policy_version 68610 (0.0010) [2023-10-08 06:35:49,932][00612] Updated weights for policy 1, policy_version 68620 (0.0008) [2023-10-08 06:35:50,301][00612] Updated weights for policy 1, policy_version 68630 (0.0009) [2023-10-08 06:35:50,661][00612] Updated weights for policy 1, policy_version 68640 (0.0008) [2023-10-08 06:35:51,015][00611] Updated weights for policy 0, policy_version 68232 (0.0007) [2023-10-08 06:35:51,385][00611] Updated weights for policy 0, policy_version 68242 (0.0009) [2023-10-08 06:35:51,759][00611] Updated weights for policy 0, policy_version 68252 (0.0009) [2023-10-08 06:35:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140181504. Throughput: 0: 1833.2, 1: 1840.8. Samples: 35049948. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 06:35:53,754][130385] Avg episode reward: [(0, '63.610'), (1, '78.010')] [2023-10-08 06:35:54,262][00612] Updated weights for policy 1, policy_version 68650 (0.0009) [2023-10-08 06:35:54,637][00612] Updated weights for policy 1, policy_version 68660 (0.0009) [2023-10-08 06:35:54,994][00612] Updated weights for policy 1, policy_version 68670 (0.0008) [2023-10-08 06:35:55,467][00611] Updated weights for policy 0, policy_version 68262 (0.0010) [2023-10-08 06:35:55,833][00611] Updated weights for policy 0, policy_version 68272 (0.0009) [2023-10-08 06:35:56,210][00611] Updated weights for policy 0, policy_version 68282 (0.0010) [2023-10-08 06:35:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 140247040. Throughput: 0: 1851.0, 1: 1846.9. Samples: 35072260. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:35:58,754][130385] Avg episode reward: [(0, '60.030'), (1, '71.530')] [2023-10-08 06:35:58,755][00612] Updated weights for policy 1, policy_version 68680 (0.0010) [2023-10-08 06:35:59,120][00612] Updated weights for policy 1, policy_version 68690 (0.0010) [2023-10-08 06:35:59,492][00612] Updated weights for policy 1, policy_version 68700 (0.0009) [2023-10-08 06:35:59,652][00611] Updated weights for policy 0, policy_version 68292 (0.0010) [2023-10-08 06:36:00,021][00611] Updated weights for policy 0, policy_version 68302 (0.0009) [2023-10-08 06:36:00,398][00611] Updated weights for policy 0, policy_version 68312 (0.0009) [2023-10-08 06:36:02,909][00612] Updated weights for policy 1, policy_version 68710 (0.0007) [2023-10-08 06:36:03,268][00612] Updated weights for policy 1, policy_version 68720 (0.0007) [2023-10-08 06:36:03,632][00612] Updated weights for policy 1, policy_version 68730 (0.0008) [2023-10-08 06:36:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 140312576. Throughput: 0: 1856.0, 1: 1832.5. Samples: 35095174. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:36:03,754][130385] Avg episode reward: [(0, '58.720'), (1, '69.410')] [2023-10-08 06:36:03,977][00611] Updated weights for policy 0, policy_version 68322 (0.0010) [2023-10-08 06:36:04,348][00611] Updated weights for policy 0, policy_version 68332 (0.0009) [2023-10-08 06:36:04,729][00611] Updated weights for policy 0, policy_version 68342 (0.0008) [2023-10-08 06:36:05,093][00611] Updated weights for policy 0, policy_version 68352 (0.0007) [2023-10-08 06:36:07,313][00612] Updated weights for policy 1, policy_version 68740 (0.0010) [2023-10-08 06:36:07,685][00612] Updated weights for policy 1, policy_version 68750 (0.0009) [2023-10-08 06:36:08,055][00612] Updated weights for policy 1, policy_version 68760 (0.0008) [2023-10-08 06:36:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 140410880. Throughput: 0: 1860.1, 1: 1845.0. Samples: 35105830. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:36:08,754][130385] Avg episode reward: [(0, '55.910'), (1, '67.340')] [2023-10-08 06:36:08,828][00611] Updated weights for policy 0, policy_version 68362 (0.0009) [2023-10-08 06:36:09,194][00611] Updated weights for policy 0, policy_version 68372 (0.0008) [2023-10-08 06:36:09,567][00611] Updated weights for policy 0, policy_version 68382 (0.0009) [2023-10-08 06:36:11,715][00612] Updated weights for policy 1, policy_version 68770 (0.0008) [2023-10-08 06:36:12,090][00612] Updated weights for policy 1, policy_version 68780 (0.0007) [2023-10-08 06:36:12,448][00612] Updated weights for policy 1, policy_version 68790 (0.0007) [2023-10-08 06:36:12,819][00612] Updated weights for policy 1, policy_version 68800 (0.0010) [2023-10-08 06:36:13,188][00611] Updated weights for policy 0, policy_version 68392 (0.0007) [2023-10-08 06:36:13,564][00611] Updated weights for policy 0, policy_version 68402 (0.0008) [2023-10-08 06:36:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 140476416. Throughput: 0: 1864.1, 1: 1833.9. Samples: 35128558. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:36:13,754][130385] Avg episode reward: [(0, '57.640'), (1, '71.390')] [2023-10-08 06:36:13,930][00611] Updated weights for policy 0, policy_version 68412 (0.0007) [2023-10-08 06:36:16,466][00612] Updated weights for policy 1, policy_version 68810 (0.0008) [2023-10-08 06:36:16,836][00612] Updated weights for policy 1, policy_version 68820 (0.0008) [2023-10-08 06:36:17,209][00612] Updated weights for policy 1, policy_version 68830 (0.0008) [2023-10-08 06:36:17,496][00611] Updated weights for policy 0, policy_version 68422 (0.0009) [2023-10-08 06:36:17,868][00611] Updated weights for policy 0, policy_version 68432 (0.0010) [2023-10-08 06:36:18,235][00611] Updated weights for policy 0, policy_version 68442 (0.0008) [2023-10-08 06:36:18,754][130385] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 140574720. Throughput: 0: 1845.7, 1: 1843.9. Samples: 35149936. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:36:18,755][130385] Avg episode reward: [(0, '57.480'), (1, '72.030')] [2023-10-08 06:36:18,769][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000068448_70090752.pth... [2023-10-08 06:36:18,769][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000068832_70483968.pth... [2023-10-08 06:36:18,803][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000066720_68321280.pth [2023-10-08 06:36:18,807][00365] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p0/milestones/checkpoint_000068448_70090752.pth [2023-10-08 06:36:18,811][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000067104_68714496.pth [2023-10-08 06:36:18,815][00425] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p1/milestones/checkpoint_000068832_70483968.pth [2023-10-08 06:36:20,797][00612] Updated weights for policy 1, policy_version 68840 (0.0008) [2023-10-08 06:36:21,164][00612] Updated weights for policy 1, policy_version 68850 (0.0010) [2023-10-08 06:36:21,540][00612] Updated weights for policy 1, policy_version 68860 (0.0011) [2023-10-08 06:36:21,782][00611] Updated weights for policy 0, policy_version 68452 (0.0008) [2023-10-08 06:36:22,151][00611] Updated weights for policy 0, policy_version 68462 (0.0007) [2023-10-08 06:36:22,521][00611] Updated weights for policy 0, policy_version 68472 (0.0007) [2023-10-08 06:36:23,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 140640256. Throughput: 0: 1864.5, 1: 1835.3. Samples: 35161692. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:36:23,755][130385] Avg episode reward: [(0, '55.940'), (1, '71.380')] [2023-10-08 06:36:25,166][00612] Updated weights for policy 1, policy_version 68870 (0.0008) [2023-10-08 06:36:25,537][00612] Updated weights for policy 1, policy_version 68880 (0.0008) [2023-10-08 06:36:25,904][00612] Updated weights for policy 1, policy_version 68890 (0.0008) [2023-10-08 06:36:26,170][00611] Updated weights for policy 0, policy_version 68482 (0.0008) [2023-10-08 06:36:26,541][00611] Updated weights for policy 0, policy_version 68492 (0.0007) [2023-10-08 06:36:26,912][00611] Updated weights for policy 0, policy_version 68502 (0.0007) [2023-10-08 06:36:27,276][00611] Updated weights for policy 0, policy_version 68512 (0.0009) [2023-10-08 06:36:28,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140705792. Throughput: 0: 1840.7, 1: 1863.1. Samples: 35183206. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:36:28,755][130385] Avg episode reward: [(0, '54.810'), (1, '69.200')] [2023-10-08 06:36:29,532][00612] Updated weights for policy 1, policy_version 68900 (0.0007) [2023-10-08 06:36:29,898][00612] Updated weights for policy 1, policy_version 68910 (0.0009) [2023-10-08 06:36:30,273][00612] Updated weights for policy 1, policy_version 68920 (0.0011) [2023-10-08 06:36:30,894][00611] Updated weights for policy 0, policy_version 68522 (0.0009) [2023-10-08 06:36:31,274][00611] Updated weights for policy 0, policy_version 68532 (0.0008) [2023-10-08 06:36:31,647][00611] Updated weights for policy 0, policy_version 68542 (0.0007) [2023-10-08 06:36:33,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140771328. Throughput: 0: 1867.4, 1: 1859.0. Samples: 35206334. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:36:33,754][130385] Avg episode reward: [(0, '58.930'), (1, '71.150')] [2023-10-08 06:36:33,894][00612] Updated weights for policy 1, policy_version 68930 (0.0008) [2023-10-08 06:36:34,254][00612] Updated weights for policy 1, policy_version 68940 (0.0007) [2023-10-08 06:36:34,620][00612] Updated weights for policy 1, policy_version 68950 (0.0008) [2023-10-08 06:36:34,987][00612] Updated weights for policy 1, policy_version 68960 (0.0010) [2023-10-08 06:36:35,182][00611] Updated weights for policy 0, policy_version 68552 (0.0008) [2023-10-08 06:36:35,547][00611] Updated weights for policy 0, policy_version 68562 (0.0010) [2023-10-08 06:36:35,929][00611] Updated weights for policy 0, policy_version 68572 (0.0007) [2023-10-08 06:36:38,605][00612] Updated weights for policy 1, policy_version 68970 (0.0007) [2023-10-08 06:36:38,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 140836864. Throughput: 0: 1842.7, 1: 1859.7. Samples: 35216554. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:36:38,754][130385] Avg episode reward: [(0, '59.350'), (1, '74.380')] [2023-10-08 06:36:38,962][00612] Updated weights for policy 1, policy_version 68980 (0.0007) [2023-10-08 06:36:39,337][00612] Updated weights for policy 1, policy_version 68990 (0.0008) [2023-10-08 06:36:39,506][00611] Updated weights for policy 0, policy_version 68582 (0.0009) [2023-10-08 06:36:39,874][00611] Updated weights for policy 0, policy_version 68592 (0.0010) [2023-10-08 06:36:40,242][00611] Updated weights for policy 0, policy_version 68602 (0.0009) [2023-10-08 06:36:42,990][00612] Updated weights for policy 1, policy_version 69000 (0.0007) [2023-10-08 06:36:43,362][00612] Updated weights for policy 1, policy_version 69010 (0.0008) [2023-10-08 06:36:43,739][00612] Updated weights for policy 1, policy_version 69020 (0.0010) [2023-10-08 06:36:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 140902400. Throughput: 0: 1866.3, 1: 1858.0. Samples: 35239852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:36:43,754][130385] Avg episode reward: [(0, '58.380'), (1, '75.920')] [2023-10-08 06:36:43,777][00611] Updated weights for policy 0, policy_version 68612 (0.0009) [2023-10-08 06:36:44,152][00611] Updated weights for policy 0, policy_version 68622 (0.0010) [2023-10-08 06:36:44,516][00611] Updated weights for policy 0, policy_version 68632 (0.0008) [2023-10-08 06:36:47,262][00612] Updated weights for policy 1, policy_version 69030 (0.0009) [2023-10-08 06:36:47,640][00612] Updated weights for policy 1, policy_version 69040 (0.0008) [2023-10-08 06:36:48,011][00612] Updated weights for policy 1, policy_version 69050 (0.0009) [2023-10-08 06:36:48,208][00611] Updated weights for policy 0, policy_version 68642 (0.0008) [2023-10-08 06:36:48,587][00611] Updated weights for policy 0, policy_version 68652 (0.0009) [2023-10-08 06:36:48,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 141000704. Throughput: 0: 1858.8, 1: 1840.7. Samples: 35261648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:36:48,754][130385] Avg episode reward: [(0, '60.510'), (1, '76.160')] [2023-10-08 06:36:48,961][00611] Updated weights for policy 0, policy_version 68662 (0.0009) [2023-10-08 06:36:49,325][00611] Updated weights for policy 0, policy_version 68672 (0.0009) [2023-10-08 06:36:51,666][00612] Updated weights for policy 1, policy_version 69060 (0.0008) [2023-10-08 06:36:52,039][00612] Updated weights for policy 1, policy_version 69070 (0.0008) [2023-10-08 06:36:52,406][00612] Updated weights for policy 1, policy_version 69080 (0.0007) [2023-10-08 06:36:53,012][00611] Updated weights for policy 0, policy_version 68682 (0.0009) [2023-10-08 06:36:53,390][00611] Updated weights for policy 0, policy_version 68692 (0.0007) [2023-10-08 06:36:53,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 141066240. Throughput: 0: 1857.3, 1: 1862.7. Samples: 35273228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:36:53,754][130385] Avg episode reward: [(0, '60.470'), (1, '69.240')] [2023-10-08 06:36:53,766][00611] Updated weights for policy 0, policy_version 68702 (0.0008) [2023-10-08 06:36:56,011][00612] Updated weights for policy 1, policy_version 69090 (0.0008) [2023-10-08 06:36:56,372][00612] Updated weights for policy 1, policy_version 69100 (0.0009) [2023-10-08 06:36:56,740][00612] Updated weights for policy 1, policy_version 69110 (0.0008) [2023-10-08 06:36:57,096][00612] Updated weights for policy 1, policy_version 69120 (0.0008) [2023-10-08 06:36:57,396][00611] Updated weights for policy 0, policy_version 68712 (0.0008) [2023-10-08 06:36:57,770][00611] Updated weights for policy 0, policy_version 68722 (0.0008) [2023-10-08 06:36:58,141][00611] Updated weights for policy 0, policy_version 68732 (0.0008) [2023-10-08 06:36:58,754][130385] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 141164544. Throughput: 0: 1852.3, 1: 1844.4. Samples: 35294912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:36:58,755][130385] Avg episode reward: [(0, '58.670'), (1, '70.210')] [2023-10-08 06:37:00,726][00612] Updated weights for policy 1, policy_version 69130 (0.0008) [2023-10-08 06:37:01,093][00612] Updated weights for policy 1, policy_version 69140 (0.0007) [2023-10-08 06:37:01,459][00612] Updated weights for policy 1, policy_version 69150 (0.0007) [2023-10-08 06:37:01,691][00611] Updated weights for policy 0, policy_version 68742 (0.0011) [2023-10-08 06:37:02,077][00611] Updated weights for policy 0, policy_version 68752 (0.0010) [2023-10-08 06:37:02,447][00611] Updated weights for policy 0, policy_version 68762 (0.0009) [2023-10-08 06:37:03,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 141230080. Throughput: 0: 1833.6, 1: 1863.7. Samples: 35316316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:37:03,755][130385] Avg episode reward: [(0, '63.110'), (1, '71.640')] [2023-10-08 06:37:05,010][00612] Updated weights for policy 1, policy_version 69160 (0.0007) [2023-10-08 06:37:05,378][00612] Updated weights for policy 1, policy_version 69170 (0.0009) [2023-10-08 06:37:05,748][00612] Updated weights for policy 1, policy_version 69180 (0.0008) [2023-10-08 06:37:06,128][00611] Updated weights for policy 0, policy_version 68772 (0.0011) [2023-10-08 06:37:06,489][00611] Updated weights for policy 0, policy_version 68782 (0.0010) [2023-10-08 06:37:06,862][00611] Updated weights for policy 0, policy_version 68792 (0.0010) [2023-10-08 06:37:08,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141295616. Throughput: 0: 1841.1, 1: 1844.4. Samples: 35327536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:37:08,754][130385] Avg episode reward: [(0, '63.540'), (1, '70.960')] [2023-10-08 06:37:09,385][00612] Updated weights for policy 1, policy_version 69190 (0.0008) [2023-10-08 06:37:09,746][00612] Updated weights for policy 1, policy_version 69200 (0.0008) [2023-10-08 06:37:10,122][00612] Updated weights for policy 1, policy_version 69210 (0.0008) [2023-10-08 06:37:10,582][00611] Updated weights for policy 0, policy_version 68802 (0.0010) [2023-10-08 06:37:10,942][00611] Updated weights for policy 0, policy_version 68812 (0.0008) [2023-10-08 06:37:11,317][00611] Updated weights for policy 0, policy_version 68822 (0.0009) [2023-10-08 06:37:11,687][00611] Updated weights for policy 0, policy_version 68832 (0.0008) [2023-10-08 06:37:13,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141361152. Throughput: 0: 1836.0, 1: 1855.7. Samples: 35349334. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:37:13,754][130385] Avg episode reward: [(0, '64.200'), (1, '68.630')] [2023-10-08 06:37:13,788][00612] Updated weights for policy 1, policy_version 69220 (0.0007) [2023-10-08 06:37:14,160][00612] Updated weights for policy 1, policy_version 69230 (0.0008) [2023-10-08 06:37:14,532][00612] Updated weights for policy 1, policy_version 69240 (0.0007) [2023-10-08 06:37:15,370][00611] Updated weights for policy 0, policy_version 68842 (0.0008) [2023-10-08 06:37:15,733][00611] Updated weights for policy 0, policy_version 68852 (0.0008) [2023-10-08 06:37:16,107][00611] Updated weights for policy 0, policy_version 68862 (0.0009) [2023-10-08 06:37:18,202][00612] Updated weights for policy 1, policy_version 69250 (0.0007) [2023-10-08 06:37:18,573][00612] Updated weights for policy 1, policy_version 69260 (0.0007) [2023-10-08 06:37:18,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 141426688. Throughput: 0: 1843.8, 1: 1847.9. Samples: 35372464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:37:18,755][130385] Avg episode reward: [(0, '68.440'), (1, '71.790')] [2023-10-08 06:37:18,928][00612] Updated weights for policy 1, policy_version 69270 (0.0009) [2023-10-08 06:37:19,295][00612] Updated weights for policy 1, policy_version 69280 (0.0012) [2023-10-08 06:37:19,761][00611] Updated weights for policy 0, policy_version 68872 (0.0008) [2023-10-08 06:37:20,138][00611] Updated weights for policy 0, policy_version 68882 (0.0007) [2023-10-08 06:37:20,520][00611] Updated weights for policy 0, policy_version 68892 (0.0009) [2023-10-08 06:37:22,831][00612] Updated weights for policy 1, policy_version 69290 (0.0009) [2023-10-08 06:37:23,204][00612] Updated weights for policy 1, policy_version 69300 (0.0010) [2023-10-08 06:37:23,569][00612] Updated weights for policy 1, policy_version 69310 (0.0007) [2023-10-08 06:37:23,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 141524992. Throughput: 0: 1842.8, 1: 1853.5. Samples: 35382888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:37:23,754][130385] Avg episode reward: [(0, '67.280'), (1, '72.920')] [2023-10-08 06:37:24,289][00611] Updated weights for policy 0, policy_version 68902 (0.0009) [2023-10-08 06:37:24,663][00611] Updated weights for policy 0, policy_version 68912 (0.0009) [2023-10-08 06:37:25,038][00611] Updated weights for policy 0, policy_version 68922 (0.0008) [2023-10-08 06:37:27,052][00612] Updated weights for policy 1, policy_version 69320 (0.0007) [2023-10-08 06:37:27,432][00612] Updated weights for policy 1, policy_version 69330 (0.0007) [2023-10-08 06:37:27,795][00612] Updated weights for policy 1, policy_version 69340 (0.0007) [2023-10-08 06:37:28,566][00611] Updated weights for policy 0, policy_version 68932 (0.0010) [2023-10-08 06:37:28,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 141590528. Throughput: 0: 1841.1, 1: 1843.1. Samples: 35405644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:37:28,755][130385] Avg episode reward: [(0, '67.490'), (1, '70.360')] [2023-10-08 06:37:28,943][00611] Updated weights for policy 0, policy_version 68942 (0.0009) [2023-10-08 06:37:29,321][00611] Updated weights for policy 0, policy_version 68952 (0.0008) [2023-10-08 06:37:31,432][00612] Updated weights for policy 1, policy_version 69350 (0.0007) [2023-10-08 06:37:31,809][00612] Updated weights for policy 1, policy_version 69360 (0.0007) [2023-10-08 06:37:32,174][00612] Updated weights for policy 1, policy_version 69370 (0.0008) [2023-10-08 06:37:32,990][00611] Updated weights for policy 0, policy_version 68962 (0.0009) [2023-10-08 06:37:33,372][00611] Updated weights for policy 0, policy_version 68972 (0.0011) [2023-10-08 06:37:33,741][00611] Updated weights for policy 0, policy_version 68982 (0.0010) [2023-10-08 06:37:33,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 141656064. Throughput: 0: 1829.3, 1: 1854.4. Samples: 35427416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:37:33,755][130385] Avg episode reward: [(0, '66.410'), (1, '72.440')] [2023-10-08 06:37:34,111][00611] Updated weights for policy 0, policy_version 68992 (0.0010) [2023-10-08 06:37:35,752][00612] Updated weights for policy 1, policy_version 69380 (0.0008) [2023-10-08 06:37:36,136][00612] Updated weights for policy 1, policy_version 69390 (0.0007) [2023-10-08 06:37:36,502][00612] Updated weights for policy 1, policy_version 69400 (0.0007) [2023-10-08 06:37:37,781][00611] Updated weights for policy 0, policy_version 69002 (0.0010) [2023-10-08 06:37:38,160][00611] Updated weights for policy 0, policy_version 69012 (0.0009) [2023-10-08 06:37:38,530][00611] Updated weights for policy 0, policy_version 69022 (0.0009) [2023-10-08 06:37:38,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 141754368. Throughput: 0: 1839.3, 1: 1838.7. Samples: 35438738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:37:38,755][130385] Avg episode reward: [(0, '68.520'), (1, '67.270')] [2023-10-08 06:37:40,130][00612] Updated weights for policy 1, policy_version 69410 (0.0007) [2023-10-08 06:37:40,494][00612] Updated weights for policy 1, policy_version 69420 (0.0009) [2023-10-08 06:37:40,863][00612] Updated weights for policy 1, policy_version 69430 (0.0011) [2023-10-08 06:37:41,234][00612] Updated weights for policy 1, policy_version 69440 (0.0009) [2023-10-08 06:37:42,230][00611] Updated weights for policy 0, policy_version 69032 (0.0009) [2023-10-08 06:37:42,601][00611] Updated weights for policy 0, policy_version 69042 (0.0009) [2023-10-08 06:37:42,975][00611] Updated weights for policy 0, policy_version 69052 (0.0007) [2023-10-08 06:37:43,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 141819904. Throughput: 0: 1834.1, 1: 1849.8. Samples: 35460688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:37:43,755][130385] Avg episode reward: [(0, '66.030'), (1, '68.220')] [2023-10-08 06:37:44,895][00612] Updated weights for policy 1, policy_version 69450 (0.0011) [2023-10-08 06:37:45,257][00612] Updated weights for policy 1, policy_version 69460 (0.0011) [2023-10-08 06:37:45,618][00612] Updated weights for policy 1, policy_version 69470 (0.0008) [2023-10-08 06:37:46,710][00611] Updated weights for policy 0, policy_version 69062 (0.0008) [2023-10-08 06:37:47,081][00611] Updated weights for policy 0, policy_version 69072 (0.0008) [2023-10-08 06:37:47,464][00611] Updated weights for policy 0, policy_version 69082 (0.0007) [2023-10-08 06:37:48,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141885440. Throughput: 0: 1837.8, 1: 1848.6. Samples: 35482204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:37:48,754][130385] Avg episode reward: [(0, '66.440'), (1, '65.690')] [2023-10-08 06:37:49,307][00612] Updated weights for policy 1, policy_version 69480 (0.0009) [2023-10-08 06:37:49,677][00612] Updated weights for policy 1, policy_version 69490 (0.0008) [2023-10-08 06:37:50,040][00612] Updated weights for policy 1, policy_version 69500 (0.0009) [2023-10-08 06:37:51,043][00611] Updated weights for policy 0, policy_version 69092 (0.0010) [2023-10-08 06:37:51,405][00611] Updated weights for policy 0, policy_version 69102 (0.0008) [2023-10-08 06:37:51,777][00611] Updated weights for policy 0, policy_version 69112 (0.0010) [2023-10-08 06:37:53,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141950976. Throughput: 0: 1840.0, 1: 1850.8. Samples: 35493622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:37:53,754][130385] Avg episode reward: [(0, '67.180'), (1, '70.380')] [2023-10-08 06:37:53,766][00612] Updated weights for policy 1, policy_version 69510 (0.0009) [2023-10-08 06:37:54,132][00612] Updated weights for policy 1, policy_version 69520 (0.0009) [2023-10-08 06:37:54,502][00612] Updated weights for policy 1, policy_version 69530 (0.0008) [2023-10-08 06:37:55,299][00611] Updated weights for policy 0, policy_version 69122 (0.0009) [2023-10-08 06:37:55,664][00611] Updated weights for policy 0, policy_version 69132 (0.0008) [2023-10-08 06:37:56,034][00611] Updated weights for policy 0, policy_version 69142 (0.0007) [2023-10-08 06:37:56,402][00611] Updated weights for policy 0, policy_version 69152 (0.0008) [2023-10-08 06:37:58,149][00612] Updated weights for policy 1, policy_version 69540 (0.0008) [2023-10-08 06:37:58,516][00612] Updated weights for policy 1, policy_version 69550 (0.0007) [2023-10-08 06:37:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 142016512. Throughput: 0: 1847.2, 1: 1848.8. Samples: 35515656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:37:58,754][130385] Avg episode reward: [(0, '68.520'), (1, '70.680')] [2023-10-08 06:37:58,880][00612] Updated weights for policy 1, policy_version 69560 (0.0008) [2023-10-08 06:38:00,122][00611] Updated weights for policy 0, policy_version 69162 (0.0008) [2023-10-08 06:38:00,492][00611] Updated weights for policy 0, policy_version 69172 (0.0008) [2023-10-08 06:38:00,870][00611] Updated weights for policy 0, policy_version 69182 (0.0009) [2023-10-08 06:38:02,721][00612] Updated weights for policy 1, policy_version 69570 (0.0008) [2023-10-08 06:38:03,090][00612] Updated weights for policy 1, policy_version 69580 (0.0008) [2023-10-08 06:38:03,461][00612] Updated weights for policy 1, policy_version 69590 (0.0008) [2023-10-08 06:38:03,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 142082048. Throughput: 0: 1844.3, 1: 1834.0. Samples: 35537986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:38:03,755][130385] Avg episode reward: [(0, '68.820'), (1, '70.990')] [2023-10-08 06:38:03,827][00612] Updated weights for policy 1, policy_version 69600 (0.0008) [2023-10-08 06:38:04,543][00611] Updated weights for policy 0, policy_version 69192 (0.0007) [2023-10-08 06:38:04,917][00611] Updated weights for policy 0, policy_version 69202 (0.0008) [2023-10-08 06:38:05,289][00611] Updated weights for policy 0, policy_version 69212 (0.0008) [2023-10-08 06:38:07,501][00612] Updated weights for policy 1, policy_version 69610 (0.0008) [2023-10-08 06:38:07,865][00612] Updated weights for policy 1, policy_version 69620 (0.0012) [2023-10-08 06:38:08,235][00612] Updated weights for policy 1, policy_version 69630 (0.0011) [2023-10-08 06:38:08,670][00611] Updated weights for policy 0, policy_version 69222 (0.0008) [2023-10-08 06:38:08,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 142180352. Throughput: 0: 1840.6, 1: 1843.2. Samples: 35548658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:38:08,755][130385] Avg episode reward: [(0, '68.810'), (1, '71.560')] [2023-10-08 06:38:09,050][00611] Updated weights for policy 0, policy_version 69232 (0.0008) [2023-10-08 06:38:09,425][00611] Updated weights for policy 0, policy_version 69242 (0.0008) [2023-10-08 06:38:11,783][00612] Updated weights for policy 1, policy_version 69640 (0.0008) [2023-10-08 06:38:12,147][00612] Updated weights for policy 1, policy_version 69650 (0.0010) [2023-10-08 06:38:12,519][00612] Updated weights for policy 1, policy_version 69660 (0.0010) [2023-10-08 06:38:13,019][00611] Updated weights for policy 0, policy_version 69252 (0.0008) [2023-10-08 06:38:13,386][00611] Updated weights for policy 0, policy_version 69262 (0.0009) [2023-10-08 06:38:13,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 142245888. Throughput: 0: 1835.9, 1: 1834.4. Samples: 35570808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:38:13,754][130385] Avg episode reward: [(0, '69.320'), (1, '72.450')] [2023-10-08 06:38:13,768][00611] Updated weights for policy 0, policy_version 69272 (0.0011) [2023-10-08 06:38:16,108][00612] Updated weights for policy 1, policy_version 69670 (0.0011) [2023-10-08 06:38:16,478][00612] Updated weights for policy 1, policy_version 69680 (0.0008) [2023-10-08 06:38:16,844][00612] Updated weights for policy 1, policy_version 69690 (0.0008) [2023-10-08 06:38:17,428][00611] Updated weights for policy 0, policy_version 69282 (0.0010) [2023-10-08 06:38:17,809][00611] Updated weights for policy 0, policy_version 69292 (0.0007) [2023-10-08 06:38:18,187][00611] Updated weights for policy 0, policy_version 69302 (0.0008) [2023-10-08 06:38:18,546][00611] Updated weights for policy 0, policy_version 69312 (0.0008) [2023-10-08 06:38:18,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14884.5). Total num frames: 142344192. Throughput: 0: 1830.1, 1: 1838.3. Samples: 35592494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:38:18,754][130385] Avg episode reward: [(0, '72.680'), (1, '74.840')] [2023-10-08 06:38:18,762][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000069696_71368704.pth... [2023-10-08 06:38:18,763][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000069312_70975488.pth... [2023-10-08 06:38:18,802][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000067968_69599232.pth [2023-10-08 06:38:18,803][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000067584_69206016.pth [2023-10-08 06:38:20,465][00612] Updated weights for policy 1, policy_version 69700 (0.0008) [2023-10-08 06:38:20,835][00612] Updated weights for policy 1, policy_version 69710 (0.0008) [2023-10-08 06:38:21,202][00612] Updated weights for policy 1, policy_version 69720 (0.0008) [2023-10-08 06:38:22,146][00611] Updated weights for policy 0, policy_version 69322 (0.0010) [2023-10-08 06:38:22,515][00611] Updated weights for policy 0, policy_version 69332 (0.0010) [2023-10-08 06:38:22,879][00611] Updated weights for policy 0, policy_version 69342 (0.0009) [2023-10-08 06:38:23,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 142409728. Throughput: 0: 1847.6, 1: 1830.2. Samples: 35604242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:38:23,754][130385] Avg episode reward: [(0, '71.290'), (1, '73.920')] [2023-10-08 06:38:24,789][00612] Updated weights for policy 1, policy_version 69730 (0.0007) [2023-10-08 06:38:25,153][00612] Updated weights for policy 1, policy_version 69740 (0.0008) [2023-10-08 06:38:25,523][00612] Updated weights for policy 1, policy_version 69750 (0.0008) [2023-10-08 06:38:25,892][00612] Updated weights for policy 1, policy_version 69760 (0.0009) [2023-10-08 06:38:26,509][00611] Updated weights for policy 0, policy_version 69352 (0.0007) [2023-10-08 06:38:26,879][00611] Updated weights for policy 0, policy_version 69362 (0.0008) [2023-10-08 06:38:27,244][00611] Updated weights for policy 0, policy_version 69372 (0.0009) [2023-10-08 06:38:28,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 142475264. Throughput: 0: 1828.4, 1: 1836.7. Samples: 35625618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:38:28,755][130385] Avg episode reward: [(0, '70.730'), (1, '73.710')] [2023-10-08 06:38:29,585][00612] Updated weights for policy 1, policy_version 69770 (0.0008) [2023-10-08 06:38:29,966][00612] Updated weights for policy 1, policy_version 69780 (0.0008) [2023-10-08 06:38:30,344][00612] Updated weights for policy 1, policy_version 69790 (0.0009) [2023-10-08 06:38:30,853][00611] Updated weights for policy 0, policy_version 69382 (0.0010) [2023-10-08 06:38:31,224][00611] Updated weights for policy 0, policy_version 69392 (0.0009) [2023-10-08 06:38:31,591][00611] Updated weights for policy 0, policy_version 69402 (0.0009) [2023-10-08 06:38:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 142540800. Throughput: 0: 1852.5, 1: 1838.0. Samples: 35648276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:38:33,754][130385] Avg episode reward: [(0, '74.670'), (1, '73.540')] [2023-10-08 06:38:34,036][00612] Updated weights for policy 1, policy_version 69800 (0.0007) [2023-10-08 06:38:34,404][00612] Updated weights for policy 1, policy_version 69810 (0.0009) [2023-10-08 06:38:34,774][00612] Updated weights for policy 1, policy_version 69820 (0.0008) [2023-10-08 06:38:35,209][00611] Updated weights for policy 0, policy_version 69412 (0.0007) [2023-10-08 06:38:35,576][00611] Updated weights for policy 0, policy_version 69422 (0.0009) [2023-10-08 06:38:35,951][00611] Updated weights for policy 0, policy_version 69432 (0.0010) [2023-10-08 06:38:38,301][00612] Updated weights for policy 1, policy_version 69830 (0.0007) [2023-10-08 06:38:38,674][00612] Updated weights for policy 1, policy_version 69840 (0.0009) [2023-10-08 06:38:38,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 142606336. Throughput: 0: 1826.8, 1: 1840.6. Samples: 35658656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:38:38,754][130385] Avg episode reward: [(0, '68.060'), (1, '72.670')] [2023-10-08 06:38:39,037][00612] Updated weights for policy 1, policy_version 69850 (0.0010) [2023-10-08 06:38:39,683][00611] Updated weights for policy 0, policy_version 69442 (0.0008) [2023-10-08 06:38:40,055][00611] Updated weights for policy 0, policy_version 69452 (0.0009) [2023-10-08 06:38:40,423][00611] Updated weights for policy 0, policy_version 69462 (0.0007) [2023-10-08 06:38:40,794][00611] Updated weights for policy 0, policy_version 69472 (0.0007) [2023-10-08 06:38:42,572][00612] Updated weights for policy 1, policy_version 69860 (0.0009) [2023-10-08 06:38:42,936][00612] Updated weights for policy 1, policy_version 69870 (0.0011) [2023-10-08 06:38:43,300][00612] Updated weights for policy 1, policy_version 69880 (0.0011) [2023-10-08 06:38:43,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 142704640. Throughput: 0: 1844.3, 1: 1846.2. Samples: 35681730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:38:43,755][130385] Avg episode reward: [(0, '69.270'), (1, '73.540')] [2023-10-08 06:38:44,337][00611] Updated weights for policy 0, policy_version 69482 (0.0011) [2023-10-08 06:38:44,701][00611] Updated weights for policy 0, policy_version 69492 (0.0007) [2023-10-08 06:38:45,075][00611] Updated weights for policy 0, policy_version 69502 (0.0009) [2023-10-08 06:38:46,972][00612] Updated weights for policy 1, policy_version 69890 (0.0009) [2023-10-08 06:38:47,331][00612] Updated weights for policy 1, policy_version 69900 (0.0011) [2023-10-08 06:38:47,690][00612] Updated weights for policy 1, policy_version 69910 (0.0008) [2023-10-08 06:38:48,058][00612] Updated weights for policy 1, policy_version 69920 (0.0008) [2023-10-08 06:38:48,711][00611] Updated weights for policy 0, policy_version 69512 (0.0008) [2023-10-08 06:38:48,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 142770176. Throughput: 0: 1846.8, 1: 1827.7. Samples: 35703336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:38:48,754][130385] Avg episode reward: [(0, '65.840'), (1, '72.390')] [2023-10-08 06:38:49,093][00611] Updated weights for policy 0, policy_version 69522 (0.0009) [2023-10-08 06:38:49,461][00611] Updated weights for policy 0, policy_version 69532 (0.0008) [2023-10-08 06:38:51,758][00612] Updated weights for policy 1, policy_version 69930 (0.0008) [2023-10-08 06:38:52,136][00612] Updated weights for policy 1, policy_version 69940 (0.0010) [2023-10-08 06:38:52,512][00612] Updated weights for policy 1, policy_version 69950 (0.0009) [2023-10-08 06:38:53,191][00611] Updated weights for policy 0, policy_version 69542 (0.0009) [2023-10-08 06:38:53,570][00611] Updated weights for policy 0, policy_version 69552 (0.0010) [2023-10-08 06:38:53,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 142835712. Throughput: 0: 1848.0, 1: 1848.8. Samples: 35715012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:38:53,754][130385] Avg episode reward: [(0, '62.190'), (1, '73.370')] [2023-10-08 06:38:53,945][00611] Updated weights for policy 0, policy_version 69562 (0.0009) [2023-10-08 06:38:56,137][00612] Updated weights for policy 1, policy_version 69960 (0.0009) [2023-10-08 06:38:56,500][00612] Updated weights for policy 1, policy_version 69970 (0.0009) [2023-10-08 06:38:56,868][00612] Updated weights for policy 1, policy_version 69980 (0.0007) [2023-10-08 06:38:57,548][00611] Updated weights for policy 0, policy_version 69572 (0.0008) [2023-10-08 06:38:57,914][00611] Updated weights for policy 0, policy_version 69582 (0.0008) [2023-10-08 06:38:58,280][00611] Updated weights for policy 0, policy_version 69592 (0.0007) [2023-10-08 06:38:58,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14884.4). Total num frames: 142934016. Throughput: 0: 1852.6, 1: 1829.0. Samples: 35736482. Policy #0 lag: (min: 31.0, avg: 32.7, max: 56.0) [2023-10-08 06:38:58,755][130385] Avg episode reward: [(0, '65.210'), (1, '71.470')] [2023-10-08 06:39:00,364][00612] Updated weights for policy 1, policy_version 69990 (0.0009) [2023-10-08 06:39:00,729][00612] Updated weights for policy 1, policy_version 70000 (0.0008) [2023-10-08 06:39:01,099][00612] Updated weights for policy 1, policy_version 70010 (0.0011) [2023-10-08 06:39:01,837][00611] Updated weights for policy 0, policy_version 69602 (0.0009) [2023-10-08 06:39:02,207][00611] Updated weights for policy 0, policy_version 69612 (0.0009) [2023-10-08 06:39:02,571][00611] Updated weights for policy 0, policy_version 69622 (0.0007) [2023-10-08 06:39:02,950][00611] Updated weights for policy 0, policy_version 69632 (0.0007) [2023-10-08 06:39:03,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 142999552. Throughput: 0: 1834.2, 1: 1852.8. Samples: 35758408. Policy #0 lag: (min: 31.0, avg: 32.7, max: 56.0) [2023-10-08 06:39:03,754][130385] Avg episode reward: [(0, '61.970'), (1, '71.960')] [2023-10-08 06:39:04,638][00612] Updated weights for policy 1, policy_version 70020 (0.0008) [2023-10-08 06:39:05,011][00612] Updated weights for policy 1, policy_version 70030 (0.0012) [2023-10-08 06:39:05,373][00612] Updated weights for policy 1, policy_version 70040 (0.0010) [2023-10-08 06:39:06,607][00611] Updated weights for policy 0, policy_version 69642 (0.0007) [2023-10-08 06:39:06,968][00611] Updated weights for policy 0, policy_version 69652 (0.0008) [2023-10-08 06:39:07,340][00611] Updated weights for policy 0, policy_version 69662 (0.0009) [2023-10-08 06:39:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 143065088. Throughput: 0: 1843.9, 1: 1842.6. Samples: 35770134. Policy #0 lag: (min: 31.0, avg: 32.7, max: 56.0) [2023-10-08 06:39:08,755][130385] Avg episode reward: [(0, '64.930'), (1, '75.980')] [2023-10-08 06:39:09,108][00612] Updated weights for policy 1, policy_version 70050 (0.0010) [2023-10-08 06:39:09,496][00612] Updated weights for policy 1, policy_version 70060 (0.0008) [2023-10-08 06:39:09,866][00612] Updated weights for policy 1, policy_version 70070 (0.0008) [2023-10-08 06:39:10,231][00612] Updated weights for policy 1, policy_version 70080 (0.0008) [2023-10-08 06:39:10,993][00611] Updated weights for policy 0, policy_version 69672 (0.0008) [2023-10-08 06:39:11,371][00611] Updated weights for policy 0, policy_version 69682 (0.0008) [2023-10-08 06:39:11,745][00611] Updated weights for policy 0, policy_version 69692 (0.0008) [2023-10-08 06:39:13,730][00612] Updated weights for policy 1, policy_version 70090 (0.0009) [2023-10-08 06:39:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 143130624. Throughput: 0: 1833.5, 1: 1857.7. Samples: 35791722. Policy #0 lag: (min: 31.0, avg: 32.7, max: 56.0) [2023-10-08 06:39:13,755][130385] Avg episode reward: [(0, '59.340'), (1, '75.810')] [2023-10-08 06:39:14,092][00612] Updated weights for policy 1, policy_version 70100 (0.0008) [2023-10-08 06:39:14,465][00612] Updated weights for policy 1, policy_version 70110 (0.0009) [2023-10-08 06:39:15,394][00611] Updated weights for policy 0, policy_version 69702 (0.0008) [2023-10-08 06:39:15,767][00611] Updated weights for policy 0, policy_version 69712 (0.0008) [2023-10-08 06:39:16,136][00611] Updated weights for policy 0, policy_version 69722 (0.0008) [2023-10-08 06:39:18,213][00612] Updated weights for policy 1, policy_version 70120 (0.0007) [2023-10-08 06:39:18,582][00612] Updated weights for policy 1, policy_version 70130 (0.0008) [2023-10-08 06:39:18,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 143196160. Throughput: 0: 1841.8, 1: 1851.9. Samples: 35814492. Policy #0 lag: (min: 31.0, avg: 32.7, max: 56.0) [2023-10-08 06:39:18,754][130385] Avg episode reward: [(0, '60.680'), (1, '73.220')] [2023-10-08 06:39:18,958][00612] Updated weights for policy 1, policy_version 70140 (0.0009) [2023-10-08 06:39:19,766][00611] Updated weights for policy 0, policy_version 69732 (0.0008) [2023-10-08 06:39:20,145][00611] Updated weights for policy 0, policy_version 69742 (0.0008) [2023-10-08 06:39:20,515][00611] Updated weights for policy 0, policy_version 69752 (0.0009) [2023-10-08 06:39:22,715][00612] Updated weights for policy 1, policy_version 70150 (0.0009) [2023-10-08 06:39:23,094][00612] Updated weights for policy 1, policy_version 70160 (0.0008) [2023-10-08 06:39:23,452][00612] Updated weights for policy 1, policy_version 70170 (0.0008) [2023-10-08 06:39:23,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 143294464. Throughput: 0: 1834.8, 1: 1856.9. Samples: 35824784. Policy #0 lag: (min: 31.0, avg: 32.7, max: 56.0) [2023-10-08 06:39:23,755][130385] Avg episode reward: [(0, '57.900'), (1, '68.890')] [2023-10-08 06:39:24,102][00611] Updated weights for policy 0, policy_version 69762 (0.0010) [2023-10-08 06:39:24,487][00611] Updated weights for policy 0, policy_version 69772 (0.0010) [2023-10-08 06:39:24,856][00611] Updated weights for policy 0, policy_version 69782 (0.0010) [2023-10-08 06:39:25,225][00611] Updated weights for policy 0, policy_version 69792 (0.0011) [2023-10-08 06:39:27,111][00612] Updated weights for policy 1, policy_version 70180 (0.0008) [2023-10-08 06:39:27,484][00612] Updated weights for policy 1, policy_version 70190 (0.0009) [2023-10-08 06:39:27,849][00612] Updated weights for policy 1, policy_version 70200 (0.0008) [2023-10-08 06:39:28,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 143360000. Throughput: 0: 1838.2, 1: 1847.9. Samples: 35847604. Policy #0 lag: (min: 31.0, avg: 32.7, max: 56.0) [2023-10-08 06:39:28,755][130385] Avg episode reward: [(0, '57.970'), (1, '69.470')] [2023-10-08 06:39:28,926][00611] Updated weights for policy 0, policy_version 69802 (0.0008) [2023-10-08 06:39:29,300][00611] Updated weights for policy 0, policy_version 69812 (0.0009) [2023-10-08 06:39:29,671][00611] Updated weights for policy 0, policy_version 69822 (0.0009) [2023-10-08 06:39:31,438][00612] Updated weights for policy 1, policy_version 70210 (0.0008) [2023-10-08 06:39:31,805][00612] Updated weights for policy 1, policy_version 70220 (0.0010) [2023-10-08 06:39:32,167][00612] Updated weights for policy 1, policy_version 70230 (0.0007) [2023-10-08 06:39:32,539][00612] Updated weights for policy 1, policy_version 70240 (0.0007) [2023-10-08 06:39:33,389][00611] Updated weights for policy 0, policy_version 69832 (0.0007) [2023-10-08 06:39:33,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 143425536. Throughput: 0: 1834.7, 1: 1859.8. Samples: 35869588. Policy #0 lag: (min: 31.0, avg: 32.7, max: 56.0) [2023-10-08 06:39:33,755][130385] Avg episode reward: [(0, '60.160'), (1, '73.170')] [2023-10-08 06:39:33,768][00611] Updated weights for policy 0, policy_version 69842 (0.0008) [2023-10-08 06:39:34,134][00611] Updated weights for policy 0, policy_version 69852 (0.0008) [2023-10-08 06:39:36,069][00612] Updated weights for policy 1, policy_version 70250 (0.0009) [2023-10-08 06:39:36,440][00612] Updated weights for policy 1, policy_version 70260 (0.0009) [2023-10-08 06:39:36,812][00612] Updated weights for policy 1, policy_version 70270 (0.0007) [2023-10-08 06:39:37,748][00611] Updated weights for policy 0, policy_version 69862 (0.0009) [2023-10-08 06:39:38,118][00611] Updated weights for policy 0, policy_version 69872 (0.0008) [2023-10-08 06:39:38,487][00611] Updated weights for policy 0, policy_version 69882 (0.0009) [2023-10-08 06:39:38,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.6, 300 sec: 14884.4). Total num frames: 143523840. Throughput: 0: 1838.9, 1: 1846.1. Samples: 35880836. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:39:38,755][130385] Avg episode reward: [(0, '63.420'), (1, '69.790')] [2023-10-08 06:39:40,426][00612] Updated weights for policy 1, policy_version 70280 (0.0009) [2023-10-08 06:39:40,797][00612] Updated weights for policy 1, policy_version 70290 (0.0008) [2023-10-08 06:39:41,163][00612] Updated weights for policy 1, policy_version 70300 (0.0009) [2023-10-08 06:39:42,166][00611] Updated weights for policy 0, policy_version 69892 (0.0010) [2023-10-08 06:39:42,554][00611] Updated weights for policy 0, policy_version 69902 (0.0008) [2023-10-08 06:39:42,931][00611] Updated weights for policy 0, policy_version 69912 (0.0009) [2023-10-08 06:39:43,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 143589376. Throughput: 0: 1832.5, 1: 1864.4. Samples: 35902846. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:39:43,755][130385] Avg episode reward: [(0, '63.430'), (1, '74.090')] [2023-10-08 06:39:44,733][00612] Updated weights for policy 1, policy_version 70310 (0.0007) [2023-10-08 06:39:45,109][00612] Updated weights for policy 1, policy_version 70320 (0.0009) [2023-10-08 06:39:45,483][00612] Updated weights for policy 1, policy_version 70330 (0.0010) [2023-10-08 06:39:46,600][00611] Updated weights for policy 0, policy_version 69922 (0.0008) [2023-10-08 06:39:46,966][00611] Updated weights for policy 0, policy_version 69932 (0.0008) [2023-10-08 06:39:47,345][00611] Updated weights for policy 0, policy_version 69942 (0.0010) [2023-10-08 06:39:47,705][00611] Updated weights for policy 0, policy_version 69952 (0.0009) [2023-10-08 06:39:48,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 143654912. Throughput: 0: 1833.3, 1: 1863.7. Samples: 35924774. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:39:48,754][130385] Avg episode reward: [(0, '64.190'), (1, '73.350')] [2023-10-08 06:39:49,068][00612] Updated weights for policy 1, policy_version 70340 (0.0009) [2023-10-08 06:39:49,435][00612] Updated weights for policy 1, policy_version 70350 (0.0010) [2023-10-08 06:39:49,801][00612] Updated weights for policy 1, policy_version 70360 (0.0007) [2023-10-08 06:39:51,365][00611] Updated weights for policy 0, policy_version 69962 (0.0009) [2023-10-08 06:39:51,730][00611] Updated weights for policy 0, policy_version 69972 (0.0008) [2023-10-08 06:39:52,109][00611] Updated weights for policy 0, policy_version 69982 (0.0007) [2023-10-08 06:39:53,559][00612] Updated weights for policy 1, policy_version 70370 (0.0007) [2023-10-08 06:39:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 143720448. Throughput: 0: 1831.1, 1: 1860.0. Samples: 35936232. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:39:53,755][130385] Avg episode reward: [(0, '61.710'), (1, '75.880')] [2023-10-08 06:39:53,942][00612] Updated weights for policy 1, policy_version 70380 (0.0010) [2023-10-08 06:39:54,310][00612] Updated weights for policy 1, policy_version 70390 (0.0007) [2023-10-08 06:39:54,673][00612] Updated weights for policy 1, policy_version 70400 (0.0010) [2023-10-08 06:39:55,677][00611] Updated weights for policy 0, policy_version 69992 (0.0008) [2023-10-08 06:39:56,046][00611] Updated weights for policy 0, policy_version 70002 (0.0007) [2023-10-08 06:39:56,417][00611] Updated weights for policy 0, policy_version 70012 (0.0007) [2023-10-08 06:39:58,257][00612] Updated weights for policy 1, policy_version 70410 (0.0007) [2023-10-08 06:39:58,624][00612] Updated weights for policy 1, policy_version 70420 (0.0007) [2023-10-08 06:39:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 143785984. Throughput: 0: 1838.0, 1: 1851.8. Samples: 35957762. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:39:58,754][130385] Avg episode reward: [(0, '63.710'), (1, '79.650')] [2023-10-08 06:39:58,994][00612] Updated weights for policy 1, policy_version 70430 (0.0008) [2023-10-08 06:39:59,933][00611] Updated weights for policy 0, policy_version 70022 (0.0008) [2023-10-08 06:40:00,312][00611] Updated weights for policy 0, policy_version 70032 (0.0008) [2023-10-08 06:40:00,689][00611] Updated weights for policy 0, policy_version 70042 (0.0011) [2023-10-08 06:40:02,570][00612] Updated weights for policy 1, policy_version 70440 (0.0007) [2023-10-08 06:40:02,930][00612] Updated weights for policy 1, policy_version 70450 (0.0008) [2023-10-08 06:40:03,307][00612] Updated weights for policy 1, policy_version 70460 (0.0009) [2023-10-08 06:40:03,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 143884288. Throughput: 0: 1845.2, 1: 1833.2. Samples: 35980020. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:40:03,754][130385] Avg episode reward: [(0, '63.130'), (1, '79.950')] [2023-10-08 06:40:04,213][00611] Updated weights for policy 0, policy_version 70052 (0.0010) [2023-10-08 06:40:04,571][00611] Updated weights for policy 0, policy_version 70062 (0.0010) [2023-10-08 06:40:04,942][00611] Updated weights for policy 0, policy_version 70072 (0.0010) [2023-10-08 06:40:06,909][00612] Updated weights for policy 1, policy_version 70470 (0.0009) [2023-10-08 06:40:07,269][00612] Updated weights for policy 1, policy_version 70480 (0.0010) [2023-10-08 06:40:07,638][00612] Updated weights for policy 1, policy_version 70490 (0.0009) [2023-10-08 06:40:08,627][00611] Updated weights for policy 0, policy_version 70082 (0.0008) [2023-10-08 06:40:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 143949824. Throughput: 0: 1845.8, 1: 1856.6. Samples: 35991394. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:40:08,754][130385] Avg episode reward: [(0, '66.430'), (1, '80.370')] [2023-10-08 06:40:08,994][00611] Updated weights for policy 0, policy_version 70092 (0.0010) [2023-10-08 06:40:09,365][00611] Updated weights for policy 0, policy_version 70102 (0.0009) [2023-10-08 06:40:09,732][00611] Updated weights for policy 0, policy_version 70112 (0.0008) [2023-10-08 06:40:11,344][00612] Updated weights for policy 1, policy_version 70500 (0.0008) [2023-10-08 06:40:11,714][00612] Updated weights for policy 1, policy_version 70510 (0.0007) [2023-10-08 06:40:12,082][00612] Updated weights for policy 1, policy_version 70520 (0.0007) [2023-10-08 06:40:13,361][00611] Updated weights for policy 0, policy_version 70122 (0.0007) [2023-10-08 06:40:13,727][00611] Updated weights for policy 0, policy_version 70132 (0.0009) [2023-10-08 06:40:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 144015360. Throughput: 0: 1851.4, 1: 1830.5. Samples: 36013288. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:40:13,754][130385] Avg episode reward: [(0, '66.990'), (1, '79.700')] [2023-10-08 06:40:14,104][00611] Updated weights for policy 0, policy_version 70142 (0.0010) [2023-10-08 06:40:15,790][00612] Updated weights for policy 1, policy_version 70530 (0.0008) [2023-10-08 06:40:16,157][00612] Updated weights for policy 1, policy_version 70540 (0.0009) [2023-10-08 06:40:16,519][00612] Updated weights for policy 1, policy_version 70550 (0.0011) [2023-10-08 06:40:16,883][00612] Updated weights for policy 1, policy_version 70560 (0.0008) [2023-10-08 06:40:17,690][00611] Updated weights for policy 0, policy_version 70152 (0.0009) [2023-10-08 06:40:18,060][00611] Updated weights for policy 0, policy_version 70162 (0.0011) [2023-10-08 06:40:18,436][00611] Updated weights for policy 0, policy_version 70172 (0.0011) [2023-10-08 06:40:18,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14884.4). Total num frames: 144113664. Throughput: 0: 1832.9, 1: 1852.1. Samples: 36035416. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 06:40:18,755][130385] Avg episode reward: [(0, '65.670'), (1, '78.050')] [2023-10-08 06:40:18,766][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000070176_71860224.pth... [2023-10-08 06:40:18,766][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000070560_72253440.pth... [2023-10-08 06:40:18,798][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000068448_70090752.pth [2023-10-08 06:40:18,809][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000068832_70483968.pth [2023-10-08 06:40:20,544][00612] Updated weights for policy 1, policy_version 70570 (0.0008) [2023-10-08 06:40:20,911][00612] Updated weights for policy 1, policy_version 70580 (0.0008) [2023-10-08 06:40:21,274][00612] Updated weights for policy 1, policy_version 70590 (0.0008) [2023-10-08 06:40:22,160][00611] Updated weights for policy 0, policy_version 70182 (0.0010) [2023-10-08 06:40:22,527][00611] Updated weights for policy 0, policy_version 70192 (0.0010) [2023-10-08 06:40:22,906][00611] Updated weights for policy 0, policy_version 70202 (0.0011) [2023-10-08 06:40:23,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 144179200. Throughput: 0: 1847.8, 1: 1833.9. Samples: 36046510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:40:23,754][130385] Avg episode reward: [(0, '66.250'), (1, '77.030')] [2023-10-08 06:40:24,710][00612] Updated weights for policy 1, policy_version 70600 (0.0008) [2023-10-08 06:40:25,083][00612] Updated weights for policy 1, policy_version 70610 (0.0010) [2023-10-08 06:40:25,455][00612] Updated weights for policy 1, policy_version 70620 (0.0009) [2023-10-08 06:40:26,738][00611] Updated weights for policy 0, policy_version 70212 (0.0010) [2023-10-08 06:40:27,127][00611] Updated weights for policy 0, policy_version 70222 (0.0009) [2023-10-08 06:40:27,498][00611] Updated weights for policy 0, policy_version 70232 (0.0007) [2023-10-08 06:40:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 144244736. Throughput: 0: 1834.9, 1: 1855.3. Samples: 36068908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:40:28,754][130385] Avg episode reward: [(0, '69.910'), (1, '77.330')] [2023-10-08 06:40:29,013][00612] Updated weights for policy 1, policy_version 70630 (0.0009) [2023-10-08 06:40:29,375][00612] Updated weights for policy 1, policy_version 70640 (0.0007) [2023-10-08 06:40:29,739][00612] Updated weights for policy 1, policy_version 70650 (0.0007) [2023-10-08 06:40:31,061][00611] Updated weights for policy 0, policy_version 70242 (0.0008) [2023-10-08 06:40:31,424][00611] Updated weights for policy 0, policy_version 70252 (0.0007) [2023-10-08 06:40:31,792][00611] Updated weights for policy 0, policy_version 70262 (0.0010) [2023-10-08 06:40:32,159][00611] Updated weights for policy 0, policy_version 70272 (0.0011) [2023-10-08 06:40:33,338][00612] Updated weights for policy 1, policy_version 70660 (0.0009) [2023-10-08 06:40:33,706][00612] Updated weights for policy 1, policy_version 70670 (0.0008) [2023-10-08 06:40:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 144310272. Throughput: 0: 1850.1, 1: 1849.4. Samples: 36091252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:40:33,757][130385] Avg episode reward: [(0, '70.340'), (1, '77.250')] [2023-10-08 06:40:34,079][00612] Updated weights for policy 1, policy_version 70680 (0.0009) [2023-10-08 06:40:35,757][00611] Updated weights for policy 0, policy_version 70282 (0.0009) [2023-10-08 06:40:36,129][00611] Updated weights for policy 0, policy_version 70292 (0.0010) [2023-10-08 06:40:36,503][00611] Updated weights for policy 0, policy_version 70302 (0.0010) [2023-10-08 06:40:37,762][00612] Updated weights for policy 1, policy_version 70690 (0.0009) [2023-10-08 06:40:38,136][00612] Updated weights for policy 1, policy_version 70700 (0.0008) [2023-10-08 06:40:38,513][00612] Updated weights for policy 1, policy_version 70710 (0.0008) [2023-10-08 06:40:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 144375808. Throughput: 0: 1827.8, 1: 1851.3. Samples: 36101790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:40:38,755][130385] Avg episode reward: [(0, '68.770'), (1, '87.240')] [2023-10-08 06:40:38,876][00612] Updated weights for policy 1, policy_version 70720 (0.0009) [2023-10-08 06:40:40,070][00611] Updated weights for policy 0, policy_version 70312 (0.0009) [2023-10-08 06:40:40,428][00611] Updated weights for policy 0, policy_version 70322 (0.0011) [2023-10-08 06:40:40,804][00611] Updated weights for policy 0, policy_version 70332 (0.0010) [2023-10-08 06:40:42,718][00612] Updated weights for policy 1, policy_version 70730 (0.0010) [2023-10-08 06:40:43,089][00612] Updated weights for policy 1, policy_version 70740 (0.0011) [2023-10-08 06:40:43,463][00612] Updated weights for policy 1, policy_version 70750 (0.0010) [2023-10-08 06:40:43,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 144474112. Throughput: 0: 1842.4, 1: 1850.5. Samples: 36123940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:40:43,755][130385] Avg episode reward: [(0, '67.560'), (1, '90.150')] [2023-10-08 06:40:43,756][00425] Saving new best policy, reward=90.150! [2023-10-08 06:40:44,497][00611] Updated weights for policy 0, policy_version 70342 (0.0010) [2023-10-08 06:40:44,869][00611] Updated weights for policy 0, policy_version 70352 (0.0008) [2023-10-08 06:40:45,246][00611] Updated weights for policy 0, policy_version 70362 (0.0009) [2023-10-08 06:40:47,035][00612] Updated weights for policy 1, policy_version 70760 (0.0007) [2023-10-08 06:40:47,398][00612] Updated weights for policy 1, policy_version 70770 (0.0007) [2023-10-08 06:40:47,756][00612] Updated weights for policy 1, policy_version 70780 (0.0007) [2023-10-08 06:40:48,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 144539648. Throughput: 0: 1839.6, 1: 1841.9. Samples: 36145690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:40:48,754][130385] Avg episode reward: [(0, '69.740'), (1, '84.620')] [2023-10-08 06:40:48,792][00611] Updated weights for policy 0, policy_version 70372 (0.0008) [2023-10-08 06:40:49,171][00611] Updated weights for policy 0, policy_version 70382 (0.0009) [2023-10-08 06:40:49,540][00611] Updated weights for policy 0, policy_version 70392 (0.0009) [2023-10-08 06:40:51,263][00612] Updated weights for policy 1, policy_version 70790 (0.0008) [2023-10-08 06:40:51,629][00612] Updated weights for policy 1, policy_version 70800 (0.0009) [2023-10-08 06:40:51,994][00612] Updated weights for policy 1, policy_version 70810 (0.0008) [2023-10-08 06:40:53,168][00611] Updated weights for policy 0, policy_version 70402 (0.0008) [2023-10-08 06:40:53,539][00611] Updated weights for policy 0, policy_version 70412 (0.0009) [2023-10-08 06:40:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 144605184. Throughput: 0: 1838.4, 1: 1846.8. Samples: 36157224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:40:53,755][130385] Avg episode reward: [(0, '72.210'), (1, '84.250')] [2023-10-08 06:40:53,902][00611] Updated weights for policy 0, policy_version 70422 (0.0008) [2023-10-08 06:40:54,274][00611] Updated weights for policy 0, policy_version 70432 (0.0008) [2023-10-08 06:40:55,667][00612] Updated weights for policy 1, policy_version 70820 (0.0008) [2023-10-08 06:40:56,040][00612] Updated weights for policy 1, policy_version 70830 (0.0010) [2023-10-08 06:40:56,417][00612] Updated weights for policy 1, policy_version 70840 (0.0009) [2023-10-08 06:40:57,850][00611] Updated weights for policy 0, policy_version 70442 (0.0011) [2023-10-08 06:40:58,214][00611] Updated weights for policy 0, policy_version 70452 (0.0011) [2023-10-08 06:40:58,581][00611] Updated weights for policy 0, policy_version 70462 (0.0009) [2023-10-08 06:40:58,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14884.4). Total num frames: 144703488. Throughput: 0: 1841.3, 1: 1845.2. Samples: 36179182. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:40:58,755][130385] Avg episode reward: [(0, '70.420'), (1, '82.030')] [2023-10-08 06:41:00,122][00612] Updated weights for policy 1, policy_version 70850 (0.0009) [2023-10-08 06:41:00,482][00612] Updated weights for policy 1, policy_version 70860 (0.0009) [2023-10-08 06:41:00,851][00612] Updated weights for policy 1, policy_version 70870 (0.0011) [2023-10-08 06:41:01,218][00612] Updated weights for policy 1, policy_version 70880 (0.0011) [2023-10-08 06:41:02,297][00611] Updated weights for policy 0, policy_version 70472 (0.0008) [2023-10-08 06:41:02,667][00611] Updated weights for policy 0, policy_version 70482 (0.0007) [2023-10-08 06:41:03,048][00611] Updated weights for policy 0, policy_version 70492 (0.0007) [2023-10-08 06:41:03,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 144769024. Throughput: 0: 1828.7, 1: 1847.1. Samples: 36200826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:41:03,755][130385] Avg episode reward: [(0, '67.910'), (1, '84.590')] [2023-10-08 06:41:04,687][00612] Updated weights for policy 1, policy_version 70890 (0.0010) [2023-10-08 06:41:05,066][00612] Updated weights for policy 1, policy_version 70900 (0.0009) [2023-10-08 06:41:05,430][00612] Updated weights for policy 1, policy_version 70910 (0.0008) [2023-10-08 06:41:06,569][00611] Updated weights for policy 0, policy_version 70502 (0.0008) [2023-10-08 06:41:06,930][00611] Updated weights for policy 0, policy_version 70512 (0.0007) [2023-10-08 06:41:07,304][00611] Updated weights for policy 0, policy_version 70522 (0.0008) [2023-10-08 06:41:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 144834560. Throughput: 0: 1843.6, 1: 1841.4. Samples: 36212334. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:41:08,755][130385] Avg episode reward: [(0, '64.290'), (1, '82.600')] [2023-10-08 06:41:09,054][00612] Updated weights for policy 1, policy_version 70920 (0.0010) [2023-10-08 06:41:09,424][00612] Updated weights for policy 1, policy_version 70930 (0.0009) [2023-10-08 06:41:09,803][00612] Updated weights for policy 1, policy_version 70940 (0.0008) [2023-10-08 06:41:10,974][00611] Updated weights for policy 0, policy_version 70532 (0.0011) [2023-10-08 06:41:11,354][00611] Updated weights for policy 0, policy_version 70542 (0.0008) [2023-10-08 06:41:11,722][00611] Updated weights for policy 0, policy_version 70552 (0.0009) [2023-10-08 06:41:13,312][00612] Updated weights for policy 1, policy_version 70950 (0.0007) [2023-10-08 06:41:13,686][00612] Updated weights for policy 1, policy_version 70960 (0.0010) [2023-10-08 06:41:13,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 144900096. Throughput: 0: 1825.7, 1: 1846.4. Samples: 36234156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:41:13,754][130385] Avg episode reward: [(0, '63.390'), (1, '80.840')] [2023-10-08 06:41:14,046][00612] Updated weights for policy 1, policy_version 70970 (0.0009) [2023-10-08 06:41:15,244][00611] Updated weights for policy 0, policy_version 70562 (0.0007) [2023-10-08 06:41:15,636][00611] Updated weights for policy 0, policy_version 70572 (0.0008) [2023-10-08 06:41:16,010][00611] Updated weights for policy 0, policy_version 70582 (0.0008) [2023-10-08 06:41:16,376][00611] Updated weights for policy 0, policy_version 70592 (0.0007) [2023-10-08 06:41:17,742][00612] Updated weights for policy 1, policy_version 70980 (0.0008) [2023-10-08 06:41:18,110][00612] Updated weights for policy 1, policy_version 70990 (0.0007) [2023-10-08 06:41:18,479][00612] Updated weights for policy 1, policy_version 71000 (0.0008) [2023-10-08 06:41:18,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 144965632. Throughput: 0: 1841.5, 1: 1831.1. Samples: 36256522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:41:18,755][130385] Avg episode reward: [(0, '58.870'), (1, '83.740')] [2023-10-08 06:41:20,130][00611] Updated weights for policy 0, policy_version 70602 (0.0007) [2023-10-08 06:41:20,502][00611] Updated weights for policy 0, policy_version 70612 (0.0008) [2023-10-08 06:41:20,866][00611] Updated weights for policy 0, policy_version 70622 (0.0008) [2023-10-08 06:41:22,186][00612] Updated weights for policy 1, policy_version 71010 (0.0007) [2023-10-08 06:41:22,554][00612] Updated weights for policy 1, policy_version 71020 (0.0007) [2023-10-08 06:41:22,924][00612] Updated weights for policy 1, policy_version 71030 (0.0007) [2023-10-08 06:41:23,284][00612] Updated weights for policy 1, policy_version 71040 (0.0007) [2023-10-08 06:41:23,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 145063936. Throughput: 0: 1832.3, 1: 1852.6. Samples: 36267610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:41:23,754][130385] Avg episode reward: [(0, '56.780'), (1, '80.190')] [2023-10-08 06:41:24,492][00611] Updated weights for policy 0, policy_version 70632 (0.0008) [2023-10-08 06:41:24,863][00611] Updated weights for policy 0, policy_version 70642 (0.0008) [2023-10-08 06:41:25,236][00611] Updated weights for policy 0, policy_version 70652 (0.0007) [2023-10-08 06:41:26,928][00612] Updated weights for policy 1, policy_version 71050 (0.0008) [2023-10-08 06:41:27,296][00612] Updated weights for policy 1, policy_version 71060 (0.0008) [2023-10-08 06:41:27,670][00612] Updated weights for policy 1, policy_version 71070 (0.0007) [2023-10-08 06:41:28,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 145129472. Throughput: 0: 1849.6, 1: 1842.2. Samples: 36290072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:41:28,755][130385] Avg episode reward: [(0, '57.780'), (1, '81.940')] [2023-10-08 06:41:28,778][00611] Updated weights for policy 0, policy_version 70662 (0.0008) [2023-10-08 06:41:29,147][00611] Updated weights for policy 0, policy_version 70672 (0.0010) [2023-10-08 06:41:29,526][00611] Updated weights for policy 0, policy_version 70682 (0.0009) [2023-10-08 06:41:31,589][00612] Updated weights for policy 1, policy_version 71080 (0.0008) [2023-10-08 06:41:31,969][00612] Updated weights for policy 1, policy_version 71090 (0.0009) [2023-10-08 06:41:32,327][00612] Updated weights for policy 1, policy_version 71100 (0.0007) [2023-10-08 06:41:33,201][00611] Updated weights for policy 0, policy_version 70692 (0.0008) [2023-10-08 06:41:33,562][00611] Updated weights for policy 0, policy_version 70702 (0.0008) [2023-10-08 06:41:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 145195008. Throughput: 0: 1845.4, 1: 1852.0. Samples: 36312074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:41:33,754][130385] Avg episode reward: [(0, '57.990'), (1, '80.750')] [2023-10-08 06:41:33,932][00611] Updated weights for policy 0, policy_version 70712 (0.0009) [2023-10-08 06:41:35,803][00612] Updated weights for policy 1, policy_version 71110 (0.0008) [2023-10-08 06:41:36,166][00612] Updated weights for policy 1, policy_version 71120 (0.0007) [2023-10-08 06:41:36,533][00612] Updated weights for policy 1, policy_version 71130 (0.0007) [2023-10-08 06:41:37,511][00611] Updated weights for policy 0, policy_version 70722 (0.0008) [2023-10-08 06:41:37,884][00611] Updated weights for policy 0, policy_version 70732 (0.0008) [2023-10-08 06:41:38,247][00611] Updated weights for policy 0, policy_version 70742 (0.0009) [2023-10-08 06:41:38,623][00611] Updated weights for policy 0, policy_version 70752 (0.0008) [2023-10-08 06:41:38,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14884.4). Total num frames: 145293312. Throughput: 0: 1850.6, 1: 1840.7. Samples: 36323332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:41:38,755][130385] Avg episode reward: [(0, '58.410'), (1, '84.250')] [2023-10-08 06:41:40,051][00612] Updated weights for policy 1, policy_version 71140 (0.0009) [2023-10-08 06:41:40,406][00612] Updated weights for policy 1, policy_version 71150 (0.0010) [2023-10-08 06:41:40,777][00612] Updated weights for policy 1, policy_version 71160 (0.0009) [2023-10-08 06:41:42,196][00611] Updated weights for policy 0, policy_version 70762 (0.0007) [2023-10-08 06:41:42,574][00611] Updated weights for policy 0, policy_version 70772 (0.0008) [2023-10-08 06:41:42,942][00611] Updated weights for policy 0, policy_version 70782 (0.0009) [2023-10-08 06:41:43,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 145358848. Throughput: 0: 1838.5, 1: 1858.0. Samples: 36345522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:41:43,754][130385] Avg episode reward: [(0, '57.420'), (1, '72.500')] [2023-10-08 06:41:44,517][00612] Updated weights for policy 1, policy_version 71170 (0.0007) [2023-10-08 06:41:44,881][00612] Updated weights for policy 1, policy_version 71180 (0.0010) [2023-10-08 06:41:45,255][00612] Updated weights for policy 1, policy_version 71190 (0.0010) [2023-10-08 06:41:45,621][00612] Updated weights for policy 1, policy_version 71200 (0.0009) [2023-10-08 06:41:46,707][00611] Updated weights for policy 0, policy_version 70792 (0.0007) [2023-10-08 06:41:47,071][00611] Updated weights for policy 0, policy_version 70802 (0.0007) [2023-10-08 06:41:47,449][00611] Updated weights for policy 0, policy_version 70812 (0.0008) [2023-10-08 06:41:48,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 145424384. Throughput: 0: 1840.3, 1: 1860.7. Samples: 36367370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:41:48,754][130385] Avg episode reward: [(0, '60.460'), (1, '70.650')] [2023-10-08 06:41:49,125][00612] Updated weights for policy 1, policy_version 71210 (0.0008) [2023-10-08 06:41:49,496][00612] Updated weights for policy 1, policy_version 71220 (0.0007) [2023-10-08 06:41:49,859][00612] Updated weights for policy 1, policy_version 71230 (0.0007) [2023-10-08 06:41:50,950][00611] Updated weights for policy 0, policy_version 70822 (0.0010) [2023-10-08 06:41:51,330][00611] Updated weights for policy 0, policy_version 70832 (0.0011) [2023-10-08 06:41:51,698][00611] Updated weights for policy 0, policy_version 70842 (0.0011) [2023-10-08 06:41:53,436][00612] Updated weights for policy 1, policy_version 71240 (0.0008) [2023-10-08 06:41:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 145489920. Throughput: 0: 1832.5, 1: 1859.3. Samples: 36378462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:41:53,754][130385] Avg episode reward: [(0, '59.630'), (1, '72.130')] [2023-10-08 06:41:53,807][00612] Updated weights for policy 1, policy_version 71250 (0.0007) [2023-10-08 06:41:54,174][00612] Updated weights for policy 1, policy_version 71260 (0.0011) [2023-10-08 06:41:55,484][00611] Updated weights for policy 0, policy_version 70852 (0.0009) [2023-10-08 06:41:55,848][00611] Updated weights for policy 0, policy_version 70862 (0.0009) [2023-10-08 06:41:56,221][00611] Updated weights for policy 0, policy_version 70872 (0.0007) [2023-10-08 06:41:57,868][00612] Updated weights for policy 1, policy_version 71270 (0.0010) [2023-10-08 06:41:58,236][00612] Updated weights for policy 1, policy_version 71280 (0.0009) [2023-10-08 06:41:58,608][00612] Updated weights for policy 1, policy_version 71290 (0.0009) [2023-10-08 06:41:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 145555456. Throughput: 0: 1836.4, 1: 1847.2. Samples: 36399916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:41:58,754][130385] Avg episode reward: [(0, '60.820'), (1, '69.200')] [2023-10-08 06:42:00,024][00611] Updated weights for policy 0, policy_version 70882 (0.0009) [2023-10-08 06:42:00,448][00611] Updated weights for policy 0, policy_version 70892 (0.0010) [2023-10-08 06:42:00,813][00611] Updated weights for policy 0, policy_version 70902 (0.0009) [2023-10-08 06:42:01,183][00611] Updated weights for policy 0, policy_version 70912 (0.0009) [2023-10-08 06:42:02,068][00612] Updated weights for policy 1, policy_version 71300 (0.0008) [2023-10-08 06:42:02,429][00612] Updated weights for policy 1, policy_version 71310 (0.0009) [2023-10-08 06:42:02,795][00612] Updated weights for policy 1, policy_version 71320 (0.0007) [2023-10-08 06:42:03,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 145653760. Throughput: 0: 1840.4, 1: 1834.8. Samples: 36421904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:42:03,755][130385] Avg episode reward: [(0, '60.800'), (1, '70.650')] [2023-10-08 06:42:04,804][00611] Updated weights for policy 0, policy_version 70922 (0.0011) [2023-10-08 06:42:05,182][00611] Updated weights for policy 0, policy_version 70932 (0.0009) [2023-10-08 06:42:05,543][00611] Updated weights for policy 0, policy_version 70942 (0.0008) [2023-10-08 06:42:06,415][00612] Updated weights for policy 1, policy_version 71330 (0.0008) [2023-10-08 06:42:06,784][00612] Updated weights for policy 1, policy_version 71340 (0.0007) [2023-10-08 06:42:07,149][00612] Updated weights for policy 1, policy_version 71350 (0.0007) [2023-10-08 06:42:07,516][00612] Updated weights for policy 1, policy_version 71360 (0.0007) [2023-10-08 06:42:08,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 145719296. Throughput: 0: 1833.8, 1: 1846.6. Samples: 36433228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:42:08,755][130385] Avg episode reward: [(0, '62.860'), (1, '69.420')] [2023-10-08 06:42:09,243][00611] Updated weights for policy 0, policy_version 70952 (0.0008) [2023-10-08 06:42:09,623][00611] Updated weights for policy 0, policy_version 70962 (0.0007) [2023-10-08 06:42:09,991][00611] Updated weights for policy 0, policy_version 70972 (0.0008) [2023-10-08 06:42:11,265][00612] Updated weights for policy 1, policy_version 71370 (0.0007) [2023-10-08 06:42:11,628][00612] Updated weights for policy 1, policy_version 71380 (0.0009) [2023-10-08 06:42:11,995][00612] Updated weights for policy 1, policy_version 71390 (0.0008) [2023-10-08 06:42:13,572][00611] Updated weights for policy 0, policy_version 70982 (0.0008) [2023-10-08 06:42:13,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 145784832. Throughput: 0: 1839.4, 1: 1829.0. Samples: 36455150. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:42:13,754][130385] Avg episode reward: [(0, '63.430'), (1, '66.800')] [2023-10-08 06:42:13,946][00611] Updated weights for policy 0, policy_version 70992 (0.0009) [2023-10-08 06:42:14,322][00611] Updated weights for policy 0, policy_version 71002 (0.0007) [2023-10-08 06:42:15,963][00612] Updated weights for policy 1, policy_version 71400 (0.0009) [2023-10-08 06:42:16,333][00612] Updated weights for policy 1, policy_version 71410 (0.0008) [2023-10-08 06:42:16,701][00612] Updated weights for policy 1, policy_version 71420 (0.0009) [2023-10-08 06:42:17,797][00611] Updated weights for policy 0, policy_version 71012 (0.0010) [2023-10-08 06:42:18,168][00611] Updated weights for policy 0, policy_version 71022 (0.0010) [2023-10-08 06:42:18,543][00611] Updated weights for policy 0, policy_version 71032 (0.0009) [2023-10-08 06:42:18,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 145850368. Throughput: 0: 1835.6, 1: 1848.3. Samples: 36477850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:42:18,755][130385] Avg episode reward: [(0, '64.940'), (1, '64.430')] [2023-10-08 06:42:18,766][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000071424_73138176.pth... [2023-10-08 06:42:18,805][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000069696_71368704.pth [2023-10-08 06:42:18,831][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000071040_72744960.pth... [2023-10-08 06:42:18,872][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000069312_70975488.pth [2023-10-08 06:42:20,228][00612] Updated weights for policy 1, policy_version 71430 (0.0008) [2023-10-08 06:42:20,598][00612] Updated weights for policy 1, policy_version 71440 (0.0009) [2023-10-08 06:42:20,961][00612] Updated weights for policy 1, policy_version 71450 (0.0010) [2023-10-08 06:42:22,093][00611] Updated weights for policy 0, policy_version 71042 (0.0007) [2023-10-08 06:42:22,464][00611] Updated weights for policy 0, policy_version 71052 (0.0007) [2023-10-08 06:42:22,834][00611] Updated weights for policy 0, policy_version 71062 (0.0010) [2023-10-08 06:42:23,204][00611] Updated weights for policy 0, policy_version 71072 (0.0009) [2023-10-08 06:42:23,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 145948672. Throughput: 0: 1846.2, 1: 1828.9. Samples: 36488710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:42:23,754][130385] Avg episode reward: [(0, '65.930'), (1, '64.690')] [2023-10-08 06:42:24,571][00612] Updated weights for policy 1, policy_version 71460 (0.0008) [2023-10-08 06:42:24,943][00612] Updated weights for policy 1, policy_version 71470 (0.0010) [2023-10-08 06:42:25,308][00612] Updated weights for policy 1, policy_version 71480 (0.0009) [2023-10-08 06:42:26,797][00611] Updated weights for policy 0, policy_version 71082 (0.0007) [2023-10-08 06:42:27,168][00611] Updated weights for policy 0, policy_version 71092 (0.0009) [2023-10-08 06:42:27,535][00611] Updated weights for policy 0, policy_version 71102 (0.0010) [2023-10-08 06:42:28,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 146014208. Throughput: 0: 1833.5, 1: 1842.5. Samples: 36510944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:42:28,754][130385] Avg episode reward: [(0, '67.920'), (1, '64.100')] [2023-10-08 06:42:28,876][00612] Updated weights for policy 1, policy_version 71490 (0.0008) [2023-10-08 06:42:29,247][00612] Updated weights for policy 1, policy_version 71500 (0.0009) [2023-10-08 06:42:29,607][00612] Updated weights for policy 1, policy_version 71510 (0.0007) [2023-10-08 06:42:29,981][00612] Updated weights for policy 1, policy_version 71520 (0.0007) [2023-10-08 06:42:31,213][00611] Updated weights for policy 0, policy_version 71112 (0.0009) [2023-10-08 06:42:31,585][00611] Updated weights for policy 0, policy_version 71122 (0.0007) [2023-10-08 06:42:31,946][00611] Updated weights for policy 0, policy_version 71132 (0.0008) [2023-10-08 06:42:33,604][00612] Updated weights for policy 1, policy_version 71530 (0.0008) [2023-10-08 06:42:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 146079744. Throughput: 0: 1849.1, 1: 1842.6. Samples: 36533496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:42:33,754][130385] Avg episode reward: [(0, '71.770'), (1, '73.170')] [2023-10-08 06:42:33,979][00612] Updated weights for policy 1, policy_version 71540 (0.0008) [2023-10-08 06:42:34,344][00612] Updated weights for policy 1, policy_version 71550 (0.0007) [2023-10-08 06:42:35,610][00611] Updated weights for policy 0, policy_version 71142 (0.0009) [2023-10-08 06:42:35,978][00611] Updated weights for policy 0, policy_version 71152 (0.0009) [2023-10-08 06:42:36,354][00611] Updated weights for policy 0, policy_version 71162 (0.0009) [2023-10-08 06:42:37,987][00612] Updated weights for policy 1, policy_version 71560 (0.0010) [2023-10-08 06:42:38,354][00612] Updated weights for policy 1, policy_version 71570 (0.0011) [2023-10-08 06:42:38,721][00612] Updated weights for policy 1, policy_version 71580 (0.0009) [2023-10-08 06:42:38,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 146145280. Throughput: 0: 1833.3, 1: 1845.0. Samples: 36543988. Policy #0 lag: (min: 25.0, avg: 30.1, max: 57.0) [2023-10-08 06:42:38,754][130385] Avg episode reward: [(0, '74.960'), (1, '74.450')] [2023-10-08 06:42:40,052][00611] Updated weights for policy 0, policy_version 71172 (0.0007) [2023-10-08 06:42:40,419][00611] Updated weights for policy 0, policy_version 71182 (0.0011) [2023-10-08 06:42:40,798][00611] Updated weights for policy 0, policy_version 71192 (0.0009) [2023-10-08 06:42:42,206][00612] Updated weights for policy 1, policy_version 71590 (0.0010) [2023-10-08 06:42:42,575][00612] Updated weights for policy 1, policy_version 71600 (0.0010) [2023-10-08 06:42:42,945][00612] Updated weights for policy 1, policy_version 71610 (0.0009) [2023-10-08 06:42:43,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 146243584. Throughput: 0: 1853.0, 1: 1849.1. Samples: 36566508. Policy #0 lag: (min: 25.0, avg: 30.1, max: 57.0) [2023-10-08 06:42:43,755][130385] Avg episode reward: [(0, '73.860'), (1, '73.940')] [2023-10-08 06:42:44,349][00611] Updated weights for policy 0, policy_version 71202 (0.0009) [2023-10-08 06:42:44,718][00611] Updated weights for policy 0, policy_version 71212 (0.0010) [2023-10-08 06:42:45,090][00611] Updated weights for policy 0, policy_version 71222 (0.0008) [2023-10-08 06:42:45,463][00611] Updated weights for policy 0, policy_version 71232 (0.0007) [2023-10-08 06:42:46,701][00612] Updated weights for policy 1, policy_version 71620 (0.0011) [2023-10-08 06:42:47,062][00612] Updated weights for policy 1, policy_version 71630 (0.0011) [2023-10-08 06:42:47,435][00612] Updated weights for policy 1, policy_version 71640 (0.0010) [2023-10-08 06:42:48,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 146309120. Throughput: 0: 1856.2, 1: 1845.2. Samples: 36588466. Policy #0 lag: (min: 25.0, avg: 30.1, max: 57.0) [2023-10-08 06:42:48,754][130385] Avg episode reward: [(0, '72.690'), (1, '71.340')] [2023-10-08 06:42:49,157][00611] Updated weights for policy 0, policy_version 71242 (0.0009) [2023-10-08 06:42:49,529][00611] Updated weights for policy 0, policy_version 71252 (0.0010) [2023-10-08 06:42:49,902][00611] Updated weights for policy 0, policy_version 71262 (0.0009) [2023-10-08 06:42:51,066][00612] Updated weights for policy 1, policy_version 71650 (0.0009) [2023-10-08 06:42:51,430][00612] Updated weights for policy 1, policy_version 71660 (0.0009) [2023-10-08 06:42:51,798][00612] Updated weights for policy 1, policy_version 71670 (0.0008) [2023-10-08 06:42:52,164][00612] Updated weights for policy 1, policy_version 71680 (0.0009) [2023-10-08 06:42:53,667][00611] Updated weights for policy 0, policy_version 71272 (0.0009) [2023-10-08 06:42:53,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 146374656. Throughput: 0: 1857.3, 1: 1839.9. Samples: 36599598. Policy #0 lag: (min: 25.0, avg: 30.1, max: 57.0) [2023-10-08 06:42:53,754][130385] Avg episode reward: [(0, '71.060'), (1, '76.250')] [2023-10-08 06:42:54,033][00611] Updated weights for policy 0, policy_version 71282 (0.0010) [2023-10-08 06:42:54,411][00611] Updated weights for policy 0, policy_version 71292 (0.0010) [2023-10-08 06:42:55,699][00612] Updated weights for policy 1, policy_version 71690 (0.0009) [2023-10-08 06:42:56,068][00612] Updated weights for policy 1, policy_version 71700 (0.0008) [2023-10-08 06:42:56,430][00612] Updated weights for policy 1, policy_version 71710 (0.0010) [2023-10-08 06:42:57,912][00611] Updated weights for policy 0, policy_version 71302 (0.0008) [2023-10-08 06:42:58,281][00611] Updated weights for policy 0, policy_version 71312 (0.0007) [2023-10-08 06:42:58,657][00611] Updated weights for policy 0, policy_version 71322 (0.0010) [2023-10-08 06:42:58,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 146440192. Throughput: 0: 1849.5, 1: 1850.3. Samples: 36621642. Policy #0 lag: (min: 25.0, avg: 30.1, max: 57.0) [2023-10-08 06:42:58,755][130385] Avg episode reward: [(0, '72.060'), (1, '83.950')] [2023-10-08 06:43:00,092][00612] Updated weights for policy 1, policy_version 71720 (0.0007) [2023-10-08 06:43:00,467][00612] Updated weights for policy 1, policy_version 71730 (0.0007) [2023-10-08 06:43:00,827][00612] Updated weights for policy 1, policy_version 71740 (0.0008) [2023-10-08 06:43:02,279][00611] Updated weights for policy 0, policy_version 71332 (0.0010) [2023-10-08 06:43:02,649][00611] Updated weights for policy 0, policy_version 71342 (0.0008) [2023-10-08 06:43:03,017][00611] Updated weights for policy 0, policy_version 71352 (0.0007) [2023-10-08 06:43:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 146538496. Throughput: 0: 1828.1, 1: 1851.1. Samples: 36643414. Policy #0 lag: (min: 25.0, avg: 30.1, max: 57.0) [2023-10-08 06:43:03,754][130385] Avg episode reward: [(0, '72.380'), (1, '88.790')] [2023-10-08 06:43:04,650][00612] Updated weights for policy 1, policy_version 71750 (0.0010) [2023-10-08 06:43:05,031][00612] Updated weights for policy 1, policy_version 71760 (0.0011) [2023-10-08 06:43:05,395][00612] Updated weights for policy 1, policy_version 71770 (0.0007) [2023-10-08 06:43:06,638][00611] Updated weights for policy 0, policy_version 71362 (0.0008) [2023-10-08 06:43:07,006][00611] Updated weights for policy 0, policy_version 71372 (0.0008) [2023-10-08 06:43:07,367][00611] Updated weights for policy 0, policy_version 71382 (0.0009) [2023-10-08 06:43:07,744][00611] Updated weights for policy 0, policy_version 71392 (0.0009) [2023-10-08 06:43:08,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 146604032. Throughput: 0: 1844.5, 1: 1843.3. Samples: 36654662. Policy #0 lag: (min: 25.0, avg: 30.1, max: 57.0) [2023-10-08 06:43:08,755][130385] Avg episode reward: [(0, '73.770'), (1, '85.940')] [2023-10-08 06:43:08,946][00612] Updated weights for policy 1, policy_version 71780 (0.0009) [2023-10-08 06:43:09,320][00612] Updated weights for policy 1, policy_version 71790 (0.0008) [2023-10-08 06:43:09,692][00612] Updated weights for policy 1, policy_version 71800 (0.0008) [2023-10-08 06:43:11,343][00611] Updated weights for policy 0, policy_version 71402 (0.0007) [2023-10-08 06:43:11,699][00611] Updated weights for policy 0, policy_version 71412 (0.0007) [2023-10-08 06:43:12,071][00611] Updated weights for policy 0, policy_version 71422 (0.0009) [2023-10-08 06:43:13,235][00612] Updated weights for policy 1, policy_version 71810 (0.0008) [2023-10-08 06:43:13,599][00612] Updated weights for policy 1, policy_version 71820 (0.0008) [2023-10-08 06:43:13,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 146669568. Throughput: 0: 1835.5, 1: 1844.0. Samples: 36676522. Policy #0 lag: (min: 25.0, avg: 30.1, max: 57.0) [2023-10-08 06:43:13,755][130385] Avg episode reward: [(0, '73.500'), (1, '86.050')] [2023-10-08 06:43:13,961][00612] Updated weights for policy 1, policy_version 71830 (0.0011) [2023-10-08 06:43:14,332][00612] Updated weights for policy 1, policy_version 71840 (0.0011) [2023-10-08 06:43:15,665][00611] Updated weights for policy 0, policy_version 71432 (0.0009) [2023-10-08 06:43:16,040][00611] Updated weights for policy 0, policy_version 71442 (0.0010) [2023-10-08 06:43:16,412][00611] Updated weights for policy 0, policy_version 71452 (0.0009) [2023-10-08 06:43:18,067][00612] Updated weights for policy 1, policy_version 71850 (0.0008) [2023-10-08 06:43:18,432][00612] Updated weights for policy 1, policy_version 71860 (0.0007) [2023-10-08 06:43:18,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 146735104. Throughput: 0: 1849.0, 1: 1827.4. Samples: 36698932. Policy #0 lag: (min: 25.0, avg: 30.1, max: 57.0) [2023-10-08 06:43:18,755][130385] Avg episode reward: [(0, '71.650'), (1, '83.360')] [2023-10-08 06:43:18,816][00612] Updated weights for policy 1, policy_version 71870 (0.0010) [2023-10-08 06:43:20,090][00611] Updated weights for policy 0, policy_version 71462 (0.0008) [2023-10-08 06:43:20,458][00611] Updated weights for policy 0, policy_version 71472 (0.0009) [2023-10-08 06:43:20,834][00611] Updated weights for policy 0, policy_version 71482 (0.0008) [2023-10-08 06:43:22,251][00612] Updated weights for policy 1, policy_version 71880 (0.0007) [2023-10-08 06:43:22,623][00612] Updated weights for policy 1, policy_version 71890 (0.0009) [2023-10-08 06:43:22,983][00612] Updated weights for policy 1, policy_version 71900 (0.0008) [2023-10-08 06:43:23,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 146833408. Throughput: 0: 1838.8, 1: 1846.3. Samples: 36709818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:43:23,754][130385] Avg episode reward: [(0, '74.640'), (1, '84.850')] [2023-10-08 06:43:24,504][00611] Updated weights for policy 0, policy_version 71492 (0.0008) [2023-10-08 06:43:24,867][00611] Updated weights for policy 0, policy_version 71502 (0.0009) [2023-10-08 06:43:25,236][00611] Updated weights for policy 0, policy_version 71512 (0.0009) [2023-10-08 06:43:26,726][00612] Updated weights for policy 1, policy_version 71910 (0.0009) [2023-10-08 06:43:27,093][00612] Updated weights for policy 1, policy_version 71920 (0.0008) [2023-10-08 06:43:27,472][00612] Updated weights for policy 1, policy_version 71930 (0.0008) [2023-10-08 06:43:28,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 146898944. Throughput: 0: 1851.4, 1: 1824.3. Samples: 36731912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:43:28,755][130385] Avg episode reward: [(0, '70.820'), (1, '84.840')] [2023-10-08 06:43:28,813][00611] Updated weights for policy 0, policy_version 71522 (0.0011) [2023-10-08 06:43:29,194][00611] Updated weights for policy 0, policy_version 71532 (0.0009) [2023-10-08 06:43:29,567][00611] Updated weights for policy 0, policy_version 71542 (0.0009) [2023-10-08 06:43:29,936][00611] Updated weights for policy 0, policy_version 71552 (0.0007) [2023-10-08 06:43:31,030][00612] Updated weights for policy 1, policy_version 71940 (0.0009) [2023-10-08 06:43:31,393][00612] Updated weights for policy 1, policy_version 71950 (0.0010) [2023-10-08 06:43:31,766][00612] Updated weights for policy 1, policy_version 71960 (0.0011) [2023-10-08 06:43:33,473][00611] Updated weights for policy 0, policy_version 71562 (0.0009) [2023-10-08 06:43:33,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 146964480. Throughput: 0: 1847.9, 1: 1841.2. Samples: 36754478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:43:33,755][130385] Avg episode reward: [(0, '66.830'), (1, '84.260')] [2023-10-08 06:43:33,852][00611] Updated weights for policy 0, policy_version 71572 (0.0008) [2023-10-08 06:43:34,226][00611] Updated weights for policy 0, policy_version 71582 (0.0009) [2023-10-08 06:43:35,360][00612] Updated weights for policy 1, policy_version 71970 (0.0007) [2023-10-08 06:43:35,732][00612] Updated weights for policy 1, policy_version 71980 (0.0009) [2023-10-08 06:43:36,097][00612] Updated weights for policy 1, policy_version 71990 (0.0008) [2023-10-08 06:43:36,461][00612] Updated weights for policy 1, policy_version 72000 (0.0010) [2023-10-08 06:43:37,903][00611] Updated weights for policy 0, policy_version 71592 (0.0009) [2023-10-08 06:43:38,285][00611] Updated weights for policy 0, policy_version 71602 (0.0009) [2023-10-08 06:43:38,655][00611] Updated weights for policy 0, policy_version 71612 (0.0009) [2023-10-08 06:43:38,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147030016. Throughput: 0: 1850.8, 1: 1826.1. Samples: 36765058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:43:38,754][130385] Avg episode reward: [(0, '62.420'), (1, '80.410')] [2023-10-08 06:43:40,166][00612] Updated weights for policy 1, policy_version 72010 (0.0011) [2023-10-08 06:43:40,533][00612] Updated weights for policy 1, policy_version 72020 (0.0009) [2023-10-08 06:43:40,897][00612] Updated weights for policy 1, policy_version 72030 (0.0009) [2023-10-08 06:43:42,359][00611] Updated weights for policy 0, policy_version 71622 (0.0007) [2023-10-08 06:43:42,726][00611] Updated weights for policy 0, policy_version 71632 (0.0008) [2023-10-08 06:43:43,095][00611] Updated weights for policy 0, policy_version 71642 (0.0011) [2023-10-08 06:43:43,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 147128320. Throughput: 0: 1846.5, 1: 1844.4. Samples: 36787732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:43:43,754][130385] Avg episode reward: [(0, '64.470'), (1, '80.960')] [2023-10-08 06:43:44,495][00612] Updated weights for policy 1, policy_version 72040 (0.0008) [2023-10-08 06:43:44,856][00612] Updated weights for policy 1, policy_version 72050 (0.0007) [2023-10-08 06:43:45,226][00612] Updated weights for policy 1, policy_version 72060 (0.0007) [2023-10-08 06:43:46,729][00611] Updated weights for policy 0, policy_version 71652 (0.0009) [2023-10-08 06:43:47,088][00611] Updated weights for policy 0, policy_version 71662 (0.0008) [2023-10-08 06:43:47,459][00611] Updated weights for policy 0, policy_version 71672 (0.0009) [2023-10-08 06:43:48,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 147193856. Throughput: 0: 1838.0, 1: 1849.7. Samples: 36809362. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:43:48,755][130385] Avg episode reward: [(0, '66.160'), (1, '78.960')] [2023-10-08 06:43:48,992][00612] Updated weights for policy 1, policy_version 72070 (0.0008) [2023-10-08 06:43:49,357][00612] Updated weights for policy 1, policy_version 72080 (0.0010) [2023-10-08 06:43:49,732][00612] Updated weights for policy 1, policy_version 72090 (0.0008) [2023-10-08 06:43:51,236][00611] Updated weights for policy 0, policy_version 71682 (0.0010) [2023-10-08 06:43:51,605][00611] Updated weights for policy 0, policy_version 71692 (0.0008) [2023-10-08 06:43:51,980][00611] Updated weights for policy 0, policy_version 71702 (0.0008) [2023-10-08 06:43:52,348][00611] Updated weights for policy 0, policy_version 71712 (0.0008) [2023-10-08 06:43:53,451][00612] Updated weights for policy 1, policy_version 72100 (0.0010) [2023-10-08 06:43:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147259392. Throughput: 0: 1837.2, 1: 1850.7. Samples: 36820618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:43:53,754][130385] Avg episode reward: [(0, '63.350'), (1, '79.020')] [2023-10-08 06:43:53,829][00612] Updated weights for policy 1, policy_version 72110 (0.0011) [2023-10-08 06:43:54,202][00612] Updated weights for policy 1, policy_version 72120 (0.0007) [2023-10-08 06:43:56,077][00611] Updated weights for policy 0, policy_version 71722 (0.0010) [2023-10-08 06:43:56,444][00611] Updated weights for policy 0, policy_version 71732 (0.0009) [2023-10-08 06:43:56,808][00611] Updated weights for policy 0, policy_version 71742 (0.0010) [2023-10-08 06:43:57,720][00612] Updated weights for policy 1, policy_version 72130 (0.0008) [2023-10-08 06:43:58,094][00612] Updated weights for policy 1, policy_version 72140 (0.0007) [2023-10-08 06:43:58,456][00612] Updated weights for policy 1, policy_version 72150 (0.0007) [2023-10-08 06:43:58,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147324928. Throughput: 0: 1828.0, 1: 1852.1. Samples: 36842128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:43:58,755][130385] Avg episode reward: [(0, '64.820'), (1, '80.740')] [2023-10-08 06:43:58,820][00612] Updated weights for policy 1, policy_version 72160 (0.0008) [2023-10-08 06:44:00,418][00611] Updated weights for policy 0, policy_version 71752 (0.0009) [2023-10-08 06:44:00,797][00611] Updated weights for policy 0, policy_version 71762 (0.0010) [2023-10-08 06:44:01,167][00611] Updated weights for policy 0, policy_version 71772 (0.0007) [2023-10-08 06:44:02,446][00612] Updated weights for policy 1, policy_version 72170 (0.0007) [2023-10-08 06:44:02,798][00612] Updated weights for policy 1, policy_version 72180 (0.0007) [2023-10-08 06:44:03,158][00612] Updated weights for policy 1, policy_version 72190 (0.0008) [2023-10-08 06:44:03,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 147423232. Throughput: 0: 1830.0, 1: 1833.1. Samples: 36863772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:44:03,755][130385] Avg episode reward: [(0, '64.320'), (1, '76.950')] [2023-10-08 06:44:04,725][00611] Updated weights for policy 0, policy_version 71782 (0.0008) [2023-10-08 06:44:05,092][00611] Updated weights for policy 0, policy_version 71792 (0.0008) [2023-10-08 06:44:05,464][00611] Updated weights for policy 0, policy_version 71802 (0.0010) [2023-10-08 06:44:06,870][00612] Updated weights for policy 1, policy_version 72200 (0.0007) [2023-10-08 06:44:07,235][00612] Updated weights for policy 1, policy_version 72210 (0.0007) [2023-10-08 06:44:07,602][00612] Updated weights for policy 1, policy_version 72220 (0.0007) [2023-10-08 06:44:08,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 147488768. Throughput: 0: 1828.5, 1: 1843.1. Samples: 36875038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:44:08,754][130385] Avg episode reward: [(0, '62.470'), (1, '83.450')] [2023-10-08 06:44:08,859][00611] Updated weights for policy 0, policy_version 71812 (0.0010) [2023-10-08 06:44:09,230][00611] Updated weights for policy 0, policy_version 71822 (0.0009) [2023-10-08 06:44:09,598][00611] Updated weights for policy 0, policy_version 71832 (0.0008) [2023-10-08 06:44:11,198][00612] Updated weights for policy 1, policy_version 72230 (0.0008) [2023-10-08 06:44:11,557][00612] Updated weights for policy 1, policy_version 72240 (0.0008) [2023-10-08 06:44:11,929][00612] Updated weights for policy 1, policy_version 72250 (0.0007) [2023-10-08 06:44:13,219][00611] Updated weights for policy 0, policy_version 71842 (0.0008) [2023-10-08 06:44:13,592][00611] Updated weights for policy 0, policy_version 71852 (0.0007) [2023-10-08 06:44:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 147554304. Throughput: 0: 1840.1, 1: 1837.0. Samples: 36897380. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 06:44:13,754][130385] Avg episode reward: [(0, '63.290'), (1, '75.100')] [2023-10-08 06:44:13,961][00611] Updated weights for policy 0, policy_version 71862 (0.0009) [2023-10-08 06:44:14,337][00611] Updated weights for policy 0, policy_version 71872 (0.0010) [2023-10-08 06:44:15,492][00612] Updated weights for policy 1, policy_version 72260 (0.0007) [2023-10-08 06:44:15,863][00612] Updated weights for policy 1, policy_version 72270 (0.0009) [2023-10-08 06:44:16,243][00612] Updated weights for policy 1, policy_version 72280 (0.0008) [2023-10-08 06:44:18,111][00611] Updated weights for policy 0, policy_version 71882 (0.0011) [2023-10-08 06:44:18,483][00611] Updated weights for policy 0, policy_version 71892 (0.0011) [2023-10-08 06:44:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147619840. Throughput: 0: 1827.7, 1: 1854.2. Samples: 36920164. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 06:44:18,755][130385] Avg episode reward: [(0, '59.600'), (1, '74.210')] [2023-10-08 06:44:18,762][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000072288_74022912.pth... [2023-10-08 06:44:18,795][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000070560_72253440.pth [2023-10-08 06:44:18,854][00611] Updated weights for policy 0, policy_version 71902 (0.0007) [2023-10-08 06:44:18,926][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000071904_73629696.pth... [2023-10-08 06:44:18,961][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000070176_71860224.pth [2023-10-08 06:44:19,785][00612] Updated weights for policy 1, policy_version 72290 (0.0008) [2023-10-08 06:44:20,144][00612] Updated weights for policy 1, policy_version 72300 (0.0007) [2023-10-08 06:44:20,516][00612] Updated weights for policy 1, policy_version 72310 (0.0008) [2023-10-08 06:44:20,887][00612] Updated weights for policy 1, policy_version 72320 (0.0009) [2023-10-08 06:44:22,319][00611] Updated weights for policy 0, policy_version 71912 (0.0008) [2023-10-08 06:44:22,700][00611] Updated weights for policy 0, policy_version 71922 (0.0009) [2023-10-08 06:44:23,078][00611] Updated weights for policy 0, policy_version 71932 (0.0008) [2023-10-08 06:44:23,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 147718144. Throughput: 0: 1845.3, 1: 1840.7. Samples: 36930928. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 06:44:23,754][130385] Avg episode reward: [(0, '59.000'), (1, '78.880')] [2023-10-08 06:44:24,535][00612] Updated weights for policy 1, policy_version 72330 (0.0008) [2023-10-08 06:44:24,910][00612] Updated weights for policy 1, policy_version 72340 (0.0008) [2023-10-08 06:44:25,269][00612] Updated weights for policy 1, policy_version 72350 (0.0007) [2023-10-08 06:44:26,898][00611] Updated weights for policy 0, policy_version 71942 (0.0008) [2023-10-08 06:44:27,286][00611] Updated weights for policy 0, policy_version 71952 (0.0011) [2023-10-08 06:44:27,655][00611] Updated weights for policy 0, policy_version 71962 (0.0010) [2023-10-08 06:44:28,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 147783680. Throughput: 0: 1828.0, 1: 1844.4. Samples: 36952994. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 06:44:28,755][130385] Avg episode reward: [(0, '59.720'), (1, '80.890')] [2023-10-08 06:44:28,866][00612] Updated weights for policy 1, policy_version 72360 (0.0008) [2023-10-08 06:44:29,231][00612] Updated weights for policy 1, policy_version 72370 (0.0009) [2023-10-08 06:44:29,602][00612] Updated weights for policy 1, policy_version 72380 (0.0008) [2023-10-08 06:44:31,327][00611] Updated weights for policy 0, policy_version 71972 (0.0008) [2023-10-08 06:44:31,700][00611] Updated weights for policy 0, policy_version 71982 (0.0007) [2023-10-08 06:44:32,072][00611] Updated weights for policy 0, policy_version 71992 (0.0009) [2023-10-08 06:44:33,294][00612] Updated weights for policy 1, policy_version 72390 (0.0008) [2023-10-08 06:44:33,664][00612] Updated weights for policy 1, policy_version 72400 (0.0009) [2023-10-08 06:44:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147849216. Throughput: 0: 1839.3, 1: 1843.0. Samples: 36975068. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 06:44:33,754][130385] Avg episode reward: [(0, '62.510'), (1, '82.520')] [2023-10-08 06:44:34,033][00612] Updated weights for policy 1, policy_version 72410 (0.0007) [2023-10-08 06:44:35,734][00611] Updated weights for policy 0, policy_version 72002 (0.0010) [2023-10-08 06:44:36,100][00611] Updated weights for policy 0, policy_version 72012 (0.0007) [2023-10-08 06:44:36,474][00611] Updated weights for policy 0, policy_version 72022 (0.0008) [2023-10-08 06:44:36,844][00611] Updated weights for policy 0, policy_version 72032 (0.0009) [2023-10-08 06:44:37,661][00612] Updated weights for policy 1, policy_version 72420 (0.0009) [2023-10-08 06:44:38,023][00612] Updated weights for policy 1, policy_version 72430 (0.0010) [2023-10-08 06:44:38,390][00612] Updated weights for policy 1, policy_version 72440 (0.0010) [2023-10-08 06:44:38,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 147947520. Throughput: 0: 1830.6, 1: 1850.8. Samples: 36986280. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 06:44:38,754][130385] Avg episode reward: [(0, '64.920'), (1, '85.920')] [2023-10-08 06:44:40,440][00611] Updated weights for policy 0, policy_version 72042 (0.0010) [2023-10-08 06:44:40,817][00611] Updated weights for policy 0, policy_version 72052 (0.0011) [2023-10-08 06:44:41,180][00611] Updated weights for policy 0, policy_version 72062 (0.0009) [2023-10-08 06:44:42,035][00612] Updated weights for policy 1, policy_version 72450 (0.0007) [2023-10-08 06:44:42,446][00612] Updated weights for policy 1, policy_version 72460 (0.0007) [2023-10-08 06:44:42,821][00612] Updated weights for policy 1, policy_version 72470 (0.0007) [2023-10-08 06:44:43,186][00612] Updated weights for policy 1, policy_version 72480 (0.0008) [2023-10-08 06:44:43,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 148013056. Throughput: 0: 1850.2, 1: 1842.3. Samples: 37008288. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 06:44:43,755][130385] Avg episode reward: [(0, '68.630'), (1, '80.580')] [2023-10-08 06:44:44,757][00611] Updated weights for policy 0, policy_version 72072 (0.0008) [2023-10-08 06:44:45,130][00611] Updated weights for policy 0, policy_version 72082 (0.0010) [2023-10-08 06:44:45,498][00611] Updated weights for policy 0, policy_version 72092 (0.0009) [2023-10-08 06:44:46,627][00612] Updated weights for policy 1, policy_version 72490 (0.0007) [2023-10-08 06:44:47,000][00612] Updated weights for policy 1, policy_version 72500 (0.0011) [2023-10-08 06:44:47,367][00612] Updated weights for policy 1, policy_version 72510 (0.0010) [2023-10-08 06:44:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 148078592. Throughput: 0: 1853.0, 1: 1849.5. Samples: 37030386. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 06:44:48,754][130385] Avg episode reward: [(0, '65.700'), (1, '81.050')] [2023-10-08 06:44:49,310][00611] Updated weights for policy 0, policy_version 72102 (0.0009) [2023-10-08 06:44:49,682][00611] Updated weights for policy 0, policy_version 72112 (0.0009) [2023-10-08 06:44:50,055][00611] Updated weights for policy 0, policy_version 72122 (0.0010) [2023-10-08 06:44:51,016][00612] Updated weights for policy 1, policy_version 72520 (0.0009) [2023-10-08 06:44:51,377][00612] Updated weights for policy 1, policy_version 72530 (0.0010) [2023-10-08 06:44:51,744][00612] Updated weights for policy 1, policy_version 72540 (0.0009) [2023-10-08 06:44:53,628][00611] Updated weights for policy 0, policy_version 72132 (0.0008) [2023-10-08 06:44:53,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 148144128. Throughput: 0: 1849.3, 1: 1845.5. Samples: 37041302. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 06:44:53,754][130385] Avg episode reward: [(0, '65.740'), (1, '79.640')] [2023-10-08 06:44:53,999][00611] Updated weights for policy 0, policy_version 72142 (0.0007) [2023-10-08 06:44:54,387][00611] Updated weights for policy 0, policy_version 72152 (0.0009) [2023-10-08 06:44:55,590][00612] Updated weights for policy 1, policy_version 72550 (0.0009) [2023-10-08 06:44:55,960][00612] Updated weights for policy 1, policy_version 72560 (0.0009) [2023-10-08 06:44:56,318][00612] Updated weights for policy 1, policy_version 72570 (0.0008) [2023-10-08 06:44:58,018][00611] Updated weights for policy 0, policy_version 72162 (0.0010) [2023-10-08 06:44:58,378][00611] Updated weights for policy 0, policy_version 72172 (0.0008) [2023-10-08 06:44:58,743][00611] Updated weights for policy 0, policy_version 72182 (0.0008) [2023-10-08 06:44:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 148209664. Throughput: 0: 1841.2, 1: 1850.8. Samples: 37063518. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 06:44:58,755][130385] Avg episode reward: [(0, '65.970'), (1, '78.130')] [2023-10-08 06:44:59,121][00611] Updated weights for policy 0, policy_version 72192 (0.0009) [2023-10-08 06:44:59,825][00612] Updated weights for policy 1, policy_version 72580 (0.0009) [2023-10-08 06:45:00,184][00612] Updated weights for policy 1, policy_version 72590 (0.0009) [2023-10-08 06:45:00,562][00612] Updated weights for policy 1, policy_version 72600 (0.0010) [2023-10-08 06:45:02,791][00611] Updated weights for policy 0, policy_version 72202 (0.0007) [2023-10-08 06:45:03,161][00611] Updated weights for policy 0, policy_version 72212 (0.0009) [2023-10-08 06:45:03,527][00611] Updated weights for policy 0, policy_version 72222 (0.0008) [2023-10-08 06:45:03,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 148307968. Throughput: 0: 1832.8, 1: 1857.6. Samples: 37086236. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 06:45:03,755][130385] Avg episode reward: [(0, '66.970'), (1, '81.630')] [2023-10-08 06:45:04,179][00612] Updated weights for policy 1, policy_version 72610 (0.0010) [2023-10-08 06:45:04,544][00612] Updated weights for policy 1, policy_version 72620 (0.0007) [2023-10-08 06:45:04,917][00612] Updated weights for policy 1, policy_version 72630 (0.0007) [2023-10-08 06:45:05,292][00612] Updated weights for policy 1, policy_version 72640 (0.0008) [2023-10-08 06:45:07,116][00611] Updated weights for policy 0, policy_version 72232 (0.0008) [2023-10-08 06:45:07,484][00611] Updated weights for policy 0, policy_version 72242 (0.0009) [2023-10-08 06:45:07,856][00611] Updated weights for policy 0, policy_version 72252 (0.0007) [2023-10-08 06:45:08,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 148373504. Throughput: 0: 1838.3, 1: 1858.0. Samples: 37097258. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 06:45:08,754][130385] Avg episode reward: [(0, '68.070'), (1, '79.360')] [2023-10-08 06:45:08,917][00612] Updated weights for policy 1, policy_version 72650 (0.0009) [2023-10-08 06:45:09,286][00612] Updated weights for policy 1, policy_version 72660 (0.0008) [2023-10-08 06:45:09,645][00612] Updated weights for policy 1, policy_version 72670 (0.0009) [2023-10-08 06:45:11,407][00611] Updated weights for policy 0, policy_version 72262 (0.0009) [2023-10-08 06:45:11,777][00611] Updated weights for policy 0, policy_version 72272 (0.0009) [2023-10-08 06:45:12,155][00611] Updated weights for policy 0, policy_version 72282 (0.0009) [2023-10-08 06:45:13,255][00612] Updated weights for policy 1, policy_version 72680 (0.0008) [2023-10-08 06:45:13,622][00612] Updated weights for policy 1, policy_version 72690 (0.0009) [2023-10-08 06:45:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 148439040. Throughput: 0: 1838.4, 1: 1861.1. Samples: 37119470. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 06:45:13,755][130385] Avg episode reward: [(0, '71.120'), (1, '78.130')] [2023-10-08 06:45:13,981][00612] Updated weights for policy 1, policy_version 72700 (0.0007) [2023-10-08 06:45:15,754][00611] Updated weights for policy 0, policy_version 72292 (0.0009) [2023-10-08 06:45:16,124][00611] Updated weights for policy 0, policy_version 72302 (0.0008) [2023-10-08 06:45:16,498][00611] Updated weights for policy 0, policy_version 72312 (0.0007) [2023-10-08 06:45:17,407][00612] Updated weights for policy 1, policy_version 72710 (0.0007) [2023-10-08 06:45:17,783][00612] Updated weights for policy 1, policy_version 72720 (0.0007) [2023-10-08 06:45:18,153][00612] Updated weights for policy 1, policy_version 72730 (0.0007) [2023-10-08 06:45:18,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 148537344. Throughput: 0: 1856.8, 1: 1839.3. Samples: 37141396. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 06:45:18,754][130385] Avg episode reward: [(0, '69.960'), (1, '78.050')] [2023-10-08 06:45:20,063][00611] Updated weights for policy 0, policy_version 72322 (0.0009) [2023-10-08 06:45:20,434][00611] Updated weights for policy 0, policy_version 72332 (0.0008) [2023-10-08 06:45:20,797][00611] Updated weights for policy 0, policy_version 72342 (0.0007) [2023-10-08 06:45:21,166][00611] Updated weights for policy 0, policy_version 72352 (0.0008) [2023-10-08 06:45:21,734][00612] Updated weights for policy 1, policy_version 72740 (0.0008) [2023-10-08 06:45:22,095][00612] Updated weights for policy 1, policy_version 72750 (0.0008) [2023-10-08 06:45:22,464][00612] Updated weights for policy 1, policy_version 72760 (0.0007) [2023-10-08 06:45:23,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 148602880. Throughput: 0: 1839.5, 1: 1863.3. Samples: 37152908. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 06:45:23,754][130385] Avg episode reward: [(0, '72.380'), (1, '74.340')] [2023-10-08 06:45:24,843][00611] Updated weights for policy 0, policy_version 72362 (0.0007) [2023-10-08 06:45:25,214][00611] Updated weights for policy 0, policy_version 72372 (0.0007) [2023-10-08 06:45:25,591][00611] Updated weights for policy 0, policy_version 72382 (0.0008) [2023-10-08 06:45:26,198][00612] Updated weights for policy 1, policy_version 72770 (0.0009) [2023-10-08 06:45:26,606][00612] Updated weights for policy 1, policy_version 72780 (0.0008) [2023-10-08 06:45:26,967][00612] Updated weights for policy 1, policy_version 72790 (0.0007) [2023-10-08 06:45:27,328][00612] Updated weights for policy 1, policy_version 72800 (0.0008) [2023-10-08 06:45:28,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 148668416. Throughput: 0: 1856.6, 1: 1835.2. Samples: 37174416. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 06:45:28,754][130385] Avg episode reward: [(0, '70.790'), (1, '73.120')] [2023-10-08 06:45:29,164][00611] Updated weights for policy 0, policy_version 72392 (0.0007) [2023-10-08 06:45:29,538][00611] Updated weights for policy 0, policy_version 72402 (0.0009) [2023-10-08 06:45:29,903][00611] Updated weights for policy 0, policy_version 72412 (0.0008) [2023-10-08 06:45:30,968][00612] Updated weights for policy 1, policy_version 72810 (0.0010) [2023-10-08 06:45:31,334][00612] Updated weights for policy 1, policy_version 72820 (0.0008) [2023-10-08 06:45:31,707][00612] Updated weights for policy 1, policy_version 72830 (0.0008) [2023-10-08 06:45:33,558][00611] Updated weights for policy 0, policy_version 72422 (0.0007) [2023-10-08 06:45:33,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 148733952. Throughput: 0: 1850.3, 1: 1855.4. Samples: 37197142. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 06:45:33,755][130385] Avg episode reward: [(0, '69.740'), (1, '71.930')] [2023-10-08 06:45:33,934][00611] Updated weights for policy 0, policy_version 72432 (0.0009) [2023-10-08 06:45:34,304][00611] Updated weights for policy 0, policy_version 72442 (0.0010) [2023-10-08 06:45:35,416][00612] Updated weights for policy 1, policy_version 72840 (0.0008) [2023-10-08 06:45:35,776][00612] Updated weights for policy 1, policy_version 72850 (0.0010) [2023-10-08 06:45:36,156][00612] Updated weights for policy 1, policy_version 72860 (0.0007) [2023-10-08 06:45:37,937][00611] Updated weights for policy 0, policy_version 72452 (0.0009) [2023-10-08 06:45:38,305][00611] Updated weights for policy 0, policy_version 72462 (0.0010) [2023-10-08 06:45:38,670][00611] Updated weights for policy 0, policy_version 72472 (0.0010) [2023-10-08 06:45:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 148799488. Throughput: 0: 1855.8, 1: 1835.9. Samples: 37207428. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 06:45:38,755][130385] Avg episode reward: [(0, '70.000'), (1, '73.360')] [2023-10-08 06:45:39,813][00612] Updated weights for policy 1, policy_version 72870 (0.0008) [2023-10-08 06:45:40,180][00612] Updated weights for policy 1, policy_version 72880 (0.0007) [2023-10-08 06:45:40,549][00612] Updated weights for policy 1, policy_version 72890 (0.0009) [2023-10-08 06:45:42,374][00611] Updated weights for policy 0, policy_version 72482 (0.0010) [2023-10-08 06:45:42,751][00611] Updated weights for policy 0, policy_version 72492 (0.0007) [2023-10-08 06:45:43,121][00611] Updated weights for policy 0, policy_version 72502 (0.0008) [2023-10-08 06:45:43,502][00611] Updated weights for policy 0, policy_version 72512 (0.0007) [2023-10-08 06:45:43,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 148897792. Throughput: 0: 1852.0, 1: 1854.2. Samples: 37230298. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 06:45:43,754][130385] Avg episode reward: [(0, '70.430'), (1, '70.710')] [2023-10-08 06:45:44,183][00612] Updated weights for policy 1, policy_version 72900 (0.0007) [2023-10-08 06:45:44,549][00612] Updated weights for policy 1, policy_version 72910 (0.0007) [2023-10-08 06:45:44,915][00612] Updated weights for policy 1, policy_version 72920 (0.0007) [2023-10-08 06:45:47,150][00611] Updated weights for policy 0, policy_version 72522 (0.0007) [2023-10-08 06:45:47,518][00611] Updated weights for policy 0, policy_version 72532 (0.0007) [2023-10-08 06:45:47,884][00611] Updated weights for policy 0, policy_version 72542 (0.0008) [2023-10-08 06:45:48,621][00612] Updated weights for policy 1, policy_version 72930 (0.0010) [2023-10-08 06:45:48,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 148963328. Throughput: 0: 1835.4, 1: 1844.7. Samples: 37251842. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-08 06:45:48,755][130385] Avg episode reward: [(0, '69.330'), (1, '70.040')] [2023-10-08 06:45:48,991][00612] Updated weights for policy 1, policy_version 72940 (0.0011) [2023-10-08 06:45:49,363][00612] Updated weights for policy 1, policy_version 72950 (0.0008) [2023-10-08 06:45:49,731][00612] Updated weights for policy 1, policy_version 72960 (0.0008) [2023-10-08 06:45:51,552][00611] Updated weights for policy 0, policy_version 72552 (0.0010) [2023-10-08 06:45:51,923][00611] Updated weights for policy 0, policy_version 72562 (0.0011) [2023-10-08 06:45:52,295][00611] Updated weights for policy 0, policy_version 72572 (0.0010) [2023-10-08 06:45:53,237][00612] Updated weights for policy 1, policy_version 72970 (0.0007) [2023-10-08 06:45:53,600][00612] Updated weights for policy 1, policy_version 72980 (0.0008) [2023-10-08 06:45:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 149028864. Throughput: 0: 1847.5, 1: 1846.2. Samples: 37263474. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-08 06:45:53,754][130385] Avg episode reward: [(0, '68.490'), (1, '72.190')] [2023-10-08 06:45:53,978][00612] Updated weights for policy 1, policy_version 72990 (0.0008) [2023-10-08 06:45:55,985][00611] Updated weights for policy 0, policy_version 72582 (0.0009) [2023-10-08 06:45:56,360][00611] Updated weights for policy 0, policy_version 72592 (0.0008) [2023-10-08 06:45:56,724][00611] Updated weights for policy 0, policy_version 72602 (0.0008) [2023-10-08 06:45:57,746][00612] Updated weights for policy 1, policy_version 73000 (0.0010) [2023-10-08 06:45:58,116][00612] Updated weights for policy 1, policy_version 73010 (0.0008) [2023-10-08 06:45:58,480][00612] Updated weights for policy 1, policy_version 73020 (0.0009) [2023-10-08 06:45:58,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 149127168. Throughput: 0: 1828.9, 1: 1845.0. Samples: 37284794. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-08 06:45:58,755][130385] Avg episode reward: [(0, '69.070'), (1, '73.090')] [2023-10-08 06:46:00,434][00611] Updated weights for policy 0, policy_version 72612 (0.0010) [2023-10-08 06:46:00,802][00611] Updated weights for policy 0, policy_version 72622 (0.0010) [2023-10-08 06:46:01,172][00611] Updated weights for policy 0, policy_version 72632 (0.0010) [2023-10-08 06:46:02,218][00612] Updated weights for policy 1, policy_version 73030 (0.0010) [2023-10-08 06:46:02,590][00612] Updated weights for policy 1, policy_version 73040 (0.0009) [2023-10-08 06:46:02,955][00612] Updated weights for policy 1, policy_version 73050 (0.0010) [2023-10-08 06:46:03,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 149192704. Throughput: 0: 1832.0, 1: 1828.8. Samples: 37306132. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-08 06:46:03,754][130385] Avg episode reward: [(0, '69.910'), (1, '75.770')] [2023-10-08 06:46:04,923][00611] Updated weights for policy 0, policy_version 72642 (0.0008) [2023-10-08 06:46:05,324][00611] Updated weights for policy 0, policy_version 72652 (0.0007) [2023-10-08 06:46:05,696][00611] Updated weights for policy 0, policy_version 72662 (0.0010) [2023-10-08 06:46:06,067][00611] Updated weights for policy 0, policy_version 72672 (0.0007) [2023-10-08 06:46:06,641][00612] Updated weights for policy 1, policy_version 73060 (0.0007) [2023-10-08 06:46:07,011][00612] Updated weights for policy 1, policy_version 73070 (0.0009) [2023-10-08 06:46:07,386][00612] Updated weights for policy 1, policy_version 73080 (0.0007) [2023-10-08 06:46:08,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 149258240. Throughput: 0: 1824.8, 1: 1834.0. Samples: 37317552. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-08 06:46:08,754][130385] Avg episode reward: [(0, '70.580'), (1, '73.180')] [2023-10-08 06:46:09,519][00611] Updated weights for policy 0, policy_version 72682 (0.0008) [2023-10-08 06:46:09,891][00611] Updated weights for policy 0, policy_version 72692 (0.0007) [2023-10-08 06:46:10,255][00611] Updated weights for policy 0, policy_version 72702 (0.0009) [2023-10-08 06:46:11,112][00612] Updated weights for policy 1, policy_version 73090 (0.0008) [2023-10-08 06:46:11,486][00612] Updated weights for policy 1, policy_version 73100 (0.0011) [2023-10-08 06:46:11,849][00612] Updated weights for policy 1, policy_version 73110 (0.0008) [2023-10-08 06:46:12,216][00612] Updated weights for policy 1, policy_version 73120 (0.0010) [2023-10-08 06:46:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 149323776. Throughput: 0: 1838.4, 1: 1832.8. Samples: 37339618. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-08 06:46:13,754][130385] Avg episode reward: [(0, '68.550'), (1, '70.090')] [2023-10-08 06:46:13,818][00611] Updated weights for policy 0, policy_version 72712 (0.0010) [2023-10-08 06:46:14,185][00611] Updated weights for policy 0, policy_version 72722 (0.0009) [2023-10-08 06:46:14,553][00611] Updated weights for policy 0, policy_version 72732 (0.0007) [2023-10-08 06:46:15,968][00612] Updated weights for policy 1, policy_version 73130 (0.0010) [2023-10-08 06:46:16,330][00612] Updated weights for policy 1, policy_version 73140 (0.0007) [2023-10-08 06:46:16,700][00612] Updated weights for policy 1, policy_version 73150 (0.0007) [2023-10-08 06:46:18,200][00611] Updated weights for policy 0, policy_version 72742 (0.0009) [2023-10-08 06:46:18,574][00611] Updated weights for policy 0, policy_version 72752 (0.0008) [2023-10-08 06:46:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 149389312. Throughput: 0: 1832.8, 1: 1835.9. Samples: 37362234. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-08 06:46:18,755][130385] Avg episode reward: [(0, '69.370'), (1, '73.610')] [2023-10-08 06:46:18,765][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000073152_74907648.pth... [2023-10-08 06:46:18,799][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000071424_73138176.pth [2023-10-08 06:46:18,950][00611] Updated weights for policy 0, policy_version 72762 (0.0008) [2023-10-08 06:46:19,167][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000072768_74514432.pth... [2023-10-08 06:46:19,206][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000071040_72744960.pth [2023-10-08 06:46:20,289][00612] Updated weights for policy 1, policy_version 73160 (0.0007) [2023-10-08 06:46:20,660][00612] Updated weights for policy 1, policy_version 73170 (0.0008) [2023-10-08 06:46:21,024][00612] Updated weights for policy 1, policy_version 73180 (0.0007) [2023-10-08 06:46:22,801][00611] Updated weights for policy 0, policy_version 72772 (0.0008) [2023-10-08 06:46:23,167][00611] Updated weights for policy 0, policy_version 72782 (0.0008) [2023-10-08 06:46:23,536][00611] Updated weights for policy 0, policy_version 72792 (0.0008) [2023-10-08 06:46:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 149454848. Throughput: 0: 1834.8, 1: 1832.9. Samples: 37372472. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-08 06:46:23,754][130385] Avg episode reward: [(0, '70.590'), (1, '74.780')] [2023-10-08 06:46:24,580][00612] Updated weights for policy 1, policy_version 73190 (0.0008) [2023-10-08 06:46:24,956][00612] Updated weights for policy 1, policy_version 73200 (0.0009) [2023-10-08 06:46:25,320][00612] Updated weights for policy 1, policy_version 73210 (0.0008) [2023-10-08 06:46:27,149][00611] Updated weights for policy 0, policy_version 72802 (0.0008) [2023-10-08 06:46:27,517][00611] Updated weights for policy 0, policy_version 72812 (0.0009) [2023-10-08 06:46:27,882][00611] Updated weights for policy 0, policy_version 72822 (0.0009) [2023-10-08 06:46:28,256][00611] Updated weights for policy 0, policy_version 72832 (0.0010) [2023-10-08 06:46:28,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 149553152. Throughput: 0: 1826.2, 1: 1840.2. Samples: 37395286. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-08 06:46:28,755][130385] Avg episode reward: [(0, '70.620'), (1, '77.040')] [2023-10-08 06:46:28,829][00612] Updated weights for policy 1, policy_version 73220 (0.0008) [2023-10-08 06:46:29,203][00612] Updated weights for policy 1, policy_version 73230 (0.0009) [2023-10-08 06:46:29,569][00612] Updated weights for policy 1, policy_version 73240 (0.0008) [2023-10-08 06:46:31,994][00611] Updated weights for policy 0, policy_version 72842 (0.0011) [2023-10-08 06:46:32,360][00611] Updated weights for policy 0, policy_version 72852 (0.0009) [2023-10-08 06:46:32,745][00611] Updated weights for policy 0, policy_version 72862 (0.0009) [2023-10-08 06:46:33,090][00612] Updated weights for policy 1, policy_version 73250 (0.0008) [2023-10-08 06:46:33,456][00612] Updated weights for policy 1, policy_version 73260 (0.0007) [2023-10-08 06:46:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 149618688. Throughput: 0: 1827.4, 1: 1840.6. Samples: 37416900. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-08 06:46:33,754][130385] Avg episode reward: [(0, '68.330'), (1, '75.690')] [2023-10-08 06:46:33,827][00612] Updated weights for policy 1, policy_version 73270 (0.0009) [2023-10-08 06:46:34,188][00612] Updated weights for policy 1, policy_version 73280 (0.0008) [2023-10-08 06:46:36,359][00611] Updated weights for policy 0, policy_version 72872 (0.0007) [2023-10-08 06:46:36,734][00611] Updated weights for policy 0, policy_version 72882 (0.0010) [2023-10-08 06:46:37,106][00611] Updated weights for policy 0, policy_version 72892 (0.0009) [2023-10-08 06:46:37,665][00612] Updated weights for policy 1, policy_version 73290 (0.0009) [2023-10-08 06:46:38,039][00612] Updated weights for policy 1, policy_version 73300 (0.0007) [2023-10-08 06:46:38,402][00612] Updated weights for policy 1, policy_version 73310 (0.0007) [2023-10-08 06:46:38,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 149716992. Throughput: 0: 1823.9, 1: 1846.9. Samples: 37428662. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:46:38,755][130385] Avg episode reward: [(0, '68.070'), (1, '75.220')] [2023-10-08 06:46:40,770][00611] Updated weights for policy 0, policy_version 72902 (0.0008) [2023-10-08 06:46:41,138][00611] Updated weights for policy 0, policy_version 72912 (0.0009) [2023-10-08 06:46:41,513][00611] Updated weights for policy 0, policy_version 72922 (0.0007) [2023-10-08 06:46:41,957][00612] Updated weights for policy 1, policy_version 73320 (0.0008) [2023-10-08 06:46:42,327][00612] Updated weights for policy 1, policy_version 73330 (0.0009) [2023-10-08 06:46:42,692][00612] Updated weights for policy 1, policy_version 73340 (0.0007) [2023-10-08 06:46:43,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 149782528. Throughput: 0: 1829.2, 1: 1841.5. Samples: 37449972. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:46:43,755][130385] Avg episode reward: [(0, '67.120'), (1, '73.250')] [2023-10-08 06:46:45,231][00611] Updated weights for policy 0, policy_version 72932 (0.0009) [2023-10-08 06:46:45,600][00611] Updated weights for policy 0, policy_version 72942 (0.0010) [2023-10-08 06:46:45,968][00611] Updated weights for policy 0, policy_version 72952 (0.0011) [2023-10-08 06:46:46,316][00612] Updated weights for policy 1, policy_version 73350 (0.0008) [2023-10-08 06:46:46,677][00612] Updated weights for policy 1, policy_version 73360 (0.0009) [2023-10-08 06:46:47,040][00612] Updated weights for policy 1, policy_version 73370 (0.0007) [2023-10-08 06:46:48,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 149848064. Throughput: 0: 1833.8, 1: 1860.9. Samples: 37472396. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:46:48,754][130385] Avg episode reward: [(0, '67.010'), (1, '73.020')] [2023-10-08 06:46:49,669][00611] Updated weights for policy 0, policy_version 72962 (0.0007) [2023-10-08 06:46:50,046][00611] Updated weights for policy 0, policy_version 72972 (0.0010) [2023-10-08 06:46:50,411][00611] Updated weights for policy 0, policy_version 72982 (0.0011) [2023-10-08 06:46:50,695][00612] Updated weights for policy 1, policy_version 73380 (0.0008) [2023-10-08 06:46:50,782][00611] Updated weights for policy 0, policy_version 72992 (0.0010) [2023-10-08 06:46:51,071][00612] Updated weights for policy 1, policy_version 73390 (0.0008) [2023-10-08 06:46:51,429][00612] Updated weights for policy 1, policy_version 73400 (0.0008) [2023-10-08 06:46:53,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 149913600. Throughput: 0: 1833.2, 1: 1841.7. Samples: 37482920. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:46:53,755][130385] Avg episode reward: [(0, '68.160'), (1, '75.610')] [2023-10-08 06:46:54,231][00611] Updated weights for policy 0, policy_version 73002 (0.0008) [2023-10-08 06:46:54,601][00611] Updated weights for policy 0, policy_version 73012 (0.0008) [2023-10-08 06:46:54,970][00611] Updated weights for policy 0, policy_version 73022 (0.0009) [2023-10-08 06:46:55,258][00612] Updated weights for policy 1, policy_version 73410 (0.0008) [2023-10-08 06:46:55,629][00612] Updated weights for policy 1, policy_version 73420 (0.0011) [2023-10-08 06:46:55,997][00612] Updated weights for policy 1, policy_version 73430 (0.0008) [2023-10-08 06:46:56,363][00612] Updated weights for policy 1, policy_version 73440 (0.0009) [2023-10-08 06:46:58,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 149979136. Throughput: 0: 1828.7, 1: 1855.9. Samples: 37505430. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:46:58,755][130385] Avg episode reward: [(0, '66.540'), (1, '72.940')] [2023-10-08 06:46:58,778][00611] Updated weights for policy 0, policy_version 73032 (0.0010) [2023-10-08 06:46:59,157][00611] Updated weights for policy 0, policy_version 73042 (0.0008) [2023-10-08 06:46:59,532][00611] Updated weights for policy 0, policy_version 73052 (0.0007) [2023-10-08 06:46:59,921][00612] Updated weights for policy 1, policy_version 73450 (0.0010) [2023-10-08 06:47:00,294][00612] Updated weights for policy 1, policy_version 73460 (0.0009) [2023-10-08 06:47:00,660][00612] Updated weights for policy 1, policy_version 73470 (0.0010) [2023-10-08 06:47:03,191][00611] Updated weights for policy 0, policy_version 73062 (0.0009) [2023-10-08 06:47:03,564][00611] Updated weights for policy 0, policy_version 73072 (0.0008) [2023-10-08 06:47:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 150044672. Throughput: 0: 1825.9, 1: 1862.3. Samples: 37528200. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:47:03,754][130385] Avg episode reward: [(0, '66.040'), (1, '77.670')] [2023-10-08 06:47:03,935][00611] Updated weights for policy 0, policy_version 73082 (0.0010) [2023-10-08 06:47:04,275][00612] Updated weights for policy 1, policy_version 73480 (0.0008) [2023-10-08 06:47:04,647][00612] Updated weights for policy 1, policy_version 73490 (0.0007) [2023-10-08 06:47:05,011][00612] Updated weights for policy 1, policy_version 73500 (0.0009) [2023-10-08 06:47:07,575][00611] Updated weights for policy 0, policy_version 73092 (0.0009) [2023-10-08 06:47:07,945][00611] Updated weights for policy 0, policy_version 73102 (0.0011) [2023-10-08 06:47:08,316][00611] Updated weights for policy 0, policy_version 73112 (0.0010) [2023-10-08 06:47:08,499][00612] Updated weights for policy 1, policy_version 73510 (0.0008) [2023-10-08 06:47:08,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 150142976. Throughput: 0: 1832.3, 1: 1856.1. Samples: 37538454. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:47:08,755][130385] Avg episode reward: [(0, '65.960'), (1, '75.110')] [2023-10-08 06:47:08,869][00612] Updated weights for policy 1, policy_version 73520 (0.0009) [2023-10-08 06:47:09,245][00612] Updated weights for policy 1, policy_version 73530 (0.0010) [2023-10-08 06:47:11,891][00611] Updated weights for policy 0, policy_version 73122 (0.0008) [2023-10-08 06:47:12,262][00611] Updated weights for policy 0, policy_version 73132 (0.0009) [2023-10-08 06:47:12,636][00611] Updated weights for policy 0, policy_version 73142 (0.0008) [2023-10-08 06:47:13,008][00611] Updated weights for policy 0, policy_version 73152 (0.0008) [2023-10-08 06:47:13,081][00612] Updated weights for policy 1, policy_version 73540 (0.0007) [2023-10-08 06:47:13,450][00612] Updated weights for policy 1, policy_version 73550 (0.0008) [2023-10-08 06:47:13,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 150208512. Throughput: 0: 1827.8, 1: 1860.6. Samples: 37561264. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:47:13,754][130385] Avg episode reward: [(0, '67.720'), (1, '76.980')] [2023-10-08 06:47:13,817][00612] Updated weights for policy 1, policy_version 73560 (0.0008) [2023-10-08 06:47:16,615][00611] Updated weights for policy 0, policy_version 73162 (0.0007) [2023-10-08 06:47:16,988][00611] Updated weights for policy 0, policy_version 73172 (0.0010) [2023-10-08 06:47:17,357][00611] Updated weights for policy 0, policy_version 73182 (0.0010) [2023-10-08 06:47:17,428][00612] Updated weights for policy 1, policy_version 73570 (0.0008) [2023-10-08 06:47:17,805][00612] Updated weights for policy 1, policy_version 73580 (0.0008) [2023-10-08 06:47:18,177][00612] Updated weights for policy 1, policy_version 73590 (0.0010) [2023-10-08 06:47:18,549][00612] Updated weights for policy 1, policy_version 73600 (0.0010) [2023-10-08 06:47:18,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 150306816. Throughput: 0: 1839.5, 1: 1842.2. Samples: 37582580. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:47:18,755][130385] Avg episode reward: [(0, '68.700'), (1, '75.700')] [2023-10-08 06:47:21,001][00611] Updated weights for policy 0, policy_version 73192 (0.0010) [2023-10-08 06:47:21,371][00611] Updated weights for policy 0, policy_version 73202 (0.0011) [2023-10-08 06:47:21,742][00611] Updated weights for policy 0, policy_version 73212 (0.0010) [2023-10-08 06:47:22,197][00612] Updated weights for policy 1, policy_version 73610 (0.0007) [2023-10-08 06:47:22,564][00612] Updated weights for policy 1, policy_version 73620 (0.0009) [2023-10-08 06:47:22,932][00612] Updated weights for policy 1, policy_version 73630 (0.0008) [2023-10-08 06:47:23,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 150372352. Throughput: 0: 1829.4, 1: 1857.4. Samples: 37594570. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 06:47:23,755][130385] Avg episode reward: [(0, '67.740'), (1, '80.820')] [2023-10-08 06:47:25,301][00611] Updated weights for policy 0, policy_version 73222 (0.0008) [2023-10-08 06:47:25,665][00611] Updated weights for policy 0, policy_version 73232 (0.0009) [2023-10-08 06:47:26,039][00611] Updated weights for policy 0, policy_version 73242 (0.0008) [2023-10-08 06:47:26,615][00612] Updated weights for policy 1, policy_version 73640 (0.0007) [2023-10-08 06:47:26,985][00612] Updated weights for policy 1, policy_version 73650 (0.0007) [2023-10-08 06:47:27,361][00612] Updated weights for policy 1, policy_version 73660 (0.0008) [2023-10-08 06:47:28,754][130385] Fps is (10 sec: 13106.6, 60 sec: 14745.4, 300 sec: 14773.3). Total num frames: 150437888. Throughput: 0: 1841.3, 1: 1842.9. Samples: 37615764. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-08 06:47:28,755][130385] Avg episode reward: [(0, '70.490'), (1, '79.450')] [2023-10-08 06:47:29,741][00611] Updated weights for policy 0, policy_version 73252 (0.0009) [2023-10-08 06:47:30,112][00611] Updated weights for policy 0, policy_version 73262 (0.0007) [2023-10-08 06:47:30,478][00611] Updated weights for policy 0, policy_version 73272 (0.0009) [2023-10-08 06:47:30,993][00612] Updated weights for policy 1, policy_version 73670 (0.0007) [2023-10-08 06:47:31,362][00612] Updated weights for policy 1, policy_version 73680 (0.0007) [2023-10-08 06:47:31,726][00612] Updated weights for policy 1, policy_version 73690 (0.0008) [2023-10-08 06:47:33,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 150503424. Throughput: 0: 1840.0, 1: 1848.6. Samples: 37638380. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-08 06:47:33,754][130385] Avg episode reward: [(0, '70.260'), (1, '79.360')] [2023-10-08 06:47:34,046][00611] Updated weights for policy 0, policy_version 73282 (0.0009) [2023-10-08 06:47:34,423][00611] Updated weights for policy 0, policy_version 73292 (0.0011) [2023-10-08 06:47:34,794][00611] Updated weights for policy 0, policy_version 73302 (0.0010) [2023-10-08 06:47:35,158][00612] Updated weights for policy 1, policy_version 73700 (0.0009) [2023-10-08 06:47:35,160][00611] Updated weights for policy 0, policy_version 73312 (0.0007) [2023-10-08 06:47:35,517][00612] Updated weights for policy 1, policy_version 73710 (0.0010) [2023-10-08 06:47:35,890][00612] Updated weights for policy 1, policy_version 73720 (0.0008) [2023-10-08 06:47:38,754][130385] Fps is (10 sec: 13108.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 150568960. Throughput: 0: 1846.4, 1: 1838.0. Samples: 37648718. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-08 06:47:38,754][130385] Avg episode reward: [(0, '68.130'), (1, '76.010')] [2023-10-08 06:47:38,793][00611] Updated weights for policy 0, policy_version 73322 (0.0007) [2023-10-08 06:47:39,164][00611] Updated weights for policy 0, policy_version 73332 (0.0009) [2023-10-08 06:47:39,414][00612] Updated weights for policy 1, policy_version 73730 (0.0008) [2023-10-08 06:47:39,548][00611] Updated weights for policy 0, policy_version 73342 (0.0009) [2023-10-08 06:47:39,779][00612] Updated weights for policy 1, policy_version 73740 (0.0008) [2023-10-08 06:47:40,148][00612] Updated weights for policy 1, policy_version 73750 (0.0007) [2023-10-08 06:47:40,518][00612] Updated weights for policy 1, policy_version 73760 (0.0009) [2023-10-08 06:47:43,163][00611] Updated weights for policy 0, policy_version 73352 (0.0007) [2023-10-08 06:47:43,543][00611] Updated weights for policy 0, policy_version 73362 (0.0008) [2023-10-08 06:47:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 150634496. Throughput: 0: 1839.4, 1: 1856.8. Samples: 37671758. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-08 06:47:43,754][130385] Avg episode reward: [(0, '72.320'), (1, '78.210')] [2023-10-08 06:47:43,912][00611] Updated weights for policy 0, policy_version 73372 (0.0010) [2023-10-08 06:47:44,207][00612] Updated weights for policy 1, policy_version 73770 (0.0009) [2023-10-08 06:47:44,579][00612] Updated weights for policy 1, policy_version 73780 (0.0007) [2023-10-08 06:47:44,950][00612] Updated weights for policy 1, policy_version 73790 (0.0007) [2023-10-08 06:47:47,793][00611] Updated weights for policy 0, policy_version 73382 (0.0007) [2023-10-08 06:47:48,169][00611] Updated weights for policy 0, policy_version 73392 (0.0007) [2023-10-08 06:47:48,537][00611] Updated weights for policy 0, policy_version 73402 (0.0009) [2023-10-08 06:47:48,750][00612] Updated weights for policy 1, policy_version 73800 (0.0008) [2023-10-08 06:47:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 150700032. Throughput: 0: 1825.6, 1: 1852.0. Samples: 37693692. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-08 06:47:48,754][130385] Avg episode reward: [(0, '76.920'), (1, '78.150')] [2023-10-08 06:47:49,131][00612] Updated weights for policy 1, policy_version 73810 (0.0008) [2023-10-08 06:47:49,500][00612] Updated weights for policy 1, policy_version 73820 (0.0008) [2023-10-08 06:47:52,145][00611] Updated weights for policy 0, policy_version 73412 (0.0009) [2023-10-08 06:47:52,513][00611] Updated weights for policy 0, policy_version 73422 (0.0009) [2023-10-08 06:47:52,884][00611] Updated weights for policy 0, policy_version 73432 (0.0007) [2023-10-08 06:47:53,017][00612] Updated weights for policy 1, policy_version 73830 (0.0008) [2023-10-08 06:47:53,388][00612] Updated weights for policy 1, policy_version 73840 (0.0007) [2023-10-08 06:47:53,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 150798336. Throughput: 0: 1833.7, 1: 1850.3. Samples: 37704232. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-08 06:47:53,754][130385] Avg episode reward: [(0, '78.280'), (1, '80.860')] [2023-10-08 06:47:53,755][00612] Updated weights for policy 1, policy_version 73850 (0.0007) [2023-10-08 06:47:56,672][00611] Updated weights for policy 0, policy_version 73442 (0.0008) [2023-10-08 06:47:57,038][00611] Updated weights for policy 0, policy_version 73452 (0.0010) [2023-10-08 06:47:57,346][00612] Updated weights for policy 1, policy_version 73860 (0.0007) [2023-10-08 06:47:57,412][00611] Updated weights for policy 0, policy_version 73462 (0.0009) [2023-10-08 06:47:57,724][00612] Updated weights for policy 1, policy_version 73870 (0.0007) [2023-10-08 06:47:57,781][00611] Updated weights for policy 0, policy_version 73472 (0.0008) [2023-10-08 06:47:58,091][00612] Updated weights for policy 1, policy_version 73880 (0.0009) [2023-10-08 06:47:58,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 150896640. Throughput: 0: 1827.0, 1: 1848.6. Samples: 37726668. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-08 06:47:58,755][130385] Avg episode reward: [(0, '78.120'), (1, '82.440')] [2023-10-08 06:48:01,444][00611] Updated weights for policy 0, policy_version 73482 (0.0007) [2023-10-08 06:48:01,822][00611] Updated weights for policy 0, policy_version 73492 (0.0008) [2023-10-08 06:48:01,916][00612] Updated weights for policy 1, policy_version 73890 (0.0011) [2023-10-08 06:48:02,200][00611] Updated weights for policy 0, policy_version 73502 (0.0009) [2023-10-08 06:48:02,287][00612] Updated weights for policy 1, policy_version 73900 (0.0008) [2023-10-08 06:48:02,648][00612] Updated weights for policy 1, policy_version 73910 (0.0010) [2023-10-08 06:48:03,012][00612] Updated weights for policy 1, policy_version 73920 (0.0010) [2023-10-08 06:48:03,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 150962176. Throughput: 0: 1828.2, 1: 1827.4. Samples: 37747082. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-08 06:48:03,754][130385] Avg episode reward: [(0, '75.970'), (1, '81.820')] [2023-10-08 06:48:05,899][00611] Updated weights for policy 0, policy_version 73512 (0.0008) [2023-10-08 06:48:06,256][00611] Updated weights for policy 0, policy_version 73522 (0.0008) [2023-10-08 06:48:06,625][00612] Updated weights for policy 1, policy_version 73930 (0.0008) [2023-10-08 06:48:06,639][00611] Updated weights for policy 0, policy_version 73532 (0.0009) [2023-10-08 06:48:06,998][00612] Updated weights for policy 1, policy_version 73940 (0.0009) [2023-10-08 06:48:07,369][00612] Updated weights for policy 1, policy_version 73950 (0.0008) [2023-10-08 06:48:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 151027712. Throughput: 0: 1827.2, 1: 1835.4. Samples: 37759386. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-08 06:48:08,754][130385] Avg episode reward: [(0, '76.270'), (1, '82.520')] [2023-10-08 06:48:10,141][00611] Updated weights for policy 0, policy_version 73542 (0.0008) [2023-10-08 06:48:10,507][00611] Updated weights for policy 0, policy_version 73552 (0.0011) [2023-10-08 06:48:10,881][00611] Updated weights for policy 0, policy_version 73562 (0.0008) [2023-10-08 06:48:11,006][00612] Updated weights for policy 1, policy_version 73960 (0.0008) [2023-10-08 06:48:11,379][00612] Updated weights for policy 1, policy_version 73970 (0.0012) [2023-10-08 06:48:11,739][00612] Updated weights for policy 1, policy_version 73980 (0.0011) [2023-10-08 06:48:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 151093248. Throughput: 0: 1832.0, 1: 1823.5. Samples: 37780258. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-08 06:48:13,754][130385] Avg episode reward: [(0, '80.410'), (1, '88.110')] [2023-10-08 06:48:14,530][00611] Updated weights for policy 0, policy_version 73572 (0.0010) [2023-10-08 06:48:14,904][00611] Updated weights for policy 0, policy_version 73582 (0.0009) [2023-10-08 06:48:15,282][00611] Updated weights for policy 0, policy_version 73592 (0.0010) [2023-10-08 06:48:15,417][00612] Updated weights for policy 1, policy_version 73990 (0.0010) [2023-10-08 06:48:15,791][00612] Updated weights for policy 1, policy_version 74000 (0.0007) [2023-10-08 06:48:16,156][00612] Updated weights for policy 1, policy_version 74010 (0.0007) [2023-10-08 06:48:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 151158784. Throughput: 0: 1829.1, 1: 1838.8. Samples: 37803438. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-08 06:48:18,755][130385] Avg episode reward: [(0, '80.060'), (1, '84.770')] [2023-10-08 06:48:18,763][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000074016_75792384.pth... [2023-10-08 06:48:18,796][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000072288_74022912.pth [2023-10-08 06:48:18,927][00611] Updated weights for policy 0, policy_version 73602 (0.0009) [2023-10-08 06:48:19,294][00611] Updated weights for policy 0, policy_version 73612 (0.0007) [2023-10-08 06:48:19,671][00611] Updated weights for policy 0, policy_version 73622 (0.0007) [2023-10-08 06:48:19,743][00612] Updated weights for policy 1, policy_version 74020 (0.0008) [2023-10-08 06:48:20,039][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000073632_75399168.pth... [2023-10-08 06:48:20,040][00611] Updated weights for policy 0, policy_version 73632 (0.0007) [2023-10-08 06:48:20,076][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000071904_73629696.pth [2023-10-08 06:48:20,107][00612] Updated weights for policy 1, policy_version 74030 (0.0008) [2023-10-08 06:48:20,473][00612] Updated weights for policy 1, policy_version 74040 (0.0009) [2023-10-08 06:48:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 151224320. Throughput: 0: 1828.8, 1: 1837.0. Samples: 37813680. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-08 06:48:23,754][130385] Avg episode reward: [(0, '76.960'), (1, '85.800')] [2023-10-08 06:48:23,803][00611] Updated weights for policy 0, policy_version 73642 (0.0007) [2023-10-08 06:48:24,045][00612] Updated weights for policy 1, policy_version 74050 (0.0009) [2023-10-08 06:48:24,175][00611] Updated weights for policy 0, policy_version 73652 (0.0008) [2023-10-08 06:48:24,414][00612] Updated weights for policy 1, policy_version 74060 (0.0008) [2023-10-08 06:48:24,537][00611] Updated weights for policy 0, policy_version 73662 (0.0008) [2023-10-08 06:48:24,770][00612] Updated weights for policy 1, policy_version 74070 (0.0010) [2023-10-08 06:48:25,134][00612] Updated weights for policy 1, policy_version 74080 (0.0008) [2023-10-08 06:48:28,329][00611] Updated weights for policy 0, policy_version 73672 (0.0009) [2023-10-08 06:48:28,707][00611] Updated weights for policy 0, policy_version 73682 (0.0009) [2023-10-08 06:48:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.6, 300 sec: 14662.3). Total num frames: 151289856. Throughput: 0: 1822.4, 1: 1838.0. Samples: 37836478. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-08 06:48:28,754][130385] Avg episode reward: [(0, '74.920'), (1, '85.360')] [2023-10-08 06:48:28,927][00612] Updated weights for policy 1, policy_version 74090 (0.0008) [2023-10-08 06:48:29,069][00611] Updated weights for policy 0, policy_version 73692 (0.0007) [2023-10-08 06:48:29,303][00612] Updated weights for policy 1, policy_version 74100 (0.0010) [2023-10-08 06:48:29,661][00612] Updated weights for policy 1, policy_version 74110 (0.0011) [2023-10-08 06:48:32,697][00611] Updated weights for policy 0, policy_version 73702 (0.0007) [2023-10-08 06:48:33,064][00611] Updated weights for policy 0, policy_version 73712 (0.0009) [2023-10-08 06:48:33,214][00612] Updated weights for policy 1, policy_version 74120 (0.0008) [2023-10-08 06:48:33,424][00611] Updated weights for policy 0, policy_version 73722 (0.0007) [2023-10-08 06:48:33,585][00612] Updated weights for policy 1, policy_version 74130 (0.0010) [2023-10-08 06:48:33,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 151388160. Throughput: 0: 1824.4, 1: 1833.7. Samples: 37858304. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-08 06:48:33,754][130385] Avg episode reward: [(0, '73.700'), (1, '81.490')] [2023-10-08 06:48:33,963][00612] Updated weights for policy 1, policy_version 74140 (0.0009) [2023-10-08 06:48:36,914][00611] Updated weights for policy 0, policy_version 73732 (0.0009) [2023-10-08 06:48:37,285][00611] Updated weights for policy 0, policy_version 73742 (0.0009) [2023-10-08 06:48:37,650][00611] Updated weights for policy 0, policy_version 73752 (0.0008) [2023-10-08 06:48:37,760][00612] Updated weights for policy 1, policy_version 74150 (0.0008) [2023-10-08 06:48:38,135][00612] Updated weights for policy 1, policy_version 74160 (0.0009) [2023-10-08 06:48:38,512][00612] Updated weights for policy 1, policy_version 74170 (0.0008) [2023-10-08 06:48:38,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 151486464. Throughput: 0: 1831.9, 1: 1842.2. Samples: 37869568. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-08 06:48:38,755][130385] Avg episode reward: [(0, '70.430'), (1, '80.650')] [2023-10-08 06:48:41,316][00611] Updated weights for policy 0, policy_version 73762 (0.0008) [2023-10-08 06:48:41,674][00611] Updated weights for policy 0, policy_version 73772 (0.0011) [2023-10-08 06:48:42,041][00611] Updated weights for policy 0, policy_version 73782 (0.0010) [2023-10-08 06:48:42,155][00612] Updated weights for policy 1, policy_version 74180 (0.0008) [2023-10-08 06:48:42,416][00611] Updated weights for policy 0, policy_version 73792 (0.0008) [2023-10-08 06:48:42,516][00612] Updated weights for policy 1, policy_version 74190 (0.0007) [2023-10-08 06:48:42,877][00612] Updated weights for policy 1, policy_version 74200 (0.0008) [2023-10-08 06:48:43,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 151552000. Throughput: 0: 1827.4, 1: 1829.0. Samples: 37891206. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-08 06:48:43,755][130385] Avg episode reward: [(0, '70.420'), (1, '80.830')] [2023-10-08 06:48:46,211][00611] Updated weights for policy 0, policy_version 73802 (0.0008) [2023-10-08 06:48:46,576][00611] Updated weights for policy 0, policy_version 73812 (0.0008) [2023-10-08 06:48:46,624][00612] Updated weights for policy 1, policy_version 74210 (0.0007) [2023-10-08 06:48:46,952][00611] Updated weights for policy 0, policy_version 73822 (0.0008) [2023-10-08 06:48:47,000][00612] Updated weights for policy 1, policy_version 74220 (0.0008) [2023-10-08 06:48:47,368][00612] Updated weights for policy 1, policy_version 74230 (0.0008) [2023-10-08 06:48:47,736][00612] Updated weights for policy 1, policy_version 74240 (0.0010) [2023-10-08 06:48:48,754][130385] Fps is (10 sec: 13107.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 151617536. Throughput: 0: 1837.2, 1: 1834.4. Samples: 37912302. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-08 06:48:48,754][130385] Avg episode reward: [(0, '70.250'), (1, '81.580')] [2023-10-08 06:48:50,679][00611] Updated weights for policy 0, policy_version 73832 (0.0008) [2023-10-08 06:48:51,058][00611] Updated weights for policy 0, policy_version 73842 (0.0008) [2023-10-08 06:48:51,307][00612] Updated weights for policy 1, policy_version 74250 (0.0007) [2023-10-08 06:48:51,421][00611] Updated weights for policy 0, policy_version 73852 (0.0007) [2023-10-08 06:48:51,668][00612] Updated weights for policy 1, policy_version 74260 (0.0009) [2023-10-08 06:48:52,041][00612] Updated weights for policy 1, policy_version 74270 (0.0008) [2023-10-08 06:48:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 151683072. Throughput: 0: 1823.6, 1: 1831.6. Samples: 37923872. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-08 06:48:53,755][130385] Avg episode reward: [(0, '69.250'), (1, '82.670')] [2023-10-08 06:48:55,149][00611] Updated weights for policy 0, policy_version 73862 (0.0008) [2023-10-08 06:48:55,524][00611] Updated weights for policy 0, policy_version 73872 (0.0010) [2023-10-08 06:48:55,785][00612] Updated weights for policy 1, policy_version 74280 (0.0009) [2023-10-08 06:48:55,892][00611] Updated weights for policy 0, policy_version 73882 (0.0008) [2023-10-08 06:48:56,151][00612] Updated weights for policy 1, policy_version 74290 (0.0009) [2023-10-08 06:48:56,519][00612] Updated weights for policy 1, policy_version 74300 (0.0008) [2023-10-08 06:48:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 151748608. Throughput: 0: 1823.7, 1: 1833.0. Samples: 37944810. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-08 06:48:58,754][130385] Avg episode reward: [(0, '68.510'), (1, '81.080')] [2023-10-08 06:48:59,602][00611] Updated weights for policy 0, policy_version 73892 (0.0008) [2023-10-08 06:48:59,966][00611] Updated weights for policy 0, policy_version 73902 (0.0008) [2023-10-08 06:49:00,206][00612] Updated weights for policy 1, policy_version 74310 (0.0008) [2023-10-08 06:49:00,341][00611] Updated weights for policy 0, policy_version 73912 (0.0007) [2023-10-08 06:49:00,575][00612] Updated weights for policy 1, policy_version 74320 (0.0010) [2023-10-08 06:49:00,934][00612] Updated weights for policy 1, policy_version 74330 (0.0010) [2023-10-08 06:49:03,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 151814144. Throughput: 0: 1824.7, 1: 1828.5. Samples: 37967834. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-08 06:49:03,754][130385] Avg episode reward: [(0, '68.290'), (1, '82.610')] [2023-10-08 06:49:03,983][00611] Updated weights for policy 0, policy_version 73922 (0.0008) [2023-10-08 06:49:04,358][00611] Updated weights for policy 0, policy_version 73932 (0.0008) [2023-10-08 06:49:04,544][00612] Updated weights for policy 1, policy_version 74340 (0.0010) [2023-10-08 06:49:04,736][00611] Updated weights for policy 0, policy_version 73942 (0.0008) [2023-10-08 06:49:04,906][00612] Updated weights for policy 1, policy_version 74350 (0.0007) [2023-10-08 06:49:05,111][00611] Updated weights for policy 0, policy_version 73952 (0.0007) [2023-10-08 06:49:05,274][00612] Updated weights for policy 1, policy_version 74360 (0.0009) [2023-10-08 06:49:08,650][00611] Updated weights for policy 0, policy_version 73962 (0.0008) [2023-10-08 06:49:08,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 151879680. Throughput: 0: 1817.5, 1: 1826.3. Samples: 37977654. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-08 06:49:08,755][130385] Avg episode reward: [(0, '68.510'), (1, '82.930')] [2023-10-08 06:49:08,941][00612] Updated weights for policy 1, policy_version 74370 (0.0008) [2023-10-08 06:49:09,020][00611] Updated weights for policy 0, policy_version 73972 (0.0011) [2023-10-08 06:49:09,304][00612] Updated weights for policy 1, policy_version 74380 (0.0007) [2023-10-08 06:49:09,386][00611] Updated weights for policy 0, policy_version 73982 (0.0007) [2023-10-08 06:49:09,679][00612] Updated weights for policy 1, policy_version 74390 (0.0007) [2023-10-08 06:49:10,048][00612] Updated weights for policy 1, policy_version 74400 (0.0008) [2023-10-08 06:49:13,038][00611] Updated weights for policy 0, policy_version 73992 (0.0010) [2023-10-08 06:49:13,402][00611] Updated weights for policy 0, policy_version 74002 (0.0009) [2023-10-08 06:49:13,690][00612] Updated weights for policy 1, policy_version 74410 (0.0009) [2023-10-08 06:49:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 151945216. Throughput: 0: 1833.0, 1: 1832.0. Samples: 38001404. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-08 06:49:13,757][130385] Avg episode reward: [(0, '67.350'), (1, '86.270')] [2023-10-08 06:49:13,784][00611] Updated weights for policy 0, policy_version 74012 (0.0008) [2023-10-08 06:49:14,065][00612] Updated weights for policy 1, policy_version 74420 (0.0009) [2023-10-08 06:49:14,431][00612] Updated weights for policy 1, policy_version 74430 (0.0008) [2023-10-08 06:49:17,383][00611] Updated weights for policy 0, policy_version 74022 (0.0009) [2023-10-08 06:49:17,754][00611] Updated weights for policy 0, policy_version 74032 (0.0009) [2023-10-08 06:49:18,122][00611] Updated weights for policy 0, policy_version 74042 (0.0009) [2023-10-08 06:49:18,199][00612] Updated weights for policy 1, policy_version 74440 (0.0009) [2023-10-08 06:49:18,562][00612] Updated weights for policy 1, policy_version 74450 (0.0010) [2023-10-08 06:49:18,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 152043520. Throughput: 0: 1825.1, 1: 1823.1. Samples: 38022474. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-08 06:49:18,755][130385] Avg episode reward: [(0, '65.530'), (1, '87.680')] [2023-10-08 06:49:18,931][00612] Updated weights for policy 1, policy_version 74460 (0.0012) [2023-10-08 06:49:21,969][00611] Updated weights for policy 0, policy_version 74052 (0.0008) [2023-10-08 06:49:22,344][00611] Updated weights for policy 0, policy_version 74062 (0.0007) [2023-10-08 06:49:22,548][00612] Updated weights for policy 1, policy_version 74470 (0.0010) [2023-10-08 06:49:22,712][00611] Updated weights for policy 0, policy_version 74072 (0.0008) [2023-10-08 06:49:22,926][00612] Updated weights for policy 1, policy_version 74480 (0.0007) [2023-10-08 06:49:23,291][00612] Updated weights for policy 1, policy_version 74490 (0.0007) [2023-10-08 06:49:23,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 152141824. Throughput: 0: 1827.3, 1: 1827.3. Samples: 38034026. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-08 06:49:23,755][130385] Avg episode reward: [(0, '63.850'), (1, '87.000')] [2023-10-08 06:49:26,273][00611] Updated weights for policy 0, policy_version 74082 (0.0009) [2023-10-08 06:49:26,645][00611] Updated weights for policy 0, policy_version 74092 (0.0010) [2023-10-08 06:49:27,009][00611] Updated weights for policy 0, policy_version 74102 (0.0009) [2023-10-08 06:49:27,059][00612] Updated weights for policy 1, policy_version 74500 (0.0008) [2023-10-08 06:49:27,381][00611] Updated weights for policy 0, policy_version 74112 (0.0007) [2023-10-08 06:49:27,429][00612] Updated weights for policy 1, policy_version 74510 (0.0008) [2023-10-08 06:49:27,800][00612] Updated weights for policy 1, policy_version 74520 (0.0008) [2023-10-08 06:49:28,754][130385] Fps is (10 sec: 16384.6, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 152207360. Throughput: 0: 1826.4, 1: 1821.6. Samples: 38055364. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-08 06:49:28,754][130385] Avg episode reward: [(0, '62.890'), (1, '79.390')] [2023-10-08 06:49:30,973][00611] Updated weights for policy 0, policy_version 74122 (0.0011) [2023-10-08 06:49:31,353][00611] Updated weights for policy 0, policy_version 74132 (0.0010) [2023-10-08 06:49:31,525][00612] Updated weights for policy 1, policy_version 74530 (0.0009) [2023-10-08 06:49:31,724][00611] Updated weights for policy 0, policy_version 74142 (0.0007) [2023-10-08 06:49:31,888][00612] Updated weights for policy 1, policy_version 74540 (0.0007) [2023-10-08 06:49:32,261][00612] Updated weights for policy 1, policy_version 74550 (0.0008) [2023-10-08 06:49:32,621][00612] Updated weights for policy 1, policy_version 74560 (0.0010) [2023-10-08 06:49:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 152272896. Throughput: 0: 1831.5, 1: 1821.8. Samples: 38076698. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-08 06:49:33,755][130385] Avg episode reward: [(0, '64.810'), (1, '82.010')] [2023-10-08 06:49:35,053][00611] Updated weights for policy 0, policy_version 74152 (0.0007) [2023-10-08 06:49:35,421][00611] Updated weights for policy 0, policy_version 74162 (0.0007) [2023-10-08 06:49:35,789][00611] Updated weights for policy 0, policy_version 74172 (0.0007) [2023-10-08 06:49:36,332][00612] Updated weights for policy 1, policy_version 74570 (0.0007) [2023-10-08 06:49:36,695][00612] Updated weights for policy 1, policy_version 74580 (0.0009) [2023-10-08 06:49:37,073][00612] Updated weights for policy 1, policy_version 74590 (0.0009) [2023-10-08 06:49:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 152338432. Throughput: 0: 1828.5, 1: 1822.2. Samples: 38088152. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-08 06:49:38,754][130385] Avg episode reward: [(0, '67.110'), (1, '83.780')] [2023-10-08 06:49:39,502][00611] Updated weights for policy 0, policy_version 74182 (0.0007) [2023-10-08 06:49:39,887][00611] Updated weights for policy 0, policy_version 74192 (0.0007) [2023-10-08 06:49:40,250][00611] Updated weights for policy 0, policy_version 74202 (0.0010) [2023-10-08 06:49:40,828][00612] Updated weights for policy 1, policy_version 74600 (0.0010) [2023-10-08 06:49:41,187][00612] Updated weights for policy 1, policy_version 74610 (0.0008) [2023-10-08 06:49:41,560][00612] Updated weights for policy 1, policy_version 74620 (0.0010) [2023-10-08 06:49:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 152403968. Throughput: 0: 1845.6, 1: 1825.8. Samples: 38110022. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-08 06:49:43,755][130385] Avg episode reward: [(0, '64.100'), (1, '83.640')] [2023-10-08 06:49:43,827][00611] Updated weights for policy 0, policy_version 74212 (0.0011) [2023-10-08 06:49:44,194][00611] Updated weights for policy 0, policy_version 74222 (0.0008) [2023-10-08 06:49:44,574][00611] Updated weights for policy 0, policy_version 74232 (0.0007) [2023-10-08 06:49:45,155][00612] Updated weights for policy 1, policy_version 74630 (0.0009) [2023-10-08 06:49:45,530][00612] Updated weights for policy 1, policy_version 74640 (0.0009) [2023-10-08 06:49:45,896][00612] Updated weights for policy 1, policy_version 74650 (0.0010) [2023-10-08 06:49:48,352][00611] Updated weights for policy 0, policy_version 74242 (0.0008) [2023-10-08 06:49:48,729][00611] Updated weights for policy 0, policy_version 74252 (0.0007) [2023-10-08 06:49:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 152469504. Throughput: 0: 1853.4, 1: 1834.0. Samples: 38133770. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-08 06:49:48,755][130385] Avg episode reward: [(0, '65.020'), (1, '84.050')] [2023-10-08 06:49:49,093][00611] Updated weights for policy 0, policy_version 74262 (0.0007) [2023-10-08 06:49:49,316][00612] Updated weights for policy 1, policy_version 74660 (0.0010) [2023-10-08 06:49:49,467][00611] Updated weights for policy 0, policy_version 74272 (0.0009) [2023-10-08 06:49:49,686][00612] Updated weights for policy 1, policy_version 74670 (0.0007) [2023-10-08 06:49:50,059][00612] Updated weights for policy 1, policy_version 74680 (0.0008) [2023-10-08 06:49:53,118][00611] Updated weights for policy 0, policy_version 74282 (0.0007) [2023-10-08 06:49:53,494][00611] Updated weights for policy 0, policy_version 74292 (0.0008) [2023-10-08 06:49:53,709][00612] Updated weights for policy 1, policy_version 74690 (0.0010) [2023-10-08 06:49:53,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 152535040. Throughput: 0: 1859.0, 1: 1835.2. Samples: 38143892. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-08 06:49:53,754][130385] Avg episode reward: [(0, '63.540'), (1, '86.180')] [2023-10-08 06:49:53,857][00611] Updated weights for policy 0, policy_version 74302 (0.0008) [2023-10-08 06:49:54,073][00612] Updated weights for policy 1, policy_version 74700 (0.0007) [2023-10-08 06:49:54,448][00612] Updated weights for policy 1, policy_version 74710 (0.0008) [2023-10-08 06:49:54,826][00612] Updated weights for policy 1, policy_version 74720 (0.0008) [2023-10-08 06:49:57,439][00611] Updated weights for policy 0, policy_version 74312 (0.0007) [2023-10-08 06:49:57,809][00611] Updated weights for policy 0, policy_version 74322 (0.0008) [2023-10-08 06:49:58,176][00611] Updated weights for policy 0, policy_version 74332 (0.0010) [2023-10-08 06:49:58,514][00612] Updated weights for policy 1, policy_version 74730 (0.0007) [2023-10-08 06:49:58,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 152633344. Throughput: 0: 1852.5, 1: 1830.9. Samples: 38167156. Policy #0 lag: (min: 31.0, avg: 46.8, max: 63.0) [2023-10-08 06:49:58,754][130385] Avg episode reward: [(0, '60.880'), (1, '82.420')] [2023-10-08 06:49:58,886][00612] Updated weights for policy 1, policy_version 74740 (0.0009) [2023-10-08 06:49:59,247][00612] Updated weights for policy 1, policy_version 74750 (0.0008) [2023-10-08 06:50:01,861][00611] Updated weights for policy 0, policy_version 74342 (0.0010) [2023-10-08 06:50:02,236][00611] Updated weights for policy 0, policy_version 74352 (0.0007) [2023-10-08 06:50:02,615][00611] Updated weights for policy 0, policy_version 74362 (0.0007) [2023-10-08 06:50:02,821][00612] Updated weights for policy 1, policy_version 74760 (0.0008) [2023-10-08 06:50:03,192][00612] Updated weights for policy 1, policy_version 74770 (0.0008) [2023-10-08 06:50:03,565][00612] Updated weights for policy 1, policy_version 74780 (0.0009) [2023-10-08 06:50:03,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 152731648. Throughput: 0: 1847.7, 1: 1828.8. Samples: 38187914. Policy #0 lag: (min: 31.0, avg: 46.8, max: 63.0) [2023-10-08 06:50:03,754][130385] Avg episode reward: [(0, '64.640'), (1, '85.360')] [2023-10-08 06:50:06,298][00611] Updated weights for policy 0, policy_version 74372 (0.0009) [2023-10-08 06:50:06,663][00611] Updated weights for policy 0, policy_version 74382 (0.0008) [2023-10-08 06:50:07,034][00611] Updated weights for policy 0, policy_version 74392 (0.0008) [2023-10-08 06:50:07,205][00612] Updated weights for policy 1, policy_version 74790 (0.0007) [2023-10-08 06:50:07,571][00612] Updated weights for policy 1, policy_version 74800 (0.0007) [2023-10-08 06:50:07,937][00612] Updated weights for policy 1, policy_version 74810 (0.0008) [2023-10-08 06:50:08,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 152797184. Throughput: 0: 1854.0, 1: 1838.9. Samples: 38200204. Policy #0 lag: (min: 31.0, avg: 46.8, max: 63.0) [2023-10-08 06:50:08,755][130385] Avg episode reward: [(0, '69.830'), (1, '85.410')] [2023-10-08 06:50:10,689][00611] Updated weights for policy 0, policy_version 74402 (0.0008) [2023-10-08 06:50:11,054][00611] Updated weights for policy 0, policy_version 74412 (0.0010) [2023-10-08 06:50:11,420][00611] Updated weights for policy 0, policy_version 74422 (0.0009) [2023-10-08 06:50:11,588][00612] Updated weights for policy 1, policy_version 74820 (0.0008) [2023-10-08 06:50:11,791][00611] Updated weights for policy 0, policy_version 74432 (0.0008) [2023-10-08 06:50:11,984][00612] Updated weights for policy 1, policy_version 74830 (0.0009) [2023-10-08 06:50:12,359][00612] Updated weights for policy 1, policy_version 74840 (0.0008) [2023-10-08 06:50:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 152862720. Throughput: 0: 1838.1, 1: 1832.5. Samples: 38220542. Policy #0 lag: (min: 31.0, avg: 46.8, max: 63.0) [2023-10-08 06:50:13,754][130385] Avg episode reward: [(0, '68.890'), (1, '83.520')] [2023-10-08 06:50:15,394][00611] Updated weights for policy 0, policy_version 74442 (0.0009) [2023-10-08 06:50:15,767][00611] Updated weights for policy 0, policy_version 74452 (0.0008) [2023-10-08 06:50:15,817][00612] Updated weights for policy 1, policy_version 74850 (0.0007) [2023-10-08 06:50:16,132][00611] Updated weights for policy 0, policy_version 74462 (0.0007) [2023-10-08 06:50:16,190][00612] Updated weights for policy 1, policy_version 74860 (0.0007) [2023-10-08 06:50:16,547][00612] Updated weights for policy 1, policy_version 74870 (0.0008) [2023-10-08 06:50:16,915][00612] Updated weights for policy 1, policy_version 74880 (0.0009) [2023-10-08 06:50:18,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 152928256. Throughput: 0: 1847.7, 1: 1851.1. Samples: 38243142. Policy #0 lag: (min: 31.0, avg: 46.8, max: 63.0) [2023-10-08 06:50:18,754][130385] Avg episode reward: [(0, '70.790'), (1, '84.470')] [2023-10-08 06:50:18,764][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000074880_76677120.pth... [2023-10-08 06:50:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000074464_76251136.pth... [2023-10-08 06:50:18,796][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000072768_74514432.pth [2023-10-08 06:50:18,798][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000073152_74907648.pth [2023-10-08 06:50:19,804][00611] Updated weights for policy 0, policy_version 74472 (0.0007) [2023-10-08 06:50:20,178][00611] Updated weights for policy 0, policy_version 74482 (0.0009) [2023-10-08 06:50:20,550][00611] Updated weights for policy 0, policy_version 74492 (0.0009) [2023-10-08 06:50:20,682][00612] Updated weights for policy 1, policy_version 74890 (0.0010) [2023-10-08 06:50:21,056][00612] Updated weights for policy 1, policy_version 74900 (0.0009) [2023-10-08 06:50:21,418][00612] Updated weights for policy 1, policy_version 74910 (0.0008) [2023-10-08 06:50:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 152993792. Throughput: 0: 1840.5, 1: 1833.4. Samples: 38253480. Policy #0 lag: (min: 31.0, avg: 46.8, max: 63.0) [2023-10-08 06:50:23,754][130385] Avg episode reward: [(0, '71.620'), (1, '84.530')] [2023-10-08 06:50:24,275][00611] Updated weights for policy 0, policy_version 74502 (0.0009) [2023-10-08 06:50:24,632][00611] Updated weights for policy 0, policy_version 74512 (0.0011) [2023-10-08 06:50:24,992][00612] Updated weights for policy 1, policy_version 74920 (0.0007) [2023-10-08 06:50:25,000][00611] Updated weights for policy 0, policy_version 74522 (0.0008) [2023-10-08 06:50:25,356][00612] Updated weights for policy 1, policy_version 74930 (0.0009) [2023-10-08 06:50:25,724][00612] Updated weights for policy 1, policy_version 74940 (0.0008) [2023-10-08 06:50:28,647][00611] Updated weights for policy 0, policy_version 74532 (0.0009) [2023-10-08 06:50:28,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 153059328. Throughput: 0: 1838.1, 1: 1856.0. Samples: 38276258. Policy #0 lag: (min: 31.0, avg: 46.8, max: 63.0) [2023-10-08 06:50:28,755][130385] Avg episode reward: [(0, '68.440'), (1, '83.970')] [2023-10-08 06:50:29,028][00611] Updated weights for policy 0, policy_version 74542 (0.0009) [2023-10-08 06:50:29,348][00612] Updated weights for policy 1, policy_version 74950 (0.0007) [2023-10-08 06:50:29,391][00611] Updated weights for policy 0, policy_version 74552 (0.0008) [2023-10-08 06:50:29,716][00612] Updated weights for policy 1, policy_version 74960 (0.0008) [2023-10-08 06:50:30,093][00612] Updated weights for policy 1, policy_version 74970 (0.0010) [2023-10-08 06:50:33,040][00611] Updated weights for policy 0, policy_version 74562 (0.0007) [2023-10-08 06:50:33,408][00611] Updated weights for policy 0, policy_version 74572 (0.0007) [2023-10-08 06:50:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 153124864. Throughput: 0: 1830.3, 1: 1844.0. Samples: 38299114. Policy #0 lag: (min: 31.0, avg: 46.8, max: 63.0) [2023-10-08 06:50:33,754][130385] Avg episode reward: [(0, '68.730'), (1, '81.640')] [2023-10-08 06:50:33,777][00611] Updated weights for policy 0, policy_version 74582 (0.0009) [2023-10-08 06:50:33,793][00612] Updated weights for policy 1, policy_version 74980 (0.0009) [2023-10-08 06:50:34,144][00611] Updated weights for policy 0, policy_version 74592 (0.0009) [2023-10-08 06:50:34,154][00612] Updated weights for policy 1, policy_version 74990 (0.0009) [2023-10-08 06:50:34,525][00612] Updated weights for policy 1, policy_version 75000 (0.0010) [2023-10-08 06:50:37,647][00611] Updated weights for policy 0, policy_version 74602 (0.0007) [2023-10-08 06:50:38,022][00611] Updated weights for policy 0, policy_version 74612 (0.0010) [2023-10-08 06:50:38,174][00612] Updated weights for policy 1, policy_version 75010 (0.0008) [2023-10-08 06:50:38,402][00611] Updated weights for policy 0, policy_version 74622 (0.0008) [2023-10-08 06:50:38,539][00612] Updated weights for policy 1, policy_version 75020 (0.0007) [2023-10-08 06:50:38,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 153223168. Throughput: 0: 1833.2, 1: 1842.8. Samples: 38309312. Policy #0 lag: (min: 31.0, avg: 46.8, max: 63.0) [2023-10-08 06:50:38,755][130385] Avg episode reward: [(0, '71.590'), (1, '83.040')] [2023-10-08 06:50:38,910][00612] Updated weights for policy 1, policy_version 75030 (0.0007) [2023-10-08 06:50:39,268][00612] Updated weights for policy 1, policy_version 75040 (0.0008) [2023-10-08 06:50:42,110][00611] Updated weights for policy 0, policy_version 74632 (0.0009) [2023-10-08 06:50:42,473][00611] Updated weights for policy 0, policy_version 74642 (0.0007) [2023-10-08 06:50:42,845][00611] Updated weights for policy 0, policy_version 74652 (0.0007) [2023-10-08 06:50:42,995][00612] Updated weights for policy 1, policy_version 75050 (0.0007) [2023-10-08 06:50:43,353][00612] Updated weights for policy 1, policy_version 75060 (0.0008) [2023-10-08 06:50:43,722][00612] Updated weights for policy 1, policy_version 75070 (0.0007) [2023-10-08 06:50:43,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 153288704. Throughput: 0: 1820.4, 1: 1842.4. Samples: 38331980. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:50:43,754][130385] Avg episode reward: [(0, '70.690'), (1, '84.730')] [2023-10-08 06:50:46,536][00611] Updated weights for policy 0, policy_version 74662 (0.0009) [2023-10-08 06:50:46,900][00611] Updated weights for policy 0, policy_version 74672 (0.0010) [2023-10-08 06:50:47,227][00612] Updated weights for policy 1, policy_version 75080 (0.0008) [2023-10-08 06:50:47,268][00611] Updated weights for policy 0, policy_version 74682 (0.0009) [2023-10-08 06:50:47,604][00612] Updated weights for policy 1, policy_version 75090 (0.0008) [2023-10-08 06:50:47,961][00612] Updated weights for policy 1, policy_version 75100 (0.0011) [2023-10-08 06:50:48,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 153387008. Throughput: 0: 1833.7, 1: 1831.6. Samples: 38352854. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:50:48,754][130385] Avg episode reward: [(0, '72.260'), (1, '85.460')] [2023-10-08 06:50:50,932][00611] Updated weights for policy 0, policy_version 74692 (0.0008) [2023-10-08 06:50:51,320][00611] Updated weights for policy 0, policy_version 74702 (0.0009) [2023-10-08 06:50:51,680][00612] Updated weights for policy 1, policy_version 75110 (0.0008) [2023-10-08 06:50:51,686][00611] Updated weights for policy 0, policy_version 74712 (0.0007) [2023-10-08 06:50:52,046][00612] Updated weights for policy 1, policy_version 75120 (0.0007) [2023-10-08 06:50:52,416][00612] Updated weights for policy 1, policy_version 75130 (0.0011) [2023-10-08 06:50:53,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 153452544. Throughput: 0: 1825.2, 1: 1846.4. Samples: 38365428. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:50:53,755][130385] Avg episode reward: [(0, '72.700'), (1, '85.520')] [2023-10-08 06:50:55,246][00611] Updated weights for policy 0, policy_version 74722 (0.0009) [2023-10-08 06:50:55,615][00611] Updated weights for policy 0, policy_version 74732 (0.0010) [2023-10-08 06:50:55,990][00611] Updated weights for policy 0, policy_version 74742 (0.0009) [2023-10-08 06:50:56,130][00612] Updated weights for policy 1, policy_version 75140 (0.0008) [2023-10-08 06:50:56,353][00611] Updated weights for policy 0, policy_version 74752 (0.0009) [2023-10-08 06:50:56,502][00612] Updated weights for policy 1, policy_version 75150 (0.0009) [2023-10-08 06:50:56,867][00612] Updated weights for policy 1, policy_version 75160 (0.0009) [2023-10-08 06:50:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 153518080. Throughput: 0: 1842.0, 1: 1830.6. Samples: 38385806. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:50:58,754][130385] Avg episode reward: [(0, '69.750'), (1, '85.880')] [2023-10-08 06:50:59,977][00611] Updated weights for policy 0, policy_version 74762 (0.0007) [2023-10-08 06:51:00,357][00611] Updated weights for policy 0, policy_version 74772 (0.0008) [2023-10-08 06:51:00,702][00612] Updated weights for policy 1, policy_version 75170 (0.0010) [2023-10-08 06:51:00,725][00611] Updated weights for policy 0, policy_version 74782 (0.0008) [2023-10-08 06:51:01,087][00612] Updated weights for policy 1, policy_version 75180 (0.0009) [2023-10-08 06:51:01,458][00612] Updated weights for policy 1, policy_version 75190 (0.0008) [2023-10-08 06:51:01,825][00612] Updated weights for policy 1, policy_version 75200 (0.0007) [2023-10-08 06:51:03,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 153583616. Throughput: 0: 1840.8, 1: 1840.0. Samples: 38408778. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:51:03,754][130385] Avg episode reward: [(0, '68.450'), (1, '81.520')] [2023-10-08 06:51:04,263][00611] Updated weights for policy 0, policy_version 74792 (0.0009) [2023-10-08 06:51:04,626][00611] Updated weights for policy 0, policy_version 74802 (0.0009) [2023-10-08 06:51:04,998][00611] Updated weights for policy 0, policy_version 74812 (0.0009) [2023-10-08 06:51:05,400][00612] Updated weights for policy 1, policy_version 75210 (0.0009) [2023-10-08 06:51:05,757][00612] Updated weights for policy 1, policy_version 75220 (0.0009) [2023-10-08 06:51:06,140][00612] Updated weights for policy 1, policy_version 75230 (0.0010) [2023-10-08 06:51:08,454][00611] Updated weights for policy 0, policy_version 74822 (0.0007) [2023-10-08 06:51:08,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 153649152. Throughput: 0: 1842.9, 1: 1831.3. Samples: 38418820. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:51:08,755][130385] Avg episode reward: [(0, '64.360'), (1, '83.640')] [2023-10-08 06:51:08,816][00611] Updated weights for policy 0, policy_version 74832 (0.0008) [2023-10-08 06:51:09,198][00611] Updated weights for policy 0, policy_version 74842 (0.0009) [2023-10-08 06:51:09,684][00612] Updated weights for policy 1, policy_version 75240 (0.0008) [2023-10-08 06:51:10,052][00612] Updated weights for policy 1, policy_version 75250 (0.0007) [2023-10-08 06:51:10,415][00612] Updated weights for policy 1, policy_version 75260 (0.0008) [2023-10-08 06:51:12,953][00611] Updated weights for policy 0, policy_version 74852 (0.0008) [2023-10-08 06:51:13,325][00611] Updated weights for policy 0, policy_version 74862 (0.0011) [2023-10-08 06:51:13,690][00611] Updated weights for policy 0, policy_version 74872 (0.0010) [2023-10-08 06:51:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 153714688. Throughput: 0: 1845.5, 1: 1835.6. Samples: 38441906. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:51:13,754][130385] Avg episode reward: [(0, '64.320'), (1, '84.750')] [2023-10-08 06:51:13,922][00612] Updated weights for policy 1, policy_version 75270 (0.0011) [2023-10-08 06:51:14,295][00612] Updated weights for policy 1, policy_version 75280 (0.0010) [2023-10-08 06:51:14,664][00612] Updated weights for policy 1, policy_version 75290 (0.0010) [2023-10-08 06:51:17,376][00611] Updated weights for policy 0, policy_version 74882 (0.0009) [2023-10-08 06:51:17,737][00611] Updated weights for policy 0, policy_version 74892 (0.0007) [2023-10-08 06:51:18,104][00611] Updated weights for policy 0, policy_version 74902 (0.0010) [2023-10-08 06:51:18,243][00612] Updated weights for policy 1, policy_version 75300 (0.0009) [2023-10-08 06:51:18,466][00611] Updated weights for policy 0, policy_version 74912 (0.0008) [2023-10-08 06:51:18,615][00612] Updated weights for policy 1, policy_version 75310 (0.0007) [2023-10-08 06:51:18,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 153812992. Throughput: 0: 1824.0, 1: 1845.2. Samples: 38464228. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:51:18,754][130385] Avg episode reward: [(0, '62.130'), (1, '86.590')] [2023-10-08 06:51:18,981][00612] Updated weights for policy 1, policy_version 75320 (0.0007) [2023-10-08 06:51:22,236][00611] Updated weights for policy 0, policy_version 74922 (0.0010) [2023-10-08 06:51:22,508][00612] Updated weights for policy 1, policy_version 75330 (0.0007) [2023-10-08 06:51:22,597][00611] Updated weights for policy 0, policy_version 74932 (0.0009) [2023-10-08 06:51:22,877][00612] Updated weights for policy 1, policy_version 75340 (0.0007) [2023-10-08 06:51:22,964][00611] Updated weights for policy 0, policy_version 74942 (0.0007) [2023-10-08 06:51:23,243][00612] Updated weights for policy 1, policy_version 75350 (0.0007) [2023-10-08 06:51:23,622][00612] Updated weights for policy 1, policy_version 75360 (0.0008) [2023-10-08 06:51:23,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 153911296. Throughput: 0: 1839.5, 1: 1849.6. Samples: 38475318. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:51:23,754][130385] Avg episode reward: [(0, '63.780'), (1, '83.750')] [2023-10-08 06:51:26,624][00611] Updated weights for policy 0, policy_version 74952 (0.0008) [2023-10-08 06:51:27,000][00611] Updated weights for policy 0, policy_version 74962 (0.0008) [2023-10-08 06:51:27,253][00612] Updated weights for policy 1, policy_version 75370 (0.0007) [2023-10-08 06:51:27,367][00611] Updated weights for policy 0, policy_version 74972 (0.0008) [2023-10-08 06:51:27,618][00612] Updated weights for policy 1, policy_version 75380 (0.0008) [2023-10-08 06:51:27,985][00612] Updated weights for policy 1, policy_version 75390 (0.0008) [2023-10-08 06:51:28,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 153976832. Throughput: 0: 1823.4, 1: 1842.7. Samples: 38496954. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 06:51:28,754][130385] Avg episode reward: [(0, '66.850'), (1, '86.770')] [2023-10-08 06:51:31,135][00611] Updated weights for policy 0, policy_version 74982 (0.0009) [2023-10-08 06:51:31,504][00611] Updated weights for policy 0, policy_version 74992 (0.0010) [2023-10-08 06:51:31,845][00612] Updated weights for policy 1, policy_version 75400 (0.0009) [2023-10-08 06:51:31,878][00611] Updated weights for policy 0, policy_version 75002 (0.0007) [2023-10-08 06:51:32,216][00612] Updated weights for policy 1, policy_version 75410 (0.0008) [2023-10-08 06:51:32,590][00612] Updated weights for policy 1, policy_version 75420 (0.0008) [2023-10-08 06:51:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 154042368. Throughput: 0: 1831.1, 1: 1838.2. Samples: 38517972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:51:33,754][130385] Avg episode reward: [(0, '68.560'), (1, '85.670')] [2023-10-08 06:51:35,746][00611] Updated weights for policy 0, policy_version 75012 (0.0009) [2023-10-08 06:51:36,115][00611] Updated weights for policy 0, policy_version 75022 (0.0008) [2023-10-08 06:51:36,246][00612] Updated weights for policy 1, policy_version 75430 (0.0007) [2023-10-08 06:51:36,489][00611] Updated weights for policy 0, policy_version 75032 (0.0008) [2023-10-08 06:51:36,617][00612] Updated weights for policy 1, policy_version 75440 (0.0008) [2023-10-08 06:51:36,977][00612] Updated weights for policy 1, policy_version 75450 (0.0009) [2023-10-08 06:51:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 154107904. Throughput: 0: 1824.2, 1: 1832.8. Samples: 38529992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:51:38,755][130385] Avg episode reward: [(0, '66.620'), (1, '85.930')] [2023-10-08 06:51:40,233][00611] Updated weights for policy 0, policy_version 75042 (0.0008) [2023-10-08 06:51:40,588][00612] Updated weights for policy 1, policy_version 75460 (0.0008) [2023-10-08 06:51:40,648][00611] Updated weights for policy 0, policy_version 75052 (0.0008) [2023-10-08 06:51:40,955][00612] Updated weights for policy 1, policy_version 75470 (0.0009) [2023-10-08 06:51:41,007][00611] Updated weights for policy 0, policy_version 75062 (0.0009) [2023-10-08 06:51:41,328][00612] Updated weights for policy 1, policy_version 75480 (0.0007) [2023-10-08 06:51:41,379][00611] Updated weights for policy 0, policy_version 75072 (0.0008) [2023-10-08 06:51:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 154173440. Throughput: 0: 1822.5, 1: 1842.9. Samples: 38550750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:51:43,754][130385] Avg episode reward: [(0, '65.390'), (1, '83.960')] [2023-10-08 06:51:44,958][00611] Updated weights for policy 0, policy_version 75082 (0.0007) [2023-10-08 06:51:44,998][00612] Updated weights for policy 1, policy_version 75490 (0.0007) [2023-10-08 06:51:45,325][00611] Updated weights for policy 0, policy_version 75092 (0.0007) [2023-10-08 06:51:45,362][00612] Updated weights for policy 1, policy_version 75500 (0.0007) [2023-10-08 06:51:45,708][00611] Updated weights for policy 0, policy_version 75102 (0.0008) [2023-10-08 06:51:45,719][00612] Updated weights for policy 1, policy_version 75510 (0.0007) [2023-10-08 06:51:46,091][00612] Updated weights for policy 1, policy_version 75520 (0.0007) [2023-10-08 06:51:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 154238976. Throughput: 0: 1823.8, 1: 1845.4. Samples: 38573894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:51:48,755][130385] Avg episode reward: [(0, '64.200'), (1, '84.100')] [2023-10-08 06:51:49,376][00611] Updated weights for policy 0, policy_version 75112 (0.0009) [2023-10-08 06:51:49,745][00611] Updated weights for policy 0, policy_version 75122 (0.0008) [2023-10-08 06:51:49,795][00612] Updated weights for policy 1, policy_version 75530 (0.0007) [2023-10-08 06:51:50,117][00611] Updated weights for policy 0, policy_version 75132 (0.0009) [2023-10-08 06:51:50,155][00612] Updated weights for policy 1, policy_version 75540 (0.0009) [2023-10-08 06:51:50,521][00612] Updated weights for policy 1, policy_version 75550 (0.0010) [2023-10-08 06:51:53,651][00611] Updated weights for policy 0, policy_version 75142 (0.0009) [2023-10-08 06:51:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 154304512. Throughput: 0: 1824.9, 1: 1839.9. Samples: 38583734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:51:53,754][130385] Avg episode reward: [(0, '61.980'), (1, '85.300')] [2023-10-08 06:51:54,020][00611] Updated weights for policy 0, policy_version 75152 (0.0007) [2023-10-08 06:51:54,262][00612] Updated weights for policy 1, policy_version 75560 (0.0008) [2023-10-08 06:51:54,391][00611] Updated weights for policy 0, policy_version 75162 (0.0007) [2023-10-08 06:51:54,635][00612] Updated weights for policy 1, policy_version 75570 (0.0007) [2023-10-08 06:51:55,005][00612] Updated weights for policy 1, policy_version 75580 (0.0007) [2023-10-08 06:51:58,023][00611] Updated weights for policy 0, policy_version 75172 (0.0011) [2023-10-08 06:51:58,393][00611] Updated weights for policy 0, policy_version 75182 (0.0009) [2023-10-08 06:51:58,723][00612] Updated weights for policy 1, policy_version 75590 (0.0007) [2023-10-08 06:51:58,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 154370048. Throughput: 0: 1825.6, 1: 1836.0. Samples: 38606676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:51:58,754][130385] Avg episode reward: [(0, '65.580'), (1, '88.010')] [2023-10-08 06:51:58,767][00611] Updated weights for policy 0, policy_version 75192 (0.0008) [2023-10-08 06:51:59,089][00612] Updated weights for policy 1, policy_version 75600 (0.0008) [2023-10-08 06:51:59,462][00612] Updated weights for policy 1, policy_version 75610 (0.0010) [2023-10-08 06:52:02,425][00611] Updated weights for policy 0, policy_version 75202 (0.0010) [2023-10-08 06:52:02,803][00611] Updated weights for policy 0, policy_version 75212 (0.0008) [2023-10-08 06:52:03,177][00611] Updated weights for policy 0, policy_version 75222 (0.0008) [2023-10-08 06:52:03,258][00612] Updated weights for policy 1, policy_version 75620 (0.0009) [2023-10-08 06:52:03,549][00611] Updated weights for policy 0, policy_version 75232 (0.0009) [2023-10-08 06:52:03,630][00612] Updated weights for policy 1, policy_version 75630 (0.0009) [2023-10-08 06:52:03,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 154468352. Throughput: 0: 1825.8, 1: 1827.9. Samples: 38628646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:52:03,755][130385] Avg episode reward: [(0, '68.070'), (1, '83.570')] [2023-10-08 06:52:03,994][00612] Updated weights for policy 1, policy_version 75640 (0.0010) [2023-10-08 06:52:07,243][00611] Updated weights for policy 0, policy_version 75242 (0.0009) [2023-10-08 06:52:07,458][00612] Updated weights for policy 1, policy_version 75650 (0.0010) [2023-10-08 06:52:07,609][00611] Updated weights for policy 0, policy_version 75252 (0.0007) [2023-10-08 06:52:07,819][00612] Updated weights for policy 1, policy_version 75660 (0.0007) [2023-10-08 06:52:07,975][00611] Updated weights for policy 0, policy_version 75262 (0.0007) [2023-10-08 06:52:08,186][00612] Updated weights for policy 1, policy_version 75670 (0.0008) [2023-10-08 06:52:08,548][00612] Updated weights for policy 1, policy_version 75680 (0.0011) [2023-10-08 06:52:08,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 154566656. Throughput: 0: 1823.7, 1: 1829.5. Samples: 38639712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:52:08,754][130385] Avg episode reward: [(0, '68.020'), (1, '82.110')] [2023-10-08 06:52:11,629][00611] Updated weights for policy 0, policy_version 75272 (0.0008) [2023-10-08 06:52:11,993][00611] Updated weights for policy 0, policy_version 75282 (0.0007) [2023-10-08 06:52:12,195][00612] Updated weights for policy 1, policy_version 75690 (0.0007) [2023-10-08 06:52:12,356][00611] Updated weights for policy 0, policy_version 75292 (0.0007) [2023-10-08 06:52:12,557][00612] Updated weights for policy 1, policy_version 75700 (0.0007) [2023-10-08 06:52:12,931][00612] Updated weights for policy 1, policy_version 75710 (0.0008) [2023-10-08 06:52:13,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 154632192. Throughput: 0: 1824.2, 1: 1829.8. Samples: 38661384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:52:13,754][130385] Avg episode reward: [(0, '70.720'), (1, '85.690')] [2023-10-08 06:52:16,121][00611] Updated weights for policy 0, policy_version 75302 (0.0008) [2023-10-08 06:52:16,489][00611] Updated weights for policy 0, policy_version 75312 (0.0007) [2023-10-08 06:52:16,500][00612] Updated weights for policy 1, policy_version 75720 (0.0010) [2023-10-08 06:52:16,861][00612] Updated weights for policy 1, policy_version 75730 (0.0009) [2023-10-08 06:52:16,862][00611] Updated weights for policy 0, policy_version 75322 (0.0008) [2023-10-08 06:52:17,232][00612] Updated weights for policy 1, policy_version 75740 (0.0008) [2023-10-08 06:52:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 154697728. Throughput: 0: 1824.3, 1: 1842.3. Samples: 38682968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:52:18,754][130385] Avg episode reward: [(0, '70.170'), (1, '80.570')] [2023-10-08 06:52:18,767][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000075328_77135872.pth... [2023-10-08 06:52:18,767][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000075744_77561856.pth... [2023-10-08 06:52:18,807][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000073632_75399168.pth [2023-10-08 06:52:18,815][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000074016_75792384.pth [2023-10-08 06:52:20,450][00611] Updated weights for policy 0, policy_version 75332 (0.0007) [2023-10-08 06:52:20,817][00611] Updated weights for policy 0, policy_version 75342 (0.0008) [2023-10-08 06:52:20,832][00612] Updated weights for policy 1, policy_version 75750 (0.0008) [2023-10-08 06:52:21,192][00611] Updated weights for policy 0, policy_version 75352 (0.0009) [2023-10-08 06:52:21,195][00612] Updated weights for policy 1, policy_version 75760 (0.0008) [2023-10-08 06:52:21,551][00612] Updated weights for policy 1, policy_version 75770 (0.0008) [2023-10-08 06:52:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 154763264. Throughput: 0: 1816.5, 1: 1831.7. Samples: 38694162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:52:23,754][130385] Avg episode reward: [(0, '71.630'), (1, '81.280')] [2023-10-08 06:52:24,896][00611] Updated weights for policy 0, policy_version 75362 (0.0008) [2023-10-08 06:52:25,270][00611] Updated weights for policy 0, policy_version 75372 (0.0008) [2023-10-08 06:52:25,292][00612] Updated weights for policy 1, policy_version 75780 (0.0008) [2023-10-08 06:52:25,638][00611] Updated weights for policy 0, policy_version 75382 (0.0010) [2023-10-08 06:52:25,655][00612] Updated weights for policy 1, policy_version 75790 (0.0010) [2023-10-08 06:52:26,008][00611] Updated weights for policy 0, policy_version 75392 (0.0007) [2023-10-08 06:52:26,022][00612] Updated weights for policy 1, policy_version 75800 (0.0007) [2023-10-08 06:52:28,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 154828800. Throughput: 0: 1834.1, 1: 1838.0. Samples: 38715996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:52:28,755][130385] Avg episode reward: [(0, '73.200'), (1, '75.610')] [2023-10-08 06:52:29,638][00612] Updated weights for policy 1, policy_version 75810 (0.0008) [2023-10-08 06:52:29,767][00611] Updated weights for policy 0, policy_version 75402 (0.0009) [2023-10-08 06:52:30,001][00612] Updated weights for policy 1, policy_version 75820 (0.0008) [2023-10-08 06:52:30,135][00611] Updated weights for policy 0, policy_version 75412 (0.0010) [2023-10-08 06:52:30,364][00612] Updated weights for policy 1, policy_version 75830 (0.0008) [2023-10-08 06:52:30,508][00611] Updated weights for policy 0, policy_version 75422 (0.0008) [2023-10-08 06:52:30,729][00612] Updated weights for policy 1, policy_version 75840 (0.0008) [2023-10-08 06:52:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 154894336. Throughput: 0: 1821.1, 1: 1839.1. Samples: 38738602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:52:33,754][130385] Avg episode reward: [(0, '69.830'), (1, '78.700')] [2023-10-08 06:52:34,315][00611] Updated weights for policy 0, policy_version 75432 (0.0010) [2023-10-08 06:52:34,435][00612] Updated weights for policy 1, policy_version 75850 (0.0010) [2023-10-08 06:52:34,689][00611] Updated weights for policy 0, policy_version 75442 (0.0007) [2023-10-08 06:52:34,811][00612] Updated weights for policy 1, policy_version 75860 (0.0007) [2023-10-08 06:52:35,063][00611] Updated weights for policy 0, policy_version 75452 (0.0008) [2023-10-08 06:52:35,176][00612] Updated weights for policy 1, policy_version 75870 (0.0008) [2023-10-08 06:52:38,599][00611] Updated weights for policy 0, policy_version 75462 (0.0008) [2023-10-08 06:52:38,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 154959872. Throughput: 0: 1820.9, 1: 1845.6. Samples: 38748726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:52:38,754][130385] Avg episode reward: [(0, '67.860'), (1, '76.470')] [2023-10-08 06:52:38,773][00612] Updated weights for policy 1, policy_version 75880 (0.0007) [2023-10-08 06:52:38,965][00611] Updated weights for policy 0, policy_version 75472 (0.0009) [2023-10-08 06:52:39,139][00612] Updated weights for policy 1, policy_version 75890 (0.0009) [2023-10-08 06:52:39,334][00611] Updated weights for policy 0, policy_version 75482 (0.0008) [2023-10-08 06:52:39,510][00612] Updated weights for policy 1, policy_version 75900 (0.0007) [2023-10-08 06:52:43,086][00611] Updated weights for policy 0, policy_version 75492 (0.0008) [2023-10-08 06:52:43,181][00612] Updated weights for policy 1, policy_version 75910 (0.0008) [2023-10-08 06:52:43,451][00611] Updated weights for policy 0, policy_version 75502 (0.0009) [2023-10-08 06:52:43,556][00612] Updated weights for policy 1, policy_version 75920 (0.0007) [2023-10-08 06:52:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 155025408. Throughput: 0: 1819.6, 1: 1843.6. Samples: 38771522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:52:43,754][130385] Avg episode reward: [(0, '71.790'), (1, '76.430')] [2023-10-08 06:52:43,820][00611] Updated weights for policy 0, policy_version 75512 (0.0007) [2023-10-08 06:52:43,921][00612] Updated weights for policy 1, policy_version 75930 (0.0007) [2023-10-08 06:52:47,364][00611] Updated weights for policy 0, policy_version 75522 (0.0007) [2023-10-08 06:52:47,602][00612] Updated weights for policy 1, policy_version 75940 (0.0008) [2023-10-08 06:52:47,738][00611] Updated weights for policy 0, policy_version 75532 (0.0009) [2023-10-08 06:52:47,971][00612] Updated weights for policy 1, policy_version 75950 (0.0008) [2023-10-08 06:52:48,098][00611] Updated weights for policy 0, policy_version 75542 (0.0010) [2023-10-08 06:52:48,332][00612] Updated weights for policy 1, policy_version 75960 (0.0007) [2023-10-08 06:52:48,469][00611] Updated weights for policy 0, policy_version 75552 (0.0007) [2023-10-08 06:52:48,754][130385] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 155156480. Throughput: 0: 1819.7, 1: 1824.6. Samples: 38792640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:52:48,754][130385] Avg episode reward: [(0, '71.410'), (1, '75.500')] [2023-10-08 06:52:52,050][00612] Updated weights for policy 1, policy_version 75970 (0.0007) [2023-10-08 06:52:52,190][00611] Updated weights for policy 0, policy_version 75562 (0.0008) [2023-10-08 06:52:52,411][00612] Updated weights for policy 1, policy_version 75980 (0.0007) [2023-10-08 06:52:52,565][00611] Updated weights for policy 0, policy_version 75572 (0.0007) [2023-10-08 06:52:52,776][00612] Updated weights for policy 1, policy_version 75990 (0.0009) [2023-10-08 06:52:52,936][00611] Updated weights for policy 0, policy_version 75582 (0.0007) [2023-10-08 06:52:53,149][00612] Updated weights for policy 1, policy_version 76000 (0.0009) [2023-10-08 06:52:53,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 155222016. Throughput: 0: 1826.8, 1: 1837.3. Samples: 38804594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:52:53,754][130385] Avg episode reward: [(0, '73.030'), (1, '77.040')] [2023-10-08 06:52:56,457][00611] Updated weights for policy 0, policy_version 75592 (0.0010) [2023-10-08 06:52:56,823][00612] Updated weights for policy 1, policy_version 76010 (0.0007) [2023-10-08 06:52:56,837][00611] Updated weights for policy 0, policy_version 75602 (0.0009) [2023-10-08 06:52:57,184][00612] Updated weights for policy 1, policy_version 76020 (0.0008) [2023-10-08 06:52:57,203][00611] Updated weights for policy 0, policy_version 75612 (0.0009) [2023-10-08 06:52:57,563][00612] Updated weights for policy 1, policy_version 76030 (0.0009) [2023-10-08 06:52:58,754][130385] Fps is (10 sec: 13106.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 155287552. Throughput: 0: 1833.4, 1: 1824.6. Samples: 38825994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:52:58,755][130385] Avg episode reward: [(0, '78.210'), (1, '71.220')] [2023-10-08 06:53:00,799][00611] Updated weights for policy 0, policy_version 75622 (0.0010) [2023-10-08 06:53:01,166][00611] Updated weights for policy 0, policy_version 75632 (0.0009) [2023-10-08 06:53:01,234][00612] Updated weights for policy 1, policy_version 76040 (0.0007) [2023-10-08 06:53:01,537][00611] Updated weights for policy 0, policy_version 75642 (0.0008) [2023-10-08 06:53:01,608][00612] Updated weights for policy 1, policy_version 76050 (0.0007) [2023-10-08 06:53:01,965][00612] Updated weights for policy 1, policy_version 76060 (0.0010) [2023-10-08 06:53:03,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 155353088. Throughput: 0: 1843.7, 1: 1825.3. Samples: 38848074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:53:03,755][130385] Avg episode reward: [(0, '77.030'), (1, '74.030')] [2023-10-08 06:53:04,978][00611] Updated weights for policy 0, policy_version 75652 (0.0009) [2023-10-08 06:53:05,344][00611] Updated weights for policy 0, policy_version 75662 (0.0011) [2023-10-08 06:53:05,668][00612] Updated weights for policy 1, policy_version 76070 (0.0008) [2023-10-08 06:53:05,723][00611] Updated weights for policy 0, policy_version 75672 (0.0009) [2023-10-08 06:53:06,036][00612] Updated weights for policy 1, policy_version 76080 (0.0008) [2023-10-08 06:53:06,410][00612] Updated weights for policy 1, policy_version 76090 (0.0011) [2023-10-08 06:53:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 155418624. Throughput: 0: 1835.7, 1: 1823.7. Samples: 38858836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:53:08,754][130385] Avg episode reward: [(0, '73.880'), (1, '73.880')] [2023-10-08 06:53:09,322][00611] Updated weights for policy 0, policy_version 75682 (0.0008) [2023-10-08 06:53:09,699][00611] Updated weights for policy 0, policy_version 75692 (0.0010) [2023-10-08 06:53:10,028][00612] Updated weights for policy 1, policy_version 76100 (0.0009) [2023-10-08 06:53:10,070][00611] Updated weights for policy 0, policy_version 75702 (0.0008) [2023-10-08 06:53:10,395][00612] Updated weights for policy 1, policy_version 76110 (0.0007) [2023-10-08 06:53:10,448][00611] Updated weights for policy 0, policy_version 75712 (0.0008) [2023-10-08 06:53:10,760][00612] Updated weights for policy 1, policy_version 76120 (0.0007) [2023-10-08 06:53:13,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 155484160. Throughput: 0: 1846.9, 1: 1831.8. Samples: 38881536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:53:13,754][130385] Avg episode reward: [(0, '74.590'), (1, '72.290')] [2023-10-08 06:53:14,139][00611] Updated weights for policy 0, policy_version 75722 (0.0010) [2023-10-08 06:53:14,382][00612] Updated weights for policy 1, policy_version 76130 (0.0009) [2023-10-08 06:53:14,507][00611] Updated weights for policy 0, policy_version 75732 (0.0008) [2023-10-08 06:53:14,747][00612] Updated weights for policy 1, policy_version 76140 (0.0007) [2023-10-08 06:53:14,876][00611] Updated weights for policy 0, policy_version 75742 (0.0008) [2023-10-08 06:53:15,117][00612] Updated weights for policy 1, policy_version 76150 (0.0008) [2023-10-08 06:53:15,483][00612] Updated weights for policy 1, policy_version 76160 (0.0008) [2023-10-08 06:53:18,397][00611] Updated weights for policy 0, policy_version 75752 (0.0007) [2023-10-08 06:53:18,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 155549696. Throughput: 0: 1852.7, 1: 1834.7. Samples: 38904540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:53:18,755][130385] Avg episode reward: [(0, '76.270'), (1, '70.300')] [2023-10-08 06:53:18,766][00611] Updated weights for policy 0, policy_version 75762 (0.0009) [2023-10-08 06:53:19,122][00612] Updated weights for policy 1, policy_version 76170 (0.0008) [2023-10-08 06:53:19,135][00611] Updated weights for policy 0, policy_version 75772 (0.0008) [2023-10-08 06:53:19,492][00612] Updated weights for policy 1, policy_version 76180 (0.0008) [2023-10-08 06:53:19,870][00612] Updated weights for policy 1, policy_version 76190 (0.0010) [2023-10-08 06:53:22,945][00611] Updated weights for policy 0, policy_version 75782 (0.0008) [2023-10-08 06:53:23,313][00611] Updated weights for policy 0, policy_version 75792 (0.0008) [2023-10-08 06:53:23,564][00612] Updated weights for policy 1, policy_version 76200 (0.0007) [2023-10-08 06:53:23,687][00611] Updated weights for policy 0, policy_version 75802 (0.0008) [2023-10-08 06:53:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 155615232. Throughput: 0: 1851.9, 1: 1834.0. Samples: 38914590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:53:23,754][130385] Avg episode reward: [(0, '76.990'), (1, '74.000')] [2023-10-08 06:53:23,935][00612] Updated weights for policy 1, policy_version 76210 (0.0007) [2023-10-08 06:53:24,306][00612] Updated weights for policy 1, policy_version 76220 (0.0008) [2023-10-08 06:53:27,442][00611] Updated weights for policy 0, policy_version 75812 (0.0008) [2023-10-08 06:53:27,804][00611] Updated weights for policy 0, policy_version 75822 (0.0008) [2023-10-08 06:53:27,881][00612] Updated weights for policy 1, policy_version 76230 (0.0007) [2023-10-08 06:53:28,174][00611] Updated weights for policy 0, policy_version 75832 (0.0008) [2023-10-08 06:53:28,241][00612] Updated weights for policy 1, policy_version 76240 (0.0007) [2023-10-08 06:53:28,612][00612] Updated weights for policy 1, policy_version 76250 (0.0008) [2023-10-08 06:53:28,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 155713536. Throughput: 0: 1846.6, 1: 1840.4. Samples: 38937434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:53:28,754][130385] Avg episode reward: [(0, '72.650'), (1, '73.750')] [2023-10-08 06:53:31,628][00611] Updated weights for policy 0, policy_version 75842 (0.0008) [2023-10-08 06:53:31,996][00611] Updated weights for policy 0, policy_version 75852 (0.0007) [2023-10-08 06:53:32,197][00612] Updated weights for policy 1, policy_version 76260 (0.0009) [2023-10-08 06:53:32,369][00611] Updated weights for policy 0, policy_version 75862 (0.0007) [2023-10-08 06:53:32,566][00612] Updated weights for policy 1, policy_version 76270 (0.0008) [2023-10-08 06:53:32,741][00611] Updated weights for policy 0, policy_version 75872 (0.0007) [2023-10-08 06:53:32,931][00612] Updated weights for policy 1, policy_version 76280 (0.0009) [2023-10-08 06:53:33,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 155811840. Throughput: 0: 1836.4, 1: 1832.8. Samples: 38957750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:53:33,754][130385] Avg episode reward: [(0, '70.270'), (1, '74.750')] [2023-10-08 06:53:36,494][00611] Updated weights for policy 0, policy_version 75882 (0.0008) [2023-10-08 06:53:36,636][00612] Updated weights for policy 1, policy_version 76290 (0.0009) [2023-10-08 06:53:36,863][00611] Updated weights for policy 0, policy_version 75892 (0.0007) [2023-10-08 06:53:37,002][00612] Updated weights for policy 1, policy_version 76300 (0.0009) [2023-10-08 06:53:37,251][00611] Updated weights for policy 0, policy_version 75902 (0.0007) [2023-10-08 06:53:37,370][00612] Updated weights for policy 1, policy_version 76310 (0.0008) [2023-10-08 06:53:37,739][00612] Updated weights for policy 1, policy_version 76320 (0.0009) [2023-10-08 06:53:38,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 155877376. Throughput: 0: 1846.8, 1: 1845.5. Samples: 38970748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:53:38,754][130385] Avg episode reward: [(0, '74.280'), (1, '77.080')] [2023-10-08 06:53:40,820][00611] Updated weights for policy 0, policy_version 75912 (0.0009) [2023-10-08 06:53:41,196][00611] Updated weights for policy 0, policy_version 75922 (0.0007) [2023-10-08 06:53:41,345][00612] Updated weights for policy 1, policy_version 76330 (0.0007) [2023-10-08 06:53:41,566][00611] Updated weights for policy 0, policy_version 75932 (0.0008) [2023-10-08 06:53:41,713][00612] Updated weights for policy 1, policy_version 76340 (0.0009) [2023-10-08 06:53:42,080][00612] Updated weights for policy 1, policy_version 76350 (0.0010) [2023-10-08 06:53:43,754][130385] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 155942912. Throughput: 0: 1829.0, 1: 1831.2. Samples: 38990702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:53:43,755][130385] Avg episode reward: [(0, '73.220'), (1, '75.390')] [2023-10-08 06:53:45,205][00611] Updated weights for policy 0, policy_version 75942 (0.0009) [2023-10-08 06:53:45,566][00611] Updated weights for policy 0, policy_version 75952 (0.0011) [2023-10-08 06:53:45,779][00612] Updated weights for policy 1, policy_version 76360 (0.0009) [2023-10-08 06:53:45,954][00611] Updated weights for policy 0, policy_version 75962 (0.0007) [2023-10-08 06:53:46,148][00612] Updated weights for policy 1, policy_version 76370 (0.0008) [2023-10-08 06:53:46,518][00612] Updated weights for policy 1, policy_version 76380 (0.0007) [2023-10-08 06:53:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 156008448. Throughput: 0: 1834.5, 1: 1846.3. Samples: 39013708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:53:48,754][130385] Avg episode reward: [(0, '70.280'), (1, '76.830')] [2023-10-08 06:53:49,723][00611] Updated weights for policy 0, policy_version 75972 (0.0008) [2023-10-08 06:53:50,082][00611] Updated weights for policy 0, policy_version 75982 (0.0007) [2023-10-08 06:53:50,286][00612] Updated weights for policy 1, policy_version 76390 (0.0007) [2023-10-08 06:53:50,453][00611] Updated weights for policy 0, policy_version 75992 (0.0007) [2023-10-08 06:53:50,652][00612] Updated weights for policy 1, policy_version 76400 (0.0008) [2023-10-08 06:53:51,016][00612] Updated weights for policy 1, policy_version 76410 (0.0007) [2023-10-08 06:53:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 156073984. Throughput: 0: 1834.1, 1: 1830.0. Samples: 39023720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:53:53,755][130385] Avg episode reward: [(0, '72.140'), (1, '80.360')] [2023-10-08 06:53:54,078][00611] Updated weights for policy 0, policy_version 76002 (0.0008) [2023-10-08 06:53:54,451][00611] Updated weights for policy 0, policy_version 76012 (0.0009) [2023-10-08 06:53:54,708][00612] Updated weights for policy 1, policy_version 76420 (0.0008) [2023-10-08 06:53:54,815][00611] Updated weights for policy 0, policy_version 76022 (0.0007) [2023-10-08 06:53:55,064][00612] Updated weights for policy 1, policy_version 76430 (0.0007) [2023-10-08 06:53:55,186][00611] Updated weights for policy 0, policy_version 76032 (0.0007) [2023-10-08 06:53:55,428][00612] Updated weights for policy 1, policy_version 76440 (0.0010) [2023-10-08 06:53:58,729][00611] Updated weights for policy 0, policy_version 76042 (0.0007) [2023-10-08 06:53:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 156139520. Throughput: 0: 1829.5, 1: 1845.2. Samples: 39046894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:53:58,754][130385] Avg episode reward: [(0, '64.650'), (1, '79.170')] [2023-10-08 06:53:58,993][00612] Updated weights for policy 1, policy_version 76450 (0.0009) [2023-10-08 06:53:59,102][00611] Updated weights for policy 0, policy_version 76052 (0.0008) [2023-10-08 06:53:59,361][00612] Updated weights for policy 1, policy_version 76460 (0.0009) [2023-10-08 06:53:59,475][00611] Updated weights for policy 0, policy_version 76062 (0.0007) [2023-10-08 06:53:59,722][00612] Updated weights for policy 1, policy_version 76470 (0.0009) [2023-10-08 06:54:00,091][00612] Updated weights for policy 1, policy_version 76480 (0.0009) [2023-10-08 06:54:03,287][00611] Updated weights for policy 0, policy_version 76072 (0.0011) [2023-10-08 06:54:03,669][00611] Updated weights for policy 0, policy_version 76082 (0.0008) [2023-10-08 06:54:03,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 156205056. Throughput: 0: 1826.2, 1: 1840.4. Samples: 39069536. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-08 06:54:03,754][130385] Avg episode reward: [(0, '66.340'), (1, '79.100')] [2023-10-08 06:54:03,763][00612] Updated weights for policy 1, policy_version 76490 (0.0009) [2023-10-08 06:54:04,036][00611] Updated weights for policy 0, policy_version 76092 (0.0009) [2023-10-08 06:54:04,133][00612] Updated weights for policy 1, policy_version 76500 (0.0007) [2023-10-08 06:54:04,493][00612] Updated weights for policy 1, policy_version 76510 (0.0008) [2023-10-08 06:54:07,597][00611] Updated weights for policy 0, policy_version 76102 (0.0008) [2023-10-08 06:54:07,980][00611] Updated weights for policy 0, policy_version 76112 (0.0008) [2023-10-08 06:54:08,165][00612] Updated weights for policy 1, policy_version 76520 (0.0009) [2023-10-08 06:54:08,341][00611] Updated weights for policy 0, policy_version 76122 (0.0007) [2023-10-08 06:54:08,538][00612] Updated weights for policy 1, policy_version 76530 (0.0009) [2023-10-08 06:54:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 156303360. Throughput: 0: 1831.3, 1: 1834.3. Samples: 39079542. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-08 06:54:08,754][130385] Avg episode reward: [(0, '68.250'), (1, '82.710')] [2023-10-08 06:54:08,915][00612] Updated weights for policy 1, policy_version 76540 (0.0009) [2023-10-08 06:54:11,894][00611] Updated weights for policy 0, policy_version 76132 (0.0008) [2023-10-08 06:54:12,269][00611] Updated weights for policy 0, policy_version 76142 (0.0008) [2023-10-08 06:54:12,516][00612] Updated weights for policy 1, policy_version 76550 (0.0007) [2023-10-08 06:54:12,649][00611] Updated weights for policy 0, policy_version 76152 (0.0008) [2023-10-08 06:54:12,884][00612] Updated weights for policy 1, policy_version 76560 (0.0008) [2023-10-08 06:54:13,251][00612] Updated weights for policy 1, policy_version 76570 (0.0011) [2023-10-08 06:54:13,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 156401664. Throughput: 0: 1830.8, 1: 1836.7. Samples: 39102468. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-08 06:54:13,754][130385] Avg episode reward: [(0, '68.820'), (1, '81.440')] [2023-10-08 06:54:16,242][00611] Updated weights for policy 0, policy_version 76162 (0.0008) [2023-10-08 06:54:16,619][00611] Updated weights for policy 0, policy_version 76172 (0.0007) [2023-10-08 06:54:16,951][00612] Updated weights for policy 1, policy_version 76580 (0.0010) [2023-10-08 06:54:16,987][00611] Updated weights for policy 0, policy_version 76182 (0.0008) [2023-10-08 06:54:17,311][00612] Updated weights for policy 1, policy_version 76590 (0.0009) [2023-10-08 06:54:17,361][00611] Updated weights for policy 0, policy_version 76192 (0.0007) [2023-10-08 06:54:17,679][00612] Updated weights for policy 1, policy_version 76600 (0.0007) [2023-10-08 06:54:18,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 156467200. Throughput: 0: 1842.1, 1: 1825.9. Samples: 39122812. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-08 06:54:18,755][130385] Avg episode reward: [(0, '66.120'), (1, '81.200')] [2023-10-08 06:54:18,766][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000076608_78446592.pth... [2023-10-08 06:54:18,766][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000076192_78020608.pth... [2023-10-08 06:54:18,796][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000074464_76251136.pth [2023-10-08 06:54:18,804][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000074880_76677120.pth [2023-10-08 06:54:21,062][00611] Updated weights for policy 0, policy_version 76202 (0.0009) [2023-10-08 06:54:21,394][00612] Updated weights for policy 1, policy_version 76610 (0.0007) [2023-10-08 06:54:21,433][00611] Updated weights for policy 0, policy_version 76212 (0.0009) [2023-10-08 06:54:21,767][00612] Updated weights for policy 1, policy_version 76620 (0.0008) [2023-10-08 06:54:21,796][00611] Updated weights for policy 0, policy_version 76222 (0.0008) [2023-10-08 06:54:22,120][00612] Updated weights for policy 1, policy_version 76630 (0.0007) [2023-10-08 06:54:22,490][00612] Updated weights for policy 1, policy_version 76640 (0.0007) [2023-10-08 06:54:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 156532736. Throughput: 0: 1828.3, 1: 1829.2. Samples: 39135340. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-08 06:54:23,755][130385] Avg episode reward: [(0, '68.710'), (1, '76.570')] [2023-10-08 06:54:25,427][00611] Updated weights for policy 0, policy_version 76232 (0.0008) [2023-10-08 06:54:25,799][00611] Updated weights for policy 0, policy_version 76242 (0.0010) [2023-10-08 06:54:26,155][00612] Updated weights for policy 1, policy_version 76650 (0.0008) [2023-10-08 06:54:26,178][00611] Updated weights for policy 0, policy_version 76252 (0.0008) [2023-10-08 06:54:26,526][00612] Updated weights for policy 1, policy_version 76660 (0.0007) [2023-10-08 06:54:26,896][00612] Updated weights for policy 1, policy_version 76670 (0.0007) [2023-10-08 06:54:28,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 156598272. Throughput: 0: 1847.2, 1: 1825.2. Samples: 39155962. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-08 06:54:28,754][130385] Avg episode reward: [(0, '68.860'), (1, '79.280')] [2023-10-08 06:54:29,774][00611] Updated weights for policy 0, policy_version 76262 (0.0007) [2023-10-08 06:54:30,145][00611] Updated weights for policy 0, policy_version 76272 (0.0010) [2023-10-08 06:54:30,511][00612] Updated weights for policy 1, policy_version 76680 (0.0007) [2023-10-08 06:54:30,521][00611] Updated weights for policy 0, policy_version 76282 (0.0010) [2023-10-08 06:54:30,869][00612] Updated weights for policy 1, policy_version 76690 (0.0008) [2023-10-08 06:54:31,234][00612] Updated weights for policy 1, policy_version 76700 (0.0008) [2023-10-08 06:54:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 156663808. Throughput: 0: 1850.0, 1: 1830.3. Samples: 39179322. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-08 06:54:33,754][130385] Avg episode reward: [(0, '69.770'), (1, '77.250')] [2023-10-08 06:54:34,183][00611] Updated weights for policy 0, policy_version 76292 (0.0011) [2023-10-08 06:54:34,553][00611] Updated weights for policy 0, policy_version 76302 (0.0008) [2023-10-08 06:54:34,921][00611] Updated weights for policy 0, policy_version 76312 (0.0008) [2023-10-08 06:54:34,970][00612] Updated weights for policy 1, policy_version 76710 (0.0007) [2023-10-08 06:54:35,335][00612] Updated weights for policy 1, policy_version 76720 (0.0009) [2023-10-08 06:54:35,701][00612] Updated weights for policy 1, policy_version 76730 (0.0009) [2023-10-08 06:54:38,609][00611] Updated weights for policy 0, policy_version 76322 (0.0009) [2023-10-08 06:54:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 156729344. Throughput: 0: 1851.6, 1: 1831.1. Samples: 39189440. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-08 06:54:38,754][130385] Avg episode reward: [(0, '70.520'), (1, '76.560')] [2023-10-08 06:54:38,986][00611] Updated weights for policy 0, policy_version 76332 (0.0009) [2023-10-08 06:54:39,122][00612] Updated weights for policy 1, policy_version 76740 (0.0009) [2023-10-08 06:54:39,353][00611] Updated weights for policy 0, policy_version 76342 (0.0008) [2023-10-08 06:54:39,487][00612] Updated weights for policy 1, policy_version 76750 (0.0007) [2023-10-08 06:54:39,726][00611] Updated weights for policy 0, policy_version 76352 (0.0007) [2023-10-08 06:54:39,843][00612] Updated weights for policy 1, policy_version 76760 (0.0008) [2023-10-08 06:54:43,448][00612] Updated weights for policy 1, policy_version 76770 (0.0008) [2023-10-08 06:54:43,461][00611] Updated weights for policy 0, policy_version 76362 (0.0007) [2023-10-08 06:54:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 156794880. Throughput: 0: 1848.6, 1: 1836.8. Samples: 39212736. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-08 06:54:43,754][130385] Avg episode reward: [(0, '70.600'), (1, '77.300')] [2023-10-08 06:54:43,813][00612] Updated weights for policy 1, policy_version 76780 (0.0007) [2023-10-08 06:54:43,826][00611] Updated weights for policy 0, policy_version 76372 (0.0007) [2023-10-08 06:54:44,177][00612] Updated weights for policy 1, policy_version 76790 (0.0008) [2023-10-08 06:54:44,199][00611] Updated weights for policy 0, policy_version 76382 (0.0007) [2023-10-08 06:54:44,547][00612] Updated weights for policy 1, policy_version 76800 (0.0009) [2023-10-08 06:54:47,908][00611] Updated weights for policy 0, policy_version 76392 (0.0007) [2023-10-08 06:54:48,067][00612] Updated weights for policy 1, policy_version 76810 (0.0009) [2023-10-08 06:54:48,280][00611] Updated weights for policy 0, policy_version 76402 (0.0009) [2023-10-08 06:54:48,440][00612] Updated weights for policy 1, policy_version 76820 (0.0007) [2023-10-08 06:54:48,652][00611] Updated weights for policy 0, policy_version 76412 (0.0007) [2023-10-08 06:54:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 156860416. Throughput: 0: 1836.0, 1: 1831.6. Samples: 39234580. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-08 06:54:48,754][130385] Avg episode reward: [(0, '70.280'), (1, '72.770')] [2023-10-08 06:54:48,800][00612] Updated weights for policy 1, policy_version 76830 (0.0007) [2023-10-08 06:54:52,302][00611] Updated weights for policy 0, policy_version 76422 (0.0007) [2023-10-08 06:54:52,323][00612] Updated weights for policy 1, policy_version 76840 (0.0008) [2023-10-08 06:54:52,681][00612] Updated weights for policy 1, policy_version 76850 (0.0008) [2023-10-08 06:54:52,687][00611] Updated weights for policy 0, policy_version 76432 (0.0007) [2023-10-08 06:54:53,050][00611] Updated weights for policy 0, policy_version 76442 (0.0008) [2023-10-08 06:54:53,051][00612] Updated weights for policy 1, policy_version 76860 (0.0008) [2023-10-08 06:54:53,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 156991488. Throughput: 0: 1850.0, 1: 1851.1. Samples: 39246090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:54:53,754][130385] Avg episode reward: [(0, '70.960'), (1, '74.810')] [2023-10-08 06:54:56,598][00611] Updated weights for policy 0, policy_version 76452 (0.0008) [2023-10-08 06:54:56,741][00612] Updated weights for policy 1, policy_version 76870 (0.0009) [2023-10-08 06:54:56,963][00611] Updated weights for policy 0, policy_version 76462 (0.0008) [2023-10-08 06:54:57,112][00612] Updated weights for policy 1, policy_version 76880 (0.0007) [2023-10-08 06:54:57,332][00611] Updated weights for policy 0, policy_version 76472 (0.0008) [2023-10-08 06:54:57,469][00612] Updated weights for policy 1, policy_version 76890 (0.0008) [2023-10-08 06:54:58,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 157057024. Throughput: 0: 1831.6, 1: 1828.8. Samples: 39267186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:54:58,754][130385] Avg episode reward: [(0, '70.940'), (1, '76.600')] [2023-10-08 06:55:00,833][00611] Updated weights for policy 0, policy_version 76482 (0.0010) [2023-10-08 06:55:01,213][00611] Updated weights for policy 0, policy_version 76492 (0.0010) [2023-10-08 06:55:01,429][00612] Updated weights for policy 1, policy_version 76900 (0.0007) [2023-10-08 06:55:01,583][00611] Updated weights for policy 0, policy_version 76502 (0.0009) [2023-10-08 06:55:01,823][00612] Updated weights for policy 1, policy_version 76910 (0.0008) [2023-10-08 06:55:01,946][00611] Updated weights for policy 0, policy_version 76512 (0.0007) [2023-10-08 06:55:02,182][00612] Updated weights for policy 1, policy_version 76920 (0.0010) [2023-10-08 06:55:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 157122560. Throughput: 0: 1842.4, 1: 1849.7. Samples: 39288952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:55:03,754][130385] Avg episode reward: [(0, '73.990'), (1, '77.290')] [2023-10-08 06:55:05,548][00611] Updated weights for policy 0, policy_version 76522 (0.0009) [2023-10-08 06:55:05,900][00612] Updated weights for policy 1, policy_version 76930 (0.0010) [2023-10-08 06:55:05,912][00611] Updated weights for policy 0, policy_version 76532 (0.0009) [2023-10-08 06:55:06,271][00612] Updated weights for policy 1, policy_version 76940 (0.0008) [2023-10-08 06:55:06,277][00611] Updated weights for policy 0, policy_version 76542 (0.0009) [2023-10-08 06:55:06,629][00612] Updated weights for policy 1, policy_version 76950 (0.0008) [2023-10-08 06:55:06,999][00612] Updated weights for policy 1, policy_version 76960 (0.0010) [2023-10-08 06:55:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 157188096. Throughput: 0: 1822.5, 1: 1836.6. Samples: 39300002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:55:08,754][130385] Avg episode reward: [(0, '74.080'), (1, '78.010')] [2023-10-08 06:55:09,986][00611] Updated weights for policy 0, policy_version 76552 (0.0011) [2023-10-08 06:55:10,362][00611] Updated weights for policy 0, policy_version 76562 (0.0011) [2023-10-08 06:55:10,731][00611] Updated weights for policy 0, policy_version 76572 (0.0008) [2023-10-08 06:55:10,793][00612] Updated weights for policy 1, policy_version 76970 (0.0008) [2023-10-08 06:55:11,159][00612] Updated weights for policy 1, policy_version 76980 (0.0007) [2023-10-08 06:55:11,530][00612] Updated weights for policy 1, policy_version 76990 (0.0010) [2023-10-08 06:55:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 157253632. Throughput: 0: 1832.2, 1: 1851.0. Samples: 39321706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:55:13,754][130385] Avg episode reward: [(0, '76.520'), (1, '78.780')] [2023-10-08 06:55:14,478][00611] Updated weights for policy 0, policy_version 76582 (0.0010) [2023-10-08 06:55:14,844][00611] Updated weights for policy 0, policy_version 76592 (0.0008) [2023-10-08 06:55:15,047][00612] Updated weights for policy 1, policy_version 77000 (0.0008) [2023-10-08 06:55:15,217][00611] Updated weights for policy 0, policy_version 76602 (0.0007) [2023-10-08 06:55:15,412][00612] Updated weights for policy 1, policy_version 77010 (0.0009) [2023-10-08 06:55:15,784][00612] Updated weights for policy 1, policy_version 77020 (0.0010) [2023-10-08 06:55:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 157319168. Throughput: 0: 1824.3, 1: 1852.4. Samples: 39344772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:55:18,755][130385] Avg episode reward: [(0, '75.250'), (1, '77.610')] [2023-10-08 06:55:18,893][00611] Updated weights for policy 0, policy_version 76612 (0.0009) [2023-10-08 06:55:19,266][00611] Updated weights for policy 0, policy_version 76622 (0.0008) [2023-10-08 06:55:19,383][00612] Updated weights for policy 1, policy_version 77030 (0.0008) [2023-10-08 06:55:19,638][00611] Updated weights for policy 0, policy_version 76632 (0.0008) [2023-10-08 06:55:19,745][00612] Updated weights for policy 1, policy_version 77040 (0.0009) [2023-10-08 06:55:20,108][00612] Updated weights for policy 1, policy_version 77050 (0.0008) [2023-10-08 06:55:23,408][00611] Updated weights for policy 0, policy_version 76642 (0.0008) [2023-10-08 06:55:23,689][00612] Updated weights for policy 1, policy_version 77060 (0.0008) [2023-10-08 06:55:23,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 157384704. Throughput: 0: 1822.3, 1: 1854.1. Samples: 39354876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:55:23,755][130385] Avg episode reward: [(0, '74.040'), (1, '78.140')] [2023-10-08 06:55:23,782][00611] Updated weights for policy 0, policy_version 76652 (0.0007) [2023-10-08 06:55:24,048][00612] Updated weights for policy 1, policy_version 77070 (0.0008) [2023-10-08 06:55:24,164][00611] Updated weights for policy 0, policy_version 76662 (0.0008) [2023-10-08 06:55:24,420][00612] Updated weights for policy 1, policy_version 77080 (0.0009) [2023-10-08 06:55:24,529][00611] Updated weights for policy 0, policy_version 76672 (0.0007) [2023-10-08 06:55:28,044][00612] Updated weights for policy 1, policy_version 77090 (0.0008) [2023-10-08 06:55:28,281][00611] Updated weights for policy 0, policy_version 76682 (0.0009) [2023-10-08 06:55:28,412][00612] Updated weights for policy 1, policy_version 77100 (0.0007) [2023-10-08 06:55:28,651][00611] Updated weights for policy 0, policy_version 76692 (0.0010) [2023-10-08 06:55:28,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 157450240. Throughput: 0: 1826.6, 1: 1845.7. Samples: 39377990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:55:28,755][130385] Avg episode reward: [(0, '73.780'), (1, '80.340')] [2023-10-08 06:55:28,781][00612] Updated weights for policy 1, policy_version 77110 (0.0008) [2023-10-08 06:55:29,021][00611] Updated weights for policy 0, policy_version 76702 (0.0009) [2023-10-08 06:55:29,147][00612] Updated weights for policy 1, policy_version 77120 (0.0011) [2023-10-08 06:55:32,724][00611] Updated weights for policy 0, policy_version 76712 (0.0009) [2023-10-08 06:55:32,891][00612] Updated weights for policy 1, policy_version 77130 (0.0008) [2023-10-08 06:55:33,097][00611] Updated weights for policy 0, policy_version 76722 (0.0007) [2023-10-08 06:55:33,258][00612] Updated weights for policy 1, policy_version 77140 (0.0007) [2023-10-08 06:55:33,461][00611] Updated weights for policy 0, policy_version 76732 (0.0008) [2023-10-08 06:55:33,627][00612] Updated weights for policy 1, policy_version 77150 (0.0008) [2023-10-08 06:55:33,754][130385] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 157581312. Throughput: 0: 1823.8, 1: 1835.6. Samples: 39399252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:55:33,755][130385] Avg episode reward: [(0, '74.820'), (1, '80.470')] [2023-10-08 06:55:37,175][00611] Updated weights for policy 0, policy_version 76742 (0.0007) [2023-10-08 06:55:37,264][00612] Updated weights for policy 1, policy_version 77160 (0.0009) [2023-10-08 06:55:37,570][00611] Updated weights for policy 0, policy_version 76752 (0.0008) [2023-10-08 06:55:37,638][00612] Updated weights for policy 1, policy_version 77170 (0.0007) [2023-10-08 06:55:37,936][00611] Updated weights for policy 0, policy_version 76762 (0.0007) [2023-10-08 06:55:37,999][00612] Updated weights for policy 1, policy_version 77180 (0.0010) [2023-10-08 06:55:38,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 157646848. Throughput: 0: 1826.9, 1: 1835.6. Samples: 39410904. Policy #0 lag: (min: 9.0, avg: 11.4, max: 35.0) [2023-10-08 06:55:38,755][130385] Avg episode reward: [(0, '79.320'), (1, '82.000')] [2023-10-08 06:55:41,400][00611] Updated weights for policy 0, policy_version 76772 (0.0007) [2023-10-08 06:55:41,743][00612] Updated weights for policy 1, policy_version 77190 (0.0007) [2023-10-08 06:55:41,772][00611] Updated weights for policy 0, policy_version 76782 (0.0008) [2023-10-08 06:55:42,107][00612] Updated weights for policy 1, policy_version 77200 (0.0008) [2023-10-08 06:55:42,136][00611] Updated weights for policy 0, policy_version 76792 (0.0009) [2023-10-08 06:55:42,470][00612] Updated weights for policy 1, policy_version 77210 (0.0008) [2023-10-08 06:55:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 157712384. Throughput: 0: 1826.8, 1: 1836.3. Samples: 39432024. Policy #0 lag: (min: 9.0, avg: 11.4, max: 35.0) [2023-10-08 06:55:43,754][130385] Avg episode reward: [(0, '78.200'), (1, '78.760')] [2023-10-08 06:55:45,924][00611] Updated weights for policy 0, policy_version 76802 (0.0009) [2023-10-08 06:55:46,161][00612] Updated weights for policy 1, policy_version 77220 (0.0008) [2023-10-08 06:55:46,304][00611] Updated weights for policy 0, policy_version 76812 (0.0008) [2023-10-08 06:55:46,552][00612] Updated weights for policy 1, policy_version 77230 (0.0008) [2023-10-08 06:55:46,669][00611] Updated weights for policy 0, policy_version 76822 (0.0009) [2023-10-08 06:55:46,915][00612] Updated weights for policy 1, policy_version 77240 (0.0008) [2023-10-08 06:55:47,045][00611] Updated weights for policy 0, policy_version 76832 (0.0007) [2023-10-08 06:55:48,754][130385] Fps is (10 sec: 13107.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 157777920. Throughput: 0: 1822.4, 1: 1835.5. Samples: 39453556. Policy #0 lag: (min: 9.0, avg: 11.4, max: 35.0) [2023-10-08 06:55:48,754][130385] Avg episode reward: [(0, '75.720'), (1, '77.500')] [2023-10-08 06:55:50,534][00611] Updated weights for policy 0, policy_version 76842 (0.0007) [2023-10-08 06:55:50,661][00612] Updated weights for policy 1, policy_version 77250 (0.0008) [2023-10-08 06:55:50,906][00611] Updated weights for policy 0, policy_version 76852 (0.0008) [2023-10-08 06:55:51,034][00612] Updated weights for policy 1, policy_version 77260 (0.0009) [2023-10-08 06:55:51,279][00611] Updated weights for policy 0, policy_version 76862 (0.0007) [2023-10-08 06:55:51,393][00612] Updated weights for policy 1, policy_version 77270 (0.0008) [2023-10-08 06:55:51,753][00612] Updated weights for policy 1, policy_version 77280 (0.0008) [2023-10-08 06:55:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 157843456. Throughput: 0: 1830.9, 1: 1829.5. Samples: 39464718. Policy #0 lag: (min: 9.0, avg: 11.4, max: 35.0) [2023-10-08 06:55:53,754][130385] Avg episode reward: [(0, '77.040'), (1, '82.240')] [2023-10-08 06:55:54,810][00611] Updated weights for policy 0, policy_version 76872 (0.0008) [2023-10-08 06:55:55,179][00611] Updated weights for policy 0, policy_version 76882 (0.0007) [2023-10-08 06:55:55,263][00612] Updated weights for policy 1, policy_version 77290 (0.0007) [2023-10-08 06:55:55,546][00611] Updated weights for policy 0, policy_version 76892 (0.0009) [2023-10-08 06:55:55,629][00612] Updated weights for policy 1, policy_version 77300 (0.0007) [2023-10-08 06:55:55,992][00612] Updated weights for policy 1, policy_version 77310 (0.0007) [2023-10-08 06:55:58,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 157908992. Throughput: 0: 1834.5, 1: 1838.3. Samples: 39486984. Policy #0 lag: (min: 9.0, avg: 11.4, max: 35.0) [2023-10-08 06:55:58,755][130385] Avg episode reward: [(0, '77.240'), (1, '84.780')] [2023-10-08 06:55:59,360][00611] Updated weights for policy 0, policy_version 76902 (0.0009) [2023-10-08 06:55:59,589][00612] Updated weights for policy 1, policy_version 77320 (0.0007) [2023-10-08 06:55:59,723][00611] Updated weights for policy 0, policy_version 76912 (0.0007) [2023-10-08 06:55:59,956][00612] Updated weights for policy 1, policy_version 77330 (0.0010) [2023-10-08 06:56:00,101][00611] Updated weights for policy 0, policy_version 76922 (0.0008) [2023-10-08 06:56:00,321][00612] Updated weights for policy 1, policy_version 77340 (0.0009) [2023-10-08 06:56:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 157974528. Throughput: 0: 1831.7, 1: 1832.0. Samples: 39509642. Policy #0 lag: (min: 9.0, avg: 11.4, max: 35.0) [2023-10-08 06:56:03,754][130385] Avg episode reward: [(0, '76.680'), (1, '84.660')] [2023-10-08 06:56:03,949][00611] Updated weights for policy 0, policy_version 76932 (0.0008) [2023-10-08 06:56:04,016][00612] Updated weights for policy 1, policy_version 77350 (0.0009) [2023-10-08 06:56:04,323][00611] Updated weights for policy 0, policy_version 76942 (0.0007) [2023-10-08 06:56:04,381][00612] Updated weights for policy 1, policy_version 77360 (0.0007) [2023-10-08 06:56:04,695][00611] Updated weights for policy 0, policy_version 76952 (0.0007) [2023-10-08 06:56:04,743][00612] Updated weights for policy 1, policy_version 77370 (0.0007) [2023-10-08 06:56:08,174][00611] Updated weights for policy 0, policy_version 76962 (0.0008) [2023-10-08 06:56:08,323][00612] Updated weights for policy 1, policy_version 77380 (0.0008) [2023-10-08 06:56:08,554][00611] Updated weights for policy 0, policy_version 76972 (0.0010) [2023-10-08 06:56:08,690][00612] Updated weights for policy 1, policy_version 77390 (0.0007) [2023-10-08 06:56:08,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 158040064. Throughput: 0: 1832.7, 1: 1831.7. Samples: 39519774. Policy #0 lag: (min: 9.0, avg: 11.4, max: 35.0) [2023-10-08 06:56:08,755][130385] Avg episode reward: [(0, '74.100'), (1, '81.300')] [2023-10-08 06:56:08,922][00611] Updated weights for policy 0, policy_version 76982 (0.0008) [2023-10-08 06:56:09,057][00612] Updated weights for policy 1, policy_version 77400 (0.0009) [2023-10-08 06:56:09,289][00611] Updated weights for policy 0, policy_version 76992 (0.0008) [2023-10-08 06:56:12,866][00612] Updated weights for policy 1, policy_version 77410 (0.0008) [2023-10-08 06:56:12,943][00611] Updated weights for policy 0, policy_version 77002 (0.0008) [2023-10-08 06:56:13,236][00612] Updated weights for policy 1, policy_version 77420 (0.0008) [2023-10-08 06:56:13,318][00611] Updated weights for policy 0, policy_version 77012 (0.0010) [2023-10-08 06:56:13,607][00612] Updated weights for policy 1, policy_version 77430 (0.0009) [2023-10-08 06:56:13,690][00611] Updated weights for policy 0, policy_version 77022 (0.0009) [2023-10-08 06:56:13,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 158105600. Throughput: 0: 1836.4, 1: 1829.2. Samples: 39542942. Policy #0 lag: (min: 9.0, avg: 11.4, max: 35.0) [2023-10-08 06:56:13,755][130385] Avg episode reward: [(0, '75.930'), (1, '84.820')] [2023-10-08 06:56:13,966][00612] Updated weights for policy 1, policy_version 77440 (0.0007) [2023-10-08 06:56:17,382][00611] Updated weights for policy 0, policy_version 77032 (0.0008) [2023-10-08 06:56:17,524][00612] Updated weights for policy 1, policy_version 77450 (0.0007) [2023-10-08 06:56:17,753][00611] Updated weights for policy 0, policy_version 77042 (0.0008) [2023-10-08 06:56:17,884][00612] Updated weights for policy 1, policy_version 77460 (0.0007) [2023-10-08 06:56:18,121][00611] Updated weights for policy 0, policy_version 77052 (0.0008) [2023-10-08 06:56:18,263][00612] Updated weights for policy 1, policy_version 77470 (0.0007) [2023-10-08 06:56:18,754][130385] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 158236672. Throughput: 0: 1824.8, 1: 1823.9. Samples: 39563444. Policy #0 lag: (min: 9.0, avg: 11.4, max: 35.0) [2023-10-08 06:56:18,755][130385] Avg episode reward: [(0, '78.090'), (1, '78.170')] [2023-10-08 06:56:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000077056_78905344.pth... [2023-10-08 06:56:18,765][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000077472_79331328.pth... [2023-10-08 06:56:18,803][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000075744_77561856.pth [2023-10-08 06:56:18,805][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000075328_77135872.pth [2023-10-08 06:56:18,808][00425] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p1/milestones/checkpoint_000077472_79331328.pth [2023-10-08 06:56:18,811][00365] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p0/milestones/checkpoint_000077056_78905344.pth [2023-10-08 06:56:21,801][00611] Updated weights for policy 0, policy_version 77062 (0.0010) [2023-10-08 06:56:21,925][00612] Updated weights for policy 1, policy_version 77480 (0.0008) [2023-10-08 06:56:22,172][00611] Updated weights for policy 0, policy_version 77072 (0.0009) [2023-10-08 06:56:22,289][00612] Updated weights for policy 1, policy_version 77490 (0.0007) [2023-10-08 06:56:22,539][00611] Updated weights for policy 0, policy_version 77082 (0.0008) [2023-10-08 06:56:22,659][00612] Updated weights for policy 1, policy_version 77500 (0.0007) [2023-10-08 06:56:23,754][130385] Fps is (10 sec: 19661.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 158302208. Throughput: 0: 1830.5, 1: 1837.1. Samples: 39575942. Policy #0 lag: (min: 9.0, avg: 11.4, max: 35.0) [2023-10-08 06:56:23,754][130385] Avg episode reward: [(0, '76.140'), (1, '76.760')] [2023-10-08 06:56:26,194][00611] Updated weights for policy 0, policy_version 77092 (0.0008) [2023-10-08 06:56:26,247][00612] Updated weights for policy 1, policy_version 77510 (0.0009) [2023-10-08 06:56:26,573][00611] Updated weights for policy 0, policy_version 77102 (0.0007) [2023-10-08 06:56:26,616][00612] Updated weights for policy 1, policy_version 77520 (0.0007) [2023-10-08 06:56:26,939][00611] Updated weights for policy 0, policy_version 77112 (0.0009) [2023-10-08 06:56:26,980][00612] Updated weights for policy 1, policy_version 77530 (0.0007) [2023-10-08 06:56:28,754][130385] Fps is (10 sec: 13107.7, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 158367744. Throughput: 0: 1824.7, 1: 1828.6. Samples: 39596422. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:56:28,754][130385] Avg episode reward: [(0, '76.140'), (1, '74.460')] [2023-10-08 06:56:30,657][00611] Updated weights for policy 0, policy_version 77122 (0.0009) [2023-10-08 06:56:30,772][00612] Updated weights for policy 1, policy_version 77540 (0.0007) [2023-10-08 06:56:31,037][00611] Updated weights for policy 0, policy_version 77132 (0.0007) [2023-10-08 06:56:31,169][00612] Updated weights for policy 1, policy_version 77550 (0.0008) [2023-10-08 06:56:31,402][00611] Updated weights for policy 0, policy_version 77142 (0.0008) [2023-10-08 06:56:31,539][00612] Updated weights for policy 1, policy_version 77560 (0.0007) [2023-10-08 06:56:31,772][00611] Updated weights for policy 0, policy_version 77152 (0.0007) [2023-10-08 06:56:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 158433280. Throughput: 0: 1832.3, 1: 1839.4. Samples: 39618780. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:56:33,754][130385] Avg episode reward: [(0, '75.390'), (1, '72.850')] [2023-10-08 06:56:35,215][00612] Updated weights for policy 1, policy_version 77570 (0.0008) [2023-10-08 06:56:35,217][00611] Updated weights for policy 0, policy_version 77162 (0.0009) [2023-10-08 06:56:35,576][00612] Updated weights for policy 1, policy_version 77580 (0.0009) [2023-10-08 06:56:35,578][00611] Updated weights for policy 0, policy_version 77172 (0.0009) [2023-10-08 06:56:35,945][00612] Updated weights for policy 1, policy_version 77590 (0.0007) [2023-10-08 06:56:35,953][00611] Updated weights for policy 0, policy_version 77182 (0.0008) [2023-10-08 06:56:36,310][00612] Updated weights for policy 1, policy_version 77600 (0.0009) [2023-10-08 06:56:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 158498816. Throughput: 0: 1823.1, 1: 1828.9. Samples: 39629062. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:56:38,755][130385] Avg episode reward: [(0, '71.170'), (1, '75.590')] [2023-10-08 06:56:39,666][00611] Updated weights for policy 0, policy_version 77192 (0.0007) [2023-10-08 06:56:39,839][00612] Updated weights for policy 1, policy_version 77610 (0.0008) [2023-10-08 06:56:40,029][00611] Updated weights for policy 0, policy_version 77202 (0.0008) [2023-10-08 06:56:40,203][00612] Updated weights for policy 1, policy_version 77620 (0.0008) [2023-10-08 06:56:40,394][00611] Updated weights for policy 0, policy_version 77212 (0.0008) [2023-10-08 06:56:40,572][00612] Updated weights for policy 1, policy_version 77630 (0.0009) [2023-10-08 06:56:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 158564352. Throughput: 0: 1827.4, 1: 1835.3. Samples: 39651804. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:56:43,754][130385] Avg episode reward: [(0, '71.210'), (1, '74.280')] [2023-10-08 06:56:44,165][00611] Updated weights for policy 0, policy_version 77222 (0.0008) [2023-10-08 06:56:44,214][00612] Updated weights for policy 1, policy_version 77640 (0.0010) [2023-10-08 06:56:44,533][00611] Updated weights for policy 0, policy_version 77232 (0.0008) [2023-10-08 06:56:44,571][00612] Updated weights for policy 1, policy_version 77650 (0.0008) [2023-10-08 06:56:44,909][00611] Updated weights for policy 0, policy_version 77242 (0.0009) [2023-10-08 06:56:44,943][00612] Updated weights for policy 1, policy_version 77660 (0.0008) [2023-10-08 06:56:48,618][00612] Updated weights for policy 1, policy_version 77670 (0.0008) [2023-10-08 06:56:48,672][00611] Updated weights for policy 0, policy_version 77252 (0.0009) [2023-10-08 06:56:48,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 158629888. Throughput: 0: 1826.4, 1: 1841.7. Samples: 39674706. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:56:48,754][130385] Avg episode reward: [(0, '69.500'), (1, '75.280')] [2023-10-08 06:56:48,987][00612] Updated weights for policy 1, policy_version 77680 (0.0008) [2023-10-08 06:56:49,040][00611] Updated weights for policy 0, policy_version 77262 (0.0007) [2023-10-08 06:56:49,358][00612] Updated weights for policy 1, policy_version 77690 (0.0008) [2023-10-08 06:56:49,404][00611] Updated weights for policy 0, policy_version 77272 (0.0007) [2023-10-08 06:56:53,003][00612] Updated weights for policy 1, policy_version 77700 (0.0007) [2023-10-08 06:56:53,172][00611] Updated weights for policy 0, policy_version 77282 (0.0008) [2023-10-08 06:56:53,374][00612] Updated weights for policy 1, policy_version 77710 (0.0007) [2023-10-08 06:56:53,546][00611] Updated weights for policy 0, policy_version 77292 (0.0008) [2023-10-08 06:56:53,730][00612] Updated weights for policy 1, policy_version 77720 (0.0007) [2023-10-08 06:56:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 158695424. Throughput: 0: 1826.2, 1: 1838.2. Samples: 39684674. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:56:53,754][130385] Avg episode reward: [(0, '72.760'), (1, '73.320')] [2023-10-08 06:56:53,916][00611] Updated weights for policy 0, policy_version 77302 (0.0008) [2023-10-08 06:56:54,277][00611] Updated weights for policy 0, policy_version 77312 (0.0007) [2023-10-08 06:56:57,382][00612] Updated weights for policy 1, policy_version 77730 (0.0007) [2023-10-08 06:56:57,754][00612] Updated weights for policy 1, policy_version 77740 (0.0008) [2023-10-08 06:56:57,890][00611] Updated weights for policy 0, policy_version 77322 (0.0009) [2023-10-08 06:56:58,115][00612] Updated weights for policy 1, policy_version 77750 (0.0008) [2023-10-08 06:56:58,265][00611] Updated weights for policy 0, policy_version 77332 (0.0009) [2023-10-08 06:56:58,484][00612] Updated weights for policy 1, policy_version 77760 (0.0008) [2023-10-08 06:56:58,624][00611] Updated weights for policy 0, policy_version 77342 (0.0007) [2023-10-08 06:56:58,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 158826496. Throughput: 0: 1822.0, 1: 1843.7. Samples: 39707896. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:56:58,755][130385] Avg episode reward: [(0, '72.830'), (1, '75.940')] [2023-10-08 06:57:02,089][00612] Updated weights for policy 1, policy_version 77770 (0.0011) [2023-10-08 06:57:02,359][00611] Updated weights for policy 0, policy_version 77352 (0.0009) [2023-10-08 06:57:02,454][00612] Updated weights for policy 1, policy_version 77780 (0.0008) [2023-10-08 06:57:02,729][00611] Updated weights for policy 0, policy_version 77362 (0.0009) [2023-10-08 06:57:02,812][00612] Updated weights for policy 1, policy_version 77790 (0.0010) [2023-10-08 06:57:03,115][00611] Updated weights for policy 0, policy_version 77372 (0.0008) [2023-10-08 06:57:03,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 158892032. Throughput: 0: 1817.6, 1: 1834.0. Samples: 39727764. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:57:03,755][130385] Avg episode reward: [(0, '72.910'), (1, '73.430')] [2023-10-08 06:57:06,383][00612] Updated weights for policy 1, policy_version 77800 (0.0011) [2023-10-08 06:57:06,726][00611] Updated weights for policy 0, policy_version 77382 (0.0008) [2023-10-08 06:57:06,746][00612] Updated weights for policy 1, policy_version 77810 (0.0008) [2023-10-08 06:57:07,085][00611] Updated weights for policy 0, policy_version 77392 (0.0008) [2023-10-08 06:57:07,117][00612] Updated weights for policy 1, policy_version 77820 (0.0010) [2023-10-08 06:57:07,464][00611] Updated weights for policy 0, policy_version 77402 (0.0010) [2023-10-08 06:57:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 158957568. Throughput: 0: 1822.4, 1: 1835.6. Samples: 39740550. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:57:08,754][130385] Avg episode reward: [(0, '74.930'), (1, '73.540')] [2023-10-08 06:57:11,001][00612] Updated weights for policy 1, policy_version 77830 (0.0008) [2023-10-08 06:57:11,117][00611] Updated weights for policy 0, policy_version 77412 (0.0009) [2023-10-08 06:57:11,362][00612] Updated weights for policy 1, policy_version 77840 (0.0007) [2023-10-08 06:57:11,501][00611] Updated weights for policy 0, policy_version 77422 (0.0009) [2023-10-08 06:57:11,726][00612] Updated weights for policy 1, policy_version 77850 (0.0007) [2023-10-08 06:57:11,869][00611] Updated weights for policy 0, policy_version 77432 (0.0009) [2023-10-08 06:57:13,754][130385] Fps is (10 sec: 13107.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 159023104. Throughput: 0: 1819.4, 1: 1831.2. Samples: 39760700. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 06:57:13,754][130385] Avg episode reward: [(0, '71.930'), (1, '71.160')] [2023-10-08 06:57:15,408][00612] Updated weights for policy 1, policy_version 77860 (0.0007) [2023-10-08 06:57:15,482][00611] Updated weights for policy 0, policy_version 77442 (0.0008) [2023-10-08 06:57:15,795][00612] Updated weights for policy 1, policy_version 77870 (0.0009) [2023-10-08 06:57:15,856][00611] Updated weights for policy 0, policy_version 77452 (0.0008) [2023-10-08 06:57:16,151][00612] Updated weights for policy 1, policy_version 77880 (0.0008) [2023-10-08 06:57:16,219][00611] Updated weights for policy 0, policy_version 77462 (0.0009) [2023-10-08 06:57:16,594][00611] Updated weights for policy 0, policy_version 77472 (0.0008) [2023-10-08 06:57:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 159088640. Throughput: 0: 1827.1, 1: 1841.8. Samples: 39783880. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-08 06:57:18,755][130385] Avg episode reward: [(0, '66.710'), (1, '71.610')] [2023-10-08 06:57:19,595][00612] Updated weights for policy 1, policy_version 77890 (0.0007) [2023-10-08 06:57:19,959][00612] Updated weights for policy 1, policy_version 77900 (0.0008) [2023-10-08 06:57:20,166][00611] Updated weights for policy 0, policy_version 77482 (0.0008) [2023-10-08 06:57:20,328][00612] Updated weights for policy 1, policy_version 77910 (0.0008) [2023-10-08 06:57:20,530][00611] Updated weights for policy 0, policy_version 77492 (0.0008) [2023-10-08 06:57:20,703][00612] Updated weights for policy 1, policy_version 77920 (0.0009) [2023-10-08 06:57:20,905][00611] Updated weights for policy 0, policy_version 77502 (0.0008) [2023-10-08 06:57:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 159154176. Throughput: 0: 1822.9, 1: 1840.0. Samples: 39793890. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-08 06:57:23,754][130385] Avg episode reward: [(0, '67.750'), (1, '71.480')] [2023-10-08 06:57:24,315][00612] Updated weights for policy 1, policy_version 77930 (0.0009) [2023-10-08 06:57:24,681][00612] Updated weights for policy 1, policy_version 77940 (0.0008) [2023-10-08 06:57:24,689][00611] Updated weights for policy 0, policy_version 77512 (0.0009) [2023-10-08 06:57:25,052][00611] Updated weights for policy 0, policy_version 77522 (0.0007) [2023-10-08 06:57:25,053][00612] Updated weights for policy 1, policy_version 77950 (0.0008) [2023-10-08 06:57:25,421][00611] Updated weights for policy 0, policy_version 77532 (0.0007) [2023-10-08 06:57:28,735][00612] Updated weights for policy 1, policy_version 77960 (0.0008) [2023-10-08 06:57:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 159219712. Throughput: 0: 1823.7, 1: 1846.8. Samples: 39816978. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-08 06:57:28,754][130385] Avg episode reward: [(0, '65.680'), (1, '71.430')] [2023-10-08 06:57:28,962][00611] Updated weights for policy 0, policy_version 77542 (0.0008) [2023-10-08 06:57:29,099][00612] Updated weights for policy 1, policy_version 77970 (0.0008) [2023-10-08 06:57:29,328][00611] Updated weights for policy 0, policy_version 77552 (0.0007) [2023-10-08 06:57:29,462][00612] Updated weights for policy 1, policy_version 77980 (0.0008) [2023-10-08 06:57:29,697][00611] Updated weights for policy 0, policy_version 77562 (0.0007) [2023-10-08 06:57:33,108][00612] Updated weights for policy 1, policy_version 77990 (0.0008) [2023-10-08 06:57:33,299][00611] Updated weights for policy 0, policy_version 77572 (0.0007) [2023-10-08 06:57:33,474][00612] Updated weights for policy 1, policy_version 78000 (0.0009) [2023-10-08 06:57:33,666][00611] Updated weights for policy 0, policy_version 77582 (0.0007) [2023-10-08 06:57:33,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 159285248. Throughput: 0: 1833.5, 1: 1835.9. Samples: 39839826. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-08 06:57:33,755][130385] Avg episode reward: [(0, '65.610'), (1, '78.010')] [2023-10-08 06:57:33,839][00612] Updated weights for policy 1, policy_version 78010 (0.0007) [2023-10-08 06:57:34,035][00611] Updated weights for policy 0, policy_version 77592 (0.0011) [2023-10-08 06:57:37,365][00612] Updated weights for policy 1, policy_version 78020 (0.0009) [2023-10-08 06:57:37,730][00612] Updated weights for policy 1, policy_version 78030 (0.0008) [2023-10-08 06:57:37,742][00611] Updated weights for policy 0, policy_version 77602 (0.0007) [2023-10-08 06:57:38,097][00612] Updated weights for policy 1, policy_version 78040 (0.0009) [2023-10-08 06:57:38,120][00611] Updated weights for policy 0, policy_version 77612 (0.0007) [2023-10-08 06:57:38,488][00611] Updated weights for policy 0, policy_version 77622 (0.0008) [2023-10-08 06:57:38,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 159383552. Throughput: 0: 1828.2, 1: 1847.0. Samples: 39850058. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-08 06:57:38,754][130385] Avg episode reward: [(0, '68.080'), (1, '80.480')] [2023-10-08 06:57:38,862][00611] Updated weights for policy 0, policy_version 77632 (0.0007) [2023-10-08 06:57:41,799][00612] Updated weights for policy 1, policy_version 78050 (0.0009) [2023-10-08 06:57:42,157][00612] Updated weights for policy 1, policy_version 78060 (0.0008) [2023-10-08 06:57:42,442][00611] Updated weights for policy 0, policy_version 77642 (0.0007) [2023-10-08 06:57:42,515][00612] Updated weights for policy 1, policy_version 78070 (0.0009) [2023-10-08 06:57:42,808][00611] Updated weights for policy 0, policy_version 77652 (0.0008) [2023-10-08 06:57:42,879][00612] Updated weights for policy 1, policy_version 78080 (0.0009) [2023-10-08 06:57:43,175][00611] Updated weights for policy 0, policy_version 77662 (0.0010) [2023-10-08 06:57:43,754][130385] Fps is (10 sec: 19661.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 159481856. Throughput: 0: 1832.8, 1: 1829.4. Samples: 39872696. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-08 06:57:43,754][130385] Avg episode reward: [(0, '68.240'), (1, '88.590')] [2023-10-08 06:57:46,424][00612] Updated weights for policy 1, policy_version 78090 (0.0010) [2023-10-08 06:57:46,790][00612] Updated weights for policy 1, policy_version 78100 (0.0009) [2023-10-08 06:57:46,917][00611] Updated weights for policy 0, policy_version 77672 (0.0008) [2023-10-08 06:57:47,159][00612] Updated weights for policy 1, policy_version 78110 (0.0007) [2023-10-08 06:57:47,289][00611] Updated weights for policy 0, policy_version 77682 (0.0008) [2023-10-08 06:57:47,658][00611] Updated weights for policy 0, policy_version 77692 (0.0009) [2023-10-08 06:57:48,754][130385] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 159547392. Throughput: 0: 1834.9, 1: 1843.0. Samples: 39893268. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-08 06:57:48,755][130385] Avg episode reward: [(0, '67.540'), (1, '92.030')] [2023-10-08 06:57:48,766][00425] Saving new best policy, reward=92.030! [2023-10-08 06:57:50,873][00612] Updated weights for policy 1, policy_version 78120 (0.0007) [2023-10-08 06:57:51,250][00612] Updated weights for policy 1, policy_version 78130 (0.0007) [2023-10-08 06:57:51,461][00611] Updated weights for policy 0, policy_version 77702 (0.0007) [2023-10-08 06:57:51,615][00612] Updated weights for policy 1, policy_version 78140 (0.0007) [2023-10-08 06:57:51,839][00611] Updated weights for policy 0, policy_version 77712 (0.0009) [2023-10-08 06:57:52,207][00611] Updated weights for policy 0, policy_version 77722 (0.0009) [2023-10-08 06:57:53,754][130385] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 159612928. Throughput: 0: 1831.2, 1: 1832.3. Samples: 39905406. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-08 06:57:53,755][130385] Avg episode reward: [(0, '71.880'), (1, '91.740')] [2023-10-08 06:57:55,342][00612] Updated weights for policy 1, policy_version 78150 (0.0008) [2023-10-08 06:57:55,708][00612] Updated weights for policy 1, policy_version 78160 (0.0008) [2023-10-08 06:57:55,811][00611] Updated weights for policy 0, policy_version 77732 (0.0007) [2023-10-08 06:57:56,082][00612] Updated weights for policy 1, policy_version 78170 (0.0007) [2023-10-08 06:57:56,189][00611] Updated weights for policy 0, policy_version 77742 (0.0009) [2023-10-08 06:57:56,556][00611] Updated weights for policy 0, policy_version 77752 (0.0007) [2023-10-08 06:57:58,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 159678464. Throughput: 0: 1830.5, 1: 1847.2. Samples: 39926198. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-08 06:57:58,754][130385] Avg episode reward: [(0, '74.090'), (1, '92.960')] [2023-10-08 06:57:58,755][00425] Saving new best policy, reward=92.960! [2023-10-08 06:57:59,912][00612] Updated weights for policy 1, policy_version 78180 (0.0007) [2023-10-08 06:58:00,276][00612] Updated weights for policy 1, policy_version 78190 (0.0007) [2023-10-08 06:58:00,382][00611] Updated weights for policy 0, policy_version 77762 (0.0008) [2023-10-08 06:58:00,649][00612] Updated weights for policy 1, policy_version 78200 (0.0009) [2023-10-08 06:58:00,758][00611] Updated weights for policy 0, policy_version 77772 (0.0008) [2023-10-08 06:58:01,129][00611] Updated weights for policy 0, policy_version 77782 (0.0008) [2023-10-08 06:58:01,500][00611] Updated weights for policy 0, policy_version 77792 (0.0008) [2023-10-08 06:58:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 159744000. Throughput: 0: 1827.4, 1: 1845.4. Samples: 39949154. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-08 06:58:03,754][130385] Avg episode reward: [(0, '71.400'), (1, '87.830')] [2023-10-08 06:58:04,151][00612] Updated weights for policy 1, policy_version 78210 (0.0007) [2023-10-08 06:58:04,555][00612] Updated weights for policy 1, policy_version 78220 (0.0008) [2023-10-08 06:58:04,921][00612] Updated weights for policy 1, policy_version 78230 (0.0008) [2023-10-08 06:58:05,026][00611] Updated weights for policy 0, policy_version 77802 (0.0008) [2023-10-08 06:58:05,289][00612] Updated weights for policy 1, policy_version 78240 (0.0007) [2023-10-08 06:58:05,391][00611] Updated weights for policy 0, policy_version 77812 (0.0007) [2023-10-08 06:58:05,771][00611] Updated weights for policy 0, policy_version 77822 (0.0009) [2023-10-08 06:58:08,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 159809536. Throughput: 0: 1834.9, 1: 1837.2. Samples: 39959136. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 06:58:08,755][130385] Avg episode reward: [(0, '71.000'), (1, '88.860')] [2023-10-08 06:58:08,902][00612] Updated weights for policy 1, policy_version 78250 (0.0009) [2023-10-08 06:58:09,268][00612] Updated weights for policy 1, policy_version 78260 (0.0009) [2023-10-08 06:58:09,501][00611] Updated weights for policy 0, policy_version 77832 (0.0008) [2023-10-08 06:58:09,632][00612] Updated weights for policy 1, policy_version 78270 (0.0010) [2023-10-08 06:58:09,866][00611] Updated weights for policy 0, policy_version 77842 (0.0009) [2023-10-08 06:58:10,238][00611] Updated weights for policy 0, policy_version 77852 (0.0008) [2023-10-08 06:58:13,231][00612] Updated weights for policy 1, policy_version 78280 (0.0009) [2023-10-08 06:58:13,609][00612] Updated weights for policy 1, policy_version 78290 (0.0009) [2023-10-08 06:58:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 159875072. Throughput: 0: 1833.7, 1: 1839.0. Samples: 39982248. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 06:58:13,754][130385] Avg episode reward: [(0, '70.870'), (1, '88.880')] [2023-10-08 06:58:13,808][00611] Updated weights for policy 0, policy_version 77862 (0.0009) [2023-10-08 06:58:13,972][00612] Updated weights for policy 1, policy_version 78300 (0.0008) [2023-10-08 06:58:14,173][00611] Updated weights for policy 0, policy_version 77872 (0.0009) [2023-10-08 06:58:14,548][00611] Updated weights for policy 0, policy_version 77882 (0.0010) [2023-10-08 06:58:17,552][00612] Updated weights for policy 1, policy_version 78310 (0.0008) [2023-10-08 06:58:17,917][00612] Updated weights for policy 1, policy_version 78320 (0.0008) [2023-10-08 06:58:18,198][00611] Updated weights for policy 0, policy_version 77892 (0.0009) [2023-10-08 06:58:18,287][00612] Updated weights for policy 1, policy_version 78330 (0.0009) [2023-10-08 06:58:18,573][00611] Updated weights for policy 0, policy_version 77902 (0.0007) [2023-10-08 06:58:18,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 159973376. Throughput: 0: 1829.4, 1: 1828.1. Samples: 40004410. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 06:58:18,754][130385] Avg episode reward: [(0, '70.380'), (1, '88.910')] [2023-10-08 06:58:18,760][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000078336_80216064.pth... [2023-10-08 06:58:18,802][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000076608_78446592.pth [2023-10-08 06:58:18,943][00611] Updated weights for policy 0, policy_version 77912 (0.0009) [2023-10-08 06:58:19,243][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000077920_79790080.pth... [2023-10-08 06:58:19,273][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000076192_78020608.pth [2023-10-08 06:58:21,909][00612] Updated weights for policy 1, policy_version 78340 (0.0009) [2023-10-08 06:58:22,279][00612] Updated weights for policy 1, policy_version 78350 (0.0009) [2023-10-08 06:58:22,537][00611] Updated weights for policy 0, policy_version 77922 (0.0008) [2023-10-08 06:58:22,640][00612] Updated weights for policy 1, policy_version 78360 (0.0008) [2023-10-08 06:58:22,908][00611] Updated weights for policy 0, policy_version 77932 (0.0008) [2023-10-08 06:58:23,285][00611] Updated weights for policy 0, policy_version 77942 (0.0011) [2023-10-08 06:58:23,651][00611] Updated weights for policy 0, policy_version 77952 (0.0010) [2023-10-08 06:58:23,754][130385] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 160071680. Throughput: 0: 1834.7, 1: 1842.9. Samples: 40015550. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 06:58:23,755][130385] Avg episode reward: [(0, '74.580'), (1, '87.280')] [2023-10-08 06:58:26,191][00612] Updated weights for policy 1, policy_version 78370 (0.0007) [2023-10-08 06:58:26,558][00612] Updated weights for policy 1, policy_version 78380 (0.0007) [2023-10-08 06:58:26,932][00612] Updated weights for policy 1, policy_version 78390 (0.0007) [2023-10-08 06:58:27,309][00612] Updated weights for policy 1, policy_version 78400 (0.0008) [2023-10-08 06:58:27,326][00611] Updated weights for policy 0, policy_version 77962 (0.0007) [2023-10-08 06:58:27,703][00611] Updated weights for policy 0, policy_version 77972 (0.0011) [2023-10-08 06:58:28,068][00611] Updated weights for policy 0, policy_version 77982 (0.0008) [2023-10-08 06:58:28,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 160137216. Throughput: 0: 1825.5, 1: 1829.7. Samples: 40037184. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 06:58:28,755][130385] Avg episode reward: [(0, '72.850'), (1, '87.940')] [2023-10-08 06:58:30,975][00612] Updated weights for policy 1, policy_version 78410 (0.0010) [2023-10-08 06:58:31,339][00612] Updated weights for policy 1, policy_version 78420 (0.0007) [2023-10-08 06:58:31,706][00612] Updated weights for policy 1, policy_version 78430 (0.0007) [2023-10-08 06:58:31,815][00611] Updated weights for policy 0, policy_version 77992 (0.0008) [2023-10-08 06:58:32,185][00611] Updated weights for policy 0, policy_version 78002 (0.0007) [2023-10-08 06:58:32,552][00611] Updated weights for policy 0, policy_version 78012 (0.0008) [2023-10-08 06:58:33,754][130385] Fps is (10 sec: 13107.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 160202752. Throughput: 0: 1828.1, 1: 1845.1. Samples: 40058560. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 06:58:33,754][130385] Avg episode reward: [(0, '71.690'), (1, '91.060')] [2023-10-08 06:58:35,236][00612] Updated weights for policy 1, policy_version 78440 (0.0009) [2023-10-08 06:58:35,609][00612] Updated weights for policy 1, policy_version 78450 (0.0010) [2023-10-08 06:58:35,983][00612] Updated weights for policy 1, policy_version 78460 (0.0009) [2023-10-08 06:58:36,202][00611] Updated weights for policy 0, policy_version 78022 (0.0008) [2023-10-08 06:58:36,572][00611] Updated weights for policy 0, policy_version 78032 (0.0007) [2023-10-08 06:58:36,944][00611] Updated weights for policy 0, policy_version 78042 (0.0010) [2023-10-08 06:58:38,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 160268288. Throughput: 0: 1830.2, 1: 1827.0. Samples: 40069980. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 06:58:38,755][130385] Avg episode reward: [(0, '74.430'), (1, '89.680')] [2023-10-08 06:58:39,725][00612] Updated weights for policy 1, policy_version 78470 (0.0007) [2023-10-08 06:58:40,096][00612] Updated weights for policy 1, policy_version 78480 (0.0007) [2023-10-08 06:58:40,460][00612] Updated weights for policy 1, policy_version 78490 (0.0008) [2023-10-08 06:58:40,468][00611] Updated weights for policy 0, policy_version 78052 (0.0009) [2023-10-08 06:58:40,849][00611] Updated weights for policy 0, policy_version 78062 (0.0008) [2023-10-08 06:58:41,220][00611] Updated weights for policy 0, policy_version 78072 (0.0011) [2023-10-08 06:58:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 160333824. Throughput: 0: 1832.9, 1: 1850.3. Samples: 40091940. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 06:58:43,754][130385] Avg episode reward: [(0, '79.120'), (1, '94.940')] [2023-10-08 06:58:43,755][00425] Saving new best policy, reward=94.940! [2023-10-08 06:58:44,052][00612] Updated weights for policy 1, policy_version 78500 (0.0008) [2023-10-08 06:58:44,419][00612] Updated weights for policy 1, policy_version 78510 (0.0008) [2023-10-08 06:58:44,782][00612] Updated weights for policy 1, policy_version 78520 (0.0009) [2023-10-08 06:58:45,162][00611] Updated weights for policy 0, policy_version 78082 (0.0009) [2023-10-08 06:58:45,561][00611] Updated weights for policy 0, policy_version 78092 (0.0007) [2023-10-08 06:58:45,930][00611] Updated weights for policy 0, policy_version 78102 (0.0009) [2023-10-08 06:58:46,306][00611] Updated weights for policy 0, policy_version 78112 (0.0010) [2023-10-08 06:58:48,347][00612] Updated weights for policy 1, policy_version 78530 (0.0010) [2023-10-08 06:58:48,721][00612] Updated weights for policy 1, policy_version 78540 (0.0011) [2023-10-08 06:58:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 160399360. Throughput: 0: 1832.9, 1: 1854.7. Samples: 40115098. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 06:58:48,755][130385] Avg episode reward: [(0, '74.440'), (1, '96.840')] [2023-10-08 06:58:49,092][00612] Updated weights for policy 1, policy_version 78550 (0.0009) [2023-10-08 06:58:49,455][00425] Saving new best policy, reward=96.840! [2023-10-08 06:58:49,457][00612] Updated weights for policy 1, policy_version 78560 (0.0009) [2023-10-08 06:58:49,912][00611] Updated weights for policy 0, policy_version 78122 (0.0009) [2023-10-08 06:58:50,289][00611] Updated weights for policy 0, policy_version 78132 (0.0007) [2023-10-08 06:58:50,655][00611] Updated weights for policy 0, policy_version 78142 (0.0007) [2023-10-08 06:58:53,141][00612] Updated weights for policy 1, policy_version 78570 (0.0009) [2023-10-08 06:58:53,510][00612] Updated weights for policy 1, policy_version 78580 (0.0007) [2023-10-08 06:58:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 160464896. Throughput: 0: 1826.5, 1: 1859.9. Samples: 40125022. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 06:58:53,754][130385] Avg episode reward: [(0, '72.090'), (1, '95.090')] [2023-10-08 06:58:53,881][00612] Updated weights for policy 1, policy_version 78590 (0.0011) [2023-10-08 06:58:54,352][00611] Updated weights for policy 0, policy_version 78152 (0.0009) [2023-10-08 06:58:54,715][00611] Updated weights for policy 0, policy_version 78162 (0.0011) [2023-10-08 06:58:55,087][00611] Updated weights for policy 0, policy_version 78172 (0.0009) [2023-10-08 06:58:57,647][00612] Updated weights for policy 1, policy_version 78600 (0.0009) [2023-10-08 06:58:58,007][00612] Updated weights for policy 1, policy_version 78610 (0.0010) [2023-10-08 06:58:58,372][00612] Updated weights for policy 1, policy_version 78620 (0.0008) [2023-10-08 06:58:58,718][00611] Updated weights for policy 0, policy_version 78182 (0.0011) [2023-10-08 06:58:58,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 160563200. Throughput: 0: 1825.4, 1: 1856.6. Samples: 40147938. Policy #0 lag: (min: 27.0, avg: 29.3, max: 59.0) [2023-10-08 06:58:58,755][130385] Avg episode reward: [(0, '71.080'), (1, '89.280')] [2023-10-08 06:58:59,099][00611] Updated weights for policy 0, policy_version 78192 (0.0008) [2023-10-08 06:58:59,461][00611] Updated weights for policy 0, policy_version 78202 (0.0007) [2023-10-08 06:59:01,969][00612] Updated weights for policy 1, policy_version 78630 (0.0010) [2023-10-08 06:59:02,343][00612] Updated weights for policy 1, policy_version 78640 (0.0010) [2023-10-08 06:59:02,712][00612] Updated weights for policy 1, policy_version 78650 (0.0011) [2023-10-08 06:59:03,152][00611] Updated weights for policy 0, policy_version 78212 (0.0008) [2023-10-08 06:59:03,528][00611] Updated weights for policy 0, policy_version 78222 (0.0008) [2023-10-08 06:59:03,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 160628736. Throughput: 0: 1820.3, 1: 1843.2. Samples: 40169268. Policy #0 lag: (min: 27.0, avg: 29.3, max: 59.0) [2023-10-08 06:59:03,755][130385] Avg episode reward: [(0, '71.160'), (1, '90.080')] [2023-10-08 06:59:03,902][00611] Updated weights for policy 0, policy_version 78232 (0.0007) [2023-10-08 06:59:06,389][00612] Updated weights for policy 1, policy_version 78660 (0.0008) [2023-10-08 06:59:06,757][00612] Updated weights for policy 1, policy_version 78670 (0.0009) [2023-10-08 06:59:07,138][00612] Updated weights for policy 1, policy_version 78680 (0.0009) [2023-10-08 06:59:07,283][00611] Updated weights for policy 0, policy_version 78242 (0.0007) [2023-10-08 06:59:07,646][00611] Updated weights for policy 0, policy_version 78252 (0.0011) [2023-10-08 06:59:08,019][00611] Updated weights for policy 0, policy_version 78262 (0.0011) [2023-10-08 06:59:08,390][00611] Updated weights for policy 0, policy_version 78272 (0.0011) [2023-10-08 06:59:08,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 160727040. Throughput: 0: 1823.7, 1: 1852.9. Samples: 40181000. Policy #0 lag: (min: 27.0, avg: 29.3, max: 59.0) [2023-10-08 06:59:08,755][130385] Avg episode reward: [(0, '72.200'), (1, '87.180')] [2023-10-08 06:59:10,745][00612] Updated weights for policy 1, policy_version 78690 (0.0009) [2023-10-08 06:59:11,103][00612] Updated weights for policy 1, policy_version 78700 (0.0009) [2023-10-08 06:59:11,464][00612] Updated weights for policy 1, policy_version 78710 (0.0010) [2023-10-08 06:59:11,836][00612] Updated weights for policy 1, policy_version 78720 (0.0010) [2023-10-08 06:59:12,238][00611] Updated weights for policy 0, policy_version 78282 (0.0007) [2023-10-08 06:59:12,606][00611] Updated weights for policy 0, policy_version 78292 (0.0008) [2023-10-08 06:59:12,978][00611] Updated weights for policy 0, policy_version 78302 (0.0008) [2023-10-08 06:59:13,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 160792576. Throughput: 0: 1820.5, 1: 1847.3. Samples: 40202232. Policy #0 lag: (min: 27.0, avg: 29.3, max: 59.0) [2023-10-08 06:59:13,754][130385] Avg episode reward: [(0, '69.140'), (1, '84.770')] [2023-10-08 06:59:15,553][00612] Updated weights for policy 1, policy_version 78730 (0.0009) [2023-10-08 06:59:15,918][00612] Updated weights for policy 1, policy_version 78740 (0.0009) [2023-10-08 06:59:16,291][00612] Updated weights for policy 1, policy_version 78750 (0.0008) [2023-10-08 06:59:16,581][00611] Updated weights for policy 0, policy_version 78312 (0.0007) [2023-10-08 06:59:16,955][00611] Updated weights for policy 0, policy_version 78322 (0.0010) [2023-10-08 06:59:17,333][00611] Updated weights for policy 0, policy_version 78332 (0.0011) [2023-10-08 06:59:18,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 160858112. Throughput: 0: 1824.1, 1: 1856.9. Samples: 40224206. Policy #0 lag: (min: 27.0, avg: 29.3, max: 59.0) [2023-10-08 06:59:18,754][130385] Avg episode reward: [(0, '66.930'), (1, '87.910')] [2023-10-08 06:59:19,807][00612] Updated weights for policy 1, policy_version 78760 (0.0009) [2023-10-08 06:59:20,177][00612] Updated weights for policy 1, policy_version 78770 (0.0009) [2023-10-08 06:59:20,539][00612] Updated weights for policy 1, policy_version 78780 (0.0007) [2023-10-08 06:59:21,059][00611] Updated weights for policy 0, policy_version 78342 (0.0008) [2023-10-08 06:59:21,441][00611] Updated weights for policy 0, policy_version 78352 (0.0007) [2023-10-08 06:59:21,806][00611] Updated weights for policy 0, policy_version 78362 (0.0009) [2023-10-08 06:59:23,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 160923648. Throughput: 0: 1822.6, 1: 1853.2. Samples: 40235394. Policy #0 lag: (min: 27.0, avg: 29.3, max: 59.0) [2023-10-08 06:59:23,755][130385] Avg episode reward: [(0, '66.830'), (1, '88.580')] [2023-10-08 06:59:24,053][00612] Updated weights for policy 1, policy_version 78790 (0.0009) [2023-10-08 06:59:24,417][00612] Updated weights for policy 1, policy_version 78800 (0.0009) [2023-10-08 06:59:24,783][00612] Updated weights for policy 1, policy_version 78810 (0.0011) [2023-10-08 06:59:25,545][00611] Updated weights for policy 0, policy_version 78372 (0.0008) [2023-10-08 06:59:25,919][00611] Updated weights for policy 0, policy_version 78382 (0.0009) [2023-10-08 06:59:26,287][00611] Updated weights for policy 0, policy_version 78392 (0.0008) [2023-10-08 06:59:28,477][00612] Updated weights for policy 1, policy_version 78820 (0.0010) [2023-10-08 06:59:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 160989184. Throughput: 0: 1825.2, 1: 1853.6. Samples: 40257486. Policy #0 lag: (min: 27.0, avg: 29.3, max: 59.0) [2023-10-08 06:59:28,754][130385] Avg episode reward: [(0, '69.180'), (1, '86.520')] [2023-10-08 06:59:28,840][00612] Updated weights for policy 1, policy_version 78830 (0.0008) [2023-10-08 06:59:29,208][00612] Updated weights for policy 1, policy_version 78840 (0.0010) [2023-10-08 06:59:29,810][00611] Updated weights for policy 0, policy_version 78402 (0.0009) [2023-10-08 06:59:30,212][00611] Updated weights for policy 0, policy_version 78412 (0.0007) [2023-10-08 06:59:30,577][00611] Updated weights for policy 0, policy_version 78422 (0.0008) [2023-10-08 06:59:30,949][00611] Updated weights for policy 0, policy_version 78432 (0.0009) [2023-10-08 06:59:32,648][00612] Updated weights for policy 1, policy_version 78850 (0.0009) [2023-10-08 06:59:33,022][00612] Updated weights for policy 1, policy_version 78860 (0.0008) [2023-10-08 06:59:33,384][00612] Updated weights for policy 1, policy_version 78870 (0.0007) [2023-10-08 06:59:33,747][00612] Updated weights for policy 1, policy_version 78880 (0.0008) [2023-10-08 06:59:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 161087488. Throughput: 0: 1828.8, 1: 1839.4. Samples: 40280170. Policy #0 lag: (min: 27.0, avg: 29.3, max: 59.0) [2023-10-08 06:59:33,755][130385] Avg episode reward: [(0, '71.200'), (1, '85.980')] [2023-10-08 06:59:34,529][00611] Updated weights for policy 0, policy_version 78442 (0.0008) [2023-10-08 06:59:34,905][00611] Updated weights for policy 0, policy_version 78452 (0.0009) [2023-10-08 06:59:35,277][00611] Updated weights for policy 0, policy_version 78462 (0.0007) [2023-10-08 06:59:37,508][00612] Updated weights for policy 1, policy_version 78890 (0.0009) [2023-10-08 06:59:37,874][00612] Updated weights for policy 1, policy_version 78900 (0.0008) [2023-10-08 06:59:38,245][00612] Updated weights for policy 1, policy_version 78910 (0.0008) [2023-10-08 06:59:38,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 161153024. Throughput: 0: 1827.7, 1: 1858.0. Samples: 40290882. Policy #0 lag: (min: 27.0, avg: 29.3, max: 59.0) [2023-10-08 06:59:38,755][130385] Avg episode reward: [(0, '72.190'), (1, '86.260')] [2023-10-08 06:59:38,977][00611] Updated weights for policy 0, policy_version 78472 (0.0008) [2023-10-08 06:59:39,341][00611] Updated weights for policy 0, policy_version 78482 (0.0009) [2023-10-08 06:59:39,709][00611] Updated weights for policy 0, policy_version 78492 (0.0009) [2023-10-08 06:59:41,717][00612] Updated weights for policy 1, policy_version 78920 (0.0009) [2023-10-08 06:59:42,089][00612] Updated weights for policy 1, policy_version 78930 (0.0009) [2023-10-08 06:59:42,467][00612] Updated weights for policy 1, policy_version 78940 (0.0008) [2023-10-08 06:59:43,337][00611] Updated weights for policy 0, policy_version 78502 (0.0008) [2023-10-08 06:59:43,701][00611] Updated weights for policy 0, policy_version 78512 (0.0007) [2023-10-08 06:59:43,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 161218560. Throughput: 0: 1832.1, 1: 1838.8. Samples: 40313130. Policy #0 lag: (min: 27.0, avg: 29.3, max: 59.0) [2023-10-08 06:59:43,754][130385] Avg episode reward: [(0, '69.810'), (1, '85.640')] [2023-10-08 06:59:44,078][00611] Updated weights for policy 0, policy_version 78522 (0.0008) [2023-10-08 06:59:46,136][00612] Updated weights for policy 1, policy_version 78950 (0.0008) [2023-10-08 06:59:46,514][00612] Updated weights for policy 1, policy_version 78960 (0.0009) [2023-10-08 06:59:46,889][00612] Updated weights for policy 1, policy_version 78970 (0.0009) [2023-10-08 06:59:47,779][00611] Updated weights for policy 0, policy_version 78532 (0.0009) [2023-10-08 06:59:48,139][00611] Updated weights for policy 0, policy_version 78542 (0.0009) [2023-10-08 06:59:48,518][00611] Updated weights for policy 0, policy_version 78552 (0.0008) [2023-10-08 06:59:48,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161284096. Throughput: 0: 1828.2, 1: 1856.2. Samples: 40335064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:59:48,754][130385] Avg episode reward: [(0, '72.530'), (1, '82.980')] [2023-10-08 06:59:50,628][00612] Updated weights for policy 1, policy_version 78980 (0.0008) [2023-10-08 06:59:50,994][00612] Updated weights for policy 1, policy_version 78990 (0.0010) [2023-10-08 06:59:51,368][00612] Updated weights for policy 1, policy_version 79000 (0.0010) [2023-10-08 06:59:52,221][00611] Updated weights for policy 0, policy_version 78562 (0.0008) [2023-10-08 06:59:52,580][00611] Updated weights for policy 0, policy_version 78572 (0.0008) [2023-10-08 06:59:52,958][00611] Updated weights for policy 0, policy_version 78582 (0.0008) [2023-10-08 06:59:53,332][00611] Updated weights for policy 0, policy_version 78592 (0.0007) [2023-10-08 06:59:53,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 161382400. Throughput: 0: 1837.6, 1: 1835.7. Samples: 40346298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:59:53,754][130385] Avg episode reward: [(0, '71.200'), (1, '78.650')] [2023-10-08 06:59:55,077][00612] Updated weights for policy 1, policy_version 79010 (0.0008) [2023-10-08 06:59:55,444][00612] Updated weights for policy 1, policy_version 79020 (0.0007) [2023-10-08 06:59:55,818][00612] Updated weights for policy 1, policy_version 79030 (0.0010) [2023-10-08 06:59:56,172][00612] Updated weights for policy 1, policy_version 79040 (0.0008) [2023-10-08 06:59:56,887][00611] Updated weights for policy 0, policy_version 78602 (0.0007) [2023-10-08 06:59:57,245][00611] Updated weights for policy 0, policy_version 78612 (0.0009) [2023-10-08 06:59:57,618][00611] Updated weights for policy 0, policy_version 78622 (0.0010) [2023-10-08 06:59:58,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 161447936. Throughput: 0: 1831.6, 1: 1858.7. Samples: 40368296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 06:59:58,755][130385] Avg episode reward: [(0, '72.550'), (1, '76.900')] [2023-10-08 06:59:59,657][00612] Updated weights for policy 1, policy_version 79050 (0.0008) [2023-10-08 07:00:00,032][00612] Updated weights for policy 1, policy_version 79060 (0.0007) [2023-10-08 07:00:00,394][00612] Updated weights for policy 1, policy_version 79070 (0.0010) [2023-10-08 07:00:01,150][00611] Updated weights for policy 0, policy_version 78632 (0.0009) [2023-10-08 07:00:01,512][00611] Updated weights for policy 0, policy_version 78642 (0.0007) [2023-10-08 07:00:01,886][00611] Updated weights for policy 0, policy_version 78652 (0.0007) [2023-10-08 07:00:03,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 161513472. Throughput: 0: 1846.1, 1: 1858.1. Samples: 40390894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:00:03,755][130385] Avg episode reward: [(0, '74.130'), (1, '74.340')] [2023-10-08 07:00:03,927][00612] Updated weights for policy 1, policy_version 79080 (0.0009) [2023-10-08 07:00:04,284][00612] Updated weights for policy 1, policy_version 79090 (0.0007) [2023-10-08 07:00:04,647][00612] Updated weights for policy 1, policy_version 79100 (0.0009) [2023-10-08 07:00:05,546][00611] Updated weights for policy 0, policy_version 78662 (0.0008) [2023-10-08 07:00:05,910][00611] Updated weights for policy 0, policy_version 78672 (0.0007) [2023-10-08 07:00:06,287][00611] Updated weights for policy 0, policy_version 78682 (0.0010) [2023-10-08 07:00:08,363][00612] Updated weights for policy 1, policy_version 79110 (0.0009) [2023-10-08 07:00:08,730][00612] Updated weights for policy 1, policy_version 79120 (0.0011) [2023-10-08 07:00:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 161579008. Throughput: 0: 1829.6, 1: 1859.3. Samples: 40401394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:00:08,754][130385] Avg episode reward: [(0, '70.380'), (1, '68.100')] [2023-10-08 07:00:09,102][00612] Updated weights for policy 1, policy_version 79130 (0.0007) [2023-10-08 07:00:09,868][00611] Updated weights for policy 0, policy_version 78692 (0.0009) [2023-10-08 07:00:10,241][00611] Updated weights for policy 0, policy_version 78702 (0.0007) [2023-10-08 07:00:10,609][00611] Updated weights for policy 0, policy_version 78712 (0.0008) [2023-10-08 07:00:12,713][00612] Updated weights for policy 1, policy_version 79140 (0.0008) [2023-10-08 07:00:13,076][00612] Updated weights for policy 1, policy_version 79150 (0.0008) [2023-10-08 07:00:13,447][00612] Updated weights for policy 1, policy_version 79160 (0.0009) [2023-10-08 07:00:13,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 161677312. Throughput: 0: 1843.5, 1: 1853.7. Samples: 40423860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:00:13,755][130385] Avg episode reward: [(0, '72.790'), (1, '70.960')] [2023-10-08 07:00:14,345][00611] Updated weights for policy 0, policy_version 78722 (0.0010) [2023-10-08 07:00:14,707][00611] Updated weights for policy 0, policy_version 78732 (0.0010) [2023-10-08 07:00:15,087][00611] Updated weights for policy 0, policy_version 78742 (0.0008) [2023-10-08 07:00:15,459][00611] Updated weights for policy 0, policy_version 78752 (0.0008) [2023-10-08 07:00:16,993][00612] Updated weights for policy 1, policy_version 79170 (0.0009) [2023-10-08 07:00:17,358][00612] Updated weights for policy 1, policy_version 79180 (0.0007) [2023-10-08 07:00:17,724][00612] Updated weights for policy 1, policy_version 79190 (0.0010) [2023-10-08 07:00:18,090][00612] Updated weights for policy 1, policy_version 79200 (0.0007) [2023-10-08 07:00:18,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 161742848. Throughput: 0: 1844.7, 1: 1834.7. Samples: 40445746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:00:18,755][130385] Avg episode reward: [(0, '74.200'), (1, '72.680')] [2023-10-08 07:00:18,766][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000079200_81100800.pth... [2023-10-08 07:00:18,798][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000077472_79331328.pth [2023-10-08 07:00:19,122][00611] Updated weights for policy 0, policy_version 78762 (0.0007) [2023-10-08 07:00:19,494][00611] Updated weights for policy 0, policy_version 78772 (0.0010) [2023-10-08 07:00:19,858][00611] Updated weights for policy 0, policy_version 78782 (0.0010) [2023-10-08 07:00:19,930][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000078784_80674816.pth... [2023-10-08 07:00:19,959][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000077056_78905344.pth [2023-10-08 07:00:21,630][00612] Updated weights for policy 1, policy_version 79210 (0.0008) [2023-10-08 07:00:21,995][00612] Updated weights for policy 1, policy_version 79220 (0.0009) [2023-10-08 07:00:22,366][00612] Updated weights for policy 1, policy_version 79230 (0.0007) [2023-10-08 07:00:23,504][00611] Updated weights for policy 0, policy_version 78792 (0.0008) [2023-10-08 07:00:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 161808384. Throughput: 0: 1845.5, 1: 1856.3. Samples: 40457462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:00:23,754][130385] Avg episode reward: [(0, '74.490'), (1, '73.720')] [2023-10-08 07:00:23,882][00611] Updated weights for policy 0, policy_version 78802 (0.0009) [2023-10-08 07:00:24,243][00611] Updated weights for policy 0, policy_version 78812 (0.0009) [2023-10-08 07:00:26,058][00612] Updated weights for policy 1, policy_version 79240 (0.0009) [2023-10-08 07:00:26,424][00612] Updated weights for policy 1, policy_version 79250 (0.0007) [2023-10-08 07:00:26,795][00612] Updated weights for policy 1, policy_version 79260 (0.0007) [2023-10-08 07:00:27,858][00611] Updated weights for policy 0, policy_version 78822 (0.0010) [2023-10-08 07:00:28,225][00611] Updated weights for policy 0, policy_version 78832 (0.0007) [2023-10-08 07:00:28,592][00611] Updated weights for policy 0, policy_version 78842 (0.0008) [2023-10-08 07:00:28,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161873920. Throughput: 0: 1848.3, 1: 1841.8. Samples: 40479186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:00:28,754][130385] Avg episode reward: [(0, '75.520'), (1, '73.200')] [2023-10-08 07:00:30,522][00612] Updated weights for policy 1, policy_version 79270 (0.0007) [2023-10-08 07:00:30,898][00612] Updated weights for policy 1, policy_version 79280 (0.0008) [2023-10-08 07:00:31,263][00612] Updated weights for policy 1, policy_version 79290 (0.0010) [2023-10-08 07:00:32,275][00611] Updated weights for policy 0, policy_version 78852 (0.0008) [2023-10-08 07:00:32,648][00611] Updated weights for policy 0, policy_version 78862 (0.0008) [2023-10-08 07:00:33,020][00611] Updated weights for policy 0, policy_version 78872 (0.0009) [2023-10-08 07:00:33,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 161972224. Throughput: 0: 1832.5, 1: 1857.8. Samples: 40501126. Policy #0 lag: (min: 22.0, avg: 26.8, max: 54.0) [2023-10-08 07:00:33,755][130385] Avg episode reward: [(0, '77.600'), (1, '71.780')] [2023-10-08 07:00:34,789][00612] Updated weights for policy 1, policy_version 79300 (0.0010) [2023-10-08 07:00:35,161][00612] Updated weights for policy 1, policy_version 79310 (0.0008) [2023-10-08 07:00:35,521][00612] Updated weights for policy 1, policy_version 79320 (0.0008) [2023-10-08 07:00:36,585][00611] Updated weights for policy 0, policy_version 78882 (0.0007) [2023-10-08 07:00:36,953][00611] Updated weights for policy 0, policy_version 78892 (0.0011) [2023-10-08 07:00:37,321][00611] Updated weights for policy 0, policy_version 78902 (0.0010) [2023-10-08 07:00:37,692][00611] Updated weights for policy 0, policy_version 78912 (0.0009) [2023-10-08 07:00:38,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 162037760. Throughput: 0: 1850.6, 1: 1842.5. Samples: 40512486. Policy #0 lag: (min: 22.0, avg: 26.8, max: 54.0) [2023-10-08 07:00:38,754][130385] Avg episode reward: [(0, '76.970'), (1, '73.630')] [2023-10-08 07:00:39,138][00612] Updated weights for policy 1, policy_version 79330 (0.0009) [2023-10-08 07:00:39,516][00612] Updated weights for policy 1, policy_version 79340 (0.0009) [2023-10-08 07:00:39,879][00612] Updated weights for policy 1, policy_version 79350 (0.0008) [2023-10-08 07:00:40,239][00612] Updated weights for policy 1, policy_version 79360 (0.0010) [2023-10-08 07:00:41,350][00611] Updated weights for policy 0, policy_version 78922 (0.0009) [2023-10-08 07:00:41,717][00611] Updated weights for policy 0, policy_version 78932 (0.0008) [2023-10-08 07:00:42,095][00611] Updated weights for policy 0, policy_version 78942 (0.0008) [2023-10-08 07:00:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 162103296. Throughput: 0: 1831.6, 1: 1857.8. Samples: 40534318. Policy #0 lag: (min: 22.0, avg: 26.8, max: 54.0) [2023-10-08 07:00:43,755][130385] Avg episode reward: [(0, '76.930'), (1, '77.780')] [2023-10-08 07:00:43,901][00612] Updated weights for policy 1, policy_version 79370 (0.0008) [2023-10-08 07:00:44,271][00612] Updated weights for policy 1, policy_version 79380 (0.0011) [2023-10-08 07:00:44,636][00612] Updated weights for policy 1, policy_version 79390 (0.0009) [2023-10-08 07:00:45,712][00611] Updated weights for policy 0, policy_version 78952 (0.0008) [2023-10-08 07:00:46,086][00611] Updated weights for policy 0, policy_version 78962 (0.0009) [2023-10-08 07:00:46,457][00611] Updated weights for policy 0, policy_version 78972 (0.0009) [2023-10-08 07:00:48,185][00612] Updated weights for policy 1, policy_version 79400 (0.0009) [2023-10-08 07:00:48,558][00612] Updated weights for policy 1, policy_version 79410 (0.0010) [2023-10-08 07:00:48,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 162168832. Throughput: 0: 1847.3, 1: 1851.6. Samples: 40557342. Policy #0 lag: (min: 22.0, avg: 26.8, max: 54.0) [2023-10-08 07:00:48,754][130385] Avg episode reward: [(0, '75.360'), (1, '79.810')] [2023-10-08 07:00:48,937][00612] Updated weights for policy 1, policy_version 79420 (0.0009) [2023-10-08 07:00:50,096][00611] Updated weights for policy 0, policy_version 78982 (0.0008) [2023-10-08 07:00:50,468][00611] Updated weights for policy 0, policy_version 78992 (0.0009) [2023-10-08 07:00:50,838][00611] Updated weights for policy 0, policy_version 79002 (0.0007) [2023-10-08 07:00:52,495][00612] Updated weights for policy 1, policy_version 79430 (0.0009) [2023-10-08 07:00:52,860][00612] Updated weights for policy 1, policy_version 79440 (0.0009) [2023-10-08 07:00:53,223][00612] Updated weights for policy 1, policy_version 79450 (0.0007) [2023-10-08 07:00:53,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 162267136. Throughput: 0: 1834.0, 1: 1863.1. Samples: 40567764. Policy #0 lag: (min: 22.0, avg: 26.8, max: 54.0) [2023-10-08 07:00:53,754][130385] Avg episode reward: [(0, '75.490'), (1, '80.340')] [2023-10-08 07:00:54,418][00611] Updated weights for policy 0, policy_version 79012 (0.0008) [2023-10-08 07:00:54,791][00611] Updated weights for policy 0, policy_version 79022 (0.0009) [2023-10-08 07:00:55,166][00611] Updated weights for policy 0, policy_version 79032 (0.0008) [2023-10-08 07:00:56,887][00612] Updated weights for policy 1, policy_version 79460 (0.0008) [2023-10-08 07:00:57,252][00612] Updated weights for policy 1, policy_version 79470 (0.0008) [2023-10-08 07:00:57,623][00612] Updated weights for policy 1, policy_version 79480 (0.0009) [2023-10-08 07:00:58,690][00611] Updated weights for policy 0, policy_version 79042 (0.0008) [2023-10-08 07:00:58,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 162332672. Throughput: 0: 1853.0, 1: 1851.4. Samples: 40590558. Policy #0 lag: (min: 22.0, avg: 26.8, max: 54.0) [2023-10-08 07:00:58,755][130385] Avg episode reward: [(0, '71.620'), (1, '81.100')] [2023-10-08 07:00:59,053][00611] Updated weights for policy 0, policy_version 79052 (0.0010) [2023-10-08 07:00:59,437][00611] Updated weights for policy 0, policy_version 79062 (0.0010) [2023-10-08 07:00:59,795][00611] Updated weights for policy 0, policy_version 79072 (0.0009) [2023-10-08 07:01:01,360][00612] Updated weights for policy 1, policy_version 79490 (0.0010) [2023-10-08 07:01:01,729][00612] Updated weights for policy 1, policy_version 79500 (0.0009) [2023-10-08 07:01:02,089][00612] Updated weights for policy 1, policy_version 79510 (0.0008) [2023-10-08 07:01:02,456][00612] Updated weights for policy 1, policy_version 79520 (0.0007) [2023-10-08 07:01:03,584][00611] Updated weights for policy 0, policy_version 79082 (0.0008) [2023-10-08 07:01:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 162398208. Throughput: 0: 1851.8, 1: 1852.8. Samples: 40612452. Policy #0 lag: (min: 22.0, avg: 26.8, max: 54.0) [2023-10-08 07:01:03,754][130385] Avg episode reward: [(0, '66.480'), (1, '78.670')] [2023-10-08 07:01:03,959][00611] Updated weights for policy 0, policy_version 79092 (0.0007) [2023-10-08 07:01:04,333][00611] Updated weights for policy 0, policy_version 79102 (0.0008) [2023-10-08 07:01:06,016][00612] Updated weights for policy 1, policy_version 79530 (0.0009) [2023-10-08 07:01:06,377][00612] Updated weights for policy 1, policy_version 79540 (0.0010) [2023-10-08 07:01:06,754][00612] Updated weights for policy 1, policy_version 79550 (0.0007) [2023-10-08 07:01:07,855][00611] Updated weights for policy 0, policy_version 79112 (0.0010) [2023-10-08 07:01:08,223][00611] Updated weights for policy 0, policy_version 79122 (0.0007) [2023-10-08 07:01:08,593][00611] Updated weights for policy 0, policy_version 79132 (0.0007) [2023-10-08 07:01:08,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14884.5). Total num frames: 162496512. Throughput: 0: 1854.6, 1: 1835.3. Samples: 40623506. Policy #0 lag: (min: 22.0, avg: 26.8, max: 54.0) [2023-10-08 07:01:08,754][130385] Avg episode reward: [(0, '73.150'), (1, '79.040')] [2023-10-08 07:01:10,400][00612] Updated weights for policy 1, policy_version 79560 (0.0007) [2023-10-08 07:01:10,762][00612] Updated weights for policy 1, policy_version 79570 (0.0008) [2023-10-08 07:01:11,131][00612] Updated weights for policy 1, policy_version 79580 (0.0007) [2023-10-08 07:01:12,207][00611] Updated weights for policy 0, policy_version 79142 (0.0007) [2023-10-08 07:01:12,576][00611] Updated weights for policy 0, policy_version 79152 (0.0007) [2023-10-08 07:01:12,953][00611] Updated weights for policy 0, policy_version 79162 (0.0008) [2023-10-08 07:01:13,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 162562048. Throughput: 0: 1847.4, 1: 1850.8. Samples: 40645604. Policy #0 lag: (min: 22.0, avg: 26.8, max: 54.0) [2023-10-08 07:01:13,759][130385] Avg episode reward: [(0, '72.390'), (1, '79.450')] [2023-10-08 07:01:14,839][00612] Updated weights for policy 1, policy_version 79590 (0.0008) [2023-10-08 07:01:15,203][00612] Updated weights for policy 1, policy_version 79600 (0.0008) [2023-10-08 07:01:15,575][00612] Updated weights for policy 1, policy_version 79610 (0.0011) [2023-10-08 07:01:16,615][00611] Updated weights for policy 0, policy_version 79172 (0.0007) [2023-10-08 07:01:16,985][00611] Updated weights for policy 0, policy_version 79182 (0.0010) [2023-10-08 07:01:17,362][00611] Updated weights for policy 0, policy_version 79192 (0.0010) [2023-10-08 07:01:18,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 162627584. Throughput: 0: 1844.0, 1: 1851.2. Samples: 40667408. Policy #0 lag: (min: 22.0, avg: 26.8, max: 54.0) [2023-10-08 07:01:18,755][130385] Avg episode reward: [(0, '72.560'), (1, '79.900')] [2023-10-08 07:01:19,279][00612] Updated weights for policy 1, policy_version 79620 (0.0009) [2023-10-08 07:01:19,645][00612] Updated weights for policy 1, policy_version 79630 (0.0008) [2023-10-08 07:01:20,016][00612] Updated weights for policy 1, policy_version 79640 (0.0009) [2023-10-08 07:01:20,952][00611] Updated weights for policy 0, policy_version 79202 (0.0009) [2023-10-08 07:01:21,323][00611] Updated weights for policy 0, policy_version 79212 (0.0008) [2023-10-08 07:01:21,693][00611] Updated weights for policy 0, policy_version 79222 (0.0008) [2023-10-08 07:01:22,059][00611] Updated weights for policy 0, policy_version 79232 (0.0008) [2023-10-08 07:01:23,542][00612] Updated weights for policy 1, policy_version 79650 (0.0007) [2023-10-08 07:01:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 162693120. Throughput: 0: 1840.7, 1: 1851.5. Samples: 40678636. Policy #0 lag: (min: 22.0, avg: 26.8, max: 54.0) [2023-10-08 07:01:23,754][130385] Avg episode reward: [(0, '76.340'), (1, '80.070')] [2023-10-08 07:01:23,914][00612] Updated weights for policy 1, policy_version 79660 (0.0007) [2023-10-08 07:01:24,286][00612] Updated weights for policy 1, policy_version 79670 (0.0008) [2023-10-08 07:01:24,659][00612] Updated weights for policy 1, policy_version 79680 (0.0007) [2023-10-08 07:01:25,897][00611] Updated weights for policy 0, policy_version 79242 (0.0008) [2023-10-08 07:01:26,268][00611] Updated weights for policy 0, policy_version 79252 (0.0008) [2023-10-08 07:01:26,642][00611] Updated weights for policy 0, policy_version 79262 (0.0009) [2023-10-08 07:01:28,130][00612] Updated weights for policy 1, policy_version 79690 (0.0009) [2023-10-08 07:01:28,503][00612] Updated weights for policy 1, policy_version 79700 (0.0008) [2023-10-08 07:01:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 162758656. Throughput: 0: 1840.4, 1: 1855.1. Samples: 40700618. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 07:01:28,754][130385] Avg episode reward: [(0, '75.720'), (1, '79.050')] [2023-10-08 07:01:28,864][00612] Updated weights for policy 1, policy_version 79710 (0.0010) [2023-10-08 07:01:30,173][00611] Updated weights for policy 0, policy_version 79272 (0.0009) [2023-10-08 07:01:30,543][00611] Updated weights for policy 0, policy_version 79282 (0.0008) [2023-10-08 07:01:30,915][00611] Updated weights for policy 0, policy_version 79292 (0.0008) [2023-10-08 07:01:32,529][00612] Updated weights for policy 1, policy_version 79720 (0.0008) [2023-10-08 07:01:32,896][00612] Updated weights for policy 1, policy_version 79730 (0.0008) [2023-10-08 07:01:33,272][00612] Updated weights for policy 1, policy_version 79740 (0.0009) [2023-10-08 07:01:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 162856960. Throughput: 0: 1845.3, 1: 1831.5. Samples: 40722800. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 07:01:33,754][130385] Avg episode reward: [(0, '77.740'), (1, '79.630')] [2023-10-08 07:01:34,500][00611] Updated weights for policy 0, policy_version 79302 (0.0011) [2023-10-08 07:01:34,883][00611] Updated weights for policy 0, policy_version 79312 (0.0009) [2023-10-08 07:01:35,247][00611] Updated weights for policy 0, policy_version 79322 (0.0010) [2023-10-08 07:01:36,966][00612] Updated weights for policy 1, policy_version 79750 (0.0009) [2023-10-08 07:01:37,344][00612] Updated weights for policy 1, policy_version 79760 (0.0007) [2023-10-08 07:01:37,721][00612] Updated weights for policy 1, policy_version 79770 (0.0007) [2023-10-08 07:01:38,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 162922496. Throughput: 0: 1846.0, 1: 1847.7. Samples: 40733984. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 07:01:38,754][130385] Avg episode reward: [(0, '78.070'), (1, '79.840')] [2023-10-08 07:01:38,848][00611] Updated weights for policy 0, policy_version 79332 (0.0010) [2023-10-08 07:01:39,225][00611] Updated weights for policy 0, policy_version 79342 (0.0008) [2023-10-08 07:01:39,598][00611] Updated weights for policy 0, policy_version 79352 (0.0008) [2023-10-08 07:01:41,351][00612] Updated weights for policy 1, policy_version 79780 (0.0008) [2023-10-08 07:01:41,713][00612] Updated weights for policy 1, policy_version 79790 (0.0010) [2023-10-08 07:01:42,086][00612] Updated weights for policy 1, policy_version 79800 (0.0009) [2023-10-08 07:01:43,362][00611] Updated weights for policy 0, policy_version 79362 (0.0007) [2023-10-08 07:01:43,738][00611] Updated weights for policy 0, policy_version 79372 (0.0009) [2023-10-08 07:01:43,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 162988032. Throughput: 0: 1840.5, 1: 1832.0. Samples: 40755820. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 07:01:43,755][130385] Avg episode reward: [(0, '76.190'), (1, '80.060')] [2023-10-08 07:01:44,103][00611] Updated weights for policy 0, policy_version 79382 (0.0009) [2023-10-08 07:01:44,477][00611] Updated weights for policy 0, policy_version 79392 (0.0010) [2023-10-08 07:01:45,624][00612] Updated weights for policy 1, policy_version 79810 (0.0010) [2023-10-08 07:01:45,995][00612] Updated weights for policy 1, policy_version 79820 (0.0010) [2023-10-08 07:01:46,375][00612] Updated weights for policy 1, policy_version 79830 (0.0011) [2023-10-08 07:01:46,749][00612] Updated weights for policy 1, policy_version 79840 (0.0011) [2023-10-08 07:01:48,161][00611] Updated weights for policy 0, policy_version 79402 (0.0007) [2023-10-08 07:01:48,525][00611] Updated weights for policy 0, policy_version 79412 (0.0008) [2023-10-08 07:01:48,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 163053568. Throughput: 0: 1824.5, 1: 1855.9. Samples: 40778068. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 07:01:48,755][130385] Avg episode reward: [(0, '73.560'), (1, '80.320')] [2023-10-08 07:01:48,895][00611] Updated weights for policy 0, policy_version 79422 (0.0008) [2023-10-08 07:01:50,463][00612] Updated weights for policy 1, policy_version 79850 (0.0008) [2023-10-08 07:01:50,830][00612] Updated weights for policy 1, policy_version 79860 (0.0009) [2023-10-08 07:01:51,188][00612] Updated weights for policy 1, policy_version 79870 (0.0010) [2023-10-08 07:01:52,498][00611] Updated weights for policy 0, policy_version 79432 (0.0008) [2023-10-08 07:01:52,876][00611] Updated weights for policy 0, policy_version 79442 (0.0007) [2023-10-08 07:01:53,250][00611] Updated weights for policy 0, policy_version 79452 (0.0007) [2023-10-08 07:01:53,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 163151872. Throughput: 0: 1835.9, 1: 1838.6. Samples: 40788858. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 07:01:53,755][130385] Avg episode reward: [(0, '73.580'), (1, '78.260')] [2023-10-08 07:01:54,811][00612] Updated weights for policy 1, policy_version 79880 (0.0009) [2023-10-08 07:01:55,172][00612] Updated weights for policy 1, policy_version 79890 (0.0008) [2023-10-08 07:01:55,552][00612] Updated weights for policy 1, policy_version 79900 (0.0009) [2023-10-08 07:01:56,987][00611] Updated weights for policy 0, policy_version 79462 (0.0007) [2023-10-08 07:01:57,353][00611] Updated weights for policy 0, policy_version 79472 (0.0009) [2023-10-08 07:01:57,726][00611] Updated weights for policy 0, policy_version 79482 (0.0008) [2023-10-08 07:01:58,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 163217408. Throughput: 0: 1820.8, 1: 1858.6. Samples: 40811180. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 07:01:58,755][130385] Avg episode reward: [(0, '75.500'), (1, '75.710')] [2023-10-08 07:01:59,123][00612] Updated weights for policy 1, policy_version 79910 (0.0008) [2023-10-08 07:01:59,489][00612] Updated weights for policy 1, policy_version 79920 (0.0008) [2023-10-08 07:01:59,850][00612] Updated weights for policy 1, policy_version 79930 (0.0008) [2023-10-08 07:02:01,393][00611] Updated weights for policy 0, policy_version 79492 (0.0008) [2023-10-08 07:02:01,770][00611] Updated weights for policy 0, policy_version 79502 (0.0010) [2023-10-08 07:02:02,133][00611] Updated weights for policy 0, policy_version 79512 (0.0009) [2023-10-08 07:02:03,481][00612] Updated weights for policy 1, policy_version 79940 (0.0008) [2023-10-08 07:02:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 163282944. Throughput: 0: 1829.9, 1: 1857.2. Samples: 40833328. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 07:02:03,755][130385] Avg episode reward: [(0, '74.830'), (1, '78.930')] [2023-10-08 07:02:03,874][00612] Updated weights for policy 1, policy_version 79950 (0.0007) [2023-10-08 07:02:04,237][00612] Updated weights for policy 1, policy_version 79960 (0.0008) [2023-10-08 07:02:05,801][00611] Updated weights for policy 0, policy_version 79522 (0.0007) [2023-10-08 07:02:06,173][00611] Updated weights for policy 0, policy_version 79532 (0.0008) [2023-10-08 07:02:06,546][00611] Updated weights for policy 0, policy_version 79542 (0.0008) [2023-10-08 07:02:06,909][00611] Updated weights for policy 0, policy_version 79552 (0.0007) [2023-10-08 07:02:07,792][00612] Updated weights for policy 1, policy_version 79970 (0.0008) [2023-10-08 07:02:08,158][00612] Updated weights for policy 1, policy_version 79980 (0.0009) [2023-10-08 07:02:08,534][00612] Updated weights for policy 1, policy_version 79990 (0.0010) [2023-10-08 07:02:08,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 163348480. Throughput: 0: 1826.7, 1: 1856.3. Samples: 40844370. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 07:02:08,754][130385] Avg episode reward: [(0, '76.960'), (1, '78.580')] [2023-10-08 07:02:08,889][00612] Updated weights for policy 1, policy_version 80000 (0.0009) [2023-10-08 07:02:10,571][00611] Updated weights for policy 0, policy_version 79562 (0.0007) [2023-10-08 07:02:10,951][00611] Updated weights for policy 0, policy_version 79572 (0.0007) [2023-10-08 07:02:11,322][00611] Updated weights for policy 0, policy_version 79582 (0.0008) [2023-10-08 07:02:12,635][00612] Updated weights for policy 1, policy_version 80010 (0.0007) [2023-10-08 07:02:13,010][00612] Updated weights for policy 1, policy_version 80020 (0.0009) [2023-10-08 07:02:13,377][00612] Updated weights for policy 1, policy_version 80030 (0.0007) [2023-10-08 07:02:13,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 163446784. Throughput: 0: 1837.8, 1: 1850.3. Samples: 40866580. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 07:02:13,755][130385] Avg episode reward: [(0, '77.300'), (1, '78.770')] [2023-10-08 07:02:14,845][00611] Updated weights for policy 0, policy_version 79592 (0.0009) [2023-10-08 07:02:15,208][00611] Updated weights for policy 0, policy_version 79602 (0.0008) [2023-10-08 07:02:15,579][00611] Updated weights for policy 0, policy_version 79612 (0.0009) [2023-10-08 07:02:17,019][00612] Updated weights for policy 1, policy_version 80040 (0.0009) [2023-10-08 07:02:17,382][00612] Updated weights for policy 1, policy_version 80050 (0.0007) [2023-10-08 07:02:17,750][00612] Updated weights for policy 1, policy_version 80060 (0.0008) [2023-10-08 07:02:18,754][130385] Fps is (10 sec: 16383.3, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 163512320. Throughput: 0: 1832.8, 1: 1842.7. Samples: 40888200. Policy #0 lag: (min: 31.0, avg: 44.5, max: 63.0) [2023-10-08 07:02:18,755][130385] Avg episode reward: [(0, '74.380'), (1, '78.510')] [2023-10-08 07:02:18,767][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000080064_81985536.pth... [2023-10-08 07:02:18,767][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000079616_81526784.pth... [2023-10-08 07:02:18,805][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000078336_80216064.pth [2023-10-08 07:02:18,809][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000077920_79790080.pth [2023-10-08 07:02:19,293][00611] Updated weights for policy 0, policy_version 79622 (0.0007) [2023-10-08 07:02:19,656][00611] Updated weights for policy 0, policy_version 79632 (0.0007) [2023-10-08 07:02:20,035][00611] Updated weights for policy 0, policy_version 79642 (0.0007) [2023-10-08 07:02:21,433][00612] Updated weights for policy 1, policy_version 80070 (0.0009) [2023-10-08 07:02:21,796][00612] Updated weights for policy 1, policy_version 80080 (0.0011) [2023-10-08 07:02:22,160][00612] Updated weights for policy 1, policy_version 80090 (0.0009) [2023-10-08 07:02:23,687][00611] Updated weights for policy 0, policy_version 79652 (0.0009) [2023-10-08 07:02:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 163577856. Throughput: 0: 1830.9, 1: 1852.6. Samples: 40899740. Policy #0 lag: (min: 31.0, avg: 44.5, max: 63.0) [2023-10-08 07:02:23,754][130385] Avg episode reward: [(0, '73.720'), (1, '78.600')] [2023-10-08 07:02:24,058][00611] Updated weights for policy 0, policy_version 79662 (0.0009) [2023-10-08 07:02:24,440][00611] Updated weights for policy 0, policy_version 79672 (0.0008) [2023-10-08 07:02:25,726][00612] Updated weights for policy 1, policy_version 80100 (0.0009) [2023-10-08 07:02:26,105][00612] Updated weights for policy 1, policy_version 80110 (0.0008) [2023-10-08 07:02:26,468][00612] Updated weights for policy 1, policy_version 80120 (0.0007) [2023-10-08 07:02:27,933][00611] Updated weights for policy 0, policy_version 79682 (0.0007) [2023-10-08 07:02:28,297][00611] Updated weights for policy 0, policy_version 79692 (0.0007) [2023-10-08 07:02:28,667][00611] Updated weights for policy 0, policy_version 79702 (0.0008) [2023-10-08 07:02:28,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 163643392. Throughput: 0: 1838.1, 1: 1846.0. Samples: 40921600. Policy #0 lag: (min: 31.0, avg: 44.5, max: 63.0) [2023-10-08 07:02:28,754][130385] Avg episode reward: [(0, '72.420'), (1, '81.620')] [2023-10-08 07:02:29,040][00611] Updated weights for policy 0, policy_version 79712 (0.0008) [2023-10-08 07:02:29,932][00612] Updated weights for policy 1, policy_version 80130 (0.0007) [2023-10-08 07:02:30,296][00612] Updated weights for policy 1, policy_version 80140 (0.0010) [2023-10-08 07:02:30,663][00612] Updated weights for policy 1, policy_version 80150 (0.0011) [2023-10-08 07:02:31,028][00612] Updated weights for policy 1, policy_version 80160 (0.0009) [2023-10-08 07:02:32,707][00611] Updated weights for policy 0, policy_version 79722 (0.0007) [2023-10-08 07:02:33,074][00611] Updated weights for policy 0, policy_version 79732 (0.0007) [2023-10-08 07:02:33,445][00611] Updated weights for policy 0, policy_version 79742 (0.0007) [2023-10-08 07:02:33,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 163741696. Throughput: 0: 1838.0, 1: 1860.2. Samples: 40944490. Policy #0 lag: (min: 31.0, avg: 44.5, max: 63.0) [2023-10-08 07:02:33,755][130385] Avg episode reward: [(0, '71.180'), (1, '83.030')] [2023-10-08 07:02:34,621][00612] Updated weights for policy 1, policy_version 80170 (0.0008) [2023-10-08 07:02:34,986][00612] Updated weights for policy 1, policy_version 80180 (0.0008) [2023-10-08 07:02:35,357][00612] Updated weights for policy 1, policy_version 80190 (0.0011) [2023-10-08 07:02:37,250][00611] Updated weights for policy 0, policy_version 79752 (0.0008) [2023-10-08 07:02:37,626][00611] Updated weights for policy 0, policy_version 79762 (0.0007) [2023-10-08 07:02:37,988][00611] Updated weights for policy 0, policy_version 79772 (0.0007) [2023-10-08 07:02:38,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 163807232. Throughput: 0: 1847.2, 1: 1854.0. Samples: 40955410. Policy #0 lag: (min: 31.0, avg: 44.5, max: 63.0) [2023-10-08 07:02:38,755][130385] Avg episode reward: [(0, '67.820'), (1, '80.240')] [2023-10-08 07:02:38,891][00612] Updated weights for policy 1, policy_version 80200 (0.0010) [2023-10-08 07:02:39,258][00612] Updated weights for policy 1, policy_version 80210 (0.0009) [2023-10-08 07:02:39,628][00612] Updated weights for policy 1, policy_version 80220 (0.0009) [2023-10-08 07:02:41,649][00611] Updated weights for policy 0, policy_version 79782 (0.0009) [2023-10-08 07:02:42,025][00611] Updated weights for policy 0, policy_version 79792 (0.0008) [2023-10-08 07:02:42,390][00611] Updated weights for policy 0, policy_version 79802 (0.0008) [2023-10-08 07:02:43,290][00612] Updated weights for policy 1, policy_version 80230 (0.0010) [2023-10-08 07:02:43,672][00612] Updated weights for policy 1, policy_version 80240 (0.0011) [2023-10-08 07:02:43,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 163872768. Throughput: 0: 1837.4, 1: 1864.3. Samples: 40977756. Policy #0 lag: (min: 31.0, avg: 44.5, max: 63.0) [2023-10-08 07:02:43,754][130385] Avg episode reward: [(0, '67.710'), (1, '78.850')] [2023-10-08 07:02:44,036][00612] Updated weights for policy 1, policy_version 80250 (0.0009) [2023-10-08 07:02:45,995][00611] Updated weights for policy 0, policy_version 79812 (0.0008) [2023-10-08 07:02:46,382][00611] Updated weights for policy 0, policy_version 79822 (0.0010) [2023-10-08 07:02:46,753][00611] Updated weights for policy 0, policy_version 79832 (0.0008) [2023-10-08 07:02:47,535][00612] Updated weights for policy 1, policy_version 80260 (0.0009) [2023-10-08 07:02:47,900][00612] Updated weights for policy 1, policy_version 80270 (0.0007) [2023-10-08 07:02:48,275][00612] Updated weights for policy 1, policy_version 80280 (0.0007) [2023-10-08 07:02:48,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 163971072. Throughput: 0: 1847.8, 1: 1840.5. Samples: 40999302. Policy #0 lag: (min: 31.0, avg: 44.5, max: 63.0) [2023-10-08 07:02:48,755][130385] Avg episode reward: [(0, '71.340'), (1, '75.870')] [2023-10-08 07:02:50,236][00611] Updated weights for policy 0, policy_version 79842 (0.0010) [2023-10-08 07:02:50,612][00611] Updated weights for policy 0, policy_version 79852 (0.0007) [2023-10-08 07:02:50,981][00611] Updated weights for policy 0, policy_version 79862 (0.0009) [2023-10-08 07:02:51,349][00611] Updated weights for policy 0, policy_version 79872 (0.0009) [2023-10-08 07:02:51,826][00612] Updated weights for policy 1, policy_version 80290 (0.0007) [2023-10-08 07:02:52,194][00612] Updated weights for policy 1, policy_version 80300 (0.0008) [2023-10-08 07:02:52,567][00612] Updated weights for policy 1, policy_version 80310 (0.0007) [2023-10-08 07:02:52,932][00612] Updated weights for policy 1, policy_version 80320 (0.0008) [2023-10-08 07:02:53,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 164036608. Throughput: 0: 1829.2, 1: 1865.6. Samples: 41010640. Policy #0 lag: (min: 31.0, avg: 44.5, max: 63.0) [2023-10-08 07:02:53,755][130385] Avg episode reward: [(0, '74.590'), (1, '75.400')] [2023-10-08 07:02:54,981][00611] Updated weights for policy 0, policy_version 79882 (0.0007) [2023-10-08 07:02:55,351][00611] Updated weights for policy 0, policy_version 79892 (0.0010) [2023-10-08 07:02:55,718][00611] Updated weights for policy 0, policy_version 79902 (0.0008) [2023-10-08 07:02:56,579][00612] Updated weights for policy 1, policy_version 80330 (0.0007) [2023-10-08 07:02:56,949][00612] Updated weights for policy 1, policy_version 80340 (0.0008) [2023-10-08 07:02:57,319][00612] Updated weights for policy 1, policy_version 80350 (0.0007) [2023-10-08 07:02:58,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 164102144. Throughput: 0: 1841.8, 1: 1843.0. Samples: 41032394. Policy #0 lag: (min: 31.0, avg: 44.5, max: 63.0) [2023-10-08 07:02:58,754][130385] Avg episode reward: [(0, '75.520'), (1, '74.430')] [2023-10-08 07:02:59,308][00611] Updated weights for policy 0, policy_version 79912 (0.0008) [2023-10-08 07:02:59,685][00611] Updated weights for policy 0, policy_version 79922 (0.0008) [2023-10-08 07:03:00,061][00611] Updated weights for policy 0, policy_version 79932 (0.0009) [2023-10-08 07:03:00,795][00612] Updated weights for policy 1, policy_version 80360 (0.0008) [2023-10-08 07:03:01,165][00612] Updated weights for policy 1, policy_version 80370 (0.0008) [2023-10-08 07:03:01,524][00612] Updated weights for policy 1, policy_version 80380 (0.0008) [2023-10-08 07:03:03,720][00611] Updated weights for policy 0, policy_version 79942 (0.0009) [2023-10-08 07:03:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 164167680. Throughput: 0: 1840.1, 1: 1878.7. Samples: 41055544. Policy #0 lag: (min: 31.0, avg: 44.5, max: 63.0) [2023-10-08 07:03:03,754][130385] Avg episode reward: [(0, '73.650'), (1, '74.230')] [2023-10-08 07:03:04,087][00611] Updated weights for policy 0, policy_version 79952 (0.0011) [2023-10-08 07:03:04,458][00611] Updated weights for policy 0, policy_version 79962 (0.0009) [2023-10-08 07:03:04,998][00612] Updated weights for policy 1, policy_version 80390 (0.0010) [2023-10-08 07:03:05,364][00612] Updated weights for policy 1, policy_version 80400 (0.0008) [2023-10-08 07:03:05,737][00612] Updated weights for policy 1, policy_version 80410 (0.0009) [2023-10-08 07:03:08,193][00611] Updated weights for policy 0, policy_version 79972 (0.0009) [2023-10-08 07:03:08,560][00611] Updated weights for policy 0, policy_version 79982 (0.0010) [2023-10-08 07:03:08,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 164233216. Throughput: 0: 1844.5, 1: 1843.2. Samples: 41065690. Policy #0 lag: (min: 31.0, avg: 44.5, max: 63.0) [2023-10-08 07:03:08,755][130385] Avg episode reward: [(0, '71.010'), (1, '70.730')] [2023-10-08 07:03:08,943][00611] Updated weights for policy 0, policy_version 79992 (0.0010) [2023-10-08 07:03:09,304][00612] Updated weights for policy 1, policy_version 80420 (0.0008) [2023-10-08 07:03:09,673][00612] Updated weights for policy 1, policy_version 80430 (0.0008) [2023-10-08 07:03:10,030][00612] Updated weights for policy 1, policy_version 80440 (0.0008) [2023-10-08 07:03:12,561][00611] Updated weights for policy 0, policy_version 80002 (0.0009) [2023-10-08 07:03:12,937][00611] Updated weights for policy 0, policy_version 80012 (0.0010) [2023-10-08 07:03:13,307][00611] Updated weights for policy 0, policy_version 80022 (0.0009) [2023-10-08 07:03:13,640][00612] Updated weights for policy 1, policy_version 80450 (0.0010) [2023-10-08 07:03:13,680][00611] Updated weights for policy 0, policy_version 80032 (0.0007) [2023-10-08 07:03:13,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 164331520. Throughput: 0: 1836.3, 1: 1880.9. Samples: 41088876. Policy #0 lag: (min: 5.0, avg: 11.9, max: 37.0) [2023-10-08 07:03:13,755][130385] Avg episode reward: [(0, '69.080'), (1, '70.090')] [2023-10-08 07:03:14,008][00612] Updated weights for policy 1, policy_version 80460 (0.0008) [2023-10-08 07:03:14,371][00612] Updated weights for policy 1, policy_version 80470 (0.0011) [2023-10-08 07:03:14,739][00612] Updated weights for policy 1, policy_version 80480 (0.0010) [2023-10-08 07:03:17,359][00611] Updated weights for policy 0, policy_version 80042 (0.0007) [2023-10-08 07:03:17,733][00611] Updated weights for policy 0, policy_version 80052 (0.0009) [2023-10-08 07:03:18,100][00611] Updated weights for policy 0, policy_version 80062 (0.0010) [2023-10-08 07:03:18,505][00612] Updated weights for policy 1, policy_version 80490 (0.0011) [2023-10-08 07:03:18,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 164397056. Throughput: 0: 1819.6, 1: 1869.9. Samples: 41110518. Policy #0 lag: (min: 5.0, avg: 11.9, max: 37.0) [2023-10-08 07:03:18,754][130385] Avg episode reward: [(0, '67.470'), (1, '73.950')] [2023-10-08 07:03:18,882][00612] Updated weights for policy 1, policy_version 80500 (0.0008) [2023-10-08 07:03:19,241][00612] Updated weights for policy 1, policy_version 80510 (0.0008) [2023-10-08 07:03:21,878][00611] Updated weights for policy 0, policy_version 80072 (0.0010) [2023-10-08 07:03:22,249][00611] Updated weights for policy 0, policy_version 80082 (0.0009) [2023-10-08 07:03:22,618][00611] Updated weights for policy 0, policy_version 80092 (0.0008) [2023-10-08 07:03:22,694][00612] Updated weights for policy 1, policy_version 80520 (0.0009) [2023-10-08 07:03:23,062][00612] Updated weights for policy 1, policy_version 80530 (0.0008) [2023-10-08 07:03:23,431][00612] Updated weights for policy 1, policy_version 80540 (0.0007) [2023-10-08 07:03:23,754][130385] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 164495360. Throughput: 0: 1833.1, 1: 1873.4. Samples: 41122204. Policy #0 lag: (min: 5.0, avg: 11.9, max: 37.0) [2023-10-08 07:03:23,754][130385] Avg episode reward: [(0, '70.420'), (1, '77.150')] [2023-10-08 07:03:26,232][00611] Updated weights for policy 0, policy_version 80102 (0.0008) [2023-10-08 07:03:26,600][00611] Updated weights for policy 0, policy_version 80112 (0.0007) [2023-10-08 07:03:26,977][00611] Updated weights for policy 0, policy_version 80122 (0.0009) [2023-10-08 07:03:27,092][00612] Updated weights for policy 1, policy_version 80550 (0.0009) [2023-10-08 07:03:27,460][00612] Updated weights for policy 1, policy_version 80560 (0.0007) [2023-10-08 07:03:27,825][00612] Updated weights for policy 1, policy_version 80570 (0.0010) [2023-10-08 07:03:28,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 164560896. Throughput: 0: 1825.3, 1: 1857.9. Samples: 41143502. Policy #0 lag: (min: 5.0, avg: 11.9, max: 37.0) [2023-10-08 07:03:28,754][130385] Avg episode reward: [(0, '73.270'), (1, '75.690')] [2023-10-08 07:03:30,751][00611] Updated weights for policy 0, policy_version 80132 (0.0010) [2023-10-08 07:03:31,143][00611] Updated weights for policy 0, policy_version 80142 (0.0008) [2023-10-08 07:03:31,519][00611] Updated weights for policy 0, policy_version 80152 (0.0007) [2023-10-08 07:03:31,603][00612] Updated weights for policy 1, policy_version 80580 (0.0010) [2023-10-08 07:03:31,970][00612] Updated weights for policy 1, policy_version 80590 (0.0011) [2023-10-08 07:03:32,341][00612] Updated weights for policy 1, policy_version 80600 (0.0009) [2023-10-08 07:03:33,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 164626432. Throughput: 0: 1833.7, 1: 1850.9. Samples: 41165110. Policy #0 lag: (min: 5.0, avg: 11.9, max: 37.0) [2023-10-08 07:03:33,755][130385] Avg episode reward: [(0, '76.040'), (1, '77.290')] [2023-10-08 07:03:34,901][00611] Updated weights for policy 0, policy_version 80162 (0.0008) [2023-10-08 07:03:35,279][00611] Updated weights for policy 0, policy_version 80172 (0.0008) [2023-10-08 07:03:35,642][00611] Updated weights for policy 0, policy_version 80182 (0.0009) [2023-10-08 07:03:36,013][00611] Updated weights for policy 0, policy_version 80192 (0.0008) [2023-10-08 07:03:36,096][00612] Updated weights for policy 1, policy_version 80610 (0.0009) [2023-10-08 07:03:36,466][00612] Updated weights for policy 1, policy_version 80620 (0.0008) [2023-10-08 07:03:36,827][00612] Updated weights for policy 1, policy_version 80630 (0.0007) [2023-10-08 07:03:37,196][00612] Updated weights for policy 1, policy_version 80640 (0.0008) [2023-10-08 07:03:38,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 164691968. Throughput: 0: 1828.3, 1: 1855.9. Samples: 41176428. Policy #0 lag: (min: 5.0, avg: 11.9, max: 37.0) [2023-10-08 07:03:38,755][130385] Avg episode reward: [(0, '75.060'), (1, '75.310')] [2023-10-08 07:03:39,660][00611] Updated weights for policy 0, policy_version 80202 (0.0010) [2023-10-08 07:03:40,028][00611] Updated weights for policy 0, policy_version 80212 (0.0010) [2023-10-08 07:03:40,394][00611] Updated weights for policy 0, policy_version 80222 (0.0008) [2023-10-08 07:03:40,740][00612] Updated weights for policy 1, policy_version 80650 (0.0007) [2023-10-08 07:03:41,108][00612] Updated weights for policy 1, policy_version 80660 (0.0010) [2023-10-08 07:03:41,484][00612] Updated weights for policy 1, policy_version 80670 (0.0009) [2023-10-08 07:03:43,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 164757504. Throughput: 0: 1835.1, 1: 1849.0. Samples: 41198176. Policy #0 lag: (min: 5.0, avg: 11.9, max: 37.0) [2023-10-08 07:03:43,755][130385] Avg episode reward: [(0, '82.410'), (1, '73.970')] [2023-10-08 07:03:44,083][00611] Updated weights for policy 0, policy_version 80232 (0.0010) [2023-10-08 07:03:44,462][00611] Updated weights for policy 0, policy_version 80242 (0.0009) [2023-10-08 07:03:44,835][00611] Updated weights for policy 0, policy_version 80252 (0.0009) [2023-10-08 07:03:44,983][00365] Saving new best policy, reward=82.410! [2023-10-08 07:03:45,172][00612] Updated weights for policy 1, policy_version 80680 (0.0010) [2023-10-08 07:03:45,542][00612] Updated weights for policy 1, policy_version 80690 (0.0009) [2023-10-08 07:03:45,909][00612] Updated weights for policy 1, policy_version 80700 (0.0007) [2023-10-08 07:03:48,610][00611] Updated weights for policy 0, policy_version 80262 (0.0009) [2023-10-08 07:03:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14773.4). Total num frames: 164823040. Throughput: 0: 1835.1, 1: 1846.7. Samples: 41221226. Policy #0 lag: (min: 5.0, avg: 11.9, max: 37.0) [2023-10-08 07:03:48,755][130385] Avg episode reward: [(0, '83.250'), (1, '77.970')] [2023-10-08 07:03:48,977][00611] Updated weights for policy 0, policy_version 80272 (0.0008) [2023-10-08 07:03:49,348][00611] Updated weights for policy 0, policy_version 80282 (0.0007) [2023-10-08 07:03:49,562][00365] Saving new best policy, reward=83.250! [2023-10-08 07:03:49,578][00612] Updated weights for policy 1, policy_version 80710 (0.0009) [2023-10-08 07:03:49,943][00612] Updated weights for policy 1, policy_version 80720 (0.0007) [2023-10-08 07:03:50,311][00612] Updated weights for policy 1, policy_version 80730 (0.0008) [2023-10-08 07:03:52,959][00611] Updated weights for policy 0, policy_version 80292 (0.0009) [2023-10-08 07:03:53,330][00611] Updated weights for policy 0, policy_version 80302 (0.0010) [2023-10-08 07:03:53,711][00611] Updated weights for policy 0, policy_version 80312 (0.0008) [2023-10-08 07:03:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 164888576. Throughput: 0: 1834.4, 1: 1843.5. Samples: 41231196. Policy #0 lag: (min: 5.0, avg: 11.9, max: 37.0) [2023-10-08 07:03:53,754][130385] Avg episode reward: [(0, '85.470'), (1, '79.710')] [2023-10-08 07:03:53,880][00612] Updated weights for policy 1, policy_version 80740 (0.0007) [2023-10-08 07:03:54,001][00365] Saving new best policy, reward=85.470! [2023-10-08 07:03:54,248][00612] Updated weights for policy 1, policy_version 80750 (0.0009) [2023-10-08 07:03:54,619][00612] Updated weights for policy 1, policy_version 80760 (0.0009) [2023-10-08 07:03:57,273][00611] Updated weights for policy 0, policy_version 80322 (0.0008) [2023-10-08 07:03:57,637][00611] Updated weights for policy 0, policy_version 80332 (0.0008) [2023-10-08 07:03:58,017][00611] Updated weights for policy 0, policy_version 80342 (0.0007) [2023-10-08 07:03:58,215][00612] Updated weights for policy 1, policy_version 80770 (0.0008) [2023-10-08 07:03:58,391][00611] Updated weights for policy 0, policy_version 80352 (0.0009) [2023-10-08 07:03:58,580][00612] Updated weights for policy 1, policy_version 80780 (0.0010) [2023-10-08 07:03:58,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 164986880. Throughput: 0: 1838.4, 1: 1841.1. Samples: 41254454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:03:58,755][130385] Avg episode reward: [(0, '84.340'), (1, '79.590')] [2023-10-08 07:03:58,953][00612] Updated weights for policy 1, policy_version 80790 (0.0012) [2023-10-08 07:03:59,319][00612] Updated weights for policy 1, policy_version 80800 (0.0009) [2023-10-08 07:04:02,110][00611] Updated weights for policy 0, policy_version 80362 (0.0007) [2023-10-08 07:04:02,483][00611] Updated weights for policy 0, policy_version 80372 (0.0007) [2023-10-08 07:04:02,854][00611] Updated weights for policy 0, policy_version 80382 (0.0008) [2023-10-08 07:04:02,966][00612] Updated weights for policy 1, policy_version 80810 (0.0009) [2023-10-08 07:04:03,331][00612] Updated weights for policy 1, policy_version 80820 (0.0007) [2023-10-08 07:04:03,702][00612] Updated weights for policy 1, policy_version 80830 (0.0007) [2023-10-08 07:04:03,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 165052416. Throughput: 0: 1829.4, 1: 1827.3. Samples: 41275068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:04:03,755][130385] Avg episode reward: [(0, '82.290'), (1, '80.770')] [2023-10-08 07:04:06,480][00611] Updated weights for policy 0, policy_version 80392 (0.0010) [2023-10-08 07:04:06,844][00611] Updated weights for policy 0, policy_version 80402 (0.0011) [2023-10-08 07:04:07,214][00611] Updated weights for policy 0, policy_version 80412 (0.0009) [2023-10-08 07:04:07,347][00612] Updated weights for policy 1, policy_version 80840 (0.0009) [2023-10-08 07:04:07,705][00612] Updated weights for policy 1, policy_version 80850 (0.0011) [2023-10-08 07:04:08,078][00612] Updated weights for policy 1, policy_version 80860 (0.0011) [2023-10-08 07:04:08,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 165150720. Throughput: 0: 1833.3, 1: 1842.3. Samples: 41287608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:04:08,755][130385] Avg episode reward: [(0, '80.180'), (1, '77.640')] [2023-10-08 07:04:10,865][00611] Updated weights for policy 0, policy_version 80422 (0.0007) [2023-10-08 07:04:11,224][00611] Updated weights for policy 0, policy_version 80432 (0.0007) [2023-10-08 07:04:11,608][00611] Updated weights for policy 0, policy_version 80442 (0.0008) [2023-10-08 07:04:11,760][00612] Updated weights for policy 1, policy_version 80870 (0.0008) [2023-10-08 07:04:12,123][00612] Updated weights for policy 1, policy_version 80880 (0.0008) [2023-10-08 07:04:12,495][00612] Updated weights for policy 1, policy_version 80890 (0.0009) [2023-10-08 07:04:13,754][130385] Fps is (10 sec: 16384.6, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 165216256. Throughput: 0: 1832.1, 1: 1833.4. Samples: 41308450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:04:13,754][130385] Avg episode reward: [(0, '80.480'), (1, '73.900')] [2023-10-08 07:04:15,213][00611] Updated weights for policy 0, policy_version 80452 (0.0007) [2023-10-08 07:04:15,609][00611] Updated weights for policy 0, policy_version 80462 (0.0009) [2023-10-08 07:04:15,982][00611] Updated weights for policy 0, policy_version 80472 (0.0010) [2023-10-08 07:04:16,190][00612] Updated weights for policy 1, policy_version 80900 (0.0008) [2023-10-08 07:04:16,565][00612] Updated weights for policy 1, policy_version 80910 (0.0007) [2023-10-08 07:04:16,932][00612] Updated weights for policy 1, policy_version 80920 (0.0007) [2023-10-08 07:04:18,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 165281792. Throughput: 0: 1844.4, 1: 1847.5. Samples: 41331244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:04:18,754][130385] Avg episode reward: [(0, '82.570'), (1, '71.950')] [2023-10-08 07:04:18,764][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000080480_82411520.pth... [2023-10-08 07:04:18,764][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000080928_82870272.pth... [2023-10-08 07:04:18,800][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000078784_80674816.pth [2023-10-08 07:04:18,803][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000079200_81100800.pth [2023-10-08 07:04:19,563][00611] Updated weights for policy 0, policy_version 80482 (0.0009) [2023-10-08 07:04:19,927][00611] Updated weights for policy 0, policy_version 80492 (0.0008) [2023-10-08 07:04:20,295][00611] Updated weights for policy 0, policy_version 80502 (0.0007) [2023-10-08 07:04:20,508][00612] Updated weights for policy 1, policy_version 80930 (0.0007) [2023-10-08 07:04:20,663][00611] Updated weights for policy 0, policy_version 80512 (0.0008) [2023-10-08 07:04:20,879][00612] Updated weights for policy 1, policy_version 80940 (0.0009) [2023-10-08 07:04:21,250][00612] Updated weights for policy 1, policy_version 80950 (0.0007) [2023-10-08 07:04:21,607][00612] Updated weights for policy 1, policy_version 80960 (0.0007) [2023-10-08 07:04:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14773.4). Total num frames: 165347328. Throughput: 0: 1841.7, 1: 1835.7. Samples: 41341906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:04:23,754][130385] Avg episode reward: [(0, '84.290'), (1, '70.310')] [2023-10-08 07:04:24,190][00611] Updated weights for policy 0, policy_version 80522 (0.0008) [2023-10-08 07:04:24,566][00611] Updated weights for policy 0, policy_version 80532 (0.0008) [2023-10-08 07:04:24,926][00611] Updated weights for policy 0, policy_version 80542 (0.0007) [2023-10-08 07:04:25,150][00612] Updated weights for policy 1, policy_version 80970 (0.0007) [2023-10-08 07:04:25,512][00612] Updated weights for policy 1, policy_version 80980 (0.0007) [2023-10-08 07:04:25,887][00612] Updated weights for policy 1, policy_version 80990 (0.0007) [2023-10-08 07:04:28,671][00611] Updated weights for policy 0, policy_version 80552 (0.0007) [2023-10-08 07:04:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 165412864. Throughput: 0: 1847.3, 1: 1855.1. Samples: 41364786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:04:28,754][130385] Avg episode reward: [(0, '88.440'), (1, '72.560')] [2023-10-08 07:04:29,043][00611] Updated weights for policy 0, policy_version 80562 (0.0008) [2023-10-08 07:04:29,425][00611] Updated weights for policy 0, policy_version 80572 (0.0008) [2023-10-08 07:04:29,514][00612] Updated weights for policy 1, policy_version 81000 (0.0008) [2023-10-08 07:04:29,561][00365] Saving new best policy, reward=88.440! [2023-10-08 07:04:29,887][00612] Updated weights for policy 1, policy_version 81010 (0.0007) [2023-10-08 07:04:30,258][00612] Updated weights for policy 1, policy_version 81020 (0.0010) [2023-10-08 07:04:33,142][00611] Updated weights for policy 0, policy_version 80582 (0.0009) [2023-10-08 07:04:33,517][00611] Updated weights for policy 0, policy_version 80592 (0.0011) [2023-10-08 07:04:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 165478400. Throughput: 0: 1839.3, 1: 1854.2. Samples: 41387432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:04:33,754][130385] Avg episode reward: [(0, '85.720'), (1, '76.080')] [2023-10-08 07:04:33,886][00611] Updated weights for policy 0, policy_version 80602 (0.0009) [2023-10-08 07:04:33,927][00612] Updated weights for policy 1, policy_version 81030 (0.0008) [2023-10-08 07:04:34,297][00612] Updated weights for policy 1, policy_version 81040 (0.0009) [2023-10-08 07:04:34,667][00612] Updated weights for policy 1, policy_version 81050 (0.0009) [2023-10-08 07:04:37,499][00611] Updated weights for policy 0, policy_version 80612 (0.0008) [2023-10-08 07:04:37,873][00611] Updated weights for policy 0, policy_version 80622 (0.0007) [2023-10-08 07:04:38,244][00611] Updated weights for policy 0, policy_version 80632 (0.0008) [2023-10-08 07:04:38,398][00612] Updated weights for policy 1, policy_version 81060 (0.0008) [2023-10-08 07:04:38,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 165576704. Throughput: 0: 1846.4, 1: 1854.7. Samples: 41397746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:04:38,754][130385] Avg episode reward: [(0, '84.470'), (1, '76.380')] [2023-10-08 07:04:38,769][00612] Updated weights for policy 1, policy_version 81070 (0.0009) [2023-10-08 07:04:39,130][00612] Updated weights for policy 1, policy_version 81080 (0.0009) [2023-10-08 07:04:41,896][00611] Updated weights for policy 0, policy_version 80642 (0.0007) [2023-10-08 07:04:42,272][00611] Updated weights for policy 0, policy_version 80652 (0.0007) [2023-10-08 07:04:42,636][00611] Updated weights for policy 0, policy_version 80662 (0.0008) [2023-10-08 07:04:42,761][00612] Updated weights for policy 1, policy_version 81090 (0.0007) [2023-10-08 07:04:43,008][00611] Updated weights for policy 0, policy_version 80672 (0.0007) [2023-10-08 07:04:43,136][00612] Updated weights for policy 1, policy_version 81100 (0.0010) [2023-10-08 07:04:43,505][00612] Updated weights for policy 1, policy_version 81110 (0.0007) [2023-10-08 07:04:43,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 165642240. Throughput: 0: 1837.6, 1: 1849.4. Samples: 41420366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:04:43,755][130385] Avg episode reward: [(0, '80.360'), (1, '76.430')] [2023-10-08 07:04:43,871][00612] Updated weights for policy 1, policy_version 81120 (0.0009) [2023-10-08 07:04:46,576][00611] Updated weights for policy 0, policy_version 80682 (0.0008) [2023-10-08 07:04:46,949][00611] Updated weights for policy 0, policy_version 80692 (0.0009) [2023-10-08 07:04:47,316][00611] Updated weights for policy 0, policy_version 80702 (0.0008) [2023-10-08 07:04:47,374][00612] Updated weights for policy 1, policy_version 81130 (0.0007) [2023-10-08 07:04:47,736][00612] Updated weights for policy 1, policy_version 81140 (0.0009) [2023-10-08 07:04:48,108][00612] Updated weights for policy 1, policy_version 81150 (0.0009) [2023-10-08 07:04:48,754][130385] Fps is (10 sec: 16383.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 165740544. Throughput: 0: 1852.9, 1: 1834.1. Samples: 41440984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:04:48,755][130385] Avg episode reward: [(0, '81.480'), (1, '76.100')] [2023-10-08 07:04:50,907][00611] Updated weights for policy 0, policy_version 80712 (0.0008) [2023-10-08 07:04:51,284][00611] Updated weights for policy 0, policy_version 80722 (0.0009) [2023-10-08 07:04:51,653][00611] Updated weights for policy 0, policy_version 80732 (0.0008) [2023-10-08 07:04:51,788][00612] Updated weights for policy 1, policy_version 81160 (0.0007) [2023-10-08 07:04:52,154][00612] Updated weights for policy 1, policy_version 81170 (0.0007) [2023-10-08 07:04:52,521][00612] Updated weights for policy 1, policy_version 81180 (0.0007) [2023-10-08 07:04:53,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 165806080. Throughput: 0: 1833.7, 1: 1848.6. Samples: 41453312. Policy #0 lag: (min: 1.0, avg: 1.8, max: 19.0) [2023-10-08 07:04:53,754][130385] Avg episode reward: [(0, '76.430'), (1, '71.020')] [2023-10-08 07:04:55,265][00611] Updated weights for policy 0, policy_version 80742 (0.0009) [2023-10-08 07:04:55,631][00611] Updated weights for policy 0, policy_version 80752 (0.0010) [2023-10-08 07:04:56,001][00611] Updated weights for policy 0, policy_version 80762 (0.0009) [2023-10-08 07:04:56,164][00612] Updated weights for policy 1, policy_version 81190 (0.0007) [2023-10-08 07:04:56,537][00612] Updated weights for policy 1, policy_version 81200 (0.0007) [2023-10-08 07:04:56,905][00612] Updated weights for policy 1, policy_version 81210 (0.0010) [2023-10-08 07:04:58,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 165871616. Throughput: 0: 1852.1, 1: 1831.8. Samples: 41474226. Policy #0 lag: (min: 1.0, avg: 1.8, max: 19.0) [2023-10-08 07:04:58,754][130385] Avg episode reward: [(0, '72.870'), (1, '71.160')] [2023-10-08 07:04:59,712][00611] Updated weights for policy 0, policy_version 80772 (0.0007) [2023-10-08 07:05:00,084][00611] Updated weights for policy 0, policy_version 80782 (0.0009) [2023-10-08 07:05:00,454][00611] Updated weights for policy 0, policy_version 80792 (0.0009) [2023-10-08 07:05:00,601][00612] Updated weights for policy 1, policy_version 81220 (0.0009) [2023-10-08 07:05:00,964][00612] Updated weights for policy 1, policy_version 81230 (0.0010) [2023-10-08 07:05:01,332][00612] Updated weights for policy 1, policy_version 81240 (0.0011) [2023-10-08 07:05:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 165937152. Throughput: 0: 1849.6, 1: 1848.7. Samples: 41497670. Policy #0 lag: (min: 1.0, avg: 1.8, max: 19.0) [2023-10-08 07:05:03,754][130385] Avg episode reward: [(0, '73.980'), (1, '73.700')] [2023-10-08 07:05:04,067][00611] Updated weights for policy 0, policy_version 80802 (0.0008) [2023-10-08 07:05:04,474][00611] Updated weights for policy 0, policy_version 80812 (0.0007) [2023-10-08 07:05:04,842][00611] Updated weights for policy 0, policy_version 80822 (0.0008) [2023-10-08 07:05:04,930][00612] Updated weights for policy 1, policy_version 81250 (0.0009) [2023-10-08 07:05:05,204][00611] Updated weights for policy 0, policy_version 80832 (0.0008) [2023-10-08 07:05:05,303][00612] Updated weights for policy 1, policy_version 81260 (0.0008) [2023-10-08 07:05:05,675][00612] Updated weights for policy 1, policy_version 81270 (0.0009) [2023-10-08 07:05:06,037][00612] Updated weights for policy 1, policy_version 81280 (0.0010) [2023-10-08 07:05:08,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 166002688. Throughput: 0: 1844.8, 1: 1833.9. Samples: 41507448. Policy #0 lag: (min: 1.0, avg: 1.8, max: 19.0) [2023-10-08 07:05:08,755][130385] Avg episode reward: [(0, '74.060'), (1, '80.580')] [2023-10-08 07:05:08,837][00611] Updated weights for policy 0, policy_version 80842 (0.0007) [2023-10-08 07:05:09,216][00611] Updated weights for policy 0, policy_version 80852 (0.0008) [2023-10-08 07:05:09,580][00611] Updated weights for policy 0, policy_version 80862 (0.0009) [2023-10-08 07:05:09,687][00612] Updated weights for policy 1, policy_version 81290 (0.0008) [2023-10-08 07:05:10,064][00612] Updated weights for policy 1, policy_version 81300 (0.0008) [2023-10-08 07:05:10,433][00612] Updated weights for policy 1, policy_version 81310 (0.0009) [2023-10-08 07:05:13,030][00611] Updated weights for policy 0, policy_version 80872 (0.0007) [2023-10-08 07:05:13,407][00611] Updated weights for policy 0, policy_version 80882 (0.0009) [2023-10-08 07:05:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 166068224. Throughput: 0: 1845.6, 1: 1842.7. Samples: 41530762. Policy #0 lag: (min: 1.0, avg: 1.8, max: 19.0) [2023-10-08 07:05:13,754][130385] Avg episode reward: [(0, '73.930'), (1, '76.260')] [2023-10-08 07:05:13,784][00611] Updated weights for policy 0, policy_version 80892 (0.0010) [2023-10-08 07:05:14,229][00612] Updated weights for policy 1, policy_version 81320 (0.0008) [2023-10-08 07:05:14,609][00612] Updated weights for policy 1, policy_version 81330 (0.0011) [2023-10-08 07:05:14,981][00612] Updated weights for policy 1, policy_version 81340 (0.0010) [2023-10-08 07:05:17,494][00611] Updated weights for policy 0, policy_version 80902 (0.0009) [2023-10-08 07:05:17,856][00611] Updated weights for policy 0, policy_version 80912 (0.0008) [2023-10-08 07:05:18,226][00611] Updated weights for policy 0, policy_version 80922 (0.0008) [2023-10-08 07:05:18,505][00612] Updated weights for policy 1, policy_version 81350 (0.0007) [2023-10-08 07:05:18,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 166166528. Throughput: 0: 1828.6, 1: 1853.6. Samples: 41553128. Policy #0 lag: (min: 1.0, avg: 1.8, max: 19.0) [2023-10-08 07:05:18,754][130385] Avg episode reward: [(0, '76.460'), (1, '77.030')] [2023-10-08 07:05:18,880][00612] Updated weights for policy 1, policy_version 81360 (0.0010) [2023-10-08 07:05:19,251][00612] Updated weights for policy 1, policy_version 81370 (0.0010) [2023-10-08 07:05:21,801][00611] Updated weights for policy 0, policy_version 80932 (0.0007) [2023-10-08 07:05:22,184][00611] Updated weights for policy 0, policy_version 80942 (0.0009) [2023-10-08 07:05:22,565][00611] Updated weights for policy 0, policy_version 80952 (0.0009) [2023-10-08 07:05:22,820][00612] Updated weights for policy 1, policy_version 81380 (0.0009) [2023-10-08 07:05:23,177][00612] Updated weights for policy 1, policy_version 81390 (0.0007) [2023-10-08 07:05:23,556][00612] Updated weights for policy 1, policy_version 81400 (0.0008) [2023-10-08 07:05:23,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 166232064. Throughput: 0: 1845.5, 1: 1847.5. Samples: 41563932. Policy #0 lag: (min: 1.0, avg: 1.8, max: 19.0) [2023-10-08 07:05:23,755][130385] Avg episode reward: [(0, '78.660'), (1, '78.830')] [2023-10-08 07:05:26,236][00611] Updated weights for policy 0, policy_version 80962 (0.0007) [2023-10-08 07:05:26,609][00611] Updated weights for policy 0, policy_version 80972 (0.0007) [2023-10-08 07:05:26,983][00611] Updated weights for policy 0, policy_version 80982 (0.0007) [2023-10-08 07:05:27,181][00612] Updated weights for policy 1, policy_version 81410 (0.0009) [2023-10-08 07:05:27,358][00611] Updated weights for policy 0, policy_version 80992 (0.0009) [2023-10-08 07:05:27,544][00612] Updated weights for policy 1, policy_version 81420 (0.0009) [2023-10-08 07:05:27,919][00612] Updated weights for policy 1, policy_version 81430 (0.0008) [2023-10-08 07:05:28,281][00612] Updated weights for policy 1, policy_version 81440 (0.0007) [2023-10-08 07:05:28,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 166330368. Throughput: 0: 1825.8, 1: 1849.7. Samples: 41585764. Policy #0 lag: (min: 1.0, avg: 1.8, max: 19.0) [2023-10-08 07:05:28,755][130385] Avg episode reward: [(0, '78.520'), (1, '81.930')] [2023-10-08 07:05:31,126][00611] Updated weights for policy 0, policy_version 81002 (0.0009) [2023-10-08 07:05:31,490][00611] Updated weights for policy 0, policy_version 81012 (0.0009) [2023-10-08 07:05:31,857][00612] Updated weights for policy 1, policy_version 81450 (0.0008) [2023-10-08 07:05:31,870][00611] Updated weights for policy 0, policy_version 81022 (0.0009) [2023-10-08 07:05:32,221][00612] Updated weights for policy 1, policy_version 81460 (0.0010) [2023-10-08 07:05:32,596][00612] Updated weights for policy 1, policy_version 81470 (0.0009) [2023-10-08 07:05:33,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 166395904. Throughput: 0: 1841.0, 1: 1852.2. Samples: 41607176. Policy #0 lag: (min: 1.0, avg: 1.8, max: 19.0) [2023-10-08 07:05:33,754][130385] Avg episode reward: [(0, '78.320'), (1, '84.310')] [2023-10-08 07:05:35,419][00611] Updated weights for policy 0, policy_version 81032 (0.0009) [2023-10-08 07:05:35,790][00611] Updated weights for policy 0, policy_version 81042 (0.0010) [2023-10-08 07:05:36,157][00611] Updated weights for policy 0, policy_version 81052 (0.0009) [2023-10-08 07:05:36,180][00612] Updated weights for policy 1, policy_version 81480 (0.0009) [2023-10-08 07:05:36,553][00612] Updated weights for policy 1, policy_version 81490 (0.0010) [2023-10-08 07:05:36,920][00612] Updated weights for policy 1, policy_version 81500 (0.0009) [2023-10-08 07:05:38,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 166461440. Throughput: 0: 1830.7, 1: 1849.5. Samples: 41618918. Policy #0 lag: (min: 1.0, avg: 1.8, max: 19.0) [2023-10-08 07:05:38,754][130385] Avg episode reward: [(0, '79.180'), (1, '82.750')] [2023-10-08 07:05:39,909][00611] Updated weights for policy 0, policy_version 81062 (0.0009) [2023-10-08 07:05:40,278][00611] Updated weights for policy 0, policy_version 81072 (0.0008) [2023-10-08 07:05:40,531][00612] Updated weights for policy 1, policy_version 81510 (0.0007) [2023-10-08 07:05:40,645][00611] Updated weights for policy 0, policy_version 81082 (0.0008) [2023-10-08 07:05:40,899][00612] Updated weights for policy 1, policy_version 81520 (0.0008) [2023-10-08 07:05:41,268][00612] Updated weights for policy 1, policy_version 81530 (0.0007) [2023-10-08 07:05:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 166526976. Throughput: 0: 1838.5, 1: 1855.0. Samples: 41640436. Policy #0 lag: (min: 17.0, avg: 27.9, max: 49.0) [2023-10-08 07:05:43,754][130385] Avg episode reward: [(0, '79.740'), (1, '80.270')] [2023-10-08 07:05:44,324][00611] Updated weights for policy 0, policy_version 81092 (0.0008) [2023-10-08 07:05:44,684][00611] Updated weights for policy 0, policy_version 81102 (0.0007) [2023-10-08 07:05:44,960][00612] Updated weights for policy 1, policy_version 81540 (0.0008) [2023-10-08 07:05:45,055][00611] Updated weights for policy 0, policy_version 81112 (0.0007) [2023-10-08 07:05:45,323][00612] Updated weights for policy 1, policy_version 81550 (0.0009) [2023-10-08 07:05:45,692][00612] Updated weights for policy 1, policy_version 81560 (0.0008) [2023-10-08 07:05:48,520][00611] Updated weights for policy 0, policy_version 81122 (0.0007) [2023-10-08 07:05:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 166592512. Throughput: 0: 1837.8, 1: 1851.0. Samples: 41663666. Policy #0 lag: (min: 17.0, avg: 27.9, max: 49.0) [2023-10-08 07:05:48,754][130385] Avg episode reward: [(0, '80.840'), (1, '80.520')] [2023-10-08 07:05:48,899][00611] Updated weights for policy 0, policy_version 81132 (0.0009) [2023-10-08 07:05:49,272][00611] Updated weights for policy 0, policy_version 81142 (0.0008) [2023-10-08 07:05:49,452][00612] Updated weights for policy 1, policy_version 81570 (0.0008) [2023-10-08 07:05:49,635][00611] Updated weights for policy 0, policy_version 81152 (0.0008) [2023-10-08 07:05:49,816][00612] Updated weights for policy 1, policy_version 81580 (0.0007) [2023-10-08 07:05:50,185][00612] Updated weights for policy 1, policy_version 81590 (0.0008) [2023-10-08 07:05:50,548][00612] Updated weights for policy 1, policy_version 81600 (0.0008) [2023-10-08 07:05:53,362][00611] Updated weights for policy 0, policy_version 81162 (0.0009) [2023-10-08 07:05:53,737][00611] Updated weights for policy 0, policy_version 81172 (0.0009) [2023-10-08 07:05:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 166658048. Throughput: 0: 1841.6, 1: 1848.7. Samples: 41673512. Policy #0 lag: (min: 17.0, avg: 27.9, max: 49.0) [2023-10-08 07:05:53,754][130385] Avg episode reward: [(0, '82.040'), (1, '84.110')] [2023-10-08 07:05:54,100][00611] Updated weights for policy 0, policy_version 81182 (0.0008) [2023-10-08 07:05:54,222][00612] Updated weights for policy 1, policy_version 81610 (0.0007) [2023-10-08 07:05:54,591][00612] Updated weights for policy 1, policy_version 81620 (0.0007) [2023-10-08 07:05:54,964][00612] Updated weights for policy 1, policy_version 81630 (0.0007) [2023-10-08 07:05:57,683][00611] Updated weights for policy 0, policy_version 81192 (0.0007) [2023-10-08 07:05:58,055][00611] Updated weights for policy 0, policy_version 81202 (0.0007) [2023-10-08 07:05:58,433][00611] Updated weights for policy 0, policy_version 81212 (0.0008) [2023-10-08 07:05:58,480][00612] Updated weights for policy 1, policy_version 81640 (0.0007) [2023-10-08 07:05:58,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 166756352. Throughput: 0: 1830.9, 1: 1851.3. Samples: 41696460. Policy #0 lag: (min: 17.0, avg: 27.9, max: 49.0) [2023-10-08 07:05:58,754][130385] Avg episode reward: [(0, '80.470'), (1, '86.060')] [2023-10-08 07:05:58,850][00612] Updated weights for policy 1, policy_version 81650 (0.0008) [2023-10-08 07:05:59,216][00612] Updated weights for policy 1, policy_version 81660 (0.0011) [2023-10-08 07:06:02,056][00611] Updated weights for policy 0, policy_version 81222 (0.0007) [2023-10-08 07:06:02,424][00611] Updated weights for policy 0, policy_version 81232 (0.0007) [2023-10-08 07:06:02,796][00611] Updated weights for policy 0, policy_version 81242 (0.0007) [2023-10-08 07:06:02,831][00612] Updated weights for policy 1, policy_version 81670 (0.0009) [2023-10-08 07:06:03,196][00612] Updated weights for policy 1, policy_version 81680 (0.0007) [2023-10-08 07:06:03,565][00612] Updated weights for policy 1, policy_version 81690 (0.0007) [2023-10-08 07:06:03,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 166821888. Throughput: 0: 1821.9, 1: 1825.1. Samples: 41717242. Policy #0 lag: (min: 17.0, avg: 27.9, max: 49.0) [2023-10-08 07:06:03,754][130385] Avg episode reward: [(0, '80.900'), (1, '86.770')] [2023-10-08 07:06:06,488][00611] Updated weights for policy 0, policy_version 81252 (0.0008) [2023-10-08 07:06:06,857][00611] Updated weights for policy 0, policy_version 81262 (0.0009) [2023-10-08 07:06:07,154][00612] Updated weights for policy 1, policy_version 81700 (0.0009) [2023-10-08 07:06:07,227][00611] Updated weights for policy 0, policy_version 81272 (0.0008) [2023-10-08 07:06:07,538][00612] Updated weights for policy 1, policy_version 81710 (0.0008) [2023-10-08 07:06:07,907][00612] Updated weights for policy 1, policy_version 81720 (0.0010) [2023-10-08 07:06:08,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 166920192. Throughput: 0: 1832.4, 1: 1853.8. Samples: 41729810. Policy #0 lag: (min: 17.0, avg: 27.9, max: 49.0) [2023-10-08 07:06:08,754][130385] Avg episode reward: [(0, '85.320'), (1, '91.390')] [2023-10-08 07:06:10,844][00611] Updated weights for policy 0, policy_version 81282 (0.0007) [2023-10-08 07:06:11,211][00611] Updated weights for policy 0, policy_version 81292 (0.0007) [2023-10-08 07:06:11,518][00612] Updated weights for policy 1, policy_version 81730 (0.0007) [2023-10-08 07:06:11,587][00611] Updated weights for policy 0, policy_version 81302 (0.0010) [2023-10-08 07:06:11,878][00612] Updated weights for policy 1, policy_version 81740 (0.0007) [2023-10-08 07:06:11,950][00611] Updated weights for policy 0, policy_version 81312 (0.0008) [2023-10-08 07:06:12,252][00612] Updated weights for policy 1, policy_version 81750 (0.0009) [2023-10-08 07:06:12,621][00612] Updated weights for policy 1, policy_version 81760 (0.0009) [2023-10-08 07:06:13,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 166985728. Throughput: 0: 1824.9, 1: 1833.3. Samples: 41750384. Policy #0 lag: (min: 17.0, avg: 27.9, max: 49.0) [2023-10-08 07:06:13,755][130385] Avg episode reward: [(0, '86.130'), (1, '91.850')] [2023-10-08 07:06:15,601][00611] Updated weights for policy 0, policy_version 81322 (0.0008) [2023-10-08 07:06:15,971][00611] Updated weights for policy 0, policy_version 81332 (0.0007) [2023-10-08 07:06:16,170][00612] Updated weights for policy 1, policy_version 81770 (0.0007) [2023-10-08 07:06:16,338][00611] Updated weights for policy 0, policy_version 81342 (0.0007) [2023-10-08 07:06:16,535][00612] Updated weights for policy 1, policy_version 81780 (0.0008) [2023-10-08 07:06:16,908][00612] Updated weights for policy 1, policy_version 81790 (0.0009) [2023-10-08 07:06:18,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 167051264. Throughput: 0: 1838.9, 1: 1846.0. Samples: 41772998. Policy #0 lag: (min: 17.0, avg: 27.9, max: 49.0) [2023-10-08 07:06:18,754][130385] Avg episode reward: [(0, '87.620'), (1, '87.250')] [2023-10-08 07:06:18,765][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000081792_83755008.pth... [2023-10-08 07:06:18,765][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000081344_83296256.pth... [2023-10-08 07:06:18,803][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000080064_81985536.pth [2023-10-08 07:06:18,805][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000079616_81526784.pth [2023-10-08 07:06:19,867][00611] Updated weights for policy 0, policy_version 81352 (0.0008) [2023-10-08 07:06:20,235][00611] Updated weights for policy 0, policy_version 81362 (0.0007) [2023-10-08 07:06:20,484][00612] Updated weights for policy 1, policy_version 81800 (0.0008) [2023-10-08 07:06:20,604][00611] Updated weights for policy 0, policy_version 81372 (0.0009) [2023-10-08 07:06:20,850][00612] Updated weights for policy 1, policy_version 81810 (0.0009) [2023-10-08 07:06:21,213][00612] Updated weights for policy 1, policy_version 81820 (0.0009) [2023-10-08 07:06:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 167116800. Throughput: 0: 1833.1, 1: 1824.3. Samples: 41783500. Policy #0 lag: (min: 17.0, avg: 27.9, max: 49.0) [2023-10-08 07:06:23,755][130385] Avg episode reward: [(0, '87.110'), (1, '84.360')] [2023-10-08 07:06:24,225][00611] Updated weights for policy 0, policy_version 81382 (0.0010) [2023-10-08 07:06:24,603][00611] Updated weights for policy 0, policy_version 81392 (0.0009) [2023-10-08 07:06:24,895][00612] Updated weights for policy 1, policy_version 81830 (0.0007) [2023-10-08 07:06:24,977][00611] Updated weights for policy 0, policy_version 81402 (0.0008) [2023-10-08 07:06:25,260][00612] Updated weights for policy 1, policy_version 81840 (0.0007) [2023-10-08 07:06:25,638][00612] Updated weights for policy 1, policy_version 81850 (0.0009) [2023-10-08 07:06:28,717][00611] Updated weights for policy 0, policy_version 81412 (0.0008) [2023-10-08 07:06:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 167182336. Throughput: 0: 1837.5, 1: 1842.3. Samples: 41806026. Policy #0 lag: (min: 17.0, avg: 27.9, max: 49.0) [2023-10-08 07:06:28,754][130385] Avg episode reward: [(0, '84.150'), (1, '85.150')] [2023-10-08 07:06:29,085][00611] Updated weights for policy 0, policy_version 81422 (0.0010) [2023-10-08 07:06:29,238][00612] Updated weights for policy 1, policy_version 81860 (0.0007) [2023-10-08 07:06:29,451][00611] Updated weights for policy 0, policy_version 81432 (0.0007) [2023-10-08 07:06:29,606][00612] Updated weights for policy 1, policy_version 81870 (0.0008) [2023-10-08 07:06:29,969][00612] Updated weights for policy 1, policy_version 81880 (0.0009) [2023-10-08 07:06:33,096][00611] Updated weights for policy 0, policy_version 81442 (0.0007) [2023-10-08 07:06:33,457][00611] Updated weights for policy 0, policy_version 81452 (0.0009) [2023-10-08 07:06:33,634][00612] Updated weights for policy 1, policy_version 81890 (0.0009) [2023-10-08 07:06:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 167247872. Throughput: 0: 1829.2, 1: 1851.3. Samples: 41829292. Policy #0 lag: (min: 17.0, avg: 27.9, max: 49.0) [2023-10-08 07:06:33,755][130385] Avg episode reward: [(0, '83.590'), (1, '83.050')] [2023-10-08 07:06:33,824][00611] Updated weights for policy 0, policy_version 81462 (0.0007) [2023-10-08 07:06:34,002][00612] Updated weights for policy 1, policy_version 81900 (0.0008) [2023-10-08 07:06:34,195][00611] Updated weights for policy 0, policy_version 81472 (0.0007) [2023-10-08 07:06:34,376][00612] Updated weights for policy 1, policy_version 81910 (0.0008) [2023-10-08 07:06:34,749][00612] Updated weights for policy 1, policy_version 81920 (0.0008) [2023-10-08 07:06:37,758][00611] Updated weights for policy 0, policy_version 81482 (0.0010) [2023-10-08 07:06:38,120][00611] Updated weights for policy 0, policy_version 81492 (0.0010) [2023-10-08 07:06:38,490][00611] Updated weights for policy 0, policy_version 81502 (0.0009) [2023-10-08 07:06:38,511][00612] Updated weights for policy 1, policy_version 81930 (0.0009) [2023-10-08 07:06:38,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 167346176. Throughput: 0: 1840.0, 1: 1847.6. Samples: 41839452. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-08 07:06:38,754][130385] Avg episode reward: [(0, '85.670'), (1, '80.530')] [2023-10-08 07:06:38,886][00612] Updated weights for policy 1, policy_version 81940 (0.0010) [2023-10-08 07:06:39,252][00612] Updated weights for policy 1, policy_version 81950 (0.0008) [2023-10-08 07:06:42,206][00611] Updated weights for policy 0, policy_version 81512 (0.0008) [2023-10-08 07:06:42,588][00611] Updated weights for policy 0, policy_version 81522 (0.0009) [2023-10-08 07:06:42,877][00612] Updated weights for policy 1, policy_version 81960 (0.0008) [2023-10-08 07:06:42,956][00611] Updated weights for policy 0, policy_version 81532 (0.0007) [2023-10-08 07:06:43,242][00612] Updated weights for policy 1, policy_version 81970 (0.0009) [2023-10-08 07:06:43,619][00612] Updated weights for policy 1, policy_version 81980 (0.0010) [2023-10-08 07:06:43,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 167411712. Throughput: 0: 1837.8, 1: 1844.9. Samples: 41862182. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-08 07:06:43,754][130385] Avg episode reward: [(0, '88.290'), (1, '80.730')] [2023-10-08 07:06:46,698][00611] Updated weights for policy 0, policy_version 81542 (0.0009) [2023-10-08 07:06:47,071][00611] Updated weights for policy 0, policy_version 81552 (0.0010) [2023-10-08 07:06:47,326][00612] Updated weights for policy 1, policy_version 81990 (0.0008) [2023-10-08 07:06:47,434][00611] Updated weights for policy 0, policy_version 81562 (0.0008) [2023-10-08 07:06:47,698][00612] Updated weights for policy 1, policy_version 82000 (0.0009) [2023-10-08 07:06:48,072][00612] Updated weights for policy 1, policy_version 82010 (0.0010) [2023-10-08 07:06:48,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 167510016. Throughput: 0: 1840.9, 1: 1831.3. Samples: 41882494. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-08 07:06:48,755][130385] Avg episode reward: [(0, '87.970'), (1, '76.610')] [2023-10-08 07:06:51,195][00611] Updated weights for policy 0, policy_version 81572 (0.0008) [2023-10-08 07:06:51,566][00611] Updated weights for policy 0, policy_version 81582 (0.0008) [2023-10-08 07:06:51,825][00612] Updated weights for policy 1, policy_version 82020 (0.0011) [2023-10-08 07:06:51,936][00611] Updated weights for policy 0, policy_version 81592 (0.0009) [2023-10-08 07:06:52,210][00612] Updated weights for policy 1, policy_version 82030 (0.0008) [2023-10-08 07:06:52,584][00612] Updated weights for policy 1, policy_version 82040 (0.0010) [2023-10-08 07:06:53,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 167575552. Throughput: 0: 1833.3, 1: 1834.8. Samples: 41894876. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-08 07:06:53,757][130385] Avg episode reward: [(0, '92.170'), (1, '77.590')] [2023-10-08 07:06:53,757][00365] Saving new best policy, reward=92.170! [2023-10-08 07:06:55,708][00611] Updated weights for policy 0, policy_version 81602 (0.0007) [2023-10-08 07:06:56,087][00611] Updated weights for policy 0, policy_version 81612 (0.0007) [2023-10-08 07:06:56,246][00612] Updated weights for policy 1, policy_version 82050 (0.0009) [2023-10-08 07:06:56,458][00611] Updated weights for policy 0, policy_version 81622 (0.0008) [2023-10-08 07:06:56,607][00612] Updated weights for policy 1, policy_version 82060 (0.0007) [2023-10-08 07:06:56,826][00611] Updated weights for policy 0, policy_version 81632 (0.0007) [2023-10-08 07:06:56,978][00612] Updated weights for policy 1, policy_version 82070 (0.0009) [2023-10-08 07:06:57,348][00612] Updated weights for policy 1, policy_version 82080 (0.0009) [2023-10-08 07:06:58,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 167641088. Throughput: 0: 1831.1, 1: 1820.4. Samples: 41914698. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-08 07:06:58,754][130385] Avg episode reward: [(0, '89.860'), (1, '72.480')] [2023-10-08 07:07:00,485][00611] Updated weights for policy 0, policy_version 81642 (0.0009) [2023-10-08 07:07:00,865][00611] Updated weights for policy 0, policy_version 81652 (0.0009) [2023-10-08 07:07:01,068][00612] Updated weights for policy 1, policy_version 82090 (0.0008) [2023-10-08 07:07:01,228][00611] Updated weights for policy 0, policy_version 81662 (0.0007) [2023-10-08 07:07:01,436][00612] Updated weights for policy 1, policy_version 82100 (0.0007) [2023-10-08 07:07:01,813][00612] Updated weights for policy 1, policy_version 82110 (0.0010) [2023-10-08 07:07:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 167706624. Throughput: 0: 1829.5, 1: 1834.7. Samples: 41937884. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-08 07:07:03,754][130385] Avg episode reward: [(0, '87.430'), (1, '73.990')] [2023-10-08 07:07:04,879][00611] Updated weights for policy 0, policy_version 81672 (0.0008) [2023-10-08 07:07:05,245][00611] Updated weights for policy 0, policy_version 81682 (0.0007) [2023-10-08 07:07:05,391][00612] Updated weights for policy 1, policy_version 82120 (0.0009) [2023-10-08 07:07:05,610][00611] Updated weights for policy 0, policy_version 81692 (0.0007) [2023-10-08 07:07:05,753][00612] Updated weights for policy 1, policy_version 82130 (0.0007) [2023-10-08 07:07:06,128][00612] Updated weights for policy 1, policy_version 82140 (0.0007) [2023-10-08 07:07:08,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 167772160. Throughput: 0: 1825.5, 1: 1829.1. Samples: 41947954. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-08 07:07:08,754][130385] Avg episode reward: [(0, '81.720'), (1, '72.720')] [2023-10-08 07:07:09,234][00611] Updated weights for policy 0, policy_version 81702 (0.0007) [2023-10-08 07:07:09,596][00611] Updated weights for policy 0, policy_version 81712 (0.0007) [2023-10-08 07:07:09,943][00612] Updated weights for policy 1, policy_version 82150 (0.0009) [2023-10-08 07:07:09,970][00611] Updated weights for policy 0, policy_version 81722 (0.0008) [2023-10-08 07:07:10,304][00612] Updated weights for policy 1, policy_version 82160 (0.0007) [2023-10-08 07:07:10,677][00612] Updated weights for policy 1, policy_version 82170 (0.0008) [2023-10-08 07:07:13,714][00611] Updated weights for policy 0, policy_version 81732 (0.0009) [2023-10-08 07:07:13,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 167837696. Throughput: 0: 1834.1, 1: 1833.5. Samples: 41971066. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-08 07:07:13,755][130385] Avg episode reward: [(0, '84.560'), (1, '71.230')] [2023-10-08 07:07:14,082][00611] Updated weights for policy 0, policy_version 81742 (0.0009) [2023-10-08 07:07:14,155][00612] Updated weights for policy 1, policy_version 82180 (0.0008) [2023-10-08 07:07:14,456][00611] Updated weights for policy 0, policy_version 81752 (0.0008) [2023-10-08 07:07:14,509][00612] Updated weights for policy 1, policy_version 82190 (0.0007) [2023-10-08 07:07:14,872][00612] Updated weights for policy 1, policy_version 82200 (0.0008) [2023-10-08 07:07:18,217][00611] Updated weights for policy 0, policy_version 81762 (0.0008) [2023-10-08 07:07:18,471][00612] Updated weights for policy 1, policy_version 82210 (0.0009) [2023-10-08 07:07:18,592][00611] Updated weights for policy 0, policy_version 81772 (0.0008) [2023-10-08 07:07:18,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 167903232. Throughput: 0: 1828.6, 1: 1827.7. Samples: 41993824. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-08 07:07:18,755][130385] Avg episode reward: [(0, '87.720'), (1, '66.150')] [2023-10-08 07:07:18,831][00612] Updated weights for policy 1, policy_version 82220 (0.0008) [2023-10-08 07:07:18,973][00611] Updated weights for policy 0, policy_version 81782 (0.0007) [2023-10-08 07:07:19,203][00612] Updated weights for policy 1, policy_version 82230 (0.0008) [2023-10-08 07:07:19,341][00611] Updated weights for policy 0, policy_version 81792 (0.0009) [2023-10-08 07:07:19,561][00612] Updated weights for policy 1, policy_version 82240 (0.0008) [2023-10-08 07:07:23,079][00611] Updated weights for policy 0, policy_version 81802 (0.0009) [2023-10-08 07:07:23,320][00612] Updated weights for policy 1, policy_version 82250 (0.0007) [2023-10-08 07:07:23,443][00611] Updated weights for policy 0, policy_version 81812 (0.0007) [2023-10-08 07:07:23,679][00612] Updated weights for policy 1, policy_version 82260 (0.0010) [2023-10-08 07:07:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 167968768. Throughput: 0: 1818.7, 1: 1832.0. Samples: 42003734. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-08 07:07:23,754][130385] Avg episode reward: [(0, '85.340'), (1, '68.940')] [2023-10-08 07:07:23,813][00611] Updated weights for policy 0, policy_version 81822 (0.0009) [2023-10-08 07:07:24,056][00612] Updated weights for policy 1, policy_version 82270 (0.0010) [2023-10-08 07:07:27,638][00611] Updated weights for policy 0, policy_version 81832 (0.0007) [2023-10-08 07:07:27,651][00612] Updated weights for policy 1, policy_version 82280 (0.0008) [2023-10-08 07:07:28,016][00611] Updated weights for policy 0, policy_version 81842 (0.0007) [2023-10-08 07:07:28,023][00612] Updated weights for policy 1, policy_version 82290 (0.0007) [2023-10-08 07:07:28,384][00611] Updated weights for policy 0, policy_version 81852 (0.0008) [2023-10-08 07:07:28,391][00612] Updated weights for policy 1, policy_version 82300 (0.0008) [2023-10-08 07:07:28,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.6, 300 sec: 14773.4). Total num frames: 168099840. Throughput: 0: 1824.9, 1: 1826.7. Samples: 42026504. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-08 07:07:28,755][130385] Avg episode reward: [(0, '81.630'), (1, '70.480')] [2023-10-08 07:07:31,944][00611] Updated weights for policy 0, policy_version 81862 (0.0009) [2023-10-08 07:07:32,013][00612] Updated weights for policy 1, policy_version 82310 (0.0010) [2023-10-08 07:07:32,312][00611] Updated weights for policy 0, policy_version 81872 (0.0009) [2023-10-08 07:07:32,382][00612] Updated weights for policy 1, policy_version 82320 (0.0008) [2023-10-08 07:07:32,684][00611] Updated weights for policy 0, policy_version 81882 (0.0007) [2023-10-08 07:07:32,763][00612] Updated weights for policy 1, policy_version 82330 (0.0007) [2023-10-08 07:07:33,754][130385] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 168165376. Throughput: 0: 1818.0, 1: 1822.4. Samples: 42046310. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-08 07:07:33,754][130385] Avg episode reward: [(0, '81.290'), (1, '70.630')] [2023-10-08 07:07:36,358][00611] Updated weights for policy 0, policy_version 81892 (0.0007) [2023-10-08 07:07:36,385][00612] Updated weights for policy 1, policy_version 82340 (0.0007) [2023-10-08 07:07:36,732][00611] Updated weights for policy 0, policy_version 81902 (0.0008) [2023-10-08 07:07:36,768][00612] Updated weights for policy 1, policy_version 82350 (0.0008) [2023-10-08 07:07:37,107][00611] Updated weights for policy 0, policy_version 81912 (0.0008) [2023-10-08 07:07:37,133][00612] Updated weights for policy 1, policy_version 82360 (0.0007) [2023-10-08 07:07:38,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 168230912. Throughput: 0: 1823.2, 1: 1838.5. Samples: 42059656. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-08 07:07:38,754][130385] Avg episode reward: [(0, '82.540'), (1, '68.120')] [2023-10-08 07:07:40,653][00611] Updated weights for policy 0, policy_version 81922 (0.0008) [2023-10-08 07:07:40,808][00612] Updated weights for policy 1, policy_version 82370 (0.0008) [2023-10-08 07:07:41,023][00611] Updated weights for policy 0, policy_version 81932 (0.0008) [2023-10-08 07:07:41,171][00612] Updated weights for policy 1, policy_version 82380 (0.0007) [2023-10-08 07:07:41,394][00611] Updated weights for policy 0, policy_version 81942 (0.0008) [2023-10-08 07:07:41,538][00612] Updated weights for policy 1, policy_version 82390 (0.0008) [2023-10-08 07:07:41,758][00611] Updated weights for policy 0, policy_version 81952 (0.0007) [2023-10-08 07:07:41,904][00612] Updated weights for policy 1, policy_version 82400 (0.0007) [2023-10-08 07:07:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 168296448. Throughput: 0: 1822.4, 1: 1836.0. Samples: 42079322. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-08 07:07:43,754][130385] Avg episode reward: [(0, '87.060'), (1, '68.330')] [2023-10-08 07:07:45,499][00611] Updated weights for policy 0, policy_version 81962 (0.0010) [2023-10-08 07:07:45,504][00612] Updated weights for policy 1, policy_version 82410 (0.0007) [2023-10-08 07:07:45,868][00612] Updated weights for policy 1, policy_version 82420 (0.0007) [2023-10-08 07:07:45,868][00611] Updated weights for policy 0, policy_version 81972 (0.0009) [2023-10-08 07:07:46,239][00611] Updated weights for policy 0, policy_version 81982 (0.0009) [2023-10-08 07:07:46,244][00612] Updated weights for policy 1, policy_version 82430 (0.0007) [2023-10-08 07:07:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 168361984. Throughput: 0: 1809.8, 1: 1842.0. Samples: 42102218. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-08 07:07:48,754][130385] Avg episode reward: [(0, '88.690'), (1, '71.580')] [2023-10-08 07:07:49,964][00612] Updated weights for policy 1, policy_version 82440 (0.0007) [2023-10-08 07:07:50,057][00611] Updated weights for policy 0, policy_version 81992 (0.0008) [2023-10-08 07:07:50,335][00612] Updated weights for policy 1, policy_version 82450 (0.0008) [2023-10-08 07:07:50,426][00611] Updated weights for policy 0, policy_version 82002 (0.0009) [2023-10-08 07:07:50,706][00612] Updated weights for policy 1, policy_version 82460 (0.0008) [2023-10-08 07:07:50,783][00611] Updated weights for policy 0, policy_version 82012 (0.0008) [2023-10-08 07:07:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 168427520. Throughput: 0: 1812.0, 1: 1841.0. Samples: 42112338. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-08 07:07:53,754][130385] Avg episode reward: [(0, '87.930'), (1, '69.280')] [2023-10-08 07:07:54,436][00612] Updated weights for policy 1, policy_version 82470 (0.0008) [2023-10-08 07:07:54,555][00611] Updated weights for policy 0, policy_version 82022 (0.0008) [2023-10-08 07:07:54,808][00612] Updated weights for policy 1, policy_version 82480 (0.0009) [2023-10-08 07:07:54,922][00611] Updated weights for policy 0, policy_version 82032 (0.0007) [2023-10-08 07:07:55,170][00612] Updated weights for policy 1, policy_version 82490 (0.0007) [2023-10-08 07:07:55,291][00611] Updated weights for policy 0, policy_version 82042 (0.0008) [2023-10-08 07:07:58,741][00612] Updated weights for policy 1, policy_version 82500 (0.0009) [2023-10-08 07:07:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 168493056. Throughput: 0: 1804.5, 1: 1846.1. Samples: 42135344. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-08 07:07:58,754][130385] Avg episode reward: [(0, '84.820'), (1, '70.030')] [2023-10-08 07:07:58,939][00611] Updated weights for policy 0, policy_version 82052 (0.0009) [2023-10-08 07:07:59,106][00612] Updated weights for policy 1, policy_version 82510 (0.0007) [2023-10-08 07:07:59,315][00611] Updated weights for policy 0, policy_version 82062 (0.0008) [2023-10-08 07:07:59,469][00612] Updated weights for policy 1, policy_version 82520 (0.0008) [2023-10-08 07:07:59,680][00611] Updated weights for policy 0, policy_version 82072 (0.0008) [2023-10-08 07:08:03,101][00612] Updated weights for policy 1, policy_version 82530 (0.0009) [2023-10-08 07:08:03,378][00611] Updated weights for policy 0, policy_version 82082 (0.0007) [2023-10-08 07:08:03,468][00612] Updated weights for policy 1, policy_version 82540 (0.0008) [2023-10-08 07:08:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 168558592. Throughput: 0: 1809.0, 1: 1841.2. Samples: 42158084. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-08 07:08:03,754][130385] Avg episode reward: [(0, '82.670'), (1, '71.130')] [2023-10-08 07:08:03,755][00611] Updated weights for policy 0, policy_version 82092 (0.0008) [2023-10-08 07:08:03,835][00612] Updated weights for policy 1, policy_version 82550 (0.0010) [2023-10-08 07:08:04,120][00611] Updated weights for policy 0, policy_version 82102 (0.0008) [2023-10-08 07:08:04,196][00612] Updated weights for policy 1, policy_version 82560 (0.0007) [2023-10-08 07:08:04,487][00611] Updated weights for policy 0, policy_version 82112 (0.0008) [2023-10-08 07:08:07,683][00612] Updated weights for policy 1, policy_version 82570 (0.0010) [2023-10-08 07:08:08,051][00612] Updated weights for policy 1, policy_version 82580 (0.0009) [2023-10-08 07:08:08,205][00611] Updated weights for policy 0, policy_version 82122 (0.0008) [2023-10-08 07:08:08,422][00612] Updated weights for policy 1, policy_version 82590 (0.0008) [2023-10-08 07:08:08,583][00611] Updated weights for policy 0, policy_version 82132 (0.0009) [2023-10-08 07:08:08,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 168656896. Throughput: 0: 1810.1, 1: 1848.0. Samples: 42168352. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-08 07:08:08,754][130385] Avg episode reward: [(0, '80.110'), (1, '71.030')] [2023-10-08 07:08:08,948][00611] Updated weights for policy 0, policy_version 82142 (0.0007) [2023-10-08 07:08:12,121][00612] Updated weights for policy 1, policy_version 82600 (0.0007) [2023-10-08 07:08:12,485][00612] Updated weights for policy 1, policy_version 82610 (0.0007) [2023-10-08 07:08:12,579][00611] Updated weights for policy 0, policy_version 82152 (0.0009) [2023-10-08 07:08:12,854][00612] Updated weights for policy 1, policy_version 82620 (0.0008) [2023-10-08 07:08:12,951][00611] Updated weights for policy 0, policy_version 82162 (0.0008) [2023-10-08 07:08:13,326][00611] Updated weights for policy 0, policy_version 82172 (0.0008) [2023-10-08 07:08:13,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 168755200. Throughput: 0: 1817.6, 1: 1840.9. Samples: 42191132. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-08 07:08:13,754][130385] Avg episode reward: [(0, '74.060'), (1, '70.990')] [2023-10-08 07:08:16,606][00612] Updated weights for policy 1, policy_version 82630 (0.0007) [2023-10-08 07:08:16,978][00612] Updated weights for policy 1, policy_version 82640 (0.0007) [2023-10-08 07:08:17,005][00611] Updated weights for policy 0, policy_version 82182 (0.0007) [2023-10-08 07:08:17,355][00612] Updated weights for policy 1, policy_version 82650 (0.0007) [2023-10-08 07:08:17,375][00611] Updated weights for policy 0, policy_version 82192 (0.0008) [2023-10-08 07:08:17,746][00611] Updated weights for policy 0, policy_version 82202 (0.0008) [2023-10-08 07:08:18,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 168820736. Throughput: 0: 1813.4, 1: 1849.2. Samples: 42211126. Policy #0 lag: (min: 0.0, avg: 24.6, max: 32.0) [2023-10-08 07:08:18,755][130385] Avg episode reward: [(0, '78.400'), (1, '72.800')] [2023-10-08 07:08:18,768][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000082208_84180992.pth... [2023-10-08 07:08:18,768][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000082656_84639744.pth... [2023-10-08 07:08:18,807][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000080928_82870272.pth [2023-10-08 07:08:18,809][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000080480_82411520.pth [2023-10-08 07:08:20,823][00612] Updated weights for policy 1, policy_version 82660 (0.0012) [2023-10-08 07:08:21,189][00612] Updated weights for policy 1, policy_version 82670 (0.0009) [2023-10-08 07:08:21,487][00611] Updated weights for policy 0, policy_version 82212 (0.0008) [2023-10-08 07:08:21,552][00612] Updated weights for policy 1, policy_version 82680 (0.0008) [2023-10-08 07:08:21,861][00611] Updated weights for policy 0, policy_version 82222 (0.0007) [2023-10-08 07:08:22,234][00611] Updated weights for policy 0, policy_version 82232 (0.0007) [2023-10-08 07:08:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 168886272. Throughput: 0: 1810.7, 1: 1831.1. Samples: 42223536. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 07:08:23,754][130385] Avg episode reward: [(0, '77.470'), (1, '72.750')] [2023-10-08 07:08:25,128][00612] Updated weights for policy 1, policy_version 82690 (0.0009) [2023-10-08 07:08:25,492][00612] Updated weights for policy 1, policy_version 82700 (0.0009) [2023-10-08 07:08:25,856][00612] Updated weights for policy 1, policy_version 82710 (0.0009) [2023-10-08 07:08:25,914][00611] Updated weights for policy 0, policy_version 82242 (0.0008) [2023-10-08 07:08:26,221][00612] Updated weights for policy 1, policy_version 82720 (0.0010) [2023-10-08 07:08:26,284][00611] Updated weights for policy 0, policy_version 82252 (0.0007) [2023-10-08 07:08:26,670][00611] Updated weights for policy 0, policy_version 82262 (0.0010) [2023-10-08 07:08:27,034][00611] Updated weights for policy 0, policy_version 82272 (0.0010) [2023-10-08 07:08:28,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 168951808. Throughput: 0: 1810.0, 1: 1858.4. Samples: 42244400. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 07:08:28,755][130385] Avg episode reward: [(0, '74.980'), (1, '74.560')] [2023-10-08 07:08:29,852][00612] Updated weights for policy 1, policy_version 82730 (0.0008) [2023-10-08 07:08:30,218][00612] Updated weights for policy 1, policy_version 82740 (0.0008) [2023-10-08 07:08:30,583][00612] Updated weights for policy 1, policy_version 82750 (0.0008) [2023-10-08 07:08:30,690][00611] Updated weights for policy 0, policy_version 82282 (0.0009) [2023-10-08 07:08:31,060][00611] Updated weights for policy 0, policy_version 82292 (0.0008) [2023-10-08 07:08:31,436][00611] Updated weights for policy 0, policy_version 82302 (0.0008) [2023-10-08 07:08:33,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 169017344. Throughput: 0: 1822.8, 1: 1852.6. Samples: 42267612. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 07:08:33,755][130385] Avg episode reward: [(0, '75.230'), (1, '73.440')] [2023-10-08 07:08:34,224][00612] Updated weights for policy 1, policy_version 82760 (0.0008) [2023-10-08 07:08:34,587][00612] Updated weights for policy 1, policy_version 82770 (0.0008) [2023-10-08 07:08:34,966][00612] Updated weights for policy 1, policy_version 82780 (0.0008) [2023-10-08 07:08:35,035][00611] Updated weights for policy 0, policy_version 82312 (0.0008) [2023-10-08 07:08:35,405][00611] Updated weights for policy 0, policy_version 82322 (0.0008) [2023-10-08 07:08:35,782][00611] Updated weights for policy 0, policy_version 82332 (0.0008) [2023-10-08 07:08:38,531][00612] Updated weights for policy 1, policy_version 82790 (0.0009) [2023-10-08 07:08:38,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 169082880. Throughput: 0: 1823.1, 1: 1851.0. Samples: 42277676. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 07:08:38,755][130385] Avg episode reward: [(0, '74.240'), (1, '72.030')] [2023-10-08 07:08:38,893][00612] Updated weights for policy 1, policy_version 82800 (0.0010) [2023-10-08 07:08:39,253][00612] Updated weights for policy 1, policy_version 82810 (0.0011) [2023-10-08 07:08:39,394][00611] Updated weights for policy 0, policy_version 82342 (0.0008) [2023-10-08 07:08:39,763][00611] Updated weights for policy 0, policy_version 82352 (0.0009) [2023-10-08 07:08:40,131][00611] Updated weights for policy 0, policy_version 82362 (0.0007) [2023-10-08 07:08:42,973][00612] Updated weights for policy 1, policy_version 82820 (0.0008) [2023-10-08 07:08:43,345][00612] Updated weights for policy 1, policy_version 82830 (0.0010) [2023-10-08 07:08:43,714][00612] Updated weights for policy 1, policy_version 82840 (0.0010) [2023-10-08 07:08:43,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 169148416. Throughput: 0: 1826.4, 1: 1848.4. Samples: 42300708. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 07:08:43,754][130385] Avg episode reward: [(0, '76.890'), (1, '71.990')] [2023-10-08 07:08:43,824][00611] Updated weights for policy 0, policy_version 82372 (0.0008) [2023-10-08 07:08:44,192][00611] Updated weights for policy 0, policy_version 82382 (0.0009) [2023-10-08 07:08:44,565][00611] Updated weights for policy 0, policy_version 82392 (0.0010) [2023-10-08 07:08:47,419][00612] Updated weights for policy 1, policy_version 82850 (0.0008) [2023-10-08 07:08:47,784][00612] Updated weights for policy 1, policy_version 82860 (0.0007) [2023-10-08 07:08:48,131][00611] Updated weights for policy 0, policy_version 82402 (0.0007) [2023-10-08 07:08:48,143][00612] Updated weights for policy 1, policy_version 82870 (0.0008) [2023-10-08 07:08:48,503][00611] Updated weights for policy 0, policy_version 82412 (0.0007) [2023-10-08 07:08:48,510][00612] Updated weights for policy 1, policy_version 82880 (0.0009) [2023-10-08 07:08:48,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 169246720. Throughput: 0: 1831.5, 1: 1830.7. Samples: 42322884. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 07:08:48,755][130385] Avg episode reward: [(0, '70.600'), (1, '73.360')] [2023-10-08 07:08:48,874][00611] Updated weights for policy 0, policy_version 82422 (0.0007) [2023-10-08 07:08:49,246][00611] Updated weights for policy 0, policy_version 82432 (0.0008) [2023-10-08 07:08:52,101][00612] Updated weights for policy 1, policy_version 82890 (0.0010) [2023-10-08 07:08:52,469][00612] Updated weights for policy 1, policy_version 82900 (0.0009) [2023-10-08 07:08:52,837][00612] Updated weights for policy 1, policy_version 82910 (0.0007) [2023-10-08 07:08:52,935][00611] Updated weights for policy 0, policy_version 82442 (0.0007) [2023-10-08 07:08:53,306][00611] Updated weights for policy 0, policy_version 82452 (0.0008) [2023-10-08 07:08:53,668][00611] Updated weights for policy 0, policy_version 82462 (0.0010) [2023-10-08 07:08:53,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 169345024. Throughput: 0: 1835.0, 1: 1851.1. Samples: 42334226. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 07:08:53,755][130385] Avg episode reward: [(0, '70.920'), (1, '75.110')] [2023-10-08 07:08:56,374][00612] Updated weights for policy 1, policy_version 82920 (0.0008) [2023-10-08 07:08:56,746][00612] Updated weights for policy 1, policy_version 82930 (0.0007) [2023-10-08 07:08:57,112][00612] Updated weights for policy 1, policy_version 82940 (0.0009) [2023-10-08 07:08:57,437][00611] Updated weights for policy 0, policy_version 82472 (0.0008) [2023-10-08 07:08:57,817][00611] Updated weights for policy 0, policy_version 82482 (0.0008) [2023-10-08 07:08:58,180][00611] Updated weights for policy 0, policy_version 82492 (0.0009) [2023-10-08 07:08:58,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 169410560. Throughput: 0: 1825.0, 1: 1834.6. Samples: 42355814. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 07:08:58,755][130385] Avg episode reward: [(0, '73.540'), (1, '75.310')] [2023-10-08 07:09:00,642][00612] Updated weights for policy 1, policy_version 82950 (0.0009) [2023-10-08 07:09:01,011][00612] Updated weights for policy 1, policy_version 82960 (0.0008) [2023-10-08 07:09:01,378][00612] Updated weights for policy 1, policy_version 82970 (0.0008) [2023-10-08 07:09:01,861][00611] Updated weights for policy 0, policy_version 82502 (0.0008) [2023-10-08 07:09:02,232][00611] Updated weights for policy 0, policy_version 82512 (0.0011) [2023-10-08 07:09:02,603][00611] Updated weights for policy 0, policy_version 82522 (0.0010) [2023-10-08 07:09:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 169476096. Throughput: 0: 1829.8, 1: 1862.9. Samples: 42377296. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 07:09:03,754][130385] Avg episode reward: [(0, '74.080'), (1, '81.830')] [2023-10-08 07:09:04,976][00612] Updated weights for policy 1, policy_version 82980 (0.0010) [2023-10-08 07:09:05,342][00612] Updated weights for policy 1, policy_version 82990 (0.0007) [2023-10-08 07:09:05,712][00612] Updated weights for policy 1, policy_version 83000 (0.0008) [2023-10-08 07:09:06,284][00611] Updated weights for policy 0, policy_version 82532 (0.0008) [2023-10-08 07:09:06,659][00611] Updated weights for policy 0, policy_version 82542 (0.0008) [2023-10-08 07:09:07,038][00611] Updated weights for policy 0, policy_version 82552 (0.0009) [2023-10-08 07:09:08,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 169541632. Throughput: 0: 1833.6, 1: 1841.7. Samples: 42388924. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 07:09:08,754][130385] Avg episode reward: [(0, '75.820'), (1, '81.800')] [2023-10-08 07:09:09,343][00612] Updated weights for policy 1, policy_version 83010 (0.0008) [2023-10-08 07:09:09,711][00612] Updated weights for policy 1, policy_version 83020 (0.0008) [2023-10-08 07:09:10,080][00612] Updated weights for policy 1, policy_version 83030 (0.0009) [2023-10-08 07:09:10,448][00612] Updated weights for policy 1, policy_version 83040 (0.0008) [2023-10-08 07:09:10,751][00611] Updated weights for policy 0, policy_version 82562 (0.0008) [2023-10-08 07:09:11,120][00611] Updated weights for policy 0, policy_version 82572 (0.0008) [2023-10-08 07:09:11,498][00611] Updated weights for policy 0, policy_version 82582 (0.0007) [2023-10-08 07:09:11,859][00611] Updated weights for policy 0, policy_version 82592 (0.0007) [2023-10-08 07:09:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 169607168. Throughput: 0: 1834.9, 1: 1856.0. Samples: 42410490. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 07:09:13,754][130385] Avg episode reward: [(0, '75.180'), (1, '83.480')] [2023-10-08 07:09:14,079][00612] Updated weights for policy 1, policy_version 83050 (0.0007) [2023-10-08 07:09:14,447][00612] Updated weights for policy 1, policy_version 83060 (0.0011) [2023-10-08 07:09:14,815][00612] Updated weights for policy 1, policy_version 83070 (0.0010) [2023-10-08 07:09:15,534][00611] Updated weights for policy 0, policy_version 82602 (0.0008) [2023-10-08 07:09:15,898][00611] Updated weights for policy 0, policy_version 82612 (0.0009) [2023-10-08 07:09:16,276][00611] Updated weights for policy 0, policy_version 82622 (0.0007) [2023-10-08 07:09:18,384][00612] Updated weights for policy 1, policy_version 83080 (0.0010) [2023-10-08 07:09:18,752][00612] Updated weights for policy 1, policy_version 83090 (0.0008) [2023-10-08 07:09:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 169672704. Throughput: 0: 1839.1, 1: 1859.3. Samples: 42434038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:09:18,754][130385] Avg episode reward: [(0, '78.270'), (1, '84.900')] [2023-10-08 07:09:19,110][00612] Updated weights for policy 1, policy_version 83100 (0.0010) [2023-10-08 07:09:19,768][00611] Updated weights for policy 0, policy_version 82632 (0.0008) [2023-10-08 07:09:20,140][00611] Updated weights for policy 0, policy_version 82642 (0.0008) [2023-10-08 07:09:20,508][00611] Updated weights for policy 0, policy_version 82652 (0.0010) [2023-10-08 07:09:22,804][00612] Updated weights for policy 1, policy_version 83110 (0.0009) [2023-10-08 07:09:23,160][00612] Updated weights for policy 1, policy_version 83120 (0.0007) [2023-10-08 07:09:23,533][00612] Updated weights for policy 1, policy_version 83130 (0.0008) [2023-10-08 07:09:23,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 169771008. Throughput: 0: 1836.5, 1: 1859.3. Samples: 42443988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:09:23,756][130385] Avg episode reward: [(0, '82.020'), (1, '84.990')] [2023-10-08 07:09:24,187][00611] Updated weights for policy 0, policy_version 82662 (0.0008) [2023-10-08 07:09:24,545][00611] Updated weights for policy 0, policy_version 82672 (0.0008) [2023-10-08 07:09:24,925][00611] Updated weights for policy 0, policy_version 82682 (0.0008) [2023-10-08 07:09:27,074][00612] Updated weights for policy 1, policy_version 83140 (0.0008) [2023-10-08 07:09:27,446][00612] Updated weights for policy 1, policy_version 83150 (0.0009) [2023-10-08 07:09:27,809][00612] Updated weights for policy 1, policy_version 83160 (0.0008) [2023-10-08 07:09:28,496][00611] Updated weights for policy 0, policy_version 82692 (0.0009) [2023-10-08 07:09:28,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 169836544. Throughput: 0: 1834.7, 1: 1855.9. Samples: 42466788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:09:28,755][130385] Avg episode reward: [(0, '83.700'), (1, '88.700')] [2023-10-08 07:09:28,862][00611] Updated weights for policy 0, policy_version 82702 (0.0009) [2023-10-08 07:09:29,233][00611] Updated weights for policy 0, policy_version 82712 (0.0008) [2023-10-08 07:09:31,415][00612] Updated weights for policy 1, policy_version 83170 (0.0007) [2023-10-08 07:09:31,789][00612] Updated weights for policy 1, policy_version 83180 (0.0007) [2023-10-08 07:09:32,148][00612] Updated weights for policy 1, policy_version 83190 (0.0007) [2023-10-08 07:09:32,515][00612] Updated weights for policy 1, policy_version 83200 (0.0007) [2023-10-08 07:09:32,927][00611] Updated weights for policy 0, policy_version 82722 (0.0007) [2023-10-08 07:09:33,288][00611] Updated weights for policy 0, policy_version 82732 (0.0007) [2023-10-08 07:09:33,659][00611] Updated weights for policy 0, policy_version 82742 (0.0007) [2023-10-08 07:09:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 169902080. Throughput: 0: 1825.5, 1: 1855.9. Samples: 42488546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:09:33,754][130385] Avg episode reward: [(0, '84.080'), (1, '84.330')] [2023-10-08 07:09:34,028][00611] Updated weights for policy 0, policy_version 82752 (0.0008) [2023-10-08 07:09:36,152][00612] Updated weights for policy 1, policy_version 83210 (0.0008) [2023-10-08 07:09:36,521][00612] Updated weights for policy 1, policy_version 83220 (0.0007) [2023-10-08 07:09:36,889][00612] Updated weights for policy 1, policy_version 83230 (0.0008) [2023-10-08 07:09:37,640][00611] Updated weights for policy 0, policy_version 82762 (0.0009) [2023-10-08 07:09:38,008][00611] Updated weights for policy 0, policy_version 82772 (0.0012) [2023-10-08 07:09:38,386][00611] Updated weights for policy 0, policy_version 82782 (0.0009) [2023-10-08 07:09:38,754][130385] Fps is (10 sec: 16384.5, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 170000384. Throughput: 0: 1829.7, 1: 1854.7. Samples: 42500020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:09:38,754][130385] Avg episode reward: [(0, '81.070'), (1, '86.020')] [2023-10-08 07:09:40,402][00612] Updated weights for policy 1, policy_version 83240 (0.0008) [2023-10-08 07:09:40,773][00612] Updated weights for policy 1, policy_version 83250 (0.0008) [2023-10-08 07:09:41,142][00612] Updated weights for policy 1, policy_version 83260 (0.0008) [2023-10-08 07:09:42,004][00611] Updated weights for policy 0, policy_version 82792 (0.0010) [2023-10-08 07:09:42,376][00611] Updated weights for policy 0, policy_version 82802 (0.0010) [2023-10-08 07:09:42,742][00611] Updated weights for policy 0, policy_version 82812 (0.0010) [2023-10-08 07:09:43,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 170065920. Throughput: 0: 1825.7, 1: 1865.1. Samples: 42521900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:09:43,755][130385] Avg episode reward: [(0, '81.220'), (1, '88.290')] [2023-10-08 07:09:44,916][00612] Updated weights for policy 1, policy_version 83270 (0.0008) [2023-10-08 07:09:45,283][00612] Updated weights for policy 1, policy_version 83280 (0.0010) [2023-10-08 07:09:45,651][00612] Updated weights for policy 1, policy_version 83290 (0.0009) [2023-10-08 07:09:46,374][00611] Updated weights for policy 0, policy_version 82822 (0.0009) [2023-10-08 07:09:46,750][00611] Updated weights for policy 0, policy_version 82832 (0.0011) [2023-10-08 07:09:47,116][00611] Updated weights for policy 0, policy_version 82842 (0.0011) [2023-10-08 07:09:48,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 170131456. Throughput: 0: 1839.5, 1: 1860.7. Samples: 42543806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:09:48,755][130385] Avg episode reward: [(0, '79.450'), (1, '88.310')] [2023-10-08 07:09:49,270][00612] Updated weights for policy 1, policy_version 83300 (0.0009) [2023-10-08 07:09:49,631][00612] Updated weights for policy 1, policy_version 83310 (0.0010) [2023-10-08 07:09:49,997][00612] Updated weights for policy 1, policy_version 83320 (0.0010) [2023-10-08 07:09:50,896][00611] Updated weights for policy 0, policy_version 82852 (0.0009) [2023-10-08 07:09:51,273][00611] Updated weights for policy 0, policy_version 82862 (0.0009) [2023-10-08 07:09:51,638][00611] Updated weights for policy 0, policy_version 82872 (0.0007) [2023-10-08 07:09:53,522][00612] Updated weights for policy 1, policy_version 83330 (0.0009) [2023-10-08 07:09:53,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 170196992. Throughput: 0: 1826.5, 1: 1860.0. Samples: 42554820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:09:53,754][130385] Avg episode reward: [(0, '77.620'), (1, '88.590')] [2023-10-08 07:09:53,889][00612] Updated weights for policy 1, policy_version 83340 (0.0009) [2023-10-08 07:09:54,263][00612] Updated weights for policy 1, policy_version 83350 (0.0008) [2023-10-08 07:09:54,634][00612] Updated weights for policy 1, policy_version 83360 (0.0009) [2023-10-08 07:09:55,263][00611] Updated weights for policy 0, policy_version 82882 (0.0008) [2023-10-08 07:09:55,641][00611] Updated weights for policy 0, policy_version 82892 (0.0009) [2023-10-08 07:09:56,013][00611] Updated weights for policy 0, policy_version 82902 (0.0008) [2023-10-08 07:09:56,390][00611] Updated weights for policy 0, policy_version 82912 (0.0008) [2023-10-08 07:09:58,213][00612] Updated weights for policy 1, policy_version 83370 (0.0012) [2023-10-08 07:09:58,584][00612] Updated weights for policy 1, policy_version 83380 (0.0009) [2023-10-08 07:09:58,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 170262528. Throughput: 0: 1843.2, 1: 1864.7. Samples: 42577344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:09:58,755][130385] Avg episode reward: [(0, '78.610'), (1, '88.240')] [2023-10-08 07:09:58,940][00612] Updated weights for policy 1, policy_version 83390 (0.0011) [2023-10-08 07:10:00,086][00611] Updated weights for policy 0, policy_version 82922 (0.0008) [2023-10-08 07:10:00,459][00611] Updated weights for policy 0, policy_version 82932 (0.0008) [2023-10-08 07:10:00,831][00611] Updated weights for policy 0, policy_version 82942 (0.0007) [2023-10-08 07:10:02,536][00612] Updated weights for policy 1, policy_version 83400 (0.0009) [2023-10-08 07:10:02,901][00612] Updated weights for policy 1, policy_version 83410 (0.0007) [2023-10-08 07:10:03,275][00612] Updated weights for policy 1, policy_version 83420 (0.0008) [2023-10-08 07:10:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 170360832. Throughput: 0: 1839.0, 1: 1837.2. Samples: 42599468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:10:03,754][130385] Avg episode reward: [(0, '77.710'), (1, '86.550')] [2023-10-08 07:10:04,420][00611] Updated weights for policy 0, policy_version 82952 (0.0009) [2023-10-08 07:10:04,798][00611] Updated weights for policy 0, policy_version 82962 (0.0008) [2023-10-08 07:10:05,162][00611] Updated weights for policy 0, policy_version 82972 (0.0008) [2023-10-08 07:10:06,957][00612] Updated weights for policy 1, policy_version 83430 (0.0008) [2023-10-08 07:10:07,336][00612] Updated weights for policy 1, policy_version 83440 (0.0008) [2023-10-08 07:10:07,712][00612] Updated weights for policy 1, policy_version 83450 (0.0008) [2023-10-08 07:10:08,679][00611] Updated weights for policy 0, policy_version 82982 (0.0009) [2023-10-08 07:10:08,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 170426368. Throughput: 0: 1839.2, 1: 1867.8. Samples: 42610806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:10:08,754][130385] Avg episode reward: [(0, '78.970'), (1, '79.880')] [2023-10-08 07:10:09,053][00611] Updated weights for policy 0, policy_version 82992 (0.0008) [2023-10-08 07:10:09,424][00611] Updated weights for policy 0, policy_version 83002 (0.0009) [2023-10-08 07:10:11,268][00612] Updated weights for policy 1, policy_version 83460 (0.0008) [2023-10-08 07:10:11,641][00612] Updated weights for policy 1, policy_version 83470 (0.0010) [2023-10-08 07:10:12,017][00612] Updated weights for policy 1, policy_version 83480 (0.0007) [2023-10-08 07:10:13,068][00611] Updated weights for policy 0, policy_version 83012 (0.0011) [2023-10-08 07:10:13,439][00611] Updated weights for policy 0, policy_version 83022 (0.0008) [2023-10-08 07:10:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 170491904. Throughput: 0: 1844.7, 1: 1843.3. Samples: 42632744. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 07:10:13,754][130385] Avg episode reward: [(0, '77.770'), (1, '77.470')] [2023-10-08 07:10:13,811][00611] Updated weights for policy 0, policy_version 83032 (0.0008) [2023-10-08 07:10:15,589][00612] Updated weights for policy 1, policy_version 83490 (0.0008) [2023-10-08 07:10:15,953][00612] Updated weights for policy 1, policy_version 83500 (0.0008) [2023-10-08 07:10:16,327][00612] Updated weights for policy 1, policy_version 83510 (0.0009) [2023-10-08 07:10:16,697][00612] Updated weights for policy 1, policy_version 83520 (0.0008) [2023-10-08 07:10:17,410][00611] Updated weights for policy 0, policy_version 83042 (0.0008) [2023-10-08 07:10:17,786][00611] Updated weights for policy 0, policy_version 83052 (0.0008) [2023-10-08 07:10:18,158][00611] Updated weights for policy 0, policy_version 83062 (0.0008) [2023-10-08 07:10:18,518][00611] Updated weights for policy 0, policy_version 83072 (0.0008) [2023-10-08 07:10:18,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 170590208. Throughput: 0: 1834.0, 1: 1870.9. Samples: 42655270. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 07:10:18,755][130385] Avg episode reward: [(0, '78.180'), (1, '80.260')] [2023-10-08 07:10:18,766][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000083520_85524480.pth... [2023-10-08 07:10:18,766][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000083072_85065728.pth... [2023-10-08 07:10:18,810][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000081344_83296256.pth [2023-10-08 07:10:18,810][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000081792_83755008.pth [2023-10-08 07:10:20,209][00612] Updated weights for policy 1, policy_version 83530 (0.0007) [2023-10-08 07:10:20,576][00612] Updated weights for policy 1, policy_version 83540 (0.0008) [2023-10-08 07:10:20,935][00612] Updated weights for policy 1, policy_version 83550 (0.0010) [2023-10-08 07:10:22,095][00611] Updated weights for policy 0, policy_version 83082 (0.0008) [2023-10-08 07:10:22,457][00611] Updated weights for policy 0, policy_version 83092 (0.0007) [2023-10-08 07:10:22,827][00611] Updated weights for policy 0, policy_version 83102 (0.0010) [2023-10-08 07:10:23,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 170655744. Throughput: 0: 1854.5, 1: 1844.3. Samples: 42666468. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 07:10:23,755][130385] Avg episode reward: [(0, '78.260'), (1, '79.670')] [2023-10-08 07:10:24,541][00612] Updated weights for policy 1, policy_version 83560 (0.0009) [2023-10-08 07:10:24,920][00612] Updated weights for policy 1, policy_version 83570 (0.0009) [2023-10-08 07:10:25,293][00612] Updated weights for policy 1, policy_version 83580 (0.0008) [2023-10-08 07:10:26,500][00611] Updated weights for policy 0, policy_version 83112 (0.0008) [2023-10-08 07:10:26,861][00611] Updated weights for policy 0, policy_version 83122 (0.0007) [2023-10-08 07:10:27,239][00611] Updated weights for policy 0, policy_version 83132 (0.0007) [2023-10-08 07:10:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 170721280. Throughput: 0: 1836.9, 1: 1867.9. Samples: 42688616. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 07:10:28,754][130385] Avg episode reward: [(0, '77.770'), (1, '79.520')] [2023-10-08 07:10:28,804][00612] Updated weights for policy 1, policy_version 83590 (0.0008) [2023-10-08 07:10:29,171][00612] Updated weights for policy 1, policy_version 83600 (0.0009) [2023-10-08 07:10:29,547][00612] Updated weights for policy 1, policy_version 83610 (0.0010) [2023-10-08 07:10:30,929][00611] Updated weights for policy 0, policy_version 83142 (0.0009) [2023-10-08 07:10:31,313][00611] Updated weights for policy 0, policy_version 83152 (0.0008) [2023-10-08 07:10:31,681][00611] Updated weights for policy 0, policy_version 83162 (0.0008) [2023-10-08 07:10:33,197][00612] Updated weights for policy 1, policy_version 83620 (0.0010) [2023-10-08 07:10:33,561][00612] Updated weights for policy 1, policy_version 83630 (0.0007) [2023-10-08 07:10:33,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 170786816. Throughput: 0: 1845.4, 1: 1870.1. Samples: 42711004. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 07:10:33,754][130385] Avg episode reward: [(0, '78.080'), (1, '86.120')] [2023-10-08 07:10:33,939][00612] Updated weights for policy 1, policy_version 83640 (0.0010) [2023-10-08 07:10:35,329][00611] Updated weights for policy 0, policy_version 83172 (0.0009) [2023-10-08 07:10:35,707][00611] Updated weights for policy 0, policy_version 83182 (0.0009) [2023-10-08 07:10:36,069][00611] Updated weights for policy 0, policy_version 83192 (0.0007) [2023-10-08 07:10:37,513][00612] Updated weights for policy 1, policy_version 83650 (0.0009) [2023-10-08 07:10:37,881][00612] Updated weights for policy 1, policy_version 83660 (0.0009) [2023-10-08 07:10:38,242][00612] Updated weights for policy 1, policy_version 83670 (0.0010) [2023-10-08 07:10:38,612][00612] Updated weights for policy 1, policy_version 83680 (0.0008) [2023-10-08 07:10:38,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 170885120. Throughput: 0: 1831.9, 1: 1872.7. Samples: 42721532. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 07:10:38,755][130385] Avg episode reward: [(0, '81.510'), (1, '83.020')] [2023-10-08 07:10:39,647][00611] Updated weights for policy 0, policy_version 83202 (0.0007) [2023-10-08 07:10:40,011][00611] Updated weights for policy 0, policy_version 83212 (0.0008) [2023-10-08 07:10:40,387][00611] Updated weights for policy 0, policy_version 83222 (0.0009) [2023-10-08 07:10:40,749][00611] Updated weights for policy 0, policy_version 83232 (0.0010) [2023-10-08 07:10:42,130][00612] Updated weights for policy 1, policy_version 83690 (0.0009) [2023-10-08 07:10:42,492][00612] Updated weights for policy 1, policy_version 83700 (0.0009) [2023-10-08 07:10:42,871][00612] Updated weights for policy 1, policy_version 83710 (0.0007) [2023-10-08 07:10:43,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 170950656. Throughput: 0: 1842.5, 1: 1857.6. Samples: 42743848. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 07:10:43,755][130385] Avg episode reward: [(0, '78.360'), (1, '83.830')] [2023-10-08 07:10:44,494][00611] Updated weights for policy 0, policy_version 83242 (0.0011) [2023-10-08 07:10:44,866][00611] Updated weights for policy 0, policy_version 83252 (0.0008) [2023-10-08 07:10:45,241][00611] Updated weights for policy 0, policy_version 83262 (0.0009) [2023-10-08 07:10:46,440][00612] Updated weights for policy 1, policy_version 83720 (0.0008) [2023-10-08 07:10:46,809][00612] Updated weights for policy 1, policy_version 83730 (0.0007) [2023-10-08 07:10:47,170][00612] Updated weights for policy 1, policy_version 83740 (0.0007) [2023-10-08 07:10:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 171016192. Throughput: 0: 1842.8, 1: 1860.8. Samples: 42766130. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 07:10:48,755][130385] Avg episode reward: [(0, '79.320'), (1, '80.340')] [2023-10-08 07:10:48,872][00611] Updated weights for policy 0, policy_version 83272 (0.0011) [2023-10-08 07:10:49,249][00611] Updated weights for policy 0, policy_version 83282 (0.0007) [2023-10-08 07:10:49,629][00611] Updated weights for policy 0, policy_version 83292 (0.0007) [2023-10-08 07:10:50,834][00612] Updated weights for policy 1, policy_version 83750 (0.0007) [2023-10-08 07:10:51,193][00612] Updated weights for policy 1, policy_version 83760 (0.0008) [2023-10-08 07:10:51,555][00612] Updated weights for policy 1, policy_version 83770 (0.0007) [2023-10-08 07:10:53,379][00611] Updated weights for policy 0, policy_version 83302 (0.0009) [2023-10-08 07:10:53,752][00611] Updated weights for policy 0, policy_version 83312 (0.0009) [2023-10-08 07:10:53,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171081728. Throughput: 0: 1842.7, 1: 1850.1. Samples: 42776982. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 07:10:53,754][130385] Avg episode reward: [(0, '78.080'), (1, '79.980')] [2023-10-08 07:10:54,131][00611] Updated weights for policy 0, policy_version 83322 (0.0011) [2023-10-08 07:10:55,246][00612] Updated weights for policy 1, policy_version 83780 (0.0009) [2023-10-08 07:10:55,612][00612] Updated weights for policy 1, policy_version 83790 (0.0009) [2023-10-08 07:10:55,968][00612] Updated weights for policy 1, policy_version 83800 (0.0010) [2023-10-08 07:10:57,696][00611] Updated weights for policy 0, policy_version 83332 (0.0011) [2023-10-08 07:10:58,071][00611] Updated weights for policy 0, policy_version 83342 (0.0009) [2023-10-08 07:10:58,438][00611] Updated weights for policy 0, policy_version 83352 (0.0009) [2023-10-08 07:10:58,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 171180032. Throughput: 0: 1839.0, 1: 1861.0. Samples: 42799242. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 07:10:58,754][130385] Avg episode reward: [(0, '79.540'), (1, '77.590')] [2023-10-08 07:10:59,615][00612] Updated weights for policy 1, policy_version 83810 (0.0009) [2023-10-08 07:11:00,017][00612] Updated weights for policy 1, policy_version 83820 (0.0010) [2023-10-08 07:11:00,389][00612] Updated weights for policy 1, policy_version 83830 (0.0010) [2023-10-08 07:11:00,752][00612] Updated weights for policy 1, policy_version 83840 (0.0008) [2023-10-08 07:11:02,093][00611] Updated weights for policy 0, policy_version 83362 (0.0009) [2023-10-08 07:11:02,475][00611] Updated weights for policy 0, policy_version 83372 (0.0008) [2023-10-08 07:11:02,850][00611] Updated weights for policy 0, policy_version 83382 (0.0009) [2023-10-08 07:11:03,221][00611] Updated weights for policy 0, policy_version 83392 (0.0008) [2023-10-08 07:11:03,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171245568. Throughput: 0: 1827.3, 1: 1860.0. Samples: 42821202. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:11:03,754][130385] Avg episode reward: [(0, '80.100'), (1, '78.060')] [2023-10-08 07:11:04,322][00612] Updated weights for policy 1, policy_version 83850 (0.0008) [2023-10-08 07:11:04,688][00612] Updated weights for policy 1, policy_version 83860 (0.0007) [2023-10-08 07:11:05,067][00612] Updated weights for policy 1, policy_version 83870 (0.0008) [2023-10-08 07:11:06,691][00611] Updated weights for policy 0, policy_version 83402 (0.0009) [2023-10-08 07:11:07,051][00611] Updated weights for policy 0, policy_version 83412 (0.0008) [2023-10-08 07:11:07,417][00611] Updated weights for policy 0, policy_version 83422 (0.0010) [2023-10-08 07:11:08,735][00612] Updated weights for policy 1, policy_version 83880 (0.0008) [2023-10-08 07:11:08,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171311104. Throughput: 0: 1835.9, 1: 1855.7. Samples: 42832592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:11:08,754][130385] Avg episode reward: [(0, '81.340'), (1, '75.140')] [2023-10-08 07:11:09,096][00612] Updated weights for policy 1, policy_version 83890 (0.0011) [2023-10-08 07:11:09,461][00612] Updated weights for policy 1, policy_version 83900 (0.0009) [2023-10-08 07:11:11,088][00611] Updated weights for policy 0, policy_version 83432 (0.0008) [2023-10-08 07:11:11,454][00611] Updated weights for policy 0, policy_version 83442 (0.0009) [2023-10-08 07:11:11,826][00611] Updated weights for policy 0, policy_version 83452 (0.0008) [2023-10-08 07:11:13,144][00612] Updated weights for policy 1, policy_version 83910 (0.0008) [2023-10-08 07:11:13,517][00612] Updated weights for policy 1, policy_version 83920 (0.0008) [2023-10-08 07:11:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171376640. Throughput: 0: 1826.5, 1: 1851.5. Samples: 42854124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:11:13,754][130385] Avg episode reward: [(0, '81.530'), (1, '74.250')] [2023-10-08 07:11:13,881][00612] Updated weights for policy 1, policy_version 83930 (0.0009) [2023-10-08 07:11:15,429][00611] Updated weights for policy 0, policy_version 83462 (0.0010) [2023-10-08 07:11:15,795][00611] Updated weights for policy 0, policy_version 83472 (0.0011) [2023-10-08 07:11:16,172][00611] Updated weights for policy 0, policy_version 83482 (0.0008) [2023-10-08 07:11:17,383][00612] Updated weights for policy 1, policy_version 83940 (0.0007) [2023-10-08 07:11:17,756][00612] Updated weights for policy 1, policy_version 83950 (0.0007) [2023-10-08 07:11:18,125][00612] Updated weights for policy 1, policy_version 83960 (0.0008) [2023-10-08 07:11:18,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 171474944. Throughput: 0: 1849.1, 1: 1830.4. Samples: 42876580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:11:18,754][130385] Avg episode reward: [(0, '76.840'), (1, '73.400')] [2023-10-08 07:11:19,724][00611] Updated weights for policy 0, policy_version 83492 (0.0007) [2023-10-08 07:11:20,108][00611] Updated weights for policy 0, policy_version 83502 (0.0010) [2023-10-08 07:11:20,478][00611] Updated weights for policy 0, policy_version 83512 (0.0008) [2023-10-08 07:11:21,714][00612] Updated weights for policy 1, policy_version 83970 (0.0008) [2023-10-08 07:11:22,078][00612] Updated weights for policy 1, policy_version 83980 (0.0007) [2023-10-08 07:11:22,452][00612] Updated weights for policy 1, policy_version 83990 (0.0007) [2023-10-08 07:11:22,817][00612] Updated weights for policy 1, policy_version 84000 (0.0007) [2023-10-08 07:11:23,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 171540480. Throughput: 0: 1840.9, 1: 1856.6. Samples: 42887918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:11:23,754][130385] Avg episode reward: [(0, '77.790'), (1, '66.160')] [2023-10-08 07:11:24,239][00611] Updated weights for policy 0, policy_version 83522 (0.0008) [2023-10-08 07:11:24,616][00611] Updated weights for policy 0, policy_version 83532 (0.0009) [2023-10-08 07:11:24,995][00611] Updated weights for policy 0, policy_version 83542 (0.0010) [2023-10-08 07:11:25,370][00611] Updated weights for policy 0, policy_version 83552 (0.0010) [2023-10-08 07:11:26,435][00612] Updated weights for policy 1, policy_version 84010 (0.0010) [2023-10-08 07:11:26,819][00612] Updated weights for policy 1, policy_version 84020 (0.0010) [2023-10-08 07:11:27,192][00612] Updated weights for policy 1, policy_version 84030 (0.0007) [2023-10-08 07:11:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 171606016. Throughput: 0: 1844.7, 1: 1833.6. Samples: 42909368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:11:28,754][130385] Avg episode reward: [(0, '79.120'), (1, '64.700')] [2023-10-08 07:11:29,119][00611] Updated weights for policy 0, policy_version 83562 (0.0007) [2023-10-08 07:11:29,486][00611] Updated weights for policy 0, policy_version 83572 (0.0008) [2023-10-08 07:11:29,859][00611] Updated weights for policy 0, policy_version 83582 (0.0009) [2023-10-08 07:11:30,869][00612] Updated weights for policy 1, policy_version 84040 (0.0009) [2023-10-08 07:11:31,226][00612] Updated weights for policy 1, policy_version 84050 (0.0007) [2023-10-08 07:11:31,595][00612] Updated weights for policy 1, policy_version 84060 (0.0007) [2023-10-08 07:11:33,529][00611] Updated weights for policy 0, policy_version 83592 (0.0008) [2023-10-08 07:11:33,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 171671552. Throughput: 0: 1838.9, 1: 1856.0. Samples: 42932404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:11:33,757][130385] Avg episode reward: [(0, '77.610'), (1, '63.500')] [2023-10-08 07:11:33,897][00611] Updated weights for policy 0, policy_version 83602 (0.0008) [2023-10-08 07:11:34,277][00611] Updated weights for policy 0, policy_version 83612 (0.0007) [2023-10-08 07:11:35,123][00612] Updated weights for policy 1, policy_version 84070 (0.0009) [2023-10-08 07:11:35,494][00612] Updated weights for policy 1, policy_version 84080 (0.0010) [2023-10-08 07:11:35,861][00612] Updated weights for policy 1, policy_version 84090 (0.0010) [2023-10-08 07:11:37,885][00611] Updated weights for policy 0, policy_version 83622 (0.0008) [2023-10-08 07:11:38,254][00611] Updated weights for policy 0, policy_version 83632 (0.0008) [2023-10-08 07:11:38,628][00611] Updated weights for policy 0, policy_version 83642 (0.0010) [2023-10-08 07:11:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 171737088. Throughput: 0: 1840.4, 1: 1839.2. Samples: 42942564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:11:38,754][130385] Avg episode reward: [(0, '79.830'), (1, '64.900')] [2023-10-08 07:11:39,554][00612] Updated weights for policy 1, policy_version 84100 (0.0011) [2023-10-08 07:11:39,931][00612] Updated weights for policy 1, policy_version 84110 (0.0008) [2023-10-08 07:11:40,300][00612] Updated weights for policy 1, policy_version 84120 (0.0008) [2023-10-08 07:11:42,428][00611] Updated weights for policy 0, policy_version 83652 (0.0010) [2023-10-08 07:11:42,802][00611] Updated weights for policy 0, policy_version 83662 (0.0009) [2023-10-08 07:11:43,164][00611] Updated weights for policy 0, policy_version 83672 (0.0007) [2023-10-08 07:11:43,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171835392. Throughput: 0: 1837.3, 1: 1859.2. Samples: 42965582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:11:43,755][130385] Avg episode reward: [(0, '75.220'), (1, '64.240')] [2023-10-08 07:11:43,987][00612] Updated weights for policy 1, policy_version 84130 (0.0008) [2023-10-08 07:11:44,363][00612] Updated weights for policy 1, policy_version 84140 (0.0007) [2023-10-08 07:11:44,730][00612] Updated weights for policy 1, policy_version 84150 (0.0007) [2023-10-08 07:11:45,104][00612] Updated weights for policy 1, policy_version 84160 (0.0007) [2023-10-08 07:11:46,534][00611] Updated weights for policy 0, policy_version 83682 (0.0007) [2023-10-08 07:11:46,916][00611] Updated weights for policy 0, policy_version 83692 (0.0011) [2023-10-08 07:11:47,285][00611] Updated weights for policy 0, policy_version 83702 (0.0007) [2023-10-08 07:11:47,656][00611] Updated weights for policy 0, policy_version 83712 (0.0008) [2023-10-08 07:11:48,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171900928. Throughput: 0: 1839.4, 1: 1859.5. Samples: 42987652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:11:48,755][130385] Avg episode reward: [(0, '75.310'), (1, '64.830')] [2023-10-08 07:11:48,769][00612] Updated weights for policy 1, policy_version 84170 (0.0009) [2023-10-08 07:11:49,133][00612] Updated weights for policy 1, policy_version 84180 (0.0008) [2023-10-08 07:11:49,503][00612] Updated weights for policy 1, policy_version 84190 (0.0008) [2023-10-08 07:11:51,218][00611] Updated weights for policy 0, policy_version 83722 (0.0007) [2023-10-08 07:11:51,593][00611] Updated weights for policy 0, policy_version 83732 (0.0008) [2023-10-08 07:11:51,959][00611] Updated weights for policy 0, policy_version 83742 (0.0007) [2023-10-08 07:11:52,985][00612] Updated weights for policy 1, policy_version 84200 (0.0009) [2023-10-08 07:11:53,361][00612] Updated weights for policy 1, policy_version 84210 (0.0007) [2023-10-08 07:11:53,725][00612] Updated weights for policy 1, policy_version 84220 (0.0008) [2023-10-08 07:11:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171966464. Throughput: 0: 1831.6, 1: 1860.8. Samples: 42998754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:11:53,755][130385] Avg episode reward: [(0, '71.840'), (1, '66.550')] [2023-10-08 07:11:55,674][00611] Updated weights for policy 0, policy_version 83752 (0.0009) [2023-10-08 07:11:56,041][00611] Updated weights for policy 0, policy_version 83762 (0.0008) [2023-10-08 07:11:56,416][00611] Updated weights for policy 0, policy_version 83772 (0.0008) [2023-10-08 07:11:57,239][00612] Updated weights for policy 1, policy_version 84230 (0.0009) [2023-10-08 07:11:57,613][00612] Updated weights for policy 1, policy_version 84240 (0.0008) [2023-10-08 07:11:57,983][00612] Updated weights for policy 1, policy_version 84250 (0.0010) [2023-10-08 07:11:58,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 172064768. Throughput: 0: 1842.4, 1: 1862.9. Samples: 43020862. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 07:11:58,754][130385] Avg episode reward: [(0, '74.070'), (1, '66.520')] [2023-10-08 07:11:59,957][00611] Updated weights for policy 0, policy_version 83782 (0.0008) [2023-10-08 07:12:00,331][00611] Updated weights for policy 0, policy_version 83792 (0.0011) [2023-10-08 07:12:00,695][00611] Updated weights for policy 0, policy_version 83802 (0.0009) [2023-10-08 07:12:01,763][00612] Updated weights for policy 1, policy_version 84260 (0.0010) [2023-10-08 07:12:02,131][00612] Updated weights for policy 1, policy_version 84270 (0.0009) [2023-10-08 07:12:02,492][00612] Updated weights for policy 1, policy_version 84280 (0.0008) [2023-10-08 07:12:03,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 172130304. Throughput: 0: 1834.8, 1: 1849.5. Samples: 43042374. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 07:12:03,755][130385] Avg episode reward: [(0, '71.090'), (1, '68.360')] [2023-10-08 07:12:04,642][00611] Updated weights for policy 0, policy_version 83812 (0.0009) [2023-10-08 07:12:05,019][00611] Updated weights for policy 0, policy_version 83822 (0.0010) [2023-10-08 07:12:05,390][00611] Updated weights for policy 0, policy_version 83832 (0.0009) [2023-10-08 07:12:06,090][00612] Updated weights for policy 1, policy_version 84290 (0.0009) [2023-10-08 07:12:06,470][00612] Updated weights for policy 1, policy_version 84300 (0.0009) [2023-10-08 07:12:06,837][00612] Updated weights for policy 1, policy_version 84310 (0.0008) [2023-10-08 07:12:07,202][00612] Updated weights for policy 1, policy_version 84320 (0.0008) [2023-10-08 07:12:08,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 172195840. Throughput: 0: 1828.5, 1: 1852.9. Samples: 43053584. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 07:12:08,755][130385] Avg episode reward: [(0, '73.480'), (1, '68.020')] [2023-10-08 07:12:09,083][00611] Updated weights for policy 0, policy_version 83842 (0.0009) [2023-10-08 07:12:09,460][00611] Updated weights for policy 0, policy_version 83852 (0.0009) [2023-10-08 07:12:09,827][00611] Updated weights for policy 0, policy_version 83862 (0.0008) [2023-10-08 07:12:10,200][00611] Updated weights for policy 0, policy_version 83872 (0.0007) [2023-10-08 07:12:10,718][00612] Updated weights for policy 1, policy_version 84330 (0.0009) [2023-10-08 07:12:11,090][00612] Updated weights for policy 1, policy_version 84340 (0.0008) [2023-10-08 07:12:11,458][00612] Updated weights for policy 1, policy_version 84350 (0.0007) [2023-10-08 07:12:13,717][00611] Updated weights for policy 0, policy_version 83882 (0.0008) [2023-10-08 07:12:13,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 172261376. Throughput: 0: 1836.4, 1: 1855.4. Samples: 43075500. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 07:12:13,754][130385] Avg episode reward: [(0, '70.460'), (1, '71.040')] [2023-10-08 07:12:14,085][00611] Updated weights for policy 0, policy_version 83892 (0.0009) [2023-10-08 07:12:14,462][00611] Updated weights for policy 0, policy_version 83902 (0.0009) [2023-10-08 07:12:15,140][00612] Updated weights for policy 1, policy_version 84360 (0.0008) [2023-10-08 07:12:15,503][00612] Updated weights for policy 1, policy_version 84370 (0.0009) [2023-10-08 07:12:15,872][00612] Updated weights for policy 1, policy_version 84380 (0.0007) [2023-10-08 07:12:17,970][00611] Updated weights for policy 0, policy_version 83912 (0.0011) [2023-10-08 07:12:18,348][00611] Updated weights for policy 0, policy_version 83922 (0.0009) [2023-10-08 07:12:18,719][00611] Updated weights for policy 0, policy_version 83932 (0.0007) [2023-10-08 07:12:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14773.4). Total num frames: 172326912. Throughput: 0: 1827.7, 1: 1851.1. Samples: 43097948. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 07:12:18,755][130385] Avg episode reward: [(0, '69.770'), (1, '69.770')] [2023-10-08 07:12:18,766][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000084384_86409216.pth... [2023-10-08 07:12:18,799][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000082656_84639744.pth [2023-10-08 07:12:18,862][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000083936_85950464.pth... [2023-10-08 07:12:18,891][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000082208_84180992.pth [2023-10-08 07:12:19,480][00612] Updated weights for policy 1, policy_version 84390 (0.0007) [2023-10-08 07:12:19,858][00612] Updated weights for policy 1, policy_version 84400 (0.0008) [2023-10-08 07:12:20,224][00612] Updated weights for policy 1, policy_version 84410 (0.0009) [2023-10-08 07:12:22,360][00611] Updated weights for policy 0, policy_version 83942 (0.0008) [2023-10-08 07:12:22,730][00611] Updated weights for policy 0, policy_version 83952 (0.0009) [2023-10-08 07:12:23,107][00611] Updated weights for policy 0, policy_version 83962 (0.0008) [2023-10-08 07:12:23,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 172425216. Throughput: 0: 1843.0, 1: 1848.9. Samples: 43108700. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 07:12:23,754][130385] Avg episode reward: [(0, '74.680'), (1, '75.960')] [2023-10-08 07:12:23,777][00612] Updated weights for policy 1, policy_version 84420 (0.0011) [2023-10-08 07:12:24,139][00612] Updated weights for policy 1, policy_version 84430 (0.0010) [2023-10-08 07:12:24,501][00612] Updated weights for policy 1, policy_version 84440 (0.0011) [2023-10-08 07:12:26,799][00611] Updated weights for policy 0, policy_version 83972 (0.0007) [2023-10-08 07:12:27,166][00611] Updated weights for policy 0, policy_version 83982 (0.0009) [2023-10-08 07:12:27,536][00611] Updated weights for policy 0, policy_version 83992 (0.0011) [2023-10-08 07:12:28,174][00612] Updated weights for policy 1, policy_version 84450 (0.0011) [2023-10-08 07:12:28,537][00612] Updated weights for policy 1, policy_version 84460 (0.0008) [2023-10-08 07:12:28,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 172490752. Throughput: 0: 1831.6, 1: 1852.8. Samples: 43131378. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 07:12:28,754][130385] Avg episode reward: [(0, '76.260'), (1, '75.060')] [2023-10-08 07:12:28,896][00612] Updated weights for policy 1, policy_version 84470 (0.0007) [2023-10-08 07:12:29,265][00612] Updated weights for policy 1, policy_version 84480 (0.0007) [2023-10-08 07:12:31,063][00611] Updated weights for policy 0, policy_version 84002 (0.0009) [2023-10-08 07:12:31,429][00611] Updated weights for policy 0, policy_version 84012 (0.0008) [2023-10-08 07:12:31,809][00611] Updated weights for policy 0, policy_version 84022 (0.0009) [2023-10-08 07:12:32,172][00611] Updated weights for policy 0, policy_version 84032 (0.0007) [2023-10-08 07:12:32,827][00612] Updated weights for policy 1, policy_version 84490 (0.0007) [2023-10-08 07:12:33,206][00612] Updated weights for policy 1, policy_version 84500 (0.0007) [2023-10-08 07:12:33,568][00612] Updated weights for policy 1, policy_version 84510 (0.0009) [2023-10-08 07:12:33,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 172589056. Throughput: 0: 1847.1, 1: 1833.2. Samples: 43153264. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 07:12:33,754][130385] Avg episode reward: [(0, '73.410'), (1, '78.080')] [2023-10-08 07:12:35,687][00611] Updated weights for policy 0, policy_version 84042 (0.0008) [2023-10-08 07:12:36,062][00611] Updated weights for policy 0, policy_version 84052 (0.0011) [2023-10-08 07:12:36,431][00611] Updated weights for policy 0, policy_version 84062 (0.0008) [2023-10-08 07:12:37,338][00612] Updated weights for policy 1, policy_version 84520 (0.0008) [2023-10-08 07:12:37,725][00612] Updated weights for policy 1, policy_version 84530 (0.0008) [2023-10-08 07:12:38,092][00612] Updated weights for policy 1, policy_version 84540 (0.0008) [2023-10-08 07:12:38,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 172654592. Throughput: 0: 1835.8, 1: 1855.0. Samples: 43164840. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 07:12:38,754][130385] Avg episode reward: [(0, '70.740'), (1, '79.650')] [2023-10-08 07:12:39,876][00611] Updated weights for policy 0, policy_version 84072 (0.0010) [2023-10-08 07:12:40,239][00611] Updated weights for policy 0, policy_version 84082 (0.0008) [2023-10-08 07:12:40,609][00611] Updated weights for policy 0, policy_version 84092 (0.0008) [2023-10-08 07:12:41,818][00612] Updated weights for policy 1, policy_version 84550 (0.0008) [2023-10-08 07:12:42,185][00612] Updated weights for policy 1, policy_version 84560 (0.0009) [2023-10-08 07:12:42,572][00612] Updated weights for policy 1, policy_version 84570 (0.0011) [2023-10-08 07:12:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 172720128. Throughput: 0: 1859.0, 1: 1826.4. Samples: 43186704. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 07:12:43,754][130385] Avg episode reward: [(0, '70.180'), (1, '78.420')] [2023-10-08 07:12:44,094][00611] Updated weights for policy 0, policy_version 84102 (0.0008) [2023-10-08 07:12:44,457][00611] Updated weights for policy 0, policy_version 84112 (0.0008) [2023-10-08 07:12:44,826][00611] Updated weights for policy 0, policy_version 84122 (0.0009) [2023-10-08 07:12:46,155][00612] Updated weights for policy 1, policy_version 84580 (0.0008) [2023-10-08 07:12:46,522][00612] Updated weights for policy 1, policy_version 84590 (0.0008) [2023-10-08 07:12:46,899][00612] Updated weights for policy 1, policy_version 84600 (0.0008) [2023-10-08 07:12:48,518][00611] Updated weights for policy 0, policy_version 84132 (0.0008) [2023-10-08 07:12:48,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 172785664. Throughput: 0: 1863.4, 1: 1843.4. Samples: 43209178. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 07:12:48,755][130385] Avg episode reward: [(0, '72.290'), (1, '76.070')] [2023-10-08 07:12:48,896][00611] Updated weights for policy 0, policy_version 84142 (0.0012) [2023-10-08 07:12:49,257][00611] Updated weights for policy 0, policy_version 84152 (0.0010) [2023-10-08 07:12:50,469][00612] Updated weights for policy 1, policy_version 84610 (0.0009) [2023-10-08 07:12:50,842][00612] Updated weights for policy 1, policy_version 84620 (0.0010) [2023-10-08 07:12:51,222][00612] Updated weights for policy 1, policy_version 84630 (0.0008) [2023-10-08 07:12:51,586][00612] Updated weights for policy 1, policy_version 84640 (0.0008) [2023-10-08 07:12:53,031][00611] Updated weights for policy 0, policy_version 84162 (0.0010) [2023-10-08 07:12:53,432][00611] Updated weights for policy 0, policy_version 84172 (0.0009) [2023-10-08 07:12:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 172851200. Throughput: 0: 1869.9, 1: 1825.6. Samples: 43219878. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:12:53,754][130385] Avg episode reward: [(0, '69.300'), (1, '79.280')] [2023-10-08 07:12:53,807][00611] Updated weights for policy 0, policy_version 84182 (0.0010) [2023-10-08 07:12:54,170][00611] Updated weights for policy 0, policy_version 84192 (0.0011) [2023-10-08 07:12:55,309][00612] Updated weights for policy 1, policy_version 84650 (0.0009) [2023-10-08 07:12:55,676][00612] Updated weights for policy 1, policy_version 84660 (0.0009) [2023-10-08 07:12:56,041][00612] Updated weights for policy 1, policy_version 84670 (0.0008) [2023-10-08 07:12:57,864][00611] Updated weights for policy 0, policy_version 84202 (0.0010) [2023-10-08 07:12:58,238][00611] Updated weights for policy 0, policy_version 84212 (0.0010) [2023-10-08 07:12:58,620][00611] Updated weights for policy 0, policy_version 84222 (0.0009) [2023-10-08 07:12:58,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14884.5). Total num frames: 172949504. Throughput: 0: 1869.2, 1: 1838.8. Samples: 43242362. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:12:58,754][130385] Avg episode reward: [(0, '70.990'), (1, '80.200')] [2023-10-08 07:12:59,559][00612] Updated weights for policy 1, policy_version 84680 (0.0007) [2023-10-08 07:12:59,926][00612] Updated weights for policy 1, policy_version 84690 (0.0008) [2023-10-08 07:13:00,288][00612] Updated weights for policy 1, policy_version 84700 (0.0008) [2023-10-08 07:13:02,197][00611] Updated weights for policy 0, policy_version 84232 (0.0009) [2023-10-08 07:13:02,565][00611] Updated weights for policy 0, policy_version 84242 (0.0007) [2023-10-08 07:13:02,932][00611] Updated weights for policy 0, policy_version 84252 (0.0009) [2023-10-08 07:13:03,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 173015040. Throughput: 0: 1843.3, 1: 1851.5. Samples: 43264214. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:13:03,755][130385] Avg episode reward: [(0, '72.030'), (1, '77.010')] [2023-10-08 07:13:03,988][00612] Updated weights for policy 1, policy_version 84710 (0.0010) [2023-10-08 07:13:04,342][00612] Updated weights for policy 1, policy_version 84720 (0.0010) [2023-10-08 07:13:04,717][00612] Updated weights for policy 1, policy_version 84730 (0.0010) [2023-10-08 07:13:06,554][00611] Updated weights for policy 0, policy_version 84262 (0.0008) [2023-10-08 07:13:06,915][00611] Updated weights for policy 0, policy_version 84272 (0.0009) [2023-10-08 07:13:07,290][00611] Updated weights for policy 0, policy_version 84282 (0.0008) [2023-10-08 07:13:08,383][00612] Updated weights for policy 1, policy_version 84740 (0.0008) [2023-10-08 07:13:08,750][00612] Updated weights for policy 1, policy_version 84750 (0.0008) [2023-10-08 07:13:08,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173080576. Throughput: 0: 1865.2, 1: 1847.3. Samples: 43275762. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:13:08,755][130385] Avg episode reward: [(0, '68.370'), (1, '75.170')] [2023-10-08 07:13:09,112][00612] Updated weights for policy 1, policy_version 84760 (0.0008) [2023-10-08 07:13:10,729][00611] Updated weights for policy 0, policy_version 84292 (0.0007) [2023-10-08 07:13:11,092][00611] Updated weights for policy 0, policy_version 84302 (0.0010) [2023-10-08 07:13:11,470][00611] Updated weights for policy 0, policy_version 84312 (0.0007) [2023-10-08 07:13:12,724][00612] Updated weights for policy 1, policy_version 84770 (0.0007) [2023-10-08 07:13:13,088][00612] Updated weights for policy 1, policy_version 84780 (0.0007) [2023-10-08 07:13:13,454][00612] Updated weights for policy 1, policy_version 84790 (0.0008) [2023-10-08 07:13:13,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173146112. Throughput: 0: 1853.6, 1: 1848.9. Samples: 43297994. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:13:13,754][130385] Avg episode reward: [(0, '71.890'), (1, '76.720')] [2023-10-08 07:13:13,827][00612] Updated weights for policy 1, policy_version 84800 (0.0008) [2023-10-08 07:13:15,013][00611] Updated weights for policy 0, policy_version 84322 (0.0009) [2023-10-08 07:13:15,395][00611] Updated weights for policy 0, policy_version 84332 (0.0009) [2023-10-08 07:13:15,762][00611] Updated weights for policy 0, policy_version 84342 (0.0009) [2023-10-08 07:13:16,133][00611] Updated weights for policy 0, policy_version 84352 (0.0010) [2023-10-08 07:13:17,438][00612] Updated weights for policy 1, policy_version 84810 (0.0011) [2023-10-08 07:13:17,806][00612] Updated weights for policy 1, policy_version 84820 (0.0009) [2023-10-08 07:13:18,170][00612] Updated weights for policy 1, policy_version 84830 (0.0009) [2023-10-08 07:13:18,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 173244416. Throughput: 0: 1874.5, 1: 1833.2. Samples: 43320112. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:13:18,754][130385] Avg episode reward: [(0, '69.810'), (1, '75.250')] [2023-10-08 07:13:19,765][00611] Updated weights for policy 0, policy_version 84362 (0.0010) [2023-10-08 07:13:20,127][00611] Updated weights for policy 0, policy_version 84372 (0.0008) [2023-10-08 07:13:20,491][00611] Updated weights for policy 0, policy_version 84382 (0.0007) [2023-10-08 07:13:21,907][00612] Updated weights for policy 1, policy_version 84840 (0.0009) [2023-10-08 07:13:22,275][00612] Updated weights for policy 1, policy_version 84850 (0.0008) [2023-10-08 07:13:22,640][00612] Updated weights for policy 1, policy_version 84860 (0.0008) [2023-10-08 07:13:23,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 173309952. Throughput: 0: 1852.4, 1: 1845.8. Samples: 43331258. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:13:23,754][130385] Avg episode reward: [(0, '64.410'), (1, '72.340')] [2023-10-08 07:13:24,194][00611] Updated weights for policy 0, policy_version 84392 (0.0010) [2023-10-08 07:13:24,568][00611] Updated weights for policy 0, policy_version 84402 (0.0011) [2023-10-08 07:13:24,953][00611] Updated weights for policy 0, policy_version 84412 (0.0009) [2023-10-08 07:13:26,281][00612] Updated weights for policy 1, policy_version 84870 (0.0008) [2023-10-08 07:13:26,646][00612] Updated weights for policy 1, policy_version 84880 (0.0008) [2023-10-08 07:13:27,020][00612] Updated weights for policy 1, policy_version 84890 (0.0011) [2023-10-08 07:13:28,453][00611] Updated weights for policy 0, policy_version 84422 (0.0009) [2023-10-08 07:13:28,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 173375488. Throughput: 0: 1854.8, 1: 1837.8. Samples: 43352870. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:13:28,755][130385] Avg episode reward: [(0, '66.380'), (1, '73.580')] [2023-10-08 07:13:28,823][00611] Updated weights for policy 0, policy_version 84432 (0.0009) [2023-10-08 07:13:29,190][00611] Updated weights for policy 0, policy_version 84442 (0.0007) [2023-10-08 07:13:30,545][00612] Updated weights for policy 1, policy_version 84900 (0.0008) [2023-10-08 07:13:30,909][00612] Updated weights for policy 1, policy_version 84910 (0.0008) [2023-10-08 07:13:31,282][00612] Updated weights for policy 1, policy_version 84920 (0.0008) [2023-10-08 07:13:32,771][00611] Updated weights for policy 0, policy_version 84452 (0.0010) [2023-10-08 07:13:33,151][00611] Updated weights for policy 0, policy_version 84462 (0.0008) [2023-10-08 07:13:33,515][00611] Updated weights for policy 0, policy_version 84472 (0.0008) [2023-10-08 07:13:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14773.4). Total num frames: 173441024. Throughput: 0: 1838.6, 1: 1851.4. Samples: 43375226. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:13:33,754][130385] Avg episode reward: [(0, '68.150'), (1, '74.550')] [2023-10-08 07:13:34,803][00612] Updated weights for policy 1, policy_version 84930 (0.0008) [2023-10-08 07:13:35,169][00612] Updated weights for policy 1, policy_version 84940 (0.0008) [2023-10-08 07:13:35,542][00612] Updated weights for policy 1, policy_version 84950 (0.0008) [2023-10-08 07:13:35,907][00612] Updated weights for policy 1, policy_version 84960 (0.0009) [2023-10-08 07:13:37,282][00611] Updated weights for policy 0, policy_version 84482 (0.0009) [2023-10-08 07:13:37,648][00611] Updated weights for policy 0, policy_version 84492 (0.0008) [2023-10-08 07:13:38,015][00611] Updated weights for policy 0, policy_version 84502 (0.0010) [2023-10-08 07:13:38,390][00611] Updated weights for policy 0, policy_version 84512 (0.0010) [2023-10-08 07:13:38,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14884.4). Total num frames: 173539328. Throughput: 0: 1852.3, 1: 1838.5. Samples: 43385966. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:13:38,754][130385] Avg episode reward: [(0, '66.440'), (1, '70.570')] [2023-10-08 07:13:39,469][00612] Updated weights for policy 1, policy_version 84970 (0.0009) [2023-10-08 07:13:39,832][00612] Updated weights for policy 1, policy_version 84980 (0.0008) [2023-10-08 07:13:40,201][00612] Updated weights for policy 1, policy_version 84990 (0.0011) [2023-10-08 07:13:42,079][00611] Updated weights for policy 0, policy_version 84522 (0.0009) [2023-10-08 07:13:42,449][00611] Updated weights for policy 0, policy_version 84532 (0.0007) [2023-10-08 07:13:42,820][00611] Updated weights for policy 0, policy_version 84542 (0.0008) [2023-10-08 07:13:43,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 173604864. Throughput: 0: 1832.8, 1: 1858.2. Samples: 43408458. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 07:13:43,755][130385] Avg episode reward: [(0, '68.160'), (1, '72.690')] [2023-10-08 07:13:43,920][00612] Updated weights for policy 1, policy_version 85000 (0.0007) [2023-10-08 07:13:44,290][00612] Updated weights for policy 1, policy_version 85010 (0.0008) [2023-10-08 07:13:44,660][00612] Updated weights for policy 1, policy_version 85020 (0.0008) [2023-10-08 07:13:46,553][00611] Updated weights for policy 0, policy_version 84552 (0.0007) [2023-10-08 07:13:46,923][00611] Updated weights for policy 0, policy_version 84562 (0.0008) [2023-10-08 07:13:47,303][00611] Updated weights for policy 0, policy_version 84572 (0.0009) [2023-10-08 07:13:48,151][00612] Updated weights for policy 1, policy_version 85030 (0.0008) [2023-10-08 07:13:48,522][00612] Updated weights for policy 1, policy_version 85040 (0.0012) [2023-10-08 07:13:48,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173670400. Throughput: 0: 1844.8, 1: 1854.0. Samples: 43430660. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 07:13:48,755][130385] Avg episode reward: [(0, '68.270'), (1, '77.720')] [2023-10-08 07:13:48,885][00612] Updated weights for policy 1, policy_version 85050 (0.0008) [2023-10-08 07:13:50,902][00611] Updated weights for policy 0, policy_version 84582 (0.0009) [2023-10-08 07:13:51,280][00611] Updated weights for policy 0, policy_version 84592 (0.0008) [2023-10-08 07:13:51,657][00611] Updated weights for policy 0, policy_version 84602 (0.0009) [2023-10-08 07:13:52,410][00612] Updated weights for policy 1, policy_version 85060 (0.0007) [2023-10-08 07:13:52,785][00612] Updated weights for policy 1, policy_version 85070 (0.0007) [2023-10-08 07:13:53,155][00612] Updated weights for policy 1, policy_version 85080 (0.0007) [2023-10-08 07:13:53,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 173768704. Throughput: 0: 1826.8, 1: 1865.4. Samples: 43441908. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 07:13:53,754][130385] Avg episode reward: [(0, '67.280'), (1, '79.850')] [2023-10-08 07:13:55,272][00611] Updated weights for policy 0, policy_version 84612 (0.0009) [2023-10-08 07:13:55,647][00611] Updated weights for policy 0, policy_version 84622 (0.0008) [2023-10-08 07:13:56,019][00611] Updated weights for policy 0, policy_version 84632 (0.0007) [2023-10-08 07:13:56,851][00612] Updated weights for policy 1, policy_version 85090 (0.0008) [2023-10-08 07:13:57,228][00612] Updated weights for policy 1, policy_version 85100 (0.0009) [2023-10-08 07:13:57,585][00612] Updated weights for policy 1, policy_version 85110 (0.0007) [2023-10-08 07:13:57,951][00612] Updated weights for policy 1, policy_version 85120 (0.0008) [2023-10-08 07:13:58,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 173834240. Throughput: 0: 1836.1, 1: 1845.9. Samples: 43463682. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 07:13:58,754][130385] Avg episode reward: [(0, '66.910'), (1, '75.600')] [2023-10-08 07:13:59,708][00611] Updated weights for policy 0, policy_version 84642 (0.0008) [2023-10-08 07:14:00,085][00611] Updated weights for policy 0, policy_version 84652 (0.0009) [2023-10-08 07:14:00,463][00611] Updated weights for policy 0, policy_version 84662 (0.0011) [2023-10-08 07:14:00,830][00611] Updated weights for policy 0, policy_version 84672 (0.0010) [2023-10-08 07:14:01,568][00612] Updated weights for policy 1, policy_version 85130 (0.0008) [2023-10-08 07:14:01,945][00612] Updated weights for policy 1, policy_version 85140 (0.0009) [2023-10-08 07:14:02,311][00612] Updated weights for policy 1, policy_version 85150 (0.0008) [2023-10-08 07:14:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 173899776. Throughput: 0: 1827.3, 1: 1853.6. Samples: 43485754. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 07:14:03,754][130385] Avg episode reward: [(0, '71.670'), (1, '72.910')] [2023-10-08 07:14:04,418][00611] Updated weights for policy 0, policy_version 84682 (0.0008) [2023-10-08 07:14:04,787][00611] Updated weights for policy 0, policy_version 84692 (0.0008) [2023-10-08 07:14:05,159][00611] Updated weights for policy 0, policy_version 84702 (0.0007) [2023-10-08 07:14:05,978][00612] Updated weights for policy 1, policy_version 85160 (0.0007) [2023-10-08 07:14:06,344][00612] Updated weights for policy 1, policy_version 85170 (0.0009) [2023-10-08 07:14:06,714][00612] Updated weights for policy 1, policy_version 85180 (0.0008) [2023-10-08 07:14:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 173965312. Throughput: 0: 1829.7, 1: 1845.6. Samples: 43496650. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 07:14:08,754][130385] Avg episode reward: [(0, '73.100'), (1, '76.100')] [2023-10-08 07:14:08,790][00611] Updated weights for policy 0, policy_version 84712 (0.0007) [2023-10-08 07:14:09,167][00611] Updated weights for policy 0, policy_version 84722 (0.0009) [2023-10-08 07:14:09,534][00611] Updated weights for policy 0, policy_version 84732 (0.0007) [2023-10-08 07:14:10,409][00612] Updated weights for policy 1, policy_version 85190 (0.0009) [2023-10-08 07:14:10,786][00612] Updated weights for policy 1, policy_version 85200 (0.0008) [2023-10-08 07:14:11,159][00612] Updated weights for policy 1, policy_version 85210 (0.0007) [2023-10-08 07:14:13,250][00611] Updated weights for policy 0, policy_version 84742 (0.0007) [2023-10-08 07:14:13,621][00611] Updated weights for policy 0, policy_version 84752 (0.0007) [2023-10-08 07:14:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 174030848. Throughput: 0: 1832.3, 1: 1856.7. Samples: 43518872. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 07:14:13,754][130385] Avg episode reward: [(0, '73.270'), (1, '75.180')] [2023-10-08 07:14:13,990][00611] Updated weights for policy 0, policy_version 84762 (0.0008) [2023-10-08 07:14:14,729][00612] Updated weights for policy 1, policy_version 85220 (0.0007) [2023-10-08 07:14:15,093][00612] Updated weights for policy 1, policy_version 85230 (0.0007) [2023-10-08 07:14:15,454][00612] Updated weights for policy 1, policy_version 85240 (0.0008) [2023-10-08 07:14:17,520][00611] Updated weights for policy 0, policy_version 84772 (0.0008) [2023-10-08 07:14:17,884][00611] Updated weights for policy 0, policy_version 84782 (0.0007) [2023-10-08 07:14:18,256][00611] Updated weights for policy 0, policy_version 84792 (0.0009) [2023-10-08 07:14:18,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 174129152. Throughput: 0: 1827.6, 1: 1865.3. Samples: 43541408. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 07:14:18,755][130385] Avg episode reward: [(0, '70.410'), (1, '78.670')] [2023-10-08 07:14:18,771][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000084800_86835200.pth... [2023-10-08 07:14:18,771][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000085248_87293952.pth... [2023-10-08 07:14:18,808][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000083072_85065728.pth [2023-10-08 07:14:18,811][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000083520_85524480.pth [2023-10-08 07:14:18,982][00612] Updated weights for policy 1, policy_version 85250 (0.0007) [2023-10-08 07:14:19,355][00612] Updated weights for policy 1, policy_version 85260 (0.0009) [2023-10-08 07:14:19,728][00612] Updated weights for policy 1, policy_version 85270 (0.0009) [2023-10-08 07:14:20,096][00612] Updated weights for policy 1, policy_version 85280 (0.0008) [2023-10-08 07:14:21,952][00611] Updated weights for policy 0, policy_version 84802 (0.0008) [2023-10-08 07:14:22,324][00611] Updated weights for policy 0, policy_version 84812 (0.0008) [2023-10-08 07:14:22,683][00611] Updated weights for policy 0, policy_version 84822 (0.0008) [2023-10-08 07:14:23,047][00611] Updated weights for policy 0, policy_version 84832 (0.0007) [2023-10-08 07:14:23,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 174194688. Throughput: 0: 1835.8, 1: 1862.4. Samples: 43552386. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 07:14:23,754][130385] Avg episode reward: [(0, '68.820'), (1, '78.580')] [2023-10-08 07:14:23,770][00612] Updated weights for policy 1, policy_version 85290 (0.0008) [2023-10-08 07:14:24,133][00612] Updated weights for policy 1, policy_version 85300 (0.0008) [2023-10-08 07:14:24,509][00612] Updated weights for policy 1, policy_version 85310 (0.0008) [2023-10-08 07:14:26,671][00611] Updated weights for policy 0, policy_version 84842 (0.0007) [2023-10-08 07:14:27,052][00611] Updated weights for policy 0, policy_version 84852 (0.0007) [2023-10-08 07:14:27,428][00611] Updated weights for policy 0, policy_version 84862 (0.0008) [2023-10-08 07:14:27,972][00612] Updated weights for policy 1, policy_version 85320 (0.0010) [2023-10-08 07:14:28,348][00612] Updated weights for policy 1, policy_version 85330 (0.0009) [2023-10-08 07:14:28,716][00612] Updated weights for policy 1, policy_version 85340 (0.0011) [2023-10-08 07:14:28,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 174260224. Throughput: 0: 1829.4, 1: 1866.2. Samples: 43574762. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 07:14:28,754][130385] Avg episode reward: [(0, '70.370'), (1, '79.710')] [2023-10-08 07:14:31,034][00611] Updated weights for policy 0, policy_version 84872 (0.0010) [2023-10-08 07:14:31,399][00611] Updated weights for policy 0, policy_version 84882 (0.0010) [2023-10-08 07:14:31,771][00611] Updated weights for policy 0, policy_version 84892 (0.0008) [2023-10-08 07:14:32,307][00612] Updated weights for policy 1, policy_version 85350 (0.0010) [2023-10-08 07:14:32,672][00612] Updated weights for policy 1, policy_version 85360 (0.0008) [2023-10-08 07:14:33,033][00612] Updated weights for policy 1, policy_version 85370 (0.0009) [2023-10-08 07:14:33,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 174358528. Throughput: 0: 1850.0, 1: 1830.6. Samples: 43596290. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 07:14:33,755][130385] Avg episode reward: [(0, '72.940'), (1, '78.070')] [2023-10-08 07:14:35,393][00611] Updated weights for policy 0, policy_version 84902 (0.0008) [2023-10-08 07:14:35,760][00611] Updated weights for policy 0, policy_version 84912 (0.0009) [2023-10-08 07:14:36,135][00611] Updated weights for policy 0, policy_version 84922 (0.0007) [2023-10-08 07:14:36,680][00612] Updated weights for policy 1, policy_version 85380 (0.0009) [2023-10-08 07:14:37,048][00612] Updated weights for policy 1, policy_version 85390 (0.0007) [2023-10-08 07:14:37,422][00612] Updated weights for policy 1, policy_version 85400 (0.0009) [2023-10-08 07:14:38,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 174424064. Throughput: 0: 1835.4, 1: 1852.8. Samples: 43607878. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:14:38,755][130385] Avg episode reward: [(0, '73.280'), (1, '77.100')] [2023-10-08 07:14:39,694][00611] Updated weights for policy 0, policy_version 84932 (0.0009) [2023-10-08 07:14:40,067][00611] Updated weights for policy 0, policy_version 84942 (0.0010) [2023-10-08 07:14:40,451][00611] Updated weights for policy 0, policy_version 84952 (0.0011) [2023-10-08 07:14:41,098][00612] Updated weights for policy 1, policy_version 85410 (0.0008) [2023-10-08 07:14:41,464][00612] Updated weights for policy 1, policy_version 85420 (0.0010) [2023-10-08 07:14:41,826][00612] Updated weights for policy 1, policy_version 85430 (0.0009) [2023-10-08 07:14:42,191][00612] Updated weights for policy 1, policy_version 85440 (0.0009) [2023-10-08 07:14:43,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 174489600. Throughput: 0: 1849.5, 1: 1832.1. Samples: 43629354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:14:43,754][130385] Avg episode reward: [(0, '75.390'), (1, '79.630')] [2023-10-08 07:14:44,152][00611] Updated weights for policy 0, policy_version 84962 (0.0010) [2023-10-08 07:14:44,536][00611] Updated weights for policy 0, policy_version 84972 (0.0010) [2023-10-08 07:14:44,915][00611] Updated weights for policy 0, policy_version 84982 (0.0010) [2023-10-08 07:14:45,288][00611] Updated weights for policy 0, policy_version 84992 (0.0009) [2023-10-08 07:14:45,893][00612] Updated weights for policy 1, policy_version 85450 (0.0010) [2023-10-08 07:14:46,267][00612] Updated weights for policy 1, policy_version 85460 (0.0008) [2023-10-08 07:14:46,638][00612] Updated weights for policy 1, policy_version 85470 (0.0007) [2023-10-08 07:14:48,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 174555136. Throughput: 0: 1847.8, 1: 1851.8. Samples: 43652234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:14:48,754][130385] Avg episode reward: [(0, '75.850'), (1, '80.600')] [2023-10-08 07:14:48,930][00611] Updated weights for policy 0, policy_version 85002 (0.0008) [2023-10-08 07:14:49,298][00611] Updated weights for policy 0, policy_version 85012 (0.0008) [2023-10-08 07:14:49,674][00611] Updated weights for policy 0, policy_version 85022 (0.0008) [2023-10-08 07:14:50,300][00612] Updated weights for policy 1, policy_version 85480 (0.0008) [2023-10-08 07:14:50,669][00612] Updated weights for policy 1, policy_version 85490 (0.0009) [2023-10-08 07:14:51,033][00612] Updated weights for policy 1, policy_version 85500 (0.0008) [2023-10-08 07:14:53,325][00611] Updated weights for policy 0, policy_version 85032 (0.0009) [2023-10-08 07:14:53,693][00611] Updated weights for policy 0, policy_version 85042 (0.0008) [2023-10-08 07:14:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14773.4). Total num frames: 174620672. Throughput: 0: 1849.8, 1: 1834.7. Samples: 43662450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:14:53,754][130385] Avg episode reward: [(0, '74.510'), (1, '78.700')] [2023-10-08 07:14:54,062][00611] Updated weights for policy 0, policy_version 85052 (0.0008) [2023-10-08 07:14:54,656][00612] Updated weights for policy 1, policy_version 85510 (0.0007) [2023-10-08 07:14:55,033][00612] Updated weights for policy 1, policy_version 85520 (0.0007) [2023-10-08 07:14:55,401][00612] Updated weights for policy 1, policy_version 85530 (0.0008) [2023-10-08 07:14:57,669][00611] Updated weights for policy 0, policy_version 85062 (0.0008) [2023-10-08 07:14:58,040][00611] Updated weights for policy 0, policy_version 85072 (0.0008) [2023-10-08 07:14:58,405][00611] Updated weights for policy 0, policy_version 85082 (0.0009) [2023-10-08 07:14:58,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 174718976. Throughput: 0: 1847.1, 1: 1856.4. Samples: 43685534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:14:58,755][130385] Avg episode reward: [(0, '76.260'), (1, '78.710')] [2023-10-08 07:14:59,196][00612] Updated weights for policy 1, policy_version 85540 (0.0010) [2023-10-08 07:14:59,592][00612] Updated weights for policy 1, policy_version 85550 (0.0007) [2023-10-08 07:14:59,963][00612] Updated weights for policy 1, policy_version 85560 (0.0007) [2023-10-08 07:15:02,061][00611] Updated weights for policy 0, policy_version 85092 (0.0008) [2023-10-08 07:15:02,436][00611] Updated weights for policy 0, policy_version 85102 (0.0007) [2023-10-08 07:15:02,803][00611] Updated weights for policy 0, policy_version 85112 (0.0009) [2023-10-08 07:15:03,529][00612] Updated weights for policy 1, policy_version 85570 (0.0008) [2023-10-08 07:15:03,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 174784512. Throughput: 0: 1830.9, 1: 1847.5. Samples: 43706936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:15:03,755][130385] Avg episode reward: [(0, '76.550'), (1, '80.620')] [2023-10-08 07:15:03,892][00612] Updated weights for policy 1, policy_version 85580 (0.0008) [2023-10-08 07:15:04,260][00612] Updated weights for policy 1, policy_version 85590 (0.0007) [2023-10-08 07:15:04,635][00612] Updated weights for policy 1, policy_version 85600 (0.0007) [2023-10-08 07:15:06,501][00611] Updated weights for policy 0, policy_version 85122 (0.0008) [2023-10-08 07:15:06,872][00611] Updated weights for policy 0, policy_version 85132 (0.0008) [2023-10-08 07:15:07,245][00611] Updated weights for policy 0, policy_version 85142 (0.0008) [2023-10-08 07:15:07,613][00611] Updated weights for policy 0, policy_version 85152 (0.0008) [2023-10-08 07:15:08,155][00612] Updated weights for policy 1, policy_version 85610 (0.0008) [2023-10-08 07:15:08,515][00612] Updated weights for policy 1, policy_version 85620 (0.0007) [2023-10-08 07:15:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 174850048. Throughput: 0: 1843.6, 1: 1844.9. Samples: 43718368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:15:08,754][130385] Avg episode reward: [(0, '74.360'), (1, '82.820')] [2023-10-08 07:15:08,880][00612] Updated weights for policy 1, policy_version 85630 (0.0007) [2023-10-08 07:15:11,219][00611] Updated weights for policy 0, policy_version 85162 (0.0011) [2023-10-08 07:15:11,594][00611] Updated weights for policy 0, policy_version 85172 (0.0010) [2023-10-08 07:15:11,962][00611] Updated weights for policy 0, policy_version 85182 (0.0010) [2023-10-08 07:15:12,523][00612] Updated weights for policy 1, policy_version 85640 (0.0008) [2023-10-08 07:15:12,899][00612] Updated weights for policy 1, policy_version 85650 (0.0007) [2023-10-08 07:15:13,274][00612] Updated weights for policy 1, policy_version 85660 (0.0009) [2023-10-08 07:15:13,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 174948352. Throughput: 0: 1833.0, 1: 1845.2. Samples: 43740282. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:15:13,755][130385] Avg episode reward: [(0, '70.490'), (1, '83.720')] [2023-10-08 07:15:15,608][00611] Updated weights for policy 0, policy_version 85192 (0.0008) [2023-10-08 07:15:15,977][00611] Updated weights for policy 0, policy_version 85202 (0.0009) [2023-10-08 07:15:16,351][00611] Updated weights for policy 0, policy_version 85212 (0.0008) [2023-10-08 07:15:16,971][00612] Updated weights for policy 1, policy_version 85670 (0.0008) [2023-10-08 07:15:17,339][00612] Updated weights for policy 1, policy_version 85680 (0.0007) [2023-10-08 07:15:17,713][00612] Updated weights for policy 1, policy_version 85690 (0.0009) [2023-10-08 07:15:18,754][130385] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 175013888. Throughput: 0: 1835.9, 1: 1837.9. Samples: 43761610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:15:18,755][130385] Avg episode reward: [(0, '71.630'), (1, '83.960')] [2023-10-08 07:15:20,026][00611] Updated weights for policy 0, policy_version 85222 (0.0008) [2023-10-08 07:15:20,397][00611] Updated weights for policy 0, policy_version 85232 (0.0008) [2023-10-08 07:15:20,767][00611] Updated weights for policy 0, policy_version 85242 (0.0007) [2023-10-08 07:15:21,386][00612] Updated weights for policy 1, policy_version 85700 (0.0011) [2023-10-08 07:15:21,766][00612] Updated weights for policy 1, policy_version 85710 (0.0008) [2023-10-08 07:15:22,123][00612] Updated weights for policy 1, policy_version 85720 (0.0009) [2023-10-08 07:15:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 175079424. Throughput: 0: 1832.8, 1: 1840.9. Samples: 43773194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:15:23,755][130385] Avg episode reward: [(0, '73.090'), (1, '84.590')] [2023-10-08 07:15:24,440][00611] Updated weights for policy 0, policy_version 85252 (0.0008) [2023-10-08 07:15:24,814][00611] Updated weights for policy 0, policy_version 85262 (0.0009) [2023-10-08 07:15:25,190][00611] Updated weights for policy 0, policy_version 85272 (0.0008) [2023-10-08 07:15:25,903][00612] Updated weights for policy 1, policy_version 85730 (0.0009) [2023-10-08 07:15:26,281][00612] Updated weights for policy 1, policy_version 85740 (0.0007) [2023-10-08 07:15:26,653][00612] Updated weights for policy 1, policy_version 85750 (0.0009) [2023-10-08 07:15:27,017][00612] Updated weights for policy 1, policy_version 85760 (0.0007) [2023-10-08 07:15:28,621][00611] Updated weights for policy 0, policy_version 85282 (0.0009) [2023-10-08 07:15:28,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 175144960. Throughput: 0: 1838.1, 1: 1836.4. Samples: 43794710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:15:28,754][130385] Avg episode reward: [(0, '75.460'), (1, '86.170')] [2023-10-08 07:15:28,994][00611] Updated weights for policy 0, policy_version 85292 (0.0007) [2023-10-08 07:15:29,376][00611] Updated weights for policy 0, policy_version 85302 (0.0010) [2023-10-08 07:15:29,744][00611] Updated weights for policy 0, policy_version 85312 (0.0008) [2023-10-08 07:15:30,520][00612] Updated weights for policy 1, policy_version 85770 (0.0008) [2023-10-08 07:15:30,881][00612] Updated weights for policy 1, policy_version 85780 (0.0008) [2023-10-08 07:15:31,248][00612] Updated weights for policy 1, policy_version 85790 (0.0009) [2023-10-08 07:15:33,420][00611] Updated weights for policy 0, policy_version 85322 (0.0007) [2023-10-08 07:15:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 175210496. Throughput: 0: 1840.7, 1: 1843.4. Samples: 43818018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:15:33,754][130385] Avg episode reward: [(0, '73.960'), (1, '85.140')] [2023-10-08 07:15:33,782][00611] Updated weights for policy 0, policy_version 85332 (0.0007) [2023-10-08 07:15:34,162][00611] Updated weights for policy 0, policy_version 85342 (0.0007) [2023-10-08 07:15:34,753][00612] Updated weights for policy 1, policy_version 85800 (0.0008) [2023-10-08 07:15:35,120][00612] Updated weights for policy 1, policy_version 85810 (0.0007) [2023-10-08 07:15:35,489][00612] Updated weights for policy 1, policy_version 85820 (0.0007) [2023-10-08 07:15:37,716][00611] Updated weights for policy 0, policy_version 85352 (0.0009) [2023-10-08 07:15:38,088][00611] Updated weights for policy 0, policy_version 85362 (0.0007) [2023-10-08 07:15:38,448][00611] Updated weights for policy 0, policy_version 85372 (0.0007) [2023-10-08 07:15:38,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 175308800. Throughput: 0: 1846.4, 1: 1845.2. Samples: 43828574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:15:38,755][130385] Avg episode reward: [(0, '75.580'), (1, '83.850')] [2023-10-08 07:15:39,079][00612] Updated weights for policy 1, policy_version 85830 (0.0008) [2023-10-08 07:15:39,443][00612] Updated weights for policy 1, policy_version 85840 (0.0009) [2023-10-08 07:15:39,815][00612] Updated weights for policy 1, policy_version 85850 (0.0007) [2023-10-08 07:15:42,156][00611] Updated weights for policy 0, policy_version 85382 (0.0008) [2023-10-08 07:15:42,519][00611] Updated weights for policy 0, policy_version 85392 (0.0007) [2023-10-08 07:15:42,897][00611] Updated weights for policy 0, policy_version 85402 (0.0007) [2023-10-08 07:15:43,403][00612] Updated weights for policy 1, policy_version 85860 (0.0008) [2023-10-08 07:15:43,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 175374336. Throughput: 0: 1838.2, 1: 1851.2. Samples: 43851558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:15:43,754][130385] Avg episode reward: [(0, '73.460'), (1, '86.230')] [2023-10-08 07:15:43,795][00612] Updated weights for policy 1, policy_version 85870 (0.0007) [2023-10-08 07:15:44,151][00612] Updated weights for policy 1, policy_version 85880 (0.0007) [2023-10-08 07:15:46,442][00611] Updated weights for policy 0, policy_version 85412 (0.0009) [2023-10-08 07:15:46,810][00611] Updated weights for policy 0, policy_version 85422 (0.0010) [2023-10-08 07:15:47,181][00611] Updated weights for policy 0, policy_version 85432 (0.0010) [2023-10-08 07:15:47,666][00612] Updated weights for policy 1, policy_version 85890 (0.0008) [2023-10-08 07:15:48,032][00612] Updated weights for policy 1, policy_version 85900 (0.0007) [2023-10-08 07:15:48,406][00612] Updated weights for policy 1, policy_version 85910 (0.0009) [2023-10-08 07:15:48,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 175439872. Throughput: 0: 1844.0, 1: 1842.5. Samples: 43872828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:15:48,755][130385] Avg episode reward: [(0, '74.170'), (1, '85.570')] [2023-10-08 07:15:48,774][00612] Updated weights for policy 1, policy_version 85920 (0.0008) [2023-10-08 07:15:50,926][00611] Updated weights for policy 0, policy_version 85442 (0.0009) [2023-10-08 07:15:51,293][00611] Updated weights for policy 0, policy_version 85452 (0.0007) [2023-10-08 07:15:51,665][00611] Updated weights for policy 0, policy_version 85462 (0.0009) [2023-10-08 07:15:52,034][00611] Updated weights for policy 0, policy_version 85472 (0.0007) [2023-10-08 07:15:52,350][00612] Updated weights for policy 1, policy_version 85930 (0.0008) [2023-10-08 07:15:52,717][00612] Updated weights for policy 1, policy_version 85940 (0.0007) [2023-10-08 07:15:53,074][00612] Updated weights for policy 1, policy_version 85950 (0.0008) [2023-10-08 07:15:53,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 175538176. Throughput: 0: 1833.9, 1: 1864.1. Samples: 43884780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:15:53,755][130385] Avg episode reward: [(0, '74.010'), (1, '85.690')] [2023-10-08 07:15:55,621][00611] Updated weights for policy 0, policy_version 85482 (0.0011) [2023-10-08 07:15:55,981][00611] Updated weights for policy 0, policy_version 85492 (0.0011) [2023-10-08 07:15:56,348][00611] Updated weights for policy 0, policy_version 85502 (0.0009) [2023-10-08 07:15:56,875][00612] Updated weights for policy 1, policy_version 85960 (0.0008) [2023-10-08 07:15:57,243][00612] Updated weights for policy 1, policy_version 85970 (0.0007) [2023-10-08 07:15:57,603][00612] Updated weights for policy 1, policy_version 85980 (0.0010) [2023-10-08 07:15:58,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 175603712. Throughput: 0: 1852.5, 1: 1833.8. Samples: 43906164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:15:58,754][130385] Avg episode reward: [(0, '71.120'), (1, '82.210')] [2023-10-08 07:16:00,030][00611] Updated weights for policy 0, policy_version 85512 (0.0009) [2023-10-08 07:16:00,408][00611] Updated weights for policy 0, policy_version 85522 (0.0011) [2023-10-08 07:16:00,777][00611] Updated weights for policy 0, policy_version 85532 (0.0010) [2023-10-08 07:16:01,050][00612] Updated weights for policy 1, policy_version 85990 (0.0010) [2023-10-08 07:16:01,420][00612] Updated weights for policy 1, policy_version 86000 (0.0011) [2023-10-08 07:16:01,790][00612] Updated weights for policy 1, policy_version 86010 (0.0010) [2023-10-08 07:16:03,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 175669248. Throughput: 0: 1855.7, 1: 1859.5. Samples: 43928792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:16:03,755][130385] Avg episode reward: [(0, '71.350'), (1, '81.820')] [2023-10-08 07:16:04,253][00611] Updated weights for policy 0, policy_version 85542 (0.0008) [2023-10-08 07:16:04,632][00611] Updated weights for policy 0, policy_version 85552 (0.0009) [2023-10-08 07:16:04,990][00611] Updated weights for policy 0, policy_version 85562 (0.0010) [2023-10-08 07:16:05,352][00612] Updated weights for policy 1, policy_version 86020 (0.0009) [2023-10-08 07:16:05,715][00612] Updated weights for policy 1, policy_version 86030 (0.0010) [2023-10-08 07:16:06,084][00612] Updated weights for policy 1, policy_version 86040 (0.0009) [2023-10-08 07:16:08,636][00611] Updated weights for policy 0, policy_version 85572 (0.0008) [2023-10-08 07:16:08,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 175734784. Throughput: 0: 1854.8, 1: 1838.0. Samples: 43939374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:16:08,755][130385] Avg episode reward: [(0, '69.610'), (1, '82.860')] [2023-10-08 07:16:09,007][00611] Updated weights for policy 0, policy_version 85582 (0.0009) [2023-10-08 07:16:09,374][00611] Updated weights for policy 0, policy_version 85592 (0.0008) [2023-10-08 07:16:09,797][00612] Updated weights for policy 1, policy_version 86050 (0.0008) [2023-10-08 07:16:10,167][00612] Updated weights for policy 1, policy_version 86060 (0.0008) [2023-10-08 07:16:10,531][00612] Updated weights for policy 1, policy_version 86070 (0.0008) [2023-10-08 07:16:10,895][00612] Updated weights for policy 1, policy_version 86080 (0.0008) [2023-10-08 07:16:13,045][00611] Updated weights for policy 0, policy_version 85602 (0.0009) [2023-10-08 07:16:13,419][00611] Updated weights for policy 0, policy_version 85612 (0.0011) [2023-10-08 07:16:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 175800320. Throughput: 0: 1850.9, 1: 1866.8. Samples: 43962008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:16:13,755][130385] Avg episode reward: [(0, '69.180'), (1, '75.610')] [2023-10-08 07:16:13,794][00611] Updated weights for policy 0, policy_version 85622 (0.0009) [2023-10-08 07:16:14,161][00611] Updated weights for policy 0, policy_version 85632 (0.0008) [2023-10-08 07:16:14,591][00612] Updated weights for policy 1, policy_version 86090 (0.0008) [2023-10-08 07:16:14,963][00612] Updated weights for policy 1, policy_version 86100 (0.0007) [2023-10-08 07:16:15,334][00612] Updated weights for policy 1, policy_version 86110 (0.0007) [2023-10-08 07:16:17,881][00611] Updated weights for policy 0, policy_version 85642 (0.0009) [2023-10-08 07:16:18,246][00611] Updated weights for policy 0, policy_version 85652 (0.0008) [2023-10-08 07:16:18,622][00611] Updated weights for policy 0, policy_version 85662 (0.0008) [2023-10-08 07:16:18,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.3). Total num frames: 175898624. Throughput: 0: 1831.1, 1: 1866.7. Samples: 43984418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:16:18,755][130385] Avg episode reward: [(0, '74.040'), (1, '77.830')] [2023-10-08 07:16:18,767][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000085664_87719936.pth... [2023-10-08 07:16:18,800][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000083936_85950464.pth [2023-10-08 07:16:18,804][00365] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p0/milestones/checkpoint_000085664_87719936.pth [2023-10-08 07:16:18,880][00612] Updated weights for policy 1, policy_version 86120 (0.0009) [2023-10-08 07:16:19,246][00612] Updated weights for policy 1, policy_version 86130 (0.0010) [2023-10-08 07:16:19,624][00612] Updated weights for policy 1, policy_version 86140 (0.0009) [2023-10-08 07:16:19,776][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000086144_88211456.pth... [2023-10-08 07:16:19,804][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000084384_86409216.pth [2023-10-08 07:16:19,809][00425] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p1/milestones/checkpoint_000086144_88211456.pth [2023-10-08 07:16:22,146][00611] Updated weights for policy 0, policy_version 85672 (0.0008) [2023-10-08 07:16:22,528][00611] Updated weights for policy 0, policy_version 85682 (0.0008) [2023-10-08 07:16:22,897][00611] Updated weights for policy 0, policy_version 85692 (0.0008) [2023-10-08 07:16:23,249][00612] Updated weights for policy 1, policy_version 86150 (0.0008) [2023-10-08 07:16:23,618][00612] Updated weights for policy 1, policy_version 86160 (0.0007) [2023-10-08 07:16:23,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 175964160. Throughput: 0: 1847.0, 1: 1858.8. Samples: 43995332. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-08 07:16:23,754][130385] Avg episode reward: [(0, '75.640'), (1, '76.760')] [2023-10-08 07:16:23,989][00612] Updated weights for policy 1, policy_version 86170 (0.0010) [2023-10-08 07:16:26,673][00611] Updated weights for policy 0, policy_version 85702 (0.0008) [2023-10-08 07:16:27,043][00611] Updated weights for policy 0, policy_version 85712 (0.0007) [2023-10-08 07:16:27,411][00611] Updated weights for policy 0, policy_version 85722 (0.0007) [2023-10-08 07:16:27,413][00612] Updated weights for policy 1, policy_version 86180 (0.0007) [2023-10-08 07:16:27,778][00612] Updated weights for policy 1, policy_version 86190 (0.0009) [2023-10-08 07:16:28,152][00612] Updated weights for policy 1, policy_version 86200 (0.0008) [2023-10-08 07:16:28,754][130385] Fps is (10 sec: 16384.6, 60 sec: 15291.7, 300 sec: 14884.5). Total num frames: 176062464. Throughput: 0: 1833.7, 1: 1863.0. Samples: 44017908. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-08 07:16:28,754][130385] Avg episode reward: [(0, '74.390'), (1, '81.940')] [2023-10-08 07:16:31,007][00611] Updated weights for policy 0, policy_version 85732 (0.0008) [2023-10-08 07:16:31,373][00611] Updated weights for policy 0, policy_version 85742 (0.0007) [2023-10-08 07:16:31,716][00612] Updated weights for policy 1, policy_version 86210 (0.0009) [2023-10-08 07:16:31,750][00611] Updated weights for policy 0, policy_version 85752 (0.0007) [2023-10-08 07:16:32,129][00612] Updated weights for policy 1, policy_version 86220 (0.0008) [2023-10-08 07:16:32,494][00612] Updated weights for policy 1, policy_version 86230 (0.0009) [2023-10-08 07:16:32,861][00612] Updated weights for policy 1, policy_version 86240 (0.0010) [2023-10-08 07:16:33,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14884.5). Total num frames: 176128000. Throughput: 0: 1844.1, 1: 1841.1. Samples: 44038660. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-08 07:16:33,754][130385] Avg episode reward: [(0, '76.260'), (1, '82.320')] [2023-10-08 07:16:35,493][00611] Updated weights for policy 0, policy_version 85762 (0.0010) [2023-10-08 07:16:35,871][00611] Updated weights for policy 0, policy_version 85772 (0.0009) [2023-10-08 07:16:36,244][00611] Updated weights for policy 0, policy_version 85782 (0.0008) [2023-10-08 07:16:36,399][00612] Updated weights for policy 1, policy_version 86250 (0.0007) [2023-10-08 07:16:36,603][00611] Updated weights for policy 0, policy_version 85792 (0.0007) [2023-10-08 07:16:36,764][00612] Updated weights for policy 1, policy_version 86260 (0.0009) [2023-10-08 07:16:37,133][00612] Updated weights for policy 1, policy_version 86270 (0.0007) [2023-10-08 07:16:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 176193536. Throughput: 0: 1834.3, 1: 1856.5. Samples: 44050866. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-08 07:16:38,754][130385] Avg episode reward: [(0, '75.080'), (1, '80.090')] [2023-10-08 07:16:40,265][00611] Updated weights for policy 0, policy_version 85802 (0.0007) [2023-10-08 07:16:40,643][00611] Updated weights for policy 0, policy_version 85812 (0.0007) [2023-10-08 07:16:40,704][00612] Updated weights for policy 1, policy_version 86280 (0.0008) [2023-10-08 07:16:41,012][00611] Updated weights for policy 0, policy_version 85822 (0.0008) [2023-10-08 07:16:41,074][00612] Updated weights for policy 1, policy_version 86290 (0.0008) [2023-10-08 07:16:41,440][00612] Updated weights for policy 1, policy_version 86300 (0.0008) [2023-10-08 07:16:43,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 176259072. Throughput: 0: 1837.1, 1: 1853.3. Samples: 44072232. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-08 07:16:43,755][130385] Avg episode reward: [(0, '75.820'), (1, '81.880')] [2023-10-08 07:16:44,695][00611] Updated weights for policy 0, policy_version 85832 (0.0009) [2023-10-08 07:16:45,060][00611] Updated weights for policy 0, policy_version 85842 (0.0007) [2023-10-08 07:16:45,161][00612] Updated weights for policy 1, policy_version 86310 (0.0009) [2023-10-08 07:16:45,435][00611] Updated weights for policy 0, policy_version 85852 (0.0009) [2023-10-08 07:16:45,534][00612] Updated weights for policy 1, policy_version 86320 (0.0010) [2023-10-08 07:16:45,892][00612] Updated weights for policy 1, policy_version 86330 (0.0009) [2023-10-08 07:16:48,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 176324608. Throughput: 0: 1835.9, 1: 1863.9. Samples: 44095282. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-08 07:16:48,754][130385] Avg episode reward: [(0, '79.600'), (1, '81.850')] [2023-10-08 07:16:49,051][00611] Updated weights for policy 0, policy_version 85862 (0.0010) [2023-10-08 07:16:49,425][00611] Updated weights for policy 0, policy_version 85872 (0.0007) [2023-10-08 07:16:49,557][00612] Updated weights for policy 1, policy_version 86340 (0.0010) [2023-10-08 07:16:49,787][00611] Updated weights for policy 0, policy_version 85882 (0.0007) [2023-10-08 07:16:49,924][00612] Updated weights for policy 1, policy_version 86350 (0.0008) [2023-10-08 07:16:50,288][00612] Updated weights for policy 1, policy_version 86360 (0.0009) [2023-10-08 07:16:53,263][00611] Updated weights for policy 0, policy_version 85892 (0.0008) [2023-10-08 07:16:53,639][00611] Updated weights for policy 0, policy_version 85902 (0.0008) [2023-10-08 07:16:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 176390144. Throughput: 0: 1835.8, 1: 1850.9. Samples: 44105274. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-08 07:16:53,754][130385] Avg episode reward: [(0, '78.990'), (1, '83.390')] [2023-10-08 07:16:54,014][00611] Updated weights for policy 0, policy_version 85912 (0.0009) [2023-10-08 07:16:54,200][00612] Updated weights for policy 1, policy_version 86370 (0.0008) [2023-10-08 07:16:54,558][00612] Updated weights for policy 1, policy_version 86380 (0.0007) [2023-10-08 07:16:54,924][00612] Updated weights for policy 1, policy_version 86390 (0.0009) [2023-10-08 07:16:55,292][00612] Updated weights for policy 1, policy_version 86400 (0.0009) [2023-10-08 07:16:57,692][00611] Updated weights for policy 0, policy_version 85922 (0.0007) [2023-10-08 07:16:58,064][00611] Updated weights for policy 0, policy_version 85932 (0.0007) [2023-10-08 07:16:58,433][00611] Updated weights for policy 0, policy_version 85942 (0.0007) [2023-10-08 07:16:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 176455680. Throughput: 0: 1844.3, 1: 1856.8. Samples: 44128560. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-08 07:16:58,755][130385] Avg episode reward: [(0, '80.510'), (1, '83.030')] [2023-10-08 07:16:58,788][00611] Updated weights for policy 0, policy_version 85952 (0.0007) [2023-10-08 07:16:58,824][00612] Updated weights for policy 1, policy_version 86410 (0.0007) [2023-10-08 07:16:59,190][00612] Updated weights for policy 1, policy_version 86420 (0.0007) [2023-10-08 07:16:59,568][00612] Updated weights for policy 1, policy_version 86430 (0.0007) [2023-10-08 07:17:02,296][00611] Updated weights for policy 0, policy_version 85962 (0.0008) [2023-10-08 07:17:02,669][00611] Updated weights for policy 0, policy_version 85972 (0.0007) [2023-10-08 07:17:03,041][00611] Updated weights for policy 0, policy_version 85982 (0.0008) [2023-10-08 07:17:03,215][00612] Updated weights for policy 1, policy_version 86440 (0.0007) [2023-10-08 07:17:03,579][00612] Updated weights for policy 1, policy_version 86450 (0.0007) [2023-10-08 07:17:03,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 176553984. Throughput: 0: 1828.9, 1: 1848.0. Samples: 44149880. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-08 07:17:03,755][130385] Avg episode reward: [(0, '80.190'), (1, '86.420')] [2023-10-08 07:17:03,939][00612] Updated weights for policy 1, policy_version 86460 (0.0010) [2023-10-08 07:17:06,705][00611] Updated weights for policy 0, policy_version 85992 (0.0008) [2023-10-08 07:17:07,072][00611] Updated weights for policy 0, policy_version 86002 (0.0008) [2023-10-08 07:17:07,454][00611] Updated weights for policy 0, policy_version 86012 (0.0009) [2023-10-08 07:17:07,491][00612] Updated weights for policy 1, policy_version 86470 (0.0010) [2023-10-08 07:17:07,855][00612] Updated weights for policy 1, policy_version 86480 (0.0008) [2023-10-08 07:17:08,232][00612] Updated weights for policy 1, policy_version 86490 (0.0008) [2023-10-08 07:17:08,754][130385] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14884.4). Total num frames: 176652288. Throughput: 0: 1842.0, 1: 1855.9. Samples: 44161736. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-08 07:17:08,755][130385] Avg episode reward: [(0, '81.450'), (1, '85.380')] [2023-10-08 07:17:11,162][00611] Updated weights for policy 0, policy_version 86022 (0.0007) [2023-10-08 07:17:11,533][00611] Updated weights for policy 0, policy_version 86032 (0.0010) [2023-10-08 07:17:11,778][00612] Updated weights for policy 1, policy_version 86500 (0.0008) [2023-10-08 07:17:11,909][00611] Updated weights for policy 0, policy_version 86042 (0.0007) [2023-10-08 07:17:12,155][00612] Updated weights for policy 1, policy_version 86510 (0.0008) [2023-10-08 07:17:12,519][00612] Updated weights for policy 1, policy_version 86520 (0.0008) [2023-10-08 07:17:13,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14884.5). Total num frames: 176717824. Throughput: 0: 1825.6, 1: 1837.9. Samples: 44182762. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-08 07:17:13,754][130385] Avg episode reward: [(0, '83.820'), (1, '88.250')] [2023-10-08 07:17:15,474][00611] Updated weights for policy 0, policy_version 86052 (0.0008) [2023-10-08 07:17:15,850][00611] Updated weights for policy 0, policy_version 86062 (0.0008) [2023-10-08 07:17:16,131][00612] Updated weights for policy 1, policy_version 86530 (0.0010) [2023-10-08 07:17:16,215][00611] Updated weights for policy 0, policy_version 86072 (0.0007) [2023-10-08 07:17:16,509][00612] Updated weights for policy 1, policy_version 86540 (0.0007) [2023-10-08 07:17:16,880][00612] Updated weights for policy 1, policy_version 86550 (0.0007) [2023-10-08 07:17:17,259][00612] Updated weights for policy 1, policy_version 86560 (0.0010) [2023-10-08 07:17:18,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 176783360. Throughput: 0: 1841.7, 1: 1854.9. Samples: 44205008. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 07:17:18,755][130385] Avg episode reward: [(0, '82.790'), (1, '82.770')] [2023-10-08 07:17:19,922][00611] Updated weights for policy 0, policy_version 86082 (0.0007) [2023-10-08 07:17:20,294][00611] Updated weights for policy 0, policy_version 86092 (0.0008) [2023-10-08 07:17:20,672][00611] Updated weights for policy 0, policy_version 86102 (0.0007) [2023-10-08 07:17:21,037][00611] Updated weights for policy 0, policy_version 86112 (0.0008) [2023-10-08 07:17:21,051][00612] Updated weights for policy 1, policy_version 86570 (0.0009) [2023-10-08 07:17:21,428][00612] Updated weights for policy 1, policy_version 86580 (0.0009) [2023-10-08 07:17:21,796][00612] Updated weights for policy 1, policy_version 86590 (0.0010) [2023-10-08 07:17:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 176848896. Throughput: 0: 1830.7, 1: 1840.0. Samples: 44216046. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 07:17:23,755][130385] Avg episode reward: [(0, '82.570'), (1, '84.270')] [2023-10-08 07:17:24,603][00611] Updated weights for policy 0, policy_version 86122 (0.0008) [2023-10-08 07:17:24,970][00611] Updated weights for policy 0, policy_version 86132 (0.0007) [2023-10-08 07:17:25,292][00612] Updated weights for policy 1, policy_version 86600 (0.0007) [2023-10-08 07:17:25,338][00611] Updated weights for policy 0, policy_version 86142 (0.0007) [2023-10-08 07:17:25,661][00612] Updated weights for policy 1, policy_version 86610 (0.0008) [2023-10-08 07:17:26,026][00612] Updated weights for policy 1, policy_version 86620 (0.0007) [2023-10-08 07:17:28,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 176914432. Throughput: 0: 1849.8, 1: 1843.1. Samples: 44238410. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 07:17:28,754][130385] Avg episode reward: [(0, '73.400'), (1, '83.210')] [2023-10-08 07:17:28,913][00611] Updated weights for policy 0, policy_version 86152 (0.0010) [2023-10-08 07:17:29,286][00611] Updated weights for policy 0, policy_version 86162 (0.0008) [2023-10-08 07:17:29,654][00611] Updated weights for policy 0, policy_version 86172 (0.0007) [2023-10-08 07:17:29,662][00612] Updated weights for policy 1, policy_version 86630 (0.0008) [2023-10-08 07:17:30,019][00612] Updated weights for policy 1, policy_version 86640 (0.0007) [2023-10-08 07:17:30,395][00612] Updated weights for policy 1, policy_version 86650 (0.0008) [2023-10-08 07:17:33,283][00611] Updated weights for policy 0, policy_version 86182 (0.0007) [2023-10-08 07:17:33,656][00611] Updated weights for policy 0, policy_version 86192 (0.0008) [2023-10-08 07:17:33,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 176979968. Throughput: 0: 1847.3, 1: 1849.9. Samples: 44261658. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 07:17:33,755][130385] Avg episode reward: [(0, '74.530'), (1, '79.180')] [2023-10-08 07:17:33,999][00612] Updated weights for policy 1, policy_version 86660 (0.0008) [2023-10-08 07:17:34,031][00611] Updated weights for policy 0, policy_version 86202 (0.0010) [2023-10-08 07:17:34,376][00612] Updated weights for policy 1, policy_version 86670 (0.0010) [2023-10-08 07:17:34,745][00612] Updated weights for policy 1, policy_version 86680 (0.0008) [2023-10-08 07:17:37,954][00611] Updated weights for policy 0, policy_version 86212 (0.0009) [2023-10-08 07:17:38,323][00612] Updated weights for policy 1, policy_version 86690 (0.0009) [2023-10-08 07:17:38,337][00611] Updated weights for policy 0, policy_version 86222 (0.0009) [2023-10-08 07:17:38,694][00612] Updated weights for policy 1, policy_version 86700 (0.0007) [2023-10-08 07:17:38,704][00611] Updated weights for policy 0, policy_version 86232 (0.0007) [2023-10-08 07:17:38,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 177045504. Throughput: 0: 1844.8, 1: 1850.7. Samples: 44271570. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 07:17:38,754][130385] Avg episode reward: [(0, '79.620'), (1, '78.750')] [2023-10-08 07:17:39,063][00612] Updated weights for policy 1, policy_version 86710 (0.0007) [2023-10-08 07:17:39,435][00612] Updated weights for policy 1, policy_version 86720 (0.0011) [2023-10-08 07:17:42,364][00611] Updated weights for policy 0, policy_version 86242 (0.0008) [2023-10-08 07:17:42,719][00611] Updated weights for policy 0, policy_version 86252 (0.0008) [2023-10-08 07:17:43,089][00611] Updated weights for policy 0, policy_version 86262 (0.0008) [2023-10-08 07:17:43,166][00612] Updated weights for policy 1, policy_version 86730 (0.0009) [2023-10-08 07:17:43,446][00611] Updated weights for policy 0, policy_version 86272 (0.0007) [2023-10-08 07:17:43,529][00612] Updated weights for policy 1, policy_version 86740 (0.0009) [2023-10-08 07:17:43,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 177143808. Throughput: 0: 1833.1, 1: 1853.9. Samples: 44294476. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 07:17:43,754][130385] Avg episode reward: [(0, '76.980'), (1, '77.880')] [2023-10-08 07:17:43,894][00612] Updated weights for policy 1, policy_version 86750 (0.0010) [2023-10-08 07:17:47,063][00611] Updated weights for policy 0, policy_version 86282 (0.0009) [2023-10-08 07:17:47,422][00611] Updated weights for policy 0, policy_version 86292 (0.0008) [2023-10-08 07:17:47,438][00612] Updated weights for policy 1, policy_version 86760 (0.0008) [2023-10-08 07:17:47,799][00611] Updated weights for policy 0, policy_version 86302 (0.0008) [2023-10-08 07:17:47,802][00612] Updated weights for policy 1, policy_version 86770 (0.0008) [2023-10-08 07:17:48,169][00612] Updated weights for policy 1, policy_version 86780 (0.0009) [2023-10-08 07:17:48,754][130385] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14884.4). Total num frames: 177242112. Throughput: 0: 1833.3, 1: 1830.8. Samples: 44314766. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 07:17:48,755][130385] Avg episode reward: [(0, '76.820'), (1, '81.960')] [2023-10-08 07:17:51,439][00611] Updated weights for policy 0, policy_version 86312 (0.0008) [2023-10-08 07:17:51,702][00612] Updated weights for policy 1, policy_version 86790 (0.0008) [2023-10-08 07:17:51,806][00611] Updated weights for policy 0, policy_version 86322 (0.0007) [2023-10-08 07:17:52,063][00612] Updated weights for policy 1, policy_version 86800 (0.0008) [2023-10-08 07:17:52,178][00611] Updated weights for policy 0, policy_version 86332 (0.0007) [2023-10-08 07:17:52,424][00612] Updated weights for policy 1, policy_version 86810 (0.0009) [2023-10-08 07:17:53,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 177307648. Throughput: 0: 1835.3, 1: 1854.3. Samples: 44327768. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 07:17:53,754][130385] Avg episode reward: [(0, '72.330'), (1, '79.920')] [2023-10-08 07:17:55,847][00611] Updated weights for policy 0, policy_version 86342 (0.0007) [2023-10-08 07:17:56,168][00612] Updated weights for policy 1, policy_version 86820 (0.0007) [2023-10-08 07:17:56,217][00611] Updated weights for policy 0, policy_version 86352 (0.0007) [2023-10-08 07:17:56,524][00612] Updated weights for policy 1, policy_version 86830 (0.0007) [2023-10-08 07:17:56,592][00611] Updated weights for policy 0, policy_version 86362 (0.0009) [2023-10-08 07:17:56,895][00612] Updated weights for policy 1, policy_version 86840 (0.0008) [2023-10-08 07:17:58,754][130385] Fps is (10 sec: 13107.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 177373184. Throughput: 0: 1840.1, 1: 1829.5. Samples: 44347896. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 07:17:58,754][130385] Avg episode reward: [(0, '74.060'), (1, '75.590')] [2023-10-08 07:18:00,189][00611] Updated weights for policy 0, policy_version 86372 (0.0008) [2023-10-08 07:18:00,555][00611] Updated weights for policy 0, policy_version 86382 (0.0008) [2023-10-08 07:18:00,710][00612] Updated weights for policy 1, policy_version 86850 (0.0008) [2023-10-08 07:18:00,934][00611] Updated weights for policy 0, policy_version 86392 (0.0008) [2023-10-08 07:18:01,072][00612] Updated weights for policy 1, policy_version 86860 (0.0007) [2023-10-08 07:18:01,451][00612] Updated weights for policy 1, policy_version 86870 (0.0008) [2023-10-08 07:18:01,813][00612] Updated weights for policy 1, policy_version 86880 (0.0009) [2023-10-08 07:18:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 177438720. Throughput: 0: 1840.5, 1: 1842.8. Samples: 44370756. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 07:18:03,754][130385] Avg episode reward: [(0, '76.760'), (1, '77.450')] [2023-10-08 07:18:04,665][00611] Updated weights for policy 0, policy_version 86402 (0.0007) [2023-10-08 07:18:05,040][00611] Updated weights for policy 0, policy_version 86412 (0.0008) [2023-10-08 07:18:05,364][00612] Updated weights for policy 1, policy_version 86890 (0.0007) [2023-10-08 07:18:05,409][00611] Updated weights for policy 0, policy_version 86422 (0.0008) [2023-10-08 07:18:05,729][00612] Updated weights for policy 1, policy_version 86900 (0.0008) [2023-10-08 07:18:05,771][00611] Updated weights for policy 0, policy_version 86432 (0.0008) [2023-10-08 07:18:06,104][00612] Updated weights for policy 1, policy_version 86910 (0.0007) [2023-10-08 07:18:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14773.4). Total num frames: 177504256. Throughput: 0: 1834.7, 1: 1826.0. Samples: 44380778. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 07:18:08,755][130385] Avg episode reward: [(0, '73.990'), (1, '75.300')] [2023-10-08 07:18:09,460][00611] Updated weights for policy 0, policy_version 86442 (0.0008) [2023-10-08 07:18:09,827][00611] Updated weights for policy 0, policy_version 86452 (0.0009) [2023-10-08 07:18:09,933][00612] Updated weights for policy 1, policy_version 86920 (0.0007) [2023-10-08 07:18:10,200][00611] Updated weights for policy 0, policy_version 86462 (0.0008) [2023-10-08 07:18:10,301][00612] Updated weights for policy 1, policy_version 86930 (0.0008) [2023-10-08 07:18:10,673][00612] Updated weights for policy 1, policy_version 86940 (0.0009) [2023-10-08 07:18:13,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 177569792. Throughput: 0: 1831.2, 1: 1846.6. Samples: 44403908. Policy #0 lag: (min: 8.0, avg: 30.1, max: 32.0) [2023-10-08 07:18:13,755][130385] Avg episode reward: [(0, '72.620'), (1, '73.310')] [2023-10-08 07:18:13,841][00611] Updated weights for policy 0, policy_version 86472 (0.0008) [2023-10-08 07:18:14,200][00612] Updated weights for policy 1, policy_version 86950 (0.0009) [2023-10-08 07:18:14,214][00611] Updated weights for policy 0, policy_version 86482 (0.0009) [2023-10-08 07:18:14,565][00612] Updated weights for policy 1, policy_version 86960 (0.0008) [2023-10-08 07:18:14,582][00611] Updated weights for policy 0, policy_version 86492 (0.0007) [2023-10-08 07:18:14,933][00612] Updated weights for policy 1, policy_version 86970 (0.0008) [2023-10-08 07:18:18,246][00611] Updated weights for policy 0, policy_version 86502 (0.0011) [2023-10-08 07:18:18,612][00611] Updated weights for policy 0, policy_version 86512 (0.0009) [2023-10-08 07:18:18,690][00612] Updated weights for policy 1, policy_version 86980 (0.0009) [2023-10-08 07:18:18,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 177635328. Throughput: 0: 1831.7, 1: 1839.2. Samples: 44426846. Policy #0 lag: (min: 8.0, avg: 30.1, max: 32.0) [2023-10-08 07:18:18,754][130385] Avg episode reward: [(0, '74.820'), (1, '72.690')] [2023-10-08 07:18:18,985][00611] Updated weights for policy 0, policy_version 86522 (0.0008) [2023-10-08 07:18:19,063][00612] Updated weights for policy 1, policy_version 86990 (0.0009) [2023-10-08 07:18:19,204][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000086528_88604672.pth... [2023-10-08 07:18:19,233][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000084800_86835200.pth [2023-10-08 07:18:19,431][00612] Updated weights for policy 1, policy_version 87000 (0.0008) [2023-10-08 07:18:19,732][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000087008_89096192.pth... [2023-10-08 07:18:19,770][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000085248_87293952.pth [2023-10-08 07:18:22,799][00611] Updated weights for policy 0, policy_version 86532 (0.0007) [2023-10-08 07:18:23,182][00612] Updated weights for policy 1, policy_version 87010 (0.0008) [2023-10-08 07:18:23,191][00611] Updated weights for policy 0, policy_version 86542 (0.0007) [2023-10-08 07:18:23,547][00612] Updated weights for policy 1, policy_version 87020 (0.0008) [2023-10-08 07:18:23,551][00611] Updated weights for policy 0, policy_version 86552 (0.0008) [2023-10-08 07:18:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 177700864. Throughput: 0: 1831.0, 1: 1833.7. Samples: 44436482. Policy #0 lag: (min: 8.0, avg: 30.1, max: 32.0) [2023-10-08 07:18:23,754][130385] Avg episode reward: [(0, '79.110'), (1, '73.180')] [2023-10-08 07:18:23,920][00612] Updated weights for policy 1, policy_version 87030 (0.0009) [2023-10-08 07:18:24,280][00612] Updated weights for policy 1, policy_version 87040 (0.0007) [2023-10-08 07:18:27,258][00611] Updated weights for policy 0, policy_version 86562 (0.0008) [2023-10-08 07:18:27,632][00611] Updated weights for policy 0, policy_version 86572 (0.0007) [2023-10-08 07:18:27,989][00612] Updated weights for policy 1, policy_version 87050 (0.0008) [2023-10-08 07:18:28,009][00611] Updated weights for policy 0, policy_version 86582 (0.0007) [2023-10-08 07:18:28,356][00612] Updated weights for policy 1, policy_version 87060 (0.0010) [2023-10-08 07:18:28,381][00611] Updated weights for policy 0, policy_version 86592 (0.0007) [2023-10-08 07:18:28,741][00612] Updated weights for policy 1, policy_version 87070 (0.0010) [2023-10-08 07:18:28,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 177799168. Throughput: 0: 1831.0, 1: 1828.8. Samples: 44459164. Policy #0 lag: (min: 8.0, avg: 30.1, max: 32.0) [2023-10-08 07:18:28,754][130385] Avg episode reward: [(0, '79.920'), (1, '71.920')] [2023-10-08 07:18:32,044][00611] Updated weights for policy 0, policy_version 86602 (0.0009) [2023-10-08 07:18:32,280][00612] Updated weights for policy 1, policy_version 87080 (0.0008) [2023-10-08 07:18:32,416][00611] Updated weights for policy 0, policy_version 86612 (0.0008) [2023-10-08 07:18:32,643][00612] Updated weights for policy 1, policy_version 87090 (0.0007) [2023-10-08 07:18:32,783][00611] Updated weights for policy 0, policy_version 86622 (0.0008) [2023-10-08 07:18:33,015][00612] Updated weights for policy 1, policy_version 87100 (0.0007) [2023-10-08 07:18:33,754][130385] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 177897472. Throughput: 0: 1830.6, 1: 1828.3. Samples: 44479414. Policy #0 lag: (min: 8.0, avg: 30.1, max: 32.0) [2023-10-08 07:18:33,755][130385] Avg episode reward: [(0, '79.670'), (1, '69.210')] [2023-10-08 07:18:36,399][00611] Updated weights for policy 0, policy_version 86632 (0.0008) [2023-10-08 07:18:36,692][00612] Updated weights for policy 1, policy_version 87110 (0.0007) [2023-10-08 07:18:36,773][00611] Updated weights for policy 0, policy_version 86642 (0.0008) [2023-10-08 07:18:37,050][00612] Updated weights for policy 1, policy_version 87120 (0.0007) [2023-10-08 07:18:37,140][00611] Updated weights for policy 0, policy_version 86652 (0.0007) [2023-10-08 07:18:37,424][00612] Updated weights for policy 1, policy_version 87130 (0.0008) [2023-10-08 07:18:38,754][130385] Fps is (10 sec: 16383.4, 60 sec: 15291.6, 300 sec: 14773.4). Total num frames: 177963008. Throughput: 0: 1824.8, 1: 1831.3. Samples: 44492294. Policy #0 lag: (min: 8.0, avg: 30.1, max: 32.0) [2023-10-08 07:18:38,755][130385] Avg episode reward: [(0, '79.270'), (1, '71.310')] [2023-10-08 07:18:40,875][00611] Updated weights for policy 0, policy_version 86662 (0.0008) [2023-10-08 07:18:41,092][00612] Updated weights for policy 1, policy_version 87140 (0.0009) [2023-10-08 07:18:41,253][00611] Updated weights for policy 0, policy_version 86672 (0.0007) [2023-10-08 07:18:41,454][00612] Updated weights for policy 1, policy_version 87150 (0.0007) [2023-10-08 07:18:41,620][00611] Updated weights for policy 0, policy_version 86682 (0.0007) [2023-10-08 07:18:41,820][00612] Updated weights for policy 1, policy_version 87160 (0.0007) [2023-10-08 07:18:43,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 178028544. Throughput: 0: 1820.1, 1: 1830.4. Samples: 44512172. Policy #0 lag: (min: 8.0, avg: 30.1, max: 32.0) [2023-10-08 07:18:43,754][130385] Avg episode reward: [(0, '80.030'), (1, '72.160')] [2023-10-08 07:18:45,229][00611] Updated weights for policy 0, policy_version 86692 (0.0008) [2023-10-08 07:18:45,483][00612] Updated weights for policy 1, policy_version 87170 (0.0011) [2023-10-08 07:18:45,589][00611] Updated weights for policy 0, policy_version 86702 (0.0010) [2023-10-08 07:18:45,841][00612] Updated weights for policy 1, policy_version 87180 (0.0010) [2023-10-08 07:18:45,966][00611] Updated weights for policy 0, policy_version 86712 (0.0009) [2023-10-08 07:18:46,211][00612] Updated weights for policy 1, policy_version 87190 (0.0008) [2023-10-08 07:18:46,574][00612] Updated weights for policy 1, policy_version 87200 (0.0007) [2023-10-08 07:18:48,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 178094080. Throughput: 0: 1818.8, 1: 1836.5. Samples: 44535244. Policy #0 lag: (min: 8.0, avg: 30.1, max: 32.0) [2023-10-08 07:18:48,754][130385] Avg episode reward: [(0, '76.000'), (1, '67.780')] [2023-10-08 07:18:49,577][00611] Updated weights for policy 0, policy_version 86722 (0.0009) [2023-10-08 07:18:49,951][00611] Updated weights for policy 0, policy_version 86732 (0.0007) [2023-10-08 07:18:50,042][00612] Updated weights for policy 1, policy_version 87210 (0.0010) [2023-10-08 07:18:50,325][00611] Updated weights for policy 0, policy_version 86742 (0.0009) [2023-10-08 07:18:50,415][00612] Updated weights for policy 1, policy_version 87220 (0.0008) [2023-10-08 07:18:50,686][00611] Updated weights for policy 0, policy_version 86752 (0.0008) [2023-10-08 07:18:50,781][00612] Updated weights for policy 1, policy_version 87230 (0.0008) [2023-10-08 07:18:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 178159616. Throughput: 0: 1821.7, 1: 1838.0. Samples: 44545468. Policy #0 lag: (min: 8.0, avg: 30.1, max: 32.0) [2023-10-08 07:18:53,754][130385] Avg episode reward: [(0, '76.730'), (1, '69.060')] [2023-10-08 07:18:54,399][00612] Updated weights for policy 1, policy_version 87240 (0.0008) [2023-10-08 07:18:54,409][00611] Updated weights for policy 0, policy_version 86762 (0.0009) [2023-10-08 07:18:54,772][00612] Updated weights for policy 1, policy_version 87250 (0.0009) [2023-10-08 07:18:54,784][00611] Updated weights for policy 0, policy_version 86772 (0.0008) [2023-10-08 07:18:55,144][00612] Updated weights for policy 1, policy_version 87260 (0.0008) [2023-10-08 07:18:55,152][00611] Updated weights for policy 0, policy_version 86782 (0.0008) [2023-10-08 07:18:58,745][00612] Updated weights for policy 1, policy_version 87270 (0.0009) [2023-10-08 07:18:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 178225152. Throughput: 0: 1811.6, 1: 1840.9. Samples: 44568268. Policy #0 lag: (min: 8.0, avg: 30.1, max: 32.0) [2023-10-08 07:18:58,754][130385] Avg episode reward: [(0, '78.220'), (1, '68.340')] [2023-10-08 07:18:58,907][00611] Updated weights for policy 0, policy_version 86792 (0.0008) [2023-10-08 07:18:59,119][00612] Updated weights for policy 1, policy_version 87280 (0.0009) [2023-10-08 07:18:59,278][00611] Updated weights for policy 0, policy_version 86802 (0.0007) [2023-10-08 07:18:59,485][00612] Updated weights for policy 1, policy_version 87290 (0.0009) [2023-10-08 07:18:59,641][00611] Updated weights for policy 0, policy_version 86812 (0.0007) [2023-10-08 07:19:03,018][00612] Updated weights for policy 1, policy_version 87300 (0.0009) [2023-10-08 07:19:03,377][00612] Updated weights for policy 1, policy_version 87310 (0.0009) [2023-10-08 07:19:03,427][00611] Updated weights for policy 0, policy_version 86822 (0.0007) [2023-10-08 07:19:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 178290688. Throughput: 0: 1809.1, 1: 1840.1. Samples: 44591060. Policy #0 lag: (min: 8.0, avg: 30.1, max: 32.0) [2023-10-08 07:19:03,754][00612] Updated weights for policy 1, policy_version 87320 (0.0008) [2023-10-08 07:19:03,754][130385] Avg episode reward: [(0, '71.430'), (1, '68.590')] [2023-10-08 07:19:03,794][00611] Updated weights for policy 0, policy_version 86832 (0.0007) [2023-10-08 07:19:04,166][00611] Updated weights for policy 0, policy_version 86842 (0.0009) [2023-10-08 07:19:07,489][00612] Updated weights for policy 1, policy_version 87330 (0.0007) [2023-10-08 07:19:07,859][00612] Updated weights for policy 1, policy_version 87340 (0.0007) [2023-10-08 07:19:07,925][00611] Updated weights for policy 0, policy_version 86852 (0.0007) [2023-10-08 07:19:08,230][00612] Updated weights for policy 1, policy_version 87350 (0.0009) [2023-10-08 07:19:08,295][00611] Updated weights for policy 0, policy_version 86862 (0.0009) [2023-10-08 07:19:08,592][00612] Updated weights for policy 1, policy_version 87360 (0.0009) [2023-10-08 07:19:08,663][00611] Updated weights for policy 0, policy_version 86872 (0.0008) [2023-10-08 07:19:08,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 178388992. Throughput: 0: 1812.4, 1: 1849.1. Samples: 44601250. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:19:08,754][130385] Avg episode reward: [(0, '71.330'), (1, '70.800')] [2023-10-08 07:19:12,333][00612] Updated weights for policy 1, policy_version 87370 (0.0008) [2023-10-08 07:19:12,413][00611] Updated weights for policy 0, policy_version 86882 (0.0009) [2023-10-08 07:19:12,706][00612] Updated weights for policy 1, policy_version 87380 (0.0008) [2023-10-08 07:19:12,781][00611] Updated weights for policy 0, policy_version 86892 (0.0008) [2023-10-08 07:19:13,078][00612] Updated weights for policy 1, policy_version 87390 (0.0010) [2023-10-08 07:19:13,152][00611] Updated weights for policy 0, policy_version 86902 (0.0007) [2023-10-08 07:19:13,523][00611] Updated weights for policy 0, policy_version 86912 (0.0010) [2023-10-08 07:19:13,754][130385] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 178487296. Throughput: 0: 1809.9, 1: 1845.5. Samples: 44623662. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:19:13,755][130385] Avg episode reward: [(0, '72.670'), (1, '69.750')] [2023-10-08 07:19:16,772][00612] Updated weights for policy 1, policy_version 87400 (0.0009) [2023-10-08 07:19:17,151][00612] Updated weights for policy 1, policy_version 87410 (0.0007) [2023-10-08 07:19:17,321][00611] Updated weights for policy 0, policy_version 86922 (0.0008) [2023-10-08 07:19:17,514][00612] Updated weights for policy 1, policy_version 87420 (0.0007) [2023-10-08 07:19:17,695][00611] Updated weights for policy 0, policy_version 86932 (0.0007) [2023-10-08 07:19:18,059][00611] Updated weights for policy 0, policy_version 86942 (0.0011) [2023-10-08 07:19:18,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 178552832. Throughput: 0: 1807.7, 1: 1847.5. Samples: 44643898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:19:18,754][130385] Avg episode reward: [(0, '72.310'), (1, '73.370')] [2023-10-08 07:19:21,142][00612] Updated weights for policy 1, policy_version 87430 (0.0008) [2023-10-08 07:19:21,514][00612] Updated weights for policy 1, policy_version 87440 (0.0007) [2023-10-08 07:19:21,747][00611] Updated weights for policy 0, policy_version 86952 (0.0008) [2023-10-08 07:19:21,888][00612] Updated weights for policy 1, policy_version 87450 (0.0010) [2023-10-08 07:19:22,116][00611] Updated weights for policy 0, policy_version 86962 (0.0007) [2023-10-08 07:19:22,489][00611] Updated weights for policy 0, policy_version 86972 (0.0007) [2023-10-08 07:19:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 178618368. Throughput: 0: 1809.3, 1: 1838.4. Samples: 44656440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:19:23,754][130385] Avg episode reward: [(0, '65.570'), (1, '75.110')] [2023-10-08 07:19:25,591][00612] Updated weights for policy 1, policy_version 87460 (0.0009) [2023-10-08 07:19:25,949][00612] Updated weights for policy 1, policy_version 87470 (0.0009) [2023-10-08 07:19:26,144][00611] Updated weights for policy 0, policy_version 86982 (0.0007) [2023-10-08 07:19:26,317][00612] Updated weights for policy 1, policy_version 87480 (0.0007) [2023-10-08 07:19:26,506][00611] Updated weights for policy 0, policy_version 86992 (0.0007) [2023-10-08 07:19:26,879][00611] Updated weights for policy 0, policy_version 87002 (0.0008) [2023-10-08 07:19:28,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 178683904. Throughput: 0: 1809.3, 1: 1845.5. Samples: 44676642. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:19:28,755][130385] Avg episode reward: [(0, '65.810'), (1, '75.680')] [2023-10-08 07:19:29,997][00612] Updated weights for policy 1, policy_version 87490 (0.0007) [2023-10-08 07:19:30,360][00612] Updated weights for policy 1, policy_version 87500 (0.0010) [2023-10-08 07:19:30,543][00611] Updated weights for policy 0, policy_version 87012 (0.0008) [2023-10-08 07:19:30,732][00612] Updated weights for policy 1, policy_version 87510 (0.0009) [2023-10-08 07:19:30,921][00611] Updated weights for policy 0, policy_version 87022 (0.0008) [2023-10-08 07:19:31,091][00612] Updated weights for policy 1, policy_version 87520 (0.0009) [2023-10-08 07:19:31,286][00611] Updated weights for policy 0, policy_version 87032 (0.0009) [2023-10-08 07:19:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 178749440. Throughput: 0: 1816.7, 1: 1844.6. Samples: 44700002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:19:33,754][130385] Avg episode reward: [(0, '65.340'), (1, '78.220')] [2023-10-08 07:19:34,781][00612] Updated weights for policy 1, policy_version 87530 (0.0008) [2023-10-08 07:19:34,798][00611] Updated weights for policy 0, policy_version 87042 (0.0007) [2023-10-08 07:19:35,145][00612] Updated weights for policy 1, policy_version 87540 (0.0007) [2023-10-08 07:19:35,165][00611] Updated weights for policy 0, policy_version 87052 (0.0008) [2023-10-08 07:19:35,509][00612] Updated weights for policy 1, policy_version 87550 (0.0008) [2023-10-08 07:19:35,534][00611] Updated weights for policy 0, policy_version 87062 (0.0010) [2023-10-08 07:19:35,902][00611] Updated weights for policy 0, policy_version 87072 (0.0007) [2023-10-08 07:19:38,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14199.6, 300 sec: 14662.3). Total num frames: 178814976. Throughput: 0: 1813.9, 1: 1841.3. Samples: 44709950. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:19:38,754][130385] Avg episode reward: [(0, '67.950'), (1, '82.670')] [2023-10-08 07:19:38,986][00612] Updated weights for policy 1, policy_version 87560 (0.0009) [2023-10-08 07:19:39,360][00612] Updated weights for policy 1, policy_version 87570 (0.0009) [2023-10-08 07:19:39,456][00611] Updated weights for policy 0, policy_version 87082 (0.0007) [2023-10-08 07:19:39,728][00612] Updated weights for policy 1, policy_version 87580 (0.0007) [2023-10-08 07:19:39,831][00611] Updated weights for policy 0, policy_version 87092 (0.0007) [2023-10-08 07:19:40,214][00611] Updated weights for policy 0, policy_version 87102 (0.0007) [2023-10-08 07:19:43,359][00612] Updated weights for policy 1, policy_version 87590 (0.0008) [2023-10-08 07:19:43,724][00612] Updated weights for policy 1, policy_version 87600 (0.0009) [2023-10-08 07:19:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 178880512. Throughput: 0: 1822.0, 1: 1845.9. Samples: 44733324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:19:43,754][130385] Avg episode reward: [(0, '67.760'), (1, '84.980')] [2023-10-08 07:19:43,931][00611] Updated weights for policy 0, policy_version 87112 (0.0009) [2023-10-08 07:19:44,097][00612] Updated weights for policy 1, policy_version 87610 (0.0010) [2023-10-08 07:19:44,308][00611] Updated weights for policy 0, policy_version 87122 (0.0008) [2023-10-08 07:19:44,684][00611] Updated weights for policy 0, policy_version 87132 (0.0008) [2023-10-08 07:19:47,829][00612] Updated weights for policy 1, policy_version 87620 (0.0008) [2023-10-08 07:19:48,212][00612] Updated weights for policy 1, policy_version 87630 (0.0010) [2023-10-08 07:19:48,377][00611] Updated weights for policy 0, policy_version 87142 (0.0007) [2023-10-08 07:19:48,577][00612] Updated weights for policy 1, policy_version 87640 (0.0007) [2023-10-08 07:19:48,746][00611] Updated weights for policy 0, policy_version 87152 (0.0008) [2023-10-08 07:19:48,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 178946048. Throughput: 0: 1820.8, 1: 1832.3. Samples: 44755452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:19:48,754][130385] Avg episode reward: [(0, '65.530'), (1, '84.500')] [2023-10-08 07:19:49,113][00611] Updated weights for policy 0, policy_version 87162 (0.0008) [2023-10-08 07:19:52,101][00612] Updated weights for policy 1, policy_version 87650 (0.0008) [2023-10-08 07:19:52,475][00612] Updated weights for policy 1, policy_version 87660 (0.0008) [2023-10-08 07:19:52,753][00611] Updated weights for policy 0, policy_version 87172 (0.0007) [2023-10-08 07:19:52,844][00612] Updated weights for policy 1, policy_version 87670 (0.0009) [2023-10-08 07:19:53,120][00611] Updated weights for policy 0, policy_version 87182 (0.0009) [2023-10-08 07:19:53,221][00612] Updated weights for policy 1, policy_version 87680 (0.0007) [2023-10-08 07:19:53,491][00611] Updated weights for policy 0, policy_version 87192 (0.0008) [2023-10-08 07:19:53,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 179044352. Throughput: 0: 1818.5, 1: 1845.5. Samples: 44766128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:19:53,754][130385] Avg episode reward: [(0, '63.800'), (1, '86.570')] [2023-10-08 07:19:56,758][00612] Updated weights for policy 1, policy_version 87690 (0.0010) [2023-10-08 07:19:57,136][00612] Updated weights for policy 1, policy_version 87700 (0.0010) [2023-10-08 07:19:57,209][00611] Updated weights for policy 0, policy_version 87202 (0.0007) [2023-10-08 07:19:57,498][00612] Updated weights for policy 1, policy_version 87710 (0.0008) [2023-10-08 07:19:57,612][00611] Updated weights for policy 0, policy_version 87212 (0.0009) [2023-10-08 07:19:57,976][00611] Updated weights for policy 0, policy_version 87222 (0.0008) [2023-10-08 07:19:58,342][00611] Updated weights for policy 0, policy_version 87232 (0.0009) [2023-10-08 07:19:58,754][130385] Fps is (10 sec: 19660.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 179142656. Throughput: 0: 1827.4, 1: 1831.9. Samples: 44788330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:19:58,755][130385] Avg episode reward: [(0, '60.680'), (1, '83.900')] [2023-10-08 07:20:00,995][00612] Updated weights for policy 1, policy_version 87720 (0.0008) [2023-10-08 07:20:01,373][00612] Updated weights for policy 1, policy_version 87730 (0.0011) [2023-10-08 07:20:01,733][00612] Updated weights for policy 1, policy_version 87740 (0.0009) [2023-10-08 07:20:01,940][00611] Updated weights for policy 0, policy_version 87242 (0.0008) [2023-10-08 07:20:02,314][00611] Updated weights for policy 0, policy_version 87252 (0.0007) [2023-10-08 07:20:02,683][00611] Updated weights for policy 0, policy_version 87262 (0.0007) [2023-10-08 07:20:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 179208192. Throughput: 0: 1826.9, 1: 1852.3. Samples: 44809462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:20:03,754][130385] Avg episode reward: [(0, '62.860'), (1, '85.180')] [2023-10-08 07:20:05,232][00612] Updated weights for policy 1, policy_version 87750 (0.0007) [2023-10-08 07:20:05,599][00612] Updated weights for policy 1, policy_version 87760 (0.0008) [2023-10-08 07:20:05,962][00612] Updated weights for policy 1, policy_version 87770 (0.0010) [2023-10-08 07:20:06,480][00611] Updated weights for policy 0, policy_version 87272 (0.0008) [2023-10-08 07:20:06,849][00611] Updated weights for policy 0, policy_version 87282 (0.0007) [2023-10-08 07:20:07,214][00611] Updated weights for policy 0, policy_version 87292 (0.0008) [2023-10-08 07:20:08,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 179273728. Throughput: 0: 1829.4, 1: 1832.2. Samples: 44821210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:20:08,754][130385] Avg episode reward: [(0, '65.230'), (1, '83.180')] [2023-10-08 07:20:09,704][00612] Updated weights for policy 1, policy_version 87780 (0.0009) [2023-10-08 07:20:10,070][00612] Updated weights for policy 1, policy_version 87790 (0.0007) [2023-10-08 07:20:10,442][00612] Updated weights for policy 1, policy_version 87800 (0.0008) [2023-10-08 07:20:10,761][00611] Updated weights for policy 0, policy_version 87302 (0.0008) [2023-10-08 07:20:11,129][00611] Updated weights for policy 0, policy_version 87312 (0.0009) [2023-10-08 07:20:11,502][00611] Updated weights for policy 0, policy_version 87322 (0.0010) [2023-10-08 07:20:13,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 179339264. Throughput: 0: 1834.1, 1: 1858.7. Samples: 44842818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:20:13,754][130385] Avg episode reward: [(0, '63.050'), (1, '81.890')] [2023-10-08 07:20:14,199][00612] Updated weights for policy 1, policy_version 87810 (0.0008) [2023-10-08 07:20:14,561][00612] Updated weights for policy 1, policy_version 87820 (0.0009) [2023-10-08 07:20:14,923][00612] Updated weights for policy 1, policy_version 87830 (0.0011) [2023-10-08 07:20:15,224][00611] Updated weights for policy 0, policy_version 87332 (0.0010) [2023-10-08 07:20:15,290][00612] Updated weights for policy 1, policy_version 87840 (0.0008) [2023-10-08 07:20:15,590][00611] Updated weights for policy 0, policy_version 87342 (0.0010) [2023-10-08 07:20:15,965][00611] Updated weights for policy 0, policy_version 87352 (0.0008) [2023-10-08 07:20:18,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 179404800. Throughput: 0: 1828.5, 1: 1858.5. Samples: 44865918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:20:18,755][130385] Avg episode reward: [(0, '61.410'), (1, '81.180')] [2023-10-08 07:20:18,766][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000087360_89456640.pth... [2023-10-08 07:20:18,804][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000085664_87719936.pth [2023-10-08 07:20:18,883][00612] Updated weights for policy 1, policy_version 87850 (0.0009) [2023-10-08 07:20:19,263][00612] Updated weights for policy 1, policy_version 87860 (0.0009) [2023-10-08 07:20:19,586][00611] Updated weights for policy 0, policy_version 87362 (0.0009) [2023-10-08 07:20:19,626][00612] Updated weights for policy 1, policy_version 87870 (0.0009) [2023-10-08 07:20:19,696][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000087872_89980928.pth... [2023-10-08 07:20:19,724][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000086144_88211456.pth [2023-10-08 07:20:19,954][00611] Updated weights for policy 0, policy_version 87372 (0.0009) [2023-10-08 07:20:20,329][00611] Updated weights for policy 0, policy_version 87382 (0.0010) [2023-10-08 07:20:20,691][00611] Updated weights for policy 0, policy_version 87392 (0.0010) [2023-10-08 07:20:23,363][00612] Updated weights for policy 1, policy_version 87880 (0.0008) [2023-10-08 07:20:23,731][00612] Updated weights for policy 1, policy_version 87890 (0.0008) [2023-10-08 07:20:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 179470336. Throughput: 0: 1831.2, 1: 1855.6. Samples: 44875858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:20:23,754][130385] Avg episode reward: [(0, '65.930'), (1, '79.140')] [2023-10-08 07:20:24,101][00612] Updated weights for policy 1, policy_version 87900 (0.0010) [2023-10-08 07:20:24,369][00611] Updated weights for policy 0, policy_version 87402 (0.0007) [2023-10-08 07:20:24,745][00611] Updated weights for policy 0, policy_version 87412 (0.0007) [2023-10-08 07:20:25,113][00611] Updated weights for policy 0, policy_version 87422 (0.0008) [2023-10-08 07:20:27,940][00612] Updated weights for policy 1, policy_version 87910 (0.0009) [2023-10-08 07:20:28,311][00612] Updated weights for policy 1, policy_version 87920 (0.0009) [2023-10-08 07:20:28,683][00612] Updated weights for policy 1, policy_version 87930 (0.0008) [2023-10-08 07:20:28,700][00611] Updated weights for policy 0, policy_version 87432 (0.0007) [2023-10-08 07:20:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 179535872. Throughput: 0: 1831.1, 1: 1844.2. Samples: 44898712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:20:28,755][130385] Avg episode reward: [(0, '65.710'), (1, '76.560')] [2023-10-08 07:20:29,063][00611] Updated weights for policy 0, policy_version 87442 (0.0008) [2023-10-08 07:20:29,437][00611] Updated weights for policy 0, policy_version 87452 (0.0009) [2023-10-08 07:20:32,342][00612] Updated weights for policy 1, policy_version 87940 (0.0008) [2023-10-08 07:20:32,717][00612] Updated weights for policy 1, policy_version 87950 (0.0008) [2023-10-08 07:20:33,094][00612] Updated weights for policy 1, policy_version 87960 (0.0007) [2023-10-08 07:20:33,109][00611] Updated weights for policy 0, policy_version 87462 (0.0008) [2023-10-08 07:20:33,482][00611] Updated weights for policy 0, policy_version 87472 (0.0010) [2023-10-08 07:20:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 179634176. Throughput: 0: 1829.6, 1: 1834.4. Samples: 44920334. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:20:33,754][130385] Avg episode reward: [(0, '67.380'), (1, '78.080')] [2023-10-08 07:20:33,852][00611] Updated weights for policy 0, policy_version 87482 (0.0008) [2023-10-08 07:20:36,741][00612] Updated weights for policy 1, policy_version 87970 (0.0008) [2023-10-08 07:20:37,152][00612] Updated weights for policy 1, policy_version 87980 (0.0007) [2023-10-08 07:20:37,525][00612] Updated weights for policy 1, policy_version 87990 (0.0008) [2023-10-08 07:20:37,596][00611] Updated weights for policy 0, policy_version 87492 (0.0008) [2023-10-08 07:20:37,889][00612] Updated weights for policy 1, policy_version 88000 (0.0007) [2023-10-08 07:20:37,967][00611] Updated weights for policy 0, policy_version 87502 (0.0008) [2023-10-08 07:20:38,334][00611] Updated weights for policy 0, policy_version 87512 (0.0011) [2023-10-08 07:20:38,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 179732480. Throughput: 0: 1836.3, 1: 1847.7. Samples: 44931910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:20:38,755][130385] Avg episode reward: [(0, '71.580'), (1, '77.150')] [2023-10-08 07:20:41,560][00612] Updated weights for policy 1, policy_version 88010 (0.0008) [2023-10-08 07:20:41,930][00612] Updated weights for policy 1, policy_version 88020 (0.0008) [2023-10-08 07:20:42,033][00611] Updated weights for policy 0, policy_version 87522 (0.0009) [2023-10-08 07:20:42,306][00612] Updated weights for policy 1, policy_version 88030 (0.0007) [2023-10-08 07:20:42,432][00611] Updated weights for policy 0, policy_version 87532 (0.0010) [2023-10-08 07:20:42,804][00611] Updated weights for policy 0, policy_version 87542 (0.0010) [2023-10-08 07:20:43,176][00611] Updated weights for policy 0, policy_version 87552 (0.0007) [2023-10-08 07:20:43,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 179798016. Throughput: 0: 1827.5, 1: 1837.1. Samples: 44953236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:20:43,754][130385] Avg episode reward: [(0, '69.730'), (1, '82.180')] [2023-10-08 07:20:45,868][00612] Updated weights for policy 1, policy_version 88040 (0.0008) [2023-10-08 07:20:46,241][00612] Updated weights for policy 1, policy_version 88050 (0.0009) [2023-10-08 07:20:46,615][00612] Updated weights for policy 1, policy_version 88060 (0.0010) [2023-10-08 07:20:46,939][00611] Updated weights for policy 0, policy_version 87562 (0.0008) [2023-10-08 07:20:47,310][00611] Updated weights for policy 0, policy_version 87572 (0.0007) [2023-10-08 07:20:47,691][00611] Updated weights for policy 0, policy_version 87582 (0.0008) [2023-10-08 07:20:48,754][130385] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 179863552. Throughput: 0: 1831.2, 1: 1835.5. Samples: 44974468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:20:48,755][130385] Avg episode reward: [(0, '67.890'), (1, '83.030')] [2023-10-08 07:20:50,233][00612] Updated weights for policy 1, policy_version 88070 (0.0008) [2023-10-08 07:20:50,604][00612] Updated weights for policy 1, policy_version 88080 (0.0009) [2023-10-08 07:20:50,971][00612] Updated weights for policy 1, policy_version 88090 (0.0008) [2023-10-08 07:20:51,293][00611] Updated weights for policy 0, policy_version 87592 (0.0008) [2023-10-08 07:20:51,672][00611] Updated weights for policy 0, policy_version 87602 (0.0008) [2023-10-08 07:20:52,046][00611] Updated weights for policy 0, policy_version 87612 (0.0007) [2023-10-08 07:20:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 179929088. Throughput: 0: 1827.0, 1: 1834.0. Samples: 44985954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:20:53,754][130385] Avg episode reward: [(0, '67.510'), (1, '82.270')] [2023-10-08 07:20:54,501][00612] Updated weights for policy 1, policy_version 88100 (0.0009) [2023-10-08 07:20:54,873][00612] Updated weights for policy 1, policy_version 88110 (0.0010) [2023-10-08 07:20:55,233][00612] Updated weights for policy 1, policy_version 88120 (0.0007) [2023-10-08 07:20:55,640][00611] Updated weights for policy 0, policy_version 87622 (0.0009) [2023-10-08 07:20:56,011][00611] Updated weights for policy 0, policy_version 87632 (0.0010) [2023-10-08 07:20:56,384][00611] Updated weights for policy 0, policy_version 87642 (0.0008) [2023-10-08 07:20:58,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 179994624. Throughput: 0: 1828.5, 1: 1836.2. Samples: 45007730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:20:58,754][130385] Avg episode reward: [(0, '70.190'), (1, '81.910')] [2023-10-08 07:20:58,824][00612] Updated weights for policy 1, policy_version 88130 (0.0010) [2023-10-08 07:20:59,177][00612] Updated weights for policy 1, policy_version 88140 (0.0007) [2023-10-08 07:20:59,542][00612] Updated weights for policy 1, policy_version 88150 (0.0009) [2023-10-08 07:20:59,911][00612] Updated weights for policy 1, policy_version 88160 (0.0008) [2023-10-08 07:21:00,079][00611] Updated weights for policy 0, policy_version 87652 (0.0009) [2023-10-08 07:21:00,446][00611] Updated weights for policy 0, policy_version 87662 (0.0007) [2023-10-08 07:21:00,824][00611] Updated weights for policy 0, policy_version 87672 (0.0008) [2023-10-08 07:21:03,541][00612] Updated weights for policy 1, policy_version 88170 (0.0009) [2023-10-08 07:21:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 180060160. Throughput: 0: 1827.0, 1: 1835.7. Samples: 45030738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:21:03,754][130385] Avg episode reward: [(0, '69.860'), (1, '78.770')] [2023-10-08 07:21:03,908][00612] Updated weights for policy 1, policy_version 88180 (0.0010) [2023-10-08 07:21:04,275][00612] Updated weights for policy 1, policy_version 88190 (0.0010) [2023-10-08 07:21:04,473][00611] Updated weights for policy 0, policy_version 87682 (0.0010) [2023-10-08 07:21:04,848][00611] Updated weights for policy 0, policy_version 87692 (0.0010) [2023-10-08 07:21:05,224][00611] Updated weights for policy 0, policy_version 87702 (0.0011) [2023-10-08 07:21:05,597][00611] Updated weights for policy 0, policy_version 87712 (0.0008) [2023-10-08 07:21:07,865][00612] Updated weights for policy 1, policy_version 88200 (0.0007) [2023-10-08 07:21:08,224][00612] Updated weights for policy 1, policy_version 88210 (0.0008) [2023-10-08 07:21:08,593][00612] Updated weights for policy 1, policy_version 88220 (0.0007) [2023-10-08 07:21:08,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 180158464. Throughput: 0: 1827.9, 1: 1836.9. Samples: 45040776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:21:08,754][130385] Avg episode reward: [(0, '68.290'), (1, '82.370')] [2023-10-08 07:21:09,192][00611] Updated weights for policy 0, policy_version 87722 (0.0008) [2023-10-08 07:21:09,555][00611] Updated weights for policy 0, policy_version 87732 (0.0008) [2023-10-08 07:21:09,925][00611] Updated weights for policy 0, policy_version 87742 (0.0007) [2023-10-08 07:21:12,392][00612] Updated weights for policy 1, policy_version 88230 (0.0009) [2023-10-08 07:21:12,760][00612] Updated weights for policy 1, policy_version 88240 (0.0007) [2023-10-08 07:21:13,139][00612] Updated weights for policy 1, policy_version 88250 (0.0008) [2023-10-08 07:21:13,520][00611] Updated weights for policy 0, policy_version 87752 (0.0008) [2023-10-08 07:21:13,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180224000. Throughput: 0: 1832.1, 1: 1842.6. Samples: 45064074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:21:13,754][130385] Avg episode reward: [(0, '69.090'), (1, '83.290')] [2023-10-08 07:21:13,882][00611] Updated weights for policy 0, policy_version 87762 (0.0009) [2023-10-08 07:21:14,253][00611] Updated weights for policy 0, policy_version 87772 (0.0009) [2023-10-08 07:21:16,756][00612] Updated weights for policy 1, policy_version 88260 (0.0009) [2023-10-08 07:21:17,121][00612] Updated weights for policy 1, policy_version 88270 (0.0010) [2023-10-08 07:21:17,492][00612] Updated weights for policy 1, policy_version 88280 (0.0010) [2023-10-08 07:21:17,819][00611] Updated weights for policy 0, policy_version 87782 (0.0008) [2023-10-08 07:21:18,184][00611] Updated weights for policy 0, policy_version 87792 (0.0008) [2023-10-08 07:21:18,558][00611] Updated weights for policy 0, policy_version 87802 (0.0009) [2023-10-08 07:21:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180289536. Throughput: 0: 1825.3, 1: 1834.9. Samples: 45085046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:21:18,755][130385] Avg episode reward: [(0, '67.740'), (1, '83.570')] [2023-10-08 07:21:21,198][00612] Updated weights for policy 1, policy_version 88290 (0.0007) [2023-10-08 07:21:21,579][00612] Updated weights for policy 1, policy_version 88300 (0.0008) [2023-10-08 07:21:21,952][00612] Updated weights for policy 1, policy_version 88310 (0.0010) [2023-10-08 07:21:22,219][00611] Updated weights for policy 0, policy_version 87812 (0.0008) [2023-10-08 07:21:22,313][00612] Updated weights for policy 1, policy_version 88320 (0.0009) [2023-10-08 07:21:22,592][00611] Updated weights for policy 0, policy_version 87822 (0.0010) [2023-10-08 07:21:22,966][00611] Updated weights for policy 0, policy_version 87832 (0.0008) [2023-10-08 07:21:23,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 180387840. Throughput: 0: 1837.0, 1: 1837.2. Samples: 45097248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:21:23,755][130385] Avg episode reward: [(0, '67.240'), (1, '86.270')] [2023-10-08 07:21:25,779][00612] Updated weights for policy 1, policy_version 88330 (0.0009) [2023-10-08 07:21:26,149][00612] Updated weights for policy 1, policy_version 88340 (0.0011) [2023-10-08 07:21:26,523][00612] Updated weights for policy 1, policy_version 88350 (0.0009) [2023-10-08 07:21:26,594][00611] Updated weights for policy 0, policy_version 87842 (0.0007) [2023-10-08 07:21:26,969][00611] Updated weights for policy 0, policy_version 87852 (0.0008) [2023-10-08 07:21:27,339][00611] Updated weights for policy 0, policy_version 87862 (0.0009) [2023-10-08 07:21:27,709][00611] Updated weights for policy 0, policy_version 87872 (0.0011) [2023-10-08 07:21:28,754][130385] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 180453376. Throughput: 0: 1824.2, 1: 1842.6. Samples: 45118240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:21:28,754][130385] Avg episode reward: [(0, '70.270'), (1, '87.240')] [2023-10-08 07:21:30,012][00612] Updated weights for policy 1, policy_version 88360 (0.0008) [2023-10-08 07:21:30,371][00612] Updated weights for policy 1, policy_version 88370 (0.0008) [2023-10-08 07:21:30,732][00612] Updated weights for policy 1, policy_version 88380 (0.0008) [2023-10-08 07:21:31,449][00611] Updated weights for policy 0, policy_version 87882 (0.0007) [2023-10-08 07:21:31,818][00611] Updated weights for policy 0, policy_version 87892 (0.0009) [2023-10-08 07:21:32,198][00611] Updated weights for policy 0, policy_version 87902 (0.0009) [2023-10-08 07:21:33,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180518912. Throughput: 0: 1835.6, 1: 1858.9. Samples: 45140720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:21:33,754][130385] Avg episode reward: [(0, '69.410'), (1, '85.760')] [2023-10-08 07:21:34,285][00612] Updated weights for policy 1, policy_version 88390 (0.0008) [2023-10-08 07:21:34,658][00612] Updated weights for policy 1, policy_version 88400 (0.0008) [2023-10-08 07:21:35,017][00612] Updated weights for policy 1, policy_version 88410 (0.0007) [2023-10-08 07:21:35,714][00611] Updated weights for policy 0, policy_version 87912 (0.0009) [2023-10-08 07:21:36,083][00611] Updated weights for policy 0, policy_version 87922 (0.0011) [2023-10-08 07:21:36,457][00611] Updated weights for policy 0, policy_version 87932 (0.0009) [2023-10-08 07:21:38,712][00612] Updated weights for policy 1, policy_version 88420 (0.0009) [2023-10-08 07:21:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 180584448. Throughput: 0: 1825.1, 1: 1854.7. Samples: 45151546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:21:38,754][130385] Avg episode reward: [(0, '72.710'), (1, '84.690')] [2023-10-08 07:21:39,077][00612] Updated weights for policy 1, policy_version 88430 (0.0008) [2023-10-08 07:21:39,446][00612] Updated weights for policy 1, policy_version 88440 (0.0007) [2023-10-08 07:21:40,075][00611] Updated weights for policy 0, policy_version 87942 (0.0007) [2023-10-08 07:21:40,440][00611] Updated weights for policy 0, policy_version 87952 (0.0010) [2023-10-08 07:21:40,810][00611] Updated weights for policy 0, policy_version 87962 (0.0007) [2023-10-08 07:21:43,026][00612] Updated weights for policy 1, policy_version 88450 (0.0008) [2023-10-08 07:21:43,399][00612] Updated weights for policy 1, policy_version 88460 (0.0007) [2023-10-08 07:21:43,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 180649984. Throughput: 0: 1843.2, 1: 1855.2. Samples: 45174158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:21:43,755][130385] Avg episode reward: [(0, '67.010'), (1, '84.760')] [2023-10-08 07:21:43,763][00612] Updated weights for policy 1, policy_version 88470 (0.0007) [2023-10-08 07:21:44,132][00612] Updated weights for policy 1, policy_version 88480 (0.0007) [2023-10-08 07:21:44,420][00611] Updated weights for policy 0, policy_version 87972 (0.0008) [2023-10-08 07:21:44,795][00611] Updated weights for policy 0, policy_version 87982 (0.0009) [2023-10-08 07:21:45,159][00611] Updated weights for policy 0, policy_version 87992 (0.0009) [2023-10-08 07:21:47,780][00612] Updated weights for policy 1, policy_version 88490 (0.0008) [2023-10-08 07:21:48,147][00612] Updated weights for policy 1, policy_version 88500 (0.0011) [2023-10-08 07:21:48,508][00612] Updated weights for policy 1, policy_version 88510 (0.0009) [2023-10-08 07:21:48,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 180748288. Throughput: 0: 1848.8, 1: 1835.9. Samples: 45196550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:21:48,754][130385] Avg episode reward: [(0, '66.840'), (1, '86.730')] [2023-10-08 07:21:48,790][00611] Updated weights for policy 0, policy_version 88002 (0.0009) [2023-10-08 07:21:49,159][00611] Updated weights for policy 0, policy_version 88012 (0.0011) [2023-10-08 07:21:49,538][00611] Updated weights for policy 0, policy_version 88022 (0.0011) [2023-10-08 07:21:49,905][00611] Updated weights for policy 0, policy_version 88032 (0.0010) [2023-10-08 07:21:52,041][00612] Updated weights for policy 1, policy_version 88520 (0.0010) [2023-10-08 07:21:52,416][00612] Updated weights for policy 1, policy_version 88530 (0.0008) [2023-10-08 07:21:52,782][00612] Updated weights for policy 1, policy_version 88540 (0.0007) [2023-10-08 07:21:53,486][00611] Updated weights for policy 0, policy_version 88042 (0.0007) [2023-10-08 07:21:53,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 180813824. Throughput: 0: 1845.5, 1: 1859.9. Samples: 45207518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:21:53,755][130385] Avg episode reward: [(0, '73.630'), (1, '85.710')] [2023-10-08 07:21:53,858][00611] Updated weights for policy 0, policy_version 88052 (0.0007) [2023-10-08 07:21:54,227][00611] Updated weights for policy 0, policy_version 88062 (0.0008) [2023-10-08 07:21:56,337][00612] Updated weights for policy 1, policy_version 88550 (0.0009) [2023-10-08 07:21:56,701][00612] Updated weights for policy 1, policy_version 88560 (0.0007) [2023-10-08 07:21:57,068][00612] Updated weights for policy 1, policy_version 88570 (0.0011) [2023-10-08 07:21:58,091][00611] Updated weights for policy 0, policy_version 88072 (0.0009) [2023-10-08 07:21:58,457][00611] Updated weights for policy 0, policy_version 88082 (0.0007) [2023-10-08 07:21:58,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180879360. Throughput: 0: 1844.8, 1: 1832.5. Samples: 45229552. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 07:21:58,754][130385] Avg episode reward: [(0, '75.830'), (1, '84.490')] [2023-10-08 07:21:58,833][00611] Updated weights for policy 0, policy_version 88092 (0.0009) [2023-10-08 07:22:00,861][00612] Updated weights for policy 1, policy_version 88580 (0.0011) [2023-10-08 07:22:01,239][00612] Updated weights for policy 1, policy_version 88590 (0.0010) [2023-10-08 07:22:01,594][00612] Updated weights for policy 1, policy_version 88600 (0.0009) [2023-10-08 07:22:02,483][00611] Updated weights for policy 0, policy_version 88102 (0.0009) [2023-10-08 07:22:02,849][00611] Updated weights for policy 0, policy_version 88112 (0.0010) [2023-10-08 07:22:03,228][00611] Updated weights for policy 0, policy_version 88122 (0.0010) [2023-10-08 07:22:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 180977664. Throughput: 0: 1833.5, 1: 1860.9. Samples: 45251296. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 07:22:03,754][130385] Avg episode reward: [(0, '75.240'), (1, '81.800')] [2023-10-08 07:22:05,139][00612] Updated weights for policy 1, policy_version 88610 (0.0009) [2023-10-08 07:22:05,513][00612] Updated weights for policy 1, policy_version 88620 (0.0011) [2023-10-08 07:22:05,885][00612] Updated weights for policy 1, policy_version 88630 (0.0012) [2023-10-08 07:22:06,238][00612] Updated weights for policy 1, policy_version 88640 (0.0010) [2023-10-08 07:22:06,909][00611] Updated weights for policy 0, policy_version 88132 (0.0010) [2023-10-08 07:22:07,274][00611] Updated weights for policy 0, policy_version 88142 (0.0011) [2023-10-08 07:22:07,640][00611] Updated weights for policy 0, policy_version 88152 (0.0010) [2023-10-08 07:22:08,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181043200. Throughput: 0: 1840.3, 1: 1832.0. Samples: 45262500. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 07:22:08,754][130385] Avg episode reward: [(0, '74.280'), (1, '79.780')] [2023-10-08 07:22:09,769][00612] Updated weights for policy 1, policy_version 88650 (0.0009) [2023-10-08 07:22:10,132][00612] Updated weights for policy 1, policy_version 88660 (0.0008) [2023-10-08 07:22:10,501][00612] Updated weights for policy 1, policy_version 88670 (0.0011) [2023-10-08 07:22:11,347][00611] Updated weights for policy 0, policy_version 88162 (0.0009) [2023-10-08 07:22:11,714][00611] Updated weights for policy 0, policy_version 88172 (0.0009) [2023-10-08 07:22:12,087][00611] Updated weights for policy 0, policy_version 88182 (0.0010) [2023-10-08 07:22:12,459][00611] Updated weights for policy 0, policy_version 88192 (0.0009) [2023-10-08 07:22:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181108736. Throughput: 0: 1833.8, 1: 1861.9. Samples: 45284546. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 07:22:13,754][130385] Avg episode reward: [(0, '76.140'), (1, '83.380')] [2023-10-08 07:22:14,175][00612] Updated weights for policy 1, policy_version 88680 (0.0008) [2023-10-08 07:22:14,543][00612] Updated weights for policy 1, policy_version 88690 (0.0007) [2023-10-08 07:22:14,907][00612] Updated weights for policy 1, policy_version 88700 (0.0010) [2023-10-08 07:22:16,103][00611] Updated weights for policy 0, policy_version 88202 (0.0007) [2023-10-08 07:22:16,462][00611] Updated weights for policy 0, policy_version 88212 (0.0007) [2023-10-08 07:22:16,838][00611] Updated weights for policy 0, policy_version 88222 (0.0008) [2023-10-08 07:22:18,703][00612] Updated weights for policy 1, policy_version 88710 (0.0008) [2023-10-08 07:22:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181174272. Throughput: 0: 1844.5, 1: 1853.9. Samples: 45307146. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 07:22:18,755][130385] Avg episode reward: [(0, '75.740'), (1, '86.600')] [2023-10-08 07:22:18,767][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000088224_90341376.pth... [2023-10-08 07:22:18,817][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000086528_88604672.pth [2023-10-08 07:22:19,084][00612] Updated weights for policy 1, policy_version 88720 (0.0009) [2023-10-08 07:22:19,457][00612] Updated weights for policy 1, policy_version 88730 (0.0011) [2023-10-08 07:22:19,670][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000088736_90865664.pth... [2023-10-08 07:22:19,699][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000087008_89096192.pth [2023-10-08 07:22:20,617][00611] Updated weights for policy 0, policy_version 88232 (0.0007) [2023-10-08 07:22:20,992][00611] Updated weights for policy 0, policy_version 88242 (0.0007) [2023-10-08 07:22:21,367][00611] Updated weights for policy 0, policy_version 88252 (0.0007) [2023-10-08 07:22:23,025][00612] Updated weights for policy 1, policy_version 88740 (0.0009) [2023-10-08 07:22:23,383][00612] Updated weights for policy 1, policy_version 88750 (0.0008) [2023-10-08 07:22:23,750][00612] Updated weights for policy 1, policy_version 88760 (0.0007) [2023-10-08 07:22:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 181239808. Throughput: 0: 1838.0, 1: 1850.1. Samples: 45317512. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 07:22:23,754][130385] Avg episode reward: [(0, '78.030'), (1, '88.370')] [2023-10-08 07:22:25,042][00611] Updated weights for policy 0, policy_version 88262 (0.0010) [2023-10-08 07:22:25,414][00611] Updated weights for policy 0, policy_version 88272 (0.0008) [2023-10-08 07:22:25,787][00611] Updated weights for policy 0, policy_version 88282 (0.0007) [2023-10-08 07:22:27,533][00612] Updated weights for policy 1, policy_version 88770 (0.0008) [2023-10-08 07:22:27,904][00612] Updated weights for policy 1, policy_version 88780 (0.0009) [2023-10-08 07:22:28,274][00612] Updated weights for policy 1, policy_version 88790 (0.0012) [2023-10-08 07:22:28,643][00612] Updated weights for policy 1, policy_version 88800 (0.0007) [2023-10-08 07:22:28,754][130385] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 181338112. Throughput: 0: 1838.8, 1: 1852.2. Samples: 45340252. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 07:22:28,754][130385] Avg episode reward: [(0, '78.570'), (1, '87.050')] [2023-10-08 07:22:29,344][00611] Updated weights for policy 0, policy_version 88292 (0.0008) [2023-10-08 07:22:29,711][00611] Updated weights for policy 0, policy_version 88302 (0.0010) [2023-10-08 07:22:30,081][00611] Updated weights for policy 0, policy_version 88312 (0.0008) [2023-10-08 07:22:32,289][00612] Updated weights for policy 1, policy_version 88810 (0.0007) [2023-10-08 07:22:32,654][00612] Updated weights for policy 1, policy_version 88820 (0.0009) [2023-10-08 07:22:33,032][00612] Updated weights for policy 1, policy_version 88830 (0.0010) [2023-10-08 07:22:33,671][00611] Updated weights for policy 0, policy_version 88322 (0.0011) [2023-10-08 07:22:33,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 181403648. Throughput: 0: 1836.8, 1: 1838.2. Samples: 45361924. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 07:22:33,755][130385] Avg episode reward: [(0, '76.550'), (1, '86.210')] [2023-10-08 07:22:34,048][00611] Updated weights for policy 0, policy_version 88332 (0.0008) [2023-10-08 07:22:34,414][00611] Updated weights for policy 0, policy_version 88342 (0.0007) [2023-10-08 07:22:34,787][00611] Updated weights for policy 0, policy_version 88352 (0.0007) [2023-10-08 07:22:36,495][00612] Updated weights for policy 1, policy_version 88840 (0.0009) [2023-10-08 07:22:36,856][00612] Updated weights for policy 1, policy_version 88850 (0.0009) [2023-10-08 07:22:37,221][00612] Updated weights for policy 1, policy_version 88860 (0.0007) [2023-10-08 07:22:38,421][00611] Updated weights for policy 0, policy_version 88362 (0.0008) [2023-10-08 07:22:38,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 181469184. Throughput: 0: 1837.4, 1: 1852.5. Samples: 45373562. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 07:22:38,755][130385] Avg episode reward: [(0, '73.790'), (1, '85.890')] [2023-10-08 07:22:38,789][00611] Updated weights for policy 0, policy_version 88372 (0.0007) [2023-10-08 07:22:39,168][00611] Updated weights for policy 0, policy_version 88382 (0.0009) [2023-10-08 07:22:40,840][00612] Updated weights for policy 1, policy_version 88870 (0.0008) [2023-10-08 07:22:41,210][00612] Updated weights for policy 1, policy_version 88880 (0.0009) [2023-10-08 07:22:41,580][00612] Updated weights for policy 1, policy_version 88890 (0.0009) [2023-10-08 07:22:42,837][00611] Updated weights for policy 0, policy_version 88392 (0.0007) [2023-10-08 07:22:43,205][00611] Updated weights for policy 0, policy_version 88402 (0.0009) [2023-10-08 07:22:43,573][00611] Updated weights for policy 0, policy_version 88412 (0.0008) [2023-10-08 07:22:43,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 181567488. Throughput: 0: 1841.3, 1: 1844.9. Samples: 45395434. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 07:22:43,754][130385] Avg episode reward: [(0, '72.760'), (1, '89.460')] [2023-10-08 07:22:45,272][00612] Updated weights for policy 1, policy_version 88900 (0.0009) [2023-10-08 07:22:45,647][00612] Updated weights for policy 1, policy_version 88910 (0.0008) [2023-10-08 07:22:46,008][00612] Updated weights for policy 1, policy_version 88920 (0.0009) [2023-10-08 07:22:47,069][00611] Updated weights for policy 0, policy_version 88422 (0.0008) [2023-10-08 07:22:47,433][00611] Updated weights for policy 0, policy_version 88432 (0.0009) [2023-10-08 07:22:47,798][00611] Updated weights for policy 0, policy_version 88442 (0.0009) [2023-10-08 07:22:48,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181633024. Throughput: 0: 1832.2, 1: 1850.1. Samples: 45416998. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-08 07:22:48,755][130385] Avg episode reward: [(0, '75.820'), (1, '88.570')] [2023-10-08 07:22:49,715][00612] Updated weights for policy 1, policy_version 88930 (0.0008) [2023-10-08 07:22:50,087][00612] Updated weights for policy 1, policy_version 88940 (0.0011) [2023-10-08 07:22:50,461][00612] Updated weights for policy 1, policy_version 88950 (0.0009) [2023-10-08 07:22:50,820][00612] Updated weights for policy 1, policy_version 88960 (0.0007) [2023-10-08 07:22:51,508][00611] Updated weights for policy 0, policy_version 88452 (0.0008) [2023-10-08 07:22:51,895][00611] Updated weights for policy 0, policy_version 88462 (0.0008) [2023-10-08 07:22:52,274][00611] Updated weights for policy 0, policy_version 88472 (0.0010) [2023-10-08 07:22:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181698560. Throughput: 0: 1841.0, 1: 1846.8. Samples: 45428450. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 07:22:53,754][130385] Avg episode reward: [(0, '78.290'), (1, '90.850')] [2023-10-08 07:22:54,194][00612] Updated weights for policy 1, policy_version 88970 (0.0009) [2023-10-08 07:22:54,560][00612] Updated weights for policy 1, policy_version 88980 (0.0010) [2023-10-08 07:22:54,936][00612] Updated weights for policy 1, policy_version 88990 (0.0009) [2023-10-08 07:22:55,965][00611] Updated weights for policy 0, policy_version 88482 (0.0010) [2023-10-08 07:22:56,329][00611] Updated weights for policy 0, policy_version 88492 (0.0008) [2023-10-08 07:22:56,696][00611] Updated weights for policy 0, policy_version 88502 (0.0009) [2023-10-08 07:22:57,069][00611] Updated weights for policy 0, policy_version 88512 (0.0007) [2023-10-08 07:22:58,570][00612] Updated weights for policy 1, policy_version 89000 (0.0008) [2023-10-08 07:22:58,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181764096. Throughput: 0: 1831.0, 1: 1852.7. Samples: 45450312. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 07:22:58,755][130385] Avg episode reward: [(0, '74.720'), (1, '89.540')] [2023-10-08 07:22:58,934][00612] Updated weights for policy 1, policy_version 89010 (0.0009) [2023-10-08 07:22:59,304][00612] Updated weights for policy 1, policy_version 89020 (0.0010) [2023-10-08 07:23:00,767][00611] Updated weights for policy 0, policy_version 88522 (0.0010) [2023-10-08 07:23:01,139][00611] Updated weights for policy 0, policy_version 88532 (0.0008) [2023-10-08 07:23:01,511][00611] Updated weights for policy 0, policy_version 88542 (0.0011) [2023-10-08 07:23:02,978][00612] Updated weights for policy 1, policy_version 89030 (0.0008) [2023-10-08 07:23:03,352][00612] Updated weights for policy 1, policy_version 89040 (0.0008) [2023-10-08 07:23:03,720][00612] Updated weights for policy 1, policy_version 89050 (0.0008) [2023-10-08 07:23:03,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 181829632. Throughput: 0: 1841.3, 1: 1838.8. Samples: 45472748. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 07:23:03,755][130385] Avg episode reward: [(0, '74.440'), (1, '92.150')] [2023-10-08 07:23:05,240][00611] Updated weights for policy 0, policy_version 88552 (0.0010) [2023-10-08 07:23:05,610][00611] Updated weights for policy 0, policy_version 88562 (0.0008) [2023-10-08 07:23:05,977][00611] Updated weights for policy 0, policy_version 88572 (0.0007) [2023-10-08 07:23:07,397][00612] Updated weights for policy 1, policy_version 89060 (0.0009) [2023-10-08 07:23:07,785][00612] Updated weights for policy 1, policy_version 89070 (0.0009) [2023-10-08 07:23:08,150][00612] Updated weights for policy 1, policy_version 89080 (0.0011) [2023-10-08 07:23:08,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 181927936. Throughput: 0: 1826.5, 1: 1856.4. Samples: 45483240. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 07:23:08,754][130385] Avg episode reward: [(0, '74.220'), (1, '90.610')] [2023-10-08 07:23:09,561][00611] Updated weights for policy 0, policy_version 88582 (0.0008) [2023-10-08 07:23:09,932][00611] Updated weights for policy 0, policy_version 88592 (0.0011) [2023-10-08 07:23:10,297][00611] Updated weights for policy 0, policy_version 88602 (0.0012) [2023-10-08 07:23:11,702][00612] Updated weights for policy 1, policy_version 89090 (0.0008) [2023-10-08 07:23:12,068][00612] Updated weights for policy 1, policy_version 89100 (0.0007) [2023-10-08 07:23:12,423][00612] Updated weights for policy 1, policy_version 89110 (0.0008) [2023-10-08 07:23:12,791][00612] Updated weights for policy 1, policy_version 89120 (0.0009) [2023-10-08 07:23:13,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 181993472. Throughput: 0: 1842.0, 1: 1839.2. Samples: 45505906. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 07:23:13,754][130385] Avg episode reward: [(0, '74.980'), (1, '89.680')] [2023-10-08 07:23:13,821][00611] Updated weights for policy 0, policy_version 88612 (0.0009) [2023-10-08 07:23:14,191][00611] Updated weights for policy 0, policy_version 88622 (0.0009) [2023-10-08 07:23:14,567][00611] Updated weights for policy 0, policy_version 88632 (0.0009) [2023-10-08 07:23:16,379][00612] Updated weights for policy 1, policy_version 89130 (0.0008) [2023-10-08 07:23:16,747][00612] Updated weights for policy 1, policy_version 89140 (0.0008) [2023-10-08 07:23:17,122][00612] Updated weights for policy 1, policy_version 89150 (0.0007) [2023-10-08 07:23:18,153][00611] Updated weights for policy 0, policy_version 88642 (0.0007) [2023-10-08 07:23:18,526][00611] Updated weights for policy 0, policy_version 88652 (0.0009) [2023-10-08 07:23:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14773.3). Total num frames: 182059008. Throughput: 0: 1844.7, 1: 1856.4. Samples: 45528474. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 07:23:18,755][130385] Avg episode reward: [(0, '76.090'), (1, '89.690')] [2023-10-08 07:23:18,906][00611] Updated weights for policy 0, policy_version 88662 (0.0009) [2023-10-08 07:23:19,284][00611] Updated weights for policy 0, policy_version 88672 (0.0009) [2023-10-08 07:23:20,647][00612] Updated weights for policy 1, policy_version 89160 (0.0011) [2023-10-08 07:23:21,011][00612] Updated weights for policy 1, policy_version 89170 (0.0010) [2023-10-08 07:23:21,373][00612] Updated weights for policy 1, policy_version 89180 (0.0011) [2023-10-08 07:23:22,916][00611] Updated weights for policy 0, policy_version 88682 (0.0010) [2023-10-08 07:23:23,296][00611] Updated weights for policy 0, policy_version 88692 (0.0011) [2023-10-08 07:23:23,666][00611] Updated weights for policy 0, policy_version 88702 (0.0009) [2023-10-08 07:23:23,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 182157312. Throughput: 0: 1845.8, 1: 1830.0. Samples: 45538970. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 07:23:23,754][130385] Avg episode reward: [(0, '73.600'), (1, '91.670')] [2023-10-08 07:23:25,135][00612] Updated weights for policy 1, policy_version 89190 (0.0009) [2023-10-08 07:23:25,497][00612] Updated weights for policy 1, policy_version 89200 (0.0009) [2023-10-08 07:23:25,870][00612] Updated weights for policy 1, policy_version 89210 (0.0009) [2023-10-08 07:23:27,569][00611] Updated weights for policy 0, policy_version 88712 (0.0009) [2023-10-08 07:23:27,943][00611] Updated weights for policy 0, policy_version 88722 (0.0010) [2023-10-08 07:23:28,306][00611] Updated weights for policy 0, policy_version 88732 (0.0012) [2023-10-08 07:23:28,754][130385] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 182222848. Throughput: 0: 1827.9, 1: 1847.5. Samples: 45560824. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 07:23:28,754][130385] Avg episode reward: [(0, '76.130'), (1, '97.310')] [2023-10-08 07:23:28,755][00425] Saving new best policy, reward=97.310! [2023-10-08 07:23:29,813][00612] Updated weights for policy 1, policy_version 89220 (0.0009) [2023-10-08 07:23:30,169][00612] Updated weights for policy 1, policy_version 89230 (0.0007) [2023-10-08 07:23:30,531][00612] Updated weights for policy 1, policy_version 89240 (0.0007) [2023-10-08 07:23:32,192][00611] Updated weights for policy 0, policy_version 88742 (0.0012) [2023-10-08 07:23:32,564][00611] Updated weights for policy 0, policy_version 88752 (0.0010) [2023-10-08 07:23:32,932][00611] Updated weights for policy 0, policy_version 88762 (0.0010) [2023-10-08 07:23:33,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 182288384. Throughput: 0: 1816.2, 1: 1835.0. Samples: 45581302. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 07:23:33,754][130385] Avg episode reward: [(0, '69.750'), (1, '95.060')] [2023-10-08 07:23:34,325][00612] Updated weights for policy 1, policy_version 89250 (0.0008) [2023-10-08 07:23:34,692][00612] Updated weights for policy 1, policy_version 89260 (0.0009) [2023-10-08 07:23:35,059][00612] Updated weights for policy 1, policy_version 89270 (0.0011) [2023-10-08 07:23:35,426][00612] Updated weights for policy 1, policy_version 89280 (0.0009) [2023-10-08 07:23:36,775][00611] Updated weights for policy 0, policy_version 88772 (0.0012) [2023-10-08 07:23:37,154][00611] Updated weights for policy 0, policy_version 88782 (0.0010) [2023-10-08 07:23:37,525][00611] Updated weights for policy 0, policy_version 88792 (0.0008) [2023-10-08 07:23:38,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 182353920. Throughput: 0: 1804.3, 1: 1827.6. Samples: 45591884. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 07:23:38,754][130385] Avg episode reward: [(0, '69.680'), (1, '94.740')] [2023-10-08 07:23:39,514][00612] Updated weights for policy 1, policy_version 89290 (0.0010) [2023-10-08 07:23:39,886][00612] Updated weights for policy 1, policy_version 89300 (0.0009) [2023-10-08 07:23:40,249][00612] Updated weights for policy 1, policy_version 89310 (0.0010) [2023-10-08 07:23:41,661][00611] Updated weights for policy 0, policy_version 88802 (0.0010) [2023-10-08 07:23:42,030][00611] Updated weights for policy 0, policy_version 88812 (0.0009) [2023-10-08 07:23:42,408][00611] Updated weights for policy 0, policy_version 88822 (0.0008) [2023-10-08 07:23:42,773][00611] Updated weights for policy 0, policy_version 88832 (0.0009) [2023-10-08 07:23:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 182419456. Throughput: 0: 1800.6, 1: 1800.4. Samples: 45612358. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-08 07:23:43,754][130385] Avg episode reward: [(0, '72.690'), (1, '89.490')] [2023-10-08 07:23:44,304][00612] Updated weights for policy 1, policy_version 89320 (0.0011) [2023-10-08 07:23:44,656][00612] Updated weights for policy 1, policy_version 89330 (0.0009) [2023-10-08 07:23:45,021][00612] Updated weights for policy 1, policy_version 89340 (0.0009) [2023-10-08 07:23:46,853][00611] Updated weights for policy 0, policy_version 88842 (0.0009) [2023-10-08 07:23:47,230][00611] Updated weights for policy 0, policy_version 88852 (0.0007) [2023-10-08 07:23:47,601][00611] Updated weights for policy 0, policy_version 88862 (0.0010) [2023-10-08 07:23:48,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 182484992. Throughput: 0: 1757.9, 1: 1787.2. Samples: 45632278. Policy #0 lag: (min: 18.0, avg: 21.4, max: 47.0) [2023-10-08 07:23:48,754][130385] Avg episode reward: [(0, '67.770'), (1, '88.870')] [2023-10-08 07:23:48,954][00612] Updated weights for policy 1, policy_version 89350 (0.0010) [2023-10-08 07:23:49,324][00612] Updated weights for policy 1, policy_version 89360 (0.0014) [2023-10-08 07:23:49,688][00612] Updated weights for policy 1, policy_version 89370 (0.0009) [2023-10-08 07:23:51,541][00611] Updated weights for policy 0, policy_version 88872 (0.0008) [2023-10-08 07:23:51,912][00611] Updated weights for policy 0, policy_version 88882 (0.0008) [2023-10-08 07:23:52,282][00611] Updated weights for policy 0, policy_version 88892 (0.0008) [2023-10-08 07:23:53,701][00612] Updated weights for policy 1, policy_version 89380 (0.0009) [2023-10-08 07:23:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 182550528. Throughput: 0: 1784.4, 1: 1767.6. Samples: 45643078. Policy #0 lag: (min: 18.0, avg: 21.4, max: 47.0) [2023-10-08 07:23:53,755][130385] Avg episode reward: [(0, '65.650'), (1, '87.690')] [2023-10-08 07:23:54,065][00612] Updated weights for policy 1, policy_version 89390 (0.0009) [2023-10-08 07:23:54,429][00612] Updated weights for policy 1, policy_version 89400 (0.0009) [2023-10-08 07:23:56,086][00611] Updated weights for policy 0, policy_version 88902 (0.0010) [2023-10-08 07:23:56,449][00611] Updated weights for policy 0, policy_version 88912 (0.0011) [2023-10-08 07:23:56,824][00611] Updated weights for policy 0, policy_version 88922 (0.0010) [2023-10-08 07:23:58,427][00612] Updated weights for policy 1, policy_version 89410 (0.0010) [2023-10-08 07:23:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 182616064. Throughput: 0: 1734.3, 1: 1765.0. Samples: 45663374. Policy #0 lag: (min: 18.0, avg: 21.4, max: 47.0) [2023-10-08 07:23:58,754][130385] Avg episode reward: [(0, '67.060'), (1, '92.450')] [2023-10-08 07:23:58,801][00612] Updated weights for policy 1, policy_version 89420 (0.0010) [2023-10-08 07:23:59,166][00612] Updated weights for policy 1, policy_version 89430 (0.0008) [2023-10-08 07:23:59,532][00612] Updated weights for policy 1, policy_version 89440 (0.0009) [2023-10-08 07:24:00,747][00611] Updated weights for policy 0, policy_version 88932 (0.0011) [2023-10-08 07:24:01,116][00611] Updated weights for policy 0, policy_version 88942 (0.0010) [2023-10-08 07:24:01,490][00611] Updated weights for policy 0, policy_version 88952 (0.0009) [2023-10-08 07:24:03,475][00612] Updated weights for policy 1, policy_version 89450 (0.0009) [2023-10-08 07:24:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182681600. Throughput: 0: 1707.8, 1: 1751.1. Samples: 45684122. Policy #0 lag: (min: 18.0, avg: 21.4, max: 47.0) [2023-10-08 07:24:03,756][130385] Avg episode reward: [(0, '64.280'), (1, '92.150')] [2023-10-08 07:24:03,842][00612] Updated weights for policy 1, policy_version 89460 (0.0009) [2023-10-08 07:24:04,216][00612] Updated weights for policy 1, policy_version 89470 (0.0010) [2023-10-08 07:24:05,291][00611] Updated weights for policy 0, policy_version 88962 (0.0008) [2023-10-08 07:24:05,660][00611] Updated weights for policy 0, policy_version 88972 (0.0008) [2023-10-08 07:24:06,030][00611] Updated weights for policy 0, policy_version 88982 (0.0010) [2023-10-08 07:24:06,410][00611] Updated weights for policy 0, policy_version 88992 (0.0011) [2023-10-08 07:24:07,815][00612] Updated weights for policy 1, policy_version 89480 (0.0008) [2023-10-08 07:24:08,185][00612] Updated weights for policy 1, policy_version 89490 (0.0009) [2023-10-08 07:24:08,555][00612] Updated weights for policy 1, policy_version 89500 (0.0012) [2023-10-08 07:24:08,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182779904. Throughput: 0: 1718.1, 1: 1741.5. Samples: 45694652. Policy #0 lag: (min: 18.0, avg: 21.4, max: 47.0) [2023-10-08 07:24:08,754][130385] Avg episode reward: [(0, '62.760'), (1, '94.330')] [2023-10-08 07:24:10,028][00611] Updated weights for policy 0, policy_version 89002 (0.0008) [2023-10-08 07:24:10,406][00611] Updated weights for policy 0, policy_version 89012 (0.0008) [2023-10-08 07:24:10,764][00611] Updated weights for policy 0, policy_version 89022 (0.0008) [2023-10-08 07:24:12,229][00612] Updated weights for policy 1, policy_version 89510 (0.0009) [2023-10-08 07:24:12,594][00612] Updated weights for policy 1, policy_version 89520 (0.0009) [2023-10-08 07:24:12,960][00612] Updated weights for policy 1, policy_version 89530 (0.0009) [2023-10-08 07:24:13,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182845440. Throughput: 0: 1723.1, 1: 1757.6. Samples: 45717452. Policy #0 lag: (min: 18.0, avg: 21.4, max: 47.0) [2023-10-08 07:24:13,754][130385] Avg episode reward: [(0, '64.380'), (1, '97.870')] [2023-10-08 07:24:13,755][00425] Saving new best policy, reward=97.870! [2023-10-08 07:24:14,490][00611] Updated weights for policy 0, policy_version 89032 (0.0010) [2023-10-08 07:24:14,857][00611] Updated weights for policy 0, policy_version 89042 (0.0007) [2023-10-08 07:24:15,221][00611] Updated weights for policy 0, policy_version 89052 (0.0007) [2023-10-08 07:24:16,501][00612] Updated weights for policy 1, policy_version 89540 (0.0007) [2023-10-08 07:24:16,872][00612] Updated weights for policy 1, policy_version 89550 (0.0008) [2023-10-08 07:24:17,238][00612] Updated weights for policy 1, policy_version 89560 (0.0008) [2023-10-08 07:24:18,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182910976. Throughput: 0: 1772.0, 1: 1743.2. Samples: 45739488. Policy #0 lag: (min: 18.0, avg: 21.4, max: 47.0) [2023-10-08 07:24:18,755][130385] Avg episode reward: [(0, '64.190'), (1, '98.400')] [2023-10-08 07:24:18,768][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000089568_91717632.pth... [2023-10-08 07:24:18,796][00611] Updated weights for policy 0, policy_version 89062 (0.0009) [2023-10-08 07:24:18,802][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000087872_89980928.pth [2023-10-08 07:24:18,805][00425] Saving new best policy, reward=98.400! [2023-10-08 07:24:19,170][00611] Updated weights for policy 0, policy_version 89072 (0.0010) [2023-10-08 07:24:19,553][00611] Updated weights for policy 0, policy_version 89082 (0.0009) [2023-10-08 07:24:19,776][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000089088_91226112.pth... [2023-10-08 07:24:19,806][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000087360_89456640.pth [2023-10-08 07:24:20,841][00612] Updated weights for policy 1, policy_version 89570 (0.0008) [2023-10-08 07:24:21,203][00612] Updated weights for policy 1, policy_version 89580 (0.0008) [2023-10-08 07:24:21,572][00612] Updated weights for policy 1, policy_version 89590 (0.0008) [2023-10-08 07:24:21,941][00612] Updated weights for policy 1, policy_version 89600 (0.0008) [2023-10-08 07:24:23,165][00611] Updated weights for policy 0, policy_version 89092 (0.0009) [2023-10-08 07:24:23,534][00611] Updated weights for policy 0, policy_version 89102 (0.0007) [2023-10-08 07:24:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 14551.2). Total num frames: 182976512. Throughput: 0: 1750.5, 1: 1776.7. Samples: 45750608. Policy #0 lag: (min: 18.0, avg: 21.4, max: 47.0) [2023-10-08 07:24:23,754][130385] Avg episode reward: [(0, '67.170'), (1, '97.920')] [2023-10-08 07:24:23,908][00611] Updated weights for policy 0, policy_version 89112 (0.0007) [2023-10-08 07:24:25,575][00612] Updated weights for policy 1, policy_version 89610 (0.0008) [2023-10-08 07:24:25,941][00612] Updated weights for policy 1, policy_version 89620 (0.0007) [2023-10-08 07:24:26,305][00612] Updated weights for policy 1, policy_version 89630 (0.0007) [2023-10-08 07:24:27,539][00611] Updated weights for policy 0, policy_version 89122 (0.0009) [2023-10-08 07:24:27,914][00611] Updated weights for policy 0, policy_version 89132 (0.0007) [2023-10-08 07:24:28,275][00611] Updated weights for policy 0, policy_version 89142 (0.0008) [2023-10-08 07:24:28,645][00611] Updated weights for policy 0, policy_version 89152 (0.0010) [2023-10-08 07:24:28,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 183074816. Throughput: 0: 1793.9, 1: 1775.5. Samples: 45772986. Policy #0 lag: (min: 18.0, avg: 21.4, max: 47.0) [2023-10-08 07:24:28,755][130385] Avg episode reward: [(0, '66.690'), (1, '95.100')] [2023-10-08 07:24:29,826][00612] Updated weights for policy 1, policy_version 89640 (0.0007) [2023-10-08 07:24:30,188][00612] Updated weights for policy 1, policy_version 89650 (0.0009) [2023-10-08 07:24:30,562][00612] Updated weights for policy 1, policy_version 89660 (0.0007) [2023-10-08 07:24:32,281][00611] Updated weights for policy 0, policy_version 89162 (0.0007) [2023-10-08 07:24:32,651][00611] Updated weights for policy 0, policy_version 89172 (0.0008) [2023-10-08 07:24:33,030][00611] Updated weights for policy 0, policy_version 89182 (0.0008) [2023-10-08 07:24:33,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 183140352. Throughput: 0: 1803.3, 1: 1811.6. Samples: 45794952. Policy #0 lag: (min: 18.0, avg: 21.4, max: 47.0) [2023-10-08 07:24:33,754][130385] Avg episode reward: [(0, '68.550'), (1, '95.880')] [2023-10-08 07:24:34,077][00612] Updated weights for policy 1, policy_version 89670 (0.0009) [2023-10-08 07:24:34,443][00612] Updated weights for policy 1, policy_version 89680 (0.0007) [2023-10-08 07:24:34,816][00612] Updated weights for policy 1, policy_version 89690 (0.0007) [2023-10-08 07:24:36,607][00611] Updated weights for policy 0, policy_version 89192 (0.0008) [2023-10-08 07:24:36,980][00611] Updated weights for policy 0, policy_version 89202 (0.0008) [2023-10-08 07:24:37,347][00611] Updated weights for policy 0, policy_version 89212 (0.0007) [2023-10-08 07:24:38,390][00612] Updated weights for policy 1, policy_version 89700 (0.0010) [2023-10-08 07:24:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 183205888. Throughput: 0: 1818.8, 1: 1817.4. Samples: 45806706. Policy #0 lag: (min: 18.0, avg: 21.4, max: 47.0) [2023-10-08 07:24:38,755][130385] Avg episode reward: [(0, '69.350'), (1, '97.570')] [2023-10-08 07:24:38,785][00612] Updated weights for policy 1, policy_version 89710 (0.0008) [2023-10-08 07:24:39,147][00612] Updated weights for policy 1, policy_version 89720 (0.0011) [2023-10-08 07:24:40,832][00611] Updated weights for policy 0, policy_version 89222 (0.0008) [2023-10-08 07:24:41,206][00611] Updated weights for policy 0, policy_version 89232 (0.0009) [2023-10-08 07:24:41,580][00611] Updated weights for policy 0, policy_version 89242 (0.0008) [2023-10-08 07:24:42,841][00612] Updated weights for policy 1, policy_version 89730 (0.0009) [2023-10-08 07:24:43,207][00612] Updated weights for policy 1, policy_version 89740 (0.0007) [2023-10-08 07:24:43,575][00612] Updated weights for policy 1, policy_version 89750 (0.0009) [2023-10-08 07:24:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 183271424. Throughput: 0: 1830.7, 1: 1832.2. Samples: 45828204. Policy #0 lag: (min: 18.0, avg: 21.4, max: 47.0) [2023-10-08 07:24:43,754][130385] Avg episode reward: [(0, '75.790'), (1, '91.570')] [2023-10-08 07:24:43,941][00612] Updated weights for policy 1, policy_version 89760 (0.0007) [2023-10-08 07:24:45,091][00611] Updated weights for policy 0, policy_version 89252 (0.0009) [2023-10-08 07:24:45,466][00611] Updated weights for policy 0, policy_version 89262 (0.0008) [2023-10-08 07:24:45,836][00611] Updated weights for policy 0, policy_version 89272 (0.0011) [2023-10-08 07:24:47,659][00612] Updated weights for policy 1, policy_version 89770 (0.0009) [2023-10-08 07:24:48,036][00612] Updated weights for policy 1, policy_version 89780 (0.0008) [2023-10-08 07:24:48,398][00612] Updated weights for policy 1, policy_version 89790 (0.0009) [2023-10-08 07:24:48,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 183369728. Throughput: 0: 1857.0, 1: 1837.9. Samples: 45850392. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 07:24:48,755][130385] Avg episode reward: [(0, '76.340'), (1, '85.650')] [2023-10-08 07:24:49,490][00611] Updated weights for policy 0, policy_version 89282 (0.0010) [2023-10-08 07:24:49,870][00611] Updated weights for policy 0, policy_version 89292 (0.0007) [2023-10-08 07:24:50,238][00611] Updated weights for policy 0, policy_version 89302 (0.0007) [2023-10-08 07:24:50,612][00611] Updated weights for policy 0, policy_version 89312 (0.0008) [2023-10-08 07:24:51,936][00612] Updated weights for policy 1, policy_version 89800 (0.0011) [2023-10-08 07:24:52,310][00612] Updated weights for policy 1, policy_version 89810 (0.0009) [2023-10-08 07:24:52,676][00612] Updated weights for policy 1, policy_version 89820 (0.0007) [2023-10-08 07:24:53,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 183435264. Throughput: 0: 1844.0, 1: 1867.3. Samples: 45861662. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 07:24:53,754][130385] Avg episode reward: [(0, '76.460'), (1, '80.990')] [2023-10-08 07:24:54,322][00611] Updated weights for policy 0, policy_version 89322 (0.0009) [2023-10-08 07:24:54,686][00611] Updated weights for policy 0, policy_version 89332 (0.0008) [2023-10-08 07:24:55,059][00611] Updated weights for policy 0, policy_version 89342 (0.0007) [2023-10-08 07:24:56,267][00612] Updated weights for policy 1, policy_version 89830 (0.0008) [2023-10-08 07:24:56,635][00612] Updated weights for policy 1, policy_version 89840 (0.0008) [2023-10-08 07:24:56,998][00612] Updated weights for policy 1, policy_version 89850 (0.0007) [2023-10-08 07:24:58,633][00611] Updated weights for policy 0, policy_version 89352 (0.0008) [2023-10-08 07:24:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 183500800. Throughput: 0: 1850.9, 1: 1837.0. Samples: 45883410. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 07:24:58,754][130385] Avg episode reward: [(0, '73.100'), (1, '81.390')] [2023-10-08 07:24:59,005][00611] Updated weights for policy 0, policy_version 89362 (0.0007) [2023-10-08 07:24:59,376][00611] Updated weights for policy 0, policy_version 89372 (0.0007) [2023-10-08 07:25:00,662][00612] Updated weights for policy 1, policy_version 89860 (0.0007) [2023-10-08 07:25:01,035][00612] Updated weights for policy 1, policy_version 89870 (0.0007) [2023-10-08 07:25:01,397][00612] Updated weights for policy 1, policy_version 89880 (0.0010) [2023-10-08 07:25:03,016][00611] Updated weights for policy 0, policy_version 89382 (0.0010) [2023-10-08 07:25:03,396][00611] Updated weights for policy 0, policy_version 89392 (0.0010) [2023-10-08 07:25:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 183566336. Throughput: 0: 1843.0, 1: 1861.1. Samples: 45906172. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 07:25:03,754][130385] Avg episode reward: [(0, '71.450'), (1, '82.090')] [2023-10-08 07:25:03,760][00611] Updated weights for policy 0, policy_version 89402 (0.0009) [2023-10-08 07:25:05,111][00612] Updated weights for policy 1, policy_version 89890 (0.0008) [2023-10-08 07:25:05,476][00612] Updated weights for policy 1, policy_version 89900 (0.0008) [2023-10-08 07:25:05,846][00612] Updated weights for policy 1, policy_version 89910 (0.0007) [2023-10-08 07:25:06,207][00612] Updated weights for policy 1, policy_version 89920 (0.0007) [2023-10-08 07:25:07,217][00611] Updated weights for policy 0, policy_version 89412 (0.0009) [2023-10-08 07:25:07,579][00611] Updated weights for policy 0, policy_version 89422 (0.0009) [2023-10-08 07:25:07,950][00611] Updated weights for policy 0, policy_version 89432 (0.0007) [2023-10-08 07:25:08,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 183664640. Throughput: 0: 1854.6, 1: 1838.9. Samples: 45916814. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 07:25:08,755][130385] Avg episode reward: [(0, '74.810'), (1, '86.980')] [2023-10-08 07:25:09,735][00612] Updated weights for policy 1, policy_version 89930 (0.0008) [2023-10-08 07:25:10,110][00612] Updated weights for policy 1, policy_version 89940 (0.0008) [2023-10-08 07:25:10,474][00612] Updated weights for policy 1, policy_version 89950 (0.0008) [2023-10-08 07:25:11,542][00611] Updated weights for policy 0, policy_version 89442 (0.0010) [2023-10-08 07:25:11,918][00611] Updated weights for policy 0, policy_version 89452 (0.0009) [2023-10-08 07:25:12,296][00611] Updated weights for policy 0, policy_version 89462 (0.0008) [2023-10-08 07:25:12,668][00611] Updated weights for policy 0, policy_version 89472 (0.0009) [2023-10-08 07:25:13,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 183730176. Throughput: 0: 1841.7, 1: 1859.2. Samples: 45939530. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 07:25:13,755][130385] Avg episode reward: [(0, '76.590'), (1, '88.050')] [2023-10-08 07:25:14,337][00612] Updated weights for policy 1, policy_version 89960 (0.0008) [2023-10-08 07:25:14,702][00612] Updated weights for policy 1, policy_version 89970 (0.0007) [2023-10-08 07:25:15,076][00612] Updated weights for policy 1, policy_version 89980 (0.0007) [2023-10-08 07:25:16,476][00611] Updated weights for policy 0, policy_version 89482 (0.0007) [2023-10-08 07:25:16,855][00611] Updated weights for policy 0, policy_version 89492 (0.0008) [2023-10-08 07:25:17,219][00611] Updated weights for policy 0, policy_version 89502 (0.0007) [2023-10-08 07:25:18,674][00612] Updated weights for policy 1, policy_version 89990 (0.0008) [2023-10-08 07:25:18,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 183795712. Throughput: 0: 1857.3, 1: 1848.0. Samples: 45961690. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 07:25:18,754][130385] Avg episode reward: [(0, '75.870'), (1, '89.170')] [2023-10-08 07:25:19,033][00612] Updated weights for policy 1, policy_version 90000 (0.0008) [2023-10-08 07:25:19,403][00612] Updated weights for policy 1, policy_version 90010 (0.0007) [2023-10-08 07:25:20,747][00611] Updated weights for policy 0, policy_version 89512 (0.0008) [2023-10-08 07:25:21,124][00611] Updated weights for policy 0, policy_version 89522 (0.0010) [2023-10-08 07:25:21,499][00611] Updated weights for policy 0, policy_version 89532 (0.0009) [2023-10-08 07:25:23,020][00612] Updated weights for policy 1, policy_version 90020 (0.0007) [2023-10-08 07:25:23,392][00612] Updated weights for policy 1, policy_version 90030 (0.0010) [2023-10-08 07:25:23,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 183861248. Throughput: 0: 1834.6, 1: 1852.5. Samples: 45972628. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 07:25:23,754][130385] Avg episode reward: [(0, '70.160'), (1, '94.130')] [2023-10-08 07:25:23,769][00612] Updated weights for policy 1, policy_version 90040 (0.0010) [2023-10-08 07:25:25,185][00611] Updated weights for policy 0, policy_version 89542 (0.0008) [2023-10-08 07:25:25,558][00611] Updated weights for policy 0, policy_version 89552 (0.0007) [2023-10-08 07:25:25,924][00611] Updated weights for policy 0, policy_version 89562 (0.0008) [2023-10-08 07:25:27,308][00612] Updated weights for policy 1, policy_version 90050 (0.0007) [2023-10-08 07:25:27,694][00612] Updated weights for policy 1, policy_version 90060 (0.0007) [2023-10-08 07:25:28,057][00612] Updated weights for policy 1, policy_version 90070 (0.0009) [2023-10-08 07:25:28,427][00612] Updated weights for policy 1, policy_version 90080 (0.0010) [2023-10-08 07:25:28,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 183959552. Throughput: 0: 1857.0, 1: 1859.5. Samples: 45995450. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 07:25:28,755][130385] Avg episode reward: [(0, '70.100'), (1, '88.010')] [2023-10-08 07:25:29,649][00611] Updated weights for policy 0, policy_version 89572 (0.0010) [2023-10-08 07:25:30,040][00611] Updated weights for policy 0, policy_version 89582 (0.0010) [2023-10-08 07:25:30,403][00611] Updated weights for policy 0, policy_version 89592 (0.0009) [2023-10-08 07:25:31,869][00612] Updated weights for policy 1, policy_version 90090 (0.0008) [2023-10-08 07:25:32,230][00612] Updated weights for policy 1, policy_version 90100 (0.0007) [2023-10-08 07:25:32,595][00612] Updated weights for policy 1, policy_version 90110 (0.0008) [2023-10-08 07:25:33,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184025088. Throughput: 0: 1848.4, 1: 1849.4. Samples: 46016792. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 07:25:33,754][130385] Avg episode reward: [(0, '66.950'), (1, '85.060')] [2023-10-08 07:25:34,084][00611] Updated weights for policy 0, policy_version 89602 (0.0010) [2023-10-08 07:25:34,449][00611] Updated weights for policy 0, policy_version 89612 (0.0008) [2023-10-08 07:25:34,820][00611] Updated weights for policy 0, policy_version 89622 (0.0009) [2023-10-08 07:25:35,192][00611] Updated weights for policy 0, policy_version 89632 (0.0009) [2023-10-08 07:25:36,150][00612] Updated weights for policy 1, policy_version 90120 (0.0008) [2023-10-08 07:25:36,513][00612] Updated weights for policy 1, policy_version 90130 (0.0009) [2023-10-08 07:25:36,881][00612] Updated weights for policy 1, policy_version 90140 (0.0009) [2023-10-08 07:25:38,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184090624. Throughput: 0: 1851.5, 1: 1846.0. Samples: 46028050. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 07:25:38,755][130385] Avg episode reward: [(0, '62.330'), (1, '84.360')] [2023-10-08 07:25:38,780][00611] Updated weights for policy 0, policy_version 89642 (0.0007) [2023-10-08 07:25:39,161][00611] Updated weights for policy 0, policy_version 89652 (0.0009) [2023-10-08 07:25:39,524][00611] Updated weights for policy 0, policy_version 89662 (0.0008) [2023-10-08 07:25:40,605][00612] Updated weights for policy 1, policy_version 90150 (0.0008) [2023-10-08 07:25:40,979][00612] Updated weights for policy 1, policy_version 90160 (0.0008) [2023-10-08 07:25:41,346][00612] Updated weights for policy 1, policy_version 90170 (0.0009) [2023-10-08 07:25:43,081][00611] Updated weights for policy 0, policy_version 89672 (0.0009) [2023-10-08 07:25:43,456][00611] Updated weights for policy 0, policy_version 89682 (0.0007) [2023-10-08 07:25:43,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184156160. Throughput: 0: 1850.1, 1: 1852.7. Samples: 46050034. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 07:25:43,755][130385] Avg episode reward: [(0, '65.170'), (1, '88.090')] [2023-10-08 07:25:43,821][00611] Updated weights for policy 0, policy_version 89692 (0.0007) [2023-10-08 07:25:44,969][00612] Updated weights for policy 1, policy_version 90180 (0.0010) [2023-10-08 07:25:45,344][00612] Updated weights for policy 1, policy_version 90190 (0.0011) [2023-10-08 07:25:45,702][00612] Updated weights for policy 1, policy_version 90200 (0.0010) [2023-10-08 07:25:47,446][00611] Updated weights for policy 0, policy_version 89702 (0.0010) [2023-10-08 07:25:47,814][00611] Updated weights for policy 0, policy_version 89712 (0.0009) [2023-10-08 07:25:48,190][00611] Updated weights for policy 0, policy_version 89722 (0.0009) [2023-10-08 07:25:48,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184254464. Throughput: 0: 1834.9, 1: 1856.7. Samples: 46072294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:25:48,754][130385] Avg episode reward: [(0, '63.520'), (1, '84.110')] [2023-10-08 07:25:49,390][00612] Updated weights for policy 1, policy_version 90210 (0.0007) [2023-10-08 07:25:49,756][00612] Updated weights for policy 1, policy_version 90220 (0.0008) [2023-10-08 07:25:50,133][00612] Updated weights for policy 1, policy_version 90230 (0.0007) [2023-10-08 07:25:50,496][00612] Updated weights for policy 1, policy_version 90240 (0.0007) [2023-10-08 07:25:51,734][00611] Updated weights for policy 0, policy_version 89732 (0.0009) [2023-10-08 07:25:52,101][00611] Updated weights for policy 0, policy_version 89742 (0.0009) [2023-10-08 07:25:52,473][00611] Updated weights for policy 0, policy_version 89752 (0.0009) [2023-10-08 07:25:53,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184320000. Throughput: 0: 1852.9, 1: 1851.5. Samples: 46083508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:25:53,754][130385] Avg episode reward: [(0, '63.650'), (1, '84.000')] [2023-10-08 07:25:54,163][00612] Updated weights for policy 1, policy_version 90250 (0.0007) [2023-10-08 07:25:54,526][00612] Updated weights for policy 1, policy_version 90260 (0.0007) [2023-10-08 07:25:54,897][00612] Updated weights for policy 1, policy_version 90270 (0.0008) [2023-10-08 07:25:56,092][00611] Updated weights for policy 0, policy_version 89762 (0.0008) [2023-10-08 07:25:56,461][00611] Updated weights for policy 0, policy_version 89772 (0.0007) [2023-10-08 07:25:56,831][00611] Updated weights for policy 0, policy_version 89782 (0.0008) [2023-10-08 07:25:57,214][00611] Updated weights for policy 0, policy_version 89792 (0.0011) [2023-10-08 07:25:58,579][00612] Updated weights for policy 1, policy_version 90280 (0.0010) [2023-10-08 07:25:58,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184385536. Throughput: 0: 1834.6, 1: 1853.4. Samples: 46105490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:25:58,755][130385] Avg episode reward: [(0, '67.640'), (1, '85.650')] [2023-10-08 07:25:58,952][00612] Updated weights for policy 1, policy_version 90290 (0.0009) [2023-10-08 07:25:59,321][00612] Updated weights for policy 1, policy_version 90300 (0.0007) [2023-10-08 07:26:00,995][00611] Updated weights for policy 0, policy_version 89802 (0.0007) [2023-10-08 07:26:01,363][00611] Updated weights for policy 0, policy_version 89812 (0.0007) [2023-10-08 07:26:01,735][00611] Updated weights for policy 0, policy_version 89822 (0.0007) [2023-10-08 07:26:03,109][00612] Updated weights for policy 1, policy_version 90310 (0.0008) [2023-10-08 07:26:03,475][00612] Updated weights for policy 1, policy_version 90320 (0.0008) [2023-10-08 07:26:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184451072. Throughput: 0: 1848.9, 1: 1843.4. Samples: 46127842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:26:03,754][130385] Avg episode reward: [(0, '69.920'), (1, '86.090')] [2023-10-08 07:26:03,839][00612] Updated weights for policy 1, policy_version 90330 (0.0009) [2023-10-08 07:26:05,279][00611] Updated weights for policy 0, policy_version 89832 (0.0010) [2023-10-08 07:26:05,654][00611] Updated weights for policy 0, policy_version 89842 (0.0010) [2023-10-08 07:26:06,018][00611] Updated weights for policy 0, policy_version 89852 (0.0009) [2023-10-08 07:26:07,445][00612] Updated weights for policy 1, policy_version 90340 (0.0008) [2023-10-08 07:26:07,820][00612] Updated weights for policy 1, policy_version 90350 (0.0008) [2023-10-08 07:26:08,192][00612] Updated weights for policy 1, policy_version 90360 (0.0007) [2023-10-08 07:26:08,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184549376. Throughput: 0: 1835.6, 1: 1849.9. Samples: 46138474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:26:08,755][130385] Avg episode reward: [(0, '70.250'), (1, '87.260')] [2023-10-08 07:26:09,595][00611] Updated weights for policy 0, policy_version 89862 (0.0008) [2023-10-08 07:26:09,974][00611] Updated weights for policy 0, policy_version 89872 (0.0009) [2023-10-08 07:26:10,342][00611] Updated weights for policy 0, policy_version 89882 (0.0009) [2023-10-08 07:26:11,891][00612] Updated weights for policy 1, policy_version 90370 (0.0007) [2023-10-08 07:26:12,265][00612] Updated weights for policy 1, policy_version 90380 (0.0009) [2023-10-08 07:26:12,637][00612] Updated weights for policy 1, policy_version 90390 (0.0009) [2023-10-08 07:26:13,014][00612] Updated weights for policy 1, policy_version 90400 (0.0008) [2023-10-08 07:26:13,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 184614912. Throughput: 0: 1846.8, 1: 1831.3. Samples: 46160962. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:26:13,754][130385] Avg episode reward: [(0, '68.620'), (1, '89.390')] [2023-10-08 07:26:14,018][00611] Updated weights for policy 0, policy_version 89892 (0.0008) [2023-10-08 07:26:14,417][00611] Updated weights for policy 0, policy_version 89902 (0.0007) [2023-10-08 07:26:14,790][00611] Updated weights for policy 0, policy_version 89912 (0.0009) [2023-10-08 07:26:16,780][00612] Updated weights for policy 1, policy_version 90410 (0.0009) [2023-10-08 07:26:17,160][00612] Updated weights for policy 1, policy_version 90420 (0.0009) [2023-10-08 07:26:17,526][00612] Updated weights for policy 1, policy_version 90430 (0.0007) [2023-10-08 07:26:18,521][00611] Updated weights for policy 0, policy_version 89922 (0.0009) [2023-10-08 07:26:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 184680448. Throughput: 0: 1848.9, 1: 1840.2. Samples: 46182800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:26:18,755][130385] Avg episode reward: [(0, '68.930'), (1, '82.940')] [2023-10-08 07:26:18,766][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000090432_92602368.pth... [2023-10-08 07:26:18,801][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000088736_90865664.pth [2023-10-08 07:26:18,897][00611] Updated weights for policy 0, policy_version 89932 (0.0008) [2023-10-08 07:26:19,265][00611] Updated weights for policy 0, policy_version 89942 (0.0009) [2023-10-08 07:26:19,642][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000089952_92110848.pth... [2023-10-08 07:26:19,645][00611] Updated weights for policy 0, policy_version 89952 (0.0009) [2023-10-08 07:26:19,671][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000088224_90341376.pth [2023-10-08 07:26:20,978][00612] Updated weights for policy 1, policy_version 90440 (0.0007) [2023-10-08 07:26:21,342][00612] Updated weights for policy 1, policy_version 90450 (0.0010) [2023-10-08 07:26:21,706][00612] Updated weights for policy 1, policy_version 90460 (0.0009) [2023-10-08 07:26:23,186][00611] Updated weights for policy 0, policy_version 89962 (0.0008) [2023-10-08 07:26:23,567][00611] Updated weights for policy 0, policy_version 89972 (0.0009) [2023-10-08 07:26:23,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184745984. Throughput: 0: 1844.9, 1: 1834.7. Samples: 46193632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:26:23,754][130385] Avg episode reward: [(0, '69.150'), (1, '82.300')] [2023-10-08 07:26:23,938][00611] Updated weights for policy 0, policy_version 89982 (0.0008) [2023-10-08 07:26:25,154][00612] Updated weights for policy 1, policy_version 90470 (0.0009) [2023-10-08 07:26:25,520][00612] Updated weights for policy 1, policy_version 90480 (0.0009) [2023-10-08 07:26:25,891][00612] Updated weights for policy 1, policy_version 90490 (0.0009) [2023-10-08 07:26:27,708][00611] Updated weights for policy 0, policy_version 89992 (0.0008) [2023-10-08 07:26:28,075][00611] Updated weights for policy 0, policy_version 90002 (0.0008) [2023-10-08 07:26:28,450][00611] Updated weights for policy 0, policy_version 90012 (0.0011) [2023-10-08 07:26:28,754][130385] Fps is (10 sec: 16384.6, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 184844288. Throughput: 0: 1844.1, 1: 1845.6. Samples: 46216074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:26:28,754][130385] Avg episode reward: [(0, '75.450'), (1, '79.930')] [2023-10-08 07:26:29,528][00612] Updated weights for policy 1, policy_version 90500 (0.0009) [2023-10-08 07:26:29,890][00612] Updated weights for policy 1, policy_version 90510 (0.0007) [2023-10-08 07:26:30,256][00612] Updated weights for policy 1, policy_version 90520 (0.0011) [2023-10-08 07:26:32,057][00611] Updated weights for policy 0, policy_version 90022 (0.0011) [2023-10-08 07:26:32,433][00611] Updated weights for policy 0, policy_version 90032 (0.0011) [2023-10-08 07:26:32,803][00611] Updated weights for policy 0, policy_version 90042 (0.0010) [2023-10-08 07:26:33,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184909824. Throughput: 0: 1824.4, 1: 1846.5. Samples: 46237488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:26:33,754][130385] Avg episode reward: [(0, '74.090'), (1, '79.360')] [2023-10-08 07:26:33,823][00612] Updated weights for policy 1, policy_version 90530 (0.0008) [2023-10-08 07:26:34,190][00612] Updated weights for policy 1, policy_version 90540 (0.0012) [2023-10-08 07:26:34,565][00612] Updated weights for policy 1, policy_version 90550 (0.0008) [2023-10-08 07:26:34,929][00612] Updated weights for policy 1, policy_version 90560 (0.0008) [2023-10-08 07:26:36,374][00611] Updated weights for policy 0, policy_version 90052 (0.0009) [2023-10-08 07:26:36,740][00611] Updated weights for policy 0, policy_version 90062 (0.0007) [2023-10-08 07:26:37,119][00611] Updated weights for policy 0, policy_version 90072 (0.0008) [2023-10-08 07:26:38,534][00612] Updated weights for policy 1, policy_version 90570 (0.0008) [2023-10-08 07:26:38,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184975360. Throughput: 0: 1832.2, 1: 1848.0. Samples: 46249118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:26:38,755][130385] Avg episode reward: [(0, '72.080'), (1, '80.970')] [2023-10-08 07:26:38,904][00612] Updated weights for policy 1, policy_version 90580 (0.0008) [2023-10-08 07:26:39,279][00612] Updated weights for policy 1, policy_version 90590 (0.0009) [2023-10-08 07:26:40,896][00611] Updated weights for policy 0, policy_version 90082 (0.0009) [2023-10-08 07:26:41,258][00611] Updated weights for policy 0, policy_version 90092 (0.0009) [2023-10-08 07:26:41,637][00611] Updated weights for policy 0, policy_version 90102 (0.0009) [2023-10-08 07:26:42,014][00611] Updated weights for policy 0, policy_version 90112 (0.0007) [2023-10-08 07:26:42,961][00612] Updated weights for policy 1, policy_version 90600 (0.0008) [2023-10-08 07:26:43,328][00612] Updated weights for policy 1, policy_version 90610 (0.0008) [2023-10-08 07:26:43,702][00612] Updated weights for policy 1, policy_version 90620 (0.0007) [2023-10-08 07:26:43,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185040896. Throughput: 0: 1824.0, 1: 1851.4. Samples: 46270880. Policy #0 lag: (min: 30.0, avg: 30.1, max: 36.0) [2023-10-08 07:26:43,754][130385] Avg episode reward: [(0, '72.160'), (1, '81.420')] [2023-10-08 07:26:45,814][00611] Updated weights for policy 0, policy_version 90122 (0.0007) [2023-10-08 07:26:46,183][00611] Updated weights for policy 0, policy_version 90132 (0.0008) [2023-10-08 07:26:46,557][00611] Updated weights for policy 0, policy_version 90142 (0.0007) [2023-10-08 07:26:47,324][00612] Updated weights for policy 1, policy_version 90630 (0.0010) [2023-10-08 07:26:47,690][00612] Updated weights for policy 1, policy_version 90640 (0.0009) [2023-10-08 07:26:48,055][00612] Updated weights for policy 1, policy_version 90650 (0.0012) [2023-10-08 07:26:48,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185139200. Throughput: 0: 1827.2, 1: 1833.0. Samples: 46292548. Policy #0 lag: (min: 30.0, avg: 30.1, max: 36.0) [2023-10-08 07:26:48,755][130385] Avg episode reward: [(0, '74.160'), (1, '80.810')] [2023-10-08 07:26:50,085][00611] Updated weights for policy 0, policy_version 90152 (0.0008) [2023-10-08 07:26:50,454][00611] Updated weights for policy 0, policy_version 90162 (0.0008) [2023-10-08 07:26:50,825][00611] Updated weights for policy 0, policy_version 90172 (0.0008) [2023-10-08 07:26:51,751][00612] Updated weights for policy 1, policy_version 90660 (0.0009) [2023-10-08 07:26:52,128][00612] Updated weights for policy 1, policy_version 90670 (0.0007) [2023-10-08 07:26:52,493][00612] Updated weights for policy 1, policy_version 90680 (0.0008) [2023-10-08 07:26:53,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185204736. Throughput: 0: 1821.1, 1: 1855.6. Samples: 46303924. Policy #0 lag: (min: 30.0, avg: 30.1, max: 36.0) [2023-10-08 07:26:53,754][130385] Avg episode reward: [(0, '74.060'), (1, '79.110')] [2023-10-08 07:26:54,463][00611] Updated weights for policy 0, policy_version 90182 (0.0010) [2023-10-08 07:26:54,836][00611] Updated weights for policy 0, policy_version 90192 (0.0010) [2023-10-08 07:26:55,215][00611] Updated weights for policy 0, policy_version 90202 (0.0009) [2023-10-08 07:26:56,138][00612] Updated weights for policy 1, policy_version 90690 (0.0008) [2023-10-08 07:26:56,514][00612] Updated weights for policy 1, policy_version 90700 (0.0007) [2023-10-08 07:26:56,881][00612] Updated weights for policy 1, policy_version 90710 (0.0007) [2023-10-08 07:26:57,251][00612] Updated weights for policy 1, policy_version 90720 (0.0007) [2023-10-08 07:26:58,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185270272. Throughput: 0: 1821.4, 1: 1836.8. Samples: 46325582. Policy #0 lag: (min: 30.0, avg: 30.1, max: 36.0) [2023-10-08 07:26:58,755][130385] Avg episode reward: [(0, '76.150'), (1, '78.740')] [2023-10-08 07:26:58,927][00611] Updated weights for policy 0, policy_version 90212 (0.0008) [2023-10-08 07:26:59,297][00611] Updated weights for policy 0, policy_version 90222 (0.0008) [2023-10-08 07:26:59,665][00611] Updated weights for policy 0, policy_version 90232 (0.0009) [2023-10-08 07:27:00,907][00612] Updated weights for policy 1, policy_version 90730 (0.0008) [2023-10-08 07:27:01,271][00612] Updated weights for policy 1, policy_version 90740 (0.0010) [2023-10-08 07:27:01,646][00612] Updated weights for policy 1, policy_version 90750 (0.0008) [2023-10-08 07:27:03,361][00611] Updated weights for policy 0, policy_version 90242 (0.0009) [2023-10-08 07:27:03,742][00611] Updated weights for policy 0, policy_version 90252 (0.0007) [2023-10-08 07:27:03,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185335808. Throughput: 0: 1826.2, 1: 1858.9. Samples: 46348632. Policy #0 lag: (min: 30.0, avg: 30.1, max: 36.0) [2023-10-08 07:27:03,755][130385] Avg episode reward: [(0, '79.330'), (1, '80.410')] [2023-10-08 07:27:04,113][00611] Updated weights for policy 0, policy_version 90262 (0.0007) [2023-10-08 07:27:04,484][00611] Updated weights for policy 0, policy_version 90272 (0.0008) [2023-10-08 07:27:05,160][00612] Updated weights for policy 1, policy_version 90760 (0.0009) [2023-10-08 07:27:05,539][00612] Updated weights for policy 1, policy_version 90770 (0.0009) [2023-10-08 07:27:05,913][00612] Updated weights for policy 1, policy_version 90780 (0.0007) [2023-10-08 07:27:08,099][00611] Updated weights for policy 0, policy_version 90282 (0.0011) [2023-10-08 07:27:08,470][00611] Updated weights for policy 0, policy_version 90292 (0.0009) [2023-10-08 07:27:08,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 185401344. Throughput: 0: 1830.3, 1: 1836.6. Samples: 46358644. Policy #0 lag: (min: 30.0, avg: 30.1, max: 36.0) [2023-10-08 07:27:08,755][130385] Avg episode reward: [(0, '82.060'), (1, '81.150')] [2023-10-08 07:27:08,838][00611] Updated weights for policy 0, policy_version 90302 (0.0009) [2023-10-08 07:27:09,519][00612] Updated weights for policy 1, policy_version 90790 (0.0007) [2023-10-08 07:27:09,889][00612] Updated weights for policy 1, policy_version 90800 (0.0008) [2023-10-08 07:27:10,251][00612] Updated weights for policy 1, policy_version 90810 (0.0008) [2023-10-08 07:27:12,418][00611] Updated weights for policy 0, policy_version 90312 (0.0007) [2023-10-08 07:27:12,792][00611] Updated weights for policy 0, policy_version 90322 (0.0007) [2023-10-08 07:27:13,166][00611] Updated weights for policy 0, policy_version 90332 (0.0007) [2023-10-08 07:27:13,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185499648. Throughput: 0: 1833.1, 1: 1852.5. Samples: 46381926. Policy #0 lag: (min: 30.0, avg: 30.1, max: 36.0) [2023-10-08 07:27:13,755][130385] Avg episode reward: [(0, '80.020'), (1, '84.370')] [2023-10-08 07:27:13,941][00612] Updated weights for policy 1, policy_version 90820 (0.0010) [2023-10-08 07:27:14,308][00612] Updated weights for policy 1, policy_version 90830 (0.0007) [2023-10-08 07:27:14,671][00612] Updated weights for policy 1, policy_version 90840 (0.0008) [2023-10-08 07:27:16,855][00611] Updated weights for policy 0, policy_version 90342 (0.0007) [2023-10-08 07:27:17,221][00611] Updated weights for policy 0, policy_version 90352 (0.0007) [2023-10-08 07:27:17,603][00611] Updated weights for policy 0, policy_version 90362 (0.0008) [2023-10-08 07:27:18,245][00612] Updated weights for policy 1, policy_version 90850 (0.0007) [2023-10-08 07:27:18,609][00612] Updated weights for policy 1, policy_version 90860 (0.0008) [2023-10-08 07:27:18,754][130385] Fps is (10 sec: 16384.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 185565184. Throughput: 0: 1837.0, 1: 1852.4. Samples: 46403510. Policy #0 lag: (min: 30.0, avg: 30.1, max: 36.0) [2023-10-08 07:27:18,754][130385] Avg episode reward: [(0, '79.150'), (1, '80.020')] [2023-10-08 07:27:18,988][00612] Updated weights for policy 1, policy_version 90870 (0.0007) [2023-10-08 07:27:19,349][00612] Updated weights for policy 1, policy_version 90880 (0.0007) [2023-10-08 07:27:21,124][00611] Updated weights for policy 0, policy_version 90372 (0.0009) [2023-10-08 07:27:21,494][00611] Updated weights for policy 0, policy_version 90382 (0.0007) [2023-10-08 07:27:21,861][00611] Updated weights for policy 0, policy_version 90392 (0.0008) [2023-10-08 07:27:22,869][00612] Updated weights for policy 1, policy_version 90890 (0.0010) [2023-10-08 07:27:23,236][00612] Updated weights for policy 1, policy_version 90900 (0.0008) [2023-10-08 07:27:23,601][00612] Updated weights for policy 1, policy_version 90910 (0.0008) [2023-10-08 07:27:23,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 185663488. Throughput: 0: 1834.7, 1: 1853.5. Samples: 46415088. Policy #0 lag: (min: 30.0, avg: 30.1, max: 36.0) [2023-10-08 07:27:23,754][130385] Avg episode reward: [(0, '80.740'), (1, '82.130')] [2023-10-08 07:27:25,479][00611] Updated weights for policy 0, policy_version 90402 (0.0008) [2023-10-08 07:27:25,857][00611] Updated weights for policy 0, policy_version 90412 (0.0008) [2023-10-08 07:27:26,217][00611] Updated weights for policy 0, policy_version 90422 (0.0009) [2023-10-08 07:27:26,594][00611] Updated weights for policy 0, policy_version 90432 (0.0008) [2023-10-08 07:27:27,335][00612] Updated weights for policy 1, policy_version 90920 (0.0008) [2023-10-08 07:27:27,699][00612] Updated weights for policy 1, policy_version 90930 (0.0007) [2023-10-08 07:27:28,071][00612] Updated weights for policy 1, policy_version 90940 (0.0007) [2023-10-08 07:27:28,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 185729024. Throughput: 0: 1841.8, 1: 1845.3. Samples: 46436800. Policy #0 lag: (min: 30.0, avg: 30.1, max: 36.0) [2023-10-08 07:27:28,755][130385] Avg episode reward: [(0, '85.350'), (1, '82.750')] [2023-10-08 07:27:30,054][00611] Updated weights for policy 0, policy_version 90442 (0.0009) [2023-10-08 07:27:30,430][00611] Updated weights for policy 0, policy_version 90452 (0.0008) [2023-10-08 07:27:30,806][00611] Updated weights for policy 0, policy_version 90462 (0.0008) [2023-10-08 07:27:31,792][00612] Updated weights for policy 1, policy_version 90950 (0.0009) [2023-10-08 07:27:32,158][00612] Updated weights for policy 1, policy_version 90960 (0.0010) [2023-10-08 07:27:32,525][00612] Updated weights for policy 1, policy_version 90970 (0.0007) [2023-10-08 07:27:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185794560. Throughput: 0: 1852.2, 1: 1840.2. Samples: 46458704. Policy #0 lag: (min: 30.0, avg: 30.1, max: 36.0) [2023-10-08 07:27:33,754][130385] Avg episode reward: [(0, '81.230'), (1, '81.060')] [2023-10-08 07:27:34,410][00611] Updated weights for policy 0, policy_version 90472 (0.0008) [2023-10-08 07:27:34,783][00611] Updated weights for policy 0, policy_version 90482 (0.0007) [2023-10-08 07:27:35,152][00611] Updated weights for policy 0, policy_version 90492 (0.0007) [2023-10-08 07:27:36,238][00612] Updated weights for policy 1, policy_version 90980 (0.0008) [2023-10-08 07:27:36,605][00612] Updated weights for policy 1, policy_version 90990 (0.0008) [2023-10-08 07:27:36,968][00612] Updated weights for policy 1, policy_version 91000 (0.0008) [2023-10-08 07:27:38,696][00611] Updated weights for policy 0, policy_version 90502 (0.0007) [2023-10-08 07:27:38,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185860096. Throughput: 0: 1852.4, 1: 1837.5. Samples: 46469972. Policy #0 lag: (min: 30.0, avg: 30.1, max: 36.0) [2023-10-08 07:27:38,754][130385] Avg episode reward: [(0, '72.960'), (1, '76.570')] [2023-10-08 07:27:39,067][00611] Updated weights for policy 0, policy_version 90512 (0.0007) [2023-10-08 07:27:39,437][00611] Updated weights for policy 0, policy_version 90522 (0.0008) [2023-10-08 07:27:40,581][00612] Updated weights for policy 1, policy_version 91010 (0.0011) [2023-10-08 07:27:40,951][00612] Updated weights for policy 1, policy_version 91020 (0.0009) [2023-10-08 07:27:41,319][00612] Updated weights for policy 1, policy_version 91030 (0.0010) [2023-10-08 07:27:41,683][00612] Updated weights for policy 1, policy_version 91040 (0.0011) [2023-10-08 07:27:43,114][00611] Updated weights for policy 0, policy_version 90532 (0.0008) [2023-10-08 07:27:43,492][00611] Updated weights for policy 0, policy_version 90542 (0.0009) [2023-10-08 07:27:43,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185925632. Throughput: 0: 1858.9, 1: 1835.3. Samples: 46491822. Policy #0 lag: (min: 15.0, avg: 21.9, max: 47.0) [2023-10-08 07:27:43,755][130385] Avg episode reward: [(0, '74.140'), (1, '78.980')] [2023-10-08 07:27:43,864][00611] Updated weights for policy 0, policy_version 90552 (0.0008) [2023-10-08 07:27:45,345][00612] Updated weights for policy 1, policy_version 91050 (0.0009) [2023-10-08 07:27:45,722][00612] Updated weights for policy 1, policy_version 91060 (0.0009) [2023-10-08 07:27:46,083][00612] Updated weights for policy 1, policy_version 91070 (0.0009) [2023-10-08 07:27:47,349][00611] Updated weights for policy 0, policy_version 90562 (0.0008) [2023-10-08 07:27:47,713][00611] Updated weights for policy 0, policy_version 90572 (0.0008) [2023-10-08 07:27:48,090][00611] Updated weights for policy 0, policy_version 90582 (0.0011) [2023-10-08 07:27:48,462][00611] Updated weights for policy 0, policy_version 90592 (0.0008) [2023-10-08 07:27:48,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186023936. Throughput: 0: 1839.4, 1: 1840.5. Samples: 46514226. Policy #0 lag: (min: 15.0, avg: 21.9, max: 47.0) [2023-10-08 07:27:48,754][130385] Avg episode reward: [(0, '74.450'), (1, '76.240')] [2023-10-08 07:27:49,832][00612] Updated weights for policy 1, policy_version 91080 (0.0008) [2023-10-08 07:27:50,205][00612] Updated weights for policy 1, policy_version 91090 (0.0007) [2023-10-08 07:27:50,570][00612] Updated weights for policy 1, policy_version 91100 (0.0008) [2023-10-08 07:27:51,994][00611] Updated weights for policy 0, policy_version 90602 (0.0008) [2023-10-08 07:27:52,369][00611] Updated weights for policy 0, policy_version 90612 (0.0008) [2023-10-08 07:27:52,751][00611] Updated weights for policy 0, policy_version 90622 (0.0010) [2023-10-08 07:27:53,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186089472. Throughput: 0: 1866.8, 1: 1835.5. Samples: 46525246. Policy #0 lag: (min: 15.0, avg: 21.9, max: 47.0) [2023-10-08 07:27:53,754][130385] Avg episode reward: [(0, '75.360'), (1, '80.830')] [2023-10-08 07:27:54,063][00612] Updated weights for policy 1, policy_version 91110 (0.0008) [2023-10-08 07:27:54,430][00612] Updated weights for policy 1, policy_version 91120 (0.0007) [2023-10-08 07:27:54,792][00612] Updated weights for policy 1, policy_version 91130 (0.0007) [2023-10-08 07:27:56,401][00611] Updated weights for policy 0, policy_version 90632 (0.0009) [2023-10-08 07:27:56,776][00611] Updated weights for policy 0, policy_version 90642 (0.0008) [2023-10-08 07:27:57,150][00611] Updated weights for policy 0, policy_version 90652 (0.0009) [2023-10-08 07:27:58,427][00612] Updated weights for policy 1, policy_version 91140 (0.0009) [2023-10-08 07:27:58,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186155008. Throughput: 0: 1835.2, 1: 1840.8. Samples: 46547350. Policy #0 lag: (min: 15.0, avg: 21.9, max: 47.0) [2023-10-08 07:27:58,755][130385] Avg episode reward: [(0, '75.600'), (1, '84.290')] [2023-10-08 07:27:58,809][00612] Updated weights for policy 1, policy_version 91150 (0.0009) [2023-10-08 07:27:59,165][00612] Updated weights for policy 1, policy_version 91160 (0.0008) [2023-10-08 07:28:00,908][00611] Updated weights for policy 0, policy_version 90662 (0.0010) [2023-10-08 07:28:01,281][00611] Updated weights for policy 0, policy_version 90672 (0.0010) [2023-10-08 07:28:01,667][00611] Updated weights for policy 0, policy_version 90682 (0.0011) [2023-10-08 07:28:02,745][00612] Updated weights for policy 1, policy_version 91170 (0.0007) [2023-10-08 07:28:03,108][00612] Updated weights for policy 1, policy_version 91180 (0.0009) [2023-10-08 07:28:03,482][00612] Updated weights for policy 1, policy_version 91190 (0.0010) [2023-10-08 07:28:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 186220544. Throughput: 0: 1859.3, 1: 1832.7. Samples: 46569650. Policy #0 lag: (min: 15.0, avg: 21.9, max: 47.0) [2023-10-08 07:28:03,754][130385] Avg episode reward: [(0, '77.700'), (1, '82.980')] [2023-10-08 07:28:03,844][00612] Updated weights for policy 1, policy_version 91200 (0.0010) [2023-10-08 07:28:05,293][00611] Updated weights for policy 0, policy_version 90692 (0.0010) [2023-10-08 07:28:05,664][00611] Updated weights for policy 0, policy_version 90702 (0.0008) [2023-10-08 07:28:06,037][00611] Updated weights for policy 0, policy_version 90712 (0.0007) [2023-10-08 07:28:07,454][00612] Updated weights for policy 1, policy_version 91210 (0.0007) [2023-10-08 07:28:07,828][00612] Updated weights for policy 1, policy_version 91220 (0.0010) [2023-10-08 07:28:08,198][00612] Updated weights for policy 1, policy_version 91230 (0.0010) [2023-10-08 07:28:08,754][130385] Fps is (10 sec: 16384.4, 60 sec: 15291.9, 300 sec: 14662.3). Total num frames: 186318848. Throughput: 0: 1836.5, 1: 1845.4. Samples: 46580774. Policy #0 lag: (min: 15.0, avg: 21.9, max: 47.0) [2023-10-08 07:28:08,754][130385] Avg episode reward: [(0, '75.070'), (1, '86.100')] [2023-10-08 07:28:09,777][00611] Updated weights for policy 0, policy_version 90722 (0.0008) [2023-10-08 07:28:10,151][00611] Updated weights for policy 0, policy_version 90732 (0.0009) [2023-10-08 07:28:10,524][00611] Updated weights for policy 0, policy_version 90742 (0.0008) [2023-10-08 07:28:10,891][00611] Updated weights for policy 0, policy_version 90752 (0.0008) [2023-10-08 07:28:11,791][00612] Updated weights for policy 1, policy_version 91240 (0.0011) [2023-10-08 07:28:12,159][00612] Updated weights for policy 1, policy_version 91250 (0.0008) [2023-10-08 07:28:12,528][00612] Updated weights for policy 1, policy_version 91260 (0.0008) [2023-10-08 07:28:13,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186384384. Throughput: 0: 1855.8, 1: 1827.7. Samples: 46602556. Policy #0 lag: (min: 15.0, avg: 21.9, max: 47.0) [2023-10-08 07:28:13,754][130385] Avg episode reward: [(0, '74.840'), (1, '84.510')] [2023-10-08 07:28:14,583][00611] Updated weights for policy 0, policy_version 90762 (0.0007) [2023-10-08 07:28:14,951][00611] Updated weights for policy 0, policy_version 90772 (0.0007) [2023-10-08 07:28:15,325][00611] Updated weights for policy 0, policy_version 90782 (0.0008) [2023-10-08 07:28:16,106][00612] Updated weights for policy 1, policy_version 91270 (0.0008) [2023-10-08 07:28:16,479][00612] Updated weights for policy 1, policy_version 91280 (0.0007) [2023-10-08 07:28:16,849][00612] Updated weights for policy 1, policy_version 91290 (0.0011) [2023-10-08 07:28:18,754][130385] Fps is (10 sec: 13106.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 186449920. Throughput: 0: 1851.4, 1: 1850.0. Samples: 46625270. Policy #0 lag: (min: 15.0, avg: 21.9, max: 47.0) [2023-10-08 07:28:18,755][130385] Avg episode reward: [(0, '72.030'), (1, '82.420')] [2023-10-08 07:28:18,766][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000091296_93487104.pth... [2023-10-08 07:28:18,802][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000089568_91717632.pth [2023-10-08 07:28:18,807][00611] Updated weights for policy 0, policy_version 90792 (0.0010) [2023-10-08 07:28:19,190][00611] Updated weights for policy 0, policy_version 90802 (0.0010) [2023-10-08 07:28:19,570][00611] Updated weights for policy 0, policy_version 90812 (0.0010) [2023-10-08 07:28:19,711][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000090816_92995584.pth... [2023-10-08 07:28:19,750][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000089088_91226112.pth [2023-10-08 07:28:20,501][00612] Updated weights for policy 1, policy_version 91300 (0.0008) [2023-10-08 07:28:20,866][00612] Updated weights for policy 1, policy_version 91310 (0.0009) [2023-10-08 07:28:21,240][00612] Updated weights for policy 1, policy_version 91320 (0.0009) [2023-10-08 07:28:23,195][00611] Updated weights for policy 0, policy_version 90822 (0.0009) [2023-10-08 07:28:23,560][00611] Updated weights for policy 0, policy_version 90832 (0.0007) [2023-10-08 07:28:23,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 186515456. Throughput: 0: 1851.1, 1: 1832.7. Samples: 46635742. Policy #0 lag: (min: 15.0, avg: 21.9, max: 47.0) [2023-10-08 07:28:23,755][130385] Avg episode reward: [(0, '74.550'), (1, '85.540')] [2023-10-08 07:28:23,924][00611] Updated weights for policy 0, policy_version 90842 (0.0010) [2023-10-08 07:28:24,830][00612] Updated weights for policy 1, policy_version 91330 (0.0007) [2023-10-08 07:28:25,196][00612] Updated weights for policy 1, policy_version 91340 (0.0007) [2023-10-08 07:28:25,574][00612] Updated weights for policy 1, policy_version 91350 (0.0009) [2023-10-08 07:28:25,951][00612] Updated weights for policy 1, policy_version 91360 (0.0010) [2023-10-08 07:28:27,630][00611] Updated weights for policy 0, policy_version 90852 (0.0011) [2023-10-08 07:28:28,005][00611] Updated weights for policy 0, policy_version 90862 (0.0010) [2023-10-08 07:28:28,378][00611] Updated weights for policy 0, policy_version 90872 (0.0008) [2023-10-08 07:28:28,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186613760. Throughput: 0: 1845.0, 1: 1858.1. Samples: 46658464. Policy #0 lag: (min: 15.0, avg: 21.9, max: 47.0) [2023-10-08 07:28:28,755][130385] Avg episode reward: [(0, '67.430'), (1, '87.050')] [2023-10-08 07:28:29,587][00612] Updated weights for policy 1, policy_version 91370 (0.0010) [2023-10-08 07:28:29,960][00612] Updated weights for policy 1, policy_version 91380 (0.0008) [2023-10-08 07:28:30,329][00612] Updated weights for policy 1, policy_version 91390 (0.0008) [2023-10-08 07:28:32,044][00611] Updated weights for policy 0, policy_version 90882 (0.0010) [2023-10-08 07:28:32,412][00611] Updated weights for policy 0, policy_version 90892 (0.0008) [2023-10-08 07:28:32,793][00611] Updated weights for policy 0, policy_version 90902 (0.0008) [2023-10-08 07:28:33,166][00611] Updated weights for policy 0, policy_version 90912 (0.0010) [2023-10-08 07:28:33,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186679296. Throughput: 0: 1830.8, 1: 1856.2. Samples: 46680140. Policy #0 lag: (min: 15.0, avg: 21.9, max: 47.0) [2023-10-08 07:28:33,754][130385] Avg episode reward: [(0, '66.240'), (1, '80.570')] [2023-10-08 07:28:34,063][00612] Updated weights for policy 1, policy_version 91400 (0.0010) [2023-10-08 07:28:34,432][00612] Updated weights for policy 1, policy_version 91410 (0.0008) [2023-10-08 07:28:34,795][00612] Updated weights for policy 1, policy_version 91420 (0.0007) [2023-10-08 07:28:36,792][00611] Updated weights for policy 0, policy_version 90922 (0.0009) [2023-10-08 07:28:37,173][00611] Updated weights for policy 0, policy_version 90932 (0.0008) [2023-10-08 07:28:37,532][00611] Updated weights for policy 0, policy_version 90942 (0.0007) [2023-10-08 07:28:38,508][00612] Updated weights for policy 1, policy_version 91430 (0.0007) [2023-10-08 07:28:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186744832. Throughput: 0: 1835.8, 1: 1856.1. Samples: 46691382. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-08 07:28:38,755][130385] Avg episode reward: [(0, '66.700'), (1, '80.380')] [2023-10-08 07:28:38,884][00612] Updated weights for policy 1, policy_version 91440 (0.0009) [2023-10-08 07:28:39,243][00612] Updated weights for policy 1, policy_version 91450 (0.0009) [2023-10-08 07:28:41,055][00611] Updated weights for policy 0, policy_version 90952 (0.0010) [2023-10-08 07:28:41,427][00611] Updated weights for policy 0, policy_version 90962 (0.0009) [2023-10-08 07:28:41,800][00611] Updated weights for policy 0, policy_version 90972 (0.0008) [2023-10-08 07:28:42,918][00612] Updated weights for policy 1, policy_version 91460 (0.0008) [2023-10-08 07:28:43,285][00612] Updated weights for policy 1, policy_version 91470 (0.0009) [2023-10-08 07:28:43,650][00612] Updated weights for policy 1, policy_version 91480 (0.0008) [2023-10-08 07:28:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186810368. Throughput: 0: 1832.5, 1: 1848.3. Samples: 46712984. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-08 07:28:43,754][130385] Avg episode reward: [(0, '68.960'), (1, '80.590')] [2023-10-08 07:28:45,594][00611] Updated weights for policy 0, policy_version 90982 (0.0008) [2023-10-08 07:28:45,981][00611] Updated weights for policy 0, policy_version 90992 (0.0008) [2023-10-08 07:28:46,336][00611] Updated weights for policy 0, policy_version 91002 (0.0009) [2023-10-08 07:28:47,211][00612] Updated weights for policy 1, policy_version 91490 (0.0010) [2023-10-08 07:28:47,582][00612] Updated weights for policy 1, policy_version 91500 (0.0009) [2023-10-08 07:28:47,944][00612] Updated weights for policy 1, policy_version 91510 (0.0008) [2023-10-08 07:28:48,315][00612] Updated weights for policy 1, policy_version 91520 (0.0011) [2023-10-08 07:28:48,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 186908672. Throughput: 0: 1841.1, 1: 1827.5. Samples: 46734738. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-08 07:28:48,755][130385] Avg episode reward: [(0, '71.190'), (1, '85.010')] [2023-10-08 07:28:50,095][00611] Updated weights for policy 0, policy_version 91012 (0.0007) [2023-10-08 07:28:50,460][00611] Updated weights for policy 0, policy_version 91022 (0.0008) [2023-10-08 07:28:50,830][00611] Updated weights for policy 0, policy_version 91032 (0.0009) [2023-10-08 07:28:52,061][00612] Updated weights for policy 1, policy_version 91530 (0.0011) [2023-10-08 07:28:52,420][00612] Updated weights for policy 1, policy_version 91540 (0.0010) [2023-10-08 07:28:52,792][00612] Updated weights for policy 1, policy_version 91550 (0.0010) [2023-10-08 07:28:53,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 186974208. Throughput: 0: 1827.8, 1: 1843.1. Samples: 46745966. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-08 07:28:53,755][130385] Avg episode reward: [(0, '67.620'), (1, '86.610')] [2023-10-08 07:28:54,499][00611] Updated weights for policy 0, policy_version 91042 (0.0009) [2023-10-08 07:28:54,869][00611] Updated weights for policy 0, policy_version 91052 (0.0008) [2023-10-08 07:28:55,235][00611] Updated weights for policy 0, policy_version 91062 (0.0007) [2023-10-08 07:28:55,602][00611] Updated weights for policy 0, policy_version 91072 (0.0008) [2023-10-08 07:28:56,330][00612] Updated weights for policy 1, policy_version 91560 (0.0008) [2023-10-08 07:28:56,705][00612] Updated weights for policy 1, policy_version 91570 (0.0008) [2023-10-08 07:28:57,072][00612] Updated weights for policy 1, policy_version 91580 (0.0008) [2023-10-08 07:28:58,754][130385] Fps is (10 sec: 13107.7, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 187039744. Throughput: 0: 1841.6, 1: 1834.9. Samples: 46767998. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-08 07:28:58,754][130385] Avg episode reward: [(0, '68.480'), (1, '85.930')] [2023-10-08 07:28:59,175][00611] Updated weights for policy 0, policy_version 91082 (0.0011) [2023-10-08 07:28:59,541][00611] Updated weights for policy 0, policy_version 91092 (0.0009) [2023-10-08 07:28:59,913][00611] Updated weights for policy 0, policy_version 91102 (0.0007) [2023-10-08 07:29:00,658][00612] Updated weights for policy 1, policy_version 91590 (0.0007) [2023-10-08 07:29:01,024][00612] Updated weights for policy 1, policy_version 91600 (0.0007) [2023-10-08 07:29:01,397][00612] Updated weights for policy 1, policy_version 91610 (0.0008) [2023-10-08 07:29:03,460][00611] Updated weights for policy 0, policy_version 91112 (0.0007) [2023-10-08 07:29:03,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187105280. Throughput: 0: 1834.6, 1: 1844.0. Samples: 46790806. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-08 07:29:03,754][130385] Avg episode reward: [(0, '67.560'), (1, '84.490')] [2023-10-08 07:29:03,835][00611] Updated weights for policy 0, policy_version 91122 (0.0007) [2023-10-08 07:29:04,203][00611] Updated weights for policy 0, policy_version 91132 (0.0007) [2023-10-08 07:29:05,057][00612] Updated weights for policy 1, policy_version 91620 (0.0008) [2023-10-08 07:29:05,426][00612] Updated weights for policy 1, policy_version 91630 (0.0008) [2023-10-08 07:29:05,799][00612] Updated weights for policy 1, policy_version 91640 (0.0009) [2023-10-08 07:29:07,733][00611] Updated weights for policy 0, policy_version 91142 (0.0008) [2023-10-08 07:29:08,098][00611] Updated weights for policy 0, policy_version 91152 (0.0011) [2023-10-08 07:29:08,465][00611] Updated weights for policy 0, policy_version 91162 (0.0009) [2023-10-08 07:29:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 187203584. Throughput: 0: 1841.1, 1: 1831.1. Samples: 46800988. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-08 07:29:08,754][130385] Avg episode reward: [(0, '67.550'), (1, '86.530')] [2023-10-08 07:29:09,317][00612] Updated weights for policy 1, policy_version 91650 (0.0008) [2023-10-08 07:29:09,687][00612] Updated weights for policy 1, policy_version 91660 (0.0008) [2023-10-08 07:29:10,049][00612] Updated weights for policy 1, policy_version 91670 (0.0008) [2023-10-08 07:29:10,418][00612] Updated weights for policy 1, policy_version 91680 (0.0008) [2023-10-08 07:29:12,097][00611] Updated weights for policy 0, policy_version 91172 (0.0007) [2023-10-08 07:29:12,465][00611] Updated weights for policy 0, policy_version 91182 (0.0009) [2023-10-08 07:29:12,837][00611] Updated weights for policy 0, policy_version 91192 (0.0009) [2023-10-08 07:29:13,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 187269120. Throughput: 0: 1837.9, 1: 1844.3. Samples: 46824160. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-08 07:29:13,755][130385] Avg episode reward: [(0, '68.630'), (1, '83.450')] [2023-10-08 07:29:13,954][00612] Updated weights for policy 1, policy_version 91690 (0.0011) [2023-10-08 07:29:14,329][00612] Updated weights for policy 1, policy_version 91700 (0.0009) [2023-10-08 07:29:14,698][00612] Updated weights for policy 1, policy_version 91710 (0.0008) [2023-10-08 07:29:16,562][00611] Updated weights for policy 0, policy_version 91202 (0.0010) [2023-10-08 07:29:16,935][00611] Updated weights for policy 0, policy_version 91212 (0.0008) [2023-10-08 07:29:17,305][00611] Updated weights for policy 0, policy_version 91222 (0.0008) [2023-10-08 07:29:17,688][00611] Updated weights for policy 0, policy_version 91232 (0.0008) [2023-10-08 07:29:18,236][00612] Updated weights for policy 1, policy_version 91720 (0.0009) [2023-10-08 07:29:18,604][00612] Updated weights for policy 1, policy_version 91730 (0.0008) [2023-10-08 07:29:18,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 187334656. Throughput: 0: 1836.5, 1: 1847.9. Samples: 46845940. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-08 07:29:18,755][130385] Avg episode reward: [(0, '70.520'), (1, '82.520')] [2023-10-08 07:29:18,973][00612] Updated weights for policy 1, policy_version 91740 (0.0011) [2023-10-08 07:29:21,470][00611] Updated weights for policy 0, policy_version 91242 (0.0008) [2023-10-08 07:29:21,841][00611] Updated weights for policy 0, policy_version 91252 (0.0008) [2023-10-08 07:29:22,208][00611] Updated weights for policy 0, policy_version 91262 (0.0008) [2023-10-08 07:29:22,685][00612] Updated weights for policy 1, policy_version 91750 (0.0007) [2023-10-08 07:29:23,071][00612] Updated weights for policy 1, policy_version 91760 (0.0008) [2023-10-08 07:29:23,436][00612] Updated weights for policy 1, policy_version 91770 (0.0010) [2023-10-08 07:29:23,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 187432960. Throughput: 0: 1835.2, 1: 1856.5. Samples: 46857506. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-08 07:29:23,754][130385] Avg episode reward: [(0, '67.800'), (1, '82.990')] [2023-10-08 07:29:25,733][00611] Updated weights for policy 0, policy_version 91272 (0.0010) [2023-10-08 07:29:26,105][00611] Updated weights for policy 0, policy_version 91282 (0.0011) [2023-10-08 07:29:26,486][00611] Updated weights for policy 0, policy_version 91292 (0.0009) [2023-10-08 07:29:26,995][00612] Updated weights for policy 1, policy_version 91780 (0.0007) [2023-10-08 07:29:27,374][00612] Updated weights for policy 1, policy_version 91790 (0.0008) [2023-10-08 07:29:27,737][00612] Updated weights for policy 1, policy_version 91800 (0.0008) [2023-10-08 07:29:28,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 187498496. Throughput: 0: 1843.3, 1: 1849.2. Samples: 46879150. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-08 07:29:28,754][130385] Avg episode reward: [(0, '70.250'), (1, '81.420')] [2023-10-08 07:29:30,055][00611] Updated weights for policy 0, policy_version 91302 (0.0010) [2023-10-08 07:29:30,437][00611] Updated weights for policy 0, policy_version 91312 (0.0009) [2023-10-08 07:29:30,808][00611] Updated weights for policy 0, policy_version 91322 (0.0007) [2023-10-08 07:29:31,299][00612] Updated weights for policy 1, policy_version 91810 (0.0008) [2023-10-08 07:29:31,666][00612] Updated weights for policy 1, policy_version 91820 (0.0009) [2023-10-08 07:29:32,028][00612] Updated weights for policy 1, policy_version 91830 (0.0009) [2023-10-08 07:29:32,408][00612] Updated weights for policy 1, policy_version 91840 (0.0008) [2023-10-08 07:29:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 187564032. Throughput: 0: 1851.9, 1: 1854.5. Samples: 46901526. Policy #0 lag: (min: 18.0, avg: 18.1, max: 24.0) [2023-10-08 07:29:33,754][130385] Avg episode reward: [(0, '72.150'), (1, '76.160')] [2023-10-08 07:29:34,320][00611] Updated weights for policy 0, policy_version 91332 (0.0008) [2023-10-08 07:29:34,686][00611] Updated weights for policy 0, policy_version 91342 (0.0008) [2023-10-08 07:29:35,060][00611] Updated weights for policy 0, policy_version 91352 (0.0009) [2023-10-08 07:29:35,909][00612] Updated weights for policy 1, policy_version 91850 (0.0007) [2023-10-08 07:29:36,284][00612] Updated weights for policy 1, policy_version 91860 (0.0010) [2023-10-08 07:29:36,659][00612] Updated weights for policy 1, policy_version 91870 (0.0009) [2023-10-08 07:29:38,520][00611] Updated weights for policy 0, policy_version 91362 (0.0011) [2023-10-08 07:29:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 187629568. Throughput: 0: 1850.3, 1: 1847.8. Samples: 46912380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:29:38,754][130385] Avg episode reward: [(0, '74.590'), (1, '83.230')] [2023-10-08 07:29:38,890][00611] Updated weights for policy 0, policy_version 91372 (0.0008) [2023-10-08 07:29:39,262][00611] Updated weights for policy 0, policy_version 91382 (0.0007) [2023-10-08 07:29:39,632][00611] Updated weights for policy 0, policy_version 91392 (0.0007) [2023-10-08 07:29:40,250][00612] Updated weights for policy 1, policy_version 91880 (0.0009) [2023-10-08 07:29:40,617][00612] Updated weights for policy 1, policy_version 91890 (0.0009) [2023-10-08 07:29:40,988][00612] Updated weights for policy 1, policy_version 91900 (0.0009) [2023-10-08 07:29:43,309][00611] Updated weights for policy 0, policy_version 91402 (0.0007) [2023-10-08 07:29:43,690][00611] Updated weights for policy 0, policy_version 91412 (0.0009) [2023-10-08 07:29:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187695104. Throughput: 0: 1853.6, 1: 1860.5. Samples: 46935130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:29:43,754][130385] Avg episode reward: [(0, '75.480'), (1, '81.330')] [2023-10-08 07:29:44,054][00611] Updated weights for policy 0, policy_version 91422 (0.0010) [2023-10-08 07:29:44,613][00612] Updated weights for policy 1, policy_version 91910 (0.0008) [2023-10-08 07:29:44,985][00612] Updated weights for policy 1, policy_version 91920 (0.0010) [2023-10-08 07:29:45,355][00612] Updated weights for policy 1, policy_version 91930 (0.0010) [2023-10-08 07:29:47,690][00611] Updated weights for policy 0, policy_version 91432 (0.0010) [2023-10-08 07:29:48,061][00611] Updated weights for policy 0, policy_version 91442 (0.0010) [2023-10-08 07:29:48,438][00611] Updated weights for policy 0, policy_version 91452 (0.0010) [2023-10-08 07:29:48,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 187793408. Throughput: 0: 1834.7, 1: 1866.4. Samples: 46957356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:29:48,754][130385] Avg episode reward: [(0, '78.760'), (1, '81.500')] [2023-10-08 07:29:49,107][00612] Updated weights for policy 1, policy_version 91940 (0.0010) [2023-10-08 07:29:49,475][00612] Updated weights for policy 1, policy_version 91950 (0.0008) [2023-10-08 07:29:49,844][00612] Updated weights for policy 1, policy_version 91960 (0.0009) [2023-10-08 07:29:52,481][00611] Updated weights for policy 0, policy_version 91462 (0.0007) [2023-10-08 07:29:52,851][00611] Updated weights for policy 0, policy_version 91472 (0.0011) [2023-10-08 07:29:53,225][00611] Updated weights for policy 0, policy_version 91482 (0.0008) [2023-10-08 07:29:53,587][00612] Updated weights for policy 1, policy_version 91970 (0.0008) [2023-10-08 07:29:53,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 187858944. Throughput: 0: 1847.3, 1: 1862.8. Samples: 46967942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:29:53,755][130385] Avg episode reward: [(0, '79.500'), (1, '79.640')] [2023-10-08 07:29:53,952][00612] Updated weights for policy 1, policy_version 91980 (0.0009) [2023-10-08 07:29:54,317][00612] Updated weights for policy 1, policy_version 91990 (0.0009) [2023-10-08 07:29:54,688][00612] Updated weights for policy 1, policy_version 92000 (0.0008) [2023-10-08 07:29:57,090][00611] Updated weights for policy 0, policy_version 91492 (0.0009) [2023-10-08 07:29:57,463][00611] Updated weights for policy 0, policy_version 91502 (0.0008) [2023-10-08 07:29:57,839][00611] Updated weights for policy 0, policy_version 91512 (0.0007) [2023-10-08 07:29:58,181][00612] Updated weights for policy 1, policy_version 92010 (0.0008) [2023-10-08 07:29:58,542][00612] Updated weights for policy 1, policy_version 92020 (0.0009) [2023-10-08 07:29:58,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 187924480. Throughput: 0: 1835.9, 1: 1858.6. Samples: 46990412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:29:58,755][130385] Avg episode reward: [(0, '79.050'), (1, '77.840')] [2023-10-08 07:29:58,904][00612] Updated weights for policy 1, policy_version 92030 (0.0007) [2023-10-08 07:30:01,361][00611] Updated weights for policy 0, policy_version 91522 (0.0008) [2023-10-08 07:30:01,728][00611] Updated weights for policy 0, policy_version 91532 (0.0007) [2023-10-08 07:30:02,098][00611] Updated weights for policy 0, policy_version 91542 (0.0008) [2023-10-08 07:30:02,471][00611] Updated weights for policy 0, policy_version 91552 (0.0007) [2023-10-08 07:30:02,607][00612] Updated weights for policy 1, policy_version 92040 (0.0007) [2023-10-08 07:30:02,975][00612] Updated weights for policy 1, policy_version 92050 (0.0007) [2023-10-08 07:30:03,346][00612] Updated weights for policy 1, policy_version 92060 (0.0007) [2023-10-08 07:30:03,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 188022784. Throughput: 0: 1845.0, 1: 1831.8. Samples: 47011396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:30:03,755][130385] Avg episode reward: [(0, '79.030'), (1, '82.100')] [2023-10-08 07:30:06,062][00611] Updated weights for policy 0, policy_version 91562 (0.0011) [2023-10-08 07:30:06,439][00611] Updated weights for policy 0, policy_version 91572 (0.0010) [2023-10-08 07:30:06,813][00611] Updated weights for policy 0, policy_version 91582 (0.0007) [2023-10-08 07:30:06,961][00612] Updated weights for policy 1, policy_version 92070 (0.0008) [2023-10-08 07:30:07,329][00612] Updated weights for policy 1, policy_version 92080 (0.0010) [2023-10-08 07:30:07,708][00612] Updated weights for policy 1, policy_version 92090 (0.0009) [2023-10-08 07:30:08,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 188088320. Throughput: 0: 1830.5, 1: 1856.0. Samples: 47023400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:30:08,754][130385] Avg episode reward: [(0, '78.670'), (1, '81.910')] [2023-10-08 07:30:10,547][00611] Updated weights for policy 0, policy_version 91592 (0.0012) [2023-10-08 07:30:10,921][00611] Updated weights for policy 0, policy_version 91602 (0.0010) [2023-10-08 07:30:11,294][00611] Updated weights for policy 0, policy_version 91612 (0.0009) [2023-10-08 07:30:11,488][00612] Updated weights for policy 1, policy_version 92100 (0.0008) [2023-10-08 07:30:11,873][00612] Updated weights for policy 1, policy_version 92110 (0.0010) [2023-10-08 07:30:12,252][00612] Updated weights for policy 1, policy_version 92120 (0.0008) [2023-10-08 07:30:13,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 188153856. Throughput: 0: 1836.0, 1: 1838.7. Samples: 47044510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:30:13,754][130385] Avg episode reward: [(0, '80.300'), (1, '82.450')] [2023-10-08 07:30:15,019][00611] Updated weights for policy 0, policy_version 91622 (0.0008) [2023-10-08 07:30:15,387][00611] Updated weights for policy 0, policy_version 91632 (0.0010) [2023-10-08 07:30:15,749][00611] Updated weights for policy 0, policy_version 91642 (0.0010) [2023-10-08 07:30:15,835][00612] Updated weights for policy 1, policy_version 92130 (0.0009) [2023-10-08 07:30:16,200][00612] Updated weights for policy 1, policy_version 92140 (0.0009) [2023-10-08 07:30:16,570][00612] Updated weights for policy 1, policy_version 92150 (0.0008) [2023-10-08 07:30:16,937][00612] Updated weights for policy 1, policy_version 92160 (0.0009) [2023-10-08 07:30:18,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 188219392. Throughput: 0: 1830.6, 1: 1847.8. Samples: 47067056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:30:18,755][130385] Avg episode reward: [(0, '82.050'), (1, '81.010')] [2023-10-08 07:30:18,766][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000092160_94371840.pth... [2023-10-08 07:30:18,767][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000091648_93847552.pth... [2023-10-08 07:30:18,805][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000090432_92602368.pth [2023-10-08 07:30:18,806][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000089952_92110848.pth [2023-10-08 07:30:19,219][00611] Updated weights for policy 0, policy_version 91652 (0.0008) [2023-10-08 07:30:19,592][00611] Updated weights for policy 0, policy_version 91662 (0.0008) [2023-10-08 07:30:19,968][00611] Updated weights for policy 0, policy_version 91672 (0.0008) [2023-10-08 07:30:20,616][00612] Updated weights for policy 1, policy_version 92170 (0.0008) [2023-10-08 07:30:20,979][00612] Updated weights for policy 1, policy_version 92180 (0.0008) [2023-10-08 07:30:21,348][00612] Updated weights for policy 1, policy_version 92190 (0.0009) [2023-10-08 07:30:23,375][00611] Updated weights for policy 0, policy_version 91682 (0.0007) [2023-10-08 07:30:23,747][00611] Updated weights for policy 0, policy_version 91692 (0.0007) [2023-10-08 07:30:23,754][130385] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 188284928. Throughput: 0: 1835.9, 1: 1830.5. Samples: 47077372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:30:23,755][130385] Avg episode reward: [(0, '76.020'), (1, '78.820')] [2023-10-08 07:30:24,121][00611] Updated weights for policy 0, policy_version 91702 (0.0007) [2023-10-08 07:30:24,484][00611] Updated weights for policy 0, policy_version 91712 (0.0008) [2023-10-08 07:30:24,890][00612] Updated weights for policy 1, policy_version 92200 (0.0010) [2023-10-08 07:30:25,263][00612] Updated weights for policy 1, policy_version 92210 (0.0009) [2023-10-08 07:30:25,630][00612] Updated weights for policy 1, policy_version 92220 (0.0009) [2023-10-08 07:30:28,141][00611] Updated weights for policy 0, policy_version 91722 (0.0008) [2023-10-08 07:30:28,518][00611] Updated weights for policy 0, policy_version 91732 (0.0008) [2023-10-08 07:30:28,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 188350464. Throughput: 0: 1830.7, 1: 1846.1. Samples: 47100588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:30:28,754][130385] Avg episode reward: [(0, '76.060'), (1, '85.140')] [2023-10-08 07:30:28,890][00611] Updated weights for policy 0, policy_version 91742 (0.0009) [2023-10-08 07:30:29,141][00612] Updated weights for policy 1, policy_version 92230 (0.0007) [2023-10-08 07:30:29,516][00612] Updated weights for policy 1, policy_version 92240 (0.0007) [2023-10-08 07:30:29,895][00612] Updated weights for policy 1, policy_version 92250 (0.0009) [2023-10-08 07:30:32,651][00611] Updated weights for policy 0, policy_version 91752 (0.0011) [2023-10-08 07:30:33,028][00611] Updated weights for policy 0, policy_version 91762 (0.0009) [2023-10-08 07:30:33,396][00611] Updated weights for policy 0, policy_version 91772 (0.0008) [2023-10-08 07:30:33,636][00612] Updated weights for policy 1, policy_version 92260 (0.0010) [2023-10-08 07:30:33,754][130385] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 188448768. Throughput: 0: 1827.7, 1: 1846.9. Samples: 47122712. Policy #0 lag: (min: 25.0, avg: 25.3, max: 37.0) [2023-10-08 07:30:33,754][130385] Avg episode reward: [(0, '76.640'), (1, '83.920')] [2023-10-08 07:30:34,004][00612] Updated weights for policy 1, policy_version 92270 (0.0012) [2023-10-08 07:30:34,381][00612] Updated weights for policy 1, policy_version 92280 (0.0010) [2023-10-08 07:30:37,079][00611] Updated weights for policy 0, policy_version 91782 (0.0009) [2023-10-08 07:30:37,455][00611] Updated weights for policy 0, policy_version 91792 (0.0008) [2023-10-08 07:30:37,831][00611] Updated weights for policy 0, policy_version 91802 (0.0009) [2023-10-08 07:30:38,143][00612] Updated weights for policy 1, policy_version 92290 (0.0010) [2023-10-08 07:30:38,513][00612] Updated weights for policy 1, policy_version 92300 (0.0011) [2023-10-08 07:30:38,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 188514304. Throughput: 0: 1829.3, 1: 1847.5. Samples: 47133394. Policy #0 lag: (min: 25.0, avg: 25.3, max: 37.0) [2023-10-08 07:30:38,754][130385] Avg episode reward: [(0, '75.830'), (1, '85.170')] [2023-10-08 07:30:38,879][00612] Updated weights for policy 1, policy_version 92310 (0.0010) [2023-10-08 07:30:39,246][00612] Updated weights for policy 1, policy_version 92320 (0.0011) [2023-10-08 07:30:41,608][00611] Updated weights for policy 0, policy_version 91812 (0.0008) [2023-10-08 07:30:41,971][00611] Updated weights for policy 0, policy_version 91822 (0.0007) [2023-10-08 07:30:42,350][00611] Updated weights for policy 0, policy_version 91832 (0.0007) [2023-10-08 07:30:42,838][00612] Updated weights for policy 1, policy_version 92330 (0.0010) [2023-10-08 07:30:43,211][00612] Updated weights for policy 1, policy_version 92340 (0.0010) [2023-10-08 07:30:43,579][00612] Updated weights for policy 1, policy_version 92350 (0.0010) [2023-10-08 07:30:43,754][130385] Fps is (10 sec: 16383.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 188612608. Throughput: 0: 1823.3, 1: 1847.5. Samples: 47155598. Policy #0 lag: (min: 25.0, avg: 25.3, max: 37.0) [2023-10-08 07:30:43,756][130385] Avg episode reward: [(0, '76.630'), (1, '84.610')] [2023-10-08 07:30:46,214][00611] Updated weights for policy 0, policy_version 91842 (0.0008) [2023-10-08 07:30:46,587][00611] Updated weights for policy 0, policy_version 91852 (0.0008) [2023-10-08 07:30:46,952][00611] Updated weights for policy 0, policy_version 91862 (0.0010) [2023-10-08 07:30:47,010][00612] Updated weights for policy 1, policy_version 92360 (0.0009) [2023-10-08 07:30:47,322][00611] Updated weights for policy 0, policy_version 91872 (0.0008) [2023-10-08 07:30:47,374][00612] Updated weights for policy 1, policy_version 92370 (0.0007) [2023-10-08 07:30:47,744][00612] Updated weights for policy 1, policy_version 92380 (0.0009) [2023-10-08 07:30:48,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 188678144. Throughput: 0: 1830.0, 1: 1841.5. Samples: 47176610. Policy #0 lag: (min: 25.0, avg: 25.3, max: 37.0) [2023-10-08 07:30:48,755][130385] Avg episode reward: [(0, '77.000'), (1, '88.140')] [2023-10-08 07:30:50,952][00611] Updated weights for policy 0, policy_version 91882 (0.0007) [2023-10-08 07:30:51,322][00611] Updated weights for policy 0, policy_version 91892 (0.0009) [2023-10-08 07:30:51,396][00612] Updated weights for policy 1, policy_version 92390 (0.0008) [2023-10-08 07:30:51,696][00611] Updated weights for policy 0, policy_version 91902 (0.0008) [2023-10-08 07:30:51,770][00612] Updated weights for policy 1, policy_version 92400 (0.0007) [2023-10-08 07:30:52,138][00612] Updated weights for policy 1, policy_version 92410 (0.0008) [2023-10-08 07:30:53,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 188743680. Throughput: 0: 1831.3, 1: 1850.8. Samples: 47189092. Policy #0 lag: (min: 25.0, avg: 25.3, max: 37.0) [2023-10-08 07:30:53,755][130385] Avg episode reward: [(0, '75.790'), (1, '86.410')] [2023-10-08 07:30:55,352][00611] Updated weights for policy 0, policy_version 91912 (0.0008) [2023-10-08 07:30:55,725][00611] Updated weights for policy 0, policy_version 91922 (0.0008) [2023-10-08 07:30:55,754][00612] Updated weights for policy 1, policy_version 92420 (0.0009) [2023-10-08 07:30:56,096][00611] Updated weights for policy 0, policy_version 91932 (0.0007) [2023-10-08 07:30:56,124][00612] Updated weights for policy 1, policy_version 92430 (0.0007) [2023-10-08 07:30:56,485][00612] Updated weights for policy 1, policy_version 92440 (0.0010) [2023-10-08 07:30:58,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 188809216. Throughput: 0: 1831.2, 1: 1842.3. Samples: 47209818. Policy #0 lag: (min: 25.0, avg: 25.3, max: 37.0) [2023-10-08 07:30:58,754][130385] Avg episode reward: [(0, '74.390'), (1, '85.380')] [2023-10-08 07:30:59,745][00611] Updated weights for policy 0, policy_version 91942 (0.0008) [2023-10-08 07:31:00,025][00612] Updated weights for policy 1, policy_version 92450 (0.0007) [2023-10-08 07:31:00,122][00611] Updated weights for policy 0, policy_version 91952 (0.0007) [2023-10-08 07:31:00,415][00612] Updated weights for policy 1, policy_version 92460 (0.0008) [2023-10-08 07:31:00,498][00611] Updated weights for policy 0, policy_version 91962 (0.0008) [2023-10-08 07:31:00,789][00612] Updated weights for policy 1, policy_version 92470 (0.0009) [2023-10-08 07:31:01,147][00612] Updated weights for policy 1, policy_version 92480 (0.0009) [2023-10-08 07:31:03,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 188874752. Throughput: 0: 1830.3, 1: 1856.9. Samples: 47232978. Policy #0 lag: (min: 25.0, avg: 25.3, max: 37.0) [2023-10-08 07:31:03,754][130385] Avg episode reward: [(0, '72.030'), (1, '90.700')] [2023-10-08 07:31:04,145][00611] Updated weights for policy 0, policy_version 91972 (0.0009) [2023-10-08 07:31:04,519][00611] Updated weights for policy 0, policy_version 91982 (0.0009) [2023-10-08 07:31:04,774][00612] Updated weights for policy 1, policy_version 92490 (0.0008) [2023-10-08 07:31:04,878][00611] Updated weights for policy 0, policy_version 91992 (0.0008) [2023-10-08 07:31:05,147][00612] Updated weights for policy 1, policy_version 92500 (0.0008) [2023-10-08 07:31:05,509][00612] Updated weights for policy 1, policy_version 92510 (0.0008) [2023-10-08 07:31:08,400][00611] Updated weights for policy 0, policy_version 92002 (0.0008) [2023-10-08 07:31:08,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 188940288. Throughput: 0: 1827.4, 1: 1848.9. Samples: 47242806. Policy #0 lag: (min: 25.0, avg: 25.3, max: 37.0) [2023-10-08 07:31:08,755][130385] Avg episode reward: [(0, '72.890'), (1, '90.320')] [2023-10-08 07:31:08,763][00611] Updated weights for policy 0, policy_version 92012 (0.0007) [2023-10-08 07:31:09,069][00612] Updated weights for policy 1, policy_version 92520 (0.0010) [2023-10-08 07:31:09,126][00611] Updated weights for policy 0, policy_version 92022 (0.0007) [2023-10-08 07:31:09,423][00612] Updated weights for policy 1, policy_version 92530 (0.0008) [2023-10-08 07:31:09,494][00611] Updated weights for policy 0, policy_version 92032 (0.0007) [2023-10-08 07:31:09,795][00612] Updated weights for policy 1, policy_version 92540 (0.0010) [2023-10-08 07:31:13,202][00611] Updated weights for policy 0, policy_version 92042 (0.0009) [2023-10-08 07:31:13,502][00612] Updated weights for policy 1, policy_version 92550 (0.0009) [2023-10-08 07:31:13,562][00611] Updated weights for policy 0, policy_version 92052 (0.0008) [2023-10-08 07:31:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 189005824. Throughput: 0: 1828.2, 1: 1855.0. Samples: 47266334. Policy #0 lag: (min: 25.0, avg: 25.3, max: 37.0) [2023-10-08 07:31:13,754][130385] Avg episode reward: [(0, '73.920'), (1, '90.680')] [2023-10-08 07:31:13,871][00612] Updated weights for policy 1, policy_version 92560 (0.0008) [2023-10-08 07:31:13,932][00611] Updated weights for policy 0, policy_version 92062 (0.0009) [2023-10-08 07:31:14,239][00612] Updated weights for policy 1, policy_version 92570 (0.0008) [2023-10-08 07:31:17,575][00611] Updated weights for policy 0, policy_version 92072 (0.0009) [2023-10-08 07:31:17,872][00612] Updated weights for policy 1, policy_version 92580 (0.0009) [2023-10-08 07:31:17,940][00611] Updated weights for policy 0, policy_version 92082 (0.0008) [2023-10-08 07:31:18,244][00612] Updated weights for policy 1, policy_version 92590 (0.0009) [2023-10-08 07:31:18,321][00611] Updated weights for policy 0, policy_version 92092 (0.0009) [2023-10-08 07:31:18,614][00612] Updated weights for policy 1, policy_version 92600 (0.0009) [2023-10-08 07:31:18,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 189104128. Throughput: 0: 1828.5, 1: 1841.7. Samples: 47287874. Policy #0 lag: (min: 25.0, avg: 25.3, max: 37.0) [2023-10-08 07:31:18,754][130385] Avg episode reward: [(0, '74.300'), (1, '88.960')] [2023-10-08 07:31:22,079][00611] Updated weights for policy 0, policy_version 92102 (0.0009) [2023-10-08 07:31:22,383][00612] Updated weights for policy 1, policy_version 92610 (0.0009) [2023-10-08 07:31:22,448][00611] Updated weights for policy 0, policy_version 92112 (0.0007) [2023-10-08 07:31:22,748][00612] Updated weights for policy 1, policy_version 92620 (0.0008) [2023-10-08 07:31:22,823][00611] Updated weights for policy 0, policy_version 92122 (0.0008) [2023-10-08 07:31:23,111][00612] Updated weights for policy 1, policy_version 92630 (0.0008) [2023-10-08 07:31:23,476][00612] Updated weights for policy 1, policy_version 92640 (0.0008) [2023-10-08 07:31:23,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 189202432. Throughput: 0: 1830.3, 1: 1855.3. Samples: 47299244. Policy #0 lag: (min: 25.0, avg: 25.3, max: 37.0) [2023-10-08 07:31:23,754][130385] Avg episode reward: [(0, '71.600'), (1, '90.640')] [2023-10-08 07:31:26,477][00611] Updated weights for policy 0, policy_version 92132 (0.0007) [2023-10-08 07:31:26,850][00611] Updated weights for policy 0, policy_version 92142 (0.0009) [2023-10-08 07:31:27,159][00612] Updated weights for policy 1, policy_version 92650 (0.0009) [2023-10-08 07:31:27,219][00611] Updated weights for policy 0, policy_version 92152 (0.0009) [2023-10-08 07:31:27,536][00612] Updated weights for policy 1, policy_version 92660 (0.0007) [2023-10-08 07:31:27,908][00612] Updated weights for policy 1, policy_version 92670 (0.0009) [2023-10-08 07:31:28,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 189267968. Throughput: 0: 1823.5, 1: 1838.5. Samples: 47320384. Policy #0 lag: (min: 25.0, avg: 25.3, max: 37.0) [2023-10-08 07:31:28,755][130385] Avg episode reward: [(0, '74.070'), (1, '89.240')] [2023-10-08 07:31:30,903][00611] Updated weights for policy 0, policy_version 92162 (0.0008) [2023-10-08 07:31:31,285][00611] Updated weights for policy 0, policy_version 92172 (0.0009) [2023-10-08 07:31:31,491][00612] Updated weights for policy 1, policy_version 92680 (0.0008) [2023-10-08 07:31:31,661][00611] Updated weights for policy 0, policy_version 92182 (0.0009) [2023-10-08 07:31:31,859][00612] Updated weights for policy 1, policy_version 92690 (0.0008) [2023-10-08 07:31:32,029][00611] Updated weights for policy 0, policy_version 92192 (0.0009) [2023-10-08 07:31:32,221][00612] Updated weights for policy 1, policy_version 92700 (0.0011) [2023-10-08 07:31:33,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 189333504. Throughput: 0: 1826.3, 1: 1843.2. Samples: 47341734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:31:33,755][130385] Avg episode reward: [(0, '76.650'), (1, '89.720')] [2023-10-08 07:31:35,710][00611] Updated weights for policy 0, policy_version 92202 (0.0009) [2023-10-08 07:31:35,971][00612] Updated weights for policy 1, policy_version 92710 (0.0008) [2023-10-08 07:31:36,082][00611] Updated weights for policy 0, policy_version 92212 (0.0008) [2023-10-08 07:31:36,335][00612] Updated weights for policy 1, policy_version 92720 (0.0007) [2023-10-08 07:31:36,443][00611] Updated weights for policy 0, policy_version 92222 (0.0007) [2023-10-08 07:31:36,699][00612] Updated weights for policy 1, policy_version 92730 (0.0007) [2023-10-08 07:31:38,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 189399040. Throughput: 0: 1817.2, 1: 1826.9. Samples: 47353078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:31:38,755][130385] Avg episode reward: [(0, '76.330'), (1, '93.820')] [2023-10-08 07:31:39,961][00611] Updated weights for policy 0, policy_version 92232 (0.0007) [2023-10-08 07:31:40,250][00612] Updated weights for policy 1, policy_version 92740 (0.0007) [2023-10-08 07:31:40,326][00611] Updated weights for policy 0, policy_version 92242 (0.0007) [2023-10-08 07:31:40,611][00612] Updated weights for policy 1, policy_version 92750 (0.0010) [2023-10-08 07:31:40,692][00611] Updated weights for policy 0, policy_version 92252 (0.0007) [2023-10-08 07:31:40,981][00612] Updated weights for policy 1, policy_version 92760 (0.0008) [2023-10-08 07:31:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 189464576. Throughput: 0: 1828.5, 1: 1840.5. Samples: 47374924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:31:43,754][130385] Avg episode reward: [(0, '76.380'), (1, '96.470')] [2023-10-08 07:31:44,385][00611] Updated weights for policy 0, policy_version 92262 (0.0007) [2023-10-08 07:31:44,679][00612] Updated weights for policy 1, policy_version 92770 (0.0007) [2023-10-08 07:31:44,751][00611] Updated weights for policy 0, policy_version 92272 (0.0007) [2023-10-08 07:31:45,082][00612] Updated weights for policy 1, policy_version 92780 (0.0008) [2023-10-08 07:31:45,127][00611] Updated weights for policy 0, policy_version 92282 (0.0009) [2023-10-08 07:31:45,460][00612] Updated weights for policy 1, policy_version 92790 (0.0009) [2023-10-08 07:31:45,823][00612] Updated weights for policy 1, policy_version 92800 (0.0007) [2023-10-08 07:31:48,685][00611] Updated weights for policy 0, policy_version 92292 (0.0008) [2023-10-08 07:31:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 189530112. Throughput: 0: 1830.1, 1: 1836.8. Samples: 47397990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:31:48,755][130385] Avg episode reward: [(0, '74.940'), (1, '91.860')] [2023-10-08 07:31:49,051][00611] Updated weights for policy 0, policy_version 92302 (0.0008) [2023-10-08 07:31:49,429][00611] Updated weights for policy 0, policy_version 92312 (0.0008) [2023-10-08 07:31:49,450][00612] Updated weights for policy 1, policy_version 92810 (0.0008) [2023-10-08 07:31:49,824][00612] Updated weights for policy 1, policy_version 92820 (0.0009) [2023-10-08 07:31:50,185][00612] Updated weights for policy 1, policy_version 92830 (0.0007) [2023-10-08 07:31:53,014][00611] Updated weights for policy 0, policy_version 92322 (0.0008) [2023-10-08 07:31:53,384][00611] Updated weights for policy 0, policy_version 92332 (0.0011) [2023-10-08 07:31:53,749][00612] Updated weights for policy 1, policy_version 92840 (0.0008) [2023-10-08 07:31:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 189595648. Throughput: 0: 1829.9, 1: 1839.8. Samples: 47407942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:31:53,754][130385] Avg episode reward: [(0, '75.630'), (1, '94.720')] [2023-10-08 07:31:53,760][00611] Updated weights for policy 0, policy_version 92342 (0.0009) [2023-10-08 07:31:54,118][00612] Updated weights for policy 1, policy_version 92850 (0.0007) [2023-10-08 07:31:54,133][00611] Updated weights for policy 0, policy_version 92352 (0.0008) [2023-10-08 07:31:54,487][00612] Updated weights for policy 1, policy_version 92860 (0.0008) [2023-10-08 07:31:57,900][00611] Updated weights for policy 0, policy_version 92362 (0.0008) [2023-10-08 07:31:58,088][00612] Updated weights for policy 1, policy_version 92870 (0.0008) [2023-10-08 07:31:58,270][00611] Updated weights for policy 0, policy_version 92372 (0.0010) [2023-10-08 07:31:58,452][00612] Updated weights for policy 1, policy_version 92880 (0.0007) [2023-10-08 07:31:58,639][00611] Updated weights for policy 0, policy_version 92382 (0.0008) [2023-10-08 07:31:58,754][130385] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 189693952. Throughput: 0: 1824.0, 1: 1835.4. Samples: 47431010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:31:58,754][130385] Avg episode reward: [(0, '71.210'), (1, '92.750')] [2023-10-08 07:31:58,813][00612] Updated weights for policy 1, policy_version 92890 (0.0008) [2023-10-08 07:32:02,385][00611] Updated weights for policy 0, policy_version 92392 (0.0009) [2023-10-08 07:32:02,411][00612] Updated weights for policy 1, policy_version 92900 (0.0009) [2023-10-08 07:32:02,750][00611] Updated weights for policy 0, policy_version 92402 (0.0007) [2023-10-08 07:32:02,777][00612] Updated weights for policy 1, policy_version 92910 (0.0008) [2023-10-08 07:32:03,118][00611] Updated weights for policy 0, policy_version 92412 (0.0008) [2023-10-08 07:32:03,143][00612] Updated weights for policy 1, policy_version 92920 (0.0008) [2023-10-08 07:32:03,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14884.5). Total num frames: 189792256. Throughput: 0: 1811.2, 1: 1823.7. Samples: 47451444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:32:03,754][130385] Avg episode reward: [(0, '71.740'), (1, '93.290')] [2023-10-08 07:32:06,767][00611] Updated weights for policy 0, policy_version 92422 (0.0010) [2023-10-08 07:32:06,956][00612] Updated weights for policy 1, policy_version 92930 (0.0008) [2023-10-08 07:32:07,134][00611] Updated weights for policy 0, policy_version 92432 (0.0008) [2023-10-08 07:32:07,315][00612] Updated weights for policy 1, policy_version 92940 (0.0008) [2023-10-08 07:32:07,516][00611] Updated weights for policy 0, policy_version 92442 (0.0008) [2023-10-08 07:32:07,677][00612] Updated weights for policy 1, policy_version 92950 (0.0008) [2023-10-08 07:32:08,044][00612] Updated weights for policy 1, policy_version 92960 (0.0009) [2023-10-08 07:32:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 189857792. Throughput: 0: 1823.6, 1: 1838.4. Samples: 47464030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:32:08,754][130385] Avg episode reward: [(0, '73.460'), (1, '95.390')] [2023-10-08 07:32:11,215][00611] Updated weights for policy 0, policy_version 92452 (0.0008) [2023-10-08 07:32:11,595][00611] Updated weights for policy 0, policy_version 92462 (0.0007) [2023-10-08 07:32:11,614][00612] Updated weights for policy 1, policy_version 92970 (0.0008) [2023-10-08 07:32:11,954][00611] Updated weights for policy 0, policy_version 92472 (0.0010) [2023-10-08 07:32:11,988][00612] Updated weights for policy 1, policy_version 92980 (0.0007) [2023-10-08 07:32:12,360][00612] Updated weights for policy 1, policy_version 92990 (0.0007) [2023-10-08 07:32:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 189923328. Throughput: 0: 1823.3, 1: 1829.6. Samples: 47484768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:32:13,754][130385] Avg episode reward: [(0, '78.640'), (1, '91.520')] [2023-10-08 07:32:15,629][00611] Updated weights for policy 0, policy_version 92482 (0.0009) [2023-10-08 07:32:15,889][00612] Updated weights for policy 1, policy_version 93000 (0.0007) [2023-10-08 07:32:16,001][00611] Updated weights for policy 0, policy_version 92492 (0.0010) [2023-10-08 07:32:16,264][00612] Updated weights for policy 1, policy_version 93010 (0.0008) [2023-10-08 07:32:16,373][00611] Updated weights for policy 0, policy_version 92502 (0.0008) [2023-10-08 07:32:16,626][00612] Updated weights for policy 1, policy_version 93020 (0.0007) [2023-10-08 07:32:16,734][00611] Updated weights for policy 0, policy_version 92512 (0.0009) [2023-10-08 07:32:18,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 189988864. Throughput: 0: 1832.4, 1: 1846.5. Samples: 47507284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:32:18,755][130385] Avg episode reward: [(0, '75.620'), (1, '97.200')] [2023-10-08 07:32:18,763][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000093024_95256576.pth... [2023-10-08 07:32:18,763][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000092512_94732288.pth... [2023-10-08 07:32:18,798][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000090816_92995584.pth [2023-10-08 07:32:18,800][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000091296_93487104.pth [2023-10-08 07:32:20,222][00611] Updated weights for policy 0, policy_version 92522 (0.0009) [2023-10-08 07:32:20,267][00612] Updated weights for policy 1, policy_version 93030 (0.0009) [2023-10-08 07:32:20,596][00611] Updated weights for policy 0, policy_version 92532 (0.0007) [2023-10-08 07:32:20,634][00612] Updated weights for policy 1, policy_version 93040 (0.0009) [2023-10-08 07:32:20,963][00611] Updated weights for policy 0, policy_version 92542 (0.0009) [2023-10-08 07:32:21,004][00612] Updated weights for policy 1, policy_version 93050 (0.0008) [2023-10-08 07:32:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 190054400. Throughput: 0: 1826.5, 1: 1827.7. Samples: 47517514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:32:23,754][130385] Avg episode reward: [(0, '75.590'), (1, '97.750')] [2023-10-08 07:32:24,697][00612] Updated weights for policy 1, policy_version 93060 (0.0009) [2023-10-08 07:32:24,711][00611] Updated weights for policy 0, policy_version 92552 (0.0008) [2023-10-08 07:32:25,059][00612] Updated weights for policy 1, policy_version 93070 (0.0007) [2023-10-08 07:32:25,078][00611] Updated weights for policy 0, policy_version 92562 (0.0008) [2023-10-08 07:32:25,427][00612] Updated weights for policy 1, policy_version 93080 (0.0008) [2023-10-08 07:32:25,445][00611] Updated weights for policy 0, policy_version 92572 (0.0007) [2023-10-08 07:32:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 190119936. Throughput: 0: 1827.3, 1: 1848.7. Samples: 47540344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:32:28,755][130385] Avg episode reward: [(0, '76.110'), (1, '94.250')] [2023-10-08 07:32:29,127][00612] Updated weights for policy 1, policy_version 93090 (0.0008) [2023-10-08 07:32:29,224][00611] Updated weights for policy 0, policy_version 92582 (0.0009) [2023-10-08 07:32:29,495][00612] Updated weights for policy 1, policy_version 93100 (0.0008) [2023-10-08 07:32:29,612][00611] Updated weights for policy 0, policy_version 92592 (0.0008) [2023-10-08 07:32:29,867][00612] Updated weights for policy 1, policy_version 93110 (0.0009) [2023-10-08 07:32:29,991][00611] Updated weights for policy 0, policy_version 92602 (0.0008) [2023-10-08 07:32:30,240][00612] Updated weights for policy 1, policy_version 93120 (0.0007) [2023-10-08 07:32:33,643][00611] Updated weights for policy 0, policy_version 92612 (0.0008) [2023-10-08 07:32:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 190185472. Throughput: 0: 1822.4, 1: 1848.5. Samples: 47563178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:32:33,754][130385] Avg episode reward: [(0, '74.850'), (1, '90.360')] [2023-10-08 07:32:34,022][00611] Updated weights for policy 0, policy_version 92622 (0.0009) [2023-10-08 07:32:34,045][00612] Updated weights for policy 1, policy_version 93130 (0.0007) [2023-10-08 07:32:34,396][00611] Updated weights for policy 0, policy_version 92632 (0.0009) [2023-10-08 07:32:34,425][00612] Updated weights for policy 1, policy_version 93140 (0.0008) [2023-10-08 07:32:34,788][00612] Updated weights for policy 1, policy_version 93150 (0.0008) [2023-10-08 07:32:38,066][00611] Updated weights for policy 0, policy_version 92642 (0.0008) [2023-10-08 07:32:38,433][00611] Updated weights for policy 0, policy_version 92652 (0.0010) [2023-10-08 07:32:38,437][00612] Updated weights for policy 1, policy_version 93160 (0.0008) [2023-10-08 07:32:38,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 190251008. Throughput: 0: 1819.4, 1: 1843.5. Samples: 47572772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:32:38,754][130385] Avg episode reward: [(0, '77.510'), (1, '91.970')] [2023-10-08 07:32:38,799][00611] Updated weights for policy 0, policy_version 92662 (0.0008) [2023-10-08 07:32:38,810][00612] Updated weights for policy 1, policy_version 93170 (0.0007) [2023-10-08 07:32:39,170][00611] Updated weights for policy 0, policy_version 92672 (0.0008) [2023-10-08 07:32:39,178][00612] Updated weights for policy 1, policy_version 93180 (0.0007) [2023-10-08 07:32:42,804][00611] Updated weights for policy 0, policy_version 92682 (0.0008) [2023-10-08 07:32:42,878][00612] Updated weights for policy 1, policy_version 93190 (0.0008) [2023-10-08 07:32:43,183][00611] Updated weights for policy 0, policy_version 92692 (0.0009) [2023-10-08 07:32:43,249][00612] Updated weights for policy 1, policy_version 93200 (0.0008) [2023-10-08 07:32:43,551][00611] Updated weights for policy 0, policy_version 92702 (0.0008) [2023-10-08 07:32:43,610][00612] Updated weights for policy 1, policy_version 93210 (0.0008) [2023-10-08 07:32:43,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190349312. Throughput: 0: 1821.9, 1: 1844.4. Samples: 47595994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:32:43,754][130385] Avg episode reward: [(0, '76.740'), (1, '94.370')] [2023-10-08 07:32:47,123][00612] Updated weights for policy 1, policy_version 93220 (0.0008) [2023-10-08 07:32:47,227][00611] Updated weights for policy 0, policy_version 92712 (0.0007) [2023-10-08 07:32:47,492][00612] Updated weights for policy 1, policy_version 93230 (0.0008) [2023-10-08 07:32:47,595][00611] Updated weights for policy 0, policy_version 92722 (0.0008) [2023-10-08 07:32:47,868][00612] Updated weights for policy 1, policy_version 93240 (0.0008) [2023-10-08 07:32:47,966][00611] Updated weights for policy 0, policy_version 92732 (0.0007) [2023-10-08 07:32:48,754][130385] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 190447616. Throughput: 0: 1824.5, 1: 1834.7. Samples: 47616110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:32:48,754][130385] Avg episode reward: [(0, '73.230'), (1, '95.210')] [2023-10-08 07:32:51,440][00612] Updated weights for policy 1, policy_version 93250 (0.0008) [2023-10-08 07:32:51,694][00611] Updated weights for policy 0, policy_version 92742 (0.0008) [2023-10-08 07:32:51,801][00612] Updated weights for policy 1, policy_version 93260 (0.0007) [2023-10-08 07:32:52,064][00611] Updated weights for policy 0, policy_version 92752 (0.0008) [2023-10-08 07:32:52,166][00612] Updated weights for policy 1, policy_version 93270 (0.0007) [2023-10-08 07:32:52,441][00611] Updated weights for policy 0, policy_version 92762 (0.0007) [2023-10-08 07:32:52,534][00612] Updated weights for policy 1, policy_version 93280 (0.0007) [2023-10-08 07:32:53,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 190513152. Throughput: 0: 1823.1, 1: 1844.0. Samples: 47629052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:32:53,754][130385] Avg episode reward: [(0, '69.340'), (1, '92.570')] [2023-10-08 07:32:56,136][00611] Updated weights for policy 0, policy_version 92772 (0.0009) [2023-10-08 07:32:56,139][00612] Updated weights for policy 1, policy_version 93290 (0.0008) [2023-10-08 07:32:56,503][00612] Updated weights for policy 1, policy_version 93300 (0.0007) [2023-10-08 07:32:56,511][00611] Updated weights for policy 0, policy_version 92782 (0.0009) [2023-10-08 07:32:56,870][00612] Updated weights for policy 1, policy_version 93310 (0.0007) [2023-10-08 07:32:56,882][00611] Updated weights for policy 0, policy_version 92792 (0.0008) [2023-10-08 07:32:58,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 190578688. Throughput: 0: 1819.5, 1: 1831.1. Samples: 47649048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:32:58,755][130385] Avg episode reward: [(0, '71.630'), (1, '90.210')] [2023-10-08 07:33:00,426][00612] Updated weights for policy 1, policy_version 93320 (0.0008) [2023-10-08 07:33:00,473][00611] Updated weights for policy 0, policy_version 92802 (0.0008) [2023-10-08 07:33:00,794][00612] Updated weights for policy 1, policy_version 93330 (0.0008) [2023-10-08 07:33:00,829][00611] Updated weights for policy 0, policy_version 92812 (0.0009) [2023-10-08 07:33:01,162][00612] Updated weights for policy 1, policy_version 93340 (0.0007) [2023-10-08 07:33:01,205][00611] Updated weights for policy 0, policy_version 92822 (0.0008) [2023-10-08 07:33:01,583][00611] Updated weights for policy 0, policy_version 92832 (0.0007) [2023-10-08 07:33:03,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 190644224. Throughput: 0: 1822.8, 1: 1841.2. Samples: 47672164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:33:03,754][130385] Avg episode reward: [(0, '72.170'), (1, '85.510')] [2023-10-08 07:33:04,844][00612] Updated weights for policy 1, policy_version 93350 (0.0009) [2023-10-08 07:33:05,216][00612] Updated weights for policy 1, policy_version 93360 (0.0009) [2023-10-08 07:33:05,292][00611] Updated weights for policy 0, policy_version 92842 (0.0009) [2023-10-08 07:33:05,585][00612] Updated weights for policy 1, policy_version 93370 (0.0008) [2023-10-08 07:33:05,655][00611] Updated weights for policy 0, policy_version 92852 (0.0008) [2023-10-08 07:33:06,025][00611] Updated weights for policy 0, policy_version 92862 (0.0010) [2023-10-08 07:33:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 190709760. Throughput: 0: 1818.9, 1: 1840.0. Samples: 47682164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:33:08,754][130385] Avg episode reward: [(0, '70.590'), (1, '88.830')] [2023-10-08 07:33:09,132][00612] Updated weights for policy 1, policy_version 93380 (0.0009) [2023-10-08 07:33:09,509][00612] Updated weights for policy 1, policy_version 93390 (0.0007) [2023-10-08 07:33:09,625][00611] Updated weights for policy 0, policy_version 92872 (0.0009) [2023-10-08 07:33:09,884][00612] Updated weights for policy 1, policy_version 93400 (0.0007) [2023-10-08 07:33:09,998][00611] Updated weights for policy 0, policy_version 92882 (0.0009) [2023-10-08 07:33:10,378][00611] Updated weights for policy 0, policy_version 92892 (0.0009) [2023-10-08 07:33:13,670][00612] Updated weights for policy 1, policy_version 93410 (0.0010) [2023-10-08 07:33:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 190775296. Throughput: 0: 1824.9, 1: 1846.3. Samples: 47705546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:33:13,754][130385] Avg episode reward: [(0, '73.430'), (1, '88.450')] [2023-10-08 07:33:14,036][00611] Updated weights for policy 0, policy_version 92902 (0.0008) [2023-10-08 07:33:14,040][00612] Updated weights for policy 1, policy_version 93420 (0.0008) [2023-10-08 07:33:14,414][00611] Updated weights for policy 0, policy_version 92912 (0.0009) [2023-10-08 07:33:14,417][00612] Updated weights for policy 1, policy_version 93430 (0.0009) [2023-10-08 07:33:14,796][00611] Updated weights for policy 0, policy_version 92922 (0.0009) [2023-10-08 07:33:14,797][00612] Updated weights for policy 1, policy_version 93440 (0.0008) [2023-10-08 07:33:18,443][00612] Updated weights for policy 1, policy_version 93450 (0.0008) [2023-10-08 07:33:18,750][00611] Updated weights for policy 0, policy_version 92932 (0.0008) [2023-10-08 07:33:18,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 190840832. Throughput: 0: 1821.4, 1: 1841.6. Samples: 47728016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:33:18,754][130385] Avg episode reward: [(0, '70.490'), (1, '88.580')] [2023-10-08 07:33:18,804][00612] Updated weights for policy 1, policy_version 93460 (0.0007) [2023-10-08 07:33:19,133][00611] Updated weights for policy 0, policy_version 92942 (0.0009) [2023-10-08 07:33:19,171][00612] Updated weights for policy 1, policy_version 93470 (0.0008) [2023-10-08 07:33:19,506][00611] Updated weights for policy 0, policy_version 92952 (0.0011) [2023-10-08 07:33:22,908][00612] Updated weights for policy 1, policy_version 93480 (0.0007) [2023-10-08 07:33:23,249][00611] Updated weights for policy 0, policy_version 92962 (0.0010) [2023-10-08 07:33:23,276][00612] Updated weights for policy 1, policy_version 93490 (0.0009) [2023-10-08 07:33:23,625][00611] Updated weights for policy 0, policy_version 92972 (0.0008) [2023-10-08 07:33:23,633][00612] Updated weights for policy 1, policy_version 93500 (0.0008) [2023-10-08 07:33:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 190906368. Throughput: 0: 1819.8, 1: 1847.6. Samples: 47737802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:33:23,754][130385] Avg episode reward: [(0, '72.270'), (1, '88.130')] [2023-10-08 07:33:23,985][00611] Updated weights for policy 0, policy_version 92982 (0.0009) [2023-10-08 07:33:24,359][00611] Updated weights for policy 0, policy_version 92992 (0.0011) [2023-10-08 07:33:27,173][00612] Updated weights for policy 1, policy_version 93510 (0.0008) [2023-10-08 07:33:27,550][00612] Updated weights for policy 1, policy_version 93520 (0.0007) [2023-10-08 07:33:27,918][00612] Updated weights for policy 1, policy_version 93530 (0.0007) [2023-10-08 07:33:28,143][00611] Updated weights for policy 0, policy_version 93002 (0.0010) [2023-10-08 07:33:28,507][00611] Updated weights for policy 0, policy_version 93012 (0.0011) [2023-10-08 07:33:28,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191004672. Throughput: 0: 1811.5, 1: 1834.8. Samples: 47760076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:33:28,755][130385] Avg episode reward: [(0, '70.340'), (1, '89.590')] [2023-10-08 07:33:28,877][00611] Updated weights for policy 0, policy_version 93022 (0.0011) [2023-10-08 07:33:31,635][00612] Updated weights for policy 1, policy_version 93540 (0.0009) [2023-10-08 07:33:32,001][00612] Updated weights for policy 1, policy_version 93550 (0.0010) [2023-10-08 07:33:32,374][00612] Updated weights for policy 1, policy_version 93560 (0.0009) [2023-10-08 07:33:32,501][00611] Updated weights for policy 0, policy_version 93032 (0.0007) [2023-10-08 07:33:32,870][00611] Updated weights for policy 0, policy_version 93042 (0.0008) [2023-10-08 07:33:33,235][00611] Updated weights for policy 0, policy_version 93052 (0.0007) [2023-10-08 07:33:33,754][130385] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 191102976. Throughput: 0: 1818.6, 1: 1838.0. Samples: 47780658. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 07:33:33,755][130385] Avg episode reward: [(0, '72.110'), (1, '87.850')] [2023-10-08 07:33:36,084][00612] Updated weights for policy 1, policy_version 93570 (0.0007) [2023-10-08 07:33:36,448][00612] Updated weights for policy 1, policy_version 93580 (0.0010) [2023-10-08 07:33:36,815][00612] Updated weights for policy 1, policy_version 93590 (0.0008) [2023-10-08 07:33:37,009][00611] Updated weights for policy 0, policy_version 93062 (0.0008) [2023-10-08 07:33:37,188][00612] Updated weights for policy 1, policy_version 93600 (0.0007) [2023-10-08 07:33:37,378][00611] Updated weights for policy 0, policy_version 93072 (0.0007) [2023-10-08 07:33:37,756][00611] Updated weights for policy 0, policy_version 93082 (0.0009) [2023-10-08 07:33:38,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 191168512. Throughput: 0: 1810.7, 1: 1831.1. Samples: 47792932. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 07:33:38,754][130385] Avg episode reward: [(0, '74.790'), (1, '85.210')] [2023-10-08 07:33:40,806][00612] Updated weights for policy 1, policy_version 93610 (0.0009) [2023-10-08 07:33:41,170][00612] Updated weights for policy 1, policy_version 93620 (0.0010) [2023-10-08 07:33:41,358][00611] Updated weights for policy 0, policy_version 93092 (0.0009) [2023-10-08 07:33:41,540][00612] Updated weights for policy 1, policy_version 93630 (0.0009) [2023-10-08 07:33:41,737][00611] Updated weights for policy 0, policy_version 93102 (0.0010) [2023-10-08 07:33:42,110][00611] Updated weights for policy 0, policy_version 93112 (0.0009) [2023-10-08 07:33:43,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191234048. Throughput: 0: 1817.3, 1: 1837.6. Samples: 47813518. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 07:33:43,754][130385] Avg episode reward: [(0, '74.230'), (1, '88.110')] [2023-10-08 07:33:44,955][00612] Updated weights for policy 1, policy_version 93640 (0.0007) [2023-10-08 07:33:45,324][00612] Updated weights for policy 1, policy_version 93650 (0.0011) [2023-10-08 07:33:45,699][00612] Updated weights for policy 1, policy_version 93660 (0.0009) [2023-10-08 07:33:45,733][00611] Updated weights for policy 0, policy_version 93122 (0.0009) [2023-10-08 07:33:46,114][00611] Updated weights for policy 0, policy_version 93132 (0.0009) [2023-10-08 07:33:46,479][00611] Updated weights for policy 0, policy_version 93142 (0.0008) [2023-10-08 07:33:46,851][00611] Updated weights for policy 0, policy_version 93152 (0.0009) [2023-10-08 07:33:48,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 191299584. Throughput: 0: 1808.2, 1: 1843.0. Samples: 47836466. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 07:33:48,755][130385] Avg episode reward: [(0, '73.390'), (1, '89.260')] [2023-10-08 07:33:49,207][00612] Updated weights for policy 1, policy_version 93670 (0.0011) [2023-10-08 07:33:49,582][00612] Updated weights for policy 1, policy_version 93680 (0.0010) [2023-10-08 07:33:49,953][00612] Updated weights for policy 1, policy_version 93690 (0.0007) [2023-10-08 07:33:50,406][00611] Updated weights for policy 0, policy_version 93162 (0.0008) [2023-10-08 07:33:50,784][00611] Updated weights for policy 0, policy_version 93172 (0.0008) [2023-10-08 07:33:51,157][00611] Updated weights for policy 0, policy_version 93182 (0.0011) [2023-10-08 07:33:53,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 191365120. Throughput: 0: 1817.1, 1: 1846.8. Samples: 47847038. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 07:33:53,754][130385] Avg episode reward: [(0, '75.390'), (1, '86.430')] [2023-10-08 07:33:53,767][00612] Updated weights for policy 1, policy_version 93700 (0.0008) [2023-10-08 07:33:54,136][00612] Updated weights for policy 1, policy_version 93710 (0.0008) [2023-10-08 07:33:54,514][00612] Updated weights for policy 1, policy_version 93720 (0.0008) [2023-10-08 07:33:54,810][00611] Updated weights for policy 0, policy_version 93192 (0.0011) [2023-10-08 07:33:55,186][00611] Updated weights for policy 0, policy_version 93202 (0.0011) [2023-10-08 07:33:55,554][00611] Updated weights for policy 0, policy_version 93212 (0.0011) [2023-10-08 07:33:58,082][00612] Updated weights for policy 1, policy_version 93730 (0.0007) [2023-10-08 07:33:58,447][00612] Updated weights for policy 1, policy_version 93740 (0.0007) [2023-10-08 07:33:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 191430656. Throughput: 0: 1816.0, 1: 1834.9. Samples: 47869838. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 07:33:58,755][130385] Avg episode reward: [(0, '75.760'), (1, '87.560')] [2023-10-08 07:33:58,808][00612] Updated weights for policy 1, policy_version 93750 (0.0008) [2023-10-08 07:33:59,175][00612] Updated weights for policy 1, policy_version 93760 (0.0008) [2023-10-08 07:33:59,216][00611] Updated weights for policy 0, policy_version 93222 (0.0009) [2023-10-08 07:33:59,593][00611] Updated weights for policy 0, policy_version 93232 (0.0008) [2023-10-08 07:33:59,960][00611] Updated weights for policy 0, policy_version 93242 (0.0010) [2023-10-08 07:34:02,999][00612] Updated weights for policy 1, policy_version 93770 (0.0008) [2023-10-08 07:34:03,361][00612] Updated weights for policy 1, policy_version 93780 (0.0009) [2023-10-08 07:34:03,724][00612] Updated weights for policy 1, policy_version 93790 (0.0007) [2023-10-08 07:34:03,727][00611] Updated weights for policy 0, policy_version 93252 (0.0010) [2023-10-08 07:34:03,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 191496192. Throughput: 0: 1825.2, 1: 1829.1. Samples: 47892460. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 07:34:03,755][130385] Avg episode reward: [(0, '77.130'), (1, '89.820')] [2023-10-08 07:34:04,119][00611] Updated weights for policy 0, policy_version 93262 (0.0010) [2023-10-08 07:34:04,505][00611] Updated weights for policy 0, policy_version 93272 (0.0011) [2023-10-08 07:34:07,263][00612] Updated weights for policy 1, policy_version 93800 (0.0007) [2023-10-08 07:34:07,623][00612] Updated weights for policy 1, policy_version 93810 (0.0007) [2023-10-08 07:34:07,983][00612] Updated weights for policy 1, policy_version 93820 (0.0007) [2023-10-08 07:34:08,080][00611] Updated weights for policy 0, policy_version 93282 (0.0008) [2023-10-08 07:34:08,454][00611] Updated weights for policy 0, policy_version 93292 (0.0010) [2023-10-08 07:34:08,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191594496. Throughput: 0: 1826.9, 1: 1846.3. Samples: 47903096. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 07:34:08,754][130385] Avg episode reward: [(0, '75.670'), (1, '83.300')] [2023-10-08 07:34:08,816][00611] Updated weights for policy 0, policy_version 93302 (0.0009) [2023-10-08 07:34:09,181][00611] Updated weights for policy 0, policy_version 93312 (0.0011) [2023-10-08 07:34:11,812][00612] Updated weights for policy 1, policy_version 93830 (0.0008) [2023-10-08 07:34:12,193][00612] Updated weights for policy 1, policy_version 93840 (0.0007) [2023-10-08 07:34:12,562][00612] Updated weights for policy 1, policy_version 93850 (0.0007) [2023-10-08 07:34:13,015][00611] Updated weights for policy 0, policy_version 93322 (0.0007) [2023-10-08 07:34:13,387][00611] Updated weights for policy 0, policy_version 93332 (0.0009) [2023-10-08 07:34:13,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191660032. Throughput: 0: 1836.3, 1: 1842.4. Samples: 47925616. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 07:34:13,755][130385] Avg episode reward: [(0, '79.180'), (1, '82.600')] [2023-10-08 07:34:13,763][00611] Updated weights for policy 0, policy_version 93342 (0.0008) [2023-10-08 07:34:16,064][00612] Updated weights for policy 1, policy_version 93860 (0.0007) [2023-10-08 07:34:16,436][00612] Updated weights for policy 1, policy_version 93870 (0.0008) [2023-10-08 07:34:16,803][00612] Updated weights for policy 1, policy_version 93880 (0.0008) [2023-10-08 07:34:17,337][00611] Updated weights for policy 0, policy_version 93352 (0.0010) [2023-10-08 07:34:17,713][00611] Updated weights for policy 0, policy_version 93362 (0.0008) [2023-10-08 07:34:18,076][00611] Updated weights for policy 0, policy_version 93372 (0.0009) [2023-10-08 07:34:18,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 191758336. Throughput: 0: 1831.6, 1: 1856.0. Samples: 47946604. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 07:34:18,755][130385] Avg episode reward: [(0, '85.080'), (1, '89.950')] [2023-10-08 07:34:18,765][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000093888_96141312.pth... [2023-10-08 07:34:18,766][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000093376_95617024.pth... [2023-10-08 07:34:18,802][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000092160_94371840.pth [2023-10-08 07:34:18,810][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000091648_93847552.pth [2023-10-08 07:34:20,411][00612] Updated weights for policy 1, policy_version 93890 (0.0008) [2023-10-08 07:34:20,784][00612] Updated weights for policy 1, policy_version 93900 (0.0011) [2023-10-08 07:34:21,151][00612] Updated weights for policy 1, policy_version 93910 (0.0010) [2023-10-08 07:34:21,510][00612] Updated weights for policy 1, policy_version 93920 (0.0010) [2023-10-08 07:34:21,700][00611] Updated weights for policy 0, policy_version 93382 (0.0008) [2023-10-08 07:34:22,064][00611] Updated weights for policy 0, policy_version 93392 (0.0007) [2023-10-08 07:34:22,422][00611] Updated weights for policy 0, policy_version 93402 (0.0007) [2023-10-08 07:34:23,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 191823872. Throughput: 0: 1837.6, 1: 1838.8. Samples: 47958368. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 07:34:23,754][130385] Avg episode reward: [(0, '82.860'), (1, '89.460')] [2023-10-08 07:34:25,290][00612] Updated weights for policy 1, policy_version 93930 (0.0009) [2023-10-08 07:34:25,651][00612] Updated weights for policy 1, policy_version 93940 (0.0008) [2023-10-08 07:34:26,005][00611] Updated weights for policy 0, policy_version 93412 (0.0008) [2023-10-08 07:34:26,022][00612] Updated weights for policy 1, policy_version 93950 (0.0007) [2023-10-08 07:34:26,381][00611] Updated weights for policy 0, policy_version 93422 (0.0009) [2023-10-08 07:34:26,740][00611] Updated weights for policy 0, policy_version 93432 (0.0007) [2023-10-08 07:34:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191889408. Throughput: 0: 1833.1, 1: 1861.1. Samples: 47979756. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 07:34:28,755][130385] Avg episode reward: [(0, '83.290'), (1, '97.010')] [2023-10-08 07:34:29,429][00612] Updated weights for policy 1, policy_version 93960 (0.0008) [2023-10-08 07:34:29,794][00612] Updated weights for policy 1, policy_version 93970 (0.0008) [2023-10-08 07:34:30,159][00612] Updated weights for policy 1, policy_version 93980 (0.0009) [2023-10-08 07:34:30,375][00611] Updated weights for policy 0, policy_version 93442 (0.0008) [2023-10-08 07:34:30,756][00611] Updated weights for policy 0, policy_version 93452 (0.0007) [2023-10-08 07:34:31,122][00611] Updated weights for policy 0, policy_version 93462 (0.0011) [2023-10-08 07:34:31,488][00611] Updated weights for policy 0, policy_version 93472 (0.0008) [2023-10-08 07:34:33,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 191954944. Throughput: 0: 1842.6, 1: 1860.1. Samples: 48003086. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 07:34:33,755][130385] Avg episode reward: [(0, '85.260'), (1, '93.560')] [2023-10-08 07:34:33,816][00612] Updated weights for policy 1, policy_version 93990 (0.0007) [2023-10-08 07:34:34,191][00612] Updated weights for policy 1, policy_version 94000 (0.0008) [2023-10-08 07:34:34,556][00612] Updated weights for policy 1, policy_version 94010 (0.0009) [2023-10-08 07:34:35,161][00611] Updated weights for policy 0, policy_version 93482 (0.0009) [2023-10-08 07:34:35,536][00611] Updated weights for policy 0, policy_version 93492 (0.0010) [2023-10-08 07:34:35,908][00611] Updated weights for policy 0, policy_version 93502 (0.0008) [2023-10-08 07:34:38,088][00612] Updated weights for policy 1, policy_version 94020 (0.0008) [2023-10-08 07:34:38,450][00612] Updated weights for policy 1, policy_version 94030 (0.0011) [2023-10-08 07:34:38,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 192020480. Throughput: 0: 1834.4, 1: 1854.9. Samples: 48013060. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 07:34:38,754][130385] Avg episode reward: [(0, '80.000'), (1, '98.320')] [2023-10-08 07:34:38,833][00612] Updated weights for policy 1, policy_version 94040 (0.0010) [2023-10-08 07:34:39,399][00611] Updated weights for policy 0, policy_version 93512 (0.0007) [2023-10-08 07:34:39,770][00611] Updated weights for policy 0, policy_version 93522 (0.0008) [2023-10-08 07:34:40,145][00611] Updated weights for policy 0, policy_version 93532 (0.0009) [2023-10-08 07:34:42,389][00612] Updated weights for policy 1, policy_version 94050 (0.0008) [2023-10-08 07:34:42,758][00612] Updated weights for policy 1, policy_version 94060 (0.0007) [2023-10-08 07:34:43,114][00612] Updated weights for policy 1, policy_version 94070 (0.0009) [2023-10-08 07:34:43,481][00612] Updated weights for policy 1, policy_version 94080 (0.0007) [2023-10-08 07:34:43,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192118784. Throughput: 0: 1837.0, 1: 1866.0. Samples: 48036472. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 07:34:43,754][130385] Avg episode reward: [(0, '83.610'), (1, '96.590')] [2023-10-08 07:34:43,803][00611] Updated weights for policy 0, policy_version 93542 (0.0009) [2023-10-08 07:34:44,175][00611] Updated weights for policy 0, policy_version 93552 (0.0009) [2023-10-08 07:34:44,533][00611] Updated weights for policy 0, policy_version 93562 (0.0008) [2023-10-08 07:34:47,024][00612] Updated weights for policy 1, policy_version 94090 (0.0007) [2023-10-08 07:34:47,404][00612] Updated weights for policy 1, policy_version 94100 (0.0008) [2023-10-08 07:34:47,767][00612] Updated weights for policy 1, policy_version 94110 (0.0011) [2023-10-08 07:34:48,339][00611] Updated weights for policy 0, policy_version 93572 (0.0007) [2023-10-08 07:34:48,722][00611] Updated weights for policy 0, policy_version 93582 (0.0009) [2023-10-08 07:34:48,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192184320. Throughput: 0: 1835.3, 1: 1845.1. Samples: 48058078. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 07:34:48,755][130385] Avg episode reward: [(0, '81.300'), (1, '94.070')] [2023-10-08 07:34:49,092][00611] Updated weights for policy 0, policy_version 93592 (0.0011) [2023-10-08 07:34:51,369][00612] Updated weights for policy 1, policy_version 94120 (0.0009) [2023-10-08 07:34:51,743][00612] Updated weights for policy 1, policy_version 94130 (0.0011) [2023-10-08 07:34:52,104][00612] Updated weights for policy 1, policy_version 94140 (0.0010) [2023-10-08 07:34:52,669][00611] Updated weights for policy 0, policy_version 93602 (0.0009) [2023-10-08 07:34:53,035][00611] Updated weights for policy 0, policy_version 93612 (0.0009) [2023-10-08 07:34:53,404][00611] Updated weights for policy 0, policy_version 93622 (0.0008) [2023-10-08 07:34:53,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 192249856. Throughput: 0: 1836.7, 1: 1863.1. Samples: 48069586. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 07:34:53,755][130385] Avg episode reward: [(0, '83.330'), (1, '94.830')] [2023-10-08 07:34:53,779][00611] Updated weights for policy 0, policy_version 93632 (0.0008) [2023-10-08 07:34:55,821][00612] Updated weights for policy 1, policy_version 94150 (0.0008) [2023-10-08 07:34:56,181][00612] Updated weights for policy 1, policy_version 94160 (0.0007) [2023-10-08 07:34:56,548][00612] Updated weights for policy 1, policy_version 94170 (0.0011) [2023-10-08 07:34:57,520][00611] Updated weights for policy 0, policy_version 93642 (0.0007) [2023-10-08 07:34:57,899][00611] Updated weights for policy 0, policy_version 93652 (0.0009) [2023-10-08 07:34:58,263][00611] Updated weights for policy 0, policy_version 93662 (0.0009) [2023-10-08 07:34:58,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 192348160. Throughput: 0: 1833.1, 1: 1848.0. Samples: 48091270. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 07:34:58,755][130385] Avg episode reward: [(0, '78.180'), (1, '90.790')] [2023-10-08 07:35:00,217][00612] Updated weights for policy 1, policy_version 94180 (0.0010) [2023-10-08 07:35:00,605][00612] Updated weights for policy 1, policy_version 94190 (0.0008) [2023-10-08 07:35:00,977][00612] Updated weights for policy 1, policy_version 94200 (0.0009) [2023-10-08 07:35:01,972][00611] Updated weights for policy 0, policy_version 93672 (0.0010) [2023-10-08 07:35:02,347][00611] Updated weights for policy 0, policy_version 93682 (0.0010) [2023-10-08 07:35:02,712][00611] Updated weights for policy 0, policy_version 93692 (0.0011) [2023-10-08 07:35:03,754][130385] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 192413696. Throughput: 0: 1827.3, 1: 1863.2. Samples: 48112676. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 07:35:03,754][130385] Avg episode reward: [(0, '78.410'), (1, '93.150')] [2023-10-08 07:35:04,470][00612] Updated weights for policy 1, policy_version 94210 (0.0008) [2023-10-08 07:35:04,836][00612] Updated weights for policy 1, policy_version 94220 (0.0007) [2023-10-08 07:35:05,205][00612] Updated weights for policy 1, policy_version 94230 (0.0008) [2023-10-08 07:35:05,572][00612] Updated weights for policy 1, policy_version 94240 (0.0008) [2023-10-08 07:35:06,393][00611] Updated weights for policy 0, policy_version 93702 (0.0010) [2023-10-08 07:35:06,766][00611] Updated weights for policy 0, policy_version 93712 (0.0010) [2023-10-08 07:35:07,135][00611] Updated weights for policy 0, policy_version 93722 (0.0008) [2023-10-08 07:35:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192479232. Throughput: 0: 1830.5, 1: 1854.2. Samples: 48124182. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 07:35:08,754][130385] Avg episode reward: [(0, '74.100'), (1, '91.900')] [2023-10-08 07:35:09,249][00612] Updated weights for policy 1, policy_version 94250 (0.0008) [2023-10-08 07:35:09,629][00612] Updated weights for policy 1, policy_version 94260 (0.0007) [2023-10-08 07:35:09,996][00612] Updated weights for policy 1, policy_version 94270 (0.0007) [2023-10-08 07:35:10,840][00611] Updated weights for policy 0, policy_version 93732 (0.0008) [2023-10-08 07:35:11,212][00611] Updated weights for policy 0, policy_version 93742 (0.0008) [2023-10-08 07:35:11,586][00611] Updated weights for policy 0, policy_version 93752 (0.0010) [2023-10-08 07:35:13,578][00612] Updated weights for policy 1, policy_version 94280 (0.0009) [2023-10-08 07:35:13,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192544768. Throughput: 0: 1826.7, 1: 1866.8. Samples: 48145962. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 07:35:13,754][130385] Avg episode reward: [(0, '74.110'), (1, '90.460')] [2023-10-08 07:35:13,941][00612] Updated weights for policy 1, policy_version 94290 (0.0007) [2023-10-08 07:35:14,307][00612] Updated weights for policy 1, policy_version 94300 (0.0008) [2023-10-08 07:35:15,154][00611] Updated weights for policy 0, policy_version 93762 (0.0008) [2023-10-08 07:35:15,517][00611] Updated weights for policy 0, policy_version 93772 (0.0008) [2023-10-08 07:35:15,887][00611] Updated weights for policy 0, policy_version 93782 (0.0008) [2023-10-08 07:35:16,257][00611] Updated weights for policy 0, policy_version 93792 (0.0008) [2023-10-08 07:35:17,741][00612] Updated weights for policy 1, policy_version 94310 (0.0007) [2023-10-08 07:35:18,107][00612] Updated weights for policy 1, policy_version 94320 (0.0007) [2023-10-08 07:35:18,475][00612] Updated weights for policy 1, policy_version 94330 (0.0010) [2023-10-08 07:35:18,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 192643072. Throughput: 0: 1832.1, 1: 1852.0. Samples: 48168872. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 07:35:18,755][130385] Avg episode reward: [(0, '71.540'), (1, '88.380')] [2023-10-08 07:35:19,986][00611] Updated weights for policy 0, policy_version 93802 (0.0009) [2023-10-08 07:35:20,343][00611] Updated weights for policy 0, policy_version 93812 (0.0007) [2023-10-08 07:35:20,715][00611] Updated weights for policy 0, policy_version 93822 (0.0010) [2023-10-08 07:35:21,967][00612] Updated weights for policy 1, policy_version 94340 (0.0011) [2023-10-08 07:35:22,333][00612] Updated weights for policy 1, policy_version 94350 (0.0008) [2023-10-08 07:35:22,694][00612] Updated weights for policy 1, policy_version 94360 (0.0009) [2023-10-08 07:35:23,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 192708608. Throughput: 0: 1829.1, 1: 1876.1. Samples: 48179796. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 07:35:23,755][130385] Avg episode reward: [(0, '69.310'), (1, '88.300')] [2023-10-08 07:35:24,381][00611] Updated weights for policy 0, policy_version 93832 (0.0010) [2023-10-08 07:35:24,747][00611] Updated weights for policy 0, policy_version 93842 (0.0007) [2023-10-08 07:35:25,111][00611] Updated weights for policy 0, policy_version 93852 (0.0007) [2023-10-08 07:35:26,423][00612] Updated weights for policy 1, policy_version 94370 (0.0008) [2023-10-08 07:35:26,793][00612] Updated weights for policy 1, policy_version 94380 (0.0007) [2023-10-08 07:35:27,161][00612] Updated weights for policy 1, policy_version 94390 (0.0010) [2023-10-08 07:35:27,529][00612] Updated weights for policy 1, policy_version 94400 (0.0010) [2023-10-08 07:35:28,622][00611] Updated weights for policy 0, policy_version 93862 (0.0010) [2023-10-08 07:35:28,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192774144. Throughput: 0: 1833.8, 1: 1841.8. Samples: 48201876. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-08 07:35:28,755][130385] Avg episode reward: [(0, '70.620'), (1, '80.370')] [2023-10-08 07:35:28,997][00611] Updated weights for policy 0, policy_version 93872 (0.0009) [2023-10-08 07:35:29,369][00611] Updated weights for policy 0, policy_version 93882 (0.0010) [2023-10-08 07:35:31,218][00612] Updated weights for policy 1, policy_version 94410 (0.0008) [2023-10-08 07:35:31,592][00612] Updated weights for policy 1, policy_version 94420 (0.0008) [2023-10-08 07:35:31,951][00612] Updated weights for policy 1, policy_version 94430 (0.0007) [2023-10-08 07:35:33,062][00611] Updated weights for policy 0, policy_version 93892 (0.0009) [2023-10-08 07:35:33,453][00611] Updated weights for policy 0, policy_version 93902 (0.0007) [2023-10-08 07:35:33,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 192839680. Throughput: 0: 1825.8, 1: 1862.9. Samples: 48224068. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-08 07:35:33,754][130385] Avg episode reward: [(0, '69.740'), (1, '81.140')] [2023-10-08 07:35:33,827][00611] Updated weights for policy 0, policy_version 93912 (0.0008) [2023-10-08 07:35:35,604][00612] Updated weights for policy 1, policy_version 94440 (0.0007) [2023-10-08 07:35:35,970][00612] Updated weights for policy 1, policy_version 94450 (0.0008) [2023-10-08 07:35:36,328][00612] Updated weights for policy 1, policy_version 94460 (0.0010) [2023-10-08 07:35:37,584][00611] Updated weights for policy 0, policy_version 93922 (0.0008) [2023-10-08 07:35:37,966][00611] Updated weights for policy 0, policy_version 93932 (0.0008) [2023-10-08 07:35:38,329][00611] Updated weights for policy 0, policy_version 93942 (0.0010) [2023-10-08 07:35:38,701][00611] Updated weights for policy 0, policy_version 93952 (0.0011) [2023-10-08 07:35:38,754][130385] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 192937984. Throughput: 0: 1831.1, 1: 1836.6. Samples: 48234632. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-08 07:35:38,755][130385] Avg episode reward: [(0, '69.930'), (1, '79.760')] [2023-10-08 07:35:39,893][00612] Updated weights for policy 1, policy_version 94470 (0.0010) [2023-10-08 07:35:40,253][00612] Updated weights for policy 1, policy_version 94480 (0.0008) [2023-10-08 07:35:40,619][00612] Updated weights for policy 1, policy_version 94490 (0.0009) [2023-10-08 07:35:42,473][00611] Updated weights for policy 0, policy_version 93962 (0.0010) [2023-10-08 07:35:42,852][00611] Updated weights for policy 0, policy_version 93972 (0.0009) [2023-10-08 07:35:43,225][00611] Updated weights for policy 0, policy_version 93982 (0.0010) [2023-10-08 07:35:43,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193003520. Throughput: 0: 1824.5, 1: 1862.7. Samples: 48257194. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-08 07:35:43,754][130385] Avg episode reward: [(0, '72.770'), (1, '80.710')] [2023-10-08 07:35:44,134][00612] Updated weights for policy 1, policy_version 94500 (0.0008) [2023-10-08 07:35:44,516][00612] Updated weights for policy 1, policy_version 94510 (0.0011) [2023-10-08 07:35:44,873][00612] Updated weights for policy 1, policy_version 94520 (0.0011) [2023-10-08 07:35:46,879][00611] Updated weights for policy 0, policy_version 93992 (0.0008) [2023-10-08 07:35:47,245][00611] Updated weights for policy 0, policy_version 94002 (0.0008) [2023-10-08 07:35:47,616][00611] Updated weights for policy 0, policy_version 94012 (0.0009) [2023-10-08 07:35:48,595][00612] Updated weights for policy 1, policy_version 94530 (0.0010) [2023-10-08 07:35:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193069056. Throughput: 0: 1830.8, 1: 1864.5. Samples: 48278964. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-08 07:35:48,754][130385] Avg episode reward: [(0, '70.030'), (1, '76.000')] [2023-10-08 07:35:48,998][00612] Updated weights for policy 1, policy_version 94540 (0.0008) [2023-10-08 07:35:49,375][00612] Updated weights for policy 1, policy_version 94550 (0.0007) [2023-10-08 07:35:49,739][00612] Updated weights for policy 1, policy_version 94560 (0.0007) [2023-10-08 07:35:51,327][00611] Updated weights for policy 0, policy_version 94022 (0.0008) [2023-10-08 07:35:51,695][00611] Updated weights for policy 0, policy_version 94032 (0.0009) [2023-10-08 07:35:52,066][00611] Updated weights for policy 0, policy_version 94042 (0.0009) [2023-10-08 07:35:53,308][00612] Updated weights for policy 1, policy_version 94570 (0.0008) [2023-10-08 07:35:53,666][00612] Updated weights for policy 1, policy_version 94580 (0.0007) [2023-10-08 07:35:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 193134592. Throughput: 0: 1830.5, 1: 1861.3. Samples: 48290314. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-08 07:35:53,754][130385] Avg episode reward: [(0, '70.500'), (1, '79.880')] [2023-10-08 07:35:54,035][00612] Updated weights for policy 1, policy_version 94590 (0.0009) [2023-10-08 07:35:55,797][00611] Updated weights for policy 0, policy_version 94052 (0.0011) [2023-10-08 07:35:56,154][00611] Updated weights for policy 0, policy_version 94062 (0.0011) [2023-10-08 07:35:56,537][00611] Updated weights for policy 0, policy_version 94072 (0.0009) [2023-10-08 07:35:57,726][00612] Updated weights for policy 1, policy_version 94600 (0.0010) [2023-10-08 07:35:58,090][00612] Updated weights for policy 1, policy_version 94610 (0.0010) [2023-10-08 07:35:58,470][00612] Updated weights for policy 1, policy_version 94620 (0.0011) [2023-10-08 07:35:58,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 193232896. Throughput: 0: 1831.6, 1: 1858.9. Samples: 48312032. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-08 07:35:58,754][130385] Avg episode reward: [(0, '75.310'), (1, '78.420')] [2023-10-08 07:36:00,193][00611] Updated weights for policy 0, policy_version 94082 (0.0008) [2023-10-08 07:36:00,562][00611] Updated weights for policy 0, policy_version 94092 (0.0009) [2023-10-08 07:36:00,930][00611] Updated weights for policy 0, policy_version 94102 (0.0008) [2023-10-08 07:36:01,306][00611] Updated weights for policy 0, policy_version 94112 (0.0009) [2023-10-08 07:36:02,108][00612] Updated weights for policy 1, policy_version 94630 (0.0008) [2023-10-08 07:36:02,478][00612] Updated weights for policy 1, policy_version 94640 (0.0007) [2023-10-08 07:36:02,845][00612] Updated weights for policy 1, policy_version 94650 (0.0010) [2023-10-08 07:36:03,754][130385] Fps is (10 sec: 16383.3, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 193298432. Throughput: 0: 1824.4, 1: 1832.8. Samples: 48333450. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-08 07:36:03,756][130385] Avg episode reward: [(0, '73.830'), (1, '78.020')] [2023-10-08 07:36:04,743][00611] Updated weights for policy 0, policy_version 94122 (0.0007) [2023-10-08 07:36:05,116][00611] Updated weights for policy 0, policy_version 94132 (0.0008) [2023-10-08 07:36:05,478][00611] Updated weights for policy 0, policy_version 94142 (0.0009) [2023-10-08 07:36:06,461][00612] Updated weights for policy 1, policy_version 94660 (0.0010) [2023-10-08 07:36:06,836][00612] Updated weights for policy 1, policy_version 94670 (0.0009) [2023-10-08 07:36:07,206][00612] Updated weights for policy 1, policy_version 94680 (0.0009) [2023-10-08 07:36:08,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 193363968. Throughput: 0: 1829.9, 1: 1839.9. Samples: 48344938. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-08 07:36:08,755][130385] Avg episode reward: [(0, '74.180'), (1, '75.660')] [2023-10-08 07:36:09,086][00611] Updated weights for policy 0, policy_version 94152 (0.0010) [2023-10-08 07:36:09,448][00611] Updated weights for policy 0, policy_version 94162 (0.0010) [2023-10-08 07:36:09,820][00611] Updated weights for policy 0, policy_version 94172 (0.0010) [2023-10-08 07:36:10,764][00612] Updated weights for policy 1, policy_version 94690 (0.0009) [2023-10-08 07:36:11,127][00612] Updated weights for policy 1, policy_version 94700 (0.0007) [2023-10-08 07:36:11,491][00612] Updated weights for policy 1, policy_version 94710 (0.0009) [2023-10-08 07:36:11,865][00612] Updated weights for policy 1, policy_version 94720 (0.0012) [2023-10-08 07:36:13,547][00611] Updated weights for policy 0, policy_version 94182 (0.0008) [2023-10-08 07:36:13,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193429504. Throughput: 0: 1828.8, 1: 1834.3. Samples: 48366712. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-08 07:36:13,755][130385] Avg episode reward: [(0, '73.130'), (1, '73.930')] [2023-10-08 07:36:13,927][00611] Updated weights for policy 0, policy_version 94192 (0.0009) [2023-10-08 07:36:14,291][00611] Updated weights for policy 0, policy_version 94202 (0.0008) [2023-10-08 07:36:15,592][00612] Updated weights for policy 1, policy_version 94730 (0.0010) [2023-10-08 07:36:15,954][00612] Updated weights for policy 1, policy_version 94740 (0.0009) [2023-10-08 07:36:16,330][00612] Updated weights for policy 1, policy_version 94750 (0.0008) [2023-10-08 07:36:17,708][00611] Updated weights for policy 0, policy_version 94212 (0.0007) [2023-10-08 07:36:18,068][00611] Updated weights for policy 0, policy_version 94222 (0.0007) [2023-10-08 07:36:18,438][00611] Updated weights for policy 0, policy_version 94232 (0.0007) [2023-10-08 07:36:18,754][130385] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193527808. Throughput: 0: 1819.8, 1: 1846.4. Samples: 48389052. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-08 07:36:18,756][130385] Avg episode reward: [(0, '73.500'), (1, '78.230')] [2023-10-08 07:36:18,768][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000094240_96501760.pth... [2023-10-08 07:36:18,768][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000094752_97026048.pth... [2023-10-08 07:36:18,808][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000092512_94732288.pth [2023-10-08 07:36:18,812][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000093024_95256576.pth [2023-10-08 07:36:18,812][00365] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p0/milestones/checkpoint_000094240_96501760.pth [2023-10-08 07:36:18,816][00425] Saving a milestone ./train_atari/atari_assault_APPO/checkpoint_p1/milestones/checkpoint_000094752_97026048.pth [2023-10-08 07:36:19,980][00612] Updated weights for policy 1, policy_version 94760 (0.0007) [2023-10-08 07:36:20,357][00612] Updated weights for policy 1, policy_version 94770 (0.0009) [2023-10-08 07:36:20,720][00612] Updated weights for policy 1, policy_version 94780 (0.0008) [2023-10-08 07:36:22,266][00611] Updated weights for policy 0, policy_version 94242 (0.0009) [2023-10-08 07:36:22,662][00611] Updated weights for policy 0, policy_version 94252 (0.0009) [2023-10-08 07:36:23,039][00611] Updated weights for policy 0, policy_version 94262 (0.0008) [2023-10-08 07:36:23,402][00611] Updated weights for policy 0, policy_version 94272 (0.0009) [2023-10-08 07:36:23,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193593344. Throughput: 0: 1832.4, 1: 1837.6. Samples: 48399780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:36:23,754][130385] Avg episode reward: [(0, '79.450'), (1, '78.530')] [2023-10-08 07:36:24,561][00612] Updated weights for policy 1, policy_version 94790 (0.0008) [2023-10-08 07:36:24,938][00612] Updated weights for policy 1, policy_version 94800 (0.0009) [2023-10-08 07:36:25,296][00612] Updated weights for policy 1, policy_version 94810 (0.0009) [2023-10-08 07:36:27,021][00611] Updated weights for policy 0, policy_version 94282 (0.0007) [2023-10-08 07:36:27,396][00611] Updated weights for policy 0, policy_version 94292 (0.0008) [2023-10-08 07:36:27,776][00611] Updated weights for policy 0, policy_version 94302 (0.0009) [2023-10-08 07:36:28,754][130385] Fps is (10 sec: 13108.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 193658880. Throughput: 0: 1828.4, 1: 1834.5. Samples: 48422026. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:36:28,754][130385] Avg episode reward: [(0, '80.680'), (1, '78.790')] [2023-10-08 07:36:28,784][00612] Updated weights for policy 1, policy_version 94820 (0.0009) [2023-10-08 07:36:29,161][00612] Updated weights for policy 1, policy_version 94830 (0.0010) [2023-10-08 07:36:29,521][00612] Updated weights for policy 1, policy_version 94840 (0.0009) [2023-10-08 07:36:31,403][00611] Updated weights for policy 0, policy_version 94312 (0.0009) [2023-10-08 07:36:31,774][00611] Updated weights for policy 0, policy_version 94322 (0.0008) [2023-10-08 07:36:32,145][00611] Updated weights for policy 0, policy_version 94332 (0.0008) [2023-10-08 07:36:33,199][00612] Updated weights for policy 1, policy_version 94850 (0.0007) [2023-10-08 07:36:33,571][00612] Updated weights for policy 1, policy_version 94860 (0.0010) [2023-10-08 07:36:33,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 193724416. Throughput: 0: 1842.9, 1: 1833.5. Samples: 48444402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:36:33,755][130385] Avg episode reward: [(0, '81.620'), (1, '82.210')] [2023-10-08 07:36:33,932][00612] Updated weights for policy 1, policy_version 94870 (0.0009) [2023-10-08 07:36:34,303][00612] Updated weights for policy 1, policy_version 94880 (0.0009) [2023-10-08 07:36:35,755][00611] Updated weights for policy 0, policy_version 94342 (0.0008) [2023-10-08 07:36:36,133][00611] Updated weights for policy 0, policy_version 94352 (0.0007) [2023-10-08 07:36:36,500][00611] Updated weights for policy 0, policy_version 94362 (0.0007) [2023-10-08 07:36:37,961][00612] Updated weights for policy 1, policy_version 94890 (0.0009) [2023-10-08 07:36:38,330][00612] Updated weights for policy 1, policy_version 94900 (0.0008) [2023-10-08 07:36:38,699][00612] Updated weights for policy 1, policy_version 94910 (0.0007) [2023-10-08 07:36:38,754][130385] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 193789952. Throughput: 0: 1832.8, 1: 1841.0. Samples: 48455634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:36:38,755][130385] Avg episode reward: [(0, '78.960'), (1, '82.370')] [2023-10-08 07:36:39,989][00611] Updated weights for policy 0, policy_version 94372 (0.0007) [2023-10-08 07:36:40,354][00611] Updated weights for policy 0, policy_version 94382 (0.0010) [2023-10-08 07:36:40,724][00611] Updated weights for policy 0, policy_version 94392 (0.0007) [2023-10-08 07:36:42,373][00612] Updated weights for policy 1, policy_version 94920 (0.0007) [2023-10-08 07:36:42,737][00612] Updated weights for policy 1, policy_version 94930 (0.0011) [2023-10-08 07:36:43,117][00612] Updated weights for policy 1, policy_version 94940 (0.0010) [2023-10-08 07:36:43,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 193888256. Throughput: 0: 1856.4, 1: 1832.4. Samples: 48478026. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:36:43,754][130385] Avg episode reward: [(0, '81.300'), (1, '79.280')] [2023-10-08 07:36:44,311][00611] Updated weights for policy 0, policy_version 94402 (0.0007) [2023-10-08 07:36:44,690][00611] Updated weights for policy 0, policy_version 94412 (0.0007) [2023-10-08 07:36:45,075][00611] Updated weights for policy 0, policy_version 94422 (0.0007) [2023-10-08 07:36:45,441][00611] Updated weights for policy 0, policy_version 94432 (0.0008) [2023-10-08 07:36:46,799][00612] Updated weights for policy 1, policy_version 94950 (0.0011) [2023-10-08 07:36:47,172][00612] Updated weights for policy 1, policy_version 94960 (0.0010) [2023-10-08 07:36:47,536][00612] Updated weights for policy 1, policy_version 94970 (0.0007) [2023-10-08 07:36:48,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 193953792. Throughput: 0: 1863.0, 1: 1835.2. Samples: 48499870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:36:48,754][130385] Avg episode reward: [(0, '79.020'), (1, '79.530')] [2023-10-08 07:36:48,982][00611] Updated weights for policy 0, policy_version 94442 (0.0009) [2023-10-08 07:36:49,363][00611] Updated weights for policy 0, policy_version 94452 (0.0009) [2023-10-08 07:36:49,732][00611] Updated weights for policy 0, policy_version 94462 (0.0009) [2023-10-08 07:36:51,037][00612] Updated weights for policy 1, policy_version 94980 (0.0009) [2023-10-08 07:36:51,402][00612] Updated weights for policy 1, policy_version 94990 (0.0010) [2023-10-08 07:36:51,772][00612] Updated weights for policy 1, policy_version 95000 (0.0011) [2023-10-08 07:36:53,503][00611] Updated weights for policy 0, policy_version 94472 (0.0008) [2023-10-08 07:36:53,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 194019328. Throughput: 0: 1855.2, 1: 1833.2. Samples: 48510916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:36:53,754][130385] Avg episode reward: [(0, '78.510'), (1, '78.170')] [2023-10-08 07:36:53,876][00611] Updated weights for policy 0, policy_version 94482 (0.0010) [2023-10-08 07:36:54,252][00611] Updated weights for policy 0, policy_version 94492 (0.0009) [2023-10-08 07:36:55,373][00612] Updated weights for policy 1, policy_version 95010 (0.0009) [2023-10-08 07:36:55,753][00612] Updated weights for policy 1, policy_version 95020 (0.0008) [2023-10-08 07:36:56,118][00612] Updated weights for policy 1, policy_version 95030 (0.0010) [2023-10-08 07:36:56,492][00612] Updated weights for policy 1, policy_version 95040 (0.0007) [2023-10-08 07:36:57,800][00611] Updated weights for policy 0, policy_version 94502 (0.0007) [2023-10-08 07:36:58,174][00611] Updated weights for policy 0, policy_version 94512 (0.0007) [2023-10-08 07:36:58,540][00611] Updated weights for policy 0, policy_version 94522 (0.0008) [2023-10-08 07:36:58,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 194084864. Throughput: 0: 1856.3, 1: 1839.8. Samples: 48533034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:36:58,754][130385] Avg episode reward: [(0, '76.660'), (1, '83.760')] [2023-10-08 07:37:00,151][00612] Updated weights for policy 1, policy_version 95050 (0.0008) [2023-10-08 07:37:00,522][00612] Updated weights for policy 1, policy_version 95060 (0.0010) [2023-10-08 07:37:00,893][00612] Updated weights for policy 1, policy_version 95070 (0.0009) [2023-10-08 07:37:02,155][00611] Updated weights for policy 0, policy_version 94532 (0.0007) [2023-10-08 07:37:02,527][00611] Updated weights for policy 0, policy_version 94542 (0.0007) [2023-10-08 07:37:02,900][00611] Updated weights for policy 0, policy_version 94552 (0.0008) [2023-10-08 07:37:03,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 194183168. Throughput: 0: 1841.8, 1: 1840.6. Samples: 48554754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:37:03,755][130385] Avg episode reward: [(0, '76.310'), (1, '78.580')] [2023-10-08 07:37:04,578][00612] Updated weights for policy 1, policy_version 95080 (0.0008) [2023-10-08 07:37:04,949][00612] Updated weights for policy 1, policy_version 95090 (0.0008) [2023-10-08 07:37:05,320][00612] Updated weights for policy 1, policy_version 95100 (0.0009) [2023-10-08 07:37:06,506][00611] Updated weights for policy 0, policy_version 94562 (0.0007) [2023-10-08 07:37:06,880][00611] Updated weights for policy 0, policy_version 94572 (0.0010) [2023-10-08 07:37:07,254][00611] Updated weights for policy 0, policy_version 94582 (0.0009) [2023-10-08 07:37:07,623][00611] Updated weights for policy 0, policy_version 94592 (0.0007) [2023-10-08 07:37:08,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 194248704. Throughput: 0: 1857.2, 1: 1834.6. Samples: 48565910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:37:08,754][130385] Avg episode reward: [(0, '76.770'), (1, '81.080')] [2023-10-08 07:37:08,972][00612] Updated weights for policy 1, policy_version 95110 (0.0010) [2023-10-08 07:37:09,341][00612] Updated weights for policy 1, policy_version 95120 (0.0009) [2023-10-08 07:37:09,707][00612] Updated weights for policy 1, policy_version 95130 (0.0007) [2023-10-08 07:37:11,431][00611] Updated weights for policy 0, policy_version 94602 (0.0009) [2023-10-08 07:37:11,814][00611] Updated weights for policy 0, policy_version 94612 (0.0008) [2023-10-08 07:37:12,184][00611] Updated weights for policy 0, policy_version 94622 (0.0008) [2023-10-08 07:37:13,291][00612] Updated weights for policy 1, policy_version 95140 (0.0008) [2023-10-08 07:37:13,661][00612] Updated weights for policy 1, policy_version 95150 (0.0007) [2023-10-08 07:37:13,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 194314240. Throughput: 0: 1833.2, 1: 1843.6. Samples: 48587484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:37:13,754][130385] Avg episode reward: [(0, '78.140'), (1, '82.590')] [2023-10-08 07:37:14,033][00612] Updated weights for policy 1, policy_version 95160 (0.0010) [2023-10-08 07:37:15,859][00611] Updated weights for policy 0, policy_version 94632 (0.0008) [2023-10-08 07:37:16,227][00611] Updated weights for policy 0, policy_version 94642 (0.0008) [2023-10-08 07:37:16,602][00611] Updated weights for policy 0, policy_version 94652 (0.0009) [2023-10-08 07:37:17,597][00612] Updated weights for policy 1, policy_version 95170 (0.0008) [2023-10-08 07:37:17,953][00612] Updated weights for policy 1, policy_version 95180 (0.0008) [2023-10-08 07:37:18,328][00612] Updated weights for policy 1, policy_version 95190 (0.0008) [2023-10-08 07:37:18,694][00612] Updated weights for policy 1, policy_version 95200 (0.0009) [2023-10-08 07:37:18,754][130385] Fps is (10 sec: 16383.5, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 194412544. Throughput: 0: 1844.2, 1: 1828.1. Samples: 48609654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:37:18,755][130385] Avg episode reward: [(0, '80.100'), (1, '83.820')] [2023-10-08 07:37:20,230][00611] Updated weights for policy 0, policy_version 94662 (0.0009) [2023-10-08 07:37:20,598][00611] Updated weights for policy 0, policy_version 94672 (0.0009) [2023-10-08 07:37:20,981][00611] Updated weights for policy 0, policy_version 94682 (0.0010) [2023-10-08 07:37:22,244][00612] Updated weights for policy 1, policy_version 95210 (0.0008) [2023-10-08 07:37:22,605][00612] Updated weights for policy 1, policy_version 95220 (0.0008) [2023-10-08 07:37:22,974][00612] Updated weights for policy 1, policy_version 95230 (0.0007) [2023-10-08 07:37:23,754][130385] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 194478080. Throughput: 0: 1821.4, 1: 1844.5. Samples: 48620598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:37:23,754][130385] Avg episode reward: [(0, '79.180'), (1, '82.810')] [2023-10-08 07:37:24,692][00611] Updated weights for policy 0, policy_version 94692 (0.0009) [2023-10-08 07:37:25,057][00611] Updated weights for policy 0, policy_version 94702 (0.0009) [2023-10-08 07:37:25,434][00611] Updated weights for policy 0, policy_version 94712 (0.0010) [2023-10-08 07:37:26,646][00612] Updated weights for policy 1, policy_version 95240 (0.0007) [2023-10-08 07:37:27,009][00612] Updated weights for policy 1, policy_version 95250 (0.0008) [2023-10-08 07:37:27,378][00612] Updated weights for policy 1, policy_version 95260 (0.0007) [2023-10-08 07:37:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 194543616. Throughput: 0: 1827.5, 1: 1828.7. Samples: 48642558. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 07:37:28,755][130385] Avg episode reward: [(0, '79.370'), (1, '80.100')] [2023-10-08 07:37:29,177][00611] Updated weights for policy 0, policy_version 94722 (0.0010) [2023-10-08 07:37:29,552][00611] Updated weights for policy 0, policy_version 94732 (0.0009) [2023-10-08 07:37:29,917][00611] Updated weights for policy 0, policy_version 94742 (0.0008) [2023-10-08 07:37:30,285][00611] Updated weights for policy 0, policy_version 94752 (0.0009) [2023-10-08 07:37:31,082][00612] Updated weights for policy 1, policy_version 95270 (0.0007) [2023-10-08 07:37:31,460][00612] Updated weights for policy 1, policy_version 95280 (0.0008) [2023-10-08 07:37:31,820][00612] Updated weights for policy 1, policy_version 95290 (0.0007) [2023-10-08 07:37:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 194609152. Throughput: 0: 1824.2, 1: 1850.4. Samples: 48665224. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 07:37:33,754][130385] Avg episode reward: [(0, '75.370'), (1, '80.840')] [2023-10-08 07:37:33,926][00611] Updated weights for policy 0, policy_version 94762 (0.0008) [2023-10-08 07:37:34,306][00611] Updated weights for policy 0, policy_version 94772 (0.0007) [2023-10-08 07:37:34,664][00611] Updated weights for policy 0, policy_version 94782 (0.0007) [2023-10-08 07:37:35,368][00612] Updated weights for policy 1, policy_version 95300 (0.0010) [2023-10-08 07:37:35,741][00612] Updated weights for policy 1, policy_version 95310 (0.0010) [2023-10-08 07:37:36,118][00612] Updated weights for policy 1, policy_version 95320 (0.0010) [2023-10-08 07:37:38,290][00611] Updated weights for policy 0, policy_version 94792 (0.0010) [2023-10-08 07:37:38,659][00611] Updated weights for policy 0, policy_version 94802 (0.0007) [2023-10-08 07:37:38,754][130385] Fps is (10 sec: 13107.6, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 194674688. Throughput: 0: 1831.2, 1: 1832.3. Samples: 48675774. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 07:37:38,754][130385] Avg episode reward: [(0, '80.600'), (1, '77.250')] [2023-10-08 07:37:39,028][00611] Updated weights for policy 0, policy_version 94812 (0.0009) [2023-10-08 07:37:39,642][00612] Updated weights for policy 1, policy_version 95330 (0.0008) [2023-10-08 07:37:40,010][00612] Updated weights for policy 1, policy_version 95340 (0.0011) [2023-10-08 07:37:40,378][00612] Updated weights for policy 1, policy_version 95350 (0.0011) [2023-10-08 07:37:40,741][00612] Updated weights for policy 1, policy_version 95360 (0.0007) [2023-10-08 07:37:42,785][00611] Updated weights for policy 0, policy_version 94822 (0.0008) [2023-10-08 07:37:43,156][00611] Updated weights for policy 0, policy_version 94832 (0.0009) [2023-10-08 07:37:43,538][00611] Updated weights for policy 0, policy_version 94842 (0.0009) [2023-10-08 07:37:43,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 194740224. Throughput: 0: 1822.4, 1: 1852.8. Samples: 48698418. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 07:37:43,754][130385] Avg episode reward: [(0, '84.650'), (1, '81.810')] [2023-10-08 07:37:44,429][00612] Updated weights for policy 1, policy_version 95370 (0.0007) [2023-10-08 07:37:44,803][00612] Updated weights for policy 1, policy_version 95380 (0.0007) [2023-10-08 07:37:45,158][00612] Updated weights for policy 1, policy_version 95390 (0.0008) [2023-10-08 07:37:47,169][00611] Updated weights for policy 0, policy_version 94852 (0.0008) [2023-10-08 07:37:47,553][00611] Updated weights for policy 0, policy_version 94862 (0.0007) [2023-10-08 07:37:47,920][00611] Updated weights for policy 0, policy_version 94872 (0.0007) [2023-10-08 07:37:48,754][130385] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 194838528. Throughput: 0: 1823.2, 1: 1850.5. Samples: 48720070. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 07:37:48,755][130385] Avg episode reward: [(0, '79.940'), (1, '78.210')] [2023-10-08 07:37:48,948][00612] Updated weights for policy 1, policy_version 95400 (0.0009) [2023-10-08 07:37:49,317][00612] Updated weights for policy 1, policy_version 95410 (0.0010) [2023-10-08 07:37:49,683][00612] Updated weights for policy 1, policy_version 95420 (0.0009) [2023-10-08 07:37:51,553][00611] Updated weights for policy 0, policy_version 94882 (0.0010) [2023-10-08 07:37:51,922][00611] Updated weights for policy 0, policy_version 94892 (0.0011) [2023-10-08 07:37:52,294][00611] Updated weights for policy 0, policy_version 94902 (0.0008) [2023-10-08 07:37:52,671][00611] Updated weights for policy 0, policy_version 94912 (0.0008) [2023-10-08 07:37:53,238][00612] Updated weights for policy 1, policy_version 95430 (0.0009) [2023-10-08 07:37:53,596][00612] Updated weights for policy 1, policy_version 95440 (0.0008) [2023-10-08 07:37:53,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 194904064. Throughput: 0: 1822.3, 1: 1856.4. Samples: 48731450. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 07:37:53,755][130385] Avg episode reward: [(0, '81.020'), (1, '77.780')] [2023-10-08 07:37:53,971][00612] Updated weights for policy 1, policy_version 95450 (0.0008) [2023-10-08 07:37:56,271][00611] Updated weights for policy 0, policy_version 94922 (0.0008) [2023-10-08 07:37:56,648][00611] Updated weights for policy 0, policy_version 94932 (0.0010) [2023-10-08 07:37:57,023][00611] Updated weights for policy 0, policy_version 94942 (0.0009) [2023-10-08 07:37:57,573][00612] Updated weights for policy 1, policy_version 95460 (0.0009) [2023-10-08 07:37:57,950][00612] Updated weights for policy 1, policy_version 95470 (0.0008) [2023-10-08 07:37:58,314][00612] Updated weights for policy 1, policy_version 95480 (0.0011) [2023-10-08 07:37:58,754][130385] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 195002368. Throughput: 0: 1826.4, 1: 1860.3. Samples: 48753382. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 07:37:58,754][130385] Avg episode reward: [(0, '79.610'), (1, '77.300')] [2023-10-08 07:38:00,597][00611] Updated weights for policy 0, policy_version 94952 (0.0010) [2023-10-08 07:38:00,968][00611] Updated weights for policy 0, policy_version 94962 (0.0009) [2023-10-08 07:38:01,343][00611] Updated weights for policy 0, policy_version 94972 (0.0009) [2023-10-08 07:38:01,948][00612] Updated weights for policy 1, policy_version 95490 (0.0009) [2023-10-08 07:38:02,308][00612] Updated weights for policy 1, policy_version 95500 (0.0008) [2023-10-08 07:38:02,677][00612] Updated weights for policy 1, policy_version 95510 (0.0009) [2023-10-08 07:38:03,042][00612] Updated weights for policy 1, policy_version 95520 (0.0010) [2023-10-08 07:38:03,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 195067904. Throughput: 0: 1839.3, 1: 1838.4. Samples: 48775152. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 07:38:03,755][130385] Avg episode reward: [(0, '82.910'), (1, '75.720')] [2023-10-08 07:38:04,837][00611] Updated weights for policy 0, policy_version 94982 (0.0007) [2023-10-08 07:38:05,212][00611] Updated weights for policy 0, policy_version 94992 (0.0007) [2023-10-08 07:38:05,580][00611] Updated weights for policy 0, policy_version 95002 (0.0009) [2023-10-08 07:38:06,768][00612] Updated weights for policy 1, policy_version 95530 (0.0009) [2023-10-08 07:38:07,148][00612] Updated weights for policy 1, policy_version 95540 (0.0010) [2023-10-08 07:38:07,510][00612] Updated weights for policy 1, policy_version 95550 (0.0011) [2023-10-08 07:38:08,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 195133440. Throughput: 0: 1839.2, 1: 1852.7. Samples: 48786734. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 07:38:08,755][130385] Avg episode reward: [(0, '84.670'), (1, '76.440')] [2023-10-08 07:38:09,242][00611] Updated weights for policy 0, policy_version 95012 (0.0009) [2023-10-08 07:38:09,617][00611] Updated weights for policy 0, policy_version 95022 (0.0008) [2023-10-08 07:38:09,986][00611] Updated weights for policy 0, policy_version 95032 (0.0007) [2023-10-08 07:38:11,208][00612] Updated weights for policy 1, policy_version 95560 (0.0007) [2023-10-08 07:38:11,582][00612] Updated weights for policy 1, policy_version 95570 (0.0007) [2023-10-08 07:38:11,945][00612] Updated weights for policy 1, policy_version 95580 (0.0008) [2023-10-08 07:38:13,691][00611] Updated weights for policy 0, policy_version 95042 (0.0008) [2023-10-08 07:38:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 195198976. Throughput: 0: 1845.4, 1: 1838.1. Samples: 48808316. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 07:38:13,755][130385] Avg episode reward: [(0, '82.030'), (1, '78.180')] [2023-10-08 07:38:14,058][00611] Updated weights for policy 0, policy_version 95052 (0.0007) [2023-10-08 07:38:14,433][00611] Updated weights for policy 0, policy_version 95062 (0.0008) [2023-10-08 07:38:14,811][00611] Updated weights for policy 0, policy_version 95072 (0.0007) [2023-10-08 07:38:15,564][00612] Updated weights for policy 1, policy_version 95590 (0.0008) [2023-10-08 07:38:15,954][00612] Updated weights for policy 1, policy_version 95600 (0.0010) [2023-10-08 07:38:16,327][00612] Updated weights for policy 1, policy_version 95610 (0.0007) [2023-10-08 07:38:18,517][00611] Updated weights for policy 0, policy_version 95082 (0.0009) [2023-10-08 07:38:18,754][130385] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 14773.3). Total num frames: 195264512. Throughput: 0: 1846.4, 1: 1844.7. Samples: 48831326. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 07:38:18,755][130385] Avg episode reward: [(0, '84.450'), (1, '81.080')] [2023-10-08 07:38:18,767][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000095616_97910784.pth... [2023-10-08 07:38:18,797][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000093888_96141312.pth [2023-10-08 07:38:18,895][00611] Updated weights for policy 0, policy_version 95092 (0.0008) [2023-10-08 07:38:19,258][00611] Updated weights for policy 0, policy_version 95102 (0.0008) [2023-10-08 07:38:19,332][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000095104_97386496.pth... [2023-10-08 07:38:19,372][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000093376_95617024.pth [2023-10-08 07:38:19,968][00612] Updated weights for policy 1, policy_version 95620 (0.0009) [2023-10-08 07:38:20,337][00612] Updated weights for policy 1, policy_version 95630 (0.0009) [2023-10-08 07:38:20,693][00612] Updated weights for policy 1, policy_version 95640 (0.0011) [2023-10-08 07:38:22,871][00611] Updated weights for policy 0, policy_version 95112 (0.0008) [2023-10-08 07:38:23,240][00611] Updated weights for policy 0, policy_version 95122 (0.0008) [2023-10-08 07:38:23,607][00611] Updated weights for policy 0, policy_version 95132 (0.0009) [2023-10-08 07:38:23,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 195330048. Throughput: 0: 1846.1, 1: 1831.8. Samples: 48841280. Policy #0 lag: (min: 8.0, avg: 31.6, max: 40.0) [2023-10-08 07:38:23,755][130385] Avg episode reward: [(0, '79.730'), (1, '82.580')] [2023-10-08 07:38:24,455][00612] Updated weights for policy 1, policy_version 95650 (0.0010) [2023-10-08 07:38:24,818][00612] Updated weights for policy 1, policy_version 95660 (0.0009) [2023-10-08 07:38:25,185][00612] Updated weights for policy 1, policy_version 95670 (0.0011) [2023-10-08 07:38:25,546][00612] Updated weights for policy 1, policy_version 95680 (0.0010) [2023-10-08 07:38:27,330][00611] Updated weights for policy 0, policy_version 95142 (0.0010) [2023-10-08 07:38:27,701][00611] Updated weights for policy 0, policy_version 95152 (0.0007) [2023-10-08 07:38:28,078][00611] Updated weights for policy 0, policy_version 95162 (0.0007) [2023-10-08 07:38:28,754][130385] Fps is (10 sec: 16384.8, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 195428352. Throughput: 0: 1840.5, 1: 1832.8. Samples: 48863718. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:38:28,754][130385] Avg episode reward: [(0, '80.010'), (1, '82.820')] [2023-10-08 07:38:29,197][00612] Updated weights for policy 1, policy_version 95690 (0.0007) [2023-10-08 07:38:29,561][00612] Updated weights for policy 1, policy_version 95700 (0.0010) [2023-10-08 07:38:29,926][00612] Updated weights for policy 1, policy_version 95710 (0.0007) [2023-10-08 07:38:31,753][00611] Updated weights for policy 0, policy_version 95172 (0.0007) [2023-10-08 07:38:32,119][00611] Updated weights for policy 0, policy_version 95182 (0.0008) [2023-10-08 07:38:32,501][00611] Updated weights for policy 0, policy_version 95192 (0.0008) [2023-10-08 07:38:33,438][00612] Updated weights for policy 1, policy_version 95720 (0.0007) [2023-10-08 07:38:33,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 195493888. Throughput: 0: 1838.3, 1: 1842.6. Samples: 48885708. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:38:33,755][130385] Avg episode reward: [(0, '80.380'), (1, '80.970')] [2023-10-08 07:38:33,815][00612] Updated weights for policy 1, policy_version 95730 (0.0007) [2023-10-08 07:38:34,177][00612] Updated weights for policy 1, policy_version 95740 (0.0008) [2023-10-08 07:38:36,109][00611] Updated weights for policy 0, policy_version 95202 (0.0007) [2023-10-08 07:38:36,486][00611] Updated weights for policy 0, policy_version 95212 (0.0007) [2023-10-08 07:38:36,856][00611] Updated weights for policy 0, policy_version 95222 (0.0009) [2023-10-08 07:38:37,221][00611] Updated weights for policy 0, policy_version 95232 (0.0008) [2023-10-08 07:38:37,741][00612] Updated weights for policy 1, policy_version 95750 (0.0008) [2023-10-08 07:38:38,105][00612] Updated weights for policy 1, policy_version 95760 (0.0007) [2023-10-08 07:38:38,475][00612] Updated weights for policy 1, policy_version 95770 (0.0009) [2023-10-08 07:38:38,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 195592192. Throughput: 0: 1839.0, 1: 1844.4. Samples: 48897206. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:38:38,754][130385] Avg episode reward: [(0, '81.040'), (1, '85.280')] [2023-10-08 07:38:40,675][00611] Updated weights for policy 0, policy_version 95242 (0.0009) [2023-10-08 07:38:41,044][00611] Updated weights for policy 0, policy_version 95252 (0.0009) [2023-10-08 07:38:41,401][00611] Updated weights for policy 0, policy_version 95262 (0.0007) [2023-10-08 07:38:42,201][00612] Updated weights for policy 1, policy_version 95780 (0.0009) [2023-10-08 07:38:42,562][00612] Updated weights for policy 1, policy_version 95790 (0.0008) [2023-10-08 07:38:42,935][00612] Updated weights for policy 1, policy_version 95800 (0.0009) [2023-10-08 07:38:43,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 195657728. Throughput: 0: 1844.4, 1: 1834.4. Samples: 48918926. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:38:43,754][130385] Avg episode reward: [(0, '82.110'), (1, '86.480')] [2023-10-08 07:38:45,034][00611] Updated weights for policy 0, policy_version 95272 (0.0008) [2023-10-08 07:38:45,409][00611] Updated weights for policy 0, policy_version 95282 (0.0007) [2023-10-08 07:38:45,779][00611] Updated weights for policy 0, policy_version 95292 (0.0009) [2023-10-08 07:38:46,639][00612] Updated weights for policy 1, policy_version 95810 (0.0009) [2023-10-08 07:38:47,007][00612] Updated weights for policy 1, policy_version 95820 (0.0008) [2023-10-08 07:38:47,370][00612] Updated weights for policy 1, policy_version 95830 (0.0007) [2023-10-08 07:38:47,737][00612] Updated weights for policy 1, policy_version 95840 (0.0010) [2023-10-08 07:38:48,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 195723264. Throughput: 0: 1842.5, 1: 1839.9. Samples: 48940856. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:38:48,754][130385] Avg episode reward: [(0, '84.870'), (1, '88.250')] [2023-10-08 07:38:49,682][00611] Updated weights for policy 0, policy_version 95302 (0.0007) [2023-10-08 07:38:50,071][00611] Updated weights for policy 0, policy_version 95312 (0.0007) [2023-10-08 07:38:50,443][00611] Updated weights for policy 0, policy_version 95322 (0.0009) [2023-10-08 07:38:51,350][00612] Updated weights for policy 1, policy_version 95850 (0.0007) [2023-10-08 07:38:51,717][00612] Updated weights for policy 1, policy_version 95860 (0.0012) [2023-10-08 07:38:52,085][00612] Updated weights for policy 1, policy_version 95870 (0.0010) [2023-10-08 07:38:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 195788800. Throughput: 0: 1836.3, 1: 1836.1. Samples: 48951994. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:38:53,754][130385] Avg episode reward: [(0, '81.570'), (1, '82.780')] [2023-10-08 07:38:53,890][00611] Updated weights for policy 0, policy_version 95332 (0.0009) [2023-10-08 07:38:54,267][00611] Updated weights for policy 0, policy_version 95342 (0.0011) [2023-10-08 07:38:54,637][00611] Updated weights for policy 0, policy_version 95352 (0.0010) [2023-10-08 07:38:55,814][00612] Updated weights for policy 1, policy_version 95880 (0.0010) [2023-10-08 07:38:56,185][00612] Updated weights for policy 1, policy_version 95890 (0.0010) [2023-10-08 07:38:56,546][00612] Updated weights for policy 1, policy_version 95900 (0.0010) [2023-10-08 07:38:58,240][00611] Updated weights for policy 0, policy_version 95362 (0.0009) [2023-10-08 07:38:58,609][00611] Updated weights for policy 0, policy_version 95372 (0.0009) [2023-10-08 07:38:58,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14773.4). Total num frames: 195854336. Throughput: 0: 1835.7, 1: 1842.6. Samples: 48973838. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:38:58,754][130385] Avg episode reward: [(0, '83.860'), (1, '82.230')] [2023-10-08 07:38:58,977][00611] Updated weights for policy 0, policy_version 95382 (0.0007) [2023-10-08 07:38:59,352][00611] Updated weights for policy 0, policy_version 95392 (0.0009) [2023-10-08 07:39:00,208][00612] Updated weights for policy 1, policy_version 95910 (0.0009) [2023-10-08 07:39:00,578][00612] Updated weights for policy 1, policy_version 95920 (0.0011) [2023-10-08 07:39:00,946][00612] Updated weights for policy 1, policy_version 95930 (0.0010) [2023-10-08 07:39:03,024][00611] Updated weights for policy 0, policy_version 95402 (0.0010) [2023-10-08 07:39:03,393][00611] Updated weights for policy 0, policy_version 95412 (0.0008) [2023-10-08 07:39:03,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 195919872. Throughput: 0: 1815.3, 1: 1847.5. Samples: 48996154. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:39:03,755][130385] Avg episode reward: [(0, '82.990'), (1, '83.750')] [2023-10-08 07:39:03,759][00611] Updated weights for policy 0, policy_version 95422 (0.0009) [2023-10-08 07:39:04,610][00612] Updated weights for policy 1, policy_version 95940 (0.0009) [2023-10-08 07:39:05,007][00612] Updated weights for policy 1, policy_version 95950 (0.0007) [2023-10-08 07:39:05,373][00612] Updated weights for policy 1, policy_version 95960 (0.0009) [2023-10-08 07:39:07,507][00611] Updated weights for policy 0, policy_version 95432 (0.0009) [2023-10-08 07:39:07,879][00611] Updated weights for policy 0, policy_version 95442 (0.0010) [2023-10-08 07:39:08,269][00611] Updated weights for policy 0, policy_version 95452 (0.0011) [2023-10-08 07:39:08,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 196018176. Throughput: 0: 1828.1, 1: 1847.6. Samples: 49006688. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:39:08,754][130385] Avg episode reward: [(0, '81.900'), (1, '81.530')] [2023-10-08 07:39:08,801][00612] Updated weights for policy 1, policy_version 95970 (0.0009) [2023-10-08 07:39:09,165][00612] Updated weights for policy 1, policy_version 95980 (0.0008) [2023-10-08 07:39:09,542][00612] Updated weights for policy 1, policy_version 95990 (0.0008) [2023-10-08 07:39:09,909][00612] Updated weights for policy 1, policy_version 96000 (0.0008) [2023-10-08 07:39:11,886][00611] Updated weights for policy 0, policy_version 95462 (0.0009) [2023-10-08 07:39:12,253][00611] Updated weights for policy 0, policy_version 95472 (0.0008) [2023-10-08 07:39:12,627][00611] Updated weights for policy 0, policy_version 95482 (0.0007) [2023-10-08 07:39:13,580][00612] Updated weights for policy 1, policy_version 96010 (0.0007) [2023-10-08 07:39:13,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 196083712. Throughput: 0: 1822.2, 1: 1856.8. Samples: 49029274. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:39:13,754][130385] Avg episode reward: [(0, '86.430'), (1, '82.510')] [2023-10-08 07:39:13,947][00612] Updated weights for policy 1, policy_version 96020 (0.0007) [2023-10-08 07:39:14,324][00612] Updated weights for policy 1, policy_version 96030 (0.0008) [2023-10-08 07:39:16,245][00611] Updated weights for policy 0, policy_version 95492 (0.0009) [2023-10-08 07:39:16,613][00611] Updated weights for policy 0, policy_version 95502 (0.0009) [2023-10-08 07:39:16,993][00611] Updated weights for policy 0, policy_version 95512 (0.0009) [2023-10-08 07:39:17,859][00612] Updated weights for policy 1, policy_version 96040 (0.0008) [2023-10-08 07:39:18,218][00612] Updated weights for policy 1, policy_version 96050 (0.0008) [2023-10-08 07:39:18,585][00612] Updated weights for policy 1, policy_version 96060 (0.0007) [2023-10-08 07:39:18,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.9, 300 sec: 14773.4). Total num frames: 196182016. Throughput: 0: 1837.5, 1: 1835.2. Samples: 49050980. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:39:18,755][130385] Avg episode reward: [(0, '84.400'), (1, '79.240')] [2023-10-08 07:39:20,500][00611] Updated weights for policy 0, policy_version 95522 (0.0008) [2023-10-08 07:39:20,860][00611] Updated weights for policy 0, policy_version 95532 (0.0011) [2023-10-08 07:39:21,237][00611] Updated weights for policy 0, policy_version 95542 (0.0010) [2023-10-08 07:39:21,604][00611] Updated weights for policy 0, policy_version 95552 (0.0008) [2023-10-08 07:39:22,059][00612] Updated weights for policy 1, policy_version 96070 (0.0009) [2023-10-08 07:39:22,422][00612] Updated weights for policy 1, policy_version 96080 (0.0009) [2023-10-08 07:39:22,796][00612] Updated weights for policy 1, policy_version 96090 (0.0007) [2023-10-08 07:39:23,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 196247552. Throughput: 0: 1819.0, 1: 1853.0. Samples: 49062444. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:39:23,755][130385] Avg episode reward: [(0, '83.600'), (1, '78.760')] [2023-10-08 07:39:25,409][00611] Updated weights for policy 0, policy_version 95562 (0.0007) [2023-10-08 07:39:25,779][00611] Updated weights for policy 0, policy_version 95572 (0.0008) [2023-10-08 07:39:26,148][00611] Updated weights for policy 0, policy_version 95582 (0.0010) [2023-10-08 07:39:26,406][00612] Updated weights for policy 1, policy_version 96100 (0.0007) [2023-10-08 07:39:26,779][00612] Updated weights for policy 1, policy_version 96110 (0.0009) [2023-10-08 07:39:27,148][00612] Updated weights for policy 1, policy_version 96120 (0.0010) [2023-10-08 07:39:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 196313088. Throughput: 0: 1835.6, 1: 1832.9. Samples: 49084006. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 07:39:28,754][130385] Avg episode reward: [(0, '80.490'), (1, '73.660')] [2023-10-08 07:39:29,671][00611] Updated weights for policy 0, policy_version 95592 (0.0008) [2023-10-08 07:39:30,041][00611] Updated weights for policy 0, policy_version 95602 (0.0010) [2023-10-08 07:39:30,412][00611] Updated weights for policy 0, policy_version 95612 (0.0008) [2023-10-08 07:39:30,927][00612] Updated weights for policy 1, policy_version 96130 (0.0009) [2023-10-08 07:39:31,300][00612] Updated weights for policy 1, policy_version 96140 (0.0008) [2023-10-08 07:39:31,660][00612] Updated weights for policy 1, policy_version 96150 (0.0009) [2023-10-08 07:39:32,038][00612] Updated weights for policy 1, policy_version 96160 (0.0009) [2023-10-08 07:39:33,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 196378624. Throughput: 0: 1835.9, 1: 1850.1. Samples: 49106724. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 07:39:33,755][130385] Avg episode reward: [(0, '88.500'), (1, '75.540')] [2023-10-08 07:39:34,068][00611] Updated weights for policy 0, policy_version 95622 (0.0008) [2023-10-08 07:39:34,457][00611] Updated weights for policy 0, policy_version 95632 (0.0007) [2023-10-08 07:39:34,830][00611] Updated weights for policy 0, policy_version 95642 (0.0008) [2023-10-08 07:39:35,691][00612] Updated weights for policy 1, policy_version 96170 (0.0007) [2023-10-08 07:39:36,050][00612] Updated weights for policy 1, policy_version 96180 (0.0008) [2023-10-08 07:39:36,423][00612] Updated weights for policy 1, policy_version 96190 (0.0007) [2023-10-08 07:39:38,520][00611] Updated weights for policy 0, policy_version 95652 (0.0007) [2023-10-08 07:39:38,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 196444160. Throughput: 0: 1839.3, 1: 1829.6. Samples: 49117098. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 07:39:38,755][130385] Avg episode reward: [(0, '81.460'), (1, '75.440')] [2023-10-08 07:39:38,894][00611] Updated weights for policy 0, policy_version 95662 (0.0010) [2023-10-08 07:39:39,255][00611] Updated weights for policy 0, policy_version 95672 (0.0010) [2023-10-08 07:39:40,054][00612] Updated weights for policy 1, policy_version 96200 (0.0008) [2023-10-08 07:39:40,419][00612] Updated weights for policy 1, policy_version 96210 (0.0008) [2023-10-08 07:39:40,797][00612] Updated weights for policy 1, policy_version 96220 (0.0009) [2023-10-08 07:39:42,856][00611] Updated weights for policy 0, policy_version 95682 (0.0007) [2023-10-08 07:39:43,215][00611] Updated weights for policy 0, policy_version 95692 (0.0009) [2023-10-08 07:39:43,592][00611] Updated weights for policy 0, policy_version 95702 (0.0008) [2023-10-08 07:39:43,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 196509696. Throughput: 0: 1838.0, 1: 1847.4. Samples: 49139684. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 07:39:43,755][130385] Avg episode reward: [(0, '79.320'), (1, '73.510')] [2023-10-08 07:39:43,965][00611] Updated weights for policy 0, policy_version 95712 (0.0011) [2023-10-08 07:39:44,455][00612] Updated weights for policy 1, policy_version 96230 (0.0008) [2023-10-08 07:39:44,822][00612] Updated weights for policy 1, policy_version 96240 (0.0007) [2023-10-08 07:39:45,194][00612] Updated weights for policy 1, policy_version 96250 (0.0008) [2023-10-08 07:39:47,662][00611] Updated weights for policy 0, policy_version 95722 (0.0010) [2023-10-08 07:39:48,037][00611] Updated weights for policy 0, policy_version 95732 (0.0009) [2023-10-08 07:39:48,406][00611] Updated weights for policy 0, policy_version 95742 (0.0010) [2023-10-08 07:39:48,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 196608000. Throughput: 0: 1834.6, 1: 1845.3. Samples: 49161752. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 07:39:48,754][130385] Avg episode reward: [(0, '82.700'), (1, '73.830')] [2023-10-08 07:39:48,821][00612] Updated weights for policy 1, policy_version 96260 (0.0010) [2023-10-08 07:39:49,196][00612] Updated weights for policy 1, policy_version 96270 (0.0009) [2023-10-08 07:39:49,562][00612] Updated weights for policy 1, policy_version 96280 (0.0007) [2023-10-08 07:39:51,947][00611] Updated weights for policy 0, policy_version 95752 (0.0008) [2023-10-08 07:39:52,313][00611] Updated weights for policy 0, policy_version 95762 (0.0007) [2023-10-08 07:39:52,688][00611] Updated weights for policy 0, policy_version 95772 (0.0008) [2023-10-08 07:39:53,228][00612] Updated weights for policy 1, policy_version 96290 (0.0008) [2023-10-08 07:39:53,639][00612] Updated weights for policy 1, policy_version 96300 (0.0008) [2023-10-08 07:39:53,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 196673536. Throughput: 0: 1852.9, 1: 1847.7. Samples: 49173214. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 07:39:53,755][130385] Avg episode reward: [(0, '81.060'), (1, '73.110')] [2023-10-08 07:39:54,012][00612] Updated weights for policy 1, policy_version 96310 (0.0008) [2023-10-08 07:39:54,376][00612] Updated weights for policy 1, policy_version 96320 (0.0007) [2023-10-08 07:39:56,346][00611] Updated weights for policy 0, policy_version 95782 (0.0008) [2023-10-08 07:39:56,721][00611] Updated weights for policy 0, policy_version 95792 (0.0008) [2023-10-08 07:39:57,100][00611] Updated weights for policy 0, policy_version 95802 (0.0009) [2023-10-08 07:39:57,920][00612] Updated weights for policy 1, policy_version 96330 (0.0008) [2023-10-08 07:39:58,279][00612] Updated weights for policy 1, policy_version 96340 (0.0008) [2023-10-08 07:39:58,647][00612] Updated weights for policy 1, policy_version 96350 (0.0009) [2023-10-08 07:39:58,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 196771840. Throughput: 0: 1838.2, 1: 1849.5. Samples: 49195222. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 07:39:58,755][130385] Avg episode reward: [(0, '79.020'), (1, '73.920')] [2023-10-08 07:40:00,566][00611] Updated weights for policy 0, policy_version 95812 (0.0008) [2023-10-08 07:40:00,933][00611] Updated weights for policy 0, policy_version 95822 (0.0008) [2023-10-08 07:40:01,308][00611] Updated weights for policy 0, policy_version 95832 (0.0007) [2023-10-08 07:40:02,386][00612] Updated weights for policy 1, policy_version 96360 (0.0007) [2023-10-08 07:40:02,765][00612] Updated weights for policy 1, policy_version 96370 (0.0008) [2023-10-08 07:40:03,122][00612] Updated weights for policy 1, policy_version 96380 (0.0010) [2023-10-08 07:40:03,754][130385] Fps is (10 sec: 16384.5, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 196837376. Throughput: 0: 1857.0, 1: 1833.6. Samples: 49217056. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 07:40:03,754][130385] Avg episode reward: [(0, '80.270'), (1, '72.460')] [2023-10-08 07:40:04,897][00611] Updated weights for policy 0, policy_version 95842 (0.0008) [2023-10-08 07:40:05,262][00611] Updated weights for policy 0, policy_version 95852 (0.0008) [2023-10-08 07:40:05,633][00611] Updated weights for policy 0, policy_version 95862 (0.0007) [2023-10-08 07:40:06,002][00611] Updated weights for policy 0, policy_version 95872 (0.0007) [2023-10-08 07:40:06,701][00612] Updated weights for policy 1, policy_version 96390 (0.0007) [2023-10-08 07:40:07,063][00612] Updated weights for policy 1, policy_version 96400 (0.0009) [2023-10-08 07:40:07,431][00612] Updated weights for policy 1, policy_version 96410 (0.0007) [2023-10-08 07:40:08,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 196902912. Throughput: 0: 1844.5, 1: 1848.2. Samples: 49228616. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 07:40:08,754][130385] Avg episode reward: [(0, '79.710'), (1, '75.480')] [2023-10-08 07:40:09,518][00611] Updated weights for policy 0, policy_version 95882 (0.0009) [2023-10-08 07:40:09,897][00611] Updated weights for policy 0, policy_version 95892 (0.0007) [2023-10-08 07:40:10,278][00611] Updated weights for policy 0, policy_version 95902 (0.0009) [2023-10-08 07:40:11,158][00612] Updated weights for policy 1, policy_version 96420 (0.0008) [2023-10-08 07:40:11,530][00612] Updated weights for policy 1, policy_version 96430 (0.0008) [2023-10-08 07:40:11,898][00612] Updated weights for policy 1, policy_version 96440 (0.0009) [2023-10-08 07:40:13,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 196968448. Throughput: 0: 1860.3, 1: 1835.3. Samples: 49250310. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 07:40:13,755][130385] Avg episode reward: [(0, '80.910'), (1, '77.680')] [2023-10-08 07:40:14,045][00611] Updated weights for policy 0, policy_version 95912 (0.0011) [2023-10-08 07:40:14,425][00611] Updated weights for policy 0, policy_version 95922 (0.0009) [2023-10-08 07:40:14,801][00611] Updated weights for policy 0, policy_version 95932 (0.0007) [2023-10-08 07:40:15,453][00612] Updated weights for policy 1, policy_version 96450 (0.0010) [2023-10-08 07:40:15,822][00612] Updated weights for policy 1, policy_version 96460 (0.0009) [2023-10-08 07:40:16,188][00612] Updated weights for policy 1, policy_version 96470 (0.0009) [2023-10-08 07:40:16,549][00612] Updated weights for policy 1, policy_version 96480 (0.0009) [2023-10-08 07:40:18,451][00611] Updated weights for policy 0, policy_version 95942 (0.0008) [2023-10-08 07:40:18,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 197033984. Throughput: 0: 1853.2, 1: 1851.1. Samples: 49273416. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 07:40:18,754][130385] Avg episode reward: [(0, '81.800'), (1, '80.230')] [2023-10-08 07:40:18,762][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000096480_98795520.pth... [2023-10-08 07:40:18,801][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000094752_97026048.pth [2023-10-08 07:40:18,817][00611] Updated weights for policy 0, policy_version 95952 (0.0009) [2023-10-08 07:40:19,190][00611] Updated weights for policy 0, policy_version 95962 (0.0010) [2023-10-08 07:40:19,411][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000095968_98271232.pth... [2023-10-08 07:40:19,439][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000094240_96501760.pth [2023-10-08 07:40:20,129][00612] Updated weights for policy 1, policy_version 96490 (0.0010) [2023-10-08 07:40:20,491][00612] Updated weights for policy 1, policy_version 96500 (0.0010) [2023-10-08 07:40:20,861][00612] Updated weights for policy 1, policy_version 96510 (0.0010) [2023-10-08 07:40:22,951][00611] Updated weights for policy 0, policy_version 95972 (0.0009) [2023-10-08 07:40:23,337][00611] Updated weights for policy 0, policy_version 95982 (0.0007) [2023-10-08 07:40:23,709][00611] Updated weights for policy 0, policy_version 95992 (0.0007) [2023-10-08 07:40:23,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 197099520. Throughput: 0: 1855.3, 1: 1842.1. Samples: 49283478. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 07:40:23,754][130385] Avg episode reward: [(0, '84.310'), (1, '77.470')] [2023-10-08 07:40:24,529][00612] Updated weights for policy 1, policy_version 96520 (0.0008) [2023-10-08 07:40:24,899][00612] Updated weights for policy 1, policy_version 96530 (0.0008) [2023-10-08 07:40:25,279][00612] Updated weights for policy 1, policy_version 96540 (0.0012) [2023-10-08 07:40:27,292][00611] Updated weights for policy 0, policy_version 96002 (0.0008) [2023-10-08 07:40:27,663][00611] Updated weights for policy 0, policy_version 96012 (0.0008) [2023-10-08 07:40:28,030][00611] Updated weights for policy 0, policy_version 96022 (0.0009) [2023-10-08 07:40:28,394][00611] Updated weights for policy 0, policy_version 96032 (0.0009) [2023-10-08 07:40:28,753][00612] Updated weights for policy 1, policy_version 96550 (0.0008) [2023-10-08 07:40:28,754][130385] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 197197824. Throughput: 0: 1850.9, 1: 1865.3. Samples: 49306912. Policy #0 lag: (min: 15.0, avg: 18.4, max: 47.0) [2023-10-08 07:40:28,754][130385] Avg episode reward: [(0, '84.660'), (1, '73.010')] [2023-10-08 07:40:29,113][00612] Updated weights for policy 1, policy_version 96560 (0.0010) [2023-10-08 07:40:29,482][00612] Updated weights for policy 1, policy_version 96570 (0.0007) [2023-10-08 07:40:32,025][00611] Updated weights for policy 0, policy_version 96042 (0.0010) [2023-10-08 07:40:32,403][00611] Updated weights for policy 0, policy_version 96052 (0.0011) [2023-10-08 07:40:32,768][00611] Updated weights for policy 0, policy_version 96062 (0.0008) [2023-10-08 07:40:33,205][00612] Updated weights for policy 1, policy_version 96580 (0.0008) [2023-10-08 07:40:33,568][00612] Updated weights for policy 1, policy_version 96590 (0.0007) [2023-10-08 07:40:33,754][130385] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197263360. Throughput: 0: 1840.1, 1: 1864.5. Samples: 49328460. Policy #0 lag: (min: 15.0, avg: 18.4, max: 47.0) [2023-10-08 07:40:33,754][130385] Avg episode reward: [(0, '85.560'), (1, '69.550')] [2023-10-08 07:40:33,942][00612] Updated weights for policy 1, policy_version 96600 (0.0008) [2023-10-08 07:40:36,391][00611] Updated weights for policy 0, policy_version 96072 (0.0010) [2023-10-08 07:40:36,763][00611] Updated weights for policy 0, policy_version 96082 (0.0008) [2023-10-08 07:40:37,131][00611] Updated weights for policy 0, policy_version 96092 (0.0009) [2023-10-08 07:40:37,489][00612] Updated weights for policy 1, policy_version 96610 (0.0008) [2023-10-08 07:40:37,874][00612] Updated weights for policy 1, policy_version 96620 (0.0009) [2023-10-08 07:40:38,244][00612] Updated weights for policy 1, policy_version 96630 (0.0008) [2023-10-08 07:40:38,612][00612] Updated weights for policy 1, policy_version 96640 (0.0007) [2023-10-08 07:40:38,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 197361664. Throughput: 0: 1840.7, 1: 1867.6. Samples: 49340090. Policy #0 lag: (min: 15.0, avg: 18.4, max: 47.0) [2023-10-08 07:40:38,755][130385] Avg episode reward: [(0, '84.750'), (1, '70.100')] [2023-10-08 07:40:40,776][00611] Updated weights for policy 0, policy_version 96102 (0.0009) [2023-10-08 07:40:41,149][00611] Updated weights for policy 0, policy_version 96112 (0.0008) [2023-10-08 07:40:41,514][00611] Updated weights for policy 0, policy_version 96122 (0.0007) [2023-10-08 07:40:42,063][00612] Updated weights for policy 1, policy_version 96650 (0.0008) [2023-10-08 07:40:42,421][00612] Updated weights for policy 1, policy_version 96660 (0.0008) [2023-10-08 07:40:42,793][00612] Updated weights for policy 1, policy_version 96670 (0.0008) [2023-10-08 07:40:43,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 197427200. Throughput: 0: 1838.1, 1: 1851.7. Samples: 49361266. Policy #0 lag: (min: 15.0, avg: 18.4, max: 47.0) [2023-10-08 07:40:43,754][130385] Avg episode reward: [(0, '82.430'), (1, '67.660')] [2023-10-08 07:40:45,148][00611] Updated weights for policy 0, policy_version 96132 (0.0008) [2023-10-08 07:40:45,521][00611] Updated weights for policy 0, policy_version 96142 (0.0009) [2023-10-08 07:40:45,898][00611] Updated weights for policy 0, policy_version 96152 (0.0007) [2023-10-08 07:40:46,435][00612] Updated weights for policy 1, policy_version 96680 (0.0008) [2023-10-08 07:40:46,805][00612] Updated weights for policy 1, policy_version 96690 (0.0008) [2023-10-08 07:40:47,173][00612] Updated weights for policy 1, policy_version 96700 (0.0008) [2023-10-08 07:40:48,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 197492736. Throughput: 0: 1836.2, 1: 1862.4. Samples: 49383494. Policy #0 lag: (min: 15.0, avg: 18.4, max: 47.0) [2023-10-08 07:40:48,754][130385] Avg episode reward: [(0, '82.680'), (1, '67.790')] [2023-10-08 07:40:49,557][00611] Updated weights for policy 0, policy_version 96162 (0.0007) [2023-10-08 07:40:49,922][00611] Updated weights for policy 0, policy_version 96172 (0.0010) [2023-10-08 07:40:50,294][00611] Updated weights for policy 0, policy_version 96182 (0.0009) [2023-10-08 07:40:50,663][00611] Updated weights for policy 0, policy_version 96192 (0.0009) [2023-10-08 07:40:50,747][00612] Updated weights for policy 1, policy_version 96710 (0.0008) [2023-10-08 07:40:51,122][00612] Updated weights for policy 1, policy_version 96720 (0.0010) [2023-10-08 07:40:51,490][00612] Updated weights for policy 1, policy_version 96730 (0.0008) [2023-10-08 07:40:53,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 197558272. Throughput: 0: 1837.8, 1: 1847.4. Samples: 49394450. Policy #0 lag: (min: 15.0, avg: 18.4, max: 47.0) [2023-10-08 07:40:53,754][130385] Avg episode reward: [(0, '85.330'), (1, '68.680')] [2023-10-08 07:40:54,500][00611] Updated weights for policy 0, policy_version 96202 (0.0009) [2023-10-08 07:40:54,863][00611] Updated weights for policy 0, policy_version 96212 (0.0008) [2023-10-08 07:40:55,122][00612] Updated weights for policy 1, policy_version 96740 (0.0007) [2023-10-08 07:40:55,230][00611] Updated weights for policy 0, policy_version 96222 (0.0009) [2023-10-08 07:40:55,487][00612] Updated weights for policy 1, policy_version 96750 (0.0009) [2023-10-08 07:40:55,855][00612] Updated weights for policy 1, policy_version 96760 (0.0010) [2023-10-08 07:40:58,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 197623808. Throughput: 0: 1830.5, 1: 1866.9. Samples: 49416694. Policy #0 lag: (min: 15.0, avg: 18.4, max: 47.0) [2023-10-08 07:40:58,755][130385] Avg episode reward: [(0, '88.390'), (1, '69.870')] [2023-10-08 07:40:58,883][00611] Updated weights for policy 0, policy_version 96232 (0.0009) [2023-10-08 07:40:59,244][00611] Updated weights for policy 0, policy_version 96242 (0.0007) [2023-10-08 07:40:59,493][00612] Updated weights for policy 1, policy_version 96770 (0.0009) [2023-10-08 07:40:59,612][00611] Updated weights for policy 0, policy_version 96252 (0.0009) [2023-10-08 07:40:59,870][00612] Updated weights for policy 1, policy_version 96780 (0.0010) [2023-10-08 07:41:00,251][00612] Updated weights for policy 1, policy_version 96790 (0.0010) [2023-10-08 07:41:00,614][00612] Updated weights for policy 1, policy_version 96800 (0.0010) [2023-10-08 07:41:03,302][00611] Updated weights for policy 0, policy_version 96262 (0.0007) [2023-10-08 07:41:03,683][00611] Updated weights for policy 0, policy_version 96272 (0.0008) [2023-10-08 07:41:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 197689344. Throughput: 0: 1825.2, 1: 1864.4. Samples: 49439446. Policy #0 lag: (min: 15.0, avg: 18.4, max: 47.0) [2023-10-08 07:41:03,754][130385] Avg episode reward: [(0, '83.650'), (1, '64.350')] [2023-10-08 07:41:04,059][00611] Updated weights for policy 0, policy_version 96282 (0.0010) [2023-10-08 07:41:04,330][00612] Updated weights for policy 1, policy_version 96810 (0.0007) [2023-10-08 07:41:04,700][00612] Updated weights for policy 1, policy_version 96820 (0.0007) [2023-10-08 07:41:05,064][00612] Updated weights for policy 1, policy_version 96830 (0.0007) [2023-10-08 07:41:07,629][00611] Updated weights for policy 0, policy_version 96292 (0.0010) [2023-10-08 07:41:07,989][00611] Updated weights for policy 0, policy_version 96302 (0.0008) [2023-10-08 07:41:08,356][00611] Updated weights for policy 0, policy_version 96312 (0.0008) [2023-10-08 07:41:08,715][00612] Updated weights for policy 1, policy_version 96840 (0.0007) [2023-10-08 07:41:08,754][130385] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 197787648. Throughput: 0: 1831.2, 1: 1861.2. Samples: 49449636. Policy #0 lag: (min: 15.0, avg: 18.4, max: 47.0) [2023-10-08 07:41:08,754][130385] Avg episode reward: [(0, '84.510'), (1, '67.840')] [2023-10-08 07:41:09,091][00612] Updated weights for policy 1, policy_version 96850 (0.0009) [2023-10-08 07:41:09,459][00612] Updated weights for policy 1, policy_version 96860 (0.0007) [2023-10-08 07:41:11,910][00611] Updated weights for policy 0, policy_version 96322 (0.0008) [2023-10-08 07:41:12,283][00611] Updated weights for policy 0, policy_version 96332 (0.0010) [2023-10-08 07:41:12,656][00611] Updated weights for policy 0, policy_version 96342 (0.0012) [2023-10-08 07:41:13,024][00611] Updated weights for policy 0, policy_version 96352 (0.0008) [2023-10-08 07:41:13,027][00612] Updated weights for policy 1, policy_version 96870 (0.0008) [2023-10-08 07:41:13,388][00612] Updated weights for policy 1, policy_version 96880 (0.0008) [2023-10-08 07:41:13,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197853184. Throughput: 0: 1825.2, 1: 1851.1. Samples: 49472348. Policy #0 lag: (min: 15.0, avg: 18.4, max: 47.0) [2023-10-08 07:41:13,754][130385] Avg episode reward: [(0, '84.770'), (1, '68.830')] [2023-10-08 07:41:13,766][00612] Updated weights for policy 1, policy_version 96890 (0.0011) [2023-10-08 07:41:16,711][00611] Updated weights for policy 0, policy_version 96362 (0.0009) [2023-10-08 07:41:17,092][00611] Updated weights for policy 0, policy_version 96372 (0.0007) [2023-10-08 07:41:17,472][00611] Updated weights for policy 0, policy_version 96382 (0.0009) [2023-10-08 07:41:17,488][00612] Updated weights for policy 1, policy_version 96900 (0.0009) [2023-10-08 07:41:17,857][00612] Updated weights for policy 1, policy_version 96910 (0.0009) [2023-10-08 07:41:18,221][00612] Updated weights for policy 1, policy_version 96920 (0.0007) [2023-10-08 07:41:18,754][130385] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 197951488. Throughput: 0: 1833.5, 1: 1826.6. Samples: 49493162. Policy #0 lag: (min: 15.0, avg: 18.4, max: 47.0) [2023-10-08 07:41:18,755][130385] Avg episode reward: [(0, '87.200'), (1, '70.630')] [2023-10-08 07:41:21,083][00611] Updated weights for policy 0, policy_version 96392 (0.0008) [2023-10-08 07:41:21,440][00611] Updated weights for policy 0, policy_version 96402 (0.0007) [2023-10-08 07:41:21,819][00611] Updated weights for policy 0, policy_version 96412 (0.0007) [2023-10-08 07:41:21,842][00612] Updated weights for policy 1, policy_version 96930 (0.0008) [2023-10-08 07:41:22,222][00612] Updated weights for policy 1, policy_version 96940 (0.0009) [2023-10-08 07:41:22,587][00612] Updated weights for policy 1, policy_version 96950 (0.0008) [2023-10-08 07:41:22,957][00612] Updated weights for policy 1, policy_version 96960 (0.0010) [2023-10-08 07:41:23,754][130385] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 198017024. Throughput: 0: 1823.6, 1: 1845.4. Samples: 49505194. Policy #0 lag: (min: 15.0, avg: 18.4, max: 47.0) [2023-10-08 07:41:23,755][130385] Avg episode reward: [(0, '85.830'), (1, '70.390')] [2023-10-08 07:41:25,469][00611] Updated weights for policy 0, policy_version 96422 (0.0009) [2023-10-08 07:41:25,848][00611] Updated weights for policy 0, policy_version 96432 (0.0008) [2023-10-08 07:41:26,216][00611] Updated weights for policy 0, policy_version 96442 (0.0008) [2023-10-08 07:41:26,693][00612] Updated weights for policy 1, policy_version 96970 (0.0010) [2023-10-08 07:41:27,053][00612] Updated weights for policy 1, policy_version 96980 (0.0009) [2023-10-08 07:41:27,428][00612] Updated weights for policy 1, policy_version 96990 (0.0007) [2023-10-08 07:41:28,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 198082560. Throughput: 0: 1831.0, 1: 1827.0. Samples: 49525876. Policy #0 lag: (min: 15.0, avg: 18.4, max: 47.0) [2023-10-08 07:41:28,754][130385] Avg episode reward: [(0, '85.700'), (1, '74.140')] [2023-10-08 07:41:29,863][00611] Updated weights for policy 0, policy_version 96452 (0.0008) [2023-10-08 07:41:30,234][00611] Updated weights for policy 0, policy_version 96462 (0.0008) [2023-10-08 07:41:30,606][00611] Updated weights for policy 0, policy_version 96472 (0.0008) [2023-10-08 07:41:31,094][00612] Updated weights for policy 1, policy_version 97000 (0.0009) [2023-10-08 07:41:31,457][00612] Updated weights for policy 1, policy_version 97010 (0.0011) [2023-10-08 07:41:31,826][00612] Updated weights for policy 1, policy_version 97020 (0.0010) [2023-10-08 07:41:33,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 198148096. Throughput: 0: 1840.1, 1: 1833.5. Samples: 49548808. Policy #0 lag: (min: 27.0, avg: 31.2, max: 59.0) [2023-10-08 07:41:33,755][130385] Avg episode reward: [(0, '84.590'), (1, '76.600')] [2023-10-08 07:41:34,075][00611] Updated weights for policy 0, policy_version 96482 (0.0009) [2023-10-08 07:41:34,444][00611] Updated weights for policy 0, policy_version 96492 (0.0009) [2023-10-08 07:41:34,805][00611] Updated weights for policy 0, policy_version 96502 (0.0008) [2023-10-08 07:41:35,180][00611] Updated weights for policy 0, policy_version 96512 (0.0007) [2023-10-08 07:41:35,490][00612] Updated weights for policy 1, policy_version 97030 (0.0010) [2023-10-08 07:41:35,857][00612] Updated weights for policy 1, policy_version 97040 (0.0009) [2023-10-08 07:41:36,234][00612] Updated weights for policy 1, policy_version 97050 (0.0010) [2023-10-08 07:41:38,752][00611] Updated weights for policy 0, policy_version 96522 (0.0009) [2023-10-08 07:41:38,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 198213632. Throughput: 0: 1842.4, 1: 1819.8. Samples: 49559248. Policy #0 lag: (min: 27.0, avg: 31.2, max: 59.0) [2023-10-08 07:41:38,754][130385] Avg episode reward: [(0, '85.410'), (1, '79.320')] [2023-10-08 07:41:39,120][00611] Updated weights for policy 0, policy_version 96532 (0.0008) [2023-10-08 07:41:39,496][00611] Updated weights for policy 0, policy_version 96542 (0.0007) [2023-10-08 07:41:39,676][00612] Updated weights for policy 1, policy_version 97060 (0.0007) [2023-10-08 07:41:40,037][00612] Updated weights for policy 1, policy_version 97070 (0.0007) [2023-10-08 07:41:40,411][00612] Updated weights for policy 1, policy_version 97080 (0.0009) [2023-10-08 07:41:43,085][00611] Updated weights for policy 0, policy_version 96552 (0.0009) [2023-10-08 07:41:43,454][00611] Updated weights for policy 0, policy_version 96562 (0.0010) [2023-10-08 07:41:43,754][130385] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 198279168. Throughput: 0: 1846.9, 1: 1832.0. Samples: 49582244. Policy #0 lag: (min: 27.0, avg: 31.2, max: 59.0) [2023-10-08 07:41:43,755][130385] Avg episode reward: [(0, '82.750'), (1, '80.850')] [2023-10-08 07:41:43,821][00611] Updated weights for policy 0, policy_version 96572 (0.0008) [2023-10-08 07:41:44,157][00612] Updated weights for policy 1, policy_version 97090 (0.0007) [2023-10-08 07:41:44,527][00612] Updated weights for policy 1, policy_version 97100 (0.0008) [2023-10-08 07:41:44,894][00612] Updated weights for policy 1, policy_version 97110 (0.0009) [2023-10-08 07:41:45,273][00612] Updated weights for policy 1, policy_version 97120 (0.0008) [2023-10-08 07:41:47,473][00611] Updated weights for policy 0, policy_version 96582 (0.0008) [2023-10-08 07:41:47,844][00611] Updated weights for policy 0, policy_version 96592 (0.0008) [2023-10-08 07:41:48,208][00611] Updated weights for policy 0, policy_version 96602 (0.0010) [2023-10-08 07:41:48,754][130385] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 198377472. Throughput: 0: 1831.0, 1: 1829.2. Samples: 49604154. Policy #0 lag: (min: 27.0, avg: 31.2, max: 59.0) [2023-10-08 07:41:48,754][130385] Avg episode reward: [(0, '86.920'), (1, '83.850')] [2023-10-08 07:41:48,999][00612] Updated weights for policy 1, policy_version 97130 (0.0010) [2023-10-08 07:41:49,374][00612] Updated weights for policy 1, policy_version 97140 (0.0009) [2023-10-08 07:41:49,747][00612] Updated weights for policy 1, policy_version 97150 (0.0011) [2023-10-08 07:41:51,889][00611] Updated weights for policy 0, policy_version 96612 (0.0009) [2023-10-08 07:41:52,262][00611] Updated weights for policy 0, policy_version 96622 (0.0008) [2023-10-08 07:41:52,627][00611] Updated weights for policy 0, policy_version 96632 (0.0008) [2023-10-08 07:41:53,357][00612] Updated weights for policy 1, policy_version 97160 (0.0010) [2023-10-08 07:41:53,719][00612] Updated weights for policy 1, policy_version 97170 (0.0009) [2023-10-08 07:41:53,754][130385] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 198443008. Throughput: 0: 1852.9, 1: 1824.6. Samples: 49615124. Policy #0 lag: (min: 27.0, avg: 31.2, max: 59.0) [2023-10-08 07:41:53,754][130385] Avg episode reward: [(0, '83.530'), (1, '84.590')] [2023-10-08 07:41:54,092][00612] Updated weights for policy 1, policy_version 97180 (0.0007) [2023-10-08 07:41:56,495][00611] Updated weights for policy 0, policy_version 96642 (0.0009) [2023-10-08 07:41:56,897][00611] Updated weights for policy 0, policy_version 96652 (0.0009) [2023-10-08 07:41:57,269][00611] Updated weights for policy 0, policy_version 96662 (0.0009) [2023-10-08 07:41:57,528][00612] Updated weights for policy 1, policy_version 97190 (0.0008) [2023-10-08 07:41:57,645][00611] Updated weights for policy 0, policy_version 96672 (0.0009) [2023-10-08 07:41:57,894][00612] Updated weights for policy 1, policy_version 97200 (0.0007) [2023-10-08 07:41:58,262][00612] Updated weights for policy 1, policy_version 97210 (0.0010) [2023-10-08 07:41:58,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 198541312. Throughput: 0: 1838.2, 1: 1836.7. Samples: 49637718. Policy #0 lag: (min: 27.0, avg: 31.2, max: 59.0) [2023-10-08 07:41:58,754][130385] Avg episode reward: [(0, '86.680'), (1, '86.740')] [2023-10-08 07:42:01,216][00611] Updated weights for policy 0, policy_version 96682 (0.0007) [2023-10-08 07:42:01,585][00611] Updated weights for policy 0, policy_version 96692 (0.0011) [2023-10-08 07:42:01,900][00612] Updated weights for policy 1, policy_version 97220 (0.0011) [2023-10-08 07:42:01,964][00611] Updated weights for policy 0, policy_version 96702 (0.0008) [2023-10-08 07:42:02,271][00612] Updated weights for policy 1, policy_version 97230 (0.0008) [2023-10-08 07:42:02,630][00612] Updated weights for policy 1, policy_version 97240 (0.0009) [2023-10-08 07:42:03,754][130385] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 198606848. Throughput: 0: 1845.5, 1: 1831.1. Samples: 49658610. Policy #0 lag: (min: 27.0, avg: 31.2, max: 59.0) [2023-10-08 07:42:03,755][130385] Avg episode reward: [(0, '89.440'), (1, '88.320')] [2023-10-08 07:42:05,696][00611] Updated weights for policy 0, policy_version 96712 (0.0008) [2023-10-08 07:42:06,077][00611] Updated weights for policy 0, policy_version 96722 (0.0009) [2023-10-08 07:42:06,367][00612] Updated weights for policy 1, policy_version 97250 (0.0008) [2023-10-08 07:42:06,441][00611] Updated weights for policy 0, policy_version 96732 (0.0008) [2023-10-08 07:42:06,742][00612] Updated weights for policy 1, policy_version 97260 (0.0009) [2023-10-08 07:42:07,112][00612] Updated weights for policy 1, policy_version 97270 (0.0008) [2023-10-08 07:42:07,472][00612] Updated weights for policy 1, policy_version 97280 (0.0008) [2023-10-08 07:42:08,754][130385] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 198672384. Throughput: 0: 1833.9, 1: 1843.6. Samples: 49670680. Policy #0 lag: (min: 27.0, avg: 31.2, max: 59.0) [2023-10-08 07:42:08,755][130385] Avg episode reward: [(0, '87.820'), (1, '90.740')] [2023-10-08 07:42:10,053][00611] Updated weights for policy 0, policy_version 96742 (0.0009) [2023-10-08 07:42:10,414][00611] Updated weights for policy 0, policy_version 96752 (0.0009) [2023-10-08 07:42:10,786][00611] Updated weights for policy 0, policy_version 96762 (0.0010) [2023-10-08 07:42:10,977][00612] Updated weights for policy 1, policy_version 97290 (0.0008) [2023-10-08 07:42:11,349][00612] Updated weights for policy 1, policy_version 97300 (0.0008) [2023-10-08 07:42:11,712][00612] Updated weights for policy 1, policy_version 97310 (0.0008) [2023-10-08 07:42:13,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 198737920. Throughput: 0: 1846.9, 1: 1839.7. Samples: 49691772. Policy #0 lag: (min: 27.0, avg: 31.2, max: 59.0) [2023-10-08 07:42:13,754][130385] Avg episode reward: [(0, '84.660'), (1, '84.490')] [2023-10-08 07:42:14,361][00611] Updated weights for policy 0, policy_version 96772 (0.0009) [2023-10-08 07:42:14,737][00611] Updated weights for policy 0, policy_version 96782 (0.0009) [2023-10-08 07:42:15,112][00611] Updated weights for policy 0, policy_version 96792 (0.0009) [2023-10-08 07:42:15,481][00612] Updated weights for policy 1, policy_version 97320 (0.0008) [2023-10-08 07:42:15,857][00612] Updated weights for policy 1, policy_version 97330 (0.0008) [2023-10-08 07:42:16,224][00612] Updated weights for policy 1, policy_version 97340 (0.0009) [2023-10-08 07:42:18,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 198803456. Throughput: 0: 1841.9, 1: 1849.4. Samples: 49714914. Policy #0 lag: (min: 27.0, avg: 31.2, max: 59.0) [2023-10-08 07:42:18,755][130385] Avg episode reward: [(0, '85.440'), (1, '89.670')] [2023-10-08 07:42:18,767][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000097344_99680256.pth... [2023-10-08 07:42:18,769][00611] Updated weights for policy 0, policy_version 96802 (0.0008) [2023-10-08 07:42:18,808][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000095616_97910784.pth [2023-10-08 07:42:19,142][00611] Updated weights for policy 0, policy_version 96812 (0.0009) [2023-10-08 07:42:19,521][00611] Updated weights for policy 0, policy_version 96822 (0.0008) [2023-10-08 07:42:19,825][00612] Updated weights for policy 1, policy_version 97350 (0.0008) [2023-10-08 07:42:19,886][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000096832_99155968.pth... [2023-10-08 07:42:19,889][00611] Updated weights for policy 0, policy_version 96832 (0.0007) [2023-10-08 07:42:19,926][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000095104_97386496.pth [2023-10-08 07:42:20,194][00612] Updated weights for policy 1, policy_version 97360 (0.0008) [2023-10-08 07:42:20,560][00612] Updated weights for policy 1, policy_version 97370 (0.0009) [2023-10-08 07:42:23,578][00611] Updated weights for policy 0, policy_version 96842 (0.0010) [2023-10-08 07:42:23,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 198868992. Throughput: 0: 1841.5, 1: 1841.1. Samples: 49724964. Policy #0 lag: (min: 27.0, avg: 31.2, max: 59.0) [2023-10-08 07:42:23,754][130385] Avg episode reward: [(0, '85.280'), (1, '92.050')] [2023-10-08 07:42:23,939][00611] Updated weights for policy 0, policy_version 96852 (0.0010) [2023-10-08 07:42:24,304][00611] Updated weights for policy 0, policy_version 96862 (0.0009) [2023-10-08 07:42:24,350][00612] Updated weights for policy 1, policy_version 97380 (0.0008) [2023-10-08 07:42:24,714][00612] Updated weights for policy 1, policy_version 97390 (0.0010) [2023-10-08 07:42:25,079][00612] Updated weights for policy 1, policy_version 97400 (0.0010) [2023-10-08 07:42:27,880][00611] Updated weights for policy 0, policy_version 96872 (0.0011) [2023-10-08 07:42:28,260][00611] Updated weights for policy 0, policy_version 96882 (0.0008) [2023-10-08 07:42:28,623][00611] Updated weights for policy 0, policy_version 96892 (0.0008) [2023-10-08 07:42:28,710][00612] Updated weights for policy 1, policy_version 97410 (0.0008) [2023-10-08 07:42:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 198934528. Throughput: 0: 1839.0, 1: 1839.6. Samples: 49747780. Policy #0 lag: (min: 27.0, avg: 31.2, max: 59.0) [2023-10-08 07:42:28,755][130385] Avg episode reward: [(0, '86.570'), (1, '95.160')] [2023-10-08 07:42:29,083][00612] Updated weights for policy 1, policy_version 97420 (0.0010) [2023-10-08 07:42:29,452][00612] Updated weights for policy 1, policy_version 97430 (0.0008) [2023-10-08 07:42:29,814][00612] Updated weights for policy 1, policy_version 97440 (0.0009) [2023-10-08 07:42:32,266][00611] Updated weights for policy 0, policy_version 96902 (0.0008) [2023-10-08 07:42:32,639][00611] Updated weights for policy 0, policy_version 96912 (0.0009) [2023-10-08 07:42:32,996][00611] Updated weights for policy 0, policy_version 96922 (0.0008) [2023-10-08 07:42:33,570][00612] Updated weights for policy 1, policy_version 97450 (0.0007) [2023-10-08 07:42:33,754][130385] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 199032832. Throughput: 0: 1830.7, 1: 1848.0. Samples: 49769696. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 07:42:33,755][130385] Avg episode reward: [(0, '86.440'), (1, '94.650')] [2023-10-08 07:42:33,934][00612] Updated weights for policy 1, policy_version 97460 (0.0008) [2023-10-08 07:42:34,304][00612] Updated weights for policy 1, policy_version 97470 (0.0007) [2023-10-08 07:42:36,531][00611] Updated weights for policy 0, policy_version 96932 (0.0008) [2023-10-08 07:42:36,912][00611] Updated weights for policy 0, policy_version 96942 (0.0009) [2023-10-08 07:42:37,275][00611] Updated weights for policy 0, policy_version 96952 (0.0009) [2023-10-08 07:42:37,851][00612] Updated weights for policy 1, policy_version 97480 (0.0009) [2023-10-08 07:42:38,214][00612] Updated weights for policy 1, policy_version 97490 (0.0007) [2023-10-08 07:42:38,590][00612] Updated weights for policy 1, policy_version 97500 (0.0010) [2023-10-08 07:42:38,754][130385] Fps is (10 sec: 19661.4, 60 sec: 15291.7, 300 sec: 14884.5). Total num frames: 199131136. Throughput: 0: 1837.8, 1: 1853.3. Samples: 49781224. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 07:42:38,754][130385] Avg episode reward: [(0, '84.060'), (1, '95.440')] [2023-10-08 07:42:40,996][00611] Updated weights for policy 0, policy_version 96962 (0.0010) [2023-10-08 07:42:41,382][00611] Updated weights for policy 0, policy_version 96972 (0.0010) [2023-10-08 07:42:41,759][00611] Updated weights for policy 0, policy_version 96982 (0.0008) [2023-10-08 07:42:42,122][00611] Updated weights for policy 0, policy_version 96992 (0.0007) [2023-10-08 07:42:42,181][00612] Updated weights for policy 1, policy_version 97510 (0.0008) [2023-10-08 07:42:42,549][00612] Updated weights for policy 1, policy_version 97520 (0.0007) [2023-10-08 07:42:42,917][00612] Updated weights for policy 1, policy_version 97530 (0.0007) [2023-10-08 07:42:43,754][130385] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 199196672. Throughput: 0: 1824.9, 1: 1842.9. Samples: 49802770. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 07:42:43,754][130385] Avg episode reward: [(0, '87.320'), (1, '94.020')] [2023-10-08 07:42:45,713][00611] Updated weights for policy 0, policy_version 97002 (0.0011) [2023-10-08 07:42:46,082][00611] Updated weights for policy 0, policy_version 97012 (0.0009) [2023-10-08 07:42:46,451][00611] Updated weights for policy 0, policy_version 97022 (0.0007) [2023-10-08 07:42:46,463][00612] Updated weights for policy 1, policy_version 97540 (0.0008) [2023-10-08 07:42:46,824][00612] Updated weights for policy 1, policy_version 97550 (0.0009) [2023-10-08 07:42:47,201][00612] Updated weights for policy 1, policy_version 97560 (0.0007) [2023-10-08 07:42:48,754][130385] Fps is (10 sec: 13106.5, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 199262208. Throughput: 0: 1843.9, 1: 1846.4. Samples: 49824676. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 07:42:48,756][130385] Avg episode reward: [(0, '90.750'), (1, '90.510')] [2023-10-08 07:42:50,111][00611] Updated weights for policy 0, policy_version 97032 (0.0011) [2023-10-08 07:42:50,486][00611] Updated weights for policy 0, policy_version 97042 (0.0011) [2023-10-08 07:42:50,842][00612] Updated weights for policy 1, policy_version 97570 (0.0011) [2023-10-08 07:42:50,856][00611] Updated weights for policy 0, policy_version 97052 (0.0009) [2023-10-08 07:42:51,209][00612] Updated weights for policy 1, policy_version 97580 (0.0008) [2023-10-08 07:42:51,587][00612] Updated weights for policy 1, policy_version 97590 (0.0009) [2023-10-08 07:42:51,947][00612] Updated weights for policy 1, policy_version 97600 (0.0008) [2023-10-08 07:42:53,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199327744. Throughput: 0: 1830.9, 1: 1833.4. Samples: 49835572. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 07:42:53,754][130385] Avg episode reward: [(0, '87.690'), (1, '89.700')] [2023-10-08 07:42:54,646][00611] Updated weights for policy 0, policy_version 97062 (0.0009) [2023-10-08 07:42:55,020][00611] Updated weights for policy 0, policy_version 97072 (0.0008) [2023-10-08 07:42:55,402][00611] Updated weights for policy 0, policy_version 97082 (0.0009) [2023-10-08 07:42:55,600][00612] Updated weights for policy 1, policy_version 97610 (0.0008) [2023-10-08 07:42:55,978][00612] Updated weights for policy 1, policy_version 97620 (0.0008) [2023-10-08 07:42:56,347][00612] Updated weights for policy 1, policy_version 97630 (0.0008) [2023-10-08 07:42:58,754][130385] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 199393280. Throughput: 0: 1843.9, 1: 1844.8. Samples: 49857760. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 07:42:58,755][130385] Avg episode reward: [(0, '83.560'), (1, '85.060')] [2023-10-08 07:42:58,886][00611] Updated weights for policy 0, policy_version 97092 (0.0009) [2023-10-08 07:42:59,264][00611] Updated weights for policy 0, policy_version 97102 (0.0008) [2023-10-08 07:42:59,638][00611] Updated weights for policy 0, policy_version 97112 (0.0007) [2023-10-08 07:43:00,094][00612] Updated weights for policy 1, policy_version 97640 (0.0009) [2023-10-08 07:43:00,469][00612] Updated weights for policy 1, policy_version 97650 (0.0010) [2023-10-08 07:43:00,833][00612] Updated weights for policy 1, policy_version 97660 (0.0009) [2023-10-08 07:43:03,190][00611] Updated weights for policy 0, policy_version 97122 (0.0007) [2023-10-08 07:43:03,556][00611] Updated weights for policy 0, policy_version 97132 (0.0007) [2023-10-08 07:43:03,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 199458816. Throughput: 0: 1846.2, 1: 1846.2. Samples: 49881074. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 07:43:03,754][130385] Avg episode reward: [(0, '83.590'), (1, '86.210')] [2023-10-08 07:43:03,925][00611] Updated weights for policy 0, policy_version 97142 (0.0008) [2023-10-08 07:43:04,300][00611] Updated weights for policy 0, policy_version 97152 (0.0009) [2023-10-08 07:43:04,354][00612] Updated weights for policy 1, policy_version 97670 (0.0008) [2023-10-08 07:43:04,728][00612] Updated weights for policy 1, policy_version 97680 (0.0008) [2023-10-08 07:43:05,090][00612] Updated weights for policy 1, policy_version 97690 (0.0007) [2023-10-08 07:43:07,912][00611] Updated weights for policy 0, policy_version 97162 (0.0007) [2023-10-08 07:43:08,281][00611] Updated weights for policy 0, policy_version 97172 (0.0007) [2023-10-08 07:43:08,656][00611] Updated weights for policy 0, policy_version 97182 (0.0009) [2023-10-08 07:43:08,662][00612] Updated weights for policy 1, policy_version 97700 (0.0007) [2023-10-08 07:43:08,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 199557120. Throughput: 0: 1843.3, 1: 1850.0. Samples: 49891162. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 07:43:08,754][130385] Avg episode reward: [(0, '81.410'), (1, '86.860')] [2023-10-08 07:43:09,032][00612] Updated weights for policy 1, policy_version 97710 (0.0008) [2023-10-08 07:43:09,397][00612] Updated weights for policy 1, policy_version 97720 (0.0009) [2023-10-08 07:43:12,434][00611] Updated weights for policy 0, policy_version 97192 (0.0008) [2023-10-08 07:43:12,808][00611] Updated weights for policy 0, policy_version 97202 (0.0008) [2023-10-08 07:43:12,862][00612] Updated weights for policy 1, policy_version 97730 (0.0008) [2023-10-08 07:43:13,180][00611] Updated weights for policy 0, policy_version 97212 (0.0007) [2023-10-08 07:43:13,230][00612] Updated weights for policy 1, policy_version 97740 (0.0009) [2023-10-08 07:43:13,593][00612] Updated weights for policy 1, policy_version 97750 (0.0009) [2023-10-08 07:43:13,754][130385] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 199622656. Throughput: 0: 1839.8, 1: 1863.5. Samples: 49914428. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 07:43:13,754][130385] Avg episode reward: [(0, '81.410'), (1, '88.240')] [2023-10-08 07:43:13,957][00612] Updated weights for policy 1, policy_version 97760 (0.0008) [2023-10-08 07:43:16,805][00611] Updated weights for policy 0, policy_version 97222 (0.0007) [2023-10-08 07:43:17,175][00611] Updated weights for policy 0, policy_version 97232 (0.0007) [2023-10-08 07:43:17,557][00611] Updated weights for policy 0, policy_version 97242 (0.0008) [2023-10-08 07:43:17,563][00612] Updated weights for policy 1, policy_version 97770 (0.0008) [2023-10-08 07:43:17,925][00612] Updated weights for policy 1, policy_version 97780 (0.0009) [2023-10-08 07:43:18,289][00612] Updated weights for policy 1, policy_version 97790 (0.0007) [2023-10-08 07:43:18,754][130385] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14884.5). Total num frames: 199720960. Throughput: 0: 1839.1, 1: 1837.9. Samples: 49935160. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 07:43:18,754][130385] Avg episode reward: [(0, '82.280'), (1, '94.290')] [2023-10-08 07:43:21,225][00611] Updated weights for policy 0, policy_version 97252 (0.0009) [2023-10-08 07:43:21,589][00611] Updated weights for policy 0, policy_version 97262 (0.0007) [2023-10-08 07:43:21,870][00612] Updated weights for policy 1, policy_version 97800 (0.0008) [2023-10-08 07:43:21,967][00611] Updated weights for policy 0, policy_version 97272 (0.0007) [2023-10-08 07:43:22,239][00612] Updated weights for policy 1, policy_version 97810 (0.0008) [2023-10-08 07:43:22,602][00612] Updated weights for policy 1, policy_version 97820 (0.0007) [2023-10-08 07:43:23,754][130385] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 199786496. Throughput: 0: 1836.4, 1: 1869.9. Samples: 49948004. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 07:43:23,754][130385] Avg episode reward: [(0, '82.260'), (1, '88.850')] [2023-10-08 07:43:25,661][00611] Updated weights for policy 0, policy_version 97282 (0.0010) [2023-10-08 07:43:26,024][00611] Updated weights for policy 0, policy_version 97292 (0.0009) [2023-10-08 07:43:26,193][00612] Updated weights for policy 1, policy_version 97830 (0.0007) [2023-10-08 07:43:26,400][00611] Updated weights for policy 0, policy_version 97302 (0.0009) [2023-10-08 07:43:26,562][00612] Updated weights for policy 1, policy_version 97840 (0.0007) [2023-10-08 07:43:26,764][00611] Updated weights for policy 0, policy_version 97312 (0.0007) [2023-10-08 07:43:26,935][00612] Updated weights for policy 1, policy_version 97850 (0.0009) [2023-10-08 07:43:28,754][130385] Fps is (10 sec: 13107.2, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 199852032. Throughput: 0: 1838.1, 1: 1831.4. Samples: 49967898. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 07:43:28,754][130385] Avg episode reward: [(0, '82.670'), (1, '86.080')] [2023-10-08 07:43:30,433][00611] Updated weights for policy 0, policy_version 97322 (0.0009) [2023-10-08 07:43:30,671][00612] Updated weights for policy 1, policy_version 97860 (0.0009) [2023-10-08 07:43:30,804][00611] Updated weights for policy 0, policy_version 97332 (0.0007) [2023-10-08 07:43:31,039][00612] Updated weights for policy 1, policy_version 97870 (0.0008) [2023-10-08 07:43:31,177][00611] Updated weights for policy 0, policy_version 97342 (0.0008) [2023-10-08 07:43:31,410][00612] Updated weights for policy 1, policy_version 97880 (0.0009) [2023-10-08 07:43:33,754][130385] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199917568. Throughput: 0: 1834.3, 1: 1859.9. Samples: 49990916. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 07:43:33,755][130385] Avg episode reward: [(0, '83.170'), (1, '86.540')] [2023-10-08 07:43:34,787][00611] Updated weights for policy 0, policy_version 97352 (0.0010) [2023-10-08 07:43:35,070][00612] Updated weights for policy 1, policy_version 97890 (0.0008) [2023-10-08 07:43:35,165][00611] Updated weights for policy 0, policy_version 97362 (0.0009) [2023-10-08 07:43:35,443][00612] Updated weights for policy 1, policy_version 97900 (0.0008) [2023-10-08 07:43:35,528][00611] Updated weights for policy 0, policy_version 97372 (0.0010) [2023-10-08 07:43:35,802][00612] Updated weights for policy 1, policy_version 97910 (0.0009) [2023-10-08 07:43:36,169][00612] Updated weights for policy 1, policy_version 97920 (0.0007) [2023-10-08 07:43:38,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 199983104. Throughput: 0: 1836.0, 1: 1837.8. Samples: 50000890. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 07:43:38,754][130385] Avg episode reward: [(0, '83.290'), (1, '88.220')] [2023-10-08 07:43:39,017][00611] Updated weights for policy 0, policy_version 97382 (0.0010) [2023-10-08 07:43:39,380][00611] Updated weights for policy 0, policy_version 97392 (0.0009) [2023-10-08 07:43:39,642][00612] Updated weights for policy 1, policy_version 97930 (0.0008) [2023-10-08 07:43:39,751][00611] Updated weights for policy 0, policy_version 97402 (0.0007) [2023-10-08 07:43:40,007][00612] Updated weights for policy 1, policy_version 97940 (0.0007) [2023-10-08 07:43:40,381][00612] Updated weights for policy 1, policy_version 97950 (0.0008) [2023-10-08 07:43:43,470][00611] Updated weights for policy 0, policy_version 97412 (0.0008) [2023-10-08 07:43:43,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 200048640. Throughput: 0: 1837.1, 1: 1863.6. Samples: 50024290. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 07:43:43,755][130385] Avg episode reward: [(0, '87.870'), (1, '85.160')] [2023-10-08 07:43:43,845][00611] Updated weights for policy 0, policy_version 97422 (0.0010) [2023-10-08 07:43:44,046][00612] Updated weights for policy 1, policy_version 97960 (0.0009) [2023-10-08 07:43:44,213][00611] Updated weights for policy 0, policy_version 97432 (0.0008) [2023-10-08 07:43:44,420][00612] Updated weights for policy 1, policy_version 97970 (0.0008) [2023-10-08 07:43:44,793][00612] Updated weights for policy 1, policy_version 97980 (0.0008) [2023-10-08 07:43:47,829][00611] Updated weights for policy 0, policy_version 97442 (0.0009) [2023-10-08 07:43:48,212][00611] Updated weights for policy 0, policy_version 97452 (0.0008) [2023-10-08 07:43:48,578][00611] Updated weights for policy 0, policy_version 97462 (0.0007) [2023-10-08 07:43:48,649][00612] Updated weights for policy 1, policy_version 97990 (0.0009) [2023-10-08 07:43:48,754][130385] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14662.3). Total num frames: 200114176. Throughput: 0: 1817.8, 1: 1860.8. Samples: 50046612. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 07:43:48,754][130385] Avg episode reward: [(0, '87.560'), (1, '85.460')] [2023-10-08 07:43:48,948][00611] Updated weights for policy 0, policy_version 97472 (0.0008) [2023-10-08 07:43:49,033][00612] Updated weights for policy 1, policy_version 98000 (0.0011) [2023-10-08 07:43:49,404][00612] Updated weights for policy 1, policy_version 98010 (0.0010) [2023-10-08 07:43:52,693][00611] Updated weights for policy 0, policy_version 97482 (0.0009) [2023-10-08 07:43:52,788][00612] Updated weights for policy 1, policy_version 98020 (0.0007) [2023-10-08 07:43:53,062][00611] Updated weights for policy 0, policy_version 97492 (0.0008) [2023-10-08 07:43:53,162][00612] Updated weights for policy 1, policy_version 98030 (0.0007) [2023-10-08 07:43:53,431][00611] Updated weights for policy 0, policy_version 97502 (0.0007) [2023-10-08 07:43:53,526][00612] Updated weights for policy 1, policy_version 98040 (0.0008) [2023-10-08 07:43:53,754][130385] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 200212480. Throughput: 0: 1826.8, 1: 1858.4. Samples: 50056998. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 07:43:53,754][130385] Avg episode reward: [(0, '88.720'), (1, '87.030')] [2023-10-08 07:43:57,087][00611] Updated weights for policy 0, policy_version 97512 (0.0007) [2023-10-08 07:43:57,225][00612] Updated weights for policy 1, policy_version 98050 (0.0009) [2023-10-08 07:43:57,459][00611] Updated weights for policy 0, policy_version 97522 (0.0007) [2023-10-08 07:43:57,591][00612] Updated weights for policy 1, policy_version 98060 (0.0007) [2023-10-08 07:43:57,823][00611] Updated weights for policy 0, policy_version 97532 (0.0009) [2023-10-08 07:43:57,945][00612] Updated weights for policy 1, policy_version 98070 (0.0008) [2023-10-08 07:43:58,309][00612] Updated weights for policy 1, policy_version 98080 (0.0009) [2023-10-08 07:43:58,754][130385] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14884.4). Total num frames: 200310784. Throughput: 0: 1821.6, 1: 1849.3. Samples: 50079622. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 07:43:58,755][130385] Avg episode reward: [(0, '90.170'), (1, '89.800')] [2023-10-08 07:44:01,564][00611] Updated weights for policy 0, policy_version 97542 (0.0008) [2023-10-08 07:44:01,935][00611] Updated weights for policy 0, policy_version 97552 (0.0009) [2023-10-08 07:44:01,966][00612] Updated weights for policy 1, policy_version 98090 (0.0008) [2023-10-08 07:44:02,317][00611] Updated weights for policy 0, policy_version 97562 (0.0007) [2023-10-08 07:44:02,330][00612] Updated weights for policy 1, policy_version 98100 (0.0007) [2023-10-08 07:44:02,700][00612] Updated weights for policy 1, policy_version 98110 (0.0007) [2023-10-08 07:44:03,754][130385] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 200376320. Throughput: 0: 1827.1, 1: 1836.0. Samples: 50100000. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 07:44:03,754][130385] Avg episode reward: [(0, '89.030'), (1, '92.570')] [2023-10-08 07:44:05,857][00611] Updated weights for policy 0, policy_version 97572 (0.0007) [2023-10-08 07:44:06,231][00611] Updated weights for policy 0, policy_version 97582 (0.0007) [2023-10-08 07:44:06,485][00612] Updated weights for policy 1, policy_version 98120 (0.0010) [2023-10-08 07:44:06,595][00611] Updated weights for policy 0, policy_version 97592 (0.0007) [2023-10-08 07:44:06,845][00612] Updated weights for policy 1, policy_version 98130 (0.0007) [2023-10-08 07:44:07,214][00612] Updated weights for policy 1, policy_version 98140 (0.0008) [2023-10-08 07:44:08,754][130385] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 200441856. Throughput: 0: 1819.8, 1: 1834.4. Samples: 50112446. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 07:44:08,754][130385] Avg episode reward: [(0, '97.070'), (1, '97.990')] [2023-10-08 07:44:08,755][00365] Saving new best policy, reward=97.070! [2023-10-08 07:44:10,286][00611] Updated weights for policy 0, policy_version 97602 (0.0007) [2023-10-08 07:44:10,649][00611] Updated weights for policy 0, policy_version 97612 (0.0007) [2023-10-08 07:44:10,842][00612] Updated weights for policy 1, policy_version 98150 (0.0008) [2023-10-08 07:44:11,022][00611] Updated weights for policy 0, policy_version 97622 (0.0008) [2023-10-08 07:44:11,214][00612] Updated weights for policy 1, policy_version 98160 (0.0008) [2023-10-08 07:44:11,400][00611] Updated weights for policy 0, policy_version 97632 (0.0010) [2023-10-08 07:44:11,591][00612] Updated weights for policy 1, policy_version 98170 (0.0007) [2023-10-08 07:44:13,754][130385] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 200507392. Throughput: 0: 1832.1, 1: 1836.3. Samples: 50132978. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 07:44:13,754][130385] Avg episode reward: [(0, '94.610'), (1, '96.120')] [2023-10-08 07:44:14,984][00611] Updated weights for policy 0, policy_version 97642 (0.0010) [2023-10-08 07:44:15,160][00612] Updated weights for policy 1, policy_version 98180 (0.0009) [2023-10-08 07:44:15,353][00611] Updated weights for policy 0, policy_version 97652 (0.0008) [2023-10-08 07:44:15,533][00612] Updated weights for policy 1, policy_version 98190 (0.0008) [2023-10-08 07:44:15,727][00611] Updated weights for policy 0, policy_version 97662 (0.0008) [2023-10-08 07:44:15,891][00612] Updated weights for policy 1, policy_version 98200 (0.0008) [2023-10-08 07:44:16,187][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000098208_100564992.pth... [2023-10-08 07:44:16,187][00646] Stopping RolloutWorker_w2... [2023-10-08 07:44:16,187][00650] Stopping RolloutWorker_w5... [2023-10-08 07:44:16,187][00654] Stopping RolloutWorker_w8... [2023-10-08 07:44:16,187][01361] Stopping RolloutWorker_w14... [2023-10-08 07:44:16,187][00646] Loop rollout_proc2_evt_loop terminating... [2023-10-08 07:44:16,187][00657] Stopping RolloutWorker_w12... [2023-10-08 07:44:16,187][00656] Stopping RolloutWorker_w10... [2023-10-08 07:44:16,187][00650] Loop rollout_proc5_evt_loop terminating... [2023-10-08 07:44:16,187][00654] Loop rollout_proc8_evt_loop terminating... [2023-10-08 07:44:16,187][01411] Stopping RolloutWorker_w15... [2023-10-08 07:44:16,187][130385] Component RolloutWorker_w8 stopped! [2023-10-08 07:44:16,187][00365] Stopping Batcher_0... [2023-10-08 07:44:16,187][00652] Stopping RolloutWorker_w4... [2023-10-08 07:44:16,187][01361] Loop rollout_proc14_evt_loop terminating... [2023-10-08 07:44:16,188][00657] Loop rollout_proc12_evt_loop terminating... [2023-10-08 07:44:16,188][00656] Loop rollout_proc10_evt_loop terminating... [2023-10-08 07:44:16,188][01411] Loop rollout_proc15_evt_loop terminating... [2023-10-08 07:44:16,188][00652] Loop rollout_proc4_evt_loop terminating... [2023-10-08 07:44:16,188][130385] Component RolloutWorker_w5 stopped! [2023-10-08 07:44:16,187][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000097664_100007936.pth... [2023-10-08 07:44:16,188][130385] Component RolloutWorker_w2 stopped! [2023-10-08 07:44:16,189][130385] Component RolloutWorker_w14 stopped! [2023-10-08 07:44:16,189][130385] Component Batcher_0 stopped! [2023-10-08 07:44:16,190][130385] Component RolloutWorker_w12 stopped! [2023-10-08 07:44:16,190][130385] Component RolloutWorker_w10 stopped! [2023-10-08 07:44:16,190][130385] Component RolloutWorker_w15 stopped! [2023-10-08 07:44:16,190][130385] Component RolloutWorker_w4 stopped! [2023-10-08 07:44:16,191][130385] Component Batcher_1 stopped! [2023-10-08 07:44:16,191][00651] Stopping RolloutWorker_w6... [2023-10-08 07:44:16,191][130385] Component RolloutWorker_w6 stopped! [2023-10-08 07:44:16,192][00651] Loop rollout_proc6_evt_loop terminating... [2023-10-08 07:44:16,192][00645] Stopping RolloutWorker_w0... [2023-10-08 07:44:16,192][130385] Component RolloutWorker_w0 stopped! [2023-10-08 07:44:16,192][00645] Loop rollout_proc0_evt_loop terminating... [2023-10-08 07:44:16,193][130385] Component RolloutWorker_w1 stopped! [2023-10-08 07:44:16,193][00644] Stopping RolloutWorker_w1... [2023-10-08 07:44:16,193][00644] Loop rollout_proc1_evt_loop terminating... [2023-10-08 07:44:16,188][00365] Loop batcher_evt_loop terminating... [2023-10-08 07:44:16,194][00659] Stopping RolloutWorker_w13... [2023-10-08 07:44:16,194][00658] Stopping RolloutWorker_w11... [2023-10-08 07:44:16,194][00653] Stopping RolloutWorker_w7... [2023-10-08 07:44:16,194][00659] Loop rollout_proc13_evt_loop terminating... [2023-10-08 07:44:16,194][00653] Loop rollout_proc7_evt_loop terminating... [2023-10-08 07:44:16,194][130385] Component RolloutWorker_w13 stopped! [2023-10-08 07:44:16,194][00658] Loop rollout_proc11_evt_loop terminating... [2023-10-08 07:44:16,195][130385] Component RolloutWorker_w11 stopped! [2023-10-08 07:44:16,195][130385] Component RolloutWorker_w7 stopped! [2023-10-08 07:44:16,196][00655] Stopping RolloutWorker_w9... [2023-10-08 07:44:16,197][130385] Component RolloutWorker_w9 stopped! [2023-10-08 07:44:16,197][00655] Loop rollout_proc9_evt_loop terminating... [2023-10-08 07:44:16,199][00647] Stopping RolloutWorker_w3... [2023-10-08 07:44:16,199][130385] Component RolloutWorker_w3 stopped! [2023-10-08 07:44:16,199][00647] Loop rollout_proc3_evt_loop terminating... [2023-10-08 07:44:16,188][00425] Stopping Batcher_1... [2023-10-08 07:44:16,213][00611] Weights refcount: 2 0 [2023-10-08 07:44:16,216][00611] Stopping InferenceWorker_p0-w0... [2023-10-08 07:44:16,216][130385] Component InferenceWorker_p0-w0 stopped! [2023-10-08 07:44:16,216][00611] Loop inference_proc0-0_evt_loop terminating... [2023-10-08 07:44:16,220][00612] Weights refcount: 2 0 [2023-10-08 07:44:16,221][00612] Stopping InferenceWorker_p1-w0... [2023-10-08 07:44:16,213][00425] Loop batcher_evt_loop terminating... [2023-10-08 07:44:16,222][00612] Loop inference_proc1-0_evt_loop terminating... [2023-10-08 07:44:16,222][130385] Component InferenceWorker_p1-w0 stopped! [2023-10-08 07:44:16,223][00425] Removing ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000096480_98795520.pth [2023-10-08 07:44:16,227][00425] Saving ./train_atari/atari_assault_APPO/checkpoint_p1/checkpoint_000098208_100564992.pth... [2023-10-08 07:44:16,229][00365] Removing ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000095968_98271232.pth [2023-10-08 07:44:16,234][00365] Saving ./train_atari/atari_assault_APPO/checkpoint_p0/checkpoint_000097664_100007936.pth... [2023-10-08 07:44:16,266][00425] Stopping LearnerWorker_p1... [2023-10-08 07:44:16,267][00425] Loop learner_proc1_evt_loop terminating... [2023-10-08 07:44:16,267][130385] Component LearnerWorker_p1 stopped! [2023-10-08 07:44:16,272][00365] Stopping LearnerWorker_p0... [2023-10-08 07:44:16,272][130385] Component LearnerWorker_p0 stopped! [2023-10-08 07:44:16,272][00365] Loop learner_proc0_evt_loop terminating... [2023-10-08 07:44:16,273][130385] Waiting for process learner_proc0 to stop... [2023-10-08 07:44:17,041][130385] Waiting for process learner_proc1 to stop... [2023-10-08 07:44:17,069][130385] Waiting for process inference_proc0-0 to join... [2023-10-08 07:44:17,070][130385] Waiting for process inference_proc1-0 to join... [2023-10-08 07:44:17,168][130385] Waiting for process rollout_proc0 to join... [2023-10-08 07:44:17,169][130385] Waiting for process rollout_proc1 to join... [2023-10-08 07:44:17,170][130385] Waiting for process rollout_proc2 to join... [2023-10-08 07:44:17,170][130385] Waiting for process rollout_proc3 to join... [2023-10-08 07:44:17,171][130385] Waiting for process rollout_proc4 to join... [2023-10-08 07:44:17,171][130385] Waiting for process rollout_proc5 to join... [2023-10-08 07:44:17,172][130385] Waiting for process rollout_proc6 to join... [2023-10-08 07:44:17,173][130385] Waiting for process rollout_proc7 to join... [2023-10-08 07:44:17,173][130385] Waiting for process rollout_proc8 to join... [2023-10-08 07:44:17,174][130385] Waiting for process rollout_proc9 to join... [2023-10-08 07:44:17,175][130385] Waiting for process rollout_proc10 to join... [2023-10-08 07:44:17,175][130385] Waiting for process rollout_proc11 to join... [2023-10-08 07:44:17,176][130385] Waiting for process rollout_proc12 to join... [2023-10-08 07:44:17,176][130385] Waiting for process rollout_proc13 to join... [2023-10-08 07:44:17,177][130385] Waiting for process rollout_proc14 to join... [2023-10-08 07:44:17,178][130385] Waiting for process rollout_proc15 to join... [2023-10-08 07:44:17,178][130385] Batcher 0 profile tree view: batching: 169.8187, releasing_batches: 0.0906 [2023-10-08 07:44:17,179][130385] Batcher 1 profile tree view: batching: 170.5377, releasing_batches: 0.0936 [2023-10-08 07:44:17,179][130385] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0001 wait_policy_total: 1599.3940 update_model: 202.1257 weight_update: 0.0009 one_step: 0.0027 handle_policy_step: 11193.2202 deserialize: 62.8052, stack: 187.2691, obs_to_device_normalize: 2489.4347, forward: 5032.5115, prepare_outputs: 2485.5381, send_messages: 459.2244 [2023-10-08 07:44:17,179][130385] InferenceWorker_p1-w0 profile tree view: wait_policy: 0.0011 wait_policy_total: 1701.0165 update_model: 202.1718 weight_update: 0.0009 one_step: 0.0027 handle_policy_step: 11101.1717 deserialize: 63.6712, stack: 190.3289, obs_to_device_normalize: 2471.3648, forward: 5010.1942, prepare_outputs: 2435.2804, send_messages: 452.1597 [2023-10-08 07:44:17,179][130385] Learner 0 profile tree view: misc: 0.0193, prepare_batch: 261.3646 train: 3640.8287 epoch_init: 0.1893, minibatch_init: 12.7940, losses_postprocess: 896.9838, kl_divergence: 31.2625, update: 388.6475, after_optimizer: 2128.8340 calculate_losses: 164.8532 losses_init: 0.4431, forward_head: 55.1772, bptt_initial: 1.4120, bptt: 1.9605, tail: 37.7632, advantages_returns: 11.0888, losses: 43.4699 [2023-10-08 07:44:17,179][130385] Learner 1 profile tree view: misc: 0.0197, prepare_batch: 262.3096 train: 3631.4370 epoch_init: 0.1888, minibatch_init: 12.9286, losses_postprocess: 891.6586, kl_divergence: 30.5347, update: 390.1351, after_optimizer: 2122.7976 calculate_losses: 166.3504 losses_init: 0.4506, forward_head: 55.6328, bptt_initial: 1.4339, bptt: 1.9938, tail: 38.2041, advantages_returns: 11.1610, losses: 43.7171 [2023-10-08 07:44:17,180][130385] RolloutWorker_w0 profile tree view: wait_for_trajectories: 1.2553, enqueue_policy_requests: 407.9659, process_policy_outputs: 194.5717, env_step: 6251.0015, finalize_trajectories: 3.5249, complete_rollouts: 2.9396 post_env_step: 379.5568 process_env_step: 85.5409 [2023-10-08 07:44:17,180][130385] RolloutWorker_w15 profile tree view: wait_for_trajectories: 1.2502, enqueue_policy_requests: 409.4777, process_policy_outputs: 193.3829, env_step: 6252.2898, finalize_trajectories: 3.4850, complete_rollouts: 2.8897 post_env_step: 381.1754 process_env_step: 85.0693 [2023-10-08 07:44:17,180][130385] Loop Runner_EvtLoop terminating... [2023-10-08 07:44:17,181][130385] Runner profile tree view: main_loop: 13673.9014 [2023-10-08 07:44:17,181][130385] Collected {0: 100007936, 1: 100564992}, FPS: 14668.3